What the Heck are Artificial Neural Networks?
Artificial Neural Networks (ANNs) are a specific tool for implementing ML.
Mathematically, ANNs are composite functions made up of simple mathematical functions like linear () and exponential () functions. Calculus is used to "train" these composite functions to do various tasks.
In the applet below you can see an ANN perform machine learning with the push of a button. Each time you press the button, backpropagation "learns" how to select weights (w's) to get and to better target and . Each press of the button logs one epoch (or learning cycle). Backpropagation is the name for the calculus optimization problem the network is solving.
TRY IT! If anything goes wrong, click the restart button in the bottom right of the applet. We'll discuss what's going on as a group.
You can click on the blue shapes and segments to see how different parts are calculated.
How many epochs does it take to get the calculated outputs to within 3 decimal places of the targets?
Reset the app with the button in the bottom right. If the learning rate is increased to 5, how many epochs does it take to get the calculated outputs to within 2 decimal places?
Take a look at the app below to explore the math behind calculating the outputs and weights used in the neural network. Explore different activation functions that are sometimes used in artificial neural networks.
After looking at the different activation functions, decide which one you think would be best to use for each of these types of activities: Predicting whether a patient has a disease Determining if reviews are positive or negative Recognizing handwritten digits Object Detection
OPTIONAL Technical Terminology:
The inputs together with your targets are often called training data.
While not included here, sometimes biases are used in addition to the data. These are optional fixed numbers that get added to the net's. They are user-specified parameters, and are not considered "learnable". They are set to 0 in the applet for simplicity.
The E's on the right are errors (or loss), or a measurement of how close the computed outputs are to your targets. In this example we're using mean square error for an error.
The learning rate adjusts the impact of each epoch of backpropagation on the weights. Higher learning rates have a bigger impact. Careful though! A learning rate too high might accidentally jump your ANN into a very bad error!
OPTIONAL Technical details about the applet:
An ANN takes input data, and uses it compute outputs that target numbers you specify.
An ANN usually starts its life doing a very bad job of computing outputs close to your targets, but ANNs use calculus to "learn" how to get their calculated outputs closer to your specified targets.
You can see an ANN above.
There are two inputs, and . Do you see them on the left? You can change them if you want, but don't just yet.
There are two targets, and . Do you seem them at the top? You can also change these, but don't just yet.
The two outputs the ANN has computed are and . Do you seem them in the gray boxes? You can't change these. These have been computed by the ANN.
You don't need to know too much about what's in between (all the w's and net's and 's), but we'll talk about them a little bit. If anyone's curious you can read more here.
The cool thing about ANNs is that they use calculus to LEARN how to adjust the w's to get the computed outputs, and , closer to your targets, and . It's called backpropagation, and each iteration of backpropagation is called an epoch.