00:00and this A1 is now going to the output neuron and once again we have the weighted sum of the
00:08inputs
00:08z3 equal to w1k which is the weight associated with this connection into A1 plus w2k the weight
00:18associated with the connection between the second neuron of the hidden layer and the output neuron
00:23and we get the weighted sum of the inputs. In this case there is no bias which is taken here.
00:30Now, the activation function once again there is an activation function here which is A3 sorry
00:36which is g and it takes passes z3 and then gives me the output A3. So, the output of your
00:48output layer
00:49this is basically your output layer is nothing but A3. Now, suppose your target is y what is the
00:59meaning of target? Target means the ground truth value the actual value is y and the predicted value
01:07from the neural network is A3. Then there is something called as a loss or the error which
01:17is a function of y and A3. It means the loss or the error term is the difference between
01:24the predicted and the ground truth ok. It is the difference between these two. Now, that means
01:34the loss is a function of your ground truth value or the target value and the output that
01:40you are getting from the neural network. The entire process that I explained till now is called
01:48as the forward propagation which means the information is passed in the forward direction.
01:58Now, what is the main aim of a neural network? The main aim is when we are predicting something
02:05whatever it is whether we are predicting a class of an image or whether we are predicting you know
02:13the weather or we are predicting anything. We want the prediction to be very very close to the actual
02:19ground truth value. Now, for that one what needs to be done is the loss or the error should be
02:27minimized.
02:29Now, how to reduce this error? Now, if we see the loss is a function of the ground truth or
02:37the target
02:37value and the output from the model. Now, we will just quickly have a till now I have explained all
02:48in terms of x1, x2, x3, x4.
02:51We will quickly have a look into a numerical example and then we will go and see how the loss
02:58can be reduced
02:59in case of neural network. So, suppose we have inputs x1 and x2 as 1 and 2 bias b1 equal
03:08to 0.5 and the activation function used is the sigmoid activation function.
03:13Once again it is a mathematical operation and which is given like gx equal to 1 by 1 plus e
03:19to the power of minus x.
03:21So, that is the activation function that we use here and it is called a sigmoid activation function.
03:26Right now we will just use this one. In my next session we will go into the details of what
03:31exactly the sigmoid activation function is.
03:34And if we have these weights w11, w12, w21, w22 all these are the values of the weights.
03:42Then for the neuron 1 in the hidden layer we do the calculations for z1 you all can do it.
03:49And in neuron 2 also this is the weighted sum of the inputs. First we have done z1 which is
03:56the weighted sum of the inputs.
03:57And then a1 has taken the output value of the weighted sum and then passed through the sigmoid activation function
04:05and we have got some value here.
04:07Similarly, z2 is for the neuron 2 and it is the weighted sum of the inputs and then it has
04:18gone through the activation function which is once again the sigmoid and this is the value.
04:23And the output neurons calculations also are shown here.
04:26So, this is a numerical example which I wanted to show so that there is you can relate it to
04:32the actual numbers that usually come in real life problem solving.
04:39Now, coming back to how to reduce the loss.
04:42So, as I was telling we have the input layer, hidden layer, output layer and your output or the loss
04:49at the output is a function of your target value and a3 which is your output in the or the
04:58model's output.
04:59Now, what happens is if we see this loss is a function of your output a3.
05:10Now, if we see here a3 is a function of z3 and z3 in while calculating the z3 all my
05:21weights have come into picture w1k w2k they are a part in the calculation of z3.
05:29Also, in z3 calculation we have taken a1 a2 also multiplied them with the weights and if we go backward
05:37we see this a1 is once again dependent on z1 which is nothing but the weighted sum of the inputs
05:45in the first neuron of the hidden layer.
05:48And, if we see this z1 also depends on the weights.
05:54So, w11 w12 w11 w21 similarly here also if we go back here because your z3 in z3 calculation we
06:05have this a2 term also and a2 is dependent on your z2.
06:09So, z2 is dependent on the weights associated with the neurons input neurons what we see here is the loss
06:19is actually dependent or is a function of all the weights associated in all the edges.
06:28So, in order to reduce the loss the loss the gradient of the loss or the error is propagated and
06:37the gradient is taken with respect to the weights and it is propagated in the backward direction.
06:43And, for time being what we will understand here is there is an algorithm which is called as the back
06:50propagation or backward propagation algorithm in which the derivative of the loss with respect to the weights are taken in
06:58the backward direction.
06:59So, using
Comments