00:03Once again I will write it h 1 here, it is h 2 equal to h 3 here. So, I
00:10have to find
00:11out here this one. So, I will just write down once more z 1 2 equal to h 1 into
00:18w 1 1 2
00:20plus h 2 into w 2 1 2 plus h 3 into w 3 1 2. Therefore, if I take
00:33the partial derivative
00:34of z 1 2 with respect to h 1, I get w 1 1 2. Therefore, so first second and
00:44the third term
00:46all 3 we have calculated now. Therefore, we can write del l by del h 1 equal to 1 by
00:54n
00:55summation y minus t into y into y 1 minus y into w 1 1 2. Now, taking all the
01:06previous
01:06values here. Putting all these equations, putting all these values in equation 3, what was the
01:20equation 3? It was the one partial derivative of l with respect to w 1 1 1 with respect to
01:35w 1 1 1 1 is equal to 1 by n summation y minus t into y into 1 minus
01:49y w 1 1 2 into h 1 into
01:561 1 1 1 2. So, taking all these values from first term,
02:06second term and third term, we get this one. 1 minus y y minus t into yes y into y
02:15minus
02:161 w 1 1 2 and then from the previous term we got h 1 into 1 minus h 1
02:21into x 1 ok. So, we
02:23have got finally, we have got the partial derivative of l with respect to w 1 1 1 1 which
02:30is a weight
02:30in the inside layer ok, between the input node and the first hidden layer. Now, if I want
02:42my gradient descent algorithm to work, I need to find out something called as grad of l. So, if I
02:55write
02:55w 1 let w 1 be all the weights of your first layer. So, all these I put them in
03:03the form of a vector and I can
03:05write it as w 1 1 1, w 1 2 1, w 1 3 1 so on, w 2 1
03:161, w 2 2 1, w 2 3 1 ok. So, if you have and if you have some more,
03:28if you write it like this, then gradient
03:34gradient of l with respect to the weights is can be written as del l by del w 1 1
03:431 1, 2 1 1,
03:52partial derivative of w 1 2, partial derivative of w 1 2, sorry partial derivative of l. So,
04:11partial derivative partial derivative L with respect to W 1 2 1 and so on ok.
04:35So, after every iteration, we want to update the weights. So, after every iteration, weights
04:49will be updated. And what is the equation for that one? Weight at time t plus 1 or tau plus
05:031 equal to weight at the iteration, 1 iteration before minus learning rate into the gradient
05:10of the loss function with respect to the weights.
05:12So, at the time this is you know W at
05:40this derivative of L with respect to this W, this W is nothing but at the time t tau ok.
05:48So, the derivative is taken at W equal to eta. And similarly, the bias terms also will get
06:01updated, p tau minus eta that in that case your gradient will be taken with respect to
06:09the weights, gradient of the loss function with respect to the weights ok. So, this gradient
06:15will be taken at W equal to W tau. So, this is the way your back propagation algorithm works.
06:26And we have seen that we have started from the output loss, then we have come one layer
06:34in using those we have found out the derivative of the loss function with respect to you know
06:40the weights that are present in the inside layers. So, it is a propagation or the error is propagated
06:47in the backward direction. Then that is the reason it is called as back propagation algorithm.
06:53So, with this one I come to the end of today's discussion. Just to summarize what we have discussed today.
06:58We started with a toy neural network with which we have started with toy neural network which we have
Comments