00:00So, in order to reduce the loss, the loss, the gradient of the loss or the error is propagated
00:08and the gradient is taken with respect to the weights and it is propagated in the backward
00:14direction.
00:15And for time being what we will understand here is there is an algorithm which is called
00:21as the back propagation or backward propagation algorithm in which the derivative of the loss
00:28with respect to the weights are taken in the backward direction and by this process the
00:33weights are updated and we reach a position where the loss is minimized, okay.
00:42We will go into the back propagation or backward propagation in much more details in my subsequent
00:49sessions.
00:50So, for right time being it is the back propagation which updates the weights and helps to reduce
00:58or minimize the loss in the output.
01:01So, this is about feed forward neural networks with the forward information carrying and
01:07back propagation of you know to read back propagation algorithm working in the backward direction
01:14to reduce the loss.
01:16Then we come to something called as the shallow neural network and deep neural network.
01:23We are all speaking about neural networks.
01:25Now, there is two terms shallow and deep neural network.
01:29We will try to see what are these two.
01:31So, if you see I have two pictures the first one is a shallow neural network where there is
01:37one hidden layer of neurons whereas in the deep neural network there are multiple hidden
01:43layers.
01:45Now, it is from the picture itself we can say that this network is very very deep because
01:52it has multiple hidden layers three or more usually and we will just have a quick look
01:58into what is the difference in these two models in terms of handling problem complexity or what
02:07what are the different applications and feature learning you know the things feature learning
02:13in an algorithm.
02:14So, we will just see in shallow neural network we have one hidden layer here it is three or
02:19more complexity wise shallow neural networks can handle simple and linearly separable problems
02:26whereas, deep neural network can excellently handle complex non-linear patterns.
02:33In applications coming to applications shallow neural network can be used in linear regression
02:39simply simple binary classification ok.
02:43So, very simple tasks whereas, deep neural networks are used for image recognition, speech
02:49recognition, natural language processing, complex predictive analysis this kind of applications.
02:56And, as we know and as we are seeing that shallow neural network is having less number of hidden
03:02layers so, it has a limited capability of feature learning whereas, deep neural networks are capable
03:10of learning very complex features.
03:13So, usually whenever there is very complex work we use deep neural networks whereas, where there
03:21is a simple work which do not require deep learning or deep neural networks we prefer going into shallow
03:30neural networks because that reduces the time complexity and the other requirements in terms of memory
03:37and processors and other things and if it can be solved in a simple way we go in the shallow
03:43neural network.
03:45Now, we will just have a quick look into feature extraction using shallow and deep neural network
03:50for a classification problem.
03:53Suppose, we have a dog's image or a puppy's image and a cat's image and I want to do a
04:00classification
04:01that means telling a dog as a dog and a cat as not a dog.
04:07So, what we will do is in a shallow neural network the image data is fed and there is a
04:14separate
04:15feature extractor which extracts the features, handcrafted features for a dog's image from
04:21a dog's image.
04:22So, this feature extractor is basically another one single hidden layer neural networks which
04:28takes out or does the feature engineering takes out or extract the features of a dog and
04:34then that is fed to the neural network and classification happens in the neural network and we get an
04:39output why that means whether it is an image of a dog or a not a dog ok.
04:44So, that is in that happens in shallow neural network whereas, in deep neural network, deep
04:51neural network does the feature extraction as well as the classification together.
04:56So, we do not require a separate feature extractor in case of deep neural network because it can learn
05:04the complex features that are present in the image and then it gives the output why.
05:10So, I think that is all for today's session.
05:16We will now have a quick summary of what all we have done or what all we have learnt today.
05:22So, we started with the multi-layer perceptrons, introduction to it, how it looks, input layer,
05:29hidden layers, output layers, how they are connected and then we have seen what is the difference
05:35between the single layer perceptron and multi-layer perceptron and for which kind of applications
05:43that is they are good. Then we have come to the feed forward neural network where we have understood
05:51what exactly happens inside the hidden layers neurons, the weighted sum of the inputs and the activation
05:58functions and what is the importance of the weights there and how the output is calculated in the feed
06:06forward neural network and we have seen these things in details and also we have seen there is a loss
06:13or
06:13the error term that comes out at the output layer and there is an algorithm called as back propagation.
06:20We have just touched upon but understood what exactly back propagation does in terms of weight
06:28updation, basically it updates the weights, updation of the weights happen after the back propagation
06:33algorithm and where we take the gradient of the loss with respect to the weights because
06:39weights are the deciding factor for this error. So, after that one we came to a numerical on feed
06:48forward neural network and we saw if the inputs are some numbers and we give a particular activation
06:57function like a sigmoid activation function like a sigmoid activation function.
06:59and they have a certain propeller function like a sigmoid activation function like a sigmoid activation function.
Comments