00:03activation will be a linear activation function and the loss function should be mean squared
00:09error loss.
00:11So, with this one we come to the end of today's session, before I come to the summary, I would
00:20like to give some reference books for you all to you know have them as your reference
00:24or text books.
00:25So, the first book is you know Bishop C.M. and Bishop H Deep Learning Foundations and Concepts
00:33from Springer Nature and the other book is Jan Godfellow's book on Deep Learning from
00:39MIT Press.
00:40So, these are the two books that you can have for the artificial neural network, the introduction
00:45to deep learning, whatever we have taught you can refer to these books.
00:49And together with these, we have taken material from very recent research papers also.
00:57And for very updated information, we have looked into some very recent blogs also.
01:03So, coming to the end of this session, I would like to summarize what we have done today.
01:07So, we started with the activation functions, what are the uses of activation functions.
01:13Basically, these activation functions are used for capturing non-linearity in the data.
01:18And they are very much required in neural networks.
01:21And then we have seen what are the types of activation function, linear and non-linear.
01:26In linear activation function, we have seen that the output is just the input value.
01:32It is same, it is also called as the identity function.
01:34Then in non-linear activation functions, we have seen sigmoid activation functions which
01:39are used for binary classification problems.
01:43We have seen tanh activation functions whose outputs are zero-centered and they are used in
01:50the hidden layers.
01:51Then we have seen softmax output functions, activation functions which are used in multi-class
01:57classification problems.
01:58Then we have seen relu activation function and whereas, in relu it was giving x value for
02:06all the positive x values and for any negative x values it was giving zero.
02:11So, liquid relu does not allow it to go, the negative values of x to go exactly to zero.
02:18And thus, it helps in avoiding the dying relu problem.
02:23Then we have come into the loss functions, we have seen for regression problem, we have the
02:29mean squared error and the mean absolute error, what are the equations for them.
02:34And then we have seen these mean squared errors, they are very sensitive to outliers.
02:39We have seen how bigger value of error, you know, makes amplifies the mean squared error.
02:47Whereas, that amplification is not happening in absolute error and we have seen it for smaller
02:52error values.
02:53It is not the case but for bigger error values which might come from your outlier data points.
03:00And also we have discussed why we always take either square or absolute value in case
03:06of loss, ok, to avoid cancellation of positive and negative errors.
03:12Then we have come to different loss functions in classification problem, like in binary classification
03:19problem what kind of loss function, it is binary cross entropy loss.
03:23Then for multi class classification, we have done categorical cross entropy loss.
03:27And in multi label classification function, we do binary cross entropy for each of the labels
03:34and then we sum over all the classes.
03:36And we have seen into all the mathematical equations that are there related to the different losses
03:43in the classification problems.
03:45Then we have looked into the KL divergence loss which is required when the models output are
03:50probability distributions.
03:52And they are majorly, we see them when the model generates some output and generating models.
03:59And we have seen the properties of KL divergence between two probability distributions P and
04:06Q over the events I.
04:08And we have seen that KL divergence between P and Q is not equal to KL divergence between Q
04:13and P.
04:14And then we have come and had a quick look into how to design a neural network means how many
04:20neurons should be there in the input layer, how many neurons should be there in the hidden
04:24layer, how many neurons should be there in output layer for various classes of applications.
04:28And we have seen for input layer, the number of features decide how many neurons will be
04:33present.
04:34In the hidden layers, it is the task that we are doing.
04:37If it is a simple task, then we use very small number of neurons.
04:40For complicated task, we use more number of neurons.
04:45And then we go to the output layer for binary classification, one neuron.
04:51For multi class classification, the number of neurons is equal to the number of classes that
04:55are present.
04:56In multi-label classification, it is the number of labels that are present in your data.
05:02So, these are the ways we fix the number of neurons in the output.
05:06So, with this one, we come to the end of the sessions on introduction to deep learning, the artificial
05:15neural networks.
05:16And in the next session, we will start with the optimization algorithms for machine learning
05:24and the back propagation algorithm.
05:26And I hope this session was engaging for us.
05:32And till we meet for the next session, please continue reading and happy learning.
05:38Thank you all.
Comments