00:031. So, you see the log x values look like this. When we do a minus log x plot, it
00:09looks
00:09like this. And if you see in the first term as well as in the second term, we have minus
00:15log of something. So, our plots will look like this. And then this is the binary cross
00:23entropy loss. The blue curve is for the true value or the true class 1 and the orange 1
00:34is for the class for which the true value is 0, that means the negative class.
00:41So, what happens here is if we see when the true value is 1, that means it is a puppy,
00:48similarly when the true value is 1, then here the true value is 1, this is the curve for
00:55true value, we see the loss is 0. When the true value is 1, but the predicted value is
01:03near to 0, the loss is high. Similarly, this is the curve for the true value equal to 0,
01:10that means this is the curve for the cat say. So, in that case, what happens is if the predicted
01:17value is 0, then the loss is minimum. If the predicted value is class 1, then the loss is
01:24maximum. So, this is the curve for binary cross entropy loss. It takes both the positive class
01:33as well as the negative class and then it shows what is the predicted value and what is the
01:38true value and then it plots the graph and this is the plot for the binary cross entropy.
01:44So, here in the from the picture, we can make it much more clearer. So, suppose your true
01:49class is the dog, that means your y value is 1 and if the y pred or the predicted class
01:55is
01:56also dog that is 1, then in that case the loss is 0. So, true class is 1 which is
02:03your 1,
02:03this 1 and prediction is also 1, in that case the loss is 0. Now, if your true class is
02:101 which
02:10is the puppy and your predicted class is 0 which is the kitten, that means your model actually
02:17is predicting a dog's image as a cat's image. So, it is true class is 1 but the predicted
02:26class is 0 like this, in that case the loss is high here. Then when the true class is actually
02:34a kitten's image and the predicted class is puppy, then also it is a high loss. So, true
02:42class is, so this is the orange graph when it is predicted class is 1 and the actual value
02:50is 0. So, in that case also the loss is high here. And when the true class is 0 and
02:58the predicted
02:59class also is 0, then the loss is 0 ok. So, the true class is 0 and the predicted class
03:06is also 0,
03:07the loss is less. So, this is how we do the binary cross entropy loss calculations. Example of binary
03:14cross entropy loss. So, we will see an example of binary cross entropy loss numerical. So, consider
03:20the given actual output and the predicted output for four samples are given in the table below.
03:24So, you have a table here where S1, S2, S3, S4 are the samples, the output actual outputs are 1001
03:31and the predicted output y prime is 0.8, 0.2, 0.6, 0.9. And if we have the
03:38binary cross entropy
03:39is equal to 1 by minus 1 by m summation of yi log yi prime plus 1 minus yi log
03:481 minus yi prime.
03:49And if you do the calculations using these values here, you will get 0.367. You can pause here and
03:59do the calculations and cross check the answer. Then loss function for multi-class classification.
04:05For multi-class classification, we have categorical cross entropy loss. And the number of outputs nodes
04:11here in the output layer is equal to number of classes. And we have seen that softmax activation
04:17function is actually used for multi-class classification problems. Loss function for
04:22multi-class classification, sorry, softmax activation function is used for multi-class classification
04:28problem. So, this softmax ensures that all the output nodes have values between 0 and 1.
04:34And the sum of all the output node values are always equals to 0. Because your output values from that
04:41is coming out of the softmax, they are actually the probability values. And the probability of
04:47all the classes taken together should be equal to 0. That is the reason the output of all output node
04:52values should be equal to 0. And output nodes should have values between 0 and 1 because these are the
04:57probability values. And if you have you know a vector like this and it goes to softmax, it gives the
05:05probability values between 0 and 1. And suppose the second one is having high probability, this is
05:11basically a predicted class. Then we will have a look into the equation for categorical cross entropy.
05:21The loss equal to minus summation i equal to 1 to c, yi into log of yi hat. So, it
05:28is also called as
05:29softmax loss. Where c is the number of classes, yi is the true label which is the one hot encoded
05:36label.
05:37Only one of them is 1, other is 0. We will look into what exactly the one hot encoding is.
05:42And yi hat is the predicted probability for the class i, which comes from the softmax output.
05:49Now, we will see what is this one hot encoding. So, one hot encoding basically if you have
05:56four types of animals with IDs 1, 2, 3 and 4. The first ID is dog, then second is cat,
06:02then third is rabbit, fourth is bird. In that case, if we do one hot coding for the first ID,
06:10it will be 1 under the dog category and all others are 0. For the second one which is cat,
06:16the all other things are 0, only other than the cat. For rabbit, only the third one is 1,
06:22other all values are 0. And for the bird, only the fourth one is 1 and all other values are
06:280.
06:28So, this is called as one hot encoding. So, suppose we have a training data set where we have
06:35images of puppies, dogs, then horses, then cats and a cheetah. And during inference,
06:42I am giving the image of a dog. It is going through the neural network layers. And these are
06:47the values that I am getting before this activation function, which are called as the logics 3.2,
06:521.3, 0.2 and 0.8. And when these values are going through the softmax activation function,
06:57we are getting the output probabilities 0.0, 0.775, 0.116, 0.039 and 0.070. Where we see
07:05the output
07:06probability for the classes are given here, the dog class is the highest one. And if we take the
07:14difference between the output probability values p and the true ground truth, where is the dog class
07:21is 1 and the others are 0. So, the difference between them gives the categorical cross entropy loss.
07:28So, suppose if there are 3 classes and the true class is the second class,
Comments