00:00Therefore, we will be going for a multi-layer perceptron which is suitable for both classification
00:05as well as regression task.
00:07Now here also we will be updating the weights, we will be updating the bias.
00:11So which means you have to after once we calculate get the output, once we calculate the error
00:16we have to come back and then update the weights.
00:18So in single layer perceptron we were using a simple perceptron learning rule to update
00:23the weights whereas in a multi-layer perceptron we will be using a back propagation algorithm.
00:28So, again in detail you will be studying in upcoming sessions.
00:33But for now multi-layer perceptron will use back propagation algorithm with gradient descent
00:39and different variants of gradient descent to update the weights and then the bias.
00:45And more you know this multi-layer perceptron we use to solve more real world problems where
00:52non-linear decision boundaries are involved.
00:55If this is a theory concept, theory behind the multi-layer perceptron, diagrammatically
01:01we can able to under see multi-layer perceptron looks something like this.
01:04So, we have seen some time back we have an output layer, then we have an hidden layer and as
01:11usual we will have an input neurons.
01:12So, here when we talk about the hidden layer, we can have one hidden layer or we can have
01:19more number of hidden layers.
01:21And when we say this is a neuron, here also we have a summation and we will have an activation.
01:27Here also we will get a summation and there is an activation, summation and activation.
01:31So, in previous session we have seen how input gets multiplied with the weight, the same concept
01:38is applied here as well.
01:39So, here this summation is the input which is coming from this and the weight associated
01:44with this, the input and the weight associated, input and the weight associated, this input
01:48and the weight associated and so on so forth.
01:50And the output of this will be this value and that will be multiplied with this weight and
01:55then you will have a summation, we will have a summation and we will have an activation
01:59function here also.
02:00So, since there are activation functions in all of these places, there are different activation
02:08functions that we will be using.
02:09There are different set of activation functions we use at an output layer and there are different
02:15set of activation functions that we use for a hidden layer and more specifically we will
02:22be going for a non-linear activation functions at an hidden layer.
02:26So, as we have seen in the theory, this is used to solve both linearly separable and non-linearly
02:33separable also.
02:34Even if we have a data set which is of this nature, if you have data where you cannot able
02:40to separate that with a straight line, you can still able to solve this problem using
02:45an multi-layer perception.
02:48So, this is all about the different types of perception, we have two types, single layer
02:54perception and multi-layer perception.
02:56Now, let us take the second numerical example where we will be still solving using a single
03:02layer perception only and here we are taking an R you know truth table.
03:10If you look into this, this is an R gate, when you where you have two inputs, therefore there
03:16are two neurons and then if you look into this, it is an binary classification problem, you just
03:23has an you just have an output either 0 or 1, therefore the number of neurons that you
03:28are placing at an output layer is just one neuron.
03:32Now, from the previous session we have seen how exactly the single layer perceptron update
03:39the weights, we have an weight updation formula, so using that we will be updating the weights.
03:46Here is an example where intentionally bias is not been taken, but that does not mean that
03:52there is no bias, definitely there will be a bias term also that will be added.
03:57But for this example, we are not considering the bias, the focus is only on updating the weights,
04:03but otherwise the bias term is also been included here, when we see the hands-on I will be using
04:10bias term also, so there is an you require there is a summation and there is an activation function
04:18and the activation function that we are using is a threshold activation where a threshold
04:22value is 0.5, anything the value greater than or equal to 0.5, the value will be 1 less
04:29than 0.5, the value will be 0, this is what is an activation function that we are defining
04:35for this problem statement and this is the learning rate that we have defined, ok.
04:40So, with this let us look into how to solve this and this is a diagrammatic representation
04:47of an or data set, if you take this when both the inputs are 0, you have the value 0,
04:55when
04:55one of the value is 0 and when one value is 1, we have a value 1, when one input
05:02is 1, when
05:02other input is 0, the value is 1, that value is this and when both are 1, when both x1
05:09and x2
05:09are 1, we get this value. So, you can see clearly there is a difference between two classes, when
05:18we plot this data set, there is a clear differentiation between this is a class 0, all this points belongs
05:26to class 1. So, between these two classes, we can able to clearly separate them using a
05:32line. So, this is what is the single layer perceptron concept, where it mainly used for
05:39a binary classification, works very good for a linearly separable data. So, let us solve this,
05:45so since we have solved one problem very detailedly, I will be showing you for one epoch for this
05:52particular problem, I will pause for a few seconds, where I want the readers to solve the problem
06:02and then you can come back to the particular slide and then you can verify your answer. So,
06:08let us look into how exactly we are going to solve this particular problem. So, since we have seen
06:14there are two inputs, the very first input what we are considering is 0 and 0, we have considered
06:210 and 0 as the first set of input. What is the target value for this particular input? The target
06:27value is 0, okay. So, what is the first step? The first step is we calculate z, how do we
06:34calculate z?
06:34x1 w1 plus x2 w2 and there is a bias term also, but we are not considering the bias term,
06:42but otherwise
06:42there will be some constant value, a small constant value that will be added here. What is x1? x1 value
06:49is 0 and what is the weight? Weight is 0.1 and what is x2? x2 is also 0 and
06:55the weight is 0.3. So,
06:57definitely the value is 0. So, and this is less than a threshold, what is a threshold value that
07:03is defined? The threshold is 0.5. Since the value is less than threshold, the predicted value is 0. So,
07:10we can say the computed value is 0. Now, the next step is that is what we have seen the
07:17computed value.
Comments