00:00Hello everyone, welcome back to the lecture series on foundations of deep learning concepts
00:07and applications.
00:08In the previous session we had seen biological inspiration for neural networks, the mathematical
00:15representation of a neuron, we have also seen what is perceptron and basic components of
00:21perceptron.
00:22We have seen a perceptron learning model and we have solved one numerical example where
00:27we have learnt the working mechanism of perceptron learning model.
00:31Today in this session we will see a types of perceptron, then we will see we will solve
00:38once again one more numerical problem and we will take an R gate you know truth table
00:45and then we will solve a problem.
00:47Then we will look into there is a limitation, there are certain limitations with respect
00:51to single layer perceptron.
00:52Once we complete this concept, once we understand the single layer perceptron limitations, then
00:59we will go to the implementation of single layer perceptron and here in the implementation
01:04we will be using Google Colab.
01:06So first for few minutes we will understand what is Google Colab, what is let us get familiarized
01:11with the Google Colab environment and then we will see the implementation of single layer
01:16perception.
01:17So, we will look into what are the different types of perceptron, there are 2 different types
01:23of perceptron, we have a single layer perceptron and we have a multi layer perceptron.
01:29Now what exactly is a single layer perceptron, single layer perceptron is one form of ANN which
01:36is mainly designed for a linearly separable problems.
01:40Now, when you look into this diagram, here we have a data set, when you plot the points,
01:52between the 2 different classes, we can clearly able to separate them with a straight line.
01:59So, this is called a linearly separable data.
02:01So, perceptron, single layer perceptron is one which is designed for a linearly separable patterns.
02:10And how does the architecture of a single layer perceptron looks?
02:14The architecture consists of only an input layer and it will have an output layer.
02:19So, here between input and output, we will not have any hidden layers.
02:24And since there is only one output layer, the activation function that is used here as we
02:31have seen in the previous session also, the activation function typically used is either we will have
02:37a step or a threshold activation function.
02:40And the single layer perceptron is the one which is mainly suitable for binary classification
02:45where you will have the classes either yes or no, 0 or 1, true or false.
02:51And here we have seen the perceptron learning rule does not use a back propagation concept.
02:57So this is what is the theoretical concept.
02:59So, when we say the perceptron learning rule here, the perceptron learning rule is used to
03:06update the weights.
03:08So, how do we update the weight?
03:10What is the formula?
03:11The new weight is old weight plus change in the weight.
03:15So, delta W is given by a learning rate and the target minus computed multiplied with the
03:20target minus computed and then we will multiply it with its associated input.
03:24And this has been added with the new old weight to get a new weight.
03:29So, the these are this is a formula which is used to update the weight.
03:34Likewise, we are also we will also be updating the bias and the formula used to update the
03:41bias is this.
03:43And diagrammatically if we have to understand the single layer perceptron, this is the diagrammatic
03:51representation.
03:52So, theoretically what we have seen, we will just have only input and then the output layer.
03:57We will not have any hidden layers in between that is what is single layer perceptron.
04:02And the neurons that are used at an I mean the activation function that is used at an output
04:07layer will be an step activation or a threshold activation.
04:11And it is mainly designed to solve a linearly separable problem.
04:16Linearly separable problems are those problems where when you plot them on a 2 dimensional graph,
04:24you can able to clearly separate them using a straight line.
04:27There is one major limitation with respect to a single layer perception.
04:32What is that limitation?
04:34It cannot handle a non-linearly separable data.
04:37So, it cannot handle this.
04:39So, what do you mean by non-linearly separable?
04:41So, here is the diagram.
04:43So, when we look into this, you cannot separate the two classes with the straight line.
04:51So, this is what is non-linearly separable data.
04:54So, single layer perception cannot handle this.
04:57That is the reason we have a second type of perception called a multi-layer perception.
05:03What is a multi-layer perception?
05:05It is once again a neural network model which is mainly designed to handle both linear as
05:11well as non-linear separable problems.
05:15If that is the case, how does the architecture of you know multi-layer perception looks?
05:21The architecture will have something called we will have an input layer, we will have an
05:25hidden layers and we will have an output layer.
05:27So, we will have hidden layers and we can have one or more hidden layers in the architecture.
05:35And because we have an hidden layer and then we have an output layer also, there are different
05:40activation functions that we will be using for an hidden layers and there are different
05:45activation functions that we will be using for an output layer.
05:48So, you will study more in detail about various activation functions in the upcoming sessions.
05:54But for this particular session, you only try to understand there are different activation
05:59functions that will be used at an hidden layer and there are different set of activation functions
06:05that we will be using at an output layer.
06:07So, the at an hidden layer, mainly we will be focusing on using a non-linear activation functions.
06:14So, such as it can be a ReLU, it can be a Leaky ReLU, etc.
06:18So, multi-layer perceptron using this particular model, you can solve both the regression as well
06:25as the classification tasks, whereas we have seen previously the perceptron, single layer perceptron
06:31is more suitable to solve a classification problem and more specifically, we will be focusing
06:36on a binary classification.
06:38But whereas, when you take a real world problems, you will have an multi-class classification,
06:43you will have a regression problems.
06:45So, in that case, single layer perceptron comes with certain limitations.
06:49Therefore, we will be going for a multi-layer perception, which is suitable for both the classification
06:54as well as the regression task.
06:56Now here also, we will be updating the weights, we will be updating the bias.
07:00So, which means you have to after once we calculate, get the output, once we calculate
07:04the error, we have to come back and then update the weights.
07:07So, in single layer perceptron, we were using a simple perceptron learning rule to update
07:12the weights, whereas in a multi-layer perception, we will be using a back propagation algorithm.
07:17So, we have another Antwort, starting up with theони�� of the
07:17Let's go back into theajes siguiente report Такil!
Comments