Download Subtitles and Captions for Any Video Easily

so today's session what all things we

are basically going to discuss so first

of all we going to discuss about

different types of machine learning

algorithm like how many different types

of machine learning

algor understand the purpose of taking

this session is to clear the interviews

okay clear the interviews once you go

for a data science interviews and all

the main purpose is to clear the

interviews I've seen people who knew

machine learning algorithms in a proper

way okay they were definitely able to

clear it because they just explain the

algorithms in a better way to the

recruiter so that they got hired first

of all is the introduction to machine

learning here I'm just specifically

going to talk about AI versus ml versus

DL versus data sign then the second

thing that we are going to talk about

over here is the difference between

supervised MS

and unsupervised ml the third thing that

we are probably going to discuss about

is something called as linear regression

so we are going to clearly understand

the maths and geometric intuition the

next thing that we are probably going to

discuss about is R square and adjusted R

square the fifth topic that we are going

to discuss about is Ridge and lasso

regression the first topic that we are

going to discuss about is AI versus ml

versus DL versus data science so this is

the first topic that we are probably

going to discuss if you really want to

understand the difference between AI

versus ml versus DL versus data science

we will go in this specific format so

just imagine the entire universe so this

entire universe I will probably call it

as an AI now specifically when I say AI

this basically means AI artificial

intelligence whatever role you are in

you are as a machine learning developer

you working as a deep learning developer

Vision developer or a data scientist or

an AI engineer at the end of the day you

are actually creating AI application so

if I really want to Define what is this

artificial intelligence you can just say

that it is a process wherein we create

some kind of applications in which it

will be able to do its task without any

human intervention so that basically

means a person need not monitor this AI

application automatically it'll be able

to make decisions it will be able to

perform its task and it will be able to

do many things so this is what an AI

application is some of the examples that

I would definitely like to consider so

the first example that I would like to

consider AI application AI module

Netflix has an AI module suppose if you

see a kind of action movie for some time

then the kind of AI work or AI work that

is basically implemented over here is

something called as recommendation

so here through this application what

happens is that when you're continuously

seeing the action movies then

automatically the AI module that is

present inside Netflix will make sure

that it gives us recommendation on

action movies second if I take an

example of comedy movie If I

continuously see comedy movie then also

it'll give us the recommendation of the

comedy movie so this through this what

happens is that it understands your

behavior and it is being able to do its

task without asking you anything the

second example that I would like to take

up in is

amazon.in now amazon.in again if you buy

an

iPhone then it may recommend you a

headphones so this kind of

recommendation is also a part of AI

module that is integrated with the

amazon.in website the ads that you see

probably when you opening my channel

through which I get paid a little bit

from my from a from the hard work that I

do in YouTube right so through that ads

how that is recommended to you uh that

is also an AI engine that is included in

the YouTube channel itself which really

plays it is a business-driven goal

understand it is a business driven

things that we basically do with the

help of AI one more example that I would

like to give you is if I consider it

self-driving cars so here you'll be able

to see self-driving cars if you take an

example of Tesla so self-driving cars

what happens based on the road it is

able ble to drive it automatically who

is doing that there is an AI application

integrated with the car itself right so

if I consider all these things these all

are AI application at the end of the day

whatever role you do you are going to

create an AI application this is the

common mistake what people do you know

like our CEO sudhansu Kumar he has

written in his profile that he's an AI

engineer that basically means his goal

is to create an AI application so

probably in a product based companies

you'll be seeing this kind of roles

called as AI engineer now let's go to

the next role which is called as machine

learning so where does machine learning

comes into existence so if I try to

create this machine learning is a subset

of AI and what is the role of machine

learning it provides stats

tools

to analyze the data visualize the data

and apart from that to do

predictions I'm

forecasting so you will be seeing a lot

of machine learning algorithms so

internally those machine learning

algorithm the equation that we are

basically using it is basically using it

is having a kind of stats tool stat

techniques because whenever we work with

data statistics is definitely very much

important so this exactly is called as

machine learning so it is a subset of AI

this is very much important to

understand ml is a subset of AI so here

you can see that it is a part of this

now let's go to the next one which is

called called as deep learning deep

learning is again a subset of ml now

let's consider why deep learning came

into existence because in 1950s 60s

scientists thought that can we make

machine learn like how we human being

learn so for that particular purpose

deep learning came into existence here

the plan is to basically mimic human

brain so when I say mimicking human

brain that basically means we are trying

to mimic the human brain to implement

something to learn something so for this

you use something called as

multi-layered neural networks so this is

what deep learning is it is a subset of

machine learning its main aim is to

mimic human brain so they actually

create multi-layer neural network and

this multi-layered neural network will

basically help you to train the machines

or applications whatever we are trying

to create and deep learning has really

really done an amazing work with the

help of deep learning we are able to

solve such a complex complex complex use

cases that we will be probably

discussing as we go ahead now if I come

to data science see this is the thing

guys if you want to say yourself as a

data scientist tomorrow you given a

business use case and situation comes

that you probably have to solve that use

case with the help of machine learning

algorithms or deep learning algorithms

again the final goal is to create an AI

application right you cannot say that I

am a data scientist and I'll just work

in machine learning I or I'll work in

deep learning or I may I don't know how

to analyze the data no you cannot do

that when I was working in Panasonic I

got various different kind of task

sometime I was told to use W powerbi to

visualize analyze the data sometime I

was given a machine learning project

sometime I was given a deep learning

project so as a data scientist if I

consider where does data scientist fall

into this it will be a part of

everything so if I talk about machine

learning and deep learning with respect

to any kind of problem statement that we

solve the majority of the business use

cases will be falling in two sections

one is supervised machine learning one

is unsupervised machine learning so most

of the problems that you are basically

solving this is with respect to this two

problem statement two different types of

machine learning algorithms that is

supervised machine learning and deep

learning if I talk about supervised

machine learning two major problem

statements that you are basically

solving here also one is regression

problem

and the other one is something called as

classification problem and in the case

of unsupervised machine learning problem

statement you are basically solving two

different types of problem one is

clustering and one is dimensionality

reduction and there is also one more

type which is called as reinforcement

learning reinforcement learning I can I

I will definitely talk about this not

right now right now we are just focusing

on all these things now understand what

happens in supervised machine learning

let's consider consider a data set so

here I have a data set which says this

is my age and this is my weight suppose

I have these two specific features let's

say that I have values like 24 62 25 63

21 72

257 uh 62 and many more data over here

let's say that my task is to basically

take this particular data and create a

model wherein so suppose my task is that

I need to create a model whenever it

takes the New Age first of all we train

this model with this data and whenever

we take age a new age it should be able

to give us the output of weight this

particular model is also called as

hypothesis okay I'll discuss about this

today when I we discussing about linear

regression now what are the important

components whenever we have this kind of

problem statement first of all you need

to understand there are two important

things one is independent features and

the other one is something called as

dependent features now let's go ahead

and discuss what is independent feature

independent feature basically means in

this particular case since the input

that I'm basically training in all those

features becomes an independent feature

now in this particular case my age is

independent feature and whatever I'm

actually predicting so when I say

predicting I know this is my output okay

this is the what I have to basically

make my model uh give this as a an

output so in this particular casee my

dependent feature becomes weight why we

specifically say a dependent feature

because this is completely dependent on

this value whenever this is increasing

or decreasing this value is basically

getting changed so that is the reason

why we basically say this has

independent and dependent feature

whenever we are solving a problem right

in the case of supervised machine

learning remember they will be one

dependent feature and there can be any

number of independent features now let's

go ahead and let's discuss about

regression and classification what is

the difference between them now let

let's go ahead and let's discuss about

two things one

is let's say I want a regression problem

statement suppose I take the same

example as age and weight so I have

values like as discussed 24 72 23

71 uh 24 or 25

71.5 okay so this kind of data I have

see this is my output variable which is

my dependent feature now in this

particular dependent feature now

whenever I'm trying to find out the

output and in this particular output you

have a continuous variable when you have

a continuous variable then this becomes

a regression problem statement now one

example I would like to give suppose

this is my data set right this is my age

this is my weight suppose I am

populating this particular data set with

the help of scatter plot then in order

to basically solve this problem what

we'll do suppose if I take an example of

linear regression I will try to draw a

straight line and this particular line

is my equation which is called as yal mx

+ C and with the help of this particular

equation I will try to find out the

predicted points so this will be my

predicted point this will be my

predicted point this this any new points

that I see over here will basically be

my predicted point with respect to Y so

in this way we basically solve a

regression problem statement so this is

very much important to understand let's

go to the always understand in a

regression problem statement your output

will be a continuous variable the second

one is basically a classification

problem now in classification problem

suppose I have a data set let's say that

number of hours study number of study

hours number of play

hours so this is my independent feature

let's say a number of sleeping hours and

finally I have my output which will will

be pass or fail so in this I have all

this as my independent features and this

is my dependent feature so I will be

having some values like this and here

either you'll be pass or fail or pass or

fail now whenever you have in your

output fixed number of categories then

that becomes a classification problem

suppose it just has two outputs then it

becomes a binary classification if you

have more than two different categories

at that time it becomes a multiclass

classification so this is the difference

between regression problem statement and

the classification problem statement now

let's go ahead and let's discuss about

something called as unsupervised machine

learning now in unsupervised machine

learning which is my second main topic

over here I'm just going to write

unsupervised machine learning now what

exactly is unsupervised machine learning

here whenever I talk about there are two

main problem statement that we solve one

is clustering

one is dimensionality reduction let's

take one example of a specific data set

over here let's say that my data set is

something called as salary and age now

in this scenario we don't have any

output variable no output variable no

dependent variable then what kind of

assumptions that we can take out from

this particular data set suppose I have

salary and age as my values so in this

particular case I would like to do

something called as clustering now why

clustering is used just understand let's

say I am going to do something called as

customer segmentation now what does this

customer segmentation do clustering

basically means that based on this data

I will try to find out similar groups

groups of people suppose this is my one

group this is my another group this is

my third group let's say that I was able

to create this many groups this many

groups are clusters I'll say cluster 1 2

three each and every cluster will be

specifying some information this cluster

May specify that this person uh he was

very young but he was able to get some

amazing salary this person it may some

specify that these people are basically

having more age and they are getting

good salary these people are like middle

class background where with respect to

the age the salary is not that much

increasing so here what we are doing we

are doing clustering we are grouping

them together main thing is grouping

this word is very much important now why

do we use this suppose my company

launches is a product and I want to just

Target this particular product to rich

people let's say product one is for rich

people product two is for middle class

people so if I make this kind of

clusters I will be able to Target my ads

only to this kind of people let's say

that this is the rich people this is the

middle class people I will be able to

Target this particular ads or this

particular product or send this

particular things to those specific

group of people by that that is

basically called as ad marketing and

this uses something called as customer

segmentation a very important example

and based on this customer segmentation

we can later apply any regression or

classification kind of problem statement

now coming to the second one after

clustering which is called as

dimensionality reduction now in

dimensionality reduction what we are

focusing on suppose if we have th000

features can we reduce this features to

lower Dimensions let's say that I want

to convert this

uh th000 feature to 100 features lower

Dimension so can we do that yes it is

possible with the help of dimensionality

deduction algorithm there are some

algorithms like PCA so I'll also try to

cover this as we go ahead understand

clustering is not a classification

problem clustering is a grouping

algorithm there is no output feature no

dependent variable in clustering sorry

in unsupervised ml so yes I will also

try to cover up LDA we'll cover up PCA

and all as we go ahead so with respect

to supervised and unsupervised so first

thing that we are going to cover is

something called as linear regression

the second algorithm that we will try to

cover after linear regression is

something called as Ridge and lasso

third that we are going to cover is

something called as logistic regression

the fourth that we are basically going

to cover is something called as decision

tree decision tree includes both

classification and regression four fifth

that we are going to cover is something

called as adab boost sixth that we are

going to cover is something called as

random Forest seventh that we are going

to cover is something called as gradient

boosting eighth that we are going to

cover is something called as XG boost N9

that we are going to cover is something

called as n bias then when we go to the

unsupervised machine learning algorithm

the first algorithm that we are going to

do is something called as K means K

means algorithm then we also have DV

scan then we are also going to do higher

C clustering there is also something

called as K nearest neighbor clustering

fifth we'll try to see about PCA then

LDA so different different things we

will try to cover up yes svm I have

missed here I'm going to include svm KNN

will also get covered so I have that in

my list probably I may miss one or two

but we are going to cover everything so

let's start our first algorithm linear

regression so let's go ahead and discuss

about linear regression linear

regression problem statement is very

simple guys so suppose I have let's say

I have two features one is my X feature

and one is my y feature let's say that X

is nothing but age and Y is nothing but

weight so based on these two features I

have some data points that has been

present over here so in linear

regression what we try to do is that we

try to create a model with the help of

this training data set so this will be

my training data set what I'm actually

going to do is that I'm going to

basically train a model and this model

is nothing but a kind of hypothesis

testing or it is just kind of hypothesis

which takes the new age and gives the

output of the weights and then with the

help of performance metrics we try to

verify whether this model is performing

well or not now in short what we are

going to do in linear regression is that

we'll try to find out a best fit line

which will actually help us to do the

prediction that basically means if I get

my new age over here then what should be

my output with respect to Y okay so with

respect to this what should be my output

over here in this particular case

whenever we are drawing a diagram like

this I can basically say that Y is a

linear function of X so this is what we

are going to do now understand how we

are going to create this best fit line

this is very much important whenever we

say linear regression it basically means

that we are going to create a linear

line over there you may be thinking sir

why to create linear line why not

nonlinear line that I'll discuss about

it as we go ahead see other other

algorithms so to begin with let's

consider this line that you see over

here right this line equation can be

given by multiple equations someone some

people people write yal mx + C some

people write uh H some people write yal

beta 0 + beta 1 into X some people write

H Theta of xal to Theta 0 + Theta 1 into

X many many equations are there for this

this straight line this straight line

many many equations are there with

respect to many many different kind of

notations but the first algorithm that I

have probably learned of linear

regression is from Andrew Ng definitely

I would like to give him the entire

credits and based on his notation

whatever he has explained I'll try to

explain you over here so the credits for

this algorithm specifically goes to

Andrew NG so let's consider this one

over here in order to create this

straight line I will basically use a

equation which is called as H Theta so

this is the equation of a straight line

if I know the equation of the straight

line whatever I can write I can write

many things yal mx + C yal beta 0 + beta

1 * X and then I can also write one more

that is H Theta of xal theta 0 + Theta 1

into X of I here also you can basically

say x of I here also you can say x of I

now let's go ahead and let's take this

equation for now let's take this

equation of now so I'm I'm going to take

out this equation and just write one

equation through which I have also

studied but I will definitely be adding

some points which probably Andrew and

could not mention mention in his video

but I'll try my level best obviously he

is the best I cannot even compare myself

to him so Theta 0 + Theta 1 into X now

let's understand what is Theta 0 Theta 1

as I said that let's say I have a

problem statement over here let's say I

this is my X and this is my y this is my

data points now what I'm doing I'm

trying to create a best fit line like

this now what is this best fit line what

is uh when I say this best fit line is

basically given by this equation what

does Theta 0 basically indicate Theta 0

over here is something called as

intercept now what exactly is intercept

intercept basically means that when your

X is zero then H Theta of X is equal to

Theta 0 so in this particular case

intercept basically indicates that at

what point you are meeting the Y AIS so

this particular point is basically

your intercept when your X is equal to 0

at that point of time you'll be seeing

that this line is intersecting the y-

AIS whatever value this will be that is

your intercept now the second thing is

about your Theta 1 what is Theta 1 this

is nothing but slope or coefficient now

what does this basically indicate this

indicates let let's say that this is the

unit one unit in the x-axis and probably

with respect to this I can find one

point over here one point over here and

if I try to draw this over here to here

this is the unit movement in y so what

does it basically say slope with the

unit movement in one one unit movement

towards the x-axis what is the unit

movement in y- axis that is basically

slope or coefficient Theta 0 and Theta 1

two things and X of I is definitely your

data points now our main aim is to

create a best fit line in such a way

that I I'll just try to show it to you

what is our main aim let's let's

understand what is the aim of a linear

regression so if I take an example of

linear regression I need to find out the

best fit line in such a way that the

distance

between this data points that I have and

the predicted points should be very very

less suppose I'm creating a best fit

line okay I'm creating a best fit line

so with respect to this data points

initially was this right but my

predicted point is this point in this

particular case my predicted point is

this point so and if I do do the

summation of all these points those

distance should be minimal then only

I'll be able to say that this is the

best fit line so I I cannot definitely

say that this is exactly the best fit

line or not how will I say when I try to

calculate the difference between this

point and the predicted Point these are

my predicted point right if I try to

calculate the distance between them then

I will basically have a aim to it should

be minimal if I do the summation of all

the distance it should be minimal

so for that what I can do is that see

you may be also thinking Krish why not

just do one thing okay suppose if these

are my data points why not just play and

create multiple lines and try to compare

what we can do is that we can compare

multiple we can create multiple lines

right like this and then whoever is

giving the best minimal point I will go

and select that but how many iteration

you will do how you will come to know

that okay this line is the best line so

for that specific purpose we should

start at one point and we should lead

towards finding the best fit line start

at one point and then we should go

towards finding the best fit line so for

this particular purpose what we do is

that we create a something called as uh

cost function I have already shown you

what is my hypothesis function my best

fit line equation is basically given as

H Theta of x equal to Theta 0 + Theta 1

* X this is my hypothesis right now

coming to the cost function which is

super super important why this it is

super important because cost function

basically what what is cost function

over here I told right right this

distance when I do the

summation this distance that I when I'm

doing the summation it should be minimal

so if I really want to find out this

particular distance I will be using one

more equation how can I use a distance

formula between the predicted and the

real point I will just say that H Theta

of x - y so when I say h Theta of x - Y

what does this basically mean this is my

real point and this is my predicted

Point predicted point is basically given

by H Theta of X and what I'm going to do

I'm going to basically do the squaring

because I may get a negative value so

because of that I really want to do the

squaring part Now understand one thing I

need to also do the

summation I = 1 to compl complete M

let's say that I'm taking the number of

data points over here as M because I

need to calculate the distance between

all the points right with respect to the

predicted and the predict with respect

to the real

points so after this I also need to

divide by 1X 2m the reason why I'm

dividing by first of all let me show you

why we are dividing by 1 by m 1 by m

will give us the average of all the

values that we have the specific reason

why we are dividing by 1 by 2 do is for

the derivation purpose it helps us to

make our equation very much simpler so

that later on when I am updating the

weights when I say weights I'm basically

updating Theta 0 and Theta 1 Theta 0 and

Theta 1 at that point of time you'll be

able to see that this particular value

when we probably do the derivative it

will help us to do it again I'm going to

repeat it I'm going to write it down for

you first of

all now in order to find find out the

best fit line I need to keep on changing

Theta 0 and Theta 1 unless and until I

get the best fit line unless and until I

don't get the best fit line I need to

keep on updating Theta 0 and Theta 1 now

if I need to keep on updating Theta 0

and Theta 1 I probably require a cost

function okay what this cost function

will do I'll just tell you so cost

function over here I will specify as J

of theta 0 comma Theta 1 is equal to now

what is cost fun function over here what

this distance I told right this distance

between the H Theta of X and Y if I do

the summation of all these things it

needs to be minimal it needs to be less

because with respect to an X point this

is my y point

right similarly with respect to this x

point this is my y point so what I'm

actually going to do I'm going to use a

cost function now in this cost function

my main aim is

to basically write H Theta of x - y s

this will be with respect to I I I why I

am saying I because this will be moving

from I equal to 1 to all the points that

is m m is basically all the points over

here now apart from this what I actually

going to do I'm going to divide by 1X 2

m I'll tell you why I'm specifically

dividing by 1X 2 m first of all by

dividing by m I will be getting an

average

output average cost function because

here I'm iterating M the reason why I'm

dividing by two because it will help us

in derivation why let's say that I have

x² if I try to find out derivative of x²

with respect to X then what will I get I

will basically get 2x right that is what

is the formula what is the derivation of

X of n it is nothing but n x of n

minus1 so that is the reason why I'm

actually making it 1 by two so that when

two comes over here this two and two

will get cancelled so I hope everybody's

able to understand so this is my cost

function Now understand what is this

called as this entire equation is

basically called as squared error

function yes mathematical Simplicity

basically means because when we are

updating Theta 0 and Theta 1 we

basically find out derivation in the

cost function so that is the reason why

we are specifically doing it squaring

off is basically done because so that we

don't get any negative values here

squared error function now let's go

towards the what we need to solve this

is my cost function okay so I need to

minimize minimize this particular value

that is 1x 2 m summation of I = 1 2 m

and then this will basically be H Theta

of X of I minus y of I whole Square we

need to minimize this by adjusting

parameter Theta 0 and Theta 1

this entirely is what this is nothing

but J of theta 0 comma Theta 1 and we

really need to minimize this so this is

our task okay this is our task now let's

go ahead and let's try to compare with

two different thing one is the

hypothesis testing and one is with

respect to the cost

function okay let's take an

example so right now my equation of

the

hypothesis is nothing but H Theta of x

equal to Theta 0 + Theta 1 *

X if Theta 0 is 0 then what does this

basically indicate can I say that it

basically the line the line the best fit

line passes through the origin and this

is nothing but s Theta of xal to Theta

1 multiplied by X can I say like this

obviously I can definitely say like this

right so my equation will be like this

so for right now let's consider that

your Theta 0 is equal to 0 so this is

what it is we have done till here we

have minimized we have written the

equation everything yes so it is passing

through the origin and this is what is

the equation I'm actually getting now

let's take one example and let's try to

solve this if I if I have H Theta of X

so this is my new hypothesis considering

that my intercept is passing through the

region so with respect to this let's say

that I will create one line over here

let's say this is

my this is my data points like X1 y1 I

have 1 2 3 I have 1 2 3 now let's

consider that if I have T I have data

points like what I have data points like

let's say I have three data points 1

comma 1 2A 2 3 comma 3 so 1A 1 is

nothing but this is my data point 2A 2

is nothing but this is my data point and

3 comma 3 is this is my data point so

these are my data points from the data

set that I

have so 2 comma 2 is this point and 3

comma 3 is basically this point let's

consider that these are my points that I

have these are my data points now if I

consider Theta 1 as 1 where do you think

the straight line will pass through

where do you think the straight line

will pass the straight line will

definitely pass like this right my

straight line will definitely pass

through all the points this same point

becomes a prediction point also right

same point let's consider that this is

also getting pass through this it passes

through all the points when Theta 1 is

equal to 1 Theta 1 is nothing but slope

when slope is equal to 1 in this

scenario it passes through all the

points now go ahead and calculate your J

of theta so what will the form of J of

theta 1 become because Theta 0 is 0 okay

we can basically write 1 by 2 m

summation of I = 1 2 three how many

points are there three right and here I

have J of H of theta of X1

sorry X of theta of x i - y i

s right now let's go ahead and compute

now in this particular scenario what

will happen 1X 2 m

then what is what is this point minus y

of I see h of X is also 1 y of I is also

one both the point are 1 so this will

become 1 - 1 whole S Plus because we are

doing summation the next point is also

falling in 2A 2 so this will become 2 -

2 s + 3 - 3 S so in total this will

become zero so when your J of theta when

Theta 1 is 1 Theta 1 is 1 so J of theta

1 is how much it is

Z right so what is this J of theta 1 it

is the cost function so let me draw the

cost function graph over here let's say

that this is my Theta and this is

my so here I have 0.5 here I have 1 here

I have 1.5 so this is my Theta here I

have two then I have 2.5 okay then

similarly I have 0. five then I have 1

1.5 2 2.5 this is my J of theta 1 so

right now what is my Theta 1 my Theta 1

is 1 at this particular Point what did I

get J of theta 1 is nothing but zero so

this will be my first point this will be

my first point guys I have discussed why

why the value will be 1X 2m basically to

make the calculation simpler we are

dividing by 1X 2 m is basically used to

average aage is the sumission that we

are actually doing over here now let's

go ahead and let's take the second

scenario in the second scenario let's

consider my Theta 1 let's say that my

Theta 1 over here is now 0.5 if my Theta

1 is 0.5 then tell me what are the

points that I will get for x equal to

1.5 * 1 so it will come as 0.5 over

here right then similarly when X is

equal to 2.5 * 2 is nothing but 1 over

here and then similarly when uh for x

equal to

35 multiplied by 3 see we are

multiplying here right5 multi by 3 is

1.5 so the next point will come over

here now when I create my best fit line

what will happen so here is my next best

fit line which I will probably create by

green

color okay so this is my second one

which is green color here definitely

slope is decreasing so if I go ahead and

calculate my J of theta let's see what

I'll get so J of theta

1 is nothing but 1X 2

m again same equation summation of I = 1

2 3 H Theta of X of

i - y of

i² so what we have for over here we have

nothing but 1X 2 m now let's do the

summation what is this point this point

is nothing but the predicted point and

this point is the real point right so in

this particular scenario the first point

that I will get is nothing but. 5 - 1

whole s how I'm getting. 5 - 1 whole

Square this is 1 this is the real Point

1 this is the predicted Point .5 so here

I'm getting. 5 - 1 whole Square the

second point will be 1 - 2 whole s right

2 so 1 - 2 whole

s and then I will finally get 1.5 - 3

whole s so finally if I do this

calculation how much I'm actually

getting 1X 2 * 3 which is 6 here I'm

getting

.25 5 Square here I'm getting 1 here I'm

getting 1.5 whole Square so my final

output will be which I have already

calculated it is nothing but point it

will be approximately equal to. 58 so 58

now with Theta as this is nothing but

Theta Theta 1 as

.5 right that is what Theta 1 as .5 we

are able to get. 58 so Theta 1 is .5

over here and. 58 will be coming

somewhere here right so this is my next

point which will be again in green color

now let's go ahead and calculate the

third condition now in third condition

what I'm actually going to write I'm

going to basically say Theta 1 as 0 at

that point of time just go and assume

what is 0 multiplied by X it will

obviously be zero so I will be getting

three points and my next line will be in

this line that is the

x-axis and this is basically all my

points now if I go ahead and calculate

this what is J of theta 1

now what is J of theta 1 now in this

particular case when my Theta 1 is equal

= to 0 1X 2 m now this part you'll be

able to see this is 0 - 1 0 - 2 0 -

3 okay so it will become 0 - 1 s 0 - 2 s

and 0 - 3

S okay so this will become 1X 6

* 1 + 4 + 9 which will not be it will be

nothing but 2.3 which is approximately

equal to

2.3 then what will happen with respect

to Theta 1 as 0 we are getting 2.3 so if

I draw this it is nothing but with

respect to zero I'm getting 2.

2

2.3 this is my point so similarly when

you start constructing with Theta 1 is

equal 2 I may get some point over here

so here when I join this points

together you will be seeing that I will

be getting this kind of

curve okay and this curve is something

called as gradient

descent and this gradient descent will

play a very very important role in

making sure that in making sure that you

get the right Theta 1 value or light

slope value now which is the most

suitable point the most suitable point

is to come over here because this is

this this point is basically called AS

Global

Minima because see out of all these

three lines which is the best fit line

this is the best fit line right this is

the best fit line when I had this best

fit line my point that came over here

was here itself this was my point that

came over here right and I want to

basically come to this region because

this is my Global

Minima when I basically am over here the

distance between the predicted and the

real point is very very less right so

this specific point is basically called

AS Global minimum but still I did not

discuss Krish you have assumed Theta 1

is 1 Theta 1 is .5 Theta 1 is 0 here

also you're assuming many things right

and then you probably calculating and

you're creating this gradient descent

but the thing should be that probably

you come to one point over here and then

you reach towards this so for that

specific reason how do you do that how

do I first of all come to a point and

then move towards This Global Minima so

for that specific case we will be using

one convergence algorithm because if I

come to one specific point after that I

just need to keep on updating Theta 1

instead of using different different

Theta 1 value so for this we use

something called as convergence

algorithm so here the convergence

algorithm basically says

repeat until

convergence that basically means I'm in

a while loop let's say and here I'm

basically going to update my Theta value

which will be given by this notation

which is continuous updation where I'll

say Theta J minus I'll talk about this

Alpha don't worry and then it will be

derivative of theta

J with respect to this J of theta

0 and Theta 1 so this should happen that

basically means after we reach to a

specific point of theta after performing

this particular operation we should be

able to come to the global Minima and

this this specific thing that you are

able to see is called as

derivative this is called as derivative

derivative basically means I'm trying to

find out the slope

derivative which I can also say it as

slope this equation will definitely work

guys trust me this will definitely work

why it will work I'll just draw it show

it to you let's say that this is my cost

function let's say that I've got this

gradient

descent and let's say that my first

point is somewhere here but I have to

reach somewhere here right now when I

reach this this is my Theta 1 and this

is my J of theta 1 suppose I reach at

this specific point and I will also have

another gradient descent which looks

like this let's say that in the initial

time I reach the point over here how we

will be coming to this minimal Global

Minima by using this equation I'll talk

about Alpha also don't worry now this is

also my Theta 1 this is also my J of

theta 1 now let's say suppose I came to

this particular point right after coming

to this particular point I will

basically apply this derivative on this

J of theta 1 okay now when I find out a

derivative that basically means we are

trying to find out the slope and in

order to find the slope we just create a

straight line like

this which will look like this I'll just

try to

create so I'll try to create a slope

like this this

slope so if you try to find out with

respect to this this is a positive slope

how do we indicate it because understand

the right hand side of the line of this

is pointing on the top wordss Direction

this is the best easy way to find out

whether it is a positive slope or

negative slope now in this particular

case this is a positive slope now when I

get a positive slope that basically

means I will update my weights or Theta

1 as Theta 1 let's say I'm writing it

over here so I will just apply this

convergence algorithm see Theta

1 colon Theta 1 minus this learning rate

which is called as Alpha this is my my

learning rate I'll talk about learning

rate don't worry then this derivative

value in this particular case since I'm

having a positive slope I will be

getting a positive value let's say that

for this Theta value I got this slope

initially now I need to come to this

location so for that I have to reduce

Theta 1 so that I come to this main

point now here you can see that I am I

subtracting Theta 1 with something which

is a positive number

right this is a positive number so

definitely I know that after some n

number of iteration I will be able to

come to the global Minima similarly if I

take the right hand side and if I try to

draw the slope in this particular case

my slope will be

negative so similarly I can write the

equation as Theta

1 = to Theta 1 minus learning rate

multiplied by a negative number so minus

into minus will be positive right

suppose initially my 1 was

here my Theta 1 was here now I'll keep

on updating the weight to come to this

Global Minima so minus into minus is

positive so I will basically get Theta 1

+

Alpha by a positive number because minus

into minus is plus so this will

definitely work so that we will be able

to come over here to the global Minima

whether it is a positive slope or a

negative slope now what is this learning

learning rate now learning rate based on

this learning rate suppose I want to

come from this point to the global

Minima by what speed I should be coming

what speed if my learning rate value is

bigger what speed I may be coming

suppose if I say usually we select

learning rate as 01 if I select a small

number then it'll start taking small

small steps to move towards the optimal

Minima but if I take a alpha value a

huge value if it is a huge huge value

then what will happen this uh this

updation of the Theta 1 will keep on

jumping here and there and the situation

will be that it will never meet it will

never reach the global Minima so it is a

very very good decision to take a alpha

small value it should also not be a very

very small value if it becomes a very

very small value then what will happen

very tiny steps it will take forever to

reach the global Minima that basically

means my model will keep on training

itself so definitely this Al is going to

work now let me talk about one

scenario one scenario will be that what

if my my cost function has a local

Minima what if I have a local Minima

because here if I

come here if I come this is a local

Minima suppose one of my points come

over here and finally I'm reaching over

here what will happen in this particular

case because in this case you'll be

seeing that what will be my equation my

equation will be simply Theta 1

Theta 1 minus Alpha in this point in

this local Minima slope will be zero so

in this particular case my Theta 1 will

be equal to Theta 1 now you may be

thinking what is if this is the scenario

then we will be stuck in local Minima

this is called as local

Minima but usually with respect to the

gradient descent and the equation that

we are using here we do not get stuck in

local Minima because our gradient

descent in this particular scenar iio

will always look like this but yes in

deep learning when we are learning about

grade in descent and a Ann at that point

of time we have lot of local Minima and

because of that we have different

different G decent algorithm like RMS

prop we have Adam optimizers which will

solve that specific problem so this one

point also I wanted to mention because

tomorrow if someone asks you as an

interview question that what if in your

uh do you see any local Minima in linear

regression you can just that the cost

function that we use will definitely not

give us local Minima but if in deep

learning techniques with that we are

trying to use like Ann we have different

different kind of optimizers which will

solve that particular problem so that is

the answer you basically have to give

now let me go ahead and write with

respect to the gradient descent

algorithm so here again I'm going to

write the gradient descent algorithm so

this will be my gradient descent

algorithm and remember guys gradient

descent is an amazing algorithm and you

you will definitely be using it so

please make sure that you know this

perfectly now some questions are that

when will convergence stop convergence

will stop when we come to near this area

where my uh J of theta will be very very

less now in gradient descent algorithm I

will again repeat it so what did I say I

said

repeat until convergence I told you

right here we have written this

algorithm

and now let's take it for Theta 0 and

Theta 1 so here I will write Theta 0

J equal to Theta

J minus learning rate of derivative of

theta

J J of theta 0 and Theta 1 so this is my

repeat until convergence now we really

need to find out what we'll try to

equate we'll try to first of all find

out what is this

now if I really want to find out

derivative

of derivative of derivative of theta J

with respect to J of theta 0 and Theta 1

so how do I write this I can definitely

write this in a easy way okay so this

will be derivative of theta J and

remember J will be 0 and 1 right because

we need to find out for 0 Theta 0 and

Theta 1 so this will be 1 by 2 m what is

what is J of theta 0a Theta 1 obviously

my cost function so I will write

summation of IAL 1 to M and here I will

basically write J of theta of X of I

minus y of I whole squar so if my J is

equal to Z so what will happen for this

so here I can specifically say that

derivative of derivative of theta 0 J of

theta 0a 1

now it's simple here what I will be

doing is that I will be simply applying

derivative function see guys what is

this derivative let's consider this is

something like this 1X 2 m x² so if I

try to find out the derivative this will

be 2x 2 MX so 2 and 2 will get cancel so

similarly I'll have 1 by m and here I

will specifically be writing summation

of I = 1 2 m h Theta of x X of I which

will be my

x - y of i² so this will be my

derivative with respect to Theta 0 this

is what I got now the second thing will

be that when J is equal to 1 derivative

of derivative of theta 1 J of theta 0

comma Theta

1 in this particular case I will be

having 1 by m summation of I = 1 to M

then again see in this particular case

Theta of 1 is there right Theta of 1

basically means what if I try to replace

this let's say that I'm trying to

replace this H Theta of X with something

else what is s Theta of X I know that

right it is Theta 0 + Theta 1 * X so

Theta 0 + Theta 1 * X so after this if

I'm trying to find out the derivative

with respect to Theta 0 this will

obviously become I will be able to get

this much right now with respect to the

second derivative what I will be writing

I will again be writing H thet of X of i

- y of i s

multiplied X of I so this Square also

went off understand this H Theta of X is

what see they H Theta of X is nothing

but Theta 0 + Theta 1 * X so if I'm

trying to find out derivative with

respect to Theta 0 nothing will be going

to come okay Theta 1 of X will become a

constant in this particular case in this

case because Theta 1 of X is there so if

I try to find out derivative of theta 1

into X only I'll be getting X Y Square

will not be there it's easy right X squ

means 2x this is the derivative of x

square right so that square went and 1X

2 1 2 by two got cancelled so this will

be now my convergence algorithm so here

we have discussed about linear

regression oh sorry I have to remove

Square here also so let me write it

again okay repeat until conver con let

me write it down again repeat until

convergence finally your two updates

will be happening one is Theta 0 so here

it will be Theta 0

minus Alpha that is my learning rate 1

by m summation of IAL 1 to M and this

will basically be H Theta of X of I

minus y of

I and similarly if I want to update

Theta 1 it will be - alpha 1 by m

summation of I = 1 to m h Theta of X of

I oh my God y of I uh multiplied by X of

I Alpha is your learning rate guys Alpha

is nothing but it is learning rate here

we have to initialize some value like

0.1 see what is s Theta of X Theta 0 +

Theta 1 into X right if I do derivative

of theta 1 into x what is derivative of

theta 1 with Theta 1 x it is nothing but

X so this x will come over here now

let's discuss about two important thing

one is R square and adjusted R square

now similarly what will happen you will

have lot of convex functions now see if

I talk about uh like if you have

multiple features like X1 X2 X3 x4 at

that point of time you will be having a

3D curve curve which looks like this

gradient

decent which will be something like this

gradient it's just like coming down a

mountain now let's discuss about two

performance metrics which is important

in this particular case one is R

square and adjusted R square

we usually use this performance metrix

to verify how our model is and how good

our model is with respect to linear

regression so R square is basically

given R square is a performance Matrix

to check how good the specific model is

so here we basically have a formula

which is like 1 minus sum of residual

divided by sum of total now this is the

formula of R squ now what is this sum of

residual I can basically write like this

summation of y i Min - y i hat whole

Square this Yi hat is nothing but H

Theta of X just consider in this way

divided by summation of Y of i - y mean

y mean y s to formula this is the

formula I'll try to explain you what

this formula definitely says okay so

first thing first let's consider that

this is my this is my problem statement

that I'm trying to solve suppose these

are my data points and if I try to

create the best fit

line This Yi hat Yi hat basically means

this specific point we are trying to

find out the difference between this

things difference between these things

let's say that these are my points I'm

trying to find out a difference between

this predicted this is my predicted the

point in green color are my predicted

points which I have denoted as y i hat

and always understand this is what Su

sum of residual is sum of residual is

nothing but difference between this

point to this point this point to this

point this point to this point this

point to this point and I doing the all

the summation of those now the next

point which is very much important here

is my X and Y what is this y IUS y y bar

Y Bar is nothing but mean mean of Y if I

calculate the mean of Y then I will

probably get a line which looks like

this I'll get a line something like this

and then I will probably try to

calculate the distance between each and

every point and this specific point with

respect to the distance between this

point and this point the denominator

will definitely be high right this value

obviously this value will be higher than

this value right the reason why it will

be higher because the mean of this

particular value distance will obviously

be higher so this 1 minus high this will

be a low value and this will be a high

value when I try to divide Low by

High Low by high then obviously this

entire number will become a small number

when this is a small number 1 minus

small number will be a big number so

this basically shows that our R square

has fitted properly right it has

basically got a very good R square now

tell me can I get this entire R square a

negative number let's say that in this

particular case I got 90% can I get this

R square as negative number there will

be situation guys what if I create a

best fit line which looks like

this if I create this best fit line

which looks like this then this value

will be quite High it is only possible

when this value will be higher

than higher than this

value okay but in the usual scenario it

will not happen because obviously we'll

try to fit a line which will be at least

good it's not just like pulling one line

somewhere we don't want to create a best

fit line which is worse than this right

worse than this so in this particular

scenario you'll be saying that in R

square now here you'll be able to see

one one amazing feature about R square

is that let's say let's say one scenario

suppose I have features like let's say

that my feature is something like uh

let's say I have a price of a house okay

so suppose this is my bedrooms how many

bedrooms I have and this is basically

the price of the house now if I if I

probably solve this Pro problem I'll

definitely get an R square value let's

say the R square value is 85% let's say

that my R square is 85% now what if if I

add one more feature the one more

feature basically says that okay if I

add

location location of the house will be

definitely correlated with price so

there is a definite chance that the R

square value will increase let's say

that R square will become 90% if I

probably have this two specific feature

and obviously it is basically increasing

the R square because this is also

correlated to price

and let me change the example see first

case I got by R square as 85% let's say

now as soon as I added location I got

90% now let's say that I added one more

feature which gender is going to stay

gender like male or female is going to

stay you know that gender is no way

correlated to price but even though I

add one feature there is a scenario that

my R square will still increase and it

may become

91% even though my feature is not that

important even gender is not that

important the R square formula Works in

such a way that if I keep on adding

features and that are not nowhere

correlated this is obviously nowhere

correlated this is not correlated with

price then also what it does is that it

is basically increasing my r² so this

specific thing should not happen whether

a male will stay or female will stay

that does not matter at all still when

you do the calculation the R square will

still increase so in order to not impact

the model because see now right now with

this particular model where I have got

90% now as soon as I see R square as 91%

because it is considering this

particular gender so this model will be

picked right because it is performing

well and is giving you a better R square

value but this should not happen because

that is not at all corelated this model

should have been picked so in order to

prevent this situation what we do we

basically Ally use something called as

adjusted R square now what is this

adjusted R square and how it will work

I'll also show it to you very very nice

concept of adjusted R square so adjusted

R square R square

adjusted is given by the

formula is given by the Formula 1 - 1 -

r² * N - 1 where n is the total number

of samples n minus P minus 1 this p p is

nothing but number of features

or predictors we'll also say or

predictors suppose initially my number

of predictors were in this particular

scenario in this scenario where I saw

this my number of predictors was two and

in this particular case my number of

predictor was three now if my predictor

is 2 I got the r squ as 90% so in this

particular scenario what all the

calculation will happen okay all the

calculation will happen and let's say

that my R square adjusted it'll be

little bit less it'll be little bit less

let's say it8 is 6% let's say that my R

square adjusted is 86% based on this

predictor 2 now when I use my predictor

3 predictor basically means number of

features that I'm going to use and now

in this one one feature is nowhere

related like gender but what we are

getting we are basically getting R

square increased to

91% now for the R square

adjusted this will not increase this

will in turn decrease right now it will

become 82% how it will become I'll show

you I've just considered some value 8682

here you can see that there is an

increase here an increase is there here

decrease is there now how this is

basically happening see this P value

that I will be putting okay if I put a p

isal 3 obviously with n minus P minus 1

this will become a little bit smaller

number or sorry little bit uh smaller

number right so now in this particular

case if it is not correlated obviously

this will be high when I'm increasing

this so this will also be high let me

write the equation something like this

just a second so this will basically

be okay now why probably this value may

have decreased let me talk about this

one what is r squ I hope everybody

understood n is the number of data

points p is the number of

predictors if p is increasing then what

will happen as P keeps on increasing

this value will keep on

decreasing this value will keep on

decreasing if this values keep on

decreasing this will be a bigger number

this will obviously be a big number a

big number divided by a small number

what it will be obviously this will be a

little bit bigger number 1 minus bigger

number we will basically get some values

which will be decreasing if my P value

is two in this particular case it will

be less smaller than this right at least

it will be greater than this this

particular value right when p is equal

to

3 so with the help of P obviously R

square is there to support you okay

whether it is correlated or not always

remember when the features are highly

correlated your R square value will

increase tremendously if it is less

correlated then it will be there will be

a small increase but there will not be a

very huge increase now if I consider p

is equal to 2 obviously when I'm trying

to find out this uh calculation n minus

P minus 1 it will obviously be greater

than p is equal to 3 when p is equal to

3 then this value will be still more

smaller and when we are dividing a

bigger number by a smaller number

obviously we are subtracting with one so

that basically means even though my R

square is 86 over here there may be a

scenario since this is nowhere

correlated I'm basically getting an 82%

because of this entire equation so I

hope you are understanding this this is

very much important to understand a very

very important property simple way to

define is that as my P value keeps on

increasing the number of predictors

keeps on increasing my R squ gets

adjusted whatever R square I'm getting

with respect to this it will always be

less than this particular R square there

was one interview question that was

asked one of my student between R square

and adjusted R square which will always

be bigger definitely the student said R

square then he told him to explain about

adjusted R square why does that specific

happen agenda one is about Ridge lasso

regression second is assumptions of

linear regression the third point that

we are probably going to discuss about

is logistic regression then the fourth

thing that we are going to discuss about

is something called as confusion

Matrix the fifth thing that we are going

to consider about

is practicals

for lead lineer Ridge lasso and logistic

so first topic uh that we are probably

going to discuss is something called as

Ridge and lasso

regression so let's understand about

Ridge and lasso regression if you

remember in our previous session what

all things we discussed linear

regression and then we had discussed

about the cost function we have

discussed about R square adjusted

adjusted R square sorry R square and

adjusted R square we have discussed

about it gradient descent we have

discussed about it it was nothing but 1

by 2 m summation of I = 1 2 m h Theta of

x i -

y - y i s so this is the cost function

that we had discussed right yesterday

and this cost function was able to give

us a

gradient descent with respect to the J

of

theta J of theta Zer or Theta not so I

can also write this as J of theta comma

Theta 0 comma Theta 1 now let me give

you a scenario let's say that I have a

scenario over here and I have this

specific scenario let's say that I just

have two points which looks like this

okay now if I have these two specific

points what will happen I will probably

try to create a best fit line the best

fit line will definitely pass through

all the points like this if I try to

calculate the cost function what will be

the value of J of theta 0 comma Theta 1

let's say that in this particular case

since it is passing through the origin

my Theta 0 will be zero okay so what

will be the value of theta 0 comma Theta

1 so here obviously you can see that

there is no difference so it will

obviously become zero Now understand

this data that you see right right this

data is basically called as training

data so this data that I have actually

plotted with two points these are

specifically called as training

data now what is the problem in this

data right now see right now exactly

whatever line is basically getting

created over here which is through the

uh hypothesis over here you can see that

it is passing through every point so

that is the reason your cost is zero and

our main aim is to basically minimize

the cost function that is absolutely

fine now in this particular case in

which my model this if this model is

getting trained initially this data is

basically called as training data now

just imagine that tomorrow new data

points comes so if my new data points

are here let's consider that I I want to

basically uh come up with this new data

point now in this particular scenario if

I want to predict with respect to this

particular Point let's say my predicted

point is here

is this the difference between the

predicted and the real Point quite

huge yes or no so this is basically

creating a condition which is called as

overfitting that basically means even

though my

model has given or trained well with the

training

data or let me write it down properly

over here so this condition since since

you can see that over here my each and

every point is basically passing through

the best fit line so because of that

what happens it causes something called

as

overfitting so you really need to

understand what is overfitting now what

does overfitting mean overfitting

basically means my model performs well

with training data but it fails to

perform well with test data now what is

the test data over here the test data is

basically this points the real test data

answer was this points but because the

my line is like this I'm actually

getting the predicted point over here so

this distance if I try to calculate it

is quite huge so in this scenario

whenever I say my model performs well

with training data and it fails to

perform well with test data then this

scenario we say it as overfitting so

this scenario when the model performs

well with training data I have a

condition which is called as low bias

and when it fails to perform with the

test data then it is basically called as

high High variance very important okay I

will make each and everyone understand

one by one if it is performing well with

the training data that is basically low

bias and whenever it performs well with

the test sorry fails to perform well

with the fails to perform well with the

test data then it is basically High

variance now similarly I may have

another scenario which is called as

underfitting so let's say that I have

something called as

underfitting now in this underfitting

what is the scenario the

model fails to perform it gives bad

accuracy I say that model always

remember whenever I talk about bias then

you can understand that it is something

related to the training data whenever I

talk about test data at that point of

time you talk about variance and that

specifically whenever you talk about

variance that basically means we are

talking about the test data so for an

overfitting you will basically have low

bias and high variance low bias with

respect to the training data and high

variance with respect to the test data

now if the model accuracy is bad with

training data and the model accuracy is

also bad with test data in this scenario

we basically say it as underfitting so

these are the two conditions that are

with respect to underfitting that

basically means that both for the

training data also the model is giving

bad accuracy and again for the test data

also it is basically having a bad

accuracy so in this particular scenario

we can definitely say two things out of

underfitting one is high bias and high

variance so this is the condition with

respect to underfitting very super

important let me just explain you once

again suppose let's consider I have one

model I have model two this is model one

this is model one this is model two and

this is model 3 okay guys so suppose

let's say that I have my model my

training accuracy is let's say

90% And my let's say that my test

accuracy is 80% now in this particular

case let's say that my training accuracy

is

92% and my test accuracy is 91% and

let's say my model three is basically

having training accuracy as

70% and my test accuracy is 65% so if I

take this particular case it is

basically overfitting if I take this

particular thing this basically becomes

my generalized model and when I talk

about this this is my I'll just say that

okay I'll also put nice color so that uh

you'll be able to understand this this

becomes our generalized model and this

finally becomes our underfitting right

under under fitting so here is my red

color I will just say it as underfitting

what are the main properties of this

overfitting as I said in this scenario

since it is performing well with the

training data so it will be low bias

High variance in this particular case it

will be low bias low variance and this

particular case it will be high bias and

high variance understand in this

terminology in this particular way

you'll be able to understand so why do

we require always a generalized model

because whenever our new data will

definitely come generalized model will

be able to give us very good output

let's go back to this particular example

here you'll be able to see this straight

line the red line that I have actually

created is basically overfitting so that

whenever I probably get the new points

which is having this real value and the

predicted points here you'll be able to

see the difference is quite huge so

because of this it will definitely be a

scenario of overfitting where it has low

bias and high weight

so again let me go ahead and take this

example so this was my line which I have

actually drawn I had two points and when

I draw this line which was a best fit

line to which is passing through both

the points this scenario is basically

causing a overfitting problem and I've

also shown you my J of theta 1 will be

zero in this scenario since it is

passing exactly and the predicted point

is also over there now understand one

thing is that what can can we take out

from this what assumptions we can take

out from this definitely if I talk about

our cost function our cost function here

is nothing but 1X 2 m summation of I = 1

2 m h Theta of X of i - y of I whole s

now let's consider that I am going to

use this H Theta X and I'm going to

basically write it as y hat okay let's

focus on this specific point so when I

take this I'm I'm just going to focus on

this particular point so here I will

definitely write it as y hat minus y of

I whole squ so this is my y y hat of I

minus y hat y i whole Square so this is

nothing but the difference between the

predicted value and the real value okay

this is what I'm actually trying to get

now in this scenario if I am adding this

values obviously I'm going to get the

value as zero now I have to make sure

that this value does not come to zero

because this is still over fitting so

that is where your Ridge regression will

come into picture Ridge and lasso will

come into picture now when I use Ridge

and lasso suppose if I use Ridge now in

Ridge what we say this this is also

called as L2

regularization now L2 regularization

what it does is that it basically adds a

unique

parameter add a One More Sample value

which is like Lambda multiplied by slope

Square now what is this slope whatever

slope of this particular line it is we

are just going to square it off now

suppose if I take my equation which

looks like this H Theta of X is equal to

Theta 0 + Theta 1 x now in this

particular case my Theta 0 was zero so

my H Theta of X is nothing but Theta 1

what is Theta 1 this is specifically

called as slope and I am basically

taking this Theta 1 I'm actually making

it as a square Square so always

understand I don't want to make this as

zero because if it becomes zero it may

lead to overfitting condition now what

will happen if I add this particular

equation if I add this particular

equation this will obviously come as

zero let's consider my Lambda value over

here my Lambda value is one I'll talk

about how do you set up Lambda value

okay let's consider that I'm

initializing it to one let's say my

Lambda value is 1 now what I will do is

that this l Lambda value is 1 Let's

consider our slope value initially is

two and because of this two I got this

best fit line I'm just going to consider

it so if I do the total sum over here if

I'm just considering this this value is

three now the cost function will not

stop over here because still it has to

minimize it has to reduce this three

value so what it will do it will again

change the Theta 1 value and let's say

that my Theta van value has changed now

it got another best fit line which looks

something like like this this is my next

best fit line I'll talk about Lambda

Lambda is a hyper parameter guys what

exactly is Lambda I'll just talk about

it now when I basically change this line

now see why I'm getting this line let's

consider I have changed my Theta 1 value

since we need to minimize now when we

need to minimize what it will do we'll

again calculate the slope of this

particular line and then we will try to

create a new line when we sorry it is

two two not three just a second guys 0 +

1 multiplied by 2 s which is nothing but

4 so now my cost function will not stop

over here so we are going to still

reduce this now in order to reduce this

again Theta 1 value will get changed and

then we will get a next best fit line

for this point now what will happen in

this scenario once we have this best fit

line we will definitely get a kind of

small difference so now if I go ahead

and consider the new equation my y hat I

minus y

i² + Lambda of slope squar this value

will be a small value now because I have

some difference and then plus again 1

multiplied by now understand whether the

slope will increase in this particular

case or whether it will decrease in this

particular case there will be some slope

value let's say that I have got some

slope of this particular line in this

particular scenario again your slope

will definitely decrease so let's say in

the case of two initially it was now it

is basically

1.36 whole squ now this small Value

Plus 1 + 1.3 squ or let me consider that

my slope is now one simple value that is

5 so if I get this it is 2.25 2.25 plus

small value it will be less than three

only right it will obviously be less

than three or equal to 3 but understand

what is happening the value is getting

reduced from 4 to 3 so this is is the

importance of Ridge now what will happen

is that you will try to get a

generalized model which has low bias and

low variance instead of this overfitting

condition you know why specifically we

are adding Ridge L2 regularization it is

basically to prevent

overfitting because here you are not

stopping here you are trying to reduce

it unless and until you get a line you

get a line which will be able to handle

the which will be able to handle as a uh

generalized model now here you can see

now if I have my new points like how I

drew over here now the distance will be

less so now you'll be able to see that

it will be able to create a generalized

model guys this will be a small value

only see initially when we have this

line obviously we have zero if we try to

slightly move here and there so here

you'll be able to see that it will just

a slight movement but what this movement

is basically specifying it is specifying

that the slope should not be steep if we

probably have a steep slope it obviously

leads to most of the time overfitting

condition it should not be steep it

should be very very it should be less

steeper but it should actually help you

to create a generalized model so you

will be seeing that after playing for

some amount of time this value will not

reduce after some point of time it'll

get almost it'll be a minimal value

it'll be a smaller value and for this

also you have to specify iterations how

many times you probably have to train

them now this iterations is also a

hyperparameter based on number of

iterations you will probably see your R

square or adjusted R square over here so

this iterations based on the number of

iterations it will never become zero

guys understand because zero it is not

possible if it becomes zero trust me it

is an overfitting model you cannot get

that is something zero now what is

Lambda coming to this Lambda this Lambda

is a

hyperparameter this is basically to

check how fast you want to lessen the

steepness or how fast you want to make a

steepness grow higher right and this

Lambda will also be selected by using

hyper parameter and this also I'll show

you today in Practical what do you mean

by iterations iteration basically means

how many time I want to change the Theta

1 value how many times you want to

change the Theta value that is the

convergence algorithm right

convergence algorithm over here L2

regularization or Ridge is basically

used in such a way that you should never

overfit why we assume Theta 0 is equal

to 0 because I'm considering that it

passes through a origin right origin

over here Lambda is a hyper

parameter steep basically means how

steep the line is if I have this line

this line is quite steep if I have this

line This is less steep now if I go to

the next regularization which is called

as lasso raso R lasso regression this is

also called as L1

regularization now here the formula will

be changing little bit here you will be

having y hat of minus of Y whole Square

here you'll be adding a parameter Lambda

but understand here you'll not be adding

slope Square no here you'll be adding

mode of slope here you'll be adding mode

of slope and this mode of slope will

work is that it will actually help you

to do feature selection now you may be

thinking how feature selection crash

let's consider a equation over here

let's say that I have many many features

I have many many many features okay so

my H Theta of X which I'm indicating

here as y hat let's say that I'm I'm

writing this equation apart from

preventing for overfitting it will also

help you to do feature selection here

let me just show you over here with an

example this H Theta of X which I'm

probably writing as y hat will basically

be indicated by something over here

you'll be able to see that it is nothing

but let's say that I have multiple

features like this now in this

particular features obviously there are

so many coefficients over here so many

slopes over here now mod of slope will

be what it will be nothing but mod of X1

plus X2 plus X3 plus X4 plus X5 like

this up to xn now in this particular

case how it is basically helping you to

sorry not X1 sorry just a second this

mod of I have taken the data point this

is not data points this should be your

mod of theta 0 + Theta 1 + Theta 2 +

theta 3 + Theta 4 + Theta 5 like this up

to Theta n so here you'll be able to see

that this is how I will basically uh

I'll basically be calculating the slope

now as we go ahead guys whichever

features are probably not playing an

amazing role the Theta value the

coefficient value the slope value will

be very very small it is just like that

entire feature is neglected that entire

feature is neglected now in this

particular case we were doing squaring

because of the squaring that value was

also increasing but here because of the

mode that value will not increase

instead it will be a condition wherein

we are basically neglecting those

features that are not at all important

in this specific problem statement so

with the help of L1 regularization that

is lasso you are able to do two

important things one is preventing

overfitting and the second case is that

if you have many features and many of

the features are not that important okay

in basically finding out your slope or

your line or the best fit line in that

particular case it will also help you to

perform feature selection so this is the

importance of the entire what is the

importance of this this is the

importance of the uh Ridge and the lasso

regression that we are doing here I'm

just going to write L1

regularization and obviously we have

discussed about L2 regularization also

now you have probably understood Lambda

is one hyperparameter okay which we will

specifically using okay and based on

this Lambda this will be found out

through cross

validation cross validation is a

technique wherein we will try to

probably train our model and try to find

out the specific things okay what should

be the exact value and there also we

play with multiple values in short what

we are doing we just trying to reduce

the cost function in such a way that uh

it will definitely never become zero but

it will basically reduce based on the

Lambda and the slope value in most of

the scenario if you ask me we should

definitely try both the regularization

and see that wherever the performance

Matrix is good we should use that what

is cross validation basically means I

will try to use different different

Lambda value and basically Ally use it

so in a short let me write it down again

for Ridge regression which is an L2 Norm

here I'm simply writing my cost function

in this particular case will be little

bit different here I can definitely

write my cost function as H Theta X of i

- y of I S Plus Lambda multiplied slope

Square what is the purpose of this the

purpose is very simple here we are

preventing overfitting this was with

respect to the Ridge Recreation that is

L2 nor now if I go ahead and discuss

about the next one which is called as

lasso regression which is also called as

L1 regularization in the case of lasso

regression your cost function will be H

Theta of X of

IUS y of

i² plus Lambda ultied mode of flow so

here you have this specific thing and

what is the purpose the purpose are two

one is prevent overfitting and the

second one is something called as

feature selection so these two are the

outcomes of the entire thing see with

respect to this lasso right you have

slopes slopes here you'll be having

Theta 0 plus Theta 1 plus Theta 2 plus

theta 3 like this up to Theta n now when

you'll have this many number of thetas

when you have many number of features

and when you have many number of

features that basically means you'll

have multiple slopes right those

features that are not performing well or

that has no contribution in finding out

your output that coefficient value will

be almost nil right it will be very much

near to zero in short you neglecting

that value by using modulus you're not

squaring them up you're not increasing

those values now I will continue and uh

probably I will also discuss about the

assumptions of linear regressions so

what are the assumptions of linear

regression in this particular scenario

so assumption is that number one point

linear regression if our features are in

normal or gion

distribution if our features follows

this particular distribution it is

obviously good our model will get

trained well so there is one concept

which is called as feature

transformation now in future

transformation always understand what

will happen if a model does not fall

follow a gan distribution then we apply

some kind of mathematical equation onto

the data and try to convert them into

normal orian distribution the second

assumption that I would definitely like

to make is that standard scalar or

standard digestion standard dig is

nothing but it is a kind of scaling your

data by using Z score I hope everybody

remembers Z score this is what we

basically apply there your mean is equal

to zero and standard deviation equal to

1 see guys wherever you have gradient

descent involved it is good to basically

do

standardization because if our initial

point is a small Point somewhere here

then to reach the global Minima or

training will happen quickly otherwise

what will happen if your values are

quite huge then your graph may be very

big and the point can come over any over

there and the third point is that this

linear regression works with respect to

linearity it works if your data is

linearly separable

I'll not say linearly separable but this

linearity will come into picture if your

data is too much linear it will

obviously be able to give a very good

answer like logistic regression also

which we are going to discuss today this

also has the same property now you may

be asking is it compulsory to do

standardization guys if you want to

increase the training time of your model

or if you want to optimize your model I

would suggest go ahead and do

standardization now coming to the fourth

Point here you really need to check

about multicolinearity

this is also one kind of check we

basically do what is multicol

linearities let's say I have X1 I have

X2 and this is my output feature I have

let's say X3 also now let's say that if

I try to see the colinearity of this two

feature how how correlated these two

feature are let's say that these two

feature are 95% correlated is it is it a

wise decision to use both the features

and let's say that let's let's say that

these two features are 95% correlated

but it is highly correlated with Y is it

necessary that we should use both the

feature in this particular scenario the

answer should be no we can drop this

particular feature okay we can drop this

particular feature any one of the

feature we can definitely drop it and

based on that I can just use one single

feature and basically we do the

prediction there is also a concept which

is called as variation inflation factor

I will try to make a dedicated video

about this multical is also solved with

the help of variation inflation Factor

one more term is there homos orc so that

kind of terminologies also we use one

more condition in this but if you almost

satisfied with this assumptions you will

definitely be able to outperform in

linear regression so you have got an

idea of the assumptions you have also

got an idea of multiple things okay now

let's go towards something called as

logistic regression now logistic

regression what logistic regression is

the first type of algorithm that we are

going to learn in classification let's

say that in classification I have one

example you know so suppose I have say

number of hours study hours and number

of play hours based on this I want to

predict whether a child is passing or

failing suppose these two are my

features I want to predict whether it is

pass or fail so here you'll be able to

see that I have some fixed number of

categories specifically in this

particular scenario I have two

categories binary logistic regression

works very well with binary

classification now the uh question comes

that can we solve multiclass

classification using logistic the answer

is simply yes you can definitely do it

so let's go ahead and let's try to

discuss about uh logistic regression now

what is the main purpose of the logistic

regression first of all let's let's uh

understand one scenario okay suppose I

have a feature which basically says um

number of study hours and this is like 1

2 3 4 5 6 7 and let's say that I have

pass this point is basically pass and

this point is basically

fail so I have this two conditions these

are my outcomes now what I'll do I will

just try to make some data points let's

say that if I study Less Than 3 hours I

will probably be fail if I study more

than 3 hours then probably I will pass

this I'll make it as fail and this I

will make it as pass so I will be having

points over here this 1 2 3 let's say

that this is my training data set now

the first question says that okay Chris

fine you have some data over here

whenever it is less than three you are

basically the person is failing if it is

greater than five greater than three it

is basically showing data points points

with respect to pass now can't we solve

this problem first with linear

regression now with the help of linear

regression here the first point will be

that yes I can definitely draw a best

fit line my best fit line in this

particular scenario may be something

like this it may it may look something

like this so here fail is nothing but

zero pass is one the middle point is

basically 0.5 so obviously with the help

of linear

regression I'm able to create this best

fit line and I'll put a scenario that

whenever the value is less

than5 whenever the value is less than

0.5 whenever the output is less than5

let's say that new data point is this

and based on this I'll try to do the

prediction I'm actually able to get the

output over here now when I'm getting

the output over here this basically is

0.25 now in this particular scenario

obviously I'm able to say that yes the

person I'll write a condition over here

saying that if my H Theta of x value is

less than 0.5 then my output should be

zero let's say less than 0.5 I'll say

not less than or equal to less than5

then my output will be zero right so in

this particular case Zero basically

means fail similarly I'll have a

scenario where I'll say that when if my

S of theta of X is greater than or equal

to 5 then this will basically be one

which is nothing but pass so this two

condition I can definitely write over

here this is my center point so that any

point that will probably come over here

let's say that this point is coming over

here right let's say new data point is

somewhere coming over here with this red

point

now what I'll do I'll basically draw a

straight line it will come over here I

will just extend this line

long I will extend this line over here

and I will extend this line over here

and here you can see that based on this

I'm actually getting this particular

prediction which is greater than 0.5 so

I will say that okay the person has

passed obviously this is fine this is

obviously working better this is

obviously working better so what what is

the problem why we are not using linear

regression okay in order to solve this

particular problem why you are

specifically having logistic regression

the answer is very much simple guys the

answer is that whenever let's say that

if I have an outlier which looks

something like this suppose I have an

outlier which comes like this over here

what is this value let's say that this

value is nothing but 7 8 9 10 let's say

that the number of study hours and I'm

studying for nine it is obviously pass

now in this particular scenario when I

have an outlier this entire line will

change now I will probably get my line

which looks something like this okay my

line will basically move something like

this it will now get moved something

like this now when it gets moves

completely like this now for even five

or even at any point that I am actually

predicting let's say that at this

particular point if I try to find out

it'll be showing less than 0. five so

here this particular value or answer

will be wrong right because if we are

studying more than 5 hours OB viously B

based on the previous line the person

had to pass but in this scenario it is

failing it is coming less than 0.5 but

the real value for this is basically

passed so I hope you are understanding

because of the outlier the entire line

is getting changed so how do we fix this

particular problem now in this two

scenarios are there first of all

obviously because of just an outlier

your entire line is getting shifted here

and there the second point is that over

here sometimes you're also getting

greater than one you you're also getting

less than one suppose if I try to

calculate for this particular point if I

project it in behind I'll be getting

some negative value so we have to squash

this function if I squash this function

then it'll become a plain line right how

do we squash it and for this we use

something called as sigmoid activation

function or sigmoid function if somebody

ask you why don't you use linear

regession in order to solve this

classification problem then your answer

should be very much simple you should

say this to specific points so we will

try to go ahead and solve some linear

regression now with the help of cost

function everything as such and we'll

try to understand how the cost function

will look for logistic regression second

reason I told you right it is greater

than zero over here the line is going

greater than zero right greater than

zero I have only Z and one and it is

becoming greater than zero but I have

already told that our maximum and

minimum value are 1 and zero so I hope

you have understood why linear Reg

cannot be used okay I showed you all the

scenarios why linear regression should

not be used now we'll continue and

probably discuss about the other things

over here and uh we will now try to

understand fine what exactly logistic

regression is all about and how the

decision boundaries basically created

now we'll go ahead and discuss about

that specific thing so let's go ahead

our values should be always between 0 to

one over here in this particular case

because it is a binary classification

problem only this should be the answer

so let's go ahead and let's define our

decision boundary so my decision

boundary decision boundary in the case

of logistic regression first of all as

usual in logistic regression we defined

our hypothesis okay guys first of all

let's see if I'm writing my my h of

theta my H Theta of X as Theta 0 + Theta

1 into x + Theta 2 into X like this X1

X2 + Theta n into xn

now in this scenario can I write this

entire equation as Theta transpose X

obviously I can definitely write this

way right and this is what is the

notation that you will probably seeing

in many places so with respect to the

decision boundary of logistic regression

our Theta see like this we can write I'm

saying okay but since we have to

consider two things one is squashing the

line okay how that squashing will

basically happen see if I have this if I

have this line

we saw in the above right if I have this

line suppose I have some data points

over here and I have some data points

over here if I want to create the best

fit line how will I create I will

basically create like this but I have to

also do two things one is squash over

here and squash over here right squash

over here and squash over here now in

order to squash I'm saying squash squash

means

okay now in order to do this I use a

function which is called as sigmoid

activation function

that basically means what happens

obviously you know this line is

basically denoted by H Theta of x equal

to how do you denote this straight line

let me write it down nicely for you so

how do you denote this straight line the

straight line is obviously denoted by

Theta 0 + Theta 1 * X1 let's say now on

top of this on top of this I have to

apply something on top of this value I

have to apply something so that I can

make this line straight instead of just

expanding in this way so my hypothesis

will basically be now G of G is

basically a function on Theta 0 and

Theta 1 * X1 so here I'm trying to

basically what I'm trying to do I will

apply a mathematical formula on top of

this linear regression to squash this

line now let's go ahead and let's try to

find out what is this G okay what is

this G I will say let Z equal to Theta 0

+ Theta 1 * X I'm just initializing this

now my H Theta of X is nothing but G of

Z now we need to understand what is this

z g of Z and how do we basically specify

what is the G function so my G function

is nothing but H Theta of x equal to 1

by 1 + e ^ of minus Z which in short if

I try to initialize Zed now it is 1 ^ of

e ^ of minus Theta 0 + Theta 1 * X so

this is what is my H Theta of X which is

my hypothesis and this obviously works

well because it is being able to squash

the function so this is basically my

hypothesis which I am definitely trying

to use it and this function that you are

actually able to see is called as

sigmoid or logistic function now you

need to understand what does this

sigmoid function look like in graph in

graph it looks something like this so

this this is my Zed value and this is my

G of Z this is my 05 your sigmoid

function will have this curve so this is

your one this is zero your value when

now from this we can make a lot of

assumptions what are the assumptions

that we can basically make your G of Zed

your G of Zed is greater than or equal

to

5.5 is obviously greater than or equal

to 0.5 when your Zed value is greater

than or equal to zero this is the major

assumptions that we can basically make

that is whenever your G of Z is greater

than your G of Z is greater than or

equal to 0.5 whenever your Zed is

greater than or equal to Z so obviously

whenever your Zed value is greater than

Z it is greater than 0.5 if your Zed

value is less than zero what it will

become it will basically be less than

0.5 so you can write that specific

condition also you want so this is the

most important condition

over here why it is called as logistic

regression see guys with the help of

regression you creating this straight

line and with the help of the concept of

sigmo you are able to squash it so they

have probably combined that name and uh

basically have written in this way will

squashing of the best fit L line help to

overcome the outlier issues yes

obviously it'll be able to help you so

let's go ahead and let's try to solve

the problem statement now usually let's

consider my training set let's consider

my training set suppose I have some

training points like this x of 1 comma y

of 1

let's say x of 2A y of 2 okay X of 3A y

of 3 like this I have lot of training

points and finally X of n comma y of n

let's say that this is my training data

so here uh my y y will belong to what

zero or 1 because I will only have two

outputs since we are solving a binary

classification problem here is my

training set with two outputs and I hope

everybody knows about J Theta of Z

it is nothing but 1 + e ^ of minus Z

here your Z is nothing but Theta 0 +

Theta 1 * X1 so this is your Theta 0 now

what we have to do we have to select

this Theta now in this particular case

let's consider that my Theta 0 is 0

because it is passing through the origin

just for time pass sake suppose my Z is

Theta 1 into X so now I need to change

what is my parameter my parameter is

Theta 1

I have to change parameter Theta 1 in

such a way that I get the best fit line

and along that I apply this sigmoid

activation function now let's go ahead

and let's first of all Define our cost

function because for this we definitely

require our cost

function now everything will be same

obviously you know the cost function of

linear regression because the first best

fit line that you are probably creating

is with the help of linear

regression now in this particular case

in the case of linear regression so here

you can basically write J J of theta 1

is nothing but 1 by m summation of I = 1

2 m 1X 2 and here you have H Theta of x

minus y of I I whole Square so this is

your entire thing of if you remember

linear regression whatever things we

have discussed yesterday okay so this is

the cost function let's consider that

for linear regression for this is for

the linear regression now for the

logistic regression what will happen for

your logistic regression I will take the

same cost function H Theta of X now you

know what is s Theta of X it is nothing

but 1 + 1 + e ^ of minus Theta 0 + Theta

sorry Theta 1 multiplied by X right this

is my with respect to logistic

regression this is my entire equation

now similarly I will try to only put

this H Theta of X let's consider that

this is my cost function only only my H

Theta of X is changing in this

particular case so if I go ahead and

write my cost function I can basically

say 1x2 h Theta of X of i - y of

i² and in this particular scenario what

is h Theta of X it is nothing but 1 + 1

+ e ^ minus Theta 1 x so this is what

this is getting replaced and this is my

logistic regression cost function I'm

just considering this cost function part

this part later on if you replace this

to this see if I replace this to this

and if I replace this to this it becomes

a logistic regression cost function

intercept I'm considering it as zero

guys now when I'm replacing this to this

this to this then it becomes a logistic

uh regression cost function but there is

one problem we cannot we cannot use we

cannot use this cost function there is a

reason for this because this equation

that you're seeing 1/ 1 + e^ of minus

Theta 1 * X this is a non-convex

function now you may be considering what

is a non-convex function so let me write

it down so here this this term this

terminology right it is a non-convex

function now what is this non-convex

function let me show you and let me

differentiate it with convex function

okay we'll try to understand what is the

difference between non-convex function

and convex function this is related to

gradient descent very important this is

related to gradient desent if you

remember with the help of linear

regression whatever gradient Dent we are

actually getting it is a convex function

like this this is the convex function

which looks like a parabola curve

Parabola curve because of this Parabola

curve whenever we use this linear

regression cost function specifically

because here my H Theta of X is what it

is nothing but Theta 0 + Theta 1 into X

because of this this equ

will always give you a parabola curve

this kind of cost function or convex

function you can say but here your s

Theta of X is changing so in the case of

if I use that cost function you will be

getting some curves which looks like

this now what is the problem with this

curve here you have lot of local Minima

if local Minima is there you will never

reach This Global Minima so that is the

reason we cannot use that c function now