Hi everyone, this is Zulaikha from Edureka, and I welcome you to this session

on Artificial Intelligence full course. In this video, I'll be covering all the domains and the concepts

involved under the umbrella of artificial intelligence, and I will also be showing you a couple of use cases and practical implementations by using Python.

So there's a lot to cover in this session, and let me quickly run you through today's agenda. So we're gonna begin the session

by understanding the history of artificial intelligence and how it cam into existence. We'll follow this by looking at

why we're talking about artificial intelligence now, why has it gotten so famous right now. Then we'll look at what exactly is artificial intelligence.

We'll discuss the applications of artificial intelligence, after which we'll discuss the basics of AI where in we'll understand

the different types of artificial intelligence. We'll follow this by understanding the different programming languages

that can be used to study AI. And we'll understand why we're gonna choose Python. Alright, I'll introduce you to Python.

And then we'll move on and discuss machine learning. Here we'll discuss the different types of machine learning, the different algorithms involved in machine learning,

which include classification algorithms, regression algorithms, clustering, and association algorithms.

To make you understand machine learning better, we'll run a couple of demos wherein we'll see how machine learning algorithms

are used to solve real world problems. After that, we'll discuss the limitations of machine learning

and why deep learning is needed. I'll introduce you to the deep learning concept, what are neurons, perceptrons,

multiple layer perceptrons and so on. We'll discuss the different types of neural networks, and we'll also look at what exactly back propagation is.

Apart from this, we'll be running a demo to understand deep learning in more depth. And finally we'll move onto the next module,

which is natural language processing. On the natural language processing, we'll try to understand what is text mining,

the difference between text mining in NLP, what are the different terminologies in NLP, and we'll end the session by looking at

the practical implementation of NLP using Python, alright. So guys, there's a lot to cover in today's session. Also, if you want to stay updated

about the recent technologies, and would like to learn more about the training technology, make sure you subscribe to our YouTube channel

to never miss out on such sessions. So let's move ahead and take a look at our first topic which is history of artificial intelligence.

So guys, the concept of artificial intelligence goes back to the classical ages. Under Greek mythology,

the concept of machines and mechanical men were well thought of. So, an example of this is Talos.

I don't know how many of you have heard of this. Talos was a giant animated bronze warrior who was programmed to guard the island of Crete.

Now these are just ideas. Nobody knows if this was actually implemented, but machine learning and AI were thought of long ago.

Now let's get back to the 19th century. Now 1950 was speculated to be one of the most important years

for the introduction of artificial intelligence. In 1950, Alan Turing published a paper in which he speculated about the possibility

of creating machines that think. So he created what is known as the Turing test. This test is basically used to determine

whether or not a computer can think intelligently like a human being. He noted that thinking is difficult to define

and devised his famous Turing test. So, basically, if a machine can carry out a conversation that was indistinguishable

from a conversation with a human being, it was reasonable to say that the machine is thinking, meaning that the machine will pass the Turing test.

Now, unfortunately, up to this date, we haven't found a machine that has fully cleared the Turing test.

So, the Turing test was actually the first serious proposal in the philosophy of artificial intelligence. Followed by this was the era of 1951.

This was also known as the game AI. So in 1951, by using the Ferranti Mark 1 machine of the University of Manchester,

a computer scientist known as Christopher Strachey wrote a checkers program. And at the same time,

a program was written for chess as well. Now, these programs were later improved and redone, but this was the first attempt

at creating programs that could play chess or that would compete with humans in playing chess. This is followed by the year 1956.

Now, this is probably the most important year in the invention of AI. Because in 1956, for the firs time,

the term artificial intelligence was coined. Alright. So the term artificial intelligence

was coined by John McCarthy at the Dartmouth Conference in 1956. Coming to the year 1959,

the first AI laboratory was established. This period marked the research era for AI. So the first AI lab where research was performed

is the MIT lab, which is still running til date. In 1960, the first robot was introduced

to the General Motors assembly line. In 1961, the first chatbot was invented. Now we have Siri, we have Alexa.

But in 1961, there was a chatbot known as Eliza, which was introduced.

This is followed by the famous IBM Deep Blue. In 1997, the news broke down that IBM's Deep Blue beats the world champion,

Garry Kasparov, in the game of chess. So this was kind of the first accomplishment of AI. It was able to beat the world champion at chess.

So in 2005, when the DARPA Grand Challenge was held, a robotic car named Stanley, which was built by Stanford's racing team,

won the DARPA Grand Challenge. That was another big accomplish of AI. In 2011, IBM's question answering system, Watson,

defeated the two greatest Jeopardy champions, Brad Rutter and Ken Jennings. So guys, this was how AI evolved.

It started off as a hypothetical situation. Right now it's the most important technology in today's world.

If you look around every where, everything around us is run through AI deep learning or machine learning.

So since the emergence of AI in the 1950s, we have actually seen an exponential growth and its potential.

So AI covers domains such as machine learning, deep learning, neural networks, natural language processing,

knowledge based, expert systems and so on. It is also made its way into computer vision and image processing.

Now the question here is if AI has been here for over half a century, why has it suddenly gain so much importance?

Why are we talking about artificial intelligence now? Let me tell you the main reasons for the demand of AI. The first reason is what we have

more computation power now. So, artificial intelligence requires a lot of computing power.

Recently, many advances have been made and complex deep learning models are deployed. And one of the greatest technology

that made this possible are GPUs. Since we have more computational power now, it is possible for us to implement AI

in our daily aspects. Second most important reason is that we have a lot of data at present.

We're generating data at an immeasurable pace. We are generating data through social media, through IoT devices.

Every possible way, there's a lot of data. So we need to find a method or a solution that can help us process this much data,

and help us derive useful insight, so that we can grow business with the help of data. Alright, so, that process

is basically artificial intelligence. So, in order to have a useful AI agent to make smart decisions

like telling which item to recommend next when you shop online, or how to classify an object from an image.

AI are trained on large data sets, and big data enables us to do this more efficiently. Next reason is now we have better algorithms.

Right now we have very effective algorithms which are based on the idea of neural networks. Neural networks is nothing but

the concept behind deep learning. Since we have better algorithms which can do better computations

and quicker computations with more accuracy, the demand for AI has increased. Another reason is that universities, governments,

startup, and tech giants are all investing in AI. Okay, so companies like Google, Amazon, Facebook, Microsoft,

all of these companies have heavily invested in artificial intelligence because they believe that AI is the future.

So AI is rapidly growing both as a field of study and also as an economy. So, actually, this is the right time

for you to understand what is AI and how it works. So let's move on and understand what exactly artificial intelligence is.

The term artificial intelligence was first coined in the year 1956 by John McCarthy at the Dartmouth Conference.

I already mentioned this before. It was the birth of AI in the 1956. Now, how did he define artificial intelligence?

John McCarthy defined AI as the science and engineering of making intelligent machines. In other words, artificial intelligence

is the theory and development of computer systems able to perform task that normally require human intelligence,

such as visual perception, speech recognition, decision making, and translation between languages. So guys, in a sense, AI is a technique of getting machines

to work and behave like humans. In the rest past, artificial intelligence has been able to accomplish this

by creating machines and robots that have been used in wide range of fields, including healthcare, robotics, marketing,

business analytics, and many more. With this in mind, let's discuss a couple of real world application of AI,

so that you understand how important artificial intelligence is in today's world. Now, one of the most famous applications

of artificial intelligence is the Google predictive search engine. When you begin typing a search term

and Google makes recommendations for you to choose from, that is artificial intelligence in action. So predictive searches are based on data

that Google collects about you, such as your browser history, your location, your age, and other personal details.

So by using artificial intelligence, Google attempts to guess what you might be trying to find. Now behind this,

there's a lot of natural language processing, deep learning, and machine learning involved. We'll be discussing all of those concepts

in the further slides. It's not very simple to create a search engine, but the logic behind Google search engine

is artificial intelligence. Moving on, in the finance sector, JP Morgan Chase's Contract Intelligence Platform

uses machine learning, artificial intelligence, and image recognition software to analyze legal documents.

Now let me tell you that manually reviewing around 12,000 agreements took over 36,000 hours. That's a lot of time.

But as soon as this task was replaced by AI machine, it was able to do this in a matter of seconds. So that's the difference between artificial intelligence

and manual or human work. Even though AI cannot think and reason like humans, but their computational power is very strong

compared to humans, because the machine learning algorithm, deep learning concepts, and natural language processing,

AI has reach a stage wherein it can compute the most complex of complex problems in a matter of seconds.

Coming to healthcare, IBM is one of the pioneers that has developed AI software, specifically for medicine.

Let me tell you that more than 230 healthcare organizations use IBM AI technology, which is basically IBM Watson.

In 2016, IBM Watson technology was able to cross reference 20 million oncology records quickly and correctly diagnose a rare leukemia

condition in a patient. So, it basically went through 20 million records, which it probably did in a matter of second or minutes,

max to max. And then it correctly diagnosed a patient with a rare leukemia.

Knowing that machines are now used in medical fields as well, it shows how important AI has become.

It has reached every domains of our lives. Let me give you another example. The Google's AI Eye Doctor

is another initiative, which is taken by Google, where they're working with an Indian eye care chain to develop artificial intelligence system

which can examine retinal scans and identify a condition called diabetic retinopathy which can cause blindness.

Now in social media platforms like Facebook, artificial intelligence is used for face verification wherein you make use of machine learning

and deep learning concept in order to detect facial features and tag your friends. All the auto tagging feature that you see in Facebook,

behind that there's machine learning, deep learning, neural networks. There's only AI behind it.

So we're actually unaware that we use AI very regularly in our life. All the social media platforms

like Instagram, Facebook, Twitter, they heavily rely on artificial intelligence. Another such example is Twitter's AI

which is being used to identify any sort of hate speech and terroristic languages in tweets. So again, it makes use of machine leaning,

deep learning, natural language processing in order to filter out any offensive or any reportable content.

Now recently, the company discovered around 300,000 terroristic link accounts and 95% of these were found by non-human

artificially intelligent machines. Coming to virtual assistants, we have virtual assistants like Siri and Alexa right now.

Let me tell you about another newly released Google's virtual assistant called the Google Duplex, which has astonished millions of people around the world.

Not only can it respond to calls and book appointments for you, it also adds a human touch.

So it adds human filters and all of that. It makes it sound very realistic. It's actually very hard to distinguish between

human and the AI speaking over the phone. Another famous application is AI is self-driving cars. So, artificial intelligence implements computer vision,

image detection, deep learning, in order to build cars that can automatically detect any objects or any obstacles

and drive around without human intervention. So these are fully automated self-driving cars. Also, Elon Musk talks a lot about how AI is implemented

in Tesla's self-driving cars. He quoted that Tesla will have fully self-driving cars ready by the end of the year,

and robo taxi version that can ferry passengers without anyone behind the wheel. So if you look at it, AI is actually used

by the tech giants. A lot of tech giant companies like Google, Tesla, Facebook, all of these data-driven companies.

In fact, Netflix also makes use of AI,. So, coming to Netflix. So with the help of artificial intelligence

and machine learning, Netflix has developed a personalized movie recommendation for each of its users.

So if each of you opened up Netflix and if you look at the type of movies that are recommended to you, they are different.

This is because Netflix studies each user's personal details, and tries to understand what each user is interested in

and what sort of movie patterns each user has, and then it recommends movies to them. So Netflix uses the watching history of other users

with similar taste to recommend what you may be most interested in watching next, so that you can stay engaged

and continue your monthly subscription. Also, there's a known fact that over 75% of what you watch is recommended by Netflix.

So their recommendation engine is brilliant. And the logic behind their recommendation engine is machine learning and artificial intelligence.

Apart from Netflix, Gmail also uses AI on a everyday basis. If you open up your inbox right now, you will notice that there are separate sections.

For example, we have primary section, social section, and all of that. Gmail has a separate section called the spam mails also.

So, what Gmail does is it makes use of concepts of artificial intelligence and machine learning algorithms

to classify emails as spam and non-spam. Many times certain words or phrases are frequently used in spam emails.

If notice your spam emails, they have words like lottery, earn, full refund. All of this denotes that the email

is more likely to be a spam one. So such words and correlations are understood by using machine learning and natural language processing

and a few other aspects of artificial intelligence. So, guys, these were the common applications of artificial intelligence.

Now let's discuss the different types of AI. So, AI is divided into three different evolutionary stages, or you can say that there are three stages

of artificial intelligence. Of course, we have artificial narrow intelligence followed by artificial general intelligence,

and that is followed by artificial super intelligence. Artificial narrow intelligence, which is also known as weak AI,

it involves applying artificial intelligence only to specific task. So, many currently existing systems

that claim to use artificial intelligence are actually operating as weak AI focused on a narrowly defined specific problem

Let me give you an example of artificial narrow intelligence. Alexa is a very good example of weak AI.

It operates within unlimited pre-defined range of functions. There's no genuine intelligence or there is no self awareness,

despite being a sophisticated example of weak AI. The Google search engine, Sophia the humanoid, self-driving cars, and even the famous AlphaGo

fall under the category of weak AI. So guys, right now we're at the stage of artificial narrow intelligence or weak AI.

We actually haven't reached artificial general intelligence or artificial super intelligence, but let's look at what exactly it would be like

if we reach artificial general intelligence. Now artificial general intelligence which is also known as strong AI,

it involves machines that posses the ability to perform any intelligent task that a human being can. Now this is actually something

that a lot of people don't realize. Machines don't posses human-like abilities. They have a very strong processing unit

that can perform high-level computations, but they're not yet capable of doing the simple and the most reasonable things that a human being can.

If you tell a machine to process like a million documents, it'll probably do that in a matter of 10 seconds, or a minute, or even 10 minutes.

But if you ask a machine to walk up to your living room and switch on the TV, a machine will take forever to learn that,

because machines don't have the reasonable way of thinking. They have a very strong processing unit, but they're not yet capable

of thinking and reasoning like a human being. So that's exactly why we're still stuck on artificial narrow intelligence.

So far we haven't developed any machine that can fully be called strong AI, even though there are examples of AlphaGo Zero

which defeated AlphaGo in the game of Go. AlphaGo Zero basically learned in a span of four months. It learned on its own without any human intervention.

But even then, it was not classified as a fully strong artificial intelligence, because it cannot reason like a human being.

Moving onto artificial super intelligence. Now this is a term referring to the time when the capabilities of a computer

will surpass that of a human being. In all actuality, I'll take a while for us to achieve artificial super intelligence.

Presently, it's seen as a hypothetical situation as depicted in movies and any science fiction books wherein machines have taken over the world,

movies like Terminator and all of that depict artificial super intelligence. These don't exist yet,

which we should be thankful for, but there are a lot of people who speculate that artificial super intelligence

will take over the world by the year 2040. So guys, these were the different types or different stages of artificial intelligence.

To summarize everything, like I said before, narrow intelligence is the only thing that exist for now. We have only weak AI or weak artificial intelligence.

All the major AI technologies that you see are artificial narrow intelligence. We don't have any machines which are capable of thinking

like human beings or reasoning like a human being. Now let's move on and discuss the different programming language for AI.

So there are actually N number of language that can be used for artificial intelligence. I'm gonna mention a few of them.

So, first, we have Python. Python is probably the most famous language for artificial intelligence.

It's also known as the most effective language for AI, because a lot of developers prefer to use Python. And a lot of scientists are also comfortable

with the Python language. This is partly because the syntaxes which belong to Python are very simple

and they can be learned very easily. It's considered to be one of the most easiest language to learn.

And also many other AI algorithms and machine learning algorithms can be easily implemented in Python,

because there are a lot of libraries which are predefined functions for these algorithms. So all you have to do is you have to call that function.

You don't actually have to call your algorithm. So, Python is considered the best choice for artificial intelligence.

With Python stands R, which is a statistical programming language. Now R is one of the most effective language

and environment for analyzing and manipulating the data for statistical purpose. It is a statistical programming language.

So using R we can easily produce well designed publication quality plots, including mathematical symbol and formula, wherever needed.

If you ask me, I think R is also one of the easiest programming language to learn. The syntax is very similar to English language,

and it also has N number of libraries that support statistics, data science, AI, machine learning, and so on.

It also has predefined functions for machine learning algorithms, natural language processing, and so on.

So R is also a very good choice if you want to get started with programming languages for machine learning or AI.

Apart from this, we have Java. Now Java can also be considered as a good choice for AI development.

Artificial intelligence has a lot to do with search algorithms, artificial neural networks, and genetic programming,

and Java provides many benefits. It's easy to use. Debugging is very easy, package services.

There is simplified work with large scale projects. There's a good user interaction, and graphical representation of data.

It has something known as the standard widget toolkit, which can be used for making graphs and interfaces. So, graphic virtualization is actually

a very important part of AI, or data science, or machine learning for that matter. Let me list out a few more languages.

We also have something known as Lisp. Now shockingly, a lot of people have not heard of this language.

This is actually the oldest and the most suited language for the development of artificial intelligence. It is considered to be a language

which is very suited for the development of artificial intelligence. Now let me tell you that this language

was invented by John McCarthy who's also known as the father of artificial intelligence. He was the person who coined the term

artificial intelligence. It has the capability of processing symbolic information. It has excellent prototyping capabilities.

It is easy, and it creates dynamic objects with a lot of ease. There's automatic garbage collection in all of that.

But over the years, because of advancements, many of these features have migrated into many other languages.

And that's why a lot of people don't go for Lisp. There are a lot of new languages which have more effective features

or which have better packages you can see. Another language I like to talk about is Prolog. Prolog is frequently used in knowledge base

and expert systems. The features provided by Prolog include pattern matching, freebase data structuring,

automatic back tracking and so on. All of these features provide a very powerful and flexible programming framework.

Prolog is actually widely used in medical projects and also for designing expert AI systems. Apart from this, we also have C++,

we have SaaS, we have JavaScript which can also be used for AI. We have MATLAB, we have Julia.

All of these languages are actually considered pretty good languages for artificial intelligence. But for now, if you ask me

which programming language should I go for, I would say Python. Python has all the possible packages,

and it is very easy to understand and easy to learn. So let's look at a couple of features of Python. We can see why we should go for Python.

First of all, Python was created in the year 1989. It is actually a very easy programming language.

That's one of the reasons why a lot of people prefer Python. It's very easy to understand.

It's very easy to grasp this language. So Python is an interpreted, object-oriented, high-level programming language,

and it can be very easily implemented. Now let me tell you a few features of Python. It's very simple and easy to learn.

Like I mentioned, it is one of the easiest programming language, and it also free and open source.

Apart from that, it is a high-level language. You don't have to worry about anything like memory allocation.

It is portable, meaning that you can use it on any platform like Linux, Windows, Macintosh, Solaris, and so on.

It support different programming paradigms like object-oriented and procedure oriented programming, and it is extensible,

meaning that it can invoke C and C++ libraries. Apart from this, let me tell you that Python is actually gaining unbelievable huge momentum in AI.

The language is used to develop data science algorithms, machine learning algorithms, and IoT projects. The other advantages to Python also,

the fact that you don't have to code much when it comes to Python for AI or machine learning. This is because there are ready-made packages.

There are predefined packages that have all the function and algorithm stored. For example, there is something known as PiBrain,

which can be used for machine learning, NumPy which can be used for scientific computation, Pandas and so on.

There are N number of libraries in Python. So guys, I'm now going to go into depth of Python. I'm now going to explain Python to you,

since this session is about artificial intelligence. So, those of you who don't know much about Python or who are new to Python,

I will leave a couple of links in the description box. You all can get started with programming and any other concepts or any other doubts

that you have on Python. We have a lot of content around programming with Python or Python for machine learning and so on.

Now let's move on and talk about one of the most important aspects of artificial intelligence,

which is machine learning. Now a lot of people always ask me this question. Is machine learning and artificial intelligence

the same thing? Well, both of them are not the same thing. The difference between AI and machine learning

is that machine learning is used in artificial intelligence. Machine learning is a method through which you can feed a lot of data to a machine

and make it learn. Now AI is a vast of field. Under AI, we have machine learning, we have NLP,

we have expert systems, we have image recognition, object detection, and so on. We have deep learning also.

So, AI is sort of a process or it's a methodology in which you make machines mimic the behavior of human beings.

Machine learning is a way in which you feed a lot of data to a machine, so that it can make it's own decisions.

Let's get into depth about machine learning. So first, we'll understand the need for machine learning or why machine learning came into existence.

Now the need for machine learning begins since the technical revolution itself. So, guys, since technology became the center of everything,

we've been generating an immeasurable amount of data. As per research, we generate around 2.5 quintillion bytes of data every single data

every single day. And it is estimated that by this year, 2020, 1.7 mb of data will be created every second

for every person on earth. So as I'm speaking to you right now, I'm generating a lot of data.

Now your watching this video on YouTube also accounts for data generation. So there's data everywhere.

So with the availability of so much data, it is finally possible to build predictive models that can study and analyze complex data

to find useful insights and deliver more accurate results. So, top tier companies like Netflix and Amazon build such machine learning models

by using tons of data in order to identify any profitable opportunity and avoid any unwanted risk.

So guys, one thing you all need to know is that the most important thing for artificial intelligence is data.

For artificial intelligence or whether it's machine learning or deep learning, it's always data.

And now that we have a lot of data, we can find a way to analyze, process, and draw useful insights from this data

in order to help us grow businesses or to find solutions to some problems. Data is the solution.

We just need to know how to handle the data. And the way to handle data is through machine learning, deep learning,

and artificial intelligence. A few reasons why machine learning is so important is, number one, due to increase in data generation.

So due to excessive production of data, we need to find a method that can be used to structure, analyze, and draw useful insights from data,

this is where machine learning comes in. It is used to solve problems and find solutions through the most complex task faced by organizations.

Apart form this, we also needed to improve decision making. So by making use of various algorithms, machine learning can be used to make

better business decisions. For example, machine learning is used to focus sales. It is used to predict any downfalls n the stock market

or identify any sort of risk and anomalies. Other reasons include that machine learning helps us uncover patterns and trends in data.

So finding hidden patterns and extracting key insights fro data is the most important part of machine learning.

So by building predictive models and using statistical techniques, machine learning allows you to dig beneath the surface

and explode the data at a minute scale. Understanding data and extracting patterns manually takes a lot of time.

It'll take several days for us to extract any useful information from data. But if you use machine learning algorithms,

you can perform similar computations in less than a second. Another reason is we need to solve complex problems. So from detecting the genes

linked to the deadly ALS disease, to building self-driving cars, machine learning can be used

to solve the most complex problems. At present, we also found a way to spot stars which are 2,400 light years away from our planet.

Okay, all of this is possible through AI, machine learning, deep learning, and these techniques. So to sum it up,

machine learning is very important at present because we're facing a lot of issues with data. We're generating a lot of data,

and we have to handle this data in such a way that in benefits us. So that's why machine learning comes in.

Moving on, what exactly is machine learning? So let me give you a short history of machine learning. So machine learning was first coined by Arthur Samuel

in the year 1959, which is just three years from when artificial intelligence was coined.

So, looking back, that year was probably the most significant in terms of technological advancement, because most of the technologies today

are based on the concept of machine learning. Most of the AI technologies itself are based on the concept of

machine learning and deep learning. Don't get confused about machine learning and deep learning.

We'll discuss about deep learning in the further slides, where we'll also see the difference between AI, machine learning, and deep learning.

So coming back to what exactly machine learning is, if we browse through the internet, you'll find a lot of definitions about

what exactly machine learning is. One of the definitions I found was a computer program is said to learn from experience E

with respect to some class of task T and performance measure P if its performance at task in T, as measured by P, improves with experience E.

That's very confusing, so let me just narrow it down to you. In simple terms, machine learning is a subset of artificial intelligence

which provides machines the ability to learn automatically and improve with experience without being explicitly programmed to do so.

In the sense, it is the practice of getting machines to solve problems by gaining the ability to think.

But now you might be thinking how can a machine think or make decisions. Now machines are very similar to humans.

Okay, if you feed a machine a good amount of data, it will learn how to interpret, process, and analyze this data by using machine learning algorithms,

and it will help you solve world problems. So what happens here is a lot of data is fed to the machine.

The machine will train on this data and it'll build a predictive model with the help of machine learning algorithms

in order to predict some outcome or in order to find some solution to a problem. So it involves data.

You're gonna train the machine and build a model by using machine learning algorithms in order to predict some outcome

or to find a solution to a problem. So that is a simple way of understanding what exactly machine learning is.

I'll be going into more depth about machine learning, so don't worry if you have understood anything as of now. Now let's discuss a couple terms

which are frequently used in machine learning. So, the first definition that we come across very often is an algorithm.

So, basically, a machine learning algorithm is a set of rules and statistical techniques that is used to learn patterns from data

and draw significant information from it. Okay. So, guys, the logic behind a machine learning model

is basically the machine learning algorithm. Okay, an example of a machine learning algorithm is linear regression, or decision tree, or a random forest.

All of these are machine learning algorithms. We'll define the logic behind a machine learning model.

Now what is a machine learning model? A model is actually the main component of a machine learning process.

Okay, so a model is trained by using the machine learning algorithm. The difference between an algorithm and a model is that

an algorithm maps all the decisions that a model is supposed to take based on the given input

in order to get the correct output. So the model will use the machine learning algorithm

in order to draw useful insights from the input and give you an outcome that is very precise. That's the machine learning model.

The next definition we have is predictor variable. Now a predictor variable is any feature of the data that can be used to predict the output.

Okay, let me give you an example to make you understand what a predictor variable is. Let's say you're trying to predict the height of a person,

depending on his weight. So here your predictor variable becomes your weight, because you're using the weight of a person

to predict the person's height. So your predictor variable becomes your weight. The next definition is response variable.

Now in the same example, height would be the response variable. Response variable is also known as

the target variable or the output variable. This is the variable that you're trying to predict by using the predictor variables.

So a response variable is the feature or the output variable that needs to be predicted by using the predictor variables.

Next, we have something known as training data. Now training and testing data are terminologies that you'll come across very often

in a machine learning process. So training data is basically the data that I used to create the machine learning model.

So, basically in a machine learning process, when you feed data into the machine, it'll be divided into two parts.

So splitting the data into two parts is also known as data splicing. So you'll take your input data,

you'll divide it into two sections. One you'll call the training data, and the other you'll call the testing data.

So then you have something known as the testing data. The training data is basically used to create the machine learning model.

The training data helps the model to identify key trends and patterns which are essential to predict the output.

Now the testing data is, after the model is trained, it must be tested in order to evaluate how accurately it can predict an outcome.

Now this is done by using the testing data. So, basically, the training data is used to train the model. The testing data is used to test

the efficiency of the model. Now let's move on and get our next topic, which is machine learning process.

So what is the machine learning process? Now the machine learning process involves building a predictive model

that can be used to find a solution for a problem statement. Now in order to solve any problem in machine learning,

there are a couple of steps that you need to follow. Let's look at the steps. The first step is you define the objective of your problem.

And the second step is data gathering, which is followed by preparing your data, data exploration, building a model,

model evaluation, and finally making predictions. Now, in order to understand the machine learning process, let's assume that you've been given a problem

that needs to be solved by using machine learning. So the problem that you need to solve is we need to predict the occurrence of rain

in your local area by using machine learning. So, basically, you need to predict the possibility of rain by studying the weather conditions.

So what we did here is we basically looked at step number one, which is define the objective of the problem.

Now here you need to answer questions such as what are we trying to predict. Is that output going to be a continuous variable,

or is it going to be a discreet variable? These are the kinds of questions that you need to answer in the first page,

which is defining the objective of the problem, right? So yeah, exactly what are the target feature. So here you need to understand

which is your target variable and what are the different predictor variables that you need in order to predict this outcome.

So here our target variable will be basically a variable that can tell us whether it's going to rain or not.

Input data is we'll need data such as maybe the temperature on a particular day or the humidity level, the precipitation, and so on.

So you need to define the objective at this stage. So basically, you have to form an idea of the problem at this storage.

Another question that you need to ask yourself is what kind of problem are you solving. Is this a binary classification problem,

or is this a clustering problem, or is this a regression problem? Now, a lo of you might not be familiar

with the terms classification clustering and regression in terms of machine learning. Don't worry, I'll explain all of these terms

in the upcoming slides. All you need to understand at step one is you need to define how you're going to solve the problem.

You need to understand what sort of data you need to solve the problem, how you're going to approach the problem,

what are you trying to predict, what variables you'll need in order to predict the outcome, and so on.

Let's move on and look at step number two, which is data gather. Now in this stage, you must be asking questions such as,

what kind of data is needed to solve this problem? And is this data available? And if it is available, from where can I get this data

and how can I get the data? Data gathering is one of the most time-consuming steps in machine learning process.

If you have to go manually and collect the data, it's going to take a lot of time. But lucky for us, there are a lot of resources online,

which were wide data sets. All you need to do is web scraping where you just have to go ahead and download data.

One of the websites I can tell you all about is Cargill. So if you're a beginner in machine learning, don't worry about data gathering and all of that.

All you have to do is go to websites such as cargill and just download the data set. So coming back to the problem that we are discussing,

which is predicting the weather, the data needed for weather forecasting includes measures like humidity level,

the temperature, the pressure, the locality, whether or not you live in a hill station, such data has to be collected or stored for analysis.

So all the data is collected during the data gathering stage. This step is followed by data preparation,

or also known as data cleaning. So if you're going around collecting data, it's almost never in the right format.

And eve if you are taking data from online resources from any website, even then, the data will require cleaning and preparation.

The data is never in the right format. You have to do some sort of preparation and some sort of cleaning

in order to make the data ready for analysis. So what you'll encounter while cleaning data is you'll encounter a lot of inconsistencies

in the data set, like you'll encounter som missing values, redundant variables, duplicate values, and all of that.

So removing such inconsistencies is very important, because they might lead to any wrongful computations and predictions.

Okay, so at this stage you can scan the data set for any inconsistencies, and you can fix them then and there.

Now let me give you a small fact about data cleaning. So there was a survey that was ran last year or so. I'm not sure.

And a lot of data scientists were asked which step was the most difficult or the most annoying and time-consuming of all.

And 80% of the data scientist said it was data cleaning. Data cleaning takes up 80% of their time.

So it's not very easy to get rid of missing values and corrupted data. And even if you get rid of missing values,

sometimes your data set might get affected. It might get biased because maybe one variable has too many missing values,

and this will affect your outcome. So you'll have to fix such issue, we'll have to deal with all of this missing data

and corrupted data. So data cleaning is actually one of the hardest steps in machine learning process.

Okay, now let's move on and look at our next step, which is exploratory data analysis. So here what you do is basically become

a detective in the stage. So this stage, which is EDA or exploratory data analysis, is like the brainstorming stage of machine learning.

Data exploration involves understanding the patterns and the trends in your data. So at this stage, all the useful insights are drawn

and any correlations between the various variables are understood. What do I mean by trends and patterns and correlations?

Now let's consider our example which is we have to predict the rainfall on a particular day.

So we know that there is a strong possibility of rain if the temperature has fallen law. So we know that our output will depend on

variables such as temperature, humidity, and so on. Now to what level it depends on these variables, we'll have to find out that.

We'll have to find out the patterns, and we'll find out the correlations between such variables.

So such patterns and trends have to be understood and mapped at this stage. So this is what exploratory data analysis is about.

It's the most important part of machine learning. This is where you'll understand what exactly your data is

and how you can form the solution to your problem. The next step in a machine learning process is building a machine learning module.

So all the insights and the patterns that you derive during the data exploration are used to build a machine learning model.

So this stage always begins by splitting the data set into two parts, which is training data and testing data. I've already discussed with you

that the data that you used in a machine learning process is always split into two parts. We have the training data and we have the testing data.

Now when you're building a model, you always use the training data. So you always make use of the training data

in order to build the model. Now a lot of you might be asking what is training data. Is it different from the input data

that you're feeding with the machine or is it different from the testing data? Now training data is the same input data

that you're feeding to the machine. The only difference is that you're splitting the data set into two.

You're randomly picking 80% of your data and you're assigning for training purpose. And the rest 20%, probably,

you'll assign it for testing purpose. So guys, always remember another thing that the training data is always much more

than your testing data, obviously because you need to train your machine. And the more data you feed the machine

during the training phase, the better it will be during the testing phase. Obviously, it'll predict better outcomes

if it is being trained on more data. Correct? So the model is basically using

the machine learning algorithm that predicts the output by using the data fed to it. Now in the case of predicting rainfall,

the output will be a categorical variable, because we'll be predicting whether it's going to rain or not.

Okay, so let's say we have an output variable called rain. The two possible values that this variable can take is yes it's going to rain and no it won't rain.

Correct, so that is out come. Our outcome is a classification or a categorical variable. So for such cases where your outcome

is a categorical variable, you'll be using classification algorithms. Again, example of a classification algorithm

is logistic regression or you can also support vector machines, you can use K nearest neighbor,

and you can also use naive Bayes, and so on. Now don't worry about these terms, I'll be discussing all these algorithms with you.

But just remember that while you're building a machine learning model, you'll make use of the training data.

You'll train the model by using the training data and the machine learning algorithm. Now like I said, choosing the machine learning algorithm,

depends on the problem statement that you're trying to solve because of N number of machine learning algorithms.

We'll have to choose the algorithm that is the most suitable for your problem statement. So step number six is model evaluation

and optimization. Now after you've done building a model by using the training data set,

it is finally time to put the model road test. The testing data set is used to check the efficiency of the model

and how accurately it can predict the outcome. So once the accuracy is calculated, any further improvements in the model

can be implemented during this stage. The various methods that can help you improve the performance of the model,

like you can use parameter tuning and cross validation methods in order to improve the performance of the model.

Now the main things you need to remember during model evaluation and optimization is that model evaluation is nothing but

you're testing how well your model can predict the outcome. So at this stage, you will be using the testing data set. In the previous stage, which is building a model,

you'll be using the training data set. But in the model evaluation stage, you'll be using the testing data set.

Now once you've tested your model, you need to calculate the accuracy. You need to calculate how accurately

your model is predicting the outcome. After that, if you find that you need to improve your model in some way or the other,

because the accuracy is not very good, then you'll use methods such as parameter tuning. Don't worry about these terms,

I'll discuss all of this with you, but I'm just trying to make sure that you're understanding the concept

behind each of the phases and machine learning. It's very important you understand each step. Okay, now let's move on and look at

the last stage of machine learning, which is predictions. Now, once a model is evaluated and once you've improved it,

it is finally used to make predictions. The final output can either be a categorical variable or a continuous variable.

Now all of this depends on your problem statement. Don't get confused about continuous variables, categorical variables.

I'll be discussing all of this. Now in our case, because we're predicting the occurrence of rainfall,

the output will be categorical variable. It's obvious because we're predicting whether it's going to rain or not.

The result, we understand that this is a classification problem because we have a categorical variable.

So that was the entire machine learning process. Now it's time to learn about the different ways in which machines can learn.

So let's move ahead and look at the types of machine learning. Now this is one of the most

interesting concepts in machine learning, the three different ways in which machines learn. There is something known as supervised learning,

unsupervised learning, and reinforcement learning. So we'll go through this one by one. We'll understand what supervised learning is first,

and then we'll look at the other two types. So defined supervised learning, it is basically a technique in which we teach

or train the machine by using the data, which is well labeled. Now, in order to understand supervised learning,

let's consider a small example. So, as kids, we all needed guidance to solve math problems. A lot of us had trouble solving math problems.

So our teachers always help us understand what addition is an dhow it is done. Similarly, you can think of supervised learning

as a type of machine learning that involves a guide. The label data set is a teacher

that will train you to understand the patterns in the data. So the label data set is nothing but the training data set. I'll explain more about this in a while.

So, to understand supervised learning better, let's look at the figure on the screen. Right here we're feeding the machine

image of Tom and Jerry, and the goal is for the machine to identify and classify the images into two classes.

One will contain images of Tom and the the other will contain images of Jerry. Now the main thing that you need to note

in supervised learning is a training data set. The training data set is going to be very well labeled. Now what do I mean when I say that

training data set is labeled. Basically, what we're doing is we're telling the machine this how Tom looks and this is how Jerry looks.

By doing this, you're training the machine by using label data. So the main thing that you're doing is you're labeling

every input data that you're feeding to the model. So, basically, you're entire training data set is labeled. Whenever you're giving an image of Tom,

there's gonna be a label there saying this is Tom. And when you're giving an image of Jerry, you're saying that this is how Jerry looks.

So, basically, you're guiding the machine and you're telling that, "Listen, this is how Tom looks, "this is how Jerry looks,

"and now you need to classify them "into two different classes." That's how supervised learning works.

Apart from that, it's the same old process. After getting the input data, you're gonna perform data cleaning.

Then there's exploratory data analysis, followed by creating the model by using the machine learning algorithm,

and then this is followed by model evaluation, and finally, your predictions. Now, one more thing to note here is that

the output that you get by using supervised learning is also labeled output. So, basically, you'll get two different classes

of name Tom and one of name Jerry, and you'll get them labeled. That is how supervised learning works.

The most important thing in supervised learning is that you're training the model by using labeled data set.

Now let's move on and look at unsupervised learning. We look at the same example and understand how unsupervised learning works.

So what exactly is unsupervised learning? Now this involves training by using unlabeled data and allowing the model to act on that information

without any guidance. Alright. Like the name suggest itself,

there is no supervision here. It's unsupervised learning. So think of unsupervised learning as a smart kid

that learns without any guidance. Okay, in this type of machine learning, the model is not fed with any label data,

as in the model has no clue that this is the image of Tom and this is Jerry. It figures out patterns

and the difference between Tom and Jerry on its own by taking in tons and tons of data. Now how do you think the machine identifies this as Tom,

and then finally gives us the output like yes this is Tom, this is Jerry. For example, it identifies prominent features of Tom,

such as pointy ears, bigger in size, and so on, to understand that this image is of type one. Similarly, it finds out features in Jerry,

and knows that this image is of type two, meaning that the first image is different from the second image.

So what the unsupervised learning algorithm or the model does is it'll form two different clusters. It'll form one cluster which are very similar,

and the other cluster which is very different from the first cluster. That's how unsupervised learning works.

So the important things that you need to know in unsupervised learning is that you're gonna feed the machine unlabeled data.

The machine has to understand the patterns and discover the output on its own. And finally, the machine will form clusters

based on feature similarity. Now let's move on and locate the last type of machine learning,

which is reinforcement learning. Reinforcement learning is quite different when compared to supervised and unsupervised learning.

What exactly is reinforcement learning? It is a part of machine learning where an agent is put in an environment,

and he learns to behave in this environment by performing certain actions, and observing the rewards which is gets from those actions.

To understand what reinforcement learning is, imagine that you were dropped off at an isolate island. What would you do?

Now panic. Yes, of course, initially, we'll all panic. But as time passes by, you will learn

how to live on the island. You will explode the environment, you will understand the climate conditions,

the type of food that grows there, the dangers of the island so on. This is exactly how reinforcement learning works.

It basically involves an agent, which is you stuck on the island, that is put in an unknown environment, which is the island,

where he must learn by observing and performing actions that result in rewards. So reinforcement learning is mainly used

in advanced machine learning areas such as self-driving cars and AlphaGo. I'm sure a lot of you have heard of AlphaGo.

So, the logic behind AlphaGo is nothing but reinforcement learning and deep learning. And in reinforcement learning,

there is not really any input data given to the agent. All he has to do is he has to explore everything from scratch

it's like a newborn baby with no information about anything. He has to go around exploring the environment, and getting rewards, and performing some actions

which results in either rewards or in some sort of punishment. Okay.

So that sums up the types of machine learning. Before we move ahead, I'd like to discuss the difference

between the three types of machine learning, just to make the concept clear to you all. So let's start by looking at the definitions of each.

In supervised learning, the machine will learn by using the label data. In unsupervised learning, they'll be unlabeled data,

and the machine has to learn without any supervision. In reinforcement learning, there'll be an agent which interacts with the environment

by producing actions and discover errors or rewards based on his actions. Now what are the type of problems

that can be solved by using supervised, unsupervised, and reinforcement learning. When it comes to supervised learning,

the two main types of problems that are solved is regression problems and classification problems. When it comes to unsupervised learning,

it is association and clustering problems. When it comes to reinforcement learning, it's reward-based problems.

I'll be discussing regression, classification, clustering, and all of this in the upcoming slides, so don't worry if you don't understand this.

Now the type of data which is used in supervised learning is labeled data. In unsupervised learning, it unlabeled.

And in reinforcement learning, we have no predefined data set. The agent has to do everything from scratch.

Now the type of training involved in each of these learnings. In supervised learning, there is external supervision,

as in there is the labeled data set which acts as a guide for the machine to learn. In unsupervised learning, there's no supervision.

Again, in reinforcement learning, there's no supervision at all. Now what is the approach to solve problems

by using supervised, unsupervised, and reinforcement learning? In supervised learning, it is simple.

You have to mal the labeled input to the known output. The machine knows what the output looks like. So you're just labeling the input to the output.

In unsupervised learning, you're going to understand the patterns and discover the output.

Here you have no clue about what the input is. It's not labeled. You just have to understand the patterns

and you'll have to form clusters and discover the output. In reinforcement learning, there is no clue at all. You'll have to follow the trial and error method.

You'll have to go around your environment. You'll have to explore the environment, and you'll have to try some actions.

And only once you perform those actions, you'll know that whether this is a reward-based action or whether this is a punishment-based action.

So, reinforcement learning is totally based on the concept of trial and error. Okay.

A popular algorithm on the supervised learning include linear regression, logistic regressions, support vector machines K nearest neighbor,

naive Bayes, and so on. Under unsupervised learning, we have the famous K-means clustering method, C-means

and all of that. Under reinforcement learning, we have the famous learning Q-learning algorithm.

I'll be discussing these algorithms in the upcoming slides. So let's move on and look at the next topic, which is the types of problems solved

using machine learning. Now this is what we were talking about earlier when I said regression, classification,

and clustering problems. Okay, so let's discuss what exactly I mean by that. In machine learning, all the problems can be

classified into three types. Every problem that is approached in machine learning can be put interest one of these three categories.

Okay, so the first type is known as a regression, then we have classification and clustering. So, first, let's look at regression type of problems.

So in this type problem, the output is always a continuous quantity. For example, if you want to predict

the speed of a car, given the distance, it is a regression problem. Now a lot of you might not be very aware

of what exactly a continuous quantity is. A continuous quantity is any quantity that can have an infinite range of values.

For example, The weight of a person, it is a continuous quantity, because our weight can be 50, 50.1,

50.001, 5.0021, 50.0321 and so on.

It can have an infinite range of values, correct? So the type of problem that you have to predict a continuous quantity to make use of regression algorithms.

So, regression problems can be solved by using supervised learning algorithms like linear regression.

Next, we have classification. Now in this type of problem, the output is always a categorical value.

Now when I say categorical value, it can be value such as the gender of a person is a categorical value.

Now classifying emails into two two classes like spam and non-spam is a classification problem that can be solved by using

supervised learning classification algorithms, like support vector machines, naive Bayes, logistic regression, K nearest neighbor, and so on.

So, again, the main aim in classification is to compute the category of the data. Coming to clustering problems.

This type of problem involves assigned input into two or more clusters based on feature similarity.

Thus when I read this sentence, you should understand that this is unsupervised learning, because you don't have enough data about your input,

and the only option that you have is to form clusters Categories are formed only when you know that your data is of two type.

Your input data is labeled and it's of two types, so it's gonna be a classification problem. But when a clustering problem happens,

when you don't have much information about your input, all you have to do is you have to find patterns and you have to understand that

data points which are similar are clustered into one group, and data points which are different from the first group

are clustered into another group. That's what clustering is. An example is in Netflix what happens is

Netflix clusters their users into similar groups based on their interest, based on their age, geography, and so on.

This can be done by using unsupervised learning algorithms like K-means. Okay.

So guys, there were the three categories of problems that can be solved by using machine learning. So, basically, what I'm trying to say

is all the problems will fall into one of these categories. So any problem that you give to a machine learning model, it'll fall into one of these categories.

Okay. Now to make things a little more interesting, I have collected real world data sets

from online resources. And what we're gonna do is we're going to try and understand if this is a regression problem,

or a clustering problem, or a classification problem. Okay. Now the problem statement in here

is to study the house sales data set, and build a machine learning model that predicts the house pricing index.

Now the most important thing you need to understand when you read a problem statement is you need to understand what is your target variable,

what are the possible predictor variable that you'll need. The first thing you should look at is your targe variable. If you want to understand if this a classification,

regression, or clustering problem, look at your target variable or your output variable that you're supposed to predict.

Here you're supposed to predict the house pricing index. Our house pricing index is obviously a continuous quantity.

So as soon as you understand that, you'll know that this is a regression problem. So for this, you can make use of

the linear regression algorithm, and you can predict the house pricing index. Linear regression is the regression algorithm.

It is a supervised learning algorithm. We'll discuss more about it in the further slides. Let's look at our next problem statement.

Here you have to study a bank credit data set, and make a decision about whether to approve the loan of an applicant

based on his profile. Now what is your output variable over here? Your output variable is to predict whether you can

approve the loan of a applicant or not. So, obviously, your output is going to be categorical. It's either going to be yes or no.

Yes is basically approved loan. No is reject loan. So here, you understand that

this is a classification problem. Okay. So you can make use of algorithms like KNN algorithm

or you can make use of support vector machines in order to do this. So, support vector machine and KNN

which is K nearest neighbor algorithms are basically supervised learning algorithm. We'll talk more about that in the upcoming slides.

Moving on to our next problem statement. Here the problem statement is to cluster a set of movies as either good or average

based on the social media outreach. Now if you look properly, your clue is in the question itself.

The first line it says is to cluster a set of movies as either good or average. Now guys, whenever you have a problem statement

that is asking you to group the data set into different groups or to form different, different clusters,

it's obviously a clustering problem. Right here you can make use of the K-means clustering algorithm,

and you can form two clusters. One will contain the popular movies and the other will contain the non-popular movies.

These alright small examples of how you can use machine learning to solve clustering problem, the regression, and classification problems.

The key is you need to identify the type of problem first. Now let's move on and discuss the different types of machine learning algorithms.

So we're gonna start by discussing the different supervised learning algorithms. So to give you a quick overview,

we'll be discussing the linear regression, logistic regression, and decision tree, random forest, naive Bayes classifier,

support vector machines, and K nearest neighbor. We'll be discussing these seven algorithms. So without any further delay,

let's look at linear regression first. Now what exactly is a linear regression algorithm? So guys, linear regression is basically

a supervised learning algorithm that is used to predict a continuous dependent variable y based on the values of independent variable x.

Okay. The important thing to note here is that the dependent variable y,

the variable that you're trying to predict, is always going to be a continuous variable. But the independent variable x,

which is basically the predictor variables, these are the variables that you'll be using to predict your output variable,

which is nothing but your dependent variable. So your independent variables or your predictive variables can either be continuous or discreet.

Okay, there is not such a restriction over here. Okay, they can be either continuous variables or they can be discreet variables.

Now, again, I'll tell you what a continuous variable is, in case you've forgotten. It is a vary that has infinite number of possibilities.

So I'll give you an example of a person's weight. It can be 160 pounds, or they can weigh 160.11 pounds, or 160.1134 pounds and so on.

So the number of possibilities for weight is limitless, and this is exactly what a continuous variable is. Now in order to understand linear regression,

let's assume that you want to predict the price of a stock over a period of time. Okay.

For such a problem, you can make use of linear regression by starting the relationship between the dependent variable,

which is the stock price, and the independent variable, which is the time. You're trying to predict the stock price

over a period of time. So basically, you're gonna check how the price of a stock varies over a period of time.