Understanding Data Science: Concepts, Importance, and Analytics Lifecycle

foreign [Music] students in this course we are going to

discuss about the data science and its tool boxes okay so in this lecture we are going to tell about the concept and

the need of data science why we are actually studying the data science after this I am going to tell you about the

life cycle of data analytics okay so first I am starting with the data science

before diving into the details of data science I am going to tell you about the data first this data is generated

randomly a lot of data millions of data is generated in one second okay so there is a need to process this data we are

going to give the processing of data different term that is data science okay so it is the task of scrutinizing and

processing the raw data so that we can reach a meaningful Insight on that so that we can reach a meaningful

conclusion on that generally what happens the data is unstructured by looking at the data we are not able

to draw any conclusion okay so for that we have to arrange the data in some manner we have to process that raw data

so that we can conclude something we can reach up to a meaningful conclusion and a lot of data is mind and classified so

that we can detect and study the behavioral pattern and the data generally what happens the data which is

generated that is in different different formats by looking at the data it is not able to draw any meaningful thing draw

any meaningful pattern so for that we classify the data we mine the data so that we find out the meaningful insights

or the patterns out of that data and this classification or mining is very important when we look at the various

aspects like when we talk about the sentiment analysis this mining of data this classification of data is very

important how this is important someone is putting the past on Twitter okay by looking at that post we are not able to

you know identify anything we are not able to see what kind of sentiments that person is having by looking at the

different different posts by having the analysis of different different posts if we see that we are able to identify the

sentiments of that person if we you know if we are able to reach up to a conclusion that person can be

saved from a you know unfortunate incident so that is why because data science is

very very important okay so if we look at the graph what is the need of this if we talk about the previous years

there was just two Zeta bytes of data in one second but if we see in the coming years say two two zero two five twenty

twenty five there will be approximately 180 Zeta bytes of data in one second what is the fun of having this data how

will we able to draw the conclusions out of that data so for that we need to apply the data science on that there are

various tools available there are various Technologies available for the data side depending upon the behavior of

data depending upon the patterns of data we will identify the technique and we will apply that on the data so that we

can get the meaningful conclusion out of that okay otherwise there is no fun of having this much data we will not be

able to identify anything out of that data there is just random data there is just unstructured data okay so for

having the conclusions in our hand we will apply the data science Technologies data science tools on that okay so in

this course we are going to study about different different tool boxes we are going to tell you about the r r

programming then we are going to tell you about the w we are going to tell you about the python and I'm a lot of

toolboxes will be started in this course so if we see at this data on the left hand side the data is in unstructured

form there is no pattern we can identify from this data okay so if I want to draw some conclusion out of that data there

is nothing okay out of this data I cannot see any pattern so if I want to find out any meaningful thing out of

this data I have to apply any data science technique on that okay so depending upon the pattern depending

upon the behavior of that data I will identify one of the technique and I will Implement that technique on that data

okay so once this technique is implemented our data is in structured form okay so to process the data from

unstructured form to structured form we have to apply some pre-processing on that we have to clean the data we have

to integrate the data we have to transform the data so once this is done our data is in a proper shape

with this proper shape we are able to identify actually what is the conclusion what is

the pattern what is the behavior here okay so when we see on the right hand side of this data we can find out that

like this is the similarity and this is not the similarity okay we can study this particular pattern but if we look

at the left hand side there is no conclusion this is just the random and unstructured data okay so if we talk

about the data itself that can be in various forms basically generally it is in four forms

that is not your data Neuer means nominal ordinal interval and ratio data it can be in any form it can be nominal

form it can be in ordinal form it can be interval form and it can be in ratio form okay so we have to see which

particular form it is following or whether it is a mix up of two or three forms first of all we have to identify

that particular thing the details of this data type will be discussed in the next lectures

so this data can be collected from any source if we talk about the sources of this this can be of two type this can be

of primary source or it can derive from a scan resource okay these primary and scan resources means the primary source

means from where the data is originated from where the data is originated it can be from the questionnaires it can be

from the interviews it can be from the you know different different websites it can be from the secondary source also

okay so the classification is it can be from the primary source or it can be from this country Source based upon that

it is identified as the primary data and the secondary data if we talk about the publicly available data sets there are

various websites which provide these data sets we have the kaggle data sets we have UCI machine repositories we have

IMDb data sets and if we talk about the medical data sites either it is belong to belonging to the cancer diseases or

to the neuromusculars or to the neurophysiological diseases we have a huge amount of data and if we talk about

a particular data set that contains um more than 40 000 genes if we talk about just one sample do we really think

these 40 000 genes adds up to one diseases no there is only one out of these 40 000 genes which actually is

used for the diagnosis of disease so what we are doing here we are going to identify that one gene out of those 40

000 genes which is actually harming the person and is causing the disease so for that we have to implement one of the

data science technique here okay so we will discuss various kinds of data science techniques in this and we have

one more that is Stanford large Network database collections so on these websites the data set is

freely available for experimentation purpose we can download the data set and we can perform our experiments and we

can Implement Implement our techniques on that so

apart from the generation of data apart from these random and unstructured data what is the need of studying data

science I am going to tell you about its need in different different sectors so if we talk about the e-commerce sector

what is the need of this for maximizing the revenue and profitability if we study the previous patterns of this we

can easily identify what is going to happen in the future what is going to happen in the future if we are going to

identify that that will somehow benefit benefit us in maximizing the revenue and the profit here then we have the finance

why we are studying the data science for finance so that we can take care the risk analysis we can find out the fraud

we can work on the Capital Management okay so in finance we can see the patterns here we can study the previous

patterns based upon the study of previous pattern we can identify what is going to happen in the future if

something is going to happen in the future and we are able to detect that then we can easily control that if we

are easily control that then we can say that we are able to you know detect the fraud which is going to happen we are

able to you know identify the risk which can happen on the data and in a retail market for the optimal pricing for

better marketing strategies and for stock management we are going to see the use of data science here then in

healthcare I have already explained how it can be useful for healthcare like for the diagnosis of disease once the

disease is diagnosed then for the patient care we can use it for the diagonal for the proper treatment for

the identification of a proper medicine for the patient we can apply the data science techniques on that okay so for

quality for patient quality care for classification of types of symptoms of patient and for predicted Health

deficiencies we can have the data science in healthcare then after that in education sector in

academic institutions for better admission scenarios for empowerment of students for successful examination

results and for all round student performance we have the need of studying data science in education sector in

human resource for building the strong leadership for employee acquisition for employer retention and Performance

Management we require the data science in human resource sector then after that in supports to analyze

the performance of these uh players we have to use the data science Technologies in that how different

players have responded in previous games based upon that we can identify how these players are going to give the

results or going to perform in this particular match based upon that we can have this strategy so that we can win

okay so in sports sector to analyze the performance of player to analyze the predicted scores to prevention of

injuries and the possibility of winning or either losing a match by a particular t Okay so for this we require the data

science toolboxes so for this there is a particular life cycle data science is a particular like

the data has to go through a particular life cycle we cannot identify a pattern we cannot reach to a conclusion from the

unstructured data it has to go through various phases so that we can have a meaningful pattern okay so under this

term we have one more thing that is data analytics okay this data analytics is comprised of six phases which is

actually a subset of the data science this data science is an umbrella term whereas this data analytics is a subset

of the data science so let's understand first what is data analytics once the data analytics is done then we will

finally move to the data science so this data analytics is comprised of six phases which are carried out in a cycle

so this cycle is this one first of all data has to be discovered we have to identify the previous things we have to

identify the previous you know patterns we have to identify the previous data we have to see which particular thing is

available with this in the office we have to see which particular thing is not available with us in the office so

once the whole data is discovered we have to prepare that data in a particular form

okay so that is the second phase that is data preparation once the data is prepared then we are going to see the

third space that is planning of data models we have to see which particular technique which particular model can be

applied on that okay so once this is identified the fourth phase is the building of data models we have to

actually Implement that particular technique on that data so this comes under fourth phase that is building of

data models once the implementation is done once all the things are implemented then we are going to communicate the

results we are going to communicate with the results with the business parties we are going to communicate the result with

the stakeholders all the stakeholders what are these stakeholders that we will discuss in this lecture only then once

this is done whether the result is positive whether we are successful or we are you know a failure then we are going

to operationalize realization it okay so these are the six phases of data analytics so these are the six phases

data Discovery data preparation then data then model planning the fourth one is building of model then communication

of result then operationalization so first thing is data discovery here the stakeholders will regularly

perform some task these tasks are examine the business Trends what is actually happening in the business what

are the trends here what particular thing is you know most famous what particular thing is not people taking

here okay so first of all they have to examine the business Trends then we have to make the case studies of the similar

data analytics so that we can have a success here and then we are going to study the domain of business industry so

these are the different things that stake here stakeholders will regularly perform and the team will make an

assessment of In-House risk resources means what are the things which are already available with us in the office

what is the infrastructure there and what is the technology requirement and how much time is required to actually

give the results okay so for these we are going to discover the data okay so this is done in the first phase of data

analytics so after all the evaluations after all the assessments the stakeholders will

start formulate the initial hypothesis for resolving all the business challenges in the terms of current

market scenario they will analyze what is the trend here they will analyze what is actually available with us so based

upon that if there is any challenge they are going to solve that challenge so that is basically done in the first

phase itself that is data Discovery phase once all the challenges are solved here we are having one particular data

with us that data is known as the particular data set okay so this is done in the data Discovery once the data is

there then we will move to the next phase of data analytics that is data preparation so the data is prepared here

by transferring it to a form that is the Legacy system into a data analytics form by using any kind of platform these

platform are sandbox platform if we talk about one big organization that is IBM they use IBM and it is a 1000 this is a

Sandbox platform which is used by the IBM company so that they can handle their data marks okay otherwise it is

not able to prepare like it is not possible to prepare the data in a particular form they have to use some

sandbox platform for that okay so I have given one example of IBM company they use IBM net is a 1000 this is a Sandbox

platform once the data is prepared once the data is in a particular form then next thing is the model planning what we

do in this this see here the proper planning of methods which are going to be adopted and the various workflows

that are going to be followed during the next phase what actually we are going to implement which technique will be used

here which technique or which workflow will be followed here that we are going to decide at this phase only okay so

this particular phase is known as model planning basically we are planning then we are planning for the next coming

phase the various divisions of the work among the team is done at this space and the feature selection basically what

we are doing here once the data is in a particular form we are going to identify the important features here what is the

relationship between these features out of all the features which particular features are giving us most you know

most important thing that we are going to select here so this particular phenomenon is known as feature selection

so at this phase only we are going to perform the feature selection so that we can have the hundred percent results

here okay so at this point of time we are going to decide which particular team will work here

we are going to decide which features are going we are going to select we are going to decide which particular

Technique we are going to implement which workflow will be followed here once this is done the next thing we have

is the model building now this is the implementation phase the things which we have decided till now the data which we

have prepared till now that will be used in this phase for the implementation so we are going to build the model here now

basically how this model is built we have a complete data set okay we have a complete data set now that whole data

set is divided into two parts first part is the training data set the second one is the testing data set the

model which we have planned the techniques which we have decided that we are going to implement on the training

data set once the model is built we have seen the performance of that model on the training data set then we are going

to test that for the testing data set whether that is you know correct or not whether that is going to give the

results on The Unknown data or not okay but for that first of all we have to prepare the data set into two parts

generally what happens we divide the data set into two things that is 70 of the data is for

training and the rest thirty percent of the data is for testing seventy percent we use for you know implementing it so

that it can train it the model can be trained once the model is trained on the training data set we test it on the

testing data set that testing data set accuracy or the results on that testing data set is actually used to see the

performance of the model which we have built okay so in this particular one we are going to execute the model in model

building phase once this is done the next thing is the communication of the result we have to check the result of

the project to see whether it is a success or it is a failure the inferences which are drawn from that

particular thing we are going to communicate that with the entirety that will be going to be summarized and we

are going to elaborate the narratives on the key findings okay on the key findings we are going to summarize it

once the results are communicated the next thing is the operationalization means everything is done the challenges

which were there we have solved that the next thing is the making of the final report so the team will be there which

are going to you know make the final report along with the briefings along with the source codes and the related

documents it involves the running of the pilot project in real time and it tests the project in real time environment

okay so these are the six phases of the data analytics this is known as the data analytics life cycle okay so I am going

I am going to give you a brief of that first of all we are going to discover the data we are going to see the trends

we are going to the previous see we are going to see the previous results after that we are going to prepare the data in

a particular form so that we can build the model on that once the data is prepared the next phase the third phase

is what the model planning we are going to plan which particular team is going to work there we are going to plan which

you know technique will be implemented there whilst everything is finalized in fourth phase that is building of model

we are going to actually implement the model of that on on that okay so for that the data set will be divided into

two parts training data set and testing data set and the actual results will be seen on the testing data set

once this is done in the next phase we are going to communicate the result to all the stakeholders after that

everything is done then we are going to make the final report in that we are going to include all the

source codes all the related documents all the briefings okay so these are the six phases so in this lecture we have

actually seen what is data science what is the need of studying data science and what is actually the life cycle of data

analytics so that's all for now guys thank you so much