Complete System Design Course: Scalable Architectures & Key Concepts

Welcome back aliens. My name is Vin Reddy, and in this video we'll talk about system design. I know you are

waiting for a long time for a system design course on Telesco, and finally we are here. Now, in this course we are

going to understand different aspect of system design. Now, why it is important? When you go for the interviews, most of

the companies will ask you for system design knowledge, and the question will be based on that. Now, of course they

will not be saying this is a system design round, but they will expect from you to understand different concepts

here, and how will you solve a complex problem. And let me give you a statement which highlights this, and I have it

here. Anyone can write a code that works. A system design is what makes it work for million people at once, and

that's the power of system design. See, when you write a code, of course when you learn a language, you learn a

framework, you can build application. That framework can I mean, the entire project will work. The moment you deploy

it, the moment it is in production, and when you have users, not just 10 users, of course it will work for 10 users, and

it should work for 10 users, but the moment you have 10 million users accessing that at the same time, will it

work? That's the question. When you talk about all your favorite applications, it can be WhatsApp, YouTube, Instagram, or

Amazon for that matter, there are not few users who are accessing that at one time. There are millions of users, but

still if you can see, they work. So, how do we do it? And that's where system design plays a huge role. Now, to

understand this, we have we will be going through these concepts. So, in this particular video, we are going to

start with the basics of system design, what is system design, what are different components, data versus

compute intensive, and functional and non-functional, and all this comes under foundation. Now, once that is done,

we'll talk about how the communication happens between different systems. So, we'll focus on DNS, API, and of course

REST, which you might be knowing if you have worked with Spring Boot or Fast or Django and any web framework. Then we'll

move towards the data. Of course, when you have a lot of users, how will you store data? Not just their personal

data, but their activities. Uh and in that we have different options like SQL databases, no SQL databases, when to use

cache, and when to use SQL and no non-SQL. Then, let's say your users are growing day by day. How are you are

going to manage your application? And that's where scaling and distribution comes into picture.

In which we'll talk about load balancers, replication, partitioning, and the CAP theorem. That this will be

an interesting the CAP theorem. And then, we'll talk about operations and reliability. And the most important part

there becomes a fault-tolerant. What if something goes wrong? Does the application do have something to do

that? In which we'll talk about message queues and monitoring as well. And of course, we'll cover different examples

here to understand different concepts. And the entire section or this part will be taken by an amazing trainer, Akshay,

who is very good in explaining stuff. And I hope you will enjoy the entire part. And do share your thoughts in the

comment section. Don't keep it blank. You what you like, what you don't like. If you have any questions, do point it

there. And we'll try to answer them. And if you enjoy this type of videos from us, let me know in the comments as well.

That is our motivation. And I hope you're motivated. Let's get started. Akshay is all set.

>> [music] >> Hello everyone. Welcome back to Teleschool. My name is Akshay Agarwal.

And today, we are going to start the system design course. In this course, we are covering each and every topic which

includes in system design. So, let's start with understanding why do we need system design at all? In systems, we

have a lot of systems around us. If we talk about human body only, we have digestive system. And also, we have a

system. Along with this we have a lot of systems, but these kind of systems are pre-built. It means we have not created

them. We just discovered them with time and we got to know about various components which are required for us to

breathe or to digest our food. So, in these systems as they are not built by us, we can't actually change anything.

Maybe through surgery we can change some of the other component, but not much. But the kind of system we talk about in

IT industry are the systems which are built by human or engineers like us only.

So, in that there is a possibility that we can pick our systems based on our requirement and also based on our

requirement we can pick the components which are required in our system and we can basically improve our system based

on the request we are getting, based on the load our system is managing, based on the kind of traffic which we are

getting in our system. So, in system design if we go through the words we have two words here, right? One is

system and one is design. What is a system? If we go through the definition, system basically means there are a lot

of components. There can be more than one components in that system. So, let me write components. And those

components are working eventually to solve a particular use case or maybe multiple use cases, but the common

purpose of the application remains same. So, that means if we write system is equal to components plus common goal,

this sums up our definition of system. Now, when we talk about design, there are various components which we are

going to learn and there are various common goals based on the requirement analysis of different applications which

we are going to cover in this particular series. But in order to start, let's not dig too much into the technical details

in this video. Let's start from a very basic. Let's say we have a requirement of creating our own application or our

own bank. Let's say we have a bank named Alien Bank because we are in Telesco, why not? And in this bank, we are not

dealing about each and every operation. Currently, we have a single operation, which is we have a cash counter, and we

have a person sitting on a cash counter. Let's name it as cashier. Okay, so here we are ready with our cashier, who is on

our cash counter, and the requirement or the flow is pretty simple. For the flow, let's understand the concept of our

bank. So, we have multiple, let's say, customers are here, and each customer will come to our counter, either

withdraw or deposit some amount, get the receipt from the cashier, and then move forward. So, let's write it down as

customer flow cycle. So, in our customer flow cycle, there are four steps, basically. Customer first will visit the

counter, will come to the counter, then the customer will either withdraw or deposit some amount, take the receipt

from the cashier, and then the transaction is done. The purpose of our application is done. So, these are the

core functionality of our system. It is very basic, very easy. You might have seen this already in some of the banks.

So, we are building the same architecture right now. The architecture basically looks like this. Now, in order

to understand system design, what it really is, and how we are going to improve the working of our Alien Bank,

let's face some issues. We are going to face five issues, and based on the issues and solutions, we are going to

understand how system design looks like. So, here comes our first issue, which is the process is really slow. By slow,

what we meant is, let's say, our cashier takes 10 minutes per customer. So, that means we are serving only six customers

in 1-hour span. So, this is a huge issue, because if the number of customers will increase to a limit, and

also we have rush hour, in those cases, our system will break, and we will be not able to serve the customers rightly.

So, in order to improve this, let's first focus on our cashier. So, let's understand our system from the cashier's

point of view. So, basically, our cashier is taking time to understand the requirement. By requirement, I mean

whether the customer needs to deposit the amount or customer needs to withdraw the amount. They have to make this

interaction, which is time-consuming, right? Now, once the requirement is locked, whether the customer is going to

deposit the amount or withdraw the amount, the cashier manually have to count the cash. And then our cashier

will prepare a receipt, which will be again given to our customer. So, these are the time-consuming tasks for our

cashier. How can we improve the cashier time? So, in order to improve the working of our cashier, we have three

major issues. First is they are not able to understand the requirement. Next is they are counting the cash manually,

which is taking time. And again, they have to prepare the receipt. So, these are the issues which our cashier is

feeling, and we are trying to solve these issues. Maybe we can train our cashier in order to count the cash very

rapidly. Okay. Also, our cashier is also responsible for maintaining the receipt. So, we will improve the typing speed of

our cashier in order to make the receipt very fast. So, the second thing is train to type fast. So, with this, what we are

doing, we are not adding any resources to our cashier. Simply, we are making our cashier super fast, right? We are

improving the skills of our cashier, which is required for them to maintain this job. And let's say through this

training, the cashier is really improved now. He is or he or she is now taking, let's

say, 5 minutes in order to fulfill one customer's need. So, this is a huge improvement. By this, we are able to

improve the process by 50%. So, let's say our second issue is that the customers which our cashier used to

attend are increasing very fast, because this is our alien bank, and it is working very fastly, right? Growing very

fastly. So, now the issue two is our customers count is increasing. This is our issue two. In this case, what we

know so far? We know that our cashier we can't improve our cashier performance because it is now at its peak. So we

have to basically handle some or the other components to our cashier in order to make the work very fluently. So maybe

the desk on which our cashier is sitting, we will increase the size of desk. So next thing which we can do is

we will provide the cash counting machine to our cashier so that the manual cash counting time will get

reduced, right? So we'll provide cash counting machine, right? So with this also the time will

get reduced and our cashier will be able to attend more customers. One more thing we can do that is instead of coming to

counter and getting the requirement at the real time, we can give a form in advance to all customers so that they

can fill it and our interaction time will get reduced, right? So our cashier will now don't have to communicate each

and every request to our customers. Uh they can simply see the form and based on the form they will fill out the

thing. And now our cashier have cash counting device also, so the process will be fast. Let's say through these

processes, through enhancing our desk or cash counter, the timing got reduced from 5 minutes per customer to 3 minutes

per customer. This is also a huge impactful thing. So now our cash counter is capable of attending more customers

in lesser time. So that means even if the customer count is increasing, we will be able to process more customers

and enroll more customers into our Alien bank. So this issue is also kind of solved. Now let's move to the third

issue. The issue three is our customers are still waiting to get their application resolved, right? Currently,

what do we have? We have our cashier who is upgraded to its maximum level. So, we can't upgrade the cashier more. Also,

the desk on which it is sitting, the cash counter, is also upgraded to its maximum limit. So, we can't upgrade our

cash counter more. So, these two things are pretty solid right now. We can't improve it now. Now, the thing is, so if

we have 10 customers in place, that means our 10th customer will have to wait for 27 minutes in order to process

the three-minute work. So, that is a huge drawback of our system right now. And why that drawback is coming? Because

we have only single cash counter in our system. So, in order to make the process more feasible, now we have reached that

point that we have to add more cash counters into our system. So, let's add them. So, now we have our

cash counter one and cash counter two. Now, at this point, we have added one more counter, and things are pretty

solid right now. So, our customers will eventually have two counters to go to, and both of those counters will be able

to solve our issues. It feels good, right? Now, we have two counters to process. So, our customers will not have

to wait for longer time. They can either go to counter one, or they can go to counter two. But, let's say if first

customer is going to counter one, and the counter one is able to process the information, and they are maintaining

their own data. And similarly, our second counter is also maintaining its own data. And they are not synced. So,

this creates two issue, right? First issue is, if in our system, the counter two doesn't know anything about the

customer, that will create an issue because they have a valid account in our bank, and they should be able to proceed

their information through it. Now, the second issue is this only. That from the first account, they can also withdraw

the amount, and they will not update the second counter. So, that means they can again withdraw the amount from cash

counter two also. This will create an issue of data discrepancy. That means our data is not synced. So, the fourth

issue here is we need to have a centralized database or central data from which the cash counter one and cash

counter two will go and validate the data and go and retrieve the data. If the account is there for any customer

and the amount is also there, so the customer one will also insert the cash into the same account. And if the

counter two is getting withdrawal request, they will also get the amount out of the same account. So, in order to

fix this, we need to have a centralized data. That means our counter will look like this. This is our counter, let's

say two. This is our counter one. And they will eventually connect to a centralized database. And as it was

before, our customers are standing in a queue waiting for their turn to come in order to get the request. Now, after

solving all these case also, we have a new issue in hand. Which is, let's say the gate of our bank is here. This is

our alien bank. So, since the gate is here, most of our customers are going to the cash counter one only, where cash

counter two is sitting idle. Or maybe processing lesser request, far lesser request than the cash counter one. This

will again end up adding the wait time to our customers. And again, we will be end up handling lesser customers than

our potential. So, let's write that down. This is our issue five, which is underutilized counter. Now, in order to

solve this issue, we need to have a middleman. Which will stand apart from our counters. Let's say this is our

counter two. Again, this is our counter one. And here are our customers. And now our customers will not go to the counter

directly. They will first go to this middleman. Then the middleman will check how much load each counter is handling.

Suppose there are 10 customers at counter one and only five customers at counter two, then our middleman can

decide that next customer should go to the counter two. And the count will increase to six,

right? And the next customer will again go to counter two, and the next customer is again going to counter two only.

When the limit reach, let's say we have 11 customers here and 11 customers here. And if then any new customer arrive,

they will again ask man where to go. Let's say on some fine day our counter one is not working. That means our

middleman will know about this and will eventually send all the customers to counter two until the counter one is

getting fixed. So, let's end this example here at this particular point and see what have we learned. Through

this story, through this single story and five issues, I have tried to capture multiple concepts of system design. So,

the first concept which we discussed is designing a system, that is for sure. Ultimately, what design we do have here?

We have one cashier and the other cashier. Each cashier is having their own counter. So, this is our counter

two, maybe this is our counter one. We have a number of customers waiting for us. Also, we have a middleman now. With

that, we also have a common database, okay? So, this database is nothing but centralized database. The customers

which are coming to our bank is nothing but requests. And if they are standing in a queue, we can say they are creating

a traffic, okay? So, these are the traffic which we are going to handle. Now, all the requests or all the traffic

is coming through a load balancer, the middleman. So, this is nothing but our load balancer. So, from the load

balancers, all the requests are coming to either of the cash counter. So, we can say that our cash counter is nothing

but servers. And the person sitting there doing the work is nothing but our application code. Now, let's see the

system again through the system design point of view. So, the first issue which we have was our process is slow. So, in

order to solve that, we improved our code quality, right? So, first thing which we do in our systems also, we

improve our code quality. So, this is where our LLD will come and this is where our DSA will come. Suppose a

particular loop is taking certain time, we can reduce that loop or we can change that loop to some for each function or

some other way in order to process that information fast. So, while solving the issue two, we increase the size of our

table, we added some functionality, we added this cash counter. That means we upgraded our server. So, in order to

upgrade our server, this term is called vertical scaling. In vertical scaling, we don't add multiple servers, we

basically update our current server only. And again, the current server updation have a particular limit. After

that limit, you can't upgrade your system. So, then comes the issue third, which is our customers need to wait for

the single server to respond. So, in this case, we added more servers. So, in this scaling, we added more server that

is called horizontal scaling. So, after having multiple servers, we need to maintain the data at the common point.

So, that's why we added databases. And in order to add a common database, we basically can move to distributed

database architecture or can move to a centralized approach of handling that database. So, this is nothing but

centralized database. So, after having centralized database, again the issue comes where we don't know how to handle

multiple requests. That means our requests are again going to one counter only or one server only. Now that we

know they are going to one server only and the other server is underutilized. So, what we did, we added a middleman.

Our middleman is no one but our load balancer. Our load balancer basically do two things. They will check whether our

servers are healthy or not and based on that, they will distribute the request equally into multiple servers. So, these

are all the concepts which we discovered through this particular example. So, if we think about the applications we are

using in our day-to-day life, we are using WhatsApp, we are using maybe Snapchat, we are using Instagram, we are

using YouTube, we are using LinkedIn, we are using Netflix, Amazon, maybe Flipkart. All of the applications, if we

think about it, are showing us the data either in the form of images or in videos, in audios or in text. So, we can

say through a large lens, the data consist of these four kind of information. And all of the applications

which we are using in our day-to-day life consist of these data's only. So, we can say that in the core of every

system, we have data and we are then finding ways to make it available to the users in the way they are expecting it.

So, in the center, as we discussed, we have data. Data can be of various form. It can be image, video, audio or text.

Now, if we think about data, it is not like we are seeing the data once only. We have to see the data and also while

storing also, we need to store the large amount of data. Now, in order to store the data, we do have databases. So,

databases are basically our first component of system design. What is functionality of databases? Pretty

simple. We need to store our data and whenever we want to retrieve some data, we need to get the data from our

databases only. They are represented by this cylindrical figure. Databases can be of multiple ways. It can be

categorized into no sequel or sequel. Now, there are various kind of databases in both of these categories. In sequel

also, we have various databases from which we can select which database we are going to use. And similarly, in no

sequel also, we have various kinds of database from which we can pick which database we need for our application

right now. But, the core remains seems that we are saving our data in some or the other database. We will discuss

about sequel and no sequel databases in the coming videos. That's why I'm not going to deep into this at this moment.

Let's think about the scenario where you and every user on the planet knows the queries of database, knows how the

tables are structured if we are talking about sequel, or how the documents or nodes are connected if we are talking

about no sequel. If we talk about both of these scenarios, if user, the end user, already know the database query

languages and know how to deal with different kind of data, in that case, we might not need any application if you

think about it. We will simply go to our database, or the database will be accessible throughout every application.

Suppose we are seeing the LinkedIn, but on LinkedIn we will see an interface on which we will write our query, and the

data will be fetched to us. That will be an hectic task. But, if users know about that, and if users start exploring that,

that will be easier, right? In that case, we don't need any application layer in between us, and in that case,

we don't need any client-side application also between us. We simply will see a database, query it, and get

the results, right? In that case, what will happen? Each query have to be managed through the users only. And

these queries, when I talk about large systems, let's say we are talking about LinkedIn, there can be multiple tables

through which our data is curating and coming to us. Every user can't do it, right? That's why we have different

layer in front of our database, which is our application code. So, in application code, what we do is we hide the

complexity of dealing with databases, different tables, different documents, and we give the endpoints to our

application or to the world so that they can access the kind of information they are looking for. For that, we create

APIs or we can say we create endpoints. There can be multiple endpoints based on different features, based on different

operations. So, through endpoints, we communicate to our application, which will again communicate to our databases

to get the data we are trying to get. There can be various operations, like we can insert some data, we can update some

data, we can retrieve some data, we can delete some data based on our request. So, the thing becomes little easier to

us. Now, we don't have to deal with all the database tables, but now we have assigned endpoints through which we can

do the work which we want to do. And endpoints are pretty simple. So, in order to get some data, we have a

get endpoint. Let's say https, and here comes our Teleschool. So, this URL or this endpoint can be used to fetch all

the students which are there in our Teleschool environment. But then, what we are doing? By introducing application

code, we are moving our logic and data separately. That means we need some kind of connections between these two. In

which we can connect through IP addresses, we can connect through some HTTP protocols, or we can connect

through some TCP protocol. There are many ways of connecting the database from the application code and getting

the data which we want. So, in order to show the APIs or in order to interact with the APIs, we have to know what is

JSON and how we are going to send the data to our APIs and how we are going to fetch the data from our API.

In this level also, if we know how to handle JSON, if we know how to handle various operations through which

endpoint, it will make our life little easier than accessing the databases, right? So, in that case, we don't need a

front end if we know how to hit the endpoints through which we'll get the data. But, for our normal users, for

other users of any applications, this is also a tedious task. So, in order to make this easier, we have a different

layer between our system that is our client application. Client applications are nothing but the applications through

which we are interacting to the database, through which we are getting those databases. So, our client

applications, be it a mobile application, be it any web application, be it any ATM machine for that matter,

they are connected with the application code, and further, the data is being consumed by us. So, again, in order to

connect the application layer and the client code, we again need some kind of communication between them. We either

can go to the same HTTP or TCP or maybe IP. Now, we have our application ready in our hand and we can use it. So, if I

draw it like we used to draw it, this is represented by our client, either a desktop or a mobile device. Now, they

are sending queries to our application code, and our application code is further getting the data from database.

Now, for our client to communicate with our application code, we need to put our application code on any server. And now,

if we think about it, our application code is already exposing the APIs to our client. That means, it can also expose

the APIs, or any other application can also use our application codes through our APIs and do the work which is

needed. So, it can be connected to any other server also, which will contain their application code. And this will

communicate with our data through our application code. For every query which we are making, we are going to our

database, and from that we are getting the data which we are trying to get. So, if we think about it, most of the

communications will start from the client side, and they will come to our application code through which we will

get the data from the database. This is pretty much the easy application or the applications which we create on our

college projects look like. Either a to-do application or a simple chatting application will look like this. Now, in

the current scenario, what we are doing with this kind of architecture is every client is going to ask application code

for some or the other information, and to fetch the data, our application code needs to get the data from the database

only. We don't have any other option. In order to make the files accessible to our server faster, because the

database's calls are heavy, which will take a lot of time, let's say, if a database query is finding the columns

from 12 tables, it is going to take time, right? Now, in order to make the frequently used data accessible faster,

we have a different layer, which is called cache. When we are going to put any data into our cache, and when we are

supposed to remove the data. There are different strategies we are going to cover that also in the upcoming videos,

but for now, let's see, this is our cache, and all the frequently used data will be stored here. So, that we don't

have to call to our database, and they can be easily accessible through our cache only, which will make our process

slightly faster. So, this is our fourth component of our system, which is cache. Now, let's move forward. Let's say our

application is big, is having multiple servers. In that case, we may have some client or this client, either the mobile

application or the web view, and we are getting the data from multiple servers. Let's say this is server one or this is

server two. Now, when we have two servers or multiple servers, our client have to decide where to go. From where

do we fetch the information? Now, in order to take this control, we have a load balancer in the middle, which will

take care of each and every request and which will take care of each and every connections of the server. So, our load

balancers are responsible for handling two tasks. First, they will check whether the server is healthy or not.

And second one, they will divide the request for each servers equally. This is what our fifth component looks like.

Now, let's come to our next component, which is messaging queue. So, in order to understand message queue, let's first

understand that in order to process one operation, there can be multiple task into that operation, right? Let's say we

are ordering something. So, when we are ordering, we are going to do multiple tasks, right? We are first creating an

inventory update. Next is we are going to process the order also. We are going to talk to the vendor who is going to

take the delivery. So, we are going to handle delivery. And then, we are also trying to send you the information, the

notifications that your order is placed successfully. Don't worry about it. So, that means we are going to handle the

mail service or the SMS service also. Now, in order to handle multiple services like this, we can't wait for

every application to respond back. There are two kind of applications basically. First is, let's say this is an

application and this is another application. Let's say this is SMS application and this is our order

application or microservice for that matter. Now, whenever an order is placed, it is going to speak to the SMS

service or SMS application to send the message to the user that your order is placed successfully. Now, there can be

two ways of handling this. First way is, whenever the SMS is sent, they will notify the order service or the order

application back that we have sent your message done. Or there can be different way of handling it that from our order

service, we simply say that okay, this is our order information. Please send the message to this number and we are

done. We will not wait for the response. We will move forward with our application and with different orders.

So, the kind of requests in which we are waiting for any confirmation or any feedback from the other service are

called synchronous calls. And the kind of service in which we will simply put down the service and we are not waiting

for any confirmation, we will move forward to do our task. Those are called asynchronous services. Don't worry about

this if you don't understand it now. We are going to dissect each and every component in details in the further

coming videos. So, for asynchronous kind of services, what we can do, we can simply push our notification or push our

request to another service and move forward. So, in those kinds of services, we have a message queue or message

broker in middle. What it will do, we are having our order service here. It will put multiple request or can put a

request time to time into this message queue. And then all the requests which our order is processing through message

queue, they will not take care of it anymore. Our message queues will take care of it. They will connect with our

SMS service or SMS applications and send the request one by one. And they will take care of these all operations. So,

our message queue is like a broker who will take the request from one application, store inside it. They will

take care of these applications. They will take care of the retries or if any request is failing, they will take care

of it. Maybe they will communicate to our order application in the end or in the end of each day, we will send some

notification, some batch processes to our order application in order to show that which requests are fulfilled and

which are not fulfilled. So, in short, our messages queues are doing two work here. They are storing each and every

request which are coming from various other services or various other applications for that matter. They are

storing it and they are taking care of those requests. They are sending those requests finally to SMS application or

service and they are taking charge that okay, my first service or my first request is being processed. Nice. Now, I

will send the next request. Now, I will send the next request and further on. If it is failed, there are various

mechanisms how messages queues work and how they bisect and dissect each and every request and make sure that they

will provide the latest status of each request. And in those kinds of environment where we have multiple

microservices, in those cases message queues plays a vital role in between them. And with that, let's move to our

final component which is monitoring and logs. In monitoring and logs, what do we do? As we know now, we have different

components in our architecture, we have a client system here. We may have a mobile application again

which is connecting through an API to our load balancer which is again further connected to

either server one or server two. Our servers are the area in which we have our application code and then they

may be connected to single database or multiple databases and those databases can also communicate with each other in

order to maintain the data. And also, let's say our application is also using some third-party application for which

we are using message queues. Our message queues are denoted by this. Now, this is our architecture. We have so many

components. So, it is very important for us to monitor those components. To monitor

whether our load balancer is working finely or not. To manage whether our application code does have any error or

not. And if there is an error, we need to know how to solve it. We need to know the complete stack trace of that

particular error. We need to know how to reproduce those errors on our local system in order to fix them. And also,

while working with the applications, we know that for every each and now, we have multiple features which are being

attached to our applications. So, we also need to monitor while adding one more feature to our application, whether

any other feature is impacted or not, or it is failing or not. So, in order to do that, we have a lot of monitoring tools

and also logs to check our system. There is a complete testing department in our team also, who takes care of these

things. This is one of the vital role in our application. So, these are most of the common components which are used in

system design. Whenever we think about any application or designing any application, we suddenly jump to the

tools which are used in that application. We start creating diagrams that this is our client, this is our

database, this is how our data is going to flow, this is how we add cache maybe. So, these all things we start doing

without noticing a single thing, that is, how is our application going to work?

What is this application? What are the limitations of this application? In system design, there is a smart way of

doing that. First question you need to ask yourself before diving into the solutions is, what is this application?

What is the type of this application? Is it a data intensive application or a compute intensive application? With this

common differentiation, you will be able to save a lot of cost to your system design, as well as you will be able to

handle the task properly into your system. Let's say, we have two applications. In both of these

application, there are the same number of users. With that, the network on which both of the applications are

working are also pretty much same. Now, let's put a scenario that our system or the application which we are working on

is facing latency. Now, what is latency? Latency means our system is taking time to respond. There are various ways of

handling this issue, of fixing the latency issue. We can add multiple databases, we can add different caching

layers, we can update the caching mechanism, or we can update or the CPU powers. Now, on which vector

we are going to do this changes defines how well we are going to solve this issue. So, now there are two type of

applications as I discussed. One is data intensive application and one is compute intensive application. Let's understand

them one by one. The basic difference on which data intensive application is different from a compute intensive

application is that in data intensive application our core focus is on data. Now, for this data we need to have

multiple operations, right? We need to store the data, plus we need to gather the data, we need to move the data time

to time. That means we need to keep updating the data because new data is entering. And also, while new data is

entering we can also, right, we need to add a lot of data. We can take example of any application

which can be Instagram because on Instagram we see a lot of content and we post a lot of content. So, that is a

data intensive application where the primary focus is on gathering the data, is on showing the data, is on the data

you are posting. But, the focus is not on how the data will be calculated. The calculation part is pretty solid. We

just need to move the data from our databases to the client applications and we are sorted. So, in these kind of

systems the data mainly resides in the database. So, first issue can be the database response is low. Another is,

since we are using network calls here and there, that means our network calls are also bulky or we need to optimize

them. The third can be our server itself that our server configuration is pretty low, that's why we are not able to get

the data faster. So, all in all, in data-intensive applications, we are not blaming the client to upgrade their CPU,

we are blaming our circle, our database calls, our network calls, our server configuration in order to get the data

fast. Most of the applications in which you work on are data-intensive application. That's why we use terms

like cache enhancement, shredding, and through all of those things we will be able to solve this issue. Now, let me

also write some common examples of this. So, the common examples here are Instagram feed where we see a lot of

data as we discussed earlier. In WhatsApp messages also, there is a high amount of data because we are not only

taking care of one chat, we are taking care of entire chat. The count of chats or the count of messages WhatsApp deal

with is around 2 to 10 million messages per day. So, this is also a heavy data-intensive application. Now, there

are bank transactions, analytic dashboard, and log process system as well. In all these application, the

primary goal is to serve the data from database to our client system faster. There is not much calculation required

for them to show on the client machine. Now, based on the data-intensive application, what are the worries of a

data-intensive application? The first worry is how fast can we read the data? The second point is how safely are we

going to store the data? The third point, as we all know, a lot of users are using our application, so how many

users can use the same data simultaneously? The next and one of the most important worry of these kind of

application is what if any server dies? What if any server breaks? What if any network call breaks? What if any

database machine breaks down? So, in those cases, how we are going to handle the data? So, these are the most common

worries of the data intensive applications. And now, in order to solve them, we don't go to our client system.

We talk about the components on which these processes lies. And those are our databases, caching, replication, maybe

data sharding, or it can be solved through data consistency. So, these are all the components or the terms we use

in order to improve our data intensive application. Don't worry about these terms if you don't know them yet. We are

going to cover each and every component in details in the coming videos. So, I hope data intensive applications are

pretty clear up to now. But, for the sake of example, we are also going to take one more example. Now, think about

Instagram feed. We are not solving Instagram feed issue through any mathematical equations. We just have to

gather a lot of data and show you that data. So, what we don't do here, we don't do any complex maths. What we do

then? We normally fetch the posts. After fetching the post, we sort them through time or relevance. We show a lot of

images or videos. And according to those content, we normally do some operations, maybe like or commenting or sharing. So,

I'm writing that. So, this is all the operations loosely we have on the Instagram feed. Now, what is the common

challenge which Instagram is facing daily? That is we we have a lot of content. We have millions of users and

million of content which we have to handle. So, let me write that down. In other words, we can also say that

million of users are reading and writing at the same time. So, with this challenge, Instagram still needs to

scale, right? How does Instagram scale? If we think about this, Instagram adds multiple databases in order to store the

data. And also, Instagram uses aggressive caching in order to store or get the data fast. Now, we also know

that Instagram heavily depends on the images and the video content form. So, in order to handle them, we can't simply

store those kind of data into our databases directly. For that, in order to fetch them faster, we need to have

certain CDN in place. And in databases, we will only store the location of that particular file which we are going to

fetch through the CDN. Now, think about this. In order to fix the latency issue on Instagram or in order to help

Instagram scale, we hardly touch the CPU or we hardly touch any GPU. We only touch the data or the components which

are associated with them. Because the issue here is not the CPU, but the data movements between the components. Now,

this is all about data-intensive applications. Let's now focus on the compute-intensive applications. Now, the

compute-intensive application is exactly opposite of a data-intensive application. Here, the amount of data

which we are retrieving from databases are pretty much low, but the calculations we need to perform are

really heavy. So, instead of putting our focus on database caching, replication, sharding maybe, we move our focus from

there to CPU and GPUs. So, let me write down the challenge now. So, we can safely say that here our systems are

compute-bound or we can say CPU or GPU-bound. Let me take some examples of this. The most common example can be

image processing, video rendering. The machine learning model training can also be a good example of this. Also, we can

talk about simulation and cryptography. Now, in all these examples, the data which we are getting from the database

are in a smaller level only, but the kind of processing we need to do in order to show these data to user is

heavy. That's why they come under the category of compute intensive applications. Now, let's discuss about

the worries of a computer intensive application. The first worry is pretty simple, that is how fast can we do the

calculation or how fast can we compute? The second worry can be can we do something some processes in parallel in

order to show the data faster to the user. You may have seen a lot of systems in which in parallel the image is being

processed and you are able to move further with the application. Let the image process and the meanwhile you are

moving to the other features of the application. The next issue can be how can we reduce the computational cost?

And also as a system designer, we can also think about can we use GPU instead of CPU? So, these are the common worries

of a computer intensive application where the data is not much, but the processing is the king. Let me also take

a good example of a computer intensive application. So, in order to create a simulator machine, it is very important

for the simulator to work exactly how the actual machine works. So, think about a scenario that our country is

hiring a lot of pilots every year and we can't spend too much on the actual aircrafts when the pilot doesn't know

how to ride it or how to fly it. In order to train them, we need to give them simulators machines and on which

they train themself first and then sit on the actual plane. So, now in simulation machine, our common worries

or the task which we are going to have is can we do anything in parallel again? Can we use GPU instead of CPU? Again,

how can we reduce the cost of our simulator? Also, we need to find a way to gather a lot of good algorithms in

order to do the task frequently. Now, at this point I think we are very clear on how our data intensive applications are

different from computer intensive applications and why we need to put our focus on computation on different

algorithms on GPU or CPU in order to make a computer intensive application works because the data is very low and

in data intensive application if we start focusing on GPU and CPU there will not be any big changes because the issue

lies in the database network calls caching algorithms or maybe server. So in order to fix them we need to put our

effort into the right category into the right components so that our application works seamlessly. And why this is

important? Because if we put our efforts into the wrong components our components are not free they are going to cost us

money and if we put the money in the wrong place and not getting any help in order to solve our issue it is going to

be a bad system design for the entire game. Now when we talk about data intensive or computer intensive

applications we often think that these two are very separate applications. But in today's time we live in the

environment of multiple applications and each application performs multiple tasks. So that means our data

intensiveness or the computer intensiveness of the application can also depend on the feature we are

working in. For example we have YouTube right? YouTube is data intensive for serving the videos. There are a lot of

videos in YouTube environment and you get the videos and we can say that it is computer intensive when it comes to the

recommendation. Because in order to get the recommendations we have to get what you are watching right now. So I think

it is pretty sorted now that in order to fix a particular issue you need to first think whether the issue which we are

facing is computer intensive or data intensive and then you get into the components and fix that. There is a

trick to understand whether the feature is data intensive or compute intensive. That is, if the time is lost in data

movement, probably it is data intensive application. Or if the time is lost in the computation, it is probably a

compute intensive feature or the task. This one distinctions save you a lot of money, time, and complexity. In order to

discuss our requirements, we divide the requirements into two categories. One is functional requirement and the another

one is non-functional requirement. So, let's go to our screen and let's discuss this. Functional requirement says,

"Please define all the points which you are building, all the features which you are building, all the working which you

are building." That means, if we define every feature that comes under the functional requirement. So, let's

understand functional requirement by taking an example of any e-commerce application, let's say Amazon. So, what

can be the functional requirements of Amazon application? That means, we are defining each and every feature which we

see on Amazon. So, let's say on Amazon application, a user can register themself. User should also be able to

log in. So, let's write that. After login, user should be able to see the products. Also, user can search the

products or filter them. Let's write the searching and filtering into two parts. So, search the products and user

can also filter the products. After filtering, searching, user should be able to add any product to the cart.

After adding any product to the cart, user should be able to get some promotions or get some discounts, right?

So, they should be able to apply the coupon codes to get the discount. And once all this done, user should be able

to place the order. After placing the order, user should be able to do the payments, maybe. And once the item is

placed, once the order is placed, user should be able to track that order. So, all of these points are the functional

requirements of Amazon or any e-commerce application for that matter. In which we define all the functionalities, all the

actions which user can take, all the working of the application is defined under functional requirement. So, if we

have to write this, what is functional requirement again? Functional requirements means all the features

which you are going to have in your application are the functional requirements. Now, let's move forward to

the non-functional requirements. For non-functional requirements also, let's start with the example of Amazon only.

Let's say the first non-functional requirement is that on our Amazon site, we have

approximately 1 million users. So, that means our system should be able to handle 1 million daily active users.

Now, in order to give our users a good experience, we can say that the response time should be less. But, in

non-functional requirements, we have to define how less. So, let's say for Amazon application,

the response time should be less than 200 milliseconds for each request. Now, our application is built and is running,

that means our application can't handle failure. But, that is not possible, right? So, we should define up to how

much is our availability of our system. That means for how much time our system will be available. That are defined

under SLAs and SLOs. So, we have to define them. That for 1 year, our system downtime should be this much amount

only. And for 99.9% time, our system should be up. That is defined under availability. After

availability, we don't want our data to be flowing easily, so we want to encrypt our data. And let's say on any sale day,

let's say Black Friday sale or Diwali sale or Holi sale, in those times our normal users can go up to 10x. So, our

system should be able to handle that much traffic also. So, let's write that down. With this, our system should be

also able to handle the faults. Because there are going to be faults in the system. No application is perfect so

far. Every system have their own faults, have their own downtime. But we need to see how can we track them and how can we

fix them as fast as possible. So, that is we need to make our system fault tolerant. So, with this example of

Amazon, we get to know that our functional requirement are the features of the system that we can say what our

system is going to do. And with non-functional requirements, we define how our system is going to perform. So,

these are the specific non-functional requirements for Amazon. If we have to write it in a generic way, there are

some generic terms which we can write. So, we can talk about scalability. Let's say our current configuration of server

is reaching to a point where it is not able to handle the kind of traffic it is getting. So, now in order to scale, will

you scale it horizontally or vertically? That will be taken care under non-functional requirements. Apart from

that, as we have discussed, availability of every system will be defined under the non-functional. As we have already

discussed, that means our system uptime will be calculated here. Which is defined under SLAs and SLOs.

So, in these agreements, it is already defined that, let's say we are using Amazon server in order to deploy our

application, then Amazon will give us some contract that we will be down up to this time. And apart from that, you will

not see any issues. Now, in those cases, if we see any issues, let's say for a day, Amazon says that we will be down

maybe for 5 minutes maximum. But then, if Amazon fails to keep that promise or that agreement, the companies which are

using the Amazon service can sue them. Because the Amazon services have given us the SLA's in writing that the

services will not be down after certain point. So, availability plays a huge concern there. Next, we have

reliability. In reliability, we can say that our system should be reliable. That means there should be no data loss in

our system. After reliability, comes the performance. Under performance, we are going to define how our system is going

to work, how much latency can we feel in the system. What is our P95? What is our P90? What is our P99? These all things

are defined under performance. So, all the words which I have said in this particular video, we are going to cover

each and every topic in details in the coming videos. So, don't worry about this if you don't get it now. After

performance, we can jump to security. In security, we can have features like authentication, authorization, and

encryption. With security, we are also dealing with a lot of code. That means we can say our code should be in perfect

modules so that each code or each chunk of code can use different APIs or different modules in order to work in

efficient way. After this, we have last but not the least requirement, that is observability. Even if our code is

pushed to production, there are a lot of use cases which are handled after that. Because we don't know the exact behavior

of users, that how they are going to use it. So, we need to keep track of all the monitoring tools, all the tools which

are going to track the user behavior on our application. So that we can improve our system and also we will be able to

track if there are any bugs and solve them. So, let me write monitoring and logging. So, this is all about

functional and non-functional requirements. I hope you all get it. But in order to completely get it, we all go

through a lot of applications on a day-to-day basis, right? We may be scrolling through Instagram, maybe

watching YouTube, maybe watching Netflix, or Amazon Prime, or purchasing anything from Flipkart, Amazon, or any

other application e-commerce platform, or we may be scrolling through our WhatsApp messages. Now, I want you,

while using your application, just sit back for a few minutes and think about all the functional features which you

are working, or all the functional requirements which you are using, and all the non-functional requirements

which the app must have configured or thought of while creating that applications. Also, comment down the

application on which you worked on, and write down three to four functional and non-functional requirements. That will

help you and me also in order to understand that we are learning on a similar pace.

How does our browser know which webpage to hit? Do they know by domain name, or do they get these things through IP

addresses? Let's find out. So, whenever on our browser or on our machine, whenever we

write any address, let's say telesco.com, it is going to redirect us to the telesco page, which is our

homepage. Maybe some banners here, maybe some courses section here. It is like this only. Now, how do our

browser figure out which page to show? Because, as humans, we communicate through names, but systems don't

communicate through names. In order to establish the communication between one system to another system, we need IP

addresses. So, that means there should be some place in between or some system in between our browser or our OS through

which we are going to get the correct IP addresses of each and every site. So, we will send telesco.com, and in return we

will get some IP addresses. So, this can be the IP address of telesco. I'm not exactly sure about the IP address, but

yeah. Like this, our DNS works. So, DNS basically stands for domain name system. Domain means any specific URL or

any specific domain. Domain is this only, telesco.com. There can be multiple domains, telesco.net, google.com,

facebook.com, whatsapp.com, .web I think. But yeah, you get the point. These all things are domain. And what is

subdomain? If I write any subdomain, let's say docs.telesco.com or any other subdomain, let's say

course. So, they both are subdomains of telesco.com. Where telesco.com acts as an umbrella and under which we can

create multiple subdomains if and when required. So, these are the domain and subdomains. But mind it, the machines,

the computers doesn't communicate with each other through these domain names. So, they need IP addresses. Let's figure

out ways on which our system can handle the IP addresses. Let's say we are building a browser. Why do we need DNS?

Why do we need another system for maintaining the domains? We can basically install or store all the

domain name list on our browser only. Now, is this even possible? Let's say if we have only two or three websites,

maybe 10 websites, we can store their names, even 100 also. But the current number of domains which exist in our

world is more than 350 million. And as a browser, browser doesn't know which sites I'm going to use, whether I'm

going to use Stack Overflow one day and Gmail the other. So, we can't store all the 350 plus domains into our browser.

That will crash our system. So, two reasons because of which we can't store the entire domain list on our browser

are first thing is it will get heavier. And to store such amount of information, we need a high storage browser. So, that

will also impact the performance of the system. Now, the other point is what if the IP changes after some time? What if,

let's say, I am running any site, teleschool.com, I have purchased my domain from maybe GoDaddy, and tomorrow

I decide to discontinue my service from GoDaddy, and I'll move to Google Domains. So, I'll purchase or renew the

domain from there. In that case, my IP can be changed. And if it restores, I need to reassure that in all the

browsers where teleschool.com is running, I need to update them. I need to send them some update. So, through

this we can estimate that it can't be stored in one browser. And also, it can't be stored in a single server,

also, because it will also get heavier. There are more than 350 million domains. That means, if we are going to store it

in a tabular manner, so the here will be the domain name, and the corresponding IP addresses accordingly. So, here we

will have any domain name such as google.com, and here we will have the IP address, 1.2.3.4,

for example. So, by this, we will create the entire list. This is not possible for a single

machine to handle, because if that machine breaks, then the entire system, the entire internet will break. Anyone

in the world will not be able to use any internet services or any web pages for that matter. No APIs will hit, because

it all depends on the DNS. So, one thing is sure, we need a certain kind of system in order to fulfill this

requirement. That's why we have one of the most beautifully designed system, DNS. Let's start from the high level.

So, this is our machine, which is going to use some internet service provider, and under this service provider, we have

something called as DNS resolver. So, this DNS resolver is responsible for us to getting the correct IP of any

websites or any domain which we are trying to process. In order to understand this clearly, let me take

this and put it in a side, so that I can draw more things onto our screen. So, the first call it is going to make is to

our root server. Now, there are 13 root servers in this entire world. That means there

are 13 companies who own 13 web servers in this entire world. They are not physically 13, but they are categorized

into 13. So, that means out of these 13, we are going to hit one of them and get the response. These root servers are

typically named from A to M like this. Again, from this A to M, they are owned from various companies. So, you can see

the complete list of the root servers with the companies which own that root servers on the screen. You don't have to

remember all of them. You just have to see and understand that these are the service providers of the root servers

from which we are going to get some information in order to process our domain. Now, let's come back to screen.

As discussed, we were searching for telesco.com. Also, I can remove this now. Now, when we are searching for

telesco.com, the root server doesn't know anything about telesco.com, but it knows about the TLDs which are top level

domains. What is top level domain? If I talk about .com, this is our top level domain. .com,

.net, .gov, .in, .uk, .us, .education maybe .edu, we use them. These all domains are the top level domains of any

website. So, that means if we need to find out where telesco.com is, we first need to know which server is handling

all the .coms. So, in order to do that, our root server will give the response that okay, you can go to one of the TLDs

which is handling the top level domain for .com and the root server will return the IP address of that. Maybe this is

the IP address and from now, we know that we are going to hit any TLD, top-level domain for a .com. Now, from

this TLD, we are again not going to get the complete IP address of the site. We are going to get the proper server or

the name server or the authoritative servers which are managing the all the IP addresses for our domains. Let's say

this is the IP address of our authoritative name server. And from this, we are going to make another call

which are our authoritative name servers. And from here, we are going to actually get where our telesco.com

resides, the IP address, which was 19283841. And once this is captured through our

browser, the IP, then it is going to make the call to telesco.com server and it is going to get the web page which is

desired for. I get it this can be a little confusing at start, so let me reiterate it for once.

So, whenever we click on any site, whenever we want to find any site, let it be telesco.com, it is first going to

our service provider, internet service provider. Also, you can see uh it can go through router and router can

also have DNS resolver. So, from internet service provider which have DNS resolver, this component is going to

resolve all our queries for us and ultimately will give us a perfectly fine working IP address for telesco.com or

any other site for that matter. So, the first step is it will go to the root servers. What is root servers? There are

13 companies who own these root servers who doesn't know the IP address of your site directly, but knows which TLD to

follow. So, from root server, it is going to check what is your TLD, top-level domain. And in this case,

it is .com. Now, as it is dot com it is going to get the IP address of the TLD which is handling the dot com domain. So

after this first call now we are going to call the TLD which we get from the root server and from this also we are

not going to get the complete IP address of telesco. We are going to get on which name server we are going to hit to get

the complete IP address of telesco.com. So from here we are going to get the authoritative name servers IP and

accordingly we are going to hit our third and final call to our authoritative name servers. Our

authoritative name servers are mostly can be configured in GoDaddy or Hostinger or any other service providers

in your current application. And from these authoritative name servers we are going to get the IP address of

telesco.com. Now in order to click one side we have to do these three operations. This can be heavy, right? So

this is only done first time and the next time when you click on anything it basically takes the value from cache.

Now cache can be stored in three locations here or more than three if you think about it, but three are definite

locations. The first is the DNS resolver can have the cache to have all the common IP addresses which we are going

to take or the all the common domains which we are trying to hit. It can store it here. Also on our system we have two

things. One is OS and one is browser. In both of these areas we can basically add that domain and the IP address in order

to get the details fast. So browser also maintain that cache OS level also we can basically maintain the cache and on our

DNS resolver site also a cache is maintained in order to get the details fast. Now when we are at this topic let

me tell you about one more thing, that whenever we create domain name systems like this dogs or subdomains, these all

entirely creates a zone in the authoritative name server. That means telesco.com including all the

subdomains. So, one zone can contain multiple domain list, which will include only the domains specifically to that

particular main domain. Okay? And through this it is easier to get to the main domain. So, in this case, whenever

we are hitting the root server, we are getting the IP of TLD, again hitting the TLD, getting the IP of authoritative

name server. And in authoritative name servers, we can have different zones attached to different domains.

And in this zone, if it is telesco zone, in that case, all the domains which are associated with telesco.com will be

here. So, if instead of telesco.com, if we are hitting courses.telesco.com, and from this, the authoritative name

server will get that this is our telesco zone. And in telesco zone, we will get the courses.telesco.com

IP address also. So, yeah, this is it for DNS for now. I hope you understand it. If you get it, let me know in the

comments. If you have any queries also, let me know in the comments. Have you ever wondered, if you see Zomato, if you

see Uber, if you see Ola, if you see Rapido, and all these applications, also Swiggy, they all have an inbuilt map.

But, did they create their own map in their applications, or do they use the existing Google Maps in order to do

that? Now, that is possible only through APIs. So, Google Maps present some APIs to other applications which they can

use. Now, in order to understand APIs in detail, let's dig into this video and go to our screen. If we think about API,

the API basically stands for application programming interface. Now, what does this mean? Suppose this is our

application. Any application will work. Let's say this is a movie application.

In which we have all the movies, plus we have all the ratings, all the reviews of the movie. Like IMDb or something, okay?

Like Rotten Tomatoes or something. So, we are already capturing all the details which includes a movie like latest

movie, latest hits of 2000, 2010, 2020, everything. Now, we already have this amount of data which is stored in our

database and our application is already working on that. Let's say now in any other corner of the world, someone is

trying to create one application based on the data which is already captured in this app. Now, they have two options.

Either they can create their own engine like this, their own application where they will again curate all the ratings,

curate all the reviews, and curate all the data. But now, where are they going to curate this from?

For this, every application leaves some end points which can be used by another application. Now, if

for our application or for this developer application, if they want to use rating of some movie application,

they can directly call the APIs of this application and through which they will get the proper information which is

required for them. Now, like this, this developer or this application don't have to create their own application for the

task which has already been done. So, they can use the application data directly. But again, can't we directly

give access to our database? This can be dangerous, right? Because then this application again have to handle a lot

of scenarios based on the database queries, plus they can also delete some important data in this database which

will affect the other applications which are using the same content. Also, there are some added functionalities. Let's

say we are giving an option to write a review again. So, in this we have already handled that case through some

or the other API. So, instead of creating a new endpoint in this application, we can directly access the

endpoint of the other application, which is going to take our review also. So, in this case, this application saves

a lot of time because they are using another application, the APIs of another application through which they are going

to do the same work. Now, in this case, if this application wants to add some data into their database also, they can

easily do this. So, this is one use case of creating an API or using APIs of other application. Also, there can be a

case where you're creating an application and in the backend you are exposing the APIs which can be used by

your front end. So, front end can be a web application or even a mobile application. Can be both also. Let's say

we are creating some backend and from for that application we are also creating front end maybe using React or

something and backend using Swift or something. In both of these cases, these languages, the React and Swift, both are

capable of handling APIs. So, they don't care about which language did the backend use in order to provide them the

API. They will directly call the APIs and use the information which they are trying to use. So, this is another

scenario where there are no two applications, there is only single application which is divided into front

end and backend part and using the data which backend is providing. Again, in this case also, you can think that they

will use the database directly, but it is restricted and prohibited for the security of data which we have already

discussed in the first case. So, with this we understand the core functionality or usage of the APIs.

Number one is if you want to give access to some other application, and number two is if you want to give access of the

endpoints or the APIs to our front end. Now, this both can also happen at the same time because it is up to us how do

we want to expose our APIs. Some applications or some backend basically restrict the APIs to be used through

their front end applications only. And if you want to make them open, you can make them open so that other application

can also use the same API. Now, let's move forward to the type of APIs. The first type of API is called as REST API.

If you are a developer, you already are using this. Also, if you are creating projects, you might already have created

them or use them in your application. Now, REST basically stands for representational state transfer. REST is

one of those APIs which you will see in most of the application nowadays because of its easiness, its structure, and its

ability to easily maintain a large amount of data with low volume. So, that's why we use REST. Mostly, in

REST we use JSON, which is JavaScript Object Notation, which is a way or a format through which

we will send or retrieve the data. It is a easy to use format, that's why we are sticking with this for this today's

video, and we are going to cover this in details in a minute. Also, we have some other types, that is SOAP. SOAP

basically stands for Simple Object Access Protocol. This is also one way of handling the data across applications,

across different platforms, and SOAP basically use XML formats in order to transfer the data.

XMLs are replaced by JSON because XMLs were really bulky in nature because in order to create every object or every

array, we have to mention the attributes again and again, which is solved by JSON. So, that's why we prefer JSON over

XML any day in today's time. But, the SOAP or the XML was famous back then. So, that means there are a lot of

applications which are still using XMLs. Why? Because there are legacy applications. They are built way earlier

when JSON was introduced and still are using XML. So, in legacy system, if the APIs are created and exposed using XML,

that means in order to make our system work, we also need to know about XMLs so that we

can integrate those APIs into our system. Also, you can use some middleware application where the data

will be taken in the form of XML and will be converted into JSON and your application can use that data

afterwards. So, that's why if you are working with a legacy system, it is preferred to understand the XMLs also so

that you can convert them in JSON or you can work them without converting them as well. So, after SOAP, let's jump to the

third one, which is GraphQL. So, GraphQL is also used widely across many applications, many platforms. So, the

basic thing in GraphQL is GraphQL basically says that I'll provide a single endpoint, only a single endpoint,

and based on that, you can send whichever query you want. So, in FE, from FE side, we will not hit multiple

endpoints, but we will send a request as a query. That's why the GraphQL stands for graph query language. So, GraphQL

also have a query language that is similar to SQL. In SQL also, we have multiple queries from which we can do

our things. In this also, only one point is exposed as I told earlier and using that only

the front end now can send multiple queries and access that particular endpoint and get the data which they

want. So, this is the core concept of GraphQL. So, as I said, GraphQL is also present in our today's time also. So,

there can be a chance that you learning GraphQL and working on it. The other type of API is gRPC. Here, the RPC

stands for remote procedural calls and for G some may say that G stands for Google because it is created by Google,

but it is gRCP remote procedural calls only. So, instead of using XML and JSON, we have a

new format here which is called protocol buffers. It is very much efficient than JSON and XML combined. So, that means

the size of protocol buffers are usually very small than XML or JSON. So, that will enable the data to transfer faster.

So, because of this only Google created this and uses this. So, this is small in size and faster to

transfer. In today's applications where we have microservices, right? So, in microservices, this is our entire

application and let's say our applications is divided into multiple applications which is combining the data

of their endpoints through each other. Let's say these are the connections and through this they are going to provide

the actual API to the outer world. Now, for this communication between these microservices, we often use gRPC. Why?

Because internally at least the communication should be faster to provide the relevant information in a

low latency. This will also help us having multiple microservices and having various kind of communications between

them before giving the actual data to the other application or the front end. Now, let's jump to the last one in our

list today, which is WebSockets. WebSockets are also used in today's time and today's application for various

purposes. For every notifications, if we see, WebSockets are used. In order to maintain a chatting stream, WebSockets

are used. Wherever you need some real-time connections, WebSockets are used. So, its architecture is slightly

different from the other services. In other services, front end or the other application will place some request to

the application and then they will provide the response. But, in WebSockets, whenever front end creates a

request, a channel is created. Now, once the channel is established, then the conversation can be lead through back

end also. Whereas, in other forms, only front end or the other application, which is trying to access the

information, they send the request and get the request. But, in this case, since the back end also can initiate the

conversation, that means it can also send some or the other information by itself. Those information can be the

chat or the notifications, which is very helpful for front end to get. Because, if this channel is not there, the front

end have to constantly ask the back end whether there are some new notifications or not or whether the chat is there or

not. But, once the channel is maintained through WebSockets, it is easy to navigate message from back end to front

end and front end to back end as well. With this also, if you have played quizzes on our Teleschool platform only,

Teleschool channel, in which you may see an application which Naveen sir presents in order to share the quiz with you all,

you use that application in order to give the responses. Now, those responses or the marks are calculated based on the

response time and the correctness of the answer. Now, those connections are also maintained through WebSockets. That's

why you see many notifications or some GIF or something something. So, this is about web socket and with this I would

like to conclude this session because it is going to be a long session and it is better to understand these things in

parts. So, these are the most used five type of APIs which we have. Now, in order to understand them briefly in the

next video, I'm going to talk about RESTful APIs in which we will discuss the various aspects or the parts of the

APIs and also see what are the recommendations and usage of those APIs.

Now that we already know about RESTful APIs and all the other different type of APIs, it's time to deep dive into one of

them. And for today's video, I have selected REST APIs to deep dive into. Why? Because it is the most used APIs

across the world. It doesn't matter if you are starting a career or you are already seasoned, you have to work with

RESTful APIs for sure. So, let's move to the screen now. RESTful APIs are also known as REST APIs. As we know, it

stands for Representational State Transfer. What does it mean? It means that REST is also one format through

which we will transfer our data from one end to another end. What are these ends? We have discussed this already. Either

we are giving some data from front end to back end or vice versa. It is also possible that one application needs to

get some data from some other application. So, in that case also, APIs are used and REST specifically is used

to transfer some data from one end to another end. So, basically, it is a way of how our applications are going to

communicate with each other. Now, before starting, let me tell you that in REST, we normally use JSON. JSON is a standard

way of sharing the data. In JSON, we can have array, we can have object and with these two combinations,

we can share our data in the form of key and value. So, let me create a simple JSON. Let's say this is a start of an

object and here we have a key, let's say name. For value, I'm writing my name. With comma, we separate different

values. So, after comma, I have to write some other key. So, let it be age. And with age, I can write any numeric value

as well. So, as we can see, this is string value and this is a numeric value. There are more values who are

accepted as a form of value, which is We have already seen string and then number. We can also use boolean, even a

nested object, arrays, and a null. Null can also be used because if we don't have any value for that particular field

directly, it is easy to say it is a null value. And the application which is using that value can take care of it.

So, this is all about the JSON which we need to know. We need to know that it is in the form of key-value pair and what

are the values which we can use. Now, let's move forward to the next part, which is endpoint. In order to

understand endpoint, we have to get two things. One is method and the other thing is path. So, in order to access

any data, we have to have some URL on which we will hit with that particular method and based on the method selection

and the path, we are going to either get or pass some data to our server. Let me start this with an example. Suppose we

are using HTTP as uh any site, my site .com. So, this is our main site and after that, we can put a slash and use a

different URL. Let's say we are separating all the APIs in the API subdomain. So, APIs is there and then we

can also segregate the APIs using version. So, this is version one and here we are going to write resource.

Now, resource can be anything. If I talk about any e-commerce application, it can be a product, it can be coupon, can be

user, can be anything, basically. A resource is an entity through which we are navigating and getting and passing

the data to the database. So, this is the URL. The URL is called the path through which we are going to either get

or pass some data. And with this, we need to have a method. So, for method, we can have a get basically. So, get

method with this path combinedly creates an endpoint and through which now we can get some data. We expect this path to

retrieve some data because we are using get. Now, there can be multiple ways of creating this URL. So, normally, we hide

the domain part of it. We don't use them in the explanations. We only use the resource part. Also, we can remove the

version also. With that, API can also be removed in order to understand the resource part directly.

So, from now on, if I create any path, let's say users, it is expected to have all this domain name API versioning if

it is there beforehand. So, I'm not writing that again and again. Cool. So, with that, if I write users here and use

get method, now I'm expecting some response from our server. I'm expecting either a list or a response from our

server. We'll come back to responses in a while, but this is how we can get some user information from our server. Now,

let's move to the second part. In this, we don't want all the users, we want a specific users. So, to find a specific

user, we have to provide them an ID. Normally, we provide them in the URL itself. So, it is get users and here we

can include the ID. So, in real time, it can be used as users/1. So, if we write one, the user with the

ID one will be retrieved. Now, with this, we can also add any user. So, the third part is

adding a user. So, again, the URL will be same users and now we are adding the user, so we are using the method post

here. So, in this with post, we are going to give some body, and in body, we will have some JSON which will store the

user information which we are trying to create. And based on our request, our user will be created in our database.

Now, once we have all the user details in our table, we may choose to update any user information, correct? So, for

that, we use another method which is put. Again, the URL will be same, users. And with this users, we will send the ID

of the user of which information we are trying to update. So, with put, we also have another method which is patch,

which can also be used in a similar way to update the user information. Now, what is the difference between put and

patch? It is simply that if we use put, our entire user object will be replaced with the new data. So, this is going to

replace the user data. And if we don't want to update all the fields, we want to update a specific field or particular

field, suppose username, name, age, or anything, so we can use patch. It can be used to partially update the user

record. Let's understand this with an example. Let's say we have any user information which have ID and have a

name and have a username, maybe age. This is all. So, these are all the values which are

user currently have. Let's we put some values. So, if our user table or user entity have these all four values, if we

use put to update a particular change only, let's say I'm going to change my username from Akshay to AK something.

So, now this change will only be done if we again send the entire body into the put request. If we only send username

like this, then all the other values of our entity will be updated with the default value. So, if we send this kind

of data in a put request, it is going to replace our entire entity. So, that means the name, the age will be replaced

by the default or null value. So, it is not recommended to use put entirely if you want to update your entity

partially. In those cases, always try to use patch. In patch, we're going to again update the particular information.

So, this is perfectly valid for a patch request. So, with this, let's move to the next part. The next kind of method

which we use is delete. In delete also, normally we pass the ID like this. So, if we write users and with {slash} we

write one, so that means the user with the ID one will be deleted. Now, this is the usual stuff. You will get this

information anywhere. Now, let me talk about something which is nested data. Let's understand nested data with an

example. Suppose you're creating a blog site. So, in blog site, we normally have our users who are going to read our

blogs. Then, we have blog, the actual content on the site. And on blog, the user can comment. So, we have clearly

three entities: users, blogs, and then comments. So, these three entities are interrelated. So, a user is going to