HTTP Protocol Fundamentals: Statelessness, Methods, CORS, Caching, and Status Codes for Backend Developers

backend is huge and if we start discussing every single component that could be part of it we will be stuck

here for years so what we will do is we will only discuss those topics which are used in majority of the code bases let's

say 90% of them with that in mind let's talk about HTTP protocol the medium through which our browsers talk to our

servers either to send data or to receive data from it and as I said there are a lot of other ways and protocols

which clients and servers use to communicate with each other and HTTP being one of the most used ones we will

focus on that now there are two ideas which are at the heart of HTTP protocol the first one being tessness what does

it mean tessness basically means it has no memory of past interactions so each HTTP request carries all the necessary

information for the server to process it such as headers or URLs and methods which we will see in a bit and after the

server responds it forgets about the request if a client makes another request the server treats it as a new

and unrelated event and it also means self-contained requests since the server does not remember pass requests each

request must include all the necessary data such as authentication tokens or session information to handle that

specific interaction for example in the case of accessing a user profile the client has to provide credentials like

cookies or tokens on every request for for the server to know which user is requesting the data so what are the

benefits of that what are the benefits of this stateless model the obvious one being Simplicity stateless design

simplifies server architecture because the server does not need to store session information which would

otherwise require additional resources and complexity and it also means scalability stateless protocol makes it

easy to distribute request across multiple servers because no single server need needs to keep track of a

session and if the server crashes it does not affect the state of a client interaction as there is no session or

memory of a request that needs to be restored and having said that because HTTP is stateless developers often

Implement State management techniques like cookies or sessions or tokens to maintain continuity in interactions

where needed like user logins or shopping cards and we'll explore those soon in this series and the second idea

is client server model in a typical HTTP request flow there is a client and there is a server always the client typically

is a web browser or an application which initiates the communication by sending a request to the server the client is

responsible for for providing all information needed by the server such as the URL of the resource or the headers

and everything and then there is a server which hosts resources like websites or

apis or other content and waits for incoming requests from the clients when the server receives a request it

processes it and sends back the appropriate response such as a web page or data or error message or a Json file

or a text file or any other kind of content now the thing to remember here is HTTP protocol states that

communication is always initiated by the client to get some kind of resp response by the server and throughout our

discussion we can go ahead and safely assume that HTTP and https are interchangeable because https to

oversimplify it just a more secure version of HTTP but the underlying principles are the same with more

security features like encryption and security certificates or TLS stuff like that which boarders too much into the

network engineering domain also to send some kind of request or receive some kind of response first the client and

the server need to establish some kind of connection mechanism right otherwise how are they going to communicate what

is the medium of the communication and for that sgtp uses TCP TCP which is a protocol which is a

transmission protocol essentially HTTP does not require the underlying transport protocol to be connection

based it only requires it to be reliable and not lose message messages at minimum presenting an error in such cases and

among the two most common transport protocols on the internet which are TCP and UDP TCP is considered to be more

reliable HTTP therefore relies on the TCP standard which is connection based now this is called an OSI model which is

often referred when we are talking about sending and receiving data over a network right and we as backend

Engineers often deal with this layer the top layer the layer 7even application layer and so a lot of discussions for

example the recent one which we talked about the TCP handshake and establishing connections and TLS encryptions they

often border into these layers so they are mostly network engineering concepts of course it's it's

good to know about those but if we start exploring those it'll be a rabbit hole and we'll have to cover a lot of

Concepts right so what we'll do is we'll focus on the application layer with a brief about you know what happens in the

netor layer like as I said HTTP uses TCP and TCP uses something like a 3-way handshake which if you're curious about

you can just look up and study more on that now throughout the years we had different versions of HTTP which kept

redefining how clients and servers send and receive data right in sttp 1.0 each request opened a new connection this led

to inefficiencies since a connection had to be established and closed for every request and response which slowed

performance in HTTP 1.1 they introduced something called as persistent connections allowing multiple request

and responses over the same connection over the same TCP connection that was established before sending the request

which significantly improved performance it also added stuff like chunk transfer encoding and better caching mechanisms

in SCP 2.0 they introduced multiplexing allowing multiple requests or responses over a single collection it uses

something called binary framing instead of text and supports header compression with Edge pack or and server push

allowing servers to send resources before the client requests them now in sttp 3.0 built on quick protocol a

transport layer protocol over UDP right instead of TCP it is designed over UDP which improved performance with faster

connection establishment reduce latency and better handling of packet loss it also continues to support multiplexing

without head ofline blocking which is still an issue in HTTP 2.0 now again that's a rabbit hole if you get too much

into the network stuff all we need to remember is that from all these discussions that client and servers

established some kind of network connection and messages are sent and received that's all you need to remember

for now now that we mentioned message let's look at what HTTP messages look like request messages or response

messages a request message is basically the one that is sent by the client and a response message is the one that is

received by the client from the server so this is what a request message looks like in HTTP and this is what a response

message looks like I intentionally took more complex messages which had more parameters and so that we can cover

different different components one by one so on a high level I'll just explain what is what in this right this part

this is called a request method this is the resource URL the one we are requesting from the server this is the

HTTP version which says we are using HTTP version 1.1 which is the currently the most used one this is the host right

which is our domain the front ends domain and these are this all of these things these are

called headers right we will talk about headers soon there will be a blank line here after all the headers to signify

that the headers are over and body starts this is called a request body some information the client wants to

send to a server right and in the response message again we have the HTTP version have the status code and the

value of the status code which basically means the status code 200 means okay right and we have again some response

headers after that there is a blank line and we have the response body so let's talk about HTTP headers

first this is very important since it's the major part of the request here and the responses on a high level what we

can see is headers are basically key value pairs key value pairs of different different parameters that is sent over

request or received over a response that is the first definition and the next question is why do we need headers why

not send the all these values in the you know the URL or we have this request body here why send why create a

different section just for sending more information or more met data about the request or the response why create

another level of abstraction and in order to understand that let's take a real life example we send Parcels right

ciers and we receive them the address of about the phone number of the recipient and different details right the the

state the the PIN code and all of these things do we keep it inside the package or do we write it on top and we write it

on top why because the one who is taking the parcel from the sender and to the receiver they need to know different

informations in order to successfully transmit the package through different modes of transmission

and if we kept all those information inside the parcel they have to open that right and which does not make any sense

but just for the sake of the example they have to open that and they have to see who is the recipient again and again

so let's say another person wants to see the address so again they have to open that so keeping the metadata the address

and the information of the recipient on top of the parel uh gives a quick way of checking different metadata about the

package sdtp headers can be thought of something like that but they have more uses before we go into the use cases

let's see what are some different kinds of headers that we see so as you can see there are a lot of different types of

headers so can we categorize them can we categorize them and different types of headers so let's see the first category

can be request headers which are sent by the client to the server to provide some kind of information about the request

itself it could be user agent which identifies what kind of client is that it is a browser or it is a it is Postman

or it is some kind of server or it's a mobile app we have authorization which sends different credentials like bar

token to the server to identify the user you have accept headers which provides informations like what kind of content

we are expecting whether it is a Json whether it is a text whether it is an HTML file so request headers help server

understand the client's environment its preferences and its capabilities then we have General headers which are used in

both request and responses which have some metadata about the message itself for example the date of the message and

different caching mechanisms like no cache or max age and the connection information whether to keep it alive or

to close it so General header have some kind of information about the request message or the response message then we

have representation headers which primarily deal with the representation of the resource being transmitted

whether it could be a request body or a response message the content type describes the media type of the request

or the response it could be Json or it could be HTML the content length describ the size of the resource in bites the

content encoding specifies any encoding like G or D flare then we have eag a unique identifier which is mostly used

for caching so representation headers provides information about the body of the message or the response ensuring the

clients and servers know how to interpret and process request or the response then we have security headers

which are used to enhance the security of the request and response by controlling behaviors like content

loading cookies and encryption hsts ensures that the client only communicates with the server over https

preventing protocol downgrade attack content security policy restricts the sources from which content like

JavaScript or css and images can be loaded helping prevent cross site scripting attacks x-frame options

prevents the web page from being embedded in iframe mitigating clickjacking attack X content type

options ensures that the browser does not try to guess the mime type of the content preventing MIM type sniffing

attack that cookie with HTTP only or secure Flags secures cookies by making them inaccessible with JavaScript and

ensuring they are sent over https so security headers helps protect the client and server from a variety of

attacks by controlling how the browser behaves with resources and enforcing security policies now there are two

ideas that we see here while talking about HTTP headers one is extensibility HTTP is highly extensible

because headers can be easily added or customized without altering the underlying protocol

right we only have to add some kind of metadata and the whole flow of the interaction changes depending on that

now headers can be defined and used for various purposes making HTTP adaptable to new technologies and use cases for

instance security enhancements as we just talked about headers like strict Transport Security and Force Security

connections custom headers developers can create custom headers like X custom header for specific application for

their own use cases content negotiation the accept and the accept language and accept encoding headers allows servers

to serve different versions of the content depending on the client's preference so the next part is the idea

of remote control sgtp headers act as kind of a remote control on the server side they allow the client to send

instructions or preferences to the server influencing how the server responds or processes requests for

example content Ty negotiations clients can request specific formats using accept header and the server can respond

with the appropriate format if the client says I want the HTML format the server sends the HTML and if the client

says they want Json then the server sends the Json format then caching and expiration control the server can use

headers like cach control or expires to control how long a resource should be cached by the client then authentication

the client can authenticate itself to the server through authorization header influencing Access Control decisions

right so these provide more capabilities on top of our messages which are very useful in a lot of instances now we have

some kind of idea about the headers in HTTP and we will explore them in more depth when we are going through the

demos let's move on to the next component of a HTTP message right the HTTP method HTTP methods exist to

represent different kinds of actions that a CL client like a browser or an API consumer can request on a server

instead of every request doing the same thing methods Define the intent the keyword note here is the

intent the intent of the interaction this gives clear semantic meaning to each type of action and it is

pretty intuitive in that sense we use get request to fetch some kind of data from the server and it should not modify

anything on the server use post to create some data in the server and post request have a body which makes sense

because how else would you send the user data to the server then we have patch which is used to update some data

for example you have a users profile page where they can update their name and this also has a request body to send

the data to the server we also have put which is also used to update data but what sets it apart from patch is

whatever data that comes in request body should completely replace the previous instance basically patch can be thought

of as an append action or a selective replacement while put is a complete replacement even though a lot of times

developers use put when they should be using patch and go against the semantics so the thumb rule is always use patch

unless you have a specific use case for put then we have delete method and as the name suggests we use this to delete

some kind of resource from the server and one prevalent idea that we have in the context of HTTP methods is the idea

of EMP poent and nonm poent and what the idea of important means is these HTTP methods can be called multiple times and

we can expect the same kind of result for example we generally consider the methods get or put

or delete in the category of it poent methods because just think about it

you're trying to fetch some data from the server right it does not matter how many times you fetch it the data should

be same you should not be able to modify any kind of data in the server so get is obviously impotent and then we have put

because put completely replaces the resource in the server so it does not matter how many times you replace an old

with the new data it will the result will be the same right so that is also considered as important then we have

delete you can only delete a resource from the server once because after that it is already deleted you cannot perform

the action multiple times and expect different results right the result will always be the same the resource can only

be deleted once and then we have the idea of nonmutant usually we consider post request to be non important because

once you submit a request to create some data let's say uh a user can create a note using

your app right they submitted a post request to create a new note and the first time they submit the request the

request goes through the response is successful and a new note is created and when they do it a second time we create

a new note right there are two different results for the same kind of request so that's why we consider post to be a

non-idempotent method because it produces different results for the same kind of request after all the sgtp

methods we have one other method which is called the options method and this has a very interesting use case which is

used in the course flow right we discussed this before in brief the course flow which is part of the same

origin policy which browsers have now options method this method you probably won't use directly as a developer but

you will see them once in a while in your browser network tab in pre-flight requests so the options method is used

to fetch the capabilities of the server for a cross origin request also known as course which is used because browsers

have same origin policy which means by default browsers follow the same origin policy which restricts web pages from

making requests to a domain different from theirs different from the one serving the web page and course is a

security mechanism enforced by browsers to control how web applications interact with resources hosted on different

domains which are cross origin and without course browsers block the request made from a web application

running on one origin like example.com to a different origin like API another example.com for security reasons course

allows servers to specify who can access their resources and how in a cross origin request there are primarily two

types of flows one is a simple request flow and another one is a pre-f flighter request flow first let's look at the

simple request imagine our client our frontend is at the Domain example.com and our server is at the Domain api.

example.com and we make a get request which looks something like like this this is how the flow looks like the

client sends a request the browser automatically adds the origin header to indicate the origin of the request the

request uses a simple method which are usually get post or head and then it reaches the server the server checks for

the origin header against it C policy if the origin is allowed the server includes the access

control allow origin header in the response and it sends the response response the server responds with the

resource and includes the necessary course headers for example Access Control allow origin header and this is

what the response looks like it has the appropriate course header which the browser looks for while it passes the

response it sees that our domain which is the example.com is different from the host domain which is the api. another

domain.com and since these two are different the browser checks whether the server's response has this header or not

whether it responds with this particular header Access Control allow origin if the server responds with the domain of

the client example.com or it could also respond with star which means allow all Origins right if either of those two

conditions are true then the browser lets the response go through to the client to whatever the JavaScript client

requesting the resource and imagine imagine if the response looked like this the client sent the request the server

got the request and assuming the server did not add the corresponding course headers or assuming the server did not

allow this particular domain as the client then what the server does it excludes this particular header from the

response so the response looks something like this and when the browser finally parses it it sees that it sees the

absence of this header and what it does it blocks the this particular response from being passed you will get an error

in our console and we will see the error in the network tab as a course error and this is how a simple request flow looks

like next we have a pre-lighted request flow for a cross origin request and how does a browser distinguish between a

simple request flow and a pre-lighted request flow so these are the three conditions the browser checks before it

decides it has to do a PF flight request which basically means it has to do a request before the original request to

inquire some stuff to inquire some capabilities and to let the server know about some capabilities right and when

does a request qualify as a pre-lighted request one is the method is not get post or head example it is a put request

or a delete request and it is a either or situation right it either of these conditions have to be true then it will

be considered as a pre-lighted request so the first condition is obviously it has to be cross Surin request which

means our domain and the server domain has to be different and the second condition is one of these three and

after that it will be a c request and inside that it will be a pre-lighted request so the first condition is it has

to be either a put request or a delete request or the request includes some nons simple headers nons simple headers

are basically anything apart from our general headers or request headers so one example could be an authorization

header or some custom headers right and the third one is or the request has a content type other than application form

URL encoded or multiart or text plan these are called General content types or simple content types so if assuming

our application our client request the data to be in Json in that case it will be considered as a

pre-lighted request and if you are a frontend engineer or a backend engineer you know that mostly we deal with Json

data right so most of our requests are considered as a pre-lighted requests these are the three conditions either of

them have to be satisfied before we move on before the browser moves on to make a pre-flight request what does a

pre-flight request looks like this is when we finally see the use of options method right a pre-flight request is

made with an options method and pre- flight request looks something like this it has the method as options it has the

appropriate resource URL and the HTTP version it has the host header which is the header of our API the origin the

domain of the front end and this header Access Control request method so it is basically asking the server whether this

method is supported for this URL or not so it is saying I am making a cross origin request and do you support this

put method for this route right and it also asks whether you support this particular header or not if it is needed

and the browser sends the options request to the server the request does not include actual data right this does

not include any request body or anything it is just a general inquiry to the server about its capabilities and after

that if the server is is properly handling course flow if the server supports cross origin requests it

responds something like this and if the server does not handle cost request then it won't respond with this and the

request will be automatically blocked by the browser and if it does handle it it requests with something like this what

it says is this is the status code of the response which means there is no content this is just a general

information we use 204 when there is no content then these are the four important headers it responds with the

first one is the access control allow origin it says yes I allow the client's domain which is example.com as a cross

origin as a valid cross origin request right it can either respond with this or it can respond with a star which

basically means I allow all types of clients to make request to me this is the one valid condition it is browser

checks it the next is we asked whether you allow put request for this resource or not so the server says yes I allow

these two different kinds of methods it is the put method and delete method for this resource for this particular route

right and the browser checks it off the next one is we asked whether you allow authorization header and the server says

yes I allow authorization header and then it is also checked off and the last one is Access Control maxage what this

means is the server says don't make make any more pre-flight request to me right it says these these configs whatever I

responded with these will be the same for at least the next 24 hours so you don't have to keep making more

pre-flight requests for every route before you send the original request so this saves some bandwidth for both our

servers and clients so this is the pre-flight request that the browser makes and the server responds with the

appropriate headers the browser checks off all these conditions and then the browser sends the final request which is

the original request that the client wanted to make and the server responds according to the original request with

whatever the operations are required this is how the typical course flow looks like and now that we understood

the theory let let's look at a real demo okay now before we start let me make this clear that we won't be looking at

any code because as I said we are learning from first principles and the rule for that is

we try to understand the concepts first how the underlying mechanism Works before we dive into code before we

understand the how but first we have to understand what for all these demos that we are going to do in this video and and

all the other videos in this series we won't be looking at any code it does not matter what language the server is

written in what framework it is written in I'll explain what I am changing in the server what I have done in the

server for all it matters it can be implemented in any language that's that and this is a tool it's called burp suit

and it is used by ethical hackers if you may it has a lot of features as you can see but primarily what we are going to

use it for is HTTP intercepting or visualizing HTTP traffic and it offers a quite a nice set of features for that

with that what I have here is I have a simple front end app again it could be in any language and what I want to show

is one for our course flow one is the simple request and one is the pre-light request and how they look like in actual

browser environment so let's fire the first simple request okay so we got this response

that we are rendering here and let's inspect the request and response for now right if you go here this is what the

request and response looks like right we already took a look at all these components right the method and the URL

and all these headers and the response the status code and the response headers the response body so let's focus on the

course here what is the first parameter for a request to be considered as a cross origin request by the browser the

first thing is the origin so this is our local host file 173 and the host right this is our apis

which is running on the port 3,000 of Local Host since the host and the origin are running on different ports in Local

Host by the browser this is considered as a cross origin request and according to the same origin policy we can only

make request from Local Host 5173 to Local Host 5173 right since we not it is considered as a cross origin request the

headers to focus here is origin header let me minimize this so the the headers to focus here is

the origin header which is our origin the front end origin and the host the domain or the port that you want to

connect to which is the apis right for the request these are the two important headers now let's go to the response now

the thing to f Focus here is this header right as I had explained the browser makes a cross origin request and when it

gets the response it checks whether it has the access control allow origin header or not or whether it has the

appropriate port or the domain so it has the front ends Port right the 5173 or it could also have star so in

that case the browser lets the response through and and it does not block it since according to the core sets this

response is now expected right this response is now allowed and now that's why we are able to get the response and

now we are able to render it in our front end right let's see what happens if you remove this header like this

Access Control allow origin header from the server for a simple request let me just go and make this change okay I have

went ahead and made the change in the server now it won't return the appri corer Access Control allow origin Let me

refresh this and let's clear this history so that we can focus on the particular

request now let's fire the same request again and before we do let's disable caching so that we can see what a new

request looks like right so let me fire this the browser blocked this response right it says course error and why let's

inspect this request and response so here if you see we have a cross origin request because our origin is 5173 and

the host is 3001 and the server does not return an access control allow origin header right because we removed it and

for that reason the browser has blocked the request because of course error so this is how the simple request flow

looks like and now let's move on to our pre-lighted request right okay I have gone ahead and enabled the course again

so that we can test the pre-flight request flow okay let me fire this all right as you can see here the

first request that went through was the options request right the pre-flight request and then the original request

went through let's look at them one by one how the whole flow looked like okay for the pre-flight request as we have

already discussed the method was options and it is a cross origin request because the referrer and the origin are

different ports right and for that what the server responded with it responded with status code 204 no

content because it is just a general inquiry right pre-flight requests are General inquiries they don't have

request bodies or response bodies and the server responded with Access Control allow origin with the front ends domain

or Port right this makes the browser allow the response wants to go through that is the first condition and before

that let's see why a pre-flight request was fired right why why this was not considered a simple request he

investigate the original request what was the conditions it is not a simple method which is get post or head right

that is that was one of the conditions so it is already satisfied but let's look at others it also has a

authorization header that also counts it out of a simple request right that is the second one and the third one is the

content type is application Json and it is sending a Json request body right all the three conditions are satisfied but

even if one of them was satisfied it would have still fired a pre-flight request for this let's go back to our

options pre-light request analysis the first one was it allowed the client's domain so that the browser lets the

response to go through the second one Access Control allowed methods so the server says all these methods get

post put delete these methods are allowed for the server it lets the client know the capabilities of the

server the third thing is Access Control allowed headers it says these are the two headers which are not simple headers

that are supported by the server which is the content type and authorization since the client asked for them if you

look at this Access Control request method put the client is asking with the pre-flight request whether the put

method is allowed or not and whether the these two headers authorization and content type these two headers are

allowed or not so for that the server is responding with these headers it says get post put delete is allowed it also

says content type and authorization is also allowed we have also set the max age for the access control to zero for

testing purposes right because if you cach this for let's say 5 minutes then pre-flight request be fired that's why

this is used for and the content length is zero because there is no content since the pre-flight request was

successful it returned 204 it is a success status code the browser let the response go through and it fired the

original request it is the put method and this is the original request it has put method the resource URL the origin

and the authorization header the host the request body and it responded with a response body right and that's how the

pre-flight request flow looks like so the simple request and the pre-flight request flow combined they make the

whole course flow and this is all you have to understand how course Works behind the scenes and why these headers

are important and how browsers react to them okay moving on the next component that I want to cover is response codes

this part this 200 okay what are these and why are they needed HTTP response codes exist to communicate the result of

a request in a standardized way know you can just look at the response code and see whether the request was successful

or not or what is the state of the server without looking into the body or without judging from the response

message that okay so if the request was successful then I would have expected this structure if the request was

unsuccessful then I would have expected this structure if the server crashed then I would have expected a null object

right we don't have to make those decisions we can judge by these status Cotes so they quickly inform the client

whether the request was successful resulted in an error or requires further action they also help clients handle

errors by providing specific codes to identify the problem for example unauthorized access will return 401 so

now the clients can check whether it is 40 one and it can log the user out saying you have to log in again right

those kinds of actions they can be judged with response codes or imagine let's say it was a bad request error

because of some invalid value through a form submission then the client can ask their user to make changes to their form

submission and resubmit right depending on those status code which was like 400 also standardization HTTP response codes

are standardized across all web services enabling consistency in how servers communicate with different clients

regardless of the platform or language used it does not matter whether you are making a server in python or golang or

rust or JavaScript or Ruby you have to follow this standard if the request was successful you have to return to 200 if

the if you created something you have to return to1 these are the standards and you have to follow them now before HTTP

status course clients would have to guess the outcome of a request based on the content of the response as I said

leading to inconsistencies and inefficiencies HTTP status code solve this by providing a universal language

that all clients and servers understand streamlining interactions and error handling on a high level response codes

are three-digit numbers they could either start with 1 2 3 4 or five and depending on the starting digit we

categorize them as different level of errors or different types of errors on a high level

the digits starting with one are informational responses two are success responses three are redirection five are

server errors four are client errors response codes starting with one this code is sent by the server to indicate

that it has received the headers and the client can proceed to send the request body and when is it used commonly used

in large uploads the client sends the headers first and if the server is okay with the request it sends a 100 Contin

so the client can send and the rest of the body also there is which is which is used for switching protocols right we

have a use case for that this indicates that the server is switching protocols as requested by the client such as

upgrading from HTTP to websocket this is not the mostly used response codes that you will encounter on your day-to-day

life so let's focus on these four which you'll see a lot of times especially 2 4 and 5 200 series as you know are used

for success responses and under that we have three mostly used ones one is 200 and

2011 and 204 200 is the most common code it indicates that the request was successful and the server is returning

the requested resource or performing the requested action for example successful get request where a resource is

retrieved 2011 it indicates that the request has been fulfilled and resulted in a

creation of a new resource for example post request or new form submissions that's where servers use this response

to indicate that the a new resource has been created then we have 204 that we just

saw in a course flow when we send options request a pre-flight request and the server responds with 204 saying that

there is no content but these are the information in the form of headers this also indicates that the request was

successful but there is no content we also sometimes use it for delete delete request right the client makes a delete

request you delete the particular resource but the server says okay I've deleted it but there is no content to

return right you can just assume that the request was successful then we have 300 and in 300 the mostly used ones are

31 302 and 304 301 means moved

permanently which means the requested resource has been permanently moved to a new UR URL and the future request should

use this new URL for example let's say initially you had a route called user and eventually you decided to move

that route to slash person so in order to maintain backwards compatibility so that old users or old applications that

are still using this route don't break what you do is you add a 301 response for this routes and read direct then to

the/ person route so it is a permanent redirect the next one is 302 which means temporary redirect the requested

resource is temporarily located at a different URL but the client should continue to use the original URL for

future request so when do we use this let's imagine you are running a campaign or something and for those couple of

hours you want to redir redirect a particular route to a new route for catching new traffic or showing a

different UI or something like that but you don't want to stick to that right you want to revert the changes so you

want to say to the client that for for now I'm making a redirect but later on you should use the original route only

and then we have 34 which says not modified it indicates that the resource has not been modified

since the last time the client requested it and this we will see soon in a bit in a uh when we explore our caching demo

and when do we use it this is mostly used in conjunction with conditional get request to allow efficient caching right

when we are using e tax to let the client know that the uh response is not modified it should use the cached one

only instead of downloading the new response so it just says that it is not modified so you should keep using your

old response the cach response then we have the 400 series errors and as a backend engineer I think you will mostly

deal with these errors because these are the client errors or errors that are triggered because of something some

behavior from client so let's see what are the common ones the first one is 400 and what this means is it says bad

request and when does it Trigger or when should we fire it for example when the client sends invalid data or illogical

data or something related to data right for example you are expecting a number and the client sends an array or a

string right you are expecting an email but the client sends a phone number something like that it's a bad request

that's what it says and you're letting the client know that there is something wrong with your request format so fix it

and make a new request the next one is 401 which is which means unauthorized when should you fire this when a request

requires authentication but the client has either failed to provide valid credentials or is not authenticated at

all so let's say you are expecting a JWT token and either if the JWT token has expired or if the client has not send

the token in the first place so in those scenarios you want to say that you are unauthorized that's when you respond

with this status code 401 next up we have 403 and it says forbidden which means

the server understood the request but it refuses to authorize and this can happen even if the client is authenticated for

example when a user tries to access a resource they don't have permission to access so let's say you are user a and

you are trying to delete a resource of user B so that's when the server says you don't have the necessary permissions

to perform this action so you are forbidden 403 next up we have 404 and it means not found I think this is the most

famous status code 404 this is fired when the client requests a resource that is unavailable

either because the URL is incorrect or the resource has been deleted so in those cases the service is 404 it is not

found then we have 405 which means method not allowed when does it get fired when an invalid HTTP method is use

such as trying to put to a resource that only accepts get or post this often happens because of typos right we are

working on front end and instead of doing a put request we do a patch request or instead of doing a post

request we do a put request so in those scenarios server says 405 which means method not allowed then we have

409 which means conflict this has a number of use cases one of which we can imagine is let's say in your app you are

allowing users to create folders right and the condition is the folder names has to be unique they cannot create two

folders with the same name when they try to create a new folder when they submit a post request you check whether the

folder is already existing or not if it is you can respond with this error 409 conflict so the client will understand

that a folder with that name already exists and it should try with a new folder name right it means conflict and

at last we have 4 to9 which means too many requests and this is mostly used when we are trying to rate limit uh the

client's request rate limit basically means if client tries to make too many requests in a particular interval let's

say in your server you have configured it to allow at most 60 requests for a client in 1 second and if the client

tries to exceed that you can respond with this response code which ISS 429 too many requests okay let's move on to

the 500 series and we have the most famous one which is the 500 which means internal server error this is often used

for unexpected conditions in server something some process broke or some exceptions were raised which were not

handled in the server so something unexpected happened at the server so instead of you know just returning a

empty response or just breaking or hanging the request we respond with 500 internal server errors the client knows

that something went wrong with the server then we have 501 which means not implemented for example example when the

server does not support the requested HTTP method or functionality but it plans to edit soon so that's when we add

a 501 which means currently it is not supported it might be in the future right so we are trying to get that

intention through to the client so that it is not implemented yet then we have 502 right it means bad gateway we

usually see this in Proxes like NX it means when a server acts as a proxy like in a load balance system or reverse

proxy and Upstream server returns an invalid response so that's when 52 is returned this is not something we uh

return intentionally this is handled mostly by Proxes and load balancers then we have

503s which means service unavailable when the service is down when the service is temporarily unable to handle

the request such as during high traffic or when it is going maintenance so that's when you return 503 to let the

client know that the service is unavailable right now and it should try again later at last we have

504 which means gway time out it is similar to 502 but this specifically means that the Upstream server failed to

respond within timeout period let's say you have you're using enginex and enginex could not get a response from

our original server which is running behind it so that's when ninx responds with 504 Gateway timeout that it did not

receive any response from our original server that's why it says timeout okay that's all that's pretty much all the

response codes you need to know to work with 95% of use cases now in order to

solidify our understanding let's look at a quick demo where we look at some of these responses okay now again we have a

friend end app for this demo where we are trying to emulate some of the response resp CES and there is a server

running which will respond with different status codes right okay so what we'll do is let's just fire all of

these requests then we will examine what the responses look like so let me go

ahead and okay

created this one one all right we have fired all these requests let's go through one by one so

as you can see we have all these options request before each original request because because these are cross original

requests the browser need to do pre-flight request to support those let's go through all the original

requests one by one to see what are the responses looks like okay the first one was 200 okay as I said this is a

successful respon which means request was successful right the status code is 200 and the next one

is a post request which says 2011 which means created and This Server response even though it is a mock request and a

mock response this is what usually it looks like the resource was created successfully and and the status code is

2011 then we have 401 bad request says bad request missing required data then we have have then we have 401

which means unauthorized which which typically means either you have not included the token the JWT token or the

cookie or whatever authentication mechanism you're using whether you are not you have not included that or even

if you have it has expired or it is not valid anymore next we have forbidden which means you're trying to perform

some action which are not authorized to do right that's why it says 40 through forbidden you do not have access next up

we have 404 which means not found and the server says something like not found the requested resource could not be

found next we have 409 conflict which is resource already exists so this could happen when as our previous example

maybe you already have a folder and you're trying to create a folder with with the same name again so that's why

the server says resource already exists right then we have internal server error which is 500 and the server just

says internal server error without letting the client know too much of the information for security reasons then we

have 503 service unavailable please try again later right that's pretty much covers most of the status Cotes that we

have discussed and that we saw in this demo okay moving on let's explore another interesting concept HTTP caching

what does http caching mean HTTP caching is a technique to store copies of responses for reuse reducing the need to

repeated request to the server this improves load time reduces uh bandwidth and decreases server load right because

the client does not need to download a lot of data and the server does not need to send a lot of data if the data is not

changed this client can just reuse the old data right that's what caching means reusing the old data if the data is not

changed in order to understand this let's just go through a demo instead of of going it theoretically how it looks

like practically in a browser environment okay let's go to the request cycle and understand how caching works

by following the trail let let me just do a refresh so when we first render the page we are doing a fetch operation so

let's start from there okay we fired all these requests and what you want is the last one right because these are just uh

JavaScript files and CSS and stuff okay okay what happened in the last request Let's see we did a get request to this

endpoint API resource and there is nothing else in the request headers that we should focus on right now right let

come let's come to the response what did the server respond with the first important thing is this one cache

control what it says is you should maintain the cash for this resource for maximum 10 seconds right this is the

first important header the next is e tag e tag is basically a hash so for the sake of this example we are using a

random number but eags are usually hashes which are computed from a response for example the server might

have taken this response hashed it and sent us that hash in the form of E tag right and we will see what is the use of

that in the next request the next important header is last modified with this what the server is trying to say is

this is the last time this request was modified and judging from this time and date we can decide whether we should use

the old resource the cash resource or request for a new one these are the three important headers that we are

going to use for caching okay so that is the initial request and in this request the server responded with a 200 along

with the requested resource whatever that is we won't focus on that we are just seeing there is a response body all

right what happens when we do a fetch operation for the same resource right let's do a

fetch okay what happened here let's look at the response let's focus on the request first we are again doing a get

request to this end point and now the headers to focus in the request is this these two headers if none match and if

modified since what we are trying to say to the server is if the E tag the hashed version of the response object the E tag

of the requested resource is not the same as this one which we have with us in the browser or if the request has

been modified after this which means we have the outdated version right we are saying if the eag does not match or if

the request has been modified after this then send us a new resource then send us the updated resource otherwise I will

just use my cach version which I have with me in the browser cache and to that what server says is it responds with 304

not modified and we just talked about different response codes right and 304 meant the requested resource has not

been modified ever since what the server did was it checked the if none match header which is which is the eag and it

also checked the last modified and since the resource matched either the eag or the last modified it send the client a

304 response which means the requested resource has not been modified ever since from the last time you fetched it

so you can use you can go ahead and use your cast version let's update the resource for now let's clear

this and let's fire an update resource okay what what happened here let's explore the first request okay

ideally we should have done a patch request or put request but for the sake of this example again we just fired a

request a post request okay we updated the resource and the server responded with 200 okay and it also sent us a new

e tag right 2943 and after that the client did a new get request in this as you can see the client sent the old dag

which which was 3141 our cast version so client did a get request with the old eag and the last modified sense to the

server and the server responded with 200 instead of 304 because the resource has been modified after that right because

we did a update resource request and the server responded with 2943 which is the updated e tag the last modified Etc and

let's go ahead again do a fet request okay and what do we have here we are again using the eag that was just

provided by the server in this request right this was 200 and the server provided as with a new e tag we are

using that e tag and the last modified value and we are doing another get request so the server checked it again

and since it is the latest version of the resource it again responded with 304 for not modified so this is how you

handle caching using the HTTP protocol using different headers in server and client even though in a production

setting it gets a lot complicated because the server has to manually Implement and manage all these e tags

and if by mistake forgot to update an e tag right then the client will continue to use the cast version with the

outdated resource which is not a good idea nowadays we have better Solutions for caching right for example react

query which is a complete client side caching so the client has the complete power over when it wants to use a cach

resource and when it wants to refetch right at what interval and all these powerful capabilities which is in my

opinion a much better solution compared to the traditional HTTP based caching but it is good to know that we have this

option if our use cases simple enough then we can go ahead and use HTTP based caching moving on another important

topic that usually comes up in client server model in HTTP is content negotiation we already looked at some of

these headers for example accept the content type application gson Etc so this is an important topic to understand

how clients and servers exchange information about different types and and coding and representation of the

content this is basically a mechanism using which client and server agree on the best format to exchange data the

client can indicate its preferred format like Json or XML or HTML and the server will try to respond with a compatible

format or if not available a fallback format that is the whole idea to look at a high level we have generally three

types of content negotiation one is a med type which means the client specifies the desired format through the

accept header which is application Json or XML then we have the language negotiation the client requests content

in a specific language using the accept language header uh it could be English or Spanish then we have encoding

negotiation right the client specifies which encoding it supports using the accept encoding header like gz or

deflate and the server responds with that compression format we have a topic which is HTTP compression which is also

part of this topic only so we will quickly see that also in the demo that's the whole idea and let's just jump into

the demo to understand how it works and again there is a server running which will help us understand different types

of content negotiation and this is a frontend client through which we will communicate with our server so let's

just try out different types of requests and we'll see how the headers differ and how the responses differ depending on

the types of headers the client is sending okay let's first just try the default one which is which is a language

is English the format is Json and the preferred encoding is jip so let's do F FD source and this is what the request

and response looks like okay so it was a get request to this endpoint and what we said is the

language we are saying is English we are sending that information using the accept language header that we prefer

the English language then what we are saying is the format is Json right we send that with accept application Json

and there is the encoding header right that we are saying accept encoding the browser supports these kinds of

encodings right JZ defl VR and zsd so we said we are expecting English and in Json format so this how this is how the

server responded it it is sending us a Json and it is in English so let's trve one

thing let's make it to Spanish and see how the response differs I made it Spanish and add did

another fetch okay so let's see how this differs in this the only thing that changed was

instead of accept language English in the header the client said accepted language is Spanish so depending on on

that the server was able to update the response so instead of sending it in English since the client prefers Spanish

the server responded with Spanish and what if we change the format from Json to XML and we do a

Fetch and we look at this what change here we are saying we accepted format is XML and the accepted language is Spanish

this is the XML format and the language is Spanish and on a high level this these are the benefits of using content

negotiation based headers right the client can let the server know what are its preferences whether it is the data

format or the language of the data and depending on that the server can decide to send according to that format it can

make the life of the client Easier by sticking to the preferences those are the primarily two type of content

negotiations while we are discussing content negotiation there is one interesting topic which falls under the

same umbrella which is HTTP based compression it could be either gzip deflate or other formats right so let's

look at why do we need compression and how does it work so now what I've done is I've gone ahead and replace the Json

response with a very large file file of 11,000 entries now let's file the request okay this is the response

response and the size is 3.8 M since it's a very large file and let's look how the response looks

like this is what it looks like because we are compressing the file on the server side with gzip encoding right

here it says content encoding is gzip because the client says it accepts encoding of gzip one of the encoding

formats now why do we use it to show that let me just go and disable compression in the server side okay and

now that I have disabled the compression let's fire the same request again and as you can see the significant

increase in the size it's the same file right it's the same file with 11,000 entries and because we are using

compression the file size was 3.8 MB and now that we have disabled it the file size becomes 26 M that's a huge increase

in size and imagine every client having to download that file it it it caes a lot of based of bandwidth and that is

why we need compression so that if the response size is very large we can compress it to a format and on the

client side the browser can decompress it and it will get the same response and this is another important topic which we

won't really work with it's good to know that it exists behind the scenes in the early days of HTTP specifically HTTP 1.0

each request response cycle required a separate connection to the server now this created inefficiencies since

establishing and closing TCP connections is resource intensive and slow to address this persistent connections were

introduced in HTTP 1.1 now with persistent connections a single TCP connection can be reused for for

multiple requests and responses avoiding the overhead of opening and closing a collection for every interaction and for

achieving that they introduced this header called keep alive keep alive is the mechanism that enables persistent

connections it allows the client and server to reuse the same connection for multiple request responses until one of

them decides to close it so what are some key points to remember here in sttp 1.1 connections are persistent by

default we won't have to do anything explicitly about them meaning they remain open for further requests unless

explicitly closed and multiple SB request and responses can be sent over a single connection this reduces latency

and saves resources as fewer connection need to be established now the second thing is the keep alive header while

persistent connections are default in HTTP 1.1 the connection keep alive header is it's still sometimes used to

explicitly ask the server to keep the connection open this header can also include option like how long the

connection should remain open with a timeout or how many requests can be sent before the connection is closed with the

max value now the third thing is when connection is set to close and when that is specified the connection is closed

after the response is sent this is the behavior in HTTP 1.0 by default and it can still explicitly enforced in HTTP

1.1 that is some amount of information you just have to understand but you usually won't be working with them the

default values work fine one last topic that I want to cover is handling large request and responses how server takes

in large request like Files video files image files audio files any kind of file which are very large compared to our

typical Json and how client can receive large responses in the same way so let's just jump into the demo and see how it

usually works now here we have two examples in one we will see how clients can send large request to the server and

in the second one we'll see how servers can send large responses to the client first one is multiart request multiart

is usually used for sending large files or any kind of files to the server from the client and the difference between

our typical Json request body is in multiart request the file the data of the file the binary data is transferred

to the server in Parts in different parts that's why it is called multiart request so let's see how it looks like I

have selected a picture and let's click on upload file now let's look at the response of

this here is what the request looks like it is a post request and we are specifying a Content length and our

content type is multiart form data and this is the important part the boundary and why we need this is since our binary

data the binary data of the file is transferred in parts we want to specify what is going to be the delimit what

will separate the parts right we need some kind of code that will separate the parts so we are saying this will be our

delimiter and if we do a search here we have the delimeter at the start of the binary data and the next

occurrence is at the end of it when the all the binary data ends in the request body

right that is the use of the boundary parameter and that is how we transfer a large file to the server and the server

can read the file responds with the some details of the file in order to say that the upload was successful the idea is

whenever we want to transfer large files to the server we should use multiart requests okay now the second thing is

receiving large responses from the server for this demo I have used a large text file in the server side and we want

to stream the data to the client side in chunks that's what we want to do so let's start the so let's start the

request and see how it looks like click on stream data so as you can see this is the first

chunk okay so if you go here you can see that we are receiving chunks continuously from the server in

different different requests so the request is pretty much a normal request it is a get request let's look at the

response what the response looks like these are the three important things that we have to to consider one is

content type it is saying that it is not a text content right it is a text event stream which says it will stream the

data to the client through different events and the second one is the connection keep alive it says keep the

connection alive until all the data is sent and if is here we are still receiving the chunks and we can keep

scrolling and scrolling and until the file is completely transferred the server will keep sending the data in

chunks and how does it work because of these headers the content type text event stream and the connection keep

live right and the client keeps appending all the data that it is receiving from the server and

constructing this whole text file and that is how we transfer a large file from a server to the client using chunk

transfer or text event stream and the last thing before we end this lesson I just want to give a brief idea about

what these terms mean SSL or TLS or https even though we don't explicitly work with them it's good to know what

are these SSL was the original protocol for securing Communications between client like a web browser and server it

encrypts data so that the sensitive information like passwords or credit card numbers cannot be intercepted by

attackers right it was the original encryption mechanism between clients and server now currently SSL is outdated due

to some some security vulnerabilities and has been replaced by TLS so this is the modern version of the encryption

that client and servers use for data transmission TLS is a modern and more secure version of SSL it encrypts data

in transit ensuring that any data sent between the client and server is protected from interception and

tampering how it works is TLS uses certificates to authenticate the server and establish an encrypted connection

preventing Eve dropping and data bries TLS is continuously updated with newer versions offering better security the

current recommended version is TLS 1. and what is https then https is basically HTTP but more secutive

features which is which are provided by SSL or TLS initially it was SSL and now it is TLS the underlying mechanism is

TLS and https is the one which uses TLS how it works is when you visit a website using

https TLS encrypts the communication between your browser and the server this protects sensitive data like login

credentials from being intercepted by the attackers and that much information is more than enough on this topic that's

all you need to know about TLS and sdps to work on application Level great we talked about a lot of stuff I hope

you're able to digest that and I hope you rewatch some of the sections so that you are able to internalize all of them

and overally this is all you need to know about HTTP at least of course there are more stuff to read if you want on

HTTP or TLS or PCP protocol different different components of HTTP but in order to work on backend systems if you

understand this much if you internalize this much and you can visualize the whole flow of all the the components

that we talked about today then you're good to go you'll be able to understand all you need to understand and you'll be

able to debug most of the stuff now that you understand how the system works behind the scenes and what are the

components that come into play in different different flows that's all about http