Advanced Virtual Machines Creation and Automation in AWS & Azure

hello everyone my name is Abhishek and welcome back to my channel so today we are talking about day four of our devops

Zero to Hero course for people who are just watching our videos for the first time we are doing a

45 days of complete devops course that is devops Zero to Hero and today we are at day four if you haven't watched our

previous videos Day Zero day one day two and day three I would highly recommend you to go and watch those videos first

so there is a demo of Zero to Hero playlist on my channel and those videos covers the previous topics and this is

typically virtual machines part two so in the previous video we talked about virtual machines and uh you know where

we talked about the evolution of virtual machines what is the server what is physical server how a physical server is

different from a virtual machine what is the concept that allows you to create virtual machines and we also looked at

how data centers are you know getting extinct you know previously even a startup used to have their own data

center but now people are relying on platforms like openstack or AWS or Azure so we also talked about those things so

it would be a really interesting one to watch part one before part two now today what we'll learn is we will

learn about virtual machines in a advanced way we will talk about how to create virtual machines we will look at

the top platforms or the top Cloud providers today you have AWS you have Azure and I'll also take a on-premise

scenario so that if you're working in people might be working in different organizations so that everybody gets a

feel of it like there is also Google Cloud platform but we cannot cover all of the topics so what I'll try to tell

you is how to create a virtual machine on these platforms I'll talk about AWS I'll talk

about Azure and then also say I'll also talk about how to create a virtual machine on on-premise let's say your

organization has not shifted to cloud and they're still using their own data center they have their own servers so

how to create a virtual machine on those so these are the different things that we would cover today

okay so before we start the video I would like to just make a simple request that you know I'm making these videos

with at most honesty and you people know that it's a free devops course so the only thing that I would expect from

everyone is to promote these videos within your uh Circle and if anybody who is looking for these videos like

somebody is passionate about learning devops then please do share these videos with them so that everybody is benefited

out of these videos perfect so going ahead firstly just a precap of uh sorry recap of what we have

learned yesterday so yesterday we learned that in devops to create a virtual machine you basically make a

request to one of your Cloud providers I'll talk about on-premise later so for now just understand that we are dealing

with Cloud providers okay so what we would do is we'll make a request with one of these Cloud

providers let's say we have we are only talking about AWS and Azure so as an end user what you do is from your personal

laptop let's say this is your personal laptop and you are a guy who is sitting in front of this laptop and you are in

search of a virtual machine so what you do you would log into something called as

AWS console okay by logging into AWS console what you do is you make a request to AWS

asking for a virtual machine in AWS terminology it is called ec2 instance don't worry if you are not aware of AWS

in this video we will directly look at the AWS console I'll show you how to create a virtual machine and today we

are not even deep diving into AWS so don't worry even if you have zero knowledge on AWS I'll show you

theoretically as well as practically how to create a virtual machine on AWS and also we'll take a look at Microsoft

azure so for theoretical understanding what happens is there is a developer or there

is a person called X Mr X and what Mr X would do is sitting in front of his laptop he'd open his favorite browser

and in his favorite browser he would search for AWS console once he go to the AWS console he can make a request with

the AWS through the UI to create a virtual machine or ec2 instance and what AWS would do is AWS in response

would send you back something like an IP address and all the specification related to

this virtual machine that you have requested This is the End thing that happens or

this is the overall thing that happens if you are trying to create a virtual machine from the UI and the same thing

happens with Microsoft Azure as well right so whether it is Microsoft Azure or whether it is AWS the process is the

same again there is a person who is sitting in front of his laptop let's call him Mr X and what Mr X would do is

instead of AWS on his favorite browser he will try to open Microsoft Azure portal which is

Azure portal right once he opens the Azure portal don't worry we'll also take a look at this uh practically on the

Azure portal he will do the same thing he would ask the Azure to create a virtual machine in Azure terminology we

call it virtual machine only and Azure would send back him the same response right the IP address plus all the

specifications that are required to login so this is the process but

the important thing that we would understand is okay for creating one virtual machine everything is fine like

you know you can log in you can go through the UI and you can create a virtual machine but as a devops engineer

always from class 1 I teach you the same thing that we have to focus on efficiency let's say you get hundred

such requests okay because in your organization you might have thousands of people and if you get hundreds of the

same requests right every day you receive the same request people ask you to create a virtual machine so would you

go to your AWS portal 100 times and create a virtual machine you can do that but that does not add efficiency so in

devops every day what we are talking is the same thing that is efficiency so as a devops engineer you should always

Target in improving the efficiency of your routine task so what you do is you would look for automation

so AWS people also provides you the support for this automation right because you are talking to AWS AWS has

to develop a feature that allows you to automate the stuff on AWS right so what happens is in general this is called a

AWS API okay so AWS provides you API so in this case we are talking about ec2 so it is

called as AWS ec2 API right what is it called AWS ec2 API because we are talking about ec2 for now we are only uh

talking about the AWS ec25 similarly if you want to create storage AWS supports AWS S3 API if you want to talk about

volumes AWS support AWS EBS API or the something else so for now understand that AWS people have developed something

called as AWS ec2 API and using this AWS ec2 API the end user or the devops engineer who is using his laptop what he

can do is instead of directly accessing the Amazon.com or sorry Amazon console from his favorite browser what he can do

is he can write a script okay what this user would do he would write a script

and using this script what he would do he would make a call to the AWS ec2

API right so he makes a call to AWS API and in this script what he'll say is I want to create 10 easy to instances okay

or he'll just write a script and every time a user comes to him and asks for ec2 instance he would run the script and

it would return one ec2 instance okay even if you are writing a script to automate the process for creating one

ec2 instance and even if it is same in one minute of your regular time okay that is also called as a automation

right because there is no manual intervention so when there is no manual intervention that

means there are no errors right because manually you might commit some errors but if you are automating the process

and automation is doing things for you so you are also avoiding errors and you are also saving your time so this is the

process now coming to the script it can be done in multiple ways okay so I'll talk about

the different ways that AWS will allow you to create or automate the stuff on AWS now the process would be same for

Microsoft Azure as well so that is the reason why I'm not repeating AWS and Azure so for now understand that the

process is same for any cloud provider today I am talking about AWS it can be Microsoft Azure it can be Google Cloud

platform digitalocean or any cloud provider okay the process is the same what they would do is for the service

that they are providing so it can be easy to service it can be storage service or any service what the cloud

provider would do is they would write a API so the developers at Amazon what they would do is they have hundreds of

services right Amazon provide you hundreds of services today and today we are dealing with ec2

service in future we'll talk about the other services but for the easy to service

there is a developer right or there are a bunch of developers what these developers would do is they would expose

the API of This ec2 Server okay so this is the API now

what happens is this API would receive a request and if it receives the request in the

expected format as a response what we will do it would send you a easy to instance right so the AWS ec2 is

AWS ec2 API is responsible for receiving the request and once it receives the request and the resource request is

valid authenticated and authorized okay what I am talking about the three things one is the request that you are doing

should be valid valid is following all the standards this ec2 API is expecting and then it should be

authenticated what is authenticated the user who is making the request should have access to the AWS and then

authorized okay sometimes as a user you might have access to AWS but you don't have access to or you are not authorized

to create ec2 instance so all these three criterias have to be matched one is the request that the user is sending

has to be valid then he has to be authenticated he has to be authorized once these three things are done what

happens is as a devops engineer now you can write a script what your script has to do your script

has to make an API call with AWS with a request that is matching all this criteria that is valid authenticated and

authorized and as a output for the script you will get a ec2 instance now what is this script I have been talking

about script script script all this while so now let's Deep dive and understand what is the script actually

so the script can be multiple types one is through CLI okay so because we are talking about AWS

let me call this as AWS CLI so as a devops engineer you can automate the creation of your virtual machines or ec2

instance through AWS CLI 1 2 you can use AWS API itself directly right if you are

familiar with the concept of uh let's say the rest API how to write a rest API and if you have exposure towards any

programming language you can directly deal with AWS API and there are some modules like if you're talking about

python python supports boto3 okay using boto3 module in Python you can directly make a request to the AWS API now this

does not have to be uh I mean you don't have to be familiar at this point of time like even if you don't know uh

python or Java or any programming languages you can still use AWS CLI to automate the process but for now you

should understand that AWS also allows you to write your custom scripts or your own scripts and you know there are open

source libraries or the models that are available and If For example I'm a familiar with python so in Python there

is a model called boto3 and boto3 allows you to directly make a API call with AWS any service ec2 service or S3 service or

any service now the next approach is AWS support something called as AWS CFT that is called as cloud formation templates

so the cloud formation templates are templating language it's a classic template uh you AWS provides a

definition or a proper structure if you provide this template to the AWS AWS will return you with 1 to n number of

virtual missions that you have requested so these are the different approaches and finally there is also a great

competitor that is available in the market that is terraform now if you put terraform aside for now

because we have a separate class where we will talk about terraform and terraform is a uh you know is something

that we cannot uh discuss for now because people have to have certain level of expertise to Deep dive into

terraform and during the terraform class we will Master this uh we will learn Zero to Hero of terraform so far now

just understand that terraform is one of the Alternatives that is available to automate your

resource creation and the advantage with terraform is that if you look at these three things they are very specific to

AWS right this is AWS CLI this is AWS API boto3 and this is AWS CFT but terraform I did not mention as AWS

because one terraform multiple Cloud providers so terraform can automate your process in AWS it can automate your

process in Azure it can automate your process in Google Cloud platform everywhere

okay now what is happening here is using all of these things or any of these things what you can do is you can

directly talk to the AWS API right so what are these things doing end of the day they are directly talking to the AWS

API so who is writing the AWS API the devops engineer or the developer at Amazon what they are doing is they are

exposing their ec2 instance using the API and as a devops engineer you can automate the process in your

organization by using one of these things that is AWS CLI AWS API CFT or terraform or if you are not aware of all

of these things what you can also do is you can also use the manual approach right as I explained even using the

manual approach you can create but the problem with manual approach is that you have to let's say you receive 100

requests so 100 times you have to log into the AWS console and you have to create virtual machines one by one that

is against the pattern of devops because you are doing it in a inefficient way in devops we always talk and if in devops

we always learn the efficient ways so these are the different processes that are available

so in an interview you can say any of these things like let's say somebody asks you in an interview how do you uh

create 10 virtual machines at once or what is the automation that you are using in your organization for

infrastructure creation what is a virtual machine virtual machine is an infrastructure right or ec2 instance is

an infrastructure so people will ask you what is the tool that you are using for infrastructure automation now the very

popular tool that is available in the market is terraform but remember as I mentioned in our previous classes don't

do something because that is popular or don't do something because it has to be done

always look at your organization what is your organization doing let's say your organization is completely focused with

AWS and in future also they only want to deal with AWS okay you can use any of these things you don't have to

definitely move to terraform right you can either use AWS CLI API or CFT and AWS also supports additional things

sorry I forgot to mention that is AWS cdk okay that is AWS Cloud development kit

which is recently introduced so you have four different options right either you can use CLI API CFT or cdk so personally

if you ask me if your organization is very focused on AWS and you know that your vision is AWS like you know next 10

years or 15 years your organization is going to stick with AWS you can use cdk over terraform because cdk provides

Advanced or more uh benefits over terraform okay so in an interview you do you don't have to always mention that I

am using terraform in the organization no you can say our organization using AWS cdk because cdk is very specific to

AWS and AWS has decided that they have their initial support let's say AWS is launching a new service

okay this service is called AWS XYZ so the developers at AWS they have to write an API for it right similar to ec2 they

have to write an API for XYZ now once they write this API who is going to get the first support terraform or cdk

obviously cdk because cdk is a proprietary tool of AWS whereas terraform is a

open source tool which is developed and maintained by hashicorp or the other community members so to have initial

support or to have initial access for any automations or database cdk is far better than terraform

similarly CLI API or CFT as well so always choose your answers wisely and whenever you are projecting yourself for

a devops interview make sure that your Basics are and your foundation is clear now you might ask me then when is

terraform used so you know there are some organizations or most of the organizations these day or these days

are choosing the hybrid Cloud pattern okay what is this pattern called it's called hybrid Cloud pattern so what we

are doing in the hybrid cloud model is that they have their virtual machines in one Cloud platform they have their other

resource infrastructure in a different Cloud platform either it can be because of the efficiency let's say Google Cloud

platform Google Cloud platform is very popular for its Ai and ml stuff okay if you are talking about machine learning

or artificial intelligence Google Cloud platform leads or it's a compared to be better than these things or you know

some people prefer kubernetes on Google so that's why these days organizations are choosing hybrid Cloud Model they

have their Ai and ml stuff on Google they have their other stuff like RDS or something on AWS it's a individual

Choice uh depending upon the organization requirements to requirements so that's why if you are

using a hybrid Cloud Model terraform is definitely a best fit for you because you have to automate infrastructure

across the different Cloud platforms whereas if you are in a single Cloud platform and your organization will

stick to be in a single Cloud platform then choose your answer wisely and go with either cdk CLI API or if you're on

Azure you can talk about Azure resource manager because those things are pnas or they have early access to any resources

on that cloud platform perfect I think now we have talked a lot about these things now let's get

practical right so you need to understand how to manage or how to deal with these things so before that firstly

I'll show you how to deal with AWS through the UI and once we understand that probably we can talk about the

automation that is using CLI or CFT or something and parallel here also show you with Microsoft azure

let me just stop sharing the screen and go with a new one uh okay

so how do I do that here if I click on share document

perfect okay sorry I am still finding my screen just keep my moment

okay perfect so now I'm sharing my AWS console screen right let's say you don't have access or you are very new to

cloud and you want to create an account with AWS it's a very simple uh practice what you have to do is you have to come

here okay go for sign in dot aws.com okay or you can simply click on uh AWS console uh just go to your browser and

say AWS console it will redirect you to this page if you have already have an account with AWS you can proceed with

your sign in provide your user mail ID or if you don't have what you can do is you can come down and you can click on

create a new AWS account once you click on create a new AWS account it would just ask you for some questions okay and

you will be proceeded with creating an AWS account so there are no uh ifs and buts there like you know you can

directly follow the step-by-step approach and finally you will be asked with the card details and once you

provide the card details it will deduct uh if you are in India it will direct two rupees INR from your account just to

validate that you are a authentic user and providing a valid card details and once you are done with that your AWS

account will be created now don't worry that AWS will not charge you for any virtual machines that you are created if

you are using a free instance I'll also show you how to use a free instance don't worry about it now let me just

stop sharing for a moment so that I can hide my user details stop share

and let me log in okay coming here so I have provided my user id uh sorry

for not sharing the screen but I have to enter my password and uh my password details are entered now let me see

let me go back and share my screen perfect so I hope you are looking at the right

screen again perfect so now what you do is once you are done with it come here and search for the services so AWS

provides you a bunch of services okay and you have to Choose Wisely what the service that you want to use so in my

case I am sure that I want to use a AWS ec2 service so you can click on the ec2 service

and what you will do is you look at the instances or the services that are available here what you do is you would

come to the AWS ec2 global view and uh perfect let me just go back here and what I'll do is I would go to the

instances and what I'll do is I'll click on the launch instance okay so click on the launch instance and provide these

details okay now what you are doing you are talking to the AWS uh CL sorry UI directly you are not doing any API calls

or this is not a process of automation what you are doing everything is a manual process okay so what do you do

here you would provide the details here let's say test okay let me call my AWS instance name as test and then you have

to choose over the wide range of operating systems that AWS provides you you can use Amazon Linux you can choose

Ubuntu Windows red hat for a first comer or if you are just learning about uh devops and everything I would prefer

Ubuntu and I would recommend you to use Ubuntu because Ubuntu is widely used in the devops community but you can go with

any other options as well and after that see this is very important choose free tire eligible okay because we want to

try out things and we are not going to pay any amount uh to the AWS for the instances that we're using so always

choose the free tire available what is the difference between free tire and the paid ones so in free tire you get very

uh restricted amount of resources so here if you see I'm only dealing with one CPU and 1GB memory but if I am going

to go to the subscription model or if I'm going to pay AWS then I can choose larger virtual machines here if you see

T2 large I'm going I'm getting two CPUs and 8GB memory but I am going to be charged well if you are using the free

tire available or free time eligible options you'll not be charged by AWS then

after that for the very first time you have to create your key value pair with AWS what is key value pair so key value

pair is something that would allow you to login to your ec2 instance okay once you create your AWS ec2 instance you

would get the IP address but there has to be a password kind of thing to log into your virtual machine right so

that's why click on create new key value pair okay let me call it as test key value pair or let me say test one one

one and now go with the defaults choose RSA you don't have to know about uh you know Ed

for now this is a different uh you know RSA is a kind of thing and Erie is a different uh encrypted public and

private key methodology but for now just go with RSA that is default and pem pem should be fine for the private key file

format now click on create key value pair that would be saved it would be downloaded

and saved in your local this would be very useful to login once your key value pair is gone then it is almost

impossible to log into your ec2 instance so make sure you save your key value pair

then you know we are not going to talk about any of these things at this point of time I am not going to explain you

what is a security group what are the vpcs and all at this point of time because today we are just learning about

installing a virtual machine and uh for the beginners who are watching our videos I don't want to confuse them by

explaining about the advanced concepts of a virtual machine or compute instances and then click on launch

instance that's it now once you click on the launch instance your virtual machine is created

you don't have to do anything else just wait for a couple of minutes go back to the instant screen your ec2 instance is

available here okay it would take a couple of minutes now while we wait for this I would also parallely show you how

to create instances on Microsoft azure you don't have to do anything new or anything fancy apart from uh the things

that we are doing on AWS what you do is click on portal.azure.com okay on your browser go for portal.azure.com it would

ask you for sign up and one good thing about Microsoft Azure is they have inbuilt integration with GitHub okay so

if you are a user for GitHub user on GitHub you already have a GitHub account like like I do what I've done is I have

signed up with GitHub if you see here this is a GitHub icon that I'm showing here so using GitHub I have logged in it

would take less than couple of minutes for you to log in once you are done with it see here I don't I'm not holding any

uh paid account with Microsoft Azure one difference between Azure and AWS here is if you're a new player or if you are

just learning devops you know AWS provides you close to one year uh free membership on AWS whereas Azure only

provides you some 30 to 45 days I'm not exactly I don't remember exactly but AWS provides you longevity so that you can

play around and try AWS for more time Azure you have to buy uh after this one month or let's say no I want to create a

virtual machine with Azure which I'm sure you I have to pay money so click on create resource okay or there is an

option directly for virtual machines so I personally feel that the Azure user interface or user experience is slightly

better than AWS like click on create virtual machine and provide the same details that you're going to provide for

nav I don't hold a free account so if you are starting with a free Azure account then you just follow the same

steps click on start I mean I'm not going to go with the pay and use I would go for start free but I'm already done

with my free membership so if you if you have a free membership then you provide the same details that you have done with

AWS and your virtual machine would be created on Azure as well so whether it's AWS or Azure the process is the same now

if you see here our AWS ec2 instance is up and running okay once this AWS ec2 instance is up and running you can

either log in from the UI or using the key value pair you can also log in using the

terminal so those things I'll show in the next class and I will also show you how to

automate these things right because we should also look at the automation for now I just showed you how to create an

account with AWS and Microsoft Azure how to create these resources through the UI but if you want to learn how to create

them how to login into them using the automation process wait for my next class which would be tomorrow and in

that video I'll show you how to automate the process I hope you watch I mean I hope you'll like this video if you like

the video click on the like button if you have any feedback for me good or bad please do post that in the comment

section I would happy to reply to each and every comment and finally don't forget to subscribe my channel if

you haven't Abhishek virmala thank you