Linux OS Fundamentals and Shell Scripting Basics for DevOps Beginners

hello everyone my name is Abhishek and welcome back to my channel so today we'll be talking about Linux operating

system and basics of shell scripting so if you haven't watched our previous videos on devops so just to let you know

that we are doing a complete free devops course it will be a fortify uh to 40 to 45 days of devops course where we'll be

talking about most of the devops things and our aim with this devops course is to make a person with zero devops

knowledge heroin devops so if you haven't watched those videos uh there is a playlist on my channel

called devops Zero to Hero uh I would highly recommend you to watch the previous videos before jumping on to

this video perfect so if you move to the agenda for the day that is Linux operating system

and the basics of shell scripting firstly what is a operating system right before

you understand the concept of Linux operating system you should know the basics of operating system or what is an

operating system so just try to understand with a practical example let's say you have a

laptop or you purchased a Raspberry Pi or any other server okay so what are the key components of any uh server right

the key components is that you are purchasing Hardware I understand you get a lot of fancy

things like your display you get uh other things mouse keyboard and all the other things that you are purchasing but

essentially you are purchasing Hardware that is nothing but your CPU

Ram and IO right or hard disk so you are purchasing these things and

what you do once you purchase the hardware is you try to use some softwares on top of your on top of your

Hardware okay let's say you purchase the laptop what would be your interest your interest is to

you make use of some softwares whether you are using this for gaming

okay or let's say you are a devops engineer so you must be using some applications like Jenkins

or you must be using some office related stuff or as a student you must uh you might want to uh learn something like

Java or python so you are essentially running some softwares right this is what you are doing so you have a

software application that you want to run against the hardware that you purchased okay

but you can directly do it okay let's say as a user you install Jenkins and you want to execute your Jenkins

pipeline your Jenkins can directly not I mean it cannot directly talk to you the CPU or Ram or the io

there has to be a medium between your software and Hardware that is nothing but your operating system

okay so what is the operating system operating system is something that acts as a bridge between your software and

Hardware so it drives as a medium for the communication between your software and Hardware that is the definition of

your operating system okay so whether it is a Linux operating system whether it is a Windows operating

system or anything so today when you purchase your laptop let's say you purchase the Dell laptop

okay so what Del does is it sells you the hardware okay and on top of the hardware

what is it doing it is installing a operating system and it is giving it to you so

you purchase the Dell laptop which comes with the CPU Ram IO and along with that what Dell is doing is it is installing

Windows operating system okay so that is the reason why you are able to directly deploy any application

or install any application and your application is able to process the requests that are coming from the user

that is you so as a user what you are trying to do is you are installing an application and

your application is talking to the operating system and your operating system is talking to your hardware and

vice versa right your Hardware is again talking to your operating system your operating system gives back the

information to the application and your application gives back the information to you

so this is the process so the communication happens in this way firstly as a user you are installing

some application on your laptop and your laptop or your server okay just for your easy understanding I'm mentioning it as

a laptop but always understand it as a server so as a user you installed application or software on your server

and your server is giving the response or sorry uh giving the request to your operating system and your operating

system is sending the request to your hardware for CPU usage or RAM usage or anything again your operating system is

giving the response back to your sorry your Hardware is giving the response back to your operating system from

operating system it goes to your software or your application and finally it reaches the user so this is the life

cycle and this is where your operating system plays the key role you must have your

applications ready you must have your Hardware ready but without operating system nothing happens so that's why

it's a core earlier it's a heart of everything again depending upon your comfort

whether you're using a apple like let's put let's say that you purchased a Apple laptop or your Mac laptop so it comes

with your Mac operating system if you purchase Dell laptop by default most of the times it comes with a Windows

operating system uh you are purchasing ThinkPad or something the Linux version of it so it comes with a Linux operating

system so depending upon the user or depending upon the vendor they come with your own operating systems but why Linux

is very popular but uh if you just think for a minute as a student or as a kid most of the times

you are familiar with Windows operating system but once you move to your software Journey you see that Linux is

used everywhere whether uh in your production system staging developer environments everything is deployed all

the applications are deployed and tested on your Linux operating system I agree that there are windows as well like but

90 to 80 of the times you test on Linux operating system and you also Deploy on Linux operating system

let's try to understand why okay so there are very good reasons why Linux

is very popular firstly first and foremost reason is Linux is a free operating system okay unlike Windows

Windows is a proprietary operating system you must know that uh it's appropriating appropriate operating

system provided by Windows right sorry Microsoft whereas Linux is a freeware or it's a open source software

so today uh anybody can create Unix like operating systems that is Linux like operating systems

and again uh one of the key reasons is Linux is very secure for example you uh have a Linux laptop

and you have a Windows laptop you don't even have to install any kind of antiviruses

for on your Windows laptop you must have installed my cafe or any other things like

antivirus softwares right anti-malware softwares but whereas you win Linux you don't have to install any kind of

antivirus It Is by default very very secure I'm not saying that it is 100 secure but most of the times your Linux

operating system is a secure software and it contain a lot of distributions like you know uh today you have uh many

distributions like uh Centos you have distributions like Ubuntu you have distributions like uh red hat and lot of

other things right but if you talk about the free distributions you have popular ones like Ubuntu sentos uh people also

use Alpine Debian so different kinds of distributions are provided so you can depending upon your comfort you can use

any uh distributions and finally one of the most important things for any application to run on your production

systems is it has to be very fast if your operating system is slow because your operating system is hard of

everything like whenever a user is trying to send a request let's say you are trying to access amazon.com you are

trying to access Netflix so if your operating system itself is very slow then it doesn't matter right whether

your application you might have used Advanced concept like multi-threading or any other things in your application but

if your operating system is not capable of taking this requests then it's of no use so that's why people always prefer

fast running operating systems or the operating systems which are not crashed or which are not slow in their

production systems that's one of the key reasons why Linux is used as a operating system in production systems so it is

very fast it is very secure and it is also free in nature so what are the things that you want right when

something comes for free when something is very secure and fast then that's a bliss right that's a blessing

so that's why people use Linux operating system over windows okay so if somebody asks you what is the

difference between Linux and windows so this is something that you answer or why Linux is used Linux is most widely used

when compared to other operating system these are some of the reasons now let's try to take a couple of

minutes to understand the architecture of it okay so I'll not bore you or I'll not go into the details of the Linux

Operator Operator architecture because it's very complicated and it's not even for me to explain the entire Linux

operating system even if I take one two hours it's not easy to understand or easy to explain so that's why what I'll

do is I'll try to keep it as crisp as possible or as simple as possible so let's say

let's try to break break it down to make it easy okay so let's say this is your operating

system OS so firstly the heart of your Linux

operating system is your kernel okay and on top of your kernel you have something called as your system

libraries don't worry I'll explain them in detail and on top of it you have some of the

things like you have your compilers you have your uh user processes and you have some

system related softwares even for your windows you have some system related softwares right so

similarly even for Linux you have some system related software so whenever you are installing Linux let's say you

install Linux on your laptop so these are the things that you are installing okay so because these are the

fundamentals of your operating system so firstly you need to understand what is a kernel so kernel is heart of your

Linux operating system why because the responsibility of Kernel is to establish a communication between your

hardware and your software like I mentioned so there is a user okay the user is installing an

application which is a software and then there is a hardware that is nothing but your CPU

Ram IO so operating system is responsible for basically

cascading your request that is from your software to Hardware so who takes the responsibility like there should be a

component in your operating system which is taking this responsibility right so that component is nothing but your

kernel so that's why kernel is again here heart of your operating system so there are some primary responsibilities

of Kernel that you have to understand that is for CPU it does okay let's try to put it in this way there are four

important aspects of your kernel even if somebody is asking in an interview what is a kernel in an operating system you

can just say that kernel has four primary responsibilities it's a heart of any operating system which does

device management okay kernel is responsible for memory management

then it is responsible for your Process Management and finally it is responsible for handling your system calls

okay for any operating system these are the key aspects and that are handled by your kernel one is device management

memory management process management and handling the system related calls after that you have system libraries right uh

like as I mentioned you can also take example of Kernel modules and other things but let's try to keep it very

simple so after the kernel you have system libraries and what you have in system libraries so system libraries are

basically responsible for performing a task let's say user is trying to perform a task so it comes from your system

libraries and then it goes to your kernel so each operating system has their own

uh supported kind of these system libraries so let's say you are installing uh uh you know a Debian

distribution or you are installing a uh a different kind of distribution like your Cent Os or federal your system

libraries might differ slightly but again most of the times the concept of the system libraries is the same

if you want to take a uh example of a system Library you can try to understand like you have your Lipsy right so lip C

is one of the system Library just for your understanding so uh there are other system libraries like the graphical gnu

or the other things but okay after that you have the final stage where you have compilers user process

and system software so compilers coming to the compiler so uh like let's say you want to run Java application or Java or

you want to run python or you want to run any other uh kind of your software so your operating system has to compile

this code right your application or these kind of things so for that reason you have compilers user processes and

some kind of system softwares even if you are using a Windows laptop you are aware of your system softwares that you

have so these are the fundamentals of any operating system like uh definitely I'm

not going to the details of each of these things because even for me to explain like even when I'm trying to

explain these things I'm trying to keep it as crisp as possible that's why I'm taking some time while explaining it

because I don't want to complicate these things and explaining these things in like 30 to 40 minutes of session is

technically not possible but you have to understand the breadth of the concepts like even in this 40 to 45

days of devops Journey if we spend two to three days in just understanding the kernel or if you spend two to three days

in understanding the operating system that will be of no use you have to understand the breadth of

the knowledge of all the different kinds of Concepts that are available in devops or most widely used in devops and once

you understand the breadth of it you can start giving your interviews and during your interviews you can go into the

depth of each and every tool for example if I'm explaining you about the configuration management you have to

understand about configuration management but you don't have to spend uh 10 days 15 days in just understanding

different tools that are available in configuration management like don't learn ansible puppet itself and

everything just learn one thing start with your interviews and then once you are comfortable with your devops journey

then probably you can understand okay what is puppet why puppet is different from ansible for now just restrict to

one thing so even today I'm just explaining you the Core Concepts of operating system

but not going into the details of Kernel not going to the details of system libraries because that will be very

complicating now so what did we learn till now we

understood what is a operating system after that we try to understand what is the definition of an operating system uh

why operating system is useful basically it is useful for establishing a communication between your hardware and

your software whenever a user sends a request it has to go from your software to your operating system to your

Hardware finally the response is also received back in the same way and then we try to see why Linux is a very

popular operating system and why windows or other operating systems are not used in your production systems because of

the free secure and the fast nature of your Linux operating system and finally we also try to understand

the architecture of your Linux operating system so now let's move ahead and try to understand the fundamentals of your

shell scripting okay so today we will not go into the details of shell scripting but we will only take

a look at the fundamentals of shell scripting so because you understood what is Linux

operating system why you have to use something like a shell scripting

basically what is shell shell is a way that you talk to your operating system let's say you want to create a file

what do you do in your windows so Windows has a very good graphical user interface but most of the times when you

are working in your software organizations you don't have a graphical user interface for your servers so if

you have issue on your production server you don't have a graphical user interface for you to navigate through

the folders to understand what is issue so you have to do it through some uh command line way okay so this command

line way through which you talk to your operating system are nothing but your shell commands

okay so in Linux the way that you talk to your Operating System is using your shell commands so shell is most of the

times like 99.9 percent of them your uh servers that you're using on your production or your Dev or staging

environments default come with the shell environment so you can if you are aware of the shell commands you can directly

talk to your Linux operating system and these commands are common across different distributions whether you are

using uh Centos whether you are using uh fed Arrow or whether you are using a Debian or Ubuntu so these uh shell

commands that we are going to talk or which we use as a devops engineer are mostly common across different

distributions and different uh yeah just the different distributions not operating systems

so that's why today we'll try to see what are some of the popular shell commands so again just to give you an

example if you want to create a file on your windows what you would do is you would navigate through your graphical

user interface that is you go to your laptop making use of your mouse or cursor you navigate through the

different folders but online Linux operating system that's not possible because most of the times your graphical

user interface GUI is not installed because it comes with some weight right if you want to install graphical user

interface again your operating system or your servers would become heavy so that's why on your production systems we

don't use graphical interface and most of the times we make use of the shell commands to talk to the

distribution or to create files or to create anything in your operating system so let me just stop sharing and in the

previous class we uh already understood how to create a virtual machine on your AWS right whether through the command

line or whether through the AWS CLI we already talked how to create This Server so now let me make use of one of the

server that I created in the previous class and show you some of the basic commands of shell scripting

so I'll stop sharing here and share my screen with the stop sharing and let me share my

terminal so you can use any terminal whether you want to use putty like I explained in the previous classes

whether you want to use mobilex term or any other thing that's totally fine you can use anything

yep so let me share my screen again here

Okay cool so what I've done is I am uh for now I'm using a Mac laptop but I've installed bash in it by default it comes

with SH so you can use zsh or you can use bash so I've installed bash like most of the times recommend bash is one

of the popular ones and most widely used so instead of learning ksh instead of learning essay just learn bash and you

can use this scripting even once you start your software journey and even in a devops Zero to Hero devops Journey

we'll be learning about bash only so I have like I mentioned I have uh this one here

the just give me a second uh for me to pull the

okay I got hold of my IP address of the newly created AWS instance in the previous class so this is the command

I'll use to login so if you just want to understand one more time so this is the key value pair that we just created I

mean in the previous class and using the identity file that is the key value pair what we do is we make use of the public

IP of the ec2 instance to log into the instance so we created a Ubuntu related image uh in the last class so that's why

the user that comes by default is something that I already know that is the Ubuntu so that's why I use the

command SSH minus I identity file Ubuntu at the rate public IP address perfect so this is the server it has

Linux operating system installed that that is nothing but your Ubuntu distribution so even in Linux operating

system you have different distributions uh most of the times we use Ubuntu or Centos

distributions are nothing but the vendors that are providing this Linux operating system right so red hat is a

vendor that provides Linux operating system uh using Red Hat uh Centos is one fedoro is one Ubuntu Debian different

vendors provide Linux operating system you can make use of anything Ubuntu is most widely used

so what is the command first of all you got into this one but you don't know how to navigate from one place to another

place right if it is Windows what you would do you directly make use of your mouse you would see what are the files

and folders that are present but on your Linux platform if you want to see what are the different files and folders that

are present here the command that you make use is LS so by using LS you try to understand what are the files and

folders LS is nothing but list okay so using LS command you list the files and folders

but if you see here it is showing the files and folders in this particular directory right but I want to see the

files and folders everywhere but for that I need to understand where I am currently

again if you are on Windows what you would do you would right click and see what is your current location but

everything in Linux has to be done through some commands for the for that what is the command that you use is PWD

so PWD stands for present working directory so always once you start logging into your server firstly make

use of the PWD command to understand what is your present working directory so here it says that I am in the slash

home slash Ubuntu working directory now I can change this present working directory by using the CD command so

using CD that is change directory okay so again uh always try to compare with the operations that you are doing on

your Windows machine if you are very used to just to understand how to

correlate right you know if you are for example uh you want to move from uh C directory to the D directory in your

windows what you would do you will make use of the cursor and click on the back you do back click and then go to the

directory but in Linux it's not possible because there is no graphical user interface most of the times so that's

why firstly you would do PWD to understand where you are currently and using LS you understand what are the

files and folders that are available in the directory and let's say you want to move from slash Ubuntu to bundle

directory so for that what you will do is you will make use of the CD command CD can CD stands for change directory CD

bundle and once you do CD bundle again if you do PWD you can clearly see that you are

present working directory has changed so you now we are present where working directory is slash home slash Ubuntu

slash bundle okay so just to repeat always once you log in to your uh Linux software or Linux system sorry Linux

system what you would do is firstly you execute the command called PWD to understand what is your present working

directory and once you understand your present working directory you need to do the ls command to list the files and

directories okay so inside this present working directory I have few files or folders as well

so for me to understand or to just uh see like let's say uh somebody asked you what are the files and folders that are

present in bundle directory so you make use of the ls command to list the files as well as folders once you list these

files and folders let's say I want to move to a directory called bundle so what I'll do is I'll make use of the CD

command to change the directory okay so using CD I'll go to the bundle folder which I am currently already in

or let's say there is a test folder so what I can do CD test once doing it again if I click on PWD you'll see the

present working directory has changed if you want to go back okay like let's say you came here and now you just want to

go back instead of giving the complete path what you can simply do is you can say CD space dot dot so what happens

when you do CD space dot dot is you go back to One Directory let's say you want to go back to two directories okay so

what you say is CD dot dot slash dot dot by doing that now you are in just the home directory and if you want to go to

two directories more or you know two directories in depth what you can do is CD

Ubuntu slash bundle okay so this is how you move from one directory to another

director and if you want to do uh with multiple like you want to go back to multiple directories or go ahead of

multiple directories you can make use of this slash operator perfect so just in uh one minute of time

we learned about three different commands that is PWD CD LS so we learned about these things right now let's say I

want to list all the files and folders with timestamp who is owner of the file uh and other information like is it a

file or a directory so you can make use of this one simple command that is LS hyphen LTR

okay LS minus LTR so what does this LS minus LTR does is it provides you uh it provides you the information like if it

is starting with d that means that it is a directory if it is not starting with t that means that it is a Phi so if you

see here this is a directory this is a directory this is a directory metadata is a directory uh like you know manifest

is a directory bundle is a directory and if it doesn't start with the D that means that it's a file after that you

have information of who is the owner of the file which group owns this file you know uh what is the content like what is

the size of the contents inside this file and after that you also have information like you know when is this

file created the timestamp of it and what is the name of the file or the folder so if you want to just list the

files and folders you can use LS command but LS supports Advanced that is using LS minus LTR you can list the files and

folders with its properties that is who is the owner who is the group owner or what is it a file and a folder and you

also get the information that what are the permissions of it and what is the size of it and the

timestamp using the ls minus LTR so these are some of the basics right and after this you might want to create a

file okay so to create a file you can simply use the touch command so using touch command you can create a file but

like let's say touch space Abhishek so now what happens is a file called wstick is created and to see if this file is

created or not you can use again LS command and you'll see a new file called Abhishek

okay but if you just want to create a file touch is fine most of the times you want

to create a file and you want to write contents inside this file so for that we use the vi command so using VI you can

not only create a file but you can create a file and write inside the file so VI space let's say test

so a file called see here a new file is being created with name bi uh sorry with name test and you can also start writing

into the file but to write content inside a Linux file okay or file in a Linux it's not straightforward because

you know when you open a file Linux will ask you do you want to read the file do you want to write content into the file

so using Escape okay always once you enter a file click on escape and tell Linux that you know I want to write

content inside the file so click on I by clicking on I you will move to the insert mode and then you can start

writing into the file hi I am Abhishek okay and using again Escape you have to use this called colon

WQ to save the file Okay so now we wrote contents inside this file and using cat you can

basically print the file now if you feel that I am moving too fast don't worry I have already created a very good video

that covers all the basics of shell scripting I'll put the link in the description don't worry uh it's almost

like you know 90 minutes of video or sorry 80 minutes of video where we covered all the fundamentals of shell

and again we created one more video that is of one hour to cover some intermediate Concepts on shell so

totally we have two end of ours content on my channel that only talks about this cell commands uh how to write some

simple shell scripts so if you haven't followed any of these things right now don't worry I'll put the links in the

description you can watch both of those videos where I covered uh all most of the concepts of shell commands like you

know from Basics to the intermediate level of shell command so also we did a shell scripting interview questions

video so watch those videos and you know you'll get very familiar clear with the shell command shell Concepts and

tomorrow's class we'll talk about some Advanced topics where you know we'll take a look at one

real-time devops project so you might be learning all of these shell commands but if you don't

understand where are these practically used so that will be a problem right so that's why what I'll do is I'll make a

video where I'll talk about a real-time devops project using shell scripting uh why shell scripting is used in an

organization just for one example and with that you know you can understand okay so this is a real time use case of

shell scripting in devops probably you can enhance or you can think of other Alternatives and you can create your own

projects and you can put them on your resumes perfect so uh this is something and then

uh you know you have uh some other commands like for creating directories you have a command called mkdir using

mkdr you can create directory and for removing the files you have command like RM okay so using RM you can remove a

file and using RM minus r you can remove a directory so these are some of the commands that we usually use as part of

day to day life okay so it says folder already exists uh let's say I do mkdar and call it something called as

abhishek1 so it creates a directory called Abhishek

Abhishek and then if I want to remove the directory that I just created I can use RM minus r Abhishek one

so these are some of the popular commands that people use it in day to day basis but apart from this so this is

about creating a file writing a file reading a file and all of these things right but let's say uh you want to log

into a uh Linux machine and you want to check the CPU memory or the performance of your

machine completely which is very important right so let's say that you are using a Windows machine

and in your Windows machine all of a sudden you realize that your Windows machine uh all the CPUs used or Ram is

used so how do you check uh basically you go to your system properties and you will see what is the Ram size and then

you will go to task manager and you will see how much RAM is used you will compare both of these things right so in

your system properties you understood that your uh your laptop is of 8GB RAM and in your task manager you realize

that okay 7gb is already used so similarly let's say you want to do the same uh things in your

Linux so for that you again you do it only through your shell commands Okay so what are the sale commands that you will

use here is for looking into the memory you use a command called free so using free uh for

better readability what you can do is you can do 3 minus mg

sorry so using free minus D what you can do is you can understand what is the memory of

your laptop that is or your server and using uh n proc using nproc you can understand uh what are the CPUs uh

number of CPUs that are because this is a basic ec2 instance that I just created right so it says that it only comes with

one CPU we know but because we are using a free ec2 instance right so that's why it just has one CPU now and finally you

want to understand what is the disk size first of all that you have a command called DF minus H so using DF minus H

you can understand what is the disk size so 24 24 percentage I have already used so it comes with 5.8 GB and 1.9 GB I

already used so just like you know you sorry 7.6 DB is the total size uh 1.9 GB is

something that I used and 5.8 is available so 24 percentage of this is something that I used if you want to see

all of these things at one place there is a very popular command that is called top so using top you can get the

complete information so it's like a memory CPU and disk everything like you can monitor everything from one place

that is using top so it says like you know what is the CPU percentage that is used what is the memory percentage that

is used so this is a very popular interview question that is what is uh the command that you will use to manage

your memory CPU or look into the details of it so the answer is using the top command and individually if you want to

do you can use for memory you can use the free Command free is the command name for CPU you can use a command

called n proc okay and for this you can use a command called DF

so all of these commands if you want to look into one place that is using the top command so these are some of the

popular shell commands that you use a day-to-day basis we looked into creating a file creating a folder removing files

writing into a file reading a file uh and a listing file how to look into the permissions of a file right and finally

we also looked into uh some of the processing or management related commands that is DF to look into this

file size and memory using free Command and CPU using the in Pro command and for the whole management we can use the top

command so we looked into all of these things right and if you have still more questions or if you are eager to learn

many more shell commands so like I mentioned I created three videos on my channel and there is also playlist

called Shell Shell scripting I guess so I'll put all of the links in the description so once you watch these

videos uh at least watch the intermediate level video where I covered a lot of shell commands before we move

to the tomorrow's topic because in tomorrow we will see a real time shell scripting devops project

perfect I hope you enjoyed the video uh where we understood the concepts of operating system basics of cell shipping

and everything if you feel that this video was useful click on the like button if you have any feedback for me

whether it's positive or negative put that feedback in the comment section so I'll make sure uh to take your feedback

and improve in the future videos and finally if somebody uh is willing to learn devops and is not a subscriber or

a viewer of our Channel please try to share this video with your friends and family

and if you haven't subscribed uh please subscribe to my channel I'll see in the next video take care