Shell script:
It is program take commands from user gives to OS
Interface between user and OS
Unix Architecture
at /etc/shells → file will show all support shells like
C
Install – sudo yum install zsh -y
Find which shell is running → echo $SHELL
Sh, → it will available in old systems default shell oldos
Bash →bourne Again shell default shell latest os
Zsh → Z shell
Ksh, → kern shell default shell AIX
csh → C shell
Extensions → sh -shell script, .py –python, .java –java
HEL/Centos/Fedora → use yum command
R
ubuntu/Debian → apt-get or apt
#!/bin/bash → all commands run under bash script
hmod 700 sh.sh file → ./sh.sh or bash sh.sh
C
# → single line comment
a=12 echo $a → print a value ********************variables don't give spaces
LInux :
)Linux intro:
1
Most used linux distros currently
-RPM based: RHEL & Centos (paid version and downloaded .rpm)
-Debian based:ubuntu server & linux mint (free avaliable and downloaded .deb )
OME important Directories:
S
Home Directories :-- /root, /home/username
User Executables: /bin, /usr/bin, /usr/local/bin
System Executable: /sbin, /usr/sbin, /usr/local/sbin
Other Mount Points: /media, /mnt
Configuration: /etc
Temporary Files: /tmp
Kernels and Bootloader : /boot
Server Data: /var, /src
System Information: /proc, /sys
Shared Libraries: /lib, /user/lib, /usr/local/lib
) commands and File systems
2
Whoami — username or root directory
Pwd —-it will show path
Ls – list files or folders
Cat /etc/os-release —- show version and os name –find out current logged in username
im — go to file
V
Default text editor in ubuntu is nano
3)Filter & IO redirection command: — go to file
)software management
4
1.Curl —--(predefined curl in linux and downloading and moving to file_name.rpm )
curl <link.rpm> -o file_name.rpm
—->(install ) rpm -ivh file_name.rpm
.yum install package and its dependencies for link or other like wget (like curl)--
2
apt command in ubuntu and yum in centos to manage packages
)Services
5
Systemctl —> systemctl status httpd
Systemctl start httpd
Systemctl restart httpd
Systemctl reload httpd
Systemctl enable httpd
Systemctl is-enabled httpd How to check if a service is enabled
String
cho $variable or ${variable} → to print the value
E
Echo ${#variable} → variable length
Echo ${variable:20} → print last 20 characters
Operators:
xpr 2 + 3
e
expr 2 \* 3
expr 10 / 2
expr 20 % 2
User interaction read command:
r ead username → ask username from user and access as $username
read -a users → list of array give a b c like [a,b,c] access ${users[*]} → all values
read -p “enter user name ” username → echo $username
read -p “enter the password” password → echo $password
Ifelse conditions:
- gt → >
-ge → >=
-le → <
-ne → <=
-eq → ==
file.sh
If [ $a -gt $b ]
then
------
—
else
—----
fi
Search the file:
r ead username
If [ -f $username ] → search in current directory
If [ -f /tmp/$username ] → search in the specified dir
then
—---------
else
—---------
fi
For loop:
for (( inst; condition; inc/dec ))
do
—---------
done
x:
E
for (( a=1; i<=10; i++ ))
do
echo $a
done → print 1 to 1o
Functions:
resk(){
g
echo “hello”
}
gresk —> calling function in shell script ****************fun call after writing only
File Management
. ls → list of directories
1
2. ls -l → list dir with file permissions and storage all details
1)Hidden Files
. ls -a → hidden files and folders
1
2)Creating Files
i file.txt
V
Shell script:
1. vi text.sh
!/bin/bash
#
echo “hello vi text.sh”
2. chmod 777 text.sh ----> give permision to text.sh
3. ./text.sh or bash text.sh or sh text.sh --->to run the vi file
insert mode --->enter into
i ---->enter
I ---->enter + curser start of line
a ---->enter + 2nd word
A --->enter + end of the line
command mode ---> esc to change
exit mode ---> esc to change
1. :wq! ---->w-->write the content save,q ---->quite
!----> force to quite
2. :e! ----->unchange to file
3. shift + o ---->new line in the above the curser
4. :sh --->switch to shell (enter ctr + shift +d -->back to vi )
5. :set nu --->numbers to line
6. :set nonu -----> remove number to line
7. vi editer:
u -->undo last action
U -->undo all last actions :e!
$ -->end of the line
Search:
1. / or ? To search
/chaitanya
?chaitanya ----->to search chaitanya (for next word n forword or N backword)
delete:
1. x ---> to delete from right one alpha
2. dw --> remove the word
3. dd ---> remove line
4. del ---->del button
Remove lines:
1. dG ---> curser(remove curser to end of file)
. d
2 lG ---->all above content remove above curser
3. : 10,15 d---> delete lines from 10 to 15
Move:
shift +g --->curser move to last of line
l+shift+g --->curser move to first of line
30+shift+g --->curser move to 30th of line
Screen Move:
shift +h --->curser move top of screen
shift + m ---->curser move middle of screen
shift +l ---->bottom of curser
Copy:
YY ---> copy
5YY --->copy 5 lines
p ---> paste
:1,5 co 9 ----copy to
:1,5 m 9 ---->move to
3)Display Content of a File
. c
1 at file.txt → read all lines
2. cat -b file.txt → with numbers all lines
4)Counting Words in a File
1. w
c text.file---> list [number lines,words,alpha]
ls |wc -l -----> to show how many files or fold in dir
ls | wc -w —-> to show how many words in dir
ls |wc -c —-> to show how many characters in dir
5)Copying Files
1. cp file.txt file.txt
6)Renaming Files
1. mv file.txt file.txt
7)Deleting Files
1. rm file1.txt file2.txt
8)File descriptions:
→ std i/p
0
1 → std o/p
2 → std error
Input and output:
) 1.sh > 2.sh copy content from 1 to 2 **********************error not go (error command not
1
found)
If you want error also then 1.sh > 2.sh 2>&1
2>&1 → error and std o/p go inside
Ex: Sh file.sh 2> error.log 1> output.log
) 1.sh >> 2.sh add file content to file 2 at end
2
3) cat < output.log → input simbol like cat only
Directory Management
cd
1. cd ~ or cd -----> directory to home
2. cd - -----> back dir
3. cd .. -----> one step back to dir
4. cd “chaita kiran” ---->to enter space dir
5. cd /home/chaitanya2/inside --->change path from home or cd inside --->change
also
1)Listing Directories
. ls dir
1
2. ls /dir1/dir2
2)Creating Directories
1. mkdir worker
3)Creating Parent Directories
1. mkdir -p worker
4)Removing Directories
1. rmdir worker1 worker2
5)Renaming Directories
1. mv worker1 worker2
File Permission / Access Modes
File permissions:
rwxr-xr-x ----->for new dir(chaitanya)755
d
-rw-r--r-- ----->for new file(chaitanya1)644
1. r-->4
w --->2
x---->1
rwx(user)rwx(group)rwx(others)---->permisions
chmod 777(user7,gp7,otrs7) text.file -->per for file or fold
2. chmod g+r u+w o-r text.file ----->give reading to group,per to write to
user,per to remove reading others
3. chmod 777 or -m=rwx text.file ---->full permisions
4. chmod u=rwx,g=rwx,o=rw text.file ---->full permisions
5. chown owner:group file.txt --->change owner and group name(new only
exited owner and group name)
chown chaitanya:user file.txt
Umask:
umaskchanges the default permissions and thus the
ermissions for all newly created files and folders, while chmodsets permissions for
p
files and folders that already exist.
---The system default permission values are 777 (rwxrwxrwx)for folders and 666
(rw-rw-rw-) for files.
- ----The default mask for a non-root user is 002, changing the folder permissions to 775
(rwxrwxr-x), and file permissions to 664 (rw-rw-r--).
- -----The default mask for a root user us 022, changing the folder permissions to 755
(rwxr-xr-x), and file permissions to 644 (rw-r--r--).
( root user—>022,non-root user -->002)
umask 022 or umask u=rwx,g=rx,u=rx
Pipes and Filters
Filter commands:
.
1 less text.file --->adject width and height for content
2. more text.file --->remaining 55%
3. head text.file -->show 1st 5 linesd
4. head -15 text.file ---> show 1st 15 lines
5. tail text.file --->show last 5 lines
6. tail -10 text.file --->show last 10 lines
7. touch text{1..100}.txt
8. wc text.file---> list [number lines,words,alpha]
ls |wc -l -----> to show how many files or fold in dir
ls | wc -w —-> to show how many words in dir
ls |wc -c —-> to show how many characters in dir
Search:
0.curl localhost:8080/80 --->to check content running on 80port
1. ls |grep -i kiran ---->search kiran in ls, -i --->case sensitive
ifconfig |grep -i inet addr ---> to show net address
2. only for files and folders
find chaitanya or locate chaitanya ----> to search files and fold
Processes Management
Listing Running Processes
. ps -f → show list all process
1
2. ps -ef |grep tomcat to check the tomcat running on which port
Stopping Processes
1. kill -9 6751(PID)
The top Command
1. top →it will show allphysical and virtual memory,CPU usage, load averages,
and your busy processes.
rocesses
p
–Top —-to see running processes
–Htop – to running process and ram status
–Ps -ef | grep httpd —-- to see the running services(all the process with their
parent process id) ps -aux — normal
–kill -9(force) PID command is to stop process forcefully
–kill PID is to stop process gracefully, child processes also will be stopped if
parent process is stopped gracefully.
Network Communication Utilities
ing <IP1> → connect b/w running server and IP1
p
telnet <IP> 22 → server running on this port or not
curl -v telnet://<IP>:22 → server running on this port or not
1. ping →ping command sends anecho request to a hostavailableon the
etwork.
n
EX:ping hostname or ip-address (ping google.com)
ftp Utility
1. T he simplest file transfer protocol to exchange filesto and from a remote
computer or network.
EX: ftp hostname(192.168.92.68)
telnet utility
1. Used to check the port open or not
EX: telnet 192.168.75.68: 8000
Shell scripting
---above regards shells
—
#!/bin/sh
cho"What is your name?"
e
read PERSON ### reads the input from the keyboard
echo"Hello, $PERSON"
Running ::
1. ./file.sh
2. sh file.sh
hell Types
S
In Unix, there are two major types of shells −
ourne shell− If you are using a Bourne-type shell,the $ character is the default
B
prompt.
C shell− If you are using a C-type shell, the % characteris the default prompt.
The Bourne Shell has the following subcategories−
ourne shell (sh)
B
Korn shell (ksh)
Bourne Again shell (bash)
POSIX shell (sh)
The different C-type shells follow −
shell (csh)
C
TENEX/TOPS C shell (tcsh)
Variable Names
The name of a variable can contain only letters (a to z or A to Z), numbers ( 0 to 9) or
the underscore character ( _)
EX :
_ALI
TOKEN_A
VAR_1
VAR_2
Not variable:
2_VAR
-VARIABLE
VAR1-VAR2
VAR_A!
Special Variables
alues passing through the arguments
V
Sh file.sh arg1 arg2
$1 →1st argument
$2 →2nd argument or $11
$# →number arguments like 2
$$ →PID number 605670
$? → previous command execution status ******************************IMP
File.sh and if then else if
Java -version
If [ $? -eq 0 ]
then
echo “java installed”
else
echo “java not installed”
Fi
X:
E
#!/bin/sh
echo"File Name: $0"
cho"First Parameter : $1"
e
echo"Second Parameter : $2"
echo"Quoted Values: $@"
echo"Quoted Values: $*"
echo"Total Number of Parameters : $#"
utput:
O
$./test.sh Zara Ali
File Name : ./test.sh
First Parameter : Zara
Second Parameter : Ali
Quoted Values: Zara Ali
Quoted Values: Zara Ali
Total Number of Parameters : 2
Shell Arrays
x:
E
#!/bin/sh
AME[0
N ]="Zara"
NAME[1 ]="Qadir"
NAME[2 ]="Mahnaz"
NAME[3 ]="Ayan"
NAME[4 ]="Daisy"
echo"First Index: ${NAME[0]}"
echo"First Method: ${NAME[*]}"
echo"Second Method: ${NAME[@]}"
utput :
O
$./test.sh
First Index: Zara
First Method: Zara Qadir Mahnaz Ayan Daisy
Second Method: Zara Qadir Mahnaz Ayan Daisy
Shell Basic Operators
1. Arithmetic Operators
2. Relational Operators
3. Boolean Operators
4. String Operators
5. File Test Operators
Note Url :https://www.tutorialspoint.com/unix/unix-basic-operators.htm
Relational Operators
Assume variable a holds 10 and variable b holds 20
specific to numeric values. These operators do not work for string values
unless their value is numeric.
Operator Description Example
-eq Checks if the value of two operands [ $a -eq $b ]
are equal or not; is not true.
-ne Checks if the value of two operands [ $a -ne $b ]
are equal or not; is true.
-gt Checks if the value of left operand is [ $a -gt $b ]
greater than the value of right is not true.
operand;
-lt Checks if the value of left operand is [ $a -lt $b ]
less than the value of right operand;. is true.
-ge Checks if the value of left operand is
[ $a -ge $b ]
greater than or equal to the value of
is not true.
right operand;
-le Checks if the value of left operand is
[ $a -le $b ]
less than or equal to the value of
is true
right operand;
Boolean Operators
Operator Description Example
! This is logical negation. This inverts a true condition
[ ! false ] is true.
into false and vice versa.
-o This is logical OR. If one of the operands is true, then [ $a -lt 20 -o $b -gt
the condition becomes true. 100 ] is true.
-a This is logical AND. If both the operands are true, then [ $a -lt 20 -a $b -gt
the condition becomes true otherwise false. 100 ] is false.
String Operators
Operator Description Example
= Checks if the value of two operands are equal or [ $a = $b ] is not true.
not; if yes, then the condition becomes true.
!= Checks if the value of two operands are equal or [ $a != $b ] is true.
not; if values are not equal then the condition
becomes true.
-z Checks if the given string operand size is zero; if it [ -z $a ] is not true.
is zero length, then it returns true.
-n Checks if the given string operand size is non-zero; [ -n $a ] is not false.
if it is nonzero length, then it returns true.
str Checks if str is not the empty string; if it is empty, [ $a ] is not false.
then it returns false.
File Test Operators
file holds an existing file name "test" the size of which is 100 bytes and has read, write
and execute permission on −
Shell Loop Control
1)While loop →loop executes the given commands untilthe given condition remains
true
#!/bin/sh
=0
a
while[$a-lt10]
do
echo $a
a=
̀ expr $a + 1`
done ##output ## 0,1,2,3,4,5,6,7,8,9
2) until loop →until loop executes until a givencondition becomes true.
#!/bin/sh
=0
a
until[!$a-lt10]
do
echo $a
a=
̀ expr $a + 1`
done ##output ## 0,1,2,3,4,5,6,7,8,9
)for loop
3
#!/bin/sh
forvarin0123456789
do
echo $var
done ############0,1,2,3,4,5,6,7,8,9
) select loop
4
select var in word1 word2 ... wordN
do
Statement(s) to be executed for every word.
done
Loop Control
In this chapter, we will learn following two statements thatareusedtocontrolshell
loops−
1)The break statement
#!/bin/sh
=0
a
while[$a-lt10]
do
echo $a
if[$a-eq5]
then
break
fi
a=
̀ expr $a + 1`
done ####### 01,2,3,4,5
Ex: 2 using for
#!/bin/sh
forvar1in123
do
forvar2in05
do
if[$var1-e
q2-a
$var2-e
q0]
then
break2
else
echo"$var1 $var2"
fi
done
done #####
Output:
1 0
1 5
2)The continue statement
#!/bin/sh
UMS="1 2 3 4 5 6 7"
N
forNUMin$NUMS
do
Q=̀ expr $NUM % 2`
if[$Q-eq0]
then
echo"Number is an even number!!"
continue
fi
echo"Found odd number"
done
utput:
O
Found odd number
Number is an even number!!
Found odd number
Number is an even number!!
Found odd number
Number is an even number!!
Found odd number
CRONTAB:
1. vim abc.sh --->u(rw),g(r),o(r)
touch /home/chaitanya/Desktop/created.txt --->in sh file
2. chmod 700 abc.sh or chmod u+x abc.sh or chmod 777 abc.sh
3. crontab -l --->to check content in the crontab
4. crontab -e ---->to set the time
57 10 * * * /bin/bash /home/chaitanya/Desktop/adc.sh
57 --->minutes(0-59)
10 --->hours (1-24)
* ---->day in month(1-31)
* ----->month in year (1-12)
* ---->day in week(0-6)(sunday=0)
5. tail -f /var/log/cron.log ortail -f /var/log/syslog|grep cron
very day more than 7 days data to move another
E
folder and 14 + data removed (7 to 14 data stored
Zip )
-------------------------------------------------------------------------------------------------------------------
!/bin/sh
#
NOW="$(date +'%d-%m-%Y')"
###create the folder using today date
#
echo "$NOW"
###### creating dir /var/log/backup/"$NOW" using today date
mkdir -p /var/log/backup/"$NOW"
###### move all files more than 7 days to /var/log/backup/"$NOW"
find /var/log/nginx/ -maxdepth 1 -mtime +7 -exec mv "{}" /var/log/backup/"$NOW"/ \;
####### remove all files from older than 14 days and remaining 7 to 14 days store in
/var/log/backup/"$NOW"
find /var/log/backup/"$NOW"/ * -mtime +14 -exec rm {} \;
#######tar folder using /var/log/backup/"$NOW"
tar fczP /var/log/backup/"$NOW".tar.gz /var/log/backup/"$NOW"
######delete the folder /var/log/backup/"$NOW"
rm -rf /var/log/backup/"$NOW"
-maxdepth 1 → select the files/folders search for object
Note::: crontab
hmod +x /opt/script/delete-old-files.sh
c
crontab -e
0 7 * * * /bin/bash /opt/script/delete-old-folders.sh
MongoDB_Backup :
Backup.sh
Backup.sh
#!/bin/bash
Variables
#
CONTAINER_ID=`docker ps -q --filter="name=pg_postgres_postgres.1*"`
BACKUP_FOLDER=./backup/pg
TIMESTAMP=`date +%F-%H%M̀
cho 'Variables-------'
e
echo 'CONTAINER_ID='${CONTAINER_ID}
echo 'BACKUP_FOLDER='${BACKUP_FOLDER}
Clean up
#
echo 'Running cleanup-------'
rm -rf ${BACKUP_FOLDER}
mkdir -p ${BACKUP_FOLDER}
Backup commands
#
echo 'Generating backup-------'
for db in $(docker exec -i ${CONTAINER_ID} psql -U postgres -t -c 'SELECT dataname FROM
pg_database WHERE NOT datistemplate')
do
echo $db
docker exec -i ${CONTAINER_ID} pg_dump -U postgres --format=c ${db} >
${BACKUP_FOLDER}/${db}.sqlc
one
d
tar -zcvf $TIMESTAMP.tar.gz ./backup/pg
aws s3 sync $BACKUP_FOLDER s3://drogoback/$TIMESTAMP
rm -rf $TIMESTAMP.tar.gz
echo 'Done-------'
r
O
#send message to mailtrap box
STR=$(du -sh BACKUP_DIR)
echo "$TIMESTAMP DB backup size is $STR" | mail -s "DB daily backup alert"
d8e1a67424-278bbd@inbox.mailtrap.io
aws s3 sync $BACKUP_DIR s3://ttinfra/backups/mongo/csop-qa/$BACKUP_NAME
r m -rf /tmp2/backup/*
done
Note::: crontab
at 5 a.m every week with:
#
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h dom mon dow command
0 0 * * * bash /home/backup.sh
0 0 * * * sh /home/single_db.sh > /dev/null 2>&1
3
~
~
~
~
~
~
~
