Operating Systems

Lab No. 4
4.1.0- Lab objective
This lab will give overview of Linux shells. You will get insight of ‘bash’ shell.

4.1.1- Background
Shells
A shell provides an interface between the user and the operating
system kernel
Either a command interpreter or a graphical user interface
Traditional Unix shells are command-line interfaces (CLIs)
Usually started automatically when you log in or open a terminal

Page - 30 -
 Operating Systems

4.2.0- Work Effectively on the Linux Command Line

4.2.1 The Bash Shell

Linux’s most popular command interpreter is called bash
The Bourne-Again Shell
o More sophisticated than the original sh by Steve Bourne
o Can be run as sh, as a replacement for the original Unix shell
o Gives you a prompt and waits for a command to be entered
Although this course concentrates on Bash, the shell tcsh is also
popular
o Based on the design of the older C Shell (csh)

4.2.2 Shell Commands

Shell commands entered consist of words
o Separated by spaces (whitespace)
o The first word is the command to run
o Subsequent words are options or arguments to the command
For several reasons, some commands are built into the shell itself
o Called builtins
o Only a small number of commands are builtins, most are
separate programs

4.2.3 Command-Line Arguments

The words after the command name are passed to a command as
a list of arguments
Most commands group these words into two categories:
o Options, usually starting with one or two hyphens
o Filenames, directories, etc., on which to operate

Page - 31 -
 Operating Systems

The options usually come first, but for most commands they do not
need to
There is a special option ‘--’ which indicates the end of the options
o Nothing after the double hyphen is treated as an option,
even if it starts with -

4.2.4 Syntax of Command-Line Options

Most Unix commands have a consistent syntax for options:
o Single letter options start with a hyphen, e.g., -B
o Less cryptic options are whole words or phrases, and start with
two hyphens, for example
--ignore-backups
Some options themselves take arguments
o Usually the argument is the next word: sort -o output_file
A few programs use different styles of command-line options
o For example, long options (not single letters) sometimes start
with a single – rather than - -

4.2.5 Examples of Command-Line Options

List all the files in the current directory:
$ ls
List the files in the ‘long format’ (giving more information):
$ ls -l
List full information about some specific files:
$ ls -l notes.txt report.txt
List full information about all the .txt files:
$ ls -l *.txt
List all files in long format, even the hidden ones:
$ ls -l -a

Page - 32 -
 Operating Systems

$ ls -la
4.2.6 Setting Shell Variables
Shell variables can be used to store temporary values
Set a shell variable’s value as follows:
$ files="notes.txt report.txt"
o The double quotes are needed because the value contains a
space
o Easiest to put them in all the time
Print out the value of a shell variable with the echo command:
$ echo $files
o The dollar ($) tells the shell to insert the variable’s value into
the command line
Use the set command (with no arguments) to list all the shell
variables

4.2.7 Environment Variables

Shell variables are private to the shell
A special type of shell variables called environment variables are
passed to programs run from the shell
A program’s environment is the set of environment variables it can
access
o In Bash, use export to export a shell variable into the
environment:
$ files="notes.txt report.txt"
$ export files
o Or combine those into one line:
$ export files="notes.txt report.txt"
The env command lists environment variables

Page - 33 -
 Operating Systems

4.2.8 Where Programs are Found

The location of a program can be specified explicitly:
o ./sample runs the sample program in the current directory
o /bin/ls runs the ls command in the /bin directory
Otherwise, the shell looks in standard places for the program
o The variable called PATH lists the directories to search in
o Directory names are separated by colon, for example:
$ echo $PATH
/bin:/usr/bin:/usr/local/bin
o So running whoami will run /bin/whoami or /usr/bin/whoami
or /usr/local/bin/whoami (whichever is found first)

4.2.9 Bash Configuration Variables

Some variables contain information which Bash itself uses
o The variable called PS1 (Prompt String 1) specifies how to
display the shell prompt
Use the echo command with a $ sign before a varable name to see
its value, e.g.
$ echo $PS1
[\u@\h \W]\$
The special characters \u, \h and \W represent shell variables
containing, respectively, your user/login name, machine’s
hostname and current working directory, i.e.,
o $USER, $HOSTNAME, $PWD

Page - 34 -
 Operating Systems

4.2.10 Using History

Previously executed commands can be edited with the Up or Ctrl+P
keys
This allows old commands to be executed again without re-entering
Bash stores a history of old commands in memory
o Use the built-in command history to display the lines
remembered
o History is stored between sessions in the file ˜/.bash_history
Bash uses the readline library to read input from the user
o Allows Emacs-like editing of the command line
o Left and Right cursor keys and Delete work as expected

4.2.11 Reusing History Items

Previous commands can be used to build new commands, using
history expansion
Use !! to refer to the previous command, for example:
$ rm index.html
$ echo !!
echo rm index.html
rm index.html
More often useful is !string, which inserts the most recent command
which started with string
o Useful for repeating particular commands without
modification:
$ ls *.txt
notes.txt report.txt
$ !ls
ls *.txt
notes.txt report.txt

Page - 35 -
 Operating Systems

4.2.12 Retrieving Arguments from the History

The event designator !$ refers to the last argument of the previous
command:
$ ls -l long_file_name.html
-rw-r--r-- 1 jeff users 11170 Feb 20 10:47 long_file_name.html
$ rm !$
rm long_file_name.html
Similarly, !ˆ refers to the first argument
A command of the form ˆstringˆreplacementˆ replaces the first
occurrence of string with replacement in the previous command,
and runs it:
$ echo $HOTSNAME
$ ˆTSˆSTˆ
echo $HOSTNAME
tiger

4.2.13 Summary of Bash Editing Keys

These are the basic editing commands by default:
o Right — move cursor to the right
o Left — move cursor to the left
o Up — previous history line
o Down — next history line
o Ctrl+A — move to start of line
o Ctrl+E — move to end of line
o Ctrl+D — delete current character
There are alternative keys, as for the Emacs editor, which can be
more comfortable to use than the cursor keys
There are other, less often used keys, which are documented in the
bash man page (section ‘Readline’)

Page - 36 -
 Operating Systems

4.2.14 Combining Commands on One Line

You can write multiple commands on one line by separating them
with ;
Useful when the first command might take a long time:
time-consuming-program; ls
Alternatively, use && to arrange for subsequent commands to run
only if earlier ones succeeded:
time-consuming-potentially-failing-program && ls

4.2.15 Repeating Commands with ‘for’

Commands can be repeated several times using for
o Structure: for varname in list; do commands...; done
For example, to rename all .txt files to .txt.old :
$ for file in *.txt;
> do
> mv -v $file $file.old;
> done
barbie.txt -> barbie.txt.old
food.txt -> food.txt.old
quirks.txt -> quirks.txt.old
The command above could also be written on a single line

4.2.16 Command Substitution

Command substitution allows the output of one command to be
used as arguments to another
For example, use the locate command to find all files called
manual.html and print information about them with ls:
$ ls -l $(locate manual.html)
$ ls -l ‘locate manual.html‘

Page - 37 -
 Operating Systems

The punctuation marks on the second form are opening single

quote characters, called backticks
o The $() form is usually preferred, but backticks are widely used
Line breaks in the output are converted to spaces
Another example: use vi to edit the last of the files found:
$ vi $(locate manual.html | tail -1)

4.2.17 Finding Files with locate

The locate command is a simple and fast way to find files
For example, to find files relating to the email program mutt:
$ locate mutt
The locate command searches a database of filenames
o The database needs to be updated regularly
o Usually this is done automatically with cron
o But locate will not find files created since the last update
The -i option makes the search case-insensitive
-r treats the pattern as a regular expression, rather than a simple
string
4.2.18 Finding Files More Flexibly: ‘find’
locate only finds files by name
find can find files by any combination of a wide number of criteria,
including name
Structure: find directories criteria
Simplest possible example: find .
Finding files with a simple criterion:
$ find . -name manual.html
Looks for files under the current directory whose name is
manual.html
The criteria always begin with a single hyphen, even though they
have long names

Page - 38 -
 Operating Systems

4.2.19 ‘find’ Criteria

find accepts many different criteria; two of the most useful are:
o -name pattern: selects files whose name matches the shell-
style wildcard pattern
o -type d, -type f: select directories or plain files, respectively
You can have complex selections involving ‘and’, ‘or’, and ‘not’

4.2.20 ‘find’ Actions: Executing Programs

find lets you specify an action for each file found; the default action
is simply to print out the name
o You can alternatively write that explicitly as -print
Other actions include executing a program; for example, to delete
all files whose name starts with manual:
find . -name ’manual*’ -exec rm ’{}’ ’;’
The command rm ’{}’ is run for each file, with ’{}’ replaced by the
filename
The {} and ; are required by find, but must be quoted to protect
them from the shell

Page - 39 -
 Operating Systems

4.3- Exercises
Q1
a. Use the df command to display the amount of used and
available space on your hard drive.
b. Check the man page for df, and use it to find an option to the
command which will display the free space in a more human-
friendly form. Try both the single-letter and long-style options.
c. Run the shell, bash, and see what happens. Remember that you
were already running it to start with. Try leaving the shell you
have started with the exit command.

Q2
a. Try ls with the -a and -A options. What is the difference between
them?
b. Write a for loop which goes through all the files in a directory and
prints out their names with echo. If you write the whole thing on
one line, then it will be easy to repeat it using the command line
history.
c. Change the loop so that it goes through the names of the
people in the room (which needn’t be the names of files) and
print greetings to them.
d. Of course, a simpler way to print a list of filenames is echo *. Why
might this be useful, when we usually use the ls command?

Page - 40 -
 Operating Systems

Q3
a. Use the find command to list all the files and directories under
your home directory. Try the -type d and -type f criteria to show
just files and just directories.
b. Use ‘locate’ to find files whose name contains the string
‘bashbug’. Try the same search with find, looking over all files on
the system. You’ll need to use the * wildcard at the end of the
pattern to match files with extensions.
c. Find out what the find criterion -iname does.

Page - 41 -
 Operating Systems

Lab No. 5
5.1.0- Lab objective
This lab will give overview of Basic File and Directory Management utilities.

5.1.1 File system Objects

A file is a place to store data: a possibly-empty sequence of bytes
A directory is a collection of files and other directories
o Directories are organized in a hierarchy, with the root
directory at the top
The root directory is referred to as /
$cp rm jeff /

5.1.2 Directory and File Names

Files and directories are organized into a file-system
Refer to files in directories and sub-directories by separating their
names with /, for example:
o /bin/ls
o /usr/share/dict/words
o /home/jeff/recipe
Paths to files either start at / (absolute) or from some ‘current’
directory

5.1.3 File Extensions

It’s common to put an extension, beginning with a dot, on the end
of a filename
The extension can indicate the type of the file:
o .txt Text file
o .gif Graphics Interchange Format image

Page - 42 -
 Operating Systems

o .jpg Joint Photographic Experts Group image

o .mp3 MPEG-2 Layer 3 audio
o .gz Compressed file
o .tar Unix ‘tape archive’ file
o .tar.gz, .tgz Compressed archive file
On Unix and Linux, file extensions are just a convention
The kernel just treats them as a normal part of the name
A few programs use extensions to determine the type of a file

5.2 Going Back to Previous Directories

The pushd command takes you to another directory, like cd
But also saves the current directory, so that you can go back later
o For example, to visit Fred’s home directory, and then go back
to where you started from:
$ pushd ˜fred
$ cd Work
$ ls
...
$ popd
popd takes you back to the directory where you last did pushd
dirs will list the directories you can pop back to

5.3 Filename Completion

Modern shells help you type the names of files and directories by
completing partial names
Type the start of the name (enough to make it unambiguous) and
press Tab
For an ambiguous name (there are several possible completions),
the shell can list the options:

Page - 43 -
 Operating Systems

o For Bash, type Tab twice in succession

o For C shells, type Ctrl+D
Both of these shells will automatically escape spaces and special
characters in the filenames

5.4 Wildcard Patterns

Give commands multiple files by specifying patterns
Use the symbol * to match any part of a filename:
$ ls *.txt
accounts.txt letter.txt report.txt
Just * produces the names of all files in the current directory
The wildcard ? matches exactly one character:
$ rm -v data.?
removing data.1
removing data.2
removing data.3
Note: wildcards are turned into filenames by the shell, so the
program you pass them to can’t tell that those names came from
wildcard expansion

5.5.0 Copying Files with cp

Syntax: cp [options] source-file destination-file
Copy multiple files into a directory: cp files directory
Common options:
o -f, force overwriting of destination files
o -i, interactively prompt before overwriting files
o -a, archive, copy the contents of directories recursively

Page - 44 -
 Operating Systems

5.5.1 Examples of cp
Copy /etc/smb.conf to the current directory:
$ cp /etc/smb.conf .
Create an identical copy of a directory called work, and call it
work-backup:
$ cp -a work work-backup
Copy all the GIF and JPEG images in the current directory into
images:
$ cp *.gif *.jpeg images/

5.6 Moving Files with mv

mv can rename files or directories, or move them to different
directories
o It is equivalent to copying and then deleting
o But is usually much faster
Options:
o -f, force overwrite, even if target already exists
o -i, ask user interactively before overwriting files
o For example, to rename poetry.txt to poems.txt:
$ mv poetry.txt poems.txt
o To move everything in the current directory somewhere else:
$ mv * ˜/old-stuff/

5.7.0 Deleting Files with rm

rm deletes (‘removes’) the specified files
You must have write permission for the directory the file is in to
remove it
o Use carefully if you are logged in as root!
Options:

Page - 45 -
 Operating Systems

o -f, delete write-protected files without prompting

o -i, interactive — ask the user before deleting files
o -r, recursively delete files and directories
o For example, clean out everything in /tmp, without prompting
to delete each file:
$ rm -rf /tmp/*

5.7.1 Deleting Files with Peculiar Names

Some files have names which make them hard to delete
o Files that begin with a minus sign:
$ rm ./-filename
$ rm -- -filename
Files that contain peculiar characters — perhaps characters that
you can’t actually type on your keyboard:
Write a wildcard pattern that matches only the name you want to
delete:
$ rm -i ./name-with-funny-characters*
o The ./ forces it to be in the current directory
Using the -i option to rm makes sure that you won’t delete anything
else by accident

5.8.0 Making Directories with mkdir

Syntax: mkdir directory-names
Options:
o -p, create intervening parent directories if they don’t already
exist
o -m mode, set the access permissions to mode

Page - 46 -
 Operating Systems

o For example, create a directory called mystuff in your home

directory with permissions so that only you can write, but
eveyone can read it:
$ mkdir -m 755 ˜/mystuff
Create a directory tree in /tmp using one command with three
subdirectories called one, two and three:
$ mkdir -p /tmp/one/two/three

5.8.1 Removing Directories with rmdir

rmdir deletes empty directories, so the files inside must be deleted
first
o For example, to delete the images directory:
$ rm images/*
$ rmdir images
For non-empty directories, use rm -r directory
The -p option to rmdir removes the complete path, if there are no
other files and directories in it
These commands are equivalent:
$ rmdir -p a/b/c
$ rmdir a/b/c a/b a

5.9 Changing Timestamps with touch

Changes the access and modification times of files
Creates files that didn’t already exist
Options:
o -a, change only the access time
o -m, change only the modification time
o -t [YYYY]MMDDhhmm[.ss], set the timestamp of the file to the
specified date and time

Page - 47 -
 Operating Systems

GNU touch has a -d option, which accepts times in a more flexible

format
o For example, change the time stamp on homework to
January 20 2001, 5:59p.m.
$ touch -t 200101201759 homework

Page - 48 -
 Operating Systems

5.10 Exercises
Q1
a. Use cd to go to your home directory, and create a new directory there
called dog.
b. Create another directory within that one called cat, and another within
that called mouse.
c. Remove all three directories. You can either remove them one at a
time, or all at once.
d. If you can delete directories with rm -r, what is the point of using rmdir
for empty directories?
e. Try creating the dog/cat/mouse directory structure with a single
command.
Q2
a. Copy the file /etc/passwd to your home directory, and then use cat to
see what’s in it.
b. Rename it to users using the mv command.
c. Make a directory called programs and copy everything from /bin into
it.
d. Delete all the files in the programs directory.
e. Delete the empty programs directory and the users file.
Q3
a. The touch command can be used to create new empty files. Try that
now, picking a name for the new
file:
$ touch baked-beans
b. Get details about the file using the ls command:
$ ls -l baked-beans
c. Wait for a minute, and then try the previous two steps again, and see
what changes. What happens

Page - 49 -
 Operating Systems

when we don’t specify a time to touch?

d. Try setting the timestamp on the file to a value in the future.
e. When you’re finished with it, delete the file.

Page - 50 -
 Operating Systems

Lab No. 6
6.1.0- Lab objective
This lab will give overview of Processing Text Streams using Text Processing
Filters

6.1.1 Working with Text Files

Unix-like systems are designed to manipulate text very well
The same techniques can be used with plain text, or text-based
formats
Most Unix configuration files are plain text
Text is usually in the ASCII character set
Non-English text might use the ISO-8859 character sets
Unicode is better, but unfortunately many Linux command-line
utilities don’t (directly) support it yet

6.1.2 Lines of Text

Text files are naturally divided into lines
In Linux a line ends in a line feed character
Character number 10, hexadecimal 0x0A
Other operating systems use different combinations
o Windows and DOS use a carriage return followed by a line
feed
o Macintosh systems use only a carriage return
Programs are available to convert between the various formats

Page - 51 -
 Operating Systems

6.2 Filtering Text and Piping

The Unix philosophy: use small programs, and link them together as
needed
Each tool should be good at one specific job
Join programs together with pipes
Indicated with the pipe character: |
o The first program prints text to its standard output
o That gets fed into the second program’s standard input
For example, to connect the output of echo to the input of wc:
$ echo "count these words, boy" | wc

6.3 Displaying Files with less

If a file is too long to fit in the terminal, display it with less:
$ less README
less also makes it easy to clear the terminal of other things, so is
useful even for small files
Often used on the end of a pipe line, especially when it is not
known how long the output will be:
$ wc *.txt | less
o Doesn’t choke on strange characters, so it won’t mess up
your terminal (unlike cat)
6.4Counting Words and Lines with wc
wc counts characters, words and lines in a file
If used with multiple files, outputs counts for each file, and a
combined total
Options:
o -c output character count
o -l output line count
o l -w output word count

Page - 52 -
 Operating Systems

o Default is –clw
Examples: display word count for essay.txt:
$ wc -w essay.txt
Display the total number of lines in several text files:
$ wc -l *.txt
6.5 Sorting Lines of Text with sort
The sort filter reads lines of text and prints them sorted into order
For example, to sort a list of words into dictionary order:
$ sort words > sorted-words
The -f option makes the sorting case-insensitive
The -n option sorts numerically, rather than lexicographically
6.6 Removing Duplicate Lines with uniq
Use uniq to find unique lines in a file
Removes consecutive duplicate lines
Usually give it sorted input, to remove all duplicates
o Example: find out how many unique words are in a dictionary:
$ sort /usr/dict/words | uniq | wc –w
sort has a -u option to do this, without using a separate program:
$ sort -u /usr/dict/words | wc -w
sort | uniq can do more than sort -u, though:
uniq -c counts how many times each line appeared
uniq -u prints only unique lines
uniq -d prints only duplicated lines
6.7 Selecting Parts of Lines with cut
Used to select columns or fields from each line of input
Select a range of
o Characters, with
o Fields, with –f
o Field separator specified with -d (defaults to tab)

Page - 53 -
 Operating Systems

A range is written as start and end position: e.g., 3-5

o Either can be omitted
The first character or field is numbered 1, not 0
o Example: select usernames of logged in users:
$ who | cut -d" " -f1 | sort -u

6.8 Using fmt to Format Text Files

Arranges words nicely into lines of consistent length
Use -u to convert to uniform spacing
One space between words, two between sentences
Use -w width to set the maximum line width in characters
o Defaults to 75
o Example: change the line length of notes.txt to a maximum of
70 characters, and display it on the screen:
$ fmt -w 70 notes.txt | less

6.9 Reading the Start of a File with head

Prints the top of its input, and discards the rest
Set the number of lines to print with -n lines or -lines
Defaults to ten lines
Print the first line of a text file (two alternatives):
$ head -n 1 notes.txt
$ head -1 notes.txt

6.10 Reading the End of a File with tail

Similar to head, but prints lines at the end of a file
The -f option watches the file forever
Continually updates the display as new entries are appended to
the end of the file

Page - 54 -
 Operating Systems

Kill it with Ctrl+C

The option -n is the same as in head (number of lines to print)

6.11 Numbering Lines of a File with nl or cat

Display the input with line numbers against each line
There are options to finely control the formatting
By default, blank lines aren’t numbered
The option -ba numbers every line
cat -n also numbers lines, including blank ones

6.12 Dividing Files into Chunks with split

Splits files into equal-sized segments
Syntax: split [options] [input] [output-prefix]
Use -l n to split a file into n-line chunks
Use -b n to split into chunks of n bytes each
Output files are named using the specified output name with aa,
ab, ac, etc., added to the end of the prefix
o Example: Split essay.txt into 30-line files, and save the output
to files short_aa, short_ab, etc:
$ split -l 30 essay.txt short_

6.13 Reversing Files with tac

Similar to cat, but in reverse
Prints the last line of the input first, the penultimate line second, and
so on
o Example: show a list of logins and logouts, but with the most
recent events at the end:
$ last | tac

Page - 55 -
 Operating Systems

6.14 Modifying Files with sed

sed uses a simple script to process each line of a file
Specify the script file with -f filename
Or give individual commands with -e command
For example, if you have a script called spelling.sed which corrects
your most common mistakes, you can feed a file through it:
$ sed -f spelling.sed < report.txt > corrected.txt

6.15 Put Files Side-by-Side with paste

paste takes lines from two or more files and puts them in columns of
the output
o Use -d char to set the delimiter between fields in the output
o The default is tab
Giving -d more than one character sets different delimiters between
each pair of columns
Example: assign passwords to users, separating them with a colon:
$ paste -d: usernames passwords > .htpasswd

Page - 56 -
 Operating Systems

6.16 Exercises

Q1
a. Type in the example on the cut slide to display a list of users logged in.
(Try just who on its own first to see what is happening.)
b. Arrange for the list of usernames in who’s output to be sorted, and
remove any duplicates.
c. Try the command last to display a record of login sessions, and then try
reversing it with tac. Which is more useful? What if you pipe the output into
less?
d. Use sed to correct the misspelling ‘enviroment’ to ‘environment’. Use it
on a test file, containing a few lines of text, to check it. Does it work if the
misspelling occurs more than once on the same line?
e. Use nl to number the lines in the output of the previous question.

Q2
a. Try making an empty file and using tail -f to monitor it. Then add lines to
it from a different terminal using a command like this:
$ echo "testing" >>filename
b. Once you have written some lines into your file, use tr to display it with
all occurances of the letters A–F changed to the numbers 0–5.
c. Try looking at the binary for the ls command (/bin/ls) with less. You can
use the -f option to force it to display the file, even though it isn’t text.
d. Try viewing the same binary with od. Try it in its default mode, as well as
with the options shown on the slide for outputting in hexadecimal.
Q3
a. Use the split command to split the binary of the ls command into 1Kb
chunks. You might want to create a directory especially for the split files,
so that it can all be easily deleted later.

Page - 57 -
 Operating Systems

b. Put your split ls command back together again, and run it to make sure
it still works. You will have to make sure you are running the new copy of it,
for example ./my_ls, and make sure that the program is marked as
‘executable’ to run it, with the following command:
$ chmod a+rx my_ls

Page - 58 -