Linux Fundamentals

Paul Cobbaut
Table of Contents
I. introduction to Linux ............................................................................................................................. ...... 1
Part II. shell expansion ............................................................................................................................................ 5
Chapter 1. commands and arguments ....................................................................................................... 6
1.1. arguments ...................................................................................................................................... 7
1.2. white space removal ...................................................................................................................... 7
1.3. single quotes .................................................................................................................................. 7
1.4. double quotes................................................................................................................................. 7
1.5. echo and quotes ............................................................................................................................. 8
1.6. commands ..................................................................................................................................... 8
1.7. aliases ............................................................................................................................................ 9
1.8. displaying shell expansion ........................................................................................................... 10
1.9. practice: commands and arguments ............................................................................................. 10
Chapter 2. control operators ..................................................................................................................... 12
2.1. ; semicolon .................................................................................................................................. 13
2.2. & ampersand ............................................................................................................................... 13
2.3. $? dollar question mark ............................................................................................................... 13
2.4. && double ampersand................................................................................................................. 13
2.5. || double vertical bar .................................................................................................................... 14
2.6. combining && and || ................................................................................................................... 14
2.7. # pound sign ................................................................................................................................ 14
2.8. \ escaping special characters ........................................................................................................ 15
2.9. practice: control operators ........................................................................................................... 15
Chapter 3. shell variables .......................................................................................................................... 17
3.1. $ dollar sign ................................................................................................................................. 18
3.2. case sensitive ............................................................................................................................... 18
3.3. creating variables......................................................................................................................... 18
3.4. quotes .......................................................................................................................................... 18
3.5. set ................................................................................................................................................ 19
3.6. unset ............................................................................................................................................ 19
3.7. $PS1 ............................................................................................................................................ 19
3.8. $PATH ......................................................................................................................................... 21
3.9. env ............................................................................................................................................... 21
3.10. export......................................................................................................................................... 22
3.11. delineate variables ..................................................................................................................... 22
3.12. unbound variables ..................................................................................................................... 22
3.13. practice: shell variables ............................................................................................................. 23
Chapter 4. shell embedding and options .................................................................................................. 24
4.1. shell embedding........................................................................................................................... 25
4.2. shell options ................................................................................................................................ 26
4.3. practice: shell embedding ............................................................................................................ 26
Chapter 5. file globbing ............................................................................................................................. 28
5.1. * asterisk...................................................................................................................................... 29
5.2. ? question mark ........................................................................................................................... 29
5.3. [] square brackets ........................................................................................................................ 29
5.4. a-z and 0-9 ranges ....................................................................................................................... 30
5.5. $LANG and square brackets ....................................................................................................... 30
5.6. preventing file globbing .............................................................................................................. 31
5.7. practice: shell globbing ............................................................................................................... 32
Chapter 6. shell history.............................................................................................................................. 33
6.1. repeating the last command ......................................................................................................... 34
6.2. repeating other commands ........................................................................................................... 34
6.3. history .......................................................................................................................................... 34
6.4. !n ................................................................................................................................................. 34
 6.5. Ctrl-r ............................................................................................................................................ 35
6.6. $HISTSIZE ................................................................................................................................. 35
6.7. $HISTFILE ................................................................................................................................. 35
6.8. $HISTFILESIZE ......................................................................................................................... 35
6.9. prevent recording a command ..................................................................................................... 36
6.10. (optional)regular expressions .................................................................................................... 36
6.12. practice: shell history................................................................................................................. 36
16.13. solution: shell history .............................................................................................................. 37
Part III. pipes and commands ................................................................................................................................ 38
Chapter 1. I/O redirection ......................................................................................................................... 39
1.1. stdin, stdout, and stderr................................................................................................................ 40
1.2. output redirection ........................................................................................................................ 40
1.3. error redirection ........................................................................................................................... 42
1.4. output redirection and pipes ........................................................................................................ 43
1.5. joining stdout and stderr .............................................................................................................. 43
1.6. input redirection .......................................................................................................................... 43
1.7. confusing redirection ................................................................................................................... 44
1.8. quick file clear ............................................................................................................................. 45
1.9. practice: input/output redirection ................................................................................................ 46
1.10. solution: input/output redirection .............................................................................................. 47
Chapter 2. filters ........................................................................................................................................ 48
2.1. cat ................................................................................................................................................ 49
2.2. tee ................................................................................................................................................ 49
2.3. grep.............................................................................................................................................. 49
2.4. cut ................................................................................................................................................ 50
2.5. tr .................................................................................................................................................. 51
2.6. wc ................................................................................................................................................ 52
2.7. sort ............................................................................................................................................... 52
2.8. uniq.............................................................................................................................................. 53
2.9. comm ........................................................................................................................................... 54
2.10. od ............................................................................................................................................... 54
2.11. sed.............................................................................................................................................. 56
2.12. pipe examples ............................................................................................................................ 56
2.13. practice: filters ........................................................................................................................... 57
Chapter 3. basic Unix tools ....................................................................................................................... 58
3.1. find .............................................................................................................................................. 59
3.2. locate ........................................................................................................................................... 59
3.3. date .............................................................................................................................................. 60
3.4. cal ................................................................................................................................................ 60
3.5. sleep............................................................................................................................................. 61
3.6. time.............................................................................................................................................. 61
3.7. gzip - gunzip ................................................................................................................................ 62
3.8. zcat - zmore ................................................................................................................................. 62
3.9. bzip2 - bunzip2 ............................................................................................................................ 62
3.10. bzcat - bzmore ........................................................................................................................... 62
3.11. practice: basic Unix tools .......................................................................................................... 63
Chapter 4. regular expressions ................................................................................................................. 64
4.1. regex versions .............................................................................................................................. 65
4.2. grep.............................................................................................................................................. 66
4.3. rename ......................................................................................................................................... 70
4.4. sed ............................................................................................................................................... 72
4.5. bash history ................................................................................................................................. 75

iii
Linux Fundamentals

iv
Part II. shell expansion
Chapter 1. commands and arguments
This chapter introduces you to shell expansion by taking a close look at commands and
arguments. Knowing shell expansion is important because many commands on your Linux
system are processed and most likely changed by the shell before they are executed.

The command line interface or shell used on most Linux systems is called bash, which stands
for Bourne again shell.

This chapter frequently uses the echo command to demonstrate shell features. The echo
command is very simple: it echoes the input that it receives.
paul@laika:~$ echo Burtonville
Burtonville
paul@laika:~$ echo Smurfs are blue
Smurfs are blue

6
 commands and arguments

1.1. arguments
One of the primary features of a shell is to perform a command line scan. When you enter
a command at the shell's command prompt and press the enter key, then the shell will start
scanning that line, cutting it up in arguments. While scanning the line, the shell may make
many changes to the arguments you typed.

This process is called shell expansion. When the shell has finished scanning and modifying
that line, then it will be executed.

1.2. white space removal

Parts that are separated by one or more consecutive white spaces (or tabs) are considered
separate arguments, any white space is removed. The first argument is the command to be
executed, the other arguments are given to the command. The shell effectively cuts your
command into one or more arguments.

This explains why the following four different command lines are the same after shell
expansion.
[paul@RHELv4u3 ~]$ echo Hello World
Hello World
[paul@RHELv4u3 ~]$ echo Hello World
Hello World
[paul@RHELv4u3 ~]$ echo Hello World
Hello World
[paul@RHELv4u3 ~]$ echo Hello World
Hello World

The echo command will display each argument it receives from the shell. The echo
command will also add a new white space between the arguments it received.

1.3. single quotes

You can prevent the removal of white spaces by quoting the spaces. The contents of the
quoted string are considered as one argument. In the screenshot below the echo receives
only one argument.
[paul@RHEL4b ~]$ echo 'A line with single quotes'
A line with single quotes
[paul@RHEL4b ~]$

1.4. double quotes

You can also prevent the removal of white spaces by double quoting the spaces. Same as
above, echo only receives one argument.
[paul@RHEL4b ~]$ echo "A line with double quotes"
A line with double quotes
[paul@RHEL4b ~]$

7
 commands and arguments

Later in this book, when discussing variables we will see important differences between
single and double quotes.

1.5. echo and quotes

Quoted lines can include special escaped characters recognised by the echo command (when
using echo -e). The screenshot below shows how to use \n for a newline and \t for a tab
(usually eight white spaces).
[paul@RHEL4b ~]$ echo -e "A line with \na newline"
A line with
a newline
[paul@RHEL4b ~]$ echo -e 'A line with \na newline'
A line with
a newline
[paul@RHEL4b ~]$ echo -e "A line with \ta tab"
A line with a tab
[paul@RHEL4b ~]$ echo -e 'A line with \ta tab'
A line with a tab
[paul@RHEL4b ~]$
The echo command can generate more than white spaces, tabs and newlines. Look in the
man page for a list of options.

1.6. commands
1.6.1. external or builtin commands ?
Not all commands are external to the shell, some are builtin. External commands are
programs that have their own binary and reside somewhere in the file system. Many external
commands are located in /bin or /sbin. Builtin commands are an integral part of the shell
program itself.

1.6.2. type
To find out whether a command given to the shell will be executed as an external command
or as a builtin command, use the type command.
paul@laika:~$ type cd
cd is a shell builtin
paul@laika:~$ type cat
cat is /bin/cat

As you can see, the cd command is builtin and the cat command is external. You can

also use this command to show you whether the command is aliased or not.

paul@laika:~$ type ls

ls is aliased to `ls --color=auto'

8
 commands and arguments

1.6.3. running external commands

Some commands have both builtin and external versions. When one of these commands is
executed, the builtin version takes priority. To run the external version, you must enter the
full path to the command.
paul@laika:~$ type -a echo

echo is a shell builtin echo is /bin/echo

paul@laika:~$ /bin/echo Running the external echo command...

Running the external echo command...

1.6.4. which
The which command will search for binaries in the $PATH environment variable (variables
will be explained later). In the screenshot below, it is determined that cd is builtin, and ls,
cp, rm, mv, mkdir, pwd, and which are external commands.
[root@RHEL4b ~]# which cp ls cd mkdir pwd
/bin/cp
/bin/ls /usr/bin/which: no cd in (/usr/kerberos/sbin:/usr/kerberos/bin:...
/bin/mkdir
/bin/pwd

1.7. aliases
1.7.1. create an alias
The shell allows you to create aliases. Aliases are often used to create an easier to remember
name for an existing command or to easily supply parameters.
[paul@RHELv4u3 ~]$ cat count.txt
one two three

[paul@RHELv4u3 ~]$ alias dog=tac

[paul@RHELv4u3 ~]$ dog count.txt
three two one

1.7.2. abbreviate commands

An alias can also be useful to abbreviate an existing command.
paul@laika:~$ alias ll='ls -lh --color=auto'
paul@laika:~$ alias c='clear'

paul@laika:~$

1.7.3. default options

Aliases can be used to supply commands with default options. The example below shows
how to set the -i option default when typing rm.

9
 commands and arguments

[paul@RHELv4u3 ~]$ rm -i winter.txt rm:

remove regular file `winter.txt'? no
[paul@RHELv4u3 ~]$ rm winter.txt
[paul@RHELv4u3 ~]$ ls winter.txt
ls: winter.txt: No such file or directory
[paul@RHELv4u3 ~]$ touch winter.txt
[paul@RHELv4u3 ~]$ alias rm='rm -i'
[paul@RHELv4u3 ~]$ rm winter.txt
rm: remove regular empty file `winter.txt'? no
[paul@RHELv4u3 ~]$
Some distributions enable default aliases to protect users from accidentally erasing files ('rm
-i', 'mv -i', 'cp -i')

1.7.4. viewing aliases

You can provide one or more aliases as arguments to the alias command to get their
definitions. Providing no arguments gives a complete list of current aliases.
paul@laika:~$ alias c ll
alias c='clear'
alias ll='ls -lh --color=auto'

1.7.5. unalias
You can undo an alias with the unalias command.
[paul@RHEL4b ~]$ which rm
/bin/rm
[paul@RHEL4b ~]$ alias rm='rm -i'
[paul@RHEL4b ~]$ which rm
alias rm='rm -i' /bin/rm
[paul@RHEL4b ~]$ unalias rm
[paul@RHEL4b ~]$ which rm
/bin/rm
[paul@RHEL4b ~]$

1.8. displaying shell expansion

You can display shell expansion with set -x, and stop displaying it with set +x. You might
want to use this further on in this course, or when in doubt about exactly what the shell is
doing with your command.

1.9. practice: commands and arguments

1. How many arguments are in this line (not counting the command itself).
touch '/etc/cron/cron.allow' 'file 42.txt' "file 33.txt"

10
 commands and arguments

2. Is tac a shell builtin command ?

3. Is there an existing alias for rm ?

4. Read the man page of rm, make sure you understand the -i option of rm. Create and
remove a file to test the -i option.

5. Execute: alias rm='rm -i' . Test your alias with a test file. Does this work as expected ?

6. List all current aliases.

7a. Create an alias called 'city' that echoes your hometown.

7b. Use your alias to test that it works.

8. Execute set -x to display shell expansion for every command.

9. Test the functionality of set -x by executing your city and rm aliases.

10 Execute set +x to stop displaying shell expansion.

11. Remove your city alias.

12. What is the location of the cat and the passwd commands ?

13. Explain the difference between the following commands:

echo

/bin/echo

14. Explain the difference between the following commands:

echo Hello

echo -n Hello

15. Display A B C with two spaces between B and C.

(optional)16. Complete the following command (do not use spaces) to display exactly the
following output:
4+4 =8
10+14 =24

17. Use echo to display the following exactly:

??\\
Find two solutions with single quotes, two with double quotes and one without quotes (and
say thank you to René and Darioush from Google for this extra).

18. Use one echo command to display three words on three lines.

11
Chapter 2. control operators
In this chapter we put more than one command on the command line using control
operators. We also briefly discuss related parameters ($?) and similar special characters(&).

12
 control operators

2.1. ; semicolon
You can put two or more commands on the same line separated by a semicolon ; . The shell
will scan the line until it reaches the semicolon. All the arguments before this semicolon
will be considered a separate command from all the arguments after the semicolon. Both
series will be executed sequentially with the shell waiting for each command to finish before
starting the next one.
[paul@RHELv4u3 ~]$ echo Hello
Hello
[paul@RHELv4u3 ~]$ echo World
World
[paul@RHELv4u3 ~]$ echo Hello ; echo World
Hello
World
[paul@RHELv4u3 ~]$

2.2. & ampersand

When a line ends with an ampersand &, the shell will not wait for the command to finish.
You will get your shell prompt back, and the command is executed in background. You will
get a message when this command has finished executing in background.
[paul@RHELv4u3 ~]$ sleep 20 &
[1] 7925
[paul@RHELv4u3 ~]$ ...wait
20 seconds...
[paul@RHELv4u3 ~]$
[1]+ Done sleep 20
The technical explanation of what happens in this case is explained in the chapter about
processes.

2.3. $? dollar question mark

The exit code of the previous command is stored in the shell variable $?. Actually $? is a
shell parameter and not a variable, since you cannot assign a value to $?.
paul@debian5:~/test$ touch file1
paul@debian5:~/test$ echo $? 0
paul@debian5:~/test$ rm file1
paul@debian5:~/test$ echo $?
0
paul@debian5:~/test$ rm file1
rm: cannot remove `file1': No such file or directory
paul@debian5:~/test$ echo $? 1
paul@debian5:~/test$

2.4. && double ampersand

The shell will interpret && as a logical AND. When using && the second command is
executed only if the first one succeeds (returns a zero exit status).

13
 control operators

paul@barry:~$ echo first && echo second

first second

paul@barry:~$ zecho first && echo second

-bash: zecho: command not found

Another example of the same logical AND principle. This example starts with a working cd
followed by ls, then a non-working cd which is not followed by ls.
[paul@RHELv4u3 ~]$ cd gen && ls
file1 file3 File55 fileab FileAB fileabc file2 File4
FileA Fileab fileab2
[paul@RHELv4u3 gen]$ cd gen && ls
-bash: cd: gen: No such file or directory

2.5. || double vertical bar

The || represents a logical OR. The second command is executed only when the first
command fails (returns a non-zero exit status).
paul@barry:~$ echo first || echo second ; echo third
first third

paul@barry:~$ zecho first || echo second ; echo third

-bash: zecho: command not found second third
paul@barry:~$

Another example of the same logical OR principle.

[paul@RHELv4u3 ~]$ cd gen || ls
[paul@RHELv4u3 gen]$ cd gen || ls -bash: cd: gen: No such
file or directory file1 file3 File55 fileab FileAB
fileabc file2 File4 FileA Fileab fileab2

2.6. combining && and ||

You can use this logical AND and logical OR to write an if-then-else structure on the
command line. This example uses echo to display whether the rm command was successful.
paul@laika:~/test$ rm file1 && echo It worked! || echo It failed!
It worked!
paul@laika:~/test$ rm file1 && echo It worked! || echo It failed!
rm: cannot remove `file1': No such file or directory
It failed!
paul@laika:~/test$

2.7. # pound sign

Everything written after a pound sign (#) is ignored by the shell. This is useful to write a
shell comment, but has no influence on the command execution or shell expansion.
paul@debian4:~$ mkdir test # we create a directory
paul@debian4:~$ cd test #### we enter the directory

14
 control operators

paul@debian4:~/test$ ls # is it empty ?
paul@debian4:~/test$

2.8. \ escaping special characters

The backslash \ character enables the use of control characters, but without the shell
interpreting it, this is called escaping characters.
[paul@RHELv4u3 ~]$ echo hello \; world

hello ; world

[paul@RHELv4u3 ~]$ echo hello\ \ \ world

hello world

[paul@RHELv4u3 ~]$ echo escaping \\\ \#\ \&\ \"\ \'

escaping \ # & " '

[paul@RHELv4u3 ~]$ echo escaping \\\?\*\"\'

escaping \?*"'

2.8.1. end of line backslash

Lines ending in a backslash are continued on the next line. The shell does not interpret the
newline character and will wait on shell expansion and execution of the command line until
a newline without backslash is encountered.
[paul@RHEL4b ~]$ echo This command line \
> is split in three \
> parts
This command line is split in three parts
[paul@RHEL4b ~]$

2.9. practice: control operators

0. Each question can be answered by one command line!

1. When you type passwd, which file is executed ?

2. What kind of file is that ?

3. Execute the pwd command twice. (remember 0.)

4. Execute ls after cd /etc, but only if cd /etc did not error.

5. Execute cd /etc after cd etc, but only if cd etc fails.

15
 control operators

6. Echo it worked when touch test42 works, and echo it failed when the touch failed. All
on one command line as a normal user (not root). Test this line in your home directory
and in /bin/ .

7. Execute sleep 6, what is this command doing ?

8. Execute sleep 200 in background (do not wait for it to finish).

9. Write a command line that executes rm file55. Your command line should print 'success'
if file55 is removed, and print 'failed' if there was a problem.

(optional)10. Use echo to display "Hello World with strange' characters \ * [ } ~ \ \ ."
(including all quotes)

16
Chapter 3. shell variables
In this chapter we learn to manage environment variables in the shell. These variables are
often needed by applications.

17
 shell variables

3.1. $ dollar sign

Another important character interpreted by the shell is the dollar sign $. The shell will look
for an environment variable named like the string following the dollar sign and replace
it with the value of the variable (or with nothing if the variable does not exist).

These are some examples using $HOSTNAME, $USER, $UID, $SHELL, and $HOME.
[paul@RHELv4u3 ~]$ echo This is the $SHELL shell
This is the /bin/bash shell
[paul@RHELv4u3 ~]$ echo This is $SHELL on computer $HOSTNAME
This is /bin/bash on computer RHELv4u3.localdomain
[paul@RHELv4u3 ~]$ echo The userid of $USER is $UID
The userid of paul is 500
[paul@RHELv4u3 ~]$ echo My homedir is $HOME
My homedir is /home/paul

3.2. case sensitive

This example shows that shell variables are case sensitive!
[paul@RHELv4u3 ~]$ echo Hello $USER
Hello paul
[paul@RHELv4u3 ~]$ echo Hello $user
Hello

3.3. creating variables

This example creates the variable $MyVar and sets its value. It then uses echo to verify the
value.
[paul@RHELv4u3 gen]$ MyVar=555
[paul@RHELv4u3 gen]$ echo $MyVar
555
[paul@RHELv4u3 gen]$

3.4. quotes
Notice that double quotes still allow the parsing of variables, whereas single quotes prevent
this.
[paul@RHELv4u3 ~]$ MyVar=555
[paul@RHELv4u3 ~]$ echo $MyVar
555
[paul@RHELv4u3 ~]$ echo "$MyVar"
555
[paul@RHELv4u3 ~]$ echo '$MyVar'
$MyVar

The bash shell will replace variables with their value in double quoted lines, but not in single
quoted lines.

18
 shell variables

paul@laika:~$ city=Burtonville
paul@laika:~$ echo "We are in $city today."
We are in Burtonville today.

paul@laika:~$ echo 'We are in $city today.'

We are in $city today.

3.5. set
You can use the set command to display a list of environment variables. On Ubuntu and
Debian systems, the set command will also list shell functions after the shell variables. Use
set | more to see the variables then.

3.6. unset
Use the unset command to remove a variable from your shell environment.
[paul@RHEL4b ~]$ MyVar=8472
[paul@RHEL4b ~]$ echo $MyVar
8472
[paul@RHEL4b ~]$ unset MyVar
[paul@RHEL4b ~]$ echo $MyVar

[paul@RHEL4b ~]$

3.7. $PS1
The $PS1 variable determines your shell prompt. You can use backslash escaped special
characters like \u for the username or \w for the working directory. The bash manual has a
complete reference.

In this example we change the value of $PS1 a couple of times.

paul@deb503:~$ PS1=prompt
prompt prompt

PS1='prompt ' prompt prompt

PS1='> ' > >

PS1='\u@\h$ '

paul@deb503$ paul@deb503$
PS1='\u@\h:\W$'
paul@deb503:~$

To avoid unrecoverable mistakes, you can set normal user prompts to green and the root
prompt to red. Add the following to your .bashrc for a green user prompt:
# color prompt by paul
RED='\[\033[01;31m\]'
WHITE='\[\033[01;00m\]'
GREEN='\[\033[01;32m\]' BLUE='\[\033[01;34m\]'

19
 shell variables

export
PS1="${debian_chroot:+($debian_chroot)}$GREEN\u$WHITE@$BLUE\h$WHITE\w\$ "

20
 shell variables

3.8. $PATH
The $PATH variable is determines where the shell is looking for commands to execute
(unless the command is builtin or aliased). This variable contains a list of directories,
separated by colons.
[[paul@RHEL4b ~]$ echo $PATH
/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:

The shell will not look in the current directory for commands to execute! (Looking for
executables in the current directory provided an easy way to hack PC-DOS computers). If
you want the shell to look in the current directory, then add a . at the end of your $PATH.
[paul@RHEL4b ~]$ PATH=$PATH:.
[paul@RHEL4b ~]$ echo $PATH
/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:.
[paul@RHEL4b ~]$

Your path might be different when using su instead of su - because the latter will take on
the environment of the target user. The root user typically has /sbin directories added to the
$PATH variable.
[paul@RHEL3 ~]$ su
Password:
[root@RHEL3 paul]# echo $PATH
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
[root@RHEL3 paul]# exit
[paul@RHEL3 ~]$ su Password:
[root@RHEL3 ~]# echo $PATH
/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:
[root@RHEL3 ~]#

3.9. env
The env command without options will display a list of exported variables. The difference
with set with options is that set lists all variables, including those not exported to child
shells.

But env can also be used to start a clean shell (a shell without any inherited environment).
The env -i command clears the environment for the subshell.

Notice in this screenshot that bash will set the $SHELL variable on startup.
[paul@RHEL4b ~]$ bash -c 'echo $SHELL $HOME $USER'
/bin/bash /home/paul paul
[paul@RHEL4b ~]$ env -i bash -c 'echo $SHELL $HOME $USER'
/bin/bash
[paul@RHEL4b ~]$

You can use the env command to set the $LANG, or any other, variable for just one instance
of bash with one command. The example below uses this to show the influence of the
$LANG variable on file globbing (see the chapter on file globbing).
[paul@RHEL4b test]$ env LANG=C bash -c 'ls File[a-z]'

21
 shell variables

Filea Fileb
[paul@RHEL4b test]$ env LANG=en_US.UTF-8 bash -c 'ls File[a-z]'
Filea FileA Fileb FileB
[paul@RHEL4b test]$

3.10. export
You can export shell variables to other shells with the export command. This will export
the variable to child shells.
[paul@RHEL4b ~]$ var3=three
[paul@RHEL4b ~]$ var4=four
[paul@RHEL4b ~]$ export var4

[paul@RHEL4b ~]$ echo $var3 $var4 three four

[paul@RHEL4b ~]$ bash

[paul@RHEL4b ~]$ echo $var3 $var4 four

But it will not export to the parent shell (previous screenshot continued).
[paul@RHEL4b ~]$ export var5=five
[paul@RHEL4b ~]$ echo $var3 $var4 $var5 four five
[paul@RHEL4b ~]$ exit exit
[paul@RHEL4b ~]$ echo $var3 $var4 $var5 three four
[paul@RHEL4b ~]$

3.11. delineate variables

Until now, we have seen that bash interprets a variable starting from a dollar sign, continuing
until the first occurrence of a non-alphanumeric character that is not an underscore. In some
situations, this can be a problem. This issue can be resolved with curly braces like in this
example.
[paul@RHEL4b ~]$ prefix=Super
[paul@RHEL4b ~]$ echo Hello $prefixman and $prefixgirl
Hello and
[paul@RHEL4b ~]$ echo Hello ${prefix}man and ${prefix}girl
Hello Superman and Supergirl
[paul@RHEL4b ~]$

3.12. unbound variables

The example below tries to display the value of the $MyVar variable, but it fails because
the variable does not exist. By default the shell will display nothing when a variable is
unbound (does not exist).
[paul@RHELv4u3 gen]$ echo $MyVar

[paul@RHELv4u3 gen]$

22
 shell variables

There is, however, the nounset shell option that you can use to generate an error when a
variable does not exist.
paul@laika:~$ set -u

paul@laika:~$ echo $Myvar

bash: Myvar: unbound variable

paul@laika:~$ set +u

paul@laika:~$ echo $Myvar

paul@laika:~$

In the bash shell set -u is identical to set -o nounset and likewise set +u is identical to set
+o nounset.

3.13. practice: shell variables

1. Use echo to display Hello followed by your username. (use a bash variable!)

2. Create a variable answer with a value of 42.

3. Copy the value of $LANG to $MyLANG.

4. List all current shell variables.

5. List all exported shell variables.

6. Do the env and set commands display your variable ?

6. Destroy your answer variable.

7. Create two variables, and export one of them.

8. Display the exported variable in an interactive child shell.

9. Create a variable, give it the value 'Dumb', create another variable with value 'do'.
Useecho and the two variables to echo Dumbledore.

10. Find the list of backslash escaped characters in the manual of bash. Add the time to
your PS1 prompt.

23
Chapter 4. shell embedding and
options
This chapter takes a brief look at child shells, embedded shells and shell options.

24
 shell embedding and options

4.1. shell embedding

Shells can be embedded on the command line, or in other words, the command line scan
can spawn new processes containing a fork of the current shell. You can use variables to
prove that new shells are created. In the screenshot below, the variable $var1 only exists in
the (temporary) sub shell.
[paul@RHELv4u3 gen]$ echo $var1

[paul@RHELv4u3 gen]$ echo $(var1=5;echo $var1)

5
[paul@RHELv4u3 gen]$ echo $var1

[paul@RHELv4u3 gen]$

You can embed a shell in an embedded shell, this is called nested embedding of shells.

This screenshot shows an embedded shell inside an embedded shell.

paul@deb503:~$ A=shell

paul@deb503:~$ echo $C$B$A $(B=sub;echo $C$B$A; echo $(C=sub;echo $C$B$A))

shell subshell subsubshell

4.1.1. backticks
Single embedding can be useful to avoid changing your current directory. The screenshot
below uses backticks instead of dollar-bracket to embed.
[paul@RHELv4u3 ~]$ echo `cd /etc; ls -d * | grep pass`
passwd passwd- passwd.OLD

[paul@RHELv4u3 ~]$

You can only use the $() notation to nest embedded shells, backticks cannot do this.

4.1.2. backticks or single quotes

Placing the embedding between backticks uses one character less than the dollar and
parenthesis combo. Be careful however, backticks are often confused with single quotes.
The technical difference between ' and ` is significant!
[paul@RHELv4u3 gen]$ echo `var1=5;echo $var1`
5
[paul@RHELv4u3 gen]$ echo 'var1=5;echo $var1' var1=5;echo $var1
[paul@RHELv4u3 gen]$

25
 shell embedding and options

4.2. shell options

Both set and unset are builtin shell commands. They can be used to set options of the bash
shell itself. The next example will clarify this. By default, the shell will treat unset variables
as a variable having no value. By setting the -u option, the shell will treat any reference to
unset variables as an error. See the man page of bash for more information.
[paul@RHEL4b ~]$ echo $var123

[paul@RHEL4b ~]$ set -u

[paul@RHEL4b ~]$ echo $var123
-bash: var123: unbound variable
[paul@RHEL4b ~]$ set +u
[paul@RHEL4b ~]$ echo $var123

[paul@RHEL4b ~]$

To list all the set options for your shell, use echo $-. The noclobber (or -C) option will be
explained later in this book (in the I/O redirection chapter).
[paul@RHEL4b ~]$ echo $himBH
[paul@RHEL4b ~]$ set -C ; set -u
[paul@RHEL4b ~]$ echo $himuBCH
[paul@RHEL4b ~]$ set +C ; set +u
[paul@RHEL4b ~]$ echo $himBH
[paul@RHEL4b ~]$

When typing set without options, you get a list of all variables without function when the
shell is on posix mode. You can set bash in posix mode typing set -o posix.

4.3. practice: shell embedding

1. Find the list of shell options in the man page of bash. What is the difference between set
-u and set -o nounset?

2. Activate nounset in your shell. Test that it shows an error message when using
nonexisting variables.

3. Deactivate nounset.

4. Execute cd /var and ls in an embedded shell.

The echo command is only needed to show the result of the ls command. Omitting will
result in the shell trying to execute the first file as a command.

5. Create the variable embvar in an embedded shell and echo it. Does the variable exist
inyour current shell now ?

26
 shell embedding and options

6. Explain what "set -x" does. Can this be useful ?

(optional)7. Given the following screenshot, add exactly four characters to that command
line so that the total output is FirstMiddleLast.
[paul@RHEL4b ~]$ echo First; echo Middle; echo Last

8. Display a long listing (ls -l) of the passwd command using the which command inside
an embedded shell.
(optional)7. Given the following screenshot, add exactly four characters to that command
line so that the total output is FirstMiddleLast.
[paul@RHEL4b ~]$ echo First; echo Middle; echo Last

echo -n First; echo -n Middle; echo Last

8. Display a long listing (ls -l) of the passwd command using the which command inside
an embedded shell.
ls -l $(which passwd)

27
Chapter 5. file globbing
The shell is also responsible for file globbing (or dynamic filename generation). This
chapter will explain file globbing.

28
 file globbing

5.1. * asterisk
The asterisk * is interpreted by the shell as a sign to generate filenames, matching the
asterisk to any combination of characters (even none). When no path is given, the shell will
use filenames in the current directory. See the man page of glob(7) for more information.
(This is part of LPI topic 1.103.3.)
[paul@RHELv4u3 gen]$ ls file1 file2 file3 File4 File55 FileA fileab
Fileab FileAB fileabc [paul@RHELv4u3 gen]$ ls File*
File4 File55 FileA Fileab FileAB
[paul@RHELv4u3 gen]$ ls file* file1
file2 file3 fileab fileabc
[paul@RHELv4u3 gen]$ ls *ile55
File55
[paul@RHELv4u3 gen]$ ls F*ile55
File55
[paul@RHELv4u3 gen]$ ls F*55
File55
[paul@RHELv4u3 gen]$

5.2. ? question mark

Similar to the asterisk, the question mark ? is interpreted by the shell as a sign to generate
filenames, matching the question mark with exactly one character.
[paul@RHELv4u3 gen]$ ls file1 file2 file3 File4 File55 FileA fileab
Fileab FileAB fileabc [paul@RHELv4u3 gen]$ ls File?
File4 FileA
[paul@RHELv4u3 gen]$ ls Fil?4
File4 [paul@RHELv4u3 gen]$
ls Fil??
File4 FileA [paul@RHELv4u3
gen]$ ls File??
File55 Fileab FileAB
[paul@RHELv4u3 gen]$

5.3. [] square brackets

The square bracket [ is interpreted by the shell as a sign to generate filenames, matching any
of the characters between [ and the first subsequent ]. The order in this list between the
brackets is not important. Each pair of brackets is replaced by exactly one character.
[paul@RHELv4u3 gen]$ ls
file1 file2 file3 File4 File55 FileA fileab Fileab FileAB fileabc
[paul@RHELv4u3 gen]$ ls File[5A]
FileA
[paul@RHELv4u3 gen]$ ls File[A5]
FileA
[paul@RHELv4u3 gen]$ ls File[A5][5b]
File55
[paul@RHELv4u3 gen]$ ls File[a5][5b]
File55 Fileab
[paul@RHELv4u3 gen]$ ls File[a5][5b][abcdefghijklm]
ls: File[a5][5b][abcdefghijklm]: No such file or directory
[paul@RHELv4u3 gen]$ ls
file[a5][5b][abcdefghijklm] fileabc
[paul@RHELv4u3 gen]$

You can also exclude characters from a list between square brackets with the exclamation
mark !. And you are allowed to make combinations of these wild cards.

29
 file globbing

[paul@RHELv4u3 gen]$ ls
file1 file2 file3 File4 File55 FileA fileab Fileab FileAB fileabc
[paul@RHELv4u3 gen]$ ls file[a5][!Z]
fileab
[paul@RHELv4u3 gen]$ ls file[!5]*
file1 file2 file3 fileab fileabc
[paul@RHELv4u3 gen]$ ls file[!5]? fileab
[paul@RHELv4u3 gen]$

5.4. a-z and 0-9 ranges

The bash shell will also understand ranges of characters between brackets.
[paul@RHELv4u3 gen]$ ls

file1 file3 File55 fileab FileAB fileabc

file2 File4 FileA Fileab fileab2

[paul@RHELv4u3 gen]$ ls file[a-z]*

fileab fileab2 fileabc

[paul@RHELv4u3 gen]$ ls file[0-9]

file1 file2 file3

[paul@RHELv4u3 gen]$ ls file[a-z][a-z][0-9]*

fileab2

[paul@RHELv4u3 gen]$

5.5. $LANG and square brackets

But, don't forget the influence of the LANG variable. Some languages include lower case
letters in an upper case range (and vice versa).
paul@RHELv4u4:~/test$ ls [A-Z]ile?
file1 file2 file3 File4
paul@RHELv4u4:~/test$ ls [a-z]ile?
file1 file2 file3 File4
paul@RHELv4u4:~/test$ echo $LANG
en_US.UTF-8
paul@RHELv4u4:~/test$ LANG=C
paul@RHELv4u4:~/test$ echo $LANG C
paul@RHELv4u4:~/test$ ls [a-z]ile?
file1 file2 file3
paul@RHELv4u4:~/test$ ls [A-Z]ile? File4
paul@RHELv4u4:~/test$

If $LC_ALL is set, then this will also need to be reset to prevent file globbing.

30
 file globbing

5.6. preventing file globbing

The screenshot below should be no surprise. The echo * will echo a * when in an empty
directory. And it will echo the names of all files when the directory is not empty.
paul@ubu1010:~$ mkdir test42
paul@ubu1010:~$ cd test42
paul@ubu1010:~/test42$ echo * *
paul@ubu1010:~/test42$ touch file42 file33
paul@ubu1010:~/test42$ echo * file33
file42

Globbing can be prevented using quotes or by escaping the special characters, as shown in
this screenshot.
paul@ubu1010:~/test42$ echo *
file33 file42
paul@ubu1010:~/test42$ echo \*
*
paul@ubu1010:~/test42$ echo '*'
*
paul@ubu1010:~/test42$ echo "*"
*

31
 file globbing

5.7. practice: shell globbing

1. Create a test directory and enter it.

2. Create the following files :

file1
file10
file11
file2
File2
File3
file33
fileAB
filea
fileA
fileAAA
file(
file 2

(the last one has 6 characters including a space)

3. List (with ls) all files starting with file

4. List (with ls) all files starting with File

5. List (with ls) all files starting with file and ending in a number.

6. List (with ls) all files starting with file and ending with a letter

7. List (with ls) all files starting with File and having a digit as fifth character.

8. List (with ls) all files starting with File and having a digit as fifth character and
nothingelse.

9. List (with ls) all files starting with a letter and ending in a number.

10. List (with ls) all files that have exactly five characters.

11. List (with ls) all files that start with f or F and end with 3 or A.

12. List (with ls) all files that start with f have i or R as second character and end in a number.

13. List all files that do not start with the letter F.

14. Copy the value of $LANG to $MyLANG.

15. Show the influence of $LANG in listing A-Z or a-z ranges.

16. You receive information that one of your servers was cracked, the cracker
probablyreplaced the ls command. You know that the echo command is safe to use. Can
echo replace ls ? How can you list the files in the current directory with echo ?

17. Is there another command besides cd to change directories ?

32
Chapter 6. shell history
The shell makes it easy for us to repeat commands, this chapter explains how.

33
 shell history

6.1. repeating the last command

To repeat the last command in bash, type !!. This is pronounced as bang bang.
paul@debian5:~/test42$ echo this will be repeated > file42.txt
paul@debian5:~/test42$ !!

echo this will be repeated > file42.txt

paul@debian5:~/test42$

6.2. repeating other commands

You can repeat other commands using one bang followed by one or more characters. The
shell will repeat the last command that started with those characters.
paul@debian5:~/test42$ touch file42
paul@debian5:~/test42$ cat file42
paul@debian5:~/test42$ !to

touch file42

paul@debian5:~/test42$

6.3. history
To see older commands, use history to display the shell command history (or use history n
to see the last n commands).
paul@debian5:~/test$ history 10
38 mkdir test
39 cd test
40 touch file1
41 echo hello > file2
42 echo It is very cold today >
winter.txt
43 ls
44 ls -l
45 cp winter.txt summer.txt
46 ls -l
47 history 10

6.4. !n
When typing ! followed by the number preceding the command you want repeated, then the
shell will echo the command and execute it.
paul@debian5:~/test$ !43

ls

34
 shell history

file1 file2 summer.txt winter.txt

6.5. Ctrl-r
Another option is to use ctrl-r to search in the history. In the screenshot below i only typed
ctrl-r followed by four characters apti and it finds the last command containing these four
consecutive characters.
paul@debian5:~$ (reverse-i-search)`apti':

sudo aptitude install screen

6.6. $HISTSIZE
The $HISTSIZE variable determines the number of commands that will be remembered in
your current environment. Most distributions default this variable to 500 or 1000.
paul@debian5:~$ echo $HISTSIZE
500

You can change it to any value you like.

paul@debian5:~$ HISTSIZE=15000
paul@debian5:~$ echo $HISTSIZE
15000

6.7. $HISTFILE
The $HISTFILE variable points to the file that contains your history. The bash shell defaults
this value to ~/.bash_history.
paul@debian5:~$ echo $HISTFILE
/home/paul/.bash_history

A session history is saved to this file when you exit the session!

Closing a gnome-terminal with the mouse, or typing reboot as root will NOT save your
terminal's history.

6.8. $HISTFILESIZE
The number of commands kept in your history file can be set using $HISTFILESIZE.
paul@debian5:~$ echo $HISTFILESIZE
15000

35
 shell history

6.9. prevent recording a command

You can prevent a command from being recorded in history using a space prefix.
paul@debian8:~/github$ echo abc abc
paul@debian8:~/github$ echo def def
paul@debian8:~/github$ echo ghi ghi
paul@debian8:~/github$ history 3
9501 echo abc
9502 echo ghi
9503 history 3

6.10. (optional)regular expressions

It is possible to use regular expressions when using the bang to repeat commands. The
screenshot below switches 1 into 2.
paul@debian5:~/test$ cat file1
paul@debian5:~/test$ !c:s/1/2 cat file2 hello
paul@debian5:~/test$

6.12. practice: shell history

1. Issue the command echo The answer to the meaning of life, the universe and
everything is 42.

2. Repeat the previous command using only two characters (there are two solutions!)

3. Display the last 5 commands you typed.

4. Issue the long echo from question 1 again, using the line numbers you received from the
command in question 3.

5. How many commands can be kept in memory for your current shell session ?

6. Where are these commands stored when exiting the shell ?

7. How many commands can be written to the history file when exiting your current shell
session ?

8. Make sure your current bash shell remembers the next 5000 commands you type.

9. Open more than one console (by press Ctrl-shift-t in gnome-terminal, or by opening
anextra putty.exe in MS Windows) with the same user account. When is command history
written to the history file ?

36
 shell history

16.13. solution: shell history

1. Issue the command echo The answer to the meaning of life, the universe and
everything is 42.
echo The answer to the meaning of life, the universe and everything is 42

2. Repeat the previous command using only two characters (there are two solutions!)
!!
OR
!e

3. Display the last 5 commands you typed.

paul@ubu1010:~$ history 5
52 ls -l
53 ls
54 df -h | grep sda
55 echo The answer to the meaning of life, the universe and everything is 42
56 history 5
You will receive different line numbers.

4. Issue the long echo from question 1 again, using the line numbers you received from the
command in question 3.
paul@ubu1010:~$ !55
echo The answer to the meaning of life, the universe and everything is 42
The answer to the meaning of life, the universe and everything is 42

5. How many commands can be kept in memory for your current shell session ?
echo $HISTSIZE

6. Where are these commands stored when exiting the shell ?

echo $HISTFILE

7. How many commands can be written to the history file when exiting your current shell
session ?
echo $HISTFILESIZE

8. Make sure your current bash shell remembers the next 5000 commands you type.
HISTSIZE=5000

9. Open more than one console (by press Ctrl-shift-t in gnome-terminal, or by opening
anextra putty.exe in MS Windows) with the same user account. When is command history
written to the history file ?
when you type exit

37