Introduction to

Linux

Arunjith A.S
Topics for Today

● Linux Overview

● Linux Interaction - Shell and Commands

● I/O redirection (pipes, etc.)

● Navigating the file system

● virtual environment python

© 2023, Probeplus. All rights reserved. 2

 Linux
What, Who, When, Where & Why

© 2023, Probeplus. All rights reserved. 3

What is Linux

●
Unix-like computer operating system assembled under the model of free and open-source software
development and distribution.

●
These operating systems share the Linux kernel.
●
Typically have the GNU utilities

●
Comes in several “distributions” to serve different purposes.

4
What is Linux
●
Bird’s eye view
Who is Linux

&

> Linux is an O/S core A set of programs written by

originally Richard Stallman and others.
written by Linus Torvalds. Now They are the GNU utilities.
almost 10,000 developers
including major technology
companies like Intel and IBM.
Where is Linux
● World Wide Web
○ 67% of the world’s web-servers run Linux (2016)

● Research/High-Performance Compute
○ Google, Amazon, NSA, 100% of TOP500 Super-computers.

● Modern Smartphones and devices

○ The Android phone

○ Amazon Kindle

○ Smart TVs/Devices
Why Linux
● Free and open-source.

● Powerful for research datacenters

● Personal for desktops and phones

● Universal

● Community (and business) driven.

Connecting
Let’s use Linux
Connection Protocols and Software
Remote Connections: Remote Graphics: Data Transfer:

Secure Shell X-Windowing Secure File Transfer Protocol

(SSH) (X, X-Win) (SFTP)

Other protocols too, but let’s start with these.

Microsoft Windows
You need software that emulates an “X” terminal and
that connects using the “SSH” Secure Shell protocol.

● Recommended: MobaXterm

○ Download: http://mobaxterm.mobatek.net/

● Alternatives:

○ SSH/X-Windows: X-Win32
https://www.bu.edu/tech/services/support/desktop/distribution/xwindows/

○ SFTP: Filezilla
https://filezilla-project.org/
Apple macOS
● SSH: Terminal

○ Built in to macOS
Applications > Utilities > Terminal

● X-Windows: Xquartz

○ Download: https://www.xquartz.org/
○ Note: This install requires a logout.

● SFTP: Your choice

○ Filezilla: https://filezilla-project.org/ (Cross-platform, open-source)

○ Cyberduck: https://cyberduck.io (macOS native, drag-and-drop)
○ Many others
Linux

● SSH: Terminal

○ Built in to Linux
Applications > System > Terminal

● X-Windows: X11

○ Built in to Linux
○ Use your package manager.

● SFTP: Your choice

○ Usually has one Built in.

○ Alternate: Filezilla (https://filezilla-project.org/)
Connecting
● Use your Shared Computing Cluster account if you have one.

● Tutorial accounts if you need one.

○ Username : ***********
○ Password : ***********
Linux: The Shell
● Program that interprets commands and sends them to the OS

● Provides:

○ Built-in commands

○ Programming control structures

○ Environment variables

● Linux supports multiple shells.

○ The default on SCC is Bash.

Linux: The “prompt”

( In Linux “ ~ ” is a shorthand for your home directory. )

Linux: Command Basics

● Command: Command/program that does one thing

● Options: Change the way a command does that one thing

○ Short form: Single-dash and one letter e.g. ls -a
○ Long form: Double-dash and a word e.g. ls --all

● Argument: Provides the input/output that the command interacts with.

For more information about any command, use man or info (e.g. “man ls”)
Commands: Hands-On
● After you connect, type
○ whoami # my login
○ hostname # name of this computer
○ echo “Hello, world” # print characters to screen
○ echo $HOME # print environment variable
○ echo my login is $(whoami) # replace $(xx) with program output
○ date # print current time/date
○ cal # print this month’s calendar
○ shazam # bad command
Commands: Hands-On Options
● Commands have three parts; command, options and arguments/parameters.

Example: cal –j 3 1999. “cal” is the command, “-j” is an option (or switch), “3” and “1999”
are arguments/parameters.

[username@scc1 ~]$ cal -j 3 1999

● What is the nature of the prompt?

● What was the system’s response to the command?

Commands: Selected text processing utilities
● awk Pattern scanning and processing language
● cat Display file(s)
● cut Extract selected fields of each line of a file
● diff Compare two files Just a few of the
● grep Search text for a pattern commands for
● head Display the first part of files text processing
● less Display files on a page-by-page basis
● sed Stream editor (esp. search and replace)
● sort Sort text files
● split Split files
● tail Display the last part of a file
● tr Translate/delete characters
● uniq Filter out repeated lines in a file
● wc Line, word and character count
Variables and Environment Variables
● Variables are named storage locations.

○ USER=augustin
○ foo=“this is foo’s value”

● “Environment variables” are variables used and shared by the shell

○ For example, $PATH tells the system where to find commands.

● Environment variables are shared with programs that the shell runs.
Bash variables
● To create a new variable, use the assignment operator ‘=‘

● The foo variable can be printed with echo

● To make $foo visible to programs run by the shell (i.e., make it an

“environment variable”), use export:
Environment Variables
● To see all currently defined environment variable, use printenv:
Command History and Command Line Editing
● Try the history command

● Choose from the command history using the up ↑ and down ↓ arrows

● To redo your last command, try !!

● To go further back in the command history try !, then the number as shown by history (e.g., !132). Or, !ls,
for example, to match the most recent ‘ls’ command.

● What do the left ← and right → arrow do on the command line?

● Try the <Del> and <Backspace> keys

Help with Commands
● Type

○ date –-help
○ man date
○ info date

● BASH built-ins

○ A little different from other commands

○ Just type the command ‘help’
○ Or ‘man bash’
On using ‘man’ with ‘less’
● The man command outputs to a pager called less, which supports
many ways of scrolling through text:

○ Space, f # page forward

○b # page backward
○< # go to first line of file
○> # go to last line of file
○/ # search forward (n to repeat)
○? # search backward (N to repeat)
○h # display help
○q # quit help
I/O Redirection
I/O redirection with pipes
● Many Linux commands print to “standard output”, which defaults to the terminal screen. The ‘|’ (pipe)
character can be used to divert or “redirect” output to another program or filter.

○w # show who’s logged on

○ w | less # pipe into the ‘less’ pager

○ w | grep ‘tuta’ # pipe into grep, print lines containing ‘tuta’

○ w | grep –v ‘tuta’ # print only lines not containing ‘tuta’

○ w | grep ‘tuta’ | sed s/tuta/scholar/g # replace all ‘tuta’ with ‘scholar

More examples of I/O redirection
● Try the following (use up arrow to avoid retyping each line):

○ w | wc # count lines
○ w | cut –d ‘ ’ –f1 | sort # sort users
○ w | cut –d ‘ ’ –f1 | sort | uniq # eliminate duplicates

● We can also redirect output into a file:

○ w | cut –d ‘ ’ –f1 | sort | uniq > users

● Note that ‘awk’ can be used instead of ‘cut’:

○ w | awk ‘{print $1;}’ | sort | uniq > users
The Filesystem
The Linux File System
● The structure resembles an upside-down tree
● Directories (a.k.a. folders) are collections of files and other directories.
● Every directory has a parent except for the root directory.
● Many directories have subdirectories.
Navigating the File System

● Essential navigation commands:

○ pwd print current directory

○ ls list files

○ cd change directory
Navigating the File System
We use pathnames to refer to files and directories in the Linux file system.
● There are two types of pathnames:

○ Absolute – The full path to a directory or file; begins with /

○ Relative – A partial path that is relative to the current working directory;

does not begin with /
Navigating the File System
● Examples:
○ cd /usr/local Change directory to /usr/local/lib

○ cd ~ Change to home directory (could just type ‘cd’)

○ pwd Print working (current) directory

○ cd .. Change directory to the “parent” directory

○ cd / Change directory to the “root”

○ ls –d pro* Listing of only the directories starting with “pro”

The ls Command
● Useful options for the “ls” command:

○ ls -a List all files, including hidden files beginning with a “.”

○ ls -ld * List details about a directory and not its contents

○ ls -F Put an indicator character at the end of each name

○ ls –L Simple long listing

○ ls –lR Recursive long listing

○ ls –lh Give human readable file sizes

Some Useful File Commands
● cp [file1] [file2] copy file
● mkdir [name] make directory
● rmdir [name] remove (empty) directory
● mv [file] [destination] move/rename file
● rm [file] remove (-r for recursive)
● file [file] identify file type
● less [file] page through file
● head -n N [file] display first N lines
● tail -n N [file] display last N lines
● ln –s [file] [new] create symbolic link
● cat [file] [file2…] display file(s)
● tac [file] [file2…] display file in reverse order
● touch [file] update modification time
● od [file] display file contents, esp. binary
Manipulating files and directories
● Examples:

○ cd # The same as cd ~
○ mkdir test
○ cd test
○ echo ‘Hello everyone’ > myfile.txt
○ echo ‘Goodbye all’ >> myfile.txt
○ less myfile.txt
○ mkdir subdir1/subdir2 # Fails. Why?
○ mkdir -p subdir1/subdir2 # Succeeds
○ mv myfile.txt subdir1/subdir2
○ cd ..
○ rmdir test # Fails. Why?
○ rm –rf test # Succeeds
Symbolic links
● Sometimes it is helpful to be able to access a file from multiple locations within the hierarchy. On a
Windows system, we might create a “shortcut.” On a Linux system, we can create a symbolic link:

○ mkdir foo # make foo directory

○ touch foo/bar # create empty file

○ ln –s foo/bar . # create link in current dir.

Finding a needle in a haystack
● The find command has a rather unfriendly syntax, but can be exceedingly helpful for locating files in
heavily nested directories.

● Examples:

○ find ~ -name bu –type d # search for “bu” directories in ~

○ find . –name my-file.txt # search for my-file.txt in .
○ find ~ -name ‘*.txt’ # search for “*.txt” in ~

● Quiz:

○ Can you use find to locate a file called “needle” in your haystack directory?
○ Extra credit: what are the contents of the “needle” file?
Processes & Job Control
Processes and Job Control
● As we interact with Linux, we create numbered instances of running programs called “processes.” You
can use the ‘ps’ command to see a listing of your processes (and others!). To see a long listing, for
example, of all processes on the system try:

● To see all the processes owned by you and other members of the class, try:
Processes and job control
● Use “top” to see active processes.
Foreground/background
● Thus far, we have run commands at the prompt and waited for them to complete. We call this running in
the “foreground.”
foreground

● Use the “&” operator, to run programs in the “background”,

○ Prompt returns immediately without waiting for the command to complete:

Process Control Practice
● Let’s look at the “countdown” script, in your scripts folder for practice

● Make the script executable with chmod:

● First, run it for a few seconds, then kill with Control-C.

Process control
● Type ‘ps’ to see your countdown process.

● Also, try running ‘jobs’ to see any jobs running in the background from this bash shell.

● To kill the job, use the ‘kill’ command, either with the five-digit process id:

○ kill 54356

● Or, you can use the job number (use ‘jobs’ to see list) with ‘%’:

○ kill %1
Backgrounding a running job with C-z and ‘bg’

● Sometimes you start a program, then decide to run it in the background.

Packages
 install packages
●
APT (Advanced Package Tool):
○ The APT package manager is a powerful and commonly used method for installing and managing software
packages on Ubuntu. You can use command-line tools like apt-get, apt, and aptitude to install packages.

●
APT GUI Tools:
○ Ubuntu includes graphical package management tools that make it easy for users who prefer a graphical interface
to search for, install, and remove software. The two most common GUI package management tools are:
○ "Software Center" or "Ubuntu Software": The official graphical software center for Ubuntu.
○ "Synaptic Package Manager": A more advanced graphical package manager with additional features.
●
Snap Packages:
○ Snap is a universal package format and packaging system supported on Ubuntu. You can use the
snap command to install, manage, and update Snap packages. Snap packages are containerized
and can be installed without affecting the system's libraries or dependencies.

●
Flatpak Packages:
○ Flatpak is another universal package format and packaging system that can be used on Ubuntu. You can use t
flatpak command to install and manage Flatpak applications.

●
PPAs (Personal Package Archives):
○ A PPA is a software repository maintained by individuals or organizations that contain packages not included
in the official Ubuntu repositories. You can add a PPA to your system and use the APT package manager to
install packages from it.
●
Source Code Compilation:
○ Some software may not be available as pre-compiled packages. In such cases, you can download the
source code and compile it on your system. This method is more advanced and requires development
tools (e.g.,build-essential) to be installed.

●
Deb Packages:
○ You can download Debian packages (.deb files) from trusted sources and use tools like dpkg to
install them manually.

●
Other Package Managers:
○ While APT is the default package manager on Ubuntu, you can also use other package managers like
dpkg,gdebi, and alien to install packages. These tools provide additional functionality or compatibility with
non-Debian packages.
 ●
AppImage Packages:
○ AppImage is another packaging format used to distribute software that includes
all dependencies. You can download and run AppImage files without installation.

Each of these methods offers different features and capabilities, and the choice of which one to use may depend
on your specific requirements and preferences. The APT package manager is the most common and
recommended way to install and manage software on Ubuntu due to its comprehensive support for the
distribution's software repositories. However, Snap, Flatpak, and other methods provide additional flexibility and
access to a wide range of software.
environment (for python)
 why we have to use python virtualenv
●
Isolation:
○ Virtual environments allow you to create isolated Python environments for different projects. Each virtual
environment can have its own set of Python packages and dependencies, which are independent of the
system-wide Python installation. This isolation prevents conflicts between packages used in different
projects.
●
Dependency Management:
○ With virtual environments, you can manage project-specific dependencies separately. This means you can
install, update, and remove packages without affecting other projects. It also ensures that the correct
versions of packages are used for each project.
●
Version Compatibility:
○ Virtual environments enable you to use different Python versions for different projects. This is crucial when
working on projects that require specific Python versions. You can create a virtual environment for Python
3.6 for one project and a different environment for Python 3.8 for another.
●
Cleaner Development:

○ Virtual environments help maintain a clean and organized development environment. You don't clutter the
system-wide Python installation with project-specific packages, which can lead to a messy and hard-to-maintain
system.

●
Reproducibility:
○ Virtual environments make it easier to reproduce the development environment on different machines. By
sharing the requirements.txt or Pipfile of a project, others can create the same virtual environment with the same
dependencies.

●
Security:

○ Isolated environments enhance security by reducing the risk of system-wide package conflicts and
vulnerabilities. If a project requires an older, potentially vulnerable version of a package, it won't impact other
projects.
●
Easier Cleanup:

○ When you're done with a project, you can simply delete its virtual environment, ensuring that no residual
project-specific files or packages are left behind on your system.
●
Development Workflow:

○ Many Python development tools and workflows are built around the concept of virtual environments. For
example, tools like pip, virtualenv, and package management systems like Pipenv work seamlessly with
virtual environments.

●
Testing and Isolation:

○ Virtual environments are valuable for testing your Python code in isolation. You can run unit tests, integration
tests, and other tests in a controlled environment to ensure that they do not interfere with the system or
other projects.

●
Compatibility with Non-root Users:

○ Virtual environments can be created and managed without requiring administrator (root) privileges.
This makes it accessible to users who don't have system-wide installation rights.
how to install ?
●
Install pip (if not already installed):

●
Install virtualenv:

●
Verify the installation:
●
Create Virtual Environment:

●
Activate Virtual Environment:

●
Deactivate Virtual Environment:
?