Linux/Unix file system

1
 Disks
•Disks are secondary storage
which are very important to
SAs
–Large space
–Permanent storage
•Hard disks
–Physical structure: tracks,
sectors/blocks
•Cylinder
–All tracks at the same
radius

2
 Disks (cont.)
• Formatting
– Formatting is needed for a new disk to write some data onto the
magnetic surface of platters
– It is also called low-level format
– Damaged sectors are avoided and reported when formatting
• Disk controller
– A disk is controlled by a device controller (a control card with
circuits)
• Performance vs. reliability
• Abstract commands, such as read and write, are used by
applications.

3
 Disks (cont.)
• Disk driver
– The disk driver is a layer of software which translates disk requests
from the OS’s abstract commands into commands which the
controller understands
• Hard disk types
– Integrated Drive Electronics (IDE): cheap but a maximum of two disks
can be used. Speed is up to 133Mbytes/sec.
– Small Computer System Interface (SCSI): expensive but fast and reliable,
up to 640Mbytes/sec with 16 bits bus. Can deal with 5 to 16 disks. Some
extended SCSI can have 16,256 disks.
– Serial ATA (SATA): Serial Advanced Technology Attachment. ATA is
also known as IDE, but IDE is now called Parallel ATA. Speed can be up
to 300MBytes/sec.
– SAS (Serial Attached SCSI): serial P2P protocol replacing parallel SCSI.
• Always power down when handling a disk!
• Solid State Drive (SSD)
4
 RAID
• RAID: Redundant Array of Independent Disks
• Redundancy levels
– RAID 0: Non-redundant
– RAID 1: Mirrored
– RAID 2: Redundancy through Hamming Code
– RAID 3: Bit-interleaved parity
– RAID 4: Block-level parity
– RAID 5: Block-level distributed parity
– RAID 6: Dual redundancy
• Refer to COSC243 textbook for computer architectures.

5
 Other devices
• USB devices: flash memory, hard drive, etc.
• Floppy disk
– Density: 1.2M or 1.44M
– Similar to hard disk except small and slow
• CD-ROM
– ISO 9660 file system, UDF for DVD-ROM
• Tapes
– Important backup media but will not be covered in this course
– Check related sites for more info.
• Device naming convention under /dev
– sda1,sda2,…, sdb1, sdb2, …,hda1, hda2,…, hdb1, hdb2,
…, fd0,fd1,…
– Disks are labelled as a,b,…, and partitions are recognised by 1, 2,
3, ...

6
 Linux/Unix directory tree
• Refer to http://www.tldp.org/LDP/Linux-
Filesystem-Hierarchy/html/index.html

7
 Unix file
•
system
A hierarchical file system
– A convenient way of organising data in directories, sub-directories,
and so on
• File system protocols
– Open, read/write, …, close a stream of bytes
• File types
– Executables, text, etc
– Different OSs use different ways to mark the file types
• Permissions and access in UNIX
– Each file may be readable, writable, and executable
– Each file has an owner and belongs to one group
– The owner can set access rights for himself, group members, and
others
– 9 bits for access rights, 3 special bits (setuid, setgid, sticky), 4 bits
to tell the type of file (directory, soft link, etc.)
8
 Access control models
• Discretionary access control
– a means of restricting access to objects based on the
identity of subjects and/or groups to which they belong.
The controls are discretionary in the sense that a subject
with a certain access permission is capable of passing
that permission (perhaps indirectly) on to any other
subject (unless restrained by mandatory access control).
– Subject centered
• Mandatory access control
– a means of restricting access to objects based on the
sensitivity (as represented by a label) of the information
contained in the objects and the formal authorization
(i.e., clearance) of subjects to access information of
such sensitivity
– Object centered
9
 Implementation of file
• systems
Implementation of UNIX file systems
– Basic units: blocks, default size 4096, can be
adjusted when a file system is created.
– Two structures are created for a file system
• Superblock (with backups) and inodes

Boot Super- Inode Data … …

block block bit map bit map

inodes data blocks

10
 Implementation of file
(cont.)
systems
• Superblock
– contains the info on the boundaries of the
partition, info about where the inode table (and number
of inodes), where data blocks start (and their size), the
first inode (root /), etc. If the superblock is lost or
damaged, the whole file system would be unreadable. It
is important to make superblock backups when a file
system is created, e.g ext2. The Linux file system check
program fsck can do this.

11
 Implementation of file
(cont.)
•
systems
Inode (index node) is the data structure which holds the specific data
about a particular file. Regardless of how large a file is, there is exactly
one inode per file.
• Three ways of addressing data are used in inodes
– Direct addressing: for files < 48KB
– Indirect addressing: for files <4 MB
– Double-indirect: for files < 4GB
– Triple-indirect: for files < 4TB
• When a file system is created it creates a fixed number (which can be
decided by SA) of inodes.
• How to decide the number of inodes?

12
 Implementation of file
systems (cont.)

COSC301 Lecture 5: Linux/Unix file … 13

Implementation of file
systems (cont.)
– Inodes elements: the device on which the file resides,
the user id and the group id of the owner, the type of
file and its protection bits, timestamps indicating the
last time the file was written to, size of the file, and
etc
• 16 bits are used for protection and other attributes, only 12 bits
are changeable by users
• rwxrwxrwx, sticky bit, setuid bit, setgid bit
• rwxrwxrwx can be expressed by numbers such as 666 or 777
• The leading 4 bits are used to represent plain file, directory,
soft link, or other purposes (modern OS use 32 bits now!)
– Inodes that are not in use are kept in a doubly linked list
called the free list.
– File names are stored in a directory structure (which
again is a file), not in the inodes themselves, with
pointers in the directory to 14the appropriate inodes.
 Structure of a
directory
A directory is a
file!
s.txt
inode

data blocks

15
 Soft/hard links
• Links (use ln to create links)
– Objects which appear in the file system and look just
like files
– Pointers to other files which are elsewhere in the strict
hierarchy
• Symbolic (soft) links
– A small file containing the name of the true file
– Can point to a non-existing file (broken link)
• Hard links
– More permanent
– To delete a file with hard links, all of the hard links
must be removed
16
 Soft/hard links (cont.)
file hlink slink Directory entries

N1 I-nodes
N2

file data “…/file” data

17
 Advanced file
• s-bits attributes
and t-bit (sticky bit)
– Apart from the read, write, and execute file attributes,
UNIX has three other flags
• Setuid bit
– chmod u+s file
– For executable files, this bit tells UNIX that regardless
of who runs the program it should be executed with the
privilege of the owner of the file.
– System admins should be aware of this because some
root programs may be run by users to allow them
limited access to root privileges (e.g. ps).
Warning: security holes!
18
 Advanced file
• Setgidattributes
bit
– chmod g+s file
– Similar to setuid bit except the program is run with the rights of the
owner’s group
– In BSD UNIX , System 5 and Linux, if this bit is set on a directory
then any new files created in that directory assume the group
ownership of the parent directory, not the login group of the user
who created the file.
• Sticky bit (t-bit)
– A file with this bit set can be kept in memory once loaded into
memory for performance reasons (obsolete in modern OS)
– A directory with this bit set restricts the deletion of files within it
(only the owner or superuser can delete her files though the
directory is set mode 777)
– Example - files under /tmp

19
 Special files
• /proc in unix (also /sys in Linux 2.6)
– Looks like a file system. It is a virtual file system. Contents are
created on-demand.
– An easy way to access kernel data structures such as the process
list
• Files in /dev
– Certain files in the UNIX OS are not really files but “handles” to
devices. They are called device nodes.
– A device node is a way into a device through a file system
interface. It is convenient to be able to use normal file operations to
access devices
– mknod is used to make i-nodes for devices
– Major and minor numbers are used for devices
• Check /usr/src/linux/Documentation/devices.txt for details
on device major and minor numbers
– Example: /dev/fd0 - the first floppy drive.

20
 Backups
• Commands for backups
– tar, dd, rsync (for incremental backups)
• Full and incremental backup
– Full backup is a copy of every file
– An incremental backup is a copy of only those files which have
changed since the last backup was taken
– Incremental backup works on the concept of levels, 0,1,2,3,4,
…9.
At each incremental backup level, you back up everything that
has
changed since the previous backup at the same or lower level
(higher number)
• Backup policies
– Daily backup, Weekly backup, Monthly
21 backup
 Journaling
• Logs changes to a journal before actually writing
to the main file system
– Journaling can have a severe impact on performance
because it requires that all data be written twice
• Metadata-only journaling
– a compromise between reliability and performance that
stores only changes to file metadata (which is usually
relatively small and hence less of a drain on
performance) in the journal.
– ensures that the file system can recover quickly
– but leaves an opportunity for data corruption because
unjournaled file data and journaled metadata can fall
out of sync with each other.
22
 ACL
• ACL: Access Control List
– more secure and convenient than traditional access control
• ACL is a list of permissions attached to an object (file)
– The list specifies who or what is allowed to access the
object and what operations are allowed to be performed
on the object
– ACL consists of entries like [user, operations]
– The operations can be R, W, X, D, and etc.

23
 Performance issues
• Block size
– How big should a block be?
• Fragmentation
– Free blocks are fragmented, which results in
that blocks of a file are scattered around the
disk.
– Can slow down disk performance

24
 Summary
• Access control models
• Unix/Linux access control bits
• Advanced file attributes (s-bits and t-bit)
• Difference of soft link and hard link
• What is incremental backup in
comparison with full backup?

25