Devices, Linux Filesystems, Filesystem Hierarchy

Standard
Create Partitions and Filesystems

Key Knowledge Areas

 Use various mkfs commands to set up partitions and create various filesystems such as:
ext2, ext3, xfs, reiserfs v3 and vfat.

Linux File Systems

In order to persist data on a disk you need to create a file system on it first. At
installation time you will be asked which type of file system should be used to format
your configured block device before installation can occur.
Linux supports many filesystems the most popular being the extended (ext2,ext3,ext4)
file system family. The ext3 filesystem has been the default Linux filesystem over the
last several years but has recently begun to be replaced by ext4, the latest iteration of
the extended filesystem.
Ext4 is backwardly compatible with ext3 and ext3 and ext4 are both backwardly
compatible with ext2. Ext4 is considered an interim solution while a more modern
filesystem, called Btrfs, is developed. The features required from a modern filesystem
include, pooling, snapshots, checksums and integral multi-device spanning, all of which
btrfs will deliver in due course.
Ext2 File System
The Second Extended File System (ext2fs or ext2), sometimes referred to as the Linux
native filesystem, was developed in 1993 and became the dominant filesystem used on
Linux. Due to its widespread use over several years, ext2 is regarded as a well tested
and reliable filesystem. It is also consequently the best supported filesystem on Linux
with a range of management tools and utilities.
The utilities are part of the e2fsprogs package available on your Linux distribution and
are often installed by default. The utilities and tools included in the e2fsprogs package
include:
 e2fsck a fsck program that checks for and corrects
inconsistencies
 mke2fs used for creating ext2 file systems
 tune2fs used to modify file system parameters
 dumpe2fs which prints superblock and block group
information.
  debugfs used to manually view or modify internal structures
of the file system
Ext3 File System
The ext3 filesystem is an ext2 filesystem with journaling capabilities. Journaling
improves reliability and eliminates the need to check the file system after an unclean
shutdown, which results in faster start ups. One of the advantages of having ext3 being
backwardly compatible with ext2 is that it allow well-tested and mature file system
maintenance utilities, such as fsck and tune2fs for monitoring, checking and repairing
ext2 file systems to also be used with ext3 without major changes.
XFS File System
XFS is a high-performance journaling file system created by Silicon Graphics, originally
for their IRIX operating system. The code was donated by Silicon Graphics and ported
to the Linux kernel. XFS has a good reputation for speed and robustness and is
particularly good at handling large files. XFS is much less popular than ext and
consequently there are fewer tools and utilities to monitor and maintain XFS
filesystems. The XFS tools are found as part of the xfsprogs package on most Linux
distributions and include:
 xfs_fsr - Used to defragment mounted XFS file systems.
When invoked with no arguments, xfs_fsr defragments all
regular files in all mounted XFS file systems. This utility also
allows users to suspend a defragmentation at a specified
time and resume from where it left off later.
 xfs_bmap - Prints the map of disk blocks used by files in an
XFS filesystem. This map list each extent used by a specified
file, as well as regions in the file with no corresponding
blocks (i.e. holes).
 xfs_info - Prints XFS file system information.
 xfs_admin - Changes the parameters of an XFS file system.
The xfs_admin utility can only modify parameters of
unmounted devices/file systems.
 xfs_copy Copies the contents of an entire XFS file system to
one or more targets in parallel.
 xfs_metadump - Copies XFS file system metadata to a file.
The xfs_metadump utility should only be used to copy
unmounted, read-only, or frozen/suspended file systems;
otherwise, generated dumps could be corrupted or
inconsistent.
Reiserfs File System
ReiserFS is another general-purpose, journalled file system designed by Hans Reiser. It
is particularly efficient at handling a large number of small files. It was one of the first
journalling file systems for Linux and is often used in situations where the drive will store
a large number of small files. Other filesystems that are not as efficient at handling small
files, end up wasting a lot of space as their minimum block allocation often exceeds the
size of the file. Reiserfs, like the XFS system, also has a fewer number of tools and
utilities than the extended file system. Utilities for reiserfs are found in the reiserfsprogs
package and include reiserfsck.

File System Formatting

In order to create a filesystem you need to format the partition. To create a file systems
while running a Linux system you need to install the associated formatting tools which
should be available as a package. The formatting tools follow the naming convention of
the filesystem type preceded with the mkfs, for make filesystem.
The formatting tool for ext2 for example is mke2fs. Similarly the formatting tool for
the xfs file system is mkfs.xfs while for reiserfs it is mkfs.reiserfs. The mkfs command is
a wrapper around each filesystem specific tool and acts as a front for all the different file
system types. The syntax for using the mkfs command is:
mkfs –t <fstype> <DEVICE>
The ext3 filesystem is created by specifying an ext2 file system and passing in the -j
parameter to enable journaling support.
Example 1: Making a xfs filesystem

Example 2: Making a ext2 filesystem

Example 3: Making a ext3 filesystem

Formatting Swap Space

Swap space is made with the mkswap command. Swap space does not have a real
filesystem as the kernel does raw read and writes to swap space to enhance the speed
with which it can access cached memory pages. To create swap space you would run
 To activate the swap space you would run the command swapon; for example:

sed terms, files and utilities

 fdisk
 mkfs
 mkswap
Maintain the integrity of filesystems
Key Knowledge Areas
 Verify the integrity of filesystems
 Monitor free space and inodes
 Repair simple filesystem problems
Once a filesystem has been created on your block device you will want to know how to
monitor the filesystem and check it for errors, recovering from errors where possible.
Fortunately the filesystem provide several command and utilities to aid in the process.

Monitoring Disk Usage

The df (disk free) and disk usage (du) commands can be used to report on the amount
of free disk space and query how much space directories and files are using. df works
on a device level, as opposed to a directory level.
The df tool shows used and available disk space. By default this is given in blocks of
1K.
The du command will display disk usage. This is done on a per directory basis. du
cannot display available space since this information is only available at a device level.
The following command will list the current usage of the /etc directory in human
readable units (using the -h switch) and will only print the grand total (using the -
s switch):

File System Checking, Repair and Maintenance

As with filesystem monitoring there are numerous tools for maintaining a Linux
filesystem which are provided by the creators of the respective filesystem.
If the file system is damaged or corrupt, then the fsck utility should be run against the
partition (the minimum requirement is that the file system be unmounted or mounted
read-only).
fsck acts as a front that automatically detects the file system type of a partition. Then as
with mkfs, the tools fsck.ext2, fsck.ext3 or fsck.xfs will be called accordingly to carry out
the system check and, if necessary, repair. Since ext3 is the main filesystem type for
Linux there is a e2fsck command that only handles this filesystem type.
You invoke the filesystem check as follows explicitly specify a file system type with the
following syntax:
fsck –t <fstype> <device>
Example: Checking a reiserfs filesystem on the /dev/sdb10 device:
Ext File System Maintenance utilities
As the extended filesystem is the most widely used and deployed filesystem on Linux,
the tools for ext filesystem support are more numerous and comprehensive than for
other filesystems.
Ext File System Debug Commands
The debugfs and dumpe2fs are seldom used but can be useful in providing low level
information about an ext2 or ext3 filesystem.
debugfs [ -b blocksize ] [ -s superblock ] [ -f cmd_file ] [ -R request ] [ -V ] [ [ -w ] [ -c ] [
-i ] [ device ] ]
The debugfs program is an interactive file system debugger. It can be used to examine
and change the state of an ext2/3 file system.
Once in the debugfs shell, internal commands can then be used to change directory,
examine inode data, remove files, create links, dump the ext3 journal logs etc. While
this is a very powerful command, it should be used with caution, generally only after the
fsck command has failed to make any headway.
dumpe2fs [ -bfhixV ] [ -ob superblock ] [ -oB blocksize ] device
dumpe2fs prints the super block and block group information for the filesystem present
on device.
 tune2fs
tune2fs allows you to adjust various filesystem parameters on Linux extended
filesystems. The following is a list of the most common parameters used to adjust
extended filesystem settings:
 -c sets the number of times a filesystem will be mounted before a filesystem
check is forced. This is usually at next boot but can be run when a filesystem is
manually unmounted.
 -C sets the number of times the filesystem was mounted since it was last
checked.
 -L sets the volume label, this used to be used to uniquely identify hard disk
partitions but is being replaced with UUIDs.
 -i sets the maximum time between filesystem checks. A filesystem check will be
forced when either the time expires or the the maximum number of mounts has been
exceeded, which ever comes first.
 -j adds journaling to an ext2 filesysem making it an ext3 filesystem.
Running tune2fs -l will print out the current settings for a filesystem.
XFS File System Maintenance Utilities
The tools that come with XFS for filesystem integrity checking are xfs_info and
xfs_metadump.
 xfs_metdump is a filesystem debugging utility, that dumps xfs filesystem meta-data to a
file. The file can then be used to debug the files or as a backup. Later the meta-data can
be restored with the xfs_restore utility.

Used terms, files and utilities

 du
 df
 fsck
 e2fsck
 mke2fs
 debugfs
 dumpe2fs
 tune2fs
 xfs tools (such as xfs_metadump and xfs_info)

Used terms, files and utilities

 du
 df
 fsck
 e2fsck
 mke2fs
 debugfs
 dumpe2fs
 tune2fs
 xfs tools (such as xfs_metadump and xfs_info)
Control mounting and unmounting of filesystems
Key Knowledge Areas
 Manually mount and unmount filesystems
 Configure filesystem mounting on bootup
 Configure user mountable removable filesystems

Mounting Filesystems at Bootup

At boot time the /etc/fstab file assigns mount points for block devices.
The /etc/fstab format
 Sample /etc/fstab

The options that are available for use with fstab file:
 atime / noatime / relatime The Unix stat structure records when files are last accessed
(atime), modified (mtime), and created (ctime). One result is that atime is written every time a
file is read, which has been heavily criticized for causing performance degradation and
increased wear. However, atime is used by some applications and desired by some users, and
thus is configurable as atime (update on access), noatime (do not update), or (in Linux) relatime
(update atime if older than mtime). Through Linux 2.6.29, atime was the default; as of 2.6.30 (9
June 2009), relatime is the default,
 auto / noauto - With the auto option, the device will be mounted automatically at bootup
or when the mount -a command is issued. auto is the default option. If you don't want the device
to be mounted automatically, use the noauto option in /etc/fstab. With noauto, the device can be
only mounted explicitly.
 dev / nodev - Interpret/do not interpret block special devices on the filesystem.
 exec / noexec - exec lets you execute binaries that are on that partition, whereas noexec
doesn't let you do that. noexec might be useful for a partition that contains no binaries, like /var,
or contains binaries you don't want to execute on your system, or that can't even be executed
on your system. Last might be the case of a Windows partition.
 ro - Mount read-only.
 rw - Mount the filesystem read-write. Again, using this option might alleviate confusion
on the part of new Linux users who are frustrated because they can't write to their floppies,
Windows partitions, or other media,
 sync / async -How the input and output to the filesystem should be done. sync means it's
done synchronously. If you look at the example fstab, you'll notice that this is the option used
with the floppy. In plain English, this means that when you, for example, copy a file to the floppy,
the changes are physically written to the floppy at the same time you issue the copy command.
 suid / nosuid - Permit/Block the operation of suid, and sgid bits.
 user / users / nouser - user permits any user to mount the filesystem. This automatically
implies noexec, nosuid, nodev unless overridden. If nouser is specified, only root can mount the
filesystem. If users is specified, every user in group users will be able to unmount the volume.
 owner - Permit the owner of device to mount,
 defaults - Use default settings. Default settings are defined per file system at the file
system level. For ext3 file systems these can be set with the tune2fs command. The normal
default for Ext3 file systems is equivalent to rw,suid,dev,exec,auto,nouser,async(no acl
support). Modern Red Hat based systems set acl support as default on the root file system but
not on user created Ext3 file systems. Some file systems such as XFS enable acls by default.
Default file system mount attributes can be over ridden in /etc/fstab.

Manually Mounting and Unmounting Filesystems

The mount command is used to make a particular device available on a specific
directory (mount point). The syntax is:
mount -t [FSTYPE] -o [OPTION S] DEVICE DIRECTORY
For example we can mount a CDROM on the mount point /media/cdrom with:

Mount maintains the list of mounted filesystems in /etc/mtab. Typing mount with no
options will show all filesystems currently mounted. The output is similar to /etc/mtab.
The kernel also keeps track of mounted filesystems in /proc/mounts.
User Mountable Configurable Systems
On a running system the /etc/fstab file also acts as a shortcut for assigning a resource
to a specific directory. For example:

The mount utility reads fstab and deduces where to mount the resource. Notice that
some of the devices are accessed using a label. Labels are assigned to devices with
the tune2fs tool:

Mount can take most of the fstab options that have been listed above. This allows a
user to mount a filesystem read-only or with noexec for example. Beside mounting
filesystems that have been defined in the /etc/fstab file, mount can also be used to
mount new devices not defined in fstab. If an external SATA drive is plugged into the
machine for example, and given the device node of /dev/sdc, its first partition can be
mounted as
 This will make the 1st partition of the device available under /mnt directory

The umount command will unmount a device. Notice that teh command is misspelled!

The syntax is:
umount DEVICE or MOUNT-POINT
For example the following commands will both unmount the CDROM device:

or

Conventionally, mount points under /media are used for removable devices such as
CDs or USB storage devices.

Used terms, files and utilities

 /etc/fstab
 /media
 mount
 umount
Manage Disk Quotas
Key Knowledge Areas
 Set up a disk quotas for a filesystem
 Edit, check and generate user quota reports

Quota Setup
Disk quotas allow system administrators to allocate the maximum amount of disk space
a user or group's files and data may take up on a machine. This functionality can be
crucial on a Linux machine that is used as a file server to prevent the hard disks from
filling up and causing system instability.
 The quota tools allow administrators to set up quotas without having to reboot. Here are
the steps.
Edit /etc/fstab and add usrquota to the options
Remount the partition:

Start the quota stats:

The preliminary aquota.user database file is generated at the top of the directory.

Editing and Quota Reporting

Edit quotas for each user:

Here a soft/hard limit must be set for both the number of blocks and inodes available for
each user.
The system will allow the user to exceed the soft limit during a certain grace period.
After the grace period has expired the soft limit will be enforced as a hard limit.
Start enforcing quotas:

Uses can query the quota status with quota. The system administrator can generate
reports with repquota or quotastats.

Used terms, files and utilities

 quota
 edquota
 repquota
 quotaon
 Manage File Permissions and Ownership
Key Knowledge Areas
 Manage access permissions on regular and special files as well as directories
 Use access modes such as suid, sgid and the sticky bit to maintain security
 Know how to change the file creation mask
 Use the group field to grant file access to group members

File and Directory Permissions

Access to directories and files on Linux is controlled by a simple file permissions
systems. Every file/directory has permissions for the file owner, the group to which the
file belongs and other, that is users who are not the owner and do not belong to the
group to which the file belongs. The permissions are known as the access mode of
file/directory and can be viewed by running the ls -l command.
File access modes are displayed symbolically as group of 3 letters or numerically as a
set of 3 octal digits, but represent a 9 bit number, with each bit representing an access
right.

The extract above is from running the ls -l command on the /etc directory. When a file is
created it is owned by the user who created the file and assigned to the default group of
the owner.

Symbolic and Octal Notation

Permissions can be read=r, write=w and execute=x. The octal values of these
permissions are listed in the next table.
Octal and symbolic permissions.

Permissions apply to the user, the group and to others. An item has a set of 3 grouped
permissions for each of these categories.
How to read a 755 or -rwxr-x-r-x permission

The Standard Permissions & UMASK

UNIX system create files and directories with standard permissions as follows:
Standard permission for:
File 666 -rw-rw-rw-
Directories 777 -rwxrwxrwx
Every user has a defined umask that alters the standard permissions.
The umask applies only at the point at which the file is created. The umask has an octal
value and is subtracted(*) from the octal standard permissions to give the file's
permission (this permission doesn't have a name and could be called the file's effective
permission).
(*) While subtraction works in most cases, it should be noted that technically the
standard permissions and the umask are combined as follows:

On systems where users belong to separate groups, the umask can have a value of
002. For systems which place all users in the users group, the umask is likely to be 022
so that files do not have group write access by default.

Devices, Linux Filesystems, Filesystem Hierarchy

Standard
Continue

Changing permissions and owners

From the previous figure we see that permissions can be acted upon with chmod. There
are 3 categories of ownership for each file and directory:
u: user
g: group
o: other
Example:
-rw-rw-r-- 1jade sales 24880 Oct 25 17:28 libcgic.a
Changing Permissions with chmod:

Changing user and group with chown and chgrp:

NOTE:
A useful option for chmod, chown and chgrp is -R which recursively changes ownership
and permissions through all files and directories indicated.
Special Permissions
SUID Permissions
An executable can be assigned a special permission which will always make it run as
the owner of this file. This permission is called SUID meaning 'set user ID'. It has a
symbolic value s or a numerical value 4000.
Administrative tools may have the SUID bit set in order to allow non-root users to
change system files. For example the passwd command can be run by any user and will
interactively change his or her current password. This password will be saved
to /etc/shadow. However this file belongs to user root with typical permissions of 600.
This problem has been solved by setting the SUID bit on passwd hence forcing it to run
as user root with the correct permissions to modify /etc/shadow.
The SUID on passwd

NOTE:
The SUID bit is shown in symbolic form in the command above. It is possible to get
more information about a file using stat as well as seeing the octal representation of the
permissions as follows:
The next examples are dangerous. Why?

SGID permissions
The SGID is a permission similar to SUID that is set for group members. The symbolic
value is s and the octal value of 2000.
Setting SGID on a directory changes the group ownership used for files subsequently
created in that directory to the directory's group ownership. No need to use newgrp to
change the effective
group of the process prior to file creation.
Examples:
 The sticky bit
The sticky bit permission with value 1000 has the following effect:
 Applied to a directory it prevents users from deleting files unless they are the owner
(ideal for directories shared by a group, or for /tmp
 Applied to a file this used to cause the file or executable to be loaded into memory and
caused later access or execution to be faster. The symbolic value for an executable file is t . It
was
supported in some versions of Unix but is not used in Linux.
Examples:

Used terms, files and utilities

 chmod
 umask
 chown
 chgrp
Create and change hard and symbolic links
Key Knowledge Areas
 Create links
 Identify hard and / or softlinks
 Copying versus linking files
 Use links to support system administration tasks
When we copy a file with the cp command, a duplicate of that file is created, sometimes
however we want to provide a link to an existing file but want that path to point to the
exact same file as the original. In this case we will use symbolic links.

Symbolic links
A soft link to a file or a directory is a special file type that simply contains the name of
the file that it "points to".

Soft links can be created across filesystems. By running ls -l we can identify whether a
directory entry is a soft link or just an ordinary file from the output. A symbolic link is
shown as follows when we run the ls -l command
mytext.txt -> mytext.sym. Notice that the reference count is 1 for both files.
-rw------- 1 root root 223 Sep 29 09:10 mytext.txt
lrwxrwxrwx 1 root root 9 Sep 29 09:10 mytext.sym -> mytext.txt
To find all symbolic links to a file you can use the find command for example
find / -lname mytext.txt
will find all symbolic links to the file mytext.txt.

Hard Links
A hard link is an additional name for the same inode and as such the reference count of
the file increases by one for every new hard link.

In the listing notice that the reference count is 2 and that both files have the same size.
In fact they are identical.
-rw------- 2 mark mark 223 Sep 26 09:06 mytextfile.txt
-rw------- 2 mark mark 223 Sep 26 09:06 mytextfile.link
Hard links can only be created within the same filesystem. Using ls, a hard link can be
identified by the reference count shown in the output, as in the above example. Another
way of finding files with hard links is to obtain the file's inode number and then run the
find command with the inode number as a parameter. To find the inode of a file run the
command:

This will output the inode number of the file, next run the find command as follows:
 Used terms, files and utilities
 ln

Finding and Placing Files within the File Hierarchy

StandardKey Knowledge Areas
 Understand the correct locations of files under the FHS
 Find files and commands on a Linux system
 Know the location and purpose of important files and directories as defined in the FHS

The Linux File System

In general Linux filesystem layout is consistent across distributions, with minor
variations. The consistency is largely due to the existence of the Filesystem Hierarchy
Standard (FHS) project, which aims to provided a recommended standard layout for
Linux and Unix like operating systems.
Below is a listing of a the most important directories, and a brief explanation of their
purpose, commonly found on Linux systems:
 /bin and /sbin Contain binaries needed to boot up the system and essential commands.
 /dev Location for device or special files
 /etcHost specific configuration files
 /lib Shared libraries for binaries in /bin and /sbin. Also contains kernel modules
 /mnt/ or /media Mount point for external filesystems
 /proc Kernel information. Read-only except for /proc/sy s/
 /boot Contains the Linux kernel, the system maps and the “second stage” bootloaders.
 /home The directories for users. Initially contains the contents from /etc/skel/
 /root The directory for user root
 /tmp Temporary files
 /usr User Specific Resource. Mainly static and shareable content
 /usr/local or /opt (optional) Add-on software applications. Can also contain shared
libraries for add-on software.
 /var/www, /var/ftp/ Location for HTML pages and anonymous FTP directories.
 /var Variable data, such as spools and logs. Contains both shareable (eg.
/var/spool/mail) and non-shareable (eg. /var/log/) subdirectories.

 Finding Files and Directories

 We will describe the find, which, whereis and locate utilities.
  locate
 Syntax:
 locate <STRING>
 When using locate all files and directories that match the expression are listed.


 The search is much faster than find. In fact locate queries

the /var/lib/slocate/slocate.db database. This database is kept up to date via a
daily cron job which runs updatedb.
 When running updatedb from the command line the /etc/updatedb.conf file is
read to determine pruned files systems (e.g NFS) and directories (e.g /tmp)
 which
 Syntax:
 which string
 This tool will return the full path to the file called string by scanning the directories
defined in the user's PATH variable only. As a result which is only used to find
commands.
 whereis
 Syntax
 whereis string
 This tool will return the full path to source or binaries as well as documentation
files matching string by scanning the PATH variable as well as a number of well
known locations
 Getting the most from ls


 Contact  English (US) 

Used terms, files and utilities

 find
 locate
 updatedb
 whereis
 which
 type