UNIT V - STORAGE MANAGEMENT

File Systems provides the mechanism for on-line storage and access to both data
and programs of the operating system and all the users of the computer system.

Mass Storage Structure- Overview

Most computer systems provide secondary storage (Mass Storage) as an extension of main
memory. The main requirement for secondary storage is that it be able to hold large quantities
of data permanently (Non-Volatile memory). The most common secondary-storage device is a
magnetic disk, which provides storage for both programs and data. Most of the secondary
storage devices are internal to the computer such as the hard disk drive, the tape disk drive and
even the compact disk drive and floppy disk drive.

Magnetic Disks
Magnetic disks provide the bulk of secondary storage for modern computer systems. Each disk
platter has a flat circular shape, like a CD. Common platter diameters range from 1.8 to 3.5
inches. The two surfaces of a platter are covered with a magnetic material. We store
information by recording it magnetically on the platters. A read–write head flies just above
each surface of every platter. The heads are attached to a disk arm that moves all the heads as
a unit. The surface of a platter is logically divided into circular tracks, which are subdivided
into sectors.

Fig 4.1 Magnetic Disk

CYLINDER: The set of tracks that are at one arm position makes up a cylinder
 Fig 4.2 Moving Head Disk Mechanism

The storage capacity of common disk drives is measured in gigabytes. When the disk is in use,
a drive motor spins it at high speed. Most drives rotate 60 to 250 times per second, specified in
terms of rotations per minute.

Magnetic Tapes
Magnetic tape was used as an early secondary-storage medium. It is relatively permanent and
can hold large quantities of data. Its access time is slow compared with that of main memory
and magnetic disk.

In addition, random access to magnetic tape is about a thousand times slower than random
access to magnetic disk, so tapes are not very useful for secondary storage. Tapes are used
mainly for backup, for storage of infrequently used information, and as a medium for
transferring information from one system to another.

Disk Structure
Magnetic disk drives are addressed as large one-dimensional arrays of logical blocks, where
the logical block is the smallest unit of transfer. The size of a logical block is usually 512 bytes,
although some disks can be low-level formatted to have a different logical block size, such as
1,024 bytes. The one-dimensional array of logical blocks is mapped onto the sectors of the disk
sequentially. Sector 0 is the first sector of the first track on the outermost cylinder. The number
of sectors per track is not constant on some drives.

1
For the disks that use Constant Linear Velocity (CLV), the density of bits per track is uniform.
In Constant Angular Velocity (CAV) the density of bits decreases from inner tracks to outer
tracks to keep the data rate constant.

Disk Scheduling
Whenever a process needs I/O to or from the disk, it issues a system call to the operating
system. If the desired disk drive and controller are available, the request can be serviced
immediately. If the drive or controller is busy, any new requests for service will be placed in
the queue of pending requests for that drive. When one request is completed, the operating
system chooses which pending request to service next. This is known as Disk Scheduling.

The request specifies several pieces of information:

• Whether this operation is input or output

• What the disk address for the transfer is
• What the memory address for the transfer is
• What the number of sectors to be transferred is
Disk Components

The three major components of the hard disk are Seek time and Rotational Latency and
bandwidth.

Seek time: The seek time is the time for the disk arm to move the heads to the cylinder
containing the desired sector.

Rotational latency: The rotational latency is the additional time for the disk to rotate the
desired sector to the disk head.

Disk bandwidth: The disk bandwidth is the total number of bytes transferred, divided by the
total time between the first request for service and the completion of the last transfer.

Disk Scheduling Algorithms

• First Come First Serve

• Shortest Seek Time First

• Scan Algorithm

• Circular Scan Algorithm

• Look Algorithm

• Circular Look Algorithm

2
FCFS Scheduling:

The simplest form of disk scheduling is, of course, the first-come, first-served (FCFS)
algorithm.

Example: Consider, for example, Given a disk with 200 cylinders and a disk queue with
requests 98, 183, 37, 122, 14, 124, 65, 67, for I/O to blocks on cylinders. Disk head is initially
at 53.

Fig 4.3 FCFS Scheduling

Current Next Block to No. of Head Movement
Block Be Accessed for Each Access
53 98 45
98 183 85
183 37 146
37 122 85
122 14 108
14 124 110
124 65 59
65 67 02
Total head movements 640

Fig 4.4 Total Head Movements Calculation for FCFS

Total No of head Movements of for FCFS is 640.

Disadvantage:

Unnecessary head movements is possible. Ex : The request from 122 to 14 and then back to
124 increases the total head movements.

3
SSTF Scheduling:

The shortest-seek-time-first (SSTF) algorithm selects the request with the least seek time
from the current head position. It chooses the pending request closest to the current head
position.

Fig 4.5 SSTF Scheduling

Disadvantage:

SSTF may cause starvation of some requests.

SCAN Scheduling:

In the SCAN algorithm, the disk arm starts at one end of the disk and moves toward the other
end, servicing requests as it reaches each cylinder, until it gets to the other end of the disk. At
the other end, the direction of head movement is reversed, and servicing continues. The head
continuously scans back and forth across the disk. The SCAN algorithm is sometimes called
the elevator algorithm.

Fig 4.6 SCAN Scheduling

Current Next Block to No. of Head Movement

Block Be Accessed for Each Access
53 37 16
37 14 23
4
 14 0 14
0 65 65
65 67 02
67 98 31
98 122 24
122 124 02
124 183 59
Total head movements 236

Fig 4.7 Total Head Movements Calculation for SCAN

Total No of head Movements of for SCAN is 236. Similarly Calculate for all other
algorithm.

Circular SCAN Algorithm:

Circular SCAN (C-SCAN) scheduling is a variant of SCAN designed to provide a more

uniform wait time.

C-SCAN moves the head from one end of the disk to the other, servicing requests along the
way. When the head reaches the other end, however, it immediately returns to the beginning
of the disk without servicing any requests on the return trip.

The C-SCAN scheduling algorithm essentially treats the cylinders as a circular list that wraps
around from the final cylinder to the first one.

Fig 4.8 Total Head Movements Calculation for SCAN

LOOK scheduling:

5
The LOOK algorithm is the same as the SCAN algorithm in that it also services the requests
on both directions of the disk head, but it ―Looks" ahead to see if there are any requests
pending in the direction of head movement.

If no requests are pending in the direction of head movement, then the disk head traversal will
be reversed to the opposite direction and requests on the other direction can be served.

In LOOK scheduling, the arm goes only as far as final requests in each direction and then
reverses direction without going all the way to the end

C-LOOK Scheduling:

This is just an enhanced version of C-SCAN.

Arm only goes as far as the last request in each direction, then reverses direction immediately,
without servicing all the way to the end of the disk and then turns the next direction to provide
the service.

File System Interface

File Concept : A file is defined as a named collection of related information that is stored
on secondary storage device. Many different types of information may be stored in a file such
as source or executable programs, numeric or text data, photos, music, video, and so on. A file
has a certain defined structure, which depends on its type.

Types of Files:

• A text file is a sequence of characters organized into lines.

• A Source file is a sequence of functions, each of which is further organized as
declarations followed by executable statements.
• An Executable file is a series of code sections that the loader can bring into memory
and execute.

File type Usual Extension Function

Executable exe, com, bin or none ready-to-run machine-language program

Object obj, o Compiled, machine language, not linked

Source code c, cc, java, perl, asm source code in various languages

Batch bat, sh Commands to the command interpreter

Markup xml, html, tex textual data, documents

Word processor xml, rtf, docx various word-processor formats

Library lib, a , so, dll Libraries of routines for programmers

6
 ASCII or binary file in a format for printing
Point or view gif, pdf, jpg
or viewing
related files are grouped, compressed for
Archive rar, zip, tar viewing
archiving or storage
mpeg, mov, mp3, mp4, avi binary file containing audio or A/V
Multimedia
information

Fig 4.10 File Types

FILE ATTRIBUTES:
A file’s attributes vary from one operating system to another but typically consist of these:

Name: The file name is the information kept in human readable form.

Identifier: This unique tag, usually a number, identifies the file within the file system

Type: This information is needed for systems that support different types of files.

Location: This information is a pointer to a device and to the location of the file on that device.

Size: The current size of the file (in bytes, words, or blocks)

Protection: Access-control information determines who can do reading, writing, executing

Time, date, and user identification: This information may be kept for creation, last
modification, and last use.

FILE OPERATIONS:

A file is an abstract data type. The operating system can provide system calls to create, write, read,
reposition, delete, and truncate files. The Basic Operations on a file includes

• Creating a File

• Writing a File

• Reading a File

• Repositioning within a File

• Deleting a file

• Truncating a file

FILE LOCKS:

File locks allow one process to lock a file and prevent other processes from gaining access to
it. File locks are useful for files that are shared by several processes. 2 types of locks are

Shared Lock: A shared lock is similar to a reader lock in that several processes can acquire
the lock concurrently.

7
Exclusive Lock: An exclusive lock behaves like a writer lock; only one process at a time can
acquire such a lock.

File Access Methods

Files store information. The information in the file can be accessed in several ways.

Sequential access Method: The simplest access method is sequential access. Information in
the file is processed in order, one record after the other.

Direct Access Method: A file is made up of fixed-length logical records that allow programs
to read and write records rapidly in no particular order.

Indexed Access Methods: These methods generally involve the construction of an index for
the file. The index contains pointers to the various blocks. To find a record in the file, we first
search the index and then use the pointer to access the file directly and to find the desired
record. With large files, the index file itself may become too large to be kept in memory.

One solution is to create an index for the index file. The primary index file contains pointers
to secondary index files, which point to the actual data items.

File Systems Mounting

File System Mounting is defined as the process of attaching an additional file system to the
currently accessible file system of a computer. A file system is a hierarchy of directories that
is used to organize files on a computer or storage media. The operating system is given the
name of the device and the mount point.

Mount Point: It is the location within the file structure where the file system is to be attached.
A mount point is an empty directory.

Example: A file system containing a user’s home directories might be mounted as /home. To
access the directory structure within that file system, we could precede the directory names
with /home, as in /home/Jane. Mounting that file system under /users would result in the path
name /users/Jane, which we could use to reach the same directory.

8
 Fig 4.11 File System (a) Existing System (b) Un mounted Volume (C) Mount Point

File Sharing
File Sharing: File sharing is very important for users who want to cooperate their files with
each other and to reduce the effort required to achieve a computing goal.
File sharing includes
• Multiple users
• Remote File Systems
• Client server model
• Distributed Information systems
• Failure Modes
• Consistency semantics - Unix Semantics, Session Semantics, Immutable Shared File
Semantics.
Multiple Users:
The system with multiple users can either allow a user to access the files of other users by
default or require that a user specifically grant access to the files.

The systems uses the concepts of file owner (or user) and group for File sharing. The owner
is the user who can change attributes and grant access and who has the most control over the
file. The group attribute defines a subset of users who can share access to the file.

The owner and group IDs of a given file are stored with the other file attributes. When a user
requests an operation on a file, the user ID can be compared with the owner attribute to
determine if the requesting user is the owner of the file.

If he is not the owner of the file, the group IDs can be compared. The result indicates which
permissions are applicable. The system then applies those permissions to the requested
operation and allows or denies it.

9
Remote File Systems:

Networking allows the sharing of resources across a campus or even around the world.

The first implemented method for remote file systems involves manually transferring files
between machines via programs like FTP. The second major method uses a distributed file
system (DFS) in which remote directories is visible from a local machine. The third method
is the World Wide Web where the browser is needed to gain access to the remote files.

Client Server Model:

The machine containing the files is the server, and the machine seeking access to the files is
the client. The server declares that a resource is available to clients and specifies exactly which
resource is shared by which clients. In the case of UNIX and its network file system (NFS),
authentication takes place via the client networking information. The user’s IDs on the client
and server must match. If they do not, the server will be unable to determine access rights to
files.

Distributed Information Systems:

Distributed information systems, also known as distributed naming services, provide

unified access to the information needed for remote computing. The domain name system
(DNS) provides host-name-to-network-address translations for the entire Internet. Distributed
information systems provide user name/password/user ID/group ID space for a distributed
facility.

In the case of Microsoft’s common Internet file system (CIFS), network information is used
in conjunction with user authentication to create a network login that the server uses to decide
whether to allow or deny access to a requested file system.

Failure Modes:
Local file systems can fail for a variety of reasons that includes
• Failure of the disk containing the file system,
• Corruption of the directory structure or other disk-management information
• Disk-controller failure,
• Cable failure,
• Host-adapter failure
• User or system-administrator failure
• Remote file systems have more failure modes because of the complexity of network
systems and the required interactions between remote machines.

10
The failure semantics are defined and implemented as part of the remote-file-system protocol.
Termination of all operations can result in users’ losing data. To implement the recovery from
failure, some kind of state information may be maintained on both the client and the server.
If both server and client maintain knowledge of their current activities and open files, then they
can recover from a failure.

Consistency Semantics:

Consistency semantics represent a criterion for evaluating any file system that supports file
sharing. The semantics specify how multiple users of a system are to access a shared file
simultaneously.

They specify when modifications of data by one user will be observable by other users.
Consistency semantics are directly related to the process synchronization algorithms.

A series of file accesses attempted by a user to the same file is always enclosed between the
open() and close() operations.

The series of accesses between the open() and close() operations makes up a file session.

The Examples of Consistency semantics includes

Unix Semantics: Writes to an open file by a user are visible immediately to other users who
have this file open. One mode of sharing allows users to share the pointer of current location
into the file. Thus, the advancing of the pointer by one user affects all sharing users

Session Semantics: Writes to an open file by a user are not visible immediately to other users
that have the same file open. Once a file is closed, the changes made to it are visible only in
sessions starting later. Already open instances of the file do not reflect these changes

Immutable Shared File Semantics:

Once a file is declared as shared by its creator, it cannot be modified. An immutable file has
two key properties: its name may not be reused, and its contents may not be altered. An
immutable file signifies that the contents of the file are fixed.

File Protection
Protection mechanisms provide controlled access by limiting the types of file access that can
be made. Access is permitted or denied depending on several factors, one of which is the type
of access requested.

File Protection is also defined as the process of protecting the file of a user from unauthorized
access or any other physical damage.

Goals of Protection:
11
To prevent malicious misuse of the system by users or programs.

To ensure that each shared resource is used only in accordance with system policies, which
may be set either by system designers or by system administrators.

To ensure that errant programs cause the minimal amount of damage possible.

Types of Access:

The need to protect files is a direct result of the ability to access files. Systems that do not
permit access to the files of other users do not need protection Several different types of
operations may be controlled:

• Read.
• Write.
• Execute.
• Append.
• Delete.
• List.
Access-control list (ACL): specifying user names and the types of access allowed for each
user.
Owner: The user who created the file is the owner.
Group: A set of users who are sharing the file and need similar access
Universe: All other users in the system constitute the universe
Password Protection: Another approach to the protection problem is to associate a password
with each file. Access to each file can be controlled with the help of passwords. If the passwords
are chosen randomly and changed often, this scheme may be effective in limiting access to a
file.

Disadvantages:
• The number of passwords that a user needs to remember may become large.
• If only one password is used for all the files, then once it is discovered, all files are
accessible; protection is on an all-or-none basis.
• In a multilevel directory structure, we need to protect not only individual files but also
collections of files in subdirectories.

12
File System Structure
The file system provides the mechanism for on-line storage and access to file contents,
including data and programs. The file system resides permanently on secondary storage, which
is designed to hold a large amount of data permanently.

A file system poses two quite different design problems.

• The first problem is defining how the file system should look to the user. This task
involves defining a file and its attributes, the operations allowed on a file, and the
directory structure files.

• The second problem is creating algorithms and data structures to map the logical file
system onto the physical secondary-storage devices.

The file system itself is generally composed of many different levels.

Fig 4.12 File System Structures

I/O Control: The I/O control level consists of device drivers and interrupts handlers to
transfer information between the main memory and the disk system.

The device driver usually writes specific bit patterns to special locations in the I/O controller’s
memory to tell the controller which device location to act on and what actions to take.

Basic File System: The basic file system needs only to issue generic commands to the
appropriate device driver to read and write physical blocks on the disk. Each physical block is
identified by its numeric disk address.

This layer also manages the memory buffers and caches that hold various file-system,
directory, and data blocks. A block in the buffer is allocated before the transfer of a disk block
can occur. Caches are used to hold frequently used file-system metadata to improve
performance.

File Organization Module:

13
The file-organization module knows about files and their logical blocks, as well as physical
blocks. The file-organization module can translate logical block addresses to physical block
addresses for the basic file system to transfer. The file-organization module also includes the
free-space manager, which tracks unallocated blocks and provides these blocks to the file-
organization module when requested.

Logical File System: The logical file system manages metadata information. The logical file
system manages the directory structure to provide the file-organization module with the
information it needs.

It maintains file structure via file-control blocks

A file control block (FCB) (an inode in UNIX file systems) contains information about the
file, including ownership, permissions, and location of the file contents.

Advantages of Layered File system:

• When a layered structure is used for file-system implementation, duplication of code is

minimized.

• Each file system can then have its own logical file-system and file-organization
modules.

Disadvantages:

• The use of layering, including the decision about how many layers to use and what each
layer should do, is a major challenge in designing new systems.

Directory Implementation
Directory can be implemented in two ways

Linear List: The simplest method of implementing a directory is to use a linear list of file names with
pointers to the data blocks. This method is simple to program but time-consuming to execute.

To create a new file, we must first search the directory to be sure that no existing file has the same name.
Then, we add a new entry at the end of the directory.

To delete a file, we search the directory for the named file and then release the space allocated to it. To
reuse the directory entry, we can do one of several things.

Disadvantage:

• Finding a file requires a linear search.

Hash Table: The hash table takes a value computed from the file name and returns a pointer to the file
time. Some provision must be made for collisions situations in which two file names hash to the same
location.

14
The major difficulties with a hash table are its generally fixed size and the dependence of the hash
function on that size. Alternatively, we can use a chained-overflow hash table.

Each hash entry can be a linked list instead of an individual value, and we can resolve collisions by
adding the new entry to the linked list. Still, this method is likely to be much faster than a linear search
through the entire directory.

Directory Allocation methods

Contiguous allocation requires that each file occupy a set of contiguous blocks on the disk.
Disk addresses define a linear ordering on the disk. It supports both direct and sequential
access. The directory entry for each file indicates the address of the starting block and the
length of the area allocated for this file.

For Example: File name mail requires length of 6 blocks, Staring block is 19, then it occupies
19, 20, 21,22, 23, 24 and 25 contiguous blocks.

Fig 4.13 Contiguous allocation

Advantages:
• Accessing a file that has been allocated contiguously is easy.

• The number of disk seeks required for accessing a file is minimal.

• It supports both direct and sequential access.

Disadvantages:

• Finding space for a new file.

• Contiguous memory allocation suffers from the problem of external fragmentation.

15
External Fragmentation: The total available space may not be enough to satisfy a request.
Storage is fragmented into a number of holes, none of which is large enough to store the data.

Linked allocation:

Linked allocation solves all problems of contiguous allocation. With linked allocation, each
file is a linked list of disk blocks; the disk blocks may be scattered anywhere on the disk. Each
directory entry has a pointer to the first disk block of the file. By accessing the First block, we
can find the address of the next blocks. To read a file, we simply read blocks by following the
pointers from block to block.

Fig 4.14 Linked allocation

Advantages:

• There is no external fragmentation with linked allocation, and any free block on the
free-space list can be used to satisfy a request.

• The size of a file need not be declared when the file is created.

• A file can continue to grow as long as free blocks are available.

Disadvantages:

• It is inefficient to support a direct-access capability for linked-allocation files.

• It requires more disk space for storing the pointers.

Solution: The usual solution to this problem is to collect blocks into multiples, called clusters,
and to allocate clusters rather than blocks.

Indexed Allocation

16
In Indexed allocation each file has its own index block, which is an array of disk-block
addresses. The directory contains the address of the index block. The index block stores all
blocks corresponding to the specific file. By accessing the index block, we can access all the
blocks for a specific file.

Fig 4.15 Indexed allocation

Advantages:

• Indexed allocation supports direct access.

• It does not suffer from External fragmentation. because any free block on the disk can
satisfy a request for more Space.

• Indexed allocation does suffer from wasted space.

Disadvantages:

• The Pointer overhead of the index block is generally greater than the pointer overhead
of linked allocation.

• Every file must have an index block, so we want the index block to be as small as
possible.

Free Space Management

Free Space management keeps track of free disk space. Since disk space is limited, we need
to reuse the space from deleted files for new files.

FREE SPACE LIST: To keep track of free disk space, the system maintains a free-space list.
The free-space list records all free disk blocks those not allocated to some file or directory.

17
To create a file, we search the free-space list for the required amount of space and allocate that
space to the new file. This space is then removed from the free-space list. When a file is deleted,
its disk space is added to the free-space list.

The Free space list can be implemented in the following ways

• Bit Vector or Bit Map

• Linked list
• Groping
• Counting
• Space maps
Bit Vector:

The free-space list is implemented as a bit map or bit vector. Each block is represented by 1
bit. If the block is free, the bit is 1; if the block is allocated, the bit is 0.

Example: Consider a disk where blocks 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26, and 27
are free and the rest of the blocks are allocated.

Free-space bit map:

01111001111110001100000011100000...

Advantages:

• The main advantage of this approach is its relative simplicity and its efficiency in
finding the first free block or n consecutive free blocks on the disk.

• Bit vectors are inefficient unless the entire vector is kept in main memory.

Disadvantages:

• If the disk size constantly increases, the problem with bit vectors will continue to
increase.

Linked list:

Linked list implementation link together all the free disk blocks, keeping a pointer to the first
free block in a special location on the disk and caching it in memory. This first block contains
a pointer to the next free disk block, and so on.

Example: Blocks 2, 3, 4, 5, 8, 9, 10, 11, 12, 13, 17, 18, 25, 26, and 27 in the disk were free
and the rest of the blocks were allocated.

18
 Fig 4.15 Linked List Method of Free Space List

We would keep a pointer to block 2 as the first free block. Block 2 would contain a pointer to
block 3, which would point to block 4, which would point to block 5, which would point to
block 8, and so on.

Disadvantages:

• This scheme is not efficient; to traverse the list, we must read each block, which requires
substantial I/O time.

Grouping:

A modification of the free-list approach stores the addresses of n free blocks in the first free
block. The first n−1 of these blocks are actually free. The last block contains the addresses of
other n free blocks, and so on.

Advantages:

• The addresses of a large number of free blocks can now be found quickly.

Counting:

Free space list keep the address of the first free block and the number (n) of free contiguous
blocks that follow the Each entry in the free-space list then consists of a disk address and a
count.

This method of tracking free space is similar to the extent method of allocating blocks. These
entries can be stored in a balanced tree, rather than a linked list, for efficient lookup, insertion,
and deletion.

Space maps:

19
A space map uses log-structured file-system techniques to record the information about the
free blocks. The space map is a log of all block activity (allocating and freeing), in time order,
in counting format.

Oracle’s ZFS file system creates Meta slabs to divide the space on the device into chunks of
manageable size. A given volume may contain hundreds of Meta slabs. Each Meta slab has an
associated space map. ZFS uses the counting algorithm to store information about free blocks.

Efficiency and Performance, Recovery

Efficiency

The efficient use of disk space depends heavily on the disk-allocation and directory algorithms
in use.

For instance, UNIX inodes are pre allocated on a volume. However, by pre allocating the
inodes and spreading them across the volume, we improve the file system’s performance.

In Solaris operating system, process table and the open-file table data structures are allocated
at system startup as fixed length. When the process table became full, no more processes could
be created. When the file table became full, no more files could be opened. The system would
fail to provide services to users. Table sizes could be increased only by recompiling the kernel
and rebooting the system.

With later releases of Solaris, all kernel structures were allocated dynamically.

Performance

Even after the basic file-system algorithms have been selected, we can still improve
performance in several ways. Most disk controllers include local memory to form an on-board
cache that is large enough to store entire tracks at a time. Once a seek is performed, the disk
controller then transfers any sector from the cache to the operating system.

Some systems maintain a separate section of main memory for a buffer cache, where blocks
are kept under the assumption that they will be used again shortly. Other systems cache file
data using a page cache. The page cache uses virtual memory techniques to cache file data
as pages rather than as file-system-oriented blocks.

Caching file data using virtual addresses is far more efficient than caching through physical
disk blocks, as accesses interface with virtual memory rather than the file system. Several
systems including Solaris, Linux, and Windows use page caching to cache both process pages
and file data. This is known as unified virtual memory.

Recovery

20
Files and directories are kept both in main memory and on disk, and care must be taken to
ensure that a system failure does not result in loss of data or in data inconsistency. A system
can recover from such a failure by using Consistency Checking, Log-Structured File
Systems, Backup and Restore.

Consistency Checking: by Scanning all the metadata regularly on each file system can
confirm or deny the consistency of the system.

Log-Structured File Systems: All metadata changes are written sequentially to a log. Each
set of operations for performing a specific task is a transaction. Once the changes are written
to this log, they are considered to be committed, and the system call can return to the user
process, allowing it to continue execution.

As the changes are made, a pointer is updated to indicate which actions have completed and
which are still incomplete. When an entire committed transaction is completed, it is removed
from the log file, The resulting implementations are known as log-based transaction-oriented
file systems.

Backup and Restore: System programs can be used to back up data from disk to another
storage device, such as a magnetic tape or other hard disk. Recovery from the loss of an
individual file, or of an entire disk, may then be a matter of restoring the data from backup.

Disk Management
The operating system is responsible for disk management. The Major Responsibility includes
Disk Formatting, Booting from Disk, Block Recovery.

Disk Formatting:

The Disk can be formatted in two ways,

1. Physical or Low Level Formatting,

2. Logical or High Level Formatting

Physical or Low Level Formatting:

Before a disk can store data, it must be divided into sectors that the disk controller can read
and write. This process is called low-level formatting, or physical formatting.

When the sector is read, the ECC is recalculated and compared with the stored value. If the
stored and calculated numbers are different, this mismatch indicates that the data area of the
sector has become corrupted and that the disk sector may be bad. It then reports a recoverable
soft error.

21
Logical Formatting or High Level Formatting:

The operating record its own data structures on the disk during Logical formatting. It does so
in two steps. The first step is to partition the disk into one or more groups of cylinders. The
second step is logical formatting, or creation of a file system. In this step, the operating system
stores the initial file-system data structures onto the disk. These data structures may include
maps of free and allocated space and an initial empty directory.

Booting from Disk: The full bootstrap program is stored in the boot blocks at a fixed location
on the disk. A disk that has a boot partition is called a boot disk or system disk.

The code in the boot ROM instructs the disk controller to read the boot blocks into memory
and then starts executing that code which in turn loads the entire Operating System.

Block Recovery:

A bad block is a damaged area of magnetic storage media that cannot reliably be used to store
and retrieve data. These blocks are handled in a variety of ways.

One strategy is to scan the disk to find bad blocks while the disk is being formatted. Any bad
blocks that are discovered are flagged as unusable so that the file system does not allocate
them.

In Some systems the controller maintains a list of bad blocks on the disk. This can be handled
in two ways.

Sector Sparing: Low-level formatting also sets aside spare sectors not visible to the operating
system. The controller can be told to replace each bad sector logically with one of the spare
sectors. This scheme is known as sector sparing or forwarding

Sector Slipping: The Process of moving all the sectors down one position from the bad sector
is called as sector slipping.

Swap Space Management

Swap-space management is another low-level task of the operating system. Virtual memory
uses disk space as an extension of main memory. Since disk access is much slower than
memory access, Swap Space management is designed to provide the best throughput for the
virtual memory system.

Systems that implement swapping may use swap space to hold an entire process image,
including the code and data segments. The amount of swap space needed on a system can
22
therefore vary from a few megabytes of disk space to gigabytes, depending on the amount of
physical memory, the amount of virtual memory it is backing, and the way in which the virtual
memory is used.

Each swap area consists of a series of 4-KB page slots, which are used to hold swapped pages.
Associated with each swap area is a swap map—an array of integer counters, each
corresponding to a page slot in the swap area. If the value of a counter is 0, the corresponding
page slot is available.

Values greater than 0 indicate that the page slot is occupied by a swapped page. The value of
the counter indicates the number of mappings to the swapped page. The data structures for
swapping on Linux systems are shown in Figure.

Fig 4.9 The data structures for swapping on Linux systems

For example, a value of 3 indicates that the swapped page is mapped to three different processes
(which can occur if the swapped page is storing a region of memory shared by three processes.

23
Understanding the OS161 - filesystem and working with test programs

OS/161 is an educational operating system. It aims to strike a balance between giving students
experience working on a real operating system, and potentially overwhelming students with
the complexity that exists in a fully fledged operating system, such as Linux. Compared to
most deployed operating systems, OS/161 is quite small (approximately 20,000 lines of code
and comments), and therefore it is much easier to develop an understanding of the entire code
base.

The source code distribution contains a full operating system source tree, including the kernel,
libraries, various utilities (ls, cat, etc.), and some test programs. The OS/161 boots on the
simulated machine in the same manner as a real system might boot on real hardware.

System/161

System/161 simulates a "real" machine to run OS/161 on. The machine features a MIPS
R2000/R3000 CPU including an MMU, but no floating point unit or cache. It also features
simplified hardware devices hooked up to lamebus. These devices are much simpler than real
hardware, and thus make it feasible for you to get your hands dirty, without having to deal with
the typical level of complexity of physical hardware.

Using a simulator has several advantages. Unlike software you have written thus far (Windows
excluded :-)), buggy software may result in completely locking up the machine, making to
difficult to debug and requiring a reboot. A simulator allows debuggers access to the machine
below the software architecture level as if debugging was built into the CPU chip. In some
senses, the simulator is similar to an in circuit emulator (ICE) that you might find in industry,
only it's done in software. The other major advantage is speed of reboot, rebooting real
hardware takes minutes, and hence the development cycle can be frustratingly slow on real
hardware.

GDB

You should already be familiar the GDB, the GNU debugger. GDB allows you to set
breakpoints to stop your program under certain conditions, inspect the state of your program
when it stops, modify its state, and continue where it left off. It is a powerful aid to the
debugging process that is worth investing the time needed to learn it. GDB allows you to
quickly find bugs that are very difficult to find with the typical printf style debugging.

24
STEPS TO BUILD SOFTWARE FROM SOURCE FILE

BUILD SOFTWARE FRAMEWORK FOR OS/161

25
OS/161 INSTALLATION

● Linux desktop with UBUNTU Version 12.04 or later.

● Internet connections to download and install packages

Pre Installation Steps

1. Install the packages gettext (to translate native language statements into English) ,
textinfo (to transalate source code into other formats) and libncurses5-dev ( allows users to
write text-base GUI)

sudo apt-get install gettext sudo apt-get install texinfo

sudo apt-get install libncurses5-dev

2. Include the paths $HOME/sys161/bin and $HOME/sys161/tools/bin into the PATH

environment variable. Add the following line at the end of the file .bashrc

export PATH=$HOME/sys161/bin:$HOME/sys161/tools/bin:$PATH

Now logout and login to get the PATH updated. You can check the current setting of the

PATH environment variable using the command

printenv PATH

Installation Steps

STEP 1: Download the following source codes one by one

● Binutils for MIPS

http://www.student.cs.uwaterloo.ca/~cs350/os161_repository/os161- binutils.tar.gz

● GCC MIPS Cross-Compiler

http://www.student.cs.uwaterloo.ca/~cs350/os161_repository/os161- gcc.tar.gz

26
● GDB for Use with OS/161

http://www.student.cs.uwaterloo.ca/~cs350/os161_repository/os161-gdb.tar.gz

● bmake for use with OS/161

http://www.student.cs.uwaterloo.ca/~cs350/os161_repository/os161- bmake.tar.gz

● mk for use with OS/161

http://www.student.cs.uwaterloo.ca/~cs350/os161_repository/os161-mk.tar.gz

● sys/161

http://www.student.cs.uwaterloo.ca/~cs350/os161_repository/sys161.tar.gz

OS/161

http://www.student.cs.uwaterloo.ca/~cs350/os161_repository/os161.tar.gz

Note : bmake and mk utilities are BSD make utilities used for OS161

STEP 2: Build and Install the Binary Utilities (Binutils)

Unpack the binutils archive:

tar -xzf os161-binutils.tar.gz

Move into the newly-created directory:

cd binutils-2.17+os161-2.0.1

Configure binutils:

./configure --nfp --disable-werror --target=mips harvard-os161 --prefix=$HOME/sys161/tools

Make binutils:

make
27
Finally, once make has succeeded, install the binutils into their final location:

make install

This will create the directory $HOME/sys161/tools/ and populate it.

Step 3: Install the GCC MIPS Cross-Compiler

Unpack the gcc archive:

tar -xzf os161-gcc.tar.gz

Move into the newly-created directory:

cd gcc-4.1.2+os161-2.0

Configure gcc

./configure -nfp --disable-shared --disable-threads -- disable-libmudflap --disable-libssp --

target=mips harvard-os161 --prefix=$HOME/sys161/tools

Make it and install it:

make

make install

Step 4: Install GDB

Unpack the gdb archive:

tar -xzf os161-gdb.tar.gz

Move into the newly-created directory:

cd gdb-6.6+os161-2.0

28
Configure gdb

./configure --target=mips-harvard-os161 -- prefix=$HOME/sys161/tools --disable-werror

Make it and install it:

Make

make install

Step 5: Install bmake

Unpack the bmake archive:

tar -xzf os161-bmake.tar.gz

Move into the newly-created directory:

cd bmake

Unpack mk within the bmake directory:

tar -xzf ../os161-mk.tar.gz

Run the bmake bootstrap script

./boot-strap --prefix=$HOME/sys161/tools

As the boot-strap script finishes, it should print a list of commands that you can run to install
bmake under $HOME/sys161/tools. The list should look something like this:

mkdir -p /home/kmsalem/sys161/tools/bin cp /home/kmsalem/bmake/Linux/bmake

/home/kmsalem/sys161/tools/bin/bmake-20101215 rm -f
/home/kmsalem/sys161/tools/bin/bmake

ln -s bmake-20101215 /home/kmsalem/sys161/tools/bin/bmake mkdir

-p /home/kmsalem/sys161/tools/share/man/cat1 cp /home/kmsalem/bmake/bmake.cat1

/home/kmsalem/sys161/tools/share/man/cat1/bmake.1 sh /home/kmsalem/bmake/mk/install-
mk

/home/kmsalem/sys161/tools/share/mk

Run the commands printed by boot-strap in the order in which they are listed in your terminal
screen.

29
Step 6: Set Up Links for Toolchain Binaries

mkdir $HOME/sys161/bin

cd $HOME/sys161/tools/bin

sh -c 'for i in mips-*; do ln -s $HOME/sys161/tools/bin/$i

$HOME/sys161/bin/cs350-`echo $i | cut -d- -f4-`; done'

ln -s $HOME/sys161/tools/bin/bmake $HOME/sys161/bin/bmake

When you are finished with these steps, a listing of the directory

$HOME/sys161/bin should look similar to this:

bmake@ cs350-gcc@ cs350-ld@ cs350-run@ cs350-addr2line@ cs350-gcc-4.1.2@ cs350-

nm@ cs350-size@ cs350-ar@ cs350-gccbug@ cs350-objcopy@ cs350- strings@ cs350-as@
cs350-gcov@ cs350-objdump@ cs350-strip@ cs350-c++filt@ cs350-gdb@ cs350-ranlib@

cs350-cpp@ cs350-gdbtui@ cs350-readelf@

Step 7: Build and Install the sys161 Simulator

Unpack the sys161 archive:

tar -xzf sys161.tar.gz

Move into the newly-created directory:

cd sys161-1.99.06

Next, configure sys161:

./configure --prefix=$HOME/sys161 mipseb

Build sys161 and install it:

make

make install

Finally, set up a link to a sample sys161 configuration file

cd $HOME/sys161

ln -s share/examples/sys161/sys161.conf.sample sys161.conf

Step 8: Install OS/161

First, create a directory to hold the OS/161 source code, your compiled OS/161 kernels, and
related test programs.
30
cd $HOME

mkdir cs350-os161

Next, move the OS/161 archive into your new directory and unpack it: mv os161.tar.gz
cs350-os161

cd cs350-os161

tar -xzf os161.tar.gz

This will create a directory called os161-1.99 (under cs350-os161) containing the OS/161
source code. You should now be able build, install, and run an OS/161 kernel and related
application and test programs by following steps.

Step 9: Configure OS/161 and Build the OS/161 Kernel

The next step is to configure OS/161 and compile the kernel. From the cs350- os161
directory, do the following:

cd os161-1.99

./configure --ostree=$HOME/cs350-os161/root -- toolprefix=cs350-

cd kern/conf

./config ASST0

cd ../compile/ASST0 bmake depend

bmake

bmake install

Step 10: Build the OS/161 User-level

Next, build the OS/161 user level utilities and test programs:

cd $HOME/cs350-os161/os161-1.99 bmake

bmake install

Step 11: Try Running OS/161

You should now be able to use the SYS/161 simulator to run the OS/161 kernel that you built
and installed. The SYS/161 simulator requires a configuration file in order to run. To obtain
one, do this:

cd $HOME/cs350-os161/root

cp $HOME/sys161/sys161.conf sys161.conf sys161 kernel-ASST0

31
You should see some output that looks something like this:

sys161: System/161 release 1.99.06, compiled Aug 23 2013 10:23:34

OS/161 base system version 1.99.05

Copyright (c) 2000, 2001, 2002, 2003, 2004, 2005, 2008, 2009

President and Fellows of Harvard College. All rights reserved. Put-your-group-name-here's

system version 0 (ASST0 #1)

316k physical memory available Device probe...

lamebus0 (system main bus) emu0 at lamebus0

ltrace0 at lamebus0 ltimer0 at lamebus0 beep0 at ltimer0 rtclock0 at ltimer0 lrandom0 at

lamebus0 random0 at lrandom0 lhd0 at lamebus0 lhd1 at lamebus0 lser0 at lamebus0 con0 at
lser0

cpu0: MIPS r3000

OS/161 kernel [? for menu]:

The last line is a command prompt from the OS/161 kernel. For now, just enter the command
q to shut down the simulation and return to your shell.

32