Week #7-Lect #16
Week #7-Lect #16
Operating System Concepts – 10th Edition Silberschatz, Galvin and Gagne ©2018
Mass-Storage Systems
Instructor : Dr. Farzana
Jabeen
Operating System Concepts – 10th Edition Silberschatz, Galvin and Gagne ©2018
Recap
Operating System Concepts – 10th Edition 11.3 Silberschatz, Galvin and Gagne ©2018
Moving-head Disk Mechanism
Operating System Concepts – 10th Edition 11.4 Silberschatz, Galvin and Gagne ©2018
Disk Attachment
▪ Host-attached storage accessed through I/O ports talking to I/O busses
▪ Several busses available, including advanced technology attachment
(ATA), serial ATA (SATA), eSATA, universal serial bus (USB), and
fibre channel (FC).
▪ Most common is SATA ( SATA specifies a transfer format and a wiring
arrangement.)
▪ Because NVM much faster than HDD, new fast interface for NVM
called NVM express (NVMe), connecting directly to PCI bus
▪ Data transfers on a bus carried out by special electronic processors
called controllers (or host-bus adapters,)
• Host controller on the computer end of the bus, device controller on
device end
• Computer places command on host controller, using memory-
mapped I/O ports
Host controller sends messages to device controller
Data transferred via DMA between device and computer DRAM
Operating System Concepts – 10th Edition 11.5 Silberschatz, Galvin and Gagne ©2018
HDD Scheduling
Operating System Concepts – 10th Edition 11.6 Silberschatz, Galvin and Gagne ©2018
Disk Scheduling (Cont.)
▪ Note that drive controllers have small buffers and can manage a
queue of I/O requests (of varying “depth”)
▪ Several algorithms exist to schedule the servicing of disk I/O requests
▪ The analysis is true for one or many platters
▪ We illustrate scheduling algorithms with a request queue (0-199)
98, 183, 37, 122, 14, 124, 65, 67
Head pointer 53
Operating System Concepts – 10th Edition 11.7 Silberschatz, Galvin and Gagne ©2018
FCFS
Illustration shows total head movement of 640 cylinders
Operating System Concepts – 10th Edition 11.8 Silberschatz, Galvin and Gagne ©2018
SCAN
▪ The disk arm starts at one end of the disk, and moves toward the
other end, servicing requests until it gets to the other end of the disk,
where the head movement is reversed and servicing continues.
▪ SCAN algorithm Sometimes called the elevator algorithm
▪ Illustration shows total head movement of 208 cylinders
▪ But note that if requests are uniformly dense, largest density at other
end of disk and those wait the longest
Operating System Concepts – 10th Edition 11.9 Silberschatz, Galvin and Gagne ©2018
SCAN (Cont.)
Operating System Concepts – 10th Edition 11.10 Silberschatz, Galvin and Gagne ©2018
C-SCAN
▪ Provides a more uniform wait time than SCAN
▪ The head moves from one end of the disk to the other, servicing
requests as it goes
• When it reaches the other end, however, it immediately returns to
the beginning of the disk, without servicing any requests on the
return trip
▪ Treats the cylinders as a circular list that wraps around from the last
cylinder to the first one
▪ Total number of cylinders?
Operating System Concepts – 10th Edition 11.11 Silberschatz, Galvin and Gagne ©2018
C-SCAN (Cont.)
Operating System Concepts – 10th Edition 11.12 Silberschatz, Galvin and Gagne ©2018
TODAY’S LECTURE
Operating System Concepts – 10th Edition 11.13 Silberschatz, Galvin and Gagne ©2018
Error Detection and Correction
▪ Fundamental aspect of many parts of computing (memory, networking,
storage)
▪ Error detection determines if there a problem has occurred (for
example a bit flipping)
• If detected, can halt the operation
• Detection frequently done via parity bit
▪ Parity one form of checksum – uses modular arithmetic to compute,
store, compare values of fixed-length words
• Another error-detection method common in networking is cyclic
redundancy check (CRC) which uses hash function to detect
multiple-bit errors
▪ Error-correction code (ECC) not only detects, but can correct some
errors
• Soft errors correctable, hard errors detected but not corrected
Operating System Concepts – 10th Edition 11.14 Silberschatz, Galvin and Gagne ©2018
Storage Device Management
▪ Low-level formatting, or physical formatting — Dividing a disk into
sectors that the disk controller can read and write
• Each sector can hold header information, plus data, plus error
correction code (ECC)
• Usually 512 bytes of data but can be selectable
▪ To use a disk to hold files, the operating system still needs to record its
own data structures on the disk
• Partition the disk into one or more groups of cylinders, each
treated as a logical disk
• Logical formatting or “making a file system”
• To increase efficiency most file systems group blocks into clusters
Disk I/O done in blocks
File I/O done in clusters
Operating System Concepts – 10th Edition 11.15 Silberschatz, Galvin and Gagne ©2018
Storage Device Management (cont.)
▪ Root partition contains the OS, other partitions can hold other
Oses, other file systems, or be raw
• Mounted at boot time
• Other partitions can mount automatically or manually
▪ At mount time, file system consistency checked
• Is all metadata correct?
If not, fix it, try again
If yes, add to mount table, allow access
▪ Boot block can point to boot volume or boot loader set of blocks that
contain enough code to know how to load the kernel from the file
system
• Or a boot management program for multi-os booting
Operating System Concepts – 10th Edition 11.16 Silberschatz, Galvin and Gagne ©2018
Device Storage Management (Cont.)
Operating System Concepts – 10th Edition 11.17 Silberschatz, Galvin and Gagne ©2018
Swap-Space Management
▪ Used for moving entire processes (swapping), or pages (paging),
from DRAM to secondary storage when DRAM not large enough for
all processes
▪ Operating system provides swap space management
• Secondary storage slower than DRAM, so important to optimize
performance
• Usually multiple swap spaces possible – decreasing I/O load on
any given device
• Best to have dedicated devices
• Can be in raw partition or a file within a file system (for
convenience of adding)
• Data structures for swapping on Linux systems:
Operating System Concepts – 10th Edition 11.18 Silberschatz, Galvin and Gagne ©2018
Storage Attachment
▪ Computers access storage in three ways
• host-attached
• network-attached
• cloud
▪ Host attached access through local I/O ports, using one of several
technologies
• To attach many devices, use storage busses such as USB, firewire,
thunderbolt
• High-end systems use fibre channel (FC)
High-speed serial architecture using fibre or copper cables
Multiple hosts and storage devices can connect to the FC fabric
Operating System Concepts – 10th Edition 11.19 Silberschatz, Galvin and Gagne ©2018
Network-Attached Storage
▪ Network-attached storage (NAS) is storage made available over a
network rather than over a local connection (such as a bus)
• Remotely attaching to file systems
▪ NFS and CIFS are common protocols
▪ Implemented via remote procedure calls (RPCs) between host and
storage over typically TCP or UDP on IP network
▪ iSCSI protocol uses IP network to carry the SCSI protocol
• Remotely attaching to devices (blocks)
Operating System Concepts – 10th Edition 11.20 Silberschatz, Galvin and Gagne ©2018
CIFS & NFS
▪ CIFS is an abbreviation for “Common Internet File System” used
by Windows operating systems for file sharing. CIFS also uses
the client-server methodology where A client makes a request of a
server program for accessing a file. The server takes the requested
action and returns a response. CIFS is an open standard version of
the Server Message Block Protocol (SMB) developed and used by
Microsoft and it uses the TCP/IP protocol.
▪ NFS is the “Network File System” specifically used for Unix and
Linux operating systems. It allows files communication
transparently between servers and end-users machines like
desktops & laptops. NFS uses a client-server methodology to allow
the user to view read and write files on a computer system. A user
can mount all or a portion of a file system via NFS.
Operating System Concepts – 10th Edition 11.21 Silberschatz, Galvin and Gagne ©2018
CIFS vs NFS
Operating System Concepts – 10th Edition 11.22 Silberschatz, Galvin and Gagne ©2018
Cloud Storage
Operating System Concepts – 10th Edition 11.23 Silberschatz, Galvin and Gagne ©2018
Storage Array
▪ Can just attach disks, or arrays of disks
▪ Avoids the NAS drawback of using network bandwidth
▪ Storage Array has controller(s), provides features to attached host(s)
• Ports to connect hosts to array
• Memory, controlling software (sometimes NVRAM, etc)
• A few to thousands of disks
• RAID, hot spares, hot swap (discussed later)
• Shared storage -> more efficiency
• Features found in some file systems
Snaphots, clones, thin provisioning, replication, deduplication,
etc
Operating System Concepts – 10th Edition 11.24 Silberschatz, Galvin and Gagne ©2018
Storage Area Network
Operating System Concepts – 10th Edition 11.25 Silberschatz, Galvin and Gagne ©2018
Storage Area Network (Cont.)
Operating System Concepts – 10th Edition 11.26 Silberschatz, Galvin and Gagne ©2018
RAID Structure
Operating System Concepts – 10th Edition 11.27 Silberschatz, Galvin and Gagne ©2018
RAID (Cont.)
▪ Disk striping uses a group of disks as one storage unit
▪ RAID is arranged into six different levels
▪ RAID schemes improve performance and improve the reliability of
the storage system by storing redundant data
• Mirroring or shadowing (RAID 1) keeps duplicate of each
disk
• Striped mirrors (RAID 1+0) or mirrored stripes (RAID 0+1)
provides high performance and high reliability
• Block interleaved parity (RAID 4, 5, 6) uses much less
redundancy
▪ RAID within a storage array can still fail if the array fails, so
automatic replication of the data between arrays is common
▪ Frequently, a small number of hot-spare disks are left
unallocated, automatically replacing a failed disk and having data
rebuilt onto them
Operating System Concepts – 10th Edition 11.28 Silberschatz, Galvin and Gagne ©2018
RAID Levels
Operating System Concepts – 10th Edition 11.29 Silberschatz, Galvin and Gagne ©2018
Operating System Concepts – 10th Edition 11.30 Silberschatz, Galvin and Gagne ©2018
RAID (0 + 1) and (1 + 0)
Operating System Concepts – 10th Edition 11.31 Silberschatz, Galvin and Gagne ©2018
Other Features
▪ Regardless of where RAID implemented, other useful features can be
added
▪ Snapshot is a view of file system before a set of changes take place
(i.e. at a point in time)
• More in Ch 12
▪ Replication is automatic duplication of writes between separate sites
• For redundancy and disaster recovery
• Can be synchronous or asynchronous
▪ Hot spare disk is unused, automatically used by RAID production if a
disk fails to replace the failed disk and rebuild the RAID set if possible
• Decreases mean time to repair
Operating System Concepts – 10th Edition 11.32 Silberschatz, Galvin and Gagne ©2018
Week 7:Lecture # 17
File-System Interface
Operating System Concepts – 10th Edition Silberschatz, Galvin and Gagne ©2018
File Concept
▪ Contiguous logical address space
▪ Types:
• Data
Numeric
Character
Binary
• Program
▪ Contents defined by file’s creator
• Many types
text file,
source file,
executable file
Operating System Concepts – 10th Edition 11.34 Silberschatz, Galvin and Gagne ©2018
What is a File System?
Operating System Concepts – 10th Edition 11.35 Silberschatz, Galvin and Gagne ©2018
Objective and Properties
Objective of File management System
• It provides I/O support for a variety of storage device types.
• Minimizes the chances of lost or destroyed data
• Helps OS to standardized I/O interface routines for user processes.
• It provides I/O support for multiple users in a multiuser systems environment.
Operating System Concepts – 10th Edition 11.36 Silberschatz, Galvin and Gagne ©2018
File Attributes
▪ Name – only information kept in human-readable form
▪ Identifier – unique tag (number) identifies file within file system
▪ Type – needed for systems that support different types
▪ Location – pointer to file location on device
▪ Size – current file size
▪ Protection – controls who can do reading, writing, executing
▪ Time, date, and user identification – data for protection, security,
and usage monitoring
▪ Information about files are kept in the directory structure, which is
maintained on the disk
▪ Many variations, including extended file attributes such as file
checksum
▪ Information kept in the directory structure
Operating System Concepts – 10th Edition 11.37 Silberschatz, Galvin and Gagne ©2018
File info Window on Mac OS X
Operating System Concepts – 10th Edition 11.38 Silberschatz, Galvin and Gagne ©2018
Directory Structure
▪ A collection of nodes containing information about all files
Operating System Concepts – 10th Edition 11.39 Silberschatz, Galvin and Gagne ©2018
File Operations
▪ Create
▪ Write – at write pointer location
▪ Read – at read pointer location
▪ Reposition within file - seek
▪ Delete
▪ Truncate
▪ Open (Fi) – search the directory structure on disk for entry Fi,
and move the content of entry to memory
▪ Close (Fi) – move the content of entry Fi in memory to directory
structure on disk
Operating System Concepts – 10th Edition 11.40 Silberschatz, Galvin and Gagne ©2018
Open Files
▪ Several pieces of data are needed to manage open files:
• Open-file table: tracks open files
• File pointer: pointer to last read/write location, per process
that has the file open
• File-open count: counter of number of times a file is open –
to allow removal of data from open-file table when last
processes closes it
• Disk location of the file: cache of data access information
• Access rights: per-process access mode information
Operating System Concepts – 10th Edition 11.41 Silberschatz, Galvin and Gagne ©2018
File Types – Name, Extension
Operating System Concepts – 10th Edition 11.42 Silberschatz, Galvin and Gagne ©2018
Access Methods
▪ A file is fixed length logical records
▪ Sequential Access
▪ Direct Access
▪ Other Access Methods
Operating System Concepts – 10th Edition 11.43 Silberschatz, Galvin and Gagne ©2018
Sequential Access
▪ OS reads the file word by word. A pointer is maintained which initially points
to the base address of the file. If the user wants to read first word of the file
then the pointer provides that word to the user and increases its value by 1
word. This process continues till the end of the file.
▪ Operations
• read next
• write next
• Reset (rewind)
▪ Figure
Operating System Concepts – 10th Edition 11.44 Silberschatz, Galvin and Gagne ©2018
Direct Access
▪ Operations
• read n
• write n
• position to n
read next
write next
rewrite n
Operating System Concepts – 10th Edition 11.45 Silberschatz, Galvin and Gagne ©2018
Simulation of Sequential Access on Direct-access File
Operating System Concepts – 10th Edition 11.46 Silberschatz, Galvin and Gagne ©2018
Index Access
▪ The Index Access method is an improvised version of the direct
access method.
▪ In this method, an index is maintained which contains the address
(or pointers) of the file blocks.
▪ So, to access any particular record of the file the operating system
will first search its address in the index and the index will then point
to the actual address of the particular block of the file that has the
desired information.
Operating System Concepts – 10th Edition 11.47 Silberschatz, Galvin and Gagne ©2018
Protection
▪ File owner/creator should be able to control:
• What can be done
• By whom
▪ Types of access
• Read
• Write
• Execute
• Append
• Delete
• List
Operating System Concepts – 10th Edition 11.48 Silberschatz, Galvin and Gagne ©2018
Access Lists and Groups in Unix
▪ Mode of access: read, write, execute
▪ Three classes of users on Unix / Linux
RWX
a) owner access 7 111
RWX
b) group access 6 110
RWX
c) public access 1 001
▪ Ask manager to create a group (unique name), say G, and add
some users to the group.
▪ For a file (say game) or subdirectory, define an appropriate access.
Operating System Concepts – 10th Edition 11.49 Silberschatz, Galvin and Gagne ©2018
A Sample UNIX Directory Listing
Operating System Concepts – 10th Edition 11.50 Silberschatz, Galvin and Gagne ©2018
Windows 7 Access-Control List Management
Operating System Concepts – 10th Edition 11.51 Silberschatz, Galvin and Gagne ©2018
LECTURE # 18
Operating System Concepts – 10th Edition 11.52 Silberschatz, Galvin and Gagne ©2018
Disk Structure
▪ Disk can be subdivided into partitions
▪ Disks or partitions can be RAID protected against failure
▪ Disk or partition can be used raw – without a file system, or
formatted with a file system
▪ Partitions also known as minidisks, slices
▪ Entity containing file system is known as a volume
▪ Each volume containing a file system also tracks that file
system’s info in device directory or volume table of contents
▪ In addition to general-purpose file systems there are many
special-purpose file systems, frequently all within the same
operating system or computer
Operating System Concepts – 10th Edition 11.53 Silberschatz, Galvin and Gagne ©2018
A Typical File-system Organization
Operating System Concepts – 10th Edition 11.54 Silberschatz, Galvin and Gagne ©2018
Types of File Systems
▪ We mostly talk of general-purpose file systems
▪ But systems frequently have may file systems, some general- and
some special- purpose
▪ Consider Solaris has
• tmpfs – memory-based volatile FS for fast, temporary I/O
• objfs – interface into kernel memory to get kernel symbols for
debugging
• ctfs – contract file system for managing daemons
• lofs – loopback file system allows one FS to be accessed in place
of another
• procfs – kernel interface to process structures
• ufs, zfs – general purpose file systems
Operating System Concepts – 10th Edition 11.55 Silberschatz, Galvin and Gagne ©2018
Directory Structure
▪ A collection of nodes containing information about all files
Operating System Concepts – 10th Edition 11.56 Silberschatz, Galvin and Gagne ©2018
Operations Performed on Directory
▪ Search for a file
▪ Create a file
▪ Delete a file
▪ List a directory
▪ Rename a file
Operating System Concepts – 10th Edition 11.57 Silberschatz, Galvin and Gagne ©2018
Directory Organization
The directory is organized logically to obtain
▪ Efficiency – locating a file quickly
▪ Naming – convenient to users
• Two users can have same name for different files
• The same file can have several different names
▪ Grouping – logical grouping of files by properties, (e.g., all
Java programs, all games, …)
Operating System Concepts – 10th Edition 11.58 Silberschatz, Galvin and Gagne ©2018
Single-Level Directory
▪ A single directory for all users
▪ Naming problem
▪ Grouping problem
Operating System Concepts – 10th Edition 11.59 Silberschatz, Galvin and Gagne ©2018
Two-Level Directory
▪ Separate directory for each user
▪ Path name
▪ Can have the same file name for different user
▪ Efficient searching
▪ No grouping capability
Operating System Concepts – 10th Edition 11.60 Silberschatz, Galvin and Gagne ©2018
Tree-Structured Directories
Operating System Concepts – 10th Edition 11.61 Silberschatz, Galvin and Gagne ©2018
Tree-Structured Directories
A tree structure is the most common directory structure. The tree has
a root directory, and every file in the system have a unique path.
Advantages:
1. Very generalize, since full path name can be given.
2. Very scalable, the probability of name collision is less.
3. Searching becomes very easy, we can use both absolute path
as well as relative.
Disadvantages:
1. Every file does not fit into the hierarchical model, files may be
saved into multiple directories.
2. We can not share files.
3. It is inefficient, because accessing a file may go under multiple
directories.
Operating System Concepts – 10th Edition 11.62 Silberschatz, Galvin and Gagne ©2018
Acyclic-Graph Directories
▪ Have shared subdirectories and files
▪ Example
Operating System Concepts – 10th Edition 11.63 Silberschatz, Galvin and Gagne ©2018
Acyclic-Graph Directories (Cont.)
▪ Two different names (aliasing)
▪ If dict deletes w/list dangling pointer
Solutions:
• Backpointers, so we can delete all pointers.
Variable size records a problem
• Backpointers using a daisy chain organization
• Entry-hold-count solution
▪ New directory entry type
• Link – another name (pointer) to an existing file
• Resolve the link – follow pointer to locate the file
Operating System Concepts – 10th Edition 11.64 Silberschatz, Galvin and Gagne ©2018
General Graph Directory
Operating System Concepts – 10th Edition 11.65 Silberschatz, Galvin and Gagne ©2018
General Graph Directory (Cont.)
▪ How do we guarantee no cycles?
• Allow only links to files not subdirectories
• Garbage collection
• Every time a new link is added use a cycle detection algorithm to
determine whether it is OK
Operating System Concepts – 10th Edition 11.66 Silberschatz, Galvin and Gagne ©2018
Current Directory
▪ Can designate one of the directories as the current (working)
directory
• cd /spell/mail/prog
• type list
▪ Creating and deleting a file is done in current directory
▪ Example of creating a new file
• If in current directory is /mail
• The command
mkdir <dir-name>
• Results in:
Operating System Concepts – 10th Edition 11.67 Silberschatz, Galvin and Gagne ©2018
File Locking
▪ Provided by some operating systems and file systems
• Similar to reader-writer locks
• Shared lock similar to reader lock – several processes can
acquire concurrently
• Exclusive lock similar to writer lock
▪ Mediates access to a file
▪ Mandatory or advisory:
• Mandatory – access is denied depending on locks held and
requested
• Advisory – processes can find status of locks and decide
what to do
Operating System Concepts – 10th Edition 11.68 Silberschatz, Galvin and Gagne ©2018
File Locking Example – Java API
import java.io.*;
import java.nio.channels.*;
public class LockingExample {
public static final boolean EXCLUSIVE = false;
public static final boolean SHARED = true;
public static void main(String arsg[]) throws IOException {
FileLock sharedLock = null;
FileLock exclusiveLock = null;
try {
RandomAccessFile raf = new RandomAccessFile("file.txt", "rw");
// get the channel for the file
FileChannel ch = raf.getChannel();
// this locks the first half of the file - exclusive
exclusiveLock = ch.lock(0, raf.length()/2, EXCLUSIVE);
/** Now modify the data . . . */
// release the lock
exclusiveLock.release();
Operating System Concepts – 10th Edition 11.69 Silberschatz, Galvin and Gagne ©2018
File Locking Example – Java API (Cont.)
Operating System Concepts – 10th Edition 11.70 Silberschatz, Galvin and Gagne ©2018
End of Chapter 13
Operating System Concepts – 10th Edition Silberschatz, Galvin and Gagne ©2018