CSE 330:
Operating Systems
Adil Ahmad
Lecture #18: File system overview and internals
The (virtual) file system recap
Disk interactions starting from a system call
User starts here open(..) system call
The abstraction we want
Virtual filesystem
to discuss next!
Block device
Disk device driver
Discussed MMIO/DMA
this so far
The (virtual) filesystem abstraction
§ High-level intuitive view of how we look at data stored on disks
§ Not just disk-related; UNIX philosophy à “everything is a file”
Can anyone tell me of other ways in which you have used files?
§ High-level intuitive view of how we look at data stored on disks
§ Not just disk-related; UNIX philosophy à “everything is a file”
§ /dev/memalloc à virtual file to communicate with modules
§ CPU features can be enabled or disabled using files.
§ E.g., entire CPUs can be disabled as follows:
echo 0 | sudo tee /sys/devices/system/cpu/cpu1/online
§ Perform console R/W à write(1, “hello”, 5)
§ 1 à file descriptor for console (terminal) output
Let’s get back to (disk-related) file
§ File à a set of blocks that the OS has combined and operates on
together
§ Recall that the filesystem is composed of the the block layer
§ Files are given identifiers (e.g., “[Link]” is a human-readable version) so
programs/users can distinguish between them
§ File system (FS) à an intricate hierarchical collection of files built using
the blocks in your storage disk
Different “names” for a file
§ Three different names typically
ü inode (low-level names)
Ø Internal name (number) given to a file by the OS
ü path (human readable)
Ø The version that we see when we open the file browser (e.g.,
Windows explorer or MacOS finder)
ü file descriptor (runtime state)
Ø Represents the runtime status of a certain file
The inode (OS-level representation)
PROMPT>: stat [Link]
§ Each file has exactly one File: ‘[Link]’
inode number Size: 5
Blocks: 8
IO Block: 4096 regular file
Device: 803h/2051d
§ Inodes are unique (at a given Inode: 119341128
Links: 1
time) within a FS Access: (0664/-rw-rw-r--)
Uid: ( 1001/ yue)
Gid: ( 1001/ yue)
Context: unconfined_u:object_r:user_home_t:s0
§ File names can be the Access: 2015-12-17 [Link].935716294 -0500
same, why can’t inodes? Modify: 2014-12-12 [Link].669625220 -0500
Change: 2014-12-12 [Link].669625220 -0500
Ø Something must be unique Birth: -
for the OS to track
The path (human-readable)
§ Human-readable interpretation /
t
tx
of every inode
ar.
/b
o
foo bar
/fo
§ Typically, represented as –
<path-to-directory , filename> Bar.
bar2 foo
txt
§ Traversing a tree – getting the [Link]
final inode from a location
§ E.g., ls /foo/[Link]
§ Gets the inode and prints details
The file descriptor
§ “Everything is a file”
§ File descriptor tracks what each ‘file’ really does in the system
If file belongs to the
disk, it has an inode
Commonly-used file system interfaces
Creating a new file or opening an existing file
§ UNIX system call: open()
int fd = open(char *path, int flag, mode_t mode);
-OR-
int fd = open(char *path, int flag);
§ open() returns a file descriptor (fd)
• A fd is an integer
• Private per process
§ fd is a handle that gives caller the power to perform certain operations
§ You can think of a “fd” as a pointer to an object of the file
open()example
open()example
open()example
open()example
Other common file system interfaces
§ UNIX system call: read() and write()
int ret = read(int fd, char *buffer, size_t size);
int ret = write(int fd, char *buffer, size_t size);
§ Can anyone tell me why they have the same arguments?
Can anyone tell me what FS calls this command invokes?
§ cat [Link]
Can anyone tell me what FS calls this command invokes?
§ cat [Link]
Can anyone tell me what FS calls this command invokes?
Can anyone tell me what FS calls this command invokes?
Can anyone tell me what FS calls this command invokes?
Can anyone tell me what FS calls this command invokes?
Can anyone tell me what FS calls this command invokes?
Are writes to disk performed exactly when you call write()?
§ No, that would be very slow since more writes could be needed later,
and they can be batched together
Ø Hence, your FS will buffer writes in-memory
§ Sometimes though, you may want to force writes, e.g., if a later
operation depends on disk write or to ensure persistence.
§ For such cases, FS provides the fsync() system call
Example of how “vim” uses the filesystem
Implementation of a filesystem
What are the questions we may need to answer to build an FS?
Ø Where to store the files in the disk?
Ø Where/how to store the information about existing files (e.g., inodes,
path, blocks, etc.) in the disk?
Ø How to quickly find free locations in the disk for new files?
Storing file “data blocks” within the disk
§ Initially, the entire disk is empty
§ We will save some space to store information
§ Use the rest to store our “data blocks” (figure below)
Now, we must answer:
Ø Where to store the files in the disk?
Ø Where/how to store the information about existing files (e.g., inodes,
path, blocks, etc.) in the disk?
Ø How to quickly find free locations in the disk for new files?
An inode data structure
§ Answers the “how to store file information” question
§ Inode à The number corresponds to a data structure consisting of :
§ Type (e.g., file or directory)
§ Size
§ Number of data blocks
§ Address of data blocks
§ User permissions (e.g., root, etc.)
§ …
§ Inodes are 128/256 bytes, depending on the FS implementation
An inode table
§ Answers the “where to store file information” question
§ In a part of the reserved disk space, as a table/array format
Visualizing the inode and its table in the disk
Visualizing the inode and its table in the disk
Visualizing the inode and its table in the disk
Now, we must answer:
Ø Where to store the files in the disk?
Ø Where/how to store the information about existing files (e.g., inodes,
path, blocks, etc.) in the disk?
Ø How to quickly find free locations in the disk for new files?
This question has two sub-questions. What are they?
Ø Where to store the files in the disk?
Ø Where/how to store the information about existing files (e.g., inodes,
path, blocks, etc.) in the disk?
Ø How to quickly find free locations in the disk for new files?
1) How to quickly find free “data blocks”?
2) How to quickly find free “inodes”?
Using bitmaps to store free data and inode blocks
§ Can anyone tell me what’s a bitmap?
Bitmap to find free data blocks: data bitmap
Bitmap to find free data blocks: inode bitmap
Is there another question that we may have missed?
Ø How to persistently keep track of which regions contain all these disk
structures (e.g., data blocks, inode table, bitmaps, etc.)?
Ø Their size may be variable (e.g., depending on disk)
The “superblock”
§ Keeps track of basic filesystem information, like
Ø Block size
Ø How many inodes are there?
Ø How much space is free?
Putting it all together
Questions? Otherwise, see you next class!
