Linux Conference

October 26- 28, 2011

Prague , Czech Republic Linux Development Center

The Extended FAT file system

Differentiating with FAT32 file system

Keshava Munegowda , Venkatraman S

Texas Instruments (India) Pvt Ltd
Bangalore.

Dr. G T Raju
1 Professor and Head, Computer Science and Engineering Department,
R N S Institute of Technology, Bangalore, India.
 ExFAT

Linux Development Center

Agenda

Ø FAT file system

Ø Need for ExFAT file system
Ø ExFAT file system organization
Ø Boot Sector of FAT and ExFAT
Ø Snapshot of Boot Sector of ExFAT
Ø Directory Entries of FAT
Ø Directory Entries of ExFAT
Ø Clusters Heap
Ø Up-Case Table
Ø Snapshot of Root directory
Ø Performance Benchmarking in Linux
Ø Performance Benchmarking in Windows
2 Ø References
 ExFAT

Linux Development Center

FAT File system

Minimum 1
BPB sector BPB - BIOS Parameter Block
– BIOS : Basic Input-Output System
FAT1 – Also Called as “Boot Sector” or “Volume Boot Record”
– Specifies
FAT2 • Number of sectors in the storage partition/disk/device
Cluster 2 • Number of FATs ( File Allocation Table)
Cluster 3 • Sectors per cluster
Cluster 4

…………. Data FAT1 – File Allocation Table

………….
clusters - Linear linking (chain) of data clusters of the
………….
file/directory

FAT2 – Backup of FAT1

FILE.TXT

cluster3 cluster7 EOF Data clusters

- Group of physical/logical sectors/blocks
3 Clusters chain / Linked list of
clusters - Contains directories or Files data
 ExFAT

Linux Development Center

Need For ExFAT file system

l FAT File system limited to support only

32GB.
– FAT does not support Higher size SDXC cards.
l NTFS
– Security features
– Optional for Removable storage devices
– Meta data overhead for file/directory
– Write caching mechanism for performance
optimizations causes the data corruption in when
4 removable storage device is unplugged.
 ExFAT

Linux Development Center

ExFAT file system Organization

24 l 12 Sectors for Boot Area

BPB Sectors
l Another 12 sectors for backup of
FAT1 Boot Area
Exists only if
TexFAT file
l FAT2 exists if the TexFAT (
FAT2 system Transactional FAT is used)
Cluster 2 Cluster heap l First few Data clusters starting
Cluster 3 Up Case Table
Cluster 4
from cluster 2 contains
1. Volume label
…………. Directory Entry – Cluster allocation bit map table
2. Allocation bit map
…………. Directory Entry – Up case table
…………. 3. Up case directory
entry l Next Cluster Contains
– Cluster allocation Bit map
Data clusters Directory Entry
– Up-case Table Directory Entry
5 – Volume label Directory Entry
 ExFAT

Linux Development Center

Key Elements of Boot Sector of ExFAT and FAT32

Field name ExFAT Boot Sector FAT32 Boot Sector

Offset Size Description Offset Size Description
( Byte) ( Byte) ( Byte) ( Byte)
File system name 3 8 “ExFAT ” “MSWIN4.1” or “MSDOS5.0”

82 8 “FAT32 ”

Volume length 72 8 Total number of Sectors 32 4 Total number of Sectors

FAT offset 80 4 Sector address of 1 st FAT

FAT length 84 4 Size of FAT in sectors 36 4 Size of 1st FAT in sectors

Cluster offset 88 4 Starting sector of cluster heap

Cluster count 92 4 Number of clusters

Root directory 96 4 First cluster of root directory 44 4 First cluster of root directory

Volume flags 106 2 Bit 0 – Active FAT

0 – 1st , 1 – 2nd
Bit 1 – Volume Dirty
0 – Clean, 1- dirty
Bits 2 & 3 – Media failure
0 – No failures, 1 – failures reported

Bytes per Sector 108 1 This is power of 2; Minimal value is 9; 29 =512 11 2 Count of bytes per sector; This can have
Bytes and maximum 2 12 =4096 Bytes following values 512,1024,2048 or 4096.

Sector per 109 1 This is power of 2; Minimal value is 1; 20 =1 13 1 Number of sectors per cluster; allowed values
cluster sector (512 Bytes) and maximum 32 MB cluster are 1,2,4,8,32,64 and 128. Maximum of 128
size in bytes sectors per cluster

6 Number of FATs 110 1 Either 1 or 2; if TexFAT is supported then it will

be 2
16 1 Usually 2
 ExFAT

Boot Sector of ExFAT

Boot sector snap shot of ExFAT file
system in 4GB USB stick

File system name “ExFAT”

Sector Address of 1st FAT Total number of sectors

Number of clusters
Cluster heap starting
data cluster
Number of FATs
Starting sector of
cluster heap
Number of sectors per cluster

Size of FAT in sectors 26 = 64 sectors

Number of Bytes per sector

29 = 512 bytes

Cluster size = 26 * 2 9 = 2 15 =32KB

7
 ExFAT

Default Cluster size of ExFAT, FAT and

NTFS

Disk size FAT16 FAT32 NTFS ExFAT Note: *

4 MB – 16MB 1KB 512 Bytes 4KB 4KB
KB – Kilo Bytes
MB – Mega Bytes
16 MB–64MB 2KB GB - Giga Bytes
64 MB–128 MB TB – Tera Bytes
128 MB–256 MB 4KB

256MB- 512MB 8KB 4KB 32KB

512MB- 1GB 16KB

1GB- 2GB 32KB

2GB – 8GB Not supported

8GB–16GB 8KB

16GB-32GB 16KB

32GB- 127GB 32KB 128KB

127GB-2TB Not supported;

2TB-16TB
*Assuming Sector size is 512
bytes, then beyond 127GB is
16TB – 32TB 8KB
not supported by FAT32, as
per FAT spec, the FAT entry
32TB-64TB 16KB
size is 32 bits but only 28 bits
64TB – 128TB are used for cluster entry. 32KB

8 128TB – 256TB

>256TB
64KB

Custom specific Custom specific

 ExFAT

Linux Development Center

ExFAT Maximum Volume Support

l The “Volume length” field of Boot sector is 8
Bytes per Sector per bytes; so the maximum value is 264 Sectors
sector cluster l The Maximum value of “Bytes per sector” field
Max value: 212 Max value: 29 of boot sector is 212 = 4096 bytes
l So, the Maximum theoretical value support is
2(64+12) = 276 which is 64 Zetta Bytes (ZB)
+ l The FAT entry size is 32, Maximum cluster
number is 232
Maximum cluster size l So, To support 64 ZB , it requires at least
= 225 = 32 MB 2(32+12)= 16 Terra Bytes (TB) cluster size.
l But, Microsoft has set the limitation that
FAT entry size cluster size should be of maximum of 32 MB
32 bits = 232 l Hence “Bytes per sector” + “Sectors per
+ Clusters cluster” of boot sector can not exceed 25.
hence the maximum value will be 225 and the
maximum cluster are 232, so it yields
maximum value of 2(32+25) = 257 Bytes = 128
257 Bytes = 128 Peta Bytes ( PB)
Peta Bytes l Practically, Maximum Supported Size is 128
9 PB
 ExFAT

Linux Development Center

Directory Entries of FAT

32 Bytes Directory Entry of FAT32 FAT Long Directory Entry Structure
Offset Size Description Offset Size Description
Name (byte) (bytes) Name (byte) (bytes)

DIR_Name 0 11 Short name. LDIR_Ord 0 1 Masked with 0x40 (LAST_LONG_ENTRY),

this indicates the entry is the last long dir
DIR_Attr 11 1 File attributes
entry in a set of long dir entries. All valid
DIR_NTRes 12 1 Reserved sets of long dir entries must begin with an
entry having this mask.
DIR_CrtTimeTenth 13 1 Millisecond stamp at
file creation time LDIR_Name1 1 10 Characters 1-5 of the long-name sub-
DIR_CrtTime 14 2 Time file was created. component in this dir entry.
LDIR_Attr 11 1 Attributes - must be ATTR_LONG_NAME
DIR_CrtDate 16 2 Date file was created.
LDIR_Type 12 1 Zero
DIR_LstAccDate 18 2 Last access date.
LDIR_Chksum 13 1 Checksum of name in the short dir entry at
DIR_FstClusHI 20 2 High word of this the end of the long dir set.
entry’s first cluster
number LDIR_Name2 14 12 Characters 6-11 of the long-name sub-
component in this dir entry.
DIR_WrtTime 22 2 Time of last write.
LDIR_FstClusLO 26 2 Must be ZERO.
DIR_WrtDate 24 2 Date of last write.
LDIR_Name3 28 4 Characters 12-13 of the long-name sub-
component in this dir entry.
DIR_FstClusLO 26 2 Low word of this
entry’s first cluster
number.

DIR_FileSize 28 4 file size in bytes.

10
 ExFAT

Linux Development Center

Directory Entries of ExFAT

l Following are the Directory Entry Types defined

– Critical Primary
l Volume Label Directory Entry
l Allocation Bitmap Directory Entry
l Up-case Table Directory Entry
l File/Folder directory Entry
– Benign primary
l Volume GUID
l TexFAT Padding
l Windows CE Access Control Table
– Critical Secondary
l Stream Extension Directory Entry
l File name Extension Directory Entry
l All Directory Entries are of 32 Bytes
11
 ExFAT

Linux Development Center

Volume Label Directory Entry

Volume Label Directory Entry of ExFAT

Offset Size Description
Name (byte) (bytes)

Entry Type 0 1 0x83 – Indicates that the

Volume label exists

0x03- Indicates that the

volume label is deleted

Character count 1 1 Number of characters in label

Volume label 2 22 Volume label in Unicode;

Maximum of 11 characters of
16 bit Unicode string

Reserved 24 8
12
 ExFAT

Linux Development Center

Allocation Bit map ( Cluster Heap)

Directory Entry

Cluster Heap Directory Entry of ExFAT

Offset Size Description
Name (byte) (bytes)

Entry Type 0 1 0x81

Bit Map Flags 1 1 Bit 0

0 – 1st cluster Heap
1 – 2nd cluster Heap

Reserved 2 18

First cluster 20 4 First data cluster number;

Usually it will be 2
Data length 24 8 Size of cluster heap in bytes
It is calculated as: Ceil (Cluster
count / 8 )
13
 ExFAT

Cluster Heap
8 Bits l Generally the cluster 2 contains the
cluster heap
Clusters 9-2 00000001 l if cluster heap size is more than
cluster size; then the index 2 of File
Clusters 17-10 00000000 Allocation Table (FAT) contains the
Clusters 25-18 next cluster number which contains
-------- the cluster heap.
Clusters 33-26 There can exist multiple clusters (
-------- l
Data cluster chain in FAT) as cluster heap
------ -------- l Every Bit in the Cluster heap
clusters
------
-------- contains the status of the data
cluster; 0 – Free; 1- Allocated
l Byte 0 contains allocation status of
cluster 2 to cluster 9
l Byte 1 contains the allocation status
Clusters N – [N-8] -------- of cluster 10 to cluster 17
l Byte 2 contains the allocation status
14 of cluster 18 to cluster 25 and so on
 ExFAT

Linux Development Center

Up-Case Table Directory Entry

Up Case Table Directory Entry of ExFAT

Offset Size (bytes) Description
Name (byte)
Entry Type 0 1 0x82

Reserved 1 3
Table checksum 4 4 Check sum of the Up-case table
Reserved 8 12
First cluster 20 4 First data cluster

Data length 24 8 Up case table size in bytes

15
 ExFAT

Linux Development Center

Up-Case Table
l The Up case Table is series of
Unicode character mappings.
l One of the uses of the Up case
table is “lower case” to “Upper
case” conversion when file/directory
searching is performed.
l Up case table is always Read-only.
l If cluster heap is only one cluster
and it is cluster 2; then the Up case
table starts from cluster 3.
l By default, Up-case table starts
immediately after the cluster heap.
l If Up case table size is more than
cluster size; File Allocation Table
(FAT) contains the cluster chain
specifying the data clusters in
which content of Up case table is
available.
l Example:
– WinHex tool Snap shot of first few
bytes of Up-case Table
– Lower case letter “a ( 0x:61)”
16 converted to Upper case “A (0x41)”
by having value 0x0041 at the 16
bit index 0x61.
 ExFAT

Linux Development Center

File Directory Entry

File/Folder Directory Entry of ExFAT
Name Offset Size Description
(byte) (bytes)
Entry Type 0 1 0x85
Secondary count 1 1 Number of Secondary directory entries ; Minimum value is 2 and maximum
value is 18.These entries will be “Stream Extension directory entry” and
“File name extension directory entries”; In general, A file/directory contains
1 File directory Entry, 1 stream extension directory and maximum of 17 file
name extension directory entries.
Set checksum 2 2 Checksum is calculated on File directory entry (this entry) + All secondary
entries
File Attributes 4 2 Attributes of the file/directory
Reserved 6 2
Creation time 8 4 Created Date and Time in DOS time stamp format
Last Modified 12 4 Last modified Date and Time in DOS time stamp format
Last Accessed 16 4 Last accessed Date and Time in DOS time stamp format
Create 10ms Increment 20 1 10ms increments between 0 – 199
Last modified 10ms Increment 21 1 10ms increments between 0 – 199
Last accessed 10ms Increment 22 1 10ms increments between 0 – 199
17 Reserved 23 9
 ExFAT

Linux Development Center

File Attributes Field structure

File Attributes field of File Directory Entry of ExFAT
Name Offset (byte) Size (bytes) Description
Read only 0 1 0 – Write permission
1 – Read only
Hidden 1 1 0 – Visible
1 – Hidden
System 2 1 1 – system
Reserved 3 1 In FAT32, this bit is used to indicate that the file/directory entry is volume
label
Directory 4 1 0 – File
1 – Directory/Folder
Archive 5 1 1- Archive
Reserved 6 10

l The Read-only, Hidden, System, Directory and Archive bit fields are file attribute are same as
FAT32 File attributes structure.
l Volume Id is reserved in file attribute of ExFAT, because there is exist a dedicated directory
entry to represent the volume name of the disk/partition

18
 ExFAT

Linux Development Center

File/Directory Timestamp Structure

Timestamp field of File Directory Entry of ExFAT

Name Offset (byte) Size (bytes) Description

Double Seconds 0 5 Represents the seconds in two-second multiples.

Valid range of values is:
0 – which represents 0 seconds
29 – represents 58 seconds
Minute 5 6 Minutes; Valid range 0 -59

Hour 11 5 Hours; Valid range 0 -23

Day 16 5 Day of month; valid range 1-31

Month 21 4 Month of year; 1= January , valid range 1-12

Year 25 7 Count of years from 1980;

Valid range is:
0 – year 1980
127 – year 2107

l The time stamp format used in the ExFAT is same as FAT

19
 ExFAT

Linux Development Center

Stream Extension Directory Entry

Stream Extension Directory Entry of ExFAT
Name Offset Size Description
(byte) (bytes)
Entry Type 0 1 0xC0
General Secondary 1 1 Bit 0 : Allocation possible
Flags 0 – No cluster allocated; 1 – cluster allocation is possible

Bit 1 : No FAT chain

0 – Yes ; The clusters of this file/directory are NOT contiguous
1 – No; The Contiguous Cluster are allocated to this file/directory;
This improves the File read performance

Bits 2 – 7 : Reserved
Reserved 2 1
Name length 3 1 Length of file name in bytes ( Maximum of 255 Unicode characters)
Name hash 4 2 Hash of the file name; Used while searching for fie/directory name
Reserved 6 2
Valid data length 8 8 The size of the file of directory in bytes
Reserved 16 4
First cluster 20 4 First data cluster of the file/directory
20 Data length 24 8 The size of the file of directory in bytes; In case of directory maximum size is 256MB
 ExFAT

Linux Development Center

File Name Extension Directory Entry

File Name Extension Directory Entry of ExFAT

Name Offset Size Description
(byte) (bytes)

Entry Type 0 1 0xC1

General Secondary 1 1 Bit 0 : Allocation possible

Flags 0 – No cluster allocated; 1 – cluster allocation is possible

Bit 1 : No FAT chain

0 – Yes ; The clusters of this file/directory are NOT contiguous
1 – No; The Contiguous Cluster are allocated to this file/directory;
This improves the File read performance

Bits 2 – 7 : Reserved

File name 2 30 15 Unicode character of the part of the file name

l Single File name Extension directory entry can contain 15 Unicode characters.
l Maximum File name size is 255 Unicode characters
l Hence , for a single long file/directory name maximum of 17 file name extension directory
entries are possible.
21
 ExFAT

Linux Development Center

File/Directory and Volume label

Deletion process

l The Most Significant Bit (MSB) of Entry Type field of the directory entry will be
changed from 1 to 0 to indicate the directory entry is deleted.
l For example, A file contains the following directory entries
– File Directory Entry , Entry Type is 0x85 = 0x10000101 in binary format
– Stream Extension Directory Entry, Entry type is 0xC0 = 0x11000000 in binary format
– File name extension Directory Entry, Entry type is 0xC1 = 0x11000001 in Binary
format
l The MSB (7th) Bit indicates “In Use” field; it means 1 – In use, 0 – deleted.
l This MSB bit is changed from 1 to 0 to indicate the deletion process;
– File Directory Entry , Entry Type is 0x85 is changed to 0x05 = 0x00000101 in binary
format
– Stream Extension Directory Entry, Entry type is 0xC0 is changed to 0x40 =
0x01000000 in binary format
– File name extension Directory Entry, Entry type is 0xC1 is changed to 0x41=
0x010000001 in binary format
l The Entry type of the volume label is 0x83 = 0x10000011 in binary format.
l This Entry type value 0x83 will changed to 0x03 = 0x00000011 to indicate that
there is no volume label.

22
 ExFAT

Linux Development Center

Snapshot of the Root Directory

Root Directory ; The starting cluster 4 ; The
WinHex tool is showing contents of cluster 4

Volume label directory entry, named “ExFATdrive”

Cluster heap directory entry; with starting data

cluster 2

Up case directory entry; with starting data cluster 3

File created with

- file /directory entry
- Stream extension entry , starting data cluster 5.
- File name extension entry with the name
“KESHAVA.TXT”

Directory created with

- file /directory entry
- Stream extension entry , starting data cluster 5.
- File name extension entry with the name “TI”

23 File named “FILE.TXT” is DELETED

 ExFAT

Linux Development Center

File systems Write performance

numbers in Linux

Writer Performance Test Setup

Computer Dell Computer, Optiplex GX620
700000 Duel core 3.4Ghz with 2GB RAM
Operating system Ubuntu 9.04 with Linux kernel version 2.6.28
600000 FAT32
Storage device Transcend 4GB Thumb drive

Ext3 Performance tool Iozone version 3.3

500000
l The fie size range 16MB to 256MB
Ext4 The buffer size : 4MB to write
Write Performance in Kilo Bytes/ Second

ExFAT l
400000
FAT32
NTFS
NTFS
l For small files higher performance
ExFAT Ext3
range is 300MB/S ( Mega bytes per
300000
Ext4
second) to 700MB/S.
l For larger files, the performance range
200000 is 1MB/S to 12MB/S.
l The ExFAT is implemented as FUSE(
100000 File system in User space) module.
l FAT32 has seen good and in
0 performance for both smaller and
16384 32768 65536 131072 262144 larger files.
24 File size in KiloBytes
 ExFAT

Linux Development Center

File systems Read performance

numbers in Linux

Test Setup
Reader Performance Computer Dell Computer, Optiplex GX620
Duel core 3.4Ghz with 2GB RAM
850000
Operating system Ubuntu 9.04 with Linux kernel version 2.6.28

FAT32 Storage device Transcend 4GB Thumb drive

800000 NTFS Performance tool Iozone version 3.3
Ext4
ExFAT
l The fie size range 16MB to 256MB
750000
FAT32
l The buffer size : 4MB to Read
Read Performance in Kilo Bytes/Second

Ext3 l For small files higher performance range

NTFS
is 700MB/S ( Mega bytes per second) to
ExFAT Ext3 850MB/S.
700000
Ext4 l For larger files, the performance range is
600MB/S to 800MB/S.
650000 l Compared to write, the read performance
degradation is less.
l FAT32 and Ext4 has seen good and
600000 consistent in performance for both
16384 32768 65536 131072 262144
smaller and larger files.
File size in Kilo Bytes
25
 ExFAT

Linux Development Center

File systems Write performance

numbers in WindowsXP

Test Setup
Writer Performance
Computer Dell Lap-top, Latitude D630

450000
Intel Core 2Duo CPU 2.4 GHz, &1.17Ghz
Operating system Windows XP Profession version 2002,
400000 Service pack3
Storage device Transcend 4GB Thumb drive
350000
Performance tool Iozone version 3.3
300000
l The fie size range 16MB to 256MB
Write peformance in kilobytes/Second

250000 ExFAT l The buffer size : 4MB to Write

NTFS FAT32
l NTFS is has highest performance range
200000 NTFS 50MB/S to 400MB/S
150000
l Both ExFAT and FAT32 has lower
performance less than 50 MB/s
100000 l The NTFS performance is higher because
of “Write caching” mechanism
50000 l In Window XP, user has to set the “Write
FAT32 ExFAT caching mechanism by enabling the
0 performance optimization policy in the
16384 32768 65536 131072 262144 Hardware tab of the drive.
26 File size in KiloBytes
 ExFAT

Linux Development Center

File systems Re-Write performance

numbers in WindowsXP

Test Setup
Re-Writer Performance
Computer Dell Lap-top, Latitude D630

900000 Intel Core 2Duo CPU 2.4 GHz, &1.17Ghz

Operating system Windows XP Profession version 2002,
800000 Service pack3
Storage device Transcend 4GB Thumb drive
700000
Performance tool Iozone version 3.3
600000
Re-Write peformance in kilobytes/Second

l Re-Write means write the file that

ExFAT
500000 ExFAT already exists.
FAT32
400000 NTFS
l The fie size range 16MB to 256MB
FAT32 l The buffer size : 4MB to Write
300000
l NTFS is has lowest performance
200000 below 100 MB/S
l Both ExFAT and FAT32 has higher
100000
NTFS
performance range 400MB/s to
0
800MB/s
16384 32768 65536 131072 262144 l ExFAT has slightly higher performance
than FAT32.
27 File size in KiloBytes
 ExFAT

Linux Development Center

File systems Read performance

numbers in WindowsXP

Test Setup
Reader Performance
Computer Dell Lap-top, Latitude D630
1200000 Intel Core 2Duo CPU 2.4 GHz, &1.17Ghz
Operating system Windows XP Profession version 2002,
Service pack3
1000000
ExFAT Storage device Transcend 4GB Thumb drive
Performance tool Iozone version 3.3
800000
Read performance in Kilobytes/Second

ExFAT
l The fie size range 16MB to 256MB
600000 FAT32 l The buffer size : 4MB to Read
FAT32 NTFS l NTFS is has lowest performance
400000
below 100 MB/S
l Both ExFAT and FAT32 has higher
performance range 400MB/s to
200000 2GB(Giga Bytes)/S
NTFS
0
16384 32768 65536 131072 262144

28 File size in KiloBytes

 ExFAT

Linux Development Center

References
l US patent Application: 12/389396, “QUICK FILENAME LOOKUP USING NAME”,
Ravishankar V.Puipeddi, Vishal V.Ghotge, Ravinder S.Thind, Microsoft Corporation.
l US patent Application: 11/752872, “EXTENDING CLUSTER ALLOCATIONS IN
EXTENSIBLE FILE SYSTEM”, Ravishankar V.Puipeddi, Vishal V.Ghotge, Sarosh
C.Havewala, Ravinder S.Thind, Mark J Zbikowski, Microsoft Corporation.
l US patent Application: 11/229485, “EXTENSIBLE FILE SYSTEM”, Ravishankar
V.Puipeddi, Vishal V.Ghotge, Sarosh C.Havewala, Ravinder S.Thind, Mark J
Zbikowski, Microsoft Corporation.
l “Reverse Engineering the Microsoft Extended FAT file system(ExFAT)”, Robert
Shullich, SANS Institute InfoSec Reading Room
l “Extended FAT file system”, Jeff Hamm, Paradigm Solutions, Techno Forensics
Conference, October 2009.
l www.ntfs.com
l Free Fuse based ExFAT implementation for Linux : http://code.google.com/p/exfat/ ,
Developer : Andrew Nayenko
l Iozone,” File systems performance benchmarking tool”: http://www.iozone.org/
l FAT Specification:
http://msdn.microsoft.com/en-us/windows/hardware/gg463080
29
 ExFAT

Linux Development Center

Questions

Queries and Feedback

– [email protected]
– [email protected]

30