OPERATING SYSTEM FILE SYSTEM

Session 10(Chapter 9)

Concept and Structure of a File

2

Contiguous logical address space Types:

None - sequence of words, bytes Simple record structure

Data

numeric character binary

Lines Fixed length Variable length

Formatted document Re-locatable load file

Complex Structures

Program

Can simulate last two with first method by inserting appropriate control characters Who decides:

Operating system Program

OperatingSystems-FileSystem

Attributes of a File
3

Name only information kept in human-readable form Identifier unique tag (number) identifies file within file system Type needed for systems that support different types Location pointer to file location on device Size current file size Protection controls who can do reading, writing, executing Time, date, and user identification data for protection, security, and usage monitoring Information about files are kept in the directory structure, which is maintained on the disk
OperatingSystems-FileSystem

File Operation
4

File is an abstract data type Create Write Read Reposition within file Delete Truncate Open(Fi) search the directory structure on disk for entry Fi, and move the content of entry to memory Close (Fi) move the content of entry Fi in memory to directory structure on disk
OperatingSystems-FileSystem

What is needed to Open Files

5

Several pieces of data are needed to manage open files:

File pointer: pointer to last read/write location, per process that has the file open File-open count: counter of number of times a file is open to allow removal of data from open-file table when last processes closes it Disk location of the file: cache of data access information Access rights: per-process access mode information

OperatingSystems-FileSystem

Open File Locking

6

Provided by some operating systems and file systems Mediates access to a file Mandatory or advisory:

Mandatory access is denied depending on locks held and requested Advisory processes can find status of locks and decide what to do

OperatingSystems-FileSystem

File type Name and Extension

7

OperatingSystems-FileSystem

What is File System?

8

The file system is a mechanism for storing data and programs. Main characteristics of Secondary storage

Large storage capacity Non-volatile nature. Give raise to the convenience of storing data and programs on secondary storage.

Consequence?

OperatingSystems-FileSystem

What do you Require of File Systems?

Naming

Should be flexible, e.g., allow multiple names for same files Support hierarchy for easy of use Want to be sure data has been written to disk in case crash occurs Want to restrict who has access to files Want to share files with other users Sequential access Random access Sequential is most common & Random next Other pattern is Keyed access (not usually provided by OS

Persistence

Sharing/Protection

Speed & Efficiency for different access patterns

OperatingSystems-FileSystem

10

What do you Require of File Systems?

Minimum Space Overhead

Disk space needed to store metadata is lost for user data

Twist: all metadata that is required to do translation must be stored on disk

Translation scheme should minimize number of additional accesses for a given access pattern Harder than, say page tables where we assumed page tables themselves are not subject to paging!

OperatingSystems-FileSystem

File System Organization

11

The file system on the disk of an operating system allows data to be stored, searched and retrieved. One part of the file system is the user interface. This interface defines the definition of a file, file attributes, operations allowed on files and a directory structure for organizing files. The other part of the file system has algorithms and data structures to map the logical file system onto the physical storage devices. A file system has a layered design

Each layer uses the features provided by the layer below it to provide features to the layer above it.

The lowest layer is the I/O control.

It consists of device drivers and interrupt handlers for information transfer between the memory and the disk. A device driver is a translator between the file system and the actual disk hardware. The device driver takes high level instructions as input.
OperatingSystems-FileSystem

File System Organization

12

The next layer, the basic file system issues generic commands to the appropriate device driver to access (read / write) physical blocks on the disk.

The layer above the basic file system is the file organization module.

This module has information about files, the logical records and the physical blocks.

The file organization module translates / maps logical addresses to physical block addresses for the basic file system.

This module also keeps track of free blocks. The logical file system makes use of the directory structure of the file system and provides information to the file organization module given a file name. Protection and security are also part of the logical file system.

OperatingSystems-FileSystem

File System Organization

13

OperatingSystems-FileSystem

Over-View
14

OperatingSystems-FileSystem

15

OperatingSystems-FileSystem

16

Steps in Opening and Reading a File

Look-up (via-directory)

Find on disks file descriptors block number

Create one if none, else increment ref count By reading on-disk file descriptor

Find entry in open file table.

Find where file data is located

Read data & return to user

OperatingSystems-FileSystem

Open File Table

17

inode represents file

at most 1 in-memory instance per unique file #number of openers & other properties
With separate offsets for byte-stream

filerepresents one or more processes using an file

dirrepresents an open directory file Generally:

None of data in OFT is persistent Reflects how processes are currently using files Lifetime of objects determined by open/close

Reference counting is used

OperatingSystems-FileSystem

File Descriptors (inodes)

18

Term inode can refer to 3 things:

1.in-memory inode

Store information about an open file, such as how many openers, corresponds to on-disk file descriptor Region on disk, entry in file descriptor table, that stores persistent information about a file who owns it, where to find its data blocks, etc. A bytewise copy of 2. in memory

2.on-disk inode

3.on-disk inode, when cached in buffer cache

Q.: Should in-memory inode store a pointer to cached on-disk inode?

OperatingSystems-FileSystem

19

What information does File System contain?

Contains super blockstores information such assize of entire file system, etc.

Location of file descriptor table & free map

Free Block Map

Bitmap used to find free blocks Typically cached in memory

Superblock & free map often replicated in different positions on disk

OperatingSystems-FileSystem

Allocation of Disk Space

20

These mechanisms looks at how efficiently files are stored and accessed.

Contiguous Linked Indexed

Commonly referred as file allocation strategies

OperatingSystems-FileSystem

Contiguous Allocation
21

Idea: allocate files in contiguous blocks File Descriptor = (first block, length) Good sequential & random access Problems:
hard to extend files may require expensive compaction external fragmentation analogous to segmentation-based VM

OperatingSystems-FileSystem

Linked Files
22

Idea: implement linked list

either with variable sized blocks or fixed sized blocks (clusters)

Solves fragmentation problem, but now

need lots of seeks for sequential accesses and random accesses

lose first block, may lose file Solution: keep linked list in memory

unreliable:

DOS: FAT File Allocation Table

OperatingSystems-FileSystem

23

OperatingSystems-FileSystem

Indexed Allocation
24

Single-index: specify maximum file-size, create index array, then note blocks in index
access ok one translation step Sequential access requires more seeks depending on contiguous allocation
Random

Drawback: hard to grow beyond maximum

OperatingSystems-FileSystem

Directory Structure
25

Directory is collection of nodes containing information about files.

Both the directory structure and the files reside on disk Backups of these two structures are kept on tapes

OperatingSystems-FileSystem

Disk Structure
26

Disk can be subdivided into partitions Disks or partitions can be RAID protected against failure Disk or partition can be used raw without a file system, or formatted with a file system Partitions also known as minidisks, slices Entity containing file system known as a volume Each volume containing file system also tracks that file systems info in device directory or volume table of contents As well as general-purpose file systems there are many special-purpose file systems, frequently all within the same operating system or computer.
OperatingSystems-FileSystem

Disk Schematics
27

OperatingSystems-FileSystem

Tracks Sectors and Cylinders

28

OperatingSystems-FileSystem

From O.S Perspective

29

Disks are big & slow -compared to RAM Access to disk requires

Seek (move arm to track) to cross all tracks anywhere from 20-50ms, on average takes 1/3. Rotational delay (wait for sector to appear under track) 7,200rpm is 8.3ms per rotation, on average takes : 4.15ms rot delay Transfer time (fast: 512 bytes at 998 Mbit/s is about 3.91us)

Seek+Rot Delay dominates Random Access is expensive

And unlikely to get better

Consequence:

Avoid seeks Seek to short distances Amortize seeks by doing bulk transfers

OperatingSystems-FileSystem

Disk Scheduling
30

Can use priority scheme Can reduce avg access time by sending requests to disk controller in certain order

Or, more commonly, have disk itself reorder requests

Like SJF in CPU scheduling, guarantees minimum avg seek time, but

SSTF: shortest seek time first

can lead to starvation

SCAN: elevator algorithm

Process requests with increasing track numbers until highest reached, then decreasing etc. repeat LOOK dont go all the way to the top without passengers C-SCAN: -only take passengers when going up

Variations:

OperatingSystems-FileSystem

31

Typical File System Organization

OperatingSystems-FileSystem

Operations Performed on a Directory

32

Search for a file Create a file Delete a file List a directory Rename a file Traverse the file system

OperatingSystems-FileSystem

Organize the Directory (Logically) to Obtain

33

Efficiency locating a file quickly Naming convenient to users

Two users can have same name for different files The same file can have several different names

Grouping logical grouping of files by properties, (e.g., all Java programs, all games, )

OperatingSystems-FileSystem

Single Directory
34

A single directory for all users

Naming and grouping problem

OperatingSystems-FileSystem

Single Directory
35

Separate directory for each user.

Path name Can have the same file name for different user Efficient searching No grouping capability
OperatingSystems-FileSystem

Tree Structured Directories

36

Efficient searching Grouping Capability Current directory (working directory)

cd /spell/mail/prog type list

OperatingSystems-FileSystem

Tree Structured Directories

37

Absolute or relative path name Creating a new file is done in current directory Delete a file
rm <file-name>

Creating a new subdirectory is done in current directory

mkdir <dir-name>

Example: if in current directory /mail

mkdir count

Deleting mail deleting the entire subtree rooted by mail

OperatingSystems-FileSystem

Acyclic Graph Directories

38

They have shared Sub-directories and files

OperatingSystems-FileSystem

Acyclic Graph Directories

39

Two different names (aliasing) If dict deletes list dangling pointer Solutions:

Back-pointers, so we can delete all pointers Variable size records a problem Back-pointers using a daisy chain organization

Entry-hold-count solution New directory entry type

Link another name (pointer) to an existing file Resolve the link follow pointer to locate the file

OperatingSystems-FileSystem

General Graph Directory

40

How do we guarantee no cycles?

Allow only links to file not subdirectories Garbage collection Every time a new link is added use a cycle detection algorithm to determine whether it is OK.

OperatingSystems-FileSystem

Directory Implementation
41

Linear List

list of file names with pointers to the data blocks is one way to implement a directory. linear search is necessary to find for a particular file The method is simple but the search is time consuming. To create a file a linear search is made to look for the existence of a file with the same file name and if no such file is found the new file created is added to the directory at the end. To delete a file a linear search for the file name is made and if found allocated space is released. Every time making a linear search consumes time and increases access time that is not desirable since a directory information is frequently used. A sorted list allows for a binary search that is time efficient compared to the linear search.

But maintaining a sorted list is an overhead especially because of file creations and OperatingSystems-FileSystem deletions.

Directory Implementation
42

Hash Table

A linear list is used to store directory entries. A hash table takes a value computed from the file name and returns a pointer to the file name in the linear list. Thus search time is greatly reduced. Insertions are prone to collisions that are resolved. The main problem is the hash function that is dependent on the hash table size. A solution to the problem is to allow for chained overflow with each hash entry being a linked list.

Directory lookups in a hash table are faster than in a linear list.

OperatingSystems-FileSystem

How is File System Mounted?

43

A file system must be mounted before it can be accessed A un-mounted file system is mounted at a mount point

(a) Existing.

(b) Unmounted Partition

Mount point
OperatingSystems-FileSystem

File Sharing
44

Sharing of files on multi-user systems is desirable Sharing may be done through a protection scheme On distributed systems, files may be shared across a network Network File System (NFS) is a common distributed filesharing method. Multiple Users

User IDs identify users, allowing permissions and protections to be per-user Group IDs allow users to be in groups, permitting group access rights

OperatingSystems-FileSystem

45

File Sharing Remote File Systems

Uses networking to allow file system access between systems

Manually via programs like FTP Automatically, seamlessly using distributed file systems Semi automatically via the world wide web

Client-server model allows clients to mount remote file systems from servers

Server can serve multiple clients Client and user-on-client identification is insecure or complicated NFS is standard UNIX client-server file sharing protocol CIFS is standard Windows protocol Standard operating system file calls are translated into remote calls

Distributed Information Systems (distributed naming services) such as LDAP, DNS, NIS, Active Directory implement unified access to information needed for remote computing

OperatingSystems-FileSystem

File Sharing Failure Modes

46

Remote file systems add new failure modes, due to network failure, server failure Recovery from failure can involve state information about status of each remote request Stateless protocols such as NFS include all information in each request, allowing easy recovery but less security

OperatingSystems-FileSystem

47

File Sharing Consistency Semantics

Consistency semantics specify how multiple users are to access a shared file simultaneously

Similar to process synchronization algorithms

Tend to be less complex due to disk I/O and network latency (for remote file systems

Andrew File System (AFS) implemented complex remote file sharing semantics Unix file system (UFS) implements:

Writes to an open file visible immediately to other users of the same open file Sharing file pointer to allow multiple users to read and write concurrently Writes only visible to sessions starting after the file is closed

AFS has session semantics

OperatingSystems-FileSystem

Protection
48

File owner/creator should be able to control:

what can be done by whom

Read Write Execute Append Delete List

Types of access

OperatingSystems-FileSystem

Access Lists and Groups

49

Mode of access: read, write, execute Three classes of users

Ask manager to create a group (unique name), say G, and add some users to the group. For a particular file (say game) or subdirectory, define an appropriate access.

Attach a group to a file

OperatingSystems-FileSystem

50

A Sample UNIX Directory Listing

OperatingSystems-FileSystem