HFS: a performance-oriented flexible file system based on building-block compositions

Reviewer: Gerald David Chandler

The title of this paper is an especially accurate one-line summary of the contents, omitting only that it is intended primarily for shared-memory multiprocessor systems. Based on the first author's 1994 thesis with an almost identical title, and augmented by later work, this paper describes how small components, some as short as a few lines, with well-defined interfaces, can be combined in multiple ways to provide file-processing algorithms that vary from file to file and even over time for the same file. This “made to measure” quality allows performance to be optimized according to whether the file is most often read or written; access is contiguous, striped, or sparse; locking is important or unimportant; or any other conceivable set of conditions. This is an interesting example of the programming organization that results from the use of well-defined objects with a well-defined communication protocol between them and that is applied at all levels. The HFS building-block model is based on three layers. At the bottom, there is a physical layer, which directly accesses disks. HFS includes predefined blocks for orthogonal handling of shape (for example, striped or contiguous), properties (such as size unit, parity, and read-write ratio), and locations (distribution and replication). Above the physical layer is what the authors call the logical layer, which could better be called the performance or system layer. This provides common disk-independent services: naming (directories), authentication, and locking. The highest, or application, layer provides an interface that is a superset of the standard Unix file I/O interface. It also provides building blocks for latency hiding and for compression and decompression. Data are given that show that this flexible multilayer system (extending to 11 layers in one example) does not significantly delay file access. The apparent reasons for this are that, while there are indeed losses due to interlayer communication, they are small because the protocols are chosen for their low overhead and the underlying interprocess communication facility of the authors' research operating system is very fast. What losses there are, are nearly compensated for by the elimination of conditional and other code from the specific blocks that are used for a particular file system (this shortens both execution time and loading time) and by the reduction of cross-address space communication. Interestingly, a related paper reports that a specially tailored compiler was able to generate efficient compositions of predefined blocks. The paper is clear but overly long. Some of the space given to repeated expositions of the same points could have been used, for example, to specify the standard interface for the pre-implemented physical layer building blocks (as given in Krieger's thesis).