A new approach to file system cache writeback of application data

In this paper we propose a new paradigm and algorithms to address cache writeback performance in servers and storage arrays. As servers and storage processors move to multi-core architecture, with ever increasing memory caches, the cost of flushing these caches to disk has become a problem. Traditional watermark based algorithms currently used in many storage arrays and NAS servers have a problem keeping up with the higher speeds of incoming application writes, often resulting in a performance penalty. The server's cache is generally used for hiding high disk latencies associated with file system data. In general, metadata performance was optimized, while application data was considered less sensitive to high latencies and was given lower priority or was written directly to disk. The new algorithms proposed here change the application data writeback from using watermark based flush to something that approximates the rate of the incoming application I/Os. The problem is more critical for network file systems where the complex client/server protocols can make writeback a serious performance barrier, particularly in light of very large I/Os and the lack of application commits. Our proposed algorithms are applicable to local file systems and remote servers as well as to storage arrays. We show test results based on dynamic traces of real file system dirty pages in the buffer cache and prove that rate based cache writeback algorithms are the most efficient replacement for watermark based flushing.