Consistent and compact data management in distributed storage systems

In this paper we consider the problem of maintaining a consistent mapping of a virtual object space to a set of memory modules, i.e. the object space can be decomposed into a set of ranges where every module is responsible for exactly one range. A module owning some range R is responsible for storing all objects in R. Besides consistency, we require the mapping to be compact, i.e. any object or consecutive range of objects should be spread out over as few memory modules as possible. A compact mapping is important for many applications such as efficiently executing programs using a large amount of space or complex search queries such as semi-group range queries. Our main result assumes a static set of memory modules of uniform capacity, but we also show how to extend this to a dynamic set of memory modules of non-uniform capacity in a decentralized environment.In both settings, new objects may be added, old objects may be deleted, or objects may be modified over time. Each object consists of a set of data blocks of uniform size. So insert, delete, or modify operations on objects can be seen as insert or delete operations of data blocks. Each module can send or receive at most one data block in each unit of time and the injection of insert or delete requests for data blocks is under adversarial control. We prove asymptotically tight upper and lower bounds on the maximum rate at which the adversary can inject requests into the system so that a consistent and compact placement can be preserved without exceeding the capacity of a module at any time. Specifically, we show that in a (1-ε)-utilized system (i.e. the available space is used up to an ε fraction) the maximum injection rate that can be sustained is Θ(ε).