Competitive algorithms from competitive equilibria: non-clairvoyant scheduling under polyhedral constraints

We introduce and study a general scheduling problem that we term the Packing Scheduling problem (PSP). In this problem, jobs can have different arrival times and sizes; a scheduler can process job j at rate x_j, subject to arbitrary packing constraints over the set of rates (x) of the outstanding jobs. The PSP framework captures a variety of scheduling problems, including the classical problems of unrelated machines scheduling, broadcast scheduling, and scheduling jobs of different parallelizability. It also captures scheduling constraints arising in diverse modern environments ranging from individual computer architectures to data centers. More concretely, PSP models multidimensional resource requirements and parallelizability, as well as network bandwidth requirements found in data center scheduling.

In this paper, we design non-clairvoyant online algorithms for PSP and its special cases -- in this setting, the scheduler is unaware of the sizes of jobs. Our results are summarized as follows.

• For minimizing total weighted completion time, we show a O(1)-competitive algorithm. Surprisingly, we achieve this result by applying the well-known Proportional Fairness algorithm (PF) to perform allocations each time instant. Though PF has been extensively studied in the context of maximizing fairness in resource allocation, we present the first analysis in adversarial and general settings for optimizing job latency. Our result is also the first O(1)-competitive algorithm for weighted completion time for several classical non-clairvoyant scheduling problems.

•For minimizing total weighted flow time, for any constant ε > 0, any O(n^1---ε)-competitive algorithm requires extra speed (resource augmentation) compared to the offline optimum. We show that PF is a O(log n)-speed O(log n)-competitive non-clairvoyant algorithm, where n is the total number of jobs. We further show that there is an instance of PSP for which no non-clairvoyant algorithm can be O(n^1---ε)-competitive with o(√log n) speed.

•For the classical problem of minimizing total flow time for unrelated machines in the non-clairvoyant setting, we present the first online algorithm which is scalable ((1 + ε)-speed O(1)-competitive for any constant ε > 0). No non-trivial results were known for this setting, and the previous scalable algorithm could handle only related machines. We develop new algorithmic techniques to handle the unrelated machines setting that build on a new single machine scheduling policy. Since unrelated machine scheduling is a special case of PSP, when contrasted with the lower bound for PSP, our result also shows that PSP is significantly harder than perhaps the most general classical scheduling settings.

Our results for PSP show that instantaneous fair scheduling algorithms can also be effective tools for minimizing the overall job latency, even when the scheduling decisions are non-clairvoyant and constrained by general packing constraints.