Dynamic batching policies for an on-demand video server

Reviewer: Arvid G. Larson

In a video-on-demand environment, continuous delivery of video streams…[implies] a hard limit on the number of streams that a video server can deliver. Multiple client requests for the same video [program] can be served with a single disk I/O stream by sending (multicasting) the same data blocks to multiple clients. This is achieved by batching (grouping) requests for the same video that arrive within a short time. …For an empirical workload, we demonstrate a substantial reduction (of the order of 60%) in the required server capacity by batching. —From the Authors' Abstract This paper explores various statistical aspects of video-on-demand service and operational strategies to ensure optimal performance. The authors hypothesize customer behavior in a structured service environment, considering such factors as acceptable customer waiting time and video request reneging if the service delay exceeds customer tolerance. Several analytic performance models are developed. For example, given a well-defined structured service environment, such models show that a first-come, first-served policy that schedules the video with the longest outstanding request can perform better than a maximum-queue-length policy that selects the video with the most outstanding requests. Furthermore, such models show that multicasting is better exploited by scheduling playback of the most popular videos at predetermined, regular intervals. Such scheduling is also shown to further improve performance, due to reduction of order reneging, if this selected playback interval is guaranteed to be the maximum waiting time experienced by any customer. The hypothesized server operational policy must consider the relationships among several competing performance objectives. It is desirable to minimize the average waiting time before a client is served, while simultaneously reducing the reneging probability for all video requests caused by unacceptable service delays. Concurrently, while client waiting-time behavior is a strong function of experienced delay, it is also desirable to be fair to all requests, irrespective of the popularity of individual videos. Given a finite server capacity and bandwidth, in addition to considerations of the impact of peak-time system loading, these operational performance tradeoffs may be more complex than is suggested by the simple dynamic batching policy utilized within this model. Nevertheless, the authors demonstrate that what seems to be an intuitively obvious best-practice operational policy can be supported by the results obtained from their analytical model. This paper may appeal most to statistically oriented system design professionals and those with a general queueing-theoretic bent. I also recommend it to students of statistical behavioral psychology.