research-article

Free access

Fair, effective, efficient and differentiated scheduling in an enterprise data warehouse

Authors:

Umesh DayalAuthors Info & Claims

EDBT '09: Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology

Pages 696 - 707

https://doi.org/10.1145/1516360.1516441

Published: 24 March 2009 Publication History

Abstract

A typical online Business Intelligence (BI) workload consists of a combination of short, less intensive queries, along with long, resource intensive queries. As such, the longest queries in a typical BI workload may take several orders of magnitude more time to execute, compared with the shortest queries in the workload. This makes it challenging to design a good Mixed Workload Scheduler (MWS). In this paper we first define the design criteria that make a 'good' MWS. We then use these criteria to design rFEED, a MWS that is fair, effective, efficient, and differentiated. We simulate real workloads and compare our rFEED MWS with models of the current best of breed commercial systems. We show that the rFEED MWS works extremely well.

References

[1]

N. Bansal and K. Pruhs. Server scheduling in the lp norm: a rising tide lifts all boat. In STOC, pages 242--250, 2003.

Digital Library

[2]

L. Becchetti, S. Leonardi, A. Marchetti-Spaccamela, and K. Pruhs. Online weighted flow time and deadline scheduling. J. Discrete Algorithms, 4(3):339--352, 2006.

[3]

A. Bedekar, S. Borst, K. Ramanan, P. Whiting, and E. Yeh. Downlink scheduling in CDMA data networks. GLOBECOM, 5:2653--2657, 1999.

[4]

M. A. Bender, S. Chakrabarti, and S. Muthukrishnan. Flow and Stretch Metrics for Scheduling Continuous Job Streams. In SODA, pages 270--279, 1998.

Digital Library

[5]

M. A. Bender, S. Muthukrishnan, and R. Rajaraman. Improved algorithms for stretch scheduling. In SODA, pages 762--771, 2002.

Digital Library

[6]

P. Brucker. Scheduling Algorithms. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 1995.

Digital Library

[7]

Carrie Ballinger. The Wild World of Mixed Workload: Priorities and resources learn to get along. Teradata Magazine Online. http://www.teradata.com/t/go.aspx/?id=114533.

[8]

B. Chandramouli, C. N. Bond, S. Babu, and J. Yang. Query suspend and resume. In SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, pages 557--568, New York, NY, USA, 2007. ACM.

Digital Library

[9]

C. Chekuri and S. Khanna. Approximation schemes for preemptive weighted flow time. In STOC, pages 297--305, 2002.

Digital Library

[10]

C. Chekuri, S. Khanna, and A. Zhu. Algorithms for minimizing weighted flow time. In STOC, pages 84--93, 2001.

Digital Library

[11]

L. Cherkasova and T. Rokicki. Alpha Message Scheduling for Packet-Switched Interconnects. Technical Report HPL-94-71, HP Labs, August 1994.

[12]

M. Crovella, R. Frangioso, and M. Harchol-Balter. Connection scheduling in web servers. In USENIX Symposium on Internet Technologies and Systems, 1999.

Digital Library

[13]

E. J. Friedman and S. G. Henderson. Fairness and efficiency in web server protocols. In SIGMETRICS, pages 229--237, 2003.

Digital Library

[14]

J. R. M. Hosking and J. F. Wallis. Parameter and quantile estimation for the generalized pareto distribution. Technometrics, 29(3):339--349, 1987.

Digital Library

[15]

IBM. DB2 Query Patroller. http://www-306.ibm.com/software/data/db2/querypatroller/.

[16]

B. Kalyanasundaram, K. Pruhs, and M. Velauthapillai. Scheduling Broadcasts in Wireless Networks. In Proceedings of the 8th Annual European Symposium on Algorithms (ESA), pages 290--301, 2000.

Digital Library

[17]

A. Legrand, A. Su, and F. Vivien. Minimizing the stretch when scheduling flows of biological requests. In SPAA, pages 103--112, 2006.

Digital Library

[18]

J. Leung, L. Kelly, and J. H. Anderson. Handbook of Scheduling: Algorithms, Models, and Performance Analysis. CRC Press, Inc., Boca Raton, FL, USA, 2004.

Digital Library

[19]

A. Mehta, C. Gupta, S. Wang, and U. Dayal. rfeed: A mixed workload scheduler for enterprise data warehouses. In ICDE '09, Accepted, 2009.

Digital Library

[20]

S. Muthukrishnan, R. Rajaraman, A. Shaheen, and J. E. Gehrke. Online scheduling to minimize average stretch. In Proceedings of the 40th Annual Symposium on Foundations of Computer Science (FOCS), page 433, 1999.

Digital Library

[21]

V. Paxson. End-to-end internet packet dynamics. IEEE/ACM Trans. Netw., 7(3):277--292, 1999.

Digital Library

[22]

B. Schroeder, M. Harchol-Balter, A. Iyengar, E. M. Nahum, and A. Wierman. How to Determine a Good Multi-Programming Level for External Scheduling. In ICDE, page 60, 2006.

Digital Library

[23]

M. A. Sharaf, P. K. Chrysanthis, A. Labrinidis, and K. Pruhs. Efficient scheduling of heterogeneous continuous queries. In VLDB, pages 511--522, 2006.

Digital Library

Cited By

Sabek IUkyab TKraska TIves ZBonifati AEl Abbadi A(2022)LSched: A Workload-Aware Learned Query Scheduler for Analytical Database SystemsProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526158(1228-1242)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3526158
Gaffney KClaus RPatel J(2021)Database isolation by schedulingProceedings of the VLDB Endowment10.14778/3461535.346153714:9(1467-1480)Online publication date: 22-Oct-2021
https://dl.acm.org/doi/10.14778/3461535.3461537
Pitoura EStefanidis KKoutrika G(2021)Fairness in rankings and recommendations: an overviewThe VLDB Journal10.1007/s00778-021-00697-y31:3(431-458)Online publication date: 2-Oct-2021
https://doi.org/10.1007/s00778-021-00697-y
Show More Cited By

Fair, effective, efficient and differentiated scheduling in an enterprise data warehouse
1. Theory of computation
  1. Design and analysis of algorithms
    1. Approximation algorithms analysis
    2. Online algorithms
      1. Online learning algorithms
  2. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning

Recommendations

Enterprise traffic with a differentiated service mechanism

As an enterprise grows, its network bandwidth requirement also increases dramatically. Network performance of an enterprise will impact its operational performance. How do we efficiently manage an enterprise network's resources in an E‐business era? ...
Efficient Execution of Range-Aggregate Queries in Data Warehouse Environments
ER '01: Proceedings of the 20th International Conference on Conceptual Modeling: Conceptual Modeling

Range-aggregate queries on the data cube are powerful tools for analysis in data warehouse environments. Cubetree is a technique materializing a data cube through an R-tree. It provides efficient data accessibility, but involves some drawbacks to ...
Fair lateness scheduling: reducing maximum lateness in G-EDF-like scheduling

In prior work on soft real-time (SRT) multiprocessor scheduling, tardiness bounds have been derived for a variety of scheduling algorithms, most notably, the global earliest-deadline-first (G-EDF) algorithm. In this paper, we devise G-EDF-like (GEL) ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EDBT '09: Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology

March 2009

1180 pages

ISBN:9781605584225

DOI:10.1145/1516360

Editors:
Martin Kersten
CWI, The Netherlands
,
Boris Novikov
University of Saint Petersburg, Russia
,
Jens Teubner
ETH Zurich, Switzerland
,
Vladimir Polutin
HP Labs, Russia
,
Stefan Manegold
CWI, The Netherlands

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 March 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

EDBT/ICDT '09

EDBT/ICDT '09: EDBT/ICDT '09 joint conference

March 24 - 26, 2009

Saint Petersburg, Russia

Acceptance Rates

Overall Acceptance Rate 7 of 10 submissions, 70%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
420
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)12

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sabek IUkyab TKraska TIves ZBonifati AEl Abbadi A(2022)LSched: A Workload-Aware Learned Query Scheduler for Analytical Database SystemsProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526158(1228-1242)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3526158
Gaffney KClaus RPatel J(2021)Database isolation by schedulingProceedings of the VLDB Endowment10.14778/3461535.346153714:9(1467-1480)Online publication date: 22-Oct-2021
https://dl.acm.org/doi/10.14778/3461535.3461537
Pitoura EStefanidis KKoutrika G(2021)Fairness in rankings and recommendations: an overviewThe VLDB Journal10.1007/s00778-021-00697-y31:3(431-458)Online publication date: 2-Oct-2021
https://doi.org/10.1007/s00778-021-00697-y
Vieira ADias LSantos MPereira GOliveira J(2019)Real-Time Supply Chain Simulation: A Big Data-Driven Approach2019 Winter Simulation Conference (WSC)10.1109/WSC40007.2019.9004717(548-559)Online publication date: Dec-2019
https://doi.org/10.1109/WSC40007.2019.9004717
Zhang MMartin PPowley WChen J(2018)Workload Management in Database Management Systems: A TaxonomyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2017.276704430:7(1386-1402)Online publication date: 1-Jul-2018
https://doi.org/10.1109/TKDE.2017.2767044
Deshmukh HMemisoglu HPatel J(2017)Adaptive Concurrent Query Execution Framework for an Analytical In-Memory Database System2017 IEEE International Congress on Big Data (BigData Congress)10.1109/BigDataCongress.2017.13(23-30)Online publication date: Jun-2017
https://doi.org/10.1109/BigDataCongress.2017.13
Borovica-Gajić RAppuswamy RAilamaki A(2016)Cheap data analytics using cold storage devicesProceedings of the VLDB Endowment10.14778/2994509.29945219:12(1029-1040)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.14778/2994509.2994521
Wust JGrund MPlattner H(2015)Dynamic Query Prioritization for In-Memory DatabasesIn Memory Data Management and Analysis10.1007/978-3-319-13960-9_5(56-68)Online publication date: 14-Jan-2015
https://doi.org/10.1007/978-3-319-13960-9_5
Jiang JPeng ZWu XLiang N(2014)Sharing-Aware Scheduling of Web ServicesWeb Technologies and Applications10.1007/978-3-319-11116-2_14(153-164)Online publication date: 2014
https://doi.org/10.1007/978-3-319-11116-2_14
Wust JGrund MHoewelmeyer KSchwalb DPlattner H(2014)Concurrent Execution of Mixed Enterprise Workloads on In-Memory DatabasesDatabase Systems for Advanced Applications10.1007/978-3-319-05810-8_9(126-140)Online publication date: 2014
https://doi.org/10.1007/978-3-319-05810-8_9
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents