Abstract
MapReduce has emerged as a prominent programming model for data-intensive computation. In this work, we study power-aware MapReduce scheduling in the speed scaling setting first introduced by Yao et al. [FOCS 1995]. We focus on the minimization of the total weighted completion time of a set of MapReduce jobs under a given budget of energy. Using a linear programming relaxation of our problem, we derive a polynomial time constant-factor approximation algorithm. We also propose a convex programming formulation that we combine with standard list scheduling policies, and we evaluate their performance using simulations.
This work was partially supported by the European Union (European Social Fund - ESF) and Greek national funds, through the Operational Program ”Education and Lifelong Learning”, under the programs THALES-ALGONOW (E. Bampis, G. Lucarelli, I. Milis) and HERACLEITUS II (G. Zois), and the project “Mathematical Programming and Non-linear Combinatorial Optimization” under the program PGMO (E. Bampis, V. Chau, G. Lucarelli).
Chapter PDF
Similar content being viewed by others
Keywords
- Completion Time
- Precedence Constraint
- Feasible Schedule
- Linear Programming Relaxation
- Total Weighted Completion Time
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Albers, S.: Algorithms for dynamic speed scaling. In: STACS, pp. 1–11 (2011)
Angel, E., Bampis, E., Kacem, F.: Energy aware scheduling for unrelated parallel machines. In: Green Computing Conference, pp. 533–540 (2012)
Bansal, N., Pruhs, K., Stein, C.: Speed scaling for weighted flow time. SIAM J. on Computing 39(4), 1294–1308 (2009)
Chang, H., Kodialam, M.S., Kompella, R.R., Lakshman, T.V., Lee, M., Mukherjee, S.: Scheduling in mapreduce-like systems for fast completion time. In: INFOCOM, pp. 3074–3082 (2011)
Chen, F., Kodialam, M.S., Lakshman, T.V.: Joint scheduling of processing and shuffle phases in mapreduce systems. In: INFOCOM, pp. 1143–1151 (2012)
Feller, E., Ramakrishnan, L., Morin, C.: On the performance and energy efficiency of Hadoop deployment models. In: BigData Conference, pp. 131–136 (2013)
Feng, B., Lu, J., Zhou, Y., Yang, N.: Energy efficiency for MapReduce workloads: An in-depth study. In: ADC, pp. 61–70 (2012)
Goiri, I., Le, K., Nguyen, T.D., Guitart, J., Torres, J., Bianchini, R.: GreenHadoop: leveraging green energy in data-processing frameworks. In: EuroSys, pp. 57–70 (2012)
Hall, L.A., Shmoys, D.B., Wein, J.: Scheduling to minimize average completion time: Off-line and on-line algorithms. In: ACM-SIAM SODA, pp. 142–151 (1996)
Mastrolilli, M., Queyranne, M., Schulz, A.S., Svensson, O., Uhan, N.A.: Minimizing the sum of weighted completion times in a concurrent open shop. Oper. Res. Letters 38(5), 390–395 (2010)
Megow, N., Verschae, J.: Dual techniques for scheduling on a machine with varying speed. In: Fomin, F.V., Freivalds, R., Kwiatkowska, M., Peleg, D. (eds.) ICALP 2013, Part I. LNCS, vol. 7965, pp. 745–756. Springer, Heidelberg (2013)
Moseley, B., Dasgupta, A., Kumar, R., Sarlós, T.: On scheduling in map-reduce and flow-shops. In: ACM-SPAA, pp. 289–298 (2011)
Phillips, C.A., Stein, C., Wein, J.: Minimizing average completion time in the presence of release dates. Math. Programming 82(1-2), 199–223 (1998)
Pruhs, K., van Stee, R., Uthaisombut, P.: Speed scaling of tasks with precedence constraints. Theory Comput. Syst. 43, 67–80 (2008)
Roemer, T.A.: A note on the complexity of the concurrent open shop problem. Journal of Scheduling 9, 389–396 (2006)
Schulz, A.S., Skutella, M.: Scheduling unrelated machines by randomized rounding. SIAM J. Discr. Mathematics 15(4), 450–469 (2002)
Wirtz, T., Ge, R.: Improving MapReduce energy efficiency for computation intensive workloads. In: IGCC, pp. 1–8 (2011)
Yao, F.F., Demers, A.J., Shenker, S.: A scheduling model for reduced cpu energy. In: IEEE- FOCS, pp. 374–382 (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Bampis, E., Chau, V., Letsios, D., Lucarelli, G., Milis, I., Zois, G. (2014). Energy Efficient Scheduling of MapReduce Jobs. In: Silva, F., Dutra, I., Santos Costa, V. (eds) Euro-Par 2014 Parallel Processing. Euro-Par 2014. Lecture Notes in Computer Science, vol 8632. Springer, Cham. https://doi.org/10.1007/978-3-319-09873-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-09873-9_17
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-09872-2
Online ISBN: 978-3-319-09873-9
eBook Packages: Computer ScienceComputer Science (R0)