Abstract
In recent years the availability of parallel computation resources has grown rapidly. Nevertheless, even for the most widely studied constraint programming problems such as SAT, solver development and applications remain largely focussed on sequential rather than parallel approaches. To ease the burden usually associated with designing, implementing and testing parallel solvers, in this chapter, we demonstrate how methods from automatic algorithm design can be used to construct effective parallel portfolio solvers from sequential components. Specifically, we discuss two prominent approaches for this problem. (I) Parallel portfolio selection involves selecting a parallel portfolio consisting of complementary sequential solvers for a specific instance to be solved (as characterised by cheaply computable instance features). Applied to a broad set of sequential SAT solvers from SAT competitions, we show that our generic approach achieves nearly linear speedup on application instances, and super-linear speedups on combinatorial and random instances. (II) Automatic construction of parallel portfolios via algorithm configuration involves a parallel portfolio of algorithm parameter configurations that is optimized for a given set of instances. Applied to gold-medal-winning parameterized SAT solvers, we show that our approach can produce significantly better-performing SAT solvers than state-of-the-art parallel solvers constructed by human experts, reducing time-outs by 17% and running time (PAR10 score) by 13% under competition conditions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Aigner, M., Biere, A., Kirsch, C., Niemetz, A., Preiner, M.: Analysis of portfolio-style parallel SAT solving on current multi-core architectures. In: Berre, D.L. (ed.) Proceedings of the Fourth Pragmatics of SAT workshop. EPiC Series in Computing, vol. 29, pp. 28–40. EasyChair (2014)
Alfonso, E., Manthey, N.: New CNF features and formula classification. In: Berre, D.L. (ed.) Proceedings of the Fifth Pragmatics of SAT workshop. EPiC Series in Computing, vol. 27, pp. 57–71. EasyChair (2014)
Alfonso, E., Manthey, N.: Riss 4.27 BlackBox. In: Belov, A., Diepold, D., Heule, M., Järvisalo, M. (eds.) Proceedings of SAT Competition 2014. Department of Computer Science Series of Publications B, vol. B-2014-2, pp. 68–69. University of Helsinki, Helsinki, Finland (2014)
Ansótegui, C., Malitsky, Y., Sellmann, M.: MaxSAT by improved instancespecific algorithm configuration. In: Brodley, C., Stone, P. (eds.) Proceedings of the Twenty-eighth National Conference on Artificial Intelligence (AAAI’14). pp. 2594–2600. AAAI Press (2014)
Ansótegui, C., Sellmann, M., Tierney, K.: A gender-based genetic algorithm for the automatic configuration of algorithms. In: Gent, I. (ed.) Proceedings of the Fifteenth International Conference on Principles and Practice of Constraint Programming (CP’09). Lecture Notes in Computer Science, vol. 5732, pp. 142–157. Springer-Verlag (2009)
Audemard, G., Hoessen, B., Jabbour, S., Lagniez, J.M., Piette, C.: Penelope, a parallel clause-freezer solver. In: Balint et al. [10], pp. 43–44
Audemard, G., Simon, L.: Glucose 2.1. in the SAT challenge 2012. In: Balint et al. [10], pp. 23–23
Audemard, G., Simon, L.: Lazy clause exchange policy for parallel SAT solvers. In: Sinz, C., Egly, U. (eds.) Proceedings of the Seventeenth International Conference on Theory and Applications of Satisfiability Testing (SAT’14). Lecture Notes in Computer Science, vol. 8561, pp. 197–205. Springer (2014)
Balaprakash, P., Birattari, M., Stützle, T.: Improvement strategies for the F-Race algorithm: Sampling design and iterative refinement. In: Bartz-Beielstein, T., Aguilera, M., Blum, C., Naujoks, B., Roli, A., Rudolph, G., Sampels, M. (eds.) International Workshop on Hybrid Metaheuristics. Lecture Notes in Computer Science, vol. 4771, pp. 108–122. Springer (2007)
Balint, A., Belov, A., Diepold, D., Gerber, S., Järvisalo, M., Sinz, C. (eds.): Proceedings of SAT Challenge 2012: Solver and Benchmark Descriptions. University of Helsinki (2012)
Balint, A., Belov, A., Heule, M., Järvisalo, M. (eds.): Proceedings of SAT Competition 2013: Solver and Benchmark Descriptions, Department of Computer Science Series of Publications B, vol. B-2013-1. University of Helsinki (2013)
Belov, A., Diepold, D., Heule, M., Järvisalo, M.: The application and the hard combinatorial benchmarks in SAT competition 2014. In: Belov, A., Diepold, D., Heule, M., Järvisalo, M. (eds.) Proceedings of SAT Competition 2014. Department of Computer Science Series of Publications B, vol. B-2014-2, pp. 80–83. University of Helsinki, Helsinki, Finland (2014)
Biere, A.: Lingeling, Plingeling, PicoSAT and PrecoSAT at SAT race 2010. Tech. Rep. 10/1, Institute for Formal Models and Verification. Johannes Kepler University (2010)
Biere, A.: Lingeling and friends at the SAT competition 2011. Technical Report FMV 11/1, Institute for Formal Models and Verification, Johannes Kepler University (2011)
Biere, A.: Lingeling, Plingeling and Treengeling entering the SAT competition 2013. In: Balint et al. [11], pp. 51–52
Biere, A.: Lingeling and friends entering the SAT race 2015. Tech. rep., Institute for Formal Models and Verification, Johannes Kepler University (2015)
Bischl, B., Kerschke, P., Kotthoff, L., Lindauer, M., Malitsky, Y., Frechétte, A., Hoos, H., Hutter, F., Leyton-Brown, K., Tierney, K., Vanschoren, J.: ASlib: A benchmark library for algorithm selection. Artificial Intelligence 237, 41–58 (2016)
Brochu, E., Cora, V., de Freitas, N.: A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. Computing Research Repository (CoRR) http://arXiv.org/abs/1012.2599 (2010)
Cameron, C., Hoos, H., Leyton-Brown, K.: Bias in algorithm portfolio performance evaluation. In: Kambhampati, S. (ed.) Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI’16). pp. 712–719. IJCAI/AAAI Press (2016)
Cheeseman, P., Kanefsky, B., Taylor, W.: Where the really hard problems are. In: Mylopoulos, J., Reiter, R. (eds.) Proceedings of the 12th International Joint Conference on Artificial Intelligence. pp. 331–340. Morgan Kaufmann (1991)
Chen, J.: Phase selection heuristics for satisfiability solvers. CoRR http://arXiv.org/abs/1106.1372(v1) (2011)
Cimatti, A., Sebastiani, R. (eds.): Proceedings of the Fifteenth International Conference on Theory and Applications of Satisfiability Testing (SAT’12), Lecture Notes in Computer Science, vol. 7317. Springer-Verlag (2012)
Falkner, S., Lindauer, M., Hutter, F.: SpySMAC: Automated configuration and performance analysis of SAT solvers. In: Heule, M., Weaver, S. (eds.) Proceedings of the Eighteenth International Conference on Theory and Applications of Satisfiability Testing (SAT’15). pp. 1–8. Lecture Notes in Computer Science, Springer-Verlag (2015)
Gebser, M., Kaufmann, B., Schaub, T.: Conflict-driven answer set solving: From theory to practice. Artificial Intelligence 187-188, 52–89 (2012)
Gebser, M., Kaufmann, B., Schaub, T.: Multi-threaded ASP solving with clasp. TPLP 12(4-5), 525–545 (2012)
van Gelder, A.: Contrasat - a contrarian SAT solver. Journal on Satisfiability, Boolean Modeling and Computation 8(1/2), 117–122 (2012)
Grinten, A., Wotzlaw, A., Speckenmeyer, E., Porschen, S.: satUZK: Solver description. In: Balint et al. [10], pp. 54–55
Hamadi, Y., Jabbour, S., Sais, L.: ManySAT: a parallel SAT solver. Journal on Satisfiability, Boolean Modeling and Computation 6, 245–262 (2009)
Hamadi, Y., Schoenauer, M. (eds.): Proceedings of the Sixth International Conference on Learning and Intelligent Optimization (LION’12), Lecture Notes in Computer Science, vol. 7219. Springer-Verlag (2012)
Harman, M., Jones, B.: Search-based software engineering. Information and Software Technology 43(14), 833–839 (2001)
Helmert, M., Röger, G., Karpas, E.: Fast Downward Stone Soup: A baseline for building planner portfolios. In: ICAPS-2011 Workshop on Planning and Learning (PAL). pp. 28–35 (2011)
Herwig, P.: Using graphs to get a better insight into satisfiability problems. Master’s thesis, Delft University of Technology, Department of Electrical Engineering, Mathematics and Computer Science (2006)
Heule, M., Dufour, M., van Zwieten, J., van Maaren, H.: March_eq: Implementing additional reasoning into an efficient look-ahead SAT solver. In: Hoos, H., Mitchell, D. (eds.) Proceedings of the Seventh International Conference on Theory and Applications of Satisfiability Testing (SAT’04). Lecture Notes in Computer Science, vol. 3542, pp. 345–359. Springer-Verlag (2004)
Holte, R., Howe, A. (eds.): Proceedings of the Twenty-second National Conference on Artificial Intelligence (AAAI’07). AAAI Press (2007)
Hoos, H.: Programming by optimization. Communications of the ACM 55(2), 70–80 (2012)
Hoos, H., Kaminski, R., Lindauer, M., Schaub, T.: aspeed: Solver scheduling via answer set programming. Theory and Practice of Logic Programming 15, 117–142 (2015)
Hoos, H., Lindauer, M., Schaub, T.: claspfolio 2: Advances in algorithm selectionfor answer set programming. Theory and Practice of Logic Programming 14, 569–585 (2014)
Hsu, E., Muise, C., Beck, C., McIlraith, S.: Probabilistically estimating backbones and variable bias: Experimental overview. In: Stuckey, P. (ed.) Proceedings of the Fourteenth International Conference on Principles and Practice of Constraint Programming (CP’08). Lecture Notes in Computer Science, vol. 5202, pp. 613–617. Springer (2008)
Huberman, B., Lukose, R., Hogg, T.: An economic approach to hard computational problems. Science 275, 51–54 (1997)
Hutter, F., Babi´c, D., Hoos, H., Hu, A.: Boosting verification by automatic tuning of decision procedures. In: O’Conner, L. (ed.) Formal Methods in Computer Aided Design (FMCAD’07). pp. 27–34. IEEE Computer Society Press (2007)
Hutter, F., Hoos, H., Leyton-Brown, K.: Automated configuration of mixed integer programming solvers. In: Lodi, A., Milano, M., Toth, P. (eds.) Proceedings of the Seventh International Conference on Integration of AI and OR Techniques in Constraint Programming (CPAIOR’10). Lecture Notes in Computer Science, vol. 6140, pp. 186–202. Springer-Verlag (2010)
Hutter, F., Hoos, H., Leyton-Brown, K.: Bayesian optimization with censored response data. In: NIPS workshop on Bayesian Optimization, Sequential Experimental Design, and Bandits (BayesOpt’11) (2011)
Hutter, F., Hoos, H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C. (ed.) Proceedings of the Fifth International Conference on Learning and Intelligent Optimization (LION’11). Lecture Notes in Computer Science, vol. 6683, pp. 507–523. Springer-Verlag (2011)
Hutter, F., Hoos, H., Leyton-Brown, K.: Parallel algorithm configuration. In: Hamadi and Schoenauer [29], pp. 55–70
Hutter, F., Hoos, H., Leyton-Brown, K., Stützle, T.: ParamILS: An automatic algorithm configuration framework. Journal of Artificial Intelligence Research 36, 267–306 (2009)
Hutter, F., Hoos, H., Stützle, T.: Automatic algorithm configuration based on local search. In: Holte and Howe [34], pp. 1152–1157
Hutter, F., Lindauer, M., Balint, A., Bayless, S., Hoos, H., Leyton-Brown, K.: The configurable SAT solver challenge (CSSC). Artificial Intelligence Journal (AIJ) 243, 1–25 (2017)
Hutter, F., Xu, L., Hoos, H., Leyton-Brown, K.: Algorithm runtime prediction: Methods and evaluation. Artificial Intelligence 206, 79–111 (2014)
Kadioglu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm selection and scheduling. In: Lee, J. (ed.) Proceedings of the Seventeenth International Conference on Principles and Practice of Constraint Programming (CP’11). Lecture Notes in Computer Science, vol. 6876, pp. 454–469. Springer-Verlag (2011)
Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC - instance-specific algorithm configuration. In: Coelho, H., Studer, R., Wooldridge, M. (eds.) Proceedings of the Nineteenth European Conference on Artificial Intelligence (ECAI’10). pp. 751–756. IOS Press (2010)
Kotthoff, L.: Algorithm selection for combinatorial search problems: A survey. AI Magazine pp. 48–60 (2014)
Kotthoff, L.: Ranking algorithms by performance. In: Pardalos, P., Resende, M. (eds.) Proceedings of the Eighth International Conference on Learning and Intelligent Optimization (LION’14). Lecture Notes in Computer Science, Springer-Verlag (2014)
Kotthoff, L., Gent, I., Miguel, I.: An evaluation of machine learning in algorithm selection for search problems. AI Communications 25(3), 257–270 (2012)
Li, C.M., Wei, W., Li, Y.: Exploiting historical relationships of clauses and variables in local search for satisfiability. In: Cimatti and Sebastiani [22], pp. 479–480
Lindauer, M., Bergdoll, D., Hutter, F.: An empirical study of per-instance algorithm scheduling. In: Festa, P., Sellmann, M., Vanschoren, J. (eds.) Proceedings of the Tenth International Conference on Learning and Intelligent Optimization (LION’16). pp. 253–259. Lecture Notes in Computer Science, Springer-Verlag (2016)
Lindauer, M., Hoos, H., Hutter, F.: From sequential algorithm selection to parallel portfolio selection. In: Dhaenens, C., Jourdan, L., Marmion, M. (eds.) Proceedings of the Ninth International Conference on Learning and Intelligent Optimization (LION’15). pp. 1–16. Lecture Notes in Computer Science, Springer-Verlag (2015)
Lindauer, M., Hoos, H., Hutter, F., Schaub, T.: Autofolio: An automatically configured algorithm selector. Journal of Artificial Intelligence Research 53, 745–778 (Aug 2015)
Lindauer, M., Hoos, H., Leyton-Brown, K., Schaub, T.: Automatic construction of parallel portfolios via algorithm configuration. Artificial Intelligence 244, 272–290 (2017)
López-Ibáñez, M., Dubois-Lacoste, J., Caceres, L.P., Birattari, M., Stützle, T.: The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives 3, 43–58 (2016)
Malitsky, Y., Mehta, D., O’Sullivan, B.: Evolving instance specific algorithm configuration. In: Helmert, M., Röger, G. (eds.) Proceedings of the Sixth Annual Symposium on Combinatorial Search (SOCS). AAAI Press (2013)
Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Parallel SAT solver selection and scheduling. In: Milano, M. (ed.) Proceedings of the Eighteenth International Conference on Principles and Practice of Constraint Programming (CP’12). Lecture Notes in Computer Science, vol. 7514, pp. 512–526. Springer-Verlag (2012)
Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm portfolios based on cost-sensitive hierarchical clustering. In: Rossi, F. (ed.) Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI’13). pp. 608–614 (2013)
Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Parallel lingeling, CCASat, and CSCH-based portfolio. In: Balint et al. [11], pp. 26–27
Maratea, M., Pulina, L., Ricca, F.: A multi-engine approach to answer-set programming. Theory and Practice of Logic Programming 14, 841–868 (2014)
Nudelman, E., Leyton-Brown, K., Andrew, G., Gomes, C., McFadden, J., Selman, B., Shoham, Y.: Satzilla 0.9 (2003), solver description, International SAT Competition
Nudelman, E., Leyton-Brown, K., Hoos, H., Devkar, A., Shoham, Y.: Understanding random SAT: beyond the clauses-to-variables ratio. In: Wallace, M. (ed.) Proceedings of the international conference on Principles and Practice of Constraint Programming. Lecture Notes in Computer Science, vol. 3258, pp. 438–452. Springer (2004)
O’Mahony, E., Hebrard, E., Holland, A., Nugent, C., O’Sullivan, B.: Using case based reasoning in an algorithm portfolio for constraint solving. In: Bridge, D., Brown, K., O’Sullivan, B., Sorensen, H. (eds.) Proceedings of the Nineteenth Irish Conference on Artificial Intelligence and Cognitive Science (AICS’08) (2008)
Pulina, L., Tacchella, A.: A self-adaptive multi-engine solver for quantified boolean formulas. Constraints 14(1), 80–116 (2009)
Rice, J.: The algorithm selection problem. Advances in Computers 15, 65–118 (1976)
Roussel, O.: Description of ppfolio (2011), available at http://www.cril.univ-artois.fr/~roussel/ppfolio/solver1.pdf
Schneider, M., Hoos, H.: Quantifying homogeneity of instance sets for algorithm configuration. In: Hamadi and Schoenauer [29], pp. 190–204
Schubert, T., Lewis, M., Becker, B.: Pamiraxt: Parallel SAT solving with threads and message passing. JSAT 6(4), 203–222 (2009)
Seipp, J., Sievers, S., Helmert, M., Hutter, F.: Automatic configuration of sequential planning portfolios. In: Bonet, B., Koenig, S. (eds.) Proceedings of the Twenty-ninth National Conference on Artificial Intelligence (AAAI’15). AAAI Press (2015)
Soos, M., Nohl, K., Castelluccia, C.: Extending SAT solvers to cryptographic problems. In: Kullmann, O. (ed.) Proceedings of the Twelfth International Conference on Theory and Applications of Satisfiability Testing (SAT’09). Lecture Notes in Computer Science, vol. 5584, pp. 244–257. Springer (2009)
Streeter, M., Golovin, D., Smith, S.: Combining multiple heuristics online. In: Holte and Howe [34], pp. 1197–1203
Thornton, C., Hutter, F., Hoos, H., Leyton-Brown, K.: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: I. Dhillon, Koren, Y., Ghani, R., Senator, T., Bradley, P., Parekh, R., He, J., Grossman, R., Uthurusamy, R. (eds.) The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’13). pp. 847–855. ACM Press (2013)
Tompkins, D., Balint, A., Hoos, H.: Captain Jack – new variable selection heuristics in local search for SAT. In: Sakallah, K., Simon, L. (eds.) Proceedings of the Fourteenth International Conference on Theory and Applications of Satisfiability Testing (SAT’11). Lecture Notes in Computer Science, vol. 6695, pp. 302–316. Springer (2011)
Wotzlaw, A., van der Grinten, A., Speckenmeyer, E., Porschen, S.: pfolioUZK: Solver description. In: Balint et al. [10], p. 45
Xu, L., Hoos, H., Leyton-Brown, K.: Hydra: Automatically configuring algorithms for portfolio-based selection. In: Fox, M., Poole, D. (eds.) Proceedings of the Twenty-fourth National Conference on Artificial Intelligence (AAAI’10). pp. 210–216. AAAI Press (2010)
Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: SATzilla: Portfolio-based algorithm selection for SAT. Journal of Artificial Intelligence Research 32, 565–606 (2008)
Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Hydra-MIP: Automated algorithm configuration and selection for mixed integer programming. In: RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJCAI) (2011)
Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Evaluating component solver contributions to portfolio-based algorithm selectors. In: Cimatti and Sebastiani [22], pp. 228–241
Xu, L., Hutter, F., Shen, J., Hoos, H., Leyton-Brown, K.: SATzilla2012: improved algorithm selection based on cost-sensitive classification models. In: Balint et al. [10], pp. 57–58
Yun, X., Epstein, S.: Learning algorithm portfolios for parallel execution. In: Hamadi and Schoenauer [29], pp. 323–338
Acknowledgement
M. Lindauer was supported by the DFG (German Research Foundation) under Emmy Noether grant HU 1900/2-1 and project SCHA 550/8-3, H. Hoos and K. Leyton-Brown by NSERC Discovery Grants, K. Leyton-Brown also by an NSERC E.W.R. Steacie Fellowship, and F. Hutter also by the DFG under Emmy Noether grant HU 1900/2-1.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Lindauer, M., Hoos, H., Hutter, F., Leyton-Brown, K. (2018). Selection and Configuration of Parallel Portfolios. In: Hamadi, Y., Sais, L. (eds) Handbook of Parallel Constraint Reasoning. Springer, Cham. https://doi.org/10.1007/978-3-319-63516-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-63516-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-63515-6
Online ISBN: 978-3-319-63516-3
eBook Packages: Computer ScienceComputer Science (R0)