Article

Parallel interval-Newton using message passing: dynamic load balancing strategies

Authors:

Mark A. StadtherrAuthors Info & Claims

SC '01: Proceedings of the 2001 ACM/IEEE conference on Supercomputing

Page 23

https://doi.org/10.1145/582034.582057

Published: 10 November 2001 Publication History

Abstract

Branch-and-prune and branch-and-bound techniques are commonly used for intelligent search in finding all solutions, or the optimal solution, within a space of interest. The corresponding binary tree structure provides a natural parallelism allowing concurrent evaluation of subproblems using parallel computing technology. Of special interest here are techniques derived from interval analysis, in particular an interval-Newton/generalized-bisection procedure. In this context, we discuss issues of load balancing and work scheduling that arise in the implementation of parallel interval-Newton on a cluster of workstations using message passing, and describe and analyze techniques for this purpose. Results using an asynchronous diffusive load balancing strategy show that a consistently high efficiency can be achieved in solving nonlinear equations, providing excellent scalability, especially with the use of a two-dimensional torus virtual network. The effectiveness of the approach used, especially in connection with a novel stack management scheme, is also demonstrated in the consistent superlinear speedups observed in performing global optimization.

References

[1]

Adjiman, C. S., Androulakis, I. P. & Floudas, C. A. (2000) Global optimization of mixed-integer nonlinear problems. AIChE J.,46, 1769.

[2]

Androulakis, I.P. & Floudas, C. A. (1998) Distributed branch and bound algorithms for global optimization. In Parallel Processing of Discrete Problems, ed. P. M. Pardalos, Springer-Verlag, Berlin.

[3]

Anstreicher, K. M., Brixius, N., Goux, J.-P. & Linderoth, J. (2000) Solving large quadratic assignment problems on computational grids. Presented at 17th International Symposium on Mathematical Programming, Atlanta, GA, August.

[4]

Balaji, G. V. & Seader, J. D. (1995) Application of interval-Newton method to chemical engineering problems. AIChE Symp. Ser.,91(304), 364.

[5]

Blumofe, R. D. & Leiserson, C. E. (1994) Scheduling multithreaded computations by work stealing. In Proceedings 35th Annual Symposium on Foundations of Computer Science, ed. S. Goldwasser, IEEE Computer Society Press, Los Alamitos, CA.

Digital Library

[6]

Blumofe, R. D. & Leiserson, C. E. (1999) Scheduling multithreaded computations by work stealing. J. ACM,46, 720.

Digital Library

[7]

Byrne, R. P. & Bogle, I. D. L. (2000) Global optimization of modular process flowsheets. Ind. Eng. Chem. Res.,39, 4296.

[8]

Clausen, J. & Traff, J. L. (1991) Implementations of parallel branch-and-bound algorithms-experience with the graph partitioning problem. Anns. Opns. Res.,33, 331.

[9]

Correa, R. C. (2000) A parallel approximation scheme for the multiprocessor scheduling problem. Parallel Computing,26, 47.

Digital Library

[10]

Correa, R. & Ferreira, A. (1996) Parallel best-first branch-and-bound in discrete optimization: A framework. In Solving Combinatorial Optimization Problems in Parallel, eds. A. Ferreira, & P. Pardalos, Springer-Verlag, Berlin.

Digital Library

[11]

Cybenko, G. (1989) Dynamic load balancing for distributed memory multiprocessors. J. Parallel Distr. Comp.,7, 278.

Digital Library

[12]

Dijkstra, E. W., Feijen, W. H. & van Gasteren, A. J. M. (1983) Derivation of a termination detection algorithm for distributed computations. Inf. Process. Lett.,16, 217.

[13]

Dijkstra, E. W. & Scholten, C. S. (1980) Termination detection for diffusing computations. Inf. Process. Lett.,11, 1.

[14]

Epperly, T. G. W. (1995) Global Optimization of Nonconvex Nonlinear Programs Using Parallel Branch and Bound. Ph.D. Thesis, University of Wisconsin, Madison, WI.

Digital Library

[15]

Finkel, R. & Manber, U. (1987) DIB-A distributed implementation of backtracking. ACM Tran. Prog. Lang. and Syst.,9, 235.

Digital Library

[16]

Foster, I. (1995) Designing and Building Parallel Programs --- Concepts and Tools for Parallel Software Engineering. Addison-Wesley, Reading, MA.

Digital Library

[17]

Frommer, A. & Mayer, G. (1989) Parallel interval multisplittings. Numer. Math.,56, 255.

Digital Library

[18]

Gan, Q., Yang, Q. & Hu, C. (1994) Parallel all-row preconditioned interval linear solver for nonlinear equations on multiprocessors. Parallel Computing,20, 1249.

Digital Library

[19]

Gau, C.-Y. & Stadtherr, M. A. (1998) Global nonlinear parameter estimation using interval analysis: Parallel computing strategies. Presented at AIChE Annual Meeting, Miami Beach, FL, November.

[20]

Gau, C.-Y. & Stadtherr, M. A. (2000) Reliable nonlinear parameter estimation using interval analysis: Error-in-variable approach. Comput. Chem. Eng.,24, 631.

[21]

Gendron, B. & Crainic, T. G. (1994) Parallel branch-and-bound algorithms-survey and synthesis. Oper. Res.,42, 1042.

Digital Library

[22]

Goux, J.-P., Kulkani, S., Linderoth, J. & Yoder, M. (2000) An enabling framework for master-worker computing applications on the computational grid. Technical Report, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL.

[23]

Gropp, W., Lusk, E. & Skjellum, A. (1994) Using MPI: Portable Parallel Programming with the Message-Passing Interface. MIT Press, Cambridge, MA.

Digital Library

[24]

Gropp, W., Lusk, E. & Thakur, R. (1999) Using MPI-2: Advanced Features of the Message-Passing Interface. MIT Press, Cambridge, MA.

Digital Library

[25]

Han, J. R., Manousiousthakis, V. & Choi, S. H. (1997) Global optimization of chemical processes using the interval analysis. Korean J. Chem. Eng.,14, 270.

[26]

Hansen, E. R. (1992). Global Optimization Using Interval Analysis, Marcel Dekkar, New York, NY.

[27]

Harding, S. T., Maranas, C. D., McDonald, C. M. & Floudas, C. A. (1997) Locating all homogeneous azeotropes in multicomponent mixtures. Ind. Eng. Chem. Res.,36, 160.

[28]

Harding, S. T. & Floudas, C. A. (2000) Locating all heterogeneous and reactive azeotropes in multicomponent mixtures. Ind. Eng. Chem. Res.,39, 1576.

[29]

Heirich, A. & Arvo, J. (1998) A competitive analysis of load balancing strategies for parallel ray tracing. J. Supercomputing,12, 57.

Digital Library

[30]

Heirich, A. & Taylor, S. (1995) Load balancing by diffusion. In Proceedings 24th International Conference on Parallel Programming, Vol. 3, CRC Press, Boca Raton, FL.

[31]

Hu, C. (1996) Parallel solutions for large-scale general sparse nonlinear systems of equations. J. Comput. Sci. Tech.,11, 257.

[32]

Hu, C., Frolov, A., Kearfott, R. B. & Yang, Q. (1995) A general iterative sparse linear solver and its parallelization for interval Newton methods. Reliable Computing,1, 251.

[33]

Kanal, L. N. & Kumar, V. (1988) Search in Artificial Intelligence. Springer-Verlag, New York, NY.

Digital Library

[34]

Kearfott, R. B. (1996). Rigorous Global Search: Continuous Problems, Kluwer Academic Publishers, Dordrecht, The Netherlands.

[35]

Klepeis, J. L. & Floudas, C. A. (2000) Deterministic global optimization and torsion angle dynamics for molecular structure prediction. Comput. Chem. Eng.,24, 1761.

[36]

Kumar. V., Grama, A., Gupta, A. & Karypis, G. (1994) Introduction to Parallel Computing: Design and Analysis of Parallel Algorithms. Benjamin-Cummings, Redwood City, CA.

Digital Library

[37]

Luling, R. & Monien, B. (1989) Two strategies for solving the vertex cover problem on a transputer network. Lect. Notes in Comp. Sci.,392, 160.

Digital Library

[38]

McDonald, C.M. & Floudas, C. A. (1997) GLOPEQ: A new computational tool for the phase and chemical equilibrium problem. Comput. Chem. Eng.21, 1.

[39]

McKeown, G. P., Rayward-Smith, V. J. & Rush, S. A. (1992) Parallel branch-and-bound. In Advances in Parallel Algorithms, eds. L. Kronsj & D. Shumsheruddin, Halsted Press, New York.

Digital Library

[40]

McKinnon, K. I. M., Millar, C. G. & Mongeau M. (1996) Global optimization for the chemical and phase equilibrium problem using interval analysis. In State of the Art in Global Optimization: Computational Methods and Applications, eds. C. A. Floudas & P. M. Pardalos, Kluwer Academic Publishers, Dordrecht, The Netherlands.

[41]

Mitra, G., Hai, I. & Hajian, M. T. (1997) A distributed processing algorithm for solving integer problems using a cluster of workstations. Parallel Computing,23, 733.

Digital Library

[42]

Neumaier, A. (1990). Interval Methods for Systems of Equations, Cambridge University Press, Cambridge, UK.

[43]

Pardalos, P. M. & Rodgers, G. P. (1990) Parallel branch-and-bound algorithms for quadratic zero-one programming on a hypercube architecture. Anns. Opns. Res.,22, 271.

Digital Library

[44]

Quinn, M. J.: Analysis and implementation of branch-and-bound algorithms on a hypercube multicomputer. IEEE Trans. Comp.,39, 384.

Digital Library

[45]

Rao, V. N. & Kumar, V. (1987) Parallel depth first search. 1. implementation. Int. J. Paral. Prog.,16, 479.

Digital Library

[46]

Rump, S. M. (1999) Fast and parallel interval arithmetic. BIT,39, 534.

Digital Library

[47]

Rushmeier, R. A. (1993) Experiments with parallel branch-and-bound algorithms for the set covering problems. Oper. Res. Lett.,13, 277.

Digital Library

[48]

Schnepper, C. A. & Stadtherr, M. A. (1993) Application of a parallel interval Newton/generalized bisection algorithm to equation-based chemical process flowsheeting. Interval Computing,1993(4), 40.

[49]

Schnepper, C. A. & Stadtherr, M. A. (1996) Robust process simulation using interval methods. Comput. Chem. Eng.,20, 187.

[50]

Sinha, M., Achenie, L. E. K. & Ostrovsky, G. M. (1999) Parallel branch and bound global optimizer for product design. Presented at AIChE Annual Meeting, Dallas, TX, November.

[51]

Smith, E. M. B. & Pantelides, C. C. (1999) A symbolic reformulation/spatial branch-and-bound algorithm for the global optimization of nonconvex MINLPs. Comput. Chem. Eng.,23, 457.

[52]

Stradi, B. A., Brennecke, J. F., Kohn, J. P. & Stadtherr, M. A. (2001) Reliable computation of mixture critical points. AIChE J,47, 212.

[53]

Subrahmanyam, S., Kudva, G. K., Bassett, M. H. & Pekny, J. F. (1996) Application of distributed computing to batch plant design and scheduling. AIChE J.,42, 1648.

[54]

Teng, S. (1990) Adaptive parallel algorithms for integral knapsack problems. J. Parallel Distr. Comp.,8, 400.

Digital Library

[55]

Troya, J. & Ortega, M. (1989) A study of parallel branch-and-bound algorithms with best-bound-first search. Parallel Computing,11, 121.

[56]

Vornberger, O. (1986) Implementing branch-and-bound in a ring of processors. Lect. Notes in Comp. Sci.,237, 157. 157-164

Digital Library

[57]

Vornberger, O. (1987) Load balancing in a network of transputers. Lect. Notes in Comp. Sci.,312, 116.

Digital Library

[58]

Xu, G., Scurto, A. M., Castier, M., Brennecke, J. F. & Stadtherr, M. A. (2000) Reliable computation of high pressure solid-fluid equilibrium. Ind. Eng. Chem. Res.,39, 1624.

Cited By

Khun JSchmidt J(2022)Interval Constraint Satisfaction: Towards Edge Acceleration2022 11th Mediterranean Conference on Embedded Computing (MECO)10.1109/MECO55406.2022.9797220(1-4)Online publication date: 7-Jun-2022
https://doi.org/10.1109/MECO55406.2022.9797220
Wu YKumar A(2012)A Parallel Interval Computation Model for Global Optimization with Automatic Load BalancingJournal of Computer Science and Technology10.1007/s11390-012-1260-x27:4(744-753)Online publication date: 12-Jul-2012
https://doi.org/10.1007/s11390-012-1260-x
Beck PNehmeier M(2012)Parallel interval newton method on CUDAProceedings of the 11th international conference on Applied Parallel and Scientific Computing10.1007/978-3-642-36803-5_34(454-464)Online publication date: 10-Jun-2012
https://dl.acm.org/doi/10.1007/978-3-642-36803-5_34
Show More Cited By

Index Terms

Parallel interval-Newton using message passing: dynamic load balancing strategies

Recommendations

A Parallel Interval Computation Model with Alternative Message Passing
IHMSC '10: Proceedings of the 2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 02

In this paper, we propose a decentralized parallel computation model for global optimization using interval analysis. The model is adaptive to any number of processors and there is no need to design an initial decomposition scheme to feed each processor ...
Dynamic and Hierarchical Load-Balancing Techniques Applied to Parallel Branch-and-Bound Methods
3PGCIC '13: Proceedings of the 2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing

Most Lipschitzian Global Optimization algorithms perform an exhaustive search using a branch-and-bound (B&B) scheme. The question is how to run multi-dimensional Lipschitz Global Optimization in parallel, such that the implementation depending on the ...
Scalable parallel numerical constraint solver using global load balancing
X10 2015: Proceedings of the ACM SIGPLAN Workshop on X10

We present a scalable parallel solver for numerical constraint satisfaction problems (NCSPs). Our parallelization scheme consists of homogeneous worker solvers, each of which runs on an available core and communicates with others via the global load ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '01: Proceedings of the 2001 ACM/IEEE conference on Supercomputing

November 2001

756 pages

ISBN:158113293X

DOI:10.1145/582034

Conference Chair:
Charles Slocomb
Los Alamos National Laboratory

Copyright © 2001 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGARCH: ACM Special Interest Group on Computer Architecture
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 November 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SC '01

Sponsor:

SIGARCH
IEEE-CS

SC '01: International Conference for High Performance Computing, Networking, Storage and Analysis

November 10 - 16, 2001

Colorado, Denver

Acceptance Rates

SC '01 Paper Acceptance Rate 60 of 240 submissions, 25%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
542
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Khun JSchmidt J(2022)Interval Constraint Satisfaction: Towards Edge Acceleration2022 11th Mediterranean Conference on Embedded Computing (MECO)10.1109/MECO55406.2022.9797220(1-4)Online publication date: 7-Jun-2022
https://doi.org/10.1109/MECO55406.2022.9797220
Wu YKumar A(2012)A Parallel Interval Computation Model for Global Optimization with Automatic Load BalancingJournal of Computer Science and Technology10.1007/s11390-012-1260-x27:4(744-753)Online publication date: 12-Jul-2012
https://doi.org/10.1007/s11390-012-1260-x
Beck PNehmeier M(2012)Parallel interval newton method on CUDAProceedings of the 11th international conference on Applied Parallel and Scientific Computing10.1007/978-3-642-36803-5_34(454-464)Online publication date: 10-Jun-2012
https://dl.acm.org/doi/10.1007/978-3-642-36803-5_34
Wu YKumar AShi P(2010)A Parallel Interval Computation Model with Alternative Message PassingProceedings of the 2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics - Volume 0210.1109/IHMSC.2010.129(120-123)Online publication date: 26-Aug-2010
https://dl.acm.org/doi/10.1109/IHMSC.2010.129
Leclerc A(2008)Interval Analysis: Parallel Methods for Global OptimizationEncyclopedia of Optimization10.1007/978-0-387-74759-0_299(1709-1717)Online publication date: 2008
https://doi.org/10.1007/978-0-387-74759-0_299

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten