Programmers typically want to write scientific programs in a high level language with semantics based on a sequential execution model. To execute efficiently on a parallel machine, however, a program typically needs to contain explicit parallelism and possibly explicit communication and synchronization. So, we need compilers to convert programs from the first of these forms to the second. There are two basic choices to be made when parallelizing a program. First, the computations of the program need to be distributed amongst the set of available processors. Second, the computations on each processor need to be ordered. My contribution has been the development of simple mathematical abstractions for representing these choices and the development of new algorithms for making these choices. I have developed a new framework that achieves good performance by minimizing communication between processors, minimizing the time processors spend waiting for messages from other processors, and ordering data accesses so as to exploit the memory hierarchy. This framework can be used by optimizing compilers, as well as by interactive transformation tools. The state of the art for vectorizing compilers is already quite good, but much work remains to bring parallelizing compilers up to the same standard. The main contribution of my work can be summarized as improving this situation by replacing existing ad hoc parallelization techniques with a sound underlying foundation on which future work can be built.
Cited By
- Kronawitter S and Lengauer C (2018). Polyhedral Search Space Exploration in the ExaStencils Code Generator, ACM Transactions on Architecture and Code Optimization, 15:4, (1-25), Online publication date: 31-Dec-2019.
- Venkat A, Hall M and Strout M (2015). Loop and data transformations for sparse matrix code, ACM SIGPLAN Notices, 50:6, (521-532), Online publication date: 7-Aug-2015.
- Venkat A, Hall M and Strout M Loop and data transformations for sparse matrix code Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, (521-532)
- Venkat A, Shantharam M, Hall M and Strout M Non-affine Extensions to Polyhedral Code Generation Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization, (185-194)
- Venkat A, Shantharam M, Hall M and Strout M Non-affine Extensions to Polyhedral Code Generation Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization, (185-194)
- Cong J, Zhang P and Zou Y Optimizing memory hierarchy allocation with loop transformations for high-level synthesis Proceedings of the 49th Annual Design Automation Conference, (1233-1238)
- Pouchet L, Bondhugula U, Bastoul C, Cohen A, Ramanujam J, Sadayappan P and Vasilache N (2011). Loop transformations, ACM SIGPLAN Notices, 46:1, (549-562), Online publication date: 26-Jan-2011.
- Pouchet L, Bondhugula U, Bastoul C, Cohen A, Ramanujam J, Sadayappan P and Vasilache N Loop transformations Proceedings of the 38th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, (549-562)
- Pouchet L, Bastoul C, Cohen A and Cavazos J Iterative optimization in the polyhedral model Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation, (90-100)
- Pouchet L, Bastoul C, Cohen A and Cavazos J (2008). Iterative optimization in the polyhedral model, ACM SIGPLAN Notices, 43:6, (90-100), Online publication date: 30-May-2008.
Recommendations
Systematic search within an optimisation space based on Unified Transformation Framework
Modern compilers have limited ability to exploit the performance improvement potential of complex transformation compositions due to their ad-hoc nature. Unified Transformation Framework (Kelly and Pugh, 1993a) provides a unified representation of ...
A Unified Compiler Framework for Control and Data Speculation
PACT '00: Proceedings of the 2000 International Conference on Parallel Architectures and Compilation TechniquesControl speculation refers to the execution of instructions before it has been determined that they would be executed in the normal flow of execution. Data speculation refers to the execution of instructions with potentially incorrect operand values, ...
A SIMD optimization framework for retargetable compilers
Retargetable C compilers are currently widely used to quickly obtain compiler support for new embedded processors and to perform early processor architecture exploration. A partially inherent problem of the retargetable compilation approach, though, is ...