A functional language architecture that supports fine-grain parallelism (extended abstract)

This abstract introduces a research project at Iowa State University whose goal is to produce a high-speed computing system by combining a multiprocessor configuration with a software paradigm based on the functional programming language SASL. The underlying architecture will evaluate combinatory code produced by the SASL compiler by graph reduction, in a manner similar to that of other proposed SKIM combinator machines. Fine-grain parallelism is supported by concurrently evaluating the body and parameters of functions, by “unrolling” recursive algorithms when possible, and by application of high-level user-defined data shaping operations, such as “map.” A simple dynamic load balancing scheme is used to evenly distribute the work of reducing the graph over all processors.

The current design, under construction, is a tightly-coupled 16 processor system. Each processor has a private program memory that contains a complete copy of the program graph. The rules that specify and control the parallel reduction of the graph guarantee that the correct value will eventually be returned to the processor assigned to the root of the graph, and that inconsistencies in copies at various processors due to local reductions are of no concern. The data memory is global, but is physically organized so that a segment of the memory is located next to each processor. We expect that once the data is distributed through the graph during reduction, there will be some degree of locality of reference that will ease bus traffic. The typical bus bandwidth limitations that degrade performance in a fine-grain architecture are mitigated somewhat by employing a locally-developed high-speed (up to 100Mb/s per channel) “party line” serial bus for data references and interprocessor messages. The bus design allows for parallel data transfers, and can exchange processor load information in broadcast mode in one bus cycle.

We have chosen to base the design of the architecture on the functional language SASL: the desire to exploit fine-grain parallelism necessitated that we abandon sequential programming languages and adopt one that shows promise of supporting parallelism through referential transparency. Because SASL does not allow side-effects (including assignment) when evaluating an expression, sub-expressions do not interfere with one another and we are free to compute their values in parallel. The degree and sources of parallelism are under investigation by us and others. A second, equally important, issue centers around the ease of which a functional language can be used to program “every day” non-recursive problems. We are addressing this problem by investigating the translation of standard FORTRAN programs into SASL prior to execution.

The object code produced by the SASL compiler is a directed graph containing combinatory code. The graph is then traversed in normal order (preorder) until the first redex is found. The node containing the redex is evaluated and the process is repeated until the graph cannot be reduced further. Combinators are instructions for a hypothetical machine sometimes called a SKIM reduction machine. We are currently using the standard combinators found in the literature, but we plan to augment that set with special purpose combinators that better support parallelism.