Chapter 1 (continued)

II. A first pass through everything

3. Programming languages and paradigms: some answers to the question "How do I program a parallel computer?"

   1. Just like you program any computer. The compiler will do everything for you, given your existing, sequential code.

      Comment: This is very hard. There have been pretty good successes in the world of vector computing. But for general, MIMD parallel computing ... too hard.

   2. Use a new programming language, designed with parallel computing in mind. The compiler for this new language will do everything for you.

      Comment: There have been lots of research projects over the last 30 years along these lines. But not a big impact in the "real" world. There are big psychological and financial impediments to forcing people to program in new languages.

   3. Use an existing language that has been extended to support parallel concepts like multiple threads or processes, data distribution, message passing.

      Comments. Again, there have been lots of research projects, a few of which have yielded systems that are used. HPF is an example of this approach in that it is an extension of F90.

   4. Use an existing language plus compiler directives. The compiler directives are handled by a preprocessor or by the compiler itself, but since they are only directives, they can be ignored by a standard sequential compiler.

      Comments. OpenMP is a widely used example of this approach.

   5. Use an existing language plus function calls to a library which implements parallel concepts like multi-threading or message passing or remote method invocation (RMI).

      Comments. This is a very common approach. Famous low-level examples of this approach are pthreads for shared memory threads and sockets for distributed memory message passing. And systems such as PVM and MPI provide higher-level interfaces for messages passing.