CS4234 Lecture Notes: 10/6/06

Discussed derivations of Amdahl's law, Gustavson's scaled speedup law and Karp's law.
Discussed implications of these laws:
- Amdahl's law assumes fixed problem size and projects scaling from the sequential program.
- Gustavson's law implies (or allows) problem size to scale with the number of processors. Speedup is projected from the observable sequential overhead in parallel executions of the program.
- Both Amdahl and Gustavson "ignore" communication. The models are useful for back-on-the-envelope calculations (e.g. how big a machine should I buy to solve a specific problem with a specific input workload, or how much performance will I gain if I parallelize this part of the code, etc.). However, the laws are not appropriate for estimating/predicting actual speedup in parallel codes, due to lack of communication. Karp's law improves on this aspect.
- Karp's law allows you to "guess" the bottleneck in a parallel program. More specifically, it tells you if communication is a bottleneck or not, and, if yes, how quickly performance saturates because of communication.