
C++ High Performance
By :

As mentioned in Chapter 11, Concurrency, the terms concurrency and parallelism can be a little hard to distinguish from each other. As a reminder, a program is said to run concurrently if it has multiple individual control flows running during overlapping time periods. On the other hand, a parallel program executes multiple tasks or subtasks simultaneously (at the exact same time), which requires hardware with multiple cores. We use parallel algorithms to optimize latency or throughput. It makes no sense to parallelize algorithms if we don't have hardware that can execute multiple tasks simultaneously to achieve better performance. A few simple formulas will now follow to help you understand what factors need to be considered when evaluating parallel algorithms.
In this chapter, speedup is defined as the ratio between a sequential and a parallel version of an algorithm, as follows:
T1 is the time it takes to...