Show understanding of the characteristics of massively parallel computers

What is Parallel Processing?

Imagine a factory where many workers (cores) can work on different parts of a product at the same time. In a computer, each core can execute instructions simultaneously, speeding up the overall task.

• Instruction‑level parallelism: one core splits a single instruction into sub‑tasks.
• Data‑level parallelism: the same instruction is applied to many data items at once.
• Task‑level parallelism: completely different tasks run on different cores.

Massively Parallel Computers (MPCs)

🧠 Definition: Systems with hundreds to thousands of cores working together on a single problem.

💻 Key Characteristics:

1. High Throughput: Can process many operations per second.
2. Scalability: Adding more cores usually improves performance.
3. Low Latency for Parallel Tasks: Tasks that can be split up finish faster.
4. Communication Overhead: Cores must share data; this can slow things down.
5. Fault Tolerance: If one core fails, others can often continue.
6. Memory Consistency Models: Rules that ensure all cores see a coherent view of memory.

🚀 Examples: GPUs for graphics, supercomputers like Summit, cloud clusters (AWS, Azure).

Amdahl’s Law – How Fast Can You Go?

When only part of a program can be parallelised, the speed‑up is limited. The formula is:

$$S = \frac{1}{(1 - P) + \frac{P}{N}}$$

• $P$ = proportion of the program that can be parallelised.
• $N$ = number of processors.
• $S$ = maximum speed‑up.

🔍 Tip: If $P=0.9$ and $N=10$, $S \approx 5.26$ – you won’t get a 10× speed‑up even with 10 cores.

Virtual Machines (VMs)

🛠️ A VM is like a computer inside a computer. The hypervisor (host) runs on real hardware and creates one or more guest OSes that think they have their own hardware.

• Isolation: Problems in one VM don’t affect others.
• Resource Sharing: CPU, memory, and storage are divided among VMs.
• Portability: Move a VM from one host to another easily.

💡 Analogy: Think of a VM as a virtual room in a building; the building (host) provides the walls, but each room can have its own furniture (guest OS).

Exam Tip Box

Question Types:

1. Define massively parallel computers and list at least three characteristics.
2. Explain Amdahl’s law and calculate the maximum speed‑up for $P=0.75$ and $N=8$.
3. Describe the role of a hypervisor in virtualisation.

Answer Strategy:

• Use bullet points for clarity.
• Show the formula and plug in numbers for calculations.
• Use analogies to demonstrate understanding.