Concurrency is not parallelism

1. Slide 1

   Two different questions

   Concurrency is about structure: is this program made of parts that could proceed independently? Parallelism is about execution: are two things happening at the same instant?

   A program can be concurrent on a single processor. An operating system from 1970 ran dozens of concurrent processes on one core, interleaving them, and none of them ran simultaneously with any other. A program can be parallel without being interestingly concurrent — adding two large arrays element by element across sixteen cores has no independent parts to coordinate, only one loop split up.

   The reason the distinction matters practically is that it tells you what a technique is for. Concurrency is a way of writing a program whose parts wait for different things. Parallelism is a way of getting a result sooner by using more hardware. Confusing them means reaching for a scheduler when you needed a core, or a core when what you needed was a way to stop blocking.

2. Slide 2

   Waiting is not working

   Most programs that feel slow are not computing anything. They are waiting, and this is the fact that makes concurrency worth having at all.

   data = fetch(url)   # 200ms, waiting
   rows = parse(data)  # 2ms, working
   save(rows)          # 10ms, waiting

   Of that request, 210 milliseconds are spent waiting for a network and a disk, and two are spent using the processor. A thread sitting in fetch is holding memory and a scheduling slot to accomplish nothing whatsoever. If a thousand requests arrive, a thousand threads sit there, and the machine spends its time switching between them rather than serving anyone.

   This is the case asynchronous programming exists for, and it is why the answer helps so dramatically. It is not making anything faster. It is allowing one thread to have ten thousand outstanding waits, because a wait costs a data structure rather than a stack.

3. Slide 3

   The event loop, and its one rule

   The commonest concurrency machinery in the world runs on exactly one thread and interleaves work at points you can see in the source.

   const data = await fetch(url);
   const rows = parse(data);

   Each await is a place where this function stops and something else runs. Between two awaits nothing can interrupt you, which is an enormous simplification: there are no data races within a stretch of synchronous code because nothing else is executing.

   The rule that follows is the one everybody learns painfully. Anything that computes without awaiting blocks everything. A single expensive loop in one request handler stops every other request on that thread, and the symptom is a server that is fast under light load and inexplicably terrible under heavy load. The event loop gives you cheap concurrency and no parallelism at all, and it hands you the whole bill in one place.

4. Slide 4

   Threads, and what they actually buy

   A thread gives you real parallelism and charges for it in a currency most programs are bad at handling.

   Two threads genuinely run at the same instant on two cores, so processor-bound work — resizing images, compressing files, computing anything — gets faster in a way that no amount of asynchrony can achieve. That is the purchase. The price is that they share memory, and any value one thread writes may be read by another at a moment neither of them chose.

   Note also that some runtimes sell you threads without the parallelism. A global interpreter lock, as in CPython's default implementation, means threads interleave but never execute simultaneously — so they help with waiting and do nothing for computation. If you have ever added threads to a Python program and measured no improvement, that is why, and it is a documented design decision rather than a bug.

5. Slide 5

   The actual hard part

   Almost everything difficult about concurrent programming reduces to one sentence: two things reading and writing the same memory without an agreed order.

   counter = counter + 1;

   That is three operations — read, add, write — and two threads running it a thousand times each will not reliably produce two thousand. Both may read the same value and both write back the same increment. The failure is silent, it is timing-dependent, it will not reproduce under a debugger, and it will happen once a fortnight in production.

   Locks solve it by making a stretch of code exclusive, and introduce their own family of problems: forget one and the bug returns, take two in different orders and the program deadlocks, hold one too long and you have serialised the thing you parallelised. This is why the field has spent forty years looking for arrangements where the question does not arise.

6. Slide 6

   Not sharing anything

   The most successful of those arrangements is to give each concurrent part its own state and let them communicate only by sending messages.

   Pid ! {add, 1},
   receive
       {ok, N} -> N
   end.

   Nothing is shared, so nothing needs locking. Erlang has worked this way since 1986 and runs telephone exchanges with it; Go's channels are the same idea with different syntax; the actor model is its academic name. A JavaScript web worker communicates only by postMessage for exactly this reason, and the restriction that looks like a limitation is the feature.

   The costs are real and worth stating. Messages are copied rather than shared, which matters for large data. Debugging a system of independent processes means reconstructing an ordering that no single place recorded. And you have not abolished the hard problem so much as moved it: two messages arriving in an unexpected order is a race condition wearing a different hat.

7. Slide 7

   Immutability as a third answer

   The other way to make the hard problem disappear is to remove one half of it.

   A race needs two things: sharing and mutation. Message passing removes the sharing. Immutability removes the mutation, and a value that cannot change can be read by any number of threads with no coordination at all, forever.

   This is the point where this lecture meets the one on immutability a few doors down. The claim that immutable data is good for concurrency is usually made without explaining why, and the why is exactly this: the entire category of bug requires a write, and there are none. It is also why the Redux store in the Hall of the Commons can be reasoned about so simply — a new state object per action means no reader ever sees a half-finished one.

8. Slide 8

   Choosing, and the honest summary

   The decision is not a matter of taste, and it follows from what your program is actually doing.

   If it is waiting — on networks, disks, users, other services — you want concurrency, and an event loop will give you enormous amounts of it cheaply. If it is computing, you want parallelism, and only threads or processes will provide it; asynchrony will do nothing but add ceremony. Most real systems are both, in different layers, and the useful skill is noticing which layer you are in.

   What is genuinely unsettled is which coordination model is best where. Shared memory with locks is the fastest and the most dangerous. Message passing is the safest and pays in copies and complexity. Immutability is the most elegant and pays in allocation. Erlang, Go, Rust and Java each bet differently, all four bets are still running, and anyone who tells you the question is closed is selling one of them.

← All lecturesAtrium