← Lectures
Back

1 / 7

Next

Slide 1 of 7: Software is not soft

Arrow keys move through the slides while these controls have focus.

The theatre is not shown, because your system asks for reduced motion. The lecture itself is below, in full.

Lecture · 7 slides

Why programs are hard to change

Software was named for the property that it can be changed. Hardware is fixed at manufacture; software is text, and text can be edited. That was the promise, and for the first version of anything it is kept.

Then something happens that everyone in the field has experienced and few can explain precisely. The same team, on the same system, takes three weeks to do in year four what took an afternoon in year one. Nobody was careless. The requests did not get harder. Every individual decision along the way was defensible, and the result is a system where a two-line change requires a fortnight of reading first.

The usual explanations are moral: there was not enough discipline, the tests were skipped, somebody took a shortcut. Sometimes true, and not the mechanism. The mechanism is structural and it operates even on careful teams, which is why it is worth understanding rather than merely deploring. This lecture tries to name it, and then to say honestly which of the standard remedies actually address it.

AssumesYou have worked on a codebase older than your involvement with it.

  1. Slide 1

    Software is not soft

    The metaphor breaks at the same place every time: changing the text is easy, and changing the text is not the job.

    The job is changing the behaviour without changing any of the behaviour that other things depend on. That second clause is where the cost lives, and it grows with the number of things that depend on you — which is a property of the system, not of the edit. Editing a function takes a minute. Establishing that no caller relied on the thing you just altered can take days, and no amount of typing speed helps.

    This is why estimates based on the size of the change are so consistently wrong. The size of the change is a poor predictor of the size of the investigation, and the investigation is nearly all of it.

  2. Slide 2

    Coupling is a count of things that must move together

    The one concept that explains most of the phenomenon is coupling, and it is worth stating as something countable rather than as a vice.

    def price(order):
    total = subtotal(order)
    total = tax(total, order.region)
    return discount(total, order.code)
    python

    Two things are coupled when a change to one forces a change to the other. That is all. It is not a synonym for "calls" — a function you call and never have to think about is not coupling in any painful sense. It becomes painful when the dependency is on something that changes: a data shape, an ordering, an assumption about who runs first.

    The reason a system slows down is that the number of pairs grows faster than the number of parts. Ten modules that all know about each other have forty-five relationships; twenty have a hundred and ninety. Nobody added the complexity; it arrived as a consequence of arithmetic, which is why "just be disciplined" is not a sufficient answer.

  3. Slide 3

    The knowledge is what couples, not the code

    The subtle form, and the one that survives every architectural fashion, is where two pieces of code share a belief rather than a call.

    Two modules that both know an identifier is twelve characters are coupled, even with no import between them. A test that constructs a fixture matching the database schema is coupled to that schema. A frontend that assumes a list comes back sorted is coupled to a decision made in a query, and the coupling is invisible until somebody changes the query for good reasons.

    This is why moving code into separate services does not by itself reduce coupling. It converts a compile error into a runtime error and puts a network between the two halves. If both sides still know the same fact, both must still change together, and now the change must be deployed in a particular order. The shared belief is the coupling; the wire is just how far apart the believers are standing.

  4. Slide 4

    Entropy has a mechanism, and it is local reasoning

    Programs decay for a reason that can be stated exactly, and it is not metaphorical.

    if (user.legacyFlag && !ctx.v2) {
  return oldPath(user);
}
    javascript

    Every change is made by someone who understands part of the system, under time pressure, choosing the option that is locally safest. Adding a flag is safer than altering the existing path, because it cannot break what already works. Copying a function and editing the copy is safer than generalising the original, for the same reason. Each of those decisions is correct in isolation, and their sum is a system with four code paths where one belonged.

    Manny Lehman formalised this in the nineteen-seventies as laws of software evolution: a system in use must keep changing or become less useful, and as it changes its complexity increases unless work is done specifically to reduce it. That last clause is the operative one. Complexity is the default outcome, not the result of failure, and holding it steady requires effort that produces no visible feature.

  5. Slide 5

    The debt metaphor, used properly

    Ward Cunningham's phrase has been taken to mean "code I do not like", which is a shame, because the original is precise and useful.

    The metaphor was about deliberately shipping something you know does not reflect your current understanding of the problem, in order to ship it now, and accepting that you will pay interest on the difference until you go back and align it. The debt is the gap between the code's model and your model. The interest is the extra work every subsequent change costs because of that gap.

    Two things follow from taking it literally. First, debt taken knowingly, to meet a real deadline, is a legitimate financial instrument and not a moral failure. Second, a mess made by people who never had a clear model is not debt at all, because you cannot repay what you never borrowed. Distinguishing the two matters: the first is fixed by returning to a decision you can find, and the second by learning the domain, which is a much longer job.

  6. Slide 6

    Rewrites usually lose

    The most attractive response to a system that resists change is to start again, and it is worth understanding why that so rarely goes as planned.

    The old system contains a large quantity of undocumented knowledge, and most of it is in the parts that look worst. Joel Spolsky's observation is that ugly code is often ugly because it is correct: each strange branch is a bug report from a real user, and a clean reimplementation begins by deleting all of them and then rediscovers them one production incident at a time.

    There is also an arithmetic problem. The rewrite must reach parity with a system that is not standing still, and during the years it takes, the old one keeps acquiring the features the business needs. The teams that manage this successfully almost always do it incrementally — putting a new implementation behind the existing interface, moving one capability at a time, and keeping the old path until the new one is proven. That is slower to start and it finishes.

  7. Slide 7

    What actually helps

    The remedies that work are unglamorous and they share a single property: they reduce how much you must know before you can safely change something.

    Tests help, and specifically the ones that describe behaviour rather than structure, because they turn "I believe nothing depended on this" into a question you can answer in ninety seconds. A test suite's real product is not correctness; it is the confidence to delete things, and a codebase where nobody dares delete anything only grows.

    Names help more than most people expect, because most of the reading is figuring out what something is for. Boundaries help when they are drawn along the lines where knowledge actually clusters, and hurt when they are drawn somewhere tidy instead. Deleting helps most of all, and is the least practised: every feature retained for a hypothetical user is paid for in every subsequent change, and the payment is due whether or not the user ever arrives.

    And the honest conclusion is that none of this stops the process, it slows it. Every long-lived system is in a race between the knowledge being added to it and the knowledge being lost from the people who wrote it. Most of what looks like technical decay is that second quantity, and it is why the most valuable thing a team can do for the version of itself that exists in three years is to write down why, not what.

Works in the collection

The arguments above are hanging on the walls of the museum, in one form or another. These are the ones worth looking at next.

The rest of the programme

  1. Compiled, interpreted, and the space betweenA distinction that stopped describing anything decades ago, why it persists, and what is actually different about the machinery underneath.
  2. What a compiler actually doesFour jobs in a row, each mechanical, none magic: text to tokens, tokens to a tree, a tree to a judgement, and a judgement to instructions.
  3. Functional programming and object orientationTwo ways of arranging a program, what each one genuinely makes easy, and the trade-off underneath the argument that neither side can escape.
  4. The principles of object orientationEncapsulation, inheritance, polymorphism and SOLID, one at a time: what each actually claims, which held up, and which its own community abandoned.
  5. Types: what they can and cannot proveA type checker proves one thing about every possible run of your program. Knowing which proposition explains both the enthusiasm and the disappointment.
  6. Null, and the mistake its inventor apologised forTony Hoare called it his billion-dollar mistake. What was actually wrong with it, what the alternatives cost, and why the fix took forty years to arrive.
  7. What abstraction costsAbstraction is sold as free and is not. What you buy, what you pay, and how to tell before writing it which of the two is larger.
  8. Immutability, and what it is not free ofValues that never change buy sharing, comparison and time travel. They are not free, and it is worth knowing where they are expensive before you commit.
  9. Recursion, and why it feels like cheatingA function that calls itself looks like an unpaid debt. What makes it terminate, what it costs on the stack, and why some problems resist any other shape.
  10. Big-O, and what it deliberately ignoresComplexity notation throws away constants, hardware and every input you will actually see. Knowing what it discards is what makes the number useful.
  11. What regular expressions cannot matchThere is a precise boundary around what a regex can recognise. It explains the famous refusal to parse HTML, and why some patterns run forever.
  12. Concurrency is not parallelismOne is a way of structuring a program, the other a way of executing it. Keeping them apart explains why async helps a web server and threads often do not.
  13. Errors: exceptions, values, and what each hidesThrowing makes the happy path readable and the failure paths invisible. Returning errors as values does the opposite. Neither side has won.

← All lecturesAtrium