Characterization tests: how to change code nobody understands

The standard objection to modernizing a legacy system is reasonable: we cannot safely change this because we do not understand what it does. The standard response — read it until you do — does not scale past a few thousand lines, and it does not work at all when the behaviour depends on fourteen years of accumulated data.

Characterization tests sidestep the problem. Instead of encoding what the system should do, they record what it currently does. You do not need to understand a behaviour to protect it.

The distinction that matters

A unit test asserts intent: given this input, the answer should be that, because the specification says so. A characterization test asserts observed reality: given this input, the system currently returns that, and I want to know immediately if that changes.

The difference is that a characterization test is never wrong at the moment you write it. It cannot be — you generated the expectation from the system itself. What it gives you is a tripwire.

# Not "this is correct" — "this is what it does today".
@pytest.mark.parametrize("quote_id", HISTORICAL_QUOTE_IDS)
def test_pricing_unchanged(quote_id, snapshot):
    result = legacy.calculate_price(load_quote(quote_id))
    snapshot.assert_match(result, f"pricing/{quote_id}.json")

How to build the corpus

Production data is the only input worth using. Synthetic cases exercise the paths you thought of, which are by definition the paths you already understand.

• Pull a representative sample of real historical inputs — a few hundred is usually plenty, spread across the full range of what the system has actually seen
• Run them through the current system and record every output as a snapshot
• Include the edge cases the operators complain about; those are where the undocumented behaviour lives
• Commit the snapshots. They are the specification you never had

What you find

This is where the technique earns its keep. On a recent engagement the snapshots surfaced eleven cases where the legacy pricing logic did something indefensible — rounding in the wrong direction, applying a discount twice under a specific combination of conditions.

The instinct is to fix them. Resist it, at least at first. Downstream reports had been reconciling against those outputs for a decade. Correcting the rounding would have been correct and would have broken the month-end reconciliation in a way nobody would have connected to the pricing change.

We asked the client which of the eleven were bugs. Nobody had ever asked them that question. Two were bugs. Nine were deliberate decisions whose rationale had been lost.

That conversation is the real output. Preserve the behaviour, document it, and let the business decide — separately, and with the consequences visible — whether to change it.

Then you can move

With the net in place, extraction becomes mechanical. Move one piece of functionality behind an interface. Run the suite. If the snapshots still match, the change was behaviour-preserving. If they do not, you know precisely which inputs diverged before anything reaches production.

This is what makes incremental modernization possible where a rewrite is not. Each increment is independently verifiable and independently reversible. You can stop after three of eight and still be meaningfully better off — which matters, because budgets and priorities move, and a strategy that only pays off at completion usually does not get there.