Building Software That Markets Can Trust
Correctness, replay and failure behaviour in pricing systems
A pricing system can return a number quickly, consistently and incorrectly. It can also calculate the right value and deliver it too late, attach the wrong status, or leave nobody able to explain which inputs it used. Trust depends on the whole path from observation to decision to delivery.
That makes pricing software a useful test of engineering discipline. The calculation needs a defensible model and suitable inputs. The surrounding system needs explicit timing, controlled changes and a record of what happened. Reliability is weakened when either half is treated as somebody else's problem.
Separate the questions
There are at least four questions to answer about a result.
Was the calculation implemented correctly? Does it follow the intended algorithm, conventions and numerical tolerances?
Was the model appropriate? Are its assumptions and limitations acceptable for this instrument and use?
Were the inputs suitable? Were the market data, reference data and configuration valid for the valuation being requested?
Was the result used correctly? Did it reach the intended consumer in time, with its provenance and status intact?
Reproducibility helps investigate all four. It does not answer them by itself. A deterministic program with an incorrect unit conversion can produce the same wrong result indefinitely.
The distinction has a formal counterpart in model-risk practice. The US banking agencies' revised 2026 guidance describes validation in terms of model reliability, limitations, conceptual soundness and outcomes analysis.[1] That is useful context for pricing engineering, though its supervisory scope should not be confused with a universal requirement for every financial system.
A disputed price
Consider an illustrative fixed-income valuation, simplified to isolate the investigation.
An instrument pays a single 1,000-unit cash flow at a specified future date. The pricing calculation multiplies that cash flow by the applicable discount factor. A recorded factor of 0.970 produces 970. A later run uses 0.968 and produces 968.
Replaying the original inputs confirms that the old build returned 970. That establishes what the software did. It does not establish that 0.970 was the right factor to use.
The next check is independent of the replay: inspect the intended curve selection, its valuation time, the market-data inputs used to construct it and the instrument's reference data. Suppose the request required a newly approved curve snapshot with factor 0.968, but a cache supplied the preceding snapshot. Both multiplications are correct. The selection of data made the original result unsuitable for that request.
For this deliberately simple case, a separate calculation verifies the arithmetic. A real instrument may require independently derived reference values, model review, sensitivity checks and analysis of conventions. A second implementation that shares the same mistaken assumptions is a weaker check than it first appears.
The investigation should end with a narrower claim than “replay proves the price.” It shows which path produced the value, while validation establishes whether that path was acceptable.
Define the replay contract
To reconstruct a calculation, preserve the inputs and state that actually influenced it. That usually includes an addressable market-data snapshot, instrument and reference-data versions, model and build identifiers, configuration, valuation time, and any relevant prior state.
For an event-driven system, arrival order can matter independently of the timestamp inside an event. Record sequence and processing decisions where they affect state. Make time a supplied dependency where practical. Randomised calculations may also require the generator version, seed or stream state and numerical settings.
Be explicit about the guarantee. Exact reconstruction in the original execution environment is different from numerical agreement within a documented tolerance after a platform change. Intel's numerical-reproducibility documentation explains how floating-point operation order and execution choices can change results.[2] Pinning a model version alone does not settle that question.
A replay runner should control side effects. It must not send a second live quote or repeat a transaction merely because an engineer is investigating the first one. Calculation and external delivery can be separated, or effectful boundaries can be replaced with controlled implementations. Code that writes during calculation is harder to replay safely, but it is not inherently unreplayable.
The record also needs a retention and retrieval design. Preserve enough to support the intended investigations, protect sensitive data, and test whether an older build and its inputs can actually be recovered. “We log everything” is not a substitute for a working reconstruction procedure.
TickTape: one ordered history, the same trading logic
TickTape provides a directly relevant example: a Rust sequencer framework with a price-time-priority order book implemented as a deterministic service. Its service contract requires the same ordered inputs and configuration to produce the same state and outputs. Time is supplied through the execution context as a sequenced timestamp, rather than read from the operating-system clock.[3]
The recovery path applies recorded frames through the service logic and discards replay-generated outputs. That separation matters in trading: reconstructing an order book must not itself resend the historical effects. It is not, by itself, a guarantee of exactly-once delivery to an external venue; the adapter, acknowledgement and reconciliation contracts still matter.[4]
The order-book example uses ordered price levels and FIFO queues. Its invariant checks reject a crossed book and account for accepted quantities as traded, cancelled or still resting, counting a trade on both sides of the ledger. These are checks on trading behaviour, not just on the ability to reproduce bytes.[5]
The boundary remains important. The service contract prohibits ambient time, randomness and I/O; it does not make arbitrary Rust code incapable of using them. Replay equality can expose a violation, but an identically wrong matching rule can pass an equality check. TickTape makes the mechanism concrete without establishing production suitability or model validity. Historical replay still belongs alongside independently checked expected values, constructed edge cases and stress scenarios absent from the recorded period.
Choose failure behaviour before the incident
Missing, delayed and contradictory inputs should have defined meanings. An hour-old observation may be acceptable for one purpose and unusable for another. Freshness depends on the instrument, market conditions and intended action; age alone is not a universal verdict on correctness.
For each important dependency, decide what happens when it is absent, stale or invalid. The response might be to withhold an executable quote, publish a clearly marked indicative value, use an approved fallback, or stop the affected calculation. A degraded mode is useful only when its result remains appropriate for the consumer.
That agreement must extend downstream. A status flag offers little protection if another service discards it or treats an indicative value as executable. Test the consuming workflow, including cached results and delayed messages.
An operator also needs a tested way to stop the relevant activity. Define the scope of the control, who can use it, what happens to work already in flight, and what evidence is required to resume. A switch that stops new calculations but leaves queued output flowing may not provide the intended protection.
The most dangerous architecture is often the one whose failure behaviour was never consciously chosen. It has failure behaviour regardless. It just belongs to whoever wrote the retry loop.
Measure the difficult periods
Average latency can describe ordinary demand or support capacity planning. It is insufficient for understanding deadline misses. A short period of severe delay can disappear inside a session-wide mean.
Google's SRE guidance recommends treating many service indicators as distributions and selecting objectives appropriate to the workload.[6] For pricing, the practical extension is to distinguish interactive quotes from batch valuation, identify the relevant deadline, and inspect both the proportion and the business significance of missed deadlines.
Observe timeouts and rejected work as well as successful responses. If the measurement drops requests that never complete, the latency chart can improve while the service deteriorates. Examine queue age, input freshness and resource use over the same interval so that a fast calculation waiting behind a long queue does not look healthy.
Slow responses can cluster during bursts, dependency failures or resource contention; they do not invariably do so. Test those conditions deliberately. A percentile from a quiet day cannot establish behaviour during a market-data surge.
Make durability and delivery explicit
An audit record is useful only if it survives the failure being investigated. Decide which events must be durable before their effects become externally visible, and how consumers identify duplicates or gaps during recovery.
LMAX's own Disruptor guide gives an example in which journalling, replication and business processing receive the same ordered events, with dependencies used to gate processing.[7] The transferable lesson is the ordering requirement. Adopting the same library would not automatically give another system the same recovery properties.
A durable record also does not prove that a downstream consumer received or acted on an output. Preserve the acknowledgements or reconciliation evidence required by the delivery contract. Calculation history and delivery history answer different questions.
Confidence with a defined scope
A trustworthy pricing system makes it possible to establish what happened, assess whether it was acceptable, and identify what remains uncertain. Replay supports investigation. Validation challenges the result. Failure controls limit consequences. Service objectives make timeliness and availability explicit.
These capabilities reinforce one another, but none replaces the others. A system that can explain its failures still needs to meet its operating obligations. The engineering standard is a result that can be used for its intended purpose, with enough evidence to defend that use and enough control to stop when the conditions no longer hold.
Sources
Federal Reserve, OCC and FDIC, “Revised Guidance on Model Risk Management”, 17 April 2026, section V. The SR 26-2 cover letter supersedes SR 11-7 and describes its applicability. Cited for the distinction between reconstruction and validation, not as jurisdiction-specific compliance advice. ↩︎
Intel, “Obtaining Numerically Reproducible Results”, oneMKL Developer Guide, 31 March 2023. This dated edition supports the numerical principle, not a claim about the current API. ↩︎
TickTape, service contract and execution context, source reviewed at commit
0a964bbon 11 September 2026.Service::Configis part of the deterministic inputs; the restrictions on service behaviour are a contract, not a general-purpose sandbox. ↩︎TickTape, runtime implementation,
replay_stepandverify_replay. Recovery drains generated outputs; replay verification compares encoded state snapshots. Neither operation validates an external delivery contract. ↩︎TickTape, order-book implementation, matching semantics and
Invariants::check. These are inspected implementation properties, not a claim of production exchange deployment or an independent audit. ↩︎Chris Jones, John Wilkes and Niall Murphy, “Service Level Objectives”, Site Reliability Engineering, 2016. ↩︎
LMAX Exchange, “LMAX Disruptor User Guide”, “Multicast Events” and “Consumer Dependency Graph,” accessed 10 September 2026. ↩︎