When data is late — freshness rules instead of pretending it's current

Every number a system shows is a claim about a moment. A commission total is not an abstract truth; it is what the source said, as of when it said it. The interesting design question is what the software does when the “as of” gets old.

A bounded window

Our source feed is valid for a bounded window — currently 24 hours for the financial data. Inside the window, the delivery is current. Outside it, the same bytes are a historical document, and treating them as today’s is a quiet lie the UI tells with complete confidence.

Two design points follow:

The window is about observation time, not file time. You cannot extend a stale delivery’s life by re-compressing it or editing a generation timestamp. The question is when the data was observed, and rewriting metadata does not observe anything.

The recovery path is the next real delivery. When a valid package arrives, everything returns to normal — there is no need to fabricate the missing days, and no need for the platform to interpolate a plausible figure for a day it never saw. A gap in the historical record is honest. A synthesised day is not.

Saying “delayed” out loud

When nothing valid has arrived inside the window, the platform says data update delayed rather than continuing to present the last figures as current.

It would be easy to do the opposite. Stale data is better than empty data for almost every screen: yesterday’s commission total is, after all, a real number that a real system produced. But “a real number produced yesterday” and “your commission total” are different claims, and the difference is exactly what an agent needs to know before repeating the number to a client.

So the delay is a first-class state. It is visible, it has a timestamp attached, and it does not require anyone to notice the problem first.

Distinguish the moment from the state

The sharpest version of this problem is not about slowness; it is about confusing two kinds of data that arrive together.

Alongside the financial facts, our feed carries observations about eligibility: whether a result is frozen, whether a payment is deferred. These are snapshots — they describe what the source saw at the moment it ran. They are not live eligibility, and they are not payment records.

A snapshot that says “frozen” is a fact about the past. Re-labelling it as a present-tense claim is not a display choice; it is asserting something the source never said. So the platform shows such observations as “as of time” and refuses to restate them as current eligibility, as already paid, or as received.

The same discipline applies to authorisation. Our deliveries explicitly carry no payment authority. That flag is a statement about what the file permits — it is not a statement that an agent’s income is zero, and the two must never be conflated in either direction.

What this looks like in practice

• Stale is labelled, never disguised. The freshness state is visible on the screen, not only in an operations log.
• Old and current are not averaged. A delayed feed does not become a slightly worse feed; it becomes a feed that is not current.
• A missing value is not a zero. The feed preserves true zeros, negatives, and unknowns as distinct things, and the UI carries the distinction through. A zero is a claim about money; a null is an admission that we do not know.
• Failure is whole-batch. A delivery that does not verify is refused entirely rather than partially applied, because a partially-applied batch is a dataset that looks complete at the point of use.

The trade-off, stated plainly

This costs us convenience. Screens go visibly stale instead of smoothly wrong. Operators get an error code instead of an approximate import. Some of that work lands on people who would rather have a number.

We take that trade because it is the only one that scales. A platform that is quietly stale once is a platform nobody trusts twice, and rebuilding that trust is far more expensive than a labelled “delayed”.