Data quality rules: protect the billing decision, not the dashboard score

A customer record can contain every required field and still produce the wrong bill. “Active” may be a permitted status, yet be wrong for a customer whose cancellation took effect yesterday. For the data platform lead, a completeness dashboard offers little protection unless the rules reflect the billing decision the data will drive.

Begin with one process and the person accountable for its result. Ask which errors would change who gets charged, the amount or the timing. That conversation sets priorities more effectively than applying the same catalogue of quality tests to every column.

The UK Government Data Quality Framework, written for the public sector, treats quality as fitness for purpose and distinguishes completeness from accuracy. Its approach is useful beyond government: a populated field is evidence that something was recorded, not that the recorded value matches reality. The operational rules still have to come from the process owner.

Turn a billing failure into a precise condition

Take a hypothetical subscription billing run. The business has agreed that an invoice line requires a valid customer, an applicable contract and the correct status for the service period. These are assumptions for the example, not universal billing rules.

A check that the status field is non-empty catches missing values. A check against an approved status list catches values the billing system cannot interpret. Neither establishes whether the chosen status is correct for that customer and date. That needs a relationship to the authoritative contract and cancellation information.

Define the effective time explicitly. “Active today” can produce a different result from “active during the invoiced period”. Decide how late-arriving cancellation events, backdated corrections and time zones are handled before coding the check. Otherwise two technically correct queries can reach conflicting conclusions.

For this process, a small set of rules might be:

• Customer reference: every proposed invoice line must resolve to a customer in the approved master. A failure holds the affected line for the customer-data team to investigate.
• Status validity: the status must belong to the billing owner’s agreed set. Unknown values are routed to the source-system owner rather than converted silently to a default.
• Contract relationship: the customer, product and service period must match an applicable contract. Ambiguous matches go to billing operations before release.
• Duplicate charge: the business-defined invoice-line key must not repeat in the release set. The owner investigates whether the repetition is an error or a legitimate separate charge.
• Freshness: required upstream feeds must have reached the agreed processing cut-off. A missing feed triggers a run-level decision because individual records may look valid while the population is incomplete.

Each rule has a consequence and a response. “Improve validity to 99%” has neither until someone defines the population, the invalid condition and what happens to the remaining records.

Choose the denominator before the tolerance

Suppose the fictional run contains 10,000 proposed invoice lines and 20 fail the contract relationship check. The pass rate is 9,980 divided by 10,000, or 99.8%. That number describes this check on this population. It does not say that billing is 99.8% accurate or that the financial exposure is small.

If one failed line is for a large annual contract, counting lines can conceal the commercial consequence. Report the affected amount alongside the count where available. Keep uncertainty explicit when the correct amount cannot yet be calculated.

For the relationship check, the billing owner might require all released lines to pass, while allowing the 20 unresolved lines to be held and investigated. That avoids choosing between billing everyone incorrectly and stopping all 10,000 lines unnecessarily. It is appropriate only if partial release is supported and the owner accepts its operational consequences.

A stale source feed needs a different response. Quarantining the few records that look wrong may not reveal customers missing from the batch entirely. This is why row-level rules and whole-run controls serve different purposes.

Make the implementation mean what the rule says

Tool semantics can change the reported result. AWS Glue’s Data Quality Definition Language distinguishes nulls, empty strings and whitespace-only strings. Its documentation also explains that, for supported row-level rules with a where filter, excluded rows are labelled as passed by default; an option can label them skipped instead.

That behaviour matters when a dashboard is assembled from row-level outcomes. “Passed” can otherwise be read as “tested successfully” even when a row was outside the check. Preserve eligible, failed and excluded counts in the reporting model, and verify them against the intended rule. A green result should not depend on readers knowing a hidden filter.

Before enabling enforcement, use a small test set containing a known good record, each failure condition, null and blank variants, boundary dates and a legitimate exception. Have the billing owner confirm the expected treatment. Test the response path too: can the operator identify the held lines, correct the source and safely release them without duplicate charging?

Repair the cause and keep the decision accountable

Run new checks in observation mode when necessary to understand their effect. Compare flagged records with actual billing decisions and resolve false positives before introducing a hard stop. Set a defined end to that observation period so a useful rule does not remain an ignored warning indefinitely.

Assign separate responsibility for changing a rule and correcting a record. The platform team can implement a threshold; it should not quietly change the business meaning of “billable” to make a failing run pass. Temporary overrides need a reason, an approver and an expiry or resolution event.

Finally, send recurring defects back to the point where the data is created or transformed. Repeatedly fixing the billing extract leaves the next run exposed to the same fault. The lasting next step is clear data ownership before further tooling, so someone can authorise the source change and accept the result.