Conflicting dashboards: agree what an active customer means

A management meeting should not need an analyst to explain which “active customer” number is on the screen. When two dashboards disagree, however, replacing the visualisation is often the least useful first move. The difference may originate in what each team is trying to decide.

For the data platform lead, the immediate job is to separate a legitimate difference in purpose from a defect in data or code. Forcing two measures to match can destroy useful information. Leaving their meaning ambiguous can send investment towards the wrong customer group.

Reconcile the population before debating the total

Consider a hypothetical subscription business. Marketing reports 1,000 accounts with a login in the past 30 days. A customer-success dashboard reports 800 paying customer accounts with a qualifying product action in the same period. Both headings say “active customers”.

Inspection shows that the first population includes 80 internal or test accounts and a further 120 customer accounts that logged in but did not perform the qualifying action. In this simplified example the groups do not overlap: 1,000 minus 80 minus 120 equals 800. There is no missing ingestion job to repair. The dashboards have different inclusion rules.

The useful response is to name and document both measures. Marketing may need a reach indicator; customer success may need a product-adoption indicator. If leadership wants to assess adoption among paying customers, it should choose the latter definition explicitly. The platform team should not make that business choice by quietly changing a query.

Now change one fact: both dashboards are meant to use the approved adoption definition, but one includes internal accounts because its exclusion join failed. That is an implementation defect. The same numerical disagreement can therefore require either a business decision or a technical repair.

Make the definition reproducible

Start with records that one measure includes and the other excludes. Look for differences in entity, event, period and adjustment. Is the entity an individual user, an account or a corporate customer? Does activity mean a login or a completed task? Is the period a rolling window or a calendar month? How are merged accounts and late-arriving events treated?

A definition needs enough precision for another analyst to reproduce the result. “Engaged customers” is a label. “Distinct paying customer accounts with at least one completed qualifying action during the defined reporting window, excluding internal accounts” is a starting rule that can be tested against examples.

Agree the reporting time zone, data cut-off and treatment of corrections. A dashboard refreshed this morning can disagree with a report frozen last Friday even when both implement the same event rule. Show the relevant freshness and period in the product so that the reader can judge what the number means.

A semantic layer distributes a decision; it does not make it

The dbt Semantic Layer documentation describes centralising metrics over data models and making those definitions available to downstream tools, with MetricFlow handling joins. This can reduce the need for separate teams to reimplement the same agreed calculation in each dashboard.

That is valuable when duplication is the problem. It cannot determine whether a login qualifies as adoption or whether two departments genuinely need different populations. Centralising an unresolved definition merely makes its ambiguity more widely available.

Adoption also needs checking. Inventory the reports, extracts and applications that use the measure, and establish which consume the shared definition. A spreadsheet with embedded logic or a dashboard that queries the raw tables can continue producing an older interpretation. Migration requires retiring or relabelling those paths as well as deploying the central calculation.

Give changes a business owner and an effective date

The UK Government’s data ownership model assigns owners accountability for central definitions in their subject areas and for guiding stewards to manage and adopt them consistently. That is a useful division for an enterprise: a business owner approves meaning, a steward maintains the definition, and technical teams implement and test it.

When the definition changes, decide whether historical figures will be recalculated or whether the series will show a break. Otherwise, a broader activity rule can appear to create customer growth overnight. Keep the effective date, rationale and affected consumers with the definition so that a later comparison can be interpreted correctly.

Use the next disputed metric as a small reconciliation exercise. Bring representative included and excluded records to the accountable owner, agree the decision the measure serves, and resolve each difference. Then translate the approved rule into data quality checks for the business process. A dashboard earns trust when its number can be explained and reproduced, including when it legitimately differs from another.