Vulnerability metrics: explain what disappeared before claiming progress

A CISO presenting a lower vulnerability total is making an implicit claim: the organisation has less exposure than it did before. The dashboard may support that claim, but only after the team explains which assets remained visible and why findings left the count.

This is a reporting problem with budget consequences. If lost scanner coverage looks like successful remediation, the programme can be rewarded for losing sight of the systems that need work. If better discovery looks like deterioration, a useful inventory improvement can be penalised.

A 40% reduction with very little remediation

Take a hypothetical monthly report. It begins with 1,000 asset-level findings and ends with 600. The headline is a 40% reduction: 400 fewer findings divided by the opening 1,000.

Now reconcile the movement. The team verified 100 fixes, added 50 newly discovered findings and removed 350 findings from the displayed total because a scanner integration stopped reporting a group of assets. The arithmetic is 1,000 minus 100 plus 50 minus 350, giving 600. Only 100 findings have evidence of remediation.

Assume the expected estate stayed at 200 assets, but only 150 had a successful assessment inside the organisation’s chosen seven-day freshness window at month end. Coverage fell from 100% to 75%. The business has gained uncertainty about a quarter of the expected estate, even as the backlog chart improves.

The right management response is to restore visibility and assess the missing assets’ importance. It is not to treat all 350 vanished findings as still exploitable forever, nor to call them fixed. Their state is unknown until the team obtains fresh evidence.

Show verified fixes, new discoveries, confirmed asset retirement, duplicate removal, accepted exceptions and lost visibility as separate movements. A genuine decommission can remove exposure; a missing sensor cannot demonstrate the same result. Where exceptions remain technically unresolved, keep them visible even if the workflow ticket has been administratively closed.

The scoring model can move too

Microsoft’s Defender Vulnerability Management documentation describes a transition between exposure-score models. The updated model combines CVSS and EPSS signals with asset context, and Microsoft warns that scores and recommendation priorities can change during the transition without corresponding changes in the environment.

That is a concrete reason to annotate a trend rather than splice two scoring regimes into one unexplained line. Establish which model the tenant uses before interpreting movement. The same documentation notes that newly discovered vulnerabilities can offset the score effect of remediation.

A stable score can therefore coexist with useful work. The question for the committee is what changed underneath it: which material exposures were reduced, what new exposures appeared, and whether the assessment population or calculation changed. A vendor score is a useful summary within its documented model; it is not a calibrated probability of financial loss.

Choose measures that lead to different decisions

Backlog volume helps estimate work, but it rewards removal of large numbers of easy findings unless paired with consequences. Show unresolved findings on critical services and their verified access paths alongside the total. That allows the service owner to explain why a small, difficult group deserves more engineering time than a large low-consequence group.

Remediation speed helps identify delivery friction, but an average calculated only from closed findings excludes the work that never finishes. State the closure window and the population included, then show the age of still-open material findings. Fast closure of routine desktop issues should not conceal an overdue gateway upgrade. The resulting decision may be to fund a maintenance window or resolve an application dependency.

Coverage measures help expose blind spots, but the denominator itself needs scrutiny. “95% assessed” is weak reassurance if the expected asset list omits newly acquired infrastructure. Reconcile that list to relevant cloud inventories, service ownership and discovery evidence, and distinguish successful recent assessment from mere enrolment. A high percentage with critical systems missing should lead to targeted coverage work.

Exception expiry shows whether accepted delays are being revisited. It does not prove that a compensating control works. Pair overdue exceptions with the last evidence that their control still blocks the relevant path. The decision is whether to renew, change or end the exception, with a named owner for the consequences.

Use consistent observation windows and state when the population changes. A stable comparison group can help explain a trend, but report excluded and newly discovered assets separately so that the comparison does not hide growth in the estate.

The next report should reconcile its headline movement before celebrating it, then identify the few unresolved exposures that require a business decision. Turn those findings into an evidence-backed remediation order. The programme’s achievement is the change it can demonstrate in important services, not simply the number of rows it can remove.