Penetration testing, red teaming and validation answer different questions

A CISO who wants to know whether the response team can recognise an intrusion needs a different engagement from an application owner who wants to find exploitable weaknesses before launch. Both may be sold as security testing. The purchase becomes clearer when the intended question comes before the service label.

Penetration testing, red teaming and ongoing control validation overlap, and suppliers use the terms differently. A useful scope explains the objective, permitted activity and evidence expected. It should also make clear what the result will leave unanswered.

A penetration test investigates a defined scope

A scoped penetration test can examine whether weaknesses in an application or environment can be combined into a meaningful security impact. Its value depends on the scope, access provided, time available and the tester’s approach. A report about one application cannot establish the security of the wider organisation.

NIST’s technical testing guide places planning, findings analysis and mitigation alongside the testing itself, and explicitly discusses techniques’ benefits and limitations. Although the guide dates from 2008, that procurement discipline remains useful: decide what evidence the exercise must produce and how the organisation will use it.

For an application release, request findings tied to the relevant business functions and a way to verify fixes. A technically detailed report that nobody can translate into development work has not completed the assurance journey.

A red-team exercise examines a route to an objective

An adversary-led exercise can test how prevention, detection and response interact around an agreed objective. The objective might concern access to a sensitive business capability rather than a comprehensive search for every weakness.

That focus creates a different evidence boundary. Success on one path can reveal an important exposure, while failure to reach the objective does not prove that every other path is blocked. The engagement’s conditions also affect what can be concluded about defenders’ performance.

The MITRE ATT&CK knowledge base provides a shared vocabulary for adversary tactics and techniques. It can help teams describe the behaviours an exercise covers. Mapping activity to ATT&CK does not itself establish that a detection worked or that a service is secure.

Repeated validation checks whether selected controls still work

Repeated control tests can help detect regressions after configuration or platform changes. They are especially useful when the expected behaviour is well defined and the result can be investigated consistently.

The limitation is selection. A repeatable library of checks says most about the behaviours included in that library under the conditions tested. Treat it as evidence about those controls, not a substitute for every exploratory assessment or a broad claim of adversary resistance.

Question	Useful starting point	Important limit
Can weaknesses in this application produce a material impact?	Scoped penetration test.	Results depend on scope, access and time.
Would the organisation recognise and respond to this adversary-led scenario?	Exercise with explicit detection and response objectives.	One scenario is not complete coverage.
Did this selected control regress after a change?	Repeatable control validation.	Unselected behaviours remain outside the result.

Buy the follow-through

Agree authorisation, boundaries and stop conditions before activity starts. Then agree how findings will be handed to the teams that can change the outcome. A detection gap may need better telemetry, a rule change, analyst training or a different escalation path. The report should help distinguish those causes.

For a hypothetical response exercise, an alert arrives but the incident stalls because nobody can approve containment. Another detection product would not directly repair that authority gap. The useful finding concerns the handover, and the retest should examine whether it now works.

Use the detection coverage guide to connect an observed behaviour to a response outcome. Commission the exercise that answers the consequential question, then reserve time and resources to act on what it reveals.