When a technology pilot has earned another phase

Before extending a pilot, ask the team to name the result that would persuade it to stop. If no plausible result would change the recommendation, the next phase is unlikely to be an experiment. It is more likely to be an implementation programme whose funding decision has remained implicit.

That distinction matters to a programme sponsor. Another month consumes engineering capacity, keeps alternatives waiting and can deepen dependence on a supplier. Continued interest from users is useful evidence, but it does not establish that the system is ready to support their work or that the next phase is worth its cost.

Test the assumption that could break the case

A pilot should address a consequential uncertainty. Can the proposed integration support the required transaction? Will users complete the new workflow? Does the service remain affordable once review, support and exceptions are included? A successful demonstration of an easy feature cannot settle a difficult dependency elsewhere in the proposal.

The GOV.UK Service Manual’s alpha guidance illustrates this with Register to vote: the team focused on whether an API could integrate with local electoral-registration systems, because the service depended on that connection. The guidance also treats prototypes as a way to test risky assumptions, with code that may be discarded. This is a useful model for enterprise pilots: protect the learning objective from the urge to make every prototype look finished.

For your review, bring back the original hypothesis and acceptance criteria. Identify which observations support it, which contradict it and which required evidence is missing. If the criteria were never agreed, acknowledge that gap and define the decision now. Do not quietly invent a passing threshold around the results already collected.

A mandatory requirement that the technology demonstrably cannot meet is different from a requirement the team has not tested. The first may justify stopping or changing the design. The second may justify another experiment, provided that experiment is feasible and its answer would affect the choice.

Give an extension one job

Consider a hypothetical AI-assisted document-review pilot. Users like the interface, but the trial used straightforward documents and did not include the complicated amendments that drive most expert review effort. The sponsor does not yet know whether the application reduces the cost of an accepted review.

A weak extension would add users and collect more general feedback. A stronger one would test the missing document class, using authorised examples, a defined acceptance rubric and a comparison with the existing workflow. It would count review time and rework as well as model processing. The question is whether the observed benefit survives the difficult work.

Suppose the proposed next phase costs £20,000 and runs for four weeks. Those are scenario assumptions, not market prices. The sponsor should know which team will supply the cases, who can judge the outputs and what decision will follow each material result. If the necessary data or reviewers are unavailable, approving the calendar extension does not make the experiment executable.

Keep the production boundary explicit. An improved result on the missing document class may justify a controlled operational trial. It does not automatically establish support coverage, integration resilience or permission handling. Name the gate being crossed and the questions reserved for the next one, so a narrow success does not become a broad deployment approval.

Also retain a credible alternative: improving the current workflow, using a simpler tool or delaying the investment. A pilot assessed only against its previous version can keep improving without becoming the best use of the organisation’s resources.

Judge the money still at stake

Money already spent does not become recoverable because another phase is approved. The continuation decision should compare the future cost and likely value of the available options. Treat “we have invested too much to stop” as an explanation of discomfort, not a forecast of benefits.

Future exit costs still count. Ending the pilot may require data export, deletion verification, contract close-out, restoration of an earlier process or reassignment of staff. Continuing may trigger a commitment, require specialist support or postpone a better project. Make those consequences visible on both sides of the decision.

Stopping can be the successful outcome of a useful test. It prevents a larger commitment to a weak idea and leaves evidence that another team can use. Preserve what was learned, including the conditions under which the result might differ. A changed requirement or a materially different product may justify a new decision later; an unchanged demonstration does not.

Approve another phase when it has a decision-relevant question, an achievable method, a bounded cost and an agreed review point. Change the pilot when its design cannot answer the question. Stop when the evidence rejects the case or no worthwhile experiment remains.

For the AI example, the next practical step is an evaluation set built around the tasks and failures that matter. It turns an open-ended request for more time into a test the sponsor can use.