Automation · 9 min read
How to identify an automation opportunity worth piloting
How to qualify, baseline, scope, control, and measure an automation pilot before committing to a wider rollout.
A frequent task is not automatically a good pilot
A strong first automation pilot combines meaningful value and enough volume with a stable, explainable process, accessible systems, measurable baselines, a safe exception path, and an accountable owner. An unpopular manual task can still be a poor candidate when the process changes weekly, exceptions dominate, or nobody can define the correct outcome.
The purpose of a pilot is to reduce uncertainty before a larger commitment—not to prove that the original idea must be scaled.
Score the opportunity before choosing technology
Review the dimensions below as favourable, partly ready, or high risk. A high-value opportunity can still be a stop when lawful access, ownership, or a safe exception path is missing.
| Dimension | Favourable evidence | Warning sign |
|---|---|---|
| Business value | Clear capacity, cycle-time, quality, control, or service benefit | The task is only an irritation with no material outcome |
| Frequency and volume | Enough representative cases to baseline and compare | Rare or seasonal work with too few cases to test |
| Process stability | The main path will remain stable during the pilot | The process or applications are being redesigned |
| Rules and variants | The normal path, decisions, and exceptions can be explained | Tacit judgement dominates or staff disagree on the correct result |
| Input readiness | Representative digital data is available and understood | Only clean samples exist while production inputs are messy |
| System access | Approved test access, identities, and interfaces are available | Production-only access or unresolved security constraints |
| Ownership | Named sponsor, process owner, and operations/exception owner | No one owns the end-to-end outcome |
| Risk and reversibility | Consequences are bounded, auditable, and recoverable | The first action changes money, rights, employment, safety, or access without approval |
| Delivery scope | One useful path can be proved independently | The pilot requires every department, system, and variant |
Good and poor first-pilot use cases
Good candidates
- Invoice intake for one document family: extract defined fields, validate identifiers and totals, route uncertain cases, and post approved records.
- Employee document generation: use approved source data to create, protect, deliver, and track a known document type.
- New-starter setup for one worker type: run a defined API workflow and route missing approvals or policy exceptions.
- Daily legacy report retrieval: use RPA for the UI-only download, followed by deterministic reconciliation and distribution.
- Service-request triage: let an agent interpret the request and draft a recommendation while a person approves consequential action.
Poor candidates
- A process that is changing during the proposed pilot.
- A broad brief such as “automate customer service” spanning many teams and systems.
- A low-volume workflow without a repeatable test set.
- A process whose correct outcome is disputed and has no decision owner.
- A UI robot for a system that already provides a suitable governed API.
- A document-extraction demo tested only on perfect files.
Choose the simplest suitable method
| Work characteristic | Start with | Control to retain |
|---|---|---|
| Predefined steps and supported interfaces | Deterministic workflow or API automation | Input validation, idempotency, retries, and reconciliation |
| Stable steps but only a UI is available | RPA | Stable selectors, application-exception handling, and manual fallback |
| Values must be extracted from documents | Document AI plus deterministic validation | Use-case-specific quality thresholds and human review |
| Open-ended goal and contextual tool choice | AI agent | Narrow tools, least privilege, evaluation, approval, and traceability |
| Mixed process | A combination | Visible boundaries between interpretation, validation, execution, and human responsibility |
Do not use an agent to execute a long fixed sequence. Do not force a growing rule tree to imitate genuine contextual judgement. The simplest reliable design is usually easier to test, support, and justify.
Baseline the current process before building
Measure a representative period and segment peaks or variants where they differ. At minimum record:
- transactions received, completed, and abandoned;
- manual handling time and end-to-end cycle time;
- common variants and their share of volume;
- errors, rework, review effort, and exception categories;
- staff and hand-offs involved;
- backlog age and service-level performance;
- labour, licence, infrastructure, and outsourced cost;
- quality, control, or customer-impact measures relevant to the outcome.
Separate touch time from waiting time. Use logs, timestamps, work queues, and sampled observation where possible. Do not calculate annual savings from a workshop estimate multiplied by every attempted case.
Design exceptions and human review as part of the solution
For every foreseeable uncertainty or failure, answer:
- What happened, and what completed already?
- What evidence does the reviewer need?
- Which role owns the exception queue?
- Can the person correct, approve, reject, retry, compensate, or complete manually?
- What is the service level and escalation route?
- Which failure, error-rate, data-quality, or risk threshold stops the automation?
For document AI, choose confidence and review thresholds from a representative evaluation and the harm of false positives and negatives. Do not copy an arbitrary vendor example. Human review needs context, authority, time, and a standard; measure its volume and correction time because a pilot can hide manual work in an exception queue.
Write a pilot charter before implementation
- Problem: the operational constraint, not a technology ambition.
- In scope: one trigger, bounded path, users, systems, data types, and eligible transaction population.
- Out of scope: deferred variants, departments, decisions, and production volumes.
- Baseline and hypothesis: exact metric definitions, data sources, and the change expected.
- Method: why workflow, API, RPA, document AI, agent, or a combination is the simplest suitable design.
- Test set: normal, peak, edge, invalid, and exception cases with expected outcomes.
- Controls: permissions, validation, approvals, audit, thresholds, stop switch, retry, and manual fallback.
- Roles: who supervises, reviews, supports, and accepts the result.
- Exit criteria: numerical go, change, and stop conditions agreed before results are known.
Run the pilot as a controlled business test
- Observe and map: confirm the real path with the people doing the work.
- Baseline and segment: define the eligible population and current measures.
- Remove avoidable complexity: fix redundant steps, unclear ownership, and weak input rules first.
- Prototype: prove the hardest integration and representative edge cases early.
- Run in shadow mode: compare proposed outputs without taking consequential action.
- Limit live exposure: cap users, volume, value, or document type and retain manual fallback.
- Review frequently: inspect failures, exceptions, adoption, benefit, and total cost.
- Decide: scale, correct one bounded issue and retest, redesign, or stop.
Agree the exit criteria before seeing results
Business outcome: cycle time, manual touch time, errors and rework, backlog, service level, cost per accepted transaction, and capacity released.
Reliability and control: accepted outcomes, straight-through and assisted rates, business and technical exceptions, reconciliation, recovery time, review effort, unauthorised-action incidents, and audit completeness.
Adoption and operability: active and repeat users, eligible work completed outside the automation, training, operator correction time, support coverage, fallback readiness, and actual licence, infrastructure, model, and maintenance cost.
Scale only when the outcome improves, quality and risk remain within limits, exceptions are manageable, users adopt it, the support model is permanent, and total cost still supports the case. Stopping is a valid result when the process, controls, ownership, or economics do not support expansion.