What does an AI pilot actually involve?

An AI pilot is a limited piece of work that tests whether a proposed approach is useful under agreed conditions. It needs a defined output, representative inputs, someone to review the result and a decision at the end: proceed, revise or stop.

A convincing demonstration is only one part of that work. A business also needs to know what happens with incomplete information, who handles exceptions and what it would take to run the system after the builder leaves.

Agree the decision the pilot will support

For a hypothetical Sydney wholesaler, a pilot might prepare a daily operations summary from an approved order export. The question could be whether the team can verify the summary and resolve exceptions with less handling than its current process.

Keep the scope specific: one input format, one report definition and one review group. Exclude changes to orders, customer messages and automatic business decisions unless they are separately agreed. Our reporting demo is a useful way to discuss this shape of work; it is not evidence that a customer pilot has achieved it.

If the pilot includes AI-written commentary, state exactly what that commentary may do. Calculation of totals can remain in ordinary code. The model's role might be explaining verified results, with unsupported explanations treated as errors.

Prepare sources before connecting tools

Identify the owner of each source and confirm which fields are needed. Agree access, permitted uses, processing services, retention and deletion arrangements before moving business information. Start development with synthetic or suitably de-identified examples where appropriate, then separately authorise any use of representative business data.

For the reporting example, define what counts as a confirmed order, which date determines inclusion and how cancelled items are handled. Set the business timezone and reporting cut-off explicitly; a Sydney business should not inherit an unexplained default from a server.

Choose one reference dataset whose expected result the process owner can verify. Keep additional cases aside for testing after the implementation has been tuned. Otherwise, success may only show that it reproduces the examples used during development.

Design the review as part of the workflow

A review screen should give someone enough evidence to make a decision. For a report, that means the snapshot time, included records, calculation definitions and exceptions, alongside any generated commentary.

Name the reviewer and a backup. Decide what happens when neither is available: the result may need to wait rather than progress automatically. Define how a rejected output is corrected and whether approval expires when its underlying information changes.

The NIST AI RMF Playbook provides voluntary guidance organised around Govern, Map, Measure and Manage. It is a useful reference for risk discussions across a project's life. The pilot steps here are our practical approach, not a NIST certification checklist or a statement of Australian legal compliance.

Make the cost picture visible

Ask for costs to be separated into setup and ongoing operation. A proposal should explain the assumptions behind each category, rather than presenting a model subscription as the whole project cost.

• Preparation and setup: process mapping, source cleanup, configuration and any agreed connections.
• Evaluation and rollout: test cases, review sessions, training and deployment work.
• Recurring services: hosting, model usage, storage and relevant software licences.
• Operation and change: monitoring, support, maintenance and changes to sources or business rules.

Clarify what is included, who holds the service accounts and what triggers additional work. Ask how expected usage and unusually large runs affect operating costs. Monkey Go agrees scope, pricing, responsibilities and third-party costs before implementation; this article does not set a package price or delivery timetable.

Test the difficult cases before expanding

Write acceptance conditions before building. For this illustrative reporting pilot, they might require the calculation to match the reference dataset, cancelled records to be excluded and an old snapshot to be clearly identified. AI commentary must not add a cause or claim that the source material does not support.

Then test beyond the clean example:

1. Remove a required field and check that the run stops or reaches the agreed exception path.
2. Introduce duplicate or conflicting records and check that they are not silently counted twice.
3. Use an old snapshot and verify the freshness warning or blocking behaviour agreed for the pilot.
4. Interrupt a source connection and check that yesterday's output is not presented as today's successful run.
5. Repeat a run and inspect the records so that retries do not create unintended duplicate actions.

These are suggested tests for a real pilot, not a claim that every behaviour is implemented in our public demo. Record failures and unresolved limits alongside successful cases. Compare the full work, including review and correction, with the baseline.

Handover includes a way to stop

Before acceptance, have the operator run through the workflow, handle an exception and practise returning to the previous process. The handover should identify the owner, configuration, operating instructions, known limits and support arrangements, with source materials and account responsibilities defined in the scope.

Agree what changes require another review: a new data source, different model, revised rule or added permission can change the result. Decide whether the evidence supports operating the pilot, adjusting it or ending it. Expansion is a separate decision.

See what a reviewable result looks like

Try the traceable reporting demo. It is an Illustrative demo / Synthetic data example using fixed records and rules, with no live AI model or business connection. Inspect the source rows and compare the current and stale snapshot states.

Let's talk about a pilot with a clear output and owner. Bring a general description of the work; confidential records are not needed in the initial enquiry.