AI pilot acceptance checklist: proceed, revise or stop

Accept an AI pilot only for a defined next stage, using evidence from agreed test cases rather than a polished demonstration. For a document-to-draft workflow, check the extracted work, the human effort and the controls separately. Require evidence that missing information stays unresolved, approval applies to the exact draft revision, prohibited actions are blocked and uncertain handoffs are reconciled before retrying. If a required test fails or remains unassessed, hold that next stage, revise the scope or stop.

This is Amulet's recommended acceptance method for Australian business owners and operating leaders. It is not a certification standard, legal advice, an accuracy target or a report of tests we have performed. Every proposed case below starts as Not assessed. The decision is whether to authorise a bounded next stage. It is not a declaration that the system is generally safe or ready for unrestricted use.

Set the boundary before testing

Amulet's public document-to-draft demonstration describes fixed-format synthetic requests, deterministic parsing and validation, an approval step and a simulated handoff. It is not a live AI model or an ERP connector. Its scope excludes PDF/OCR accuracy, authenticated approver identity and production security, monitoring and support.1

Use it to discuss where work should stop, not as evidence that your documents, users or destination system have passed these tests. The schedule below does not report results from running the public demonstration.

Before testing, have the workflow owner complete a short test brief:

Job and next stage sought: name the incoming document, the required draft and the action being considered. Distinguish preparing drafts for review from sending them to a business system.
Version under test: record the input pack, reference data, model or parser, prompt, rules, settings and destination environment. Keep the versions needed to reproduce a result.
People: name the test operator, business approver, system administrator and destination owner. Say who can accept each result and who can stop the pilot. A supplier's demonstration is not the business owner's acceptance.
Expected work: define required fields, permitted corrections, unresolved-item handling and which actions are forbidden until approval. Decide which errors block progression and what review workload the team can accept before seeing the results.
Safe test conditions: agree permitted data and access, a test destination and how to contain deliberate failures. Use synthetic material until representative business records are authorised. Do not inject failures into a live process merely to complete this checklist.

Keep cases used to tune the workflow separate from the acceptance set. Include the document formats and exceptions relevant to the proposed next stage. Record exclusions: testing clean text is not a reason to mark scanned-document handling as assessed.

Keep the simpler option in the test

Give the same required output and approval boundary to the manual process, relevant existing-software configuration and any proposed custom workflow. Ask your administrator to verify the native option under your actual product, plan, permissions and settings. Do not assume a feature exists or is included.

If a manual draft template and named reviewer meet the need, retain them. If existing software passes the required cases with acceptable effort, prefer that over commissioning equivalent custom work. Where explicit rules can do the job, test those before adding AI. If none of the options can be assessed because source records or responsibilities are unclear, fix the process first.

The partner selection guide covers supplier questions, and the implementation cost worksheet covers the resulting cost and ownership commitments. This schedule is for deciding what the pilot has demonstrated.

Measure extraction and review work separately

For an AI-assisted option, ask the business approver to set the expected result for each acceptance document before the run. Record each required field as correct, incorrect or unresolved, including whether its source reference supports the proposed value. Separately record whether the workflow surfaced each exception it was expected to catch.

Do not combine cosmetic differences, consequential quantity errors and missed exceptions into a single pass score. Set the acceptable error types and limits for this workflow before testing; this guide supplies no universal accuracy percentage. Record the case mix and the number of documents assessed beside any result. Keep synthetic control checks separate from observations on authorised representative documents.

Have the intended operator record the whole task: preparation, checking, corrections, approval, exception handling and recovery. Include rejected and abandoned work rather than timing only successful drafts. Compare that effort with the manual or native option on the same job, and ask the manager responsible for the team to accept the workload.

A well-extracted draft does not earn permission to skip an approval test, and a passing control case is not an extraction benchmark.

Run a case-based control schedule

Copy each case into the record below and assign actual names to the roles. The suggested runners and accepting roles are recommendations, not people already appointed to your pilot.

For every case, save the input and draft revisions, action attempted, actor and permission context, timestamps, workflow status and destination evidence. Where an action must be blocked, check both the workflow record and the agreed destination for unintended changes. Do not accept a success banner or an error message alone as the result.

A case may be marked Pass only when the agreed behaviour is observed and its evidence is saved. Mark an observed breach Fail. Missing evidence, an unavailable test environment or an unresolved outcome stays Not assessed, with a reason. If a case is outside the proposed scope, record the owner's exclusion and limit the next stage accordingly; do not count it as a pass.

Missing or conflicting input

Run: The operator submits a request missing a required quantity, then a separate request with contradictory delivery information. The business approver accepts the result.

Expected evidence: The draft identifies the affected fields and source passages, leaves the uncertainty visible and routes it to the named exception owner. Check that neither case creates a released destination record. The reviewer records the correction or rejection and its reason.

Fail or retest: Fail if a required value is silently guessed, a conflict disappears or release proceeds without resolution. After a fix, rerun both cases and a complete request to check that ordinary work can still reach review.