Use AI to improve productivity is too broad an instruction to evaluate. Draft a short internal summary of an approved meeting note, with a person checking every fact, is a task you can test. A useful pilot begins with a bounded activity and a description of an acceptable result.
Choose reversible work and information your organisation has approved for the tool. Use fictional or appropriately de-identified examples when testing a workflow. Consequential decisions need a separate assessment, not an assumption that success on a simple drafting task will transfer to them.
Establish the ordinary process
Complete representative examples using the current method. Record the time and checks required, including corrections. That baseline prevents a quick first draft from being mistaken for a quicker completed task. Include awkward examples as well as clean ones: an unresolved action, conflicting dates or an unfamiliar abbreviation can reveal failures absent from a polished demonstration.
Write the scorecard before seeing the output
For the summary example, assess factual accuracy, missing decisions, action attribution and review time. Define serious errors separately. An invented deadline should not disappear inside an average score for good formatting. The NIST AI RMF measurement guidance emphasises appropriate methods, documented tests and context. This small workplace example applies that general idea; it is not a claim of formal compliance.
Record the input version, instructions, tool version where available, output and corrections in approved storage. Avoid retaining unnecessary sensitive information. When changing instructions, repeat selected examples to see whether fixing one problem creates another. A reviewer should be able to understand what changed without relying on someone’s memory of a demonstration.
Count review work as work
A hypothetical trial might produce drafts faster while making verification slower. That is useful evidence, not an inconvenience to remove from the results. It may support a narrower task, clearer source material or a decision not to continue. Examine individual failures alongside averages, particularly when a mistake would be costly.
Set a review date and name the person responsible for the decision. Consider quality, total effort, privacy, accessibility and failure handling. Keep a manual route available during assessment. Expansion should follow evidence that the task works under realistic conditions, not excitement about one impressive output. Stopping an unsuitable pilot is also a useful result.