Evaluate a model change against the task, its sources and the decisions that follow.

A task-based comparison

Imagine a team evaluating two proposed ways to draft a short explanation of an internal procedure. A reader asks which instruction applies to a particular request. The useful result is not simply a fluent paragraph: it must identify the relevant source, describe its scope and leave any unresolved conflict visible. The comparison begins with that task definition, before a model name or a preferred interface enters the discussion.

Prepare fictional source material that makes the task reviewable. An ordinary case might contain one current instruction with a clear owner. A conflicting case might contain two versions whose status is unclear. A missing-information case might omit the location or condition that determines which instruction applies. These examples serve different purposes. Success on the ordinary case does not answer the question raised by the conflict, and an invented answer to the missing-information case would not make it complete.

Describe the acceptance conditions in language a reviewer can use. The draft should point to the source that supports its explanation, preserve the meaning of the relevant instruction and distinguish an answer from an unresolved question. If the source does not establish which version is current, a reviewable output should say what remains uncertain. Those conditions give the comparison a defined endpoint without asserting that any particular model can meet it.

Keep the comparison conditions visible. Record which fictional documents, question and drafting instructions were used, and identify any change between the two proposed approaches. If the source material changes at the same time as the model, the resulting difference cannot be attributed to the model alone without further investigation. A useful record helps another reviewer understand what was compared and what was left outside the exercise.

Inspect the output in relation to the task. A concise draft may omit an important qualification; a longer one may preserve the qualification while making the answer harder to find. Examine the relevant source passage and the reader’s intended decision before deciding whether either output is usable. The aim is to explain the observed difference, rather than choose a winner based on tone or length alone.

Follow the result into the next step of the workflow. In this fictional example, a reviewer still decides whether the explanation is suitable for the request. The draft does not approve a procedure change, publish an instruction or grant permission to act. If a proposed model change would also alter those responsibilities, describe that as a change in the workflow’s scope and examine it separately from the wording of the summary.

Record failures in a form that supports another decision. An output might cite an obsolete version, miss a condition or present a conflict as settled. State the source and expectation involved, preserve the unresolved question and identify what would need to be reconsidered. Avoid treating every disappointing result as the same problem. A source gap, an unclear instruction and a draft mismatch can require different follow-up work.

The next decision might be to refine the task, improve the fictional source set, narrow the proposed use or stop the comparison. Keep the conclusion proportional to the examples examined. A bounded exercise can reveal a specific weakness or a useful presentation choice; it cannot establish general superiority, organisation-wide value or operational readiness. Revisit the decision when the task, sources, review conditions or intended use change.

About this evaluation guide

Illustrative evaluation material. No provider integration, benchmark result, customer quotation or deployed capability is claimed.

This guide is original educational material for reasoning about a proposed change. It does not announce support for a named provider, reproduce a benchmark, describe a customer deployment or quote an employee. The fictional procedure task is deliberately limited so that its inputs, uncertainties and review decisions can be discussed without implying evidence from a real organisation.

Different tasks require their own source material and acceptance conditions. A procedure summary, a document classification and a proposed action do not share an identical endpoint. Adapt the questions to the work being examined, keep the responsible decisions explicit and use only material appropriate for the exercise. A familiar model label is not a substitute for describing those conditions.

Continue the review

Write a short record of the task, the fictional sources, the expected result and the question that remains open. Use it to explain the next decision to someone who did not watch the demonstration. The related reading below offers additional original material on reviewable prompts and source-aware evaluation. No form submission, provider connection or external account is required to read these materials.