Purpose
Use this lesson to judge AI output against observable criteria instead of a vague impression. The aim is to produce work that a colleague can review and act on, not merely text that sounds convincing.
Lesson Explanation
Scenario: Evaluate a draft customer FAQ.
Better instruction: Score the FAQ from 1–5 for factual accuracy, completeness, tone, scannability, and policy compliance. Require a cited reason and revision for every score below 4.
A rubric turns “Is this good?” into repeatable review. Define criteria, scale, evidence required, threshold, and what happens when a response fails. Use the rubric on examples before relying on it broadly. Every low score includes a reason and a specific revision.
Work in a short cycle: provide the relevant input, state the result and limits, inspect the draft against the acceptance criteria, then revise the instruction when a requirement is missing. Keep the final human decision with the person responsible for the work.
Practice Questions
-
What is the main work outcome in “Creating an Evaluation Rubric”?
View Answer
judge AI output against observable criteria instead of a vague impression.
-
Why is this request incomplete: Evaluate a draft customer FAQ.
View Answer
It leaves important decisions to guesswork. The lesson shows how to supply the missing purpose, context, boundary, or format.
-
Which instruction makes the request operational: Score the FAQ from 1–5 for factual accuracy, completeness, tone, scannability, and policy compliance. Require a cited reason and revision for every score below 4.
View Answer
It defines a concrete result that can be checked instead of asking for a vague response.
-
What principle should guide your prompt for this lesson?
View Answer
A rubric turns “Is this good?” into repeatable review. Define criteria, scale, evidence required, threshold, and what happens when a response fails. Use the rubric on examples before relying on it broadly.
-
What would count as an acceptable result here?
View Answer
Every low score includes a reason and a specific revision.
-
Name one detail you would verify before using an AI result for “Creating an Evaluation Rubric”.
View Answer
Verify the source facts, inputs, numbers, names, dates, policy limits, or assumptions that affect the real decision.
-
What should you add if the result is polished but not usable for the scenario in this lesson?
View Answer
Add the missing acceptance criteria or output structure, then regenerate and compare against the stated requirement.
-
Which is safer: asking for a general answer or stating the business context in this lesson? Why?
View Answer
State the business context because it reduces irrelevant guesses and lets the AI tailor the work to the actual situation.
-
How would you test the output from “Creating an Evaluation Rubric” before sharing it?
View Answer
Check it against the requested facts, format, limits, and acceptance criteria; then have the appropriate human reviewer approve it.
-
What should the AI do when a required fact is missing in this scenario?
View Answer
Flag the missing fact or ask a focused question rather than silently inventing an answer.
-
What risk does the lesson warn about for “Creating an Evaluation Rubric”?
View Answer
Treating fluent output as automatically correct, complete, approved, or fit for real-world use.
-
Write the shortest useful improvement to the weak request in this lesson.
View Answer
Score the FAQ from 1–5 for factual accuracy, completeness, tone, scannability, and policy compliance. Require a cited reason and revision for every score below 4.