The Prompt Eval Playbook

Most prompt engineers never test their prompts. They ship a change, eyeball one or two outputs, call it done. This playbook is the alternative: a 4-category test suite (regression, hallucination, red team, boundary), a simulation prompt that runs each test, a judge prompt that scores them automatically, and a side-by-side scorecard that shows whether your new prompt is actually better.

• The 4-category test framework: regression, hallucination, red team, boundary. What each catches
• Reusable simulation prompt that runs any test case against your production prompt with mocked tool fixtures
• Reusable judge prompt that scores each output against a hard-rules checklist with pass/fail per rule