Prompting & Context WindowsPrompt Swap Audit
Progress

Prompt Swap Audit

medium
Start small, think out loud.

The wording of a prompt is code, and swapping it deserves a code-review discipline: the same fixed question set graded under the old wording and the new, so the comparison is apples to apples. In production this looks like a spreadsheet of questions with a pass or fail column per wording; here it is two boolean lists. Implement prompt_audit(before, after): both lists record how the same questions were graded under the old prompt and the new one, True for a question that passed, in order, and the lists always have the same length. Return [old_rate, new_rate, broken], where each rate is the True count divided by the list length rounded to 4 decimal places, and broken counts the positions where before is True and after is False - the questions the swap silently broke, the piece a single pass-rate comparison hides, because a new wording can move a point up while pushing two others out the back door. Two lists of equal length, one pass in each direction, and the audit reports exactly what a real prompt change review should: did the rate move, and who paid for it. An empty list means no questions were graded, and an audit of nothing is zeros, not a crash.

Time:O(number of questions)
Space:O(1)
evalsprompt-changesregression-auditai-track
solution.py
⌘↵ run
Loading...
OUTPUT

Run your code to see output