Judge Calibration Agreement

Your team ships an AI grader: a model that reads saved support transcripts and stamps each one pass, partial, or fail, encoded as 2, 1, and 0. Releases depend on its grades, so before trusting it, a teammate grades the same transcripts by hand and you measure how often the AI matches the human. That match check is the core of judge calibration. Implement judge_agreement(pairs): pairs is a list of tuples (judge, human), integers on a 0-2 axis, and the function returns a two-item list of fractions - first the share of grades in exact agreement, then the share within one step of the axis, meaning an absolute difference of 1 still counts, while a 2-step gap such as judge 0 against human 2 does not. Round each fraction to 4 decimal places. An empty list returns [0.0, 0.0]: no labeled grades means no measured agreement, and defaulting to 1.0 would be exactly the false confidence this topic exists to catch. The second number is the interview trap - a judge with a perfect within-one rate can still carry a one-directional bias that quietly upgrades failing answers to partial credit.