All question banks

Cost Metering Interview Questions

The bill is the one line every serving decision eventually meets, and the meter sees things the product does not: retries that product analytics never counted, prompt discounts the output side cannot reach, and spend nobody owns. These questions walk the meter end to end, from one request to a monthly ledger, with every number derived in the open. The linked terms are the glossary entries behind each answer.

Warm-up

1. What does one user turn cost? Walk the arithmetic for a chat request end to end.

The meter prices the two sides separately, because they measure differently - prompt tokens are read and cacheable, output tokens are generated and never are. One turn is prompt tokens times the input rate plus output tokens times the output rate, over a million because the rates quote per million tokens. The output side of even a small chat turn is the heavy half, which is the first lesson: output pricing is the lever sitting under every rate card.

Worked answer

  1. A chat turn: 800 input tokens at $0.80 and 300 output tokens at $4.00 per million.
  2. Input line: 800 x $0.80 = $640 at million-token scale, over 1e6 = $0.00064 billed.
  3. Output line: 300 x $4.00 = 1,200, over 1e6 = $0.0012.
  4. The turn: $0.00064 + $0.0012 = $0.00184, and the output side alone is 65 percent of it.

A chat turn bills $0.00184, and $0.0012 of it is output - the generated side, not the read side, is where the money is.

Core

2. The same 6 million requests, three request profiles. What does the profile decide?

Tokens per request decide the bill before the model does. A chat assistant, a copilot, and a research agent can all run 6 million requests a month and stay inside one tier conversation, yet bill in different orders of magnitude, because each prompt carries its reading and each answer carries its generation. That is why the interview answer prices a product in tokens per request - a profile you can name with its numbers beats a request count you cannot bill with.

Worked answer

  1. Chat profile on the fast tier, $0.00184 per turn: 6,000,000 x 0.00184 = $11,040 a month.
  2. Research agent on the flagship tier at 12,000 in and 4,000 out, $0.096 per request: 6,000,000 x 0.096 = $576,000 a month.
  3. The spread: 576,000 / 11,040 = about 52.2x, from the request profile alone.
  4. The tokens per request explain it: 16,000 against 1,100, about 15x, plus a 5x out-rate tier difference on top.

The same 6 million requests bill $11,040 as chat turns and $576,000 as agent requests - about 52.2x, all of it tokens per request.

Core

3. A copilot moves 2,400 tokens in and 600 out per request. What does the tier jump cost?

A tier jump multiplies the whole bill, not a line of it, because both sides reprice. The copilot profile on the fast tier, the flagship tier and the frontier tier shows the ladder: each step multiplies the input rate and the output rate together, and the last step is the expensive one. The interview answer reads the ladder in ratios, because a tier migration gets approved on the step size, not the absolute dollars.

Worked answer

  1. Fast tier ($0.80, $4.00): 2,400 x 0.8 + 600 x 4 = 4,320, over 1e6 = $0.00432 per request.
  2. Flagship tier ($3, $15): 2,400 x 3 + 600 x 15 = 16,200, $0.0162 per request.
  3. Frontier tier ($15, $75): 2,400 x 15 + 600 x 75 = 81,000, $0.081 per request.
  4. The fast-to-frontier step: 0.081 / 0.00432 = 18.75x for the identical request.

The same copilot request bills $0.00432, $0.0162 and $0.081 on the three tiers - the fast-to-frontier jump alone is 18.75x.

Core

4. Where does the prompt-cache discount land, and where does it never reach?

The cached share of the prompt bills at a tenth of the fresh rate, so the discount is real and it is prompt-only. The output line bills fresh every time a request runs, and the agent profile is the case where that matters: its weight sits on the generated side, so even a deep prompt discount leaves most of the bill standing. The feature that writes long answers is the one the cache saves least - which is the follow-up question interviewers ask about every caching story.

Worked answer

  1. The flagship agent request, fresh: 12,000 x $3 / 1e6 + 4,000 x $15 / 1e6 = $0.036 + $0.06 = $0.096.
  2. Output share without any cache: 0.06 / 0.096 = 62.5 percent of the bill.
  3. With a 90 percent prompt cache: the prompt line bills at a factor of 0.9 x 0.1 + 0.1 = 0.19, so 12,000 x 0.19 = 2,280 billable-units, $0.00684.
  4. Total with cache: $0.00684 + $0.06 = $0.0668, about $0.0668 against $0.096 - a 30.4 percent cut, none of it touching the output line.

A 90 percent prompt cache cuts the flagship agent request from $0.096 to about $0.0668, a 30.4 percent cut - while $0.06 of output bills untouched at 62.5 percent of the fresh bill.

Core

5. An agent reads a 12-million-token prompt and writes 4 million tokens of output at the frontier tier, 90 percent cached. Show the bill.

This is the bill shape interviewers want written out: a discounted prompt line, an untouched output line, and the two numbers that say what the cache is worth. The cached 90 percent of the prompt bills at a tenth of $15, the fresh 10 percent bills at $15, the whole 4 million of output bills at $75, and the fresh-rate bill is the baseline to compare against. The result shows why the discount forgone is the first meter to read when a bill surprises you.

Worked answer

  1. Prompt line at a 90 percent hit: 12M x (0.9 x 0.1 + 0.1) x $15 = 12M x 0.19 = 2.28M billable units x $15 = $34.20.
  2. Output line: 4M x $75 = $300.00, billed fresh as always.
  3. The bill: $34.20 + $300.00 = $334.20.
  4. The same request at the fresh rate: 12M x $15 + 4M x $75 = $180.00 + $300.00 = $480.00, so the cache took $145.80 off and $300.00 of output rode through untouched.

The cached bill is $334.20 against a $480.00 fresh-rate bill - the cache saved $145.80 on the $180 prompt line and nothing on the $300 output line.

Core

6. A $5 million gross month with a 20 percent retry rate and 50 percent of the bill tagged. What does the ledger actually show?

Two multipliers stack. The retry factor adds dollars - a retry re-bills the whole request at full prompt and output price, and product analytics never register the re-bill because no new traffic happened. The tag factor then names whatever the retry factor just added, and what carries no tag has no owner: no chargeback, no margin math, and no anomaly alert reads it. Both leaks are metering fixes, not analytics fixes - one is a retry rate to manage, the other is a required tag on every metered call.

Worked answer

  1. The retry factor: $5,000,000 x (1 + 20 / 100) = $6,000,000 billed, a 20 percent uplift over gross.
  2. The tag factor: the untagged share is (100 - 50) / 100 of the billed total.
  3. Unattributed dollars: $6,000,000 x 0.5 = $3,000,000 - half the billed month, with no owner.
  4. Read together: the retries added $1,000,000 of bill and the tagging gap lost a further $3,000,000 of visibility.

A $5,000,000 gross month becomes $6,000,000 billed and $3,000,000 unattributed - $3,000,000 of ghost dollars the ledger cannot name.

Hard

7. A 20 percent retry rate on a 2-million-request agent month. Price the retry tax end to end.

A retry is a second workload nobody planned for. Because the re-issued request carries the whole prompt again, the per-request bill multiplies by the retry factor, and over a request count the tax is the re-billed requests at full price. The meter is the only place these requests exist: product analytics still reports the same 2 million requests, and the difference between what the product counted and what the bill counted is exactly the tax.

Worked answer

  1. One flagship agent request, fresh: $0.096 (12,000 x $3 + 4,000 x $15, over 1e6).
  2. With a 20 percent retry rate: 0.096 x 1.2 = $0.1152 expected per issued request.
  3. The clean month: 2,000,000 x 0.096 = $192,000.
  4. The billed month: 2,000,000 x 0.1152 = $230,400 - a retry tax of $38,400, all of it re-billed whole requests.

A 20 percent retry rate turns a $192,000 agent month into $230,400 - $38,400 of re-bills the product analytics never saw.

Core

8. The copilot runs a clean $122,472 flagship month, 5 percent retries, half the bill tagged. What is unowned?

The realistic case, at realistic scale. The retry factor lifts the bill by 5 percent - a leak small enough to survive several review cycles - and the tag factor loses half of everything billed. The unowned line is big enough to fund a feature, and it has no owner to defend it, which is what the untagged share is: not an accounting error, but real spend the ledgers and the anomaly alerts cannot see. The fix order is cheap first: a required tag costs nothing in dollars and closes the whole visibility leak.

Worked answer

  1. The retry factor: $122,472 x 1.05 = $128,595.60 billed.
  2. The tag factor: (100 - 50) / 100 of the billed total has no tag.
  3. Unowned dollars: $128,595.60 x 0.5 = $64,297.80.
  4. Halving the tag gap costs nothing: tagging at 100 percent names the full $128,595.60 while the retry rate is worked down separately.

A 5 percent retry and half-tagged bill leaves $64,297.80 of a $128,595.60 ledger unowned - the visibility leak, at real scale.

Warm-up

9. What is the blended rate, and why does it disagree with the monthly bill?

The blended rate is the bill divided by every token the month moved, cached and fresh together - one number, scale-free, that you can compare across months and across profiles without a tier table. It disagrees with the bill on purpose: the bill answers a dollars question and the blended rate answers a rate-per-use question, and the failure each catches is different. A bill that grew with traffic is fine; a blended rate that grew with flat traffic usually means the cache discount or the profile changed, and that is the diagnosis the bill alone cannot name.

Worked answer

  1. Tokens the month moved: 2,000,000 requests x (12,000 + 4,000) = 32,000 million tokens.
  2. The billed month with a 20 percent retry rate: $230,400.
  3. Blended rate: 230,400 / 32,000 = $7.20 per million tokens moved.
  4. The same month clean would blend at 192,000 / 32,000 = $6.00 - so $1.20 per million of the movement is retries, visible in one division.

2 million agent requests move 32,000 million tokens and bill $230,400 with retries, a blended $7.20 per million against a clean $6.00 - the rate-per-use view the bill cannot give.

Hard

10. The monthly bill jumped with flat traffic. Which meter do you read first?

The cache hit share, because the discount forgone is the first thing a silent change can take away. A prompt edit near the head of the preamble voids the cache for every request at once, and nothing in the traffic counts changes. The arithmetic of the loss scales fast: dropping the hit share from 90 to 40 percent on a big agent volume moves the prompt line from a 0.19 factor to a 0.64 factor, and the prompt line is the only part of the bill the cache was ever discounting.

Worked answer

  1. Per frontier agent request at a 90 percent hit: 12M x 0.19 x $15 + 4M x $75 = $334.20.
  2. The same request at a 40 percent hit: 12M x (0.4 x 0.1 + 0.6) x $15 = 12M x 0.64 = 7.68M billable units, $115.20, plus $300.00 of output = $415.20.
  3. Per-request loss: 415.20 - 334.20 = $81.00, a 24 percent rise from 50 points of lost hit share.
  4. Across a 6,000-request month of these deep-research calls: 6,000 x $81.00 = $486,000 - the surprise, located in one knob.

Losing 50 points of cache hit share costs $81.00 per frontier agent request and $486,000 across a 6,000-request month - which is why the hit share meter reads before the bill does.

Need the vocabulary first? The glossary defines each of these terms with a worked example, and the other banks cover the other half of the interview.