Azure OpenAI Provisioned Throughput, Explained with Actual Numbers

Provisioned Throughput Units are the most misunderstood pricing construct in Azure AI. Teams either overbuy PTUs for workloads that idle overnight, or hammer pay-as-you-go until 429s throttle their product. The right answer is usually both — in the right proportions.

The mental model

• Pay-as-you-go: you rent seats on a shared bus. Cheap, but at rush hour you might not board (429s).
• PTU: you lease the bus. Guaranteed latency and throughput, paid for whether it's full or empty.

PTU pricing rewards sustained, predictable load. The break-even question is simply: what fraction of the day does your traffic keep the reserved capacity busy?

Doing the math

Suppose your chat workload averages 400K tokens/minute during business hours (10 h/day) and near zero otherwise.

The hybrid pattern: size PTUs near your median sustained load, then configure spillover so burst traffic overflows to a pay-as-you-go deployment.

Client → APIM (managed identity, retry policy)
          ├── primary: PTU deployment      (p50 traffic, guaranteed)
          └── fallback: PAYG deployment    (bursts, batch, off-hours)

API Management's retry-on-429 policy makes the failover invisible to callers, and gives you one place to log token usage per consumer — which you will need for chargeback the moment finance notices the bill.

The three mistakes to avoid

1. Sizing PTUs for peak. You inherit the idle cost of your spikiest hour, forever.
2. Ignoring monthly reservations. Committed PTU pricing is dramatically cheaper than hourly.
3. No per-consumer metering. Without APIM-level token logging, your first cost conversation is an archaeology project.

Run the utilization report after two weeks of production traffic and re-size. PTU allocation is not a one-time decision — it's a quarterly tuning loop, and the teams that treat it that way cut AI spend by a third without touching product behaviour.