What to Do When Your AI Provider Goes Down

Every major AI provider has shipped a status-page incident at some point — elevated error rates, degraded latency, or a full outage. That's not a knock on any one provider; it's the reality of depending on external infrastructure you don't control. The difference between a minor blip and a real incident for your own users usually comes down to how fast you notice and how ready your fallback path is.

Detection is the first failure point

The most common way teams learn about a provider outage is a user complaint, not a monitoring alert — which means the incident has already been running for however long it took someone to notice and report it. Relying on the provider's own status page isn't much better: status pages are updated by humans, after they've confirmed an issue internally, which lags the actual start of degraded service.

What real-time health monitoring should catch

• Error rate by provider, tracked continuously — a jump from a normal 0.5% to 8% is a signal well before it's a full outage.
• Latency drift — a provider that's technically 'up' but responding 5x slower than normal degrades your product just as badly as a hard failure, and won't show as an error.
• Per-model granularity — an outage often affects one model tier, not the whole provider; treating 'OpenAI is down' as all-or-nothing misses cases where only one model is degraded.
• A history of uptime by provider — knowing that one provider has had three incidents this month versus another's zero is useful input for where to route traffic by default, not just during an active incident.

Building a fallback path before you need one

The teams that handle an outage smoothly aren't improvising a fallback in the moment — they've already decided, in advance, which secondary provider or model handles traffic when the primary is degraded, and that logic is already wired into the code path, not something someone has to deploy under pressure. That doesn't mean every feature needs multi-provider redundancy; it means the ones where downtime is genuinely costly should have a tested fallback, decided calmly, ahead of time.

What a fallback actually costs

Failing over to a secondary provider during an incident is very rarely the cheapest per-token option — that's not the point during an active outage. It's cheaper than the alternative, which is a broken user-facing feature. The cost conversation belongs afterward: reviewing how much the failover period cost against a normal period, and deciding whether the secondary provider is worth keeping warm for next time or was a one-off necessity.

Provider reliability isn't something any single company controls, which is exactly why it's worth monitoring like any other dependency — continuously, per provider, per model, with alerting fast enough to beat a user complaint. The outage itself is usually out of your hands. How long it affects your users is not.