Why alert on burn rate instead of error rate
A fixed threshold such as "page when errors exceed 1%" is wrong in both directions. It wakes someone for a two-minute blip that cost almost nothing, and it stays silent through a slow leak of 0.3% errors that quietly eats the whole month's budget. Burn rate fixes this by asking a different question: at the current pace, how quickly will the error budget run out?
A burn rate of 1× spends the budget exactly over the SLO window. At 14.4× a 30-day budget is gone in about two days, and 2% of it disappears in a single hour — clearly worth a page. At 1× over three days, 10% is gone: worth a ticket, not a phone call at night.
The long window
Decides whether the problem is significant. A 1-hour window with a 14.4× rate means a real share of the budget is already spent.
The short window
Decides whether it is still happening. Requiring the same rate over the last 5 minutes makes the alert clear soon after recovery instead of ringing for an hour.
Fast burn → page
2% of the budget in 1 hour, or 5% in 6 hours. Someone needs to act now, whatever the time.
Slow burn → ticket
10% of the budget in 3 days. A persistent, low-level problem to fix during working hours.
Using the generated rules
The rules assume a counter named http_requests_total with a code label. Replace the two selectors with your own good and total event queries — the thresholds stay the same. If you record the error ratios as recording rules first, the alert expressions become shorter and cheaper to evaluate.
Frequently asked questions
What is a burn rate?
The speed at which a service consumes its error budget, relative to the speed that would use it up exactly at the end of the SLO window. Burn rate = observed error rate ÷ (1 − SLO).
Where do 14.4, 6 and 1 come from?
They are the burn rates that spend 2% of a 30-day budget in 1 hour, 5% in 6 hours and 10% in 3 days. For a different window length the calculator recomputes them.
Why two windows per alert?
The long window gives significance and the short window gives a fast reset. Without the short window an alert keeps firing long after the incident is over, because the long average takes time to drop.
Will these alerts work for a low-traffic service?
With very few requests a single failure can exceed the threshold. Add a minimum request-rate condition to the expression, or use synthetic checks to create a steady signal.