Alerting sla

SLO and SLI for Mobile

Instructions

An SLO is a promise with teeth: cross it, and releases stop until the budget is recovered. Mobile SLOs are per-release and per-platform; global averages hide regressions.

1. Pick the Right SLIs

From the core mobile RUM set (see mobile-metrics), pick 4-6 SLIs as the SLIs the team is accountable for. A good minimum:

• Crash-free sessions.
• Crash-free users.
• ANR rate (Android) / App hang rate (iOS).
• Cold start p95.
• Network error rate on critical endpoints.
• Checkout (or other primary) success rate.

Resist the urge to add more. Every SLO is a commitment to page on it.

2. Define Each SLO

SLI	SLO	Window	Error budget
Crash-free sessions	>= 99.5%	30 days	0.5%
Crash-free users	>= 99.8%	30 days	0.2%
ANR rate	<= 0.1% of DAU	7 days	0.1%
App hang rate	<= 0.5% of sessions	7 days	0.5%
Cold start p95 (mid-tier)	<= 2500 ms Android / 1800 ms iOS	7 days	per-release
Checkout success rate	>= 98%	7 days	2%

Document each SLO in a versioned slo.yaml, including: SLI query, target, window, owner team, runbook link.

3. Error Budgets

Error budget = 1 - SLO. Measured as cumulative consumption over the window.

• If the rolling budget is > 50% remaining: green, normal release velocity.
• 20-50% remaining: yellow, require release review; feature flag new risky changes.
• < 20%: red, release freeze for non-critical changes; fixes only.
• < 0%: paging on-call, postmortem required.

4. Multi-Window Burn-Rate Alerts

Each SLO needs two alerts:

• Fast burn: short window (5-30 min) consuming budget at 14.4x or more ('2% of 30-day budget in 1 hour'). Pages the on-call.
• Slow burn: long window (6 h) consuming budget at 3x-6x. Opens a ticket during business hours.

- name: crash_free_sessions_fast_burn
  severity: page
  expr: (1 - crash_free_sessions_ratio_5m{release="current"}) > 14.4 * 0.005
  for: 5m
- name: crash_free_sessions_slow_burn
  severity: ticket
  expr: (1 - crash_free_sessions_ratio_6h{release="current"}) > 3 * 0.005
  for: 1h

5. Rollout Halts

Integrate SLOs with the release pipeline:

• Play Store staged rollouts (1% -> 5% -> 20% -> 50% -> 100%): each promotion is gated by the previous stage's SLOs meeting target over a 24-hour window.
• App Store phased release: same logic; if the first 24 h SLOs regress, halt via App Store Connect.
• Feature flags: any new feature behind a flag is auto-disabled when the flag's cohort violates its SLO.

# pseudo-code for the release gate
def can_promote(release: str, stage: str) -> bool:
    return (
        crash_free_sessions(release, stage) >= 0.995 and
        anr_rate(release, stage) <= 0.001 and
        cold_start_p95(release, stage) <= threshold_for(stage)
    )

6. Per-Release, Per-Platform SLOs

• SLOs are measured per release because mobile has long-tail versions in the wild.
• A release in < 1% adoption is not yet a reliable signal; set a minimum adoption floor (e.g. 10k sessions) before its SLO starts counting.
• Aggregate SLOs across releases are for company reporting, not for rollout gating.

7. Error Budget Policy

A written policy should say:

• Who declares a freeze (typically the mobile engineering lead).
• What counts as "critical" during a freeze (security, SLO fix, compliance).
• How budgets recover: recovery is automatic by time windowing, but you can request a budget exception for a large planned release with stakeholder sign-off.
• How postmortems feed back: every budget breach requires a postmortem with action items tracked to completion.

8. Reporting

• Weekly: per-platform SLO dashboard showing budget remaining and burn rate per SLO.
• Monthly: executive summary of budget consumption trend.
• Quarterly: review SLO thresholds. Tightening an SLO that has been consistently green is expected; loosening an SLO requires written justification.

9. Anti-Patterns

• SLO on metrics that users do not feel (CPU, memory, requests per second).
• "Aspirational" SLOs no one actually enforces.
• Global SLOs that average across releases so regressions in the newest release are invisible.
• Page-on-every-miss, which trains the team to ignore alerts.

10. Example: Tying It Together

• checkout_success_rate SLO is 98% over 7 days.
• The current release has 18% adoption, meets the adoption floor, ratio is 97.2%, error budget consumed at 140%.
• Slow-burn alert fires, ticket opens.
• Feature-flag on the new payment UI is auto-disabled; release pipeline refuses to promote to next stage.
• Postmortem is scheduled; once the flag is fixed and a hotfix release is above 99% over 24 h, rollout resumes.

Checklist

• 4-6 SLIs chosen; each has SLO, window, owner, and runbook recorded in slo.yaml.
• Error budgets are defined and their thresholds trigger documented behavior (release freeze, flag disable).
• Multi-window burn-rate alerts exist per SLO (fast-burn pages, slow-burn tickets).
• Release pipeline (Play staged rollout, App Store phased release) is gated on per-stage SLOs.
• Per-release measurement with an adoption floor is configured.
• Weekly dashboard and monthly/quarterly reviews are scheduled.
• Error Budget Policy is written, signed off, and linked from runbooks.
• Postmortem loop feeds action items back into the SLO targets.