What SRE is (and how it relates to DevOps)

What SRE is

Ben Treynor Sloss, who founded the first SRE team at Google, described it as what happens when you ask a software engineer to design an operations function. Traditional operations scales with headcount: more services and more traffic mean more people performing more manual procedures. SRE refuses that model. Work that a machine can do is automated, and engineers spend their time making the system need less human attention.

In practice an SRE's work falls into two kinds. Operational work is being on call, responding to incidents, handling tickets and doing manual releases. Engineering work is writing software and designing systems that improve reliability or remove future operational work: automation, better monitoring, capacity models, safer rollout tooling. A defining SRE rule is that operational work is capped at about half of a team's time, so that there is always capacity to fix the causes and not only the symptoms. When operations exceed the cap, that is treated as a signal that something is wrong, not as the normal state.

The other defining feature is that reliability is expressed as numbers agreed in advance, not as an aspiration. "The site should be fast and always up" cannot be engineered. "99.9% of checkout requests succeed within 500 ms, measured over 28 days" can, and it tells you when to stop as well as when to act.

SRE and DevOps

The two grew up separately around the same time and address the same root problem: the wall between the people who change systems and the people who keep them stable. Developers are rewarded for shipping, operators for stability, and because most outages are caused by change, the two incentives collide.

DevOps is a broad cultural and professional movement. It describes outcomes and values: break down silos, accept failure as normal, change gradually, automate, and measure everything. It is deliberately not prescriptive about how. SRE is one specific, opinionated way of doing it. Google's own summary is that SRE implements DevOps, in the way a class implements an interface.

If you have worked through the DevOps track, you have the delivery side: pipelines, infrastructure as code, Kubernetes, GitOps. SRE picks up where that ends, with the questions of how reliable the running system should be, how you know, and what you do when it is not.

Reliability is a feature, and it has a price

Reliability is arguably the most important feature of any product, because no other feature works when the system is down. Users do not distinguish between "the payment feature has a bug" and "the payment service is unavailable"; both mean they could not pay.

But reliability is not free, and it is not unlimited. Each additional "nine" of availability costs far more than the last: redundancy across zones, then regions, slower and more careful releases, more testing, more people on call. At some point the extra reliability is invisible to users, because their own phone, Wi-Fi and network are less reliable than your service. Money and engineering time spent beyond that point would have produced more value as features.

Look at what the table implies for how you operate. At three nines a human can be paged, wake up, and fix a problem within the monthly allowance. At four nines, four minutes a month leaves no time for a human at all: detection and recovery must be automatic. The target you choose determines the architecture, the process and the cost, which is why choosing it is a business decision and not a purely technical one.

def allowed_downtime_minutes(slo_percent: float, days: int = 30) -> float:
    """Minutes of full outage permitted by an availability target."""
    total_minutes = days * 24 * 60
    return total_minutes * (1 - slo_percent / 100)


for slo in (99.0, 99.9, 99.95, 99.99):
    print(f"{slo}% -> {allowed_downtime_minutes(slo):.1f} minutes per 30 days")
# 99.0% -> 432.0, 99.9% -> 43.2, 99.95% -> 21.6, 99.99% -> 4.3

The core principles

The rest of this track expands each of these. Together they form a system: each one makes the others work.

• Service level objectives. Define reliability as a measurable target for what users experience. Everything else hangs from this.
• Error budgets. The gap between the target and 100% is a budget that may be spent on releases, experiments and risk. While budget remains, ship. When it is gone, the priority shifts to reliability. This replaces arguments between development and operations with a shared number.
• Eliminating toil. Manual, repetitive, automatable work is measured and engineered away, so that operational load does not grow with the service.
• Monitoring and alerting on symptoms. Page a human only when users are affected and a human needs to act. Everything else is a ticket or a dashboard.
• Release engineering. Most outages are caused by changes, so changes are made small, gradual, observable and reversible.
• Blameless postmortems. Incidents are studied for systemic causes, without blaming individuals, so that people report problems honestly and the organisation learns.
• Simplicity. Every line of code and every component is a liability as well as an asset. Boring, well-understood systems are easier to keep reliable.

Team models and shared ownership

There is no single correct way to organise SRE. Common models include the following.

Whatever the structure, one rule prevents SRE from turning back into traditional operations: ownership is shared, and support is conditional. Developers stay involved in production, commonly by taking part in the on-call rotation and handling a share of tickets. A service must meet a bar before an SRE team takes it on, checked in a production readiness review. And if a service keeps exceeding its error budget or generating excessive operational load, SRE can hand the pager back to the developers until it is fixed. That last mechanism is what keeps the incentive to build reliable software where it belongs, with the people who write it.

For a small company, none of this needs a team with SRE in its name. Two or three engineers can define SLOs for their main user journeys, put symptom-based alerts on them, run blameless reviews after incidents, and track how much time goes on repetitive operations. That is SRE.