Mandate 24-hour Sev-1 outage auto-escalation to estaff and AE

Situation

After the Fyr Fuzzball outage ran roughly 5 days without surfacing to leadership and the customer had to ping Horn directly to force escalation, Peter directed that any Sev-1 or customer outage open past 24 hours must automatically notify estaff and the account AE, built into tooling so it does not depend on a human judgment call. Ryan operationalized it as Zendesk automation plus CCing named-account AEs at ticket creation.

Reasoning

The technical recovery was handled competently; the real failure was that leadership and the AE learned about a multi-day Sev-1 from the customer rather than from internal channels. Peter wants escalation to be structural and automatic, not dependent on someone choosing to raise it. He framed it as a systemic fix and not blame (nobody is getting fired, I just want to understand the breakdown). He set a fixed 24-hour bound with no manual judgment because judgment is exactly what failed here.

Additional Context

Triggered by the Fyr/Fuzzball postgres cert expiry outage. Peter also asked Ryan for a COE and asked whether the rest of CIQ certs are at risk of expiring since the company just passed its one-year founding mark.

Observed Evidence

Peter in department-heads: Whatever we need to build to ensure that that happens should happen (re 24h estaff notification). Ryan back to Peter: Any Sev-1 open for 24 hours triggers automatic notification to the AE and estaff, no manual judgment, built into automation; AEs on named accounts CCd at ticket creation going forward.

Confidence Breakdown

Reasoning Depth Analysis