Official vs observed: when status pages admit an outage

Status page honesty, last 90 days

Delay = when the vendor first acknowledged the problem, minus our first failed direct check. Negative = the status page was first. When the vendor posted an incident, we use the time it was posted from the vendor's own incident feed (incidents.json), which is exact; our side is a check every 5 minutes, so the outage may have begun up to 5 minutes before our first failed check. Without a posted incident, the delay comes from our 5-minute polls of the status page and is accurate to about ±5 minutes. A service with 0 episodes had no confirmed failure of our direct check in the window.

Rating (A–F) appears only for services with 3 or more compared episodes. Each episode scores 4 if the status page showed the problem within 5 minutes of our first failed check (or before it), 3 within 15, 2 within 30, 1 within 60, and 0 if later or never; A is an average of 3.5 or more, B 2.5, C 1.5, D 0.5, F below. It rates how fast the vendor admitted the outages that our single location in Germany saw, not the vendor's reliability, and a regional problem the vendor rightly left off its page still counts against it.

Episodes

Methodology

• Observed. One plain HTTPS GET every 5 minutes to an API or site run by the service itself, from one checker location: Vigil's server at Hetzner in Falkenstein, Germany. A request that times out or cannot connect is retried once before it counts as failed. What each probe requests is listed on every episode page.
• Official. Every 5 minutes we read the overall indicator on the vendor's own status page (Atlassian Statuspage-style status.json). Minor, major and critical count as "showing a problem"; maintenance does not.
• Episode. 2 or more consecutive failed direct checks, from the first failed check to the first passing one. A single failure is a blip and is ignored. Episodes still in progress appear once they end.
• Our side, not theirs. If 3 or more other services failed within 10 minutes of the start, the checker itself (network, DNS, host) is the likelier cause, and the episode is not published.
• Acknowledgement. The earliest of: an incident the vendor posted (its created_at in the status page's public incidents.json) and the first status-page poll that shows a problem, from 30 minutes before the episode to 60 minutes after it ended. Some vendors backdate an incident's "started" time; we use the time it was posted. Nothing in that window = "never acknowledged". If the vendor's page could not be read, or has shown one standing incident for most of a month, there is no comparison.
• What this cannot tell you. A failed request from Germany is not proof that every user was affected, and a status page that stays green may be right if the problem was regional. Pages for services without a machine-readable status page are kept out of search engines.