{% extends "base.html" %} {% block title %}Alert channels{% endblock %} {% block content %}

Alert channels

Where should alerts go? Add one or more channels, then pick them per monitor. Every alert carries the monitor name, what happened and a link back here.

Your channels {{ channels|length }}

{% if channels %}
{% for c in channels %}
{{ icons.get(c.kind, '◈') }}
{{ c.name }}{{ labels.get(c.kind, c.kind) }}
{{ target(c.kind, c.config) }}
{% endfor %}

A test message says it's a test and names this workspace ({{ ws_email }}), so it's safe to send to shared rooms.

{% else %}

No channels yet

Add one on the right. Until then, alerts only show on the dashboard{% if smtp_enabled %} and go to {{ ws_email }} by email{% endif %}.

{% endif %}

How alerts look

[{{ app_name }}] DOWN: Marketing sitedown
https://example.com returned HTTP 503 after 2 failed checks (latency 1240 ms).
View: {{ base_url }}/app/monitors/12
[{{ app_name }}] UP: Marketing siteup
Recovered after 4m 30s of downtime.

Chat apps get a rich card (bold title, red/green colour, open-monitor link); push services get an urgent-priority push for outages; webhooks get JSON with event, monitor, title, message, check, diff, link; PagerDuty, Opsgenie, ServiceNow, Jira and Zendesk get a ticket that opens on the outage and resolves on recovery; Splunk, Datadog and New Relic receive every event.

Add a channel

{% for gtitle, gmeta in kind_groups %} {% if not loop.first %}
{{ gtitle }}
{% endif %} {% for k, label, ic, blurb in gmeta %} {% endfor %} {% endfor %}
{# ---- email #}
{% if not smtp_enabled %}
Email delivery isn't configured on this server yet.
{% endif %}
{# ---- URL-based chat webhooks #}
Slack: Apps → Incoming Webhooks → Add to Slack, choose the channel and copy the https://hooks.slack.com/services/… URL. Alerts arrive as a Block Kit card with a red/green bar.
Discord: channel → Edit channel → Integrations → Webhooks → New webhook → Copy URL. Alerts arrive as a coloured embed with a link to the monitor.
Teams: in the channel choose … → Workflows → “Post to a channel when a webhook request is received”, finish the wizard and copy the URL (looks like https://….logic.azure.com/…). Legacy Incoming Webhook connector URLs also work. Alerts arrive as an Adaptive Card.
Google Chat: open the space → space name → Apps & integrations → Webhooks → Add webhook, name it “{{ app_name }}” and copy the https://chat.googleapis.com/v1/spaces/… URL. Requires a Google Workspace account.
Any HTTPS endpoint. By default we send a JSON body: event, monitor{id,name,url,mode}, title, message, check{status_code,latency_ms,…}, diff, at, link.
{# ---- webhook extras #}
One Name: value per line, or a JSON object.
Leave empty to get the default JSON. Placeholders: {{ '{{' }}title{{ '}}' }} {{ '{{' }}message{{ '}}' }} {{ '{{' }}event{{ '}}' }} {{ '{{' }}monitor_name{{ '}}' }} {{ '{{' }}monitor_url{{ '}}' }} {{ '{{' }}link{{ '}}' }} {{ '{{' }}status_code{{ '}}' }} {{ '{{' }}latency_ms{{ '}}' }}. If the template starts with { values are JSON-escaped, so it stays valid JSON.
When set, every request carries X-Vigil-Signature: sha256=<HMAC-SHA256 of the raw body> so your endpoint can verify it came from {{ app_name }}.
{# ---- telegram #}
Message @BotFather in Telegram → /newbot → copy the token.
Send your bot a message (or add it to the group), then open https://api.telegram.org/bot<token>/getUpdates and copy chat.id. Groups have negative ids; public channels can use @channelname.
Only for forum-style groups with topics: the message_thread_id shown in the topic's link.
{# ---- ntfy #}
Leave as https://ntfy.sh or point at your self-hosted ntfy.
Topics are public on ntfy.sh: pick something unguessable like vigil-<random>, then subscribe to it in the ntfy app. Outages are sent with urgent priority.
Only for protected topics / self-hosted servers with auth.
{# ---- gotify #}
In Gotify go to Apps → Create application (call it “{{ app_name }}”) and copy its token. Not a client token.
{# ---- pushover #}
The 30-character key at the top of your pushover.net dashboard.
Create an application at pushover.net/apps/build and copy its API token. Outages are sent as high priority (bypasses quiet hours).
{# ---- matrix #}
Room Settings → Advanced → Internal room ID. Invite the bot account to the room first.
Best from a dedicated bot account: Element → Settings → Help & About → Advanced → Access Token.
{# ---- incident management: shared explanation #}
Outage-type alerts (down, escalation, health, dependency, ssl, domain, dns, security, error) open an item; the monitor's up alert resolves it. Everything is correlated by the key vigil-monitor-<id>, so re-opening a channel or restarting {{ app_name }} never loses track. Send test only checks the credentials: it never opens a real ticket.
{# ---- pagerduty #}
In PagerDuty open the Service → Integrations → Add integration → “Events API V2” and copy the Integration Key. Outages trigger a critical incident (other problems warning), recovery resolves it via the dedup key. Informational alerts (content/price/keyword change, slow, reports) are sent as non-paging change events on the service timeline. The test button triggers and immediately resolves a throwaway vigil-test incident (severity info).
{# ---- opsgenie #}
In Opsgenie open Teams → your team → Integrations → Add integration → “API”, allow create/close alerts and copy the API key. Outages create a P1 alert, other problems P3, informational events P5 (de-duplicated per monitor + event); recovery closes the alert by alias.
Comma-separated team names to notify. Leave empty to use the integration's own routing.
{# ---- servicenow #}
For https://acme.service-now.com enter acme.
A dedicated integration user with the itil role (User Administration → Users → New → Roles → itil), so it can create and resolve incidents through the Table API.
Group name or sys_id. Outages open with urgency 1 / impact 2, other problems 3 / 3, category Network; recovery sets state Resolved with close code “Solved (Permanently)”. Informational events are not turned into incidents.
Defaults to incident; use another table that has the same fields if your process needs it.
{# ---- jira #}
Create one at id.atlassian.com → Security → Create API token for the account above (ideally a bot account that can create issues in the project).
Defaults to Incident; if the project doesn't have it we fall back to Task. Issues are labelled vigil and vigil-monitor-<id>; on recovery every open issue with that label gets a “Resolved by {{ app_name }}” comment and is transitioned to the first Done status. Informational events are not ticketed.
{# ---- zendesk #}
For https://acme.zendesk.com enter acme.
Admin Center → Apps and integrations → APIs → Zendesk API: enable token access and add a token. Outages open an urgent ticket (other problems normal) tagged vigil with external id vigil-monitor-<id>; recovery marks matching unsolved tickets solved with an internal note. Informational events are not ticketed.
{# ---- splunk #}
Host and port of the HTTP Event Collector (Splunk Cloud: https://http-inputs-<stack>.splunkcloud.com). We POST to /services/collector/event.
Settings → Data inputs → HTTP Event Collector → New token; make sure HEC is enabled in Global Settings. Every alert (including recoveries and informational events) is sent as one event.
{# ---- datadog #}
Organization settings → API keys → New key (an API key, not an application key). Every alert becomes an event: error for outages, success for recovery, warning for slow, info otherwise, tagged source:vigil monitor:<id> event:<event> plus the monitor's tags, aggregated by vigil-monitor-<id>.
The site shown in your Datadog URL.
{# ---- newrelic #}
API keys → Create a key → Ingest - License. Every alert is sent to the Logs API with the full payload flattened into attributes (monitor.name, check.status_code, event, …), logtype: vigil.alert.
{% endblock %}