Downtime & Timeline

Downtimes in the Polycrate API record availability incidents of monitored objects and are visualized with a timeline graph and activity feed; this page covers the downtime object, timeline events, sources and SLO impact.

Downtime & Timeline

Overview

A Downtime in the Polycrate API is a completed or ongoing event during which one or more objects (e.g. K8sApps, endpoints, hosts, K8sClusters) were unavailable or only partially available. Downtimes are generated automatically from endpoint/heartbeat monitoring or created manually (e.g. for planned maintenance).

Since Polycrate API 0.15.0, downtimes are no longer visualized only as a table but with a timeline graph that shows, across the entire downtime, how many objects were affected at any given point in time.

Downtime Object

Field Description
Name Descriptive title (e.g. "Network outage DC Frankfurt")
Start / End Period of the downtime (End=null → ongoing)
Timeline Structured log of the events within the downtime
Affected Objects Number and concrete list of affected objects
Severity info, warning, critical
Workspace / Organization Assignment

Timeline Events

The Timeline is a JSON field that records events chronologically:

  • object_affected (object fails)
  • object_recovered (object recovers)
  • note (free-form descriptive event entry)
  • escalation, root_cause_identified etc.

Each event has a timestamp, type, payload and optionally a link to the affected ManagedObject.

Timeline Graph

The detail view of a downtime contains a horizontal time-series graph:

  • X axis: Time (start to end or now)
  • Y axis: Number of simultaneously affected objects
  • Annotations: Visual markers on the time axis for timeline events
  • Hover: Event details with object names

The graph replaces the former simple HTML list and makes the following visible at a glance:

  • When was the peak of the downtime?
  • Were there recovery phases (dips)?
  • How is an ongoing incident developing?

Technically, the graph is based on the Metric Functions Framework – the Downtime provides a synthetic step function that is rendered like any other metric.

Activity Timeline

In addition to the metrics graph, the Activity tab shows a chronological feed of all events related to a downtime or maintenance – populated automatically from notifications, status changes and manual entries. Concept and activity kinds: Activity Timeline.

What Appears in the Activity Timeline?

Event Source
Downtime opened / closed Endpoint monitoring
Maintenance planned / started / completed Maintenance lifecycle
Notification sent Notification system (Mattermost, email, …)
Note created / answered User
Affected object added K8sApp, endpoint, …

Workspace Timeline

Every workspace also has a Timeline tab that aggregates all events of the workspace (across all objects) in one feed – useful for an overview of the activity of an environment.

Pagination & API

By default, the timeline loads the last 20 entries. Older entries are loaded via "Load more". The underlying endpoint:

GET /api/v1/activities/?object_type=downtime&object_id={id}&page_size=20

Where Do Downtimes Come From?

Source Behavior
Endpoint Check Failure Several consecutive failed checks open a downtime, recovery closes it
Heartbeat Missing Missing heartbeat from an agent/host
Manual Creation An administrator creates a downtime for planned maintenance (severity info)
Aggregation Downtimes with the same cause/time can be bundled into a collective downtime

Maintenance vs. Downtime

Polycrate distinguishes three kinds of appointments/events that are visible together in the timeline:

  • Maintenance — a planned event with start/end (e.g. "Upgrade on Sunday 02:00–04:00"). Proactively informs users and, if applicable, suppresses alerts during the announced period.
  • Maintenance Window — a recurring policy ("every Sunday 02:00–04:00"). Does not generate events itself, but defines the rules within which a maintenance should be scheduled.
  • Downtimes — availability problems that actually occurred (past/now), generated automatically or recorded manually afterwards.

If an outage actually occurs during a maintenance, it can be recorded as a downtime within the maintenance slot.

Where Do I See Downtimes?

  • Monitoring → Downtimes – global overview with filters by severity, status, scope
  • Object detail (e.g. K8sApp) → Downtimes tab – only downtimes affecting this object
  • Dashboard – current/latest downtimes as a widget

SLO & Availability

Downtimes feed into the availability/SLO calculation:

  • Per object, available_time / total_time is calculated (minus planned maintenance)
  • Aggregation over periods (last 30/90 days) for reports
  • Usable in Audit & Compliance contexts (availability reports)

See Also