Skip to main content

Managed Updates

While many Teleport resources support agentless mode, agent deployments are sometimes simpler and more convenient. However, large Teleport deployments can create an additional burden: updating all agents.

There are two versions of Teleport Agent Managed Updates:

  • Version 1: Supports systemd-based Linux distributions using apt, yum or zypper package managers, as well as Kubernetes clusters.
  • Version 2: Removes the package manager dependency from v1, supports a wider range of setups (e.g., Teleport Community Edition, Teleport Enterprise, FIPS, and multiple agents on a single host) and offers more control over the update schedule.

Update logic and failure modes​

An updater is a piece of software deployed next to an agent which is responsible for updating it. Updating multiple agents requires multiple updaters.

We designed the updater to be as decoupled from Teleport as possible. The updater can update agents even when they cannot join the Teleport cluster. Pushing a broken version can happen, but a rollback/roll-forward must always be possible without manually connecting to the resource and fixing the agent.

The updater recurrently fetches the target version from a version server and updates the agent to the target version. Because restarting the agent can disrupt currently open sessions, it will only update the agent in two cases: during a maintenance window or when the agent is unhealthy.

Version 2 update logic​

In Managed Updates version 2, the Teleport cluster itself drives the updates by telling each agent if it should update or not. Centralizing the update decision offers more control over the rollout. As the Teleport cluster sees every agent, it can more complex decisions such as gradually rolling out an update, or detecting a faulty update.

In version 2, the updater will monitor the agent after an update. If the agent fails to come back healthy, the updater will automatically initiate a rollback to the last working version in an attempt to restore connectivity.

Version 1 update logic​

In Managed Updates version 1, the updater drives the update, and Teleport is only responsible for providing it the version schedule.

When a maintenance schedule is available, the updater will honor it. However, if the updater cannot find the maintenance schedule, it will consider the agent unhealthy and perform updates as soon as possible. Similarly, if the updater detects the agent is unhealthy, it immediately applies any pending update to try to recover from a degraded state.

We implemented an additional failsafe: the critical maintenance toggle. The version server can specify that an update is critical. Critical updates are applied even if the updater is outside its regular maintenance window.

Security​

Version 2 security invariants​

The version 2 updater fetches binaries from its CDN over TLS and validates integrity via a shasum. The agent uses the official Teleport CDN by default. A custom CDN can be configured at the updater level. For security reasons, there are no way to remotely reconfigure the updater CDN and have it install arbitrary binaries.

Binaries are currently not signed, this might change in the future. OCI artifacts (Docker images) are signed using cosign's signature format .

Version 1 security invariants​

When updating the agent, the updater will ensure the new version's authenticity before deploying it. On Linux distributions using apt, yum or zypper, it relies on the existing package signature system. On Kubernetes-based environments, it validates the OCI image signature (using cosign's signature ).

Version server and source of truth​

The agent version is subject to the following constraints:

  • the agent must never exceed the Proxy or Auth Service version,
  • the agent must always be no more than one major version below the Proxy or Auth Service version.

The best practice is to always align the agent version with the version of the Proxy Service and Auth Service. To upgrade the Auth Service and Proxy Service, follow the Teleport Cluster upgrade guide.

Updaters are discovering the version they should run by querying the Teleport proxy.

Teleport Cloud​

If you are using Teleport Cloud the target version is managed for you. In Managed Updates version 1, you can configure the maintenance window. In Managed Updates version 2, you can configure custom update schedules and control when and in which order agents are updated.

Self-hosted Teleport​

If you are self-hosting Teleport Enterprise you can set up automatic agent updates. You are responsible for configuring the desired agent version. You must also monitor the agent's health and rollout status to ensure agents are healthy and updated properly.

Next steps​

Configure automatic updates v2.

After that, you can set enroll agents in automatic updates as part of the upgrading procedure.