Podman Health Checks: Monitor and Restart Unhealthy Containers

A running container is not always a healthy container. Podman can run a command inside the container namespace on a timer, record each result, and expose a small state machine: starting, healthy, or unhealthy. That state appears in podman ps, in podman inspect, and — when you wire Quadlet correctly — in systemd unit activation.

This guide walks through configuring probes at podman run time, reading health history, running a check on demand, handling slow startups, reacting to sustained failure, and connecting health to image builds and Quadlet services.

How Podman health checks work

Podman executes the probe inside the container network and mount namespace — the same environment your application sees. Each run records start time, end time, exit code, and optional output. Failures increment FailingStreak; enough consecutive failures after any grace window moves status to unhealthy.

Three statuses matter in daily operations:

podman ps appends the status in parentheses:

podman ps --filter name=hc-ok --format "table {{.Names}}\t{{.Status}}"

Sample output:

NAMES       STATUS
hc-ok       Up 15 seconds (healthy)

Nothing in this mechanism replaces application logging or external monitoring. Health checks answer a narrow question: “does this command exit 0 right now?” Design probes that fail when the service is unusable, not when an unrelated file is missing.

Configure a runtime health check

Attach a probe when the container starts with --health-cmd. The command runs through the container shell when you use shell syntax:

podman run -d --name hc-ok \
  --health-cmd "test -f /tmp/ready || (touch /tmp/ready && exit 1)" \
  --health-interval 5s --health-retries 3 --health-timeout 3s \
  docker.io/library/alpine:latest sleep 3600

The lab probe simulates a service that is not ready on the first tick: the first run creates /tmp/ready but exits 1; later runs find the file and exit 0.

After the interval elapses a few times, confirm status:

podman ps --filter name=hc-ok --format "table {{.Names}}\t{{.Status}}"

Sample output:

NAMES       STATUS
hc-ok       Up 15 seconds (healthy)

Read the persisted structure:

podman inspect --format '{{json .State.Health}}' hc-ok | python3 -m json.tool

Sample output (timestamps trimmed):

{
    "Status": "healthy",
    "FailingStreak": 0,
    "Log": [
        {
            "Start": "2026-08-23T08:49:37.385135917+05:30",
            "End": "2026-08-23T08:49:37.986785776+05:30",
            "ExitCode": 1,
            "Output": ""
        },
        {
            "Start": "2026-08-23T08:49:42.986785776+05:30",
            "End": "2026-08-23T08:49:43.173541054+05:30",
            "ExitCode": 0,
            "Output": ""
        },
        {
            "Start": "2026-08-23T08:49:48.173541054+05:30",
            "End": "2026-08-23T08:49:48.385135917+05:30",
            "ExitCode": 0,
            "Output": ""
        }
    ]
}

The first log entry shows the intentional failure; the following entries show success. FailingStreak reset to 0 once the probe passed.

Disable an image-level check at runtime with --no-healthcheck, or override it with a new --health-cmd. Pass --health-cmd none to clear a inherited check without adding a replacement.

Set interval, timeout, and retries

Timing flags control how aggressive Podman is about declaring failure:

Build a probe that always fails to see the unhealthy transition:

podman run -d --name hc-bad \
  --health-cmd "exit 1" \
  --health-interval 5s --health-retries 2 --health-timeout 2s \
  docker.io/library/alpine:latest sleep 3600

Wait for several intervals, then read status:

podman inspect --format '{{.State.Health.Status}}' hc-bad

Sample output:

unhealthy

Inspect the log:

podman inspect --format '{{json .State.Health}}' hc-bad | python3 -m json.tool

Sample output:

{
    "Status": "unhealthy",
    "FailingStreak": 4,
    "Log": [
        { "ExitCode": 1, "Output": "" },
        { "ExitCode": 1, "Output": "" },
        { "ExitCode": 1, "Output": "" },
        { "ExitCode": 1, "Output": "" }
    ]
}

FailingStreak can exceed --health-retries because Podman keeps scheduling probes after the container is already unhealthy. Use retries to tune how quickly the first unhealthy transition happens, not as a cap on later log entries.

Additional log controls from podman run --help:

• --health-log-destination — local is the default and stores health-check history in Podman's local container storage; a directory path or events_logger can also be selected.
• --health-max-log-count — default 5 entries (0 = unlimited)
• --health-max-log-size — default 500 bytes per entry (0 = unlimited)

For HTTP services, a common pattern is wget -qO- http://127.0.0.1/ || exit 1 or curl -sf http://127.0.0.1:8080/health. Use the address and port the process listens on inside the container, not the published host port.

Grace period with --health-start-period

Some applications need time to compile assets, run migrations, or warm caches before probes should count as failures. --health-start-period defines that grace window.

Start a container whose probe fails until you create a marker file:

podman run -d --name hc-start \
  --health-cmd "test -f /tmp/ready" \
  --health-interval 5s --health-retries 2 --health-start-period 30s \
  docker.io/library/alpine:latest sleep 3600

After 10 seconds the status is still in grace:

podman inspect --format '{{.State.Health.Status}}' hc-start

starting

After 35 seconds total — start period elapsed, file still missing — status remains starting until enough failures accumulate, then moves to unhealthy if the file never appears. In this lab run the status stayed starting through the first 35 seconds because failures during the start period do not immediately flip to unhealthy.

Create the readiness marker manually:

podman exec hc-start touch /tmp/ready

After one more probe interval:

podman inspect --format '{{.State.Health.Status}}' hc-start

healthy

During --health-start-period, failed probes keep status at starting rather than healthy. The container process keeps running; only the health label reflects incomplete readiness.

Startup health checks (--health-startup-cmd)

Regular --health-cmd probes run on the main interval even while an application is still booting. Startup checks use a separate command and timing model so you can probe more frequently early in life, then fall back to the standard interval.

Podman 5.8.2 exposes:

Run a slow-start container that creates /tmp/ready after eight seconds:

podman run -d --name hc-startup \
  --health-cmd "test -f /tmp/ready" \
  --health-startup-cmd "test -f /tmp/ready" \
  --health-startup-interval 3s --health-startup-retries 5 \
  --health-interval 10s --health-retries 2 \
  docker.io/library/alpine:latest sh -c 'sleep 8 && touch /tmp/ready && sleep 3600'

After 15 seconds inspect health:

podman inspect --format '{{json .State.Health}}' hc-startup | python3 -m json.tool

Sample output:

{
    "Status": "healthy",
    "FailingStreak": 0,
    "Log": [
        { "ExitCode": 1 },
        { "ExitCode": 1 },
        { "ExitCode": 1 },
        { "ExitCode": 0 }
    ]
}

Startup probes failed until the main process created /tmp/ready; the next startup probe succeeded and status became healthy. Use startup checks when boot is bursty; use --health-start-period when the same probe is valid but should not count failures for the first N seconds.

Check container health from the CLI

podman ps is the fastest human check — look for (healthy), (starting), or (unhealthy) in STATUS.

For automation, print only the status string:

podman inspect --format '{{.State.Health.Status}}' hc-ok

healthy

Parse the full JSON when scripts need log history or streak counts:

podman inspect --format '{{json .State.Health}}' hc-ok

Containers without any health configuration have no .State.Health object. podman inspect --format '{{.State.Health.Status}}' on such a container errors with a nil pointer — guard scripts with a format that tolerates absence, or check podman inspect --format '{{json .Config.Healthcheck}}' first.

Image-defined checks appear under .Config.Healthcheck after a Docker-format build (covered below). Runtime flags override image defaults.

Run a health check manually

podman healthcheck has one subcommand on Podman 5.8.2: run. It executes the configured probe immediately.

podman healthcheck run hc-ok

Healthy containers produce no stdout and exit 0:

echo $?

0

Failed probes print the status and exit 1:

podman healthcheck run hc-bad

unhealthy

The shell exit code is 1 after a failed probe:

echo $?

1

Use manual runs after changing application configuration, before marking a deployment complete, or when debugging a probe that behaves differently from an interactive podman exec shell.

React to failure with --health-on-failure

By default (--health-on-failure none), an unhealthy container keeps running. Operators see the label in podman ps but the main process is untouched. Set an explicit action when unattended operation should stop or recycle bad instances.

Allowed values on Podman 5.8.2: none, kill, stop, restart.

none (default)

The failing hc-bad container from earlier stayed running:

podman ps --filter name=hc-bad --format "table {{.Names}}\t{{.Status}}\t{{.State}}"

Sample output:

NAMES       STATUS                         STATE
hc-bad      Up About a minute (unhealthy)  running

kill

SIGKILL the main process when health becomes unhealthy:

podman run -d --name hc-kill \
  --health-cmd "exit 1" \
  --health-interval 5s --health-retries 2 --health-on-failure kill \
  docker.io/library/alpine:latest sleep 3600

After failures accumulate:

NAMES       STATUS                                  STATE
hc-kill     Exited (137) 20 seconds ago (unhealthy) exited

Exit code 137 indicates the process received SIGKILL (128 + 9).

stop

Stop the container cleanly when unhealthy:

podman run -d --name hc-stop \
  --health-cmd "exit 1" \
  --health-interval 5s --health-retries 2 --health-on-failure stop \
  docker.io/library/alpine:latest sleep 3600

On the lab host the container also ended with exit 137 after the stop path — treat exit codes as hints and confirm with podman inspect --format '{{.State.Status}} {{.State.ExitCode}}'.

restart

Restart the container when it becomes unhealthy:

podman run -d --name hc-restart \
  --health-cmd "exit 1" \
  --health-interval 5s --health-retries 2 --health-on-failure restart \
  docker.io/library/alpine:latest sleep 3600

With a probe that never succeeds, Podman entered a restart loop — RestartCount climbed to 3 and status returned to starting after each restart. Do not combine --health-on-failure restart with aggressive --restart policies unless you intend continuous recycling.

Pick kill or stop when a supervisor outside Podman should notice the exited unit. Pick restart only when a fresh container might recover. Most production setups pair none with external orchestration or systemd restart policies instead.

Embed checks in a Containerfile

The HEALTHCHECK instruction documents the intended probe in the image itself:

FROM docker.io/library/alpine:latest
HEALTHCHECK --interval=5s --timeout=3s --retries=2 \
  CMD test -f /tmp/ready || exit 1
CMD ["sleep", "3600"]

Build with default OCI output and Podman warns:

podman build -t hc-buildtest:oci -f Containerfile .

Sample output:

warning msg="HEALTHCHECK is not supported for OCI image format and will be ignored. Must use `docker` format"

A container from that image has no runtime health state — podman inspect --format '{{json .Config.Healthcheck}}' returns null.

Build with Docker image format so metadata persists:

podman build --format docker -t hc-buildtest:docker -f Containerfile .

Run without overriding health flags, wait for probes, and inspect:

podman inspect --format '{{json .Config.Healthcheck}}' hc-buildtest

Sample output:

{
    "Test": ["CMD-SHELL", "test -f /tmp/ready || exit 1"],
    "Interval": 5000000000,
    "Timeout": 3000000000,
    "Retries": 2
}

Durations are nanoseconds in inspect JSON. Override or disable at podman run with --health-cmd or --no-healthcheck when a single image serves multiple deployment profiles.

Quadlet health checks and Notify=healthy

Quadlet maps health flags into [Container] directives — see Podman Quadlet container file for the full task reference. A minimal pattern:

[Container]
Image=docker.io/library/nginx:latest
HealthCmd=/bin/sh -c "curl -s -o /dev/null http://127.0.0.1:80/"
HealthInterval=30s
HealthTimeout=5s
HealthRetries=3

Use the port inside the container namespace, not the host port from PublishPort.

Systemd normally considers a Type=notify service active when conmon reports container start. That is earlier than application readiness. Set:

Notify=healthy

Podman adds --sdnotify=healthy to the generated podman run line. The unit stays in activating until .State.Health.Status is healthy. Downstream units with After= or Requires= on that service then start only after the probe passes.

That pattern matters for Podman auto-update with Quadlet: rollback logic needs systemd to see a failed restart when a new image does not pass health. Notify=healthy is a practical substitute when the application does not speak sdnotify itself.

Run the same probe manually on a Quadlet-managed container:

podman healthcheck run web

healthy

Quadlet syntax details, HealthOnFailure, and pod-level ordering belong in the Quadlet articles — this guide owns the probe semantics and state fields those units rely on.

Choose probes that match the failure mode

Effective health checks are cheap, idempotent, and tied to user-visible behavior.

• Prefer an HTTP endpoint or socket check that exercises the real code path over pidof alone.
• Keep interval and timeout aligned with SLOs — a 30s interval means up to 30s before the first failure after an outage.
• Set --health-start-period or --health-startup-cmd for slow boots so deploys do not flap to unhealthy during migration.
• Use --health-on-failure deliberately; an infinite restart loop on a bad image wastes CPU and hides the root cause.
• For OCI-format images, pass runtime --health-cmd or build with --format docker so checks actually exist.
• Wire Notify=healthy when systemd ordering or auto-update rollback should wait for readiness, not just a running PID.

When health state disagrees with application logs, compare podman inspect log exit codes with a manual podman exec of the same probe command — SELinux, missing binaries, and wrong ports inside the probe are the usual culprits.

Cleanup

Remove lab containers when finished:

podman rm -f hc-ok hc-bad hc-start hc-startup hc-kill hc-stop hc-restart 2>/dev/null

Alpine sleep containers sometimes need SIGKILL after podman rm -f times out — that is normal on busy hosts.

References

• podman-run(1) — health check flags and --health-on-failure
• podman-healthcheck(1) — manual run subcommand
• podman-inspect(1) — .State.Health and .Config.Healthcheck
• podman-systemd.unit(5) — Quadlet HealthCmd, HealthInterval, Notify=
• Podman Containerfile — HEALTHCHECK instruction
• Podman Quadlet container file — health directives and Notify=healthy