Helm and Kubernetes Deployments

This page covers troubleshooting specific to Anchore Enterprise deployments running on Kubernetes with the Helm chart.

Pods Restart Shortly After Starting

Symptom: One or more Anchore Enterprise pods repeatedly restart soon after starting — sometimes within about ten seconds — and kubectl get pods shows a climbing RESTARTS count or pods in CrashLoopBackOff. Running kubectl describe pod <pod-name> shows Liveness probe failed or Readiness probe failed events, and the container is killed and restarted.

Cause: The default liveness and readiness probe timings can be too aggressive for slower or resource-constrained clusters, or for services that legitimately take longer to become ready — for example during first boot, a database schema upgrade or migration, or a large vulnerability feed sync. When the liveness probe fails before the service has finished starting, Kubernetes kills and restarts the container before it can come up, producing a restart loop.

Fix: Increase the probe tolerances in your values file. The chart exposes a top-level probes block that applies to all Anchore Enterprise services. Raising initialDelaySeconds, timeoutSeconds, periodSeconds, and failureThreshold gives services more time to start and respond before Kubernetes considers them unhealthy:

probes:
  liveness:
    initialDelaySeconds: 120
    timeoutSeconds: 30
    periodSeconds: 30
    failureThreshold: 6
    successThreshold: 1
  readiness:
    initialDelaySeconds: 60
    timeoutSeconds: 30
    periodSeconds: 30
    failureThreshold: 3
    successThreshold: 1

Apply the change with helm upgrade, then watch the pods stabilize:

helm upgrade <release> -n <namespace> anchore/enterprise -f anchore_values.yaml
kubectl get pods -n <namespace> -w

Services Fail to Start After an Upgrade (Version Mismatch)

Symptom: After a Helm upgrade, one or more services fail to start and their logs report a database schema or version mismatch — the service code is a newer version than the database schema it is connecting to.

Cause: The chart upgrades the Anchore Enterprise database schema with a dedicated upgrade job (upgradeJob), which runs as a Helm hook during helm upgrade. If the upgrade job is disabled or did not complete, the database schema is never brought up to the new version, and the services refuse to start against a mismatched schema.

Fix: Ensure the upgrade job is enabled in your values file (it is enabled by default):

upgradeJob:
  enabled: true

upgradeJob.enabled must be true for the schema migration to run. With it disabled, helm upgrade only updates the deployment specs and images — it does not migrate the database — so the services come up against a mismatched schema and fail.

Within a single helm upgrade, the upgrade job retries the migration several times (approximately four attempts) before it gives up and the release fails. If the job fails, inspect its pod logs — not just the job object — to find the underlying cause, for example the database being unreachable or the credentials lacking the required permissions:

kubectl get jobs -n <namespace>
kubectl logs -n <namespace> job/<upgrade-job-name>          # streams the job's pod logs
kubectl logs -n <namespace> -l job-name=<upgrade-job-name>  # or target the pod(s) directly

After resolving the cause, delete the failed upgrade job (not just its pod) before re-running helm upgrade — a leftover job can block or conflict with the next attempt:

kubectl delete job <upgrade-job-name> -n <namespace>
helm upgrade <release> -n <namespace> anchore/enterprise -f anchore_values.yaml --timeout 30m

If the job failed, its logs will show why — for example, the database was unreachable or the credentials lacked the permissions needed to run the upgrade. Resolve the underlying issue and run the upgrade again.

helm upgrade Times Out During a v5 to v6 Migration

Symptom: helm upgrade returns an error such as timed out waiting for the condition a few minutes into a v5.x → v6.x migration, even though the upgrade job is still running.

Cause: helm upgrade waits for its hooks to complete and defaults to a 5-minute timeout. The v5.x → v6.x database schema upgrade and Legacy Imported SBOM migration can take considerably longer than five minutes on larger datasets, so Helm gives up and reports a failure while the underlying job is still working.

Fix: Raise the Helm timeout to a value that comfortably exceeds your expected migration time:

helm upgrade <release> -n <namespace> anchore/enterprise -f anchore_values.yaml --timeout 30m