Dynamiq
Self-Hosted Deployment

Operations & Troubleshooting

Day-two operations for self-hosted Dynamiq: health checks, logs, scaling, backup and restore, and fixes for the failures you'll actually hit.

Once the platform is installed, most of running it is checking health, scaling to your workload, backing up state, and diagnosing the occasional failed pod. This page collects those day-two tasks for the canonical install — release dynamiq in namespace dynamiq. It assumes you've been through Install on Kubernetes (Helm); the Troubleshooting section links back to the exact install steps a failure traces to.

Health and logs

The chart runs five Deployments in the release namespace. In a healthy install every pod is Ready:

kubectl -n dynamiq get pods

You should see one pod per Deployment at steady state — nexus, synapse, catalyst, runtime, and ui — all Running and Ready (more per service if you've raised replicas or enabled autoscaling).

Each backend service serves a liveness and readiness probe on /health at container port 8080; the ui probes / on its http port instead. Hit the public API health endpoint from outside the cluster the same way the install verifies it:

curl https://api.dynamiq.example.com/health

For logs, address each Deployment by name:

kubectl -n dynamiq logs deploy/nexus          # or synapse, catalyst, runtime, ui
kubectl -n dynamiq logs deploy/catalyst -f    # follow live
kubectl -n dynamiq logs deploy/nexus --previous   # the crashed container, after a restart

nexus is the place to start for platform-level problems — it owns authentication, projects, and scheduling — while catalyst and runtime carry the execution workloads.

Scaling

Every service block accepts replicaCount and an autoscaling block. Set a static count, or turn on autoscaling and let an HPA own the replica count between minReplicas and maxReplicas — the two are mutually exclusive per service. The full pattern (requests, limits, HPA targets, scheduling) is in Sizing, scheduling, autoscaling.

  • All five services scale horizontally. They're stateless — state lives in Postgres, S3, and NATS — so you can run several replicas of each. Keep at least 2 replicas of the public services for zero-downtime rollouts.
  • catalyst and runtime are the resource-heaviest. They ship the largest default requests and limits because they run the execution workloads (sizing baseline); scale and size those first as load grows.
  • Workload namespaces scale independently. The inference and managed-database pods that run in the feature namespaces are scheduled by nexus, not the chart, and follow the resource settings of the deployed resource — see Deploy & Integrate.

Backup and restore

Self-hosted state lives in the services you provide, not in the chart. Back up each on its own schedule:

StateLives inBack up with
Orgs, projects, workflows, deployment metadata, authPostgreSQL (nexus, synapse, catalyst databases)pg_dump per database, or your managed provider's automated snapshots
Files, run artifacts, Knowledge Base documentsS3 bucketBucket versioning and cross-region replication
In-flight jobs and streamsNATS JetStreamTreat as transient; back up its persistent volumes only if you provisioned them
Signing keys, database and provider credentials, the licenseKubernetes SecretsKeep the source manifests under version control — you created these, the chart doesn't

Postgres is the system of record: orgs, projects, workflows, and every deployment's metadata live there. Losing it loses the platform, so verify your dumps restore cleanly.

Restore the database before rolling the app back. Because Atlas migrations are forward-only, recovering from a bad schema-migrating upgrade means restoring the Postgres snapshot first, then rolling the release back to the version that matches it. See Upgrades & Rollback → Roll back.

Troubleshooting

Getting help

If a problem outlasts the fixes above, contact support@getdynamiq.ai. Include:

  • The chart version — helm list -n dynamiq.
  • Pod states — kubectl -n dynamiq get pods.
  • Relevant logs — the output of kubectl -n dynamiq logs deploy/<svc> (or job/dynamiq) for the failing component.

Next steps

On this page