Deploying on Snowpark Container Services
valved serve — the REST API plus the scheduler, reaper, archiver and worker pool — can run as a long-lived Snowpark Container Services service. Scheduled pipelines then fire inside your Snowflake account, with no machine of yours left running and no warehouse credentials stored anywhere.
Everything needed to stand one up ships in the repository under deploy/spcs/: the image, setup.sql for the Snowflake-side objects, service specs with and without a bundled Postgres, and a full runbook covering upgrades, rollback and troubleshooting. This page is the orientation — what the deployment looks like, and the five properties of Valved that decide its shape.
Added in 0.5.0.
Five things that decide the shape
Section titled “Five things that decide the shape”Skipping these is how a deploy reaches READY and then does nothing useful.
Postgres is mandatory
Section titled “Postgres is mandatory”Valved rejects any state-store URL that is not postgresql:// or postgresql+psycopg://. There is no SQLite fallback. Runs, jobs, logs, schedules, API tokens and events all live there, and DATABASE_URL is the lever. See choosing a state store below.
Egress is denied by default
Section titled “Egress is denied by default”An SPCS container reaches nothing until an external access integration says otherwise. Valved needs:
api.anthropic.comfor the agent loop,- your Postgres host, unless you run the sidecar,
- the AWS endpoints, if dlt uses Secrets Manager or S3,
- the source API of every component you run.
That last one is the one that gets forgotten, because the rest of the list is infrastructure and looks complete without it. Write an explicit :443 on every entry.
Valved’s own warehouse connection needs no rule — it uses the platform-internal host and the mounted OAuth token. A component that brings its own Snowflake credentials does need one.
Run exactly one instance
Section titled “Run exactly one instance”Leader election is not implemented. Two instances means two schedulers firing the same cron. Scale with --workers N inside the one instance, or with separate valved worker services.
Whoever owns the service is the running container
Section titled “Whoever owns the service is the running container”A container connecting back with /snowflake/session/token gets a session whose primary role is the service’s owner role, and it cannot switch away.
setup.sql therefore uses two roles: an admin role that builds the deployment, and a runtime role that runs CREATE SERVICE so it owns the result. SNOWFLAKE_ROLE in the spec must name that same owner role — Valved passes it as role= on connect, and any other value fails.
Collapsing the two into one role works and is simpler. It also means Valved’s agent loop — shell tools included — executes SQL as the role that owns your secrets, image repository and compute pool.
The API bind host is a config file, not a flag
Section titled “The API bind host is a config file, not a flag”[api] host defaults to 127.0.0.1, and valved serve has no flag or environment variable to override it. Bound to loopback, the service starts, fails its readiness probe forever, and serves nothing at the ingress URL. The shipped project template sets 0.0.0.0 for exactly this reason.
Choosing a state store
Section titled “Choosing a state store”Three options, and none dominates. The trade is network fragility against operational access.
| Reachable from outside SPCS | Network setup for the container | Extra infrastructure | |
|---|---|---|---|
| Snowflake Postgres | yes | egress rule and an ingress range that is not contractually stable | none — it is a Snowflake object |
| External Postgres (RDS, Cloud SQL, …) | yes | egress rule; you control the far side | one database you run |
| Sidecar container | no | none — loopback inside the instance | none |
Network fragility. Snowflake Postgres works from a container but depends on allowlisting a NAT range Snowflake does not document as stable. The sidecar has no network boundary to configure at all; an external database has one you own.
Operational access. Reaching the state store from a laptop is what lets you run valved auth rotate, take a pg_dump, or query runs when the API itself is the broken thing. That is real but narrower than it looks: the bootstrap token arrives via the mounted stage either way, and valved runs / valved logs serve run history over the ingress URL whenever the API is up. What the sidecar actually costs you is direct SQL while the service is down.
The sidecar pins the service to one instance through its block volume — but leader election already pins you there, so that constraint costs nothing.
The shape of a deploy
Section titled “The shape of a deploy”Six steps, covered in full by the runbook in deploy/spcs/README.md:
- Snowflake objects — database, roles, compute pool, image repository, stages, and the external access integrations that open egress (
setup.sql). - Build the image — multi-stage,
uv.lock-pinned, non-root, with the dbt engine pre-baked. - Push it to the Snowflake image repository.
- Pick a spec — with a bundled Postgres sidecar, or pointing at your own — and edit it.
- Create the service.
- Get the API token, published to the mounted stage on first boot.
What you get
Section titled “What you get”The REST API is served at the ingress URL, with OAuth-backed access. /api/docs serves Swagger UI same-origin — the ingress CSP blocks the CDN it would otherwise load from, so the assets are fetched at build time and served by the container itself.
Scheduled pipelines fire from the in-container scheduler. Warehouse access uses the platform’s own OAuth token, so no Snowflake credential is stored in the image or the spec. If dlt needs AWS — Secrets Manager, S3 — a web-identity token refresher keeps that working without a stored AWS key.
Getting the full runbook
Section titled “Getting the full runbook”deploy/spcs/README.md in the repository is the operational document: exact commands for each step, upgrade and rollback procedure, the three things that commonly bite on AWS access, and how to run a dlt component from another repository.