Web console

The console is an interactive web application over Valved’s REST API: a dashboard that answers did anything break?, a screen for nearly every /api/v1 operation, a calendar of run history and scheduled firings, and logs that stream while a step is still running.

Where it ships

The console is in the container image, not in the Python wheel. pip install valved gets you the API that backs it and no console — the bundle is built by this repository’s Dockerfile and copied into the image.

That is a deliberate boundary, not an oversight. The static web UI remains the UI that ships with the package, and it is unchanged.

So there are two ways to reach a console:

Deploy the container — see Deploying on Snowpark Container Services. The image sets console_assets_dir for you and the console is served at the ingress URL.
Build it yourself from web/ in the repository and point [api] console_assets_dir at the output. This is what contributors do; web/README.md covers it.

Reaching it

The console is served same-origin at /console, and / redirects there:

curl -sI http://127.0.0.1:8765/            # 307 → /console/

It does not own the root. /api/*, /healthz and /readyz are live there already, and a root-mounted single-page app with a catch-all silently shadows every route added after it — the failure being a new endpoint that returns the app shell instead of JSON, which reads as a routing bug nowhere near its cause.

If console_assets_dir is unset, the mount is skipped and the API serves exactly as it does without a console. If it is set but the directory has no index.html, the mount is skipped and a warning is logged — a UI must not be able to take the API down.

The dashboard

The landing screen answers three questions, in the order an operator asks them:

Is anything broken now? Failed runs in the window, each showing its error inline and linking straight to the run and its logs — not to a list you then have to search. A pipeline failing the same way again and again is one row, with a ×N count and the times it failed, each linking to its run; a different error is a row of its own. Crashed jobs and dead workers surface here too.
Is anything running? In-flight runs with elapsed time and current step; queued jobs with the worker label they need, and whether a worker advertising it is alive. A job queued behind no matching worker is a silent stall otherwise.
What does recent history look like? Success and failure counts over the window, schedule health, AI cost, today’s calendar strip, pipeline trends for the pipelines that ran in the last two weeks (the metrics screen keeps them all), and the five steps most in need of a look.

A state store the API cannot reach renders a banner and whatever is cached — never a blank page.

Runs and triggering

A run has a page of its own, because a failed run is the thing people arrive to look at and a table row cannot answer why. It leads with the error, then the outcome — duration, token spend, cost, and who triggered it — and carries both the run’s logs and its events. A run that finished with warnings leads with the warning instead, in yellow under Finished with warnings, so it is not read as a failure. Cancel and retry are offered per state.

Logs are live: the dlt and dbt step types stream output as it is produced, so a build in progress reads as one. Polling stops the moment the run can no longer change.

Triggering is asynchronous, and the console follows it. POST /runs returns a job id, not a run id — a worker creates the run when it claims the job. A trigger therefore lands on a job page that waits, names the stall if the job is queued behind a label no live worker advertises, and redirects to the run the moment one exists.

A run can be triggered with per-run parameters — a dlt step’s env for a backfill window, a dbt step’s select/vars/full_refresh for an on-demand subset build, or a run-operation step’s macro/args (say, {"dry_run": false}) — without touching the pipeline definition. A blank field is omitted from the request rather than sent empty, so it cannot override a pipeline’s configured defaults with nothing.

The pipeline graph

Each pipeline has a page (/console/pipelines/<name>) that draws its steps as a graph: dependency depth runs left to right, and steps with no path between them — the ones that run in parallel — stack top to bottom in one column. The same definition always opens in the same arrangement. Steps can be dragged about and spring back; Realign undoes any dragging, and a toggle turns the graph top-down.

Every step is labelled with how long it usually takes — mean and p90 over the last 5, 10 or 20 finished runs (GET /api/v1/pipelines/definitions/<name>/step-stats?runs=N). Only completed steps are timed, and only a run’s last attempt of a step.

Edit turns the page into an editor: add a step and configure it in the side panel, and drag from one step’s port onto another to make it wait. A loop is refused as you drag. Saving sends the steps and the [pipeline] block — never the seed_schedule — and the server validates the DAG before anything is written. New pipeline on the pipelines screen opens the same editor empty.

A run page opens on this graph, coloured by where that run is. Each remaining step shows an estimated finish — its average after its latest dependency’s estimated finish — and the run’s own estimate is the latest of those, its critical path. A step with no history has no estimate, and the run’s estimate then says it is a lower bound. Logs and events are the other two tabs (?tab=logs, ?tab=events).

The dbt graph

A dbt step is one box on the pipeline graph, but a whole-project build behind it can be hundreds of models. Click a dbt step on a run’s graph — or on a pipeline’s, for its newest run — and the models that step ran open as a graph of their own, coloured as they build: progress from dbt’s own N of TOTAL, what is running now, what failed and why, and how long each model took. A ranked table sits under the graph.

Four things keep a big build readable:

Schemas are bands. Each schema gets a band with its own progress bar, and a schema that finishes cleanly folds into a single node, so staging collapses while the marts are still building. Click a schema’s chip or band to fold or unfold it yourself; double-click a chip to show only that schema, and double-click again (or the Only … pill) to show them all.
Tests are a list, not nodes. Each model carries a bar for its tests — passed, failed, still to run — and opens the list on demand, failures first.
Problems narrows the graph to what failed plus everything upstream of it, and Not done to what has not finished. Find a model… searches, and selecting a model focuses its lineage in this run.
It is the run’s graph, not the project’s. Only what the step’s selection ran appears: models, seeds and snapshots.

The graph is recorded as the step runs: the step asks dbt-core to write its structured event log to a temporary directory, tails it, and saves the graph (every few seconds, stretching to every 15 seconds on a big build), then settles it on run_results.json when dbt exits. The step’s log is dbt’s console output and is unchanged — but because dbt’s file log now goes to that temporary directory, a pipeline’s dbt steps no longer write the project’s logs/dbt.log. run-operation steps and Fusion engines record no graph. The API is GET /api/v1/runs/{run_id}/steps/{step_id}/dbt, and GET /api/v1/pipelines/definitions/{name}/steps/{step_id}/dbt for a pipeline’s newest run.

Pipeline trends

GET /api/v1/metrics/pipelines aggregates ok/failed counts, success rate, mean and p95 duration, and a drift figure across several windows. The dashboard and the metrics screen carry the fleet view; expanding a row on the pipelines screen opens that one pipeline’s trend, fetched on expand so collapsed rows cost no queries.

Two things worth knowing about the numbers:

Means and p95s cover successful runs only. Averaging in a run that died in its first second makes a pipeline look faster the more it breaks.
The p95 is usually the number you want. The mean says what a run costs; the p95 says what the bad ones cost, which is what a schedule collides with. A pipeline whose mean holds steady while its tail doubles is a failure that has not happened yet.

The aggregation is done in SQL rather than over a page of runs, because computing it in the browser quietly becomes “the last 200 runs” on exactly the deployment big enough to need the screen.

Step trends

The metrics screen also has the same table one level down: one row per pipeline step (GET /api/v1/metrics/steps), with attempts passed and failed per window, retries, warnings and skips, mean and p95 duration, drift, and the step’s last failure. It counts attempts, not runs, so a step that only gets through on a retry is visible. Steps that are still failing sort to the top, and Problems only — on by default — hides every step that passed first time.

Webhooks: sending a test

Each webhook row has Send test. It posts one sample of every message type a subscriber can receive — a failure, a run lost with its worker, a warning, a success, a cancel, a diagnosis — or just the one you pick, in that subscription’s format, and shows what the receiver answered for each. Samples are marked as tests (a Slack message starts with [test]), go to that subscriber only, and are recorded nowhere.

Exporting pipeline definitions

Export JSON on the pipelines screen downloads every definition the current filters match — pick a tag chip first to export just those — as one list in the shape Bulk register accepts. Edit it (retag twenty pipelines at once, say) and paste it back into Bulk register.

The calendar

Run history and scheduled firings on one timeline, one lane per pipeline. Three kinds of block are drawn so they cannot be confused: a finished run is solid and coloured by outcome, an in-flight one is striped and anchored to its real start, and a scheduled firing is dashed and hollow. A run that finished with warnings is yellow — not green, which would say there is nothing to read, and not the amber of a cancel, since a block carries no label and the colour is the whole message.

A scheduled block’s width is an estimate — the mean duration of that pipeline’s last five successful runs, falling back to a default for a pipeline with no history. It is drawn dashed for exactly that reason.

The day being drawn is the day that is fetched, so “nothing ran in this window” is a statement about the window rather than about the page size. A window with no blocks names the most recent run’s day and offers one click to it, because “nothing ran today” and “this console cannot see my runs” otherwise render identically.

Filtering, search and paging

Every list pages at 50 — the API’s own default — and filters live in the URL, so a filtered view is shareable and one screen can link into another’s filtered state.

A column filter is one of two things, and the console shows you which:

Server-side, naming a query parameter the endpoint accepts — it narrows the whole collection.
Page-scoped, applied to the rows already loaded — the truth for an endpoint that returns its collection whole.

A page-scoped filter on a paginated endpoint is refused: it would show three rows out of the fifty on the page and imply three exist. A screen whose search box cannot reach the whole collection is labelled Search this page rather than Search.

Status filters are dropdowns rather than text boxes, because the values are spelled in ways nobody guesses — a run ends success while a job ends succeeded — and a typed filter silently matches nothing whenever you use the other table’s word.

GET /api/v1/me names the caller. Users are provisioned just-in-time: the first page load is the provisioning event, and an operator never provisions a Valved user on top of their Snowflake user.

A user record is attribution, not an account — no password, no role, no permission column. Your deployment’s posture is still the only authorization boundary, and every admitted caller may do everything.

How you sign in depends on what is in front of the API:

Behind an authenticating proxy — the SPCS case — there is no sign-in screen. The ingress already authenticated you, /me already names you, and the console renders straight into the dashboard.
Everywhere else, sign-in is a token exchange, not a password. POST /auth/session takes an existing API token and returns a __Host--prefixed, HttpOnly, Secure session cookie; DELETE /auth/session signs out. The token itself is untouched by signing out, so a CI job holding the same token keeps working.

This second path is off by default and must stay off behind an authenticating proxy — behind an ingress the container sees the proxy’s address on every request, so per-address throttling would either do nothing or lock out every user at once. Sessions expire (default 12 hours) even though tokens do not: a browser credential with no expiry is what makes a stolen laptop permanent.

Revoking or rotating a token invalidates every session derived from it — session resolution re-checks the parent token on each request rather than trusting the session row.

Configuration

All under [api] in valved/api.toml:

Option	Default	What it does
`console_assets_dir`	unset	Directory holding the built console. Unset serves no console; set but missing `index.html` logs a warning and serves no console.
`trusted_origins`	empty	Page origins allowed to drive a mutating request. `*` is refused; entries must be `scheme://host[:port]`.
`session_auth`	`false`	Enables the `POST /auth/session` token exchange. Refused at startup if `trusted_origins` is empty.
`session_ttl_hours`	`12`	How long a browser session lives.

trusted_origins is the cross-origin write protection a cookie credential requires, which is why enabling session_auth with an empty allowlist is refused rather than warned: Valved is the one handing out the cookie, so it fails closed.

What it is not

Not an authoring surface. The console triggers plan and build and shows their output; the CLI and the agent harness own authoring.
Not a permissions system. Identity here is attribution and personalization. Per-user authorization is not part of it.
Not server-rendered. It is a static bundle; no Node runtime enters the container’s runtime image.
Not a replacement for the static web UI, which still ships in the package and is unchanged.