Skip to content

Valved

Agent-native data engineering, from natural language to production pipelines. A control plane plus an AI harness over independently-versioned dlt + dbt + sql — self-hosted, Apache 2.0.

Valved takes a goal in plain English — “make stg_orders incremental”, “ingest the Stripe charges API into a Snowflake table” — turns it into a reviewable plan, and, when you approve, materializes the working code and opens a pull request in your repo: dbt models, Python dlt sources, SQL, tests, schema docs.

Agents are built to verify by executing, not by guessing. The DLT engineer holds gated bash and is directed not to call its work done until the pipeline runs green. The dbt engineer holds a dbt_verify tool that runs real dbt build / dbt test against the models it just wrote and reads the per-model result.

The authored diff then goes to adversarial reviewers before the build is recorded, and only after that does the control plane schedule, run, and monitor the pipeline on a durable job queue.

Terminal window
valved plan "ingest the Stripe charges API into raw_stripe, then a staging model stg_charges"
valved build <plan_id> # writes the dlt source and the dbt model into your tree
valved run <pipeline> --watch # execute on demand; data lands in your warehouse
valved deploy <pipeline> # open a reviewable PR with the built code

AI harness for data work

A Claude-Code-style agentic engine — a main orchestrator that delegates to domain subagents (a DLT engineer, a dbt engineer, a pipeline engineer, a SQL specialist, an explorer, plus a separate recovery agent for failed runs), armed with terminal-grade tools and running behind a permission gate that clamps each one to the narrower of the caller’s mode and its own declared ceiling.

A narrow control plane

Scheduled execution of dlt + dbt + sql pipelines over a Postgres job queue: multi-worker claiming, crash-recovery, retries, structured logs, and alerts. Deliberately not a general-purpose orchestrator.

Adversarial review gate

When agents author a diff, the orchestrator routes it through read-only reviewers before the build is recorded — dlt-qa and dlt-security for dlt work, dbt-qa for dbt. Each sees only {diff, goal}, never the engineer’s transcript. A blocker or major finding stops the build: no Build row, and the findings come back to you. Which builds this covers →

Headless by default

The valved CLI is the complete surface. A FastAPI REST API carries the core loop, and an MCP server is generated from that API’s OpenAPI schema — so MCP tracks REST by construction. The bundled HTML UI is static and read-only. These are not four equal surfaces; see below.

Bring your own, or build with AI

Build-with-Valved (agents author components into your repos) or orchestration-only (bring existing dlt/dbt/sql; Valved just composes, schedules, and monitors). Both are first-class.

Valved is headless in the sense that it is driven by machines as well as people — but the four surfaces are not interchangeable, and it’s worth knowing where the edges are before you build against one.

Surface Covers
CLI Everything. Project setup (init, connect, auth, target, component), the core loop (plan, build, deploy, run), and the inspection commands.
REST The core loop and the control plane: create a plan, run a build, deploy, enqueue and stream runs, list jobs/workers/pipelines, pause and resume schedules, mint tokens, register webhooks, read metrics and memory.
MCP Generated from REST’s OpenAPI document, one tool per operation, minus the log-streaming endpoint. It can never be wider than REST, and it talks to a running valved serve.
Static UI Read-only viewing — runs, per-run logs, pipelines, agents, skills. No authoring, no actions.

Two gaps are worth calling out, because they’ll bite if you assume parity:

  • Project setup is CLI-only. valved init, valved connect, valved auth, creating or renaming targets, graduating a component to its own repo, editing memory files, registering an external MCP server, and running EL artifacts directly all exist only as CLI commands. REST exposes read endpoints for targets, components, agents, skills, and registered MCP servers, but no way to create or change them.
  • POST /plans takes only {goal, pipeline_name}. The CLI’s valved plan --refine <plan_id> (which records a new plan with parent_plan_id set) and its --table / --database / --schema pre-seeds have no REST equivalent at all, and therefore no MCP tool. A client driving Valved over REST or MCP can create a plan and build it, but cannot iterate on one.

Open source & governance. Valved is Apache 2.0. There is no license check, entitlement flag, or paid tier anywhere in the codebase — the CLI, the REST API, the MCP server, the built-in agents, and the UI all ship in the repository, and no endpoint or MCP tool is withheld from it. The project’s OSS_COMMITMENT.md is the governance promise about where that line stays.