Introduction
Valved is a control plane plus an AI harness, over independently-versioned dlt/dbt/sql components — not a project that contains them. The value proposition: build, schedule, and monitor pipelines — all with AI.
Valved schedules, orchestrates, and monitors dlt (extract & load) + dbt (transform) + sql pipelines. Its AI harness builds and deploys those components for you — or you bring your own, built outside Valved, and Valved just orchestrates them.
The two halves
Section titled “The two halves”The control plane is the thing you run (valved serve, backed by Postgres). It holds the orchestration entities — pipelines, steps, schedules, jobs, runs, deploys — plus versioned references to your code components. It does not contain the components’ code. Each component is independently versioned and follows its own repo / CI-CD / lifecycle. A pipeline references each component by name and composes them into a step DAG.
The AI harness is a Claude-Code-style agentic engine that authors those components. A main orchestration loop classifies your intent and delegates to domain subagents:
- a DLT engineer — writes dlt sources, resources, and pipelines
- a dbt engineer — writes dbt models, tests, and schema docs
- a pipeline engineer — composes components into a step DAG
- a recovery engineer — diagnoses failed runs and proposes reviewable fixes
- an explorer — answers read-only questions about your project (
valved ask)
Each is armed with terminal-grade tools (edit, bash, glob, grep, web_fetch, web_search), runs behind a permission gate (read_only / plan / build / deploy modes), and verifies its work by executing it rather than guessing.
Two adoption modes, both first-class
Section titled “Two adoption modes, both first-class”- Build-with-Valved — the AI authors components into your repos, and the control plane schedules them.
- Orchestration-only — you bring existing dlt/dbt/sql, Valved references them by version and only composes, schedules, and monitors. This is a central path, not a corner case — it’s exactly why the control plane references components rather than owning them.
Who Valved is for
Section titled “Who Valved is for”- The staff data engineer at a mid-size company who owns the data platform, has dbt on Snowflake, maintains a few dozen pipelines, and wants to delegate the routine 60% of their work.
- The analytics engineer who lives in dbt and wants agent assistance for model authoring, refactoring, and test generation that respects their team’s conventions.
- The agent developer building their own data-stack agent who wants a programmable backend (REST / MCP) that can plan/build/run pipelines — Valved is headless by default.
Valved targets the SMB-to-mid-market segment. It is not designed for the long tail of large-enterprise platform requirements (multi-cluster orchestration, fine-grained RBAC across thousands of resources, custom asset-graph orchestration) — those teams are better served by Airflow or Dagster.