MCPcopy
hub / github.com/dagucloud/dagu

github.com/dagucloud/dagu @v2.9.1 sqlite

repository ↗ · DeepWiki ↗ · release v2.9.1 ↗
24,336 symbols 111,350 edges 1,904 files 7,172 documented · 29%
README

Dagu Workflow Schema at a Glance

This is the repository-level overview of Dagu's workflow YAML schema. It is meant to help a visitor understand how a workflow is shaped, what the current canonical syntax is, and where to look when they need the full field-level reference.

Dagu workflows are DAGs described in YAML. The current schema has a simple execution split:

  • run: for local shell commands and scripts
  • action: for named builtin actions, custom actions, or versioned remote action packages
  • actions: for reusable custom action definitions

The older v1 execution fields (command:, script:, step-level type:, call:, and step_types:) are still loadable for compatibility, but new workflows should use the syntax below.

Minimal Workflow

name: release-check
type: graph

params:
  ENVIRONMENT: staging

steps:
  - id: test
    run: go test ./...

  - id: health
    action: http.request
    with:
      method: GET
      url: https://example.com/health
    depends: [test]

  - id: deploy
    run: ./deploy.sh ${ENVIRONMENT}
    depends: [health]
    retry_policy:
      limit: 3
      interval_sec: 10

The root type: controls how the workflow executes:

  • graph runs steps according to depends: and can run independent steps in parallel.
  • graph is the default when type: is omitted.
  • chain runs steps in order.

Do not confuse root type: with legacy step-level type:. Step-level type: is deprecated; use action: for named executors.

Do not use scalar step shorthand such as - echo hello. It is deprecated; write explicit step objects with run: instead.

Mental Model

A Dagu file has three layers:

Layer Fields Purpose
Workflow metadata name, description, group, labels Identify and organize the DAG.
Workflow runtime schedule, params, env, working_dir, shell, shell_args, queue, worker_selector, timeout_sec, max_active_runs, artifacts, log_output Configure when and where the DAG runs.
Step graph steps, handler_on, defaults, actions Define executable work, shared step defaults, lifecycle handlers, and custom actions.

Most workflows only need name, type, params, and steps.

Step Shape

Every step has common workflow-control fields plus one execution field.

steps:
  - id: step_id
    run: echo hello
    depends: [previous_step]
    env:
      - LOG_LEVEL: info
    timeout_sec: 300
    retry_policy:
      limit: 2
      interval_sec: 5
    output: RESULT

Common step fields:

Field Meaning
id Stable identifier for dependencies and output references. Prefer this on every step.
name Optional display name.
description Step description.
depends Step dependency or dependency list.
env Step environment variables.
working_dir Step working directory.
timeout_sec Step timeout in seconds.
retry_policy Retry behavior for this step.
repeat_policy Repeat or polling behavior.
continue_on Continue after selected failure, skip, exit-code, or output conditions.
preconditions Conditions that must pass before the step starts.
worker_selector Required worker labels.
stdout, stderr, log_output Step log output configuration. stdout can also publish DAG/action outputs.
output Captured stdout variable or structured step-scoped output.
output_schema JSON Schema for stdout JSON validation.
approval Human approval gate after step execution.
container Container context for a local run: command.

run: for Local Commands

Use run: for local shell commands and scripts.

steps:
  - id: hello
    run: echo hello

Multi-line run: becomes a script:

steps:
  - id: build
    run: |
      set -e
      npm ci
      npm test

run: can also be an array of commands:

steps:
  - id: build_and_test
    run:
      - go build ./...
      - go test ./...

Only shell settings are accepted in with: for a run: step:

steps:
  - id: bash_step
    run: echo "$SHELL"
    with:
      shell: bash -e
      shell_args: [-c]
      shell_packages: [curl]

Important: object-form run: is not part of the current schema. This is invalid:

steps:
  - run:
      type: shell
      input: echo hello

action: for Named Execution

Use action: when the step is not just a local shell command.

steps:
  - id: create_job
    action: http.request
    with:
      method: POST
      url: https://api.example.com/jobs
      body: '{"queue":"default"}'

action: names select builtin actions, custom actions, or versioned remote action packages. Inputs and executor-specific options go under with:.

Current builtin actions:

Action Use for Key inputs
http.request HTTP calls method, url, headers/body config
ssh.run Remote shell over SSH command, SSH connection config
exec Direct process execution without shell parsing command, optional args
docker.run Docker executor optional command, Docker config
container.run Container executor optional command, container config
k8s.run, kubernetes.run Kubernetes job execution optional command, Kubernetes config
postgres.query, sqlite.query SQL queries query, database config
postgres.import, sqlite.import SQL imports import, database config
redis.<operation> Redis operations Redis config; operation comes from the action suffix
jq.filter jq transforms filter, plus data or input
dag.run Child DAG execution dag, optional params
dag.enqueue Asynchronous child DAG enqueue dag, optional params, optional queue
router.route Conditional routing value, routes
chat.completion LLM chat completion prompt or messages, model config
harness.run CLI coding-agent harnesses prompt, provider config, optional stdin
template.render Text/template rendering template, optional data/config
log.write Log messages message
mail.send Email sending mail executor config
archive.create, archive.extract, archive.list Archive operations archive config
file.stat, file.read, file.write, file.copy, file.move, file.delete, file.mkdir, file.list File operations path/source/destination/content config
git.checkout Git repository checkout repository, path, optional ref, depth, auth config
wait.duration, wait.until, wait.file, wait.http Wait or poll for time, files, or HTTP readiness duration, until, path, url, optional polling config
s3.upload, s3.download, s3.list, s3.delete S3 operations S3 config
sftp.upload, sftp.download SFTP transfers SFTP config
noop Output-only or approval-only placeholder step no with, or empty with
owner/repo@version, name@version, source:target@version Remote action package caller input object under with:

run: and action: are mutually exclusive on a step. Do not combine either with legacy execution fields such as command:, script:, step-level type:, call:, messages:, llm:, value:, or routes:.

Remote action packages contain a dagu-action.yaml manifest and a DAG entrypoint. GitHub refs such as acme/dagu-action-notify@v1.2.0 and official refs such as slack@v1.2.0 are portable across worker pools when the process executing the action can resolve the Git ref. Explicit source: refs support local paths, file:// paths, and Git URLs; local paths must exist on the worker executing the action step.

The action manifest inputs schema validates the caller's with: object before the action DAG starts. The outputs schema validates the final action output object published by stdout.outputs or action: outputs.write before it is exposed to the parent step as ${step.outputs.*}.

Common Action Examples

SQL Query

steps:
  - id: active_users
    action: postgres.query
    with:
      dsn: ${DATABASE_URL}
      query: SELECT id, email FROM users WHERE active = true

The built-in SQL action family supports PostgreSQL and SQLite. Use the official duckdb@v1 remote action when a workflow needs DuckDB while keeping the Dagu core binary cgo-free.

steps:
  - id: analyze
    action: duckdb@v1
    with:
      query: SELECT 42 AS answer

Child DAG

Use dag.run when the parent workflow must wait for the child DAG result:

steps:
  - id: process_account
    action: dag.run
    with:
      dag: workflows/process-account
      params:
        ACCOUNT_ID: acct_123
        REGION: us-east-1

Use dag.enqueue when the parent only needs to create a queued child DAG run and continue:

steps:
  - id: queue_account_report
    action: dag.enqueue
    with:
      dag: workflows/account-report
      params:
        ACCOUNT_ID: acct_123
      queue: background

dag.enqueue accepts the same with.dag and with.params inputs as dag.run, plus with.queue to override the queued child run's queue.

parallel: currently requires action: dag.run or action: dag.enqueue:

steps:
  - id: fanout
    action: dag.run
    with:
      dag: workflows/process-account
      params:
        ACCOUNT_ID: ${ITEM.account_id}
        REGION: ${ITEM.region}
    parallel:
      max_concurrent: 3
      items:
        - account_id: acct_1
          region: us-east-1
        - account_id: acct_2
          region: eu-west-1

Wait for HTTP Readiness

steps:
  - id: wait_for_api
    action: wait.http
    with:
      url: https://api.example.com/health
      status: 200
      poll_interval: 5s
      request_timeout: 10s
    timeout_sec: 300

Use timeout_sec to cap total wait time for polling actions such as wait.file and wait.http.

Harness

harnesses:
  codex-cli:
    binary: codex
    prefix_args: [exec]
    prompt_mode: arg

steps:
  - id: review
    action: harness.run
    with:
      provider: codex-cli
      prompt: Review the current branch and list actionable issues.

Prefer action: harness.run for new workflows. Compatibility note: a top-level harness: config still causes steps without an explicit executor type to infer the harness executor. Do not mix top-level harness: with ordinary shell run: steps unless prompt inference is intended.

Reusable Custom Actions

Use top-level actions: to define a validated reusable action.

actions:
  release.announce:
    description: Print a release announcement
    input_schema:
      type: object
      additionalProperties: false
      required: [channel, version]
      properties:
        channel:
          type: string
          enum: [changelog, slack]
        version:
          type: string
    template:
      run: echo {{ json .input.channel }} {{ json .input.version }}

steps:
  - id: announce
    action: release.announce
    with:
      channel: changelog
      version: v1.2.3

Custom action rules:

  • The action name must match ^[A-Za-z][A-Za-z0-9_-]*(\.[A-Za-z][A-Za-z0-9_-]*)*$.
  • input_schema is required and must resolve to an object schema.
  • template is required and must contain exactly one of run or action.
  • type: is not supported inside an actions: definition.
  • Legacy execution fields are rejected inside action templates.
  • with: at the call site is validated against input_schema.
  • Custom actions can call other custom actions; recursive references are rejected.

Legacy step_types: remains loadable, but it is deprecated. Use actions: for new reusable definitions.

Outputs

String-form output: captures trimmed stdout into a flat variable:

steps:
  - id: version
    run: git rev-parse --short HEAD
    output: VERSION

  - id: publish
    run: echo "Publishing ${VERSION}"
    depends: [version]

Object-form output: publishes structured step output:

steps:
  - id: inspect
    run: echo '{"version":"v1.2.3","artifact":{"url":"https://example.test/app.tgz"}}'
    output:
      version:
        from: stdout
        decode: json
        select: .version
      artifact:
        from: stdout
        decode: json
        select: .artifact

  - id: publish
    depends: [inspect]
    run: echo "${inspect.output.version} ${inspect.output.artifact.url}"

Structured output sources are stdout, stderr, and file. Decoders are text, json, and yaml. select requires json or yaml.

Use stdout.outputs or action: outputs.write when a DAG or remote action needs to return values to its caller:

steps:
  - id: notify
    run: ./notify.sh
    stdout:
      outputs:
        fields:
          messageId:
            decode: json
            select: .id

  - id: publish
    action: outputs.write
    with:
      values:
        status: sent

Caller-visible outputs are available as ${step.outputs} or ${step.outputs.messageId}. This is separate from object-form output:, which is step-scoped and remains available through ${step.output.*}.

Lifecycle Handlers

Use handler_on for lifecycle steps:

handler_on:
  failure:
    action: log.write
    with:
      message: Workflow failed
  exit:
    run: ./cleanup.sh

steps:
  - id: main
    run: ./run.sh

Handlers use the same current step execution syntax as normal steps.

Validation and Source of Truth

Use these commands when editing workflow YAML:

dagu validate workflow.yaml
dagu schema dag
dagu schema dag steps

Implementation-level references:

  • internal/cmn/schema/dag.schema.json - generated JSON Schema used by editor tooling and schema navigation
  • internal/core/spec/step_v2.go - run: and action: normalization
  • `inter

Extension points exported contracts — how you extend this code

TokenSecretProvider (Interface)
TokenSecretProvider resolves the JWT signing secret from a configured source. Implementations may perform I/O (file read [12 …
internal/auth/token_secret.go
DispatchTaskStore (Interface)
DispatchTaskStore manages the shared distributed dispatch queue. [6 implementers]
internal/core/exec/distributed.go
WatermarkStore (Interface)
WatermarkStore persists scheduler watermark state to durable storage. [10 implementers]
internal/service/scheduler/watermark.go
RetryPolicy (Interface)
RetryPolicy defines the interface for retry policies. [6 implementers]
internal/cmn/backoff/retrypolicy.go
AuditStore (Interface)
AuditStore is a file-backed audit store with an optional background cleaner. [51 implementers]
internal/persis/file/service_stores.go
ContextRunner (Interface)
ContextRunner is a minimal interface for background services that are driven by a cancellable context. [56 implementers]
internal/testutil/context_runner.go
StatusPusher (Interface)
StatusPusher reports DAG run status outside the current execution process. [6 implementers]
internal/runtime/reporter.go
CacheStore (Interface)
CacheStore provides persistence for upgrade check cache data. [10 implementers]
internal/upgrade/store.go

Core symbols most depended-on inside this repo

Equal
called by 7362
internal/service/eventstore/notifications.go
Errorf
called by 4518
internal/cmn/logger/logger.go
Run
called by 3432
internal/runtime/executor/executor.go
Error
called by 2530
internal/cmn/logger/logger.go
Len
called by 1217
internal/core/exec/queue.go
WriteHeader
called by 1021
internal/runtime/builtin/sql/result.go
Unlock
called by 899
internal/core/exec/proc.go
Lock
called by 834
internal/core/exec/proc.go

Shape

Function 11,295
Method 8,986
Struct 2,677
TypeAlias 706
Interface 487
FuncType 106
Enum 61
Class 18

Languages

Go93%
TypeScript7%

Modules by API surface

api/v1/api.gen.go3,594 symbols
proto/coordinator/v1/coordinator_protoopaque.pb.go744 symbols
proto/coordinator/v1/coordinator.pb.go744 symbols
internal/core/spec/dag.go165 symbols
internal/service/frontend/api/v1/dagruns.go147 symbols
internal/core/spec/step.go124 symbols
internal/service/frontend/api/v1/api.go122 symbols
proto/coordinator/v1/coordinator_grpc.pb.go111 symbols
internal/service/worker/remote_handler_test.go104 symbols
internal/runtime/builtin/kubernetes/config.go99 symbols
internal/gitsync/service.go97 symbols
internal/service/coordinator/handler.go96 symbols

Dependencies from manifests, versioned

4d63.com/gocheckcompilerdirectivesv1.3.0 · 1×
4d63.com/gochecknoglobalsv0.2.2 · 1×
al.essio.dev/pkg/shellescapev1.5.1 · 1×
charm.land/lipgloss/v2v2.0.3 · 1×
codeberg.org/chavacava/garifv0.2.0 · 1×
codeberg.org/polyfloyd/go-errorlintv1.9.0 · 1×
dario.cat/mergov1.0.2 · 1×
dev.gaijin.team/go/exhaustruct/v4v4.0.0 · 1×
dev.gaijin.team/go/golibv0.6.0 · 1×
filippo.io/edwards25519v1.1.1 · 1×
github.com/4meepo/tagalignv1.4.3 · 1×
github.com/Abirdcfly/dupwordv0.1.7 · 1×

Datastores touched

secretsCollection · 1 repos
profilesCollection · 1 repos
dag_stateCollection · 1 repos
schedulerCollection · 1 repos
active_runsCollection · 1 repos
dag-settingsCollection · 1 repos
dag_run_leasesCollection · 1 repos
dispatch_tasksCollection · 1 repos

For agents

$ claude mcp add dagu \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact