Smarter Week

How to automate it

How to automate “fix broken pipelines and failed jobs”

Here are 2 ways to spend less time on this, best first. Each comes with steps you can follow today and, for AI fixes, a prompt to copy.

60 min
typically, a few times a week
40%
of the time can be automated
Some setup
to set up

Fix 1 of 2

Software featureBest fix

Catch broken data before stakeholders do with tests and observability

Data tests and observability tools spot freshness, volume and schema problems and point to the upstream cause, so failures are found and fixed faster.

Typically saves about 35% of the time4 h to set up
  1. 1Add freshness, not-null and uniqueness tests to your most-used models (dbt tests or Elementary).
  2. 2Turn on an observability tool (Monte Carlo, Metaplane, Elementary, Bigeye) for anomaly detection and lineage.
  3. 3Send alerts to a data-team channel with the owner tagged.
  4. 4Use lineage to tell affected dashboard owners before they notice.

Tools: dbt tests · Elementary · Monte Carlo · Metaplane · Bigeye

Fix 2 of 2

AI

Use a coding agent on your dbt or pipeline repo

Claude Code, Codex, Cursor and Copilot can read the project, write models and tests, run dbt build and fix errors. You review the logic and the data it produces.

Typically saves about 30% of the time30 min to set up
  1. 1Add a short instructions file to the repo (CLAUDE.md or AGENTS.md) with naming conventions and how to run dbt and tests.
  2. 2Give the agent a well-scoped task with the prompt below.
  3. 3Let it run the build and tests against a dev target, never production.
  4. 4Review the SQL and check row counts against a known source before merging.
Prompt to copy
In this [DBT / AIRFLOW / DAGSTER] project, [TASK, e.g. add a model that calculates monthly active customers from events]. Follow the conventions in [EXAMPLE MODEL]. Add tests for [UNIQUENESS, NOT NULL, ACCEPTED VALUES] and a description for each column. Run the build against the dev target, fix any errors, and summarize what you changed plus anything you were unsure about. Do not run anything against production.

Tools: Claude Code, OpenAI Codex, Cursor or GitHub Copilot

Quick wins

Have you tried…

Have you had a coding agent write dbt models or pipeline code and run the build for you?
Tools like Claude Code, Codex or Cursor read your project, write the model and tests, run them against a dev target and fix errors. You review the logic and the numbers.
Do you get an alert when data is late or broken before someone tells you a dashboard is wrong?
Data tests and observability tools like Monte Carlo or Elementary watch freshness, volume and schema changes and alert the right owner with the upstream cause.

Who does this task

Roles in our library that list this as one of their common tasks. Each guide covers the rest of that role’s week.

HourLeak · the 8-minute work audit

How many hours does this cost you?

The free 8-minute check works out where your week goes and gives you your top fixes. The team scan does the same for everyone and adds it up, so you know which leaks to fix first.

Answers are anonymous. Leaders only see team totals.

Other common tasks for Data engineers