Reading job status, task graphs, and failure rates to monitor pipeline health

Lakeflow Jobs (formerly Databricks Workflows) provides a UI to monitor multi-task job runs through a DAG-based task graph, letting you see run status, spot failed or blocked upstream tasks, and review run history and durations. Understanding the job run states and how task dependencies propagate failures is key to diagnosing pipeline health quickly.

1 · Learn the must-know

• 1Job run states include Pending, Running, Succeeded, Failed, and Terminated/Skipped; a task's color in the DAG view (e.g., red for failed, gray for skipped) instantly shows where a pipeline broke.
• 2If an upstream task fails, downstream tasks that depend on it are skipped by default unless the task's dependency condition (e.g., run even if a dependency fails) is explicitly configured.
• 3The Runs tab lists historical runs with start time, duration, and status, enabling you to track run times and failure rates over time to spot trends or regressions.
• 4Clicking a task node in the DAG opens task-level details including logs, output, and duration, which is the fastest way to isolate the root cause of a failure in a multi-task job.
• 5Job-level and task-level retry policies can be configured; retried attempts appear in the run history, so a run can show 'Succeeded' overall even after earlier task attempts failed.
• 6Email or webhook notifications can be configured on job start, success, or failure, allowing proactive monitoring without manually checking the Jobs UI.

2 · Check your understanding