Repairing, restarting, stopping, and rerunning Lakeflow Jobs

Lakeflow Jobs provides run-level controls—Run Now, Stop, and Repair Run—that let engineers manage failed or in-progress multi-task job executions without always re-running an entire pipeline. Understanding when each control applies, and what state it preserves, is essential for efficient troubleshooting and cost control.

1 · Learn the must-know

• 1Repair Run only re-executes tasks that failed or were skipped due to an upstream failure; successfully completed tasks in that run are not re-run, preserving their outputs and reducing cost/time.
• 2Repair Run is available only for multi-task (workflow) jobs that have already completed with a failed or partially failed status; it is not available for a job's very first attempt or for single-task jobs.
• 3Run Now always starts a brand-new run of the entire job from scratch, ignoring the status of any previous run, and is used when you want a full fresh execution rather than a targeted fix.
• 4Stop cancels a currently active (running or queued) job run; a stopped run is marked as 'Stopped/Cancelled' and cannot later be repaired the same way a failed run can—you typically trigger Run Now again.
• 5Each task can define a retry policy (number of retries and interval), which automatically re-attempts a failed task before the overall run is marked as failed—this is separate from manually invoking Repair Run after the run completes.
• 6Troubleshooting relies on the Runs tab (matrix/list view) plus per-task driver logs, Spark UI, and stdout/stderr output to identify root cause before choosing whether to Repair, re-Run, or Stop a job.

2 · Check your understanding