Retries, branching, and looping in a Lakeflow Jobs pipeline
Lakeflow Jobs let you orchestrate multi-task pipelines with control flow features like task retries, conditional branching via 'Run if' dependencies, and the If/else and For each task types. These constructs let you handle failures gracefully and dynamically execute paths based on upstream task outcomes or a list of inputs.
Must-know
- Each task can define a retry policy (number of retries and interval in seconds) so transient failures auto-retry before the task is marked failed.
- The 'Run if dependencies' setting on a task controls execution based on upstream task status: All succeeded (default), At least one succeeded, None failed, All done, At least one failed, or All failed.
- The If/else condition task type evaluates a boolean expression (comparing task values, parameters, etc.) and routes execution to different downstream tasks based on the result.
- The For each task type loops over a list of inputs, running a nested task once per item, with a configurable concurrency limit for parallel iterations.
- Task values (set via dbutils.jobs.taskValues) can be passed between tasks and referenced in downstream conditions or For each input lists to drive dynamic control flow.
- A job run is marked failed if any task fails and no downstream task's Run if condition allows the job to continue, so branching logic must account for overall job success criteria.
A Lakeflow Job task calls an external API that occasionally times out due to transient network issues. The team wants the task to automatically retry up to 3 times with a 2-minute pause between attempts before the job is marked failed. Which configuration accomplishes this?
What you have tried across Databricks DEA's objectives, not a readiness score.
Databricks Intelligence Platform
Data Ingestion and Loading
- Batch, streaming, and incremental loading patterns, and where the data comes from
- Loading files from cloud storage into governed tables with COPY INTO
- Landing data with Auto Loader, and handling schema enforcement and evolution
- Setting up Lakeflow Connect to ingest from enterprise sources reliably
- Pulling data through JDBC, ODBC, or REST clients and scheduling the job
- Choosing the right ingestion method for a given volume, frequency, and governance need
- Bringing semi-structured and unstructured data into governed Delta tables
Data Transformation and Modeling
- Cleaning bronze data into silver tables with PySpark and SQL
- Joining and combining DataFrames with the different join and union types
- Reshaping columns, rows, and arrays in a table
- Deduplicating and aggregating DataFrames
- Tuning Spark's core parameters and measuring what changed
- Building Gold-layer views and tables for BI and analytics
- Validating Silver and Gold datasets for quality
Working with Lakeflow Jobs
Implementing CI/CD
- Branching, committing, and opening pull requests from inside the Databricks workspace
- Promoting one codebase across dev, test, and prod with bundle variables and overrides
- Packaging and deploying jobs and pipelines with Automation Bundles
- Validating and managing bundle deployments from the Databricks CLI
Troubleshooting, Monitoring, and Optimization
- Spotting performance trends in a job's run history
- Reading job status, task graphs, and failure rates to monitor pipeline health
- Diagnosing skew, shuffle, and spill from Spark UI stage metrics
- What Liquid Clustering and predictive optimization actually do
- Diagnosing cluster startup failures, library conflicts, and out-of-memory errors
Governance and Security
Coverage checked against the published exam guide on Jul 26, 2026.
These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.