Spotting performance trends in a job's run history
The Lakeflow Jobs run history view lists every past run of a job with start time, duration, trigger type, and status, letting engineers spot whether a job is slowing down, speeding up, or failing more often over time. Comparing recent runs against a historical baseline (e.g. typical duration over past weeks) helps distinguish a one-off anomaly from a genuine performance regression. This view is the primary UI-based tool for trend analysis before diving into Spark UI or cluster metrics for root cause.
Must-know
- Run history is accessed from the Jobs UI under a specific job's 'Runs' tab, showing a chronological list with duration, start/end time, trigger (scheduled, manual, API), and status (succeeded, failed, running).
- Sorting or scanning run duration over time reveals trends such as gradually increasing runtime, which often points to data volume growth, skew, or cluster under-provisioning rather than a sudden failure.
- Each run's detail page links to the Spark UI for that run, enabling drill-down from a duration anomaly into stage/task-level bottlenecks.
- Comparing a current run against historical baselines helps separate transient issues (e.g. cloud provider throttling, spot instance eviction) from systemic problems (e.g. an unoptimized query or schema drift causing shuffle-heavy operations).
- Job clusters that use autoscaling or new cluster-per-run configurations can introduce duration variance unrelated to code changes, so baseline comparisons should account for cluster startup/provisioning time separately from task execution time.
- Repeated runs with increasing duration despite stable data volume is a common gotcha suggesting cluster resource contention, small file problems, or the need for table optimization (e.g. OPTIMIZE/Z-ORDER on Delta tables).
A data engineer opens the run history for a nightly ETL job and sees the last 30 runs averaged 12 minutes, but the past 5 runs each took 27-31 minutes with no code or schedule changes. The team wants to confirm whether this is a genuine performance regression before investigating further. Which action should the engineer take first?
What you have tried across Databricks DEA's objectives, not a readiness score.
Databricks Intelligence Platform
Data Ingestion and Loading
- Batch, streaming, and incremental loading patterns, and where the data comes from
- Loading files from cloud storage into governed tables with COPY INTO
- Landing data with Auto Loader, and handling schema enforcement and evolution
- Setting up Lakeflow Connect to ingest from enterprise sources reliably
- Pulling data through JDBC, ODBC, or REST clients and scheduling the job
- Choosing the right ingestion method for a given volume, frequency, and governance need
- Bringing semi-structured and unstructured data into governed Delta tables
Data Transformation and Modeling
- Cleaning bronze data into silver tables with PySpark and SQL
- Joining and combining DataFrames with the different join and union types
- Reshaping columns, rows, and arrays in a table
- Deduplicating and aggregating DataFrames
- Tuning Spark's core parameters and measuring what changed
- Building Gold-layer views and tables for BI and analytics
- Validating Silver and Gold datasets for quality
Working with Lakeflow Jobs
Implementing CI/CD
- Branching, committing, and opening pull requests from inside the Databricks workspace
- Promoting one codebase across dev, test, and prod with bundle variables and overrides
- Packaging and deploying jobs and pipelines with Automation Bundles
- Validating and managing bundle deployments from the Databricks CLI
Troubleshooting, Monitoring, and Optimization
- Spotting performance trends in a job's run history
- Reading job status, task graphs, and failure rates to monitor pipeline health
- Diagnosing skew, shuffle, and spill from Spark UI stage metrics
- What Liquid Clustering and predictive optimization actually do
- Diagnosing cluster startup failures, library conflicts, and out-of-memory errors
Governance and Security
Coverage checked against the published exam guide on Jul 27, 2026.
These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.