Batch, streaming, and incremental loading patterns, and where the data comes from
Databricks supports three core ingestion patterns: one-time batch loads, continuous/triggered streaming with Structured Streaming, and incremental batch loads that pick up only new files since the last run. Local/cloud file ingestion is typically handled via Auto Loader (cloudFiles) or COPY INTO, while Lakeflow Connect provides standard connectors (for common SaaS/file sources) and managed connectors (fully hosted ingestion pipelines) for scalable, low-maintenance data import.
Must-know
- Auto Loader (cloudFiles format) incrementally and idempotently processes new files landing in cloud storage, tracking processed files via checkpointing so files are not reprocessed.
- COPY INTO is a SQL command best suited for idempotent, incremental batch loading of files into a Delta table when file counts are moderate (thousands, not millions); Auto Loader scales better for very large or continuously arriving file volumes.
- Streaming ingestion in Databricks uses Structured Streaming with triggers: trigger(availableNow=True) processes all currently available data once then stops (micro-batch style incremental processing), while continuous/fixed-interval triggers keep the stream running.
- Lakeflow Connect standard connectors provide simplified, UI/config-driven ingestion from common external sources (e.g., cloud storage, message buses) without hand-written streaming code.
- Lakeflow Connect managed connectors are fully managed, Databricks-hosted ingestion pipelines for enterprise applications and databases (e.g., Salesforce, SQL Server, Workday) that handle scheduling, schema evolution, and incremental refresh automatically.
- Schema inference and evolution (rescued data column) are key gotchas with Auto Loader: unexpected or new columns are captured in a rescued data column rather than causing job failure, unless schema enforcement is explicitly configured.
A team wants to ingest data from a supported SaaS application (such as Salesforce) directly into Unity Catalog tables, with Databricks handling schema detection, incremental extraction, and orchestration of ongoing syncs automatically. Which Lakeflow Connect option should they use?
What you have tried across Databricks DEA's objectives, not a readiness score.
Databricks Intelligence Platform
Data Ingestion and Loading
- Batch, streaming, and incremental loading patterns, and where the data comes from
- Loading files from cloud storage into governed tables with COPY INTO
- Landing data with Auto Loader, and handling schema enforcement and evolution
- Setting up Lakeflow Connect to ingest from enterprise sources reliably
- Pulling data through JDBC, ODBC, or REST clients and scheduling the job
- Choosing the right ingestion method for a given volume, frequency, and governance need
- Bringing semi-structured and unstructured data into governed Delta tables
Data Transformation and Modeling
- Cleaning bronze data into silver tables with PySpark and SQL
- Joining and combining DataFrames with the different join and union types
- Reshaping columns, rows, and arrays in a table
- Deduplicating and aggregating DataFrames
- Tuning Spark's core parameters and measuring what changed
- Building Gold-layer views and tables for BI and analytics
- Validating Silver and Gold datasets for quality
Working with Lakeflow Jobs
Implementing CI/CD
- Branching, committing, and opening pull requests from inside the Databricks workspace
- Promoting one codebase across dev, test, and prod with bundle variables and overrides
- Packaging and deploying jobs and pipelines with Automation Bundles
- Validating and managing bundle deployments from the Databricks CLI
Troubleshooting, Monitoring, and Optimization
- Spotting performance trends in a job's run history
- Reading job status, task graphs, and failure rates to monitor pipeline health
- Diagnosing skew, shuffle, and spill from Spark UI stage metrics
- What Liquid Clustering and predictive optimization actually do
- Diagnosing cluster startup failures, library conflicts, and out-of-memory errors
Governance and Security
Coverage checked against the published exam guide on Jul 25, 2026.
These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.