Loading files from cloud storage into governed tables with COPY INTO
COPY INTO is a SQL command that idempotently and incrementally loads new files from a cloud storage location into a Delta table, tracking which files have already been ingested so reruns skip them. It suits append-only, file-based ingestion at moderate volumes and works with tables registered in Unity Catalog, provided the external location and storage credential are set up for governed access.
Must-know
- COPY INTO tracks already-loaded files internally (via table metadata), so re-running the same command against the same source path will not reload files already ingested, making it safe to schedule repeatedly.
- The target table must already exist (COPY INTO does not create tables); for Unity Catalog use, you create the Delta table first with CREATE TABLE, then run COPY INTO against it.
- Reading from cloud storage (ADLS, S3, GCS) under Unity Catalog requires a configured storage credential and external location (or a volume) with proper access grants; direct cloud URIs need governed access unless using a UC volume path.
COPY_OPTIONSsuch as mergeSchema (to allow schema evolution) and FORCE (to reprocess files even if already loaded) let you override default idempotent, schema-enforced behavior.- For very large numbers of new files or continuous/streaming ingestion, Auto Loader is generally preferred over COPY INTO, which is better suited to periodic batch loads of moderate file counts.
- COPY INTO supports common file formats (CSV, JSON, Parquet, Avro, ORC, text) and lets you specify FILEFORMAT and
FORMAT_OPTIONS(e.g., header, inferSchema) to control how source files are parsed.
A data engineer runs the following command daily to load new Parquet files from an external ADLS location into a Unity Catalog table: COPY INTO main.sales.transactions FROM 'abfss://[email protected]/transactions/' FILEFORMAT = PARQUET; On day two, 500 new files have landed alongside the original 10,000 files, none of which have been modified or deleted. What happens when the command runs again?
What you have tried across Databricks DEA's objectives, not a readiness score.
Databricks Intelligence Platform
Data Ingestion and Loading
- Batch, streaming, and incremental loading patterns, and where the data comes from
- Loading files from cloud storage into governed tables with COPY INTO
- Landing data with Auto Loader, and handling schema enforcement and evolution
- Setting up Lakeflow Connect to ingest from enterprise sources reliably
- Pulling data through JDBC, ODBC, or REST clients and scheduling the job
- Choosing the right ingestion method for a given volume, frequency, and governance need
- Bringing semi-structured and unstructured data into governed Delta tables
Data Transformation and Modeling
- Cleaning bronze data into silver tables with PySpark and SQL
- Joining and combining DataFrames with the different join and union types
- Reshaping columns, rows, and arrays in a table
- Deduplicating and aggregating DataFrames
- Tuning Spark's core parameters and measuring what changed
- Building Gold-layer views and tables for BI and analytics
- Validating Silver and Gold datasets for quality
Working with Lakeflow Jobs
Implementing CI/CD
- Branching, committing, and opening pull requests from inside the Databricks workspace
- Promoting one codebase across dev, test, and prod with bundle variables and overrides
- Packaging and deploying jobs and pipelines with Automation Bundles
- Validating and managing bundle deployments from the Databricks CLI
Troubleshooting, Monitoring, and Optimization
- Spotting performance trends in a job's run history
- Reading job status, task graphs, and failure rates to monitor pipeline health
- Diagnosing skew, shuffle, and spill from Spark UI stage metrics
- What Liquid Clustering and predictive optimization actually do
- Diagnosing cluster startup failures, library conflicts, and out-of-memory errors
Governance and Security
Coverage checked against the published exam guide on Jul 25, 2026.
These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.