Loading files from cloud storage into governed tables with COPY INTO

COPY INTO is a SQL command that idempotently and incrementally loads new files from a cloud storage location into a Delta table, tracking which files have already been ingested so reruns skip them. It suits append-only, file-based ingestion at moderate volumes and works with tables registered in Unity Catalog, provided the external location and storage credential are set up for governed access.

1 · Learn the must-know

• 1COPY INTO tracks already-loaded files internally (via table metadata), so re-running the same command against the same source path will not reload files already ingested, making it safe to schedule repeatedly.
• 2The target table must already exist (COPY INTO does not create tables); for Unity Catalog use, you create the Delta table first with CREATE TABLE, then run COPY INTO against it.
• 3Reading from cloud storage (ADLS, S3, GCS) under Unity Catalog requires a configured storage credential and external location (or a volume) with proper access grants; direct cloud URIs need governed access unless using a UC volume path.
• 4COPY_OPTIONS such as mergeSchema (to allow schema evolution) and FORCE (to reprocess files even if already loaded) let you override default idempotent, schema-enforced behavior.
• 5For very large numbers of new files or continuous/streaming ingestion, Auto Loader is generally preferred over COPY INTO, which is better suited to periodic batch loads of moderate file counts.
• 6COPY INTO supports common file formats (CSV, JSON, Parquet, Avro, ORC, text) and lets you specify FILEFORMAT and FORMAT_OPTIONS (e.g., header, inferSchema) to control how source files are parsed.

2 · Check your understanding