Packaging and deploying jobs and pipelines with Automation Bundles

Declarative Automation Bundles (formerly Databricks Asset Bundles) let teams define jobs, pipelines, and other workspace assets as YAML source files that can be validated, deployed, and run consistently across environments. Bundles are managed with the Databricks CLI and typically follow a databricks.yml root config with environment-specific target overrides for dev, test, and prod.

1 · Learn the must-know

• 1A bundle is defined by a databricks.yml file at the project root, which can include resource definitions (jobs, pipelines) directly or reference separate YAML files under a resources folder.
• 2Bundles use targets to define per-environment configuration (e.g., dev, staging, prod), allowing overrides such as workspace host, run_as identity, and resource-level settings without duplicating the whole config.
• 3The core CLI workflow is databricks bundle validate, databricks bundle deploy, and databricks bundle run, executed against a chosen target with the -t flag.
• 4Deploying a bundle uploads source files and notebooks to a workspace path and creates or updates the corresponding jobs/pipelines as defined, enabling reproducible, version-controlled deployments instead of manual UI configuration.
• 5Bundle configuration supports variables and substitutions so common values (catalog names, cluster specs, paths) can be parameterized and reused across targets and resources.
• 6Default dev targets typically deploy resources scoped to the deploying user (e.g., prefixed job names, isolated paths) to avoid collisions, while prod targets are configured with fixed names and explicit run_as service principals or users.

2 · Check your understanding