Applying CI/CD practices to a Vector Search index, prompts, and agent components

CI/CD for GenAI apps on Databricks extends standard software practices to cover data- and model-specific artifacts: Vector Search indexes, prompts, and agent components. Because these artifacts drift and change independently of code, pipelines must version, test, and promote them explicitly across dev/staging/prod, typically using separate catalogs/schemas per environment via Unity Catalog and Databricks Asset Bundles for deployment orchestration.

1 · Learn the must-know

• 1Vector Search indexes should be kept in sync with source Delta tables using managed pipelines (Delta Sync Index) with scheduled or continuous refresh so retrieval doesn't serve stale embeddings after code deploys.
• 2Prompts and prompt templates should be version-controlled (e.g., stored as code artifacts or registered/tracked via MLflow) rather than hardcoded, so they can be promoted across dev/staging/prod like any other reviewed artifact.
• 3MLflow is the backbone for packaging agents (via MLflow models/pyfunc or agent frameworks) so the same logged model version can be validated in staging and promoted to production without re-authoring.
• 4Databricks Asset Bundles (DABs) are the recommended way to define and deploy jobs, endpoints, and other workspace resources as code, enabling repeatable environment-specific deployments.
• 5Individual agent components (retrievers, tools, prompt chains, guardrails) should be unit-tested in isolation before integration/end-to-end evaluation, since failures often trace to a single component rather than the whole chain.
• 6Use separate Unity Catalog catalogs/schemas (and separate model/endpoint names) per environment so promotion is an explicit, auditable step rather than mutating a shared production resource in place.

2 · Check your understanding