Authenticating data access with service principals

Service principals are non-human identities used to authenticate automated tools, jobs, and applications to Azure Databricks and Unity Catalog without relying on a user's personal credentials. They can be granted Unity Catalog permissions, added to Databricks groups, and assigned as job run-as identities, enabling secure, auditable, and least-privilege automation. Managing access through service principals is the recommended practice for production workloads, CI/CD pipelines, and scheduled jobs.

1 · Learn the must-know

• 1A service principal is created in Microsoft Entra ID (Azure AD) first, then added to the Azure Databricks account and workspace before it can be granted permissions.
• 2Unity Catalog permissions (GRANT/REVOKE on catalogs, schemas, tables, etc.) can be assigned to a service principal just like a user or group, following the same privilege model.
• 3Authentication for a service principal typically uses an OAuth token (client ID/secret via Entra ID) or a Databricks personal access token generated for that service principal, not interactive login.
• 4Jobs and workflows should be set to run as a service principal (rather than an individual user) so that ownership and permissions persist even if the creating user leaves or changes roles.
• 5Service principals must be explicitly added to a Databricks workspace and assigned appropriate entitlements (e.g., workspace access, cluster creation) in addition to Unity Catalog data permissions.
• 6Using service principals for automated access avoids embedding personal user credentials in scripts/pipelines, supporting auditability and adherence to least-privilege security practices.

2 · Check your understanding