Scheduling a job to run

Azure Databricks Jobs let you automate and schedule tasks or multi-task workflows using a cron-based scheduler, so notebooks, JARs, Python scripts, SQL queries, or pipelines run automatically without manual intervention. Job scheduling is configured in the Workflows UI (or via REST API/CLI) under a job's 'Schedule' or 'Trigger' tab, where you define the cadence, timezone, and cluster resources to use.

1 · Learn the must-know

• 1Jobs can be triggered by a cron-based schedule, a file arrival trigger (new files landing in a specified location), a continuous trigger, or manually/on-demand via 'Run Now'.
• 2The schedule uses the Quartz cron syntax, and you must explicitly set the timezone, since UTC is not always the default depending on workspace configuration.
• 3Each job run can use either a new job cluster (spun up fresh per run, more cost-efficient for scheduled/automated jobs) or an existing all-purpose cluster (faster startup but incurs cluster idle costs).
• 4You can configure email or webhook notifications for job start, success, and failure events, which is critical for monitoring scheduled pipeline health.
• 5Concurrent run behavior is configurable: by default, a new scheduled run will not start if a previous run of the same job is still in progress unless you explicitly allow concurrent runs.
• 6Job schedules can be paused or the job can be disabled without deleting its configuration, and permissions (Can Manage, Can Manage Run, Can View) control who can edit or trigger the schedule.

2 · Check your understanding