Checking whether a source is stale with freshness rules
Source freshness in dbt checks how up-to-date the raw data in your source tables is by comparing the most recent loaded_at timestamp against configurable thresholds. It's configured in the sources.yml file and executed via the 'dbt source freshness' command, producing pass/warn/error statuses that help teams detect upstream data pipeline failures before they affect downstream models.
1 · Learn the must-know
- Freshness is configured with a '
loaded_at_field' (a timestamp column) plus 'warn_after' and 'error_after' blocks, each specifying a count and period (minute, hour, or day). - The command to check freshness is 'dbt source freshness' (the older alias 'dbt source snapshot-freshness' is deprecated).
- Freshness checks run a MAX(
loaded_at_field) query directly against the source table, so they do not require the source to be materialized by dbt and can be costly on very large tables without proper filtering. - You can use '
loaded_at_field: _metadata.last_modified' style expressions or warehouse-specific metadata columns (e.g., Snowflake's _metadata) when a reliableloaded_attimestamp column doesn't exist, and some adapters support 'freshness' based on metadata instead of a column scan. - A 'filter' config can be added to the freshness block to limit the rows scanned (e.g., only recent partitions), improving performance on large tables.
- Freshness results are recorded in run results/artifacts and can be used with 'dbt build'/'dbt source freshness' in CI or orchestration to fail pipelines early when upstream data is stale, and thresholds can be omitted at the source level while set per-table, or vice versa, following YAML inheritance rules.
2 · Check your understanding
A source table is configured as follows:
sources:
- name: crm
tables:
- name: opportunities
loaded_at_field: _loaded_at
freshness:
warn_after: {count: 12, period: hour}
error_after: {count: 24, period: hour}
You run dbt source freshness at 09:00. The most recent row in opportunities has _loaded_at timestamped 18 hours earlier. What status will dbt report for this table?
What you have tried across dbt Analytics Engineering's objectives, not a readiness score.
Developing and optimizing dbt models45.16% of the exam*0 of 14 tried
Managing dbt models governance9.68% of the exam*0 of 3 tried
Debugging data modeling errors16.13% of the exam*0 of 5 tried
Troubleshooting and optimizing dbt pipelines6.45% of the exam*0 of 2 tried
Implementing dbt tests9.68% of the exam*0 of 3 tried
Implementing and maintaining external dependencies6.45% of the exam*0 of 2 tried
Leveraging the dbt state6.45% of the exam*0 of 2 tried
* Our estimate. dbt Labs publishes no section weights.
3 · Keep going
Ready for more? Take a weighted mock or try free practice questions.