Aging, tiering, and expiring data over its lifecycle
Lifecycle management automates moving or deleting data as it ages, reducing storage costs while meeting retention and compliance needs. On Google Cloud, this is primarily implemented via Cloud Storage Object Lifecycle Management rules, BigQuery table/partition expiration, and Bigtable garbage collection policies. Choosing the right mechanism per service is a common exam focus.
1 · Learn the must-know
- Cloud Storage Object Lifecycle Management uses conditions (age, createdBefore, matchesStorageClass, numNewerVersions, isLive, daysSinceCustomTime, daysSinceNoncurrentTime) combined with actions (SetStorageClass, Delete) defined in a JSON lifecycle configuration applied to a bucket.
- Lifecycle transitions in Cloud Storage only move data to a 'colder' (cheaper) storage class or delete it, you cannot use lifecycle rules to transition to a warmer/more expensive class.
- Lifecycle rule evaluation in Cloud Storage runs asynchronously (checked roughly once per day), so actions are not instantaneous when an object first meets a condition.
- BigQuery supports dataset-level default table expiration and per-table expiration (expirationTime), plus partition expiration on partitioned tables, so BigQuery handles its own lifecycle rather than relying on Cloud Storage rules.
- Bigtable lifecycle is managed through garbage collection (GC) policies per column family, based on age (MaxAge) or version count (MaxVersions), not through Cloud Storage-style lifecycle rules.
- For compliance/legal holds, Cloud Storage offers Bucket Lock (retention policies) and Object Hold, which can override or block lifecycle deletion until the hold/retention period expires.
2 · Check your understanding
A data engineer creates a BigQuery dataset with defaultTableExpirationMs set to 90 days (about 7,776,000,000 ms), expecting that in a new date-partitioned table, only partitions older than 90 days will be deleted while the table itself continues to receive new data indefinitely. Instead, after 90 days the entire table disappears. What is the cause, and what should the engineer configure instead?
What you have tried across GCP PDE's objectives, not a readiness score.
Designing data processing systems~22% of the exam0 of 17 tried
Ingesting and processing the data~25% of the exam0 of 11 tried
Storing the data~20% of the exam0 of 13 tried
Preparing and using data for analysis~15% of the exam0 of 11 tried
Maintaining and automating data workloads~18% of the exam0 of 15 tried
3 · Keep going
Ready for more? Take a weighted mock or try free practice questions.