Preparing for corrupted or missing data before it happens

Preparing for data corruption and missing data on Google Cloud means designing pipelines and storage so bad or absent data is detected early and recovery is fast. Key controls span versioning, backups, validation checks, and dead-letter/error handling so downstream analytics are not silently poisoned by bad records.

1 · Learn the must-know

• 1Enable Object Versioning on Cloud Storage buckets so overwritten or deleted source files can be restored, and pair this with lifecycle rules to manage version retention costs.
• 2Use BigQuery time travel (default 7 days, configurable up to 7) and table snapshots to recover accidentally deleted or corrupted table data without a full backup restore.
• 3Cloud SQL and Spanner support automated backups plus point-in-time recovery, which should be enabled and tested so corrupted transactional data can be rolled back to a known-good state.
• 4Dataflow and Pub/Sub pipelines should route unparseable or schema-invalid records to a dead-letter topic/table rather than dropping them, preserving them for later inspection and reprocessing.
• 5Validate data quality early using schema enforcement (e.g., BigQuery schema validation, JSON schema checks in Pub/Sub) and tools like Dataplex data quality tasks to catch missing or malformed fields before they propagate.
• 6Idempotent, deduplicated pipeline design (using unique record IDs or Pub/Sub message ordering/exactly-once delivery) reduces the risk of missing data caused by retries or at-least-once delivery semantics.

2 · Check your understanding