Architecting storage around the access patterns it must support
Choosing the right storage architecture on Google Cloud depends on how data will be accessed: by query pattern (analytical vs transactional), latency requirements, data structure, and access frequency. The Professional Data Engineer exam expects you to map access patterns, OLTP, OLAP, real-time serving, batch analytics, file-based ML training, to the correct storage service and schema design.
1 · Learn the must-know
- Cloud SQL and AlloyDB suit low-latency, transactional (OLTP) access with strong consistency and relational joins, while BigQuery is optimized for large-scale analytical (OLAP) scans, not row-level lookups.
- Bigtable is designed for high-throughput, low-latency key-based access patterns (time series, IoT, ad serving) and requires careful row-key design to avoid hotspotting and support range scans.
- Spanner supports globally distributed, strongly consistent transactional workloads that need horizontal scale beyond a single Cloud SQL instance, using interleaved tables and choice of primary keys to avoid hotspots.
- Cloud Storage is the default choice for unstructured/semi-structured data and staging (data lakes), with storage classes (Standard, Nearline, Coldline, Archive) chosen based on access frequency to optimize cost.
- Firestore/Datastore fit document-style, hierarchical access patterns for mobile/web apps needing flexible schema and offline sync, whereas Memorystore serves ultra-low-latency caching for frequently accessed data.
- A common gotcha: picking BigQuery for high-frequency single-row transactional updates (bad fit due to cost/latency) or Bigtable for ad hoc SQL analytics (lacks native SQL joins) indicates a mismatch between access pattern and storage engine.
2 · Check your understanding
A logistics company ingests telemetry from 2 million delivery vehicles, writing a GPS coordinate and speed reading for each vehicle every second. Analysts need sub-10-millisecond reads of a single vehicle's most recent 500 readings, and the write throughput must scale linearly as the fleet grows. Which storage architecture best supports this access pattern?
What you have tried across GCP PDE's objectives, not a readiness score.
Designing data processing systems~22% of the exam0 of 17 tried
Ingesting and processing the data~25% of the exam0 of 11 tried
Storing the data~20% of the exam0 of 13 tried
Preparing and using data for analysis~15% of the exam0 of 11 tried
Maintaining and automating data workloads~18% of the exam0 of 15 tried
3 · Keep going
Ready for more? Take a weighted mock or try free practice questions.