Shaping data during the ingestion pipeline itself

Data transformation on Google Cloud spans batch and streaming pipelines, with Dataflow (Apache Beam) as the unified serverless engine, alongside Dataproc for Spark/Hadoop-based ETL and Data Fusion for GUI-driven pipeline design. Choosing the right transformation tool and pattern depends on data volume, latency needs, existing skill sets, and whether transformations are ELT (in-warehouse, e.g., BigQuery SQL) or ETL (pre-load, e.g., Dataflow).

Must-know

• Apache Beam's unified model lets the same pipeline code run in both batch and streaming mode on Dataflow, simplifying reprocessing and code reuse.
• Dataflow autoscaling and dynamic work rebalancing handle uneven data distribution and late-arriving data using windowing, watermarks, and triggers for streaming transformations.
• BigQuery supports in-place ELT transformations via SQL, scheduled queries, and materialized views, avoiding the need to move data out for many transformation tasks.
• Dataproc is preferred when migrating existing Spark/Hadoop jobs with minimal code changes, but requires more cluster management than serverless options like Dataflow.
• Cloud Data Fusion provides a visual, plugin-based interface for building ETL/ELT pipelines without writing code, useful for teams without deep programming expertise.
• For side inputs, joins, or deduplication in streaming pipelines, careful windowing and state management in Beam/Dataflow are essential to avoid incorrect or incomplete results.

Coverage checked against the published exam guide on Aug 11, 2026.

These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.