Watching a Dataflow job and spotting where it stalls
The Dataflow job UI in Google Cloud Console provides real-time visibility into pipeline execution, letting you track progress, diagnose bottlenecks, and inspect step-by-step performance. It combines a visual execution graph with job metrics, logs, and worker details in a single interface. Associate Data Practitioners should know how to navigate it to monitor both batch and streaming jobs.
Must-know
- The Job Graph tab shows a visual DAG of pipeline stages/transforms, with each box color-coded by status (running, succeeded, failed) and updated in near real-time.
- Clicking any stage in the Job Graph reveals per-step metrics such as elapsed time, input/output element counts, and system lag, which helps pinpoint slow or stuck stages (a common cause is a 'hot key' or unbalanced data skew).
- The Job Metrics tab offers aggregate charts for autoscaling (worker count over time), CPU utilization, memory, and I/O, useful for spotting resource bottlenecks or under/over-provisioning.
- For streaming jobs, watch the 'System Lag' and 'Data Freshness' metrics closely, since rising values indicate the pipeline is falling behind incoming data.
- The Logs panel is integrated with Cloud Logging and can be filtered by severity or worker, letting you correlate errors/exceptions directly with the graph stage that produced them.
- Job status values (Running, Succeeded, Failed, Cancelled, Drained) appear at the top of the job details page, and a failed job's error summary often points directly to the offending stage without needing to dig through raw logs.
A data practitioner is monitoring a streaming Dataflow job in the Google Cloud console and notices that overall pipeline throughput has dropped compared to earlier in the day. They want to identify which specific stage of the pipeline is causing the slowdown before making any changes. Which part of the Dataflow job UI should they examine first?
What you have tried across GCP ADP's objectives, not a readiness score.
Data Preparation and Ingestion
- When to load first and when to transform first, and what sits between the two
- Picking a way to move existing data into Google Cloud
- Judging whether a dataset is trustworthy enough to build on
- Fixing messy records before they reach a report
- Telling CSV, JSON, Parquet, Avro, and relational tables apart, and where each fits
- Picking how to pull data out of a source system
- Matching a workload to the right storage or database service
- Getting files and tables loaded with a CLI, a transfer service, or a client library
Data Analysis and Presentation
- Writing BigQuery SQL that answers a reporting question
- Exploring and charting data inside a hosted notebook
- Turning a question from the business into an analysis that settles it
- Building a dashboard and getting it in front of the right people
- Deciding whether a job calls for Looker or for Looker Studio
- Editing LookML to change what a model exposes
- Spotting a problem worth solving with BigQuery ML or AutoML
- Calling a hosted Google language model straight from BigQuery
- Sequencing a machine learning project from raw data to served predictions
- Building, fitting, and scoring a model with SQL alone
- Running predictions against a model you already trained
- Keeping trained models catalogued in one place
Data Pipeline Orchestration
- Matching a transformation job to Dataproc, Dataflow, Dataform, or a managed alternative
- Weighing whether the transform belongs before or after the load
- Assembling the services a simple transformation pipeline needs
- Putting a query on a schedule and keeping it running
- Watching a Dataflow job and spotting where it stalls
- Reading logs and metrics to work out what a pipeline actually did
- Choosing what should drive a multi-step workflow
- Streaming messages into BigQuery as they arrive rather than in batches
- Wiring a trigger so one event starts the next step
Data Management
- Granting only the access a person or service actually needs
- Controlling who can read a bucket, and what uniform access changes
- Sharing a dataset with another team or company without copying it
- Matching a storage class to how often the data gets read
- Expiring old data automatically so it stops costing money
- Picking somewhere to park data that must be kept but is rarely read
- Comparing the managed backup and restore options across services
- Working out when a second copy is worth what it costs
- Regions, dual-regions, multi-regions, and zones as redundancy choices
- Deciding who should hold the encryption keys
- What a key management service does for creating, rotating, and revoking keys
- Protecting data on the wire versus data sitting on a disk
Coverage checked against the published exam guide on Aug 12, 2026.
These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.