Choosing between a persistent cluster and a job-based one

For Dataproc, choose between persistent (long-running) clusters and ephemeral, job-scoped clusters based on workload patterns, cost sensitivity, and isolation needs. Ephemeral clusters are the Google-recommended default for most batch workloads because you pay only for the job's runtime and avoid idle capacity. Persistent clusters remain appropriate for interactive, low-latency, or continuously running workloads (e.g., Spark shells, notebooks, streaming) where cluster startup time is unacceptable.

Must-know

• Ephemeral clusters are created per job (often via Workflow Templates or an orchestrator like Cloud Composer) and deleted immediately after completion, minimizing cost since billing is per-second while nodes run.
• Persistent clusters incur continuous cost even when idle, so they must be actively monitored, autoscaled, or scheduled for start/stop to control spend.
• Storing data and metadata outside the cluster lifecycle (e.g., in Cloud Storage rather than HDFS, and using an external Hive metastore like Dataproc Metastore) is essential for ephemeral clusters so state isn't lost when the cluster is deleted.
• Job-based clusters provide strong workload isolation, each job gets a fresh environment, avoiding resource contention, dependency conflicts, and noisy-neighbor issues common on shared persistent clusters.
• Cluster startup time (typically 1-2+ minutes) is a real tradeoff for ephemeral clusters, making persistent clusters preferable for latency-sensitive interactive or streaming use cases.
• Dataproc Workflow Templates are the standard mechanism to define, parameterize, and automate the create-cluster → run-jobs → delete-cluster pattern for ephemeral clusters.

Coverage checked against the published exam guide on Aug 11, 2026.

These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.