Reshaping columns, rows, and arrays in a table
Spark SQL and DataFrame APIs provide DDL-style and functional operations to reshape tables: adding/dropping columns, renaming, splitting strings into multiple columns, filtering rows, and exploding array columns into multiple rows. These operations are used constantly when cleaning raw or semi-structured data during ETL/ELT transformations in Databricks notebooks.
Must-know
- ALTER TABLE ... ADD COLUMNS adds new columns; ALTER TABLE ... DROP COLUMN drops one (requires Delta column mapping enabled for DROP COLUMN); withColumn() adds/replaces a column in a DataFrame, while drop() removes one.
- Renaming uses ALTER TABLE ... RENAME COLUMN in SQL or withColumnRenamed() in DataFrames; renaming does not change underlying data, only metadata.
- split(col, pattern) splits a string column into an array, and array elements can be pulled into separate columns via indexing (e.g., split(col,',')[0]) or getItem().
- Filtering uses WHERE in SQL or .filter()/.where() on DataFrames; both accept boolean column expressions and can be chained for multiple conditions.
- explode(
array_col) turns each element of an array (or map) into a separate row, duplicating the other column values across rows; rows with NULL or empty arrays are dropped by explode but kept byexplode_outer. - Column and row operations can be combined in a single SELECT/DataFrame chain, but exploding before filtering (or vice versa) changes row counts and results, so operation order matters.
A data engineer has a table orders with a column items containing an array of struct values (product_id, qty) per order. The team needs one row per product per order, preserving the order_id and all other columns, for downstream aggregation. Which function should the engineer use?
What you have tried across Databricks DEA's objectives, not a readiness score.
Databricks Intelligence Platform
Data Ingestion and Loading
- Batch, streaming, and incremental loading patterns, and where the data comes from
- Loading files from cloud storage into governed tables with COPY INTO
- Landing data with Auto Loader, and handling schema enforcement and evolution
- Setting up Lakeflow Connect to ingest from enterprise sources reliably
- Pulling data through JDBC, ODBC, or REST clients and scheduling the job
- Choosing the right ingestion method for a given volume, frequency, and governance need
- Bringing semi-structured and unstructured data into governed Delta tables
Data Transformation and Modeling
- Cleaning bronze data into silver tables with PySpark and SQL
- Joining and combining DataFrames with the different join and union types
- Reshaping columns, rows, and arrays in a table
- Deduplicating and aggregating DataFrames
- Tuning Spark's core parameters and measuring what changed
- Building Gold-layer views and tables for BI and analytics
- Validating Silver and Gold datasets for quality
Working with Lakeflow Jobs
Implementing CI/CD
- Branching, committing, and opening pull requests from inside the Databricks workspace
- Promoting one codebase across dev, test, and prod with bundle variables and overrides
- Packaging and deploying jobs and pipelines with Automation Bundles
- Validating and managing bundle deployments from the Databricks CLI
Troubleshooting, Monitoring, and Optimization
- Spotting performance trends in a job's run history
- Reading job status, task graphs, and failure rates to monitor pipeline health
- Diagnosing skew, shuffle, and spill from Spark UI stage metrics
- What Liquid Clustering and predictive optimization actually do
- Diagnosing cluster startup failures, library conflicts, and out-of-memory errors
Governance and Security
Coverage checked against the published exam guide on Jul 26, 2026.
These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.