Cleaning bronze data into silver tables with PySpark and SQL

In the medallion architecture, silver tables clean and conform bronze data by removing nulls, deduplicating, and enforcing correct data types before further transformation. This step typically uses PySpark DataFrame operations or SQL to read from a bronze Delta table, apply cleaning logic, and write the result as a new managed or external Delta table.

Must-know

• Use dropna()/na.drop() or SQL WHERE col IS NOT NULL to remove nulls, and fillna()/na.fill() or COALESCE to impute default values instead.
• Use dropDuplicates() or SQL DISTINCT/ROW_NUMBER() window logic to remove duplicate records, since bronze data often contains raw, duplicated ingested events.
• Use cast() in PySpark or CAST()/:: in SQL to standardize data types (e.g., string to timestamp, string to decimal) since bronze tables often store all fields loosely typed or as strings.
• Write cleaned output with df.write.format('delta').mode('overwrite'|'append').saveAsTable() or CREATE TABLE ... AS SELECT (CTAS) to persist the silver table.
• Column pruning and renaming (select, withColumnRenamed, alias) commonly happen at this stage to conform to a clean, documented schema for downstream consumers.
• Schema enforcement is automatic with Delta Lake, so mismatched types on write will throw an error unless mergeSchema or overwriteSchema options are explicitly set.

Coverage checked against the published exam guide on Jul 27, 2026.

These are independent practice questions, written against this certification's published exam guide. They are not the certification vendor's own questions, and not the real exam.