Preparing data for feature engineering and model training and serving

Preparing data for ML on Google Cloud centers on BigQuery ML, which lets you build, train, and serve models directly on data stored in BigQuery using SQL, minimizing data movement and simplifying feature engineering. Google Cloud emphasizes consistent preprocessing between training and serving to avoid training-serving skew, and provides built-in functions and clauses in BigQuery ML to encode this logic once.

1 · Learn the must-know

• 1The TRANSFORM clause in CREATE MODEL lets you define feature preprocessing (e.g., ML.STANDARDIZE, ML.BUCKETIZE, ML.FEATURE_CROSS) that is automatically reapplied during ML.PREDICT and ML.EVALUATE, preventing training-serving skew.
• 2BigQuery ML automatically splits data into training and evaluation sets by default, but you can control this explicitly via the DATA_SPLIT_METHOD and DATA_SPLIT_COL options for reproducible, leakage-free splits.
• 3Categorical columns in BigQuery ML are automatically one-hot encoded unless you specify otherwise, so explicit encoding steps are often unnecessary for basic models.
• 4ML.EVALUATE, ML.PREDICT, and ML.EXPLAIN_PREDICT are the standard functions to assess and serve predictions from a trained BigQuery ML model without exporting data elsewhere.
• 5For non-SQL or more complex feature pipelines, Vertex AI Feature Store (now part of Vertex AI Feature Store / Feature Registry) is the managed service for storing, sharing, and serving consistent features across training and low-latency online serving.
• 6Highly imbalanced label data and missing/null values are common gotchas: BigQuery ML requires you to handle nulls and consider class weighting (e.g., CLASS_WEIGHTS option) before or during model creation to get reliable results.

2 · Check your understanding