Calling a hosted Google language model straight from BigQuery

BigQuery lets you call Google's pretrained large language models directly from SQL by creating a remote model object that points to a Vertex AI endpoint through a BigQuery Cloud resource connection. This enables generative AI tasks like text generation and embeddings without moving data out of BigQuery or standing up separate ML infrastructure.

1 · Learn the must-know

• 1You must first create a BigQuery Cloud resource connection (CREATE CONNECTION) which provisions a service account used to authenticate to Vertex AI.
• 2The connection's service account needs the Vertex AI User IAM role (or equivalent permissions) granted before the remote model will work.
• 3A remote model is created with CREATE MODEL ... REMOTE WITH CONNECTION, specifying the connection and the target pretrained endpoint (e.g., a Gemini model).
• 4Once the remote model exists, ML.GENERATE_TEXT is used to generate text output and ML.GENERATE_EMBEDDING is used to produce vector embeddings from a table's text column, both via standard SQL.
• 5The BigQuery dataset, the connection, and the Vertex AI endpoint region must be compatible (often same region) or model creation/inference will fail.
• 6Calls to these remote LLM functions incur Vertex AI prediction charges in addition to normal BigQuery query costs, so usage should be monitored.

2 · Check your understanding