Picking the best LLM for an application's own attributes

Choosing an LLM for a GenAI application means matching model attributes—context length, latency/throughput, cost, quality on the target task, and licensing/deployment constraints—to the application's requirements rather than always picking the largest or newest model. On Databricks, this decision also involves whether to use a hosted foundation model via Foundation Model APIs, a fine-tuned/custom model served on Model Serving, or an open-source model registered in Unity Catalog/MLflow.

1 · Learn the must-know

• 1Larger or more capable models generally cost more and have higher latency, so a smaller/cheaper model that meets quality requirements is often the better production choice.
• 2Context window size matters when the application needs to process long documents or maintain long conversation history; exceeding the context window requires chunking or retrieval strategies.
• 3Task type matters: some models are optimized for chat/instruction-following, others for code generation, embeddings, or summarization, so the model family should match the use case.
• 4Licensing and deployment constraints (open-source vs. proprietary, commercial-use terms, data residency/governance needs) can eliminate otherwise strong candidates.
• 5Foundation Model APIs on Databricks let teams query and compare multiple pay-per-token or provisioned-throughput models without managing infrastructure, easing evaluation before committing to one model.
• 6Evaluation should be empirical: use representative prompts/datasets and metrics (accuracy, latency, cost per request) via tools like MLflow evaluation to compare candidate models before final selection.

2 · Check your understanding