Building agentic systems with MLflow and Agent Framework

MLflow and the Databricks Mosaic AI Agent Framework work together to help engineers build, trace, evaluate, and deploy agentic systems in a standardized, production-ready way. MLflow provides logging, tracing, and registry capabilities, while the Agent Framework supplies standardized agent interfaces and evaluation tooling built on top of it.

1 · Learn the must-know

• 1Use MLflow's 'Models from Code' logging approach to log agent code (e.g., built with LangChain, LlamaIndex, or plain pyfunc) directly as a file rather than pickling the object, which avoids serialization issues and improves reproducibility.
• 2MLflow Tracing (via mlflow.trace or framework autologging) captures step-by-step execution details—LLM calls, retriever calls, tool invocations—giving visibility into agent reasoning for debugging and observability.
• 3Agent Framework defines standardized agent authoring interfaces (pyfunc-based, e.g., ChatAgent) so agents expose consistent inputs/outputs and support multi-turn conversation, tool-calling, and streaming responses.
• 4Agents should be registered to Unity Catalog through the MLflow Model Registry before being deployed to a Mosaic AI Model Serving endpoint, enabling governed lifecycle management and scalable serving.
• 5Agent Evaluation, part of the Agent Framework, uses LLM-judge metrics and custom metrics to assess response quality, groundedness, cost, and latency, and integrates with the Review App to collect human feedback on agent outputs.
• 6A common gotcha is deploying an agent without validating its input/output signature locally (e.g., via mlflow.models.predict); mismatched signatures often cause failures only at serving time, not at logging time.

2 · Check your understanding