Databricks GenAI Engineer · Application Development

A RAG chatbot on Databricks must retrieve context from a Databricks Vector Search index, format it into a prompt template, and call a Foundation Model API endpoint. These steps need to be composed declaratively so each component can be swapped independently and the chain supports streaming output token by token. Which approach satisfies this?

• AWrap all three steps inside a single custom Python class that hardcodes the retriever, prompt, and model calls in sequence
• BUse LangChain's legacy LLMChain class, which supports memory injection but does not natively support streaming
• CUse LlamaIndex's SimpleDirectoryReader to load the vector index and query it directly, bypassing prompt templating
• DCompose the retriever, prompt template, and model using LangChain Expression Language (LCEL) with the pipe (|) operator