Spotting quality and safety issues in a model's responses

When developing LLM-powered applications, engineers must qualitatively assess model responses to catch issues before they reach production, since automated metrics alone often miss subtle quality and safety problems. This assessment typically happens during interactive development and testing, using direct inspection of outputs against a rubric of known failure modes. Recognizing these patterns early helps engineers decide whether to adjust prompts, retrieval logic, or model parameters.

1 · Learn the must-know

• 1Hallucination is a key quality issue to watch for: the model generates plausible-sounding but factually incorrect or unsupported content, which is especially risky in RAG applications where responses should be grounded in retrieved context.
• 2Toxicity and harmful content (biased, offensive, or inappropriate language) are safety issues that should be manually reviewed for, particularly when testing edge-case or adversarial prompts.
• 3Responses should be checked for relevance and coherence—whether the answer actually addresses the user's question and stays logically consistent, rather than being off-topic or contradictory.
• 4Verbosity or conciseness issues matter for user experience: overly long, repetitive, or overly terse responses indicate the prompt or system instructions may need tuning.
• 5For RAG systems specifically, engineers should qualitatively check groundedness—whether the response is actually supported by the retrieved context—and note when the model ignores retrieved documents in favor of its own prior knowledge.
• 6Qualitative spot-checking of a diverse sample of inputs (including edge cases) is a necessary complement to quantitative evaluation metrics, since metrics can look acceptable while specific failure modes go undetected.

2 · Check your understanding