Definition
The risk also applies to responses written fluently and convincingly.
An AI model hallucination occurs when a model generates a convincing response without sufficient grounds or provides information that conflicts with the source. It may invent a fact, combine data incorrectly, cite a nonexistent document, or interpret ambiguous text with too much confidence. Simply choosing a larger model does not eliminate the risk. Controlled context, RAG, rule-based validation, source citations, testing, and designed human involvement reduce it. The greater the impact of the result on a customer, employee, or finances, the stronger the verification must be. Review the security principles for AI solutions.