Building Safe AI Chatbots for Rural Mothers Seeking Breastfeeding Support: Understanding Hallucinations and How to Mitigate Them

Abstract

AI-enabled chatbots are increasingly positioned as a remedy for breastfeeding support gaps in rural maternal health, offering private, immediate assistance amid persistent shortages of lactation specialists and limited access to care. However, their clinical promise remains constrained by the probabilistic nature of large language models, which can generate hallucinations that undermine maternal–infant safety. This article argues that safely integrating AI into breastfeeding support requires treating hallucination not as a singular technical flaw but as a systems-level risk shaped by design, governance, and use context. We identified key risks of AI systems that could result in hallucination such as, false citations, transcription errors, prompt injection and jailbreaking, and incorrect generalization or personalization, and analyze how each error introduces distinct safety vulnerabilities. Drawing from systems thinking, we outline mitigation strategies including retrieval-augmented generation grounded in authoritative breastfeeding sources, layered guardrails, adversarial testing, uncertainty-aware messaging, and domain-specific fine-tuning. By linking AI system design choices to downstream health consequences in resource-constrained settings, this paper reframes AI-assisted breastfeeding support as a governance challenge central to equitable, safe maternal health innovation.