"Doctor, Is It Normal?" Enabling Medical Chatbots to Provide Certified Replies to Normalcy Questions.

This paper presents a work in progress to enhance a Retrieval-Augmented Generation (RAG) pipeline for a medical chatbot designed to address evaluative questions related to patient concerns about “normalcy”. The chatbot uses a novel approach called Hypothetical Document Embeddings (HyDoc) to augment queries and improve the retrieval of certified medical information. In the first evaluation of the chatbot, it emerged that evaluative queries often fail to retrieve relevant documents as well as to produce appropriately framed responses. We, therefore, experiment with the impact of an additional naive-RAG module to improve the retrieval and a Chain-of-Thought (CoT) inspired prompting strategy to contextualize the queries better and advance response generation. Results demonstrate that this method enhances document retrieval and the framing of generated replies, improving the chatbot’s ability to generate responses that consider emotional and communicative aspects.