Toward Proactive Dialogic AI Agents

This paper introduces an ongoing research on the development of a proactive dialogic AI agent, focusing on enhancing an LLM’s pragmatic competence in goal-oriented dialogues. We investigate proactivity as a collaborative behaviour that enables to provide relevant and useful information that has not been explicitly requested, thereby improving interaction efficiency and dialogue naturalness. Our approach is grounded in a corpus-based analysis of proactive behaviours in human-human dialogues across five goal-oriented dialogue corpora, leading to the creation of the D-Pro Corpus, a manually annotated resource for studying proactivity. Its analysis provides information on qualitative and distributional features of proactivity in human dialogues, as well as clues on recurrent linguistics structures that co-occur with the display of proactive behaviours. We then leverage the D-Pro Corpus to evaluate the performance of a GPT-4o model in proactivity annotation, addressing the task by providing a 4-turns context size and by targeting the last utterance for proactivity prediction. By experimenting with parameter setting and prompt configurations, we assess the model’s performance across multiple dialogue corpora, obtaining encouraging results toward human-like performance, particularly with the NESPOLE! corpus. We propose to advance our research by exploring the potential of open-source models for cost-effective, large-scale automatic annotation of unlabelled dialogic data. As a final step we plan to use the large-scale annotated corpus to instruction-tune an open model, expanding its pragmatic competence for the development of more proactive and contextually aware dialogic AI system and more natural human-machine conversation.