Automated generation of test inputs has seen widespread adoption, yet the challenge of producing correct and meaningful test oracles remains largely unsolved. In particular, automatically inferring assertions that reflect the intended behavior of a program is difficult in the absence of formal specifications or high-level semantic information. We present AugmenTest, a framework that leverages Large Language Models (LLMs) to synthesize JUnit assertions using only natural language artifacts such as documentation and comments. By treating assertion inference as a prompt-driven task, AugmenTest enables context-aware oracle generation that scales across a wide range of software systems without requiring source code access or dynamic instrumentation. The architecture is model-agnostic and supports multiple LLM backends, prompt strategies, and validation workflows. We describe the design and capabilities of AugmenTest, along with its application to a curated benchmark of real-world Java test cases. The tool, datasets, and artifacts are publicly released to support reproducibility and ongoing research in LLM-assisted software testing.
AugmenTest: A tool for improving test quality through automatic assertion generation
Khandaker, Shaker Mahmud;Kifetew, Fitsum;Prandi, Davide;Susi, Angelo
2026-01-01
Abstract
Automated generation of test inputs has seen widespread adoption, yet the challenge of producing correct and meaningful test oracles remains largely unsolved. In particular, automatically inferring assertions that reflect the intended behavior of a program is difficult in the absence of formal specifications or high-level semantic information. We present AugmenTest, a framework that leverages Large Language Models (LLMs) to synthesize JUnit assertions using only natural language artifacts such as documentation and comments. By treating assertion inference as a prompt-driven task, AugmenTest enables context-aware oracle generation that scales across a wide range of software systems without requiring source code access or dynamic instrumentation. The architecture is model-agnostic and supports multiple LLM backends, prompt strategies, and validation workflows. We describe the design and capabilities of AugmenTest, along with its application to a curated benchmark of real-world Java test cases. The tool, datasets, and artifacts are publicly released to support reproducibility and ongoing research in LLM-assisted software testing.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
