Risk-Driven Compliance Assurance for Collaborative AI Systems: A Vision Paper

Context and motivation. Collaborative AI systems aim at working together with humans in a shared space. Building these systems, which comply with quality requirements, domain specific standards and regulations is a challenging research direction. This challenge is even more exacerbated for new generation of systems that leverage on machine learning components rather than deductive (top-down programmed) AI. Question/problem. How can requirements engineering, together with software and systems engineering, contribute towards the objective of building flexible and compliant collaborative AI with strong assurances? Principal idea/results. In this paper, we identify three main research directions: automated specification and management of compliance requirements, and their alignment with assurance cases; risk management; and risk-driven assurance methods. Each one tackles challenges that currently hinder engineering processes in this context. Contributions. This vision paper aims at fostering further discussion on the challenges and research directions towards appropriate methods and tools to engineer collaborative AI systems in compliance with existing standards, norms, and regulations.