Explainability for Vulnerability Identification in AI Systems

The Research and Development submission to the Strategic Review (SR20) recognised the need to advance MOD's ability to adopt critical and game-changing technology, enabling autonomous systems on the battlefield and in the command space through the use of artificial intelligence. It proposed to do this by establishing a Defence AI Centre with the science and technology component delivered by a Defence AI Centre Experimentation hub (DAIC-X) led by Dstl. A key objective for DAIC-X is to understand and develop good practice in managing AI verification, validation, vulnerabilities as well as wider issues including trust and transparency and legal and ethical considerations. This task will research the potential to exploit artificial intelligence explainability (XAI) methodologies to identify and expose vulnerabilities in neural network-based machine vision algorithms. Please see the attached Tasking Form for further information regarding this award.