Counter-AI Tool System Design for AI System Adversarial Testing and Evaluation

This work consists of the initial recommendations and conclusions found while soliciting functional requirements for the research, design and development of a Counter-AI Tool for conducting adversarial testing and evaluation of artificial intelligence (AI) systems. The report includes a literature review of relevant AI concepts and extensive research within the adversarial AI domain. An intensive stakeholder analysis, consisting of requirement elicitation from over twenty governmental and non-governmental organizations, assisted in determining what functional requirements should be included in the system design of a Counter-AI Tool. The subsequent system architecture diagram takes user input, tests for various types of adversarial AI attacks, and outputs the vulnerabilities of the AI model. Prior to the operationalization of this tool, iterative experimentation will be conducted by partner organizations, which is the next step in the development and deployment of this Counter-AI Tool.
Adversarial AI, System Design Architecture, AI Security, AI Resiliency, Testing and Evaluation
Nathan Byington, Carter Davis, Matthew Meehan, Caroline Vincent, David Woodward, and Nathaniel Bastian. "Counter-AI Tool System Design for AI System Adversarial Testing and Evaluation". Proceedings of the Annual General Donald R. Keith Memorial Conference, 2022.