Best Practices for Addressing New Challenges in Testing and Evaluating Artificial Intelligence Enabled Systems

Mark your calendars for March 14th to join us as Dr. Freeman provides her perspective on a currently set of best practices for T&E for AI.  

The integration of artificial intelligence (AI) and statistical machine learning (ML) into complex systems exposes a variety of challenges in traditional test and evaluation (T&E) practices. As more decisions at varying levels are handled by AI-enabled systems (AIES), we need T&E processes that provide a basis for ensuring system effectiveness, suitability, and survivability.  This involves methods for assessing the component ML models and AI algorithms, including the ability to show how they result in repeatable and explainable decisions, as well as an understanding of any failure modes and failure mitigation techniques. Moreover, there is a need for AI assurance to certify that AI algorithms operate as intended and are free of vulnerabilities arising either from faulty design or from adversarially inserted data or algorithm code. In this talk I will share my perspective on a currently set of best practices for T&E for AI.  I will highlight current areas of research and gaps that need to be resolved as we deploy ML/AI in mission critical systems.