Microsoft on Thursday announced the release of PyRIT, an open access red teaming tool designed to help security professionals and ML engineers identify risks in generative AI.
PyRIT, Microsoft says, increases audit efficiency by automating tasks and flagging areas that require further investigation, essentially augmenting manual red teaming.
Red teaming generative AI, the tech giant notes, is different from probing classical AI systems or traditional systems, mainly because it requires identifying both security risks and responsible AI risks, generative AI is more probabilistic, and due to the wide variations in generative AI system architectures.
Generative AI could produce ungrounded or inaccurate content and its output is influenced even by small input variations, and red teaming these systems needs to consider these risks as well.
Furthermore, generative AI systems may vary from stand-alone applications to integrations, and their output may vary greatly as well, Microsoft notes.
PyRIT (Python Risk Identification Toolkit for generative AI), which started in 2022 as a set of scripts for red teaming generative AI, has already proven its efficiency in red teaming various systems, including Copilot.
“PyRIT is not a replacement for manual red teaming of generative AI systems. Instead, it augments an AI red teamer’s existing domain expertise and automates the tedious tasks for them. PyRIT shines light on the hot spots of where the risk could be, which the security professional can incisively explore,” Microsoft explains.
The tool provides the user with control over the strategy and execution of the AI red team operation, can generate additional harmful prompts based on the set it was fed with, and changes tactics based on the responses received from the generative AI system.
PyRIT includes support for various generative AI target formulations, can be fed a dynamic prompt template or a static set of malicious prompts, provides two options for scoring the target system’s outputs, supports two styles of attack strategy, and can save intermediate input and output interactions for follow-up analysis.
“PyRIT was created in response to our belief that the sharing of AI red teaming resources across the industry raises all boats. We encourage our peers across the industry to spend time with the toolkit and see how it can be adopted for red teaming your own generative AI application,” Microsoft notes.
PyRIT is available on GitHub.
Related: Google Open Sources AI-Aided Fuzzing Framework
Related: Critical Vulnerabilities Found in Open Source AI/ML Platforms
Related: AI’s Future Could Be Open-Source or Closed. Tech Giants Are Divided as They Lobby Regulators
