NIST AI Risk Management Framework

Policies, processes, procedures, and practices across the organization related to the mapping, measuring, and managing of AI risks are in place, transparent, and implemented effectively.

Accountability structures are in place so that the appropriate teams and individuals are empowered, responsible, and trained for mapping, measuring, and managing AI risks.

Organizational teams are committed to a culture that considers and communicates AI risk. Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization.

The risk or impact tolerance of the organization or context is established, communicated, and updated. Organizational risk tolerance for AI risks is documented and applied.

Organizational teams are committed to a culture that considers and communicates AI risk. Policies and procedures are in place to address AI risks and benefits arising from third-party software and data and other supply chain issues.

Organizational policies and practices are in place to foster a critical thinking and safety-first mindset in the design, development, deployment, and uses of AI systems to minimize potential negative impacts.

Context is established for the AI risk assessment. The organizational mission and relevant AI policies, processes, and procedures are understood. Intended purpose, potentially beneficial uses, context of use, and assumptions are documented.

Organizational risk tolerances are determined and documented. The AI system to be deployed presents risks that fall within organizational risk tolerances.

The scientific basis of the claimed or potential benefits and costs of the AI system are understood. Potential costs, including harms, are properly identified and documented.

Likelihood and magnitude of each identified impact based on expected use, past uses of AI systems in similar contexts, public incident reports, feedback from those external to the team that developed or deployed the AI system, or other data are identified and documented.

Approaches and metrics for measurement of AI risks, impacts, and related harms are identified and documented. Measurement approaches for identifying AI risks are aligned with the AI risk management framework.

Test sets, metrics, and details about the tools used during test, evaluation, validation, and verification (TEVV) are documented. AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment setting(s).

The AI system to be deployed is demonstrated to be valid and reliable through systematic evaluation. AI system performance, or assurance criteria, are evaluated against established metrics. Bias testing is performed.

A determination is made as to whether the AI system achieves its intended purposes and stated objectives and whether its development or deployment should proceed. Risks or other undesirable impacts are treated, transferred, accepted, or avoided.

Resources required to manage AI risks are taken into account – along with viable non-AI alternative systems, approaches, or methods – to reduce the magnitude or likelihood of potential impacts. Mechanisms are in place to inventory AI systems and their associated risks.

Post-deployment AI system monitoring plans are in place and are implemented, including mechanisms for capturing and evaluating input from users and other relevant AI actors. Residual risks not identified in pre-deployment testing are documented.