FB pixel

NIST introduces program to analyze real-world AI risks

NIST introduces program to analyze real-world AI risks
 

It is critical to evaluate the societal implications of using artificial intelligence, especially in situations where people regularly engage with AI. 

The National Institute of Standards and Technology (NIST) has introduced a new initiative, the Assessing Risks and Impacts of AI (ARIA) program, designed to assess the interactions between AI systems and human users to understand how these systems impact and are influenced by social factors. 

According to NIST, the program will offer measurable metrics for assessing the performance of these systems, with a focus on aspects such as validity, reliability, safety, security, privacy, and fairness. The objective is to support the U.S. AI Safety Institute‘s endeavors to establish reliable AI systems.

“This new effort will support the U.S. AI Safety Institute, expand NIST’s already broad engagement with the research community, and help establish reliable methods for testing and evaluating AI’s functionality in the real world,” says Laurie E. Locascio, NIST director and under secretary of Commerce for Standards and Technology.

The initiative includes three tiers of assessment: model testing, red teaming, and field testing. Model testing aims to verify the capabilities of AI models, while red teaming focuses on stress testing applications to identify vulnerabilities. Field testing is used to examine how individuals interact with AI in everyday scenarios. Some specific instances where AI models will be evaluated include TV spoilers, meal planning, and pathfinding.

Looking ahead, NIST anticipates expanding the ARIA evaluation to add generative AI technologies, such as text-to-image models, as well as other AI methods like recommender systems or decision support tools.

One of NIST’s most well-known contributions to the field of AI is the ongoing evaluation of face biometrics and analysis models.

“ARIA will consider AI beyond the model and assess systems in context, including what happens when people interact with AI technology in realistic settings under regular use. This gives a broader, more holistic view of the net effects of these technologies,” says Reva Schwartz, NIST Information Technology Lab’s ARIA program lead.

The ARIA program builds on the AI Risk Management Framework released in January 2023. The framework outlines methods for analyzing and monitoring the risks and impacts of AI systems using both quantitative and qualitative techniques. The framework was developed to guide companies in managing the risks associated with AI systems.

Related Posts

Article Topics

 |   |   |   |   | 

Latest Biometrics News

 

Physical, digital spoof detection for ID documents upgraded by IDScan.net

IDScan.net is introducing new tools to detect more instances of tampering on identity documents. The new features to detect physical…

 

Social media giants face the wrath of new Australian government committee

Australia continues to harden its stance on social media. In a release from the office of Minister for Communications Michelle…

 

New Zealand trust framework looks to onboard digital ID providers

In an interview with Radio New Zealand (RNZ), Digital Identity New Zealand Executive Director Colin Wallis confirms that the country’s…

 

Google introduces passkeys for users at high risk of cyberattacks

Google users now have the option to use passkeys for account security in signing up for its Advanced Protection Program…

 

EU residency permits to exempt Brits from biometric checks at UK border

The UK government has confirmed its intention to implement the changes necessary to accommodate the European Union’s new digital Entry/Exit…

 

DHS to hold industry day for biometric scanner contract

A hybrid industry day will be held by the U.S. Department of Homeland Security on July 29 to share information…

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Most Read This Week

Featured Company

Biometrics Insight, Opinion

Digital ID In-Depth

Biometrics White Papers

Biometrics Events