How are biometric systems evaluated?

Key metrics explained

May 20, 2024, 2:13 am EDT | Abhishek Jadhav

The evaluation metrics used to assess the performance of biometric systems are important but not necessarily easy to understand.

Evaluation itself is an increasingly necessary step for biometrics market participation. Continuous performance evaluation allows the operator to improve and optimize the biometric system over time. It helps identify weaknesses and areas for enhancement while meeting the regulatory requirements for security and privacy.

– False Acceptance Rate (FAR) measures the likelihood of the system accepting an unauthorized individual. It is also referred to as False Positive Identification Rate (FPIR).

Where FP is the number of false positives (unauthorized access granted), and TN is the number of true negatives (correctly denied access).

– False Rejection Rate (FRR) measures the likelihood of rejecting an authorized individual. It is also referred to as False Non-Match Rate (FNIR).

Where FN is the number of false negatives (authorized access denied), and TP is the number of true positives (correctly granted access).

– The rate at which both the FAR and FRR are equal is the Equal Error Rate (EER). It provides a value representing the point where both types of errors balance each other.

– False Match Rate (FMR) refers to the proportion of unauthorized access attempts that are falsely declared to match a template of another subject.

– False Non-Match Rate (FNMR) is the proportion of authorized access attempts falsely reported as non-match for a template of the same subject.

Note: When it comes to biometric authentication, the False Rejection Rate (FRR) and False Acceptance Rate (FAR) are commonly referred to as FNMR and FMR. Despite this, it’s important to note that they are not interchangeable. Specifically, FMR is equivalent to FAR and FNMR is equivalent to FRR, but only when the system utilizes a single attempt from a user to match their stored template.

– False Positive Identification Rate (FPIR) measures the probability that a biometric system incorrectly identifies an individual who is not in the database as being a match. The FPIR is approximately equal to the FMR in a one-to-one verification system multiplied by the number of entries in the database.

– False Negative Identification Rate (FNIR) measures the probability that a system fails to correctly identify a registered user. According to the U.S. National Institute of Standards and Technology, the probability of a system failing to identify a match in a 1:N setting approximates the probability of failing to verify a match in a 1:1 (FNMR 1:1) setting under similar conditions.

The well-known facial recognition vendor evaluations by NIST measure FNIR with the FPIR set at a given threshold.

– Failure To Capture (FTC) occurs when the biometric system is unable to capture the sample. This is the percentage of time when the system fails to capture the biometric characteristics.

– Failure To Enrol (FTE) is the percentage of time when users are unable to enroll in the system.

– Attack Presentation Classification Error Rate (APCER) measures the system’s accuracy in detecting and identifying presentation attacks.

– Bona Fide Classification Error Rate (BPCER) measures how well the system avoids false alarms.

– Impostor Attack Presentation Match Rate (IAPMR) assesses the vulnerability of biometric systems to impostor attacks, where unauthorized users attempt to gain access.

Biometric presentation attack detection (PAD) evaluations based on the ISO/IEC 30107-3 standard set limits for the BPCER and FNMR and test the APCER when evaluating subsystems. For full PAD systems, IAPMR is tested.

– Receiver Operating Characteristic (ROC) curve is a method used to evaluate the performance of a biometric system with the objective of creating a curve that represents FAR in relation to FRR.

Performance evaluation has become a fundamental aspect of ensuring that biometric systems are reliable, secure, and suitable for their intended applications.