NeurIPS 2024: New research tackles biometric security, bias and model transparency

Three of the papers selected for presentation at Safe Generative AI Workshop at NeurIPS 2024, held in Vancouver, Canada tackle challenges and advancements in the field of biometrics.

The highlighted studies cover methodologies in biometric model interpretation, explore the weaknesses and security risks in identity verification, and investigate bias concerns within facial recognition models. Two of the papers were written by researchers from the Idiap Research Institute in Switzerland.

The papers in brief

The paper titled “Model Pairing Using Embedding Translation for Backdoor Attack Detection on Open-Set Classification Tasks” introduces “Model Agnostic Prototypical Explanations” (MAPE) as a novel approach to interpreting AI behavior in biometric systems. The authors propose using prototypical examples to explain and interpret the decisions of black-box biometric models, making it possible for users to better understand why certain biometric predictions are made.

According to the paper, the MAPE approach uses a variety of model-agnostic methods to analyze and evaluate explanations. These methods aim to achieve both accuracy and transparency by comparing embedding representations of biometrics, allowing for a more interpretable similarity score across predictions. MAPE leverages prototypes as benchmarks for comparison, generating a similarity score that reveals how closely a prediction aligns with previously identified patterns.

The framework holds promise for enhancing trustworthiness in biometric systems by making model outputs more interpretable. These interpretations could be particularly impactful in applications that demand high-stakes biometric verification, such as border security or financial services, where explainable AI can help decision-makers verify and trust model outputs.

The paper titled “HyperFace: Generating Synthetic Face Recognition Datasets by Exploring Face Embedding Hypersphere” examines biometric systems’ robustness by analyzing how different AI models can be compromised by adversarial inputs. The study focuses on various attack types, including noise-based and pattern-distorting attacks, to determine how easily biometric systems could be deceived.

The authors, anonymized for double-blind review, present a taxonomy of attacks and countermeasures to help enhance security in biometric systems, evaluating the effectiveness of different adversarial defense strategies in a range of use cases. By simulating attacks on facial recognition and fingerprint identification systems, they demonstrate both the vulnerabilities present and the potential for resilient designs through strategic modifications.

A third paper, “Unveiling Synthetic Faces: How Synthetic Datasets Can Expose Real Identities,” sheds light on a long-standing issue in biometrics: cross-regional bias in facial recognition models. Bias in biometric systems can lead to inequitable outcomes, particularly in facial recognition where certain demographic groups may be disproportionately misidentified. The paper explores how facial recognition models trained on data from specific regions often underperform when applied to individuals from other geographical areas, highlighting an issue of fairness that can impact global deployments of this technology.

The authors conducted experiments with multiple face datasets, scrutinizing the accuracy of regionalized face recognition models and examining how discrepancies arise across different populations. Their findings reveal that the region in which a model is trained has a significant impact on its accuracy across diverse populations, leading to systemic biases that are often overlooked in model evaluations.

Implications and future directions

These papers, spotlighted by Sebastien Marcel, professor and senior researcher in biometrics security and privacy, collectively address some pressing issues in biometrics today: explainability, security, and fairness. By advancing model-agnostic explanations, defending against adversarial attacks, and tackling cross-regional bias, researchers are pushing biometric technology toward a more transparent future.

The Safe Generative AI Workshop at NeurIPS 2024 will be held December 14 and 15.

Article Topics

biometric-bias  |  biometrics  |  biometrics research  |  demographic fairness  |  explainability  |  facial recognition  |  identity verification  |  Idiap  |  synthetic data