← Back to Blog

Research RadarFace RecognitionarXivOctober 2026

Monthly arXiv Radar

October 2026 Face Recognition Papers: Hard-Sample Attention, Data Audits, and Thermal Scale

October's face recognition work looks beyond a single leaderboard score. ViT-HarDis changes how a transformer is regularized during training, the MS1MV3 audit measures which data repairs actually help, and VTF900 tests cross-spectral matching at a scale where attractive AUC values can conceal weak operating-point performance.

What This Month Signals

ViT-HarDis directs attention-head decorrelation toward identity-ambiguous samples and leads the compared ViT systems on IJB-B, IJB-C, and TinyFace without changing inference. A controlled MS1MV3 audit shows that removing 15.44% duplicate images slightly improves accuracy and reaches baseline performance with 23.7% fewer steps, while broad filtering is not automatically beneficial. VTF900 then expands visible-thermal evaluation to 900 subjects: a 97.68% AUC corresponds to only 51.10% TAR at 1% FAR in the frontal setting, and off-pose performance falls further. Together, the papers argue that training dynamics, data defects, gallery scale, and operating thresholds must be evaluated separately.

Paper 012026-10-30cs.CV

Hard Sample-Conditioned Decorrelated Attention for Robust Face Recognition

Authors & Institutions

Tahar Chettaoui

Fraunhofer Institute for Computer Graphics Research IGD

Guray Ozgur

Fraunhofer Institute for Computer Graphics Research IGD

TU Darmstadt

Eduarda Caldeira

Fraunhofer Institute for Computer Graphics Research IGD

TU Darmstadt

Naser Damer

Fraunhofer Institute for Computer Graphics Research IGD

TU Darmstadt

Fadi Boutros

Fraunhofer Institute for Computer Graphics Research IGD

What Problem It Solves

ViT-HarDis targets redundant attention specifically where the embedding is dangerously close to another identity. It is designed to improve large-scale and low-resolution face recognition while keeping inference latency, model shape, and embedding interfaces unchanged.

Key Result

On the controlled ViT-S ablation, TAR at FAR 1e-4 rises from 84.46% to 85.72% on IJB-B and from 88.18% to 89.09% on IJB-C. With ViT-B trained on WebFace4M, the method reports 95.84% on IJB-B, 97.24% on IJB-C, and 75.97% TinyFace Rank-1, leading the compared ViT-based systems. Inference cost is unchanged because the penalty is training-only, although the authors report roughly 30% longer training on WebFace4M.

Abstract

Multi-head attention in Vision Transformers (ViTs) allows independent attention patterns across heads, but without explicit regularization, heads may learn redundant patterns rather than complementary roles. Correlation-based penalties encourage head disentanglement, yet apply uniform regularization across all training samples. In face recognition (FR), hard samples whose embeddings lie close to competing identities are critical for optimization and generalization. Motivated by this, we propose ViT-HarDis, which conditions attention head disentanglement on sample difficulty: samples with high cosine similarity to negative class centers receive stronger head decorrelation, enforcing specialization where identity ambiguity is highest. Experiments show improvements over both the baseline and uniform disentanglement on most benchmarks. Our analysis shows that ViT-HarDis reduces the similarity of hard samples to competing class centers while decreasing inter-head attention similarity, and that applying disentanglement only to the final transformer layers outperforms regularizing the entire network. Trained on WebFace4M, ViT-HarDis achieves SOTA results among ViT-based methods on IJB-B, IJB-C, and TinyFace Rank-1, while achieving the best IJB-B and TinyFace results when trained on MS1MV3, improving FR without architectural modifications or additional inference cost. Code: https://github.com/TaharChettaoui/Hard-Sample-Conditioned-Attention-for-FR.

Research Starting Point

Vision Transformers can allocate different facial regions to different attention heads, but those heads often converge on redundant patterns. Uniform decorrelation penalties treat an easy frontal portrait and an identity-ambiguous hard sample alike, even though difficult samples near competing class centers are the ones most likely to determine verification errors. The paper asks whether training capacity can be focused on those difficult identities without changing the deployed architecture.

Method

The method measures sample hardness from cosine similarity to the closest negative class center. For each selected transformer block it computes an inter-head attention-correlation penalty, weights that penalty more strongly for harder correctly classified samples, and adds it to the normal margin-based face-recognition loss. Experiments compare the baseline, uniform disentanglement, and hardness-conditioned disentanglement; layer ablations show that regularizing only the final quarter of transformer blocks is more effective than applying the constraint everywhere.

Paper Summary

For teams already using a ViT face encoder, this is a training-time improvement rather than a deployment rewrite. The gains are strongest on difficult large-scale and low-resolution tests, but they come with extra training time and still need validation on the target population.

Paper 022026-10-20cs.CV

Different Defects, Different Effects: A Controlled Audit of Face Recognition Training Data

Authors & Institutions

Ana Dias

University of Beira Interior

Instituto de Telecomunicacoes

NOVA LINCS

João Ribeiro Pinto

Amadeus

Hugo Proença

University of Beira Interior

Instituto de Telecomunicacoes

João C. Neves

University of Beira Interior

NOVA LINCS

What Problem It Solves

The study separates five defect classes and measures the effect of correcting each one under the same IResNet-50 and ArcFace training protocol. This lets data owners distinguish interventions that improve training efficiency from broad filtering that can accidentally remove informative samples.

Key Result

The best duplicate-removal setting deletes 15.44% of images, raises average validation accuracy from 97.20% to 97.27%, and reaches baseline performance with 23.7% fewer optimization steps. A matched random removal reaches only 97.18%, so dataset shrinkage alone does not explain the result. Merging split identities retains all images while reaching baseline performance with 21.8% fewer steps; removing invalid content helps similarly, while combining every validity filter does not automatically produce the best model.

Abstract

The development of face recognition has been strongly supported by the availability of large-scale training datasets. Among them, MS-Celeb-1M has played an important role by providing millions of facial images from nearly 100K identities. However, the size and heterogeneity of such datasets increase the likelihood of annotation errors, redundant samples, and other forms of data inconsistency. This motivated the introduction of refined versions, such as MS1MV2 and MS1MV3, aimed at improving the quality of the training data. Despite these cleaning efforts, we show that several types of training-data defects remain in MS1MV3. By proposing complementary data-auditing strategies, we identify five defect categories: exact and near-duplicate images, split identities, crop errors, multiple-face ambiguity, and invalid content. We then assess the effect of correcting these defects under a controlled face-recognition training protocol based on IResNet-50 and ArcFace. The results show that targeted interventions can preserve, and in some cases improve, recognition performance while requiring fewer optimization steps to reach the performance of the original training set. Duplicate removal provides the clearest overall benefit, while split-identity merging and invalid-content removal also improve training efficiency. Overall, the results indicate that the effect of dataset cleaning depends strongly on the type of defect being addressed and provide further evidence that training-data quality remains an important and comparatively underexplored factor in large-scale face recognition. The training manifests and code used in this work will be released upon acceptance.

Research Starting Point

Large face-recognition datasets are usually described as clean or noisy in aggregate, yet duplicates, split identities, bad crops, multiple faces, and invalid images create different supervision errors. MS1MV3 is already a refined benchmark, so remaining defects provide a useful test of whether another cleaning pass improves recognition or merely removes useful diversity.

Method

Exact and near duplicates are found with perceptual hashes; split identities are discovered using identity-level representations followed by image verification; crop errors and multiple-face ambiguity use geometric and OFIQ checks; invalid content is identified with BLIP visual question answering. The authors create controlled dataset variants, include a random-removal control matched by identity and image count, train each variant identically, and evaluate both validation accuracy and early checkpoints on IJB-B and IJB-C.

Paper Summary

The operational lesson is to audit defect types separately and measure learning efficiency as well as final accuracy. Duplicate removal and identity repair look worthwhile here, while indiscriminate quality filtering is not a guaranteed win.

Paper 032026-10-17cs.CV

VTF900: A 900-Subject Visible-Thermal Face Recognition Dataset and Benchmark

Authors & Institutions

Jacob Abaare

University of Nebraska-Lincoln

Bianca Ianetta

University of Nebraska-Lincoln

Cedric Nimpa Fondje

AeroVironment

Shuowen Hu

DEVCOM Army Research Laboratory

Benjamin S. Riggan

University of Tennessee, Knoxville

What Problem It Solves

VTF900 supplies a 900-subject paired visible-thermal benchmark for verification and closed-set identification. It is intended to reveal how cross-spectral methods scale when training data comes from a different collection and when probe pose, expression, and occlusion vary.

Key Result

At the full 900-person gallery, ViT-B reaches 97.68% AUC but only 51.10% TAR at 1% FAR under frontal conditions. DPIT-ResNet50 is more resilient off pose, with 94.68% AUC and 36.08% TAR at 1% FAR. The widening gap between headline AUC and operational TAR, together with degradation as gallery size grows, shows why results from approximately 100-person thermal benchmarks do not establish production readiness.

Abstract

Despite advancements in visible-thermal face recognition, where thermal probe (query) face images are matched against visible enrollment templates, existing benchmarks are currently limited by small gallery sizes (N approximately 100 subjects). This limitation inhibits the ability to accurately evaluate visible-thermal facial recognition at scale and to report significant recognition performance metrics, which have better operational relevance. Therefore, we introduce VTF900, a large-scale visible-thermal face recognition benchmark with 900 subjects and paired imagery that includes expression, pose, and occlusion variations. VTF900 is designed to support large-scale (i.e., large gallery) evaluations across verification (1:1 matching) and closed-set identification (1:N matching) for visible-thermal face recognition. Furthermore, VTF900 specifically addresses the challenging combination of cross-spectral and cross-dataset analysis. In this paper, after training models on the ARL-VTF dataset, we provide benchmark analyses across convolutional (baseline), vision transformers, and domain-adaptive neural network architectures to assess how these methods scale. At full gallery scale (N = 900), transformer models achieve strong discriminative performance under frontal conditions (e.g., ViT-B: 97.68% AUC, 51.10% TAR@1%FAR), while domain-adaptive methods such as DPIT-ResNet50 demonstrate greater robustness under off-pose conditions (94.68%AUC, 36.08% TAR@1%FAR). Our results show that high AUC at small scale does not translate to reliable performance at deployment scale, and that pose variation significantly amplifies performance degradation. VTF900 exposes these limitations, enabling more rigorous evaluation of cross-spectral face recognition systems.

Research Starting Point

Visible-to-thermal face recognition is attractive for low-light and nighttime operation, but most public evaluations contain roughly one hundred identities. Small galleries can make AUC look strong while hiding the false matches and rank errors that appear when a system must search a deployment-sized population, especially under pose and sensor shift.

Method

The dataset pairs visible enrollment images with thermal probes for 900 consenting subjects and defines controlled and exhaustive opportunistic protocols. Models are trained on ARL-VTF rather than VTF900, then tested across increasing gallery sizes using convolutional, transformer, and domain-adaptive architectures. The analysis reports verification and identification metrics separately for frontal and off-pose conditions.

Paper Summary

VTF900 is useful for buyers evaluating nighttime identity systems because it tests both sensor mismatch and gallery scale. Its results argue for reporting operating-point and pose-stratified metrics, not AUC alone.