Hiding in Plain Sight: An Effective Physical Adversarial Patch Attack against Visual-Infrared Fused Face Detection
Authors & Institutions
Qiucheng Yu
City University of Hong Kong, Hong Kong, China
Tao Ni
King Abdullah University of Science and Technology, Saudi Arabia
Yihe Zhou
City University of Hong Kong, Hong Kong, China
Jiayimei Wang
City University of Hong Kong, Hong Kong, China
Qingchuan Zhao
City University of Hong Kong, Hong Kong, China
What Problem It Solves
Prior patches transfer poorly to fused detectors because optimization for the visible image can conflict with the infrared response, and conspicuous patterns are easier for people or secondary systems to notice. VIPatch seeks a physically realizable, joint, and visually unobtrusive attack.
Key Result
The digital black-box attack averages 94.59% ASR across visible detectors and 97.40% across infrared detectors. In the physical fused setting, VIPatch reaches 97.27% ASR on YOLOv8-Face and MTCNN and 100% on TFW and OpenCV. Its visible physical ASR remains 80.29% in 12,000-lux light and ranges from 92.51% to 100% across the tested one-to-three-meter distances.
Abstract
Deep learning-based visual-infrared fused face detection models are increasingly deployed across a wide range of applications, yet they remain susceptible to adversarial patch attacks. Most prior attacks target either the visual or the infrared image alone in the digital domain, which renders them ineffective against fused models in the physical world. Moreover, many of these methods are readily noticeable, as their patch patterns deviate substantially from those seen in the real world. In this paper, we introduce VIPatch (Visual-Infrared Patch), a novel physical adversarial patch attack that produces inconspicuous, realistic, and natural-looking patches for facial images. Specifically, VIPatch crafts a gradient-color mask together with a band-aid sticker across both the visual and infrared images, and jointly optimizes these two elements; the resulting digital patches further guide the fabrication of their physical counterparts. Experimental results show that VIPatch achieves competitive attack success rates (over 90%) in both the digital and physical domains, while keeping the patches unobtrusive to human observers.
Research Starting Point
Visual-infrared fusion is used to keep face detection working in low light and temperature-screening environments, but most adversarial research attacks only one modality or remains digital. A production system needs to know whether an ordinary-looking object can suppress both channels after printing, placement, lighting changes, and camera movement.
Method
VIPatch jointly optimizes a gradient-colored face mask for the visible channel and a bandage-like thermal sticker for the infrared channel, then fabricates physical counterparts. The evaluation combines multiple visible and infrared detectors, white-box and black-box transfer, and physical tests over viewing angle, illumination from 200 to 12,000 lux, and distances from one to three meters.
Paper Summary
Multispectral sensing should be threat-modeled as a coupled system, not as two independent defenses. The attack is a red-team result rather than a mitigation, but it gives detector vendors concrete physical conditions, transfer models, and attack-success targets for evaluating patch detection, sensor consistency checks, and fallback policy.