Abstract
•Micro-expressions contain information pertinent to person recognition.•LBP-TOP is well suited to model appearance and motion changes in micro-expression.•Optimal weight distribution is important for traditional and soft biometric fusion.
Soft biometrics, although not discriminant enough for person recognition provides additional information that aids traditional person recognition. Initially, attempts were made to integrate appearance-based facial soft biometrics, such as facial marks, skin color, and hair color/style, but more recently behavior-based facial soft biometrics, such as head dynamics, visual speech, and facial expressions have also been studied. Facial expressions are further classified as macro and micro-expressions and most of the existing studies using facial expressions as a soft biometric have focused on macro-expressions. Therefore, in this study, we investigate the utility of micro-expressions as a soft biometric for person recognition. The proposed system is based on the fusion of traditional facial features that model the facial appearance with soft biometric features that model the micro-expressions in an image sequence. We tested a texture-based traditional feature extraction technique, two motion-based soft biometric techniques, and several fusion methods at feature, rank, and decision level. The experiments were conducted on three commonly used micro-expression databases and exhibit an improvement of around 5% identification rate when soft biometric traits are fused with traditional face recognition at decision level.