Problem
CFT builds an identification service for the client "Golden Crown" (Zolotaya Korona), where users take loans. The user sends a document and a selfie, or a face is captured at a point of sale. The service first checks that a real person is in front of the camera, then that the face matches the document.
The first model, a small single-photo classifier, showed metrics near 100% and satisfied the business. It still missed "maskers", persistent attackers with printed masks.
Example from the article: a masker wears a hoodie, hat or scarf, attaches a cut-out face with a rubber band and cuts out the eyes to show real ones.
Solution in brief
Clean the labels and cluster photos by person so one person never lands in both train and validation. Then retrain with a third class "mask", focal loss, label smoothing and RSC dropout, using two EfficientNetV2 encoders (full frame and face crop).
Questions this case answers
- How to detect presentation attacks such as printed photos, screens and masks in a selfie identity check
- How to build a validation split that avoids leakage when the same person appears in many sessions
- How to detect printed-photo, screen and mask spoofing in selfie identity checks for loan onboarding
- How to stop train/validation leakage when the same person appears in many sessions
Tools and methods
- EfficientNetV2 (smallest variant, 7M parameters) — encoder for the baseline and final model
- Face detector with key points — face crop, crops of eyes, nose and mouth for augmentation, completeness check
- Face embeddings and clustering — find sessions of one person across train and validation
- Cluster sampling and domain sampler — balance persons and the two domains (selfies, points of sale)
- Focal loss, label smoothing, RSC dropout — calibration and generalization on the three-class setup
Google or a one-time email code. No password.