Stanford Trust & Safety Research Conference Lunch Workshop: Evaluating model biases in identity fraud detection

​Every day, millions of identity fraud attempts hit online services, using a mix of stolen and fabricated identities. They use fabricated identities, often including deepfakes, and repeated attempts designed to find a way through verification. Identity verification systems must therefore detect signs of fraud across the full submission while minimizing friction for legitimate users.

​These systems can prevent abuse, but differences in model performance may also affect who encounters false rejection and which attacks evade detection. Evaluating those differences is especially important for deepfake detectors, whose performance can vary across image sources, synthetic-image generators, and perceived demographic groups.

​To study whether deepfake-detection performance varies across populations, we assembled an exploratory balanced dataset of bona-fide and generated images

​In this session, we present exploratory research comparing Persona’s and 7 other public deepfake detection systems. We evaluate performance across six intersections of perceived gender presentation and Monk Skin Tone at a common operating policy. We discuss which groups experience elevated friction or weaker attack-detection coverage, the limitations of the current sample, and the balanced confirmatory study needed to support stronger claims about demographic performance bias.