Huixin Zhan | One Nucleotide Is Enough: Adversarial Robustness of Genomic Foundation Models
Foresight Institute’s Computation Group
One Nucleotide Is Enough: Adversarial Robustness of Genomic Foundation Models
Abstract: Protein language models fine-tuned for clinical variant interpretation demonstrate strong baseline performance, scoring an Area Under the ROC Curve (AUC) of 0.761 on a cardiomyopathy evaluation set—a result competitive with established benchmark predictors at clinically relevant operating points. However, introducing a single, non-label-altering nucleotide edit elsewhere in the gene causes the model's performance to drop to an AUC of 0.394. Rather than degrading toward random chance, the model's ranking mechanism systematically inverts.
This presentation examines the technical mechanisms driving this instability and explores its broader clinical implications. Beginning with the fundamentals of sequence-based language models, the discussion contrasts natural language corpora with protein sequence modeling before deriving core concepts in adversarial machine learning, ranging from the Fast Gradient Sign Method (FGSM) to advanced iterative attacks.
The talk introduces SafeGenes, an evaluation framework designed to assess model robustness under two clinically motivated threat scenarios. SafeGenes evaluates models across continuous embedding perturbations, biologically plausible single-nucleotide edits, and representation-level stress tests of model reuse. Across five pretrained backbones and two cardiac variant datasets, iterative attacks increase the false positive rate to nearly 1.0 at 95% sensitivity, while displacing one-third of true pathogenic variants from a fixed-size clinical review queue.
Finally, the session distinguishes between theoretical robustness stress tests and practical deployment exploits, concluding with an analysis of the emerging security perimeter as these models are integrated into agentic pipelines involving evidence retrieval, knowledge graph traversal, and shared database writes.
Speaker Bio: Huixin Zhan is an Assistant Professor of Computer Science and Engineering at New Mexico Institute of Mining and Technology. Her research develops and stress tests machine learning for high stakes scientific and clinical settings, spanning genomic foundation models, the adversarial robustness and biosecurity of scientific AI, agentic systems with knowledge graph reasoning, and multimodal surgical AI. She is the lead author of DYNA, a disease specific language model for variant pathogenicity in cardiac and regulatory genomics, published in Nature Machine Intelligence, and of SafeGenes, an open auditing framework for adversarial robustness in genomic AI. She collaborates with Cedars-Sinai Medical Center and Los Alamos National Laboratory. huixin-zhan-ph-d-9a0b3a143
A group of scientists, engineers, and entrepreneurs in computer science, ML, cryptography, and related fields who leverage those technologies to improve voluntary cooperation across humans, and ultimately AIs.
Subscribe to our newsletter
Nominate a seminar presenter/topic
Share this application with colleagues who’d like to join
Our book Gaming the Future: Technologies for Intelligent Voluntary Cooperation is now online on Substack.
Zoom link: https://us02web.zoom.us/j/81623367983