Incorporating LLM Embeddings for Variation Across the Human Genome
This provides a foundational resource for large-scale genomic discovery and precision medicine by addressing the gap in variant-level representations, though it is incremental in applying existing LLM methods to new genomic data.
The authors tackled the problem of representing genomic variation at the variant level across the entire human genome by generating embeddings for 8.9 billion possible variants using LLMs, achieving high predictive accuracy for variant properties and enabling downstream applications like enhanced genome-wide association studies and genetic risk prediction.
Recent advances in large language model (LLM) embeddings have enabled powerful representations for biological data, but most applications to date focus only on gene-level information. We present one of the first systematic frameworks to generate variant-level embeddings across the entire human genome. Using curated annotations from FAVOR, ClinVar, and the GWAS Catalog, we constructed semantic text descriptions for 8.9 billion possible variants and generated embeddings at three scales: 1.5 million HapMap3+MEGA variants, ~90 million imputed UK Biobank variants, and ~9 billion all possible variants. Embeddings were produced with both OpenAI's text-embedding-3-large and the open-source Qwen3-Embedding-0.6B models. Baseline experiments demonstrate high predictive accuracy for variant properties, validating the embeddings as structured representations of genomic variation. We outline two downstream applications: embedding-informed hypothesis testing by extending the Frequentist And Bayesian framework to genome-wide association studies, and embedding-augmented genetic risk prediction that enhances standard polygenic risk scores. These resources, publicly available on Hugging Face, provide a foundation for advancing large-scale genomic discovery and precision medicine.