Codon language embeddings provide strong signals for use in protein engineering

Outeiral, C; Deane, CM

AI Collection

Journal article

Codon language embeddings provide strong signals for use in protein engineering

Abstract:: The task of understanding and interpreting the complex information encoded within genomic sequences remains a grand challenge in biological research and clinical applications. In this context, recent advancements in large language model research have led to the development of both encoder-only and decoder-only foundation models designed to decode intricate information in DNA sequences. However, several issues persist, particularly regarding the efficient management of long-range dependencies inherent in genomic sequences, the effective representation of nucleotide variations, and the considerable computational costs associated with large model architectures and extensive pretraining datasets. Current genomic foundation models often face a critical tradeoff: smaller models with mediocre performance versus large models with improved performance. To address these challenges, we introduce dnaGrinder, a unique and efficient genomic foundation model. dnaGrinder excels at managing long-range dependencies within genomic sequences while minimizing computational costs without compromising performance. It achieves results that are not just comparable but often superior to leading DNA models such as Nucleotide Transformer and DNABERT-2. Furthermore, dnaGrinder is designed for easy fine-tuning on workstation-grade GPUs, accommodating input lengths exceeding 17,000 tokens. On a single high-performance GPU, it supports sequences longer than 140,000 tokens, making it a highly efficient and accessible tool for both basic biological research and clinical applications

Publication status:: Published

Peer review status:: Peer reviewed

Actions

Email

Email this record

Send the bibliographic details of this record to your email address.

Your Email
Please enter the email address that the record information will be sent to.

-
Your message (optional)
Please add any additional information to be included within the email.
Share
Cite

Cite this record

APA Style

Outeiral, C., & Deane, C. M. (2024). Codon language embeddings provide strong signals for use in protein engineering. Nature Machine Intelligence, 6(2), 170–179.

MLA Style

Outeiral, C, and CM Deane. “Codon Language Embeddings Provide Strong Signals for Use in Protein Engineering.” Nature Machine Intelligence, vol. 6, no. 2, 2024, pp. 170–79.

Chicago Style

Outeiral, C, and CM Deane. 2024. “Codon Language Embeddings Provide Strong Signals for Use in Protein Engineering.” Nature Machine Intelligence 6 (2): 170–79.
Print