Contrastive learning on protein embeddings enlightens midnight zone.

TitleContrastive learning on protein embeddings enlightens midnight zone.
Publication TypeJournal Article
Year of Publication2022
AuthorsHeinzinger, M, Littmann, M, Sillitoe, I, Bordin, N, Orengo, C, Rost, B
JournalNAR Genom Bioinform
Date Published2022 Jun

Experimental structures are leveraged through multiple sequence alignments, or more generally through homology-based inference (HBI), facilitating the transfer of information from a protein with known annotation to a query without any annotation. A recent alternative expands the concept of HBI from sequence-distance lookup to embedding-based annotation transfer (EAT). These embeddings are derived from protein Language Models (pLMs). Here, we introduce using single protein representations from pLMs for contrastive learning. This learning procedure creates a new set of embeddings that optimizes constraints captured by hierarchical classifications of protein 3D structures defined by the CATH resource. The approach, dubbed , has an improved ability to recognize distant homologous relationships than more traditional techniques such as threading or fold recognition. Thus, these embeddings have allowed sequence comparison to step into the 'midnight zone' of protein similarity, i.e. the region in which distantly related sequences have a seemingly random pairwise sequence similarity. The novelty of this work is in the particular combination of tools and sampling techniques that ascertained good performance comparable or better to existing state-of-the-art sequence comparison methods. Additionally, since this method does not need to generate alignments it is also orders of magnitudes faster. The code is available at

Alternate JournalNAR Genom Bioinform
PubMed ID35702380
PubMed Central IDPMC9188115