Harnessing Transformers to Generate Protein Sequences Prone to Liquid Liquid Phase Separation

Abstract Understanding the molecular grammar that governs protein phase separation is essential for advancements in bioinformatics and protein engineering. This study leverages Generative Pre-trained Transformer (GPT)-based Protein Language Models (PLMs) to decode the complex grammar of proteins prone to liquid-liquid phase separation (LLPS). We trained three distinct GPT models on datasets comprising amino acid sequences with varying LLPS propensities: highly predisposed (LLPS+ GPT), moderate (LLPS-GPT), and resistant (PDB* GPT). As training progressed, the LLPS-prone model began to learn embeddings that were distinct from those in LLPS-resistant sequences. These models generated 18,000 protein sequences ranging from 20 to 200 amino acids, which exhibited low similarity to known sequences in the SwissProt database. Statistical analysis revealed subtle but significant differences in amino acid occurrence probabilities between sequences from LLPS-prone and LLPS-resistant models, suggesting distinct molecular grammar underlying their phase separation abilities. Notably, sequences from LLPS+ GPT showed fewer aromatic residues and a higher fraction of charge decoration. Short peptides (20-25 amino acids) generated from LLPS+ GPT underwent computational and wet-lab validation, demonstrating their ability to form phase-separated states in vitro. The generated sequences enriched the existing database and enabled the development of a robust classifier that accurately distinguishes LLPS-prone from non-LLPS sequences. This research marks a significant advancement in using computational models to explore and engineer the vast protein sequence space associated with LLPS-prone proteins.