Open LLM List
esm2_t33_650 M_ U R50 D
facebookIntroduction
ESM-2 is a state-of-the-art protein model developed by Meta, designed for tasks involving protein sequences. It is based on a masked language modeling objective and is suitable for fine-tuning across a variety of protein-related tasks. Detailed information regarding the model can be found in the accompanying research paper.
Architecture
ESM-2 is available in several checkpoints with varying sizes and capacities, each offering different numbers of layers and parameters:
- esm2_t48_15B_UR50D: 48 layers, 15 billion parameters
- esm2_t36_3B_UR50D: 36 layers, 3 billion parameters
- esm2_t33_650M_UR50D: 33 layers, 650 million parameters
- esm2_t30_150M_UR50D: 30 layers, 150 million parameters
- esm2_t12_35M_UR50D: 12 layers, 35 million parameters
- esm2_t6_8M_UR50D: 6 layers, 8 million parameters
Generally, larger models offer better accuracy at the cost of increased memory and computational resources.
Training
The ESM-2 model is trained using a masked language modeling approach. It requires substantial computational resources, particularly for larger models. The training process involves the use of extensive protein sequence data to optimize the model for various protein analysis tasks.
Guide: Running Locally
To run ESM-2 models locally, follow these steps:
- Environment Setup: Ensure you have Python and the necessary libraries installed, including PyTorch or TensorFlow, depending on your preference.
- Download Model: Obtain the desired ESM-2 model checkpoint from Hugging Face's Model Hub.
- Fine-Tuning: Use provided demo notebooks for fine-tuning:
- Run Locally: Execute the model using your local setup for specific tasks.
For optimal performance, especially with larger models, consider using cloud GPUs such as those available on AWS, GCP, or Azure.
License
ESM-2 is released under the MIT License, allowing for broad use and modification, provided that appropriate credit is given and copies of the license are included with distributions.
