L3 70 B Euryale v2.1

Introduction

L3-70B-Euryale-V2.1 is a large-scale language model designed for improved prompt adherence, creativity, and handling of unique formats. It is part of the family of models developed as a successor to Stheno v3.2.

Architecture

The model is a 70 billion parameter model, which allows for superior awareness and handling of subtle contexts compared to smaller models like 7B or 13B. It has been fine-tuned using LoRA, a more cost-effective method compared to FFT.

Training

L3-70B-Euryale-V2.1 was trained using 8x H100 SXM GPUs, with further training conducted beyond this initial setup. The training dataset is the same as used for Stheno v3.2.

Guide: Running Locally

To run the model locally:

1. Set Up Environment: Ensure you have PyTorch and necessary libraries installed.
2. Download the Model: Obtain the model files from the Hugging Face repository.
3. Configure Settings: Use recommended sampler settings for optimal performance:
   • Temperature: 1.17
   • Minimum Probability (min_p): 0.075
   • Repetition Penalty: 1.10
4. Run Inference: Utilize SillyTavern Instruct settings with the context template Llama-3-Instruct-Names and instruct presets Euryale-v2.1-Llama-3-Instruct.

For enhanced performance, consider using cloud GPUs such as AWS EC2 or Google Cloud GPUs.

License

The model is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (cc-by-nc-4.0), which allows for sharing and adaptation for non-commercial purposes with attribution.