em_german_leo_mistral

Introduction

EM German is a model family based on Llama2, Mistral, and LeoLM architectures, fine-tuned on a comprehensive dataset of German-language instructions. These models are designed to excel in understanding, generating, and interacting with German content. They include versions based on 7b, 13b, and 70b Llama-2, Mistral, and LeoLM models with continued pretraining on German texts.

Architecture

The EM German model family utilizes the recent Mistral architecture, which may not be supported by all libraries. It offers models optimized for German text generation, leveraging the underlying capabilities of Llama2 and LeoLM with advanced pretraining enhancements.

Training

The models in the EM German family have been fine-tuned on diverse German-language datasets. These models benefit from extensive pretraining and optimization efforts, enhancing their performance in generating and understanding German text. The training incorporates various instructions and tasks to ensure proficiency in language interactions.

Guide: Running Locally

1. Clone the Repository: Start by cloning the EM German GitHub repository for access to model files and examples.
2. Install Dependencies: Ensure you have Python and the required libraries, such as PyTorch and Transformers, installed.
3. Download Model Weights: Select and download the model weights from Hugging Face, opting for the LeoLM Mistral version for optimal performance.
4. Run the Model: Utilize the provided scripts or load the model using a Python script to generate text based on your input.
5. Cloud GPUs: For enhanced performance and large-scale tasks, consider running the models on cloud GPUs available through providers like AWS, Google Cloud, or OVH Cloud.

License

The EM German model family is licensed under the Apache-2.0 license. This means it is open for use in research and development, with the condition that users adhere to the terms of this license.