Llama 3 Open Ko 8 B

Introduction

Llama-3-Open-Ko-8B is a language model developed by Beomi, based on the Llama-3-8B architecture. It is trained using more than 17.7 billion tokens and is designed to handle text generation tasks in both English and Korean languages. This model is intended for commercial and research use and can be adapted for various natural language generation tasks.

Architecture

Llama-3-Open-Ko-8B is built on an optimized transformer architecture, utilizing an auto-regressive language model approach. The model supports input and output of text and code, and it is part of the Meta Llama 3 family of large language models.

Training

The model was trained using publicly available resources, with a dataset of over 60GB of deduplicated texts. Training was conducted on TPUv5e-256 hardware, supported by Google's TRC program. It differs from its predecessor by using a new Llama-3 tokenizer and exceeding the token count of the Llama-2-Ko tokenizer.

Guide: Running Locally

1. Clone the Repository: Begin by cloning the model repository from Hugging Face.
2. Install Dependencies: Ensure that Python and necessary libraries like PyTorch are installed.
3. Load the Model: Use the Hugging Face Transformers library to load the model into your environment.
4. Run Inference: Implement a script to input text data and generate responses using the model.

For optimal performance, consider running the model on a cloud GPU service such as AWS EC2, Google Cloud Platform, or Azure.

License

The Llama-3-Open-Ko-8B model is distributed under the Llama3 License. For more details, refer to the license documentation.