Llama-3-Taiwan-8B-Instruct

Introduction

Llama-3-Taiwan-8B-Instruct is a language model optimized for Traditional Mandarin and English. It excels in language understanding, generation, and multi-turn dialogue, making it suitable for various NLP tasks.

Architecture

The model contains 70 billion parameters and supports an 8K context length. It is built on the Llama-3 architecture, specifically fine-tuned for tasks in Traditional Mandarin and English, covering domains like legal, manufacturing, medical, and electronics.

Training

Training was performed using the NVIDIA NeMo Framework on NVIDIA DGX H100 systems. The model was trained with a batch size of 2 million tokens per step. The training data was provided by several sponsors, including Chang Gung Memorial Hospital and NVIDIA.

Guide: Running Locally

1. Install Dependencies: Ensure you have Python and the Hugging Face Transformers library installed.
2. Initialize the Model:

   import torch
   from transformers import pipeline
   
   llm = pipeline("text-generation", model="yentinglin/Llama-3-Taiwan-70B-Instruct-rc1", device_map="auto")
   

3. Run the Model: Use the initialized model to generate text based on input data.
4. Cloud GPUs: Use services like AWS, Google Cloud, or Azure for access to powerful GPUs if required.

License

The model is released under the Llama-3 license, allowing for open and flexible use.