Llama-3.1-Korean-8B-Instruct

Introduction

Llama-3.1-Korean-8B-Instruct is a fine-tuned language model based on Meta-Llama-3.1, designed specifically for Korean text generation. It utilizes several datasets to enhance its performance in generating and understanding Korean language content.

Architecture

This model is derived from the Meta-Llama-3.1-8B-Instruct base model. It incorporates advanced transformer architectures and is designed to handle complex text generation tasks in Korean.

Training

The model was fine-tuned using a variety of datasets:

• maywell/ko_wikidata_QA: A dataset focused on Korean Wikidata for question-answering tasks.
• lcw99/wikipedia-korean-20240501-1million-qna: A large-scale dataset of Korean Wikipedia Q&A.
• jojo0217/korean_rlhf_dataset: A dataset for reward learning from human feedback in Korean.
• MarkrAI/KoCommercial-Dataset: A dataset aimed at commercial applications in Korean.

Guide: Running Locally

To run Llama-3.1-Korean-8B-Instruct locally, follow these steps:

1. Install Requirements: Ensure transformers and accelerate libraries are up-to-date:

   pip install -qU transformers accelerate
   

2. Setup Model and Tokenizer: Use the AutoTokenizer and transformers.pipeline for text generation:

   from transformers import AutoTokenizer, pipeline
   import torch
   
   model = "sh2orc/Llama-3.1-Korean-8B-Instruct"
   tokenizer = AutoTokenizer.from_pretrained(model)
   
   pipeline = pipeline(
       "text-generation",
       model=model,
       torch_dtype=torch.float16,
       device_map="auto",
   )
   

3. Generate Text: Create prompts and generate text using the model's pipeline:

   messages = [{"role": "user", "content": "What is a large language model?"}]
   prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
   outputs = pipeline(prompt, max_new_tokens=2048, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
   print(outputs[0]["generated_text"])
   

4. Cloud GPUs: For optimal performance, consider using cloud-based GPUs such as AWS EC2, Google Cloud, or Azure's GPU instances.

License

Llama-3.1-Korean-8B-Instruct is distributed under the license specified by the model provider. Ensure compliance with the terms and conditions provided in the license agreement.