E X A O N E 3.0 7.8 B Instruct

Introduction

EXAONE-3.0-7.8B-Instruct is a bilingual (English and Korean) generative model with 7.8 billion parameters, developed by LG AI Research. It is pre-trained with 8 trillion curated tokens and further enhanced with supervised fine-tuning and direct preference optimization. This model is highly competitive against other state-of-the-art models of similar size.

Architecture

The model is designed for text generation tasks and is built using transformers. It supports both English and Korean languages and has been fine-tuned for specific instructions, making it suitable for a variety of conversational and custom code applications.

Training

The model underwent pre-training with a vast dataset of 8 trillion tokens, followed by supervised fine-tuning and direct preference optimization. It has been evaluated against similar-sized models, demonstrating superior performance on various benchmarks.

Guide: Running Locally

To run EXAONE-3.0-7.8B-Instruct locally, follow these steps:

1. Install Transformers Library: Ensure you have transformers v4.41 or later.

2. Import Libraries:

   import torch
   from transformers import AutoModelForCausalLM, AutoTokenizer
   

3. Load Model and Tokenizer:

   model = AutoModelForCausalLM.from_pretrained(
       "LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct",
       torch_dtype=torch.bfloat16,
       trust_remote_code=True,
       device_map="auto"
   )
   tokenizer = AutoTokenizer.from_pretrained("LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct")
   

4. Generate Text:

   prompt = "Explain who you are"  # Or a Korean example: "너의 소원을 말해봐"
   messages = [
       {"role": "system", "content": "You are EXAONE model from LG AI Research, a helpful assistant."},
       {"role": "user", "content": prompt}
   ]
   input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
   output = model.generate(input_ids.to("cuda"), eos_token_id=tokenizer.eos_token_id, max_new_tokens=128)
   print(tokenizer.decode(output[0]))
   

5. Cloud GPUs Suggestion: For optimal performance, consider using cloud-based GPUs such as AWS EC2, Google Cloud, or Azure.

License

The EXAONE-3.0-7.8B-Instruct model is available under the EXAONE AI Model License Agreement 1.1 - NC. More details can be found in the license document.