wiroai turkish llm 9b G G U F

Introduction

The WIROAI-TURKISH-LLM-9B-GGUF is a quantized version of the WiroAI Turkish language model designed to provide robust support for Turkish language and culture. It is built using Google's Gemma architecture and fine-tuned with high-quality Turkish instructions, making it suitable for various Turkish language processing tasks.

Architecture

The model utilizes a decoder-only transformer architecture based on Google Gemma 2 9B. It is specifically designed to handle Turkish language nuances and local contexts effectively.

Training

The model is fine-tuned with over 500,000 high-quality Turkish instructions using the LoRA method, without quantization. This approach ensures the model's adaptability to the Turkish culture and enhances its performance in language processing tasks.

Guide: Running Locally

1. Install the required Python libraries:

   pip install transformers torch
   

2. Use the following Python script to run the model:

   import transformers
   import torch
   
   model_id = "WiroAI/wiroai-turkish-llm-9b"
   
   pipeline = transformers.pipeline(
       "text-generation",
       model=model_id,
       model_kwargs={"torch_dtype": torch.bfloat16},
       device_map="auto",
   )
   
   pipeline.model.eval()
   instruction = "Bana İstanbul ile alakalı bir sosyal medya postu hazırlar mısın?"
   
   messages = [
       {"role": "user", "content": f"{instruction}"}
   ]
   
   prompt = pipeline.tokenizer.apply_chat_template(
       messages, 
       tokenize=False, 
       add_generation_prompt=True
   )
   
   terminators = [
       pipeline.tokenizer.eos_token_id,
       pipeline.tokenizer.convert_tokens_to_ids("<end_of_turn>")
   ]
   
   outputs = pipeline(
       prompt,
       max_new_tokens=512,
       eos_token_id=terminators,
       do_sample=True,
       temperature=0.9,
   )
   
   print(outputs[0]["generated_text"][len(prompt):])
   

3. For optimal performance, consider using a cloud GPU from providers such as AWS, Google Cloud, or Azure.

License

This model is provided under Google's Gemma license. Users must review and accept the license terms before using the model.