Cohere For A I_aya expanse 32b exl2

Introduction

Aya Expanse 32B is a multilingual large language model developed by Cohere For AI, optimized for 23 languages. It represents advanced capabilities in multilingual preference training and safety tuning, resulting from dedicated research.

Architecture

Aya Expanse 32B is an auto-regressive language model leveraging an optimized transformer architecture. It includes post-training processes such as supervised finetuning, preference training, and model merging. The model supports a context length of 128K and is designed to handle text input and output exclusively.

Training

The model's training incorporates elements like data arbitrage, multilingual preference training, and safety tuning. It is part of the Command family of models, which are highly performant and multilingual. The Aya Expanse model has undergone extensive evaluation and fine-tuning to ensure robust multilingual capabilities.

Guide: Running Locally

1. Install Dependencies: Ensure you have the transformers library installed. You can do this by running:

   pip install 'git+https://github.com/huggingface/transformers.git'
   

2. Load the Model:

   from transformers import AutoTokenizer, AutoModelForCausalLM
   
   model_id = "CohereForAI/aya-expanse-32b"
   tokenizer = AutoTokenizer.from_pretrained(model_id)
   model = AutoModelForCausalLM.from_pretrained(model_id)
   

3. Prepare Input:

   messages = [{"role": "user", "content": "Your message here"}]
   input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
   

4. Generate Text:

   gen_tokens = model.generate(
       input_ids, 
       max_new_tokens=100, 
       do_sample=True, 
       temperature=0.3,
   )
   gen_text = tokenizer.decode(gen_tokens[0])
   print(gen_text)
   

Cloud GPUs

For optimal performance, consider using cloud-based GPUs such as AWS EC2 instances, Google Cloud's GPU offerings, or Azure's machine learning services.

License

Aya Expanse 32B is released under the CC-BY-NC license. Use of the model must comply with C4AI's Acceptable Use Policy. The license allows for non-commercial use only and requires agreement to Cohere’s privacy policy and terms.