c4ai command r7b 12 2024

Introduction

C4AI Command R7B is a 7 billion parameter multilingual model developed by Cohere and Cohere For AI. It is designed for a variety of tasks such as reasoning, summarization, question answering, and code. The model supports 23 languages and excels in Retrieval Augmented Generation (RAG), tool use, and code tasks.

Architecture

C4AI Command R7B utilizes an auto-regressive transformer architecture. It has three layers with sliding window attention and ROPE for efficient context modeling, and a fourth layer using global attention. The model accommodates a context length of 128K and supports 23 languages, including English, French, Spanish, and more.

Training

The model undergoes supervised fine-tuning and preference training to align its behavior to human preferences for helpfulness and safety. It has been evaluated on various benchmarks, demonstrating top performance compared to similarly sized models.

Guide: Running Locally

1. Install Transformers Library:

   pip install 'git+https://github.com/huggingface/transformers.git'
   

2. Load the Model and Tokenizer:

   from transformers import AutoTokenizer, AutoModelForCausalLM
   
   model_id = "CohereForAI/c4ai-command-r7b-12-2024"
   tokenizer = AutoTokenizer.from_pretrained(model_id)
   model = AutoModelForCausalLM.from_pretrained(model_id)
   

3. Generate Text:

   messages = [{"role": "user", "content": "Hello, how are you?"}]
   input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
   
   gen_tokens = model.generate(
       input_ids,
       max_new_tokens=100,
       do_sample=True,
       temperature=0.3,
   )
   
   gen_text = tokenizer.decode(gen_tokens[0], skip_special_tokens=True)
   print(gen_text)
   

4. Suggested Cloud GPUs: Consider using cloud services like AWS, Google Cloud, or Azure for accessing powerful GPU instances to efficiently run the model.

License

The model is released under the CC-BY-NC license, which requires adherence to the C4AI Acceptable Use Policy. It is intended for non-commercial use only. More details on the license can be found here.