Cipher 20 B

Introduction

Cipher-20B is a causal language model with 20 billion parameters, developed for code generation tasks. It is designed to assist in programming by generating code, detecting errors, and explaining code snippets. The model is compatible with multiple programming languages.

Architecture

Cipher-20B leverages the transformers library to facilitate text generation, particularly for code-related tasks. It uses a causal language modeling approach to predict subsequent tokens in a sequence, making it suitable for code completion and generation.

Training

Cipher-20B was trained on an extensive dataset that includes code, programming tasks, and technical documentation. It has been fine-tuned to handle multiple programming languages such as Python, JavaScript, and C++. The model is capable of generating code, identifying errors, and explaining code logic, although it may produce verbose outputs or struggle with ambiguous inputs.

Guide: Running Locally

To run Cipher-20B locally, follow these steps:

1. Install Transformers: Ensure you have the transformers library installed.

   pip install transformers
   

2. Load the Model and Tokenizer:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model = AutoModelForCausalLM.from_pretrained("HelpingAI/Cipher-20B")
   tokenizer = AutoTokenizer.from_pretrained("HelpingAI/Cipher-20B")
   

3. Prepare Input and Generate Code:

   code_task = [
       {"role": "system", "content": "You are Cipher"},
       {"role": "user", "content": "Write a Python function to calculate the Fibonacci sequence."}
   ]
   
   inputs = tokenizer.apply_chat_template(
       code_task,
       add_generation_prompt=True,
       return_tensors="pt"
   )
   
   outputs = model.generate(
       inputs,
       max_new_tokens=256,
       temperature=0.7,
       top_p=0.9,
   )
   
   print(tokenizer.decode(outputs[0], skip_special_tokens=True))
   

4. Hardware Requirements: For efficient execution, consider using cloud GPUs such as those offered by AWS, Google Cloud, or Azure.

License

Cipher-20B is licensed under the HelpingAI license. For more details, visit helpingai.co/license.