70 B L3.3 Cirrus x1

Introduction

The 70B-L3.3-CIRRUS-X1 model is a text generation model developed by Sao10K, leveraging the Transformers library. It utilizes the Llama-3.3 architecture with 70 billion parameters, designed for stable and stylistic text generation.

Architecture

The model is based on the Llama-3.3-70B-Instruct architecture. It employs the same data composition techniques as the Freya model but with extended training and checkpoint merging, resulting in improved stability and performance.

Training

Training was conducted over approximately 22 hours on an 8xH100 node, followed by an additional 3 hours for merging multiple epoch checkpoints on a 2xH200 node. The process involved continuous model experimentation and required substantial computational resources, which were funded personally by the developer.

Guide: Running Locally

To run the 70B-L3.3-CIRRUS-X1 model locally, follow these steps:

1. Environment Setup: Ensure you have Python installed along with the necessary libraries, including Transformers and Safetensors.
2. Download Model: Access the model files from the Hugging Face repository.
3. Load Model: Use the Transformers library to load the model into your environment.
4. Run Inference: Employ the model for text generation tasks using appropriate input prompts.

Cloud GPUs: Consider using cloud GPU services like Runpod or Vast for efficient model training and inference, especially for handling large-scale computations.

License

The 70B-L3.3-CIRRUS-X1 model is distributed under the Llama3.3 license, which should be reviewed to understand usage rights and restrictions.