Mini Thinky v2 1 B Llama 3.2

Introduction

MiniThinky-v2-1B-Llama-3.2 is an updated version of the MiniThinky-1B-Llama model, designed for text generation tasks. It features improvements in loss reduction and utilizes a chat template similar to Llama 3.

Architecture

The model is built using the transformers library and is based on the meta-llama/Llama-3.2-1B-Instruct. It incorporates tags such as trl and sft and uses the ngxson/MiniThinky-dataset for training.

Training

MiniThinky-v2-1B was trained using a Hugging Face space equipped with 4xL40S GPUs for a duration of 5 hours, achieving an evaluation loss of approximately 0.8.

Guide: Running Locally

To run MiniThinky-v2-1B-Llama-3.2 locally, follow these steps:

1. Ensure you have Python and the necessary libraries installed, such as transformers.
2. Download the model from the Hugging Face repository.
3. Set up the required system message: "You are MiniThinky, a helpful AI assistant. You always think before giving the answer. Use <|thinking|> before thinking and <|answer|> before giving the answer."
4. Initiate the model and input your queries.

For improved performance, especially during training or large-scale inference, consider using cloud GPUs such as those provided by AWS or Google Cloud.

License

The usage and distribution of MiniThinky-v2-1B-Llama-3.2 are subject to the terms specified by Hugging Face.