Home 3 B v3 G G U F

Introduction

The "Home" model, known as Home-3B-v3-GGUF, is a specialized version of the StableLM-3B-Zephyr model from Stability AI. It is designed to control smart home devices and perform basic question answering. The model supports multiple languages and is fine-tuned on a custom dataset to enable function calling.

Architecture

Home-3B-v3-GGUF is based on the StableLM model, which offers increased accuracy and includes features like basic multi-personality and multi-language support. It is quantized using Lama.cpp to run efficiently on low-resource environments, such as Raspberry Pis typically used in Home Assistant setups. The model functions as an instruct-type model using the Zephyr prompt format, allowing it to perform tasks related to home automation.

Training

The model was trained using the LoRA technique on an NVIDIA RTX 3090 with 24GB memory. The LoRA has a rank of 64 and alpha of 128, targeting specific projection modules. The final model is a merged version of these components. It achieves a 97.11% accuracy for JSON function calling on its test dataset.

Guide: Running Locally

To run the Home-3B-v3-GGUF model locally:

1. Setup Environment: Ensure you have a Python environment set up with necessary libraries such as torch.
2. Download Model: Clone the model repository or download the model files directly from Hugging Face.
3. Install Lama.cpp: Use Lama.cpp for model quantization if running on low-resource hardware.
4. Run Inference: Use a script or tool to load the model and provide it with system prompts for home automation tasks.

For enhanced performance, consider using cloud GPUs offered by providers like AWS, Google Cloud, or Azure.

License

The model is released under the STABILITY AI NON-COMMERCIAL RESEARCH COMMUNITY LICENSE AGREEMENT. It is intended for non-commercial use only, following the licensing terms of the original StableLM model series from Stability AI.