OpenMath-8B-GGUF

Introduction

OpenMath-8B-GGUF is a text generation model designed for computational tasks involving mathematical data. It leverages the NVIDIA OpenMathInstruct-2 dataset and supports 8-bit precision for efficient processing.

Architecture

The model is based on the NVIDIA OpenMath2-Llama3.1-8B architecture. It employs transformers and safetensors within the GGUF framework, focusing on delivering high-performance text generation.

Training

OpenMath-8B-GGUF was trained using the NVIDIA OpenMathInstruct-2 dataset. It supports various precision formats including full precision (F16) and quantized versions (Q4, Q5, Q8).

Guide: Running Locally

Download and Install Ollama

1. Download Ollama: Obtain the software from ollama.com/download and install it on your Windows or Mac system.

Steps to Run GGUF Models

1. Create the Model File: Name your model file, such as metallama.

2. Add the Template Command: In your model file, add a line like:

   FROM Llama-3.2-1B.F16.gguf
   

3. Create and Patch the Model: Use the terminal to run:

   ollama create metallama -f ./metallama
   

   Verify the model creation with:

   ollama list
   

Running the Model

Run your model with:

ollama run metallama

Sample Usage

Execute in terminal:

D:\>ollama run metallama
>>> write a mini passage about space x

Cloud GPUs

For enhanced performance, consider using cloud GPUs from providers like AWS, Google Cloud, or Azure.

License

This model is distributed under the CreativeML OpenRAIL-M license.