Duck D B N S Q L 7 B v0.1 G G U F

Introduction

DuckDB-NSQL-7B is a member of the NSQL family, designed specifically for SQL generation tasks. It is a quantized version based on Meta's Llama-2 7B model, tailored for generating SQL queries for DuckDB.

Architecture

The model is built upon the Meta's Llama-2 7B, further pre-trained and fine-tuned with SQL generation capabilities. It uses a family of autoregressive models for generating valid SQL statements, including those using DuckDB's extensions.

Training

The model was trained on 200k synthetically generated DuckDB text-to-SQL pairs using Mixtral-8x7B-Instruct-v0.1. It also used text-to-SQL pairs from NSText2SQL, transpiled to DuckDB SQL. Training employed cross-entropy loss on 80GB A100 GPUs, utilizing data and model parallelism for 10 epochs.

Guide: Running Locally

1. Installation: Install PyTorch and Hugging Face's transformers library.
2. Load Model: Use AutoTokenizer and AutoModelForCausalLM from transformers to load the model.
3. Generate SQL: Prepare your input text following the prompt format, tokenize it, and generate SQL using the model.
4. Output: Decode the generated IDs to obtain the SQL statement.

Cloud GPUs like those from AWS or Google Cloud can be used to handle the computational load effectively.

License

The DuckDB-NSQL-7B model is licensed under the Llama2 license.