Llama 3.1 8 B Book Adventures

Introduction

KoboldAI's Llama-3.1-8B-BookAdventures model is designed for long-form writing, intended as a research model to facilitate co-writing and adventure mode experiences. It is an intermediate release due to the absence of suitable Chat RP data, allowing expansion by communities with access to superior private data.

Architecture

The model is based on the KoboldAI/LLaMA-3.1-8B-Infinity3M-Kobo, optimized for longer writing capabilities. It operates with a 32K context, and its design focuses on generating narrative-like text, integrating user prompts into story form. While it supports guided co-writing, it lacks the ability to perform Chat RP as a chatbot.

Training

Training utilized the Alpaca format with a subset of the Infinity3M dataset. It was tuned by removing short-form writing data to enhance long-form bias, avoiding premature story endings. The PromptGen tool was employed to generate instruct prompts for the Pike dataset, ensuring no direct copying of existing works. The model also incorporates elements from the Floyd adventure data.

Guide: Running Locally

To run the model locally:

1. Clone the KoboldAI repository and navigate to the model directory.
2. Ensure you have the necessary dependencies installed.
3. Run the model using the KoboldAI Lite interface for best results.
4. Consider using cloud GPU services like Google Cloud, AWS, or Azure for optimal performance, especially given the model's size and context length.

License

The Llama-3.1-8B-BookAdventures model is licensed under CC-BY-NC-SA-4.0, intended for research purposes only. It may be used privately by AI hobbyists but is not available for commercial use.