L3-Dark-Planet-8B-GGUF

Introduction

L3-Dark-Planet-8B-GGUF is a LLama3-based model optimized for creative writing tasks such as fiction and roleplaying. It supports a variety of writing styles and genres including horror, science fiction, and romance. The model is designed for high performance with a low perplexity level, making it suitable for detailed storytelling and prose generation.

Architecture

This model supports a maximum context length of 8192 tokens, which can be extended to 32k using "rope" settings. It features enhanced quantization techniques for improved performance across different computational settings. The architecture is designed to handle various temperature settings and operates with all parameters, ensuring flexibility in text generation.

Training

The L3-Dark-Planet-8B-GGUF model is constructed using components from models like L3-8B-Stheno-v3.2 and Llama-3-Lumimaid-8B-v0.1-OAS. It includes enhanced prose and fiction writing capabilities compared to similar models. The model has been updated with refreshed and upgraded quants, incorporating the latest LLAMACPP improvements for better instruction following and output generation.

Guide: Running Locally

To run the L3-Dark-Planet-8B-GGUF model locally:

Setup Environment: Ensure you have Python installed along with packages required for text generation. You might need tools like transformers and torch.
Download Model: Obtain the model files from the Hugging Face repository. Choose between Bfloat16 and Float32 versions based on your computational resources.
Run Inference: Use a text generation web UI like oobabooga or systems like KoboldCpp for smoother operation. Adjust parameters like temperature and repetition penalty to suit your needs.
Cloud GPUs: Consider using cloud services offering GPU access for more efficient processing, such as AWS, Google Cloud, or Azure.

License

The L3-Dark-Planet-8B-GGUF model is licensed under the Apache-2.0 License, allowing for both personal and commercial use with proper attribution.