Open LLM List
creative writing control vectors v3.0
jukofyorkIntroduction
The Creative Writing Control Vectors v3.0 repository provides pre-generated control vectors in GGUF format for use with large language models (LLMs) like llama.cpp. These control vectors enable fine-tuned control over LLM outputs for more precise and targeted text generation. They are particularly useful for creative writing applications.
Architecture
The control vectors are implemented using a format that allows them to integrate with the llama.cpp framework. They require the use of respective de-bias control vectors to function correctly. The vectors were generated with the --num_prompt_samples option set to the model's hidden state dimension, ensuring compatibility with the LLM architecture.
Training
Control vectors were generated using a systematic approach involving pre/post prompt stems, creative-writing continuations, and a large collection of writing prompts. The process uses a cross-covariance matrix, eigendecomposition, and projection techniques to ensure vectors align with desired writing styles and attributes. This method allows for the creation of balanced and opposing axes, such as those used in the Dark Tetrad model.
Guide: Running Locally
- Installation: Clone the repository and navigate to the directory.
- Dependencies: Ensure
llama.cppis installed and up-to-date, as multi-vector support was added on 27/06/24. - Running Models:
- Use the command line to apply control vectors with the
--control-vectoroption. - Example for applying a de-bias vector:
llama-cli --model <model>.gguf --control-vector mistral-large:123b-language__debias.gguf - Apply multiple vectors or use scaled vectors for partial application.
- Use the command line to apply control vectors with the
- Cloud GPUs: For computational efficiency, consider running models on cloud GPUs such as AWS EC2 with GPU instances or Google Cloud Platform.
License
The Creative Writing Control Vectors v3.0 repository is licensed under the Apache-2.0 License, which allows for both personal and commercial use, modification, distribution, and private use.
