Fw F Qwen 7 B 0.2
bunnycoreIntroduction
FwF-Qwen-7B-0.2 is a merged large language model designed for a variety of tasks including reasoning, long thinking, roleplay, and general tasks. It combines the strengths of different specialized base models to achieve enhanced performance in these areas.
Architecture
FwF-Qwen-7B-0.2 is constructed by merging four distinct models:
bunnycore/FuseQwQen-7B
prithivMLmods/QwQ-LCoT-7B-Instruct
qingy2024/UwU-7B-Instruct
fblgit/cybertron-v4-qw7B-UNAMGS
The merging process employed a linear method, further detailed in the research paper arxiv: 2203.05482.
Training
The model was assembled using a YAML configuration specifying different weightings for each base model:
bunnycore/FuseQwQen-7B
: 0.9qingy2024/UwU-7B-Instruct
: 0.9prithivMLmods/QwQ-LCoT-7B-Instruct
: 0.5fblgit/cybertron-v4-qw7B-UNAMGS
: 0.5
The following settings were used:
merge_method
: linearnormalize
: falseint8_mask
: truedtype
: bfloat16
Guide: Running Locally
To run the FwF-Qwen-7B-0.2 model locally, follow these steps:
- Install Dependencies: Ensure you have Python and the
transformers
library installed.pip install transformers
- Download the Model: Use the Hugging Face
transformers
library to load the model.from transformers import AutoModelForCausalLM, AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("bunnycore/FwF-Qwen-7B-0.2") model = AutoModelForCausalLM.from_pretrained("bunnycore/FwF-Qwen-7B-0.2")
- Cloud GPUs: For optimal performance, consider using cloud services such as AWS, GCP, or Azure, which provide access to high-performance GPUs.
License
The licensing details for the FwF-Qwen-7B-0.2 model are not specified in the provided information. Users should refer to the Hugging Face model card or repository for any licensing terms or restrictions.