Mirai 3.0 70 B
BlackrootIntroduction
Mirai-3.0-70B is a model developed by Blackroot, leveraging the Llama 3 Instruct format. It represents an evolution in merging strategies and architecture, deviating from previous methods by employing the TIES merging technique to preserve model integrity and tokenization.
Architecture
Mirai-3.0-70B utilizes a complex merging approach involving multiple base models and instruct models. The architecture incorporates a stockmerge and TIES model, facilitated by the mergekit tool. This strategy uses a combination of "True Merge" and TIES to amplify model strengths while addressing issues with end-of-sequence (EOS) tokens. The model integrates several base models, including PKU-Baichuan-MLSystemLab/Llama3-PBM-Nova-70B, yentinglin/Llama-3-Taiwan-70B-Instruct, and others through a series of merges, including a custom base model stock merge.
Training
The training process involves evolutionary strategies, iterating on previous generations by adding or removing models. The focus is on preserving the EOS token while enhancing storytelling diversity. The model employs geometric interpolation and TIES merging to maintain token consistency. Despite the challenges, the TIES method was adopted for its ability to preserve EOS tokens and improve overall model performance.
Guide: Running Locally
- Clone the Repository: Begin by cloning the model repository from Hugging Face.
- Install Dependencies: Ensure you have all necessary libraries and dependencies installed, such as
transformers
andtorch
. - Load the Model: Use the Hugging Face
transformers
library to load the Mirai-3.0-70B model. - Configure Environment: Set up your environment to support large model inference, which may include using optimized libraries or settings.
- Run Inference: Execute the model using sample inputs to verify its performance.
Cloud GPUs
For optimal performance, consider using cloud GPUs such as those offered by AWS, Google Cloud, or Azure. These platforms provide scalable resources that can handle the computational demands of running large models like Mirai-3.0-70B.
License
The licensing details for Mirai-3.0-70B should be reviewed in the model repository on Hugging Face to ensure compliance with any usage restrictions or requirements.