Deep Seek V3 slice jp64

Introduction

DeepSeek-V3-Slice-JP64 is an experimental model derived from DeepSeek-V3, optimized for Japanese language outputs. It utilizes a tailored selection of the Mixture of Experts (MoE) layers to balance performance and stability in Japanese text generation.

Architecture

This model is based on DeepSeek-V3, which initially includes 256 experts in its MoE architecture. For DeepSeek-V3-Slice-JP64, the model has been adjusted to use 64 experts per layer, focusing on those most frequently needed for Japanese output. This adjustment aims to enhance the model's stability and performance.

Training

The model has been restructured by selecting the top 64 experts per layer based on the frequency of usage in Japanese example outputs. The scripts/layer_topk_idx_distribution.json records the rank of the top 128 experts for each layer. A script, scripts/deepseek_slice.py, facilitates the creation of this modified model from the original bf16 format.

Guide: Running Locally

To run the model locally, follow these steps:

1. Setup Environment: Ensure you have Python installed along with necessary dependencies. Use a virtual environment for isolation.
2. Download Model: Clone or download the model files from the repository.
3. Run Tests: Use scripts/model_test.py to execute test scripts and verify model performance. This script includes example sentences to measure expert frequency.
4. GPU Recommendation: Given the model's complexity, using a cloud GPU, such as those offered by AWS or Google Cloud, is recommended for efficient processing.

License

Before using the model, review the license file. The DeepSeek-V3-Slice-JP64 model follows the same licensing terms as the original DeepSeek-V3 model.