bloom 389m zh

Introduction

The BLOOM-389M-ZH model is a text generation model designed for the Chinese language. It is a pruned version of the bigscience/bloom-560m model, aimed at reducing GPU memory usage by optimizing vocabulary size.

Architecture

The model is implemented using the Transformers library in PyTorch. It features a reduced vocabulary size, from the original 250,880 tokens to 42,437 tokens, tailored to Chinese text, resulting in a total of 389 million parameters.

Training

The model was derived from bigscience/bloom-560m, focusing on efficiency improvements for Chinese text processing by pruning unnecessary vocabulary tokens. This adaptation helps in reducing the computational load while maintaining performance for text generation tasks in Chinese.

Guide: Running Locally

To run the BLOOM-389M-ZH model locally, follow these steps:

1. Install Dependencies: Ensure you have Python installed along with the transformers library:

   pip install transformers
   

2. Load the Model and Tokenizer: Use the following Python code to load the model and tokenizer:

   from transformers import BloomTokenizerFast, BloomForCausalLM
   
   tokenizer = BloomTokenizerFast.from_pretrained('Langboat/bloom-389m-zh')
   model = BloomForCausalLM.from_pretrained('Langboat/bloom-389m-zh')
   
   print(tokenizer.batch_decode(model.generate(tokenizer.encode('中国的首都是', return_tensors='pt'))))
   

3. Execution: Run the code to generate text based on the input Chinese prompt.

4. Cloud GPUs: For better performance and to handle larger workloads, consider using cloud GPU services such as AWS EC2, Google Cloud Platform, or Azure.

License

The BLOOM-389M-ZH model is released under the bigscience-bloom-rail-1.0 license, which should be reviewed to understand the permissions and limitations associated with its use.