nllb jaen 1.3 B lightnovels
thefrigidliquidationIntroduction
The NLLB-JAEN-1.3B-LIGHTNOVELS model is fine-tuned specifically for translating Japanese light and web novels to English. It supports translation of text up to 512 tokens and utilizes diverse beam search for optimal results.
Architecture
This model leverages the NLLB (No Language Left Behind) framework, based on the m2m_100 architecture. It employs transformers and is implemented using PyTorch. The model supports both English and Japanese languages.
Training
The model is fine-tuned on a dataset of Japanese light novels to enhance its translation capabilities from Japanese to English. It is configured to handle translations with custom noun and character name replacements, as well as optional handling of honorifics.
Guide: Running Locally
To run this model locally, follow these steps:
- Install dependencies: Ensure that you have Python and the Hugging Face Transformers library installed.
- Load the model: Use the following code snippet to load the tokenizer and model:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("thefrigidliquidation/nllb-jaen-1.3B-lightnovels") model = AutoModelForSeq2SeqLM.from_pretrained("thefrigidliquidation/nllb-jaen-1.3B-lightnovels")
- Generate translations: Use the model to generate translations with the provided settings for diverse beam search.
- Cloud GPUs: For optimal performance, consider using cloud GPU services from providers like AWS, Google Cloud, or Azure.
License
The model is licensed under the Creative Commons Attribution-NonCommercial 4.0 International (cc-by-nc-4.0) license. This allows for use and modification for non-commercial purposes, provided appropriate credit is given.