Open LLM List
Virtuoso Small
arcee-aiIntroduction
Virtuoso-Small is the initial public release in the Virtuoso series by Arcee.ai, designed to provide efficient generative AI capabilities in a compact form. With 14 billion parameters, it is suitable for high-quality text generation, instruction-following, and business-oriented tasks. The model is a part of a scalable ecosystem that includes larger variants like Virtuoso-Medium and Virtuoso-Large.
Architecture
Virtuoso-Small is built with 14 billion parameters, offering a balance between performance and efficiency. It is optimized for smaller hardware configurations, making it accessible for various applications without compromising on quality. The model is designed to integrate seamlessly with its larger counterparts in the Virtuoso series, allowing for scalability as requirements grow.
Training
Virtuoso-Small is evaluated on several datasets and tasks, showcasing its capabilities in text generation. Key metrics from these evaluations include:
- IFEval (0-Shot): 79.35% strict accuracy
- BBH (3-Shot): 50.4% normalized accuracy
- MATH Lvl 5 (4-Shot): 34.29% exact match
- GPQA (0-shot): 11.52% normalized accuracy
- MuSR (0-shot): 14.44% normalized accuracy
- MMLU-PRO (5-shot): 46.57% accuracy
These results are derived from evaluations on the Open LLM Leaderboard, highlighting the model's performance across diverse tasks.
Guide: Running Locally
- Install Dependencies: Ensure you have Python and necessary libraries installed. Use
pipto install any required packages. - Download the Model: Access the model files from the Hugging Face repository or the Virtuoso-Small GGUF link.
- Set Up Environment: Configure your environment for running the model, ensuring compatibility with your hardware.
- Execute Locally: Use a script or interface to interact with the model for your specific use case.
For optimal performance, using cloud GPUs such as those from AWS, Google Cloud, or Azure is recommended, especially when handling larger datasets or more complex tasks.
License
Virtuoso-Small is released under the Apache-2.0 license, allowing for both personal and commercial use while ensuring compliance with the license terms.
