Qwen. Q V Q 72 B Preview G G U F

Introduction

The Qwen.QVQ-72B-Preview-GGUF is an advanced model hosted on Hugging Face, developed by DEVQUASAR. It is designed for image-text-to-text tasks and is compatible with inference endpoints for conversational AI applications.

Architecture

This model belongs to the Qwen/QVQ-72B series, featuring a GGUF library designed for high performance in generating descriptive text from images. The structure supports complex image processing and text generation tasks.

Training

Details on the specific training process are not provided in the README. However, the model leverages the capabilities of the Qwen/QVQ-72B architecture to handle sophisticated image and text data, offering a preview of its full capabilities.

Guide: Running Locally

To run the model locally, you can use the llama-qwen2vl-cli tool. Follow these steps:

1. Build the CLI tool:

   cmake --build build --config Release --target llama-qwen2vl-cli
   

2. Run a sample inference:

   build/bin/llama-qwen2vl-cli -m ../Qwen.QVQ-72B-Preview-GGUF/Q8/QVQ-72B-Preview-Q8_0-00001-of-00006.gguf --mmproj ../Qwen.QVQ-72B-Preview-GGUF/qwen.qvq-72b-preview-vision.gguf -p "Describe this image." --image ~/test_img.jpg
   

For optimal performance, it's recommended to use cloud GPUs from providers like AWS, Google Cloud, or Azure.

License

The README does not specify the licensing details for the Qwen.QVQ-72B-Preview-GGUF model. Users are encouraged to visit the model's page on Hugging Face for more information and to comply with any usage restrictions or licensing agreements.