Open LLM List
llava v1.6 vicuna 7b
liuhaotianIntroduction
The LLaVA-V1.6-Vicuna-7B model is an open-source chatbot developed by fine-tuning a language model (LLM) on multimodal instruction-following data. It employs an auto-regressive language model, leveraging the transformer architecture. The base LLM used is lmsys/vicuna-7b-v1.5. This model is designed primarily for research in large multimodal models and chatbots, targeting users such as researchers and hobbyists in fields like computer vision, natural language processing, machine learning, and artificial intelligence.
Architecture
LLaVA-V1.6-Vicuna-7B is based on the transformer architecture. It functions as an auto-regressive language model, which is trained to predict the next token in a sequence, given prior tokens. This makes it suitable for applications such as chatbots where generating coherent and contextually relevant text is crucial.
Training
The training dataset for LLaVA includes:
- 558K filtered image-text pairs from datasets like LAION, CC, and SBU, captioned using BLIP.
- 158K GPT-generated multimodal instruction-following data.
- 500K academic-task-oriented visual question answering (VQA) data.
- 50K GPT-4V data.
- 40K ShareGPT data.
The model's evaluation is based on 12 benchmarks, consisting of 5 academic VQA benchmarks and 7 tailored for instruction-following language models.
Guide: Running Locally
To run the LLaVA-V1.6-Vicuna-7B model locally, follow these steps:
- Set Up Environment: Ensure you have Python and necessary libraries such as PyTorch installed.
- Download Model: Access the model files from the Hugging Face repository.
- Install Dependencies: Use a package manager to install any additional dependencies.
- Load Model: Use a suitable framework like Transformers to load the model.
- Run Inference: Input data as specified and run inference to generate outputs.
For optimal performance, consider using a cloud GPU service like AWS, Google Cloud, or Azure to handle the computational requirements.
License
The LLaVA-V1.6-Vicuna-7B model is licensed under the LLAMA 2 Community License, with all rights reserved by Meta Platforms, Inc. For questions or comments, users are directed to the GitHub issues page.
