Athene V2 Agent

Introduction

Athene-V2-Agent is an open-source Large Language Model (LLM) designed by Nexusflow. It surpasses existing models like GPT-4o in function calling and agentic tasks, excelling in environments with complex dependencies and tool use. The model is derived from Qwen-2.5-72B-Instruct and is licensed under the Nexusflow Research License.

Architecture

Athene-V2-Agent functions as an agent model capable of reasoning and planning in environments requiring multiple tool calls for single queries. It is designed to generalize to unseen settings without prior training on specific functions or agentic scenarios.

Training

The model achieves superior performance without specific training on evaluated functions or settings. It surpasses GPT-4o in function calling success rates by 18% and in agentic success rates by 17%.

Guide: Running Locally

To run Athene-V2-Agent, utilize the VLLM Docker image compatible with OpenAI APIs:

1. Docker Setup:

   docker run --name athene-v2-agent \
       --runtime nvidia --gpus '"device=0,1,2,3,4,5,6,7"' \
       -v ~/.cache/huggingface:/root/.cache/huggingface \
       --env "HUGGING_FACE_HUB_TOKEN=<secret>" \
       -p <port>:8000 \
       --ipc=host \
       ghcr.io/nexusflowai/athene-v2-vllm:latest \
       --model Nexusflow/Athene-V2-Agent \
       --dtype=auto \
       --tensor-parallel-size=8 \
       --enable-auto-tool-choice \
       --tool-call-parser Athene-V2-Agent
   

2. Cloud GPUs: Consider using cloud GPU services like AWS, Google Cloud, or Azure for optimal performance.

3. Prompting Tips:

   • Use detailed and well-indented docstrings.
   • Set sampling to False and temperature to zero for best results.
   • Utilize the Docker image for accurate performance.

4. Example Usage: Test with example agents provided in the repository for weather and RAG-based queries.

License

Athene-V2-Agent is available under the Nexusflow Research License. For detailed terms, refer to the license document.