Sensei 7 B V1

Introduction

Sensei-7B-V1 is a Large Language Model (LLM) fine-tuned from the mistral-ft-optimized-1218 model, based on the Mistral-7B architecture. It specializes in retrieval-augmented generation (RAG) using detailed web search results to generate accurate and well-cited summaries. The model is designed for use in scenarios requiring efficient information retrieval and summarization.

Architecture

• Base Model: mistral-ft-optimized-1218
• Features:
  • Transformer-based model
  • Grouped-Query Attention
  • Sliding-Window Attention
  • Byte-fallback BPE tokenizer

Training

Sensei-7B-V1 is fine-tuned with a fully synthetic dataset to enhance its performance in retrieval-augmented generation tasks. The model is optimized to use search results as context to provide accurate answers to user queries.

Guide: Running Locally

To use Sensei-7B-V1 locally, follow these steps:

1. Set API Key:

   export SCIPHI_API_KEY=MY_SCIPHI_API_KEY
   

2. Run a Query Using AgentSearch:

   python -m agent_search.scripts.run_rag run --query="What is Fermat's last theorem?"
   

Alternatively, provide your own search context directly to the model using the specified JSON format. Ensure the response includes the prefix {"summary": for proper JSON formatting.

For enhanced performance, consider using cloud GPUs from providers like AWS, GCP, or Azure.

License

The usage of Sensei-7B-V1 may be subject to licensing terms specified by its developers. Users should refer to the official documentation or contact the providers for detailed licensing information.