iSA-02-Nano-1B-Preview

Introduction

The iSA-02-Nano-1B-Preview is a language model developed by NeuraLake, focusing on reasoning capabilities with a "think before you speak" philosophy. It is part of the first small reasoning models series and uses synthetic data for enhanced reasoning.

Architecture

• Type: Transformer-based
• Layers: 16
• Hidden Size: 2048
• Heads: 32
• Key/Value Size: 64
• Feed-Forward Size: 8192
• Vocabulary Size: 128,256

Training

• Mixed Precision: fp16
• Context Window Size:
  • Text generation: 1024–4096 tokens
  • Logical reasoning: 16,000–64,000 tokens

Guide: Running Locally

Basic Steps

1. Setup Environment: Install necessary libraries such as PyTorch.
2. Download Model: Obtain the model from Hugging Face.
3. Load Model: Use a framework like Transformers to load the model.
4. Run Inference: Input text prompts and receive generated text outputs.

Cloud GPUs

For enhanced performance, consider using cloud GPU services like AWS EC2, Google Cloud, or Azure.

License

This model is licensed under the Apache 2.0 License.