Nehanda Model
Read the full research report: Epistemic Fine-Tuning of Open-Weight LLMs for Deep Research
Overview
Nehanda v3 is a fine-tuned Qwen3.6-27B VL model trained for RAG synthesis — the capability to read source documents and produce grounded responses without fabricating claims. It scores 88.7% on FACTS Grounding, surpassing frontier models including Gemini 2.5 Pro (87.8%), Claude 3.5 Sonnet (83.8%), and GPT-4o (79.8%).
The result demonstrates that epistemic behavior — source fidelity, evidence boundary enforcement, refusal to fabricate — is a trainable capability that targeted fine-tuning installs more efficiently than scale alone. The model was trained with 1.15% of parameters for approximately $135 of GPU time on a single NVIDIA L40S.
Key Results
| Model | FACTS Grounding Score | Parameters | Access |
|---|---|---|---|
| Nehanda v3 | 88.7% | 27B | Open Weight |
| Gemini 2.5 Pro | 87.8% | Proprietary | Closed |
| Claude 3.5 Sonnet | 83.8% | Proprietary | Closed |
| GPT-4o | 79.8% | Proprietary | Closed |
| Gemma 3 27B | 74.9% | 27B | Open Weight |
Nehanda v3 outperforms Gemma 3 27B — the same-size open-weight model from Google — by 13.8 percentage points. The gap is attributable to the training pipeline, not the base model: both are 27B, both are open-weight, but only one has been fine-tuned for source fidelity.
Training Pipeline
Nehanda v3 uses a five-stage stacked QLoRA pipeline. LoRA adapters (r=64, α=128, 7 target modules covering all attention and MLP projections) are initialized once and trained continuously across all stages. An eval gate follows each SFT stage — the pipeline halts if the model’s epistemic behavior regresses.
| Stage | Purpose | Learning Rate |
|---|---|---|
| 1. Epistemic Foundation | Premise correction, evidence boundary enforcement | 2e-4 |
| 2. Evidence Hardening | Source citation, claim verification | 8e-5 |
| 3. RAG Synthesis | Multi-document synthesis with inline citation | 2e-5 |
| 4. Constitutional Alignment | Refusal to fabricate, calibration | 2e-5 |
| 5. Preference Optimization (DPO) | Preference tuning for grounded responses | 2.7e-7 |
The learning rate decays across stages because each stage builds on an increasingly fragile foundation — large updates late in the pipeline would disrupt the epistemic behavior installed earlier.
Base Model
- Architecture: Qwen3.6-27B VL (native vision-language)
- Context window: 262,144 tokens
- Vision: Integrated vision encoder (training is text-only SFT/DPO, vision weights untouched)
- Training data: Energy regulatory documents, intelligence analysis reports, general-purpose synthesis tasks
- Training cost: ~$135 GPU time on a single NVIDIA L40S
Prompt Schema
Nehanda v3 uses a persona-based prompt schema (SEP-020):
{persona}
### Task:
{input}
### Response:
Model Variants
| Variant | Format | Use Case |
|---|---|---|
asoba/nehanda-v3-27b |
Full weights (HuggingFace) | Full-precision inference, further fine-tuning |
asoba/nehanda-rag-synthesis-27b-gguf |
Quantized q4_k_m GGUF | Local inference via llama.cpp or LM Studio |
Trade-offs
Nehanda v3 sacrifices general capability for epistemic reliability. The model is not trained for creative writing, code generation, or open-ended chat. It is trained to read documents and say what they support.
For applications where source fidelity is the core capability — regulatory analysis, intelligence assessment, due diligence, academic research — the trade-off is favorable. For applications where general capability matters more, a frontier model is the better choice.
Citation
Samudzi, S. (2026). Epistemic Fine-Tuning of Open-Weight LLMs for Deep Research:
Nehanda v3 and the FACTS Grounding Benchmark. Asoba Corporation Technical Report.
Model: asoba/nehanda-v3-27b.