Nehanda Model - Asoba Open Source

View on HuggingFace Download Quantized GGUF

Read the full research report: Epistemic Fine-Tuning of Open-Weight LLMs for Deep Research

Overview

Nehanda v3.1 is a fine-tuned Qwen3.6-27B VL model trained for RAG synthesis — the capability to read source documents and produce grounded responses without fabricating claims. Evaluated on the public FACTS Grounding benchmark under a 3-judge majority vote protocol across three distinct model families (Gemini 3.7 Flash, GPT-OSS 120B, and glm-5-turbo), it achieves 82.21% factuality, outperforming GPT-4o (80.00%) and Gemma 3 27B (74.90%).

The result demonstrates that epistemic behavior — source fidelity, evidence boundary enforcement, refusal to fabricate — is a trainable capability that targeted fine-tuning installs more efficiently than scale alone. The model was trained with 1.15% of parameters for approximately $135 of GPU time on a single NVIDIA L40S.

Key Results

Model	FACTS Grounding Score	Parameters	Evaluation Protocol
Gemini 2.5 Pro Preview	87.80%	Proprietary	Kaggle leaderboard
Gemini 2.5 Flash	85.30%	Proprietary	Kaggle leaderboard
Claude 3.5 Sonnet	83.80%	Proprietary	Paper (Jacovi et al.)
Nehanda v3.1	82.21%	27B	3-judge majority vote (Gemini 3.7, GPT-OSS 120B, GLM-5)
GPT-4o	80.00%	Proprietary	Paper (Jacovi et al.)
Gemma 3 27B	74.90%	27B	Kaggle leaderboard

Nehanda v3.1 outperforms Gemma 3 27B — the same-size open-weight model from Google — by 7.31 percentage points. The gap is attributable to the training pipeline, not the base model: both are 27B, both are open-weight, but only Nehanda has been fine-tuned for source fidelity.

Training Pipeline

Nehanda v3.1 uses a five-stage stacked QLoRA pipeline. LoRA adapters are trained across all stages to build epistemic discipline while preserving core base capability.

Stage	Purpose	Learning Rate	Sequence Length
1. Epistemic Foundation	Core calibration: refusal when unanswerable, uncertainty expression	2e-4	2048
2. Evidence Hardening	Strict source-boundary enforcement, claim verification	5e-5	2048
3. RAG Synthesis	Multi-document synthesis with inline citation & conflict preservation	2e-5	4096
4. Constitutional Alignment	Preference optimization pairing grounded vs fabricated responses	5e-6	2048
5. Epistemic DPO	Advanced preference tuning contrasting evidence-based vs sycophantic reasoning	5e-6	2048

The learning rate decays across stages because each stage builds on an increasingly structured foundation — large updates late in the pipeline would disrupt the epistemic behavior installed earlier.

Base Model

Architecture: Qwen3.6-27B VL (native vision-language)
Context window: 262,144 tokens
Vision: Integrated vision encoder (training is text-only SFT/DPO, vision weights untouched)
Training data: Energy regulatory documents, intelligence analysis reports, general-purpose synthesis tasks
Training cost: ~$135 GPU time on a single NVIDIA L40S

Prompt Schema

Nehanda v3.1 uses a persona-based prompt schema (SEP-020):

{persona}

### Task:
{input}

### Response:

Model Variants

Variant	Format	Use Case
`asoba/nehanda-v3-27b`	Full weights (HuggingFace)	Full-precision inference, further fine-tuning
`asoba/nehanda-rag-synthesis-27b-gguf`	Quantized GGUF	Local inference via llama.cpp or LM Studio

Trade-offs

Nehanda v3.1 sacrifices general capability for epistemic reliability. The model is not trained for creative writing, code generation, or open-ended chat. It is trained to read documents and say what they support.

For applications where source fidelity is the core capability — regulatory analysis, intelligence assessment, due diligence, academic research — the trade-off is favorable. For applications where general capability matters more, a frontier model is the better choice.

Citation

Samudzi, S. (2026). Epistemic Fine-Tuning of Open-Weight LLMs for Deep Research:
Nehanda v3.1 and the FACTS Grounding Benchmark. Asoba Corporation Technical Report.
Model: asoba/nehanda-v3-27b.

Nehanda Research Assistant