Asoba Asoba Open Source

Providers & Inference Backends

The provider layer provides a unified interface across cloud frontier models and local self-hosted engines. Nehanda CLI features an interactive Goose-style configuration wizard, automatic live model discovery, provider credential isolation, and capability probing for tool use and reasoning tokens.


Unified /model Configuration Wizard

Provider selection and credential setup are handled in a single interactive terminal flow via /model:

❯ /model

◆  Which provider?
│  ● OpenRouter
│  ○ Anthropic Claude
│  ○ Ollama (Local or LAN)
│  ○ LM Studio
│  ○ Nehanda Cloud
│  ○ Zhipu AI
│  ○ Any OpenAI-compatible API
└
◆  API key
│  sk-or-v1-…
└
◆  API host
│  https://openrouter.ai/api/v1
└
  Fetching models from provider…
◆  Model
│  ● openai/gpt-4o
│  ○ anthropic/claude-3.5-sonnet
│  ○ meta-llama/llama-3.3-70b-instruct
│  ○ deepseek/deepseek-chat
└
  ✓ Provider / host / model saved — session active

Wizard Features


Provider Code Paths

All providers route through one of two execution paths, both supporting streaming completions and the deterministic turn loop:

Code Path Providers SDK / Client Native Tool Support
Anthropic SDK Anthropic Claude, Bedrock @anthropic-ai/sdk Yes (tool_use blocks)
OpenAI-Compatible OpenRouter, Ollama, LM Studio, Zhipu, Nehanda, Custom Custom SSE streaming client (openaiCompat.mjs) Yes (or [TOOL_CALL] rescue path)

Supported Providers

1. OpenRouter

Access dozens of frontier and open-weights models through a single API key:

2. Anthropic Claude

Connects using the official Anthropic SDK with native tool calling:

3. Ollama (Local or LAN)

Connects to local or remote Ollama instances:

4. LM Studio

Connects to local inference servers:

5. Nehanda Cloud (vLLM)

Connects to Nehanda’s domain-tuned Qwen 27B model:


Wire Model Resolution

resolveWireModel(model_config) translates configured model IDs into the exact string required by the provider endpoint:


Environment Variables

Variable Provider Purpose
OPENROUTER_API_KEY OpenRouter API key
ANTHROPIC_API_KEY Claude API key
ANTHROPIC_AUTH_TOKEN Claude OAuth bearer token
OLLAMA_BASE_URL Ollama Custom Ollama host URL
OPENAI_API_KEY OpenAI-compatible Generic API key
OPENAI_BASE_URL OpenAI-compatible Custom base URL endpoint
NEHANDA_API_KEY Nehanda Cloud API key