Providers

The provider layer abstracts away the differences between LLM providers behind a unified interface. Nehanda CLI supports five provider categories, each routing through one of two code paths.

Provider Code Paths

Both paths implement the same behavior: stream a chat completion, collect tool calls, execute them, and loop. The only differences are the SDK used and how tool schemas are sent.

Supported Providers

Nehanda Cloud

The default provider. Connects to the Nehanda vLLM deployment behind a Lambda proxy.

Tool calling: Uses the [TOOL_CALL] rescue path. The nehandaMlProxy Lambda strips tools and tool_choice from requests, so tool schemas are injected into the system prompt and parsed from model text.

Anthropic Claude

Uses the official Anthropic SDK with native tool support.

Authentication: Supports two modes:

• API key: ANTHROPIC_API_KEY environment variable
• OAuth bearer: ANTHROPIC_AUTH_TOKEN environment variable (uses oauth-2025-04-20 beta header)

Tool calling: Native. Tools are sent in the tools parameter and results come back as structured tool_use content blocks.

LM Studio

Connects to a locally running LM Studio instance.

Tool calling: Native when available. Tool set is reduced to six core tools (Read, Write, Edit, Bash, Glob, Grep) to conserve context window.

Ollama

Connects to a local or remote Ollama instance.

Tool support detection: On first use, the engine calls Ollama’s /api/show endpoint to check if the model’s chat template includes .Tools or its capabilities list includes tools. Results are cached per model. If not supported, falls back to the [TOOL_CALL] rescue path automatically.

Tool calling: Native if detected, rescue path if not.

Zhipu

Connects to Zhipu AI’s GLM models.

Tool calling: OpenAI-compatible native.

Any OpenAI-Compatible API

Connects to any server that implements the OpenAI chat completions API.

Switching Providers

From the REPL:

❯ /model
# Lists available models from the active provider

❯ /model claude-sonnet-4-20250514
# Switches model (stays on current provider)

❯ /config set model_config.provider ollama
❯ /config set model_config.base_url http://localhost:11434/v1
❯ /model llama3
# Switches to Ollama with llama3

From a settings file:

{
  "model_config": {
    "provider": "ollama",
    "model_id": "deepseek-coder-v2:latest",
    "base_url": "http://my-server:11434/v1"
  }
}

Model Resolution

resolveWireModel(model_config) maps the configured model_id to the actual wire format the provider expects. For most providers this is a pass-through, but some providers require transformations (e.g., stripping provider prefixes).

Environment Variables