nimble

Name: nimble
Author: Bespoke Labs

4 TagsUpdated Sep 30, 2026Apache-2.0by Bespoke Labs

Decision model by Bespoke Labs: a LoRA on Qwen3.5-9B trained on contrastive pairs. Nimble reads the whole request as a JSON schema and scores each option by the next-token logit of its code; Ollaya applies the author's temperature, so its probabilities are calibrated. Up to 255 options.

decisionfine-tuned9b

ollaya run nimble --preset triage "I was charged twice for my subscription this month and want a refund."

curl http://localhost:11435/api/decide \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nimble",
    "state": "I was charged twice for my subscription this month and want a refund.",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Payments, invoices and refunds",
          "technical": "Bugs, errors and outages",
          "account": "Login, profile and settings"
        }
      },
      "refund": {
        "type": "noul",
        "instructions": "Is the customer asking for a refund?"
      }
    }
  }'

# Already using a TypeSafe SDK? Set TYPESAFE_BASE_URL=http://localhost:11435 instead.
import requests

response = requests.post(
    "http://localhost:11435/api/decide",
    json={
        "model": "nimble",
        "state": "I was charged twice for my subscription this month and want a refund.",
        "questions": {
            "department": {
                "type": "choice",
                "instructions": "Which team should handle this?",
                "criteria": {
                    "billing": "Payments, invoices and refunds",
                    "technical": "Bugs, errors and outages",
                    "account": "Login, profile and settings"
                }
            },
            "refund": {
                "type": "noul",
                "instructions": "Is the customer asking for a refund?"
            }
        }
    },
)
answers = response.json()["answers"]
print(answers["department"]["choice"], answers["refund"]["noul"])

const response = await fetch("http://localhost:11435/api/decide", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "nimble",
    state: "I was charged twice for my subscription this month and want a refund.",
    questions: {
      department: {
        type: "choice",
        instructions: "Which team should handle this?",
        criteria: {
          billing: "Payments, invoices and refunds",
          technical: "Bugs, errors and outages",
          account: "Login, profile and settings"
        }
      },
      refund: {
        type: "noul",
        instructions: "Is the customer asking for a refund?"
      }
    }
  }),
});
const { answers } = await response.json();
console.log(answers.department.choice, answers.refund.noul);

Models

Every tag

Name	Accuracy	Latency	Size	Context	Input
Most accurateHighest typed-decisions accuracy of the family.
nimblelatestSame as nimble:9b.	0.665	1015 ms	19.5 GB	8192	Text · English
FasterFastest first; accuracy as measured on the bench.
nimble:9b-fp16Bespoke Labs' Nimble v2: a LoRA on Qwen3.5-9B that answers every question from the next-token logits of its option codes, calibrated with the author's temperature. Up to 255 options. Needs about 18 GB of memory; best on a 24 GB GPU. This tag computes in fp16 on NVIDIA GPUs: faster than nimble:9b, with at least 99.5 % of its decisions the same. On a CPU it runs, slowly; use nimble:9b there.	0.665	428 ms	19.5 GB	8192	Text · English
nimble:9b-v3Bespoke-Nimble-9B v3: v2's prompts with a new LoRA, served as the author's Decision Index engine. CC BY-NC 4.0 (non-commercial). Needs a 24 GB GPU.	0.647	942 ms	20.0 GB	32768	Text · English
nimble:9bBespoke-Nimble-9B v2 (LoRA on Qwen3.5-9B) with the author's temperature 2.18. Needs a 24 GB GPU.	0.665	1015 ms	19.5 GB	8192	Text · English

Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.

Readme

Nimble is a decision model by Bespoke Labs: a LoRA adapter on Qwen3.5-9B, trained on contrastive pairs (two examples that differ by one fact that flips the answer), released under Apache-2.0. For each question it reads the whole request as a JSON schema and scores every option by the next-token logit of its code (A, B, ...). It never generates text.

Needs Ollaya 0.8.0 or newer, the first release that runs the nimble-codes-v1 layout.

Models

Tag	Base	Checkpoint	Params	Typed-decisions accuracy
`nimble:latest`, `nimble:9b`	Qwen3.5-9B	Bespoke-Nimble-9B-v2 (2026-09-23)	9B	0.665
`nimble:9b-v3`	Qwen3.5-9B	Bespoke-Nimble-9B-v3 (2026-10)	9B	0.647

Typed-decisions accuracy is the argmax against the majority label on all 400 typed-decisions states, measured by Ollaya. nimble:9b-v3's adapter is licensed CC BY-NC 4.0: it may not be used commercially. It runs as the author's Decision Index engine serves it: options without a description get none, rows run to 32,768 tokens, and probabilities are the plain softmax the author reads (no temperature is shipped). On Bespoke Labs' public benchmark (13 human-labeled datasets, 3,880 questions), see the comparison with Ollama.

Usage

ollaya run nimble --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."

Point any TypeSafe client at http://localhost:11435 and set the model to nimble.

Speed

RTX 5090, through the HTTP API (Ollaya Bench): the triage preset (five questions) on a short message takes 1015 ms at the median (about 3,000 input tokens: each of the five rows carries the whole request).
State length: every question's row repeats the whole request (the state and every question's schema), so the cost grows with the number of questions times the request length.
CPU: a 9B model belongs on a GPU.

How it works

Prompt. Built exactly as the author's serving_schema.prepare_prompts builds it, with TypeSafe questions mapped as the author's /v1/systemone server maps them: noul to a true/false field, choice to an enum of its labels, score to an enum of its levels.
Calibrated. Probabilities use the author's recommended temperature, 2.179 (temperature_config.json). A plain softmax of the code logits, what Ollama returns, is overconfident.
Up to 255 options. Questions with more than 26 options use the author's serving extension: two-letter codes and the "short code" prompt.
Weights. Qwen3.5-9B from Qwen and Bespoke Labs' adapter, from Hugging Face, pinned to a commit and verified by sha256. The adapter is applied at run time, not merged; Ollaya hosts only the ONNX graph (14 MB).
Parity. Ollaya's Rust runtime matches the author's code in fp32 on CUDA: identical token rows, the same rejections and the same decision on all 492 test questions, probabilities within 6.5e-6. nimble:9b-v3 matches the author's Decision Index engine the same way on 498 questions, probabilities within 1.1e-5.
fp16 tag. nimble:9b-fp16 runs the trunk in fp16 on NVIDIA GPUs (how): 428 ms for five questions on the RTX 5090 instead of 1015 ms, 492 of 492 decisions the same as nimble:9b on the parity set, and 0.665 on typed decisions. On a CPU use nimble:9b.

Limits

Context. Each question's row, the whole request included, can be up to 8,192 tokens; a longer one is rejected, never cut.
State. The state must not be empty; instructions and choice labels must not be blank.
Memory. The weights stay BF16 in memory: about 18 GB, so a 24 GB GPU. With nimble:9b-v3, a row of 10,000 tokens overflows a 24 GB GPU: it runs, slowly, and slows the model until it is unloaded.

Badge

For the model card or a README. Day-0 support for a new model

[![Run on Ollaya](https://ollaya.dev/badge/run-on-ollaya.svg)](https://ollaya.dev/library/nimble)