kev
6 TagsUpdated Apache-2.0by Jared Palmer
Decision models by Jared Palmer: a LoRA on a Qwen3.5 base plus a pointer head that scores every option at its own span, in one forward pass per question. Calibrated with Kev's own temperature.
decision0.76b4.2b7.9b
ollaya run kev --preset triage "I was charged twice for my subscription this month and want a refund."curl http://localhost:11435/api/decide \
-H "Content-Type: application/json" \
-d '{
"model": "kev",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
}'# Already using a TypeSafe SDK? Set TYPESAFE_BASE_URL=http://localhost:11435 instead.
import requests
response = requests.post(
"http://localhost:11435/api/decide",
json={
"model": "kev",
"state": "I was charged twice for my subscription this month and want a refund.",
"questions": {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, invoices and refunds",
"technical": "Bugs, errors and outages",
"account": "Login, profile and settings"
}
},
"refund": {
"type": "noul",
"instructions": "Is the customer asking for a refund?"
}
}
},
)
answers = response.json()["answers"]
print(answers["department"]["choice"], answers["refund"]["noul"])const response = await fetch("http://localhost:11435/api/decide", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
model: "kev",
state: "I was charged twice for my subscription this month and want a refund.",
questions: {
department: {
type: "choice",
instructions: "Which team should handle this?",
criteria: {
billing: "Payments, invoices and refunds",
technical: "Bugs, errors and outages",
account: "Login, profile and settings"
}
},
refund: {
type: "noul",
instructions: "Is the customer asking for a refund?"
}
}
}),
});
const { answers } = await response.json();
console.log(answers.department.choice, answers.refund.noul);Models
Every tag| Name | Accuracy | Latency | |||
|---|---|---|---|---|---|
| Most accurateHighest typed-decisions accuracy of the family. | |||||
| kev:9b-fp16Largest Kev (Qwen3.5-9B base, LoRA and pointer head). Needs about 17 GB of memory; best on a 24 GB GPU. This tag computes in fp16 on NVIDIA GPUs: faster than kev:9b, with at least 99.5 % of its decisions the same. On a CPU it runs, slowly; use kev:9b there. | 0.723 | 125 ms | |||
| FasterFastest first; accuracy as measured on the bench. | |||||
| kev:0.8bQwen3.5-0.8B base, round 15, 0.460 on typed decisions: small and fast. | 0.460 | 62 ms | |||
| kev:4b-fp16Kev on Qwen3.5-4B base (LoRA and pointer head): more accurate than 0.8b out of domain. Needs about 9 GB of memory. This tag computes in fp16 on NVIDIA GPUs: faster than kev:4b, with at least 99.5 % of its decisions the same. On a CPU it runs, slowly; use kev:4b there. | 0.669 | 98 ms | |||
| kev:4bQwen3.5-4B base, round 10, 0.669 on typed decisions. Best on a GPU. | 0.669 | 171 ms | |||
| kevlatestSame as kev:4b. | 0.669 | 171 ms | |||
| kev:9bQwen3.5-9B base, 0.722 on typed decisions: the most accurate Kev. Needs a 24 GB GPU. | 0.722 | 248 ms | |||
Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.
Readme
Kev is a family of decision models by Jared Palmer: a LoRA adapter on a Qwen3.5 base model plus a small pointer head, released under Apache-2.0. For each question it lays out the state, the question and one span per option, then scores every option at the end of its own span against the question, in one forward pass. It never generates text.
Models
| Tag | Base | Checkpoint | Params | Typed-decisions accuracy | Out-of-domain (upstream) |
|---|---|---|---|---|---|
kev:0.8b | Qwen3.5-0.8B | round 15 | 0.76B | 0.460 | 0.697 |
kev:latest, kev:4b | Qwen3.5-4B | round 10 | 4.2B | 0.669 | 0.838 |
kev:9b | Qwen3.5-9B | 2026-09-21 | 7.9B | 0.722 | 0.852 |
Typed-decisions accuracy is the argmax against the majority label on all 400 typed-decisions states. For comparison, decider:2b scores 0.608, nli 0.536 and laya:en 0.357. The labels have low annotator agreement, so compare the numbers against each other rather than reading them as absolutes. The last column is upstream's locked out-of-domain test (transfer-v4); Jev scores 0.857 on its development items.
Usage
ollaya run kev --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."
ollaya run kev:9b --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."Point any TypeSafe client at http://localhost:11435 and set the model to kev (the same as kev:4b), kev:0.8b or kev:9b. Before 2026-09-27, kev meant kev:0.8b.
Speed
- RTX 5090, through the HTTP API (Ollaya Bench): the triage preset (five questions) on a short message takes 62 ms on
0.8b, 171 ms on4band 248 ms on9bat the median. - State length: every question reads the whole state again, so the cost grows with the number of questions times the state length.
- CPU:
0.8btakes 2908 ms for five questions with a short state on the Threadripper 3960X (24 cores).4band9bbelong on a GPU.
How it works
- One row per question. The state comes first, then the question, then one span per option, then a decide token. The pointer head compares each option's closing token with the decide token.
- Calibrated. The scores are divided by the temperature each checkpoint ships, fitted upstream on development data: 2.35 (
0.8b), 2.41 (4b), 2.30 (9b). - Weights. From Hugging Face, pinned to a commit and verified by sha256: the Qwen3.5 base from Qwen, the adapter and
head.ptfrom Jared Palmer. The adapter is applied at run time, not merged; Ollaya hosts only the ONNX graph (10 to 14 MB). - Precision. The fp32 tags compute in fp32 on the BF16 base weights.
0.8bwidens them when it loads;4band9bkeep them BF16, half the memory, and widen each layer just before it runs. - Parity. Ollaya's Rust runtime matches upstream Kev exactly: identical token rows and option positions, and the same decision on every test question, on CPU and CUDA.
- fp16 tags.
kev:4b-fp16andkev:9b-fp16run the trunk in fp16 on NVIDIA GPUs (how). On the RTX 5090: 98 and 125 ms for five questions (fp32: 171 and 248 ms), 478 and 480 of 480 parity decisions the same as fp32, 0.669 and 0.723 on typed decisions.kev:0.8bhas no fp16 tag (its fp32 graph is as fast). On a CPU use the fp32 tags.
Limits
- Choice criteria. Give
choicecriteria as an object of option to description ({"refund": null, ...}). A plain list is rejected, as upstream Kev does. - Context. A question's row, state included, can be up to 8,192 tokens. Kev was trained on shorter states, so very long states are untested upstream.
- Memory. On the GPU, with requests of about 1,400 tokens,
4btook about 11 GB and9babout 18 GB; the weights stay BF16.9bneeds a 24 GB GPU and peaked at 22 GB on rows of 2,000 tokens, so much longer states do not fit. The downloads are 9.5 GB (4b) and 19.5 GB (9b).
Badge
For the model card or a README. Day-0 support for a new model
[](https://ollaya.dev/library/kev)