Skip to content

pplx-decider

3 TagsUpdated Apache-2.0by Perplexity AI

Perplexity's decision model: Qwen3.5-4B distilled from its 27B decision models, with a readout over 255 answer codes. Every question of a request shares one sequence and reads the whole request, and the probabilities are calibrated with Perplexity's temperature. Up to 255 options.

decisionmultilingualfine-tuned4b
ollaya run pplx-decider --preset triage "I was charged twice for my subscription this month and want a refund."
NameAccuracyLatency
Most accurateHighest typed-decisions accuracy of the family.
pplx-deciderlatestSame as pplx-decider:flash.0.726127 ms
FasterFastest first; accuracy as measured on the bench.
pplx-decider:flash-fp16pplx-decider-flash v1.2: Qwen3.5-4B post-trained by Perplexity (distilled from its 27B decision models), with a 255-code readout. Every question of a request shares one sequence, read at a marker after each question; the full-attention layers see the whole request. Up to 255 options and levels, 8,192 tokens per sequence. Needs about 9 GB of memory. This tag computes in fp16 on NVIDIA GPUs: faster than pplx-decider:flash, with at least 99.5 % of its decisions the same. On a CPU it runs, slowly; use pplx-decider:flash there.0.72690 ms
pplx-decider:flashpplx-decider-flash-v1.2 (Qwen3.5-4B), about 9 GB in memory.0.726127 ms

Accuracy: typed-decisions, argmax against the majority label on 400 states (left out for a tag trained on that data). Latency: the median five-question request on an RTX 5090, Ollaya Bench.

Readme

pplx-decider is Perplexity's family of decision models, released under Apache-2.0 with their code in each model repository. Ollaya runs the flash model: Qwen3.5-4B post-trained by Perplexity, distilled from its 27B decision models, with a readout over 255 answer codes. A request is one sequence: the state, then every question with its options, each followed by a marker where the readout picks the answer. It never generates text.

Models

TagBaseCheckpointParamsDecision Index 0.3 (upstream)
pplx-decider:latest, pplx-decider:flashQwen3.5-4Bflash v1.2, 2026-10 (pplx-decider-flash-v1.2)4B58.9 joint, 60.1 one question per request

The Decision Index numbers are from the model card. The 27B models (pplx-decider-v1.2, -v1.1-27b) use the same layout but need about 54 GB of memory in Ollaya's fp32 runtime, so they are not in the library.

Usage

ollaya run pplx-decider --preset triage "My order never arrived and support ignores me. Refund me today or I'm switching to your competitor."

Point any TypeSafe client at http://localhost:11435 and set the model to pplx-decider.

Speed

  • RTX 5090, over HTTP (Ollaya Bench): 127 ms at the median for five triage questions (one forward pass), 232 ms per question on Bespoke Labs' public benchmark and 380 ms per five-question typed-decisions request.

Accuracy

Measured by Ollaya on an RTX 5090: 0.824 macro accuracy on Bespoke Labs' public benchmark (3,880 human-labeled questions, pooled ECE 0.016) and 0.726 on the 400 typed-decisions states. Its public score is the highest of any model Ollaya or Ollama 0.40 serves, Ollama's 27B pplx-decider included (0.816 public, 0.702 typed). See the comparison with Ollama.

How it works

  • One sequence per request. Every question sees the state and the other questions. The model card finds the flash model more accurate with one question per request on most multi-question tasks: send related questions together and independent ones apart.
  • Calibrated. The readout logits are divided by the temperature Perplexity fitted on held-out requests (1.55).
  • Weights. The files download from Hugging Face, pinned to a commit and verified by sha256, byte for byte Perplexity's. Ollaya hosts only the ONNX graph (11 MB).
  • Parity. Ollaya's Rust runtime matches upstream pplx_decider in fp32: identical token sequences, the same rejections and the same decision on all 696 test questions, probabilities within 6.8e-6.
  • fp16 tag. pplx-decider:flash-fp16 runs the trunk in fp16 on NVIDIA GPUs (how): 90 ms for five questions on the RTX 5090 instead of 127 ms, 695 of 696 decisions the same as pplx-decider:flash on the parity set, and 0.726 on typed decisions. On a CPU use pplx-decider:flash.

Limits

  • Context. 8,192 tokens per sequence. A longer request is split by its questions, every part keeping the whole state; a question that does not fit alone is rejected.
  • Not ported. Images (upstream accepts them).

Badge

Run on Ollaya

For the model card or a README. Day-0 support for a new model

[![Run on Ollaya](https://ollaya.dev/badge/run-on-ollaya.svg)](https://ollaya.dev/library/pplx-decider)