Bespoke Labs’ Bespoke-Nimble-9B is a nine-billion-parameter model adapted from Qwen3.5-9B to make decisions within a defined set of answers. Give it text and a schema of boolean or enum questions, and it returns a selected answer and probabilities for the available options. The project positions this as an open alternative to TypeSafe AI’s Jev, with a public model, training recipe, evaluation setup and data-curation pipeline. DataCamp’s technical overview describes that implementation.
The attraction is the ability to inspect and adapt the process behind those decisions. The performance evidence is narrower: Bespoke reports roughly 90% accuracy on its own curated evaluation, compared with roughly 93% for Jev. That result supports a closer look at the training approach; it does not establish equivalent performance across applications.
How Nimble makes typed decisions
The schema defines what the model may choose. A boolean question has two answers; an enum is a list of permitted string values. According to the Nimble repository, the latest release supports between 1 and 255 choices per enum field. The interface accepts text rather than inputs such as images.
DataCamp describes a flat schema containing boolean or enum questions and says training and inference operate directly over candidate logits, rather than generating an answer token by token. These logits are the scores associated with the candidate answers. The repository says Nimble makes each decision in one step without first writing out reasoning. Bespoke’s announcement describes the serving approach as parallel constrained decoding.
Those descriptions explain the focus on a restricted answer space: the output is a decision among supplied choices, accompanied by probabilities. Developers looking for this behavior should assess whether their task can be expressed through the supported fields. The documented interface provides a concrete boundary for evaluation rather than evidence of a general-purpose agent that can handle any task.
Nimble uses LoRA, or Low-Rank Adaptation, on Qwen3.5-9B; DataCamp specifies a rank-16 adapter for typed decision making. It also describes a preparation step that downloads bespokelabs/Bespoke-Nimble-9B and, when necessary, merges the adapter with the pinned Qwen3.5 base model. That is a model adaptation workflow rather than a new model trained from scratch.
The training recipe emphasizes difficult negatives
In his launch announcement, Bespoke Labs CEO Mahesh Sathiamoorthy described contrastive data curation: slightly changing facts to create negative examples intended to improve discrimination. He said the data was entirely synthetic, covered 10 categories and was split into training and evaluation sets.
The repository records publication of 2,676 training examples and 324 held-out examples on September 20, 2026. It also acknowledges that those examples had been omitted from the first release by mistake. Their subsequent publication matters because readers can examine the material underlying the reported result, rather than relying solely on a score.
Sathiamoorthy characterized training as distillation-free, with Jev used only for evaluation, and said reinforcement learning had not been used at that stage. This distinction matters when interpreting the project as an open Jev alternative: the stated method adapts Qwen to the decision task without using Jev as a training teacher.
The announcement attributes importance to contrastive examples, but also says no ablation experiments were performed. Without tests that isolate the contribution of that component, the improvement cannot be assigned to data curation alone. The evidence supports the reported outcome of the combined approach, while leaving the relative contribution of its parts unresolved.
What the accuracy numbers establish
AI Engineering’s report gives the more precise figures for the team’s 324-example held-out test set. Bespoke’s announcement uses rounded percentages and also reports the Qwen baseline.
| Model | Reported accuracy | Evaluation context |
|---|---|---|
| Qwen baseline | 66% | Bespoke’s curated evaluation, rounded launch figure |
| Bespoke-Nimble-9B | 90.12% | Team’s 324-example held-out test set |
| Jev | 93.21% | Same reported held-out comparison |
These are reported project results, not an independently reproduced benchmark in the supplied evidence. The rounded 90% and 93% figures are consistent with the more precise report. Nimble remained below Jev in that comparison.
Some coverage frames the result more strongly. explainx.ai’s article describes the score as matching Jev in its introductory summary, while also identifying the result as self-reported and not independently verified. The numerical comparison supports proximity on this test, rather than equality.
Sathiamoorthy explicitly warned that there was no standard benchmark for measuring this performance and that Nimble could fare much worse than Jev on other benchmarks. The repository separately records a public benchmark suite, added by Edgar Dyck on September 18, that can run both models on the same records from 13 public subsets with human labels. That suite supplies a way to broaden testing; its availability does not supply an additional accuracy result here.
Context limits and probability calibration changed after launch
The repository’s September 24 update describes a checkpoint with an 8,192-token context and up to 255 choices per field. It preserves the original release under the original-2676 tag and tells users to update their checkout before loading the new release.
The hosted API’s prompt limit had increased to 8,192 tokens per question on September 19. However, the repository says the model was trained on prompts of up to 2,048 tokens, making shorter prompts better tested. DataCamp’s supplied preparation example also specifies max_input_tokens as 2048. Developers should distinguish the later supported context limit from the documented training range and the limit in that example.
There is a similar distinction around probabilities. The launch announcement describes calibration as implicit and says training examples need no probability labels. A September 22 repository update subsequently reports fitting a temperature so probabilities better reflect answer correctness, without changing the selected answers. It warns users who compare probabilities or Score values against thresholds to test those thresholds again. The September 24 checkpoint update lists its default as T=1.0.
These updates make checkpoint and probability settings relevant to evaluation. Accuracy concerns which answer is selected; calibration concerns how closely its probability reflects correctness. A threshold tested with one set of probabilities should not be assumed to behave identically after those probabilities change.
Local deployment offers control, with concrete limits
The repository documents execution on Apple Silicon Macs or machines with NVIDIA GPUs. It says unquantized 9B weights alone occupy about 18 GB; that figure should not be treated as a complete runtime memory requirement. Sathiamoorthy reports approximately 100 milliseconds on an H100, but the supplied announcement does not specify enough measurement conditions to turn that into a general latency guarantee.
AI Engineering reports that the training data, weights, recipes and inference code are public under Apache 2.0. For developers studying or adapting typed decision models, that openness is the clearest practical value of the release. Its usefulness in a particular workflow still depends on testing the supported schema, relevant text lengths, accuracy and probability thresholds against that workflow’s own requirements.
