Fast decisions: a small model so the large ones stay quiet

What this layer does

The typed decision model returns one of two typed answers: a choice among options I list, or a calibrated score between 0 and 1. It does not write text and it does not chat. If it cannot answer, the call falls back to its previous route, so no decision is lost.

Source: my verdict log, test rows excluded, 21 Sep 2026 to 1 Nov 2026, refreshed every 6 hours.

Decisions per day

Source: verdict log on my server, read at build time.

How fast, how often it falls back

Fallback means the typed model did not return a usable verdict and the call site used its backup route. It measures availability, not correctness. I do not have a labelled truth set for these decisions, so I do not report an accuracy figure for my own data.

What it replaced

A call site is one place in my code that asks for a decision. One site can ask more than one type of question. These are counts of call sites, not measured before-and-after savings.

Where the decisions come from

About three quarters of all decisions come from one private workflow in my work. I report its volume and nothing else about it.

What it costs

On a measured sample of 50,695 calls since 7 Oct 2026, the typed decision layer processed 30,469,510 input tokens and 5,617,334 output tokens (36,086,844 total tokens, about 712 tokens per call) and cost $1.2797, an average of about $0.0500 per day.

Output tokens are free on this service; list price was $0.042 per million input tokens.

Earlier verdicts in the log are not metered for tokens. Because I do not record the tokens a chat model would have used for each decision, I do not publish an estimated reduction figure. The measured fact is narrower: 197,937 decisions were handled by this layer instead of calling a chat model.

Where I never use it

A typed model is probabilistic, so I exclude it by file from anything irreversible or trust critical. It never decides:

• Sending email or messages
• Deleting files or approving cleanup
• Anything touching money, finance or payments
• Secrets, credentials and vaults
• Lists of people who asked not to be contacted

These paths stay deterministic plain code or wait for my explicit approval. The exclusion list is checked by the same registry that manages live sites.

Limits

The typed decision model is not free of mistakes. On public classification datasets, a third party measured this model at 76 to 95 depending on the dataset, against 77 to 96 for a large chat model and 88 to 91 for a fine-tuned small classifier. Those benchmarks come from public datasets and are not my data; I treat them as an external reference, not a result on my workflows.

In my own use, 0.05% of calls returned no usable answer and went to the backup route. It answers only the narrow, typed questions I configure; it cannot reason through anything open-ended or explain why it reached a decision.

The current service runs on limited credit, so I am testing a free and local typed decision model to take over the same role. This page describes an architectural role rather than a permanent vendor.

Decisions that matter stay deterministic or come to me because a cheap wrong answer that cannot be undone is not cheap.

How typed decisions fit into my broader cost strategy is documented in the savings ladder on Model routing and tiered compute