Insights · Applied AI

Cutting LLM cost with a Thai decision model: where it worked, where it failed

VRDevAI engineering · September 2026 · 7 min read

A lot of what we ask large language models to do is not writing at all. It is picking one answer from a list we already know. We tested whether a small, self-hosted Thai model could take that work off the LLM, and published the misses along with the wins.

The problem: paying LLM prices for a multiple-choice question

Look inside most AI assistants and you will find steps like these: which topic is this message about, which team should handle it, is this urgent, does this document answer the question. Each one is a multiple-choice decision. Yet the usual way to build them is to send the whole conversation to a general-purpose LLM, ask for JSON back, and parse the reply, which occasionally breaks.

That works, but you pay for every input token, wait for text generation, and get no reliable signal for how sure the model is.

What a “decision model” is

In September 2026 TypeSafe AI released Jev, a model that does not generate text. You give it a state (text or JSON) and typed questions, and it returns a typed answer with probabilities and a confidence value. It offers three question types:

• Choice: pick one of up to 255 options (intent, routing target).
• Score: rate against 2–10 ordered levels (difficulty, urgency).
• Noul: a yes/no statement returned as a probability from 0 to 1 (“does this passage answer the question?”).

An open re-implementation, OpenThai-SystemOne by iApp, uses the same request contract. It is Apache-2.0, has 0.8B parameters, and received roughly 5B tokens of extra Thai training. Because it can run on your own infrastructure, it was the one we tested.

What we tested

Both tests ran offline on a development machine (CPU, no GPU), in September 2026.

Test A: routing website visitors · 35 messages, mixed Thai and English

The job: sort messages of the kind visitors send to VIRA, the assistant on this website, into a fixed set of topics, including “other / off-topic”. We wrote the 35 test messages ourselves; no real visitor conversations were used.

35 hand-written messages. A small test set: read it as a direction, not a guarantee.

A hybrid setup (keywords first, model only when keywords say “other”) reached the same 28/35 while calling the model on 21 of 35 messages. The remaining errors were in categories whose meanings overlap. Sharper category descriptions are the fix.

Test B: does this document answer the question? · 8 short sample HR policy passages we wrote, 16 questions

What it does to cost

Take one routing call. Assumptions: a typical LLM routing prompt carries about 1,500 input tokens (instructions plus chat history) and the reply is about 80. The decision model needs only the latest message and two previous turns, about 400 tokens.

On these assumptions, the hosted decision model is roughly 56× cheaper per decision (we did not test the hosted service; our tests used the self-hosted model). Our assumption is that 20–30% of messages (low confidence, or genuinely open-ended) would still go to the LLM, so the saving applies to the rest. The same “decide cheaply first” idea also applies one level up: score how hard a request is, and send the easy ones to a cheaper model.

The pattern we recommend

Log the model version, the choice and the confidence for every decision. That is how you tune the threshold and how you answer an auditor later.

Before you put this in production

• Personal data (PDPA). According to public reports (September 2026), TypeSafe does not use requests for training and offers zero data retention to enterprise customers, but does not state where data is processed. Check the vendor's current terms yourself. A self-hosted model can keep personal data on infrastructure you control, but you still need a lawful basis, access control and a retention rule. This is engineering guidance, not legal advice.
• Never expose the model server directly. The reference server ships without authentication. Keep it on an internal network or behind your own gateway.
• User text can steer the answer. A crafted message can change a classification. Do not make it the only gate for anything security-related; pair it with rules you control.
• Pin the model version. A “latest” tag can change underneath you and silently move your confidence threshold.
• Know the blind spots. Counting, date comparison, multi-step reasoning, and saying “there is no answer”.

The takeaway

A decision model does not replace your LLM. It is a cheap first step in front of it. It is useful when the answer is a choice, when you actually use the confidence value, and when the data stays where it should. Where the job is explaining, writing or reasoning, keep the LLM.

If you want to know where a decision step like this would pay back in your workflows, tell us about the process, or ask VIRA in the corner of this page.

Sources

• Jev (AI model), Wikipedia
• A deep dive into Jev, flaviocopes.com (pricing, limits, weaknesses)
• iapp/OpenThai-SystemOne, Hugging Face (licence, size, Thai training)
• 70B-class LLM price: one public list price for Llama 3.3 70B inference, September 2026. Prices vary by provider.