Jev vs BERT: deciding better or deciding at home

Lee este artículo en español.

On 15 September 2026, the US company TypeSafe launched Jev, an AI model that does not write: it reads a text, picks among the options you give it and tells you how sure it is. We tested it alongside Kev, its open replica, on almost 500 texts, including real documents from Spain's Official State Gazette (BOE). Jev let the fewest sensitive texts through.

TypeSafe sells Jev on speed and price. There is a catch, though: to use it, you have to send the text to its servers, which its privacy policy places in the United States. And when the question is whether a text may leave your company, where the decision is made matters at least as much as how well it is made.

That is the comparison we care about with BERT, the family of small models that has been making decisions about text, without writing a word, since 2018. We should say it up front: Moviwa, our product, uses models from this family.

Jev and BERT in two minutes

An assistant like ChatGPT writes an answer. Jev writes nothing. You give it a text and a set of options ("public", "internal", "confidential"…) and it returns a probability for each one. You can change the question with every request, without training anything.

It charges $0.042 per million input tokens (the word fragments these models use to measure text), and output is free. All the requests in our study, just over 2,000 including the robustness tests, cost $0.065. It is in early access and runs only in the cloud.

Kev, by contrast, can be installed on your own machine: we analysed it in depth a few weeks ago.

BERT, which Google published in 2018, also decides without writing: it reads a text and gives it a label. It is small, between 110 and 340 million parameters in the original version, against the billions of a chat model. That is why it answers in milliseconds on an ordinary processor, with no graphics card, and can run on your own server.

The trade-off: it cannot do your task until you train it with examples. And how well it decides depends on those examples.

What we measured

We gave Jev and Kev the same question: which of five confidentiality levels does each text belong to. They both got the same guide with the definitions of each level, fixed in a protocol we registered before running the tests, and two groups of texts:

• 329 made-up texts written for the test: emails, chat messages and document excerpts, with and without sensitive information. Four Anthropic models (Claude Opus, Sonnet, Haiku and Fable) wrote them from our instructions. Three of them (Opus, Sonnet and Fable), without seeing the expected answer, confirmed the label of each one, and they agreed on every text about whether it was sensitive or not.
• 150 real excerpts from the BOE from September 2026. They are public by definition, so any "confidential" flag on them is a false alarm.

We only counted whether each text ended up as sensitive or not, because that is the decision that matters for letting it out, and two kinds of error:

• A leak is a sensitive text the model lets through. It is the error that does the most damage.
• A false alarm is a harmless text the model blocks. It leaks nothing, but it interrupts whoever wrote it for no reason.

Jev let the fewest sensitive texts through. Kev let more through, but it raised no false alarms at all, not even on the BOE. With this number of texts, the difference in leaks between the two is not statistically significant: a paired test (McNemar, corrected for multiple comparisons) gives p = 0.06.

The leaks clustered in the same kind of text: secrets that are secret only because they have not been published yet. They were 6 of Kev's 8 and Jev's only one. A bid for a public tender, the rate agreed with a customer or a takeover offer before it is announced look a lot like any other business text. No model knows what your company has already made public and what it has not.

There is no BERT-type model in the table. Each one is as good as the examples it is trained on, so there is no reference model to compare with, and the ones we use in Moviwa are part of a commercial product whose figures we do not publish. That is why this article does not claim that a BERT-type model is as accurate as Jev. What does work for anyone is the method: a test bench like this one, with leaks and false alarms counted separately, before you trust a model with a decision.

What this measurement does not tell you

• There are no real confidential documents. Leaks were measured on made-up texts. The real part, the BOE, only measures false alarms.
• The made-up texts are too clear-cut. We asked for texts with a well-defined level, and the fact that the reviewers agreed on all of them confirms it. That is good for fixing the labels, but it leaves out the borderline cases. On real messages, both models will fail more.
• The generators and reviewers are models from the same company, Anthropic's Claude family. None is from TypeSafe. It is a bias we could not remove.
• The guide matters. Jev and Kev got the same definitions as the reviewers, which favours them. We also tried a different wording: Kev raised false alarms on 15 of the 150 BOE documents, and Jev went from 1 leak to 3.
• The timings do not come from the same machine. Jev includes the network from Granada; Kev ran on a Mac's graphics card; the BERT-type model, on the same Mac's processor.
• Jev changes. We measured version 1.13 on 5 October 2026. It is in early access, and its results and terms may vary.

Why the best decider is not always the right choice

Jev deciding well does not make it the best option for every company. There are three reasons to think twice, and the first one weighs the most.

To ask it, you have to send it

Jev decides on TypeSafe's servers. To find out whether a message contains a secret, you have to send it there first, whole.

If what you want is to stop sensitive information from leaving the company, sending it out to ask is starting at the wrong end. Its privacy policy says it does not train on what it receives, but the text leaves all the same.

If those texts contain personal data, Chapter V of the General Data Protection Regulation (GDPR) treats sending them to the United States as an international transfer. Your company would have to check which safeguard covers it and document it: for example, whether the recipient is certified under the EU-US Data Privacy Framework or signs standard contractual clauses. That is the first thing to ask TypeSafe.

A BERT-type model, on the other hand, can decide inside your own infrastructure, without the text going through any outside provider.

Every text has to wait for the answer

If you check every message before it goes out, or every document before it is filed, the time the decision takes is added to each one. In our test, Jev took almost 300 milliseconds per text, network included.

Kev took over 700 on a Mac's graphics card, and its own project estimates it would be 20 to 54 times slower on an ordinary processor. In our tests, a fine-tuned BERT-type model took under 20 milliseconds median on the same Mac's processor, without using the graphics card. With long documents, like those in the BOE, it was around 100.

A file that only changes when you change it

Jev is a service run by a young company, and its price, its behaviour and its continuity depend on that company. A fine-tuned BERT-type model is a file on your server: it always gives the same answer to the same text and only changes when you retrain it. If one day you need to explain why a document went out, you know which version decided and you can repeat the decision.

What about Kev, which is open and can run in-house?

Kev passes the first and third tests: you can host it wherever you like and it is a file you control. It fails the second. It needs a graphics card to answer in time. So it fits better in a different role: teaching.

A big model teaches, a small one decides

The big drawback of a BERT-type model is the labelled examples. Gathering them from a company's real documents is slow and delicate, because those are exactly the documents that should not be passed around.

A big model can do that work. It can write thousands of made-up texts for each category, or label the ones you already have. A public model such as ALIA, the Spanish government's model, Kev on your own machines, or Jev, with texts that are allowed to leave only, can all do it.

Ask it for borderline cases too, because made-up texts tend to be too clear-cut. The small model is trained on those examples, and before it goes to work it is checked against a test bench that includes real documents.

When each one suits you

Jev fits when your categories change often, you have no examples and the texts can leave with no legal or contractual problem. Kev, when you have no examples either but the texts cannot leave and you have a graphics card. A small model, when the question is always the same, the volume is high and you need to audit what it decides.

The question that matters

Before choosing a model to decide on sensitive information, it pays to answer four questions: how accurate it is, who reads the texts while it decides, how long it takes, and what happens if the provider changes its terms. Jev wins on the first. A small model in your own infrastructure wins on the other three. When the texts belong to your customers, however accurate Jev is, we put the security of the data first.

If you are choosing a model to classify or filter sensitive information, we measure it with you, as in this test, keeping leaks and false alarms apart. Models you install in-house, with your texts and on your infrastructure; cloud models, only with texts that are allowed to leave. It is part of our services. Tell us about your case.

Sources

• Jev announcement: TypeSafe, "Introducing System One models and Jev", 15 September 2026, accessed 5 October 2026 — typesafe.ai
• TypeSafe privacy policy, accessed 5 October 2026 — typesafe.ai/legal/privacy-policy
• Kev repository, version 1.0, Apache 2.0 licence — github.com/jaredpalmer/kev
• Devlin, Chang, Lee and Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", 11 October 2018 — arxiv.org/abs/1810.04805
• Regulation (EU) 2016/679 (GDPR), Chapter V, transfers of personal data to third countries — eur-lex.europa.eu
• Commission Implementing Decision (EU) 2023/1795 of 10 July 2023 on the EU-US Data Privacy Framework — eur-lex.europa.eu
• Spain's Official State Gazette (BOE), September 2026 issues — boe.es
• Montevive's own measurement, 5 October 2026, unpublished: Jev 1.13 via its API and Kev-4B 1.0 run locally, on 329 made-up texts and 150 BOE excerpts, with a protocol registered before running the tests; timings of a fine-tuned BERT-type model on the same computer's processor.

This article reflects the state of Jev and of the regulatory framework as of 8 October 2026. It is for information only and is not legal advice.