Field tests

Every number we publish about the size of OPVS messages comes from this page. We wrote down the method before the first count, rendered every format with a program, counted on five tokenizers, and published the data and the script, so that anyone can repeat it.

Run: 8 November 2026. Method: written and committed before the first count. Repeat it: one command, below.

In short

• OPVS messages were smaller than every format we compared, on every tokenizer we ran. On 224 test messages, OPVS used about 21% fewer tokens than compact JSON, 42% fewer than indented JSON, 48% fewer than A2A JSON-RPC, 55% fewer than FIPA-style XML, and 6% fewer than plain English carrying the same fields.

• Most of the saving is in the routing line. With the message body removed, the OPVS routing line and context block are about 40% smaller than the same fields as compact JSON or YAML. The longer the body, the smaller the saving, because the body is the same text in every format.

• Memory notes: an OPVS note is 13–18% smaller than the same note as compact JSON or YAML, and 11–13% larger than the same note as one plain sentence. Notes are summaries: ours kept 86% of the facts of the memory they came from.

• Board tasks: about 42% smaller than YAML per task, and 51% as one board document.

• Symbols: 16 of the 17 marks are one token on every tokenizer we ran. /# is two tokens after a space, which is where it appears in a routing line.

• Not measured: the Claude and Gemini tokenizers (their token-count services refused our keys; Gemma 3 stands in for Gemini), and whether a model understands an OPVS message as well as a JSON one.

Method

The question. How many tokens does the same information take in OPVS, compared with the formats agents use today?

1. One record, every format, written by a program. Each test message is one record with the fields of an OPVS message. Eight renderers turn it into OPVS, indented JSON, compact JSON, the JSON style of the original paper, A2A JSON-RPC (message/send, spec 0.3), YAML, FIPA-style XML and plain English. The OPVS text is written by the shipped OPVS formatter and read back by the shipped parser. Nothing in any format is written by hand, so no format can be tuned against another.

2. Every format carries the same information. A field that is empty, or at its default, is left out of every format, because OPVS leaves it out. Routing fields that A2A has no slot for go in its metadata.

3. The test set. All synthetic, with no real mail or memories:

   • 240 messages across 12 kinds of work (support, sales, research, status, engineering, scheduling, billing, incidents, content, data enrichment, people, hand-offs), one third each with a short (1 sentence), medium (2–4) or long (5–9) body.

   • 51 memories of six kinds, each written as a list of facts, with a short OPVS note and a record of which facts the note keeps, written before any count.

   • 50 board tasks.

   • The original paper's 6 messages and 10 memories, rendered by the same programs, to compare with its numbers.

4. Five tokenizers. OpenAI cl100k_base and o200k_base (tiktoken 0.12.0); Gemma 3, Qwen 2.5 and Llama 3.2 (Hugging Face tokenizers 0.22.1, each file pinned by revision and SHA-256).

5. Statistics. For each tokenizer and baseline: the total saving (1 − all OPVS tokens ÷ all baseline tokens) with a 95% bootstrap interval (10,000 resamples, fixed seed), and the per-message spread. Breakdowns by body length, address form and "envelope only" (the message with its body removed) were fixed in advance.

Left out, as the method required: the 16 messages that use a sub-address (@alice+task1>bob). The parser of 8 November read them back wrong, so their OPVS form was not a correct message. That leaves 224 messages. The parsers have been fixed since (see Versions and changes).

One change after the first count: our own XML renderer repeated the message kind. Fixing it moved the XML saving from 55.6% to 55.2% (o200k). It is recorded in the method file with every other change.

Results

Messages (224)

Total saving of OPVS against each format, with the 95% interval:

One number hides a wide spread. Per message (o200k), the saving against compact JSON runs from 6% to 47% (median 25%; 10th–90th percentile 12–40%). Against plain English it runs from −10% to +31%: for 17 of the 224 messages the plain sentence is shorter, and 12 more come out even.

What decides the saving: the body

The body is the same text in every format, so the longer it is, the smaller the saving (o200k):

The last column is what the notation itself does. The address form changes little: 20–25% against compact JSON for every form (direct, domain, pool, team, everyone).

The paper's own 6 messages, rendered by the same programs, give 37.8% against the paper's JSON style (cl100k), close to the paper's 38%. So the paper's figure is fair for those six short messages; on a wider mix of messages the same JSON style gives 33%.

Memory notes (51)

Tokens of each format ÷ tokens of the OPVS note (above 1 means the OPVS note is smaller):

Why this is not the paper's 2.7x. The paper compared its notes with wordy prose: its ten prose memories average 92 tokens (cl100k) and their notes 33, and our programs reproduce its ratio on them (2.74x). Our 51 memories are written as short, plain facts (42 tokens on average; their notes 39). A ratio against prose measures how wordy the prose was. The rows that hold the same note in each format measure the notation.

Fidelity. A note is a summary. Our notes keep 160 of 185 facts (86.5%), and 32 of 51 keep all of them. The paper's ten notes keep 24 of 31 facts. A note is not a lossless copy of the memory.

In a 5,000-token budget: 127 OPVS notes against 119–120 prose memories (cl100k and o200k), and 113 against 114 on Gemma 3.

Board tasks (50)

As one board document (all 50 tasks together), OPVS uses 1,160 tokens against 2,349 for YAML (o200k), a saving of 51%.

Symbols

All 17 marks were counted alone and after a space. 16 are one token in both positions on all five tokenizers. /# (not this tag) is one token alone but two after a space ( / + #), and in a routing line it always follows a space. Four marks (::, /#, <-, !=) are two characters long.

What this changes in earlier claims

Limits

1. Two major tokenizers are missing. The Claude and Gemini token-count services refused our keys. Gemma 3 stands in for Gemini (Google describes its tokenizer as Gemini 2.0's; later Gemini models may differ). The five tokenizers we ran agree within 6 points everywhere, which suggests, but does not show, that the other two would too.

2. We wrote the test set. An AI session working for OPVS wrote the bodies, memories and tasks. Rendering by program stops one format being tuned against another, but the mix of bodies, lengths and fields is still ours. It is published in full so that others can replace it.

3. The body length mix sets the headline. More long messages shrink every saving; more short ones grow it. The body-length table gives each length separately.

4. Baseline choices move the numbers. We used OPVS's own field names and flat objects, which favour JSON, and an A2A shape that carries the same fields as OPVS. Shorter key names, or another A2A shape, give other numbers.

5. Tokens are not quality. We measured size only. We did not test whether a model understands, routes or answers an OPVS message as well as a JSON one.

6. Memory notes are an author's choice. How short a note is, and which facts it keeps, decides both its size and its fidelity.

7. Live use is not measured. These are test messages, not counts from OPVS mail in use.

8. What we knew beforehand. Before the method was written, the paper's six messages had already been measured against compact JSON (about 22–24%). The method file says so.

Data

Reproduce

Download the zip, then run one command. It needs Python 3.10 or later and internet access the first time (pinned packages and three tokenizer files, each checked against its SHA-256):

unzip opvs-field-tests-2026-11-08.zip && cd opvs-field-tests && bash reproduce.sh

It counts every item of the published test set on all five tokenizers, recomputes every table, and checks that the results match the published ones byte for byte. With ANTHROPIC_API_KEY or GEMINI_API_KEY set, it also counts the Claude or Gemini tokenizer.

Rebuilding the test set itself (build_corpus.py) uses the OPVS Python parser, which is not published as a package yet (see Parser libraries).

Related

• The paper

• Memory notes in OPVS form

• Parser libraries

• Versions and changes