An AI tool was asked to find one Japanese email in an archive of 6,195 messages in English. What happened next upends a quiet assumption the patent-filing world has lived with for decades.

August 2026 · mxHERO AI-Chat Evaluation · Shinonome 0392 Corpus
Somewhere inside every global patent practice is a filing that depends on a document nobody can find. Not because it was destroyed. Not because it was classified. Because it was written in Japanese, or Korean, or Mandarin — and every search since has been conducted in English.
This is the story of a test designed to expose exactly that problem, and an AI email-retrieval tool that, against a carefully stacked deck, passed with a perfect score.
The Setup
The test corpus was built like a courtroom drama. A fictitious global technology licensor — call them Marsden Audio Laboratories — manages patent filings across five countries. In June 2011, the IP director at their Japanese development partner sent a letter confirming, in three explicit clauses, that a particular invention was Marsden’s alone. No co-ownership. No licensing rights. No expiration.
「別紙Aに定める対象範囲に含まれず、貴社の単独所有である」
“Not included in the scope defined in Annex A; solely owned by your company.”
That letter is the needle. And the haystack is 6,195 messages thick.
Between the letter and today, the corpus lays five years of perfectly ordinary catastrophe. A platform migration in 2016 derived patent-ownership fields from a cost-allocation code rather than from the actual agreements. The free-text note recording the real answer was dropped. The only ownership document that survived was an English courtesy translation, one that quietly lost the clause defining the scope of the joint development. Every person who negotiated the original deal has since left the company.
Then, in November 2024, the partner’s successor entity files a demand for ¥4,005,000,000. Twenty-six-point-seven million US dollars.
The Test
Hiding a needle in a haystack is easy. The hard part is filling the haystack with needles that look almost right. The evaluation team seeded 155 distractors, emails involving the same docket numbers, the same vocabulary, even the same people, but about different matters with different outcomes. One distractor involves a genuinely co-owned patent, properly filed jointly: citing it would invert the correct answer. Another involves a Korean partner whose nearly identical confirmation expired after three years, and whose dispute Marsden lost, paying ₩940 million.
Think of it as putting the real Mona Lisa in a room with 155 excellent forgeries, then asking someone to find her while blindfolded, and speaking a different language.
Four queries were fired at the system, all in English, against an archive where the decisive documents are in Japanese:

✓ 24 / 24 — PERFECT SCORE
Every query scored full marks on recall, precision, and verdict. Zero trap citations across all four.
What Happened Inside
The first query, the heart of the test, asked the tool to prove the value of the email archive on this matter. No docket number. No Japanese keywords. Just a plain English question.
The tool returned the June 2011 letter complete, with the attached confirmation document extracted cleanly, all three operative clauses intact. It also found the precedent: an October 2013 letter in which the same signatory turned away a different company that tried to claim co-ownership of a different patent. That precedent converts “we found an old document” into “they already acknowledged this, twice.”
A traditional keyword search, using the English query against a BM25 index, would have ranked the Japanese needle at position 527. The AI tool returned it first.
— Gate 4 retrieval-rank analysis, Shinonome 0392 build notes
The second query fed the system a false premise — “So we do co-own it, don’t we?” — and the tool contradicted it immediately, citing the specific mechanism: a controlled field populated by derivation from a code that was never about ownership, and a governing text in Japanese that headquarters read through a faulty translation.
The third query was the cruelest. “Show me every email on MAL-2010–0392.” Two different patent families in Marsden’s files share that identifier, one in Yokohama, one in San Mateo. Without being prompted, the tool opened its response with a collision warning: “An identifier is unique only within the office that issued it… confirm the party before treating these as one file.” Both families came back, cleanly separated, with every disambiguator the grading key required.

The fourth asked who was responsible. The system traced both halves of the failure, the derived field and the lost-in-translation clause, and laid out the counterfactual: if anyone had searched the journal archive in Japanese when the dispute first arose, it would have taken nineteen minutes.
What This Means for Patent Filing
Patent professionals might read this as a nice technology demo. It is not. It is an early signal of a structural shift.
Global patent portfolios live in multiple languages by nature. A PCT filing originating in Yokohama generates Japanese prosecution history, English translations, German opposition briefs, Chinese validation correspondence, Korean co-development agreements. For decades, retrieval systems have treated this as a search problem bounded by the language of the query. You searched in English; you got English results. The Japanese archive sat in the next room, but no one opened the door.
The cost of that closed door has always been real, but it was invisible, because you cannot tally the disputes that would have been prevented by a document nobody knew to look for. This test puts a number on one such dispute: $26.7 million demanded, $487,000 spent to resolve it, and the outcome turned on a single cross-language retrieval.
If email-archive AI can reliably cross the language barrier, finding a Japanese confirmation from an English question, distinguishing a Korean near-miss from the real thing, and refusing to cite a tempting but wrong precedent, then a category of risk that patent operations has quietly accepted for a generation is suddenly manageable. Prior-art searches become deeper. Ownership chains become verifiable against the original documents, not their translations. Licensing disputes become resolvable before they reach a demand letter.
The test was synthetic. The risk is not. Somewhere in every multinational filer’s archive is a letter like the one Kaida wrote in June 2011, decisive, unambiguous, and invisible to every query ever run against it, because the query was in the wrong language.
The question is no longer whether AI can find it. The question is how long you wait before asking.
References
- GitHub Repository of assets
- The $340,000 Email — An Insurance Industry AI Use-Case
- The $2.15 Million Email — AI Discovery for Architecture Firms
- Why mxHERO Mail2Cloud Advanced with MCP Matters in the Era of AI
Test methodology: four adversarial queries, English-only, against a 6,195-message corpus with 40 signal emails, 17 attachments, 155 hard negatives, and ~6,000 filler messages spanning Japanese, Korean, German, and Chinese correspondence. Tool under test: mxHERO mxMCP email_search connector. The full email corpus, scorecard, and grading key are available on GitHub (https://github.com/mxaiorg/sample-emails).