Limited local Japanese typo suggestions — provider decision

Decision date: 2026-10-07. Tracks #7378.

Decision and scope

Implement a dependency-free, renderer-local literal matcher with exactly the three rules below. Keep it off by default, separately opt-in from native spelling, and describe it as limited Japanese typo suggestions (日本語の表記候補（限定）). Offer an explicit replacement candidate; never rewrite automatically or offer fix-all. Existing macOS word spelling remains independent.

This is a concrete implementation decision, not approval to ship or merge the combined feature. Browser worker/CSP, native interaction and resource gates below remain open. The broader request for Japanese grammar checking is not achieved: this provider cannot judge particles, syntax, semantics, register or sentence quality. No warning means only that these three literal forms were not found in eligible text.

Three rules are a useful first slice because they cover a memory-related expression in everyday correspondence and two terms used in technical/business prose, with an inspectable one-candidate substitution. They are deliberately low recall. Adding grammatical guesses to increase the count would repeat the precision problems observed in the textlint spike. Additions require a new reviewed positive/negative corpus and an explicit decision; do not silently import a preset or upstream dictionary.

Alternatives and observed evidence

The initial synthetic spike used @textlint/kernel@15.8.0, textlint-rule-ja-no-successive-word@2.0.1, textlint-rule-ja-no-abusage@3.0.0 and kuromoji@0.1.2 on Linux/Node.

The initial Node run measured 461.13 ms for its first check and 1.283 ms median over 20 warm checks. RSS grew from 49,987,584 to 428,122,112 bytes (360.62 MiB). That is an observed process RSS change, not a measured peak or retained worker memory after GC. It establishes neither browser failure nor browser success.

Upstream descriptions: successive-word rule, abusage rule and its two dictionary strategies, official worker compiler protocol. These sources describe the alternatives; the selected implementation does not copy their implementation or load their dictionaries.

Exact v1 rules

Provider ID: ja-local-typos; rule-table version: 1; category: usage. All findings are advisory suggestions. The UI message is Consider “{replacement}” for this limited Japanese spelling suggestion. Keep the original if intentional. There is one optional user-selectable edit per finding, never a mandatory correction.

Match exact code units, case-sensitively. Do not normalize Unicode, collapse spaces, join lines, strip Markdown, expand spelling variants or infer word boundaries from a tokenizer. Highlight and edit both cover only the complete literal for this provider, although the shared diagnostic contract must keep them independent.

Apply these additional boundaries, exactly as the executable probe below:

1. Reject if the immediately adjacent code point on either side is an ASCII letter, digit, _, Unicode Script=Katakana, Unicode mark (\p{M}), 々 or ー. This avoids embedded identifiers, extended katakana names and combining sequences. It intentionally misses some compounds.

2. After the literal, require paragraph end, whitespace, one of 。、！？!?.,:;：；（）()[]【】, or a prefix from です, でした, だ, で, の, を, は, が, に, と, も, する, し. This is a literal suffix allowlist, not morphological analysis.

3. Enumerate rules in table order, nonoverlapping matches of each rule from left to right; stop after 100 results per job, then sort by source offset. Coverage beyond that cap is intentionally partial; never label the document error-free.

Unquoted invented names and intentional nonstandard spelling can still collide. For example, a deliberately named うる覚えの会 can trigger; this is a known false-positive possibility, not a grammar verdict. Quotation exclusion, dismissal, opt-out and explicit acceptance limit disruption; they do not establish universal precision. Native Add to Dictionary must not claim to suppress these local rules. Do not add casual こんにちわ, repetitions, ら抜き言葉, 全然大丈夫, disputed idioms, or contextual substitutions such as 適応 → 適用 to this table.

Paragraph eligibility and masks

The implementation owner must build a Japanese-specific eligibility wrapper around the existing CodeMirror Markdown syntax tree. Existing tauri-app/renderer/extensions/spellcheck.ts supplies useful masked-node names, but collectCheckableSegments splits text around masks and clips visible ranges. Its output alone does not satisfy the complete-paragraph contract here.

• Consider full prose paragraphs intersecting visible ranges, expanded to their true boundaries, preserving soft line breaks. Do not check a clipped viewport fragment. A selected job has at most 20,000 UTF-16 units of original paragraph text; skip any single larger paragraph, rather than truncate it.

• Exclude an entire paragraph if any part intersects inline/fenced/indented code, a URL/autolink, link or image (including label, destination and title), HTML, comments, or a Markdown blockquote, including lazy continuation lines. Exclude frontmatter, headings, tables and directives in v1; ordinary list-item prose may be checked if the syntax tree identifies a complete paragraph. Keep the stricter Japanese masks separate from native spelling behavior.

• Exclude the entire paragraph if it contains any of 「」『』“”‘’"', balanced or not. This intentionally misses prose surrounding a quotation and English apostrophes. Do not attempt quote attribution or repair an unmatched delimiter.

• An opening frontmatter --- on the first line excludes through its closing --- or ...; if none exists, exclude the remainder of the document. An unclosed code fence is excluded to document end. Skip incomplete/unparsed or ambiguous syntax, raw URL-like tokens (http:, https:, www., mailto:), and unresolved code/link delimiters in a candidate paragraph. Recheck when parsing completes; no hand-written regex replacement of a Markdown parser.

• Preserve original offsets by passing complete original paragraph substrings with their original base offsets. Never concatenate around excluded material. The whole-paragraph exclusion is intentionally more conservative than necessary for three strings, avoiding lost context and making exclusions explainable.

The 10 eligible=false fixtures in the probe are a boundary contract, not proof that the actual Markdown extractor recognizes those constructs. Implementation tests must parse those documents and assert no eligible paragraph/finding, plus test lazy blockquotes, nested quotes, incomplete parsing, directives, headings, tables, links with titles, and viewport edges against the actual editor extension.

Coordinates, state and explicit edits

The authoritative source is a frozen CodeMirror document snapshot, with offsets in zero-based UTF-16 code units and half-open ranges. The worker receives that snapshot's paragraph strings without conversion. Use String.indexOf and literal.length for offsets; add the paragraph's snapshot base. Never interpret UTF-8 bytes, Unicode scalar counts, grapheme counts or line/column positions as editor offsets. Array.from in the probe is only for inspecting a neighbor, never for position calculation.

For 😀うる覚えです。, highlight/edit must be [2,6), not [1,5). Combining か\u3099 and variation selectors remain unchanged; reject matches with an adjacent combining mark. Reject any boundary splitting a surrogate pair. CRLF must remain two code units if present in the source passed to the pure checker. At the editor boundary, use the exact serialized representation whose offsets agree with state.doc (normally LF); do not convert a file string's CRLF offsets to editor offsets by guesswork. If source length/representation cannot be proven, reject the result. Include both raw-CRLF pure tests and editor-LF tests.

Each result carries provider, rule ID, message/category, request ID, view and document identity, revision/source identity, highlight, and optional edit {from, to, expectedText, insert}. The shared safety owner defines the actual DTO. Before displaying and again before applying, validate identities, exact snapshot, integer bounds, code-point-safe boundaries and exact original edit text. These guards supplement a proven offset convention; matching the same text at a wrong offset is not evidence of correctness. Never search nearby to salvage stale work.

Invalidate on any local/remote edit, document switch, readonly transition, composition start, unmount or disable. Do not decorate/update candidates during IME composition. Apply only the selected candidate in one isolated editor history transaction (input.spellcheck), with one-step undo and focus restoration through the shared inline-correction contract. Repeated identical literals are separate positions; replacing one must never replace another occurrence or all occurrences.

Worker and privacy contract

Implement the pure rule table/matcher and a bundled ESM worker in the renderer; no Rust/native IPC extension, new dependency or dictionary asset is needed. Use the existing Vite worker packaging and current worker-src 'self' policy; do not weaken CSP to accommodate a library, use blob workers, fetch code from a CDN, or make external text requests. There is no cloud-model fallback.

Lazy-start only after explicit opt-in. Use 400 ms debounce, one worker and one inflight job per renderer realm, bounded round-robin scheduling across active split editors, and replace each editor's queued request with its latest revision. Jobs carry request IDs. Cancel/remove pending work on opt-out and unmount; drop stale responses. Terminate/restart a failed worker at most once per enable cycle, then surface unavailable state until a deliberate re-enable. No retry loop on keystrokes. Release snapshots and results after invalidation; no persistent text caches, document-text logging, telemetry payloads or network calls from this provider.

Synthetic evidence and resource budgets

The dependency-free probe below was run on Linux x64, Node v24.19.0, AMD EPYC 7763 64-Core Processor, on 2026-10-07. It checks 11 positive sentences (14 expected suggestions), 24 negative sentences and 10 prefiltered exclusions. All assertions passed. Expected literal slices and emoji [2,6) were checked. Agent review accepted the 14 substitutions in these synthetic sentences; zero findings occurred in the 24 negative cases. This is a small selected corpus, not an estimate of precision or recall on user documents.

Examples deliberately left untouched include あるある, そうそう, ドキドキ, すももももももものうち, polite hedges, English, technical names, correct spellings, こんにちわ, 食べれる, 全然大丈夫 and ambiguous idioms. Repeated particles are also untouched: v1 does not implement that grammar objective. The probe contains the complete corpus so a later worker can preserve these expectations.

Each size uses 10 fresh Node processes and 200 warm calls per process. Input creation plus first matcher call is timed after module loading; process startup, worker initialization, Markdown parsing, serialization and UI work are excluded. Warm p95 is nearest-rank sample 190 of 200; the table shows the worst p95 of the 10 processes. Heap/RSS deltas are after explicit GC, not peak or worker memory.

The serialized rule table plus matcher function is 1,413 UTF-8 bytes / 775 gzip bytes. This is a kernel-only feasibility measurement, not a production bundle size. The repeated-positive performance input hits the 100-result cap at 5k/20k; implementation measurements must also include clean text, rejected-neighbor matches, mask-heavy text and many small paragraphs.

Provisional production renderer evidence

On 2026-10-07, the integration owner verified the Vite production renderer and tested the emitted worker in system Chromium 151.0.7922.173 on Linux x64, AMD EPYC 7763 64-Core Processor. The worker asset was 1,253 bytes raw and 778 bytes gzip. A synthetic production-asset probe under worker-src 'self' recorded no CSP violations and only same-origin local-server requests. It ran 10 fresh workers and 100 warm samples at 1,000, 5,000 and 20,000 UTF-16 units. These round-trip timings cover worker startup for cold samples, message transfer and literal matching; they exclude editor parsing, the 400 ms debounce and full-app startup. At 5,000 units, the cold median was 9.55 ms and maximum 20.425 ms; the warm p95 was 0.45 ms.

A separate system-Chromium app interaction pass used a deterministic mock native spell service. It covered exact Japanese offsets after emoji, Markdown and quote exclusions, candidate click with isolated undo/redo, native left-click and close-without-mutation behavior, Alt+Enter navigation, and opt-out clearing the popup and marks. It recorded no page errors or external requests. The repository's pinned Chromium 145 download returned HTTP 403, so Chromium 151 is supplemental evidence rather than the pinned CI browser result.

The final production reports were also compared with parent commit 255fe5648: 1,722 versus 1,723 emitted asset rows, with reported gzip totals of 7,196.16 kB and 7,206.94 kB. Including the separately measured 778-byte worker, the reported increase is 11.558 decimal kB. Allowing a conservative 0.01 kB rounding error for every row in both reports bounds the increase at 46.008 kB, below the 5 MiB ceiling. The other non-gzip-reported worker/glue assets are unchanged; no dependency manifests or dictionary assets were added. This is a bound from production output reports, not an exact byte-by-byte artifact comparison.

These checks do not measure retained per-worker memory or verify actual offline operation or native macOS WKWebView behavior. The renderer interaction pass also does not replace the real macOS IME/Vim/NSSpellChecker acceptance run.

Keep these acceptance budgets unchanged for the actual implementation:

Run the matrix in a named supported browser and real macOS WKWebView. Where retained-worker memory is not directly measurable, report the proxy and unresolved gate to the integration owner; never call an unavailable measurement a pass. If a budget fails, profile/reduce overhead within this scope or return the decision to the parent. Do not relax thresholds or replace local checks with external AI.

Handoff and remaining gates

The renderer implementation now provides the three local rules, default-off setting, shared diagnostics and explicit candidate popup. Its React and CodeMirror integration is covered by a focused hook-order regression test. The system-Chromium production worker and popup evidence above is provisional; pinned Chromium 145 and real macOS WKWebView behavior remain unverified.

Remaining gates include retained worker memory, complete feature timing, a true offline run, and native macOS IME/Vim/NSSpellChecker acceptance, tracked in #7382. The evidence above does not claim grammar checking, mixed-language accuracy, or successful native verification. Keep the acceptance budgets in this decision unchanged.

Reproduce the lightweight probe

Save the following as literal-probe.cjs in a scratch directory and run node --expose-gc literal-probe.cjs. It uses only Node builtins, synthetic data and no network. The original run used /workspace/scratch/japanese-proofreading-spike/literal-probe.cjs; raw samples are in that directory's literal-results.json. The original textlint input and output remain in probe.cjs, results.json and evidence.txt there. These session paths are supporting evidence, not product runtime paths. The embedded probe preserves reproducibility after that disposable directory is removed.

// Synthetic feasibility probe only. No Markdown parser, Worker, or UI integration.
const {performance} = require('node:perf_hooks');
const fs = require('node:fs');
const os = require('node:os');
const {spawnSync} = require('node:child_process');
const {gzipSync} = require('node:zlib');
const assert = require('node:assert/strict');
const rules = [
  ['ja-typo-urooboe', 'うる覚え', 'うろ覚え'],
  ['ja-typo-simulation', 'シュミレーション', 'シミュレーション'],
  ['ja-typo-communication', 'コミニュケーション', 'コミュニケーション'],
];
function check(text, eligible = true) {
  if (!eligible || text.length > 20000 || /[「」『』“”‘’"']/.test(text)) return [];
  const output = [];
  const neighbor = /[A-Za-z0-9_\p{Script=Katakana}\p{M}々ー]/u;
  const tail = /^(?:$|[\s。、！？!?.,:;：；（）()\[\]【】]|です|でした|だ|で|の|を|は|が|に|と|も|する|し)/u;
  for (const [ruleId, expectedText, insert] of rules) {
    for (let from = text.indexOf(expectedText); from !== -1; from = text.indexOf(expectedText, from + expectedText.length)) {
      const to = from + expectedText.length;
      // Array.from is used ONLY to obtain the prior code point, never to compute offsets.
      const before = Array.from(text.slice(Math.max(0, from - 2), from)).at(-1) || '';
      const after = String.fromCodePoint(text.codePointAt(to) || 0);
      if (neighbor.test(before) || neighbor.test(after) || !tail.test(text.slice(to))) continue;
      output.push({ruleId, from, to, expectedText, insert});
      if (output.length === 100) return output.sort((a,b) => a.from-b.from || a.ruleId.localeCompare(b.ruleId));
    }
  }
  return output.sort((a,b) => a.from-b.from || a.ruleId.localeCompare(b.ruleId));
}
const positive = [
  ['うる覚えです。', 1], ['手順はうる覚えでした。', 1], ['😀うる覚えです。', 1],
  ['か\u3099。うる覚えです。', 1], ['確認します。\r\nうる覚えです。', 1],
  ['シュミレーションを実行します。', 1], ['明日はシュミレーションです。', 1],
  ['コミニュケーションが大切です。', 1], ['チームのコミニュケーションを改善する。', 1],
  ['うる覚えです。うる覚えでした。', 2],
  ['うる覚えです。シュミレーションを実行し、コミニュケーションを改善する。', 3],
];
const negative = [
  'うろ覚えです。シミュレーションを実行し、コミュニケーションを改善する。',
  'あるあるですね。そうそう、それです。', '今日はドキドキした。すももももももものうち。',
  'ご確認いただければと思います。明日は雨かもしれません。よろしくお願いいたします。',
  '食べれる？ 全然大丈夫。こんにちわ。', '役不足と力不足。確信犯。煮詰まる。',
  'これはは問題ある文章です。', 'これは\nは問題ある文章です。',
  'GitHubとTypeScriptのAPIを確認します。', 'This is a valid English sentence.',
  '鈴木さんと株式会社シミュレーション技研。', 'うる\n覚えです。',
  '「うる覚え」は表記の例です。', '『コミニュケーション』という作品です。',
  '“シュミレーション” is the example.', '「うる覚えです。', 'うる覚えです。」',
  'ABCシュミレーションを呼ぶ。', 'シュミレーションズの新作。',
  'コミニュケーションAPI', 'xうる覚えです。', 'うる覚え\u3099です。',
  'か\u3099うる覚えです。', 'コミニュケーション\u{E0100}です。',
];
const masked = [
  '`うる覚えです。`', '```\nシュミレーションです。\n```',
  '    コミニュケーションです。', 'https://example.test/うる覚えです。',
  '[うる覚えです。](https://example.test)', '> うる覚えです。',
  '---\ntitle: うる覚えです。\n---', '<div>うる覚えです。</div>',
  '<!-- コミニュケーションです。 -->', '---\ntitle: うる覚えです。',
];
const makeText = n => {
  const unit = '確認します。😀か\u3099。うる覚えです。\r\n';
  return (unit.repeat(Math.floor(n / unit.length)) + 'あ'.repeat(n % unit.length));
};
if (process.argv[2] === 'sample') {
  const n = Number(process.argv[3]);
  global.gc(); const before = process.memoryUsage();
  const start = performance.now(); const source = makeText(n); check(source); const first = performance.now()-start;
  const warm = [];
  for(let i=0;i<200;i++) { const t = performance.now(); check(source); warm.push(performance.now()-t); }
  global.gc(); const after = process.memoryUsage();
  warm.sort((a,b)=>a-b);
  console.log(JSON.stringify({n,first,warmP95:warm[189],warmMedian:warm[99],heapDelta:after.heapUsed-before.heapUsed,rssDelta:after.rss-before.rss}));
} else {
  for (const [text,count] of positive) {
    const results=check(text); assert.equal(results.length,count,text);
    for(const result of results) assert.equal(text.slice(result.from,result.to),result.expectedText);
  }
  for (const text of negative) assert.deepEqual(check(text),[],text);
  // Only prefiltered eligibility is exercised, not Markdown extraction.
  for (const text of masked) assert.deepEqual(check(text,false),[],text);
  assert.deepEqual(check('あ'.repeat(20001)),[]);
  assert.deepEqual(check('😀うる覚えです。').map(x=>[x.from,x.to]),[[2,6]]);
  const measurements=[];
  for(const n of [1000,5000,20000]) for(let run=0;run<10;run++) {
    const child=spawnSync(process.execPath,['--expose-gc',__filename,'sample',String(n)],{encoding:'utf8'});
    assert.equal(child.status,0,child.stderr); measurements.push({...JSON.parse(child.stdout),run});
  }
  const kernel=JSON.stringify(rules)+'\n'+check.toString();
  const result={runtime:process.version,platform:os.platform(),arch:os.arch(),cpu:os.cpus()[0].model,
    positiveCases:positive.length,positiveDiagnostics:positive.reduce((sum,x)=>sum+x[1],0),negativeCases:negative.length,
    prefilteredCases:masked.length,sourceBytes:Buffer.byteLength(kernel),gzipBytes:gzipSync(kernel).length,measurements};
  fs.writeFileSync('literal-results.json',JSON.stringify(result,null,2));
  console.log(JSON.stringify({...result,measurements:undefined}));
  for(const n of [1000,5000,20000]) {
    const samples=measurements.filter(s=>s.n===n);
    console.log(JSON.stringify({n,firstMin:Math.min(...samples.map(x=>x.first)),firstMax:Math.max(...samples.map(x=>x.first)),warmP95Max:Math.max(...samples.map(x=>x.warmP95)),heapDeltaMax:Math.max(...samples.map(x=>x.heapDelta)),rssDeltaMax:Math.max(...samples.map(x=>x.rssDelta))}));
  }
}