fix: support large files in comparisons

Replace the 16 MiB comparison rejection with independent bounded per-side loading and frozen Working blobs. Bound diff computation, defer large inline highlighting, index source-row navigation, and expose actual load failures. Verified the real 52.3 MB parser through repository/session APIs and terminal rendering with EOF/start navigation.
2026-10-17 00:03:43 +02:00
parent adbd2c3f81
commit b9f29d258c
11 changed files with 986 additions and 199 deletions
@@ -101,7 +101,8 @@ Diff styling checks cover independent snapshot language/shebang/injection detect
 - PTY throughput alone cannot prove terminal presentation stays smooth: capture the editor and terminal while the user reproduces a pause. Synchronized-output commands must not reach the terminal during frame computation, even when individual frames and queued-input catch-up appear fast.
 - Annotation coordinate validation fast-paths ASCII grapheme lengths using byte lengths with CRLF pair accounting, retaining Unicode segmentation for non-ASCII text. Validation remains atomic and still checks every token; large Unicode buffers or diagnostic batches can require foreground work.
 - Legacy synchronous plugin invocation may pause input for up to 500 ms before a host timeout. Async plugins opt into background replies; pending requests and transport queues are bounded. Config/plugin startup has a 2-second deadline per host. Buffer context/document snapshots must fit within a 4 MiB JSON frame; a too-large frame disables that plugin, not editing.
- Repository diff currently requires Unix subprocess-group cancellation and Git-backed jj (or Git alone). Each file is limited to 16 MiB; each subprocess output stream and Working/conflict snapshot to 64 MiB; each command to 60 seconds. Limit, transport, and non-commit-tag errors are explicit rather than silently truncated. Temporary comparison stores are removed when the diff closes.
+- Repository diff requires Unix subprocess-group cancellation and Git-backed jj (or Git alone). Selected comparison sides support 64 MiB and 4,000,000 source lines each; frozen Working snapshots support 512 MiB total. Subprocess streams remain bounded to 64 MiB and commands to 60 seconds. Working text is frozen into private Git blobs; catalogs stay lazy and selected blobs load separately without concatenated batch copies. Limits and transport errors are explicit; temporary stores are removed on close. Picker/gutter annotation budgets remain separate.
+- Large line comparisons skip equal prefixes/suffixes before allocating diff tokens; added/removed files bypass the diff algorithm. Myers work has a 100 ms deadline, after which unresolved regions can be conservatively shown as replacements without dropping text. Inline highlighting is viewport-lazy above 20,000 combined source rows; large/pathological lines use whole-line change shading. Source/display navigation uses indexed contiguous ranges rather than scanning all preceding rows.
 - Normal-picker status uses the gutter worker's five-second command and 16 MiB stream limits. Catalogs are bounded to 100,000 paths; dirty-buffer batches to 1,024 files/64 MiB; individual overlays, conflict files, and rename candidates to 16 MiB; conflict/rename data to 64 MiB. Temporary baseline stores are retained for warm metadata polling and removed when the worker stops.

 See [syntax caches](architecture.md#syntax-caches) for the current full-tree highlighting ceiling.