fix: support large files in comparisons

Replace the 16 MiB comparison rejection with independent bounded per-side loading and frozen Working blobs. Bound diff computation, defer large inline highlighting, index source-row navigation, and expose actual load failures. Verified the real 52.3 MB parser through repository/session APIs and terminal rendering with EOF/start navigation.
2026-10-17 00:03:43 +02:00
parent adbd2c3f81
commit b9f29d258c
11 changed files with 986 additions and 199 deletions
@@ -101,7 +101,8 @@ Diff styling checks cover independent snapshot language/shebang/injection detect
 - PTY throughput alone cannot prove terminal presentation stays smooth: capture the editor and terminal while the user reproduces a pause. Synchronized-output commands must not reach the terminal during frame computation, even when individual frames and queued-input catch-up appear fast.
 - Annotation coordinate validation fast-paths ASCII grapheme lengths using byte lengths with CRLF pair accounting, retaining Unicode segmentation for non-ASCII text. Validation remains atomic and still checks every token; large Unicode buffers or diagnostic batches can require foreground work.
 - Legacy synchronous plugin invocation may pause input for up to 500 ms before a host timeout. Async plugins opt into background replies; pending requests and transport queues are bounded. Config/plugin startup has a 2-second deadline per host. Buffer context/document snapshots must fit within a 4 MiB JSON frame; a too-large frame disables that plugin, not editing.
- Repository diff currently requires Unix subprocess-group cancellation and Git-backed jj (or Git alone). Each file is limited to 16 MiB; each subprocess output stream and Working/conflict snapshot to 64 MiB; each command to 60 seconds. Limit, transport, and non-commit-tag errors are explicit rather than silently truncated. Temporary comparison stores are removed when the diff closes.
+- Repository diff requires Unix subprocess-group cancellation and Git-backed jj (or Git alone). Selected comparison sides support 64 MiB and 4,000,000 source lines each; frozen Working snapshots support 512 MiB total. Subprocess streams remain bounded to 64 MiB and commands to 60 seconds. Working text is frozen into private Git blobs; catalogs stay lazy and selected blobs load separately without concatenated batch copies. Limits and transport errors are explicit; temporary stores are removed on close. Picker/gutter annotation budgets remain separate.
+- Large line comparisons skip equal prefixes/suffixes before allocating diff tokens; added/removed files bypass the diff algorithm. Myers work has a 100 ms deadline, after which unresolved regions can be conservatively shown as replacements without dropping text. Inline highlighting is viewport-lazy above 20,000 combined source rows; large/pathological lines use whole-line change shading. Source/display navigation uses indexed contiguous ranges rather than scanning all preceding rows.
 - Normal-picker status uses the gutter worker's five-second command and 16 MiB stream limits. Catalogs are bounded to 100,000 paths; dirty-buffer batches to 1,024 files/64 MiB; individual overlays, conflict files, and rename candidates to 16 MiB; conflict/rename data to 64 MiB. Temporary baseline stores are retained for warm metadata polling and removed when the worker stops.

 See [syntax caches](architecture.md#syntax-caches) for the current full-tree highlighting ceiling.
@@ -517,6 +517,8 @@ Both diff panes apply available semantic tokens from their own buffer, independe

 Binary data, missing final newlines, permission/type changes, symlink targets, and submodule pointers have explicit notices. Symlinks are not followed and submodule contents are not traversed. jj conflicts use jj's file-content projection and carry conflict notices; ordinary files remain readable. Identical-content moves are shown as renames; modified moves appear as an added and removed file.

+Large comparisons load each selected side independently, up to 64 MiB and 4,000,000 source lines per side. Frozen Working snapshots have a 512 MiB aggregate budget. Exceeding a limit displays the actual error in the document pane. Large files use the normal plain-text syntax fallback; pathological comparisons may show broader replacement regions or whole-line highlights rather than block input searching for a minimal diff. Full source text remains available.
+
 ## Jujutsu conflict resolution

 Run **`:resolve`** in Normal mode for the current jj working copy. It opens a dedicated tab and selects the current conflicted file (otherwise the first conflict). Historical comparisons remain read-only; resolving never implicitly checks out or creates another change.