fix: keep large-file editing responsive

Bound synchronous full-source syntax parsing, cache dirty comparisons, edit ordinary inline changes in place, retain changed-row undo history, and buffer atomic saves. Skip repository snapshots beyond existing worker limits before copying their text.
This commit is contained in:
2026-10-16 21:14:16 +02:00
parent 02d1422a04
commit adbd2c3f81
13 changed files with 639 additions and 55 deletions
+2
View File
@@ -115,6 +115,8 @@ The same cancellable worker collects normal-picker status for the startup projec
Each buffer has a frontend-owned parser/tree cache. Unchanged revision/path skips comparison and parsing; unchanged text at a later revision also avoids parsing. Actual text changes produce a minimal UTF-8-safe byte edit with newline-aware Tree-sitter Points, edit the prior tree, then parse using that tree for incremental reuse. Grammar changes reset the tree, including TypeScript versus TSX despite their shared display name. Native highlight/local/injection queries resolve capture precedence and scoped definitions into nonoverlapping original-line spans; unchanged injected trees are reused. Buffer switching restores its cache, and closing retires it. Incomplete text and grammar ERROR nodes remain highlightable; genuine language/query/parser failures expose a status error and readable plain text. Full-tree highlight queries still run after incremental parsing—a deliberate ceiling to revisit if measured costly. There is no local semantic inference engine or runtime grammar download.
Full-source syntax work is bounded to 1 MiB, with a maximum 64 KiB source line (UTF-8 bytes, including normalized line separators in the total). Oversized buffers retain language detection and an explicit plain-text status, but allocate no source copy, per-row spans, scopes, or parse tree. Crossing either budget releases existing parser/cache allocations; unchanged oversized revisions return immediately, and shrinking below both budgets resumes normal parsing. The same policy covers ordinary buffers, diff sources including their separately stored final newline, and text-only hover caches.
Diff highlighting lazily owns two instances of the same syntax cache, keyed by the accepted comparison request ID and selected file. Before detects language from the old path when renamed; After uses the new path. Scrolling, focus changes, and explorer toggling reuse parsed spans without copying/reparsing snapshots. A fresh comparison identity prevents stale Working-checkout colors even when labels and paths are unchanged. Closing the view releases snapshot caches. Syntax styles patch foreground/modifiers over diff backgrounds; historical panes never receive live-buffer semantic tokens.
Scope context is a lazily built row index over the existing primary/injected syntax trees, invalidated by source or grammar changes and warmed by diff updates. Ordinary buffer highlighting does not build it. The frontend's visual-row projection inserts inline/sticky scope labels in wrapped and unwrapped diffs; source alignment and excerpt ranges contain no label rows. Coloring, native selection/caret overlays, and source navigation skip these presentation rows. Before/After labels use their independent trees rather than live-workspace symbol analysis.
+4 -1
View File
@@ -92,7 +92,10 @@ Diff styling checks cover independent snapshot language/shebang/injection detect
## Performance and process limits
- Undo uses full-buffer snapshots and movement scans the current line. This favors straightforward correctness over large-file performance.
- Retained undo/redo history stores only changed rows. Insert/replay/completion transactions still use temporary full-buffer snapshots, and undo/redo reconstructs the buffer. Ordinary single-line edits with one pane and no semantic overlay modify the line in place; multiline, multi-edit, shared-pane, and semantic-overlay batches retain the general whole-buffer path. Dirty-state comparisons are cached until text or the saved baseline changes. Movement and wrapping still scan the current line.
- Syntax falls back to plain text above 1 MiB of source or a 64 KiB line, before source copies, tree construction, or per-row syntax allocations. The detected language and fallback reason remain visible; reducing the buffer below both budgets restores highlighting. These are optional syntax limits, not file-opening or editing limits.
- Saves use buffered writes before flushing, syncing, and atomically replacing the destination. External-change checks remain independent of dirty-state caching. Large-file verification must include exact saved bytes and edit/undo/redo round-trips, not startup alone.
- Blame and gutter snapshots are size-checked before joining buffer lines; files above their existing 16 MiB limits retain editing but have no annotations. Eligibility is cached with the source revision and recovers after shrinking.
- Rapid-wheel verification must replay bursts against the running editor and measure arrival at the final viewport after input stops, not just individual frame duration. Cheap frames can still accumulate a visible backlog if each queued report requires its own redraw.
- Keyboard viewport scrolling coalesces up to 64 consecutive inputs for the same resolved scroll action into one redraw. Reversals, changes between row/configured-step scrolling, and cursor motions end the batch so boundary clamping and caret reveal happen before the next action. Burst checks must include direction changes at the start/end of the buffer, not just throughput.
- PTY throughput alone cannot prove terminal presentation stays smooth: capture the editor and terminal while the user reproduces a pause. Synchronized-output commands must not reach the terminal during frame computation, even when individual frames and queued-input catch-up appear fast.
+2
View File
@@ -590,6 +590,8 @@ config.bind "leader f", config.core.follow_link
Buffers highlight automatically using [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) and compiled-in grammar/query packages. The selected language set is Rust, Ruby, Python, JSON, Bash, Makefile, Markdown, JavaScript/JSX, TypeScript/TSX, HTML, CSS, YAML, TOML, C, C++, Go, Zig (`.zig`), and Honey (`.hon`). Detection uses filename/extension or the in-memory first-line shebang, including common special filenames such as Makefile, Gemfile, and Cargo.lock; no file reread occurs. Unsupported formats use the normal foreground. The detected language appears in status, and save-as redetects it.
Files remain editable above the syntax budget: sources over 1 MiB, or with any line over 64 KiB, use plain text and show the reason in the language status. Syntax highlighting resumes automatically after shrinking below both limits. This avoids building a full parse tree for huge generated files; navigation, search, selections, undo/redo, and saving remain available.
Colors retain the terminal palette: comments dark gray, strings green, keywords magenta, numbers yellow, types/markup cyan, and functions light blue. Grammar queries handle multiline/nested constructs, even when their opening is above the viewport. Markdown uses its inline grammar and named fenced-code injections; HTML script/style content uses JavaScript/CSS. Known tagged-template and raw-string injections use the same selected grammar registry; unknown injected languages retain host highlighting or plain text. Highlighting styles whole graphemes and expanded tab cells; partially clipped wide placeholders remain unchanged. Search, Visual selections, and secondary carets retain foreground/background precedence.
Honey uses its bundled grammar and highlight queries, including declarations, primitive types, literals, and comments. Markdown fences accept `honey`, `honeylang`, or `hon`; the language ID is `honey`.
+1 -1
View File
@@ -13,7 +13,7 @@ use std::{
time::{Duration, Instant},
};
const LIMIT: usize = 16 * 1024 * 1024;
pub(crate) const LIMIT: usize = 16 * 1024 * 1024;
const TIMEOUT: Duration = Duration::from_secs(5);
static NEXT_TEMP: AtomicU64 = AtomicU64::new(0);
type ResultFile = Result<Option<Vec<Option<Arc<LineBlame>>>>, String>;
+161 -17
View File
@@ -295,6 +295,40 @@ struct Snapshot {
primary: usize,
}
/// Retain only changed rows between undo groups, not another copy of the file.
#[cfg_attr(test, derive(Debug))]
struct HistoryEntry {
start: usize,
end: usize,
before: Snapshot,
}
impl HistoryEntry {
fn new(mut before: Snapshot, after: &[String]) -> Self {
let start = before
.lines
.iter()
.zip(after)
.take_while(|(a, b)| a == b)
.count();
let suffix = before.lines[start..]
.iter()
.rev()
.zip(after[start..].iter().rev())
.take_while(|(a, b)| a == b)
.count();
before.lines = before
.lines
.drain(start..before.lines.len() - suffix)
.collect();
Self {
start,
end: after.len() - suffix,
before,
}
}
}
struct Edit<'a> {
target: Option<usize>,
start: usize,
@@ -1868,6 +1902,8 @@ struct Buffer {
snapshot_root: Option<PathBuf>,
lines: Vec<String>,
saved_lines: Vec<String>,
// Invalidate whenever text or the saved baseline changes, including temporary swaps.
dirty_cache: Cell<Option<bool>>,
disk_stamp: Option<FileStamp>,
reload_error: Option<String>,
text_revision: u64,
@@ -1878,9 +1914,8 @@ struct Buffer {
view: ViewState,
path: Option<PathBuf>,
identity: Option<PathBuf>,
// ponytail: full-buffer history costs O(buffer size) per action; use edit deltas for large files.
undo: Vec<Snapshot>,
redo: Vec<Snapshot>,
undo: Vec<HistoryEntry>,
redo: Vec<HistoryEntry>,
}
impl Buffer {
@@ -1926,6 +1961,7 @@ impl Buffer {
snapshot_document_path: None,
snapshot_root: None,
saved_lines: lines.clone(),
dirty_cache: Cell::new(Some(false)),
disk_stamp,
reload_error: None,
lines,
@@ -1946,7 +1982,15 @@ impl Buffer {
}
fn dirty(&self) -> bool {
!self.resolver_readonly && self.diff_snapshot.is_none() && self.lines != self.saved_lines
if self.resolver_readonly || self.diff_snapshot.is_some() {
return false;
}
if let Some(dirty) = self.dirty_cache.get() {
return dirty;
}
let dirty = self.lines != self.saved_lines;
self.dirty_cache.set(Some(dirty));
dirty
}
fn refresh_file(&mut self) -> io::Result<bool> {
@@ -1994,6 +2038,7 @@ impl Buffer {
}
self.replace_lines(split_lines(&text));
self.saved_lines.clone_from(&self.lines);
self.dirty_cache.set(Some(false));
self.undo.clear();
self.redo.clear();
self.view.reload(&self.lines);
@@ -2029,12 +2074,17 @@ impl Buffer {
if let Some(tokens) = &mut self.semantic_tokens {
retain_semantic_tokens(tokens, &self.lines, &lines);
}
self.text_revision = self.text_revision.wrapping_add(1);
self.diagnostics.clear();
self.view.search_match = None;
self.text_changed();
}
std::mem::replace(&mut self.lines, lines)
}
fn text_changed(&mut self) {
self.text_revision = self.text_revision.wrapping_add(1);
self.dirty_cache.set(None);
self.diagnostics.clear();
self.view.search_match = None;
}
}
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
@@ -2601,7 +2651,8 @@ impl Editor {
let seed = split_lines(projection.seed());
if buffer.lines != seed {
let previous = buffer.replace_lines(seed);
buffer.undo.push(Snapshot {
buffer.undo.push(HistoryEntry::new(
Snapshot {
lines: previous,
targets: if buffer.view.targets.is_empty() {
vec![Target::caret(Cursor::default())]
@@ -2609,7 +2660,9 @@ impl Editor {
buffer.view.targets.clone()
},
primary: buffer.view.primary,
});
},
&buffer.lines,
));
buffer.redo.clear();
buffer.view.clamp(&buffer.lines);
for pane in self.panes.values_mut() {
@@ -2675,12 +2728,14 @@ impl Editor {
buffer.view.clamp(&buffer.lines);
}
buffer.saved_lines = saved;
buffer.dirty_cache.set(None);
buffer.identity = Some(target_path.clone());
buffer.path = Some(target_path);
buffer.resolver_readonly = false;
} else if !buffer.dirty() {
buffer.replace_lines(vec!["Non-text resolution saved; reopen the file to inspect its current type.".into()]);
buffer.saved_lines.clone_from(&buffer.lines);
buffer.dirty_cache.set(Some(false));
buffer.path = None;
buffer.identity = None;
buffer.resolver_readonly = true;
@@ -2983,6 +3038,7 @@ impl Editor {
if buffer.lines != alternative {
buffer.replace_lines(alternative.clone());
buffer.saved_lines = alternative;
buffer.dirty_cache.set(Some(false));
buffer.view.clamp(&buffer.lines);
for pane in self.panes.values_mut() {
if let Some(view) = pane.views.get_mut(&alternative_buffer) {
@@ -5024,6 +5080,7 @@ impl Editor {
buffer.snapshot_document_path = Some(document_path);
buffer.snapshot_root = Some(root);
buffer.lines = lines;
buffer.dirty_cache.set(None);
buffer
}
@@ -10028,6 +10085,7 @@ impl Editor {
match atomic_save(&path, &self.active.lines, expected) {
Ok(stamp) => {
self.active.saved_lines.clone_from(&self.active.lines);
self.active.dirty_cache.set(Some(false));
self.active.save_revision = self.active.save_revision.wrapping_add(1);
self.active.disk_stamp = Some(stamp);
self.active.reload_error = None;
@@ -10063,7 +10121,9 @@ impl Editor {
fn record(&mut self, before: Snapshot) {
if before.lines != self.active.lines {
self.active.view.search_match = None;
self.active.undo.push(before);
self.active
.undo
.push(HistoryEntry::new(before, &self.active.lines));
self.active.redo.clear();
}
}
@@ -11369,6 +11429,7 @@ impl Editor {
let linewise = self.active.view.linewise;
let search_match = self.active.view.search_match;
let revision = self.active.text_revision;
let dirty_cache = self.active.dirty_cache.get();
let diagnostics = std::mem::take(&mut self.active.diagnostics);
let semantic_tokens = self.active.semantic_tokens.clone();
let mut restored = false;
@@ -11380,6 +11441,7 @@ impl Editor {
// Validate the entire replay first, then publish while the original
// document is restored, and only then commit the computed deletion.
std::mem::swap(&mut self.active.lines, &mut before.lines);
self.active.dirty_cache.set(None);
std::mem::swap(&mut self.active.view.targets, &mut before.targets);
std::mem::swap(&mut self.active.view.primary, &mut before.primary);
restored = true;
@@ -11389,6 +11451,7 @@ impl Editor {
shape,
)?;
std::mem::swap(&mut self.active.lines, &mut before.lines);
self.active.dirty_cache.set(None);
std::mem::swap(&mut self.active.view.targets, &mut before.targets);
std::mem::swap(&mut self.active.view.primary, &mut before.primary);
restored = false;
@@ -11412,6 +11475,7 @@ impl Editor {
self.active.view.primary = before.primary;
}
self.active.text_revision = revision;
self.active.dirty_cache.set(dirty_cache);
self.active.diagnostics = diagnostics;
self.active.semantic_tokens = semantic_tokens;
self.active.view.mode = mode;
@@ -11677,13 +11741,35 @@ impl Editor {
self.normalize_targets();
}
fn restore_history(&mut self, entry: HistoryEntry) -> HistoryEntry {
let mut lines = self.active.lines.clone();
let end = entry.start + entry.before.lines.len();
let before = Snapshot {
lines: lines
.splice(entry.start..entry.end, entry.before.lines)
.collect(),
targets: self.active.view.targets.clone(),
primary: self.active.view.primary,
};
self.restore(Snapshot {
lines,
targets: entry.before.targets,
primary: entry.before.primary,
});
HistoryEntry {
start: entry.start,
end,
before,
}
}
fn undo(&mut self) {
if self.mutation_blocked() {
return;
}
if let Some(before) = self.active.undo.pop() {
self.active.redo.push(self.snapshot());
self.restore(before);
let reverse = self.restore_history(before);
self.active.redo.push(reverse);
} else {
self.set_message("Nothing to undo");
}
@@ -11694,8 +11780,8 @@ impl Editor {
return;
}
if let Some(after) = self.active.redo.pop() {
self.active.undo.push(self.snapshot());
self.restore(after);
let reverse = self.restore_history(after);
self.active.undo.push(reverse);
} else {
self.set_message("Nothing to redo");
}
@@ -11951,6 +12037,61 @@ impl Editor {
if edits.is_empty() {
return;
}
// Ordinary typing/deletion can edit one line without copying the entire file.
// Snapshot previews, shared panes and semantic overlays use the general remapping path.
if original.is_none()
&& edits.len() == 1
&& self.panes.len() == 1
&& self.active.semantic_tokens.is_none()
&& !edits[0].text.contains('\n')
{
let edit = &edits[0];
let mut offsets = line_offsets(&self.active.lines);
let row = offsets
.partition_point(|offset| *offset <= edit.start)
.saturating_sub(1);
let start = edit.start - offsets[row];
let end = edit.end - offsets[row];
if end <= self.active.lines[row].len() {
let mut destinations: Vec<_> = self
.active
.view
.targets
.iter()
.map(|target| {
(
translate_edit_offset(
cursor_offset(&self.active.lines, &offsets, target.active),
true,
std::iter::once(edit),
),
true,
)
})
.collect();
if let Some(target) = edit.target {
destinations[target] = (edit.start + edit.caret, edit.round_up);
}
if self.active.lines[row][start..end] != *edit.text {
self.active.lines[row].replace_range(start..end, &edit.text);
self.active.text_changed();
}
for offset in &mut offsets[row + 1..] {
*offset = *offset - (end - start) + edit.text.len();
}
for (target, (byte, round_up)) in
self.active.view.targets.iter_mut().zip(destinations)
{
target.active = offset_cursor(&self.active.lines, &offsets, byte, round_up);
target.anchor = target.active;
target.preferred_columns = None;
}
if !preserve_targets {
self.normalize_targets();
}
return;
}
}
edits.sort_by_key(|edit| (edit.start, edit.end));
let mut unique: Vec<usize> = Vec::with_capacity(edits.len());
for (index, edit) in edits.iter().enumerate() {
@@ -13115,7 +13256,7 @@ fn atomic_save(
.parent()
.filter(|parent| !parent.as_os_str().is_empty())
.unwrap_or(Path::new("."));
let (temporary, mut file) = loop {
let (temporary, file) = loop {
let id = TEMP_ID.fetch_add(1, Ordering::Relaxed);
let temporary = parent.join(format!(".vamp-{}-{id}.tmp", std::process::id()));
let mut options = OpenOptions::new();
@@ -13129,12 +13270,15 @@ fn atomic_save(
}
};
let result = (|| {
let mut writer = io::BufWriter::new(&file);
for (index, line) in lines.iter().enumerate() {
if index > 0 {
file.write_all(b"\n")?;
writer.write_all(b"\n")?;
}
file.write_all(line.as_bytes())?;
writer.write_all(line.as_bytes())?;
}
writer.flush()?;
drop(writer);
if let Some(metadata) = metadata {
file.set_permissions(metadata.permissions())?;
}
+3
View File
@@ -109,6 +109,7 @@ impl Editor {
.is_some_and(|binding| binding.owner == self.tab_id)
{
buffer.saved_lines.clone_from(&buffer.lines);
buffer.dirty_cache.set(Some(false));
}
}
}
@@ -268,6 +269,7 @@ impl Editor {
}) {
buffer.review_buffer = None;
buffer.saved_lines.clone_from(&buffer.lines);
buffer.dirty_cache.set(Some(false));
}
}
self.autosave_local_review()?;
@@ -849,6 +851,7 @@ impl Editor {
self.active.lines = split_lines(text);
self.active.saved_lines.clone_from(&self.active.lines);
self.active.text_revision = self.active.text_revision.wrapping_add(1);
self.active.dirty_cache.set(Some(false));
self.normalize_targets();
}
} else {
+1 -1
View File
@@ -19,7 +19,7 @@ use std::{
thread::{self, JoinHandle},
};
const FILE_LIMIT: usize = 16 * 1024 * 1024;
pub(crate) const FILE_LIMIT: usize = 16 * 1024 * 1024;
const BATCH_LIMIT: usize = 64 * 1024 * 1024;
const FILE_COUNT_LIMIT: usize = 1024;
type Files = Vec<(usize, PathBuf, Arc<str>)>;
+37 -9
View File
@@ -425,7 +425,18 @@ struct GutterSource {
save_revision: u64,
path: PathBuf,
dirty: bool,
contents: Arc<str>,
contents: Option<Arc<str>>,
}
fn text_fits(lines: &[String], limit: usize) -> bool {
lines
.iter()
.try_fold(lines.len().saturating_sub(1), |bytes, line| {
bytes
.checked_add(line.len())
.filter(|bytes| *bytes <= limit)
})
.is_some()
}
/// Owns command eligibility and execution hosts together. Frontends can inspect
@@ -1479,8 +1490,11 @@ impl EditorSession {
source.revision == document.revision && source.path == path
})
.map_or_else(
|| Arc::from(document.lines.join("\n")),
|source| Arc::clone(&source.contents),
|| {
text_fits(document.lines, crate::gutter::FILE_LIMIT)
.then(|| Arc::from(document.lines.join("\n")))
},
|source| source.contents.clone(),
);
Some(GutterSource {
buffer: document.id,
@@ -1492,6 +1506,16 @@ impl EditorSession {
})
})
.collect();
for source in &self.gutter_sources {
if source.contents.is_none() {
self.editor.set_gutter_changes(
source.buffer,
source.revision,
&source.path,
Vec::new(),
);
}
}
}
self.gutter_request_id = self.gutter_request_id.wrapping_add(1);
self.last_gutter_refresh = Some(now);
@@ -1508,12 +1532,12 @@ impl EditorSession {
.gutter_sources
.iter()
.filter(|_| line_gutter)
.map(|source| {
(
.filter_map(|source| {
Some((
source.buffer,
source.path.clone(),
Arc::clone(&source.contents),
)
Arc::clone(source.contents.as_ref()?),
))
})
.collect();
let picker = self.picker_source.as_ref().map(|root| {
@@ -1521,7 +1545,9 @@ impl EditorSession {
.gutter_sources
.iter()
.filter(|source| source.dirty)
.map(|source| (source.path.clone(), Arc::clone(&source.contents)))
.filter_map(|source| {
Some((source.path.clone(), Arc::clone(source.contents.as_ref()?)))
})
.collect();
(root.clone(), buffers)
});
@@ -1624,7 +1650,9 @@ impl EditorSession {
self.blame_request_id = self.blame_request_id.wrapping_add(1);
self.blame_lines = None;
changed |= self.editor.clear_line_blame();
if let Some(source) = &self.blame_source {
if let Some(source) = &self.blame_source
&& text_fits(self.editor.lines(), crate::blame::LIMIT)
{
if self.blame_worker.is_none() {
self.blame_worker = BlameWorker::start().ok();
}
+109 -6
View File
@@ -14,6 +14,12 @@ pub(crate) const RUST_ID: &str = "rust";
pub(crate) const RUBY_ID: &str = "ruby";
pub(crate) const PYTHON_ID: &str = "python";
// Full trees and their indexes are optional; editing remains available beyond these budgets.
const MAX_SYNTAX_BYTES: usize = 1024 * 1024;
const MAX_SYNTAX_LINE_BYTES: usize = 64 * 1024;
const SOURCE_BUDGET_NOTICE: &str = "source exceeds 1 MiB syntax budget";
const LINE_BUDGET_NOTICE: &str = "line exceeds 64 KiB syntax budget";
pub(crate) fn is_markdown_path(path: &Path) -> bool {
languages::detect(Some(path), None).is_some_and(|syntax| syntax.name == "Markdown")
}
@@ -69,6 +75,7 @@ pub struct SyntaxCache {
path: Option<PathBuf>,
final_newline: bool,
syntax: Option<&'static LanguageSpec>,
budget_exceeded: Option<&'static str>,
source: String,
next_source: String,
line_starts: Vec<usize>,
@@ -100,7 +107,34 @@ impl SyntaxCache {
}
pub(crate) fn update_text(&mut self, text: &str, path: &Path) {
if self.source == text
// Check lengths before equality, splitting into owned lines, or copying the source.
let budget_exceeded = if text.len() > MAX_SYNTAX_BYTES {
Some(SOURCE_BUDGET_NOTICE)
} else if text
.split('\n')
.any(|line| line.len() > MAX_SYNTAX_LINE_BYTES)
{
Some(LINE_BUDGET_NOTICE)
} else {
None
};
if let Some(reason) = budget_exceeded {
let syntax = languages::detect(Some(path), None)
.or_else(|| languages::detect(None, Some(syntax_prefix(text))));
let language_changed = !same_language(self.syntax, syntax);
self.revision = None;
self.final_newline = false;
if self.path.as_deref() != Some(path) {
self.path = Some(path.to_path_buf());
}
self.syntax = syntax;
if self.budget_exceeded != Some(reason) || language_changed {
self.use_plain_text(reason);
}
return;
}
if self.budget_exceeded.is_none()
&& self.source == text
&& self.path.as_deref() == Some(path)
&& self.highlighted.iter().all(|highlighted| *highlighted)
{
@@ -168,8 +202,11 @@ impl SyntaxCache {
let path_changed = self.path.as_deref() != path;
let revision_changed =
self.revision != Some(revision) || self.final_newline != final_newline;
if self.budget_exceeded.is_some() && !path_changed && !revision_changed {
return;
}
let syntax = if path_changed || revision_changed {
languages::detect(path, lines.first().map(String::as_str))
languages::detect(path, lines.first().map(|line| syntax_prefix(line)))
} else {
self.syntax
};
@@ -181,12 +218,29 @@ impl SyntaxCache {
}
self.syntax = syntax;
let length = if revision_changed {
syntax_source_length(lines, final_newline)
} else if let Some(reason) = self.budget_exceeded {
Err(reason)
} else {
Ok(self.source.len())
};
let length = match length {
Ok(length) => length,
Err(reason) => {
if self.budget_exceeded != Some(reason) || language_changed {
self.use_plain_text(reason);
}
return;
}
};
if self.budget_exceeded.take().is_some() {
self.error = None;
}
let mut edit = None;
if revision_changed {
self.next_source.clear();
let length = lines.iter().map(String::len).sum::<usize>()
+ lines.len().saturating_sub(1)
+ usize::from(final_newline);
self.next_source.reserve(length);
for (row, line) in lines.iter().enumerate() {
if row > 0 {
@@ -289,6 +343,27 @@ impl SyntaxCache {
}
}
fn use_plain_text(&mut self, reason: &'static str) {
// Drop parser arenas as well as source/span capacities when crossing the budget.
*self = Self {
revision: self.revision,
path: self.path.take(),
final_newline: self.final_newline,
syntax: self.syntax,
budget_exceeded: Some(reason),
error: self
.syntax
.map(|syntax| format!("{}: plain text ({reason})", syntax.name)),
#[cfg(test)]
parses: self.parses,
#[cfg(test)]
incremental_parses: self.incremental_parses,
#[cfg(test)]
injection_parses: self.injection_parses,
..Self::default()
};
}
/// Sorted, non-overlapping, end-exclusive ranges. Unclassified text uses the normal foreground.
pub fn line(&self, row: usize) -> &[SyntaxSpan] {
self.lines.get(row).map_or(&[], Vec::as_slice)
@@ -352,7 +427,7 @@ impl SyntaxCache {
self.syntax.map(|syntax| syntax.name)
}
/// Genuine grammar/query failures disable highlighting; ERROR nodes in user text do not.
/// Why highlighting is unavailable: a deliberate plain-text fallback or grammar/query failure.
pub fn error(&self) -> Option<&str> {
self.error.as_deref()
}
@@ -546,6 +621,34 @@ impl SyntaxCache {
}
}
fn syntax_prefix(text: &str) -> &str {
let mut end = text.len().min(MAX_SYNTAX_LINE_BYTES);
while !text.is_char_boundary(end) {
end -= 1;
}
text[..end].split('\n').next().unwrap_or_default()
}
fn syntax_source_length(lines: &[String], final_newline: bool) -> Result<usize, &'static str> {
let mut length = lines
.len()
.saturating_sub(1)
.saturating_add(usize::from(final_newline));
if length > MAX_SYNTAX_BYTES {
return Err(SOURCE_BUDGET_NOTICE);
}
for line in lines {
if line.len() > MAX_SYNTAX_LINE_BYTES {
return Err(LINE_BUDGET_NOTICE);
}
length = length.saturating_add(line.len());
if length > MAX_SYNTAX_BYTES {
return Err(SOURCE_BUDGET_NOTICE);
}
}
Ok(length)
}
fn has_scope_language(language: &str) -> bool {
matches!(
language,
+4 -1
View File
@@ -1844,7 +1844,10 @@ fn column_view_preserves_split_gaps_unified_sides_and_folded_coordinates() {
let ColumnRow::Text(line) = before.row(1).unwrap() else {
panic!("unified insertion must paint the after source");
};
assert_eq!((line.text, line.side, line.source_row), (Some("added"), Focus::After, 1));
assert_eq!(
(line.text, line.side, line.source_row),
(Some("added"), Focus::After, 1)
);
assert_eq!(line.numbers, [None, Some(1)]);
assert!(line.changed);
assert_eq!(before.native_row(1), None);
+122 -13
View File
@@ -323,6 +323,7 @@ fn resolver_explorer_retains_last_jj_conflict_state_until_confirmation() {
ResolverStatus::Unconfirmed
);
editor.active.saved_lines.clone_from(&editor.active.lines);
editor.active.dirty_cache.set(None);
editor.refresh_resolver_status();
assert_eq!(
editor.resolver().unwrap().files[0].status,
@@ -3245,7 +3246,10 @@ fn review_identity_changes_retire_drafts_before_reusing_comment_ids() {
let restored = root.join("restored.json");
editor.execute(&format!("diff.review.save {}", restored.display()));
assert_eq!(
crate::review::Review::load(&restored, &root).unwrap().comments[0].body,
crate::review::Review::load(&restored, &root)
.unwrap()
.comments[0]
.body,
"Original feedback"
);
@@ -7145,19 +7149,25 @@ mod system_clipboard {
editor
.apply(vec![Effect::Replace("aa\nbb\ncc".into())])
.unwrap();
editor.active.saved_lines.clone_from(&editor.active.lines);
editor.active.dirty_cache.set(Some(false));
assert!(!editor.dirty());
type_text(&mut editor, "Vj");
fixture.fail("fail-write", true);
let before = state(&editor);
editor.handle(Input::Char('.'));
assert_eq!(state(&editor), before);
assert!(!editor.dirty());
assert_eq!(fixture.bytes("state"), b"a\n");
assert!(editor.visual_linewise());
fixture.fail("fail-write", false);
editor.handle(Input::Char('.'));
assert_eq!(editor.context().text, "cc");
assert_eq!(fixture.bytes("state"), b"aa\nbb\n");
assert!(editor.dirty());
editor.handle(Input::Char('u'));
assert_eq!(editor.context().text, "aa\nbb\ncc");
assert!(!editor.dirty());
assert_eq!(editor.mode(), Mode::Normal);
}
@@ -7795,11 +7805,12 @@ fn filter_escape_resume_and_clear_do_not_edit_underlying_buffers_or_history() {
editor.count = Some(7);
let snapshot =
|state: &Snapshot| (state.lines.clone(), state.targets.clone(), state.primary);
let history = |entry: &HistoryEntry| (entry.start, entry.end, snapshot(&entry.before));
let lines = editor.lines().to_owned();
let targets = editor.active.view.targets.clone();
let primary = editor.active.view.primary;
let undo: Vec<_> = editor.active.undo.iter().map(&snapshot).collect();
let redo: Vec<_> = editor.active.redo.iter().map(&snapshot).collect();
let undo: Vec<_> = editor.active.undo.iter().map(&history).collect();
let redo: Vec<_> = editor.active.redo.iter().map(&history).collect();
let insert = editor.active.view.insert_start.as_ref().map(snapshot);
let buffer = editor.buffer_id();
let buffers = editor.buffer_count();
@@ -7845,11 +7856,11 @@ fn filter_escape_resume_and_clear_do_not_edit_underlying_buffers_or_history() {
assert_eq!(editor.buffer_id(), buffer);
assert_eq!(editor.buffer_count(), buffers);
assert_eq!(
editor.active.undo.iter().map(&snapshot).collect::<Vec<_>>(),
editor.active.undo.iter().map(&history).collect::<Vec<_>>(),
undo
);
assert_eq!(
editor.active.redo.iter().map(&snapshot).collect::<Vec<_>>(),
editor.active.redo.iter().map(&history).collect::<Vec<_>>(),
redo
);
assert_eq!(
@@ -8078,11 +8089,12 @@ fn selection_navigation_wraps_without_touching_any_underlying_mode() {
assert!(len > 1);
let snapshot =
|state: &Snapshot| (state.lines.clone(), state.targets.clone(), state.primary);
let history = |entry: &HistoryEntry| (entry.start, entry.end, snapshot(&entry.before));
let text = editor.context().text;
let targets = editor.active.view.targets.clone();
let primary = editor.active.view.primary;
let undo: Vec<_> = editor.active.undo.iter().map(&snapshot).collect();
let redo: Vec<_> = editor.active.redo.iter().map(&snapshot).collect();
let undo: Vec<_> = editor.active.undo.iter().map(&history).collect();
let redo: Vec<_> = editor.active.redo.iter().map(&history).collect();
let insert = editor.active.view.insert_start.as_ref().map(snapshot);
editor.count = Some(7);
for (input, selected) in [
@@ -8104,11 +8116,11 @@ fn selection_navigation_wraps_without_touching_any_underlying_mode() {
assert_eq!(editor.active.view.command_line, "original prompt");
assert_eq!(editor.count(), Some(7));
assert_eq!(
editor.active.undo.iter().map(&snapshot).collect::<Vec<_>>(),
editor.active.undo.iter().map(&history).collect::<Vec<_>>(),
undo
);
assert_eq!(
editor.active.redo.iter().map(&snapshot).collect::<Vec<_>>(),
editor.active.redo.iter().map(&history).collect::<Vec<_>>(),
redo
);
assert_eq!(
@@ -12182,6 +12194,7 @@ fn palette_cancellation_preserves_every_underlying_mode_and_insert_group() {
}
let snapshot =
|state: &Snapshot| (state.lines.clone(), state.targets.clone(), state.primary);
let history = |entry: &HistoryEntry| (entry.start, entry.end, snapshot(&entry.before));
let lines = editor.active.lines.clone();
let targets = editor.active.view.targets.clone();
let primary = editor.active.view.primary;
@@ -12190,8 +12203,8 @@ fn palette_cancellation_preserves_every_underlying_mode_and_insert_group() {
let search = editor.active.view.search_query.clone();
let found = editor.active.view.search_match;
let message = editor.message.clone();
let undo: Vec<_> = editor.active.undo.iter().map(&snapshot).collect();
let redo: Vec<_> = editor.active.redo.iter().map(&snapshot).collect();
let undo: Vec<_> = editor.active.undo.iter().map(&history).collect();
let redo: Vec<_> = editor.active.redo.iter().map(&history).collect();
let insert = editor.active.view.insert_start.as_ref().map(snapshot);
editor.handle(Input::Ctrl('p'));
@@ -12218,11 +12231,11 @@ fn palette_cancellation_preserves_every_underlying_mode_and_insert_group() {
assert_eq!(editor.active.view.search_match, found);
assert_eq!(editor.message, message);
assert_eq!(
editor.active.undo.iter().map(&snapshot).collect::<Vec<_>>(),
editor.active.undo.iter().map(&history).collect::<Vec<_>>(),
undo
);
assert_eq!(
editor.active.redo.iter().map(&snapshot).collect::<Vec<_>>(),
editor.active.redo.iter().map(&history).collect::<Vec<_>>(),
redo
);
assert_eq!(
@@ -13489,6 +13502,56 @@ fn insert_sessions_and_open_lines_are_single_undo_actions() {
assert_eq!(editor.context().text, "hello\n世界");
}
#[test]
fn large_buffer_history_retains_changed_rows_and_round_trips_all_groups() {
let directory = Directory::new();
let path = directory.path("large.txt");
let source = format!("head\n{}tail\n", "unchanged contents\n".repeat(10_000));
fs::write(&path, &source).unwrap();
let mut editor = Editor::open(Some(path), vec![]).unwrap();
for _ in 0..12 {
editor.handle(Input::Char('i'));
editor.handle(Input::Char('界'));
editor.handle(Input::Escape);
}
assert_eq!(editor.lines()[0], format!("{}head", "界".repeat(12)));
let retained_bytes: usize = editor
.active
.undo
.iter()
.flat_map(|entry| &entry.before.lines)
.map(String::len)
.sum();
assert!(
retained_bytes < 4096,
"small edits retained {retained_bytes} bytes of history"
);
for _ in 0..12 {
editor.handle(Input::Char('u'));
}
assert_eq!(editor.lines().join("\n"), source);
assert!(!editor.dirty());
for _ in 0..12 {
editor.handle(Input::Char('U'));
}
assert_eq!(editor.lines()[0], format!("{}head", "界".repeat(12)));
assert_eq!(&editor.lines()[1..], &split_lines(&source)[1..]);
assert!(editor.dirty());
editor.handle(Input::Char('u'));
editor.handle(Input::Char('i'));
editor.handle(Input::Enter);
editor.handle(Input::Escape);
assert!(editor.active.redo.is_empty());
editor.handle(Input::Char('u'));
assert_eq!(editor.lines()[0], format!("{}head", "界".repeat(11)));
editor.execute("w");
assert!(!editor.dirty());
assert_eq!(
fs::read_to_string(editor.path().unwrap()).unwrap(),
format!("{}{source}", "界".repeat(11))
);
}
#[test]
fn grapheme_navigation_and_deletion_do_not_split_unicode() {
let mut editor = editor();
@@ -14076,6 +14139,52 @@ fn saves_preserve_empty_files_lf_trailing_newlines_and_dirty_history() {
assert!(editor.should_quit());
}
#[test]
fn dirty_cache_tracks_saved_text_across_edits_undo_restore_and_reload() {
let directory = Directory::new();
let path = directory.path("file.txt");
fs::write(&path, "saved\n").unwrap();
let mut editor = Editor::open(Some(path.clone()), vec![]).unwrap();
editor.set_clipboard_mode("separate").unwrap();
assert!(!editor.dirty());
editor
.apply(vec![Effect::Insert("changed".into())])
.unwrap();
assert!(editor.dirty());
for _ in 0..3 {
assert!(editor.dirty());
assert!(editor.active.info(true).dirty);
assert!(editor.active.document().dirty);
assert!(!editor.active.refresh_file().unwrap());
}
editor.handle(Input::Char('u'));
assert!(!editor.dirty());
editor.handle(Input::Char('U'));
assert!(editor.dirty());
editor
.apply(vec![Effect::Replace("saved\n".into())])
.unwrap();
assert!(!editor.dirty());
editor.apply(vec![Effect::Insert("new ".into())]).unwrap();
assert!(editor.dirty());
let revision = editor.text_revision();
editor.execute("w");
assert_eq!(editor.text_revision(), revision);
assert!(!editor.dirty());
editor.handle(Input::Char('u'));
assert!(editor.dirty());
editor.handle(Input::Char('U'));
assert!(!editor.dirty());
fs::write(&path, "reloaded content\n").unwrap();
assert!(editor.active.refresh_file().unwrap());
assert_eq!(editor.lines(), ["reloaded content", ""]);
assert!(!editor.dirty());
assert!(!editor.active.refresh_file().unwrap());
}
#[test]
fn save_as_and_failed_wq_keep_path_dirty_state_and_quit_correct() {
let directory = Directory::new();
+41
View File
@@ -4,6 +4,47 @@ use std::{
sync::atomic::{AtomicU64, Ordering},
};
#[test]
fn oversized_repository_text_skips_snapshots_and_recovers_after_shrinking() {
let fixture = Fixture::new();
let path = fixture.file("buffer.txt", "small\n");
let mut session = EditorSession::open(Some(path.clone()), vec![], None).unwrap();
let buffer = session.editor.buffer_id();
session.editor.set_gutter_changes(
buffer,
session.editor.text_revision(),
&path,
vec![Some(crate::gutter::Change::Added)],
);
assert_eq!(
session.editor.gutter_change(0),
Some(crate::gutter::Change::Added)
);
fs::write(&path, "x".repeat(crate::gutter::FILE_LIMIT + 1)).unwrap();
assert!(session.editor.refresh_files());
session.poll_gutter_at(Instant::now());
session.synchronize_documents();
let source = session
.gutter_sources
.iter()
.find(|source| source.buffer == buffer)
.unwrap();
assert!(source.contents.is_none());
assert!(session.blame_worker.is_none());
assert_eq!(session.editor.gutter_change(0), None);
fs::write(&path, "small again\n").unwrap();
assert!(session.editor.refresh_files());
session.poll_gutter_at(Instant::now());
session.synchronize_documents();
let source = session
.gutter_sources
.iter()
.find(|source| source.buffer == buffer)
.unwrap();
assert_eq!(source.contents.as_deref(), Some("small again\n"));
assert!(session.blame_worker.is_some());
}
#[test]
fn resolver_load_errors_return_through_real_session_without_touching_the_draft() {
let fixture = Fixture::new();
+146
View File
@@ -729,6 +729,152 @@ fn unchanged_revision_and_same_source_do_not_allocate_or_parse() {
assert_eq!(cache.incremental_parses, 0);
}
fn assert_plain_text_storage(cache: &SyntaxCache) {
assert!(cache.tree.is_none());
assert!(cache.context.is_none());
assert!(cache.scopes.get().is_none());
assert_eq!(cache.source.capacity(), 0);
assert_eq!(cache.next_source.capacity(), 0);
assert_eq!(cache.line_starts.capacity(), 0);
assert_eq!(cache.lines.capacity(), 0);
assert_eq!(cache.string_spans.capacity(), 0);
assert_eq!(cache.highlighted.capacity(), 0);
assert_eq!(cache.injections.capacity(), 0);
assert_eq!(cache.captures.capacity(), 0);
assert_eq!(cache.highlights.capacity(), 0);
assert_eq!(cache.events.capacity(), 0);
assert!(cache.line(0).is_empty());
assert!(cache.string_spans(0).is_empty());
assert!(cache.scope_at(0).is_none());
}
#[test]
fn syntax_budget_crossing_releases_state_and_reenables_after_shrinking() {
let small = "fn main() { let value = \"ok\"; }";
let mut input = lines(small);
let mut remaining = MAX_SYNTAX_BYTES - small.len();
while remaining > 0 {
let length = (remaining - 1).min(16 * 1024);
input.push(" ".repeat(length));
remaining -= length + 1;
}
let path = Some(Path::new("budget.rs"));
let mut cache = SyntaxCache::new();
cache.update_visible(&input, path, 0, std::slice::from_ref(&(0..1)));
assert_eq!(cache.source.len(), MAX_SYNTAX_BYTES);
assert_eq!(cache.error(), None);
assert_eq!(cache.parses, 1);
assert_token(&cache, small, 0, "fn", SyntaxKind::Keyword);
assert!(cache.scope_at(0).is_some());
input.last_mut().unwrap().push(' ');
cache.update_visible(&input, path, 1, std::slice::from_ref(&(0..1)));
assert_eq!(cache.language(), Some("Rust"));
assert!(cache.error().unwrap().contains("plain text"));
assert_eq!(cache.parses, 1);
assert_plain_text_storage(&cache);
let notice = cache.error.as_ref().unwrap().as_ptr();
cache.update(&input, path, 1);
assert_eq!(cache.error.as_ref().unwrap().as_ptr(), notice);
assert_eq!(cache.parses, 1);
assert_plain_text_storage(&cache);
cache.update(&input, Some(Path::new("budget.py")), 1);
assert_eq!(cache.language(), Some("Python"));
assert!(cache.error().unwrap().starts_with("Python"));
assert_plain_text_storage(&cache);
cache.update(&input, Some(Path::new("budget.txt")), 1);
assert_eq!(cache.language(), None);
assert_eq!(cache.error(), None);
assert_plain_text_storage(&cache);
cache.update(&lines(small), path, 2);
assert_eq!(cache.parses, 2);
assert_eq!(cache.error(), None);
assert_token(&cache, small, 0, "fn", SyntaxKind::Keyword);
assert!(cache.scope_at(0).is_some());
}
#[test]
fn diff_syntax_budget_counts_the_separate_final_newline() {
let mut input = vec!["//".to_owned() + &" ".repeat(16 * 1024 - 2); 63];
input.push("//".to_owned() + &" ".repeat(16 * 1024 - 65));
let path = Some(Path::new("budget.rs"));
let mut cache = SyntaxCache::new();
cache.update_diff(&input, path, 0, false, None);
assert_eq!(cache.source.len(), MAX_SYNTAX_BYTES);
assert_eq!(cache.parses, 1);
assert_eq!(kind_at(&cache, 0, 0), Some(SyntaxKind::Comment));
cache.update_diff(&input, path, 0, true, None);
assert_eq!(cache.parses, 1);
assert_plain_text_storage(&cache);
cache.update_diff(&input, path, 0, false, Some(std::slice::from_ref(&(0..1))));
assert_eq!(cache.parses, 2);
assert_eq!(cache.error(), None);
assert_eq!(kind_at(&cache, 0, 0), Some(SyntaxKind::Comment));
}
#[test]
fn long_lines_and_large_text_never_copy_or_parse_and_clear_injections() {
let markdown = "# Title\n```rust\nfn main() { \"ok\"; }\n```";
let mut cache = self::cache(markdown, "hover.md");
assert!(!cache.injections.is_empty());
assert!(cache.scope_at(2).is_some());
let parses = cache.parse_counts();
let long_line = "//".to_owned() + &" ".repeat(MAX_SYNTAX_LINE_BYTES - 1);
cache.update_text(&long_line, Path::new("large.rs"));
assert_eq!(cache.language(), Some("Rust"));
assert_eq!(cache.parse_counts(), parses);
assert!(cache.error().unwrap().contains("line"));
assert_plain_text_storage(&cache);
let large = " ".repeat(52 * 1024 * 1024);
cache.update_text(&large, Path::new("parser.c"));
assert_eq!(cache.language(), Some("C"));
assert_eq!(cache.parse_counts(), parses);
assert_plain_text_storage(&cache);
let notice = cache.error.as_ref().unwrap().as_ptr();
cache.update_text(&large, Path::new("parser.c"));
assert_eq!(cache.error.as_ref().unwrap().as_ptr(), notice);
assert_plain_text_storage(&cache);
let large_lines = vec![large];
cache.update(&large_lines, Some(Path::new("parser.c")), 10);
assert_eq!(cache.language(), Some("C"));
assert_eq!(cache.parse_counts(), parses);
assert_plain_text_storage(&cache);
cache.update_diff(&large_lines, Some(Path::new("parser.c")), 10, true, None);
assert_eq!(cache.parse_counts(), parses);
assert_plain_text_storage(&cache);
cache.update_text(markdown, Path::new("hover.md"));
assert!(cache.parse_counts().0 > parses.0);
assert_eq!(cache.error(), None);
assert!(!cache.injections.is_empty());
assert!(cache.scope_at(2).is_some());
let boundary = "//".to_owned() + &" ".repeat(MAX_SYNTAX_LINE_BYTES - 2);
cache.update(&lines(&boundary), Some(Path::new("line.rs")), 50);
assert_eq!(cache.error(), None);
assert_eq!(kind_at(&cache, 0, 0), Some(SyntaxKind::Comment));
cache.update(&lines(&long_line), Some(Path::new("line.rs")), 51);
assert_plain_text_storage(&cache);
cache.update_text("", Path::new("line.rs"));
assert_eq!(cache.language(), Some("Rust"));
assert_eq!(cache.error(), None);
assert!(cache.budget_exceeded.is_none());
assert!(cache.line(0).is_empty());
assert!(cache.scope_at(0).is_none());
let mut script = vec!["#!/usr/bin/env python3".to_owned(), long_line.clone()];
cache.update(&script, None, 60);
assert_eq!(cache.language(), Some("Python"));
assert_plain_text_storage(&cache);
script[0] = "#!/usr/bin/env ruby".to_owned();
cache.update(&script, None, 61);
assert_eq!(cache.language(), Some("Ruby"));
assert!(cache.error().unwrap().starts_with("Ruby"));
assert_plain_text_storage(&cache);
script.pop();
cache.update(&script, None, 62);
assert_eq!(cache.language(), Some("Ruby"));
assert_eq!(cache.error(), None);
assert!(cache.tree.is_some());
}
#[test]
fn unicode_minimal_edits_use_byte_points_and_never_split_codepoints() {
let old = "x\n\t\"é 👩‍💻\"\n";