Compare commits

...

3 Commits

17 changed files with 3001 additions and 208 deletions
+1 -1
View File
@@ -43,7 +43,7 @@ Ruby configuration and plugins run with your account's filesystem/process permis
- One visible buffer at a time; no splits/tabs or graphical frontend. - One visible buffer at a time; no splits/tabs or graphical frontend.
- Modal editing, not full Vim compatibility: characterwise Visual selections, but no line/block selection, named registers, or Vim operators. Search is literal, case-sensitive, and single-line. - Modal editing, not full Vim compatibility: characterwise Visual selections, but no line/block selection, named registers, or Vim operators. Search is literal, case-sensitive, and single-line.
- Native opt-in LSP supports diagnostics, go-to-definition, and plain-text hover—not completion, rename, formatting, or code actions. - Native opt-in LSP supports diagnostics, go-to-definition, plain-text hover, and automatic semantic highlighting—not completion, rename, formatting, or code actions.
- System clipboard access supports macOS and Linux. Separate clipboard mode keeps internal editing available on unsupported/headless environments. - System clipboard access supports macOS and Linux. Separate clipboard mode keeps internal editing available on unsupported/headless environments.
Performance and subprocess limits are documented in [development](docs/development.md#performance-and-process-limits). Performance and subprocess limits are documented in [development](docs/development.md#performance-and-process-limits).
+10 -5
View File
@@ -11,9 +11,9 @@ One crate; no workspace or terminal-coordinate types in the editing backend:
- `editor`: owns buffer lifecycle and per-buffer text, saved state, primary/selection set, modes, search, and undo/redo; session-wide clipboard policy, independent registers and owned-system fragment metadata, semantic native repeat, action metadata, command-palette and file-picker focus/dispatch, key notation, leader/count prefixes and binding resolution, modal paste policy, saving, command ownership, capabilities, and validated edit effects. Its low-level interface returns an `Invocation` when an external command is selected. - `editor`: owns buffer lifecycle and per-buffer text, saved state, primary/selection set, modes, search, and undo/redo; session-wide clipboard policy, independent registers and owned-system fragment metadata, semantic native repeat, action metadata, command-palette and file-picker focus/dispatch, key notation, leader/count prefixes and binding resolution, modal paste policy, saving, command ownership, capabilities, and validated edit effects. Its low-level interface returns an `Invocation` when an external command is selected.
- `files`: owns the gitignore-aware directory/file catalog, authoritative OS paths, compact tree projection, query filtering, browse/search focus, expansion/selection restoration, navigation, and scan/error state shared by floating and docked presentations. - `files`: owns the gitignore-aware directory/file catalog, authoritative OS paths, compact tree projection, query filtering, browse/search focus, expansion/selection restoration, navigation, and scan/error state shared by floating and docked presentations.
- `clipboard`: owns native plain-text provider selection and bounded subprocess I/O; it never computes selections or editing offsets. - `clipboard`: owns native plain-text provider selection and bounded subprocess I/O; it never computes selections or editing offsets.
- `syntax`: owns Tree-sitter grammar detection, per-buffer incremental trees, bundled highlight/local/injection queries, and semantic byte spans; it has no terminal color/geometry or editing dependency. - `syntax`: owns Tree-sitter grammar detection, per-buffer incremental trees, bundled highlight/local/injection queries, and syntax byte spans; it has no terminal color/geometry or editing dependency.
- `plugin`: owns generic document snapshots/events, zero-based grapheme positions, diagnostics, and versioned asynchronous update contracts. It has no LSP protocol types. - `plugin`: owns generic document snapshots/events, zero-based grapheme positions, diagnostics, frontend-independent semantic token kinds/modifiers, and versioned asynchronous update contracts. Semantic token updates are native-only, not part of the Ruby wire protocol; it has no LSP protocol types.
- `lsp`: owns opt-in native server workers, bounded framed JSON-RPC, synchronization/capability negotiation, settings, UTF-16/file-URI conversion, and definition/hover results. Its private configuration resolver supplies presets without restricting language IDs. - `lsp`: owns opt-in native server workers, bounded framed JSON-RPC, synchronization/capability negotiation, settings, UTF-16/file-URI conversion, definition/hover results, and semantic token requests/legend decoding. Its private configuration resolver supplies presets without restricting language IDs.
- `session`: `EditorSession` owns the editor, native LSP, and Ruby hosts together. It loads configuration once, registers eligible plugins before installing shared bindings, tracks document open/change/save/close events, polls native and plugin results, validates versioned diagnostics/navigation, and retires failed plugin ownership. - `session`: `EditorSession` owns the editor, native LSP, and Ruby hosts together. It loads configuration once, registers eligible plugins before installing shared bindings, tracks document open/change/save/close events, polls native and plugin results, validates versioned diagnostics/navigation, and retires failed plugin ownership.
- `tui`: translates terminal events to session input, renders read-only active-buffer state, keeps frontend-owned viewports and syntax caches by stable buffer ID, and advertises `text`, `styled_text`, and `diagnostics` capabilities. It maps semantic tokens and diagnostics to terminal styling and has no server/protocol logic or paste acceptance policy. - `tui`: translates terminal events to session input, renders read-only active-buffer state, keeps frontend-owned viewports and syntax caches by stable buffer ID, and advertises `text`, `styled_text`, and `diagnostics` capabilities. It maps semantic tokens and diagnostics to terminal styling and has no server/protocol logic or paste acceptance policy.
- `ruby`: evaluates configuration and runs isolated plugin hosts. Protocol, deadlines, and process cleanup stay inside this module. - `ruby`: evaluates configuration and runs isolated plugin hosts. Protocol, deadlines, and process cleanup stay inside this module.
@@ -29,7 +29,8 @@ Frontends inspect `count()` for the optional pending motion prefix; parsing and
Frontends inspect `leader_pending()` for the cached display of a pending leader sequence; prefix recognition, cancellation, and dispatch remain editor-owned. Frontends inspect `leader_pending()` for the cached display of a pending leader sequence; prefix recognition, cancellation, and dispatch remain editor-owned.
Frontends inspect `buffer_id()`, `buffer_count()`, `has_buffer(id)`, and the ascending-ID `buffers()` iterator of borrowed path/dirty/active metadata. Existing text, cursor, selection, search, and mode methods describe only the active buffer. Frontends inspect `buffer_id()`, `buffer_count()`, `has_buffer(id)`, and the ascending-ID `buffers()` iterator of borrowed path/dirty/active metadata. Existing text, cursor, selection, search, and mode methods describe only the active buffer.
Frontends inspect `file_picker()` for query, presentation, view, explorer-local INSERT/filter subfocus, selection, and errors, and iterate borrowed `FileRow` metadata (actual path, display label, directory flag, visible depth, expansion). Dock side is a logical preference. They allocate rectangles, add folder suffixes/colors, and render; hierarchy, filtering, parent/child navigation, focus transitions, and selected-file opening remain backend-owned. Frontends inspect `file_picker()` for query, presentation, view, explorer-local INSERT/filter subfocus, selection, and errors, and iterate borrowed `FileRow` metadata (actual path, display label, directory flag, visible depth, expansion). Dock side is a logical preference. They allocate rectangles, add folder suffixes/colors, and render; hierarchy, filtering, parent/child navigation, focus transitions, and selected-file opening remain backend-owned.
Frontends inspect `diagnostics()` and `diagnostic_at_cursor()` for validated active-buffer annotations. Diagnostics are owned by core LSP or a plugin and buffer revision; text changes and owner/server retirement clear them. The TUI underlines annotated text, colors affected gutter numbers by severity, and displays the primary caret's diagnostic message without replacing search/selection backgrounds. Frontends inspect `diagnostics()` and `diagnostic_at_cursor()` for validated active-buffer annotations. Diagnostics are owned by core LSP or a plugin and buffer revision; text changes clear them. Native server retirement removes only that server's contribution, preserving sibling diagnostics. The TUI underlines annotated text, colors affected gutter numbers by severity, and displays the primary caret's diagnostic message without replacing search/selection backgrounds.
Frontends borrow `semantic_tokens()` for sorted, single-line grapheme spans from the active buffer. Immutable `Arc` slices are shared by native LSP and buffer storage; painting does not copy tokens. Text/path changes discard overlays, and native updates validate the whole applicable annotation batch before mutation. The TUI merges semantic styling into its existing glyph pass above syntax foregrounds but below diagnostic/search/selection/caret styling.
Frontends inspect `hover()` for optional read-only content and logical-line/grapheme scroll offsets. Popup request eligibility, dismissal, and scrolling stay editor/session-owned; the TUI clips the panel inside the document workspace without changing the primary cursor or viewport. Frontends inspect `hover()` for optional read-only content and logical-line/grapheme scroll offsets. Popup request eligibility, dismissal, and scrolling stay editor/session-owned; the TUI clips the panel inside the document workspace without changing the primary cursor or viewport.
## Terminal lifecycle and layout ## Terminal lifecycle and layout
@@ -42,7 +43,7 @@ The buffer viewport includes a fixed left gutter with right-aligned hybrid line
## Syntax caches ## Syntax caches
Each buffer has a frontend-owned parser/tree cache. Unchanged revision/path skips comparison and parsing; unchanged text at a later revision also avoids parsing. Actual text changes produce a minimal UTF-8-safe byte edit with newline-aware Tree-sitter Points, edit the prior tree, then parse using that tree for incremental reuse. Grammar changes reset the tree, including TypeScript versus TSX despite their shared display name. Native highlight/local/injection queries resolve capture precedence and scoped definitions into nonoverlapping original-line spans; unchanged injected trees are reused. Buffer switching restores its cache, and closing retires it. Incomplete text and grammar ERROR nodes remain highlightable; genuine language/query/parser failures expose a status error and readable plain text. Full-tree highlight queries still run after incremental parsing—a deliberate ceiling to revisit if measured costly. There is no second highlighting engine or runtime grammar download. Each buffer has a frontend-owned parser/tree cache. Unchanged revision/path skips comparison and parsing; unchanged text at a later revision also avoids parsing. Actual text changes produce a minimal UTF-8-safe byte edit with newline-aware Tree-sitter Points, edit the prior tree, then parse using that tree for incremental reuse. Grammar changes reset the tree, including TypeScript versus TSX despite their shared display name. Native highlight/local/injection queries resolve capture precedence and scoped definitions into nonoverlapping original-line spans; unchanged injected trees are reused. Buffer switching restores its cache, and closing retires it. Incomplete text and grammar ERROR nodes remain highlightable; genuine language/query/parser failures expose a status error and readable plain text. Full-tree highlight queries still run after incremental parsing—a deliberate ceiling to revisit if measured costly. There is no local semantic inference engine or runtime grammar download.
## Clipboard providers ## Clipboard providers
@@ -52,6 +53,10 @@ macOS uses fixed JXA/AppKit `NSPasteboardTypeString` access through `osascript`,
The native module initializes each server, sends `initialization_options` as `initializationOptions`, answers `workspace/configuration` from `settings` (including dotted sections; missing sections return `null`), honors open/change/save capabilities, synchronizes in-memory text including hidden dirty buffers, and sends close events. Incremental servers receive whole-document range replacements. Process/pipe I/O and requests stay on bounded background workers; input and idle diagnostic redraw do not wait for server responses. The native module initializes each server, sends `initialization_options` as `initializationOptions`, answers `workspace/configuration` from `settings` (including dotted sections; missing sections return `null`), honors open/change/save capabilities, synchronizes in-memory text including hidden dirty buffers, and sends close events. Incremental servers receive whole-document range replacements. Process/pipe I/O and requests stay on bounded background workers; input and idle diagnostic redraw do not wait for server responses.
Language configurations can own an ordered server stack. The first matching language wins extension overlap; all its matching servers receive document events, while the first handles hover/definition. The Python preset uses ty then Ruff. LSP merges each worker's latest diagnostic snapshot in assignment order before emitting the unchanged core-owned update. Empty publications/retirement remove only their source; revision/path/root changes and close discard all contributions. Worker snapshots retain identity so queued pre-rename diagnostics cannot attach to a new path with the same text revision. Combined diagnostic message bytes remain bounded to 4 MiB; an overflowing publication retires its worker instead of discarding healthy sibling annotations. Workers are stored in config/root order, avoiding per-poll key cloning/sorting. Shutdown flushes the exit notification and waits for peer termination within the existing bounded grace period before process cleanup.
Semantic tokens belong only to the first assigned server. Initialization advertises standard token types/modifiers, relative encoding, full/range requests, and server refresh support; deltas, overlapping/multiline tokens, and dynamic registration are not advertised. Each buffer has at most one request in flight; changes/refreshes coalesce into the latest snapshot. Full requests are preferred, with whole-document range fallback. A linear UTF-16 decoder validates raw coordinates and legend indices before outward grapheme rounding; overlong ends truncate at EOL, unknown types and invisible spans fall back to syntax. Both wire frames and decoded spans are bounded to 4 MiB. Snapshot identity rejects stale/renamed/closed replies before decoding. Invalid semantic replies clear the overlay and report an error without retiring diagnostics/navigation; primary retirement clears its overlays, and sibling retirement leaves them intact.
## Ruby action catalog ## Ruby action catalog
Ruby builds `config.core` from the Rust action catalog before loading configuration or plugin scripts. Dotted identifiers create nested namespaces automatically; each namespace and Action value is frozen. Action names are not duplicated in Ruby, so Rust catalog additions/renames immediately change the Ruby interface without code generation or handwritten exports. Ruby builds `config.core` from the Rust action catalog before loading configuration or plugin scripts. Dotted identifiers create nested namespaces automatically; each namespace and Action value is frozen. Action names are not duplicated in Ruby, so Rust catalog additions/renames immediately change the Ruby interface without code generation or handwritten exports.
+30 -5
View File
@@ -138,7 +138,7 @@ LSP runs in Rust, independently of Ruby plugin hosts. **Every language server is
Vamp.configure do |config| Vamp.configure do |config|
config.lsp["rust"] = true # rust-analyzer, extensions: ["rs"] config.lsp["rust"] = true # rust-analyzer, extensions: ["rs"]
config.lsp["ruby"] = true # solargraph stdio, extensions: ["rb"] config.lsp["ruby"] = true # solargraph stdio, extensions: ["rb"]
config.lsp["python"] = false # disabled (also the default when absent) config.lsp["python"] = true # ty server + ruff server, extensions: ["py"]
config.lsp["my-language"] = { config.lsp["my-language"] = {
command: ["my-language-server", "--stdio"], command: ["my-language-server", "--stdio"],
@@ -152,13 +152,34 @@ Vamp.configure do |config|
end end
``` ```
Keys are arbitrary nonempty strings, used as the protocol's `languageId`; they need no Tree-sitter grammar or hardcoded language enum. Absent/`false` entries are disabled. `true` enables a built-in preset: `"rust"` uses `rust-analyzer`/`rs`, `"python"` uses `pylsp`/`py`, and `"ruby"` uses `solargraph stdio`/`rb`. Unknown keys need a configuration object with nonempty `command` argv and `extensions` (without leading dots). Objects also explicitly enable servers; known presets supply omitted command/extensions. For example, `config.lsp["rust"] = { command: ["/custom/path/rust-analyzer"] }` overrides the executable while retaining `rs`. Keys are arbitrary nonempty strings, used as the protocol's `languageId`; they need no Tree-sitter grammar or hardcoded language enum. Absent/`false` entries are disabled. `true` enables a built-in preset: `"rust"` uses `rust-analyzer`/`rs`, `"python"` uses `ty server` plus `ruff server`/`py`, and `"ruby"` uses `solargraph stdio`/`rb`. Unknown keys need a configuration object with nonempty `command` argv or a `servers` list, plus `extensions` (without leading dots). Objects also explicitly enable servers; known presets supply omitted command/extensions. An explicit `command` replaces the entire preset with one server: `config.lsp["rust"] = { command: ["/custom/path/rust-analyzer"] }` retains `rs`, while a Python command override does not also launch Ruff.
Only `command`, `extensions`, `initialization_options`, and `settings` are accepted. Both option/settings values must be JSON-compatible objects and default to `{}`. Invalid declarations reject configuration with a notice rather than activating a guessed server. Server commands are executable/argument arrays, never shell commands; use an absolute path if the executable is not on PATH. No server installation or restart is automatic. Only `command`, `extensions`, `initialization_options`, `settings`, and `servers` are accepted. Options/settings must be JSON-compatible objects. Without an explicit command/list, options apply to every default server; explicit `initialization_options` replaces preset defaults. `servers` must be a nonempty ordered list of objects with explicit `command` and optional per-server `initialization_options`/`settings`; it shares language-level extensions and cannot be combined with top-level command/options/settings. Invalid declarations reject configuration with a notice rather than activating a guessed server. Commands are executable/argument arrays, never shell commands; use an absolute path if the executable is not on PATH. No server installation or restart is automatic.
Install Solargraph yourself with `gem install solargraph` in the project's Ruby environment. The Ruby preset launches `["solargraph", "stdio"]`, following [Solargraph's stdio integration](https://solargraph.org/guides/language-server), and uses [YARD annotations](https://solargraph.org/guides/yard) for type information. Explicit `command` overrides replace the entire argv: for a bundled installation, use `config.lsp["ruby"] = { command: ["bundle", "exec", "solargraph", "stdio"] }`; Vamp retains the `rb` extension but does not append `stdio` to your override. Install Solargraph yourself with `gem install solargraph` in the project's Ruby environment. The Ruby preset launches `["solargraph", "stdio"]`, following [Solargraph's stdio integration](https://solargraph.org/guides/language-server), and uses [YARD annotations](https://solargraph.org/guides/yard) for type information. Explicit `command` overrides replace the entire argv: for a bundled installation, use `config.lsp["ruby"] = { command: ["bundle", "exec", "solargraph", "stdio"] }`; Vamp retains the `rb` extension but does not append `stdio` to your override.
Servers start lazily only when a matching buffer opens. Extension matching is case-sensitive; if enabled languages overlap, the first language in sorted key order wins. One worker runs per language configuration/project root, with at most 16 enabled languages/server-project keys. Start vamp from your project root: its startup working directory is the workspace; automatic root discovery is not implemented. **Python (Astral):** install [ty](https://docs.astral.sh/ty/editors/) and [Ruff](https://docs.astral.sh/ruff/editors/setup/) yourself, for example with `uv tool install ty` and `uv tool install ruff`. The Python preset sends the same documents to both servers, uses ty for hover/definition, and merges type errors with Ruff lint diagnostics. Clearing or losing one server preserves the other's diagnostics. Formatting and code actions remain unsupported by Vamp.
Use `servers` to customize the stack independently; order selects the hover/definition server:
```ruby
config.lsp["python"] = {
servers: [
{
command: ["ty", "server"],
initialization_options: { experimental: { useUv: "off" } }
},
{
command: ["ruff", "server"],
initialization_options: { settings: { lint: { extendSelect: ["I"] } } }
}
]
}
```
**uv remains optional project tooling.** Run project setup/synchronization yourself; [ty discovers an existing `.venv`](https://docs.astral.sh/ty/reference/configuration/#python) without a uv launcher. Vamp neither runs `uv sync` nor changes dependencies/lockfiles. The default ty preset sets `experimental.useUv: "off"` so inherited `TY_UV` cannot enable ty's [experimental environment-management integration](https://docs.astral.sh/ty/reference/editor-settings/). Ruff's initialization options default to `{}`. When replacing ty's initialization options, retain that flag if you want the same policy.
Servers start lazily only when a matching buffer opens. Extension matching is case-sensitive; if enabled languages overlap, the first language in sorted key order wins. All matching servers in that language run, and the first handles hover/definition, semantic highlighting, and the diagnostic-status command without fallback to sibling servers. Each server/project root gets one worker, with at most 16 configured servers and 16 server/project keys; Python uses two slots. Start vamp from your project root: its startup working directory is the workspace; automatic root discovery is not implemented.
See [native LSP architecture](architecture.md#native-lsp) for synchronization and worker behavior. See [native LSP architecture](architecture.md#native-lsp) for synchronization and worker behavior.
@@ -174,4 +195,8 @@ Diagnostics appear as severity-colored gutter numbers and text underlines; the p
Hover accepts plaintext/Markdown MarkupContent and string/code MarkedString forms, including arrays. Multiline content is preserved; Markdown is shown as plain text, not rendered markup. Horizontal scrolling exposes long lines without splitting graphemes. Missing/unsupported/invalid hover results report a message without changing text, selections, or undo history. Content is bounded to 4 MiB and terminal controls are sanitized. Hover accepts plaintext/Markdown MarkupContent and string/code MarkedString forms, including arrays. Multiline content is preserved; Markdown is shown as plain text, not rendered markup. Horizontal scrolling exposes long lines without splitting graphemes. Missing/unsupported/invalid hover results report a message without changing text, selections, or undo history. Content is bounded to 4 MiB and terminal controls are sanitized.
Native LSP currently provides synchronization, diagnostics, go-to-definition, and hover—not completion, rename, formatting, code actions, workspace edits, or semantic highlighting. Tree-sitter remains the syntax highlighter. Semantic highlighting is **enabled automatically** when the first server advertises [LSP semantic tokens](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_semanticTokens). No additional configuration or binding is needed. Full-document requests are preferred; range-only servers receive a whole-document range request. Token colors override Tree-sitter where recognized; unsupported servers, unknown custom token types, missing tokens, and failed replies retain syntax highlighting. Text/path changes clear old semantic colors immediately, and stale replies cannot restore them. Same-text refreshes keep the current overlay until the replacement arrives.
rust-analyzer and ty support semantic tokens. Solargraph 0.61.0 and Ruff do not; Ruby therefore retains Tree-sitter colors with the Solargraph preset, while Python uses ty's tokens and Ruff's diagnostics. Declaration/definition modifiers use bold, deprecated symbols use strikethrough, and readonly variables/properties use yellow. Diagnostics, search, selections, and carets retain their existing styling precedence.
Native LSP currently provides synchronization, diagnostics, go-to-definition, hover, and semantic highlighting—not completion, rename, formatting, code actions, or workspace edits. Semantic token deltas, dynamic registration, and partial-result streaming are not implemented.
+2 -1
View File
@@ -35,8 +35,9 @@ Rust tests live alongside their modules; native LSP integration checks are in [`
- Editor checks cover Unicode editing, multi-target state, undo/redo, saving failures, and modal paste acceptance and rejection atomicity. - Editor checks cover Unicode editing, multi-target state, undo/redo, saving failures, and modal paste acceptance and rejection atomicity.
- Session and Ruby checks exercise real Ruby execution, command/capability ownership, effect validation, configuration fallback, plugin retirement, crash/timeout recovery, and inherited-pipe cleanup. - Session and Ruby checks exercise real Ruby execution, command/capability ownership, effect validation, configuration fallback, plugin retirement, crash/timeout recovery, and inherited-pipe cleanup.
- Unix terminal-hook checks use isolated child processes and real PTYs to keep process-global hooks out of the parallel runner. They cover cleanup on entry failure, normal return, I/O errors, and panics, including original panic-payload propagation. - Unix terminal-hook checks use isolated child processes and real PTYs to keep process-global hooks out of the parallel runner. They cover cleanup on entry failure, normal return, I/O errors, and panics, including original panic-payload propagation.
- Native LSP integration checks launch framed-protocol fixture subprocesses, covering initialization/settings, opt-in startup, synchronization, UTF-16/file-URI conversion, dirty targets, stale requests/diagnostics, and transport failures. - Native LSP integration checks launch framed-protocol fixture subprocesses, covering initialization/settings, opt-in startup, synchronization, UTF-16/file-URI conversion, dirty targets, stale requests/diagnostics, and transport failures. Stack checks cover independent initialization, diagnostic merging/clearing, sibling failure isolation in active/parked buffers, same-revision rename/close identity, aggregate limits, first-server position routing, and both shutdowns. Semantic ordering uses protocol/file gates; elapsed deadlines only bound stuck subprocesses.
- Hover regression checks cover content forms, sanitization, cancelled/superseded replies, invalid results, scrolling/dismissal, Unicode clipping, and preserved editing state. - Hover regression checks cover content forms, sanitization, cancelled/superseded replies, invalid results, scrolling/dismissal, Unicode clipping, and preserved editing state.
- Semantic-token checks cover full/range requests, legend/modifier remapping, UTF-16 boundaries and grapheme rounding, refresh/change coalescing, stale rename/close replies, null/empty/error fallback, primary/sibling retirement, and rendered cell precedence with syntax, diagnostics, search, selections, and carets.
Clipboard checks use isolated providers rather than reading or overwriting the desktop clipboard. Clipboard checks use isolated providers rather than reading or overwriting the desktop clipboard.
+4
View File
@@ -175,3 +175,7 @@ Space begins a leader sequence in Normal/Visual by default. Press the suffix key
Buffers highlight automatically using [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) and compiled-in grammar/query packages. The selected language set is Rust, Ruby, Python, JSON, Bash, Makefile, Markdown, JavaScript/JSX, TypeScript/TSX, HTML, CSS, YAML, TOML, C, C++, and Go. Detection uses filename/extension or the in-memory first-line shebang, including common special filenames such as Makefile, Gemfile, and Cargo.lock; no file reread occurs. Unsupported formats use the normal foreground. The detected language appears in status, and save-as redetects it. Buffers highlight automatically using [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) and compiled-in grammar/query packages. The selected language set is Rust, Ruby, Python, JSON, Bash, Makefile, Markdown, JavaScript/JSX, TypeScript/TSX, HTML, CSS, YAML, TOML, C, C++, and Go. Detection uses filename/extension or the in-memory first-line shebang, including common special filenames such as Makefile, Gemfile, and Cargo.lock; no file reread occurs. Unsupported formats use the normal foreground. The detected language appears in status, and save-as redetects it.
Colors retain the terminal palette: comments dark gray, strings green, keywords magenta, numbers yellow, types/markup cyan, and functions light blue. Grammar queries handle multiline/nested constructs, even when their opening is above the viewport. Markdown uses its inline grammar and named fenced-code injections; HTML script/style content uses JavaScript/CSS. Known tagged-template and raw-string injections use the same selected grammar registry; unknown injected languages retain host highlighting or plain text. Highlighting styles whole graphemes and expanded tab cells; partially clipped wide placeholders remain unchanged. Search, Visual selections, and secondary carets retain foreground/background precedence. Colors retain the terminal palette: comments dark gray, strings green, keywords magenta, numbers yellow, types/markup cyan, and functions light blue. Grammar queries handle multiline/nested constructs, even when their opening is above the viewport. Markdown uses its inline grammar and named fenced-code injections; HTML script/style content uses JavaScript/CSS. Known tagged-template and raw-string injections use the same selected grammar registry; unknown injected languages retain host highlighting or plain text. Highlighting styles whole graphemes and expanded tab cells; partially clipped wide placeholders remain unchanged. Search, Visual selections, and secondary carets retain foreground/background precedence.
Markdown heading markers (`#` through `######`, or Setext underlines) share the heading text's markup color. Plain-text hashes and other punctuation keep their existing styling.
An enabled supporting LSP server supplies semantic tokens automatically, overriding syntax colors only on recognized spans. Variables/parameters use light cyan, properties blue, and readonly variables/properties yellow; declarations/definitions are bold, documentation comments italic, and deprecated symbols struck through. Tree-sitter fills gaps and remains available while new tokens are pending after edits. Search, Visual selections, diagnostics, and carets keep their existing precedence. See [native LSP configuration](configuration.md#native-lsp) for provider support and server-stack routing.
+1
View File
@@ -13,6 +13,7 @@ Vamp.configure do |config|
# Native LSP is disabled by default; install a server and explicitly opt in. # Native LSP is disabled by default; install a server and explicitly opt in.
# config.lsp["rust"] = true # built-in rust-analyzer preset # config.lsp["rust"] = true # built-in rust-analyzer preset
# config.lsp["ruby"] = true # built-in Solargraph stdio preset (gem install solargraph) # config.lsp["ruby"] = true # built-in Solargraph stdio preset (gem install solargraph)
# config.lsp["python"] = true # built-in ty + Ruff stack (install both separately)
# config.lsp["my-language"] = { # config.lsp["my-language"] = {
# command: ["my-language-server", "--stdio"], extensions: ["myl"], # command: ["my-language-server", "--stdio"], extensions: ["myl"],
# initialization_options: {}, settings: {} # initialization_options: {}, settings: {}
+252 -1
View File
@@ -1,6 +1,6 @@
use crate::clipboard::{ClipboardMode, SystemClipboard}; use crate::clipboard::{ClipboardMode, SystemClipboard};
use crate::files::{DockSide, FilePicker, FilePresentation, FileView}; use crate::files::{DockSide, FilePicker, FilePresentation, FileView};
use crate::plugin::{Diagnostic, TextPosition}; use crate::plugin::{Diagnostic, SemanticToken, TextPosition};
use serde::Serialize; use serde::Serialize;
use std::borrow::Cow; use std::borrow::Cow;
use std::collections::{BTreeMap, HashMap, HashSet}; use std::collections::{BTreeMap, HashMap, HashSet};
@@ -10,6 +10,7 @@ use std::io::{self, Write};
#[cfg(unix)] #[cfg(unix)]
use std::os::unix::fs::OpenOptionsExt; use std::os::unix::fs::OpenOptionsExt;
use std::path::{Component, Path, PathBuf}; use std::path::{Component, Path, PathBuf};
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::atomic::{AtomicU64, Ordering};
use unicode_segmentation::{GraphemeCursor, UnicodeSegmentation}; use unicode_segmentation::{GraphemeCursor, UnicodeSegmentation};
use unicode_width::UnicodeWidthStr; use unicode_width::UnicodeWidthStr;
@@ -773,6 +774,7 @@ struct Buffer {
text_revision: u64, text_revision: u64,
save_revision: u64, save_revision: u64,
diagnostics: BTreeMap<String, AnnotationSet>, diagnostics: BTreeMap<String, AnnotationSet>,
semantic_tokens: Option<Arc<[SemanticToken]>>,
targets: Vec<Target>, targets: Vec<Target>,
primary: usize, primary: usize,
mode: Mode, mode: Mode,
@@ -814,6 +816,7 @@ impl Buffer {
text_revision: 0, text_revision: 0,
save_revision: 0, save_revision: 0,
diagnostics: BTreeMap::new(), diagnostics: BTreeMap::new(),
semantic_tokens: None,
targets: vec![Target::caret(Cursor::default())], targets: vec![Target::caret(Cursor::default())],
primary: 0, primary: 0,
mode: Mode::Normal, mode: Mode::Normal,
@@ -855,6 +858,7 @@ impl Buffer {
fn text_changed(&mut self) { fn text_changed(&mut self) {
self.text_revision = self.text_revision.wrapping_add(1); self.text_revision = self.text_revision.wrapping_add(1);
self.diagnostics.clear(); self.diagnostics.clear();
self.semantic_tokens = None;
} }
} }
@@ -1074,6 +1078,101 @@ impl Editor {
} }
} }
pub fn semantic_tokens(&self) -> &[SemanticToken] {
self.active.semantic_tokens.as_deref().unwrap_or_default()
}
pub(crate) fn validate_semantic_tokens(
&self,
buffer_id: usize,
revision: u64,
tokens: Option<&[SemanticToken]>,
) -> Result<bool, String> {
let Some(buffer) = self.buffer(buffer_id) else {
return Ok(false);
};
if buffer.text_revision != revision {
return Ok(false);
}
let Some(tokens) = tokens else {
return Ok(true);
};
if tokens.len() > MAX_EDIT_BYTES / std::mem::size_of::<SemanticToken>() {
return Err("Semantic tokens exceed 4 MiB".into());
}
let mut previous = None;
let mut row_length = None;
for token in tokens {
if previous.is_some_and(|start| token.start < start) {
return Err("Semantic token starts must be sorted".into());
}
if token.start.row != token.end.row || token.start >= token.end {
return Err("Semantic tokens must have nonempty single-line ranges".into());
}
if token.modifiers & !1023 != 0 {
return Err("Semantic token modifiers are not canonical".into());
}
if row_length.is_none_or(|(row, _)| row != token.start.row) {
let line = buffer
.lines
.get(token.start.row)
.ok_or("Semantic token position is outside the document")?;
row_length = Some((token.start.row, grapheme_count(line)));
}
if token.end.column > row_length.unwrap().1 {
return Err("Semantic token position is outside the document".into());
}
previous = Some(token.start);
}
Ok(true)
}
pub fn replace_semantic_tokens(
&mut self,
buffer_id: usize,
revision: u64,
tokens: Option<Arc<[SemanticToken]>>,
) -> Result<bool, String> {
if !self.validate_semantic_tokens(buffer_id, revision, tokens.as_deref())? {
return Ok(false);
}
Ok(self.replace_validated_semantic_tokens(buffer_id, revision, tokens))
}
/// Only session's fully prevalidated batch may use this storage path.
pub(crate) fn replace_validated_semantic_tokens(
&mut self,
buffer_id: usize,
revision: u64,
tokens: Option<Arc<[SemanticToken]>>,
) -> bool {
let buffer = if buffer_id == self.active.id {
&mut self.active
} else if let Some(buffer) = self.parked.get_mut(&buffer_id) {
buffer
} else {
return false;
};
if buffer.text_revision != revision {
return false;
}
let tokens = tokens.filter(|tokens| !tokens.is_empty());
match (&buffer.semantic_tokens, &tokens) {
(None, None) => return false,
(Some(current), Some(next)) if Arc::ptr_eq(current, next) => return false,
_ => {}
}
buffer.semantic_tokens = tokens;
true
}
pub fn clear_semantic_tokens(&mut self) {
self.active.semantic_tokens = None;
for buffer in self.parked.values_mut() {
buffer.semantic_tokens = None;
}
}
pub fn diagnostics(&self) -> impl Iterator<Item = &Diagnostic> { pub fn diagnostics(&self) -> impl Iterator<Item = &Diagnostic> {
self.active self.active
.diagnostics .diagnostics
@@ -2754,6 +2853,9 @@ impl Editor {
Ok(()) => { Ok(()) => {
self.active.saved_lines.clone_from(&self.active.lines); self.active.saved_lines.clone_from(&self.active.lines);
self.active.save_revision = self.active.save_revision.wrapping_add(1); self.active.save_revision = self.active.save_revision.wrapping_add(1);
if self.active.path.as_ref() != Some(&path) {
self.active.semantic_tokens = None;
}
self.active.path = Some(path); self.active.path = Some(path);
self.active.identity = Some(identity); self.active.identity = Some(identity);
self.set_message("Written"); self.set_message("Written");
@@ -4316,6 +4418,155 @@ mod tests {
editor editor
} }
#[test]
fn semantic_snapshots_are_borrowed_buffer_local_and_invalidated_by_text_and_path() {
use crate::plugin::SemanticTokenKind;
let directory = Directory::new();
let a = directory.path("a.py");
let b = directory.path("b.py");
fs::write(&a, "e\u{301}界").unwrap();
fs::write(&b, "other").unwrap();
let mut editor = Editor::open(Some(a.clone()), vec![]).unwrap();
editor.set_clipboard_mode("separate").unwrap();
let tokens: Arc<[SemanticToken]> = Arc::from([SemanticToken {
start: TextPosition { row: 0, column: 0 },
end: TextPosition { row: 0, column: 2 },
kind: SemanticTokenKind::Variable,
modifiers: 1023,
}]);
assert!(
editor
.replace_semantic_tokens(1, 0, Some(tokens.clone()))
.unwrap()
);
assert!(std::ptr::eq(editor.semantic_tokens(), tokens.as_ref()));
assert!(!editor.dirty());
assert!(editor.active.undo.is_empty());
editor.execute(&format!("edit {}", b.display()));
assert!(editor.semantic_tokens().is_empty());
assert!(
editor
.replace_semantic_tokens(2, 0, Some(tokens.clone()))
.unwrap()
);
editor.execute("buffer 1");
assert!(std::ptr::eq(editor.semantic_tokens(), tokens.as_ref()));
assert!(
!editor
.replace_semantic_tokens(1, 0, Some(tokens.clone()))
.unwrap()
);
editor.execute("w");
assert_eq!(editor.semantic_tokens(), tokens.as_ref());
editor.execute(&format!("w {}", directory.path("renamed.py").display()));
assert_eq!(editor.text_revision(), 0);
assert!(editor.semantic_tokens().is_empty());
assert!(
editor
.replace_semantic_tokens(1, 0, Some(tokens.clone()))
.unwrap()
);
editor.handle(Input::Char('x'));
assert!(editor.semantic_tokens().is_empty());
editor.handle(Input::Char('u'));
assert_eq!(editor.lines(), ["e\u{301}界"]);
assert!(editor.semantic_tokens().is_empty());
assert!(
!editor
.replace_semantic_tokens(1, 0, Some(tokens.clone()))
.unwrap()
);
editor.execute("buffer 2");
assert_eq!(editor.semantic_tokens(), tokens.as_ref());
editor.clear_semantic_tokens();
assert!(editor.semantic_tokens().is_empty());
editor.execute("buffer 1");
assert!(editor.semantic_tokens().is_empty());
}
#[test]
fn semantic_validation_rejects_malformed_live_tokens_but_ignores_stale_and_unloaded() {
use crate::plugin::SemanticTokenKind;
let mut editor = editor();
editor
.apply(vec![Effect::Replace("e\u{301}界\nx".into())])
.unwrap();
let revision = editor.text_revision();
let token = SemanticToken {
start: TextPosition { row: 0, column: 0 },
end: TextPosition { row: 0, column: 2 },
kind: SemanticTokenKind::Variable,
modifiers: 0,
};
let invalid = [
SemanticToken {
end: TextPosition { row: 0, column: 3 },
..token
},
SemanticToken {
end: token.start,
..token
},
SemanticToken {
end: TextPosition { row: 1, column: 1 },
..token
},
SemanticToken {
start: TextPosition { row: 9, column: 0 },
end: TextPosition { row: 9, column: 1 },
..token
},
SemanticToken {
modifiers: 1024,
..token
},
];
for token in invalid {
assert!(
editor
.validate_semantic_tokens(1, revision, Some(&[token]))
.is_err()
);
assert!(
!editor
.validate_semantic_tokens(1, revision - 1, Some(&[token]))
.unwrap()
);
assert!(
!editor
.validate_semantic_tokens(999, revision, Some(&[token]))
.unwrap()
);
}
let later = SemanticToken {
start: TextPosition { row: 0, column: 1 },
..token
};
assert!(
editor
.validate_semantic_tokens(1, revision, Some(&[later, token]))
.is_err()
);
// Raw adjacent LSP tokens may map outward to the same grapheme.
assert!(
editor
.validate_semantic_tokens(1, revision, Some(&[token, token, later]))
.unwrap()
);
let oversized = vec![token; MAX_EDIT_BYTES / std::mem::size_of::<SemanticToken>() + 1];
assert!(
editor
.validate_semantic_tokens(1, revision, Some(&oversized))
.is_err()
);
let tokens: Arc<[SemanticToken]> = Arc::from([token]);
editor
.replace_semantic_tokens(1, revision, Some(tokens))
.unwrap();
assert!(editor.replace_semantic_tokens(1, revision, None).unwrap());
assert!(editor.semantic_tokens().is_empty());
}
#[test] #[test]
fn hover_scroll_and_escape_preserve_carets_and_edits_dismiss_normally() { fn hover_scroll_and_escape_preserve_carets_and_edits_dismiss_normally() {
let mut editor = editor(); let mut editor = editor();
+727 -75
View File
File diff suppressed because it is too large Load Diff
+107 -30
View File
@@ -10,10 +10,10 @@ pub fn resolve(value: Value) -> Result<Vec<ServerConfig>, String> {
if language_id.is_empty() || language_id.chars().any(char::is_control) { if language_id.is_empty() || language_id.chars().any(char::is_control) {
return Err("LSP language keys must not be empty or contain control characters".into()); return Err("LSP language keys must not be empty or contain control characters".into());
} }
let preset: Option<(&[&str], &str)> = match language_id.as_str() { let preset: Option<(&[&[&str]], &str)> = match language_id.as_str() {
"rust" => Some((&["rust-analyzer"], "rs")), "rust" => Some((&[&["rust-analyzer"]], "rs")),
"python" => Some((&["pylsp"], "py")), "python" => Some((&[&["ty", "server"], &["ruff", "server"]], "py")),
"ruby" => Some((&["solargraph", "stdio"], "rb")), "ruby" => Some((&[&["solargraph", "stdio"]], "rb")),
_ => None, _ => None,
}; };
let mut options = match value { let mut options = match value {
@@ -36,55 +36,134 @@ pub fn resolve(value: Value) -> Result<Vec<ServerConfig>, String> {
for key in options.keys() { for key in options.keys() {
if !matches!( if !matches!(
key.as_str(), key.as_str(),
"command" | "extensions" | "initialization_options" | "settings" "command" | "extensions" | "initialization_options" | "settings" | "servers"
) { ) {
return Err(format!( return Err(format!(
"LSP language {language_id:?} has unknown option {key:?}" "LSP language {language_id:?} has unknown option {key:?}"
)); ));
} }
} }
let command = match options.remove("command") { let mut extensions = match options.remove("extensions") {
Some(value) => command(value, &language_id)?,
None => match preset {
Some((command, _)) => command.iter().map(|argument| (*argument).into()).collect(),
None => {
return Err(format!(
"LSP language {language_id:?} requires command argv"
));
}
},
};
let extensions = match options.remove("extensions") {
Some(value) => extensions(value, &language_id)?, Some(value) => extensions(value, &language_id)?,
None => match preset { None => match preset {
Some((_, extension)) => vec![extension.into()], Some((_, extension)) => vec![extension.into()],
None => return Err(format!("LSP language {language_id:?} requires extensions")), None => return Err(format!("LSP language {language_id:?} requires extensions")),
}, },
}; };
let explicit_servers = options.contains_key("servers");
let configurations = match options.remove("servers") {
Some(value) => {
if ["command", "initialization_options", "settings"]
.iter()
.any(|key| options.contains_key(*key))
{
return Err(format!(
"LSP language {language_id:?} servers cannot be combined with command, initialization_options, or settings"
));
}
let Value::Array(configurations) = value else {
return Err(format!(
"LSP language {language_id:?} servers must be a nonempty array of configuration objects"
));
};
if configurations.is_empty() {
return Err(format!(
"LSP language {language_id:?} servers must be a nonempty array of configuration objects"
));
}
configurations
}
None if options.contains_key("command") => vec![Value::Object(options)],
None => {
let Some((commands, _)) = preset else {
return Err(format!(
"LSP language {language_id:?} requires command argv or servers"
));
};
let mut configurations = Vec::with_capacity(commands.len());
for (index, command) in commands.iter().enumerate() {
let mut configuration = if index + 1 == commands.len() {
std::mem::take(&mut options)
} else {
options.clone()
};
configuration.insert(
"command".into(),
Value::Array(
command
.iter()
.map(|argument| Value::from(*argument))
.collect(),
),
);
if language_id == "python" && index == 0 {
configuration
.entry("initialization_options")
.or_insert_with(
|| serde_json::json!({"experimental": {"useUv": "off"}}),
);
}
configurations.push(Value::Object(configuration));
}
configurations
}
};
if servers.len() + configurations.len() > 16 {
return Err("LSP configuration supports at most 16 configured servers".into());
}
let configuration_count = configurations.len();
let mut language_id = language_id;
for (index, configuration) in configurations.into_iter().enumerate() {
let context = if explicit_servers {
format!("LSP language {language_id:?} servers[{index}]")
} else {
format!("LSP language {language_id:?}")
};
let Value::Object(mut configuration) = configuration else {
return Err(format!("{context} must be a configuration object"));
};
for key in configuration.keys() {
if !matches!(
key.as_str(),
"command" | "initialization_options" | "settings"
) {
return Err(format!("{context} has unknown option {key:?}"));
}
}
let command = match configuration.remove("command") {
Some(value) => command(value, &context)?,
None => return Err(format!("{context} requires command argv")),
};
let initialization_options = object( let initialization_options = object(
options.remove("initialization_options"), configuration.remove("initialization_options"),
&language_id, &context,
"initialization_options", "initialization_options",
)?; )?;
let settings = object(options.remove("settings"), &language_id, "settings")?; let settings = object(configuration.remove("settings"), &context, "settings")?;
servers.push(ServerConfig { servers.push(ServerConfig {
language_id, language_id: if index + 1 == configuration_count {
std::mem::take(&mut language_id)
} else {
language_id.clone()
},
command, command,
extensions, extensions: if index + 1 == configuration_count {
std::mem::take(&mut extensions)
} else {
extensions.clone()
},
initialization_options, initialization_options,
settings, settings,
}); });
if servers.len() > 16 {
return Err("LSP configuration supports at most 16 enabled languages".into());
} }
} }
Ok(servers) Ok(servers)
} }
fn command(value: Value, language: &str) -> Result<Vec<String>, String> { fn command(value: Value, context: &str) -> Result<Vec<String>, String> {
let error = || { let error = || {
format!( format!(
"LSP language {language:?} command must be nonempty argv with a nonempty executable and no NUL bytes" "{context} command must be nonempty argv with a nonempty executable and no NUL bytes"
) )
}; };
let Value::Array(values) = value else { let Value::Array(values) = value else {
@@ -137,12 +216,10 @@ fn extensions(value: Value, language: &str) -> Result<Vec<String>, String> {
Ok(extensions) Ok(extensions)
} }
fn object(value: Option<Value>, language: &str, option: &str) -> Result<Value, String> { fn object(value: Option<Value>, context: &str, option: &str) -> Result<Value, String> {
match value { match value {
None => Ok(Value::Object(serde_json::Map::new())), None => Ok(Value::Object(serde_json::Map::new())),
Some(value) if value.is_object() => Ok(value), Some(value) if value.is_object() => Ok(value),
Some(_) => Err(format!( Some(_) => Err(format!("{context} {option} must be a JSON object")),
"LSP language {language:?} {option} must be a JSON object"
)),
} }
} }
+3 -1
View File
@@ -37,9 +37,11 @@ Ruby config: $XDG_CONFIG_HOME/vamp/config.rb or ~/.config/vamp/config.rb
Set VAMP_RUBY to choose the Ruby executable. Set VAMP_RUBY to choose the Ruby executable.
Native LSP is disabled by default: config.lsp[\"rust\"] = true enables a preset Native LSP is disabled by default: config.lsp[\"rust\"] = true enables a preset
Presets: rust -> rust-analyzer, python -> pylsp, ruby -> solargraph stdio Presets: rust -> rust-analyzer, python -> ty server + ruff server, ruby -> solargraph stdio
Custom languages: config.lsp[\"my-language\"] = { command: [...], extensions: [...] } Custom languages: config.lsp[\"my-language\"] = { command: [...], extensions: [...] }
Server stacks: config.lsp[\"python\"] = { servers: [{ command: [...] }, { command: [...] }] }
LSP commands: :lsp.definition, :lsp.diagnostics, :lsp.hover LSP commands: :lsp.definition, :lsp.diagnostics, :lsp.hover
Semantic highlighting: automatic when the first enabled server supports semantic tokens
Hover binding: config.bind \"K\", config.core.lsp.hover (no default key)"; Hover binding: config.bind \"K\", config.core.lsp.hover (no default key)";
struct Options { struct Options {
+61 -1
View File
@@ -1,4 +1,4 @@
use std::path::PathBuf; use std::{path::PathBuf, sync::Arc};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -52,6 +52,59 @@ pub struct Diagnostic {
pub message: String, pub message: String,
} }
/// Standard LSP semantic token categories, independent of frontend colors.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum SemanticTokenKind {
Namespace,
Type,
Class,
Enum,
Interface,
Struct,
TypeParameter,
Parameter,
Variable,
Property,
EnumMember,
Event,
Function,
Method,
Macro,
Keyword,
Modifier,
Comment,
String,
Number,
Regexp,
Operator,
Decorator,
}
/// Canonical modifier bits; server legend bit positions are translated by LSP.
#[repr(u16)]
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum SemanticTokenModifier {
Declaration = 1,
Definition = 1 << 1,
Readonly = 1 << 2,
Static = 1 << 3,
Deprecated = 1 << 4,
Abstract = 1 << 5,
Async = 1 << 6,
Modification = 1 << 7,
Documentation = 1 << 8,
DefaultLibrary = 1 << 9,
}
/// A single-line foreground annotation using whole extended-grapheme positions.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct SemanticToken {
pub start: TextPosition,
pub end: TextPosition,
pub kind: SemanticTokenKind,
pub modifiers: u16,
}
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)] #[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]
pub enum UpdateEffect { pub enum UpdateEffect {
@@ -60,6 +113,13 @@ pub enum UpdateEffect {
revision: u64, revision: u64,
items: Vec<Diagnostic>, items: Vec<Diagnostic>,
}, },
#[serde(skip_deserializing)]
SemanticTokens {
buffer_id: usize,
revision: u64,
/// None clears the overlay. Immutable slices are shared without copying.
tokens: Option<Arc<[SemanticToken]>>,
},
Navigate { Navigate {
buffer_id: usize, buffer_id: usize,
revision: u64, revision: u64,
+226 -5
View File
@@ -981,9 +981,9 @@ end"#,
end"#, end"#,
); );
let config = load_config(&path).unwrap(); let config = load_config(&path).unwrap();
assert_eq!(config.lsp.len(), 4);
for (language, command, extension) in [ for (language, command, extension) in [
("rust", &["rust-analyzer"][..], "rs"), ("rust", &["rust-analyzer"][..], "rs"),
("python", &["pylsp"][..], "py"),
("ruby", &["solargraph", "stdio"][..], "rb"), ("ruby", &["solargraph", "stdio"][..], "rb"),
] { ] {
let server = config let server = config
@@ -996,6 +996,25 @@ end"#,
assert_eq!(server.initialization_options, serde_json::json!({})); assert_eq!(server.initialization_options, serde_json::json!({}));
assert_eq!(server.settings, serde_json::json!({})); assert_eq!(server.settings, serde_json::json!({}));
} }
let python: Vec<_> = config
.lsp
.iter()
.filter(|server| server.language_id == "python")
.collect();
assert_eq!(python.len(), 2);
for (server, command, initialization_options) in [
(
python[0],
["ty", "server"],
serde_json::json!({"experimental": {"useUv": "off"}}),
),
(python[1], ["ruff", "server"], serde_json::json!({})),
] {
assert_eq!(server.command, command);
assert_eq!(server.extensions, ["py"]);
assert_eq!(server.initialization_options, initialization_options);
assert_eq!(server.settings, serde_json::json!({}));
}
let path = fixture.file( let path = fixture.file(
"config.rb", "config.rb",
@@ -1005,7 +1024,7 @@ end"#,
initialization_options: {cargo: {features: ["α"]}}, initialization_options: {cargo: {features: ["α"]}},
settings: {"rust-analyzer" => {check: {command: "clippy"}}} settings: {"rust-analyzer" => {check: {command: "clippy"}}}
} }
config.lsp["python"] = {command: ["custom pylsp", "--stdio"]} config.lsp["python"] = {command: ["custom-python-server", "--stdio"]}
config.lsp["ruby"] = {command: ["bundle", "exec", "solargraph", "stdio"]} config.lsp["ruby"] = {command: ["bundle", "exec", "solargraph", "stdio"]}
config.lsp["my-language.c++"] = { config.lsp["my-language.c++"] = {
command: ["custom server", "--flag", ""], command: ["custom server", "--flag", ""],
@@ -1035,10 +1054,18 @@ end"#,
.iter() .iter()
.find(|server| server.language_id == "python") .find(|server| server.language_id == "python")
.unwrap(); .unwrap();
assert_eq!(python.command, ["custom pylsp", "--stdio"]); assert_eq!(python.command, ["custom-python-server", "--stdio"]);
assert_eq!(python.extensions, ["py"]); assert_eq!(python.extensions, ["py"]);
assert_eq!(python.initialization_options, serde_json::json!({})); assert_eq!(python.initialization_options, serde_json::json!({}));
assert_eq!(python.settings, serde_json::json!({})); assert_eq!(python.settings, serde_json::json!({}));
assert_eq!(
config
.lsp
.iter()
.filter(|server| server.language_id == "python")
.count(),
1
);
let ruby = config let ruby = config
.lsp .lsp
.iter() .iter()
@@ -1060,6 +1087,182 @@ end"#,
); );
} }
#[test]
fn lsp_python_common_options_and_ordered_server_overrides() {
let fixture = Fixture::new();
let path = fixture.file(
"config.rb",
r#"Vamp.configure do |config|
config.lsp["python"] = {
extensions: ["py", "pyi"],
initialization_options: {diagnosticMode: "workspace"},
settings: {shared: true}
}
end"#,
);
let config = load_config(&path).unwrap();
assert_eq!(config.lsp.len(), 2);
for (server, command) in config
.lsp
.iter()
.zip([["ty", "server"], ["ruff", "server"]])
{
assert_eq!(server.language_id, "python");
assert_eq!(server.command, command);
assert_eq!(server.extensions, ["py", "pyi"]);
assert_eq!(
server.initialization_options,
serde_json::json!({"diagnosticMode": "workspace"})
);
assert_eq!(server.settings, serde_json::json!({"shared": true}));
}
let path = fixture.file(
"config.rb",
r#"Vamp.configure do |config|
config.lsp["python"] = {servers: [
{command: ["type-server", "--stdio"], initialization_options: {experimental: {useUv: "off"}}, settings: {first: true}},
{command: ["lint-server"], initialization_options: {settings: {lint: {enable: true}}}}
]}
config.lsp["custom"] = {extensions: ["custom"], servers: [
{command: ["custom-first"]}, {command: ["custom-second"], settings: {second: true}}
]}
end"#,
);
let config = load_config(&path).unwrap();
for (language, commands, extensions) in [
(
"python",
[&["type-server", "--stdio"][..], &["lint-server"][..]],
["py"],
),
(
"custom",
[&["custom-first"][..], &["custom-second"][..]],
["custom"],
),
] {
let servers: Vec<_> = config
.lsp
.iter()
.filter(|server| server.language_id == language)
.collect();
assert_eq!(servers.len(), 2);
for (server, command) in servers.iter().zip(commands) {
assert_eq!(server.command, command);
assert_eq!(server.extensions, extensions);
}
}
let python: Vec<_> = config
.lsp
.iter()
.filter(|server| server.language_id == "python")
.collect();
assert_eq!(
python[0].initialization_options,
serde_json::json!({"experimental": {"useUv": "off"}})
);
assert_eq!(python[0].settings, serde_json::json!({"first": true}));
assert_eq!(
python[1].initialization_options,
serde_json::json!({"settings": {"lint": {"enable": true}}})
);
assert_eq!(python[1].settings, serde_json::json!({}));
let custom: Vec<_> = config
.lsp
.iter()
.filter(|server| server.language_id == "custom")
.collect();
assert_eq!(custom[0].initialization_options, serde_json::json!({}));
assert_eq!(custom[1].settings, serde_json::json!({"second": true}));
let path = fixture.file(
"config.rb",
r#"Vamp.configure { |config| config.lsp["python"] = {settings: {shared: true}} }"#,
);
let config = load_config(&path).unwrap();
assert_eq!(
config.lsp[0].initialization_options,
serde_json::json!({"experimental": {"useUv": "off"}})
);
assert_eq!(config.lsp[1].initialization_options, serde_json::json!({}));
}
#[test]
fn lsp_server_lists_reject_invalid_shapes_and_ambiguous_options_locally() {
let fixture = Fixture::new();
for (options, error) in [
("{servers: nil}", "nonempty array"),
("{servers: {command: ['server']}}", "nonempty array"),
("{servers: []}", "nonempty array"),
(
"{servers: [true]}",
"servers[0] must be a configuration object",
),
("{servers: [{}]}", "servers[0] requires command argv"),
("{servers: [{command: []}]}", "servers[0] command must be"),
(
"{servers: [{command: ['server'], extensions: ['py']}]}",
"servers[0] has unknown option",
),
(
"{servers: [{command: ['server'], servers: []}]}",
"servers[0] has unknown option",
),
(
"{servers: [{command: ['server'], initialization_options: []}]}",
"servers[0] initialization_options must be",
),
(
"{servers: [{command: ['server']}, {command: ['other'], settings: nil}]}",
"servers[1] settings must be",
),
(
"{servers: [{command: ['server']}], command: ['other']}",
"cannot be combined",
),
(
"{servers: [{command: ['server']}], initialization_options: {}}",
"cannot be combined",
),
(
"{servers: [{command: ['server']}], settings: {}}",
"cannot be combined",
),
] {
let path = fixture.file(
"config.rb",
&format!("Vamp.configure {{ |config| config.lsp[\"python\"] = {options} }}"),
);
let actual = load_config(&path).unwrap_err();
assert!(
actual.starts_with("Ruby config: LSP language \"python\""),
"{actual}"
);
assert!(actual.contains(error), "{options}: {actual}");
}
}
#[test]
fn lsp_server_limit_is_enforced_at_configuration_startup() {
let fixture = Fixture::new();
for count in [16, 17] {
let path = fixture.file(
"config.rb",
&format!(
"Vamp.configure {{ |config| config.lsp[\"python\"] = {{servers: Array.new({count}) {{ {{command: [\"server\"]}} }} }} }}"
),
);
let result = load_config(&path);
if count == 16 {
assert_eq!(result.unwrap().lsp.len(), 16);
} else {
assert_eq!(
result.unwrap_err(),
"Ruby config: LSP configuration supports at most 16 configured servers"
);
}
}
}
#[test] #[test]
fn lsp_configuration_rejects_invalid_keys_values_and_json_options() { fn lsp_configuration_rejects_invalid_keys_values_and_json_options() {
let fixture = Fixture::new(); let fixture = Fixture::new();
@@ -1074,6 +1277,7 @@ end"#,
r#"config.lsp["rust"] = "yes""#, r#"config.lsp["rust"] = "yes""#,
r#"config.lsp["custom"] = {}"#, r#"config.lsp["custom"] = {}"#,
r#"config.lsp["custom"] = {command: ["server"]}"#, r#"config.lsp["custom"] = {command: ["server"]}"#,
r#"config.lsp["custom"] = {servers: [{command: ["server"]}]}"#,
r#"config.lsp["custom"] = {extensions: ["custom"]}"#, r#"config.lsp["custom"] = {extensions: ["custom"]}"#,
r#"config.lsp["rust"] = {command: "server"}"#, r#"config.lsp["rust"] = {command: "server"}"#,
r#"config.lsp["rust"] = {command: []}"#, r#"config.lsp["rust"] = {command: []}"#,
@@ -1146,6 +1350,7 @@ end"#,
serde_json::json!({"": false}), serde_json::json!({"": false}),
serde_json::json!({"bad\u{0085}key": false}), serde_json::json!({"bad\u{0085}key": false}),
serde_json::json!({"rust": null}), serde_json::json!({"rust": null}),
serde_json::json!({"python": []}),
serde_json::json!({"unknown": true}), serde_json::json!({"unknown": true}),
serde_json::json!({"custom": {}}), serde_json::json!({"custom": {}}),
serde_json::json!({"rust": {"command": ["tool", "bad\0argument"]}}), serde_json::json!({"rust": {"command": ["tool", "bad\0argument"]}}),
@@ -1153,6 +1358,11 @@ end"#,
serde_json::json!({"rust": {"initialization_options": []}}), serde_json::json!({"rust": {"initialization_options": []}}),
serde_json::json!({"rust": {"settings": "not-an-object"}}), serde_json::json!({"rust": {"settings": "not-an-object"}}),
serde_json::json!({"rust": {"unexpected": true}}), serde_json::json!({"rust": {"unexpected": true}}),
serde_json::json!({"python": {"servers": []}}),
serde_json::json!({"python": {"servers": [null]}}),
serde_json::json!({"python": {"servers": [{}]}}),
serde_json::json!({"python": {"servers": [{"command": ["server"], "extensions": ["py"]}]}}),
serde_json::json!({"python": {"servers": [{"command": ["server"]}], "settings": {}}}),
] { ] {
frame["lsp"] = value; frame["lsp"] = value;
let Response::Config { lsp, .. } = let Response::Config { lsp, .. } =
@@ -1162,7 +1372,7 @@ end"#,
}; };
assert!(crate::lsp_config::resolve(lsp).is_err(), "{frame}"); assert!(crate::lsp_config::resolve(lsp).is_err(), "{frame}");
} }
let languages: serde_json::Map<String, serde_json::Value> = (0..17) let mut languages: serde_json::Map<String, serde_json::Value> = (0..14)
.map(|index| { .map(|index| {
( (
format!("custom-{index}"), format!("custom-{index}"),
@@ -1170,13 +1380,24 @@ end"#,
) )
}) })
.collect(); .collect();
languages.insert("python".into(), serde_json::json!(true));
frame["lsp"] = serde_json::Value::Object(languages); frame["lsp"] = serde_json::Value::Object(languages);
let Response::Config { lsp, .. } = let Response::Config { lsp, .. } =
serde_json::from_value::<Response>(frame.clone()).unwrap() serde_json::from_value::<Response>(frame.clone()).unwrap()
else { else {
panic!("expected config frame"); panic!("expected config frame");
}; };
assert!(crate::lsp_config::resolve(lsp).is_err()); assert_eq!(crate::lsp_config::resolve(lsp).unwrap().len(), 16);
frame["lsp"]["ruby"] = serde_json::json!(true);
let Response::Config { lsp, .. } =
serde_json::from_value::<Response>(frame.clone()).unwrap()
else {
panic!("expected config frame");
};
assert_eq!(
crate::lsp_config::resolve(lsp).unwrap_err(),
"LSP configuration supports at most 16 configured servers"
);
frame.as_object_mut().unwrap().remove("lsp"); frame.as_object_mut().unwrap().remove("lsp");
assert!(serde_json::from_value::<Response>(frame).is_err()); assert!(serde_json::from_value::<Response>(frame).is_err());
} }
+126
View File
@@ -390,6 +390,7 @@ impl EditorSession {
return Ok(false); return Ok(false);
} }
// Validate the whole applicable annotation batch before changing any buffer. // Validate the whole applicable annotation batch before changing any buffer.
let validation = (|| -> Result<(), String> {
for effect in &update.effects { for effect in &update.effects {
match effect { match effect {
UpdateEffect::Diagnostics { UpdateEffect::Diagnostics {
@@ -400,6 +401,20 @@ impl EditorSession {
self.editor self.editor
.validate_diagnostics(*buffer_id, *revision, items)?; .validate_diagnostics(*buffer_id, *revision, items)?;
} }
UpdateEffect::SemanticTokens {
buffer_id,
revision,
tokens,
} => {
if owner != "core" {
return Err("Semantic tokens are native LSP updates only".into());
}
self.editor.validate_semantic_tokens(
*buffer_id,
*revision,
tokens.as_deref(),
)?;
}
UpdateEffect::Navigate { UpdateEffect::Navigate {
buffer_id, buffer_id,
revision, revision,
@@ -435,6 +450,14 @@ impl EditorSession {
UpdateEffect::Message { .. } => {} UpdateEffect::Message { .. } => {}
} }
} }
Ok(())
})();
if let Err(error) = validation {
if owner == "core" {
self.editor.clear_semantic_tokens();
}
return Err(error);
}
let mut changed = false; let mut changed = false;
for effect in update.effects { for effect in update.effects {
match effect { match effect {
@@ -447,6 +470,15 @@ impl EditorSession {
.editor .editor
.replace_validated_diagnostics(owner, buffer_id, revision, items); .replace_validated_diagnostics(owner, buffer_id, revision, items);
} }
UpdateEffect::SemanticTokens {
buffer_id,
revision,
tokens,
} => {
changed |= self
.editor
.replace_validated_semantic_tokens(buffer_id, revision, tokens);
}
UpdateEffect::Navigate { UpdateEffect::Navigate {
buffer_id, buffer_id,
revision, revision,
@@ -595,6 +627,100 @@ mod tests {
static NEXT_FIXTURE: AtomicU64 = AtomicU64::new(0); static NEXT_FIXTURE: AtomicU64 = AtomicU64::new(0);
#[test]
fn native_semantic_batches_are_atomic_and_malformed_updates_clear_hidden_overlays() {
use crate::plugin::{SemanticToken, SemanticTokenKind};
use std::sync::Arc;
let fixture = Fixture::new();
fixture.file("buffer.txt", "source");
let target = fixture.file("target.txt", "target");
let mut session = fixture.open("");
let token = SemanticToken {
start: TextPosition { row: 0, column: 0 },
end: TextPosition { row: 0, column: 1 },
kind: SemanticTokenKind::Variable,
modifiers: 0,
};
let tokens: Arc<[SemanticToken]> = Arc::from([token]);
let semantic = |buffer_id, revision, tokens| UpdateEffect::SemanticTokens {
buffer_id,
revision,
tokens,
};
let update = |effects| PluginUpdate {
request_id: None,
effects,
};
session
.apply_update("core", update(vec![semantic(1, 0, Some(tokens.clone()))]))
.unwrap();
session.execute(&format!("edit {}", target.display()));
session
.apply_update("core", update(vec![semantic(2, 0, Some(tokens.clone()))]))
.unwrap();
let diagnostic = crate::plugin::Diagnostic {
start: token.start,
end: token.end,
severity: crate::plugin::DiagnosticSeverity::Warning,
message: "unchanged".into(),
};
session
.editor
.replace_diagnostics("plugin", 2, 0, vec![diagnostic.clone()])
.unwrap();
let malformed: Arc<[SemanticToken]> = Arc::from([SemanticToken {
end: TextPosition { row: 99, column: 0 },
..token
}]);
let before_message = session.editor.message().to_owned();
let result = session.apply_update(
"core",
update(vec![
UpdateEffect::Message {
text: "must not apply".into(),
},
UpdateEffect::Diagnostics {
buffer_id: 2,
revision: 0,
items: vec![diagnostic.clone()],
},
semantic(1, 0, Some(tokens.clone())),
semantic(2, 0, Some(malformed.clone())),
]),
);
assert!(result.is_err());
assert_eq!(session.editor.message(), before_message);
assert!(session.editor.semantic_tokens().is_empty());
assert_eq!(session.editor.diagnostics().count(), 1);
session.execute("buffer 1");
assert!(session.editor.semantic_tokens().is_empty());
assert!(
session
.apply_update(
"core",
update(vec![
semantic(999, 0, Some(malformed.clone())),
semantic(1, 99, Some(malformed)),
semantic(1, 0, Some(tokens.clone())),
])
)
.unwrap()
);
assert!(std::ptr::eq(
session.editor.semantic_tokens(),
tokens.as_ref()
));
assert_eq!(session.editor.lines(), ["source"]);
assert_eq!(session.editor.text_revision(), 0);
assert!(!session.editor.dirty());
assert!(
session
.apply_update("core", update(vec![semantic(1, 0, None)]))
.unwrap()
);
assert!(session.editor.semantic_tokens().is_empty());
}
struct Fixture(PathBuf); struct Fixture(PathBuf);
impl Fixture { impl Fixture {
+20 -1
View File
@@ -20,7 +20,7 @@ pub enum SyntaxKind {
Markup, Markup,
} }
/// A semantic foreground range in the original line's bytes, excluding its newline. /// A syntax foreground range in the original line's bytes, excluding its newline.
#[derive(Clone, Copy, Debug, Eq, PartialEq)] #[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct SyntaxSpan { pub struct SyntaxSpan {
pub start_byte: usize, pub start_byte: usize,
@@ -1263,6 +1263,25 @@ mod tests {
assert_token(&cache, source, 3, "42", SyntaxKind::Number); assert_token(&cache, source, 3, "42", SyntaxKind::Number);
} }
#[test]
fn markdown_heading_markers_share_title_colors_without_coloring_plain_hashes() {
let source = "# One\n## Two\n### Three\n#### Four\n##### Five\n###### Six\n\nSetext\n======\n\nplain # text";
let cache = self::cache(source, "headings.md");
for row in 0..6 {
assert_token(
&cache,
source,
row,
&"#".repeat(row + 1),
SyntaxKind::Markup,
);
}
assert_token(&cache, source, 0, "One", SyntaxKind::Markup);
assert_token(&cache, source, 7, "Setext", SyntaxKind::Markup);
assert_token(&cache, source, 8, "======", SyntaxKind::Markup);
assert_eq!(kind_at(&cache, 10, 6), None);
}
#[test] #[test]
fn markdown_inline_and_fences_inject_known_languages_and_update_incrementally() { fn markdown_inline_and_fences_inject_known_languages_and_update_incrementally() {
let original = "A **bold** and `code` link.\n\n```rust\nfn main() { let n = 42; }\n```\n\n```python\ndef hello():\n return 7\n```\n\n```not-a-language\nfn not_rust() { 99; }\n```"; let original = "A **bold** and `code` link.\n\n```rust\nfn main() { let n = 42; }\n```\n\n```python\ndef hello():\n return 7\n```\n\n```not-a-language\nfn not_rust() { 99; }\n```";
+19 -1
View File
@@ -46,6 +46,24 @@ static CPP_HIGHLIGHTS: LazyLock<String> = LazyLock::new(|| {
] ]
.join("\n") .join("\n")
}); });
static MD_HIGHLIGHTS: LazyLock<String> = LazyLock::new(|| {
[
tree_sitter_md::HIGHLIGHT_QUERY_BLOCK,
"
[
(atx_h1_marker)
(atx_h2_marker)
(atx_h3_marker)
(atx_h4_marker)
(atx_h5_marker)
(atx_h6_marker)
(setext_h1_underline)
(setext_h2_underline)
] @text.title
",
]
.join("\n")
});
// The MIT-licensed upstream C++ crate bundles, but does not export, injections.scm. // The MIT-licensed upstream C++ crate bundles, but does not export, injections.scm.
// This minimal AST query uses its raw-string delimiter/content nodes: // This minimal AST query uses its raw-string delimiter/content nodes:
@@ -129,7 +147,7 @@ static LANGUAGES: LazyLock<[LanguageSpec; 18]> = LazyLock::new(|| {
spec( spec(
"Markdown", "Markdown",
tree_sitter_md::LANGUAGE.into(), tree_sitter_md::LANGUAGE.into(),
tree_sitter_md::HIGHLIGHT_QUERY_BLOCK, MD_HIGHLIGHTS.as_str(),
tree_sitter_md::INJECTION_QUERY_BLOCK, tree_sitter_md::INJECTION_QUERY_BLOCK,
"", "",
), ),
+183 -18
View File
@@ -31,7 +31,9 @@ use unicode_width::UnicodeWidthStr;
use crate::{ use crate::{
editor::{Editor, Hover, Input, Mode, Palette, StartupDiagnostic}, editor::{Editor, Hover, Input, Mode, Palette, StartupDiagnostic},
files::{DockSide, FilePicker, FilePresentation, FileView}, files::{DockSide, FilePicker, FilePresentation, FileView},
plugin::{Diagnostic, DiagnosticSeverity}, plugin::{
Diagnostic, DiagnosticSeverity, SemanticToken, SemanticTokenKind, SemanticTokenModifier,
},
session::EditorSession, session::EditorSession,
syntax::{SyntaxCache, SyntaxKind}, syntax::{SyntaxCache, SyntaxKind},
}; };
@@ -345,15 +347,25 @@ fn draw(
.collect(); .collect();
frame.render_widget(Paragraph::new(lines), viewport); frame.render_widget(Paragraph::new(lines), viewport);
let buffer = frame.buffer_mut(); let buffer = frame.buffer_mut();
let semantic = editor.semantic_tokens();
let mut semantic_index = semantic.partition_point(|token| token.start.row < *top);
for (row, line) in editor.lines().iter().enumerate().skip(*top).take(height) { for (row, line) in editor.lines().iter().enumerate().skip(*top).take(height) {
let spans = syntax.line(row); let spans = syntax.line(row);
if spans.is_empty() { while semantic_index < semantic.len() && semantic[semantic_index].start.row < row {
semantic_index += 1;
}
if spans.is_empty()
&& semantic
.get(semantic_index)
.is_none_or(|token| token.start.row != row)
{
continue; continue;
} }
let mut span_index = 0; let mut span_index = 0;
let mut token_style = None;
let mut column = 0; let mut column = 0;
let right = left.saturating_add(width); let right = left.saturating_add(width);
for (byte, grapheme) in line.grapheme_indices(true) { for (index, (byte, grapheme)) in line.grapheme_indices(true).enumerate() {
if column >= right { if column >= right {
break; break;
} }
@@ -366,22 +378,39 @@ fn draw(
while span_index < spans.len() && spans[span_index].end_byte <= byte { while span_index < spans.len() && spans[span_index].end_byte <= byte {
span_index += 1; span_index += 1;
} }
if let Some(span) = spans.get(span_index) while semantic_index < semantic.len()
&& span.start_byte <= byte && semantic[semantic_index].start.row == row
&& end > *left && semantic[semantic_index].end.column <= index
&& (grapheme == "\t" || (column >= *left && end <= right))
{ {
let color = match span.kind { semantic_index += 1;
token_style = None;
}
let style = semantic
.get(semantic_index)
.filter(|token| token.start.row == row && token.start.column <= index)
.map(|token| *token_style.get_or_insert_with(|| semantic_style(token)))
.or_else(|| {
spans
.get(span_index)
.filter(|span| span.start_byte <= byte)
.map(|span| {
Style::default().fg(match span.kind {
SyntaxKind::Comment => Color::DarkGray, SyntaxKind::Comment => Color::DarkGray,
SyntaxKind::String => Color::Green, SyntaxKind::String => Color::Green,
SyntaxKind::Keyword => Color::Magenta, SyntaxKind::Keyword => Color::Magenta,
SyntaxKind::Number => Color::Yellow, SyntaxKind::Number => Color::Yellow,
SyntaxKind::Type | SyntaxKind::Markup => Color::Cyan, SyntaxKind::Type | SyntaxKind::Markup => Color::Cyan,
SyntaxKind::Function => Color::LightBlue, SyntaxKind::Function => Color::LightBlue,
}; })
})
});
if let Some(style) = style
&& end > *left
&& (grapheme == "\t" || (column >= *left && end <= right))
{
let y = viewport.y + (row - *top) as u16; let y = viewport.y + (row - *top) as u16;
for cell in column.max(*left)..end.min(right) { for cell in column.max(*left)..end.min(right) {
buffer[(viewport.x + (cell - *left) as u16, y)].set_fg(color); buffer[(viewport.x + (cell - *left) as u16, y)].set_style(style);
} }
} }
column = end; column = end;
@@ -1044,6 +1073,36 @@ fn draw_palette(frame: &mut Frame, palette: &Palette, mode: Mode, navigation_hin
); );
} }
fn semantic_style(token: &SemanticToken) -> Style {
use SemanticTokenKind::*;
let readonly = token.modifiers & SemanticTokenModifier::Readonly as u16 != 0;
let color = match token.kind {
Namespace | Type | Class | Enum | Interface | Struct | TypeParameter => Color::Cyan,
Function | Method | Event | Macro => Color::LightBlue,
Parameter | Variable | Property if readonly => Color::Yellow,
Parameter | Variable => Color::LightCyan,
Property => Color::Blue,
EnumMember | Number => Color::Yellow,
Keyword | Modifier | Operator | Decorator => Color::Magenta,
String | Regexp => Color::Green,
Comment => Color::DarkGray,
};
let mut style = Style::default().fg(color);
if token.modifiers
& (SemanticTokenModifier::Declaration as u16 | SemanticTokenModifier::Definition as u16)
!= 0
{
style = style.add_modifier(ratatui::style::Modifier::BOLD);
}
if token.kind == Comment && token.modifiers & SemanticTokenModifier::Documentation as u16 != 0 {
style = style.add_modifier(ratatui::style::Modifier::ITALIC);
}
if token.modifiers & SemanticTokenModifier::Deprecated as u16 != 0 {
style = style.add_modifier(ratatui::style::Modifier::CROSSED_OUT);
}
style
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -1150,6 +1209,21 @@ mod tests {
} }
} }
fn semantic_token(
row: usize,
start: usize,
end: usize,
kind: SemanticTokenKind,
modifiers: u16,
) -> SemanticToken {
SemanticToken {
start: crate::plugin::TextPosition { row, column: start },
end: crate::plugin::TextPosition { row, column: end },
kind,
modifiers,
}
}
#[test] #[test]
fn diagnostic_underlines_preserve_syntax_and_whole_graphemes_at_viewport_edges() { fn diagnostic_underlines_preserve_syntax_and_whole_graphemes_at_viewport_edges() {
let directory = Directory::new(); let directory = Directory::new();
@@ -1579,15 +1653,54 @@ end
} }
#[test] #[test]
fn syntax_colors_whole_unicode_graphemes_and_clipped_tabs_without_painting_wide_blanks() { fn semantic_colors_modifiers_and_syntax_fallback_cover_whole_graphemes_and_clipped_tabs() {
let directory = Directory::new(); let directory = Directory::new();
let source = "#a\t界e\u{301}👩‍🚀z"; let source = "#a\t界e\u{301}👩‍🚀z";
let mut editor = Editor::open(Some(directory.file("unicode.py", source)), vec![]).unwrap(); let mut editor = Editor::open(Some(directory.file("unicode.py", source)), vec![]).unwrap();
editor.set_clipboard_mode("separate").unwrap();
editor
.replace_semantic_tokens(
editor.buffer_id(),
editor.text_revision(),
Some(Arc::from([
semantic_token(
0,
0,
1,
SemanticTokenKind::Comment,
SemanticTokenModifier::Documentation as u16,
),
semantic_token(
0,
2,
6,
SemanticTokenKind::Property,
SemanticTokenModifier::Readonly as u16
| SemanticTokenModifier::Definition as u16
| SemanticTokenModifier::Deprecated as u16,
),
semantic_token(0, 2, 3, SemanticTokenKind::Variable, 0),
])),
)
.unwrap();
editor
.replace_diagnostics(
"fixture",
editor.buffer_id(),
editor.text_revision(),
vec![annotation(
(0, 2),
(0, 6),
DiagnosticSeverity::Warning,
"semantic",
)],
)
.unwrap();
let mut syntax = SyntaxCache::new(); let mut syntax = SyntaxCache::new();
for (offset, width, caret, colors) in [ for (offset, width, caret, colors) in [
(2, 5, 4, vec![Color::DarkGray; 5]), (2, 5, 4, vec![Color::Yellow; 5]),
(5, 2, 4, vec![Color::Reset, Color::DarkGray]), (5, 2, 4, vec![Color::Reset, Color::Yellow]),
(7, 2, 5, vec![Color::DarkGray; 2]), (7, 2, 5, vec![Color::Yellow; 2]),
(8, 2, 6, vec![Color::Reset, Color::DarkGray]), (8, 2, 6, vec![Color::Reset, Color::DarkGray]),
( (
0, 0,
@@ -1596,8 +1709,8 @@ end
vec![ vec![
Color::DarkGray, Color::DarkGray,
Color::DarkGray, Color::DarkGray,
Color::DarkGray, Color::Yellow,
Color::DarkGray, Color::Yellow,
Color::Reset, Color::Reset,
], ],
), ),
@@ -1632,21 +1745,73 @@ end
assert_eq!(buffer[position].symbol(), expected[position].symbol()); assert_eq!(buffer[position].symbol(), expected[position].symbol());
assert_eq!(rendered[position].fg, color, "left={offset}, cell={cell}"); assert_eq!(rendered[position].fg, color, "left={offset}, cell={cell}");
assert_eq!(rendered[position].bg, Color::Reset); assert_eq!(rendered[position].bg, Color::Reset);
let semantic_cell = color == Color::Yellow;
assert_eq!(
rendered[position].modifier.contains(Modifier::BOLD),
semantic_cell
);
assert_eq!(
rendered[position].modifier.contains(Modifier::CROSSED_OUT),
semantic_cell
);
assert_eq!(
rendered[position].modifier.contains(Modifier::UNDERLINED),
semantic_cell
);
if semantic_cell {
assert_eq!(rendered[position].underline_color, Color::Yellow);
} }
assert_eq!(buffer[(area.x, area.y)].fg, Color::Cyan); assert_eq!(
rendered[position].modifier.contains(Modifier::ITALIC),
offset == 0 && cell == 0
);
}
assert_eq!(buffer[(area.x, area.y)].fg, Color::Yellow);
assert_eq!(buffer[(0, 0)].fg, Color::Reset); assert_eq!(buffer[(0, 0)].fg, Color::Reset);
assert_eq!(editor.lines(), &[source]); assert_eq!(editor.lines(), &[source]);
} }
editor.handle(Input::Char('x'));
assert!(editor.semantic_tokens().is_empty());
let mut terminal = Terminal::new(ratatui::backend::TestBackend::new(20, 4)).unwrap();
let (mut top, mut left) = (0, 0);
terminal
.draw(|frame| draw(frame, &editor, &mut top, &mut left, &mut syntax))
.unwrap();
assert_eq!(terminal.backend().buffer()[(2, 0)].fg, Color::DarkGray);
assert!(
!terminal.backend().buffer()[(2, 0)]
.modifier
.contains(Modifier::ITALIC)
);
} }
#[test] #[test]
fn search_visual_selection_and_secondary_carets_override_syntax_foreground() { fn search_visual_selection_and_secondary_carets_override_semantic_foreground() {
let directory = Directory::new(); let directory = Directory::new();
let mut editor = Editor::open( let mut editor = Editor::open(
Some(directory.file("overlays.py", "# word\n# word\n# word")), Some(directory.file("overlays.py", "# word\n# word\n# word")),
vec![], vec![],
) )
.unwrap(); .unwrap();
editor
.replace_semantic_tokens(
editor.buffer_id(),
editor.text_revision(),
Some(Arc::from(
(0..3)
.map(|row| {
semantic_token(
row,
2,
6,
SemanticTokenKind::Function,
SemanticTokenModifier::Declaration as u16,
)
})
.collect::<Vec<_>>(),
)),
)
.unwrap();
editor editor
.replace_diagnostics( .replace_diagnostics(
"fixture", "fixture",
+1173 -7
View File
File diff suppressed because it is too large Load Diff