feat: migrate LSP to opt-in native language servers

README.md

@@ -200,14 +200,15 @@ Leader bindings require Normal/Visual and the target's existing mode eligibility
 
 ## Module seams
 
-One crate, eight modules, no workspace or terminal-coordinate types in the editing backend:
+One crate; no workspace or terminal-coordinate types in the editing backend:
 
 - `editor`: owns buffer lifecycle and per-buffer text, saved state, primary/selection set, modes, search, and undo/redo; session-wide clipboard policy, independent registers and owned-system fragment metadata, semantic native repeat, action metadata, command-palette and file-picker focus/dispatch, key notation, leader/count prefixes and binding resolution, modal paste policy, saving, command ownership, capabilities, and validated edit effects. Its low-level interface returns an `Invocation` when an external command is selected.
 - `files`: owns the gitignore-aware directory/file catalog, authoritative OS paths, compact tree projection, query filtering, browse/search focus, expansion/selection restoration, navigation, and scan/error state shared by floating and docked presentations.
 - `clipboard`: owns native plain-text provider selection and bounded subprocess I/O; it never computes selections or editing offsets.
 - `syntax`: owns Tree-sitter grammar detection, per-buffer incremental trees, bundled highlight/local/injection queries, and semantic byte spans; it has no terminal color/geometry or editing dependency.
 - `plugin`: owns generic document snapshots/events, zero-based grapheme positions, diagnostics, and versioned asynchronous update contracts. It has no LSP protocol types.
-- `session`: `EditorSession` owns the editor and its Ruby hosts together. It loads configuration once, registers eligible plugins before installing shared bindings, tracks document open/change/save/close events, polls asynchronous results, validates versioned diagnostics/navigation, and retires host and command ownership on invocation or effect-validation failure.
+- `lsp`: owns opt-in native server workers, bounded framed JSON-RPC, synchronization/capability negotiation, settings, UTF-16/file-URI conversion, and definition results. Its private configuration resolver supplies presets without restricting language IDs.
+- `session`: `EditorSession` owns the editor, native LSP, and Ruby hosts together. It loads configuration once, registers eligible plugins before installing shared bindings, tracks document open/change/save/close events, polls native and plugin results, validates versioned diagnostics/navigation, and retires failed plugin ownership.
 - `tui`: translates terminal events to session input, renders read-only active-buffer state, keeps frontend-owned viewports and syntax caches by stable buffer ID, and advertises `text`, `styled_text`, and `diagnostics` capabilities. It maps semantic tokens and diagnostics to terminal styling and has no server/protocol logic or paste acceptance policy.
 - `ruby`: evaluates configuration and runs isolated plugin hosts. Protocol, deadlines, and process cleanup stay inside this module.
 - The executable selects paths and frontend capabilities, opens an editor session, and launches the TUI.
@@ -220,7 +221,7 @@ Frontends inspect `count()` for the optional pending motion prefix; parsing and
 Frontends inspect `leader_pending()` for the cached display of a pending leader sequence; prefix recognition, cancellation, and dispatch remain editor-owned.
 Frontends inspect `buffer_id()`, `buffer_count()`, `has_buffer(id)`, and the ascending-ID `buffers()` iterator of borrowed path/dirty/active metadata. Existing text, cursor, selection, search, and mode methods describe only the active buffer.
 Frontends inspect `file_picker()` for query, presentation, view, explorer-local INSERT/filter subfocus, selection, and errors, and iterate borrowed `FileRow` metadata (actual path, display label, directory flag, visible depth, expansion). Dock side is a logical preference. They allocate rectangles, add folder suffixes/colors, and render; hierarchy, filtering, parent/child navigation, focus transitions, and selected-file opening remain backend-owned.
-Frontends inspect `diagnostics()` and `diagnostic_at_cursor()` for validated active-buffer annotations. Diagnostics are owned by a plugin and buffer revision; text changes and owner retirement clear them. The TUI underlines annotated text, colors affected gutter numbers by severity, and displays the primary caret's diagnostic message without replacing search/selection backgrounds.
+Frontends inspect `diagnostics()` and `diagnostic_at_cursor()` for validated active-buffer annotations. Diagnostics are owned by core LSP or a plugin and buffer revision; text changes and owner/server retirement clear them. The TUI underlines annotated text, colors affected gutter numbers by severity, and displays the primary caret's diagnostic message without replacing search/selection backgrounds.
 
 The terminal adapter owns raw mode, alternate-screen state, bracketed paste, and its panic hook for one session lifetime. Entry failure, normal return, I/O errors, and unwinding panics restore the caller's hook; active-terminal cleanup runs before caller panic diagnostics. Panics retain their original payload and propagate rather than being swallowed. Terminal state and panic hooks are process-global: run one TUI at a time and do not replace its hook while that session is active.
 
@@ -242,27 +243,38 @@ Built-in Ex commands register as the protected `core` feature in the same comman
 
 A feature declares all capabilities it needs. Registration is atomic: if any requirement is unavailable or any command conflicts, none of its commands are installed. For example, `Vamp.plugin "canvas", requires: ["graphics"]` will be refused by this TUI. Capability names are extensible strings, not a terminal-vs-GUI switch. Capability declarations express compatibility; they are not security permissions.
 
-### Optional LSP plugin
+### Native LSP
 
-LSP is implemented in `examples/plugins/lsp.rb`, not the editing core. It requires Ruby and an independently installed, trusted language server; it is not loaded by default. Enable it in your config (the plugin path is relative to that config):
+LSP runs in Rust, independently of Ruby plugin hosts. **Every language server is disabled by default.** Ruby evaluates configuration only at startup; no LSP plugin needs loading. Install a trusted server yourself and explicitly opt in:
 
 ```ruby
-config.plugin "plugins/lsp.rb", options: {
-  servers: [
-    { language_id: "rust", extensions: ["rs"], command: ["rust-analyzer"] },
-    { language_id: "python", extensions: ["py"], command: ["pylsp"] }
-  ]
-}
-config.bind "leader d", "lsp_definition"
+Vamp.configure do |config|
+  config.lsp["rust"] = true # rust-analyzer, extensions: ["rs"]
+  config.lsp["python"] = false # disabled (also the default when absent)
+
+  config.lsp["my-language"] = {
+    command: ["my-language-server", "--stdio"],
+    extensions: ["myl"],
+    initialization_options: {},
+    settings: { "my-language-server" => { "exampleOption" => true } }
+  }
+  config.bind "leader d", "lsp_definition"
+end
 ```
 
-Server commands are argv arrays, never shell commands. The first matching extension configuration wins. One server runs per configuration/project root, using the startup working directory as its workspace. The plugin initializes the server, honors its open/change/save capabilities, synchronizes in-memory text (including hidden dirty buffers), and sends close events. Servers requesting incremental synchronization receive whole-document range replacements. Server transport and requests run in plugin worker threads; the editor continues processing input and idle diagnostics.
+Keys are arbitrary nonempty strings, used as the protocol's `languageId`; they need no Tree-sitter grammar or hardcoded language enum. Absent/`false` entries are disabled. `true` enables a built-in preset: currently `"rust"` uses `rust-analyzer`/`rs`, and `"python"` uses `pylsp`/`py`. Unknown keys need a configuration object with nonempty `command` argv and `extensions` (without leading dots). Objects also explicitly enable servers; known presets supply omitted command/extensions. For example, `config.lsp["rust"] = { command: ["/custom/path/rust-analyzer"] }` overrides the executable while retaining `rs`.
+
+Only `command`, `extensions`, `initialization_options`, and `settings` are accepted. Both option/settings values must be JSON-compatible objects and default to `{}`. Invalid declarations reject configuration with a notice rather than activating a guessed server. Server commands are executable/argument arrays, never shell commands; use an absolute path if the executable is not on PATH. No server installation or restart is automatic.
+
+Servers start lazily only when a matching buffer opens. Extension matching is case-sensitive; if enabled languages overlap, the first language in sorted key order wins. One worker runs per language configuration/project root, with at most 16 enabled languages/server-project keys. Start vamp from your project root: its startup working directory is the workspace; automatic root discovery is not implemented.
+
+The native module initializes each server, sends `initialization_options` as `initializationOptions`, answers `workspace/configuration` from `settings` (including dotted sections; missing sections return `null`), honors open/change/save capabilities, synchronizes in-memory text including hidden dirty buffers, and sends close events. Incremental servers receive whole-document range replacements. Process/pipe I/O and requests stay on bounded background workers; input and idle diagnostic redraw do not wait for server responses.
 
 `:lsp_definition` or the configured Space d binding requests a definition at the primary caret, once regardless of selection count. Both Location and LocationLink results are supported; multiple results use the first. Local-file navigation reuses loaded buffers without overwriting unsaved text. LSP UTF-16 positions and percent-encoded file URIs are translated to validated grapheme coordinates. Navigation is ignored if its source buffer, revision, caret, or latest request has changed.
 
 Diagnostics appear as severity-colored gutter numbers and text underlines; the primary caret's diagnostic appears in the message row. `:lsp_diagnostics` reports the current document's server state. Older versioned diagnostics are discarded; unversioned diagnostics can only be associated with the arrival-time document revision. Editing clears previous annotations immediately. Server startup/transport failures are reported and clear that server's diagnostics; other servers and native editing remain available. There are no automatic restarts.
 
-This plugin currently provides synchronization, diagnostics, and go-to-definition—not completion, hover, rename, formatting, code actions, workspace edits, or semantic highlighting. Tree-sitter remains the syntax highlighter. Server configuration requests return `null` for defaults; custom server settings and workspace discovery are not implemented.
+Native LSP currently provides synchronization, diagnostics, and go-to-definition—not completion, hover, rename, formatting, code actions, workspace edits, or semantic highlighting. Tree-sitter remains the syntax highlighter. The former Ruby LSP plugin has been removed; migrate its `servers` entries to `config.lsp[language_id]` objects without the `language_id` field.
 
 ## Ruby configuration
 
@@ -381,7 +393,7 @@ JSON uses Ruby's standard library. Protocol Buffers is not installed or required
 
 ## Current limits
 
-- One visible buffer at a time; Normal, Insert, Command, Search, and characterwise Visual modes. No named registers, line/block selection, Vim operators, splits/tabs, regex/multiline search, or graphical frontend. Optional LSP currently supports diagnostics and definitions only. Native system clipboard access currently supports macOS and Linux; Separate mode retains internal editing on unsupported/headless environments.
+- One visible buffer at a time; Normal, Insert, Command, Search, and characterwise Visual modes. No named registers, line/block selection, Vim operators, splits/tabs, regex/multiline search, or graphical frontend. Native opt-in LSP currently supports diagnostics and definitions only. Native system clipboard access currently supports macOS and Linux; Separate mode retains internal editing on unsupported/headless environments.
 - Undo uses full-buffer snapshots and movement scans the current line. This favors straightforward correctness over large-file performance.
 - Legacy synchronous plugin invocation may pause input for up to 500 ms before a host timeout. Async plugins opt into background replies; pending requests and transport queues are bounded. Config/plugin startup has a 2-second deadline per host. Buffer context/document snapshots must fit within a 4 MiB JSON frame; a too-large frame disables that plugin, not editing.
 - Disabling a plugin removes its commands and bindings until the next launch. There is no hot reload.
@@ -399,4 +411,4 @@ Regression checks exercise Unicode edits, undo/redo, saving and failed saves, ca
 
 Unix terminal-hook checks run in isolated child processes and real PTYs to keep global hooks out of the parallel test runner. They cover entry failure, normal return, I/O error, panic cleanup before caller diagnostics, original panic-payload propagation, and later panics producing no stale terminal-restoration output.
 
-LSP integration checks launch real framed-protocol fixture subprocesses through the optional Ruby plugin: fragmented/adjacent frames, initialization and synchronization capabilities, open/change/save/close, UTF-16 and encoded Unicode paths, dirty-buffer definitions, stale/superseded requests and diagnostics, unsupported server requests, crashes, malformed frames, unavailable servers, and surviving native editing. A real executable PTY smoke also exercised idle diagnostic redraw, active-server editing, definition navigation into a dirty loaded buffer, save/close notifications, and clean shutdown. No desktop clipboard or user files are touched.
+LSP integration checks launch real framed-protocol fixture subprocesses directly from native Rust: fragmented/adjacent frames, initialization/custom settings, arbitrary language IDs, opt-in matching-only startup, negotiated synchronization, open/change/save/close/shutdown, UTF-16 and encoded Unicode paths, dirty targets without enabled LSPs, stale/superseded requests and diagnostics, moved carets, unsupported requests, crashes, malformed frames, unavailable servers, and surviving native editing. Executable PTY smoke exercised idle diagnostic redraw, active-server edits, definition navigation into a dirty non-LSP buffer, Unicode cursor placement, settings, saving, and clean shutdown; its server's parent PID confirmed there was no Ruby LSP host. No desktop clipboard or user files are touched. Windows transport/process cleanup is implemented but not exercised on this macOS workstation.