From 5c08aa0a67aa6fb26a7038deb93202e0c7528172 Mon Sep 17 00:00:00 2001 From: Valdemar Lorenzen Date: Wed, 21 Oct 2026 11:28:59 +0200 Subject: [PATCH] chore: isolate tests from external services --- TODO.md | 2 +- docs/development.md | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/TODO.md b/TODO.md index 7ecd5a1..5e23333 100644 --- a/TODO.md +++ b/TODO.md @@ -3,7 +3,7 @@ - [x] improve: pull requests - add a `🙏` emoji left of the PR entry in the PR list if the user in question (me) has been requested as reviewer. - also change the checkmark displayed for PRs approved by me to the `👍` emoji. - make the symbols displayed for these states configurable. -- [ ] chore: check to make sure we're mocking everything in tests, not pinging/fetching real data from real websites/APIs. +- [x] chore: check to make sure we're mocking everything in tests, not pinging/fetching real data from real websites/APIs. - [ ] feature: diff view - GitHub code review integration after the local review UX is usable. - publish local reviews explicitly: general PR comments, inline line/range comments, review summaries, approval, and requests for changes. - preserve explicit PR identity and reviewed commit IDs; validate anchors against GitHub's PR diff and recheck the PR head before submission. diff --git a/docs/development.md b/docs/development.md index d707b20..4347d6d 100644 --- a/docs/development.md +++ b/docs/development.md @@ -41,6 +41,26 @@ Rust tests live under [`tests/`](../tests/). Private unit tests are in [`tests/u `cargo test` runs both groups; `cargo test --lib` and `cargo test --bin vamp` select the library and CLI unit suites. Do not name a unit file `tests/unit/main.rs`: Cargo would discover it as a separate integration target. +### External-service isolation + +Tests must not contact real websites or APIs. Forge tests inject JSON responses through `page_with`, `metadata_with`, and `resolve_with`; listing workers inject fetch closures. Repository remotes are temporary local repositories (including missing local paths for failure cases). LSP servers, Ruby plugins, and shell commands are local fixture subprocesses. The SSH alias check uses `ssh -G` with a temporary config to inspect configuration without connecting. + +On macOS, verify external-service isolation with networking denied for Cargo and all descendants: + +```sh +sandbox-exec -p '(version 1) (allow default) (deny network*)' \ + cargo test --offline --no-fail-fast -- --test-threads=1 \ + --skip shell::tests::completed_shell_does_not_leave_background_jobs_holding_output_open \ + --skip shell::tests::dropping_shell_kills_and_reaps_group_with_pipe_inheriting_descendant +cargo test --offline --lib shell::tests:: -- --test-threads=1 +``` + +Dependencies must already be cached. `--offline` prevents Cargo downloads; it does not block networking by tests or subprocesses. The sandbox supplies that boundary, including for real Git/jj operations and fixture programs; do not replace those local integration checks with mocks. + +The two skipped shell cleanup checks execute `/bin/ps`, which macOS refuses to launch even under an allow-all sandbox profile; run them outside the sandbox as shown. Serial execution avoids contention in subprocess fixtures with production deadlines. + +### Coverage + - Editor checks cover Unicode editing, multi-target state, undo/redo, saving failures, and modal paste acceptance and rejection atomicity. - Prompt-history checks cover separate session-wide command/search entries, Unicode draft restoration, editable recalls, clamped/empty/bounded history, duplicate/empty/cancelled submissions, remapped/disabled keys, and preserved Visual/read-only diff targets. Live PTY smoke exercises arrows and raw Ctrl+k/j, restores drafts, accepts a recalled search with Enter, and checks command/search isolation and unchanged source bytes. - Command-completion checks cover forward/backward alphabetical cycling/wrap across core names, aliases and live plugin registrations, reverse-first completion and direction changes retaining the prefix, explicit execution, edits/history/cancel resets, empty/unknown/Unicode prefixes, untouched arguments, remapped/disabled keys, and preserved Visual/read-only diff targets and literal search tabs. Live PTY smoke cycles `tab.*` in both directions through wraparound, submits a completed `tab.new`, and exits with unchanged source bytes.