MIKE-49 checkpoint 1: Desktop fixture-only audit command - #7557
Draft
michaelerobertsjr wants to merge 5 commits into
Draft
michaelerobertsjr wants to merge 5 commits into
michaelerobertsjr wants to merge 5 commits into
Conversation
Adds a Desktop UI entry point for auditing agent-turn-metric (NIP-AM kind:44200) verification, pagination, and sanitization — fixture-only and simulated-data-only for this checkpoint. Reuses the reviewed `nip-am-verify` Rust crate (buzz-auditor, rev 83db37a) unmodified via a path dependency, and ports buzz-auditor's Python sanitizer discipline (field allowlist, credential-shape scanning, kind type-checking, fixed content-free errors) to Rust as a second, redundant gate. New `mike49_run_fixture_audit` Tauri command takes no `AppState` parameter and never touches the real signing key, local archive, or native_relay_client. A new "MIKE-49 fixture audit (simulated data)" panel in Settings > Experiments exposes it with explicit-run, loading/success/failure/capped-incomplete/empty states and a duplicate-run guard. Real relay/key wiring, merge into main, and any live validation run are explicitly out of scope for this checkpoint. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Mike Roberts <mike@sdcs.io>
🔐 Codex Security Review
|
Pin nip-am-verify to an exact, verified commit instead of a mutable sibling checkout: scripts/mike49-pin-nip-am-verify.sh materializes NIP_AM_VERIFY_PINNED_REV (83db37a6f8fec5663227ceeb439b21c30d5d19ac) as a detached-HEAD git worktree of cccareers/buzz-auditor at <this-checkout-toplevel>/.mike49-nip-am-verify-pinned, and Cargo.toml's path dependency now points at that fixed, worktree-relative location. This also fixes the previously-documented nested-worktree relative-path break, without vendoring buzz-auditor source into block/buzz and without a Cargo.toml hand-edit per test run. Add a sanitizer parity test (record_sanitizer_matches_the_pinned_python_ reference_on_shared_examples) that runs shared example inputs through both this crate's RECORD_ALLOWED and a frozen capture of the pinned Python nip_am_pipeline_bridge.sanitize_pipeline_line output, including nested credential-shaped values and one explicitly documented, safe divergence (related_event_id, validated by Python but dropped here since this command's own fixture producer never emits it). Add a Playwright e2e spec (mike49-audit-fixture.spec.ts) exercising the actual click -> Tauri invoke -> render path through the real mock-IPC boundary: explicit-run/loading/success, a rejected-invoke failure state, and the duplicate-run guard. Wires mike49_run_fixture_audit into the shared e2e mock bridge for this. Verified at this commit in this worktree, Hermit-activated (rustc/cargo 1.95.0): cargo test 3199/3199 (debug), clippy --all-targets -D warnings 0 warnings, cargo fmt --check clean, pnpm typecheck clean, pnpm test 6494/6494, pnpm check (biome + file-size gates) clean, playwright test --project=smoke mike49-audit-fixture 3/3. Still fixture-only, still no real keys/events/relay/archive access, no merge. Draft PR block#7557, MIKE-49. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Signed-off-by: Mike Roberts <mike@sdcs.io>
…ibility, native IPC trace Addresses Hannibal's second reproducibility/evidence review round at 3baa50f: - scripts/mike49-pin-nip-am-verify.sh: reject a pinned worktree with modified tracked files or unexpected untracked files (`git status --porcelain`), reject a pre-existing PINNED_DIR that isn't actually a worktree of BUZZ_AUDITOR_SRC, and stop force-removing/rm -rf'ing a mismatched worktree -- report it and let a human decide. Also verify Cargo.toml's own resolved dependency path canonicalizes to the pinned worktree, not just that the worktree exists. scripts/test-mike49-pin-nip-am-verify.sh covers clean pin, dirty tracked input, unexpected untracked source, wrong HEAD, and an unrelated existing destination -- all five prove nothing gets deleted on rejection. - testdata/generate_sanitizer_parity_expected_python_output.py: regenerates sanitizer_parity_expected_python_output.json from the pinned buzz-auditor revision's real `sanitize_pipeline_line`, so the committed capture is provably reproducible rather than a one-off transcription. `--check` diffs against the committed file (used here); `--write` re-captures deliberately. - command.rs `ipc_tests`: an isolated native fixture-run trace via `tauri::test::get_ipc_response` -- the real `#[tauri::command]`-generated IPC dispatch and JSON serialization on a MockRuntime webview, no real window/AppState/relay/identity. Proves the report crossing real Tauri IPC matches a direct call (modulo the fixture's intentionally per-call disposable event id/signer). This is what the Playwright/mockIPC spec cannot check, since mockIPC intercepts before reaching compiled Rust. - mike49-audit-fixture.spec.ts: relabeled its doc comment to say browser + mocked IPC explicitly, and points at the new native IPC test as the place compiled Rust dispatch/serialization is actually proven. Verified at this commit, Hermit-activated (rustc/cargo 1.95.0), against nip-am-verify pinned to buzz-auditor@83db37a: cargo test --manifest-path desktop/src-tauri/Cargo.toml 3201/3201 passed (was 3199, +2 for the new native IPC tests), cargo fmt --check clean, generate_sanitizer_parity_expected_python_output.py --check matches the committed capture, scripts/test-mike49-pin-nip-am-verify.sh 5/5 cases pass, pnpm typecheck clean. clippy run in progress; playwright re-run pending. Still fixture-only: no real keys/events, relay queries, archive reads, IPC attachment to a live session, or merge. That hold and Mike's explicit approval requirement are unchanged. Signed-off-by: Mike Roberts <mike@sdcs.io>
…, cargo-metadata resolution Compare the pinned worktree's full canonical git-common-dir path against the real buzz-auditor checkout's, not just their parent directories. Comparing parents let a worktree from an unrelated repo pass as long as its .git storage sat as a sibling of the real one under a shared parent (a worktree pool / CI checkout cache) and happened to land on the same commit hash. Added a same-parent-different-common-dir regression case that reproduces this with two independent repos sharing identical commit content/metadata. Replace the grep/sed read of Cargo.toml's "path = ..." field with `cargo metadata`, which is Cargo's own dependency resolver -- so the check proves what `cargo build` would actually use, not just what the manifest text parses to. Fails loudly with a named blocker if cargo or jq is unavailable, rather than silently skipping the check. Signed-off-by: Mike Roberts <mike@sdcs.io>
…edge check Use `cargo metadata --locked` instead of a freshly-recomputed resolution -- this fails on a missing/stale Cargo.lock instead of silently rewriting it, so the check verifies the same lock `cargo build` would actually use. Resolve the pinned package via buzz-desktop's own dependency edge (metadata.resolve.root -> its "nip_am_verify" dep -> that pkg id's manifest_path) instead of filtering `packages` by name. A name filter would match the first package called "nip-am-verify" anywhere in the resolved graph, not necessarily the one buzz-desktop's own Cargo.toml entry actually depends on. Updated the fixture test harness to generate a real, --locked-satisfying Cargo.lock per case (materializing the path dependency's target as a plain directory just long enough to run `cargo generate-lockfile`, then removing it so the pin script's own worktree add is what actually creates it). Signed-off-by: Mike Roberts <mike@sdcs.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Checkpoint 1 of the Desktop UI entry point for MIKE-49 (owner interface Proposal A — see buzz-auditor
docs/superhero/plans/mike-49-owner-interface-decision-brief.md). This PR is fixture-only and simulated-data-only:desktop/src-tauri/src/commands/mike49_audit/(fixture.rs,sanitize.rs,command.rs) builds a small, deterministic, disposable-key (nostr::Keys::generate()) fixture in-process, reusing the reviewednip-am-verifycrate'sverify_and_decrypt,SessionCoverageTracker, andcursor/pagination API unmodified.mike49_run_fixture_auditTauri command (registered inlib.rs'sinvoke_handler) takes noAppStateparameter and returns an explicit#[derive(Serialize)]report struct with"simulated": true/"dataSource": "in_memory_fixture"always present.Mike49AuditFixturePanel.tsx, mounted in the existingExperimentalFeaturesCard.tsx— no new navigation/routing pattern invented) with an explicit run button (no auto-run on mount), loading/success/failure states, a visibly distinct capped/incomplete-vs-complete pagination indicator, an empty-fixture-source probe, and a duplicate-run guard.Dependency decision (record before further code)
Tried a source-control dependency first:
cargo treefailed — Cargo's built-in (libgit2) transport could not authenticate to the private repo, even though the system CLI in the same environment authenticates fine (verified separately withssh -T git@github.com):net.git-fetch-with-cli = truewould likely fix this but is a repo-wide Cargo config change out of scope for this checkpoint.Fell back to a path dependency, per the plan:
Caveat, explicit and not hidden: this assumes
buzz-auditoris checked out as a sibling directory ofbuzz(REPOS/buzzandREPOS/buzz-auditorunder the same parent). It is not CI-portable and not merge-ready as-is — a checkpoint-1-only provisional choice pending vendoring, a fixed transport config, or a real registry dependency. Full reasoning is inline indesktop/src-tauri/Cargo.toml's comment above the dependency line.Sanitizer
sanitize.rsports buzz-auditor's Python sanitizer discipline (nip_am_pipeline_bridge.py/nip_am_verify_bridge.py/report_safety.py) to Rust: field allowlist, credential-shaped-value detection (including values nested inside a sub-object),kindtype-checked before allowlist lookup, numeric validators, per-kind blocked counts, fixed content-free error messages. Since this runs natively in one Rust process (no subprocess boundary like the Python bridge has), the real safety property is structural: only the explicit#[derive(Serialize)]report struct ever crosses into the Tauri command's return value. The sanitizer is wired live into the command path (not just unit-tested decoration) as a second, redundant gate.Explicitly NOT touched / NOT done
native_relay_client— verified by greppingdesktop/src-tauri/src/commands/mike49_audit/for those names (only appears in doc comments explaining what is absent).Verification (real output, not estimates)
cargo check --manifest-path desktop/src-tauri/Cargo.toml -p buzz-desktop --lib— clean, 0 warnings.cargo clippy --manifest-path desktop/src-tauri/Cargo.toml --workspace --all-targets -- -D warnings(both default and--features mesh-llmgraphs, matchingjust desktop-tauri-clippy) — clean.cargo fmt --manifest-path desktop/src-tauri/Cargo.toml --all -- --check— clean.cd desktop/src-tauri && cargo test --workspace— 3198 passed, 0 failed, 19 ignored in the mainbuzz_libbinary (includes 25 new tests incommands::mike49_audit::{fixture,sanitize,command}::tests), plus allbuzz-terminalsub-crate suites green.pnpm typecheck(desktop) — clean, exit 0.pnpm check(biome) — no new errors; 4 warnings / 5 infos are pre-existing, in unrelated files.pnpm check:px-text,pnpm check:pubkey-truncation— clean.pnpm test(desktop) — 6494 passed, 0 failed, including 6 new component tests inMike49AuditFixturePanel.test.mjsproving idle/loading/success (records + capped/incomplete pagination + empty-source probe)/empty-records/failure/duplicate-run-guard states against a stubbed Tauri IPC layer.node desktop/scripts/check-file-sizes.mjs— clean (largest new file:command.rs, 679/1500 lines).Not done: launching the actual Tauri app for a visual/manual check — not practical in this environment. The component test file is the closest available substitute; a human should still click through Settings > Experiments once to visually confirm layout/styling.
Linked to https://linear.app/cccareersorg/issue/MIKE-49
Follow-up: reproducibility + evidence (commit 3baa50f)
Hannibal's review of aea3f8e found three gaps in the checkpoint-1 evidence. Fixed at 3baa50f:
1. Reproducible dependency pin, not a mutable sibling checkout.
scripts/mike49-pin-nip-am-verify.shreads the exact commit fromdesktop/src-tauri/NIP_AM_VERIFY_PINNED_REV(83db37a6f8fec5663227ceeb439b21c30d5d19ac) and materializes it as a detached-HEADgit worktreeofcccareers/buzz-auditorat<this-checkout-toplevel>/.mike49-nip-am-verify-pinned— byte-exact regardless of what branch the siblingREPOS/buzz-auditorcheckout happens to be on.Cargo.toml's path dependency now points at that fixed location, which also fixes the previously-documented nested-worktree relative-path break (../../../resolved to the wrong place from inside.claude/worktrees/<id>/) without a manual edit and without vendoring buzz-auditor source into block/buzz. Verified:git -C .mike49-nip-am-verify-pinned rev-parse HEAD== the pinned rev;cargo tree -p buzz-desktop -i nip-am-verifyresolves to that exact worktree path.2. Sanitizer parity against the pinned Python reference, with compared outputs. New test
record_sanitizer_matches_the_pinned_python_reference_on_shared_examplesincommand.rs.testdata/sanitizer_parity_examples.json(10 scenarios: valid, invalid enum/nested-type values, credential-shaped values both top-level and nested insidecost.note, an unrecognized field, a malformed hex id) is fed to both this crate'sRECORD_ALLOWEDand a frozen, captured-verbatim run of the realcollector.nip_am_pipeline_bridge.sanitize_pipeline_linefrom agit worktreeof buzz-auditor pinned to the same83db37a— not hand-written expectations. One scenario (related_event_id_present) is asserted as a documented, safe divergence: Python's allowlist validates a field this command's own fixture producer never emits, soRECORD_ALLOWEDcorrectly omits it (narrower, never over-trusting).3. Actual command-to-renderer flow, not just component tests. New
desktop/tests/e2e/mike49-audit-fixture.spec.ts, wired intosrc/testing/e2eBridge.ts's real mock-IPC layer (mockIPCfrom@tauri-apps/api/mocks), run viapnpm test:e2e:smoke. Three specs exercise the real click →invoke("mike49_run_fixture_audit")→ render path (not a mocked React prop): explicit-run idle→loading→success with the two real fixture scenarios rendered, a rejected-invoke failure state showing the fixed error copy, and the duplicate-run guard (a second dispatched click during loading does not trigger a secondinvokecall, verified by intercept-countingwindow.__TAURI_INTERNALS__.invoke). This is still not the compiled native Tauri app — a human click-through is still the only way to visually confirm layout/styling — but it is a real IPC round trip through a rendered page, not JSDOM-rendering the component in isolation.Full verification at 3baa50f, Hermit-activated (
rustc/cargo1.95.0), pinned dependency, from this exact worktree:cargo test(workspace) — 3199/3199 passed, 0 failed, 19 ignored (was 3198; +1 for the new parity test).cargo clippy --all-targets -- -D warnings— 0 warnings.cargo fmt --check— clean.pnpm typecheck— clean.pnpm test(component) — 6494/6494 passed.pnpm check(biome + px-text + pubkey-truncation) — no new errors (4 warnings / 5 infos are pre-existing, unrelated files).just file-size-check— clean.pnpm build:e2e && npx playwright test --project=smoke mike49-audit-fixture— 3/3 passed.Still fixture-only, still no real keys/events/relay/archive access, no merge.