Files
Netcatty/infrastructure/ai/sessionExecutionQueue.test.ts
T
陈大猫andClaude Opus 4.7 cab4fc36ab Serialize Catty terminal_execute per session to fix the parallel tool_use race (#1108)
* fix(ai): serialize Catty terminal_execute calls per session

Issue #1101 problem 3. When the LLM emits multiple tool_use blocks in
one assistant turn (DeepSeek's Anthropic-compat happily does this for
parallel info-gathering), the Vercel AI SDK dispatches every tool
through `Promise.all(toolCalls.map(execute))`. All of them then race
into the same per-session mutex inside the main-process bridge
(`mcpServerBridge.reserveSessionExecution`). One wins; the rest get
`{ ok: false, error: "Session already has another command in
progress..." }` synthesized back as the tool result. The LLM sees a
turn full of synthetic errors instead of the answers it asked for, the
UI cards look stuck on "executing", and the Anthropic API has
sometimes rejected the resulting trace as
`tool_use ids were found without tool_result blocks`.

Adds an in-renderer Promise-chain queue keyed by
`${chatSessionId}:${terminalSessionId}` so the actual
`bridge.aiExec()` calls are issued one at a time per terminal. The LLM
can still parallel-call as many tool_use blocks as it wants — they
just resolve sequentially with real output. Approval prompts are kept
*outside* the queue: three parallel tool_use blocks still surface
three approval cards together, so the user can dispatch them in any
order rather than waiting for each prior command to finish before the
next prompt appears. The bridge-side mutex stays as defense-in-depth
for non-LLM paths (terminal_start, MCP, etc.).

`chainBySessionKey` is a self-contained helper with cleanup logic so
the queue map doesn't leak across many short-lived sessions:
- Each task awaits the previous tail via `.then(task, task)` so a
  failure doesn't poison the chain.
- A non-rejecting wrapper is stored as the new tail to keep that
  contract regardless of how the task settles.
- The cleanup `finally` only deletes the map entry when the current
  tail is still ours, so a caller that arrived between
  `set` and `finally` doesn't have its tail evicted.

Four unit tests cover dispatch ordering, rejection isolation,
per-key parallelism, and the cleanup path. Full suite 1255/1255.

Refs #1101.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ai): preserve LLM emission order + honor abort in serialized queue

Codex local review on the first cut of this branch flagged two real
issues plus a weak cleanup test:

1. **Approval order vs queue order** — the previous version awaited
   approval *before* reserving a queue slot. With three parallel
   tool_use blocks the user could approve B's prompt first, and B
   would slip into the queue ahead of A, running out of LLM emission
   order. Reserving the slot synchronously up front fixes that:
   `Promise.all`-dispatched executes each grab a slot at the same
   instant in their dispatch order, then wait on approval while
   holding it, then await `slot.ready` and run.

2. **Abort not honored** — once approvals were settled, queued-but-
   not-started commands ran to completion even after the user hit
   Stop. Their results were ignored by the SDK but the side effects
   still happened. Two `abortSignal` checks now short-circuit before
   the queue wait and again after.

3. **Cleanup test was vacuous** — the previous "drains" test would
   pass even if the cleanup logic were removed. Exposed
   `getSessionExecutionQueueSizeForTests` and assert the Map is empty
   after the only queued task settles.

Queue API now has two entry points: high-level `chainBySessionKey`
for run-it-when-it's-your-turn callers, and lower-level
`reserveSessionSlot` for callers (like `terminal_execute`) that
need to do non-blocking pre-work in parallel with the queue wait.

Two new tests cover the reservation-order-vs-prework-order guarantee
and the skip-without-serialized-work path.

Refs #1101.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ai): re-issue cancel on abort to close IPC-transit race

Codex local review (round 2) flagged a small remaining race: between
the abort check after \`slot.ready\` and the main-process bridge
registering the new exec into \`activePtyExecs\`, the user can hit
Stop. \`handleStop\` does issue \`aiCattyCancelExec\`, but if that
cancel IPC arrives at main *before* the exec has finished
registering, the cancel finds nothing to cancel and the exec keeps
running. Result: the user already cancelled, but the command runs
anyway.

Plug the gap by re-issuing the cancel from the tool's own abort
listener. Once the exec has registered into \`activePtyExecs\` (a
synchronous step on the main side once the IPC lands), the duplicate
cancel finds the entry and cancels it. \`cancelPtyExecsForSession\`
is already idempotent — it iterates the live tracker and skips
entries with mismatched \`chatSessionId\` — so double-firing from
both \`handleStop\` and the tool is safe.

Extends \`NetcattyBridge\` with the optional
\`aiCattyCancelExec(chatSessionId)\` method. Already exposed in
\`electron/preload.cjs\` so the runtime call works on the live
bridge; the type addition just makes it visible to the tool.

Refs codex review on #1101 problem 3 fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ai): register pending-cancel marker before SSH exec-channel opens

Codex local review (round 3) caught a remaining race in
`execViaChannel`: the cancellation marker is only registered inside
`sshClient.exec`'s open callback, which is async. If a cancel arrives
in the window between `sshClient.exec(...)` being dispatched and the
callback firing, `cancelPtyExecsForSession` finds nothing for the
session and is a no-op. The channel then opens, the marker
registers, and the command runs to completion — exactly the "user
already cancelled, but the command still ran" failure mode.

Plug the window by registering a *pending* marker synchronously
before `sshClient.exec`. The pending marker carries a `cancel()` that
just latches a `cancelled` flag and a `cleanup()` that is a no-op.
When the open callback fires it removes the pending marker and
checks the latch: if it tripped, close the just-opened stream and
resolve with `{ ok: false, error: "Cancelled" }`. If not, the normal
post-open registration takes over with the real `execStream` close.

The PTY (`execViaPty`) and raw-serial (`execViaRawPty`) paths
already register their marker before any async wait, so they don't
share this race.

Two regression tests in `ptyExec.test.cjs`:
- The pending marker is present immediately after `execViaChannel`
  returns, before the open callback runs.
- A cancel during the pre-open window short-circuits the eventual
  callback with `{ ok: false, error: "Cancelled" }` and closes the
  now-unwanted stream.

Refs codex review on #1101 problem 3 fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ai): clean up pending-cancel marker when sshClient.exec throws sync

Codex local review (round 4) noted that if `sshClient.exec(...)`
itself throws synchronously — e.g. because the underlying ssh2 client
was destroyed between the session lookup and the actual `.exec`
call — the pending-cancel marker stays in `activePtyExecs`
indefinitely and the surrounding Promise rejects instead of
resolving with the normal `{ ok, error }` shape the tool layer
expects.

Wraps the `sshClient.exec` invocation in `try/catch` so:
- Pending marker is removed when the throw escapes.
- The Promise resolves cleanly with `{ ok: false, error: err.message }`
  so the tool layer gets the same shape it does for every other
  failure mode (closed stream, timeout, cancellation).

Adds a regression test covering this exact path: a fake client whose
`.exec` throws synchronously; the result is a clean failure and the
cancellation map is empty afterwards.

Refs codex review on #1101 problem 3 fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 00:45:37 +08:00

188 lines
5.8 KiB
TypeScript

import assert from "node:assert/strict";
import test from "node:test";
import {
chainBySessionKey,
getSessionExecutionQueueSizeForTests,
reserveSessionSlot,
resetSessionExecutionQueueForTests,
} from "./shared/sessionExecutionQueue";
function defer<T = void>(): { promise: Promise<T>; resolve: (value: T) => void; reject: (err: unknown) => void } {
let resolve!: (value: T) => void;
let reject!: (err: unknown) => void;
const promise = new Promise<T>((res, rej) => {
resolve = res;
reject = rej;
});
return { promise, resolve, reject };
}
// Drain pending microtasks and immediates so cleanup `.finally`s have run.
async function flushMicrotasks(): Promise<void> {
for (let i = 0; i < 5; i += 1) {
await new Promise((r) => setImmediate(r));
}
}
test("chainBySessionKey serializes calls sharing the same key in dispatch order", async (t) => {
t.afterEach(resetSessionExecutionQueueForTests);
const events: string[] = [];
const gateA = defer();
const gateB = defer();
const gateC = defer();
const a = chainBySessionKey("session-1", async () => {
events.push("a:start");
await gateA.promise;
events.push("a:end");
return "a";
});
const b = chainBySessionKey("session-1", async () => {
events.push("b:start");
await gateB.promise;
events.push("b:end");
return "b";
});
const c = chainBySessionKey("session-1", async () => {
events.push("c:start");
await gateC.promise;
events.push("c:end");
return "c";
});
await flushMicrotasks();
assert.deepEqual(events, ["a:start"]);
gateA.resolve();
await flushMicrotasks();
assert.deepEqual(events, ["a:start", "a:end", "b:start"]);
gateB.resolve();
await flushMicrotasks();
assert.deepEqual(events, ["a:start", "a:end", "b:start", "b:end", "c:start"]);
gateC.resolve();
assert.equal(await a, "a");
assert.equal(await b, "b");
assert.equal(await c, "c");
});
test("chainBySessionKey does not let one task's rejection poison the next", async (t) => {
t.afterEach(resetSessionExecutionQueueForTests);
const a = chainBySessionKey("session-2", async () => {
throw new Error("a boom");
});
const b = chainBySessionKey("session-2", async () => "b ok");
await assert.rejects(a, /a boom/);
assert.equal(await b, "b ok");
});
test("chainBySessionKey isolates work across keys (different sessions run in parallel)", async (t) => {
t.afterEach(resetSessionExecutionQueueForTests);
const gateA = defer();
const gateB = defer();
const a = chainBySessionKey("session-a", async () => {
await gateA.promise;
return "a";
});
const b = chainBySessionKey("session-b", async () => {
await gateB.promise;
return "b";
});
// Resolve B first; it should finish even though A is still blocked.
gateB.resolve();
assert.equal(await b, "b");
gateA.resolve();
assert.equal(await a, "a");
});
test("queue Map drops the entry once the last task drains", async (t) => {
t.afterEach(resetSessionExecutionQueueForTests);
resetSessionExecutionQueueForTests();
assert.equal(getSessionExecutionQueueSizeForTests(), 0);
await chainBySessionKey("session-cleanup", async () => "done");
await flushMicrotasks();
// Without the cleanup `finally`, the resolved tail would stay parked
// in the map; this is the regression guard that the previous version
// of this test was missing.
assert.equal(getSessionExecutionQueueSizeForTests(), 0);
});
test("reserveSessionSlot fixes order at reservation time regardless of pre-work duration", async (t) => {
t.afterEach(resetSessionExecutionQueueForTests);
const events: string[] = [];
// Simulate three tool_use calls firing in parallel where each has
// some pre-work (e.g. an approval prompt) of varying length. The
// serialized work order must still match the slot-reservation order.
async function run(name: string, prework: Promise<void>) {
const slot = reserveSessionSlot("session-order");
try {
events.push(`${name}:reserved`);
await prework;
events.push(`${name}:prework-done`);
await slot.ready;
events.push(`${name}:serial-start`);
// Trivial serial work, just so we can observe the start order.
await new Promise((r) => setImmediate(r));
events.push(`${name}:serial-end`);
return name;
} finally {
slot.release();
}
}
const gateA = defer();
const gateB = defer();
const gateC = defer();
const a = run("A", gateA.promise);
const b = run("B", gateB.promise);
const c = run("C", gateC.promise);
// Approvals land in reverse order — C first, then B, then A.
await flushMicrotasks();
gateC.resolve();
await flushMicrotasks();
gateB.resolve();
await flushMicrotasks();
gateA.resolve();
await flushMicrotasks();
assert.deepEqual(await Promise.all([a, b, c]), ["A", "B", "C"]);
const serialStarts = events.filter((e) => e.endsWith(":serial-start"));
assert.deepEqual(
serialStarts,
["A:serial-start", "B:serial-start", "C:serial-start"],
`serial work ran in reservation order, not approval order; got: ${events.join(", ")}`,
);
});
test("reserveSessionSlot lets later slots proceed once an early slot releases without serialized work", async (t) => {
t.afterEach(resetSessionExecutionQueueForTests);
const events: string[] = [];
async function maybeRun(name: string, run: boolean) {
const slot = reserveSessionSlot("session-skip");
try {
events.push(`${name}:reserved`);
if (!run) return `${name}:skipped`;
await slot.ready;
events.push(`${name}:ran`);
return `${name}:ran`;
} finally {
slot.release();
}
}
// Slot A reserves but skips serial work (mimics user denying approval
// or aborting). Slot B should still proceed.
const a = maybeRun("A", false);
const b = maybeRun("B", true);
assert.equal(await a, "A:skipped");
assert.equal(await b, "B:ran");
});