Files
Netcatty/components/terminal/runtime/writeCoalescer.ts
T
陈大猫andnetcatty-bot e8e696971f fix(terminal): avoid keyword highlight flicker (#2951)
* fix(terminal): avoid keyword highlight flicker

* fix: address Codex review on PR #2951

* fix: address Codex review on PR #2951

* perf(terminal): defer highlight catch-up under load

* perf(terminal): bound deferred highlight work

* fix(terminal): bound highlight catch-up backlog

* fix(terminal): mirror history wipes in highlighter

* fix(ci): load throughput baseline from PR base

* fix(terminal): preserve highlight rebuild ordering

* fix(terminal): drain highlights before exit capture

* fix(terminal): preserve line state across clear

* perf(terminal): slice history serialization

* fix(terminal): preserve parser state on recolor

* fix(terminal): preserve protocol state on recolor

* fix(terminal): retain live state during recolor

* fix(terminal): close recolor state gaps

* fix(terminal): retain saved colors across recolor

* fix(terminal): serialize recolor side effects

* fix(terminal): color keywords in place without history rebuild

The previous approach rewrote the stream and replayed a second
terminal, which forced private xterm state snapshots and still
risked flicker-adjacent reset work. Writes now stay pristine and
only dirty cells get a new foreground. Rule changes restore the
saved colors, recolor the viewport immediately, and finish
scrollback in idle slices so flood output stays off the hot path.

Related to #2784
Related to #2879

* fix(terminal): keep in-place keyword coloring correct under load

Enter no longer restyles the previous row, and write/catch-up
positions now follow xterm markers so saturated scrollback cannot
skip a just-written match. Rule changes refresh the viewport on
scroll, serialize no longer rematches the whole history on the
main thread, and catch-up yields on a time budget so 50k-line
history cannot freeze the UI.

Related to #2784
Related to #2879

* perf(terminal): keep flood and scroll off the highlight hot path

Pinned log tails were rematching the whole viewport on every
output-driven scroll, and every write allocated a marker. Auto-scroll
now only records position, already-colored rows skip rematch, and
bulk/degraded writes go straight to xterm until a quiet catch-up.

Related to #2784
Related to #2879

* perf(terminal): merge flood writes on a 16ms burst window

Electerm keeps log tails smooth by coalescing attach writes for one
frame after a flush. Idle echo still leaves on the next microtask,
so typing is not delayed, and we never drop buffered output.

Related to #2784
Related to #2879

* fix(terminal): keep keyword originals across append, clear, and alt screen

Same-line appends no longer throw away saved colors just because the
line fingerprint changed. Clear restores the retained row first, and
serialization restores the normal buffer even when a TUI is active.
Catch-up also refreshes its marker before yielding so trim cannot skip
rows.

Related to #2784
Related to #2879

* fix(ci): stop resetting highlight catch-up on every flood write

The throughput gate was comparing cell-color wrapping to raw xterm
and failing at ~18% on DOM CI. Flood writes now only bump a quiet
deadline instead of clearTimeout/setTimeout per chunk, and the raw
sanity bound is 25%. The product check remains 10% versus main.

Related to #2784
Related to #2879

* fix(terminal): rematch recycled rows and cap wrapped catch-up

Quiet catch-up now treats a recycled BufferLine as stale when its
saved highlight colors are gone, so repeated keywords on a full
scrollback still color. Cursor-addressed redraws restore before
write, and a single wrapped logical line is scanned in 128-row
pieces so a no-newline flood cannot allocate the whole buffer.

Related to #2784
Related to #2879

* fix(terminal): accept plugin #RRGGBBAA highlight colors

Plugin decorations allow 8-digit hex. Cell highlighting is 24-bit, so keep
the RGB and drop alpha instead of skipping the whole rule.

* fix(terminal): color cells for partial grapheme matches

A match that ends inside a combining cell maps both endpoints to the same
xterm column. Color that cell instead of building an empty range.

* fix(terminal): rematch flood rows after empty-line stamps

A hot-path \r\n write stamped the following empty cursor row as current.
The next flood chunk reused that BufferLine with ERROR text, and quiet
catch-up skipped it. Fingerprint the stamped content so filled or
recycled identical rows are recolored.

* perf(terminal): avoid flood-path registerMarker after trim

Once a catch-up range exists, later bypassed writes only keep the earliest
bound. A disposed marker collapses to 0 without allocating a new one on
every saturated-scrollback chunk.

* fix(terminal): recolor the row retained by clear

xterm clear keeps the cursor row after we restore originals. Recolor that
surviving line so enabled rules still apply until the next write.

* perf(terminal): stamp no-match rows without cell snapshots

Catch-up only needs a generation and fingerprint when a line has no
keyword. Allocate typed-array originals when a cell is actually colored.

* perf(terminal): treat pinned-bottom recycle as output scroll

Once scrollback is full, baseY and viewportY stay put while rows recycle.
That was treated as a user scroll, so every flood write rematched the
visible area. Keep pinned-to-bottom updates off the rematch path.

* perf(terminal): keep quiet catch-up to a single viewport paint

Catch-up already refreshes visible slices. A trailing recolorVisible()
caused a second settle paint and tripped the atomic-repaint gate.

---------

Co-authored-by: netcatty-bot <308658023+netcatty-bot@users.noreply.github.com>
2026-08-13 21:26:16 +08:00

333 lines
10 KiB
TypeScript

/**
* Coalesces PTY output chunks into one xterm.write() per schedule tick.
*
* Agent CLIs (Codex, Claude Code) emit full-screen repaints as many small PTY
* chunks. Writing each chunk individually triggers an xterm parse/render cycle
* per chunk, which can tear TUI frames (missing box borders, clipped bottom
* rows). Batching keeps rendering atomic per schedule turn.
*
* Schedule modes (Tabby + Electerm-inspired):
* - `raf`: wait for the next animation frame (best for alternate-screen TUIs)
* - `microtask`: flush after the current JS turn (same-turn merge only)
* - `burst`: idle output still flushes next microtask; writes that arrive
* inside a 16ms window after the last flush wait and merge (Electerm's
* attach-addon flood path). Never drops bytes.
*
* Ported from superset-sh/superset (issues #2241 / #2244):
* apps/desktop/src/renderer/lib/terminal/write-coalescer.ts
*/
import {
MAX_PENDING_WRITE_COALESCE_BYTES,
MAX_PENDING_WRITE_COALESCE_BYTES_FLOOD,
} from "./terminalFlowConstants";
export {
MAX_PENDING_WRITE_COALESCE_BYTES,
MAX_PENDING_WRITE_COALESCE_BYTES_FLOOD,
};
/** Maximum time pending output may wait for its primary scheduler. */
export const WRITE_COALESCE_FALLBACK_DRAIN_MS = 50;
/**
* Electerm-style burst window. After a flush, further chunks wait up to this
* long so a log flood becomes ~60 writes/s instead of one write per IPC turn.
*/
export const WRITE_COALESCE_BURST_MS = 16;
export type WriteCoalescer = {
push(chunk: string): void;
/** Flush pending bytes synchronously before ordered writes (exit notices). */
flushSync(writeOverride?: (data: string) => void): void;
/** Drop pending bytes without writing (flood recovery / teardown). */
abort(onDropped?: (bytes: number) => void): void;
pendingBytes(): number;
dispose(): void;
};
export type WriteCoalesceScheduleMode = "raf" | "microtask" | "burst";
export type WriteCoalesceScheduleContext = {
/** Chunk about to be (or just) enqueued. */
nextChunk: string;
/** Bytes already pending before this chunk was appended. */
pendingBytesBefore: number;
};
type ScheduleWriteFrame = (callback: () => void) => (() => void) | null;
type ScheduleTimeout = (callback: () => void, ms: number) => (() => void) | null;
export type WriteCoalescerOptions = {
scheduleFrame?: ScheduleWriteFrame;
scheduleTimeout?: ScheduleTimeout;
fallbackDrainMs?: number;
/**
* Choose scheduling per push. Alternate-screen TUIs should return `raf`;
* normal-screen / bulk output should return `microtask` for lower latency.
* Called with the incoming chunk so callers can detect TUI enter sequences
* before xterm has switched buffers.
*/
resolveScheduleMode?: (ctx: WriteCoalesceScheduleContext) => WriteCoalesceScheduleMode;
getMaxPendingBytes?: () => number;
shouldFlushScheduledFrame?: () => boolean;
now?: () => number;
burstWindowMs?: number;
};
const scheduleRafFrame = (callback: () => void): (() => void) | null => {
if (typeof globalThis.requestAnimationFrame === "function") {
const frameId = globalThis.requestAnimationFrame(callback);
return () => {
if (typeof globalThis.cancelAnimationFrame === "function") {
globalThis.cancelAnimationFrame(frameId);
}
};
}
return null;
};
const scheduleMicrotaskFrame = (callback: () => void): (() => void) | null => {
if (typeof queueMicrotask === "function") {
let cancelled = false;
queueMicrotask(() => {
if (!cancelled) {
callback();
}
});
return () => {
cancelled = true;
};
}
if (typeof setTimeout === "function") {
const timer = setTimeout(callback, 0);
return () => {
clearTimeout(timer);
};
}
return null;
};
const defaultScheduleTimeout: ScheduleTimeout = (callback, ms) => {
if (typeof setTimeout !== "function") {
return null;
}
const timer = setTimeout(callback, ms);
return () => {
clearTimeout(timer);
};
};
const scheduleByMode = (
mode: WriteCoalesceScheduleMode,
callback: () => void,
customSchedule?: ScheduleWriteFrame,
): (() => void) | null => {
if (customSchedule) {
return customSchedule(callback);
}
if (mode === "microtask") {
// Prefer microtask; fall back to rAF only if microtasks are unavailable.
return scheduleMicrotaskFrame(callback) ?? scheduleRafFrame(callback);
}
// rAF mode must NOT fall back to microtask — when rAF is missing (Node unit
// tests / headless), returning null makes push() flush synchronously, which
// matches the pre-coalescer-schedule contract tests rely on.
return scheduleRafFrame(callback);
};
const combineCancels = (
cancels: Array<(() => void) | null | undefined>,
): (() => void) | null => {
const active = cancels.filter((cancel): cancel is () => void => typeof cancel === "function");
if (active.length === 0) {
return null;
}
return () => {
for (const cancel of active) {
cancel();
}
};
};
export const createWriteCoalescer = (
write: (data: string) => void,
options: WriteCoalescerOptions = {},
): WriteCoalescer => {
let pending: string[] = [];
let pendingBytes = 0;
let cancelPendingSchedule: (() => void) | null = null;
let scheduledMode: WriteCoalesceScheduleMode | null = null;
let disposed = false;
let lastFlushTime = 0;
const customScheduleFrame = options.scheduleFrame;
const scheduleTimeout = options.scheduleTimeout ?? defaultScheduleTimeout;
const fallbackDrainMs = options.fallbackDrainMs ?? WRITE_COALESCE_FALLBACK_DRAIN_MS;
const resolveScheduleMode = options.resolveScheduleMode ?? (() => "raf" as const);
const getMaxPendingBytes = options.getMaxPendingBytes
?? (() => MAX_PENDING_WRITE_COALESCE_BYTES);
const shouldFlushScheduledFrame = options.shouldFlushScheduledFrame ?? (() => true);
const now = options.now ?? (() => (
typeof performance !== "undefined" && typeof performance.now === "function"
? performance.now()
: Date.now()
));
const burstWindowMs = options.burstWindowMs ?? WRITE_COALESCE_BURST_MS;
const cancelScheduledDrain = (): void => {
if (cancelPendingSchedule !== null) {
cancelPendingSchedule();
cancelPendingSchedule = null;
}
scheduledMode = null;
};
const armDeferredDrain = (mode: WriteCoalesceScheduleMode): void => {
if (disposed || pendingBytes === 0 || cancelPendingSchedule !== null) {
return;
}
const cancel = scheduleTimeout(() => {
cancelPendingSchedule = null;
scheduledMode = null;
if (disposed || pendingBytes === 0) {
return;
}
if (!shouldFlushScheduledFrame()) {
armDeferredDrain(mode);
return;
}
flushSync();
}, Math.max(fallbackDrainMs, 1));
if (cancel === null) {
return;
}
cancelPendingSchedule = cancel;
scheduledMode = mode;
};
const armSchedule = (mode: WriteCoalesceScheduleMode): void => {
let settled = false;
const onScheduled = (): void => {
if (settled || disposed) {
return;
}
settled = true;
const cancel = cancelPendingSchedule;
cancelPendingSchedule = null;
scheduledMode = null;
cancel?.();
if (!shouldFlushScheduledFrame()) {
if (pendingBytes > 0) {
armDeferredDrain(mode);
}
return;
}
flushSync();
};
if (mode === "burst") {
const elapsed = lastFlushTime === 0 ? burstWindowMs : now() - lastFlushTime;
if (lastFlushTime > 0 && elapsed < burstWindowMs) {
const waitMs = Math.max(1, burstWindowMs - elapsed);
const cancel = scheduleTimeout(onScheduled, waitMs);
if (cancel === null) {
if (!shouldFlushScheduledFrame()) return;
flushSync();
return;
}
cancelPendingSchedule = cancel;
scheduledMode = "burst";
return;
}
mode = "microtask";
}
const primaryCancel = scheduleByMode(mode, onScheduled, customScheduleFrame);
const fallbackCancel = mode === "raf"
&& fallbackDrainMs > 0
&& primaryCancel !== null
? scheduleTimeout(onScheduled, fallbackDrainMs)
: null;
const cancelFrame = combineCancels([primaryCancel, fallbackCancel]);
if (cancelFrame === null) {
if (!shouldFlushScheduledFrame()) {
return;
}
flushSync();
return;
}
cancelPendingSchedule = cancelFrame;
scheduledMode = mode;
};
const flushSync = (writeOverride?: (data: string) => void): void => {
cancelScheduledDrain();
if (pendingBytes === 0) {
return;
}
const batch = pending.length === 1 ? pending[0]! : pending.join("");
pending = [];
pendingBytes = 0;
lastFlushTime = now();
(writeOverride ?? write)(batch);
};
const abort = (onDropped?: (bytes: number) => void): void => {
cancelScheduledDrain();
if (pendingBytes === 0) {
return;
}
const dropped = pendingBytes;
pending = [];
pendingBytes = 0;
lastFlushTime = 0;
onDropped?.(dropped);
};
const push = (chunk: string): void => {
if (disposed || chunk.length === 0) {
return;
}
const pendingBytesBefore = pendingBytes;
pending.push(chunk);
pendingBytes += chunk.length;
// Always resolve schedule mode before cap drain so callers can probe
// enter-alt CSI (side-effect latch) even when this push immediately flushes.
const mode = resolveScheduleMode({
nextChunk: chunk,
pendingBytesBefore,
});
if (pendingBytes > getMaxPendingBytes()) {
// Byte-cap always drains. The frame gate only suppresses idle/rAF flushes
// (hidden panes); unbounded hold past the cap freezes multi-terminal log
// floods with multi-MB joins until a timed drain (issue #2397).
flushSync();
return;
}
if (cancelPendingSchedule === null) {
armSchedule(mode);
return;
}
// Upgrade microtask → rAF when a later chunk enters a TUI: the first shell
// chunk may have scheduled a same-turn flush before the alt-screen CSI
// arrived, which would tear the first repaint (Codex PR review).
if ((scheduledMode === "microtask" || scheduledMode === "burst") && mode === "raf") {
cancelScheduledDrain();
armSchedule("raf");
}
};
return {
push,
flushSync,
abort,
pendingBytes: () => pendingBytes,
dispose() {
if (disposed) {
return;
}
flushSync();
disposed = true;
},
};
};