mirror of
https://github.com/binaricat/Netcatty.git
synced 2026-11-22 16:10:50 +00:00
* fix(sftp): parallel folder discovery and unblock transfer stalls
Address #2712 / #2556 leftovers after the pipelined fail-closed work
(#2449 / #2720 / #2761): keep bulk body I/O on concurrent READ/WRITE only,
surface upload digest prep as verifying instead of a silent hang, make the
fastPut path-gate wait cancelable (#2755), parallelize multi-select enqueue,
and bound-parallelize directory listings so folder totals appear faster.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* feat(sftp): folder pre-scan and skip unchanged files
Absorb rsync strengths without an rsync backend: optionally walk the
full tree before transfer for stable file totals, and skip files whose
size and mtime already match the destination.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): drop unused DiscoveredTransferFile import
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix: address Codex review on PR #2772
* fix(sftp): address Codex review on path gate, aliases, and worker drain
- Invalidate same-transfer fallback when a force-released path gate gets a late truncating OPEN
- Use per-branch ancestors so sibling symlink aliases copy under pre-scan BFS
- Keep interleaved subdirectory walks sequential for deterministic manifest indices
- Drain transfer/bounded workers before propagating errors; ASCII punctuation
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): canonical pre-scan file order and pre-2001 mtime skip
Sort discovered files with compareDirectoryTraversalPaths before index
assignment, and treat timestamps >= 1e11 as milliseconds so year-2000
local mtimes still match SFTP second precision.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): bound multi-select downloads and preserve dest mtime
Address Codex review on PR #2772: serialize local blob reads, bound
multi-select root downloads before pre-scan, replace remaining Unicode
em dashes, and stamp destination mtimes after successful transfers so
skip-unchanged can match on a later pass.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): keep in-place OPEN poison until callback settles
Address Codex review on e2cc8241: do not force-release path gates for
in-place truncating OPENs, fail closed on pending-OPEN wait timeouts
without clearing poison, and replace remaining Unicode em dashes in
ASCII-only transfer modules.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): fail closed when in-place OPEN poison is held
When an isolated in-place truncating OPEN leaves the path gate poisoned
after channel error/cancel, mark the transfer terminal (noTransferFallback)
so fastPut/concurrent-shared cannot wait forever on the unreleased prior
gate.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): omit cyclic dirs from pre-scan plan and lower mtime cutoff
Only record discovered directories after a visit is accepted so symlink
cycles are not mkdir'd as empty targets, and treat magnitudes >= 1e10 as
milliseconds so early-1970s local Date values still match SFTP seconds.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): count max-depth symlink gaps and skip symlink quick-check
Pre-scan now surfaces max-depth symlink directory omissions as transfer
errors (matching interleaved walk), and discovered file symlinks carry
isSymlink so skip-unchanged does not trust link-node size/mtime.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): unit-aware skip mtimes and interleaved symlink bypass
Declare mtimeUnit on skip identities so early-1970 millisecond values
normalize without magnitude cutoffs, and bypass skip-unchanged for
symlinks on the interleaved (non-prescan) directory walk as well.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): bound local pre-scan and preserve SCP destination mtimes
Apply the same directory/entry traversal budget to local folder pre-scan,
and best-effort touch SCP upload targets after commit so skip-unchanged
can match on a later pass.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): address Codex review on pause, skip restat, mtime bound
- Honor transfer pause during folder pre-scan BFS waves/listings
- Re-stat source immediately before skip-unchanged on pre-scanned files
- Bound best-effort destination setStat/utimes so sendComplete cannot hang
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): harden skip restat and pre-transfer mtime capture
- Require a fresh source stat before skip-unchanged (prescan + interleaved)
- Capture source soft identity before bytes move for non-resumable/SCP
- Drop post-commit mtime re-stat fallback; keep ASCII in touched comments
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): propagate fresh re-stat size into interleaved transfers
Use the re-statted source size/mtime for child tasks on the non-prescan
path so grown files are not truncated to the stale listing size.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): omit stale listing size after re-stat failure
When source re-stat fails, clear child totalBytes/mtime so
startStreamTransfer omits an explicit snapshot and the bridge
re-stats on the dedicated session instead of truncating to the
stale listing size.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
* fix(sftp): make soft-identity capture cancelable
Race pre-transfer source soft-identity capture against cancel and
pass the transfer abort signal into requireSftpChannel so a dead
SFTP reopen cannot block cancel for the channel timeout.
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: 陈大猫 <binaricat@users.noreply.github.com>
Co-authored-by: netcatty-bot <308658023+netcatty-bot@users.noreply.github.com>
48 lines
1.9 KiB
TypeScript
48 lines
1.9 KiB
TypeScript
/**
|
|
* rsync-style "generator" skip: when size and mtime already match the target,
|
|
* the file does not need to be transferred again.
|
|
*
|
|
* SFTP mtimes are typically second-precision; compare on whole seconds so a
|
|
* local millisecond timestamp does not defeat an otherwise identical remote.
|
|
*/
|
|
|
|
export type TransferMtimeUnit = "ms" | "s";
|
|
|
|
export interface TransferSkipIdentity {
|
|
size: number;
|
|
lastModified: number;
|
|
/**
|
|
* Preferred: adapters declare the unit. Local fs and Netcatty SFTP stats use
|
|
* milliseconds (`Date` / `mtime * 1000`). Raw ssh2 attrs use seconds.
|
|
* When omitted, a magnitude heuristic is used only as a legacy fallback.
|
|
*/
|
|
mtimeUnit?: TransferMtimeUnit;
|
|
}
|
|
|
|
export function normalizeTransferMtimeSeconds(
|
|
lastModified: number,
|
|
mtimeUnit?: TransferMtimeUnit,
|
|
): number {
|
|
if (!Number.isFinite(lastModified) || lastModified <= 0) return 0;
|
|
if (mtimeUnit === "ms") return Math.floor(lastModified / 1000);
|
|
if (mtimeUnit === "s") return Math.floor(lastModified);
|
|
// Legacy heuristic: magnitudes at/above 1e10 (~year 2286 as seconds) are ms.
|
|
// Prefer passing mtimeUnit from adapters so early-1970 ms values are exact.
|
|
return lastModified >= 1e10 ? Math.floor(lastModified / 1000) : Math.floor(lastModified);
|
|
}
|
|
|
|
export function isUnchangedTransferCandidate(
|
|
source: TransferSkipIdentity,
|
|
target: TransferSkipIdentity,
|
|
): boolean {
|
|
const sourceSize = Number(source.size);
|
|
const targetSize = Number(target.size);
|
|
if (!Number.isFinite(sourceSize) || !Number.isFinite(targetSize)) return false;
|
|
if (sourceSize !== targetSize) return false;
|
|
if (sourceSize < 0 || targetSize < 0) return false;
|
|
const sourceMtime = normalizeTransferMtimeSeconds(source.lastModified, source.mtimeUnit);
|
|
const targetMtime = normalizeTransferMtimeSeconds(target.lastModified, target.mtimeUnit);
|
|
if (sourceMtime <= 0 || targetMtime <= 0) return false;
|
|
return sourceMtime === targetMtime;
|
|
}
|