Compare commits

..
75 Commits
Author SHA1 Message Date
Akizon 9a4d730507 Merge pull request #706 from carexdk/fix/theme-extract-dir-perm
fix(theme): create extracted directories with 0755
2026-11-07 11:32:06 +08:00
Akizon 10d9463c06 移除/优化部分测试,允许空的ping任务 2026-11-06 15:38:06 +08:00
komari-fix 63ffadd49b fix(theme): create extracted directories with 0755
Theme archives frequently store directory entries without the execute
bit (e.g. 0664). Theme extraction passed that mode straight to
os.MkdirAll, producing directories that cannot be entered, so writing
the first file inside them failed with EACCES:

    创建文件失败: open data/theme/<name>/dist/__next.__PAGE__.txt: permission denied

This makes market installation fail for affected themes. Create all
extracted directories with 0755 instead; individual file modes keep
being taken from the archive.
2026-11-06 15:35:15 +08:00
Akizon 96cf0f79d8 插件支持分类了喵 2026-10-26 11:58:41 +08:00
Akizon77 9e5f9a3a71 让插件支持监听ws 2026-10-24 16:41:01 +08:00
Akizon 70bbceaf09 支持了新功能在pgsql和mysql的兼容 2026-10-23 15:47:20 +08:00
Akizon e30b9fdbd4 优化查询的CPU性能,但是稍微牺牲一点点内存 2026-10-23 13:33:19 +08:00
Akizon d5689efeea 优化查询性能 2026-10-23 11:07:55 +08:00
Akizon c24c4b9c7c 允许插件修改自己的内建对象 2026-10-22 15:02:05 +08:00
Akizon d50cc0a9fd JS运行时换成moejs 2026-10-22 13:20:51 +08:00
Akizon 528cea23be Merge pull request #695 from cristianchiriac/fix-ping-queue-size
fix(metricstore): give the ping queue its own capacity
2026-10-20 09:00:35 +08:00
Akizon b7042672db Merge main into PR 695 and remove added queue test 2026-10-20 08:45:10 +08:00
Akizon 7468dcb12b 感谢新的金主 MooProxy 2026-10-19 19:59:23 +08:00
Akizon dc7f4cb541 感谢新的金主 MooProxy 2026-10-19 19:58:11 +08:00
Akizon 48af63ca7b 查询时对象分配数量可以少一点 2026-10-19 17:22:10 +08:00
Akizon 3cf8445ad4 优化单次查询性能 2026-10-19 15:00:24 +08:00
Akizon b38b0b2347 让opus5.5 简化、优化1下代码 2026-10-19 11:19:34 +08:00
cristianchiriac fa5a04d8e2 fix(metricstore): give the ping queue its own capacity
2c3f228 lowered reportBatchQueueSize from 4096 to 512, and the ping queue
shared that size. A ping schedule round from many clients arrives within
a second or two, while the batcher drains every 3 seconds, so results
past the 512th were rejected with ErrPingBatchQueueFull. The latest
results, from the slowest routes, were the ones lost.

Size the ping queue with its own pingBatchQueueSize of 4096, the value
before 2c3f228. The report queue stays at 512.

Fixes #694
2026-10-17 17:41:45 +03:00
Akizon 4b24f1e196 Update README_zh-cn.md 2026-10-17 15:06:20 +08:00
Akizon 5006f55ca0 Update payment options in README
Removed WeChat Pay section from README and retained TRON Network.
2026-10-17 15:05:27 +08:00
Akizon 1b76b7ad5b js通知沿用旧版本的逻辑 2026-10-14 17:00:19 +08:00
Akizon 13cbdca545 和浏览器1起协商ws压缩算法 2026-10-10 17:32:08 +08:00
Akizon e48aa1e769 加1个通知升级提醒 2026-10-08 10:04:56 +08:00
Akizon a68704a425 js通知先加回来 2026-10-08 09:31:51 +08:00
Akizon 39cb3f59b2 识别长时间断报优化一下 2026-09-28 11:35:26 +08:00
Akizon 9812acfd11 Agent支持切换版本协议 2026-09-23 13:59:50 +08:00
Akizon 70df32096f 对于#675,有一个更简洁的方案可以试1下 2026-09-23 10:38:09 +08:00
Akizon 29d309f4a1 支持拿到Agent的运行参数了 2026-09-23 09:26:51 +08:00
Akizon 7d692d2faf 剪切板也是1个用的比较少的功能,移除了喵 2026-09-22 15:15:37 +08:00
Akizon 68c284d043 xterm设置适用面有1点太少了 2026-09-22 09:32:05 +08:00
Akizon 14bc87d1e9 没有落地的访客审计功能先删掉吧 2026-09-21 17:02:03 +08:00
Akizon 154d2b8af2 添加1个小参数优化内存 2026-09-21 10:00:27 +08:00
Akizon 2c3f2281fa 1点点小参数优化内存 2026-09-21 09:58:03 +08:00
Akizon 7e91794f24 优化内存的分配喵,再压1点! 2026-09-20 16:52:33 +08:00
Akizon f57461f87f 裁剪1下 2026-09-20 15:32:23 +08:00
Akizon d15f0512c7 重写1下通知 2026-09-20 14:01:47 +08:00
Akizon 4ec22edf1f 重定向旧版本的主题接口? 2026-09-20 09:29:32 +08:00
Akizon77 0ca87aafd1 移除流量通知 2026-09-14 21:13:26 +08:00
Akizon f18ad72c17 Merge pull request #669 from MousseC/fix/upgrade-request-detection
fix(plugin): 仅当 Connection 与 Upgrade 头同时存在时判定协议升级请求
2026-09-14 13:44:49 +08:00
Akizon 8e4a3f8709 Merge pull request #672 from MiaM1ku/fix/latest-status-gpu-usage
fix: getNodesLatestStatus 写入实时 GPU 占用
2026-09-14 13:43:40 +08:00
Nya 80add06a12 fix: getNodeRecentStatus 同样写入 GPU 占用
该接口扁平化近期上报时 Gpu 也被写死为 0,与 getNodesLatestStatus 同类问题。
2026-09-11 23:56:18 +08:00
Nya 1333d63ce0 fix: getNodesLatestStatus 写入实时 GPU 占用
此前 recordLike.Gpu 被写死为 0,主题实时 GPU 一直显示 0,而历史 gpu.usage 正常。改为使用 report.GPU.AverageUsage,并附带 gpu_count / gpu_average_usage / gpu_detailed_info。
2026-09-11 23:52:18 +08:00
MousseC 44ed3035f1 chore(plugin): shorten isUpgradeRequest doc comment 2026-09-08 20:34:23 +08:00
MousseC 7ec36157d5 test(plugin): cover upgrade detection and injection behind bare Connection header 2026-09-08 18:41:58 +08:00
MousseC 360fd5f50b fix(plugin): detect protocol upgrades only when both headers are present
isUpgradeRequest previously flagged any request carrying a bare
`Connection: upgrade` header -- or a bare `Upgrade: websocket` header --
as a protocol upgrade, bypassing plugin HTML injection and
request/response hooks. Some reverse-proxy templates (e.g. sites created
by 1Panel) set `proxy_set_header Connection upgrade;` unconditionally, so
every proxied page request skipped the plugin middleware silently while
plugins still reported as loaded.

Per RFC 9110 section 7.8 a valid upgrade request carries both headers:
Connection lists the upgrade token and Upgrade names the target protocol.
Require the Upgrade header to be present and the Connection list to
contain the upgrade token, and accept any upgrade protocol rather than
only websocket.
2026-09-08 18:41:55 +08:00
Akizon77 b11ffd3aa7 使用zstd压缩前端,节约空间! 2026-09-04 16:16:41 +08:00
Akizon77 d7d6135c49 让SSRF策略可以调整1下! 2026-09-04 13:59:37 +08:00
Akizon77 ac037f2177 换了1个更好的按安装脚本! 2026-09-04 11:43:42 +08:00
Akizon77 5405bdf377 使用人类友好的时间格式 2026-09-02 16:09:16 +08:00
Akizon f754f01f53 Merge pull request #652 from YZoiu/fix/metric-store-remote-write-timeout
fix(metricstore): break report counter restore retry loop on slow remote databases
2026-09-02 15:41:27 +08:00
Akizon77 434ba108e1 后续版本将不再支持v1协议了捏 2026-09-02 15:30:30 +08:00
Yuzhong ZhangandAkizon77 24254749b5 现在只剩v2了 (#653)
Co-authored-by: Akizon77 <hytf233g@gmail.com>
2026-09-02 15:01:04 +08:00
Akizon c6b02af000 感谢新的金主喵 2026-09-01 15:50:56 +08:00
Akizon b28ac9d42c 感谢新的金主喵 2026-09-01 15:50:03 +08:00
Akizon77 2ff7c11db0 refactor: remove v1 protocol and stabilize file transfer 2026-08-29 19:03:53 +08:00
Akizon77 46fd2af44c feat: 自动推断分片大小 2026-08-29 15:03:14 +08:00
Akizon77 4d2eb70c62 fix: 分片文件操作过早关闭io导致502 2026-08-29 12:38:32 +08:00
Akizon77 7a6692b79a pref: 文件上传下载链路 2026-08-29 11:24:22 +08:00
Akizon77 63d78b6ed4 pref: 下载速度优化 2026-08-28 23:41:27 +08:00
Akizon77 2319ea77f6 fix: 上传大小限制 2026-08-28 22:02:27 +08:00
Akizon77 0c39a52e0b feat: file access / editor 2026-08-28 20:11:18 +08:00
Akizon77 e31a032d89 feat(terminal): support session reattachment on WebSocket reconnect 2026-08-26 14:40:01 +08:00
Akizon77 da4d5187c1 docs: update Rainyun deployment descriptions 2026-08-17 14:59:36 +08:00
Akizon77 bf6b45ec3a fix(metricstore): #646 skip traffic deltas on agent restart or counter reset 2026-08-13 17:10:59 +08:00
Akizon77 4a3400bee6 feat(notifier): warn users to migrate before traffic report removal 2026-08-13 15:45:56 +08:00
Akizon77 4caa067556 feat(plugin): reload plugin after config update 2026-08-13 14:28:32 +08:00
Akizon77 2f3959adde fix(child_process): stop inheriting runtime timeout for child processes
Replace default runtime timeout with explicit timeout only, avoiding unexpected termination of long-running child processes when a runtime-level timeout is configured.
2026-08-13 14:13:26 +08:00
Akizon77 63a7ffddcf fix(plugin): normalize RPC results before resolving promises
RPC results were passed directly to goja's resolve, which could fail for
non-JSON-compatible Go types. Add normalizeRPCValue to round-trip the
result through JSON, ensuring it becomes plain data structures. Handle
serialization errors properly by rejecting the promise.
2026-08-13 13:05:12 +08:00
Akizon77 92d679ffdb feat(console): format values as JSON and error messages
Improve console output by replacing fmt.Sprint with a dedicated
value formatter that renders null, strings, errors, and JSON-serializable
objects more intuitively for JavaScript runtime users.
2026-08-13 11:45:10 +08:00
Akizon77 7c30fc4dc9 feat: #635 add historical backup cleanup and harden upgrade rollback 2026-08-13 09:43:05 +08:00
Akizon77 773a3715dc feat(upload): add chunked upload store for themes/plugins 2026-08-12 22:38:07 +08:00
Akizon77 75d7b6c7a3 feat(migration): cap structure upgrade progress at 80% 2026-08-12 20:38:31 +08:00
Akizon77 c294c17642 feat: add managed configuration selectors 2026-08-12 16:46:05 +08:00
Akizon77 2883bf6295 feat: add raw SQL query and exec RPC methods 2026-08-11 17:41:46 +08:00
Akizon77 14bf8bde7c build: use COPY --chmod in Dockerfile 2026-08-07 16:57:13 +08:00
291 changed files with 18392 additions and 10839 deletions

No files matched your search

+15 -3
View File
@@ -23,6 +23,15 @@ runs:
with:
node-version: ${{ inputs.node-version }}
- name: Install zstd
shell: bash
run: |
set -euxo pipefail
if ! command -v zstd >/dev/null 2>&1; then
sudo apt-get update
sudo apt-get install -y zstd
fi
- name: Clone and build frontend
shell: bash
run: |
@@ -38,9 +47,12 @@ runs:
npm run build
cd ..
mkdir -p web/public/defaultTheme/dist
rm -rf web/public/defaultTheme/dist/*
cp -r komari-web/dist/* web/public/defaultTheme/dist/
mkdir -p web/public/defaultTheme
rm -rf web/public/defaultTheme/dist web/public/defaultTheme/dist.tar.zst
test -f komari-web/dist/index.html
tar -cf "$RUNNER_TEMP/komari-default-dist.tar" -C komari-web/dist .
zstd -19 -T0 -q -f "$RUNNER_TEMP/komari-default-dist.tar" -o web/public/defaultTheme/dist.tar.zst
rm -f "$RUNNER_TEMP/komari-default-dist.tar"
cp -f komari-web/komari-theme.json web/public/defaultTheme/
if [ -f komari-web/preview.png ]; then cp -f komari-web/preview.png web/public/defaultTheme/; fi
if [ -f komari-web/perview.png ]; then cp -f komari-web/perview.png web/public/defaultTheme/; fi
@@ -22,3 +22,4 @@ runs:
mkdir -p web/public/defaultTheme
rm -rf web/public/defaultTheme/*
tar -xzf "$RUNNER_TEMP/frontend-artifact/${{ inputs.artifact-name }}.tar.gz" -C web/public/defaultTheme
test -s web/public/defaultTheme/dist.tar.zst
+5 -1
View File
@@ -98,7 +98,11 @@ jobs:
fi
VERSION="${{ github.sha }}"
VERSION_HASH="${{ github.sha }}"
go build -trimpath -ldflags="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}" -o $BINARY_NAME
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
if [ "${{ matrix.goos }}" = "linux" ]; then
LDFLAGS="$LDFLAGS -extldflags=-Wl,--strip-all"
fi
go build -trimpath -ldflags="$LDFLAGS" -o "$BINARY_NAME"
- name: Upload binary as artifact
uses: actions/upload-artifact@v4
+1 -6
View File
@@ -14,11 +14,6 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4
#- name: Create dummy index.html
# run: |
# mkdir -p web/public/defaultTheme/dist
# echo "<html><body><h1>This is komari development server.</h1></body></html>" > web/public/defaultTheme/dist/index.html
- name: Clone and build frontend
uses: ./.github/actions/build-frontend
@@ -33,7 +28,7 @@ jobs:
GOARCH: amd64
run: |
VERSION_SHA="${{ github.sha }}"
go build -ldflags "-X github.com/komari-monitor/komari/utils.CurrentVersion=dev -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_SHA}" -o komari .
go build -ldflags "-s -w -extldflags=-Wl,--strip-all -X github.com/komari-monitor/komari/utils.CurrentVersion=dev -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_SHA}" -o komari .
- name: Upload binary as artifact
uses: actions/upload-artifact@v4
+1 -1
View File
@@ -50,7 +50,7 @@ jobs:
run: |
VERSION="${{ github.ref_name }}"
VERSION_HASH="${{ github.sha }}"
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
LDFLAGS="-s -w -extldflags=-Wl,--strip-all -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
echo "Building for linux/amd64..."
GOARCH=amd64 CC="zig cc -target x86_64-linux-musl" go build -trimpath -ldflags="$LDFLAGS" -o komari-linux-amd64
+5 -1
View File
@@ -102,7 +102,11 @@ jobs:
run: |
BINARY_NAME=komari-${{ matrix.goos }}-${{ matrix.goarch }}${{ matrix.binary_extension }}
VERSION_HASH="$(git rev-parse HEAD)"
go build -trimpath -ldflags="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${RELEASE_TAG} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}" -o "$BINARY_NAME"
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${RELEASE_TAG} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
if [ "${{ matrix.goos }}" = "linux" ]; then
LDFLAGS="$LDFLAGS -extldflags=-Wl,--strip-all"
fi
go build -trimpath -ldflags="$LDFLAGS" -o "$BINARY_NAME"
- name: Upload binary to release
env:
+1 -1
View File
@@ -56,7 +56,7 @@ jobs:
VERSION: ${{ github.event.inputs.version }}
run: |
VERSION_HASH="${{ github.sha }}"
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
LDFLAGS="-s -w -extldflags=-Wl,--strip-all -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
echo "Building for linux/amd64..."
GOARCH=amd64 CC="zig cc -target x86_64-linux-musl" go build -trimpath -ldflags="$LDFLAGS" -o komari-linux-amd64
+5 -1
View File
@@ -121,7 +121,11 @@ jobs:
fi
VERSION="${{ github.event.release.tag_name }}"
VERSION_HASH="${{ github.sha }}" # Use commit SHA for hash
go build -trimpath -ldflags="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}" -o $BINARY_NAME
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
if [ "${{ matrix.goos }}" = "linux" ]; then
LDFLAGS="$LDFLAGS -extldflags=-Wl,--strip-all"
fi
go build -trimpath -ldflags="$LDFLAGS" -o "$BINARY_NAME"
- name: Upload binary as artifact
uses: actions/upload-artifact@v4
+5 -1
View File
@@ -150,7 +150,11 @@ jobs:
fi
VERSION="${{ needs.prepare.outputs.version }}"
VERSION_HASH="${{ github.sha }}" # Use commit SHA for hash
go build -trimpath -ldflags="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}" -o $BINARY_NAME
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
if [ "${{ matrix.goos }}" = "linux" ]; then
LDFLAGS="$LDFLAGS -extldflags=-Wl,--strip-all"
fi
go build -trimpath -ldflags="$LDFLAGS" -o "$BINARY_NAME"
- name: Upload binary as artifact
uses: actions/upload-artifact@v4
+57
View File
@@ -0,0 +1,57 @@
name: Test
on:
pull_request:
branches:
- main
push:
branches:
- dev
workflow_dispatch:
concurrency:
group: test-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
cache: true
- name: Download dependencies
run: go mod download
# NOTE: the repository stores Go sources with CRLF line endings, which
# gofmt normalises to LF, so "gofmt -l" currently lists ~327 files. Until
# that is normalised repo-wide this step is advisory only; flip it to a
# hard failure once the tree is clean.
- name: Formatting report (advisory)
run: |
count=$(gofmt -l . | grep -v '^web/node_modules' | wc -l)
echo "gofmt reports $count file(s) needing formatting."
gofmt -l . | grep -v '^web/node_modules' | head -20 || true
- name: Vet
run: go vet ./...
- name: Test
run: go test ./... -count=1 -timeout 10m
- name: Test (repeat for isolation)
run: go test ./... -count=2 -shuffle=on -timeout 15m
- name: Race detector
run: go test -race ./pkg/metric/ ./internal/metricstore/ ./internal/plugin/ ./web/connection/ ./web/api/terminal/ -count=1 -timeout 15m
- name: Coverage
run: |
go test ./... -count=1 -coverpkg=./... -coverprofile=coverage.out -timeout 15m
go tool cover -func=coverage.out | tail -1
+3
View File
@@ -15,6 +15,7 @@ komari.db
/utils/geoip/data
.vscode/
/backup
/cache
# Test binary, built with `go test -c`
*.test
@@ -36,3 +37,5 @@ go.work.sum
komari
/dist/
.codex
.gocache/
/web/filemanager/data
+2 -4
View File
@@ -8,13 +8,11 @@ ARG TARGETARCH
RUN apk add --no-cache ca-certificates curl tzdata
COPY komari-${TARGETOS}-${TARGETARCH} /app/komari
RUN chmod +x /app/komari
COPY --chmod=755 komari-${TARGETOS}-${TARGETARCH} /app/komari
ENV GIN_MODE=release
ENV KOMARI_LISTEN=0.0.0.0:25774
ENV GODEBUG=disablethp=1
EXPOSE 25774
+9 -7
View File
@@ -24,10 +24,10 @@ Komari is a lightweight, self-hosted server monitoring solution. It provides a s
## Quick Start
| Platform | Description |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| <a href="https://app.rainyun.com/apps/rca/store/6780/NzYxNzAz_"><img src="https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-cn.svg" alt="Rainyun" width="180"></a> | Deploy websites, databases, and hundreds of popular apps in seconds with flexible hourly billing. |
| <a href="https://apps.fit2cloud.com/1panel/komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/1panel-logo-blue.png" alt="1Panel App Store" width="180"></a> | A modern, open-source Linux server management panel for websites, databases, containers, files, backups, security, and AI, with one-click deployment from its app store. |
| Platform | Description |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <a href="https://app.rainyun.com/apps/rca/store/6780/NzYxNzAz_"><img src="https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-cn.svg" alt="Rainyun" width="180"></a> | Deploy websites, databases, and hundreds of popular apps in seconds with flexible hourly billing. [Get started for just ¥5/month. Deploy now!](https://app.rainyun.com/apps/rca/store/6780/NzYxNzAz_) |
| <a href="https://apps.fit2cloud.com/1panel/komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/1panel-logo-blue.png" alt="1Panel App Store" width="180"></a> | A modern, open-source Linux server management panel for websites, databases, containers, files, backups, security, and AI, with one-click deployment from its app store. |
For instructions on Docker deployment, binary installation, building from source, and updates, see the [installation guide](https://www.komari.wiki/en/install/quick-start).
@@ -48,6 +48,8 @@ Interested in sponsoring Komari? Contact the developer via [email](mailto:komari
| Sponsor | Description |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <a href="https://mooproxy.net/?utm_source=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/Mooproxy-logo-transparent.png" alt="mooproxy.net" width="180"></a> | [MooProxy · Rotating residential proxies from $0.20/GB. Sign up for 500MB of free traffic and try it today!](https://mooproxy.net/?utm_source=komari) |
| <a href="https://axisnow.io/zh?utm=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/AxisNow.jpg" alt="AxisNow" width="180"></a> | [Self-Hosted Private CDN \| Subscription-Based CDN-Like Service \| A Fully Controlled, Flexible, Modular CDN Network](https://axisnow.io/zh?utm=komari) |
| <a href="https://whmcs.as211392.com/aff.php?aff=110"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/dreamcloud.png" alt="Dream Cloud" width="180"></a> | Cost-effective Asia-Pacific hosting with direct connectivity and robust DDoS protection, backed by transparent capacity claims. |
| <a href="https://sharon.io"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/sharon-networks.webp" alt="Sharon Networks" width="180"></a> | Premium China-optimized connectivity from Asia-Pacific data centers, featuring low latency, high bandwidth, and Tbps-scale local DDoS mitigation. Join the [Telegram community](https://t.me/SharonNetwork) to participate in charitable initiatives and community giveaways. |
@@ -61,6 +63,6 @@ Thanks to everyone who has contributed code, themes, plugins, documentation, tra
If Komari has been useful to you, consider buying me a coffee. Thank you for your support!
| WeChat Pay | TRON Network |
| ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| ![WeChat Pay QR code](https://b2.akz.moe/awesome-pictures/%E5%BE%AE%E4%BF%A1%E8%B5%9E%E8%B5%8F%E7%A0%81.png) | ![TRON Network QR code](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
| TRON Network |
| ------------------------------------------------------------------------------------------- |
| ![TRON Network QR code](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
+9 -7
View File
@@ -24,10 +24,10 @@ Komari 是一款轻量级的自托管服务器监控工具,旨在提供简单
## 快速开始
| 平台 | 介绍 |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| <a href="https://app.rainyun.com/apps/rca/store/6780/NzYxNzAz_"><img src="https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-cn.svg" alt="Rainyun" width="180"></a> | 秒级部署网站、数据库及数百款热门 App,并采用按小时灵活计费。 |
| <a href="https://apps.fit2cloud.com/1panel/komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/1panel-logo-blue.png" alt="1Panel Appstore" width="180"></a> | 现代化、开源的Linux 服务器运维管理面板,提供网站、数据库、容器、文件、备份、安全与AI 管理能力,支持应用商店一键部署。 |
| 平台 | 介绍 |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| <a href="https://app.rainyun.com/apps/rca/store/6780/NzYxNzAz_"><img src="https://rainyun-apps.cn-nb1.rains3.com/materials/deploy-on-rainyun-cn.svg" alt="Rainyun" width="180"></a> | 秒级部署网站、数据库及数百款热门 App,并采用按小时灵活计费。[每月5元,立即部署](https://app.rainyun.com/apps/rca/store/6780/NzYxNzAz_) |
| <a href="https://apps.fit2cloud.com/1panel/komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/1panel-logo-blue.png" alt="1Panel Appstore" width="180"></a> | 现代化、开源的Linux 服务器运维管理面板,提供网站、数据库、容器、文件、备份、安全与AI 管理能力,支持应用商店一键部署。 |
Docker、二进制文件、源码构建和更新说明,请参阅 [安装指南](https://www.komari.wiki/install/quick-start).
@@ -48,6 +48,8 @@ Docker、二进制文件、源码构建和更新说明,请参阅 [安装指南
| 赞助商 | 描述 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <a href="https://mooproxy.net/?utm_source=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/Mooproxy-logo-transparent.png" alt="mooproxy.net" width="180"></a> | [MooProxy · 动态住宅代理IP低至 $0.20/GB,注册即送 500MB 免费流量,立即体验!](https://mooproxy.net/?utm_source=komari) |
| <a href="https://axisnow.io/zh?utm=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/AxisNow.jpg" alt="AxisNow" width="180"></a> | [自建私有部署CDN \| 订阅式高仿CDN \| 自主可控、灵活组合的CDN网络](https://axisnow.io/zh?utm=komari) |
| <a href="https://whmcs.as211392.com/aff.php?aff=110"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/dreamcloud.png" alt="Dream Cloud" width="180"></a> | 极高性价比解锁直连亚太高防,真高防,不虚标,打死退款 |
| <a href="https://sharon.io"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/sharon-networks.webp" alt="Sharon Networks" width="180"></a> | 亚太数据中心提供顶级的中国优化网络接入 · 低延时&高带宽&提供Tbps级本地清洗高防服务, 为您的业务保驾护航, 为您的客户提供极致体验. 加入社区 [Telegram群组](https://t.me/SharonNetwork) 可参与公益募捐或群内抽奖免费使用 |
@@ -61,6 +63,6 @@ Docker、二进制文件、源码构建和更新说明,请参阅 [安装指南
如果 Komari 对你有所帮助,欢迎请作者喝一杯奶茶。感谢你的支持!
| 微信赞赏码 | TRON Network |
| ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
| ![wechat](https://b2.akz.moe/awesome-pictures/%E5%BE%AE%E4%BF%A1%E8%B5%9E%E8%B5%8F%E7%A0%81.png) | ![TRON](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
| TRON Network |
| --------------------------------------------------------------------------- |
| ![TRON](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
-20
View File
@@ -1,20 +0,0 @@
package flags
import "testing"
func TestNormalizeDatabaseType(t *testing.T) {
tests := map[string]string{
"": DatabaseTypeSQLite,
"sqlite": DatabaseTypeSQLite,
" SQLite ": DatabaseTypeSQLite,
"SQLITE": DatabaseTypeSQLite,
"postgres": "postgres",
" postgres": "postgres",
}
for input, want := range tests {
if got := NormalizeDatabaseType(input); got != want {
t.Fatalf("NormalizeDatabaseType(%q) = %q, want %q", input, got, want)
}
}
}
+5
View File
@@ -6,6 +6,7 @@ import (
appserver "github.com/komari-monitor/komari/internal/server"
logger "github.com/komari-monitor/komari/utils/log"
frontendpublic "github.com/komari-monitor/komari/web/public"
"github.com/komari-monitor/komari/cmd/flags"
"github.com/spf13/cobra"
@@ -32,6 +33,10 @@ func init() {
// 具体各阶段的职责与顺序见 internal/server.App。这里只负责串联:
// 任一初始化阶段失败即中止启动,避免在半初始化状态下对外提供服务。
func RunServer() {
if err := frontendpublic.PrepareDefaultDist(); err != nil {
logger.Fatalf("server", "prepare default frontend failed: %v", err)
}
app := appserver.New(appserver.Options{ListenAddr: flags.Listen})
if err := app.Bootstrap(); err != nil {
_ = app.Shutdown()
+3 -13
View File
@@ -35,21 +35,11 @@ func Enable2Fa(uuid, secret string) error {
func Verify2Fa(uuid, code string) (bool, error) {
db := dbcore.GetDBInstance()
var user models.User
err := db.Where("uuid = ?", uuid).First(&user).Error
if err != nil {
if err := db.Where("uuid = ?", uuid).First(&user).Error; err != nil {
return false, err
}
if user.TwoFactor == "" {
return false, nil // 用户未启用2FA
}
valid := totp.Validate(code, user.TwoFactor)
if !valid {
return false, nil
}
return true, nil
// 用户未启用2FA时视为校验失败
return user.TwoFactor != "" && totp.Validate(code, user.TwoFactor), nil
}
func Disable2Fa(uuid string) error {
+24 -56
View File
@@ -14,15 +14,12 @@ import (
const constantSalt = "06Wm4Jv1Hkxx"
// CheckPassword 检查密码是否正确
//
// 如果密码正确,返回用户的 UUID 和 true;否则返回空字符串和 false
func CheckPassword(username, passwd string) (uuid string, success bool) {
// CheckPassword 检查密码,正确时返回用户 UUID 和 true
func CheckPassword(username, passwd string) (uuid string, ok bool) {
db := dbcore.GetDBInstance()
var user models.User
result := db.Where("username = ?", username).First(&user)
if result.Error != nil {
// 静默处理错误,不显示日志
return "", false
}
if hashPasswd(passwd) != user.Passwd {
@@ -32,7 +29,7 @@ func CheckPassword(username, passwd string) (uuid string, success bool) {
}
// ForceResetPassword 强制重置用户密码
func ForceResetPassword(username, passwd string) (err error) {
func ForceResetPassword(username, passwd string) error {
db := dbcore.GetDBInstance()
result := db.Model(&models.User{}).Where("username = ?", username).Update("passwd", hashPasswd(passwd))
if result.Error != nil {
@@ -46,82 +43,54 @@ func ForceResetPassword(username, passwd string) (err error) {
// hashPasswd 对密码进行加盐哈希
func hashPasswd(passwd string) string {
saltedPassword := passwd + constantSalt
hash := sha256.New()
hash.Write([]byte(saltedPassword))
hashedPassword := base64.StdEncoding.EncodeToString(hash.Sum(nil))
return hashedPassword
h := sha256.Sum256([]byte(passwd + constantSalt))
return base64.StdEncoding.EncodeToString(h[:])
}
func CreateAccount(username, passwd string) (user models.User, err error) {
return CreateAccountWithDB(dbcore.GetDBInstance(), username, passwd)
}
func CreateAccountWithDB(db *gorm.DB, username, passwd string) (user models.User, err error) {
hashedPassword := hashPasswd(passwd)
user = models.User{
func CreateAccountWithDB(db *gorm.DB, username, passwd string) (models.User, error) {
user := models.User{
UUID: uuid.New().String(),
Username: username,
Passwd: hashedPassword,
Passwd: hashPasswd(passwd),
}
err = db.Create(&user).Error
if err != nil {
return models.User{}, err
}
return user, nil
return user, db.Create(&user).Error
}
func DeleteAccountByUsername(username string) (err error) {
return DeleteAccountByUsernameWithDB(dbcore.GetDBInstance(), username)
}
func DeleteAccountByUsernameWithDB(db *gorm.DB, username string) (err error) {
err = db.Where("username = ?", username).Delete(&models.User{}).Error
if err != nil {
return err
}
return nil
func DeleteAccountByUsernameWithDB(db *gorm.DB, username string) error {
return db.Where("username = ?", username).Delete(&models.User{}).Error
}
func GetUserByUUID(uuid string) (user models.User, err error) {
func GetUserByUUID(uuid string) (models.User, error) {
var user models.User
return user, dbcore.GetDBInstance().Where("uuid = ?", uuid).First(&user).Error
}
// GetUserBySSO 通过 SSO 信息获取用户
func GetUserBySSO(ssoID string) (models.User, error) {
db := dbcore.GetDBInstance()
err = db.Where("uuid = ?", uuid).First(&user).Error
if err != nil {
return models.User{}, err
var user models.User
if err := db.Where("sso_id = ?", ssoID).First(&user).Error; err != nil {
return models.User{}, fmt.Errorf("用户不存在:%s", ssoID)
}
return user, nil
}
// 通过 SSO 信息获取用户
func GetUserBySSO(ssoID string) (user models.User, err error) {
db := dbcore.GetDBInstance()
// 首先尝试查找已存在的用户
err = db.Where("sso_id = ?", ssoID).First(&user).Error
if err == nil {
return user, nil
}
// 如果找不到用户,返回明确的错误信息
return models.User{}, fmt.Errorf("用户不存在:%s", ssoID)
}
func BindingExternalAccount(uuid string, sso_id string) error {
db := dbcore.GetDBInstance()
err := db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", sso_id).Error
if err != nil {
return err
}
return nil
return db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", sso_id).Error
}
func UnbindExternalAccount(uuid string) error {
db := dbcore.GetDBInstance()
err := db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", "").Error
if err != nil {
return err
}
return nil
return db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", "").Error
}
func UpdateUser(uuid string, name, password, sso_type *string) error {
@@ -143,8 +112,7 @@ func UpdateUser(uuid string, name, password, sso_type *string) error {
updates["sso_type"] = *sso_type
}
updates["updated_at"] = time.Now().UTC()
err := db.Model(&models.User{}).Where("uuid = ?", uuid).Updates(updates).Error
if err != nil {
if err := db.Model(&models.User{}).Where("uuid = ?", uuid).Updates(updates).Error; err != nil {
return err
}
if password != nil {
+220
View File
@@ -0,0 +1,220 @@
package accounts
import (
"testing"
"time"
"github.com/pquerna/otp/totp"
)
// TestHashPasswordIsSaltedAndStable pins the password hash contract: stable
// for the same input, different for different passwords, and never equal to
// the plaintext.
func TestHashPasswordIsSaltedAndStable(t *testing.T) {
a := hashPasswd("correct horse battery staple")
b := hashPasswd("correct horse battery staple")
if a != b {
t.Fatal("hash is not deterministic for the same password")
}
if hashPasswd("other password") == a {
t.Fatal("different passwords produced the same hash")
}
if a == "correct horse battery staple" {
t.Fatal("hash equals the plaintext")
}
if a == hashPasswd("correct horse battery staplex") {
t.Fatal("hash is not sensitive to a one-character change")
}
}
// TestCheckPassword covers the credential check used by the login endpoint.
func TestCheckPassword(t *testing.T) {
resetAccountsState(t)
user, err := CreateAccount("alice", "Strong-password1")
if err != nil {
t.Fatalf("create account: %v", err)
}
if uuid, ok := CheckPassword("alice", "Strong-password1"); !ok || uuid != user.UUID {
t.Fatalf("correct password rejected: uuid=%q ok=%v", uuid, ok)
}
if _, ok := CheckPassword("alice", "wrong-password"); ok {
t.Fatal("wrong password accepted")
}
if _, ok := CheckPassword("nobody", "Strong-password1"); ok {
t.Fatal("unknown user accepted")
}
}
// TestForceResetPasswordChangesCredential guards the admin recovery path.
func TestForceResetPasswordChangesCredential(t *testing.T) {
resetAccountsState(t)
if _, err := CreateAccount("bob", "Old-password1"); err != nil {
t.Fatalf("create account: %v", err)
}
if err := ForceResetPassword("bob", "New-password1"); err != nil {
t.Fatalf("force reset: %v", err)
}
if _, ok := CheckPassword("bob", "Old-password1"); ok {
t.Fatal("old password still works after reset")
}
if _, ok := CheckPassword("bob", "New-password1"); !ok {
t.Fatal("new password does not work after reset")
}
if err := ForceResetPassword("nobody", "New-password1"); err == nil {
t.Fatal("reset of a missing user should fail")
}
}
// TestSessionLifecycle covers creation, lookup and deletion.
func TestSessionLifecycle(t *testing.T) {
resetAccountsState(t)
user, err := CreateAccount("carol", "Strong-password1")
if err != nil {
t.Fatalf("create account: %v", err)
}
session, err := CreateSession(user.UUID, 3600, "ua", "127.0.0.1", "password")
if err != nil {
t.Fatalf("create session: %v", err)
}
if got, err := GetSession(session); err != nil || got != user.UUID {
t.Fatalf("GetSession = %q, %v; want %q", got, err, user.UUID)
}
if err := DeleteSession(session); err != nil {
t.Fatalf("delete session: %v", err)
}
if _, err := GetSession(session); err == nil {
t.Fatal("deleted session still resolves")
}
}
// TestExpiredSessionIsRejectedAndRemoved verifies GetSession both refuses an
// expired session and cleans it up.
func TestExpiredSessionIsRejectedAndRemoved(t *testing.T) {
resetAccountsState(t)
user, err := CreateAccount("dave", "Strong-password1")
if err != nil {
t.Fatalf("create account: %v", err)
}
// expires=-1 puts the record in the past.
session, err := CreateSession(user.UUID, -1, "ua", "127.0.0.1", "password")
if err != nil {
t.Fatalf("create session: %v", err)
}
if _, err := GetSession(session); err == nil {
t.Fatal("expired session resolved successfully")
}
if got := countSessions(t, "session = ?", session); got != 0 {
t.Fatalf("expired session was not removed, %d rows remain", got)
}
}
// TestRemoveExpiredSessionsOnlyDropsExpired pins the janitor's selectivity.
func TestRemoveExpiredSessionsOnlyDropsExpired(t *testing.T) {
resetAccountsState(t)
user, err := CreateAccount("erin", "Strong-password1")
if err != nil {
t.Fatalf("create account: %v", err)
}
live, err := CreateSession(user.UUID, 3600, "ua", "127.0.0.1", "password")
if err != nil {
t.Fatalf("create live session: %v", err)
}
stale, err := CreateSession(user.UUID, -1, "ua", "127.0.0.1", "password")
if err != nil {
t.Fatalf("create stale session: %v", err)
}
if err := RemoveExpiredSessions(); err != nil {
t.Fatalf("remove expired sessions: %v", err)
}
if _, err := GetSession(live); err != nil {
t.Fatalf("live session was removed: %v", err)
}
if _, err := GetSession(stale); err == nil {
t.Fatal("stale session survived the janitor")
}
}
// TestUpdateUserRotatesPasswordAndDropsSessions pins the rule that changing a
// password invalidates every existing session.
func TestUpdateUserRotatesPasswordAndDropsSessions(t *testing.T) {
resetAccountsState(t)
user, err := CreateAccount("frank", "Strong-password1")
if err != nil {
t.Fatalf("create account: %v", err)
}
session, err := CreateSession(user.UUID, 3600, "ua", "127.0.0.1", "password")
if err != nil {
t.Fatalf("create session: %v", err)
}
newPassword := "Rotated-password1"
if err := UpdateUser(user.UUID, nil, &newPassword, nil); err != nil {
t.Fatalf("update user: %v", err)
}
if _, err := GetSession(session); err == nil {
t.Fatal("session survived a password rotation")
}
if _, ok := CheckPassword("frank", newPassword); !ok {
t.Fatal("rotated password does not work")
}
if err := UpdateUser("missing-uuid", nil, &newPassword, nil); err == nil {
t.Fatal("updating a missing user should fail")
}
}
// TestTwoFactorLifecycle covers enable, verify and disable.
func TestTwoFactorLifecycle(t *testing.T) {
resetAccountsState(t)
user, err := CreateAccount("grace", "Strong-password1")
if err != nil {
t.Fatalf("create account: %v", err)
}
// Without a secret, verification must fail closed.
if ok, err := Verify2Fa(user.UUID, "123456"); err != nil || ok {
t.Fatalf("Verify2Fa before enabling = %v, %v; want false, nil", ok, err)
}
secret, img, err := Generate2Fa()
if err != nil {
t.Fatalf("generate 2fa: %v", err)
}
if secret == "" || img == nil {
t.Fatal("Generate2Fa returned an empty secret or image")
}
if err := Enable2Fa(user.UUID, secret); err != nil {
t.Fatalf("enable 2fa: %v", err)
}
code, err := totp.GenerateCode(secret, time.Now())
if err != nil {
t.Fatalf("generate totp code: %v", err)
}
if ok, err := Verify2Fa(user.UUID, code); err != nil || !ok {
t.Fatalf("Verify2Fa with a valid code = %v, %v; want true, nil", ok, err)
}
if ok, err := Verify2Fa(user.UUID, "000000"); err != nil || ok {
t.Fatalf("Verify2Fa with a wrong code = %v, %v; want false, nil", ok, err)
}
if err := Disable2Fa(user.UUID); err != nil {
t.Fatalf("disable 2fa: %v", err)
}
if ok, err := Verify2Fa(user.UUID, code); err != nil || ok {
t.Fatalf("Verify2Fa after disabling = %v, %v; want false, nil", ok, err)
}
}
// TestVerify2FaUnknownUserIsAnError distinguishes "no such user" from "wrong
// code" so callers do not silently treat a lookup failure as a pass.
func TestVerify2FaUnknownUserIsAnError(t *testing.T) {
resetAccountsState(t)
if _, err := Verify2Fa("missing-uuid", "123456"); err == nil {
t.Fatal("Verify2Fa for a missing user should return an error")
}
}
+47
View File
@@ -0,0 +1,47 @@
package accounts
import (
"os"
"testing"
"github.com/komari-monitor/komari/cmd/flags"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
// TestMain wires a shared in-memory SQLite database for the accounts tests.
func TestMain(m *testing.M) {
flags.DatabaseType = flags.DatabaseTypeSQLite
flags.DatabaseFile = "file:komari_accounts_test?mode=memory&cache=shared"
db := dbcore.GetDBInstance()
if sqlDB, err := db.DB(); err == nil {
sqlDB.SetMaxOpenConns(1)
}
os.Exit(m.Run())
}
// resetAccountsState clears users and sessions so each test starts clean.
func resetAccountsState(t *testing.T) {
t.Helper()
clear := func() {
db := dbcore.GetDBInstance()
if err := db.Where("1 = 1").Delete(&models.User{}).Error; err != nil {
t.Fatalf("clear users: %v", err)
}
if err := db.Where("1 = 1").Delete(&models.Session{}).Error; err != nil {
t.Fatalf("clear sessions: %v", err)
}
}
clear()
t.Cleanup(clear)
}
// countSessions returns how many session rows match the query.
func countSessions(t *testing.T, query string, args ...any) int64 {
t.Helper()
var count int64
if err := dbcore.GetDBInstance().Model(&models.Session{}).Where(query, args...).Count(&count).Error; err != nil {
t.Fatalf("count sessions: %v", err)
}
return count
}
+11 -33
View File
@@ -16,13 +16,10 @@ import (
)
// GetAllSessions 获取所有会话
func GetAllSessions() (sessions []models.Session, err error) {
func GetAllSessions() ([]models.Session, error) {
db := dbcore.GetDBInstance()
err = db.Find(&sessions).Error
if err != nil {
return nil, err
}
return sessions, nil
var sessions []models.Session
return sessions, db.Find(&sessions).Error
}
// CreateSession 创建新会话
@@ -57,19 +54,14 @@ func CreateSession(uuid string, expires int, userAgent, ip, login_method string)
}
}()
err := db.Create(&sessionRecord).Error
if err != nil {
return "", err
}
return session, nil
return session, db.Create(&sessionRecord).Error
}
// GetSession 根据会话 ID 获取 UUID
func GetSession(session string) (uuid string, err error) {
func GetSession(session string) (string, error) {
db := dbcore.GetDBInstance()
var sessionRecord models.Session
err = db.Where("session = ?", session).First(&sessionRecord).Error
if err != nil {
if err := db.Where("session = ?", session).First(&sessionRecord).Error; err != nil {
return "", err
}
@@ -83,32 +75,22 @@ func GetSession(session string) (uuid string, err error) {
}
func GetUserBySession(session string) (models.User, error) {
db := dbcore.GetDBInstance()
var sessionRecord models.Session
err := db.Where("session = ?", session).First(&sessionRecord).Error
if err != nil {
if err := dbcore.GetDBInstance().Where("session = ?", session).First(&sessionRecord).Error; err != nil {
return models.User{}, err
}
return GetUserByUUID(sessionRecord.UUID)
}
// DeleteSession 删除指定会话
func DeleteSession(session string) (err error) {
func DeleteSession(session string) error {
db := dbcore.GetDBInstance()
result := db.Where("session = ?", session).Delete(&models.Session{})
if result.Error != nil {
return result.Error
}
return nil
return db.Where("session = ?", session).Delete(&models.Session{}).Error
}
func DeleteAllSessions() error {
db := dbcore.GetDBInstance()
result := db.Where("1 = 1").Delete(&models.Session{})
if result.Error != nil {
return result.Error
}
return nil
return db.Where("1 = 1").Delete(&models.Session{}).Error
}
func UpdateLatest(session, useragent, ip string) error {
@@ -122,9 +104,5 @@ func UpdateLatest(session, useragent, ip string) error {
func RemoveExpiredSessions() error {
db := dbcore.GetDBInstance()
result := db.Where("expires < ?", time.Now().UTC()).Delete(&models.Session{})
if result.Error != nil {
return result.Error
}
return nil
return db.Where("expires < ?", time.Now().UTC()).Delete(&models.Session{}).Error
}
+13 -54
View File
@@ -16,12 +16,7 @@ import (
)
func DeleteClient(clientUuid string) error {
db := dbcore.GetDBInstance()
err := db.Delete(&models.Client{}, "uuid = ?", clientUuid).Error
if err != nil {
return err
}
return nil
return dbcore.GetDBInstance().Delete(&models.Client{}, "uuid = ?", clientUuid).Error
}
func SaveClientInfo(update map[string]interface{}) error {
@@ -31,11 +26,6 @@ func SaveClientInfo(update map[string]interface{}) error {
return fmt.Errorf("invalid client UUID")
}
// 确保更新的字段不为空
if len(update) == 0 {
return fmt.Errorf("no fields to update")
}
update["updated_at"] = time.Now().UTC()
toFloat64 := func(value interface{}) (float64, bool) {
@@ -114,11 +104,7 @@ func SaveClientInfo(update map[string]interface{}) error {
return err
}
err := db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(update).Error
if err != nil {
return err
}
return nil
return db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(update).Error
}
// CreateClient 创建新客户端
@@ -135,8 +121,7 @@ func CreateClient() (clientUUID, token string, err error) {
UpdatedAt: time.Now().UTC(),
}
err = db.Create(&client).Error
if err != nil {
if err := db.Create(&client).Error; err != nil {
return "", "", err
}
if err := tasks.AddDefaultOnClientUUID(clientUUID); err != nil {
@@ -160,8 +145,7 @@ func CreateClientWithName(name string) (clientUUID, token string, err error) {
UpdatedAt: time.Now().UTC(),
}
err = db.Create(&client).Error
if err != nil {
if err := db.Create(&client).Error; err != nil {
return "", "", err
}
if err := tasks.AddDefaultOnClientUUID(clientUUID); err != nil {
@@ -170,41 +154,25 @@ func CreateClientWithName(name string) (clientUUID, token string, err error) {
return clientUUID, token, nil
}
/*
// GetAllClients 获取所有客户端配置
func getAllClients() (clients []models.Client, err error) {
db := dbcore.GetDBInstance()
err = db.Find(&clients).Error
if err != nil {
return nil, err
}
return clients, nil
}
*/
func GetClientByUUID(uuid string) (client models.Client, err error) {
db := dbcore.GetDBInstance()
err = db.Where("uuid = ?", uuid).First(&client).Error
if err != nil {
func GetClientByUUID(uuid string) (models.Client, error) {
var client models.Client
if err := dbcore.GetDBInstance().Where("uuid = ?", uuid).First(&client).Error; err != nil {
return models.Client{}, err
}
return client, nil
}
func GetClientTokenByUUID(uuid string) (token string, err error) {
db := dbcore.GetDBInstance()
func GetClientTokenByUUID(uuid string) (string, error) {
var client models.Client
err = db.Where("uuid = ?", uuid).First(&client).Error
if err != nil {
if err := dbcore.GetDBInstance().Where("uuid = ?", uuid).First(&client).Error; err != nil {
return "", err
}
return client.Token, nil
}
func GetAllClientBasicInfo() (clients []models.Client, err error) {
db := dbcore.GetDBInstance()
err = db.Find(&clients).Error
if err != nil {
func GetAllClientBasicInfo() ([]models.Client, error) {
var clients []models.Client
if err := dbcore.GetDBInstance().Find(&clients).Error; err != nil {
return nil, err
}
return clients, nil
@@ -217,11 +185,6 @@ func SaveClient(updates map[string]interface{}) error {
return fmt.Errorf("invalid client UUID")
}
// 确保更新的字段不为空
if len(updates) == 0 {
return fmt.Errorf("no fields to update")
}
if v, exists := updates["traffic_limit"]; exists {
if val, ok := v.(float64); ok {
if val < 0 || val > math.MaxInt64-1 {
@@ -254,9 +217,5 @@ func SaveClient(updates map[string]interface{}) error {
updates["updated_at"] = time.Now().UTC()
err := db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(updates).Error
if err != nil {
return err
}
return nil
return db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(updates).Error
}
+6 -13
View File
@@ -6,22 +6,19 @@ import (
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
v1 "github.com/komari-monitor/komari/protocol/v1"
v2 "github.com/komari-monitor/komari/protocol/v2"
)
func GetClientUUIDByToken(token string) (clientUUID string, err error) {
db := dbcore.GetDBInstance()
func GetClientUUIDByToken(token string) (string, error) {
var client models.Client
err = db.Where("token = ?", token).First(&client).Error
if err != nil {
if err := dbcore.GetDBInstance().Where("token = ?", token).First(&client).Error; err != nil {
return "", err
}
return client.UUID, nil
}
// 检查数据防止异常数据导致数据库损坏
func ReportVerify(report v1.Report) error {
// 防止输入不合理范围
// ReportVerify 检查上报数据,防止异常数据导致数据库损坏
func ReportVerify(report v2.Report) error {
if report.CPU.Usage < 0 || report.CPU.Usage > 100 {
return fmt.Errorf("CPU.Usage must be between 0 and 100")
}
@@ -56,28 +53,24 @@ func ReportVerify(report v1.Report) error {
}
// [int64] 防止数据溢出
// Ram 验证
if err := checkInt64("Ram.Used", report.Ram.Used); err != nil {
return err
}
if err := checkInt64("Ram.Total", report.Ram.Total); err != nil {
return err
}
// Swap 验证
if err := checkInt64("Swap.Used", report.Swap.Used); err != nil {
return err
}
if err := checkInt64("Swap.Total", report.Swap.Total); err != nil {
return err
}
// Disk 验证
if err := checkInt64("Disk.Used", report.Disk.Used); err != nil {
return err
}
if err := checkInt64("Disk.Total", report.Disk.Total); err != nil {
return err
}
// Network 验证
if err := checkInt64("Network.Up", report.Network.Up); err != nil {
return err
}
@@ -90,7 +83,7 @@ func ReportVerify(report v1.Report) error {
if err := checkInt64("Network.TotalDown", report.Network.TotalDown); err != nil {
return err
}
// 拒绝所有负数Int
// 拒绝所有负数 Int
if report.Process < 0 {
return fmt.Errorf("Process must be non-negative: %d", report.Process)
}
-64
View File
@@ -1,64 +0,0 @@
package clipboard
import (
"time"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
// CreateClipboard 创建剪贴板记录
func CreateClipboard(cb *models.Clipboard) error {
db := dbcore.GetDBInstance()
cb.CreatedAt = time.Now().UTC()
cb.UpdatedAt = cb.CreatedAt
return db.Create(cb).Error
}
// GetClipboardByID 根据ID获取剪贴板记录
func GetClipboardByID(id int) (*models.Clipboard, error) {
var cb models.Clipboard
db := dbcore.GetDBInstance()
if err := db.First(&cb, id).Error; err != nil {
return nil, err
}
return &cb, nil
}
// UpdateClipboardFields 更新剪贴板记录
func UpdateClipboardFields(id int, fields map[string]interface{}) error {
db := dbcore.GetDBInstance()
delete(fields, "created_at")
fields["updated_at"] = time.Now().UTC()
return db.Model(&models.Clipboard{}).Where("id = ?", id).Updates(fields).Error
}
// DeleteClipboard 删除剪贴板记录
func DeleteClipboard(id int) error {
db := dbcore.GetDBInstance()
// Check if record exists first
var cb models.Clipboard
if err := db.First(&cb, id).Error; err != nil {
return err // Record not found or other error
}
return db.Delete(&cb).Error
}
// DeleteClipboardBatch 批量删除剪贴板记录
func DeleteClipboardBatch(ids []int) error {
if len(ids) == 0 {
return nil
}
db := dbcore.GetDBInstance()
return db.Where("id IN ?", ids).Delete(&models.Clipboard{}).Error
}
// ListClipboard 列出所有剪贴板记录
func ListClipboard() ([]models.Clipboard, error) {
var list []models.Clipboard
db := dbcore.GetDBInstance()
if err := db.Find(&list).Error; err != nil {
return nil, err
}
return list, nil
}
+16 -7
View File
@@ -186,6 +186,13 @@ const (
// versionID 是当前构建的版本标识,由 SetVersionID 在 Initialize 前注入。
var versionID string
// nil means a fresh install; an empty value means an unmarked legacy install.
var lastRunVersion *string
func LastRunVersion() *string {
return lastRunVersion
}
// dbFileExistedAtStartup 记录本次进程启动、打开数据库之前 komari.db 是否已存在,
// 用于区分“全新安装”与“从旧版升级(无版本标记)”。在 doInitialize 打开数据库
// 之前采集。
@@ -221,13 +228,17 @@ func resolveDatabaseFile() string {
//
// 备份失败不阻止启动,但打印明确错误;备份成功(或无需备份)后写入/更新版本。
func backupOnVersionUpgrade() {
if versionID == "" {
return
}
prevVersion, readErr := config.GetAs[string](SystemVersionKey)
prevVersion = strings.TrimSpace(prevVersion)
versionRecorded := readErr == nil && prevVersion != ""
lastRunVersion = nil
if versionRecorded || dbFileExistedAtStartup {
lastRunVersion = &prevVersion
}
if versionID == "" {
return
}
// 版本未变化,无需备份。
if versionRecorded && prevVersion == versionID {
@@ -444,15 +455,13 @@ func doInitialize() error {
// 若升级时旧表仍存在,管理员可通过升级向导显式导入并清理。
// models.Record / models.PingRecord / models.GPURecord 结构体仍作为
// metric store 的读写 DTO 和旧表导入 DTO 保留在 models 包中。
// models.TrafficReportNotification 同样仅为旧数据库兼容保留,不再建表。
err = instance.AutoMigrate(
&models.User{},
&models.Client{},
&models.Log{},
&models.Clipboard{},
&models.LoadNotification{},
&models.OfflineNotification{},
&models.TrafficReportNotification{},
&models.PingTask{},
&models.OidcProvider{},
&models.MessageSenderProvider{},
-13
View File
@@ -1,13 +0,0 @@
package models
import "time"
type Clipboard struct {
Id int `json:"id" gorm:"primaryKey;autoIncrement;unique"`
Text string `json:"text" gorm:"type:longtext"`
Name string `json:"name" gorm:"type:varchar(255)"`
Weight int `json:"weight" gorm:"type:int"`
Remark string `json:"remark" gorm:"type:text"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
@@ -8,7 +8,4 @@ const (
Login = "Login"
Alert = "Alert"
Traffic = "Traffic"
DReport = "DReport" // 日报
WReport = "WReport" // 周报
MReport = "MReport" // 月报
)
-12
View File
@@ -12,18 +12,6 @@ type OfflineNotification struct {
LastNotified *time.Time `json:"last_notified"` // 上次通知时间
}
// LoadNotification 定义了基于资源占用达标时间比的负载通知规则
type LoadNotification struct {
Id uint `json:"id,omitempty" gorm:"primaryKey;autoIncrement"`
Name string `json:"name" gorm:"type:varchar(255)"`
Clients StringArray `json:"clients" gorm:"type:longtext"`
Metric string `json:"metric" gorm:"type:varchar(50);not null;default:'cpu'"` // 监控指标,如 cpu, ram, load
Threshold float32 `json:"threshold" gorm:"type:decimal(5,2);not null;default:80.00"` // 阈值百分比
Ratio float32 `json:"ratio" gorm:"type:decimal(5,2);not null;default:0.80"` // 达标时间比
Interval int `json:"interval" gorm:"type:int;not null;default:15"` // 监测间隔(分钟)
LastNotified *time.Time `json:"last_notified"` // 上次通知时间
}
// TrafficReportNotification 定义了流量定时报告的数据库模型
type TrafficReportNotification struct {
Client string `json:"client" gorm:"type:varchar(36);not null;index;unique;constraint:OnDelete:CASCADE,OnUpdate:CASCADE;foreignKey:client;references:UUID"`
+25 -4
View File
@@ -15,16 +15,37 @@ const (
// Theme represents a komari theme information
type Theme struct {
Name string `json:"name"` // 主题名称
Name any `json:"name"` // 主题名称,支持字符串或多语言对象
Short string `json:"short"` // 短名称,用作文件夹名
Description string `json:"description"` // 主题描述
Description any `json:"description"` // 主题描述,支持字符串或多语言对象
Version string `json:"version"` // 版本号
Author string `json:"author"` // 作者
Author any `json:"author"` // 作者,支持字符串或多语言对象
URL string `json:"url"` // 主题URL
Preview string `json:"preview"` // 预览图片相对路径
Configuration Configuration `json:"configuration"` // 声明配置项
}
// IsLocalizedText reports whether a manifest text field has a usable value.
func IsLocalizedText(value any) bool {
switch text := value.(type) {
case string:
return strings.TrimSpace(text) != ""
case map[string]any:
for _, item := range text {
if itemText, ok := item.(string); ok && strings.TrimSpace(itemText) != "" {
return true
}
}
case map[string]string:
for _, item := range text {
if strings.TrimSpace(item) != "" {
return true
}
}
}
return false
}
type Configuration struct {
Type string `json:"type"` // managed raw redirect
Icon string `json:"icon"` // 图标
@@ -36,7 +57,7 @@ type ManagedThemeConfigurationItem struct {
Key string `json:"key"`
Name any `json:"name"`
Required bool `json:"required"`
Type string `json:"type"` // string number select switch title
Type string `json:"type"` // string number select switch title textbox richtext nodes pingtasks
Options string `json:"options"`
Default any `json:"default"`
Help any `json:"help"`
+2 -2
View File
@@ -13,7 +13,7 @@ import (
)
func TestTimeFieldsStoreAndSerializeAsUTCWithNanoseconds(t *testing.T) {
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
dsn := fmt.Sprintf("file:%s-%d?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"), time.Now().UnixNano())
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
NowFunc: func() time.Time { return time.Now().UTC() },
@@ -48,7 +48,7 @@ func TestTimeFieldsStoreAndSerializeAsUTCWithNanoseconds(t *testing.T) {
}
func TestNullableTimeFieldReadsAndSerializesNull(t *testing.T) {
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
dsn := fmt.Sprintf("file:%s-%d?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"), time.Now().UnixNano())
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatalf("open database: %v", err)
-63
View File
@@ -1,63 +0,0 @@
package notification
import (
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/utils/notifier"
"gorm.io/gorm"
)
func AddLoadNotification(clients []string, name string, metric string, threshold float32, ratio float32, interval int) (uint, error) {
db := dbcore.GetDBInstance()
notification := models.LoadNotification{
Clients: clients,
Name: name,
Metric: metric,
Threshold: threshold,
Ratio: ratio,
Interval: interval,
}
if err := db.Create(&notification).Error; err != nil {
return 0, err
}
return notification.Id, ReloadLoadNotificationSchedule()
}
func DeleteLoadNotification(id []uint) error {
db := dbcore.GetDBInstance()
result := db.Where("id IN ?", id).Delete(&models.LoadNotification{})
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
return ReloadLoadNotificationSchedule()
}
func EditLoadNotification(notifications []*models.LoadNotification) error {
db := dbcore.GetDBInstance()
for _, notification := range notifications {
result := db.Model(&models.LoadNotification{}).Where("id = ?", notification.Id).Updates(notification)
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
}
return ReloadLoadNotificationSchedule()
}
func GetAllLoadNotifications() ([]models.LoadNotification, error) {
db := dbcore.GetDBInstance()
var notifications []models.LoadNotification
if err := db.Find(&notifications).Error; err != nil {
return nil, err
}
return notifications, nil
}
func ReloadLoadNotificationSchedule() error {
db := dbcore.GetDBInstance()
var loadNotifications []models.LoadNotification
if err := db.Find(&loadNotifications).Error; err != nil {
return err
}
return notifier.ReloadLoadNotificationSchedule(loadNotifications)
}
-128
View File
@@ -1,128 +0,0 @@
package notification
import (
"fmt"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"gorm.io/gorm/clause"
)
func validateTrafficReportNotification(notification models.TrafficReportNotification) error {
if notification.Client == "" {
return fmt.Errorf("client UUID cannot be empty")
}
if notification.Enable && !notification.Daily && !notification.Weekly && !notification.Monthly {
return fmt.Errorf("at least one cadence must be selected when enabling traffic reports")
}
return nil
}
func ValidateTrafficReportNotifications(notifications []models.TrafficReportNotification) error {
for _, notification := range notifications {
if err := validateTrafficReportNotification(notification); err != nil {
return err
}
}
return nil
}
func buildEnabledTrafficReportNotifications(uuids []string, existing []models.TrafficReportNotification) ([]models.TrafficReportNotification, error) {
if len(uuids) == 0 {
return nil, fmt.Errorf("at least one client UUID is required")
}
existingByClient := make(map[string]models.TrafficReportNotification, len(existing))
for _, notification := range existing {
existingByClient[notification.Client] = notification
}
notifications := make([]models.TrafficReportNotification, 0, len(uuids))
for _, uuid := range uuids {
if uuid == "" {
return nil, fmt.Errorf("client UUID cannot be empty")
}
existingNotification, ok := existingByClient[uuid]
if !ok || (!existingNotification.Daily && !existingNotification.Weekly && !existingNotification.Monthly) {
return nil, fmt.Errorf("at least one cadence must be selected when enabling traffic reports")
}
notifications = append(notifications, models.TrafficReportNotification{
Client: uuid,
Enable: true,
})
}
return notifications, nil
}
// ListTrafficReportNotifications 获取所有流量定时报告配置(关联客户端信息)
func ListTrafficReportNotifications() ([]models.TrafficReportNotification, error) {
db := dbcore.GetDBInstance()
var notifications []models.TrafficReportNotification
err := db.Model(&models.TrafficReportNotification{}).Preload("ClientInfo").Find(&notifications).Error
return notifications, err
}
// EditTrafficReportNotifications 批量更新流量定时报告配置
func EditTrafficReportNotifications(notifications []models.TrafficReportNotification) error {
if err := ValidateTrafficReportNotifications(notifications); err != nil {
return err
}
db := dbcore.GetDBInstance()
return db.Model(&models.TrafficReportNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable", "daily", "weekly", "monthly"}),
}).
Select("client", "enable", "daily", "weekly", "monthly").
Create(notifications).Error
}
// EnableTrafficReportNotifications 批量启用(仅更新 enable 字段)
func EnableTrafficReportNotifications(uuids []string) error {
if len(uuids) == 0 {
return fmt.Errorf("at least one client UUID is required")
}
db := dbcore.GetDBInstance()
var existing []models.TrafficReportNotification
if err := db.Where("client IN ?", uuids).Find(&existing).Error; err != nil {
return err
}
notifications, err := buildEnabledTrafficReportNotifications(uuids, existing)
if err != nil {
return err
}
return db.Model(&models.TrafficReportNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable"}),
}).
Select("client", "enable").
Create(notifications).Error
}
// DisableTrafficReportNotifications 批量禁用
func DisableTrafficReportNotifications(uuids []string) error {
if len(uuids) == 0 {
return fmt.Errorf("at least one client UUID is required")
}
db := dbcore.GetDBInstance()
var notifications []models.TrafficReportNotification
for _, uuid := range uuids {
if uuid == "" {
return fmt.Errorf("client UUID cannot be empty")
}
notifications = append(notifications, models.TrafficReportNotification{
Client: uuid,
Enable: false,
})
}
return db.Model(&models.TrafficReportNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable"}),
}).
Select("client", "enable").
Create(notifications).Error
}
@@ -1,36 +0,0 @@
package notification
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/komari-monitor/komari/database/models"
)
func TestValidateTrafficReportNotificationsRejectsEnabledWithoutCadence(t *testing.T) {
err := ValidateTrafficReportNotifications([]models.TrafficReportNotification{{
Client: "client-a",
Enable: true,
}})
assert.Error(t, err)
}
func TestBuildEnabledTrafficReportNotificationsRequiresExistingCadence(t *testing.T) {
_, err := buildEnabledTrafficReportNotifications([]string{"client-a"}, nil)
assert.Error(t, err)
_, err = buildEnabledTrafficReportNotifications([]string{"client-a"}, []models.TrafficReportNotification{{Client: "client-a"}})
assert.Error(t, err)
notifications, err := buildEnabledTrafficReportNotifications([]string{"client-a"}, []models.TrafficReportNotification{{
Client: "client-a",
Daily: true,
}})
require.NoError(t, err)
require.Len(t, notifications, 1)
assert.Equal(t, "client-a", notifications[0].Client)
assert.True(t, notifications[0].Enable)
}
+1 -5
View File
@@ -20,9 +20,5 @@ func DeleteOidcConfigByName(name string) error {
}
func SaveOidcConfig(config *models.OidcProvider) error {
db := dbcore.GetDBInstance()
if err := db.Save(config).Error; err != nil {
return err
}
return nil
return dbcore.GetDBInstance().Save(config).Error
}
+63
View File
@@ -0,0 +1,63 @@
package records
import "github.com/komari-monitor/komari/database/models"
// LoadProjection is the field subset selected by a load_type request.
type LoadProjection struct {
Cpu *float32 `json:"cpu,omitempty"`
Gpu *float32 `json:"gpu,omitempty"`
Ram *int64 `json:"ram,omitempty"`
RamTotal *int64 `json:"ram_total,omitempty"`
Swap *int64 `json:"swap,omitempty"`
SwapTotal *int64 `json:"swap_total,omitempty"`
Load *float32 `json:"load,omitempty"`
Temp *float32 `json:"temp,omitempty"`
Disk *int64 `json:"disk,omitempty"`
DiskTotal *int64 `json:"disk_total,omitempty"`
NetIn *int64 `json:"net_in,omitempty"`
NetOut *int64 `json:"net_out,omitempty"`
NetTotalUp *int64 `json:"net_total_up,omitempty"`
NetTotalDown *int64 `json:"net_total_down,omitempty"`
Process *int `json:"process,omitempty"`
Connections *int `json:"connections,omitempty"`
ConnectionsUdp *int `json:"connections_udp,omitempty"`
}
// ProjectLoad returns the load_type projection for one record.
func ProjectLoad(r models.Record, loadType string) LoadProjection {
var p LoadProjection
switch loadType {
case "cpu":
p.Cpu = ptr(r.Cpu)
case "gpu":
p.Gpu = ptr(r.Gpu)
case "ram":
p.Ram = ptr(r.Ram)
p.RamTotal = ptr(r.RamTotal)
case "swap":
p.Swap = ptr(r.Swap)
p.SwapTotal = ptr(r.SwapTotal)
case "load":
p.Load = ptr(r.Load)
case "temp":
p.Temp = ptr(r.Temp)
case "disk":
p.Disk = ptr(r.Disk)
p.DiskTotal = ptr(r.DiskTotal)
case "network":
p.NetIn = ptr(r.NetIn)
p.NetOut = ptr(r.NetOut)
p.NetTotalUp = ptr(r.NetTotalUp)
p.NetTotalDown = ptr(r.NetTotalDown)
case "process":
p.Process = ptr(r.Process)
case "connections":
p.Connections = ptr(r.Connections)
p.ConnectionsUdp = ptr(r.ConnectionsUdp)
default:
p.Cpu = ptr(r.Cpu)
}
return p
}
func ptr[T any](v T) *T { return &v }
+65 -9
View File
@@ -4,8 +4,8 @@ import (
"context"
"time"
"github.com/komari-monitor/komari/internal/metricstore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/metricstore"
)
// 历史监控数据已完全迁移到 metric store(默认 SQLite ./data/metrics.db,或配置的
@@ -19,21 +19,77 @@ func DeleteAll() error {
return metricstore.DeleteAllRecords(context.Background())
}
// GetGPURecordsByClientAndTime 获取 GPU 记录数据。
// QueryGPURecords returns GPU records using the caller's cancellation context.
func QueryGPURecords(ctx context.Context, uuid string, start, end time.Time) ([]models.GPURecord, error) {
return metricstore.GetGPURecordsByClientAndTime(ctx, uuid, start, end)
}
// GetGPURecordsByClientAndTime is a compatibility-only wrapper without cancellation.
// New request handlers should use QueryGPURecords.
func GetGPURecordsByClientAndTime(uuid string, start, end time.Time) ([]models.GPURecord, error) {
return metricstore.GetGPURecordsByClientAndTime(context.Background(), uuid, start, end)
return QueryGPURecords(context.Background(), uuid, start, end)
}
func GetRecordsByClientAndTime(uuid string, start, end time.Time) ([]models.Record, error) {
return metricstore.GetRecordsByClientAndTime(context.Background(), uuid, start, end)
// RecordQueryOptions narrows metric selection and the response point budget.
type RecordQueryOptions = metricstore.RecordQueryOptions
// LoadTypeMetricNames maps the RPC load_type to the stored metrics that back
// that field. An empty result means "all load metrics" (including temp and
// unknown values, which stay on the compatible full-query path).
func LoadTypeMetricNames(loadType string) []string {
var names []string
switch loadType {
case "cpu":
return []string{metricstore.MetricCPU}
case "gpu":
names = []string{metricstore.MetricGPU}
case "ram":
names = []string{metricstore.MetricRAM}
case "swap":
names = []string{metricstore.MetricSwap}
case "load":
names = []string{metricstore.MetricLoad}
case "disk":
names = []string{metricstore.MetricDisk}
case "network":
names = []string{metricstore.MetricNetIn, metricstore.MetricNetOut, metricstore.MetricNetTotalUp, metricstore.MetricNetTotalDown}
case "process":
names = []string{metricstore.MetricProcess}
case "connections":
names = []string{metricstore.MetricConnections, metricstore.MetricConnectionsUDP}
default:
return nil
}
// CPU is the common presence anchor: the previous implementation built one
// record per timestamp where any load metric existed. Including CPU keeps
// those timestamps (and zero values for a missing selected metric) without
// going back to all fifteen metrics.
return append(names, metricstore.MetricCPU)
}
// GetRecordMetricMaxByClientAndTime 获取单项监控指标在各时间桶内的最大值。
// QueryRecords returns one client's records, or all clients when uuid is empty.
// Request handlers should use this context-aware entry point.
func QueryRecords(ctx context.Context, uuid string, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
if uuid == "" {
return metricstore.GetRecordsByTime(ctx, start, end, opts...)
}
return metricstore.GetRecordsByClientAndTime(ctx, uuid, start, end, opts...)
}
// GetRecordsByClientAndTime is a compatibility-only wrapper without cancellation.
// New request handlers should use QueryRecords.
func GetRecordsByClientAndTime(uuid string, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
return metricstore.GetRecordsByClientAndTime(context.Background(), uuid, start, end, opts...)
}
// GetRecordMetricMaxByClientAndTime is a compatibility-only wrapper without cancellation.
// New request handlers should use metricstore.GetRecordMetricMaxByClientAndTime.
func GetRecordMetricMaxByClientAndTime(uuid, metricName string, start, end time.Time) ([]models.Record, error) {
return metricstore.GetRecordMetricMaxByClientAndTime(context.Background(), uuid, metricName, start, end)
}
// GetRecordsByTime 获取所有客户端在时间范围内的记录。
func GetRecordsByTime(start, end time.Time) ([]models.Record, error) {
return metricstore.GetRecordsByTime(context.Background(), start, end)
// GetRecordsByTime is a compatibility-only wrapper without cancellation.
// New request handlers should use QueryRecords with an empty uuid.
func GetRecordsByTime(start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
return metricstore.GetRecordsByTime(context.Background(), start, end, opts...)
}
+5 -33
View File
@@ -2,8 +2,8 @@ package tasks
import (
"context"
"slices"
"sort"
"time"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
@@ -28,17 +28,8 @@ func AddPingTask(clients []string, defaultOn bool, name string, target, task_typ
if err := tx.Create(&task).Error; err != nil {
return err
}
// Append by id to avoid races between concurrent create requests.
result := tx.Model(&models.PingTask{}).Where("id = ?", task.Id).Update("weight", int(task.Id))
if result.Error != nil {
return result.Error
}
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
return nil
return tx.Model(&models.PingTask{}).Where("id = ?", task.Id).Update("weight", int(task.Id)).Error
})
if err != nil {
return 0, err
@@ -138,10 +129,8 @@ func UpdatePingTaskOrder(order map[uint]int) error {
}
for _, id := range ids {
weight := order[id]
result := tx.Model(&models.PingTask{}).Where("id = ?", id).Update("weight", weight)
if result.Error != nil {
return result.Error
if err := tx.Model(&models.PingTask{}).Where("id = ?", id).Update("weight", order[id]).Error; err != nil {
return err
}
}
return nil
@@ -178,27 +167,14 @@ func ReloadPingSchedule() error {
// AddDefaultOnClientUUID 在新客户端注册后,把该 UUID 追加到所有 default_on=true 的任务的 clients 中(去重)。
func AddDefaultOnClientUUID(uuid string) error {
if uuid == "" {
return nil
}
db := dbcore.GetDBInstance()
var tasks []models.PingTask
if err := db.Where("all_clients = ?", true).Find(&tasks).Error; err != nil {
return err
}
if len(tasks) == 0 {
return nil
}
changed := false
for _, task := range tasks {
exists := false
for _, c := range task.Clients {
if c == uuid {
exists = true
break
}
}
if exists {
if slices.Contains(task.Clients, uuid) {
continue
}
next := append(models.StringArray{}, task.Clients...)
@@ -213,7 +189,3 @@ func AddDefaultOnClientUUID(uuid string) error {
}
return nil
}
func GetPingRecords(uuid string, taskId int, start, end time.Time) ([]models.PingRecord, error) {
return metricstore.GetPingRecords(context.Background(), uuid, taskId, start, end)
}
+180
View File
@@ -0,0 +1,180 @@
package tasks
import (
"math"
"sort"
"time"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/sampling"
)
// PingClientSummary is the per-client aggregate shared by ping RPC adapters.
type PingClientSummary struct {
Total int
Loss int
Min int
Max int
LossPercent float64
}
// PingTaskSummary is the per-task aggregate shared by ping RPC adapters.
type PingTaskSummary struct {
Total int
Loss int
Valid int
Min int
Max int
Avg int
Latest int
P50 int
P99 int
LossPercent float64
P99P50Ratio float64
}
// PingSummary aggregates records once so legacy and public ping handlers only
// differ in response shape.
type PingSummary struct {
Clients map[string]PingClientSummary
Tasks map[uint]PingTaskSummary
}
type pingClientAccumulator struct {
total, loss, min, max int
}
type pingTaskAccumulator struct {
total, loss, min, max, sum, valid int
latest int
latestTime time.Time
latencies []int
}
// SummarizePingRecords computes client and task aggregates in one pass.
func SummarizePingRecords(recs []models.PingRecord) PingSummary {
clients := make(map[string]*pingClientAccumulator)
taskMap := make(map[uint]*pingTaskAccumulator)
for _, r := range recs {
cs := clients[r.Client]
if cs == nil {
cs = &pingClientAccumulator{}
clients[r.Client] = cs
}
cs.total++
ts := taskMap[r.TaskId]
if ts == nil {
ts = &pingTaskAccumulator{latest: -1}
taskMap[r.TaskId] = ts
}
ts.total++
if r.Value < 0 {
cs.loss++
ts.loss++
continue
}
if cs.min == 0 || r.Value < cs.min {
cs.min = r.Value
}
if r.Value > cs.max {
cs.max = r.Value
}
ts.valid++
ts.sum += r.Value
if ts.min == 0 || r.Value < ts.min {
ts.min = r.Value
}
if r.Value > ts.max {
ts.max = r.Value
}
if ts.latestTime.IsZero() || r.Time.After(ts.latestTime) {
ts.latestTime = r.Time
ts.latest = r.Value
}
ts.latencies = append(ts.latencies, r.Value)
}
out := PingSummary{Clients: make(map[string]PingClientSummary, len(clients)), Tasks: make(map[uint]PingTaskSummary, len(taskMap))}
for client, v := range clients {
loss := 0.0
if v.total > 0 {
loss = float64(v.loss) / float64(v.total) * 100
}
out.Clients[client] = PingClientSummary{Total: v.total, Loss: v.loss, Min: v.min, Max: v.max, LossPercent: loss}
}
for id, v := range taskMap {
loss := 0.0
if v.total > 0 {
loss = float64(v.loss) / float64(v.total) * 100
}
avg := 0
if v.valid > 0 {
avg = v.sum / v.valid
}
p50, p99 := pingPercentiles(v.latencies)
ratio := 0.0
if p50 > 0 && p99 >= p50 {
adjustedBase := math.Max(math.Min(float64(p50), 50.0), 10.0)
ratio = float64(p99-p50) / adjustedBase
}
out.Tasks[id] = PingTaskSummary{
Total: v.total, Loss: v.loss, Valid: v.valid, Min: v.min, Max: v.max,
Avg: avg, Latest: v.latest, P50: p50, P99: p99, LossPercent: loss, P99P50Ratio: ratio,
}
}
return out
}
// SamplePingRecordsByTask applies the legacy ping maxCount policy: allocate
// points proportionally by task, sample each task evenly and return ascending
// by time. Statistics must be computed from the unsampled records.
func SamplePingRecordsByTask(recs []models.PingRecord, maxCount int) []models.PingRecord {
if maxCount <= 0 || len(recs) <= maxCount {
return recs
}
groups := make(map[uint][]models.PingRecord)
for _, r := range recs {
groups[r.TaskId] = append(groups[r.TaskId], r)
}
meta := make([]sampling.Group[uint], 0, len(groups))
for id, points := range groups {
sort.Slice(points, func(i, j int) bool { return points[i].Time.Before(points[j].Time) })
groups[id] = points
meta = append(meta, sampling.Group[uint]{Key: id, Length: len(points)})
}
targets := sampling.AllocateTargets(meta, maxCount)
out := make([]models.PingRecord, 0, maxCount)
for id, points := range groups {
out = append(out, sampling.SampleEvenly(points, targets[id])...)
}
sort.Slice(out, func(i, j int) bool { return out[i].Time.Before(out[j].Time) })
return out
}
func pingPercentiles(values []int) (int, int) {
if len(values) == 0 {
return 0, 0
}
sort.Ints(values)
return pingPercentile(values, 0.50), pingPercentile(values, 0.99)
}
func pingPercentile(values []int, percentile float64) int {
if len(values) == 0 {
return 0
}
if percentile <= 0 {
return values[0]
}
if percentile >= 1 {
return values[len(values)-1]
}
pos := float64(len(values)-1) * percentile
lo := int(math.Floor(pos))
hi := int(math.Ceil(pos))
if lo == hi {
return values[lo]
}
frac := pos - float64(lo)
return int(math.Round(float64(values[lo]) + (float64(values[hi])-float64(values[lo]))*frac))
}
+3 -4
View File
@@ -9,7 +9,6 @@ import (
func CreateTask(taskId string, clients []string, command string) error {
db := dbcore.GetDBInstance()
// Create a new task in the database with clients as JSON array
task := models.Task{
TaskId: taskId,
Clients: models.StringArray(clients),
@@ -29,10 +28,10 @@ func CreateTask(taskId string, clients []string, command string) error {
CreatedAt: time.Now().UTC(),
})
}
if len(taskResults) > 0 {
return db.Create(&taskResults).Error
if len(taskResults) == 0 {
return nil
}
return nil
return db.Create(&taskResults).Error
}
func GetTaskByTaskId(taskId string) (*models.Task, error) {
var task models.Task
+58 -8
View File
@@ -1,6 +1,7 @@
package tasks
import (
"os"
"testing"
"time"
@@ -10,8 +11,7 @@ import (
)
func TestClearTaskResultsByTimeBeforeUsesUTCTimeValue(t *testing.T) {
flags.DatabaseType = flags.DatabaseTypeSQLite
flags.DatabaseFile = "file:task_cleanup_time?mode=memory&cache=shared"
resetTaskTables(t)
db := dbcore.GetDBInstance()
taskID := "task-cleanup-time"
@@ -42,8 +42,7 @@ func TestClearTaskResultsByTimeBeforeUsesUTCTimeValue(t *testing.T) {
}
func TestGetAllPingTasksOrdersByWeightThenID(t *testing.T) {
flags.DatabaseType = flags.DatabaseTypeSQLite
flags.DatabaseFile = "file:ping_task_order?mode=memory&cache=shared"
resetTaskTables(t)
db := dbcore.GetDBInstance()
tasks := []models.PingTask{
@@ -68,8 +67,7 @@ func TestGetAllPingTasksOrdersByWeightThenID(t *testing.T) {
}
func TestUpdatePingTaskOrderAllowsUnchangedWeights(t *testing.T) {
flags.DatabaseType = flags.DatabaseTypeSQLite
flags.DatabaseFile = "file:ping_task_update_order?mode=memory&cache=shared"
resetTaskTables(t)
db := dbcore.GetDBInstance()
items := []models.PingTask{
@@ -100,8 +98,7 @@ func TestUpdatePingTaskOrderAllowsUnchangedWeights(t *testing.T) {
}
func TestGetPingTasksByClientOrdersByWeightThenID(t *testing.T) {
flags.DatabaseType = flags.DatabaseTypeSQLite
flags.DatabaseFile = "file:ping_task_client_order?mode=memory&cache=shared"
resetTaskTables(t)
db := dbcore.GetDBInstance()
items := []models.PingTask{
@@ -118,3 +115,56 @@ func TestGetPingTasksByClientOrdersByWeightThenID(t *testing.T) {
t.Fatalf("ping task order = %#v, want ids [%d %d]", ordered, items[1].Id, items[0].Id)
}
}
// TestMain pins a single shared in-memory database. dbcore.Initialize runs
// once per process, so per-test DSN assignment would be ignored and leave the
// tests fighting over whichever DSN happened to win.
func TestMain(m *testing.M) {
flags.DatabaseType = flags.DatabaseTypeSQLite
flags.DatabaseFile = "file:komari_tasks_test?mode=memory&cache=shared"
db := dbcore.GetDBInstance()
if sqlDB, err := db.DB(); err == nil {
sqlDB.SetMaxOpenConns(1)
}
os.Exit(m.Run())
}
// resetTaskTables clears the shared in-memory tables so repeated runs and
// shuffled execution start from a clean slate.
func resetTaskTables(t *testing.T) {
t.Helper()
db := dbcore.GetDBInstance()
for _, model := range []any{&models.Task{}, &models.TaskResult{}, &models.PingTask{}} {
if err := db.Where("1 = 1").Delete(model).Error; err != nil {
t.Fatalf("reset %T: %v", model, err)
}
}
t.Cleanup(func() {
db := dbcore.GetDBInstance()
for _, model := range []any{&models.Task{}, &models.TaskResult{}, &models.PingTask{}} {
_ = db.Where("1 = 1").Delete(model).Error
}
})
}
+31 -53
View File
@@ -3,16 +3,17 @@ package database
import (
"context"
"encoding/json"
logger "github.com/komari-monitor/komari/utils/log"
"os"
"path/filepath"
"strings"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/internal/metricstore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/config"
"github.com/komari-monitor/komari/internal/managedconfig"
"github.com/komari-monitor/komari/internal/metricstore"
logger "github.com/komari-monitor/komari/utils/log"
"github.com/komari-monitor/komari/web/public"
)
func GetPublicInfo() (map[string]interface{}, error) {
@@ -69,58 +70,20 @@ func GetPublicInfo() (map[string]interface{}, error) {
if err != nil {
logger.Infof("database", "%v", err)
}
// Try to load theme declaration file and merge defaults for managed configuration
// Theme declarations live in ./data/theme/<short>/komari-theme.json
if cst.Theme != "" && cst.Theme != "default" {
themeConfigPath := filepath.Join("./data/theme", cst.Theme, "komari-theme.json")
if _, err := os.Stat(themeConfigPath); err == nil {
b, err := os.ReadFile(themeConfigPath)
if err == nil {
var themeDecl struct {
Configuration struct {
Type string `json:"type"`
Data []models.ManagedThemeConfigurationItem `json:"data"`
} `json:"configuration"`
}
if err := json.Unmarshal(b, &themeDecl); err == nil {
if themeDecl.Configuration.Type == "managed" {
for _, item := range themeDecl.Configuration.Data {
if item.Key == "" {
continue
}
// missing
if _, exists := tc_data[item.Key]; !exists {
var def any = item.Default
// select
if item.Type == "select" {
if def == nil || def == "" {
if item.Options != "" {
opts := strings.Split(item.Options, ",")
if len(opts) > 0 {
def = strings.TrimSpace(opts[0])
}
}
}
}
// number->0, string->"", switch->false
if def == nil {
switch item.Type {
case "number":
def = 0
case "switch":
def = false
default:
def = ""
}
}
tc_data[item.Key] = def
}
}
}
}
items := themeConfigurationItems(cst.Theme)
if cst.Theme != "default" {
for _, item := range items {
if item.Key == "" {
continue
}
if _, exists := tc_data[item.Key]; !exists {
tc_data[item.Key] = managedconfig.DefaultValue(item)
}
}
}
if err := managedconfig.ResolveForOutput(tc_data, items); err != nil {
return nil, err
}
return gin.H{
"sitename": cst.Sitename,
@@ -135,8 +98,23 @@ func GetPublicInfo() (map[string]interface{}, error) {
"record_preserve_time": retention.MaxDays * 24,
"ping_record_preserve_time": retention.MaxDays * 24,
"private_site": cst.PrivateSite,
"visitor_audit_enabled": cst.VisitorAuditEnabled,
"theme": cst.Theme,
"theme_settings": tc_data,
}, nil
}
func themeConfigurationItems(short string) []models.ManagedThemeConfigurationItem {
var manifest models.Theme
if short == "default" {
data, err := public.PublicFS.ReadFile("defaultTheme/komari-theme.json")
if err != nil || json.Unmarshal(data, &manifest) != nil {
return nil
}
} else {
data, err := os.ReadFile(filepath.Join("./data/theme", short, "komari-theme.json"))
if err != nil || json.Unmarshal(data, &manifest) != nil {
return nil
}
}
return managedconfig.Items(manifest.Configuration)
}
+4 -10
View File
@@ -3,8 +3,7 @@ module github.com/komari-monitor/komari
go 1.25.0
require (
github.com/dop251/goja v0.0.0-20251008123653-cf18d89f3cf6
github.com/dop251/goja_nodejs v0.0.0-20251015164255-5e94316bedaf
github.com/Calcium-Ion/moejs v0.1.0-alpha.9
github.com/gin-gonic/gin v1.10.0
github.com/go-sql-driver/mysql v1.10.0
github.com/google/uuid v1.6.0
@@ -16,13 +15,13 @@ require (
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pquerna/otp v1.5.0
github.com/spf13/cobra v1.9.1
github.com/stretchr/testify v1.11.1
github.com/stretchr/testify v1.12.1
golang.org/x/sys v0.33.0
gorm.io/driver/sqlite v1.5.7
gorm.io/gorm v1.26.1
)
require github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217 // indirect
require go.yaml.in/yaml/v3 v3.0.5 // indirect
// Misuse of ServerConfig.PublicKeyCallback may cause authorization bypass in golang.org/x/crypto #1
// golang.org/x/crypto Vulnerable to Denial of Service (DoS) via Slow or Incomplete Key Exchange #3
@@ -30,7 +29,7 @@ require golang.org/x/crypto v0.39.0
// HTTP Proxy bypass using IPv6 Zone IDs in golang.org/x/net #2
// golang.org/x/net vulnerable to Cross-site Scripting #4
require golang.org/x/net v0.41.0 // indirect
require golang.org/x/net v0.41.0
require (
filippo.io/edwards25519 v1.2.0 // indirect
@@ -38,17 +37,13 @@ require (
github.com/bytedance/sonic v1.13.2 // indirect
github.com/bytedance/sonic/loader v0.2.4 // indirect
github.com/cloudwego/base64x v0.1.5 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dlclark/regexp2 v1.11.4 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gin-contrib/sse v1.0.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.26.0 // indirect
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
@@ -63,7 +58,6 @@ require (
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rogpeppe/go-internal v1.8.0 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
+6 -21
View File
@@ -1,7 +1,7 @@
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/Masterminds/semver/v3 v3.2.1 h1:RN9w6+7QoMeJVGyfmbcgs28Br8cvmnucEXnY0rYXWg0=
github.com/Masterminds/semver/v3 v3.2.1/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Calcium-Ion/moejs v0.1.0-alpha.9 h1:WY1ebChegtafhPjHSQx3Q+gfnrjqYwHewHuO9k+v7e0=
github.com/Calcium-Ion/moejs v0.1.0-alpha.9/go.mod h1:9YqHj36EV7pzV/vlEYSw49f3oggeJFdmqcjM9TmDpHs=
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc h1:biVzkmvwrH8WK8raXaxBx6fRVTlJILwEwQGL1I/ByEI=
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/bytedance/sonic v1.13.2 h1:8/H1FempDZqC4VqjptGo14QQlJx8VdZJegxs6wwfqpQ=
@@ -17,14 +17,6 @@ github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ3
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.4 h1:rPYF9/LECdNymJufQKmri9gV604RvvABwgOA8un7yAo=
github.com/dlclark/regexp2 v1.11.4/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217 h1:16iT9CBDOniJwFGPI41MbUDfEk74hFaKTqudrX8kenY=
github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217/go.mod h1:eIb+f24U+eWQCIsj9D/ah+MD9UP+wdxuqzsdLD+mhGM=
github.com/dop251/goja v0.0.0-20251008123653-cf18d89f3cf6 h1:6dE1TmjqkY6tehR4A67gDNhvDtuZ54ocu7ab4K9o540=
github.com/dop251/goja v0.0.0-20251008123653-cf18d89f3cf6/go.mod h1:MxLav0peU43GgvwVgNbLAj1s/bSGboKkhuULvq/7hx4=
github.com/dop251/goja_nodejs v0.0.0-20251015164255-5e94316bedaf h1:gbmvliZnCut4NjaPSNOQlfqBoZ9C5Dpf72mHMMYhgVE=
github.com/dop251/goja_nodejs v0.0.0-20251015164255-5e94316bedaf/go.mod h1:Tb7Xxye4LX7cT3i8YLvmPMGCV92IOi4CDZvm/V8ylc0=
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
github.com/gin-contrib/sse v1.0.0 h1:y3bT1mUWUxDpW4JLQg/HnTqV4rozuW4tC9eFKTxYI9E=
@@ -39,8 +31,6 @@ github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJn
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.26.0 h1:SP05Nqhjcvz81uJaRfEV0YBSSSGMc/iMaVtFbr3Sw2k=
github.com/go-playground/validator/v10 v10.26.0/go.mod h1:I5QpIEbmr8On7W0TktmJAumgzX4CA1XNl4ZmDuVHKKo=
github.com/go-sourcemap/sourcemap v2.1.4+incompatible h1:a+iTbH5auLKxaNwQFg0B+TCYl6lbukKPc7b5x0n1s6Q=
github.com/go-sourcemap/sourcemap v2.1.4+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg=
github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw=
github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
@@ -48,8 +38,6 @@ github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PU
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 h1:FKHo8hFI3A+7w0aUQuYXQ+6EN5stWmeY/AZqtM8xk9k=
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8/go.mod h1:K1liHPHnj73Fdn/EKuT8nrFqBihUSKXoLYU0BuatOYo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
@@ -101,7 +89,6 @@ github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTK
github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
github.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pquerna/otp v1.5.0 h1:NMMR+WrmaqXU4EzdGJEE1aUUI0AMRzsp96fFFWNPwxs=
github.com/pquerna/otp v1.5.0/go.mod h1:dkJfzwRKNiegxyNb54X/3fLwhCynbMspSyWKnvi1AEg=
@@ -123,14 +110,14 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/arch v0.15.0 h1:QtOrQd0bTUnhNVNndMpLHNWrDmYzZ2KDqSrEymqInZw=
golang.org/x/arch v0.15.0/go.mod h1:JmwW7aLIoRUKgaTzhkiEFxvcEiQGyOg9BMonBJUS7EE=
golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
@@ -151,8 +138,6 @@ gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+1060 -224
View File
File diff suppressed because it is too large. Load diff
+2 -3
View File
@@ -16,7 +16,7 @@ type Settings struct {
AutoDiscoveryKey string `json:"auto_discovery_key" default:""` // 自动发现密钥
ScriptDomain string `json:"script_domain" default:""` // 自定义脚本域名
SendIpAddrToGuest bool `json:"send_ip_addr_to_guest" default:"false"` // 是否向访客页面发送 IP 地址,默认 false
VisitorAuditEnabled bool `json:"visitor_audit_enabled" default:"false"` // 是否允许公开访客事件写入审计日志,默认 false
SSRFProtectionEnabled bool `json:"ssrf_protection_enabled" default:"false"` // 是否启用 SSRF 防护,默认关闭
EulaAccepted bool `json:"eula_accepted" default:"false"`
BaseScriptsURLKey string `json:"base_scripts_url" default:""`
// GeoIP 配置
@@ -54,7 +54,7 @@ const (
AutoDiscoveryKeyKey = "auto_discovery_key"
ScriptDomainKey = "script_domain"
SendIpAddrToGuestKey = "send_ip_addr_to_guest"
VisitorAuditEnabledKey = "visitor_audit_enabled"
SSRFProtectionEnabledKey = "ssrf_protection_enabled"
EulaAcceptedKey = "eula_accepted"
BaseScriptsURLKey = "base_scripts_url"
GeoIpEnabledKey = "geo_ip_enabled"
@@ -73,7 +73,6 @@ const (
LoginNotificationKey = "login_notification"
TrafficLimitPercentageKey = "traffic_limit_percentage"
UpdatedAtKey = "updated_at"
XtermjsSettingsKey = "xtermjs_settings"
ThemeMarketSourcesKey = "theme_market_sources"
PluginMarketSourcesKey = "plugin_market_sources"
)
+144
View File
@@ -0,0 +1,144 @@
// Package managedconfig implements the shared managed-configuration behavior
// used by themes and plugins.
package managedconfig
import (
"encoding/json"
"strings"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
const (
TypeNodes = "nodes"
TypePingTasks = "pingtasks"
)
// Items returns the declared managed configuration items. An omitted type is
// managed, matching the theme configuration default.
func Items(configuration models.Configuration) []models.ManagedThemeConfigurationItem {
if configuration.Type != "" && !strings.EqualFold(configuration.Type, models.ThemeConfigurationManaged) {
return nil
}
raw, err := json.Marshal(configuration.Data)
if err != nil {
return nil
}
var items []models.ManagedThemeConfigurationItem
if err := json.Unmarshal(raw, &items); err != nil {
return nil
}
return items
}
func DefaultValue(item models.ManagedThemeConfigurationItem) any {
value := item.Default
if item.Type == "select" && (value == nil || value == "") && item.Options != "" {
options := strings.Split(item.Options, ",")
if len(options) > 0 {
return strings.TrimSpace(options[0])
}
}
if value != nil {
return value
}
switch item.Type {
case "number":
return float64(0)
case "switch":
return false
case TypeNodes, TypePingTasks:
return "[]"
default:
return ""
}
}
// ResolveForOutput decodes declared selectors, drops deleted references, and
// returns typed arrays suitable for public theme settings and plugin config.
func ResolveForOutput(values map[string]any, items []models.ManagedThemeConfigurationItem) error {
hasNodes := false
hasPingTasks := false
for _, item := range items {
switch item.Type {
case TypeNodes:
hasNodes = true
case TypePingTasks:
hasPingTasks = true
}
}
db := dbcore.GetDBInstance()
liveNodes := map[string]struct{}{}
if hasNodes {
var nodes []models.Client
if err := db.Select("uuid").Find(&nodes).Error; err != nil {
return err
}
for _, node := range nodes {
liveNodes[node.UUID] = struct{}{}
}
}
livePingTasks := map[uint]struct{}{}
if hasPingTasks {
var tasks []models.PingTask
if err := db.Select("id").Find(&tasks).Error; err != nil {
return err
}
for _, task := range tasks {
livePingTasks[task.Id] = struct{}{}
}
}
for _, item := range items {
if item.Key == "" {
continue
}
switch item.Type {
case TypeNodes:
selected := NodeIDs(values[item.Key])
filtered := make([]string, 0, len(selected))
for _, id := range selected {
if _, ok := liveNodes[id]; ok {
filtered = append(filtered, id)
}
}
values[item.Key] = filtered
case TypePingTasks:
selected := PingTaskIDs(values[item.Key])
filtered := make([]uint, 0, len(selected))
for _, id := range selected {
if _, ok := livePingTasks[id]; ok {
filtered = append(filtered, id)
}
}
values[item.Key] = filtered
}
}
return nil
}
func NodeIDs(value any) []string {
raw, ok := value.(string)
if !ok {
return nil
}
var ids []string
if json.Unmarshal([]byte(raw), &ids) != nil {
return nil
}
return ids
}
func PingTaskIDs(value any) []uint {
raw, ok := value.(string)
if !ok {
return nil
}
var ids []uint
if json.Unmarshal([]byte(raw), &ids) != nil {
return nil
}
return ids
}
-7
View File
@@ -65,9 +65,6 @@ func targetFingerprint(cfg *MetricStoreConfig) string {
// autoMigrate 控制是否在 Open 时自动建表:正式初始化/热加载时为 true,
// 仅做连接测试时为 false(不写入 schema,避免对目标库产生副作用)。
func buildMetricConfig(cfg *MetricStoreConfig, autoMigrate bool) (metric.Config, error) {
if cfg == nil {
return metric.Config{}, fmt.Errorf("metric store config is nil")
}
driver := ResolveDriverFromConfig(cfg.Driver, cfg.DSN)
tablePrefix := cfg.TablePrefix
@@ -131,10 +128,6 @@ func defaultRollupPolicy() metric.RollupPolicy {
}
func rollupPolicyFromConfig(cfg *MetricStoreConfig) (metric.RollupPolicy, error) {
if cfg == nil {
return metric.RollupPolicy{}, fmt.Errorf("metric store config is nil")
}
minuteRetention := cfg.RollupMinuteRetentionMinutes
fiveMinuteRetention := cfg.RollupFiveMinuteRetentionMinutes
hourRetention := cfg.RollupHourRetentionHours
+110 -116
View File
@@ -10,14 +10,32 @@ import (
"github.com/komari-monitor/komari/pkg/metric"
)
// GetRecordsByClientAndTime 从 metric store 查询记录并重构为 models.Record
func GetRecordsByClientAndTime(ctx context.Context, clientUUID string, start, end time.Time) ([]models.Record, error) {
// RecordQueryOptions narrows the record reconstruction work. MetricNames
// limits which stored load metrics are queried; empty means all load metrics.
// MaxPoints is a total response-point budget; it is applied before aggregation,
// so an all-client query does not materialize thousands of points per client
// only to sample them afterwards.
type RecordQueryOptions struct {
MetricNames []string
MaxPoints int
}
// GetRecordsByClientAndTime 从 metric store 查询记录并重构为 models.Record。
// opts 为空表示兼容旧行为;有值时用于指标选择和点数预算。
func GetRecordsByClientAndTime(ctx context.Context, clientUUID string, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
return getRecordsByClientAndTimeFromSeries(ctx, s, clientUUID, start, end)
return getRecordsFromSeries(ctx, s, clientUUID, start, end, false, firstRecordOptions(opts))
}
func firstRecordOptions(opts []RecordQueryOptions) RecordQueryOptions {
if len(opts) == 0 {
return RecordQueryOptions{}
}
return opts[0]
}
// GetRecordMetricMaxByClientAndTime 查询单项监控指标在各时间桶内的最大值。
@@ -30,28 +48,17 @@ func GetRecordMetricMaxByClientAndTime(ctx context.Context, clientUUID, recordMe
return getRecordMetricMaxByClientAndTimeFromSeries(ctx, s, clientUUID, recordMetric, start, end)
}
// GetRecordsByTime 从 metric store 查询所有客户端在时间范围内的记录
func GetRecordsByTime(ctx context.Context, start, end time.Time) ([]models.Record, error) {
// GetRecordsByTime 从 metric store 查询所有客户端在时间范围内的记录。
// opts 为空表示兼容旧行为;有值时用于指标选择和总点数预算。
func GetRecordsByTime(ctx context.Context, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
interval := recordSeriesInterval(s, start, end, time.Now().UTC())
entityIDs, err := listRecordEntityIDs(ctx, s, start, end, interval)
if err != nil {
return nil, err
}
var records []models.Record
for _, entityID := range entityIDs {
items, err := getRecordsByClientAndTimeFromSeries(ctx, s, entityID, start, end)
if err != nil {
return nil, err
}
records = append(records, items...)
}
sortRecords(records)
return records, nil
// Query all entities together. Entity-only grouping retains the old per-client
// aggregation across tags without repeating the tier scan for every client.
return getRecordsFromSeries(ctx, s, "", start, end, true, firstRecordOptions(opts))
}
type recordSeriesKey struct {
@@ -59,25 +66,58 @@ type recordSeriesKey struct {
ts int64
}
func getRecordsByClientAndTimeFromSeries(ctx context.Context, s *metric.Store, clientUUID string, start, end time.Time) ([]models.Record, error) {
now := time.Now().UTC()
interval := recordSeriesInterval(s, start, end, now)
recordMap := make(map[recordSeriesKey]*models.Record)
func getRecordsFromSeries(ctx context.Context, s *metric.Store, clientUUID string, start, end time.Time, allEntities bool, opts RecordQueryOptions) ([]models.Record, error) {
metricNames := opts.MetricNames
if len(metricNames) == 0 {
metricNames = loadRecordMetricNames
}
specs := make([]MetricSpec, 0, len(metricNames))
for _, name := range metricNames {
specs = append(specs, MetricSpec{Name: name, Aggregations: []metric.Aggregation{recordMetricAggregation(name)}})
}
entities := EntityScope{All: allEntities}
grouping := GroupMerged
if allEntities {
grouping = GroupEntity
} else {
entities.IDs = []string{clientUUID}
}
rounding := RoundFloor
if opts.MaxPoints > 0 {
rounding = RoundCeil
}
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QuerySeries, Source: SourceRollup,
OmitTags: true, OmitLabels: true, CompactDefinitions: true,
Entities: entities, Specs: specs, Start: start, End: end,
Order: metric.OrderAsc, Grouping: grouping,
Budget: PointBudget{PerSeries: 500, Total: opts.MaxPoints},
Rounding: rounding, AllowMissing: true,
})
if err != nil {
return nil, fmt.Errorf("failed to query metrics: %w", err)
}
return assembleRecordsFromSeries(loaded.Values, clientUUID), nil
}
for _, metricName := range loadRecordMetricNames {
points, err := s.Series(ctx, metric.AggregateQuery{
Query: metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
},
Aggregation: recordMetricAggregation(metricName),
Interval: interval,
}, now)
if err != nil {
return nil, fmt.Errorf("failed to query metric %s: %w", metricName, err)
// assembleRecordsFromSeries unions metric buckets without allocating a record per key.
func assembleRecordsFromSeries(values map[string]map[metric.Aggregation][]metric.AggregatePoint, clientUUID string) []models.Record {
// Aligned metrics normally share buckets: the largest selected metric is a
// conservative capacity estimate, not the sum of the metric fanout. Missing
// buckets can grow the union beyond it (and duplicate seconds can shrink it).
capacity := 0
for name, aggregations := range values {
if n := len(aggregations[recordMetricAggregation(name)]); n > capacity {
capacity = n
}
}
records := make([]models.Record, 0, capacity)
recordIndex := make(map[recordSeriesKey]int, capacity)
for metricName, aggMap := range values {
agg := recordMetricAggregation(metricName)
points, ok := aggMap[agg]
if !ok {
continue
}
for _, point := range points {
entityID := point.EntityID
@@ -85,22 +125,22 @@ func getRecordsByClientAndTimeFromSeries(ctx context.Context, s *metric.Store, c
entityID = clientUUID
}
key := recordSeriesKey{client: entityID, ts: point.Bucket.Unix()}
if recordMap[key] == nil {
recordMap[key] = &models.Record{
index, exists := recordIndex[key]
if !exists {
index = len(records)
recordIndex[key] = index
records = append(records, models.Record{
Client: entityID,
Time: point.Bucket.UTC(),
}
})
}
applyRecordMetricValue(recordMap[key], metricName, point.Value)
// Resolve the address after append: the backing array may have moved.
applyRecordMetricValue(&records[index], metricName, point.Value)
}
}
records := make([]models.Record, 0, len(recordMap))
for _, rec := range recordMap {
records = append(records, *rec)
}
sortRecords(records)
return records, nil
return records
}
func getRecordMetricMaxByClientAndTimeFromSeries(ctx context.Context, s *metric.Store, clientUUID, recordMetric string, start, end time.Time) ([]models.Record, error) {
@@ -109,22 +149,19 @@ func getRecordMetricMaxByClientAndTimeFromSeries(ctx context.Context, s *metric.
return nil, fmt.Errorf("unsupported record metric %q", recordMetric)
}
now := time.Now().UTC()
points, err := s.Series(ctx, metric.AggregateQuery{
Query: metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
},
Aggregation: metric.AggMax,
Interval: recordSeriesInterval(s, start, end, now),
}, now)
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QuerySeries, Source: SourceRollup,
OmitTags: true, OmitLabels: true, CompactDefinitions: true,
Entities: EntityScope{IDs: []string{clientUUID}},
Specs: []MetricSpec{{Name: metricName, Aggregations: []metric.Aggregation{metric.AggMax}}},
Start: start, End: end, Order: metric.OrderAsc, Grouping: GroupMerged,
Budget: PointBudget{PerSeries: 500}, Rounding: RoundFloor, AllowMissing: true,
})
if err != nil {
return nil, fmt.Errorf("failed to query metric %s: %w", metricName, err)
}
points := loaded.Values[metricName][metric.AggMax]
records := make([]models.Record, 0, len(points))
for _, point := range points {
entityID := point.EntityID
@@ -149,49 +186,6 @@ func recordMetricAggregation(metricName string) metric.Aggregation {
}
}
func recordSeriesInterval(s *metric.Store, start, end, now time.Time) time.Duration {
interval := recordDownsampleInterval(end.Sub(start), 500)
return s.CompatibleSeriesInterval(start, now, interval)
}
func recordDownsampleInterval(rangeDuration time.Duration, maxPoints int) time.Duration {
if maxPoints <= 0 {
maxPoints = 500
}
nanos := rangeDuration.Nanoseconds()
if nanos <= 0 {
return time.Second
}
interval := time.Duration((nanos + int64(maxPoints) - 1) / int64(maxPoints))
if interval < time.Second {
return time.Second
}
return metric.FloorStandardInterval(interval)
}
func listRecordEntityIDs(ctx context.Context, s *metric.Store, start, end time.Time, interval time.Duration) ([]string, error) {
seen := make(map[string]struct{})
for _, metricName := range loadRecordMetricNames {
ids, err := s.EntityIDs(ctx, metric.Query{
MetricName: metricName,
Start: start.Add(-interval),
End: end,
})
if err != nil {
return nil, err
}
for _, id := range ids {
seen[id] = struct{}{}
}
}
out := make([]string, 0, len(seen))
for id := range seen {
out = append(out, id)
}
sort.Strings(out)
return out, nil
}
func applyRecordMetricValue(rec *models.Record, metricName string, value float64) {
switch metricName {
case MetricCPU:
@@ -253,17 +247,21 @@ func GetGPURecordsByClientAndTime(ctx context.Context, clientUUID string, start,
}
recordMap := make(map[gpuKey]*models.GPURecord)
specs := make([]MetricSpec, 0, len(gpuMetrics))
for _, name := range gpuMetrics {
specs = append(specs, MetricSpec{Name: name})
}
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QueryPoints, Source: SourceRaw,
OmitLabels: true, CompactDefinitions: true,
Entities: EntityScope{IDs: []string{clientUUID}}, Specs: specs,
Start: start, End: end, Order: metric.OrderAsc, AllowMissing: true,
})
if err != nil {
return nil, fmt.Errorf("failed to query GPU metrics: %w", err)
}
for _, metricName := range gpuMetrics {
points, err := s.Query(ctx, metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
})
if err != nil {
continue // GPU 数据可能不存在
}
points := loaded.Raw[metricName]
for _, p := range points {
deviceIndex := 0
@@ -307,7 +305,3 @@ func GetGPURecordsByClientAndTime(ctx context.Context, clientUUID string, start,
return records, nil
}
// GetPingRecords 从 metric store 查询兼容旧接口的 ping 记录。
//
// 旧接口过去直接读取 ping_records。这里使用与 queryMetrics 相同的 Series
+7 -7
View File
@@ -9,7 +9,7 @@ import (
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/metric"
v1 "github.com/komari-monitor/komari/protocol/v1"
v2 "github.com/komari-monitor/komari/protocol/v2"
)
func TestDefaultRollupPolicy(t *testing.T) {
@@ -572,15 +572,15 @@ func TestGetRecordsByClientAndTimeReadsRollupsAfterRawCompaction(t *testing.T) {
Connections: 321,
ConnectionsUdp: 12,
}
if _, err := WriteReport(ctx, v1.Report{
if _, err := WriteReport(ctx, v2.Report{
UUID: rec.Client,
UpdatedAt: ts,
CPU: v1.CPUReport{Usage: float64(rec.Cpu)},
Ram: v1.RamReport{Used: rec.Ram, Total: rec.RamTotal},
Load: v1.LoadReport{Load1: float64(rec.Load)},
Disk: v1.DiskReport{Used: rec.Disk, Total: rec.DiskTotal},
CPU: v2.CPUReport{Usage: float64(rec.Cpu)},
Ram: v2.RamReport{Used: rec.Ram, Total: rec.RamTotal},
Load: v2.LoadReport{Load1: float64(rec.Load)},
Disk: v2.DiskReport{Used: rec.Disk, Total: rec.DiskTotal},
Process: rec.Process,
Connections: v1.ConnectionsReport{
Connections: v2.ConnectionsReport{
TCP: rec.Connections,
UDP: rec.ConnectionsUdp,
},
+11 -31
View File
@@ -62,39 +62,33 @@ func writePingRecords(ctx context.Context, records []models.PingRecord) error {
return s.WriteBatch(ctx, points)
}
// GetPingRecords returns legacy-shaped ping records, newest first.
func GetPingRecords(ctx context.Context, clientUUID string, taskID int, start, end time.Time) ([]models.PingRecord, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
query := metric.Query{
MetricName: MetricPingLatency,
Start: start,
End: end,
Order: metric.OrderAsc,
query := MetricsQuery{
Store: s, Mode: QuerySeries, Source: SourceRollup,
CompactDefinitions: true,
Entities: EntityScope{All: clientUUID == ""},
Specs: []MetricSpec{{Name: MetricPingLatency, Aggregations: []metric.Aggregation{metric.AggLast}}},
Start: start, End: end, Order: metric.OrderAsc, Grouping: GroupSeries,
Budget: PointBudget{PerSeries: 4000}, Rounding: RoundFloor, AllowMissing: true,
}
if clientUUID != "" {
query.EntityID = clientUUID
query.Entities.IDs = []string{clientUUID}
}
if taskID >= 0 {
query.Tags = map[string]string{"task_id": fmt.Sprintf("%d", taskID)}
}
interval := pingQueryInterval(end.Sub(start), 4000)
interval = s.CompatibleSeriesInterval(start, time.Now().UTC(), interval)
points, err := s.Series(ctx, metric.AggregateQuery{
Query: query,
Aggregation: metric.AggLast,
Interval: interval,
PreserveSeries: true,
}, time.Now().UTC())
loaded, err := QueryMetrics(ctx, query)
if err != nil {
return nil, err
}
points := loaded.Values[MetricPingLatency][metric.AggLast]
records := make([]models.PingRecord, 0, len(points))
for _, p := range points {
taskIDVal := uint(0)
@@ -117,17 +111,3 @@ func GetPingRecords(ctx context.Context, clientUUID string, taskID int, start, e
return records, nil
}
func pingQueryInterval(rangeDuration time.Duration, maxPoints int) time.Duration {
if maxPoints <= 0 {
maxPoints = 4000
}
if rangeDuration <= 0 {
return time.Second
}
interval := time.Duration((rangeDuration.Nanoseconds() + int64(maxPoints) - 1) / int64(maxPoints))
if interval < time.Second {
return time.Second
}
return metric.FloorStandardInterval(interval)
}
+766
View File
@@ -0,0 +1,766 @@
package metricstore
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/komari-monitor/komari/pkg/metric"
)
// QueryMode selects exact samples, adaptive series, or exclusive-boundary lookups.
type QueryMode string
const (
QueryPoints QueryMode = "points"
QuerySeries QueryMode = "series"
QueryLatestBefore QueryMode = "latest_before"
)
type QuerySource string
const (
SourceAuto QuerySource = "auto"
SourceRaw QuerySource = "raw"
SourceRollup QuerySource = "rollup"
)
type QueryGrouping string
const (
GroupMerged QueryGrouping = "merged"
GroupEntity QueryGrouping = "entity"
GroupSeries QueryGrouping = "series"
)
type IntervalRounding string
const (
RoundFloor IntervalRounding = "floor"
RoundCeil IntervalRounding = "ceil"
)
// EntityScope deliberately distinguishes an empty authorized set from all entities.
// All and IDs are mutually exclusive. IDs are deduplicated without modifying callers.
type EntityScope struct {
All bool
IDs []string
}
type MetricSpec struct {
Name string
Aggregations []metric.Aggregation
// Positive intervals are explicit and are preserved, even for historical data.
Interval time.Duration
// Zero inherits Budget.PerSeries, then 500. -1 disables adaptive downsampling:
// a zero Interval then selects the finest compatible tier, not a whole-window bucket.
MaxPoints int
}
// PointBudget controls adaptive interval planning, not a raw-row SQL LIMIT.
// Total divides the point target across entities (metrics may be columns of one
// legacy record); it is not a hard cap across metrics, aggregations or tag variants.
// Raw queries and explicit intervals retain their exact historical semantics.
// Response adapters can impose their existing final sampling policy separately.
type PointBudget struct {
Total int
PerSeries int
}
type FillPolicy struct {
Gaps bool
PingNegativeAsNull bool
}
// QueryProjection controls retained result representations. The default keeps all
// source data for compatibility; chart requires Fill and retains only Filled.
type QueryProjection string
const (
ProjectionFull QueryProjection = "full"
ProjectionChart QueryProjection = "chart"
)
type MetricsQuery struct {
// Output omissions are opt-in; filtering and aggregation identity are unchanged.
OmitTags bool
OmitLabels bool
CompactDefinitions bool
Projection QueryProjection
Store *metric.Store
Mode QueryMode
Source QuerySource
Entities EntityScope
Specs []MetricSpec
Start, End time.Time
Tags map[string]string
Order metric.Order
Grouping QueryGrouping
Budget PointBudget
Rounding IntervalRounding
Fill FillPolicy
AllowMissing bool
Now time.Time
}
type MetricsResult struct {
Definitions map[string]metric.Definition
Raw map[string][]metric.Point
Values map[string]map[metric.Aggregation][]metric.AggregatePoint
Intervals map[string]time.Duration
IsRaw bool
Filled []MetricSeries
// Latest is keyed by metric name, then entity; missing samples are absent.
Latest map[string]map[string]metric.Point
}
type MetricSeries struct {
MetricName, EntityID string
Aggregation metric.Aggregation
Tags map[string]string
Interval time.Duration
Points []MetricValue
}
type MetricValue struct {
Time time.Time
Value *float64
Count int
Tags, Labels map[string]string
}
// DownsampleInterval returns a standard interval that keeps rangeDuration
// within about maxPoints buckets, never finer than one second. roundUp is used
// by point budgets: flooring can exceed the budget, while compatibility queries
// preserve the historical floor behavior.
func DownsampleInterval(rangeDuration time.Duration, maxPoints int, roundUp bool) time.Duration {
if maxPoints < 1 {
maxPoints = 1
}
points := time.Duration(maxPoints)
interval := (rangeDuration + points - 1) / points
if interval < time.Second {
return time.Second
}
if roundUp {
return metric.CeilStandardInterval(interval)
}
return metric.FloorStandardInterval(interval)
}
// QueryMetrics is the shared query policy boundary. Raw and aggregate slices are
// transferred directly from batch primitives; nullable chart data is allocated only
// when explicitly requested by Fill. All work uses ctx and one captured clock.
func QueryMetrics(ctx context.Context, req MetricsQuery) (MetricsResult, error) {
if ctx == nil {
return MetricsResult{}, fmt.Errorf("%w: context is required", metric.ErrInvalidArgument)
}
if err := ctx.Err(); err != nil {
return MetricsResult{}, err
}
result := MetricsResult{Definitions: map[string]metric.Definition{}, Raw: map[string][]metric.Point{}, Values: map[string]map[metric.Aggregation][]metric.AggregatePoint{}, Intervals: map[string]time.Duration{}, Latest: map[string]map[string]metric.Point{}}
if req.Mode == "" {
req.Mode = QuerySeries
}
if req.Source == "" {
req.Source = SourceAuto
}
if req.Grouping == "" {
req.Grouping = GroupMerged
}
if req.Order == "" {
req.Order = metric.OrderAsc
}
if req.Rounding == "" {
req.Rounding = RoundFloor
}
if err := validateMetricsQuery(req); err != nil {
return result, err
}
now := req.Now
if now.IsZero() {
now = time.Now()
}
now = now.UTC()
req.Start, req.End = req.Start.UTC(), req.End.UTC()
req.Entities.IDs = uniqueQueryIDs(req.Entities.IDs)
// Never let the primitive's empty entity filter broaden an empty authorized set.
s := req.Store
if s == nil {
s = GetStore()
}
if s == nil {
return result, fmt.Errorf("metric store not enabled")
}
names := make([]string, 0, len(req.Specs))
for _, spec := range req.Specs {
names = append(names, spec.Name)
}
loadDefinitions := s.GetMetrics
if req.CompactDefinitions {
loadDefinitions = s.GetMetricsCompact
}
if !req.Entities.All && len(req.Entities.IDs) == 0 {
definitions, err := loadDefinitions(ctx, names)
if err != nil {
return result, err
}
result.Definitions = definitions
result.IsRaw = req.Source == SourceRaw || (req.Source == SourceAuto && (req.Mode == QueryPoints || UsesRawWindow(req.Start, req.End, now)))
if err := checkQueryDefinitions(names, definitions, req.AllowMissing); err != nil {
return result, err
}
return result, ctx.Err()
}
if req.Mode == QueryLatestBefore {
definitions, err := loadDefinitions(ctx, names)
if err != nil {
return result, err
}
result.Definitions = definitions
if err := checkQueryDefinitions(names, definitions, req.AllowMissing); err != nil {
return result, err
}
for _, name := range names {
if _, ok := definitions[name]; !ok {
continue
}
ids := req.Entities.IDs
for _, id := range ids {
if err := ctx.Err(); err != nil {
return result, err
}
point, found, err := s.LatestBefore(ctx, name, id, req.End)
if err != nil {
return result, err
}
if found {
if result.Latest[name] == nil {
result.Latest[name] = map[string]metric.Point{}
}
result.Latest[name][id] = point
}
}
}
return result, ctx.Err()
}
result.IsRaw = req.Source == SourceRaw || (req.Source == SourceAuto && (req.Mode == QueryPoints || UsesRawWindow(req.Start, req.End, now)))
if result.IsRaw {
definitions, err := loadDefinitions(ctx, names)
if err != nil {
return result, err
}
result.Definitions = definitions
if err := checkQueryDefinitions(names, definitions, req.AllowMissing); err != nil {
return result, err
}
known := make([]string, 0, len(names))
for _, name := range names {
if _, ok := definitions[name]; ok {
known = append(known, name)
result.Intervals[name] = 0
}
}
if len(known) > 0 {
result.Raw, err = s.QueryBatch(ctx, metric.BatchQuery{MetricNames: known, EntityIDs: req.Entities.IDs, Start: req.Start, End: req.End, Tags: req.Tags, Order: req.Order, OmitTags: req.OmitTags, OmitLabels: req.OmitLabels, IncludeSeriesIdentity: req.Fill.Gaps || req.Fill.PingNegativeAsNull})
if err != nil {
return result, err
}
}
} else {
entityCount := len(req.Entities.IDs)
if needsTotalEntityCount(req) {
if req.Entities.All {
count, err := s.CountEntities(ctx, names)
if err != nil {
return result, err
}
entityCount = count
}
// Explicit IDs are a conservative dictionary-count upper bound; never
// scan every metric/tier simply to plan a response budget.
}
if entityCount < 1 {
entityCount = 1
}
specs := make([]metric.BatchSeriesSpec, 0, len(req.Specs))
for _, spec := range req.Specs {
interval := spec.Interval
if interval == 0 {
points := spec.MaxPoints
if points == 0 {
points = req.Budget.PerSeries
}
if points == 0 {
points = 500
}
if points == 500 && spec.MaxPoints == 0 && req.Budget.PerSeries == 0 && req.Budget.Total == -1 {
points = -1
}
interval = time.Second
if points != -1 {
if req.Budget.Total > 0 {
budget := req.Budget.Total / entityCount
if budget < 1 {
budget = 1
}
if budget < points {
points = budget
}
}
interval = DownsampleInterval(req.End.Sub(req.Start), points, req.Rounding == RoundCeil)
}
interval = s.CompatibleSeriesInterval(req.Start, now, interval)
if interval <= 0 {
interval = time.Second
}
}
aggregations := spec.Aggregations
if len(aggregations) == 0 {
aggregations = []metric.Aggregation{metric.AggAvg}
}
specs = append(specs, metric.BatchSeriesSpec{MetricName: spec.Name, Aggregations: aggregations, Interval: interval, PreserveSeries: req.Grouping == GroupSeries, GroupByEntity: req.Grouping == GroupEntity})
result.Intervals[spec.Name] = interval
}
batch, err := s.SeriesBatch(ctx, metric.BatchSeriesQuery{Specs: specs, EntityIDs: req.Entities.IDs, Start: req.Start, End: req.End, Tags: req.Tags, Order: req.Order, OmitTags: req.OmitTags, CompactDefinitions: req.CompactDefinitions, IncludeSeriesIdentity: req.Fill.Gaps || req.Fill.PingNegativeAsNull}, now)
if err != nil {
return result, err
}
result.Definitions, result.Values = batch.Definitions, batch.Values
// SeriesBatch already loads metadata. Validate its result rather than issue a
// second metadata query; unknown metrics otherwise look like empty series.
if err := checkQueryDefinitions(names, batch.Definitions, req.AllowMissing); err != nil {
return result, err
}
for _, name := range names {
if _, ok := batch.Definitions[name]; !ok {
delete(result.Intervals, name)
delete(result.Values, name)
}
}
}
if req.Fill.Gaps || req.Fill.PingNegativeAsNull {
filled, err := buildQuerySeriesOwned(ctx, req, result, req.Projection == ProjectionChart)
if err != nil {
return result, err
}
result.Filled = filled
if req.Projection == ProjectionChart {
result.Raw, result.Values = nil, nil
}
}
return result, ctx.Err()
}
// needsTotalEntityCount excludes explicit intervals and unlimited specs: neither
// consumes the total adaptive budget. Keep its precedence aligned with planning.
func needsTotalEntityCount(req MetricsQuery) bool {
if req.Budget.Total <= 0 {
return false
}
for _, spec := range req.Specs {
if spec.Interval != 0 {
continue
}
points := spec.MaxPoints
if points == 0 {
points = req.Budget.PerSeries
}
if points != -1 {
return true
}
}
return false
}
func validateMetricsQuery(req MetricsQuery) error {
invalid := func(message string) error { return fmt.Errorf("%w: %s", metric.ErrInvalidArgument, message) }
if req.OmitTags && (req.Fill.Gaps || req.Fill.PingNegativeAsNull) {
return invalid("omitting tags with filled output is unsupported: series identity must be preserved")
}
if req.Mode == QueryLatestBefore && (req.OmitTags || req.OmitLabels) {
return invalid("latest-before output omissions are unsupported")
}
switch req.Mode {
case QueryPoints, QuerySeries, QueryLatestBefore:
default:
return invalid("unsupported query mode")
}
switch req.Source {
case SourceAuto, SourceRaw, SourceRollup:
default:
return invalid("unsupported query source")
}
switch req.Grouping {
case GroupMerged, GroupEntity, GroupSeries:
default:
return invalid("unsupported query grouping")
}
switch req.Projection {
case "", ProjectionFull:
case ProjectionChart:
if req.Mode == QueryLatestBefore || (!req.Fill.Gaps && !req.Fill.PingNegativeAsNull) {
return invalid("chart projection requires fill and does not support latest-before")
}
default:
return invalid("unsupported query projection")
}
switch req.Rounding {
case RoundFloor, RoundCeil:
default:
return invalid("unsupported interval rounding")
}
if req.Order != metric.OrderAsc && req.Order != metric.OrderDesc {
return invalid("unsupported query order")
}
if req.Entities.All && len(req.Entities.IDs) > 0 {
return invalid("all entities and explicit IDs are mutually exclusive")
}
for _, id := range req.Entities.IDs {
if strings.TrimSpace(id) == "" {
return invalid("entity ID cannot be empty")
}
}
if req.End.IsZero() {
return invalid("end time is required")
}
if req.Mode != QueryLatestBefore && (req.Start.IsZero() || req.End.Before(req.Start)) {
return invalid("valid start and end times are required")
}
if req.Mode == QueryLatestBefore && (len(req.Tags) > 0 || req.Source != SourceAuto) {
return invalid("latest-before requires auto source and does not support tag filters")
}
if req.Mode == QueryPoints && req.Source == SourceRollup {
return invalid("points mode requires raw or auto source")
}
if req.Mode == QueryLatestBefore && req.Entities.All {
return invalid("latest-before requires explicit entity IDs")
}
if req.Budget.Total < -1 || req.Budget.PerSeries < -1 {
return invalid("invalid point budget")
}
if len(req.Specs) == 0 {
return invalid("at least one metric specification is required")
}
seen := map[string]bool{}
for _, spec := range req.Specs {
if strings.TrimSpace(spec.Name) == "" || seen[spec.Name] {
return invalid("metric names must be nonempty and unique")
}
seen[spec.Name] = true
if spec.Interval < 0 || spec.MaxPoints < -1 {
return invalid("invalid metric interval or point budget")
}
for _, agg := range spec.Aggregations {
// Reuse primitive aggregation validation even when the selected source is raw.
if err := (metric.AggregateQuery{Query: metric.Query{MetricName: spec.Name, Start: req.End, End: req.End}, Aggregation: agg, Interval: time.Second}).Validate(); err != nil {
return err
}
}
}
return nil
}
// UsesRawWindow selects the public adaptive raw window; its end boundary is strict.
func UsesRawWindow(start, end, now time.Time) bool {
return end.Sub(start) <= 10*time.Minute && end.After(now.Add(-10*time.Minute))
}
func uniqueQueryIDs(ids []string) []string {
out := make([]string, 0, len(ids))
seen := map[string]bool{}
for _, id := range ids {
if !seen[id] {
out = append(out, id)
seen[id] = true
}
}
return out
}
func checkQueryDefinitions(names []string, definitions map[string]metric.Definition, allowMissing bool) error {
if !allowMissing {
for _, name := range names {
if _, ok := definitions[name]; !ok {
return fmt.Errorf("%w: metric %q", metric.ErrNotFound, name)
}
}
}
return nil
}
// querySeriesKey consumes storage's immutable, collision-free output identity.
// Manually constructed points (and low-level queries without the opt-in) retain
// the old canonical fallback. Neither grouping nor ordering uses a hash.
func querySeriesKey(entity string, tags map[string]string, identity *metric.SeriesIdentity) string {
if key := identity.Key(); key != "" {
return key
}
return metric.NewSeriesIdentity(entity, tags).Key()
}
func buildQuerySeries(ctx context.Context, req MetricsQuery, result MetricsResult) ([]MetricSeries, error) {
return buildQuerySeriesOwned(ctx, req, result, false)
}
// detach is only valid for QueryMetrics-owned source maps. Release each source
// array after conversion, before allocating gaps or processing the next metric.
func buildQuerySeriesOwned(ctx context.Context, req MetricsQuery, result MetricsResult, detach bool) ([]MetricSeries, error) {
out := []MetricSeries{}
for _, spec := range req.Specs {
if err := ctx.Err(); err != nil {
return nil, err
}
if _, ok := result.Definitions[spec.Name]; !ok {
continue
}
aggregations := spec.Aggregations
if result.IsRaw {
aggregations = []metric.Aggregation{""}
} else if len(aggregations) == 0 {
aggregations = []metric.Aggregation{metric.AggAvg}
}
for aggIndex, agg := range aggregations {
if err := ctx.Err(); err != nil {
return nil, err
}
var seenEntities map[string]bool
if len(req.Entities.IDs) > 0 && (result.IsRaw || req.Grouping != GroupMerged) {
seenEntities = make(map[string]bool, len(req.Entities.IDs))
}
groups := map[string]*MetricSeries{}
add := func(entity string, tags map[string]string, identity *metric.SeriesIdentity, value MetricValue) {
// Exact samples always preserve identity; aggregate identity is determined by
// the requested primitive grouping and must not be collapsed again here.
key := querySeriesKey(entity, tags, identity)
series := groups[key]
if series == nil {
series = &MetricSeries{MetricName: spec.Name, EntityID: entity, Aggregation: agg, Tags: tags, Interval: result.Intervals[spec.Name]}
groups[key] = series
if seenEntities != nil {
seenEntities[entity] = true
}
}
if req.Fill.PingNegativeAsNull && (spec.Name == MetricPingLatency || spec.Name == MetricPingLoss) && value.Value != nil && *value.Value == -1 {
value.Value = nil
}
series.Points = append(series.Points, value)
}
if result.IsRaw {
for i, p := range result.Raw[spec.Name] {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
v := p.Value
add(p.EntityID, p.Tags, p.SeriesIdentity, MetricValue{Time: p.Timestamp, Value: &v, Count: 1, Tags: p.Tags, Labels: p.Labels})
}
} else {
for i, p := range result.Values[spec.Name][agg] {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
v := p.Value
add(p.EntityID, p.Tags, p.SeriesIdentity, MetricValue{Time: p.Bucket, Value: &v, Count: p.Count, Tags: p.Tags})
}
}
if detach {
if result.IsRaw {
delete(result.Raw, spec.Name)
} else {
// Duplicate aggregation specs historically produce duplicate series.
needed := false
for _, next := range aggregations[aggIndex+1:] {
needed = needed || next == agg
}
if !needed {
delete(result.Values[spec.Name], agg)
}
}
}
// Seed missing entities directly; do not scan all groups for each ID or
// allocate a placeholder point only to discard it.
seed := func(id string) {
groups[querySeriesKey(id, req.Tags, nil)] = &MetricSeries{MetricName: spec.Name, EntityID: id, Aggregation: agg, Tags: req.Tags, Interval: result.Intervals[spec.Name]}
if seenEntities != nil {
seenEntities[id] = true
}
}
if result.IsRaw || req.Grouping != GroupMerged {
for i, id := range req.Entities.IDs {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
if !seenEntities[id] {
seed(id)
}
}
}
if len(groups) == 0 {
ids := req.Entities.IDs
if len(ids) == 0 || (!result.IsRaw && req.Grouping == GroupMerged) {
ids = []string{""}
}
for _, id := range ids {
seed(id)
}
}
keys := make([]string, 0, len(groups))
for key := range groups {
if len(keys)%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
keys = append(keys, key)
}
if err := ctx.Err(); err != nil {
return nil, err
}
sort.Strings(keys)
for _, key := range keys {
if err := ctx.Err(); err != nil {
return nil, err
}
series := *groups[key]
sort.SliceStable(series.Points, func(i, j int) bool { return series.Points[i].Time.Before(series.Points[j].Time) })
if err := ctx.Err(); err != nil {
return nil, err
}
if req.Fill.Gaps {
var err error
series, err = fillMetricSeriesOwned(ctx, series, req.Start, req.End)
if err != nil {
return nil, err
}
}
if req.Order == metric.OrderDesc {
for i, j := 0, len(series.Points)-1; i < j; i, j = i+1, j-1 {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
series.Points[i], series.Points[j] = series.Points[j], series.Points[i]
}
}
out = append(out, series)
}
}
}
return out, ctx.Err()
}
// FillMetricSeries matches the public chart's adaptive lower-quartile gap policy.
func FillMetricSeries(series MetricSeries, start, end time.Time) MetricSeries {
filled, _ := fillMetricSeries(context.Background(), series, start, end)
return filled
}
func fillMetricSeries(ctx context.Context, series MetricSeries, start, end time.Time) (MetricSeries, error) {
if err := ctx.Err(); err != nil {
return series, err
}
// Public callers retain ownership, including spare capacity. Never write there.
series.Points = append([]MetricValue(nil), series.Points...)
return fillMetricSeriesOwned(ctx, series, start, end)
}
// fillMetricSeriesOwned inserts gaps backwards into exclusively owned storage.
// Dense series reuse their point array; sparse series grow only when capacity
// cannot accommodate the exact number of gaps, rather than always copying into
// a second full point array. The public wrapper above remains non-mutating.
func fillMetricSeriesOwned(ctx context.Context, series MetricSeries, start, end time.Time) (MetricSeries, error) {
if err := ctx.Err(); err != nil {
return series, err
}
deltas := make([]time.Duration, 0, len(series.Points))
for i := 1; i < len(series.Points); i++ {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return series, err
}
}
if d := series.Points[i].Time.Sub(series.Points[i-1].Time); d > 0 {
deltas = append(deltas, d)
}
}
if err := ctx.Err(); err != nil {
return series, err
}
expected := series.Interval
if len(deltas) >= 2 {
sort.Slice(deltas, func(i, j int) bool { return deltas[i] < deltas[j] })
if observed := deltas[(len(deltas)-1)/4]; observed > expected {
expected = observed
}
}
if err := ctx.Err(); err != nil {
return series, err
}
if expected > 0 {
series.Interval = expected
}
null := func(at time.Time) MetricValue { return MetricValue{Time: at.UTC(), Tags: series.Tags} }
n := len(series.Points)
if n == 0 {
series.Points = append(series.Points, null(start), null(end))
return series, ctx.Err()
}
leading := start.Before(series.Points[0].Time)
gap := func(i int) bool {
return i > 0 && expected > 0 && series.Points[i-1].Value != nil && series.Points[i].Value != nil && series.Points[i].Time.Sub(series.Points[i-1].Time) > expected+expected/2
}
extra := 0
if leading {
extra++
}
for i := 1; i < n; i++ {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return series, err
}
}
if gap(i) {
extra++
}
}
if extra == 0 {
return series, ctx.Err()
}
if cap(series.Points) < n+extra {
points := make([]MetricValue, n+extra)
copy(points, series.Points)
series.Points = points
} else {
series.Points = series.Points[:n+extra]
}
write := n + extra - 1
for i := n - 1; i >= 0; i-- {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return series, err
}
}
// Decide before moving: write may equal i, but never precedes it.
insert := gap(i)
var at time.Time
if insert {
at = series.Points[i-1].Time.Add(expected)
}
series.Points[write] = series.Points[i]
write--
if insert {
series.Points[write] = null(at)
write--
}
}
if leading {
series.Points[0] = null(start)
}
return series, ctx.Err()
}
+63
View File
@@ -0,0 +1,63 @@
package metricstore
import (
"context"
"database/sql"
"fmt"
"github.com/komari-monitor/komari/pkg/metric"
)
// QueryContext runs a raw read query against the active metric store. The
// caller must invoke the returned release function after closing rows.
func QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, metric.Driver, func(), error) {
return QueryForDriver(ctx, func(metric.Driver) (string, error) {
return query, nil
}, args...)
}
// QueryForDriver runs a raw read query whose SQL depends on the active store's
// driver. The caller must invoke the returned release function after closing rows.
func QueryForDriver(ctx context.Context, query func(metric.Driver) (string, error), args ...any) (*sql.Rows, metric.Driver, func(), error) {
if err := storeOperations.AcquireShared(ctx); err != nil {
return nil, "", nil, fmt.Errorf("wait for metric store operations before query: %w", err)
}
storeMu.RLock()
activeStore := store
storeMu.RUnlock()
if activeStore == nil {
storeOperations.ReleaseShared()
return nil, "", nil, ErrStoreNotInitialized
}
statement, err := query(activeStore.Driver())
if err != nil {
storeOperations.ReleaseShared()
return nil, "", nil, err
}
rows, err := activeStore.QueryContext(ctx, statement, args...)
if err != nil {
storeOperations.ReleaseShared()
return nil, "", nil, err
}
return rows, activeStore.Driver(), storeOperations.ReleaseShared, nil
}
// ExecContext runs a raw statement against the active metric store.
func ExecContext(ctx context.Context, query string, args ...any) (sql.Result, metric.Driver, error) {
if err := storeOperations.Acquire(ctx); err != nil {
return nil, "", fmt.Errorf("wait for metric store operations before execution: %w", err)
}
defer storeOperations.Release()
storeMu.RLock()
activeStore := store
storeMu.RUnlock()
if activeStore == nil {
return nil, "", ErrStoreNotInitialized
}
result, err := activeStore.ExecContext(ctx, query, args...)
return result, activeStore.Driver(), err
}
+78 -87
View File
@@ -11,7 +11,7 @@ import (
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/metric"
v1 "github.com/komari-monitor/komari/protocol/v1"
v2 "github.com/komari-monitor/komari/protocol/v2"
)
type reportTrafficState struct {
@@ -25,17 +25,25 @@ type reportTrafficValues struct {
timestamp time.Time
hasUp bool
totalUp int64
totalUpAt time.Time
hasDown bool
totalDown int64
totalDownAt time.Time
}
var reportTrafficStates sync.Map
var (
reportTrafficStates sync.Map
reportWriteMu sync.Mutex
)
const (
reportBatchInterval = 3 * time.Second
reportBatchQueueSize = 4096
reportBatchQueueSize = 512
pingBatchQueueSize = 4096
pingBatchMaxRecords = 512
reportBatchWriteTimeout = 10 * time.Second
// Longer gaps have no reliable per-bucket traffic attribution.
maxTrafficReportGap = 15 * time.Minute
)
var (
@@ -57,7 +65,7 @@ type reportBatchRequest struct {
type reportBatchWorker struct {
mu sync.Mutex
queue chan v1.Report
queue chan v2.Report
pingQueue chan models.PingRecord
requests chan reportBatchRequest
done chan struct{}
@@ -73,8 +81,8 @@ func StartReportBatcher() {
return
}
worker := &reportBatchWorker{
queue: make(chan v1.Report, reportBatchQueueSize),
pingQueue: make(chan models.PingRecord, reportBatchQueueSize),
queue: make(chan v2.Report, reportBatchQueueSize),
pingQueue: make(chan models.PingRecord, pingBatchQueueSize),
requests: make(chan reportBatchRequest, 1),
done: make(chan struct{}),
}
@@ -150,16 +158,16 @@ func FlushReportBatch(ctx context.Context) error {
// WriteReport persists one agent report and adds it to in-memory minute
// summaries using the same server receive time. Traffic deltas remain summable
// after rollup.
func WriteReport(ctx context.Context, report v1.Report) (v1.Report, error) {
func WriteReport(ctx context.Context, report v2.Report) (v2.Report, error) {
if report.UUID == "" {
return v1.Report{}, fmt.Errorf("report UUID is required")
return v2.Report{}, fmt.Errorf("report UUID is required")
}
if report.UpdatedAt.IsZero() {
return v1.Report{}, fmt.Errorf("report receive time is required")
return v2.Report{}, fmt.Errorf("report receive time is required")
}
report.UpdatedAt = report.UpdatedAt.UTC()
if GetStore() == nil {
return v1.Report{}, fmt.Errorf("metric store not enabled")
return v2.Report{}, fmt.Errorf("metric store not enabled")
}
reportBatcherMu.Lock()
@@ -167,19 +175,19 @@ func WriteReport(ctx context.Context, report v1.Report) (v1.Report, error) {
reportBatcherMu.Unlock()
if worker != nil {
if err := worker.enqueue(ctx, report); err != nil {
return v1.Report{}, err
return v2.Report{}, err
}
return report, nil
}
saved, err := writeReportBatch(ctx, []v1.Report{report})
saved, err := writeReportBatch(ctx, []v2.Report{report})
if err != nil {
return v1.Report{}, err
return v2.Report{}, err
}
return saved[0], nil
}
func (w *reportBatchWorker) enqueue(ctx context.Context, report v1.Report) error {
func (w *reportBatchWorker) enqueue(ctx context.Context, report v2.Report) error {
w.mu.Lock()
defer w.mu.Unlock()
if w.stopping {
@@ -200,13 +208,13 @@ func (w *reportBatchWorker) run() {
ticker := time.NewTicker(reportBatchInterval)
defer ticker.Stop()
var pending []v1.Report
var pending []v2.Report
var pendingPings []models.PingRecord
for {
select {
case request := <-w.requests:
pending = append(pending, drainReportQueue(w.queue, reportBatchQueueSize)...)
pendingPings = append(pendingPings, drainPingQueue(w.pingQueue, reportBatchQueueSize)...)
pending = append(pending, drainQueue(w.queue, reportBatchQueueSize)...)
pendingPings = append(pendingPings, drainQueue(w.pingQueue, pingBatchQueueSize)...)
err := errors.Join(
writePendingReports(request.ctx, &pending),
writePendingPingRecords(request.ctx, &pendingPings),
@@ -221,11 +229,11 @@ func (w *reportBatchWorker) run() {
}
request.done <- err
case <-ticker.C:
pendingPings = append(pendingPings, drainPingQueue(w.pingQueue, reportBatchQueueSize)...)
pendingPings = append(pendingPings, drainQueue(w.pingQueue, pingBatchQueueSize)...)
if err := writePendingPingRecords(context.Background(), &pendingPings); err != nil {
logger.Errorf("metricstore", "failed to flush ping batch: %v", err)
}
pending = append(pending, drainReportQueue(w.queue, reportBatchQueueSize)...)
pending = append(pending, drainQueue(w.queue, reportBatchQueueSize)...)
if err := writePendingReports(context.Background(), &pending); err != nil {
logger.Errorf("metricstore", "failed to flush metric report batch: %v", err)
}
@@ -250,69 +258,36 @@ func (w *reportBatchWorker) enqueuePing(ctx context.Context, record models.PingR
}
}
func drainReportQueue(queue <-chan v1.Report, limit int) []v1.Report {
if limit <= 0 {
// drainQueue takes up to limit queued items without blocking.
func drainQueue[T any](queue <-chan T, limit int) []T {
n := min(len(queue), limit)
if n <= 0 {
return nil
}
capacity := len(queue)
if capacity > limit {
capacity = limit
}
if capacity == 0 {
return nil
}
reports := make([]v1.Report, 0, capacity)
for len(reports) < limit {
items := make([]T, 0, n)
for len(items) < limit {
select {
case report := <-queue:
reports = append(reports, report)
case item := <-queue:
items = append(items, item)
default:
return reports
return items
}
}
return reports
return items
}
func drainPingQueue(queue <-chan models.PingRecord, limit int) []models.PingRecord {
if limit <= 0 {
return nil
}
capacity := len(queue)
if capacity > limit {
capacity = limit
}
if capacity == 0 {
return nil
}
records := make([]models.PingRecord, 0, capacity)
for len(records) < limit {
select {
case record := <-queue:
records = append(records, record)
default:
return records
}
}
return records
}
func writePendingReports(ctx context.Context, pending *[]v1.Report) error {
// writePendingReports commits all pending reports as one batch and keeps them
// for the next attempt when the write fails.
func writePendingReports(ctx context.Context, pending *[]v2.Report) error {
if len(*pending) == 0 {
return nil
}
batchSize := len(*pending)
for len(*pending) > 0 {
if batchSize > len(*pending) {
batchSize = len(*pending)
}
writeCtx, cancel := context.WithTimeout(ctx, reportBatchWriteTimeout)
_, err := writeReportBatch(writeCtx, (*pending)[:batchSize])
cancel()
if err != nil {
return err
}
*pending = (*pending)[batchSize:]
writeCtx, cancel := context.WithTimeout(ctx, reportBatchWriteTimeout)
defer cancel()
if _, err := writeReportBatch(writeCtx, *pending); err != nil {
return err
}
*pending = (*pending)[:0]
return nil
}
@@ -333,7 +308,7 @@ func writePendingPingRecords(ctx context.Context, pending *[]models.PingRecord)
return nil
}
func writeReportBatch(ctx context.Context, reports []v1.Report) ([]v1.Report, error) {
func writeReportBatch(ctx context.Context, reports []v2.Report) ([]v2.Report, error) {
if len(reports) == 0 {
return nil, nil
}
@@ -342,12 +317,17 @@ func writeReportBatch(ctx context.Context, reports []v1.Report) ([]v1.Report, er
}
defer storeOperations.ReleaseShared()
// Serialize baseline reads and commits, including direct writes without
// the background batcher.
reportWriteMu.Lock()
defer reportWriteMu.Unlock()
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
prepared := make([]v1.Report, len(reports))
prepared := make([]v2.Report, len(reports))
copy(prepared, reports)
points := make([]metric.Point, 0, len(reports)*20)
pendingStates := make(map[*reportTrafficState]reportTrafficValues)
@@ -361,43 +341,50 @@ func writeReportBatch(ctx context.Context, reports []v1.Report) ([]v1.Report, er
state.mu.Unlock()
}
if !values.initialized {
totalUp, hasUp, err := latestReportCounter(ctx, s, MetricNetTotalUp, report.UUID, report.UpdatedAt)
if err == nil {
values.totalUp = totalUp
values.hasUp = hasUp
} else if ctx.Err() == nil {
totalUp, hasUp, err := latestReportPoint(ctx, s, MetricNetTotalUp, report.UUID, report.UpdatedAt)
if err != nil {
logger.Errorf("metricstore", "failed to restore previous upload counter for %s: %v", report.UUID, err)
} else {
values.totalUp = int64(totalUp.Value)
values.totalUpAt = totalUp.Timestamp
values.hasUp = hasUp
}
totalDown, hasDown, err := latestReportCounter(ctx, s, MetricNetTotalDown, report.UUID, report.UpdatedAt)
if err == nil {
values.totalDown = totalDown
values.hasDown = hasDown
} else if ctx.Err() == nil {
totalDown, hasDown, err := latestReportPoint(ctx, s, MetricNetTotalDown, report.UUID, report.UpdatedAt)
if err != nil {
logger.Errorf("metricstore", "failed to restore previous download counter for %s: %v", report.UUID, err)
}
if err := ctx.Err(); err != nil {
return nil, err
} else {
values.totalDown = int64(totalDown.Value)
values.totalDownAt = totalDown.Timestamp
values.hasDown = hasDown
}
values.initialized = true
// Keep only the restored baseline on failure. WriteBatch may have
// accepted raw samples before a rollup flush fails, so restoring
// again on retry could otherwise read this uncommitted batch.
state.mu.Lock()
state.reportTrafficValues = values
state.mu.Unlock()
}
if !values.timestamp.IsZero() && !report.UpdatedAt.After(values.timestamp) {
report.UpdatedAt = values.timestamp.Add(time.Millisecond)
}
trafficUp := int64(0)
if values.hasUp {
if values.hasUp && !values.totalUpAt.IsZero() && report.UpdatedAt.Sub(values.totalUpAt) <= maxTrafficReportGap {
trafficUp = TrafficCounterDelta(report.Network.TotalUp, values.totalUp)
}
trafficDown := int64(0)
if values.hasDown {
if values.hasDown && !values.totalDownAt.IsZero() && report.UpdatedAt.Sub(values.totalDownAt) <= maxTrafficReportGap {
trafficDown = TrafficCounterDelta(report.Network.TotalDown, values.totalDown)
}
points = append(points, reportMetricPoints(report, trafficUp, trafficDown)...)
values.timestamp = report.UpdatedAt
values.hasUp = true
values.totalUp = report.Network.TotalUp
values.totalUpAt = report.UpdatedAt
values.hasDown = true
values.totalDown = report.Network.TotalDown
values.totalDownAt = report.UpdatedAt
pendingStates[state] = values
prepared[i] = report
}
@@ -405,10 +392,14 @@ func writeReportBatch(ctx context.Context, reports []v1.Report) ([]v1.Report, er
if err := s.WriteBatch(ctx, points); err != nil {
return nil, err
}
// Advance counters and timestamps only after success. Retried batches
// must replace the same samples, not manufacture resets or new timestamps.
for state, values := range pendingStates {
state.mu.Lock()
state.reportTrafficValues = values
state.mu.Unlock()
}
return prepared, nil
}
+29 -4
View File
@@ -8,10 +8,10 @@ import (
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/metric"
v1 "github.com/komari-monitor/komari/protocol/v1"
v2 "github.com/komari-monitor/komari/protocol/v2"
)
func reportMetricPoints(report v1.Report, trafficUp, trafficDown int64) []metric.Point {
func reportMetricPoints(report v2.Report, trafficUp, trafficDown int64) []metric.Point {
entityID := report.UUID
ts := report.UpdatedAt
points := []metric.Point{
@@ -49,8 +49,23 @@ func reportMetricPoints(report v1.Report, trafficUp, trafficDown int64) []metric
return points
}
// latestReportPoint preserves independent counter restoration errors: one failed
// direction must not prevent the other from restoring its baseline.
func latestReportPoint(ctx context.Context, s *metric.Store, metricName, entityID string, before time.Time) (metric.Point, bool, error) {
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QueryLatestBefore, Source: SourceAuto,
Entities: EntityScope{IDs: []string{entityID}},
Specs: []MetricSpec{{Name: metricName}}, End: before, AllowMissing: true,
})
if err != nil {
return metric.Point{}, false, err
}
point, ok := loaded.Latest[metricName][entityID]
return point, ok, nil
}
func latestReportCounter(ctx context.Context, s *metric.Store, metricName, entityID string, before time.Time) (int64, bool, error) {
point, ok, err := s.LatestBefore(ctx, metricName, entityID, before)
point, ok, err := latestReportPoint(ctx, s, metricName, entityID, before)
if err != nil {
return 0, false, err
}
@@ -93,8 +108,15 @@ func GetLatestTrafficBefore(ctx context.Context, entityIDs []string, before time
return result, nil
}
// maxResetAwareDelta is the largest per-sample increase accepted when a
// counter appears to reset. A "reset" whose current value is larger than this
// is treated as jitter of a still-huge counter, not a wrap or reboot leftover.
const maxResetAwareDelta int64 = 64 << 30
// TrafficCounterDelta returns a reset-aware increase between two cumulative
// traffic counters. After a reset, the current counter is the new increase.
// traffic counters. After a wrap or reboot, the current counter is the new
// increase. A tiny dip of a TB-scale counter is not a reset: counting the
// still-huge current value would inject a multi-terabyte spike.
func TrafficCounterDelta(current, previous int64) int64 {
if current < 0 || previous < 0 {
return 0
@@ -102,6 +124,9 @@ func TrafficCounterDelta(current, previous int64) int64 {
if current >= previous {
return current - previous
}
if current > maxResetAwareDelta {
return 0
}
return current
}
+426 -26
View File
@@ -2,14 +2,18 @@ package metricstore
import (
"context"
"database/sql"
"database/sql/driver"
"errors"
"fmt"
"sync/atomic"
"testing"
"time"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/metric"
v1 "github.com/komari-monitor/komari/protocol/v1"
v2 "github.com/komari-monitor/komari/protocol/v2"
sqlite3 "github.com/mattn/go-sqlite3"
)
func useReportTestStore(t *testing.T, policy *metric.RollupPolicy) *metric.Store {
@@ -44,6 +48,198 @@ func useReportTestStore(t *testing.T, policy *metric.RollupPolicy) *metric.Store
return s
}
type reportCounterFault struct {
remaining atomic.Int32
denied atomic.Int32
failWrite atomic.Bool
}
func (f *reportCounterFault) denyRollupRead() bool {
for {
remaining := f.remaining.Load()
if remaining <= 0 {
return false
}
if f.remaining.CompareAndSwap(remaining, remaining-1) {
f.denied.Add(1)
return true
}
}
}
type reportSQLiteConnector struct {
driver *sqlite3.SQLiteDriver
dsn string
}
func (c *reportSQLiteConnector) Connect(context.Context) (driver.Conn, error) {
return c.driver.Open(c.dsn)
}
func (c *reportSQLiteConnector) Driver() driver.Driver {
return c.driver
}
// useReportCounterFailureStore denies exactly two rollup reads after schema
// setup, exercising failed counter restoration while leaving writes usable.
func useReportCounterFailureStore(t *testing.T) (*metric.Store, *reportCounterFault) {
t.Helper()
fault := &reportCounterFault{}
dsn := fmt.Sprintf("file:report-counter-fault-%d?mode=memory&cache=shared", time.Now().UnixNano())
driver := &sqlite3.SQLiteDriver{
ConnectHook: func(conn *sqlite3.SQLiteConn) error {
conn.RegisterAuthorizer(func(op int, arg1, _, _ string) int {
if op == sqlite3.SQLITE_TRANSACTION && arg1 == "BEGIN" && fault.failWrite.Load() {
return sqlite3.SQLITE_DENY
}
if op == sqlite3.SQLITE_READ && arg1 == "metric_rollups" && fault.denyRollupRead() {
return sqlite3.SQLITE_DENY
}
return sqlite3.SQLITE_OK
})
return nil
},
}
db := sql.OpenDB(&reportSQLiteConnector{driver: driver, dsn: dsn})
s, err := metric.Open(context.Background(), metric.SQLite("", metric.WithDB(db), metric.WithMaxOpenConns(1)))
if err != nil {
_ = db.Close()
t.Fatalf("open metric store: %v", err)
}
if err := createMetricDefinitions(context.Background(), s); err != nil {
_ = s.Close()
_ = db.Close()
t.Fatalf("create metric definitions: %v", err)
}
fault.remaining.Store(2)
storeMu.Lock()
previous := store
store = s
storeMu.Unlock()
t.Cleanup(func() {
clearReportTrafficStates()
storeMu.Lock()
store = previous
storeMu.Unlock()
_ = s.Close()
_ = db.Close()
})
return s, fault
}
func TestReportBatchCounterRestoreFailureInitializesStateOnce(t *testing.T) {
s, fault := useReportCounterFailureStore(t)
ctx := context.Background()
base := time.Now().UTC().Truncate(time.Second)
first := v2.Report{
UUID: "counter-restore-failure",
UpdatedAt: base,
CPU: v2.CPUReport{Usage: 10},
Network: v2.NetworkReport{TotalUp: 100, TotalDown: 200},
}
if _, err := writeReportBatch(ctx, []v2.Report{first}); err != nil {
t.Fatalf("write first report after counter restore failure: %v", err)
}
if fault.denied.Load() != 2 {
t.Fatalf("counter restore queries = %d, want 2", fault.denied.Load())
}
stateValue, ok := reportTrafficStates.Load(first.UUID)
if !ok {
t.Fatal("report traffic state was not persisted")
}
state := stateValue.(*reportTrafficState)
state.mu.Lock()
initialized := state.initialized
state.mu.Unlock()
if !initialized {
t.Fatal("report traffic state was not initialized after restore failure")
}
second := first
second.UpdatedAt = base.Add(time.Second)
second.Network.TotalUp = 150
second.Network.TotalDown = 260
if _, err := writeReportBatch(ctx, []v2.Report{second}); err != nil {
t.Fatalf("write second report: %v", err)
}
if fault.denied.Load() != 2 {
t.Fatalf("counter restore queries after second batch = %d, want no repeat", fault.denied.Load())
}
assertMetricValues(t, s, MetricTrafficUp, first.UUID, base.Add(-time.Second), base.Add(2*time.Second), []float64{0, 50})
assertMetricValues(t, s, MetricTrafficDown, first.UUID, base.Add(-time.Second), base.Add(2*time.Second), []float64{0, 60})
}
func TestReportBatchRetryDoesNotCountHistoricalTraffic(t *testing.T) {
for _, existingBaseline := range []bool{false, true} {
t.Run(fmt.Sprintf("existing-baseline-%v", existingBaseline), func(t *testing.T) {
s, fault := useReportCounterFailureStore(t)
fault.remaining.Store(0)
ctx := context.Background()
base := time.Now().UTC().Truncate(time.Minute).Add(-2 * time.Minute)
report := v2.Report{
UUID: "retry-node", UpdatedAt: base,
Network: v2.NetworkReport{TotalUp: 12 << 30, TotalDown: 30 << 30},
}
if existingBaseline {
if _, err := writeReportBatch(ctx, []v2.Report{report}); err != nil {
t.Fatal(err)
}
}
first := report
first.UpdatedAt = base.Add(3 * time.Second)
first.Network.TotalUp += 100
first.Network.TotalDown += 200
second := first
second.UpdatedAt = base.Add(6 * time.Second)
second.Network.TotalUp += 50
second.Network.TotalDown += 60
pending := []v2.Report{first, second}
fault.failWrite.Store(true)
for attempt := 0; attempt < 2; attempt++ {
if err := writePendingReports(ctx, &pending); err == nil {
t.Fatal("expected injected rollup write failure")
}
if len(pending) != 2 {
t.Fatal("failed reports were removed from pending batch")
}
value, _ := reportTrafficStates.Load(report.UUID)
state := value.(*reportTrafficState)
state.mu.Lock()
baseline := state.reportTrafficValues
state.mu.Unlock()
if baseline.hasUp != existingBaseline || baseline.hasDown != existingBaseline ||
(existingBaseline && (baseline.totalUp != report.Network.TotalUp ||
baseline.totalDown != report.Network.TotalDown || !baseline.timestamp.Equal(base))) {
t.Fatal("failed write advanced the committed counter baseline")
}
}
fault.failWrite.Store(false)
if err := writePendingReports(ctx, &pending); err != nil {
t.Fatalf("retry failed: %v", err)
}
if len(pending) != 0 {
t.Fatal("successful reports remained pending")
}
wantUp, wantDown := []float64{0, 50}, []float64{0, 60}
if existingBaseline {
wantUp, wantDown = []float64{0, 100, 50}, []float64{0, 200, 60}
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), wantUp)
assertMetricValues(t, s, MetricTrafficDown, report.UUID, base.Add(-time.Second), base.Add(time.Minute), wantDown)
var sumUp, sumDown float64
for i := range wantUp {
sumUp += wantUp[i]
sumDown += wantDown[i]
}
assertMetricAggregate(t, s, MetricTrafficUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), metric.AggSum, sumUp, len(wantUp))
assertMetricAggregate(t, s, MetricTrafficDown, report.UUID, base.Add(-time.Second), base.Add(time.Minute), metric.AggSum, sumDown, len(wantDown))
})
}
}
func TestWriteReportStoresMinuteMetricsAndResetAwareTraffic(t *testing.T) {
ctx := context.Background()
policy := defaultRollupPolicy()
@@ -51,20 +247,20 @@ func TestWriteReportStoresMinuteMetricsAndResetAwareTraffic(t *testing.T) {
base := time.Now().UTC().Truncate(time.Minute).Add(5 * time.Second)
now := base.Add(45 * time.Second)
report := v1.Report{
report := v2.Report{
UUID: "node-a",
UpdatedAt: base,
CPU: v1.CPUReport{Usage: 12.5},
Ram: v1.RamReport{Used: 100, Total: 1000},
Swap: v1.RamReport{Used: 20, Total: 200},
Load: v1.LoadReport{Load1: 0.5},
Disk: v1.DiskReport{Used: 300, Total: 3000},
Network: v1.NetworkReport{Up: 3, Down: 4, TotalUp: 100, TotalDown: 200},
CPU: v2.CPUReport{Usage: 12.5},
Ram: v2.RamReport{Used: 100, Total: 1000},
Swap: v2.RamReport{Used: 20, Total: 200},
Load: v2.LoadReport{Load1: 0.5},
Disk: v2.DiskReport{Used: 300, Total: 3000},
Network: v2.NetworkReport{Up: 3, Down: 4, TotalUp: 100, TotalDown: 200},
Process: 7,
Connections: v1.ConnectionsReport{TCP: 8, UDP: 9},
GPU: &v1.GPUDetailReport{
Connections: v2.ConnectionsReport{TCP: 8, UDP: 9},
GPU: &v2.GPUDetailReport{
AverageUsage: 25,
DetailedInfo: []v1.GPUDeviceInfo{{
DetailedInfo: []v2.GPUDeviceInfo{{
Name: "GPU 0", MemoryUsed: 400, MemoryTotal: 800, Utilization: 30, Temperature: 55,
}},
},
@@ -126,7 +322,7 @@ func TestWriteReportSkipsMetricsWithoutAgentData(t *testing.T) {
ctx := context.Background()
s := useReportTestStore(t, nil)
timestamp := time.Now().UTC()
if _, err := WriteReport(ctx, v1.Report{
if _, err := WriteReport(ctx, v2.Report{
UUID: "node-without-gpu", UpdatedAt: timestamp,
}); err != nil {
t.Fatalf("write report: %v", err)
@@ -154,11 +350,11 @@ func TestReportBatcherFlushesQueuedReports(t *testing.T) {
})
base := time.Now().UTC().Truncate(time.Minute).Add(10 * time.Second)
first := v1.Report{
first := v2.Report{
UUID: "batched-node",
UpdatedAt: base,
CPU: v1.CPUReport{Usage: 10},
Network: v1.NetworkReport{TotalUp: 100, TotalDown: 200},
CPU: v2.CPUReport{Usage: 10},
Network: v2.NetworkReport{TotalUp: 100, TotalDown: 200},
}
second := first
second.UpdatedAt = base.Add(3 * time.Second)
@@ -271,11 +467,11 @@ func TestReportBatchKeepsEverySample(t *testing.T) {
ctx := context.Background()
s := useReportTestStore(t, nil)
base := time.Now().UTC().Truncate(time.Second)
pending := []v1.Report{
{UUID: "node-a", UpdatedAt: base, CPU: v1.CPUReport{Usage: 10}, Network: v1.NetworkReport{TotalUp: 100}},
{UUID: "node-a", UpdatedAt: base, CPU: v1.CPUReport{Usage: 20}, Network: v1.NetworkReport{TotalUp: 150}},
{UUID: "node-b", UpdatedAt: base, CPU: v1.CPUReport{Usage: 30}, Network: v1.NetworkReport{TotalUp: 200}},
{UUID: "node-b", UpdatedAt: base.Add(time.Second), CPU: v1.CPUReport{Usage: 40}, Network: v1.NetworkReport{TotalUp: 260}},
pending := []v2.Report{
{UUID: "node-a", UpdatedAt: base, CPU: v2.CPUReport{Usage: 10}, Network: v2.NetworkReport{TotalUp: 100}},
{UUID: "node-a", UpdatedAt: base, CPU: v2.CPUReport{Usage: 20}, Network: v2.NetworkReport{TotalUp: 150}},
{UUID: "node-b", UpdatedAt: base, CPU: v2.CPUReport{Usage: 30}, Network: v2.NetworkReport{TotalUp: 200}},
{UUID: "node-b", UpdatedAt: base.Add(time.Second), CPU: v2.CPUReport{Usage: 40}, Network: v2.NetworkReport{TotalUp: 260}},
}
if err := writePendingReports(ctx, &pending); err != nil {
@@ -292,11 +488,11 @@ func TestReportQueueFullReturnsError(t *testing.T) {
ctx := context.Background()
useReportTestStore(t, nil)
worker := &reportBatchWorker{
queue: make(chan v1.Report, 1),
queue: make(chan v2.Report, 1),
requests: make(chan reportBatchRequest, 1),
done: make(chan struct{}),
}
worker.queue <- v1.Report{UUID: "already-queued"}
worker.queue <- v2.Report{UUID: "already-queued"}
reportBatcherMu.Lock()
reportBatcher = worker
reportBatcherMu.Unlock()
@@ -308,7 +504,7 @@ func TestReportQueueFullReturnsError(t *testing.T) {
reportBatcherMu.Unlock()
})
report := v1.Report{
report := v2.Report{
UUID: "realtime-node",
UpdatedAt: time.Now().UTC(),
}
@@ -348,6 +544,10 @@ func TestRecordReconstructionUsesMetricSpecificAggregation(t *testing.T) {
}
func TestTrafficCounterDelta(t *testing.T) {
const (
oneGB = int64(1_000_000_000)
oneTB = int64(1_000_000_000_000)
)
tests := []struct {
name string
current int64
@@ -360,6 +560,11 @@ func TestTrafficCounterDelta(t *testing.T) {
{name: "counter reset", current: 15, previous: 250, want: 15},
{name: "negative current", current: -1, previous: 100, want: 0},
{name: "negative previous", current: 15, previous: -1, want: 0},
{name: "tb-scale monotonic", current: 2*oneTB + 5*oneGB, previous: 2 * oneTB, want: 5 * oneGB},
{name: "32-bit wrap leftover", current: 800_000_000, previous: 3*oneGB + 900_000_000, want: 800_000_000},
{name: "reboot leftover below cap", current: 8 * oneGB, previous: 2 * oneTB, want: 8 * oneGB},
{name: "tb-scale tiny dip", current: 2*oneTB - 100, previous: 2 * oneTB, want: 0},
{name: "tb-scale false reset", current: 2*oneTB + 5*oneGB - 1000, previous: 2*oneTB + 5*oneGB, want: 0},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
@@ -370,16 +575,211 @@ func TestTrafficCounterDelta(t *testing.T) {
}
}
func TestWriteReportRebasesTrafficAfterAgentRestart(t *testing.T) {
ctx := context.Background()
s := useReportTestStore(t, nil)
base := time.Now().UTC().Truncate(time.Minute).Add(5 * time.Second)
report := v2.Report{
UUID: "restarted-node",
UpdatedAt: base,
Uptime: 1000,
Network: v2.NetworkReport{TotalUp: 100, TotalDown: 200},
}
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write first report: %v", err)
}
report.UpdatedAt = base.Add(3 * time.Second)
report.Uptime = 1003
report.Network.TotalUp = 150
report.Network.TotalDown = 260
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write continuous report: %v", err)
}
report.UpdatedAt = base.Add(6 * time.Second)
report.Uptime = 1
report.Network.TotalUp = 155
report.Network.TotalDown = 265
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write report after agent restart: %v", err)
}
report.UpdatedAt = base.Add(9 * time.Second)
report.Uptime = 4
report.Network.TotalUp = 180
report.Network.TotalDown = 300
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write report after new baseline: %v", err)
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 50, 5, 25})
assertMetricValues(t, s, MetricTrafficDown, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 60, 5, 35})
}
func TestWriteReportCountsTBScaleTrafficDeltas(t *testing.T) {
const (
oneGB = int64(1_000_000_000)
oneTB = int64(1_000_000_000_000)
)
ctx := context.Background()
s := useReportTestStore(t, nil)
base := time.Now().UTC().Truncate(time.Minute).Add(5 * time.Second)
report := v2.Report{
UUID: "tb-node",
UpdatedAt: base,
Uptime: 10_000,
Network: v2.NetworkReport{TotalUp: 2 * oneTB, TotalDown: 3 * oneTB},
}
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write first report: %v", err)
}
report.UpdatedAt = base.Add(3 * time.Second)
report.Uptime = 10_003
report.Network.TotalUp = 2*oneTB + 5*oneGB
report.Network.TotalDown = 3*oneTB + 7*oneGB
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write growing report: %v", err)
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, float64(5 * oneGB)})
assertMetricValues(t, s, MetricTrafficDown, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, float64(7 * oneGB)})
}
func TestWriteReportRestoresTBScaleCountersFromStore(t *testing.T) {
const (
oneGB = int64(1_000_000_000)
oneTB = int64(1_000_000_000_000)
)
ctx := context.Background()
policy := defaultRollupPolicy()
s := useReportTestStore(t, &policy)
base := time.Now().UTC().Truncate(time.Minute).Add(5 * time.Second)
now := base.Add(45 * time.Second)
report := v2.Report{
UUID: "tb-restore-node",
UpdatedAt: base,
Uptime: 10_000,
Network: v2.NetworkReport{TotalUp: 2*oneTB + 123, TotalDown: 4*oneTB + 456},
}
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write first report: %v", err)
}
restoredUp, hasUp, err := latestReportCounter(ctx, s, MetricNetTotalUp, report.UUID, now)
if err != nil {
t.Fatalf("restore upload counter: %v", err)
}
if !hasUp || restoredUp != report.Network.TotalUp {
t.Fatalf("restored upload counter = %d (has=%v), want %d", restoredUp, hasUp, report.Network.TotalUp)
}
restoredDown, hasDown, err := latestReportCounter(ctx, s, MetricNetTotalDown, report.UUID, now)
if err != nil {
t.Fatalf("restore download counter: %v", err)
}
if !hasDown || restoredDown != report.Network.TotalDown {
t.Fatalf("restored download counter = %d (has=%v), want %d", restoredDown, hasDown, report.Network.TotalDown)
}
if _, err := s.Compact(ctx, now); err != nil {
t.Fatalf("compact reports: %v", err)
}
deleteReportTrafficState(report.UUID)
report.UpdatedAt = now
report.Uptime = 10_045
report.Network.TotalUp = 2*oneTB + 123 + 5*oneGB
report.Network.TotalDown = 4*oneTB + 456 + 7*oneGB
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write after restoring tb-scale baseline: %v", err)
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, now.Add(-time.Second), now.Add(time.Second), []float64{float64(5 * oneGB)})
assertMetricValues(t, s, MetricTrafficDown, report.UUID, now.Add(-time.Second), now.Add(time.Second), []float64{float64(7 * oneGB)})
}
func TestWriteReportCountsTrafficAfterHighRateCounterWrap(t *testing.T) {
const oneGB = int64(1_000_000_000)
ctx := context.Background()
s := useReportTestStore(t, nil)
base := time.Now().UTC().Truncate(time.Minute).Add(5 * time.Second)
report := v2.Report{
UUID: "wrap-node",
UpdatedAt: base,
Uptime: 10_000,
Network: v2.NetworkReport{TotalUp: 3*oneGB + 900_000_000, TotalDown: 3*oneGB + 800_000_000},
}
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write first report: %v", err)
}
report.UpdatedAt = base.Add(3 * time.Second)
report.Uptime = 10_003
report.Network.TotalUp = 800_000_000
report.Network.TotalDown = 900_000_000
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write wrapped report: %v", err)
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 800_000_000})
assertMetricValues(t, s, MetricTrafficDown, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 900_000_000})
}
func TestWriteReportIgnoresTinyDipOfTBScaleCounter(t *testing.T) {
const (
oneGB = int64(1_000_000_000)
oneTB = int64(1_000_000_000_000)
)
ctx := context.Background()
s := useReportTestStore(t, nil)
base := time.Now().UTC().Truncate(time.Minute).Add(5 * time.Second)
report := v2.Report{
UUID: "tb-jitter-node",
UpdatedAt: base,
Uptime: 10_000,
Network: v2.NetworkReport{TotalUp: 2 * oneTB, TotalDown: 3 * oneTB},
}
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write first report: %v", err)
}
report.UpdatedAt = base.Add(3 * time.Second)
report.Uptime = 10_003
report.Network.TotalUp = 2*oneTB + 5*oneGB
report.Network.TotalDown = 3*oneTB + 7*oneGB
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write growing report: %v", err)
}
report.UpdatedAt = base.Add(6 * time.Second)
report.Uptime = 10_006
report.Network.TotalUp = 2*oneTB + 5*oneGB - 1000
report.Network.TotalDown = 3*oneTB + 7*oneGB - 2000
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write jitter report: %v", err)
}
report.UpdatedAt = base.Add(9 * time.Second)
report.Uptime = 10_009
report.Network.TotalUp = 2*oneTB + 6*oneGB
report.Network.TotalDown = 3*oneTB + 8*oneGB
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write recovered report: %v", err)
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, float64(5 * oneGB), 0, float64(oneGB + 1000)})
assertMetricValues(t, s, MetricTrafficDown, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, float64(7 * oneGB), 0, float64(oneGB + 2000)})
}
func TestWriteReportNormalizesReceiveTimeToUTC(t *testing.T) {
ctx := context.Background()
s := useReportTestStore(t, nil)
local := time.FixedZone("UTC+8", 8*60*60)
receiveTime := time.Now().In(local).Add(-10 * time.Second)
report := v1.Report{
report := v2.Report{
UUID: "utc-report",
UpdatedAt: receiveTime,
CPU: v1.CPUReport{Usage: 10},
Network: v1.NetworkReport{TotalUp: 1, TotalDown: 2},
CPU: v2.CPUReport{Usage: 10},
Network: v2.NetworkReport{TotalUp: 1, TotalDown: 2},
}
saved, err := WriteReport(ctx, report)
+29 -61
View File
@@ -37,99 +37,67 @@ func StructureUpgradeRequired(ctx context.Context) (bool, error) {
}
func structureUpgradeRequiredForConfig(ctx context.Context, cfg *MetricStoreConfig) (bool, error) {
s, err := openWithoutMigration(ctx, cfg)
if err != nil {
return false, err
}
defer s.Close()
return s.NeedsRestructure(ctx)
}
// openWithoutMigration opens cfg without creating or altering any table.
func openWithoutMigration(ctx context.Context, cfg *MetricStoreConfig) (*metric.Store, error) {
metricCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return false, err
return nil, err
}
store, err := metric.Open(ctx, metricCfg)
if err != nil {
return false, err
}
defer store.Close()
return store.NeedsRestructure(ctx)
return metric.Open(ctx, metricCfg)
}
// RestructureConfiguredStore performs the explicit data copy, table swap and
// one physical reclaim pass used by the upgrade guide.
func RestructureConfiguredStore(ctx context.Context, report func(RestructureProgress)) (RestructureResult, error) {
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
return RestructureResult{}, err
}
metricCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return RestructureResult{}, err
}
store, err := metric.Open(ctx, metricCfg)
if err != nil {
return RestructureResult{}, err
}
defer store.Close()
before, err := store.LegacyStorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure legacy metric storage: %w", err)
}
result, err := store.Restructure(ctx, func(progress metric.RestructureProgress) {
if report == nil {
return
}
report(RestructureProgress{Phase: progress.Phase, CurrentMetric: progress.Current, RowsDone: progress.RowsDone, RowsTotal: progress.RowsTotal, MetricsDone: progress.MetricsDone, MetricsTotal: progress.MetricsTotal})
})
if err != nil {
return RestructureResult{}, err
}
if report != nil {
report(RestructureProgress{Phase: "reclaiming", RowsDone: result.RowsCopied, RowsTotal: result.RowsCopied, MetricsDone: result.Metrics, MetricsTotal: result.Metrics})
}
if err := store.ReclaimSpace(ctx); err != nil {
return RestructureResult{}, fmt.Errorf("reclaim metric storage: %w", err)
}
after, err := store.StorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure rebuilt metric storage: %w", err)
}
return RestructureResult{BeforeBytes: before, AfterBytes: after, RowsCopied: result.RowsCopied, Metrics: result.Metrics}, nil
return rebuildConfiguredStore(ctx, report, (*metric.Store).Restructure)
}
// DiscardConfiguredStoreHistory completes the same schema upgrade as
// RestructureConfiguredStore but drops every historical sample instead of
// copying it. Metric definitions remain available for new ingest.
func DiscardConfiguredStoreHistory(ctx context.Context, report func(RestructureProgress)) (RestructureResult, error) {
return rebuildConfiguredStore(ctx, report, (*metric.Store).DiscardHistory)
}
type storeRebuild func(*metric.Store, context.Context, func(metric.RestructureProgress)) (metric.RestructureResult, error)
func rebuildConfiguredStore(ctx context.Context, report func(RestructureProgress), rebuild storeRebuild) (RestructureResult, error) {
if report == nil {
report = func(RestructureProgress) {}
}
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
return RestructureResult{}, err
}
metricCfg, err := buildMetricConfig(cfg, false)
s, err := openWithoutMigration(ctx, cfg)
if err != nil {
return RestructureResult{}, err
}
store, err := metric.Open(ctx, metricCfg)
if err != nil {
return RestructureResult{}, err
}
defer store.Close()
defer s.Close()
before, err := store.LegacyStorageSize(ctx)
before, err := s.LegacyStorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure legacy metric storage: %w", err)
}
result, err := store.DiscardHistory(ctx, func(progress metric.RestructureProgress) {
if report == nil {
return
}
result, err := rebuild(s, ctx, func(progress metric.RestructureProgress) {
report(RestructureProgress{Phase: progress.Phase, CurrentMetric: progress.Current, RowsDone: progress.RowsDone, RowsTotal: progress.RowsTotal, MetricsDone: progress.MetricsDone, MetricsTotal: progress.MetricsTotal})
})
if err != nil {
return RestructureResult{}, err
}
if report != nil {
report(RestructureProgress{Phase: "reclaiming", RowsDone: result.RowsCopied, RowsTotal: result.RowsCopied, MetricsDone: result.Metrics, MetricsTotal: result.Metrics})
}
if err := store.ReclaimSpace(ctx); err != nil {
report(RestructureProgress{Phase: "reclaiming", RowsDone: result.RowsCopied, RowsTotal: result.RowsCopied, MetricsDone: result.Metrics, MetricsTotal: result.Metrics})
if err := s.ReclaimSpace(ctx); err != nil {
return RestructureResult{}, fmt.Errorf("reclaim metric storage: %w", err)
}
after, err := store.StorageSize(ctx)
after, err := s.StorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure rebuilt metric storage: %w", err)
}
+36 -69
View File
@@ -53,13 +53,6 @@ func openStoreWithDefaultRetention(ctx context.Context, cfg *MetricStoreConfig,
return s, nil
}
// OpenStore opens an isolated metric store using the supplied configuration.
// It is used by the pre-start upgrade flow before the process-wide store is
// initialized. The caller owns the returned store and must close it.
// func OpenStore(ctx context.Context, cfg *MetricStoreConfig) (*metric.Store, error) {
// return openStore(ctx, cfg)
// }
// OpenStoreForMigration opens an isolated target and uses the legacy data span
// as the initial retention for definitions that do not exist yet. Existing
// definitions keep their configured retention, including an explicit zero.
@@ -73,17 +66,11 @@ func OpenStoreForMigration(ctx context.Context, cfg *MetricStoreConfig, legacyRe
// TestConnection 使用给定配置尝试连接 metrics 数据库(不影响当前运行的 store)。
// 仅打开连接并 Ping,不执行自动建表,连接成功后立即关闭。失败时返回可读错误。
func TestConnection(ctx context.Context, cfg *MetricStoreConfig) error {
metricCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return err
}
s, err := metric.Open(ctx, metricCfg)
s, err := openWithoutMigration(ctx, cfg)
if err != nil {
return err
}
defer s.Close()
return s.Ping(ctx)
}
@@ -116,10 +103,7 @@ func InitializeStore() error {
return err
}
storeMu.Lock()
store = s
storeFingerprint = targetFingerprint(cfg)
storeMu.Unlock()
activateStore(s, targetFingerprint(cfg), "initialization")
clearStoreClosing()
logger.Infof("metricstore", "Metric store initialized successfully (driver=%s)", ResolveDriverFromConfig(cfg.Driver, cfg.DSN))
@@ -130,10 +114,6 @@ func InitializeStore() error {
// from the restricted recovery page. It records that target as the current
// manual-migration source, but does not copy data from the unavailable store.
func RecoverStore(ctx context.Context, cfg *MetricStoreConfig) error {
if cfg == nil {
return fmt.Errorf("metric store recovery config is nil")
}
storeInitMu.Lock()
defer storeInitMu.Unlock()
if err := storeOperations.Acquire(ctx); err != nil {
@@ -147,18 +127,14 @@ func RecoverStore(ctx context.Context, cfg *MetricStoreConfig) error {
recovered := *cfg
recovered.DSN = strings.TrimSpace(recovered.DSN)
recovered.Driver = string(ResolveDriverFromConfig(recovered.Driver, recovered.DSN))
target := targetFingerprint(&recovered)
restructureRequired, err := structureUpgradeRequiredForConfig(ctx, &recovered)
if err != nil {
return err
}
if restructureRequired {
target := targetFingerprint(&recovered)
if err := config.SetMany(map[string]any{
MetricDBDriverKey: recovered.Driver,
MetricDBDSNKey: recovered.DSN,
MigrationTargetKey: target,
}); err != nil {
return fmt.Errorf("save recovered metric store config: %w", err)
if err := saveRecoveredTarget(&recovered, target); err != nil {
return err
}
logger.Infof("metricstore", "Metric store connection recovered; structure upgrade is required (driver=%s)", recovered.Driver)
return nil
@@ -167,31 +143,43 @@ func RecoverStore(ctx context.Context, cfg *MetricStoreConfig) error {
if err != nil {
return err
}
if err := saveRecoveredTarget(&recovered, target); err != nil {
_ = s.Close()
return err
}
activateStore(s, target, "recovery")
clearStoreClosing()
logger.Infof("metricstore", "Metric store recovered successfully (driver=%s)", recovered.Driver)
return nil
}
target := targetFingerprint(&recovered)
// saveRecoveredTarget persists the recovered connection and records it as the
// default source for the next manual store migration.
func saveRecoveredTarget(cfg *MetricStoreConfig, target string) error {
if err := config.SetMany(map[string]any{
MetricDBDriverKey: recovered.Driver,
MetricDBDSNKey: recovered.DSN,
MetricDBDriverKey: cfg.Driver,
MetricDBDSNKey: cfg.DSN,
MigrationTargetKey: target,
}); err != nil {
_ = s.Close()
return fmt.Errorf("save recovered metric store config: %w", err)
}
return nil
}
// activateStore publishes s as the active store, then closes the store it
// replaced. Callers must hold the exclusive operation gate or storeInitMu.
func activateStore(s *metric.Store, fingerprint, reason string) {
storeMu.Lock()
old := store
store = s
storeFingerprint = target
storeFingerprint = fingerprint
storeMu.Unlock()
clearStoreClosing()
if old != nil {
if closeErr := old.Close(); closeErr != nil {
logger.Errorf("metricstore", "Failed to close previous metric store during recovery: %v", closeErr)
}
if old == nil {
return
}
if err := old.Close(); err != nil {
logger.Errorf("metricstore", "Failed to close previous metric store during %s: %v", reason, err)
}
logger.Infof("metricstore", "Metric store recovered successfully (driver=%s)", recovered.Driver)
return nil
}
// Reload 根据最新配置热重载 metric store,无需重启进程。
@@ -218,16 +206,7 @@ func Reload(ctx context.Context) error {
// point-backed schema. That schema must go through the authenticated
// structure-upgrade guide first; otherwise index creation can reference
// columns such as resolution_id that do not exist yet.
checkCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return err
}
checkStore, err := metric.Open(ctx, checkCfg)
if err != nil {
return err
}
needsRestructure, err := checkStore.NeedsRestructure(ctx)
_ = checkStore.Close()
needsRestructure, err := structureUpgradeRequiredForConfig(ctx, cfg)
if err != nil {
return fmt.Errorf("inspect metric store structure: %w", err)
}
@@ -241,17 +220,7 @@ func Reload(ctx context.Context) error {
return err
}
storeMu.Lock()
old := store
store = s
storeFingerprint = targetFingerprint(cfg)
storeMu.Unlock()
if old != nil {
if cerr := old.Close(); cerr != nil {
logger.Errorf("metricstore", "Failed to close previous metric store on reload: %v", cerr)
}
}
activateStore(s, targetFingerprint(cfg), "reload")
logger.Infof("metricstore", "Metric store reloaded successfully (driver=%s)", ResolveDriverFromConfig(cfg.Driver, cfg.DSN))
return nil
}
@@ -266,7 +235,7 @@ func GetStore() *metric.Store {
// CloseStoreContext stops the asynchronous store migration before taking the
// store write lock, so shutdown cannot wait forever on the migration's lease.
func CloseStoreContext(ctx context.Context) error {
if err := stopStoreMigrationForClose(ctx); err != nil {
if err := StopStoreMigrationForShutdown(ctx); err != nil {
clearStoreClosing()
return err
}
@@ -278,13 +247,11 @@ func CloseStoreContext(ctx context.Context) error {
storeMu.Lock()
defer storeMu.Unlock()
var err error
if store != nil {
err := store.Close()
err = store.Close()
store = nil
storeFingerprint = ""
return err
}
storeFingerprint = ""
return nil
return err
}
+22 -40
View File
@@ -224,54 +224,36 @@ func finishStoreMigration(status string, err error) {
func CancelStoreMigration() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return stopStoreMigration(ctx, true)
}
func stopStoreMigration(ctx context.Context, requireRunning bool) error {
storeMigMu.Lock()
cancel := storeMigCancel
done := storeMigDone
stop, done := storeMigCancel, storeMigDone
storeMigMu.Unlock()
if cancel == nil {
if requireRunning {
return fmt.Errorf("no store migration is currently running")
}
return nil
}
cancel()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
}
}
func stopStoreMigrationForClose(ctx context.Context) error {
storeMigMu.Lock()
storeClosing = true
cancel := storeMigCancel
done := storeMigDone
storeMigMu.Unlock()
if cancel == nil {
return nil
}
cancel()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
if stop == nil {
return fmt.Errorf("no store migration is currently running")
}
return awaitStoreMigrationStop(ctx, stop, done)
}
// StopStoreMigrationForShutdown prevents a new store migration and waits for
// an active one to release its exclusive lease before report batches flush.
func StopStoreMigrationForShutdown(ctx context.Context) error {
return stopStoreMigrationForClose(ctx)
storeMigMu.Lock()
storeClosing = true
stop, done := storeMigCancel, storeMigDone
storeMigMu.Unlock()
if stop == nil {
return nil
}
return awaitStoreMigrationStop(ctx, stop, done)
}
func awaitStoreMigrationStop(ctx context.Context, stop context.CancelFunc, done <-chan struct{}) error {
stop()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
}
}
func clearStoreClosing() {
-11
View File
@@ -112,9 +112,6 @@ func Run(ctx Context) error {
if err := migrateLegacyClientInfo(db); err != nil {
return err
}
if err := migrateLegacyLoadNotification(db); err != nil {
return err
}
if err := migrateLegacyPingAllClientsExpansion(db); err != nil {
return err
}
@@ -175,14 +172,6 @@ func hasTableColumn(db *gorm.DB, tableName, columnName string) bool {
return false
}
func migrateLegacyLoadNotification(db *gorm.DB) error {
if db.Migrator().HasColumn(&models.LoadNotification{}, "client") {
logger.InfoArgs("migration", "[>0.1.4] Rebuilding LoadNotification table....")
return db.Migrator().DropTable(&models.LoadNotification{})
}
return nil
}
func migrateLegacyOidcConfig(db *gorm.DB) error {
if db.Migrator().HasTable(&models.OidcProvider{}) {
return nil
+2 -1
View File
@@ -2,6 +2,7 @@ package migrations
import (
"encoding/json"
"fmt"
"strings"
"testing"
"time"
@@ -15,7 +16,7 @@ import (
func openTestDB(t *testing.T, name string) *gorm.DB {
t.Helper()
dsn := "file:" + strings.ReplaceAll(name, " ", "_") + "?mode=memory&cache=shared"
dsn := fmt.Sprintf("file:%s-%d?mode=memory&cache=shared", strings.ReplaceAll(name, " ", "_"), time.Now().UnixNano())
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatalf("open sqlite test db: %v", err)
-3
View File
@@ -35,10 +35,7 @@ var legacyTimestampColumns = [...]timestampColumn{
{table: "sessions", column: "expires"},
{table: "sessions", column: "created_at"},
{table: "logs", column: "time"},
{table: "clipboards", column: "created_at"},
{table: "clipboards", column: "updated_at"},
{table: "offline_notifications", column: "last_notified"},
{table: "load_notifications", column: "last_notified"},
{table: "task_results", column: "finished_at"},
{table: "task_results", column: "created_at"},
{table: "records", column: "time"},
+1 -1
View File
@@ -61,7 +61,7 @@ func TestParseLegacyTimestampAcceptsHistoricalEpochUnits(t *testing.T) {
func openTimestampMigrationDB(t *testing.T) *gorm.DB {
t.Helper()
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
dsn := fmt.Sprintf("file:%s-%d?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"), time.Now().UnixNano())
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatalf("open database: %v", err)
+64
View File
@@ -0,0 +1,64 @@
// Package notifications wires the built-in notification channels into the
// shared messageSender registry.
package notifications
import (
"encoding/json"
"errors"
"github.com/komari-monitor/komari/database"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/managedconfig"
"github.com/komari-monitor/komari/utils/messageSender"
"github.com/komari-monitor/komari/utils/messageSender/javascript"
"github.com/komari-monitor/komari/utils/messageSender/webhook"
)
// Initialize registers the built-in channels and seeds their defaults.
func Initialize() (err error) {
if err := webhook.Register(); err != nil {
return err
}
registered := []string{"webhook"}
defer func() {
if err != nil {
for _, id := range registered {
err = errors.Join(err, messageSender.UnregisterNotificationChannel(id))
}
}
}()
if err := javascript.Register(); err != nil {
return err
}
registered = append(registered, javascript.ChannelID)
for _, item := range messageSender.ListNotificationChannels() {
if _, err := database.GetMessageSenderConfigByName(item.ID); err == nil {
continue
}
values := map[string]any{}
for _, field := range managedconfig.Items(item.Configuration) {
if field.Key != "" {
values[field.Key] = managedconfig.DefaultValue(field)
}
}
raw, err := json.Marshal(values)
if err != nil {
return err
}
if err := database.SaveMessageSenderConfig(&models.MessageSenderProvider{
Name: item.ID,
Addition: string(raw),
}); err != nil {
return err
}
}
return nil
}
// Shutdown unregisters the built-in channels and releases their runtimes.
func Shutdown() error {
return errors.Join(
messageSender.UnregisterNotificationChannel(javascript.ChannelID),
messageSender.UnregisterNotificationChannel("webhook"),
)
}
+8 -43
View File
@@ -5,10 +5,10 @@ import (
"errors"
"fmt"
"path/filepath"
"strings"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/managedconfig"
"gorm.io/gorm"
)
@@ -26,7 +26,11 @@ func GetConfiguration(short string) (map[string]any, error) {
if err != nil {
return values, nil // not installed (or unreadable): keep saved values
}
mergeConfigurationDefaults(values, info.Configuration)
items := managedconfig.Items(info.Configuration)
mergeConfigurationDefaults(values, items)
if err := managedconfig.ResolveForOutput(values, items); err != nil {
return nil, fmt.Errorf("resolve plugin configuration: %w", err)
}
return values, nil
}
@@ -48,18 +52,7 @@ func savedConfiguration(short string) (map[string]any, error) {
// mergeConfigurationDefaults fills manifest-declared defaults for keys that
// are missing from the saved values, following the theme merge rules.
func mergeConfigurationDefaults(values map[string]any, configuration models.Configuration) {
if configuration.Type != models.ThemeConfigurationManaged {
return
}
raw, err := json.Marshal(configuration.Data)
if err != nil {
return
}
var items []models.ManagedThemeConfigurationItem
if err := json.Unmarshal(raw, &items); err != nil {
return
}
func mergeConfigurationDefaults(values map[string]any, items []models.ManagedThemeConfigurationItem) {
for _, item := range items {
if item.Key == "" {
continue
@@ -67,38 +60,10 @@ func mergeConfigurationDefaults(values map[string]any, configuration models.Conf
if _, exists := values[item.Key]; exists {
continue
}
values[item.Key] = configurationDefault(item)
values[item.Key] = managedconfig.DefaultValue(item)
}
}
// configurationDefault resolves one item's effective default, mirroring the
// theme rules: select falls back to its first option; nil defaults become
// 0 for number, false for switch and "" otherwise.
func configurationDefault(item models.ManagedThemeConfigurationItem) any {
def := item.Default
if item.Type == "select" {
if def == nil || def == "" {
if item.Options != "" {
opts := strings.Split(item.Options, ",")
if len(opts) > 0 {
return strings.TrimSpace(opts[0])
}
}
}
}
if def == nil {
switch item.Type {
case "number":
return float64(0) // JSON numbers decode as float64
case "switch":
return false
default:
return ""
}
}
return def
}
// SaveConfiguration upserts the configuration values of one plugin.
func SaveConfiguration(short string, data map[string]any) error {
if data == nil {
+21 -19
View File
@@ -5,8 +5,10 @@ import (
"net/http"
"strings"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/rpc"
)
@@ -17,41 +19,41 @@ const principalContextKey = "principal"
// routeRequestContext builds the JS request context with the caller identity
// resolved by IdentityMiddleware. Raw credentials (session/client tokens) are
// intentionally not exposed.
func routeRequestContext(vm *goja.Runtime, c *gin.Context) *goja.Object {
ctx := vm.NewObject()
func routeRequestContext(vm *moejs.Runtime, c *gin.Context) *engine.Object {
ctx := vm.Realm().NewObject()
principal := rpcPrincipalFromGin(c)
p := vm.NewObject()
p := vm.Realm().NewObject()
if principal != nil {
_ = p.Set("type", principalTypeString(principal.Type))
_ = p.Set("roles", principal.Roles)
_ = p.Set("user_uuid", principal.UserUUID)
_ = p.Set("client_uuid", principal.ClientUUID)
_ = p.Set("is_api_key", principal.IsAPIKey)
_ = native.Set(vm, p, "type", principalTypeString(principal.Type))
_ = native.Set(vm, p, "roles", principal.Roles)
_ = native.Set(vm, p, "user_uuid", principal.UserUUID)
_ = native.Set(vm, p, "client_uuid", principal.ClientUUID)
_ = native.Set(vm, p, "is_api_key", principal.IsAPIKey)
}
_ = ctx.Set("principal", p)
_ = native.Set(vm, ctx, "principal", p)
if role, ok := c.Get("role"); ok {
if s, ok := role.(string); ok {
_ = ctx.Set("role", s)
_ = native.Set(vm, ctx, "role", s)
}
}
if uuid, ok := c.Get("uuid"); ok {
_ = ctx.Set("user_uuid", uuid)
_ = native.Set(vm, ctx, "user_uuid", uuid)
}
if clientUUID, ok := c.Get("client_uuid"); ok {
_ = ctx.Set("client_uuid", clientUUID)
_ = native.Set(vm, ctx, "client_uuid", clientUUID)
}
_ = ctx.Set("remote_ip", c.ClientIP())
_ = ctx.Set("user_agent", c.GetHeader("User-Agent"))
_ = native.Set(vm, ctx, "remote_ip", c.ClientIP())
_ = native.Set(vm, ctx, "user_agent", c.GetHeader("User-Agent"))
return ctx
}
// hookRequestContext builds the JS request context at the raw HTTP layer
// (before the gin identity middleware runs): only network metadata is
// available there.
func hookRequestContext(vm *goja.Runtime, r *http.Request) *goja.Object {
ctx := vm.NewObject()
_ = ctx.Set("remote_ip", remoteIP(r))
_ = ctx.Set("user_agent", r.Header.Get("User-Agent"))
func hookRequestContext(vm *moejs.Runtime, r *http.Request) *engine.Object {
ctx := vm.Realm().NewObject()
_ = native.Set(vm, ctx, "remote_ip", remoteIP(r))
_ = native.Set(vm, ctx, "user_agent", r.Header.Get("User-Agent"))
return ctx
}
+8 -7
View File
@@ -5,8 +5,9 @@ import (
"fmt"
"sync/atomic"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/internal/scheduler"
"github.com/komari-monitor/komari/pkg/jsruntime"
)
// cronJobSeq gives every registered cron job a process-unique id. Job names
@@ -20,7 +21,7 @@ var cronJobSeq atomic.Uint64
// evaluation and takes the manager lock itself; the critical section is short
// and never waits on the event loop, so no lock cycle is possible. The job is
// removed automatically when the plugin is unloaded or fails to load.
func (m *Manager) registerCron(short, spec string, fn goja.Callable) error {
func (m *Manager) registerCron(short, spec string, fn moejs.Value) error {
m.mu.Lock()
defer m.mu.Unlock()
inst, ok := m.instances[short]
@@ -37,7 +38,7 @@ func (m *Manager) registerCron(short, spec string, fn goja.Callable) error {
name := fmt.Sprintf("plugin:%s:%d", short, cronJobSeq.Add(1))
if err := scheduler.AddContextFunc(name, spec, false, func(ctx context.Context) {
m.runCronJob(short, name, fn)
m.runCronJob(short, name, fn, host)
}); err != nil {
return err
}
@@ -51,7 +52,7 @@ func (m *Manager) registerCron(short, spec string, fn goja.Callable) error {
// on a scheduler goroutine and drops the fire when the plugin was unloaded
// (the job is cancelled with the instance, but a fire may already be queued).
// Errors thrown by the handler are reported to the plugin log by RunJob.
func (m *Manager) runCronJob(short, name string, fn goja.Callable) {
func (m *Manager) runCronJob(short, name string, fn moejs.Value, owner *jsruntime.Host) {
inst := m.instanceFor(short)
if inst == nil {
return
@@ -60,12 +61,12 @@ func (m *Manager) runCronJob(short, name string, fn goja.Callable) {
host := inst.host
alive := inst.runtime != nil
inst.mu.RUnlock()
if !alive || host == nil {
if !alive || host == nil || host != owner {
return
}
host.RunOnLoop(func(vm *goja.Runtime) {
host.RunOnLoop(func(vm *moejs.Runtime) {
_ = host.RunJob(vm, "plugin cron "+name, func() error {
_, err := fn(goja.Undefined())
_, err := vm.CallFunction(fn, moejs.Undefined())
return err
})
})
+1
View File
@@ -398,6 +398,7 @@ func TestGetConfigurationMergesManifestDefaults(t *testing.T) {
t.Errorf("mode still default = %v", values["mode"])
}
}
func TestGetConfigFromScript(t *testing.T) {
withTempDataDir(t)
gin.SetMode(gin.TestMode)
+50 -39
View File
@@ -14,9 +14,11 @@ import (
"sync"
"time"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
// hookKind identifies the HTTP hook phase.
@@ -31,7 +33,7 @@ const (
type hookEntry struct {
short string
kind hookKind
fn goja.Callable
fn moejs.Value
host *jsruntime.Host
matcher *hookMatcher // nil matches every request
bodyLimit int64 // plugin-declared maxHTTPBodyBytes (0 = default)
@@ -211,36 +213,36 @@ func (m *Manager) wrapHandler(next http.Handler) http.Handler {
func (m *Manager) runRequestHook(r *http.Request, body []byte, h *hookEntry) ([]byte, error) {
nextBody := body
var hookErr error
queued, timedOut := runHookTurn(h.host, "plugin request hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin request hook "+h.short, func(vm *moejs.Runtime) {
reqObj := hookRequestObject(vm, r, nextBody)
hookErr = h.host.RunJob(vm, "plugin request hook "+h.short, func() error {
_, err := h.fn(goja.Undefined(), reqObj)
_, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(reqObj))
return err
})
if hookErr != nil {
return
}
if value := reqObj.Get("method"); value != nil && !goja.IsUndefined(value) {
if method := strings.ToUpper(strings.TrimSpace(value.String())); method != "" {
if value := native.Get(vm, reqObj, "method"); !value.IsUndefined() {
if method := strings.ToUpper(strings.TrimSpace(native.String(vm, value))); method != "" {
r.Method = method
}
}
if value := reqObj.Get("url"); value != nil && !goja.IsUndefined(value) {
if raw := strings.TrimSpace(value.String()); raw != "" {
if value := native.Get(vm, reqObj, "url"); !value.IsUndefined() {
if raw := strings.TrimSpace(native.String(vm, value)); raw != "" {
if parsed, err := url.Parse(raw); err == nil {
r.URL = parsed
r.RequestURI = raw
}
}
}
if value := reqObj.Get("headers"); value != nil && !goja.IsUndefined(value) {
if value := native.Get(vm, reqObj, "headers"); !value.IsUndefined() {
var raw map[string]any
if err := vm.ExportTo(value, &raw); err == nil {
if err := native.ExportTo(vm, value, &raw); err == nil {
r.Header = normalizeHeader(raw)
}
}
if value := reqObj.Get("body"); value != nil && !goja.IsUndefined(value) {
nextBody = []byte(value.String())
if value := native.Get(vm, reqObj, "body"); !value.IsUndefined() {
nextBody = []byte(native.String(vm, value))
}
})
if !queued {
@@ -256,29 +258,29 @@ func (m *Manager) runRequestHook(r *http.Request, body []byte, h *hookEntry) ([]
// the JS-side modifications to the captured response.
func (m *Manager) runResponseHook(r *http.Request, bw *bufferedResponseWriter, h *hookEntry) error {
var hookErr error
queued, timedOut := runHookTurn(h.host, "plugin response hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin response hook "+h.short, func(vm *moejs.Runtime) {
resObj := hookResponseObject(vm, bw)
hookErr = h.host.RunJob(vm, "plugin response hook "+h.short, func() error {
_, err := h.fn(goja.Undefined(), hookRequestObject(vm, r, nil), resObj)
_, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(hookRequestObject(vm, r, nil)), engine.ObjectValue(resObj))
return err
})
if hookErr != nil {
return
}
status := bw.status()
if value := resObj.Get("statusCode"); value != nil && !goja.IsUndefined(value) {
status = int(value.ToInteger())
if value := native.Get(vm, resObj, "statusCode"); !value.IsUndefined() {
status = int(native.Int(vm, value))
}
header := bw.Header().Clone()
if value := resObj.Get("headers"); value != nil && !goja.IsUndefined(value) {
if value := native.Get(vm, resObj, "headers"); !value.IsUndefined() {
var raw map[string]any
if err := vm.ExportTo(value, &raw); err == nil {
if err := native.ExportTo(vm, value, &raw); err == nil {
header = normalizeHeader(raw)
}
}
responseBody := bw.bodyBytes()
if value := resObj.Get("body"); value != nil && !goja.IsUndefined(value) {
responseBody = []byte(value.String())
if value := native.Get(vm, resObj, "body"); !value.IsUndefined() {
responseBody = []byte(native.String(vm, value))
}
bw.apply(status, header, responseBody)
})
@@ -298,16 +300,16 @@ func (m *Manager) runResponseHook(r *http.Request, bw *bufferedResponseWriter, h
// runHookTurn queues one hook callback on the plugin event loop and waits
// for it with the runtime timeout. It reports whether the job was queued and
// whether it timed out.
func runHookTurn(host *jsruntime.Host, name string, job func(vm *goja.Runtime)) (queued, timedOut bool) {
func runHookTurn(host *jsruntime.Host, name string, job func(vm *moejs.Runtime)) (queued, timedOut bool) {
return runHookTurnTimeout(host, name, host.Timeout(), job)
}
// runHookTurnTimeout is runHookTurn with an explicit wait budget. The
// callback keeps running on the plugin loop after a timeout; the waiter gives
// up and the caller falls back to passing the message through.
func runHookTurnTimeout(host *jsruntime.Host, name string, timeout time.Duration, job func(vm *goja.Runtime)) (queued, timedOut bool) {
func runHookTurnTimeout(host *jsruntime.Host, name string, timeout time.Duration, job func(vm *moejs.Runtime)) (queued, timedOut bool) {
done := make(chan struct{})
if !host.RunOnLoop(func(vm *goja.Runtime) {
if !host.RunOnLoop(func(vm *moejs.Runtime) {
defer close(done)
job(vm)
}) {
@@ -326,29 +328,29 @@ func (m *Manager) logHookError(h *hookEntry, err error) {
}
// hookRequestObject builds the mutable JS request object.
func hookRequestObject(vm *goja.Runtime, r *http.Request, body []byte) *goja.Object {
req := vm.NewObject()
_ = req.Set("method", r.Method)
_ = req.Set("url", r.URL.RequestURI())
_ = req.Set("headers", headerToMap(r.Header))
func hookRequestObject(vm *moejs.Runtime, r *http.Request, body []byte) *engine.Object {
req := vm.Realm().NewObject()
_ = native.Set(vm, req, "method", r.Method)
_ = native.Set(vm, req, "url", r.URL.RequestURI())
_ = native.Set(vm, req, "headers", headerToMap(r.Header))
query := make(map[string]string, len(r.URL.Query()))
for name, values := range r.URL.Query() {
query[name] = strings.Join(values, ",")
}
_ = req.Set("query", query)
_ = req.Set("body", string(body))
_ = req.Set("context", hookRequestContext(vm, r))
_ = native.Set(vm, req, "query", query)
_ = native.Set(vm, req, "body", string(body))
_ = native.Set(vm, req, "context", hookRequestContext(vm, r))
return req
}
// hookResponseObject builds the mutable JS response object from the captured
// response.
func hookResponseObject(vm *goja.Runtime, bw *bufferedResponseWriter) *goja.Object {
res := vm.NewObject()
_ = res.Set("statusCode", bw.status())
_ = res.Set("statusMessage", "")
_ = res.Set("headers", headerToMap(bw.Header()))
_ = res.Set("body", string(bw.bodyBytes()))
func hookResponseObject(vm *moejs.Runtime, bw *bufferedResponseWriter) *engine.Object {
res := vm.Realm().NewObject()
_ = native.Set(vm, res, "statusCode", bw.status())
_ = native.Set(vm, res, "statusMessage", "")
_ = native.Set(vm, res, "headers", headerToMap(bw.Header()))
_ = native.Set(vm, res, "body", string(bw.bodyBytes()))
return res
}
@@ -392,9 +394,18 @@ func headerToMap(header http.Header) map[string]any {
return out
}
// isUpgradeRequest reports whether r is a genuine protocol-upgrade request:
// both Connection: upgrade and an Upgrade header must be present (RFC 9110 §7.8).
func isUpgradeRequest(r *http.Request) bool {
return strings.EqualFold(strings.TrimSpace(r.Header.Get("Connection")), "upgrade") ||
strings.EqualFold(r.Header.Get("Upgrade"), "websocket")
if r.Header.Get("Upgrade") == "" {
return false
}
for _, token := range strings.Split(r.Header.Get("Connection"), ",") {
if strings.EqualFold(strings.TrimSpace(token), "upgrade") {
return true
}
}
return false
}
// bufferedResponseWriter captures status/headers/body so response hooks can
+19 -2
View File
@@ -8,6 +8,7 @@ import (
"github.com/komari-monitor/komari/cmd/flags"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
func TestMain(m *testing.M) {
@@ -20,8 +21,9 @@ func TestMain(m *testing.M) {
os.Exit(m.Run())
}
// withTempDataDir redirects DataDir and StorageDir and resets the global
// manager for the duration of a test.
// withTempDataDir redirects DataDir and StorageDir, resets the global
// manager, and clears the plugin configuration rows shared through the
// process-wide test database, so each test starts from a clean slate.
func withTempDataDir(t *testing.T) {
t.Helper()
oldDir := DataDir
@@ -29,6 +31,11 @@ func withTempDataDir(t *testing.T) {
oldGlobal := global
DataDir = t.TempDir()
StorageDir = t.TempDir()
// SaveConfiguration writes to the shared in-memory database created by
// TestMain, so leftover rows would leak into later tests (and repeated
// runs). Clear them before and after each test.
clearPluginConfigurations(t)
t.Cleanup(func() { clearPluginConfigurations(t) })
mgr := &Manager{
instances: make(map[string]*Instance),
routes: make(map[string]map[string]bool),
@@ -43,6 +50,16 @@ func withTempDataDir(t *testing.T) {
})
}
// clearPluginConfigurations removes every stored plugin configuration row.
func clearPluginConfigurations(t *testing.T) {
t.Helper()
if err := dbcore.GetDBInstance().
Where("1 = 1").
Delete(&models.PluginConfiguration{}).Error; err != nil {
t.Fatalf("clear plugin configurations: %v", err)
}
}
// writePluginZip creates a plugin ZIP from a name -> content map.
func writePluginZip(t *testing.T, files map[string]string) string {
t.Helper()
+2 -22
View File
@@ -41,7 +41,7 @@ func validateManifest(info *models.Plugin) error {
if !validShort(info.Short) {
return fmt.Errorf("plugin short %q is invalid: only letters, digits, '_' and '-' are allowed", info.Short)
}
if !localizedText(info.Name) {
if !models.IsLocalizedText(info.Name) {
return fmt.Errorf("plugin name is required")
}
if info.Entry == "" {
@@ -90,7 +90,7 @@ func validatePage(page *models.PluginPage) error {
default:
return fmt.Errorf("plugin page type %q is invalid: use \"iframe\" or \"redirect\"", page.Type)
}
if !localizedText(page.Title) {
if !models.IsLocalizedText(page.Title) {
return fmt.Errorf("plugin page %q requires a title", page.Title)
}
return nil
@@ -113,23 +113,3 @@ func isSafeInternalPath(target string) bool {
}
return true
}
// localizedText reports whether v is a usable name/description/author value:
// a non-empty string, or an i18n object with at least one non-empty value.
func localizedText(v any) bool {
switch t := v.(type) {
case nil:
return false
case string:
return strings.TrimSpace(t) != ""
case map[string]any:
for _, value := range t {
if s, ok := value.(string); ok && strings.TrimSpace(s) != "" {
return true
}
}
return false
default:
return false
}
}
+163
View File
@@ -0,0 +1,163 @@
package plugin
import (
"context"
"encoding/json"
"fmt"
"sync"
"time"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/utils/messageSender"
)
// registerNotificationChannel registers one channel owned by a plugin. The
// channel is tied to the current runtime, so unloading the plugin removes it.
func (m *Manager) registerNotificationChannel(
short string,
id string,
configuration models.Configuration,
handler moejs.Value,
host *jsruntime.Host,
) error {
inst := m.instanceFor(short)
if inst == nil {
return fmt.Errorf("plugin %q is not loaded", short)
}
if host == nil {
return fmt.Errorf("plugin %q notification runtime is unavailable", short)
}
notificationChannel := &pluginNotificationChannel{
short: short,
host: host,
handler: handler,
}
inst.mu.Lock()
defer inst.mu.Unlock()
if inst.runtime == nil {
return fmt.Errorf("plugin %q notification runtime is closed", short)
}
if err := messageSender.RegisterNotificationChannel(id, configuration, notificationChannel); err != nil {
return err
}
if inst.notificationChannels == nil {
inst.notificationChannels = make(map[string]struct{})
}
inst.notificationChannels[id] = struct{}{}
return nil
}
type pluginNotificationChannel struct {
short string
host *jsruntime.Host
handler moejs.Value
}
func (c *pluginNotificationChannel) Send(ctx context.Context, notification messageSender.Notification, config map[string]any) error {
if c.host == nil || c.handler.IsUndefined() {
return fmt.Errorf("plugin %q notification channel is closed", c.short)
}
result := make(chan error, 1)
var once sync.Once
finish := func(err error) {
once.Do(func() { result <- err })
}
queued := c.host.RunOnLoop(func(vm *moejs.Runtime) {
var value moejs.Value
runErr := c.host.RunJob(vm, "plugin notification "+c.short, func() error {
raw, err := json.Marshal(notification)
if err != nil {
return err
}
var notificationValue map[string]any
if err := json.Unmarshal(raw, &notificationValue); err != nil {
return err
}
var callErr error
value, callErr = vm.CallFunction(c.handler,
moejs.Undefined(),
native.From(vm, notificationValue),
native.From(vm, config),
)
return callErr
})
if runErr != nil {
finish(runErr)
return
}
_, _, ok := moejs.PromiseResult(value)
if !ok {
finish(nil)
return
}
then := native.Get(vm, native.Object(vm, value), "then")
if !engine.IsCallable(then) {
finish(fmt.Errorf("plugin %q notification channel returned a Promise without then", c.short))
return
}
onFulfilled := native.Bind(vm, "notificationFulfilled", 0, func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
finish(nil)
return moejs.Undefined(), nil
})
onRejected := native.Bind(vm, "notificationRejected", 1, func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
finish(fmt.Errorf("plugin %q notification channel failed: %s", c.short, native.String(vm, moejs.Arg(args, 0))))
return moejs.Undefined(), nil
})
if _, err := vm.CallFunction(then, value, onFulfilled, onRejected); err != nil {
finish(fmt.Errorf("plugin %q notification channel Promise failed: %w", c.short, err))
}
})
if !queued {
return fmt.Errorf("plugin %q notification runtime is closed", c.short)
}
timeout := c.host.Timeout()
if deadline, ok := ctx.Deadline(); ok {
if remaining := time.Until(deadline); remaining < timeout {
timeout = remaining
}
}
if timeout <= 0 {
return context.DeadlineExceeded
}
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
case err := <-result:
return err
case <-ctx.Done():
return ctx.Err()
case <-timer.C:
return fmt.Errorf("plugin %q notification channel timed out", c.short)
}
}
func (c *pluginNotificationChannel) Unload() error {
return nil
}
func (m *Manager) unregisterNotificationChannels(inst *Instance) error {
if inst == nil {
return nil
}
inst.mu.Lock()
ids := make([]string, 0, len(inst.notificationChannels))
for id := range inst.notificationChannels {
ids = append(ids, id)
}
inst.notificationChannels = nil
inst.mu.Unlock()
var firstErr error
for _, id := range ids {
if err := messageSender.UnregisterNotificationChannel(id); err != nil && firstErr == nil {
firstErr = err
}
}
return firstErr
}
+40 -15
View File
@@ -29,12 +29,12 @@ import (
"sync"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/require"
"github.com/Calcium-Ion/moejs"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/scheduler"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"github.com/komari-monitor/komari/pkg/rpc"
"github.com/komari-monitor/komari/web/connection"
)
@@ -113,15 +113,16 @@ type Manager struct {
// Instance is one loaded plugin runtime plus its host bindings.
type Instance struct {
mu sync.RWMutex
info models.Plugin
dir string
runtime *jsruntime.Runtime
host *jsruntime.Host
handlers map[string]goja.Callable // "METHOD path" -> current route handler
statics map[string]*staticConfig // mount path -> static folder config
rpcMethods map[string]goja.Callable // registered RPC method -> JS handler
cronJobs []string // scheduler job names, removed on unload
mu sync.RWMutex
info models.Plugin
dir string
runtime *jsruntime.Runtime
host *jsruntime.Host
handlers map[string]moejs.Value // "METHOD path" -> current route handler
statics map[string]*staticConfig // mount path -> static folder config
rpcMethods map[string]moejs.Value // registered RPC method -> JS handler
cronJobs []string // scheduler job names, removed on unload
notificationChannels map[string]struct{}
}
// global is the process-wide plugin manager.
@@ -153,6 +154,13 @@ func SetEnabled(short string, enabled, approved bool) error {
return global.setEnabled(short, enabled, approved)
}
// Reload unloads and loads an enabled plugin. The load hook completes before
// this function returns; a failed load is persisted as the plugin's last
// error and disables the plugin, matching the normal enable/startup path.
func Reload(short string) error {
return global.reload(short)
}
// CloseAll unloads every loaded plugin. Used during server shutdown.
func CloseAll() error {
return global.closeAll()
@@ -254,7 +262,7 @@ func (m *Manager) load(short string) error {
logs.Reset()
_, _ = logs.Write([]byte("[plugin] loading " + short + "\n"))
inst := &Instance{info: info, dir: dir, handlers: make(map[string]goja.Callable), statics: make(map[string]*staticConfig)}
inst := &Instance{info: info, dir: dir, handlers: make(map[string]moejs.Value), statics: make(map[string]*staticConfig)}
m.mu.Lock()
if _, ok := m.instances[short]; ok {
m.mu.Unlock()
@@ -298,8 +306,8 @@ func (m *Manager) load(short string) error {
if rt.HasFunction("load") {
if err := rt.CallVoid("load"); err != nil {
rt.Close()
m.dropInstance(short, inst)
rt.Close()
return fmt.Errorf("plugin %q load() failed: %w", short, err)
}
}
@@ -321,11 +329,15 @@ func (m *Manager) load(short string) error {
// unload behavior. The caller must not hold any Manager lock.
func (m *Manager) dropInstance(short string, inst *Instance) {
m.mu.Lock()
defer m.mu.Unlock()
if m.instances[short] != inst {
m.mu.Unlock()
return // replaced by a newer load or already unloaded
}
delete(m.instances, short)
m.mu.Unlock()
_ = m.unregisterNotificationChannels(inst)
m.mu.Lock()
defer m.mu.Unlock()
inst.mu.Lock()
for method := range inst.rpcMethods {
rpc.Unregister(method)
@@ -378,6 +390,9 @@ func (m *Manager) unload(short string) error {
}
}
}
if err := m.unregisterNotificationChannels(inst); err != nil && unloadErr == nil {
unloadErr = err
}
m.mu.Lock()
inst.mu.Lock()
for method := range inst.rpcMethods {
@@ -521,6 +536,16 @@ func (m *Manager) restartPlugin(short string) error {
return nil
}
func (m *Manager) reload(short string) error {
if !m.stateStore().get(short).Enabled {
return nil
}
if err := m.unload(short); err != nil && !errors.Is(err, errNotLoaded) {
return err
}
return m.restartPlugin(short)
}
// disableWithError persists the auto-disable state for one plugin and wraps
// the load failure for the startup error summary.
func (m *Manager) disableWithError(short string, st PluginState, err error) error {
@@ -659,7 +684,7 @@ func (m *Manager) hooksOf(kind hookKind) []*hookEntry {
// during script evaluation; it takes the manager lock itself. matcher is
// nil for unfiltered hooks; the plugin-declared body limit is captured at
// registration so unloaded plugins never contribute a stale limit.
func (m *Manager) registerHook(short string, kind hookKind, fn goja.Callable, matcher *hookMatcher) {
func (m *Manager) registerHook(short string, kind hookKind, fn moejs.Value, matcher *hookMatcher) {
m.mu.Lock()
defer m.mu.Unlock()
inst := m.instances[short]
+507
View File
@@ -0,0 +1,507 @@
package plugin
import (
"errors"
"fmt"
"math"
"net"
"sync"
"time"
"unicode/utf8"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gorilla/websocket"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/web/connection"
)
const (
routeWSOpen = 1
routeWSClosing = 2
routeWSClosed = 3
routeWSMaxQueuedFrames = 64
routeWSMaxQueuedBytes = 16 << 20
routeWSWriteTimeout = 5 * time.Second
routeWSCloseTimeout = 5 * time.Second
)
type routeWSFrame struct {
kind int
data []byte
}
type routeWSCloseEvent struct {
code int
reason string
clean bool
message string
}
// routeWebSocket is registered before hijacking: runtime shutdown must be able
// to close the network connection even while UpgradeSafeConn waits for a
// synchronous wsConnect hook. No SafeConn operation or hook runs on the VM loop.
// mu protects transport state only; jsObject belongs exclusively to the loop.
type routeWebSocket struct {
host *jsruntime.Host
mu sync.Mutex
resourceID uint64
netConn net.Conn
conn *connection.SafeConn
raw *websocket.Conn
state int
started bool
finished bool
notified bool
result routeWSCloseEvent
closeTimer *time.Timer
queue []routeWSFrame
queuedCount int // includes the writer's in-flight frame
queuedBytes int
wake chan struct{}
done chan struct{}
closeRequested bool
closeCode int
closeReason string
closeWritten bool
peerClosed bool
peerCode int
peerReason string
pong string
pongPending bool
jsObject *engine.Object
}
func newRouteWebSocket(host *jsruntime.Host) *routeWebSocket {
ws := &routeWebSocket{
host: host, state: routeWSClosed,
wake: make(chan struct{}, 1), done: make(chan struct{}),
}
// AddResource may call abort synchronously, or shutdown may race with its
// return. Never publish the ID without checking whether teardown already ran.
id := host.AddResource(ws.abort)
ws.mu.Lock()
ws.resourceID = id
finished := ws.finished
ws.mu.Unlock()
if finished {
host.RemoveResource(id)
}
return ws
}
// captureConn is called by the HTTP writer's Hijack method, before the
// upgrader can enter any hooks. It also handles shutdown winning that race.
func (ws *routeWebSocket) captureConn(conn net.Conn) {
ws.mu.Lock()
finished := ws.finished
if !finished {
ws.netConn = conn
}
ws.mu.Unlock()
if finished && conn != nil {
_ = conn.Close()
}
}
// attach runs on the HTTP pump, never the VM loop: GetConn takes SafeConn's
// write mutex. The underlying Gorilla connection is private to these pumps.
func (ws *routeWebSocket) attach(conn *connection.SafeConn) bool {
if conn == nil {
return false
}
raw := conn.GetConn()
raw.SetReadLimit(maxWSFrameBytes)
raw.SetCloseHandler(func(code int, reason string) error {
ws.mu.Lock()
if !ws.finished {
ws.peerClosed, ws.peerCode, ws.peerReason = true, code, reason
// The peer cannot consume further data. Drop queued frames and
// prioritize the close reply (the in-flight writer stays counted).
for _, frame := range ws.queue {
ws.queuedCount--
ws.queuedBytes -= len(frame.data)
}
ws.queue = nil
ws.pong, ws.pongPending = "", false
if !ws.closeRequested {
ws.requestCloseLocked(code, reason)
}
}
ws.mu.Unlock()
ws.signal()
return nil // the writer, not the reader, sends the close reply
})
raw.SetPingHandler(func(data string) error {
ws.mu.Lock()
if !ws.finished {
// A ping flood must not allocate an unbounded control queue. RFC 6455
// allows replying only to the most recently processed ping.
ws.pong, ws.pongPending = data, true
}
ws.mu.Unlock()
ws.signal()
return nil
})
ws.mu.Lock()
if ws.finished || ws.conn != nil {
ws.mu.Unlock()
_ = raw.Close()
go func() { _ = conn.Close() }()
return false
}
ws.conn, ws.raw, ws.state = conn, raw, routeWSOpen
if ws.netConn == nil {
ws.netConn = raw.UnderlyingConn()
}
ws.mu.Unlock()
return true
}
// abort is a non-waiting resource callback. Close the captured network socket
// first to unblock I/O; SafeConn.Close may wait for same-plugin hooks, so it
// must happen separately, after the socket is already closed.
func (ws *routeWebSocket) abort() {
ws.finish(routeWSCloseEvent{code: websocket.CloseAbnormalClosure})
}
func (ws *routeWebSocket) finish(result routeWSCloseEvent) {
ws.mu.Lock()
if ws.finished {
ws.mu.Unlock()
return
}
ws.finished, ws.state, ws.result = true, routeWSClosed, result
ws.queue = nil
ws.queuedCount, ws.queuedBytes = 0, 0
if ws.closeTimer != nil {
ws.closeTimer.Stop()
}
netConn, conn, id := ws.netConn, ws.conn, ws.resourceID
notify := ws.started && !ws.notified
if notify {
ws.notified = true
}
ws.mu.Unlock()
if netConn != nil {
_ = netConn.Close()
}
ws.host.RemoveResource(id)
close(ws.done)
if conn != nil {
go func() { _ = conn.Close() }()
}
if notify {
ws.notifyClose(result)
}
}
// start is called on a later loop turn than promise resolution. This barrier
// lets .then/await reactions install handlers before any data or close event.
func (ws *routeWebSocket) start() {
ws.mu.Lock()
if ws.started {
ws.mu.Unlock()
return
}
ws.started = true
if ws.finished {
notify, result := !ws.notified, ws.result
ws.notified = true
ws.mu.Unlock()
if notify {
ws.notifyClose(result)
}
return
}
attached := ws.conn != nil
ws.mu.Unlock()
if !attached || !ws.host.ResourcesOpen() {
ws.abort()
return
}
go ws.writePump()
go ws.readPump()
}
func (ws *routeWebSocket) signal() {
select {
case ws.wake <- struct{}{}:
default:
}
}
func (ws *routeWebSocket) requestCloseLocked(code int, reason string) {
ws.state, ws.closeRequested = routeWSClosing, true
ws.closeCode, ws.closeReason = code, reason
ws.closeTimer = time.AfterFunc(routeWSCloseTimeout, func() {
ws.fail(errors.New("WebSocket close handshake timed out"))
})
}
func (ws *routeWebSocket) fail(err error) {
ws.finish(routeWSCloseEvent{code: websocket.CloseAbnormalClosure, message: err.Error()})
}
func (ws *routeWebSocket) writePump() {
for {
ws.mu.Lock()
if ws.finished {
ws.mu.Unlock()
return
}
if ws.closeWritten && ws.peerClosed {
result := routeWSCloseEvent{code: ws.peerCode, reason: ws.peerReason, clean: true}
ws.mu.Unlock()
ws.finish(result)
return
}
var frame routeWSFrame
control, haveFrame := false, false
switch {
case ws.pongPending && !ws.closeWritten:
frame = routeWSFrame{kind: websocket.PongMessage, data: []byte(ws.pong)}
ws.pong, ws.pongPending, control, haveFrame = "", false, true, true
case len(ws.queue) != 0:
frame, haveFrame = ws.queue[0], true
ws.queue[0] = routeWSFrame{}
ws.queue = ws.queue[1:]
case ws.closeRequested && !ws.closeWritten:
frame = routeWSFrame{kind: websocket.CloseMessage, data: websocket.FormatCloseMessage(ws.closeCode, ws.closeReason)}
control, haveFrame = true, true
}
conn, raw := ws.conn, ws.raw
ws.mu.Unlock()
if !haveFrame {
select {
case <-ws.wake:
case <-ws.done:
return
}
continue
}
deadline := time.Now().Add(routeWSWriteTimeout)
var err error
if control {
err = conn.WriteControl(frame.kind, frame.data, deadline)
} else {
err = raw.SetWriteDeadline(deadline)
if err == nil {
err = conn.WriteMessage(frame.kind, frame.data)
}
}
if err != nil {
ws.fail(err)
return
}
ws.mu.Lock()
if !ws.finished {
if frame.kind == websocket.CloseMessage {
ws.closeWritten = true
} else if !control {
ws.queuedCount--
ws.queuedBytes -= len(frame.data)
}
}
ws.mu.Unlock()
}
}
func (ws *routeWebSocket) readPump() {
for {
kind, data, err := ws.conn.ReadMessage()
if err != nil {
ws.mu.Lock()
peerClosed := ws.peerClosed
ws.mu.Unlock()
if peerClosed {
ws.signal() // the writer completes the close handshake
} else {
ws.fail(err)
}
return
}
if len(data) > maxWSFrameBytes {
ws.fail(fmt.Errorf("WebSocket message exceeds %d bytes", maxWSFrameBytes))
return
}
if kind != websocket.TextMessage && kind != websocket.BinaryMessage {
continue
}
if kind == websocket.TextMessage && !utf8.Valid(data) {
ws.fail(errors.New("WebSocket text message is not valid UTF-8"))
return
}
ack := make(chan struct{})
if !ws.host.RunOnLoop(func(vm *moejs.Runtime) {
defer close(ack)
jobErr := ws.host.RunJob(vm, "plugin WebSocket.onmessage", func() error {
ws.mu.Lock()
open := ws.state == routeWSOpen
ws.mu.Unlock()
if !open || ws.jsObject == nil {
return nil
}
var value any = string(data)
if kind == websocket.BinaryMessage {
buffer, err := vm.Realm().NewArrayBuffer(data)
if err != nil {
return err
}
value = engine.ObjectValue(buffer)
}
event := vm.Realm().NewObject()
if err := native.Set(vm, event, "data", value); err != nil {
return err
}
return ws.callHandler(vm, "onmessage", event)
})
if jobErr != nil {
ws.fail(jobErr)
}
}) {
ws.abort()
return
}
// Only one inbound payload may wait for JavaScript. Cancellation must
// also release this reader when queued jobs are discarded on shutdown.
select {
case <-ack:
case <-ws.done:
return
}
}
}
// All methods below that touch jsObject or JS values run only on the host loop.
func (ws *routeWebSocket) object(vm *moejs.Runtime) *engine.Object {
if ws.jsObject != nil {
return ws.jsObject
}
obj := vm.Realm().NewObject()
ws.jsObject = obj
for _, name := range []string{"onmessage", "onerror", "onclose"} {
_ = native.Set(vm, obj, name, moejs.Null())
}
getter := native.Bind(vm, "get readyState", 0, func(_ *moejs.Realm, _ moejs.Value, _ []moejs.Value) (moejs.Value, error) {
ws.mu.Lock()
state := ws.state
ws.mu.Unlock()
return native.From(vm, state), nil
})
desc := engine.AccessorDescriptor(getter.AsObject(), nil, 0)
desc.SetEnumerable(true)
if err := obj.DefinePropertyOrThrow(vm.Realm(), vm.Realm().KeyFromGoString("readyState"), desc); err != nil {
panic(err)
}
_ = native.Function(vm, obj, "send", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := moejs.Arg(args, 0)
kind := websocket.BinaryMessage
var data []byte
if value.IsString() {
kind = websocket.TextMessage
text := native.String(vm, value)
if len(text) > maxWSFrameBytes {
return moejs.Undefined(), native.TypeError(vm, "WebSocket message exceeds %d bytes", maxWSFrameBytes)
}
data = []byte(text)
} else if value.IsObject() {
var ok bool
data, ok = value.AsObject().BufferData()
if !ok {
return moejs.Undefined(), native.TypeError(vm, "WebSocket.send requires a string, ArrayBuffer or ArrayBuffer view")
}
} else {
return moejs.Undefined(), native.TypeError(vm, "WebSocket.send requires a string, ArrayBuffer or ArrayBuffer view")
}
if len(data) > maxWSFrameBytes {
return moejs.Undefined(), native.TypeError(vm, "WebSocket message exceeds %d bytes", maxWSFrameBytes)
}
ws.mu.Lock()
if ws.state != routeWSOpen {
ws.mu.Unlock()
return moejs.Undefined(), native.TypeError(vm, "WebSocket is not open")
}
if ws.queuedCount >= routeWSMaxQueuedFrames || len(data) > routeWSMaxQueuedBytes-ws.queuedBytes {
ws.mu.Unlock()
return moejs.Undefined(), native.TypeError(vm, "WebSocket send queue is full")
}
ws.queue = append(ws.queue, routeWSFrame{kind: kind, data: append([]byte(nil), data...)})
ws.queuedCount++
ws.queuedBytes += len(data)
ws.mu.Unlock()
ws.signal()
return moejs.Undefined(), nil
})
_ = native.Function(vm, obj, "close", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
code := websocket.CloseNormalClosure
if v := moejs.Arg(args, 0); !v.IsUndefined() {
number := native.Float(vm, v)
if math.IsNaN(number) || math.IsInf(number, 0) || math.Trunc(number) != number || (number != 1000 && (number < 3000 || number > 4999)) {
return moejs.Undefined(), native.TypeError(vm, "WebSocket close code must be 1000 or between 3000 and 4999")
}
code = int(number)
}
reason := ""
if v := moejs.Arg(args, 1); !v.IsUndefined() {
reason = native.String(vm, v)
}
if len(reason) > 123 || !utf8.ValidString(reason) {
return moejs.Undefined(), native.TypeError(vm, "WebSocket close reason must be valid UTF-8 and at most 123 bytes")
}
ws.mu.Lock()
if ws.state == routeWSOpen {
ws.requestCloseLocked(code, reason)
}
ws.mu.Unlock()
ws.signal()
return moejs.Undefined(), nil
})
return obj
}
func (ws *routeWebSocket) callHandler(vm *moejs.Runtime, name string, event *engine.Object) error {
if ws.jsObject == nil {
return nil
}
callback, err := vm.Get(engine.ObjectValue(ws.jsObject), name)
if err != nil || !engine.IsCallable(callback) {
return err
}
_, err = vm.CallFunction(callback, engine.ObjectValue(ws.jsObject), engine.ObjectValue(event))
return err
}
func (ws *routeWebSocket) notifyClose(result routeWSCloseEvent) {
ws.host.RunOnLoop(func(vm *moejs.Runtime) {
if ws.jsObject == nil {
return
}
defer func() { ws.jsObject = nil }()
if result.message != "" {
_ = ws.host.RunJob(vm, "plugin WebSocket.onerror", func() error {
event := vm.Realm().NewObject()
if err := native.Set(vm, event, "message", result.message); err != nil {
return err
}
return ws.callHandler(vm, "onerror", event)
})
}
// A throwing onerror must not suppress the final onclose callback.
_ = ws.host.RunJob(vm, "plugin WebSocket.onclose", func() error {
event := vm.Realm().NewObject()
_ = native.Set(vm, event, "code", result.code)
_ = native.Set(vm, event, "reason", result.reason)
_ = native.Set(vm, event, "wasClean", result.clean)
return ws.callHandler(vm, "onclose", event)
})
})
}
+24 -22
View File
@@ -7,7 +7,8 @@ import (
"strings"
"time"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/rpc"
)
@@ -15,7 +16,7 @@ import (
// JavaScript function receiving the JSON-RPC params and returning the result
// (or throwing an Error with optional code/data). Called from the plugin's
// own event loop during script evaluation; it takes the manager lock itself.
func (m *Manager) registerRPC(short, method string, handler goja.Callable) error {
func (m *Manager) registerRPC(short, method string, handler moejs.Value) error {
method = strings.TrimSpace(method)
if method == "" || strings.HasPrefix(method, "rpc.") {
return fmt.Errorf("invalid RPC method name %q", method)
@@ -28,7 +29,7 @@ func (m *Manager) registerRPC(short, method string, handler goja.Callable) error
}
inst.mu.Lock()
if inst.rpcMethods == nil {
inst.rpcMethods = make(map[string]goja.Callable)
inst.rpcMethods = make(map[string]moejs.Value)
}
if _, exists := inst.rpcMethods[method]; exists {
inst.mu.Unlock()
@@ -57,24 +58,24 @@ func (m *Manager) rpcHandler(short, method string) rpc.Handler {
host := inst.host
alive := inst.runtime != nil
inst.mu.RUnlock()
if !alive || handler == nil || host == nil {
if !alive || handler.IsUndefined() || host == nil {
return nil, &rpc.JsonRpcError{Code: rpc.MethodNotFound, Message: "method not found", Data: method}
}
resultCh := make(chan any, 1)
errCh := make(chan *rpc.JsonRpcError, 1)
queued := host.RunOnLoop(func(vm *goja.Runtime) {
queued := host.RunOnLoop(func(vm *moejs.Runtime) {
_ = host.RunJob(vm, "plugin rpc "+method, func() error {
params := vm.ToValue(req.Params)
if params == nil {
params = goja.Null()
params := native.From(vm, req.Params)
if params.IsUndefined() {
params = moejs.Null()
}
value, callErr := handler(goja.Undefined(), params)
value, callErr := vm.CallFunction(handler, moejs.Undefined(), params)
if callErr != nil {
errCh <- jsErrorToRPCError(vm, callErr)
return nil // error is captured, do not double-report
}
exported, exportErr := exportJSValue(value)
exported, exportErr := exportJSValue(vm, value)
if exportErr != nil {
errCh <- &rpc.JsonRpcError{Code: rpc.InternalError, Message: exportErr.Error()}
return nil
@@ -99,38 +100,39 @@ func (m *Manager) rpcHandler(short, method string) rpc.Handler {
}
}
// exportJSValue exports a goja value into a JSON-RPC result. Circular
// exportJSValue exports a native JavaScript value into a JSON-RPC result. Circular
// structures and other export failures become an error instead of panicking
// inside the event loop.
func exportJSValue(value goja.Value) (result any, err error) {
func exportJSValue(vm *moejs.Runtime, value moejs.Value) (result any, err error) {
defer func() {
if recovered := recover(); recovered != nil {
result = nil
err = fmt.Errorf("export JavaScript value: %v", recovered)
}
}()
return value.Export(), nil
return vm.ToGo(value)
}
// jsErrorToRPCError converts a thrown JavaScript error into a JSON-RPC error,
// honoring optional code/data properties.
func jsErrorToRPCError(vm *goja.Runtime, err error) *rpc.JsonRpcError {
func jsErrorToRPCError(vm *moejs.Runtime, err error) *rpc.JsonRpcError {
code := rpc.InternalError
message := err.Error()
var data any
var exception *goja.Exception
var exception *moejs.Exception
if errors.As(err, &exception) {
if obj, ok := exception.Value().(*goja.Object); ok {
if value := obj.Get("code"); value != nil && !goja.IsUndefined(value) {
if n := value.ToInteger(); n != 0 {
if exception.Value.IsObject() {
obj := exception.Value.AsObject()
if value := native.Get(vm, obj, "code"); !value.IsUndefined() {
if n := native.Int(vm, value); n != 0 {
code = int(n)
}
}
if value := obj.Get("message"); value != nil && !goja.IsUndefined(value) && value.String() != "" {
message = value.String()
if value := native.Get(vm, obj, "message"); !value.IsUndefined() && native.String(vm, value) != "" {
message = native.String(vm, value)
}
if value := obj.Get("data"); value != nil && !goja.IsUndefined(value) {
if exported, exportErr := exportJSValue(value); exportErr == nil {
if value := native.Get(vm, obj, "data"); !value.IsUndefined() {
if exported, exportErr := exportJSValue(vm, value); exportErr == nil {
data = exported
}
}
+363 -137
View File
@@ -1,10 +1,13 @@
package plugin
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net"
"net/http"
"os"
"path/filepath"
@@ -13,13 +16,17 @@ import (
"sync/atomic"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gin-gonic/gin"
"github.com/gorilla/websocket"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"github.com/komari-monitor/komari/pkg/rpc"
"github.com/komari-monitor/komari/web/api"
)
// registerServerModule registers the "server" native module for one plugin
@@ -27,7 +34,8 @@ import (
//
// server.route(method, path, handler) register an HTTP route on the host
// engine; handler receives (req, res)
// and must call res.end() to finish
// and finishes via res.end() or an
// awaited req.upgradeWebSocket()
// server.static(path, dir, opts) serve a folder from the plugin
// directory at a mount path; opts.spa
// makes unmatched paths fall back to
@@ -52,6 +60,8 @@ import (
// result or rejects with an Error
// carrying code/message/data
// server.registerRPC(method, handler) register a plugin-owned RPC method
// server.registerNotificationChannel(id, configuration, handler)
// register a notification channel
// server.getConfig() resolve the saved plugin configuration
// server.cron(expr, fn) run fn on the plugin event loop each
// time the cron expression fires
@@ -67,49 +77,50 @@ import (
// The host engine keeps a registered route slot after unload; requests then
// receive 404 until the plugin is loaded again.
func (m *Manager) registerServerModule(host *jsruntime.Host, registry *require.Registry, inst *Instance) {
registry.RegisterNativeModule("server", func(vm *goja.Runtime, module *goja.Object) {
exports := vm.NewObject()
_ = exports.Set("route", func(call goja.FunctionCall) goja.Value {
method := strings.ToUpper(strings.TrimSpace(call.Argument(0).String()))
path := call.Argument(1).String()
handler, ok := goja.AssertFunction(call.Argument(2))
registry.RegisterNativeModule("server", func(vm *moejs.Runtime, module *engine.Object) {
exports := vm.Realm().NewObject()
_ = native.Function(vm, exports, "route", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
method := strings.ToUpper(strings.TrimSpace(native.String(vm, moejs.Arg(args, 0))))
path := native.String(vm, moejs.Arg(args, 1))
handler := moejs.Arg(args, 2)
ok := engine.IsCallable(handler)
if method == "" {
panic(vm.NewTypeError("server.route requires an HTTP method"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires an HTTP method")
}
if path == "" || !strings.HasPrefix(path, "/") {
panic(vm.NewTypeError("server.route requires a path starting with /"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires a path starting with /")
}
if !ok {
panic(vm.NewTypeError("server.route requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires a function handler")
}
if !inst.info.Permissions.AllowRoutes {
panic(vm.NewTypeError("server.route requires the \"route\" permission (allowRoutes)"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires the \"route\" permission (allowRoutes)")
}
if err := m.registerRoute(inst.info.Short, method, path, handler); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("static", func(call goja.FunctionCall) goja.Value {
mount := strings.TrimSpace(call.Argument(0).String())
_ = native.Function(vm, exports, "static", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
mount := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
mount = strings.TrimRight(mount, "/")
dir := strings.TrimSpace(call.Argument(1).String())
dir := strings.TrimSpace(native.String(vm, moejs.Arg(args, 1)))
if mount == "" || mount == "/" || !strings.HasPrefix(mount, "/") {
panic(vm.NewTypeError("server.static requires a mount path starting with / (and not \"/\")"))
return moejs.Undefined(), native.TypeError(vm, "server.static requires a mount path starting with / (and not \"/\")")
}
if dir == "" {
panic(vm.NewTypeError("server.static requires a folder name"))
return moejs.Undefined(), native.TypeError(vm, "server.static requires a folder name")
}
if !filepath.IsLocal(dir) {
panic(vm.NewTypeError("server.static folder must be a relative path inside the plugin directory"))
return moejs.Undefined(), native.TypeError(vm, "server.static folder must be a relative path inside the plugin directory")
}
if !inst.info.Permissions.AllowRoutes {
panic(vm.NewTypeError("server.static requires the \"route\" permission (allowRoutes)"))
return moejs.Undefined(), native.TypeError(vm, "server.static requires the \"route\" permission (allowRoutes)")
}
var opts map[string]any
if !goja.IsUndefined(call.Argument(2)) && !goja.IsNull(call.Argument(2)) {
if err := vm.ExportTo(call.Argument(2), &opts); err != nil {
panic(vm.NewTypeError("server.static options must be an object like { spa: true }"))
if !moejs.Arg(args, 2).IsUndefined() && !moejs.Arg(args, 2).IsNull() {
if err := native.ExportTo(vm, moejs.Arg(args, 2), &opts); err != nil {
return moejs.Undefined(), native.TypeError(vm, "server.static options must be an object like { spa: true }")
}
}
spa := false
@@ -117,137 +128,172 @@ func (m *Manager) registerServerModule(host *jsruntime.Host, registry *require.R
spa, _ = v.(bool)
}
if err := m.registerStatic(inst.info.Short, mount, dir, spa); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("call", func(call goja.FunctionCall) goja.Value {
method := strings.TrimSpace(call.Argument(0).String())
_ = native.Function(vm, exports, "call", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
method := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
if method == "" {
panic(vm.NewTypeError("server.call requires an RPC method name"))
return moejs.Undefined(), native.TypeError(vm, "server.call requires an RPC method name")
}
params := callParams(call)
params := callParams(vm, args)
promise, resolve, reject := vm.NewPromise()
go func() {
if !inst.info.Permissions.AllowSystemRPC {
host.RunOnLoop(func(vm *goja.Runtime) {
_ = reject(vm.NewGoError(errors.New("server.call requires the \"system RPC\" permission (allowSystemRPC)")))
host.RunOnLoop(func(vm *moejs.Runtime) {
_ = reject(native.GoError(vm, errors.New("server.call requires the \"system RPC\" permission (allowSystemRPC)")))
})
return
}
meta := &rpc.ContextMeta{Permission: rpc.RoleAdmin, Principal: rpc.PrincipalFromRole(rpc.RoleAdmin)}
ctx := rpc.NewContextWithMeta(context.Background(), meta)
resp := rpc.CallWithContext(ctx, nil, method, params)
host.RunOnLoop(func(vm *goja.Runtime) {
host.RunOnLoop(func(vm *moejs.Runtime) {
if resp.Error != nil {
_ = reject(jsRPCError(vm, resp.Error))
_ = reject(engine.ObjectValue(jsRPCError(vm, resp.Error)))
return
}
_ = resolve(resp.Result)
result, err := normalizeRPCValue(resp.Result)
if err != nil {
_ = reject(native.GoError(vm, err))
return
}
_ = resolve(native.From(vm, result))
})
}()
return vm.ToValue(promise)
return promise, nil
})
_ = exports.Set("hook", func(call goja.FunctionCall) goja.Value {
kind := strings.ToLower(strings.TrimSpace(call.Argument(0).String()))
_ = native.Function(vm, exports, "hook", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
kind := strings.ToLower(strings.TrimSpace(native.String(vm, moejs.Arg(args, 0))))
switch kind {
case "request", "response", string(hookWSConnect), string(hookWSMessage), string(hookWSSend), string(hookWSClose):
if !inst.info.Permissions.AllowHooks {
panic(vm.NewTypeError("server.hook requires the \"hook\" permission (allowHooks)"))
return moejs.Undefined(), native.TypeError(vm, "server.hook requires the \"hook\" permission (allowHooks)")
}
default:
panic(vm.NewTypeError("server.hook kind must be \"request\", \"response\", \"wsConnect\", \"wsMessage\", \"wsSend\" or \"wsClose\""))
return moejs.Undefined(), native.TypeError(vm, "server.hook kind must be \"request\", \"response\", \"wsConnect\", \"wsMessage\", \"wsSend\" or \"wsClose\"")
}
// server.hook(kind, fn) or server.hook(kind, matcher, fn) where
// matcher is "METHOD /path", "/path" or "/path/*". The ws kinds
// only accept a path matcher: every upgrade is a GET request.
fnValue := call.Argument(1)
fnValue := moejs.Arg(args, 1)
var matcher *hookMatcher
if goja.IsString(fnValue) {
parsed, err := parseHookMatcher(fnValue.String())
if fnValue.IsString() {
parsed, err := parseHookMatcher(native.String(vm, fnValue))
if err != nil {
panic(vm.NewTypeError(err.Error()))
return moejs.Undefined(), native.TypeError(vm, "%s", err.Error())
}
if isWSHookKind(kind) && parsed.method != "" {
panic(vm.NewTypeError("server.hook \"" + kind + "\" only accepts a path matcher like \"/api/*\""))
return moejs.Undefined(), native.TypeError(vm, "server.hook %q only accepts a path matcher like /api/*", kind)
}
matcher = parsed
fnValue = call.Argument(2)
fnValue = moejs.Arg(args, 2)
}
fn, ok := goja.AssertFunction(fnValue)
fn := fnValue
ok := engine.IsCallable(fn)
if !ok {
panic(vm.NewTypeError("server.hook requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.hook requires a function handler")
}
m.registerHook(inst.info.Short, hookKind(kind), fn, matcher)
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("injectHTML", func(call goja.FunctionCall) goja.Value {
_ = native.Function(vm, exports, "injectHTML", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
if !inst.info.Permissions.AllowHTMLInject {
panic(vm.NewTypeError("server.injectHTML requires the \"HTML inject\" permission (allowHTMLInject)"))
return moejs.Undefined(), native.TypeError(vm, "server.injectHTML requires the \"HTML inject\" permission (allowHTMLInject)")
}
m.registerInject(inst.info.Short, call.Argument(0).String(), call.Argument(1).String())
return goja.Undefined()
m.registerInject(inst.info.Short, native.String(vm, moejs.Arg(args, 0)), native.String(vm, moejs.Arg(args, 1)))
return moejs.Undefined(), nil
})
_ = exports.Set("cron", func(call goja.FunctionCall) goja.Value {
spec := strings.TrimSpace(call.Argument(0).String())
fn, ok := goja.AssertFunction(call.Argument(1))
_ = native.Function(vm, exports, "cron", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
spec := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
fn := moejs.Arg(args, 1)
ok := engine.IsCallable(fn)
if spec == "" {
panic(vm.NewTypeError("server.cron requires a cron expression"))
return moejs.Undefined(), native.TypeError(vm, "server.cron requires a cron expression")
}
if !ok {
panic(vm.NewTypeError("server.cron requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.cron requires a function handler")
}
if err := m.registerCron(inst.info.Short, spec, fn); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("registerRPC", func(call goja.FunctionCall) goja.Value {
method := strings.TrimSpace(call.Argument(0).String())
fn, ok := goja.AssertFunction(call.Argument(1))
_ = native.Function(vm, exports, "registerRPC", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
method := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
fn := moejs.Arg(args, 1)
ok := engine.IsCallable(fn)
if method == "" {
panic(vm.NewTypeError("server.registerRPC requires an RPC method name"))
return moejs.Undefined(), native.TypeError(vm, "server.registerRPC requires an RPC method name")
}
if !ok {
panic(vm.NewTypeError("server.registerRPC requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.registerRPC requires a function handler")
}
if err := m.registerRPC(inst.info.Short, method, fn); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("getConfig", func(call goja.FunctionCall) goja.Value {
_ = native.Function(vm, exports, "registerNotificationChannel", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
id := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
handler := moejs.Arg(args, 2)
ok := engine.IsCallable(handler)
if id == "" {
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a channel id")
}
if !ok {
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a function handler")
}
rawConfiguration, err := exportJSValue(vm, moejs.Arg(args, 1))
if err != nil {
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a managed configuration object")
}
configurationJSON, err := json.Marshal(rawConfiguration)
if err != nil {
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a managed configuration object")
}
var configuration models.Configuration
if err := json.Unmarshal(configurationJSON, &configuration); err != nil {
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a managed configuration object")
}
if err := m.registerNotificationChannel(inst.info.Short, id, configuration, handler, host); err != nil {
return moejs.Undefined(), err
}
return moejs.Undefined(), nil
})
_ = native.Function(vm, exports, "getConfig", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
promise, resolve, reject := vm.NewPromise()
go func() {
data, err := GetConfiguration(inst.info.Short)
host.RunOnLoop(func(vm *goja.Runtime) {
host.RunOnLoop(func(vm *moejs.Runtime) {
if err != nil {
_ = reject(vm.NewGoError(err))
_ = reject(native.GoError(vm, err))
return
}
_ = resolve(data)
_ = resolve(native.From(vm, data))
})
}()
return vm.ToValue(promise)
return promise, nil
})
_ = module.Set("exports", exports)
_ = native.Set(vm, module, "exports", exports)
})
}
// callParams converts JS arguments into JSON-RPC params: no params, a single
// value, or a positional array. Values that cannot be exported (for example
// circular objects) are skipped so the call rejects instead of panicking.
func callParams(call goja.FunctionCall) any {
switch n := len(call.Arguments); {
func callParams(vm *moejs.Runtime, args []moejs.Value) any {
switch n := len(args); {
case n <= 1:
return nil
case n == 2:
exported, _ := exportJSValue(call.Argument(1))
exported, _ := exportJSValue(vm, moejs.Arg(args, 1))
return exported
default:
params := make([]any, 0, n-1)
for i := 1; i < n; i++ {
exported, err := exportJSValue(call.Argument(i))
exported, err := exportJSValue(vm, moejs.Arg(args, i))
if err != nil {
return nil
}
@@ -257,20 +303,33 @@ func callParams(call goja.FunctionCall) any {
}
}
func jsRPCError(vm *goja.Runtime, e *rpc.JsonRpcError) *goja.Object {
obj := vm.NewGoError(fmt.Errorf("%s", e.Message))
_ = obj.Set("code", e.Code)
func jsRPCError(vm *moejs.Runtime, e *rpc.JsonRpcError) *engine.Object {
obj := native.Object(vm, native.GoError(vm, fmt.Errorf("%s", e.Message)))
_ = native.Set(vm, obj, "code", e.Code)
if e.Data != nil {
_ = obj.Set("data", e.Data)
_ = native.Set(vm, obj, "data", e.Data)
}
return obj
}
func normalizeRPCValue(value any) (any, error) {
data, err := json.Marshal(value)
if err != nil {
return nil, fmt.Errorf("serialize RPC result: %w", err)
}
var normalized any
if err := json.Unmarshal(data, &normalized); err != nil {
return nil, fmt.Errorf("decode RPC result: %w", err)
}
return normalized, nil
}
// registerRoute registers (or reuses) a gin route slot for one plugin. It is
// called from the plugin's own event loop during script evaluation and takes
// the manager lock itself. The critical section is short and never waits on
// the event loop, so no lock cycle is possible.
func (m *Manager) registerRoute(short, method, path string, handler goja.Callable) (err error) {
func (m *Manager) registerRoute(short, method, path string, handler moejs.Value) (err error) {
key := method + " " + path
m.mu.Lock()
defer m.mu.Unlock()
@@ -286,7 +345,8 @@ func (m *Manager) registerRoute(short, method, path string, handler goja.Callabl
}
slotExists := m.routes[short][key]
inst.mu.Lock()
if !slotExists && inst.handlers[key] != nil {
// A missing map entry yields a zero moejs.Value, not JavaScript undefined.
if _, registered := inst.handlers[key]; !slotExists && registered {
inst.mu.Unlock()
return nil // already registered in this load
}
@@ -325,7 +385,7 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
alive := inst.runtime != nil
limit := inst.info.Permissions.MaxHTTPBodyBytes
inst.mu.RUnlock()
if !alive || handler == nil || host == nil {
if !alive || handler.IsUndefined() || host == nil {
c.Status(http.StatusNotFound)
return
}
@@ -346,11 +406,23 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
chunks: make(chan []byte, 4),
abortCh: make(chan struct{}),
}
queued := host.RunOnLoop(func(vm *goja.Runtime) {
req := routeRequest(vm, c, body)
state.upgrades = make(chan *routeUpgrade, 1)
defer state.seal()
// Gin recycles c when the request returns, including on a timeout while
// this turn is still queued. Only the HTTP pump uses the original.
snapshot := c.Copy()
snapshot.Request = c.Request.Clone(c.Request.Context())
if principal := rpcPrincipalFromGin(c); principal != nil {
copy := *principal
copy.Roles = append([]string(nil), principal.Roles...)
snapshot.Set(principalContextKey, &copy)
}
queued := host.RunOnLoop(func(vm *moejs.Runtime) {
req := routeRequest(vm, snapshot, body)
res := routeResponseObject(vm, state)
bindRouteUpgrade(vm, req, res, state, host, snapshot.Request)
runErr := host.RunJob(vm, "plugin route "+short, func() error {
_, callErr := handler(goja.Undefined(), req, res)
_, callErr := vm.CallFunction(handler, moejs.Undefined(), engine.ObjectValue(req), engine.ObjectValue(res))
return callErr
})
if runErr != nil {
@@ -391,6 +463,36 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
committed := false
for {
select {
case upgrade := <-state.upgrades:
state.mu.Lock()
state.sealed = true
state.upgradeClaimed = true
state.mu.Unlock()
idle.Stop()
if c.Request.Context().Err() != nil || !host.ResourcesOpen() {
upgrade.ws.abort()
upgrade.complete(errors.New("WebSocket upgrade canceled"))
return
}
stopCancel := context.AfterFunc(c.Request.Context(), upgrade.ws.abort)
original := c.Writer
c.Writer = &routeUpgradeWriter{ResponseWriter: original, ws: upgrade.ws}
socket, upgradeErr := api.UpgradeSafeConn(c, func(upgrader *websocket.Upgrader) {
upgrader.HandshakeTimeout = host.Timeout()
})
c.Writer = original
stopCancel()
if upgradeErr != nil {
upgrade.ws.abort()
upgrade.complete(upgradeErr)
return
}
if !upgrade.ws.attach(socket) {
upgrade.complete(errors.New("WebSocket upgrade canceled"))
return
}
upgrade.complete(nil)
return
case chunk := <-state.chunks:
if !committed {
state.mu.Lock()
@@ -416,19 +518,18 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
}
idle.Reset(host.Timeout())
case <-state.done:
state.seal()
if !committed {
writeFinal(0, nil)
}
return
case <-c.Request.Context().Done():
state.aborted.Store(true)
close(state.abortCh)
state.seal()
return
case <-idle.C:
state.seal()
if state.isStreaming() {
// Streaming idle timeout: end the stream and notify JS
state.aborted.Store(true)
close(state.abortCh)
return
}
writeFinal(http.StatusGatewayTimeout, []byte("plugin route handler timed out"))
@@ -571,12 +672,100 @@ type routeResponse struct {
chunks chan []byte
aborted atomic.Bool
abortCh chan struct{}
wrote bool
sealed bool
upgradeClaimed bool
upgrade *routeUpgrade
upgrades chan *routeUpgrade
}
func (state *routeResponse) markStreaming() {
// routeUpgrade keeps the Promise on its original loop. No network goroutine
// touches the resolver or socket object directly.
type routeUpgrade struct {
ws *routeWebSocket
complete func(error)
}
type routeUpgradeWriter struct {
gin.ResponseWriter
ws *routeWebSocket
}
func (w *routeUpgradeWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
conn, rw, err := w.ResponseWriter.Hijack()
if err == nil {
w.ws.captureConn(conn)
}
return conn, rw, err
}
func bindRouteUpgrade(vm *moejs.Runtime, req, res *engine.Object, state *routeResponse, host *jsruntime.Host, request *http.Request) {
_ = native.Function(vm, req, "upgradeWebSocket", func(_ *moejs.Realm, _ moejs.Value, _ []moejs.Value) (moejs.Value, error) {
promise, resolve, reject := vm.NewPromise()
fail := func(message string) (moejs.Value, error) {
_ = reject(native.GoError(vm, errors.New(message)))
return native.From(vm, promise), nil
}
streaming := native.Bool(native.Get(vm, res, "streaming"))
state.mu.Lock()
if state.sealed || state.ended || state.upgrade != nil || state.wrote || state.streaming || streaming {
state.mu.Unlock()
return fail("HTTP response is no longer available for WebSocket upgrade")
}
if request.Method != http.MethodGet || !websocket.IsWebSocketUpgrade(request) {
state.mu.Unlock()
return fail("require a GET WebSocket upgrade request")
}
ws := newRouteWebSocket(host)
u := &routeUpgrade{ws: ws}
u.complete = func(upgradeErr error) {
if !host.RunOnLoop(func(vm *moejs.Runtime) {
if !host.ResourcesOpen() {
ws.abort()
return
}
err := host.RunJob(vm, "WebSocket upgrade", func() error {
if upgradeErr != nil {
return reject(native.GoError(vm, upgradeErr))
}
return resolve(engine.ObjectValue(ws.object(vm)))
})
if err != nil || upgradeErr != nil {
ws.abort()
return
}
// Promise continuations install handlers before a later loop turn
// starts delivering messages (including an immediate peer send).
if !host.RunOnLoop(func(_ *moejs.Runtime) { ws.start() }) {
ws.abort()
}
}) {
ws.abort()
}
}
state.upgrade = u
state.mu.Unlock()
state.upgrades <- u
return native.From(vm, promise), nil
})
}
// seal prevents queued JavaScript from writing or upgrading a returned request.
func (state *routeResponse) seal() {
state.mu.Lock()
state.streaming = true
wasSealed := state.sealed
state.sealed = true
pending := state.upgrade
claimed := state.upgradeClaimed
state.mu.Unlock()
if state.aborted.CompareAndSwap(false, true) {
close(state.abortCh)
}
if pending != nil && !claimed && !wasSealed {
pending.ws.abort()
pending.complete(errors.New("WebSocket upgrade canceled"))
}
}
func (state *routeResponse) isStreaming() bool {
@@ -599,7 +788,7 @@ func (state *routeResponse) pushChunk(p []byte) bool {
func (state *routeResponse) finish(status int, body []byte) {
state.mu.Lock()
defer state.mu.Unlock()
if state.ended {
if state.ended || state.sealed || state.upgrade != nil {
return
}
if status != 0 {
@@ -612,70 +801,107 @@ func (state *routeResponse) finish(status int, body []byte) {
close(state.done)
}
func routeRequest(vm *goja.Runtime, c *gin.Context, body []byte) *goja.Object {
func routeRequest(vm *moejs.Runtime, c *gin.Context, body []byte) *engine.Object {
req := hookRequestObject(vm, c.Request, body)
_ = req.Set("context", routeRequestContext(vm, c))
_ = native.Set(vm, req, "context", routeRequestContext(vm, c))
return req
}
func routeResponseObject(vm *goja.Runtime, state *routeResponse) *goja.Object {
res := vm.NewObject()
_ = res.Set("statusCode", http.StatusOK)
_ = res.Set("statusMessage", "")
_ = res.Set("streaming", false)
_ = res.Set("isAborted", func() bool { return state.aborted.Load() })
_ = res.Set("setHeader", func(name string, value goja.Value) *goja.Object {
state.mu.Lock()
defer state.mu.Unlock()
var values []string
if err := vm.ExportTo(value, &values); err == nil {
state.headers[name] = values
} else {
state.headers.Set(name, value.String())
}
return res
func routeResponseObject(vm *moejs.Runtime, state *routeResponse) *engine.Object {
res := vm.Realm().NewObject()
_ = native.Set(vm, res, "statusCode", http.StatusOK)
_ = native.Set(vm, res, "statusMessage", "")
_ = native.Set(vm, res, "streaming", false)
_ = native.Function(vm, res, "isAborted", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, state.aborted.Load()), nil
})
_ = res.Set("getHeader", func(name string) goja.Value {
_ = native.Function(vm, res, "setHeader", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
name, value := native.String(vm, moejs.Arg(args, 0)), moejs.Arg(args, 1)
var values []string
if err := native.ExportTo(vm, value, &values); err != nil {
values = []string{native.String(vm, value)}
}
state.mu.Lock()
defer state.mu.Unlock()
values := state.headers.Values(name)
if state.upgrade != nil {
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if !state.sealed {
state.headers[http.CanonicalHeaderKey(name)] = values
}
return engine.ObjectValue(res), nil
})
_ = native.Function(vm, res, "getHeader", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.mu.Lock()
defer state.mu.Unlock()
values := state.headers.Values(native.String(vm, moejs.Arg(args, 0)))
if len(values) == 0 {
return goja.Undefined()
return moejs.Undefined(), nil
}
if len(values) == 1 {
return vm.ToValue(values[0])
return native.From(vm, values[0]), nil
}
return vm.ToValue(values)
return native.From(vm, values), nil
})
_ = res.Set("removeHeader", func(name string) {
_ = native.Function(vm, res, "removeHeader", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
state.headers.Del(name)
state.mu.Unlock()
})
_ = res.Set("write", func(call goja.FunctionCall) goja.Value {
data := buffer.Bytes(vm, call.Argument(0))
if value := res.Get("streaming"); value != nil && value.ToBoolean() {
state.markStreaming()
_ = state.pushChunk(data)
return vm.ToValue(true)
defer state.mu.Unlock()
if state.upgrade != nil {
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if !state.sealed {
state.headers.Del(name)
}
return moejs.Undefined(), nil
})
_ = native.Function(vm, res, "write", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := moejs.Arg(args, 0)
var data []byte
if err := native.ExportTo(vm, value, &data); err != nil {
data = []byte(native.String(vm, value))
}
streaming := native.Bool(native.Get(vm, res, "streaming"))
state.mu.Lock()
if state.upgrade != nil {
state.mu.Unlock()
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if state.sealed || state.ended {
state.mu.Unlock()
return native.From(vm, false), nil
}
state.wrote = true
if streaming {
state.streaming = true
state.mu.Unlock()
return native.From(vm, state.pushChunk(data)), nil
}
_, _ = state.body.Write(data)
state.mu.Unlock()
return vm.ToValue(true)
return native.From(vm, true), nil
})
_ = res.Set("end", func(call goja.FunctionCall) goja.Value {
state.mu.Lock()
if len(call.Arguments) > 0 && !goja.IsUndefined(call.Argument(0)) && !goja.IsNull(call.Argument(0)) {
_, _ = state.body.WriteString(call.Argument(0).String())
_ = native.Function(vm, res, "end", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
var tail string
if len(args) > 0 && !native.IsNil(args[0]) {
tail = native.String(vm, args[0])
}
state.statusCode = int(res.Get("statusCode").ToInteger())
status := int(native.Int(vm, native.Get(vm, res, "statusCode")))
state.mu.Lock()
defer state.mu.Unlock()
if state.upgrade != nil {
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if state.sealed || state.ended {
return engine.ObjectValue(res), nil
}
_, _ = state.body.WriteString(tail)
state.statusCode = status
if !state.ended {
state.ended = true
close(state.done)
}
state.mu.Unlock()
return res
return engine.ObjectValue(res), nil
})
return res
}
+1 -4
View File
@@ -75,9 +75,6 @@ func (s *State) saveLocked() {
file := struct {
Plugins map[string]PluginState `json:"plugins"`
}{Plugins: s.current}
data, err := json.MarshalIndent(file, "", " ")
if err != nil {
return
}
data, _ := json.MarshalIndent(file, "", " ")
_ = os.WriteFile(s.path, data, 0644)
}
+99
View File
@@ -0,0 +1,99 @@
package plugin
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gin-gonic/gin"
)
// TestIsUpgradeRequest 覆盖升级请求判定:真正的协议升级必须同时携带
// Connection: upgrade 记号与 Upgrade 目标协议头(RFC 9110 §7.8);仅带
// 其中一个头的普通请求(如反代模板给每个请求无条件附加
// Connection: upgrade)不得被视为升级。
func TestIsUpgradeRequest(t *testing.T) {
cases := []struct {
name string
connection string
upgrade string
want bool
}{
{"websocket upgrade with both headers", "upgrade", "websocket", true},
{"case insensitive", "Upgrade", "WebSocket", true},
{"connection list keeps other tokens", "keep-alive, upgrade", "websocket", true},
{"bare connection upgrade is not an upgrade", "upgrade", "", false},
{"upgrade header alone is not an upgrade", "", "websocket", false},
{"plain request", "", "", false},
{"keep-alive only", "keep-alive", "", false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/", nil)
if tc.connection != "" {
req.Header.Set("Connection", tc.connection)
}
if tc.upgrade != "" {
req.Header.Set("Upgrade", tc.upgrade)
}
if got := isUpgradeRequest(req); got != tc.want {
t.Fatalf("isUpgradeRequest(Connection=%q, Upgrade=%q) = %v, want %v",
tc.connection, tc.upgrade, got, tc.want)
}
})
}
}
// TestInjectSurvivesBareConnectionUpgradeHeader 回归:部分反代模板(如 1Panel
// 的 OpenResty 站点模板)会给每个代理请求无条件附加 Connection: upgrade。
// 这类普通页面请求必须照常获得插件 HTML 注入;只有同时携带两个头的真实升级
// 才走放行路径。
func TestInjectSurvivesBareConnectionUpgradeHeader(t *testing.T) {
withTempDataDir(t)
gin.SetMode(gin.TestMode)
engine := gin.New()
engine.GET("/admin/dashboard", func(c *gin.Context) {
c.Data(http.StatusOK, "text/html; charset=utf-8",
[]byte("<!DOCTYPE html><html><head></head><body></body></html>"))
})
Init(engine)
zipPath := writePluginZip(t, map[string]string{
"komari-plugin.json": `{"name":"Guard","short":"guard","version":"1.0.0","permissions":{"allowHTMLInject":true,"timeout":5}}`,
"script.js": `
const server = require("server");
function load() {
server.injectHTML("", "<script>window.__guard = 1;</script>");
}
`,
})
if _, err := InstallZip(zipPath); err != nil {
t.Fatal(err)
}
if err := SetEnabled("guard", true, true); err != nil {
t.Fatal(err)
}
defer func() { _ = SetEnabled("guard", false, false) }()
handler := HTMLInjectHandler(engine)
// 仅 Connection: upgrade(反代常见行为)→ 注入照常生效
req := httptest.NewRequest(http.MethodGet, "/admin/dashboard", nil)
req.Header.Set("Connection", "upgrade")
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if !strings.Contains(rec.Body.String(), "__guard") {
t.Fatalf("bare Connection: upgrade must not skip injection, body = %q", rec.Body.String())
}
// 两个头齐全的真实升级 → 放行,不注入
req2 := httptest.NewRequest(http.MethodGet, "/admin/dashboard", nil)
req2.Header.Set("Connection", "upgrade")
req2.Header.Set("Upgrade", "websocket")
rec2 := httptest.NewRecorder()
handler.ServeHTTP(rec2, req2)
if strings.Contains(rec2.Body.String(), "__guard") {
t.Fatalf("real upgrade must skip injection, body = %q", rec2.Body.String())
}
}
+51 -41
View File
@@ -3,8 +3,10 @@ package plugin
import (
"time"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gorilla/websocket"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/web/connection"
)
@@ -15,14 +17,14 @@ import (
// GET. Like the HTTP kinds, the input is lowercased, so any casing works
// from JS.
const (
hookWSConnect hookKind = "wsconnect"
hookWSMessage hookKind = "wsmessage"
hookWSSend hookKind = "wssend"
hookWSClose hookKind = "wsclose"
hookWSConnect hookKind = "wsconnect"
hookWSMessage hookKind = "wsmessage"
hookWSSend hookKind = "wssend"
hookWSClose hookKind = "wsclose"
)
// wsFrameHookTimeout bounds one frame-level callback. The agent read pumps
// run on deadlines (v1: 11s, v2: readWait); a frame hook must never stall
// run on the agent read deadline; a frame hook must never stall
// them, so the wait is cut short and the frame passes through untouched.
const wsFrameHookTimeout = time.Second
@@ -47,21 +49,21 @@ func (m *Manager) OnConnect(info *connection.ConnInfo) (deny bool, reason string
for _, h := range m.wsHooksFor(hookWSConnect, info.Path) {
var denied bool
var why string
queued, timedOut := runHookTurn(h.host, "plugin wsConnect hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin wsConnect hook "+h.short, func(vm *moejs.Runtime) {
_ = h.host.RunJob(vm, "plugin wsConnect hook "+h.short, func() error {
res, err := h.fn(goja.Undefined(), wsHookContext(vm, info))
res, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(wsHookContext(vm, info)))
if err != nil {
return err
}
if goja.IsUndefined(res) || goja.IsNull(res) {
if res.IsUndefined() || res.IsNull() {
return nil
}
obj := res.ToObject(vm)
if v := obj.Get("deny"); v != nil && v.ToBoolean() {
obj := native.Object(vm, res)
if v := native.Get(vm, obj, "deny"); native.Bool(v) {
denied = true
}
if v := obj.Get("reason"); v != nil && !goja.IsUndefined(v) {
why = v.String()
if v := native.Get(vm, obj, "reason"); !v.IsUndefined() {
why = native.String(vm, v)
}
return nil
})
@@ -103,25 +105,25 @@ func (m *Manager) runFrameHooks(kind hookKind, info *connection.ConnInfo, frameT
for _, h := range hooks {
var drop bool
var hookErr error
queued, timedOut := runHookTurnTimeout(h.host, "plugin "+string(kind)+" hook "+h.short, wsFrameHookTimeout, func(vm *goja.Runtime) {
queued, timedOut := runHookTurnTimeout(h.host, "plugin "+string(kind)+" hook "+h.short, wsFrameHookTimeout, func(vm *moejs.Runtime) {
msg := wsFrameObject(vm, info, frameType, next)
hookErr = h.host.RunJob(vm, "plugin "+string(kind)+" hook "+h.short, func() error {
res, err := h.fn(goja.Undefined(), wsHookContext(vm, info), msg)
res, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(wsHookContext(vm, info)), engine.ObjectValue(msg))
if err != nil {
return err
}
if goja.IsUndefined(res) || goja.IsNull(res) {
if res.IsUndefined() || res.IsNull() {
return nil
}
obj := res.ToObject(vm)
if v := obj.Get("drop"); v != nil && v.ToBoolean() {
obj := native.Object(vm, res)
if v := native.Get(vm, obj, "drop"); native.Bool(v) {
drop = true
return nil
}
if v := obj.Get("type"); v != nil && !goja.IsUndefined(v) {
frameType = int(v.ToInteger())
if v := native.Get(vm, obj, "type"); !v.IsUndefined() {
frameType = int(native.Int(vm, v))
}
if v := obj.Get("data"); v != nil && !goja.IsUndefined(v) && !goja.IsNull(v) {
if v := native.Get(vm, obj, "data"); !v.IsUndefined() && !v.IsNull() {
if data := wsFrameBytes(vm, v); data != nil {
next = data
}
@@ -151,9 +153,9 @@ func (m *Manager) runFrameHooks(kind hookKind, info *connection.ConnInfo, frameT
func (m *Manager) OnClose(info *connection.ConnInfo) {
for _, h := range m.wsHooksFor(hookWSClose, info.Path) {
var hookErr error
queued, timedOut := runHookTurn(h.host, "plugin wsClose hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin wsClose hook "+h.short, func(vm *moejs.Runtime) {
hookErr = h.host.RunJob(vm, "plugin wsClose hook "+h.short, func() error {
_, err := h.fn(goja.Undefined(), wsHookContext(vm, info))
_, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(wsHookContext(vm, info)))
return err
})
})
@@ -184,41 +186,41 @@ func (m *Manager) wsHooksFor(kind hookKind, path string) []*hookEntry {
}
// wsHookContext builds the immutable per-connection context object.
func wsHookContext(vm *goja.Runtime, info *connection.ConnInfo) *goja.Object {
ctx := vm.NewObject()
_ = ctx.Set("path", info.Path)
_ = ctx.Set("connId", info.ID)
_ = ctx.Set("remoteIp", info.RemoteIP)
_ = ctx.Set("userAgent", info.UserAgent)
func wsHookContext(vm *moejs.Runtime, info *connection.ConnInfo) *engine.Object {
ctx := vm.Realm().NewObject()
_ = native.Set(vm, ctx, "path", info.Path)
_ = native.Set(vm, ctx, "connId", info.ID)
_ = native.Set(vm, ctx, "remoteIp", info.RemoteIP)
_ = native.Set(vm, ctx, "userAgent", info.UserAgent)
if info.ClientUUID != "" {
_ = ctx.Set("clientUuid", info.ClientUUID)
_ = native.Set(vm, ctx, "clientUuid", info.ClientUUID)
}
return ctx
}
// wsFrameObject builds the frame object handed to wsMessage/wsSend hooks.
// Text frames arrive as strings, binary frames as ArrayBuffers.
func wsFrameObject(vm *goja.Runtime, info *connection.ConnInfo, frameType int, data []byte) *goja.Object {
msg := vm.NewObject()
_ = msg.Set("type", frameType)
func wsFrameObject(vm *moejs.Runtime, info *connection.ConnInfo, frameType int, data []byte) *engine.Object {
msg := vm.Realm().NewObject()
_ = native.Set(vm, msg, "type", frameType)
if frameType == websocket.BinaryMessage {
_ = msg.Set("data", vm.NewArrayBuffer(append([]byte(nil), data...)))
_ = native.Set(vm, msg, "data", newFrameBuffer(vm, data))
} else {
_ = msg.Set("data", string(data))
_ = native.Set(vm, msg, "data", string(data))
}
_ = msg.Set("connId", info.ID)
_ = msg.Set("path", info.Path)
_ = native.Set(vm, msg, "connId", info.ID)
_ = native.Set(vm, msg, "path", info.Path)
return msg
}
// wsFrameBytes converts a hook-returned payload (string or ArrayBuffer) back
// to bytes; anything else reports nil and leaves the frame untouched.
func wsFrameBytes(vm *goja.Runtime, v goja.Value) []byte {
if goja.IsString(v) {
return []byte(v.String())
func wsFrameBytes(vm *moejs.Runtime, v moejs.Value) []byte {
if v.IsString() {
return []byte(native.String(vm, v))
}
var data []byte
if err := vm.ExportTo(v, &data); err == nil {
if err := native.ExportTo(vm, v, &data); err == nil {
return data
}
return nil
@@ -231,3 +233,11 @@ func (m *Manager) logWSHookTimeout(h *hookEntry) {
// The Manager is the process-wide WebSocket frame interceptor.
var _ connection.FrameInterceptor = (*Manager)(nil)
func newFrameBuffer(vm *moejs.Runtime, data []byte) moejs.Value {
obj, err := vm.Realm().NewArrayBuffer(append([]byte(nil), data...))
if err != nil {
panic(err)
}
return engine.ObjectValue(obj)
}
+147
View File
@@ -0,0 +1,147 @@
// Package sampling provides deterministic proportional downsampling used by
// history responses. It is independent of transport and storage.
package sampling
import (
"math"
"sort"
)
// Group is one named set of candidate points.
type Group[K comparable] struct {
Key K
Length int
}
type remainder struct {
idx int
frac float64
}
// AllocateTargets splits maxTotal across groups proportionally.
func AllocateTargets[K comparable](groups []Group[K], maxTotal int) map[K]int {
result := make(map[K]int, len(groups))
if maxTotal < 0 {
for _, g := range groups {
result[g.Key] = g.Length
}
return result
}
total := 0
for _, g := range groups {
total += g.Length
}
if total <= maxTotal {
for _, g := range groups {
result[g.Key] = g.Length
}
return result
}
remainders := make([]remainder, 0, len(groups))
sumTargets := 0
for i, g := range groups {
if g.Length <= 0 {
result[g.Key] = 0
continue
}
raw := float64(g.Length) * float64(maxTotal) / float64(total)
t := int(math.Floor(raw))
if t > g.Length {
t = g.Length
}
result[groups[i].Key] = t
sumTargets += t
remainders = append(remainders, remainder{i, raw - float64(t)})
}
if sumTargets < maxTotal {
need := maxTotal - sumTargets
sortRemainders(remainders, false)
for _, r := range remainders {
if need == 0 {
break
}
g := groups[r.idx]
if result[g.Key] < g.Length {
result[g.Key]++
need--
}
}
for need > 0 {
for _, g := range groups {
if need == 0 {
break
}
if result[g.Key] < g.Length {
result[g.Key]++
need--
}
}
if need > 0 {
break
}
}
} else if sumTargets > maxTotal {
over := sumTargets - maxTotal
sortRemainders(remainders, true)
for _, r := range remainders {
if over == 0 {
break
}
g := groups[r.idx]
if result[g.Key] > 0 {
result[g.Key]--
over--
}
}
for over > 0 {
for _, g := range groups {
if over == 0 {
break
}
if result[g.Key] > 0 {
result[g.Key]--
over--
}
}
if over > 0 {
break
}
}
}
return result
}
func sortRemainders(r []remainder, asc bool) {
sort.Slice(r, func(i, j int) bool {
if asc {
return r[i].frac < r[j].frac
}
return r[i].frac > r[j].frac
})
}
// SampleEvenly selects k points including both endpoints.
func SampleEvenly[T any](in []T, k int) []T {
n := len(in)
if k <= 0 || n == 0 {
return []T{}
}
if k >= n {
return in
}
out := make([]T, 0, k)
if k == 1 {
out = append(out, in[n-1])
return out
}
for i := 0; i < k; i++ {
idx := int(math.Round(float64(i) * float64(n-1) / float64(k-1)))
if idx < 0 {
idx = 0
} else if idx >= n {
idx = n - 1
}
out = append(out, in[idx])
}
return out
}
+7 -3
View File
@@ -5,9 +5,9 @@ import (
"fmt"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/internal/notifications"
"github.com/komari-monitor/komari/utils/geoip"
logger "github.com/komari-monitor/komari/utils/log"
"github.com/komari-monitor/komari/utils/messageSender"
"github.com/komari-monitor/komari/web/oauth"
)
@@ -18,8 +18,12 @@ func (a *App) InitProviders() error {
go geoip.InitGeoIp()
a.addCleanup("geoip", func(context.Context) error { return geoip.Shutdown() })
messageSender.Initialize()
a.addCleanup("message-sender", func(context.Context) error { return messageSender.Shutdown() })
if err := notifications.Initialize(); err != nil {
return fmt.Errorf("initialize notification channels: %w", err)
}
a.addCleanup("notification-channels", func(context.Context) error {
return notifications.Shutdown()
})
return nil
}
+1 -12
View File
@@ -14,7 +14,6 @@ import (
"github.com/komari-monitor/komari/database"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/auditlog"
d_notification "github.com/komari-monitor/komari/database/notification"
"github.com/komari-monitor/komari/database/tasks"
"github.com/komari-monitor/komari/internal/config"
"github.com/komari-monitor/komari/internal/lifecycle"
@@ -23,7 +22,6 @@ import (
"github.com/komari-monitor/komari/internal/scheduler"
"github.com/komari-monitor/komari/utils/geoip"
logger "github.com/komari-monitor/komari/utils/log"
"github.com/komari-monitor/komari/utils/messageSender"
"github.com/komari-monitor/komari/utils/notifier"
"github.com/komari-monitor/komari/web/api"
"github.com/komari-monitor/komari/web/oauth"
@@ -76,11 +74,6 @@ func (a *App) registerReloadHandlers(cors *security.CorsController) {
go geoip.InitGeoIp()
}
})
a.reload.Register("message-sender", func(event config.ConfigEvent) {
if event.IsChanged(config.NotificationMethodKey) {
go messageSender.Initialize()
}
})
a.reload.Register("cors", func(event config.ConfigEvent) { cors.Update(event) })
}
@@ -188,9 +181,6 @@ func registerScheduledWork() {
if err := tasks.ReloadPingSchedule(); err != nil {
logger.ErrorArgs("server", "Failed to reload ping schedule:", err)
}
if err := d_notification.ReloadLoadNotificationSchedule(); err != nil {
logger.ErrorArgs("server", "Failed to reload load notification schedule:", err)
}
if err := scheduler.AddFunc("records:cleanup", "@every 30m", cleanupScheduledData); err != nil {
logger.ErrorArgs("server", "Failed to add cleanup scheduled task:", err)
}
@@ -203,10 +193,9 @@ func registerScheduledWork() {
if err := scheduler.AddFunc("notifier:traffic", "@every 1m", notifier.CheckTraffic); err != nil {
logger.ErrorArgs("server", "Failed to add traffic notification task:", err)
}
if err := scheduler.AddFunc("notifier:expire", "0 0 9 * * *", notifier.CheckExpireScheduledWork); err != nil {
if err := scheduler.AddFunc("notifier:expire", "0 0 9 * * *", notifier.CheckExpire); err != nil {
logger.ErrorArgs("server", "Failed to add expire notification task:", err)
}
notifier.InitTrafficReportSchedule()
}
const taskResultRetentionDays = 30
-21
View File
@@ -1,21 +0,0 @@
package sqlitetune
import (
"testing"
"time"
)
func TestNormalizeAllowsUnlimitedJournalSize(t *testing.T) {
options, err := normalize(Options{
BusyTimeout: 5 * time.Second,
CacheSizeKB: 8,
WALAutoCheckpoint: 1,
JournalSizeLimitBytes: -1,
})
if err != nil {
t.Fatalf("normalize unlimited journal size: %v", err)
}
if options.JournalSizeLimitBytes != -1 {
t.Fatalf("journal size limit = %d, want -1", options.JournalSizeLimitBytes)
}
}
+3 -2
View File
@@ -365,7 +365,7 @@ const value = Buffer.from("komari").toString("hex");
| `domainToASCII/domainToUnicode` | `可用` |
| 旧式 `parse/format/resolve/resolveObject/urlToHttpOptions` | `未实现` |
`URL` 和 `URLSearchParams` 不会自动成为全局变量,应从模块取得:
`URL` 和 `URLSearchParams` 会作为全局变量注入,也可以从模块取得:
```js
const { URL, URLSearchParams } = require("node:url");
@@ -705,7 +705,8 @@ function runtimeInfo() {
| ChildProcess `kill/ref/unref/disconnect/send` | `部分实现` 或调用抛错,见下文 |
支持的 options:`cwd`、`env`、`shell`、`timeout`、`encoding`、`maxBuffer`。
`cwd` 受 `BaseDir` 限制;请求的 timeout 和 maxBuffer 只能收紧 Go 侧全局上限,不能放大。
`cwd` 受 `BaseDir` 限制;子进程默认不继承 runtime 的 `Timeout`,只有显式传入 `timeout` 时才会自动结束。
请求的 `maxBuffer` 只能收紧 Go 侧全局上限,不能放大。
`stdio/input/detached/uid/gid/windowsHide/windowsVerbatimArguments/serialization` 等未实现。
- `spawn()` 的 `stdin` 是 `stream.Writable`(写入经后台写队列串行执行,`end()` 后关闭管道),
+13
View File
@@ -0,0 +1,13 @@
Copyright (c) 2016 Dmitry Panov
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+196
View File
@@ -0,0 +1,196 @@
// Package buffer implements Node buffers using native JavaScript typed arrays.
package buffer
import (
_ "embed"
"encoding/base64"
"encoding/hex"
moe "github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"strings"
"unicode/utf16"
"unicode/utf8"
)
const ModuleName = "buffer"
//go:embed buffer.js
var source string
func codecName(s string) string {
switch strings.ToLower(s) {
case "", "utf8", "utf-8":
return "utf8"
case "hex", "base64", "base64url", "ascii":
return strings.ToLower(s)
case "latin1", "binary":
return "latin1"
case "ucs2", "ucs-2", "utf16le", "utf-16le":
return "utf16le"
}
return ""
}
func decode(s, enc string) []byte {
switch codecName(enc) {
case "hex":
b, _ := hex.DecodeString(s)
return b
case "base64", "base64url":
s = strings.Map(func(r rune) rune {
if r == '-' {
return '+'
}
if r == '_' {
return '/'
}
if r > 127 {
return -1
}
if strings.ContainsRune("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=", r) {
return r
}
return -1
}, s)
s = strings.TrimRight(s, "=")
b, _ := base64.RawStdEncoding.DecodeString(s)
return b
case "latin1", "ascii":
u := utf16.Encode([]rune(s))
b := make([]byte, len(u))
for i, v := range u {
b[i] = byte(v)
}
return b
case "utf16le":
u := utf16.Encode([]rune(s))
b := make([]byte, len(u)*2)
for i, v := range u {
b[2*i] = byte(v)
b[2*i+1] = byte(v >> 8)
}
return b
default:
return []byte(s)
}
}
func encode(b []byte, enc string) string {
switch codecName(enc) {
case "hex":
return hex.EncodeToString(b)
case "base64":
return base64.StdEncoding.EncodeToString(b)
case "base64url":
return base64.RawURLEncoding.EncodeToString(b)
case "latin1", "ascii":
r := make([]rune, len(b))
for i, v := range b {
if codecName(enc) == "ascii" {
v &= 127
}
r[i] = rune(v)
}
return string(r)
case "utf16le":
u := make([]uint16, len(b)/2)
for i := range u {
u[i] = uint16(b[2*i]) | uint16(b[2*i+1])<<8
}
return string(utf16.Decode(u))
default:
if utf8.Valid(b) {
return string(b)
}
var out strings.Builder
for len(b) > 0 {
r, n := utf8.DecodeRune(b)
out.WriteRune(r)
b = b[n:]
}
return out.String()
}
}
// Bytes returns a live view of the bytes of ArrayBuffer, TypedArray and DataView.
func Bytes(vm *moe.Runtime, v moe.Value) []byte {
if v.IsObject() {
if b, ok := v.AsObject().BufferData(); ok {
return b
}
}
if v.IsString() {
return []byte(native.String(vm, v))
}
var b []byte
if err := native.ExportTo(vm, v, &b); err != nil {
panic(vm.Realm().TypeError("Expected a buffer or byte array"))
}
return b
}
func DecodeBytes(vm *moe.Runtime, arg, enc moe.Value) []byte {
if arg.IsString() {
s := "utf8"
if !native.IsNil(enc) {
s = native.String(vm, enc)
}
return decode(native.String(vm, arg), s)
}
return Bytes(vm, arg)
}
// WrapBytes shares data with a native ArrayBuffer; callers must synchronize access.
func WrapBytes(vm *moe.Runtime, data []byte) *engine.Object {
ab, err := vm.Realm().NewArrayBuffer(data)
if err != nil {
panic(err)
}
m := require.Require(vm, ModuleName)
ctor := native.Get(vm, m.AsObject(), "Buffer")
from := native.Get(vm, ctor.AsObject(), "from")
v, err := vm.CallFunction(from, ctor, engine.ObjectValue(ab))
if err != nil {
panic(err)
}
return v.AsObject()
}
func EncodeBytes(vm *moe.Runtime, data []byte, enc moe.Value) moe.Value {
if !native.IsNil(enc) {
s := native.String(vm, enc)
if codecName(s) != "" {
return native.From(vm, encode(data, s))
}
}
return engine.ObjectValue(WrapBytes(vm, data))
}
func Enable(vm *moe.Runtime) {
m := require.Require(vm, ModuleName)
if err := vm.SetGlobal("Buffer", native.Get(vm, m.AsObject(), "Buffer")); err != nil {
panic(err)
}
}
func Require(vm *moe.Runtime, module *engine.Object) {
factory, err := native.RunScript(vm, "buffer.js", source)
if err != nil {
panic(err)
}
enc := native.Bind(vm, "encode", 2, moe.NativeFunc(func(_ *moe.Realm, _ moe.Value, a []moe.Value) (moe.Value, error) {
return native.From(vm, encode(Bytes(vm, moe.Arg(a, 0)), native.String(vm, moe.Arg(a, 1)))), nil
}))
dec := native.Bind(vm, "decode", 2, moe.NativeFunc(func(_ *moe.Realm, _ moe.Value, a []moe.Value) (moe.Value, error) {
b := decode(native.String(vm, moe.Arg(a, 0)), native.String(vm, moe.Arg(a, 1)))
ab, e := vm.Realm().NewArrayBuffer(b)
if e != nil {
return moe.Undefined(), e
}
return engine.ObjectValue(ab), nil
}))
exports, err := vm.CallFunction(factory, moe.Undefined(), enc, dec)
if err != nil {
panic(err)
}
if err = native.Set(vm, module, "exports", exports); err != nil {
panic(err)
}
}
func init() { require.RegisterCoreModule(ModuleName, Require) }
+45
View File
@@ -0,0 +1,45 @@
(function(encode, decode) {
'use strict';
const encodings = ['utf8','utf-8','hex','base64','base64url','ascii','latin1','binary','ucs2','ucs-2','utf16le','utf-16le'];
function encoding(e) { e = e === undefined ? 'utf8' : String(e).toLowerCase(); if (!encodings.includes(e)) throw new TypeError('Unknown encoding: '+e); return e; }
function size(n) { if (typeof n !== 'number') throw new TypeError('size must be a number'); if (!Number.isInteger(n)||n<0||n>0x7fffffff) throw new RangeError('Invalid buffer size'); return n; }
function bounds(b,o,n) { if (!Number.isInteger(o)||!Number.isInteger(n)||n<0||o<0||o+n>b.length) throw new RangeError('Offset is outside the bounds of the Buffer'); }
class Buffer extends Uint8Array {
constructor(a,b,c) { if(typeof a==='string') super(decode(a,encoding(b))); else super(a,b,c); }
static from(a,b,c) { if(typeof a==='number'||a==null) throw new TypeError('Invalid Buffer input'); if(typeof a==='string')return new Buffer(decode(a,encoding(b))); if(a instanceof ArrayBuffer) return new Buffer(a,b,c); if(a.type==='Buffer'&&Array.isArray(a.data)) a=a.data; if(a.valueOf){const v=a.valueOf();if(v!==a)return Buffer.from(v,b,c);}if(typeof a.length!=='number'&&!ArrayBuffer.isView(a)&&!a[Symbol.iterator])throw new TypeError('Invalid Buffer input'); return new Buffer(a); }
static alloc(n,fill,enc) {const b=new Buffer(size(n));if(fill!==undefined)b.fill(fill,0,b.length,enc);return b;}
static allocUnsafe(n) {return new Buffer(size(n));}
static allocUnsafeSlow(n) {return Buffer.allocUnsafe(n);}
static isBuffer(b) {return b instanceof Buffer;}
static isEncoding(e) {return typeof e==='string'&&encodings.includes(e.toLowerCase());}
static byteLength(s,e) {if(typeof s!=='string'){if(s instanceof ArrayBuffer||ArrayBuffer.isView(s))return s.byteLength;throw new TypeError('Expected string or buffer');}e=encoding(e);return e==='hex'?Math.floor(s.length/2):decode(s,e).byteLength;}
static concat(list,total) {if(!Array.isArray(list))throw new TypeError('Expected array');if(total===undefined)total=list.reduce((n,b)=>n+b.length,0);const b=Buffer.alloc(total);let o=0;for(const p of list){const n=Math.min(p.length,total-o);if(n<=0)break;b.set(p.subarray(0,n),o);o+=n;}return b;}
static compare(a,b) {for(let i=0;i<Math.min(a.length,b.length);i++)if(a[i]!==b[i])return a[i]<b[i]?-1:1;return a.length===b.length?0:(a.length<b.length?-1:1);}
toString(e,start,end) {e=encoding(e);start=Math.max(0,start===undefined?0:Math.trunc(start));end=Math.min(this.length,end===undefined?this.length:Math.trunc(end));return encode(this.subarray(start,Math.max(start,end)),e);}
toLocaleString() {return this.toString();}
toJSON() {return {type:'Buffer',data:Array.from(this)};}
slice(s,e) {return this.subarray(s,e);}
equals(b) {return Buffer.compare(this,b)===0;}
compare(b,ts,te,ss,se) {return Buffer.compare(this.subarray(ss,se),b.subarray(ts,te));}
copy(target,ts,ss,se) {ts=ts===undefined?0:ts;ss=ss===undefined?0:ss;se=se===undefined?this.length:se;if(ts<0||ss<0||se<0)throw new RangeError('Invalid offset');const n=Math.max(0,Math.min(se-ss,this.length-ss,target.length-ts));if(n>0)target.set(this.subarray(ss,ss+n),ts);return n;}
fill(v,start,end,e) {if(typeof start==='string'){e=start;start=0;}if(typeof end==='string'){e=end;end=this.length;}start=start===undefined?0:start;end=end===undefined?this.length:end;bounds(this,start,Math.max(0,end-start));if(typeof v==='number') {super.fill(v,start,end);return this;}const p=typeof v==='string'?Buffer.from(v,e):v;if(!p||!p.length){if(v===''){super.fill(0,start,end);return this;}throw new TypeError('Invalid fill value');}for(let i=start;i<end;i++)this[i]=p[(i-start)%p.length];return this;}
write(s,o,n,e) {if(typeof o==='string'){e=o;o=0;n=this.length;}o=o===undefined?0:o;if(typeof n==='string'){e=n;n=undefined;}n=n===undefined?this.length-o:n;bounds(this,o,n);let b=Buffer.from(s,e);let count=Math.min(n,b.length);if((e===undefined||/^utf-?8$/i.test(e))&&count<b.length){while(count>0&&(b[count]&192)===128)count--;}this.set(b.subarray(0,count),o);return count;}
indexOf(v,o,e) {return this._find(v,o,e,false);}
lastIndexOf(v,o,e) {return this._find(v,o,e,true);}
includes(v,o,e) {return this.indexOf(v,o,e)!==-1;}
_find(v,o,e,reverse) {if(typeof o==='string'){e=o;o=undefined;}if(typeof v==='number')v=Buffer.from([v]);else if(typeof v==='string')v=Buffer.from(v,e);o=o===undefined?(reverse?this.length:0):Math.trunc(o);if(o<0)o=Math.max(0,this.length+o);if(!v.length)return Math.min(o,this.length);if(reverse){for(let i=Math.min(o,this.length-v.length);i>=0;i--){let j=0;while(j<v.length&&this[i+j]===v[j])j++;if(j===v.length)return i;}}else{for(let i=o;i<=this.length-v.length;i++){let j=0;while(j<v.length&&this[i+j]===v[j])j++;if(j===v.length)return i;}}return -1;}
}
for(const width of [8,16,32,64]) for(const little of [false,true]) for(const signed of [false,true]) {
if(width===8&&little)continue;
const big=width===64, bytes=width/8, suffix=width+(width===8?'':little?'LE':'BE');
const kind=big?(signed?'BigInt':'BigUInt'):(signed?'Int':'UInt');
const dvkind=big?(signed?'BigInt64':'BigUint64'):(signed?'Int':'Uint')+width;
Buffer.prototype['read'+kind+suffix]=function(o){o=o===undefined?0:o;bounds(this,o,bytes);return new DataView(this.buffer,this.byteOffset,this.byteLength)['get'+dvkind](o,little);};
Buffer.prototype['write'+kind+suffix]=function(v,o){o=o===undefined?0:o;bounds(this,o,bytes);if(big&&(typeof v!=='bigint'||v<(signed?-(1n<<63n):0n)||v>(signed?(1n<<63n)-1n:(1n<<64n)-1n)))throw new RangeError('Value out of range');if(!big&&(!Number.isFinite(v)||v<(signed?-(2**(width-1)):0)||v>(signed?2**(width-1)-1:2**width-1)))throw new RangeError('Value out of range');new DataView(this.buffer,this.byteOffset,this.byteLength)['set'+dvkind](o,v,little);return o+bytes;};
if(!signed){Buffer.prototype['read'+(big?'BigUint':'Uint')+suffix]=Buffer.prototype['read'+kind+suffix];Buffer.prototype['write'+(big?'BigUint':'Uint')+suffix]=Buffer.prototype['write'+kind+suffix];}
}
for(const pair of [['Float',32],['Double',64]])for(const little of [false,true]){const name=pair[0]+(little?'LE':'BE'),bytes=pair[1]/8;Buffer.prototype['read'+name]=function(o){o=o===undefined?0:o;bounds(this,o,bytes);return new DataView(this.buffer,this.byteOffset,this.byteLength)['getFloat'+pair[1]](o,little);};Buffer.prototype['write'+name]=function(v,o){o=o===undefined?0:o;bounds(this,o,bytes);new DataView(this.buffer,this.byteOffset,this.byteLength)['setFloat'+pair[1]](o,v,little);return o+bytes;};}
for(const little of [false,true])for(const signed of [false,true]){const name=(signed?'Int':'UInt')+(little?'LE':'BE');Buffer.prototype['read'+name]=function(o,n){if(!Number.isInteger(n)||n<1||n>6)throw new RangeError('Invalid byteLength');bounds(this,o,n);let v=0;for(let i=0;i<n;i++)v=v*256+this[o+(little?n-i-1:i)];if(signed&&v>=2**(n*8-1))v-=2**(n*8);return v;};Buffer.prototype['write'+name]=function(v,o,n){if(!Number.isInteger(n)||n<1||n>6)throw new RangeError('Invalid byteLength');bounds(this,o,n);const min=signed?-(2**(n*8-1)):0,max=signed?2**(n*8-1)-1:2**(n*8)-1;if(!Number.isFinite(v)||v<min||v>max)throw new RangeError('Value out of range');if(v<0)v+=2**(n*8);for(let i=0;i<n;i++){this[o+(little?i:n-i-1)]=v%256;v=Math.floor(v/256);}return o+n;};if(!signed){Buffer.prototype['readUint'+(little?'LE':'BE')]=Buffer.prototype['read'+name];Buffer.prototype['writeUint'+(little?'LE':'BE')]=Buffer.prototype['write'+name];}}
for(const n of [2,4,8])Buffer.prototype['swap'+n*8]=function(){if(this.length%n)throw new RangeError('Buffer size must be a multiple of '+n);for(let i=0;i<this.length;i+=n)for(let j=0;j<n/2;j++){const x=this[i+j];this[i+j]=this[i+n-j-1];this[i+n-j-1]=x;}return this;};
return {Buffer,SlowBuffer:Buffer.alloc,INSPECT_MAX_BYTES:50,kMaxLength:0x7fffffff,constants:{MAX_LENGTH:0x7fffffff,MAX_STRING_LENGTH:0x1fffffe8}};
})
+83
View File
@@ -0,0 +1,83 @@
package buffer
import (
moe "github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"testing"
)
func TestNativeBuffer(t *testing.T) {
vm := moe.NewRuntime(moe.Options{})
require.NewRegistry().Enable(vm)
Enable(vm)
for _, s := range []string{
`Object.getPrototypeOf(Buffer.prototype) === Uint8Array.prototype`,
`Object.isFrozen(Object.prototype)&&Object.isFrozen(Uint8Array.prototype)`,
`(function(){const b=Buffer.alloc(1);return b.copy(Buffer.alloc(0),999)===0&&Buffer.byteLength("zz","hex")===1})()`,
`(function(){for(const f of [()=>Buffer.from({}),()=>Buffer.alloc(4).readUIntLE(0,1.5),()=>Buffer.alloc(8).writeBigUInt64LE(-1n),()=>Buffer.alloc(8).writeUIntLE(NaN,0,2)]){try{f();return false;}catch(e){}}return true;})()`,
`(function(){const a=new Uint8Array([1,2,3]);const b=Buffer.from(a);b[0]=9;const ab=a.buffer;const v=Buffer.from(ab,1,1);v[0]=7;return a[0]===1&&a[1]===7&&v.length===1&&b instanceof Uint8Array})()`,
`Buffer.from("hello").toString() === "hello"`,
`Buffer.from("hello").toString("hex") === "68656c6c6f"`,
`Buffer.from("68656c6c6f", "hex").toString() === "hello"`,
`Buffer.from("通知").toString() === "通知"`,
`Buffer.from("aGVsbG8=", "base64").toString() === "hello"`,
`Buffer.from("hello") instanceof Buffer`,
`Buffer.alloc(4, "ab").toString() === "abab"`,
`Buffer.concat([Buffer.from("a"),Buffer.from("b")]).toString() === "ab"`,
`(function(){const b=Buffer.alloc(16);b.writeUInt32LE(0x12345678,1);b.writeInt16BE(-123,5);b.writeDoubleBE(1.25,8);return b.readUInt32LE(1)===0x12345678&&b.readInt16BE(5)===-123&&b.readDoubleBE(8)===1.25})()`,
`(function(){const b=Buffer.from("abcd");const v=b.slice(1,3);v[0]=120;return b.toString()==="axcd"&&Buffer.isBuffer(v)})()`,
`(function(){const b=Buffer.alloc(8);b.writeBigUInt64LE(123n);return b.readBigUInt64LE()===123n})()`,
} {
v, err := native.Run(vm, s)
if err != nil {
t.Fatalf("%s: %v", s, err)
}
if !native.Bool(v) {
t.Errorf("false: %s", s)
}
}
data := []byte("wrapped")
b := WrapBytes(vm, data)
if err := vm.SetGlobal("wrapped", engine.ObjectValue(b)); err != nil {
t.Fatal(err)
}
if _, err := native.Run(vm, "wrapped[0]=65"); err != nil {
t.Fatal(err)
}
if data[0] != 65 {
t.Fatal("WrapBytes did not share backing storage")
}
view, err := native.Run(vm, "wrapped.subarray(1,3)")
if err != nil {
t.Fatal(err)
}
raw := Bytes(vm, view)
raw[0] = 66
if data[1] != 66 || len(raw) != 2 {
t.Fatal("Bytes did not return a live bounded subarray")
}
for _, source := range []string{"new DataView(wrapped.buffer,1,2)", "new Uint16Array(wrapped.buffer,0,2)", "wrapped.buffer"} {
v, err := native.Run(vm, source)
if err != nil {
t.Fatal(err)
}
b := Bytes(vm, v)
if len(b) == 0 {
t.Fatalf("empty bytes: %s", source)
}
b[0] = 77
offset := 0
if source == "new DataView(wrapped.buffer,1,2)" {
offset = 1
if len(b) != 2 {
t.Fatal("unbounded DataView")
}
}
if data[offset] != 77 {
t.Fatalf("bytes not live: %s", source)
}
}
}
+231 -220
View File
@@ -6,6 +6,15 @@ import (
_ "embed"
"errors"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/fs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/writequeue"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"io"
"os"
"os/exec"
@@ -13,14 +22,6 @@ import (
"strings"
"sync"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/fs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/writequeue"
)
type childCommandOptions struct {
@@ -31,7 +32,6 @@ type childCommandOptions struct {
encoding string
maxBuffer int
}
type childOutputLimitError struct {
stream string
}
@@ -65,9 +65,12 @@ func (b *childOutputBuffer) Write(data []byte) (int, error) {
b.once.Do(b.cancel)
return len(data), nil
}
func (b *childOutputBuffer) Len() int { return b.buffer.Len() }
func (b *childOutputBuffer) Bytes() []byte { return b.buffer.Bytes() }
func (b *childOutputBuffer) Len() int {
return b.buffer.Len()
}
func (b *childOutputBuffer) Bytes() []byte {
return b.buffer.Bytes()
}
type Module struct {
runtime *bridge.Runtime
@@ -79,134 +82,131 @@ type Module struct {
func New(runtime *bridge.Runtime, filesystem *fs.Module, allowExec bool, maxOutput int) *Module {
return &Module{runtime: runtime, filesystem: filesystem, allowExec: allowExec, maxOutput: maxOutput}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
if !m.allowExec {
panic(vm.NewGoError(fmt.Errorf("child_process requires AllowExec")))
panic(native.GoError(vm, fmt.Errorf("child_process requires AllowExec")))
}
exports := vm.NewObject()
_ = exports.Set("spawn", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, false)
return m.spawnChild(vm, command, arguments, options)
})
_ = exports.Set("exec", func(call goja.FunctionCall) goja.Value {
command := call.Argument(0).String()
optionsValue, callbackValue := call.Argument(1), call.Argument(2)
if _, ok := goja.AssertFunction(optionsValue); ok {
callbackValue, optionsValue = optionsValue, goja.Undefined()
exports := vm.Realm().NewObject()
_ = native.Set(vm, exports, "spawn", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, false)
return native.From(vm, m.spawnChild(vm, command, arguments, options)), nil
}))
_ = native.Set(vm, exports, "exec", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command := native.String(vm, moejs.Arg(args, 0))
optionsValue, callbackValue := moejs.Arg(args, 1), moejs.Arg(args, 2)
if _, ok := checkedFunction(optionsValue); ok {
callbackValue, optionsValue = optionsValue, moejs.Undefined()
}
callback, ok := goja.AssertFunction(callbackValue)
callback, ok := checkedFunction(callbackValue)
if !ok {
panic(vm.NewTypeError("callback must be a function"))
panic(native.TypeError(vm, "callback must be a function"))
}
options := m.childOptions(vm, optionsValue)
options.shell = true
return m.execChild(vm, command, nil, options, callback)
})
_ = exports.Set("execFile", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, true)
return native.From(vm, m.execChild(vm, command, nil, options, callback)), nil
}))
_ = native.Set(vm, exports, "execFile", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, true)
callbackIndex := 1
if _, ok := call.Argument(1).Export().([]any); ok {
if _, ok := native.Export(vm, moejs.Arg(args, 1)).([]any); ok {
callbackIndex = 2
}
callbackValue := call.Argument(callbackIndex)
if _, ok := goja.AssertFunction(callbackValue); !ok {
callbackValue = call.Argument(callbackIndex + 1)
callbackValue := moejs.Arg(args, callbackIndex)
if _, ok := checkedFunction(callbackValue); !ok {
callbackValue = moejs.Arg(args, callbackIndex+1)
}
callback, ok := goja.AssertFunction(callbackValue)
callback, ok := checkedFunction(callbackValue)
if !ok {
panic(vm.NewTypeError("callback must be a function"))
panic(native.TypeError(vm, "callback must be a function"))
}
return m.execChild(vm, command, arguments, options, callback)
})
_ = exports.Set("spawnSync", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, false)
return m.spawnChildSync(vm, command, arguments, options)
})
_ = exports.Set("execFileSync", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, false)
return native.From(vm, m.execChild(vm, command, arguments, options, callback)), nil
}))
_ = native.Set(vm, exports, "spawnSync", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, false)
return native.From(vm, m.spawnChildSync(vm, command, arguments, options)), nil
}))
_ = native.Set(vm, exports, "execFileSync", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, false)
result := m.runChildSync(command, arguments, options)
if result.err != nil {
panic(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
panic(engine.ObjectValue(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr)))
}
return encodeChildOutput(vm, result.stdout, options.encoding)
})
_ = exports.Set("execSync", func(call goja.FunctionCall) goja.Value {
options := m.childOptions(vm, call.Argument(1))
return native.From(vm, encodeChildOutput(vm, result.stdout, options.encoding)), nil
}))
_ = native.Set(vm, exports, "execSync", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
options := m.childOptions(vm, moejs.Arg(args, 1))
options.shell = true
result := m.runChildSync(call.Argument(0).String(), nil, options)
result := m.runChildSync(native.String(vm, moejs.Arg(args, 0)), nil, options)
if result.err != nil {
panic(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
panic(engine.ObjectValue(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr)))
}
return encodeChildOutput(vm, result.stdout, options.encoding)
})
_ = exports.Set("fork", func() {
panic(vm.NewGoError(fmt.Errorf("child_process.fork is unavailable because this runtime is not a Node executable")))
})
_ = module.Set("exports", exports)
return native.From(vm, encodeChildOutput(vm, result.stdout, options.encoding)), nil
}))
_ = native.Set(vm, exports, "fork", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("child_process.fork is unavailable because this runtime is not a Node executable")))
}))
_ = native.Set(vm, module, "exports", exports)
}
func (m *Module) childCommandArguments(vm *goja.Runtime, call goja.FunctionCall, callbackExpected bool) (string, []string, childCommandOptions) {
command := call.Argument(0).String()
func (m *Module) childCommandArguments(vm *moejs.Runtime, args []moejs.Value, callbackExpected bool) (string, []string, childCommandOptions) {
command := native.String(vm, moejs.Arg(args, 0))
arguments := []string{}
optionsIndex := 1
if value := call.Argument(1); !goja.IsUndefined(value) && !goja.IsNull(value) {
if value := moejs.Arg(args, 1); !value.IsUndefined() && !value.IsNull() {
var exported []string
if err := vm.ExportTo(value, &exported); err == nil {
if err := native.ExportTo(vm, value, &exported); err == nil {
arguments = exported
optionsIndex = 2
}
}
options := m.childOptions(vm, call.Argument(optionsIndex))
options := m.childOptions(vm, moejs.Arg(args, optionsIndex))
_ = callbackExpected
return command, arguments, options
}
func (m *Module) childOptions(vm *goja.Runtime, value goja.Value) childCommandOptions {
options := childCommandOptions{cwd: m.filesystem.Cwd(), env: os.Environ(), timeout: m.runtime.Timeout(), maxBuffer: m.maxOutput}
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func (m *Module) childOptions(vm *moejs.Runtime, value moejs.Value) childCommandOptions {
options := childCommandOptions{cwd: m.filesystem.Cwd(), env: os.Environ(), maxBuffer: m.maxOutput}
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return options
}
object := value.ToObject(vm)
if cwd := object.Get("cwd"); cwd != nil && !goja.IsUndefined(cwd) && !goja.IsNull(cwd) {
resolved, err := m.filesystem.Resolve(cwd.String(), false)
object := native.Object(vm, value)
if cwd := native.Get(vm, object, "cwd"); !native.IsNil(cwd) && !cwd.IsUndefined() && !cwd.IsNull() {
resolved, err := m.filesystem.Resolve(native.String(vm, cwd), false)
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
options.cwd = resolved
}
if environment := object.Get("env"); environment != nil && !goja.IsUndefined(environment) && !goja.IsNull(environment) {
if environment := native.Get(vm, object, "env"); !native.IsNil(environment) && !environment.IsUndefined() && !environment.IsNull() {
var values map[string]string
if err := vm.ExportTo(environment, &values); err != nil {
panic(vm.NewTypeError("env must be an object"))
if err := native.ExportTo(vm, environment, &values); err != nil {
panic(native.TypeError(vm, "env must be an object"))
}
options.env = make([]string, 0, len(values))
for name, value := range values {
options.env = append(options.env, name+"="+value)
}
}
if shell := object.Get("shell"); shell != nil {
options.shell = shell.ToBoolean()
if shell := native.Get(vm, object, "shell"); !native.IsNil(shell) {
options.shell = native.Bool(shell)
}
if timeoutValue := object.Get("timeout"); timeoutValue != nil && timeoutValue.ToInteger() > 0 {
timeout := timeoutValue.ToInteger()
options.timeout = min(options.timeout, time.Duration(timeout)*time.Millisecond)
if timeoutValue := native.Get(vm, object, "timeout"); !native.IsNil(timeoutValue) && native.Int(vm, timeoutValue) > 0 {
timeout := native.Int(vm, timeoutValue)
options.timeout = time.Duration(timeout) * time.Millisecond
}
if encoding := object.Get("encoding"); encoding != nil && !goja.IsUndefined(encoding) && !goja.IsNull(encoding) {
options.encoding = encoding.String()
if encoding := native.Get(vm, object, "encoding"); !native.IsNil(encoding) && !encoding.IsUndefined() && !encoding.IsNull() {
options.encoding = native.String(vm, encoding)
}
if maxBuffer := object.Get("maxBuffer"); maxBuffer != nil {
requested := maxBuffer.ToInteger()
if maxBuffer := native.Get(vm, object, "maxBuffer"); !native.IsNil(maxBuffer) {
requested := native.Int(vm, maxBuffer)
if requested > 0 && requested < int64(options.maxBuffer) {
options.maxBuffer = int(requested)
}
}
return options
}
func childExecCommand(command string, arguments []string, options childCommandOptions) (*exec.Cmd, context.CancelFunc) {
ctx := context.Background()
cancel := func() {}
cancel := func() {
}
if options.timeout > 0 {
ctx, cancel = context.WithTimeout(ctx, options.timeout)
}
@@ -222,15 +222,13 @@ func childExecCommand(command string, arguments []string, options childCommandOp
}
return exec.CommandContext(ctx, command, arguments...), cancel
}
func (m *Module) childOutputLimit(options childCommandOptions) int {
if options.maxBuffer > 0 && options.maxBuffer < m.maxOutput {
return options.maxBuffer
}
return m.maxOutput
}
func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string, options childCommandOptions) *goja.Object {
func (m *Module) spawnChild(vm *moejs.Runtime, command string, arguments []string, options childCommandOptions) *engine.Object {
child := events.NewEmitter(vm)
cmd, cancel := childExecCommand(command, arguments, options)
cmd.Dir = options.cwd
@@ -239,40 +237,45 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
stdoutReader, stdoutErr := cmd.StdoutPipe()
stderrReader, stderrErr := cmd.StderrPipe()
if err := errors.Join(stdinErr, stdoutErr, stderrErr); err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
stdinStream, stdout, stderr := m.childStreams(vm, stdin)
_ = child.Set("stdout", stdout)
_ = child.Set("stderr", stderr)
_ = child.Set("stdin", stdinStream)
_ = child.Set("stdio", []any{stdinStream, stdout, stderr})
_ = child.Set("pid", goja.Undefined())
_ = child.Set("exitCode", goja.Null())
_ = child.Set("signalCode", goja.Null())
_ = child.Set("killed", false)
_ = child.Set("connected", false)
_ = native.Set(vm, child, "stdout", stdout)
_ = native.Set(vm, child, "stderr", stderr)
_ = native.Set(vm, child, "stdin", stdinStream)
_ = native.Set(vm, child, "stdio", []any{stdinStream, stdout, stderr})
_ = native.Set(vm, child, "pid", moejs.Undefined())
_ = native.Set(vm, child, "exitCode", moejs.Null())
_ = native.Set(vm, child, "signalCode", moejs.Null())
_ = native.Set(vm, child, "killed", false)
_ = native.Set(vm, child, "connected", false)
if err := cmd.Start(); err != nil {
cancel()
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.spawn", func() error {
return events.Emit(vm, child, "error", vm.NewGoError(err))
return events.Emit(vm, child, "error", native.GoError(vm, err))
})
}, 0)
return child
}
_ = child.Set("pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() { _ = cmd.Process.Kill(); cancel() })
_ = native.Set(vm, child, "pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() {
_ = cmd.Process.Kill()
cancel()
})
if resourceID == 0 {
_ = stdin.Close()
_ = stdoutReader.Close()
_ = stderrReader.Close()
go func() { _ = cmd.Wait() }()
go func() {
_ = cmd.Wait()
}()
return child
}
_ = child.Set("kill", func(signal string) bool {
_ = native.Set(vm, child, "kill", native.Bind(vm, "", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
signal := native.String(vm, moejs.Arg(args, 0))
if cmd.Process == nil {
return false
return native.From(vm, false), nil
}
var err error
if signal == "SIGKILL" {
@@ -281,24 +284,26 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
err = cmd.Process.Signal(os.Interrupt)
}
if err == nil {
_ = child.Set("killed", true)
_ = native.Set(vm, child, "killed", true)
}
return err == nil
})
_ = child.Set("ref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("child_process.ChildProcess.ref is not supported by jsruntime; the event loop is host-driven")))
})
_ = child.Set("unref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("child_process.ChildProcess.unref is not supported by jsruntime; the event loop is host-driven")))
})
_ = child.Set("disconnect", func() { _ = events.Emit(vm, child, "disconnect") })
_ = child.Set("send", func(call goja.FunctionCall) goja.Value {
if callback, ok := goja.AssertFunction(call.Argument(len(call.Arguments) - 1)); ok {
_, _ = callback(goja.Undefined(), vm.NewGoError(fmt.Errorf("IPC is not enabled")))
return native.From(vm, err == nil), nil
}))
_ = native.Set(vm, child, "ref", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("child_process.ChildProcess.ref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, child, "unref", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("child_process.ChildProcess.unref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, child, "disconnect", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
_ = events.Emit(vm, child, "disconnect")
return moejs.Undefined(), nil
}))
_ = native.Set(vm, child, "send", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if callback, ok := checkedFunction(moejs.Arg(args, len(args)-1)); ok {
_, _ = native.Call(vm, callback, moejs.Undefined(), native.GoError(vm, fmt.Errorf("IPC is not enabled")))
}
return vm.ToValue(false)
})
return native.From(vm, false), nil
}))
m.pipeChildOutput(vm, stdoutReader, stdout, options.encoding)
m.pipeChildOutput(vm, stderrReader, stderr, options.encoding)
go func() {
@@ -313,13 +318,13 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
exitCode = -1
}
}
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process", func() error {
_ = child.Set("exitCode", exitCode)
if emitErr := events.Emit(vm, child, "exit", exitCode, goja.Null()); emitErr != nil {
_ = native.Set(vm, child, "exitCode", exitCode)
if emitErr := events.Emit(vm, child, "exit", exitCode, moejs.Null()); emitErr != nil {
return emitErr
}
return events.Emit(vm, child, "close", exitCode, goja.Null())
return events.Emit(vm, child, "close", exitCode, moejs.Null())
})
})
}()
@@ -329,96 +334,99 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
//go:embed stdio.js
var childStdioSource string
// childStreams creates the Writable stdin and Readable stdout/stderr streams
// for one spawned child process. The Go side owns the underlying pipes and
// serializes writes through a per-child queue.
func (m *Module) childStreams(vm *goja.Runtime, stdin io.WriteCloser) (*goja.Object, *goja.Object, *goja.Object) {
factoryValue, err := vm.RunString(childStdioSource)
func (m *Module) childStreams(vm *moejs.Runtime, stdin io.WriteCloser) (*engine.Object, *engine.Object, *engine.Object) {
factoryValue, err := native.Run(vm, childStdioSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load child_process stdio: %w", err)))
panic(native.GoError(vm, fmt.Errorf("load child_process stdio: %w", err)))
}
factory, _ := goja.AssertFunction(factoryValue)
hooks := vm.NewObject()
factory, _ := checkedFunction(factoryValue)
hooks := vm.Realm().NewObject()
var writes writequeue.Queue
_ = hooks.Set("stdinWrite", func(chunk goja.Value, callback goja.Callable) {
_ = native.Set(vm, hooks, "stdinWrite", native.Bind(vm, "", 2, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
chunk := moejs.Arg(args, 0)
callback := moejs.Arg(args, 1)
data := append([]byte(nil), buffer.Bytes(vm, chunk)...)
writes.Submit(func() {
_, err := stdin.Write(data)
if callback == nil {
if native.IsNil(callback) {
return
}
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.stdin.write", func() error {
if err != nil {
_, callbackErr := callback(goja.Undefined(), vm.NewGoError(err))
_, callbackErr := native.Call(vm, callback, moejs.Undefined(), native.GoError(vm, err))
return callbackErr
}
_, callbackErr := callback(goja.Undefined())
_, callbackErr := native.Call(vm, callback, moejs.Undefined())
return callbackErr
})
})
})
})
_ = hooks.Set("stdinEnd", func(callback goja.Callable) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "stdinEnd", native.Bind(vm, "", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
callback := moejs.Arg(args, 0)
writes.Submit(func() {
err := stdin.Close()
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.stdin.end", func() error {
if callback != nil {
if !native.IsNil(callback) {
if err != nil {
_, callbackErr := callback(goja.Undefined(), vm.NewGoError(err))
_, callbackErr := native.Call(vm, callback, moejs.Undefined(), native.GoError(vm, err))
return callbackErr
}
_, callbackErr := callback(goja.Undefined())
_, callbackErr := native.Call(vm, callback, moejs.Undefined())
return callbackErr
}
return nil
})
})
})
})
_ = hooks.Set("stdinDestroy", func() {
go func() { _ = stdin.Close() }()
})
streams, err := factory(goja.Undefined(), require.Require(vm, "stream"), hooks)
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "stdinDestroy", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
go func() {
_ = stdin.Close()
}()
return moejs.Undefined(), nil
}))
streams, err := native.Call(vm, factory, moejs.Undefined(), require.Require(vm, "stream"), engine.ObjectValue(hooks))
if err != nil {
panic(err)
}
streamsObject := streams.ToObject(vm)
createStdin, _ := goja.AssertFunction(streamsObject.Get("createStdin"))
createOutput, _ := goja.AssertFunction(streamsObject.Get("createOutput"))
if createStdin == nil || createOutput == nil {
panic(vm.NewGoError(fmt.Errorf("child_process stdio factory is incomplete")))
streamsObject := native.Object(vm, streams)
createStdin, _ := checkedFunction(native.Get(vm, streamsObject, "createStdin"))
createOutput, _ := checkedFunction(native.Get(vm, streamsObject, "createOutput"))
if native.IsNil(createStdin) || native.IsNil(createOutput) {
panic(native.GoError(vm, fmt.Errorf("child_process stdio factory is incomplete")))
}
stdinValue, err := createStdin(goja.Undefined())
stdinValue, err := native.Call(vm, createStdin, moejs.Undefined())
if err != nil {
panic(err)
}
stdoutValue, err := createOutput(goja.Undefined())
stdoutValue, err := native.Call(vm, createOutput, moejs.Undefined())
if err != nil {
panic(err)
}
stderrValue, err := createOutput(goja.Undefined())
stderrValue, err := native.Call(vm, createOutput, moejs.Undefined())
if err != nil {
panic(err)
}
return stdinValue.ToObject(vm), stdoutValue.ToObject(vm), stderrValue.ToObject(vm)
return native.Object(vm, stdinValue), native.Object(vm, stdoutValue), native.Object(vm, stderrValue)
}
func childCallback(call goja.FunctionCall) goja.Callable {
for index := len(call.Arguments) - 1; index >= 0; index-- {
if callback, ok := goja.AssertFunction(call.Arguments[index]); ok {
func childCallback(args []moejs.Value) moejs.Value {
for index := len(args) - 1; index >= 0; index-- {
if callback, ok := checkedFunction(args[index]); ok {
return callback
}
}
return nil
return moejs.Undefined()
}
func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goja.Object, encoding string) {
push, _ := goja.AssertFunction(stream.Get("push"))
setEncoding, _ := goja.AssertFunction(stream.Get("setEncoding"))
if encoding != "" && setEncoding != nil {
_, _ = setEncoding(stream, vm.ToValue(encoding))
func (m *Module) pipeChildOutput(vm *moejs.Runtime, reader io.Reader, stream *engine.Object, encoding string) {
push, _ := checkedFunction(native.Get(vm, stream, "push"))
setEncoding, _ := checkedFunction(native.Get(vm, stream, "setEncoding"))
if encoding != "" && !native.IsNil(setEncoding) {
_, _ = native.Call(vm, setEncoding, engine.ObjectValue(stream), native.From(vm, encoding))
}
go func() {
data := make([]byte, 32*1024)
@@ -426,11 +434,12 @@ func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goj
count, err := reader.Read(data)
if count > 0 {
chunk := append([]byte(nil), data[:count]...)
delivered := make(chan struct{})
if !m.runtime.RunOnLoop(func(vm *goja.Runtime) {
delivered := make(chan struct {
})
if !m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
defer close(delivered)
_ = m.runtime.RunJob(vm, "child_process stream", func() error {
_, pushErr := push(stream, buffer.WrapBytes(vm, chunk))
_, pushErr := native.Call(vm, push, engine.ObjectValue(stream), engine.ObjectValue(buffer.WrapBytes(vm, chunk)))
return pushErr
})
}) {
@@ -439,9 +448,9 @@ func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goj
<-delivered
}
if err != nil {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process stream close", func() error {
_, pushErr := push(stream, goja.Null())
_, pushErr := native.Call(vm, push, engine.ObjectValue(stream), moejs.Null())
return pushErr
})
})
@@ -450,11 +459,10 @@ func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goj
}
}()
}
func (m *Module) execChild(vm *goja.Runtime, command string, arguments []string, options childCommandOptions, callback goja.Callable) *goja.Object {
func (m *Module) execChild(vm *moejs.Runtime, command string, arguments []string, options childCommandOptions, callback moejs.Value) *engine.Object {
child := events.NewEmitter(vm)
_ = child.Set("pid", goja.Undefined())
_ = child.Set("killed", false)
_ = native.Set(vm, child, "pid", moejs.Undefined())
_ = native.Set(vm, child, "killed", false)
cmd, cancel := childExecCommand(command, arguments, options)
cmd.Dir, cmd.Env = options.cwd, options.env
outputLimit := m.childOutputLimit(options)
@@ -463,29 +471,34 @@ func (m *Module) execChild(vm *goja.Runtime, command string, arguments []string,
cmd.Stdout, cmd.Stderr = stdout, stderr
if err := cmd.Start(); err != nil {
cancel()
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.exec", func() error {
if _, callbackErr := callback(goja.Undefined(), childProcessError(vm, err, -1, nil, nil), goja.Undefined(), goja.Undefined()); callbackErr != nil {
if _, callbackErr := native.Call(vm, callback, moejs.Undefined(), engine.ObjectValue(childProcessError(vm, err, -1, nil, nil)), moejs.Undefined(), moejs.Undefined()); callbackErr != nil {
return callbackErr
}
return events.Emit(vm, child, "error", vm.NewGoError(err))
return events.Emit(vm, child, "error", native.GoError(vm, err))
})
}, 0)
return child
}
_ = child.Set("pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() { _ = cmd.Process.Kill(); cancel() })
_ = native.Set(vm, child, "pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() {
_ = cmd.Process.Kill()
cancel()
})
if resourceID == 0 {
go func() { _ = cmd.Wait() }()
go func() {
_ = cmd.Wait()
}()
return child
}
_ = child.Set("kill", func() bool {
_ = native.Set(vm, child, "kill", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
err := cmd.Process.Kill()
if err == nil {
_ = child.Set("killed", true)
_ = native.Set(vm, child, "killed", true)
}
return err == nil
})
return native.From(vm, err == nil), nil
}))
go func() {
err := cmd.Wait()
err = errors.Join(err, stdout.err, stderr.err)
@@ -498,20 +511,20 @@ func (m *Module) execChild(vm *goja.Runtime, command string, arguments []string,
} else if err != nil {
exitCode = -1
}
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.exec", func() error {
stdoutValue := encodeChildOutput(vm, stdout.Bytes(), options.encoding)
stderrValue := encodeChildOutput(vm, stderr.Bytes(), options.encoding)
var errorValue goja.Value = goja.Null()
var errorValue moejs.Value = moejs.Null()
if err != nil {
errorValue = childProcessError(vm, err, exitCode, stdout.Bytes(), stderr.Bytes())
errorValue = engine.ObjectValue(childProcessError(vm, err, exitCode, stdout.Bytes(), stderr.Bytes()))
}
if _, callbackErr := callback(goja.Undefined(), errorValue, stdoutValue, stderrValue); callbackErr != nil {
if _, callbackErr := native.Call(vm, callback, moejs.Undefined(), errorValue, stdoutValue, stderrValue); callbackErr != nil {
return callbackErr
}
_ = child.Set("exitCode", exitCode)
_ = events.Emit(vm, child, "exit", exitCode, goja.Null())
return events.Emit(vm, child, "close", exitCode, goja.Null())
_ = native.Set(vm, child, "exitCode", exitCode)
_ = events.Emit(vm, child, "exit", exitCode, moejs.Null())
return events.Emit(vm, child, "close", exitCode, moejs.Null())
})
})
}()
@@ -548,40 +561,38 @@ func (m *Module) runChildSync(command string, arguments []string, options childC
}
return result
}
func (m *Module) spawnChildSync(vm *goja.Runtime, command string, arguments []string, options childCommandOptions) *goja.Object {
func (m *Module) spawnChildSync(vm *moejs.Runtime, command string, arguments []string, options childCommandOptions) *engine.Object {
result := m.runChildSync(command, arguments, options)
object := vm.NewObject()
_ = object.Set("pid", result.pid)
_ = object.Set("status", result.exitCode)
_ = object.Set("signal", goja.Null())
_ = object.Set("stdout", encodeChildOutput(vm, result.stdout, options.encoding))
_ = object.Set("stderr", encodeChildOutput(vm, result.stderr, options.encoding))
_ = object.Set("output", []any{goja.Null(), encodeChildOutput(vm, result.stdout, options.encoding), encodeChildOutput(vm, result.stderr, options.encoding)})
object := vm.Realm().NewObject()
_ = native.Set(vm, object, "pid", result.pid)
_ = native.Set(vm, object, "status", result.exitCode)
_ = native.Set(vm, object, "signal", moejs.Null())
_ = native.Set(vm, object, "stdout", encodeChildOutput(vm, result.stdout, options.encoding))
_ = native.Set(vm, object, "stderr", encodeChildOutput(vm, result.stderr, options.encoding))
_ = native.Set(vm, object, "output", []any{moejs.Null(), encodeChildOutput(vm, result.stdout, options.encoding), encodeChildOutput(vm, result.stderr, options.encoding)})
if result.err != nil {
_ = object.Set("error", childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
_ = native.Set(vm, object, "error", childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
}
return object
}
func encodeChildOutput(vm *goja.Runtime, data []byte, encoding string) goja.Value {
func encodeChildOutput(vm *moejs.Runtime, data []byte, encoding string) moejs.Value {
if encoding == "" || encoding == "buffer" {
return buffer.WrapBytes(vm, append([]byte(nil), data...))
return engine.ObjectValue(buffer.WrapBytes(vm, append([]byte(nil), data...)))
}
return buffer.EncodeBytes(vm, data, vm.ToValue(encoding))
return buffer.EncodeBytes(vm, data, native.From(vm, encoding))
}
func childProcessError(vm *goja.Runtime, err error, exitCode int, stdout, stderr []byte) *goja.Object {
object := vm.NewGoError(err)
func childProcessError(vm *moejs.Runtime, err error, exitCode int, stdout, stderr []byte) *engine.Object {
object := native.Object(vm, native.GoError(vm, err))
var limitError *childOutputLimitError
if errors.As(err, &limitError) {
_ = object.Set("code", "ERR_CHILD_PROCESS_STDIO_MAXBUFFER")
_ = native.Set(vm, object, "code", "ERR_CHILD_PROCESS_STDIO_MAXBUFFER")
} else {
_ = object.Set("code", exitCode)
_ = native.Set(vm, object, "code", exitCode)
}
_ = object.Set("killed", false)
_ = object.Set("signal", goja.Null())
_ = object.Set("stdout", encodeChildOutput(vm, stdout, ""))
_ = object.Set("stderr", encodeChildOutput(vm, stderr, ""))
_ = native.Set(vm, object, "killed", false)
_ = native.Set(vm, object, "signal", moejs.Null())
_ = native.Set(vm, object, "stdout", encodeChildOutput(vm, stdout, ""))
_ = native.Set(vm, object, "stderr", encodeChildOutput(vm, stderr, ""))
return object
}
func checkedFunction(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
+76 -95
View File
@@ -1,16 +1,14 @@
package console
// JavaScript compatibility: console.assert(), debug(), error(), exception(),
// info(), log(), trace(), and warn(). Supports basic %s/%d/%i/%f/%o/%O/%c/%%
// substitutions. assert(false, ...) and trace() include a stack trace.
import (
"encoding/json"
"fmt"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
logger "github.com/komari-monitor/komari/utils/log"
"io"
"strings"
"github.com/dop251/goja"
logger "github.com/komari-monitor/komari/utils/log"
)
type level uint8
@@ -22,81 +20,78 @@ const (
errorLevel
)
type Module struct {
output io.Writer
}
type Module struct{ output io.Writer }
func New(output io.Writer) *Module {
return &Module{output: output}
}
func (m *Module) Inject(vm *goja.Runtime) error {
console := vm.NewObject()
console.Set("assert", func(call goja.FunctionCall) goja.Value {
if len(call.Arguments) > 0 && call.Argument(0).ToBoolean() {
return goja.Undefined()
func New(w io.Writer) *Module { return &Module{output: w} }
func (m *Module) Inject(vm *moe.Runtime) error {
o := vm.Realm().NewObject()
for _, entry := range []struct {
name string
level level
stack bool
}{{"debug", debug, false}, {"info", info, false}, {"log", info, false}, {"warn", warn, false}, {"error", errorLevel, false}, {"exception", errorLevel, false}, {"trace", debug, true}} {
entry := entry
if err := native.Function(vm, o, entry.name, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
m.write(vm, entry.level, args, entry.stack)
return moe.Undefined(), nil
}); err != nil {
return err
}
values := call.Arguments
if len(values) <= 1 {
values = []goja.Value{vm.ToValue("Assertion failed")}
} else {
values = append([]goja.Value{vm.ToValue("Assertion failed:")}, values[1:]...)
}
if err := native.Function(vm, o, "assert", func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
if len(args) > 0 && native.Bool(args[0]) {
return moe.Undefined(), nil
}
values := []moe.Value{moe.String("Assertion failed:")}
if len(args) > 1 {
values = append(values, args[1:]...)
}
m.write(vm, errorLevel, values, true)
return goja.Undefined()
})
console.Set("debug", func(call goja.FunctionCall) goja.Value {
m.write(vm, debug, call.Arguments, false)
return goja.Undefined()
})
error := func(call goja.FunctionCall) goja.Value {
m.write(vm, errorLevel, call.Arguments, false)
return goja.Undefined()
return moe.Undefined(), nil
}); err != nil {
return err
}
console.Set("error", error)
console.Set("exception", error)
console.Set("info", func(call goja.FunctionCall) goja.Value {
m.write(vm, info, call.Arguments, false)
return goja.Undefined()
})
console.Set("log", func(call goja.FunctionCall) goja.Value {
m.write(vm, info, call.Arguments, false)
return goja.Undefined()
})
console.Set("trace", func(call goja.FunctionCall) goja.Value {
m.write(vm, debug, call.Arguments, true)
return goja.Undefined()
})
console.Set("warn", func(call goja.FunctionCall) goja.Value {
m.write(vm, warn, call.Arguments, false)
return goja.Undefined()
})
return vm.Set("console", console)
return vm.SetGlobal("console", e.ObjectValue(o))
}
func (m *Module) WriteError(vm *goja.Runtime, values []goja.Value, withStack bool) {
m.write(vm, errorLevel, values, withStack)
func (m *Module) WriteError(vm *moe.Runtime, values []moe.Value, stack bool) {
m.write(vm, errorLevel, values, stack)
}
func (m *Module) Report(vm *goja.Runtime, name string, err error) {
m.WriteError(vm, []goja.Value{vm.ToValue(fmt.Sprintf("%s callback failed: %v", name, err))}, false)
func (m *Module) Report(vm *moe.Runtime, name string, err error) {
m.WriteError(vm, []moe.Value{moe.String(fmt.Sprintf("%s callback failed: %v", name, err))}, false)
}
func (m *Module) write(vm *goja.Runtime, messageLevel level, values []goja.Value, withStack bool) {
message := formatConsoleValues(values)
if withStack {
if stack := stackTrace(vm); stack != "" {
if message != "" {
message += "\n"
func (m *Module) write(vm *moe.Runtime, l level, values []moe.Value, withStack bool) {
args := make([]any, len(values))
for i, v := range values {
x, err := vm.ToGo(v)
if err != nil {
x = native.String(vm, v)
}
args[i] = x
}
message := ""
if len(args) > 0 {
if format, ok := args[0].(string); ok && strings.Contains(format, "%") {
message = formatConsoleString(format, args[1:])
} else {
parts := make([]string, len(args))
for i, x := range args {
parts[i] = formatConsoleValue(x)
}
message += stack
message = strings.Join(parts, " ")
}
}
if withStack {
errObj := vm.Realm().NewError(e.KindError, "")
stack := vm.Realm().StackTrace(e.ObjectValue(errObj))
if stack != "" {
message += "\n" + stack
}
}
if m.output != nil {
_, _ = fmt.Fprintln(m.output, message)
return
}
switch messageLevel {
switch l {
case debug:
logger.Debug("JavaScript", message)
case warn:
@@ -107,23 +102,20 @@ func (m *Module) write(vm *goja.Runtime, messageLevel level, values []goja.Value
logger.Info("JavaScript", message)
}
}
func formatConsoleValue(value any) string {
switch value := value.(type) {
case nil:
return "null"
case string:
return value
case error:
return value.Error()
}
func formatConsoleValues(values []goja.Value) string {
if len(values) == 0 {
return ""
if data, err := json.Marshal(value); err == nil {
return string(data)
}
args := make([]any, 0, len(values))
for _, value := range values {
args = append(args, value.Export())
}
if format, ok := args[0].(string); ok && strings.Contains(format, "%") {
return formatConsoleString(format, args[1:])
}
parts := make([]string, 0, len(args))
for _, arg := range args {
parts = append(parts, fmt.Sprint(arg))
}
return strings.Join(parts, " ")
return fmt.Sprint(value)
}
func formatConsoleString(format string, args []any) string {
@@ -154,7 +146,7 @@ func formatConsoleString(format string, args []any) string {
argIndex++
switch format[i] {
case 's', 'o', 'O':
builder.WriteString(fmt.Sprint(arg))
builder.WriteString(formatConsoleValue(arg))
case 'd', 'i':
builder.WriteString(fmt.Sprintf("%d", toConsoleInteger(arg)))
case 'f':
@@ -168,7 +160,7 @@ func formatConsoleString(format string, args []any) string {
if argIndex < len(args) {
remaining := make([]string, 0, len(args)-argIndex)
for _, arg := range args[argIndex:] {
remaining = append(remaining, fmt.Sprint(arg))
remaining = append(remaining, formatConsoleValue(arg))
}
if builder.Len() > 0 {
builder.WriteByte(' ')
@@ -205,14 +197,3 @@ func toConsoleFloat(value any) float64 {
return 0
}
}
func stackTrace(vm *goja.Runtime) string {
if vm == nil {
return ""
}
value, err := vm.RunString("new Error().stack")
if err != nil {
return ""
}
return value.String()
}
File diff suppressed because it is too large. Load diff
+140
View File
@@ -0,0 +1,140 @@
package crypto
import (
"crypto/ed25519"
"crypto/rand"
"crypto/x509"
"encoding/pem"
"strconv"
"testing"
"time"
moe "github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/loop"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
func runNativeCryptoRegression(t *testing.T, script string) {
t.Helper()
results := make(chan string, 8)
l := loop.NewEventLoop(loop.WithOnError(func(err error) { results <- err.Error() }))
t.Cleanup(l.Terminate)
services := bridge.New(l, 10*time.Second)
services.SetErrorReporter(func(_ *moe.Runtime, _ string, err error) { results <- err.Error() })
if err := l.Start(); err != nil {
t.Fatal(err)
}
l.RunOnLoop(func(vm *moe.Runtime) {
buffer.Enable(vm)
module := vm.Realm().NewObject()
New(services).Load(vm, module)
_ = vm.SetGlobal("crypto", native.Get(vm, module, "exports"))
_ = vm.SetGlobal("done", native.Bind(vm, "done", 1, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
results <- native.String(vm, moe.Arg(args, 0))
return moe.Undefined(), nil
}))
if _, err := native.RunScript(vm, "crypto-native-test.js", script); err != nil {
results <- err.Error()
}
})
select {
case result := <-results:
if result != "ok" {
t.Fatal(result)
}
case <-time.After(12 * time.Second):
t.Fatal("native crypto callback was not delivered")
}
}
func TestNativeCryptoRandomIntSyncAndAsync(t *testing.T) {
runNativeCryptoRegression(t, `
const a = crypto.randomInt(1);
const b = crypto.randomInt(-3, -2);
if (typeof a !== "number" || a !== 0 || b !== -3) throw new Error("randomInt must return synchronously without callback");
let rejected = false;
try { crypto.randomFill(new Uint8Array(1)); } catch (e) { rejected = e instanceof TypeError; }
if (!rejected) throw new Error("missing randomFill callback accepted");
crypto.randomInt(4, 5, (err, value) => done(!err && value === 4 ? "ok" : "randomInt callback"));
`)
}
func TestNativeCryptoRandomFillLoopOwnership(t *testing.T) {
runNativeCryptoRegression(t, `
const backing = new ArrayBuffer(128);
const view = new Uint16Array(backing, 8, 32);
const raw = new Uint8Array(backing);
crypto.randomFill(view, 2, 16, (err, result) => {
if (err || result !== view) return done("randomFill callback identity");
if (raw.slice(0, 12).some(x => x !== 123) || raw.slice(44).some(x => x !== 123)) return done("randomFill wrote outside requested view range");
if (raw.slice(12, 44).every(x => x === 123)) return done("randomFill did not copy on loop");
done("ok");
});
// No loop callback can run yet. This must not overlap a worker writing raw.
for (let i = 0; i < 1000; i++) raw.fill(123);
`)
}
func TestNativeCryptoRandomFillDetachAndResize(t *testing.T) {
for _, mutation := range []string{"ab.transfer()", "ab.resize(4)"} {
t.Run(mutation, func(t *testing.T) {
runNativeCryptoRegression(t, `
const ab = new ArrayBuffer(64, {maxByteLength: 128});
const view = new Uint8Array(ab, 8, 32);
let calls = 0;
crypto.randomFill(view, 2, 16, (err, value) => {
if (++calls !== 1 || !(err instanceof Error) || value !== undefined) return done("detach/resize must callback with error");
done("ok");
});
`+mutation+`;`)
})
}
}
func TestNativeCryptoRandomFillReacquiresResizedStorage(t *testing.T) {
runNativeCryptoRegression(t, `
const ab = new ArrayBuffer(64, {maxByteLength: 1024});
const view = new DataView(ab, 4, 32);
crypto.randomFill(view, 3, 24, (err, value) => {
const raw = new Uint8Array(ab);
if (err || value !== view || raw.slice(7, 31).every(x => x === 0)) return done("stale randomFill storage after resize");
if (raw.slice(0, 7).some(x => x !== 0) || raw.slice(31).some(x => x !== 0)) return done("randomFill resize range");
done("ok");
});
ab.resize(1024);
`)
}
func TestNativeCryptoAsyncInputSnapshots(t *testing.T) {
public, private, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
privateDER, err := x509.MarshalPKCS8PrivateKey(private)
if err != nil {
t.Fatal(err)
}
publicDER, err := x509.MarshalPKIXPublicKey(public)
if err != nil {
t.Fatal(err)
}
privatePEM := string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privateDER}))
publicPEM := string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: publicDER}))
runNativeCryptoRegression(t, "const privatePEM = "+strconv.Quote(privatePEM)+"; const publicPEM = "+strconv.Quote(publicPEM)+";"+`
const password = Buffer.from("password"), salt = Buffer.from("salt");
const expectedPBKDF = crypto.pbkdf2Sync(password, salt, 1000, 32, "sha256").toString("hex");
const expectedScrypt = crypto.scryptSync(password, salt, 32, {N: 16, r: 1, p: 1}).toString("hex");
const data = Buffer.from("message"), privateKey = Buffer.from(privatePEM), publicKey = Buffer.from(publicPEM);
const signature = crypto.sign(null, data, privateKey);
let remaining = 4;
function checked(ok) { if (!ok) throw new Error("async crypto input was not snapshotted"); if (--remaining === 0) done("ok"); }
crypto.pbkdf2(password, salt, 1000, 32, "sha256", (err, value) => checked(!err && value.toString("hex") === expectedPBKDF));
crypto.scrypt(password, salt, 32, {N: 16, r: 1, p: 1}, (err, value) => checked(!err && value.toString("hex") === expectedScrypt));
crypto.sign(null, data, privateKey, (err, value) => checked(!err && value.toString("hex") === expectedSignature));
const expectedSignature = signature.toString("hex");
crypto.verify(null, data, publicKey, signature, (err, value) => checked(!err && value === true));
password.fill(0); salt.fill(0); data.fill(0); privateKey.fill(0); publicKey.fill(0); signature.fill(0);
`)
}
+26 -28
View File
@@ -2,46 +2,44 @@ package events
import (
_ "embed"
"fmt"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/require"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
)
func Load(vm *goja.Runtime, module *goja.Object) {
constructor, err := vm.RunString(eventEmitterSource)
func Load(vm *moe.Runtime, module *e.Object) {
constructor, err := native.Run(vm, eventEmitterSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load events module: %w", err)))
panic(err)
}
object := native.Object(vm, constructor)
if err = native.Set(vm, object, "EventEmitter", constructor); err != nil {
panic(err)
}
_ = native.Set(vm, object, "defaultMaxListeners", 10)
if err = native.Set(vm, module, "exports", constructor); err != nil {
panic(err)
}
_ = constructor.ToObject(vm).Set("EventEmitter", constructor)
_ = constructor.ToObject(vm).Set("defaultMaxListeners", 10)
_ = module.Set("exports", constructor)
}
func NewEmitter(vm *goja.Runtime) *goja.Object {
constructor := require.Require(vm, "events")
object, err := vm.New(constructor)
func NewEmitter(vm *moe.Runtime) *e.Object {
object, err := native.New(vm, require.Require(vm, "events"))
if err != nil {
panic(err)
}
return object
}
func Emit(vm *goja.Runtime, emitter *goja.Object, name string, values ...any) error {
emit, ok := goja.AssertFunction(emitter.Get("emit"))
if !ok {
return fmt.Errorf("EventEmitter.emit is not callable")
}
arguments := make([]goja.Value, 1, len(values)+1)
arguments[0] = vm.ToValue(name)
for _, value := range values {
if jsValue, ok := value.(goja.Value); ok {
arguments = append(arguments, jsValue)
} else {
arguments = append(arguments, vm.ToValue(value))
func Emit(vm *moe.Runtime, emitter *e.Object, name string, values ...any) error {
args := make([]moe.Value, 1, len(values)+1)
args[0] = moe.String(name)
for _, x := range values {
v, err := vm.FromGo(x)
if err != nil {
return err
}
args = append(args, v)
}
_, err := emit(emitter, arguments...)
_, err := vm.CallFunction(native.Get(vm, emitter, "emit"), e.ObjectValue(emitter), args...)
return err
}
+120 -138
View File
@@ -1,10 +1,17 @@
package fetch
import (
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
import (
"bytes"
"context"
"errors"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"io"
"mime"
"mime/multipart"
@@ -14,17 +21,12 @@ import (
"strconv"
"strings"
"sync"
"github.com/dop251/goja"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
)
type abortSignalState struct {
aborted bool
requests map[int64]context.CancelFunc
}
type nativeRequest struct {
method string
url string
@@ -33,7 +35,6 @@ type nativeRequest struct {
signalID int64
redirect string
}
type httpResult struct {
statusCode int
statusText string
@@ -42,7 +43,6 @@ type httpResult struct {
url string
redirected bool
}
type formEntry struct {
Name string
Value string
@@ -51,10 +51,9 @@ type formEntry struct {
Type string
File bool
}
type Module struct {
runtime *bridge.Runtime
vm *goja.Runtime
vm *moejs.Runtime
client *http.Client
maxBodyBytes int64
abortMu sync.Mutex
@@ -63,51 +62,57 @@ type Module struct {
}
func New(runtime *bridge.Runtime, client *http.Client, maxBodyBytes int64) *Module {
return &Module{
runtime: runtime,
client: client,
maxBodyBytes: maxBodyBytes,
abortState: make(map[int64]*abortSignalState),
}
return &Module{runtime: runtime, client: client, maxBodyBytes: maxBodyBytes, abortState: make(map[int64]*abortSignalState)}
}
func (m *Module) Inject(vm *goja.Runtime) error {
func (m *Module) Inject(vm *moejs.Runtime) error {
m.vm = vm
if err := vm.Set("__komariBodyBuffer", m.bodyBuffer); err != nil {
if err := vm.SetGlobal("__komariBodyBuffer", native.Bind(vm, "__komariBodyBuffer", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.bodyBuffer(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariBodyText", m.bodyText); err != nil {
if err := vm.SetGlobal("__komariBodyText", native.Bind(vm, "__komariBodyText", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.bodyText(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariEncodeFormData", m.encodeFormData); err != nil {
if err := vm.SetGlobal("__komariEncodeFormData", native.Bind(vm, "__komariEncodeFormData", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.encodeFormData(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariParseFormData", m.parseFormData); err != nil {
if err := vm.SetGlobal("__komariParseFormData", native.Bind(vm, "__komariParseFormData", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.parseFormData(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariNewAbortSignal", m.newAbortSignal); err != nil {
if err := vm.SetGlobal("__komariNewAbortSignal", native.Bind(vm, "__komariNewAbortSignal", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.newAbortSignal(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariAbortSignal", m.abortSignal); err != nil {
if err := vm.SetGlobal("__komariAbortSignal", native.Bind(vm, "__komariAbortSignal", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.abortSignal(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariFetch", m.createFetchFunction(false)); err != nil {
if err := vm.SetGlobal("__komariFetch", native.Bind(vm, "__komariFetch", 1, m.createFetchFunction(false))); err != nil {
return err
}
if err := vm.Set("__komariFetchSync", m.createFetchFunction(true)); err != nil {
if err := vm.SetGlobal("__komariFetchSync", native.Bind(vm, "__komariFetchSync", 1, m.createFetchFunction(true))); err != nil {
return err
}
if _, err := vm.RunString(fetchAPISource); err != nil {
if _, err := native.Run(vm, fetchAPISource); err != nil {
return fmt.Errorf("inject Fetch API: %w", err)
}
return nil
}
func (m *Module) createFetchFunction(syncRequest bool) func(goja.FunctionCall) goja.Value {
return func(call goja.FunctionCall) goja.Value {
request, err := m.exportNativeRequest(call.Argument(0))
func (m *Module) createFetchFunction(syncRequest bool) moejs.NativeFunc {
vm := m.vm
return func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
request, err := m.exportNativeRequest(moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("%s", err))
panic(native.TypeError(m.vm, "%s", err))
}
ctx, releaseSignal := m.requestContext(request.signalID)
ctx, cancelTimeout := context.WithTimeout(ctx, m.runtime.Timeout())
@@ -121,13 +126,12 @@ func (m *Module) createFetchFunction(syncRequest bool) func(goja.FunctionCall) g
if requestErr != nil {
panic(webAPIError(m.vm, "TypeError", fmt.Sprintf("fetch failed: %v", requestErr)))
}
return m.httpResultValue(m.vm, result)
return native.From(vm, m.httpResultValue(m.vm, result)), nil
}
promise, resolve, reject := m.vm.NewPromise()
go func() {
result, requestErr := m.doHTTPRequest(ctx, request)
if !m.runtime.RunOnLoop(func(vm *goja.Runtime) {
if !m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
defer release()
m.runtime.RunJob(vm, "fetch", func() error {
if ctx.Err() != nil {
@@ -146,30 +150,25 @@ func (m *Module) createFetchFunction(syncRequest bool) func(goja.FunctionCall) g
release()
}
}()
return m.vm.ToValue(promise)
return native.From(vm, native.From(m.vm, promise)), nil
}
}
func (m *Module) exportNativeRequest(value goja.Value) (nativeRequest, error) {
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func (m *Module) exportNativeRequest(value moejs.Value) (nativeRequest, error) {
vm := m.vm
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return nativeRequest{}, errors.New("missing request data")
}
object := value.ToObject(m.vm)
request := nativeRequest{
method: object.Get("method").String(),
url: object.Get("url").String(),
signalID: object.Get("signalId").ToInteger(),
redirect: object.Get("redirect").String(),
}
object := native.Object(m.vm, value)
request := nativeRequest{method: native.String(vm, native.Get(vm, object, "method")), url: native.String(vm, native.Get(vm, object, "url")), signalID: native.Int(vm, native.Get(vm, object, "signalId")), redirect: native.String(vm, native.Get(vm, object, "redirect"))}
if request.method == "" {
request.method = http.MethodGet
}
if request.redirect == "" {
request.redirect = "follow"
}
if value := object.Get("headers"); value != nil && !goja.IsUndefined(value) {
if value := native.Get(vm, object, "headers"); !native.IsNil(value) && !value.IsUndefined() {
var headers [][]string
if err := m.vm.ExportTo(value, &headers); err != nil {
if err := native.ExportTo(m.vm, value, &headers); err != nil {
return nativeRequest{}, fmt.Errorf("invalid request headers: %w", err)
}
for _, header := range headers {
@@ -179,7 +178,7 @@ func (m *Module) exportNativeRequest(value goja.Value) (nativeRequest, error) {
request.headers = append(request.headers, [2]string{header[0], header[1]})
}
}
if value := object.Get("body"); value != nil && !goja.IsUndefined(value) && !goja.IsNull(value) {
if value := native.Get(vm, object, "body"); !native.IsNil(value) && !value.IsUndefined() && !value.IsNull() {
body, err := exportBytes(m.vm, value)
if err != nil {
return nativeRequest{}, fmt.Errorf("invalid request body: %w", err)
@@ -188,7 +187,6 @@ func (m *Module) exportNativeRequest(value goja.Value) (nativeRequest, error) {
}
return request, nil
}
func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (httpResult, error) {
httpRequest, err := http.NewRequestWithContext(ctx, request.method, request.url, bytes.NewReader(request.body))
if err != nil {
@@ -201,7 +199,6 @@ func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (http
}
httpRequest.Header.Add(header[0], header[1])
}
client := *m.client
switch request.redirect {
case "manual":
@@ -216,7 +213,6 @@ func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (http
default:
return httpResult{}, fmt.Errorf("unsupported redirect mode %q", request.redirect)
}
response, err := client.Do(httpRequest)
if err != nil {
return httpResult{}, err
@@ -231,75 +227,60 @@ func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (http
responseURL = response.Request.URL.String()
}
statusText := strings.TrimSpace(strings.TrimPrefix(response.Status, strconv.Itoa(response.StatusCode)))
return httpResult{
statusCode: response.StatusCode,
statusText: statusText,
headers: response.Header.Clone(),
body: body,
url: responseURL,
redirected: responseURL != request.url,
}, nil
return httpResult{statusCode: response.StatusCode, statusText: statusText, headers: response.Header.Clone(), body: body, url: responseURL, redirected: responseURL != request.url}, nil
}
func (m *Module) httpResultValue(vm *goja.Runtime, result httpResult) goja.Value {
object := vm.NewObject()
_ = object.Set("status", result.statusCode)
_ = object.Set("statusText", result.statusText)
_ = object.Set("url", result.url)
_ = object.Set("redirected", result.redirected)
_ = object.Set("body", vm.NewArrayBuffer(append([]byte(nil), result.body...)))
func (m *Module) httpResultValue(vm *moejs.Runtime, result httpResult) moejs.Value {
object := vm.Realm().NewObject()
_ = native.Set(vm, object, "status", result.statusCode)
_ = native.Set(vm, object, "statusText", result.statusText)
_ = native.Set(vm, object, "url", result.url)
_ = native.Set(vm, object, "redirected", result.redirected)
_ = native.Set(vm, object, "body", native.From(vm, append([]byte(nil), result.body...)))
headers := make([][2]string, 0, len(result.headers))
for name, values := range result.headers {
for _, value := range values {
headers = append(headers, [2]string{name, value})
}
}
_ = object.Set("headers", headers)
return object
_ = native.Set(vm, object, "headers", headers)
return engine.ObjectValue(object)
}
func (m *Module) bodyBuffer(call goja.FunctionCall) goja.Value {
data, err := exportBytes(m.vm, call.Argument(0))
func (m *Module) bodyBuffer(args []moejs.Value) moejs.Value {
data, err := exportBytes(m.vm, moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("unsupported body value: %v", err))
panic(native.TypeError(m.vm, "unsupported body value: %v", err))
}
return m.vm.ToValue(m.vm.NewArrayBuffer(data))
return native.From(m.vm, native.From(m.vm, data))
}
func (m *Module) bodyText(call goja.FunctionCall) goja.Value {
data, err := exportBytes(m.vm, call.Argument(0))
func (m *Module) bodyText(args []moejs.Value) moejs.Value {
data, err := exportBytes(m.vm, moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("invalid body buffer: %v", err))
panic(native.TypeError(m.vm, "invalid body buffer: %v", err))
}
return m.vm.ToValue(strings.ToValidUTF8(string(data), "\uFFFD"))
return native.From(m.vm, strings.ToValidUTF8(string(data), "\uFFFD"))
}
func exportBytes(vm *goja.Runtime, value goja.Value) ([]byte, error) {
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func exportBytes(vm *moejs.Runtime, value moejs.Value) ([]byte, error) {
if native.IsNil(value) {
return nil, nil
}
if stringValue, ok := value.Export().(string); ok {
return []byte(stringValue), nil
if value.IsString() {
return []byte(native.String(vm, value)), nil
}
if buffer, ok := value.Export().(goja.ArrayBuffer); ok {
return append([]byte(nil), buffer.Bytes()...), nil
}
var bytesValue []byte
if err := vm.ExportTo(value, &bytesValue); err != nil {
var data []byte
if err := native.ExportTo(vm, value, &data); err != nil {
return nil, err
}
return append([]byte(nil), bytesValue...), nil
return append([]byte(nil), data...), nil
}
func (m *Module) newAbortSignal(goja.FunctionCall) goja.Value {
func (m *Module) newAbortSignal(args []moejs.Value) moejs.Value {
m.abortMu.Lock()
defer m.abortMu.Unlock()
m.abortIDs++
return m.vm.ToValue(m.abortIDs)
return native.From(m.vm, m.abortIDs)
}
func (m *Module) abortSignal(call goja.FunctionCall) goja.Value {
id := call.Argument(0).ToInteger()
func (m *Module) abortSignal(args []moejs.Value) moejs.Value {
vm := m.vm
id := native.Int(vm, moejs.Arg(args, 0))
m.abortMu.Lock()
state := m.abortState[id]
if state == nil {
@@ -311,15 +292,13 @@ func (m *Module) abortSignal(call goja.FunctionCall) goja.Value {
cancel()
}
m.abortMu.Unlock()
return goja.Undefined()
return moejs.Undefined()
}
func (m *Module) requestContext(signalID int64) (context.Context, context.CancelFunc) {
ctx, cancel := context.WithCancel(context.Background())
if signalID <= 0 {
return ctx, cancel
}
m.abortMu.Lock()
state := m.abortState[signalID]
if state != nil && state.aborted {
@@ -335,7 +314,6 @@ func (m *Module) requestContext(signalID int64) (context.Context, context.Cancel
requestID := m.abortIDs
state.requests[requestID] = cancel
m.abortMu.Unlock()
return ctx, func() {
m.abortMu.Lock()
if current := m.abortState[signalID]; current != nil {
@@ -348,7 +326,6 @@ func (m *Module) requestContext(signalID int64) (context.Context, context.Cancel
cancel()
}
}
func (m *Module) Close() {
m.abortMu.Lock()
defer m.abortMu.Unlock()
@@ -359,36 +336,34 @@ func (m *Module) Close() {
}
}
}
func webAPIError(vm *goja.Runtime, name, message string) *goja.Object {
object := vm.NewGoError(errors.New(message))
_ = object.Set("name", name)
func webAPIError(vm *moejs.Runtime, name, message string) moejs.Value {
object := native.GoError(vm, errors.New(message))
_ = native.Set(vm, native.Object(vm, object), "name", name)
return object
}
func (m *Module) encodeFormData(call goja.FunctionCall) goja.Value {
func (m *Module) encodeFormData(args []moejs.Value) moejs.Value {
vm := m.vm
var entries []formEntry
array := call.Argument(0).ToObject(m.vm)
length := int(array.Get("length").ToInteger())
array := native.Object(m.vm, moejs.Arg(args, 0))
length := int(native.Int(vm, native.Get(vm, array, "length")))
for index := 0; index < length; index++ {
entryObject := array.Get(strconv.Itoa(index)).ToObject(m.vm)
entry := formEntry{Name: entryObject.Get("name").String()}
value := entryObject.Get("value")
if entryObject.Get("file").ToBoolean() {
entryObject := native.Object(m.vm, native.Get(vm, array, strconv.Itoa(index)))
entry := formEntry{Name: native.String(vm, native.Get(vm, entryObject, "name"))}
value := native.Get(vm, entryObject, "value")
if native.Bool(native.Get(vm, entryObject, "file")) {
entry.File = true
entry.Filename = entryObject.Get("filename").String()
entry.Type = entryObject.Get("type").String()
entry.Filename = native.String(vm, native.Get(vm, entryObject, "filename"))
entry.Type = native.String(vm, native.Get(vm, entryObject, "type"))
data, err := exportBytes(m.vm, value)
if err != nil {
panic(m.vm.NewTypeError("invalid FormData file: %v", err))
panic(native.TypeError(m.vm, "invalid FormData file: %v", err))
}
entry.Bytes = data
} else {
entry.Value = value.String()
entry.Value = native.String(vm, value)
}
entries = append(entries, entry)
}
var output bytes.Buffer
writer := multipart.NewWriter(&output)
for _, entry := range entries {
@@ -403,27 +378,27 @@ func (m *Module) encodeFormData(call goja.FunctionCall) goja.Value {
}
part, err := writer.CreatePart(header)
if err != nil {
panic(m.vm.NewGoError(err))
panic(native.GoError(m.vm, err))
}
_, _ = part.Write(entry.Bytes)
}
if err := writer.Close(); err != nil {
panic(m.vm.NewGoError(err))
panic(native.GoError(m.vm, err))
}
result := m.vm.NewObject()
_ = result.Set("body", m.vm.NewArrayBuffer(output.Bytes()))
_ = result.Set("contentType", writer.FormDataContentType())
return result
result := m.vm.Realm().NewObject()
_ = native.Set(vm, result, "body", native.From(m.vm, output.Bytes()))
_ = native.Set(vm, result, "contentType", writer.FormDataContentType())
return native.From(vm, result)
}
func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
data, err := exportBytes(m.vm, call.Argument(0))
func (m *Module) parseFormData(args []moejs.Value) moejs.Value {
vm := m.vm
data, err := exportBytes(m.vm, moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("invalid FormData body: %v", err))
panic(native.TypeError(m.vm, "invalid FormData body: %v", err))
}
mediaType, parameters, err := mime.ParseMediaType(call.Argument(1).String())
mediaType, parameters, err := mime.ParseMediaType(native.String(vm, moejs.Arg(args, 1)))
if err != nil {
panic(m.vm.NewTypeError("invalid form content type: %v", err))
panic(native.TypeError(m.vm, "invalid form content type: %v", err))
}
entries := make([]map[string]any, 0)
switch mediaType {
@@ -435,13 +410,13 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
nameValue := strings.SplitN(field, "=", 2)
name, parseErr := url.QueryUnescape(nameValue[0])
if parseErr != nil {
panic(m.vm.NewTypeError("invalid form field name: %v", parseErr))
panic(native.TypeError(m.vm, "invalid form field name: %v", parseErr))
}
value := ""
if len(nameValue) == 2 {
value, parseErr = url.QueryUnescape(nameValue[1])
if parseErr != nil {
panic(m.vm.NewTypeError("invalid form field value: %v", parseErr))
panic(native.TypeError(m.vm, "invalid form field value: %v", parseErr))
}
}
entries = append(entries, map[string]any{"name": name, "value": value})
@@ -449,7 +424,7 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
case "multipart/form-data":
boundary := parameters["boundary"]
if boundary == "" {
panic(m.vm.NewTypeError("multipart form data has no boundary"))
panic(native.TypeError(m.vm, "multipart form data has no boundary"))
}
reader := multipart.NewReader(bytes.NewReader(data), boundary)
for {
@@ -458,11 +433,11 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
break
}
if partErr != nil {
panic(m.vm.NewTypeError("invalid multipart body: %v", partErr))
panic(native.TypeError(m.vm, "invalid multipart body: %v", partErr))
}
partBytes, readErr := io.ReadAll(part)
if readErr != nil {
panic(m.vm.NewTypeError("read multipart body: %v", readErr))
panic(native.TypeError(m.vm, "read multipart body: %v", readErr))
}
entry := map[string]any{"name": part.FormName()}
if part.FileName() == "" {
@@ -471,12 +446,19 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
entry["file"] = true
entry["filename"] = part.FileName()
entry["type"] = part.Header.Get("Content-Type")
entry["value"] = m.vm.NewArrayBuffer(partBytes)
entry["value"] = native.From(m.vm, partBytes)
}
entries = append(entries, entry)
}
default:
panic(m.vm.NewTypeError("unsupported form content type %q", mediaType))
panic(native.TypeError(m.vm, "unsupported form content type %q", mediaType))
}
return m.vm.ToValue(entries)
return native.From(m.vm, entries)
}
func callable(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
func asObject(v moejs.Value) (*engine.Object, bool) {
if !v.IsObject() {
return nil, false
}
return v.AsObject(), true
}
+45
View File
@@ -0,0 +1,45 @@
package fetch
import (
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"net/http"
"testing"
)
func TestNativeFormDataAndBody(t *testing.T) {
vm := moejs.NewRuntime(moejs.Options{})
m := New(nil, http.DefaultClient, 1<<20)
if err := m.Inject(vm); err != nil {
t.Fatal(err)
}
value, err := native.Run(vm, `(async function () {
const blob = new Blob(["ab", new Uint8Array([99])], {type:"TEXT/PLAIN"});
if (blob.size !== 3 || await blob.text() !== "abc") throw new Error("blob bytes changed");
const form = new FormData();
form.append("field", "one"); form.append("field", "two");
form.append("file", new File([blob], "test.txt", {type:blob.type}));
const request = new Request("https://example.test/", {method:"POST", body:form});
const response = new Response(await request.arrayBuffer(), {headers:request.headers});
const parsed = await response.formData();
if (parsed.getAll("field").join(",") !== "one,two") throw new Error("form fields changed");
const file = parsed.get("file");
if (file.name !== "test.txt" || file.type !== "text/plain" || await file.text() !== "abc") throw new Error("multipart file changed");
const urlencoded = await new Response("hello=world&hello=again", {headers:{"content-type":"application/x-www-form-urlencoded"}}).formData();
if (urlencoded.getAll("hello").join(",") !== "world,again") throw new Error("urlencoded changed");
return true;
})()`)
if err != nil {
t.Fatal(err)
}
state, result, ok := moejs.PromiseResult(value)
if !ok {
t.Fatal("expected promise")
}
if state != moejs.PromiseFulfilled {
t.Fatalf("promise state %v result %v", state, native.String(vm, result))
}
if !native.Bool(result) {
t.Fatal("body/form verification failed")
}
}
+81 -89
View File
@@ -3,55 +3,50 @@ package fs
import (
"errors"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"io"
"io/fs"
"os"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
)
type fsAsyncOperation struct {
run func() (any, error)
values func(*goja.Runtime, any) []goja.Value
values func(*moejs.Runtime, any) []moejs.Value
}
type fsReadFileResult struct {
data []byte
encoding string
}
type fsReadDirResult struct {
entries []fs.DirEntry
withTypes bool
}
type fsReadResult struct {
count int
data []byte
target goja.Value
target moejs.Value
offset int
}
type fsWriteResult struct {
count int
target goja.Value
target moejs.Value
}
func fsArgument(arguments []goja.Value, index int) goja.Value {
func fsArgument(arguments []moejs.Value, index int) moejs.Value {
if index < 0 || index >= len(arguments) {
return goja.Undefined()
return moejs.Undefined()
}
return arguments[index]
}
func fsNoValues(*goja.Runtime, any) []goja.Value { return nil }
func fsSingleValue(vm *goja.Runtime, result any) []goja.Value {
return []goja.Value{vm.ToValue(result)}
func fsNoValues(*moejs.Runtime, any) []moejs.Value {
return nil
}
func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.Value) fsAsyncOperation {
func fsSingleValue(vm *moejs.Runtime, result any) []moejs.Value {
return []moejs.Value{native.From(vm, result)}
}
func (m *Module) prepareFSAsync(vm *moejs.Runtime, name string, arguments []moejs.Value) fsAsyncOperation {
cwd := m.Cwd()
follow := func(path string, allowMissing bool) (string, error) {
return m.resolveNodePathAt(path, cwd, allowMissing, nodePathFollowFinal)
@@ -59,29 +54,25 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
noFollow := func(path string, allowMissing bool) (string, error) {
return m.resolveNodePathAt(path, cwd, allowMissing, nodePathNoFollowFinal)
}
switch name {
case "readFile":
pathName := fsArgument(arguments, 0).String()
encoding := fsEncoding(fsArgument(arguments, 1))
return fsAsyncOperation{
run: func() (any, error) {
path, err := follow(pathName, false)
if err != nil {
return nil, err
}
data, err := m.nodeReadFile(path)
return fsReadFileResult{data: data, encoding: encoding}, err
},
values: func(vm *goja.Runtime, result any) []goja.Value {
value := result.(fsReadFileResult)
return []goja.Value{encodeFSData(vm, value.data, value.encoding)}
},
}
pathName := native.String(vm, fsArgument(arguments, 0))
encoding := fsEncoding(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err != nil {
return nil, err
}
data, err := m.nodeReadFile(path)
return fsReadFileResult{data: data, encoding: encoding}, err
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsReadFileResult)
return []moejs.Value{encodeFSData(vm, value.data, value.encoding)}
}}
case "writeFile", "appendFile":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
data := append([]byte(nil), buffer.Bytes(vm, fsArgument(arguments, 1))...)
mode := fsMode(fsArgument(arguments, 2), 0o666)
mode := fsMode(vm, fsArgument(arguments, 2), 0o666)
appendMode := name == "appendFile"
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, true)
@@ -100,8 +91,8 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, errors.Join(writeErr, file.Close())
}, values: fsNoValues}
case "access":
pathName := fsArgument(arguments, 0).String()
mode := fsAccessMode(fsArgument(arguments, 1))
pathName := native.String(vm, fsArgument(arguments, 0))
mode := fsAccessMode(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -110,7 +101,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "stat", "lstat":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
isLstat := name == "lstat"
return fsAsyncOperation{run: func() (any, error) {
var path string
@@ -127,12 +118,12 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return m.nodeStat(path, false)
}
return m.nodeStat(path, true)
}, values: func(vm *goja.Runtime, result any) []goja.Value {
return []goja.Value{fsStatObject(vm, result.(os.FileInfo))}
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
return []moejs.Value{native.From(vm, fsStatObject(vm, result.(os.FileInfo)))}
}}
case "readdir":
pathName := fsArgument(arguments, 0).String()
withTypes := fsWithFileTypes(fsArgument(arguments, 1))
pathName := native.String(vm, fsArgument(arguments, 0))
withTypes := fsWithFileTypes(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err != nil {
@@ -140,7 +131,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
}
entries, err := m.nodeReadDir(path)
return fsReadDirResult{entries: entries, withTypes: withTypes}, err
}, values: func(vm *goja.Runtime, result any) []goja.Value {
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsReadDirResult)
entries := make([]any, 0, len(value.entries))
for _, entry := range value.entries {
@@ -150,12 +141,12 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
entries = append(entries, entry.Name())
}
}
return []goja.Value{vm.ToValue(entries)}
return []moejs.Value{native.From(vm, entries)}
}}
case "mkdir":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
options := fsArgument(arguments, 1)
recursive, mode := fsBooleanOption(options, "recursive"), fsMode(options, 0o777)
recursive, mode := fsBooleanOption(vm, options, "recursive"), fsMode(vm, options, 0o777)
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(pathName, true)
if err != nil {
@@ -169,9 +160,9 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "rm":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
options := fsArgument(arguments, 1)
recursive, force := fsBooleanOption(options, "recursive"), fsBooleanOption(options, "force")
recursive, force := fsBooleanOption(vm, options, "recursive"), fsBooleanOption(vm, options, "force")
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(pathName, true)
if err != nil {
@@ -188,7 +179,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "unlink", "rmdir":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(pathName, false)
if err == nil {
@@ -197,7 +188,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "rename":
oldName, newName := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).String()
oldName, newName := native.String(vm, fsArgument(arguments, 0)), native.String(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
oldPath, err := noFollow(oldName, false)
if err != nil {
@@ -210,7 +201,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "copyFile":
sourceName, targetName := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).String()
sourceName, targetName := native.String(vm, fsArgument(arguments, 0)), native.String(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
source, err := follow(sourceName, false)
if err != nil {
@@ -233,7 +224,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, errors.Join(copyErr, output.Close())
}, values: fsNoValues}
case "realpath", "readlink":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
readLink := name == "readlink"
return fsAsyncOperation{run: func() (any, error) {
if readLink {
@@ -246,7 +237,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return follow(pathName, false)
}, values: fsSingleValue}
case "symlink":
targetName, linkName := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).String()
targetName, linkName := native.String(vm, fsArgument(arguments, 0)), native.String(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
target, err := follow(targetName, true)
if err != nil {
@@ -259,7 +250,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "truncate":
pathName, size := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).ToInteger()
pathName, size := native.String(vm, fsArgument(arguments, 0)), native.Int(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -268,8 +259,8 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "chmod":
pathName := fsArgument(arguments, 0).String()
mode := os.FileMode(fsArgument(arguments, 1).ToInteger())
pathName := native.String(vm, fsArgument(arguments, 0))
mode := os.FileMode(native.Int(vm, fsArgument(arguments, 1)))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -278,8 +269,8 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "utimes":
pathName := fsArgument(arguments, 0).String()
accessTime, modifyTime := fsTime(fsArgument(arguments, 1).Export()), fsTime(fsArgument(arguments, 2).Export())
pathName := native.String(vm, fsArgument(arguments, 0))
accessTime, modifyTime := fsTime(native.Export(vm, fsArgument(arguments, 1))), fsTime(native.Export(vm, fsArgument(arguments, 2)))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -288,7 +279,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "mkdtemp":
prefix := fsArgument(arguments, 0).String()
prefix := native.String(vm, fsArgument(arguments, 0))
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(prefix, true)
if err != nil {
@@ -297,9 +288,9 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return m.nodeMkdirTemp(path)
}, values: fsSingleValue}
case "open":
pathName := fsArgument(arguments, 0).String()
flags := fsArgument(arguments, 1).String()
mode := fsMode(fsArgument(arguments, 2), 0o666)
pathName := native.String(vm, fsArgument(arguments, 0))
flags := native.String(vm, fsArgument(arguments, 1))
mode := fsMode(vm, fsArgument(arguments, 2), 0o666)
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, true)
if err != nil {
@@ -308,21 +299,23 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return m.fsOpenPath(path, flags, mode)
}, values: fsSingleValue}
case "close":
fd := int(fsArgument(arguments, 0).ToInteger())
return fsAsyncOperation{run: func() (any, error) { return nil, m.fsClose(fd) }, values: fsNoValues}
fd := int(native.Int(vm, fsArgument(arguments, 0)))
return fsAsyncOperation{run: func() (any, error) {
return nil, m.fsClose(fd)
}, values: fsNoValues}
case "fstat":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
if err != nil {
return nil, err
}
return file.Stat()
}, values: func(vm *goja.Runtime, result any) []goja.Value {
return []goja.Value{fsStatObject(vm, result.(os.FileInfo))}
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
return []moejs.Value{native.From(vm, fsStatObject(vm, result.(os.FileInfo)))}
}}
case "fsync":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
if err == nil {
@@ -331,16 +324,16 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "read":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
target := fsArgument(arguments, 1)
offset := int(fsArgument(arguments, 2).ToInteger())
length := int(fsArgument(arguments, 3).ToInteger())
offset := int(native.Int(vm, fsArgument(arguments, 2)))
length := int(native.Int(vm, fsArgument(arguments, 3)))
positionValue := fsArgument(arguments, 4)
hasPosition := !goja.IsNull(positionValue) && !goja.IsUndefined(positionValue)
position := positionValue.ToInteger()
hasPosition := !positionValue.IsNull() && !positionValue.IsUndefined()
position := native.Int(vm, positionValue)
bufferLength := len(buffer.Bytes(vm, target))
if offset < 0 || length < 0 || offset+length > bufferLength {
panic(vm.NewTypeError("buffer range is out of bounds"))
panic(native.TypeError(vm, "buffer range is out of bounds"))
}
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
@@ -358,28 +351,28 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
err = nil
}
return fsReadResult{count: count, data: data[:count], target: target, offset: offset}, err
}, values: func(vm *goja.Runtime, result any) []goja.Value {
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsReadResult)
copy(buffer.Bytes(vm, value.target)[value.offset:], value.data)
return []goja.Value{vm.ToValue(value.count), value.target}
return []moejs.Value{native.From(vm, value.count), value.target}
}}
case "write":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
target := fsArgument(arguments, 1)
data := append([]byte(nil), buffer.Bytes(vm, target)...)
positionValue := fsArgument(arguments, 4)
if _, isString := target.Export().(string); !isString {
offset := int(fsArgument(arguments, 2).ToInteger())
length := int(fsArgument(arguments, 3).ToInteger())
if _, isString := native.Export(vm, target).(string); !isString {
offset := int(native.Int(vm, fsArgument(arguments, 2)))
length := int(native.Int(vm, fsArgument(arguments, 3)))
if offset < 0 || length < 0 || offset+length > len(data) {
panic(vm.NewTypeError("buffer range is out of bounds"))
panic(native.TypeError(vm, "buffer range is out of bounds"))
}
data = data[offset : offset+length]
} else {
positionValue = fsArgument(arguments, 2)
}
hasPosition := !goja.IsNull(positionValue) && !goja.IsUndefined(positionValue)
position := positionValue.ToInteger()
hasPosition := !positionValue.IsNull() && !positionValue.IsUndefined()
position := native.Int(vm, positionValue)
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
if err != nil {
@@ -392,9 +385,9 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
count, err = file.Write(data)
}
return fsWriteResult{count: count, target: target}, err
}, values: func(vm *goja.Runtime, result any) []goja.Value {
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsWriteResult)
return []goja.Value{vm.ToValue(value.count), value.target}
return []moejs.Value{native.From(vm, value.count), value.target}
}}
default:
return fsAsyncOperation{run: func() (any, error) {
@@ -402,7 +395,6 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
}, values: fsNoValues}
}
}
func (m *Module) fsOpenPath(path, flags string, mode os.FileMode) (int, error) {
file, err := m.nodeOpenFile(path, fsOpenFlags(flags), mode)
if err != nil {
+11 -10
View File
@@ -2,14 +2,15 @@ package fs
import (
"errors"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"io/fs"
"os"
"strings"
"github.com/dop251/goja"
)
func nodeErrorObject(vm *goja.Runtime, err error, fallbackSyscall string) *goja.Object {
func nodeErrorObject(vm *moejs.Runtime, err error, fallbackSyscall string) *engine.Object {
operation, path, destination := fallbackSyscall, "", ""
underlying := err
var linkError *os.LinkError
@@ -41,16 +42,16 @@ func nodeErrorObject(vm *goja.Runtime, err error, fallbackSyscall string) *goja.
if errno == 0 {
errno = nodeErrnoForCode(code)
}
object := vm.NewGoError(err)
_ = object.Set("name", "Error")
_ = object.Set("code", code)
_ = object.Set("errno", errno)
_ = object.Set("syscall", operation)
object := native.Object(vm, native.GoError(vm, err))
_ = native.Set(vm, object, "name", "Error")
_ = native.Set(vm, object, "code", code)
_ = native.Set(vm, object, "errno", errno)
_ = native.Set(vm, object, "syscall", operation)
if path != "" {
_ = object.Set("path", path)
_ = native.Set(vm, object, "path", path)
}
if destination != "" {
_ = object.Set("dest", destination)
_ = native.Set(vm, object, "dest", destination)
}
return object
}
+345 -342
View File
File diff suppressed because it is too large. Load diff
+6 -3
View File
@@ -3,16 +3,19 @@ package jsruntime
import (
_ "embed"
"fmt"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
nodeurl "github.com/komari-monitor/komari/pkg/jsruntime/url"
"github.com/komari-monitor/komari/pkg/jsruntime/xhr"
)
// JavaScript compatibility: injects bounded timer APIs, console, Fetch API,
// XMLHttpRequest, and queueMicrotask.
// JavaScript compatibility: injects bounded timer APIs, console, URL APIs,
// Fetch API, XMLHttpRequest, and queueMicrotask.
func (r *Runtime) injectGlobals() error {
if err := r.consoleMod.Inject(r.vm); err != nil {
return fmt.Errorf("inject console: %w", err)
}
nodeurl.Enable(r.vm)
if err := r.timersMod.Inject(r.vm); err != nil {
return fmt.Errorf("inject timers: %w", err)
}
@@ -27,7 +30,7 @@ func (r *Runtime) injectGlobals() error {
if err := xhr.Inject(r.vm); err != nil {
return fmt.Errorf("inject XMLHttpRequest: %w", err)
}
if _, err := r.vm.RunString(queueMicrotaskSource); err != nil {
if _, err := native.Run(r.vm, queueMicrotaskSource); err != nil {
return fmt.Errorf("inject queueMicrotask: %w", err)
}
return nil
+24 -3
View File
@@ -9,7 +9,7 @@ package jsruntime
import (
"time"
"github.com/dop251/goja"
moe "github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
)
@@ -22,14 +22,14 @@ type Host struct {
// RunOnLoop queues job on the runtime's event loop. It returns false when
// the loop is no longer running, for example after Runtime.Close.
func (h *Host) RunOnLoop(job func(*goja.Runtime)) bool {
func (h *Host) RunOnLoop(job func(*moe.Runtime)) bool {
return h.runtime.RunOnLoop(job)
}
// RunJob runs one asynchronous host callback as a bounded JavaScript turn,
// reporting errors through the runtime's error reporter (the console by
// default). Exit code 0 is treated as a normal termination.
func (h *Host) RunJob(vm *goja.Runtime, name string, job func() error) error {
func (h *Host) RunJob(vm *moe.Runtime, name string, job func() error) error {
return h.runtime.RunJob(vm, name, job)
}
@@ -37,3 +37,24 @@ func (h *Host) RunJob(vm *goja.Runtime, name string, job func() error) error {
func (h *Host) Timeout() time.Duration {
return h.runtime.Timeout()
}
// AddResource registers a closer to run when the runtime shuts down. If shutdown
// has already started, it runs the closer immediately and returns 0. Closers
// must be nonblocking and must not access the JavaScript VM or wait for its event
// loop; they may run outside the loop during shutdown.
func (h *Host) AddResource(close func()) uint64 {
return h.runtime.AddResource(close)
}
// RemoveResource unregisters a resource without calling its closer. Unknown IDs
// and 0 are ignored.
func (h *Host) RemoveResource(id uint64) {
h.runtime.RemoveResource(id)
}
// ResourcesOpen reports whether the runtime still accepts resources, not whether
// any resources are registered. The result is a snapshot; AddResource handles
// registration racing with shutdown by immediately closing the new resource.
func (h *Host) ResourcesOpen() bool {
return h.runtime.ResourcesOpen()
}
+292 -249
View File
@@ -1,22 +1,25 @@
package http
import (
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
import (
"bytes"
_ "embed"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"net"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
)
type nodeHTTPResponse struct {
@@ -26,61 +29,63 @@ type nodeHTTPResponse struct {
statusCode int
statusText string
ended bool
done chan struct{}
done chan struct {
}
}
type Module struct {
runtime *bridge.Runtime
allowListen bool
maxBodyBytes int64
runtime *bridge.Runtime
allowListen bool
maxBodyBytes int64
streamMu sync.Mutex
createIncoming goja.Callable
createResponse goja.Callable
createIncoming moejs.Value
createResponse moejs.Value
}
func New(runtime *bridge.Runtime, allowListen bool, maxBodyBytes int64) *Module {
return &Module{runtime: runtime, allowListen: allowListen, maxBodyBytes: maxBodyBytes}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
exports := vm.NewObject()
_ = exports.Set("createServer", func(call goja.FunctionCall) goja.Value {
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
exports := vm.Realm().NewObject()
_ = native.Set(vm, exports, "createServer", native.Bind(vm, "createServer", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
server := m.newHTTPServer(vm)
listener := call.Argument(0)
if _, ok := goja.AssertFunction(listener); !ok {
listener = call.Argument(1)
listener := moejs.Arg(args, 0)
if _, ok := callable(listener); !ok {
listener = moejs.Arg(args, 1)
}
if _, ok := goja.AssertFunction(listener); ok {
on, _ := goja.AssertFunction(server.Get("on"))
_, _ = on(server, vm.ToValue("request"), listener)
if _, ok := callable(listener); ok {
on, _ := callable(native.Get(vm, server, "on"))
_, _ = native.Call(vm, on, native.From(vm, server), native.From(vm, "request"), native.From(vm, listener))
}
return server
})
_ = exports.Set("METHODS", []string{"ACL", "BIND", "CHECKOUT", "CONNECT", "COPY", "DELETE", "GET", "HEAD", "LINK", "LOCK", "M-SEARCH", "MERGE", "MKACTIVITY", "MKCALENDAR", "MKCOL", "MOVE", "NOTIFY", "OPTIONS", "PATCH", "POST", "PROPFIND", "PROPPATCH", "PURGE", "PUT", "QUERY", "REBIND", "REPORT", "SEARCH", "SOURCE", "SUBSCRIBE", "TRACE", "UNBIND", "UNLINK", "UNLOCK", "UNSUBSCRIBE"})
return native.From(vm, server), nil
}))
_ = native.Set(vm, exports, "METHODS", []string{"ACL", "BIND", "CHECKOUT", "CONNECT", "COPY", "DELETE", "GET", "HEAD", "LINK", "LOCK", "M-SEARCH", "MERGE", "MKACTIVITY", "MKCALENDAR", "MKCOL", "MOVE", "NOTIFY", "OPTIONS", "PATCH", "POST", "PROPFIND", "PROPPATCH", "PURGE", "PUT", "QUERY", "REBIND", "REPORT", "SEARCH", "SOURCE", "SUBSCRIBE", "TRACE", "UNBIND", "UNLINK", "UNLOCK", "UNSUBSCRIBE"})
statusCodes := make(map[int]string)
for code := 100; code <= 599; code++ {
if text := http.StatusText(code); text != "" {
statusCodes[code] = text
}
}
_ = exports.Set("STATUS_CODES", statusCodes)
_ = exports.Set("maxHeaderSize", 1<<20)
_ = exports.Set("validateHeaderName", func(name string) {
_ = native.Set(vm, exports, "STATUS_CODES", statusCodes)
_ = native.Set(vm, exports, "maxHeaderSize", 1<<20)
_ = native.Set(vm, exports, "validateHeaderName", native.Bind(vm, "validateHeaderName", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
if !validHTTPToken(name) {
panic(vm.NewTypeError("Invalid HTTP header name"))
panic(native.TypeError(vm, "Invalid HTTP header name"))
}
})
_ = exports.Set("validateHeaderValue", func(name, value string) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, exports, "validateHeaderValue", native.Bind(vm, "validateHeaderValue", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
value := native.String(vm, moejs.Arg(args, 1))
if strings.ContainsAny(value, "\x00\r\n") {
panic(vm.NewTypeError("Invalid value for header %s", name))
panic(native.TypeError(vm, "Invalid value for header %s", name))
}
})
return moejs.Undefined(), nil
}))
m.attachHTTPClient(vm, exports)
_ = module.Set("exports", exports)
_ = native.Set(vm, module, "exports", exports)
}
func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
func (m *Module) newHTTPServer(vm *moejs.Runtime) *engine.Object {
serverObject := events.NewEmitter(vm)
var server *http.Server
var listener net.Listener
@@ -89,35 +94,38 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
var listenGeneration uint64
var listenPending bool
var readTimeout time.Duration
_ = serverObject.Set("listening", false)
_ = serverObject.Set("timeout", 0)
_ = serverObject.Set("keepAliveTimeout", 5000)
_ = serverObject.Set("headersTimeout", 60000)
_ = serverObject.Set("requestTimeout", 300000)
_ = native.Set(vm, serverObject, "listening", false)
_ = native.Set(vm, serverObject, "timeout", 0)
_ = native.Set(vm, serverObject, "keepAliveTimeout", 5000)
_ = native.Set(vm, serverObject, "headersTimeout", 60000)
_ = native.Set(vm, serverObject, "requestTimeout", 300000)
handler := http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
body, err := httpbody.ReadAll(request.Body, m.maxBodyBytes)
if err != nil {
http.Error(response, err.Error(), http.StatusRequestEntityTooLarge)
return
}
state := &nodeHTTPResponse{headers: make(http.Header), statusCode: http.StatusOK, done: make(chan struct{})}
queued := m.runtime.RunOnLoop(func(vm *goja.Runtime) {
state := &nodeHTTPResponse{headers: make(http.Header), statusCode: http.StatusOK, done: make(chan struct {
})}
queued := m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
incoming := m.httpIncomingMessage(vm, request, body)
outgoing := m.httpServerResponse(vm, state)
err := m.runtime.RunJob(vm, "http.Server request", func() error { return events.Emit(vm, serverObject, "request", incoming, outgoing) })
err := m.runtime.RunJob(vm, "http.Server request", func() error {
return events.Emit(vm, serverObject, "request", incoming, outgoing)
})
if err != nil {
state.finish(http.StatusInternalServerError, nil)
return
}
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.IncomingMessage body", func() error {
push, _ := goja.AssertFunction(incoming.Get("push"))
push, _ := callable(native.Get(vm, incoming, "push"))
if len(body) > 0 {
if _, pushErr := push(incoming, buffer.WrapBytes(vm, append([]byte(nil), body...))); pushErr != nil {
if _, pushErr := native.Call(vm, push, native.From(vm, incoming), native.From(vm, buffer.WrapBytes(vm, append([]byte(nil), body...)))); pushErr != nil {
return pushErr
}
}
_, pushErr := push(incoming, goja.Null())
_, pushErr := native.Call(vm, push, native.From(vm, incoming), native.From(vm, moejs.Null()))
return pushErr
})
}, 0)
@@ -147,15 +155,15 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
response.WriteHeader(statusCode)
_, _ = response.Write(responseBody)
})
_ = serverObject.Set("listen", func(call goja.FunctionCall) goja.Value {
_ = native.Set(vm, serverObject, "listen", native.Bind(vm, "listen", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if !m.allowListen {
panic(vm.NewGoError(fmt.Errorf("http.Server.listen requires AllowListen")))
panic(native.GoError(vm, fmt.Errorf("http.Server.listen requires AllowListen")))
}
address, callback := listenArguments(vm, call)
address, callback := listenArguments(vm, args)
serverMu.Lock()
if listenPending || listener != nil {
serverMu.Unlock()
panic(vm.NewGoError(fmt.Errorf("server is already listening")))
panic(native.GoError(vm, fmt.Errorf("server is already listening")))
}
listenGeneration++
generation := listenGeneration
@@ -172,19 +180,15 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
}
serverMu.Unlock()
if active {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server error", func() error {
return events.Emit(vm, serverObject, "error", vm.NewGoError(err))
return events.Emit(vm, serverObject, "error", native.GoError(vm, err))
})
})
}
return
}
createdServer := &http.Server{
Handler: handler,
ReadHeaderTimeout: m.runtime.Timeout(),
ReadTimeout: configuredReadTimeout,
}
createdServer := &http.Server{Handler: handler, ReadHeaderTimeout: m.runtime.Timeout(), ReadTimeout: configuredReadTimeout}
serverMu.Lock()
if !listenPending || listenGeneration != generation {
serverMu.Unlock()
@@ -205,19 +209,19 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
resourceID = id
listenPending = false
serverMu.Unlock()
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
serverMu.RLock()
active := listenGeneration == generation && server == createdServer && listener == created
serverMu.RUnlock()
if !active {
return
}
if callback != nil {
once, _ := goja.AssertFunction(serverObject.Get("once"))
_, _ = once(serverObject, vm.ToValue("listening"), vm.ToValue(callback))
if !native.IsNil(callback) {
once, _ := callable(native.Get(vm, serverObject, "once"))
_, _ = native.Call(vm, once, native.From(vm, serverObject), native.From(vm, "listening"), native.From(vm, callback))
}
_ = m.runtime.RunJob(vm, "http.Server listening", func() error {
_ = serverObject.Set("listening", true)
_ = native.Set(vm, serverObject, "listening", true)
return events.Emit(vm, serverObject, "listening")
})
})
@@ -232,19 +236,19 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
m.runtime.RemoveResource(id)
}
if active && err != nil && err != http.ErrServerClosed {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server error", func() error {
return events.Emit(vm, serverObject, "error", vm.NewGoError(err))
return events.Emit(vm, serverObject, "error", native.GoError(vm, err))
})
})
}
}()
return serverObject
})
_ = serverObject.Set("close", func(call goja.FunctionCall) goja.Value {
if callback, ok := goja.AssertFunction(call.Argument(0)); ok {
once, _ := goja.AssertFunction(serverObject.Get("once"))
_, _ = once(serverObject, vm.ToValue("close"), vm.ToValue(callback))
return native.From(vm, serverObject), nil
}))
_ = native.Set(vm, serverObject, "close", native.Bind(vm, "close", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if callback, ok := callable(moejs.Arg(args, 0)); ok {
once, _ := callable(native.Get(vm, serverObject, "once"))
_, _ = native.Call(vm, once, native.From(vm, serverObject), native.From(vm, "close"), native.From(vm, callback))
}
serverMu.Lock()
wasActive := listenPending || listener != nil
@@ -262,55 +266,59 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
if currentResourceID != 0 {
m.runtime.RemoveResource(currentResourceID)
}
_ = serverObject.Set("listening", false)
_ = native.Set(vm, serverObject, "listening", false)
if wasActive {
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server close", func() error { return events.Emit(vm, serverObject, "close") })
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server close", func() error {
return events.Emit(vm, serverObject, "close")
})
}, 0)
}
return serverObject
})
_ = serverObject.Set("closeAllConnections", func() {
return native.From(vm, serverObject), nil
}))
_ = native.Set(vm, serverObject, "closeAllConnections", native.Bind(vm, "closeAllConnections", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
serverMu.RLock()
current := server
serverMu.RUnlock()
if current != nil {
_ = current.Close()
}
})
_ = serverObject.Set("closeIdleConnections", func() {
panic(vm.NewGoError(fmt.Errorf("http.Server.closeIdleConnections is not supported by jsruntime; net/http does not expose individual idle server connections")))
})
_ = serverObject.Set("address", func() goja.Value {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, serverObject, "closeIdleConnections", native.Bind(vm, "closeIdleConnections", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.Server.closeIdleConnections is not supported by jsruntime; net/http does not expose individual idle server connections")))
}))
_ = native.Set(vm, serverObject, "address", native.Bind(vm, "address", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
serverMu.RLock()
current := listener
serverMu.RUnlock()
if current == nil {
return goja.Null()
return native.From(vm, moejs.Null()), nil
}
host, port, _ := net.SplitHostPort(current.Addr().String())
portNumber, _ := strconv.Atoi(port)
return vm.ToValue(map[string]any{"address": host, "family": addressFamily(host), "port": portNumber})
})
_ = serverObject.Set("setTimeout", func(milliseconds int64, callback goja.Value) *goja.Object {
return native.From(vm, map[string]any{"address": host, "family": addressFamily(host), "port": portNumber}), nil
}))
_ = native.Set(vm, serverObject, "setTimeout", native.Bind(vm, "setTimeout", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
milliseconds := native.Int(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
serverMu.Lock()
readTimeout = time.Duration(milliseconds) * time.Millisecond
serverMu.Unlock()
if function, ok := goja.AssertFunction(callback); ok {
on, _ := goja.AssertFunction(serverObject.Get("on"))
_, _ = on(serverObject, vm.ToValue("timeout"), vm.ToValue(function))
if function, ok := callable(callback); ok {
on, _ := callable(native.Get(vm, serverObject, "on"))
_, _ = native.Call(vm, on, native.From(vm, serverObject), native.From(vm, "timeout"), native.From(vm, function))
}
return serverObject
})
_ = serverObject.Set("ref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("http.Server.ref is not supported by jsruntime; the event loop is host-driven")))
})
_ = serverObject.Set("unref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("http.Server.unref is not supported by jsruntime; the event loop is host-driven")))
})
return native.From(vm, serverObject), nil
}))
_ = native.Set(vm, serverObject, "ref", native.Bind(vm, "ref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.Server.ref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, serverObject, "unref", native.Bind(vm, "unref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.Server.unref is not supported by jsruntime; the event loop is host-driven")))
}))
return serverObject
}
func (state *nodeHTTPResponse) finish(status int, body []byte) {
state.mu.Lock()
if state.ended {
@@ -327,14 +335,13 @@ func (state *nodeHTTPResponse) finish(status int, body []byte) {
close(state.done)
state.mu.Unlock()
}
func (m *Module) httpIncomingMessage(vm *goja.Runtime, request *http.Request, body []byte) *goja.Object {
func (m *Module) httpIncomingMessage(vm *moejs.Runtime, request *http.Request, body []byte) *engine.Object {
m.initHTTPStreams(vm)
value, err := m.createIncoming(goja.Undefined())
value, err := native.Call(vm, m.createIncoming, native.From(vm, moejs.Undefined()))
if err != nil {
panic(err)
}
incoming := value.ToObject(vm)
incoming := native.Object(vm, value)
headers := make(map[string]string, len(request.Header))
rawHeaders := make([]string, 0, len(request.Header)*2)
for name, values := range request.Header {
@@ -343,26 +350,28 @@ func (m *Module) httpIncomingMessage(vm *goja.Runtime, request *http.Request, bo
rawHeaders = append(rawHeaders, name, value)
}
}
_ = incoming.Set("method", request.Method)
_ = incoming.Set("url", request.URL.RequestURI())
_ = incoming.Set("headers", headers)
_ = incoming.Set("headersDistinct", request.Header.Clone())
_ = incoming.Set("rawHeaders", rawHeaders)
_ = incoming.Set("httpVersion", fmt.Sprintf("%d.%d", request.ProtoMajor, request.ProtoMinor))
_ = incoming.Set("httpVersionMajor", request.ProtoMajor)
_ = incoming.Set("httpVersionMinor", request.ProtoMinor)
_ = incoming.Set("complete", true)
_ = incoming.Set("aborted", false)
_ = incoming.Set("socket", httpSocketInfo(vm, request))
_ = incoming.Set("connection", incoming.Get("socket"))
_ = incoming.Set("_destroy", func(err goja.Value, callback goja.Callable) {
_ = native.Set(vm, incoming, "method", request.Method)
_ = native.Set(vm, incoming, "url", request.URL.RequestURI())
_ = native.Set(vm, incoming, "headers", headers)
_ = native.Set(vm, incoming, "headersDistinct", request.Header.Clone())
_ = native.Set(vm, incoming, "rawHeaders", rawHeaders)
_ = native.Set(vm, incoming, "httpVersion", fmt.Sprintf("%d.%d", request.ProtoMajor, request.ProtoMinor))
_ = native.Set(vm, incoming, "httpVersionMajor", request.ProtoMajor)
_ = native.Set(vm, incoming, "httpVersionMinor", request.ProtoMinor)
_ = native.Set(vm, incoming, "complete", true)
_ = native.Set(vm, incoming, "aborted", false)
_ = native.Set(vm, incoming, "socket", httpSocketInfo(vm, request))
_ = native.Set(vm, incoming, "connection", native.Get(vm, incoming, "socket"))
_ = native.Set(vm, incoming, "_destroy", native.Bind(vm, "_destroy", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
err := moejs.Arg(args, 0)
callback := moejs.Arg(args, 1)
_ = request.Body.Close()
_, _ = callback(goja.Undefined(), err)
})
_, _ = native.Call(vm, callback, native.From(vm, moejs.Undefined()), native.From(vm, err))
return moejs.Undefined(), nil
}))
return incoming
}
func httpSocketInfo(vm *goja.Runtime, request *http.Request) *goja.Object {
func httpSocketInfo(vm *moejs.Runtime, request *http.Request) *engine.Object {
socket := events.NewEmitter(vm)
remoteHost, remotePort, _ := net.SplitHostPort(request.RemoteAddr)
localHost, localPort := "", 0
@@ -370,161 +379,185 @@ func httpSocketInfo(vm *goja.Runtime, request *http.Request) *goja.Object {
localHost, localPort = splitAddress(local)
}
remotePortNumber, _ := strconv.Atoi(remotePort)
_ = socket.Set("remoteAddress", remoteHost)
_ = socket.Set("remotePort", remotePortNumber)
_ = socket.Set("remoteFamily", addressFamily(remoteHost))
_ = socket.Set("localAddress", localHost)
_ = socket.Set("localPort", localPort)
_ = socket.Set("localFamily", addressFamily(localHost))
_ = socket.Set("encrypted", request.TLS != nil)
_ = native.Set(vm, socket, "remoteAddress", remoteHost)
_ = native.Set(vm, socket, "remotePort", remotePortNumber)
_ = native.Set(vm, socket, "remoteFamily", addressFamily(remoteHost))
_ = native.Set(vm, socket, "localAddress", localHost)
_ = native.Set(vm, socket, "localPort", localPort)
_ = native.Set(vm, socket, "localFamily", addressFamily(localHost))
_ = native.Set(vm, socket, "encrypted", request.TLS != nil)
return socket
}
func (m *Module) httpServerResponse(vm *goja.Runtime, state *nodeHTTPResponse) *goja.Object {
func (m *Module) httpServerResponse(vm *moejs.Runtime, state *nodeHTTPResponse) *engine.Object {
m.initHTTPStreams(vm)
hooks := vm.NewObject()
_ = hooks.Set("write", func(response *goja.Object, chunk goja.Value, callback goja.Callable) {
hooks := vm.Realm().NewObject()
_ = native.Set(vm, hooks, "write", native.Bind(vm, "write", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
response := native.Object(vm, moejs.Arg(args, 0))
chunk := moejs.Arg(args, 1)
callback := moejs.Arg(args, 2)
state.mu.Lock()
_, _ = state.body.Write(buffer.Bytes(vm, chunk))
state.mu.Unlock()
_ = response.Set("headersSent", true)
if callback != nil {
_, _ = callback(goja.Undefined())
_ = native.Set(vm, response, "headersSent", true)
if !native.IsNil(callback) {
_, _ = native.Call(vm, callback, native.From(vm, moejs.Undefined()))
}
})
_ = hooks.Set("finish", func(response *goja.Object, callback goja.Callable) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "finish", native.Bind(vm, "finish", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
response := native.Object(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
state.mu.Lock()
state.statusCode = int(response.Get("statusCode").ToInteger())
state.statusText = response.Get("statusMessage").String()
state.statusCode = int(native.Int(vm, native.Get(vm, response, "statusCode")))
state.statusText = native.String(vm, native.Get(vm, response, "statusMessage"))
if !state.ended {
state.ended = true
close(state.done)
}
state.mu.Unlock()
_ = response.Set("headersSent", true)
if callback != nil {
_, _ = callback(goja.Undefined())
_ = native.Set(vm, response, "headersSent", true)
if !native.IsNil(callback) {
_, _ = native.Call(vm, callback, native.From(vm, moejs.Undefined()))
}
})
_ = hooks.Set("destroy", func(*goja.Object) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "destroy", native.Bind(vm, "destroy", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.finish(0, nil)
})
value, err := m.createResponse(goja.Undefined(), hooks)
return moejs.Undefined(), nil
}))
value, err := native.Call(vm, m.createResponse, native.From(vm, moejs.Undefined()), native.From(vm, hooks))
if err != nil {
panic(err)
}
response := value.ToObject(vm)
_ = response.Set("statusCode", http.StatusOK)
_ = response.Set("statusMessage", "")
_ = response.Set("headersSent", false)
_ = response.Set("sendDate", true)
_ = response.Set("setHeader", func(name string, value goja.Value) *goja.Object {
response := native.Object(vm, value)
_ = native.Set(vm, response, "statusCode", http.StatusOK)
_ = native.Set(vm, response, "statusMessage", "")
_ = native.Set(vm, response, "headersSent", false)
_ = native.Set(vm, response, "sendDate", true)
_ = native.Set(vm, response, "setHeader", native.Bind(vm, "setHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
value := moejs.Arg(args, 1)
state.mu.Lock()
defer state.mu.Unlock()
var values []string
if err := vm.ExportTo(value, &values); err == nil {
if err := native.ExportTo(vm, value, &values); err == nil {
state.headers[name] = values
} else {
state.headers.Set(name, value.String())
state.headers.Set(name, native.String(vm, value))
}
return response
})
_ = response.Set("appendHeader", func(name, value string) *goja.Object {
return native.From(vm, response), nil
}))
_ = native.Set(vm, response, "appendHeader", native.Bind(vm, "appendHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
value := native.String(vm, moejs.Arg(args, 1))
state.mu.Lock()
state.headers.Add(name, value)
state.mu.Unlock()
return response
})
_ = response.Set("getHeader", func(name string) goja.Value {
return native.From(vm, response), nil
}))
_ = native.Set(vm, response, "getHeader", native.Bind(vm, "getHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
defer state.mu.Unlock()
values := state.headers.Values(name)
if len(values) == 0 {
return goja.Undefined()
return native.From(vm, moejs.Undefined()), nil
}
if len(values) == 1 {
return vm.ToValue(values[0])
return native.From(vm, values[0]), nil
}
return vm.ToValue(values)
})
_ = response.Set("getHeaders", func() http.Header { state.mu.Lock(); defer state.mu.Unlock(); return state.headers.Clone() })
_ = response.Set("getHeaderNames", func() []string {
return native.From(vm, values), nil
}))
_ = native.Set(vm, response, "getHeaders", native.Bind(vm, "getHeaders", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.mu.Lock()
defer state.mu.Unlock()
return native.From(vm, state.headers.Clone()), nil
}))
_ = native.Set(vm, response, "getHeaderNames", native.Bind(vm, "getHeaderNames", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.mu.Lock()
defer state.mu.Unlock()
names := make([]string, 0, len(state.headers))
for name := range state.headers {
names = append(names, strings.ToLower(name))
}
return names
})
_ = response.Set("hasHeader", func(name string) bool {
return native.From(vm, names), nil
}))
_ = native.Set(vm, response, "hasHeader", native.Bind(vm, "hasHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
defer state.mu.Unlock()
return len(state.headers.Values(name)) > 0
})
_ = response.Set("removeHeader", func(name string) { state.mu.Lock(); state.headers.Del(name); state.mu.Unlock() })
_ = response.Set("writeHead", func(call goja.FunctionCall) goja.Value {
_ = response.Set("statusCode", call.Argument(0).ToInteger())
if text := call.Argument(1); !goja.IsUndefined(text) {
if _, ok := text.(*goja.Object); ok {
return native.From(vm, len(state.headers.Values(name)) > 0), nil
}))
_ = native.Set(vm, response, "removeHeader", native.Bind(vm, "removeHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
state.headers.Del(name)
state.mu.Unlock()
return moejs.Undefined(), nil
}))
_ = native.Set(vm, response, "writeHead", native.Bind(vm, "writeHead", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
_ = native.Set(vm, response, "statusCode", native.Int(vm, moejs.Arg(args, 0)))
if text := moejs.Arg(args, 1); !text.IsUndefined() {
if _, ok := asObject(text); ok {
applyHTTPHeaders(vm, response, text)
} else {
_ = response.Set("statusMessage", text.String())
applyHTTPHeaders(vm, response, call.Argument(2))
_ = native.Set(vm, response, "statusMessage", native.String(vm, text))
applyHTTPHeaders(vm, response, moejs.Arg(args, 2))
}
}
_ = response.Set("headersSent", true)
return response
})
_ = response.Set("flushHeaders", func() { _ = response.Set("headersSent", true) })
_ = response.Set("writeContinue", func() {
panic(vm.NewGoError(fmt.Errorf("http.ServerResponse.writeContinue is not supported by jsruntime; the Go server never sends 100 Continue")))
})
_ = response.Set("setTimeout", func(milliseconds int64, callback goja.Value) *goja.Object {
if function, ok := goja.AssertFunction(callback); ok {
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
_ = native.Set(vm, response, "headersSent", true)
return native.From(vm, response), nil
}))
_ = native.Set(vm, response, "flushHeaders", native.Bind(vm, "flushHeaders", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
_ = native.Set(vm, response, "headersSent", true)
return moejs.Undefined(), nil
}))
_ = native.Set(vm, response, "writeContinue", native.Bind(vm, "writeContinue", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.ServerResponse.writeContinue is not supported by jsruntime; the Go server never sends 100 Continue")))
}))
_ = native.Set(vm, response, "setTimeout", native.Bind(vm, "setTimeout", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
milliseconds := native.Int(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
if function, ok := callable(callback); ok {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.ServerResponse timeout", func() error {
_, err := function(response)
_, err := native.Call(vm, function, native.From(vm, response))
return err
})
}, time.Duration(milliseconds)*time.Millisecond)
}
return response
})
return native.From(vm, response), nil
}))
return response
}
func applyHTTPHeaders(vm *goja.Runtime, response *goja.Object, value goja.Value) {
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func applyHTTPHeaders(vm *moejs.Runtime, response *engine.Object, value moejs.Value) {
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return
}
setHeader, _ := goja.AssertFunction(response.Get("setHeader"))
object := value.ToObject(vm)
for _, name := range object.Keys() {
_, _ = setHeader(response, vm.ToValue(name), object.Get(name))
setHeader, _ := callable(native.Get(vm, response, "setHeader"))
object := native.Object(vm, value)
for _, name := range objectKeys(object) {
_, _ = native.Call(vm, setHeader, native.From(vm, response), native.From(vm, name), native.From(vm, native.Get(vm, object, name)))
}
}
func listenArguments(vm *goja.Runtime, call goja.FunctionCall) (string, goja.Callable) {
func listenArguments(vm *moejs.Runtime, args []moejs.Value) (string, moejs.Value) {
host := "127.0.0.1"
port := int(call.Argument(0).ToInteger())
port := int(native.Int(vm, moejs.Arg(args, 0)))
callbackIndex := 1
if object, ok := call.Argument(0).(*goja.Object); ok {
port = int(object.Get("port").ToInteger())
if value := object.Get("host"); !goja.IsUndefined(value) {
host = value.String()
if object, ok := asObject(moejs.Arg(args, 0)); ok {
port = int(native.Int(vm, native.Get(vm, object, "port")))
if value := native.Get(vm, object, "host"); !value.IsUndefined() {
host = native.String(vm, value)
}
} else if value := call.Argument(1); !goja.IsUndefined(value) {
if _, ok := goja.AssertFunction(value); !ok {
host = value.String()
} else if value := moejs.Arg(args, 1); !value.IsUndefined() {
if _, ok := callable(value); !ok {
host = native.String(vm, value)
callbackIndex = 2
}
}
callback, _ := goja.AssertFunction(call.Argument(callbackIndex))
callback, _ := callable(moejs.Arg(args, callbackIndex))
return net.JoinHostPort(host, strconv.Itoa(port)), callback
}
func splitAddress(address net.Addr) (string, int) {
if address == nil {
return "", 0
@@ -533,30 +566,27 @@ func splitAddress(address net.Addr) (string, int) {
portNumber, _ := strconv.Atoi(port)
return host, portNumber
}
func addressFamily(host string) string {
if ip := net.ParseIP(host); ip != nil && ip.To4() == nil {
return "IPv6"
}
return "IPv4"
}
func (m *Module) attachHTTPClient(vm *goja.Runtime, exports *goja.Object) {
factoryValue, err := vm.RunString(httpClientSource)
func (m *Module) attachHTTPClient(vm *moejs.Runtime, exports *engine.Object) {
factoryValue, err := native.Run(vm, httpClientSource)
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
factory, _ := goja.AssertFunction(factoryValue)
client, err := factory(goja.Undefined(), require.Require(vm, "events"), vm.Get("fetch"), vm.Get("AbortController"), vm.Get("Blob"))
factory, _ := callable(factoryValue)
client, err := native.Call(vm, factory, native.From(vm, moejs.Undefined()), native.From(vm, require.Require(vm, "events")), native.From(vm, native.Global(vm, "fetch")), native.From(vm, native.Global(vm, "AbortController")), native.From(vm, native.Global(vm, "Blob")))
if err != nil {
panic(err)
}
clientObject := client.ToObject(vm)
for _, name := range clientObject.Keys() {
_ = exports.Set(name, clientObject.Get(name))
clientObject := native.Object(vm, client)
for _, name := range objectKeys(clientObject) {
_ = native.Set(vm, exports, name, native.Get(vm, clientObject, name))
}
}
func validHTTPToken(value string) bool {
if value == "" {
return false
@@ -569,33 +599,46 @@ func validHTTPToken(value string) bool {
return true
}
func (m *Module) initHTTPStreams(vm *moejs.Runtime) {
m.streamMu.Lock()
defer m.streamMu.Unlock()
if engine.IsCallable(m.createIncoming) {
return
}
factoryValue, err := native.Run(vm, httpStreamsSource)
if err != nil {
panic(native.GoError(vm, fmt.Errorf("load http streams: %w", err)))
}
factory, _ := callable(factoryValue)
value, err := native.Call(vm, factory, native.From(vm, moejs.Undefined()), native.From(vm, require.Require(vm, "stream")))
if err != nil {
panic(err)
}
object := native.Object(vm, value)
m.createIncoming, _ = callable(native.Get(vm, object, "createIncomingMessage"))
m.createResponse, _ = callable(native.Get(vm, object, "createServerResponse"))
if !engine.IsCallable(m.createIncoming) || !engine.IsCallable(m.createResponse) {
panic(native.GoError(vm, fmt.Errorf("http streams factory is incomplete")))
}
}
func callable(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
func asObject(v moejs.Value) (*engine.Object, bool) {
if !v.IsObject() {
return nil, false
}
return v.AsObject(), true
}
func objectKeys(o *engine.Object) []string {
keys := o.OwnEnumerableStringKeys()
out := make([]string, len(keys))
for i, k := range keys {
out[i] = k.GoString()
}
return out
}
//go:embed client.js
var httpClientSource string
//go:embed streams.js
var httpStreamsSource string
// initHTTPStreams loads the embedded factory that builds IncomingMessage
// (Readable) and ServerResponse (Writable) instances from the stream module.
func (m *Module) initHTTPStreams(vm *goja.Runtime) {
m.streamMu.Lock()
defer m.streamMu.Unlock()
if m.createIncoming != nil {
return
}
factoryValue, err := vm.RunString(httpStreamsSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load http streams: %w", err)))
}
factory, _ := goja.AssertFunction(factoryValue)
value, err := factory(goja.Undefined(), require.Require(vm, "stream"))
if err != nil {
panic(err)
}
object := value.ToObject(vm)
m.createIncoming, _ = goja.AssertFunction(object.Get("createIncomingMessage"))
m.createResponse, _ = goja.AssertFunction(object.Get("createServerResponse"))
if m.createIncoming == nil || m.createResponse == nil {
panic(vm.NewGoError(fmt.Errorf("http streams factory is incomplete")))
}
}
Loaded 100 of 291 files, more files were not shown because too many files have changed in this diff. Show more