Compare commits

...
122 Commits
Author SHA1 Message Date
Akizon77 be3daa42fc remove unused database connection parameters 2026-07-17 17:22:24 +08:00
Akizon77 42b290f169 refactor: standardize time fields 2026-07-17 16:08:15 +08:00
Akizon77 99fda6b023 refactor: clean up metric constants and refactor record downsampling 2026-07-17 13:45:48 +08:00
Akizon77 7f128509e9 Revert "feat(admin): add database compression inspection and configuration endpoints"
This reverts commit fee8c860d7.
2026-07-17 13:14:03 +08:00
Akizon77 fee8c860d7 feat(admin): add database compression inspection and configuration endpoints
Add GET and POST routes for /admin/database/compression to allow
viewing current compression settings and applying new compression
configuration. Implement adminGetDatabaseCompression and
adminSetDatabaseCompression RPC methods, including validation,
locking, and audit logging.
2026-07-17 13:06:00 +08:00
Akizon77 3c278230f4 feat: add legacy upgrade server and refactor OAuth initialization
Introduce a restricted HTTP server to handle migration of legacy monitoring data
before the main application starts. Refactor OAuth initialization into a reusable
initOAuth method to support being called early by the upgrade server without
duplicating cleanup registrations.
2026-07-17 11:39:08 +08:00
Akizon77 d853a83ec6 refactor(jsonrpc): remove deprecated Tag field from public metric types
The Tag field was an alias for Tags and caused duplication in JSON output and internal logic. Removed it entirely to simplify the data model and avoid confusion, using Tags consistently everywhere.
2026-07-17 10:02:32 +08:00
Akizon77 a78ef878eb feat(metricstore): batch metric report writes in single transactions
Collect reports from agents for a short window (3s) and flush them together to reduce database transaction overhead. Add StartReportBatcher, StopReportBatcher, and FlushReportBatch for lifecycle control. Update WriteReport to enqueue when batcher is active, falling back to immediate write. Integrate startup/cleanup in cmd/app.go.
2026-07-17 09:45:35 +08:00
Akizon77 312e320c8f refactor: remove deprecated WriteRecord/WriteGPURecord and refactor cron jobs
Remove unused `WriteRecord` and `WriteGPURecord` functions from metric store, delete `minuteScheduledWork` wrapper, and directly schedule `notifier.CheckTraffic` as a separate cron job. Also drop the unused `report_cache` import.
2026-07-17 00:01:42 +08:00
Akizon77 73c3c436a4 Make metric chart gaps adaptive 2026-07-16 22:18:35 +08:00
Akizon77 f42cfe7ddc refactor: remove Nezha compatibility 2026-07-16 21:31:12 +08:00
Akizon77 0f6f2f4294 refactor: remove unused database functions for sessions, clients, and metric store 2026-07-16 21:13:02 +08:00
Akizon77 fc3febc3be feat: #414 add support for loong64 architecture 2026-07-16 20:23:19 +08:00
Akizon77 5b90804a13 feat(metric): add compaction watermark upsert and update storage queries 2026-07-16 19:36:09 +08:00
Akizon77 43547b947f feat: clean metric store before deleting client or ping task
When deleting a client or a ping task, the associated metric records in the independent metric store were not removed, leaving orphaned data. This change adds cleanup calls to delete metric records before removing the entity from the main database, ensuring data consistency.
2026-07-15 23:13:37 +08:00
Akizon77 e2a8284c03 fix(metricstore): use Series to read ping records for rollup support
Switch GetPingRecords from raw point query to Series with aggregation to correctly read both raw and rolled-up ping latency data. This prevents data loss after metric rollup compaction removes old raw points. Add pingQueryInterval helper to choose an appropriate aggregation interval based on the query time range, preserving the task_id tag and returning results in descending time order.
2026-07-15 23:05:49 +08:00
Akizon77 c1178ebfe1 fix: reduce default built-in metric retention to 1 day for testing 2026-07-15 22:45:51 +08:00
Akizon77 7baf2edf26 refactor(metrics): remove DefaultRetentionDays config option and enforce explicit retention
The `DefaultRetentionDays` configuration parameter has been removed. All metrics must now provide an explicit `RetentionDays` value upon creation. This simplifies the codebase by eliminating implicit defaults and the associated special-case logic in metric migration. The scheduled cleanup of task results now uses a hardcoded constant (30 days) instead of relying on retention summaries. Test files have been updated to set `RetentionDays` explicitly where previously omitted.
2026-07-15 21:59:49 +08:00
Akizon77 66b7691e01 fix: preserve zero-retention metric definitions across operations
When a metric has RetentionDays set to 0 (disabled), the system now correctly preserves that state during definition creation, migration between stores, and compaction. Previously, zero retention was either ignored or overwritten, leading to unintended retention periods and potential data leak. This change ensures disabled metrics remain disabled and that compaction deletes their series explicitly.
2026-07-15 21:05:56 +08:00
Akizon77 7f368817e0 refactor: replace static retention config with dynamic retention summary
Remove MetricStoreConfig.RetentionDays and related constants. Use metricstore.GetRetentionSummary to determine retention for task result cleanup. Adjust default rollup policy to use fixed durations instead of computing from config, simplifying the policy and relying on the metric store's own retention logic.
2026-07-15 20:50:53 +08:00
Akizon 5fa59ab7de Merge pull request #602 from sanrokamlan-prog/fix/visitor-audit-hardening
fix: harden visitor audit logging
2026-07-15 14:22:14 +08:00
sanrokamlan 0c80f0f23a fix: harden visitor audit logging 2026-07-15 12:44:24 +08:00
Akizon 1eb3549ed9 Merge pull request #597 from sanrokamlan-prog/visitor-audit-rpc
feat(rpc): add public visitor audit event endpoint
2026-07-15 10:57:55 +08:00
Akizon 5c73eba475 Update generate-release-notes.yml 2026-07-13 21:04:19 +08:00
sanrokamlanandClaude Opus 4.8 76794305c8 feat: add visitor audit rpc endpoint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-13 13:41:09 +08:00
Akizon77 724dd3b666 disable downsampling by default 2026-07-12 23:01:23 +08:00
Akizon77 c828653d20 refactor: improve metric store lifecycle management with context-aware close and operation gate 2026-07-12 16:52:50 +08:00
Akizon77 ff358edd8b Fix metric rollup compaction and retention 2026-07-12 12:36:27 +08:00
Akizon77 3b48339209 fix(metrics): select compatible rollup intervals 2026-07-12 01:01:03 +08:00
Akizon77 42ef993cd6 feat(metricstore): add downsampling toggle, raise default retention to 90d, and cleanup expired data on compact 2026-07-11 23:48:58 +08:00
Akizon77 39c5f7bb69 Include commits in release notes 2026-07-11 22:38:09 +08:00
Akizon77 9831df0af8 feat(metric): preserve series identity in rollup queries
Add PreserveSeries support to rollup aggregation so rollup and mixed raw paths can keep entity/tag identities instead of always merging matched series. Thread series-aware rollup keys through folding and FillEmpty handling to keep output points aligned with raw aggregation behavior.
2026-07-11 22:00:49 +08:00
Akizon77 29eecf55f7 Use stable release base for snapshot notes 2026-07-11 21:49:39 +08:00
Akizon77 0660ee74d4 Read release note workflow inputs 2026-07-11 21:24:01 +08:00
Akizon77 5ade710f32 Improve release notes generation 2026-07-11 21:07:34 +08:00
Akizon77 855ff82f97 feat(metricstore): track ping packet loss in metrics
Add a ping loss gauge and write it alongside ping latency records so failed ping samples are captured as loss indicators.

Update ping deletion and aggregation paths to include loss data, keeping cleanup and public ping stats consistent across latency and packet loss metrics.
2026-07-11 20:00:01 +08:00
Akizon77 8910c2e9c7 ci: generate AI-assisted release notes from stable tags 2026-07-11 18:14:32 +08:00
Akizon77 b2fe9ab068 feat(metricstore): aggregate record queries via series rollups
Refactor record loading to query metric series with calculated downsample intervals and shared metric mapping. Add all-client time-range loading by discovering record entity IDs, and centralize record sorting to keep large range queries efficient and ordered.
2026-07-11 17:28:13 +08:00
Akizon77 8a529b4eb0 ci: reuse frontend and toolchain actions in workflows
Extract repeated frontend build, frontend restore, and Zig setup steps into shared GitHub Actions so CI workflows use the same artifact and toolchain setup path. Update binary and Docker builds to depend on the frontend artifact, and require an explicit Docker dispatch version for controlled image tagging.
2026-07-11 17:10:01 +08:00
Akizon77 46b4e37178 fix(metricstore): rotate compaction cursor per metric
Track compaction progress across individual metric definitions instead of treating the metric store as one operation. This preserves the rotating cursor through full cycles, resumes from the failed metric on errors, and reports how many rollup buckets were written when compaction fails.

Add a regression test covering cursor stability after a complete rotated compaction pass.
2026-07-11 16:48:10 +08:00
Akizon77 5ad791fe50 feat(metricstore): add scheduled rollup compaction 2026-07-11 16:16:38 +08:00
Akizon77 8911c08a5b fix(metric): preserve series identity during aggregation
Group aggregate buckets by metric, entity, and tag fingerprint so mixed-series inputs do not collapse into one bucket. Carry cloned tags into aggregate output and filled empty buckets to keep results deterministic and isolated by series.
2026-07-11 15:52:29 +08:00
Akizon77 7d95240bc1 feat: add metric retention policy management 2026-07-11 15:08:00 +08:00
Akizon77 7e0d542694 refactor(metrics): use metric retention for cleanup settings
Move scheduled cleanup to MetricStoreConfig.RetentionDays and apply the same cutoff to records, task results, and ping history. Expose derived legacy public settings for theme compatibility while removing record retention fields from Settings.
2026-07-10 23:38:46 +08:00
Akizon77 84130dfff5 fix: run metric store migrations before startup migration
Run one-shot metric store migrations after the metric store is initialized, before backend-switch startup migration begins.

Move legacy monitoring table cleanup out of dbcore initialization so old records are imported through the migration flow before being removed.
2026-07-09 22:55:46 +08:00
Akizon77 e0094447a6 Use UTC+8 for snapshot versions 2026-07-08 23:09:09 +08:00
Akizon77 ab31fc69fe refactor(oauth): remove Cloudflare Access provider support
Remove the Cloudflare Access provider registration and clean up any persisted CloudflareAccess OIDC config during initialization so deployments do not keep referencing the removed provider.

Add a database helper for deleting OIDC configs by name and drop now-unused OIDC/oauth2 dependencies from the module files.
2026-07-08 21:04:46 +08:00
Akizon77 73071ff1e8 refactor: remove built-in Cloudflare tunnel support
Remove cloudflared from the Docker image and server startup lifecycle so the app no longer downloads, starts, or shuts down a managed tunnel process.

Drop the Cloudflare tunnel token setting from config, migrations, and admin routing to keep configuration aligned with the removed integration.
2026-07-08 19:38:30 +08:00
Akizon77 4ad190a7f4 feat(metrics): always enable metric store with startup migration and version-based backup
Enable the metric store unconditionally, dropping the deprecated
metric_store_enabled toggle. When unconfigured it uses SQLite
(./data/metrics.db), otherwise the configured MySQL/PostgreSQL backend.
Initialization failure now aborts startup instead of silently falling
back to the legacy records table.

On startup, run a one-time migration that moves data from the previous
metrics target to the current one when the storage backend changes
(e.g. SQLite to MySQL). Migration failure also aborts startup with a
clear error. The legacy komari.db monitoring tables are fully retired.

Inject a build version ID via dbcore.SetVersionID before Initialize so
dbcore can auto-backup ./data on version upgrades. The version is now
persisted in the configs table (system_version key), replacing the
./data/.komari-version file so it travels with config backups.

Remove records.CompactRecord calls and drop obsolete report cache tests.
2026-07-08 08:37:20 +08:00
Akizon77 6ddb143365 refactor: split startup into app lifecycle stages 2026-07-07 21:44:53 +08:00
airium eb4ca83a13 ci: align snapshot workflow behavior 2026-07-07 18:12:11 +08:00
Akizon77 43c80d238b fix(migration): 恢复迁移时保留已迁移进度计数,避免从0重新累计 2026-07-01 23:28:01 +08:00
Akizon77 416ea871be feat(metrics): 支持迁移暂停/恢复,进度持久化与重启后孤儿状态修复 2026-07-01 23:06:07 +08:00
Akizon77 5c2bd4ec74 perf: 优化 metrics 迁移性能 - 批量写入 + keyset 分页 2026-07-01 22:24:21 +08:00
Akizon77 be6af33d85 merge: metrics DSN autodetect 2026-06-30 23:44:14 +08:00
Akizon77 d02d9347dc feat(metrics): infer DB driver from DSN automatically 2026-06-30 23:43:47 +08:00
Akizon77 64998b527c fix: avoid sqlite table locks during metric migration 2026-06-30 22:43:43 +08:00
Akizon77 9e5b6df196 feat(metrics): hot-reload metrics DB config with connection test before save 2026-06-30 21:28:04 +08:00
Akizon77 332053d8ec fix: avoid sqlite database is locked failures for record writes 2026-06-30 19:28:00 +08:00
Akizon b6fcdd396d Merge pull request #583 from Torther/main
fix(telegram): 非 200 响应时返回 API 错误描述而非仅状态码
2026-06-30 18:39:06 +08:00
Akizonandguocn 92ec774249 fix(theme): prevent path traversal in theme operations (#584)
DeleteTheme/UpdateTheme/SetTheme used the user-supplied theme name
(short) directly in filepath.Join("./data/theme", short) before
filesystem operations (os.RemoveAll / os.Stat). Without validation,
a crafted short value like "../../" normalizes outside the theme
directory, letting an authenticated admin delete or access arbitrary
directories within the process working directory.

Reuse the existing isValidThemeShort validator (allowlist of
[a-zA-Z0-9_-]) in all three handlers before the path is built.
Add a regression test covering traversal payloads and absolute paths.

Co-authored-by: guocn <guocn@users.noreply.github.com>
2026-06-30 18:38:20 +08:00
guocn 404d97d846 fix(theme): prevent path traversal in theme operations
DeleteTheme/UpdateTheme/SetTheme used the user-supplied theme name
(short) directly in filepath.Join("./data/theme", short) before
filesystem operations (os.RemoveAll / os.Stat). Without validation,
a crafted short value like "../../" normalizes outside the theme
directory, letting an authenticated admin delete or access arbitrary
directories within the process working directory.

Reuse the existing isValidThemeShort validator (allowlist of
[a-zA-Z0-9_-]) in all three handlers before the path is built.
Add a regression test covering traversal payloads and absolute paths.
2026-06-29 23:12:42 +08:00
Akizon77 0132d1cbd8 Merge branch 'refactor/metrics-unify-storage': unify storage through metric store + getNodes map 2026-06-29 21:54:00 +08:00
Akizon77 ff6daa6ce3 fix(rpc): common:getNodes returns uuid-keyed map instead of array
Each value still contains its own uuid field. Single-uuid query keeps returning a single object.
2026-06-29 21:52:16 +08:00
Akizon77 6a5f3ef04d refactor(metrics): route all storage through metric store when enabled
- Fix all-clients query bypassing metric store (empty uuid path)

- Write records/GPU to metric store when enabled; traffic baseline from metric store

- Fix traffic report bypassing metric store (would report zero traffic)

- Use dedicated gpu.device.usage metric to avoid collision with entity-level gpu.usage

- Remove legacy GPU tables (gpu_records/gpu_records_long_term) and related code

- Remove never-used dead code: RecordOne/RecordGPU/GetLatestRecord/GetAllRecords/SaveClientReport/SaveReport/ParseReport
2026-06-29 21:46:53 +08:00
Torther da047cf612 fix(telegram): include API error description in non-200 responses
SendTextMessage checked resp.StatusCode before decoding the JSON body, so non-200 responses returned only the status code without the error reason.

Move JSON decode before the status check. When the status is not 200, use the description field from the API response as the error message. Fall back to the original status code message if decode fails or description is empty.
2026-06-29 10:12:38 +08:00
Akizon77 f0877d6d32 fix(database): use FromTime() for time queries and improve cache cleanup 2026-06-28 22:28:57 +08:00
Akizon77 f2c46dbfb1 feat: route metric reads and writes to dedicated store 2026-06-28 22:08:35 +08:00
Akizon77 5f50d1ba00 Merge branch 'fix/temp-share-private-site' 2026-06-28 21:00:52 +08:00
Akizon77 7e19909dd6 fix: allow temp-share guests to bypass private site block
Private site dispatch only checked anonymous principal against the login whitelist and ignored meta.TempShareValid, so valid temp_key share links were rejected with -32041 on the analysis methods. Add !meta.TempShareValid to the gate; CheckPrincipal still limits anonymous callers to public:* (guest).
2026-06-28 21:00:41 +08:00
Akizon77 9db4daac10 ci: limit build workflow to PRs and fix snapshot release creation 2026-06-28 20:14:17 +08:00
Akizon 0a4cd6bd8c Merge pull request #578 from komari-monitor/feat/vacuum-database
feat: add database vacuum and size API endpoints
2026-06-28 19:33:44 +08:00
Akizon77 20c865ed20 feat(install): add TUI support with whiptail/dialog and text fallback 2026-06-28 19:32:07 +08:00
Akizon77 e740bf2797 feat(install): add stable/snapshot channel selection for binary install and upgrade 2026-06-28 19:28:43 +08:00
Akizon77 074958fad7 feat: add database vacuum and size API endpoints
Add admin RPC methods getDatabaseSize and vacuumDatabase, and register /api/admin/database/size and /api/admin/database/vacuum routes. VACUUM compacts the SQLite database to reclaim disk space.
2026-06-28 18:59:16 +08:00
Akizon77 11d3ba8236 fix(rpc): allow login-page endpoints in private-site mode
Fixes #567 where enabling private site blocked the login page itself,
making it impossible to log in from external IPs.

The Dispatch private-site check was too broad, blocking all anonymous
requests including the meta endpoints needed to render the login page
(getMe, getPublicSettings, getVersion). Now exempt these from the
private-site block via whitelist.

Adds regression test to guard the required whitelist entries.
2026-06-28 00:38:29 +08:00
Akizon77 ce41745edb refactor(rpc): migrate ACL to capability-based Principal model 2026-06-28 00:19:48 +08:00
Akizon77 036121c148 refactor(rpc): unify identity into Principal model 2026-06-27 23:50:21 +08:00
Akizonandyuanhhs dd0539392b Preserve RPC node order (#566)
Co-authored-by: yuanhhs <yuanhhs@users.noreply.github.com>
2026-06-27 23:23:36 +08:00
Akizon77 50ba8310ca Merge PR #571: fix(rpc): enforce sensitive-operation 2FA at the RPC boundary 2026-06-27 23:20:35 +08:00
carbofish 50fe286528 fix(rpc): enforce sensitive-operation 2FA at the RPC boundary
Sensitive methods (admin:exec) were gated by sensitive-operation 2FA only through REST route middleware. Move the enforcement to the JSON-RPC boundary so every entrypoint applies the same check:

  - pkg/rpc: add a transport-agnostic sensitive-method registry (MarkSensitive / IsSensitive).
  - web/rpc/jsonrpc: verify sensitive-operation 2FA in dispatchWithSensitive before dispatch, covering the REST bridge, /api/rpc2 POST and WebSocket.
  - mark admin:exec sensitive.

API-key callers and the existing no-2FA-configured semantics are preserved; non-sensitive methods are unaffected.
2026-06-27 05:18:45 +08:00
yuanhhs 144efdc8ad Preserve RPC node order 2026-06-22 07:42:49 +08:00
Akizon77 e446bee900 ci: expand release notes 2026-06-21 20:53:30 +08:00
Akizon77 a980befd60 fix(metric): isolate compaction scan and delete to prevent data loss
Add compactTxOptions to the dialect interface so the raw scan and raw
deletion in CompactMetric observe a consistent view. PostgreSQL and
MySQL escalate to SERIALIZABLE, ensuring a point inserted between the
scan and the deletion cannot be removed before it is rolled up; on
PostgreSQL this surfaces as a serialization failure that is retried,
while InnoDB blocks the conflicting insert via locking reads. SQLite
returns nil since its single write connection already serializes writes.

Add regression tests covering partial fine-row overwrites of coarse
rollups during compaction.
2026-06-21 17:00:01 +08:00
Akizon77 2e0c598dae feat(metric): support arbitrary percentile aggregations 2026-06-21 01:11:07 +08:00
Akizon77 771a192a3d chore(deps): upgrade Go and add SQL database drivers
Bump the module target to Go 1.25 and refresh dependency metadata.

Add MySQL, PostgreSQL, and SQLite drivers as direct dependencies to support SQL backends, while updating related transitive modules and checksums.
2026-06-20 20:11:16 +08:00
Akizon77 95cc65e28d docs: add abuse warning and remote control disclosure 2026-06-17 19:53:46 +08:00
airium b6e36ee0c4 Merge pull request #555 from airium/fix-traffic-report
fix: 修复流量报告为 0 B 问题
2026-06-17 19:21:30 +08:00
Akizon 25f3e1d2bd Merge pull request #551 from komari-monitor/fix/rpc2-route-contract-regressions
fix(router): 修复 RPC2 迁移引入的接口契约回归
2026-06-11 17:06:51 +08:00
Akizon77andClaude Opus 4.8 091eafa743 fix(router): 修复 RPC2 迁移引入的接口契约回归
对照 1.1.7 与重构前基线逐路由核对,修复 3 处契约破坏:

- /api/clients 是 WebSocket 端点(api.GetClients,客户端发 "get"/"get <uuid>"
  拉取在线列表与最新上报),迁移时误绑为 REST public:getNodesInformation,
  导致 wss:// 连接失败。还原为 api.GetClients。
- POST /api/admin/settings/oidc 与 /message-sender:原响应为
  {status,message:"",data:{message:"..."}},误用 WithFlat() 变成
  {status,message:"..."}(丢失 data 层)。移除 WithFlat,恢复 data.message 契约。

其余路由(WS/流端点、响应渲染器、完整路径)经与 1.1.7 全量比对确认一致。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 17:03:24 +08:00
Akizon cbe665591f Merge pull request #550 from komari-monitor/refactor/rpc2-unified-pipeline
refactor(rpc): 统一 RPC2 管道并全量迁移 REST 接口
2026-06-11 16:36:57 +08:00
Akizon77andClaude Opus 4.8 0abd9ff9bc refactor(rpc): 统一 RPC2 管道并全量迁移 REST 接口
将混乱的「Web 接口 → 业务逻辑」收敛为统一管道:
接口 → 生成 RPC 请求 → Dispatch(权限+命名空间) → RPC handler(业务),
为后续 JS 插件接入打地基(插件本身未实现)。

核心:
- pkg/rpc/permission.go:声明式 ACL + 通配符权限,按特异性裁决
  (精确 > 字面前缀更长的通配 > 全局 *)。Allow/RegisterNamespace 供插件扩展。
- pkg/rpc:新增 Unregister(动态注销,供插件卸载);ContextMeta 增 SessionToken/TempShareValid。
- web/rpc/jsonrpc 分层:dispatch(统一 Dispatch+OnInternalRequest)、
  transport(HTTP/WS+CallFromGin)、register、bridge(声明式路由桥)。
  WS 改同步写,删除旧 JsonRpc.go 的死代码 registry。

全量迁移:
- 所有 JSON 接口迁为 RPC2 方法(admin/public/client 三命名空间),
  router 经 bridge.Bind 直接绑定,删除整个 per-resource gin handler 层。
- 渲染器覆盖全部响应形状:standard/flat/raw/message。
- agent v1/v2 上报逻辑统一到 web/api/client/ingest.go。

文件收敛:
- web/api/admin 从 ~20 文件+5 子目录降到 6 个 REST-binary 文件
  (2fa/theme/download/upload/update/oauth),update 子包合并。
- 二进制/流/重定向/特殊鉴权类保留为 REST。

开发者文档见 web/rpc/jsonrpc/README.md。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:05:49 +08:00
airium 618ced3b8f test(auth): remove obsolete dev cookie bypass coverage (#547) 2026-06-11 12:22:27 +08:00
Akizon77 f7fd459723 fix(auth): only set secure session cookie over HTTPS 2026-06-10 20:48:43 +08:00
Akizon77 bb22a818bf fix(client): fill GeoIP region for v2 basic info 2026-06-10 01:12:02 +08:00
Akizon77 b66fe47487 fix(config): rename CORS toggle to origin check setting
Replace the legacy allow_cors setting with cors_origin_check_enabled
throughout server setup, public info output, and configuration constants.
This clarifies that the option controls API CORS origin validation rather
than broadly enabling CORS.
2026-06-09 17:27:06 +08:00
Akizon77 98122fa4d1 ci: generate release notes from GitHub 2026-06-09 16:46:33 +08:00
Akizon77 c3f65b1a32 fix go tests 2026-06-08 21:14:27 +08:00
Akizon77 7da616ce4c feat(security): add configurable CORS and origin checks
Add settings for allowed CORS origins and WebSocket origin validation so
deployments can restrict cross-origin access instead of using a global
wildcard policy.

Move CORS handling into the security middleware and update report cache
usage to use the dedicated report package.
2026-06-08 20:45:00 +08:00
airium 9d914e70b9 fix: harden traffic report accounting and validation (#542)
- Store per-record traffic deltas and sum them for scheduled reports
- Preserve traffic delta totals during long-term record compaction
- Compact only complete 15-minute buckets to avoid partial-bucket undercounts
- Serialize report saves per client so concurrent uploads cannot double-count deltas
- Enforce enabled traffic report cadence validation across edit and enable paths
- Add regression coverage for compaction boundaries, cadence validation, and client save locking
2026-06-08 11:26:13 +08:00
Akizon77 6f0624e9b4 Merge branch 'main' into codex/theme-configuration-types 2026-06-08 01:58:52 +08:00
Akizon77 3cc67e11cd fix: align theme redirect and html language handling 2026-06-08 01:53:28 +08:00
Akizon77 742a5e953e feat: support raw and redirect themes 2026-06-08 01:53:28 +08:00
Akizon77 3b82cee4b1 fix: align theme redirect and html language handling 2026-06-08 01:53:20 +08:00
Akizon77 2b010954a3 feat: support raw and redirect themes 2026-06-08 00:48:37 +08:00
Akizon77 6f23565517 refactor: isolate startup migrations 2026-06-08 00:01:32 +08:00
airium 7c2896c3fe fix(remote-exec): preserve command text during validation (#543)
Reject whitespace-only remote exec commands without trimming the submitted
command body, so multiline scripts and meaningful leading/trailing content are
preserved through dispatch.
2026-06-07 23:54:43 +08:00
Akizon77 721d61bc4e docs: update default theme build path 2026-06-06 22:59:38 +08:00
kylvenp 014b236745 feat: add network test rpc protocol structs 2026-06-06 21:34:48 +08:00
Akizon77 80c8d7e5ee Merge pull request #537 from airium/fix-http-development 2026-06-06 20:05:30 +08:00
Akizon77 8423881cc4 Merge pull request #541 from airium/support-phy-cpu-count 2026-06-06 19:59:16 +08:00
Akizon77 a1d644977c Merge pull request #538 from airium/fix-theme-path 2026-06-06 19:59:05 +08:00
Akizon77 301fb536f2 Merge pull request #539 from aijwfsibari/main 2026-06-06 19:53:17 +08:00
airium 49a8d772c8 feat: support physical CPU core count 2026-06-06 19:15:03 +08:00
aijwfsibari a9adae29d2 add Traffic report patch 2026-06-05 15:04:21 +08:00
airium d6e23baa19 fix: correct defaultTheme path across CI and docs
The Go embed directive in web/public/public.go resolves defaultTheme
relative to the package directory (web/public/defaultTheme/),
but all CI workflows and documentation referenced public/defaultTheme/
which would have caused builds to fail or embed an empty directory.
2026-06-02 13:24:23 +08:00
airium 658e7603f8 fix: allow insecure session cookies in development build 2026-06-02 02:31:01 +08:00
Akizon77 fb0b9b9910 feat(agent): add v2 pull events for queued agent tasks
Introduce agent.pull protocol types with event metadata and ACK fields.
Route ping dispatch through the agent API so tasks can be queued for
pull-based clients instead of only sent over active WebSocket sessions.
2026-05-31 19:01:02 +08:00
Akizon77 e149e8b8b7 refactor: remove legacy client APIs and support v2 pings
Remove deprecated client database helpers tied to common protocol types and
update report handling to use protocol/v1 models. Send ping tasks with the
v2 JSON-RPC payload for v2 clients while preserving existing behavior for
legacy websocket clients.
2026-05-31 16:50:04 +08:00
Akizon77 d2cc1503de refactor: centralize scheduled jobs with corn package
Replace ad-hoc ticker loops with named cron-based scheduled tasks for cleanup,
minute reporting, and expiration checks. Add scheduler shutdown handling and
log errors when schedules fail to reload or register.

Update imports to use the relocated pkg/config package across affected code.
2026-05-31 15:54:23 +08:00
Akizon77 f86803feff refactor(web): move HTTP route setup out of server
Relocate API routing, OAuth, and Nezha compatibility wiring into the web package to keep server startup focused on initialization concerns and improve code organization.
2026-05-31 15:35:47 +08:00
flyflas a2e3c7218d Merge pull request #525 from flyflas/feat/addXtermjsSetting
Feat/add xtermjs setting
2026-05-31 14:14:16 +08:00
282 changed files with 29506 additions and 9475 deletions

No files matched your search

+55
View File
@@ -0,0 +1,55 @@
name: Build frontend theme
description: Clone komari-web, build the default theme, and optionally upload it as an artifact.
inputs:
node-version:
description: Node.js version used to build komari-web.
default: "23"
repository:
description: komari-web repository URL.
default: https://github.com/komari-monitor/komari-web
artifact-name:
description: Optional artifact name for the packed frontend theme.
default: ""
runs:
using: composite
steps:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
- name: Clone and build frontend
shell: bash
run: |
set -euxo pipefail
rm -rf komari-web
git clone --depth=1 "${{ inputs.repository }}" komari-web
cd komari-web
npm install
npm run build
cd ..
mkdir -p web/public/defaultTheme/dist
rm -rf web/public/defaultTheme/dist/*
cp -r komari-web/dist/* web/public/defaultTheme/dist/
cp -f komari-web/komari-theme.json web/public/defaultTheme/
if [ -f komari-web/preview.png ]; then cp -f komari-web/preview.png web/public/defaultTheme/; fi
if [ -f komari-web/perview.png ]; then cp -f komari-web/perview.png web/public/defaultTheme/; fi
if [ -f web/public/defaultTheme/preview.png ] && [ ! -f web/public/defaultTheme/perview.png ]; then cp -f web/public/defaultTheme/preview.png web/public/defaultTheme/perview.png; fi
if [ -f web/public/defaultTheme/perview.png ] && [ ! -f web/public/defaultTheme/preview.png ]; then cp -f web/public/defaultTheme/perview.png web/public/defaultTheme/preview.png; fi
- name: Pack frontend artifact
if: ${{ inputs.artifact-name != '' }}
shell: bash
run: |
set -euxo pipefail
tar -czf "$RUNNER_TEMP/${{ inputs.artifact-name }}.tar.gz" -C web/public/defaultTheme .
- name: Upload frontend artifact
if: ${{ inputs.artifact-name != '' }}
uses: actions/upload-artifact@v4
with:
name: ${{ inputs.artifact-name }}
path: ${{ runner.temp }}/${{ inputs.artifact-name }}.tar.gz
@@ -0,0 +1,24 @@
name: Restore frontend theme
description: Download and unpack a previously built frontend theme artifact.
inputs:
artifact-name:
description: Frontend theme artifact name.
default: komari-frontend
runs:
using: composite
steps:
- name: Download frontend artifact
uses: actions/download-artifact@v4
with:
name: ${{ inputs.artifact-name }}
path: ${{ runner.temp }}/frontend-artifact
- name: Extract frontend artifact
shell: bash
run: |
set -euxo pipefail
mkdir -p web/public/defaultTheme
rm -rf web/public/defaultTheme/*
tar -xzf "$RUNNER_TEMP/frontend-artifact/${{ inputs.artifact-name }}.tar.gz" -C web/public/defaultTheme
+31
View File
@@ -0,0 +1,31 @@
name: Setup Zig
description: Restore or install the Zig toolchain used for CGO cross-compilation.
inputs:
zig-version:
description: Zig version to install.
default: 0.14.1
runs:
using: composite
steps:
- name: Restore Zig cache
id: zig-cache
uses: actions/cache@v4
with:
path: ~/.cache/komari-zig/zig-x86_64-linux-${{ inputs.zig-version }}
key: zig-${{ runner.os }}-x86_64-${{ inputs.zig-version }}
- name: Download Zig
if: ${{ steps.zig-cache.outputs.cache-hit != 'true' }}
shell: bash
run: |
set -euxo pipefail
mkdir -p "$HOME/.cache/komari-zig"
curl -fsSL "https://ziglang.org/download/${{ inputs.zig-version }}/zig-x86_64-linux-${{ inputs.zig-version }}.tar.xz" -o "$RUNNER_TEMP/zig.tar.xz"
tar -C "$HOME/.cache/komari-zig" -Jxf "$RUNNER_TEMP/zig.tar.xz"
- name: Add Zig to PATH
shell: bash
run: |
echo "$HOME/.cache/komari-zig/zig-x86_64-linux-${{ inputs.zig-version }}" >> "$GITHUB_PATH"
+21 -31
View File
@@ -1,16 +1,26 @@
name: Build Binaries on Main Push and PR
name: Build Binaries on PR
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:
jobs:
build-frontend:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Build frontend artifact
uses: ./.github/actions/build-frontend
with:
artifact-name: komari-frontend
build-binaries:
needs: build-frontend
runs-on: ubuntu-latest
strategy:
@@ -49,6 +59,10 @@ jobs:
goarch: riscv64
zig_target_triple: riscv64-linux-musl
binary_extension: ""
- goos: linux
goarch: loong64
zig_target_triple: loongarch64-linux-musl
binary_extension: ""
exclude:
- goos: windows
@@ -58,27 +72,8 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "23"
- name: Clone and build frontend
run: |
git clone https://github.com/komari-monitor/komari-web web
cd web
npm install
npm run build
cd ..
mkdir -p public/defaultTheme/dist
rm -rf public/defaultTheme/dist/*
cp -r web/dist/* public/defaultTheme/dist/
cp -f web/komari-theme.json public/defaultTheme/
if [ -f web/preview.png ]; then cp -f web/preview.png public/defaultTheme/; fi
if [ -f web/perview.png ]; then cp -f web/perview.png public/defaultTheme/; fi
# Compatibility: ensure both filenames exist if only one is present
if [ -f public/defaultTheme/preview.png ] && [ ! -f public/defaultTheme/perview.png ]; then cp -f public/defaultTheme/preview.png public/defaultTheme/perview.png; fi
if [ -f public/defaultTheme/perview.png ] && [ ! -f public/defaultTheme/preview.png ]; then cp -f public/defaultTheme/perview.png public/defaultTheme/preview.png; fi
- name: Restore frontend artifact
uses: ./.github/actions/restore-frontend
- name: Set up Go
uses: actions/setup-go@v5
@@ -86,12 +81,7 @@ jobs:
go-version: "1.23"
- name: Install Zig cross-compiler toolchain
run: |
sudo apt-get update # Update apt package lists
sudo apt-get install -y build-essential curl
wget https://ziglang.org/download/0.14.1/zig-x86_64-linux-0.14.1.tar.xz
sudo tar -C /usr/local -Jxf zig-x86_64-linux-0.14.1.tar.xz
echo "PATH=$PATH:/usr/local/zig-x86_64-linux-0.14.1" >> $GITHUB_ENV
uses: ./.github/actions/setup-zig
- name: Build binary
env:
+3 -17
View File
@@ -16,25 +16,11 @@ jobs:
#- name: Create dummy index.html
# run: |
# mkdir -p public/defaultTheme/dist
# echo "<html><body><h1>This is komari development server.</h1></body></html>" > public/defaultTheme/dist/index.html
# mkdir -p web/public/defaultTheme/dist
# echo "<html><body><h1>This is komari development server.</h1></body></html>" > web/public/defaultTheme/dist/index.html
- name: Clone and build frontend
run: |
git clone https://github.com/komari-monitor/komari-web web
cd web
npm install
npm run build
cd ..
mkdir -p public/defaultTheme/dist
rm -rf public/defaultTheme/dist/*
cp -r web/dist/* public/defaultTheme/dist/
cp -f web/komari-theme.json public/defaultTheme/
if [ -f web/preview.png ]; then cp -f web/preview.png public/defaultTheme/; fi
if [ -f web/perview.png ]; then cp -f web/perview.png public/defaultTheme/; fi
# Compatibility: ensure both filenames exist if only one is present
if [ -f public/defaultTheme/preview.png ] && [ ! -f public/defaultTheme/perview.png ]; then cp -f public/defaultTheme/preview.png public/defaultTheme/perview.png; fi
if [ -f public/defaultTheme/perview.png ] && [ ! -f public/defaultTheme/preview.png ]; then cp -f public/defaultTheme/perview.png public/defaultTheme/preview.png; fi
uses: ./.github/actions/build-frontend
- name: Set up Go
uses: actions/setup-go@v5
+39 -28
View File
@@ -11,37 +11,28 @@ env:
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push-multi-arch:
build-frontend:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
- name: Build frontend artifact
uses: ./.github/actions/build-frontend
with:
node-version: "23"
artifact-name: komari-frontend
- name: Clone and build frontend
run: |
git clone https://github.com/komari-monitor/komari-web web
cd web
npm install
npm run build
cd ..
mkdir -p public/defaultTheme/dist
rm -rf public/defaultTheme/dist/*
cp -r web/dist/* public/defaultTheme/dist/
cp -f web/komari-theme.json public/defaultTheme/
if [ -f web/preview.png ]; then cp -f web/preview.png public/defaultTheme/; fi
if [ -f web/perview.png ]; then cp -f web/perview.png public/defaultTheme/; fi
# Compatibility: ensure both filenames exist if only one is present
if [ -f public/defaultTheme/preview.png ] && [ ! -f public/defaultTheme/perview.png ]; then cp -f public/defaultTheme/preview.png public/defaultTheme/perview.png; fi
if [ -f public/defaultTheme/perview.png ] && [ ! -f public/defaultTheme/preview.png ]; then cp -f public/defaultTheme/perview.png public/defaultTheme/preview.png; fi
build-docker-binaries:
needs: build-frontend
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Restore frontend artifact
uses: ./.github/actions/restore-frontend
- name: Set up Go
uses: actions/setup-go@v5
@@ -49,10 +40,7 @@ jobs:
go-version: "1.23"
- name: Install Zig
run: |
wget https://ziglang.org/download/0.14.1/zig-x86_64-linux-0.14.1.tar.xz
sudo tar -C /usr/local -Jxf zig-x86_64-linux-0.14.1.tar.xz
echo "PATH=$PATH:/usr/local/zig-x86_64-linux-0.14.1" >> $GITHUB_ENV
uses: ./.github/actions/setup-zig
- name: Build static binaries for multi-arch
env:
@@ -70,6 +58,29 @@ jobs:
echo "Building for linux/arm64..."
GOARCH=arm64 CC="zig cc -target aarch64-linux-musl" go build -trimpath -ldflags="$LDFLAGS" -o komari-linux-arm64
- name: Upload Docker binaries
uses: actions/upload-artifact@v4
with:
name: komari-docker-binaries
path: komari-linux-*
build-and-push-multi-arch:
needs: build-docker-binaries
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download Docker binaries
uses: actions/download-artifact@v4
with:
name: komari-docker-binaries
path: .
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -0,0 +1,608 @@
name: Generate Release Notes
on:
release:
types: [published]
workflow_dispatch:
inputs:
tag:
description: "Release tag to regenerate notes for. Leave empty to use the latest release."
required: false
type: string
base_ref:
description: "Base ref or SHA for the release comparison. Snapshot builds pass the previous main SHA."
required: false
type: string
permissions:
contents: write
jobs:
update-release-notes:
runs-on: ubuntu-latest
if: ${{ github.event_name != 'release' || !startsWith(github.event.release.tag_name, 'Snapshot-') }}
steps:
- name: Generate and apply release notes
uses: actions/github-script@v9
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENAI_BASE_URL: ${{ secrets.OPENAI_BASE_URL }}
OPENAI_MODEL: ${{ vars.OPENAI_MODEL }}
with:
script: |
const { owner, repo } = context.repo;
const webRepo = { owner: "komari-monitor", repo: "komari-web" };
const AI_CONTEXT_CHAR_LIMIT = 120000;
const AI_MAX_COMMITS = 160;
const AI_MAX_CHANGED_FILES = 120;
const AI_MAX_PULL_REQUESTS = 80;
const AI_MAX_PATCH_CHARS_PER_FILE = 2200;
const OPENAI_TIMEOUT_MS = 120000;
const clean = (value) => (typeof value === "string" ? value.trim() : "");
const isSnapshotTag = (tag) => /^Snapshot-/i.test(clean(tag));
const releaseTimestamp = (release) => Date.parse(release.published_at || release.created_at || "");
const isStableRelease = (release) => !release.draft && !release.prerelease && !isSnapshotTag(release.tag_name);
function limitText(value, maxLength) {
const text = clean(value);
if (text.length <= maxLength) {
return text;
}
return `${text.slice(0, maxLength)}\n...[truncated ${text.length - maxLength} chars]`;
}
function appendWithinLimit(parts, value, maxLength) {
const text = clean(value);
if (!text) {
return false;
}
const currentLength = parts.reduce((total, part) => total + part.length + 2, 0);
if (currentLength + text.length + 2 <= maxLength) {
parts.push(text);
return true;
}
const remaining = maxLength - currentLength - 64;
if (remaining > 500) {
parts.push(`${text.slice(0, remaining)}\n...[context truncated]`);
}
return false;
}
async function resolveRelease() {
const requestedTag = clean(context.payload.release?.tag_name)
|| clean(context.payload.client_payload?.tag)
|| clean(context.payload.inputs?.tag);
if (requestedTag) {
const release = context.payload.release?.tag_name === requestedTag
? context.payload.release
: (await github.rest.repos.getReleaseByTag({ owner, repo, tag: requestedTag })).data;
return { tag: requestedTag, release };
}
const release = (await github.rest.repos.getLatestRelease({ owner, repo })).data;
return { tag: release.tag_name, release };
}
async function getPreviousReleaseTag(tag, release) {
const releases = await github.paginate(github.rest.repos.listReleases, {
owner,
repo,
per_page: 100,
});
const currentTime = releaseTimestamp(release);
const currentIsSnapshot = isSnapshotTag(tag);
const currentIsStable = isStableRelease(release);
const candidates = releases
.filter((item) => !item.draft && item.tag_name !== tag)
.filter((item) => {
if (!Number.isFinite(currentTime)) {
return true;
}
const itemTime = releaseTimestamp(item);
return Number.isFinite(itemTime) && itemTime < currentTime;
})
.filter((item) => {
if (currentIsSnapshot || currentIsStable) {
return isStableRelease(item);
}
return item.prerelease && !isSnapshotTag(item.tag_name);
})
.sort((left, right) => releaseTimestamp(right) - releaseTimestamp(left));
return candidates[0]?.tag_name || "";
}
async function resolveComparisonBase(tag, release) {
const requestedBaseRef = clean(context.payload.client_payload?.base_ref)
|| clean(context.payload.inputs?.base_ref);
if (requestedBaseRef) {
return {
ref: requestedBaseRef,
previousTag: "",
source: "requested base ref",
};
}
const previousTag = await getPreviousReleaseTag(tag, release);
if (previousTag) {
return {
ref: previousTag,
previousTag,
source: isSnapshotTag(tag) ? "previous stable release" : "previous release in the same channel",
};
}
if (isSnapshotTag(tag)) {
try {
const commit = (await github.rest.repos.getCommit({
owner,
repo,
ref: release.target_commitish || tag,
})).data;
const parentSha = commit.parents?.[0]?.sha || "";
if (parentSha) {
return {
ref: parentSha,
previousTag: "",
source: "snapshot target parent commit",
};
}
} catch (error) {
core.warning(`Failed to resolve parent commit for ${tag}: ${error.message}`);
}
}
return {
ref: "",
previousTag: "",
source: "none",
};
}
async function getGeneratedNotes(tag, release, previousTag) {
const params = {
owner,
repo,
tag_name: tag,
target_commitish: release.target_commitish || undefined,
};
if (previousTag) {
params.previous_tag_name = previousTag;
}
const generated = await github.request(
"POST /repos/{owner}/{repo}/releases/generate-notes",
params,
);
return clean(generated.data.body);
}
async function getComparison(baseRef, tag) {
if (!baseRef) {
return { commits: [], files: [], htmlUrl: "" };
}
const commits = [];
let files = [];
let htmlUrl = "";
for (let page = 1; page <= 20; page += 1) {
const response = await github.rest.repos.compareCommitsWithBasehead({
owner,
repo,
basehead: `${baseRef}...${tag}`,
per_page: 100,
page,
});
if (page === 1) {
files = response.data.files || [];
htmlUrl = response.data.html_url || "";
}
commits.push(...response.data.commits);
if (!response.headers.link?.includes('rel="next"')) {
break;
}
}
return { commits, files, htmlUrl };
}
function renderCommitLine(commit) {
const subject = clean(commit.commit.message.split("\n")[0]) || "(no subject)";
const shortSha = commit.sha.slice(0, 7);
const author = commit.author?.login
? ` @${commit.author.login}`
: (commit.commit.author?.name ? ` ${commit.commit.author.name}` : "");
return `- ${subject} ([${shortSha}](${commit.html_url}))${author}`;
}
function buildWhatsChangedSection(baseRef, tag, comparison) {
if (!baseRef) {
return "## What's Changed\n\nNo comparison base was found for this release.";
}
if (comparison.commits.length === 0) {
return `## What's Changed\n\nNo commits found between \`${baseRef}\` and \`${tag}\`.`;
}
const lines = ["## What's Changed", "", ...comparison.commits.map(renderCommitLine)];
if (comparison.htmlUrl) {
lines.push("", `**Full Changelog**: ${comparison.htmlUrl}`);
}
return lines.join("\n");
}
function appendCommitsToGeneratedNotes(generatedNotes, baseRef, tag, comparison) {
const notes = clean(generatedNotes);
if (!notes || comparison.commits.length === 0) {
return notes || buildWhatsChangedSection(baseRef, tag, comparison);
}
const lines = notes.split("\n");
const fullChangelogIndex = lines.findIndex((line) => /^\*\*Full Changelog\*\*:/i.test(line.trim()));
const nextTopLevelSectionIndex = lines.findIndex((line, index) => (
index > 0
&& /^##\s+/.test(line)
&& !/^##\s+What's Changed\b/i.test(line)
));
const insertionIndex = nextTopLevelSectionIndex >= 0
? nextTopLevelSectionIndex
: (fullChangelogIndex >= 0 ? fullChangelogIndex : lines.length);
const commitSection = [
"### Commits",
"",
...comparison.commits.map(renderCommitLine),
"",
];
lines.splice(insertionIndex, 0, ...commitSection);
return clean(lines.join("\n"));
}
async function getPullRequestContext(commits) {
const pullRequests = new Map();
for (const commit of commits.slice(0, AI_MAX_COMMITS)) {
try {
const response = await github.rest.repos.listPullRequestsAssociatedWithCommit({
owner,
repo,
commit_sha: commit.sha,
});
for (const pullRequest of response.data || []) {
pullRequests.set(pullRequest.html_url, pullRequest);
}
} catch (error) {
core.warning(`Failed to load pull requests for ${commit.sha.slice(0, 7)}: ${error.message}`);
}
if (pullRequests.size >= AI_MAX_PULL_REQUESTS) {
break;
}
}
return [...pullRequests.values()].slice(0, AI_MAX_PULL_REQUESTS);
}
function buildChangedFilesContext(files) {
if (!files.length) {
return "No changed file metadata was returned by GitHub.";
}
return files.slice(0, AI_MAX_CHANGED_FILES).map((file) => {
const lines = [
`File: ${file.filename}`,
`Status: ${file.status}; additions: ${file.additions}; deletions: ${file.deletions}; changes: ${file.changes}`,
];
if (file.patch) {
lines.push("Patch excerpt:");
lines.push(limitText(file.patch, AI_MAX_PATCH_CHARS_PER_FILE));
}
return lines.join("\n");
}).join("\n\n");
}
function buildCommitContext(commits) {
if (!commits.length) {
return "No commits were returned by GitHub.";
}
return commits.slice(0, AI_MAX_COMMITS).map((commit) => {
const shortSha = commit.sha.slice(0, 7);
const author = commit.author?.login || commit.commit.author?.name || "unknown";
const message = limitText(commit.commit.message, 1800);
return [
`Commit: ${shortSha}`,
`Author: ${author}`,
`URL: ${commit.html_url}`,
"Message:",
message,
].join("\n");
}).join("\n\n");
}
function buildPullRequestContext(pullRequests) {
if (!pullRequests.length) {
return "No associated pull requests were found for the compared commits.";
}
return pullRequests.map((pullRequest) => {
const labels = (pullRequest.labels || []).map((label) => label.name).filter(Boolean).join(", ") || "none";
return [
`PR: #${pullRequest.number} ${pullRequest.title}`,
`URL: ${pullRequest.html_url}`,
`Author: ${pullRequest.user?.login || "unknown"}`,
`Labels: ${labels}`,
"Body:",
limitText(pullRequest.body, 1800) || "(empty)",
].join("\n");
}).join("\n\n");
}
function buildAiContext({ tag, release, comparisonBase, whatsChangedSection, comparison, pullRequests, webReleaseSection }) {
const sections = [];
appendWithinLimit(sections, [
`Repository: ${owner}/${repo}`,
`Release tag: ${tag}`,
`Release title: ${release.name || tag}`,
`Prerelease: ${release.prerelease ? "yes" : "no"}`,
`Snapshot tag: ${isSnapshotTag(tag) ? "yes" : "no"}`,
`Target commitish: ${release.target_commitish || "(unknown)"}`,
`Comparison base: ${comparisonBase.ref || "(none)"}`,
`Comparison base source: ${comparisonBase.source}`,
`Compare URL: ${comparison.htmlUrl || "(unavailable)"}`,
].join("\n"), AI_CONTEXT_CHAR_LIMIT);
appendWithinLimit(sections, `What's Changed section:\n${whatsChangedSection}`, AI_CONTEXT_CHAR_LIMIT);
appendWithinLimit(sections, `Changed files and patch excerpts:\n${buildChangedFilesContext(comparison.files)}`, AI_CONTEXT_CHAR_LIMIT);
appendWithinLimit(sections, `Associated pull requests:\n${buildPullRequestContext(pullRequests)}`, AI_CONTEXT_CHAR_LIMIT);
appendWithinLimit(sections, `Commit details:\n${buildCommitContext(comparison.commits)}`, AI_CONTEXT_CHAR_LIMIT);
if (webReleaseSection) {
appendWithinLimit(sections, `Komari Web release notes for the same tag:\n${webReleaseSection}`, AI_CONTEXT_CHAR_LIMIT);
}
return sections.join("\n\n---\n\n");
}
function buildChatCompletionsUrl(baseUrl) {
const trimmed = clean(baseUrl).replace(/\/+$/, "");
if (!trimmed) {
return "";
}
if (trimmed.endsWith("/chat/completions")) {
return trimmed;
}
if (trimmed.endsWith("/v1")) {
return `${trimmed}/chat/completions`;
}
return `${trimmed}/v1/chat/completions`;
}
function normalizeReleaseSummary(markdown) {
const summary = clean(markdown);
if (!summary) {
return "";
}
const withoutAiHeading = clean(summary
.replace(/^#{1,6}\s+AI Summary\b\s*/im, "")
.replace(/^\*\*AI Summary\*\*\s*/im, ""));
const whatsChangedIndex = withoutAiHeading.search(/^##\s+What's Changed\b/im);
if (whatsChangedIndex >= 0) {
return clean(withoutAiHeading.slice(0, whatsChangedIndex));
}
return withoutAiHeading;
}
async function callOpenAiChatCompletion(url, apiKey, model, messages) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), OPENAI_TIMEOUT_MS);
try {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ model, messages, temperature: 0 }),
signal: controller.signal,
});
const responseText = await response.text();
let responseJson = {};
try {
responseJson = responseText ? JSON.parse(responseText) : {};
} catch {
responseJson = {};
}
if (!response.ok) {
const message = clean(responseJson.error?.message) || limitText(responseText, 500);
throw new Error(`OpenAI request failed with ${response.status}: ${message}`);
}
return clean(responseJson.choices?.[0]?.message?.content);
} finally {
clearTimeout(timeout);
}
}
async function buildReleaseSummarySection({ tag, release, comparisonBase, whatsChangedSection, comparison, webReleaseSection }) {
const apiKey = clean(process.env.OPENAI_API_KEY);
const baseUrl = clean(process.env.OPENAI_BASE_URL);
const model = clean(process.env.OPENAI_MODEL);
if (!apiKey || !baseUrl || !model) {
core.warning("OPENAI_API_KEY, OPENAI_BASE_URL, or OPENAI_MODEL is missing. Skipping generated release summary.");
return "";
}
const pullRequests = await getPullRequestContext(comparison.commits);
const aiContext = buildAiContext({
tag,
release,
comparisonBase,
whatsChangedSection,
comparison,
pullRequests,
webReleaseSection,
});
const messages = [
{
role: "system",
content: [
"You write accurate release notes for the Komari server project.",
"Use only the supplied repository context.",
"Include both Komari server changes and Komari Web frontend changes when frontend release notes are supplied.",
"Prioritize user-visible behavior, compatibility or migration notes, operations/deployment changes, security fixes, and important bug fixes.",
"Do not invent features or implementation details that are not supported by the context.",
"Write concise Markdown in Simplified Chinese.",
].join(" "),
},
{
role: "user",
content: [
`Create the release summary section for release ${tag}.`,
comparisonBase.ref
? `Summarize only changes in the comparison ${comparisonBase.ref}...${tag}.`
: "No comparison base was found, so summarize only the supplied release context.",
"Return only Markdown for the summary section that appears before '## What's Changed'.",
"Do not use the heading 'AI Summary'. Do not add a 'What's Changed' section.",
"Use these top-level headings, in this order: '## 主要更新内容', '## Bug修复', and '## 其他'.",
"Omit '## Bug修复' when there are no concrete bug fixes. Keep bullets short and specific.",
"",
aiContext,
].join("\n"),
},
];
try {
const url = buildChatCompletionsUrl(baseUrl);
const summary = await callOpenAiChatCompletion(url, apiKey, model, messages);
return normalizeReleaseSummary(summary);
} catch (error) {
core.warning(`Generated release summary failed: ${error.message}`);
return "";
}
}
async function buildWebReleaseSection(tag) {
const demoteMarkdownHeadings = (markdown) => markdown.replace(/^(#{1,5})(\s+)/gm, "#$1$2");
try {
const webRelease = (await github.rest.repos.getReleaseByTag({
owner: webRepo.owner,
repo: webRepo.repo,
tag,
})).data;
const webBody = clean(webRelease.body);
if (!webBody) {
return `## Komari Web Release Notes\n\nKomari Web release [${tag}](${webRelease.html_url}) exists, but its release notes are empty.`;
}
return [
"## Komari Web Release Notes",
"",
`Source: [komari-web ${tag}](${webRelease.html_url})`,
"",
demoteMarkdownHeadings(webBody),
].join("\n");
} catch (error) {
if (error.status === 404) {
core.info(`Komari Web release ${tag} does not exist yet. Skipping web release notes for now.`);
return "";
}
throw error;
}
}
const { tag, release } = await resolveRelease();
if (!tag) {
core.setFailed("No release tag was provided or found.");
return;
}
const comparisonBase = await resolveComparisonBase(tag, release);
core.info(comparisonBase.ref
? `Comparison base for ${tag}: ${comparisonBase.ref} (${comparisonBase.source}).`
: `No comparison base found for ${tag}.`);
const comparison = await getComparison(comparisonBase.ref, tag);
const whatsChangedSection = comparisonBase.previousTag
? appendCommitsToGeneratedNotes(
await getGeneratedNotes(tag, release, comparisonBase.previousTag),
comparisonBase.ref,
tag,
comparison,
)
: (!comparisonBase.ref && !isSnapshotTag(tag)
? await getGeneratedNotes(tag, release, "")
: buildWhatsChangedSection(comparisonBase.ref, tag, comparison));
if (!whatsChangedSection) {
core.setFailed(`Failed to build What's Changed section for ${tag}.`);
return;
}
const webReleaseSection = await buildWebReleaseSection(tag);
const releaseSummarySection = await buildReleaseSummarySection({
tag,
release,
comparisonBase,
whatsChangedSection,
comparison,
webReleaseSection,
});
const body = [
releaseSummarySection,
whatsChangedSection,
].filter(Boolean).join("\n\n");
if (clean(release.body) === clean(body)) {
core.info(`Release notes for ${tag} are already up to date.`);
} else {
await github.rest.repos.updateRelease({
owner,
repo,
release_id: release.id,
body,
});
}
await core.summary
.addHeading("Release Notes")
.addRaw(`Updated release notes for ${tag}.`)
.write();
+65 -51
View File
@@ -1,9 +1,12 @@
name: Publish Docker Image
name: Publish Docker Image Manually
on:
release:
types: [published]
workflow_dispatch:
inputs:
version:
description: Docker image tag and embedded version.
required: true
type: string
permissions:
contents: read
@@ -14,7 +17,61 @@ env:
IMAGE_NAME: ${{ github.repository }}
jobs:
build-frontend:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Build frontend artifact
uses: ./.github/actions/build-frontend
with:
artifact-name: komari-frontend
build-docker-binaries:
needs: build-frontend
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Restore frontend artifact
uses: ./.github/actions/restore-frontend
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.23"
- name: Install Zig
uses: ./.github/actions/setup-zig
- name: Build static binaries for multi-arch
env:
GOOS: linux
CGO_ENABLED: 1
GIN_MODE: release
VERSION: ${{ github.event.inputs.version }}
run: |
VERSION_HASH="${{ github.sha }}"
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
echo "Building for linux/amd64..."
GOARCH=amd64 CC="zig cc -target x86_64-linux-musl" go build -trimpath -ldflags="$LDFLAGS" -o komari-linux-amd64
echo "Building for linux/arm64..."
GOARCH=arm64 CC="zig cc -target aarch64-linux-musl" go build -trimpath -ldflags="$LDFLAGS" -o komari-linux-arm64
- name: Upload Docker binaries
uses: actions/upload-artifact@v4
with:
name: komari-docker-binaries
path: komari-linux-*
build-and-push-multi-arch:
needs: build-docker-binaries
runs-on: ubuntu-latest
permissions:
contents: read
@@ -24,54 +81,11 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
- name: Download Docker binaries
uses: actions/download-artifact@v4
with:
node-version: "23"
- name: Clone and build frontend
run: |
git clone https://github.com/komari-monitor/komari-web web
cd web
npm install
npm run build
cd ..
mkdir -p public/defaultTheme/dist
rm -rf public/defaultTheme/dist/*
cp -r web/dist/* public/defaultTheme/dist/
cp -f web/komari-theme.json public/defaultTheme/
if [ -f web/preview.png ]; then cp -f web/preview.png public/defaultTheme/; fi
if [ -f web/perview.png ]; then cp -f web/perview.png public/defaultTheme/; fi
# Compatibility: ensure both filenames exist if only one is present
if [ -f public/defaultTheme/preview.png ] && [ ! -f public/defaultTheme/perview.png ]; then cp -f public/defaultTheme/preview.png public/defaultTheme/perview.png; fi
if [ -f public/defaultTheme/perview.png ] && [ ! -f public/defaultTheme/preview.png ]; then cp -f public/defaultTheme/perview.png public/defaultTheme/preview.png; fi
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.23"
- name: Install Zig
run: |
wget https://ziglang.org/download/0.14.1/zig-x86_64-linux-0.14.1.tar.xz
sudo tar -C /usr/local -Jxf zig-x86_64-linux-0.14.1.tar.xz
echo "PATH=$PATH:/usr/local/zig-x86_64-linux-0.14.1" >> $GITHUB_ENV
- name: Build static binaries for multi-arch
env:
GOOS: linux
CGO_ENABLED: 1
GIN_MODE: release
run: |
VERSION="${{ github.event.release.tag_name }}"
VERSION_HASH="${{ github.sha }}"
LDFLAGS="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}"
echo "Building for linux/amd64..."
GOARCH=amd64 CC="zig cc -target x86_64-linux-musl" go build -trimpath -ldflags="$LDFLAGS" -o komari-linux-amd64
echo "Building for linux/arm64..."
GOARCH=arm64 CC="zig cc -target aarch64-linux-musl" go build -trimpath -ldflags="$LDFLAGS" -o komari-linux-arm64
name: komari-docker-binaries
path: .
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -92,7 +106,7 @@ jobs:
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=raw,value=${{ github.event.release.tag_name }}
type=raw,value=${{ github.event.inputs.version }}
type=raw,value=latest
- name: Build and push Docker image
+77 -27
View File
@@ -13,7 +13,20 @@ env:
IMAGE_NAME: ${{ github.repository }}
jobs:
build-frontend:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Build frontend artifact
uses: ./.github/actions/build-frontend
with:
artifact-name: komari-frontend
build-and-attach-binaries:
needs: build-frontend
runs-on: ubuntu-latest
strategy:
@@ -52,6 +65,10 @@ jobs:
goarch: riscv64
zig_target_triple: riscv64-linux-musl
binary_extension: ""
- goos: linux
goarch: loong64
zig_target_triple: loongarch64-linux-musl
binary_extension: ""
exclude:
- goos: windows
@@ -61,27 +78,8 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "23"
- name: Clone and build frontend
run: |
git clone https://github.com/komari-monitor/komari-web web
cd web
npm install
npm run build
cd ..
mkdir -p public/defaultTheme/dist
rm -rf public/defaultTheme/dist/*
cp -r web/dist/* public/defaultTheme/dist/
cp -f web/komari-theme.json public/defaultTheme/
if [ -f web/preview.png ]; then cp -f web/preview.png public/defaultTheme/; fi
if [ -f web/perview.png ]; then cp -f web/perview.png public/defaultTheme/; fi
# Compatibility: ensure both filenames exist if only one is present
if [ -f public/defaultTheme/preview.png ] && [ ! -f public/defaultTheme/perview.png ]; then cp -f public/defaultTheme/preview.png public/defaultTheme/perview.png; fi
if [ -f public/defaultTheme/perview.png ] && [ ! -f public/defaultTheme/preview.png ]; then cp -f public/defaultTheme/perview.png public/defaultTheme/preview.png; fi
- name: Restore frontend artifact
uses: ./.github/actions/restore-frontend
- name: Set up Go
uses: actions/setup-go@v5
@@ -89,12 +87,7 @@ jobs:
go-version: "1.23"
- name: Install Zig cross-compiler toolchain
run: |
sudo apt-get update # Update apt package lists
sudo apt-get install -y build-essential curl
wget https://ziglang.org/download/0.14.1/zig-x86_64-linux-0.14.1.tar.xz
sudo tar -C /usr/local -Jxf zig-x86_64-linux-0.14.1.tar.xz
echo "PATH=$PATH:/usr/local/zig-x86_64-linux-0.14.1" >> $GITHUB_ENV
uses: ./.github/actions/setup-zig
- name: Build binary
env:
@@ -112,6 +105,12 @@ jobs:
VERSION_HASH="${{ github.sha }}" # Use commit SHA for hash
go build -trimpath -ldflags="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}" -o $BINARY_NAME
- name: Upload binary as artifact
uses: actions/upload-artifact@v4
with:
name: komari-${{ matrix.goos }}-${{ matrix.goarch }}
path: komari-${{ matrix.goos }}-${{ matrix.goarch }}*
- name: Upload binary to release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
@@ -124,3 +123,54 @@ jobs:
# Upload the binary as a release asset using GitHub CLI
gh release upload ${{ github.event.release.tag_name }} $BINARY_NAME --repo ${{ github.repository }}
shell: bash
build-and-push-docker:
needs: build-and-attach-binaries
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download Docker binaries
uses: actions/download-artifact@v4
with:
pattern: komari-linux-*
path: .
merge-multiple: true
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=raw,value=${{ github.event.release.tag_name }}
type=raw,value=latest
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
platforms: linux/amd64,linux/arm64
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
no-cache: true
provenance: false
+293
View File
@@ -0,0 +1,293 @@
name: Publish Snapshot
on:
workflow_dispatch:
push:
branches:
- main
permissions:
contents: write
packages: write
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
concurrency:
group: snapshot-${{ github.ref }}
cancel-in-progress: true
jobs:
validate-ref:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Ensure snapshot target is main
run: |
if [ "${GITHUB_REF}" != "refs/heads/main" ]; then
echo "Snapshot builds must run from refs/heads/main. Current ref: ${GITHUB_REF}"
exit 1
fi
git fetch --no-tags --depth=1 origin main
MAIN_SHA="$(git rev-parse FETCH_HEAD)"
if [ "${GITHUB_SHA}" != "${MAIN_SHA}" ]; then
echo "Snapshot builds must use the current main commit."
echo "Workflow SHA: ${GITHUB_SHA}"
echo "Current main: ${MAIN_SHA}"
exit 1
fi
prepare:
needs: validate-ref
runs-on: ubuntu-latest
outputs:
version: ${{ steps.version.outputs.version }}
steps:
- name: Generate snapshot version
id: version
run: |
# Snapshot 版本号格式: Snapshot-yymmddhhMM (UTC+8)
VERSION="Snapshot-$(date -u -d '+8 hours' +%y%m%d%H%M)"
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
echo "Snapshot version: ${VERSION}"
build-frontend:
needs: validate-ref
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Build frontend artifact
uses: ./.github/actions/build-frontend
with:
artifact-name: komari-frontend
build-and-attach-binaries:
needs: [prepare, build-frontend]
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
goos: [windows, linux]
goarch: [amd64, arm64, 386, riscv64]
include:
# Windows
- goos: windows
goarch: amd64
zig_target_triple: x86_64-windows-gnu
binary_extension: .exe
- goos: windows
goarch: arm64
zig_target_triple: aarch64-windows-gnu
binary_extension: .exe
- goos: windows
goarch: 386
zig_target_triple: x86-windows-gnu
binary_extension: .exe
# Linux
- goos: linux
goarch: amd64
zig_target_triple: x86_64-linux-musl
binary_extension: ""
- goos: linux
goarch: arm64
zig_target_triple: aarch64-linux-musl
binary_extension: ""
- goos: linux
goarch: 386
zig_target_triple: x86-linux-musl
binary_extension: ""
- goos: linux
goarch: riscv64
zig_target_triple: riscv64-linux-musl
binary_extension: ""
- goos: linux
goarch: loong64
zig_target_triple: loongarch64-linux-musl
binary_extension: ""
exclude:
- goos: windows
goarch: riscv64
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Restore frontend artifact
uses: ./.github/actions/restore-frontend
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "1.23"
- name: Install Zig cross-compiler toolchain
uses: ./.github/actions/setup-zig
- name: Build binary
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
CGO_ENABLED: 1
CC: zig cc -target ${{ matrix.zig_target_triple }}
GIN_MODE: release
run: |
BINARY_NAME=komari-${{ matrix.goos }}-${{ matrix.goarch }}
if [ "${{ matrix.binary_extension }}" != "" ]; then
BINARY_NAME=${BINARY_NAME}${{ matrix.binary_extension }}
fi
VERSION="${{ needs.prepare.outputs.version }}"
VERSION_HASH="${{ github.sha }}" # Use commit SHA for hash
go build -trimpath -ldflags="-s -w -X github.com/komari-monitor/komari/utils.CurrentVersion=${VERSION} -X github.com/komari-monitor/komari/utils.VersionHash=${VERSION_HASH}" -o $BINARY_NAME
- name: Upload binary as artifact
uses: actions/upload-artifact@v4
with:
name: komari-${{ matrix.goos }}-${{ matrix.goarch }}
path: komari-${{ matrix.goos }}-${{ matrix.goarch }}*
publish-snapshot-release:
needs: [prepare, build-and-attach-binaries]
runs-on: ubuntu-latest
permissions:
actions: write
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Ensure commit is still current main
run: |
git fetch --no-tags --depth=1 origin main
MAIN_SHA="$(git rev-parse FETCH_HEAD)"
if [ "${GITHUB_SHA}" != "${MAIN_SHA}" ]; then
echo "Snapshot publish skipped because a newer main commit exists."
echo "Workflow SHA: ${GITHUB_SHA}"
echo "Current main: ${MAIN_SHA}"
exit 1
fi
- name: Download all binaries
uses: actions/download-artifact@v4
with:
path: artifacts
merge-multiple: true
- name: Create snapshot prerelease and upload binaries
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VERSION: ${{ needs.prepare.outputs.version }}
run: |
set -euxo pipefail
echo "Snapshot version: ${VERSION}"
ls -alR artifacts
# 如果同名 release/tag 已存在(例如重跑),先删除避免创建冲突
if gh release view "${VERSION}" --repo "${{ github.repository }}" >/dev/null 2>&1; then
echo "Release ${VERSION} 已存在,正在删除以重新创建..."
gh release delete "${VERSION}" --repo "${{ github.repository }}" --cleanup-tag --yes
fi
# 创建预发布并上传所有二进制文件
gh release create "${VERSION}" \
--repo "${{ github.repository }}" \
--target "${{ github.sha }}" \
--title "${VERSION}" \
--notes "Automated snapshot build (${VERSION}) from commit ${{ github.sha }}." \
--prerelease \
artifacts/komari-*
gh workflow run generate-release-notes.yml \
--repo "${{ github.repository }}" \
--ref main \
-f tag="${VERSION}"
gh release list \
--repo "${{ github.repository }}" \
--limit 1000 \
--json tagName,isPrerelease \
--jq '.[] | select(.isPrerelease and (.tagName | startswith("Snapshot-"))) | .tagName' |
while IFS= read -r tag; do
if [ -z "${tag}" ] || [ "${tag}" = "${VERSION}" ]; then
continue
fi
echo "Deleting old snapshot release ${tag}..."
gh release delete "${tag}" --repo "${{ github.repository }}" --cleanup-tag --yes
done
shell: bash
build-and-push-docker:
needs: [prepare, build-and-attach-binaries, publish-snapshot-release]
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Ensure commit is still current main
run: |
git fetch --no-tags --depth=1 origin main
MAIN_SHA="$(git rev-parse FETCH_HEAD)"
if [ "${GITHUB_SHA}" != "${MAIN_SHA}" ]; then
echo "Snapshot Docker publish skipped because a newer main commit exists."
echo "Workflow SHA: ${GITHUB_SHA}"
echo "Current main: ${MAIN_SHA}"
exit 1
fi
- name: Download Docker binaries
uses: actions/download-artifact@v4
with:
pattern: komari-linux-*
path: .
merge-multiple: true
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=raw,value=snapshot
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
platforms: linux/amd64,linux/arm64
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
no-cache: true
provenance: false
+2 -1
View File
@@ -33,4 +33,5 @@ go.work.sum
.env
# build
komari
komari
.codex
+1 -18
View File
@@ -8,29 +8,12 @@ ARG TARGETARCH
RUN apk add --no-cache ca-certificates curl tzdata
RUN set -eux; \
case "${TARGETARCH}" in \
amd64) cloudflared_arch="amd64" ;; \
386) cloudflared_arch="386" ;; \
arm64) cloudflared_arch="arm64" ;; \
arm) cloudflared_arch="arm" ;; \
*) echo "Unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
esac; \
curl -fsSL "https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-${cloudflared_arch}" -o /usr/local/bin/cloudflared; \
chmod +x /usr/local/bin/cloudflared
COPY komari-${TARGETOS}-${TARGETARCH} /app/komari
RUN chmod +x /app/komari
ENV GIN_MODE=release
ENV KOMARI_DB_TYPE=sqlite
ENV KOMARI_DB_FILE=/app/data/komari.db
ENV KOMARI_DB_HOST=localhost
ENV KOMARI_DB_PORT=3306
ENV KOMARI_DB_USER=root
ENV KOMARI_DB_PASS=
ENV KOMARI_DB_NAME=komari
ENV KOMARI_LISTEN=0.0.0.0:25774
EXPOSE 25774
+6 -1
View File
@@ -9,6 +9,11 @@
Komari is a lightweight, self-hosted server monitoring tool designed to provide a simple and efficient solution for monitoring server performance. It supports viewing server status through a web interface and collects data through a lightweight agent.
> [!WARNING]
> Komari is a self-hosted monitoring and control program intended only for systems you own or are authorized to administer. Do not weaponize Komari, deploy it without consent, or use it for unauthorized access, persistence, command execution, or other abusive activity. For context on real-world abuse risks, see Huntress' analysis: [Komari C2 agent abuse](https://www.huntress.com/blog/komari-c2-agent-abuse).
> Users are solely responsible for how they deploy and operate Komari. The developers do not accept responsibility for unauthorized or abusive use, or for any resulting consequences.
> On Windows, when remote control is enabled, the client displays a Windows notification at each user login to remind the user that Komari is remote control software.
[Documentation](https://komari-document.pages.dev/) | [文档(镜像站 By Geekertao)](https://www.komari.wiki) | [Telegram Group](https://t.me/komari_monitor)
## Features
@@ -93,7 +98,7 @@ sudo ./install-komari.sh
git clone https://github.com/komari-monitor/komari
cd komari
```
Copy the static files generated in step 1 to the `/public/defaultTheme/dist` folder in the root of the `komari` project, and copy `komari-theme.json` + `preview.png`/`perview.png` to `/public/defaultTheme`.
Copy the static files generated in step 1 to the `/web/public/defaultTheme/dist` folder in the root of the `komari` project, and copy `komari-theme.json` + `preview.png`/`perview.png` to `/web/public/defaultTheme`.
```bash
go build -o komari
```
-126
View File
@@ -1,126 +0,0 @@
package api
import (
"log"
"net/http"
"time"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/common"
"github.com/patrickmn/go-cache"
"strconv"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/utils"
)
var (
Records = cache.New(1*time.Minute, 1*time.Minute)
)
func SaveClientReportToDB() error {
lastMinute := time.Now().Add(-time.Minute).Unix()
var records []models.Record
var gpuRecords []models.GPURecord
// 遍历所有客户端记录
for uuid, x := range Records.Items() {
if uuid == "" {
continue
}
reports, ok := x.Object.([]common.Report)
if !ok {
log.Printf("Invalid report type for UUID %s", uuid)
continue
}
// 过滤一分钟前的记录
var filtered []common.Report
for _, r := range reports {
if r.UpdatedAt.Unix() >= lastMinute {
filtered = append(filtered, r)
}
}
// 更新缓存
Records.Set(uuid, filtered, cache.DefaultExpiration)
// 计算平均报告并添加到记录列表
if len(filtered) > 0 {
r := utils.AverageReport(uuid, time.Now(), filtered, 0.3)
records = append(records, r)
// 使用与其他数据相同的聚合逻辑处理GPU数据
gpuAggregated := utils.AverageGPUReports(uuid, time.Now(), filtered, 0.3)
gpuRecords = append(gpuRecords, gpuAggregated...)
}
}
// 批量插入数据库前去重(client与time共同构成唯一键)
db := dbcore.GetDBInstance()
if len(records) > 0 {
unique := make(map[string]models.Record)
for _, rec := range records {
key := rec.Client + "_" + strconv.FormatInt(rec.Time.ToTime().Unix(), 10)
unique[key] = rec
}
var deduped []models.Record
for _, rec := range unique {
deduped = append(deduped, rec)
}
if err := db.Model(&models.Record{}).Create(&deduped).Error; err != nil {
log.Printf("Failed to save records to database: %v", err)
return err
}
}
// 批量插入GPU记录
if len(gpuRecords) > 0 {
// GPU记录也需要去重,防止重复插入
gpuUnique := make(map[string]models.GPURecord)
for _, rec := range gpuRecords {
key := rec.Client + "_" + strconv.Itoa(rec.DeviceIndex) + "_" + strconv.FormatInt(rec.Time.ToTime().Unix(), 10)
gpuUnique[key] = rec
}
var gpuDeduped []models.GPURecord
for _, rec := range gpuUnique {
gpuDeduped = append(gpuDeduped, rec)
}
if err := db.Model(&models.GPURecord{}).Create(&gpuDeduped).Error; err != nil {
log.Printf("Failed to save GPU records to database: %v", err)
return err
}
}
return nil
}
type Response struct {
Status string `json:"status"`
Message string `json:"message"`
Data interface{} `json:"data,omitempty"`
}
// Respond sends a standardized JSON response.
func Respond(c *gin.Context, httpStatus int, status string, message string, data interface{}) {
c.JSON(httpStatus, Response{Status: status, Message: message, Data: data})
}
// RespondSuccess sends a success response with data.
func RespondSuccess(c *gin.Context, data interface{}) {
Respond(c, http.StatusOK, "success", "", data)
}
// RespondSuccessMessage sends a success response with message and data.
func RespondSuccessMessage(c *gin.Context, message string, data interface{}) {
Respond(c, http.StatusOK, "success", message, data)
}
// RespondError sends an error response with message.
func RespondError(c *gin.Context, httpStatus int, message string) {
Respond(c, httpStatus, "error", message, nil)
}
-137
View File
@@ -1,137 +0,0 @@
package admin
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/database/clients"
"github.com/komari-monitor/komari/database/records"
"github.com/komari-monitor/komari/ws"
)
func AddClient(c *gin.Context) {
var req struct {
Name string `json:"name"`
}
if err := c.ShouldBindJSON(&req); err != nil || req.Name == "" {
uuid, token, err := clients.CreateClient()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"status": "error", "message": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"status": "success", "uuid": uuid, "token": token})
return
}
uuid, token, err := clients.CreateClientWithName(req.Name)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"status": "error", "message": err.Error()})
return
}
user_uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), user_uuid.(string), "create client:"+uuid, "info")
c.JSON(http.StatusOK, gin.H{"status": "success", "uuid": uuid, "token": token, "message": ""})
}
func EditClient(c *gin.Context) {
var req = make(map[string]interface{})
uuid := c.Param("uuid")
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"status": "error", "message": err.Error()})
return
}
if uuid == "" {
c.JSON(http.StatusBadRequest, gin.H{"status": "error", "message": "Invalid or missing UUID"})
return
}
req["uuid"] = uuid
err := clients.SaveClient(req)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"status": "error", "message": err.Error()})
return
}
user_uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), user_uuid.(string), "edit client:"+uuid, "info")
c.JSON(http.StatusOK, gin.H{"status": "success"})
}
func RemoveClient(c *gin.Context) {
uuid := c.Param("uuid")
err := clients.DeleteClient(uuid)
if err != nil {
c.JSON(500, gin.H{
"status": "error",
"error": "Failed to delete client" + err.Error(),
})
return
}
user_uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), user_uuid.(string), "delete client:"+uuid, "warn")
c.JSON(200, gin.H{"status": "success"})
ws.DeleteConnectedClients(uuid)
ws.DeleteLatestReport(uuid)
}
func ClearRecord(c *gin.Context) {
if err := records.DeleteAll(); err != nil {
c.JSON(500, gin.H{
"status": "error",
"message": "Failed to delete Record" + err.Error(),
})
return
}
user_uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), user_uuid.(string), "clear records", "warn")
c.JSON(200, gin.H{"status": "success"})
}
func GetClient(c *gin.Context) {
uuid := c.Param("uuid")
if uuid == "" {
c.JSON(400, gin.H{
"status": "error",
"message": "Invalid or missing UUID",
})
return
}
result, err := clients.GetClientByUUID(uuid)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{
"status": "error",
"message": err.Error(),
})
return
}
c.JSON(http.StatusOK, result)
}
func ListClients(c *gin.Context) {
cls, err := clients.GetAllClientBasicInfo()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"status": "error", "message": err.Error()})
return
}
c.JSON(http.StatusOK, cls)
}
func GetClientToken(c *gin.Context) {
uuid := c.Param("uuid")
if uuid == "" {
c.JSON(400, gin.H{
"status": "error",
"message": "Invalid or missing UUID",
})
return
}
token, err := clients.GetClientTokenByUUID(uuid)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"status": "error", "message": err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"status": "success", "token": token, "message:": ""})
}
-28
View File
@@ -1,28 +0,0 @@
package admin
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
func OrderWeight(c *gin.Context) {
var req = make(map[string]int)
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, 400, "Invalid or missing request body: "+err.Error())
return
}
db := dbcore.GetDBInstance()
for uuid, weight := range req {
err := db.Model(&models.Client{}).Where("uuid = ?", uuid).Update("weight", weight).Error
if err != nil {
api.RespondError(c, 500, "Failed to update client weight: "+err.Error())
return
}
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "order clients", "info")
api.RespondSuccess(c, nil)
}
-115
View File
@@ -1,115 +0,0 @@
package clipboard
import (
"net/http"
"strconv"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/auditlog"
clipboardDB "github.com/komari-monitor/komari/database/clipboard"
"github.com/komari-monitor/komari/database/models"
)
// GetClipboard retrieves a clipboard entry by ID
func GetClipboard(c *gin.Context) {
idStr := c.Param("id")
id, err := strconv.Atoi(idStr)
if err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid ID")
return
}
cb, err := clipboardDB.GetClipboardByID(id)
if err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to get clipboard: "+err.Error())
return
}
api.RespondSuccess(c, cb)
}
// ListClipboard lists all clipboard entries
func ListClipboard(c *gin.Context) {
list, err := clipboardDB.ListClipboard()
if err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to list clipboard: "+err.Error())
return
}
api.RespondSuccess(c, list)
}
// CreateClipboard creates a new clipboard entry
func CreateClipboard(c *gin.Context) {
var req models.Clipboard
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if err := clipboardDB.CreateClipboard(&req); err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to create clipboard: "+err.Error())
return
}
userUUID, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), userUUID.(string), "create clipboard:"+strconv.Itoa(req.Id), "info")
api.RespondSuccess(c, req)
}
// UpdateClipboard updates an existing clipboard entry
func UpdateClipboard(c *gin.Context) {
idStr := c.Param("id")
id, err := strconv.Atoi(idStr)
if err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid ID")
return
}
var fields map[string]interface{}
if err := c.ShouldBindJSON(&fields); err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if err := clipboardDB.UpdateClipboardFields(id, fields); err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to update clipboard: "+err.Error())
return
}
userUUID, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), userUUID.(string), "update clipboard:"+strconv.Itoa(id), "info")
api.RespondSuccess(c, nil)
}
// DeleteClipboard deletes a clipboard entry
func DeleteClipboard(c *gin.Context) {
idStr := c.Param("id")
id, err := strconv.Atoi(idStr)
if err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid ID")
return
}
if err := clipboardDB.DeleteClipboard(id); err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to delete clipboard: "+err.Error())
return
}
userUUID, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), userUUID.(string), "delete clipboard:"+strconv.Itoa(id), "warn")
api.RespondSuccess(c, nil)
}
// BatchDeleteClipboard deletes multiple clipboard entries
func BatchDeleteClipboard(c *gin.Context) {
var req struct {
IDs []int `json:"ids" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid request: "+err.Error())
return
}
if len(req.IDs) == 0 {
api.RespondError(c, http.StatusBadRequest, "IDs cannot be empty")
return
}
if err := clipboardDB.DeleteClipboardBatch(req.IDs); err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to batch delete clipboard: "+err.Error())
return
}
userUUID, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), userUUID.(string), "batch delete clipboard: "+strconv.Itoa(len(req.IDs))+" items", "warn")
api.RespondSuccess(c, nil)
}
-104
View File
@@ -1,104 +0,0 @@
package admin
import (
"net/http"
"strings"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/utils/cloudflared"
)
const cloudflaredStopConfirmText = "STOP CLOUDFLARED"
func GetCloudflaredStatus(c *gin.Context) {
api.RespondSuccess(c, cloudflared.Status())
}
func StartCloudflared(c *gin.Context) {
var req struct {
Token string `json:"token"`
}
if err := c.ShouldBindJSON(&req); err != nil && err.Error() != "EOF" {
api.RespondError(c, http.StatusBadRequest, "Invalid request body: "+err.Error())
return
}
token := strings.TrimSpace(req.Token)
if token != "" {
if err := cloudflared.SaveToken(token); err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to save Cloudflare Tunnel token: "+err.Error())
return
}
}
if err := cloudflared.Start(token); err != nil {
api.RespondError(c, http.StatusBadRequest, err.Error())
return
}
if uuid, ok := c.Get("uuid"); ok {
auditlog.Log(c.ClientIP(), uuid.(string), "started cloudflared tunnel", "warn")
}
api.RespondSuccess(c, cloudflared.Status())
}
func StopCloudflared(c *gin.Context) {
var req struct {
CurrentPassword string `json:"current_password"`
ConfirmText string `json:"confirm_text"`
}
if err := c.ShouldBindJSON(&req); err != nil && err.Error() != "EOF" {
api.RespondError(c, http.StatusBadRequest, "Invalid request body: "+err.Error())
return
}
disablePasswordLogin, _ := config.GetAs[bool](config.DisablePasswordLoginKey, false)
if !disablePasswordLogin {
rawUUID, exists := c.Get("uuid")
if !exists {
api.RespondError(c, http.StatusUnauthorized, "Unauthorized.")
return
}
user, err := accounts.GetUserByUUID(rawUUID.(string))
if err != nil {
api.RespondError(c, http.StatusUnauthorized, "Failed to verify current user")
return
}
if strings.TrimSpace(req.CurrentPassword) == "" {
api.RespondError(c, http.StatusBadRequest, "Current password is required")
return
}
if _, ok := accounts.CheckPassword(user.Username, req.CurrentPassword); !ok {
api.RespondError(c, http.StatusUnauthorized, "Current password is incorrect")
return
}
} else if strings.TrimSpace(req.ConfirmText) != cloudflaredStopConfirmText {
api.RespondError(c, http.StatusBadRequest, "Type STOP CLOUDFLARED to confirm stopping cloudflared")
return
}
if err := cloudflared.Stop(); err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to stop cloudflared: "+err.Error())
return
}
if uuid, ok := c.Get("uuid"); ok {
auditlog.Log(c.ClientIP(), uuid.(string), "stopped cloudflared tunnel", "warn")
}
api.RespondSuccess(c, cloudflared.Status())
}
func RemoveCloudflaredToken(c *gin.Context) {
if err := cloudflared.RemoveToken(); err != nil {
api.RespondError(c, http.StatusBadRequest, "Failed to remove Cloudflare Tunnel token: "+err.Error())
return
}
if uuid, ok := c.Get("uuid"); ok {
auditlog.Log(c.ClientIP(), uuid.(string), "removed cloudflared tunnel token", "warn")
}
api.RespondSuccess(c, cloudflared.Status())
}
-98
View File
@@ -1,98 +0,0 @@
package admin
import (
"encoding/json"
"time"
"github.com/gin-gonic/gin"
"github.com/gorilla/websocket"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/database/tasks"
"github.com/komari-monitor/komari/utils"
"github.com/komari-monitor/komari/ws"
)
// 接受数据类型:
// - command: string
// - clients: []string (客户端 UUID 列表)
func Exec(c *gin.Context) {
var req struct {
Command string `json:"command" binding:"required"`
Clients []string `json:"clients" binding:"required"`
}
var onlineClients []string
var offlineClients []string
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, 400, "Invalid or missing request body: "+err.Error())
return
}
// for uuid := range ws.GetConnectedClients() {
// if contain(req.Clients, uuid) {
// onlineClients = append(onlineClients, uuid)
// }
// // else {
// // api.RespondError(c, 400, "Client not connected: "+uuid)
// // return
// // }
// }
for _, uuid := range req.Clients {
if client := ws.GetConnectedClients()[uuid]; client != nil {
onlineClients = append(onlineClients, uuid)
} else {
offlineClients = append(offlineClients, uuid)
}
}
if len(onlineClients) == 0 {
api.RespondError(c, 400, "No clients connected")
return
}
taskId := utils.GenerateRandomString(16)
if err := tasks.CreateTask(taskId, append(onlineClients, offlineClients...), req.Command); err != nil {
api.RespondError(c, 500, "Failed to create task: "+err.Error())
return
}
for _, uuid := range onlineClients {
var send struct {
Message string `json:"message"`
Command string `json:"command"`
TaskId string `json:"task_id"`
}
send.Message = "exec"
send.Command = req.Command
send.TaskId = taskId
payload, _ := json.Marshal(send)
client := ws.GetConnectedClients()[uuid]
if client != nil {
if err := client.WriteMessage(websocket.TextMessage, payload); err != nil {
api.RespondError(c, 400, "Client connection is broke: "+uuid)
return
}
} else {
api.RespondError(c, 400, "Client connection is null: "+uuid)
return
}
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "REC, task id: "+taskId, "warn")
api.RespondSuccess(c, gin.H{
"task_id": taskId,
"clients": onlineClients,
})
if len(offlineClients) > 0 {
for _, uuid := range offlineClients {
tasks.SaveTaskResult(taskId, uuid, "Client offline!", -1, models.FromTime(time.Now()))
}
}
}
// func contain(clients []string, uuid string) bool {
// for _, client := range clients {
// if client == uuid {
// return true
// }
// }
// return false
// }
-49
View File
@@ -1,49 +0,0 @@
package log
import (
"strconv"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
func GetLogs(c *gin.Context) {
limit := c.Query("limit")
if limit == "" {
limit = "100" // Default to 100 logs if not specified
}
page := c.Query("page")
if page == "" {
page = "1" // Default to page 1 if not specified
}
// Convert limit and page to integers
// If conversion fails, return an error
limitInt, err := strconv.Atoi(limit)
if err != nil || limitInt <= 0 {
api.RespondError(c, 400, "Invalid limit: "+limit)
return
}
pageInt, err := strconv.Atoi(page)
if err != nil || pageInt <= 0 {
api.RespondError(c, 400, "Invalid page: "+page)
return
}
db := dbcore.GetDBInstance()
var logs []models.Log
// 添加分页:计算偏移量并限制数量
offset := (pageInt - 1) * limitInt
var total int64
if err := db.Model(&models.Log{}).Count(&total).Error; err != nil {
api.RespondError(c, 500, "Failed to count logs: "+err.Error())
return
}
if err := db.Order("time desc").Limit(limitInt).Offset(offset).Find(&logs).Error; err != nil {
api.RespondError(c, 500, "Failed to retrieve logs: "+err.Error())
return
}
api.RespondSuccess(c, gin.H{"logs": logs, "total": total})
}
-64
View File
@@ -1,64 +0,0 @@
package admin
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/utils/messageSender"
"github.com/komari-monitor/komari/utils/messageSender/factory"
)
func GetMessageSenderProvider(c *gin.Context) {
provider := c.Query("provider")
if provider != "" {
// 如果指定了provider,返回单个提供者的配置
config, err := database.GetMessageSenderConfigByName(provider)
if err != nil {
api.RespondError(c, 404, "Provider not found: "+err.Error())
return
}
api.RespondSuccess(c, config)
return
}
// 否则返回所有提供者的配置项模板
providers := factory.GetSenderConfigs()
if len(providers) == 0 {
api.RespondError(c, 404, "No message sender providers found")
return
}
api.RespondSuccess(c, providers)
}
func SetMessageSenderProvider(c *gin.Context) {
var senderConfig models.MessageSenderProvider
if err := c.ShouldBindJSON(&senderConfig); err != nil {
api.RespondError(c, 400, "Invalid configuration: "+err.Error())
return
}
if senderConfig.Name == "" {
api.RespondError(c, 400, "Provider name is required")
return
}
_, exists := factory.GetConstructor(senderConfig.Name)
if !exists {
api.RespondError(c, 404, "Provider not found: "+senderConfig.Name)
return
}
if err := database.SaveMessageSenderConfig(&senderConfig); err != nil {
api.RespondError(c, 500, "Failed to save message sender provider configuration: "+err.Error())
return
}
NotificationMethod, _ := config.GetAs[string](config.NotificationMethodKey, "none")
// 正在使用,重载
if NotificationMethod == senderConfig.Name {
err := messageSender.LoadProvider(senderConfig.Name, senderConfig.Addition)
if err != nil {
api.RespondError(c, 500, "Failed to load message sender provider: "+err.Error())
return
}
}
api.RespondSuccess(c, gin.H{"message": "Message sender provider set successfully"})
}
-90
View File
@@ -1,90 +0,0 @@
package notification
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/database/notification"
)
// POST body: clients []string, name string, metric string, threshold float32, ratio float32, interval int
func AddLoadNotification(c *gin.Context) {
var req struct {
Clients []string `json:"clients" binding:"required"`
Name string `json:"name"`
Metric string `json:"metric" binding:"required"`
Threshold float32 `json:"threshold" binding:"required"` // 阈值百分比
Ratio float32 `json:"ratio" binding:"required"` // 达标时间比
Interval int `json:"interval" binding:"required"` // 间隔时间,单位秒
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, err.Error())
return
}
if req.Interval > 4*60 || req.Interval <= 0 {
api.RespondError(c, http.StatusBadRequest, "Interval must be between 1 and 240 minutes")
return
}
if req.Ratio <= 0 || req.Ratio > 1 {
api.RespondError(c, http.StatusBadRequest, "Ratio must be between 0 and 1")
return
}
if taskID, err := notification.AddLoadNotification(req.Clients, req.Name, req.Metric, req.Threshold, req.Ratio, req.Interval); err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
} else {
api.RespondSuccess(c, gin.H{"task_id": taskID})
}
}
// POST body: id []uint
func DeleteLoadNotification(c *gin.Context) {
var req struct {
ID []uint `json:"id" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, err.Error())
return
}
if err := notification.DeleteLoadNotification(req.ID); err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
} else {
api.RespondSuccess(c, nil)
}
}
// POST body: id []uint, updates map[string]interface{}
func EditLoadNotification(c *gin.Context) {
var req struct {
Notifications []*models.LoadNotification `json:"notifications" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid request data")
return
}
if err := notification.EditLoadNotification(req.Notifications); err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
} else {
// for _, notification := range req.Notifications {
// notification.DeleteLoadNotification([]uint{notification.Id})
// }
api.RespondSuccess(c, nil)
}
}
func GetAllLoadNotifications(c *gin.Context) {
notifications, err := notification.GetAllLoadNotifications()
if err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
return
}
api.RespondSuccess(c, notifications)
}
-109
View File
@@ -1,109 +0,0 @@
package notification
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"gorm.io/gorm/clause"
)
// POST body : []uuid
func EnableOfflineNotification(c *gin.Context) {
var uuids []string
if err := c.ShouldBindJSON(&uuids); err != nil {
api.RespondError(c, 400, "Invalid request body: "+err.Error())
return
}
var notifications []models.OfflineNotification
for _, uuid := range uuids {
notifications = append(notifications, models.OfflineNotification{
Client: uuid,
Enable: true,
})
}
err := dbcore.GetDBInstance().Model(&models.OfflineNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable"}),
}).
Select("client", "enable").
Create(notifications).Error
if err != nil {
api.RespondError(c, 500, "Failed to enable offline notifications: "+err.Error())
return
}
api.RespondSuccess(c, nil)
}
// POST body : []uuid
func DisableOfflineNotification(c *gin.Context) {
var uuids []string
if err := c.ShouldBindJSON(&uuids); err != nil {
api.RespondError(c, 400, "Invalid request body: "+err.Error())
return
}
var notifications []models.OfflineNotification
for _, uuid := range uuids {
notifications = append(notifications, models.OfflineNotification{
Client: uuid,
Enable: false,
})
}
err := dbcore.GetDBInstance().Model(&models.OfflineNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable"}),
}).
Select("client", "enable").
Create(notifications).Error
if err != nil {
api.RespondError(c, 500, "Failed to disable offline notifications: "+err.Error())
return
}
api.RespondSuccess(c, nil)
}
func EditOfflineNotification(c *gin.Context) {
var notifications []models.OfflineNotification
if err := c.ShouldBindJSON(&notifications); err != nil {
api.RespondError(c, 400, "Invalid request body: "+err.Error())
return
}
if len(notifications) == 0 {
api.RespondError(c, 400, "At least one notification is required")
return
}
for _, noti := range notifications {
if noti.Client == "" {
api.RespondError(c, 400, "Client UUID cannot be empty")
return
}
if noti.GracePeriod <= 0 {
api.RespondError(c, 400, "GracePeriod must be a positive integer")
return
}
}
err := dbcore.GetDBInstance().Model(&models.OfflineNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable", "grace_period"}),
}).
Select("*").
Create(notifications).Error
if err != nil {
api.RespondError(c, 500, "Failed to edit offline notifications: "+err.Error())
return
}
api.RespondSuccess(c, nil)
}
func ListOfflineNotifications(c *gin.Context) {
var notifications []models.OfflineNotification
err := dbcore.GetDBInstance().Model(&models.OfflineNotification{}).Find(&notifications).Error
if err != nil {
api.RespondError(c, 500, "Failed to list offline notifications: "+err.Error())
return
}
api.RespondSuccess(c, notifications)
}
-93
View File
@@ -1,93 +0,0 @@
package admin
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/utils/oauth"
"github.com/komari-monitor/komari/utils/oauth/factory"
)
func BindingExternalAccount(c *gin.Context) {
session, _ := c.Cookie("session_token")
user, err := accounts.GetUserBySession(session)
if err != nil {
api.RespondError(c, 500, "No user found: "+err.Error())
return
}
c.SetCookie("binding_external_account", user.UUID, 3600, "/", "", false, true)
c.Redirect(302, "/api/oauth")
}
func UnbindExternalAccount(c *gin.Context) {
session, _ := c.Cookie("session_token")
user, err := accounts.GetUserBySession(session)
if err != nil {
api.RespondError(c, 500, "No user found: "+err.Error())
return
}
err = accounts.UnbindExternalAccount(user.UUID)
if err != nil {
api.RespondError(c, 500, "Failed to unbind external account: "+err.Error())
return
}
api.RespondSuccess(c, nil)
}
func GetOidcProvider(c *gin.Context) {
provider := c.Query("provider")
if provider != "" {
// 如果指定了provider,返回单个提供者的配置
config, err := database.GetOidcConfigByName(provider)
if err != nil {
api.RespondError(c, 404, "Provider not found: "+err.Error())
return
}
api.RespondSuccess(c, config)
return
}
// 否则返回所有提供者的配置
providers := factory.GetProviderConfigs()
if len(providers) == 0 {
api.RespondError(c, 404, "No OIDC providers found")
return
}
api.RespondSuccess(c, providers)
}
func SetOidcProvider(c *gin.Context) {
var oidcConfig models.OidcProvider
if err := c.ShouldBindJSON(&oidcConfig); err != nil {
api.RespondError(c, 400, "Invalid configuration: "+err.Error())
return
}
if oidcConfig.Name == "" {
api.RespondError(c, 400, "Provider name is required")
return
}
_, exists := factory.GetConstructor(oidcConfig.Name)
if !exists {
api.RespondError(c, 404, "Provider not found: "+oidcConfig.Name)
return
}
if err := database.SaveOidcConfig(&oidcConfig); err != nil {
api.RespondError(c, 500, "Failed to save OIDC provider configuration: "+err.Error())
return
}
OAuthProvider, _ := config.GetAs[string](config.OAuthProviderKey, "github")
// 正在使用,重载
if OAuthProvider == oidcConfig.Name {
err := oauth.LoadProvider(oidcConfig.Name, oidcConfig.Addition)
if err != nil {
api.RespondError(c, 500, "Failed to load OIDC provider: "+err.Error())
return
}
}
api.RespondSuccess(c, gin.H{"message": "OIDC provider set successfully"})
}
-120
View File
@@ -1,120 +0,0 @@
package admin
import (
"net/http"
"strconv"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/database/tasks"
)
// AddPingTask 处理新增延迟监测任务请求。
// POST body: clients []string, default_on bool, target, task_type string, interval int
func AddPingTask(c *gin.Context) {
var req struct {
Clients []string `json:"clients"`
DefaultOn bool `json:"default_on"`
Name string `json:"name" binding:"required"`
Target string `json:"target" binding:"required"`
TaskType string `json:"type" binding:"required"` // icmp, tcp, http
Interval int `json:"interval" binding:"required"` // 间隔时间,单位秒
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, err.Error())
return
}
if !req.DefaultOn && len(req.Clients) == 0 {
api.RespondError(c, http.StatusBadRequest, "clients is required when default_on is false")
return
}
if taskID, err := tasks.AddPingTask(req.Clients, req.DefaultOn, req.Name, req.Target, req.TaskType, req.Interval); err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
} else {
api.RespondSuccess(c, gin.H{"task_id": taskID})
}
}
// POST body: id []uint
func DeletePingTask(c *gin.Context) {
var req struct {
ID []uint `json:"id" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, err.Error())
return
}
if err := tasks.DeletePingTask(req.ID); err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
} else {
api.RespondSuccess(c, nil)
}
}
// EditPingTask 处理延迟监测任务编辑请求。
// POST body: id []uint, updates map[string]interface{}
func EditPingTask(c *gin.Context) {
var req struct {
Tasks []*models.PingTask `json:"tasks" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid request data")
return
}
for _, task := range req.Tasks {
// 编辑时只拦截空任务对象,允许把任务临时保存为未绑定服务器状态。
if task == nil {
api.RespondError(c, http.StatusBadRequest, "Invalid request data")
return
}
}
if err := tasks.EditPingTask(req.Tasks); err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
} else {
// for _, task := range req.Tasks {
// tasks.DeletePingRecords([]uint{task.Id})
// }
api.RespondSuccess(c, nil)
}
}
func GetAllPingTasks(c *gin.Context) {
tasks, err := tasks.GetAllPingTasks()
if err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
return
}
api.RespondSuccess(c, tasks)
}
func OrderPingTask(c *gin.Context) {
var req map[string]int
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid or missing request body: "+err.Error())
return
}
order := make(map[uint]int, len(req))
for idStr, weight := range req {
id, err := strconv.ParseUint(idStr, 10, 64)
if err != nil {
api.RespondError(c, http.StatusBadRequest, "Invalid task id: "+idStr)
return
}
order[uint(id)] = weight
}
if err := tasks.UpdatePingTaskOrder(order); err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
return
}
api.RespondSuccess(c, nil)
}
-50
View File
@@ -1,50 +0,0 @@
package admin
import (
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/gin-gonic/gin"
)
func GetSessions(c *gin.Context) {
ss, err := accounts.GetAllSessions()
if err != nil {
api.RespondError(c, 500, "Failed to retrieve sessions: "+err.Error())
return
}
current, _ := c.Cookie("session_token")
c.JSON(200, gin.H{"status": "success", "current": current, "data": ss})
}
func DeleteSession(c *gin.Context) {
var req struct {
Session string `json:"session" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, 400, "Invalid request: "+err.Error())
return
}
err := accounts.DeleteSession(req.Session)
if err != nil {
api.RespondError(c, 500, "Failed to delete session: "+err.Error())
return
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "delete session", "info")
api.RespondSuccess(c, nil)
}
func DeleteAllSession(c *gin.Context) {
err := accounts.DeleteAllSessions()
if err != nil {
api.RespondError(c, 500, "Failed to delete all sessions: "+err.Error())
return
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "delete all sessions", "warn")
api.RespondSuccess(c, nil)
}
-54
View File
@@ -1,54 +0,0 @@
package admin
import (
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/database/records"
"github.com/komari-monitor/komari/database/tasks"
"github.com/gin-gonic/gin"
)
// GetSettings 获取自定义配置
func GetSettings(c *gin.Context) {
cst, err := config.GetAll()
if err != nil {
api.RespondError(c, 500, "Failed to get settings: "+err.Error())
return
}
api.RespondSuccess(c, cst)
}
// EditSettings 更新自定义配置
func EditSettings(c *gin.Context) {
cfg := make(map[string]interface{})
if err := c.ShouldBindJSON(&cfg); err != nil {
api.RespondError(c, 400, "Invalid or missing request body: "+err.Error())
return
}
if err := config.SetMany(cfg); err != nil {
api.RespondError(c, 500, "Failed to update settings: "+err.Error())
return
}
uuid, _ := c.Get("uuid")
message := "update settings: "
for key := range cfg {
message += key + ", "
}
if len(message) > 2 {
message = message[:len(message)-2]
}
auditlog.Log(c.ClientIP(), uuid.(string), message, "info")
api.RespondSuccess(c, nil)
}
func ClearAllRecords(c *gin.Context) {
records.DeleteAll()
tasks.DeleteAllPingRecords()
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "clear all records", "info")
api.RespondSuccess(c, nil)
}
-154
View File
@@ -1,154 +0,0 @@
package admin
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/tasks"
)
func GetTasks(c *gin.Context) {
dbTasks, err := tasks.GetAllTasks()
if err != nil {
api.RespondError(c, 500, "Failed to retrieve tasks: "+err.Error())
return
}
var responseTasks []gin.H
for _, t := range dbTasks {
results, err := tasks.GetTaskResultsByTaskId(t.TaskId)
if err != nil {
api.RespondError(c, 500, "Failed to retrieve task results: "+err.Error())
return
}
var filteredResults []gin.H
for _, r := range results {
filteredResults = append(filteredResults, gin.H{
"client": r.Client,
"result": r.Result,
"exit_code": r.ExitCode,
"finished_at": r.FinishedAt,
"created_at": r.CreatedAt,
})
}
responseTasks = append(responseTasks, gin.H{
"task_id": t.TaskId,
"clients": t.Clients,
"command": t.Command,
"results": filteredResults,
})
}
api.RespondSuccess(c, responseTasks)
}
func GetTaskById(c *gin.Context) {
taskId := c.Param("task_id")
if taskId == "" {
api.RespondError(c, 400, "Task ID is required")
return
}
task, err := tasks.GetTaskByTaskId(taskId)
if err != nil {
api.RespondError(c, 500, "Failed to retrieve task: "+err.Error())
return
}
if task == nil {
api.RespondError(c, 404, "Task not found")
return
}
results, err := tasks.GetTaskResultsByTaskId(taskId)
if err != nil {
api.RespondError(c, 500, "Failed to retrieve task results: "+err.Error())
return
}
var filteredResults []gin.H
for _, r := range results {
filteredResults = append(filteredResults, gin.H{
"client": r.Client,
"result": r.Result,
"exit_code": r.ExitCode,
"finished_at": r.FinishedAt,
"created_at": r.CreatedAt,
})
}
api.RespondSuccess(c, gin.H{
"task_id": task.TaskId,
"clients": task.Clients,
"command": task.Command,
"results": filteredResults,
})
}
func GetTasksByClientId(c *gin.Context) {
clientId := c.Param("uuid")
if clientId == "" {
api.RespondError(c, 400, "Client ID is required")
return
}
tasks, err := tasks.GetTasksByClientId(clientId)
if err != nil {
api.RespondError(c, 500, "Failed to retrieve tasks: "+err.Error())
return
}
if len(tasks) == 0 {
api.RespondError(c, 404, "No tasks found for this client")
return
}
api.RespondSuccess(c, tasks)
}
func GetSpecificTaskResult(c *gin.Context) {
taskId := c.Param("task_id")
clientId := c.Param("uuid")
if taskId == "" || clientId == "" {
api.RespondError(c, 400, "Task ID and Client ID are required")
return
}
result, err := tasks.GetSpecificTaskResult(taskId, clientId)
if err != nil {
api.RespondError(c, 500, "Failed to retrieve task result: "+err.Error())
return
}
if result == nil {
api.RespondError(c, 404, "No result found for this task and client")
return
}
api.RespondSuccess(c, result)
}
// Param: task_id
func GetTaskResultsByTaskId(c *gin.Context) {
taskId := c.Param("task_id")
if taskId == "" {
api.RespondError(c, 400, "Task ID is required")
return
}
results, err := tasks.GetTaskResultsByTaskId(taskId)
if err != nil {
api.RespondError(c, 500, "Failed to retrieve task results: "+err.Error())
return
}
if len(results) == 0 {
api.RespondError(c, 404, "No results found for this task")
return
}
api.RespondSuccess(c, results)
}
func GetAllTaskResultByUUID(c *gin.Context) {
clientId := c.Param("uuid")
if clientId == "" {
api.RespondError(c, 400, "Client ID is required")
return
}
results, err := tasks.GetAllTasksResultByUUID(clientId)
if err != nil {
api.RespondError(c, 500, "Failed to retrieve tasks: "+err.Error())
return
}
if len(results) == 0 {
api.RespondError(c, 404, "No tasks found for this client")
return
}
api.RespondSuccess(c, results)
}
-50
View File
@@ -1,50 +0,0 @@
package test
import (
"net"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/utils/geoip"
"github.com/komari-monitor/komari/utils/messageSender"
)
func TestSendMessage(c *gin.Context) {
err := messageSender.SendEvent(models.EventMessage{
Event: "Test",
Message: "This is a test message from Komari.",
})
if err != nil {
api.RespondError(c, 500, "Failed to send message: "+err.Error())
return
}
api.RespondSuccess(c, nil)
}
func TestGeoIp(c *gin.Context) {
ip := c.Query("ip")
if ip == "" {
if cfIP := c.GetHeader("CF-Connecting-IP"); cfIP != "" {
ip = cfIP
} else {
ip = c.ClientIP()
}
}
cfg, err := config.GetAs[bool](config.GeoIpEnabledKey, false)
if err != nil {
api.RespondError(c, 500, "Failed to get configuration: "+err.Error())
return
}
if !cfg {
api.RespondError(c, 400, "GeoIP is not enabled in the configuration.")
return
}
GeoIpRecord, err := geoip.GetGeoInfo(net.ParseIP(ip))
if err != nil {
api.RespondError(c, 500, "Failed to get GeoIP record: "+err.Error())
return
}
api.RespondSuccess(c, GeoIpRecord)
}
-46
View File
@@ -1,46 +0,0 @@
package update
import (
"io"
"net/http"
"os"
"strings"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/auditlog"
)
func UploadFavicon(c *gin.Context) {
c.Request.Body = http.MaxBytesReader(c.Writer, c.Request.Body, 5<<20) // 5MB
data, err := io.ReadAll(c.Request.Body)
if err != nil {
if strings.Contains(err.Error(), "request body too large") {
api.RespondError(c, http.StatusRequestEntityTooLarge, "File too large. Maximum size is 5MB")
} else {
api.RespondError(c, http.StatusBadRequest, err.Error())
}
return
}
if err := os.WriteFile("./data/favicon.ico", data, 0644); err != nil {
api.RespondError(c, http.StatusInternalServerError, "Failed to save favicon: "+err.Error())
return
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "Favicon uploaded", "info")
api.RespondSuccess(c, nil)
}
func DeleteFavicon(c *gin.Context) {
if err := os.Remove("./data/favicon.ico"); err != nil {
if os.IsNotExist(err) {
api.RespondError(c, http.StatusNotFound, "Favicon not found")
} else {
api.RespondError(c, http.StatusInternalServerError, "Failed to delete favicon: "+err.Error())
}
return
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "Favicon deleted", "info")
api.RespondSuccess(c, nil)
}
-18
View File
@@ -1,18 +0,0 @@
package update
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/utils/geoip"
)
func UpdateMmdbGeoIP(c *gin.Context) {
if err := geoip.UpdateDatabase(); err != nil {
api.RespondError(c, 500, "Failed to update GeoIP database "+err.Error())
return
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "GeoIP database updated", "info")
api.RespondSuccess(c, nil)
}
-40
View File
@@ -1,40 +0,0 @@
package update
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/auditlog"
)
func UpdateUser(c *gin.Context) {
var req struct {
Uuid string `json:"uuid" binding:"required"`
Name *string `json:"username"`
Password *string `json:"password"`
SsoType *string `json:"sso_type"`
}
if err := c.ShouldBindJSON(&req); err != nil {
api.RespondError(c, 400, "Invalid or missing request body: "+err.Error())
return
}
if req.Password == nil && req.Name == nil {
api.RespondError(c, 400, "At least one field (username or password) must be provided")
return
}
if req.Name != nil && len(*req.Name) < 3 {
api.RespondError(c, 400, "Username must be at least 3 characters long")
return
}
if req.Password != nil && len(*req.Password) < 6 {
api.RespondError(c, 400, "Password must be at least 6 characters long")
return
}
if err := accounts.UpdateUser(req.Uuid, req.Name, req.Password, req.SsoType); err != nil {
api.RespondError(c, 500, "Failed to update user: "+err.Error())
return
}
uuid, _ := c.Get("uuid")
auditlog.Log(c.ClientIP(), uuid.(string), "User updated", "warn")
api.RespondSuccess(c, gin.H{"uuid": req.Uuid})
}
-55
View File
@@ -1,55 +0,0 @@
package client
import (
"net/http"
"time"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/database/tasks"
)
func GetPingTasks(c *gin.Context) {
uuid, _ := c.Get("client_uuid")
clientUUID, _ := uuid.(string)
if clientUUID == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "client_uuid not found"})
return
}
result := tasks.GetPingTasksByClient(clientUUID)
c.JSON(http.StatusOK, result)
}
func UploadPingResult(c *gin.Context) {
uuid, _ := c.Get("client_uuid")
clientUUID, _ := uuid.(string)
if clientUUID == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "client_uuid not found"})
return
}
var req struct {
TaskID uint `json:"task_id" binding:"required"`
Value int `json:"value"`
PingType string `json:"ping_type"`
FinishedAt time.Time `json:"finished_at" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid request: " + err.Error()})
return
}
record := models.PingRecord{
Client: clientUUID,
TaskId: req.TaskID,
Value: req.Value,
Time: models.FromTime(req.FinishedAt),
}
if err := tasks.SavePingRecord(record); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to save ping result: " + err.Error()})
return
}
c.JSON(http.StatusOK, gin.H{"status": "success"})
}
-36
View File
@@ -1,36 +0,0 @@
package client
import (
"time"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/database/clients"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/database/tasks"
)
func TaskResult(c *gin.Context) {
token := c.Query("token")
clientId, _ := clients.GetClientUUIDByToken(token)
if clientId == "" {
c.JSON(400, gin.H{"status": "error", "message": "Invalid or missing token"})
return
}
var req struct {
TaskId string `json:"task_id" binding:"required"`
Result string `json:"result"`
ExitCode int `json:"exit_code"`
FinishedAt time.Time `json:"finished_at" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(400, gin.H{"status": "error", "message": "Invalid request"})
return
}
if err := tasks.SaveTaskResult(req.TaskId, clientId, req.Result, req.ExitCode, models.FromTime(req.FinishedAt)); err != nil {
c.JSON(500, gin.H{"status": "error", "message": "Failed to update task result: " + err.Error()})
return
}
c.JSON(200, gin.H{"status": "success", "message": "Task result updated successfully"})
}
-85
View File
@@ -1,85 +0,0 @@
package client
import (
"net"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/clients"
"github.com/komari-monitor/komari/utils/geoip"
"github.com/gin-gonic/gin"
)
func getClientIPType(ip net.IP) int {
// 0:ipv4 1:ipv6 -1:错误的输入
if ip == nil {
return -1
}
if ip.To4() == nil {
return 1
} else {
return 0
}
}
func UploadBasicInfo(c *gin.Context) {
var cbi = map[string]interface{}{}
if err := c.ShouldBindJSON(&cbi); err != nil {
c.JSON(400, gin.H{"status": "error", "error": "Invalid or missing data"})
return
}
token := c.Query("token")
uuid, err := clients.GetClientUUIDByToken(token)
if uuid == "" || err != nil {
c.JSON(400, gin.H{"status": "error", "error": "Invalid token"})
return
}
cbi["uuid"] = uuid
// 如果没有传入 IP 信息,尝试从请求中获取
if (func() bool {
if v4, ok := cbi["ipv4"].(string); !ok || v4 == "" {
if v6, ok := cbi["ipv6"].(string); !ok || v6 == "" {
return true
}
}
return false
})() {
ipStr := c.ClientIP()
ip := net.ParseIP(ipStr)
ipType := getClientIPType(ip)
switch ipType {
case 0:
cbi["ipv4"] = ipStr
case 1:
cbi["ipv6"] = ipStr
default:
break
}
}
if cfg, err := config.GetAs[bool](config.GeoIpEnabledKey); err == nil && cfg {
if ipv4, ok := cbi["ipv4"].(string); ok && ipv4 != "" {
ip4 := net.ParseIP(ipv4)
ip4_record, _ := geoip.GetGeoInfo(ip4)
if ip4_record != nil {
cbi["region"] = geoip.GetRegionUnicodeEmoji(ip4_record.ISOCode)
}
} else if ipv6, ok := cbi["ipv6"].(string); ok && ipv6 != "" {
ip6 := net.ParseIP(ipv6)
ip6_record, _ := geoip.GetGeoInfo(ip6)
if ip6_record != nil {
cbi["region"] = geoip.GetRegionUnicodeEmoji(ip6_record.ISOCode)
}
}
}
if err := clients.SaveClientInfo(cbi); err != nil {
c.JSON(500, gin.H{"status": "error", "error": err})
return
}
c.JSON(200, gin.H{"status": "success"})
}
-273
View File
@@ -1,273 +0,0 @@
package jsonRpc
import (
"context"
"encoding/json"
"errors"
"io"
"net/http"
"reflect"
"strings"
"sync"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/clients"
"github.com/komari-monitor/komari/utils/rpc"
"github.com/komari-monitor/komari/ws"
)
// OnInternalRequest 内部调用 RPC 方法,支持权限控制
// group: 调用者的权限分组 (guest/client/admin)
// method: RPC 方法名 (如 "common:getNodes")
// params: 方法参数
func OnInternalRequest(ctx context.Context, group string, method string, params interface{}) *rpc.JsonRpcResponse {
// 检查私有站点
PrivateSite, _ := config.GetAs[bool](config.PrivateSiteKey)
if PrivateSite && group == "guest" {
return rpc.ErrorResponse(nil, rpc.PermissionDenied, "Private site enabled, please login first", nil)
}
// 解析方法命名空间
fc := strings.Split(method, ":")
namespace := "common"
if len(fc) >= 2 {
namespace = fc[0]
}
// 权限检查
allowed := false
switch namespace {
case "guest", "", "rpc", "common":
allowed = true
case "client":
if group == "client" || group == "admin" {
allowed = true
}
case "admin":
if group == "admin" {
allowed = true
}
}
if !allowed {
return rpc.ErrorResponse(nil, rpc.PermissionDenied, "Permission denied", nil)
}
// 构建 context meta
meta := &rpc.ContextMeta{Permission: group}
return rpc.CallWithContext(rpc.NewContextWithMeta(ctx, meta), nil, method, params)
}
// Json Rpc2 over websocket, /api/rpc2
func OnRpcRequest(c *gin.Context) {
AllowCors, _ := config.GetAs[bool](config.AllowCorsKey)
// GET -> WebSocket
if c.Request.Method == http.MethodGet {
_conn, err := ws.UpgradeRequest(c, func(r *http.Request) bool {
if AllowCors {
return true
}
return ws.CheckOrigin(r)
})
conn := ws.NewSafeConn(_conn)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"status": "error", "error": "Failed to upgrade to WebSocket." + err.Error()})
return
}
permissionGroup := detectPermissionGroup(c)
meta := buildContextMeta(c, permissionGroup)
defer conn.Close()
for {
var req rpc.JsonRpcRequest
err := conn.ReadJSON(&req)
if err != nil {
// JSON 解析
var se *json.SyntaxError
var ute *json.UnmarshalTypeError
if errors.As(err, &se) || errors.As(err, &ute) {
conn.WriteJSON(rpc.ErrorResponse(nil, rpc.InvalidRequest, "bad request: "+err.Error(), nil))
continue
}
// 其它视为连接/IO 错误
break
}
if jerr := req.Validate(); jerr != nil {
conn.WriteJSON(jerr.ResponseWithID(req.ID))
continue
}
dispatchByPermissionWithMeta(conn, permissionGroup, meta, &req)
}
return
}
// 否则按 HTTP POST JSON-RPC 处理 (支持单个或批量)
if c.Request.Method != http.MethodPost {
c.JSON(http.StatusMethodNotAllowed, gin.H{"error": "method not allowed"})
return
}
body, err := io.ReadAll(c.Request.Body)
if err != nil {
c.JSON(http.StatusBadRequest, rpc.ErrorResponse(nil, rpc.ParseError, "read body error", err.Error()))
return
}
requests, jerr := rpc.ParseRequests(body)
if jerr != nil {
c.JSON(http.StatusBadRequest, jerr.Response())
return
}
permissionGroup := detectPermissionGroup(c)
meta := buildContextMeta(c, permissionGroup)
// 批量
responses := make([]*rpc.JsonRpcResponse, 0, len(requests))
for _, rreq := range requests {
// 权限与命名空间
fc := strings.Split(rreq.Method, ":")
if len(fc) == 1 {
fc[0] = "common"
}
allowed := false
switch fc[0] {
case "guest", "", "rpc", "common":
allowed = true
case "client":
if permissionGroup == "client" || permissionGroup == "admin" {
allowed = true
}
case "admin":
if permissionGroup == "admin" {
allowed = true
}
default:
responses = append(responses, rpc.ErrorResponse(rreq.ID, 401, "Unauthorized", nil))
continue
}
if !allowed {
responses = append(responses, rpc.ErrorResponse(rreq.ID, 401, "Unauthorized", nil))
continue
}
responses = append(responses, rpc.CallWithContext(rpc.NewContextWithMeta(context.TODO(), meta), rreq.ID, rreq.Method, rreq.Params))
}
// 单个请求直接对象,批量请求数组 (符合 JSON-RPC 2.0)
if len(responses) == 1 {
c.JSON(http.StatusOK, responses[0])
} else {
c.JSON(http.StatusOK, responses)
}
}
// detectPermissionGroup 提取权限分组,与原逻辑保持一致
func detectPermissionGroup(c *gin.Context) string {
permissionGroup := "guest"
token := c.Query("Authorization")
if _, err := clients.GetClientUUIDByToken(token); err == nil {
permissionGroup = "client"
}
if session_token, _ := c.Cookie("session_token"); session_token != "" {
if _, err := accounts.GetUserBySession(session_token); err == nil {
permissionGroup = "admin"
}
}
apiKey := c.GetHeader("Authorization")
if len(apiKey) <= len("Bearer ") {
return permissionGroup
}
cfg, _ := config.GetAs[string](config.ApiKeyKey, "")
if len(cfg) > 8 && apiKey == "Bearer "+cfg {
permissionGroup = "admin"
}
return permissionGroup
}
// buildContextMeta 从 gin.Context 构建 *rpc.ContextMeta
func buildContextMeta(c *gin.Context, permissionGroup string) *rpc.ContextMeta {
meta := &rpc.ContextMeta{Permission: permissionGroup}
// 提取客户端 token (query Authorization / Header Authorization Bearer token)
token := c.Query("Authorization")
if token == "" {
// 兼容 header Bearer <token>
hAuth := c.GetHeader("Authorization")
if strings.HasPrefix(hAuth, "Bearer ") {
token = strings.TrimPrefix(hAuth, "Bearer ")
}
}
if token != "" {
if uuid, err := clients.GetClientUUIDByToken(token); err == nil {
meta.ClientToken = token
meta.ClientUUID = uuid
}
}
// 提取用户 (session cookie)
if session_token, _ := c.Cookie("session_token"); session_token != "" {
if user, err := accounts.GetUserBySession(session_token); err == nil {
meta.User = &user
meta.UserUUID = user.UUID
}
}
meta.RemoteIP = c.ClientIP()
meta.UserAgent = c.GetHeader("User-Agent")
return meta
}
// dispatchByPermissionWithMeta 与原函数类似,但会携带 meta 上下文给 handler
func dispatchByPermissionWithMeta(conn *ws.SafeConn, permissionGroup string, meta *rpc.ContextMeta, req *rpc.JsonRpcRequest) {
fc := strings.Split(req.Method, ":")
if len(fc) == 1 {
fc[0] = "common"
}
ctx := rpc.NewContextWithMeta(context.TODO(), meta)
switch fc[0] {
case "guest", "", "rpc", "common":
go conn.WriteJSON(rpc.CallWithContext(ctx, req.ID, req.Method, req.Params))
case "client":
if permissionGroup == "client" || permissionGroup == "admin" {
go conn.WriteJSON(rpc.CallWithContext(ctx, req.ID, req.Method, req.Params))
} else {
conn.WriteJSON(rpc.ErrorResponse(req.ID, 401, "Unauthorized", nil))
}
case "admin":
if permissionGroup == "admin" {
go conn.WriteJSON(rpc.CallWithContext(ctx, req.ID, req.Method, req.Params))
} else {
conn.WriteJSON(rpc.ErrorResponse(req.ID, 401, "Unauthorized", nil))
}
default:
conn.WriteJSON(rpc.ErrorResponse(req.ID, 401, "Unauthorized", nil))
}
}
// registry holds method handlers keyed by "namespace:MethodName"
var (
registryMu sync.RWMutex
registry = make(map[string]reflect.Value)
)
// Register 将以分组common注册
func Register(name string, cb rpc.Handler) error {
return RegisterWithGroupAndMeta(name, "common", cb, &rpc.MethodMeta{
Name: name,
Summary: "This method does not provide a summary",
Description: "This method does not provide a description",
})
}
// RegisterWithGroup 将回调按分组注册。
// 传入函数: Register("ns", someFunc) -> 注册 ns:SomeFunc
// group 为空时使用默认分组 "common"。
func RegisterWithGroupAndMeta(name, group string, cb rpc.Handler, meta *rpc.MethodMeta) error {
return rpc.RegisterWithMeta(group+":"+name, cb, meta)
}
// GetRegisteredKeys 返回已注册的方法 key 列表 (调试用)。
func GetRegisteredKeys() []string {
registryMu.RLock()
defer registryMu.RUnlock()
keys := make([]string, 0, len(registry))
for k := range registry {
keys = append(keys, k)
}
return keys
}
-45
View File
@@ -1,45 +0,0 @@
package public
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
func GetClientRecentRecords(c *gin.Context) {
uuid := c.Param("uuid")
if uuid == "" {
api.RespondError(c, 400, "UUID is required")
return
}
// 登录状态检查
isLogin := false
session, _ := c.Cookie("session_token")
_, err := accounts.GetUserBySession(session)
if err == nil {
isLogin = true
}
// 仅在未登录时需要 Hidden 信息做过滤
hiddenMap := map[string]bool{}
if !isLogin {
var hiddenClients []models.Client
db := dbcore.GetDBInstance()
_ = db.Select("uuid").Where("hidden = ?", true).Find(&hiddenClients).Error
for _, cli := range hiddenClients {
hiddenMap[cli.UUID] = true
}
if hiddenMap[uuid] {
api.RespondError(c, 400, "UUID is required") //防止未登录用户获取隐藏客户端数据
return
}
}
records, _ := api.Records.Get(uuid)
api.RespondSuccess(c, records)
}
-27
View File
@@ -1,27 +0,0 @@
package public
import (
"github.com/komari-monitor/komari/database/accounts"
"github.com/gin-gonic/gin"
)
func GetMe(c *gin.Context) {
session, err := c.Cookie("session_token")
if err != nil {
c.JSON(200, gin.H{"username": "Guest", "logged_in": false})
return
}
uuid, err := accounts.GetSession(session)
if err != nil {
c.JSON(200, gin.H{"username": "Guest", "logged_in": false})
return
}
user, err := accounts.GetUserByUUID(uuid)
if err != nil {
c.JSON(200, gin.H{"username": "Guest", "logged_in": false})
return
}
c.JSON(200, gin.H{"username": user.Username, "logged_in": true, "uuid": user.UUID, "sso_type": user.SSOType, "sso_id": user.SSOID, "2fa_enabled": user.TwoFactor != ""})
}
-95
View File
@@ -1,95 +0,0 @@
package public
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/database/accounts"
"github.com/stretchr/testify/assert"
)
func TestGetMe(t *testing.T) {
// 设置测试模式
gin.SetMode(gin.TestMode)
// 创建测试用户
user, err := accounts.CreateAccount("testuser", "password")
if err != nil {
t.Fatalf("Failed to create test user: %v", err)
}
uuid := user.UUID
sessionToken, _ := accounts.CreateSession(uuid, 2592000, "test_user_agent", "127.0.0.1", "oauth")
tests := []struct {
name string
sessionToken string
expectedStatus int
expectedBody map[string]interface{}
}{
{
name: "有效的会话",
sessionToken: sessionToken,
expectedStatus: http.StatusOK,
expectedBody: map[string]interface{}{
"username": "testuser",
},
},
{
name: "无效的会话",
sessionToken: "invalid_token",
expectedStatus: http.StatusOK,
expectedBody: map[string]interface{}{
"username": "Guest",
},
},
{
name: "无会话",
sessionToken: "",
expectedStatus: http.StatusOK,
expectedBody: map[string]interface{}{
"username": "Guest",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// 创建测试路由
router := gin.New()
router.GET("/me", GetMe)
// 创建测试请求
req, _ := http.NewRequest("GET", "/me", nil)
if tt.sessionToken != "" {
req.AddCookie(&http.Cookie{
Name: "session_token",
Value: tt.sessionToken,
})
}
// 创建响应记录器
w := httptest.NewRecorder()
// 执行请求
router.ServeHTTP(w, req)
// 断言状态码
assert.Equal(t, tt.expectedStatus, w.Code)
// 解析响应体
var response map[string]interface{}
err := json.Unmarshal(w.Body.Bytes(), &response)
assert.NoError(t, err)
// 断言响应体
assert.Equal(t, tt.expectedBody, response)
})
}
// 清除测试数据
accounts.DeleteAccountByUsername("testuser")
accounts.DeleteAllSessions()
}
-40
View File
@@ -1,40 +0,0 @@
package public
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/clients"
)
func GetNodesInformation(c *gin.Context) {
clientList, err := clients.GetAllClientBasicInfo()
if err != nil {
api.RespondError(c, 500, "Failed to retrieve client information: "+err.Error())
return
}
isLogin := false
session, _ := c.Cookie("session_token")
_, err = accounts.GetUserBySession(session)
if err == nil {
isLogin = true
}
// 过滤掉 Hidden 的客户端,并清理需要隐藏的字段
j := 0
for i := 0; i < len(clientList); i++ {
if clientList[i].Hidden && !isLogin { // 不返回 Hidden 客户端
continue
}
clientList[i].IPv4 = ""
clientList[i].IPv6 = ""
clientList[i].Remark = "" // 私有备注不展示
clientList[i].Version = ""
clientList[i].Token = ""
clientList[j] = clientList[i]
j++
}
clientList = clientList[:j]
api.RespondSuccess(c, clientList)
}
-41
View File
@@ -1,41 +0,0 @@
package public
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/tasks"
)
// PublicPingTask 是对外暴露的延迟监测任务信息。
type PublicPingTask struct {
Id uint `json:"id"`
Name string `json:"name"`
Clients []string `json:"clients"`
DefaultOn bool `json:"default_on"`
Type string `json:"type"`
Interval int `json:"interval"`
}
func GetPublicPingTasks(c *gin.Context) {
tasks, err := tasks.GetAllPingTasks()
if err != nil {
api.RespondError(c, http.StatusInternalServerError, err.Error())
return
}
publicTasks := make([]PublicPingTask, len(tasks))
for i, task := range tasks {
publicTasks[i] = PublicPingTask{
Id: task.Id,
Name: task.Name,
Clients: task.Clients,
DefaultOn: task.DefaultOn,
Type: task.Type,
Interval: task.Interval,
}
}
api.RespondSuccess(c, publicTasks)
}
-47
View File
@@ -1,47 +0,0 @@
package public
import (
"time"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database"
)
func GetPublicSettings(c *gin.Context) {
p, e := database.GetPublicInfo()
if e != nil {
api.RespondError(c, 500, e.Error())
return
}
// 临时访问许可
if func() bool {
tempKey, err := c.Cookie("temp_key")
if err != nil {
return false
}
tempKeyExpireTime, err := config.GetAs[int64]("tempory_share_token_expire_at", 0)
if err != nil {
return false
}
allowTempKey, err := config.GetAs[string]("tempory_share_token", "")
if err != nil {
return false
}
if allowTempKey == "" || tempKey != allowTempKey {
return false
}
now := time.Now().Unix()
if tempKeyExpireTime < now {
return false
}
return true
}() {
p["private_site"] = false
}
api.RespondSuccess(c, p)
}
-446
View File
@@ -1,446 +0,0 @@
package public
import (
"strconv"
"time"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
records "github.com/komari-monitor/komari/database/records"
"github.com/komari-monitor/komari/database/tasks"
)
func GetRecordsByUUID(c *gin.Context) {
uuid := c.Query("uuid")
loadType := c.Query("load_type")
// 登录状态检查
isLogin := false
session, _ := c.Cookie("session_token")
_, err := accounts.GetUserBySession(session)
if err == nil {
isLogin = true
}
// 仅在未登录时需要 Hidden 信息做过滤
hiddenMap := map[string]bool{}
if !isLogin {
var hiddenClients []models.Client
db := dbcore.GetDBInstance()
_ = db.Select("uuid").Where("hidden = ?", true).Find(&hiddenClients).Error
for _, cli := range hiddenClients {
hiddenMap[cli.UUID] = true
}
if hiddenMap[uuid] {
api.RespondError(c, 400, "UUID is required") //防止未登录用户获取隐藏客户端数据
return
}
}
hours := c.Query("hours")
if uuid == "" {
api.RespondError(c, 400, "UUID is required")
return
}
if hours == "" {
hours = "4"
}
hoursInt, err := strconv.Atoi(hours)
if err != nil {
api.RespondError(c, 400, "Invalid hours parameter")
return
}
// 验证 load_type 参数
validLoadTypes := map[string]bool{
"cpu": true, "ram": true, "swap": true,
"load": true, "temp": true, "disk": true, "network": true,
"process": true, "connections": true, "all": true, "": true,
}
if !validLoadTypes[loadType] {
api.RespondError(c, 400, "Invalid load_type parameter")
return
}
clientRecords, err := records.GetRecordsByClientAndTime(uuid, time.Now().Add(-time.Duration(hoursInt)*time.Hour), time.Now())
if err != nil {
api.RespondError(c, 500, "Failed to fetch records: "+err.Error())
return
}
// 准备基本响应
response := gin.H{
"records": clientRecords,
"count": len(clientRecords),
}
// 如果有load_type过滤,应用过滤
if loadType != "" && loadType != "all" {
filteredRecords := filterRecordsByLoadType(clientRecords, loadType)
response = gin.H{
"records": filteredRecords,
"count": len(filteredRecords),
"load_type": loadType,
}
}
// 自动检测是否有GPU数据并附加到响应中
if loadType == "" || loadType == "all" || loadType == "gpu" {
gpuRecords, err := records.GetGPURecordsByClientAndTime(uuid, time.Now().Add(-time.Duration(hoursInt)*time.Hour), time.Now())
if err == nil && len(gpuRecords) > 0 {
// 按设备索引分组数据,构建简化的设备结构
gpuDevices := make(map[string]interface{})
for _, record := range gpuRecords {
deviceKey := strconv.Itoa(record.DeviceIndex)
// 如果设备还没有初始化,创建设备信息
if gpuDevices[deviceKey] == nil {
gpuDevices[deviceKey] = gin.H{
"device_index": record.DeviceIndex,
"device_name": record.DeviceName,
"records": []models.GPURecord{},
}
}
// 添加记录到设备
device := gpuDevices[deviceKey].(gin.H)
records := device["records"].([]models.GPURecord)
device["records"] = append(records, record)
gpuDevices[deviceKey] = device
}
// 添加优化后的GPU数据结构到响应
response["gpu_devices"] = gpuDevices
response["has_gpu_data"] = true
} else {
response["has_gpu_data"] = false
}
}
api.RespondSuccess(c, response)
}
// filterRecordsByLoadType 根据 load_type 过滤记录,只返回相关字段
func filterRecordsByLoadType(records []models.Record, loadType string) []gin.H {
filteredRecords := make([]gin.H, 0, len(records))
for _, r := range records {
record := gin.H{
"client": r.Client,
"time": r.Time,
}
switch loadType {
case "cpu":
record["cpu"] = r.Cpu
case "gpu":
record["gpu"] = r.Gpu
case "ram":
record["ram"] = r.Ram
record["ram_total"] = r.RamTotal
if r.RamTotal > 0 {
record["ram_percent"] = float32(r.Ram) / float32(r.RamTotal) * 100
}
case "swap":
record["swap"] = r.Swap
record["swap_total"] = r.SwapTotal
if r.SwapTotal > 0 {
record["swap_percent"] = float32(r.Swap) / float32(r.SwapTotal) * 100
}
case "load":
record["load"] = r.Load
case "temp":
record["temp"] = r.Temp
case "disk":
record["disk"] = r.Disk
record["disk_total"] = r.DiskTotal
if r.DiskTotal > 0 {
record["disk_percent"] = float32(r.Disk) / float32(r.DiskTotal) * 100
}
case "network":
record["net_in"] = r.NetIn
record["net_out"] = r.NetOut
record["net_total_up"] = r.NetTotalUp
record["net_total_down"] = r.NetTotalDown
case "process":
record["process"] = r.Process
case "connections":
record["connections"] = r.Connections
record["connections_udp"] = r.ConnectionsUdp
record["connections_tcp"] = r.Connections - r.ConnectionsUdp
}
filteredRecords = append(filteredRecords, record)
}
return filteredRecords
}
// GET query: uuid string, task_id int, hours int
// 支持三种查询模式:
// 1. 仅 uuid - 获取该客户端的所有 ping 任务记录
// 2. 仅 task_id - 获取该任务的所有客户端记录
// 3. uuid + task_id - 获取特定客户端在特定任务的记录
func GetPingRecords(c *gin.Context) {
uuid := c.Query("uuid")
taskIdStr := c.Query("task_id")
// 必须提供 uuid 或 task_id 其中至少一个
if uuid == "" && taskIdStr == "" {
api.RespondError(c, 400, "UUID or task_id is required")
return
}
// 登录状态检查
isLogin := false
session, _ := c.Cookie("session_token")
_, err := accounts.GetUserBySession(session)
if err == nil {
isLogin = true
}
type RecordsResp struct {
TaskId uint `json:"task_id,omitempty"`
Time string `json:"time"`
Value int `json:"value"`
Client string `json:"client,omitempty"`
}
type ClientBasicInfo struct {
Client string `json:"client"`
Loss float64 `json:"loss"`
Min int `json:"min"`
Max int `json:"max"`
}
type Resp struct {
Count int `json:"count"`
BasicInfo []ClientBasicInfo `json:"basic_info,omitempty"`
Records []RecordsResp `json:"records"`
Tasks []gin.H `json:"tasks,omitempty"`
}
var records []models.PingRecord
hiddenMap := map[string]bool{}
response := &Resp{
Count: 0,
Records: []RecordsResp{},
}
// 仅在未登录时需要 Hidden 信息做过滤
if !isLogin {
var hiddenClients []models.Client
db := dbcore.GetDBInstance()
_ = db.Select("uuid").Where("hidden = ?", true).Find(&hiddenClients).Error
for _, cli := range hiddenClients {
hiddenMap[cli.UUID] = true
}
if uuid != "" {
if hiddenMap[uuid] {
api.RespondSuccess(c, response) // 对于尝试获取隐藏uuid一键哈气
return
}
}
}
hours := c.Query("hours")
if hours == "" {
hours = "4"
}
hoursInt, err := strconv.Atoi(hours)
if err != nil {
hoursInt = 4
}
endTime := time.Now()
startTime := endTime.Add(-time.Duration(hoursInt) * time.Hour)
// 解析 task_id,支持同时传入 uuid 和 task_id
taskId := -1
if taskIdStr != "" {
taskId, err = strconv.Atoi(taskIdStr)
if err != nil {
api.RespondError(c, 400, "Invalid task_id parameter")
return
}
}
// 查询记录,现在支持 uuid + task_id 组合查询
records, err = tasks.GetPingRecords(uuid, taskId, startTime, endTime)
if err != nil {
api.RespondError(c, 500, "Failed to fetch ping records: "+err.Error())
return
}
// 用于统计每个客户端的信息(按 task_id 查询时使用)
clientStats := make(map[string]struct {
total int
loss int
min int
max int
})
for _, r := range records {
if r.Client != "" && !isLogin {
if hiddenMap[r.Client] {
continue // 跳过隐藏的节点
}
}
toTime := r.Time.ToTime()
rec := RecordsResp{
Time: toTime.Format(time.RFC3339),
Value: r.Value,
}
rec.Client = r.Client
stats := clientStats[r.Client]
stats.total++
if r.Value < 0 {
stats.loss++
} else {
if stats.min == 0 || r.Value < stats.min {
stats.min = r.Value
}
if r.Value > stats.max {
stats.max = r.Value
}
}
clientStats[r.Client] = stats
rec.TaskId = r.TaskId
response.Records = append(response.Records, rec)
}
// 返回 BasicInfo - 按客户端分组的统计信息
// 在以下情况下特别有用:
// 1. 仅 task_id 查询 - 查看该任务下所有客户端的表现
// 2. uuid + task_id 查询 - 查看特定客户端在特定任务的表现
if len(clientStats) > 0 {
response.BasicInfo = make([]ClientBasicInfo, 0, len(clientStats))
for client, stats := range clientStats {
if client != "" && !isLogin {
if hiddenMap[client] {
continue // 跳过隐藏的节点
}
}
loss := float64(0)
if stats.total > 0 {
loss = float64(stats.loss) / float64(stats.total) * 100
}
response.BasicInfo = append(response.BasicInfo, ClientBasicInfo{
Client: client,
Loss: loss,
Min: stats.min,
Max: stats.max,
})
}
// 如果同时指定了 uuid 和 task_id,BasicInfo 应该只有一条记录
// 这种情况下可以在响应中标记查询模式
if uuid != "" && taskId != -1 && len(response.BasicInfo) == 1 {
// 这是精确查询模式
}
}
// 优化后的任务信息返回逻辑
// 1. uuid != "" - 返回该客户端参与的所有任务信息
// 2. uuid != "" && taskId != -1 - 返回该客户端在指定任务的信息
// 3. taskId != -1 && uuid == "" - 返回该任务的所有客户端统计(通过 BasicInfo)
if uuid != "" || taskId != -1 {
// 获取所有 pingTasks
pingTasks, err := tasks.GetAllPingTasks()
if err != nil {
api.RespondError(c, 500, "Failed to fetch ping tasks: "+err.Error())
return
}
tasksList := make([]gin.H, 0, len(pingTasks))
for _, t := range pingTasks {
// 如果指定了 taskId,只处理该任务
if taskId != -1 {
if t.Id != uint(taskId) {
continue
}
}
// 如果指定了 uuid,检查任务是否分配给该客户端
if uuid != "" {
if !t.AppliesToClient(uuid) {
continue
}
}
// 计算该任务的丢包率和延迟统计
totalCount := 0
lossCount := 0
minLatency := 0
maxLatency := 0
avgLatency := 0
sumLatency := 0
validCount := 0
for _, r := range records {
// 根据查询模式过滤记录
if r.TaskId != t.Id {
continue
}
// 如果同时指定了 uuid 和 task_id,只统计该客户端的记录
if uuid != "" && r.Client != uuid {
continue
}
totalCount++
if r.Value < 0 {
lossCount++
} else {
validCount++
sumLatency += r.Value
if minLatency == 0 || r.Value < minLatency {
minLatency = r.Value
}
if r.Value > maxLatency {
maxLatency = r.Value
}
}
}
var lossRate float64 = 0
if totalCount > 0 {
lossRate = float64(lossCount) / float64(totalCount) * 100
}
if validCount > 0 {
avgLatency = sumLatency / validCount
}
taskInfo := gin.H{
"id": t.Id,
"name": t.Name,
"type": t.Type,
"interval": t.Interval,
"default_on": t.DefaultOn,
"loss": lossRate,
"min": minLatency,
"max": maxLatency,
"avg": avgLatency,
"total": totalCount,
}
// 如果是仅 task_id 查询,添加客户端列表信息
if uuid == "" && taskId != -1 {
taskInfo["clients"] = t.Clients
}
tasksList = append(tasksList, taskInfo)
}
response.Tasks = tasksList
}
response.Count = len(response.Records) // 计算最后结果保持计数一致
api.RespondSuccess(c, response)
}
-14
View File
@@ -1,14 +0,0 @@
package public
import (
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/utils"
)
func GetVersion(c *gin.Context) {
api.RespondSuccess(c, gin.H{
"version": utils.CurrentVersion,
"hash": utils.VersionHash,
})
}
+498
View File
@@ -0,0 +1,498 @@
package cmd
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"path/filepath"
"strings"
"syscall"
"time"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/cmd/flags"
"github.com/komari-monitor/komari/database"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/metricstore"
"github.com/komari-monitor/komari/database/models"
d_notification "github.com/komari-monitor/komari/database/notification"
"github.com/komari-monitor/komari/database/tasks"
"github.com/komari-monitor/komari/pkg/config"
"github.com/komari-monitor/komari/pkg/corn"
"github.com/komari-monitor/komari/pkg/migrations"
"github.com/komari-monitor/komari/utils"
"github.com/komari-monitor/komari/utils/geoip"
logutil "github.com/komari-monitor/komari/utils/log"
"github.com/komari-monitor/komari/utils/messageSender"
"github.com/komari-monitor/komari/utils/notifier"
"github.com/komari-monitor/komari/web/api"
"github.com/komari-monitor/komari/web/oauth"
frontendpublic "github.com/komari-monitor/komari/web/public"
"github.com/komari-monitor/komari/web/router"
"github.com/komari-monitor/komari/web/security"
upgradeweb "github.com/komari-monitor/komari/web/update"
)
// cleanupFunc 是一个关闭阶段执行的清理函数。
type cleanupFunc struct {
name string
fn func(ctx context.Context) error
}
// App 显式建模服务端的启动生命周期。
//
// 过去 RunServer 把目录创建、数据库、metric store、GeoIP、定时任务、通知、OAuth、
// Gin 中间件、路由、HTTP 启动与 shutdown 全部混在一个函数里,
// 启动顺序只能靠通读整段代码推断,异步初始化过早且吞掉错误,关闭也不完整。
//
// App 把这些拆成有序阶段:
//
// Bootstrap 基础设施:目录、数据库、默认管理员、配置快照
// InitStores 存储:metric store
// InitProviders 外部 provider:OAuth(同步,路由依赖)、GeoIP、消息发送
// StartBackground 后台:定时任务
// BuildRouter 构建 Gin 引擎与路由
// Run 启动 HTTP 服务并阻塞直到收到信号
// Shutdown 反序执行已登记的清理函数
//
// 每个阶段返回错误即可让上层决定是否中止启动;各资源在创建时把对应清理登记到
// cleanup 栈,关闭时按后进先出(LIFO)顺序释放。
type App struct {
settings *config.Settings
engine *gin.Engine
server *http.Server
reload *ReloadManager
dbReady bool
oauthReady bool
cleanups []cleanupFunc
}
// NewApp 构造一个空的 App。真正的初始化在各阶段方法中完成。
func NewApp() *App {
return &App{
reload: NewReloadManager(),
}
}
// addCleanup 登记一个关闭阶段的清理函数(后进先出执行)。
func (a *App) addCleanup(name string, fn func(ctx context.Context) error) {
a.cleanups = append(a.cleanups, cleanupFunc{name: name, fn: fn})
}
// Bootstrap 初始化基础设施:数据目录、数据库、默认管理员账号、配置快照。
func (a *App) Bootstrap() error {
if err := os.MkdirAll("./data/theme", os.ModePerm); err != nil {
return fmt.Errorf("failed to create theme directory: %w", err)
}
// 注入版本标识,供 dbcore 在检测到版本升级时自动备份 ./data。
// 必须在 Initialize() 之前设置。
dbcore.SetVersionID(utils.CurrentVersion + "-" + utils.VersionHash)
// 显式初始化数据库(返回错误而非在 getter 里 log.Fatal)。
if err := dbcore.Initialize(); err != nil {
return fmt.Errorf("failed to initialize database: %w", err)
}
a.dbReady = true
a.addCleanup("database", func(context.Context) error {
return dbcore.Close()
})
if utils.VersionHash != "unknown" {
gin.SetMode(gin.ReleaseMode)
}
// 首次启动创建默认管理员账号。
if err := ensureDefaultAdmin(); err != nil {
return fmt.Errorf("failed to ensure default admin account: %w", err)
}
conf, err := config.GetManyAs[config.Settings]()
if err != nil {
return fmt.Errorf("failed to load settings: %w", err)
}
a.settings = conf
return nil
}
// InitStores 初始化独立存储组件(metric store)并执行 metrics 迁移。
//
// metric store 现在始终启用(旧的 metric_store_enabled 开关已废弃):
// 未显式配置时使用 SQLite(./data/metrics.db),否则使用配置的 MySQL/PostgreSQL。
// 初始化失败即启动失败,不再静默 fallback 到旧 records 表。
//
// 初始化成功后先执行需要 metric store 的一次性迁移,再执行启动迁移:当 metrics
// 存储后端发生变化(例如从默认 SQLite 切换到 MySQL/PostgreSQL)时,把上一个
// metrics 目标库的数据搬运到当前目标。迁移失败同样让启动失败,并打印明确错误。
func (a *App) InitStores() error {
if err := metricstore.InitializeStore(); err != nil {
auditlog.EventLog("error", fmt.Sprintf("Failed to initialize metric store: %v", err))
return fmt.Errorf("failed to initialize metric store: %w", err)
}
a.addCleanup("metric-store", func(ctx context.Context) error {
return metricstore.CloseStoreContext(ctx)
})
if err := migrations.RunMetricStoreMigrations(migrations.MetricContext{
DB: dbcore.GetDBInstance(),
Store: metricstore.GetStore(),
}); err != nil {
auditlog.EventLog("error", fmt.Sprintf("Metric store one-shot migrations failed: %v", err))
return fmt.Errorf("metric store one-shot migrations failed: %w", err)
}
// 存储后端切换时把上一个 metrics 目标库的数据搬运到当前目标(失败即启动失败)。
if err := metricstore.RunStartupMigration(); err != nil {
auditlog.EventLog("error", fmt.Sprintf("Metrics startup migration failed: %v", err))
return fmt.Errorf("metrics startup migration failed: %w", err)
}
metricstore.StartReportBatcher()
a.addCleanup("metric-report-batcher", func(ctx context.Context) error {
return metricstore.StopReportBatcher(ctx)
})
return nil
}
// InitProviders 初始化外部 provider。
//
// OAuth 必须在 HTTP 服务开始接收请求之前同步完成,否则 oauth.CurrentProvider()
// 存在空指针风险;GeoIP 与消息发送允许后台初始化。
func (a *App) InitProviders() error {
a.initOAuth()
// GeoIP:可能涉及下载/加载 mmdb,放后台执行避免拖慢启动。
go geoip.InitGeoIp()
a.addCleanup("geoip", func(context.Context) error {
return geoip.Shutdown()
})
// 消息发送 provider。
messageSender.Initialize()
a.addCleanup("message-sender", func(context.Context) error {
return messageSender.Shutdown()
})
return nil
}
// initOAuth initializes the provider once. The restricted upgrade server also
// needs OAuth login, so provider setup may happen before the normal provider
// stage without registering duplicate cleanup work.
func (a *App) initOAuth() {
if a.oauthReady {
return
}
if err := oauth.Initialize(); err != nil {
// Keep password login available when an OAuth provider is misconfigured.
log.Printf("Failed to initialize OAuth provider: %v", err)
auditlog.EventLog("error", fmt.Sprintf("Failed to initialize OAuth provider: %v", err))
}
a.oauthReady = true
a.addCleanup("oauth", func(context.Context) error {
return oauth.Shutdown()
})
}
func (a *App) LegacyUpgradeRequired() (bool, migrations.LegacyMonitoringSummary, error) {
return migrations.LegacyMonitoringMigrationRequired(dbcore.GetDBInstance())
}
// RunLegacyUpgrade starts a restricted HTTP server on the normal listen
// address. It returns only after the migration finishes and the restricted
// listener has released the port, or after shutdown/interruption.
func (a *App) RunLegacyUpgrade(summary migrations.LegacyMonitoringSummary) (bool, error) {
a.initOAuth()
controller := upgradeweb.NewController(dbcore.GetDBInstance(), summary)
controller.Activate()
defer controller.Deactivate()
r := gin.New()
r.Use(logutil.GinLogger())
r.Use(logutil.GinRecovery())
cors := security.NewCorsController(a.settings.CorsOriginCheckEnabled, a.settings.CorsAllowedOrigins)
r.Use(cors.Middleware())
r.Use(api.IdentityMiddleware())
r.Use(func(c *gin.Context) {
if strings.HasPrefix(c.Request.URL.Path, "/api") {
c.Header("Cache-Control", "no-store")
}
c.Next()
})
controller.Register(r)
frontendpublic.Static(r.Group("/"), func(handlers ...gin.HandlerFunc) {
r.NoRoute(func(c *gin.Context) {
requestPath := c.Request.URL.Path
if strings.HasPrefix(requestPath, "/api") {
api.RespondError(c, http.StatusNotFound, "Not found in upgrade mode")
return
}
if c.Request.Method == http.MethodGet && requestPath != upgradeweb.PagePath && filepath.Ext(requestPath) == "" {
c.Redirect(http.StatusTemporaryRedirect, upgradeweb.PagePath)
return
}
for _, handler := range handlers {
handler(c)
if c.IsAborted() {
return
}
}
})
})
a.engine = r
a.server = &http.Server{Addr: flags.Listen, Handler: r}
serverErr := make(chan error, 1)
log.Printf("Legacy monitoring data requires the 1.2.7 upgrade wizard on %s", flags.Listen)
go func() {
if err := a.server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
serverErr <- err
}
}()
quit := make(chan os.Signal, 1)
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
defer signal.Stop(quit)
select {
case err := <-serverErr:
return false, fmt.Errorf("listen in upgrade mode: %w", err)
case <-quit:
return false, a.Shutdown()
case <-controller.Done():
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := a.server.Shutdown(ctx); err != nil {
return false, fmt.Errorf("stop upgrade server: %w", err)
}
a.server = nil
a.engine = nil
return true, nil
}
}
// StartBackground 启动后台工作:定时任务。
func (a *App) StartBackground() error {
registerScheduledWork()
a.addCleanup("scheduler", func(context.Context) error {
corn.StopAll()
return nil
})
return nil
}
// registerReloadHandlers 把此前散落各处的 config.Subscribe 统一登记到 reload 管理器。
func (a *App) registerReloadHandlers(cors *security.CorsController) {
// OAuth provider 切换。
a.reload.Register("oauth-provider", func(event config.ConfigEvent) {
if ok, t := config.IsChangedT[string](event, config.OAuthProviderKey); ok {
if t == "" || t == "none" {
t = "github"
}
oidcProvider, err := database.GetOidcConfigByName(t)
if err != nil {
log.Printf("Failed to get OIDC provider config: %v", err)
return
}
log.Printf("Using %s as OIDC provider", oidcProvider.Name)
if err := oauth.LoadProvider(oidcProvider.Name, oidcProvider.Addition); err != nil {
auditlog.EventLog("error", fmt.Sprintf("Failed to load OIDC provider: %v", err))
}
}
})
// GeoIP provider 切换。
a.reload.Register("geoip-provider", func(event config.ConfigEvent) {
if event.IsChanged(config.GeoIpProviderKey) {
go geoip.InitGeoIp()
}
})
// 消息发送方式切换。
a.reload.Register("message-sender", func(event config.ConfigEvent) {
if event.IsChanged(config.NotificationMethodKey) {
go messageSender.Initialize()
}
})
// CORS 配置热更新。
a.reload.Register("cors", func(event config.ConfigEvent) {
cors.Update(event)
})
}
// BuildRouter 构建 Gin 引擎、中间件与全部路由,并登记热重载处理器。
func (a *App) BuildRouter() error {
r := gin.New()
r.Use(logutil.GinLogger())
r.Use(logutil.GinRecovery())
cors := security.NewCorsController(a.settings.CorsOriginCheckEnabled, a.settings.CorsAllowedOrigins)
r.Use(cors.Middleware())
r.Use(api.IdentityMiddleware())
r.Use(api.PrivateSiteMiddleware())
r.Use(func(c *gin.Context) {
if len(c.Request.URL.Path) >= 4 && c.Request.URL.Path[:4] == "/api" {
c.Header("Cache-Control", "no-store")
}
c.Next()
})
router.Register(r)
// 集中登记并启动热重载订阅。
a.registerReloadHandlers(cors)
a.reload.Start()
a.engine = r
return nil
}
// Run 启动 HTTP 服务并阻塞直到收到中断信号或服务异常退出。
func (a *App) Run() error {
a.server = &http.Server{
Addr: flags.Listen,
Handler: a.engine,
}
serverErr := make(chan error, 1)
log.Printf("Starting server on %s ...", flags.Listen)
go func() {
if err := a.server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
serverErr <- err
}
}()
quit := make(chan os.Signal, 1)
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
select {
case err := <-serverErr:
a.onFatal(err)
return fmt.Errorf("listen: %w", err)
case <-quit:
return a.Shutdown()
}
}
// Shutdown 优雅关闭:先停止接收新请求,再反序执行已登记的清理函数。
func (a *App) Shutdown() error {
if a.dbReady {
auditlog.Log("", "", "server is shutting down", "info")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
// 先关闭 HTTP 服务,停止接收新请求。
if a.server != nil {
if err := a.server.Shutdown(ctx); err != nil {
log.Printf("HTTP server forced to shutdown: %v", err)
}
}
// 反序释放资源(后进先出)。
for i := len(a.cleanups) - 1; i >= 0; i-- {
c := a.cleanups[i]
if err := c.fn(ctx); err != nil {
log.Printf("cleanup %q failed: %v", c.name, err)
}
}
return nil
}
// onFatal 处理 HTTP 服务致命错误:尽力释放已登记资源。
func (a *App) onFatal(err error) {
if a.dbReady {
auditlog.Log("", "", "server encountered a fatal error: "+err.Error(), "error")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
for i := len(a.cleanups) - 1; i >= 0; i-- {
c := a.cleanups[i]
if cerr := c.fn(ctx); cerr != nil {
log.Printf("cleanup %q failed: %v", c.name, cerr)
}
}
}
// ensureDefaultAdmin 首次启动(无任何用户)时创建默认管理员账号。
func ensureDefaultAdmin() error {
var count int64
dbcore.GetDBInstance().Model(&models.User{}).Count(&count)
if count != 0 {
return nil
}
user, passwd, err := accounts.CreateDefaultAdminAccount()
if err != nil {
return err
}
log.Println("Default admin account created. Username:", user, ", Password:", passwd)
return nil
}
// registerScheduledWork 注册所有定时任务与首启动同步逻辑。
func registerScheduledWork() {
if err := tasks.ReloadPingSchedule(); err != nil {
log.Println("Failed to reload ping schedule:", err)
}
if err := d_notification.ReloadLoadNotificationSchedule(); err != nil {
log.Println("Failed to reload load notification schedule:", err)
}
if err := corn.AddFunc("records:cleanup", "@every 30m", cleanupScheduledData); err != nil {
log.Println("Failed to add cleanup scheduled task:", err)
}
if err := corn.AddContextFunc("metrics:compact", "@every 5m", true, compactMetricStore); err != nil {
log.Println("Failed to add metric compact scheduled task:", err)
}
if err := corn.AddFunc("notifier:traffic", "@every 1m", notifier.CheckTraffic); err != nil {
log.Println("Failed to add traffic notification task:", err)
}
if err := corn.AddFunc("notifier:expire", "0 0 9 * * *", notifier.CheckExpireScheduledWork); err != nil {
log.Println("Failed to add expire notification scheduled task:", err)
}
notifier.InitTrafficReportSchedule()
}
const taskResultRetentionDays = 30
func cleanupScheduledData() {
before := time.Now().UTC().Add(-24 * time.Hour * taskResultRetentionDays)
if err := tasks.ClearTaskResultsByTimeBefore(before); err != nil {
log.Printf("Failed to clean expired task results: %v", err)
}
auditlog.RemoveOldLogs()
accounts.RemoveExpiredSessions()
}
func compactMetricStore(ctx context.Context) {
compactCtx, cancel := context.WithTimeout(ctx, 15*time.Minute)
defer cancel()
written, err := metricstore.Compact(compactCtx, time.Now().UTC())
if errors.Is(err, metricstore.ErrCompactInProgress) {
return
}
if err != nil {
log.Printf("Failed to compact metric store after writing %d rollup buckets: %v", written, err)
return
}
if written > 0 {
log.Printf("Metric store compacted %d rollup buckets", written)
}
}
+28 -6
View File
@@ -1,14 +1,36 @@
package flags
import "strings"
const (
DatabaseTypeSQLite = "sqlite"
)
var (
// 数据库配置
DatabaseType string // 数据库类型:sqlite, mysql
DatabaseType string // 数据库类型:sqlite
DatabaseFile string // SQLite数据库文件路径
DatabaseHost string // MySQL/其他数据库主机地址
DatabasePort string // MySQL/其他数据库端口
DatabaseUser string // MySQL/其他数据库用户名
DatabasePass string // MySQL/其他数据库密码
DatabaseName string // MySQL/其他数据库名称
Listen string
)
func NormalizeDatabaseType(databaseType string) string {
databaseType = strings.ToLower(strings.TrimSpace(databaseType))
if databaseType == "" {
return DatabaseTypeSQLite
}
return databaseType
}
func ApplyDatabaseTypeNormalization() string {
DatabaseType = NormalizeDatabaseType(DatabaseType)
return DatabaseType
}
func IsSQLite() bool {
return NormalizeDatabaseType(DatabaseType) == DatabaseTypeSQLite
}
func SupportedDatabaseTypes() string {
return DatabaseTypeSQLite
}
+20
View File
@@ -0,0 +1,20 @@
package flags
import "testing"
func TestNormalizeDatabaseType(t *testing.T) {
tests := map[string]string{
"": DatabaseTypeSQLite,
"sqlite": DatabaseTypeSQLite,
" SQLite ": DatabaseTypeSQLite,
"SQLITE": DatabaseTypeSQLite,
"postgres": "postgres",
" postgres": "postgres",
}
for input, want := range tests {
if got := NormalizeDatabaseType(input); got != want {
t.Fatalf("NormalizeDatabaseType(%q) = %q, want %q", input, got, want)
}
}
}
-404
View File
@@ -1,404 +0,0 @@
package cmd
import (
"context"
"errors"
"io"
"log"
"math"
"net"
"strings"
"sync"
"time"
apiClient "github.com/komari-monitor/komari/api/client"
"github.com/komari-monitor/komari/common"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/utils/geoip"
"github.com/komari-monitor/komari/utils/notifier"
"github.com/komari-monitor/komari/ws"
"github.com/komari-monitor/komari/compat/nezha/proto"
"google.golang.org/grpc"
"google.golang.org/grpc/keepalive"
"google.golang.org/grpc/metadata"
"gorm.io/gorm/clause"
)
// [Deprecated] Use StartNezhaCompat instead.
func StartNezhaCompatServer(addr string) error {
boot := uint64(time.Now().Unix())
lis, err := net.Listen("tcp", addr)
if err != nil {
return err
}
srv := &nezhaCompatServer{bootTime: boot}
// Interceptors for metadata auth presence (lightweight; validation is inside handlers too)
unary := func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
return handler(ctx, req)
}
stream := func(srvIface interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
return handler(srvIface, ss)
}
// Enable gRPC keepalive to tolerate long-idle streams
gs := grpc.NewServer(
grpc.UnaryInterceptor(unary),
grpc.StreamInterceptor(stream),
// Keepalive: allow client pings and keep connections stable without app traffic
grpc.KeepaliveParams(keepalive.ServerParameters{
Time: 2 * time.Minute,
Timeout: 20 * time.Second,
}),
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
MinTime: 20 * time.Second,
PermitWithoutStream: true,
}),
)
proto.RegisterNezhaServiceServer(gs, srv)
log.Printf("Nezha compat gRPC listening on %s", addr)
return gs.Serve(lis)
}
// ---- Manual start/stop support ----
var (
nezhaSrv *grpc.Server
nezhaLis net.Listener
nezhaOnceM sync.Mutex
)
// StartNezhaCompat starts the Nezha compatible gRPC server asynchronously.
// Returns error if already started.
func StartNezhaCompat(addr string) error {
nezhaOnceM.Lock()
defer nezhaOnceM.Unlock()
if nezhaSrv != nil {
return errors.New("nezha compat server already started")
}
lis, err := net.Listen("tcp", addr)
if err != nil {
return err
}
boot := uint64(time.Now().Unix())
sImpl := &nezhaCompatServer{bootTime: boot}
unary := func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
return handler(ctx, req)
}
stream := func(srvIface interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
return handler(srvIface, ss)
}
gs := grpc.NewServer(
grpc.UnaryInterceptor(unary),
grpc.StreamInterceptor(stream),
grpc.KeepaliveParams(keepalive.ServerParameters{Time: 2 * time.Minute, Timeout: 20 * time.Second}),
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{MinTime: 20 * time.Second, PermitWithoutStream: true}),
)
proto.RegisterNezhaServiceServer(gs, sImpl)
nezhaSrv = gs
nezhaLis = lis
go func() {
if err := gs.Serve(lis); err != nil {
log.Printf("Nezha compat gRPC server stopped: %v", err)
}
}()
log.Printf("Nezha compat gRPC started on %s", addr)
return nil
}
// StopNezhaCompat stops the server if running.
func StopNezhaCompat() error {
nezhaOnceM.Lock()
defer nezhaOnceM.Unlock()
if nezhaSrv == nil {
return errors.New("nezha compat server not running")
}
// 强制立即断开所有连接与流,不等待在途 RPC 完成。
nezhaSrv.Stop()
// Listener close (Serve already returns after GracefulStop, but close to be explicit)
if nezhaLis != nil {
_ = nezhaLis.Close()
}
nezhaSrv = nil
nezhaLis = nil
log.Printf("Nezha compat gRPC stopped")
return nil
}
type nezhaCompatServer struct {
proto.UnimplementedNezhaServiceServer
bootTime uint64
}
// authentication helpers
func getAuth(ctx context.Context) (uuid string, secret string, err error) {
md, ok := metadata.FromIncomingContext(ctx)
if !ok {
return "", "", errors.New("missing metadata")
}
getFirst := func(key string) string {
vals := md.Get(key)
if len(vals) > 0 {
return vals[0]
}
return ""
}
uuid = getFirst("client_uuid")
secret = getFirst("client_secret")
if uuid == "" || secret == "" {
return "", "", errors.New("unauthorized: missing client_uuid/client_secret")
}
return uuid, secret, nil
}
// ReportSystemInfo: upsert static host info
func (s *nezhaCompatServer) ReportSystemInfo(ctx context.Context, host *proto.Host) (*proto.Receipt, error) {
uuid, secret, err := getAuth(ctx)
if err != nil {
return nil, err
}
if err := upsertClientFromHost(uuid, secret, host); err != nil {
return nil, err
}
return &proto.Receipt{Proced: true}, nil
}
// ReportSystemInfo2: same as above but returns dashboard boot time
func (s *nezhaCompatServer) ReportSystemInfo2(ctx context.Context, host *proto.Host) (*proto.Uint64Receipt, error) {
uuid, secret, err := getAuth(ctx)
if err != nil {
return nil, err
}
if err := upsertClientFromHost(uuid, secret, host); err != nil {
return nil, err
}
return &proto.Uint64Receipt{Data: s.bootTime}, nil
}
// ReportSystemState: ingest time-series records
func (s *nezhaCompatServer) ReportSystemState(stream proto.NezhaService_ReportSystemStateServer) error {
ctx := stream.Context()
uuid, _, err := getAuth(ctx)
if err != nil {
return err
}
// presence start
connID := time.Now().UnixNano()
ws.SetPresence(uuid, connID, true)
go notifier.OnlineNotification(uuid, connID)
defer func() {
ws.SetPresence(uuid, connID, false)
notifier.OfflineNotification(uuid, connID)
}()
for {
st, err := stream.Recv()
if err == io.EOF {
return nil
}
if err != nil {
return err
}
// refresh presence TTL on every frame
ws.KeepAlivePresence(uuid, connID, 30*time.Second)
if err := ingestState(uuid, st); err != nil {
// still ack to avoid client stuck; log error
log.Printf("Nezha ingest state error: %v", err)
}
if err := stream.Send(&proto.Receipt{Proced: true}); err != nil {
return err
}
}
}
// RequestTask: do not dispatch tasks, just drain results to avoid Unimplemented
func (s *nezhaCompatServer) RequestTask(stream proto.NezhaService_RequestTaskServer) error {
ctx := stream.Context()
uuid, _, err := getAuth(ctx)
if err != nil {
return err
}
ticker := time.NewTicker(10 * time.Second)
defer ticker.Stop()
connID := time.Now().UnixNano()
ws.SetPresence(uuid, connID, true)
defer ws.SetPresence(uuid, connID, false)
// receive results in background
recvErr := make(chan error, 1)
go func() {
for {
_, rerr := stream.Recv()
if rerr == io.EOF {
recvErr <- nil
return
}
if rerr != nil {
recvErr <- rerr
return
}
// refresh presence TTL when result received
ws.KeepAlivePresence(uuid, connID, 30*time.Second)
}
}()
// send heartbeat tasks periodically
for {
select {
case <-ctx.Done():
return ctx.Err()
case err := <-recvErr:
return err
case <-ticker.C:
if err := stream.Send(&proto.Task{}); err != nil {
return err
}
}
}
}
// Unimplemented methods intentionally left as default (IOStream, ReportGeoIP)
// upsertClientFromHost maps Host into models.Client and upserts by UUID
func upsertClientFromHost(uuid, secret string, h *proto.Host) error {
db := dbcore.GetDBInstance()
now := models.FromTime(time.Now())
// token guard: if existing record has different token, reject
var exist models.Client
if err := db.Where("uuid = ?", uuid).First(&exist).Error; err == nil {
if exist.Token != "" && exist.Token != secret {
return errors.New("unauthorized: token mismatch")
}
}
cpuName := ""
if len(h.Cpu) > 0 {
cpuName = h.Cpu[0]
}
gpuName := strings.Join(h.Gpu, "; ")
osName := h.Platform
if h.PlatformVersion != "" {
osName = h.Platform + " " + h.PlatformVersion
}
// clamp uint64 to int64 safely
clamp := func(v uint64) int64 {
if v > uint64(math.MaxInt64) {
return math.MaxInt64
}
return int64(v)
}
c := models.Client{
UUID: uuid,
Token: secret,
Name: "nezha_" + uuid[0:8],
CpuName: cpuName,
Arch: h.Arch,
CpuCores: len(h.Cpu),
OS: osName,
KernelVersion: h.PlatformVersion,
Virtualization: h.Virtualization,
GpuName: gpuName,
MemTotal: clamp(h.MemTotal),
SwapTotal: clamp(h.SwapTotal),
DiskTotal: clamp(h.DiskTotal),
Version: h.Version,
UpdatedAt: now,
}
// Upsert by UUID; don't override existing Token if already set
updates := map[string]interface{}{
"cpu_name": c.CpuName,
"arch": c.Arch,
"cpu_cores": c.CpuCores,
"os": c.OS,
"kernel_version": c.KernelVersion,
"virtualization": c.Virtualization,
"gpu_name": c.GpuName,
"mem_total": c.MemTotal,
"swap_total": c.SwapTotal,
"disk_total": c.DiskTotal,
"version": c.Version,
"updated_at": time.Now(),
}
return db.Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "uuid"}},
DoUpdates: clause.Assignments(updates),
}).Create(&c).Error
}
// ingestState maps Nezha State into common.Report then saves a Record
func ingestState(uuid string, st *proto.State) error {
// we may need totals from client
db := dbcore.GetDBInstance()
var client models.Client
if err := db.Where("uuid = ?", uuid).First(&client).Error; err != nil {
// If missing, create with minimal defaults to avoid failing ingestion
client = models.Client{UUID: uuid, Token: "", Name: "nezha_" + uuid[0:8]}
auditlog.EventLog("info", "auto created client "+client.Name)
_ = db.Create(&client).Error
}
rep := common.Report{
CPU: common.CPUReport{Usage: st.Cpu},
Ram: common.RamReport{Total: client.MemTotal, Used: int64(st.MemUsed)},
Swap: common.RamReport{Total: client.SwapTotal, Used: int64(st.SwapUsed)},
Load: common.LoadReport{Load1: st.Load1, Load5: st.Load5, Load15: st.Load15},
Disk: common.DiskReport{Total: client.DiskTotal, Used: int64(st.DiskUsed)},
Network: common.NetworkReport{
Up: int64(st.NetOutSpeed),
Down: int64(st.NetInSpeed),
TotalUp: int64(st.NetOutTransfer),
TotalDown: int64(st.NetInTransfer),
},
Uptime: int64(st.Uptime),
Process: int(st.ProcessCount),
UpdatedAt: time.Now(),
}
// 更新实时缓存供前端使用
ws.SetLatestReport(uuid, &rep)
// 写入内存缓存,入库交由定时聚合任务处理
return apiClient.SaveClientReport(uuid, rep)
}
// ReportGeoIP: 保存 Agent 上报的 IP,并回填国家码/面板启动时间
func (s *nezhaCompatServer) ReportGeoIP(ctx context.Context, in *proto.GeoIP) (*proto.GeoIP, error) {
uuid, _, err := getAuth(ctx)
if err != nil {
return nil, err
}
updates := map[string]interface{}{
"updated_at": time.Now(),
}
var iso string
if in != nil && in.Ip != nil {
if v4 := strings.TrimSpace(in.Ip.Ipv4); v4 != "" {
updates["ipv4"] = v4
if GeoIpEnabled, _ := config.GetAs[bool](config.GeoIpEnabledKey, false); GeoIpEnabled {
if ip := net.ParseIP(v4); ip != nil {
if gi, _ := geoip.GetGeoInfo(ip); gi != nil {
iso = gi.ISOCode
}
}
}
}
if v6 := strings.TrimSpace(in.Ip.Ipv6); v6 != "" {
updates["ipv6"] = v6
if iso == "" { // 优先使用 v4 的国家码
if GeoIpEnabled, _ := config.GetAs[bool](config.GeoIpEnabledKey, false); GeoIpEnabled {
if ip := net.ParseIP(v6); ip != nil {
if gi, _ := geoip.GetGeoInfo(ip); gi != nil {
iso = gi.ISOCode
}
}
}
}
}
}
if iso != "" {
// UI 使用旗帜 emoji
updates["region"] = geoip.GetRegionUnicodeEmoji(iso)
}
if len(updates) > 0 {
_ = dbcore.GetDBInstance().Model(&models.Client{}).Where("uuid = ?", uuid).Updates(updates).Error
}
// 回写 GeoIP(包含国家码与面板启动时间)
resp := &proto.GeoIP{Use6: in.GetUse6(), Ip: in.GetIp(), CountryCode: iso, DashboardBootTime: s.bootTime}
return resp, nil
}
+3 -8
View File
@@ -4,9 +4,8 @@ import (
"os"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/config"
"github.com/spf13/cobra"
"gorm.io/gorm"
)
var PermitPasswordLoginCmd = &cobra.Command{
@@ -14,12 +13,8 @@ var PermitPasswordLoginCmd = &cobra.Command{
Short: "Force permit password login",
Long: `Force permit password login`,
Run: func(cmd *cobra.Command, args []string) {
db := dbcore.GetDBInstance()
err := db.Transaction(func(tx *gorm.DB) error {
return tx.Model(&models.Config{}).Where("id = ?", 1).
Update("disable_password_login", false).Error
})
if err != nil {
dbcore.GetDBInstance()
if err := config.Set(config.DisablePasswordLoginKey, false); err != nil {
cmd.Println("Error:", err)
os.Exit(1)
}
+67
View File
@@ -0,0 +1,67 @@
package cmd
import (
"log"
"github.com/komari-monitor/komari/pkg/config"
)
// reloadHandler 是单个配置热重载处理器。
//
// name 仅用于日志定位,handler 接收配置变更事件,自行判断关心的 key 是否变化。
// 每个 handler 相互独立:一个 panic 不应影响其它 handler。
type reloadHandler struct {
name string
handler func(config.ConfigEvent)
}
// ReloadManager 统一收敛所有配置热重载逻辑。
//
// 过去 config.Subscribe 分散在 cmd/server.go、cors.go 等多处,导致:
// - 谁监听哪些 key 不清晰;
// - 某个 handler panic 会连带影响同一订阅回调里的其它逻辑;
// - 关闭时无法统一停止。
//
// ReloadManager 把这些 handler 注册在一处,用单个 config.Subscribe 分发,
// 并对每个 handler 做 panic 隔离。
type ReloadManager struct {
handlers []reloadHandler
started bool
}
// NewReloadManager 创建一个空的热重载管理器。
func NewReloadManager() *ReloadManager {
return &ReloadManager{}
}
// Register 注册一个命名的热重载处理器。必须在 Start 之前调用。
func (m *ReloadManager) Register(name string, handler func(config.ConfigEvent)) {
if handler == nil {
return
}
m.handlers = append(m.handlers, reloadHandler{name: name, handler: handler})
}
// Start 向 config 订阅一次,之后每个事件会分发给全部已注册 handler。
// 重复调用无副作用。
func (m *ReloadManager) Start() {
if m.started {
return
}
m.started = true
config.Subscribe(func(event config.ConfigEvent) {
for _, h := range m.handlers {
m.dispatch(h, event)
}
})
}
// dispatch 执行单个 handler,并隔离 panic,避免一个 handler 影响其它。
func (m *ReloadManager) dispatch(h reloadHandler, event config.ConfigEvent) {
defer func() {
if r := recover(); r != nil {
log.Printf("config reload handler %q panicked: %v", h.name, r)
}
}()
h.handler(event)
}
+2 -19
View File
@@ -17,17 +17,6 @@ func GetEnv(key, defaultValue string) string {
return value
}
// 从环境变量获取默认值
var (
dbTypeEnv = GetEnv("KOMARI_DB_TYPE", "sqlite")
dbFileEnv = GetEnv("KOMARI_DB_FILE", "./data/komari.db")
dbHostEnv = GetEnv("KOMARI_DB_HOST", "localhost")
dbPortEnv = GetEnv("KOMARI_DB_PORT", "3306")
dbUserEnv = GetEnv("KOMARI_DB_USER", "root")
dbPassEnv = GetEnv("KOMARI_DB_PASS", "")
dbNameEnv = GetEnv("KOMARI_DB_NAME", "komari")
)
var RootCmd = &cobra.Command{
Use: "Komari",
Short: "Komari is a simple server monitoring tool",
@@ -47,12 +36,6 @@ func Execute() {
}
func init() {
// 设置命令行参数,提供环境变量作为默认值
RootCmd.PersistentFlags().StringVarP(&flags.DatabaseType, "db-type", "t", dbTypeEnv, "Database type (sqlite, mysql) [env: KOMARI_DB_TYPE]")
RootCmd.PersistentFlags().StringVarP(&flags.DatabaseFile, "database", "d", dbFileEnv, "SQLite database file path [env: KOMARI_DB_FILE]")
RootCmd.PersistentFlags().StringVar(&flags.DatabaseHost, "db-host", dbHostEnv, "MySQL/Other database host address [env: KOMARI_DB_HOST]")
RootCmd.PersistentFlags().StringVar(&flags.DatabasePort, "db-port", dbPortEnv, "MySQL/Other database port [env: KOMARI_DB_PORT]")
RootCmd.PersistentFlags().StringVar(&flags.DatabaseUser, "db-user", dbUserEnv, "MySQL/Other database username [env: KOMARI_DB_USER]")
RootCmd.PersistentFlags().StringVar(&flags.DatabasePass, "db-pass", dbPassEnv, "MySQL/Other database password [env: KOMARI_DB_PASS]")
RootCmd.PersistentFlags().StringVar(&flags.DatabaseName, "db-name", dbNameEnv, "MySQL/Other database name [env: KOMARI_DB_NAME]")
RootCmd.PersistentFlags().StringVarP(&flags.DatabaseType, "db-type", "t", "sqlite", "Database type (sqlite)")
RootCmd.PersistentFlags().StringVarP(&flags.DatabaseFile, "database", "d", "./data/komari.db", "SQLite database file path")
}
+39 -402
View File
@@ -1,53 +1,12 @@
package cmd
import (
"context"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/api"
"github.com/komari-monitor/komari/api/admin"
"github.com/komari-monitor/komari/api/admin/clipboard"
log_api "github.com/komari-monitor/komari/api/admin/log"
"github.com/komari-monitor/komari/api/admin/notification"
"github.com/komari-monitor/komari/api/admin/test"
"github.com/komari-monitor/komari/api/admin/update"
"github.com/komari-monitor/komari/api/client"
"github.com/komari-monitor/komari/api/jsonRpc"
public_api "github.com/komari-monitor/komari/api/public"
"github.com/komari-monitor/komari/api/terminal"
"github.com/komari-monitor/komari/cmd/flags"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database"
"github.com/komari-monitor/komari/database/accounts"
"github.com/komari-monitor/komari/database/auditlog"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
d_notification "github.com/komari-monitor/komari/database/notification"
"github.com/komari-monitor/komari/database/records"
"github.com/komari-monitor/komari/database/tasks"
"github.com/komari-monitor/komari/public"
"github.com/komari-monitor/komari/utils"
"github.com/komari-monitor/komari/utils/cloudflared"
"github.com/komari-monitor/komari/utils/geoip"
logutil "github.com/komari-monitor/komari/utils/log"
"github.com/komari-monitor/komari/utils/messageSender"
"github.com/komari-monitor/komari/utils/notifier"
"github.com/komari-monitor/komari/utils/oauth"
"github.com/spf13/cobra"
)
var (
DynamicCorsEnabled bool = false
)
var ServerCmd = &cobra.Command{
Use: "server",
Short: "Start the server",
@@ -64,375 +23,53 @@ func init() {
RootCmd.AddCommand(ServerCmd)
}
// RunServer 按显式的生命周期阶段启动服务端。
//
// 具体各阶段的职责与顺序见 App(cmd/app.go)。这里只负责串联:
// 任一初始化阶段失败即中止启动,避免在半初始化状态下对外提供服务。
func RunServer() {
// #region 初始化
if err := os.MkdirAll("./data/theme", os.ModePerm); err != nil {
log.Fatalf("Failed to create theme directory: %v", err)
app := NewApp()
if err := app.Bootstrap(); err != nil {
_ = app.Shutdown()
log.Fatalf("server startup failed at %q: %v", "bootstrap", err)
}
InitDatabase()
if utils.VersionHash != "unknown" {
gin.SetMode(gin.ReleaseMode)
}
conf, err := config.GetManyAs[config.Legacy]()
required, summary, err := app.LegacyUpgradeRequired()
if err != nil {
log.Fatal(err)
_ = app.Shutdown()
log.Fatalf("server startup failed at %q: %v", "detect-1.2.7-upgrade", err)
}
go geoip.InitGeoIp()
go DoScheduledWork()
go messageSender.Initialize()
// oidcInit
go oauth.Initialize()
if conf.NezhaCompatEnabled {
go func() {
if err := StartNezhaCompat(conf.NezhaCompatListen); err != nil {
log.Printf("Nezha compat server error: %v", err)
auditlog.EventLog("error", fmt.Sprintf("Nezha compat server error: %v", err))
}
}()
}
config.Subscribe(func(event config.ConfigEvent) {
if ok, t := config.IsChangedT[string](event, config.OAuthProviderKey); ok {
if t == "" || t == "none" {
t = "github"
}
oidcProvider, err := database.GetOidcConfigByName(t)
if err != nil {
log.Printf("Failed to get OIDC provider config: %v", err)
} else {
log.Printf("Using %s as OIDC provider", oidcProvider.Name)
}
err = oauth.LoadProvider(oidcProvider.Name, oidcProvider.Addition)
if err != nil {
auditlog.EventLog("error", fmt.Sprintf("Failed to load OIDC provider: %v", err))
}
}
if ok, t := config.IsChangedT[bool](event, config.NezhaCompatEnabledKey); ok {
if t {
l, _ := config.GetAs[string](config.NezhaCompatListenKey)
if err := StartNezhaCompat(l); err != nil {
log.Printf("start Nezha compat server error: %v", err)
auditlog.EventLog("error", fmt.Sprintf("start Nezha compat server error: %v", err))
}
} else {
if err := StopNezhaCompat(); err != nil {
log.Printf("stop Nezha compat server error: %v", err)
auditlog.EventLog("error", fmt.Sprintf("stop Nezha compat server error: %v", err))
}
}
}
})
// 初始化 cloudflared
if err := cloudflared.AutoStart(GetEnv("KOMARI_CLOUDFLARED_TOKEN", "")); err != nil {
log.Printf("failed to auto start cloudflared: %v", err)
}
r := gin.New()
r.Use(logutil.GinLogger())
r.Use(logutil.GinRecovery())
// 动态 CORS 中间件
DynamicCorsEnabled = conf.AllowCors
config.Subscribe(func(event config.ConfigEvent) {
if ok, t := config.IsChangedT[bool](event, config.AllowCorsKey); ok {
DynamicCorsEnabled = t
}
if event.IsChanged(config.GeoIpProviderKey) {
go geoip.InitGeoIp()
}
if event.IsChanged(config.NotificationMethodKey) {
go messageSender.Initialize()
}
})
r.Use(func(c *gin.Context) {
if DynamicCorsEnabled {
c.Header("Access-Control-Allow-Origin", "*")
c.Header("Access-Control-Allow-Methods", "GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS")
c.Header("Access-Control-Allow-Headers", "Origin, Content-Length, Content-Type, Authorization, Accept, X-CSRF-Token, X-Requested-With, Set-Cookie")
c.Header("Access-Control-Expose-Headers", "Content-Length, Authorization, Set-Cookie")
c.Header("Access-Control-Allow-Credentials", "true")
c.Header("Access-Control-Max-Age", "43200") // 12 hours
if c.Request.Method == "OPTIONS" {
c.AbortWithStatus(204)
return
}
}
c.Next()
})
r.Use(api.IdentityMiddleware())
r.Use(api.PrivateSiteMiddleware())
r.Use(func(c *gin.Context) {
if len(c.Request.URL.Path) >= 4 && c.Request.URL.Path[:4] == "/api" {
c.Header("Cache-Control", "no-store")
}
c.Next()
})
r.Any("/ping", func(c *gin.Context) {
c.String(200, "pong")
})
// #region 公开路由
r.POST("/api/login", public_api.Login)
r.GET("/api/me", public_api.GetMe)
r.GET("/api/clients", api.GetClients)
r.GET("/api/nodes", public_api.GetNodesInformation)
r.GET("/api/public", public_api.GetPublicSettings)
r.GET("/api/oauth", public_api.OAuth)
r.GET("/api/oauth_callback", public_api.OAuthCallback)
r.GET("/api/logout", public_api.Logout)
r.GET("/api/version", public_api.GetVersion)
r.GET("/api/recent/:uuid", public_api.GetClientRecentRecords)
r.GET("/api/records/load", public_api.GetRecordsByUUID)
r.GET("/api/records/ping", public_api.GetPingRecords)
r.GET("/api/task/ping", public_api.GetPublicPingTasks)
r.GET("/api/rpc2", jsonRpc.OnRpcRequest)
r.POST("/api/rpc2", jsonRpc.OnRpcRequest)
r.GET("/api/mjpeg_live", public_api.MjpegLiveHandler)
// #region Agent
r.POST("/api/clients/register", client.RegisterClient)
tokenAuthrized := r.Group("/api/clients", api.RequireRole(api.RoleAdmin, api.RoleClient))
{
tokenAuthrized.GET("/report", client.WebSocketReport) // websocket
tokenAuthrized.POST("/uploadBasicInfo", client.UploadBasicInfo)
tokenAuthrized.POST("/report", client.UploadReport)
tokenAuthrized.GET("/terminal", terminal.EstablishConnection)
tokenAuthrized.POST("/task/result", client.TaskResult)
tokenAuthrized.GET("/ping/tasks", client.GetPingTasks)
tokenAuthrized.POST("/ping/result", client.UploadPingResult)
}
// #region 管理员
adminAuthrized := r.Group("/api/admin", api.RequireRole(api.RoleAdmin))
{
adminAuthrized.GET("/download/backup", admin.DownloadBackup)
adminAuthrized.POST("/upload/backup", admin.UploadBackup)
// test
testGroup := adminAuthrized.Group("/test")
{
testGroup.GET("/geoip", test.TestGeoIp)
testGroup.POST("/sendMessage", test.TestSendMessage)
}
// update
updateGroup := adminAuthrized.Group("/update")
{
updateGroup.POST("/mmdb", update.UpdateMmdbGeoIP)
updateGroup.POST("/user", update.UpdateUser)
updateGroup.PUT("/favicon", update.UploadFavicon)
updateGroup.POST("/favicon", update.DeleteFavicon)
}
// tasks
taskGroup := adminAuthrized.Group("/task")
{
taskGroup.GET("/all", admin.GetTasks)
taskGroup.POST("/exec", admin.Exec)
taskGroup.GET("/:task_id", admin.GetTaskById)
taskGroup.GET("/:task_id/result", admin.GetTaskResultsByTaskId)
taskGroup.GET("/:task_id/result/:uuid", admin.GetSpecificTaskResult)
taskGroup.GET("/client/:uuid", admin.GetTasksByClientId)
}
// settings
settingsGroup := adminAuthrized.Group("/settings")
{
settingsGroup.GET("/", admin.GetSettings)
settingsGroup.POST("/", admin.EditSettings)
settingsGroup.POST("/oidc", admin.SetOidcProvider)
settingsGroup.GET("/oidc", admin.GetOidcProvider)
settingsGroup.POST("/message-sender", admin.SetMessageSenderProvider)
settingsGroup.GET("/message-sender", admin.GetMessageSenderProvider)
settingsGroup.GET("/cloudflared", admin.GetCloudflaredStatus)
settingsGroup.POST("/cloudflared/start", admin.StartCloudflared)
settingsGroup.POST("/cloudflared/stop", admin.StopCloudflared)
settingsGroup.POST("/cloudflared/remove-token", admin.RemoveCloudflaredToken)
}
// themes
themeGroup := adminAuthrized.Group("/theme")
{
themeGroup.PUT("/upload", admin.UploadTheme)
themeGroup.GET("/list", admin.ListThemes)
themeGroup.POST("/delete", admin.DeleteTheme)
themeGroup.GET("/set", admin.SetTheme)
themeGroup.POST("/update", admin.UpdateTheme)
themeGroup.POST("/import", admin.ImportTheme)
themeGroup.POST("/settings", admin.UpdateThemeSettings)
}
// clients
clientGroup := adminAuthrized.Group("/client")
{
clientGroup.POST("/add", admin.AddClient)
clientGroup.GET("/list", admin.ListClients)
clientGroup.GET("/:uuid", admin.GetClient)
clientGroup.POST("/:uuid/edit", admin.EditClient)
clientGroup.POST("/:uuid/remove", admin.RemoveClient)
clientGroup.GET("/:uuid/token", admin.GetClientToken)
clientGroup.POST("/order", admin.OrderWeight)
// client terminal
clientGroup.GET("/:uuid/terminal", terminal.RequestTerminal)
}
// records
recordGroup := adminAuthrized.Group("/record")
{
recordGroup.POST("/clear", admin.ClearRecord)
recordGroup.POST("/clear/all", admin.ClearAllRecords)
}
// oauth2
oauth2Group := adminAuthrized.Group("/oauth2")
{
oauth2Group.GET("/bind", admin.BindingExternalAccount)
oauth2Group.POST("/unbind", admin.UnbindExternalAccount)
}
sessionGroup := adminAuthrized.Group("/session")
{
sessionGroup.GET("/get", admin.GetSessions)
sessionGroup.POST("/remove", admin.DeleteSession)
sessionGroup.POST("/remove/all", admin.DeleteAllSession)
}
two_factorGroup := adminAuthrized.Group("/2fa")
{
two_factorGroup.GET("/generate", admin.Generate2FA)
two_factorGroup.POST("/enable", admin.Enable2FA)
two_factorGroup.POST("/disable", admin.Disable2FA)
}
adminAuthrized.GET("/logs", log_api.GetLogs)
// clipboard
clipboardGroup := adminAuthrized.Group("/clipboard")
{
clipboardGroup.GET("/:id", clipboard.GetClipboard)
clipboardGroup.GET("", clipboard.ListClipboard)
clipboardGroup.POST("", clipboard.CreateClipboard)
clipboardGroup.POST("/:id", clipboard.UpdateClipboard)
clipboardGroup.POST("/remove", clipboard.BatchDeleteClipboard)
clipboardGroup.POST("/:id/remove", clipboard.DeleteClipboard)
}
notificationGroup := adminAuthrized.Group("/notification")
{
// offline notifications
notificationGroup.GET("/offline", notification.ListOfflineNotifications)
notificationGroup.POST("/offline/edit", notification.EditOfflineNotification)
notificationGroup.POST("/offline/enable", notification.EnableOfflineNotification)
notificationGroup.POST("/offline/disable", notification.DisableOfflineNotification)
loadAlertGroup := notificationGroup.Group("/load")
{
loadAlertGroup.GET("/", notification.GetAllLoadNotifications)
loadAlertGroup.POST("/add", notification.AddLoadNotification)
loadAlertGroup.POST("/delete", notification.DeleteLoadNotification)
loadAlertGroup.POST("/edit", notification.EditLoadNotification)
}
}
pingTaskGroup := adminAuthrized.Group("/ping")
{
pingTaskGroup.GET("/", admin.GetAllPingTasks)
pingTaskGroup.POST("/add", admin.AddPingTask)
pingTaskGroup.POST("/delete", admin.DeletePingTask)
pingTaskGroup.POST("/edit", admin.EditPingTask)
pingTaskGroup.POST("/order", admin.OrderPingTask)
}
}
public.Static(r.Group("/"), func(handlers ...gin.HandlerFunc) {
r.NoRoute(handlers...)
})
srv := &http.Server{
Addr: flags.Listen,
Handler: r,
}
log.Printf("Starting server on %s ...", flags.Listen)
go func() {
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
OnFatal(err)
log.Fatalf("listen: %s\n", err)
}
}()
quit := make(chan os.Signal, 1)
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
<-quit
OnShutdown()
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := srv.Shutdown(ctx); err != nil {
log.Fatalf("Server forced to shutdown: %v", err)
}
}
func InitDatabase() {
// // 打印数据库类型和连接信息
// if flags.DatabaseType == "mysql" {
// log.Printf("使用 MySQL 数据库连接: %s@%s:%s/%s",
// flags.DatabaseUser, flags.DatabaseHost, flags.DatabasePort, flags.DatabaseName)
// log.Printf("环境变量配置: [KOMARI_DB_TYPE=%s] [KOMARI_DB_HOST=%s] [KOMARI_DB_PORT=%s] [KOMARI_DB_USER=%s] [KOMARI_DB_NAME=%s]",
// os.Getenv("KOMARI_DB_TYPE"), os.Getenv("KOMARI_DB_HOST"), os.Getenv("KOMARI_DB_PORT"),
// os.Getenv("KOMARI_DB_USER"), os.Getenv("KOMARI_DB_NAME"))
// } else {
// log.Printf("使用 SQLite 数据库文件: %s", flags.DatabaseFile)
// log.Printf("环境变量配置: [KOMARI_DB_TYPE=%s] [KOMARI_DB_FILE=%s]",
// os.Getenv("KOMARI_DB_TYPE"), os.Getenv("KOMARI_DB_FILE"))
// }
var count int64 = 0
if dbcore.GetDBInstance().Model(&models.User{}).Count(&count); count == 0 {
user, passwd, err := accounts.CreateDefaultAdminAccount()
if required {
completed, err := app.RunLegacyUpgrade(summary)
if err != nil {
panic(err)
_ = app.Shutdown()
log.Fatalf("server startup failed at %q: %v", "run-1.2.7-upgrade", err)
}
log.Println("Default admin account created. Username:", user, ", Password:", passwd)
}
}
// #region 定时任务
func DoScheduledWork() {
if err := tasks.MigrateAllClientsExpansion(); err != nil {
log.Println("Failed to migrate ping task all_clients expansion:", err)
}
tasks.ReloadPingSchedule()
d_notification.ReloadLoadNotificationSchedule()
ticker := time.NewTicker(time.Minute * 30)
minute := time.NewTicker(60 * time.Second)
//records.DeleteRecordBefore(time.Now().Add(-time.Hour * 24 * 30))
records.CompactRecord()
go notifier.CheckExpireScheduledWork()
for {
cfg, _ := config.GetManyAs[config.Legacy]()
select {
case <-ticker.C:
records.DeleteRecordBefore(time.Now().Add(-time.Hour * time.Duration(cfg.RecordPreserveTime)))
records.CompactRecord()
tasks.ClearTaskResultsByTimeBefore(time.Now().Add(-time.Hour * time.Duration(cfg.RecordPreserveTime)))
tasks.DeletePingRecordsBefore(time.Now().Add(-time.Hour * time.Duration(cfg.PingRecordPreserveTime)))
auditlog.RemoveOldLogs()
accounts.RemoveExpiredSessions()
case <-minute.C:
api.SaveClientReportToDB()
if !cfg.RecordEnabled {
records.DeleteAll()
tasks.DeleteAllPingRecords()
}
// 每分钟检查一次流量提醒
go notifier.CheckTraffic()
if !completed {
return
}
}
}
// 初始化阶段:任一步失败都不应继续对外服务。
type stage struct {
name string
fn func() error
}
stages := []stage{
{"init-stores", app.InitStores},
{"init-providers", app.InitProviders},
{"start-background", app.StartBackground},
{"build-router", app.BuildRouter},
}
for _, s := range stages {
if err := s.fn(); err != nil {
// 已登记的资源尽力回收,再退出。
_ = app.Shutdown()
log.Fatalf("server startup failed at %q: %v", s.name, err)
}
}
func OnShutdown() {
auditlog.Log("", "", "server is shutting down", "info")
cloudflared.Shutdown()
}
func OnFatal(err error) {
auditlog.Log("", "", "server encountered a fatal error: "+err.Error(), "error")
cloudflared.Shutdown()
if err := app.Run(); err != nil {
log.Fatalf("server exited with error: %v", err)
}
}
-135
View File
@@ -1,135 +0,0 @@
package common
import "time"
type Message struct {
Type string `json:"type"`
Content string `json:"content"`
Sender string `json:"sender"`
Timestamp int64 `json:"timestamp"`
}
type ClientConfig struct {
ClientUUID string `json:"client_uuid" gorm:"type:uuid;primaryKey;foreignKey:ClientUUID;references:UUID;constraint:OnDelete:CASCADE"`
CPU bool `json:"cpu" gorm:"default:true"`
GPU bool `json:"gpu" gorm:"default:true"`
RAM bool `json:"ram" gorm:"default:true"`
SWAP bool `json:"swap" gorm:"default:true"`
LOAD bool `json:"load" gorm:"default:true"`
UPTIME bool `json:"uptime" gorm:"default:true"`
TEMP bool `json:"temp" gorm:"default:true"`
OS bool `json:"os" gorm:"default:true"`
DISK bool `json:"disk" gorm:"default:true"`
NET bool `json:"net" gorm:"default:true"`
PROCESS bool `json:"process" gorm:"default:true"`
Connections bool `json:"connections" gorm:"default:true"`
Interval int `json:"interval" gorm:"default:3;check:Interval >= 1"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
// ClientInfo stores static client information
// Deprecated: Use models.Client instead.
type ClientInfo struct {
UUID string `json:"uuid,omitempty" gorm:"type:varchar(36);primaryKey;foreignKey:ClientUUID;references:UUID;constraint:OnDelete:CASCADE"`
Name string `json:"name" gorm:"type:varchar(100);not null"`
CpuName string `json:"cpu_name" gorm:"type:varchar(100)"`
Virtualization string `json:"virtualization" gorm:"type:varchar(50)"`
Arch string `json:"arch" gorm:"type:varchar(50)"`
CpuCores int `json:"cpu_cores" gorm:"type:int"`
OS string `json:"os" gorm:"type:varchar(100)"`
GpuName string `json:"gpu_name" gorm:"type:varchar(100)"`
IPv4 string `json:"ipv4,omitempty" gorm:"type:varchar(100)"`
IPv6 string `json:"ipv6,omitempty" gorm:"type:varchar(100)"`
Region string `json:"region" gorm:"type:varchar(100)"`
Remark string `json:"remark,omitempty" gorm:"type:longtext"`
PublicRemark string `json:"public_remark,omitempty" gorm:"type:longtext"`
MemTotal int64 `json:"mem_total" gorm:"type:bigint"`
SwapTotal int64 `json:"swap_total" gorm:"type:bigint"`
DiskTotal int64 `json:"disk_total" gorm:"type:bigint"`
Version string `json:"version,omitempty" gorm:"type:varchar(100)"`
Weight int `json:"weight" gorm:"type:int"`
Price float64 `json:"price"`
BillingCycle int `json:"billing_cycle"`
ExpiredAt time.Time `json:"expired_at" gorm:"type:timestamp"`
CreatedAt time.Time
UpdatedAt time.Time
}
type IPAddress struct {
Ipv4 string `json:"ipv4"`
Ipv6 string `json:"ipv6"`
}
type Report struct {
UUID string `json:"uuid,omitempty"`
CPU CPUReport `json:"cpu"`
Ram RamReport `json:"ram"`
Swap RamReport `json:"swap"`
Load LoadReport `json:"load"`
Disk DiskReport `json:"disk"`
Network NetworkReport `json:"network"`
Connections ConnectionsReport `json:"connections"`
GPU *GPUDetailReport `json:"gpu,omitempty"` // 新增GPU详细信息
Uptime int64 `json:"uptime"`
Process int `json:"process"`
Message string `json:"message"`
Method string `json:"method,omitempty"`
UpdatedAt time.Time `json:"updated_at"`
}
type CPUReport struct {
Name string `json:"name,omitempty"`
Cores int `json:"cores,omitempty"`
Arch string `json:"arch,omitempty"`
Usage float64 `json:"usage,omitempty"`
}
// GPUDetailReport 详细GPU信息报告
type GPUDetailReport struct {
Count int `json:"count"` // GPU数量
AverageUsage float64 `json:"average_usage"` // 平均使用率
DetailedInfo []GPUDeviceInfo `json:"detailed_info"` // 每个GPU的详细信息
}
// GPUDeviceInfo 单个GPU设备信息
type GPUDeviceInfo struct {
Name string `json:"name"` // GPU型号名称
MemoryTotal int64 `json:"memory_total"` // 显存总量(字节)
MemoryUsed int64 `json:"memory_used"` // 显存已用(字节)
Utilization float64 `json:"utilization"` // GPU使用率(%)
Temperature int `json:"temperature"` // GPU温度(°C)
}
// 保持向后兼容的旧GPUReport结构
type GPUReport struct {
Name string `json:"name,omitempty"`
Usage float64 `json:"usage,omitempty"`
}
type RamReport struct {
Total int64 `json:"total"`
Used int64 `json:"used"`
}
type LoadReport struct {
Load1 float64 `json:"load1"`
Load5 float64 `json:"load5"`
Load15 float64 `json:"load15"`
}
type DiskReport struct {
Total int64 `json:"total"`
Used int64 `json:"used"`
}
type NetworkReport struct {
Up int64 `json:"up"`
Down int64 `json:"down"`
TotalUp int64 `json:"totalUp"`
TotalDown int64 `json:"totalDown"`
}
type ConnectionsReport struct {
TCP int `json:"tcp"`
UDP int `json:"udp"`
}
-83
View File
@@ -1,83 +0,0 @@
syntax = "proto3";
package proto;
option go_package = "github.com/komari-monitor/komari/compat/nezha/proto";
service NezhaService {
rpc ReportSystemState(stream State) returns (stream Receipt) {}
rpc ReportSystemInfo(Host) returns (Receipt) {}
rpc RequestTask(stream TaskResult) returns (stream Task) {}
rpc IOStream(stream IOStreamData) returns (stream IOStreamData) {}
rpc ReportGeoIP(GeoIP) returns (GeoIP) {}
rpc ReportSystemInfo2(Host) returns (Uint64Receipt) {}
}
message Host {
string platform = 1;
string platform_version = 2;
repeated string cpu = 3;
uint64 mem_total = 4;
uint64 disk_total = 5;
uint64 swap_total = 6;
string arch = 7;
string virtualization = 8;
uint64 boot_time = 9;
string version = 10;
repeated string gpu = 11;
}
message State {
double cpu = 1;
uint64 mem_used = 2;
uint64 swap_used = 3;
uint64 disk_used = 4;
uint64 net_in_transfer = 5;
uint64 net_out_transfer = 6;
uint64 net_in_speed = 7;
uint64 net_out_speed = 8;
uint64 uptime = 9;
double load1 = 10;
double load5 = 11;
double load15 = 12;
uint64 tcp_conn_count = 13;
uint64 udp_conn_count = 14;
uint64 process_count = 15;
repeated State_SensorTemperature temperatures = 16;
repeated double gpu = 17;
}
message State_SensorTemperature {
string name = 1;
double temperature = 2;
}
message Task {
uint64 id = 1;
uint64 type = 2;
string data = 3;
}
message TaskResult {
uint64 id = 1;
uint64 type = 2;
float delay = 3;
string data = 4;
bool successful = 5;
}
message Receipt { bool proced = 1; }
message Uint64Receipt { uint64 data = 1; }
message IOStreamData { bytes data = 1; }
message GeoIP {
bool use6 = 1;
IP ip = 2;
string country_code = 3;
uint64 dashboard_boot_time = 4;
}
message IP {
string ipv4 = 1;
string ipv6 = 2;
}
File diff suppressed because it is too large. Load diff
-374
View File
@@ -1,374 +0,0 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
package proto
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
const _ = grpc.SupportPackageIsVersion7
// NezhaServiceClient is the client API for NezhaService service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type NezhaServiceClient interface {
ReportSystemState(ctx context.Context, opts ...grpc.CallOption) (NezhaService_ReportSystemStateClient, error)
ReportSystemInfo(ctx context.Context, in *Host, opts ...grpc.CallOption) (*Receipt, error)
RequestTask(ctx context.Context, opts ...grpc.CallOption) (NezhaService_RequestTaskClient, error)
IOStream(ctx context.Context, opts ...grpc.CallOption) (NezhaService_IOStreamClient, error)
ReportGeoIP(ctx context.Context, in *GeoIP, opts ...grpc.CallOption) (*GeoIP, error)
ReportSystemInfo2(ctx context.Context, in *Host, opts ...grpc.CallOption) (*Uint64Receipt, error)
}
type nezhaServiceClient struct {
cc grpc.ClientConnInterface
}
func NewNezhaServiceClient(cc grpc.ClientConnInterface) NezhaServiceClient {
return &nezhaServiceClient{cc}
}
func (c *nezhaServiceClient) ReportSystemState(ctx context.Context, opts ...grpc.CallOption) (NezhaService_ReportSystemStateClient, error) {
stream, err := c.cc.NewStream(ctx, &_NezhaService_serviceDesc.Streams[0], "/proto.NezhaService/ReportSystemState", opts...)
if err != nil {
return nil, err
}
x := &nezhaServiceReportSystemStateClient{stream}
return x, nil
}
type NezhaService_ReportSystemStateClient interface {
Send(*State) error
Recv() (*Receipt, error)
grpc.ClientStream
}
type nezhaServiceReportSystemStateClient struct {
grpc.ClientStream
}
func (x *nezhaServiceReportSystemStateClient) Send(m *State) error {
return x.ClientStream.SendMsg(m)
}
func (x *nezhaServiceReportSystemStateClient) Recv() (*Receipt, error) {
m := new(Receipt)
if err := x.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func (c *nezhaServiceClient) ReportSystemInfo(ctx context.Context, in *Host, opts ...grpc.CallOption) (*Receipt, error) {
out := new(Receipt)
err := c.cc.Invoke(ctx, "/proto.NezhaService/ReportSystemInfo", in, out, opts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *nezhaServiceClient) RequestTask(ctx context.Context, opts ...grpc.CallOption) (NezhaService_RequestTaskClient, error) {
stream, err := c.cc.NewStream(ctx, &_NezhaService_serviceDesc.Streams[1], "/proto.NezhaService/RequestTask", opts...)
if err != nil {
return nil, err
}
x := &nezhaServiceRequestTaskClient{stream}
return x, nil
}
type NezhaService_RequestTaskClient interface {
Send(*TaskResult) error
Recv() (*Task, error)
grpc.ClientStream
}
type nezhaServiceRequestTaskClient struct {
grpc.ClientStream
}
func (x *nezhaServiceRequestTaskClient) Send(m *TaskResult) error {
return x.ClientStream.SendMsg(m)
}
func (x *nezhaServiceRequestTaskClient) Recv() (*Task, error) {
m := new(Task)
if err := x.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func (c *nezhaServiceClient) IOStream(ctx context.Context, opts ...grpc.CallOption) (NezhaService_IOStreamClient, error) {
stream, err := c.cc.NewStream(ctx, &_NezhaService_serviceDesc.Streams[2], "/proto.NezhaService/IOStream", opts...)
if err != nil {
return nil, err
}
x := &nezhaServiceIOStreamClient{stream}
return x, nil
}
type NezhaService_IOStreamClient interface {
Send(*IOStreamData) error
Recv() (*IOStreamData, error)
grpc.ClientStream
}
type nezhaServiceIOStreamClient struct {
grpc.ClientStream
}
func (x *nezhaServiceIOStreamClient) Send(m *IOStreamData) error {
return x.ClientStream.SendMsg(m)
}
func (x *nezhaServiceIOStreamClient) Recv() (*IOStreamData, error) {
m := new(IOStreamData)
if err := x.ClientStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func (c *nezhaServiceClient) ReportGeoIP(ctx context.Context, in *GeoIP, opts ...grpc.CallOption) (*GeoIP, error) {
out := new(GeoIP)
err := c.cc.Invoke(ctx, "/proto.NezhaService/ReportGeoIP", in, out, opts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *nezhaServiceClient) ReportSystemInfo2(ctx context.Context, in *Host, opts ...grpc.CallOption) (*Uint64Receipt, error) {
out := new(Uint64Receipt)
err := c.cc.Invoke(ctx, "/proto.NezhaService/ReportSystemInfo2", in, out, opts...)
if err != nil {
return nil, err
}
return out, nil
}
// NezhaServiceServer is the server API for NezhaService service.
// All implementations must embed UnimplementedNezhaServiceServer
// for forward compatibility
type NezhaServiceServer interface {
ReportSystemState(NezhaService_ReportSystemStateServer) error
ReportSystemInfo(context.Context, *Host) (*Receipt, error)
RequestTask(NezhaService_RequestTaskServer) error
IOStream(NezhaService_IOStreamServer) error
ReportGeoIP(context.Context, *GeoIP) (*GeoIP, error)
ReportSystemInfo2(context.Context, *Host) (*Uint64Receipt, error)
mustEmbedUnimplementedNezhaServiceServer()
}
// UnimplementedNezhaServiceServer must be embedded to have forward compatible implementations.
type UnimplementedNezhaServiceServer struct {
}
func (UnimplementedNezhaServiceServer) ReportSystemState(NezhaService_ReportSystemStateServer) error {
return status.Errorf(codes.Unimplemented, "method ReportSystemState not implemented")
}
func (UnimplementedNezhaServiceServer) ReportSystemInfo(context.Context, *Host) (*Receipt, error) {
return nil, status.Errorf(codes.Unimplemented, "method ReportSystemInfo not implemented")
}
func (UnimplementedNezhaServiceServer) RequestTask(NezhaService_RequestTaskServer) error {
return status.Errorf(codes.Unimplemented, "method RequestTask not implemented")
}
func (UnimplementedNezhaServiceServer) IOStream(NezhaService_IOStreamServer) error {
return status.Errorf(codes.Unimplemented, "method IOStream not implemented")
}
func (UnimplementedNezhaServiceServer) ReportGeoIP(context.Context, *GeoIP) (*GeoIP, error) {
return nil, status.Errorf(codes.Unimplemented, "method ReportGeoIP not implemented")
}
func (UnimplementedNezhaServiceServer) ReportSystemInfo2(context.Context, *Host) (*Uint64Receipt, error) {
return nil, status.Errorf(codes.Unimplemented, "method ReportSystemInfo2 not implemented")
}
func (UnimplementedNezhaServiceServer) mustEmbedUnimplementedNezhaServiceServer() {}
// UnsafeNezhaServiceServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to NezhaServiceServer will
// result in compilation errors.
type UnsafeNezhaServiceServer interface {
mustEmbedUnimplementedNezhaServiceServer()
}
func RegisterNezhaServiceServer(s *grpc.Server, srv NezhaServiceServer) {
s.RegisterService(&_NezhaService_serviceDesc, srv)
}
func _NezhaService_ReportSystemState_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(NezhaServiceServer).ReportSystemState(&nezhaServiceReportSystemStateServer{stream})
}
type NezhaService_ReportSystemStateServer interface {
Send(*Receipt) error
Recv() (*State, error)
grpc.ServerStream
}
type nezhaServiceReportSystemStateServer struct {
grpc.ServerStream
}
func (x *nezhaServiceReportSystemStateServer) Send(m *Receipt) error {
return x.ServerStream.SendMsg(m)
}
func (x *nezhaServiceReportSystemStateServer) Recv() (*State, error) {
m := new(State)
if err := x.ServerStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func _NezhaService_ReportSystemInfo_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(Host)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(NezhaServiceServer).ReportSystemInfo(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: "/proto.NezhaService/ReportSystemInfo",
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(NezhaServiceServer).ReportSystemInfo(ctx, req.(*Host))
}
return interceptor(ctx, in, info, handler)
}
func _NezhaService_RequestTask_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(NezhaServiceServer).RequestTask(&nezhaServiceRequestTaskServer{stream})
}
type NezhaService_RequestTaskServer interface {
Send(*Task) error
Recv() (*TaskResult, error)
grpc.ServerStream
}
type nezhaServiceRequestTaskServer struct {
grpc.ServerStream
}
func (x *nezhaServiceRequestTaskServer) Send(m *Task) error {
return x.ServerStream.SendMsg(m)
}
func (x *nezhaServiceRequestTaskServer) Recv() (*TaskResult, error) {
m := new(TaskResult)
if err := x.ServerStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func _NezhaService_IOStream_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(NezhaServiceServer).IOStream(&nezhaServiceIOStreamServer{stream})
}
type NezhaService_IOStreamServer interface {
Send(*IOStreamData) error
Recv() (*IOStreamData, error)
grpc.ServerStream
}
type nezhaServiceIOStreamServer struct {
grpc.ServerStream
}
func (x *nezhaServiceIOStreamServer) Send(m *IOStreamData) error {
return x.ServerStream.SendMsg(m)
}
func (x *nezhaServiceIOStreamServer) Recv() (*IOStreamData, error) {
m := new(IOStreamData)
if err := x.ServerStream.RecvMsg(m); err != nil {
return nil, err
}
return m, nil
}
func _NezhaService_ReportGeoIP_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(GeoIP)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(NezhaServiceServer).ReportGeoIP(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: "/proto.NezhaService/ReportGeoIP",
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(NezhaServiceServer).ReportGeoIP(ctx, req.(*GeoIP))
}
return interceptor(ctx, in, info, handler)
}
func _NezhaService_ReportSystemInfo2_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(Host)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(NezhaServiceServer).ReportSystemInfo2(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: "/proto.NezhaService/ReportSystemInfo2",
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(NezhaServiceServer).ReportSystemInfo2(ctx, req.(*Host))
}
return interceptor(ctx, in, info, handler)
}
var _NezhaService_serviceDesc = grpc.ServiceDesc{
ServiceName: "proto.NezhaService",
HandlerType: (*NezhaServiceServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "ReportSystemInfo",
Handler: _NezhaService_ReportSystemInfo_Handler,
},
{
MethodName: "ReportGeoIP",
Handler: _NezhaService_ReportGeoIP_Handler,
},
{
MethodName: "ReportSystemInfo2",
Handler: _NezhaService_ReportSystemInfo2_Handler,
},
},
Streams: []grpc.StreamDesc{
{
StreamName: "ReportSystemState",
Handler: _NezhaService_ReportSystemState_Handler,
ServerStreams: true,
ClientStreams: true,
},
{
StreamName: "RequestTask",
Handler: _NezhaService_RequestTask_Handler,
ServerStreams: true,
ClientStreams: true,
},
{
StreamName: "IOStream",
Handler: _NezhaService_IOStream_Handler,
ServerStreams: true,
ClientStreams: true,
},
},
Metadata: "compat/nezha/nezha.proto",
}
-133
View File
@@ -1,133 +0,0 @@
package config
import "time"
type Legacy struct {
ID uint `json:"id,omitempty"` // 1
Sitename string `json:"sitename" default:"Komari"` // 站点名称,默认 "Komari"
Description string `json:"description" default:"A simple server monitor tool."` // 站点描述
AllowCors bool `json:"allow_cors" default:"false"` // 是否允许跨域,默认 false
Theme string `json:"theme" default:"default"` // 主题名称,默认 'default'
PrivateSite bool `json:"private_site" default:"false"` // 是否为私有站点,默认 false
ApiKey string `json:"api_key" default:""` // API 密钥,默认空字符串
AutoDiscoveryKey string `json:"auto_discovery_key" default:""` // 自动发现密钥
ScriptDomain string `json:"script_domain" default:""` // 自定义脚本域名
SendIpAddrToGuest bool `json:"send_ip_addr_to_guest" default:"false"` // 是否向访客页面发送 IP 地址,默认 false
EulaAccepted bool `json:"eula_accepted" default:"false"`
BaseScriptsURLKey string `json:"base_scripts_url" default:""`
// GeoIP 配置
GeoIpEnabled bool `json:"geo_ip_enabled" default:"true"`
GeoIpProvider string `json:"geo_ip_provider" default:"ipinfo"` // empty, mmdb, ip-api, geojs
// Nezha 兼容(Agent gRPC)
NezhaCompatEnabled bool `json:"nezha_compat_enabled" default:"false"`
NezhaCompatListen string `json:"nezha_compat_listen" default:""` // 例如 0.0.0.0:5555
// OAuth 配置
OAuthEnabled bool `json:"o_auth_enabled" default:"false"`
OAuthProvider string `json:"o_auth_provider" default:"github"`
DisablePasswordLogin bool `json:"disable_password_login" default:"false"`
CloudflareTunnelToken string `json:"cloudflare_tunnel_token" default:""`
// 自定义美化
CustomHead string `json:"custom_head" default:""`
CustomBody string `json:"custom_body" default:""`
// 通知
NotificationEnabled bool `json:"notification_enabled" default:"true"` // 通知总开关
NotificationMethod string `json:"notification_method" default:"none"`
NotificationTemplate string `json:"notification_template" default:"{{emoji}}{{emoji}}{{emoji}}\nEvent: {{event}}\nClients: {{client}}\nMessage: {{message}}\nTime: {{time}}"`
ExpireNotificationEnabled bool `json:"expire_notification_enabled" default:"true"` // 是否启用过期通知
ExpireNotificationLeadDays int `json:"expire_notification_lead_days" default:"7"` // 过期前多少天通知,默认7天
LoginNotification bool `json:"login_notification" default:"true"` // 登录通知
TrafficLimitPercentage float64 `json:"traffic_limit_percentage" default:"80.00"` // 流量限制百分比,默认80.00%
// Record
RecordEnabled bool `json:"record_enabled" default:"true"` // 是否启用记录功能
RecordPreserveTime int `json:"record_preserve_time" default:"720"` // 记录保留时间,单位小时,默认30天
PingRecordPreserveTime int `json:"ping_record_preserve_time" default:"24"` // Ping 记录保留时间,单位小时,默认1天
UpdatedAt time.Time
}
const (
SitenameKey = "sitename"
DescriptionKey = "description"
AllowCorsKey = "allow_cors"
ThemeKey = "theme"
PrivateSiteKey = "private_site"
ApiKeyKey = "api_key"
AutoDiscoveryKeyKey = "auto_discovery_key"
ScriptDomainKey = "script_domain"
SendIpAddrToGuestKey = "send_ip_addr_to_guest"
EulaAcceptedKey = "eula_accepted"
BaseScriptsURLKey = "base_scripts_url"
GeoIpEnabledKey = "geo_ip_enabled"
GeoIpProviderKey = "geo_ip_provider"
NezhaCompatEnabledKey = "nezha_compat_enabled"
NezhaCompatListenKey = "nezha_compat_listen"
OAuthEnabledKey = "o_auth_enabled"
OAuthProviderKey = "o_auth_provider"
DisablePasswordLoginKey = "disable_password_login"
CloudflareTunnelTokenKey = "cloudflare_tunnel_token"
CustomHeadKey = "custom_head"
CustomBodyKey = "custom_body"
NotificationEnabledKey = "notification_enabled"
NotificationMethodKey = "notification_method"
NotificationTemplateKey = "notification_template"
ExpireNotificationEnabledKey = "expire_notification_enabled"
ExpireNotificationLeadDaysKey = "expire_notification_lead_days"
LoginNotificationKey = "login_notification"
TrafficLimitPercentageKey = "traffic_limit_percentage"
RecordEnabledKey = "record_enabled"
RecordPreserveTimeKey = "record_preserve_time"
PingRecordPreserveTimeKey = "ping_record_preserve_time"
UpdatedAtKey = "updated_at"
)
func (Legacy) TableName() string {
return "configs"
}
// Decrepted
/*
func Update(cst map[string]interface{}) error {
oldConfig, _ := GetManyAs[Legacy]()
// Proceed with update
cst["updated_at"] = time.Now().Unix()
delete(cst, "created_at")
delete(cst, "CreatedAt")
// 至少有一种登录方式启用
newDisablePasswordLogin := oldConfig.DisablePasswordLogin
newOAuthEnabled := oldConfig.OAuthEnabled
if val, exists := cst["disable_password_login"]; exists {
newDisablePasswordLogin = val.(bool)
}
if val, exists := cst["o_auth_enabled"]; exists {
newOAuthEnabled = val.(bool)
}
if newDisablePasswordLogin && !newOAuthEnabled {
return errors.New("at least one login method must be enabled (password/oauth)")
}
// 没绑定账号也不能禁用
if newDisablePasswordLogin {
usr := &models.User{}
if err := Db.Model(&models.User{}).First(usr).Error; err != nil {
return errors.Join(err, errors.New("failed to retrieve user"))
}
if usr.SSOID == "" {
return errors.New("cannot disable password login when no SSO-bound account exists")
}
}
err := Db.Transaction(func(tx *gorm.DB) error {
if err := tx.Model(&models.Config{}).Where("id = ?", oldConfig.ID).Updates(cst).Error; err != nil {
return errors.Join(err, errors.New("failed to update configuration"))
}
newConfig := &models.Config{}
if err := tx.Where("id = ?", oldConfig.ID).First(newConfig).Error; err != nil {
return errors.Join(err, errors.New("failed to retrieve updated configuration"))
}
//publishEvent(oldConfig, *newConfig)
return nil
})
if err != nil {
return err
}
return nil
}
*/
+3 -3
View File
@@ -100,8 +100,8 @@ func CreateDefaultAdminAccount() (username, passwd string, err error) {
Username: username,
Passwd: hashedPassword,
SSOID: "",
CreatedAt: models.FromTime(time.Now()),
UpdatedAt: models.FromTime(time.Now()),
CreatedAt: time.Now().UTC(),
UpdatedAt: time.Now().UTC(),
}
err = db.Create(&user).Error
@@ -171,7 +171,7 @@ func UpdateUser(uuid string, name, password, sso_type *string) error {
if sso_type != nil {
updates["sso_type"] = *sso_type
}
updates["updated_at"] = time.Now()
updates["updated_at"] = time.Now().UTC()
err := db.Model(&models.User{}).Where("uuid = ?", uuid).Updates(updates).Error
if err != nil {
return err
+7 -21
View File
@@ -6,10 +6,10 @@ import (
"net"
"time"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
messageevent "github.com/komari-monitor/komari/database/models/messageEvent"
"github.com/komari-monitor/komari/pkg/config"
"github.com/komari-monitor/komari/utils"
"github.com/komari-monitor/komari/utils/geoip"
"github.com/komari-monitor/komari/utils/messageSender"
@@ -33,11 +33,11 @@ func CreateSession(uuid string, expires int, userAgent, ip, login_method string)
sessionRecord := models.Session{
UUID: uuid,
Session: session,
Expires: models.FromTime(time.Now().Add(time.Duration(expires) * time.Second)),
Expires: time.Now().UTC().Add(time.Duration(expires) * time.Second),
UserAgent: userAgent,
Ip: ip,
LoginMethod: login_method,
LatestOnline: models.FromTime(time.Now()),
LatestOnline: time.Now().UTC(),
}
go func() {
LoginNotification, _ := config.GetAs[bool](config.LoginNotificationKey, false)
@@ -50,7 +50,7 @@ func CreateSession(uuid string, expires int, userAgent, ip, login_method string)
}
messageSender.SendEvent(models.EventMessage{
Event: messageevent.Login,
Time: time.Now(),
Time: time.Now().UTC(),
Message: fmt.Sprintf("%s: %s (%s)\n%s", login_method, ip, loc, userAgent),
Emoji: "🔑",
})
@@ -73,7 +73,7 @@ func GetSession(session string) (uuid string, err error) {
return "", err
}
if time.Now().After(sessionRecord.Expires.ToTime()) {
if time.Now().UTC().After(sessionRecord.Expires) {
// 会话已过期,删除它
_ = DeleteSession(session)
return "", errors.New("session expired")
@@ -111,24 +111,10 @@ func DeleteAllSessions() error {
return nil
}
func UpdateLatestOnline(session string) error {
db := dbcore.GetDBInstance()
return db.Model(&models.Session{}).Where("session = ?", session).Update("latest_online", time.Now()).Error
}
func UpdateLatestUserAgent(session, userAgent string) error {
db := dbcore.GetDBInstance()
return db.Model(&models.Session{}).Where("session = ?", session).Update("latest_user_agent", userAgent).Error
}
func UpdateLatestIp(session, ip string) error {
db := dbcore.GetDBInstance()
return db.Model(&models.Session{}).Where("session = ?", session).Update("latest_ip", ip).Error
}
func UpdateLatest(session, useragent, ip string) error {
db := dbcore.GetDBInstance()
return db.Model(&models.Session{}).Where("session = ?", session).Updates(map[string]interface{}{
"latest_online": time.Now(),
"latest_online": time.Now().UTC(),
"latest_user_agent": useragent,
"latest_ip": ip,
}).Error
@@ -136,7 +122,7 @@ func UpdateLatest(session, useragent, ip string) error {
func RemoveExpiredSessions() error {
db := dbcore.GetDBInstance()
result := db.Where("expires < ?", time.Now()).Delete(&models.Session{})
result := db.Where("expires < ?", time.Now().UTC()).Delete(&models.Session{})
if result.Error != nil {
return result.Error
}
+2 -3
View File
@@ -9,14 +9,13 @@ import (
)
func Log(ip, uuid, message, msgType string) {
now := time.Now()
db := dbcore.GetDBInstance()
logEntry := &models.Log{
IP: ip,
UUID: uuid,
Message: message,
MsgType: msgType,
Time: models.FromTime(now),
Time: time.Now().UTC(),
}
db.Create(logEntry)
}
@@ -28,7 +27,7 @@ func EventLog(eventType, message string) {
// Delete logs older than 30 days
func RemoveOldLogs() {
db := dbcore.GetDBInstance()
threshold := time.Now().AddDate(0, 0, -30)
threshold := time.Now().UTC().AddDate(0, 0, -30)
if err := db.Where("time < ?", threshold).Delete(&models.Log{}).Error; err != nil {
log.Println("Failed to remove old logs:", err)
}
+93 -159
View File
@@ -1,33 +1,28 @@
package clients
import (
"context"
"encoding/json"
"fmt"
"log"
"math"
"time"
"github.com/komari-monitor/komari/common"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/metricstore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/database/tasks"
"github.com/komari-monitor/komari/utils"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"fmt"
"github.com/google/uuid"
)
// Deprecated: DeleteClientConfig is deprecated and will be removed in a future release. Use DeleteClient instead.
func DeleteClientConfig(clientUuid string) error {
db := dbcore.GetDBInstance()
err := db.Delete(&common.ClientConfig{ClientUUID: clientUuid}).Error
if err != nil {
func DeleteClient(clientUuid string) error {
// Clean the independent metric store before removing the client from the main DB.
if err := metricstore.DeleteEntity(context.Background(), clientUuid); err != nil {
return err
}
return nil
}
func DeleteClient(clientUuid string) error {
db := dbcore.GetDBInstance()
err := db.Delete(&models.Client{}, "uuid = ?", clientUuid).Error
if err != nil {
@@ -36,79 +31,6 @@ func DeleteClient(clientUuid string) error {
return nil
}
// Deprecated: UpdateOrInsertBasicInfo is deprecated and will be removed in a future release. Use SaveClientInfo instead.
func UpdateOrInsertBasicInfo(cbi common.ClientInfo) error {
db := dbcore.GetDBInstance()
updates := make(map[string]interface{})
if cbi.Name != "" {
updates["name"] = cbi.Name
}
if cbi.CpuName != "" {
updates["cpu_name"] = cbi.CpuName
}
if cbi.Arch != "" {
updates["arch"] = cbi.Arch
}
if cbi.CpuCores > 0 || cbi.CpuCores < math.MaxInt-1 {
updates["cpu_cores"] = cbi.CpuCores
}
if cbi.OS != "" {
updates["os"] = cbi.OS
}
if cbi.GpuName != "" {
updates["gpu_name"] = cbi.GpuName
}
if cbi.IPv4 != "" {
updates["ipv4"] = cbi.IPv4
}
if cbi.IPv6 != "" {
updates["ipv6"] = cbi.IPv6
}
if cbi.Region != "" {
updates["region"] = cbi.Region
}
if cbi.Remark != "" {
updates["remark"] = cbi.Remark
}
updates["mem_total"] = cbi.MemTotal
updates["swap_total"] = cbi.SwapTotal
updates["disk_total"] = cbi.DiskTotal
updates["version"] = cbi.Version
updates["updated_at"] = time.Now()
// 转换为更新Client表
client := models.Client{
UUID: cbi.UUID,
}
err := db.Model(&client).Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "uuid"}},
DoUpdates: clause.Assignments(updates),
}).Create(map[string]interface{}{
"uuid": cbi.UUID,
"name": cbi.Name,
"cpu_name": cbi.CpuName,
"arch": cbi.Arch,
"cpu_cores": cbi.CpuCores,
"os": cbi.OS,
"gpu_name": cbi.GpuName,
"ipv4": cbi.IPv4,
"ipv6": cbi.IPv6,
"region": cbi.Region,
"remark": cbi.Remark,
"mem_total": cbi.MemTotal,
"swap_total": cbi.SwapTotal,
"disk_total": cbi.DiskTotal,
"version": cbi.Version,
"updated_at": time.Now(),
}).Error
if err != nil {
return err
}
return nil
}
func SaveClientInfo(update map[string]interface{}) error {
db := dbcore.GetDBInstance()
clientUUID, ok := update["uuid"].(string)
@@ -121,29 +43,75 @@ func SaveClientInfo(update map[string]interface{}) error {
return fmt.Errorf("no fields to update")
}
update["updated_at"] = time.Now()
update["updated_at"] = time.Now().UTC()
checkInt64 := func(name string, val float64) error {
if val < 0 {
return fmt.Errorf("%s must be non-negative, got %d", name, int64(val))
toFloat64 := func(value interface{}) (float64, bool) {
switch typed := value.(type) {
case float64:
return typed, true
case float32:
return float64(typed), true
case int:
return float64(typed), true
case int8:
return float64(typed), true
case int16:
return float64(typed), true
case int32:
return float64(typed), true
case int64:
return float64(typed), true
case uint:
return float64(typed), true
case uint8:
return float64(typed), true
case uint16:
return float64(typed), true
case uint32:
return float64(typed), true
case uint64:
return float64(typed), true
case json.Number:
parsed, err := typed.Float64()
if err != nil {
return 0, false
}
return parsed, true
default:
return 0, false
}
if val > math.MaxInt64-1 {
return fmt.Errorf("%s exceeds int64 max limit: %d", name, int64(val))
}
checkOptionalInt := func(name, key string, maxValue float64) error {
value, exists := update[key]
if !exists || value == nil {
return nil
}
numericValue, ok := toFloat64(value)
if !ok {
return fmt.Errorf("%s must be a valid number", name)
}
if numericValue < 0 || numericValue > maxValue {
return fmt.Errorf("%s must be a valid non-negative number: %v", name, value)
}
return nil
}
verify := func(update map[string]interface{}) error {
if update["cpu_cores"].(float64) < 0 || update["cpu_cores"].(float64) > math.MaxInt-1 {
return fmt.Errorf("Cpu.Cores be not a valid int64 number: %d", update["cpu_cores"])
}
if err := checkInt64("Ram.Total", update["mem_total"].(float64)); err != nil {
if err := checkOptionalInt("Cpu.Cores", "cpu_cores", math.MaxInt-1); err != nil {
return err
}
if err := checkInt64("Swap.Total", update["swap_total"].(float64)); err != nil {
if err := checkOptionalInt("Cpu.PhysicalCores", "cpu_physical_cores", math.MaxInt-1); err != nil {
return err
}
if err := checkInt64("Disk.Total", update["disk_total"].(float64)); err != nil {
if err := checkOptionalInt("Ram.Total", "mem_total", math.MaxInt64-1); err != nil {
return err
}
if err := checkOptionalInt("Swap.Total", "swap_total", math.MaxInt64-1); err != nil {
return err
}
if err := checkOptionalInt("Disk.Total", "disk_total", math.MaxInt64-1); err != nil {
return err
}
return nil
@@ -160,49 +128,6 @@ func SaveClientInfo(update map[string]interface{}) error {
return nil
}
// 更新客户端设置
func UpdateClientConfig(config common.ClientConfig) error {
db := dbcore.GetDBInstance()
err := db.Save(&config).Error
if err != nil {
return err
}
return nil
}
func EditClientName(clientUUID, clientName string) error {
db := dbcore.GetDBInstance()
err := db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Update("name", clientName).Error
if err != nil {
return err
}
return nil
}
/*
// UpdateClientByUUID 更新指定 UUID 的客户端配置
func UpdateClientByUUID(config common.ClientConfig) error {
db := dbcore.GetDBInstance()
result := db.Model(&common.ClientConfig{}).Where("client_uuid = ?", config.ClientUUID).Updates(config)
if result.Error != nil {
return result.Error
}
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
return nil
}
*/
func EditClientToken(clientUUID, token string) error {
db := dbcore.GetDBInstance()
err := db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Update("token", token).Error
if err != nil {
return err
}
return nil
}
// CreateClient 创建新客户端
func CreateClient() (clientUUID, token string, err error) {
db := dbcore.GetDBInstance()
@@ -213,8 +138,8 @@ func CreateClient() (clientUUID, token string, err error) {
UUID: clientUUID,
Token: token,
Name: "client_" + clientUUID[0:8],
CreatedAt: models.FromTime(time.Now()),
UpdatedAt: models.FromTime(time.Now()),
CreatedAt: time.Now().UTC(),
UpdatedAt: time.Now().UTC(),
}
err = db.Create(&client).Error
@@ -238,8 +163,8 @@ func CreateClientWithName(name string) (clientUUID, token string, err error) {
UUID: clientUUID,
Token: token,
Name: name,
CreatedAt: models.FromTime(time.Now()),
UpdatedAt: models.FromTime(time.Now()),
CreatedAt: time.Now().UTC(),
UpdatedAt: time.Now().UTC(),
}
err = db.Create(&client).Error
@@ -273,19 +198,6 @@ func GetClientByUUID(uuid string) (client models.Client, err error) {
return client, nil
}
// GetClientBasicInfo 获取指定 UUID 的客户端基本信息
func GetClientBasicInfo(uuid string) (client models.Client, err error) {
db := dbcore.GetDBInstance()
err = db.Where("uuid = ?", uuid).First(&client).Error
if err != nil {
if err == gorm.ErrRecordNotFound {
return models.Client{}, fmt.Errorf("客户端不存在: %s", uuid)
}
return models.Client{}, err
}
return client, nil
}
func GetClientTokenByUUID(uuid string) (token string, err error) {
db := dbcore.GetDBInstance()
var client models.Client
@@ -324,8 +236,30 @@ func SaveClient(updates map[string]interface{}) error {
}
}
}
if value, exists := updates["expired_at"]; exists {
switch typed := value.(type) {
case nil:
updates["expired_at"] = nil
case time.Time:
updates["expired_at"] = typed.UTC()
case *time.Time:
if typed == nil {
updates["expired_at"] = nil
} else {
updates["expired_at"] = typed.UTC()
}
case string:
stamp, err := time.Parse(time.RFC3339Nano, typed)
if err != nil {
return fmt.Errorf("expired_at must be an RFC3339 timestamp with a timezone: %w", err)
}
updates["expired_at"] = stamp.UTC()
default:
return fmt.Errorf("expired_at must be an RFC3339 timestamp with a timezone")
}
}
updates["updated_at"] = time.Now()
updates["updated_at"] = time.Now().UTC()
err := db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(updates).Error
if err != nil {
+2 -150
View File
@@ -1,35 +1,14 @@
package clients
import (
"encoding/json"
"fmt"
"math"
"time"
"github.com/komari-monitor/komari/common"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"gorm.io/gorm"
v1 "github.com/komari-monitor/komari/protocol/v1"
)
// Report 表示客户端报告数据
// SaveReport 保存报告数据
func SaveReport(uuid string, data map[string]interface{}) (err error) {
report, err := ParseReport(data)
if err != nil {
return err
}
err = SaveClientReport(uuid, report)
if err != nil {
return err
}
return nil
}
func GetClientUUIDByToken(token string) (clientUUID string, err error) {
db := dbcore.GetDBInstance()
var client models.Client
@@ -40,20 +19,8 @@ func GetClientUUIDByToken(token string) (clientUUID string, err error) {
return client.UUID, nil
}
func ParseReport(data map[string]interface{}) (report common.Report, err error) {
jsonData, err := json.Marshal(data)
if err != nil {
return common.Report{}, err
}
err = json.Unmarshal(jsonData, &report)
if err != nil {
return common.Report{}, err
}
return report, nil
}
// 检查数据防止异常数据导致数据库损坏
func ReportVerify(report common.Report) error {
func ReportVerify(report v1.Report) error {
// 防止输入不合理范围
if report.CPU.Usage < 0 || report.CPU.Usage > 100 {
return fmt.Errorf("CPU.Usage must be between 0 and 100")
@@ -135,118 +102,3 @@ func ReportVerify(report common.Report) error {
}
return nil
}
// SaveClientReport 保存客户端报告到 Record 表
func SaveClientReport(clientUUID string, report common.Report) (err error) {
db := dbcore.GetDBInstance()
if err := ReportVerify(report); err != nil {
return fmt.Errorf("failed to save Record: %v", err)
}
// 保存GPU详细记录到独立表
currentTime := time.Now()
if report.GPU != nil && len(report.GPU.DetailedInfo) > 0 {
for idx, gpu := range report.GPU.DetailedInfo {
gpuRecord := models.GPURecord{
Client: clientUUID,
Time: models.FromTime(currentTime),
DeviceIndex: idx,
DeviceName: gpu.Name,
MemTotal: gpu.MemoryTotal,
MemUsed: gpu.MemoryUsed,
Utilization: float32(gpu.Utilization),
Temperature: gpu.Temperature,
}
if err := db.Create(&gpuRecord).Error; err != nil {
return fmt.Errorf("failed to save GPU record: %v", err)
}
}
}
// 计算平均GPU使用率,用于向后兼容
averageGPUUsage := float32(0)
if report.GPU != nil && len(report.GPU.DetailedInfo) > 0 {
averageGPUUsage = float32(report.GPU.AverageUsage)
}
Record := models.Record{
Client: clientUUID,
Time: models.FromTime(currentTime),
Cpu: float32(report.CPU.Usage),
Gpu: averageGPUUsage, // 使用平均GPU使用率
Ram: report.Ram.Used,
RamTotal: report.Ram.Total,
Swap: report.Swap.Used,
SwapTotal: report.Swap.Total,
Load: float32(report.Load.Load1), // 使用 Load1 作为主要负载指标
Temp: 0, // Report 未提供 TEMP,设为 0(与原 nil 行为类似)
Disk: report.Disk.Used,
DiskTotal: report.Disk.Total,
NetIn: report.Network.Down,
NetOut: report.Network.Up,
NetTotalUp: report.Network.TotalUp,
NetTotalDown: report.Network.TotalDown,
Process: report.Process,
Connections: report.Connections.TCP,
ConnectionsUdp: report.Connections.UDP,
//Uptime: report.Uptime,
}
// 使用事务确保 Record 和 ClientsInfo 一致性
err = db.Transaction(func(tx *gorm.DB) error {
// 保存 Record
if err := tx.Create(&Record).Error; err != nil {
return fmt.Errorf("failed to save Record: %v", err)
}
return nil
})
if err != nil {
return err
}
return nil
}
/*
// getString 从 map 中获取字符串
func getString(data map[string]interface{}, key string) string {
if val, ok := data[key]; ok {
if str, ok := val.(string); ok {
return str
}
}
return ""
}
// getInt 从 map 中获取整数
func getInt(data map[string]interface{}, key string) int {
if val, ok := data[key]; ok {
if num, ok := val.(float64); ok {
return int(num)
}
}
return 0
}
// getInt64 从 map 中获取 int64
func getInt64(data map[string]interface{}, key string) int64 {
if val, ok := data[key]; ok {
if num, ok := val.(float64); ok {
return int64(num)
}
}
return 0
}
// getFloat 从 map 中获取 float64
func getFloat(data map[string]interface{}, key string) float64 {
if val, ok := data[key]; ok {
if num, ok := val.(float64); ok {
return num
}
}
return 0.0
}
*/
+6
View File
@@ -1,6 +1,8 @@
package clipboard
import (
"time"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
@@ -8,6 +10,8 @@ import (
// CreateClipboard 创建剪贴板记录
func CreateClipboard(cb *models.Clipboard) error {
db := dbcore.GetDBInstance()
cb.CreatedAt = time.Now().UTC()
cb.UpdatedAt = cb.CreatedAt
return db.Create(cb).Error
}
@@ -24,6 +28,8 @@ func GetClipboardByID(id int) (*models.Clipboard, error) {
// UpdateClipboardFields 更新剪贴板记录
func UpdateClipboardFields(id int, fields map[string]interface{}) error {
db := dbcore.GetDBInstance()
delete(fields, "created_at")
fields["updated_at"] = time.Now().UTC()
return db.Model(&models.Clipboard{}).Where("id = ?", id).Updates(fields).Error
}
+292 -332
View File
@@ -2,7 +2,6 @@ package dbcore
import (
"archive/zip"
"encoding/json"
"fmt"
"io"
"log"
@@ -13,9 +12,9 @@ import (
"time"
"github.com/komari-monitor/komari/cmd/flags"
"github.com/komari-monitor/komari/common"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/config"
"github.com/komari-monitor/komari/pkg/migrations"
logutil "github.com/komari-monitor/komari/utils/log"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
@@ -166,342 +165,303 @@ func unzipToDir(zipPath, dstDir string) error {
return nil
}
// mergeClientInfo 将旧版ClientInfo数据迁移到新版Client表
func mergeClientInfo(db *gorm.DB) {
var clientInfos []common.ClientInfo
if err := db.Find(&clientInfos).Error; err != nil {
log.Printf("Failed to read ClientInfo table: %v", err)
return
}
for _, info := range clientInfos {
var client models.Client
if err := db.Where("uuid = ?", info.UUID).First(&client).Error; err != nil {
log.Printf("Could not find Client record with UUID %s: %v", info.UUID, err)
continue
}
// 更新Client记录
client.Name = info.Name
client.CpuName = info.CpuName
client.Virtualization = info.Virtualization
client.Arch = info.Arch
client.CpuCores = info.CpuCores
client.OS = info.OS
client.GpuName = info.GpuName
client.IPv4 = info.IPv4
client.IPv6 = info.IPv6
client.Region = info.Region
client.Remark = info.Remark
client.PublicRemark = info.PublicRemark
client.MemTotal = info.MemTotal
client.SwapTotal = info.SwapTotal
client.DiskTotal = info.DiskTotal
client.Version = info.Version
client.Weight = info.Weight
client.Price = info.Price
client.BillingCycle = info.BillingCycle
client.ExpiredAt = models.FromTime(info.ExpiredAt)
// Save updated Client record
if err := db.Save(&client).Error; err != nil {
log.Printf("Failed to update Client record: %v", err)
continue
}
}
// Backup and rename old table after migration
if err := db.Migrator().RenameTable("client_infos", "client_infos_backup"); err != nil {
log.Printf("Failed to backup ClientInfo table: %v", err)
return
}
log.Println("Data migration completed, old table has been backed up as client_infos_backup")
}
func MergeDatabase(db *gorm.DB) {
if db.Migrator().HasTable("client_infos") {
log.Println("[>0.0.5] Legacy ClientInfo table detected, starting data migration...")
mergeClientInfo(db)
}
if db.Migrator().HasColumn(&models.Config{}, "allow_cros") {
log.Println("[>0.0.5a] Renaming column 'allow_cros' to 'allow_cors' in config table...")
db.Migrator().RenameColumn(&models.Config{}, "allow_cros", "allow_cors")
}
if db.Migrator().HasColumn(&models.LoadNotification{}, "client") {
log.Println("[>0.1.4] Rebuilding LoadNotification table....")
db.Migrator().DropTable(&models.LoadNotification{})
}
if !db.Migrator().HasTable(&models.OidcProvider{}) && db.Migrator().HasTable(&models.Config{}) {
log.Println("[>1.0.2] Merge OidcProvider table....")
var config struct {
OAuthClientID string `json:"o_auth_client_id" gorm:"type:varchar(255)"`
OAuthClientSecret string `json:"o_auth_client_secret" gorm:"type:varchar(255)"`
}
if err := db.Raw("SELECT * FROM configs LIMIT 1").Scan(&config).Error; err != nil {
log.Println("Failed to get config for OIDC provider migration:", err)
}
db.AutoMigrate(&models.OidcProvider{})
j, err := json.Marshal(&map[string]string{
"client_id": config.OAuthClientID,
"client_secret": config.OAuthClientSecret,
})
if err != nil {
log.Println("Failed to marshal OIDC provider config:", err)
return
}
db.Save(&models.OidcProvider{
Name: "github",
Addition: string(j),
})
db.AutoMigrate(&models.Config{})
db.Model(&models.Config{}).Where("id = 1").Update("o_auth_provider", "github")
}
if !db.Migrator().HasTable(&models.MessageSenderProvider{}) && db.Migrator().HasTable(&models.Config{}) {
log.Println("[>1.0.2] Migrate MessageSender configuration....")
var config struct {
TelegramBotToken string `json:"telegram_bot_token" gorm:"type:varchar(255)"`
TelegramChatID string `json:"telegram_chat_id" gorm:"type:varchar(255)"`
TelegramEndpoint string `json:"telegram_endpoint" gorm:"type:varchar(255)"`
EmailHost string `json:"email_host" gorm:"type:varchar(255)"`
EmailPort int `json:"email_port" gorm:"type:int"`
EmailUsername string `json:"email_username" gorm:"type:varchar(255)"`
EmailPassword string `json:"email_password" gorm:"type:varchar(255)"`
EmailSender string `json:"email_sender" gorm:"type:varchar(255)"`
EmailReceiver string `json:"email_receiver" gorm:"type:varchar(255)"`
EmailUseSSL bool `json:"email_use_ssl" gorm:"type:boolean"`
NotificationMethod string `json:"notification_method" gorm:"type:varchar(50)"`
}
if err := db.Raw("SELECT * FROM configs LIMIT 1").Scan(&config).Error; err != nil {
log.Println("Failed to get config for MessageSender migration:", err)
}
db.AutoMigrate(&models.MessageSenderProvider{})
// 迁移Telegram配置
if config.NotificationMethod == "telegram" && config.TelegramBotToken != "" {
telegramConfig := map[string]interface{}{
"bot_token": config.TelegramBotToken,
"chat_id": config.TelegramChatID,
"endpoint": config.TelegramEndpoint,
}
if telegramConfig["endpoint"] == "" {
telegramConfig["endpoint"] = "https://api.telegram.org/bot"
}
telegramConfigJSON, err := json.Marshal(telegramConfig)
if err != nil {
log.Println("Failed to marshal Telegram config:", err)
} else {
db.Save(&models.MessageSenderProvider{
Name: "telegram",
Addition: string(telegramConfigJSON),
})
}
}
// 迁移Email配置
if config.NotificationMethod == "email" && config.EmailHost != "" {
emailConfig := map[string]interface{}{
"host": config.EmailHost,
"port": config.EmailPort,
"username": config.EmailUsername,
"password": config.EmailPassword,
"sender": config.EmailSender,
"receiver": config.EmailReceiver,
"use_ssl": config.EmailUseSSL,
}
emailConfigJSON, err := json.Marshal(emailConfig)
if err != nil {
log.Println("Failed to marshal Email config:", err)
} else {
db.Save(&models.MessageSenderProvider{
Name: "email",
Addition: string(emailConfigJSON),
})
}
}
// 删除旧的配置字段
if db.Migrator().HasColumn(&models.Config{}, "telegram_bot_token") {
db.Migrator().DropColumn(&models.Config{}, "telegram_bot_token")
}
if db.Migrator().HasColumn(&models.Config{}, "telegram_chat_id") {
db.Migrator().DropColumn(&models.Config{}, "telegram_chat_id")
}
if db.Migrator().HasColumn(&models.Config{}, "telegram_endpoint") {
db.Migrator().DropColumn(&models.Config{}, "telegram_endpoint")
}
if db.Migrator().HasColumn(&models.Config{}, "email_host") {
db.Migrator().DropColumn(&models.Config{}, "email_host")
}
if db.Migrator().HasColumn(&models.Config{}, "email_port") {
db.Migrator().DropColumn(&models.Config{}, "email_port")
}
if db.Migrator().HasColumn(&models.Config{}, "email_username") {
db.Migrator().DropColumn(&models.Config{}, "email_username")
}
if db.Migrator().HasColumn(&models.Config{}, "email_password") {
db.Migrator().DropColumn(&models.Config{}, "email_password")
}
if db.Migrator().HasColumn(&models.Config{}, "email_sender") {
db.Migrator().DropColumn(&models.Config{}, "email_sender")
}
if db.Migrator().HasColumn(&models.Config{}, "email_receiver") {
db.Migrator().DropColumn(&models.Config{}, "email_receiver")
}
if db.Migrator().HasColumn(&models.Config{}, "email_use_ssl") {
db.Migrator().DropColumn(&models.Config{}, "email_use_ssl")
}
}
}
var (
instance *gorm.DB
once sync.Once
initErr error
)
func GetDBInstance() *gorm.DB {
// SystemVersionKey 是记录“上次启动所用版本标识”的配置键(存于 configs 表)。
// 取代旧的 ./data/.komari-version 文件:版本标识随配置库一起备份/恢复,
// 也避免额外的裸文件依赖。
const SystemVersionKey = "system_version"
// versionID 是当前构建的版本标识,由 SetVersionID 在 Initialize 前注入。
var versionID string
// dbFileExistedAtStartup 记录本次进程启动、打开数据库之前 komari.db 是否已存在,
// 用于区分“全新安装”与“从旧版升级(无版本标记)”。在 doInitialize 打开数据库
// 之前采集。
var dbFileExistedAtStartup bool
// SetVersionID 设置当前构建的版本标识(通常为 CurrentVersion+"-"+VersionHash),
// 用于版本升级检测与自动备份。应在 Initialize() 之前调用;为空则跳过升级备份。
func SetVersionID(id string) {
versionID = id
}
// resolveDatabaseFile 返回当前使用的 SQLite 数据库文件路径。
func resolveDatabaseFile() string {
dbFile := flags.DatabaseFile
if dbFile == "" {
dbFile = "./data/komari.db"
}
return dbFile
}
// backupOnVersionUpgrade 在检测到版本升级时,把当前 ./data 打包到
// ./backup/upgrade-{time}.zip,便于升级(含 metrics 迁移)异常时回滚。
//
// 版本标识存放于配置库(configs 表,键 system_version),因此本函数必须在
// config.SetDb 之后、一次性 metrics 迁移(InitStores)之前调用。
//
// 触发规则:
// - versionID 为空:跳过(未注入版本,如部分测试场景)。
// - 配置中无版本且启动前无数据库文件:全新安装,仅写版本,不备份。
// - 配置中无版本但启动前已有数据库文件:从无版本标记的旧稳定版升级,备份。
// - 配置中版本与当前不同:版本升级,备份。
// - 配置中版本与当前一致:无需备份。
//
// 备份失败不阻止启动,但打印明确错误;备份成功(或无需备份)后写入/更新版本。
func backupOnVersionUpgrade() {
if versionID == "" {
return
}
prevVersion, readErr := config.GetAs[string](SystemVersionKey)
prevVersion = strings.TrimSpace(prevVersion)
versionRecorded := readErr == nil && prevVersion != ""
// 版本未变化,无需备份。
if versionRecorded && prevVersion == versionID {
return
}
// 全新安装:配置中无版本且启动前无数据库文件,直接写版本不备份。
if !versionRecorded && !dbFileExistedAtStartup {
writeVersionMarker()
return
}
// 需要备份(升级或从旧稳定版首次带版本标记启动)。
// 先做一次 WAL checkpoint,确保 komari.db 主文件包含最新数据,
// 避免备份出的库缺少仍留在 -wal 中的写入。
if instance != nil {
instance.Exec("PRAGMA wal_checkpoint(TRUNCATE);")
}
if err := os.MkdirAll("./backup", 0755); err != nil {
log.Printf("[upgrade-backup] failed to create backup dir: %v", err)
return
}
tsName := time.Now().UTC().Format("20060102-150405")
bakPath := filepath.Join("./backup", fmt.Sprintf("upgrade-%s.zip", tsName))
backupZipPath := filepath.Join(".", "data", "backup.zip")
if zipErr := zipDirectoryExcluding("./data", bakPath, map[string]struct{}{backupZipPath: {}}); zipErr != nil {
log.Printf("[upgrade-backup] failed to backup ./data before upgrade (from %q to %q): %v", prevVersion, versionID, zipErr)
return
}
log.Printf("[upgrade-backup] ./data backed up to %s before upgrade (from %q to %q)", bakPath, prevVersion, versionID)
writeVersionMarker()
}
// writeVersionMarker 将当前 versionID 写入配置库。
func writeVersionMarker() {
if err := config.Set(SystemVersionKey, versionID); err != nil {
log.Printf("[upgrade-backup] failed to persist version marker: %v", err)
}
}
func buildSQLiteDSN(databaseFile string) string {
if databaseFile == "" {
databaseFile = "./data/komari.db"
}
params := "_busy_timeout=5000&_txlock=immediate&_journal_mode=WAL&_synchronous=NORMAL"
separator := "?"
if strings.Contains(databaseFile, "?") {
separator = "&"
}
if strings.HasPrefix(databaseFile, "file:") {
return databaseFile + separator + params
}
if databaseFile == ":memory:" {
return "file::memory:?cache=shared&" + params
}
return "file:" + filepath.ToSlash(databaseFile) + separator + params
}
// Initialize 显式初始化数据库连接与表结构,仅执行一次。
// 与 GetDBInstance 不同,Initialize 返回错误而非直接退出进程,
// 便于启动生命周期统一处理错误、以及在测试/CLI 命令中做隔离。
func Initialize() error {
once.Do(func() {
var err error
// 在数据库初始化前执行:如果存在 ./data/backup.zip,则进行恢复逻辑
func() {
backupZipPath := filepath.Join(".", "data", "backup.zip")
if _, statErr := os.Stat(backupZipPath); statErr == nil {
// 4. 把除了 ./data/backup.zip 之外的所有文件压缩到 ./backup/{time}.zip
if err := os.MkdirAll("./backup", 0755); err != nil {
log.Printf("[restore] failed to create backup dir: %v", err)
} else {
tsName := time.Now().Format("20060102-150405")
bakPath := filepath.Join("./backup", fmt.Sprintf("%s.zip", tsName))
if zipErr := zipDirectoryExcluding("./data", bakPath, map[string]struct{}{backupZipPath: {}}); zipErr != nil {
log.Printf("[restore] failed to zip current data: %v", zipErr)
} else {
log.Printf("[restore] current data zipped to %s", bakPath)
}
}
// 5. 删除除了 ./data/backup.zip 之外的所有文件
if delErr := removeAllInDirExcept("./data", map[string]struct{}{backupZipPath: {}}); delErr != nil {
log.Printf("[restore] failed to cleanup data dir: %v", delErr)
}
// 6. 解压 ./data/backup.zip 到 ./data
if unzipErr := unzipToDir(backupZipPath, "./data"); unzipErr != nil {
log.Printf("[restore] failed to unzip backup into data: %v", unzipErr)
} else {
log.Printf("[restore] backup.zip extracted to ./data")
}
// 7. 删除 ./data/backup.zip
if rmErr := os.Remove(backupZipPath); rmErr != nil {
log.Printf("[restore] failed to remove backup.zip: %v", rmErr)
} else {
log.Printf("[restore] backup.zip removed")
}
// 8. 删除标记
if rmErr := os.Remove("./data/komari-backup-markup"); rmErr != nil {
log.Printf("[restore] failed to remove komari-backup-markup: %v", rmErr)
} else {
log.Printf("[restore] komari-backup-markup removed")
}
}
}()
logConfig := &gorm.Config{
Logger: logutil.NewGormLogger(),
}
// 根据数据库类型选择不同的连接方式
switch flags.DatabaseType {
case "sqlite", "":
// SQLite 连接
instance, err = gorm.Open(sqlite.Open(flags.DatabaseFile), logConfig)
if err != nil {
log.Fatalf("Failed to connect to SQLite3 database: %v", err)
}
log.Printf("Using SQLite database file: %s", flags.DatabaseFile)
instance.Exec("PRAGMA wal = ON;")
if err := instance.Exec("PRAGMA journal_mode = WAL;").Error; err != nil {
log.Printf("Failed to enable WAL mode for SQLite: %v", err)
}
instance.Exec("PRAGMA synchronous = NORMAL;")
instance.Exec("PRAGMA cache_size = -65536;")
instance.Exec("PRAGMA temp_store = MEMORY;")
instance.Exec("PRAGMA wal_checkpoint(TRUNCATE);")
// case "mysql":
// // MySQL 连接
// dsn := fmt.Sprintf("%s:%s@tcp(%s:%s)/%s?charset=utf8mb4&collation=utf8mb4_unicode_ci&parseTime=True&loc=Local",
// flags.DatabaseUser,
// flags.DatabasePass,
// flags.DatabaseHost,
// flags.DatabasePort,
// flags.DatabaseName)
// instance, err = gorm.Open(mysql.Open(dsn), logConfig)
// if err != nil {
// log.Fatalf("Failed to connect to MySQL database: %v", err)
// }
// log.Printf("Using MySQL database: %s@%s:%s/%s", flags.DatabaseUser, flags.DatabaseHost, flags.DatabasePort, flags.DatabaseName)
default:
log.Fatalf("Unsupported database type: %s", flags.DatabaseType)
}
config.SetDb(instance)
MergeDatabase(instance)
// 自动迁移模型
err = instance.AutoMigrate(
&models.User{},
&models.Client{},
&models.Record{},
&models.GPURecord{},
// &models.Config{},
&models.Log{},
&models.Clipboard{},
&models.LoadNotification{},
&models.OfflineNotification{},
&models.PingRecord{},
&models.PingTask{},
&models.OidcProvider{},
&models.MessageSenderProvider{},
&models.ThemeConfiguration{},
)
if err != nil {
log.Fatalf("Failed to create tables: %v", err)
}
err = instance.Table("records_long_term").AutoMigrate(
&models.Record{},
)
if err != nil {
log.Printf("Failed to create records_long_term table, it may already exist: %v", err)
}
err = instance.Table("gpu_records_long_term").AutoMigrate(
&models.GPURecord{},
)
if err != nil {
log.Printf("Failed to create gpu_records_long_term table, it may already exist: %v", err)
}
err = instance.AutoMigrate(
&models.Session{},
)
if err != nil {
log.Printf("Failed to create Session table, it may already exist: %v", err)
}
err = instance.AutoMigrate(
&models.Task{},
&models.TaskResult{},
)
if err != nil {
log.Printf("Failed to create Task and TaskResult table, it may already exist: %v", err)
}
// Manually create composite indexes
if flags.DatabaseType == "sqlite" || flags.DatabaseType == "" {
instance.Exec("CREATE INDEX IF NOT EXISTS idx_record_client_time ON records(client, time)")
instance.Exec("CREATE INDEX IF NOT EXISTS idx_record_lt_client_time ON records_long_term(client, time)")
instance.Exec("CREATE INDEX IF NOT EXISTS idx_gpu_record_client_time ON gpu_records(client, time)")
instance.Exec("CREATE INDEX IF NOT EXISTS idx_gpu_record_lt_client_time ON gpu_records_long_term(client, time)")
instance.Exec("CREATE INDEX IF NOT EXISTS idx_ping_record_client_time ON ping_records(client, time)")
}
initErr = doInitialize()
})
return initErr
}
// GetDBInstance 返回全局数据库实例。
// 为兼容既有大量调用点,这里保留“出错即退出”的语义;
// 需要错误处理的启动流程应优先调用 Initialize()。
func GetDBInstance() *gorm.DB {
if err := Initialize(); err != nil {
log.Fatalf("Failed to initialize database: %v", err)
}
return instance
}
// Close 关闭底层数据库连接,供关闭流程调用。
func Close() error {
if instance == nil {
return nil
}
sqlDB, err := instance.DB()
if err != nil {
return err
}
return sqlDB.Close()
}
func doInitialize() error {
var err error
// 在数据库初始化前执行:如果存在 ./data/backup.zip,则进行恢复逻辑
func() {
backupZipPath := filepath.Join(".", "data", "backup.zip")
if _, statErr := os.Stat(backupZipPath); statErr == nil {
// 4. 把除了 ./data/backup.zip 之外的所有文件压缩到 ./backup/{time}.zip
if err := os.MkdirAll("./backup", 0755); err != nil {
log.Printf("[restore] failed to create backup dir: %v", err)
} else {
tsName := time.Now().UTC().Format("20060102-150405")
bakPath := filepath.Join("./backup", fmt.Sprintf("%s.zip", tsName))
if zipErr := zipDirectoryExcluding("./data", bakPath, map[string]struct{}{backupZipPath: {}}); zipErr != nil {
log.Printf("[restore] failed to zip current data: %v", zipErr)
} else {
log.Printf("[restore] current data zipped to %s", bakPath)
}
}
// 5. 删除除了 ./data/backup.zip 之外的所有文件
if delErr := removeAllInDirExcept("./data", map[string]struct{}{backupZipPath: {}}); delErr != nil {
log.Printf("[restore] failed to cleanup data dir: %v", delErr)
}
// 6. 解压 ./data/backup.zip 到 ./data
if unzipErr := unzipToDir(backupZipPath, "./data"); unzipErr != nil {
log.Printf("[restore] failed to unzip backup into data: %v", unzipErr)
} else {
log.Printf("[restore] backup.zip extracted to ./data")
}
// 7. 删除 ./data/backup.zip
if rmErr := os.Remove(backupZipPath); rmErr != nil {
log.Printf("[restore] failed to remove backup.zip: %v", rmErr)
} else {
log.Printf("[restore] backup.zip removed")
}
// 8. 删除标记
if rmErr := os.Remove("./data/komari-backup-markup"); rmErr != nil {
log.Printf("[restore] failed to remove komari-backup-markup: %v", rmErr)
} else {
log.Printf("[restore] komari-backup-markup removed")
}
}
}()
// 记录“打开数据库之前”komari.db 是否已存在,用于区分全新安装与旧版升级。
// 必须在(可能的)恢复逻辑之后、gorm.Open 之前采集:恢复会解压出旧库,
// gorm.Open 会创建空库。
if _, statErr := os.Stat(resolveDatabaseFile()); statErr == nil {
dbFileExistedAtStartup = true
}
logConfig := &gorm.Config{
Logger: logutil.NewGormLogger(),
NowFunc: func() time.Time { return time.Now().UTC() },
}
// 根据数据库类型选择不同的连接方式
switch flags.ApplyDatabaseTypeNormalization() {
case flags.DatabaseTypeSQLite:
// SQLite 连接
// 通过 DSN 传入 _busy_timeout / _txlock 等参数,确保连接池中的每一条连接
// 都生效:
// - _busy_timeout=5000:遇到写锁时最多等待 5s 再返回,避免瞬时
// "database is locked" 直接失败(仅靠后续 PRAGMA Exec 只作用于
// 当时执行该语句的单条连接,池内其它连接不生效)。
// - _txlock=immediate:事务一开始即获取写锁,避免「先 SELECT 后写」
// 的锁升级在并发写入下产生死锁式的立即 SQLITE_BUSY。
// - _journal_mode=WAL / _synchronous=NORMAL:与下方 PRAGMA 保持一致,
// 在 DSN 层为所有连接预设。
dsn := buildSQLiteDSN(flags.DatabaseFile)
instance, err = gorm.Open(sqlite.Open(dsn), logConfig)
if err != nil {
return fmt.Errorf("failed to connect to SQLite3 database: %w", err)
}
log.Printf("Using SQLite database file: %s", flags.DatabaseFile)
if sqlDB, dbErr := instance.DB(); dbErr == nil {
// SQLite 同一时刻只允许一个写者;限制连接数可避免连接池层面的写写竞争。
// 负载历史每分钟会执行包含读和写的事务,若连接池允许多个连接,容易与
// ping 结果等短写入撞锁并导致整批负载记录回滚。
sqlDB.SetMaxOpenConns(1)
sqlDB.SetMaxIdleConns(1)
sqlDB.SetConnMaxLifetime(0)
} else {
log.Printf("Failed to access underlying sql.DB for SQLite tuning: %v", dbErr)
}
instance.Exec("PRAGMA wal = ON;")
if err := instance.Exec("PRAGMA journal_mode = WAL;").Error; err != nil {
log.Printf("Failed to enable WAL mode for SQLite: %v", err)
}
instance.Exec("PRAGMA synchronous = NORMAL;")
instance.Exec("PRAGMA busy_timeout = 5000;")
instance.Exec("PRAGMA cache_size = -65536;")
instance.Exec("PRAGMA temp_store = MEMORY;")
instance.Exec("PRAGMA wal_checkpoint(TRUNCATE);")
default:
return fmt.Errorf("unsupported database type: %s (supported: %s)", flags.DatabaseType, flags.SupportedDatabaseTypes())
}
if err := migrations.Run(migrations.Context{DB: instance}); err != nil {
return fmt.Errorf("failed to run startup migrations: %w", err)
}
config.SetDb(instance)
// 配置库就绪后、执行后续 AutoMigrate 与一次性 metrics 迁移之前:
// 基于配置中的版本标记检测升级并自动备份 ./data,便于回滚。
backupOnVersionUpgrade()
// 自动迁移模型
//
// 注意:负载/GPU/ping 历史监控数据运行期全部走 metric store(默认 SQLite
// ./data/metrics.db,或配置的 MySQL/PostgreSQL)。旧的 records /
// records_long_term / gpu_records / ping_records 表不再建表、不再写入。
// 若升级时旧表仍存在,会在 pkg/migrations.RunMetricStoreMigrations 中先导入再清理。
// models.Record / models.PingRecord / models.GPURecord 结构体仍作为
// metric store 的读写 DTO 和旧表导入 DTO 保留在 models 包中。
err = instance.AutoMigrate(
&models.User{},
&models.Client{},
&models.Log{},
&models.Clipboard{},
&models.LoadNotification{},
&models.OfflineNotification{},
&models.TrafficReportNotification{},
&models.PingTask{},
&models.OidcProvider{},
&models.MessageSenderProvider{},
&models.ThemeConfiguration{},
)
if err != nil {
return fmt.Errorf("failed to create tables: %w", err)
}
if err := instance.AutoMigrate(
&models.Session{},
); err != nil {
log.Printf("Failed to create Session table, it may already exist: %v", err)
}
if err := instance.AutoMigrate(
&models.Task{},
&models.TaskResult{},
); err != nil {
log.Printf("Failed to create Task and TaskResult table, it may already exist: %v", err)
}
return nil
}
+92
View File
@@ -0,0 +1,92 @@
package dbcore
import (
"context"
"database/sql"
"errors"
"fmt"
"os"
"strings"
"sync"
"github.com/komari-monitor/komari/cmd/flags"
)
var maintenanceMu sync.Mutex
// StorageSize returns the bytes occupied by the main SQLite database and its
// WAL/SHM sidecar files.
func StorageSize() (int64, error) {
if !flags.IsSQLite() {
return 0, errors.New("main database size is only available for SQLite")
}
return sqliteFileSetSize(resolveDatabaseFile())
}
// ReclaimSpace checkpoints the main database WAL and rewrites the SQLite file.
func ReclaimSpace(ctx context.Context) error {
if !flags.IsSQLite() {
return errors.New("main database maintenance is only supported for SQLite")
}
maintenanceMu.Lock()
defer maintenanceMu.Unlock()
db, err := GetDBInstance().DB()
if err != nil {
return fmt.Errorf("get main database connection: %w", err)
}
if err := checkpointSQLiteWAL(ctx, db); err != nil {
return err
}
if _, err := db.ExecContext(ctx, "VACUUM"); err != nil {
return fmt.Errorf("vacuum main database: %w", err)
}
return checkpointSQLiteWAL(ctx, db)
}
func checkpointSQLiteWAL(ctx context.Context, db *sql.DB) error {
var busy, logFrames, checkpointedFrames int
if err := db.QueryRowContext(ctx, "PRAGMA wal_checkpoint(TRUNCATE)").Scan(&busy, &logFrames, &checkpointedFrames); err != nil {
return fmt.Errorf("checkpoint main database WAL: %w", err)
}
if busy != 0 {
return fmt.Errorf(
"checkpoint main database WAL: database is busy (%d log frames, %d checkpointed)",
logFrames,
checkpointedFrames,
)
}
return nil
}
func sqliteFileSetSize(dsn string) (int64, error) {
path := strings.TrimPrefix(strings.TrimSpace(dsn), "file:")
if index := strings.IndexByte(path, '?'); index >= 0 {
path = path[:index]
}
if path == "" || path == ":memory:" || strings.Contains(strings.ToLower(dsn), "mode=memory") {
return 0, errors.New("database is not backed by a local file")
}
var total int64
foundDatabase := false
for _, suffix := range []string{"", "-wal", "-shm"} {
info, err := os.Stat(path + suffix)
switch {
case err == nil:
total += info.Size()
if suffix == "" {
foundDatabase = true
}
case errors.Is(err, os.ErrNotExist):
continue
default:
return 0, fmt.Errorf("stat database file %q: %w", path+suffix, err)
}
}
if !foundDatabase {
return 0, fmt.Errorf("database file %q does not exist", path)
}
return total, nil
}
+37
View File
@@ -0,0 +1,37 @@
package dbcore
import (
"os"
"path/filepath"
"testing"
)
func TestSQLiteFileSetSizeIncludesWALAndSHM(t *testing.T) {
databasePath := filepath.Join(t.TempDir(), "komari.db")
files := map[string]int{
databasePath: 11,
databasePath + "-wal": 7,
databasePath + "-shm": 5,
}
for path, size := range files {
if err := os.WriteFile(path, make([]byte, size), 0o600); err != nil {
t.Fatalf("write fixture %s: %v", path, err)
}
}
got, err := sqliteFileSetSize("file:" + filepath.ToSlash(databasePath) + "?mode=rwc")
if err != nil {
t.Fatalf("sqliteFileSetSize: %v", err)
}
if got != 23 {
t.Fatalf("size = %d, want 23", got)
}
}
func TestSQLiteFileSetSizeRejectsNonFileDatabase(t *testing.T) {
for _, dsn := range []string{":memory:", "file:metrics?MODE=MEMORY&cache=shared"} {
if _, err := sqliteFileSetSize(dsn); err == nil {
t.Fatalf("sqliteFileSetSize(%q) should fail", dsn)
}
}
}
+95
View File
@@ -0,0 +1,95 @@
package metricstore
import (
"context"
"errors"
"fmt"
"github.com/komari-monitor/komari/pkg/metric"
)
var (
ErrStoreNotInitialized = errors.New("metric store not initialized")
ErrStoreBusy = errors.New("metric store is busy")
)
// StorageInfo describes the physical storage owned by the active metric store.
// Size remains useful even when the store points at an external database because
// pkg/metric limits its query to the three tables managed by this Store.
type StorageInfo struct {
Driver metric.Driver
Action metric.MaintenanceAction
Size int64
}
// MaintenanceResult keeps measurement failures separate from the maintenance
// error. A database may allow table maintenance while denying catalog queries,
// and callers should still be able to report that the operation succeeded.
type MaintenanceResult struct {
Driver metric.Driver
Action metric.MaintenanceAction
Before int64
After int64
BeforeSizeError error
AfterSizeError error
}
// InspectStorage reads physical storage information while preventing a store
// reload from closing the active connection underneath the query.
func InspectStorage(ctx context.Context) (StorageInfo, error) {
if err := storeOperations.Acquire(ctx); err != nil {
return StorageInfo{}, fmt.Errorf("wait for metric store operations before inspection: %w", err)
}
defer storeOperations.Release()
storeMu.RLock()
activeStore := store
storeMu.RUnlock()
if activeStore == nil {
return StorageInfo{}, ErrStoreNotInitialized
}
info := StorageInfo{
Driver: activeStore.Driver(),
Action: activeStore.MaintenanceAction(),
}
size, err := activeStore.StorageSize(ctx)
info.Size = size
return info, err
}
// ReclaimSpace performs the driver-specific physical maintenance operation.
// It shares the exclusive operation lock with Compact so retention compaction
// and a table/file rewrite can never run against the metric store concurrently.
func ReclaimSpace(ctx context.Context) (MaintenanceResult, error) {
if !storeOperations.TryAcquire() {
storeMu.RLock()
defer storeMu.RUnlock()
if store == nil {
return MaintenanceResult{}, ErrStoreNotInitialized
}
return MaintenanceResult{
Driver: store.Driver(),
Action: store.MaintenanceAction(),
BeforeSizeError: ErrStoreBusy,
AfterSizeError: ErrStoreBusy,
}, ErrStoreBusy
}
defer storeOperations.Release()
storeMu.RLock()
activeStore := store
storeMu.RUnlock()
if activeStore == nil {
return MaintenanceResult{}, ErrStoreNotInitialized
}
result := MaintenanceResult{
Driver: activeStore.Driver(),
Action: activeStore.MaintenanceAction(),
}
result.Before, result.BeforeSizeError = activeStore.StorageSize(ctx)
maintenanceErr := activeStore.ReclaimSpace(ctx)
result.After, result.AfterSizeError = activeStore.StorageSize(ctx)
return result, maintenanceErr
}
+167
View File
@@ -0,0 +1,167 @@
package metricstore
import (
"context"
"errors"
"testing"
"time"
"github.com/komari-monitor/komari/pkg/metric"
)
func TestInspectAndReclaimStorage(t *testing.T) {
ctx := context.Background()
s, err := metric.Open(ctx, metric.SQLite(":memory:"))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
installTestStore(t, s)
info, err := InspectStorage(ctx)
if err != nil {
t.Fatalf("inspect storage: %v", err)
}
if info.Driver != metric.DriverSQLite || info.Action != metric.MaintenanceVacuum {
t.Fatalf("unexpected storage info: %#v", info)
}
if info.Size != 0 {
t.Fatalf("in-memory storage size = %d, want 0", info.Size)
}
result, err := ReclaimSpace(ctx)
if err != nil {
t.Fatalf("reclaim space: %v", err)
}
if result.Driver != metric.DriverSQLite || result.Action != metric.MaintenanceVacuum {
t.Fatalf("unexpected maintenance result: %#v", result)
}
if result.BeforeSizeError != nil || result.AfterSizeError != nil {
t.Fatalf("unexpected size errors: before=%v after=%v", result.BeforeSizeError, result.AfterSizeError)
}
}
func TestReclaimSpaceReportsBusyStore(t *testing.T) {
s, err := metric.Open(context.Background(), metric.SQLite(":memory:"))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
installTestStore(t, s)
if !storeOperations.TryAcquire() {
t.Fatal("acquire store operation gate")
}
defer storeOperations.Release()
result, err := ReclaimSpace(context.Background())
if !errors.Is(err, ErrStoreBusy) {
t.Fatalf("reclaim error = %v, want %v", err, ErrStoreBusy)
}
if result.Driver != metric.DriverSQLite || result.Action != metric.MaintenanceVacuum {
t.Fatalf("busy result lost store metadata: %#v", result)
}
if !errors.Is(result.BeforeSizeError, ErrStoreBusy) || !errors.Is(result.AfterSizeError, ErrStoreBusy) {
t.Fatalf("busy result should mark both measurements unavailable: %#v", result)
}
if _, err := Compact(context.Background(), time.Now()); !errors.Is(err, ErrCompactInProgress) {
t.Fatalf("compact error = %v, want %v", err, ErrCompactInProgress)
}
}
func TestInspectStorageRequiresInitializedStore(t *testing.T) {
storeMu.Lock()
previous := store
store = nil
storeMu.Unlock()
t.Cleanup(func() {
storeMu.Lock()
store = previous
storeMu.Unlock()
})
if _, err := InspectStorage(context.Background()); !errors.Is(err, ErrStoreNotInitialized) {
t.Fatalf("inspect error = %v, want %v", err, ErrStoreNotInitialized)
}
}
func TestCloseStoreContextCancelsMigrationBeforeTakingStoreLock(t *testing.T) {
s, err := metric.Open(context.Background(), metric.SQLite(":memory:"))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
installTestStore(t, s)
migrationCtx, cancelMigration := context.WithCancel(context.Background())
done := make(chan struct{})
if !storeOperations.TryAcquire() {
t.Fatal("acquire store operation gate")
}
storeMigMu.Lock()
storeMigCancel = cancelMigration
storeMigDone = done
storeMigMu.Unlock()
go func() {
<-migrationCtx.Done()
storeOperations.Release()
storeMigMu.Lock()
storeMigCancel = nil
storeMigDone = nil
close(done)
storeMigMu.Unlock()
}()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := CloseStoreContext(ctx); err != nil {
t.Fatalf("close store: %v", err)
}
if migrationCtx.Err() == nil {
t.Fatal("store migration was not canceled before close")
}
if err := Reload(context.Background()); !errors.Is(err, ErrStoreBusy) {
t.Fatalf("reload after close error = %v, want %v", err, ErrStoreBusy)
}
}
func TestStoreOperationWaitsRespectContext(t *testing.T) {
if !storeOperations.TryAcquire() {
t.Fatal("acquire store operation gate")
}
defer storeOperations.Release()
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err := InspectStorage(ctx); !errors.Is(err, context.Canceled) {
t.Fatalf("inspect error = %v, want context canceled", err)
}
if err := Reload(ctx); !errors.Is(err, context.Canceled) {
t.Fatalf("reload error = %v, want context canceled", err)
}
if err := CloseStoreContext(ctx); !errors.Is(err, context.Canceled) {
t.Fatalf("close error = %v, want context canceled", err)
}
}
func installTestStore(t *testing.T, s *metric.Store) {
t.Helper()
storeMigMu.Lock()
previousClosing := storeClosing
storeClosing = false
storeMigMu.Unlock()
storeMu.Lock()
previous := store
previousFingerprint := storeFingerprint
store = s
storeFingerprint = "test|memory"
storeMu.Unlock()
t.Cleanup(func() {
storeMigMu.Lock()
storeClosing = previousClosing
storeMigMu.Unlock()
storeMu.Lock()
store = previous
storeFingerprint = previousFingerprint
storeMu.Unlock()
_ = s.Close()
})
}
+61
View File
@@ -0,0 +1,61 @@
package metricstore
const (
MetricCPU = "cpu.usage"
MetricGPU = "gpu.usage"
MetricGPUDeviceUsage = "gpu.device.usage"
MetricGPUMem = "gpu.memory.used"
MetricGPUMemTotal = "gpu.memory.total"
MetricGPUTemp = "gpu.temperature"
MetricRAM = "memory.used"
MetricRAMTotal = "memory.total"
MetricSwap = "swap.used"
MetricSwapTotal = "swap.total"
MetricLoad = "load.average"
MetricTemp = "temperature"
MetricDisk = "disk.used"
MetricDiskTotal = "disk.total"
MetricNetIn = "net.in.rate"
MetricNetOut = "net.out.rate"
MetricNetTotalUp = "net.total.up"
MetricNetTotalDown = "net.total.down"
MetricTrafficUp = "traffic.up"
MetricTrafficDown = "traffic.down"
MetricProcess = "process.count"
MetricConnections = "connections.tcp"
MetricConnectionsUDP = "connections.udp"
MetricPingLatency = "ping.latency_ms"
MetricPingLoss = "ping.loss"
)
// loadRecordMetricNames are the entity-level metrics used to reconstruct the
// legacy Record response shape.
var loadRecordMetricNames = []string{
MetricCPU, MetricGPU, MetricRAM, MetricRAMTotal, MetricSwap, MetricSwapTotal,
MetricLoad, MetricTemp, MetricDisk, MetricDiskTotal, MetricNetIn, MetricNetOut,
MetricNetTotalUp, MetricNetTotalDown, MetricTrafficUp, MetricTrafficDown,
MetricProcess, MetricConnections, MetricConnectionsUDP,
}
// gpuDeviceRecordMetricNames are stored separately from the entity-level GPU
// average and are included when deleting all system records.
var gpuDeviceRecordMetricNames = []string{
MetricGPUDeviceUsage, MetricGPUMem, MetricGPUMemTotal, MetricGPUTemp,
}
var recordMetricNames = joinMetricNames(loadRecordMetricNames, gpuDeviceRecordMetricNames)
// Ping has an independent retention and cleanup boundary.
var pingMetricNames = []string{MetricPingLatency, MetricPingLoss}
func joinMetricNames(groups ...[]string) []string {
total := 0
for _, group := range groups {
total += len(group)
}
names := make([]string, 0, total)
for _, group := range groups {
names = append(names, group...)
}
return names
}
+999
View File
@@ -0,0 +1,999 @@
package metricstore
import (
"context"
"errors"
"fmt"
"log"
"sort"
"strings"
"sync"
"time"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/config"
"github.com/komari-monitor/komari/pkg/metric"
)
var (
store *metric.Store
storeFingerprint string
storeMu sync.RWMutex
storeOnce sync.Once
storeOperations = newStoreOperationGate()
compactAt int
)
var ErrCompactInProgress = errors.New("metric store compact already in progress")
const (
// DefaultRollupRawRetention keeps a short hot raw window; older samples are
// served from rollups after compaction.
DefaultRollupRawRetention = 15 * time.Minute
DefaultRollupFinestTier = time.Minute
)
// MetricStoreConfig 保存 metric store 配置。
//
// 注意:metric store 现在始终启用(旧的 metric_store_enabled 开关已废弃)。
// 未显式配置时默认使用 SQLite(./data/metrics.db)。
type MetricStoreConfig struct {
Driver string `json:"metric_db_driver" default:"sqlite"` // 数据库类型: sqlite, mysql, postgresql
DSN string `json:"metric_db_dsn" default:"./data/metrics.db"` // 数据库连接串
DownsamplingEnabled bool `json:"metric_downsampling_enabled" default:"false"` // 是否启用分层降采样
TablePrefix string `json:"metric_table_prefix" default:"metric_"` // 表名前缀
MaxOpenConns int `json:"metric_max_open_conns" default:"25"` // 最大连接数
MaxIdleConns int `json:"metric_max_idle_conns" default:"5"` // 最大空闲连接数
}
// MetricStoreConfigKeys 配置键
//
// MetricStoreEnabledKey 已废弃:metric store 始终启用,保留常量仅用于清理旧配置。
const (
MetricStoreEnabledKey = "metric_store_enabled" // Deprecated: metric store 始终启用
MetricDBDriverKey = "metric_db_driver"
MetricDBDSNKey = "metric_db_dsn"
MetricDownsamplingEnabledKey = "metric_downsampling_enabled"
MetricTablePrefixKey = "metric_table_prefix"
MetricMaxOpenConnsKey = "metric_max_open_conns"
MetricMaxIdleConnsKey = "metric_max_idle_conns"
// MigrationTargetKey 记录上一次成功完成启动迁移的目标指纹(driver+dsn)。
// 当目标发生变化(例如从 SQLite 切换到 MySQL/PostgreSQL)时,启动迁移会
// 重新执行,把上一个目标库的数据搬运到新的目标 metrics 库。
MigrationTargetKey = "metric_migration_target"
)
// buildMetricConfig 根据 MetricStoreConfig 构造底层 metric.Config。
// autoMigrate 控制是否在 Open 时自动建表:正式初始化/热加载时为 true,
// 仅做连接测试时为 false(不写入 schema,避免对目标库产生副作用)。
func buildMetricConfig(cfg *MetricStoreConfig, autoMigrate bool) (metric.Config, error) {
driver := ResolveDriverFromConfig(cfg.Driver, cfg.DSN)
tablePrefix := cfg.TablePrefix
if tablePrefix == "" {
tablePrefix = "metric_"
}
opts := []metric.Option{
metric.WithTablePrefix(tablePrefix),
metric.WithAutoMigrate(autoMigrate),
}
if cfg.DownsamplingEnabled {
opts = append(opts, metric.WithRollupPolicy(defaultRollupPolicy()))
}
switch driver {
case metric.DriverSQLite:
dsn := cfg.DSN
if dsn == "" || dsn == "./data/metrics.db" {
// 注意:刻意不使用 cache=shared。SQLite 共享缓存模式使用表级锁,
// 当一个连接持有读锁、另一个连接尝试写入时会立即返回
// SQLITE_LOCKED("database table is locked"),且 busy_timeout
// 对共享缓存的表级锁无效,迁移期间与前台查询/实时写入并发时必然报错。
// _txlock=immediate 让写事务开始即获取写锁,避免锁升级死锁。
dsn = "file:./data/metrics.db?mode=rwc&_txlock=immediate"
} else {
// 用户自定义 DSN 时,剥离 cache=shared,避免上述表级锁问题。
dsn = stripSharedCache(dsn)
}
// SQLite 串行化写入:固定单写连接以避免 "database is locked" 竞争,
// 同时启用独立的 WAL 只读连接池提升前台查询并发(写仍走单主连接)。
// 这里刻意忽略 cfg.MaxOpenConns/MaxIdleConns —— 对 SQLite 而言多写连接
// 只会引入锁竞争而非提升吞吐。
opts = append(opts,
metric.WithMaxOpenConns(1),
metric.WithMaxIdleConns(1),
metric.WithSQLiteReadPool(4),
)
return metric.SQLite(dsn, opts...), nil
case metric.DriverMySQL:
opts = append(opts,
metric.WithMaxOpenConns(cfg.MaxOpenConns),
metric.WithMaxIdleConns(cfg.MaxIdleConns),
)
return metric.MySQL(cfg.DSN, opts...), nil
case metric.DriverPostgreSQL:
opts = append(opts,
metric.WithMaxOpenConns(cfg.MaxOpenConns),
metric.WithMaxIdleConns(cfg.MaxIdleConns),
)
return metric.PostgreSQL(cfg.DSN, opts...), nil
default:
return metric.Config{}, fmt.Errorf("unsupported metric database driver: %s", cfg.Driver)
}
}
func defaultRollupPolicy() metric.RollupPolicy {
return metric.RollupPolicy{
RawRetention: DefaultRollupRawRetention,
Tiers: []metric.RollupTier{
{Interval: DefaultRollupFinestTier, Retention: 48 * time.Hour},
{Interval: 5 * time.Minute, Retention: 14 * 24 * time.Hour},
{Interval: time.Hour, Retention: 14 * 24 * time.Hour},
},
}
}
// ResolveDriverFromConfig 根据 DSN 自动推断 metrics 数据库类型;当 DSN 不能可靠
// 识别时回退到旧配置中的 driver,以兼容已有配置和非常规 DSN。
func ResolveDriverFromConfig(configuredDriver, dsn string) metric.Driver {
if driver, ok := InferDriverFromDSN(dsn); ok {
return driver
}
switch driver := metric.Driver(strings.ToLower(strings.TrimSpace(configuredDriver))); driver {
case metric.DriverSQLite, metric.DriverMySQL, metric.DriverPostgreSQL:
return driver
default:
return metric.DriverSQLite
}
}
// InferDriverFromDSN 尽量根据常见 DSN 格式推断数据库类型。
// 返回 ok=false 表示格式不够明确,调用方应使用已有配置作为兜底。
func InferDriverFromDSN(dsn string) (metric.Driver, bool) {
raw := strings.TrimSpace(dsn)
if raw == "" {
return metric.DriverSQLite, true
}
lower := strings.ToLower(raw)
// PostgreSQL URL DSN: postgres://... 或 postgresql://...
if strings.HasPrefix(lower, "postgres://") || strings.HasPrefix(lower, "postgresql://") {
return metric.DriverPostgreSQL, true
}
// SQLite 常见文件/内存 DSN。
if raw == ":memory:" || strings.HasPrefix(lower, "file:") || strings.HasPrefix(lower, "sqlite://") || strings.HasPrefix(lower, "sqlite3://") {
return metric.DriverSQLite, true
}
// MySQL URL(虽然 go-sql-driver/mysql 原生 DSN 通常不是 URL,但这里用于给出
// 类型推断;连接测试仍会校验 DSN 是否可被驱动接受)。
if strings.HasPrefix(lower, "mysql://") {
return metric.DriverMySQL, true
}
// PostgreSQL 关键字/值 DSN: host=... user=... dbname=...
if looksLikePostgreSQLKeyValueDSN(lower) {
return metric.DriverPostgreSQL, true
}
// go-sql-driver/mysql DSN: user:pass@tcp(host:3306)/db、user@unix(...)/db、user:pass@/db 等。
if looksLikeMySQLDSN(lower) {
return metric.DriverMySQL, true
}
// SQLite 路径:./data/metrics.db、/var/lib/metrics.sqlite3、metrics.sqlite 等。
if looksLikeSQLitePath(lower) {
return metric.DriverSQLite, true
}
return "", false
}
func looksLikePostgreSQLKeyValueDSN(lower string) bool {
if !strings.Contains(lower, "=") || strings.Contains(lower, "://") {
return false
}
keys := []string{"host=", "user=", "password=", "dbname=", "port=", "sslmode="}
matched := 0
for _, key := range keys {
if strings.Contains(lower, key) {
matched++
}
}
// dbname= 基本是 PostgreSQL libpq DSN 的强特征;否则至少匹配两个常见键。
return strings.Contains(lower, "dbname=") || matched >= 2
}
func looksLikeMySQLDSN(lower string) bool {
if strings.Contains(lower, "://") || strings.Contains(lower, " ") {
return false
}
if strings.Contains(lower, "@tcp(") || strings.Contains(lower, "@unix(") || strings.Contains(lower, "@/") {
return true
}
// user:pass@host/db、user@host/db 这类虽然不是推荐格式,但也明显偏 MySQL。
return strings.Contains(lower, "@") && strings.Contains(lower, "/")
}
func looksLikeSQLitePath(lower string) bool {
path := lower
if idx := strings.IndexAny(path, "?"); idx >= 0 {
path = path[:idx]
}
return strings.HasSuffix(path, ".db") || strings.HasSuffix(path, ".sqlite") || strings.HasSuffix(path, ".sqlite3")
}
// stripSharedCache 从 SQLite DSN 中移除 cache=shared 参数,避免共享缓存模式下的
// 表级锁(SQLITE_LOCKED "database table is locked")。其它参数保持不变。
func stripSharedCache(dsn string) string {
if !strings.Contains(dsn, "cache=shared") {
return dsn
}
idx := strings.Index(dsn, "?")
if idx < 0 {
return dsn
}
base := dsn[:idx]
query := dsn[idx+1:]
parts := strings.Split(query, "&")
kept := parts[:0]
for _, p := range parts {
if p == "cache=shared" {
continue
}
kept = append(kept, p)
}
if len(kept) == 0 {
return base
}
return base + "?" + strings.Join(kept, "&")
}
// openStore 按配置打开 metric store 并创建指标定义。
func openStore(ctx context.Context, cfg *MetricStoreConfig) (*metric.Store, error) {
return openStoreWithDefaultRetention(ctx, cfg, defaultBuiltinMetricRetentionDays)
}
func openStoreWithDefaultRetention(ctx context.Context, cfg *MetricStoreConfig, defaultRetentionDays int) (*metric.Store, error) {
metricCfg, err := buildMetricConfig(cfg, true)
if err != nil {
return nil, err
}
s, err := metric.Open(ctx, metricCfg)
if err != nil {
return nil, fmt.Errorf("failed to open metric store: %w", err)
}
if err := createMetricDefinitionsWithDefaultRetention(ctx, s, defaultRetentionDays); err != nil {
s.Close()
return nil, fmt.Errorf("failed to create metric definitions: %w", err)
}
return s, nil
}
// OpenStore opens an isolated metric store using the supplied configuration.
// It is used by the pre-start upgrade flow before the process-wide store is
// initialized. The caller owns the returned store and must close it.
func OpenStore(ctx context.Context, cfg *MetricStoreConfig) (*metric.Store, error) {
return openStore(ctx, cfg)
}
// OpenStoreForMigration opens an isolated target and uses the legacy data span
// as the initial retention for definitions that do not exist yet. Existing
// definitions keep their configured retention, including an explicit zero.
func OpenStoreForMigration(ctx context.Context, cfg *MetricStoreConfig, legacyRetentionDays int) (*metric.Store, error) {
if legacyRetentionDays < defaultBuiltinMetricRetentionDays {
legacyRetentionDays = defaultBuiltinMetricRetentionDays
}
return openStoreWithDefaultRetention(ctx, cfg, legacyRetentionDays)
}
// TestConnection 使用给定配置尝试连接 metrics 数据库(不影响当前运行的 store)。
// 仅打开连接并 Ping,不执行自动建表,连接成功后立即关闭。失败时返回可读错误。
func TestConnection(ctx context.Context, cfg *MetricStoreConfig) error {
metricCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return err
}
s, err := metric.Open(ctx, metricCfg)
if err != nil {
return err
}
defer s.Close()
return s.Ping(ctx)
}
// InitializeStore 初始化 metric store(启动时调用,仅执行一次)。
func InitializeStore() error {
var initErr error
storeOnce.Do(func() {
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
initErr = fmt.Errorf("failed to load metric store config: %w", err)
return
}
// metric store 始终启用;未配置时默认 SQLite(./data/metrics.db)。
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
s, err := openStore(ctx, cfg)
if err != nil {
initErr = err
return
}
storeMu.Lock()
store = s
storeFingerprint = targetFingerprint(cfg)
storeMu.Unlock()
clearStoreClosing()
log.Printf("Metric store initialized successfully (driver=%s)", ResolveDriverFromConfig(cfg.Driver, cfg.DSN))
})
return initErr
}
// Reload 根据最新配置热重载 metric store,无需重启进程。
// metric store 始终启用:用新配置打开并建表(内部已 Ping 校验连接),
// 成功后再替换运行中的 store,最后关闭旧实例。任何失败都会保留旧 store 不变。
//
// 注意:Reload 只切换运行中的连接,不会把旧目标(如 SQLite)中的历史数据
// 搬运到新目标(如 MySQL/PostgreSQL)。跨库数据迁移由启动迁移
// (RunStartupMigration)在下次启动时按目标指纹自动完成。
func Reload(ctx context.Context) error {
if err := storeOperations.Acquire(ctx); err != nil {
return fmt.Errorf("wait for metric store operations before reload: %w", err)
}
defer storeOperations.Release()
if isStoreClosing() {
return ErrStoreBusy
}
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
return fmt.Errorf("failed to load metric store config: %w", err)
}
// 用新配置打开并建表(内部已 Ping 校验连接)。
s, err := openStore(ctx, cfg)
if err != nil {
return err
}
storeMu.Lock()
old := store
store = s
storeFingerprint = targetFingerprint(cfg)
storeMu.Unlock()
if old != nil {
if cerr := old.Close(); cerr != nil {
log.Printf("Failed to close previous metric store on reload: %v", cerr)
}
}
log.Printf("Metric store reloaded successfully (driver=%s)", ResolveDriverFromConfig(cfg.Driver, cfg.DSN))
return nil
}
// GetStore 获取 metric store 实例(如果未启用返回 nil)
func GetStore() *metric.Store {
storeMu.RLock()
defer storeMu.RUnlock()
return store
}
// RetentionSummary is the compatibility view of all persisted metric policies.
type RetentionSummary struct {
AllPositive bool
MaxDays int
}
// GetRetentionSummary aggregates the active store's metric definitions. An
// empty definition set is not considered record-enabled.
func GetRetentionSummary(ctx context.Context) (RetentionSummary, error) {
s := GetStore()
if s == nil {
return RetentionSummary{}, fmt.Errorf("metric store not initialized")
}
defs, err := s.ListMetrics(ctx)
if err != nil {
return RetentionSummary{}, err
}
return summarizeRetentionDefinitions(defs), nil
}
func summarizeRetentionDefinitions(defs []metric.Definition) RetentionSummary {
if len(defs) == 0 {
return RetentionSummary{}
}
summary := RetentionSummary{AllPositive: true}
for _, def := range defs {
if def.RetentionDays <= 0 {
summary.AllPositive = false
}
if def.RetentionDays > summary.MaxDays {
summary.MaxDays = def.RetentionDays
}
}
return summary
}
func Compact(ctx context.Context, now time.Time) (int, error) {
if !storeOperations.TryAcquire() {
return 0, ErrCompactInProgress
}
defer storeOperations.Release()
storeMu.RLock()
activeStore := store
storeMu.RUnlock()
if activeStore == nil {
return 0, fmt.Errorf("metric store not initialized")
}
defs, err := activeStore.ListMetrics(ctx)
if err != nil {
return 0, err
}
if len(defs) == 0 {
compactAt = 0
return 0, nil
}
if compactAt >= len(defs) {
compactAt = 0
}
total := 0
start := compactAt
for i := 0; i < len(defs); i++ {
idx := (start + i) % len(defs)
n, err := activeStore.CompactMetric(ctx, defs[idx].Name, now)
if err != nil {
compactAt = idx
return total, fmt.Errorf("compact metric %q: %w", defs[idx].Name, err)
}
total += n
compactAt = (idx + 1) % len(defs)
}
if _, err := activeStore.CleanupExpired(ctx, now); err != nil {
return total, fmt.Errorf("clean up expired raw metrics: %w", err)
}
return total, nil
}
// CloseStoreContext stops the asynchronous store migration before taking the
// store write lock, so shutdown cannot wait forever on the migration's lease.
func CloseStoreContext(ctx context.Context) error {
if err := stopStoreMigrationForClose(ctx); err != nil {
clearStoreClosing()
return err
}
if err := storeOperations.Acquire(ctx); err != nil {
clearStoreClosing()
return fmt.Errorf("wait for metric store operations before close: %w", err)
}
defer storeOperations.Release()
storeMu.Lock()
defer storeMu.Unlock()
if store != nil {
err := store.Close()
store = nil
storeFingerprint = ""
return err
}
storeFingerprint = ""
return nil
}
const defaultBuiltinMetricRetentionDays = 1
// createMetricDefinitions creates built-in definitions with explicit policies.
func createMetricDefinitions(ctx context.Context, s *metric.Store) error {
return createMetricDefinitionsWithDefaultRetention(ctx, s, defaultBuiltinMetricRetentionDays)
}
func createMetricDefinitionsWithDefaultRetention(ctx context.Context, s *metric.Store, defaultRetentionDays int) error {
if defaultRetentionDays < defaultBuiltinMetricRetentionDays {
defaultRetentionDays = defaultBuiltinMetricRetentionDays
}
definitions := []metric.Definition{
{Name: MetricCPU, Type: metric.TypeGauge, Unit: "%", Description: "CPU usage percentage", RetentionDays: defaultRetentionDays},
{Name: MetricGPU, Type: metric.TypeGauge, Unit: "%", Description: "GPU usage percentage", RetentionDays: defaultRetentionDays},
{Name: MetricGPUDeviceUsage, Type: metric.TypeGauge, Unit: "%", Description: "Per-device GPU utilization", RetentionDays: defaultRetentionDays},
{Name: MetricGPUMem, Type: metric.TypeGauge, Unit: "bytes", Description: "GPU memory used", RetentionDays: defaultRetentionDays},
{Name: MetricGPUMemTotal, Type: metric.TypeGauge, Unit: "bytes", Description: "GPU memory total", RetentionDays: defaultRetentionDays},
{Name: MetricGPUTemp, Type: metric.TypeGauge, Unit: "°C", Description: "GPU temperature", RetentionDays: defaultRetentionDays},
{Name: MetricRAM, Type: metric.TypeGauge, Unit: "bytes", Description: "RAM used", RetentionDays: defaultRetentionDays},
{Name: MetricRAMTotal, Type: metric.TypeGauge, Unit: "bytes", Description: "RAM total", RetentionDays: defaultRetentionDays},
{Name: MetricSwap, Type: metric.TypeGauge, Unit: "bytes", Description: "Swap used", RetentionDays: defaultRetentionDays},
{Name: MetricSwapTotal, Type: metric.TypeGauge, Unit: "bytes", Description: "Swap total", RetentionDays: defaultRetentionDays},
{Name: MetricLoad, Type: metric.TypeGauge, Unit: "", Description: "System load average", RetentionDays: defaultRetentionDays},
{Name: MetricTemp, Type: metric.TypeGauge, Unit: "°C", Description: "System temperature", RetentionDays: defaultRetentionDays},
{Name: MetricDisk, Type: metric.TypeGauge, Unit: "bytes", Description: "Disk used", RetentionDays: defaultRetentionDays},
{Name: MetricDiskTotal, Type: metric.TypeGauge, Unit: "bytes", Description: "Disk total", RetentionDays: defaultRetentionDays},
{Name: MetricNetIn, Type: metric.TypeGauge, Unit: "bytes/s", Description: "Network in rate", RetentionDays: defaultRetentionDays},
{Name: MetricNetOut, Type: metric.TypeGauge, Unit: "bytes/s", Description: "Network out rate", RetentionDays: defaultRetentionDays},
{Name: MetricNetTotalUp, Type: metric.TypeCounter, Unit: "bytes", Description: "Network total upload", RetentionDays: defaultRetentionDays},
{Name: MetricNetTotalDown, Type: metric.TypeCounter, Unit: "bytes", Description: "Network total download", RetentionDays: defaultRetentionDays},
{Name: MetricTrafficUp, Type: metric.TypeGauge, Unit: "bytes", Description: "Traffic upload delta", RetentionDays: defaultRetentionDays},
{Name: MetricTrafficDown, Type: metric.TypeGauge, Unit: "bytes", Description: "Traffic download delta", RetentionDays: defaultRetentionDays},
{Name: MetricProcess, Type: metric.TypeGauge, Unit: "count", Description: "Process count", RetentionDays: defaultRetentionDays},
{Name: MetricConnections, Type: metric.TypeGauge, Unit: "count", Description: "TCP connections", RetentionDays: defaultRetentionDays},
{Name: MetricConnectionsUDP, Type: metric.TypeGauge, Unit: "count", Description: "UDP connections", RetentionDays: defaultRetentionDays},
{Name: MetricPingLatency, Type: metric.TypeGauge, Unit: "ms", Description: "Ping latency", RetentionDays: defaultRetentionDays},
{Name: MetricPingLoss, Type: metric.TypeGauge, Unit: "ratio", Description: "Ping packet loss indicator", RetentionDays: defaultRetentionDays},
}
for _, def := range definitions {
existing, err := s.GetMetric(ctx, def.Name)
if err != nil && !errors.Is(err, metric.ErrNotFound) {
return fmt.Errorf("failed to get metric %s: %w", def.Name, err)
}
if err == nil {
if existing.RetentionDays == 0 {
if _, err := s.SetMetricRetention(ctx, def.Name, 0); err != nil {
return fmt.Errorf("failed to preserve disabled metric %s: %w", def.Name, err)
}
continue
}
def.RetentionDays = existing.RetentionDays
}
if err := s.UpsertMetric(ctx, def); err != nil {
return fmt.Errorf("failed to create metric %s: %w", def.Name, err)
}
}
return nil
}
// WritePingRecord 将 ping 记录写入 metric store
func WritePingRecord(ctx context.Context, rec models.PingRecord) error {
s := GetStore()
if s == nil {
return fmt.Errorf("metric store not enabled")
}
ts := rec.Time
entityID := rec.Client
tags := map[string]string{
"task_id": fmt.Sprintf("%d", rec.TaskId),
}
loss := 0.0
if rec.Value < 0 {
loss = 1
}
points := []metric.Point{
{
MetricName: MetricPingLatency,
EntityID: entityID,
Timestamp: ts,
Value: float64(rec.Value),
Tags: tags,
},
{
MetricName: MetricPingLoss,
EntityID: entityID,
Timestamp: ts,
Value: loss,
Tags: tags,
},
}
return s.WriteBatch(ctx, points)
}
// GetRecordsByClientAndTime 从 metric store 查询记录并重构为 models.Record
func GetRecordsByClientAndTime(ctx context.Context, clientUUID string, start, end time.Time) ([]models.Record, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
return getRecordsByClientAndTimeFromSeries(ctx, s, clientUUID, start, end)
}
// GetRecordsByTime 从 metric store 查询所有客户端在时间范围内的记录
func GetRecordsByTime(ctx context.Context, start, end time.Time) ([]models.Record, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
interval := recordSeriesInterval(s, start, end, time.Now().UTC())
entityIDs, err := listRecordEntityIDs(ctx, s, start, end, interval)
if err != nil {
return nil, err
}
var records []models.Record
for _, entityID := range entityIDs {
items, err := getRecordsByClientAndTimeFromSeries(ctx, s, entityID, start, end)
if err != nil {
return nil, err
}
records = append(records, items...)
}
sortRecords(records)
return records, nil
}
type recordSeriesKey struct {
client string
ts int64
}
func getRecordsByClientAndTimeFromSeries(ctx context.Context, s *metric.Store, clientUUID string, start, end time.Time) ([]models.Record, error) {
now := time.Now().UTC()
interval := recordSeriesInterval(s, start, end, now)
recordMap := make(map[recordSeriesKey]*models.Record)
for _, metricName := range loadRecordMetricNames {
points, err := s.Series(ctx, metric.AggregateQuery{
Query: metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
},
Aggregation: recordMetricAggregation(metricName),
Interval: interval,
}, now)
if err != nil {
return nil, fmt.Errorf("failed to query metric %s: %w", metricName, err)
}
for _, point := range points {
entityID := point.EntityID
if entityID == "" {
entityID = clientUUID
}
key := recordSeriesKey{client: entityID, ts: point.Bucket.Unix()}
if recordMap[key] == nil {
recordMap[key] = &models.Record{
Client: entityID,
Time: point.Bucket.UTC(),
}
}
applyRecordMetricValue(recordMap[key], metricName, point.Value)
}
}
records := make([]models.Record, 0, len(recordMap))
for _, rec := range recordMap {
records = append(records, *rec)
}
sortRecords(records)
return records, nil
}
func recordMetricAggregation(metricName string) metric.Aggregation {
switch metricName {
case MetricTrafficUp, MetricTrafficDown:
return metric.AggSum
case MetricNetTotalUp, MetricNetTotalDown:
return metric.AggLast
default:
return metric.AggAvg
}
}
func recordSeriesInterval(s *metric.Store, start, end, now time.Time) time.Duration {
interval := recordDownsampleInterval(end.Sub(start), 500)
return s.CompatibleSeriesInterval(start, now, interval)
}
func recordDownsampleInterval(rangeDuration time.Duration, maxPoints int) time.Duration {
if maxPoints <= 0 {
maxPoints = 500
}
nanos := rangeDuration.Nanoseconds()
if nanos <= 0 {
return time.Second
}
interval := time.Duration((nanos + int64(maxPoints) - 1) / int64(maxPoints))
if interval < time.Second {
return time.Second
}
return metric.FloorStandardInterval(interval)
}
func listRecordEntityIDs(ctx context.Context, s *metric.Store, start, end time.Time, interval time.Duration) ([]string, error) {
seen := make(map[string]struct{})
for _, metricName := range loadRecordMetricNames {
ids, err := s.EntityIDs(ctx, metric.Query{
MetricName: metricName,
Start: start.Add(-interval),
End: end,
})
if err != nil {
return nil, err
}
for _, id := range ids {
seen[id] = struct{}{}
}
}
out := make([]string, 0, len(seen))
for id := range seen {
out = append(out, id)
}
sort.Strings(out)
return out, nil
}
func applyRecordMetricValue(rec *models.Record, metricName string, value float64) {
switch metricName {
case MetricCPU:
rec.Cpu = float32(value)
case MetricGPU:
rec.Gpu = float32(value)
case MetricRAM:
rec.Ram = int64(value)
case MetricRAMTotal:
rec.RamTotal = int64(value)
case MetricSwap:
rec.Swap = int64(value)
case MetricSwapTotal:
rec.SwapTotal = int64(value)
case MetricLoad:
rec.Load = float32(value)
case MetricTemp:
rec.Temp = float32(value)
case MetricDisk:
rec.Disk = int64(value)
case MetricDiskTotal:
rec.DiskTotal = int64(value)
case MetricNetIn:
rec.NetIn = int64(value)
case MetricNetOut:
rec.NetOut = int64(value)
case MetricNetTotalUp:
rec.NetTotalUp = int64(value)
case MetricNetTotalDown:
rec.NetTotalDown = int64(value)
case MetricTrafficUp:
rec.TrafficUp = int64(value)
case MetricTrafficDown:
rec.TrafficDown = int64(value)
case MetricProcess:
rec.Process = int(value)
case MetricConnections:
rec.Connections = int(value)
case MetricConnectionsUDP:
rec.ConnectionsUdp = int(value)
}
}
func sortRecords(records []models.Record) {
sort.Slice(records, func(i, j int) bool {
if records[i].Client != records[j].Client {
return records[i].Client < records[j].Client
}
return records[i].Time.Before(records[j].Time)
})
}
// GetGPURecordsByClientAndTime 从 metric store 查询 GPU 记录
func GetGPURecordsByClientAndTime(ctx context.Context, clientUUID string, start, end time.Time) ([]models.GPURecord, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
// 查询 GPU 相关指标(每设备利用率使用独立指标 gpu.device.usage)
gpuMetrics := []string{MetricGPUDeviceUsage, MetricGPUMem, MetricGPUMemTotal, MetricGPUTemp}
// 按设备索引和时间组织数据
type gpuKey struct {
deviceIndex int
timestamp int64
}
recordMap := make(map[gpuKey]*models.GPURecord)
for _, metricName := range gpuMetrics {
points, err := s.Query(ctx, metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
})
if err != nil {
continue // GPU 数据可能不存在
}
for _, p := range points {
deviceIndex := 0
deviceName := ""
if idx, ok := p.Tags["device_index"]; ok {
fmt.Sscanf(idx, "%d", &deviceIndex)
}
if name, ok := p.Tags["device_name"]; ok {
deviceName = name
}
key := gpuKey{deviceIndex: deviceIndex, timestamp: p.Timestamp.Unix()}
if recordMap[key] == nil {
recordMap[key] = &models.GPURecord{
Client: clientUUID,
Time: p.Timestamp.UTC(),
DeviceIndex: deviceIndex,
DeviceName: deviceName,
}
}
rec := recordMap[key]
switch metricName {
case MetricGPUDeviceUsage:
rec.Utilization = float32(p.Value)
case MetricGPUMem:
rec.MemUsed = int64(p.Value)
case MetricGPUMemTotal:
rec.MemTotal = int64(p.Value)
case MetricGPUTemp:
rec.Temperature = int(p.Value)
}
}
}
// 转换为切片
records := make([]models.GPURecord, 0, len(recordMap))
for _, rec := range recordMap {
records = append(records, *rec)
}
return records, nil
}
// GetPingRecords 从 metric store 查询兼容旧接口的 ping 记录。
//
// 旧接口过去直接读取 ping_records 的原始点。启用 metric rollup 后,较旧
// 的 raw 点会被压入 rollup 并删除,因此这里使用 Series 走与 queryMetrics
// 相同的 raw/rollup 混合读取路径,并保留 task_id 标签。
func GetPingRecords(ctx context.Context, clientUUID string, taskID int, start, end time.Time) ([]models.PingRecord, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
query := metric.Query{
MetricName: MetricPingLatency,
Start: start,
End: end,
Order: metric.OrderAsc,
}
if clientUUID != "" {
query.EntityID = clientUUID
}
if taskID >= 0 {
query.Tags = map[string]string{"task_id": fmt.Sprintf("%d", taskID)}
}
interval := pingQueryInterval(end.Sub(start), 4000)
interval = s.CompatibleSeriesInterval(start, time.Now().UTC(), interval)
points, err := s.Series(ctx, metric.AggregateQuery{
Query: query,
Aggregation: metric.AggLast,
Interval: interval,
PreserveSeries: true,
}, time.Now().UTC())
if err != nil {
return nil, err
}
records := make([]models.PingRecord, 0, len(points))
for _, p := range points {
taskIDVal := uint(0)
if tid, ok := p.Tags["task_id"]; ok {
var t uint64
fmt.Sscanf(tid, "%d", &t)
taskIDVal = uint(t)
}
records = append(records, models.PingRecord{
Client: p.EntityID,
TaskId: taskIDVal,
Time: p.Bucket.UTC(),
Value: int(p.Value),
})
}
sort.Slice(records, func(i, j int) bool {
return records[i].Time.After(records[j].Time)
})
return records, nil
}
func pingQueryInterval(rangeDuration time.Duration, maxPoints int) time.Duration {
if maxPoints <= 0 {
maxPoints = 4000
}
if rangeDuration <= 0 {
return time.Second
}
interval := time.Duration((rangeDuration.Nanoseconds() + int64(maxPoints) - 1) / int64(maxPoints))
if interval < time.Second {
return time.Second
}
return metric.FloorStandardInterval(interval)
}
// farFuture 返回一个足够远的未来时间,用于以 DeleteBefore 语义清空某指标的全部数据。
func farFuture() time.Time {
return time.Now().UTC().Add(24 * 365 * time.Hour)
}
// DeleteAllRecords 删除所有负载/系统类记录(保留指标定义,不含 ping)。
func DeleteAllRecords(ctx context.Context) error {
s := GetStore()
if s == nil {
return fmt.Errorf("metric store not enabled")
}
for _, metricName := range recordMetricNames {
if _, err := s.DeleteBefore(ctx, metricName, farFuture()); err != nil {
log.Printf("Failed to delete metric %s: %v", metricName, err)
}
}
clearReportTrafficStates()
return nil
}
// DeleteAllPingRecords 删除全部 ping 记录(保留指标定义)。
func DeleteAllPingRecords(ctx context.Context) error {
s := GetStore()
if s == nil {
return fmt.Errorf("metric store not enabled")
}
for _, metricName := range pingMetricNames {
if _, err := s.DeleteBefore(ctx, metricName, farFuture()); err != nil {
return fmt.Errorf("failed to delete ping records: %w", err)
}
}
return nil
}
// DeletePingRecordsByTask 删除指定任务(task_id)的全部 ping 记录。
func DeletePingRecordsByTask(ctx context.Context, taskIDs []uint) error {
s := GetStore()
if s == nil {
return fmt.Errorf("metric store not enabled")
}
for _, id := range taskIDs {
for _, metricName := range pingMetricNames {
if _, err := s.DeleteSeries(ctx, metric.Query{
MetricName: metricName,
Tags: map[string]string{"task_id": fmt.Sprintf("%d", id)},
}); err != nil {
return fmt.Errorf("failed to delete ping records for task %d: %w", id, err)
}
}
}
return nil
}
// DeleteEntity 删除指定 agent 在所有指标下的历史数据。
func DeleteEntity(ctx context.Context, entityID string) error {
s := GetStore()
if s == nil {
return fmt.Errorf("metric store not enabled")
}
if _, err := s.DeleteEntity(ctx, entityID); err != nil {
return fmt.Errorf("failed to delete metric records for entity %s: %w", entityID, err)
}
deleteReportTrafficState(entityID)
return nil
}
+481
View File
@@ -0,0 +1,481 @@
package metricstore
import (
"context"
"testing"
"time"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/metric"
v1 "github.com/komari-monitor/komari/protocol/v1"
)
func TestDefaultRollupPolicy(t *testing.T) {
policy := defaultRollupPolicy()
if err := policy.Validate(); err != nil {
t.Fatalf("default rollup policy should validate: %v", err)
}
if policy.RawRetention != DefaultRollupRawRetention {
t.Fatalf("raw retention = %s, want %s", policy.RawRetention, DefaultRollupRawRetention)
}
if len(policy.Tiers) != 3 {
t.Fatalf("expected 3 rollup tiers, got %d", len(policy.Tiers))
}
wantIntervals := []time.Duration{time.Minute, 5 * time.Minute, time.Hour}
wantRetentions := []time.Duration{48 * time.Hour, 14 * 24 * time.Hour, 14 * 24 * time.Hour}
for i := range wantIntervals {
if policy.Tiers[i].Interval != wantIntervals[i] {
t.Fatalf("tier %d interval = %s, want %s", i, policy.Tiers[i].Interval, wantIntervals[i])
}
if policy.Tiers[i].Retention != wantRetentions[i] {
t.Fatalf("tier %d retention = %s, want %s", i, policy.Tiers[i].Retention, wantRetentions[i])
}
}
}
func TestBuildMetricConfigEnablesDefaultRollupPolicy(t *testing.T) {
cfg, err := buildMetricConfig(&MetricStoreConfig{
Driver: "sqlite",
DSN: ":memory:",
DownsamplingEnabled: true,
TablePrefix: "metric_",
}, false)
if err != nil {
t.Fatalf("build metric config: %v", err)
}
if !cfg.RollupPolicy.Enabled() {
t.Fatal("expected default rollup policy to be enabled")
}
if cfg.RollupPolicy.RawRetention != DefaultRollupRawRetention {
t.Fatalf("raw retention = %s, want %s", cfg.RollupPolicy.RawRetention, DefaultRollupRawRetention)
}
}
func TestBuildMetricConfigLeavesFinalRetentionToMetricDefinition(t *testing.T) {
cfg, err := buildMetricConfig(&MetricStoreConfig{
Driver: "sqlite",
DSN: ":memory:",
DownsamplingEnabled: true,
}, false)
if err != nil {
t.Fatalf("build metric config: %v", err)
}
wantRollupRetention := 14 * 24 * time.Hour
lastTier := cfg.RollupPolicy.Tiers[len(cfg.RollupPolicy.Tiers)-1]
if lastTier.Retention != wantRollupRetention {
t.Fatalf("rollup retention = %s, want %s", lastTier.Retention, wantRollupRetention)
}
}
func TestBuildMetricConfigCanDisableDownsampling(t *testing.T) {
cfg, err := buildMetricConfig(&MetricStoreConfig{
Driver: "sqlite",
DSN: ":memory:",
DownsamplingEnabled: false,
}, false)
if err != nil {
t.Fatalf("build metric config: %v", err)
}
if cfg.RollupPolicy.Enabled() {
t.Fatal("expected rollup policy to be disabled")
}
}
func TestGetPingRecordsReadsRollupsAfterRawCompaction(t *testing.T) {
ctx := context.Background()
now := time.Now().UTC().Truncate(time.Second)
s, err := metric.Open(ctx, metric.SQLite(":memory:",
metric.WithMaxOpenConns(1),
metric.WithRollupPolicy(defaultRollupPolicy()),
))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
defer s.Close()
if err := s.UpsertMetric(ctx, metric.Definition{
Name: MetricPingLatency,
Type: metric.TypeGauge,
RetentionDays: 30,
}); err != nil {
t.Fatalf("create ping metric: %v", err)
}
if err := s.WriteBatch(ctx, []metric.Point{
{MetricName: MetricPingLatency, EntityID: "node-a", Timestamp: now.Add(-20 * time.Minute), Value: 20, Tags: map[string]string{"task_id": "7"}},
{MetricName: MetricPingLatency, EntityID: "node-a", Timestamp: now.Add(-10 * time.Minute), Value: 10, Tags: map[string]string{"task_id": "7"}},
{MetricName: MetricPingLatency, EntityID: "node-a", Timestamp: now.Add(-5 * time.Minute), Value: 5, Tags: map[string]string{"task_id": "7"}},
}); err != nil {
t.Fatalf("write ping points: %v", err)
}
if _, err := s.Compact(ctx, now); err != nil {
t.Fatalf("compact ping points: %v", err)
}
storeMu.Lock()
oldStore := store
store = s
storeMu.Unlock()
defer func() {
storeMu.Lock()
store = oldStore
storeMu.Unlock()
}()
records, err := GetPingRecords(ctx, "node-a", 7, now.Add(-30*time.Minute), now)
if err != nil {
t.Fatalf("get ping records: %v", err)
}
if len(records) != 3 {
t.Fatalf("expected 3 ping records across raw and rollup data, got %d: %#v", len(records), records)
}
if records[0].Value != 5 || records[1].Value != 10 || records[2].Value != 20 {
t.Fatalf("unexpected ping values in descending order: %#v", records)
}
}
func TestCreateMetricDefinitionsUsesExplicitRetentionAndPreservesOverrides(t *testing.T) {
if defaultBuiltinMetricRetentionDays != 1 {
t.Fatalf("default built-in metric retention = %d, want 1 day", defaultBuiltinMetricRetentionDays)
}
ctx := context.Background()
s, err := metric.Open(ctx, metric.SQLite(":memory:", metric.WithMaxOpenConns(1)))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
defer s.Close()
if err := createMetricDefinitions(ctx, s); err != nil {
t.Fatalf("create definitions: %v", err)
}
defs, err := s.ListMetrics(ctx)
if err != nil {
t.Fatalf("list definitions: %v", err)
}
if len(defs) != 25 {
t.Fatalf("definition count = %d, want 25", len(defs))
}
for _, def := range defs {
if def.RetentionDays != defaultBuiltinMetricRetentionDays {
t.Fatalf("%s retention = %d, want %d", def.Name, def.RetentionDays, defaultBuiltinMetricRetentionDays)
}
}
cpu, err := s.GetMetric(ctx, MetricCPU)
if err != nil {
t.Fatalf("get cpu definition: %v", err)
}
cpu.RetentionDays = 60
if err := s.UpsertMetric(ctx, cpu); err != nil {
t.Fatalf("override cpu retention: %v", err)
}
if err := createMetricDefinitions(ctx, s); err != nil {
t.Fatalf("recreate definitions: %v", err)
}
cpu, err = s.GetMetric(ctx, MetricCPU)
if err != nil {
t.Fatalf("reload cpu definition: %v", err)
}
if cpu.RetentionDays != 60 {
t.Fatalf("cpu retention = %d, want preserved override 60", cpu.RetentionDays)
}
if _, err := s.SetMetricRetention(ctx, MetricCPU, 0); err != nil {
t.Fatalf("disable cpu retention: %v", err)
}
if err := createMetricDefinitions(ctx, s); err != nil {
t.Fatalf("refresh disabled definition: %v", err)
}
cpu, err = s.GetMetric(ctx, MetricCPU)
if err != nil {
t.Fatalf("reload disabled cpu definition: %v", err)
}
if cpu.RetentionDays != 0 {
t.Fatalf("cpu retention = %d, want preserved disabled state", cpu.RetentionDays)
}
}
func TestCreateMetricDefinitionsUsesLegacySpanOnlyForNewDefinitions(t *testing.T) {
ctx := context.Background()
s, err := metric.Open(ctx, metric.SQLite(":memory:", metric.WithMaxOpenConns(1)))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
defer s.Close()
if err := createMetricDefinitionsWithDefaultRetention(ctx, s, 10); err != nil {
t.Fatalf("create migration definitions: %v", err)
}
defs, err := s.ListMetrics(ctx)
if err != nil {
t.Fatalf("list migration definitions: %v", err)
}
for _, def := range defs {
if def.RetentionDays != 10 {
t.Fatalf("%s retention = %d, want legacy span 10", def.Name, def.RetentionDays)
}
}
cpu, err := s.GetMetric(ctx, MetricCPU)
if err != nil {
t.Fatalf("get CPU definition: %v", err)
}
cpu.RetentionDays = 3
if err := s.UpsertMetric(ctx, cpu); err != nil {
t.Fatalf("override CPU retention: %v", err)
}
if err := createMetricDefinitionsWithDefaultRetention(ctx, s, 20); err != nil {
t.Fatalf("refresh migration definitions: %v", err)
}
cpu, err = s.GetMetric(ctx, MetricCPU)
if err != nil {
t.Fatalf("reload CPU definition: %v", err)
}
if cpu.RetentionDays != 3 {
t.Fatalf("existing CPU retention = %d, want preserved 3", cpu.RetentionDays)
}
}
func TestGetRetentionSummaryUsesAllMetricDefinitions(t *testing.T) {
ctx := context.Background()
s, err := metric.Open(ctx, metric.SQLite(":memory:", metric.WithMaxOpenConns(1)))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
defer s.Close()
storeMu.Lock()
oldStore := store
store = s
storeMu.Unlock()
defer func() {
storeMu.Lock()
store = oldStore
storeMu.Unlock()
}()
empty, err := GetRetentionSummary(ctx)
if err != nil {
t.Fatalf("summarize empty store: %v", err)
}
if empty.AllPositive || empty.MaxDays != 0 {
t.Fatalf("unexpected empty summary: %#v", empty)
}
for _, def := range []metric.Definition{
{Name: "short", Type: metric.TypeGauge, RetentionDays: 7},
{Name: "long", Type: metric.TypeGauge, RetentionDays: 60},
} {
if err := s.UpsertMetric(ctx, def); err != nil {
t.Fatalf("upsert %s: %v", def.Name, err)
}
}
summary, err := GetRetentionSummary(ctx)
if err != nil {
t.Fatalf("summarize definitions: %v", err)
}
if !summary.AllPositive || summary.MaxDays != 60 {
t.Fatalf("unexpected summary: %#v", summary)
}
if _, err := s.SetMetricRetention(ctx, "short", 0); err != nil {
t.Fatalf("disable short metric: %v", err)
}
summary, err = GetRetentionSummary(ctx)
if err != nil {
t.Fatalf("summarize disabled metric: %v", err)
}
if summary.AllPositive || summary.MaxDays != 60 {
t.Fatalf("unexpected disabled summary: %#v", summary)
}
}
func TestSummarizeRetentionDefinitionsRequiresEveryMetricToBePositive(t *testing.T) {
summary := summarizeRetentionDefinitions([]metric.Definition{
{Name: "enabled", RetentionDays: 30},
{Name: "disabled", RetentionDays: 0},
{Name: "long", RetentionDays: 60},
})
if summary.AllPositive || summary.MaxDays != 60 {
t.Fatalf("unexpected summary: %#v", summary)
}
}
func TestCompactCleansExpiredRawPointsWhenDownsamplingDisabled(t *testing.T) {
ctx := context.Background()
s, err := metric.Open(ctx, metric.SQLite(":memory:", metric.WithMaxOpenConns(1)))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
if err := s.UpsertMetric(ctx, metric.Definition{
Name: "raw.metric",
Type: metric.TypeGauge,
RetentionDays: 1,
}); err != nil {
t.Fatalf("upsert metric: %v", err)
}
now := time.Date(2026, 7, 11, 12, 0, 0, 0, time.UTC)
if err := s.WriteBatch(ctx, []metric.Point{
{MetricName: "raw.metric", EntityID: "node", Timestamp: now.Add(-48 * time.Hour), Value: 1},
{MetricName: "raw.metric", EntityID: "node", Timestamp: now.Add(-time.Hour), Value: 2},
}); err != nil {
t.Fatalf("write points: %v", err)
}
storeMu.Lock()
oldStore := store
oldCompactAt := compactAt
store = s
compactAt = 0
storeMu.Unlock()
defer func() {
storeMu.Lock()
store = oldStore
compactAt = oldCompactAt
storeMu.Unlock()
_ = s.Close()
}()
if _, err := Compact(ctx, now); err != nil {
t.Fatalf("compact: %v", err)
}
points, err := s.Query(ctx, metric.Query{
MetricName: "raw.metric",
EntityID: "node",
Start: now.Add(-72 * time.Hour),
End: now,
})
if err != nil {
t.Fatalf("query points: %v", err)
}
if len(points) != 1 || points[0].Value != 2 {
t.Fatalf("expected only the retained raw point, got %#v", points)
}
}
func TestCompactKeepsRotatingCursorAfterFullCycle(t *testing.T) {
ctx := context.Background()
s, err := metric.Open(ctx, metric.SQLite(":memory:",
metric.WithMaxOpenConns(1),
metric.WithRollupPolicy(defaultRollupPolicy()),
))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
for _, name := range []string{"a.metric", "b.metric", "c.metric"} {
if err := s.UpsertMetric(ctx, metric.Definition{Name: name, Type: metric.TypeGauge}); err != nil {
t.Fatalf("upsert metric %s: %v", name, err)
}
}
storeMu.Lock()
oldStore := store
oldCompactAt := compactAt
store = s
compactAt = 1
storeMu.Unlock()
defer func() {
storeMu.Lock()
store = oldStore
compactAt = oldCompactAt
storeMu.Unlock()
_ = s.Close()
}()
if _, err := Compact(ctx, time.Date(2026, 7, 11, 12, 0, 0, 0, time.UTC)); err != nil {
t.Fatalf("compact: %v", err)
}
if compactAt != 1 {
t.Fatalf("compact cursor = %d, want 1 after a complete rotated cycle", compactAt)
}
}
func TestGetRecordsByClientAndTimeReadsRollupsAfterRawCompaction(t *testing.T) {
ctx := context.Background()
s, err := metric.Open(ctx, metric.SQLite(":memory:",
metric.WithMaxOpenConns(1),
metric.WithRollupPolicy(defaultRollupPolicy()),
))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
if err := createMetricDefinitions(ctx, s); err != nil {
t.Fatalf("create metric definitions: %v", err)
}
storeMu.Lock()
oldStore := store
oldCompactAt := compactAt
store = s
compactAt = 0
storeMu.Unlock()
defer func() {
storeMu.Lock()
store = oldStore
compactAt = oldCompactAt
storeMu.Unlock()
_ = s.Close()
}()
now := time.Now().UTC().Truncate(time.Minute)
ts := now.Add(-time.Hour)
rec := models.Record{
Client: "node-a",
Time: ts,
Cpu: 42.5,
Ram: 123456,
RamTotal: 999999,
Disk: 456789,
DiskTotal: 777777,
Load: 0.75,
Connections: 321,
ConnectionsUdp: 12,
}
if _, err := WriteReport(ctx, v1.Report{
UUID: rec.Client,
UpdatedAt: ts,
CPU: v1.CPUReport{Usage: float64(rec.Cpu)},
Ram: v1.RamReport{Used: rec.Ram, Total: rec.RamTotal},
Load: v1.LoadReport{Load1: float64(rec.Load)},
Disk: v1.DiskReport{Used: rec.Disk, Total: rec.DiskTotal},
Process: rec.Process,
Connections: v1.ConnectionsReport{
TCP: rec.Connections,
UDP: rec.ConnectionsUdp,
},
}); err != nil {
t.Fatalf("write record: %v", err)
}
if _, err := s.Compact(ctx, now); err != nil {
t.Fatalf("compact raw into rollup: %v", err)
}
raw, err := s.Query(ctx, metric.Query{
MetricName: MetricCPU,
EntityID: rec.Client,
Start: ts.Add(-time.Minute),
End: now,
})
if err != nil {
t.Fatalf("query raw cpu: %v", err)
}
if len(raw) != 0 {
t.Fatalf("expected old raw cpu point to be deleted after compaction, got %d", len(raw))
}
got, err := GetRecordsByClientAndTime(ctx, rec.Client, ts.Add(-time.Minute), now)
if err != nil {
t.Fatalf("get records: %v", err)
}
if len(got) != 1 {
t.Fatalf("expected 1 reconstructed record from rollup, got %d: %#v", len(got), got)
}
if got[0].Cpu == 0 || got[0].Ram == 0 || got[0].Disk == 0 || got[0].Connections == 0 {
t.Fatalf("record was not reconstructed from rollup: %#v", got[0])
}
all, err := GetRecordsByTime(ctx, ts.Add(-time.Minute), now)
if err != nil {
t.Fatalf("get all records: %v", err)
}
if len(all) != 1 || all[0].Client != rec.Client || all[0].Cpu == 0 {
t.Fatalf("all-client records were not reconstructed from rollup: %#v", all)
}
}
+72
View File
@@ -0,0 +1,72 @@
package metricstore
import (
"fmt"
"log"
"strings"
"github.com/komari-monitor/komari/pkg/config"
)
// migration.go
//
// 启动阶段的 metrics 数据迁移。这里只处理 metrics 存储后端切换时的数据搬运
// (例如默认 SQLite ./data/metrics.db 切换到 MySQL/PostgreSQL)。旧 komari.db
// 监控表导入属于一次性迁移,见 pkg/migrations.RunMetricStoreMigrations。
// targetFingerprint 返回当前 metrics 目标库的指纹(driver + 归一化 DSN),
// 用于判断 metrics 存储后端是否发生变化(例如从 SQLite 切换到 MySQL/PostgreSQL)。
func targetFingerprint(cfg *MetricStoreConfig) string {
driver := ResolveDriverFromConfig(cfg.Driver, cfg.DSN)
dsn := strings.TrimSpace(cfg.DSN)
return fmt.Sprintf("%s|%s", driver, dsn)
}
// RunStartupMigration 在服务启动时检查 metrics 存储后端是否发生变化,
// 若变化则把上一个 metrics 目标库的全部数据搬运到当前目标。
//
// 判定依据 MigrationTargetKey(记录“数据当前完整所在的 metrics 目标指纹”):
// - saved == current:数据已在当前目标,无需处理。
// - saved == "":无历史目标记录。老快照的 metrics 数据固定落在默认 SQLite
// (./data/metrics.db);若当前目标不是默认 SQLite(例如已切到 MySQL/PostgreSQL),
// 把默认 SQLite 中可能存在的历史数据搬运过来,否则仅登记指纹。
// - saved != current:后端已切换,把上一个目标(saved)的数据搬运到当前目标。
//
// 搬运以 upsert 写入,幂等,中断后重启可安全重跑。任一失败都返回错误,
// 调用方应让启动失败并打印明确错误。
func RunStartupMigration() error {
s := GetStore()
if s == nil {
return fmt.Errorf("metric store not initialized")
}
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
return fmt.Errorf("failed to load metric store config: %w", err)
}
current := targetFingerprint(cfg)
saved, _ := config.GetAs[string](MigrationTargetKey, "")
// 数据已完整位于当前目标:无需搬运。
if saved == current {
return nil
}
// 上一个目标:优先使用已保存指纹;无记录时按老快照默认 SQLite 推断。
prev := saved
if prev == "" {
prev = defaultSQLiteFingerprint()
}
if prev != current {
if err := migrateFromPreviousStore(prev, cfg, s); err != nil {
return err
}
}
if err := config.Set(MigrationTargetKey, current); err != nil {
log.Printf("Failed to persist migration target fingerprint: %v", err)
}
return nil
}
+211
View File
@@ -0,0 +1,211 @@
package metricstore
import (
"context"
"fmt"
"log"
"strings"
"time"
"github.com/komari-monitor/komari/pkg/metric"
)
// storeMigrationWindow 是 store-to-store 迁移时按时间分窗查询的窗口大小。
// 采样点按时间窗口分批拉取,避免一次性把整段序列读入内存。
const storeMigrationWindow = 6 * time.Hour
// configFromFingerprint 从目标指纹(driver|dsn)重建一个 MetricStoreConfig,
// 用于以只读方式打开“上一次的 metrics 目标库”。表前缀、保留天数、连接数等
// 沿用当前配置(切换后端时这些通常不变)。
func configFromFingerprint(fingerprint string, base *MetricStoreConfig) (*MetricStoreConfig, error) {
idx := strings.Index(fingerprint, "|")
if idx < 0 {
return nil, fmt.Errorf("invalid target fingerprint: %q", fingerprint)
}
driver := fingerprint[:idx]
dsn := fingerprint[idx+1:]
if strings.TrimSpace(driver) == "" {
return nil, fmt.Errorf("empty driver in target fingerprint: %q", fingerprint)
}
return &MetricStoreConfig{
Driver: driver,
DSN: dsn,
DownsamplingEnabled: base.DownsamplingEnabled,
TablePrefix: base.TablePrefix,
MaxOpenConns: base.MaxOpenConns,
MaxIdleConns: base.MaxIdleConns,
}, nil
}
// openSourceStore 打开一个已存在的 metrics 目标库作为数据搬运的源库读取。
//
// 使用 autoMigrate=true:GORM 的 AutoMigrate 只新增表/列/索引、从不删除数据,
// 因此对真实旧库幂等无害;而当源库文件/表不存在(例如老快照记录了 completed
// 但 metrics.db 缺失)时,可创建空表让后续 ListMetrics 返回空集而非报
// "no such table",从而把“无历史可迁移”识别为正常情况而非错误。
func openSourceStore(ctx context.Context, cfg *MetricStoreConfig) (*metric.Store, error) {
metricCfg, err := buildMetricConfig(cfg, true)
if err != nil {
return nil, err
}
return metric.Open(ctx, metricCfg)
}
// defaultSQLiteFingerprint 返回默认 SQLite metrics 库(./data/metrics.db)的目标指纹。
// 老快照的 metrics 数据固定落在该 SQLite 文件,用于在缺失指纹时推断上一个源库。
func defaultSQLiteFingerprint() string {
return targetFingerprint(&MetricStoreConfig{Driver: "sqlite", DSN: "./data/metrics.db"})
}
// migrateFromPreviousStore 打开由 prevFingerprint 指定的上一个 metrics 目标库作为源,
// 把其中的全部指标搬运到当前目标 dst(例如 SQLite metrics.db → MySQL/PostgreSQL)。
//
// 该过程幂等:dst 以 (metric, entity, tags, ts) upsert 写入,中断后重启可安全重跑。
func migrateFromPreviousStore(prevFingerprint string, cfg *MetricStoreConfig, dst *metric.Store) error {
prevCfg, err := configFromFingerprint(prevFingerprint, cfg)
if err != nil {
return fmt.Errorf("parse previous metrics target %q: %w", prevFingerprint, err)
}
ctx := context.Background()
src, err := openSourceStore(ctx, prevCfg)
if err != nil {
return fmt.Errorf("open previous metrics store (%s): %w", prevFingerprint, err)
}
defer src.Close()
log.Printf("Migrating metrics from previous store %s to current target...", prevFingerprint)
total, err := MigrateBetweenStores(ctx, src, dst)
if err != nil {
return fmt.Errorf("store-to-store metrics migration failed: %w", err)
}
log.Printf("Store-to-store metrics migration completed (%d points)", total)
return nil
}
// storeMigrationObserver 在 store-to-store 迁移过程中接收进度回调。
// - currentMetric:当前正在搬运的指标名。
// - metricIndex:该指标在全部指标中的序号(0 起),即已完成的指标数。
// - totalMetrics:指标定义总数。
// - addedPoints:本次新写入目标库的采样点数(用于外部累计)。
type storeMigrationObserver func(currentMetric string, metricIndex, totalMetrics int, addedPoints int64)
// MigrateBetweenStores 把 src metric store 的全部指标定义与采样点搬运到 dst。
//
// 用于 metrics 后端切换(例如默认 SQLite metrics.db → MySQL/PostgreSQL):
// - 先在 dst 建立/更新全部指标定义;
// - 再按指标、按时间窗口分批读取源库采样点并写入 dst。
//
// 采样点在 dst 以 (metric_name, entity_id, tags, ts) upsert 写入,窗口边界重叠也
// 幂等,因此整个过程可安全重试。返回搬运的采样点总数。
func MigrateBetweenStores(ctx context.Context, src, dst *metric.Store) (int64, error) {
return migrateBetweenStores(ctx, src, dst, nil)
}
// migrateBetweenStores 是 MigrateBetweenStores 的内部实现,额外接受一个可选的进度
// 观察者 observe(为 nil 时行为与旧版本完全一致)。启动迁移走 nil,WebUI/API 触发
// 的迁移传入回调以实时更新进度。
func migrateBetweenStores(ctx context.Context, src, dst *metric.Store, observe storeMigrationObserver) (int64, error) {
if src == nil || dst == nil {
return 0, fmt.Errorf("source or destination metric store is nil")
}
defs, err := src.ListMetrics(ctx)
if err != nil {
return 0, fmt.Errorf("list source metrics: %w", err)
}
var total int64
for i, def := range defs {
if observe != nil {
// 进入下一个指标:已完成 i 个指标。
observe(def.Name, i, len(defs), 0)
}
// 目标库先建立指标定义,保证后续写入的指标存在。
if err := dst.UpsertMetric(ctx, def); err != nil {
return total, fmt.Errorf("upsert metric %q on target: %w", def.Name, err)
}
if def.RetentionDays == 0 {
continue
}
earliest, latest, ok, err := metricTimeBounds(ctx, src, def.Name)
if err != nil {
return total, fmt.Errorf("resolve time bounds for metric %q: %w", def.Name, err)
}
if !ok {
// 该指标只有定义没有数据,跳过。
continue
}
var migrated int64
for windowStart := earliest; !windowStart.After(latest); windowStart = windowStart.Add(storeMigrationWindow) {
select {
case <-ctx.Done():
return total, ctx.Err()
default:
}
windowEnd := windowStart.Add(storeMigrationWindow)
points, err := src.Query(ctx, metric.Query{
MetricName: def.Name,
Start: windowStart,
End: windowEnd,
Order: metric.OrderAsc,
})
if err != nil {
return total, fmt.Errorf("query metric %q window [%s,%s): %w", def.Name, windowStart, windowEnd, err)
}
if len(points) == 0 {
continue
}
if err := dst.WriteBatch(ctx, points); err != nil {
return total, fmt.Errorf("write metric %q batch to target: %w", def.Name, err)
}
migrated += int64(len(points))
total += int64(len(points))
if observe != nil {
observe(def.Name, i, len(defs), int64(len(points)))
}
}
if migrated > 0 {
log.Printf("[store-migration] metric %q: migrated %d points", def.Name, migrated)
}
}
return total, nil
}
// metricTimeBounds 返回某指标在 src 中最早/最晚采样时间。ok=false 表示无数据。
// 通过升/降序各取一条采样点定位边界,避免把整段序列读入内存。
func metricTimeBounds(ctx context.Context, src *metric.Store, name string) (time.Time, time.Time, bool, error) {
wide := metric.Query{
MetricName: name,
Start: time.Unix(0, 0),
End: time.Now().UTC().Add(24 * time.Hour),
}
asc := wide
asc.Order = metric.OrderAsc
asc.Limit = 1
first, err := src.Query(ctx, asc)
if err != nil {
return time.Time{}, time.Time{}, false, err
}
if len(first) == 0 {
return time.Time{}, time.Time{}, false, nil
}
desc := wide
desc.Order = metric.OrderDesc
desc.Limit = 1
last, err := src.Query(ctx, desc)
if err != nil {
return time.Time{}, time.Time{}, false, err
}
if len(last) == 0 {
return time.Time{}, time.Time{}, false, nil
}
return first[0].Timestamp, last[0].Timestamp, true, nil
}
+38
View File
@@ -0,0 +1,38 @@
package metricstore
import "context"
// storeOperationGate serializes operations that need a stable active Store.
// A channel-backed gate supports both non-blocking scheduled work and
// context-aware admin/shutdown waits.
type storeOperationGate struct {
token chan struct{}
}
func newStoreOperationGate() *storeOperationGate {
gate := &storeOperationGate{token: make(chan struct{}, 1)}
gate.token <- struct{}{}
return gate
}
func (g *storeOperationGate) Acquire(ctx context.Context) error {
select {
case <-g.token:
return nil
case <-ctx.Done():
return ctx.Err()
}
}
func (g *storeOperationGate) TryAcquire() bool {
select {
case <-g.token:
return true
default:
return false
}
}
func (g *storeOperationGate) Release() {
g.token <- struct{}{}
}
+467
View File
@@ -0,0 +1,467 @@
package metricstore
import (
"context"
"errors"
"fmt"
"log"
"strconv"
"sync"
"time"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/metric"
v1 "github.com/komari-monitor/komari/protocol/v1"
)
type reportTrafficState struct {
mu sync.Mutex
reportTrafficValues
}
type reportTrafficValues struct {
initialized bool
timestamp time.Time
hasUp bool
totalUp int64
hasDown bool
totalDown int64
}
var reportTrafficStates sync.Map
const (
reportBatchInterval = 3 * time.Second
reportBatchQueueSize = 4096
reportBatchMaxReports = 256 * 21
reportBatchWriteTimeout = 10 * time.Second
)
var (
reportBatcherMu sync.Mutex
reportBatcher *reportBatchWorker
)
var (
ErrReportBatchQueueFull = errors.New("metric report batch queue is full")
ErrReportBatchStopped = errors.New("metric report batcher is stopped")
)
type reportBatchRequest struct {
ctx context.Context
done chan error
stop bool
}
type reportBatchWorker struct {
mu sync.Mutex
queue chan v1.Report
requests chan reportBatchRequest
done chan struct{}
stopping bool
}
// StartReportBatcher starts the shared report writer. Reports are collected for
// a short window so many agents can be persisted in one database transaction.
func StartReportBatcher() {
reportBatcherMu.Lock()
defer reportBatcherMu.Unlock()
if reportBatcher != nil {
return
}
worker := &reportBatchWorker{
queue: make(chan v1.Report, reportBatchQueueSize),
requests: make(chan reportBatchRequest, 1),
done: make(chan struct{}),
}
reportBatcher = worker
go worker.run()
}
// StopReportBatcher stops the report writer after flushing all queued reports.
func StopReportBatcher(ctx context.Context) error {
reportBatcherMu.Lock()
worker := reportBatcher
if worker == nil {
reportBatcherMu.Unlock()
return nil
}
worker.mu.Lock()
worker.stopping = true
worker.mu.Unlock()
reportBatcherMu.Unlock()
request := reportBatchRequest{ctx: ctx, done: make(chan error, 1), stop: true}
select {
case worker.requests <- request:
case <-ctx.Done():
return ctx.Err()
}
select {
case err := <-request.done:
<-worker.done
reportBatcherMu.Lock()
if reportBatcher == worker {
reportBatcher = nil
}
reportBatcherMu.Unlock()
return err
case <-ctx.Done():
return ctx.Err()
}
}
// FlushReportBatch synchronously flushes the current queue. It is useful for
// controlled handoff points and deterministic tests; normal operation relies
// on the three-second ticker.
func FlushReportBatch(ctx context.Context) error {
reportBatcherMu.Lock()
worker := reportBatcher
reportBatcherMu.Unlock()
if worker == nil {
return nil
}
request := reportBatchRequest{ctx: ctx, done: make(chan error, 1)}
worker.mu.Lock()
stopping := worker.stopping
worker.mu.Unlock()
if stopping {
return ErrReportBatchStopped
}
select {
case worker.requests <- request:
case <-worker.done:
return ErrReportBatchStopped
case <-ctx.Done():
return ctx.Err()
}
select {
case err := <-request.done:
return err
case <-ctx.Done():
return ctx.Err()
}
}
// WriteReport persists one agent report as raw metric points sharing the same
// server receive time. Traffic deltas are derived from the previous counters
// and stored alongside the raw counters so they remain summable after rollup.
func WriteReport(ctx context.Context, report v1.Report) (v1.Report, error) {
if report.UUID == "" {
return v1.Report{}, fmt.Errorf("report UUID is required")
}
if report.UpdatedAt.IsZero() {
return v1.Report{}, fmt.Errorf("report receive time is required")
}
report.UpdatedAt = report.UpdatedAt.UTC()
if GetStore() == nil {
return v1.Report{}, fmt.Errorf("metric store not enabled")
}
reportBatcherMu.Lock()
worker := reportBatcher
reportBatcherMu.Unlock()
if worker != nil {
if err := worker.enqueue(ctx, report); err != nil {
return v1.Report{}, err
}
return report, nil
}
saved, err := writeReportBatch(ctx, []v1.Report{report})
if err != nil {
return v1.Report{}, err
}
return saved[0], nil
}
func (w *reportBatchWorker) enqueue(ctx context.Context, report v1.Report) error {
w.mu.Lock()
defer w.mu.Unlock()
if w.stopping {
return ErrReportBatchStopped
}
if err := ctx.Err(); err != nil {
return err
}
select {
case w.queue <- report:
return nil
default:
return ErrReportBatchQueueFull
}
}
func (w *reportBatchWorker) run() {
ticker := time.NewTicker(reportBatchInterval)
defer ticker.Stop()
var pending []v1.Report
for {
select {
case report := <-w.queue:
pending = append(pending, report)
case request := <-w.requests:
pending = append(pending, drainReportQueue(w.queue, reportBatchMaxReports-len(pending))...)
err := writePendingReports(request.ctx, &pending)
if request.stop {
if err != nil {
log.Printf("failed to flush metric report batch during shutdown: %v", err)
}
close(w.done)
request.done <- err
return
}
request.done <- err
case <-ticker.C:
pending = append(pending, drainReportQueue(w.queue, reportBatchMaxReports-len(pending))...)
if err := writePendingReports(context.Background(), &pending); err != nil {
log.Printf("failed to flush metric report batch: %v", err)
}
}
}
}
func drainReportQueue(queue <-chan v1.Report, limit int) []v1.Report {
if limit <= 0 {
return nil
}
reports := make([]v1.Report, 0, limit)
for len(reports) < limit {
select {
case report := <-queue:
reports = append(reports, report)
default:
return reports
}
}
return reports
}
func writePendingReports(ctx context.Context, pending *[]v1.Report) error {
if len(*pending) == 0 {
return nil
}
writeCtx, cancel := context.WithTimeout(ctx, reportBatchWriteTimeout)
defer cancel()
if err := func() error {
reports := *pending
_, err := writeReportBatch(writeCtx, reports)
return err
}(); err != nil {
return err
}
*pending = (*pending)[:0]
return nil
}
func writeReportBatch(ctx context.Context, reports []v1.Report) ([]v1.Report, error) {
if len(reports) == 0 {
return nil, nil
}
if err := storeOperations.Acquire(ctx); err != nil {
return nil, fmt.Errorf("wait for metric store operation before writing reports: %w", err)
}
defer storeOperations.Release()
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
prepared := make([]v1.Report, len(reports))
copy(prepared, reports)
points := make([]metric.Point, 0, len(reports)*20)
pendingStates := make(map[*reportTrafficState]reportTrafficValues)
for i, report := range prepared {
stateValue, _ := reportTrafficStates.LoadOrStore(report.UUID, &reportTrafficState{})
state := stateValue.(*reportTrafficState)
values, ok := pendingStates[state]
if !ok {
state.mu.Lock()
values = state.reportTrafficValues
state.mu.Unlock()
}
if !values.initialized {
var err error
values.totalUp, values.hasUp, err = latestReportCounter(ctx, s, MetricNetTotalUp, report.UUID, report.UpdatedAt)
if err != nil {
return nil, fmt.Errorf("load previous upload counter: %w", err)
}
values.totalDown, values.hasDown, err = latestReportCounter(ctx, s, MetricNetTotalDown, report.UUID, report.UpdatedAt)
if err != nil {
return nil, fmt.Errorf("load previous download counter: %w", err)
}
values.initialized = true
}
if !values.timestamp.IsZero() && !report.UpdatedAt.After(values.timestamp) {
report.UpdatedAt = values.timestamp.Add(time.Nanosecond)
}
trafficUp := int64(0)
if values.hasUp {
trafficUp = TrafficCounterDelta(report.Network.TotalUp, values.totalUp)
}
trafficDown := int64(0)
if values.hasDown {
trafficDown = TrafficCounterDelta(report.Network.TotalDown, values.totalDown)
}
points = append(points, reportMetricPoints(report, trafficUp, trafficDown)...)
values.timestamp = report.UpdatedAt
values.hasUp = true
values.totalUp = report.Network.TotalUp
values.hasDown = true
values.totalDown = report.Network.TotalDown
pendingStates[state] = values
prepared[i] = report
}
if err := s.WriteBatch(ctx, points); err != nil {
return nil, err
}
for state, values := range pendingStates {
state.mu.Lock()
state.reportTrafficValues = values
state.mu.Unlock()
}
return prepared, nil
}
func reportMetricPoints(report v1.Report, trafficUp, trafficDown int64) []metric.Point {
entityID := report.UUID
ts := report.UpdatedAt
gpuUsage := 0.0
if report.GPU != nil {
gpuUsage = report.GPU.AverageUsage
}
points := []metric.Point{
{MetricName: MetricCPU, EntityID: entityID, Timestamp: ts, Value: report.CPU.Usage},
{MetricName: MetricGPU, EntityID: entityID, Timestamp: ts, Value: gpuUsage},
{MetricName: MetricRAM, EntityID: entityID, Timestamp: ts, Value: float64(report.Ram.Used)},
{MetricName: MetricRAMTotal, EntityID: entityID, Timestamp: ts, Value: float64(report.Ram.Total)},
{MetricName: MetricSwap, EntityID: entityID, Timestamp: ts, Value: float64(report.Swap.Used)},
{MetricName: MetricSwapTotal, EntityID: entityID, Timestamp: ts, Value: float64(report.Swap.Total)},
{MetricName: MetricLoad, EntityID: entityID, Timestamp: ts, Value: report.Load.Load1},
{MetricName: MetricTemp, EntityID: entityID, Timestamp: ts, Value: 0},
{MetricName: MetricDisk, EntityID: entityID, Timestamp: ts, Value: float64(report.Disk.Used)},
{MetricName: MetricDiskTotal, EntityID: entityID, Timestamp: ts, Value: float64(report.Disk.Total)},
{MetricName: MetricNetIn, EntityID: entityID, Timestamp: ts, Value: float64(report.Network.Down)},
{MetricName: MetricNetOut, EntityID: entityID, Timestamp: ts, Value: float64(report.Network.Up)},
{MetricName: MetricNetTotalUp, EntityID: entityID, Timestamp: ts, Value: float64(report.Network.TotalUp)},
{MetricName: MetricNetTotalDown, EntityID: entityID, Timestamp: ts, Value: float64(report.Network.TotalDown)},
{MetricName: MetricTrafficUp, EntityID: entityID, Timestamp: ts, Value: float64(trafficUp)},
{MetricName: MetricTrafficDown, EntityID: entityID, Timestamp: ts, Value: float64(trafficDown)},
{MetricName: MetricProcess, EntityID: entityID, Timestamp: ts, Value: float64(report.Process)},
{MetricName: MetricConnections, EntityID: entityID, Timestamp: ts, Value: float64(report.Connections.TCP)},
{MetricName: MetricConnectionsUDP, EntityID: entityID, Timestamp: ts, Value: float64(report.Connections.UDP)},
}
if report.GPU == nil {
return points
}
for deviceIndex, gpu := range report.GPU.DetailedInfo {
tags := map[string]string{
"device_index": strconv.Itoa(deviceIndex),
"device_name": gpu.Name,
}
points = append(points,
metric.Point{MetricName: MetricGPUMem, EntityID: entityID, Timestamp: ts, Value: float64(gpu.MemoryUsed), Tags: tags},
metric.Point{MetricName: MetricGPUMemTotal, EntityID: entityID, Timestamp: ts, Value: float64(gpu.MemoryTotal), Tags: tags},
metric.Point{MetricName: MetricGPUDeviceUsage, EntityID: entityID, Timestamp: ts, Value: gpu.Utilization, Tags: tags},
metric.Point{MetricName: MetricGPUTemp, EntityID: entityID, Timestamp: ts, Value: float64(gpu.Temperature), Tags: tags},
)
}
return points
}
func latestReportCounter(ctx context.Context, s *metric.Store, metricName, entityID string, before time.Time) (int64, bool, error) {
def, err := s.GetMetric(ctx, metricName)
if errors.Is(err, metric.ErrNotFound) {
return 0, false, nil
}
if err != nil {
return 0, false, err
}
if def.RetentionDays <= 0 {
return 0, false, nil
}
end := before.UTC().Add(-time.Nanosecond)
start := end.Add(-time.Duration(def.RetentionDays) * 24 * time.Hour)
interval := s.CompatibleSeriesInterval(start, before, 24*time.Hour)
points, err := s.Series(ctx, metric.AggregateQuery{
Query: metric.Query{
MetricName: metricName,
EntityID: entityID,
Start: start,
End: end,
Order: metric.OrderAsc,
},
Aggregation: metric.AggLast,
Interval: interval,
PreserveSeries: true,
}, before)
if err != nil {
return 0, false, err
}
if len(points) == 0 {
return 0, false, nil
}
return int64(points[len(points)-1].Value), true, nil
}
// GetLatestTrafficBefore returns the latest retained upload/download counters
// before a boundary, transparently reading raw points or rollups.
func GetLatestTrafficBefore(ctx context.Context, entityIDs []string, before time.Time) (map[string]models.Record, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
result := make(map[string]models.Record, len(entityIDs))
for _, entityID := range entityIDs {
if entityID == "" {
continue
}
up, hasUp, err := latestReportCounter(ctx, s, MetricNetTotalUp, entityID, before)
if err != nil {
return nil, err
}
down, hasDown, err := latestReportCounter(ctx, s, MetricNetTotalDown, entityID, before)
if err != nil {
return nil, err
}
if !hasUp && !hasDown {
continue
}
result[entityID] = models.Record{
Client: entityID,
Time: before.UTC().Add(-time.Nanosecond),
NetTotalUp: up,
NetTotalDown: down,
}
}
return result, nil
}
// TrafficCounterDelta returns a reset-aware increase between two cumulative
// traffic counters. After a reset, the current counter is the new increase.
func TrafficCounterDelta(current, previous int64) int64 {
if current < 0 || previous < 0 {
return 0
}
if current >= previous {
return current - previous
}
return current
}
func deleteReportTrafficState(entityID string) {
reportTrafficStates.Delete(entityID)
}
func clearReportTrafficStates() {
reportTrafficStates.Range(func(key, _ any) bool {
reportTrafficStates.Delete(key)
return true
})
}
+277
View File
@@ -0,0 +1,277 @@
package metricstore
import (
"context"
"fmt"
"testing"
"time"
"github.com/komari-monitor/komari/pkg/metric"
v1 "github.com/komari-monitor/komari/protocol/v1"
)
func useReportTestStore(t *testing.T, policy *metric.RollupPolicy) *metric.Store {
t.Helper()
ctx := context.Background()
opts := []metric.Option{metric.WithMaxOpenConns(1)}
if policy != nil {
opts = append(opts, metric.WithRollupPolicy(*policy))
}
dsn := fmt.Sprintf("file:report-%d?mode=memory&cache=shared", time.Now().UnixNano())
s, err := metric.Open(ctx, metric.SQLite(dsn, opts...))
if err != nil {
t.Fatalf("open metric store: %v", err)
}
if err := createMetricDefinitions(ctx, s); err != nil {
_ = s.Close()
t.Fatalf("create metric definitions: %v", err)
}
storeMu.Lock()
previous := store
store = s
storeMu.Unlock()
clearReportTrafficStates()
t.Cleanup(func() {
clearReportTrafficStates()
storeMu.Lock()
store = previous
storeMu.Unlock()
_ = s.Close()
})
return s
}
func TestWriteReportStoresRawMetricsAndResetAwareTraffic(t *testing.T) {
ctx := context.Background()
policy := defaultRollupPolicy()
s := useReportTestStore(t, &policy)
now := time.Now().UTC().Truncate(time.Minute)
base := now.Add(-30 * time.Minute)
report := v1.Report{
UUID: "node-a",
UpdatedAt: base,
CPU: v1.CPUReport{Usage: 12.5},
Ram: v1.RamReport{Used: 100, Total: 1000},
Swap: v1.RamReport{Used: 20, Total: 200},
Load: v1.LoadReport{Load1: 0.5},
Disk: v1.DiskReport{Used: 300, Total: 3000},
Network: v1.NetworkReport{Up: 3, Down: 4, TotalUp: 100, TotalDown: 200},
Process: 7,
Connections: v1.ConnectionsReport{TCP: 8, UDP: 9},
GPU: &v1.GPUDetailReport{
AverageUsage: 25,
DetailedInfo: []v1.GPUDeviceInfo{{
Name: "GPU 0", MemoryUsed: 400, MemoryTotal: 800, Utilization: 30, Temperature: 55,
}},
},
}
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write first report: %v", err)
}
report.UpdatedAt = base.Add(3 * time.Second)
report.Network.TotalUp = 150
report.Network.TotalDown = 260
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write second report: %v", err)
}
report.UpdatedAt = base.Add(6 * time.Second)
report.Network.TotalUp = 20
report.Network.TotalDown = 30
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write reset report: %v", err)
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 50, 20})
assertMetricValues(t, s, MetricTrafficDown, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 60, 30})
assertMetricValues(t, s, MetricNetTotalUp, report.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{100, 150, 20})
gpuPoints, err := s.Query(ctx, metric.Query{
MetricName: MetricGPUDeviceUsage,
EntityID: report.UUID,
Start: base.Add(-time.Second),
End: base.Add(time.Minute),
Tags: map[string]string{"device_index": "0"},
Order: metric.OrderAsc,
})
if err != nil {
t.Fatalf("query GPU points: %v", err)
}
if len(gpuPoints) != 3 || gpuPoints[0].Timestamp != base || gpuPoints[0].Tags["device_name"] != "GPU 0" {
t.Fatalf("unexpected GPU points: %#v", gpuPoints)
}
if _, err := s.Compact(ctx, now); err != nil {
t.Fatalf("compact reports: %v", err)
}
deleteReportTrafficState(report.UUID)
report.UpdatedAt = now
report.Network.TotalUp = 35
report.Network.TotalDown = 50
if _, err := WriteReport(ctx, report); err != nil {
t.Fatalf("write after restoring rollup baseline: %v", err)
}
assertMetricValues(t, s, MetricTrafficUp, report.UUID, now.Add(-time.Second), now.Add(time.Second), []float64{15})
assertMetricValues(t, s, MetricTrafficDown, report.UUID, now.Add(-time.Second), now.Add(time.Second), []float64{20})
}
func TestReportBatcherFlushesQueuedReports(t *testing.T) {
ctx := context.Background()
s := useReportTestStore(t, nil)
StartReportBatcher()
t.Cleanup(func() {
if err := StopReportBatcher(ctx); err != nil {
t.Errorf("stop report batcher: %v", err)
}
})
base := time.Now().UTC().Truncate(time.Second)
first := v1.Report{
UUID: "batched-node",
UpdatedAt: base,
CPU: v1.CPUReport{Usage: 10},
Network: v1.NetworkReport{TotalUp: 100, TotalDown: 200},
}
second := first
second.UpdatedAt = base.Add(3 * time.Second)
second.CPU.Usage = 20
second.Network.TotalUp = 150
second.Network.TotalDown = 260
if _, err := WriteReport(ctx, first); err != nil {
t.Fatalf("queue first report: %v", err)
}
if _, err := WriteReport(ctx, second); err != nil {
t.Fatalf("queue second report: %v", err)
}
points, err := s.Query(ctx, metric.Query{
MetricName: MetricCPU,
EntityID: first.UUID,
Start: base.Add(-time.Second),
End: base.Add(time.Minute),
Order: metric.OrderAsc,
})
if err != nil {
t.Fatalf("query before flush: %v", err)
}
if len(points) != 0 {
t.Fatalf("queued reports were written before flush: %#v", points)
}
if err := FlushReportBatch(ctx); err != nil {
t.Fatalf("flush report batch: %v", err)
}
assertMetricValues(t, s, MetricCPU, first.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{10, 20})
assertMetricValues(t, s, MetricTrafficUp, first.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 50})
assertMetricValues(t, s, MetricTrafficDown, first.UUID, base.Add(-time.Second), base.Add(time.Minute), []float64{0, 60})
}
func TestRecordReconstructionUsesMetricSpecificAggregation(t *testing.T) {
ctx := context.Background()
s := useReportTestStore(t, nil)
base := time.Now().UTC().Truncate(time.Minute)
entityID := "node-aggregation"
points := []metric.Point{
{MetricName: MetricCPU, EntityID: entityID, Timestamp: base.Add(time.Second), Value: 10},
{MetricName: MetricCPU, EntityID: entityID, Timestamp: base.Add(2 * time.Second), Value: 30},
{MetricName: MetricNetTotalUp, EntityID: entityID, Timestamp: base.Add(time.Second), Value: 100},
{MetricName: MetricNetTotalUp, EntityID: entityID, Timestamp: base.Add(2 * time.Second), Value: 200},
{MetricName: MetricTrafficUp, EntityID: entityID, Timestamp: base.Add(time.Second), Value: 10},
{MetricName: MetricTrafficUp, EntityID: entityID, Timestamp: base.Add(2 * time.Second), Value: 20},
}
if err := s.WriteBatch(ctx, points); err != nil {
t.Fatalf("write points: %v", err)
}
records, err := GetRecordsByClientAndTime(ctx, entityID, base, base.Add(time.Hour))
if err != nil {
t.Fatalf("reconstruct records: %v", err)
}
if len(records) != 1 {
t.Fatalf("records = %#v, want one bucket", records)
}
if records[0].Cpu != 20 || records[0].NetTotalUp != 200 || records[0].TrafficUp != 30 {
t.Fatalf("unexpected aggregation result: %#v", records[0])
}
}
func TestTrafficCounterDelta(t *testing.T) {
tests := []struct {
name string
current int64
previous int64
want int64
}{
{name: "previous zero", current: 120, previous: 0, want: 120},
{name: "monotonic counter", current: 250, previous: 200, want: 50},
{name: "unchanged counter", current: 100, previous: 100, want: 0},
{name: "counter reset", current: 15, previous: 250, want: 15},
{name: "negative current", current: -1, previous: 100, want: 0},
{name: "negative previous", current: 15, previous: -1, want: 0},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
if got := TrafficCounterDelta(test.current, test.previous); got != test.want {
t.Fatalf("TrafficCounterDelta(%d, %d) = %d, want %d", test.current, test.previous, got, test.want)
}
})
}
}
func TestWriteReportNormalizesReceiveTimeToUTC(t *testing.T) {
ctx := context.Background()
s := useReportTestStore(t, nil)
local := time.FixedZone("UTC+8", 8*60*60)
receiveTime := time.Date(2026, 7, 17, 9, 30, 0, 123456789, local)
report := v1.Report{
UUID: "utc-report",
UpdatedAt: receiveTime,
CPU: v1.CPUReport{Usage: 10},
Network: v1.NetworkReport{TotalUp: 1, TotalDown: 2},
}
saved, err := WriteReport(ctx, report)
if err != nil {
t.Fatalf("write report: %v", err)
}
if !saved.UpdatedAt.Equal(receiveTime) || saved.UpdatedAt.Location() != time.UTC {
t.Fatalf("saved receive time = %s (%s), want UTC", saved.UpdatedAt, saved.UpdatedAt.Location())
}
points, err := s.Query(ctx, metric.Query{
MetricName: MetricCPU,
EntityID: report.UUID,
Start: receiveTime.Add(-time.Nanosecond),
End: receiveTime.Add(time.Nanosecond),
})
if err != nil {
t.Fatalf("query stored point: %v", err)
}
if len(points) != 1 || points[0].Timestamp.Location() != time.UTC || points[0].Timestamp.Nanosecond() != 123456789 {
t.Fatalf("stored points = %#v, want one UTC nanosecond point", points)
}
}
func assertMetricValues(t *testing.T, s *metric.Store, metricName, entityID string, start, end time.Time, want []float64) {
t.Helper()
points, err := s.Query(context.Background(), metric.Query{
MetricName: metricName,
EntityID: entityID,
Start: start,
End: end,
Order: metric.OrderAsc,
})
if err != nil {
t.Fatalf("query %s: %v", metricName, err)
}
if len(points) != len(want) {
t.Fatalf("%s point count = %d, want %d: %#v", metricName, len(points), len(want), points)
}
for i := range want {
if points[i].Value != want[i] {
t.Fatalf("%s point %d = %v, want %v", metricName, i, points[i].Value, want[i])
}
}
}
+312
View File
@@ -0,0 +1,312 @@
package metricstore
import (
"context"
"fmt"
"log"
"strings"
"sync"
"time"
"github.com/komari-monitor/komari/pkg/config"
"github.com/komari-monitor/komari/pkg/metric"
)
// store_migration.go
//
// 面向 WebUI/API 的“存储后端迁移”支撑:把一个 metrics 源库(默认本地 SQLite
// ./data/metrics.db)中的全部指标数据搬运到当前正在运行的 metrics 目标库
// (通常是用户新配置并热重载后的 MySQL/PostgreSQL)。
//
// 与启动迁移(RunStartupMigration)的区别:
// - 启动迁移在进程启动时自动、同步执行(komari.db → 当前 metrics 目标,或上一个
// metrics 目标 → 当前目标)。
// - 此处的 store 迁移由管理员在 WebUI 中「切换数据库后」手动触发,异步执行并可
// 查询进度 / 取消。数据以 upsert 写入,可安全重复执行。
//
// 典型流程:
// 1. 管理员在设置中把 metric_db_dsn 改为 MySQL/PostgreSQL,保存 → 后端连接测试
// 通过后热重载,当前 store 切到远端(此时远端为空)。
// 2. 管理员调用 startMetricMigration(不带 source 参数),系统以上一个 metrics
// 目标(默认 SQLite metrics.db)为源,把历史数据搬运到远端。
// 3. 完成后登记目标指纹,下次启动不再重复搬运。
// StoreMigrationProgress 描述一次 store-to-store 迁移的实时进度。
type StoreMigrationProgress struct {
Status string `json:"status"` // idle, running, completed, failed, canceled
SourceDriver string `json:"source_driver"` // 源库驱动
SourceDSN string `json:"source_dsn"` // 源库 DSN(脱敏后)
TargetDriver string `json:"target_driver"` // 目标库驱动
TargetDSN string `json:"target_dsn"` // 目标库 DSN(脱敏后)
TotalMetrics int `json:"total_metrics"` // 指标定义总数
CurrentMetric string `json:"current_metric"` // 当前正在搬运的指标名
MetricsDone int `json:"metrics_done"` // 已完成的指标数
MigratedPoints int64 `json:"migrated_points"` // 已搬运的采样点数
StartTime time.Time `json:"start_time"` // 开始时间
EndTime time.Time `json:"end_time"` // 结束时间
Error string `json:"error,omitempty"` // 错误信息
}
var (
storeMigMu sync.Mutex
storeMigProgress StoreMigrationProgress
storeMigCancel context.CancelFunc
storeMigDone chan struct{}
storeClosing bool
)
// IsStoreMigrationRunning 报告是否有 store-to-store 迁移正在运行。
func IsStoreMigrationRunning() bool {
storeMigMu.Lock()
defer storeMigMu.Unlock()
return storeMigCancel != nil
}
// GetStoreMigrationProgress 返回当前 store-to-store 迁移进度快照。
func GetStoreMigrationProgress() StoreMigrationProgress {
storeMigMu.Lock()
defer storeMigMu.Unlock()
return storeMigProgress
}
// ResolveStoreMigrationSourceFingerprint 推断本次 store 迁移的源库指纹。
//
// 优先级:
// 1. 显式传入的 driver+dsn(driver 可留空,交由 DSN 推断)。
// 2. 已保存的上一个 metrics 目标指纹(MigrationTargetKey)。
// 3. 默认 SQLite(./data/metrics.db)。
func ResolveStoreMigrationSourceFingerprint(driver, dsn string) string {
dsn = strings.TrimSpace(dsn)
if dsn != "" {
d := ResolveDriverFromConfig(driver, dsn)
return fmt.Sprintf("%s|%s", d, dsn)
}
if saved, _ := config.GetAs[string](MigrationTargetKey, ""); saved != "" {
return saved
}
return defaultSQLiteFingerprint()
}
// StartStoreMigration 异步启动一次 store-to-store 迁移:把源库(由 sourceDriver +
// sourceDSN 指定,留空则自动推断)的全部指标数据搬运到当前运行中的 metrics 目标库。
//
// 返回错误表示“未能启动”(例如已有迁移在跑、源与目标相同、目标未初始化等);
// 迁移过程中的错误通过 GetStoreMigrationProgress().Status == "failed" 与 Error 暴露。
func StartStoreMigration(sourceDriver, sourceDSN string) error {
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
return fmt.Errorf("failed to load metric store config: %w", err)
}
targetFP := targetFingerprint(cfg)
sourceFP := ResolveStoreMigrationSourceFingerprint(sourceDriver, sourceDSN)
if sourceFP == targetFP {
return fmt.Errorf("source and target metrics database are the same; nothing to migrate")
}
srcCfg, err := configFromFingerprint(sourceFP, cfg)
if err != nil {
return fmt.Errorf("resolve source metrics store: %w", err)
}
if !storeOperations.TryAcquire() {
return ErrStoreBusy
}
exclusiveLeaseHandedOff := false
defer func() {
if !exclusiveLeaseHandedOff {
storeOperations.Release()
}
}()
storeMu.RLock()
dst := store
activeFingerprint := storeFingerprint
storeMu.RUnlock()
if dst == nil {
return ErrStoreNotInitialized
}
if activeFingerprint != targetFP {
return fmt.Errorf("active metric store does not match the configured migration target; wait for hot reload and retry")
}
storeMigMu.Lock()
if storeClosing {
storeMigMu.Unlock()
return ErrStoreBusy
}
if storeMigCancel != nil {
storeMigMu.Unlock()
return fmt.Errorf("a store migration is already in progress")
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
storeMigCancel = cancel
storeMigDone = done
storeMigProgress = StoreMigrationProgress{
Status: "running",
SourceDriver: string(ResolveDriverFromConfig(srcCfg.Driver, srcCfg.DSN)),
SourceDSN: maskDSN(srcCfg.DSN),
TargetDriver: string(ResolveDriverFromConfig(cfg.Driver, cfg.DSN)),
TargetDSN: maskDSN(cfg.DSN),
StartTime: time.Now().UTC(),
}
storeMigMu.Unlock()
exclusiveLeaseHandedOff = true
go runStoreMigration(ctx, cancel, done, srcCfg, cfg, dst, targetFP)
return nil
}
// runStoreMigration 执行实际的搬运逻辑(在独立 goroutine 中)。
func runStoreMigration(ctx context.Context, cancel context.CancelFunc, done chan struct{}, srcCfg *MetricStoreConfig, cfg *MetricStoreConfig, dst *metric.Store, targetFP string) {
defer func() {
cancel()
storeOperations.Release()
storeMigMu.Lock()
storeMigCancel = nil
storeMigDone = nil
close(done)
storeMigMu.Unlock()
}()
src, err := openSourceStore(ctx, srcCfg)
if err != nil {
finishStoreMigration("failed", fmt.Errorf("open source metrics store: %w", err))
return
}
defer src.Close()
observe := func(currentMetric string, metricIndex, totalMetrics int, addedPoints int64) {
storeMigMu.Lock()
storeMigProgress.CurrentMetric = currentMetric
storeMigProgress.MetricsDone = metricIndex
storeMigProgress.TotalMetrics = totalMetrics
storeMigProgress.MigratedPoints += addedPoints
storeMigMu.Unlock()
}
total, err := migrateBetweenStores(ctx, src, dst, observe)
if err != nil {
if ctx.Err() != nil {
finishStoreMigration("canceled", nil)
log.Printf("[store-migration] canceled after %d points", total)
return
}
finishStoreMigration("failed", err)
return
}
// 搬运成功:登记当前目标指纹,避免下次启动重复搬运。
if err := config.Set(MigrationTargetKey, targetFP); err != nil {
log.Printf("[store-migration] failed to persist migration target fingerprint: %v", err)
}
finishStoreMigration("completed", nil)
log.Printf("[store-migration] completed via API (%d points) target_driver=%s", total, ResolveDriverFromConfig(cfg.Driver, cfg.DSN))
}
// finishStoreMigration 统一收尾:设置终态状态、结束时间与错误信息。
func finishStoreMigration(status string, err error) {
storeMigMu.Lock()
defer storeMigMu.Unlock()
storeMigProgress.Status = status
storeMigProgress.EndTime = time.Now().UTC()
if err != nil {
storeMigProgress.Error = err.Error()
}
// 完成/取消时把当前指标标记为已全部处理,便于前端进度条归位。
if status == "completed" && storeMigProgress.TotalMetrics > 0 {
storeMigProgress.MetricsDone = storeMigProgress.TotalMetrics
}
}
// CancelStoreMigration 请求取消正在运行的 store-to-store 迁移,并等待其退出。
// 由于写入是幂等 upsert,取消后可安全重新发起,不会产生重复数据。
func CancelStoreMigration() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return stopStoreMigration(ctx, true)
}
func stopStoreMigration(ctx context.Context, requireRunning bool) error {
storeMigMu.Lock()
cancel := storeMigCancel
done := storeMigDone
storeMigMu.Unlock()
if cancel == nil {
if requireRunning {
return fmt.Errorf("no store migration is currently running")
}
return nil
}
cancel()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
}
}
func stopStoreMigrationForClose(ctx context.Context) error {
storeMigMu.Lock()
storeClosing = true
cancel := storeMigCancel
done := storeMigDone
storeMigMu.Unlock()
if cancel == nil {
return nil
}
cancel()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
}
}
func clearStoreClosing() {
storeMigMu.Lock()
storeClosing = false
storeMigMu.Unlock()
}
func isStoreClosing() bool {
storeMigMu.Lock()
defer storeMigMu.Unlock()
return storeClosing
}
// maskDSN 对 DSN 做粗粒度脱敏,避免在 API 响应/日志中泄露密码。
// 处理两类常见格式:URL(scheme://user:pass@host/...)与 key=value(password=...)。
func maskDSN(dsn string) string {
dsn = strings.TrimSpace(dsn)
if dsn == "" {
return ""
}
// key=value 形式:屏蔽 password=... 段。
if strings.Contains(dsn, "password=") {
parts := strings.Fields(dsn)
for i, p := range parts {
if strings.HasPrefix(strings.ToLower(p), "password=") {
parts[i] = "password=***"
}
}
return strings.Join(parts, " ")
}
// URL / user:pass@host 形式:屏蔽 user:pass@ 中的密码。
if at := strings.LastIndex(dsn, "@"); at > 0 {
head := dsn[:at]
tail := dsn[at:]
if colon := strings.LastIndex(head, ":"); colon >= 0 {
// 保留 scheme://user,屏蔽密码。
return head[:colon] + ":***" + tail
}
}
return dsn
}
-121
View File
@@ -1,121 +0,0 @@
package models
import (
"database/sql/driver"
"fmt"
"log"
"os"
"strings"
"sync"
"time"
)
// LocalTime is a custom time type for GORM to handle time.Time correctly.
// It ensures that all times are stored and retrieved based on the application's configured timezone.
type LocalTime time.Time
// Value implements the driver.Valuer interface.
// It converts UTCTime to a TEXT format that the database understands.
// The time is formatted as a string in the application's local timezone, without timezone information.
func (t LocalTime) Value() (driver.Value, error) {
if time.Time(t).IsZero() {
return nil, nil
}
return time.Time(t).In(GetAppLocation()).Format("2006-01-02 15:04:05.0000000"), nil
}
// Scan implements the sql.Scanner interface.
// It scans a value from the database into a UTCTime object.
func (t *LocalTime) Scan(v interface{}) error {
if v == nil {
*t = LocalTime(time.Time{})
return nil
}
loc := GetAppLocation()
switch val := v.(type) {
case time.Time:
// CRITICAL FIX: When the driver reads a timezone-less string (e.g., "16:00:00"),
// it often incorrectly assumes it's UTC. We must correct this by re-interpreting
// the date and clock values in the application's actual timezone.
year, month, day := val.Date()
hour, min, sec := val.Clock()
nanosec := val.Nanosecond()
*t = LocalTime(time.Date(year, month, day, hour, min, sec, nanosec, loc))
return nil
case []byte:
return t.parseTime(string(val), loc)
case string:
return t.parseTime(val, loc)
default:
return fmt.Errorf("UTCTime scan source was not string, []byte or time.Time: %T (%v)", v, v)
}
}
// parseTime handles parsing a string into UTCTime.
func (t *LocalTime) parseTime(timeStr string, loc *time.Location) error {
timeStr = strings.TrimSpace(timeStr)
if timeStr == "" {
*t = LocalTime(time.Time{})
return nil
}
layouts := []string{
time.RFC3339Nano, time.RFC3339,
"2006-01-02 15:04:05.0000000-07:00", "2006-01-02 15:04:05-07:00",
"2006-01-02 15:04:05.0000000", "2006-01-02 15:04:05", "2006-01-02",
}
for _, layout := range layouts {
if parsedTime, err := time.ParseInLocation(layout, timeStr, loc); err == nil {
*t = LocalTime(parsedTime)
return nil
}
}
return fmt.Errorf("unable to parse time string '%s' into UTCTime", timeStr)
}
// MarshalJSON implements the json.Marshaler interface.
// Serializes UTCTime to JSON in RFC3339 format with the correct timezone offset.
func (t LocalTime) MarshalJSON() ([]byte, error) {
if time.Time(t).IsZero() {
return []byte("null"), nil
}
formattedTime := time.Time(t).In(GetAppLocation()).Format(time.RFC3339)
return []byte(fmt.Sprintf(`"%s"`, formattedTime)), nil
}
// ToTime converts UTCTime to Go's native time.Time type.
func (t LocalTime) ToTime() time.Time { return time.Time(t) }
// FromTime converts Go's native time.Time type to UTCTime.
func FromTime(t time.Time) LocalTime { return LocalTime(t) }
// Now returns the current time in the application's configured location.
func Now() LocalTime { return LocalTime(time.Now().In(GetAppLocation())) }
var (
appLocation *time.Location
locationOnce sync.Once
)
// GetAppLocation retrieves the application's global timezone from the "TZ" environment variable.
func GetAppLocation() *time.Location {
locationOnce.Do(func() {
tz := os.Getenv("TZ")
if tz == "" {
tz = "UTC"
}
loc, err := time.LoadLocation(tz)
if err != nil {
log.Printf("Warning: Failed to load timezone '%s', falling back to UTC. Error: %v", tz, err)
appLocation = time.UTC
} else {
appLocation = loc
}
time.Local = appLocation
log.Printf("Application timezone is set to '%s'.", appLocation.String())
})
return appLocation
}
+4 -2
View File
@@ -1,11 +1,13 @@
package models
import "time"
type Clipboard struct {
Id int `json:"id" gorm:"primaryKey;autoIncrement;unique"`
Text string `json:"text" gorm:"type:longtext"`
Name string `json:"name" gorm:"type:varchar(255)"`
Weight int `json:"weight" gorm:"type:int"`
Remark string `json:"remark" gorm:"type:text"`
CreatedAt LocalTime `json:"created_at"`
UpdatedAt LocalTime `json:"updated_at"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
-42
View File
@@ -1,42 +0,0 @@
package models
type Config struct {
ID uint `json:"id,omitempty" gorm:"primaryKey;autoIncrement"` // 1
Sitename string `json:"sitename" gorm:"type:varchar(100);not null"`
Description string `json:"description" gorm:"type:text"`
AllowCors bool `json:"allow_cors" gorm:"column:allow_cors;default:false"`
Theme string `json:"theme" gorm:"type:varchar(100);default:'default'"` // 主题名称,默认 'default'
PrivateSite bool `json:"private_site" gorm:"default:false"` // 是否为私有站点,默认 false
ApiKey string `json:"api_key" gorm:"type:varchar(255);default:''"`
AutoDiscoveryKey string `json:"auto_discovery_key" gorm:"type:varchar(255);default:''"` // 自动发现密钥
ScriptDomain string `json:"script_domain" gorm:"type:varchar(255);default:''"` // 自定义脚本域名
SendIpAddrToGuest bool `json:"send_ip_addr_to_guest" gorm:"default:false"` // 是否向访客页面发送 IP 地址,默认 false
EulaAccepted bool `json:"eula_accepted" gorm:"default:false"`
// GeoIP 配置
GeoIpEnabled bool `json:"geo_ip_enabled" gorm:"default:true"`
GeoIpProvider string `json:"geo_ip_provider" gorm:"type:varchar(20);default:'ip-api'"` // empty, mmdb, ip-api, geojs
// Nezha 兼容(Agent gRPC)
NezhaCompatEnabled bool `json:"nezha_compat_enabled" gorm:"default:false"`
NezhaCompatListen string `json:"nezha_compat_listen" gorm:"type:varchar(100);default:''"` // 例如 0.0.0.0:5555
// OAuth 配置
OAuthEnabled bool `json:"o_auth_enabled" gorm:"default:false"`
OAuthProvider string `json:"o_auth_provider" gorm:"type:varchar(50);default:'github'"`
DisablePasswordLogin bool `json:"disable_password_login" gorm:"default:false"`
// 自定义美化
CustomHead string `json:"custom_head" gorm:"type:longtext"`
CustomBody string `json:"custom_body" gorm:"type:longtext"`
// 通知
NotificationEnabled bool `json:"notification_enabled" gorm:"default:false"` // 通知总开关
NotificationMethod string `json:"notification_method" gorm:"type:varchar(64);default:'none'"`
NotificationTemplate string `json:"notification_template" gorm:"type:longtext;default:'{{emoji}}{{emoji}}{{emoji}}\nEvent: {{event}}\nClients: {{client}}\nMessage: {{message}}\nTime: {{time}}'"`
ExpireNotificationEnabled bool `json:"expire_notification_enabled" gorm:"default:false"` // 是否启用过期通知
ExpireNotificationLeadDays int `json:"expire_notification_lead_days" gorm:"default:7"` // 过期前多少天通知,默认7天
LoginNotification bool `json:"login_notification" gorm:"default:false"` // 登录通知
TrafficLimitPercentage float64 `json:"traffic_limit_percentage" gorm:"default:80.00"` // 流量限制百分比,默认80.00%
// Record
RecordEnabled bool `json:"record_enabled" gorm:"default:true"` // 是否启用记录功能
RecordPreserveTime int `json:"record_preserve_time" gorm:"default:720"` // 记录保留时间,单位小时,默认30天
PingRecordPreserveTime int `json:"ping_record_preserve_time" gorm:"default:24"` // Ping 记录保留时间,单位小时,默认1天
CreatedAt LocalTime
UpdatedAt LocalTime
}
+4 -2
View File
@@ -1,10 +1,12 @@
package models
import "time"
type Log struct {
ID uint `json:"id,omitempty" gorm:"primaryKey;autoIncrement"`
IP string `json:"ip" gorm:"type:varchar(45);"` // IPv4 or IPv6
UUID string `json:"uuid" gorm:"type:varchar(36);"`
Message string `json:"message" gorm:"type:text;not null"`
MsgType string `json:"msg_type" gorm:"type:varchar(20);not null"`
Time LocalTime `json:"time" gorm:"autoCreateTime;not null"`
MsgType string `json:"msg_type" gorm:"type:varchar(20);not null;index:idx_logs_msg_type_time,priority:1"`
Time time.Time `json:"time" gorm:"autoCreateTime;not null;index:idx_logs_time;index:idx_logs_msg_type_time,priority:2"`
}
@@ -8,4 +8,7 @@ const (
Login = "Login"
Alert = "Alert"
Traffic = "Traffic"
DReport = "DReport" // 日报
WReport = "WReport" // 周报
MReport = "MReport" // 月报
)
+65 -49
View File
@@ -4,42 +4,44 @@ import (
"database/sql/driver"
"encoding/json"
"fmt"
"time"
)
// Client represents a registered client device
type Client struct {
UUID string `json:"uuid,omitempty" gorm:"type:varchar(36);primaryKey"`
Token string `json:"token,omitempty" gorm:"type:varchar(255);unique;not null"`
Name string `json:"name" gorm:"type:varchar(100)"`
CpuName string `json:"cpu_name" gorm:"type:varchar(100)"`
Virtualization string `json:"virtualization" gorm:"type:varchar(50)"`
Arch string `json:"arch" gorm:"type:varchar(50)"`
CpuCores int `json:"cpu_cores" gorm:"type:int"`
OS string `json:"os" gorm:"type:varchar(100)"`
KernelVersion string `json:"kernel_version" gorm:"type:varchar(100)"`
GpuName string `json:"gpu_name" gorm:"type:varchar(100)"`
IPv4 string `json:"ipv4,omitempty" gorm:"type:varchar(100)"`
IPv6 string `json:"ipv6,omitempty" gorm:"type:varchar(100)"`
Region string `json:"region" gorm:"type:varchar(100)"`
Remark string `json:"remark,omitempty" gorm:"type:longtext"`
PublicRemark string `json:"public_remark,omitempty" gorm:"type:longtext"`
MemTotal int64 `json:"mem_total" gorm:"type:bigint"`
SwapTotal int64 `json:"swap_total" gorm:"type:bigint"`
DiskTotal int64 `json:"disk_total" gorm:"type:bigint"`
Version string `json:"version,omitempty" gorm:"type:varchar(100)"`
Weight int `json:"weight" gorm:"type:int"`
Price float64 `json:"price"`
BillingCycle int `json:"billing_cycle"`
AutoRenewal bool `json:"auto_renewal" gorm:"default:false"` // 是否自动续费
Currency string `json:"currency" gorm:"type:varchar(20);default:'$'"`
ExpiredAt LocalTime `json:"expired_at" gorm:"type:timestamp"`
Group string `json:"group" gorm:"type:varchar(100)"`
Tags string `json:"tags" gorm:"type:text"` // split by ';'
Hidden bool `json:"hidden" gorm:"default:false"`
TrafficLimit int64 `json:"traffic_limit" gorm:"type:bigint"`
TrafficLimitType string `json:"traffic_limit_type" gorm:"type:varchar(10);default:'max'"` // 流量阈值类型:sum max min up down
CreatedAt LocalTime `json:"created_at"`
UpdatedAt LocalTime `json:"updated_at"`
UUID string `json:"uuid,omitempty" gorm:"type:varchar(36);primaryKey"`
Token string `json:"token,omitempty" gorm:"type:varchar(255);unique;not null"`
Name string `json:"name" gorm:"type:varchar(100)"`
CpuName string `json:"cpu_name" gorm:"type:varchar(100)"`
Virtualization string `json:"virtualization" gorm:"type:varchar(50)"`
Arch string `json:"arch" gorm:"type:varchar(50)"`
CpuCores int `json:"cpu_cores" gorm:"type:int"`
CpuPhysicalCores int `json:"cpu_physical_cores" gorm:"type:int"`
OS string `json:"os" gorm:"type:varchar(100)"`
KernelVersion string `json:"kernel_version" gorm:"type:varchar(100)"`
GpuName string `json:"gpu_name" gorm:"type:varchar(100)"`
IPv4 string `json:"ipv4,omitempty" gorm:"type:varchar(100)"`
IPv6 string `json:"ipv6,omitempty" gorm:"type:varchar(100)"`
Region string `json:"region" gorm:"type:varchar(100)"`
Remark string `json:"remark,omitempty" gorm:"type:longtext"`
PublicRemark string `json:"public_remark,omitempty" gorm:"type:longtext"`
MemTotal int64 `json:"mem_total" gorm:"type:bigint"`
SwapTotal int64 `json:"swap_total" gorm:"type:bigint"`
DiskTotal int64 `json:"disk_total" gorm:"type:bigint"`
Version string `json:"version,omitempty" gorm:"type:varchar(100)"`
Weight int `json:"weight" gorm:"type:int"`
Price float64 `json:"price"`
BillingCycle int `json:"billing_cycle"`
AutoRenewal bool `json:"auto_renewal" gorm:"default:false"` // 是否自动续费
Currency string `json:"currency" gorm:"type:varchar(20);default:'$'"`
ExpiredAt *time.Time `json:"expired_at" gorm:"type:timestamp"`
Group string `json:"group" gorm:"type:varchar(100)"`
Tags string `json:"tags" gorm:"type:text"` // split by ';'
Hidden bool `json:"hidden" gorm:"default:false"`
TrafficLimit int64 `json:"traffic_limit" gorm:"type:bigint"`
TrafficLimitType string `json:"traffic_limit_type" gorm:"type:varchar(10);default:'max'"` // 流量阈值类型:sum max min up down
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
// User represents an authenticated user
@@ -51,8 +53,8 @@ type User struct {
SSOID string `json:"sso_id" gorm:"type:varchar(100)"` // OAuth provider's user ID
TwoFactor string `json:"two_factor,omitempty" gorm:"type:varchar(255)"` // 2FA secret
Sessions []Session `json:"sessions,omitempty" gorm:"foreignKey:UUID;references:UUID;constraint:OnDelete:CASCADE,OnUpdate:CASCADE"`
CreatedAt LocalTime `json:"created_at"`
UpdatedAt LocalTime `json:"updated_at"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
// Session manages user sessions
@@ -62,17 +64,17 @@ type Session struct {
UserAgent string `json:"user_agent" gorm:"type:text"`
Ip string `json:"ip" gorm:"type:varchar(100)"`
LoginMethod string `json:"login_method" gorm:"type:varchar(50)"`
LatestOnline LocalTime `json:"latest_online" gorm:"type:timestamp"`
LatestOnline time.Time `json:"latest_online" gorm:"type:timestamp"`
LatestUserAgent string `json:"latest_user_agent" gorm:"type:text"`
LatestIp string `json:"latest_ip" gorm:"type:varchar(100)"`
Expires LocalTime `json:"expires" gorm:"not null"`
CreatedAt LocalTime `json:"created_at"`
Expires time.Time `json:"expires" gorm:"not null"`
CreatedAt time.Time `json:"created_at"`
}
// Record logs client metrics over time
type Record struct {
Client string `json:"client" gorm:"type:varchar(36);index"`
Time LocalTime `json:"time" gorm:"index"`
Time time.Time `json:"time" gorm:"index"`
Cpu float32 `json:"cpu" gorm:"type:decimal(5,2)"` // e.g., 75.50%
Gpu float32 `json:"gpu" gorm:"type:decimal(5,2)"`
Ram int64 `json:"ram" gorm:"type:bigint"`
@@ -87,6 +89,8 @@ type Record struct {
NetOut int64 `json:"net_out" gorm:"type:bigint"`
NetTotalUp int64 `json:"net_total_up" gorm:"type:bigint"`
NetTotalDown int64 `json:"net_total_down" gorm:"type:bigint"`
TrafficUp int64 `json:"traffic_up" gorm:"type:bigint"`
TrafficDown int64 `json:"traffic_down" gorm:"type:bigint"`
Process int `json:"process"`
Connections int `json:"connections"`
ConnectionsUdp int `json:"connections_udp"`
@@ -95,14 +99,14 @@ type Record struct {
// GPURecord logs individual GPU metrics over time
type GPURecord struct {
Client string `json:"client" gorm:"type:varchar(36);index"` // 客户端UUID
Time LocalTime `json:"time" gorm:"index"` // 记录时间
DeviceIndex int `json:"device_index" gorm:"index"` // GPU设备索引 (0,1,2...)
DeviceName string `json:"device_name" gorm:"type:varchar(100)"` // GPU型号
MemTotal int64 `json:"mem_total" gorm:"type:bigint"` // 显存总量(字节)
MemUsed int64 `json:"mem_used" gorm:"type:bigint"` // 显存使用(字节)
Utilization float32 `json:"utilization" gorm:"type:decimal(5,2)"` // GPU使用率(%)
Temperature int `json:"temperature"` // GPU温度(°C)
Client string `json:"client" gorm:"type:varchar(36);index"` // 客户端UUID
Time time.Time `json:"time" gorm:"index"` // 记录时间
DeviceIndex int `json:"device_index" gorm:"index"` // GPU设备索引 (0,1,2...)
DeviceName string `json:"device_name" gorm:"type:varchar(100)"` // GPU型号
MemTotal int64 `json:"mem_total" gorm:"type:bigint"` // 显存总量(字节)
MemUsed int64 `json:"mem_used" gorm:"type:bigint"` // 显存使用(字节)
Utilization float32 `json:"utilization" gorm:"type:decimal(5,2)"` // GPU使用率(%)
Temperature int `json:"temperature"` // GPU温度(°C)
}
// StringArray represents a slice of strings stored as JSON in the database
@@ -110,9 +114,21 @@ type GPURecord struct {
type StringArray []string
func (sa *StringArray) Scan(value interface{}) error {
bytes, ok := value.([]byte)
if !ok {
return fmt.Errorf("failed to scan StringArray: value is not []byte")
var bytes []byte
switch v := value.(type) {
case nil:
*sa = StringArray{}
return nil
case []byte:
bytes = v
case string:
bytes = []byte(v)
default:
return fmt.Errorf("failed to scan StringArray: unsupported value type %T", value)
}
if len(bytes) == 0 {
*sa = StringArray{}
return nil
}
return json.Unmarshal(bytes, sa)
}
+30
View File
@@ -0,0 +1,30 @@
package models
import "testing"
func TestStringArrayScanAcceptsSQLiteTextShapes(t *testing.T) {
tests := map[string]interface{}{
"bytes": []byte(`["a","b"]`),
"text": `["a","b"]`,
"empty": "",
"nil": nil,
}
for name, input := range tests {
t.Run(name, func(t *testing.T) {
var got StringArray
if err := got.Scan(input); err != nil {
t.Fatalf("scan StringArray: %v", err)
}
if input == "" || input == nil {
if len(got) != 0 {
t.Fatalf("expected empty StringArray, got %v", got)
}
return
}
if len(got) != 2 || got[0] != "a" || got[1] != "b" {
t.Fatalf("unexpected StringArray: %v", got)
}
})
}
}
+15 -3
View File
@@ -1,13 +1,15 @@
package models
import "time"
// Notification 定义了通知相关的数据库模型
type OfflineNotification struct {
Client string `json:"client" gorm:"type:varchar(36);not null;index;unique;constraint:OnDelete:CASCADE,OnUpdate:CASCADE;foreignKey:client;references:UUID"`
ClientInfo Client `json:"client_info,omitempty" gorm:"foreignKey:Client;references:UUID"`
Enable bool `json:"enable" gorm:"type:boolean;default:false"`
//Cooldown int `json:"cooldown" gorm:"type:int;not null;default:1800"` // 冷却时间(秒),默认 30 分钟
GracePeriod int `json:"grace_period" gorm:"type:int;not null;default:180"` // 宽限期(秒),默认 3 分钟
LastNotified LocalTime `json:"last_notified"` // 上次通知时间
GracePeriod int `json:"grace_period" gorm:"type:int;not null;default:180"` // 宽限期(秒),默认 3 分钟
LastNotified *time.Time `json:"last_notified"` // 上次通知时间
}
// LoadNotification 定义了基于资源占用达标时间比的负载通知规则
@@ -19,5 +21,15 @@ type LoadNotification struct {
Threshold float32 `json:"threshold" gorm:"type:decimal(5,2);not null;default:80.00"` // 阈值百分比
Ratio float32 `json:"ratio" gorm:"type:decimal(5,2);not null;default:0.80"` // 达标时间比
Interval int `json:"interval" gorm:"type:int;not null;default:15"` // 监测间隔(分钟)
LastNotified LocalTime `json:"last_notified"` // 上次通知时间
LastNotified *time.Time `json:"last_notified"` // 上次通知时间
}
// TrafficReportNotification 定义了流量定时报告的数据库模型
type TrafficReportNotification struct {
Client string `json:"client" gorm:"type:varchar(36);not null;index;unique;constraint:OnDelete:CASCADE,OnUpdate:CASCADE;foreignKey:client;references:UUID"`
ClientInfo Client `json:"client_info,omitempty" gorm:"foreignKey:Client;references:UUID"`
Enable bool `json:"enable" gorm:"type:boolean;default:false"`
Daily bool `json:"daily" gorm:"type:boolean;default:false"` // 日报
Weekly bool `json:"weekly" gorm:"type:boolean;default:false"` // 周报
Monthly bool `json:"monthly" gorm:"type:boolean;default:false"` // 月报
}
+3 -1
View File
@@ -1,11 +1,13 @@
package models
import "time"
type PingRecord struct {
Client string `json:"client" gorm:"type:varchar(36);not null;index"`
ClientInfo Client `json:"client_info" gorm:"foreignKey:Client;references:UUID;constraint:OnDelete:CASCADE,OnUpdate:CASCADE"`
TaskId uint `json:"task_id" gorm:"not null;index"`
Task PingTask `json:"task" gorm:"foreignKey:TaskId;references:Id;constraint:OnDelete:CASCADE,OnUpdate:CASCADE;"`
Time LocalTime `json:"time" gorm:"index;not null"`
Time time.Time `json:"time" gorm:"index;not null"`
Value int `json:"value" gorm:"type:int;not null"` // Ping 值,单位毫秒
}
+4 -2
View File
@@ -1,5 +1,7 @@
package models
import "time"
type Task struct {
TaskId string `json:"task_id" gorm:"type:varchar(36);primaryKey;unique"`
Clients StringArray `json:"clients" gorm:"type:longtext"`
@@ -13,6 +15,6 @@ type TaskResult struct {
ClientInfo Client `json:"client_info" gorm:"foreignKey:Client;references:UUID;constraint:OnDelete:CASCADE,OnUpdate:CASCADE"`
Result string `json:"result" gorm:"type:longtext"`
ExitCode *int `json:"exit_code" gorm:"type:int"`
FinishedAt *LocalTime `json:"finished_at" gorm:"type:timestamp"`
CreatedAt LocalTime `json:"created_at" gorm:"type:timestamp"`
FinishedAt *time.Time `json:"finished_at" gorm:"type:timestamp"`
CreatedAt time.Time `json:"created_at" gorm:"type:timestamp"`
}
+112 -1
View File
@@ -1,5 +1,18 @@
package models
import (
"fmt"
"net/url"
"path"
"strings"
)
const (
ThemeConfigurationManaged = "managed"
ThemeConfigurationRaw = "raw"
ThemeConfigurationRedirect = "redirect"
)
// Theme represents a komari theme information
type Theme struct {
Name string `json:"name"` // 主题名称
@@ -13,7 +26,7 @@ type Theme struct {
}
type Configuration struct {
Type string `json:"type"` // managed
Type string `json:"type"` // managed raw redirect
Icon string `json:"icon"` // 图标
Name any `json:"name"`
Data any `json:"data"` // 配置数据
@@ -33,3 +46,101 @@ type ThemeConfiguration struct {
Short string `json:"short" gorm:"primaryKey;unique;not null"`
Data string `json:"data" gorm:"type:longtext" default:"{}"`
}
func (t Theme) ConfigurationType() string {
typ := strings.ToLower(strings.TrimSpace(t.Configuration.Type))
if typ == "" {
return ThemeConfigurationManaged
}
return typ
}
func (t Theme) RawHTML() (string, bool) {
if t.ConfigurationType() != ThemeConfigurationRaw {
return "", false
}
return configurationDataString(t.Configuration.Data)
}
func (t Theme) RedirectTarget() (string, bool) {
if t.ConfigurationType() != ThemeConfigurationRedirect {
return "", false
}
return NormalizeThemeRedirectTarget(t.Configuration.Data)
}
func (t Theme) ValidateConfiguration() error {
switch t.ConfigurationType() {
case ThemeConfigurationManaged:
return nil
case ThemeConfigurationRaw:
html, ok := t.RawHTML()
if !ok || strings.TrimSpace(html) == "" {
return fmt.Errorf("raw 类型主题需要在 configuration.data 中提供 HTML 字符串")
}
return nil
case ThemeConfigurationRedirect:
if _, ok := t.RedirectTarget(); !ok {
return fmt.Errorf("redirect 类型主题需要在 configuration.data 中提供站内相对路径")
}
return nil
default:
return fmt.Errorf("不支持的主题类型: %s", t.Configuration.Type)
}
}
func NormalizeThemeRedirectTarget(data any) (string, bool) {
target, ok := configurationDataString(data)
if !ok {
return "", false
}
target = strings.TrimSpace(target)
if target == "" || strings.Contains(target, "\\") || strings.HasPrefix(target, "//") {
return "", false
}
parsed, err := url.Parse(target)
if err != nil || parsed.IsAbs() || parsed.Host != "" {
return "", false
}
cleanInputPath := parsed.Path
if strings.HasPrefix(cleanInputPath, "/") {
cleanInputPath = strings.TrimLeft(cleanInputPath, "/")
} else {
for strings.HasPrefix(cleanInputPath, "../") {
cleanInputPath = strings.TrimPrefix(cleanInputPath, "../")
}
}
for _, segment := range strings.Split(cleanInputPath, "/") {
if segment == ".." {
return "", false
}
}
cleanPath := cleanInputPath
if cleanPath == "" {
cleanPath = "/"
} else {
cleanPath = path.Clean(cleanPath)
if cleanPath == "." {
cleanPath = "/"
} else {
cleanPath = "/" + strings.TrimPrefix(cleanPath, "/")
}
}
normalized := url.URL{
Path: cleanPath,
RawQuery: parsed.RawQuery,
Fragment: parsed.Fragment,
}
return normalized.String(), true
}
func configurationDataString(data any) (string, bool) {
value, ok := data.(string)
return value, ok
}
+141
View File
@@ -0,0 +1,141 @@
package models
import "testing"
func TestThemeValidateConfiguration(t *testing.T) {
tests := map[string]struct {
theme Theme
wantErr bool
}{
"managed empty data": {
theme: Theme{
Configuration: Configuration{Type: ThemeConfigurationManaged},
},
},
"missing type defaults to managed": {
theme: Theme{},
},
"raw html": {
theme: Theme{
Configuration: Configuration{
Type: ThemeConfigurationRaw,
Data: "<!doctype html><title>raw</title>",
},
},
},
"raw rejects empty html": {
theme: Theme{
Configuration: Configuration{
Type: ThemeConfigurationRaw,
Data: " ",
},
},
wantErr: true,
},
"redirect relative path": {
theme: Theme{
Configuration: Configuration{
Type: ThemeConfigurationRedirect,
Data: "/dashboard?tab=nodes",
},
},
},
"redirect rejects absolute url": {
theme: Theme{
Configuration: Configuration{
Type: ThemeConfigurationRedirect,
Data: "https://example.com",
},
},
wantErr: true,
},
"redirect rejects parent traversal": {
theme: Theme{
Configuration: Configuration{
Type: ThemeConfigurationRedirect,
Data: "/../admin",
},
},
wantErr: true,
},
"unknown type": {
theme: Theme{
Configuration: Configuration{Type: "hosted"},
},
wantErr: true,
},
}
for name, tt := range tests {
t.Run(name, func(t *testing.T) {
err := tt.theme.ValidateConfiguration()
if tt.wantErr && err == nil {
t.Fatal("expected error, got nil")
}
if !tt.wantErr && err != nil {
t.Fatalf("unexpected error: %v", err)
}
})
}
}
func TestNormalizeThemeRedirectTarget(t *testing.T) {
tests := map[string]struct {
input string
want string
ok bool
}{
"leading slash": {
input: "/nodes",
want: "/nodes",
ok: true,
},
"missing leading slash": {
input: "nodes",
want: "/nodes",
ok: true,
},
"parent prefix points to root path": {
input: "../settings",
want: "/settings",
ok: true,
},
"multiple parent prefixes point to root path": {
input: "../../settings/site",
want: "/settings/site",
ok: true,
},
"query and fragment": {
input: "nodes?id=1#cpu",
want: "/nodes?id=1#cpu",
ok: true,
},
"absolute parent traversal": {
input: "/../admin",
},
"nested parent traversal": {
input: "nodes/../admin",
},
"protocol relative": {
input: "//example.com",
},
"backslash": {
input: `\admin`,
},
"absolute url": {
input: "https://example.com",
},
}
for name, tt := range tests {
t.Run(name, func(t *testing.T) {
got, ok := NormalizeThemeRedirectTarget(tt.input)
if ok != tt.ok {
t.Fatalf("ok = %v, want %v", ok, tt.ok)
}
if got != tt.want {
t.Fatalf("target = %q, want %q", got, tt.want)
}
})
}
}
+77
View File
@@ -0,0 +1,77 @@
package models
import (
"encoding/json"
"fmt"
"strings"
"testing"
"time"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func TestTimeFieldsStoreAndSerializeAsUTCWithNanoseconds(t *testing.T) {
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
NowFunc: func() time.Time { return time.Now().UTC() },
})
if err != nil {
t.Fatalf("open database: %v", err)
}
if err := db.AutoMigrate(&Log{}); err != nil {
t.Fatalf("migrate log: %v", err)
}
stamp := time.Date(2026, 7, 17, 1, 2, 3, 123456789, time.UTC)
entry := Log{IP: "127.0.0.1", UUID: "test", Message: "test", MsgType: "info", Time: stamp}
if err := db.Create(&entry).Error; err != nil {
t.Fatalf("create log: %v", err)
}
var loaded Log
if err := db.First(&loaded, entry.ID).Error; err != nil {
t.Fatalf("load log: %v", err)
}
if !loaded.Time.Equal(stamp) || loaded.Time.Nanosecond() != stamp.Nanosecond() {
t.Fatalf("loaded time = %s, want %s", loaded.Time, stamp)
}
encoded, err := json.Marshal(loaded)
if err != nil {
t.Fatalf("marshal log: %v", err)
}
if !strings.Contains(string(encoded), `"time":"2026-07-17T01:02:03.123456789Z"`) {
t.Fatalf("unexpected JSON: %s", encoded)
}
}
func TestNullableTimeFieldReadsAndSerializesNull(t *testing.T) {
dsn := fmt.Sprintf("file:%s?mode=memory&cache=shared", strings.ReplaceAll(t.Name(), "/", "_"))
db, err := gorm.Open(sqlite.Open(dsn), &gorm.Config{Logger: logger.Default.LogMode(logger.Silent)})
if err != nil {
t.Fatalf("open database: %v", err)
}
if err := db.AutoMigrate(&Client{}); err != nil {
t.Fatalf("migrate client: %v", err)
}
if err := db.Exec(`INSERT INTO clients (uuid, token, expired_at) VALUES (?, ?, NULL)`, "nullable", "token").Error; err != nil {
t.Fatalf("insert client: %v", err)
}
var client Client
if err := db.First(&client, "uuid = ?", "nullable").Error; err != nil {
t.Fatalf("load client: %v", err)
}
if client.ExpiredAt != nil {
t.Fatalf("expired_at = %s, want nil", client.ExpiredAt)
}
encoded, err := json.Marshal(client)
if err != nil {
t.Fatalf("marshal client: %v", err)
}
if !strings.Contains(string(encoded), `"expired_at":null`) {
t.Fatalf("unexpected JSON: %s", encoded)
}
}
+144
View File
@@ -0,0 +1,144 @@
package notification
import (
"fmt"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
"gorm.io/gorm/clause"
)
func validateTrafficReportNotification(notification models.TrafficReportNotification) error {
if notification.Client == "" {
return fmt.Errorf("client UUID cannot be empty")
}
if notification.Enable && !notification.Daily && !notification.Weekly && !notification.Monthly {
return fmt.Errorf("at least one cadence must be selected when enabling traffic reports")
}
return nil
}
func ValidateTrafficReportNotifications(notifications []models.TrafficReportNotification) error {
for _, notification := range notifications {
if err := validateTrafficReportNotification(notification); err != nil {
return err
}
}
return nil
}
func buildEnabledTrafficReportNotifications(uuids []string, existing []models.TrafficReportNotification) ([]models.TrafficReportNotification, error) {
if len(uuids) == 0 {
return nil, fmt.Errorf("at least one client UUID is required")
}
existingByClient := make(map[string]models.TrafficReportNotification, len(existing))
for _, notification := range existing {
existingByClient[notification.Client] = notification
}
notifications := make([]models.TrafficReportNotification, 0, len(uuids))
for _, uuid := range uuids {
if uuid == "" {
return nil, fmt.Errorf("client UUID cannot be empty")
}
existingNotification, ok := existingByClient[uuid]
if !ok || (!existingNotification.Daily && !existingNotification.Weekly && !existingNotification.Monthly) {
return nil, fmt.Errorf("at least one cadence must be selected when enabling traffic reports")
}
notifications = append(notifications, models.TrafficReportNotification{
Client: uuid,
Enable: true,
})
}
return notifications, nil
}
// ListTrafficReportNotifications 获取所有流量定时报告配置(关联客户端信息)
func ListTrafficReportNotifications() ([]models.TrafficReportNotification, error) {
db := dbcore.GetDBInstance()
var notifications []models.TrafficReportNotification
err := db.Model(&models.TrafficReportNotification{}).Preload("ClientInfo").Find(&notifications).Error
return notifications, err
}
// EditTrafficReportNotifications 批量更新流量定时报告配置
func EditTrafficReportNotifications(notifications []models.TrafficReportNotification) error {
if err := ValidateTrafficReportNotifications(notifications); err != nil {
return err
}
db := dbcore.GetDBInstance()
return db.Model(&models.TrafficReportNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable", "daily", "weekly", "monthly"}),
}).
Select("client", "enable", "daily", "weekly", "monthly").
Create(notifications).Error
}
// EnableTrafficReportNotifications 批量启用(仅更新 enable 字段)
func EnableTrafficReportNotifications(uuids []string) error {
if len(uuids) == 0 {
return fmt.Errorf("at least one client UUID is required")
}
db := dbcore.GetDBInstance()
var existing []models.TrafficReportNotification
if err := db.Where("client IN ?", uuids).Find(&existing).Error; err != nil {
return err
}
notifications, err := buildEnabledTrafficReportNotifications(uuids, existing)
if err != nil {
return err
}
return db.Model(&models.TrafficReportNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable"}),
}).
Select("client", "enable").
Create(notifications).Error
}
// DisableTrafficReportNotifications 批量禁用
func DisableTrafficReportNotifications(uuids []string) error {
if len(uuids) == 0 {
return fmt.Errorf("at least one client UUID is required")
}
db := dbcore.GetDBInstance()
var notifications []models.TrafficReportNotification
for _, uuid := range uuids {
if uuid == "" {
return fmt.Errorf("client UUID cannot be empty")
}
notifications = append(notifications, models.TrafficReportNotification{
Client: uuid,
Enable: false,
})
}
return db.Model(&models.TrafficReportNotification{}).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "client"}},
DoUpdates: clause.AssignmentColumns([]string{"enable"}),
}).
Select("client", "enable").
Create(notifications).Error
}
// GetEnabledTrafficReportByType 获取启用了指定类型报告的客户端配置
func GetEnabledTrafficReportByType(daily, weekly, monthly bool) ([]models.TrafficReportNotification, error) {
db := dbcore.GetDBInstance()
var notifications []models.TrafficReportNotification
query := db.Model(&models.TrafficReportNotification{}).Where("enable = ?", true)
if daily {
query = query.Where("daily = ?", true)
} else if weekly {
query = query.Where("weekly = ?", true)
} else if monthly {
query = query.Where("monthly = ?", true)
}
err := query.Find(&notifications).Error
return notifications, err
}
@@ -0,0 +1,36 @@
package notification
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/komari-monitor/komari/database/models"
)
func TestValidateTrafficReportNotificationsRejectsEnabledWithoutCadence(t *testing.T) {
err := ValidateTrafficReportNotifications([]models.TrafficReportNotification{{
Client: "client-a",
Enable: true,
}})
assert.Error(t, err)
}
func TestBuildEnabledTrafficReportNotificationsRequiresExistingCadence(t *testing.T) {
_, err := buildEnabledTrafficReportNotifications([]string{"client-a"}, nil)
assert.Error(t, err)
_, err = buildEnabledTrafficReportNotifications([]string{"client-a"}, []models.TrafficReportNotification{{Client: "client-a"}})
assert.Error(t, err)
notifications, err := buildEnabledTrafficReportNotifications([]string{"client-a"}, []models.TrafficReportNotification{{
Client: "client-a",
Daily: true,
}})
require.NoError(t, err)
require.Len(t, notifications, 1)
assert.Equal(t, "client-a", notifications[0].Client)
assert.True(t, notifications[0].Enable)
}
+5
View File
@@ -23,6 +23,11 @@ func GetOidcConfigByName(name string) (*models.OidcProvider, error) {
return &config, nil
}
func DeleteOidcConfigByName(name string) error {
db := dbcore.GetDBInstance()
return db.Delete(&models.OidcProvider{}, "name = ?", name).Error
}
func SaveOidcConfig(config *models.OidcProvider) error {
db := dbcore.GetDBInstance()
if err := db.Save(config).Error; err != nil {
+15 -484
View File
@@ -1,503 +1,34 @@
package records
import (
"log"
"sort"
"strings"
"context"
"time"
"gorm.io/gorm"
"github.com/komari-monitor/komari/cmd/flags"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/metricstore"
"github.com/komari-monitor/komari/database/models"
)
func RecordOne(rec models.Record) error {
db := dbcore.GetDBInstance()
return db.Create(&rec).Error
}
func RecordGPU(rec models.GPURecord) error {
db := dbcore.GetDBInstance()
return db.Create(&rec).Error
}
// 历史监控数据已完全迁移到 metric store(默认 SQLite ./data/metrics.db,或配置的
// MySQL/PostgreSQL)。旧的 records / records_long_term / gpu_records / ping_records
// 表仅在启动一次性迁移时作为数据源读取,运行期的读写全部走 metric store。
//
// 因此本文件不再保留对旧表的任何读写、压缩(rollup)与流量增量修复逻辑:
// metric store 自行管理 rollup 与保留策略,旧表压缩已无意义。
func DeleteAll() error {
db := dbcore.GetDBInstance()
if err := db.Exec("DELETE FROM records_long_term").Error; err != nil {
return err
}
if err := db.Exec("DELETE FROM gpu_records_long_term").Error; err != nil {
return err
}
if err := db.Exec("DELETE FROM gpu_records").Error; err != nil {
return err
}
return db.Exec("DELETE FROM records").Error
return metricstore.DeleteAllRecords(context.Background())
}
// GetGPURecordsByClientAndTime 获取GPU记录数据
// GetGPURecordsByClientAndTime 获取 GPU 记录数据。
func GetGPURecordsByClientAndTime(uuid string, start, end time.Time) ([]models.GPURecord, error) {
db := dbcore.GetDBInstance()
var records []models.GPURecord
fourHoursAgo := time.Now().Add(-4*time.Hour - time.Minute)
var recentRecords []models.GPURecord
recentStart := start
if end.After(fourHoursAgo) {
if recentStart.Before(fourHoursAgo) {
recentStart = fourHoursAgo
}
err := db.Where("client = ? AND time >= ? AND time <= ?", uuid, recentStart, end).
Order("time ASC, device_index ASC").Find(&recentRecords).Error
if err != nil {
log.Printf("Error fetching recent GPU records for client %s between %s and %s: %v", uuid, recentStart, end, err)
return nil, err
}
}
var longTermRecords []models.GPURecord
err := db.Table("gpu_records_long_term").Where("client = ? AND time >= ? AND time <= ?", uuid, start, end).
Order("time ASC, device_index ASC").Find(&longTermRecords).Error
if err != nil {
log.Printf("Error fetching long-term GPU records for client %s between %s and %s: %v", uuid, start, end, err)
return recentRecords, nil
}
// 合并结果 - 不再需要类型转换
records = append(records, recentRecords...)
records = append(records, longTermRecords...)
return records, nil
}
func GetLatestRecord(uuid string) (Record []models.Record, err error) {
db := dbcore.GetDBInstance()
err = db.Where("client = ?", uuid).Order("time DESC").Limit(1).Find(&Record).Error
return
}
func DeleteRecordBefore(before time.Time) error {
db := dbcore.GetDBInstance()
db.Table("records_long_term").Where("time < ?", before).Delete(&models.Record{})
db.Table("gpu_records_long_term").Where("time < ?", before).Delete(&models.GPURecord{})
db.Where("time < ?", before).Delete(&models.GPURecord{})
return db.Where("time < ?", before).Delete(&models.Record{}).Error
return metricstore.GetGPURecordsByClientAndTime(context.Background(), uuid, start, end)
}
func GetRecordsByClientAndTime(uuid string, start, end time.Time) ([]models.Record, error) {
db := dbcore.GetDBInstance()
var records []models.Record
fourHoursAgo := time.Now().Add(-4*time.Hour - time.Minute)
var recentRecords []models.Record
recentStart := start
if end.After(fourHoursAgo) {
if recentStart.Before(fourHoursAgo) {
recentStart = fourHoursAgo
}
err := db.Where("client = ? AND time >= ? AND time <= ?", uuid, recentStart, end).Order("time ASC").Find(&recentRecords).Error
if err != nil {
log.Printf("Error fetching recent records for client %s between %s and %s: %v", uuid, recentStart, end, err)
return nil, err
}
}
var long_term []models.Record
err := db.Table("records_long_term").Where("client = ? AND time >= ? AND time <= ?", uuid, start, end).Order("time ASC").Find(&long_term).Error
if err != nil {
log.Printf("Error fetching long-term records for client %s between %s and %s: %v", uuid, start, end, err)
return recentRecords, nil
}
if len(long_term) == 0 {
// 没有查到long_term,返回全部recentRecords
records = append(records, recentRecords...)
return records, nil
}
// 查到了long_term,recentRecords按15分钟分组,每组只保留一条(取最新一条)
grouped := make(map[string]models.Record)
for _, rec := range recentRecords {
key := rec.Time.ToTime().Truncate(15 * time.Minute).Format(time.RFC3339)
if old, ok := grouped[key]; !ok || rec.Time.ToTime().After(old.Time.ToTime()) {
grouped[key] = rec
}
}
var groupedList []models.Record
for _, rec := range grouped {
groupedList = append(groupedList, rec)
}
sort.Slice(groupedList, func(i, j int) bool {
return groupedList[i].Time.ToTime().Before(groupedList[j].Time.ToTime())
})
records = append(records, groupedList...)
records = append(records, long_term...)
return records, nil
return metricstore.GetRecordsByClientAndTime(context.Background(), uuid, start, end)
}
func GetAllRecords() ([]models.Record, error) {
db := dbcore.GetDBInstance()
var records []models.Record
var long_term []models.Record
err := db.Table("records").Order("time ASC").Find(&records).Error
if err != nil {
log.Printf("Error fetching all records: %v", err)
return nil, err
}
err = db.Table("records_long_term").Order("time ASC").Find(&long_term).Error
if err != nil {
log.Printf("Error fetching long-term records: %v", err)
return records, nil
}
records = append(records, long_term...)
return records, nil
}
// 压缩数据库
func CompactRecord() error {
db := dbcore.GetDBInstance()
err := migrateOldRecords(db)
if err != nil {
log.Printf("Error migrating old records: %v", err)
return err
}
err = migrateGPURecords(db)
if err != nil {
log.Printf("Error migrating GPU records: %v", err)
return err
}
if flags.DatabaseType == "sqlite" {
db.Exec("PRAGMA wal_checkpoint(PASSIVE);")
}
//log.Printf("Record compaction completed")
return nil
}
func migrateOldRecords(db *gorm.DB) error {
// 计算 4 小时前的时间
fourHoursAgo := time.Now().Add(-4 * time.Hour)
// 查询 records 表中超过 4 小时的记录
var records []models.Record
if err := db.Table("records").Where("time < ?", fourHoursAgo).Find(&records).Error; err != nil {
return err
}
if len(records) == 0 {
return nil
}
// 按 Client 和 15 分钟时间段分组,并存储所有记录以计算分位数
type groupData struct {
Cpu []float32
Gpu []float32
Load []float32
Temp []float32
Ram []int64
RamTotal []int64
Swap []int64
SwapTotal []int64
Disk []int64
DiskTotal []int64
NetIn []int64
NetOut []int64
NetTotalUp []int64
NetTotalDown []int64
Process []int
Connections []int
ConnectionsUdp []int
Uptime []int64
}
groupedRecords := make(map[string]*groupData)
for _, record := range records {
key := record.Client + "_" + record.Time.ToTime().Truncate(15*time.Minute).Format(time.RFC3339)
if _, ok := groupedRecords[key]; !ok {
groupedRecords[key] = &groupData{}
}
data := groupedRecords[key]
data.Cpu = append(data.Cpu, record.Cpu)
data.Gpu = append(data.Gpu, record.Gpu)
data.Load = append(data.Load, record.Load)
data.Temp = append(data.Temp, record.Temp)
data.Ram = append(data.Ram, record.Ram)
data.RamTotal = append(data.RamTotal, record.RamTotal)
data.Swap = append(data.Swap, record.Swap)
data.SwapTotal = append(data.SwapTotal, record.SwapTotal)
data.Disk = append(data.Disk, record.Disk)
data.DiskTotal = append(data.DiskTotal, record.DiskTotal)
data.NetIn = append(data.NetIn, record.NetIn)
data.NetOut = append(data.NetOut, record.NetOut)
data.NetTotalUp = append(data.NetTotalUp, record.NetTotalUp)
data.NetTotalDown = append(data.NetTotalDown, record.NetTotalDown)
data.Process = append(data.Process, record.Process)
data.Connections = append(data.Connections, record.Connections)
data.ConnectionsUdp = append(data.ConnectionsUdp, record.ConnectionsUdp)
//data.Uptime = append(data.Uptime, record.Uptime)
}
getPercentile := func(values []float64, percentile float64) float64 {
if len(values) == 0 {
return 0
}
sortedValues := make([]float64, len(values))
copy(sortedValues, values)
sort.Float64s(sortedValues)
index := float64(len(sortedValues)-1) * percentile
lowerIndex := int(index)
if lowerIndex >= len(sortedValues)-1 {
return sortedValues[len(sortedValues)-1]
}
frac := index - float64(lowerIndex)
return sortedValues[lowerIndex] + frac*(sortedValues[lowerIndex+1]-sortedValues[lowerIndex])
}
getIntPercentile := func(values []int64, percentile float64) int64 {
if len(values) == 0 {
return 0
}
floats := make([]float64, len(values))
for i, v := range values {
floats[i] = float64(v)
}
return int64(getPercentile(floats, percentile))
}
getInt32Percentile := func(values []int, percentile float64) int {
if len(values) == 0 {
return 0
}
floats := make([]float64, len(values))
for i, v := range values {
floats[i] = float64(v)
}
return int(getPercentile(floats, percentile))
}
return db.Transaction(func(tx *gorm.DB) error {
for key, data := range groupedRecords {
// 解析 Client 和时间
parts := strings.Split(key, "_")
clientUUID := parts[0]
timeSlot, err := time.Parse(time.RFC3339, strings.Join(parts[1:], "_"))
if err != nil {
return err
}
cpuFloats := make([]float64, len(data.Cpu))
for i, v := range data.Cpu {
cpuFloats[i] = float64(v)
}
gpuFloats := make([]float64, len(data.Gpu))
for i, v := range data.Gpu {
gpuFloats[i] = float64(v)
}
loadFloats := make([]float64, len(data.Load))
for i, v := range data.Load {
loadFloats[i] = float64(v)
}
tempFloats := make([]float64, len(data.Temp))
for i, v := range data.Temp {
tempFloats[i] = float64(v)
}
// 取高位
high_percentile := 0.7
// 检查 records_long_term 表中是否已存在相同的记录
var existingCount int64
if err := tx.Table("records_long_term").Where("client = ? AND time = ?", clientUUID, timeSlot).Count(&existingCount).Error; err != nil {
return err
}
newRec := models.Record{
Client: clientUUID,
Time: models.FromTime(timeSlot),
Cpu: float32(getPercentile(cpuFloats, high_percentile)),
Gpu: float32(getPercentile(gpuFloats, high_percentile)),
Load: float32(getPercentile(loadFloats, high_percentile)),
Temp: float32(getPercentile(tempFloats, high_percentile)),
Ram: getIntPercentile(data.Ram, high_percentile),
RamTotal: getIntPercentile(data.RamTotal, high_percentile),
Swap: getIntPercentile(data.Swap, high_percentile),
SwapTotal: getIntPercentile(data.SwapTotal, high_percentile),
Disk: getIntPercentile(data.Disk, high_percentile),
DiskTotal: getIntPercentile(data.DiskTotal, high_percentile),
NetIn: getIntPercentile(data.NetIn, 0.2),
NetOut: getIntPercentile(data.NetOut, 0.2),
NetTotalUp: getIntPercentile(data.NetTotalUp, high_percentile),
NetTotalDown: getIntPercentile(data.NetTotalDown, high_percentile),
Process: getInt32Percentile(data.Process, high_percentile),
Connections: getInt32Percentile(data.Connections, high_percentile),
ConnectionsUdp: getInt32Percentile(data.ConnectionsUdp, high_percentile),
//Uptime: getIntPercentile(data.Uptime, high_percentile),
}
// 如果记录已存在则更新,否则创建新记录
if existingCount > 0 {
if err := tx.Table("records_long_term").Where("client = ? AND time = ?", clientUUID, timeSlot).Updates(&newRec).Error; err != nil {
return err
}
} else {
if err := tx.Table("records_long_term").Create(&newRec).Error; err != nil {
return err
}
}
}
// 删除 records 表中的旧数据
if err := tx.Table("records").Where("time < ?", fourHoursAgo.Add(-1*time.Hour)).Delete(&models.Record{}).Error; err != nil {
return err
}
return nil
})
}
// migrateGPURecords 压缩GPU记录数据
func migrateGPURecords(db *gorm.DB) error {
fourHoursAgo := time.Now().Add(-4 * time.Hour)
// 查询超过4小时的GPU记录
var gpuRecords []models.GPURecord
if err := db.Where("time < ?", fourHoursAgo).Find(&gpuRecords).Error; err != nil {
return err
}
if len(gpuRecords) == 0 {
return nil
}
// 按Client + DeviceIndex + 15分钟时间窗口分组
type gpuGroupKey struct {
Client string
DeviceIndex int
TimeSlot time.Time
DeviceName string
}
type gpuGroupData struct {
MemTotal []int64
MemUsed []int64
Utilization []float32
Temperature []int
}
groupedGPUs := make(map[gpuGroupKey]*gpuGroupData)
for _, record := range gpuRecords {
key := gpuGroupKey{
Client: record.Client,
DeviceIndex: record.DeviceIndex,
TimeSlot: record.Time.ToTime().Truncate(15 * time.Minute),
DeviceName: record.DeviceName,
}
if _, ok := groupedGPUs[key]; !ok {
groupedGPUs[key] = &gpuGroupData{}
}
data := groupedGPUs[key]
data.MemTotal = append(data.MemTotal, record.MemTotal)
data.MemUsed = append(data.MemUsed, record.MemUsed)
data.Utilization = append(data.Utilization, record.Utilization)
data.Temperature = append(data.Temperature, record.Temperature)
}
// 百分位数计算函数 (复用传统Record压缩逻辑)
getPercentile := func(values []float64, percentile float64) float64 {
if len(values) == 0 {
return 0
}
sortedValues := make([]float64, len(values))
copy(sortedValues, values)
sort.Float64s(sortedValues)
index := float64(len(sortedValues)-1) * percentile
lowerIndex := int(index)
if lowerIndex >= len(sortedValues)-1 {
return sortedValues[len(sortedValues)-1]
}
frac := index - float64(lowerIndex)
return sortedValues[lowerIndex] + frac*(sortedValues[lowerIndex+1]-sortedValues[lowerIndex])
}
getIntPercentile := func(values []int64, percentile float64) int64 {
if len(values) == 0 {
return 0
}
floats := make([]float64, len(values))
for i, v := range values {
floats[i] = float64(v)
}
return int64(getPercentile(floats, percentile))
}
// 温度数据转换辅助函数
convertIntToInt64 := func(values []int) []int64 {
result := make([]int64, len(values))
for i, v := range values {
result[i] = int64(v)
}
return result
}
getFloat32Percentile := func(values []float32, percentile float64) float32 {
if len(values) == 0 {
return 0
}
floats := make([]float64, len(values))
for i, v := range values {
floats[i] = float64(v)
}
return float32(getPercentile(floats, percentile))
}
// 保持与传统Record压缩的一致性
high_percentile := 0.7
return db.Transaction(func(tx *gorm.DB) error {
for key, data := range groupedGPUs {
// 检查是否已存在记录
var existingCount int64
if err := tx.Table("gpu_records_long_term").Where("client = ? AND device_index = ? AND time = ?",
key.Client, key.DeviceIndex, key.TimeSlot).Count(&existingCount).Error; err != nil {
return err
}
compressedGPU := models.GPURecord{
Client: key.Client,
DeviceIndex: key.DeviceIndex,
Time: models.FromTime(key.TimeSlot),
DeviceName: key.DeviceName,
MemTotal: getIntPercentile(data.MemTotal, high_percentile),
MemUsed: getIntPercentile(data.MemUsed, high_percentile),
Utilization: getFloat32Percentile(data.Utilization, high_percentile),
Temperature: int(getIntPercentile(convertIntToInt64(data.Temperature), high_percentile)),
}
if existingCount > 0 {
// 更新已存在记录
if err := tx.Table("gpu_records_long_term").Where("client = ? AND device_index = ? AND time = ?",
key.Client, key.DeviceIndex, key.TimeSlot).Updates(&compressedGPU).Error; err != nil {
return err
}
} else {
// 创建新记录
if err := tx.Table("gpu_records_long_term").Create(&compressedGPU).Error; err != nil {
return err
}
}
}
// 删除已压缩的原始GPU数据
if err := tx.Where("time < ?", fourHoursAgo.Add(-1*time.Hour)).Delete(&models.GPURecord{}).Error; err != nil {
return err
}
return nil
})
// GetRecordsByTime 获取所有客户端在时间范围内的记录。
func GetRecordsByTime(start, end time.Time) ([]models.Record, error) {
return metricstore.GetRecordsByTime(context.Background(), start, end)
}
-130
View File
@@ -1,130 +0,0 @@
package records
import (
"encoding/csv"
"os"
"strconv"
"testing"
"time"
"github.com/stretchr/testify/assert"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"github.com/komari-monitor/komari/database/models"
)
var uuid = "7901508c-304f-49aa-b84f-957c33ae6f8a"
var _ = func() bool {
// 确保 Test 环境中使用 sqlite 内存数据库
return true
}()
// TestCompactRecord tests the database compaction logic by inserting 4h30m of data (one record per minute),
// then running migrateOldRecords and verifying the aggregation and cleanup.
func TestCompactRecord(t *testing.T) {
const totalMinutes = 12*60 + 30
now := time.Now()
threshold := now.Add(-4 * time.Hour)
// 使用 sqlite 内存数据库并迁移表结构
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
assert.NoError(t, err)
assert.NoError(t, db.AutoMigrate(&models.Record{}))
assert.NoError(t, db.Table("records_long_term").AutoMigrate(&models.Record{}))
expectedGroups := make(map[time.Time]struct{})
expectedRemain := 0
// 插入数据
for i := 0; i < totalMinutes; i++ {
recTime := now.Add(-time.Duration(i) * time.Minute)
rec := models.Record{Client: uuid, Time: models.FromTime(recTime), Cpu: float32(i), Gpu: float32(i), Load: float32(i), Temp: float32(i), Ram: int64(i)}
err := db.Create(&rec).Error
assert.NoError(t, err)
if recTime.Before(threshold) {
slot := recTime.Truncate(time.Hour)
expectedGroups[slot] = struct{}{}
} else {
expectedRemain++
}
}
// 导出原始数据到 CSV
os.MkdirAll("../../data", 0755)
var origRecs []models.Record
db.Order("time desc").Find(&origRecs)
fOrig, err := os.Create("../../data/original.csv")
assert.NoError(t, err)
defer fOrig.Close()
wOrig := csv.NewWriter(fOrig)
defer wOrig.Flush()
wOrig.Write([]string{"Client", "Time", "Cpu", "Gpu", "Load", "Temp", "Ram"})
for _, r := range origRecs {
wOrig.Write([]string{
r.Client,
r.Time.ToTime().Format(time.RFC3339),
strconv.FormatFloat(float64(r.Cpu), 'f', -1, 32),
strconv.FormatFloat(float64(r.Gpu), 'f', -1, 32),
strconv.FormatFloat(float64(r.Load), 'f', -1, 32),
strconv.FormatFloat(float64(r.Temp), 'f', -1, 32),
strconv.FormatInt(r.Ram, 10),
})
}
// 运行压缩(迁移)逻辑
err = migrateOldRecords(db)
assert.NoError(t, err)
// 验证 long-term 表中的聚合记录数
var longCount int64
assert.NoError(t, db.Table("records_long_term").Count(&longCount).Error)
assert.Equal(t, int64(len(expectedGroups)), longCount)
// 验证原始表中剩余记录数
var remainCount int64
assert.NoError(t, db.Table("records").Count(&remainCount).Error)
assert.Equal(t, int64(expectedRemain), remainCount+1)
// 导出压缩后的数据到 CSV
var compRecs []models.Record
db.Table("records_long_term").Order("time desc").Find(&compRecs)
fComp, err := os.Create("../../data/compressed.csv")
assert.NoError(t, err)
defer fComp.Close()
wComp := csv.NewWriter(fComp)
defer wComp.Flush()
wComp.Write([]string{"Client", "Time", "Cpu", "Gpu", "Load", "Temp", "Ram"})
for _, r := range compRecs {
wComp.Write([]string{
r.Client,
r.Time.ToTime().Format(time.RFC3339),
strconv.FormatFloat(float64(r.Cpu), 'f', -1, 32),
strconv.FormatFloat(float64(r.Gpu), 'f', -1, 32),
strconv.FormatFloat(float64(r.Load), 'f', -1, 32),
strconv.FormatFloat(float64(r.Temp), 'f', -1, 32),
strconv.FormatInt(r.Ram, 10),
})
}
db.Table("records").Order("time desc").Find(&compRecs)
fComp, err = os.Create("../../data/compressed_records.csv")
assert.NoError(t, err)
defer fComp.Close()
wComp = csv.NewWriter(fComp)
defer wComp.Flush()
wComp.Write([]string{"Client", "Time", "Cpu", "Gpu", "Load", "Temp", "Ram"})
for _, r := range compRecs {
wComp.Write([]string{
r.Client,
r.Time.ToTime().Format(time.RFC3339),
strconv.FormatFloat(float64(r.Cpu), 'f', -1, 32),
strconv.FormatFloat(float64(r.Gpu), 'f', -1, 32),
strconv.FormatFloat(float64(r.Load), 'f', -1, 32),
strconv.FormatFloat(float64(r.Temp), 'f', -1, 32),
strconv.FormatInt(r.Ram, 10),
})
}
}
+16 -68
View File
@@ -1,9 +1,11 @@
package tasks
import (
"context"
"time"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/metricstore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/utils"
"gorm.io/gorm"
@@ -45,6 +47,12 @@ func AddPingTask(clients []string, defaultOn bool, name string, target, task_typ
}
func DeletePingTask(id []uint) error {
// The metric store is independent from the main database, so clean it first
// to avoid leaving history that can no longer be addressed through the task.
if err := DeletePingRecords(id); err != nil {
return err
}
db := dbcore.GetDBInstance()
result := db.Where("id IN ?", id).Delete(&models.PingTask{})
if result.RowsAffected == 0 {
@@ -125,34 +133,21 @@ func UpdatePingTaskOrder(order map[uint]int) error {
return nil
}
func SavePingRecord(record models.PingRecord) error {
db := dbcore.GetDBInstance()
return db.Create(&record).Error
}
// ping 记录已完全迁移到 metric store(指标 ping.latency_ms),运行期读写全部走
// metric store,旧 ping_records 表不再参与。
func DeletePingRecordsBefore(time time.Time) error {
db := dbcore.GetDBInstance()
err := db.Where("time < ?", time).Delete(&models.PingRecord{}).Error
return err
func SavePingRecord(record models.PingRecord) error {
return metricstore.WritePingRecord(context.Background(), record)
}
func DeletePingRecords(id []uint) error {
db := dbcore.GetDBInstance()
result := db.Where("task_id IN ?", id).Delete(&models.PingRecord{})
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
return result.Error
return metricstore.DeletePingRecordsByTask(context.Background(), id)
}
func DeleteAllPingRecords() error {
db := dbcore.GetDBInstance()
result := db.Exec("DELETE FROM ping_records")
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
return result.Error
return metricstore.DeleteAllPingRecords(context.Background())
}
func ReloadPingSchedule() error {
db := dbcore.GetDBInstance()
var pingTasks []models.PingTask
@@ -200,53 +195,6 @@ func AddDefaultOnClientUUID(uuid string) error {
return nil
}
// MigrateAllClientsExpansion 启动时把旧版 default_on=true 且 clients 为空的任务展开为当前所有客户端 UUID。
// 迁移后 clients 始终是显式列表,调度路径不再依赖 default_on。
func MigrateAllClientsExpansion() error {
db := dbcore.GetDBInstance()
var tasks []models.PingTask
if err := db.Where("all_clients = ?", true).Find(&tasks).Error; err != nil {
return err
}
if len(tasks) == 0 {
return nil
}
var clients []models.Client
if err := db.Select("uuid").Find(&clients).Error; err != nil {
return err
}
if len(clients) == 0 {
return nil
}
allUUIDs := make(models.StringArray, 0, len(clients))
for _, c := range clients {
if c.UUID != "" {
allUUIDs = append(allUUIDs, c.UUID)
}
}
for _, task := range tasks {
if len(task.Clients) > 0 {
continue
}
if err := db.Model(&models.PingTask{}).Where("id = ?", task.Id).Update("clients", allUUIDs).Error; err != nil {
return err
}
}
return nil
}
func GetPingRecords(uuid string, taskId int, start, end time.Time) ([]models.PingRecord, error) {
db := dbcore.GetDBInstance()
var records []models.PingRecord
dbQuery := db.Model(&models.PingRecord{})
if uuid != "" {
dbQuery = dbQuery.Where("client = ?", uuid)
}
if taskId >= 0 {
dbQuery = dbQuery.Where("task_id = ?", uint(taskId))
}
if err := dbQuery.Where("time >= ? AND time <= ?", start, end).Order("time DESC").Find(&records).Error; err != nil {
return nil, err
}
return records, nil
return metricstore.GetPingRecords(context.Background(), uuid, taskId, start, end)
}
+4 -15
View File
@@ -26,7 +26,7 @@ func CreateTask(taskId string, clients []string, command string) error {
Result: "",
ExitCode: nil,
FinishedAt: nil,
CreatedAt: models.FromTime(time.Now()),
CreatedAt: time.Now().UTC(),
})
}
if len(taskResults) > 0 {
@@ -57,13 +57,6 @@ func GetSpecificTaskResult(taskId, clientId string) (*models.TaskResult, error)
return &result, nil
}
func GetAllTasksResultByUUID(uuid string) ([]models.TaskResult, error) {
var results []models.TaskResult
if err := dbcore.GetDBInstance().Where("client = ?", uuid).Find(&results).Error; err != nil {
return nil, err
}
return results, nil
}
func GetAllTasks() ([]models.Task, error) {
var tasks []models.Task
if err := dbcore.GetDBInstance().Find(&tasks).Error; err != nil {
@@ -79,21 +72,17 @@ func GetTaskResultsByTaskId(taskId string) ([]models.TaskResult, error) {
}
return results, nil
}
func DeleteTaskByTaskId(taskId string) error {
return dbcore.GetDBInstance().Where("task_id = ?", taskId).Delete(&models.Task{}).Error
}
func SaveTaskResult(taskId, clientId, result string, exitCode int, timestamp models.LocalTime) error {
func SaveTaskResult(taskId, clientId, result string, exitCode int, timestamp time.Time) error {
return dbcore.GetDBInstance().
Model(&models.TaskResult{}).
Where("task_id = ? AND client = ?", taskId, clientId).
Updates(map[string]interface{}{
"result": result,
"exit_code": exitCode,
"finished_at": timestamp,
"finished_at": timestamp.UTC(),
}).Error
}
func ClearTaskResultsByTimeBefore(before time.Time) error {
return dbcore.GetDBInstance().Where("created_at < ?", before.Format(time.RFC3339)).Delete(&models.TaskResult{}).Error
return dbcore.GetDBInstance().Where("created_at < ?", before.UTC()).Delete(&models.TaskResult{}).Error
}
+42
View File
@@ -0,0 +1,42 @@
package tasks
import (
"testing"
"time"
"github.com/komari-monitor/komari/cmd/flags"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
)
func TestClearTaskResultsByTimeBeforeUsesUTCTimeValue(t *testing.T) {
flags.DatabaseType = flags.DatabaseTypeSQLite
flags.DatabaseFile = "file:task_cleanup_time?mode=memory&cache=shared"
db := dbcore.GetDBInstance()
taskID := "task-cleanup-time"
if err := db.Create(&models.Task{TaskId: taskID, Command: "true"}).Error; err != nil {
t.Fatalf("create task: %v", err)
}
cutoff := time.Date(2026, 7, 17, 12, 0, 0, 123456789, time.UTC)
results := []models.TaskResult{
{TaskId: taskID, Client: "old", CreatedAt: cutoff.Add(-time.Nanosecond)},
{TaskId: taskID, Client: "boundary", CreatedAt: cutoff},
{TaskId: taskID, Client: "new", CreatedAt: cutoff.Add(time.Nanosecond)},
}
if err := db.Create(&results).Error; err != nil {
t.Fatalf("create task results: %v", err)
}
localCutoff := cutoff.In(time.FixedZone("UTC+8", 8*60*60))
if err := ClearTaskResultsByTimeBefore(localCutoff); err != nil {
t.Fatalf("clear task results: %v", err)
}
var remaining []models.TaskResult
if err := db.Where("task_id = ?", taskID).Order("created_at").Find(&remaining).Error; err != nil {
t.Fatalf("load remaining results: %v", err)
}
if len(remaining) != 2 || remaining[0].Client != "boundary" || remaining[1].Client != "new" {
t.Fatalf("remaining results = %#v, want boundary and new", remaining)
}
}
+13 -16
View File
@@ -1,6 +1,7 @@
package database
import (
"context"
"encoding/json"
"log"
"os"
@@ -8,13 +9,14 @@ import (
"strings"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/config"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/metricstore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/config"
)
func GetPublicInfo() (map[string]interface{}, error) {
cstPtr, err := config.GetManyAs[config.Legacy]()
cstPtr, err := config.GetManyAs[config.Settings]()
if err != nil {
return nil, err
}
@@ -42,15 +44,6 @@ func GetPublicInfo() (map[string]interface{}, error) {
if !hasKey("o_auth_provider") {
cst.OAuthProvider = "github"
}
if !hasKey("record_enabled") {
cst.RecordEnabled = true
}
if !hasKey("record_preserve_time") {
cst.RecordPreserveTime = 720
}
if !hasKey("ping_record_preserve_time") {
cst.PingRecordPreserveTime = 24
}
// Fallback defaults if we couldn't enumerate keys.
if allErr != nil {
@@ -61,7 +54,10 @@ func GetPublicInfo() (map[string]interface{}, error) {
cst.Description = "Komari Monitor, a simple server monitoring tool."
}
}
retention, err := metricstore.GetRetentionSummary(context.Background())
if err != nil {
return nil, err
}
db := dbcore.GetDBInstance()
tc := models.ThemeConfiguration{}
err = db.Model(&models.ThemeConfiguration{}).Where("short = ?", cst.Theme).First(&tc).Error
@@ -134,11 +130,12 @@ func GetPublicInfo() (map[string]interface{}, error) {
"oauth_enable": cst.OAuthEnabled,
"oauth_provider": cst.OAuthProvider,
"disable_password_login": cst.DisablePasswordLogin,
"allow_cors": cst.AllowCors,
"record_enabled": cst.RecordEnabled,
"record_preserve_time": cst.RecordPreserveTime,
"ping_record_preserve_time": cst.PingRecordPreserveTime,
"cors_origin_check_enabled": cst.CorsOriginCheckEnabled,
"record_enabled": retention.AllPositive, // 兼容旧版本主题
"record_preserve_time": retention.MaxDays * 24,
"ping_record_preserve_time": retention.MaxDays * 24,
"private_site": cst.PrivateSite,
"visitor_audit_enabled": cst.VisitorAuditEnabled,
"theme": cst.Theme,
"theme_settings": tc_data,
}, nil
+6 -1
View File
@@ -6,6 +6,11 @@
Komariは、サーバーのパフォーマンスを監視するためのシンプルで効率的なソリューションを提供することを目的とした、軽量の自己ホスト型サーバー監視ツールです。Webインターフェースを介してサーバーのステータスを表示し、軽量エージェントを介してデータを収集します。
> [!WARNING]
> Komariは、自己ホスト型の監視/制御プログラムであり、所有しているシステム、または管理権限を得ているシステムでのみ使用してください。Komariを武器化したり、許可なく展開、アクセス、永続化、コマンド実行、その他の不正利用に使用したりしないでください。実際の悪用リスクについては、Huntressの分析をご参照ください:[Komari C2 agent abuse](https://www.huntress.com/blog/komari-c2-agent-abuse)。
> Komariの展開および運用方法については、利用者自身が責任を負います。開発者は、無許可または不正な利用、およびその結果について責任を負いません。
> Windows端末でリモートコントロールを有効にした場合、クライアントはユーザーがログインするたびに、KomariがリモートコントロールソフトウェアであることをWindows通知で知らせます。
[ドキュメント](https://komari-document.pages.dev/) | [Telegramグループ](https://t.me/komari_monitor)
## 特徴
@@ -90,7 +95,7 @@ sudo ./install-komari.sh
git clone https://github.com/komari-monitor/komari
cd komari
```
ステップ1で生成された静的ファイルを `komari` プロジェクトのルートにある `/public/defaultTheme/dist` フォルダにコピーし、`komari-theme.json` と `preview.png`/`perview.png` を `/public/defaultTheme` にコピーします。
ステップ1で生成された静的ファイルを `komari` プロジェクトのルートにある `/web/public/defaultTheme/dist` フォルダにコピーし、`komari-theme.json` と `preview.png`/`perview.png` を `/web/public/defaultTheme` にコピーします。
```bash
go build -o komari
```
Loaded 100 of 282 files, more files were not shown because too many files have changed in this diff. Show more