Compare commits

...
21 Commits
Author SHA1 Message Date
Akizon 96cf0f79d8 插件支持分类了喵 2026-10-26 11:58:41 +08:00
Akizon77 9e5f9a3a71 让插件支持监听ws 2026-10-24 16:41:01 +08:00
Akizon 70bbceaf09 支持了新功能在pgsql和mysql的兼容 2026-10-23 15:47:20 +08:00
Akizon e30b9fdbd4 优化查询的CPU性能,但是稍微牺牲一点点内存 2026-10-23 13:33:19 +08:00
Akizon d5689efeea 优化查询性能 2026-10-23 11:07:55 +08:00
Akizon c24c4b9c7c 允许插件修改自己的内建对象 2026-10-22 15:02:05 +08:00
Akizon d50cc0a9fd JS运行时换成moejs 2026-10-22 13:20:51 +08:00
Akizon 528cea23be Merge pull request #695 from cristianchiriac/fix-ping-queue-size
fix(metricstore): give the ping queue its own capacity
2026-10-20 09:00:35 +08:00
Akizon b7042672db Merge main into PR 695 and remove added queue test 2026-10-20 08:45:10 +08:00
Akizon 7468dcb12b 感谢新的金主 MooProxy 2026-10-19 19:59:23 +08:00
Akizon dc7f4cb541 感谢新的金主 MooProxy 2026-10-19 19:58:11 +08:00
Akizon 48af63ca7b 查询时对象分配数量可以少一点 2026-10-19 17:22:10 +08:00
Akizon 3cf8445ad4 优化单次查询性能 2026-10-19 15:00:24 +08:00
Akizon b38b0b2347 让opus5.5 简化、优化1下代码 2026-10-19 11:19:34 +08:00
cristianchiriac fa5a04d8e2 fix(metricstore): give the ping queue its own capacity
2c3f228 lowered reportBatchQueueSize from 4096 to 512, and the ping queue
shared that size. A ping schedule round from many clients arrives within
a second or two, while the batcher drains every 3 seconds, so results
past the 512th were rejected with ErrPingBatchQueueFull. The latest
results, from the slowest routes, were the ones lost.

Size the ping queue with its own pingBatchQueueSize of 4096, the value
before 2c3f228. The report queue stays at 512.

Fixes #694
2026-10-17 17:41:45 +03:00
Akizon 4b24f1e196 Update README_zh-cn.md 2026-10-17 15:06:20 +08:00
Akizon 5006f55ca0 Update payment options in README
Removed WeChat Pay section from README and retained TRON Network.
2026-10-17 15:05:27 +08:00
Akizon 1b76b7ad5b js通知沿用旧版本的逻辑 2026-10-14 17:00:19 +08:00
Akizon 13cbdca545 和浏览器1起协商ws压缩算法 2026-10-10 17:32:08 +08:00
Akizon e48aa1e769 加1个通知升级提醒 2026-10-08 10:04:56 +08:00
Akizon a68704a425 js通知先加回来 2026-10-08 09:31:51 +08:00
150 changed files with 8689 additions and 5203 deletions

No files matched your search

+1 -1
View File
@@ -37,5 +37,5 @@ go.work.sum
komari
/dist/
.codex
.gocache/
/web/filemanager/data
+4 -3
View File
@@ -48,6 +48,7 @@ Interested in sponsoring Komari? Contact the developer via [email](mailto:komari
| Sponsor | Description |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <a href="https://mooproxy.net/?utm_source=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/Mooproxy-logo-transparent.png" alt="mooproxy.net" width="180"></a> | [MooProxy · Rotating residential proxies from $0.20/GB. Sign up for 500MB of free traffic and try it today!](https://mooproxy.net/?utm_source=komari) |
| <a href="https://axisnow.io/zh?utm=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/AxisNow.jpg" alt="AxisNow" width="180"></a> | [Self-Hosted Private CDN \| Subscription-Based CDN-Like Service \| A Fully Controlled, Flexible, Modular CDN Network](https://axisnow.io/zh?utm=komari) |
| <a href="https://whmcs.as211392.com/aff.php?aff=110"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/dreamcloud.png" alt="Dream Cloud" width="180"></a> | Cost-effective Asia-Pacific hosting with direct connectivity and robust DDoS protection, backed by transparent capacity claims. |
| <a href="https://sharon.io"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/sharon-networks.webp" alt="Sharon Networks" width="180"></a> | Premium China-optimized connectivity from Asia-Pacific data centers, featuring low latency, high bandwidth, and Tbps-scale local DDoS mitigation. Join the [Telegram community](https://t.me/SharonNetwork) to participate in charitable initiatives and community giveaways. |
@@ -62,6 +63,6 @@ Thanks to everyone who has contributed code, themes, plugins, documentation, tra
If Komari has been useful to you, consider buying me a coffee. Thank you for your support!
| WeChat Pay | TRON Network |
| ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- |
| ![WeChat Pay QR code](https://b2.akz.moe/awesome-pictures/%E5%BE%AE%E4%BF%A1%E8%B5%9E%E8%B5%8F%E7%A0%81.png) | ![TRON Network QR code](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
| TRON Network |
| ------------------------------------------------------------------------------------------- |
| ![TRON Network QR code](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
+4 -3
View File
@@ -48,6 +48,7 @@ Docker、二进制文件、源码构建和更新说明,请参阅 [安装指南
| 赞助商 | 描述 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| <a href="https://mooproxy.net/?utm_source=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/Mooproxy-logo-transparent.png" alt="mooproxy.net" width="180"></a> | [MooProxy · 动态住宅代理IP低至 $0.20/GB,注册即送 500MB 免费流量,立即体验!](https://mooproxy.net/?utm_source=komari) |
| <a href="https://axisnow.io/zh?utm=komari"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/AxisNow.jpg" alt="AxisNow" width="180"></a> | [自建私有部署CDN \| 订阅式高仿CDN \| 自主可控、灵活组合的CDN网络](https://axisnow.io/zh?utm=komari) |
| <a href="https://whmcs.as211392.com/aff.php?aff=110"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/dreamcloud.png" alt="Dream Cloud" width="180"></a> | 极高性价比解锁直连亚太高防,真高防,不虚标,打死退款 |
| <a href="https://sharon.io"><img src="https://raw.githubusercontent.com/komari-monitor/public/refs/heads/main/images/sharon-networks.webp" alt="Sharon Networks" width="180"></a> | 亚太数据中心提供顶级的中国优化网络接入 · 低延时&高带宽&提供Tbps级本地清洗高防服务, 为您的业务保驾护航, 为您的客户提供极致体验. 加入社区 [Telegram群组](https://t.me/SharonNetwork) 可参与公益募捐或群内抽奖免费使用 |
@@ -62,6 +63,6 @@ Docker、二进制文件、源码构建和更新说明,请参阅 [安装指南
如果 Komari 对你有所帮助,欢迎请作者喝一杯奶茶。感谢你的支持!
| 微信赞赏码 | TRON Network |
| ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- |
| ![wechat](https://b2.akz.moe/awesome-pictures/%E5%BE%AE%E4%BF%A1%E8%B5%9E%E8%B5%8F%E7%A0%81.png) | ![TRON](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
| TRON Network |
| --------------------------------------------------------------------------- |
| ![TRON](https://b2.akz.moe/awesome-pictures/PixPin_2026-08-07_15-16-52.png) |
-20
View File
@@ -1,20 +0,0 @@
package flags
import "testing"
func TestNormalizeDatabaseType(t *testing.T) {
tests := map[string]string{
"": DatabaseTypeSQLite,
"sqlite": DatabaseTypeSQLite,
" SQLite ": DatabaseTypeSQLite,
"SQLITE": DatabaseTypeSQLite,
"postgres": "postgres",
" postgres": "postgres",
}
for input, want := range tests {
if got := NormalizeDatabaseType(input); got != want {
t.Fatalf("NormalizeDatabaseType(%q) = %q, want %q", input, got, want)
}
}
}
+3 -13
View File
@@ -35,21 +35,11 @@ func Enable2Fa(uuid, secret string) error {
func Verify2Fa(uuid, code string) (bool, error) {
db := dbcore.GetDBInstance()
var user models.User
err := db.Where("uuid = ?", uuid).First(&user).Error
if err != nil {
if err := db.Where("uuid = ?", uuid).First(&user).Error; err != nil {
return false, err
}
if user.TwoFactor == "" {
return false, nil // 用户未启用2FA
}
valid := totp.Validate(code, user.TwoFactor)
if !valid {
return false, nil
}
return true, nil
// 用户未启用2FA时视为校验失败
return user.TwoFactor != "" && totp.Validate(code, user.TwoFactor), nil
}
func Disable2Fa(uuid string) error {
+24 -56
View File
@@ -14,15 +14,12 @@ import (
const constantSalt = "06Wm4Jv1Hkxx"
// CheckPassword 检查密码是否正确
//
// 如果密码正确,返回用户的 UUID 和 true;否则返回空字符串和 false
func CheckPassword(username, passwd string) (uuid string, success bool) {
// CheckPassword 检查密码,正确时返回用户 UUID 和 true
func CheckPassword(username, passwd string) (uuid string, ok bool) {
db := dbcore.GetDBInstance()
var user models.User
result := db.Where("username = ?", username).First(&user)
if result.Error != nil {
// 静默处理错误,不显示日志
return "", false
}
if hashPasswd(passwd) != user.Passwd {
@@ -32,7 +29,7 @@ func CheckPassword(username, passwd string) (uuid string, success bool) {
}
// ForceResetPassword 强制重置用户密码
func ForceResetPassword(username, passwd string) (err error) {
func ForceResetPassword(username, passwd string) error {
db := dbcore.GetDBInstance()
result := db.Model(&models.User{}).Where("username = ?", username).Update("passwd", hashPasswd(passwd))
if result.Error != nil {
@@ -46,82 +43,54 @@ func ForceResetPassword(username, passwd string) (err error) {
// hashPasswd 对密码进行加盐哈希
func hashPasswd(passwd string) string {
saltedPassword := passwd + constantSalt
hash := sha256.New()
hash.Write([]byte(saltedPassword))
hashedPassword := base64.StdEncoding.EncodeToString(hash.Sum(nil))
return hashedPassword
h := sha256.Sum256([]byte(passwd + constantSalt))
return base64.StdEncoding.EncodeToString(h[:])
}
func CreateAccount(username, passwd string) (user models.User, err error) {
return CreateAccountWithDB(dbcore.GetDBInstance(), username, passwd)
}
func CreateAccountWithDB(db *gorm.DB, username, passwd string) (user models.User, err error) {
hashedPassword := hashPasswd(passwd)
user = models.User{
func CreateAccountWithDB(db *gorm.DB, username, passwd string) (models.User, error) {
user := models.User{
UUID: uuid.New().String(),
Username: username,
Passwd: hashedPassword,
Passwd: hashPasswd(passwd),
}
err = db.Create(&user).Error
if err != nil {
return models.User{}, err
}
return user, nil
return user, db.Create(&user).Error
}
func DeleteAccountByUsername(username string) (err error) {
return DeleteAccountByUsernameWithDB(dbcore.GetDBInstance(), username)
}
func DeleteAccountByUsernameWithDB(db *gorm.DB, username string) (err error) {
err = db.Where("username = ?", username).Delete(&models.User{}).Error
if err != nil {
return err
}
return nil
func DeleteAccountByUsernameWithDB(db *gorm.DB, username string) error {
return db.Where("username = ?", username).Delete(&models.User{}).Error
}
func GetUserByUUID(uuid string) (user models.User, err error) {
func GetUserByUUID(uuid string) (models.User, error) {
var user models.User
return user, dbcore.GetDBInstance().Where("uuid = ?", uuid).First(&user).Error
}
// GetUserBySSO 通过 SSO 信息获取用户
func GetUserBySSO(ssoID string) (models.User, error) {
db := dbcore.GetDBInstance()
err = db.Where("uuid = ?", uuid).First(&user).Error
if err != nil {
return models.User{}, err
var user models.User
if err := db.Where("sso_id = ?", ssoID).First(&user).Error; err != nil {
return models.User{}, fmt.Errorf("用户不存在:%s", ssoID)
}
return user, nil
}
// 通过 SSO 信息获取用户
func GetUserBySSO(ssoID string) (user models.User, err error) {
db := dbcore.GetDBInstance()
// 首先尝试查找已存在的用户
err = db.Where("sso_id = ?", ssoID).First(&user).Error
if err == nil {
return user, nil
}
// 如果找不到用户,返回明确的错误信息
return models.User{}, fmt.Errorf("用户不存在:%s", ssoID)
}
func BindingExternalAccount(uuid string, sso_id string) error {
db := dbcore.GetDBInstance()
err := db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", sso_id).Error
if err != nil {
return err
}
return nil
return db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", sso_id).Error
}
func UnbindExternalAccount(uuid string) error {
db := dbcore.GetDBInstance()
err := db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", "").Error
if err != nil {
return err
}
return nil
return db.Model(&models.User{}).Where("uuid = ?", uuid).Update("sso_id", "").Error
}
func UpdateUser(uuid string, name, password, sso_type *string) error {
@@ -143,8 +112,7 @@ func UpdateUser(uuid string, name, password, sso_type *string) error {
updates["sso_type"] = *sso_type
}
updates["updated_at"] = time.Now().UTC()
err := db.Model(&models.User{}).Where("uuid = ?", uuid).Updates(updates).Error
if err != nil {
if err := db.Model(&models.User{}).Where("uuid = ?", uuid).Updates(updates).Error; err != nil {
return err
}
if password != nil {
+11 -33
View File
@@ -16,13 +16,10 @@ import (
)
// GetAllSessions 获取所有会话
func GetAllSessions() (sessions []models.Session, err error) {
func GetAllSessions() ([]models.Session, error) {
db := dbcore.GetDBInstance()
err = db.Find(&sessions).Error
if err != nil {
return nil, err
}
return sessions, nil
var sessions []models.Session
return sessions, db.Find(&sessions).Error
}
// CreateSession 创建新会话
@@ -57,19 +54,14 @@ func CreateSession(uuid string, expires int, userAgent, ip, login_method string)
}
}()
err := db.Create(&sessionRecord).Error
if err != nil {
return "", err
}
return session, nil
return session, db.Create(&sessionRecord).Error
}
// GetSession 根据会话 ID 获取 UUID
func GetSession(session string) (uuid string, err error) {
func GetSession(session string) (string, error) {
db := dbcore.GetDBInstance()
var sessionRecord models.Session
err = db.Where("session = ?", session).First(&sessionRecord).Error
if err != nil {
if err := db.Where("session = ?", session).First(&sessionRecord).Error; err != nil {
return "", err
}
@@ -83,32 +75,22 @@ func GetSession(session string) (uuid string, err error) {
}
func GetUserBySession(session string) (models.User, error) {
db := dbcore.GetDBInstance()
var sessionRecord models.Session
err := db.Where("session = ?", session).First(&sessionRecord).Error
if err != nil {
if err := dbcore.GetDBInstance().Where("session = ?", session).First(&sessionRecord).Error; err != nil {
return models.User{}, err
}
return GetUserByUUID(sessionRecord.UUID)
}
// DeleteSession 删除指定会话
func DeleteSession(session string) (err error) {
func DeleteSession(session string) error {
db := dbcore.GetDBInstance()
result := db.Where("session = ?", session).Delete(&models.Session{})
if result.Error != nil {
return result.Error
}
return nil
return db.Where("session = ?", session).Delete(&models.Session{}).Error
}
func DeleteAllSessions() error {
db := dbcore.GetDBInstance()
result := db.Where("1 = 1").Delete(&models.Session{})
if result.Error != nil {
return result.Error
}
return nil
return db.Where("1 = 1").Delete(&models.Session{}).Error
}
func UpdateLatest(session, useragent, ip string) error {
@@ -122,9 +104,5 @@ func UpdateLatest(session, useragent, ip string) error {
func RemoveExpiredSessions() error {
db := dbcore.GetDBInstance()
result := db.Where("expires < ?", time.Now().UTC()).Delete(&models.Session{})
if result.Error != nil {
return result.Error
}
return nil
return db.Where("expires < ?", time.Now().UTC()).Delete(&models.Session{}).Error
}
+13 -54
View File
@@ -16,12 +16,7 @@ import (
)
func DeleteClient(clientUuid string) error {
db := dbcore.GetDBInstance()
err := db.Delete(&models.Client{}, "uuid = ?", clientUuid).Error
if err != nil {
return err
}
return nil
return dbcore.GetDBInstance().Delete(&models.Client{}, "uuid = ?", clientUuid).Error
}
func SaveClientInfo(update map[string]interface{}) error {
@@ -31,11 +26,6 @@ func SaveClientInfo(update map[string]interface{}) error {
return fmt.Errorf("invalid client UUID")
}
// 确保更新的字段不为空
if len(update) == 0 {
return fmt.Errorf("no fields to update")
}
update["updated_at"] = time.Now().UTC()
toFloat64 := func(value interface{}) (float64, bool) {
@@ -114,11 +104,7 @@ func SaveClientInfo(update map[string]interface{}) error {
return err
}
err := db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(update).Error
if err != nil {
return err
}
return nil
return db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(update).Error
}
// CreateClient 创建新客户端
@@ -135,8 +121,7 @@ func CreateClient() (clientUUID, token string, err error) {
UpdatedAt: time.Now().UTC(),
}
err = db.Create(&client).Error
if err != nil {
if err := db.Create(&client).Error; err != nil {
return "", "", err
}
if err := tasks.AddDefaultOnClientUUID(clientUUID); err != nil {
@@ -160,8 +145,7 @@ func CreateClientWithName(name string) (clientUUID, token string, err error) {
UpdatedAt: time.Now().UTC(),
}
err = db.Create(&client).Error
if err != nil {
if err := db.Create(&client).Error; err != nil {
return "", "", err
}
if err := tasks.AddDefaultOnClientUUID(clientUUID); err != nil {
@@ -170,41 +154,25 @@ func CreateClientWithName(name string) (clientUUID, token string, err error) {
return clientUUID, token, nil
}
/*
// GetAllClients 获取所有客户端配置
func getAllClients() (clients []models.Client, err error) {
db := dbcore.GetDBInstance()
err = db.Find(&clients).Error
if err != nil {
return nil, err
}
return clients, nil
}
*/
func GetClientByUUID(uuid string) (client models.Client, err error) {
db := dbcore.GetDBInstance()
err = db.Where("uuid = ?", uuid).First(&client).Error
if err != nil {
func GetClientByUUID(uuid string) (models.Client, error) {
var client models.Client
if err := dbcore.GetDBInstance().Where("uuid = ?", uuid).First(&client).Error; err != nil {
return models.Client{}, err
}
return client, nil
}
func GetClientTokenByUUID(uuid string) (token string, err error) {
db := dbcore.GetDBInstance()
func GetClientTokenByUUID(uuid string) (string, error) {
var client models.Client
err = db.Where("uuid = ?", uuid).First(&client).Error
if err != nil {
if err := dbcore.GetDBInstance().Where("uuid = ?", uuid).First(&client).Error; err != nil {
return "", err
}
return client.Token, nil
}
func GetAllClientBasicInfo() (clients []models.Client, err error) {
db := dbcore.GetDBInstance()
err = db.Find(&clients).Error
if err != nil {
func GetAllClientBasicInfo() ([]models.Client, error) {
var clients []models.Client
if err := dbcore.GetDBInstance().Find(&clients).Error; err != nil {
return nil, err
}
return clients, nil
@@ -217,11 +185,6 @@ func SaveClient(updates map[string]interface{}) error {
return fmt.Errorf("invalid client UUID")
}
// 确保更新的字段不为空
if len(updates) == 0 {
return fmt.Errorf("no fields to update")
}
if v, exists := updates["traffic_limit"]; exists {
if val, ok := v.(float64); ok {
if val < 0 || val > math.MaxInt64-1 {
@@ -254,9 +217,5 @@ func SaveClient(updates map[string]interface{}) error {
updates["updated_at"] = time.Now().UTC()
err := db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(updates).Error
if err != nil {
return err
}
return nil
return db.Model(&models.Client{}).Where("uuid = ?", clientUUID).Updates(updates).Error
}
+4 -11
View File
@@ -9,19 +9,16 @@ import (
v2 "github.com/komari-monitor/komari/protocol/v2"
)
func GetClientUUIDByToken(token string) (clientUUID string, err error) {
db := dbcore.GetDBInstance()
func GetClientUUIDByToken(token string) (string, error) {
var client models.Client
err = db.Where("token = ?", token).First(&client).Error
if err != nil {
if err := dbcore.GetDBInstance().Where("token = ?", token).First(&client).Error; err != nil {
return "", err
}
return client.UUID, nil
}
// 检查数据防止异常数据导致数据库损坏
// ReportVerify 检查上报数据,防止异常数据导致数据库损坏
func ReportVerify(report v2.Report) error {
// 防止输入不合理范围
if report.CPU.Usage < 0 || report.CPU.Usage > 100 {
return fmt.Errorf("CPU.Usage must be between 0 and 100")
}
@@ -56,28 +53,24 @@ func ReportVerify(report v2.Report) error {
}
// [int64] 防止数据溢出
// Ram 验证
if err := checkInt64("Ram.Used", report.Ram.Used); err != nil {
return err
}
if err := checkInt64("Ram.Total", report.Ram.Total); err != nil {
return err
}
// Swap 验证
if err := checkInt64("Swap.Used", report.Swap.Used); err != nil {
return err
}
if err := checkInt64("Swap.Total", report.Swap.Total); err != nil {
return err
}
// Disk 验证
if err := checkInt64("Disk.Used", report.Disk.Used); err != nil {
return err
}
if err := checkInt64("Disk.Total", report.Disk.Total); err != nil {
return err
}
// Network 验证
if err := checkInt64("Network.Up", report.Network.Up); err != nil {
return err
}
@@ -90,7 +83,7 @@ func ReportVerify(report v2.Report) error {
if err := checkInt64("Network.TotalDown", report.Network.TotalDown); err != nil {
return err
}
// 拒绝所有负数Int
// 拒绝所有负数 Int
if report.Process < 0 {
return fmt.Errorf("Process must be non-negative: %d", report.Process)
}
+15 -4
View File
@@ -186,6 +186,13 @@ const (
// versionID 是当前构建的版本标识,由 SetVersionID 在 Initialize 前注入。
var versionID string
// nil means a fresh install; an empty value means an unmarked legacy install.
var lastRunVersion *string
func LastRunVersion() *string {
return lastRunVersion
}
// dbFileExistedAtStartup 记录本次进程启动、打开数据库之前 komari.db 是否已存在,
// 用于区分“全新安装”与“从旧版升级(无版本标记)”。在 doInitialize 打开数据库
// 之前采集。
@@ -221,13 +228,17 @@ func resolveDatabaseFile() string {
//
// 备份失败不阻止启动,但打印明确错误;备份成功(或无需备份)后写入/更新版本。
func backupOnVersionUpgrade() {
if versionID == "" {
return
}
prevVersion, readErr := config.GetAs[string](SystemVersionKey)
prevVersion = strings.TrimSpace(prevVersion)
versionRecorded := readErr == nil && prevVersion != ""
lastRunVersion = nil
if versionRecorded || dbFileExistedAtStartup {
lastRunVersion = &prevVersion
}
if versionID == "" {
return
}
// 版本未变化,无需备份。
if versionRecorded && prevVersion == versionID {
+1 -5
View File
@@ -20,9 +20,5 @@ func DeleteOidcConfigByName(name string) error {
}
func SaveOidcConfig(config *models.OidcProvider) error {
db := dbcore.GetDBInstance()
if err := db.Save(config).Error; err != nil {
return err
}
return nil
return dbcore.GetDBInstance().Save(config).Error
}
+63
View File
@@ -0,0 +1,63 @@
package records
import "github.com/komari-monitor/komari/database/models"
// LoadProjection is the field subset selected by a load_type request.
type LoadProjection struct {
Cpu *float32 `json:"cpu,omitempty"`
Gpu *float32 `json:"gpu,omitempty"`
Ram *int64 `json:"ram,omitempty"`
RamTotal *int64 `json:"ram_total,omitempty"`
Swap *int64 `json:"swap,omitempty"`
SwapTotal *int64 `json:"swap_total,omitempty"`
Load *float32 `json:"load,omitempty"`
Temp *float32 `json:"temp,omitempty"`
Disk *int64 `json:"disk,omitempty"`
DiskTotal *int64 `json:"disk_total,omitempty"`
NetIn *int64 `json:"net_in,omitempty"`
NetOut *int64 `json:"net_out,omitempty"`
NetTotalUp *int64 `json:"net_total_up,omitempty"`
NetTotalDown *int64 `json:"net_total_down,omitempty"`
Process *int `json:"process,omitempty"`
Connections *int `json:"connections,omitempty"`
ConnectionsUdp *int `json:"connections_udp,omitempty"`
}
// ProjectLoad returns the load_type projection for one record.
func ProjectLoad(r models.Record, loadType string) LoadProjection {
var p LoadProjection
switch loadType {
case "cpu":
p.Cpu = ptr(r.Cpu)
case "gpu":
p.Gpu = ptr(r.Gpu)
case "ram":
p.Ram = ptr(r.Ram)
p.RamTotal = ptr(r.RamTotal)
case "swap":
p.Swap = ptr(r.Swap)
p.SwapTotal = ptr(r.SwapTotal)
case "load":
p.Load = ptr(r.Load)
case "temp":
p.Temp = ptr(r.Temp)
case "disk":
p.Disk = ptr(r.Disk)
p.DiskTotal = ptr(r.DiskTotal)
case "network":
p.NetIn = ptr(r.NetIn)
p.NetOut = ptr(r.NetOut)
p.NetTotalUp = ptr(r.NetTotalUp)
p.NetTotalDown = ptr(r.NetTotalDown)
case "process":
p.Process = ptr(r.Process)
case "connections":
p.Connections = ptr(r.Connections)
p.ConnectionsUdp = ptr(r.ConnectionsUdp)
default:
p.Cpu = ptr(r.Cpu)
}
return p
}
func ptr[T any](v T) *T { return &v }
+65 -9
View File
@@ -4,8 +4,8 @@ import (
"context"
"time"
"github.com/komari-monitor/komari/internal/metricstore"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/metricstore"
)
// 历史监控数据已完全迁移到 metric store(默认 SQLite ./data/metrics.db,或配置的
@@ -19,21 +19,77 @@ func DeleteAll() error {
return metricstore.DeleteAllRecords(context.Background())
}
// GetGPURecordsByClientAndTime 获取 GPU 记录数据。
// QueryGPURecords returns GPU records using the caller's cancellation context.
func QueryGPURecords(ctx context.Context, uuid string, start, end time.Time) ([]models.GPURecord, error) {
return metricstore.GetGPURecordsByClientAndTime(ctx, uuid, start, end)
}
// GetGPURecordsByClientAndTime is a compatibility-only wrapper without cancellation.
// New request handlers should use QueryGPURecords.
func GetGPURecordsByClientAndTime(uuid string, start, end time.Time) ([]models.GPURecord, error) {
return metricstore.GetGPURecordsByClientAndTime(context.Background(), uuid, start, end)
return QueryGPURecords(context.Background(), uuid, start, end)
}
func GetRecordsByClientAndTime(uuid string, start, end time.Time) ([]models.Record, error) {
return metricstore.GetRecordsByClientAndTime(context.Background(), uuid, start, end)
// RecordQueryOptions narrows metric selection and the response point budget.
type RecordQueryOptions = metricstore.RecordQueryOptions
// LoadTypeMetricNames maps the RPC load_type to the stored metrics that back
// that field. An empty result means "all load metrics" (including temp and
// unknown values, which stay on the compatible full-query path).
func LoadTypeMetricNames(loadType string) []string {
var names []string
switch loadType {
case "cpu":
return []string{metricstore.MetricCPU}
case "gpu":
names = []string{metricstore.MetricGPU}
case "ram":
names = []string{metricstore.MetricRAM}
case "swap":
names = []string{metricstore.MetricSwap}
case "load":
names = []string{metricstore.MetricLoad}
case "disk":
names = []string{metricstore.MetricDisk}
case "network":
names = []string{metricstore.MetricNetIn, metricstore.MetricNetOut, metricstore.MetricNetTotalUp, metricstore.MetricNetTotalDown}
case "process":
names = []string{metricstore.MetricProcess}
case "connections":
names = []string{metricstore.MetricConnections, metricstore.MetricConnectionsUDP}
default:
return nil
}
// CPU is the common presence anchor: the previous implementation built one
// record per timestamp where any load metric existed. Including CPU keeps
// those timestamps (and zero values for a missing selected metric) without
// going back to all fifteen metrics.
return append(names, metricstore.MetricCPU)
}
// GetRecordMetricMaxByClientAndTime 获取单项监控指标在各时间桶内的最大值。
// QueryRecords returns one client's records, or all clients when uuid is empty.
// Request handlers should use this context-aware entry point.
func QueryRecords(ctx context.Context, uuid string, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
if uuid == "" {
return metricstore.GetRecordsByTime(ctx, start, end, opts...)
}
return metricstore.GetRecordsByClientAndTime(ctx, uuid, start, end, opts...)
}
// GetRecordsByClientAndTime is a compatibility-only wrapper without cancellation.
// New request handlers should use QueryRecords.
func GetRecordsByClientAndTime(uuid string, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
return metricstore.GetRecordsByClientAndTime(context.Background(), uuid, start, end, opts...)
}
// GetRecordMetricMaxByClientAndTime is a compatibility-only wrapper without cancellation.
// New request handlers should use metricstore.GetRecordMetricMaxByClientAndTime.
func GetRecordMetricMaxByClientAndTime(uuid, metricName string, start, end time.Time) ([]models.Record, error) {
return metricstore.GetRecordMetricMaxByClientAndTime(context.Background(), uuid, metricName, start, end)
}
// GetRecordsByTime 获取所有客户端在时间范围内的记录。
func GetRecordsByTime(start, end time.Time) ([]models.Record, error) {
return metricstore.GetRecordsByTime(context.Background(), start, end)
// GetRecordsByTime is a compatibility-only wrapper without cancellation.
// New request handlers should use QueryRecords with an empty uuid.
func GetRecordsByTime(start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
return metricstore.GetRecordsByTime(context.Background(), start, end, opts...)
}
+5 -33
View File
@@ -2,8 +2,8 @@ package tasks
import (
"context"
"slices"
"sort"
"time"
"github.com/komari-monitor/komari/database/dbcore"
"github.com/komari-monitor/komari/database/models"
@@ -28,17 +28,8 @@ func AddPingTask(clients []string, defaultOn bool, name string, target, task_typ
if err := tx.Create(&task).Error; err != nil {
return err
}
// Append by id to avoid races between concurrent create requests.
result := tx.Model(&models.PingTask{}).Where("id = ?", task.Id).Update("weight", int(task.Id))
if result.Error != nil {
return result.Error
}
if result.RowsAffected == 0 {
return gorm.ErrRecordNotFound
}
return nil
return tx.Model(&models.PingTask{}).Where("id = ?", task.Id).Update("weight", int(task.Id)).Error
})
if err != nil {
return 0, err
@@ -138,10 +129,8 @@ func UpdatePingTaskOrder(order map[uint]int) error {
}
for _, id := range ids {
weight := order[id]
result := tx.Model(&models.PingTask{}).Where("id = ?", id).Update("weight", weight)
if result.Error != nil {
return result.Error
if err := tx.Model(&models.PingTask{}).Where("id = ?", id).Update("weight", order[id]).Error; err != nil {
return err
}
}
return nil
@@ -178,27 +167,14 @@ func ReloadPingSchedule() error {
// AddDefaultOnClientUUID 在新客户端注册后,把该 UUID 追加到所有 default_on=true 的任务的 clients 中(去重)。
func AddDefaultOnClientUUID(uuid string) error {
if uuid == "" {
return nil
}
db := dbcore.GetDBInstance()
var tasks []models.PingTask
if err := db.Where("all_clients = ?", true).Find(&tasks).Error; err != nil {
return err
}
if len(tasks) == 0 {
return nil
}
changed := false
for _, task := range tasks {
exists := false
for _, c := range task.Clients {
if c == uuid {
exists = true
break
}
}
if exists {
if slices.Contains(task.Clients, uuid) {
continue
}
next := append(models.StringArray{}, task.Clients...)
@@ -213,7 +189,3 @@ func AddDefaultOnClientUUID(uuid string) error {
}
return nil
}
func GetPingRecords(uuid string, taskId int, start, end time.Time) ([]models.PingRecord, error) {
return metricstore.GetPingRecords(context.Background(), uuid, taskId, start, end)
}
+180
View File
@@ -0,0 +1,180 @@
package tasks
import (
"math"
"sort"
"time"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/sampling"
)
// PingClientSummary is the per-client aggregate shared by ping RPC adapters.
type PingClientSummary struct {
Total int
Loss int
Min int
Max int
LossPercent float64
}
// PingTaskSummary is the per-task aggregate shared by ping RPC adapters.
type PingTaskSummary struct {
Total int
Loss int
Valid int
Min int
Max int
Avg int
Latest int
P50 int
P99 int
LossPercent float64
P99P50Ratio float64
}
// PingSummary aggregates records once so legacy and public ping handlers only
// differ in response shape.
type PingSummary struct {
Clients map[string]PingClientSummary
Tasks map[uint]PingTaskSummary
}
type pingClientAccumulator struct {
total, loss, min, max int
}
type pingTaskAccumulator struct {
total, loss, min, max, sum, valid int
latest int
latestTime time.Time
latencies []int
}
// SummarizePingRecords computes client and task aggregates in one pass.
func SummarizePingRecords(recs []models.PingRecord) PingSummary {
clients := make(map[string]*pingClientAccumulator)
taskMap := make(map[uint]*pingTaskAccumulator)
for _, r := range recs {
cs := clients[r.Client]
if cs == nil {
cs = &pingClientAccumulator{}
clients[r.Client] = cs
}
cs.total++
ts := taskMap[r.TaskId]
if ts == nil {
ts = &pingTaskAccumulator{latest: -1}
taskMap[r.TaskId] = ts
}
ts.total++
if r.Value < 0 {
cs.loss++
ts.loss++
continue
}
if cs.min == 0 || r.Value < cs.min {
cs.min = r.Value
}
if r.Value > cs.max {
cs.max = r.Value
}
ts.valid++
ts.sum += r.Value
if ts.min == 0 || r.Value < ts.min {
ts.min = r.Value
}
if r.Value > ts.max {
ts.max = r.Value
}
if ts.latestTime.IsZero() || r.Time.After(ts.latestTime) {
ts.latestTime = r.Time
ts.latest = r.Value
}
ts.latencies = append(ts.latencies, r.Value)
}
out := PingSummary{Clients: make(map[string]PingClientSummary, len(clients)), Tasks: make(map[uint]PingTaskSummary, len(taskMap))}
for client, v := range clients {
loss := 0.0
if v.total > 0 {
loss = float64(v.loss) / float64(v.total) * 100
}
out.Clients[client] = PingClientSummary{Total: v.total, Loss: v.loss, Min: v.min, Max: v.max, LossPercent: loss}
}
for id, v := range taskMap {
loss := 0.0
if v.total > 0 {
loss = float64(v.loss) / float64(v.total) * 100
}
avg := 0
if v.valid > 0 {
avg = v.sum / v.valid
}
p50, p99 := pingPercentiles(v.latencies)
ratio := 0.0
if p50 > 0 && p99 >= p50 {
adjustedBase := math.Max(math.Min(float64(p50), 50.0), 10.0)
ratio = float64(p99-p50) / adjustedBase
}
out.Tasks[id] = PingTaskSummary{
Total: v.total, Loss: v.loss, Valid: v.valid, Min: v.min, Max: v.max,
Avg: avg, Latest: v.latest, P50: p50, P99: p99, LossPercent: loss, P99P50Ratio: ratio,
}
}
return out
}
// SamplePingRecordsByTask applies the legacy ping maxCount policy: allocate
// points proportionally by task, sample each task evenly and return ascending
// by time. Statistics must be computed from the unsampled records.
func SamplePingRecordsByTask(recs []models.PingRecord, maxCount int) []models.PingRecord {
if maxCount <= 0 || len(recs) <= maxCount {
return recs
}
groups := make(map[uint][]models.PingRecord)
for _, r := range recs {
groups[r.TaskId] = append(groups[r.TaskId], r)
}
meta := make([]sampling.Group[uint], 0, len(groups))
for id, points := range groups {
sort.Slice(points, func(i, j int) bool { return points[i].Time.Before(points[j].Time) })
groups[id] = points
meta = append(meta, sampling.Group[uint]{Key: id, Length: len(points)})
}
targets := sampling.AllocateTargets(meta, maxCount)
out := make([]models.PingRecord, 0, maxCount)
for id, points := range groups {
out = append(out, sampling.SampleEvenly(points, targets[id])...)
}
sort.Slice(out, func(i, j int) bool { return out[i].Time.Before(out[j].Time) })
return out
}
func pingPercentiles(values []int) (int, int) {
if len(values) == 0 {
return 0, 0
}
sort.Ints(values)
return pingPercentile(values, 0.50), pingPercentile(values, 0.99)
}
func pingPercentile(values []int, percentile float64) int {
if len(values) == 0 {
return 0
}
if percentile <= 0 {
return values[0]
}
if percentile >= 1 {
return values[len(values)-1]
}
pos := float64(len(values)-1) * percentile
lo := int(math.Floor(pos))
hi := int(math.Ceil(pos))
if lo == hi {
return values[lo]
}
frac := pos - float64(lo)
return int(math.Round(float64(values[lo]) + (float64(values[hi])-float64(values[lo]))*frac))
}
+3 -4
View File
@@ -9,7 +9,6 @@ import (
func CreateTask(taskId string, clients []string, command string) error {
db := dbcore.GetDBInstance()
// Create a new task in the database with clients as JSON array
task := models.Task{
TaskId: taskId,
Clients: models.StringArray(clients),
@@ -29,10 +28,10 @@ func CreateTask(taskId string, clients []string, command string) error {
CreatedAt: time.Now().UTC(),
})
}
if len(taskResults) > 0 {
return db.Create(&taskResults).Error
if len(taskResults) == 0 {
return nil
}
return nil
return db.Create(&taskResults).Error
}
func GetTaskByTaskId(taskId string) (*models.Task, error) {
var task models.Task
+4 -10
View File
@@ -3,8 +3,7 @@ module github.com/komari-monitor/komari
go 1.25.0
require (
github.com/dop251/goja v0.0.0-20251008123653-cf18d89f3cf6
github.com/dop251/goja_nodejs v0.0.0-20251015164255-5e94316bedaf
github.com/Calcium-Ion/moejs v0.1.0-alpha.9
github.com/gin-gonic/gin v1.10.0
github.com/go-sql-driver/mysql v1.10.0
github.com/google/uuid v1.6.0
@@ -16,13 +15,13 @@ require (
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pquerna/otp v1.5.0
github.com/spf13/cobra v1.9.1
github.com/stretchr/testify v1.11.1
github.com/stretchr/testify v1.12.1
golang.org/x/sys v0.33.0
gorm.io/driver/sqlite v1.5.7
gorm.io/gorm v1.26.1
)
require github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217 // indirect
require go.yaml.in/yaml/v3 v3.0.5 // indirect
// Misuse of ServerConfig.PublicKeyCallback may cause authorization bypass in golang.org/x/crypto #1
// golang.org/x/crypto Vulnerable to Denial of Service (DoS) via Slow or Incomplete Key Exchange #3
@@ -30,7 +29,7 @@ require golang.org/x/crypto v0.39.0
// HTTP Proxy bypass using IPv6 Zone IDs in golang.org/x/net #2
// golang.org/x/net vulnerable to Cross-site Scripting #4
require golang.org/x/net v0.41.0 // indirect
require golang.org/x/net v0.41.0
require (
filippo.io/edwards25519 v1.2.0 // indirect
@@ -38,17 +37,13 @@ require (
github.com/bytedance/sonic v1.13.2 // indirect
github.com/bytedance/sonic/loader v0.2.4 // indirect
github.com/cloudwego/base64x v0.1.5 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dlclark/regexp2 v1.11.4 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gin-contrib/sse v1.0.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.26.0 // indirect
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
@@ -63,7 +58,6 @@ require (
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rogpeppe/go-internal v1.8.0 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
+6 -21
View File
@@ -1,7 +1,7 @@
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/Masterminds/semver/v3 v3.2.1 h1:RN9w6+7QoMeJVGyfmbcgs28Br8cvmnucEXnY0rYXWg0=
github.com/Masterminds/semver/v3 v3.2.1/go.mod h1:qvl/7zhW3nngYb5+80sSMF+FG2BjYrf8m9wsX0PNOMQ=
github.com/Calcium-Ion/moejs v0.1.0-alpha.9 h1:WY1ebChegtafhPjHSQx3Q+gfnrjqYwHewHuO9k+v7e0=
github.com/Calcium-Ion/moejs v0.1.0-alpha.9/go.mod h1:9YqHj36EV7pzV/vlEYSw49f3oggeJFdmqcjM9TmDpHs=
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc h1:biVzkmvwrH8WK8raXaxBx6fRVTlJILwEwQGL1I/ByEI=
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/bytedance/sonic v1.13.2 h1:8/H1FempDZqC4VqjptGo14QQlJx8VdZJegxs6wwfqpQ=
@@ -17,14 +17,6 @@ github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ3
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.4 h1:rPYF9/LECdNymJufQKmri9gV604RvvABwgOA8un7yAo=
github.com/dlclark/regexp2 v1.11.4/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217 h1:16iT9CBDOniJwFGPI41MbUDfEk74hFaKTqudrX8kenY=
github.com/dop251/base64dec v0.0.0-20231022112746-c6c9f9a96217/go.mod h1:eIb+f24U+eWQCIsj9D/ah+MD9UP+wdxuqzsdLD+mhGM=
github.com/dop251/goja v0.0.0-20251008123653-cf18d89f3cf6 h1:6dE1TmjqkY6tehR4A67gDNhvDtuZ54ocu7ab4K9o540=
github.com/dop251/goja v0.0.0-20251008123653-cf18d89f3cf6/go.mod h1:MxLav0peU43GgvwVgNbLAj1s/bSGboKkhuULvq/7hx4=
github.com/dop251/goja_nodejs v0.0.0-20251015164255-5e94316bedaf h1:gbmvliZnCut4NjaPSNOQlfqBoZ9C5Dpf72mHMMYhgVE=
github.com/dop251/goja_nodejs v0.0.0-20251015164255-5e94316bedaf/go.mod h1:Tb7Xxye4LX7cT3i8YLvmPMGCV92IOi4CDZvm/V8ylc0=
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
github.com/gin-contrib/sse v1.0.0 h1:y3bT1mUWUxDpW4JLQg/HnTqV4rozuW4tC9eFKTxYI9E=
@@ -39,8 +31,6 @@ github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJn
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.26.0 h1:SP05Nqhjcvz81uJaRfEV0YBSSSGMc/iMaVtFbr3Sw2k=
github.com/go-playground/validator/v10 v10.26.0/go.mod h1:I5QpIEbmr8On7W0TktmJAumgzX4CA1XNl4ZmDuVHKKo=
github.com/go-sourcemap/sourcemap v2.1.4+incompatible h1:a+iTbH5auLKxaNwQFg0B+TCYl6lbukKPc7b5x0n1s6Q=
github.com/go-sourcemap/sourcemap v2.1.4+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg=
github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw=
github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
@@ -48,8 +38,6 @@ github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PU
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 h1:FKHo8hFI3A+7w0aUQuYXQ+6EN5stWmeY/AZqtM8xk9k=
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8/go.mod h1:K1liHPHnj73Fdn/EKuT8nrFqBihUSKXoLYU0BuatOYo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
@@ -101,7 +89,6 @@ github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTK
github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
github.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pquerna/otp v1.5.0 h1:NMMR+WrmaqXU4EzdGJEE1aUUI0AMRzsp96fFFWNPwxs=
github.com/pquerna/otp v1.5.0/go.mod h1:dkJfzwRKNiegxyNb54X/3fLwhCynbMspSyWKnvi1AEg=
@@ -123,14 +110,14 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/arch v0.15.0 h1:QtOrQd0bTUnhNVNndMpLHNWrDmYzZ2KDqSrEymqInZw=
golang.org/x/arch v0.15.0/go.mod h1:JmwW7aLIoRUKgaTzhkiEFxvcEiQGyOg9BMonBJUS7EE=
golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
@@ -151,8 +138,6 @@ gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-7
View File
@@ -65,9 +65,6 @@ func targetFingerprint(cfg *MetricStoreConfig) string {
// autoMigrate 控制是否在 Open 时自动建表:正式初始化/热加载时为 true,
// 仅做连接测试时为 false(不写入 schema,避免对目标库产生副作用)。
func buildMetricConfig(cfg *MetricStoreConfig, autoMigrate bool) (metric.Config, error) {
if cfg == nil {
return metric.Config{}, fmt.Errorf("metric store config is nil")
}
driver := ResolveDriverFromConfig(cfg.Driver, cfg.DSN)
tablePrefix := cfg.TablePrefix
@@ -131,10 +128,6 @@ func defaultRollupPolicy() metric.RollupPolicy {
}
func rollupPolicyFromConfig(cfg *MetricStoreConfig) (metric.RollupPolicy, error) {
if cfg == nil {
return metric.RollupPolicy{}, fmt.Errorf("metric store config is nil")
}
minuteRetention := cfg.RollupMinuteRetentionMinutes
fiveMinuteRetention := cfg.RollupFiveMinuteRetentionMinutes
hourRetention := cfg.RollupHourRetentionHours
+110 -116
View File
@@ -10,14 +10,32 @@ import (
"github.com/komari-monitor/komari/pkg/metric"
)
// GetRecordsByClientAndTime 从 metric store 查询记录并重构为 models.Record
func GetRecordsByClientAndTime(ctx context.Context, clientUUID string, start, end time.Time) ([]models.Record, error) {
// RecordQueryOptions narrows the record reconstruction work. MetricNames
// limits which stored load metrics are queried; empty means all load metrics.
// MaxPoints is a total response-point budget; it is applied before aggregation,
// so an all-client query does not materialize thousands of points per client
// only to sample them afterwards.
type RecordQueryOptions struct {
MetricNames []string
MaxPoints int
}
// GetRecordsByClientAndTime 从 metric store 查询记录并重构为 models.Record。
// opts 为空表示兼容旧行为;有值时用于指标选择和点数预算。
func GetRecordsByClientAndTime(ctx context.Context, clientUUID string, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
return getRecordsByClientAndTimeFromSeries(ctx, s, clientUUID, start, end)
return getRecordsFromSeries(ctx, s, clientUUID, start, end, false, firstRecordOptions(opts))
}
func firstRecordOptions(opts []RecordQueryOptions) RecordQueryOptions {
if len(opts) == 0 {
return RecordQueryOptions{}
}
return opts[0]
}
// GetRecordMetricMaxByClientAndTime 查询单项监控指标在各时间桶内的最大值。
@@ -30,28 +48,17 @@ func GetRecordMetricMaxByClientAndTime(ctx context.Context, clientUUID, recordMe
return getRecordMetricMaxByClientAndTimeFromSeries(ctx, s, clientUUID, recordMetric, start, end)
}
// GetRecordsByTime 从 metric store 查询所有客户端在时间范围内的记录
func GetRecordsByTime(ctx context.Context, start, end time.Time) ([]models.Record, error) {
// GetRecordsByTime 从 metric store 查询所有客户端在时间范围内的记录。
// opts 为空表示兼容旧行为;有值时用于指标选择和总点数预算。
func GetRecordsByTime(ctx context.Context, start, end time.Time, opts ...RecordQueryOptions) ([]models.Record, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
interval := recordSeriesInterval(s, start, end, time.Now().UTC())
entityIDs, err := listRecordEntityIDs(ctx, s, start, end, interval)
if err != nil {
return nil, err
}
var records []models.Record
for _, entityID := range entityIDs {
items, err := getRecordsByClientAndTimeFromSeries(ctx, s, entityID, start, end)
if err != nil {
return nil, err
}
records = append(records, items...)
}
sortRecords(records)
return records, nil
// Query all entities together. Entity-only grouping retains the old per-client
// aggregation across tags without repeating the tier scan for every client.
return getRecordsFromSeries(ctx, s, "", start, end, true, firstRecordOptions(opts))
}
type recordSeriesKey struct {
@@ -59,25 +66,58 @@ type recordSeriesKey struct {
ts int64
}
func getRecordsByClientAndTimeFromSeries(ctx context.Context, s *metric.Store, clientUUID string, start, end time.Time) ([]models.Record, error) {
now := time.Now().UTC()
interval := recordSeriesInterval(s, start, end, now)
recordMap := make(map[recordSeriesKey]*models.Record)
func getRecordsFromSeries(ctx context.Context, s *metric.Store, clientUUID string, start, end time.Time, allEntities bool, opts RecordQueryOptions) ([]models.Record, error) {
metricNames := opts.MetricNames
if len(metricNames) == 0 {
metricNames = loadRecordMetricNames
}
specs := make([]MetricSpec, 0, len(metricNames))
for _, name := range metricNames {
specs = append(specs, MetricSpec{Name: name, Aggregations: []metric.Aggregation{recordMetricAggregation(name)}})
}
entities := EntityScope{All: allEntities}
grouping := GroupMerged
if allEntities {
grouping = GroupEntity
} else {
entities.IDs = []string{clientUUID}
}
rounding := RoundFloor
if opts.MaxPoints > 0 {
rounding = RoundCeil
}
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QuerySeries, Source: SourceRollup,
OmitTags: true, OmitLabels: true, CompactDefinitions: true,
Entities: entities, Specs: specs, Start: start, End: end,
Order: metric.OrderAsc, Grouping: grouping,
Budget: PointBudget{PerSeries: 500, Total: opts.MaxPoints},
Rounding: rounding, AllowMissing: true,
})
if err != nil {
return nil, fmt.Errorf("failed to query metrics: %w", err)
}
return assembleRecordsFromSeries(loaded.Values, clientUUID), nil
}
for _, metricName := range loadRecordMetricNames {
points, err := s.Series(ctx, metric.AggregateQuery{
Query: metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
},
Aggregation: recordMetricAggregation(metricName),
Interval: interval,
}, now)
if err != nil {
return nil, fmt.Errorf("failed to query metric %s: %w", metricName, err)
// assembleRecordsFromSeries unions metric buckets without allocating a record per key.
func assembleRecordsFromSeries(values map[string]map[metric.Aggregation][]metric.AggregatePoint, clientUUID string) []models.Record {
// Aligned metrics normally share buckets: the largest selected metric is a
// conservative capacity estimate, not the sum of the metric fanout. Missing
// buckets can grow the union beyond it (and duplicate seconds can shrink it).
capacity := 0
for name, aggregations := range values {
if n := len(aggregations[recordMetricAggregation(name)]); n > capacity {
capacity = n
}
}
records := make([]models.Record, 0, capacity)
recordIndex := make(map[recordSeriesKey]int, capacity)
for metricName, aggMap := range values {
agg := recordMetricAggregation(metricName)
points, ok := aggMap[agg]
if !ok {
continue
}
for _, point := range points {
entityID := point.EntityID
@@ -85,22 +125,22 @@ func getRecordsByClientAndTimeFromSeries(ctx context.Context, s *metric.Store, c
entityID = clientUUID
}
key := recordSeriesKey{client: entityID, ts: point.Bucket.Unix()}
if recordMap[key] == nil {
recordMap[key] = &models.Record{
index, exists := recordIndex[key]
if !exists {
index = len(records)
recordIndex[key] = index
records = append(records, models.Record{
Client: entityID,
Time: point.Bucket.UTC(),
}
})
}
applyRecordMetricValue(recordMap[key], metricName, point.Value)
// Resolve the address after append: the backing array may have moved.
applyRecordMetricValue(&records[index], metricName, point.Value)
}
}
records := make([]models.Record, 0, len(recordMap))
for _, rec := range recordMap {
records = append(records, *rec)
}
sortRecords(records)
return records, nil
return records
}
func getRecordMetricMaxByClientAndTimeFromSeries(ctx context.Context, s *metric.Store, clientUUID, recordMetric string, start, end time.Time) ([]models.Record, error) {
@@ -109,22 +149,19 @@ func getRecordMetricMaxByClientAndTimeFromSeries(ctx context.Context, s *metric.
return nil, fmt.Errorf("unsupported record metric %q", recordMetric)
}
now := time.Now().UTC()
points, err := s.Series(ctx, metric.AggregateQuery{
Query: metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
},
Aggregation: metric.AggMax,
Interval: recordSeriesInterval(s, start, end, now),
}, now)
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QuerySeries, Source: SourceRollup,
OmitTags: true, OmitLabels: true, CompactDefinitions: true,
Entities: EntityScope{IDs: []string{clientUUID}},
Specs: []MetricSpec{{Name: metricName, Aggregations: []metric.Aggregation{metric.AggMax}}},
Start: start, End: end, Order: metric.OrderAsc, Grouping: GroupMerged,
Budget: PointBudget{PerSeries: 500}, Rounding: RoundFloor, AllowMissing: true,
})
if err != nil {
return nil, fmt.Errorf("failed to query metric %s: %w", metricName, err)
}
points := loaded.Values[metricName][metric.AggMax]
records := make([]models.Record, 0, len(points))
for _, point := range points {
entityID := point.EntityID
@@ -149,49 +186,6 @@ func recordMetricAggregation(metricName string) metric.Aggregation {
}
}
func recordSeriesInterval(s *metric.Store, start, end, now time.Time) time.Duration {
interval := recordDownsampleInterval(end.Sub(start), 500)
return s.CompatibleSeriesInterval(start, now, interval)
}
func recordDownsampleInterval(rangeDuration time.Duration, maxPoints int) time.Duration {
if maxPoints <= 0 {
maxPoints = 500
}
nanos := rangeDuration.Nanoseconds()
if nanos <= 0 {
return time.Second
}
interval := time.Duration((nanos + int64(maxPoints) - 1) / int64(maxPoints))
if interval < time.Second {
return time.Second
}
return metric.FloorStandardInterval(interval)
}
func listRecordEntityIDs(ctx context.Context, s *metric.Store, start, end time.Time, interval time.Duration) ([]string, error) {
seen := make(map[string]struct{})
for _, metricName := range loadRecordMetricNames {
ids, err := s.EntityIDs(ctx, metric.Query{
MetricName: metricName,
Start: start.Add(-interval),
End: end,
})
if err != nil {
return nil, err
}
for _, id := range ids {
seen[id] = struct{}{}
}
}
out := make([]string, 0, len(seen))
for id := range seen {
out = append(out, id)
}
sort.Strings(out)
return out, nil
}
func applyRecordMetricValue(rec *models.Record, metricName string, value float64) {
switch metricName {
case MetricCPU:
@@ -253,17 +247,21 @@ func GetGPURecordsByClientAndTime(ctx context.Context, clientUUID string, start,
}
recordMap := make(map[gpuKey]*models.GPURecord)
specs := make([]MetricSpec, 0, len(gpuMetrics))
for _, name := range gpuMetrics {
specs = append(specs, MetricSpec{Name: name})
}
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QueryPoints, Source: SourceRaw,
OmitLabels: true, CompactDefinitions: true,
Entities: EntityScope{IDs: []string{clientUUID}}, Specs: specs,
Start: start, End: end, Order: metric.OrderAsc, AllowMissing: true,
})
if err != nil {
return nil, fmt.Errorf("failed to query GPU metrics: %w", err)
}
for _, metricName := range gpuMetrics {
points, err := s.Query(ctx, metric.Query{
MetricName: metricName,
EntityID: clientUUID,
Start: start,
End: end,
Order: metric.OrderAsc,
})
if err != nil {
continue // GPU 数据可能不存在
}
points := loaded.Raw[metricName]
for _, p := range points {
deviceIndex := 0
@@ -307,7 +305,3 @@ func GetGPURecordsByClientAndTime(ctx context.Context, clientUUID string, start,
return records, nil
}
// GetPingRecords 从 metric store 查询兼容旧接口的 ping 记录。
//
// 旧接口过去直接读取 ping_records。这里使用与 queryMetrics 相同的 Series
+11 -31
View File
@@ -62,39 +62,33 @@ func writePingRecords(ctx context.Context, records []models.PingRecord) error {
return s.WriteBatch(ctx, points)
}
// GetPingRecords returns legacy-shaped ping records, newest first.
func GetPingRecords(ctx context.Context, clientUUID string, taskID int, start, end time.Time) ([]models.PingRecord, error) {
s := GetStore()
if s == nil {
return nil, fmt.Errorf("metric store not enabled")
}
query := metric.Query{
MetricName: MetricPingLatency,
Start: start,
End: end,
Order: metric.OrderAsc,
query := MetricsQuery{
Store: s, Mode: QuerySeries, Source: SourceRollup,
CompactDefinitions: true,
Entities: EntityScope{All: clientUUID == ""},
Specs: []MetricSpec{{Name: MetricPingLatency, Aggregations: []metric.Aggregation{metric.AggLast}}},
Start: start, End: end, Order: metric.OrderAsc, Grouping: GroupSeries,
Budget: PointBudget{PerSeries: 4000}, Rounding: RoundFloor, AllowMissing: true,
}
if clientUUID != "" {
query.EntityID = clientUUID
query.Entities.IDs = []string{clientUUID}
}
if taskID >= 0 {
query.Tags = map[string]string{"task_id": fmt.Sprintf("%d", taskID)}
}
interval := pingQueryInterval(end.Sub(start), 4000)
interval = s.CompatibleSeriesInterval(start, time.Now().UTC(), interval)
points, err := s.Series(ctx, metric.AggregateQuery{
Query: query,
Aggregation: metric.AggLast,
Interval: interval,
PreserveSeries: true,
}, time.Now().UTC())
loaded, err := QueryMetrics(ctx, query)
if err != nil {
return nil, err
}
points := loaded.Values[MetricPingLatency][metric.AggLast]
records := make([]models.PingRecord, 0, len(points))
for _, p := range points {
taskIDVal := uint(0)
@@ -117,17 +111,3 @@ func GetPingRecords(ctx context.Context, clientUUID string, taskID int, start, e
return records, nil
}
func pingQueryInterval(rangeDuration time.Duration, maxPoints int) time.Duration {
if maxPoints <= 0 {
maxPoints = 4000
}
if rangeDuration <= 0 {
return time.Second
}
interval := time.Duration((rangeDuration.Nanoseconds() + int64(maxPoints) - 1) / int64(maxPoints))
if interval < time.Second {
return time.Second
}
return metric.FloorStandardInterval(interval)
}
+766
View File
@@ -0,0 +1,766 @@
package metricstore
import (
"context"
"fmt"
"sort"
"strings"
"time"
"github.com/komari-monitor/komari/pkg/metric"
)
// QueryMode selects exact samples, adaptive series, or exclusive-boundary lookups.
type QueryMode string
const (
QueryPoints QueryMode = "points"
QuerySeries QueryMode = "series"
QueryLatestBefore QueryMode = "latest_before"
)
type QuerySource string
const (
SourceAuto QuerySource = "auto"
SourceRaw QuerySource = "raw"
SourceRollup QuerySource = "rollup"
)
type QueryGrouping string
const (
GroupMerged QueryGrouping = "merged"
GroupEntity QueryGrouping = "entity"
GroupSeries QueryGrouping = "series"
)
type IntervalRounding string
const (
RoundFloor IntervalRounding = "floor"
RoundCeil IntervalRounding = "ceil"
)
// EntityScope deliberately distinguishes an empty authorized set from all entities.
// All and IDs are mutually exclusive. IDs are deduplicated without modifying callers.
type EntityScope struct {
All bool
IDs []string
}
type MetricSpec struct {
Name string
Aggregations []metric.Aggregation
// Positive intervals are explicit and are preserved, even for historical data.
Interval time.Duration
// Zero inherits Budget.PerSeries, then 500. -1 disables adaptive downsampling:
// a zero Interval then selects the finest compatible tier, not a whole-window bucket.
MaxPoints int
}
// PointBudget controls adaptive interval planning, not a raw-row SQL LIMIT.
// Total divides the point target across entities (metrics may be columns of one
// legacy record); it is not a hard cap across metrics, aggregations or tag variants.
// Raw queries and explicit intervals retain their exact historical semantics.
// Response adapters can impose their existing final sampling policy separately.
type PointBudget struct {
Total int
PerSeries int
}
type FillPolicy struct {
Gaps bool
PingNegativeAsNull bool
}
// QueryProjection controls retained result representations. The default keeps all
// source data for compatibility; chart requires Fill and retains only Filled.
type QueryProjection string
const (
ProjectionFull QueryProjection = "full"
ProjectionChart QueryProjection = "chart"
)
type MetricsQuery struct {
// Output omissions are opt-in; filtering and aggregation identity are unchanged.
OmitTags bool
OmitLabels bool
CompactDefinitions bool
Projection QueryProjection
Store *metric.Store
Mode QueryMode
Source QuerySource
Entities EntityScope
Specs []MetricSpec
Start, End time.Time
Tags map[string]string
Order metric.Order
Grouping QueryGrouping
Budget PointBudget
Rounding IntervalRounding
Fill FillPolicy
AllowMissing bool
Now time.Time
}
type MetricsResult struct {
Definitions map[string]metric.Definition
Raw map[string][]metric.Point
Values map[string]map[metric.Aggregation][]metric.AggregatePoint
Intervals map[string]time.Duration
IsRaw bool
Filled []MetricSeries
// Latest is keyed by metric name, then entity; missing samples are absent.
Latest map[string]map[string]metric.Point
}
type MetricSeries struct {
MetricName, EntityID string
Aggregation metric.Aggregation
Tags map[string]string
Interval time.Duration
Points []MetricValue
}
type MetricValue struct {
Time time.Time
Value *float64
Count int
Tags, Labels map[string]string
}
// DownsampleInterval returns a standard interval that keeps rangeDuration
// within about maxPoints buckets, never finer than one second. roundUp is used
// by point budgets: flooring can exceed the budget, while compatibility queries
// preserve the historical floor behavior.
func DownsampleInterval(rangeDuration time.Duration, maxPoints int, roundUp bool) time.Duration {
if maxPoints < 1 {
maxPoints = 1
}
points := time.Duration(maxPoints)
interval := (rangeDuration + points - 1) / points
if interval < time.Second {
return time.Second
}
if roundUp {
return metric.CeilStandardInterval(interval)
}
return metric.FloorStandardInterval(interval)
}
// QueryMetrics is the shared query policy boundary. Raw and aggregate slices are
// transferred directly from batch primitives; nullable chart data is allocated only
// when explicitly requested by Fill. All work uses ctx and one captured clock.
func QueryMetrics(ctx context.Context, req MetricsQuery) (MetricsResult, error) {
if ctx == nil {
return MetricsResult{}, fmt.Errorf("%w: context is required", metric.ErrInvalidArgument)
}
if err := ctx.Err(); err != nil {
return MetricsResult{}, err
}
result := MetricsResult{Definitions: map[string]metric.Definition{}, Raw: map[string][]metric.Point{}, Values: map[string]map[metric.Aggregation][]metric.AggregatePoint{}, Intervals: map[string]time.Duration{}, Latest: map[string]map[string]metric.Point{}}
if req.Mode == "" {
req.Mode = QuerySeries
}
if req.Source == "" {
req.Source = SourceAuto
}
if req.Grouping == "" {
req.Grouping = GroupMerged
}
if req.Order == "" {
req.Order = metric.OrderAsc
}
if req.Rounding == "" {
req.Rounding = RoundFloor
}
if err := validateMetricsQuery(req); err != nil {
return result, err
}
now := req.Now
if now.IsZero() {
now = time.Now()
}
now = now.UTC()
req.Start, req.End = req.Start.UTC(), req.End.UTC()
req.Entities.IDs = uniqueQueryIDs(req.Entities.IDs)
// Never let the primitive's empty entity filter broaden an empty authorized set.
s := req.Store
if s == nil {
s = GetStore()
}
if s == nil {
return result, fmt.Errorf("metric store not enabled")
}
names := make([]string, 0, len(req.Specs))
for _, spec := range req.Specs {
names = append(names, spec.Name)
}
loadDefinitions := s.GetMetrics
if req.CompactDefinitions {
loadDefinitions = s.GetMetricsCompact
}
if !req.Entities.All && len(req.Entities.IDs) == 0 {
definitions, err := loadDefinitions(ctx, names)
if err != nil {
return result, err
}
result.Definitions = definitions
result.IsRaw = req.Source == SourceRaw || (req.Source == SourceAuto && (req.Mode == QueryPoints || UsesRawWindow(req.Start, req.End, now)))
if err := checkQueryDefinitions(names, definitions, req.AllowMissing); err != nil {
return result, err
}
return result, ctx.Err()
}
if req.Mode == QueryLatestBefore {
definitions, err := loadDefinitions(ctx, names)
if err != nil {
return result, err
}
result.Definitions = definitions
if err := checkQueryDefinitions(names, definitions, req.AllowMissing); err != nil {
return result, err
}
for _, name := range names {
if _, ok := definitions[name]; !ok {
continue
}
ids := req.Entities.IDs
for _, id := range ids {
if err := ctx.Err(); err != nil {
return result, err
}
point, found, err := s.LatestBefore(ctx, name, id, req.End)
if err != nil {
return result, err
}
if found {
if result.Latest[name] == nil {
result.Latest[name] = map[string]metric.Point{}
}
result.Latest[name][id] = point
}
}
}
return result, ctx.Err()
}
result.IsRaw = req.Source == SourceRaw || (req.Source == SourceAuto && (req.Mode == QueryPoints || UsesRawWindow(req.Start, req.End, now)))
if result.IsRaw {
definitions, err := loadDefinitions(ctx, names)
if err != nil {
return result, err
}
result.Definitions = definitions
if err := checkQueryDefinitions(names, definitions, req.AllowMissing); err != nil {
return result, err
}
known := make([]string, 0, len(names))
for _, name := range names {
if _, ok := definitions[name]; ok {
known = append(known, name)
result.Intervals[name] = 0
}
}
if len(known) > 0 {
result.Raw, err = s.QueryBatch(ctx, metric.BatchQuery{MetricNames: known, EntityIDs: req.Entities.IDs, Start: req.Start, End: req.End, Tags: req.Tags, Order: req.Order, OmitTags: req.OmitTags, OmitLabels: req.OmitLabels, IncludeSeriesIdentity: req.Fill.Gaps || req.Fill.PingNegativeAsNull})
if err != nil {
return result, err
}
}
} else {
entityCount := len(req.Entities.IDs)
if needsTotalEntityCount(req) {
if req.Entities.All {
count, err := s.CountEntities(ctx, names)
if err != nil {
return result, err
}
entityCount = count
}
// Explicit IDs are a conservative dictionary-count upper bound; never
// scan every metric/tier simply to plan a response budget.
}
if entityCount < 1 {
entityCount = 1
}
specs := make([]metric.BatchSeriesSpec, 0, len(req.Specs))
for _, spec := range req.Specs {
interval := spec.Interval
if interval == 0 {
points := spec.MaxPoints
if points == 0 {
points = req.Budget.PerSeries
}
if points == 0 {
points = 500
}
if points == 500 && spec.MaxPoints == 0 && req.Budget.PerSeries == 0 && req.Budget.Total == -1 {
points = -1
}
interval = time.Second
if points != -1 {
if req.Budget.Total > 0 {
budget := req.Budget.Total / entityCount
if budget < 1 {
budget = 1
}
if budget < points {
points = budget
}
}
interval = DownsampleInterval(req.End.Sub(req.Start), points, req.Rounding == RoundCeil)
}
interval = s.CompatibleSeriesInterval(req.Start, now, interval)
if interval <= 0 {
interval = time.Second
}
}
aggregations := spec.Aggregations
if len(aggregations) == 0 {
aggregations = []metric.Aggregation{metric.AggAvg}
}
specs = append(specs, metric.BatchSeriesSpec{MetricName: spec.Name, Aggregations: aggregations, Interval: interval, PreserveSeries: req.Grouping == GroupSeries, GroupByEntity: req.Grouping == GroupEntity})
result.Intervals[spec.Name] = interval
}
batch, err := s.SeriesBatch(ctx, metric.BatchSeriesQuery{Specs: specs, EntityIDs: req.Entities.IDs, Start: req.Start, End: req.End, Tags: req.Tags, Order: req.Order, OmitTags: req.OmitTags, CompactDefinitions: req.CompactDefinitions, IncludeSeriesIdentity: req.Fill.Gaps || req.Fill.PingNegativeAsNull}, now)
if err != nil {
return result, err
}
result.Definitions, result.Values = batch.Definitions, batch.Values
// SeriesBatch already loads metadata. Validate its result rather than issue a
// second metadata query; unknown metrics otherwise look like empty series.
if err := checkQueryDefinitions(names, batch.Definitions, req.AllowMissing); err != nil {
return result, err
}
for _, name := range names {
if _, ok := batch.Definitions[name]; !ok {
delete(result.Intervals, name)
delete(result.Values, name)
}
}
}
if req.Fill.Gaps || req.Fill.PingNegativeAsNull {
filled, err := buildQuerySeriesOwned(ctx, req, result, req.Projection == ProjectionChart)
if err != nil {
return result, err
}
result.Filled = filled
if req.Projection == ProjectionChart {
result.Raw, result.Values = nil, nil
}
}
return result, ctx.Err()
}
// needsTotalEntityCount excludes explicit intervals and unlimited specs: neither
// consumes the total adaptive budget. Keep its precedence aligned with planning.
func needsTotalEntityCount(req MetricsQuery) bool {
if req.Budget.Total <= 0 {
return false
}
for _, spec := range req.Specs {
if spec.Interval != 0 {
continue
}
points := spec.MaxPoints
if points == 0 {
points = req.Budget.PerSeries
}
if points != -1 {
return true
}
}
return false
}
func validateMetricsQuery(req MetricsQuery) error {
invalid := func(message string) error { return fmt.Errorf("%w: %s", metric.ErrInvalidArgument, message) }
if req.OmitTags && (req.Fill.Gaps || req.Fill.PingNegativeAsNull) {
return invalid("omitting tags with filled output is unsupported: series identity must be preserved")
}
if req.Mode == QueryLatestBefore && (req.OmitTags || req.OmitLabels) {
return invalid("latest-before output omissions are unsupported")
}
switch req.Mode {
case QueryPoints, QuerySeries, QueryLatestBefore:
default:
return invalid("unsupported query mode")
}
switch req.Source {
case SourceAuto, SourceRaw, SourceRollup:
default:
return invalid("unsupported query source")
}
switch req.Grouping {
case GroupMerged, GroupEntity, GroupSeries:
default:
return invalid("unsupported query grouping")
}
switch req.Projection {
case "", ProjectionFull:
case ProjectionChart:
if req.Mode == QueryLatestBefore || (!req.Fill.Gaps && !req.Fill.PingNegativeAsNull) {
return invalid("chart projection requires fill and does not support latest-before")
}
default:
return invalid("unsupported query projection")
}
switch req.Rounding {
case RoundFloor, RoundCeil:
default:
return invalid("unsupported interval rounding")
}
if req.Order != metric.OrderAsc && req.Order != metric.OrderDesc {
return invalid("unsupported query order")
}
if req.Entities.All && len(req.Entities.IDs) > 0 {
return invalid("all entities and explicit IDs are mutually exclusive")
}
for _, id := range req.Entities.IDs {
if strings.TrimSpace(id) == "" {
return invalid("entity ID cannot be empty")
}
}
if req.End.IsZero() {
return invalid("end time is required")
}
if req.Mode != QueryLatestBefore && (req.Start.IsZero() || req.End.Before(req.Start)) {
return invalid("valid start and end times are required")
}
if req.Mode == QueryLatestBefore && (len(req.Tags) > 0 || req.Source != SourceAuto) {
return invalid("latest-before requires auto source and does not support tag filters")
}
if req.Mode == QueryPoints && req.Source == SourceRollup {
return invalid("points mode requires raw or auto source")
}
if req.Mode == QueryLatestBefore && req.Entities.All {
return invalid("latest-before requires explicit entity IDs")
}
if req.Budget.Total < -1 || req.Budget.PerSeries < -1 {
return invalid("invalid point budget")
}
if len(req.Specs) == 0 {
return invalid("at least one metric specification is required")
}
seen := map[string]bool{}
for _, spec := range req.Specs {
if strings.TrimSpace(spec.Name) == "" || seen[spec.Name] {
return invalid("metric names must be nonempty and unique")
}
seen[spec.Name] = true
if spec.Interval < 0 || spec.MaxPoints < -1 {
return invalid("invalid metric interval or point budget")
}
for _, agg := range spec.Aggregations {
// Reuse primitive aggregation validation even when the selected source is raw.
if err := (metric.AggregateQuery{Query: metric.Query{MetricName: spec.Name, Start: req.End, End: req.End}, Aggregation: agg, Interval: time.Second}).Validate(); err != nil {
return err
}
}
}
return nil
}
// UsesRawWindow selects the public adaptive raw window; its end boundary is strict.
func UsesRawWindow(start, end, now time.Time) bool {
return end.Sub(start) <= 10*time.Minute && end.After(now.Add(-10*time.Minute))
}
func uniqueQueryIDs(ids []string) []string {
out := make([]string, 0, len(ids))
seen := map[string]bool{}
for _, id := range ids {
if !seen[id] {
out = append(out, id)
seen[id] = true
}
}
return out
}
func checkQueryDefinitions(names []string, definitions map[string]metric.Definition, allowMissing bool) error {
if !allowMissing {
for _, name := range names {
if _, ok := definitions[name]; !ok {
return fmt.Errorf("%w: metric %q", metric.ErrNotFound, name)
}
}
}
return nil
}
// querySeriesKey consumes storage's immutable, collision-free output identity.
// Manually constructed points (and low-level queries without the opt-in) retain
// the old canonical fallback. Neither grouping nor ordering uses a hash.
func querySeriesKey(entity string, tags map[string]string, identity *metric.SeriesIdentity) string {
if key := identity.Key(); key != "" {
return key
}
return metric.NewSeriesIdentity(entity, tags).Key()
}
func buildQuerySeries(ctx context.Context, req MetricsQuery, result MetricsResult) ([]MetricSeries, error) {
return buildQuerySeriesOwned(ctx, req, result, false)
}
// detach is only valid for QueryMetrics-owned source maps. Release each source
// array after conversion, before allocating gaps or processing the next metric.
func buildQuerySeriesOwned(ctx context.Context, req MetricsQuery, result MetricsResult, detach bool) ([]MetricSeries, error) {
out := []MetricSeries{}
for _, spec := range req.Specs {
if err := ctx.Err(); err != nil {
return nil, err
}
if _, ok := result.Definitions[spec.Name]; !ok {
continue
}
aggregations := spec.Aggregations
if result.IsRaw {
aggregations = []metric.Aggregation{""}
} else if len(aggregations) == 0 {
aggregations = []metric.Aggregation{metric.AggAvg}
}
for aggIndex, agg := range aggregations {
if err := ctx.Err(); err != nil {
return nil, err
}
var seenEntities map[string]bool
if len(req.Entities.IDs) > 0 && (result.IsRaw || req.Grouping != GroupMerged) {
seenEntities = make(map[string]bool, len(req.Entities.IDs))
}
groups := map[string]*MetricSeries{}
add := func(entity string, tags map[string]string, identity *metric.SeriesIdentity, value MetricValue) {
// Exact samples always preserve identity; aggregate identity is determined by
// the requested primitive grouping and must not be collapsed again here.
key := querySeriesKey(entity, tags, identity)
series := groups[key]
if series == nil {
series = &MetricSeries{MetricName: spec.Name, EntityID: entity, Aggregation: agg, Tags: tags, Interval: result.Intervals[spec.Name]}
groups[key] = series
if seenEntities != nil {
seenEntities[entity] = true
}
}
if req.Fill.PingNegativeAsNull && (spec.Name == MetricPingLatency || spec.Name == MetricPingLoss) && value.Value != nil && *value.Value == -1 {
value.Value = nil
}
series.Points = append(series.Points, value)
}
if result.IsRaw {
for i, p := range result.Raw[spec.Name] {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
v := p.Value
add(p.EntityID, p.Tags, p.SeriesIdentity, MetricValue{Time: p.Timestamp, Value: &v, Count: 1, Tags: p.Tags, Labels: p.Labels})
}
} else {
for i, p := range result.Values[spec.Name][agg] {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
v := p.Value
add(p.EntityID, p.Tags, p.SeriesIdentity, MetricValue{Time: p.Bucket, Value: &v, Count: p.Count, Tags: p.Tags})
}
}
if detach {
if result.IsRaw {
delete(result.Raw, spec.Name)
} else {
// Duplicate aggregation specs historically produce duplicate series.
needed := false
for _, next := range aggregations[aggIndex+1:] {
needed = needed || next == agg
}
if !needed {
delete(result.Values[spec.Name], agg)
}
}
}
// Seed missing entities directly; do not scan all groups for each ID or
// allocate a placeholder point only to discard it.
seed := func(id string) {
groups[querySeriesKey(id, req.Tags, nil)] = &MetricSeries{MetricName: spec.Name, EntityID: id, Aggregation: agg, Tags: req.Tags, Interval: result.Intervals[spec.Name]}
if seenEntities != nil {
seenEntities[id] = true
}
}
if result.IsRaw || req.Grouping != GroupMerged {
for i, id := range req.Entities.IDs {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
if !seenEntities[id] {
seed(id)
}
}
}
if len(groups) == 0 {
ids := req.Entities.IDs
if len(ids) == 0 || (!result.IsRaw && req.Grouping == GroupMerged) {
ids = []string{""}
}
for _, id := range ids {
seed(id)
}
}
keys := make([]string, 0, len(groups))
for key := range groups {
if len(keys)%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
keys = append(keys, key)
}
if err := ctx.Err(); err != nil {
return nil, err
}
sort.Strings(keys)
for _, key := range keys {
if err := ctx.Err(); err != nil {
return nil, err
}
series := *groups[key]
sort.SliceStable(series.Points, func(i, j int) bool { return series.Points[i].Time.Before(series.Points[j].Time) })
if err := ctx.Err(); err != nil {
return nil, err
}
if req.Fill.Gaps {
var err error
series, err = fillMetricSeriesOwned(ctx, series, req.Start, req.End)
if err != nil {
return nil, err
}
}
if req.Order == metric.OrderDesc {
for i, j := 0, len(series.Points)-1; i < j; i, j = i+1, j-1 {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return nil, err
}
}
series.Points[i], series.Points[j] = series.Points[j], series.Points[i]
}
}
out = append(out, series)
}
}
}
return out, ctx.Err()
}
// FillMetricSeries matches the public chart's adaptive lower-quartile gap policy.
func FillMetricSeries(series MetricSeries, start, end time.Time) MetricSeries {
filled, _ := fillMetricSeries(context.Background(), series, start, end)
return filled
}
func fillMetricSeries(ctx context.Context, series MetricSeries, start, end time.Time) (MetricSeries, error) {
if err := ctx.Err(); err != nil {
return series, err
}
// Public callers retain ownership, including spare capacity. Never write there.
series.Points = append([]MetricValue(nil), series.Points...)
return fillMetricSeriesOwned(ctx, series, start, end)
}
// fillMetricSeriesOwned inserts gaps backwards into exclusively owned storage.
// Dense series reuse their point array; sparse series grow only when capacity
// cannot accommodate the exact number of gaps, rather than always copying into
// a second full point array. The public wrapper above remains non-mutating.
func fillMetricSeriesOwned(ctx context.Context, series MetricSeries, start, end time.Time) (MetricSeries, error) {
if err := ctx.Err(); err != nil {
return series, err
}
deltas := make([]time.Duration, 0, len(series.Points))
for i := 1; i < len(series.Points); i++ {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return series, err
}
}
if d := series.Points[i].Time.Sub(series.Points[i-1].Time); d > 0 {
deltas = append(deltas, d)
}
}
if err := ctx.Err(); err != nil {
return series, err
}
expected := series.Interval
if len(deltas) >= 2 {
sort.Slice(deltas, func(i, j int) bool { return deltas[i] < deltas[j] })
if observed := deltas[(len(deltas)-1)/4]; observed > expected {
expected = observed
}
}
if err := ctx.Err(); err != nil {
return series, err
}
if expected > 0 {
series.Interval = expected
}
null := func(at time.Time) MetricValue { return MetricValue{Time: at.UTC(), Tags: series.Tags} }
n := len(series.Points)
if n == 0 {
series.Points = append(series.Points, null(start), null(end))
return series, ctx.Err()
}
leading := start.Before(series.Points[0].Time)
gap := func(i int) bool {
return i > 0 && expected > 0 && series.Points[i-1].Value != nil && series.Points[i].Value != nil && series.Points[i].Time.Sub(series.Points[i-1].Time) > expected+expected/2
}
extra := 0
if leading {
extra++
}
for i := 1; i < n; i++ {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return series, err
}
}
if gap(i) {
extra++
}
}
if extra == 0 {
return series, ctx.Err()
}
if cap(series.Points) < n+extra {
points := make([]MetricValue, n+extra)
copy(points, series.Points)
series.Points = points
} else {
series.Points = series.Points[:n+extra]
}
write := n + extra - 1
for i := n - 1; i >= 0; i-- {
if i%256 == 0 {
if err := ctx.Err(); err != nil {
return series, err
}
}
// Decide before moving: write may equal i, but never precedes it.
insert := gap(i)
var at time.Time
if insert {
at = series.Points[i-1].Time.Add(expected)
}
series.Points[write] = series.Points[i]
write--
if insert {
series.Points[write] = null(at)
write--
}
}
if leading {
series.Points[0] = null(start)
}
return series, ctx.Err()
}
+25 -57
View File
@@ -39,6 +39,7 @@ var (
const (
reportBatchInterval = 3 * time.Second
reportBatchQueueSize = 512
pingBatchQueueSize = 4096
pingBatchMaxRecords = 512
reportBatchWriteTimeout = 10 * time.Second
// Longer gaps have no reliable per-bucket traffic attribution.
@@ -81,7 +82,7 @@ func StartReportBatcher() {
}
worker := &reportBatchWorker{
queue: make(chan v2.Report, reportBatchQueueSize),
pingQueue: make(chan models.PingRecord, reportBatchQueueSize),
pingQueue: make(chan models.PingRecord, pingBatchQueueSize),
requests: make(chan reportBatchRequest, 1),
done: make(chan struct{}),
}
@@ -212,8 +213,8 @@ func (w *reportBatchWorker) run() {
for {
select {
case request := <-w.requests:
pending = append(pending, drainReportQueue(w.queue, reportBatchQueueSize)...)
pendingPings = append(pendingPings, drainPingQueue(w.pingQueue, reportBatchQueueSize)...)
pending = append(pending, drainQueue(w.queue, reportBatchQueueSize)...)
pendingPings = append(pendingPings, drainQueue(w.pingQueue, pingBatchQueueSize)...)
err := errors.Join(
writePendingReports(request.ctx, &pending),
writePendingPingRecords(request.ctx, &pendingPings),
@@ -228,11 +229,11 @@ func (w *reportBatchWorker) run() {
}
request.done <- err
case <-ticker.C:
pendingPings = append(pendingPings, drainPingQueue(w.pingQueue, reportBatchQueueSize)...)
pendingPings = append(pendingPings, drainQueue(w.pingQueue, pingBatchQueueSize)...)
if err := writePendingPingRecords(context.Background(), &pendingPings); err != nil {
logger.Errorf("metricstore", "failed to flush ping batch: %v", err)
}
pending = append(pending, drainReportQueue(w.queue, reportBatchQueueSize)...)
pending = append(pending, drainQueue(w.queue, reportBatchQueueSize)...)
if err := writePendingReports(context.Background(), &pending); err != nil {
logger.Errorf("metricstore", "failed to flush metric report batch: %v", err)
}
@@ -257,69 +258,36 @@ func (w *reportBatchWorker) enqueuePing(ctx context.Context, record models.PingR
}
}
func drainReportQueue(queue <-chan v2.Report, limit int) []v2.Report {
if limit <= 0 {
// drainQueue takes up to limit queued items without blocking.
func drainQueue[T any](queue <-chan T, limit int) []T {
n := min(len(queue), limit)
if n <= 0 {
return nil
}
capacity := len(queue)
if capacity > limit {
capacity = limit
}
if capacity == 0 {
return nil
}
reports := make([]v2.Report, 0, capacity)
for len(reports) < limit {
items := make([]T, 0, n)
for len(items) < limit {
select {
case report := <-queue:
reports = append(reports, report)
case item := <-queue:
items = append(items, item)
default:
return reports
return items
}
}
return reports
}
func drainPingQueue(queue <-chan models.PingRecord, limit int) []models.PingRecord {
if limit <= 0 {
return nil
}
capacity := len(queue)
if capacity > limit {
capacity = limit
}
if capacity == 0 {
return nil
}
records := make([]models.PingRecord, 0, capacity)
for len(records) < limit {
select {
case record := <-queue:
records = append(records, record)
default:
return records
}
}
return records
return items
}
// writePendingReports commits all pending reports as one batch and keeps them
// for the next attempt when the write fails.
func writePendingReports(ctx context.Context, pending *[]v2.Report) error {
if len(*pending) == 0 {
return nil
}
batchSize := len(*pending)
for len(*pending) > 0 {
if batchSize > len(*pending) {
batchSize = len(*pending)
}
writeCtx, cancel := context.WithTimeout(ctx, reportBatchWriteTimeout)
_, err := writeReportBatch(writeCtx, (*pending)[:batchSize])
cancel()
if err != nil {
return err
}
*pending = (*pending)[batchSize:]
writeCtx, cancel := context.WithTimeout(ctx, reportBatchWriteTimeout)
defer cancel()
if _, err := writeReportBatch(writeCtx, *pending); err != nil {
return err
}
*pending = (*pending)[:0]
return nil
}
@@ -373,7 +341,7 @@ func writeReportBatch(ctx context.Context, reports []v2.Report) ([]v2.Report, er
state.mu.Unlock()
}
if !values.initialized {
totalUp, hasUp, err := s.LatestBefore(ctx, MetricNetTotalUp, report.UUID, report.UpdatedAt)
totalUp, hasUp, err := latestReportPoint(ctx, s, MetricNetTotalUp, report.UUID, report.UpdatedAt)
if err != nil {
logger.Errorf("metricstore", "failed to restore previous upload counter for %s: %v", report.UUID, err)
} else {
@@ -381,7 +349,7 @@ func writeReportBatch(ctx context.Context, reports []v2.Report) ([]v2.Report, er
values.totalUpAt = totalUp.Timestamp
values.hasUp = hasUp
}
totalDown, hasDown, err := s.LatestBefore(ctx, MetricNetTotalDown, report.UUID, report.UpdatedAt)
totalDown, hasDown, err := latestReportPoint(ctx, s, MetricNetTotalDown, report.UUID, report.UpdatedAt)
if err != nil {
logger.Errorf("metricstore", "failed to restore previous download counter for %s: %v", report.UUID, err)
} else {
+16 -1
View File
@@ -49,8 +49,23 @@ func reportMetricPoints(report v2.Report, trafficUp, trafficDown int64) []metric
return points
}
// latestReportPoint preserves independent counter restoration errors: one failed
// direction must not prevent the other from restoring its baseline.
func latestReportPoint(ctx context.Context, s *metric.Store, metricName, entityID string, before time.Time) (metric.Point, bool, error) {
loaded, err := QueryMetrics(ctx, MetricsQuery{
Store: s, Mode: QueryLatestBefore, Source: SourceAuto,
Entities: EntityScope{IDs: []string{entityID}},
Specs: []MetricSpec{{Name: metricName}}, End: before, AllowMissing: true,
})
if err != nil {
return metric.Point{}, false, err
}
point, ok := loaded.Latest[metricName][entityID]
return point, ok, nil
}
func latestReportCounter(ctx context.Context, s *metric.Store, metricName, entityID string, before time.Time) (int64, bool, error) {
point, ok, err := s.LatestBefore(ctx, metricName, entityID, before)
point, ok, err := latestReportPoint(ctx, s, metricName, entityID, before)
if err != nil {
return 0, false, err
}
+29 -61
View File
@@ -37,99 +37,67 @@ func StructureUpgradeRequired(ctx context.Context) (bool, error) {
}
func structureUpgradeRequiredForConfig(ctx context.Context, cfg *MetricStoreConfig) (bool, error) {
s, err := openWithoutMigration(ctx, cfg)
if err != nil {
return false, err
}
defer s.Close()
return s.NeedsRestructure(ctx)
}
// openWithoutMigration opens cfg without creating or altering any table.
func openWithoutMigration(ctx context.Context, cfg *MetricStoreConfig) (*metric.Store, error) {
metricCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return false, err
return nil, err
}
store, err := metric.Open(ctx, metricCfg)
if err != nil {
return false, err
}
defer store.Close()
return store.NeedsRestructure(ctx)
return metric.Open(ctx, metricCfg)
}
// RestructureConfiguredStore performs the explicit data copy, table swap and
// one physical reclaim pass used by the upgrade guide.
func RestructureConfiguredStore(ctx context.Context, report func(RestructureProgress)) (RestructureResult, error) {
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
return RestructureResult{}, err
}
metricCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return RestructureResult{}, err
}
store, err := metric.Open(ctx, metricCfg)
if err != nil {
return RestructureResult{}, err
}
defer store.Close()
before, err := store.LegacyStorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure legacy metric storage: %w", err)
}
result, err := store.Restructure(ctx, func(progress metric.RestructureProgress) {
if report == nil {
return
}
report(RestructureProgress{Phase: progress.Phase, CurrentMetric: progress.Current, RowsDone: progress.RowsDone, RowsTotal: progress.RowsTotal, MetricsDone: progress.MetricsDone, MetricsTotal: progress.MetricsTotal})
})
if err != nil {
return RestructureResult{}, err
}
if report != nil {
report(RestructureProgress{Phase: "reclaiming", RowsDone: result.RowsCopied, RowsTotal: result.RowsCopied, MetricsDone: result.Metrics, MetricsTotal: result.Metrics})
}
if err := store.ReclaimSpace(ctx); err != nil {
return RestructureResult{}, fmt.Errorf("reclaim metric storage: %w", err)
}
after, err := store.StorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure rebuilt metric storage: %w", err)
}
return RestructureResult{BeforeBytes: before, AfterBytes: after, RowsCopied: result.RowsCopied, Metrics: result.Metrics}, nil
return rebuildConfiguredStore(ctx, report, (*metric.Store).Restructure)
}
// DiscardConfiguredStoreHistory completes the same schema upgrade as
// RestructureConfiguredStore but drops every historical sample instead of
// copying it. Metric definitions remain available for new ingest.
func DiscardConfiguredStoreHistory(ctx context.Context, report func(RestructureProgress)) (RestructureResult, error) {
return rebuildConfiguredStore(ctx, report, (*metric.Store).DiscardHistory)
}
type storeRebuild func(*metric.Store, context.Context, func(metric.RestructureProgress)) (metric.RestructureResult, error)
func rebuildConfiguredStore(ctx context.Context, report func(RestructureProgress), rebuild storeRebuild) (RestructureResult, error) {
if report == nil {
report = func(RestructureProgress) {}
}
cfg, err := config.GetManyAs[MetricStoreConfig]()
if err != nil {
return RestructureResult{}, err
}
metricCfg, err := buildMetricConfig(cfg, false)
s, err := openWithoutMigration(ctx, cfg)
if err != nil {
return RestructureResult{}, err
}
store, err := metric.Open(ctx, metricCfg)
if err != nil {
return RestructureResult{}, err
}
defer store.Close()
defer s.Close()
before, err := store.LegacyStorageSize(ctx)
before, err := s.LegacyStorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure legacy metric storage: %w", err)
}
result, err := store.DiscardHistory(ctx, func(progress metric.RestructureProgress) {
if report == nil {
return
}
result, err := rebuild(s, ctx, func(progress metric.RestructureProgress) {
report(RestructureProgress{Phase: progress.Phase, CurrentMetric: progress.Current, RowsDone: progress.RowsDone, RowsTotal: progress.RowsTotal, MetricsDone: progress.MetricsDone, MetricsTotal: progress.MetricsTotal})
})
if err != nil {
return RestructureResult{}, err
}
if report != nil {
report(RestructureProgress{Phase: "reclaiming", RowsDone: result.RowsCopied, RowsTotal: result.RowsCopied, MetricsDone: result.Metrics, MetricsTotal: result.Metrics})
}
if err := store.ReclaimSpace(ctx); err != nil {
report(RestructureProgress{Phase: "reclaiming", RowsDone: result.RowsCopied, RowsTotal: result.RowsCopied, MetricsDone: result.Metrics, MetricsTotal: result.Metrics})
if err := s.ReclaimSpace(ctx); err != nil {
return RestructureResult{}, fmt.Errorf("reclaim metric storage: %w", err)
}
after, err := store.StorageSize(ctx)
after, err := s.StorageSize(ctx)
if err != nil {
return RestructureResult{}, fmt.Errorf("measure rebuilt metric storage: %w", err)
}
+36 -69
View File
@@ -53,13 +53,6 @@ func openStoreWithDefaultRetention(ctx context.Context, cfg *MetricStoreConfig,
return s, nil
}
// OpenStore opens an isolated metric store using the supplied configuration.
// It is used by the pre-start upgrade flow before the process-wide store is
// initialized. The caller owns the returned store and must close it.
// func OpenStore(ctx context.Context, cfg *MetricStoreConfig) (*metric.Store, error) {
// return openStore(ctx, cfg)
// }
// OpenStoreForMigration opens an isolated target and uses the legacy data span
// as the initial retention for definitions that do not exist yet. Existing
// definitions keep their configured retention, including an explicit zero.
@@ -73,17 +66,11 @@ func OpenStoreForMigration(ctx context.Context, cfg *MetricStoreConfig, legacyRe
// TestConnection 使用给定配置尝试连接 metrics 数据库(不影响当前运行的 store)。
// 仅打开连接并 Ping,不执行自动建表,连接成功后立即关闭。失败时返回可读错误。
func TestConnection(ctx context.Context, cfg *MetricStoreConfig) error {
metricCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return err
}
s, err := metric.Open(ctx, metricCfg)
s, err := openWithoutMigration(ctx, cfg)
if err != nil {
return err
}
defer s.Close()
return s.Ping(ctx)
}
@@ -116,10 +103,7 @@ func InitializeStore() error {
return err
}
storeMu.Lock()
store = s
storeFingerprint = targetFingerprint(cfg)
storeMu.Unlock()
activateStore(s, targetFingerprint(cfg), "initialization")
clearStoreClosing()
logger.Infof("metricstore", "Metric store initialized successfully (driver=%s)", ResolveDriverFromConfig(cfg.Driver, cfg.DSN))
@@ -130,10 +114,6 @@ func InitializeStore() error {
// from the restricted recovery page. It records that target as the current
// manual-migration source, but does not copy data from the unavailable store.
func RecoverStore(ctx context.Context, cfg *MetricStoreConfig) error {
if cfg == nil {
return fmt.Errorf("metric store recovery config is nil")
}
storeInitMu.Lock()
defer storeInitMu.Unlock()
if err := storeOperations.Acquire(ctx); err != nil {
@@ -147,18 +127,14 @@ func RecoverStore(ctx context.Context, cfg *MetricStoreConfig) error {
recovered := *cfg
recovered.DSN = strings.TrimSpace(recovered.DSN)
recovered.Driver = string(ResolveDriverFromConfig(recovered.Driver, recovered.DSN))
target := targetFingerprint(&recovered)
restructureRequired, err := structureUpgradeRequiredForConfig(ctx, &recovered)
if err != nil {
return err
}
if restructureRequired {
target := targetFingerprint(&recovered)
if err := config.SetMany(map[string]any{
MetricDBDriverKey: recovered.Driver,
MetricDBDSNKey: recovered.DSN,
MigrationTargetKey: target,
}); err != nil {
return fmt.Errorf("save recovered metric store config: %w", err)
if err := saveRecoveredTarget(&recovered, target); err != nil {
return err
}
logger.Infof("metricstore", "Metric store connection recovered; structure upgrade is required (driver=%s)", recovered.Driver)
return nil
@@ -167,31 +143,43 @@ func RecoverStore(ctx context.Context, cfg *MetricStoreConfig) error {
if err != nil {
return err
}
if err := saveRecoveredTarget(&recovered, target); err != nil {
_ = s.Close()
return err
}
activateStore(s, target, "recovery")
clearStoreClosing()
logger.Infof("metricstore", "Metric store recovered successfully (driver=%s)", recovered.Driver)
return nil
}
target := targetFingerprint(&recovered)
// saveRecoveredTarget persists the recovered connection and records it as the
// default source for the next manual store migration.
func saveRecoveredTarget(cfg *MetricStoreConfig, target string) error {
if err := config.SetMany(map[string]any{
MetricDBDriverKey: recovered.Driver,
MetricDBDSNKey: recovered.DSN,
MetricDBDriverKey: cfg.Driver,
MetricDBDSNKey: cfg.DSN,
MigrationTargetKey: target,
}); err != nil {
_ = s.Close()
return fmt.Errorf("save recovered metric store config: %w", err)
}
return nil
}
// activateStore publishes s as the active store, then closes the store it
// replaced. Callers must hold the exclusive operation gate or storeInitMu.
func activateStore(s *metric.Store, fingerprint, reason string) {
storeMu.Lock()
old := store
store = s
storeFingerprint = target
storeFingerprint = fingerprint
storeMu.Unlock()
clearStoreClosing()
if old != nil {
if closeErr := old.Close(); closeErr != nil {
logger.Errorf("metricstore", "Failed to close previous metric store during recovery: %v", closeErr)
}
if old == nil {
return
}
if err := old.Close(); err != nil {
logger.Errorf("metricstore", "Failed to close previous metric store during %s: %v", reason, err)
}
logger.Infof("metricstore", "Metric store recovered successfully (driver=%s)", recovered.Driver)
return nil
}
// Reload 根据最新配置热重载 metric store,无需重启进程。
@@ -218,16 +206,7 @@ func Reload(ctx context.Context) error {
// point-backed schema. That schema must go through the authenticated
// structure-upgrade guide first; otherwise index creation can reference
// columns such as resolution_id that do not exist yet.
checkCfg, err := buildMetricConfig(cfg, false)
if err != nil {
return err
}
checkStore, err := metric.Open(ctx, checkCfg)
if err != nil {
return err
}
needsRestructure, err := checkStore.NeedsRestructure(ctx)
_ = checkStore.Close()
needsRestructure, err := structureUpgradeRequiredForConfig(ctx, cfg)
if err != nil {
return fmt.Errorf("inspect metric store structure: %w", err)
}
@@ -241,17 +220,7 @@ func Reload(ctx context.Context) error {
return err
}
storeMu.Lock()
old := store
store = s
storeFingerprint = targetFingerprint(cfg)
storeMu.Unlock()
if old != nil {
if cerr := old.Close(); cerr != nil {
logger.Errorf("metricstore", "Failed to close previous metric store on reload: %v", cerr)
}
}
activateStore(s, targetFingerprint(cfg), "reload")
logger.Infof("metricstore", "Metric store reloaded successfully (driver=%s)", ResolveDriverFromConfig(cfg.Driver, cfg.DSN))
return nil
}
@@ -266,7 +235,7 @@ func GetStore() *metric.Store {
// CloseStoreContext stops the asynchronous store migration before taking the
// store write lock, so shutdown cannot wait forever on the migration's lease.
func CloseStoreContext(ctx context.Context) error {
if err := stopStoreMigrationForClose(ctx); err != nil {
if err := StopStoreMigrationForShutdown(ctx); err != nil {
clearStoreClosing()
return err
}
@@ -278,13 +247,11 @@ func CloseStoreContext(ctx context.Context) error {
storeMu.Lock()
defer storeMu.Unlock()
var err error
if store != nil {
err := store.Close()
err = store.Close()
store = nil
storeFingerprint = ""
return err
}
storeFingerprint = ""
return nil
return err
}
+22 -40
View File
@@ -224,54 +224,36 @@ func finishStoreMigration(status string, err error) {
func CancelStoreMigration() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return stopStoreMigration(ctx, true)
}
func stopStoreMigration(ctx context.Context, requireRunning bool) error {
storeMigMu.Lock()
cancel := storeMigCancel
done := storeMigDone
stop, done := storeMigCancel, storeMigDone
storeMigMu.Unlock()
if cancel == nil {
if requireRunning {
return fmt.Errorf("no store migration is currently running")
}
return nil
}
cancel()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
}
}
func stopStoreMigrationForClose(ctx context.Context) error {
storeMigMu.Lock()
storeClosing = true
cancel := storeMigCancel
done := storeMigDone
storeMigMu.Unlock()
if cancel == nil {
return nil
}
cancel()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
if stop == nil {
return fmt.Errorf("no store migration is currently running")
}
return awaitStoreMigrationStop(ctx, stop, done)
}
// StopStoreMigrationForShutdown prevents a new store migration and waits for
// an active one to release its exclusive lease before report batches flush.
func StopStoreMigrationForShutdown(ctx context.Context) error {
return stopStoreMigrationForClose(ctx)
storeMigMu.Lock()
storeClosing = true
stop, done := storeMigCancel, storeMigDone
storeMigMu.Unlock()
if stop == nil {
return nil
}
return awaitStoreMigrationStop(ctx, stop, done)
}
func awaitStoreMigrationStop(ctx context.Context, stop context.CancelFunc, done <-chan struct{}) error {
stop()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("wait for store migration to cancel: %w", ctx.Err())
}
}
func clearStoreClosing() {
+21 -4
View File
@@ -4,19 +4,33 @@ package notifications
import (
"encoding/json"
"errors"
"github.com/komari-monitor/komari/database"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/managedconfig"
"github.com/komari-monitor/komari/utils/messageSender"
"github.com/komari-monitor/komari/utils/messageSender/javascript"
"github.com/komari-monitor/komari/utils/messageSender/webhook"
)
// Initialize registers the built-in webhook channel and seeds its defaults.
func Initialize() error {
// Initialize registers the built-in channels and seeds their defaults.
func Initialize() (err error) {
if err := webhook.Register(); err != nil {
return err
}
registered := []string{"webhook"}
defer func() {
if err != nil {
for _, id := range registered {
err = errors.Join(err, messageSender.UnregisterNotificationChannel(id))
}
}
}()
if err := javascript.Register(); err != nil {
return err
}
registered = append(registered, javascript.ChannelID)
for _, item := range messageSender.ListNotificationChannels() {
if _, err := database.GetMessageSenderConfigByName(item.ID); err == nil {
continue
@@ -41,7 +55,10 @@ func Initialize() error {
return nil
}
// Shutdown unregisters the built-in webhook channel.
// Shutdown unregisters the built-in channels and releases their runtimes.
func Shutdown() error {
return messageSender.UnregisterNotificationChannel("webhook")
return errors.Join(
messageSender.UnregisterNotificationChannel(javascript.ChannelID),
messageSender.UnregisterNotificationChannel("webhook"),
)
}
+21 -19
View File
@@ -5,8 +5,10 @@ import (
"net/http"
"strings"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/rpc"
)
@@ -17,41 +19,41 @@ const principalContextKey = "principal"
// routeRequestContext builds the JS request context with the caller identity
// resolved by IdentityMiddleware. Raw credentials (session/client tokens) are
// intentionally not exposed.
func routeRequestContext(vm *goja.Runtime, c *gin.Context) *goja.Object {
ctx := vm.NewObject()
func routeRequestContext(vm *moejs.Runtime, c *gin.Context) *engine.Object {
ctx := vm.Realm().NewObject()
principal := rpcPrincipalFromGin(c)
p := vm.NewObject()
p := vm.Realm().NewObject()
if principal != nil {
_ = p.Set("type", principalTypeString(principal.Type))
_ = p.Set("roles", principal.Roles)
_ = p.Set("user_uuid", principal.UserUUID)
_ = p.Set("client_uuid", principal.ClientUUID)
_ = p.Set("is_api_key", principal.IsAPIKey)
_ = native.Set(vm, p, "type", principalTypeString(principal.Type))
_ = native.Set(vm, p, "roles", principal.Roles)
_ = native.Set(vm, p, "user_uuid", principal.UserUUID)
_ = native.Set(vm, p, "client_uuid", principal.ClientUUID)
_ = native.Set(vm, p, "is_api_key", principal.IsAPIKey)
}
_ = ctx.Set("principal", p)
_ = native.Set(vm, ctx, "principal", p)
if role, ok := c.Get("role"); ok {
if s, ok := role.(string); ok {
_ = ctx.Set("role", s)
_ = native.Set(vm, ctx, "role", s)
}
}
if uuid, ok := c.Get("uuid"); ok {
_ = ctx.Set("user_uuid", uuid)
_ = native.Set(vm, ctx, "user_uuid", uuid)
}
if clientUUID, ok := c.Get("client_uuid"); ok {
_ = ctx.Set("client_uuid", clientUUID)
_ = native.Set(vm, ctx, "client_uuid", clientUUID)
}
_ = ctx.Set("remote_ip", c.ClientIP())
_ = ctx.Set("user_agent", c.GetHeader("User-Agent"))
_ = native.Set(vm, ctx, "remote_ip", c.ClientIP())
_ = native.Set(vm, ctx, "user_agent", c.GetHeader("User-Agent"))
return ctx
}
// hookRequestContext builds the JS request context at the raw HTTP layer
// (before the gin identity middleware runs): only network metadata is
// available there.
func hookRequestContext(vm *goja.Runtime, r *http.Request) *goja.Object {
ctx := vm.NewObject()
_ = ctx.Set("remote_ip", remoteIP(r))
_ = ctx.Set("user_agent", r.Header.Get("User-Agent"))
func hookRequestContext(vm *moejs.Runtime, r *http.Request) *engine.Object {
ctx := vm.Realm().NewObject()
_ = native.Set(vm, ctx, "remote_ip", remoteIP(r))
_ = native.Set(vm, ctx, "user_agent", r.Header.Get("User-Agent"))
return ctx
}
+8 -7
View File
@@ -5,8 +5,9 @@ import (
"fmt"
"sync/atomic"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/internal/scheduler"
"github.com/komari-monitor/komari/pkg/jsruntime"
)
// cronJobSeq gives every registered cron job a process-unique id. Job names
@@ -20,7 +21,7 @@ var cronJobSeq atomic.Uint64
// evaluation and takes the manager lock itself; the critical section is short
// and never waits on the event loop, so no lock cycle is possible. The job is
// removed automatically when the plugin is unloaded or fails to load.
func (m *Manager) registerCron(short, spec string, fn goja.Callable) error {
func (m *Manager) registerCron(short, spec string, fn moejs.Value) error {
m.mu.Lock()
defer m.mu.Unlock()
inst, ok := m.instances[short]
@@ -37,7 +38,7 @@ func (m *Manager) registerCron(short, spec string, fn goja.Callable) error {
name := fmt.Sprintf("plugin:%s:%d", short, cronJobSeq.Add(1))
if err := scheduler.AddContextFunc(name, spec, false, func(ctx context.Context) {
m.runCronJob(short, name, fn)
m.runCronJob(short, name, fn, host)
}); err != nil {
return err
}
@@ -51,7 +52,7 @@ func (m *Manager) registerCron(short, spec string, fn goja.Callable) error {
// on a scheduler goroutine and drops the fire when the plugin was unloaded
// (the job is cancelled with the instance, but a fire may already be queued).
// Errors thrown by the handler are reported to the plugin log by RunJob.
func (m *Manager) runCronJob(short, name string, fn goja.Callable) {
func (m *Manager) runCronJob(short, name string, fn moejs.Value, owner *jsruntime.Host) {
inst := m.instanceFor(short)
if inst == nil {
return
@@ -60,12 +61,12 @@ func (m *Manager) runCronJob(short, name string, fn goja.Callable) {
host := inst.host
alive := inst.runtime != nil
inst.mu.RUnlock()
if !alive || host == nil {
if !alive || host == nil || host != owner {
return
}
host.RunOnLoop(func(vm *goja.Runtime) {
host.RunOnLoop(func(vm *moejs.Runtime) {
_ = host.RunJob(vm, "plugin cron "+name, func() error {
_, err := fn(goja.Undefined())
_, err := vm.CallFunction(fn, moejs.Undefined())
return err
})
})
+39 -37
View File
@@ -14,9 +14,11 @@ import (
"sync"
"time"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
// hookKind identifies the HTTP hook phase.
@@ -31,7 +33,7 @@ const (
type hookEntry struct {
short string
kind hookKind
fn goja.Callable
fn moejs.Value
host *jsruntime.Host
matcher *hookMatcher // nil matches every request
bodyLimit int64 // plugin-declared maxHTTPBodyBytes (0 = default)
@@ -211,36 +213,36 @@ func (m *Manager) wrapHandler(next http.Handler) http.Handler {
func (m *Manager) runRequestHook(r *http.Request, body []byte, h *hookEntry) ([]byte, error) {
nextBody := body
var hookErr error
queued, timedOut := runHookTurn(h.host, "plugin request hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin request hook "+h.short, func(vm *moejs.Runtime) {
reqObj := hookRequestObject(vm, r, nextBody)
hookErr = h.host.RunJob(vm, "plugin request hook "+h.short, func() error {
_, err := h.fn(goja.Undefined(), reqObj)
_, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(reqObj))
return err
})
if hookErr != nil {
return
}
if value := reqObj.Get("method"); value != nil && !goja.IsUndefined(value) {
if method := strings.ToUpper(strings.TrimSpace(value.String())); method != "" {
if value := native.Get(vm, reqObj, "method"); !value.IsUndefined() {
if method := strings.ToUpper(strings.TrimSpace(native.String(vm, value))); method != "" {
r.Method = method
}
}
if value := reqObj.Get("url"); value != nil && !goja.IsUndefined(value) {
if raw := strings.TrimSpace(value.String()); raw != "" {
if value := native.Get(vm, reqObj, "url"); !value.IsUndefined() {
if raw := strings.TrimSpace(native.String(vm, value)); raw != "" {
if parsed, err := url.Parse(raw); err == nil {
r.URL = parsed
r.RequestURI = raw
}
}
}
if value := reqObj.Get("headers"); value != nil && !goja.IsUndefined(value) {
if value := native.Get(vm, reqObj, "headers"); !value.IsUndefined() {
var raw map[string]any
if err := vm.ExportTo(value, &raw); err == nil {
if err := native.ExportTo(vm, value, &raw); err == nil {
r.Header = normalizeHeader(raw)
}
}
if value := reqObj.Get("body"); value != nil && !goja.IsUndefined(value) {
nextBody = []byte(value.String())
if value := native.Get(vm, reqObj, "body"); !value.IsUndefined() {
nextBody = []byte(native.String(vm, value))
}
})
if !queued {
@@ -256,29 +258,29 @@ func (m *Manager) runRequestHook(r *http.Request, body []byte, h *hookEntry) ([]
// the JS-side modifications to the captured response.
func (m *Manager) runResponseHook(r *http.Request, bw *bufferedResponseWriter, h *hookEntry) error {
var hookErr error
queued, timedOut := runHookTurn(h.host, "plugin response hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin response hook "+h.short, func(vm *moejs.Runtime) {
resObj := hookResponseObject(vm, bw)
hookErr = h.host.RunJob(vm, "plugin response hook "+h.short, func() error {
_, err := h.fn(goja.Undefined(), hookRequestObject(vm, r, nil), resObj)
_, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(hookRequestObject(vm, r, nil)), engine.ObjectValue(resObj))
return err
})
if hookErr != nil {
return
}
status := bw.status()
if value := resObj.Get("statusCode"); value != nil && !goja.IsUndefined(value) {
status = int(value.ToInteger())
if value := native.Get(vm, resObj, "statusCode"); !value.IsUndefined() {
status = int(native.Int(vm, value))
}
header := bw.Header().Clone()
if value := resObj.Get("headers"); value != nil && !goja.IsUndefined(value) {
if value := native.Get(vm, resObj, "headers"); !value.IsUndefined() {
var raw map[string]any
if err := vm.ExportTo(value, &raw); err == nil {
if err := native.ExportTo(vm, value, &raw); err == nil {
header = normalizeHeader(raw)
}
}
responseBody := bw.bodyBytes()
if value := resObj.Get("body"); value != nil && !goja.IsUndefined(value) {
responseBody = []byte(value.String())
if value := native.Get(vm, resObj, "body"); !value.IsUndefined() {
responseBody = []byte(native.String(vm, value))
}
bw.apply(status, header, responseBody)
})
@@ -298,16 +300,16 @@ func (m *Manager) runResponseHook(r *http.Request, bw *bufferedResponseWriter, h
// runHookTurn queues one hook callback on the plugin event loop and waits
// for it with the runtime timeout. It reports whether the job was queued and
// whether it timed out.
func runHookTurn(host *jsruntime.Host, name string, job func(vm *goja.Runtime)) (queued, timedOut bool) {
func runHookTurn(host *jsruntime.Host, name string, job func(vm *moejs.Runtime)) (queued, timedOut bool) {
return runHookTurnTimeout(host, name, host.Timeout(), job)
}
// runHookTurnTimeout is runHookTurn with an explicit wait budget. The
// callback keeps running on the plugin loop after a timeout; the waiter gives
// up and the caller falls back to passing the message through.
func runHookTurnTimeout(host *jsruntime.Host, name string, timeout time.Duration, job func(vm *goja.Runtime)) (queued, timedOut bool) {
func runHookTurnTimeout(host *jsruntime.Host, name string, timeout time.Duration, job func(vm *moejs.Runtime)) (queued, timedOut bool) {
done := make(chan struct{})
if !host.RunOnLoop(func(vm *goja.Runtime) {
if !host.RunOnLoop(func(vm *moejs.Runtime) {
defer close(done)
job(vm)
}) {
@@ -326,29 +328,29 @@ func (m *Manager) logHookError(h *hookEntry, err error) {
}
// hookRequestObject builds the mutable JS request object.
func hookRequestObject(vm *goja.Runtime, r *http.Request, body []byte) *goja.Object {
req := vm.NewObject()
_ = req.Set("method", r.Method)
_ = req.Set("url", r.URL.RequestURI())
_ = req.Set("headers", headerToMap(r.Header))
func hookRequestObject(vm *moejs.Runtime, r *http.Request, body []byte) *engine.Object {
req := vm.Realm().NewObject()
_ = native.Set(vm, req, "method", r.Method)
_ = native.Set(vm, req, "url", r.URL.RequestURI())
_ = native.Set(vm, req, "headers", headerToMap(r.Header))
query := make(map[string]string, len(r.URL.Query()))
for name, values := range r.URL.Query() {
query[name] = strings.Join(values, ",")
}
_ = req.Set("query", query)
_ = req.Set("body", string(body))
_ = req.Set("context", hookRequestContext(vm, r))
_ = native.Set(vm, req, "query", query)
_ = native.Set(vm, req, "body", string(body))
_ = native.Set(vm, req, "context", hookRequestContext(vm, r))
return req
}
// hookResponseObject builds the mutable JS response object from the captured
// response.
func hookResponseObject(vm *goja.Runtime, bw *bufferedResponseWriter) *goja.Object {
res := vm.NewObject()
_ = res.Set("statusCode", bw.status())
_ = res.Set("statusMessage", "")
_ = res.Set("headers", headerToMap(bw.Header()))
_ = res.Set("body", string(bw.bodyBytes()))
func hookResponseObject(vm *moejs.Runtime, bw *bufferedResponseWriter) *engine.Object {
res := vm.Realm().NewObject()
_ = native.Set(vm, res, "statusCode", bw.status())
_ = native.Set(vm, res, "statusMessage", "")
_ = native.Set(vm, res, "headers", headerToMap(bw.Header()))
_ = native.Set(vm, res, "body", string(bw.bodyBytes()))
return res
}
+22 -19
View File
@@ -7,9 +7,11 @@ import (
"sync"
"time"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/utils/messageSender"
)
@@ -19,7 +21,7 @@ func (m *Manager) registerNotificationChannel(
short string,
id string,
configuration models.Configuration,
handler goja.Callable,
handler moejs.Value,
host *jsruntime.Host,
) error {
inst := m.instanceFor(short)
@@ -52,11 +54,11 @@ func (m *Manager) registerNotificationChannel(
type pluginNotificationChannel struct {
short string
host *jsruntime.Host
handler goja.Callable
handler moejs.Value
}
func (c *pluginNotificationChannel) Send(ctx context.Context, notification messageSender.Notification, config map[string]any) error {
if c.host == nil || c.handler == nil {
if c.host == nil || c.handler.IsUndefined() {
return fmt.Errorf("plugin %q notification channel is closed", c.short)
}
result := make(chan error, 1)
@@ -65,8 +67,8 @@ func (c *pluginNotificationChannel) Send(ctx context.Context, notification messa
once.Do(func() { result <- err })
}
queued := c.host.RunOnLoop(func(vm *goja.Runtime) {
var value goja.Value
queued := c.host.RunOnLoop(func(vm *moejs.Runtime) {
var value moejs.Value
runErr := c.host.RunJob(vm, "plugin notification "+c.short, func() error {
raw, err := json.Marshal(notification)
if err != nil {
@@ -77,10 +79,10 @@ func (c *pluginNotificationChannel) Send(ctx context.Context, notification messa
return err
}
var callErr error
value, callErr = c.handler(
goja.Undefined(),
vm.ToValue(notificationValue),
vm.ToValue(config),
value, callErr = vm.CallFunction(c.handler,
moejs.Undefined(),
native.From(vm, notificationValue),
native.From(vm, config),
)
return callErr
})
@@ -88,27 +90,28 @@ func (c *pluginNotificationChannel) Send(ctx context.Context, notification messa
finish(runErr)
return
}
_, ok := value.Export().(*goja.Promise)
_, _, ok := moejs.PromiseResult(value)
if !ok {
finish(nil)
return
}
then, ok := goja.AssertFunction(value.ToObject(vm).Get("then"))
if !ok {
then := native.Get(vm, native.Object(vm, value), "then")
if !engine.IsCallable(then) {
finish(fmt.Errorf("plugin %q notification channel returned a Promise without then", c.short))
return
}
onFulfilled := vm.ToValue(func(goja.FunctionCall) goja.Value {
onFulfilled := native.Bind(vm, "notificationFulfilled", 0, func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
finish(nil)
return goja.Undefined()
return moejs.Undefined(), nil
})
onRejected := vm.ToValue(func(call goja.FunctionCall) goja.Value {
finish(fmt.Errorf("plugin %q notification channel failed: %v", c.short, call.Argument(0)))
return goja.Undefined()
onRejected := native.Bind(vm, "notificationRejected", 1, func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
finish(fmt.Errorf("plugin %q notification channel failed: %s", c.short, native.String(vm, moejs.Arg(args, 0))))
return moejs.Undefined(), nil
})
if _, err := then(value, onFulfilled, onRejected); err != nil {
if _, err := vm.CallFunction(then, value, onFulfilled, onRejected); err != nil {
finish(fmt.Errorf("plugin %q notification channel Promise failed: %w", c.short, err))
}
})
if !queued {
return fmt.Errorf("plugin %q notification runtime is closed", c.short)
+6 -6
View File
@@ -29,12 +29,12 @@ import (
"sync"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/require"
"github.com/Calcium-Ion/moejs"
"github.com/gin-gonic/gin"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/internal/scheduler"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"github.com/komari-monitor/komari/pkg/rpc"
"github.com/komari-monitor/komari/web/connection"
)
@@ -118,9 +118,9 @@ type Instance struct {
dir string
runtime *jsruntime.Runtime
host *jsruntime.Host
handlers map[string]goja.Callable // "METHOD path" -> current route handler
handlers map[string]moejs.Value // "METHOD path" -> current route handler
statics map[string]*staticConfig // mount path -> static folder config
rpcMethods map[string]goja.Callable // registered RPC method -> JS handler
rpcMethods map[string]moejs.Value // registered RPC method -> JS handler
cronJobs []string // scheduler job names, removed on unload
notificationChannels map[string]struct{}
}
@@ -262,7 +262,7 @@ func (m *Manager) load(short string) error {
logs.Reset()
_, _ = logs.Write([]byte("[plugin] loading " + short + "\n"))
inst := &Instance{info: info, dir: dir, handlers: make(map[string]goja.Callable), statics: make(map[string]*staticConfig)}
inst := &Instance{info: info, dir: dir, handlers: make(map[string]moejs.Value), statics: make(map[string]*staticConfig)}
m.mu.Lock()
if _, ok := m.instances[short]; ok {
m.mu.Unlock()
@@ -684,7 +684,7 @@ func (m *Manager) hooksOf(kind hookKind) []*hookEntry {
// during script evaluation; it takes the manager lock itself. matcher is
// nil for unfiltered hooks; the plugin-declared body limit is captured at
// registration so unloaded plugins never contribute a stale limit.
func (m *Manager) registerHook(short string, kind hookKind, fn goja.Callable, matcher *hookMatcher) {
func (m *Manager) registerHook(short string, kind hookKind, fn moejs.Value, matcher *hookMatcher) {
m.mu.Lock()
defer m.mu.Unlock()
inst := m.instances[short]
+507
View File
@@ -0,0 +1,507 @@
package plugin
import (
"errors"
"fmt"
"math"
"net"
"sync"
"time"
"unicode/utf8"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gorilla/websocket"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/web/connection"
)
const (
routeWSOpen = 1
routeWSClosing = 2
routeWSClosed = 3
routeWSMaxQueuedFrames = 64
routeWSMaxQueuedBytes = 16 << 20
routeWSWriteTimeout = 5 * time.Second
routeWSCloseTimeout = 5 * time.Second
)
type routeWSFrame struct {
kind int
data []byte
}
type routeWSCloseEvent struct {
code int
reason string
clean bool
message string
}
// routeWebSocket is registered before hijacking: runtime shutdown must be able
// to close the network connection even while UpgradeSafeConn waits for a
// synchronous wsConnect hook. No SafeConn operation or hook runs on the VM loop.
// mu protects transport state only; jsObject belongs exclusively to the loop.
type routeWebSocket struct {
host *jsruntime.Host
mu sync.Mutex
resourceID uint64
netConn net.Conn
conn *connection.SafeConn
raw *websocket.Conn
state int
started bool
finished bool
notified bool
result routeWSCloseEvent
closeTimer *time.Timer
queue []routeWSFrame
queuedCount int // includes the writer's in-flight frame
queuedBytes int
wake chan struct{}
done chan struct{}
closeRequested bool
closeCode int
closeReason string
closeWritten bool
peerClosed bool
peerCode int
peerReason string
pong string
pongPending bool
jsObject *engine.Object
}
func newRouteWebSocket(host *jsruntime.Host) *routeWebSocket {
ws := &routeWebSocket{
host: host, state: routeWSClosed,
wake: make(chan struct{}, 1), done: make(chan struct{}),
}
// AddResource may call abort synchronously, or shutdown may race with its
// return. Never publish the ID without checking whether teardown already ran.
id := host.AddResource(ws.abort)
ws.mu.Lock()
ws.resourceID = id
finished := ws.finished
ws.mu.Unlock()
if finished {
host.RemoveResource(id)
}
return ws
}
// captureConn is called by the HTTP writer's Hijack method, before the
// upgrader can enter any hooks. It also handles shutdown winning that race.
func (ws *routeWebSocket) captureConn(conn net.Conn) {
ws.mu.Lock()
finished := ws.finished
if !finished {
ws.netConn = conn
}
ws.mu.Unlock()
if finished && conn != nil {
_ = conn.Close()
}
}
// attach runs on the HTTP pump, never the VM loop: GetConn takes SafeConn's
// write mutex. The underlying Gorilla connection is private to these pumps.
func (ws *routeWebSocket) attach(conn *connection.SafeConn) bool {
if conn == nil {
return false
}
raw := conn.GetConn()
raw.SetReadLimit(maxWSFrameBytes)
raw.SetCloseHandler(func(code int, reason string) error {
ws.mu.Lock()
if !ws.finished {
ws.peerClosed, ws.peerCode, ws.peerReason = true, code, reason
// The peer cannot consume further data. Drop queued frames and
// prioritize the close reply (the in-flight writer stays counted).
for _, frame := range ws.queue {
ws.queuedCount--
ws.queuedBytes -= len(frame.data)
}
ws.queue = nil
ws.pong, ws.pongPending = "", false
if !ws.closeRequested {
ws.requestCloseLocked(code, reason)
}
}
ws.mu.Unlock()
ws.signal()
return nil // the writer, not the reader, sends the close reply
})
raw.SetPingHandler(func(data string) error {
ws.mu.Lock()
if !ws.finished {
// A ping flood must not allocate an unbounded control queue. RFC 6455
// allows replying only to the most recently processed ping.
ws.pong, ws.pongPending = data, true
}
ws.mu.Unlock()
ws.signal()
return nil
})
ws.mu.Lock()
if ws.finished || ws.conn != nil {
ws.mu.Unlock()
_ = raw.Close()
go func() { _ = conn.Close() }()
return false
}
ws.conn, ws.raw, ws.state = conn, raw, routeWSOpen
if ws.netConn == nil {
ws.netConn = raw.UnderlyingConn()
}
ws.mu.Unlock()
return true
}
// abort is a non-waiting resource callback. Close the captured network socket
// first to unblock I/O; SafeConn.Close may wait for same-plugin hooks, so it
// must happen separately, after the socket is already closed.
func (ws *routeWebSocket) abort() {
ws.finish(routeWSCloseEvent{code: websocket.CloseAbnormalClosure})
}
func (ws *routeWebSocket) finish(result routeWSCloseEvent) {
ws.mu.Lock()
if ws.finished {
ws.mu.Unlock()
return
}
ws.finished, ws.state, ws.result = true, routeWSClosed, result
ws.queue = nil
ws.queuedCount, ws.queuedBytes = 0, 0
if ws.closeTimer != nil {
ws.closeTimer.Stop()
}
netConn, conn, id := ws.netConn, ws.conn, ws.resourceID
notify := ws.started && !ws.notified
if notify {
ws.notified = true
}
ws.mu.Unlock()
if netConn != nil {
_ = netConn.Close()
}
ws.host.RemoveResource(id)
close(ws.done)
if conn != nil {
go func() { _ = conn.Close() }()
}
if notify {
ws.notifyClose(result)
}
}
// start is called on a later loop turn than promise resolution. This barrier
// lets .then/await reactions install handlers before any data or close event.
func (ws *routeWebSocket) start() {
ws.mu.Lock()
if ws.started {
ws.mu.Unlock()
return
}
ws.started = true
if ws.finished {
notify, result := !ws.notified, ws.result
ws.notified = true
ws.mu.Unlock()
if notify {
ws.notifyClose(result)
}
return
}
attached := ws.conn != nil
ws.mu.Unlock()
if !attached || !ws.host.ResourcesOpen() {
ws.abort()
return
}
go ws.writePump()
go ws.readPump()
}
func (ws *routeWebSocket) signal() {
select {
case ws.wake <- struct{}{}:
default:
}
}
func (ws *routeWebSocket) requestCloseLocked(code int, reason string) {
ws.state, ws.closeRequested = routeWSClosing, true
ws.closeCode, ws.closeReason = code, reason
ws.closeTimer = time.AfterFunc(routeWSCloseTimeout, func() {
ws.fail(errors.New("WebSocket close handshake timed out"))
})
}
func (ws *routeWebSocket) fail(err error) {
ws.finish(routeWSCloseEvent{code: websocket.CloseAbnormalClosure, message: err.Error()})
}
func (ws *routeWebSocket) writePump() {
for {
ws.mu.Lock()
if ws.finished {
ws.mu.Unlock()
return
}
if ws.closeWritten && ws.peerClosed {
result := routeWSCloseEvent{code: ws.peerCode, reason: ws.peerReason, clean: true}
ws.mu.Unlock()
ws.finish(result)
return
}
var frame routeWSFrame
control, haveFrame := false, false
switch {
case ws.pongPending && !ws.closeWritten:
frame = routeWSFrame{kind: websocket.PongMessage, data: []byte(ws.pong)}
ws.pong, ws.pongPending, control, haveFrame = "", false, true, true
case len(ws.queue) != 0:
frame, haveFrame = ws.queue[0], true
ws.queue[0] = routeWSFrame{}
ws.queue = ws.queue[1:]
case ws.closeRequested && !ws.closeWritten:
frame = routeWSFrame{kind: websocket.CloseMessage, data: websocket.FormatCloseMessage(ws.closeCode, ws.closeReason)}
control, haveFrame = true, true
}
conn, raw := ws.conn, ws.raw
ws.mu.Unlock()
if !haveFrame {
select {
case <-ws.wake:
case <-ws.done:
return
}
continue
}
deadline := time.Now().Add(routeWSWriteTimeout)
var err error
if control {
err = conn.WriteControl(frame.kind, frame.data, deadline)
} else {
err = raw.SetWriteDeadline(deadline)
if err == nil {
err = conn.WriteMessage(frame.kind, frame.data)
}
}
if err != nil {
ws.fail(err)
return
}
ws.mu.Lock()
if !ws.finished {
if frame.kind == websocket.CloseMessage {
ws.closeWritten = true
} else if !control {
ws.queuedCount--
ws.queuedBytes -= len(frame.data)
}
}
ws.mu.Unlock()
}
}
func (ws *routeWebSocket) readPump() {
for {
kind, data, err := ws.conn.ReadMessage()
if err != nil {
ws.mu.Lock()
peerClosed := ws.peerClosed
ws.mu.Unlock()
if peerClosed {
ws.signal() // the writer completes the close handshake
} else {
ws.fail(err)
}
return
}
if len(data) > maxWSFrameBytes {
ws.fail(fmt.Errorf("WebSocket message exceeds %d bytes", maxWSFrameBytes))
return
}
if kind != websocket.TextMessage && kind != websocket.BinaryMessage {
continue
}
if kind == websocket.TextMessage && !utf8.Valid(data) {
ws.fail(errors.New("WebSocket text message is not valid UTF-8"))
return
}
ack := make(chan struct{})
if !ws.host.RunOnLoop(func(vm *moejs.Runtime) {
defer close(ack)
jobErr := ws.host.RunJob(vm, "plugin WebSocket.onmessage", func() error {
ws.mu.Lock()
open := ws.state == routeWSOpen
ws.mu.Unlock()
if !open || ws.jsObject == nil {
return nil
}
var value any = string(data)
if kind == websocket.BinaryMessage {
buffer, err := vm.Realm().NewArrayBuffer(data)
if err != nil {
return err
}
value = engine.ObjectValue(buffer)
}
event := vm.Realm().NewObject()
if err := native.Set(vm, event, "data", value); err != nil {
return err
}
return ws.callHandler(vm, "onmessage", event)
})
if jobErr != nil {
ws.fail(jobErr)
}
}) {
ws.abort()
return
}
// Only one inbound payload may wait for JavaScript. Cancellation must
// also release this reader when queued jobs are discarded on shutdown.
select {
case <-ack:
case <-ws.done:
return
}
}
}
// All methods below that touch jsObject or JS values run only on the host loop.
func (ws *routeWebSocket) object(vm *moejs.Runtime) *engine.Object {
if ws.jsObject != nil {
return ws.jsObject
}
obj := vm.Realm().NewObject()
ws.jsObject = obj
for _, name := range []string{"onmessage", "onerror", "onclose"} {
_ = native.Set(vm, obj, name, moejs.Null())
}
getter := native.Bind(vm, "get readyState", 0, func(_ *moejs.Realm, _ moejs.Value, _ []moejs.Value) (moejs.Value, error) {
ws.mu.Lock()
state := ws.state
ws.mu.Unlock()
return native.From(vm, state), nil
})
desc := engine.AccessorDescriptor(getter.AsObject(), nil, 0)
desc.SetEnumerable(true)
if err := obj.DefinePropertyOrThrow(vm.Realm(), vm.Realm().KeyFromGoString("readyState"), desc); err != nil {
panic(err)
}
_ = native.Function(vm, obj, "send", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := moejs.Arg(args, 0)
kind := websocket.BinaryMessage
var data []byte
if value.IsString() {
kind = websocket.TextMessage
text := native.String(vm, value)
if len(text) > maxWSFrameBytes {
return moejs.Undefined(), native.TypeError(vm, "WebSocket message exceeds %d bytes", maxWSFrameBytes)
}
data = []byte(text)
} else if value.IsObject() {
var ok bool
data, ok = value.AsObject().BufferData()
if !ok {
return moejs.Undefined(), native.TypeError(vm, "WebSocket.send requires a string, ArrayBuffer or ArrayBuffer view")
}
} else {
return moejs.Undefined(), native.TypeError(vm, "WebSocket.send requires a string, ArrayBuffer or ArrayBuffer view")
}
if len(data) > maxWSFrameBytes {
return moejs.Undefined(), native.TypeError(vm, "WebSocket message exceeds %d bytes", maxWSFrameBytes)
}
ws.mu.Lock()
if ws.state != routeWSOpen {
ws.mu.Unlock()
return moejs.Undefined(), native.TypeError(vm, "WebSocket is not open")
}
if ws.queuedCount >= routeWSMaxQueuedFrames || len(data) > routeWSMaxQueuedBytes-ws.queuedBytes {
ws.mu.Unlock()
return moejs.Undefined(), native.TypeError(vm, "WebSocket send queue is full")
}
ws.queue = append(ws.queue, routeWSFrame{kind: kind, data: append([]byte(nil), data...)})
ws.queuedCount++
ws.queuedBytes += len(data)
ws.mu.Unlock()
ws.signal()
return moejs.Undefined(), nil
})
_ = native.Function(vm, obj, "close", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
code := websocket.CloseNormalClosure
if v := moejs.Arg(args, 0); !v.IsUndefined() {
number := native.Float(vm, v)
if math.IsNaN(number) || math.IsInf(number, 0) || math.Trunc(number) != number || (number != 1000 && (number < 3000 || number > 4999)) {
return moejs.Undefined(), native.TypeError(vm, "WebSocket close code must be 1000 or between 3000 and 4999")
}
code = int(number)
}
reason := ""
if v := moejs.Arg(args, 1); !v.IsUndefined() {
reason = native.String(vm, v)
}
if len(reason) > 123 || !utf8.ValidString(reason) {
return moejs.Undefined(), native.TypeError(vm, "WebSocket close reason must be valid UTF-8 and at most 123 bytes")
}
ws.mu.Lock()
if ws.state == routeWSOpen {
ws.requestCloseLocked(code, reason)
}
ws.mu.Unlock()
ws.signal()
return moejs.Undefined(), nil
})
return obj
}
func (ws *routeWebSocket) callHandler(vm *moejs.Runtime, name string, event *engine.Object) error {
if ws.jsObject == nil {
return nil
}
callback, err := vm.Get(engine.ObjectValue(ws.jsObject), name)
if err != nil || !engine.IsCallable(callback) {
return err
}
_, err = vm.CallFunction(callback, engine.ObjectValue(ws.jsObject), engine.ObjectValue(event))
return err
}
func (ws *routeWebSocket) notifyClose(result routeWSCloseEvent) {
ws.host.RunOnLoop(func(vm *moejs.Runtime) {
if ws.jsObject == nil {
return
}
defer func() { ws.jsObject = nil }()
if result.message != "" {
_ = ws.host.RunJob(vm, "plugin WebSocket.onerror", func() error {
event := vm.Realm().NewObject()
if err := native.Set(vm, event, "message", result.message); err != nil {
return err
}
return ws.callHandler(vm, "onerror", event)
})
}
// A throwing onerror must not suppress the final onclose callback.
_ = ws.host.RunJob(vm, "plugin WebSocket.onclose", func() error {
event := vm.Realm().NewObject()
_ = native.Set(vm, event, "code", result.code)
_ = native.Set(vm, event, "reason", result.reason)
_ = native.Set(vm, event, "wasClean", result.clean)
return ws.callHandler(vm, "onclose", event)
})
})
}
+24 -22
View File
@@ -7,7 +7,8 @@ import (
"strings"
"time"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/rpc"
)
@@ -15,7 +16,7 @@ import (
// JavaScript function receiving the JSON-RPC params and returning the result
// (or throwing an Error with optional code/data). Called from the plugin's
// own event loop during script evaluation; it takes the manager lock itself.
func (m *Manager) registerRPC(short, method string, handler goja.Callable) error {
func (m *Manager) registerRPC(short, method string, handler moejs.Value) error {
method = strings.TrimSpace(method)
if method == "" || strings.HasPrefix(method, "rpc.") {
return fmt.Errorf("invalid RPC method name %q", method)
@@ -28,7 +29,7 @@ func (m *Manager) registerRPC(short, method string, handler goja.Callable) error
}
inst.mu.Lock()
if inst.rpcMethods == nil {
inst.rpcMethods = make(map[string]goja.Callable)
inst.rpcMethods = make(map[string]moejs.Value)
}
if _, exists := inst.rpcMethods[method]; exists {
inst.mu.Unlock()
@@ -57,24 +58,24 @@ func (m *Manager) rpcHandler(short, method string) rpc.Handler {
host := inst.host
alive := inst.runtime != nil
inst.mu.RUnlock()
if !alive || handler == nil || host == nil {
if !alive || handler.IsUndefined() || host == nil {
return nil, &rpc.JsonRpcError{Code: rpc.MethodNotFound, Message: "method not found", Data: method}
}
resultCh := make(chan any, 1)
errCh := make(chan *rpc.JsonRpcError, 1)
queued := host.RunOnLoop(func(vm *goja.Runtime) {
queued := host.RunOnLoop(func(vm *moejs.Runtime) {
_ = host.RunJob(vm, "plugin rpc "+method, func() error {
params := vm.ToValue(req.Params)
if params == nil {
params = goja.Null()
params := native.From(vm, req.Params)
if params.IsUndefined() {
params = moejs.Null()
}
value, callErr := handler(goja.Undefined(), params)
value, callErr := vm.CallFunction(handler, moejs.Undefined(), params)
if callErr != nil {
errCh <- jsErrorToRPCError(vm, callErr)
return nil // error is captured, do not double-report
}
exported, exportErr := exportJSValue(value)
exported, exportErr := exportJSValue(vm, value)
if exportErr != nil {
errCh <- &rpc.JsonRpcError{Code: rpc.InternalError, Message: exportErr.Error()}
return nil
@@ -99,38 +100,39 @@ func (m *Manager) rpcHandler(short, method string) rpc.Handler {
}
}
// exportJSValue exports a goja value into a JSON-RPC result. Circular
// exportJSValue exports a native JavaScript value into a JSON-RPC result. Circular
// structures and other export failures become an error instead of panicking
// inside the event loop.
func exportJSValue(value goja.Value) (result any, err error) {
func exportJSValue(vm *moejs.Runtime, value moejs.Value) (result any, err error) {
defer func() {
if recovered := recover(); recovered != nil {
result = nil
err = fmt.Errorf("export JavaScript value: %v", recovered)
}
}()
return value.Export(), nil
return vm.ToGo(value)
}
// jsErrorToRPCError converts a thrown JavaScript error into a JSON-RPC error,
// honoring optional code/data properties.
func jsErrorToRPCError(vm *goja.Runtime, err error) *rpc.JsonRpcError {
func jsErrorToRPCError(vm *moejs.Runtime, err error) *rpc.JsonRpcError {
code := rpc.InternalError
message := err.Error()
var data any
var exception *goja.Exception
var exception *moejs.Exception
if errors.As(err, &exception) {
if obj, ok := exception.Value().(*goja.Object); ok {
if value := obj.Get("code"); value != nil && !goja.IsUndefined(value) {
if n := value.ToInteger(); n != 0 {
if exception.Value.IsObject() {
obj := exception.Value.AsObject()
if value := native.Get(vm, obj, "code"); !value.IsUndefined() {
if n := native.Int(vm, value); n != 0 {
code = int(n)
}
}
if value := obj.Get("message"); value != nil && !goja.IsUndefined(value) && value.String() != "" {
message = value.String()
if value := native.Get(vm, obj, "message"); !value.IsUndefined() && native.String(vm, value) != "" {
message = native.String(vm, value)
}
if value := obj.Get("data"); value != nil && !goja.IsUndefined(value) {
if exported, exportErr := exportJSValue(value); exportErr == nil {
if value := native.Get(vm, obj, "data"); !value.IsUndefined() {
if exported, exportErr := exportJSValue(vm, value); exportErr == nil {
data = exported
}
}
+327 -149
View File
@@ -1,11 +1,13 @@
package plugin
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"net"
"net/http"
"os"
"path/filepath"
@@ -14,14 +16,17 @@ import (
"sync/atomic"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gin-gonic/gin"
"github.com/gorilla/websocket"
"github.com/komari-monitor/komari/database/models"
"github.com/komari-monitor/komari/pkg/jsruntime"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"github.com/komari-monitor/komari/pkg/rpc"
"github.com/komari-monitor/komari/web/api"
)
// registerServerModule registers the "server" native module for one plugin
@@ -29,7 +34,8 @@ import (
//
// server.route(method, path, handler) register an HTTP route on the host
// engine; handler receives (req, res)
// and must call res.end() to finish
// and finishes via res.end() or an
// awaited req.upgradeWebSocket()
// server.static(path, dir, opts) serve a folder from the plugin
// directory at a mount path; opts.spa
// makes unmatched paths fall back to
@@ -71,49 +77,50 @@ import (
// The host engine keeps a registered route slot after unload; requests then
// receive 404 until the plugin is loaded again.
func (m *Manager) registerServerModule(host *jsruntime.Host, registry *require.Registry, inst *Instance) {
registry.RegisterNativeModule("server", func(vm *goja.Runtime, module *goja.Object) {
exports := vm.NewObject()
_ = exports.Set("route", func(call goja.FunctionCall) goja.Value {
method := strings.ToUpper(strings.TrimSpace(call.Argument(0).String()))
path := call.Argument(1).String()
handler, ok := goja.AssertFunction(call.Argument(2))
registry.RegisterNativeModule("server", func(vm *moejs.Runtime, module *engine.Object) {
exports := vm.Realm().NewObject()
_ = native.Function(vm, exports, "route", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
method := strings.ToUpper(strings.TrimSpace(native.String(vm, moejs.Arg(args, 0))))
path := native.String(vm, moejs.Arg(args, 1))
handler := moejs.Arg(args, 2)
ok := engine.IsCallable(handler)
if method == "" {
panic(vm.NewTypeError("server.route requires an HTTP method"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires an HTTP method")
}
if path == "" || !strings.HasPrefix(path, "/") {
panic(vm.NewTypeError("server.route requires a path starting with /"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires a path starting with /")
}
if !ok {
panic(vm.NewTypeError("server.route requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires a function handler")
}
if !inst.info.Permissions.AllowRoutes {
panic(vm.NewTypeError("server.route requires the \"route\" permission (allowRoutes)"))
return moejs.Undefined(), native.TypeError(vm, "server.route requires the \"route\" permission (allowRoutes)")
}
if err := m.registerRoute(inst.info.Short, method, path, handler); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("static", func(call goja.FunctionCall) goja.Value {
mount := strings.TrimSpace(call.Argument(0).String())
_ = native.Function(vm, exports, "static", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
mount := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
mount = strings.TrimRight(mount, "/")
dir := strings.TrimSpace(call.Argument(1).String())
dir := strings.TrimSpace(native.String(vm, moejs.Arg(args, 1)))
if mount == "" || mount == "/" || !strings.HasPrefix(mount, "/") {
panic(vm.NewTypeError("server.static requires a mount path starting with / (and not \"/\")"))
return moejs.Undefined(), native.TypeError(vm, "server.static requires a mount path starting with / (and not \"/\")")
}
if dir == "" {
panic(vm.NewTypeError("server.static requires a folder name"))
return moejs.Undefined(), native.TypeError(vm, "server.static requires a folder name")
}
if !filepath.IsLocal(dir) {
panic(vm.NewTypeError("server.static folder must be a relative path inside the plugin directory"))
return moejs.Undefined(), native.TypeError(vm, "server.static folder must be a relative path inside the plugin directory")
}
if !inst.info.Permissions.AllowRoutes {
panic(vm.NewTypeError("server.static requires the \"route\" permission (allowRoutes)"))
return moejs.Undefined(), native.TypeError(vm, "server.static requires the \"route\" permission (allowRoutes)")
}
var opts map[string]any
if !goja.IsUndefined(call.Argument(2)) && !goja.IsNull(call.Argument(2)) {
if err := vm.ExportTo(call.Argument(2), &opts); err != nil {
panic(vm.NewTypeError("server.static options must be an object like { spa: true }"))
if !moejs.Arg(args, 2).IsUndefined() && !moejs.Arg(args, 2).IsNull() {
if err := native.ExportTo(vm, moejs.Arg(args, 2), &opts); err != nil {
return moejs.Undefined(), native.TypeError(vm, "server.static options must be an object like { spa: true }")
}
}
spa := false
@@ -121,168 +128,172 @@ func (m *Manager) registerServerModule(host *jsruntime.Host, registry *require.R
spa, _ = v.(bool)
}
if err := m.registerStatic(inst.info.Short, mount, dir, spa); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("call", func(call goja.FunctionCall) goja.Value {
method := strings.TrimSpace(call.Argument(0).String())
_ = native.Function(vm, exports, "call", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
method := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
if method == "" {
panic(vm.NewTypeError("server.call requires an RPC method name"))
return moejs.Undefined(), native.TypeError(vm, "server.call requires an RPC method name")
}
params := callParams(call)
params := callParams(vm, args)
promise, resolve, reject := vm.NewPromise()
go func() {
if !inst.info.Permissions.AllowSystemRPC {
host.RunOnLoop(func(vm *goja.Runtime) {
_ = reject(vm.NewGoError(errors.New("server.call requires the \"system RPC\" permission (allowSystemRPC)")))
host.RunOnLoop(func(vm *moejs.Runtime) {
_ = reject(native.GoError(vm, errors.New("server.call requires the \"system RPC\" permission (allowSystemRPC)")))
})
return
}
meta := &rpc.ContextMeta{Permission: rpc.RoleAdmin, Principal: rpc.PrincipalFromRole(rpc.RoleAdmin)}
ctx := rpc.NewContextWithMeta(context.Background(), meta)
resp := rpc.CallWithContext(ctx, nil, method, params)
host.RunOnLoop(func(vm *goja.Runtime) {
host.RunOnLoop(func(vm *moejs.Runtime) {
if resp.Error != nil {
_ = reject(jsRPCError(vm, resp.Error))
_ = reject(engine.ObjectValue(jsRPCError(vm, resp.Error)))
return
}
result, err := normalizeRPCValue(resp.Result)
if err != nil {
_ = reject(vm.NewGoError(err))
_ = reject(native.GoError(vm, err))
return
}
_ = resolve(result)
_ = resolve(native.From(vm, result))
})
}()
return vm.ToValue(promise)
return promise, nil
})
_ = exports.Set("hook", func(call goja.FunctionCall) goja.Value {
kind := strings.ToLower(strings.TrimSpace(call.Argument(0).String()))
_ = native.Function(vm, exports, "hook", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
kind := strings.ToLower(strings.TrimSpace(native.String(vm, moejs.Arg(args, 0))))
switch kind {
case "request", "response", string(hookWSConnect), string(hookWSMessage), string(hookWSSend), string(hookWSClose):
if !inst.info.Permissions.AllowHooks {
panic(vm.NewTypeError("server.hook requires the \"hook\" permission (allowHooks)"))
return moejs.Undefined(), native.TypeError(vm, "server.hook requires the \"hook\" permission (allowHooks)")
}
default:
panic(vm.NewTypeError("server.hook kind must be \"request\", \"response\", \"wsConnect\", \"wsMessage\", \"wsSend\" or \"wsClose\""))
return moejs.Undefined(), native.TypeError(vm, "server.hook kind must be \"request\", \"response\", \"wsConnect\", \"wsMessage\", \"wsSend\" or \"wsClose\"")
}
// server.hook(kind, fn) or server.hook(kind, matcher, fn) where
// matcher is "METHOD /path", "/path" or "/path/*". The ws kinds
// only accept a path matcher: every upgrade is a GET request.
fnValue := call.Argument(1)
fnValue := moejs.Arg(args, 1)
var matcher *hookMatcher
if goja.IsString(fnValue) {
parsed, err := parseHookMatcher(fnValue.String())
if fnValue.IsString() {
parsed, err := parseHookMatcher(native.String(vm, fnValue))
if err != nil {
panic(vm.NewTypeError(err.Error()))
return moejs.Undefined(), native.TypeError(vm, "%s", err.Error())
}
if isWSHookKind(kind) && parsed.method != "" {
panic(vm.NewTypeError("server.hook \"" + kind + "\" only accepts a path matcher like \"/api/*\""))
return moejs.Undefined(), native.TypeError(vm, "server.hook %q only accepts a path matcher like /api/*", kind)
}
matcher = parsed
fnValue = call.Argument(2)
fnValue = moejs.Arg(args, 2)
}
fn, ok := goja.AssertFunction(fnValue)
fn := fnValue
ok := engine.IsCallable(fn)
if !ok {
panic(vm.NewTypeError("server.hook requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.hook requires a function handler")
}
m.registerHook(inst.info.Short, hookKind(kind), fn, matcher)
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("injectHTML", func(call goja.FunctionCall) goja.Value {
_ = native.Function(vm, exports, "injectHTML", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
if !inst.info.Permissions.AllowHTMLInject {
panic(vm.NewTypeError("server.injectHTML requires the \"HTML inject\" permission (allowHTMLInject)"))
return moejs.Undefined(), native.TypeError(vm, "server.injectHTML requires the \"HTML inject\" permission (allowHTMLInject)")
}
m.registerInject(inst.info.Short, call.Argument(0).String(), call.Argument(1).String())
return goja.Undefined()
m.registerInject(inst.info.Short, native.String(vm, moejs.Arg(args, 0)), native.String(vm, moejs.Arg(args, 1)))
return moejs.Undefined(), nil
})
_ = exports.Set("cron", func(call goja.FunctionCall) goja.Value {
spec := strings.TrimSpace(call.Argument(0).String())
fn, ok := goja.AssertFunction(call.Argument(1))
_ = native.Function(vm, exports, "cron", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
spec := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
fn := moejs.Arg(args, 1)
ok := engine.IsCallable(fn)
if spec == "" {
panic(vm.NewTypeError("server.cron requires a cron expression"))
return moejs.Undefined(), native.TypeError(vm, "server.cron requires a cron expression")
}
if !ok {
panic(vm.NewTypeError("server.cron requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.cron requires a function handler")
}
if err := m.registerCron(inst.info.Short, spec, fn); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("registerRPC", func(call goja.FunctionCall) goja.Value {
method := strings.TrimSpace(call.Argument(0).String())
fn, ok := goja.AssertFunction(call.Argument(1))
_ = native.Function(vm, exports, "registerRPC", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
method := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
fn := moejs.Arg(args, 1)
ok := engine.IsCallable(fn)
if method == "" {
panic(vm.NewTypeError("server.registerRPC requires an RPC method name"))
return moejs.Undefined(), native.TypeError(vm, "server.registerRPC requires an RPC method name")
}
if !ok {
panic(vm.NewTypeError("server.registerRPC requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.registerRPC requires a function handler")
}
if err := m.registerRPC(inst.info.Short, method, fn); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("registerNotificationChannel", func(call goja.FunctionCall) goja.Value {
id := strings.TrimSpace(call.Argument(0).String())
handler, ok := goja.AssertFunction(call.Argument(2))
_ = native.Function(vm, exports, "registerNotificationChannel", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
id := strings.TrimSpace(native.String(vm, moejs.Arg(args, 0)))
handler := moejs.Arg(args, 2)
ok := engine.IsCallable(handler)
if id == "" {
panic(vm.NewTypeError("server.registerNotificationChannel requires a channel id"))
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a channel id")
}
if !ok {
panic(vm.NewTypeError("server.registerNotificationChannel requires a function handler"))
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a function handler")
}
rawConfiguration, err := exportJSValue(call.Argument(1))
rawConfiguration, err := exportJSValue(vm, moejs.Arg(args, 1))
if err != nil {
panic(vm.NewTypeError("server.registerNotificationChannel requires a managed configuration object"))
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a managed configuration object")
}
configurationJSON, err := json.Marshal(rawConfiguration)
if err != nil {
panic(vm.NewTypeError("server.registerNotificationChannel requires a managed configuration object"))
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a managed configuration object")
}
var configuration models.Configuration
if err := json.Unmarshal(configurationJSON, &configuration); err != nil {
panic(vm.NewTypeError("server.registerNotificationChannel requires a managed configuration object"))
return moejs.Undefined(), native.TypeError(vm, "server.registerNotificationChannel requires a managed configuration object")
}
if err := m.registerNotificationChannel(inst.info.Short, id, configuration, handler, host); err != nil {
panic(vm.NewGoError(err))
return moejs.Undefined(), err
}
return goja.Undefined()
return moejs.Undefined(), nil
})
_ = exports.Set("getConfig", func(call goja.FunctionCall) goja.Value {
_ = native.Function(vm, exports, "getConfig", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
promise, resolve, reject := vm.NewPromise()
go func() {
data, err := GetConfiguration(inst.info.Short)
host.RunOnLoop(func(vm *goja.Runtime) {
host.RunOnLoop(func(vm *moejs.Runtime) {
if err != nil {
_ = reject(vm.NewGoError(err))
_ = reject(native.GoError(vm, err))
return
}
_ = resolve(data)
_ = resolve(native.From(vm, data))
})
}()
return vm.ToValue(promise)
return promise, nil
})
_ = module.Set("exports", exports)
_ = native.Set(vm, module, "exports", exports)
})
}
// callParams converts JS arguments into JSON-RPC params: no params, a single
// value, or a positional array. Values that cannot be exported (for example
// circular objects) are skipped so the call rejects instead of panicking.
func callParams(call goja.FunctionCall) any {
switch n := len(call.Arguments); {
func callParams(vm *moejs.Runtime, args []moejs.Value) any {
switch n := len(args); {
case n <= 1:
return nil
case n == 2:
exported, _ := exportJSValue(call.Argument(1))
exported, _ := exportJSValue(vm, moejs.Arg(args, 1))
return exported
default:
params := make([]any, 0, n-1)
for i := 1; i < n; i++ {
exported, err := exportJSValue(call.Argument(i))
exported, err := exportJSValue(vm, moejs.Arg(args, i))
if err != nil {
return nil
}
@@ -292,11 +303,11 @@ func callParams(call goja.FunctionCall) any {
}
}
func jsRPCError(vm *goja.Runtime, e *rpc.JsonRpcError) *goja.Object {
obj := vm.NewGoError(fmt.Errorf("%s", e.Message))
_ = obj.Set("code", e.Code)
func jsRPCError(vm *moejs.Runtime, e *rpc.JsonRpcError) *engine.Object {
obj := native.Object(vm, native.GoError(vm, fmt.Errorf("%s", e.Message)))
_ = native.Set(vm, obj, "code", e.Code)
if e.Data != nil {
_ = obj.Set("data", e.Data)
_ = native.Set(vm, obj, "data", e.Data)
}
return obj
}
@@ -318,7 +329,7 @@ func normalizeRPCValue(value any) (any, error) {
// called from the plugin's own event loop during script evaluation and takes
// the manager lock itself. The critical section is short and never waits on
// the event loop, so no lock cycle is possible.
func (m *Manager) registerRoute(short, method, path string, handler goja.Callable) (err error) {
func (m *Manager) registerRoute(short, method, path string, handler moejs.Value) (err error) {
key := method + " " + path
m.mu.Lock()
defer m.mu.Unlock()
@@ -334,7 +345,8 @@ func (m *Manager) registerRoute(short, method, path string, handler goja.Callabl
}
slotExists := m.routes[short][key]
inst.mu.Lock()
if !slotExists && inst.handlers[key] != nil {
// A missing map entry yields a zero moejs.Value, not JavaScript undefined.
if _, registered := inst.handlers[key]; !slotExists && registered {
inst.mu.Unlock()
return nil // already registered in this load
}
@@ -373,7 +385,7 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
alive := inst.runtime != nil
limit := inst.info.Permissions.MaxHTTPBodyBytes
inst.mu.RUnlock()
if !alive || handler == nil || host == nil {
if !alive || handler.IsUndefined() || host == nil {
c.Status(http.StatusNotFound)
return
}
@@ -394,11 +406,23 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
chunks: make(chan []byte, 4),
abortCh: make(chan struct{}),
}
queued := host.RunOnLoop(func(vm *goja.Runtime) {
req := routeRequest(vm, c, body)
state.upgrades = make(chan *routeUpgrade, 1)
defer state.seal()
// Gin recycles c when the request returns, including on a timeout while
// this turn is still queued. Only the HTTP pump uses the original.
snapshot := c.Copy()
snapshot.Request = c.Request.Clone(c.Request.Context())
if principal := rpcPrincipalFromGin(c); principal != nil {
copy := *principal
copy.Roles = append([]string(nil), principal.Roles...)
snapshot.Set(principalContextKey, &copy)
}
queued := host.RunOnLoop(func(vm *moejs.Runtime) {
req := routeRequest(vm, snapshot, body)
res := routeResponseObject(vm, state)
bindRouteUpgrade(vm, req, res, state, host, snapshot.Request)
runErr := host.RunJob(vm, "plugin route "+short, func() error {
_, callErr := handler(goja.Undefined(), req, res)
_, callErr := vm.CallFunction(handler, moejs.Undefined(), engine.ObjectValue(req), engine.ObjectValue(res))
return callErr
})
if runErr != nil {
@@ -439,6 +463,36 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
committed := false
for {
select {
case upgrade := <-state.upgrades:
state.mu.Lock()
state.sealed = true
state.upgradeClaimed = true
state.mu.Unlock()
idle.Stop()
if c.Request.Context().Err() != nil || !host.ResourcesOpen() {
upgrade.ws.abort()
upgrade.complete(errors.New("WebSocket upgrade canceled"))
return
}
stopCancel := context.AfterFunc(c.Request.Context(), upgrade.ws.abort)
original := c.Writer
c.Writer = &routeUpgradeWriter{ResponseWriter: original, ws: upgrade.ws}
socket, upgradeErr := api.UpgradeSafeConn(c, func(upgrader *websocket.Upgrader) {
upgrader.HandshakeTimeout = host.Timeout()
})
c.Writer = original
stopCancel()
if upgradeErr != nil {
upgrade.ws.abort()
upgrade.complete(upgradeErr)
return
}
if !upgrade.ws.attach(socket) {
upgrade.complete(errors.New("WebSocket upgrade canceled"))
return
}
upgrade.complete(nil)
return
case chunk := <-state.chunks:
if !committed {
state.mu.Lock()
@@ -464,19 +518,18 @@ func (m *Manager) routeHandler(short string) gin.HandlerFunc {
}
idle.Reset(host.Timeout())
case <-state.done:
state.seal()
if !committed {
writeFinal(0, nil)
}
return
case <-c.Request.Context().Done():
state.aborted.Store(true)
close(state.abortCh)
state.seal()
return
case <-idle.C:
state.seal()
if state.isStreaming() {
// Streaming idle timeout: end the stream and notify JS
state.aborted.Store(true)
close(state.abortCh)
return
}
writeFinal(http.StatusGatewayTimeout, []byte("plugin route handler timed out"))
@@ -619,12 +672,100 @@ type routeResponse struct {
chunks chan []byte
aborted atomic.Bool
abortCh chan struct{}
wrote bool
sealed bool
upgradeClaimed bool
upgrade *routeUpgrade
upgrades chan *routeUpgrade
}
func (state *routeResponse) markStreaming() {
// routeUpgrade keeps the Promise on its original loop. No network goroutine
// touches the resolver or socket object directly.
type routeUpgrade struct {
ws *routeWebSocket
complete func(error)
}
type routeUpgradeWriter struct {
gin.ResponseWriter
ws *routeWebSocket
}
func (w *routeUpgradeWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
conn, rw, err := w.ResponseWriter.Hijack()
if err == nil {
w.ws.captureConn(conn)
}
return conn, rw, err
}
func bindRouteUpgrade(vm *moejs.Runtime, req, res *engine.Object, state *routeResponse, host *jsruntime.Host, request *http.Request) {
_ = native.Function(vm, req, "upgradeWebSocket", func(_ *moejs.Realm, _ moejs.Value, _ []moejs.Value) (moejs.Value, error) {
promise, resolve, reject := vm.NewPromise()
fail := func(message string) (moejs.Value, error) {
_ = reject(native.GoError(vm, errors.New(message)))
return native.From(vm, promise), nil
}
streaming := native.Bool(native.Get(vm, res, "streaming"))
state.mu.Lock()
if state.sealed || state.ended || state.upgrade != nil || state.wrote || state.streaming || streaming {
state.mu.Unlock()
return fail("HTTP response is no longer available for WebSocket upgrade")
}
if request.Method != http.MethodGet || !websocket.IsWebSocketUpgrade(request) {
state.mu.Unlock()
return fail("require a GET WebSocket upgrade request")
}
ws := newRouteWebSocket(host)
u := &routeUpgrade{ws: ws}
u.complete = func(upgradeErr error) {
if !host.RunOnLoop(func(vm *moejs.Runtime) {
if !host.ResourcesOpen() {
ws.abort()
return
}
err := host.RunJob(vm, "WebSocket upgrade", func() error {
if upgradeErr != nil {
return reject(native.GoError(vm, upgradeErr))
}
return resolve(engine.ObjectValue(ws.object(vm)))
})
if err != nil || upgradeErr != nil {
ws.abort()
return
}
// Promise continuations install handlers before a later loop turn
// starts delivering messages (including an immediate peer send).
if !host.RunOnLoop(func(_ *moejs.Runtime) { ws.start() }) {
ws.abort()
}
}) {
ws.abort()
}
}
state.upgrade = u
state.mu.Unlock()
state.upgrades <- u
return native.From(vm, promise), nil
})
}
// seal prevents queued JavaScript from writing or upgrading a returned request.
func (state *routeResponse) seal() {
state.mu.Lock()
state.streaming = true
wasSealed := state.sealed
state.sealed = true
pending := state.upgrade
claimed := state.upgradeClaimed
state.mu.Unlock()
if state.aborted.CompareAndSwap(false, true) {
close(state.abortCh)
}
if pending != nil && !claimed && !wasSealed {
pending.ws.abort()
pending.complete(errors.New("WebSocket upgrade canceled"))
}
}
func (state *routeResponse) isStreaming() bool {
@@ -647,7 +788,7 @@ func (state *routeResponse) pushChunk(p []byte) bool {
func (state *routeResponse) finish(status int, body []byte) {
state.mu.Lock()
defer state.mu.Unlock()
if state.ended {
if state.ended || state.sealed || state.upgrade != nil {
return
}
if status != 0 {
@@ -660,70 +801,107 @@ func (state *routeResponse) finish(status int, body []byte) {
close(state.done)
}
func routeRequest(vm *goja.Runtime, c *gin.Context, body []byte) *goja.Object {
func routeRequest(vm *moejs.Runtime, c *gin.Context, body []byte) *engine.Object {
req := hookRequestObject(vm, c.Request, body)
_ = req.Set("context", routeRequestContext(vm, c))
_ = native.Set(vm, req, "context", routeRequestContext(vm, c))
return req
}
func routeResponseObject(vm *goja.Runtime, state *routeResponse) *goja.Object {
res := vm.NewObject()
_ = res.Set("statusCode", http.StatusOK)
_ = res.Set("statusMessage", "")
_ = res.Set("streaming", false)
_ = res.Set("isAborted", func() bool { return state.aborted.Load() })
_ = res.Set("setHeader", func(name string, value goja.Value) *goja.Object {
state.mu.Lock()
defer state.mu.Unlock()
var values []string
if err := vm.ExportTo(value, &values); err == nil {
state.headers[name] = values
} else {
state.headers.Set(name, value.String())
}
return res
func routeResponseObject(vm *moejs.Runtime, state *routeResponse) *engine.Object {
res := vm.Realm().NewObject()
_ = native.Set(vm, res, "statusCode", http.StatusOK)
_ = native.Set(vm, res, "statusMessage", "")
_ = native.Set(vm, res, "streaming", false)
_ = native.Function(vm, res, "isAborted", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, state.aborted.Load()), nil
})
_ = res.Set("getHeader", func(name string) goja.Value {
_ = native.Function(vm, res, "setHeader", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
name, value := native.String(vm, moejs.Arg(args, 0)), moejs.Arg(args, 1)
var values []string
if err := native.ExportTo(vm, value, &values); err != nil {
values = []string{native.String(vm, value)}
}
state.mu.Lock()
defer state.mu.Unlock()
values := state.headers.Values(name)
if state.upgrade != nil {
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if !state.sealed {
state.headers[http.CanonicalHeaderKey(name)] = values
}
return engine.ObjectValue(res), nil
})
_ = native.Function(vm, res, "getHeader", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.mu.Lock()
defer state.mu.Unlock()
values := state.headers.Values(native.String(vm, moejs.Arg(args, 0)))
if len(values) == 0 {
return goja.Undefined()
return moejs.Undefined(), nil
}
if len(values) == 1 {
return vm.ToValue(values[0])
return native.From(vm, values[0]), nil
}
return vm.ToValue(values)
return native.From(vm, values), nil
})
_ = res.Set("removeHeader", func(name string) {
_ = native.Function(vm, res, "removeHeader", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
state.headers.Del(name)
state.mu.Unlock()
})
_ = res.Set("write", func(call goja.FunctionCall) goja.Value {
data := buffer.Bytes(vm, call.Argument(0))
if value := res.Get("streaming"); value != nil && value.ToBoolean() {
state.markStreaming()
_ = state.pushChunk(data)
return vm.ToValue(true)
defer state.mu.Unlock()
if state.upgrade != nil {
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if !state.sealed {
state.headers.Del(name)
}
return moejs.Undefined(), nil
})
_ = native.Function(vm, res, "write", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := moejs.Arg(args, 0)
var data []byte
if err := native.ExportTo(vm, value, &data); err != nil {
data = []byte(native.String(vm, value))
}
streaming := native.Bool(native.Get(vm, res, "streaming"))
state.mu.Lock()
if state.upgrade != nil {
state.mu.Unlock()
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if state.sealed || state.ended {
state.mu.Unlock()
return native.From(vm, false), nil
}
state.wrote = true
if streaming {
state.streaming = true
state.mu.Unlock()
return native.From(vm, state.pushChunk(data)), nil
}
_, _ = state.body.Write(data)
state.mu.Unlock()
return vm.ToValue(true)
return native.From(vm, true), nil
})
_ = res.Set("end", func(call goja.FunctionCall) goja.Value {
state.mu.Lock()
if len(call.Arguments) > 0 && !goja.IsUndefined(call.Argument(0)) && !goja.IsNull(call.Argument(0)) {
_, _ = state.body.WriteString(call.Argument(0).String())
_ = native.Function(vm, res, "end", func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
var tail string
if len(args) > 0 && !native.IsNil(args[0]) {
tail = native.String(vm, args[0])
}
state.statusCode = int(res.Get("statusCode").ToInteger())
status := int(native.Int(vm, native.Get(vm, res, "statusCode")))
state.mu.Lock()
defer state.mu.Unlock()
if state.upgrade != nil {
return moejs.Undefined(), native.TypeError(vm, "HTTP response is unavailable after WebSocket upgrade")
}
if state.sealed || state.ended {
return engine.ObjectValue(res), nil
}
_, _ = state.body.WriteString(tail)
state.statusCode = status
if !state.ended {
state.ended = true
close(state.done)
}
state.mu.Unlock()
return res
return engine.ObjectValue(res), nil
})
return res
}
+1 -4
View File
@@ -75,9 +75,6 @@ func (s *State) saveLocked() {
file := struct {
Plugins map[string]PluginState `json:"plugins"`
}{Plugins: s.current}
data, err := json.MarshalIndent(file, "", " ")
if err != nil {
return
}
data, _ := json.MarshalIndent(file, "", " ")
_ = os.WriteFile(s.path, data, 0644)
}
+50 -40
View File
@@ -3,8 +3,10 @@ package plugin
import (
"time"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/gorilla/websocket"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/web/connection"
)
@@ -15,10 +17,10 @@ import (
// GET. Like the HTTP kinds, the input is lowercased, so any casing works
// from JS.
const (
hookWSConnect hookKind = "wsconnect"
hookWSMessage hookKind = "wsmessage"
hookWSSend hookKind = "wssend"
hookWSClose hookKind = "wsclose"
hookWSConnect hookKind = "wsconnect"
hookWSMessage hookKind = "wsmessage"
hookWSSend hookKind = "wssend"
hookWSClose hookKind = "wsclose"
)
// wsFrameHookTimeout bounds one frame-level callback. The agent read pumps
@@ -47,21 +49,21 @@ func (m *Manager) OnConnect(info *connection.ConnInfo) (deny bool, reason string
for _, h := range m.wsHooksFor(hookWSConnect, info.Path) {
var denied bool
var why string
queued, timedOut := runHookTurn(h.host, "plugin wsConnect hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin wsConnect hook "+h.short, func(vm *moejs.Runtime) {
_ = h.host.RunJob(vm, "plugin wsConnect hook "+h.short, func() error {
res, err := h.fn(goja.Undefined(), wsHookContext(vm, info))
res, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(wsHookContext(vm, info)))
if err != nil {
return err
}
if goja.IsUndefined(res) || goja.IsNull(res) {
if res.IsUndefined() || res.IsNull() {
return nil
}
obj := res.ToObject(vm)
if v := obj.Get("deny"); v != nil && v.ToBoolean() {
obj := native.Object(vm, res)
if v := native.Get(vm, obj, "deny"); native.Bool(v) {
denied = true
}
if v := obj.Get("reason"); v != nil && !goja.IsUndefined(v) {
why = v.String()
if v := native.Get(vm, obj, "reason"); !v.IsUndefined() {
why = native.String(vm, v)
}
return nil
})
@@ -103,25 +105,25 @@ func (m *Manager) runFrameHooks(kind hookKind, info *connection.ConnInfo, frameT
for _, h := range hooks {
var drop bool
var hookErr error
queued, timedOut := runHookTurnTimeout(h.host, "plugin "+string(kind)+" hook "+h.short, wsFrameHookTimeout, func(vm *goja.Runtime) {
queued, timedOut := runHookTurnTimeout(h.host, "plugin "+string(kind)+" hook "+h.short, wsFrameHookTimeout, func(vm *moejs.Runtime) {
msg := wsFrameObject(vm, info, frameType, next)
hookErr = h.host.RunJob(vm, "plugin "+string(kind)+" hook "+h.short, func() error {
res, err := h.fn(goja.Undefined(), wsHookContext(vm, info), msg)
res, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(wsHookContext(vm, info)), engine.ObjectValue(msg))
if err != nil {
return err
}
if goja.IsUndefined(res) || goja.IsNull(res) {
if res.IsUndefined() || res.IsNull() {
return nil
}
obj := res.ToObject(vm)
if v := obj.Get("drop"); v != nil && v.ToBoolean() {
obj := native.Object(vm, res)
if v := native.Get(vm, obj, "drop"); native.Bool(v) {
drop = true
return nil
}
if v := obj.Get("type"); v != nil && !goja.IsUndefined(v) {
frameType = int(v.ToInteger())
if v := native.Get(vm, obj, "type"); !v.IsUndefined() {
frameType = int(native.Int(vm, v))
}
if v := obj.Get("data"); v != nil && !goja.IsUndefined(v) && !goja.IsNull(v) {
if v := native.Get(vm, obj, "data"); !v.IsUndefined() && !v.IsNull() {
if data := wsFrameBytes(vm, v); data != nil {
next = data
}
@@ -151,9 +153,9 @@ func (m *Manager) runFrameHooks(kind hookKind, info *connection.ConnInfo, frameT
func (m *Manager) OnClose(info *connection.ConnInfo) {
for _, h := range m.wsHooksFor(hookWSClose, info.Path) {
var hookErr error
queued, timedOut := runHookTurn(h.host, "plugin wsClose hook "+h.short, func(vm *goja.Runtime) {
queued, timedOut := runHookTurn(h.host, "plugin wsClose hook "+h.short, func(vm *moejs.Runtime) {
hookErr = h.host.RunJob(vm, "plugin wsClose hook "+h.short, func() error {
_, err := h.fn(goja.Undefined(), wsHookContext(vm, info))
_, err := vm.CallFunction(h.fn, moejs.Undefined(), engine.ObjectValue(wsHookContext(vm, info)))
return err
})
})
@@ -184,41 +186,41 @@ func (m *Manager) wsHooksFor(kind hookKind, path string) []*hookEntry {
}
// wsHookContext builds the immutable per-connection context object.
func wsHookContext(vm *goja.Runtime, info *connection.ConnInfo) *goja.Object {
ctx := vm.NewObject()
_ = ctx.Set("path", info.Path)
_ = ctx.Set("connId", info.ID)
_ = ctx.Set("remoteIp", info.RemoteIP)
_ = ctx.Set("userAgent", info.UserAgent)
func wsHookContext(vm *moejs.Runtime, info *connection.ConnInfo) *engine.Object {
ctx := vm.Realm().NewObject()
_ = native.Set(vm, ctx, "path", info.Path)
_ = native.Set(vm, ctx, "connId", info.ID)
_ = native.Set(vm, ctx, "remoteIp", info.RemoteIP)
_ = native.Set(vm, ctx, "userAgent", info.UserAgent)
if info.ClientUUID != "" {
_ = ctx.Set("clientUuid", info.ClientUUID)
_ = native.Set(vm, ctx, "clientUuid", info.ClientUUID)
}
return ctx
}
// wsFrameObject builds the frame object handed to wsMessage/wsSend hooks.
// Text frames arrive as strings, binary frames as ArrayBuffers.
func wsFrameObject(vm *goja.Runtime, info *connection.ConnInfo, frameType int, data []byte) *goja.Object {
msg := vm.NewObject()
_ = msg.Set("type", frameType)
func wsFrameObject(vm *moejs.Runtime, info *connection.ConnInfo, frameType int, data []byte) *engine.Object {
msg := vm.Realm().NewObject()
_ = native.Set(vm, msg, "type", frameType)
if frameType == websocket.BinaryMessage {
_ = msg.Set("data", vm.NewArrayBuffer(append([]byte(nil), data...)))
_ = native.Set(vm, msg, "data", newFrameBuffer(vm, data))
} else {
_ = msg.Set("data", string(data))
_ = native.Set(vm, msg, "data", string(data))
}
_ = msg.Set("connId", info.ID)
_ = msg.Set("path", info.Path)
_ = native.Set(vm, msg, "connId", info.ID)
_ = native.Set(vm, msg, "path", info.Path)
return msg
}
// wsFrameBytes converts a hook-returned payload (string or ArrayBuffer) back
// to bytes; anything else reports nil and leaves the frame untouched.
func wsFrameBytes(vm *goja.Runtime, v goja.Value) []byte {
if goja.IsString(v) {
return []byte(v.String())
func wsFrameBytes(vm *moejs.Runtime, v moejs.Value) []byte {
if v.IsString() {
return []byte(native.String(vm, v))
}
var data []byte
if err := vm.ExportTo(v, &data); err == nil {
if err := native.ExportTo(vm, v, &data); err == nil {
return data
}
return nil
@@ -231,3 +233,11 @@ func (m *Manager) logWSHookTimeout(h *hookEntry) {
// The Manager is the process-wide WebSocket frame interceptor.
var _ connection.FrameInterceptor = (*Manager)(nil)
func newFrameBuffer(vm *moejs.Runtime, data []byte) moejs.Value {
obj, err := vm.Realm().NewArrayBuffer(append([]byte(nil), data...))
if err != nil {
panic(err)
}
return engine.ObjectValue(obj)
}
+147
View File
@@ -0,0 +1,147 @@
// Package sampling provides deterministic proportional downsampling used by
// history responses. It is independent of transport and storage.
package sampling
import (
"math"
"sort"
)
// Group is one named set of candidate points.
type Group[K comparable] struct {
Key K
Length int
}
type remainder struct {
idx int
frac float64
}
// AllocateTargets splits maxTotal across groups proportionally.
func AllocateTargets[K comparable](groups []Group[K], maxTotal int) map[K]int {
result := make(map[K]int, len(groups))
if maxTotal < 0 {
for _, g := range groups {
result[g.Key] = g.Length
}
return result
}
total := 0
for _, g := range groups {
total += g.Length
}
if total <= maxTotal {
for _, g := range groups {
result[g.Key] = g.Length
}
return result
}
remainders := make([]remainder, 0, len(groups))
sumTargets := 0
for i, g := range groups {
if g.Length <= 0 {
result[g.Key] = 0
continue
}
raw := float64(g.Length) * float64(maxTotal) / float64(total)
t := int(math.Floor(raw))
if t > g.Length {
t = g.Length
}
result[groups[i].Key] = t
sumTargets += t
remainders = append(remainders, remainder{i, raw - float64(t)})
}
if sumTargets < maxTotal {
need := maxTotal - sumTargets
sortRemainders(remainders, false)
for _, r := range remainders {
if need == 0 {
break
}
g := groups[r.idx]
if result[g.Key] < g.Length {
result[g.Key]++
need--
}
}
for need > 0 {
for _, g := range groups {
if need == 0 {
break
}
if result[g.Key] < g.Length {
result[g.Key]++
need--
}
}
if need > 0 {
break
}
}
} else if sumTargets > maxTotal {
over := sumTargets - maxTotal
sortRemainders(remainders, true)
for _, r := range remainders {
if over == 0 {
break
}
g := groups[r.idx]
if result[g.Key] > 0 {
result[g.Key]--
over--
}
}
for over > 0 {
for _, g := range groups {
if over == 0 {
break
}
if result[g.Key] > 0 {
result[g.Key]--
over--
}
}
if over > 0 {
break
}
}
}
return result
}
func sortRemainders(r []remainder, asc bool) {
sort.Slice(r, func(i, j int) bool {
if asc {
return r[i].frac < r[j].frac
}
return r[i].frac > r[j].frac
})
}
// SampleEvenly selects k points including both endpoints.
func SampleEvenly[T any](in []T, k int) []T {
n := len(in)
if k <= 0 || n == 0 {
return []T{}
}
if k >= n {
return in
}
out := make([]T, 0, k)
if k == 1 {
out = append(out, in[n-1])
return out
}
for i := 0; i < k; i++ {
idx := int(math.Round(float64(i) * float64(n-1) / float64(k-1)))
if idx < 0 {
idx = 0
} else if idx >= n {
idx = n - 1
}
out = append(out, in[idx])
}
return out
}
+1 -1
View File
@@ -193,7 +193,7 @@ func registerScheduledWork() {
if err := scheduler.AddFunc("notifier:traffic", "@every 1m", notifier.CheckTraffic); err != nil {
logger.ErrorArgs("server", "Failed to add traffic notification task:", err)
}
if err := scheduler.AddFunc("notifier:expire", "0 0 9 * * *", notifier.CheckExpireScheduledWork); err != nil {
if err := scheduler.AddFunc("notifier:expire", "0 0 9 * * *", notifier.CheckExpire); err != nil {
logger.ErrorArgs("server", "Failed to add expire notification task:", err)
}
}
-21
View File
@@ -1,21 +0,0 @@
package sqlitetune
import (
"testing"
"time"
)
func TestNormalizeAllowsUnlimitedJournalSize(t *testing.T) {
options, err := normalize(Options{
BusyTimeout: 5 * time.Second,
CacheSizeKB: 8,
WALAutoCheckpoint: 1,
JournalSizeLimitBytes: -1,
})
if err != nil {
t.Fatalf("normalize unlimited journal size: %v", err)
}
if options.JournalSizeLimitBytes != -1 {
t.Fatalf("journal size limit = %d, want -1", options.JournalSizeLimitBytes)
}
}
+13
View File
@@ -0,0 +1,13 @@
Copyright (c) 2016 Dmitry Panov
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+196
View File
@@ -0,0 +1,196 @@
// Package buffer implements Node buffers using native JavaScript typed arrays.
package buffer
import (
_ "embed"
"encoding/base64"
"encoding/hex"
moe "github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"strings"
"unicode/utf16"
"unicode/utf8"
)
const ModuleName = "buffer"
//go:embed buffer.js
var source string
func codecName(s string) string {
switch strings.ToLower(s) {
case "", "utf8", "utf-8":
return "utf8"
case "hex", "base64", "base64url", "ascii":
return strings.ToLower(s)
case "latin1", "binary":
return "latin1"
case "ucs2", "ucs-2", "utf16le", "utf-16le":
return "utf16le"
}
return ""
}
func decode(s, enc string) []byte {
switch codecName(enc) {
case "hex":
b, _ := hex.DecodeString(s)
return b
case "base64", "base64url":
s = strings.Map(func(r rune) rune {
if r == '-' {
return '+'
}
if r == '_' {
return '/'
}
if r > 127 {
return -1
}
if strings.ContainsRune("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=", r) {
return r
}
return -1
}, s)
s = strings.TrimRight(s, "=")
b, _ := base64.RawStdEncoding.DecodeString(s)
return b
case "latin1", "ascii":
u := utf16.Encode([]rune(s))
b := make([]byte, len(u))
for i, v := range u {
b[i] = byte(v)
}
return b
case "utf16le":
u := utf16.Encode([]rune(s))
b := make([]byte, len(u)*2)
for i, v := range u {
b[2*i] = byte(v)
b[2*i+1] = byte(v >> 8)
}
return b
default:
return []byte(s)
}
}
func encode(b []byte, enc string) string {
switch codecName(enc) {
case "hex":
return hex.EncodeToString(b)
case "base64":
return base64.StdEncoding.EncodeToString(b)
case "base64url":
return base64.RawURLEncoding.EncodeToString(b)
case "latin1", "ascii":
r := make([]rune, len(b))
for i, v := range b {
if codecName(enc) == "ascii" {
v &= 127
}
r[i] = rune(v)
}
return string(r)
case "utf16le":
u := make([]uint16, len(b)/2)
for i := range u {
u[i] = uint16(b[2*i]) | uint16(b[2*i+1])<<8
}
return string(utf16.Decode(u))
default:
if utf8.Valid(b) {
return string(b)
}
var out strings.Builder
for len(b) > 0 {
r, n := utf8.DecodeRune(b)
out.WriteRune(r)
b = b[n:]
}
return out.String()
}
}
// Bytes returns a live view of the bytes of ArrayBuffer, TypedArray and DataView.
func Bytes(vm *moe.Runtime, v moe.Value) []byte {
if v.IsObject() {
if b, ok := v.AsObject().BufferData(); ok {
return b
}
}
if v.IsString() {
return []byte(native.String(vm, v))
}
var b []byte
if err := native.ExportTo(vm, v, &b); err != nil {
panic(vm.Realm().TypeError("Expected a buffer or byte array"))
}
return b
}
func DecodeBytes(vm *moe.Runtime, arg, enc moe.Value) []byte {
if arg.IsString() {
s := "utf8"
if !native.IsNil(enc) {
s = native.String(vm, enc)
}
return decode(native.String(vm, arg), s)
}
return Bytes(vm, arg)
}
// WrapBytes shares data with a native ArrayBuffer; callers must synchronize access.
func WrapBytes(vm *moe.Runtime, data []byte) *engine.Object {
ab, err := vm.Realm().NewArrayBuffer(data)
if err != nil {
panic(err)
}
m := require.Require(vm, ModuleName)
ctor := native.Get(vm, m.AsObject(), "Buffer")
from := native.Get(vm, ctor.AsObject(), "from")
v, err := vm.CallFunction(from, ctor, engine.ObjectValue(ab))
if err != nil {
panic(err)
}
return v.AsObject()
}
func EncodeBytes(vm *moe.Runtime, data []byte, enc moe.Value) moe.Value {
if !native.IsNil(enc) {
s := native.String(vm, enc)
if codecName(s) != "" {
return native.From(vm, encode(data, s))
}
}
return engine.ObjectValue(WrapBytes(vm, data))
}
func Enable(vm *moe.Runtime) {
m := require.Require(vm, ModuleName)
if err := vm.SetGlobal("Buffer", native.Get(vm, m.AsObject(), "Buffer")); err != nil {
panic(err)
}
}
func Require(vm *moe.Runtime, module *engine.Object) {
factory, err := native.RunScript(vm, "buffer.js", source)
if err != nil {
panic(err)
}
enc := native.Bind(vm, "encode", 2, moe.NativeFunc(func(_ *moe.Realm, _ moe.Value, a []moe.Value) (moe.Value, error) {
return native.From(vm, encode(Bytes(vm, moe.Arg(a, 0)), native.String(vm, moe.Arg(a, 1)))), nil
}))
dec := native.Bind(vm, "decode", 2, moe.NativeFunc(func(_ *moe.Realm, _ moe.Value, a []moe.Value) (moe.Value, error) {
b := decode(native.String(vm, moe.Arg(a, 0)), native.String(vm, moe.Arg(a, 1)))
ab, e := vm.Realm().NewArrayBuffer(b)
if e != nil {
return moe.Undefined(), e
}
return engine.ObjectValue(ab), nil
}))
exports, err := vm.CallFunction(factory, moe.Undefined(), enc, dec)
if err != nil {
panic(err)
}
if err = native.Set(vm, module, "exports", exports); err != nil {
panic(err)
}
}
func init() { require.RegisterCoreModule(ModuleName, Require) }
+45
View File
@@ -0,0 +1,45 @@
(function(encode, decode) {
'use strict';
const encodings = ['utf8','utf-8','hex','base64','base64url','ascii','latin1','binary','ucs2','ucs-2','utf16le','utf-16le'];
function encoding(e) { e = e === undefined ? 'utf8' : String(e).toLowerCase(); if (!encodings.includes(e)) throw new TypeError('Unknown encoding: '+e); return e; }
function size(n) { if (typeof n !== 'number') throw new TypeError('size must be a number'); if (!Number.isInteger(n)||n<0||n>0x7fffffff) throw new RangeError('Invalid buffer size'); return n; }
function bounds(b,o,n) { if (!Number.isInteger(o)||!Number.isInteger(n)||n<0||o<0||o+n>b.length) throw new RangeError('Offset is outside the bounds of the Buffer'); }
class Buffer extends Uint8Array {
constructor(a,b,c) { if(typeof a==='string') super(decode(a,encoding(b))); else super(a,b,c); }
static from(a,b,c) { if(typeof a==='number'||a==null) throw new TypeError('Invalid Buffer input'); if(typeof a==='string')return new Buffer(decode(a,encoding(b))); if(a instanceof ArrayBuffer) return new Buffer(a,b,c); if(a.type==='Buffer'&&Array.isArray(a.data)) a=a.data; if(a.valueOf){const v=a.valueOf();if(v!==a)return Buffer.from(v,b,c);}if(typeof a.length!=='number'&&!ArrayBuffer.isView(a)&&!a[Symbol.iterator])throw new TypeError('Invalid Buffer input'); return new Buffer(a); }
static alloc(n,fill,enc) {const b=new Buffer(size(n));if(fill!==undefined)b.fill(fill,0,b.length,enc);return b;}
static allocUnsafe(n) {return new Buffer(size(n));}
static allocUnsafeSlow(n) {return Buffer.allocUnsafe(n);}
static isBuffer(b) {return b instanceof Buffer;}
static isEncoding(e) {return typeof e==='string'&&encodings.includes(e.toLowerCase());}
static byteLength(s,e) {if(typeof s!=='string'){if(s instanceof ArrayBuffer||ArrayBuffer.isView(s))return s.byteLength;throw new TypeError('Expected string or buffer');}e=encoding(e);return e==='hex'?Math.floor(s.length/2):decode(s,e).byteLength;}
static concat(list,total) {if(!Array.isArray(list))throw new TypeError('Expected array');if(total===undefined)total=list.reduce((n,b)=>n+b.length,0);const b=Buffer.alloc(total);let o=0;for(const p of list){const n=Math.min(p.length,total-o);if(n<=0)break;b.set(p.subarray(0,n),o);o+=n;}return b;}
static compare(a,b) {for(let i=0;i<Math.min(a.length,b.length);i++)if(a[i]!==b[i])return a[i]<b[i]?-1:1;return a.length===b.length?0:(a.length<b.length?-1:1);}
toString(e,start,end) {e=encoding(e);start=Math.max(0,start===undefined?0:Math.trunc(start));end=Math.min(this.length,end===undefined?this.length:Math.trunc(end));return encode(this.subarray(start,Math.max(start,end)),e);}
toLocaleString() {return this.toString();}
toJSON() {return {type:'Buffer',data:Array.from(this)};}
slice(s,e) {return this.subarray(s,e);}
equals(b) {return Buffer.compare(this,b)===0;}
compare(b,ts,te,ss,se) {return Buffer.compare(this.subarray(ss,se),b.subarray(ts,te));}
copy(target,ts,ss,se) {ts=ts===undefined?0:ts;ss=ss===undefined?0:ss;se=se===undefined?this.length:se;if(ts<0||ss<0||se<0)throw new RangeError('Invalid offset');const n=Math.max(0,Math.min(se-ss,this.length-ss,target.length-ts));if(n>0)target.set(this.subarray(ss,ss+n),ts);return n;}
fill(v,start,end,e) {if(typeof start==='string'){e=start;start=0;}if(typeof end==='string'){e=end;end=this.length;}start=start===undefined?0:start;end=end===undefined?this.length:end;bounds(this,start,Math.max(0,end-start));if(typeof v==='number') {super.fill(v,start,end);return this;}const p=typeof v==='string'?Buffer.from(v,e):v;if(!p||!p.length){if(v===''){super.fill(0,start,end);return this;}throw new TypeError('Invalid fill value');}for(let i=start;i<end;i++)this[i]=p[(i-start)%p.length];return this;}
write(s,o,n,e) {if(typeof o==='string'){e=o;o=0;n=this.length;}o=o===undefined?0:o;if(typeof n==='string'){e=n;n=undefined;}n=n===undefined?this.length-o:n;bounds(this,o,n);let b=Buffer.from(s,e);let count=Math.min(n,b.length);if((e===undefined||/^utf-?8$/i.test(e))&&count<b.length){while(count>0&&(b[count]&192)===128)count--;}this.set(b.subarray(0,count),o);return count;}
indexOf(v,o,e) {return this._find(v,o,e,false);}
lastIndexOf(v,o,e) {return this._find(v,o,e,true);}
includes(v,o,e) {return this.indexOf(v,o,e)!==-1;}
_find(v,o,e,reverse) {if(typeof o==='string'){e=o;o=undefined;}if(typeof v==='number')v=Buffer.from([v]);else if(typeof v==='string')v=Buffer.from(v,e);o=o===undefined?(reverse?this.length:0):Math.trunc(o);if(o<0)o=Math.max(0,this.length+o);if(!v.length)return Math.min(o,this.length);if(reverse){for(let i=Math.min(o,this.length-v.length);i>=0;i--){let j=0;while(j<v.length&&this[i+j]===v[j])j++;if(j===v.length)return i;}}else{for(let i=o;i<=this.length-v.length;i++){let j=0;while(j<v.length&&this[i+j]===v[j])j++;if(j===v.length)return i;}}return -1;}
}
for(const width of [8,16,32,64]) for(const little of [false,true]) for(const signed of [false,true]) {
if(width===8&&little)continue;
const big=width===64, bytes=width/8, suffix=width+(width===8?'':little?'LE':'BE');
const kind=big?(signed?'BigInt':'BigUInt'):(signed?'Int':'UInt');
const dvkind=big?(signed?'BigInt64':'BigUint64'):(signed?'Int':'Uint')+width;
Buffer.prototype['read'+kind+suffix]=function(o){o=o===undefined?0:o;bounds(this,o,bytes);return new DataView(this.buffer,this.byteOffset,this.byteLength)['get'+dvkind](o,little);};
Buffer.prototype['write'+kind+suffix]=function(v,o){o=o===undefined?0:o;bounds(this,o,bytes);if(big&&(typeof v!=='bigint'||v<(signed?-(1n<<63n):0n)||v>(signed?(1n<<63n)-1n:(1n<<64n)-1n)))throw new RangeError('Value out of range');if(!big&&(!Number.isFinite(v)||v<(signed?-(2**(width-1)):0)||v>(signed?2**(width-1)-1:2**width-1)))throw new RangeError('Value out of range');new DataView(this.buffer,this.byteOffset,this.byteLength)['set'+dvkind](o,v,little);return o+bytes;};
if(!signed){Buffer.prototype['read'+(big?'BigUint':'Uint')+suffix]=Buffer.prototype['read'+kind+suffix];Buffer.prototype['write'+(big?'BigUint':'Uint')+suffix]=Buffer.prototype['write'+kind+suffix];}
}
for(const pair of [['Float',32],['Double',64]])for(const little of [false,true]){const name=pair[0]+(little?'LE':'BE'),bytes=pair[1]/8;Buffer.prototype['read'+name]=function(o){o=o===undefined?0:o;bounds(this,o,bytes);return new DataView(this.buffer,this.byteOffset,this.byteLength)['getFloat'+pair[1]](o,little);};Buffer.prototype['write'+name]=function(v,o){o=o===undefined?0:o;bounds(this,o,bytes);new DataView(this.buffer,this.byteOffset,this.byteLength)['setFloat'+pair[1]](o,v,little);return o+bytes;};}
for(const little of [false,true])for(const signed of [false,true]){const name=(signed?'Int':'UInt')+(little?'LE':'BE');Buffer.prototype['read'+name]=function(o,n){if(!Number.isInteger(n)||n<1||n>6)throw new RangeError('Invalid byteLength');bounds(this,o,n);let v=0;for(let i=0;i<n;i++)v=v*256+this[o+(little?n-i-1:i)];if(signed&&v>=2**(n*8-1))v-=2**(n*8);return v;};Buffer.prototype['write'+name]=function(v,o,n){if(!Number.isInteger(n)||n<1||n>6)throw new RangeError('Invalid byteLength');bounds(this,o,n);const min=signed?-(2**(n*8-1)):0,max=signed?2**(n*8-1)-1:2**(n*8)-1;if(!Number.isFinite(v)||v<min||v>max)throw new RangeError('Value out of range');if(v<0)v+=2**(n*8);for(let i=0;i<n;i++){this[o+(little?i:n-i-1)]=v%256;v=Math.floor(v/256);}return o+n;};if(!signed){Buffer.prototype['readUint'+(little?'LE':'BE')]=Buffer.prototype['read'+name];Buffer.prototype['writeUint'+(little?'LE':'BE')]=Buffer.prototype['write'+name];}}
for(const n of [2,4,8])Buffer.prototype['swap'+n*8]=function(){if(this.length%n)throw new RangeError('Buffer size must be a multiple of '+n);for(let i=0;i<this.length;i+=n)for(let j=0;j<n/2;j++){const x=this[i+j];this[i+j]=this[i+n-j-1];this[i+n-j-1]=x;}return this;};
return {Buffer,SlowBuffer:Buffer.alloc,INSPECT_MAX_BYTES:50,kMaxLength:0x7fffffff,constants:{MAX_LENGTH:0x7fffffff,MAX_STRING_LENGTH:0x1fffffe8}};
})
+83
View File
@@ -0,0 +1,83 @@
package buffer
import (
moe "github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"testing"
)
func TestNativeBuffer(t *testing.T) {
vm := moe.NewRuntime(moe.Options{})
require.NewRegistry().Enable(vm)
Enable(vm)
for _, s := range []string{
`Object.getPrototypeOf(Buffer.prototype) === Uint8Array.prototype`,
`Object.isFrozen(Object.prototype)&&Object.isFrozen(Uint8Array.prototype)`,
`(function(){const b=Buffer.alloc(1);return b.copy(Buffer.alloc(0),999)===0&&Buffer.byteLength("zz","hex")===1})()`,
`(function(){for(const f of [()=>Buffer.from({}),()=>Buffer.alloc(4).readUIntLE(0,1.5),()=>Buffer.alloc(8).writeBigUInt64LE(-1n),()=>Buffer.alloc(8).writeUIntLE(NaN,0,2)]){try{f();return false;}catch(e){}}return true;})()`,
`(function(){const a=new Uint8Array([1,2,3]);const b=Buffer.from(a);b[0]=9;const ab=a.buffer;const v=Buffer.from(ab,1,1);v[0]=7;return a[0]===1&&a[1]===7&&v.length===1&&b instanceof Uint8Array})()`,
`Buffer.from("hello").toString() === "hello"`,
`Buffer.from("hello").toString("hex") === "68656c6c6f"`,
`Buffer.from("68656c6c6f", "hex").toString() === "hello"`,
`Buffer.from("通知").toString() === "通知"`,
`Buffer.from("aGVsbG8=", "base64").toString() === "hello"`,
`Buffer.from("hello") instanceof Buffer`,
`Buffer.alloc(4, "ab").toString() === "abab"`,
`Buffer.concat([Buffer.from("a"),Buffer.from("b")]).toString() === "ab"`,
`(function(){const b=Buffer.alloc(16);b.writeUInt32LE(0x12345678,1);b.writeInt16BE(-123,5);b.writeDoubleBE(1.25,8);return b.readUInt32LE(1)===0x12345678&&b.readInt16BE(5)===-123&&b.readDoubleBE(8)===1.25})()`,
`(function(){const b=Buffer.from("abcd");const v=b.slice(1,3);v[0]=120;return b.toString()==="axcd"&&Buffer.isBuffer(v)})()`,
`(function(){const b=Buffer.alloc(8);b.writeBigUInt64LE(123n);return b.readBigUInt64LE()===123n})()`,
} {
v, err := native.Run(vm, s)
if err != nil {
t.Fatalf("%s: %v", s, err)
}
if !native.Bool(v) {
t.Errorf("false: %s", s)
}
}
data := []byte("wrapped")
b := WrapBytes(vm, data)
if err := vm.SetGlobal("wrapped", engine.ObjectValue(b)); err != nil {
t.Fatal(err)
}
if _, err := native.Run(vm, "wrapped[0]=65"); err != nil {
t.Fatal(err)
}
if data[0] != 65 {
t.Fatal("WrapBytes did not share backing storage")
}
view, err := native.Run(vm, "wrapped.subarray(1,3)")
if err != nil {
t.Fatal(err)
}
raw := Bytes(vm, view)
raw[0] = 66
if data[1] != 66 || len(raw) != 2 {
t.Fatal("Bytes did not return a live bounded subarray")
}
for _, source := range []string{"new DataView(wrapped.buffer,1,2)", "new Uint16Array(wrapped.buffer,0,2)", "wrapped.buffer"} {
v, err := native.Run(vm, source)
if err != nil {
t.Fatal(err)
}
b := Bytes(vm, v)
if len(b) == 0 {
t.Fatalf("empty bytes: %s", source)
}
b[0] = 77
offset := 0
if source == "new DataView(wrapped.buffer,1,2)" {
offset = 1
if len(b) != 2 {
t.Fatal("unbounded DataView")
}
}
if data[offset] != 77 {
t.Fatalf("bytes not live: %s", source)
}
}
}
+229 -218
View File
@@ -6,6 +6,15 @@ import (
_ "embed"
"errors"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/fs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/writequeue"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"io"
"os"
"os/exec"
@@ -13,14 +22,6 @@ import (
"strings"
"sync"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/fs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/writequeue"
)
type childCommandOptions struct {
@@ -31,7 +32,6 @@ type childCommandOptions struct {
encoding string
maxBuffer int
}
type childOutputLimitError struct {
stream string
}
@@ -65,9 +65,12 @@ func (b *childOutputBuffer) Write(data []byte) (int, error) {
b.once.Do(b.cancel)
return len(data), nil
}
func (b *childOutputBuffer) Len() int { return b.buffer.Len() }
func (b *childOutputBuffer) Bytes() []byte { return b.buffer.Bytes() }
func (b *childOutputBuffer) Len() int {
return b.buffer.Len()
}
func (b *childOutputBuffer) Bytes() []byte {
return b.buffer.Bytes()
}
type Module struct {
runtime *bridge.Runtime
@@ -79,134 +82,131 @@ type Module struct {
func New(runtime *bridge.Runtime, filesystem *fs.Module, allowExec bool, maxOutput int) *Module {
return &Module{runtime: runtime, filesystem: filesystem, allowExec: allowExec, maxOutput: maxOutput}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
if !m.allowExec {
panic(vm.NewGoError(fmt.Errorf("child_process requires AllowExec")))
panic(native.GoError(vm, fmt.Errorf("child_process requires AllowExec")))
}
exports := vm.NewObject()
_ = exports.Set("spawn", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, false)
return m.spawnChild(vm, command, arguments, options)
})
_ = exports.Set("exec", func(call goja.FunctionCall) goja.Value {
command := call.Argument(0).String()
optionsValue, callbackValue := call.Argument(1), call.Argument(2)
if _, ok := goja.AssertFunction(optionsValue); ok {
callbackValue, optionsValue = optionsValue, goja.Undefined()
exports := vm.Realm().NewObject()
_ = native.Set(vm, exports, "spawn", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, false)
return native.From(vm, m.spawnChild(vm, command, arguments, options)), nil
}))
_ = native.Set(vm, exports, "exec", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command := native.String(vm, moejs.Arg(args, 0))
optionsValue, callbackValue := moejs.Arg(args, 1), moejs.Arg(args, 2)
if _, ok := checkedFunction(optionsValue); ok {
callbackValue, optionsValue = optionsValue, moejs.Undefined()
}
callback, ok := goja.AssertFunction(callbackValue)
callback, ok := checkedFunction(callbackValue)
if !ok {
panic(vm.NewTypeError("callback must be a function"))
panic(native.TypeError(vm, "callback must be a function"))
}
options := m.childOptions(vm, optionsValue)
options.shell = true
return m.execChild(vm, command, nil, options, callback)
})
_ = exports.Set("execFile", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, true)
return native.From(vm, m.execChild(vm, command, nil, options, callback)), nil
}))
_ = native.Set(vm, exports, "execFile", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, true)
callbackIndex := 1
if _, ok := call.Argument(1).Export().([]any); ok {
if _, ok := native.Export(vm, moejs.Arg(args, 1)).([]any); ok {
callbackIndex = 2
}
callbackValue := call.Argument(callbackIndex)
if _, ok := goja.AssertFunction(callbackValue); !ok {
callbackValue = call.Argument(callbackIndex + 1)
callbackValue := moejs.Arg(args, callbackIndex)
if _, ok := checkedFunction(callbackValue); !ok {
callbackValue = moejs.Arg(args, callbackIndex+1)
}
callback, ok := goja.AssertFunction(callbackValue)
callback, ok := checkedFunction(callbackValue)
if !ok {
panic(vm.NewTypeError("callback must be a function"))
panic(native.TypeError(vm, "callback must be a function"))
}
return m.execChild(vm, command, arguments, options, callback)
})
_ = exports.Set("spawnSync", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, false)
return m.spawnChildSync(vm, command, arguments, options)
})
_ = exports.Set("execFileSync", func(call goja.FunctionCall) goja.Value {
command, arguments, options := m.childCommandArguments(vm, call, false)
return native.From(vm, m.execChild(vm, command, arguments, options, callback)), nil
}))
_ = native.Set(vm, exports, "spawnSync", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, false)
return native.From(vm, m.spawnChildSync(vm, command, arguments, options)), nil
}))
_ = native.Set(vm, exports, "execFileSync", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
command, arguments, options := m.childCommandArguments(vm, args, false)
result := m.runChildSync(command, arguments, options)
if result.err != nil {
panic(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
panic(engine.ObjectValue(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr)))
}
return encodeChildOutput(vm, result.stdout, options.encoding)
})
_ = exports.Set("execSync", func(call goja.FunctionCall) goja.Value {
options := m.childOptions(vm, call.Argument(1))
return native.From(vm, encodeChildOutput(vm, result.stdout, options.encoding)), nil
}))
_ = native.Set(vm, exports, "execSync", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
options := m.childOptions(vm, moejs.Arg(args, 1))
options.shell = true
result := m.runChildSync(call.Argument(0).String(), nil, options)
result := m.runChildSync(native.String(vm, moejs.Arg(args, 0)), nil, options)
if result.err != nil {
panic(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
panic(engine.ObjectValue(childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr)))
}
return encodeChildOutput(vm, result.stdout, options.encoding)
})
_ = exports.Set("fork", func() {
panic(vm.NewGoError(fmt.Errorf("child_process.fork is unavailable because this runtime is not a Node executable")))
})
_ = module.Set("exports", exports)
return native.From(vm, encodeChildOutput(vm, result.stdout, options.encoding)), nil
}))
_ = native.Set(vm, exports, "fork", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("child_process.fork is unavailable because this runtime is not a Node executable")))
}))
_ = native.Set(vm, module, "exports", exports)
}
func (m *Module) childCommandArguments(vm *goja.Runtime, call goja.FunctionCall, callbackExpected bool) (string, []string, childCommandOptions) {
command := call.Argument(0).String()
func (m *Module) childCommandArguments(vm *moejs.Runtime, args []moejs.Value, callbackExpected bool) (string, []string, childCommandOptions) {
command := native.String(vm, moejs.Arg(args, 0))
arguments := []string{}
optionsIndex := 1
if value := call.Argument(1); !goja.IsUndefined(value) && !goja.IsNull(value) {
if value := moejs.Arg(args, 1); !value.IsUndefined() && !value.IsNull() {
var exported []string
if err := vm.ExportTo(value, &exported); err == nil {
if err := native.ExportTo(vm, value, &exported); err == nil {
arguments = exported
optionsIndex = 2
}
}
options := m.childOptions(vm, call.Argument(optionsIndex))
options := m.childOptions(vm, moejs.Arg(args, optionsIndex))
_ = callbackExpected
return command, arguments, options
}
func (m *Module) childOptions(vm *goja.Runtime, value goja.Value) childCommandOptions {
func (m *Module) childOptions(vm *moejs.Runtime, value moejs.Value) childCommandOptions {
options := childCommandOptions{cwd: m.filesystem.Cwd(), env: os.Environ(), maxBuffer: m.maxOutput}
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return options
}
object := value.ToObject(vm)
if cwd := object.Get("cwd"); cwd != nil && !goja.IsUndefined(cwd) && !goja.IsNull(cwd) {
resolved, err := m.filesystem.Resolve(cwd.String(), false)
object := native.Object(vm, value)
if cwd := native.Get(vm, object, "cwd"); !native.IsNil(cwd) && !cwd.IsUndefined() && !cwd.IsNull() {
resolved, err := m.filesystem.Resolve(native.String(vm, cwd), false)
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
options.cwd = resolved
}
if environment := object.Get("env"); environment != nil && !goja.IsUndefined(environment) && !goja.IsNull(environment) {
if environment := native.Get(vm, object, "env"); !native.IsNil(environment) && !environment.IsUndefined() && !environment.IsNull() {
var values map[string]string
if err := vm.ExportTo(environment, &values); err != nil {
panic(vm.NewTypeError("env must be an object"))
if err := native.ExportTo(vm, environment, &values); err != nil {
panic(native.TypeError(vm, "env must be an object"))
}
options.env = make([]string, 0, len(values))
for name, value := range values {
options.env = append(options.env, name+"="+value)
}
}
if shell := object.Get("shell"); shell != nil {
options.shell = shell.ToBoolean()
if shell := native.Get(vm, object, "shell"); !native.IsNil(shell) {
options.shell = native.Bool(shell)
}
if timeoutValue := object.Get("timeout"); timeoutValue != nil && timeoutValue.ToInteger() > 0 {
timeout := timeoutValue.ToInteger()
if timeoutValue := native.Get(vm, object, "timeout"); !native.IsNil(timeoutValue) && native.Int(vm, timeoutValue) > 0 {
timeout := native.Int(vm, timeoutValue)
options.timeout = time.Duration(timeout) * time.Millisecond
}
if encoding := object.Get("encoding"); encoding != nil && !goja.IsUndefined(encoding) && !goja.IsNull(encoding) {
options.encoding = encoding.String()
if encoding := native.Get(vm, object, "encoding"); !native.IsNil(encoding) && !encoding.IsUndefined() && !encoding.IsNull() {
options.encoding = native.String(vm, encoding)
}
if maxBuffer := object.Get("maxBuffer"); maxBuffer != nil {
requested := maxBuffer.ToInteger()
if maxBuffer := native.Get(vm, object, "maxBuffer"); !native.IsNil(maxBuffer) {
requested := native.Int(vm, maxBuffer)
if requested > 0 && requested < int64(options.maxBuffer) {
options.maxBuffer = int(requested)
}
}
return options
}
func childExecCommand(command string, arguments []string, options childCommandOptions) (*exec.Cmd, context.CancelFunc) {
ctx := context.Background()
cancel := func() {}
cancel := func() {
}
if options.timeout > 0 {
ctx, cancel = context.WithTimeout(ctx, options.timeout)
}
@@ -222,15 +222,13 @@ func childExecCommand(command string, arguments []string, options childCommandOp
}
return exec.CommandContext(ctx, command, arguments...), cancel
}
func (m *Module) childOutputLimit(options childCommandOptions) int {
if options.maxBuffer > 0 && options.maxBuffer < m.maxOutput {
return options.maxBuffer
}
return m.maxOutput
}
func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string, options childCommandOptions) *goja.Object {
func (m *Module) spawnChild(vm *moejs.Runtime, command string, arguments []string, options childCommandOptions) *engine.Object {
child := events.NewEmitter(vm)
cmd, cancel := childExecCommand(command, arguments, options)
cmd.Dir = options.cwd
@@ -239,40 +237,45 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
stdoutReader, stdoutErr := cmd.StdoutPipe()
stderrReader, stderrErr := cmd.StderrPipe()
if err := errors.Join(stdinErr, stdoutErr, stderrErr); err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
stdinStream, stdout, stderr := m.childStreams(vm, stdin)
_ = child.Set("stdout", stdout)
_ = child.Set("stderr", stderr)
_ = child.Set("stdin", stdinStream)
_ = child.Set("stdio", []any{stdinStream, stdout, stderr})
_ = child.Set("pid", goja.Undefined())
_ = child.Set("exitCode", goja.Null())
_ = child.Set("signalCode", goja.Null())
_ = child.Set("killed", false)
_ = child.Set("connected", false)
_ = native.Set(vm, child, "stdout", stdout)
_ = native.Set(vm, child, "stderr", stderr)
_ = native.Set(vm, child, "stdin", stdinStream)
_ = native.Set(vm, child, "stdio", []any{stdinStream, stdout, stderr})
_ = native.Set(vm, child, "pid", moejs.Undefined())
_ = native.Set(vm, child, "exitCode", moejs.Null())
_ = native.Set(vm, child, "signalCode", moejs.Null())
_ = native.Set(vm, child, "killed", false)
_ = native.Set(vm, child, "connected", false)
if err := cmd.Start(); err != nil {
cancel()
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.spawn", func() error {
return events.Emit(vm, child, "error", vm.NewGoError(err))
return events.Emit(vm, child, "error", native.GoError(vm, err))
})
}, 0)
return child
}
_ = child.Set("pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() { _ = cmd.Process.Kill(); cancel() })
_ = native.Set(vm, child, "pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() {
_ = cmd.Process.Kill()
cancel()
})
if resourceID == 0 {
_ = stdin.Close()
_ = stdoutReader.Close()
_ = stderrReader.Close()
go func() { _ = cmd.Wait() }()
go func() {
_ = cmd.Wait()
}()
return child
}
_ = child.Set("kill", func(signal string) bool {
_ = native.Set(vm, child, "kill", native.Bind(vm, "", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
signal := native.String(vm, moejs.Arg(args, 0))
if cmd.Process == nil {
return false
return native.From(vm, false), nil
}
var err error
if signal == "SIGKILL" {
@@ -281,24 +284,26 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
err = cmd.Process.Signal(os.Interrupt)
}
if err == nil {
_ = child.Set("killed", true)
_ = native.Set(vm, child, "killed", true)
}
return err == nil
})
_ = child.Set("ref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("child_process.ChildProcess.ref is not supported by jsruntime; the event loop is host-driven")))
})
_ = child.Set("unref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("child_process.ChildProcess.unref is not supported by jsruntime; the event loop is host-driven")))
})
_ = child.Set("disconnect", func() { _ = events.Emit(vm, child, "disconnect") })
_ = child.Set("send", func(call goja.FunctionCall) goja.Value {
if callback, ok := goja.AssertFunction(call.Argument(len(call.Arguments) - 1)); ok {
_, _ = callback(goja.Undefined(), vm.NewGoError(fmt.Errorf("IPC is not enabled")))
return native.From(vm, err == nil), nil
}))
_ = native.Set(vm, child, "ref", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("child_process.ChildProcess.ref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, child, "unref", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("child_process.ChildProcess.unref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, child, "disconnect", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
_ = events.Emit(vm, child, "disconnect")
return moejs.Undefined(), nil
}))
_ = native.Set(vm, child, "send", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if callback, ok := checkedFunction(moejs.Arg(args, len(args)-1)); ok {
_, _ = native.Call(vm, callback, moejs.Undefined(), native.GoError(vm, fmt.Errorf("IPC is not enabled")))
}
return vm.ToValue(false)
})
return native.From(vm, false), nil
}))
m.pipeChildOutput(vm, stdoutReader, stdout, options.encoding)
m.pipeChildOutput(vm, stderrReader, stderr, options.encoding)
go func() {
@@ -313,13 +318,13 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
exitCode = -1
}
}
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process", func() error {
_ = child.Set("exitCode", exitCode)
if emitErr := events.Emit(vm, child, "exit", exitCode, goja.Null()); emitErr != nil {
_ = native.Set(vm, child, "exitCode", exitCode)
if emitErr := events.Emit(vm, child, "exit", exitCode, moejs.Null()); emitErr != nil {
return emitErr
}
return events.Emit(vm, child, "close", exitCode, goja.Null())
return events.Emit(vm, child, "close", exitCode, moejs.Null())
})
})
}()
@@ -329,96 +334,99 @@ func (m *Module) spawnChild(vm *goja.Runtime, command string, arguments []string
//go:embed stdio.js
var childStdioSource string
// childStreams creates the Writable stdin and Readable stdout/stderr streams
// for one spawned child process. The Go side owns the underlying pipes and
// serializes writes through a per-child queue.
func (m *Module) childStreams(vm *goja.Runtime, stdin io.WriteCloser) (*goja.Object, *goja.Object, *goja.Object) {
factoryValue, err := vm.RunString(childStdioSource)
func (m *Module) childStreams(vm *moejs.Runtime, stdin io.WriteCloser) (*engine.Object, *engine.Object, *engine.Object) {
factoryValue, err := native.Run(vm, childStdioSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load child_process stdio: %w", err)))
panic(native.GoError(vm, fmt.Errorf("load child_process stdio: %w", err)))
}
factory, _ := goja.AssertFunction(factoryValue)
hooks := vm.NewObject()
factory, _ := checkedFunction(factoryValue)
hooks := vm.Realm().NewObject()
var writes writequeue.Queue
_ = hooks.Set("stdinWrite", func(chunk goja.Value, callback goja.Callable) {
_ = native.Set(vm, hooks, "stdinWrite", native.Bind(vm, "", 2, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
chunk := moejs.Arg(args, 0)
callback := moejs.Arg(args, 1)
data := append([]byte(nil), buffer.Bytes(vm, chunk)...)
writes.Submit(func() {
_, err := stdin.Write(data)
if callback == nil {
if native.IsNil(callback) {
return
}
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.stdin.write", func() error {
if err != nil {
_, callbackErr := callback(goja.Undefined(), vm.NewGoError(err))
_, callbackErr := native.Call(vm, callback, moejs.Undefined(), native.GoError(vm, err))
return callbackErr
}
_, callbackErr := callback(goja.Undefined())
_, callbackErr := native.Call(vm, callback, moejs.Undefined())
return callbackErr
})
})
})
})
_ = hooks.Set("stdinEnd", func(callback goja.Callable) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "stdinEnd", native.Bind(vm, "", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
callback := moejs.Arg(args, 0)
writes.Submit(func() {
err := stdin.Close()
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.stdin.end", func() error {
if callback != nil {
if !native.IsNil(callback) {
if err != nil {
_, callbackErr := callback(goja.Undefined(), vm.NewGoError(err))
_, callbackErr := native.Call(vm, callback, moejs.Undefined(), native.GoError(vm, err))
return callbackErr
}
_, callbackErr := callback(goja.Undefined())
_, callbackErr := native.Call(vm, callback, moejs.Undefined())
return callbackErr
}
return nil
})
})
})
})
_ = hooks.Set("stdinDestroy", func() {
go func() { _ = stdin.Close() }()
})
streams, err := factory(goja.Undefined(), require.Require(vm, "stream"), hooks)
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "stdinDestroy", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
go func() {
_ = stdin.Close()
}()
return moejs.Undefined(), nil
}))
streams, err := native.Call(vm, factory, moejs.Undefined(), require.Require(vm, "stream"), engine.ObjectValue(hooks))
if err != nil {
panic(err)
}
streamsObject := streams.ToObject(vm)
createStdin, _ := goja.AssertFunction(streamsObject.Get("createStdin"))
createOutput, _ := goja.AssertFunction(streamsObject.Get("createOutput"))
if createStdin == nil || createOutput == nil {
panic(vm.NewGoError(fmt.Errorf("child_process stdio factory is incomplete")))
streamsObject := native.Object(vm, streams)
createStdin, _ := checkedFunction(native.Get(vm, streamsObject, "createStdin"))
createOutput, _ := checkedFunction(native.Get(vm, streamsObject, "createOutput"))
if native.IsNil(createStdin) || native.IsNil(createOutput) {
panic(native.GoError(vm, fmt.Errorf("child_process stdio factory is incomplete")))
}
stdinValue, err := createStdin(goja.Undefined())
stdinValue, err := native.Call(vm, createStdin, moejs.Undefined())
if err != nil {
panic(err)
}
stdoutValue, err := createOutput(goja.Undefined())
stdoutValue, err := native.Call(vm, createOutput, moejs.Undefined())
if err != nil {
panic(err)
}
stderrValue, err := createOutput(goja.Undefined())
stderrValue, err := native.Call(vm, createOutput, moejs.Undefined())
if err != nil {
panic(err)
}
return stdinValue.ToObject(vm), stdoutValue.ToObject(vm), stderrValue.ToObject(vm)
return native.Object(vm, stdinValue), native.Object(vm, stdoutValue), native.Object(vm, stderrValue)
}
func childCallback(call goja.FunctionCall) goja.Callable {
for index := len(call.Arguments) - 1; index >= 0; index-- {
if callback, ok := goja.AssertFunction(call.Arguments[index]); ok {
func childCallback(args []moejs.Value) moejs.Value {
for index := len(args) - 1; index >= 0; index-- {
if callback, ok := checkedFunction(args[index]); ok {
return callback
}
}
return nil
return moejs.Undefined()
}
func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goja.Object, encoding string) {
push, _ := goja.AssertFunction(stream.Get("push"))
setEncoding, _ := goja.AssertFunction(stream.Get("setEncoding"))
if encoding != "" && setEncoding != nil {
_, _ = setEncoding(stream, vm.ToValue(encoding))
func (m *Module) pipeChildOutput(vm *moejs.Runtime, reader io.Reader, stream *engine.Object, encoding string) {
push, _ := checkedFunction(native.Get(vm, stream, "push"))
setEncoding, _ := checkedFunction(native.Get(vm, stream, "setEncoding"))
if encoding != "" && !native.IsNil(setEncoding) {
_, _ = native.Call(vm, setEncoding, engine.ObjectValue(stream), native.From(vm, encoding))
}
go func() {
data := make([]byte, 32*1024)
@@ -426,11 +434,12 @@ func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goj
count, err := reader.Read(data)
if count > 0 {
chunk := append([]byte(nil), data[:count]...)
delivered := make(chan struct{})
if !m.runtime.RunOnLoop(func(vm *goja.Runtime) {
delivered := make(chan struct {
})
if !m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
defer close(delivered)
_ = m.runtime.RunJob(vm, "child_process stream", func() error {
_, pushErr := push(stream, buffer.WrapBytes(vm, chunk))
_, pushErr := native.Call(vm, push, engine.ObjectValue(stream), engine.ObjectValue(buffer.WrapBytes(vm, chunk)))
return pushErr
})
}) {
@@ -439,9 +448,9 @@ func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goj
<-delivered
}
if err != nil {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process stream close", func() error {
_, pushErr := push(stream, goja.Null())
_, pushErr := native.Call(vm, push, engine.ObjectValue(stream), moejs.Null())
return pushErr
})
})
@@ -450,11 +459,10 @@ func (m *Module) pipeChildOutput(vm *goja.Runtime, reader io.Reader, stream *goj
}
}()
}
func (m *Module) execChild(vm *goja.Runtime, command string, arguments []string, options childCommandOptions, callback goja.Callable) *goja.Object {
func (m *Module) execChild(vm *moejs.Runtime, command string, arguments []string, options childCommandOptions, callback moejs.Value) *engine.Object {
child := events.NewEmitter(vm)
_ = child.Set("pid", goja.Undefined())
_ = child.Set("killed", false)
_ = native.Set(vm, child, "pid", moejs.Undefined())
_ = native.Set(vm, child, "killed", false)
cmd, cancel := childExecCommand(command, arguments, options)
cmd.Dir, cmd.Env = options.cwd, options.env
outputLimit := m.childOutputLimit(options)
@@ -463,29 +471,34 @@ func (m *Module) execChild(vm *goja.Runtime, command string, arguments []string,
cmd.Stdout, cmd.Stderr = stdout, stderr
if err := cmd.Start(); err != nil {
cancel()
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.exec", func() error {
if _, callbackErr := callback(goja.Undefined(), childProcessError(vm, err, -1, nil, nil), goja.Undefined(), goja.Undefined()); callbackErr != nil {
if _, callbackErr := native.Call(vm, callback, moejs.Undefined(), engine.ObjectValue(childProcessError(vm, err, -1, nil, nil)), moejs.Undefined(), moejs.Undefined()); callbackErr != nil {
return callbackErr
}
return events.Emit(vm, child, "error", vm.NewGoError(err))
return events.Emit(vm, child, "error", native.GoError(vm, err))
})
}, 0)
return child
}
_ = child.Set("pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() { _ = cmd.Process.Kill(); cancel() })
_ = native.Set(vm, child, "pid", cmd.Process.Pid)
resourceID := m.runtime.AddResource(func() {
_ = cmd.Process.Kill()
cancel()
})
if resourceID == 0 {
go func() { _ = cmd.Wait() }()
go func() {
_ = cmd.Wait()
}()
return child
}
_ = child.Set("kill", func() bool {
_ = native.Set(vm, child, "kill", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
err := cmd.Process.Kill()
if err == nil {
_ = child.Set("killed", true)
_ = native.Set(vm, child, "killed", true)
}
return err == nil
})
return native.From(vm, err == nil), nil
}))
go func() {
err := cmd.Wait()
err = errors.Join(err, stdout.err, stderr.err)
@@ -498,20 +511,20 @@ func (m *Module) execChild(vm *goja.Runtime, command string, arguments []string,
} else if err != nil {
exitCode = -1
}
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "child_process.exec", func() error {
stdoutValue := encodeChildOutput(vm, stdout.Bytes(), options.encoding)
stderrValue := encodeChildOutput(vm, stderr.Bytes(), options.encoding)
var errorValue goja.Value = goja.Null()
var errorValue moejs.Value = moejs.Null()
if err != nil {
errorValue = childProcessError(vm, err, exitCode, stdout.Bytes(), stderr.Bytes())
errorValue = engine.ObjectValue(childProcessError(vm, err, exitCode, stdout.Bytes(), stderr.Bytes()))
}
if _, callbackErr := callback(goja.Undefined(), errorValue, stdoutValue, stderrValue); callbackErr != nil {
if _, callbackErr := native.Call(vm, callback, moejs.Undefined(), errorValue, stdoutValue, stderrValue); callbackErr != nil {
return callbackErr
}
_ = child.Set("exitCode", exitCode)
_ = events.Emit(vm, child, "exit", exitCode, goja.Null())
return events.Emit(vm, child, "close", exitCode, goja.Null())
_ = native.Set(vm, child, "exitCode", exitCode)
_ = events.Emit(vm, child, "exit", exitCode, moejs.Null())
return events.Emit(vm, child, "close", exitCode, moejs.Null())
})
})
}()
@@ -548,40 +561,38 @@ func (m *Module) runChildSync(command string, arguments []string, options childC
}
return result
}
func (m *Module) spawnChildSync(vm *goja.Runtime, command string, arguments []string, options childCommandOptions) *goja.Object {
func (m *Module) spawnChildSync(vm *moejs.Runtime, command string, arguments []string, options childCommandOptions) *engine.Object {
result := m.runChildSync(command, arguments, options)
object := vm.NewObject()
_ = object.Set("pid", result.pid)
_ = object.Set("status", result.exitCode)
_ = object.Set("signal", goja.Null())
_ = object.Set("stdout", encodeChildOutput(vm, result.stdout, options.encoding))
_ = object.Set("stderr", encodeChildOutput(vm, result.stderr, options.encoding))
_ = object.Set("output", []any{goja.Null(), encodeChildOutput(vm, result.stdout, options.encoding), encodeChildOutput(vm, result.stderr, options.encoding)})
object := vm.Realm().NewObject()
_ = native.Set(vm, object, "pid", result.pid)
_ = native.Set(vm, object, "status", result.exitCode)
_ = native.Set(vm, object, "signal", moejs.Null())
_ = native.Set(vm, object, "stdout", encodeChildOutput(vm, result.stdout, options.encoding))
_ = native.Set(vm, object, "stderr", encodeChildOutput(vm, result.stderr, options.encoding))
_ = native.Set(vm, object, "output", []any{moejs.Null(), encodeChildOutput(vm, result.stdout, options.encoding), encodeChildOutput(vm, result.stderr, options.encoding)})
if result.err != nil {
_ = object.Set("error", childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
_ = native.Set(vm, object, "error", childProcessError(vm, result.err, result.exitCode, result.stdout, result.stderr))
}
return object
}
func encodeChildOutput(vm *goja.Runtime, data []byte, encoding string) goja.Value {
func encodeChildOutput(vm *moejs.Runtime, data []byte, encoding string) moejs.Value {
if encoding == "" || encoding == "buffer" {
return buffer.WrapBytes(vm, append([]byte(nil), data...))
return engine.ObjectValue(buffer.WrapBytes(vm, append([]byte(nil), data...)))
}
return buffer.EncodeBytes(vm, data, vm.ToValue(encoding))
return buffer.EncodeBytes(vm, data, native.From(vm, encoding))
}
func childProcessError(vm *goja.Runtime, err error, exitCode int, stdout, stderr []byte) *goja.Object {
object := vm.NewGoError(err)
func childProcessError(vm *moejs.Runtime, err error, exitCode int, stdout, stderr []byte) *engine.Object {
object := native.Object(vm, native.GoError(vm, err))
var limitError *childOutputLimitError
if errors.As(err, &limitError) {
_ = object.Set("code", "ERR_CHILD_PROCESS_STDIO_MAXBUFFER")
_ = native.Set(vm, object, "code", "ERR_CHILD_PROCESS_STDIO_MAXBUFFER")
} else {
_ = object.Set("code", exitCode)
_ = native.Set(vm, object, "code", exitCode)
}
_ = object.Set("killed", false)
_ = object.Set("signal", goja.Null())
_ = object.Set("stdout", encodeChildOutput(vm, stdout, ""))
_ = object.Set("stderr", encodeChildOutput(vm, stderr, ""))
_ = native.Set(vm, object, "killed", false)
_ = native.Set(vm, object, "signal", moejs.Null())
_ = native.Set(vm, object, "stdout", encodeChildOutput(vm, stdout, ""))
_ = native.Set(vm, object, "stderr", encodeChildOutput(vm, stderr, ""))
return object
}
func checkedFunction(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
+61 -97
View File
@@ -1,17 +1,14 @@
package console
// JavaScript compatibility: console.assert(), debug(), error(), exception(),
// info(), log(), trace(), and warn(). Supports basic %s/%d/%i/%f/%o/%O/%c/%%
// substitutions. assert(false, ...) and trace() include a stack trace.
import (
"encoding/json"
"fmt"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
logger "github.com/komari-monitor/komari/utils/log"
"io"
"strings"
"github.com/dop251/goja"
logger "github.com/komari-monitor/komari/utils/log"
)
type level uint8
@@ -23,81 +20,78 @@ const (
errorLevel
)
type Module struct {
output io.Writer
}
type Module struct{ output io.Writer }
func New(output io.Writer) *Module {
return &Module{output: output}
}
func (m *Module) Inject(vm *goja.Runtime) error {
console := vm.NewObject()
console.Set("assert", func(call goja.FunctionCall) goja.Value {
if len(call.Arguments) > 0 && call.Argument(0).ToBoolean() {
return goja.Undefined()
func New(w io.Writer) *Module { return &Module{output: w} }
func (m *Module) Inject(vm *moe.Runtime) error {
o := vm.Realm().NewObject()
for _, entry := range []struct {
name string
level level
stack bool
}{{"debug", debug, false}, {"info", info, false}, {"log", info, false}, {"warn", warn, false}, {"error", errorLevel, false}, {"exception", errorLevel, false}, {"trace", debug, true}} {
entry := entry
if err := native.Function(vm, o, entry.name, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
m.write(vm, entry.level, args, entry.stack)
return moe.Undefined(), nil
}); err != nil {
return err
}
values := call.Arguments
if len(values) <= 1 {
values = []goja.Value{vm.ToValue("Assertion failed")}
} else {
values = append([]goja.Value{vm.ToValue("Assertion failed:")}, values[1:]...)
}
if err := native.Function(vm, o, "assert", func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
if len(args) > 0 && native.Bool(args[0]) {
return moe.Undefined(), nil
}
values := []moe.Value{moe.String("Assertion failed:")}
if len(args) > 1 {
values = append(values, args[1:]...)
}
m.write(vm, errorLevel, values, true)
return goja.Undefined()
})
console.Set("debug", func(call goja.FunctionCall) goja.Value {
m.write(vm, debug, call.Arguments, false)
return goja.Undefined()
})
error := func(call goja.FunctionCall) goja.Value {
m.write(vm, errorLevel, call.Arguments, false)
return goja.Undefined()
return moe.Undefined(), nil
}); err != nil {
return err
}
console.Set("error", error)
console.Set("exception", error)
console.Set("info", func(call goja.FunctionCall) goja.Value {
m.write(vm, info, call.Arguments, false)
return goja.Undefined()
})
console.Set("log", func(call goja.FunctionCall) goja.Value {
m.write(vm, info, call.Arguments, false)
return goja.Undefined()
})
console.Set("trace", func(call goja.FunctionCall) goja.Value {
m.write(vm, debug, call.Arguments, true)
return goja.Undefined()
})
console.Set("warn", func(call goja.FunctionCall) goja.Value {
m.write(vm, warn, call.Arguments, false)
return goja.Undefined()
})
return vm.Set("console", console)
return vm.SetGlobal("console", e.ObjectValue(o))
}
func (m *Module) WriteError(vm *goja.Runtime, values []goja.Value, withStack bool) {
m.write(vm, errorLevel, values, withStack)
func (m *Module) WriteError(vm *moe.Runtime, values []moe.Value, stack bool) {
m.write(vm, errorLevel, values, stack)
}
func (m *Module) Report(vm *goja.Runtime, name string, err error) {
m.WriteError(vm, []goja.Value{vm.ToValue(fmt.Sprintf("%s callback failed: %v", name, err))}, false)
func (m *Module) Report(vm *moe.Runtime, name string, err error) {
m.WriteError(vm, []moe.Value{moe.String(fmt.Sprintf("%s callback failed: %v", name, err))}, false)
}
func (m *Module) write(vm *goja.Runtime, messageLevel level, values []goja.Value, withStack bool) {
message := formatConsoleValues(values)
if withStack {
if stack := stackTrace(vm); stack != "" {
if message != "" {
message += "\n"
func (m *Module) write(vm *moe.Runtime, l level, values []moe.Value, withStack bool) {
args := make([]any, len(values))
for i, v := range values {
x, err := vm.ToGo(v)
if err != nil {
x = native.String(vm, v)
}
args[i] = x
}
message := ""
if len(args) > 0 {
if format, ok := args[0].(string); ok && strings.Contains(format, "%") {
message = formatConsoleString(format, args[1:])
} else {
parts := make([]string, len(args))
for i, x := range args {
parts[i] = formatConsoleValue(x)
}
message += stack
message = strings.Join(parts, " ")
}
}
if withStack {
errObj := vm.Realm().NewError(e.KindError, "")
stack := vm.Realm().StackTrace(e.ObjectValue(errObj))
if stack != "" {
message += "\n" + stack
}
}
if m.output != nil {
_, _ = fmt.Fprintln(m.output, message)
return
}
switch messageLevel {
switch l {
case debug:
logger.Debug("JavaScript", message)
case warn:
@@ -108,25 +102,6 @@ func (m *Module) write(vm *goja.Runtime, messageLevel level, values []goja.Value
logger.Info("JavaScript", message)
}
}
func formatConsoleValues(values []goja.Value) string {
if len(values) == 0 {
return ""
}
args := make([]any, 0, len(values))
for _, value := range values {
args = append(args, value.Export())
}
if format, ok := args[0].(string); ok && strings.Contains(format, "%") {
return formatConsoleString(format, args[1:])
}
parts := make([]string, 0, len(args))
for _, arg := range args {
parts = append(parts, formatConsoleValue(arg))
}
return strings.Join(parts, " ")
}
func formatConsoleValue(value any) string {
switch value := value.(type) {
case nil:
@@ -222,14 +197,3 @@ func toConsoleFloat(value any) float64 {
return 0
}
}
func stackTrace(vm *goja.Runtime) string {
if vm == nil {
return ""
}
value, err := vm.RunString("new Error().stack")
if err != nil {
return ""
}
return value.String()
}
File diff suppressed because it is too large. Load diff
+140
View File
@@ -0,0 +1,140 @@
package crypto
import (
"crypto/ed25519"
"crypto/rand"
"crypto/x509"
"encoding/pem"
"strconv"
"testing"
"time"
moe "github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/loop"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
func runNativeCryptoRegression(t *testing.T, script string) {
t.Helper()
results := make(chan string, 8)
l := loop.NewEventLoop(loop.WithOnError(func(err error) { results <- err.Error() }))
t.Cleanup(l.Terminate)
services := bridge.New(l, 10*time.Second)
services.SetErrorReporter(func(_ *moe.Runtime, _ string, err error) { results <- err.Error() })
if err := l.Start(); err != nil {
t.Fatal(err)
}
l.RunOnLoop(func(vm *moe.Runtime) {
buffer.Enable(vm)
module := vm.Realm().NewObject()
New(services).Load(vm, module)
_ = vm.SetGlobal("crypto", native.Get(vm, module, "exports"))
_ = vm.SetGlobal("done", native.Bind(vm, "done", 1, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
results <- native.String(vm, moe.Arg(args, 0))
return moe.Undefined(), nil
}))
if _, err := native.RunScript(vm, "crypto-native-test.js", script); err != nil {
results <- err.Error()
}
})
select {
case result := <-results:
if result != "ok" {
t.Fatal(result)
}
case <-time.After(12 * time.Second):
t.Fatal("native crypto callback was not delivered")
}
}
func TestNativeCryptoRandomIntSyncAndAsync(t *testing.T) {
runNativeCryptoRegression(t, `
const a = crypto.randomInt(1);
const b = crypto.randomInt(-3, -2);
if (typeof a !== "number" || a !== 0 || b !== -3) throw new Error("randomInt must return synchronously without callback");
let rejected = false;
try { crypto.randomFill(new Uint8Array(1)); } catch (e) { rejected = e instanceof TypeError; }
if (!rejected) throw new Error("missing randomFill callback accepted");
crypto.randomInt(4, 5, (err, value) => done(!err && value === 4 ? "ok" : "randomInt callback"));
`)
}
func TestNativeCryptoRandomFillLoopOwnership(t *testing.T) {
runNativeCryptoRegression(t, `
const backing = new ArrayBuffer(128);
const view = new Uint16Array(backing, 8, 32);
const raw = new Uint8Array(backing);
crypto.randomFill(view, 2, 16, (err, result) => {
if (err || result !== view) return done("randomFill callback identity");
if (raw.slice(0, 12).some(x => x !== 123) || raw.slice(44).some(x => x !== 123)) return done("randomFill wrote outside requested view range");
if (raw.slice(12, 44).every(x => x === 123)) return done("randomFill did not copy on loop");
done("ok");
});
// No loop callback can run yet. This must not overlap a worker writing raw.
for (let i = 0; i < 1000; i++) raw.fill(123);
`)
}
func TestNativeCryptoRandomFillDetachAndResize(t *testing.T) {
for _, mutation := range []string{"ab.transfer()", "ab.resize(4)"} {
t.Run(mutation, func(t *testing.T) {
runNativeCryptoRegression(t, `
const ab = new ArrayBuffer(64, {maxByteLength: 128});
const view = new Uint8Array(ab, 8, 32);
let calls = 0;
crypto.randomFill(view, 2, 16, (err, value) => {
if (++calls !== 1 || !(err instanceof Error) || value !== undefined) return done("detach/resize must callback with error");
done("ok");
});
`+mutation+`;`)
})
}
}
func TestNativeCryptoRandomFillReacquiresResizedStorage(t *testing.T) {
runNativeCryptoRegression(t, `
const ab = new ArrayBuffer(64, {maxByteLength: 1024});
const view = new DataView(ab, 4, 32);
crypto.randomFill(view, 3, 24, (err, value) => {
const raw = new Uint8Array(ab);
if (err || value !== view || raw.slice(7, 31).every(x => x === 0)) return done("stale randomFill storage after resize");
if (raw.slice(0, 7).some(x => x !== 0) || raw.slice(31).some(x => x !== 0)) return done("randomFill resize range");
done("ok");
});
ab.resize(1024);
`)
}
func TestNativeCryptoAsyncInputSnapshots(t *testing.T) {
public, private, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
privateDER, err := x509.MarshalPKCS8PrivateKey(private)
if err != nil {
t.Fatal(err)
}
publicDER, err := x509.MarshalPKIXPublicKey(public)
if err != nil {
t.Fatal(err)
}
privatePEM := string(pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privateDER}))
publicPEM := string(pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY", Bytes: publicDER}))
runNativeCryptoRegression(t, "const privatePEM = "+strconv.Quote(privatePEM)+"; const publicPEM = "+strconv.Quote(publicPEM)+";"+`
const password = Buffer.from("password"), salt = Buffer.from("salt");
const expectedPBKDF = crypto.pbkdf2Sync(password, salt, 1000, 32, "sha256").toString("hex");
const expectedScrypt = crypto.scryptSync(password, salt, 32, {N: 16, r: 1, p: 1}).toString("hex");
const data = Buffer.from("message"), privateKey = Buffer.from(privatePEM), publicKey = Buffer.from(publicPEM);
const signature = crypto.sign(null, data, privateKey);
let remaining = 4;
function checked(ok) { if (!ok) throw new Error("async crypto input was not snapshotted"); if (--remaining === 0) done("ok"); }
crypto.pbkdf2(password, salt, 1000, 32, "sha256", (err, value) => checked(!err && value.toString("hex") === expectedPBKDF));
crypto.scrypt(password, salt, 32, {N: 16, r: 1, p: 1}, (err, value) => checked(!err && value.toString("hex") === expectedScrypt));
crypto.sign(null, data, privateKey, (err, value) => checked(!err && value.toString("hex") === expectedSignature));
const expectedSignature = signature.toString("hex");
crypto.verify(null, data, publicKey, signature, (err, value) => checked(!err && value === true));
password.fill(0); salt.fill(0); data.fill(0); privateKey.fill(0); publicKey.fill(0); signature.fill(0);
`)
}
+26 -28
View File
@@ -2,46 +2,44 @@ package events
import (
_ "embed"
"fmt"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/require"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
)
func Load(vm *goja.Runtime, module *goja.Object) {
constructor, err := vm.RunString(eventEmitterSource)
func Load(vm *moe.Runtime, module *e.Object) {
constructor, err := native.Run(vm, eventEmitterSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load events module: %w", err)))
panic(err)
}
object := native.Object(vm, constructor)
if err = native.Set(vm, object, "EventEmitter", constructor); err != nil {
panic(err)
}
_ = native.Set(vm, object, "defaultMaxListeners", 10)
if err = native.Set(vm, module, "exports", constructor); err != nil {
panic(err)
}
_ = constructor.ToObject(vm).Set("EventEmitter", constructor)
_ = constructor.ToObject(vm).Set("defaultMaxListeners", 10)
_ = module.Set("exports", constructor)
}
func NewEmitter(vm *goja.Runtime) *goja.Object {
constructor := require.Require(vm, "events")
object, err := vm.New(constructor)
func NewEmitter(vm *moe.Runtime) *e.Object {
object, err := native.New(vm, require.Require(vm, "events"))
if err != nil {
panic(err)
}
return object
}
func Emit(vm *goja.Runtime, emitter *goja.Object, name string, values ...any) error {
emit, ok := goja.AssertFunction(emitter.Get("emit"))
if !ok {
return fmt.Errorf("EventEmitter.emit is not callable")
}
arguments := make([]goja.Value, 1, len(values)+1)
arguments[0] = vm.ToValue(name)
for _, value := range values {
if jsValue, ok := value.(goja.Value); ok {
arguments = append(arguments, jsValue)
} else {
arguments = append(arguments, vm.ToValue(value))
func Emit(vm *moe.Runtime, emitter *e.Object, name string, values ...any) error {
args := make([]moe.Value, 1, len(values)+1)
args[0] = moe.String(name)
for _, x := range values {
v, err := vm.FromGo(x)
if err != nil {
return err
}
args = append(args, v)
}
_, err := emit(emitter, arguments...)
_, err := vm.CallFunction(native.Get(vm, emitter, "emit"), e.ObjectValue(emitter), args...)
return err
}
+120 -138
View File
@@ -1,10 +1,17 @@
package fetch
import (
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
import (
"bytes"
"context"
"errors"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"io"
"mime"
"mime/multipart"
@@ -14,17 +21,12 @@ import (
"strconv"
"strings"
"sync"
"github.com/dop251/goja"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
)
type abortSignalState struct {
aborted bool
requests map[int64]context.CancelFunc
}
type nativeRequest struct {
method string
url string
@@ -33,7 +35,6 @@ type nativeRequest struct {
signalID int64
redirect string
}
type httpResult struct {
statusCode int
statusText string
@@ -42,7 +43,6 @@ type httpResult struct {
url string
redirected bool
}
type formEntry struct {
Name string
Value string
@@ -51,10 +51,9 @@ type formEntry struct {
Type string
File bool
}
type Module struct {
runtime *bridge.Runtime
vm *goja.Runtime
vm *moejs.Runtime
client *http.Client
maxBodyBytes int64
abortMu sync.Mutex
@@ -63,51 +62,57 @@ type Module struct {
}
func New(runtime *bridge.Runtime, client *http.Client, maxBodyBytes int64) *Module {
return &Module{
runtime: runtime,
client: client,
maxBodyBytes: maxBodyBytes,
abortState: make(map[int64]*abortSignalState),
}
return &Module{runtime: runtime, client: client, maxBodyBytes: maxBodyBytes, abortState: make(map[int64]*abortSignalState)}
}
func (m *Module) Inject(vm *goja.Runtime) error {
func (m *Module) Inject(vm *moejs.Runtime) error {
m.vm = vm
if err := vm.Set("__komariBodyBuffer", m.bodyBuffer); err != nil {
if err := vm.SetGlobal("__komariBodyBuffer", native.Bind(vm, "__komariBodyBuffer", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.bodyBuffer(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariBodyText", m.bodyText); err != nil {
if err := vm.SetGlobal("__komariBodyText", native.Bind(vm, "__komariBodyText", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.bodyText(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariEncodeFormData", m.encodeFormData); err != nil {
if err := vm.SetGlobal("__komariEncodeFormData", native.Bind(vm, "__komariEncodeFormData", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.encodeFormData(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariParseFormData", m.parseFormData); err != nil {
if err := vm.SetGlobal("__komariParseFormData", native.Bind(vm, "__komariParseFormData", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.parseFormData(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariNewAbortSignal", m.newAbortSignal); err != nil {
if err := vm.SetGlobal("__komariNewAbortSignal", native.Bind(vm, "__komariNewAbortSignal", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.newAbortSignal(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariAbortSignal", m.abortSignal); err != nil {
if err := vm.SetGlobal("__komariAbortSignal", native.Bind(vm, "__komariAbortSignal", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return m.abortSignal(args), nil
})); err != nil {
return err
}
if err := vm.Set("__komariFetch", m.createFetchFunction(false)); err != nil {
if err := vm.SetGlobal("__komariFetch", native.Bind(vm, "__komariFetch", 1, m.createFetchFunction(false))); err != nil {
return err
}
if err := vm.Set("__komariFetchSync", m.createFetchFunction(true)); err != nil {
if err := vm.SetGlobal("__komariFetchSync", native.Bind(vm, "__komariFetchSync", 1, m.createFetchFunction(true))); err != nil {
return err
}
if _, err := vm.RunString(fetchAPISource); err != nil {
if _, err := native.Run(vm, fetchAPISource); err != nil {
return fmt.Errorf("inject Fetch API: %w", err)
}
return nil
}
func (m *Module) createFetchFunction(syncRequest bool) func(goja.FunctionCall) goja.Value {
return func(call goja.FunctionCall) goja.Value {
request, err := m.exportNativeRequest(call.Argument(0))
func (m *Module) createFetchFunction(syncRequest bool) moejs.NativeFunc {
vm := m.vm
return func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
request, err := m.exportNativeRequest(moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("%s", err))
panic(native.TypeError(m.vm, "%s", err))
}
ctx, releaseSignal := m.requestContext(request.signalID)
ctx, cancelTimeout := context.WithTimeout(ctx, m.runtime.Timeout())
@@ -121,13 +126,12 @@ func (m *Module) createFetchFunction(syncRequest bool) func(goja.FunctionCall) g
if requestErr != nil {
panic(webAPIError(m.vm, "TypeError", fmt.Sprintf("fetch failed: %v", requestErr)))
}
return m.httpResultValue(m.vm, result)
return native.From(vm, m.httpResultValue(m.vm, result)), nil
}
promise, resolve, reject := m.vm.NewPromise()
go func() {
result, requestErr := m.doHTTPRequest(ctx, request)
if !m.runtime.RunOnLoop(func(vm *goja.Runtime) {
if !m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
defer release()
m.runtime.RunJob(vm, "fetch", func() error {
if ctx.Err() != nil {
@@ -146,30 +150,25 @@ func (m *Module) createFetchFunction(syncRequest bool) func(goja.FunctionCall) g
release()
}
}()
return m.vm.ToValue(promise)
return native.From(vm, native.From(m.vm, promise)), nil
}
}
func (m *Module) exportNativeRequest(value goja.Value) (nativeRequest, error) {
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func (m *Module) exportNativeRequest(value moejs.Value) (nativeRequest, error) {
vm := m.vm
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return nativeRequest{}, errors.New("missing request data")
}
object := value.ToObject(m.vm)
request := nativeRequest{
method: object.Get("method").String(),
url: object.Get("url").String(),
signalID: object.Get("signalId").ToInteger(),
redirect: object.Get("redirect").String(),
}
object := native.Object(m.vm, value)
request := nativeRequest{method: native.String(vm, native.Get(vm, object, "method")), url: native.String(vm, native.Get(vm, object, "url")), signalID: native.Int(vm, native.Get(vm, object, "signalId")), redirect: native.String(vm, native.Get(vm, object, "redirect"))}
if request.method == "" {
request.method = http.MethodGet
}
if request.redirect == "" {
request.redirect = "follow"
}
if value := object.Get("headers"); value != nil && !goja.IsUndefined(value) {
if value := native.Get(vm, object, "headers"); !native.IsNil(value) && !value.IsUndefined() {
var headers [][]string
if err := m.vm.ExportTo(value, &headers); err != nil {
if err := native.ExportTo(m.vm, value, &headers); err != nil {
return nativeRequest{}, fmt.Errorf("invalid request headers: %w", err)
}
for _, header := range headers {
@@ -179,7 +178,7 @@ func (m *Module) exportNativeRequest(value goja.Value) (nativeRequest, error) {
request.headers = append(request.headers, [2]string{header[0], header[1]})
}
}
if value := object.Get("body"); value != nil && !goja.IsUndefined(value) && !goja.IsNull(value) {
if value := native.Get(vm, object, "body"); !native.IsNil(value) && !value.IsUndefined() && !value.IsNull() {
body, err := exportBytes(m.vm, value)
if err != nil {
return nativeRequest{}, fmt.Errorf("invalid request body: %w", err)
@@ -188,7 +187,6 @@ func (m *Module) exportNativeRequest(value goja.Value) (nativeRequest, error) {
}
return request, nil
}
func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (httpResult, error) {
httpRequest, err := http.NewRequestWithContext(ctx, request.method, request.url, bytes.NewReader(request.body))
if err != nil {
@@ -201,7 +199,6 @@ func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (http
}
httpRequest.Header.Add(header[0], header[1])
}
client := *m.client
switch request.redirect {
case "manual":
@@ -216,7 +213,6 @@ func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (http
default:
return httpResult{}, fmt.Errorf("unsupported redirect mode %q", request.redirect)
}
response, err := client.Do(httpRequest)
if err != nil {
return httpResult{}, err
@@ -231,75 +227,60 @@ func (m *Module) doHTTPRequest(ctx context.Context, request nativeRequest) (http
responseURL = response.Request.URL.String()
}
statusText := strings.TrimSpace(strings.TrimPrefix(response.Status, strconv.Itoa(response.StatusCode)))
return httpResult{
statusCode: response.StatusCode,
statusText: statusText,
headers: response.Header.Clone(),
body: body,
url: responseURL,
redirected: responseURL != request.url,
}, nil
return httpResult{statusCode: response.StatusCode, statusText: statusText, headers: response.Header.Clone(), body: body, url: responseURL, redirected: responseURL != request.url}, nil
}
func (m *Module) httpResultValue(vm *goja.Runtime, result httpResult) goja.Value {
object := vm.NewObject()
_ = object.Set("status", result.statusCode)
_ = object.Set("statusText", result.statusText)
_ = object.Set("url", result.url)
_ = object.Set("redirected", result.redirected)
_ = object.Set("body", vm.NewArrayBuffer(append([]byte(nil), result.body...)))
func (m *Module) httpResultValue(vm *moejs.Runtime, result httpResult) moejs.Value {
object := vm.Realm().NewObject()
_ = native.Set(vm, object, "status", result.statusCode)
_ = native.Set(vm, object, "statusText", result.statusText)
_ = native.Set(vm, object, "url", result.url)
_ = native.Set(vm, object, "redirected", result.redirected)
_ = native.Set(vm, object, "body", native.From(vm, append([]byte(nil), result.body...)))
headers := make([][2]string, 0, len(result.headers))
for name, values := range result.headers {
for _, value := range values {
headers = append(headers, [2]string{name, value})
}
}
_ = object.Set("headers", headers)
return object
_ = native.Set(vm, object, "headers", headers)
return engine.ObjectValue(object)
}
func (m *Module) bodyBuffer(call goja.FunctionCall) goja.Value {
data, err := exportBytes(m.vm, call.Argument(0))
func (m *Module) bodyBuffer(args []moejs.Value) moejs.Value {
data, err := exportBytes(m.vm, moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("unsupported body value: %v", err))
panic(native.TypeError(m.vm, "unsupported body value: %v", err))
}
return m.vm.ToValue(m.vm.NewArrayBuffer(data))
return native.From(m.vm, native.From(m.vm, data))
}
func (m *Module) bodyText(call goja.FunctionCall) goja.Value {
data, err := exportBytes(m.vm, call.Argument(0))
func (m *Module) bodyText(args []moejs.Value) moejs.Value {
data, err := exportBytes(m.vm, moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("invalid body buffer: %v", err))
panic(native.TypeError(m.vm, "invalid body buffer: %v", err))
}
return m.vm.ToValue(strings.ToValidUTF8(string(data), "\uFFFD"))
return native.From(m.vm, strings.ToValidUTF8(string(data), "\uFFFD"))
}
func exportBytes(vm *goja.Runtime, value goja.Value) ([]byte, error) {
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func exportBytes(vm *moejs.Runtime, value moejs.Value) ([]byte, error) {
if native.IsNil(value) {
return nil, nil
}
if stringValue, ok := value.Export().(string); ok {
return []byte(stringValue), nil
if value.IsString() {
return []byte(native.String(vm, value)), nil
}
if buffer, ok := value.Export().(goja.ArrayBuffer); ok {
return append([]byte(nil), buffer.Bytes()...), nil
}
var bytesValue []byte
if err := vm.ExportTo(value, &bytesValue); err != nil {
var data []byte
if err := native.ExportTo(vm, value, &data); err != nil {
return nil, err
}
return append([]byte(nil), bytesValue...), nil
return append([]byte(nil), data...), nil
}
func (m *Module) newAbortSignal(goja.FunctionCall) goja.Value {
func (m *Module) newAbortSignal(args []moejs.Value) moejs.Value {
m.abortMu.Lock()
defer m.abortMu.Unlock()
m.abortIDs++
return m.vm.ToValue(m.abortIDs)
return native.From(m.vm, m.abortIDs)
}
func (m *Module) abortSignal(call goja.FunctionCall) goja.Value {
id := call.Argument(0).ToInteger()
func (m *Module) abortSignal(args []moejs.Value) moejs.Value {
vm := m.vm
id := native.Int(vm, moejs.Arg(args, 0))
m.abortMu.Lock()
state := m.abortState[id]
if state == nil {
@@ -311,15 +292,13 @@ func (m *Module) abortSignal(call goja.FunctionCall) goja.Value {
cancel()
}
m.abortMu.Unlock()
return goja.Undefined()
return moejs.Undefined()
}
func (m *Module) requestContext(signalID int64) (context.Context, context.CancelFunc) {
ctx, cancel := context.WithCancel(context.Background())
if signalID <= 0 {
return ctx, cancel
}
m.abortMu.Lock()
state := m.abortState[signalID]
if state != nil && state.aborted {
@@ -335,7 +314,6 @@ func (m *Module) requestContext(signalID int64) (context.Context, context.Cancel
requestID := m.abortIDs
state.requests[requestID] = cancel
m.abortMu.Unlock()
return ctx, func() {
m.abortMu.Lock()
if current := m.abortState[signalID]; current != nil {
@@ -348,7 +326,6 @@ func (m *Module) requestContext(signalID int64) (context.Context, context.Cancel
cancel()
}
}
func (m *Module) Close() {
m.abortMu.Lock()
defer m.abortMu.Unlock()
@@ -359,36 +336,34 @@ func (m *Module) Close() {
}
}
}
func webAPIError(vm *goja.Runtime, name, message string) *goja.Object {
object := vm.NewGoError(errors.New(message))
_ = object.Set("name", name)
func webAPIError(vm *moejs.Runtime, name, message string) moejs.Value {
object := native.GoError(vm, errors.New(message))
_ = native.Set(vm, native.Object(vm, object), "name", name)
return object
}
func (m *Module) encodeFormData(call goja.FunctionCall) goja.Value {
func (m *Module) encodeFormData(args []moejs.Value) moejs.Value {
vm := m.vm
var entries []formEntry
array := call.Argument(0).ToObject(m.vm)
length := int(array.Get("length").ToInteger())
array := native.Object(m.vm, moejs.Arg(args, 0))
length := int(native.Int(vm, native.Get(vm, array, "length")))
for index := 0; index < length; index++ {
entryObject := array.Get(strconv.Itoa(index)).ToObject(m.vm)
entry := formEntry{Name: entryObject.Get("name").String()}
value := entryObject.Get("value")
if entryObject.Get("file").ToBoolean() {
entryObject := native.Object(m.vm, native.Get(vm, array, strconv.Itoa(index)))
entry := formEntry{Name: native.String(vm, native.Get(vm, entryObject, "name"))}
value := native.Get(vm, entryObject, "value")
if native.Bool(native.Get(vm, entryObject, "file")) {
entry.File = true
entry.Filename = entryObject.Get("filename").String()
entry.Type = entryObject.Get("type").String()
entry.Filename = native.String(vm, native.Get(vm, entryObject, "filename"))
entry.Type = native.String(vm, native.Get(vm, entryObject, "type"))
data, err := exportBytes(m.vm, value)
if err != nil {
panic(m.vm.NewTypeError("invalid FormData file: %v", err))
panic(native.TypeError(m.vm, "invalid FormData file: %v", err))
}
entry.Bytes = data
} else {
entry.Value = value.String()
entry.Value = native.String(vm, value)
}
entries = append(entries, entry)
}
var output bytes.Buffer
writer := multipart.NewWriter(&output)
for _, entry := range entries {
@@ -403,27 +378,27 @@ func (m *Module) encodeFormData(call goja.FunctionCall) goja.Value {
}
part, err := writer.CreatePart(header)
if err != nil {
panic(m.vm.NewGoError(err))
panic(native.GoError(m.vm, err))
}
_, _ = part.Write(entry.Bytes)
}
if err := writer.Close(); err != nil {
panic(m.vm.NewGoError(err))
panic(native.GoError(m.vm, err))
}
result := m.vm.NewObject()
_ = result.Set("body", m.vm.NewArrayBuffer(output.Bytes()))
_ = result.Set("contentType", writer.FormDataContentType())
return result
result := m.vm.Realm().NewObject()
_ = native.Set(vm, result, "body", native.From(m.vm, output.Bytes()))
_ = native.Set(vm, result, "contentType", writer.FormDataContentType())
return native.From(vm, result)
}
func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
data, err := exportBytes(m.vm, call.Argument(0))
func (m *Module) parseFormData(args []moejs.Value) moejs.Value {
vm := m.vm
data, err := exportBytes(m.vm, moejs.Arg(args, 0))
if err != nil {
panic(m.vm.NewTypeError("invalid FormData body: %v", err))
panic(native.TypeError(m.vm, "invalid FormData body: %v", err))
}
mediaType, parameters, err := mime.ParseMediaType(call.Argument(1).String())
mediaType, parameters, err := mime.ParseMediaType(native.String(vm, moejs.Arg(args, 1)))
if err != nil {
panic(m.vm.NewTypeError("invalid form content type: %v", err))
panic(native.TypeError(m.vm, "invalid form content type: %v", err))
}
entries := make([]map[string]any, 0)
switch mediaType {
@@ -435,13 +410,13 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
nameValue := strings.SplitN(field, "=", 2)
name, parseErr := url.QueryUnescape(nameValue[0])
if parseErr != nil {
panic(m.vm.NewTypeError("invalid form field name: %v", parseErr))
panic(native.TypeError(m.vm, "invalid form field name: %v", parseErr))
}
value := ""
if len(nameValue) == 2 {
value, parseErr = url.QueryUnescape(nameValue[1])
if parseErr != nil {
panic(m.vm.NewTypeError("invalid form field value: %v", parseErr))
panic(native.TypeError(m.vm, "invalid form field value: %v", parseErr))
}
}
entries = append(entries, map[string]any{"name": name, "value": value})
@@ -449,7 +424,7 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
case "multipart/form-data":
boundary := parameters["boundary"]
if boundary == "" {
panic(m.vm.NewTypeError("multipart form data has no boundary"))
panic(native.TypeError(m.vm, "multipart form data has no boundary"))
}
reader := multipart.NewReader(bytes.NewReader(data), boundary)
for {
@@ -458,11 +433,11 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
break
}
if partErr != nil {
panic(m.vm.NewTypeError("invalid multipart body: %v", partErr))
panic(native.TypeError(m.vm, "invalid multipart body: %v", partErr))
}
partBytes, readErr := io.ReadAll(part)
if readErr != nil {
panic(m.vm.NewTypeError("read multipart body: %v", readErr))
panic(native.TypeError(m.vm, "read multipart body: %v", readErr))
}
entry := map[string]any{"name": part.FormName()}
if part.FileName() == "" {
@@ -471,12 +446,19 @@ func (m *Module) parseFormData(call goja.FunctionCall) goja.Value {
entry["file"] = true
entry["filename"] = part.FileName()
entry["type"] = part.Header.Get("Content-Type")
entry["value"] = m.vm.NewArrayBuffer(partBytes)
entry["value"] = native.From(m.vm, partBytes)
}
entries = append(entries, entry)
}
default:
panic(m.vm.NewTypeError("unsupported form content type %q", mediaType))
panic(native.TypeError(m.vm, "unsupported form content type %q", mediaType))
}
return m.vm.ToValue(entries)
return native.From(m.vm, entries)
}
func callable(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
func asObject(v moejs.Value) (*engine.Object, bool) {
if !v.IsObject() {
return nil, false
}
return v.AsObject(), true
}
+45
View File
@@ -0,0 +1,45 @@
package fetch
import (
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"net/http"
"testing"
)
func TestNativeFormDataAndBody(t *testing.T) {
vm := moejs.NewRuntime(moejs.Options{})
m := New(nil, http.DefaultClient, 1<<20)
if err := m.Inject(vm); err != nil {
t.Fatal(err)
}
value, err := native.Run(vm, `(async function () {
const blob = new Blob(["ab", new Uint8Array([99])], {type:"TEXT/PLAIN"});
if (blob.size !== 3 || await blob.text() !== "abc") throw new Error("blob bytes changed");
const form = new FormData();
form.append("field", "one"); form.append("field", "two");
form.append("file", new File([blob], "test.txt", {type:blob.type}));
const request = new Request("https://example.test/", {method:"POST", body:form});
const response = new Response(await request.arrayBuffer(), {headers:request.headers});
const parsed = await response.formData();
if (parsed.getAll("field").join(",") !== "one,two") throw new Error("form fields changed");
const file = parsed.get("file");
if (file.name !== "test.txt" || file.type !== "text/plain" || await file.text() !== "abc") throw new Error("multipart file changed");
const urlencoded = await new Response("hello=world&hello=again", {headers:{"content-type":"application/x-www-form-urlencoded"}}).formData();
if (urlencoded.getAll("hello").join(",") !== "world,again") throw new Error("urlencoded changed");
return true;
})()`)
if err != nil {
t.Fatal(err)
}
state, result, ok := moejs.PromiseResult(value)
if !ok {
t.Fatal("expected promise")
}
if state != moejs.PromiseFulfilled {
t.Fatalf("promise state %v result %v", state, native.String(vm, result))
}
if !native.Bool(result) {
t.Fatal("body/form verification failed")
}
}
+81 -89
View File
@@ -3,55 +3,50 @@ package fs
import (
"errors"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"io"
"io/fs"
"os"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
)
type fsAsyncOperation struct {
run func() (any, error)
values func(*goja.Runtime, any) []goja.Value
values func(*moejs.Runtime, any) []moejs.Value
}
type fsReadFileResult struct {
data []byte
encoding string
}
type fsReadDirResult struct {
entries []fs.DirEntry
withTypes bool
}
type fsReadResult struct {
count int
data []byte
target goja.Value
target moejs.Value
offset int
}
type fsWriteResult struct {
count int
target goja.Value
target moejs.Value
}
func fsArgument(arguments []goja.Value, index int) goja.Value {
func fsArgument(arguments []moejs.Value, index int) moejs.Value {
if index < 0 || index >= len(arguments) {
return goja.Undefined()
return moejs.Undefined()
}
return arguments[index]
}
func fsNoValues(*goja.Runtime, any) []goja.Value { return nil }
func fsSingleValue(vm *goja.Runtime, result any) []goja.Value {
return []goja.Value{vm.ToValue(result)}
func fsNoValues(*moejs.Runtime, any) []moejs.Value {
return nil
}
func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.Value) fsAsyncOperation {
func fsSingleValue(vm *moejs.Runtime, result any) []moejs.Value {
return []moejs.Value{native.From(vm, result)}
}
func (m *Module) prepareFSAsync(vm *moejs.Runtime, name string, arguments []moejs.Value) fsAsyncOperation {
cwd := m.Cwd()
follow := func(path string, allowMissing bool) (string, error) {
return m.resolveNodePathAt(path, cwd, allowMissing, nodePathFollowFinal)
@@ -59,29 +54,25 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
noFollow := func(path string, allowMissing bool) (string, error) {
return m.resolveNodePathAt(path, cwd, allowMissing, nodePathNoFollowFinal)
}
switch name {
case "readFile":
pathName := fsArgument(arguments, 0).String()
encoding := fsEncoding(fsArgument(arguments, 1))
return fsAsyncOperation{
run: func() (any, error) {
path, err := follow(pathName, false)
if err != nil {
return nil, err
}
data, err := m.nodeReadFile(path)
return fsReadFileResult{data: data, encoding: encoding}, err
},
values: func(vm *goja.Runtime, result any) []goja.Value {
value := result.(fsReadFileResult)
return []goja.Value{encodeFSData(vm, value.data, value.encoding)}
},
}
pathName := native.String(vm, fsArgument(arguments, 0))
encoding := fsEncoding(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err != nil {
return nil, err
}
data, err := m.nodeReadFile(path)
return fsReadFileResult{data: data, encoding: encoding}, err
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsReadFileResult)
return []moejs.Value{encodeFSData(vm, value.data, value.encoding)}
}}
case "writeFile", "appendFile":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
data := append([]byte(nil), buffer.Bytes(vm, fsArgument(arguments, 1))...)
mode := fsMode(fsArgument(arguments, 2), 0o666)
mode := fsMode(vm, fsArgument(arguments, 2), 0o666)
appendMode := name == "appendFile"
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, true)
@@ -100,8 +91,8 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, errors.Join(writeErr, file.Close())
}, values: fsNoValues}
case "access":
pathName := fsArgument(arguments, 0).String()
mode := fsAccessMode(fsArgument(arguments, 1))
pathName := native.String(vm, fsArgument(arguments, 0))
mode := fsAccessMode(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -110,7 +101,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "stat", "lstat":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
isLstat := name == "lstat"
return fsAsyncOperation{run: func() (any, error) {
var path string
@@ -127,12 +118,12 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return m.nodeStat(path, false)
}
return m.nodeStat(path, true)
}, values: func(vm *goja.Runtime, result any) []goja.Value {
return []goja.Value{fsStatObject(vm, result.(os.FileInfo))}
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
return []moejs.Value{native.From(vm, fsStatObject(vm, result.(os.FileInfo)))}
}}
case "readdir":
pathName := fsArgument(arguments, 0).String()
withTypes := fsWithFileTypes(fsArgument(arguments, 1))
pathName := native.String(vm, fsArgument(arguments, 0))
withTypes := fsWithFileTypes(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err != nil {
@@ -140,7 +131,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
}
entries, err := m.nodeReadDir(path)
return fsReadDirResult{entries: entries, withTypes: withTypes}, err
}, values: func(vm *goja.Runtime, result any) []goja.Value {
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsReadDirResult)
entries := make([]any, 0, len(value.entries))
for _, entry := range value.entries {
@@ -150,12 +141,12 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
entries = append(entries, entry.Name())
}
}
return []goja.Value{vm.ToValue(entries)}
return []moejs.Value{native.From(vm, entries)}
}}
case "mkdir":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
options := fsArgument(arguments, 1)
recursive, mode := fsBooleanOption(options, "recursive"), fsMode(options, 0o777)
recursive, mode := fsBooleanOption(vm, options, "recursive"), fsMode(vm, options, 0o777)
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(pathName, true)
if err != nil {
@@ -169,9 +160,9 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "rm":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
options := fsArgument(arguments, 1)
recursive, force := fsBooleanOption(options, "recursive"), fsBooleanOption(options, "force")
recursive, force := fsBooleanOption(vm, options, "recursive"), fsBooleanOption(vm, options, "force")
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(pathName, true)
if err != nil {
@@ -188,7 +179,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "unlink", "rmdir":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(pathName, false)
if err == nil {
@@ -197,7 +188,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "rename":
oldName, newName := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).String()
oldName, newName := native.String(vm, fsArgument(arguments, 0)), native.String(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
oldPath, err := noFollow(oldName, false)
if err != nil {
@@ -210,7 +201,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "copyFile":
sourceName, targetName := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).String()
sourceName, targetName := native.String(vm, fsArgument(arguments, 0)), native.String(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
source, err := follow(sourceName, false)
if err != nil {
@@ -233,7 +224,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, errors.Join(copyErr, output.Close())
}, values: fsNoValues}
case "realpath", "readlink":
pathName := fsArgument(arguments, 0).String()
pathName := native.String(vm, fsArgument(arguments, 0))
readLink := name == "readlink"
return fsAsyncOperation{run: func() (any, error) {
if readLink {
@@ -246,7 +237,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return follow(pathName, false)
}, values: fsSingleValue}
case "symlink":
targetName, linkName := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).String()
targetName, linkName := native.String(vm, fsArgument(arguments, 0)), native.String(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
target, err := follow(targetName, true)
if err != nil {
@@ -259,7 +250,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "truncate":
pathName, size := fsArgument(arguments, 0).String(), fsArgument(arguments, 1).ToInteger()
pathName, size := native.String(vm, fsArgument(arguments, 0)), native.Int(vm, fsArgument(arguments, 1))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -268,8 +259,8 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "chmod":
pathName := fsArgument(arguments, 0).String()
mode := os.FileMode(fsArgument(arguments, 1).ToInteger())
pathName := native.String(vm, fsArgument(arguments, 0))
mode := os.FileMode(native.Int(vm, fsArgument(arguments, 1)))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -278,8 +269,8 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "utimes":
pathName := fsArgument(arguments, 0).String()
accessTime, modifyTime := fsTime(fsArgument(arguments, 1).Export()), fsTime(fsArgument(arguments, 2).Export())
pathName := native.String(vm, fsArgument(arguments, 0))
accessTime, modifyTime := fsTime(native.Export(vm, fsArgument(arguments, 1))), fsTime(native.Export(vm, fsArgument(arguments, 2)))
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, false)
if err == nil {
@@ -288,7 +279,7 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "mkdtemp":
prefix := fsArgument(arguments, 0).String()
prefix := native.String(vm, fsArgument(arguments, 0))
return fsAsyncOperation{run: func() (any, error) {
path, err := noFollow(prefix, true)
if err != nil {
@@ -297,9 +288,9 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return m.nodeMkdirTemp(path)
}, values: fsSingleValue}
case "open":
pathName := fsArgument(arguments, 0).String()
flags := fsArgument(arguments, 1).String()
mode := fsMode(fsArgument(arguments, 2), 0o666)
pathName := native.String(vm, fsArgument(arguments, 0))
flags := native.String(vm, fsArgument(arguments, 1))
mode := fsMode(vm, fsArgument(arguments, 2), 0o666)
return fsAsyncOperation{run: func() (any, error) {
path, err := follow(pathName, true)
if err != nil {
@@ -308,21 +299,23 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return m.fsOpenPath(path, flags, mode)
}, values: fsSingleValue}
case "close":
fd := int(fsArgument(arguments, 0).ToInteger())
return fsAsyncOperation{run: func() (any, error) { return nil, m.fsClose(fd) }, values: fsNoValues}
fd := int(native.Int(vm, fsArgument(arguments, 0)))
return fsAsyncOperation{run: func() (any, error) {
return nil, m.fsClose(fd)
}, values: fsNoValues}
case "fstat":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
if err != nil {
return nil, err
}
return file.Stat()
}, values: func(vm *goja.Runtime, result any) []goja.Value {
return []goja.Value{fsStatObject(vm, result.(os.FileInfo))}
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
return []moejs.Value{native.From(vm, fsStatObject(vm, result.(os.FileInfo)))}
}}
case "fsync":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
if err == nil {
@@ -331,16 +324,16 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
return nil, err
}, values: fsNoValues}
case "read":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
target := fsArgument(arguments, 1)
offset := int(fsArgument(arguments, 2).ToInteger())
length := int(fsArgument(arguments, 3).ToInteger())
offset := int(native.Int(vm, fsArgument(arguments, 2)))
length := int(native.Int(vm, fsArgument(arguments, 3)))
positionValue := fsArgument(arguments, 4)
hasPosition := !goja.IsNull(positionValue) && !goja.IsUndefined(positionValue)
position := positionValue.ToInteger()
hasPosition := !positionValue.IsNull() && !positionValue.IsUndefined()
position := native.Int(vm, positionValue)
bufferLength := len(buffer.Bytes(vm, target))
if offset < 0 || length < 0 || offset+length > bufferLength {
panic(vm.NewTypeError("buffer range is out of bounds"))
panic(native.TypeError(vm, "buffer range is out of bounds"))
}
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
@@ -358,28 +351,28 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
err = nil
}
return fsReadResult{count: count, data: data[:count], target: target, offset: offset}, err
}, values: func(vm *goja.Runtime, result any) []goja.Value {
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsReadResult)
copy(buffer.Bytes(vm, value.target)[value.offset:], value.data)
return []goja.Value{vm.ToValue(value.count), value.target}
return []moejs.Value{native.From(vm, value.count), value.target}
}}
case "write":
fd := int(fsArgument(arguments, 0).ToInteger())
fd := int(native.Int(vm, fsArgument(arguments, 0)))
target := fsArgument(arguments, 1)
data := append([]byte(nil), buffer.Bytes(vm, target)...)
positionValue := fsArgument(arguments, 4)
if _, isString := target.Export().(string); !isString {
offset := int(fsArgument(arguments, 2).ToInteger())
length := int(fsArgument(arguments, 3).ToInteger())
if _, isString := native.Export(vm, target).(string); !isString {
offset := int(native.Int(vm, fsArgument(arguments, 2)))
length := int(native.Int(vm, fsArgument(arguments, 3)))
if offset < 0 || length < 0 || offset+length > len(data) {
panic(vm.NewTypeError("buffer range is out of bounds"))
panic(native.TypeError(vm, "buffer range is out of bounds"))
}
data = data[offset : offset+length]
} else {
positionValue = fsArgument(arguments, 2)
}
hasPosition := !goja.IsNull(positionValue) && !goja.IsUndefined(positionValue)
position := positionValue.ToInteger()
hasPosition := !positionValue.IsNull() && !positionValue.IsUndefined()
position := native.Int(vm, positionValue)
return fsAsyncOperation{run: func() (any, error) {
file, err := m.fsFile(fd)
if err != nil {
@@ -392,9 +385,9 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
count, err = file.Write(data)
}
return fsWriteResult{count: count, target: target}, err
}, values: func(vm *goja.Runtime, result any) []goja.Value {
}, values: func(vm *moejs.Runtime, result any) []moejs.Value {
value := result.(fsWriteResult)
return []goja.Value{vm.ToValue(value.count), value.target}
return []moejs.Value{native.From(vm, value.count), value.target}
}}
default:
return fsAsyncOperation{run: func() (any, error) {
@@ -402,7 +395,6 @@ func (m *Module) prepareFSAsync(vm *goja.Runtime, name string, arguments []goja.
}, values: fsNoValues}
}
}
func (m *Module) fsOpenPath(path, flags string, mode os.FileMode) (int, error) {
file, err := m.nodeOpenFile(path, fsOpenFlags(flags), mode)
if err != nil {
+11 -10
View File
@@ -2,14 +2,15 @@ package fs
import (
"errors"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"io/fs"
"os"
"strings"
"github.com/dop251/goja"
)
func nodeErrorObject(vm *goja.Runtime, err error, fallbackSyscall string) *goja.Object {
func nodeErrorObject(vm *moejs.Runtime, err error, fallbackSyscall string) *engine.Object {
operation, path, destination := fallbackSyscall, "", ""
underlying := err
var linkError *os.LinkError
@@ -41,16 +42,16 @@ func nodeErrorObject(vm *goja.Runtime, err error, fallbackSyscall string) *goja.
if errno == 0 {
errno = nodeErrnoForCode(code)
}
object := vm.NewGoError(err)
_ = object.Set("name", "Error")
_ = object.Set("code", code)
_ = object.Set("errno", errno)
_ = object.Set("syscall", operation)
object := native.Object(vm, native.GoError(vm, err))
_ = native.Set(vm, object, "name", "Error")
_ = native.Set(vm, object, "code", code)
_ = native.Set(vm, object, "errno", errno)
_ = native.Set(vm, object, "syscall", operation)
if path != "" {
_ = object.Set("path", path)
_ = native.Set(vm, object, "path", path)
}
if destination != "" {
_ = object.Set("dest", destination)
_ = native.Set(vm, object, "dest", destination)
}
return object
}
+345 -342
View File
File diff suppressed because it is too large. Load diff
+3 -2
View File
@@ -3,8 +3,9 @@ package jsruntime
import (
_ "embed"
"fmt"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
nodeurl "github.com/dop251/goja_nodejs/url"
nodeurl "github.com/komari-monitor/komari/pkg/jsruntime/url"
"github.com/komari-monitor/komari/pkg/jsruntime/xhr"
)
@@ -29,7 +30,7 @@ func (r *Runtime) injectGlobals() error {
if err := xhr.Inject(r.vm); err != nil {
return fmt.Errorf("inject XMLHttpRequest: %w", err)
}
if _, err := r.vm.RunString(queueMicrotaskSource); err != nil {
if _, err := native.Run(r.vm, queueMicrotaskSource); err != nil {
return fmt.Errorf("inject queueMicrotask: %w", err)
}
return nil
+24 -3
View File
@@ -9,7 +9,7 @@ package jsruntime
import (
"time"
"github.com/dop251/goja"
moe "github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
)
@@ -22,14 +22,14 @@ type Host struct {
// RunOnLoop queues job on the runtime's event loop. It returns false when
// the loop is no longer running, for example after Runtime.Close.
func (h *Host) RunOnLoop(job func(*goja.Runtime)) bool {
func (h *Host) RunOnLoop(job func(*moe.Runtime)) bool {
return h.runtime.RunOnLoop(job)
}
// RunJob runs one asynchronous host callback as a bounded JavaScript turn,
// reporting errors through the runtime's error reporter (the console by
// default). Exit code 0 is treated as a normal termination.
func (h *Host) RunJob(vm *goja.Runtime, name string, job func() error) error {
func (h *Host) RunJob(vm *moe.Runtime, name string, job func() error) error {
return h.runtime.RunJob(vm, name, job)
}
@@ -37,3 +37,24 @@ func (h *Host) RunJob(vm *goja.Runtime, name string, job func() error) error {
func (h *Host) Timeout() time.Duration {
return h.runtime.Timeout()
}
// AddResource registers a closer to run when the runtime shuts down. If shutdown
// has already started, it runs the closer immediately and returns 0. Closers
// must be nonblocking and must not access the JavaScript VM or wait for its event
// loop; they may run outside the loop during shutdown.
func (h *Host) AddResource(close func()) uint64 {
return h.runtime.AddResource(close)
}
// RemoveResource unregisters a resource without calling its closer. Unknown IDs
// and 0 are ignored.
func (h *Host) RemoveResource(id uint64) {
h.runtime.RemoveResource(id)
}
// ResourcesOpen reports whether the runtime still accepts resources, not whether
// any resources are registered. The result is a snapshot; AddResource handles
// registration racing with shutdown by immediately closing the new resource.
func (h *Host) ResourcesOpen() bool {
return h.runtime.ResourcesOpen()
}
+292 -249
View File
@@ -1,22 +1,25 @@
package http
import (
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
import (
"bytes"
_ "embed"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"net"
"net/http"
"strconv"
"strings"
"sync"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/httpbody"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
)
type nodeHTTPResponse struct {
@@ -26,61 +29,63 @@ type nodeHTTPResponse struct {
statusCode int
statusText string
ended bool
done chan struct{}
done chan struct {
}
}
type Module struct {
runtime *bridge.Runtime
allowListen bool
maxBodyBytes int64
runtime *bridge.Runtime
allowListen bool
maxBodyBytes int64
streamMu sync.Mutex
createIncoming goja.Callable
createResponse goja.Callable
createIncoming moejs.Value
createResponse moejs.Value
}
func New(runtime *bridge.Runtime, allowListen bool, maxBodyBytes int64) *Module {
return &Module{runtime: runtime, allowListen: allowListen, maxBodyBytes: maxBodyBytes}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
exports := vm.NewObject()
_ = exports.Set("createServer", func(call goja.FunctionCall) goja.Value {
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
exports := vm.Realm().NewObject()
_ = native.Set(vm, exports, "createServer", native.Bind(vm, "createServer", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
server := m.newHTTPServer(vm)
listener := call.Argument(0)
if _, ok := goja.AssertFunction(listener); !ok {
listener = call.Argument(1)
listener := moejs.Arg(args, 0)
if _, ok := callable(listener); !ok {
listener = moejs.Arg(args, 1)
}
if _, ok := goja.AssertFunction(listener); ok {
on, _ := goja.AssertFunction(server.Get("on"))
_, _ = on(server, vm.ToValue("request"), listener)
if _, ok := callable(listener); ok {
on, _ := callable(native.Get(vm, server, "on"))
_, _ = native.Call(vm, on, native.From(vm, server), native.From(vm, "request"), native.From(vm, listener))
}
return server
})
_ = exports.Set("METHODS", []string{"ACL", "BIND", "CHECKOUT", "CONNECT", "COPY", "DELETE", "GET", "HEAD", "LINK", "LOCK", "M-SEARCH", "MERGE", "MKACTIVITY", "MKCALENDAR", "MKCOL", "MOVE", "NOTIFY", "OPTIONS", "PATCH", "POST", "PROPFIND", "PROPPATCH", "PURGE", "PUT", "QUERY", "REBIND", "REPORT", "SEARCH", "SOURCE", "SUBSCRIBE", "TRACE", "UNBIND", "UNLINK", "UNLOCK", "UNSUBSCRIBE"})
return native.From(vm, server), nil
}))
_ = native.Set(vm, exports, "METHODS", []string{"ACL", "BIND", "CHECKOUT", "CONNECT", "COPY", "DELETE", "GET", "HEAD", "LINK", "LOCK", "M-SEARCH", "MERGE", "MKACTIVITY", "MKCALENDAR", "MKCOL", "MOVE", "NOTIFY", "OPTIONS", "PATCH", "POST", "PROPFIND", "PROPPATCH", "PURGE", "PUT", "QUERY", "REBIND", "REPORT", "SEARCH", "SOURCE", "SUBSCRIBE", "TRACE", "UNBIND", "UNLINK", "UNLOCK", "UNSUBSCRIBE"})
statusCodes := make(map[int]string)
for code := 100; code <= 599; code++ {
if text := http.StatusText(code); text != "" {
statusCodes[code] = text
}
}
_ = exports.Set("STATUS_CODES", statusCodes)
_ = exports.Set("maxHeaderSize", 1<<20)
_ = exports.Set("validateHeaderName", func(name string) {
_ = native.Set(vm, exports, "STATUS_CODES", statusCodes)
_ = native.Set(vm, exports, "maxHeaderSize", 1<<20)
_ = native.Set(vm, exports, "validateHeaderName", native.Bind(vm, "validateHeaderName", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
if !validHTTPToken(name) {
panic(vm.NewTypeError("Invalid HTTP header name"))
panic(native.TypeError(vm, "Invalid HTTP header name"))
}
})
_ = exports.Set("validateHeaderValue", func(name, value string) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, exports, "validateHeaderValue", native.Bind(vm, "validateHeaderValue", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
value := native.String(vm, moejs.Arg(args, 1))
if strings.ContainsAny(value, "\x00\r\n") {
panic(vm.NewTypeError("Invalid value for header %s", name))
panic(native.TypeError(vm, "Invalid value for header %s", name))
}
})
return moejs.Undefined(), nil
}))
m.attachHTTPClient(vm, exports)
_ = module.Set("exports", exports)
_ = native.Set(vm, module, "exports", exports)
}
func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
func (m *Module) newHTTPServer(vm *moejs.Runtime) *engine.Object {
serverObject := events.NewEmitter(vm)
var server *http.Server
var listener net.Listener
@@ -89,35 +94,38 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
var listenGeneration uint64
var listenPending bool
var readTimeout time.Duration
_ = serverObject.Set("listening", false)
_ = serverObject.Set("timeout", 0)
_ = serverObject.Set("keepAliveTimeout", 5000)
_ = serverObject.Set("headersTimeout", 60000)
_ = serverObject.Set("requestTimeout", 300000)
_ = native.Set(vm, serverObject, "listening", false)
_ = native.Set(vm, serverObject, "timeout", 0)
_ = native.Set(vm, serverObject, "keepAliveTimeout", 5000)
_ = native.Set(vm, serverObject, "headersTimeout", 60000)
_ = native.Set(vm, serverObject, "requestTimeout", 300000)
handler := http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
body, err := httpbody.ReadAll(request.Body, m.maxBodyBytes)
if err != nil {
http.Error(response, err.Error(), http.StatusRequestEntityTooLarge)
return
}
state := &nodeHTTPResponse{headers: make(http.Header), statusCode: http.StatusOK, done: make(chan struct{})}
queued := m.runtime.RunOnLoop(func(vm *goja.Runtime) {
state := &nodeHTTPResponse{headers: make(http.Header), statusCode: http.StatusOK, done: make(chan struct {
})}
queued := m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
incoming := m.httpIncomingMessage(vm, request, body)
outgoing := m.httpServerResponse(vm, state)
err := m.runtime.RunJob(vm, "http.Server request", func() error { return events.Emit(vm, serverObject, "request", incoming, outgoing) })
err := m.runtime.RunJob(vm, "http.Server request", func() error {
return events.Emit(vm, serverObject, "request", incoming, outgoing)
})
if err != nil {
state.finish(http.StatusInternalServerError, nil)
return
}
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.IncomingMessage body", func() error {
push, _ := goja.AssertFunction(incoming.Get("push"))
push, _ := callable(native.Get(vm, incoming, "push"))
if len(body) > 0 {
if _, pushErr := push(incoming, buffer.WrapBytes(vm, append([]byte(nil), body...))); pushErr != nil {
if _, pushErr := native.Call(vm, push, native.From(vm, incoming), native.From(vm, buffer.WrapBytes(vm, append([]byte(nil), body...)))); pushErr != nil {
return pushErr
}
}
_, pushErr := push(incoming, goja.Null())
_, pushErr := native.Call(vm, push, native.From(vm, incoming), native.From(vm, moejs.Null()))
return pushErr
})
}, 0)
@@ -147,15 +155,15 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
response.WriteHeader(statusCode)
_, _ = response.Write(responseBody)
})
_ = serverObject.Set("listen", func(call goja.FunctionCall) goja.Value {
_ = native.Set(vm, serverObject, "listen", native.Bind(vm, "listen", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if !m.allowListen {
panic(vm.NewGoError(fmt.Errorf("http.Server.listen requires AllowListen")))
panic(native.GoError(vm, fmt.Errorf("http.Server.listen requires AllowListen")))
}
address, callback := listenArguments(vm, call)
address, callback := listenArguments(vm, args)
serverMu.Lock()
if listenPending || listener != nil {
serverMu.Unlock()
panic(vm.NewGoError(fmt.Errorf("server is already listening")))
panic(native.GoError(vm, fmt.Errorf("server is already listening")))
}
listenGeneration++
generation := listenGeneration
@@ -172,19 +180,15 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
}
serverMu.Unlock()
if active {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server error", func() error {
return events.Emit(vm, serverObject, "error", vm.NewGoError(err))
return events.Emit(vm, serverObject, "error", native.GoError(vm, err))
})
})
}
return
}
createdServer := &http.Server{
Handler: handler,
ReadHeaderTimeout: m.runtime.Timeout(),
ReadTimeout: configuredReadTimeout,
}
createdServer := &http.Server{Handler: handler, ReadHeaderTimeout: m.runtime.Timeout(), ReadTimeout: configuredReadTimeout}
serverMu.Lock()
if !listenPending || listenGeneration != generation {
serverMu.Unlock()
@@ -205,19 +209,19 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
resourceID = id
listenPending = false
serverMu.Unlock()
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
serverMu.RLock()
active := listenGeneration == generation && server == createdServer && listener == created
serverMu.RUnlock()
if !active {
return
}
if callback != nil {
once, _ := goja.AssertFunction(serverObject.Get("once"))
_, _ = once(serverObject, vm.ToValue("listening"), vm.ToValue(callback))
if !native.IsNil(callback) {
once, _ := callable(native.Get(vm, serverObject, "once"))
_, _ = native.Call(vm, once, native.From(vm, serverObject), native.From(vm, "listening"), native.From(vm, callback))
}
_ = m.runtime.RunJob(vm, "http.Server listening", func() error {
_ = serverObject.Set("listening", true)
_ = native.Set(vm, serverObject, "listening", true)
return events.Emit(vm, serverObject, "listening")
})
})
@@ -232,19 +236,19 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
m.runtime.RemoveResource(id)
}
if active && err != nil && err != http.ErrServerClosed {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server error", func() error {
return events.Emit(vm, serverObject, "error", vm.NewGoError(err))
return events.Emit(vm, serverObject, "error", native.GoError(vm, err))
})
})
}
}()
return serverObject
})
_ = serverObject.Set("close", func(call goja.FunctionCall) goja.Value {
if callback, ok := goja.AssertFunction(call.Argument(0)); ok {
once, _ := goja.AssertFunction(serverObject.Get("once"))
_, _ = once(serverObject, vm.ToValue("close"), vm.ToValue(callback))
return native.From(vm, serverObject), nil
}))
_ = native.Set(vm, serverObject, "close", native.Bind(vm, "close", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if callback, ok := callable(moejs.Arg(args, 0)); ok {
once, _ := callable(native.Get(vm, serverObject, "once"))
_, _ = native.Call(vm, once, native.From(vm, serverObject), native.From(vm, "close"), native.From(vm, callback))
}
serverMu.Lock()
wasActive := listenPending || listener != nil
@@ -262,55 +266,59 @@ func (m *Module) newHTTPServer(vm *goja.Runtime) *goja.Object {
if currentResourceID != 0 {
m.runtime.RemoveResource(currentResourceID)
}
_ = serverObject.Set("listening", false)
_ = native.Set(vm, serverObject, "listening", false)
if wasActive {
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server close", func() error { return events.Emit(vm, serverObject, "close") })
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.Server close", func() error {
return events.Emit(vm, serverObject, "close")
})
}, 0)
}
return serverObject
})
_ = serverObject.Set("closeAllConnections", func() {
return native.From(vm, serverObject), nil
}))
_ = native.Set(vm, serverObject, "closeAllConnections", native.Bind(vm, "closeAllConnections", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
serverMu.RLock()
current := server
serverMu.RUnlock()
if current != nil {
_ = current.Close()
}
})
_ = serverObject.Set("closeIdleConnections", func() {
panic(vm.NewGoError(fmt.Errorf("http.Server.closeIdleConnections is not supported by jsruntime; net/http does not expose individual idle server connections")))
})
_ = serverObject.Set("address", func() goja.Value {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, serverObject, "closeIdleConnections", native.Bind(vm, "closeIdleConnections", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.Server.closeIdleConnections is not supported by jsruntime; net/http does not expose individual idle server connections")))
}))
_ = native.Set(vm, serverObject, "address", native.Bind(vm, "address", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
serverMu.RLock()
current := listener
serverMu.RUnlock()
if current == nil {
return goja.Null()
return native.From(vm, moejs.Null()), nil
}
host, port, _ := net.SplitHostPort(current.Addr().String())
portNumber, _ := strconv.Atoi(port)
return vm.ToValue(map[string]any{"address": host, "family": addressFamily(host), "port": portNumber})
})
_ = serverObject.Set("setTimeout", func(milliseconds int64, callback goja.Value) *goja.Object {
return native.From(vm, map[string]any{"address": host, "family": addressFamily(host), "port": portNumber}), nil
}))
_ = native.Set(vm, serverObject, "setTimeout", native.Bind(vm, "setTimeout", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
milliseconds := native.Int(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
serverMu.Lock()
readTimeout = time.Duration(milliseconds) * time.Millisecond
serverMu.Unlock()
if function, ok := goja.AssertFunction(callback); ok {
on, _ := goja.AssertFunction(serverObject.Get("on"))
_, _ = on(serverObject, vm.ToValue("timeout"), vm.ToValue(function))
if function, ok := callable(callback); ok {
on, _ := callable(native.Get(vm, serverObject, "on"))
_, _ = native.Call(vm, on, native.From(vm, serverObject), native.From(vm, "timeout"), native.From(vm, function))
}
return serverObject
})
_ = serverObject.Set("ref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("http.Server.ref is not supported by jsruntime; the event loop is host-driven")))
})
_ = serverObject.Set("unref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("http.Server.unref is not supported by jsruntime; the event loop is host-driven")))
})
return native.From(vm, serverObject), nil
}))
_ = native.Set(vm, serverObject, "ref", native.Bind(vm, "ref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.Server.ref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, serverObject, "unref", native.Bind(vm, "unref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.Server.unref is not supported by jsruntime; the event loop is host-driven")))
}))
return serverObject
}
func (state *nodeHTTPResponse) finish(status int, body []byte) {
state.mu.Lock()
if state.ended {
@@ -327,14 +335,13 @@ func (state *nodeHTTPResponse) finish(status int, body []byte) {
close(state.done)
state.mu.Unlock()
}
func (m *Module) httpIncomingMessage(vm *goja.Runtime, request *http.Request, body []byte) *goja.Object {
func (m *Module) httpIncomingMessage(vm *moejs.Runtime, request *http.Request, body []byte) *engine.Object {
m.initHTTPStreams(vm)
value, err := m.createIncoming(goja.Undefined())
value, err := native.Call(vm, m.createIncoming, native.From(vm, moejs.Undefined()))
if err != nil {
panic(err)
}
incoming := value.ToObject(vm)
incoming := native.Object(vm, value)
headers := make(map[string]string, len(request.Header))
rawHeaders := make([]string, 0, len(request.Header)*2)
for name, values := range request.Header {
@@ -343,26 +350,28 @@ func (m *Module) httpIncomingMessage(vm *goja.Runtime, request *http.Request, bo
rawHeaders = append(rawHeaders, name, value)
}
}
_ = incoming.Set("method", request.Method)
_ = incoming.Set("url", request.URL.RequestURI())
_ = incoming.Set("headers", headers)
_ = incoming.Set("headersDistinct", request.Header.Clone())
_ = incoming.Set("rawHeaders", rawHeaders)
_ = incoming.Set("httpVersion", fmt.Sprintf("%d.%d", request.ProtoMajor, request.ProtoMinor))
_ = incoming.Set("httpVersionMajor", request.ProtoMajor)
_ = incoming.Set("httpVersionMinor", request.ProtoMinor)
_ = incoming.Set("complete", true)
_ = incoming.Set("aborted", false)
_ = incoming.Set("socket", httpSocketInfo(vm, request))
_ = incoming.Set("connection", incoming.Get("socket"))
_ = incoming.Set("_destroy", func(err goja.Value, callback goja.Callable) {
_ = native.Set(vm, incoming, "method", request.Method)
_ = native.Set(vm, incoming, "url", request.URL.RequestURI())
_ = native.Set(vm, incoming, "headers", headers)
_ = native.Set(vm, incoming, "headersDistinct", request.Header.Clone())
_ = native.Set(vm, incoming, "rawHeaders", rawHeaders)
_ = native.Set(vm, incoming, "httpVersion", fmt.Sprintf("%d.%d", request.ProtoMajor, request.ProtoMinor))
_ = native.Set(vm, incoming, "httpVersionMajor", request.ProtoMajor)
_ = native.Set(vm, incoming, "httpVersionMinor", request.ProtoMinor)
_ = native.Set(vm, incoming, "complete", true)
_ = native.Set(vm, incoming, "aborted", false)
_ = native.Set(vm, incoming, "socket", httpSocketInfo(vm, request))
_ = native.Set(vm, incoming, "connection", native.Get(vm, incoming, "socket"))
_ = native.Set(vm, incoming, "_destroy", native.Bind(vm, "_destroy", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
err := moejs.Arg(args, 0)
callback := moejs.Arg(args, 1)
_ = request.Body.Close()
_, _ = callback(goja.Undefined(), err)
})
_, _ = native.Call(vm, callback, native.From(vm, moejs.Undefined()), native.From(vm, err))
return moejs.Undefined(), nil
}))
return incoming
}
func httpSocketInfo(vm *goja.Runtime, request *http.Request) *goja.Object {
func httpSocketInfo(vm *moejs.Runtime, request *http.Request) *engine.Object {
socket := events.NewEmitter(vm)
remoteHost, remotePort, _ := net.SplitHostPort(request.RemoteAddr)
localHost, localPort := "", 0
@@ -370,161 +379,185 @@ func httpSocketInfo(vm *goja.Runtime, request *http.Request) *goja.Object {
localHost, localPort = splitAddress(local)
}
remotePortNumber, _ := strconv.Atoi(remotePort)
_ = socket.Set("remoteAddress", remoteHost)
_ = socket.Set("remotePort", remotePortNumber)
_ = socket.Set("remoteFamily", addressFamily(remoteHost))
_ = socket.Set("localAddress", localHost)
_ = socket.Set("localPort", localPort)
_ = socket.Set("localFamily", addressFamily(localHost))
_ = socket.Set("encrypted", request.TLS != nil)
_ = native.Set(vm, socket, "remoteAddress", remoteHost)
_ = native.Set(vm, socket, "remotePort", remotePortNumber)
_ = native.Set(vm, socket, "remoteFamily", addressFamily(remoteHost))
_ = native.Set(vm, socket, "localAddress", localHost)
_ = native.Set(vm, socket, "localPort", localPort)
_ = native.Set(vm, socket, "localFamily", addressFamily(localHost))
_ = native.Set(vm, socket, "encrypted", request.TLS != nil)
return socket
}
func (m *Module) httpServerResponse(vm *goja.Runtime, state *nodeHTTPResponse) *goja.Object {
func (m *Module) httpServerResponse(vm *moejs.Runtime, state *nodeHTTPResponse) *engine.Object {
m.initHTTPStreams(vm)
hooks := vm.NewObject()
_ = hooks.Set("write", func(response *goja.Object, chunk goja.Value, callback goja.Callable) {
hooks := vm.Realm().NewObject()
_ = native.Set(vm, hooks, "write", native.Bind(vm, "write", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
response := native.Object(vm, moejs.Arg(args, 0))
chunk := moejs.Arg(args, 1)
callback := moejs.Arg(args, 2)
state.mu.Lock()
_, _ = state.body.Write(buffer.Bytes(vm, chunk))
state.mu.Unlock()
_ = response.Set("headersSent", true)
if callback != nil {
_, _ = callback(goja.Undefined())
_ = native.Set(vm, response, "headersSent", true)
if !native.IsNil(callback) {
_, _ = native.Call(vm, callback, native.From(vm, moejs.Undefined()))
}
})
_ = hooks.Set("finish", func(response *goja.Object, callback goja.Callable) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "finish", native.Bind(vm, "finish", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
response := native.Object(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
state.mu.Lock()
state.statusCode = int(response.Get("statusCode").ToInteger())
state.statusText = response.Get("statusMessage").String()
state.statusCode = int(native.Int(vm, native.Get(vm, response, "statusCode")))
state.statusText = native.String(vm, native.Get(vm, response, "statusMessage"))
if !state.ended {
state.ended = true
close(state.done)
}
state.mu.Unlock()
_ = response.Set("headersSent", true)
if callback != nil {
_, _ = callback(goja.Undefined())
_ = native.Set(vm, response, "headersSent", true)
if !native.IsNil(callback) {
_, _ = native.Call(vm, callback, native.From(vm, moejs.Undefined()))
}
})
_ = hooks.Set("destroy", func(*goja.Object) {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, hooks, "destroy", native.Bind(vm, "destroy", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.finish(0, nil)
})
value, err := m.createResponse(goja.Undefined(), hooks)
return moejs.Undefined(), nil
}))
value, err := native.Call(vm, m.createResponse, native.From(vm, moejs.Undefined()), native.From(vm, hooks))
if err != nil {
panic(err)
}
response := value.ToObject(vm)
_ = response.Set("statusCode", http.StatusOK)
_ = response.Set("statusMessage", "")
_ = response.Set("headersSent", false)
_ = response.Set("sendDate", true)
_ = response.Set("setHeader", func(name string, value goja.Value) *goja.Object {
response := native.Object(vm, value)
_ = native.Set(vm, response, "statusCode", http.StatusOK)
_ = native.Set(vm, response, "statusMessage", "")
_ = native.Set(vm, response, "headersSent", false)
_ = native.Set(vm, response, "sendDate", true)
_ = native.Set(vm, response, "setHeader", native.Bind(vm, "setHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
value := moejs.Arg(args, 1)
state.mu.Lock()
defer state.mu.Unlock()
var values []string
if err := vm.ExportTo(value, &values); err == nil {
if err := native.ExportTo(vm, value, &values); err == nil {
state.headers[name] = values
} else {
state.headers.Set(name, value.String())
state.headers.Set(name, native.String(vm, value))
}
return response
})
_ = response.Set("appendHeader", func(name, value string) *goja.Object {
return native.From(vm, response), nil
}))
_ = native.Set(vm, response, "appendHeader", native.Bind(vm, "appendHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
value := native.String(vm, moejs.Arg(args, 1))
state.mu.Lock()
state.headers.Add(name, value)
state.mu.Unlock()
return response
})
_ = response.Set("getHeader", func(name string) goja.Value {
return native.From(vm, response), nil
}))
_ = native.Set(vm, response, "getHeader", native.Bind(vm, "getHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
defer state.mu.Unlock()
values := state.headers.Values(name)
if len(values) == 0 {
return goja.Undefined()
return native.From(vm, moejs.Undefined()), nil
}
if len(values) == 1 {
return vm.ToValue(values[0])
return native.From(vm, values[0]), nil
}
return vm.ToValue(values)
})
_ = response.Set("getHeaders", func() http.Header { state.mu.Lock(); defer state.mu.Unlock(); return state.headers.Clone() })
_ = response.Set("getHeaderNames", func() []string {
return native.From(vm, values), nil
}))
_ = native.Set(vm, response, "getHeaders", native.Bind(vm, "getHeaders", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.mu.Lock()
defer state.mu.Unlock()
return native.From(vm, state.headers.Clone()), nil
}))
_ = native.Set(vm, response, "getHeaderNames", native.Bind(vm, "getHeaderNames", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
state.mu.Lock()
defer state.mu.Unlock()
names := make([]string, 0, len(state.headers))
for name := range state.headers {
names = append(names, strings.ToLower(name))
}
return names
})
_ = response.Set("hasHeader", func(name string) bool {
return native.From(vm, names), nil
}))
_ = native.Set(vm, response, "hasHeader", native.Bind(vm, "hasHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
defer state.mu.Unlock()
return len(state.headers.Values(name)) > 0
})
_ = response.Set("removeHeader", func(name string) { state.mu.Lock(); state.headers.Del(name); state.mu.Unlock() })
_ = response.Set("writeHead", func(call goja.FunctionCall) goja.Value {
_ = response.Set("statusCode", call.Argument(0).ToInteger())
if text := call.Argument(1); !goja.IsUndefined(text) {
if _, ok := text.(*goja.Object); ok {
return native.From(vm, len(state.headers.Values(name)) > 0), nil
}))
_ = native.Set(vm, response, "removeHeader", native.Bind(vm, "removeHeader", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
state.mu.Lock()
state.headers.Del(name)
state.mu.Unlock()
return moejs.Undefined(), nil
}))
_ = native.Set(vm, response, "writeHead", native.Bind(vm, "writeHead", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
_ = native.Set(vm, response, "statusCode", native.Int(vm, moejs.Arg(args, 0)))
if text := moejs.Arg(args, 1); !text.IsUndefined() {
if _, ok := asObject(text); ok {
applyHTTPHeaders(vm, response, text)
} else {
_ = response.Set("statusMessage", text.String())
applyHTTPHeaders(vm, response, call.Argument(2))
_ = native.Set(vm, response, "statusMessage", native.String(vm, text))
applyHTTPHeaders(vm, response, moejs.Arg(args, 2))
}
}
_ = response.Set("headersSent", true)
return response
})
_ = response.Set("flushHeaders", func() { _ = response.Set("headersSent", true) })
_ = response.Set("writeContinue", func() {
panic(vm.NewGoError(fmt.Errorf("http.ServerResponse.writeContinue is not supported by jsruntime; the Go server never sends 100 Continue")))
})
_ = response.Set("setTimeout", func(milliseconds int64, callback goja.Value) *goja.Object {
if function, ok := goja.AssertFunction(callback); ok {
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
_ = native.Set(vm, response, "headersSent", true)
return native.From(vm, response), nil
}))
_ = native.Set(vm, response, "flushHeaders", native.Bind(vm, "flushHeaders", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
_ = native.Set(vm, response, "headersSent", true)
return moejs.Undefined(), nil
}))
_ = native.Set(vm, response, "writeContinue", native.Bind(vm, "writeContinue", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("http.ServerResponse.writeContinue is not supported by jsruntime; the Go server never sends 100 Continue")))
}))
_ = native.Set(vm, response, "setTimeout", native.Bind(vm, "setTimeout", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
milliseconds := native.Int(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
if function, ok := callable(callback); ok {
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "http.ServerResponse timeout", func() error {
_, err := function(response)
_, err := native.Call(vm, function, native.From(vm, response))
return err
})
}, time.Duration(milliseconds)*time.Millisecond)
}
return response
})
return native.From(vm, response), nil
}))
return response
}
func applyHTTPHeaders(vm *goja.Runtime, response *goja.Object, value goja.Value) {
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func applyHTTPHeaders(vm *moejs.Runtime, response *engine.Object, value moejs.Value) {
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return
}
setHeader, _ := goja.AssertFunction(response.Get("setHeader"))
object := value.ToObject(vm)
for _, name := range object.Keys() {
_, _ = setHeader(response, vm.ToValue(name), object.Get(name))
setHeader, _ := callable(native.Get(vm, response, "setHeader"))
object := native.Object(vm, value)
for _, name := range objectKeys(object) {
_, _ = native.Call(vm, setHeader, native.From(vm, response), native.From(vm, name), native.From(vm, native.Get(vm, object, name)))
}
}
func listenArguments(vm *goja.Runtime, call goja.FunctionCall) (string, goja.Callable) {
func listenArguments(vm *moejs.Runtime, args []moejs.Value) (string, moejs.Value) {
host := "127.0.0.1"
port := int(call.Argument(0).ToInteger())
port := int(native.Int(vm, moejs.Arg(args, 0)))
callbackIndex := 1
if object, ok := call.Argument(0).(*goja.Object); ok {
port = int(object.Get("port").ToInteger())
if value := object.Get("host"); !goja.IsUndefined(value) {
host = value.String()
if object, ok := asObject(moejs.Arg(args, 0)); ok {
port = int(native.Int(vm, native.Get(vm, object, "port")))
if value := native.Get(vm, object, "host"); !value.IsUndefined() {
host = native.String(vm, value)
}
} else if value := call.Argument(1); !goja.IsUndefined(value) {
if _, ok := goja.AssertFunction(value); !ok {
host = value.String()
} else if value := moejs.Arg(args, 1); !value.IsUndefined() {
if _, ok := callable(value); !ok {
host = native.String(vm, value)
callbackIndex = 2
}
}
callback, _ := goja.AssertFunction(call.Argument(callbackIndex))
callback, _ := callable(moejs.Arg(args, callbackIndex))
return net.JoinHostPort(host, strconv.Itoa(port)), callback
}
func splitAddress(address net.Addr) (string, int) {
if address == nil {
return "", 0
@@ -533,30 +566,27 @@ func splitAddress(address net.Addr) (string, int) {
portNumber, _ := strconv.Atoi(port)
return host, portNumber
}
func addressFamily(host string) string {
if ip := net.ParseIP(host); ip != nil && ip.To4() == nil {
return "IPv6"
}
return "IPv4"
}
func (m *Module) attachHTTPClient(vm *goja.Runtime, exports *goja.Object) {
factoryValue, err := vm.RunString(httpClientSource)
func (m *Module) attachHTTPClient(vm *moejs.Runtime, exports *engine.Object) {
factoryValue, err := native.Run(vm, httpClientSource)
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
factory, _ := goja.AssertFunction(factoryValue)
client, err := factory(goja.Undefined(), require.Require(vm, "events"), vm.Get("fetch"), vm.Get("AbortController"), vm.Get("Blob"))
factory, _ := callable(factoryValue)
client, err := native.Call(vm, factory, native.From(vm, moejs.Undefined()), native.From(vm, require.Require(vm, "events")), native.From(vm, native.Global(vm, "fetch")), native.From(vm, native.Global(vm, "AbortController")), native.From(vm, native.Global(vm, "Blob")))
if err != nil {
panic(err)
}
clientObject := client.ToObject(vm)
for _, name := range clientObject.Keys() {
_ = exports.Set(name, clientObject.Get(name))
clientObject := native.Object(vm, client)
for _, name := range objectKeys(clientObject) {
_ = native.Set(vm, exports, name, native.Get(vm, clientObject, name))
}
}
func validHTTPToken(value string) bool {
if value == "" {
return false
@@ -569,33 +599,46 @@ func validHTTPToken(value string) bool {
return true
}
func (m *Module) initHTTPStreams(vm *moejs.Runtime) {
m.streamMu.Lock()
defer m.streamMu.Unlock()
if engine.IsCallable(m.createIncoming) {
return
}
factoryValue, err := native.Run(vm, httpStreamsSource)
if err != nil {
panic(native.GoError(vm, fmt.Errorf("load http streams: %w", err)))
}
factory, _ := callable(factoryValue)
value, err := native.Call(vm, factory, native.From(vm, moejs.Undefined()), native.From(vm, require.Require(vm, "stream")))
if err != nil {
panic(err)
}
object := native.Object(vm, value)
m.createIncoming, _ = callable(native.Get(vm, object, "createIncomingMessage"))
m.createResponse, _ = callable(native.Get(vm, object, "createServerResponse"))
if !engine.IsCallable(m.createIncoming) || !engine.IsCallable(m.createResponse) {
panic(native.GoError(vm, fmt.Errorf("http streams factory is incomplete")))
}
}
func callable(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
func asObject(v moejs.Value) (*engine.Object, bool) {
if !v.IsObject() {
return nil, false
}
return v.AsObject(), true
}
func objectKeys(o *engine.Object) []string {
keys := o.OwnEnumerableStringKeys()
out := make([]string, len(keys))
for i, k := range keys {
out[i] = k.GoString()
}
return out
}
//go:embed client.js
var httpClientSource string
//go:embed streams.js
var httpStreamsSource string
// initHTTPStreams loads the embedded factory that builds IncomingMessage
// (Readable) and ServerResponse (Writable) instances from the stream module.
func (m *Module) initHTTPStreams(vm *goja.Runtime) {
m.streamMu.Lock()
defer m.streamMu.Unlock()
if m.createIncoming != nil {
return
}
factoryValue, err := vm.RunString(httpStreamsSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load http streams: %w", err)))
}
factory, _ := goja.AssertFunction(factoryValue)
value, err := factory(goja.Undefined(), require.Require(vm, "stream"))
if err != nil {
panic(err)
}
object := value.ToObject(vm)
m.createIncoming, _ = goja.AssertFunction(object.Get("createIncomingMessage"))
m.createResponse, _ = goja.AssertFunction(object.Get("createServerResponse"))
if m.createIncoming == nil || m.createResponse == nil {
panic(vm.NewGoError(fmt.Errorf("http streams factory is incomplete")))
}
}
+8 -33
View File
@@ -3,44 +3,19 @@ package bridge
import (
"errors"
"fmt"
"github.com/dop251/goja"
moe "github.com/Calcium-Ion/moejs"
)
// ExitRequestedError stops a script turn with the requested process exit code.
// Code 0 is treated as a normal termination; nonzero codes are surfaced as
// failures.
type ExitRequestedError struct {
Code int64
}
type ExitRequestedError struct{ Code int64 }
func (e *ExitRequestedError) Error() string {
return fmt.Sprintf("process.exit(%d) requested", e.Code)
}
// ExitCodeFromError returns the exit code when err was thrown by process.exit.
func (e *ExitRequestedError) Error() string { return fmt.Sprintf("process.exit(%d) requested", e.Code) }
func ExitCodeFromError(err error) (int64, bool) {
var exitErr *ExitRequestedError
if !errors.As(err, &exitErr) {
var exit *ExitRequestedError
if !errors.As(err, &exit) {
return 0, false
}
return exitErr.Code, true
return exit.Code, true
}
// ExitCodeFromValue returns the exit code when a rejected Promise carries the
// process.exit error object.
func ExitCodeFromValue(value goja.Value) (int64, bool) {
object, ok := value.(*goja.Object)
if !ok {
return 0, false
}
raw := object.Get("value")
if raw == nil || goja.IsUndefined(raw) || goja.IsNull(raw) {
return 0, false
}
err, ok := raw.Export().(error)
if !ok {
return 0, false
}
return ExitCodeFromError(err)
func ExitCodeFromValue(value moe.Value) (int64, bool) {
return ExitCodeFromError(&moe.Exception{Value: value})
}
+17 -12
View File
@@ -8,19 +8,19 @@ import (
"sync/atomic"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/eventloop"
moe "github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/loop"
)
var ErrExecutionTimeout = errors.New("JavaScript execution timeout")
type TurnRunner func(*goja.Runtime, func() error) error
type TurnRunner func(*moe.Runtime, func() error) error
type ErrorReporter func(*goja.Runtime, string, error)
type ErrorReporter func(*moe.Runtime, string, error)
// Runtime is the host-facing service boundary used by JavaScript modules.
type Runtime struct {
loop *eventloop.EventLoop
loop *loop.EventLoop
timeout time.Duration
callbackMu sync.RWMutex
@@ -33,7 +33,7 @@ type Runtime struct {
resourcesClosed bool
}
func New(loop *eventloop.EventLoop, timeout time.Duration) *Runtime {
func New(loop *loop.EventLoop, timeout time.Duration) *Runtime {
return &Runtime{
loop: loop,
timeout: timeout,
@@ -41,7 +41,7 @@ func New(loop *eventloop.EventLoop, timeout time.Duration) *Runtime {
}
}
func (r *Runtime) Loop() *eventloop.EventLoop { return r.loop }
func (r *Runtime) Loop() *loop.EventLoop { return r.loop }
func (r *Runtime) Timeout() time.Duration { return r.timeout }
@@ -57,11 +57,11 @@ func (r *Runtime) SetErrorReporter(report ErrorReporter) {
r.callbackMu.Unlock()
}
func (r *Runtime) RunOnLoop(job func(*goja.Runtime)) bool {
func (r *Runtime) RunOnLoop(job func(*moe.Runtime)) bool {
return r.loop.RunOnLoop(job)
}
func (r *Runtime) RunTurn(vm *goja.Runtime, job func() error) error {
func (r *Runtime) RunTurn(vm *moe.Runtime, job func() error) error {
r.callbackMu.RLock()
run := r.runTurn
r.callbackMu.RUnlock()
@@ -72,7 +72,7 @@ func (r *Runtime) RunTurn(vm *goja.Runtime, job func() error) error {
}
// RunJob executes one asynchronous host callback as a bounded JavaScript turn.
func (r *Runtime) RunJob(vm *goja.Runtime, name string, job func() error) error {
func (r *Runtime) RunJob(vm *moe.Runtime, name string, job func() error) error {
if !r.ResourcesOpen() {
return nil
}
@@ -146,7 +146,7 @@ func (r *Runtime) CloseResources() {
}
// RunWithDeadline interrupts synchronous JavaScript that exceeds deadline.
func RunWithDeadline(vm *goja.Runtime, deadline time.Time, fn func() error) error {
func RunWithDeadline(vm *moe.Runtime, deadline time.Time, fn func() error) error {
remaining := time.Until(deadline)
if remaining <= 0 {
return ErrExecutionTimeout
@@ -164,7 +164,9 @@ func RunWithDeadline(vm *goja.Runtime, deadline time.Time, fn func() error) erro
}
})
err := fn()
var err error
var recovered any
func() { defer func() { recovered = recover() }(); err = fn() }()
if running.CompareAndSwap(true, false) {
if !timer.Stop() {
<-timerDone
@@ -176,5 +178,8 @@ func RunWithDeadline(vm *goja.Runtime, deadline time.Time, fn func() error) erro
if timedOut.Load() {
return ErrExecutionTimeout
}
if recovered != nil {
panic(recovered)
}
return err
}
+46
View File
@@ -0,0 +1,46 @@
package jsruntime
import (
"io"
"testing"
"time"
)
func TestMutableIntrinsicsAreIsolated(t *testing.T) {
for _, nodeJS := range []bool{false, true} {
name := "Web"
if nodeJS {
name = "Node"
}
t.Run(name, func(t *testing.T) {
opts := Options{NodeJS: nodeJS, BaseDir: t.TempDir(), Console: io.Discard, Timeout: time.Second}
first, err := New(`
Object.defineProperty(Reflect,"decorate",{value:function(){return "decorated"},configurable:true,writable:true});
Object.defineProperty(Array.prototype,"__komariPluginProbe",{value:"private",configurable:true});
function verify(){return Reflect.decorate()==="decorated"&&[].__komariPluginProbe==="private";}
`, opts)
if err != nil {
t.Fatal(err)
}
defer first.Close()
if err := first.Call("verify"); err != nil {
t.Fatal(err)
}
second, err := New(`
if(Object.prototype.hasOwnProperty.call(Reflect,"decorate")||Object.prototype.hasOwnProperty.call(Array.prototype,"__komariPluginProbe")){throw new Error("intrinsics leaked between plugins");}
Reflect.decorate=function(){return "second"};
function verify(){return Reflect.decorate()==="second"&&typeof [].__komariPluginProbe==="undefined";}
`, opts)
if err != nil {
t.Fatal(err)
}
defer second.Close()
if err := second.Call("verify"); err != nil {
t.Fatal(err)
}
if err := first.Call("verify"); err != nil {
t.Fatalf("second runtime changed first: %v", err)
}
})
}
}
+13
View File
@@ -0,0 +1,13 @@
Copyright (c) 2016 Dmitry Panov
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+140
View File
@@ -0,0 +1,140 @@
// Package loop serializes native MoeJS host jobs.
package loop
import (
"github.com/Calcium-Ion/moejs"
upstream "github.com/Calcium-Ion/moejs/eventloop"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"sync/atomic"
"time"
)
type EventLoop struct {
core *upstream.Loop
registry *require.Registry
onError func(error)
setTimeout, setInterval, clearTimeout, clearInterval moejs.Value
}
type Option func(*EventLoop)
func WithRegistry(r *require.Registry) Option { return func(l *EventLoop) { l.registry = r } }
func WithOnError(f func(error)) Option { return func(l *EventLoop) { l.onError = f } }
func NewEventLoop(opts ...Option) *EventLoop {
l := &EventLoop{}
for _, opt := range opts {
opt(l)
}
// Plugins may extend builtins (for example reflect-metadata). Keep those
// changes private to this runtime instead of using shared frozen intrinsics.
vm := moejs.NewRuntime(moejs.Options{MutableIntrinsics: true})
var err error
l.core, err = upstream.New(vm, upstream.Options{OnError: func(err error) {
if l.onError != nil {
l.onError(err)
}
}})
if err != nil {
panic(err)
}
if l.registry == nil {
l.registry = require.NewRegistry()
}
l.registry.Enable(vm)
l.setTimeout = native.Global(vm, "setTimeout")
l.setInterval = native.Global(vm, "setInterval")
l.clearTimeout = native.Global(vm, "clearTimeout")
l.clearInterval = native.Global(vm, "clearInterval")
return l
}
func (l *EventLoop) Runtime() *moejs.Runtime { return l.core.Runtime() }
func (l *EventLoop) Start() error { return l.core.Start() }
func (l *EventLoop) Run(fn func(*moejs.Runtime)) error {
return l.core.Run(func(vm *moejs.Runtime) error {
if fn != nil {
fn(vm)
}
return nil
})
}
func (l *EventLoop) RunOnLoop(fn func(*moejs.Runtime)) bool { return l.core.RunOnLoop(fn) }
func (l *EventLoop) Stop() error { return l.core.Stop() }
func (l *EventLoop) StopNoWait() { l.core.StopNoWait() }
// Terminate is an external-goroutine close: interrupt, join, drop pending work.
func (l *EventLoop) Terminate() { l.core.Terminate() }
func (l *EventLoop) Interrupt(v any) { l.core.Interrupt(v) }
type timerState struct {
cancelled atomic.Bool
value moejs.Value
}
type Timer struct{ state timerState }
type Interval struct{ state timerState }
func (l *EventLoop) schedule(s *timerState, fn func(*moejs.Runtime), delay time.Duration, repeating bool) bool {
return l.RunOnLoop(func(vm *moejs.Runtime) {
if s.cancelled.Load() {
return
}
cb := native.Bind(vm, "hostTimer", 0, func(_ *moejs.Realm, _ moejs.Value, _ []moejs.Value) (moejs.Value, error) {
if !s.cancelled.Load() {
fn(vm)
}
return moejs.Undefined(), nil
})
setter := l.setTimeout
if repeating {
setter = l.setInterval
}
v, err := vm.CallFunction(setter, moejs.Undefined(), cb, native.From(vm, float64(delay)/float64(time.Millisecond)))
if err != nil {
if l.onError != nil {
l.onError(err)
}
return
}
s.value = v
})
}
func (l *EventLoop) SetTimeout(fn func(*moejs.Runtime), delay time.Duration) *Timer {
t := &Timer{}
if !l.schedule(&t.state, fn, delay, false) {
return nil
}
return t
}
func (l *EventLoop) SetInterval(fn func(*moejs.Runtime), delay time.Duration) *Interval {
t := &Interval{}
if !l.schedule(&t.state, fn, delay, true) {
return nil
}
return t
}
func (l *EventLoop) cancel(s *timerState, repeating bool) {
s.cancelled.Store(true)
l.RunOnLoop(func(vm *moejs.Runtime) {
if s.value.IsUndefined() {
return
}
clear := l.clearTimeout
if repeating {
clear = l.clearInterval
}
_, err := vm.CallFunction(clear, moejs.Undefined(), s.value)
if err != nil && l.onError != nil {
l.onError(err)
}
})
}
func (l *EventLoop) ClearTimeout(t *Timer) {
if t != nil {
l.cancel(&t.state, false)
}
}
func (l *EventLoop) ClearInterval(t *Interval) {
if t != nil {
l.cancel(&t.state, true)
}
}
+65
View File
@@ -0,0 +1,65 @@
package loop
import (
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"sync/atomic"
"testing"
"time"
)
func TestQueueAndTerminate(t *testing.T) {
l := NewEventLoop()
if err := l.Start(); err != nil {
t.Fatal(err)
}
done := make(chan struct{})
if !l.RunOnLoop(func(vm *moejs.Runtime) { close(done) }) {
t.Fatal("rejected")
}
<-done
l.Terminate()
if l.RunOnLoop(func(*moejs.Runtime) {}) {
t.Fatal("accepted after close")
}
if l.SetTimeout(func(*moejs.Runtime) {}, time.Millisecond) != nil {
t.Fatal("timer after close")
}
}
func TestHostAndJSTimers(t *testing.T) {
l := NewEventLoop()
defer l.Terminate()
var count atomic.Int32
timer := l.SetTimeout(func(*moejs.Runtime) { count.Add(100) }, time.Millisecond)
l.ClearTimeout(timer)
l.SetTimeout(func(*moejs.Runtime) { count.Add(1) }, time.Millisecond)
if err := l.Run(func(vm *moejs.Runtime) {
if _, err := native.Run(vm, "var n=0;var t=setInterval(function(){n++;if(n===2)clearInterval(t)},1)"); err != nil {
t.Error(err)
}
}); err != nil {
t.Fatal(err)
}
if count.Load() != 1 {
t.Fatal(count.Load())
}
if n := native.Int(l.Runtime(), native.Global(l.Runtime(), "n")); n != 2 {
t.Fatal(n)
}
}
func TestTerminateInterrupts(t *testing.T) {
l := NewEventLoop()
if err := l.Start(); err != nil {
t.Fatal(err)
}
started := make(chan struct{})
l.RunOnLoop(func(vm *moejs.Runtime) { close(started); _, _ = native.Run(vm, "for(;;){}") })
<-started
done := make(chan struct{})
go func() { l.Terminate(); close(done) }()
select {
case <-done:
case <-time.After(time.Second * 3):
t.Fatal("terminate did not interrupt")
}
}
@@ -0,0 +1,33 @@
package jsruntime
import (
"io"
"testing"
"time"
)
// Migration regressions exercise engine/host boundaries rather than language-only
// microbenchmarks. Keep this suite identical in the goja comparison worktree.
func TestRuntimeMigrationBoundaries(t *testing.T) {
tests := []struct{ name, script string }{
{"hostThrowCatch", `function run() { try { require("fs").readFileSync("missing-file"); } catch(e) { return e instanceof Error && typeof e.message === "string"; } return false; }`},
{"bufferLiveWrite", `function run() { const b=Buffer.alloc(8); const view=b.subarray(2,6); require("crypto").randomFillSync(view); return b[0]===0 && b[1]===0 && b[6]===0 && b[7]===0 && view.some(x=>x!==0); }`},
{"floatRandomRejected", `function run() { try { crypto.getRandomValues(new Float32Array(4)); } catch(e) { return true; } return false; }`},
{"constructorAndURL", `function run() { const u=new URL("/a?x=1", "https://example.com"); return u.hostname==="example.com" && u.searchParams.get("x")==="1"; }`},
{"promiseRejectCatch", `async function run() { try { await Promise.reject(new Error("expected")); } catch(e) { return e.message==="expected"; } return false; }`},
{"commonJSIdentity", `function run() { return require("buffer")===require("node:buffer") && Buffer===require("buffer").Buffer; }`},
}
for _, tc := range tests {
b := tc
t.Run(b.name, func(t *testing.T) {
rt, err := New(b.script, Options{BaseDir: t.TempDir(), NodeJS: true, Console: io.Discard, Timeout: time.Second})
if err != nil {
t.Fatal(err)
}
defer rt.Close()
if err := rt.Call("run"); err != nil {
t.Fatal(err)
}
})
}
}
+171
View File
@@ -0,0 +1,171 @@
// Package native contains small operations on MoeJS values for Komari host modules.
// It defines no runtime/value/object wrappers and performs no reflected binding.
package native
import (
"fmt"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"math"
)
func Callable(v moe.Value) (moe.Value, bool) { return v, e.IsCallable(v) }
func AsObject(v moe.Value) (*e.Object, bool) {
if !v.IsObject() {
return nil, false
}
return v.AsObject(), true
}
func IsNil(v moe.Value) bool { return v.IsUndefined() || v.IsNull() }
func From(vm *moe.Runtime, x any) moe.Value {
v, err := vm.FromGo(x)
if err != nil {
panic(err)
}
return v
}
func Get(vm *moe.Runtime, o *e.Object, key string) moe.Value {
if o == nil {
return moe.Undefined()
}
v, err := vm.Get(e.ObjectValue(o), key)
if err != nil {
panic(err)
}
return v
}
func Set(vm *moe.Runtime, o *e.Object, key string, x any) error {
v, err := vm.FromGo(x)
if err != nil {
return err
}
return o.SetProp(vm.Realm(), vm.Realm().KeyFromGoString(key), v)
}
func Global(vm *moe.Runtime, name string) moe.Value { return Get(vm, vm.Realm().Global, name) }
func Run(vm *moe.Runtime, src string) (moe.Value, error) { return RunScript(vm, "", src) }
func RunScript(vm *moe.Runtime, name, src string) (moe.Value, error) {
s, err := moe.CompileScript(name, src)
if err != nil {
return moe.Undefined(), err
}
return vm.RunScript(s)
}
func Object(vm *moe.Runtime, v moe.Value) *e.Object {
o, err := vm.Realm().ToObject(v)
if err != nil {
panic(err)
}
return o
}
func String(vm *moe.Runtime, v moe.Value) string {
s, err := vm.Realm().ToString(v)
if err != nil {
panic(err)
}
return s.GoString()
}
func Float(vm *moe.Runtime, v moe.Value) float64 {
n, err := vm.Realm().ToNumber(v)
if err != nil {
panic(err)
}
return n
}
func Int(vm *moe.Runtime, v moe.Value) int64 {
n := Float(vm, v)
if math.IsNaN(n) || n == 0 {
return 0
}
if n >= math.MaxInt64 {
return math.MaxInt64
}
if n <= math.MinInt64 {
return math.MinInt64
}
return int64(n)
}
func Bool(v moe.Value) bool {
switch v.Type() {
case e.TypeNull, e.TypeUndefined:
return false
case e.TypeBoolean:
return v.AsBool()
case e.TypeNumber:
n := v.AsNumber()
return n != 0 && !math.IsNaN(n)
case e.TypeString:
return v.AsString().Len() != 0
case e.TypeBigInt:
return !v.AsBigInt().IsZero()
default:
return true
}
}
func Export(vm *moe.Runtime, v moe.Value) any {
x, err := vm.ToGo(v)
if err != nil {
panic(err)
}
return x
}
func ExportTo(vm *moe.Runtime, v moe.Value, target any) error {
if p, ok := target.(*[]byte); ok && v.IsObject() {
if b, ok := v.AsObject().BufferData(); ok {
*p = b
return nil
}
}
return vm.ToGoInto(v, target)
}
func Call(vm *moe.Runtime, fn, this moe.Value, args ...moe.Value) (moe.Value, error) {
return vm.CallFunction(fn, this, args...)
}
func New(vm *moe.Runtime, ctor moe.Value, args ...moe.Value) (*e.Object, error) {
r := vm.Realm()
r.HoldJobs()
v, err := r.Construct(ctor, args, nil)
err = r.ReleaseJobs(err)
if err != nil {
return nil, err
}
if !v.IsObject() {
return nil, fmt.Errorf("constructor returned non-object")
}
return v.AsObject(), nil
}
func TypeError(vm *moe.Runtime, format string, args ...any) error {
return vm.Realm().TypeError(format, args...)
}
func GoError(vm *moe.Runtime, err error) moe.Value {
v, ok := vm.ThrownValue(err)
if !ok {
return moe.Undefined()
}
return v
}
func Throw(vm *moe.Runtime, err error) moe.Value { panic(err) }
// Bind installs an explicit native signature, recovering only errors thrown by
// host operations. This is not an automatic Go function/argument adapter.
func Bind(vm *moe.Runtime, name string, arity int, fn moe.NativeFunc) moe.Value {
wrapped := func(r *moe.Realm, this moe.Value, args []moe.Value) (out moe.Value, err error) {
defer func() {
if p := recover(); p != nil {
switch x := p.(type) {
case error:
err = x
case moe.Value:
err = &e.Exception{Value: x}
default:
panic(p)
}
}
}()
return fn(r, this, args)
}
return e.ObjectValue(vm.Realm().NewNativeFunction(e.FromGoString(name), arity, wrapped))
}
func Function(vm *moe.Runtime, o *e.Object, name string, fn moe.NativeFunc) error {
return Set(vm, o, name, Bind(vm, name, 0, fn))
}
+18
View File
@@ -0,0 +1,18 @@
package native
import (
"errors"
moe "github.com/Calcium-Ion/moejs"
"testing"
)
func TestGoErrorNative(t *testing.T) {
rt := moe.NewRuntime(moe.Options{})
v := GoError(rt, errors.New("sentinel"))
if !v.IsObject() {
t.Fatalf("error value %v", v)
}
if String(rt, Get(rt, v.AsObject(), "message")) != "sentinel" {
t.Fatal("message lost")
}
}
+23
View File
@@ -0,0 +1,23 @@
package jsruntime
import (
"io"
"testing"
"time"
)
func TestNativeTypedArrayBoundaries(t *testing.T) {
rt, err := New(`function run(){
const c=require("crypto");
for(const C of [Float32Array,Float64Array]){let rejected=false;try{c.getRandomValues(new C(4))}catch(e){rejected=e instanceof TypeError}if(!rejected)throw new Error("float accepted")}
const v=new Uint16Array(4);if(c.getRandomValues(v)!==v)throw new Error("identity");
const b=new Uint8Array([1,2,3,4]);c.randomFillSync(new DataView(b.buffer,1,2));if(b[0]!==1||b[3]!==4)throw new Error("view bounds");
return true;}`, Options{NodeJS: true, BaseDir: t.TempDir(), Console: io.Discard, Timeout: time.Second})
if err != nil {
t.Fatal(err)
}
defer rt.Close()
if err := rt.Call("run"); err != nil {
t.Fatal(err)
}
}
+50
View File
@@ -0,0 +1,50 @@
package jsruntime
import (
"io"
"strings"
"testing"
"time"
)
func TestNativeLookupAndPromiseGetterDeadline(t *testing.T) {
for _, tc := range []struct {
name, source string
check bool
}{
{"hasFunction", `Object.defineProperty(globalThis,"bad",{get(){while(true){}}}); function healthy(){return true}`, true},
{"callLookup", `Object.defineProperty(globalThis,"bad",{get(){while(true){}}}); function healthy(){return true}`, false},
{"promiseThen", `function bad(){const p=Promise.resolve(true);Object.defineProperty(p,"then",{get(){while(true){}}});return p} function healthy(){return true}`, false},
} {
t.Run(tc.name, func(t *testing.T) {
rt, err := New(tc.source, Options{BaseDir: t.TempDir(), Console: io.Discard, Timeout: 50 * time.Millisecond})
if err != nil {
t.Fatal(err)
}
defer rt.Close()
done := make(chan error, 1)
go func() {
if tc.check {
if rt.HasFunction("bad") {
done <- io.ErrUnexpectedEOF
return
}
done <- nil
} else {
done <- rt.Call("bad")
}
}()
select {
case err := <-done:
if !tc.check && (err == nil || !strings.Contains(err.Error(), "timeout")) {
t.Fatalf("expected timeout: %v", err)
}
case <-time.After(2 * time.Second):
t.Fatal("lookup escaped deadline")
}
if err := rt.Call("healthy"); err != nil {
t.Fatalf("runtime not reusable: %v", err)
}
})
}
}
+270 -233
View File
@@ -1,9 +1,18 @@
package net
import (
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
import (
"context"
"errors"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/writequeue"
"io"
"net"
"strconv"
@@ -11,12 +20,6 @@ import (
"sync"
"sync/atomic"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/writequeue"
)
type Module struct {
@@ -24,47 +27,59 @@ type Module struct {
allowListen bool
sockets sync.Map
}
type socketState struct {
connection net.Conn
resourceID uint64
readerDone chan struct{}
detaching atomic.Bool
closed atomic.Bool
readerDone chan struct {
}
detaching atomic.Bool
closed atomic.Bool
}
func New(runtime *bridge.Runtime, allowListen bool) *Module {
return &Module{runtime: runtime, allowListen: allowListen}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
exports := vm.NewObject()
_ = exports.Set("createServer", func(call goja.FunctionCall) goja.Value {
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
exports := vm.Realm().NewObject()
_ = native.Set(vm, exports, "createServer", native.Bind(vm, "createServer", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
server := m.newNetServer(vm)
listener := call.Argument(0)
if _, ok := goja.AssertFunction(listener); !ok {
listener = call.Argument(1)
listener := moejs.Arg(args, 0)
if _, ok := callable(listener); !ok {
listener = moejs.Arg(args, 1)
}
if _, ok := goja.AssertFunction(listener); ok {
on, _ := goja.AssertFunction(server.Get("on"))
_, _ = on(server, vm.ToValue("connection"), listener)
if _, ok := callable(listener); ok {
on, _ := callable(native.Get(vm, server, "on"))
_, _ = native.Call(vm, on, native.From(vm, server), native.From(vm, "connection"), native.From(vm, listener))
}
return server
})
connect := func(call goja.FunctionCall) goja.Value { return m.netConnect(vm, call) }
_ = exports.Set("connect", connect)
_ = exports.Set("createConnection", connect)
_ = exports.Set("isIP", parseIPVersion)
_ = exports.Set("isIPv4", func(value string) bool { ip := net.ParseIP(value); return ip != nil && ip.To4() != nil })
_ = exports.Set("isIPv6", func(value string) bool { ip := net.ParseIP(value); return ip != nil && ip.To4() == nil })
_ = exports.Set("getDefaultAutoSelectFamily", func() bool { return true })
_ = exports.Set("setDefaultAutoSelectFamily", func(bool) {
panic(vm.NewGoError(fmt.Errorf("net.setDefaultAutoSelectFamily is not supported by jsruntime; autoSelectFamily is always enabled")))
})
_ = module.Set("exports", exports)
return native.From(vm, server), nil
}))
connect := func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, m.netConnect(vm, args)), nil
}
_ = native.Set(vm, exports, "connect", native.Bind(vm, "connect", 0, connect))
_ = native.Set(vm, exports, "createConnection", native.Bind(vm, "connect", 0, connect))
_ = native.Set(vm, exports, "isIP", native.Bind(vm, "isIP", 1, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, parseIPVersion(native.String(vm, moejs.Arg(args, 0)))), nil
}))
_ = native.Set(vm, exports, "isIPv4", native.Bind(vm, "isIPv4", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := native.String(vm, moejs.Arg(args, 0))
ip := net.ParseIP(value)
return native.From(vm, ip != nil && ip.To4() != nil), nil
}))
_ = native.Set(vm, exports, "isIPv6", native.Bind(vm, "isIPv6", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := native.String(vm, moejs.Arg(args, 0))
ip := net.ParseIP(value)
return native.From(vm, ip != nil && ip.To4() == nil), nil
}))
_ = native.Set(vm, exports, "getDefaultAutoSelectFamily", native.Bind(vm, "getDefaultAutoSelectFamily", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, true), nil
}))
_ = native.Set(vm, exports, "setDefaultAutoSelectFamily", native.Bind(vm, "setDefaultAutoSelectFamily", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.setDefaultAutoSelectFamily is not supported by jsruntime; autoSelectFamily is always enabled")))
}))
_ = native.Set(vm, module, "exports", exports)
}
func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
func (m *Module) newNetServer(vm *moejs.Runtime) *engine.Object {
server := events.NewEmitter(vm)
var listener net.Listener
var resourceID uint64
@@ -72,17 +87,17 @@ func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
var listenGeneration uint64
var listenPending bool
var connections atomic.Int64
_ = server.Set("listening", false)
_ = server.Set("maxConnections", 0)
_ = server.Set("listen", func(call goja.FunctionCall) goja.Value {
_ = native.Set(vm, server, "listening", false)
_ = native.Set(vm, server, "maxConnections", 0)
_ = native.Set(vm, server, "listen", native.Bind(vm, "listen", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if !m.allowListen {
panic(vm.NewGoError(fmt.Errorf("server.listen requires AllowListen")))
panic(native.GoError(vm, fmt.Errorf("server.listen requires AllowListen")))
}
address, callback := netListenArguments(vm, call)
address, callback := netListenArguments(vm, args)
serverMu.Lock()
if listenPending || listener != nil {
serverMu.Unlock()
panic(vm.NewGoError(fmt.Errorf("server is already listening")))
panic(native.GoError(vm, fmt.Errorf("server is already listening")))
}
listenGeneration++
generation := listenGeneration
@@ -98,9 +113,9 @@ func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
}
serverMu.Unlock()
if active {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Server error", func() error {
return events.Emit(vm, server, "error", vm.NewGoError(err))
return events.Emit(vm, server, "error", native.GoError(vm, err))
})
})
}
@@ -112,7 +127,9 @@ func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
_ = created.Close()
return
}
id := m.runtime.AddResource(func() { _ = created.Close() })
id := m.runtime.AddResource(func() {
_ = created.Close()
})
if id == 0 {
listenPending = false
serverMu.Unlock()
@@ -122,19 +139,19 @@ func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
resourceID = id
listenPending = false
serverMu.Unlock()
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
serverMu.RLock()
active := listenGeneration == generation && listener == created
serverMu.RUnlock()
if !active {
return
}
if callback != nil {
once, _ := goja.AssertFunction(server.Get("once"))
_, _ = once(server, vm.ToValue("listening"), vm.ToValue(callback))
if !native.IsNil(callback) {
once, _ := callable(native.Get(vm, server, "once"))
_, _ = native.Call(vm, once, native.From(vm, server), native.From(vm, "listening"), native.From(vm, callback))
}
_ = m.runtime.RunJob(vm, "net.Server listening", func() error {
_ = server.Set("listening", true)
_ = native.Set(vm, server, "listening", true)
return events.Emit(vm, server, "listening")
})
})
@@ -145,21 +162,25 @@ func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
active := listenGeneration == generation && listener == created
serverMu.RUnlock()
if active {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Server error", func() error {
return events.Emit(vm, server, "error", vm.NewGoError(acceptErr))
return events.Emit(vm, server, "error", native.GoError(vm, acceptErr))
})
})
}
return
}
connections.Add(1)
if !m.runtime.RunOnLoop(func(vm *goja.Runtime) {
socket := m.newNetSocket(vm, connection, func() { connections.Add(-1) })
if socket.Get("destroyed").ToBoolean() {
if !m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
socket := m.newNetSocket(vm, connection, func() {
connections.Add(-1)
})
if native.Bool(native.Get(vm, socket, "destroyed")) {
return
}
_ = m.runtime.RunJob(vm, "net.Server connection", func() error { return events.Emit(vm, server, "connection", socket) })
_ = m.runtime.RunJob(vm, "net.Server connection", func() error {
return events.Emit(vm, server, "connection", socket)
})
}) {
connections.Add(-1)
_ = connection.Close()
@@ -167,12 +188,12 @@ func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
}
}
}()
return server
})
_ = server.Set("close", func(call goja.FunctionCall) goja.Value {
if callback, ok := goja.AssertFunction(call.Argument(0)); ok {
once, _ := goja.AssertFunction(server.Get("once"))
_, _ = once(server, vm.ToValue("close"), vm.ToValue(callback))
return native.From(vm, server), nil
}))
_ = native.Set(vm, server, "close", native.Bind(vm, "close", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if callback, ok := callable(moejs.Arg(args, 0)); ok {
once, _ := callable(native.Get(vm, server, "once"))
_, _ = native.Call(vm, once, native.From(vm, server), native.From(vm, "close"), native.From(vm, callback))
}
serverMu.Lock()
wasActive := listenPending || listener != nil
@@ -185,125 +206,135 @@ func (m *Module) newNetServer(vm *goja.Runtime) *goja.Object {
_ = currentListener.Close()
m.runtime.RemoveResource(currentResourceID)
}
_ = server.Set("listening", false)
_ = native.Set(vm, server, "listening", false)
if wasActive {
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
_ = m.runtime.RunJob(vm, "net.Server close", func() error { return events.Emit(vm, server, "close") })
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Server close", func() error {
return events.Emit(vm, server, "close")
})
}, 0)
}
return server
})
_ = server.Set("closeAllConnections", func() {
panic(vm.NewGoError(fmt.Errorf("net.Server.closeAllConnections is not supported by jsruntime; connections are not tracked individually")))
})
_ = server.Set("closeIdleConnections", func() {
panic(vm.NewGoError(fmt.Errorf("net.Server.closeIdleConnections is not supported by jsruntime; connections are not tracked individually")))
})
_ = server.Set("address", func() goja.Value {
return native.From(vm, server), nil
}))
_ = native.Set(vm, server, "closeAllConnections", native.Bind(vm, "closeAllConnections", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Server.closeAllConnections is not supported by jsruntime; connections are not tracked individually")))
}))
_ = native.Set(vm, server, "closeIdleConnections", native.Bind(vm, "closeIdleConnections", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Server.closeIdleConnections is not supported by jsruntime; connections are not tracked individually")))
}))
_ = native.Set(vm, server, "address", native.Bind(vm, "address", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
serverMu.RLock()
currentListener := listener
serverMu.RUnlock()
if currentListener == nil {
return goja.Null()
return native.From(vm, moejs.Null()), nil
}
host, port, _ := net.SplitHostPort(currentListener.Addr().String())
portNumber, _ := strconv.Atoi(port)
return vm.ToValue(map[string]any{"address": host, "family": netAddressFamily(host), "port": portNumber})
})
_ = server.Set("getConnections", func(callback goja.Callable) {
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
return native.From(vm, map[string]any{"address": host, "family": netAddressFamily(host), "port": portNumber}), nil
}))
_ = native.Set(vm, server, "getConnections", native.Bind(vm, "getConnections", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
callback := moejs.Arg(args, 0)
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Server.getConnections", func() error {
_, err := callback(goja.Undefined(), goja.Null(), vm.ToValue(connections.Load()))
_, err := native.Call(vm, callback, native.From(vm, moejs.Undefined()), native.From(vm, moejs.Null()), native.From(vm, connections.Load()))
return err
})
}, 0)
})
_ = server.Set("ref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Server.ref is not supported by jsruntime; the event loop is host-driven")))
})
_ = server.Set("unref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Server.unref is not supported by jsruntime; the event loop is host-driven")))
})
return moejs.Undefined(), nil
}))
_ = native.Set(vm, server, "ref", native.Bind(vm, "ref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Server.ref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, server, "unref", native.Bind(vm, "unref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Server.unref is not supported by jsruntime; the event loop is host-driven")))
}))
return server
}
func netListenArguments(vm *goja.Runtime, call goja.FunctionCall) (string, goja.Callable) {
func netListenArguments(vm *moejs.Runtime, args []moejs.Value) (string, moejs.Value) {
host := "127.0.0.1"
port := int(call.Argument(0).ToInteger())
port := int(native.Int(vm, moejs.Arg(args, 0)))
callbackIndex := 1
if object, ok := call.Argument(0).(*goja.Object); ok {
port = int(object.Get("port").ToInteger())
if value := object.Get("host"); !goja.IsUndefined(value) {
host = value.String()
if object, ok := asObject(moejs.Arg(args, 0)); ok {
port = int(native.Int(vm, native.Get(vm, object, "port")))
if value := native.Get(vm, object, "host"); !value.IsUndefined() {
host = native.String(vm, value)
}
callbackIndex = 1
} else if value := call.Argument(1); !goja.IsUndefined(value) {
if _, ok := goja.AssertFunction(value); !ok {
host = value.String()
} else if value := moejs.Arg(args, 1); !value.IsUndefined() {
if _, ok := callable(value); !ok {
host = native.String(vm, value)
callbackIndex = 2
}
}
callback, _ := goja.AssertFunction(call.Argument(callbackIndex))
callback, _ := callable(moejs.Arg(args, callbackIndex))
return net.JoinHostPort(host, strconv.Itoa(port)), callback
}
func (m *Module) netConnect(vm *goja.Runtime, call goja.FunctionCall) *goja.Object {
func (m *Module) netConnect(vm *moejs.Runtime, args []moejs.Value) *engine.Object {
host := "127.0.0.1"
port := int(call.Argument(0).ToInteger())
port := int(native.Int(vm, moejs.Arg(args, 0)))
callbackIndex := 1
if object, ok := call.Argument(0).(*goja.Object); ok {
port = int(object.Get("port").ToInteger())
if value := object.Get("host"); !goja.IsUndefined(value) {
host = value.String()
if object, ok := asObject(moejs.Arg(args, 0)); ok {
port = int(native.Int(vm, native.Get(vm, object, "port")))
if value := native.Get(vm, object, "host"); !value.IsUndefined() {
host = native.String(vm, value)
}
} else if _, ok := goja.AssertFunction(call.Argument(1)); !ok && !goja.IsUndefined(call.Argument(1)) {
host = call.Argument(1).String()
} else if _, ok := callable(moejs.Arg(args, 1)); !ok && !moejs.Arg(args, 1).IsUndefined() {
host = native.String(vm, moejs.Arg(args, 1))
callbackIndex = 2
}
placeholder := events.NewEmitter(vm)
_ = placeholder.Set("connecting", true)
_ = native.Set(vm, placeholder, "connecting", true)
var pendingEncoding string
_ = placeholder.Set("setEncoding", func(value string) *goja.Object { pendingEncoding = value; return placeholder })
_ = placeholder.Set("setTimeout", func(milliseconds int64, callback goja.Value) *goja.Object {
if function, ok := goja.AssertFunction(callback); ok {
once, _ := goja.AssertFunction(placeholder.Get("once"))
_, _ = once(placeholder, vm.ToValue("timeout"), vm.ToValue(function))
_ = native.Set(vm, placeholder, "setEncoding", native.Bind(vm, "setEncoding", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := native.String(vm, moejs.Arg(args, 0))
pendingEncoding = value
return native.From(vm, placeholder), nil
}))
_ = native.Set(vm, placeholder, "setTimeout", native.Bind(vm, "setTimeout", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
milliseconds := native.Int(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
if function, ok := callable(callback); ok {
once, _ := callable(native.Get(vm, placeholder, "once"))
_, _ = native.Call(vm, once, native.From(vm, placeholder), native.From(vm, "timeout"), native.From(vm, function))
}
if milliseconds > 0 {
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
_ = m.runtime.RunJob(vm, "net.Socket timeout", func() error { return events.Emit(vm, placeholder, "timeout") })
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Socket timeout", func() error {
return events.Emit(vm, placeholder, "timeout")
})
}, time.Duration(milliseconds)*time.Millisecond)
}
return placeholder
})
_ = placeholder.Set("setNoDelay", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.setNoDelay is not supported by jsruntime before the connection is established")))
})
_ = placeholder.Set("setKeepAlive", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.setKeepAlive is not supported by jsruntime before the connection is established")))
})
_ = placeholder.Set("ref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.ref is not supported by jsruntime; the event loop is host-driven")))
})
_ = placeholder.Set("unref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.unref is not supported by jsruntime; the event loop is host-driven")))
})
if callback, ok := goja.AssertFunction(call.Argument(callbackIndex)); ok {
once, _ := goja.AssertFunction(placeholder.Get("once"))
_, _ = once(placeholder, vm.ToValue("connect"), vm.ToValue(callback))
return native.From(vm, placeholder), nil
}))
_ = native.Set(vm, placeholder, "setNoDelay", native.Bind(vm, "setNoDelay", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.setNoDelay is not supported by jsruntime before the connection is established")))
}))
_ = native.Set(vm, placeholder, "setKeepAlive", native.Bind(vm, "setKeepAlive", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.setKeepAlive is not supported by jsruntime before the connection is established")))
}))
_ = native.Set(vm, placeholder, "ref", native.Bind(vm, "ref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.ref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, placeholder, "unref", native.Bind(vm, "unref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.unref is not supported by jsruntime; the event loop is host-driven")))
}))
if callback, ok := callable(moejs.Arg(args, callbackIndex)); ok {
once, _ := callable(native.Get(vm, placeholder, "once"))
_, _ = native.Call(vm, once, native.From(vm, placeholder), native.From(vm, "connect"), native.From(vm, callback))
}
dialContext, cancelDial := context.WithCancel(context.Background())
pendingResourceID := m.runtime.AddResource(cancelDial)
if pendingResourceID == 0 {
_ = placeholder.Set("connecting", false)
_ = placeholder.Set("destroyed", true)
_ = native.Set(vm, placeholder, "connecting", false)
_ = native.Set(vm, placeholder, "destroyed", true)
return placeholder
}
go func() {
connection, err := (&net.Dialer{Timeout: m.runtime.Timeout()}).DialContext(dialContext, "tcp", net.JoinHostPort(host, strconv.Itoa(port)))
cancelDial()
m.runtime.RemoveResource(pendingResourceID)
queued := m.runtime.RunOnLoop(func(vm *goja.Runtime) {
queued := m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
attached := false
defer func() {
if !attached && connection != nil {
@@ -312,19 +343,19 @@ func (m *Module) netConnect(vm *goja.Runtime, call goja.FunctionCall) *goja.Obje
}()
_ = m.runtime.RunJob(vm, "net.Socket connect", func() error {
if err != nil {
_ = placeholder.Set("connecting", false)
return events.Emit(vm, placeholder, "error", vm.NewGoError(err))
_ = native.Set(vm, placeholder, "connecting", false)
return events.Emit(vm, placeholder, "error", native.GoError(vm, err))
}
if !m.configureNetSocket(vm, placeholder, connection, nil) {
return nil
}
attached = true
if pendingEncoding != "" {
if setEncoding, ok := goja.AssertFunction(placeholder.Get("setEncoding")); ok {
_, _ = setEncoding(placeholder, vm.ToValue(pendingEncoding))
if setEncoding, ok := callable(native.Get(vm, placeholder, "setEncoding")); ok {
_, _ = native.Call(vm, setEncoding, native.From(vm, placeholder), native.From(vm, pendingEncoding))
}
}
_ = placeholder.Set("connecting", false)
_ = native.Set(vm, placeholder, "connecting", false)
return events.Emit(vm, placeholder, "connect")
})
})
@@ -334,26 +365,20 @@ func (m *Module) netConnect(vm *goja.Runtime, call goja.FunctionCall) *goja.Obje
}()
return placeholder
}
func (m *Module) newNetSocket(vm *goja.Runtime, connection net.Conn, onClose func()) *goja.Object {
func (m *Module) newNetSocket(vm *moejs.Runtime, connection net.Conn, onClose func()) *engine.Object {
socket := events.NewEmitter(vm)
if !m.configureNetSocket(vm, socket, connection, onClose) {
_ = socket.Set("destroyed", true)
_ = socket.Set("readyState", "closed")
_ = native.Set(vm, socket, "destroyed", true)
_ = native.Set(vm, socket, "readyState", "closed")
}
return socket
}
// AttachSocket configures a JavaScript socket around an established
// connection. It is shared with the TLS module after a handshake completes.
func (m *Module) AttachSocket(vm *goja.Runtime, socket *goja.Object, connection net.Conn, onClose func()) bool {
func (m *Module) AttachSocket(vm *moejs.Runtime, socket *engine.Object, connection net.Conn, onClose func()) bool {
return m.configureNetSocket(vm, socket, connection, onClose)
}
// DetachSocket stops a socket's reader and transfers ownership of its
// connection to the caller. It is used to upgrade an SMTP connection with
// STARTTLS without closing the underlying TCP stream.
func (m *Module) DetachSocket(socket *goja.Object) (net.Conn, error) {
func (m *Module) DetachSocket(socket *engine.Object) (net.Conn, error) {
value, ok := m.sockets.Load(socket)
if !ok {
return nil, errors.New("socket is not connected")
@@ -378,21 +403,22 @@ func (m *Module) DetachSocket(socket *goja.Object) (net.Conn, error) {
}
return state.connection, nil
}
func (m *Module) configureNetSocket(vm *goja.Runtime, socket *goja.Object, connection net.Conn, onClose func()) bool {
func (m *Module) configureNetSocket(vm *moejs.Runtime, socket *engine.Object, connection net.Conn, onClose func()) bool {
localHost, localPort := splitNetAddress(connection.LocalAddr())
remoteHost, remotePort := splitNetAddress(connection.RemoteAddr())
_ = socket.Set("connecting", false)
_ = socket.Set("pending", false)
_ = socket.Set("destroyed", false)
_ = socket.Set("readyState", "open")
_ = socket.Set("remoteAddress", remoteHost)
_ = socket.Set("remotePort", remotePort)
_ = socket.Set("remoteFamily", netAddressFamily(remoteHost))
_ = socket.Set("localAddress", localHost)
_ = socket.Set("localPort", localPort)
_ = socket.Set("localFamily", netAddressFamily(localHost))
resourceID := m.runtime.AddResource(func() { _ = connection.Close() })
_ = native.Set(vm, socket, "connecting", false)
_ = native.Set(vm, socket, "pending", false)
_ = native.Set(vm, socket, "destroyed", false)
_ = native.Set(vm, socket, "readyState", "open")
_ = native.Set(vm, socket, "remoteAddress", remoteHost)
_ = native.Set(vm, socket, "remotePort", remotePort)
_ = native.Set(vm, socket, "remoteFamily", netAddressFamily(remoteHost))
_ = native.Set(vm, socket, "localAddress", localHost)
_ = native.Set(vm, socket, "localPort", localPort)
_ = native.Set(vm, socket, "localFamily", netAddressFamily(localHost))
resourceID := m.runtime.AddResource(func() {
_ = connection.Close()
})
if resourceID == 0 {
_ = connection.Close()
if onClose != nil {
@@ -400,11 +426,8 @@ func (m *Module) configureNetSocket(vm *goja.Runtime, socket *goja.Object, conne
}
return false
}
state := &socketState{
connection: connection,
resourceID: resourceID,
readerDone: make(chan struct{}),
}
state := &socketState{connection: connection, resourceID: resourceID, readerDone: make(chan struct {
})}
m.sockets.Store(socket, state)
var encoding atomic.Value
encoding.Store("")
@@ -419,33 +442,33 @@ func (m *Module) configureNetSocket(vm *goja.Runtime, socket *goja.Object, conne
}
}
}
_ = socket.Set("write", func(call goja.FunctionCall) goja.Value {
data := append([]byte(nil), buffer.Bytes(vm, call.Argument(0))...)
callback := netCallback(call)
_ = native.Set(vm, socket, "write", native.Bind(vm, "write", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
data := append([]byte(nil), buffer.Bytes(vm, moejs.Arg(args, 0))...)
callback := netCallback(args)
writes.Submit(func() {
_, err := connection.Write(data)
if callback != nil {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
if !native.IsNil(callback) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Socket.write", func() error {
if err != nil {
_, callbackErr := callback(goja.Undefined(), vm.NewGoError(err))
_, callbackErr := native.Call(vm, callback, native.From(vm, moejs.Undefined()), native.From(vm, native.GoError(vm, err)))
return callbackErr
}
_, callbackErr := callback(goja.Undefined())
_, callbackErr := native.Call(vm, callback, native.From(vm, moejs.Undefined()))
return callbackErr
})
})
}
})
return vm.ToValue(true)
})
_ = socket.Set("end", func(call goja.FunctionCall) goja.Value {
return native.From(vm, true), nil
}))
_ = native.Set(vm, socket, "end", native.Bind(vm, "end", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
var data []byte
_, firstArgumentIsCallback := goja.AssertFunction(call.Argument(0))
if len(call.Arguments) > 0 && !firstArgumentIsCallback && !goja.IsUndefined(call.Argument(0)) && !goja.IsNull(call.Argument(0)) {
data = append([]byte(nil), buffer.Bytes(vm, call.Argument(0))...)
_, firstArgumentIsCallback := callable(moejs.Arg(args, 0))
if len(args) > 0 && !firstArgumentIsCallback && !moejs.Arg(args, 0).IsUndefined() && !moejs.Arg(args, 0).IsNull() {
data = append([]byte(nil), buffer.Bytes(vm, moejs.Arg(args, 0))...)
}
callback := netCallback(call)
callback := netCallback(args)
writes.Submit(func() {
var err error
if len(data) > 0 {
@@ -461,62 +484,73 @@ func (m *Module) configureNetSocket(vm *goja.Runtime, socket *goja.Object, conne
if err != nil {
closeSocket()
}
if callback != nil {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
if !native.IsNil(callback) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Socket.end", func() error {
if err != nil {
_, callbackErr := callback(goja.Undefined(), vm.NewGoError(err))
_, callbackErr := native.Call(vm, callback, native.From(vm, moejs.Undefined()), native.From(vm, native.GoError(vm, err)))
return callbackErr
}
_, callbackErr := callback(goja.Undefined())
_, callbackErr := native.Call(vm, callback, native.From(vm, moejs.Undefined()))
return callbackErr
})
})
}
})
return socket
})
_ = socket.Set("destroy", func() *goja.Object { closeSocket(); return socket })
_ = socket.Set("setEncoding", func(value string) *goja.Object { encoding.Store(value); return socket })
_ = socket.Set("setTimeout", func(milliseconds int64, callback goja.Value) *goja.Object {
return native.From(vm, socket), nil
}))
_ = native.Set(vm, socket, "destroy", native.Bind(vm, "destroy", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
closeSocket()
return native.From(vm, socket), nil
}))
_ = native.Set(vm, socket, "setEncoding", native.Bind(vm, "setEncoding", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := native.String(vm, moejs.Arg(args, 0))
encoding.Store(value)
return native.From(vm, socket), nil
}))
_ = native.Set(vm, socket, "setTimeout", native.Bind(vm, "setTimeout", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
milliseconds := native.Int(vm, moejs.Arg(args, 0))
callback := moejs.Arg(args, 1)
if milliseconds <= 0 {
_ = connection.SetDeadline(time.Time{})
} else {
_ = connection.SetDeadline(time.Now().Add(time.Duration(milliseconds) * time.Millisecond))
}
if function, ok := goja.AssertFunction(callback); ok {
once, _ := goja.AssertFunction(socket.Get("once"))
_, _ = once(socket, vm.ToValue("timeout"), vm.ToValue(function))
if function, ok := callable(callback); ok {
once, _ := callable(native.Get(vm, socket, "once"))
_, _ = native.Call(vm, once, native.From(vm, socket), native.From(vm, "timeout"), native.From(vm, function))
}
return socket
})
_ = socket.Set("setNoDelay", func(enabled bool) *goja.Object {
return native.From(vm, socket), nil
}))
_ = native.Set(vm, socket, "setNoDelay", native.Bind(vm, "setNoDelay", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
enabled := native.Bool(moejs.Arg(args, 0))
if tcp, ok := connection.(*net.TCPConn); ok {
_ = tcp.SetNoDelay(enabled)
}
return socket
})
_ = socket.Set("setKeepAlive", func(enabled bool) *goja.Object {
return native.From(vm, socket), nil
}))
_ = native.Set(vm, socket, "setKeepAlive", native.Bind(vm, "setKeepAlive", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
enabled := native.Bool(moejs.Arg(args, 0))
if tcp, ok := connection.(*net.TCPConn); ok {
_ = tcp.SetKeepAlive(enabled)
}
return socket
})
_ = socket.Set("address", func() map[string]any {
return map[string]any{"address": localHost, "family": netAddressFamily(localHost), "port": localPort}
})
_ = socket.Set("pause", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.pause is not supported by jsruntime; no backpressure")))
})
_ = socket.Set("resume", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.resume is not supported by jsruntime; no backpressure")))
})
_ = socket.Set("ref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.ref is not supported by jsruntime; the event loop is host-driven")))
})
_ = socket.Set("unref", func() *goja.Object {
panic(vm.NewGoError(fmt.Errorf("net.Socket.unref is not supported by jsruntime; the event loop is host-driven")))
})
return native.From(vm, socket), nil
}))
_ = native.Set(vm, socket, "address", native.Bind(vm, "address", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, map[string]any{"address": localHost, "family": netAddressFamily(localHost), "port": localPort}), nil
}))
_ = native.Set(vm, socket, "pause", native.Bind(vm, "pause", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.pause is not supported by jsruntime; no backpressure")))
}))
_ = native.Set(vm, socket, "resume", native.Bind(vm, "resume", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.resume is not supported by jsruntime; no backpressure")))
}))
_ = native.Set(vm, socket, "ref", native.Bind(vm, "ref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.ref is not supported by jsruntime; the event loop is host-driven")))
}))
_ = native.Set(vm, socket, "unref", native.Bind(vm, "unref", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("net.Socket.unref is not supported by jsruntime; the event loop is host-driven")))
}))
go func() {
defer close(state.readerDone)
data := make([]byte, 32*1024)
@@ -524,13 +558,14 @@ func (m *Module) configureNetSocket(vm *goja.Runtime, socket *goja.Object, conne
count, err := connection.Read(data)
if count > 0 {
chunk := append([]byte(nil), data[:count]...)
delivered := make(chan struct{})
if !m.runtime.RunOnLoop(func(vm *goja.Runtime) {
delivered := make(chan struct {
})
if !m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
defer close(delivered)
_ = m.runtime.RunJob(vm, "net.Socket data", func() error {
var value any = buffer.WrapBytes(vm, chunk)
if currentEncoding := encoding.Load().(string); currentEncoding != "" {
value = buffer.EncodeBytes(vm, chunk, vm.ToValue(currentEncoding))
value = buffer.EncodeBytes(vm, chunk, native.From(vm, currentEncoding))
}
return events.Emit(vm, socket, "data", value)
})
@@ -545,12 +580,12 @@ func (m *Module) configureNetSocket(vm *goja.Runtime, socket *goja.Object, conne
return
}
closeSocket()
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "net.Socket close", func() error {
_ = socket.Set("destroyed", true)
_ = socket.Set("readyState", "closed")
_ = native.Set(vm, socket, "destroyed", true)
_ = native.Set(vm, socket, "readyState", "closed")
if !errors.Is(err, io.EOF) && !isNetClosed(err) {
if emitErr := events.Emit(vm, socket, "error", vm.NewGoError(err)); emitErr != nil {
if emitErr := events.Emit(vm, socket, "error", native.GoError(vm, err)); emitErr != nil {
return emitErr
}
}
@@ -566,7 +601,6 @@ func (m *Module) configureNetSocket(vm *goja.Runtime, socket *goja.Object, conne
}()
return true
}
func splitNetAddress(address net.Addr) (string, int) {
if address == nil {
return "", 0
@@ -575,27 +609,23 @@ func splitNetAddress(address net.Addr) (string, int) {
portNumber, _ := strconv.Atoi(port)
return host, portNumber
}
func netAddressFamily(host string) string {
if ip := net.ParseIP(host); ip != nil && ip.To4() == nil {
return "IPv6"
}
return "IPv4"
}
func isNetClosed(err error) bool {
return errors.Is(err, net.ErrClosed) || strings.Contains(err.Error(), "closed network connection")
}
func netCallback(call goja.FunctionCall) goja.Callable {
for index := len(call.Arguments) - 1; index >= 0; index-- {
if callback, ok := goja.AssertFunction(call.Arguments[index]); ok {
func netCallback(args []moejs.Value) moejs.Value {
for index := len(args) - 1; index >= 0; index-- {
if callback, ok := callable(args[index]); ok {
return callback
}
}
return nil
return moejs.Undefined()
}
func parseIPVersion(value string) int {
ip := net.ParseIP(value)
if ip == nil {
@@ -606,3 +636,10 @@ func parseIPVersion(value string) int {
}
return 6
}
func callable(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
func asObject(v moejs.Value) (*engine.Object, bool) {
if !v.IsObject() {
return nil, false
}
return v.AsObject(), true
}
+7 -7
View File
@@ -5,10 +5,10 @@ import (
stdos "os"
"path/filepath"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
osmodule "github.com/komari-monitor/komari/pkg/jsruntime/os"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
streammodule "github.com/komari-monitor/komari/pkg/jsruntime/stream"
)
@@ -40,21 +40,21 @@ func (r *Runtime) registerNodeModule(registry *require.Registry, name string, lo
func (r *Runtime) injectNodeGlobals() error {
buffer.Enable(r.vm)
processValue := require.Require(r.vm, "process")
if err := r.vm.Set("process", processValue); err != nil {
if err := r.vm.SetGlobal("process", processValue); err != nil {
return fmt.Errorf("inject process: %w", err)
}
if err := r.vm.Set("global", r.vm.GlobalObject()); err != nil {
if err := r.vm.SetGlobal("global", r.vm.Realm().Global); err != nil {
return fmt.Errorf("inject global: %w", err)
}
cwd := r.fsModule.Cwd()
if err := r.vm.Set("__dirname", cwd); err != nil {
if err := r.vm.SetGlobal("__dirname", cwd); err != nil {
return fmt.Errorf("inject __dirname: %w", err)
}
if err := r.vm.Set("__filename", filepath.Join(cwd, "script.js")); err != nil {
if err := r.vm.SetGlobal("__filename", filepath.Join(cwd, "script.js")); err != nil {
return fmt.Errorf("inject __filename: %w", err)
}
if r.storageDir != "" {
if err := r.vm.Set("__storageDir__", r.storageDir); err != nil {
if err := r.vm.SetGlobal("__storageDir__", r.storageDir); err != nil {
return fmt.Errorf("inject __storageDir__: %w", err)
}
}
+75 -47
View File
@@ -1,80 +1,111 @@
package os
import (
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/metrics"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"net"
"os"
"os/user"
"runtime"
"unsafe"
"github.com/dop251/goja"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/metrics"
)
func Load(vm *goja.Runtime, module *goja.Object) {
exports := vm.NewObject()
func Load(vm *moejs.Runtime, module *engine.Object) {
exports := vm.Realm().NewObject()
hostname, _ := os.Hostname()
home, _ := os.UserHomeDir()
currentUser, _ := user.Current()
systemMetrics := func() metrics.System {
result, err := metrics.ReadSystem()
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
return result
}
_ = exports.Set("arch", func() string { return metrics.Arch() })
_ = exports.Set("platform", func() string { return metrics.Platform() })
_ = exports.Set("type", func() string {
_ = native.Set(vm, exports, "arch", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, metrics.Arch()), nil
}))
_ = native.Set(vm, exports, "platform", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, metrics.Platform()), nil
}))
_ = native.Set(vm, exports, "type", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
switch runtime.GOOS {
case "windows":
return "Windows_NT"
return native.From(vm, "Windows_NT"), nil
case "darwin":
return "Darwin"
return native.From(vm, "Darwin"), nil
case "freebsd":
return "FreeBSD"
return native.From(vm, "FreeBSD"), nil
case "openbsd":
return "OpenBSD"
return native.From(vm, "OpenBSD"), nil
case "netbsd":
return "NetBSD"
return native.From(vm, "NetBSD"), nil
case "aix":
return "AIX"
return native.From(vm, "AIX"), nil
default:
return runtime.GOOS
return native.From(vm, runtime.GOOS), nil
}
})
_ = exports.Set("release", func() string { return systemMetrics().Release })
_ = exports.Set("version", func() string { return systemMetrics().Version })
_ = exports.Set("machine", func() string { return runtime.GOARCH })
_ = exports.Set("hostname", func() string { return hostname })
_ = exports.Set("homedir", func() string { return home })
_ = exports.Set("tmpdir", os.TempDir)
_ = exports.Set("endianness", func() string {
}))
_ = native.Set(vm, exports, "release", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, systemMetrics().Release), nil
}))
_ = native.Set(vm, exports, "version", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, systemMetrics().Version), nil
}))
_ = native.Set(vm, exports, "machine", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, runtime.GOARCH), nil
}))
_ = native.Set(vm, exports, "hostname", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, hostname), nil
}))
_ = native.Set(vm, exports, "homedir", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, home), nil
}))
_ = native.Set(vm, exports, "tmpdir", native.Bind(vm, "tmpdir", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, os.TempDir()), nil
}))
_ = native.Set(vm, exports, "endianness", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
var value uint16 = 1
if *(*byte)(unsafe.Pointer(&value)) == 1 {
return "LE"
return native.From(vm, "LE"), nil
}
return "BE"
})
_ = exports.Set("EOL", map[bool]string{true: "\r\n", false: "\n"}[runtime.GOOS == "windows"])
_ = exports.Set("devNull", map[bool]string{true: `\\.\nul`, false: "/dev/null"}[runtime.GOOS == "windows"])
_ = exports.Set("uptime", func() float64 { return systemMetrics().Uptime })
_ = exports.Set("loadavg", func() []float64 { result := systemMetrics(); return result.LoadAvg[:] })
_ = exports.Set("totalmem", func() uint64 { return systemMetrics().TotalMem })
_ = exports.Set("freemem", func() uint64 { return systemMetrics().FreeMem })
_ = exports.Set("availableParallelism", runtime.NumCPU)
_ = exports.Set("cpus", func() []map[string]any { return metrics.CPUInfoValues(systemMetrics().CPUs) })
_ = exports.Set("userInfo", func() map[string]any {
return native.From(vm, "BE"), nil
}))
_ = native.Set(vm, exports, "EOL", map[bool]string{true: "\r\n", false: "\n"}[runtime.GOOS == "windows"])
_ = native.Set(vm, exports, "devNull", map[bool]string{true: `\\.\nul`, false: "/dev/null"}[runtime.GOOS == "windows"])
_ = native.Set(vm, exports, "uptime", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, systemMetrics().Uptime), nil
}))
_ = native.Set(vm, exports, "loadavg", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
result := systemMetrics()
return native.From(vm, result.LoadAvg[:]), nil
}))
_ = native.Set(vm, exports, "totalmem", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, systemMetrics().TotalMem), nil
}))
_ = native.Set(vm, exports, "freemem", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, systemMetrics().FreeMem), nil
}))
_ = native.Set(vm, exports, "availableParallelism", native.Bind(vm, "availableParallelism", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, runtime.NumCPU()), nil
}))
_ = native.Set(vm, exports, "cpus", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, metrics.CPUInfoValues(systemMetrics().CPUs)), nil
}))
_ = native.Set(vm, exports, "userInfo", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if currentUser == nil {
return map[string]any{"uid": -1, "gid": -1, "username": "", "homedir": home, "shell": ""}
return native.From(vm, map[string]any{"uid": -1, "gid": -1, "username": "", "homedir": home, "shell": ""}), nil
}
return map[string]any{"uid": currentUser.Uid, "gid": currentUser.Gid, "username": currentUser.Username, "homedir": currentUser.HomeDir, "shell": ""}
})
_ = exports.Set("networkInterfaces", nodeNetworkInterfaces)
_ = exports.Set("constants", map[string]any{"signals": map[string]int{"SIGINT": 2, "SIGTERM": 15, "SIGKILL": 9}, "errno": map[string]int{}})
_ = module.Set("exports", exports)
return native.From(vm, map[string]any{"uid": currentUser.Uid, "gid": currentUser.Gid, "username": currentUser.Username, "homedir": currentUser.HomeDir, "shell": ""}), nil
}))
_ = native.Set(vm, exports, "networkInterfaces", native.Bind(vm, "networkInterfaces", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, nodeNetworkInterfaces()), nil
}))
_ = native.Set(vm, exports, "constants", map[string]any{"signals": map[string]int{"SIGINT": 2, "SIGTERM": 15, "SIGKILL": 9}, "errno": map[string]int{}})
_ = native.Set(vm, module, "exports", exports)
}
func nodeNetworkInterfaces() map[string][]map[string]any {
result := make(map[string][]map[string]any)
interfaces, _ := net.Interfaces()
@@ -89,10 +120,7 @@ func nodeNetworkInterfaces() map[string][]map[string]any {
if ip.To4() != nil {
family = "IPv4"
}
result[networkInterface.Name] = append(result[networkInterface.Name], map[string]any{
"address": ip.String(), "netmask": net.IP(network.Mask).String(), "family": family,
"mac": networkInterface.HardwareAddr.String(), "internal": networkInterface.Flags&net.FlagLoopback != 0, "cidr": address.String(),
})
result[networkInterface.Name] = append(result[networkInterface.Name], map[string]any{"address": ip.String(), "netmask": net.IP(network.Mask).String(), "family": family, "mac": networkInterface.HardwareAddr.String(), "internal": networkInterface.Flags&net.FlagLoopback != 0, "cidr": address.String()})
}
}
return result
+95 -91
View File
@@ -1,11 +1,12 @@
package path
import (
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"path"
"runtime"
"strings"
"github.com/dop251/goja"
)
type Module struct {
@@ -15,16 +16,14 @@ type Module struct {
func New(cwd func() string) *Module {
return &Module{cwd: cwd}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
defaultPath := m.pathExports(vm, runtime.GOOS == "windows")
_ = defaultPath.Set("posix", m.pathExports(vm, false))
_ = defaultPath.Set("win32", m.pathExports(vm, true))
_ = module.Set("exports", defaultPath)
_ = native.Set(vm, defaultPath, "posix", m.pathExports(vm, false))
_ = native.Set(vm, defaultPath, "win32", m.pathExports(vm, true))
_ = native.Set(vm, module, "exports", defaultPath)
}
func (m *Module) pathExports(vm *goja.Runtime, windows bool) *goja.Object {
exports := vm.NewObject()
func (m *Module) pathExports(vm *moejs.Runtime, windows bool) *engine.Object {
exports := vm.Realm().NewObject()
separator, delimiter := "/", ":"
if windows {
separator, delimiter = `\`, ";"
@@ -66,93 +65,104 @@ func (m *Module) pathExports(vm *goja.Runtime, windows bool) *goja.Object {
}
return path.IsAbs(value)
}
extname := func(value string) string { return nodeExtname(value, windows) }
_ = exports.Set("sep", separator)
_ = exports.Set("delimiter", delimiter)
_ = exports.Set("normalize", normalize)
_ = exports.Set("isAbsolute", isAbs)
_ = exports.Set("join", func(call goja.FunctionCall) goja.Value {
parts := make([]string, len(call.Arguments))
for index, value := range call.Arguments {
parts[index] = value.String()
extname := func(value string) string {
return nodeExtname(value, windows)
}
_ = native.Set(vm, exports, "sep", separator)
_ = native.Set(vm, exports, "delimiter", delimiter)
_ = native.Set(vm, exports, "normalize", native.Bind(vm, "normalize", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, normalize(native.String(vm, moejs.Arg(args, 0)))), nil
}))
_ = native.Set(vm, exports, "isAbsolute", native.Bind(vm, "isAbsolute", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, isAbs(native.String(vm, moejs.Arg(args, 0)))), nil
}))
_ = native.Set(vm, exports, "join", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
parts := make([]string, len(args))
for index, value := range args {
parts[index] = native.String(vm, value)
}
return vm.ToValue(join(parts))
})
_ = exports.Set("resolve", func(call goja.FunctionCall) goja.Value {
parts := make([]string, len(call.Arguments))
for index, value := range call.Arguments {
parts[index] = value.String()
return native.From(vm, join(parts)), nil
}))
_ = native.Set(vm, exports, "resolve", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
parts := make([]string, len(args))
for index, value := range args {
parts[index] = native.String(vm, value)
}
return vm.ToValue(resolve(parts))
})
_ = exports.Set("relative", func(from, to string) string {
return native.From(vm, resolve(parts)), nil
}))
_ = native.Set(vm, exports, "relative", native.Bind(vm, "", 2, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
from := native.String(vm, moejs.Arg(args, 0))
to := native.String(vm, moejs.Arg(args, 1))
if windows {
return winPathRelative(m.cwd(), from, to)
return native.From(vm, winPathRelative(m.cwd(), from, to)), nil
}
return posixPathRelative(m.cwd(), from, to)
})
_ = exports.Set("dirname", dirname)
_ = exports.Set("basename", func(call goja.FunctionCall) goja.Value {
result := basename(call.Argument(0).String())
if suffix := call.Argument(1); !goja.IsUndefined(suffix) && strings.HasSuffix(result, suffix.String()) {
result = strings.TrimSuffix(result, suffix.String())
return native.From(vm, posixPathRelative(m.cwd(), from, to)), nil
}))
_ = native.Set(vm, exports, "dirname", native.Bind(vm, "dirname", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, dirname(native.String(vm, moejs.Arg(args, 0)))), nil
}))
_ = native.Set(vm, exports, "basename", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
result := basename(native.String(vm, moejs.Arg(args, 0)))
if suffix := moejs.Arg(args, 1); !suffix.IsUndefined() && strings.HasSuffix(result, native.String(vm, suffix)) {
result = strings.TrimSuffix(result, native.String(vm, suffix))
}
return vm.ToValue(result)
})
_ = exports.Set("extname", extname)
_ = exports.Set("parse", func(value string) map[string]string {
return native.From(vm, result), nil
}))
_ = native.Set(vm, exports, "extname", native.Bind(vm, "extname", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, extname(native.String(vm, moejs.Arg(args, 0)))), nil
}))
_ = native.Set(vm, exports, "parse", native.Bind(vm, "", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := native.String(vm, moejs.Arg(args, 0))
if windows {
return winPathParse(value)
return native.From(vm, winPathParse(value)), nil
}
return posixPathParse(value)
})
_ = exports.Set("format", func(call goja.FunctionCall) goja.Value {
object := call.Argument(0).ToObject(vm)
resultDir := jsStringProperty(object, "dir")
return native.From(vm, posixPathParse(value)), nil
}))
_ = native.Set(vm, exports, "format", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
object := native.Object(vm, moejs.Arg(args, 0))
resultDir := jsStringProperty(vm, object, "dir")
if resultDir == "" {
resultDir = jsStringProperty(object, "root")
resultDir = jsStringProperty(vm, object, "root")
}
resultBase := jsStringProperty(object, "base")
resultBase := jsStringProperty(vm, object, "base")
if resultBase == "" {
extension := jsStringProperty(object, "ext")
extension := jsStringProperty(vm, object, "ext")
if extension != "" && !strings.HasPrefix(extension, ".") {
extension = "." + extension
}
resultBase = jsStringProperty(object, "name") + extension
resultBase = jsStringProperty(vm, object, "name") + extension
}
if resultDir == "" {
return vm.ToValue(resultBase)
return native.From(vm, resultBase), nil
}
return vm.ToValue(join([]string{resultDir, resultBase}))
})
_ = exports.Set("toNamespacedPath", func(value string) string {
return native.From(vm, join([]string{resultDir, resultBase})), nil
}))
_ = native.Set(vm, exports, "toNamespacedPath", native.Bind(vm, "", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
value := native.String(vm, moejs.Arg(args, 0))
if !windows || value == "" {
return value
return native.From(vm, value), nil
}
resolved := winPathResolve(m.cwd(), []string{value})
if strings.HasPrefix(resolved, `\\?\`) || strings.HasPrefix(resolved, `\\.\`) {
return resolved
return native.From(vm, resolved), nil
}
if strings.HasPrefix(resolved, `\\`) {
return `\\?\UNC\` + strings.TrimLeft(resolved, `\`)
return native.From(vm, `\\?\UNC\`+strings.TrimLeft(resolved, `\`)), nil
}
if winDriveVolume(resolved) != "" {
return `\\?\` + resolved
return native.From(vm, `\\?\`+resolved), nil
}
return value
})
return native.From(vm, value), nil
}))
return exports
}
func jsStringProperty(object *goja.Object, name string) string {
value := object.Get(name)
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func jsStringProperty(vm *moejs.Runtime, object *engine.Object, name string) string {
value := native.Get(vm, object, name)
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return ""
}
return value.String()
return native.String(vm, value)
}
func posixPathResolve(cwd string, parts []string) string {
cwd = strings.ReplaceAll(cwd, `\`, "/")
if volume := winDriveVolume(cwd); volume != "" {
@@ -180,7 +190,6 @@ func posixPathResolve(cwd string, parts []string) string {
}
return result
}
func posixPathNormalize(value string) string {
if value == "" {
return "."
@@ -192,7 +201,6 @@ func posixPathNormalize(value string) string {
}
return result
}
func posixPathJoin(parts []string) string {
nonEmpty := make([]string, 0, len(parts))
for _, part := range parts {
@@ -205,7 +213,6 @@ func posixPathJoin(parts []string) string {
}
return posixPathNormalize(strings.Join(nonEmpty, "/"))
}
func posixPathParts(value string) []string {
value = strings.Trim(value, "/")
if value == "" {
@@ -213,7 +220,6 @@ func posixPathParts(value string) []string {
}
return strings.Split(value, "/")
}
func posixPathRelative(cwd, from, to string) string {
from = posixPathResolve(cwd, []string{from})
to = posixPathResolve(cwd, []string{to})
@@ -233,7 +239,6 @@ func posixPathRelative(cwd, from, to string) string {
result = append(result, toParts[common:]...)
return strings.Join(result, "/")
}
func posixPathBasename(value string) string {
end := len(value)
for end > 0 && value[end-1] == '/' {
@@ -248,7 +253,6 @@ func posixPathBasename(value string) string {
}
return value[start:end]
}
func posixPathParse(value string) map[string]string {
root := ""
rootEnd := 0
@@ -276,16 +280,15 @@ func posixPathParse(value string) map[string]string {
extension := nodeExtname(base, false)
return map[string]string{"root": root, "dir": dir, "base": base, "ext": extension, "name": strings.TrimSuffix(base, extension)}
}
func isWinSeparator(character byte) bool { return character == '\\' || character == '/' }
func isWinSeparator(character byte) bool {
return character == '\\' || character == '/'
}
func winDriveVolume(value string) string {
if len(value) >= 2 && ((value[0] >= 'A' && value[0] <= 'Z') || (value[0] >= 'a' && value[0] <= 'z')) && value[1] == ':' {
return value[:2]
}
return ""
}
func splitWinPath(value string) (volume, root, rest string) {
value = strings.ReplaceAll(value, "/", `\`)
if drive := winDriveVolume(value); drive != "" {
@@ -296,7 +299,9 @@ func splitWinPath(value string) (volume, root, rest string) {
return
}
if strings.HasPrefix(value, `\\`) {
parts := strings.FieldsFunc(strings.TrimLeft(value, `\`), func(character rune) bool { return character == '\\' })
parts := strings.FieldsFunc(strings.TrimLeft(value, `\`), func(character rune) bool {
return character == '\\'
})
if len(parts) >= 2 {
volume = `\\` + parts[0] + `\` + parts[1]
root = `\`
@@ -313,9 +318,10 @@ func splitWinPath(value string) (volume, root, rest string) {
}
return "", "", value
}
func cleanWinParts(rest string, absolute bool) []string {
parts := strings.FieldsFunc(rest, func(character rune) bool { return character == '\\' || character == '/' })
parts := strings.FieldsFunc(rest, func(character rune) bool {
return character == '\\' || character == '/'
})
result := make([]string, 0, len(parts))
for _, part := range parts {
switch part {
@@ -332,7 +338,6 @@ func cleanWinParts(rest string, absolute bool) []string {
}
return result
}
func winPathNormalize(value string) string {
if value == "" {
return "."
@@ -351,7 +356,6 @@ func winPathNormalize(value string) string {
}
return result
}
func winPathJoin(parts []string) string {
nonEmpty := make([]string, 0, len(parts))
for _, part := range parts {
@@ -364,7 +368,6 @@ func winPathJoin(parts []string) string {
}
return winPathNormalize(strings.Join(nonEmpty, `\`))
}
func winPathResolve(cwd string, parts []string) string {
resolvedDevice, resolvedTail := "", ""
resolvedAbsolute := false
@@ -416,7 +419,6 @@ func winPathResolve(cwd string, parts []string) string {
}
return result
}
func winPathRelative(cwd, from, to string) string {
from = winPathResolve(cwd, []string{from})
to = winPathResolve(cwd, []string{to})
@@ -441,9 +443,10 @@ func winPathRelative(cwd, from, to string) string {
result = append(result, toParts[common:]...)
return strings.Join(result, `\`)
}
func winPathBasename(value string) string {
value = strings.TrimRightFunc(value, func(character rune) bool { return character == '\\' || character == '/' })
value = strings.TrimRightFunc(value, func(character rune) bool {
return character == '\\' || character == '/'
})
if value == "" {
return ""
}
@@ -454,7 +457,6 @@ func winPathBasename(value string) string {
}
return value[start:]
}
func winPathParse(value string) map[string]string {
value = strings.ReplaceAll(value, "/", `\`)
volume, rootSeparator, rest := splitWinPath(value)
@@ -487,10 +489,11 @@ func winPathParse(value string) map[string]string {
extension := nodeExtname(base, true)
return map[string]string{"root": root, "dir": dir, "base": base, "ext": extension, "name": strings.TrimSuffix(base, extension)}
}
func winPathDirname(value string) string {
volume, root, _ := splitWinPath(value)
trimmed := strings.TrimRightFunc(value, func(character rune) bool { return character == '\\' || character == '/' })
trimmed := strings.TrimRightFunc(value, func(character rune) bool {
return character == '\\' || character == '/'
})
index := strings.LastIndexAny(trimmed, `/\`)
if index < len(volume) {
if root != "" {
@@ -501,13 +504,14 @@ func winPathDirname(value string) string {
}
return "."
}
result := strings.TrimRightFunc(trimmed[:index], func(character rune) bool { return character == '\\' || character == '/' })
result := strings.TrimRightFunc(trimmed[:index], func(character rune) bool {
return character == '\\' || character == '/'
})
if result == volume {
return volume + root
}
return result
}
func nodeExtname(value string, windows bool) string {
startDot, startPart, end := -1, 0, -1
matchedSlash, preDotState := true, 0
+29 -24
View File
@@ -4,30 +4,32 @@ import (
_ "embed"
"errors"
"fmt"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
func (m *Module) initNextTickScheduler(vm *goja.Runtime) error {
factoryValue, err := vm.RunString(nextTickSchedulerSource)
func (m *Module) initNextTickScheduler(vm *moejs.Runtime) error {
factoryValue, err := native.Run(vm, nextTickSchedulerSource)
if err != nil {
return fmt.Errorf("create process.nextTick scheduler: %w", err)
}
factory, _ := goja.AssertFunction(factoryValue)
schedulerValue, err := factory(goja.Undefined(), vm.ToValue(func() { m.drainNextTicks(vm) }))
factory, _ := checkedFunction(factoryValue)
schedulerValue, err := native.Call(vm, factory, moejs.Undefined(), native.From(vm, native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
m.drainNextTicks(vm)
return moejs.Undefined(), nil
})))
if err != nil {
return fmt.Errorf("initialize process.nextTick scheduler: %w", err)
}
scheduler := schedulerValue.ToObject(vm)
m.nextTickTurn, _ = goja.AssertFunction(scheduler.Get("run"))
m.nextTickSchedule, _ = goja.AssertFunction(scheduler.Get("schedule"))
if m.nextTickTurn == nil || m.nextTickSchedule == nil {
scheduler := native.Object(vm, schedulerValue)
m.nextTickTurn, _ = checkedFunction(native.Get(vm, scheduler, "run"))
m.nextTickSchedule, _ = checkedFunction(native.Get(vm, scheduler, "schedule"))
if native.IsNil(m.nextTickTurn) || native.IsNil(m.nextTickSchedule) {
return errors.New("process.nextTick scheduler is incomplete")
}
return nil
}
func (m *Module) enqueueNextTick(vm *goja.Runtime, tick nodeNextTick) {
func (m *Module) enqueueNextTick(vm *moejs.Runtime, tick nodeNextTick) {
m.nextTickMu.Lock()
m.nextTicks = append(m.nextTicks, tick)
shouldSchedule := !m.nextTickScheduled && !m.nextTickDraining
@@ -38,15 +40,16 @@ func (m *Module) enqueueNextTick(vm *goja.Runtime, tick nodeNextTick) {
if !shouldSchedule {
return
}
if m.nextTickSchedule != nil {
if _, err := m.nextTickSchedule(goja.Undefined()); err == nil {
if !native.IsNil(m.nextTickSchedule) {
if _, err := native.Call(vm, m.nextTickSchedule, moejs.Undefined()); err == nil {
return
}
}
m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) { m.drainNextTicks(vm) }, 0)
m.runtime.Loop().SetTimeout(func(vm *moejs.Runtime) {
m.drainNextTicks(vm)
}, 0)
}
func (m *Module) drainNextTicks(vm *goja.Runtime) {
func (m *Module) drainNextTicks(vm *moejs.Runtime) {
m.nextTickMu.Lock()
m.nextTickDraining = true
m.nextTickScheduled = false
@@ -62,24 +65,26 @@ func (m *Module) drainNextTicks(vm *goja.Runtime) {
m.nextTicks[0] = nodeNextTick{}
m.nextTicks = m.nextTicks[1:]
m.nextTickMu.Unlock()
if _, err := tick.callback(goja.Undefined(), tick.arguments...); err != nil {
if _, err := native.Call(vm, tick.callback, moejs.Undefined(), tick.arguments...); err != nil {
if m.reportError != nil {
m.reportError(vm, []goja.Value{vm.ToValue(fmt.Sprintf("process.nextTick callback failed: %v", err))})
m.reportError(vm, []moejs.Value{native.From(vm, fmt.Sprintf("process.nextTick callback failed: %v", err))})
}
}
}
}
func (m *Module) RunTurn(vm *goja.Runtime, job func() error) error {
if m.nextTickTurn == nil {
func (m *Module) RunTurn(vm *moejs.Runtime, job func() error) error {
if native.IsNil(m.nextTickTurn) {
return job()
}
m.nextTickMu.Lock()
m.nextTickScheduled = true
m.nextTickMu.Unlock()
var jobErr error
jobValue := vm.ToValue(func() { jobErr = job() })
_, turnErr := m.nextTickTurn(goja.Undefined(), jobValue)
jobValue := native.From(vm, native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
jobErr = job()
return moejs.Undefined(), nil
}))
_, turnErr := native.Call(vm, m.nextTickTurn, moejs.Undefined(), jobValue)
return errors.Join(jobErr, turnErr)
}
+126 -118
View File
@@ -3,48 +3,46 @@ package process
import (
_ "embed"
"fmt"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/buffer"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/fs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/metrics"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"github.com/komari-monitor/komari/utils"
"math/big"
"os"
"runtime"
"strings"
"sync"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/require"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/fs"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/metrics"
"github.com/komari-monitor/komari/utils"
)
type nodeNextTick struct {
callback goja.Callable
arguments []goja.Value
callback moejs.Value
arguments []moejs.Value
}
type Module struct {
runtime *bridge.Runtime
fs *fs.Module
allowExec bool
startedAt time.Time
reportError func(*goja.Runtime, []goja.Value)
runtime *bridge.Runtime
fs *fs.Module
allowExec bool
startedAt time.Time
reportError func(*moejs.Runtime, []moejs.Value)
nextTickMu sync.Mutex
nextTicks []nodeNextTick
nextTickScheduled bool
nextTickDraining bool
nextTickTurn goja.Callable
nextTickSchedule goja.Callable
nextTickTurn moejs.Value
nextTickSchedule moejs.Value
}
func New(runtime *bridge.Runtime, filesystem *fs.Module, allowExec bool, startedAt time.Time, reportError func(*goja.Runtime, []goja.Value)) *Module {
func New(runtime *bridge.Runtime, filesystem *fs.Module, allowExec bool, startedAt time.Time, reportError func(*moejs.Runtime, []moejs.Value)) *Module {
return &Module{runtime: runtime, fs: filesystem, allowExec: allowExec, startedAt: startedAt, reportError: reportError}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
process := events.NewEmitter(vm)
environment := make(map[string]string)
for _, item := range os.Environ() {
@@ -53,77 +51,78 @@ func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
environment[parts[0]] = parts[1]
}
}
_ = process.Set("env", environment)
_ = process.Set("argv", append([]string{os.Args[0], "script.js"}, os.Args[1:]...))
_ = process.Set("execArgv", []string{})
_ = process.Set("execPath", os.Args[0])
_ = process.Set("pid", os.Getpid())
_ = process.Set("ppid", os.Getppid())
_ = process.Set("platform", metrics.Platform())
_ = process.Set("arch", metrics.Arch())
_ = process.Set("version", "v"+strings.TrimPrefix(runtime.Version(), "go"))
_ = process.Set("versions", map[string]string{"node": "0.0.0-goja", "go": runtime.Version(), "komari": utils.CurrentVersion, "hash": utils.VersionHash})
_ = process.Set("release", map[string]string{"name": "node", "sourceUrl": "", "headersUrl": ""})
_ = process.Set("title", "komari-jsruntime")
_ = process.Set("exitCode", 0)
_ = process.Set("connected", false)
_ = process.Set("config", map[string]any{})
_ = process.Set("cwd", func() string { return m.fs.Cwd() })
_ = process.Set("chdir", func(path string) {
_ = native.Set(vm, process, "env", environment)
_ = native.Set(vm, process, "argv", append([]string{os.Args[0], "script.js"}, os.Args[1:]...))
_ = native.Set(vm, process, "execArgv", []string{})
_ = native.Set(vm, process, "execPath", os.Args[0])
_ = native.Set(vm, process, "pid", os.Getpid())
_ = native.Set(vm, process, "ppid", os.Getppid())
_ = native.Set(vm, process, "platform", metrics.Platform())
_ = native.Set(vm, process, "arch", metrics.Arch())
_ = native.Set(vm, process, "version", "v"+strings.TrimPrefix(runtime.Version(), "go"))
_ = native.Set(vm, process, "versions", map[string]string{"node": "0.0.0-moejs", "go": runtime.Version(), "komari": utils.CurrentVersion, "hash": utils.VersionHash})
_ = native.Set(vm, process, "release", map[string]string{"name": "node", "sourceUrl": "", "headersUrl": ""})
_ = native.Set(vm, process, "title", "komari-jsruntime")
_ = native.Set(vm, process, "exitCode", 0)
_ = native.Set(vm, process, "connected", false)
_ = native.Set(vm, process, "config", map[string]any{})
_ = native.Set(vm, process, "cwd", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, m.fs.Cwd()), nil
}))
_ = native.Set(vm, process, "chdir", native.Bind(vm, "", 1, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
path := native.String(vm, moejs.Arg(args, 0))
if err := m.fs.Chdir(path); err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
})
_ = process.Set("uptime", func() float64 { return time.Since(m.startedAt).Seconds() })
memoryUsage := vm.ToValue(func() map[string]uint64 {
return moejs.Undefined(), nil
}))
_ = native.Set(vm, process, "uptime", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, time.Since(m.startedAt).Seconds()), nil
}))
memoryUsage := native.Object(vm, native.From(vm, native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
var memory runtime.MemStats
runtime.ReadMemStats(&memory)
usage, err := metrics.ReadProcess()
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
return map[string]uint64{"rss": usage.RSS, "heapTotal": memory.HeapSys, "heapUsed": memory.HeapAlloc, "external": memory.OtherSys, "arrayBuffers": 0}
}).ToObject(vm)
_ = memoryUsage.Set("rss", func() uint64 {
return native.From(vm, map[string]uint64{"rss": usage.RSS, "heapTotal": memory.HeapSys, "heapUsed": memory.HeapAlloc, "external": memory.OtherSys, "arrayBuffers": 0}), nil
})))
_ = native.Set(vm, memoryUsage, "rss", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
usage, err := metrics.ReadProcess()
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
return usage.RSS
})
_ = process.Set("memoryUsage", memoryUsage)
_ = process.Set("cpuUsage", func(call goja.FunctionCall) goja.Value {
return native.From(vm, usage.RSS), nil
}))
_ = native.Set(vm, process, "memoryUsage", memoryUsage)
_ = native.Set(vm, process, "cpuUsage", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
usage, err := metrics.ReadProcess()
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
userMicros, systemMicros := usage.UserCPU.Microseconds(), usage.SystemCPU.Microseconds()
if previous := call.Argument(0); !goja.IsUndefined(previous) && !goja.IsNull(previous) {
object := previous.ToObject(vm)
userMicros -= object.Get("user").ToInteger()
systemMicros -= object.Get("system").ToInteger()
if previous := moejs.Arg(args, 0); !previous.IsUndefined() && !previous.IsNull() {
object := native.Object(vm, previous)
userMicros -= native.Int(vm, native.Get(vm, object, "user"))
systemMicros -= native.Int(vm, native.Get(vm, object, "system"))
}
return vm.ToValue(map[string]int64{"user": max(userMicros, 0), "system": max(systemMicros, 0)})
})
_ = process.Set("resourceUsage", func() map[string]any {
return native.From(vm, map[string]int64{"user": max(userMicros, 0), "system": max(systemMicros, 0)}), nil
}))
_ = native.Set(vm, process, "resourceUsage", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
usage, err := metrics.ReadProcess()
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
return map[string]any{
"userCPUTime": usage.UserCPU.Microseconds(), "systemCPUTime": usage.SystemCPU.Microseconds(), "maxRSS": usage.MaxRSS,
"minorPageFault": usage.MinorPageFault, "majorPageFault": usage.MajorPageFault,
"fsRead": usage.FSRead, "fsWrite": usage.FSWrite,
"voluntaryContextSwitches": usage.VoluntarySwitches, "involuntaryContextSwitches": usage.InvoluntarySwitches,
}
})
hrtime := vm.ToValue(func(call goja.FunctionCall) goja.Value {
return native.From(vm, map[string]any{"userCPUTime": usage.UserCPU.Microseconds(), "systemCPUTime": usage.SystemCPU.Microseconds(), "maxRSS": usage.MaxRSS, "minorPageFault": usage.MinorPageFault, "majorPageFault": usage.MajorPageFault, "fsRead": usage.FSRead, "fsWrite": usage.FSWrite, "voluntaryContextSwitches": usage.VoluntarySwitches, "involuntaryContextSwitches": usage.InvoluntarySwitches}), nil
}))
hrtime := native.Object(vm, native.From(vm, native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
now := time.Since(m.startedAt)
seconds := int64(now / time.Second)
nanoseconds := int64(now % time.Second)
if previous := call.Argument(0); !goja.IsUndefined(previous) {
if previous := moejs.Arg(args, 0); !previous.IsUndefined() {
var values []int64
if err := vm.ExportTo(previous, &values); err == nil && len(values) >= 2 {
if err := native.ExportTo(vm, previous, &values); err == nil && len(values) >= 2 {
seconds -= values[0]
nanoseconds -= values[1]
if nanoseconds < 0 {
@@ -132,32 +131,36 @@ func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
}
}
}
return vm.ToValue([]int64{seconds, nanoseconds})
}).ToObject(vm)
_ = hrtime.Set("bigint", func() *big.Int { return big.NewInt(time.Since(m.startedAt).Nanoseconds()) })
_ = process.Set("hrtime", hrtime)
_ = process.Set("nextTick", func(call goja.FunctionCall) goja.Value {
callback, ok := goja.AssertFunction(call.Argument(0))
return native.From(vm, []int64{seconds, nanoseconds}), nil
})))
_ = native.Set(vm, hrtime, "bigint", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, big.NewInt(time.Since(m.startedAt).Nanoseconds())), nil
}))
_ = native.Set(vm, process, "hrtime", hrtime)
_ = native.Set(vm, process, "nextTick", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
callback, ok := checkedFunction(moejs.Arg(args, 0))
if !ok {
panic(vm.NewTypeError("callback must be a function"))
panic(native.TypeError(vm, "callback must be a function"))
}
arguments := append([]goja.Value(nil), call.Arguments[1:]...)
arguments := append([]moejs.Value(nil), args[1:]...)
m.enqueueNextTick(vm, nodeNextTick{callback: callback, arguments: arguments})
return goja.Undefined()
})
_ = process.Set("emitWarning", func(call goja.FunctionCall) goja.Value {
return native.From(vm, moejs.Undefined()), nil
}))
_ = native.Set(vm, process, "emitWarning", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
if m.reportError != nil {
m.reportError(vm, []goja.Value{vm.ToValue("Warning: " + call.Argument(0).String())})
m.reportError(vm, []moejs.Value{native.From(vm, "Warning: "+native.String(vm, moejs.Arg(args, 0)))})
}
return goja.Undefined()
})
_ = process.Set("kill", func(pid int, signal string) bool {
return native.From(vm, moejs.Undefined()), nil
}))
_ = native.Set(vm, process, "kill", native.Bind(vm, "", 2, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
pid := int(native.Int(vm, moejs.Arg(args, 0)))
signal := native.String(vm, moejs.Arg(args, 1))
if !m.allowExec {
panic(vm.NewGoError(fmt.Errorf("process.kill requires AllowExec")))
panic(native.GoError(vm, fmt.Errorf("process.kill requires AllowExec")))
}
process, err := os.FindProcess(pid)
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
if signal == "SIGKILL" {
err = process.Kill()
@@ -165,36 +168,39 @@ func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
err = process.Signal(os.Interrupt)
}
if err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
return true
})
_ = process.Set("exit", func(call goja.FunctionCall) goja.Value {
code := call.Argument(0).ToInteger()
_ = process.Set("exitCode", code)
panic(vm.NewGoError(&bridge.ExitRequestedError{Code: code}))
})
_ = process.Set("abort", func() { panic(vm.NewGoError(fmt.Errorf("process.abort() requested"))) })
return native.From(vm, true), nil
}))
_ = native.Set(vm, process, "exit", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
code := native.Int(vm, moejs.Arg(args, 0))
_ = native.Set(vm, process, "exitCode", code)
panic(native.GoError(vm, &bridge.ExitRequestedError{Code: code}))
}))
_ = native.Set(vm, process, "abort", native.Bind(vm, "", 0, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
panic(native.GoError(vm, fmt.Errorf("process.abort() requested")))
}))
m.attachProcessStreams(vm, process)
if err := m.initNextTickScheduler(vm); err != nil {
panic(vm.NewGoError(err))
panic(native.GoError(vm, err))
}
_ = module.Set("exports", process)
_ = native.Set(vm, module, "exports", process)
}
//go:embed stdio.js
var stdioSource string
// attachProcessStreams wires process.stdout/stderr as Writable streams and
// process.stdin as a Readable stream backed by the jsruntime stream module.
func (m *Module) attachProcessStreams(vm *goja.Runtime, process *goja.Object) {
factoryValue, err := vm.RunString(stdioSource)
func (m *Module) attachProcessStreams(vm *moejs.Runtime, process *engine.Object) {
factoryValue, err := native.Run(vm, stdioSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load process stdio: %w", err)))
panic(native.GoError(vm, fmt.Errorf("load process stdio: %w", err)))
}
factory, _ := goja.AssertFunction(factoryValue)
hooks := vm.NewObject()
_ = hooks.Set("write", func(name string, chunk goja.Value, callback goja.Callable) {
factory, _ := checkedFunction(factoryValue)
hooks := vm.Realm().NewObject()
_ = native.Set(vm, hooks, "write", native.Bind(vm, "", 3, func(_ *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := native.String(vm, moejs.Arg(args, 0))
chunk := moejs.Arg(args, 1)
callback := moejs.Arg(args, 2)
var output *os.File
switch name {
case "stdout":
@@ -202,22 +208,24 @@ func (m *Module) attachProcessStreams(vm *goja.Runtime, process *goja.Object) {
case "stderr":
output = os.Stderr
default:
panic(vm.NewGoError(fmt.Errorf("process.%s.write is not supported by jsruntime; the stream is not connected", name)))
panic(native.GoError(vm, fmt.Errorf("process.%s.write is not supported by jsruntime; the stream is not connected", name)))
}
_, _ = output.Write(buffer.Bytes(vm, chunk))
if callback != nil {
_, _ = callback(goja.Undefined())
if !native.IsNil(callback) {
_, _ = native.Call(vm, callback, moejs.Undefined())
}
})
streams, err := factory(goja.Undefined(), require.Require(vm, "stream"), hooks)
return moejs.Undefined(), nil
}))
streams, err := native.Call(vm, factory, moejs.Undefined(), require.Require(vm, "stream"), engine.ObjectValue(hooks))
if err != nil {
panic(err)
}
streamsObject := streams.ToObject(vm)
streamsObject := native.Object(vm, streams)
for _, name := range []string{"stdout", "stderr", "stdin"} {
stream := streamsObject.Get(name).ToObject(vm)
_ = stream.Set("isTTY", false)
_ = stream.Set("fd", -1)
_ = process.Set(name, stream)
stream := native.Object(vm, native.Get(vm, streamsObject, name))
_ = native.Set(vm, stream, "isTTY", false)
_ = native.Set(vm, stream, "fd", -1)
_ = native.Set(vm, process, name, stream)
}
}
func checkedFunction(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
+13
View File
@@ -0,0 +1,13 @@
Copyright (c) 2016 Dmitry Panov
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+1
View File
@@ -0,0 +1 @@
The require resolver and registry derive from github.com/dop251/goja_nodejs (Dmitry Panov), previously carried in pkg/moenode. Native MoeJS integration and local modifications retain the accompanying MIT license. The sibling buffer, url, util and loop packages also retain a copy of the upstream MIT license for any carried-over implementation material.
+269
View File
@@ -0,0 +1,269 @@
package require
import (
"errors"
"io"
"io/fs"
"os"
"path"
"path/filepath"
"runtime"
"sync"
"syscall"
"text/template"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
n "github.com/komari-monitor/komari/pkg/jsruntime/native"
)
type ModuleLoader func(*moejs.Runtime, *engine.Object)
// SourceLoader represents a function that returns a file data at a given path.
// The function should return ModuleFileDoesNotExistError if the file either doesn't exist or is a directory.
// This error will be ignored by the resolver and the search will continue. Any other errors will be propagated.
type SourceLoader func(path string) ([]byte, error)
// PathResolver is a function that should return a canonical path of the path parameter relative to the base. The base
// is expected to be already canonical as it would be a result of a previous call to the PathResolver for all cases
// except for the initial evaluation, but it's a responsibility of the caller to ensure that the name of the script
// is a canonical path. To match Node JS behaviour, it should resolve symlinks.
// The path parameter is the argument of the require() call. The returned value will be supplied to the SourceLoader.
type PathResolver func(base, path string) string
var (
InvalidModuleError = errors.New("Invalid module")
IllegalModuleNameError = errors.New("Illegal module name")
NoSuchBuiltInModuleError = errors.New("No such built-in module")
ModuleFileDoesNotExistError = errors.New("module file does not exist")
)
var native, builtin map[string]ModuleLoader
// Registry contains a cache of compiled modules which can be used by multiple Runtimes
type Registry struct {
sync.Mutex
native map[string]ModuleLoader
compiled map[string]*moejs.Script
srcLoader SourceLoader
pathResolver PathResolver
globalFolders []string
}
type RequireModule struct {
r *Registry
runtime *moejs.Runtime
modules map[string]*engine.Object
nodeModules map[string]*engine.Object
}
func NewRegistry(opts ...Option) *Registry {
r := &Registry{}
for _, opt := range opts {
opt(r)
}
return r
}
func NewRegistryWithLoader(srcLoader SourceLoader) *Registry {
return NewRegistry(WithLoader(srcLoader))
}
type Option func(*Registry)
// WithLoader sets a function which will be called by the require() function in order to get a source code for a
// module at the given path. Unlike the upstream goja_nodejs implementation,
// the moejs-backed runtime does not use this loader for external source maps.
// This only affects modules loaded by require(); it does not change runtime
// parser settings or relax the caller-provided module path restrictions.
func WithLoader(srcLoader SourceLoader) Option {
return func(r *Registry) {
r.srcLoader = srcLoader
}
}
// WithPathResolver sets a function which will be used to resolve paths (see PathResolver). If not specified, the
// DefaultPathResolver is used.
func WithPathResolver(pathResolver PathResolver) Option {
return func(r *Registry) {
r.pathResolver = pathResolver
}
}
// WithGlobalFolders appends the given paths to the registry's list of
// global folders to search if the requested module is not found
// elsewhere. By default, a registry's global folders list is empty.
// In the reference Node.js implementation, the default global folders
// list is $NODE_PATH, $HOME/.node_modules, $HOME/.node_libraries and
// $PREFIX/lib/node, see
// https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders.
func WithGlobalFolders(globalFolders ...string) Option {
return func(r *Registry) {
r.globalFolders = globalFolders
}
}
// Enable adds the require() function to the specified runtime.
func (r *Registry) Enable(runtime *moejs.Runtime) *RequireModule {
rrt := &RequireModule{
r: r,
runtime: runtime,
modules: make(map[string]*engine.Object),
nodeModules: make(map[string]*engine.Object),
}
if err := runtime.SetGlobal("require", n.Bind(runtime, "require", 1, rrt.require)); err != nil {
panic(err)
}
return rrt
}
func (r *Registry) RegisterNativeModule(name string, loader ModuleLoader) {
r.Lock()
defer r.Unlock()
if r.native == nil {
r.native = make(map[string]ModuleLoader)
}
name = filepathClean(name)
r.native[name] = loader
}
// DefaultSourceLoader is used if none was set (see WithLoader()). It simply loads files from the host's filesystem.
func DefaultSourceLoader(filename string) ([]byte, error) {
f, err := os.Open(filename)
if err != nil {
if errors.Is(err, fs.ErrNotExist) {
err = ModuleFileDoesNotExistError
} else if runtime.GOOS == "windows" {
if errors.Is(err, syscall.Errno(0x7b)) { // ERROR_INVALID_NAME, The filename, directory name, or volume label syntax is incorrect.
err = ModuleFileDoesNotExistError
}
}
return nil, err
}
defer f.Close()
// On some systems (e.g. plan9 and FreeBSD) it is possible to use the standard read() call on directories
// which means we cannot rely on read() returning an error, we have to do stat() instead.
if fi, err := f.Stat(); err == nil {
if fi.IsDir() {
return nil, ModuleFileDoesNotExistError
}
} else {
return nil, err
}
return io.ReadAll(f)
}
// DefaultPathResolver is used if none was set (see WithPathResolver). It converts the path using filepath.FromSlash(),
// then joins it with base and resolves symlinks on the resulting path.
// Note, it does not make the path absolute, so to match nodejs behaviour, the initial script name should be set
// to an absolute path.
// The implementation is somewhat suboptimal because it runs filepath.EvalSymlinks() on the joint path, not using the
// fact that the base path is already resolved. This is because there is no way to resolve symlinks only in a portion
// of a path without re-implementing a significant part of filepath.FromSlash().
func DefaultPathResolver(base, path string) string {
p := filepath.Join(base, filepath.FromSlash(path))
if resolved, err := filepath.EvalSymlinks(p); err == nil {
p = resolved
}
return p
}
func (r *Registry) getSource(p string) ([]byte, error) {
srcLoader := r.srcLoader
if srcLoader == nil {
srcLoader = DefaultSourceLoader
}
return srcLoader(p)
}
func (r *Registry) getCompiledSource(p string) (*moejs.Script, error) {
r.Lock()
defer r.Unlock()
prg := r.compiled[p]
if prg == nil {
buf, err := r.getSource(p)
if err != nil {
return nil, err
}
s := string(buf)
if filepath.Ext(p) == ".json" {
s = "module.exports = JSON.parse('" + template.JSEscapeString(s) + "')"
}
source := "(function(exports,require,module,__filename,__dirname){" + s + "\n})"
// Native MoeJS compiles source directly; custom source-map loading
// via WithLoader is not supported by this runtime. Module source loading
// and the caller-provided path resolver remain unchanged.
prg, err = moejs.CompileScript(p, source)
if err == nil {
if r.compiled == nil {
r.compiled = make(map[string]*moejs.Script)
}
r.compiled[p] = prg
}
return prg, err
}
return prg, nil
}
func (r *RequireModule) require(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
return r.Require(n.String(r.runtime, moejs.Arg(args, 0)))
}
func filepathClean(p string) string {
return path.Clean(p)
}
// Require can be used to import modules from Go source (similar to JS require() function).
func (r *RequireModule) Require(p string) (ret moejs.Value, err error) {
module, err := r.resolve(p)
if err != nil {
return
}
ret, err = r.runtime.Get(engine.ObjectValue(module), "exports")
return
}
func Require(runtime *moejs.Runtime, name string) moejs.Value {
v, err := runtime.CallFunction(n.Global(runtime, "require"), moejs.Undefined(), n.From(runtime, name))
if err != nil {
panic(err)
}
return v
}
// RegisterNativeModule registers a module that isn't loaded through a SourceLoader, but rather through
// a provided ModuleLoader. Typically, this will be a module implemented in Go (although theoretically
// it can be anything, depending on the ModuleLoader implementation).
// Such modules take precedence over modules loaded through a SourceLoader, i.e. if a module name resolves as
// native, the native module is loaded, and the SourceLoader is not consulted.
// The binding is global and affects all instances of Registry.
// It should be called from a package init() function as it may not be used concurrently with require() calls.
// For registry-specific bindings see Registry.RegisterNativeModule.
func RegisterNativeModule(name string, loader ModuleLoader) {
if native == nil {
native = make(map[string]ModuleLoader)
}
name = filepathClean(name)
native[name] = loader
}
// RegisterCoreModule registers a nodejs core module. If the name does not start with "node:", the module
// will also be loadable as "node:<name>". Hence, for "builtin" modules (such as buffer, console, etc.)
// the name should not include the "node:" prefix, but for prefix-only core modules (such as "node:test")
// it should include the prefix.
func RegisterCoreModule(name string, loader ModuleLoader) {
if builtin == nil {
builtin = make(map[string]ModuleLoader)
}
name = filepathClean(name)
builtin[name] = loader
}
+70
View File
@@ -0,0 +1,70 @@
package require
import (
"errors"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
n "github.com/komari-monitor/komari/pkg/jsruntime/native"
"path/filepath"
"testing"
)
func TestModules(t *testing.T) {
files := map[string]string{
"app/main.js": "exports.a=require('./a'); exports.json=require('./data.json'); exports.later=function(){return require('./a')}; exports.pkg=require('pkg')",
"app/a.js": "exports.n=42;exports.cycle=require('./b').a",
"app/b.js": "exports.a=require('./a')",
"app/data.json": `{"ok":true}`,
"app/node_modules/pkg/package.json": `{"main":"lib.js"}`,
"app/node_modules/pkg/lib.js": "module.exports=7",
}
loads := map[string]int{}
reg := NewRegistry(WithLoader(func(p string) ([]byte, error) {
p = filepath.ToSlash(p)
loads[p]++
if s, ok := files[p]; ok {
return []byte(s), nil
}
return nil, ModuleFileDoesNotExistError
}))
vm := moejs.NewRuntime(moejs.Options{})
reg.Enable(vm)
v, err := n.RunScript(vm, "entry.js", "var m=require('./app/main'); m.a.n===42 && m.a.cycle===m.a && m.json.ok && m.later()===m.a && m.pkg===7")
if err != nil || !n.Bool(v) {
t.Fatalf("result %v error %v loads %v", v, err, loads)
}
if loads["app/a.js"] != 1 {
t.Fatalf("module loaded %d times", loads["app/a.js"])
}
// Compiled scripts are reusable across runtimes, exports are not shared.
vm2 := moejs.NewRuntime(moejs.Options{})
reg.Enable(vm2)
if _, err := n.Run(vm2, "require('./app/main')"); err != nil {
t.Fatal(err)
}
if loads["app/a.js"] != 1 {
t.Fatal("compiled cache missed")
}
}
func TestNativeAndPermissions(t *testing.T) {
denied := errors.New("permission denied")
reg := NewRegistry(WithLoader(func(p string) ([]byte, error) { return nil, denied }))
reg.RegisterNativeModule("test", func(vm *moejs.Runtime, m *engine.Object) {
if err := n.Set(vm, m, "exports", 12); err != nil {
panic(err)
}
})
vm := moejs.NewRuntime(moejs.Options{})
r := reg.Enable(vm)
v, err := r.Require("test")
if err != nil || n.Int(vm, v) != 12 {
t.Fatal(v, err)
}
if _, err := r.Require("./secret"); !errors.Is(err, denied) {
t.Fatal("permission lost", err)
}
v, err = n.Run(vm, "try{require('./secret');false}catch(e){e.message==='permission denied'}")
if err != nil || !n.Bool(v) {
t.Fatal(v, err)
}
}
+288
View File
@@ -0,0 +1,288 @@
package require
import (
"encoding/json"
"errors"
"path/filepath"
"runtime"
"strings"
"github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
n "github.com/komari-monitor/komari/pkg/jsruntime/native"
)
const NodePrefix = "node:"
func (r *RequireModule) resolvePath(base, name string) string {
if r.r.pathResolver != nil {
return r.r.pathResolver(base, name)
}
return DefaultPathResolver(base, name)
}
// NodeJS module search algorithm described by
// https://nodejs.org/api/modules.html#modules_all_together
func (r *RequireModule) resolve(modpath string) (*engine.Object, error) {
return r.resolveFrom(modpath, r.getCurrentModulePath())
}
func (r *RequireModule) resolveFrom(modpath, start string) (module *engine.Object, err error) {
if filepath.IsAbs(modpath) {
start = ""
}
p := r.resolvePath(start, modpath)
if isFileOrDirectoryPath(modpath) {
if module = r.modules[p]; module != nil {
return
}
module, err = r.loadAsFileOrDirectory(p)
if err == nil && module != nil {
r.modules[p] = module
}
} else {
module, err = r.loadNative(modpath)
if err == nil {
return
} else {
if err == InvalidModuleError {
err = nil
} else {
return
}
}
if module = r.nodeModules[p]; module != nil {
return
}
module, err = r.loadNodeModules(modpath, start)
if err == nil && module != nil {
r.nodeModules[p] = module
}
}
if module == nil && err == nil {
err = InvalidModuleError
}
return
}
func (r *RequireModule) loadNative(path string) (*engine.Object, error) {
module := r.modules[path]
if module != nil {
return module, nil
}
var ldr ModuleLoader
if ldr = r.r.native[path]; ldr == nil {
ldr = native[path]
}
var isBuiltIn, withPrefix bool
if ldr == nil {
ldr = builtin[path]
if ldr == nil && strings.HasPrefix(path, NodePrefix) {
ldr = builtin[path[len(NodePrefix):]]
if ldr == nil {
return nil, NoSuchBuiltInModuleError
}
withPrefix = true
}
isBuiltIn = true
}
if ldr != nil {
module = r.createModuleObject()
r.modules[path] = module
if isBuiltIn {
if withPrefix {
r.modules[path[len(NodePrefix):]] = module
} else {
if !strings.HasPrefix(path, NodePrefix) {
r.modules[NodePrefix+path] = module
}
}
}
ldr(r.runtime, module)
return module, nil
}
return nil, InvalidModuleError
}
func (r *RequireModule) loadAsFileOrDirectory(path string) (module *engine.Object, err error) {
if module, err = r.loadAsFile(path); module != nil || err != nil {
return
}
return r.loadAsDirectory(path)
}
func (r *RequireModule) loadAsFile(path string) (module *engine.Object, err error) {
if module, err = r.loadModule(path); module != nil || err != nil {
return
}
p := path + ".js"
if module, err = r.loadModule(p); module != nil || err != nil {
return
}
p = path + ".json"
return r.loadModule(p)
}
func (r *RequireModule) loadIndex(modpath string) (module *engine.Object, err error) {
p := r.resolvePath(modpath, "index.js")
if module, err = r.loadModule(p); module != nil || err != nil {
return
}
p = r.resolvePath(modpath, "index.json")
return r.loadModule(p)
}
func (r *RequireModule) loadAsDirectory(modpath string) (module *engine.Object, err error) {
p := r.resolvePath(modpath, "package.json")
buf, err := r.r.getSource(p)
if err != nil {
if !errors.Is(err, ModuleFileDoesNotExistError) {
return nil, err
}
return r.loadIndex(modpath)
}
var pkg struct {
Main string
}
err = json.Unmarshal(buf, &pkg)
if err != nil || len(pkg.Main) == 0 {
return r.loadIndex(modpath)
}
m := r.resolvePath(modpath, pkg.Main)
if module, err = r.loadAsFile(m); module != nil || err != nil {
return
}
return r.loadIndex(m)
}
func (r *RequireModule) loadNodeModule(modpath, start string) (*engine.Object, error) {
return r.loadAsFileOrDirectory(r.resolvePath(start, modpath))
}
func (r *RequireModule) loadNodeModules(modpath, start string) (module *engine.Object, err error) {
for _, dir := range r.r.globalFolders {
if module, err = r.loadNodeModule(modpath, dir); module != nil || err != nil {
return
}
}
for {
var p string
if filepath.Base(start) != "node_modules" {
p = filepath.Join(start, "node_modules")
} else {
p = start
}
if module, err = r.loadNodeModule(modpath, p); module != nil || err != nil {
return
}
if start == ".." { // Dir('..') is '.'
break
}
parent := filepath.Dir(start)
if parent == start {
break
}
start = parent
}
return
}
func (r *RequireModule) getCurrentModulePath() string {
// Capture native engine frames, not a textual stack trace. The first bytecode
// caller is the script that requested this module (native frames have no code).
frames := r.runtime.Realm().NewError(engine.KindError, "").ErrorData().Frames()
for _, f := range frames {
if code := f.Code(); code != nil && code.Source != nil && code.Source.Name != "" {
return filepath.Dir(code.Source.Name)
}
}
return "."
}
func (r *RequireModule) createModuleObject() *engine.Object {
module := r.runtime.Realm().NewObject()
if err := n.Set(r.runtime, module, "exports", r.runtime.Realm().NewObject()); err != nil {
panic(err)
}
return module
}
func (r *RequireModule) loadModule(path string) (*engine.Object, error) {
module := r.modules[path]
if module == nil {
module = r.createModuleObject()
r.modules[path] = module
err := r.loadModuleFile(path, module)
if err != nil {
module = nil
delete(r.modules, path)
if errors.Is(err, ModuleFileDoesNotExistError) {
err = nil
}
}
return module, err
}
return module, nil
}
func (r *RequireModule) loadModuleFile(path string, module *engine.Object) error {
script, err := r.r.getCompiledSource(path)
if err != nil {
return err
}
fn, err := r.runtime.RunScript(script)
if err != nil {
return err
}
if !engine.IsCallable(fn) {
return InvalidModuleError
}
exports, err := r.runtime.Get(engine.ObjectValue(module), "exports")
if err != nil {
return err
}
// Each module gets a lexical require rooted at its own filename. This also
// works when an exported function invokes require after initialization.
req := n.Bind(r.runtime, "require", 1, func(_ *moejs.Realm, _ moejs.Value, args []moejs.Value) (moejs.Value, error) {
name := n.String(r.runtime, moejs.Arg(args, 0))
return r.requireFrom(name, filepath.Dir(path))
})
_, err = r.runtime.CallFunction(fn, exports, exports, req, engine.ObjectValue(module), n.From(r.runtime, path), n.From(r.runtime, filepath.Dir(path)))
return err
}
func (r *RequireModule) requireFrom(name, base string) (moejs.Value, error) {
module, err := r.resolveFrom(name, base)
if err != nil {
return moejs.Undefined(), err
}
return r.runtime.Get(engine.ObjectValue(module), "exports")
}
func isFileOrDirectoryPath(path string) bool {
result := path == "." || path == ".." ||
strings.HasPrefix(path, "/") ||
strings.HasPrefix(path, "./") ||
strings.HasPrefix(path, "../")
if runtime.GOOS == "windows" {
result = result ||
strings.HasPrefix(path, `.\`) ||
strings.HasPrefix(path, `..\`) ||
filepath.IsAbs(path)
}
return result
}
+95 -83
View File
@@ -1,9 +1,9 @@
// Package jsruntime provides the JavaScript runtime used by server-side
// scripts.
//
// JavaScript compatibility: ECMAScript and Promise support provided by goja,
// CommonJS require and the event loop provided by goja_nodejs, and the host
// APIs assembled in globals.go. This is not a complete browser environment.
// JavaScript executes directly on native MoeJS values and explicit host callbacks.
// CommonJS and Node-style host capabilities are implemented in this package tree;
// scheduling uses MoeJS eventloop. This is not a complete browser environment.
package jsruntime
import (
@@ -18,11 +18,9 @@ import (
"sync/atomic"
"time"
"github.com/dop251/goja"
_ "github.com/dop251/goja_nodejs/buffer"
"github.com/dop251/goja_nodejs/eventloop"
"github.com/dop251/goja_nodejs/require"
_ "github.com/dop251/goja_nodejs/util"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
_ "github.com/komari-monitor/komari/pkg/jsruntime/buffer"
childprocess "github.com/komari-monitor/komari/pkg/jsruntime/child_process"
"github.com/komari-monitor/komari/pkg/jsruntime/console"
cryptomodule "github.com/komari-monitor/komari/pkg/jsruntime/crypto"
@@ -30,11 +28,15 @@ import (
"github.com/komari-monitor/komari/pkg/jsruntime/fs"
httpmodule "github.com/komari-monitor/komari/pkg/jsruntime/http"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
eventloop "github.com/komari-monitor/komari/pkg/jsruntime/loop"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
netmodule "github.com/komari-monitor/komari/pkg/jsruntime/net"
pathmodule "github.com/komari-monitor/komari/pkg/jsruntime/path"
processmodule "github.com/komari-monitor/komari/pkg/jsruntime/process"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"github.com/komari-monitor/komari/pkg/jsruntime/timers"
tlsmodule "github.com/komari-monitor/komari/pkg/jsruntime/tls"
_ "github.com/komari-monitor/komari/pkg/jsruntime/util"
)
const defaultTimeout = 30 * time.Second
@@ -103,12 +105,12 @@ type Options struct {
// Runtime owns one isolated JavaScript VM and its event loop. Public
// operations are serialized, and all VM access runs on the event-loop
// goroutine because goja runtimes are not goroutine-safe.
// goroutine because moejs runtimes are not goroutine-safe.
type Runtime struct {
mu sync.Mutex
loop *eventloop.EventLoop
host *bridge.Runtime
vm *goja.Runtime
vm *moe.Runtime
timeout time.Duration
closed bool
nodeJS bool
@@ -195,7 +197,6 @@ func New(script string, options Options) (*Runtime, error) {
}))
registry := require.NewRegistry(registryOptions...)
runtimeLoop := eventloop.NewEventLoop(
eventloop.EnableConsole(false),
eventloop.WithRegistry(registry),
)
host := bridge.New(runtimeLoop, timeout)
@@ -233,7 +234,7 @@ func New(script string, options Options) (*Runtime, error) {
}
if options.NodeJS {
runtime.pathModule = pathmodule.New(filesystem.Cwd)
runtime.processModule = processmodule.New(host, filesystem, options.AllowExec, startedAt, func(vm *goja.Runtime, values []goja.Value) {
runtime.processModule = processmodule.New(host, filesystem, options.AllowExec, startedAt, func(vm *moe.Runtime, values []moe.Value) {
runtime.consoleMod.WriteError(vm, values, false)
})
runtime.childProcessModule = childprocess.New(host, filesystem, options.AllowExec, maxChildOutputBytes)
@@ -243,13 +244,13 @@ func New(script string, options Options) (*Runtime, error) {
runtime.cryptoModule = cryptomodule.New(host)
runtime.registerNodeModules(registry)
}
host.SetTurnRunner(func(vm *goja.Runtime, job func() error) error {
host.SetTurnRunner(func(vm *moe.Runtime, job func() error) error {
if runtime.processModule != nil {
return runtime.processModule.RunTurn(vm, job)
}
return job()
})
host.SetErrorReporter(func(vm *goja.Runtime, name string, reportErr error) {
host.SetErrorReporter(func(vm *moe.Runtime, name string, reportErr error) {
runtime.consoleMod.Report(vm, name, reportErr)
})
if options.ConfigureHost != nil {
@@ -280,7 +281,7 @@ func New(script string, options Options) (*Runtime, error) {
runtime.loop.Start()
initialized := make(chan error, 1)
if !runtime.loop.RunOnLoop(func(vm *goja.Runtime) {
if !runtime.loop.RunOnLoop(func(vm *moe.Runtime) {
runtime.vm = vm
if err := runtime.injectGlobals(); err != nil {
initialized <- err
@@ -291,7 +292,7 @@ func New(script string, options Options) (*Runtime, error) {
if baseDir != "" {
sourceName = filepath.Join(baseDir, sourceName)
}
if _, err := vm.RunScript(sourceName, script); err != nil {
if _, err := native.RunScript(vm, sourceName, script); err != nil {
return fmt.Errorf("failed to load JavaScript script: %v", err)
}
return nil
@@ -414,9 +415,6 @@ func pathWithinBaseDir(baseDir, path string) bool {
// Close stops the event loop and cancels its pending timers. A closed runtime
// cannot be called again.
func (r *Runtime) Close() {
if r == nil {
return
}
r.mu.Lock()
defer r.mu.Unlock()
if r.closed {
@@ -443,28 +441,35 @@ func (r *Runtime) closeHostResources() {
// HasFunction reports whether name resolves to a callable JavaScript
// function.
func (r *Runtime) HasFunction(name string) bool {
if r == nil {
return false
}
r.mu.Lock()
defer r.mu.Unlock()
if r.closed {
return false
}
result := make(chan bool, 1)
if !r.loop.RunOnLoop(func(vm *goja.Runtime) {
value := vm.Get(name)
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
result <- false
return
}
_, ok := goja.AssertFunction(value)
result <- ok
deadline := time.Now().Add(r.timeout)
if !r.loop.RunOnLoop(func(vm *moe.Runtime) {
callable := false
err := bridge.RunWithDeadline(vm, deadline, func() error {
v, err := vm.Get(e.ObjectValue(vm.Realm().Global), name)
if err != nil {
return err
}
callable = e.IsCallable(v)
return nil
})
result <- err == nil && callable
}) {
return false
}
return <-result
timer := time.NewTimer(time.Until(deadline))
defer timer.Stop()
select {
case ok := <-result:
return ok
case <-timer.C:
return false
}
}
// Call invokes a named JavaScript function and accepts either a truthy
@@ -481,10 +486,6 @@ func (r *Runtime) CallVoid(name string, args ...any) error {
}
func (r *Runtime) call(name string, requireTruthy bool, args ...any) error {
if r == nil {
return errors.New("JavaScript runtime is nil")
}
r.mu.Lock()
defer r.mu.Unlock()
if r.closed {
@@ -500,12 +501,30 @@ func (r *Runtime) call(name string, requireTruthy bool, args ...any) error {
}
}
if !r.loop.RunOnLoop(func(vm *goja.Runtime) {
if !r.loop.RunOnLoop(func(vm *moe.Runtime) {
if time.Now().After(deadline) {
finish(bridge.ErrExecutionTimeout)
return
}
r.callOnLoop(vm, name, args, deadline, finish, requireTruthy)
err := bridge.RunWithDeadline(vm, deadline, func() (err error) {
defer func() {
if p := recover(); p != nil {
switch x := p.(type) {
case error:
err = x
case moe.Value:
err = &moe.Exception{Value: x}
default:
panic(p)
}
}
}()
r.callOnLoop(vm, name, args, deadline, finish, requireTruthy)
return nil
})
if err != nil {
finish(err)
}
}) {
return errors.New("JavaScript event loop is not running")
}
@@ -521,30 +540,30 @@ func (r *Runtime) call(name string, requireTruthy bool, args ...any) error {
}
}
func (r *Runtime) callOnLoop(vm *goja.Runtime, name string, args []any, deadline time.Time, finish func(error), requireTruthy bool) {
function := vm.Get(name)
if function == nil || goja.IsUndefined(function) {
func (r *Runtime) callOnLoop(vm *moe.Runtime, name string, args []any, deadline time.Time, finish func(error), requireTruthy bool) {
function := native.Global(vm, name)
if function.IsUndefined() {
finish(fmt.Errorf("%s function not defined in script", name))
return
}
call, ok := goja.AssertFunction(function)
if !ok {
if !e.IsCallable(function) {
finish(fmt.Errorf("%s is not a function", name))
return
}
values := make([]goja.Value, len(args))
values := make([]moe.Value, len(args))
for i, arg := range args {
values[i] = vm.ToValue(arg)
v, err := vm.FromGo(arg)
if err != nil {
finish(err)
return
}
values[i] = v
}
var callResult goja.Value
err := bridge.RunWithDeadline(vm, deadline, func() error {
return r.host.RunTurn(vm, func() error {
var err error
callResult, err = call(goja.Undefined(), values...)
return err
})
var result moe.Value
err := r.host.RunTurn(vm, func() error {
var err error
result, err = vm.CallFunction(function, moe.Undefined(), values...)
return err
})
if err != nil {
if errors.Is(err, bridge.ErrExecutionTimeout) {
@@ -556,54 +575,47 @@ func (r *Runtime) callOnLoop(vm *goja.Runtime, name string, args []any, deadline
}
return
}
_, ok = callResult.Export().(*goja.Promise)
if !ok {
finish(okResult(name, requireTruthy, callResult))
if !result.IsObject() {
finish(okResult(name, requireTruthy, result))
return
}
then, ok := goja.AssertFunction(callResult.ToObject(vm).Get("then"))
if !ok {
finish(errors.New("JavaScript Promise has no callable then method"))
_, _, promise := result.AsObject().PromiseResult()
if !promise {
finish(okResult(name, requireTruthy, result))
return
}
onFulfilled := vm.ToValue(func(call goja.FunctionCall) goja.Value {
finish(okResult(name, requireTruthy, call.Argument(0)))
return goja.Undefined()
then := native.Get(vm, result.AsObject(), "then")
fulfilled := native.Bind(vm, "fulfilled", 1, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
finish(okResult(name, requireTruthy, moe.Arg(args, 0)))
return moe.Undefined(), nil
})
onRejected := vm.ToValue(func(call goja.FunctionCall) goja.Value {
if code, ok := bridge.ExitCodeFromValue(call.Argument(0)); ok {
rejected := native.Bind(vm, "rejected", 1, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
v := moe.Arg(args, 0)
if code, ok := bridge.ExitCodeFromValue(v); ok {
finish(exitCodeError(code))
return goja.Undefined()
} else {
finish(fmt.Errorf("Promise rejected: %s", native.String(vm, v)))
}
finish(fmt.Errorf("Promise rejected: %v", call.Argument(0)))
return goja.Undefined()
return moe.Undefined(), nil
})
if _, err := then(callResult, onFulfilled, onRejected); err != nil {
finish(fmt.Errorf("failed to observe Promise: %v", err))
return
if _, err := vm.CallFunction(then, result, fulfilled, rejected); err != nil {
finish(fmt.Errorf("failed to observe Promise: %w", err))
}
}
// exitCodeError converts a script-requested process exit into a result.
// Exit code 0 is a normal termination; nonzero codes are failures.
func exitCodeError(code int64) error {
if code == 0 {
return nil
}
return fmt.Errorf("JavaScript process exited with code %d", code)
}
func okResult(name string, requireTruthy bool, value goja.Value) error {
if !requireTruthy {
func okResult(name string, required bool, v moe.Value) error {
if !required {
return nil
}
return truthyResult(name, value)
return truthyResult(name, v)
}
func truthyResult(name string, value goja.Value) error {
if value != nil && value.ToBoolean() {
func truthyResult(name string, v moe.Value) error {
if native.Bool(v) {
return nil
}
return fmt.Errorf("%s returned false", name)
+14 -12
View File
@@ -12,8 +12,10 @@ import (
"testing"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/require"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
)
func TestNewReturnsRuntimeWithInjectedGlobals(t *testing.T) {
@@ -325,9 +327,9 @@ func TestRequireConfigurationComposesWithBaseDir(t *testing.T) {
Console: io.Discard,
Timeout: time.Second,
ConfigureRequire: func(registry *require.Registry) {
registry.RegisterNativeModule("komari", func(vm *goja.Runtime, module *goja.Object) {
exports := module.Get("exports").ToObject(vm)
_ = exports.Set("version", "test")
registry.RegisterNativeModule("komari", func(vm *moe.Runtime, module *e.Object) {
exports := native.Object(vm,native.Get(vm,module,"exports"))
_ = native.Set(vm,exports,"version", "test")
})
},
})
@@ -352,9 +354,9 @@ func TestConfigureRequireCanOverrideNodeModule(t *testing.T) {
Console: io.Discard,
Timeout: time.Second,
ConfigureRequire: func(registry *require.Registry) {
registry.RegisterNativeModule("path", func(vm *goja.Runtime, module *goja.Object) {
exports := module.Get("exports").ToObject(vm)
_ = exports.Set("source", "custom")
registry.RegisterNativeModule("path", func(vm *moe.Runtime, module *e.Object) {
exports := native.Object(vm,native.Get(vm,module,"exports"))
_ = native.Set(vm,exports,"source", "custom")
})
},
})
@@ -537,9 +539,9 @@ func TestConfigureHostProvidesHostServicesAndModules(t *testing.T) {
Timeout: time.Second,
ConfigureHost: func(h *Host, registry *require.Registry) {
host = h
registry.RegisterNativeModule("server", func(vm *goja.Runtime, module *goja.Object) {
exports := module.Get("exports").ToObject(vm)
_ = exports.Set("name", "test-server")
registry.RegisterNativeModule("server", func(vm *moe.Runtime, module *e.Object) {
exports := native.Object(vm,native.Get(vm,module,"exports"))
_ = native.Set(vm,exports,"name", "test-server")
})
},
})
@@ -558,7 +560,7 @@ func TestConfigureHostProvidesHostServicesAndModules(t *testing.T) {
// Host services must be usable from another goroutine while the runtime
// is idle: queue a job and wait for it to run on the event loop.
ran := make(chan bool, 1)
if !host.RunOnLoop(func(vm *goja.Runtime) {
if !host.RunOnLoop(func(vm *moe.Runtime) {
ran <- true
}) {
t.Fatal("RunOnLoop rejected a job on an open runtime")
+21 -22
View File
@@ -2,37 +2,36 @@ package stream
import (
_ "embed"
"fmt"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/require"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
)
// ModuleName is the CommonJS name of the Node.js stream compatibility module.
const ModuleName = "stream"
// PromisesModuleName is the CommonJS name of the stream/promises module.
const PromisesModuleName = "stream/promises"
//go:embed stream.js
var streamSource string
// Load registers the Node.js-compatible stream module (Readable, Writable,
// Duplex, Transform, PassThrough, pipeline, finished, ...).
func Load(vm *goja.Runtime, module *goja.Object) {
exports, err := vm.RunString(streamSource)
func Load(vm *moe.Runtime, module *e.Object) {
exports, err := native.Run(vm, streamSource)
if err != nil {
panic(vm.NewGoError(fmt.Errorf("load stream module: %w", err)))
panic(err)
}
if err = native.Set(vm, module, "exports", exports); err != nil {
panic(err)
}
_ = module.Set("exports", exports)
}
// LoadPromises registers the stream/promises module with Promise-based
// pipeline and finished helpers.
func LoadPromises(vm *goja.Runtime, module *goja.Object) {
streamModule := require.Require(vm, ModuleName).ToObject(vm)
exports := vm.NewObject()
_ = exports.Set("pipeline", streamModule.Get("pipeline"))
_ = exports.Set("finished", streamModule.Get("finished"))
_ = module.Set("exports", exports)
func LoadPromises(vm *moe.Runtime, module *e.Object) {
stream := native.Object(vm, require.Require(vm, ModuleName))
exports := vm.Realm().NewObject()
for _, key := range []string{"pipeline", "finished"} {
if err := native.Set(vm, exports, key, native.Get(vm, stream, key)); err != nil {
panic(err)
}
}
if err := native.Set(vm, module, "exports", exports); err != nil {
panic(err)
}
}
+68 -79
View File
@@ -2,101 +2,90 @@ package timers
import (
"errors"
"time"
"github.com/dop251/goja"
"github.com/dop251/goja_nodejs/eventloop"
moe "github.com/Calcium-Ion/moejs"
e "github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
"github.com/komari-monitor/komari/pkg/jsruntime/loop"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"time"
)
type Module struct {
runtime *bridge.Runtime
next int64
handles map[int64]any
}
func New(runtime *bridge.Runtime) *Module {
return &Module{runtime: runtime}
}
func (m *Module) Inject(vm *goja.Runtime) error {
if err := vm.Set("setTimeout", func(call goja.FunctionCall) goja.Value {
callback, args := timerCallback(vm, call, 2)
timer := m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.RunJob(vm, "setTimeout", func() error {
_, err := callback(goja.Undefined(), args...)
return err
})
}, timerDelay(call.Argument(1)))
return vm.ToValue(timer)
}); err != nil {
return err
}
if err := vm.Set("setInterval", func(call goja.FunctionCall) goja.Value {
callback, args := timerCallback(vm, call, 2)
var interval *eventloop.Interval
interval = m.runtime.Loop().SetInterval(func(vm *goja.Runtime) {
err := m.runtime.RunJob(vm, "setInterval", func() error {
_, err := callback(goja.Undefined(), args...)
return err
})
if errors.Is(err, bridge.ErrExecutionTimeout) {
m.runtime.Loop().ClearInterval(interval)
func New(r *bridge.Runtime) *Module { return &Module{runtime: r, handles: make(map[int64]any)} }
func (m *Module) Inject(vm *moe.Runtime) error {
for _, name := range []string{"setTimeout", "setInterval", "setImmediate"} {
name := name
fn := native.Bind(vm, name, 2, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
callback := moe.Arg(args, 0)
if !e.IsCallable(callback) {
return moe.Undefined(), native.TypeError(vm, "timer callback must be a function")
}
}, timerDelay(call.Argument(1)))
return vm.ToValue(interval)
}); err != nil {
return err
first := 2
delay := time.Duration(0)
if name == "setImmediate" {
first = 1
} else {
delay = timerDelay(vm, moe.Arg(args, 1))
}
var params []moe.Value
if len(args) > first {
params = append(params, args[first:]...)
}
m.next++
id := m.next
job := func(rt *moe.Runtime) {
if name != "setInterval" {
delete(m.handles, id)
}
err := m.runtime.RunJob(rt, name, func() error { _, err := rt.CallFunction(callback, moe.Undefined(), params...); return err })
if errors.Is(err, bridge.ErrExecutionTimeout) {
m.clear(id)
}
}
if name == "setInterval" {
m.handles[id] = m.runtime.Loop().SetInterval(job, delay)
} else {
m.handles[id] = m.runtime.Loop().SetTimeout(job, delay)
}
return moe.Int(id), nil
})
if err := vm.SetGlobal(name, fn); err != nil {
return err
}
}
if err := vm.Set("setImmediate", func(call goja.FunctionCall) goja.Value {
callback, args := timerCallback(vm, call, 1)
timer := m.runtime.Loop().SetTimeout(func(vm *goja.Runtime) {
m.runtime.RunJob(vm, "setImmediate", func() error {
_, err := callback(goja.Undefined(), args...)
return err
})
}, 0)
return vm.ToValue(timer)
}); err != nil {
return err
clear := native.Bind(vm, "clearTimer", 1, func(_ *moe.Realm, _ moe.Value, args []moe.Value) (moe.Value, error) {
m.clear(native.Int(vm, moe.Arg(args, 0)))
return moe.Undefined(), nil
})
for _, name := range []string{"clearTimeout", "clearInterval", "clearImmediate"} {
if err := vm.SetGlobal(name, clear); err != nil {
return err
}
}
clear := func(call goja.FunctionCall) goja.Value { return m.clearTimer(call) }
if err := vm.Set("clearTimeout", clear); err != nil {
return err
}
if err := vm.Set("clearInterval", clear); err != nil {
return err
}
return vm.Set("clearImmediate", clear)
return nil
}
func timerCallback(vm *goja.Runtime, call goja.FunctionCall, firstArgument int) (goja.Callable, []goja.Value) {
callback, ok := goja.AssertFunction(call.Argument(0))
if !ok {
panic(vm.NewTypeError("timer callback must be a function"))
}
if len(call.Arguments) <= firstArgument {
return callback, nil
}
return callback, append([]goja.Value(nil), call.Arguments[firstArgument:]...)
}
func (m *Module) clearTimer(call goja.FunctionCall) goja.Value {
switch timer := call.Argument(0).Export().(type) {
case *eventloop.Timer:
func (m *Module) clear(id int64) {
switch timer := m.handles[id].(type) {
case *loop.Timer:
m.runtime.Loop().ClearTimeout(timer)
case *eventloop.Interval:
case *loop.Interval:
m.runtime.Loop().ClearInterval(timer)
}
return goja.Undefined()
delete(m.handles, id)
}
func timerDelay(value goja.Value) time.Duration {
milliseconds := value.ToInteger()
if milliseconds <= 0 {
func timerDelay(vm *moe.Runtime, v moe.Value) time.Duration {
n := native.Int(vm, v)
if n <= 0 {
return 0
}
const maxMilliseconds = int64((1<<63 - 1) / int64(time.Millisecond))
if milliseconds > maxMilliseconds {
milliseconds = maxMilliseconds
const max = int64((1<<63 - 1) / int64(time.Millisecond))
if n > max {
n = max
}
return time.Duration(milliseconds) * time.Millisecond
return time.Duration(n) * time.Millisecond
}
+66 -74
View File
@@ -1,20 +1,22 @@
package tlsmodule
import (
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
import (
"context"
"crypto/tls"
"crypto/x509"
"fmt"
"net"
"strconv"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/events"
"github.com/komari-monitor/komari/pkg/jsruntime/internal/bridge"
netmodule "github.com/komari-monitor/komari/pkg/jsruntime/net"
"net"
"strconv"
)
// Module provides the subset of node:tls needed by notification plugins.
type Module struct {
runtime *bridge.Runtime
netModule *netmodule.Module
@@ -23,79 +25,65 @@ type Module struct {
func New(runtime *bridge.Runtime, netModule *netmodule.Module) *Module {
return &Module{runtime: runtime, netModule: netModule}
}
func (m *Module) Load(vm *goja.Runtime, module *goja.Object) {
exports := vm.NewObject()
connect := func(call goja.FunctionCall) goja.Value { return m.connect(vm, call) }
_ = exports.Set("connect", connect)
_ = exports.Set("createConnection", connect)
_ = module.Set("exports", exports)
}
func (m *Module) connect(vm *goja.Runtime, call goja.FunctionCall) *goja.Object {
options, ok := call.Argument(0).(*goja.Object)
if !ok {
panic(vm.NewTypeError("tls.connect requires an options object"))
func (m *Module) Load(vm *moejs.Runtime, module *engine.Object) {
exports := vm.Realm().NewObject()
connect := func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
return native.From(vm, m.connect(vm, args)), nil
}
host := stringOption(options, "host")
port := intOption(options, "port")
serverName := stringOption(options, "servername")
_ = native.Set(vm, exports, "connect", native.Bind(vm, "connect", 1, connect))
_ = native.Set(vm, exports, "createConnection", native.Bind(vm, "createConnection", 1, connect))
_ = native.Set(vm, module, "exports", exports)
}
func (m *Module) connect(vm *moejs.Runtime, args []moejs.Value) *engine.Object {
options, ok := asObject(moejs.Arg(args, 0))
if !ok {
panic(native.TypeError(vm, "tls.connect requires an options object"))
}
host := stringOption(vm, options, "host")
port := intOption(vm, options, "port")
serverName := stringOption(vm, options, "servername")
if serverName == "" {
serverName = host
}
var sourceSocket *goja.Object
if value := options.Get("socket"); value != nil && !goja.IsUndefined(value) && !goja.IsNull(value) {
sourceSocket, ok = value.(*goja.Object)
var sourceSocket *engine.Object
if value := native.Get(vm, options, "socket"); !native.IsNil(value) && !value.IsUndefined() && !value.IsNull() {
sourceSocket, ok = asObject(value)
if !ok {
panic(vm.NewTypeError("tls.connect socket must be a net.Socket"))
panic(native.TypeError(vm, "tls.connect socket must be a net.Socket"))
}
}
if sourceSocket == nil && (host == "" || port < 1) {
panic(vm.NewTypeError("tls.connect requires host and port"))
panic(native.TypeError(vm, "tls.connect requires host and port"))
}
if serverName == "" {
panic(vm.NewTypeError("tls.connect requires servername when upgrading a socket"))
panic(native.TypeError(vm, "tls.connect requires servername when upgrading a socket"))
}
config := &tls.Config{ServerName: serverName, MinVersion: tls.VersionTLS12}
if ca := stringOption(options, "ca"); ca != "" {
if ca := stringOption(vm, options, "ca"); ca != "" {
roots := x509.NewCertPool()
if !roots.AppendCertsFromPEM([]byte(ca)) {
panic(vm.NewTypeError("tls.connect ca does not contain a certificate"))
panic(native.TypeError(vm, "tls.connect ca does not contain a certificate"))
}
config.RootCAs = roots
}
socket := events.NewEmitter(vm)
_ = socket.Set("connecting", true)
_ = socket.Set("encrypted", true)
_ = socket.Set("authorized", false)
_ = socket.Set("servername", serverName)
_ = native.Set(vm, socket, "connecting", true)
_ = native.Set(vm, socket, "encrypted", true)
_ = native.Set(vm, socket, "authorized", false)
_ = native.Set(vm, socket, "servername", serverName)
ctx, cancel := context.WithTimeout(context.Background(), m.runtime.Timeout())
_ = socket.Set("destroy", func() *goja.Object {
_ = native.Set(vm, socket, "destroy", native.Bind(vm, "destroy", 0, func(r *moejs.Realm, this moejs.Value, args []moejs.Value) (moejs.Value, error) {
cancel()
return socket
})
if callback, ok := goja.AssertFunction(call.Argument(1)); ok {
once, _ := goja.AssertFunction(socket.Get("once"))
_, _ = once(socket, vm.ToValue("secureConnect"), vm.ToValue(callback))
return native.From(vm, socket), nil
}))
if callback, ok := callable(moejs.Arg(args, 1)); ok {
once, _ := callable(native.Get(vm, socket, "once"))
_, _ = native.Call(vm, once, native.From(vm, socket), native.From(vm, "secureConnect"), native.From(vm, callback))
}
go m.establish(ctx, cancel, socket, sourceSocket, host, port, config)
return socket
}
func (m *Module) establish(
ctx context.Context,
cancel context.CancelFunc,
socket *goja.Object,
sourceSocket *goja.Object,
host string,
port int,
config *tls.Config,
) {
func (m *Module) establish(ctx context.Context, cancel context.CancelFunc, socket *engine.Object, sourceSocket *engine.Object, host string, port int, config *tls.Config) {
defer cancel()
var connection net.Conn
var err error
@@ -108,8 +96,9 @@ func (m *Module) establish(
m.emitError(socket, err)
return
}
resourceID := m.runtime.AddResource(func() { _ = connection.Close() })
resourceID := m.runtime.AddResource(func() {
_ = connection.Close()
})
if resourceID == 0 {
return
}
@@ -120,14 +109,13 @@ func (m *Module) establish(
m.emitError(socket, fmt.Errorf("TLS handshake failed: %w", err))
return
}
queued := m.runtime.RunOnLoop(func(vm *goja.Runtime) {
queued := m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "tls secureConnect", func() error {
if !m.netModule.AttachSocket(vm, socket, secureConnection, nil) {
return fmt.Errorf("TLS socket runtime is closed")
}
m.runtime.RemoveResource(resourceID)
_ = socket.Set("authorized", true)
_ = native.Set(vm, socket, "authorized", true)
return events.Emit(vm, socket, "secureConnect")
})
})
@@ -136,28 +124,32 @@ func (m *Module) establish(
_ = secureConnection.Close()
}
}
func (m *Module) emitError(socket *goja.Object, err error) {
m.runtime.RunOnLoop(func(vm *goja.Runtime) {
func (m *Module) emitError(socket *engine.Object, err error) {
m.runtime.RunOnLoop(func(vm *moejs.Runtime) {
_ = m.runtime.RunJob(vm, "tls error", func() error {
_ = socket.Set("connecting", false)
return events.Emit(vm, socket, "error", vm.NewGoError(err))
_ = native.Set(vm, socket, "connecting", false)
return events.Emit(vm, socket, "error", native.GoError(vm, err))
})
})
}
func stringOption(options *goja.Object, name string) string {
value := options.Get(name)
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func stringOption(vm *moejs.Runtime, options *engine.Object, name string) string {
value := native.Get(vm, options, name)
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return ""
}
return value.String()
return native.String(vm, value)
}
func intOption(options *goja.Object, name string) int {
value := options.Get(name)
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
func intOption(vm *moejs.Runtime, options *engine.Object, name string) int {
value := native.Get(vm, options, name)
if native.IsNil(value) || value.IsUndefined() || value.IsNull() {
return 0
}
return int(value.ToInteger())
return int(native.Int(vm, value))
}
func callable(v moejs.Value) (moejs.Value, bool) { return v, engine.IsCallable(v) }
func asObject(v moejs.Value) (*engine.Object, bool) {
if !v.IsObject() {
return nil, false
}
return v.AsObject(), true
}
+13
View File
@@ -0,0 +1,13 @@
Copyright (c) 2016 Dmitry Panov
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+161
View File
@@ -0,0 +1,161 @@
// Package url supplies native URL parsing and JavaScript web URL objects.
package url
import (
_ "embed"
"encoding/json"
moe "github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"golang.org/x/net/idna"
"net"
neturl "net/url"
"strconv"
"strings"
)
const ModuleName = "url"
//go:embed url.js
var source string
func parse(input, base string) (map[string]string, error) {
u, err := neturl.Parse(strings.TrimSpace(input))
if err != nil {
return nil, err
}
if base != "" {
b, e := neturl.Parse(base)
if e != nil || b.Scheme == "" {
return nil, &neturl.Error{Op: "parse", URL: base, Err: net.InvalidAddrError("Invalid base URL")}
}
u = b.ResolveReference(u)
}
if u.Scheme == "" {
return nil, net.InvalidAddrError("Invalid URL")
}
u.Scheme = strings.ToLower(u.Scheme)
hostname := strings.ToLower(u.Hostname())
if h, e := idna.Lookup.ToASCII(hostname); e == nil {
hostname = h
}
port := u.Port()
if port != "" {
p, e := strconv.Atoi(port)
if e != nil || p > 65535 {
return nil, net.InvalidAddrError("Invalid port")
}
port = strconv.Itoa(p)
}
if (u.Scheme == "http" || u.Scheme == "https" || u.Scheme == "ftp" || u.Scheme == "ws" || u.Scheme == "wss") && hostname == "" {
return nil, net.InvalidAddrError("Invalid host")
}
if ((u.Scheme == "https" || u.Scheme == "wss") && port == "443") || ((u.Scheme == "http" || u.Scheme == "ws") && port == "80") || (u.Scheme == "ftp" && port == "21") {
port = ""
}
host := hostname
if strings.Contains(host, ":") {
host = "[" + host + "]"
}
if port != "" {
host += ":" + port
}
u.Host = host
if host != "" && u.Path == "" {
u.Path = "/"
}
username, password := "", ""
if u.User != nil {
parts := strings.SplitN(u.User.String(), ":", 2)
username = parts[0]
if len(parts) == 2 {
password = parts[1]
}
}
origin := "null"
if u.Scheme == "http" || u.Scheme == "https" || u.Scheme == "ftp" || u.Scheme == "ws" || u.Scheme == "wss" {
origin = u.Scheme + "://" + host
}
search := ""
if u.RawQuery != "" || u.ForceQuery {
search = "?" + u.RawQuery
}
hash := ""
if u.Fragment != "" {
hash = "#" + u.EscapedFragment()
}
return map[string]string{"href": u.String(), "protocol": u.Scheme + ":", "hostname": func() string {
if strings.Contains(hostname, ":") {
return "[" + hostname + "]"
}
return hostname
}(), "host": host, "port": port, "pathname": u.EscapedPath(), "search": search, "hash": hash, "username": username, "password": password, "origin": origin}, nil
}
func Require(vm *moe.Runtime, module *engine.Object) {
factory, err := native.RunScript(vm, "url.js", source)
if err != nil {
panic(err)
}
parser := native.Bind(vm, "parseURL", 2, moe.NativeFunc(func(_ *moe.Realm, _ moe.Value, a []moe.Value) (moe.Value, error) {
base := ""
if !native.IsNil(moe.Arg(a, 1)) {
base = native.String(vm, moe.Arg(a, 1))
}
m, e := parse(native.String(vm, moe.Arg(a, 0)), base)
if e != nil {
return moe.Undefined(), vm.Realm().TypeError("Invalid URL: %s", e)
}
b, _ := json.Marshal(m)
return native.From(vm, string(b)), nil
}))
domain := native.Bind(vm, "domain", 2, moe.NativeFunc(func(_ *moe.Realm, _ moe.Value, a []moe.Value) (moe.Value, error) {
s := native.String(vm, moe.Arg(a, 0))
var result string
var e error
if native.Bool(moe.Arg(a, 1)) {
result, e = idna.Lookup.ToUnicode(s)
} else {
result, e = idna.Lookup.ToASCII(s)
}
if e != nil {
result = ""
}
return native.From(vm, result), nil
}))
decodeForm := native.Bind(vm, "decodeForm", 1, moe.NativeFunc(func(_ *moe.Realm, _ moe.Value, a []moe.Value) (moe.Value, error) {
s := native.String(vm, moe.Arg(a, 0))
var out strings.Builder
for i := 0; i < len(s); i++ {
if s[i] == '+' {
out.WriteByte(' ')
continue
}
if s[i] == '%' && i+2 < len(s) {
if b, e := strconv.ParseUint(s[i+1:i+3], 16, 8); e == nil {
out.WriteByte(byte(b))
i += 2
continue
}
}
out.WriteByte(s[i])
}
return native.From(vm, string([]rune(out.String()))), nil
}))
v, err := vm.CallFunction(factory, moe.Undefined(), parser, domain, decodeForm)
if err != nil {
panic(err)
}
if err = native.Set(vm, module, "exports", v); err != nil {
panic(err)
}
}
func Enable(vm *moe.Runtime) {
m := require.Require(vm, ModuleName).AsObject()
for _, name := range []string{"URL", "URLSearchParams"} {
if err := vm.SetGlobal(name, native.Get(vm, m, name)); err != nil {
panic(err)
}
}
}
func init() { require.RegisterCoreModule(ModuleName, Require) }
+57
View File
@@ -0,0 +1,57 @@
(function(parseURL, domain, decodeForm) {
'use strict';
function scalar(s){return String(s).replace(/[\uD800-\uDBFF][\uDC00-\uDFFF]|[\uD800-\uDFFF]/g,c=>c.length===2?c:'\uFFFD');}
function escape(s){return encodeURIComponent(s).replace(/[!'()~]/g,c=>'%'+c.charCodeAt(0).toString(16).toUpperCase()).replace(/%20/g,'+');}
function unescape(s){return decodeForm(s);}
class URLSearchParams {
constructor(init){this._pairs=[];this._change=null;this._load(init);}
_load(init){this._pairs=[];if(init==null)return;if(typeof init==='string'){if(init[0]==='?')init=init.slice(1);for(const p of init.split('&')){if(!p)continue;const i=p.indexOf('=');this._pairs.push([unescape(i<0?p:p.slice(0,i)),unescape(i<0?'':p.slice(i+1))]);}}else if(init[Symbol.iterator]){for(const pair of init){const p=Array.from(pair);if(p.length!==2)throw new TypeError('Expected name/value pair');this._pairs.push([scalar(p[0]),scalar(p[1])]);}}else{for(const k of Object.keys(init))this._pairs.push([k,scalar(init[k])]);}}
_notify(){if(this._change)this._change(this.toString());}
get size(){return this._pairs.length;}
append(k,v){this._pairs.push([scalar(k),scalar(v)]);this._notify();}
delete(k,v){k=scalar(k);if(v!==undefined)v=scalar(v);this._pairs=this._pairs.filter(p=>p[0]!==k||(v!==undefined&&p[1]!==v));this._notify();}
get(k){k=scalar(k);for(const p of this._pairs)if(p[0]===k)return p[1];return null;}
getAll(k){k=scalar(k);return this._pairs.filter(p=>p[0]===k).map(p=>p[1]);}
has(k,v){k=scalar(k);return this._pairs.some(p=>p[0]===k&&(v===undefined||p[1]===scalar(v)));}
set(k,v){k=scalar(k);v=scalar(v);let found=false;this._pairs=this._pairs.filter(p=>{if(p[0]!==k)return true;if(found)return false;p[1]=v;found=true;return true;});if(!found)this._pairs.push([k,v]);this._notify();}
sort(){this._pairs.sort((a,b)=>a[0]<b[0]?-1:a[0]>b[0]?1:0);this._notify();}
*entries(){for(let i=0;i<this._pairs.length;i++)yield this._pairs[i].slice();}
*keys(){for(const p of this._pairs)yield p[0];}
*values(){for(const p of this._pairs)yield p[1];}
[Symbol.iterator](){return this.entries();}
forEach(fn,thisArg){for(let i=0;i<this._pairs.length;i++){const p=this._pairs[i];fn.call(thisArg,p[1],p[0],this);}}
toString(){return this._pairs.map(p=>escape(p[0])+'='+escape(p[1])).join('&');}
}
class URL {
constructor(input,base){this._data=JSON.parse(parseURL(scalar(input),base===undefined?undefined:scalar(base)));this._params=new URLSearchParams(this._data.search);this._params._change=s=>{this._set('search',s?'?'+s:'',false);};}
_parse(s,sync){const d=JSON.parse(parseURL(s));this._data=d;if(sync!==false)this._params._load(d.search);}
_set(k,v,sync){
v=scalar(v);if(k==='href'){this._parse(v,sync);return;}
const d=Object.assign({},this._data), special=p=>/^(https?|wss?|ftp):$/.test(p);
if(k==='protocol'){v=v.endsWith(':')?v:v+':';v=v.toLowerCase();if(!/^[a-z][a-z0-9+.-]*:$/.test(v)||special(v)!==special(d.protocol))return;}
if(['host','hostname','port','username','password'].includes(k)&&(!d.host||d.protocol==='file:'))return;
if(k==='port'&&(!/^\d*$/.test(v)||Number(v)>65535))return;
if(k==='hostname'&&(v.includes('/')||v.includes('?')||v.includes('#')||(!v.startsWith('[')&&v.includes(':'))))return;
if(k==='pathname'&&!d.host&&!d.pathname.startsWith('/'))return;
d[k]=v;
if(k==='host'){d.host=v.split(/[/?#]/)[0];}
if(k==='hostname')d.host=v+(d.port?':'+d.port:'');
if(k==='port')d.host=d.hostname+(v?':'+v:'');
if(k==='username'||k==='password')d[k]=encodeURIComponent(v);
if(k==='search')d.search=v?(v[0]==='?'?v:'?'+v).replace(/#/g,'%23').replace(/ /g,'%20'):'';
if(k==='hash')d.hash=v?(v[0]==='#'?v:'#'+v).replace(/ /g,'%20'):'';
if(k==='pathname')d.pathname=(v[0]==='/'?v:'/'+v).replace(/\?/g,'%3F').replace(/#/g,'%23');
const auth=d.username||d.password?d.username+(d.password?':'+d.password:'')+'@':'';
try{this._parse(d.protocol+(d.host?'//'+auth+d.host:'')+d.pathname+d.search+d.hash,sync);}catch(e){/* Invalid component assignments leave the URL unchanged. */}
}
get searchParams(){return this._params;}
get origin(){return this._data.origin;}
toString(){return this.href;}
toJSON(){return this.href;}
static canParse(s,b){try{new URL(s,b);return true;}catch(e){return false;}}
static parse(s,b){try{return new URL(s,b);}catch(e){return null;}}
}
for(const k of ['href','protocol','username','password','host','hostname','port','pathname','search','hash'])Object.defineProperty(URL.prototype,k,{enumerable:true,configurable:true,get(){return this._data[k];},set(v){this._set(k,v);}});
Object.defineProperty(URL.prototype,Symbol.toStringTag,{value:'URL'});Object.defineProperty(URLSearchParams.prototype,Symbol.toStringTag,{value:'URLSearchParams'});
return {URL,URLSearchParams,domainToASCII:s=>domain(scalar(s),false),domainToUnicode:s=>domain(scalar(s),true)};
})
+41
View File
@@ -0,0 +1,41 @@
package url
import (
moe "github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"testing"
)
func TestNativeURL(t *testing.T) {
vm := moe.NewRuntime(moe.Options{})
require.NewRegistry().Enable(vm)
Enable(vm)
for _, s := range []string{
`new URL("/a?x=1","https://example.com").hostname==="example.com"`,
`new URL("/a?x=1","https://example.com").searchParams.get("x")==="1"`,
`(function(){const p=new URLSearchParams();p.set("chat_id","-100 123");p.set("text","通知 & test");return p.toString()==="chat_id=-100+123&text=%E9%80%9A%E7%9F%A5+%26+test"})()`,
`(function(){const u=new URL("https://user:pass@example.com:443/a?x=1#h");const p=u.searchParams;p.append("x","2");if(u.href!=="https://user:pass@example.com/a?x=1&x=2#h")return false;u.search="?a=3";return p===u.searchParams&&p.get("a")==="3"})()`,
`(function(){const p=new URLSearchParams([["b","2"],["a","1"],["a","3"]]);p.sort();return p.getAll("a").join()==="1,3"&&Array.from(p.keys()).join()==="a,a,b"})()`,
`require("url").domainToASCII("例え.テスト")==="xn--r8jz45g.xn--zckzah"`,
`new URL("https://example.com").pathname==="/"`,
`!URL.canParse("relative")`,
`new URLSearchParams("x=%E9%80%9A%E7%9F%A5%FF&y=%ZZ").get("x")==="通知\uFFFD"`,
`new URLSearchParams([["x","\uD800"]]).toString()==="x=%EF%BF%BD"`,
`String(new URL("https://example.com"))==="https://example.com/"`,
`Object.isFrozen(Object.prototype)&&Object.isFrozen(Uint8Array.prototype)`,
`(function(){const u=new URL("https://a.test:81/x");u.hostname="b.test";u.port="443";u.protocol="http:";u.pathname="a?b#c";u.search="?q=#";u.hash="x y";return u.href==="http://b.test/a%3Fb%23c?q=%23#x%20y"})()`,
`(function(){const u=new URL("https://a.test/");u.username="a b";u.password="x@y";u.pathname="ok";return u.username==="a%20b"&&u.password==="x%40y"&&u.href==="https://a%20b:x%40y@a.test/ok"})()`,
`(function(){const u=new URL("https://a.test/");const p=u.searchParams;u.href="https://b.test/?a=1";p.append("b","2");return p===u.searchParams&&u.href==="https://b.test/?a=1&b=2"})()`,
`(function(){const u=new URL("https://a.test/");u.port="99999";u.protocol="file:";u.hostname="bad:host";return u.href==="https://a.test/"&&!URL.canParse("http://a:99999")&&!URL.canParse("http:")})()`,
`new URL("wss://a.test:443/").port===""`,
} {
v, err := native.Run(vm, s)
if err != nil {
t.Fatalf("%s: %v", s, err)
}
if !native.Bool(v) {
t.Errorf("false: %s", s)
}
}
}
+13
View File
@@ -0,0 +1,13 @@
Copyright (c) 2016 Dmitry Panov
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+44
View File
@@ -0,0 +1,44 @@
// Package util supplies JavaScript formatting without reflected Go callbacks.
package util
import (
"bytes"
_ "embed"
moe "github.com/Calcium-Ion/moejs"
"github.com/Calcium-Ion/moejs/engine"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
)
const ModuleName = "util"
//go:embed util.js
var source string
func Require(vm *moe.Runtime, module *engine.Object) {
v, err := native.RunScript(vm, "util.js", source)
if err != nil {
panic(err)
}
if err = native.Set(vm, module, "exports", v); err != nil {
panic(err)
}
}
type Util struct{ runtime *moe.Runtime }
func New(vm *moe.Runtime) *Util { return &Util{runtime: vm} }
func (u *Util) Format(b *bytes.Buffer, format string, args ...moe.Value) {
values := append([]moe.Value{native.From(u.runtime, format)}, args...)
b.WriteString(FormatConsole(u.runtime, values))
}
func FormatConsole(vm *moe.Runtime, args []moe.Value) string {
m := require.Require(vm, ModuleName)
f := native.Get(vm, m.AsObject(), "format")
v, err := vm.CallFunction(f, m, args...)
if err != nil {
panic(err)
}
return native.String(vm, v)
}
func init() { require.RegisterCoreModule(ModuleName, Require) }
+11
View File
@@ -0,0 +1,11 @@
(function(){
'use strict';
function inspect(v,opts){const seen=new Set();const depth=opts&&opts.depth!==undefined?opts.depth:2;function show(x,d){if(typeof x==='string')return "'"+x.replace(/'/g,"\'")+"'";if(typeof x==='bigint')return String(x)+'n';if(x===null||typeof x!=='object')return String(x);if(x instanceof Error)return x.stack||String(x);if(seen.has(x))return '[Circular]';if(d<0)return Array.isArray(x)?'[Array]':'[Object]';seen.add(x);let s;if(Array.isArray(x))s='[ '+x.map(v=>show(v,d-1)).join(', ')+' ]';else s='{ '+Object.keys(x).map(k=>(/^[a-zA-Z_$][a-zA-Z0-9_$]*$/.test(k)?k:show(k,d-1))+': '+show(x[k],d-1)).join(', ')+' }';seen.delete(x);return s;}return show(v,depth);}
function text(v){return typeof v==='object'&&v!==null?inspect(v):String(v);}
function format(...args){if(!args.length)return '';const first=args[0];if(typeof first!=='string')return args.map(text).join(' ');let i=1;const out=first.replace(/%[%sdifjoOc]/g,code=>{if(code==='%%')return '%';if(i>=args.length)return code;const v=args[i++];switch(code){case '%s':return String(v);case '%d':return typeof v==='bigint'?String(v)+'n':String(Number(v));case '%i':return String(parseInt(v,10));case '%f':return String(parseFloat(v));case '%j':try{return String(JSON.stringify(v));}catch(e){return '[Circular]';}case '%o':case '%O':return inspect(v);case '%c':return '';default:return code;}});return out+(i<args.length?' '+args.slice(i).map(text).join(' '):'');}
function inherits(ctor,superCtor){ctor.super_=superCtor;Object.setPrototypeOf(ctor.prototype,superCtor.prototype);}
function promisify(fn){return function(...args){return new Promise((resolve,reject)=>{fn.call(this,...args,(err,v)=>err?reject(err):resolve(v));});};}
function callbackify(fn){return function(...args){const cb=args.pop();if(typeof cb!=='function')throw new TypeError('Callback must be a function');Promise.resolve(fn.apply(this,args)).then(v=>cb(null,v),e=>cb(e||new Error('Promise rejected with falsy value')));};}
const types={isArrayBuffer:v=>v instanceof ArrayBuffer,isArrayBufferView:ArrayBuffer.isView,isTypedArray:v=>ArrayBuffer.isView(v)&&!(v instanceof DataView),isUint8Array:v=>v instanceof Uint8Array,isDate:v=>v instanceof Date,isRegExp:v=>v instanceof RegExp,isNativeError:v=>v instanceof Error,isPromise:v=>v instanceof Promise,isMap:v=>v instanceof Map,isSet:v=>v instanceof Set};
return {format,formatWithOptions:(opts,...args)=>format(...args),inspect,inherits,promisify,callbackify,types,isArray:Array.isArray,isBuffer:v=>typeof Buffer!=='undefined'&&Buffer.isBuffer(v),isString:v=>typeof v==='string',isNumber:v=>typeof v==='number',isFunction:v=>typeof v==='function',isNull:v=>v===null,isUndefined:v=>v===undefined,isObject:v=>v!==null&&typeof v==='object'};
})()
+29
View File
@@ -0,0 +1,29 @@
package util
import (
moe "github.com/Calcium-Ion/moejs"
"github.com/komari-monitor/komari/pkg/jsruntime/native"
"github.com/komari-monitor/komari/pkg/jsruntime/require"
"testing"
)
func TestNativeFormat(t *testing.T) {
vm := moe.NewRuntime(moe.Options{})
require.NewRegistry().Enable(vm)
for _, s := range []string{
`require("util").format("value=%d",2)==="value=2"`,
`require("util").format("%s %d %j %%","ok",3,{a:1})==='ok 3 {"a":1} %'`,
`require("util").format("%i %f", "12px", "1.5em")==="12 1.5"`,
`require("util").format("x",1,true)==="x 1 true"`,
`require("util").format()===""`,
`(function(){const x={};x.x=x;return require("util").format("%j",x)==="[Circular]"})()`,
} {
v, err := native.Run(vm, s)
if err != nil {
t.Fatalf("%s: %v", s, err)
}
if !native.Bool(v) {
t.Errorf("false: %s", s)
}
}
}
+6 -4
View File
@@ -1,13 +1,15 @@
package xhr
import (
"github.com/komari-monitor/komari/pkg/jsruntime/native"
)
import (
"fmt"
"github.com/dop251/goja"
"github.com/Calcium-Ion/moejs"
)
func Inject(vm *goja.Runtime) error {
if _, err := vm.RunString(xhrAPISource); err != nil {
func Inject(vm *moejs.Runtime) error {
if _, err := native.Run(vm, xhrAPISource); err != nil {
return fmt.Errorf("inject XMLHttpRequest: %w", err)
}
return nil
+189 -71
View File
@@ -1,10 +1,11 @@
package metric
import (
"cmp"
"context"
"database/sql"
"fmt"
"sort"
"slices"
"strings"
"time"
)
@@ -27,12 +28,13 @@ type seriesIdentity struct {
}
type seriesReadMeta struct {
id int64
metricName string
entityID string
tagsHash string
tagsJSON string
tags map[string]string
outputIdentity *SeriesIdentity
id int64
metricName string
entityID string
tagsHash string
tagsJSON string
tags map[string]string
}
type rollupRateSample struct {
@@ -41,19 +43,23 @@ type rollupRateSample struct {
}
type rollupAggregateState struct {
summary *rollupBucket
rateSamples []rollupRateSample
tags map[string]string
outputIdentity *SeriesIdentity
summary *queryRollupSummary
rateSamples []rollupRateSample
tags map[string]string
}
type metricSeriesAccumulator struct {
spec BatchSeriesSpec
compression float64
needDigest bool
needSummary bool
needRate bool
groups map[rollupKey]*rollupAggregateState
rateGroups map[rollupKey]*rollupAggregateState
includeSeriesIdentity bool
entityIdentities map[string]*SeriesIdentity
spec BatchSeriesSpec
compression float64
needDigest bool
needSummary bool
needRate bool
groups map[rollupKey]*rollupAggregateState
rateGroups map[rollupKey]*rollupAggregateState
states *queryStateArena
}
// SeriesBatch is the single rollup query implementation. It resolves metric
@@ -80,27 +86,50 @@ func (s *Store) GetMetrics(ctx context.Context, metricNames []string) (map[strin
return s.loadBatchDefinitions(ctx, metricNames)
}
// GetMetricsCompact loads only execution metadata: Name, Type, and RetentionDays.
// GetMetrics continues to return complete definitions.
func (s *Store) GetMetricsCompact(ctx context.Context, metricNames []string) (map[string]Definition, error) {
if err := s.ensureOpen(); err != nil {
return nil, err
}
if len(metricNames) == 0 {
return map[string]Definition{}, nil
}
return s.loadBatchDefinitionsProjected(ctx, metricNames, true)
}
func (s *Store) seriesBatchAt(ctx context.Context, query BatchSeriesQuery, now time.Time, forcedResolutions map[string]time.Duration) (BatchSeriesResult, error) {
metricNames := make([]string, 0, len(query.Specs))
for _, spec := range query.Specs {
metricNames = append(metricNames, spec.MetricName)
}
definitions, err := s.loadBatchDefinitions(ctx, metricNames)
definitions, err := s.loadBatchDefinitionsProjected(ctx, metricNames, query.CompactDefinitions)
if err != nil {
return BatchSeriesResult{}, err
}
return s.seriesBatchDefined(ctx, query, now, forcedResolutions, definitions)
}
func (s *Store) seriesBatchDefined(ctx context.Context, query BatchSeriesQuery, now time.Time, forcedResolutions map[string]time.Duration, definitions map[string]Definition) (BatchSeriesResult, error) {
metricNames := make([]string, 0, len(query.Specs))
for _, spec := range query.Specs {
metricNames = append(metricNames, spec.MetricName)
}
result := BatchSeriesResult{
Definitions: definitions,
Values: make(map[string]map[Aggregation][]AggregatePoint, len(query.Specs)),
}
var states queryStateArena
defer states.release() // Includes SQL, decoding, cancellation and output errors.
accumulators := make(map[string]*metricSeriesAccumulator, len(query.Specs))
groups := make(map[batchSeriesGroupKey]*batchSeriesGroup)
for _, spec := range query.Specs {
values := make(map[Aggregation][]AggregatePoint, len(spec.Aggregations))
accumulator := &metricSeriesAccumulator{
spec: spec, compression: s.cfg.RollupPolicy.compression(),
groups: make(map[rollupKey]*rollupAggregateState), rateGroups: make(map[rollupKey]*rollupAggregateState),
spec: spec, compression: s.cfg.RollupPolicy.compression(), states: &states,
includeSeriesIdentity: query.IncludeSeriesIdentity,
groups: make(map[rollupKey]*rollupAggregateState), rateGroups: make(map[rollupKey]*rollupAggregateState),
}
for _, aggregation := range spec.Aggregations {
values[aggregation] = []AggregatePoint{}
@@ -126,9 +155,9 @@ func (s *Store) seriesBatchAt(ctx context.Context, query BatchSeriesQuery, now t
if len(policy.Tiers) == 0 {
continue
}
resolution := seriesResolutionForPolicy(query.Start, spec.Interval, now, policy)
if forced, ok := forcedResolutions[spec.MetricName]; ok {
resolution = forced
resolution, forced := forcedResolutions[spec.MetricName]
if !forced {
resolution = seriesResolutionForPolicy(query.Start, spec.Interval, now, policy)
}
key := batchSeriesGroupKey{resolution: resolution, needDigest: accumulator.needDigest}
group := groups[key]
@@ -140,11 +169,12 @@ func (s *Store) seriesBatchAt(ctx context.Context, query BatchSeriesQuery, now t
group.fields |= rollupFieldsForAggregations(spec.Aggregations)
}
seriesByID, seriesByIdentity, err := s.loadSeriesDictionary(ctx, seriesDictionaryPlan{
MetricNames: metricNames,
EntityIDs: query.EntityIDs,
Tags: query.Tags,
})
// Dictionary IDs can be created by a hot-to-persisted flush. Resolve them
// under the same view lock as both scans so no continuously present series
// disappears between dictionary lookup and backing-tier selection.
s.rollupViewMu.RLock()
defer s.rollupViewMu.RUnlock()
seriesByID, seriesByIdentity, err := s.loadSeriesDictionary(ctx, seriesDictionaryPlan{MetricNames: metricNames, EntityIDs: query.EntityIDs, Tags: query.Tags, OmitTags: query.OmitTags})
if err != nil {
return BatchSeriesResult{}, err
}
@@ -153,15 +183,19 @@ func (s *Store) seriesBatchAt(ctx context.Context, query BatchSeriesQuery, now t
for _, group := range groups {
orderedGroups = append(orderedGroups, group)
}
sort.Slice(orderedGroups, func(i, j int) bool {
if orderedGroups[i].key.resolution != orderedGroups[j].key.resolution {
return orderedGroups[i].key.resolution < orderedGroups[j].key.resolution
slices.SortFunc(orderedGroups, func(a, b *batchSeriesGroup) int {
if c := cmp.Compare(a.key.resolution, b.key.resolution); c != 0 {
return c
}
return !orderedGroups[i].key.needDigest && orderedGroups[j].key.needDigest
if a.key.needDigest == b.key.needDigest {
return 0
}
if a.key.needDigest {
return 1
}
return -1
})
s.rollupViewMu.RLock()
defer s.rollupViewMu.RUnlock()
for _, group := range orderedGroups {
if err := s.scanPersistedRollupGroup(ctx, query, group, seriesByID, accumulators); err != nil {
return BatchSeriesResult{}, err
@@ -182,15 +216,23 @@ func (s *Store) seriesBatchAt(ctx context.Context, query BatchSeriesQuery, now t
}
func (s *Store) loadBatchDefinitions(ctx context.Context, metricNames []string) (map[string]Definition, error) {
return s.loadBatchDefinitionsProjected(ctx, metricNames, false)
}
func (s *Store) loadBatchDefinitionsProjected(ctx context.Context, metricNames []string, compact bool) (map[string]Definition, error) {
args := make([]any, 0, len(metricNames))
placeholders := make([]string, 0, len(metricNames))
for _, metricName := range metricNames {
args = append(args, metricName)
placeholders = append(placeholders, s.dialect.placeholder(len(args)))
}
columns := "name, type, unit, description, retention_days, metadata, created_at_milli, updated_at_milli"
if compact {
columns = "name, type, retention_days"
}
rows, err := s.reader().QueryContext(ctx, fmt.Sprintf(
"SELECT name, type, unit, description, retention_days, metadata, created_at_milli, updated_at_milli FROM %s WHERE name IN (%s) ORDER BY name ASC",
s.tables.definitions, strings.Join(placeholders, ", "),
"SELECT %s FROM %s WHERE name IN (%s) ORDER BY name ASC",
columns, s.tables.definitions, strings.Join(placeholders, ", "),
), args...)
if err != nil {
return nil, err
@@ -198,7 +240,13 @@ func (s *Store) loadBatchDefinitions(ctx context.Context, metricNames []string)
defer rows.Close()
definitions := make(map[string]Definition, len(metricNames))
for rows.Next() {
definition, err := scanDefinition(rows)
var definition Definition
var err error
if compact {
err = rows.Scan(&definition.Name, &definition.Type, &definition.RetentionDays)
} else {
definition, err = scanDefinition(rows)
}
if err != nil {
return nil, err
}
@@ -219,16 +267,22 @@ func (s *Store) loadSeriesDictionary(ctx context.Context, plan seriesDictionaryP
for rows.Next() {
var meta seriesReadMeta
var rawTags any
if err := rows.Scan(&meta.id, &meta.metricName, &meta.entityID, &meta.tagsHash, &rawTags); err != nil {
targets := []any{&meta.id, &meta.metricName, &meta.entityID, &meta.tagsHash}
if !plan.OmitTags {
targets = append(targets, &rawTags)
}
if err := rows.Scan(targets...); err != nil {
return nil, nil, err
}
meta.tagsJSON, err = rawJSONToString(rawTags)
if err != nil {
return nil, nil, err
}
meta.tags, err = rollupTagsFromJSON(meta.tagsJSON)
if err != nil {
return nil, nil, err
if !plan.OmitTags {
meta.tagsJSON, err = rawJSONToString(rawTags)
if err != nil {
return nil, nil, err
}
meta.tags, err = rollupTagsFromJSON(meta.tagsJSON)
if err != nil {
return nil, nil, err
}
}
byID[meta.id] = &meta
byIdentity[seriesIdentity{metricName: meta.metricName, entityID: meta.entityID, tagsHash: meta.tagsHash}] = &meta
@@ -246,18 +300,45 @@ func (s *Store) scanPersistedRollupGroup(ctx context.Context, query BatchSeriesQ
if len(seriesIDs) == 0 {
return nil
}
sort.Slice(seriesIDs, func(i, j int) bool { return seriesIDs[i] < seriesIDs[j] })
slices.Sort(seriesIDs)
resolutionID, found, err := s.resolveResolutionID(ctx, group.key.resolution)
if err != nil || !found {
return err
}
rendered := s.dialect.renderRollupRead(s.tables, s.cfg.TablePrefix+"rollups_resolution_bucket_idx", rollupReadPlan{
plan := rollupReadPlan{
SeriesIDs: seriesIDs,
ResolutionID: resolutionID,
StartMilli: bucketStartMillis(query.Start.UnixMilli(), group.key.resolution.Milliseconds()),
EndMilli: query.End.UnixMilli(),
Fields: group.fields,
}
indexName := plan.indexName(s.cfg.TablePrefix)
return forEachRollupReadPlan(s.cfg.Driver, plan, func(chunk rollupReadPlan) error {
return s.scanPersistedRollupPlan(ctx, chunk, indexName, seriesByID, accumulators)
})
}
// MySQL prepared statements allow at most 65535 parameters. Leave headroom for
// fixed predicates and future fields; other dialects bind the series as one array.
const mysqlRollupSeriesChunkSize = 60000
func forEachRollupReadPlan(driver Driver, plan rollupReadPlan, consume func(rollupReadPlan) error) error {
chunkSize := len(plan.SeriesIDs)
if driver == DriverMySQL && chunkSize > mysqlRollupSeriesChunkSize {
chunkSize = mysqlRollupSeriesChunkSize
}
for start := 0; start < len(plan.SeriesIDs); start += chunkSize {
chunk := plan
chunk.SeriesIDs = plan.SeriesIDs[start:min(start+chunkSize, len(plan.SeriesIDs))]
if err := consume(chunk); err != nil {
return err
}
}
return nil
}
func (s *Store) scanPersistedRollupPlan(ctx context.Context, plan rollupReadPlan, indexName string, seriesByID map[int64]*seriesReadMeta, accumulators map[string]*metricSeriesAccumulator) error {
rendered := s.dialect.renderRollupRead(s.tables, indexName, plan)
rows, err := s.reader().QueryContext(ctx, rendered.Query, rendered.Args...)
if err != nil {
return err
@@ -267,25 +348,25 @@ func (s *Store) scanPersistedRollupGroup(ctx context.Context, query BatchSeriesQ
var sum, sumSq, min, max, firstVal, lastVal float64
var digestBlob []byte
destinations := []any{&seriesID, &bucket, &count}
if group.fields&rollupReadSum != 0 {
if plan.Fields&rollupReadSum != 0 {
destinations = append(destinations, &sum)
}
if group.fields&rollupReadSumSq != 0 {
if plan.Fields&rollupReadSumSq != 0 {
destinations = append(destinations, &sumSq)
}
if group.fields&rollupReadMin != 0 {
if plan.Fields&rollupReadMin != 0 {
destinations = append(destinations, &min)
}
if group.fields&rollupReadMax != 0 {
if plan.Fields&rollupReadMax != 0 {
destinations = append(destinations, &max)
}
if group.fields&rollupReadFirst != 0 {
if plan.Fields&rollupReadFirst != 0 {
destinations = append(destinations, &firstVal, &firstTS)
}
if group.fields&rollupReadLast != 0 {
if plan.Fields&rollupReadLast != 0 {
destinations = append(destinations, &lastVal, &lastTS)
}
if group.fields&rollupReadDigest != 0 {
if plan.Fields&rollupReadDigest != 0 {
destinations = append(destinations, &digestBlob)
}
for rows.Next() {
@@ -364,7 +445,7 @@ func (s *Store) scanHotRollupGroup(query BatchSeriesQuery, group *batchSeriesGro
continue
}
}
meta, matched, err := memorySeriesMeta(seriesByIdentity, key.metricName, key.entityID, key.tagsHash, bucket.tagsJSON, query.Tags)
meta, matched, err := memorySeriesMeta(seriesByIdentity, key.metricName, key.entityID, key.tagsHash, bucket.tagsJSON, query.Tags, query.OmitTags)
if err != nil {
return err
}
@@ -403,7 +484,7 @@ func (s *Store) scanCoarseRollupGroup(query BatchSeriesQuery, group *batchSeries
}
sortRollupKeys(childKeys)
firstChild := parent.children[childKeys[0]]
meta, matched, err := memorySeriesMeta(seriesByIdentity, key.metricName, key.entityID, key.tagsHash, firstChild.tagsJSON, query.Tags)
meta, matched, err := memorySeriesMeta(seriesByIdentity, key.metricName, key.entityID, key.tagsHash, firstChild.tagsJSON, query.Tags, query.OmitTags)
if err != nil {
return err
}
@@ -420,20 +501,27 @@ func (s *Store) scanCoarseRollupGroup(query BatchSeriesQuery, group *batchSeries
return nil
}
func memorySeriesMeta(seriesByIdentity map[seriesIdentity]*seriesReadMeta, metricName, entityID, tagsHash, tagsJSON string, filter map[string]string) (*seriesReadMeta, bool, error) {
func memorySeriesMeta(seriesByIdentity map[seriesIdentity]*seriesReadMeta, metricName, entityID, tagsHash, tagsJSON string, filter map[string]string, omitTags bool) (*seriesReadMeta, bool, error) {
identity := seriesIdentity{metricName: metricName, entityID: entityID, tagsHash: tagsHash}
if meta := seriesByIdentity[identity]; meta != nil {
return meta, true, nil
}
tags, err := rollupTagsFromJSON(tagsJSON)
if err != nil {
return nil, false, err
var tags map[string]string
if !omitTags || len(filter) > 0 {
var err error
tags, err = rollupTagsFromJSON(tagsJSON)
if err != nil {
return nil, false, err
}
}
for key, value := range filter {
if tags[key] != value {
return nil, false, nil
}
}
if omitTags {
tags = nil
}
meta := &seriesReadMeta{metricName: metricName, entityID: entityID, tagsHash: tagsHash, tagsJSON: tagsJSON, tags: tags}
seriesByIdentity[identity] = meta
return meta, true, nil
@@ -449,20 +537,23 @@ func stringSet(values []string) map[string]struct{} {
func (a *metricSeriesAccumulator) consume(meta *seriesReadMeta, sourceBucket, count int64, sum, sumSq, min, max, firstVal float64, firstTS int64, lastVal float64, lastTS int64, digest *TDigest) *rollupAggregateState {
key := rollupKey{bucket: bucketStartMillis(sourceBucket, a.spec.Interval.Milliseconds())}
if a.spec.PreserveSeries {
if a.spec.PreserveSeries || a.spec.GroupByEntity {
key.entityID = meta.entityID
}
if a.spec.PreserveSeries {
key.tagsHash = meta.tagsHash
}
if a.needSummary {
state := a.groups[key]
if state == nil {
state = &rollupAggregateState{summary: newRollupBucketWithDigest(a.compression, a.needDigest)}
state = a.states.acquire(a.compression, true, a.needDigest)
if a.spec.PreserveSeries {
state.tags = meta.tags
}
state.outputIdentity = a.outputIdentity(meta, key.entityID, a.spec.PreserveSeries)
a.groups[key] = state
}
mergeRollupValues(state.summary, count, sum, sumSq, min, max, firstVal, firstTS, lastVal, lastTS, digest, a.needDigest)
state.summary.merge(count, sum, sumSq, min, max, firstVal, firstTS, lastVal, lastTS, digest, a.needDigest)
if !a.needRate {
return state
}
@@ -475,7 +566,9 @@ func (a *metricSeriesAccumulator) consume(meta *seriesReadMeta, sourceBucket, co
}
state := a.rateGroups[rateKey]
if state == nil {
state = &rollupAggregateState{tags: meta.tags}
// Rate-only state is smaller than a summary slot; do not reserve an
// unused summary/digest for every rate output bucket.
state = &rollupAggregateState{tags: meta.tags, outputIdentity: a.outputIdentity(meta, meta.entityID, true)}
a.rateGroups[rateKey] = state
}
state.rateSamples = append(state.rateSamples, rollupRateSample{timestamp: lastTS, value: lastVal})
@@ -483,6 +576,30 @@ func (a *metricSeriesAccumulator) consume(meta *seriesReadMeta, sourceBucket, co
return a.groups[key]
}
// outputIdentity reuses the dictionary series snapshot across all buckets and
// aggregations. Merged/entity output has nil Tags and is cached separately;
// using tagsHash here would wrongly split omitted Tags and risk hash collisions.
func (a *metricSeriesAccumulator) outputIdentity(meta *seriesReadMeta, entity string, preserve bool) *SeriesIdentity {
if !a.includeSeriesIdentity {
return nil
}
if preserve {
if meta.outputIdentity == nil {
meta.outputIdentity = NewSeriesIdentity(entity, meta.tags)
}
return meta.outputIdentity
}
if a.entityIdentities == nil {
a.entityIdentities = make(map[string]*SeriesIdentity)
}
identity := a.entityIdentities[entity]
if identity == nil {
identity = NewSeriesIdentity(entity, nil)
a.entityIdentities[entity] = identity
}
return identity
}
func mergeEncodedRollupDigest(target *TDigest, count int64, min, max float64, blob []byte) error {
if count == 0 {
return nil
@@ -547,7 +664,7 @@ func (a *metricSeriesAccumulator) points(order Order) (map[Aggregation][]Aggrega
rateKeys = make([]rollupKey, 0, len(a.rateGroups))
for key, state := range a.rateGroups {
rateKeys = append(rateKeys, key)
sort.SliceStable(state.rateSamples, func(i, j int) bool { return state.rateSamples[i].timestamp < state.rateSamples[j].timestamp })
slices.SortStableFunc(state.rateSamples, func(a, b rollupRateSample) int { return cmp.Compare(a.timestamp, b.timestamp) })
}
sortRollupKeys(rateKeys)
}
@@ -578,12 +695,13 @@ func (a *metricSeriesAccumulator) points(order Order) (map[Aggregation][]Aggrega
count = int(state.summary.count)
}
points = append(points, AggregatePoint{
MetricName: a.spec.MetricName,
EntityID: key.entityID,
Bucket: fromMillis(key.bucket),
Value: value,
Count: count,
Tags: state.tags,
SeriesIdentity: state.outputIdentity,
MetricName: a.spec.MetricName,
EntityID: key.entityID,
Bucket: fromMillis(key.bucket),
Value: value,
Count: count,
Tags: state.tags,
})
}
values[aggregation] = points
+27 -2
View File
@@ -3,6 +3,7 @@ package metric
import (
"context"
"database/sql"
"math"
"sort"
"time"
)
@@ -95,6 +96,7 @@ func (s *Store) addCoarseChildren(metricName string, interval time.Duration, hig
children: make(map[rollupKey]*rollupBucket),
}
s.coarse[parentKey] = parent
s.noteCoarseCloseLocked(parentKey)
}
parent.children[childKey] = child.clone(true)
}
@@ -168,9 +170,15 @@ func (s *Store) filterClosedCoarseRollups(ctx context.Context, closed []closedCo
func (s *Store) takeClosedCoarseRollups(now time.Time) []closedCoarseRollup {
s.coarseMu.Lock()
defer s.coarseMu.Unlock()
nowMilli := now.UnixMilli()
if s.coarseNextClose > nowMilli {
return nil
}
var interval time.Duration
nextClose := int64(math.MaxInt64)
for key := range s.coarse {
if key.bucket+key.interval.Milliseconds()+coarseRollupGrace.Milliseconds() > now.UnixMilli() {
if closeAt := coarseCloseAt(key); closeAt > nowMilli {
nextClose = min(nextClose, closeAt)
continue
}
if interval == 0 || key.interval < interval {
@@ -178,11 +186,14 @@ func (s *Store) takeClosedCoarseRollups(now time.Time) []closedCoarseRollup {
}
}
if interval == 0 {
s.coarseNextClose = nextClose
return nil
}
// coarseNextClose stays <= now: closable parents of coarser tiers remain
// and the caller takes them on its next pass.
closed := make([]closedCoarseRollup, 0)
for key, parent := range s.coarse {
if key.interval != interval || key.bucket+key.interval.Milliseconds()+coarseRollupGrace.Milliseconds() > now.UnixMilli() {
if key.interval != interval || coarseCloseAt(key) > nowMilli {
continue
}
bucket := newRollupBucket(s.cfg.RollupPolicy.compression())
@@ -206,6 +217,19 @@ func (s *Store) takeClosedCoarseRollups(now time.Time) []closedCoarseRollup {
return closed
}
// coarseCloseAt is when a parent leaves its late-arrival window.
func coarseCloseAt(key coarseRollupKey) int64 {
return key.bucket + key.interval.Milliseconds() + coarseRollupGrace.Milliseconds()
}
// noteCoarseCloseLocked keeps coarseNextClose a lower bound after inserting
// a parent. The caller must hold coarseMu.
func (s *Store) noteCoarseCloseLocked(key coarseRollupKey) {
if closeAt := coarseCloseAt(key); closeAt < s.coarseNextClose {
s.coarseNextClose = closeAt
}
}
func (s *Store) restoreClosedCoarseRollups(closed []closedCoarseRollup) {
s.coarseMu.Lock()
defer s.coarseMu.Unlock()
@@ -213,6 +237,7 @@ func (s *Store) restoreClosedCoarseRollups(closed []closedCoarseRollup) {
if _, exists := s.coarse[item.key]; exists {
continue
}
s.noteCoarseCloseLocked(item.key)
s.coarse[item.key] = &coarseRollup{
higher: append([]time.Duration(nil), item.higher...),
children: item.children,
+7 -1
View File
@@ -158,7 +158,13 @@ func DefaultConfig(driver Driver, dsn string) Config {
SQLite: SQLiteOptions{
PerformanceProfile: SQLiteProfileBalanced,
BusyTimeout: 5 * time.Second,
CacheSizeKB: 4 * 1024,
// 16 MB per connection: the rollup read plans walk one time range at a
// time, and a small page cache turns that walk into repeated pager
// fetches. Override with WithSQLiteCacheSizeKB when memory is tighter.
//
// 每连接 16 MB:rollup 读计划按时间窗连续扫描,页缓存过小会让扫描不断
// 重复取页。内存更紧张时可用 WithSQLiteCacheSizeKB 覆盖。
CacheSizeKB: 16 * 1024,
TempStoreMemory: false,
MMapSizeBytes: 0,
WALAutoCheckpoint: 4000,
+27 -8
View File
@@ -2,6 +2,7 @@ package metric
import (
"context"
"math"
"sort"
"time"
)
@@ -44,6 +45,7 @@ func (s *Store) writePreparedHotRollups(ctx context.Context, prepared []prepared
bucket.tagsHash, bucket.tagsJSON = point.tagsHash, point.tagsJSON
bucket.labelsHash, bucket.labelsJSON = point.labelsHash, point.labelsJSON
s.hot[key] = bucket
s.noteHotBucketLocked(key.bucket)
}
bucket.addPoint(point.value, point.timestamp)
}
@@ -109,6 +111,7 @@ func (s *Store) rebuildHotRollupsLocked(keys map[hotRollupKey]struct{}, compress
}
s.hot[key] = bucket
s.hotReplace[key] = struct{}{}
s.noteHotBucketLocked(key.bucket)
}
}
@@ -156,18 +159,33 @@ func (s *Store) flushAllHotRollups(ctx context.Context) error {
func (s *Store) takeClosedHotRollups(currentBucket int64) []hotRollup {
s.hotMu.Lock()
defer s.hotMu.Unlock()
closed := make([]hotRollup, 0)
for key, bucket := range s.hot {
if key.bucket < currentBucket {
_, replace := s.hotReplace[key]
closed = append(closed, hotRollup{key: key, bucket: bucket, replace: replace})
delete(s.hot, key)
delete(s.hotReplace, key)
}
if s.hotOldest >= currentBucket {
return nil
}
closed := make([]hotRollup, 0)
oldest := int64(math.MaxInt64)
for key, bucket := range s.hot {
if key.bucket >= currentBucket {
oldest = min(oldest, key.bucket)
continue
}
_, replace := s.hotReplace[key]
closed = append(closed, hotRollup{key: key, bucket: bucket, replace: replace})
delete(s.hot, key)
delete(s.hotReplace, key)
}
s.hotOldest = oldest
return closed
}
// noteHotBucketLocked keeps hotOldest a lower bound after inserting a key.
// The caller must hold hotMu.
func (s *Store) noteHotBucketLocked(bucket int64) {
if bucket < s.hotOldest {
s.hotOldest = bucket
}
}
func (s *Store) restoreHotRollups(closed []hotRollup) {
s.hotMu.Lock()
defer s.hotMu.Unlock()
@@ -176,6 +194,7 @@ func (s *Store) restoreHotRollups(closed []hotRollup) {
_, currentReplaces := s.hotReplace[item.key]
if !exists {
s.hot[item.key] = item.bucket
s.noteHotBucketLocked(item.key.bucket)
if item.replace {
s.hotReplace[item.key] = struct{}{}
}
+204
View File
@@ -0,0 +1,204 @@
package metric
import (
"context"
"database/sql"
"fmt"
"strings"
)
// normalizedIndex 的形状修复。
//
// CREATE INDEX IF NOT EXISTS 在索引名已存在时是空操作,所以只改
// normalizedIndexesFor 里的列定义只会影响新建的数据库:已有库会留着旧的窄
// 索引,同时静默丢掉覆盖读计划(series 过滤回落到逐候选行回表)。Migrate
// 因此需要把库里存的索引定义和声明的定义比对一次,不一致就重建。
//
// 这里只比对按规范名管理的索引;手工添加的索引不会被删除。
// rebuildStaleIndexes drops normalized indexes whose stored column list no
// longer matches the declared one, so createNormalizedIndexes rebuilds them.
//
// rebuildStaleIndexes 删除列定义已经过时的规范化索引,交给后续创建步骤重建。
func (s *Store) rebuildStaleIndexes(ctx context.Context) error {
for _, index := range s.normalizedIndexes() {
existing, found, err := s.indexColumns(ctx, index)
if err != nil {
return err
}
if !found || sameIndexColumns(existing, index.columns) {
continue
}
if err := s.rebuildIndex(ctx, index); err != nil {
return err
}
}
return nil
}
// rebuildIndex replaces one stale definition.
//
// MySQL refuses to drop an index that a foreign key still relies on: the rollups
// table references resolutions and the only index leading with resolution_id is
// the one being replaced. The replacement is therefore created under a temporary
// name first, which satisfies the constraint immediately, and only then does the
// old index go away and the new one take the canonical name.
//
// rebuildIndex 替换一个过时的索引定义。MySQL 不允许删除仍被外键依赖的索引,所以
// 先以临时名建好新索引(立刻满足外键约束),再删旧的,最后把新索引改成正式名。
func (s *Store) rebuildIndex(ctx context.Context, index normalizedIndex) error {
if s.cfg.Driver != DriverMySQL {
return s.dropIndex(ctx, index)
}
staging := index.name + "_rebuild"
// Clear a staging index left behind by an interrupted run.
if err := s.dropIndex(ctx, normalizedIndex{name: staging, table: index.table}); err != nil {
return err
}
if _, err := s.db.ExecContext(ctx, fmt.Sprintf("CREATE INDEX %s ON %s (%s)", staging, index.table, index.columns)); err != nil {
return err
}
if _, err := s.db.ExecContext(ctx, fmt.Sprintf("ALTER TABLE %s DROP INDEX %s", index.table, index.name)); err != nil {
// The staged index stays behind and the next start retries cleanly.
return err
}
_, err := s.db.ExecContext(ctx, fmt.Sprintf("ALTER TABLE %s RENAME INDEX %s TO %s", index.table, staging, index.name))
return err
}
// indexColumnsQuery returns the introspection statement for one backend. It is
// separate from execution so the placeholder style of every dialect can be
// asserted without a server: SQLite and MySQL bind "?" while PostgreSQL binds
// "$1", and sending "?" to PostgreSQL fails to parse.
//
// indexColumnsQuery 返回对应后端的索引列探测语句。与执行分离,便于在没有服务端
// 的情况下断言占位符风格:SQLite 与 MySQL 用 "?",PostgreSQL 用 "$1"。
func (s *Store) indexColumnsQuery(index normalizedIndex) (string, []any) {
switch s.cfg.Driver {
case DriverMySQL:
return "SELECT COLUMN_NAME FROM information_schema.STATISTICS " +
"WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = ? AND INDEX_NAME = ? " +
"ORDER BY SEQ_IN_INDEX", []any{index.table, index.name}
case DriverPostgreSQL:
// pg_get_indexdef returns the stored DDL, which parses with the same
// helper as SQLite and avoids depending on pg_index.indkey ordering and
// int2vector casting, both of which vary across server versions.
//
// pg_get_indexdef 直接返回索引 DDL,可与 SQLite 共用解析器,避免依赖
// pg_index.indkey 的排序与 int2vector 转换(这两者在不同版本上行为不同)。
return "SELECT pg_get_indexdef(c.oid) FROM pg_class c " +
"JOIN pg_namespace n ON n.oid = c.relnamespace " +
"WHERE c.relname = " + s.dialect.placeholder(1) + " AND c.relkind = 'i'",
[]any{index.name}
default:
return "SELECT sql FROM sqlite_master WHERE type = 'index' AND name = ?", []any{index.name}
}
}
// indexColumns reads the stored column list for one index. found is false when
// the index does not exist yet, which is not an error.
//
// indexColumns 读取一个索引当前存储的列顺序;found 为 false 表示索引还不存在。
func (s *Store) indexColumns(ctx context.Context, index normalizedIndex) ([]string, bool, error) {
query, args := s.indexColumnsQuery(index)
if s.cfg.Driver == DriverMySQL {
rows, err := s.db.QueryContext(ctx, query, args...)
if err != nil {
return nil, false, err
}
return collectIndexColumns(rows)
}
var statement sql.NullString
err := s.db.QueryRowContext(ctx, query, args...).Scan(&statement)
if err == sql.ErrNoRows || !statement.Valid {
return nil, false, nil
}
if err != nil {
return nil, false, err
}
return indexColumnsFromDDL(statement.String), true, nil
}
func collectIndexColumns(rows *sql.Rows) ([]string, bool, error) {
defer rows.Close()
columns := make([]string, 0, 4)
for rows.Next() {
var column string
if err := rows.Scan(&column); err != nil {
return nil, false, err
}
columns = append(columns, normalizeIndexColumn(column))
}
if err := rows.Err(); err != nil {
return nil, false, err
}
if len(columns) == 0 {
return nil, false, nil
}
return columns, true, nil
}
// indexColumnsFromDDL extracts the column list from a CREATE INDEX statement.
// SQLite and PostgreSQL both expose the stored DDL, so both take this path.
// Entries keep their first token, enough to detect a stale definition.
//
// indexColumnsFromDDL 从 CREATE INDEX 语句中提取列定义;SQLite 与 PostgreSQL
// 都直接暴露存储的 DDL,因此共用这条路径。
func indexColumnsFromDDL(statement string) []string {
open := strings.Index(statement, "(")
closing := strings.LastIndex(statement, ")")
if open < 0 || closing <= open {
return nil
}
parts := strings.Split(statement[open+1:closing], ",")
columns := make([]string, 0, len(parts))
for _, part := range parts {
columns = append(columns, normalizeIndexColumn(part))
}
return columns
}
func normalizeIndexColumn(column string) string {
column = strings.ToLower(strings.TrimSpace(column))
// Drop ASC/DESC, COLLATE, and operator-class suffixes before unquoting, so a
// quoted name followed by a direction still reduces to the bare column.
if fields := strings.Fields(column); len(fields) > 0 {
column = fields[0]
}
return strings.Trim(column, "\"\x60'[]")
}
// sameIndexColumns compares a stored column list with a declared one.
//
// sameIndexColumns 比较存储的列顺序和声明的列顺序是否一致。
func sameIndexColumns(existing []string, declared string) bool {
want := strings.Split(declared, ",")
if len(existing) != len(want) {
return false
}
for i, column := range want {
if existing[i] != normalizeIndexColumn(column) {
return false
}
}
return true
}
// dropIndex removes one normalized index when it exists.
//
// dropIndex 删除一个已存在的规范化索引。
func (s *Store) dropIndex(ctx context.Context, index normalizedIndex) error {
if s.cfg.Driver == DriverMySQL {
exists, err := s.mysqlIndexExists(ctx, index.table, index.name)
if err != nil {
return err
}
if !exists {
return nil
}
_, err = s.db.ExecContext(ctx, fmt.Sprintf("ALTER TABLE %s DROP INDEX %s", index.table, index.name))
return err
}
_, err := s.db.ExecContext(ctx, "DROP INDEX IF EXISTS "+index.name)
return err
}
+8
View File
@@ -73,7 +73,15 @@ func canonicalTags(m map[string]string) (string, error) {
// tags_hash key 列。相同标签 map(无论 Go map 迭代顺序如何)都会得到相同 hash,
// 因此每种不同标签组合都会成为自己的 rollup 序列;空或 nil 标签 map 会得到 "{}"
// 的指纹。
const (
emptyTagsHash = "44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a"
emptyTagsJSON = "{}"
)
func tagsFingerprint(m map[string]string) (hash string, canonical string, err error) {
if len(m) == 0 {
return emptyTagsHash, emptyTagsJSON, nil
}
canonical, err = canonicalTags(m)
if err != nil {
return "", "", err
Loaded 100 of 150 files, more files were not shown because too many files have changed in this diff. Show more