Compare commits
16
Commits
main
...
feat-chv-p1fix
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
971e054938 | ||
|
|
8c8d62f70d | ||
|
|
e718f0f326 | ||
|
|
54f20dfe55 | ||
|
|
2f3ff8b3ec | ||
|
|
df8c64bd64 | ||
|
|
42de20da68 | ||
|
|
c908a0147a | ||
|
|
baf26beb78 | ||
|
|
003ea357f9 | ||
|
|
036323e620 | ||
|
|
445376743d | ||
|
|
9771ed9e8c | ||
|
|
22b83f118a | ||
|
|
fde7bf4c3e | ||
|
|
c371bbdf1e |
No files matched your search
@@ -199,6 +199,19 @@ python3 -m cloud.server # 默认 127.0.0.1:8765,生产经 nginx + TLS
|
||||
# 除 GET /health、/webhooks/*、/oauth/* 外,所有端点都要 X-API-Key 请求头
|
||||
# 任务实时链路:GET /tasks/{id}/trace(快照)与 GET /tasks/{id}/trace/stream(SSE)
|
||||
# 命令台:GET /console(每次现读 cloud/console.html,无需重启)
|
||||
# CHV 前端入口(实验,默认关闭):新建任务与新建环境的执行器下拉里有
|
||||
# cloud-hypervisor 选项;任务详情与任务列表的执行器筛选同样展示它。
|
||||
# 前端只传 executor 值,安全边界仍在后端白名单(任务接口 allowlist 含
|
||||
# cloud-hypervisor,环境接口 allowlist 为 local/docker/cloud-hypervisor);
|
||||
# 需后端有 KVM 宿主上的 CHV worker 才会认领,否则任务一直排队,带附件
|
||||
# 时建任务即被后端拒绝,拒绝文案经控制台原样展示(“提交失败:…”)。
|
||||
# 生产级前端新增:新建任务页与 Workers 区用现有 /workers 展示 CHV worker
|
||||
# 在线数(非管理员只能看到聚合数时写未知、不写成 0);零 CHV worker 在线
|
||||
# 时选 CHV 出强提示但不硬拦提交(worker 随后上线可认领,后端是最终权威);
|
||||
# 选 CHV 时附件输入禁用并可一键清空;排队中的 CHV 任务在列表与详情里给
|
||||
# “可能原因:无 CHV worker 在线”提示(基于 /workers 推断)。
|
||||
# 详情页只有实验提示,没有 VM 暂停/恢复/关机/快照等控制按钮;VM 状态、
|
||||
# IP、快照、预热池字段后端任务接口不返回,前端不展示、不编造。
|
||||
```
|
||||
|
||||
- **任务状态机**:`queued → running → completed / failed /
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# Cloud Hypervisor 客机镜像(chv-guest)——内容清单与版本化构建说明
|
||||
|
||||
> 本目录有清单与真实构建步骤(build.sh,补缺口轨已实做),
|
||||
> 但镜像本身不在本轨内实际构建发布。原因写在前面,不藏着:
|
||||
> 当前没有带 /dev/kvm 的宿主可以构建并实测 VM 镜像(生产
|
||||
> 主机是 CVM、无嵌套虚拟化;本 VM 容器内同样无 /dev/kvm)。
|
||||
> 构建与实测见 `docs/chv-production-readiness.md` 的“需镜像
|
||||
> 构建 / 需 KVM 宿主实测”项,绝不在本轨声称镜像已建成、
|
||||
> 已发布、已跑通。
|
||||
|
||||
## 1. 镜像定位
|
||||
|
||||
- 基础:Ubuntu 云镜像(与 `cloud/docker/Dockerfile` 的任务镜像
|
||||
对齐能力,不是另起一套环境口径)。
|
||||
- 载体:Cloud Hypervisor 直接内核启动(vmlinux)+ raw 磁盘
|
||||
镜像,执行器用 `CHV_KERNEL` / `CHV_ROOTFS` 指向它们。
|
||||
- 客内 host.py 位置:`/app/host.py`(与 Docker 的 `/app:ro`
|
||||
同款约定,只是载体从容器只读挂载换成 VM 镜像内置代码;
|
||||
代码版本 = 构建时仓库 commit,见 §3 版本化)。
|
||||
- 绝不进镜像的东西:任何模型 key、系统 key、用户账号数据、
|
||||
生产凭证。模型 key 由客机 cloud-init 预置的运行环境变量名
|
||||
注入(与 Docker 的 `-e <名>` 同款惯例),不写进镜像层。
|
||||
|
||||
## 2. 内容清单(按现有 Docker 任务镜像能力对齐)
|
||||
|
||||
| 类别 | 内容 | 来源/版本口径 |
|
||||
| --- | --- | --- |
|
||||
| 语言与工具链 | Python 3.12、git(Node.js 不装:Playwright 驱动是 Python playwright,见 PLAYWRIGHT=1 分支;任务镜像 Dockerfile 里也没有 Node.js,本清单不虚构它) | 与任务镜像同款大版本,构建时钉小版本并记入构建日志 |
|
||||
| 文档/图片工具 | pandoc、poppler-utils(pdftotext)、latexmk、texlive-latex-base、python-docx、openpyxl、python-pptx、PyMuPDF、Pillow | 逐项照抄 `cloud/docker/Dockerfile` 的 apt/pip 清单,不擅自加减 |
|
||||
| agent 引擎 | codex、codex-code-mode-host、codex-resources/bwrap(成套,约 367M,缺 sibling 直接构建失败) | codex 0.164.0-alpha.9(与生产任务镜像一致);`CODEX_BIN_DIR` 由构建方提供,bin/ 不进仓库 |
|
||||
| 浏览器 | Playwright + Chromium(浏览器自动化 MCP 用,PLAYWRIGHT=1 时才装) | Python playwright 装当时的 Chromium,未在 BUILDINFO 里钉浏览器版本——这是已知缺口,不是已钉死 |
|
||||
| 客机基础 | openssh-server(SSH 通道用)、cloud-init、qemu-guest-agent(可选,宿主信息用) | sshd 只允许 key 登录,禁止密码登录——build.sh 写入 /etc/ssh/sshd_config.d/99-chv.conf 并经 sshd -t + grep 断言(feat-chv-secfix,M8,此前只是文档声称) |
|
||||
| 任务代码 | codex-app-server 仓库代码置于 `/app`(host.py 入口) | 版本见 §3,不挂载宿主代码目录(VM 与宿主无共享路径) |
|
||||
|
||||
中文 LaTeX 说明:与 Docker 镜像同款口径——xelatex + CJK
|
||||
字体会再增数百 MB,本清单不默认包含;需要时单独一版镜像,
|
||||
不在本轨悄悄加。
|
||||
|
||||
## 3. 版本化
|
||||
|
||||
镜像 tag 格式(构建时三项缺一即失败,不生成无版本镜像):
|
||||
|
||||
```
|
||||
chv-guest-<ubuntu 版本>-<codex 版本>-<仓库 commit 短 hash>
|
||||
```
|
||||
|
||||
例如(示例格式,非已建成):`chv-guest-24.04-0.164.0-alpha.9-aebc623`。
|
||||
|
||||
构建产物清单(构建方必须输出,执行器用它做 preflight 之外的
|
||||
版本核对,v1 执行器暂不校验 tag,见就绪清单):
|
||||
|
||||
- `vmlinux`(与 Ubuntu 云镜像配套的内核)
|
||||
- `disk.raw`(根文件系统镜像)
|
||||
- `BUILDINFO.json`:`{ubuntu, codex_version, repo_commit, built_at,
|
||||
contents: [清单逐项名]}`——注意:build.sh 生成的 contents 只有
|
||||
名字列表,没有逐项版本号(feat-chv-fix 纠正 §3 旧措辞);
|
||||
逐项版本只留在构建日志里,未进 BUILDINFO,这是已知缺口。
|
||||
|
||||
## 4. 构建脚本(补缺口轨:已从骨架改成真实步骤)
|
||||
|
||||
见 `build.sh`:参数校验 → 下载 Ubuntu 云镜像 → qemu-img
|
||||
转 raw → virt-customize 装 §2 清单、拷入 codex 成套二进制
|
||||
与仓库代码到 /app、写 host key 不重生成配置 →
|
||||
virt-get-kernel 取内核 → 生成并校验 BUILDINFO.json。
|
||||
`./build.sh --dry-run out/` 只打印命令、不执行、不要求
|
||||
KVM;全量模式在无 /dev/kvm 的机器上第一步就明确退出
|
||||
(`CHV_GUEST_BUILD needs a KVM host`),绝不假装构建成功。
|
||||
本轨未在真机上跑过全量模式,镜像未建成、未发布,仍属
|
||||
“需 KVM 宿主实测”项。
|
||||
|
||||
## 5. Host key 首启注入硬性要求(补缺口轨新增)
|
||||
|
||||
执行器的 per-VM 预置方案(见 `cloud/chv/hostkeys.py`)能
|
||||
跑通的前提是客机镜像必须满足以下全部,否则预置在真机
|
||||
上跑不通,执行器会 fail-fast,不回落成弱方案:
|
||||
- 装 cloud-init 且支持 NoCloud seed(seed 磁盘(CIDATA ISO,feat-chv-fix 起挂进 VmConfig
|
||||
第二块只读盘)由执行器在 VM 启动前生成,含 meta-data 与 user-data);
|
||||
- 首启时先应用 seed 再启动 sshd,seed 里的 user-data
|
||||
把该 VM 的 host key 对写进 /etc/ssh/ 并重启 sshd;
|
||||
- 镜像里已钉死 `ssh_deletekeys: false` 与
|
||||
`ssh_genkeytypes: []`(build.sh 已写),首启不得无条件
|
||||
重新生成 host key——重生成会使预置的 known_hosts 立即
|
||||
失效,SSH 钉死直接拒绝,这是设计,不是 bug;
|
||||
- sshd 只允许 key 登录,禁止密码登录(build.sh 写入
|
||||
/etc/ssh/sshd_config.d/99-chv.conf:PasswordAuthentication no、
|
||||
PermitRootLogin prohibit-password、ChallengeResponseAuthentication no、
|
||||
KbdInteractiveAuthentication no、PubkeyAuthentication yes,并经
|
||||
sshd -t + grep 断言;feat-chv-secfix,M8);
|
||||
- seed 磁盘是 per-VM 敏感物,执行器在 VM 关闭/
|
||||
回收时删除,镜像构建方不得把任何一把 host key 固化
|
||||
进镜像(所有 VM 同 key 是安全事故)。
|
||||
|
||||
## 6. 与执行器的对接
|
||||
|
||||
- 执行器 preflight 只校验内核/镜像文件存在且可读,不校验镜像
|
||||
内容版本——内容版本核对(BUILDINFO)是镜像发布流程的责任,
|
||||
在就绪清单里标为“需镜像构建”项,不在本轨假装已做。
|
||||
- 每任务磁盘:执行器 copy 模式下把 `disk.raw` 复制为每任务
|
||||
磁盘,任务之间文件系统独立;reuse 模式仅单租户实验,preflight 拦截(须同时设 CHV_REUSE_SINGLE_TENANT_EXPERIMENT=1 并审计放行,feat-chv-fix)。
|
||||
- 快照基准:`CHV_SNAPSHOT_URL` 指向的快照必须由本镜像的干净
|
||||
启动状态制成(pause 后 snapshot),预热池用它做干净 VM 来源。
|
||||
Executable
+178
@@ -0,0 +1,178 @@
|
||||
#!/bin/bash
|
||||
# chv-guest 镜像构建脚本(补缺口轨:从 TODO 骨架改成真实步骤)。
|
||||
#
|
||||
# 用法(必须在带 /dev/kvm 的宿主上跑全量模式):
|
||||
# CODEX_BIN_DIR=bin UBUNTU_VERSION=24.04 CODEX_VERSION=0.164.0-alpha.9 \
|
||||
# REPO_COMMIT=$(git rev-parse --short HEAD) ./build.sh out/
|
||||
# ./build.sh --dry-run out/ # 只打印命令,不执行、不要求 KVM
|
||||
#
|
||||
# 产物(out/ 内):
|
||||
# vmlinux # virt-get-kernel 从云镜像取出的内核(CHV --kernel 指向它)
|
||||
# disk.raw # qemu-img 转成的 raw 根文件系统镜像(CHV_ROOTFS 指向它)
|
||||
# BUILDINFO.json # {tag, ubuntu, codex_version, repo_commit, built_at, contents}
|
||||
# + versions {dpkg, pip}(构建时从客机读回的实际版本)+
|
||||
# ubuntu_image_sha256_set(SHA256 必填,见下)
|
||||
# 绝不进镜像的东西:任何模型 key、系统 key、用户账号数据、
|
||||
# 生产凭证;模型 key 由客机 cloud-init 预置的运行环境变量名
|
||||
# 注入,不写进镜像层。
|
||||
# feat-chv-secfix(M4/M8):UBUNTU_IMAGE_SHA256 必填;
|
||||
# sshd 加固由构建写入 /etc/ssh/sshd_config.d/99-chv.conf
|
||||
# 并经 sshd -t + grep 断言(README 的“只允许 key 登录”
|
||||
# 从此是构建断言,不再只是文档声称)。
|
||||
#
|
||||
# 客机 host key 硬性要求(见 README §5,补缺口轨新增):
|
||||
# 客机必须装 cloud-init 且支持 NoCloud seed,首启先应用
|
||||
# seed 再启动 sshd,且不得无条件重新生成 host key,否则
|
||||
# 执行器的 per-VM 预置钉死即失效。本脚本在镜像里写
|
||||
# /etc/cloud/cloud.cfg.d/99-chv-hostkey.cfg(ssh_deletekeys:
|
||||
# false、ssh_genkeytypes: []),把“不重生成”钉进镜像。
|
||||
set -eu
|
||||
|
||||
DRY_RUN=0
|
||||
if [ "${1:-}" = "--dry-run" ]; then
|
||||
DRY_RUN=1
|
||||
shift
|
||||
fi
|
||||
OUT="${1:-out}"
|
||||
: "${CODEX_BIN_DIR:?CODEX_BIN_DIR 未设:需 npm 安装 @openai/codex 后的配对二进制目录}"
|
||||
: "${UBUNTU_VERSION:?UBUNTU_VERSION 未设}"
|
||||
: "${CODEX_VERSION:?CODEX_VERSION 未设}"
|
||||
: "${REPO_COMMIT:?REPO_COMMIT 未设:构建时仓库 commit 短 hash}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO_DIR="${REPO_DIR:-$(cd "$SCRIPT_DIR/.." && pwd)}"
|
||||
TAG="chv-guest-${UBUNTU_VERSION}-${CODEX_VERSION}-${REPO_COMMIT}"
|
||||
UBUNTU_IMAGE_URL="${UBUNTU_IMAGE_URL:-https://cloud-images.ubuntu.com/releases/${UBUNTU_VERSION}/release/ubuntu-${UBUNTU_VERSION}-server-cloudimg-amd64.img}"
|
||||
|
||||
run() {
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo "DRY-RUN: $*"
|
||||
else
|
||||
echo "+ $*"
|
||||
"$@"
|
||||
fi
|
||||
}
|
||||
|
||||
# 参数校验(干跑与全量都做,不依赖 KVM)。
|
||||
for bin in codex codex-code-mode-host; do
|
||||
if [ ! -x "$CODEX_BIN_DIR/$bin" ]; then
|
||||
echo "缺少配对二进制:$CODEX_BIN_DIR/$bin,构建失败" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
if [ ! -d "$CODEX_BIN_DIR/codex-resources" ]; then
|
||||
echo "缺少 codex-resources(bwrap 载体),构建失败" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ ! -f "$REPO_DIR/host.py" ] || [ ! -d "$REPO_DIR/cloud" ]; then
|
||||
echo "REPO_DIR 不是 codex-app-server 仓库根(缺 host.py/cloud/):$REPO_DIR" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$DRY_RUN" = "0" ]; then
|
||||
if [ ! -e /dev/kvm ]; then
|
||||
echo "CHV_GUEST_BUILD needs a KVM host: /dev/kvm 不存在" >&2
|
||||
exit 1
|
||||
fi
|
||||
for tool in curl qemu-img virt-customize virt-get-kernel python3 virt-cat; do
|
||||
if ! command -v "$tool" >/dev/null 2>&1; then
|
||||
echo "缺少构建工具:$tool,构建失败" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
run mkdir -p "$OUT"
|
||||
IMG="$OUT/ubuntu-${UBUNTU_VERSION}-server-cloudimg-amd64.img"
|
||||
# feat-chv-secfix(M4):云镜像 SHA256 改为必填(干跑只验
|
||||
# “已设”,不验文件;全量模式下载后验哈希)。未设即失败,
|
||||
# 不再只依赖 HTTPS 传输层作唯一锚。
|
||||
if [ -z "${UBUNTU_IMAGE_SHA256:-}" ]; then
|
||||
echo "UBUNTU_IMAGE_SHA256 未设:云镜像须以哈希锚定,构建失败" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo "DRY-RUN: curl -fL $UBUNTU_IMAGE_URL -o $IMG"
|
||||
echo "DRY-RUN: [ -s $IMG ] || exit 1"
|
||||
echo "DRY-RUN: sha256sum -c (UBUNTU_IMAGE_SHA256)"
|
||||
else
|
||||
if [ ! -s "$IMG" ]; then
|
||||
curl -fL "$UBUNTU_IMAGE_URL" -o "$IMG"
|
||||
fi
|
||||
[ -s "$IMG" ] || { echo "下载的云镜像为空:$IMG" >&2; exit 1; }
|
||||
echo "$UBUNTU_IMAGE_SHA256 $IMG" | sha256sum -c -
|
||||
fi
|
||||
run qemu-img convert -f qcow2 -O raw "$IMG" "$OUT/disk.raw"
|
||||
# 清单软件(照抄 cloud/docker/Dockerfile 的 apt/pip 清单,
|
||||
# 中文 LaTeX xelatex+CJK 故意不装,见 README;Playwright
|
||||
# 默认不装,PLAYWRIGHT=1 时才加装,属可选步骤)。
|
||||
run virt-customize -a "$OUT/disk.raw" \
|
||||
--install git,pandoc,poppler-utils,latexmk,texlive-latex-base,openssh-server,cloud-init,qemu-guest-agent,python3,python3-pip \
|
||||
--run-command 'pip3 install --no-cache-dir --break-system-packages python-docx openpyxl python-pptx PyMuPDF Pillow || pip3 install --no-cache-dir python-docx openpyxl python-pptx PyMuPDF Pillow' \
|
||||
--copy-in "$CODEX_BIN_DIR/codex:/usr/local/bin/" \
|
||||
--copy-in "$CODEX_BIN_DIR/codex-code-mode-host:/usr/local/bin/" \
|
||||
--copy-in "$CODEX_BIN_DIR/codex-resources:/usr/local/bin/" \
|
||||
--run-command 'chmod +x /usr/local/bin/codex /usr/local/bin/codex-code-mode-host; ln -sf /usr/local/bin/codex-resources/bwrap /usr/local/bin/bwrap' \
|
||||
--mkdir /app \
|
||||
--copy-in "$REPO_DIR/host.py:/app/" \
|
||||
--copy-in "$REPO_DIR/cloud:/app/" \
|
||||
--run-command 'printf "ssh_deletekeys: false\nssh_genkeytypes: []\n" > /etc/cloud/cloud.cfg.d/99-chv-hostkey.cfg; systemctl enable ssh || true' \
|
||||
--run-command 'printf "PasswordAuthentication no\nPermitRootLogin prohibit-password\nChallengeResponseAuthentication no\nKbdInteractiveAuthentication no\nPubkeyAuthentication yes\n" > /etc/ssh/sshd_config.d/99-chv.conf; sshd -t && grep -q "^PasswordAuthentication no" /etc/ssh/sshd_config.d/99-chv.conf && grep -q "^PermitRootLogin prohibit-password" /etc/ssh/sshd_config.d/99-chv.conf' \
|
||||
--run-command 'dpkg-query -W -f "${Package} ${Version}\n" > /app/guest-versions-dpkg.txt; pip3 freeze > /app/guest-versions-pip.txt || true'
|
||||
if [ "${PLAYWRIGHT:-0}" = "1" ]; then
|
||||
run virt-customize -a "$OUT/disk.raw" \
|
||||
--run-command 'pip3 install --no-cache-dir playwright; python3 -m playwright install --with-deps chromium'
|
||||
fi
|
||||
run virt-get-kernel -a "$OUT/disk.raw" -o "$OUT/vmlinux"
|
||||
# feat-chv-secfix(M4):构建时把客机内实际装好的版本
|
||||
# 读回 BUILDINFO.versions(dpkg + pip freeze),名字列表
|
||||
# 之外再有逐项版本——可审计、可复现比对。事前 lockfile
|
||||
# 钉死(apt/pip 精确版本预先锁定)仍待构建宿主生成,
|
||||
# 见 README 与就绪清单 §11;本步先保证“建成什么版本
|
||||
# 都有记录”,不再只有名字。
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo "DRY-RUN: virt-cat -a $OUT/disk.raw /app/guest-versions-dpkg.txt > $OUT/guest-versions-dpkg.txt"
|
||||
echo "DRY-RUN: virt-cat -a $OUT/disk.raw /app/guest-versions-pip.txt > $OUT/guest-versions-pip.txt"
|
||||
else
|
||||
virt-cat -a "$OUT/disk.raw" /app/guest-versions-dpkg.txt > "$OUT/guest-versions-dpkg.txt"
|
||||
virt-cat -a "$OUT/disk.raw" /app/guest-versions-pip.txt > "$OUT/guest-versions-pip.txt"
|
||||
fi
|
||||
# 产物校验(干跑只打印,不执行校验命令的副作用部分)。
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo "DRY-RUN: test -s $OUT/vmlinux && test -s $OUT/disk.raw"
|
||||
echo "DRY-RUN: python3 生成 $OUT/BUILDINFO.json 并校验 tag=$TAG"
|
||||
else
|
||||
test -s "$OUT/vmlinux" || { echo "内核产物为空:$OUT/vmlinux" >&2; exit 1; }
|
||||
test -s "$OUT/disk.raw" || { echo "磁盘产物为空:$OUT/disk.raw" >&2; exit 1; }
|
||||
python3 - "$OUT/BUILDINFO.json" "$TAG" "$UBUNTU_VERSION" "$CODEX_VERSION" "$REPO_COMMIT" <<'PYEOF'
|
||||
import json, sys, time
|
||||
path, tag, ubuntu, codex, commit = sys.argv[1:6]
|
||||
def _read_lines(p):
|
||||
try:
|
||||
with open(p) as fh:
|
||||
return [l.strip() for l in fh if l.strip()]
|
||||
except OSError:
|
||||
return []
|
||||
import os
|
||||
outdir = os.path.dirname(path)
|
||||
info = {"tag": tag, "ubuntu": ubuntu, "codex_version": codex,
|
||||
"repo_commit": commit, "built_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
|
||||
"contents": ["git", "pandoc", "poppler-utils", "latexmk", "texlive-latex-base",
|
||||
"openssh-server", "cloud-init", "qemu-guest-agent",
|
||||
"python-docx", "openpyxl", "python-pptx", "PyMuPDF", "Pillow",
|
||||
"codex", "codex-code-mode-host", "codex-resources/bwrap", "/app/host.py"],
|
||||
"versions": {
|
||||
"dpkg": _read_lines(os.path.join(outdir, "guest-versions-dpkg.txt")),
|
||||
"pip": _read_lines(os.path.join(outdir, "guest-versions-pip.txt")),
|
||||
},
|
||||
"ubuntu_image_sha256_set": True}
|
||||
with open(path, "w") as fh:
|
||||
json.dump(info, fh, indent=2)
|
||||
PYEOF
|
||||
python3 -c "import json; d=json.load(open('$OUT/BUILDINFO.json')); assert d['tag'] == '$TAG', d; assert d['versions']['dpkg'], 'BUILDINFO versions.dpkg 为空'; assert d['versions']['pip'], 'BUILDINFO versions.pip 为空'"
|
||||
fi
|
||||
echo "tag=$TAG"
|
||||
echo "out=$OUT"
|
||||
if [ "$DRY_RUN" = "1" ]; then
|
||||
echo "干跑结束:未执行任何构建步骤,镜像未建成"
|
||||
else
|
||||
echo "构建完成:$OUT/vmlinux $OUT/disk.raw $OUT/BUILDINFO.json"
|
||||
fi
|
||||
@@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env python3
|
||||
"""校验已构建镜像的 BUILDINFO 供应链清单(事后 lockfile 检查)。
|
||||
|
||||
事前 apt/pip 精确 lockfile 无法在本轨生成:apt 版本在构建
|
||||
宿主上解析、pip 版本在客机里解析,没有构建宿主就编不出
|
||||
版本号——本脚本绝不编造。构建时 build.sh 已从客机读回
|
||||
dpkg + pip freeze 写进 BUILDINFO.versions(事后 lockfile),
|
||||
本脚本校验它:versions 非空、tag 与预期一致、可选的冻结
|
||||
清单(--frozen JSON,形如 {"dpkg": ["pkg version", ...]})
|
||||
逐项比对,漂移即退出非 0。
|
||||
|
||||
用法:
|
||||
python3 check_lockfile.py out/BUILDINFO.json [--tag TAG]
|
||||
[--frozen frozen.json]
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 2:
|
||||
print("用法:check_lockfile.py BUILDINFO.json [--tag TAG] "
|
||||
"[--frozen frozen.json]", file=sys.stderr)
|
||||
return 2
|
||||
path = argv[1]
|
||||
tag = None
|
||||
frozen_path = None
|
||||
i = 2
|
||||
while i < len(argv):
|
||||
if argv[i] == "--tag" and i + 1 < len(argv):
|
||||
tag = argv[i + 1]
|
||||
i += 2
|
||||
elif argv[i] == "--frozen" and i + 1 < len(argv):
|
||||
frozen_path = argv[i + 1]
|
||||
i += 2
|
||||
else:
|
||||
print("未知参数:%s" % argv[i], file=sys.stderr)
|
||||
return 2
|
||||
with open(path) as fh:
|
||||
data = json.load(fh)
|
||||
if tag is not None and data.get("tag") != tag:
|
||||
print("tag 不一致:%s != %s" % (data.get("tag"), tag),
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
versions = data.get("versions") or {}
|
||||
if not versions.get("dpkg") or not versions.get("pip"):
|
||||
print("BUILDINFO.versions.dpkg/pip 为空,供应链清单缺失",
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
if frozen_path:
|
||||
with open(frozen_path) as fh:
|
||||
frozen = json.load(fh)
|
||||
for kind in ("dpkg", "pip"):
|
||||
want = set(frozen.get(kind) or [])
|
||||
got = set(versions.get(kind) or [])
|
||||
missing = want - got
|
||||
if missing:
|
||||
print("%s 漂移:冻结清单里有、构建产物里无:%s"
|
||||
% (kind, sorted(missing)), file=sys.stderr)
|
||||
return 1
|
||||
print("lockfile 校验通过:tag=%s dpkg=%d pip=%d"
|
||||
% (data.get("tag"), len(versions["dpkg"]),
|
||||
len(versions["pip"])))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
@@ -0,0 +1,160 @@
|
||||
#!/usr/bin/env python3
|
||||
"""在构建宿主上生成 CHV 客机镜像的供应链 lockfile(to10 轨)。
|
||||
|
||||
事后校验见 check_lockfile.py(校验 BUILDINFO.versions);
|
||||
本脚本是事前/事中生成端:在构建宿主上跑,读 BUILDINFO
|
||||
的 tag 与 contents(仅名字清单,不含版本号),逐项从本
|
||||
机 dpkg / pip 查真实版本,写出 lockfile.json:
|
||||
|
||||
{"tag": ..., "image_sha256": ... | null,
|
||||
"generated_at": UTC ISO 时间,
|
||||
"versions": {"dpkg": ["name version", ...],
|
||||
"pip": ["name version", ...]},
|
||||
"unresolved": ["查不到版本的名", ...]}
|
||||
|
||||
绝不编造版本号:查不到的进 unresolved,不猜、不填占位
|
||||
版本。--verify 模式校验一份 lockfile:tag 与 BUILDINFO
|
||||
一致、dpkg/pip 非空、每条都有非空版本、unresolved 为空;
|
||||
任一不满足即退出非 0(unresolved 非空 = 清单不完整,
|
||||
不是“生成成功”)。
|
||||
|
||||
用法(均在构建宿主上跑):
|
||||
python3 generate_lockfile.py BUILDINFO.json --output lockfile.json
|
||||
python3 generate_lockfile.py --verify lockfile.json \
|
||||
--buildinfo BUILDINFO.json
|
||||
本轨无构建宿主:脚本已落地且可跑(见回执里的 /tmp 合成
|
||||
BUILDINFO 实测),真实清单待构建宿主产出,绝不把合成
|
||||
数据写成生产清单。
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def _dpkg_version(name):
|
||||
"""dpkg 查版本,查不到回 None(不抛)。"""
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["dpkg-query", "-W", "-f=${Version}", name],
|
||||
capture_output=True, text=True, timeout=30)
|
||||
except Exception:
|
||||
return None
|
||||
if out.returncode != 0:
|
||||
return None
|
||||
version = out.stdout.strip()
|
||||
return version or None
|
||||
|
||||
|
||||
def _pip_version(name):
|
||||
"""pip 查版本,查不到回 None(不抛)。"""
|
||||
try:
|
||||
out = subprocess.run(
|
||||
[sys.executable, "-m", "pip", "show", name],
|
||||
capture_output=True, text=True, timeout=30)
|
||||
except Exception:
|
||||
return None
|
||||
if out.returncode != 0:
|
||||
return None
|
||||
for line in out.stdout.splitlines():
|
||||
if line.startswith("Version:"):
|
||||
version = line.split(":", 1)[1].strip()
|
||||
return version or None
|
||||
return None
|
||||
|
||||
|
||||
def generate(buildinfo_path, output_path):
|
||||
with open(buildinfo_path) as fh:
|
||||
buildinfo = json.load(fh)
|
||||
names = buildinfo.get("contents") or []
|
||||
if not isinstance(names, list):
|
||||
print("BUILDINFO.contents 不是列表", file=sys.stderr)
|
||||
return 2
|
||||
dpkg, pip, unresolved = [], [], []
|
||||
for name in names:
|
||||
version = _dpkg_version(name)
|
||||
if version is not None:
|
||||
dpkg.append("%s %s" % (name, version))
|
||||
continue
|
||||
version = _pip_version(name)
|
||||
if version is not None:
|
||||
pip.append("%s %s" % (name, version))
|
||||
continue
|
||||
unresolved.append(name)
|
||||
lockfile = {
|
||||
"tag": buildinfo.get("tag"),
|
||||
"image_sha256": buildinfo.get("image_sha256")
|
||||
or buildinfo.get("sha256"),
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"versions": {"dpkg": sorted(dpkg), "pip": sorted(pip)},
|
||||
"unresolved": sorted(unresolved),
|
||||
}
|
||||
with open(output_path, "w") as fh:
|
||||
json.dump(lockfile, fh, indent=2, sort_keys=True)
|
||||
fh.write("\n")
|
||||
if unresolved:
|
||||
print("警告:%d 项查不到版本,已列入 unresolved:%s"
|
||||
% (len(unresolved), ", ".join(sorted(unresolved))),
|
||||
file=sys.stderr)
|
||||
return 0
|
||||
|
||||
|
||||
def verify(lockfile_path, buildinfo_path):
|
||||
with open(lockfile_path) as fh:
|
||||
lockfile = json.load(fh)
|
||||
if buildinfo_path:
|
||||
with open(buildinfo_path) as fh:
|
||||
buildinfo = json.load(fh)
|
||||
if lockfile.get("tag") != buildinfo.get("tag"):
|
||||
print("tag 不一致:lockfile %s != BUILDINFO %s"
|
||||
% (lockfile.get("tag"), buildinfo.get("tag")),
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
versions = lockfile.get("versions") or {}
|
||||
for kind in ("dpkg", "pip"):
|
||||
entries = versions.get(kind)
|
||||
if not entries:
|
||||
print("versions.%s 为空,供应链清单缺失" % kind,
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
for entry in entries:
|
||||
parts = str(entry).split()
|
||||
if len(parts) < 2 or not parts[1]:
|
||||
print("版本为空的条目:%s %r" % (kind, entry),
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
unresolved = lockfile.get("unresolved") or []
|
||||
if unresolved:
|
||||
print("清单不完整,unresolved 非空:%s" % sorted(unresolved),
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
print("lockfile 校验通过:tag=%s dpkg=%d pip=%d"
|
||||
% (lockfile.get("tag"), len(versions["dpkg"]),
|
||||
len(versions["pip"])))
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv):
|
||||
if "--verify" in argv:
|
||||
lockfile_path = argv[argv.index("--verify") + 1]
|
||||
buildinfo_path = None
|
||||
if "--buildinfo" in argv:
|
||||
buildinfo_path = argv[argv.index("--buildinfo") + 1]
|
||||
return verify(lockfile_path, buildinfo_path)
|
||||
if len(argv) < 2:
|
||||
print("用法:generate_lockfile.py BUILDINFO.json "
|
||||
"--output lockfile.json | --verify lockfile.json "
|
||||
"[--buildinfo BUILDINFO.json]", file=sys.stderr)
|
||||
return 2
|
||||
output_path = None
|
||||
if "--output" in argv:
|
||||
output_path = argv[argv.index("--output") + 1]
|
||||
if not output_path:
|
||||
print("缺少 --output", file=sys.stderr)
|
||||
return 2
|
||||
return generate(argv[1], output_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv))
|
||||
@@ -1288,6 +1288,75 @@ interrupted 且终态任务 cancel 返回 400、心跳后 sweep 不扫刚心跳
|
||||
错误文本或返回结果。worker CLI 的 `--executor` 新增 cloud 选项,
|
||||
构造 CloudExecutor;worker 的 run_one 走 dispatch_task,
|
||||
CloudExecutor 的 dispatch_task/run_task 直接委托给 execute。
|
||||
- CloudHypervisorExecutor(生产级 v1,`cloud/chv/`,旧路径
|
||||
`cloud/sandbox.py` 只留兼容 re-export,执行器名
|
||||
`cloud-hypervisor`,绝不用 `cloud` 缩写):
|
||||
单任务单 VM:一个任务一个 VM、独立内核/文件系统/网络,
|
||||
任务结束销毁,租户之间无共享路径、无东西向互访;VM 内
|
||||
agent 可有 root,但只能影响本 VM。密钥按现有惯例走运行
|
||||
时环境变量名注入,不进镜像、不进代码、不进任务 prompt
|
||||
落库(SSH argv 里只有 key 文件路径,绝无 key 值)。
|
||||
唤醒是 API 模式:preflight(见下)→ 容量门 → 磁盘配额门
|
||||
→ 每任务磁盘供给(copy 模式复制基础镜像)→ 只以
|
||||
`--api-socket` 起 VMM → 等 `vmm.ping` 通 → `PUT vm.create`
|
||||
(VmConfig)→ `PUT vm.boot` → 等 guest SSH 就绪(默认
|
||||
120 秒)才跑 host.py;超时关 VM 并报分类失败。直接启动
|
||||
命令(`build_vm_command`)只作手动等价写法,执行器不走它。
|
||||
暂停/快照/休眠:`pause_vm`/`resume_vm` 走 ch-remote;
|
||||
`restore_vm` 先 restore 再显式 resume(恢复后默认 paused,
|
||||
不 resume 不算醒);CHV 无原生 hibernate 端点,`hibernate_vm`
|
||||
用 pause + snapshot + shutdown 等价实现,恢复走 restore。
|
||||
预热池 `WarmPool`:预热条目是一次性干净快照 VM,acquire
|
||||
消耗、release 一律销毁,绝不跨任务复用脏 VM;池空回落
|
||||
冷启动,绝不降级 Docker/Local;池状态另落
|
||||
`state_dir/warm-pool.json` registry,worker 重启后载入的
|
||||
条目一律当孤儿、只能 `reconcile_orphans()` 清理,绝不
|
||||
重新发给任务,registry 损坏时 quarantine 成 `.corrupt` 并
|
||||
从空池开始。SSH host key 按 VM 钉死(#216 的
|
||||
`StrictHostKeyChecking=no` + `/dev/null` 已废除):每 VM
|
||||
在 `state_dir/<vm>/known_hosts` 有独立文件,首次启动经
|
||||
ssh-keyscan 在本机回环端口捕获 key 写入该文件(首次信任
|
||||
TOFU,只防首次之后的替换,不验证首次本身),此后 SSH
|
||||
一律 `StrictHostKeyChecking=yes` + 该文件校验;客机用户
|
||||
(`CHV_SSH_USER`,默认 root,显式空串直接拒绝)与端口
|
||||
(`CHV_SSH_PORT`,默认 2222,1-65535)为显式配置。
|
||||
`execute()` 只读已运行 VM:未运行报 `vm_not_running`
|
||||
分类失败,不为一次 status 起 VM。长任务中途死亡
|
||||
(传输异常、或结果已 failed 且登记 VMM 进程已死)判
|
||||
`interrupted` 而非笼统 failed;中途心跳经评估未做
|
||||
(host.py 单次 SSH 长回合无客内进度信号可采,另起轮询
|
||||
只能证明 VM 活着)。关闭与清理:任务结束/
|
||||
失败/取消时 `shutdown_vm`(ch-remote shutdown + 杀进程 +
|
||||
删 socket),`cleanup_stale_vms` 处理已死/已关/socket 残留
|
||||
三种情况。审计:VM 的 create/boot/ready/pause/resume/
|
||||
snapshot/restore/shutdown/destroy/failed/recycled 均经
|
||||
audit_callback 发出,worker 接线后转 AuthStore 审计;失败
|
||||
带 kind 分类(kvm_missing/image_missing/guest_not_ready/
|
||||
api_timeout/host_capacity_exceeded/disk_quota_exceeded 等),
|
||||
不笼统 failed。资源与成本:每 VM 的 CPU/内存/磁盘配额、
|
||||
超时、最大轮次(goal 上限 100)复用 worker 现有 quota 语义,
|
||||
不另起一套计费;空闲休眠省成本;autoscaler 按 executor 过
|
||||
滤队列深度并透传 `--executor` 给拉起的 worker。网络与治理:
|
||||
VM 出口默认必须过 CHV_EGRESS_PROXY 白名单代理(未设且未
|
||||
显式允许直连时 preflight 直接失败);审批门仍走任务的
|
||||
approval 与 server 的 merge_gate,VM 内 agent 不能自批。
|
||||
配置(环境变量,见该类 docstring 全表):CHV_BINARY、
|
||||
CHV_KERNEL、CHV_ROOTFS(或 CHV_IMAGE 别名)、CHV_CPU、
|
||||
CHV_MEM、CHV_STATE_DIR、CHV_SSH_KEY/USER/PORT、CHV_CMDLINE、
|
||||
CHV_NET(直接命令用)/CHV_NET_JSON(API 用)、CHV_REMOTE、
|
||||
CHV_MAX_VMS、CHV_DISK_MODE(copy 默认/reuse 仅实验)、
|
||||
CHV_DISK_GB、CHV_EGRESS_PROXY、CHV_TASK_TIMEOUT、
|
||||
CHV_SNAPSHOT_URL、CHV_WARM_POOL_SIZE、CHV_API_TIMEOUT、
|
||||
CHV_READY_TIMEOUT/INTERVAL。默认关闭:server 默认执行器仍
|
||||
是 docker,worker 默认 local,CHV 只在任务创建时显式选
|
||||
`executor=cloud-hypervisor` 且有 CHV worker 认领时才跑;
|
||||
任务内 LLM 绝不能发现/调用它(调度层选择,只在创建层可见:
|
||||
POST /tasks 可选值、本文档、控制台选项,不新增任务内 MCP/
|
||||
工具)。真实验证边界:未做真实 VM 启动验证——生产主机
|
||||
本身是 CVM、无 /dev/kvm、无嵌套虚拟化,本 VM 容器内同样
|
||||
无 /dev/kvm;真 VM、快照恢复、压测、安全审计均在
|
||||
`docs/chv-production-readiness.md` 里标为待办,绝不声称
|
||||
已生产可用。客机镜像内容清单与构建骨架见 `chv-guest/`。
|
||||
- 自动 PR(默认关闭,显式触发,`POST /tasks/{id}/pr`,需项目
|
||||
maintainer 以上权限 approve_merge):仅当任务状态为 merged 时
|
||||
可调(其它状态 409;任务不存在 404;权限不足 403)。逻辑按顺序
|
||||
|
||||
@@ -304,6 +304,14 @@ class Autoscaler:
|
||||
if not any(a == "--host-name" or a.startswith("--host-name=")
|
||||
for a in self.worker_args):
|
||||
cmd.extend(["--host-name", socket.gethostname()])
|
||||
# 执行器透传:scaler 的 --executor 只算该类型队列深度,
|
||||
# 不透传时拉起的 worker 默认 local,永远认领不到该类型
|
||||
# 任务(cloud-hypervisor 池就是这样被饿死的)。worker_args
|
||||
# 里已显式给 --executor 时以显式为准,不覆盖。
|
||||
if self.executor and not any(
|
||||
a == "--executor" or a.startswith("--executor=")
|
||||
for a in self.worker_args):
|
||||
cmd.extend(["--executor", self.executor])
|
||||
return cmd
|
||||
|
||||
def _start_worker(self, worker_id):
|
||||
@@ -553,6 +561,8 @@ def main(argv=None):
|
||||
help="各 worker 子进程的日志目录")
|
||||
parser.add_argument("--executor", default=None,
|
||||
help="只按该执行器类型的就绪任务算深度;"
|
||||
"同时透传给拉起的 worker(worker_args 里"
|
||||
"已显式给 --executor 时以显式为准);"
|
||||
"不给时按全部任务算(可能轻微多扩,"
|
||||
"受 max-workers 封顶)")
|
||||
parser.add_argument("worker_args", nargs=argparse.REMAINDER,
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
"""Cloud Hypervisor 执行器包(从 cloud/sandbox.py 巨类拆出)。
|
||||
|
||||
公开名与 #216 一致,cloud/sandbox.py 会 re-export 生产
|
||||
接口,保证旧导入路径可用——但 WarmPool 除外(本轨破环
|
||||
性清理,见下):
|
||||
- CloudHypervisorExecutor:执行器本体(executor.py)
|
||||
- ChvFailure:分类失败(errors.py)
|
||||
- reconcile_orphans_for_executor:生产残留清理唯一入口
|
||||
(orphans.py,只读 registry 文件做清理,不构造池)
|
||||
|
||||
破环性清理(feat-chv-arch-maint-fe,未合并前即生效,
|
||||
合并时按 PR 描述的清单检查调用方):WarmPool 已从
|
||||
cloud/chv/pool.py 移到 cloud/chv/experimental_pool.py,
|
||||
本包与 cloud/sandbox.py 不再导出它。旧路径
|
||||
`from cloud.chv import WarmPool`、
|
||||
`from cloud.chv.pool import WarmPool`、
|
||||
`from cloud.sandbox import WarmPool` 一律 ImportError;
|
||||
实验用法只走
|
||||
`from cloud.chv.experimental_pool import WarmPool`。
|
||||
清理理由:CHV 尚未合并、尚无生产用户,兼容保留只是
|
||||
“怕断 import”,留着反而让生产代码有路可走进实验接口。
|
||||
|
||||
子模块分工见 executor.py 头注释与 docs/chv-architecture.md
|
||||
ADR-12。本包顶层 import 不触 cloud.sandbox(执行器方法
|
||||
内按需 import 共享 helper),所以任何导入顺序都不会循环。
|
||||
"""
|
||||
|
||||
from cloud.chv.errors import ChvFailure
|
||||
from cloud.chv.executor import CloudHypervisorExecutor
|
||||
from cloud.chv.orphans import reconcile_orphans_for_executor
|
||||
|
||||
__all__ = ["ChvFailure", "CloudHypervisorExecutor",
|
||||
"reconcile_orphans_for_executor"]
|
||||
@@ -0,0 +1,143 @@
|
||||
"""Cloud Hypervisor REST API 传输(叶子模块,不依赖
|
||||
cloud.sandbox,避免循环导入)。
|
||||
|
||||
只用标准库 raw HTTP over Unix socket,无第三方依赖。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
|
||||
|
||||
def _read_header(sock):
|
||||
"""读满响应头:返回 (header_text, 已收体部前缀)。
|
||||
|
||||
超时或提前关流即 RuntimeError,绝不把半截头当结束
|
||||
(feat-chv-fix P1-6 的语义,纯移动)。"""
|
||||
buf = b""
|
||||
header_end = None
|
||||
timed_out = False
|
||||
while header_end is None:
|
||||
try:
|
||||
data = sock.recv(65536)
|
||||
except socket.timeout:
|
||||
timed_out = True
|
||||
break
|
||||
if not data:
|
||||
break
|
||||
buf += data
|
||||
header_end = buf.find(b"\r\n\r\n")
|
||||
if timed_out or header_end is None:
|
||||
raise RuntimeError("CHV API 响应头不完整(超时或提前关流)")
|
||||
header = buf[:header_end].decode("utf-8", errors="replace")
|
||||
return header, buf[header_end + 4:]
|
||||
|
||||
|
||||
def _parse_content_length(header):
|
||||
"""从响应头取 Content-Length:chunked 或无长度即拒绝。
|
||||
|
||||
chunked 明确拒绝(CHV REST 不应发 chunked);无长度则
|
||||
无法证完整性,同样拒绝。"""
|
||||
content_length = None
|
||||
is_chunked = False
|
||||
for line in header.split("\r\n")[1:]:
|
||||
if ":" not in line:
|
||||
continue
|
||||
name, value = line.split(":", 1)
|
||||
name = name.strip().lower()
|
||||
value = value.strip()
|
||||
if name == "content-length":
|
||||
try:
|
||||
content_length = int(value)
|
||||
except ValueError:
|
||||
raise RuntimeError("CHV API Content-Length 非法")
|
||||
elif name == "transfer-encoding" \
|
||||
and "chunked" in value.lower():
|
||||
is_chunked = True
|
||||
if is_chunked:
|
||||
raise RuntimeError("CHV API 响应为 chunked,拒绝解析")
|
||||
if content_length is None:
|
||||
raise RuntimeError("CHV API 响应无 Content-Length,拒绝解析")
|
||||
return content_length
|
||||
|
||||
|
||||
def _read_body(sock, body, content_length):
|
||||
"""按 Content-Length 读满体部:超时或提前关流即截断错误。"""
|
||||
while len(body) < content_length:
|
||||
try:
|
||||
data = sock.recv(65536)
|
||||
except socket.timeout:
|
||||
raise RuntimeError(
|
||||
"CHV API 响应体截断:已收 %d / 需 %d 字节"
|
||||
% (len(body), content_length))
|
||||
if not data:
|
||||
raise RuntimeError(
|
||||
"CHV API 响应体截断:已收 %d / 需 %d 字节"
|
||||
% (len(body), content_length))
|
||||
body += data
|
||||
return body[:content_length]
|
||||
|
||||
|
||||
def _parse_response(header, body):
|
||||
"""从已读满的头与体部解析 (status, body):能解析 JSON
|
||||
时为 dict,否则为原文本;状态行解析失败时 status 为 0。"""
|
||||
text = header + "\r\n\r\n" + body.decode("utf-8", errors="replace")
|
||||
status = 0
|
||||
if text.startswith("HTTP/"):
|
||||
try:
|
||||
status = int(text.split()[1])
|
||||
except (IndexError, ValueError):
|
||||
status = 0
|
||||
body_text = body.decode("utf-8", errors="replace")
|
||||
try:
|
||||
parsed = json.loads(body_text) if body_text.strip() else {}
|
||||
except ValueError:
|
||||
parsed = body_text
|
||||
return status, parsed
|
||||
|
||||
|
||||
def _unix_socket_api(socket_path, method, path, payload=None):
|
||||
"""Cloud Hypervisor REST API 的默认传输:标准库 raw HTTP
|
||||
over Unix socket(无第三方依赖)。返回 (status, body),
|
||||
body 能解析 JSON 时为 dict,否则为原文本。socket 不存在
|
||||
或连接失败时 raise RuntimeError(调用方据此判 api 超时
|
||||
/失败,不编造 200)。读满语义见 _read_header /
|
||||
_parse_content_length / _read_body(P1-6,纯移动)。"""
|
||||
if not os.path.exists(socket_path):
|
||||
raise RuntimeError("api socket 不存在:%s" % socket_path)
|
||||
body = b""
|
||||
if payload is not None:
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
req_lines = ["%s /api/v1%s HTTP/1.1" % (method, path),
|
||||
"Host: localhost",
|
||||
"Accept: application/json",
|
||||
"Content-Type: application/json",
|
||||
"Content-Length: %d" % len(body),
|
||||
"Connection: close", "", ""]
|
||||
raw = "\r\n".join(req_lines).encode("utf-8") + body
|
||||
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
sock.settimeout(5)
|
||||
try:
|
||||
sock.connect(socket_path)
|
||||
sock.sendall(raw)
|
||||
header, body = _read_header(sock)
|
||||
content_length = _parse_content_length(header)
|
||||
body = _read_body(sock, body, content_length)
|
||||
finally:
|
||||
sock.close()
|
||||
return _parse_response(header, body)
|
||||
|
||||
|
||||
def normalize_api_result(result: dict) -> tuple:
|
||||
"""API 调用归一化:fake api_client 可返回 (status, body)
|
||||
或 {"status":..., "body":...} 或带 .status 的对象,本
|
||||
函数统一成 (status:int, body)。"""
|
||||
if isinstance(result, tuple) and len(result) == 2:
|
||||
return int(result[0]), result[1]
|
||||
if isinstance(result, dict):
|
||||
status = result.get("status", result.get("status_code"))
|
||||
body = result.get("body", result)
|
||||
return (int(status) if status is not None else 0), body
|
||||
status = getattr(result, "status", None) \
|
||||
or getattr(result, "status_code", None)
|
||||
return (int(status) if status is not None else 0), result
|
||||
@@ -0,0 +1,113 @@
|
||||
"""#215(feat-arch-converge)接口对齐层,CHV 本地实现。
|
||||
|
||||
背景:#215 未合并,本基线里没有 cloud/timeouts.py、
|
||||
cloud/gates.py、cloud/quota.py、cloud/sandbox_policy.py。
|
||||
本模块绝不复制 #215 的文件,只把 CHV 真正能映射的
|
||||
接口写成与 #215 相同的签名,#215 合并后替换为 import
|
||||
即是机械的(见 docs/chv-architecture.md ADR-8)。
|
||||
|
||||
映射表(逐个核对过 #215 远程分支的真实函数,不是凭空编的):
|
||||
- gates.resolve_inside(base, rel):目录门,CHV 快照/磁盘
|
||||
路径白名单正好需要它——本模块实现同语义并已在
|
||||
hardening.validate_snapshot_url 里使用,#215 合并后
|
||||
改为 from cloud.gates import resolve_inside 即可。
|
||||
- sandbox_policy.tmpfs_isolated(executor_kind, path):CHV
|
||||
客机 /tmp 落在客机磁盘上,不是宿主 tmpfs,故对
|
||||
cloud-hypervisor 一律返回 (False, ...)——fail-closed,
|
||||
绝不宣称隔离。未知执行器同样 fail-closed。
|
||||
- timeouts.resolve_module_timeout / quota.decide:无映射。
|
||||
前者是审批/队列超时(None=不过期、0=不过期),CHV
|
||||
的 api/ready 超时是秒数且 0 无效,语义不同,硬套会
|
||||
改行为;后者是按月美元配额,CHV 的配额是 VM 数/磁盘
|
||||
GB/快照数,单位不同。两者在本模块里 deliberately
|
||||
不提供,测试钉住“CHV 不 import 它们”。
|
||||
|
||||
本模块只依赖标准库,是叶子模块,不 import cloud.sandbox。
|
||||
|
||||
合并后替换清单(REPLACEMENT_POINTS,本轨从“接口形状”
|
||||
推进到可机械执行的清单;#215 未合并前一律不 import
|
||||
cloud.gates / cloud.sandbox_policy / cloud.timeouts /
|
||||
cloud.quota,测试钉住这些模块在本基线里不存在):
|
||||
- R1 gates.resolve_inside:唯一生产调用方是
|
||||
hardening._is_within(快照 URL 白名单)。合并后两步:
|
||||
① hardening 改为 `from cloud.gates import resolve_inside`;
|
||||
② 删本模块 resolve_inside。语义必须逐字一致,改完
|
||||
重跑快照白名单测试(逃逸/符号链接/绝对路径三例)。
|
||||
- R2 sandbox_policy.tmpfs_isolated:形状占位,当前无
|
||||
生产调用方(只有测试)。合并后一步:删本模块
|
||||
tmpfs_isolated;将来有调用方时直接 import #215 版,
|
||||
不得再本地复刻。CHV 语义是“一律不隔离(fail-closed)”,
|
||||
若 #215 版对 cloud-hypervisor 返回 True,替换即改行为,
|
||||
必须先停下来核对,不得机械替换。
|
||||
- R3 timeouts / quota:永不替换。#215 的 timeouts 是
|
||||
审批/队列超时(None/0 = 不过期),quota 是按月美元
|
||||
配额;CHV 的 api/ready 超时是秒数(0 无效)、配额是
|
||||
VM 数/磁盘 GB/快照数,单位与语义都不同。清单里明写
|
||||
“不替换”,防止合并时有人按名字硬套。
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
# 合并后替换清单:每项的 local/replacement/test 字段由测试
|
||||
# 逐项核对(local 函数真实存在、#215 模块当前不存在、
|
||||
# test 命名的测试类真实存在),清单本身不得漂移。
|
||||
REPLACEMENT_POINTS = (
|
||||
{
|
||||
"id": "R1",
|
||||
"upstream": "cloud.gates.resolve_inside",
|
||||
"local": "cloud.chv.compat215.resolve_inside",
|
||||
"caller": "cloud.chv.hardening._is_within",
|
||||
"replacement": "hardening 改为 from cloud.gates import "
|
||||
"resolve_inside,并删除本地 resolve_inside",
|
||||
"test": "Compat215Test + 快照白名单测试",
|
||||
},
|
||||
{
|
||||
"id": "R2",
|
||||
"upstream": "cloud.sandbox_policy.tmpfs_isolated",
|
||||
"local": "cloud.chv.compat215.tmpfs_isolated",
|
||||
"caller": None,
|
||||
"replacement": "删除本地 tmpfs_isolated;有调用方时"
|
||||
"直接 import #215 版(先核对 CHV 一律 False 语义)",
|
||||
"test": "Compat215Test.test_tmpfs_isolated_chv_and_unknown_"
|
||||
"fail_closed",
|
||||
},
|
||||
{
|
||||
"id": "R3",
|
||||
"upstream": "cloud.timeouts / cloud.quota",
|
||||
"local": None,
|
||||
"caller": None,
|
||||
"replacement": None,
|
||||
"test": "Compat215Test.test_no_timeout_or_quota_map",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def resolve_inside(base: str, rel: str) -> str | None:
|
||||
"""目录门:base 内的相对路径安全解析,越界回 None。
|
||||
|
||||
空串回 base 本身;绝对路径、.. 逃逸、符号链接逃逸、
|
||||
commonpath 跨盘符 ValueError 一律 None。签名与 #215
|
||||
的 gates.resolve_inside 一致,合并后可直接替换为 import。"""
|
||||
if not rel:
|
||||
return base
|
||||
if os.path.isabs(rel):
|
||||
return None
|
||||
real_base = os.path.realpath(base)
|
||||
target = os.path.realpath(os.path.join(real_base, rel))
|
||||
try:
|
||||
if os.path.commonpath([real_base, target]) != real_base:
|
||||
return None
|
||||
except ValueError:
|
||||
return None
|
||||
return target
|
||||
|
||||
|
||||
def tmpfs_isolated(executor_kind: str, path: str) -> tuple[bool, str]:
|
||||
"""某执行器下某路径是否走 tmpfs 隔离:(isolated, reason)。
|
||||
|
||||
cloud-hypervisor:客机 /tmp 落在客机磁盘上,不走宿主
|
||||
tmpfs,一律 (False, ...);未知执行器 fail-closed 同样
|
||||
False。签名与 #215 的 sandbox_policy.tmpfs_isolated 一致。"""
|
||||
if executor_kind == "cloud-hypervisor":
|
||||
return False, "CHV 客机 /tmp 落在客机磁盘上,不走宿主 tmpfs"
|
||||
return False, "未知执行器,按不隔离处理(fail-closed 口径)"
|
||||
@@ -0,0 +1,309 @@
|
||||
"""Cloud Hypervisor 执行器配置解析(叶子模块,不依赖
|
||||
cloud.sandbox,避免循环导入)。
|
||||
|
||||
职责只有两件:内存字符串解析,以及把构造参数 + 环境变量
|
||||
解析成一份显式配置。参数优先于环境变量、环境变量优先于
|
||||
默认值——与 #216 原 __init__ 的 `or` 链完全一致,只收紧了
|
||||
两处原来会静默 fallback 的 nonsense 输入(见下文),收紧点
|
||||
在 PR 描述里明写、单测钉住:
|
||||
|
||||
- CHV_SSH_USER 显式传空字符串:原来 `or` 链会静默 fallback
|
||||
成 root(调用方以为没给用户、实际跑了 root),现在直接
|
||||
ValueError。
|
||||
- CHV_SSH_PORT 显式传 0、或解析后超出 1-65535:原来 0 会
|
||||
静默 fallback 成 2222、其它越界值会原样进 ssh 命令,现在
|
||||
直接 ValueError。
|
||||
|
||||
与 #215 的关系(只列清单,不复制模块):#215 未合并,
|
||||
本轨不假装复用它。上一轨 ADR-7 的泛化映射已被证伪
|
||||
(#215 的 timeouts 是审批/队列超时、quota 是按月美元
|
||||
配额,与 CHV 的秒数超时、VM 数/磁盘 GB 单位语义不同,
|
||||
硬套会改行为),真实映射只有两处,见
|
||||
cloud/chv/compat215.py 的 REPLACEMENT_POINTS 清单
|
||||
(合并时按清单机械替换,不在本轨硬 import 未合并模块)。
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
|
||||
|
||||
def _to_int(value: object) -> int:
|
||||
"""动态配置值转 int(env/参数均未定型):bool 拒绝,
|
||||
int/float 按 int(),str 按 strip 后 int(),其它 ValueError。
|
||||
旧行为是 int(value) 抛 TypeError/ValueError 后由调用方
|
||||
转 ValueError;这里直接统一成 ValueError,文案由调用方定。"""
|
||||
if isinstance(value, bool):
|
||||
raise ValueError("布尔值不是整数")
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
if isinstance(value, float):
|
||||
return int(value)
|
||||
if isinstance(value, str):
|
||||
return int(value.strip())
|
||||
raise ValueError("不是整数:%r" % (value,))
|
||||
|
||||
|
||||
def _to_float(value: object) -> float:
|
||||
"""动态配置值转 float,与 _to_int 同款口径。"""
|
||||
if isinstance(value, bool):
|
||||
raise ValueError("布尔值不是数字")
|
||||
if isinstance(value, (int, float)):
|
||||
return float(value)
|
||||
if isinstance(value, str):
|
||||
return float(value.strip())
|
||||
raise ValueError("不是数字:%r" % (value,))
|
||||
|
||||
|
||||
def _parse_memory_bytes(value: str) -> int:
|
||||
"""内存字符串转字节:2G/512M/1024K/2048(无单位=字节)。
|
||||
非法值 ValueError,绝不静默按默认算。"""
|
||||
m = re.match(r"^\s*(\d+)\s*([KMGkmg]?)\s*[Bb]?\s*$",
|
||||
str(value))
|
||||
if not m:
|
||||
raise ValueError("CHV_MEM 格式非法:%r" % (value,))
|
||||
mult = {"": 1, "K": 1024, "M": 1024 ** 2, "G": 1024 ** 3}
|
||||
return int(m.group(1)) * mult[m.group(2).upper()]
|
||||
|
||||
|
||||
class ChvConfig:
|
||||
"""解析后的 CHV 配置:纯数据对象,字段名与
|
||||
CloudHypervisorExecutor 原 __init__ 的属性名一一对应,
|
||||
执行器 __init__ 直接把字段抄到 self 上,旧测试读属性
|
||||
的方式全部不变。"""
|
||||
|
||||
def __init__(self, **fields: object) -> None:
|
||||
for key, value in fields.items():
|
||||
setattr(self, key, value)
|
||||
|
||||
|
||||
|
||||
def _base_fragment(params: dict, env: dict) -> dict:
|
||||
"""身份/资源基底片段:二进制、内核、根盘、CPU/内存、
|
||||
state_dir、KVM/cpuinfo 路径、ch-remote、模型。无校验,
|
||||
取值口径与旧 build_chv_config 内联 base 逐字一致,
|
||||
只有一处新增(feat-chv-p1fix,P1-07):能解析出的
|
||||
CPU/内存非正值直接 ValueError,不再一路进 VmConfig
|
||||
到 VMM 侧才炸;解析不出的格式仍留给 preflight 报
|
||||
config_invalid(构造时不改旧报错时机分工)。"""
|
||||
# 显式 cpus=0 不再经 or 链静默回落成 2:0 是非法值,
|
||||
# 应报出来(P1-07);None 仍走 env/默认值。
|
||||
_cpus_raw = params.get("cpus") if params.get("cpus") is not None \
|
||||
else (env.get("CHV_CPU") or "2")
|
||||
cpus = str(_cpus_raw)
|
||||
# 解析不出的格式留给 preflight(旧时机分工不变):
|
||||
# 只对“解析成功且 <=0”的值在这里报,格式错误不报。
|
||||
try:
|
||||
_cpus_parsed = int(cpus)
|
||||
except (TypeError, ValueError):
|
||||
_cpus_parsed = None
|
||||
if _cpus_parsed is not None and _cpus_parsed <= 0:
|
||||
raise ValueError("CHV_CPU 必须是大于 0 的整数")
|
||||
memory = params.get("memory") or env.get("CHV_MEM") or "2G"
|
||||
try:
|
||||
_mem_parsed = _parse_memory_bytes(memory)
|
||||
except ValueError:
|
||||
_mem_parsed = None
|
||||
if _mem_parsed is not None and _mem_parsed <= 0:
|
||||
raise ValueError("CHV_MEM 必须是大于 0")
|
||||
return {
|
||||
"binary": params.get("binary") or env.get("CHV_BINARY")
|
||||
or "cloud-hypervisor",
|
||||
"kernel": params.get("kernel") or env.get("CHV_KERNEL"),
|
||||
"rootfs": params.get("rootfs") or params.get("image")
|
||||
or env.get("CHV_ROOTFS") or env.get("CHV_IMAGE"),
|
||||
"cpus": cpus,
|
||||
"memory": memory,
|
||||
"state_dir": params.get("state_dir")
|
||||
or env.get("CHV_STATE_DIR")
|
||||
or os.path.join(os.path.expanduser("~"),
|
||||
".local", "share", "codex-app-server", "chv-vms"),
|
||||
"kvm_path": params.get("kvm_path")
|
||||
if params.get("kvm_path") is not None else "/dev/kvm",
|
||||
"cpuinfo_path": params.get("cpuinfo_path")
|
||||
if params.get("cpuinfo_path") is not None else "/proc/cpuinfo",
|
||||
"ch_remote": params.get("ch_remote") or env.get("CHV_REMOTE")
|
||||
or "ch-remote",
|
||||
"model": params.get("model"),
|
||||
}
|
||||
|
||||
|
||||
def _ssh_fragment(params: dict, env: dict) -> dict:
|
||||
"""SSH/命令面片段:密钥、用户、端口、host key 模式、
|
||||
客内 host.py 路径、内核命令行。错误文案与原内联代码
|
||||
逐字一致,校验时机不变(构造时)。网络字段(net、
|
||||
ssh_host、egress)已移到 _network_fragment,不在本
|
||||
片段,避免“SSH”片段名不副实。"""
|
||||
ssh_key = params.get("ssh_key") or env.get("CHV_SSH_KEY")
|
||||
if params.get("ssh_user") is not None \
|
||||
and params.get("ssh_user") == "":
|
||||
raise ValueError("CHV_SSH_USER 不能为空字符串")
|
||||
ssh_user = params.get("ssh_user") or env.get("CHV_SSH_USER") \
|
||||
or "root"
|
||||
if params.get("ssh_port") is not None:
|
||||
raw_port = params.get("ssh_port")
|
||||
else:
|
||||
raw_port = env.get("CHV_SSH_PORT") or 2222
|
||||
try:
|
||||
ssh_port = _to_int(raw_port)
|
||||
except (TypeError, ValueError):
|
||||
raise ValueError("CHV_SSH_PORT 必须是整数")
|
||||
if not 1 <= ssh_port <= 65535:
|
||||
raise ValueError("CHV_SSH_PORT 必须在 1-65535 之间")
|
||||
host_key_mode = params.get("host_key_mode") \
|
||||
or env.get("CHV_HOST_KEY_MODE") or "provisioned"
|
||||
if host_key_mode not in ("provisioned", "keyscan"):
|
||||
raise ValueError(
|
||||
"CHV_HOST_KEY_MODE 只能是 provisioned 或 keyscan:%s"
|
||||
% host_key_mode)
|
||||
return {
|
||||
"ssh_key": ssh_key, "ssh_user": ssh_user,
|
||||
"ssh_port": ssh_port, "host_key_mode": host_key_mode,
|
||||
"guest_host_py": params.get("guest_host_py") or "/app/host.py",
|
||||
"cmdline": params.get("cmdline") or env.get("CHV_CMDLINE")
|
||||
or "console=hvc0 root=/dev/vda1 rw",
|
||||
}
|
||||
|
||||
|
||||
def _network_fragment(params: dict, env: dict) -> dict:
|
||||
"""网络片段:tap/桥接描述、SSH 目标、端口转发开关、
|
||||
egress 代理与直连开关。无数值校验(校验在 preflight
|
||||
的 net/egress 门里,不在配置解析层)。egress 从旧
|
||||
quota 片段移到这里:它是网络出口策略,不是 VM 配额。"""
|
||||
return {
|
||||
"net": params.get("net") or env.get("CHV_NET"),
|
||||
"net_json": params.get("net_json") or env.get("CHV_NET_JSON"),
|
||||
"ssh_host": params.get("ssh_host") or env.get("CHV_SSH_HOST"),
|
||||
"ssh_forward": env.get("CHV_SSH_FORWARD") == "1",
|
||||
"egress_proxy": env.get("CHV_EGRESS_PROXY"),
|
||||
"allow_direct_egress": env.get("CHV_ALLOW_DIRECT_EGRESS")
|
||||
== "1",
|
||||
}
|
||||
|
||||
|
||||
def _quota_fragment(env: dict) -> dict:
|
||||
"""配额片段:仅容量数——最大并发 VM 数与预热池大小。
|
||||
磁盘/快照数上限不在这里,它们是磁盘行为,见
|
||||
_disk_fragment。校验文案与旧代码逐字一致。"""
|
||||
try:
|
||||
max_vms = _to_int(env.get("CHV_MAX_VMS") or 4)
|
||||
except ValueError:
|
||||
raise ValueError("CHV_MAX_VMS 必须是整数")
|
||||
try:
|
||||
warm_pool_size = _to_int(env.get("CHV_WARM_POOL_SIZE") or 0)
|
||||
except ValueError:
|
||||
raise ValueError("CHV_WARM_POOL_SIZE 必须是整数")
|
||||
return {"max_vms": max_vms, "warm_pool_size": warm_pool_size}
|
||||
|
||||
|
||||
def _disk_fragment(env: dict) -> dict:
|
||||
"""磁盘片段:磁盘模式/大小、单租户 reuse 实验开关、
|
||||
快照数/总量/保留期/目录/远端开关与快照 URL。校验
|
||||
文案与旧 quota 片段逐字一致(disk_mode 白名单、
|
||||
disk_gb > 0、快照数非负)。"""
|
||||
disk_mode = env.get("CHV_DISK_MODE") or "copy"
|
||||
if disk_mode not in ("copy", "reuse"):
|
||||
raise ValueError(
|
||||
"CHV_DISK_MODE 只能是 copy 或 reuse:%s" % disk_mode)
|
||||
try:
|
||||
disk_gb = _to_float(env.get("CHV_DISK_GB")) \
|
||||
if env.get("CHV_DISK_GB") else 50.0
|
||||
except ValueError:
|
||||
raise ValueError("CHV_DISK_GB 必须是数字")
|
||||
if disk_gb <= 0:
|
||||
raise ValueError("CHV_DISK_GB 必须大于 0")
|
||||
try:
|
||||
snapshot_max_count = _to_int(env.get("CHV_SNAPSHOT_MAX_COUNT")
|
||||
or 10)
|
||||
snapshot_max_total_gb = _to_float(
|
||||
env.get("CHV_SNAPSHOT_MAX_TOTAL_GB") or 20.0)
|
||||
snapshot_retention_days = _to_float(
|
||||
env.get("CHV_SNAPSHOT_RETENTION_DAYS") or 30.0)
|
||||
except ValueError:
|
||||
raise ValueError("CHV_SNAPSHOT_* 必须是数字")
|
||||
if snapshot_max_count < 0 or snapshot_max_total_gb < 0 \
|
||||
or snapshot_retention_days < 0:
|
||||
raise ValueError("CHV_SNAPSHOT_* 不能为负数")
|
||||
return {
|
||||
"disk_mode": disk_mode, "disk_gb": disk_gb,
|
||||
"reuse_experiment": env.get(
|
||||
"CHV_REUSE_SINGLE_TENANT_EXPERIMENT") == "1",
|
||||
"single_tenant_project": env.get("CHV_SINGLE_TENANT_PROJECT"),
|
||||
"snapshot_max_count": snapshot_max_count,
|
||||
"snapshot_max_total_gb": snapshot_max_total_gb,
|
||||
"snapshot_retention_days": snapshot_retention_days,
|
||||
"snapshot_dir": env.get("CHV_SNAPSHOT_DIR"),
|
||||
"allow_remote_snapshot": env.get("CHV_ALLOW_REMOTE_SNAPSHOT")
|
||||
== "1",
|
||||
"snapshot_url": env.get("CHV_SNAPSHOT_URL"),
|
||||
}
|
||||
|
||||
|
||||
def _timeout_fragment(params: dict, env: dict) -> dict:
|
||||
"""超时片段:任务超时(None = 不限,旧语义)、VMM API
|
||||
超时、客机就绪超时/间隔。ready_interval <= 0 时回落
|
||||
5 秒(旧语义,非校验失败)。校验文案逐字一致。"""
|
||||
task_timeout_raw = env.get("CHV_TASK_TIMEOUT")
|
||||
task_timeout = _to_float(task_timeout_raw) \
|
||||
if task_timeout_raw else None
|
||||
# feat-chv-p1fix(P1-07):显式设的任务超时非正即非法。
|
||||
# None = 不限(旧语义,保留);0/负数过去一路进 VmConfig
|
||||
# 相关路径(runner timeout=0/负),现在构造时即 ValueError。
|
||||
if task_timeout is not None and task_timeout <= 0:
|
||||
raise ValueError("CHV_TASK_TIMEOUT 必须是大于 0 的数字")
|
||||
try:
|
||||
api_timeout = _to_float(
|
||||
params.get("api_timeout")
|
||||
if params.get("api_timeout") is not None
|
||||
else env.get("CHV_API_TIMEOUT") or 30)
|
||||
ready_timeout = _to_float(
|
||||
params.get("ready_timeout")
|
||||
if params.get("ready_timeout") is not None
|
||||
else env.get("CHV_READY_TIMEOUT") or 120)
|
||||
ready_interval = _to_float(
|
||||
params.get("ready_interval")
|
||||
if params.get("ready_interval") is not None
|
||||
else env.get("CHV_READY_INTERVAL") or 5)
|
||||
except ValueError:
|
||||
raise ValueError("CHV_*_TIMEOUT/INTERVAL 必须是数字")
|
||||
if ready_interval <= 0:
|
||||
ready_interval = 5
|
||||
return {
|
||||
"task_timeout": task_timeout, "api_timeout": api_timeout,
|
||||
"ready_timeout": ready_timeout,
|
||||
"ready_interval": ready_interval,
|
||||
}
|
||||
|
||||
|
||||
def _assemble_chv_config(fragments: list) -> "ChvConfig":
|
||||
"""把各片段 dict 合并成 ChvConfig(纯合并,不改值)。
|
||||
|
||||
片段键集必须互不相交——键冲突是编程错误,直接
|
||||
ValueError,绝不让后一片段静默覆盖前一片段的值。"""
|
||||
fields: dict = {}
|
||||
for fragment in fragments:
|
||||
for key, value in fragment.items():
|
||||
if key in fields:
|
||||
raise ValueError(
|
||||
"CHV 配置片段键冲突:%s" % key)
|
||||
fields[key] = value
|
||||
return ChvConfig(**fields)
|
||||
|
||||
|
||||
def build_chv_config(params: dict, env: dict) -> "ChvConfig":
|
||||
"""把显式构造参数与环境变量解析成 ChvConfig。
|
||||
|
||||
片段调用顺序即旧校验顺序(SSH → 配额 → 磁盘 →
|
||||
超时),只有一处可观测的顺序变化:warm_pool_size
|
||||
从旧超时片段移到配额片段,因此“CHV_WARM_POOL_SIZE
|
||||
非法 + 磁盘非法同时出现”时先报前者;单项非法时
|
||||
报错与旧代码逐字一致。字段集合与取值与旧代码一致,
|
||||
由测试的字段集断言钉住。"""
|
||||
env = env if env is not None else {}
|
||||
return _assemble_chv_config([
|
||||
_base_fragment(params, env),
|
||||
_ssh_fragment(params, env),
|
||||
_quota_fragment(env),
|
||||
_disk_fragment(env),
|
||||
_network_fragment(params, env),
|
||||
_timeout_fragment(params, env),
|
||||
])
|
||||
@@ -0,0 +1,311 @@
|
||||
"""Cloud Hypervisor 每任务磁盘供给(叶子模块)。
|
||||
|
||||
单任务单 VM 的文件系统独立性就靠这里:copy 模式每任务
|
||||
复制一份基础镜像,两个任务绝不写同一文件;reuse 模式
|
||||
直接返回基础镜像路径(仅单租户实验,生产拦截见执行器
|
||||
preflight,须同时设 CHV_REUSE_SINGLE_TENANT_EXPERIMENT=1
|
||||
并审计放行)。
|
||||
|
||||
feat-chv-fix(P1-5)配额口径:不再只比基础镜像大小。
|
||||
check_disk_quota 改为三项:state_dir 已用总量 + 本次
|
||||
拷贝(基础镜像大小)+ 宿主剩余空间必须容纳本次拷贝;
|
||||
镜像不可读时不再静默跳过,报 disk_quota_unverified
|
||||
(preflight 在 wake 路径上会先报 image_missing,这里
|
||||
是直接调用者的第二道门)。provision_task_disk 对已存在
|
||||
磁盘校验大小:与源不一致即重拷;源不可读即报失败,
|
||||
绝不复用脏磁盘。
|
||||
"""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
|
||||
from cloud.chv.errors import ChvFailure
|
||||
|
||||
|
||||
def _dir_total_bytes(path):
|
||||
total = 0
|
||||
for root, _dirs, files in os.walk(path):
|
||||
for name in files:
|
||||
try:
|
||||
total += os.path.getsize(os.path.join(root, name))
|
||||
except OSError:
|
||||
pass
|
||||
return total
|
||||
|
||||
|
||||
def check_disk_quota(rootfs: str, disk_gb: float, state_dir: str | None = None) -> None:
|
||||
"""磁盘配额门(feat-chv-fix 口径):
|
||||
|
||||
- disk_gb 未设时不检查;
|
||||
- 镜像不可读时报 disk_quota_unverified,不静默跳过;
|
||||
- 镜像大小超过 disk_gb 即 disk_quota_exceeded;
|
||||
- state_dir 已给时:已用总量 + 本次拷贝超过 disk_gb
|
||||
即 disk_quota_exceeded;宿主剩余空间不足以容纳本次
|
||||
拷贝即 disk_quota_exceeded(宿主余量校验)。
|
||||
"""
|
||||
if disk_gb is None or not rootfs:
|
||||
return
|
||||
try:
|
||||
size = os.path.getsize(rootfs)
|
||||
except OSError:
|
||||
raise ChvFailure(
|
||||
"disk_quota_unverified",
|
||||
"磁盘镜像不可读,配额无法校验:%s" % rootfs)
|
||||
if size > disk_gb * 1024 ** 3:
|
||||
raise ChvFailure(
|
||||
"disk_quota_exceeded",
|
||||
"磁盘镜像 %d 字节超过配额 %.1f GB" % (size, disk_gb))
|
||||
if state_dir and os.path.isdir(state_dir):
|
||||
used = _dir_total_bytes(state_dir)
|
||||
if used + size > disk_gb * 1024 ** 3:
|
||||
raise ChvFailure(
|
||||
"disk_quota_exceeded",
|
||||
"磁盘总量 %d 字节 + 本次 %d 字节超过配额 %.1f GB"
|
||||
% (used, size, disk_gb))
|
||||
try:
|
||||
free = shutil.disk_usage(state_dir).free
|
||||
except OSError:
|
||||
free = None
|
||||
if free is not None and free < size:
|
||||
raise ChvFailure(
|
||||
"disk_quota_exceeded",
|
||||
"宿主剩余空间 %d 字节不足以容纳本次磁盘拷贝 %d 字节"
|
||||
% (free, size))
|
||||
|
||||
|
||||
def provision_task_disk(state_dir: str, vm_name: str, rootfs: str, disk_mode: str) -> str:
|
||||
"""每任务磁盘路径:copy 模式复制到
|
||||
state_dir/<vm>/disk.raw;reuse 模式直接返回基础镜像路径。
|
||||
|
||||
feat-chv-fix:已存在磁盘须校验——大小与源不一致即
|
||||
重拷(脏磁盘不复用);源不可读即 ChvFailure,绝不
|
||||
静默返回旧磁盘。"""
|
||||
if disk_mode == "reuse":
|
||||
return rootfs
|
||||
task_dir = os.path.join(state_dir, vm_name)
|
||||
os.makedirs(task_dir, exist_ok=True)
|
||||
task_disk = os.path.join(task_dir, "disk.raw")
|
||||
# feat-chv-secfix(L1):目标是符号链接时先删链接本身,
|
||||
# 绝不跟随链接写入目标文件(copyfile 跟随链接会截断并
|
||||
# 写进链接目标)。删链接与重建之间的 TOCTOU 窗口已在
|
||||
# 文档里明写:state_dir 须 0700 且仅 worker 用户可写,
|
||||
# 宿主本地攻击者本就写不进该目录,L1 修的是“链接已
|
||||
# 预置”情形下的最后一道门。
|
||||
if os.path.islink(task_disk):
|
||||
os.remove(task_disk)
|
||||
if os.path.exists(task_disk):
|
||||
try:
|
||||
same = os.path.getsize(task_disk) == os.path.getsize(rootfs)
|
||||
except OSError:
|
||||
raise ChvFailure(
|
||||
"disk_quota_unverified",
|
||||
"任务磁盘或源镜像不可读,无法校验复用:%s" % task_disk)
|
||||
if same:
|
||||
return task_disk
|
||||
os.remove(task_disk)
|
||||
try:
|
||||
shutil.copyfile(rootfs, task_disk)
|
||||
except OSError as exc:
|
||||
raise ChvFailure(
|
||||
"disk_quota_unverified",
|
||||
"任务磁盘拷贝失败:%s" % exc)
|
||||
return task_disk
|
||||
|
||||
|
||||
def task_disk_path(state_dir: str, vm_name: str) -> str:
|
||||
"""该 VM 的任务磁盘路径:state_dir/<vm>/disk.raw。"""
|
||||
return os.path.join(state_dir, vm_name, "disk.raw")
|
||||
|
||||
|
||||
def delete_task_disk(state_dir: str, vm_name: str) -> bool:
|
||||
"""删任务磁盘(feat-chv-secfix,H3):stop/recycle/
|
||||
清理路径都调它。返回 True 表示确实删掉了一个文件,
|
||||
False 表示本来就没有;删失败抛 OSError(调用方记审计,
|
||||
绝不静默——残留盘由 gc_stale_disks 下次再清)。
|
||||
符号链接只删链接本身,不跟随。"""
|
||||
path = task_disk_path(state_dir, vm_name)
|
||||
if os.path.islink(path) or os.path.exists(path):
|
||||
os.remove(path)
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
# -- 快照登记与上限(feat-chv-secfix,M3)-------------------------
|
||||
# 登记文件:state_dir/<vm>/snapshots.json,内容为
|
||||
# [{url, path(file:// 才有), size, created_at}]。登记是
|
||||
# 执行器在宿主侧的账本,不是 VMM 的快照元数据;VMM 不认
|
||||
# 它,删登记不等于删快照文件(反之亦然),两者在
|
||||
# prune/record 里成对处理并审计。
|
||||
|
||||
def _snapshots_path(state_dir, vm_name):
|
||||
return os.path.join(state_dir, vm_name, "snapshots.json")
|
||||
|
||||
|
||||
def _load_snapshots(state_dir, vm_name):
|
||||
import json
|
||||
path = _snapshots_path(state_dir, vm_name)
|
||||
try:
|
||||
with open(path, "r") as fh:
|
||||
data = json.load(fh)
|
||||
except (OSError, ValueError):
|
||||
return []
|
||||
return data if isinstance(data, list) else []
|
||||
|
||||
|
||||
def _save_snapshots(state_dir, vm_name, entries):
|
||||
import json
|
||||
path = _snapshots_path(state_dir, vm_name)
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
tmp_path = path + ".tmp"
|
||||
with open(tmp_path, "w") as fh:
|
||||
json.dump(entries, fh)
|
||||
os.replace(tmp_path, path)
|
||||
|
||||
|
||||
def prune_snapshots(state_dir: str, vm_name: str,
|
||||
retention_days: float | None,
|
||||
now: float | None = None,
|
||||
delete_files: bool = True,
|
||||
events: list | None = None) -> tuple[list, list]:
|
||||
"""按保留期剪枝:删过期条目;delete_files 时同时删
|
||||
对应的 file:// 快照文件(只删登记里记的 path,不猜
|
||||
文件名)。返回(保留条目,剪掉条目)。
|
||||
|
||||
feat-chv-p1fix(P1-08):删文件动作不再静默——
|
||||
events 非空列表时,每条剪枝追加一条事件:
|
||||
("deleted", entry, None) / ("delete_failed", entry,
|
||||
error) / ("pruned_no_file", entry, None)(登记条目
|
||||
被剪、但无对应文件可删)。调用方(executor.snapshot_vm)
|
||||
据此发 vm_snapshot_pruned / vm_snapshot_prune_failed
|
||||
审计;events 为 None 时行为与旧版一致(仍删、仍
|
||||
best-effort),不改旧调用点返回值。
|
||||
"""
|
||||
import time
|
||||
now = time.time() if now is None else now
|
||||
kept, pruned = [], []
|
||||
for entry in _load_snapshots(state_dir, vm_name):
|
||||
created = entry.get("created_at") or 0
|
||||
if retention_days is not None \
|
||||
and now - created > retention_days * 86400:
|
||||
pruned.append(entry)
|
||||
if delete_files and entry.get("path"):
|
||||
try:
|
||||
if os.path.exists(entry["path"]):
|
||||
os.remove(entry["path"])
|
||||
if events is not None:
|
||||
events.append(("deleted", entry, None))
|
||||
elif events is not None:
|
||||
events.append(("pruned_no_file", entry, None))
|
||||
except OSError as exc:
|
||||
if events is not None:
|
||||
events.append(("delete_failed", entry, exc))
|
||||
elif events is not None:
|
||||
events.append(("pruned_no_file", entry, None))
|
||||
else:
|
||||
kept.append(entry)
|
||||
if pruned:
|
||||
_save_snapshots(state_dir, vm_name, kept)
|
||||
return kept, pruned
|
||||
|
||||
|
||||
def check_snapshot_limits(state_dir: str, vm_name: str, max_count: int,
|
||||
max_total_gb: float,
|
||||
retention_days: float | None,
|
||||
events: list | None = None) -> list:
|
||||
"""快照上限预检(snapshot_vm 调 ch-remote 之前调):
|
||||
先按保留期剪枝,再查数量(已有数 + 本次 > 上限即
|
||||
snapshot_quota_exceeded)与总量(已有总量已达上限即
|
||||
拒绝;本次大小事前未知,总量在 record 时二次收紧,
|
||||
见 record_snapshot)。返回剪枝后条目。events 透传给
|
||||
prune_snapshots(见 P1-08),供调用方审计剪枝动作。"""
|
||||
kept, _ = prune_snapshots(state_dir, vm_name, retention_days,
|
||||
events=events)
|
||||
if len(kept) + 1 > max_count:
|
||||
raise ChvFailure(
|
||||
"snapshot_quota_exceeded",
|
||||
"快照数量已达上限 %d(VM %s)" % (max_count, vm_name))
|
||||
total: float = 0.0
|
||||
for entry in kept:
|
||||
size = entry.get("size")
|
||||
if isinstance(size, (int, float)):
|
||||
total += size
|
||||
elif entry.get("path") and os.path.exists(entry["path"]):
|
||||
try:
|
||||
total += os.path.getsize(entry["path"])
|
||||
except OSError:
|
||||
pass
|
||||
if total >= max_total_gb * 1024 ** 3:
|
||||
raise ChvFailure(
|
||||
"snapshot_quota_exceeded",
|
||||
"快照总量 %d 字节已达上限 %.1f GB(VM %s)"
|
||||
% (total, max_total_gb, vm_name))
|
||||
return kept
|
||||
|
||||
|
||||
def record_snapshot(state_dir: str, vm_name: str, snapshot_url: str, max_total_gb: float,
|
||||
events: list | None = None) -> tuple[list, list]:
|
||||
"""快照成功后登记;file:// 目标记实际大小。登记后若
|
||||
总量超上限,按最旧优先删登记外的旧快照文件(总量硬
|
||||
帽,删动作由调用方审计——本函数返回被删的旧条目)。
|
||||
返回(全部条目,被总量帽删掉的旧条目)。
|
||||
|
||||
feat-chv-p1fix(P1-06):单条快照本身大于总量帽时,
|
||||
在任何落盘/删旧动作之前直接拒绝
|
||||
(snapshot_quota_exceeded),不悄悄保留超大快照——
|
||||
旧循环 `while len > 1` 在单条超帽时直接停、超大快照
|
||||
被保留。大小未知(size 为 None,如远端目标或文件
|
||||
缺失)时无法判定,不拒绝,沿用旧口径。
|
||||
feat-chv-p1fix(P1-08):总量帽删文件动作记 events
|
||||
(("deleted"/"delete_failed"/"pruned_no_file", entry,
|
||||
error)),供调用方审计;删失败时登记条目仍移除
|
||||
(旧语义不变),但失败本身不再静默。
|
||||
"""
|
||||
import time
|
||||
path = None
|
||||
size = None
|
||||
if snapshot_url.startswith("file://"):
|
||||
path = snapshot_url[len("file://"):]
|
||||
try:
|
||||
size = os.path.getsize(path)
|
||||
except OSError:
|
||||
size = None
|
||||
if max_total_gb is not None and size is not None \
|
||||
and size > max_total_gb * 1024 ** 3:
|
||||
raise ChvFailure(
|
||||
"snapshot_quota_exceeded",
|
||||
"单条快照 %d 字节已超过总量上限 %.1f GB(VM %s),"
|
||||
"拒绝记录" % (size, max_total_gb, vm_name))
|
||||
entries = _load_snapshots(state_dir, vm_name)
|
||||
entries.append({"url": snapshot_url, "path": path,
|
||||
"size": size, "created_at": time.time()})
|
||||
_save_snapshots(state_dir, vm_name, entries)
|
||||
removed = _tighten_snapshot_cap(entries, max_total_gb, events)
|
||||
if removed:
|
||||
_save_snapshots(state_dir, vm_name, entries)
|
||||
return entries, removed
|
||||
|
||||
|
||||
def _tighten_snapshot_cap(entries, max_total_gb, events):
|
||||
"""总量帽收紧(record_snapshot 提取,守 50 行门禁):
|
||||
总量超帽时按最旧优先删登记外的旧快照文件,返回被删
|
||||
的旧条目;删动作记 events(见 record_snapshot)。"""
|
||||
removed = []
|
||||
if max_total_gb is None:
|
||||
return removed
|
||||
cap = max_total_gb * 1024 ** 3
|
||||
total = sum(e.get("size") or 0 for e in entries)
|
||||
while len(entries) > 1 and total > cap:
|
||||
oldest = entries.pop(0)
|
||||
if oldest.get("path") and os.path.exists(oldest["path"]):
|
||||
try:
|
||||
os.remove(oldest["path"])
|
||||
if events is not None:
|
||||
events.append(("deleted", oldest, None))
|
||||
except OSError as exc:
|
||||
if events is not None:
|
||||
events.append(("delete_failed", oldest, exc))
|
||||
elif events is not None:
|
||||
events.append(("pruned_no_file", oldest, None))
|
||||
total -= oldest.get("size") or 0
|
||||
removed.append(oldest)
|
||||
return removed
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Cloud Hypervisor 执行器的分类失败(叶子模块,不依赖
|
||||
cloud.sandbox,避免循环导入)。"""
|
||||
|
||||
class ChvFailure(RuntimeError):
|
||||
"""Cloud Hypervisor 分类失败:RuntimeError 的子类(worker
|
||||
现有 except Exception 路径照常捕获),另带 kind 字段供
|
||||
审计与失败分类用,绝不笼统只写 failed。kind 取值见
|
||||
CloudHypervisorExecutor.preflight / wake_vm 的注释。"""
|
||||
|
||||
def __init__(self, kind: str, message: str) -> None:
|
||||
super().__init__("chv_failure:%s: %s" % (kind, message))
|
||||
self.kind = kind
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,308 @@
|
||||
"""快照预热池 WarmPool(纯实验模块,非生产路径)。
|
||||
|
||||
本轨(feat-chv-arch-maint-fe)破环性清理:WarmPool 从
|
||||
cloud/chv/pool.py 移到本文件,cloud/chv/__init__ 与
|
||||
cloud/sandbox.py 不再 re-export 它,生产代码 import 本
|
||||
模块即违规(见 docs/chv-architecture.md ADR-12 的禁
|
||||
|
||||
import 清单与测试钉住)。实验用法只走
|
||||
`from cloud.chv.experimental_pool import WarmPool`。
|
||||
生产残留清理已彻底分文件:见 cloud/chv/orphans.py 的
|
||||
reconcile_orphans_for_executor(),它不 import 本模块。
|
||||
|
||||
定案(架构决策,见 docs/chv-architecture.md ADR-5):WarmPool
|
||||
不再宣称为生产快速启动特性。原因:预热 VM 的磁盘是干净快照
|
||||
盘,不是任务盘;dispatch 前查池若要真正接线,必须在派任务
|
||||
前给已暂停 VM 重新供给任务盘(热换盘/二盘挂载/host.py 磁盘
|
||||
语义三者都未实现),把预热盘直接当任务盘用会击穿“任务盘
|
||||
隔离”不变量。acquire 内 resume 只是实验性接口,worker
|
||||
dispatch 永远走冷启动(wake_vm),_run_with_vm 绝不查池。
|
||||
本模块的 reconcile_orphans() 只留给实验/旧测试直接调用;
|
||||
生产启动/关闭走 orphans facade,不再经本类。
|
||||
warm_pool_wired_to_dispatch 恒为 False,任何把它改成
|
||||
True 的改动都必须先实现任务盘重供给并补不变量测试,
|
||||
否则不得合入。
|
||||
|
||||
|
||||
隔离红线(与 #216 一致,绝不妥协):预热条目是一次性的
|
||||
干净快照 VM,acquire 消耗掉它,release 一律销毁,绝不把
|
||||
一个任务用过的 VM(带任务文件、带 agent 痕迹)翻出来给
|
||||
下一个任务/租户复用;池子空了就回落冷启动(wake_vm),
|
||||
绝不因为池空就降级 Docker/Local。
|
||||
|
||||
Registry(本轨新增):
|
||||
- 文件:state_dir/warm-pool.json(registry_path 可覆盖),
|
||||
内容为 {<合成名>: {"state", "vm", "task_id", "snapshot_url"}}。
|
||||
每次 prewarm/acquire/release 后立即落盘(临时文件 + 原子
|
||||
rename),worker 崩溃后重启仍能发现上一世的条目。
|
||||
- 载入时绝不复用:新池实例从 registry 读到的条目一律标为
|
||||
orphan(不计入 warm_count,acquire 绝不发给任务),因为
|
||||
旧 VMM 进程已随旧 worker 消失,无法验证它还是当初那个
|
||||
干净快照 VM。它们只能走 reconcile_orphans() 清理。
|
||||
- reconcile_orphans():worker 启动/清理时的显式入口,对
|
||||
每个 orphan/assigned 条目尽力调 executor.shutdown_vm
|
||||
(清 socket 与残留登记),条目清掉并落盘,返回已清理的
|
||||
VM 名列表。registry 里没有的残留 socket 仍由执行器的
|
||||
cleanup_stale_vms() 扫 state_dir 处理,两者互补,不互相
|
||||
替代;reconcile 绝不在构造时自动执行(同一 state_dir 只
|
||||
能有一个活池,构造时自动清理会误杀另一个活池的 VM)。
|
||||
- 损坏 fail-safe:registry 不是合法 JSON / 顶层不是 dict 时,
|
||||
整个文件 quarantine 成 <path>.corrupt(尽力而为),池子
|
||||
从空开始,绝不用损坏数据复用 VM;单条目结构非法时只
|
||||
丢该条(记 registry_corrupt_entries),不连坐其它条目。
|
||||
registry 写失败时只记日志警告、不炸任务流(池子内存态
|
||||
仍可用,重启后孤儿改由 socket 扫描兜底)。
|
||||
"""
|
||||
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from cloud.chv.executor import CloudHypervisorExecutor
|
||||
|
||||
import os
|
||||
|
||||
from cloud.chv.orphans import _REGISTRY_FILENAME, _read_registry, _write_registry
|
||||
|
||||
|
||||
class WarmPool:
|
||||
"""快照预热池(实验性,见模块头定案)。
|
||||
|
||||
wired_to_dispatch = False:生产 dispatch 不查池。
|
||||
|
||||
|
||||
目标:冷启动(preflight + 起 VMM + create + boot + SSH
|
||||
就绪)是慢路径;预热池提前把干净 VM 起好并暂停,任务
|
||||
来时 acquire 直接 resume,省掉冷启动的大头。
|
||||
状态机:cold(未预热)→ prewarming(wake 中)→ warm
|
||||
(已暂停待分配)→ assigned(已分配给某任务)→
|
||||
running(已 resume)→ recycled/destroyed(release 后)。
|
||||
prewarm 的实现:对每个条目调 executor.wake_vm(冷启动)
|
||||
再 pause_vm,得到 paused 的干净 VM;mock 单测里 wake/
|
||||
pause 均可注入假实现,真机上这一步需要 KVM 宿主(见
|
||||
就绪清单“需 KVM 宿主实测”)。snapshot_url 未设时
|
||||
prewarm 返回 [](无基准快照可预热,不是静默造快照)。
|
||||
"""
|
||||
|
||||
wired_to_dispatch = False
|
||||
|
||||
def __init__(self, executor: "CloudHypervisorExecutor", size: int | None = None,
|
||||
snapshot_url: str | None = None,
|
||||
registry_path: str | None = None) -> None:
|
||||
self.executor = executor
|
||||
self.size = int(size if size is not None
|
||||
else getattr(executor, "warm_pool_size", 0))
|
||||
self.snapshot_url = snapshot_url \
|
||||
if snapshot_url is not None \
|
||||
else getattr(executor, "snapshot_url", None)
|
||||
self._entries: dict = {} # 合成名 -> {"state", "vm", "task", ...}
|
||||
self.registry_path = registry_path
|
||||
if self.registry_path is None:
|
||||
state_dir = getattr(executor, "state_dir", None)
|
||||
if state_dir:
|
||||
self.registry_path = os.path.join(
|
||||
state_dir, _REGISTRY_FILENAME)
|
||||
self.registry_corrupt = False
|
||||
self.registry_corrupt_entries = 0
|
||||
self._load_registry()
|
||||
|
||||
# -- registry ------------------------------------------------------
|
||||
def _load_registry(self) -> None:
|
||||
"""载入 registry:读到的条目一律标 orphan(见模块
|
||||
docstring,绝不直接当 warm 发给任务)。无文件时为空
|
||||
池;损坏时 quarantine 并从空开始。文件 IO 与损坏
|
||||
语义唯一来源是 cloud.chv.orphans(生产模块),本
|
||||
方法只做“条目 → 内存 orphan 态”的映射。"""
|
||||
if not self.registry_path:
|
||||
return
|
||||
data, corrupt = _read_registry(self.registry_path)
|
||||
if corrupt:
|
||||
self.registry_corrupt = True
|
||||
return
|
||||
if data is None:
|
||||
return
|
||||
for name, entry in data.items():
|
||||
if not isinstance(entry, dict) \
|
||||
or not entry.get("vm") \
|
||||
or entry.get("state") not in ("warm", "assigned"):
|
||||
self.registry_corrupt_entries += 1
|
||||
continue
|
||||
self._entries[name] = {
|
||||
"state": "orphan",
|
||||
"vm": entry["vm"],
|
||||
"task": {"id": entry.get("task_id") or name},
|
||||
"snapshot_url": entry.get("snapshot_url"),
|
||||
}
|
||||
|
||||
def _save_registry(self) -> None:
|
||||
"""落盘当前条目(只写 warm/assigned 态;orphan 是
|
||||
待清理的上一世条目,reconcile 后即删,不写回)。
|
||||
写失败只记日志,不炸任务流。落盘 IO 委托 orphans。"""
|
||||
if not self.registry_path:
|
||||
return
|
||||
data = {}
|
||||
for name, entry in self._entries.items():
|
||||
if entry["state"] not in ("warm", "assigned"):
|
||||
continue
|
||||
task = entry.get("task") or {}
|
||||
task_id = task.get("id") if isinstance(task, dict) \
|
||||
else None
|
||||
data[name] = {
|
||||
"state": entry["state"],
|
||||
"vm": entry.get("vm"),
|
||||
"task_id": task_id,
|
||||
"snapshot_url": entry.get("snapshot_url"),
|
||||
}
|
||||
_write_registry(self.registry_path, data)
|
||||
|
||||
def reconcile_orphans(self) -> list:
|
||||
"""清理本池内存里的 orphan/assigned 条目(实验入口)。
|
||||
|
||||
对每个条目尽力 shutdown_vm,条目清掉并落盘,返回
|
||||
已清理的 VM 名列表。生产启动/关闭不走本方法,走
|
||||
cloud.chv.orphans.reconcile_orphans_for_executor()。
|
||||
feat-chv-p1fix(P1-01 同款语义):shutdown 失败的
|
||||
条目不进 cleaned、不从内存条目里 pop(下次 reconcile
|
||||
可重试),并记 failed 审计;文件落盘仍只写
|
||||
warm/assigned 态(见 P2-16,实验路径的文件语义
|
||||
本轨不改,内存态保留即重试依据)。"""
|
||||
cleaned = []
|
||||
failed = []
|
||||
for name, entry in list(self._entries.items()):
|
||||
if entry["state"] not in ("orphan", "assigned"):
|
||||
continue
|
||||
vm = entry.get("vm") or name
|
||||
try:
|
||||
self.executor.shutdown_vm(vm)
|
||||
except Exception:
|
||||
failed.append(vm)
|
||||
continue
|
||||
self._entries.pop(name, None)
|
||||
cleaned.append(vm)
|
||||
if cleaned:
|
||||
self._save_registry()
|
||||
try:
|
||||
self.executor._emit(
|
||||
"warm_pool_reconcile", {"id": None},
|
||||
"cleaned=%s" % ",".join(cleaned))
|
||||
except Exception:
|
||||
pass
|
||||
if failed:
|
||||
try:
|
||||
self.executor._emit(
|
||||
"warm_pool_reconcile_failed", {"id": None},
|
||||
"failed=%s cleaned=%s" % (
|
||||
",".join(failed), ",".join(cleaned)))
|
||||
except Exception:
|
||||
pass
|
||||
return cleaned
|
||||
|
||||
# -- 池操作 ----------------------------------------------------------
|
||||
def warm_count(self) -> int:
|
||||
return sum(1 for e in self._entries.values()
|
||||
if e["state"] == "warm")
|
||||
|
||||
def orphan_count(self) -> int:
|
||||
"""上一世残留条目数(待 reconcile,不可分配)。"""
|
||||
return sum(1 for e in self._entries.values()
|
||||
if e["state"] in ("orphan", "assigned"))
|
||||
|
||||
def prewarm(self, count: int | None = None) -> list:
|
||||
"""预热到目标数:已有 warm 条目不重复建;每个新条目
|
||||
用一个合成任务名 wake + pause(合成任务 id 为
|
||||
warm-<序号>,绝不与真实任务 id 撞名)。返回新增的
|
||||
warm VM 名列表。"""
|
||||
if self.snapshot_url is None:
|
||||
return []
|
||||
target = int(count) if count is not None else self.size
|
||||
added = []
|
||||
i = 0
|
||||
while self.warm_count() < target:
|
||||
i += 1
|
||||
name = "warm-%d" % i
|
||||
if name in self._entries:
|
||||
continue
|
||||
task = {"id": name}
|
||||
self._entries[name] = {"state": "prewarming",
|
||||
"task": task}
|
||||
try:
|
||||
self.executor.wake_vm(task)
|
||||
self.executor.pause_vm(task)
|
||||
except Exception:
|
||||
self._entries.pop(name, None)
|
||||
raise
|
||||
vm = self.executor.vm_name(task)
|
||||
self._entries[name] = {
|
||||
"state": "warm", "task": task, "vm": vm,
|
||||
"snapshot_url": self.snapshot_url}
|
||||
added.append(name)
|
||||
if added:
|
||||
self._save_registry()
|
||||
return added
|
||||
|
||||
def acquire(self, task: dict) -> dict:
|
||||
"""分配一个预热 VM 给任务(feat-chv-fix,P1-3 接线):
|
||||
|
||||
有 warm 条目时先调 executor.resume_vm() 把暂停的 VM
|
||||
真正唤醒(resume 失败即条目作废、继续找下一个,
|
||||
绝不把 paused 的 VM 当 running 发给任务),再标
|
||||
assigned 并返回 mode=warm;resume 全部失败或无 warm
|
||||
条目时返回 mode=cold(调用方回落 wake_vm 冷启动)。
|
||||
注意:本方法只解决“acquire 不再悬空 resume 责任”
|
||||
这一半;worker dispatch 前查池的另一半仍未接线
|
||||
(见执行器/就绪文档的“未接线缺口”标注),预热收益
|
||||
在生产路径上仍不是已交付特性。
|
||||
feat-chv-secfix(L5,接线时的盘对应关系定义——接线
|
||||
本身仍未做,不假装已接):预热 VM 的磁盘是干净快照
|
||||
盘,不是任务盘。将来 dispatch 前查池接线时必须做到:
|
||||
acquire 返回的 VM 在派任务前仍须按任务重新供给磁盘
|
||||
(provision_task_disk 的 copy 盘,或校验快照来源与该
|
||||
任务的对应关系后才能用快照盘),绝不能把预热盘直接
|
||||
当任务盘用;接线时须补“任务盘来源”不变量的测试,
|
||||
否则 acquire 的 warm 路径不得合入生产路径。"""
|
||||
for name, entry in list(self._entries.items()):
|
||||
if entry["state"] != "warm":
|
||||
continue
|
||||
resume = getattr(self.executor, "resume_vm", None)
|
||||
if resume is not None:
|
||||
try:
|
||||
resume(entry.get("task") or task)
|
||||
except Exception:
|
||||
self._entries.pop(name, None)
|
||||
self._save_registry()
|
||||
continue
|
||||
# 无 resume_vm 的鸭子型执行器(旧测试替身):
|
||||
# 保持旧 assigned 语义,不因此丢弃条目;真实
|
||||
# CloudHypervisorExecutor 一律有 resume_vm,走
|
||||
# 上面分支,责任不悬空。
|
||||
entry["state"] = "assigned"
|
||||
entry["task"] = task
|
||||
self._save_registry()
|
||||
try:
|
||||
self.executor._emit(
|
||||
"warm_pool_acquire", task,
|
||||
"vm=%s" % name)
|
||||
except Exception:
|
||||
pass
|
||||
return {"mode": "warm", "vm": name,
|
||||
"pool_task": entry["task"]}
|
||||
return {"mode": "cold", "vm": None, "pool_task": None}
|
||||
|
||||
def release(self, task: dict, pool_vm: str | None = None) -> bool:
|
||||
"""释放:无论 warm 还是 cold 路径,VM 一律销毁
|
||||
(executor.shutdown_vm),池条目清掉;绝不把用过
|
||||
的 VM 放回 warm 池。返回 True 表示确实销毁了一个
|
||||
条目或 VM。"""
|
||||
destroyed = False
|
||||
if pool_vm and pool_vm in self._entries:
|
||||
self._entries.pop(pool_vm, None)
|
||||
destroyed = True
|
||||
try:
|
||||
if self.executor.shutdown_vm(task):
|
||||
destroyed = True
|
||||
except Exception:
|
||||
pass
|
||||
self._save_registry()
|
||||
self.executor._emit(
|
||||
"warm_pool_release", task,
|
||||
"vm=%s destroyed=%s" % (pool_vm, destroyed))
|
||||
return destroyed
|
||||
@@ -0,0 +1,213 @@
|
||||
"""CHV 安全加固 helper(feat-chv-secfix,叶子模块)。
|
||||
|
||||
这里只放不依赖执行器类的纯校验/文件操作:state_dir 安全
|
||||
目录、VMM socket 活性、net 条目隔离校验、快照 URL 白名单、
|
||||
egress 代理地址校验。宿主防火墙规则、tap 桥接本身是运维
|
||||
外部状态,代码建不出来也不校验——那部分只写进部署清单
|
||||
(docs/chv-production-readiness.md §11),绝不在代码里
|
||||
假装已隔离。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
|
||||
from cloud.chv.errors import ChvFailure
|
||||
|
||||
# NET_JSON 条目字段白名单(L4):tap/mac/ip/mask 之外的字段
|
||||
# 一律拒绝——未知字段原样送进 VMM API 的行为不确定,静默
|
||||
# 透传等于把 VMM 的字段语义赌在运维手写上。
|
||||
_NET_FIELDS = frozenset({"tap", "mac", "ip", "mask"})
|
||||
|
||||
|
||||
def ensure_secure_state_dir(state_dir: str) -> str:
|
||||
"""建 state_dir 并保证 0700(H6):目录不存在即按 0700
|
||||
建;已存在但组/其它位可访时 chmod 0700;失败即
|
||||
ChvFailure(state_dir_insecure),绝不带着 /tmp 式
|
||||
宽松目录启动 VM。umask 影响 mkdir 的默认权限,故建
|
||||
后一律显式 chmod 校验一次。"""
|
||||
try:
|
||||
os.makedirs(state_dir, exist_ok=True)
|
||||
os.chmod(state_dir, 0o700)
|
||||
except OSError as exc:
|
||||
raise ChvFailure(
|
||||
"state_dir_insecure",
|
||||
"state_dir 无法建成安全目录(0700):%s:%s"
|
||||
% (state_dir, exc))
|
||||
try:
|
||||
mode = os.stat(state_dir).st_mode & 0o777
|
||||
except OSError as exc:
|
||||
raise ChvFailure(
|
||||
"state_dir_insecure",
|
||||
"state_dir 无法校验权限:%s:%s" % (state_dir, exc))
|
||||
if mode & 0o077:
|
||||
raise ChvFailure(
|
||||
"state_dir_insecure",
|
||||
"state_dir 权限仍宽松(%o),拒绝启动:%s"
|
||||
% (mode, state_dir))
|
||||
return state_dir
|
||||
|
||||
|
||||
def socket_is_live(socket_path: str) -> bool:
|
||||
"""VMM api-socket 是否真有监听者(M7):能连通才算活。
|
||||
普通文件、空文件、已死 VMM 的残留 socket 一律算死——
|
||||
容量计数不再被残留文件占满。连接失败的任何异常都算
|
||||
死(不区分 refused/超时,计数是尽力而为的读操作)。
|
||||
注意竞态:刚启动、尚未 listen 的 VMM 会被判死,容量
|
||||
门在并发唤醒时可能短暂低估——文档明写该窗口,容量门
|
||||
是“拒绝超限”的护栏,不是精确的分布式锁。"""
|
||||
try:
|
||||
if not socket_path or not os.path.exists(socket_path):
|
||||
return False
|
||||
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
try:
|
||||
sock.settimeout(0.5)
|
||||
return sock.connect_ex(socket_path) == 0
|
||||
finally:
|
||||
sock.close()
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def parse_net_entries(net_json: str) -> list:
|
||||
"""解析 CHV_NET_JSON 为条目 list(非法 JSON / 非
|
||||
list/dict 即 net_config_invalid)。"""
|
||||
if not net_json:
|
||||
return []
|
||||
try:
|
||||
net = json.loads(net_json)
|
||||
except ValueError:
|
||||
raise ChvFailure("net_config_invalid",
|
||||
"CHV_NET_JSON 不是合法 JSON")
|
||||
entries = net if isinstance(net, list) else [net]
|
||||
if not all(isinstance(e, dict) for e in entries):
|
||||
raise ChvFailure("net_config_invalid",
|
||||
"CHV_NET_JSON 条目须为 dict")
|
||||
return entries
|
||||
|
||||
|
||||
def validate_net_entries(entries: list, existing_entries: list) -> list:
|
||||
"""net 条目隔离校验(H7 + L4):
|
||||
|
||||
- 字段白名单:未知字段即 net_config_invalid;
|
||||
- 条目内 tap/mac/ip 归一化(mac 小写、空 tap 视为未
|
||||
指定,不参与 tap 冲突比较——空 tap 是“VMM 默认网卡”
|
||||
的运维写法,代码不编造 tap 名,但空 tap 多 VM 共用
|
||||
默认网卡的风险仍在部署清单里明写);
|
||||
- 本次条目内 tap/mac/ip 重复即 net_config_invalid;
|
||||
- 与已登记 VM(existing_entries,每项为 {tap, mac, ip,
|
||||
vm})的 tap/mac/ip 任一重复即 net_tap_conflict,
|
||||
拒绝启动并由调用方审计。
|
||||
返回归一化后的条目 list。"""
|
||||
normalized = [_normalize_net_entry(e) for e in entries]
|
||||
_check_net_duplicates(normalized)
|
||||
_check_net_conflicts(normalized, existing_entries)
|
||||
return normalized
|
||||
|
||||
|
||||
def _normalize_net_entry(entry: dict) -> dict:
|
||||
"""单条 net 条目归一化:字段白名单 + tap/mac/ip 归一化。
|
||||
|
||||
空 tap 视为未指定,不参与 tap 冲突比较(见
|
||||
validate_net_entries 的说明,纯移动)。"""
|
||||
unknown = set(entry) - _NET_FIELDS
|
||||
if unknown:
|
||||
raise ChvFailure(
|
||||
"net_config_invalid",
|
||||
"CHV_NET_JSON 条目含未知字段:%s(白名单:%s)"
|
||||
% (",".join(sorted(unknown)),
|
||||
",".join(sorted(_NET_FIELDS))))
|
||||
return {"tap": str(entry.get("tap") or "").strip(),
|
||||
"mac": str(entry.get("mac") or "").strip().lower(),
|
||||
"ip": str(entry.get("ip") or "").strip(),
|
||||
"mask": str(entry.get("mask") or "")}
|
||||
|
||||
|
||||
def _check_net_duplicates(normalized: list) -> None:
|
||||
"""本次条目内 tap/mac/ip 重复即 net_config_invalid。"""
|
||||
seen: dict[str, set[str]] = {"tap": set(), "mac": set(), "ip": set()}
|
||||
for entry in normalized:
|
||||
for kind in ("tap", "mac", "ip"):
|
||||
value = entry.get(kind)
|
||||
if not value:
|
||||
continue
|
||||
if value in seen[kind]:
|
||||
raise ChvFailure(
|
||||
"net_config_invalid",
|
||||
"CHV_NET_JSON 内 %s 重复:%s" % (kind, value))
|
||||
seen[kind].add(value)
|
||||
|
||||
|
||||
def _check_net_conflicts(normalized: list, existing_entries: list) -> None:
|
||||
"""与已登记 VM 的 tap/mac/ip 任一重复即 net_tap_conflict。"""
|
||||
for entry in normalized:
|
||||
for other in existing_entries:
|
||||
for kind in ("tap", "mac", "ip"):
|
||||
value = entry.get(kind)
|
||||
if value and value == other.get(kind):
|
||||
raise ChvFailure(
|
||||
"net_tap_conflict",
|
||||
"net %s %s 已被 VM %s 使用,拒绝共享"
|
||||
" tap/mac/ip:每 VM 必须独立"
|
||||
% (kind, value, other.get("vm")))
|
||||
|
||||
|
||||
def validate_egress_proxy(egress_proxy: str) -> str | None:
|
||||
"""egress 代理地址校验(H1 代码门的一部分):须为
|
||||
http/https URL 且带主机名;否则 net_config_invalid…
|
||||
实际 kind 用 config_invalid(preflight 的配置类失败)。
|
||||
空值由调用方另行处理(未设 + 未显式允许直连时
|
||||
preflight 已有 egress_not_configured)。"""
|
||||
if not egress_proxy:
|
||||
return None
|
||||
try:
|
||||
from urllib.parse import urlparse
|
||||
parsed = urlparse(egress_proxy)
|
||||
except ValueError:
|
||||
parsed = None
|
||||
if parsed is None or parsed.scheme not in ("http", "https") \
|
||||
or not parsed.hostname:
|
||||
raise ChvFailure(
|
||||
"config_invalid",
|
||||
"CHV_EGRESS_PROXY 必须是 http/https 代理地址:%r"
|
||||
% (egress_proxy,))
|
||||
return egress_proxy
|
||||
|
||||
|
||||
def _is_within(directory, path):
|
||||
"""目录内判定:经 compat215.resolve_inside(#215 目录门
|
||||
同签名),越界回 None 即不在内。绝对路径先转相对路径
|
||||
再过门,.. 逃逸与符号链接逃逸一律挡。"""
|
||||
from cloud.chv.compat215 import resolve_inside
|
||||
rel = os.path.relpath(path, directory)
|
||||
return resolve_inside(directory, rel) is not None
|
||||
|
||||
|
||||
def validate_snapshot_url(snapshot_url: str, state_dir: str,
|
||||
snapshot_dir: str | None = None,
|
||||
allow_remote: bool = False) -> str | None:
|
||||
"""快照 URL 白名单(L2):
|
||||
|
||||
- file://:目标须在 state_dir 或 CHV_SNAPSHOT_DIR 内,
|
||||
否则 snapshot_path_not_allowed;
|
||||
- 其它协议(含无 :// 的裸路径由调用方先补 file://):
|
||||
须显式 CHV_ALLOW_REMOTE_SNAPSHOT=1,否则
|
||||
snapshot_remote_not_allowed。
|
||||
返回 file:// 目标的本地路径(非 file:// 时返回 None)。"""
|
||||
if snapshot_url.startswith("file://"):
|
||||
path = snapshot_url[len("file://"):]
|
||||
allowed = [state_dir]
|
||||
if snapshot_dir:
|
||||
allowed.append(snapshot_dir)
|
||||
if not any(_is_within(d, path) for d in allowed):
|
||||
raise ChvFailure(
|
||||
"snapshot_path_not_allowed",
|
||||
"file:// 快照须在 state_dir 或 CHV_SNAPSHOT_DIR "
|
||||
"内:%s" % snapshot_url)
|
||||
return path
|
||||
if not allow_remote:
|
||||
raise ChvFailure(
|
||||
"snapshot_remote_not_allowed",
|
||||
"非 file:// 快照须显式设 CHV_ALLOW_REMOTE_SNAPSHOT=1:%s"
|
||||
% snapshot_url)
|
||||
return None
|
||||
@@ -0,0 +1,248 @@
|
||||
"""按 VM 生成的 SSH host key 预置(替代 TOFU 的强方案)。
|
||||
|
||||
安全模型(补缺口轨,替代 #221 的 ssh-keyscan 首次捕获):
|
||||
- 绝不用镜像固定同一把 host key:所有 VM 共用一把 key 时,
|
||||
一台 VM 泄漏即全网可冒充,属安全事故。
|
||||
- 每 VM 一次性生成:宿主在 VMM 启动前为该 VM 生成一对
|
||||
ed25519 host key(私钥只活在该 VM 的 seed 里,随 VM
|
||||
销毁而删,绝不跨 VM 复用、绝不写进镜像、绝不进已知
|
||||
主机文件之外的共享位置)。
|
||||
- 公钥预置钉死:生成后立即把公钥写进该 VM 的
|
||||
state_dir/<vm>/known_hosts(格式 [127.0.0.1]:<port> ...),
|
||||
执行器在 VM 启动前就已钉死,启动后 SSH 直接用该文件
|
||||
校验,换 key 即拒绝——不再靠启动后 ssh-keyscan 抓到的
|
||||
key 来“事后钉死”。
|
||||
- 私钥注入通道(选 cloud-init NoCloud,写清唯一通道):
|
||||
seed 目录 state_dir/<vm>/seed/ 含 meta-data(instance-id
|
||||
即 VM 名)、user-data(write_files 写 /etc/ssh/ 的 host
|
||||
key 对并重启 sshd)。客机镜像硬性要求(见
|
||||
chv-guest/README.md §5):必须装 cloud-init 且支持
|
||||
NoCloud seed、首启时先应用 seed 再启动 sshd、且不得在
|
||||
seed 应用前/后无条件重新生成 host key(否则预置即失效)。
|
||||
seed 磁盘/目录是 per-VM 敏感物,VM 关闭/回收时必须删。
|
||||
- 回落边界:ssh-keyscan 首次捕获只作弱回落
|
||||
(CHV_HOST_KEY_MODE=keyscan,且文档标为弱方案):
|
||||
它只能防首次之后的替换,不能验证首次本身。生产部署
|
||||
必须用 provisioned 模式;provisioned 模式下 seed 工具
|
||||
或 keygen 不可用时直接 fail-fast,绝不静默回落 keyscan。
|
||||
|
||||
本模块只做纯函数与文件操作,不跑 VM、不联网、不含
|
||||
任何私钥值(测试用假 runner 时私钥由测试自己写文件)。
|
||||
"""
|
||||
|
||||
import os
|
||||
import shlex
|
||||
from collections.abc import Callable
|
||||
|
||||
# seed 投递的客内密钥环境文件路径(feat-chv-secfix,H2):
|
||||
# 只在 provisioned 模式的 seed user-data 里写出,0600 root,
|
||||
# 客内 host.py 命令执行前由远程命令 source 进环境;宿主侧
|
||||
# 除 seed 目录/user-data(随 VM 销毁而删)外不另存该文件。
|
||||
GUEST_ENV_PATH = "/etc/chv/guest-env"
|
||||
|
||||
|
||||
def host_key_paths(state_dir: str, vm_name: str) -> dict:
|
||||
"""该 VM 的 host key 文件路径(私钥/公钥/known_hosts)。"""
|
||||
base = os.path.join(state_dir, vm_name)
|
||||
return {
|
||||
"private": os.path.join(base, "ssh_host_ed25519_key"),
|
||||
"public": os.path.join(base, "ssh_host_ed25519_key.pub"),
|
||||
"known_hosts": os.path.join(base, "known_hosts"),
|
||||
"seed_dir": os.path.join(base, "seed"),
|
||||
}
|
||||
|
||||
|
||||
def dirty_marker_path(state_dir: str, vm_name: str) -> str:
|
||||
"""清理失败脏标记路径(feat-chv-secfix,M2):stop_vm
|
||||
清理 seed/私钥/磁盘任一步失败时写下该标记;标记存在
|
||||
时同名 VM 拒绝启动(清理失败即拒起),人工清理并删
|
||||
标记后才能再起——“已审计即可”不再当作根治。"""
|
||||
return os.path.join(state_dir, vm_name, ".cleanup-dirty")
|
||||
|
||||
|
||||
def delete_host_private_key(state_dir: str, vm_name: str) -> str:
|
||||
"""删宿主侧 host 私钥文件(feat-chv-secfix,H5):seed
|
||||
盘建好后宿主私钥已无存在必要(客机已有私钥、宿主只需
|
||||
公钥 + known_hosts 做校验)。删失败即 OSError,由调用
|
||||
方 fail-fast(VM 尚未启动,此时拒绝是最安全的)。"""
|
||||
path = host_key_paths(state_dir, vm_name)["private"]
|
||||
if os.path.exists(path):
|
||||
os.remove(path)
|
||||
return path
|
||||
|
||||
|
||||
def known_hosts_line(port: int, public_key_text: str, host: str = "127.0.0.1") -> str:
|
||||
"""公钥文件内容转 known_hosts 行:只取前两列(类型+数据)。
|
||||
|
||||
公钥文件形如 'ssh-ed25519 AAAA... [comment]',known_hosts
|
||||
行形如 '[<host>]:<port> ssh-ed25519 AAAA...'(host 缺省
|
||||
回环,仅作兼容;客机 IP 方案下 host 应为解析出的客机 IP)。
|
||||
非法内容 ValueError,绝不把陌生行原样写进文件。"""
|
||||
parts = (public_key_text or "").strip().split()
|
||||
if len(parts) < 2 or not parts[0].startswith("ssh-"):
|
||||
raise ValueError("host 公钥格式非法")
|
||||
return "[%s]:%s %s %s" % (host, port, parts[0], parts[1])
|
||||
|
||||
|
||||
def seed_disk_path(state_dir: str, vm_name: str) -> str:
|
||||
"""该 VM 的 NoCloud seed 磁盘镜像路径(CIDATA 卷标 ISO)。"""
|
||||
return os.path.join(state_dir, vm_name, "seed.iso")
|
||||
|
||||
|
||||
def build_seed_disk(seed_dir: str, seed_disk: str,
|
||||
runner: Callable[[list[str]], object] | None = None) -> str:
|
||||
"""把 seed 目录做成 CIDATA 卷标 ISO(feat-chv-fix,P0-1 的
|
||||
注入载体):genisoimage -volid CIDATA -joliet -rock。
|
||||
|
||||
runner 不给时用 subprocess;测试可注入假 runner(假
|
||||
runner 负责把 seed_disk 文件写出来)。工具失败、返回码
|
||||
非 0、或产物不存在/为空时一律 RuntimeError,调用方
|
||||
(wake_vm provisioned 路径)fail-fast,绝不带着“seed
|
||||
目录已写但未挂进 VmConfig”的 VM 继续启动。seed 目录
|
||||
本身只是中间物,不再被当成注入通道。"""
|
||||
cmd = ["genisoimage", "-output", seed_disk, "-volid", "CIDATA",
|
||||
"-joliet", "-rock", seed_dir]
|
||||
if runner is not None:
|
||||
result = runner(cmd)
|
||||
rc = getattr(result, "returncode", 0)
|
||||
if rc not in (0, None):
|
||||
raise RuntimeError("genisoimage 失败:returncode=%s" % rc)
|
||||
else:
|
||||
import subprocess
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError("genisoimage 失败:%s" % result.stderr)
|
||||
if not os.path.isfile(seed_disk) or os.path.getsize(seed_disk) == 0:
|
||||
raise RuntimeError("seed 磁盘未生成:%s" % seed_disk)
|
||||
# feat-chv-secfix(H5):seed.iso 内含 host 私钥明文
|
||||
# user-data,生成即 0600,不再取 umask 默认 0644。
|
||||
try:
|
||||
os.chmod(seed_disk, 0o600)
|
||||
except OSError as exc:
|
||||
raise RuntimeError("seed 磁盘 chmod 0600 失败:%s" % exc)
|
||||
return seed_disk
|
||||
|
||||
|
||||
def build_seed_user_data(private_key_text: str, public_key_text: str) -> str:
|
||||
"""cloud-init user-data:把 host key 对写进客机 /etc/ssh。
|
||||
|
||||
注意:user-data 本身含私钥,只写进 per-VM seed 目录
|
||||
(0600),随 VM 销毁而删,绝不进镜像、绝不进日志、
|
||||
绝不进任务 prompt 落库。"""
|
||||
if not private_key_text or not public_key_text:
|
||||
raise ValueError("host key 对缺失,无法生成 seed")
|
||||
lines = [
|
||||
"#cloud-config",
|
||||
"ssh_deletekeys: false",
|
||||
"write_files:",
|
||||
" - path: /etc/ssh/ssh_host_ed25519_key",
|
||||
" permissions: '0600'",
|
||||
" owner: root:root",
|
||||
" content: |",
|
||||
]
|
||||
for line in private_key_text.rstrip("\n").split("\n"):
|
||||
lines.append(" " + line)
|
||||
lines += [
|
||||
" - path: /etc/ssh/ssh_host_ed25519_key.pub",
|
||||
" permissions: '0644'",
|
||||
" owner: root:root",
|
||||
" content: |",
|
||||
]
|
||||
for line in public_key_text.rstrip("\n").split("\n"):
|
||||
lines.append(" " + line)
|
||||
lines += [
|
||||
"runcmd:",
|
||||
" - chmod 600 /etc/ssh/ssh_host_ed25519_key",
|
||||
" - systemctl restart ssh || service ssh restart || true",
|
||||
]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def write_seed(seed_dir: str, vm_name: str, private_key_text: str,
|
||||
public_key_text: str,
|
||||
secret_env: dict | None = None) -> str:
|
||||
"""写 seed 目录:meta-data + user-data(均为 0600)。
|
||||
|
||||
返回 seed 目录路径。seed 目录本身不含已知主机之外的
|
||||
共享数据,调用方(执行器关闭/回收)负责删除它。
|
||||
feat-chv-secfix(H2):secret_env 给定时,user-data 再
|
||||
写一个 write_files 条目,把密钥环境变量写进客内
|
||||
GUEST_ENV_PATH(0600 root),值经 shlex.quote 引用后
|
||||
才进文件内容——远程命令 source 该文件即可得到密钥
|
||||
环境,密钥值从不出现在宿主 ssh argv、宿主进程环境与
|
||||
审计明文里。secret_env 为空时不写该条目(客内文件不
|
||||
存在时远程命令也不 source 它,见执行器)。"""
|
||||
os.makedirs(seed_dir, exist_ok=True)
|
||||
meta_path = os.path.join(seed_dir, "meta-data")
|
||||
user_path = os.path.join(seed_dir, "user-data")
|
||||
with open(meta_path, "w") as fh:
|
||||
fh.write("instance-id: %s\nlocal-hostname: %s\n"
|
||||
% (vm_name, vm_name))
|
||||
os.chmod(meta_path, 0o600)
|
||||
user_data = build_seed_user_data(private_key_text, public_key_text)
|
||||
if secret_env:
|
||||
# 注意:只能把条目并进已有的 write_files 列表,绝不能
|
||||
# 再写一个 write_files: 键(YAML 重复键会覆盖掉 host
|
||||
# key 的两条写文件条目,预置即失效)。条目插在 runcmd
|
||||
# 段之前、仍属同一列表。
|
||||
lines = [
|
||||
" - path: %s" % GUEST_ENV_PATH,
|
||||
" permissions: '0600'",
|
||||
" owner: root:root",
|
||||
" content: |",
|
||||
]
|
||||
for key in sorted(secret_env):
|
||||
lines.append(" %s=%s"
|
||||
% (key, shlex.quote(str(secret_env[key]))))
|
||||
# 条目插在 runcmd 段之前、仍属同一 write_files 列表。
|
||||
marker = "runcmd:\n"
|
||||
if marker in user_data:
|
||||
head, tail = user_data.split(marker, 1)
|
||||
user_data = head + "\n".join(lines) + "\n" + marker + tail
|
||||
else:
|
||||
user_data = user_data + "\n".join(lines) + "\n"
|
||||
with open(user_path, "w") as fh:
|
||||
fh.write(user_data)
|
||||
os.chmod(user_path, 0o600)
|
||||
return seed_dir
|
||||
|
||||
|
||||
def generate_host_keypair(state_dir: str, vm_name: str,
|
||||
runner: Callable[[list[str]], object] | None = None,
|
||||
keygen: str = "ssh-keygen") -> dict:
|
||||
"""为该 VM 生成(或复用本 VM 已有的)host key 对。
|
||||
|
||||
同一 VM 重启/恢复时复用已有对(否则钉死的 known_hosts
|
||||
立即失效);不同 VM 的 state_dir/<vm> 不同,绝不共用。
|
||||
runner 不给时用 subprocess,测试可注入假 runner(假
|
||||
runner 负责把私钥/公钥文件写出来)。keygen 失败即
|
||||
RuntimeError,调用方不得回落 keyscan(provisioned
|
||||
模式)或必须显式标弱回落(keyscan 模式,见执行器)。"""
|
||||
paths = host_key_paths(state_dir, vm_name)
|
||||
if (os.path.exists(paths["private"])
|
||||
and os.path.exists(paths["public"])):
|
||||
return paths
|
||||
base = os.path.dirname(paths["private"])
|
||||
os.makedirs(base, exist_ok=True)
|
||||
cmd = [keygen, "-t", "ed25519", "-N", "",
|
||||
"-f", paths["private"]]
|
||||
if runner is not None:
|
||||
result = runner(cmd)
|
||||
rc = getattr(result, "returncode", 0)
|
||||
if rc not in (0, None):
|
||||
raise RuntimeError("ssh-keygen 失败:returncode=%s" % rc)
|
||||
else:
|
||||
import subprocess
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
raise RuntimeError("ssh-keygen 失败:%s" % result.stderr)
|
||||
if not (os.path.exists(paths["private"])
|
||||
and os.path.exists(paths["public"])):
|
||||
raise RuntimeError("ssh-keygen 未生成 host key 对")
|
||||
try:
|
||||
os.chmod(paths["private"], 0o600)
|
||||
except OSError as exc:
|
||||
# feat-chv-secfix(H5):私钥 chmod 失败不再静默
|
||||
# pass——权限不对的私钥文件绝不能留着继续用。
|
||||
raise RuntimeError("host 私钥 chmod 0600 失败:%s" % exc)
|
||||
return paths
|
||||
@@ -0,0 +1,107 @@
|
||||
"""CHV state_dir 跨进程锁(feat-chv-p1fix,P1-05)。
|
||||
|
||||
问题:容量 / net / 磁盘的 check→act 原来只靠进程内
|
||||
threading.Lock 护 `_vms` dict;多 worker 进程共用同一
|
||||
state_dir 时,两路 wake 可双双通过容量/磁盘/net 门并
|
||||
双双建 VM(check-then-act 竞态),`_purge_dead_registry`
|
||||
的无锁 pop 与持锁遍历并发还可致字典迭代异常。
|
||||
|
||||
解法:同一把锁护住 check→act 全程——文件锁(flock)
|
||||
+ 进程内可重入锁的组合:
|
||||
|
||||
- 文件锁:state_dir/chv-state.lock 上的 `LOCK_EX`。
|
||||
不同进程(多 worker)靠它互斥;同一进程内不同
|
||||
executor 实例(各有自己的 StateLock 对象)同样靠它
|
||||
互斥,因为 flock 按打开的文件描述符区分,两个实例
|
||||
各自 open → 第二个会阻塞,直到第一个释放。
|
||||
- 进程内可重入:同一 executor 实例内,wake 全程会
|
||||
嵌套调用同样需要锁的方法(如 wake → 失败时 stop →
|
||||
cleanup_stale_vms 等路径),因此 StateLock 自带
|
||||
threading.RLock + 深度计数:同一线程重入时不再重复
|
||||
flock,只在最外层加锁/释放,避免同一线程把自己锁死。
|
||||
注意:同一线程经“不同 StateLock 实例”嵌套加锁不在
|
||||
保证内——代码里不存在这条路径(wake/restore/cleanup/
|
||||
reconcile 各自只用本实例或本 registry 的锁,且
|
||||
reconcile 不在 wake 持锁期间被调用)。
|
||||
|
||||
持有范围(见 executor.py):wake_vm 全程(prechecks →
|
||||
prepare → seed → boot 含 SSH 就绪等待)、restore_vm 全程、
|
||||
cleanup_stale_vms 全程(含 gc/密钥扫描/socket 扫描/purge)、
|
||||
orphans.reconcile 全程。锁内串行化 wake 是有意的代价:
|
||||
容量门只有“查完立即占位、占位前不放行”才成立;把锁
|
||||
提前释放到 vm.create 之后会在 socket 尚未绑定前留出
|
||||
计数窗口,仍可超限。串行 wake 不影响已运行 VM 的任务
|
||||
执行(_run_guest 不持此锁)。
|
||||
|
||||
本模块只依赖标准库,是叶子模块;fcntl 只在 POSIX 上
|
||||
可用——本执行器只面向 Linux 宿主(KVM 路径本身即
|
||||
Linux 专属),非 POSIX 平台 import 即失败,不静默回落
|
||||
纯线程锁(回落会把多 worker 保证悄悄降格)。
|
||||
"""
|
||||
|
||||
import fcntl
|
||||
import os
|
||||
import threading
|
||||
|
||||
LOCK_FILENAME = "chv-state.lock"
|
||||
|
||||
|
||||
def lock_path_for_state_dir(state_dir: str) -> str:
|
||||
"""state_dir 对应的锁文件路径。"""
|
||||
return os.path.join(state_dir, LOCK_FILENAME)
|
||||
|
||||
|
||||
class StateLock:
|
||||
"""state_dir 跨进程 + 进程内可重入锁(上下文管理器)。
|
||||
|
||||
用法:`with executor_state_lock:`。acquire 时先拿进程内
|
||||
RLock,再在深度为 0 时 open 锁文件并 flock LOCK_EX
|
||||
(阻塞,直到其它进程/实例释放);release 时深度归零才
|
||||
解锁并关闭文件描述符。锁文件本身永不删除(删文件会
|
||||
破坏 flock 的互斥语义:两个进程可能各持一个已删文件
|
||||
的锁而互不感知)。
|
||||
"""
|
||||
|
||||
def __init__(self, state_dir: str) -> None:
|
||||
self.state_dir = state_dir
|
||||
self.lock_path = lock_path_for_state_dir(state_dir)
|
||||
self._rlock = threading.RLock()
|
||||
self._depth = 0
|
||||
self._fd: int | None = None
|
||||
|
||||
def acquire(self) -> None:
|
||||
self._rlock.acquire()
|
||||
try:
|
||||
if self._depth == 0:
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
fd = os.open(self.lock_path,
|
||||
os.O_RDWR | os.O_CREAT, 0o600)
|
||||
try:
|
||||
fcntl.flock(fd, fcntl.LOCK_EX)
|
||||
except Exception:
|
||||
os.close(fd)
|
||||
raise
|
||||
self._fd = fd
|
||||
self._depth += 1
|
||||
except Exception:
|
||||
self._rlock.release()
|
||||
raise
|
||||
|
||||
def release(self) -> None:
|
||||
try:
|
||||
self._depth -= 1
|
||||
if self._depth == 0 and self._fd is not None:
|
||||
try:
|
||||
fcntl.flock(self._fd, fcntl.LOCK_UN)
|
||||
finally:
|
||||
os.close(self._fd)
|
||||
self._fd = None
|
||||
finally:
|
||||
self._rlock.release()
|
||||
|
||||
def __enter__(self) -> "StateLock":
|
||||
self.acquire()
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc_info: object) -> None:
|
||||
self.release()
|
||||
@@ -0,0 +1,227 @@
|
||||
"""CHV 上一世残留 VM 清理(生产路径唯一入口)。
|
||||
|
||||
本模块是叶子模块,只依赖标准库,绝不 import
|
||||
cloud.chv.experimental_pool(实验接口)与 cloud.sandbox。
|
||||
生产代码(worker 启动/关闭)只调
|
||||
reconcile_orphans_for_executor(),它直接读 registry 文件、
|
||||
对每个上一世条目调 executor.shutdown_vm(),不构造任何
|
||||
池对象、不调 prewarm/acquire/release。
|
||||
|
||||
Registry 语义与旧 WarmPool._load_registry 完全一致
|
||||
(feat-chv-arch95plus 的 fail-safe 口径,本轨只搬位置、
|
||||
不改语义):
|
||||
- 文件:state_dir/warm-pool.json,内容为
|
||||
{<合成名>: {"state", "vm", "task_id", "snapshot_url"}}。
|
||||
- 读到的 warm/assigned 条目一律当 orphan:旧 VMM 进程
|
||||
已随旧 worker 消失,无法验证它还是当初那个干净快照
|
||||
VM,绝不发给任务,只能清理。
|
||||
- 损坏 fail-safe:文件不是合法 JSON / 顶层不是 dict 时,
|
||||
整个文件 quarantine 成 <path>.corrupt,清理从空开始;
|
||||
损坏本身发审计(feat-chv-p1fix,P1-02),绝不静默
|
||||
返回空——损坏文件的孤儿 VM 此前只能靠 socket 扫描。
|
||||
单条目结构非法(无 vm、或 state 不在 warm/assigned)
|
||||
时只丢该条,不连坐其它条目;非法条目发审计并计数
|
||||
(feat-chv-p1fix,P1-02),落盘时不再写回。
|
||||
- 清理后落盘:全部成功时为空 dict;有失败项时只写回
|
||||
失败项的原始条目(原子 tmp + rename),下次 reconcile
|
||||
可重试,失败项绝不被“已清理”掩盖(feat-chv-p1fix,
|
||||
P1-01)。写失败只记日志警告,不炸 worker 启动/关闭。
|
||||
shutdown_vm 抛错的条目不进 cleaned、不触发“全清”,
|
||||
而是记 failed 列表并经 executor._emit 发审计;
|
||||
_emit 失败同样 best-effort。整个 reconcile 在
|
||||
state_dir 的 flock 下跑(见 locking.py),与 wake 互斥。
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
|
||||
_LOGGER = logging.getLogger("cloud.cloud_hypervisor")
|
||||
|
||||
_REGISTRY_FILENAME = "warm-pool.json"
|
||||
|
||||
|
||||
def registry_path_for_executor(executor: object) -> str | None:
|
||||
"""执行器的 registry 路径:state_dir 下的 warm-pool.json。
|
||||
|
||||
无 state_dir 时返回 None(调用方按“无残留”处理)。"""
|
||||
state_dir = getattr(executor, "state_dir", None)
|
||||
if not state_dir:
|
||||
return None
|
||||
return os.path.join(state_dir, _REGISTRY_FILENAME)
|
||||
|
||||
|
||||
def _quarantine_registry(path: str) -> None:
|
||||
"""把损坏的 registry 文件挪到 <path>.corrupt 旁(尽力而为)。"""
|
||||
try:
|
||||
os.replace(path, path + ".corrupt")
|
||||
except OSError:
|
||||
_LOGGER.warning("chv_pool_registry_quarantine_failed "
|
||||
"path=%s", path)
|
||||
|
||||
|
||||
def _read_registry(path: str) -> tuple[dict | None, bool]:
|
||||
"""读 registry 原始 dict:(data, corrupt)。
|
||||
|
||||
无文件 → (None, False);损坏 → quarantine 后 (None, True);
|
||||
顶层非 dict 同样算损坏。调用方按 corrupt 决定是否记
|
||||
registry_corrupt(实验池需要区分“无文件”与“损坏”)。"""
|
||||
try:
|
||||
with open(path, "r") as fh:
|
||||
data = json.load(fh)
|
||||
except FileNotFoundError:
|
||||
return None, False
|
||||
except (OSError, ValueError):
|
||||
_quarantine_registry(path)
|
||||
return None, True
|
||||
if not isinstance(data, dict):
|
||||
_quarantine_registry(path)
|
||||
return None, True
|
||||
return data, False
|
||||
|
||||
|
||||
def _write_registry(path: str, data: dict) -> None:
|
||||
"""原子落盘 registry(tmp + rename);写失败只记日志。"""
|
||||
try:
|
||||
directory = os.path.dirname(path)
|
||||
if directory:
|
||||
os.makedirs(directory, exist_ok=True)
|
||||
tmp_path = path + ".tmp"
|
||||
with open(tmp_path, "w") as fh:
|
||||
json.dump(data, fh)
|
||||
os.replace(tmp_path, path)
|
||||
except OSError:
|
||||
_LOGGER.warning("chv_pool_registry_save_failed path=%s", path)
|
||||
|
||||
|
||||
def _orphan_entries(data: dict) -> list[dict]:
|
||||
"""从原始 registry dict 里筛出可清理的上一世条目。
|
||||
|
||||
只留 state 为 warm/assigned 且有 vm 的条目;非法条目
|
||||
不连坐其它条目(计数与审计见 reconcile 本体,不再
|
||||
静默丢弃)。返回条目为 {"vm", "task_id",
|
||||
"snapshot_url"} 的新 dict,不改原数据。"""
|
||||
entries: list[dict] = []
|
||||
for _key, entry in _orphan_entries_with_keys(data)[0]:
|
||||
entries.append(entry)
|
||||
return entries
|
||||
|
||||
|
||||
def _orphan_entries_with_keys(data: dict) -> tuple[list, list]:
|
||||
"""筛条目并保留原始 registry 键:返回(可清理条目,
|
||||
非法条目键)。
|
||||
|
||||
可清理条目为 (原始键, {"vm", "task_id",
|
||||
"snapshot_url"});非法条目只记键(键本身不含密钥,
|
||||
可进审计 detail),原始值不进审计。"""
|
||||
entries: list = []
|
||||
invalid_keys: list = []
|
||||
for key, entry in data.items():
|
||||
if not isinstance(entry, dict):
|
||||
invalid_keys.append(key)
|
||||
continue
|
||||
if not entry.get("vm"):
|
||||
invalid_keys.append(key)
|
||||
continue
|
||||
if entry.get("state") not in ("warm", "assigned"):
|
||||
invalid_keys.append(key)
|
||||
continue
|
||||
entries.append((key, {
|
||||
"vm": entry["vm"],
|
||||
"task_id": entry.get("task_id"),
|
||||
"snapshot_url": entry.get("snapshot_url"),
|
||||
}))
|
||||
return entries, invalid_keys
|
||||
|
||||
|
||||
def _best_effort_emit(executor: object, event: str, detail: str) -> None:
|
||||
"""经 executor._emit 发审计(best-effort,兼容只接受
|
||||
(event, task, detail) 三参的假执行器;无 _emit 时跳过)。"""
|
||||
emit = getattr(executor, "_emit", None)
|
||||
if not callable(emit):
|
||||
return
|
||||
try:
|
||||
emit(event, {"id": None}, detail)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def reconcile_orphans_for_executor(executor: object) -> list[str]:
|
||||
"""生产路径唯一的残留清理入口:返回已清理的 VM 名列表。
|
||||
|
||||
只做清理:读 registry → 每个上一世条目尽力
|
||||
shutdown_vm → 落盘 → best-effort 审计事件。
|
||||
绝不构造 WarmPool、绝不调 prewarm/acquire/release,
|
||||
绝不把任何条目发给任务。无 registry 路径时返回 []。
|
||||
|
||||
feat-chv-p1fix(P1-01/P1-02/P1-05)语义:
|
||||
- shutdown 失败的条目不进 cleaned,记 failed 并审计,
|
||||
落盘时只写回失败项原始条目(可重试),绝不清库;
|
||||
- registry 损坏时 quarantine 后仍发审计,不静默返回空;
|
||||
- 非法条目发审计并计数,落盘时丢弃(已审计,不再
|
||||
每次重启重复审计同一批非法条目);
|
||||
- 全程持 state_dir flock,与 wake/restore/cleanup 互斥。
|
||||
返回值仍只含 cleaned(调用方兼容不变);失败与
|
||||
损坏/非法的可见性走 executor._emit(worker 接线后
|
||||
即任务 timeline 审计),不靠返回值传递。
|
||||
"""
|
||||
path = registry_path_for_executor(executor)
|
||||
if not path:
|
||||
return []
|
||||
if not os.path.exists(path):
|
||||
return []
|
||||
state_dir = getattr(executor, "state_dir", None)
|
||||
if state_dir:
|
||||
from cloud.chv.locking import StateLock
|
||||
lock = StateLock(state_dir)
|
||||
else: # pragma: no cover - state_dir 缺失时 path 已为 None
|
||||
from contextlib import nullcontext
|
||||
lock = nullcontext() # type: ignore[assignment]
|
||||
with lock:
|
||||
return _reconcile_locked(executor, path)
|
||||
|
||||
|
||||
def _reconcile_locked(executor: object, path: str) -> list[str]:
|
||||
"""持锁的 reconcile 本体(见 reconcile_orphans_for_executor)。"""
|
||||
data, corrupt = _read_registry(path)
|
||||
if data is None:
|
||||
if corrupt:
|
||||
_best_effort_emit(
|
||||
executor, "warm_pool_registry_corrupt",
|
||||
"path=%s registry corrupt, quarantined" % path)
|
||||
return []
|
||||
entries, invalid_keys = _orphan_entries_with_keys(data)
|
||||
if invalid_keys:
|
||||
_best_effort_emit(
|
||||
executor, "warm_pool_registry_invalid_entries",
|
||||
"invalid=%d keys=%s"
|
||||
% (len(invalid_keys), ",".join(str(k) for k in invalid_keys)))
|
||||
cleaned: list[str] = []
|
||||
failed: list[str] = []
|
||||
failed_errors: list[str] = []
|
||||
failed_keys: list[str] = []
|
||||
for key, entry in entries:
|
||||
vm = entry["vm"]
|
||||
try:
|
||||
executor.shutdown_vm(vm) # type: ignore[attr-defined]
|
||||
except Exception as exc:
|
||||
failed.append(vm)
|
||||
failed_keys.append(key)
|
||||
failed_errors.append("%s:%s" % (vm, type(exc).__name__))
|
||||
continue
|
||||
cleaned.append(vm)
|
||||
if cleaned or failed or invalid_keys:
|
||||
remaining = {key: data[key] for key in failed_keys
|
||||
if key in data}
|
||||
_write_registry(path, remaining)
|
||||
if cleaned:
|
||||
_best_effort_emit(
|
||||
executor, "warm_pool_reconcile",
|
||||
"cleaned=%s" % ",".join(cleaned))
|
||||
if failed:
|
||||
_best_effort_emit(
|
||||
executor, "warm_pool_reconcile_failed",
|
||||
"failed=%s cleaned=%s errors=%s"
|
||||
% (",".join(failed), ",".join(cleaned),
|
||||
";".join(failed_errors)))
|
||||
return cleaned
|
||||
@@ -0,0 +1,180 @@
|
||||
"""Cloud Hypervisor 客机 SSH 传输(叶子模块,不依赖
|
||||
cloud.sandbox,避免循环导入)。
|
||||
|
||||
安全模型(本轨的核心变化,替代 #216 的
|
||||
StrictHostKeyChecking=no + UserKnownHostsFile=/dev/null):
|
||||
|
||||
- 每 VM 一个独立 known_hosts 文件:
|
||||
state_dir/<vm 名>/known_hosts(与每任务磁盘同目录,
|
||||
任务销毁时不自动删——known_hosts 留在宿主 state_dir 里,
|
||||
供事后审计该 VM 当时钉死的 key;VM 名本身已含任务 id,
|
||||
不会串到别的 VM 上)。
|
||||
- 首次信任(TOFU,边界必须写清,不假装是强验证):
|
||||
首次启动时客机 host key 未知,执行器用 ssh-keyscan 经
|
||||
宿主转发端口(127.0.0.1:<port>,只走本机回环)抓一次
|
||||
当时呈现的 key,写入该 VM 的 known_hosts 文件;此后
|
||||
同一 VM 的每次 SSH 都用 StrictHostKeyChecking=yes +
|
||||
该文件校验,VM 被替换 / 中间人换 key 时 SSH 直接拒绝,
|
||||
不再静默接受。首次捕获本身不验证“这台客机就是我们要
|
||||
的那台”——它只防首次之后的替换;更强的方案是镜像构建
|
||||
/cloud-init 时把客机 host key 预置进 known_hosts(未来
|
||||
工作,见 docs/chv-production-readiness.md 的待办),本轨
|
||||
在 mock 测试里只能验证“捕获→写文件→后续命令带钉死
|
||||
选项”的管线,绝不声称 mock 验证了首次密钥的真实性。
|
||||
- 密钥红线不变且收紧(feat-chv-secfix,H2/M1):argv 里只有
|
||||
key 文件路径,绝无 key 值;网关密钥类变量(见
|
||||
_CHV_SECRET_ENV_NAMES,与 Docker 的 _DOCKER_SECRET_ENV_NAMES
|
||||
同款“只传名”口径,但 CHV 更进一步:值连宿主子进程环境
|
||||
都不进,改走 seed 盘里的客内文件,见 hostkeys.write_seed)
|
||||
出现在 extra_env 里时本函数直接 ValueError——传输层
|
||||
fail-closed,任何调用方都不能再把密钥值拼回 argv。
|
||||
非密钥 extra_env 仍可走 env 前缀(值在 argv 里可见,
|
||||
但按定义不是密钥)。
|
||||
- 远程 shell 转义(feat-chv-secfix,M1):SSH 协议的 exec
|
||||
请求本身是一条字符串,ssh 客户端把多个参数用空格拼成
|
||||
远程 shell 命令——此前 inner 参数原样进 argv,prompt 里
|
||||
的 ;/$()/引号/换行会在远程 shell 里被二次解析。现在每个
|
||||
远程 token 都经 shlex.quote 引用后仍以列表元素形式返回
|
||||
(ssh 拼接后远程解析结果与本地 list 语义一致);客内
|
||||
agent 本来就是 root,本项修的是“任务执行语义不被 prompt
|
||||
改写 + 审计不失真”,不是“防客内提权”(客内 root 无
|
||||
需注入即可为所欲为,边界见执行器文档)。
|
||||
"""
|
||||
|
||||
import os
|
||||
import shlex
|
||||
|
||||
# CHV 密钥类环境变量名:值绝不进 ssh argv。Docker 线是
|
||||
# “-e 只传名、值由 CLI 子进程环境提供”(值仍在宿主进程
|
||||
# 环境里,/proc/<pid>/environ 对同用户可见);CHV 线是
|
||||
# “值只进 seed 盘客内文件”(宿主进程表与宿主进程环境里
|
||||
# 都没有值),比 Docker 线更严——差异见执行器文档。
|
||||
_CHV_SECRET_ENV_NAMES = frozenset({"SUB2API_API_KEY"})
|
||||
|
||||
|
||||
def split_secret_env(extra_env: dict | None) -> tuple[dict, dict]:
|
||||
"""把 extra_env 拆成(密钥子集,非密钥子集)。"""
|
||||
secrets, plain = {}, {}
|
||||
for key, value in dict(extra_env or {}).items():
|
||||
if key in _CHV_SECRET_ENV_NAMES:
|
||||
secrets[key] = value
|
||||
else:
|
||||
plain[key] = value
|
||||
return secrets, plain
|
||||
|
||||
|
||||
def remote_command(inner_cmd: list, egress_proxy: str | None = None,
|
||||
extra_env: dict | None = None,
|
||||
guest_env_file: str | None = None) -> list:
|
||||
"""构造远程 shell 命令的 token 列表(不含 ssh 前缀)。
|
||||
|
||||
每个 token 都经 shlex.quote:ssh 客户端把 argv 用空格拼
|
||||
成一条远程命令时,远程 shell 解析回来的恰好是本地 list
|
||||
语义。guest_env_file 给定时,远程命令先 `set -a; . <文件>;
|
||||
set +a`(把 seed 投递的密钥文件 source 进环境并 export),
|
||||
再接 env 前缀与客内命令;文件路径本身不是密钥。
|
||||
extra_env 含密钥名时 ValueError(调用方须先 split 出
|
||||
密钥子集走 seed 投递,见 split_secret_env)。"""
|
||||
if extra_env:
|
||||
secrets, _ = split_secret_env(extra_env)
|
||||
if secrets:
|
||||
raise ValueError(
|
||||
"密钥类环境变量 %s 不得进 ssh argv,须走 seed "
|
||||
"文件投递" % ",".join(sorted(secrets)))
|
||||
tokens = []
|
||||
if guest_env_file:
|
||||
tokens += ["set", "-a;", ".",
|
||||
shlex.quote(guest_env_file) + ";", "set", "+a;"]
|
||||
env_assignments = []
|
||||
if extra_env:
|
||||
for key in sorted(extra_env):
|
||||
env_assignments.append("%s=%s"
|
||||
% (key, shlex.quote(
|
||||
str(extra_env[key]))))
|
||||
if egress_proxy:
|
||||
env_assignments += [
|
||||
"HTTPS_PROXY=%s" % shlex.quote(egress_proxy),
|
||||
"HTTP_PROXY=%s" % shlex.quote(egress_proxy),
|
||||
"NO_PROXY=localhost,127.0.0.1"]
|
||||
inner = [shlex.quote(str(arg)) for arg in inner_cmd]
|
||||
if env_assignments:
|
||||
tokens += ["env"] + env_assignments + inner
|
||||
else:
|
||||
tokens += inner
|
||||
return tokens
|
||||
|
||||
|
||||
def known_hosts_path(state_dir: str, vm_name: str) -> str:
|
||||
"""该 VM 的 known_hosts 文件路径:state_dir/<vm>/known_hosts。"""
|
||||
return os.path.join(state_dir, vm_name, "known_hosts")
|
||||
|
||||
|
||||
def build_ssh_command(ssh_port: int, ssh_key: str | None,
|
||||
ssh_user: str, egress_proxy: str | None,
|
||||
known_hosts_file: str, inner_cmd: list,
|
||||
ssh_host: str = "127.0.0.1",
|
||||
extra_env: dict | None = None,
|
||||
guest_env_file: str | None = None) -> list:
|
||||
"""构造 SSH 命令:ssh -p <端口> -i <key 文件> + 钉死选项
|
||||
+ user@<ssh_host> + 客内命令(远程 token 均经 shlex.quote,
|
||||
见 remote_command)。
|
||||
|
||||
feat-chv-fix(P0-4):ssh_host 不再在传输层写死回环,
|
||||
由执行器解析传入(显式 CHV_SSH_HOST / net 客机 IP /
|
||||
显式转发宣称下的回环);本函数缺省回环仅为旧调用点
|
||||
兼容,执行器路径绝不依赖该缺省。
|
||||
feat-chv-secfix(H2):extra_env 只接受非密钥变量;
|
||||
密钥类变量须先经 split_secret_env 拆出、走 seed 盘客内
|
||||
文件投递,调用方把 guest_env_file 传进来(客内 seed
|
||||
写出的 /etc/chv/guest-env),本函数在远程命令前 source
|
||||
它;密钥值绝不出现在本函数返回的任何 argv 元素里。
|
||||
egress_proxy 已设时客内命令前加 env 前缀(代理地址本身
|
||||
不是密钥;客机出口只走该白名单代理——但这只是 env 约
|
||||
定,网络层强制见宿主防火墙部署清单,代码不声称已隔离)。"""
|
||||
remote = remote_command(inner_cmd, egress_proxy=egress_proxy,
|
||||
extra_env=extra_env,
|
||||
guest_env_file=guest_env_file)
|
||||
return ["ssh", "-p", str(ssh_port),
|
||||
"-i", ssh_key,
|
||||
"-o", "StrictHostKeyChecking=yes",
|
||||
"-o", "UserKnownHostsFile=%s" % known_hosts_file,
|
||||
"-o", "BatchMode=yes",
|
||||
"%s@%s" % (ssh_user, ssh_host)] + remote
|
||||
|
||||
|
||||
def parse_keyscan_output(text: str, host: str = "127.0.0.1") -> list:
|
||||
"""从 ssh-keyscan 的 stdout 里挑出目标主机的 host key
|
||||
行:形如 "[<host>]:<port> ssh-ed25519 AAAA..."(-p 时
|
||||
的标准写法)或 "<host> ssh-rsa AAAA..."。其它主机行、
|
||||
注释行、空行一律丢掉,绝不把陌生主机的 key 写进文件。
|
||||
host 缺省回环仅为旧调用点兼容;客机 IP 方案下应传
|
||||
解析出的客机 IP。"""
|
||||
kept = []
|
||||
for line in (text or "").splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) < 3:
|
||||
continue
|
||||
line_host, keytype = parts[0], parts[1]
|
||||
if line_host != host \
|
||||
and not line_host.startswith("[%s]:" % host):
|
||||
continue
|
||||
if not keytype.startswith("ssh-"):
|
||||
continue
|
||||
kept.append(line)
|
||||
return kept
|
||||
|
||||
|
||||
def write_known_hosts(path: str, key_lines: list) -> None:
|
||||
"""原子写 known_hosts:先写同目录临时文件再 rename,
|
||||
文件权限 0600(只含公钥行,仍按敏感文件对待)。"""
|
||||
directory = os.path.dirname(path)
|
||||
if directory:
|
||||
os.makedirs(directory, exist_ok=True)
|
||||
tmp_path = path + ".tmp"
|
||||
with open(tmp_path, "w") as fh:
|
||||
fh.write("\n".join(key_lines) + "\n")
|
||||
os.chmod(tmp_path, 0o600)
|
||||
os.replace(tmp_path, path)
|
||||
@@ -0,0 +1,196 @@
|
||||
"""CHV worker 适配模块(to10 轨,见 ADR-13)。
|
||||
|
||||
worker.py 里散落的 CHV 强相关接线抽到这里,worker 只留
|
||||
薄包装(方法名、签名、调用点不变,旧测试继续走 worker
|
||||
方法)。本模块只依赖标准库 + cloud.chv 叶子模块 +
|
||||
cloud.logging_config,不 import cloud.worker,绝无循环。
|
||||
|
||||
抽入本模块的(行为与错误文案逐字搬运):
|
||||
- call_goal_resume_breakpoint:通用断点调用(原 worker
|
||||
顶层函数 _call_goal_resume_breakpoint 的本体)。
|
||||
- make_chv_audit_callback:VM 生命周期事件进审计的回调
|
||||
(原 Worker.__init__ 里的闭包本体)。
|
||||
- reconcile_warm_pool:生产残留清理 facade 调用 + 审计
|
||||
(原 Worker._reconcile_warm_pool 的本体)。
|
||||
- cancel_watchdog_loop / quota_watchdog_loop:同步路径
|
||||
两条看门狗循环(原 Worker._chv_*_watchdog 的本体,
|
||||
self 依赖全部改成回调参数)。
|
||||
- watchdog_vm_name:看门狗启动时的 VM 名派生(原
|
||||
_run_attempt 里的 try/except 块)。
|
||||
|
||||
明确不抽的(见 ADR-13 诚实清单,worker 通用流程):
|
||||
cancel() 里的 cancel_vm 分支、cleanup_stale_containers
|
||||
的 CHV 分支、网关守卫、_executor_name 的 CHV 分支、
|
||||
_run_attempt 的看门狗线程启动、VM 死亡写回——它们都
|
||||
嵌在 Docker/Local 共用的通用流程里,硬抽只会把通用
|
||||
流程撕成两半,行为不变的收益为零。
|
||||
"""
|
||||
|
||||
import inspect
|
||||
import threading
|
||||
from collections.abc import Callable
|
||||
|
||||
from cloud.chv.executor import CloudHypervisorExecutor
|
||||
from cloud.logging_config import log_event
|
||||
|
||||
|
||||
def is_chv_executor(executor: object) -> bool:
|
||||
"""是否为 CHV 执行器(worker 各处 isinstance 的同款判断)。"""
|
||||
return isinstance(executor, CloudHypervisorExecutor)
|
||||
|
||||
|
||||
def call_goal_resume_breakpoint(executor: object, task: dict,
|
||||
model_provider: str | None = None,
|
||||
extra_env: dict | None = None) -> object:
|
||||
"""通用断点调用:执行器提供 goal_resume_breakpoint_task
|
||||
时调用它。
|
||||
|
||||
Local 仍走原路径(调用方直接传 model_provider,不经过
|
||||
这里,以免改生产语义);这里只用于非 Local 执行器:
|
||||
签名接受 model_provider 时传它(CHV/Docker 均接受),
|
||||
否则只传 task(旧假执行器单参)。feat-chv-secfix(H2):
|
||||
CHV 的 gateway_env 不再经 SSH env 前缀注入(密钥值进
|
||||
argv 的旧路径已废除),改走 provisioned seed 盘客内文件
|
||||
投递;CHV 没有 Local 那样的进程 env 通道,seed 文件是
|
||||
唯一通道,keyscan 模式下密钥投递 fail-closed。"""
|
||||
method = getattr(executor, "goal_resume_breakpoint_task")
|
||||
try:
|
||||
params = inspect.signature(method).parameters
|
||||
except (TypeError, ValueError):
|
||||
return method(task)
|
||||
kwargs: dict = {}
|
||||
if "model_provider" in params:
|
||||
kwargs["model_provider"] = model_provider
|
||||
if "extra_env" in params and extra_env is not None:
|
||||
kwargs["extra_env"] = extra_env
|
||||
return method(task, **kwargs)
|
||||
|
||||
|
||||
def make_chv_audit_callback(audit_fn: Callable) -> Callable:
|
||||
"""VM 生命周期事件进审计的回调(原 __init__ 闭包本体)。
|
||||
|
||||
audit_fn 即 Worker._audit(best-effort,失败不影响任务);
|
||||
时间线展示 VM 事件时读审计表即可,不另起一套 VM 事件表;
|
||||
成本/配额仍走 worker 现有 quota 语义,本回调绝不记账。"""
|
||||
|
||||
def _chv_audit(event, task, detail, kind=None):
|
||||
try:
|
||||
project_id = task.get("project_id") \
|
||||
if isinstance(task, dict) else None
|
||||
audit_fn(project_id, "chv_vm_" + event, detail or "")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return _chv_audit
|
||||
|
||||
|
||||
def reconcile_warm_pool(executor: object, audit_fn: Callable,
|
||||
when: str) -> list:
|
||||
"""显式调生产残留清理 facade,返回已清理 VM 名。
|
||||
|
||||
仅 CHV worker 有效(其它执行器返回 [])。实现是
|
||||
cloud.chv.orphans.reconcile_orphans_for_executor():
|
||||
直接读 registry 文件做清理,不构造 WarmPool、不碰
|
||||
实验接口。调用失败不炸主流程,但进审计(失败本身
|
||||
也审计,不静默丢弃)。orphan 条目绝不发给任务,
|
||||
reconcile 只做清理。"""
|
||||
if not isinstance(executor, CloudHypervisorExecutor):
|
||||
return []
|
||||
try:
|
||||
from cloud.chv.orphans import reconcile_orphans_for_executor as _reconcile
|
||||
cleaned = _reconcile(executor) or []
|
||||
except Exception as exc:
|
||||
try:
|
||||
audit_fn(None, "chv_warm_pool_reconcile_failed",
|
||||
"when=%s error=%s" % (when, exc))
|
||||
except Exception:
|
||||
pass
|
||||
return []
|
||||
if cleaned:
|
||||
try:
|
||||
audit_fn(None, "chv_warm_pool_reconcile",
|
||||
"when=%s cleaned=%s" % (when, ",".join(cleaned)))
|
||||
except Exception:
|
||||
pass
|
||||
return cleaned
|
||||
|
||||
|
||||
def cancel_watchdog_loop(task_id: str, vm_name: str | None,
|
||||
stop_event: threading.Event,
|
||||
cancel_requested: set,
|
||||
get_task: Callable,
|
||||
cancel_vm: Callable | None,
|
||||
logger: object) -> None:
|
||||
"""CHV 同步路径的取消看门狗(feat-chv-fix,P1-2,Docker
|
||||
取消看门狗同款模式):同步 SSH 阻塞时 worker 的 cancel()
|
||||
只能改库、清 _running 项,阻塞中的 run 本身不会醒;
|
||||
本线程每 1 秒查库/取消标志,命中时调 cancel_vm 真关
|
||||
VM,然后退出。查库失败时当没取消,不误杀。"""
|
||||
while not stop_event.wait(1.0):
|
||||
try:
|
||||
if task_id in cancel_requested:
|
||||
cancelled = True
|
||||
else:
|
||||
db_task = get_task(task_id)
|
||||
cancelled = (db_task is not None and
|
||||
db_task.get("status") == "interrupted")
|
||||
except Exception:
|
||||
cancelled = False
|
||||
if not cancelled:
|
||||
continue
|
||||
try:
|
||||
if cancel_vm is not None and vm_name:
|
||||
cancel_vm(vm_name)
|
||||
log_event(logger, "worker_chv_killed",
|
||||
task_id=task_id,
|
||||
message="cancel watchdog killed VM %s "
|
||||
"(task interrupted/cancelled)"
|
||||
% vm_name)
|
||||
except Exception:
|
||||
pass
|
||||
return
|
||||
|
||||
|
||||
def quota_watchdog_loop(task_id: str, vm_name: str | None,
|
||||
stop_event: threading.Event,
|
||||
quota_check: Callable | None,
|
||||
mark_quota_hit: Callable,
|
||||
cancel_vm: Callable | None,
|
||||
logger: object,
|
||||
interval: float = 30.0) -> None:
|
||||
"""CHV 同步路径的配额看门狗(feat-chv-fix,P1-2,Docker
|
||||
配额看门狗同款):命中时记 _quota_hit_tasks 并调
|
||||
cancel_vm 真关 VM;_run_attempt 在同步返回后把结果
|
||||
改写成 quota_exceeded,走统一配额中断收尾。"""
|
||||
while not stop_event.wait(interval):
|
||||
try:
|
||||
over = bool(quota_check()) if quota_check else False
|
||||
except Exception:
|
||||
over = False
|
||||
if not over:
|
||||
continue
|
||||
mark_quota_hit(task_id)
|
||||
try:
|
||||
if cancel_vm is not None and vm_name:
|
||||
cancel_vm(vm_name)
|
||||
log_event(logger, "worker_chv_killed",
|
||||
task_id=task_id,
|
||||
message="quota watchdog killed VM %s "
|
||||
"(quota exceeded)" % vm_name)
|
||||
except Exception:
|
||||
pass
|
||||
return
|
||||
|
||||
|
||||
def watchdog_vm_name(executor: object, task: dict) -> str | None:
|
||||
"""看门狗启动时的 VM 名派生(原 _run_attempt 块):
|
||||
executor.vm_name(task) 抛错时回 None(不建看门狗),
|
||||
绝不炸主派发流程。executor 无 vm_name 时等同抛错
|
||||
(原代码里 AttributeError 同样被 except 吞掉回 None)。"""
|
||||
get_vm_name = getattr(executor, "vm_name", None)
|
||||
if not callable(get_vm_name):
|
||||
return None
|
||||
try:
|
||||
return get_vm_name(task)
|
||||
except Exception:
|
||||
return None
|
||||
@@ -31,20 +31,20 @@ npm run build:console
|
||||
`npm run build:console` = `tsc -b && vite build`(vite-plugin-singlefile
|
||||
把 JS/CSS 全部内联成一个 `dist/index.html`),再由
|
||||
`scripts/copy-build.mjs` **原样**拷到
|
||||
`../console-redesign/console-react.html`(审阅用),并打印字节数与
|
||||
sha256。脚本不改动产物内容。
|
||||
`../console.html`(即 GET /console 现读的文件),并打印字节数与
|
||||
sha256、校验拷后内容与 dist 一致。脚本不改动产物内容。
|
||||
旧目标 `../console-redesign/console-react.html` 在仓库里不存在,
|
||||
已废弃,不再靠手工拷兜底。
|
||||
|
||||
## 自动部署链路(不留手工步骤)
|
||||
|
||||
```
|
||||
改 src → npm run build:console → 产物拷到 cloud/console.html → 提交 → 部署
|
||||
改 src → npm run build:console → 产物直接拷到 cloud/console.html → 提交 → 部署
|
||||
```
|
||||
|
||||
具体到仓库里的一步(由主 agent 在审阅后执行,本项目不部署生产):
|
||||
具体到仓库里的一步(本项目不部署生产):
|
||||
|
||||
```bash
|
||||
cp console-redesign/console-react.html <仓库>/cloud/console.html
|
||||
# 或在仓库分支里直接用构建产物覆盖 cloud/console.html
|
||||
git add cloud/console.html && git commit -m "console: React 版"
|
||||
# 随后走仓库已有的 PR → CI → 合并 → deploy.py git 模式部署
|
||||
```
|
||||
@@ -58,10 +58,10 @@ git add cloud/console.html && git commit -m "console: React 版"
|
||||
## 怎么验证
|
||||
|
||||
```bash
|
||||
npm run build:console # 必须 0 退出,tsc 无错误
|
||||
ls -l ../console-redesign/console-react.html
|
||||
sha256sum ../console-redesign/console-react.html
|
||||
grep -E 'src="/assets|href="/assets' ../console-redesign/console-react.html # 应无输出
|
||||
npm run build:console # 必须 0 退出,tsc 无错误,且拷后与 dist/index.html 内容一致
|
||||
ls -l ../console.html
|
||||
sha256sum ../console.html dist/index.html # 两者 sha 应相同
|
||||
grep -E 'src="/assets|href="/assets' ../console.html # 应无输出
|
||||
```
|
||||
|
||||
部署后另行核对(主 agent 执行):公网 `/console` 的字节数/sha 与
|
||||
|
||||
Generated
+252
-1
@@ -17,7 +17,8 @@
|
||||
"@vitejs/plugin-react": "^4.4.1",
|
||||
"typescript": "~5.8.3",
|
||||
"vite": "^6.3.5",
|
||||
"vite-plugin-singlefile": "^2.2.0"
|
||||
"vite-plugin-singlefile": "^2.2.0",
|
||||
"vitest": "^5.0.5"
|
||||
}
|
||||
},
|
||||
"node_modules/@babel/code-frame": {
|
||||
@@ -1213,6 +1214,24 @@
|
||||
"@babel/types": "^7.28.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/chai": {
|
||||
"version": "5.2.3",
|
||||
"resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz",
|
||||
"integrity": "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/deep-eql": "*",
|
||||
"assertion-error": "^2.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/deep-eql": {
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz",
|
||||
"integrity": "sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/estree": {
|
||||
"version": "1.0.9",
|
||||
"resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz",
|
||||
@@ -1261,6 +1280,26 @@
|
||||
"vite": "^4.2.0 || ^5.0.0 || ^6.0.0 || ^7.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@vitest/spy": {
|
||||
"version": "5.0.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-5.0.5.tgz",
|
||||
"integrity": "sha512-lwJRExuH417257Q5wv9QYugN9IMJkoxcrKrYxGPQkBzl7wlNpTUxD/5yNX4ESKVUIShKPgkGchYPFMWOOk0wzw==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/vitest"
|
||||
}
|
||||
},
|
||||
"node_modules/assertion-error": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz",
|
||||
"integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/baseline-browser-mapping": {
|
||||
"version": "2.11.34",
|
||||
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.11.34.tgz",
|
||||
@@ -1329,6 +1368,16 @@
|
||||
],
|
||||
"license": "CC-BY-4.0"
|
||||
},
|
||||
"node_modules/chai": {
|
||||
"version": "6.3.0",
|
||||
"resolved": "https://registry.npmjs.org/chai/-/chai-6.3.0.tgz",
|
||||
"integrity": "sha512-XWAtwJ6OHO+tj0EKCs0Y2UamnyOxseZWltU4x2U2wh8g4AigdjwvtUjvLP2tqkA/avxHEtzxNaqGq/YGNwckKg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/convert-source-map": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-2.0.0.tgz",
|
||||
@@ -1368,6 +1417,13 @@
|
||||
"dev": true,
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/es-module-lexer": {
|
||||
"version": "2.3.2",
|
||||
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.3.2.tgz",
|
||||
"integrity": "sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/esbuild": {
|
||||
"version": "0.25.12",
|
||||
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.12.tgz",
|
||||
@@ -1420,6 +1476,26 @@
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/estree-walker": {
|
||||
"version": "3.0.3",
|
||||
"resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz",
|
||||
"integrity": "sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/estree": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/expect-type": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.4.0.tgz",
|
||||
"integrity": "sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==",
|
||||
"dev": true,
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=12.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/fdir": {
|
||||
"version": "6.5.0",
|
||||
"resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz",
|
||||
@@ -1506,6 +1582,16 @@
|
||||
"yallist": "^3.0.2"
|
||||
}
|
||||
},
|
||||
"node_modules/magic-string": {
|
||||
"version": "1.4.5",
|
||||
"resolved": "https://registry.npmjs.org/magic-string/-/magic-string-1.4.5.tgz",
|
||||
"integrity": "sha512-oLLC2+YvQUfAa+vxoufE/yc1icb87M9iXIRqAoR/2EhE/8cb3SF/+Gi0c2By1B+5Vz2e2vQtUvHiu1+63Znh+w==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@jridgewell/sourcemap-codec": "^1.6.0"
|
||||
}
|
||||
},
|
||||
"node_modules/ms": {
|
||||
"version": "2.1.3",
|
||||
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
|
||||
@@ -1542,6 +1628,20 @@
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/obug": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/obug/-/obug-3.0.0.tgz",
|
||||
"integrity": "sha512-5vvB5+W7ePv+p3uqxi+RcW1XAzLW0/hxt3/4X4Lc4qHudzOhmBiBwOY6DRob4WnanAEGvNcLjF+KNOufrUoEQw==",
|
||||
"dev": true,
|
||||
"funding": [
|
||||
"https://github.com/sponsors/sxzz",
|
||||
"https://opencollective.com/debug"
|
||||
],
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=12.20.0"
|
||||
}
|
||||
},
|
||||
"node_modules/picocolors": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
|
||||
@@ -1694,6 +1794,33 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/std-env": {
|
||||
"version": "4.3.0",
|
||||
"resolved": "https://registry.npmjs.org/std-env/-/std-env-4.3.0.tgz",
|
||||
"integrity": "sha512-OtU/EgQ1kIm5KwqQpBC6ZEMXrZRui11w8zgfTWp8cdO9B8OaPsbA8bTHO2P+HNo1VlUTGMVBwPhydu6poeXiag==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/tinybench": {
|
||||
"version": "6.2.2",
|
||||
"resolved": "https://registry.npmjs.org/tinybench/-/tinybench-6.2.2.tgz",
|
||||
"integrity": "sha512-nIZ2N2dFpjBsg2hgitKu7mA+FvsIJuGJOwysRngmVRRmWgR/UyPjwJj8c/fT/dhm8EkHnF5EOogRf+0Z2qrimA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tinyexec": {
|
||||
"version": "1.3.1",
|
||||
"resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-1.3.1.tgz",
|
||||
"integrity": "sha512-GCvB3aoys96IuDFBMcTB46JOR6mdMtAToqwiW8JlWhsoh1mhHi/xn9ss/Dg7N555GiJyEt2qzoG/NHCwM6h1EA==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/tinyglobby": {
|
||||
"version": "0.2.17",
|
||||
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.17.tgz",
|
||||
@@ -1853,6 +1980,130 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/vitest": {
|
||||
"version": "5.0.5",
|
||||
"resolved": "https://registry.npmjs.org/vitest/-/vitest-5.0.5.tgz",
|
||||
"integrity": "sha512-cPODOMF8YnkiiYtDdHo6n4wWmBrOBBbOIYH2d33M4l3CdNn72h+qMmO6uXffUN6A5AZFXNND/d+krS9mgkXRfg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/chai": "^5.2.2",
|
||||
"@vitest/mocker": "5.0.5",
|
||||
"chai": "^6.3.0",
|
||||
"es-module-lexer": "^2.3.2",
|
||||
"expect-type": "^1.4.0",
|
||||
"magic-string": "^1.4.3",
|
||||
"obug": "^3.0.0",
|
||||
"picomatch": "^4.0.7",
|
||||
"std-env": "^4.3.0",
|
||||
"tinybench": "^6.2.2",
|
||||
"tinyexec": "^1.3.1",
|
||||
"tinyglobby": "^0.2.17",
|
||||
"why-is-node-running": "3.2.1"
|
||||
},
|
||||
"bin": {
|
||||
"vitest": "vitest.mjs"
|
||||
},
|
||||
"engines": {
|
||||
"node": "^22.12.0 || ^24.0.0 || >=26.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/vitest"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@edge-runtime/vm": "*",
|
||||
"@opentelemetry/api": "^1.9.0",
|
||||
"@types/node": "^22.0.0 || >=24.0.0",
|
||||
"@vitest/browser-playwright": "5.0.5",
|
||||
"@vitest/browser-preview": "5.0.5",
|
||||
"@vitest/browser-webdriverio": "^5.0.0-beta.5 || >=5.0.0",
|
||||
"@vitest/coverage-istanbul": "5.0.5",
|
||||
"@vitest/coverage-v8": "5.0.5",
|
||||
"@vitest/ui": "5.0.5",
|
||||
"happy-dom": "*",
|
||||
"jsdom": "*",
|
||||
"vite": "^6.4.0 || ^7.0.0 || ^8.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@edge-runtime/vm": {
|
||||
"optional": true
|
||||
},
|
||||
"@opentelemetry/api": {
|
||||
"optional": true
|
||||
},
|
||||
"@types/node": {
|
||||
"optional": true
|
||||
},
|
||||
"@vitest/browser-playwright": {
|
||||
"optional": true
|
||||
},
|
||||
"@vitest/browser-preview": {
|
||||
"optional": true
|
||||
},
|
||||
"@vitest/browser-webdriverio": {
|
||||
"optional": true
|
||||
},
|
||||
"@vitest/coverage-istanbul": {
|
||||
"optional": true
|
||||
},
|
||||
"@vitest/coverage-v8": {
|
||||
"optional": true
|
||||
},
|
||||
"@vitest/ui": {
|
||||
"optional": true
|
||||
},
|
||||
"happy-dom": {
|
||||
"optional": true
|
||||
},
|
||||
"jsdom": {
|
||||
"optional": true
|
||||
},
|
||||
"vite": {
|
||||
"optional": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/vitest/node_modules/@vitest/mocker": {
|
||||
"version": "5.0.5",
|
||||
"resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-5.0.5.tgz",
|
||||
"integrity": "sha512-xLL3lkDegJZw+MGoGaCCsFayDZthGgpAtr3Vziyfc4p7xH89K7KRPcRBBh/kEYm0Aqkt5DtTmkYpBtRV/FrHZg==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@jridgewell/trace-mapping": "0.3.31",
|
||||
"@vitest/spy": "5.0.5",
|
||||
"estree-walker": "^3.0.3",
|
||||
"magic-string": "^1.4.3"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://opencollective.com/vitest"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"msw": "^2.4.9 || ^3.0.0",
|
||||
"vite": "^6.0.0 || ^7.0.0 || ^8.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"msw": {
|
||||
"optional": true
|
||||
},
|
||||
"vite": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/why-is-node-running": {
|
||||
"version": "3.2.1",
|
||||
"resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-3.2.1.tgz",
|
||||
"integrity": "sha512-Tb2FUhB4vUsGQlfSquQLYkApkuPAFQXGFzxWKHHumVz2dK+X1RUm/HnID4+TfIGYJ1kTcwOaCk/buYCEJr6YjQ==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"bin": {
|
||||
"why-is-node-running": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.11"
|
||||
}
|
||||
},
|
||||
"node_modules/yallist": {
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/yallist/-/yallist-3.1.1.tgz",
|
||||
|
||||
@@ -7,7 +7,8 @@
|
||||
"build": "tsc -b && vite build",
|
||||
"dev": "vite",
|
||||
"preview": "vite preview",
|
||||
"build:console": "npm run build && node scripts/copy-build.mjs"
|
||||
"build:console": "npm run build && node scripts/copy-build.mjs",
|
||||
"test": "vitest run"
|
||||
},
|
||||
"dependencies": {
|
||||
"react": "^19.1.0",
|
||||
@@ -19,6 +20,7 @@
|
||||
"@vitejs/plugin-react": "^4.4.1",
|
||||
"typescript": "~5.8.3",
|
||||
"vite": "^6.3.5",
|
||||
"vite-plugin-singlefile": "^2.2.0"
|
||||
"vite-plugin-singlefile": "^2.2.0",
|
||||
"vitest": "^5.0.5"
|
||||
}
|
||||
}
|
||||
@@ -1,17 +1,23 @@
|
||||
// build:console 的第二步:把 vite-plugin-singlefile 产出的单文件
|
||||
// 原样拷到可部署位置。绝不改动产物内容——以后改页面只改 src 再构建。
|
||||
// 在本施工目录,目标是 ../console-redesign/console-react.html(主 agent
|
||||
// 审阅用);进仓库后,主 agent 把同一文件拷到 cloud/console.html。
|
||||
import { copyFileSync, statSync } from "node:fs";
|
||||
import { createHash } from "node:crypto";
|
||||
// 目标就是 GET /console 现读的文件 cloud/console.html
|
||||
// (cloud/server.py 与 console.html 同目录、每次请求现读),
|
||||
// 不再拷到仓库里不存在的 cloud/console-redesign/console-react.html,
|
||||
// 也不再靠手工拷兜底:npm run build:console 一次跑通即产物就位。
|
||||
import { copyFileSync } from "node:fs";
|
||||
import { createHash, timingSafeEqual } from "node:crypto";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { dirname, join } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
|
||||
const src = join(root, "dist", "index.html");
|
||||
const dest = join(root, "..", "console-redesign", "console-react.html");
|
||||
const dest = join(root, "..", "console.html");
|
||||
copyFileSync(src, dest);
|
||||
const buf = readFileSync(dest);
|
||||
console.log(`console-react.html ${buf.length} bytes sha256=${createHash("sha256").update(buf).digest("hex")}`);
|
||||
const srcBuf = readFileSync(src);
|
||||
const destBuf = readFileSync(dest);
|
||||
if (srcBuf.length !== destBuf.length || !timingSafeEqual(srcBuf, destBuf)) {
|
||||
throw new Error("copy-build: dest 与 dist/index.html 内容不一致,构建失败");
|
||||
}
|
||||
console.log(`console.html ${destBuf.length} bytes sha256=${createHash("sha256").update(destBuf).digest("hex")}`);
|
||||
console.log(`built: ${src} -> ${dest}`);
|
||||
@@ -0,0 +1,85 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { chvAttachmentBlocked, chvAvailability, chvAvailabilityText, chvQueueHint, chvVmBadge, chvVmDisplay, chvVmStatusText, countChvWorkers, isWorkerAlive } from "./chv";
|
||||
|
||||
// closeup 轨补边角:上一轨 chv.test.ts 已覆主路径后仍缺的
|
||||
// 显式 false、undefined、优先级与“旧 bug 文案不复发”断言。
|
||||
// 只测 chv.ts 纯逻辑,不渲染组件、不发请求。
|
||||
describe("chvVmDisplay 边角(closeup)", () => {
|
||||
it("字段显式 false 与 absent 同属 not-reported", () => {
|
||||
const d = chvVmDisplay({ executor: "cloud-hypervisor", chv_vm_dead: false });
|
||||
expect(d.state).toBe("not-reported");
|
||||
expect(d.text).toContain("不代表 VM 活着");
|
||||
});
|
||||
it("任务 undefined 与 null 同属 unknown", () => {
|
||||
expect(chvVmDisplay(undefined).state).toBe("unknown");
|
||||
expect(chvVmDisplay(undefined).text).toContain("无法判断");
|
||||
});
|
||||
it("非 CHV 且无 executor 字段时也是 not-applicable", () => {
|
||||
expect(chvVmDisplay({}).state).toBe("not-applicable");
|
||||
expect(chvVmDisplay({ executor: undefined }).text).toBe("");
|
||||
expect(chvVmStatusText({ executor: "cloud-hypervisor", chv_vm_dead: false })).toContain("未报告 VM 死亡");
|
||||
});
|
||||
it("列表徽标:显式 false 不出徽标", () => {
|
||||
expect(chvVmBadge({ executor: "cloud-hypervisor", chv_vm_dead: false })).toBe("");
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvAvailability 优先级(closeup)", () => {
|
||||
it("加载失败优先于聚合文案:不把失败写成 aggregate-only", () => {
|
||||
expect(chvAvailability(null, "完整 worker 清单仅全局管理员可见", "加载失败:x")).toEqual(
|
||||
{ state: "unknown", reason: "load-failed" },
|
||||
);
|
||||
});
|
||||
it("有清单时忽略错误与聚合文案", () => {
|
||||
expect(chvAvailability([{ executor: "cloud-hypervisor", alive: false }], "完整 worker 清单仅全局管理员可见", "加载失败:x")).toEqual(
|
||||
{ state: "known", online: 0, total: 1 },
|
||||
);
|
||||
});
|
||||
it("空清单与空文案是 not-loaded,不是已知 0", () => {
|
||||
expect(chvAvailability(null, "", "")).toEqual({ state: "unknown", reason: "not-loaded" });
|
||||
});
|
||||
it("空 worker 数组是已知 0(与 null 区分)", () => {
|
||||
expect(chvAvailability([], "", "")).toEqual({ state: "known", online: 0, total: 0 });
|
||||
expect(countChvWorkers([])).toEqual({ online: 0, total: 0 });
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvAvailabilityText:Workers 旧 bug 不复发(closeup)", () => {
|
||||
it("load-failed / not-loaded 绝不写“仅全局管理员可见”", () => {
|
||||
expect(chvAvailabilityText({ state: "unknown", reason: "load-failed" })).not.toContain("仅全局管理员可见");
|
||||
expect(chvAvailabilityText({ state: "unknown", reason: "not-loaded" })).not.toContain("仅全局管理员可见");
|
||||
});
|
||||
it("known 总数为 0 时仍是已知 0,不写未知", () => {
|
||||
const text = chvAvailabilityText({ state: "known", online: 0, total: 0 });
|
||||
expect(text).toContain("在线 0 / 共 0");
|
||||
expect(text).not.toContain("未知");
|
||||
});
|
||||
});
|
||||
|
||||
describe("附件拦截与排队提示边角(closeup)", () => {
|
||||
it("executor 为 undefined/null 时附件不拦", () => {
|
||||
expect(chvAttachmentBlocked(undefined, true)).toBe(false);
|
||||
expect(chvAttachmentBlocked(null, true)).toBe(false);
|
||||
});
|
||||
it("排队提示:未知三原因一律为空串", () => {
|
||||
for (const reason of ["aggregate-only", "load-failed", "not-loaded"] as const) {
|
||||
expect(chvQueueHint("cloud-hypervisor", "queued", { state: "unknown", reason })).toBe("");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("isWorkerAlive 边界(closeup)", () => {
|
||||
// 精确边界不用活时钟测:原写法先取 Date.now() 再让被测函数
|
||||
// 内部重取一次时钟,age = 120 + 两次取时钟的差,两次取时钟
|
||||
// 跨过毫秒边界时 age = 120.001,<=120 判定为 false,约 7%
|
||||
// 概率抖动(30 轮 2 败,均是本用例)。生产逻辑 <=120 本身
|
||||
// 没错,错的是用活时钟测精确边界。
|
||||
it("恰好 120 秒算在线(<=120,用不取时钟的字段确定性测)", () => {
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor", last_seen_seconds_ago: 120 })).toBe(true);
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor", last_seen_seconds_ago: 121 })).toBe(false);
|
||||
});
|
||||
it("活时钟只测带 1 秒余量的内侧,不测精确边界", () => {
|
||||
const now = Date.now() / 1000;
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor", last_seen: now - 119 })).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,155 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { chvAttachmentBlocked, chvAvailability, chvAvailabilityText, chvQueueHint, chvVmBadge, chvVmDisplay, chvVmStatusText, countChvWorkers, isChvExecutor, isWorkerAlive } from "./chv";
|
||||
|
||||
// 纯逻辑测试:只测 chv.ts,不渲染组件、不发请求、不读 localStorage。
|
||||
// alive 规则与 WorkersSection 现有口径一致:alive 字段优先,
|
||||
// 缺失时用 last_seen 距今 <= 120 秒判定在线。
|
||||
describe("isWorkerAlive", () => {
|
||||
it("alive 字段优先", () => {
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor", alive: true })).toBe(true);
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor", alive: false, last_seen: Date.now() / 1000 })).toBe(false);
|
||||
});
|
||||
it("无 alive 时用 last_seen 120 秒门槛", () => {
|
||||
const now = Date.now() / 1000;
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor", last_seen: now - 10 })).toBe(true);
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor", last_seen: now - 121 })).toBe(false);
|
||||
expect(isWorkerAlive({ executor: "cloud-hypervisor" })).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("countChvWorkers", () => {
|
||||
it("只数 cloud-hypervisor,不数 docker/local", () => {
|
||||
const now = Date.now() / 1000;
|
||||
expect(countChvWorkers([
|
||||
{ executor: "cloud-hypervisor", alive: true },
|
||||
{ executor: "cloud-hypervisor", last_seen: now - 9999 },
|
||||
{ executor: "docker", alive: true },
|
||||
])).toEqual({ online: 1, total: 2 });
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvAvailability", () => {
|
||||
it("有清单时按清单数", () => {
|
||||
expect(chvAvailability([{ executor: "cloud-hypervisor", alive: true }], "", "")).toEqual(
|
||||
{ state: "known", online: 1, total: 1 },
|
||||
);
|
||||
});
|
||||
it("无清单但聚合文案时是未知,不是 0", () => {
|
||||
expect(chvAvailability(null, "在线 2 / 离线 1(共 3;完整 worker 清单仅全局管理员可见)", "")).toEqual(
|
||||
{ state: "unknown", reason: "aggregate-only" },
|
||||
);
|
||||
});
|
||||
it("无 worker 注册时是已知 0", () => {
|
||||
expect(chvAvailability(null, "无 worker 注册", "")).toEqual({ state: "known", online: 0, total: 0 });
|
||||
});
|
||||
it("加载失败时是未知", () => {
|
||||
expect(chvAvailability(null, "", "加载失败:x")).toEqual({ state: "unknown", reason: "load-failed" });
|
||||
});
|
||||
it("尚未加载时是未知", () => {
|
||||
expect(chvAvailability(null, "未加载", "")).toEqual({ state: "unknown", reason: "not-loaded" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("isChvExecutor", () => {
|
||||
it("精确值才算 CHV,不做大小写/前缀宽松匹配", () => {
|
||||
expect(isChvExecutor("cloud-hypervisor")).toBe(true);
|
||||
expect(isChvExecutor("Cloud-Hypervisor")).toBe(false);
|
||||
expect(isChvExecutor("cloud-hypervisor-x")).toBe(false);
|
||||
expect(isChvExecutor("docker")).toBe(false);
|
||||
expect(isChvExecutor(undefined)).toBe(false);
|
||||
expect(isChvExecutor(null)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvAttachmentBlocked", () => {
|
||||
it("仅 CHV + 带附件时拦(后端 400 的前端提示口径)", () => {
|
||||
expect(chvAttachmentBlocked("cloud-hypervisor", true)).toBe(true);
|
||||
expect(chvAttachmentBlocked("cloud-hypervisor", false)).toBe(false);
|
||||
expect(chvAttachmentBlocked("docker", true)).toBe(false);
|
||||
expect(chvAttachmentBlocked("cloud", true)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvVmStatusText", () => {
|
||||
it("已报告死亡时给死亡文案", () => {
|
||||
expect(chvVmStatusText({ executor: "cloud-hypervisor", chv_vm_dead: true })).toContain("VM 中途死亡");
|
||||
});
|
||||
it("CHV 无字段时给未报告文案,绝不写成活着", () => {
|
||||
const text = chvVmStatusText({ executor: "cloud-hypervisor" });
|
||||
expect(text).toContain("未报告 VM 死亡");
|
||||
expect(text).toContain("不代表 VM 活着");
|
||||
});
|
||||
it("非 CHV 任务不显示 VM 行", () => {
|
||||
expect(chvVmStatusText({ executor: "docker" })).toBe("");
|
||||
expect(chvVmStatusText({ executor: "docker", chv_vm_dead: true })).toBe("");
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvVmBadge", () => {
|
||||
it("只有 CHV + 已报告死亡时才出徽标", () => {
|
||||
expect(chvVmBadge({ executor: "cloud-hypervisor", chv_vm_dead: true })).toBe("VM 已报告死亡");
|
||||
expect(chvVmBadge({ executor: "cloud-hypervisor" })).toBe("");
|
||||
expect(chvVmBadge({ executor: "docker", chv_vm_dead: true })).toBe("");
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvQueueHint", () => {
|
||||
it("仅 CHV + 排队 + 已知零在线时给提示", () => {
|
||||
const knownZero = { state: "known" as const, online: 0, total: 0 };
|
||||
expect(chvQueueHint("cloud-hypervisor", "queued", knownZero)).toContain("可能原因");
|
||||
expect(chvQueueHint("docker", "queued", knownZero)).toBe("");
|
||||
expect(chvQueueHint("cloud-hypervisor", "running", knownZero)).toBe("");
|
||||
expect(chvQueueHint("cloud-hypervisor", "queued", { state: "unknown" as const, reason: "aggregate-only" })).toBe("");
|
||||
expect(chvQueueHint("cloud-hypervisor", "queued", { state: "known" as const, online: 1, total: 1 })).toBe("");
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvVmDisplay(to10 四态)", () => {
|
||||
it("字段 present=true → dead", () => {
|
||||
const d = chvVmDisplay({ executor: "cloud-hypervisor", chv_vm_dead: true });
|
||||
expect(d.state).toBe("dead");
|
||||
expect(d.text).toContain("VM 中途死亡");
|
||||
});
|
||||
it("字段 absent → not-reported,不猜 live", () => {
|
||||
const d = chvVmDisplay({ executor: "cloud-hypervisor" });
|
||||
expect(d.state).toBe("not-reported");
|
||||
expect(d.text).toContain("不代表 VM 活着");
|
||||
});
|
||||
it("任务未加载 → unknown", () => {
|
||||
const d = chvVmDisplay(null);
|
||||
expect(d.state).toBe("unknown");
|
||||
expect(d.text).toContain("无法判断");
|
||||
});
|
||||
it("非 CHV → not-applicable,text 为空", () => {
|
||||
const d = chvVmDisplay({ executor: "docker", chv_vm_dead: true });
|
||||
expect(d.state).toBe("not-applicable");
|
||||
expect(d.text).toBe("");
|
||||
});
|
||||
it("旧入口仍是 display 的 text(unknown/not-applicable 回空)", () => {
|
||||
expect(chvVmStatusText({ executor: "cloud-hypervisor", chv_vm_dead: true })).toContain("VM 中途死亡");
|
||||
expect(chvVmStatusText(null)).toBe("");
|
||||
expect(chvVmStatusText({ executor: "docker" })).toBe("");
|
||||
});
|
||||
});
|
||||
|
||||
describe("chvAvailabilityText(to10 四处统一文案)", () => {
|
||||
it("known 零在线:不拦提交、后端是最终权威", () => {
|
||||
const text = chvAvailabilityText({ state: "known", online: 0, total: 2 });
|
||||
expect(text).toContain("在线 0 / 共 2");
|
||||
expect(text).toContain("不拦提交");
|
||||
expect(text).toContain("最终权威");
|
||||
});
|
||||
it("known 非零:只有在线数,无排队警告", () => {
|
||||
const text = chvAvailabilityText({ state: "known", online: 1, total: 1 });
|
||||
expect(text).toContain("在线 1 / 共 1");
|
||||
expect(text).not.toContain("排队");
|
||||
});
|
||||
it("未知三原因各有不同文案,绝不写成 0", () => {
|
||||
expect(chvAvailabilityText({ state: "unknown", reason: "aggregate-only" })).toContain("仅全局管理员可见");
|
||||
expect(chvAvailabilityText({ state: "unknown", reason: "load-failed" })).toContain("加载失败");
|
||||
expect(chvAvailabilityText({ state: "unknown", reason: "not-loaded" })).toContain("尚未加载");
|
||||
for (const reason of ["aggregate-only", "load-failed", "not-loaded"] as const) {
|
||||
expect(chvAvailabilityText({ state: "unknown", reason })).not.toContain("在线 0");
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,163 @@
|
||||
import type { Worker } from "./types";
|
||||
import { workerAgeText } from "./client";
|
||||
|
||||
// CHV 可用性纯逻辑:不发请求、不拼 URL,只从 ctx 已有的
|
||||
// /workers 数据推断。非管理员的 /workers 只返回聚合数
|
||||
// (workers_total/online/offline),没有按执行器的分项,
|
||||
// 此时 CHV 在线数是未知,绝不能把未知写成 0,也绝不能
|
||||
// 把聚合在线数当成 CHV 在线数。
|
||||
export function isWorkerAlive(w: Worker): boolean {
|
||||
if (w.alive !== undefined && w.alive !== null) return !!w.alive;
|
||||
const info = workerAgeText(w);
|
||||
return info.age !== null && info.age <= 120;
|
||||
}
|
||||
|
||||
export function countChvWorkers(workers: Worker[]): { online: number; total: number } {
|
||||
const chv = workers.filter((w) => w.executor === "cloud-hypervisor");
|
||||
return { online: chv.filter(isWorkerAlive).length, total: chv.length };
|
||||
}
|
||||
|
||||
export type ChvAvailability =
|
||||
| { state: "known"; online: number; total: number }
|
||||
| { state: "unknown"; reason: "not-loaded" | "aggregate-only" | "load-failed" };
|
||||
|
||||
export function chvAvailability(
|
||||
workers: Worker[] | null,
|
||||
workersSummary: string,
|
||||
workersError: string,
|
||||
): ChvAvailability {
|
||||
if (workers) return { state: "known", ...countChvWorkers(workers) };
|
||||
if (workersError) return { state: "unknown", reason: "load-failed" };
|
||||
if (workersSummary.includes("无 worker 注册")) return { state: "known", online: 0, total: 0 };
|
||||
if (workersSummary.includes("完整 worker 清单仅全局管理员可见")) {
|
||||
return { state: "unknown", reason: "aggregate-only" };
|
||||
}
|
||||
return { state: "unknown", reason: "not-loaded" };
|
||||
}
|
||||
|
||||
// 执行器选择谓词:派发下拉与列表筛选都以字符串相等为准,
|
||||
// 不做大小写/前缀宽松匹配(后端白名单是精确值,见 server
|
||||
// POST /tasks 的 executor 白名单;宽松匹配会把非法值放进
|
||||
// “CHV 任务”分支,造成前端提示与后端拒绝不一致)。
|
||||
export function isChvExecutor(executor: string | undefined | null): boolean {
|
||||
return executor === "cloud-hypervisor";
|
||||
}
|
||||
|
||||
// 附件守卫谓词:CHV + 带附件 → 后端建任务即拒绝(server
|
||||
// POST /tasks 的附件绑定分支,错误文案“附件不支持
|
||||
// cloud-hypervisor 执行器”)。本函数只作前端提示口径,
|
||||
// 绝不代替后端白名单:前端照常提交,后端拒绝为准。
|
||||
export function chvAttachmentBlocked(
|
||||
executor: string | undefined | null,
|
||||
hasAttachments: boolean,
|
||||
): boolean {
|
||||
return isChvExecutor(executor) && hasAttachments;
|
||||
}
|
||||
|
||||
// VM 只读状态:四种情形各有明确文案,绝不编造第五种,
|
||||
// 绝不猜 live(to10 精确化,见 ADR-15):
|
||||
// - dead:CHV 任务且 chv_vm_dead 为 true(字段已 present;
|
||||
// 来源:worker 终局写回的 interrupted_reason,见 server
|
||||
// _chv_vm_dead_from_row);
|
||||
// - not-reported:CHV 任务但字段 absent(后端只在死亡时
|
||||
// 才返回该字段;absent 只表示“未报告”,且明写“后端无
|
||||
// live 状态字段,不代表 VM 活着”,server 没有 VM 登记表);
|
||||
// - unknown:任务详情本身未加载(task 为 null),无法判断;
|
||||
// - not-applicable:非 CHV 任务,不显示 VM 行(text 为空)。
|
||||
export type ChvVmDisplay =
|
||||
| { state: "dead"; text: string }
|
||||
| { state: "not-reported"; text: string }
|
||||
| { state: "unknown"; text: string }
|
||||
| { state: "not-applicable"; text: string };
|
||||
|
||||
export function chvVmDisplay(task: {
|
||||
executor?: string | null;
|
||||
chv_vm_dead?: boolean;
|
||||
} | null | undefined): ChvVmDisplay {
|
||||
if (task === null || task === undefined) {
|
||||
return {
|
||||
state: "unknown",
|
||||
text: "任务详情未加载,无法判断 VM 状态(后端无 live 状态字段,详情加载后才显示)",
|
||||
};
|
||||
}
|
||||
// 死亡文案同样先过执行器门:后端只对 CHV 任务派生该字段,
|
||||
// 万一某天非 CHV 行误带该字段,前端也不把 docker 任务
|
||||
// 写成“VM 死亡”。
|
||||
if (task.chv_vm_dead && isChvExecutor(task.executor)) {
|
||||
return {
|
||||
state: "dead",
|
||||
text: "VM 中途死亡,任务未完成、断点可续跑(后端只读字段 chv_vm_dead)",
|
||||
};
|
||||
}
|
||||
if (isChvExecutor(task.executor)) {
|
||||
return {
|
||||
state: "not-reported",
|
||||
text: "未报告 VM 死亡(后端无 live 状态字段,不代表 VM 活着)",
|
||||
};
|
||||
}
|
||||
return { state: "not-applicable", text: "" };
|
||||
}
|
||||
|
||||
// 旧入口(保留给已测调用点):即 chvVmDisplay 的 text,
|
||||
// not-applicable 与 unknown 均回空串(调用点只在已加载的
|
||||
// CHV 行上用它,unknown 分支由详情页的新入口处理)。
|
||||
export function chvVmStatusText(task: {
|
||||
executor?: string | null;
|
||||
chv_vm_dead?: boolean;
|
||||
} | null | undefined): string {
|
||||
const display = chvVmDisplay(task);
|
||||
return display.state === "not-applicable" ||
|
||||
display.state === "unknown"
|
||||
? ""
|
||||
: display.text;
|
||||
}
|
||||
|
||||
// CHV 可用性统一文案(to10):新建任务、Workers、环境
|
||||
// 三处此前各写各的,Workers 的未知分支甚至恒写“仅全局
|
||||
// 管理员可见”(加载失败/未加载时也是这句,是文案 bug)。
|
||||
// 四处统一走本函数:known 零在线时不拦提交、后端是最终
|
||||
// 权威;未知时按 reason 分三句,绝不把未知写成 0。
|
||||
export function chvAvailabilityText(
|
||||
availability: ChvAvailability,
|
||||
): string {
|
||||
if (availability.state === "known") {
|
||||
const base = `CHV worker 在线 ${availability.online} / 共 ${availability.total}(来自 /workers)`;
|
||||
return availability.online === 0
|
||||
? `${base}:当前无 CHV worker 在线,任务将一直排队,请先确认宿主 worker 已起;本提示不拦提交,worker 随后上线可认领,后端是最终权威。`
|
||||
: base;
|
||||
}
|
||||
if (availability.reason === "aggregate-only") {
|
||||
return "CHV worker 在线数未知:完整 worker 清单仅全局管理员可见,非管理员只能看到聚合数,无法按执行器分项。";
|
||||
}
|
||||
if (availability.reason === "load-failed") {
|
||||
return "CHV worker 在线数未知:/workers 加载失败,请到 Workers 区刷新后重试。";
|
||||
}
|
||||
return "CHV worker 在线数未知:/workers 尚未加载,请到 Workers 区刷新。";
|
||||
}
|
||||
|
||||
// 列表行徽标:只有已报告死亡时才出现短徽标;未报告时
|
||||
// 列表行不写 VM 状态(详情里才写“未报告”长文案,避免
|
||||
// 列表行被未知状态刷屏)。
|
||||
export function chvVmBadge(task: {
|
||||
executor?: string | null;
|
||||
chv_vm_dead?: boolean;
|
||||
}): string {
|
||||
return task.chv_vm_dead && isChvExecutor(task.executor)
|
||||
? "VM 已报告死亡"
|
||||
: "";
|
||||
}
|
||||
|
||||
// 排队原因提示:只在“已知零 CHV worker 在线 + CHV 任务排队”时
|
||||
// 给出,文案明确写“可能原因”,且不编造后端未返回的
|
||||
// VM 状态/IP/快照。未知时返回空串(不提示、不猜测)。
|
||||
export function chvQueueHint(
|
||||
executor: string | undefined | null,
|
||||
status: string | undefined | null,
|
||||
availability: ChvAvailability,
|
||||
): string {
|
||||
if (executor !== "cloud-hypervisor" || status !== "queued") return "";
|
||||
if (availability.state === "known" && availability.online === 0) {
|
||||
return "可能原因:当前无 cloud-hypervisor worker 在线,任务在排队(基于 /workers 推断;VM 状态/IP/快照等后端未返回字段不展示)。请先确认 KVM 宿主上的 CHV worker 已起,worker 上线后可认领。";
|
||||
}
|
||||
return "";
|
||||
}
|
||||
@@ -21,6 +21,10 @@ export interface Task {
|
||||
coordinator_part?: string; goal_mode?: number | boolean; verify_command?: string;
|
||||
no_time_limit?: number | boolean;
|
||||
environment_id?: string; attachments?: string[]; model?: string; effort?: string;
|
||||
// feat-chv-arch95plus 只读字段:仅 CHV 任务且 worker 已把
|
||||
// VM 死亡写进 interrupted_reason 时 server 才返回 true,
|
||||
// 其它情形字段不存在(绝不把“未报告死亡”当成“VM 活着”)。
|
||||
chv_vm_dead?: boolean;
|
||||
child_model?: string; child_effort?: string; approval?: string;
|
||||
result_text?: string; error_text?: string;
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@ import { useRef, useState } from "react";
|
||||
import { useConsole } from "../context";
|
||||
import { Card, Bar } from "../components/ui";
|
||||
import { gatherAttachments } from "../api/client";
|
||||
import { chvAttachmentBlocked, chvAvailability, chvAvailabilityText, isChvExecutor } from "../api/chv";
|
||||
import { EFFORTS } from "../api/types";
|
||||
|
||||
export default function DispatchSection() {
|
||||
@@ -22,7 +23,15 @@ export default function DispatchSection() {
|
||||
const filesRef = useRef<HTMLInputElement>(null);
|
||||
const dirRef = useRef<HTMLInputElement>(null);
|
||||
const imgRef = useRef<HTMLInputElement>(null);
|
||||
const [hasAttachments, setHasAttachments] = useState(false);
|
||||
const [msg, setMsg] = useState("");
|
||||
const chvAvail = chvAvailability(ctx.workers, ctx.workersSummary, ctx.workersError);
|
||||
const isChv = isChvExecutor(executor);
|
||||
function refreshAttachments() {
|
||||
setHasAttachments(
|
||||
!!(filesRef.current?.files?.length || dirRef.current?.files?.length || imgRef.current?.files?.length),
|
||||
);
|
||||
}
|
||||
|
||||
async function submit() {
|
||||
setMsg("");
|
||||
@@ -60,6 +69,7 @@ export default function DispatchSection() {
|
||||
if (goalMode || goalInvincible || planMode) { setMsg("协调员任务不能与 goal/计划模式同用:协调员任务本身不跑模型回合,请取消 goal/计划勾选,或清空协调员 JSON"); return; }
|
||||
}
|
||||
const atts = gatherAttachments(filesRef.current?.files || null, dirRef.current?.files || null, imgRef.current?.files || null);
|
||||
if (chvAttachmentBlocked(executor, atts.length > 0)) { setMsg("cloud-hypervisor 任务不支持附件:请先清空附件再提交(后端对带附件的 CHV 任务会 400)"); return; }
|
||||
try {
|
||||
const result = await ctx.dispatch(body, atts);
|
||||
setMsg(result);
|
||||
@@ -67,6 +77,7 @@ export default function DispatchSection() {
|
||||
if (filesRef.current) filesRef.current.value = "";
|
||||
if (dirRef.current) dirRef.current.value = "";
|
||||
if (imgRef.current) imgRef.current.value = "";
|
||||
setHasAttachments(false);
|
||||
} catch (e: any) { setMsg("提交失败:" + (e?.message || String(e))); }
|
||||
}
|
||||
|
||||
@@ -78,12 +89,19 @@ export default function DispatchSection() {
|
||||
{ctx.envs.map((e) => <option key={e.id} value={e.id} disabled={e.status !== "ready"}>{e.name}({e.status})</option>)}
|
||||
</select>
|
||||
<select aria-label="执行器" value={executor} onChange={(e) => setExecutor(e.target.value)}>
|
||||
<option value="docker">docker(默认,多用户下隔离最强)</option><option value="local">local</option><option value="cloud">cloud</option>
|
||||
<option value="docker">docker(默认,多用户下隔离最强)</option><option value="local">local</option><option value="cloud">cloud</option><option value="cloud-hypervisor">cloud-hypervisor(实验,默认关闭)</option>
|
||||
</select>
|
||||
<label><input type="checkbox" checked={goalMode} onChange={(e) => setGoalMode(e.target.checked)} /> goal 模式</label>
|
||||
<label><input type="checkbox" checked={goalInvincible} onChange={(e) => setGoalInvincible(e.target.checked)} /> goal 模式无敌版</label>
|
||||
<label><input type="checkbox" checked={planMode} onChange={(e) => setPlanMode(e.target.checked)} /> 计划模式(只读)</label>
|
||||
</Bar>
|
||||
{isChv ? <div className="muted">实验执行器,默认关闭:需后端已开启 CHV worker 且宿主有 KVM,否则任务无人认领、一直排队;带附件时建任务即被后端拒绝(错误文案原样显示在下方)。前端只传 executor 值,不绕过后端白名单;默认仍是 docker。</div> : null}
|
||||
{isChv ? (
|
||||
<div className="muted" aria-live="polite">
|
||||
{chvAvailabilityText(chvAvail)}
|
||||
{" "}<button type="button" onClick={ctx.refreshWorkers}>刷新 CHV 可用性</button>
|
||||
</div>
|
||||
) : null}
|
||||
<textarea className="prompt stacked" placeholder="任务 prompt(必填)" value={prompt} onChange={(e) => setPrompt(e.target.value)} />
|
||||
<textarea className="coord stacked" placeholder="协调员子任务 JSON(可选),留空为普通任务" value={coordParts} onChange={(e) => setCoordParts(e.target.value)} />
|
||||
<div className="muted">协调员格式:[{"{"}"name":"后端","prompt":"做后端"{"}"},{"{"}"name":"前端","prompt":"做前端"{"}"}];提交后 worker 并行建子任务、等齐、合流汇总。不能与 goal/计划模式同用。协调员与 goal/计划互斥:勾了 goal 或计划模式时不要填协调员 JSON,提交前会校验并拦截。子任务费用记父任务创建人账;父任务创建时按分工数预留配额,余额不足时整个父任务被挡下。</div>
|
||||
@@ -117,13 +135,14 @@ export default function DispatchSection() {
|
||||
{goalInvincible ? <div className="muted">goal 模式无敌版:goal 模式 + 任务不限时(去掉 worker/docker 总超时;单轮 600 秒护栏还在,不限的是任务总时长与总轮次)。服务端默认的自动续跑与断点镜像同时生效。</div> : null}
|
||||
<Bar>
|
||||
<span className="muted">附件(放进任务工作区,图片与普通文件不区分):</span>
|
||||
<input ref={filesRef} type="file" multiple title="文件" />
|
||||
<input ref={dirRef} type="file" multiple title="文件夹" {...({ webkitdirectory: "" } as any)} />
|
||||
<input ref={imgRef} type="file" accept="image/*" multiple title="图片" />
|
||||
<input ref={filesRef} type="file" multiple title="文件" aria-label="附件文件" disabled={isChv} onChange={refreshAttachments} />
|
||||
<input ref={dirRef} type="file" multiple title="文件夹" aria-label="附件文件夹" disabled={isChv} onChange={refreshAttachments} {...({ webkitdirectory: "" } as any)} />
|
||||
<input ref={imgRef} type="file" accept="image/*" multiple title="图片" aria-label="附件图片" disabled={isChv} onChange={refreshAttachments} />
|
||||
</Bar>
|
||||
{isChv ? <div className="muted">cloud-hypervisor 任务不支持附件:附件输入已禁用(后端对带附件的 CHV 任务会 400,错误文案原样显示在下方;切换回其他执行器可恢复附件输入)。{hasAttachments ? <>当前仍有已选附件,请先清空再提交。 <button type="button" onClick={() => { if (filesRef.current) filesRef.current.value = ""; if (dirRef.current) dirRef.current.value = ""; if (imgRef.current) imgRef.current.value = ""; setHasAttachments(false); }}>清空附件</button></> : ""}</div> : null}
|
||||
<Bar>
|
||||
<button className="seal" onClick={submit}>提交任务</button>
|
||||
<span className="muted">{msg || ctx.dispatchMsg}</span>
|
||||
<span className="muted" role="status" aria-live="polite">{msg || ctx.dispatchMsg}</span>
|
||||
</Bar>
|
||||
</Card>
|
||||
);
|
||||
|
||||
@@ -2,12 +2,14 @@ import { useState } from "react";
|
||||
import { useConsole } from "../context";
|
||||
import { Card, Bar, DataListRaw } from "../components/ui";
|
||||
import { shortId } from "../api/client";
|
||||
import { chvAvailability, chvAvailabilityText } from "../api/chv";
|
||||
|
||||
export default function EnvSection() {
|
||||
const ctx = useConsole();
|
||||
const [name, setName] = useState("");
|
||||
const [branch, setBranch] = useState("");
|
||||
const [executor, setExecutor] = useState("");
|
||||
const chvAvail = chvAvailability(ctx.workers, ctx.workersSummary, ctx.workersError);
|
||||
return (
|
||||
<Card id="nav-env" title={<>环境 <span className="muted">(选、建、管:当前项目的基准检出)</span></>}>
|
||||
<Bar>
|
||||
@@ -31,10 +33,11 @@ export default function EnvSection() {
|
||||
<input type="text" placeholder="新环境名" value={name} onChange={(e) => setName(e.target.value)} />
|
||||
<input type="text" placeholder="分支(可选,空=默认分支)" value={branch} onChange={(e) => setBranch(e.target.value)} />
|
||||
<select aria-label="新环境执行器" value={executor} onChange={(e) => setExecutor(e.target.value)}>
|
||||
<option value="">执行器:不限</option><option value="local">local</option><option value="docker">docker</option>
|
||||
<option value="">执行器:不限</option><option value="local">local</option><option value="docker">docker</option><option value="cloud-hypervisor">cloud-hypervisor(实验,默认关闭)</option>
|
||||
</select>
|
||||
<button onClick={async () => { await ctx.createEnv(name, branch, executor); setName(""); setBranch(""); }}>新建环境</button>
|
||||
</Bar>
|
||||
{executor === "cloud-hypervisor" ? <div className="muted" aria-live="polite">实验执行器,默认关闭:环境创建白名单同样允许 cloud-hypervisor(后端环境接口 allowlist:local / docker / cloud-hypervisor,不含 cloud);仍需后端有 KVM 宿主上的 CHV worker 才会认领,否则任务排队。前端只传值,不绕过后端白名单。 {chvAvailabilityText(chvAvail)}</div> : null}
|
||||
<span className="muted">{ctx.envMsg}</span>
|
||||
</Card>
|
||||
);
|
||||
|
||||
@@ -4,6 +4,7 @@ import { Card } from "../components/ui";
|
||||
import { shortId, fmt } from "../api/client";
|
||||
import { STATUS_EXPLAIN } from "../api/types";
|
||||
import { firstChar, durationOf, tokenOf, fmtCostUsd } from "../api/taskMeta";
|
||||
import { chvAvailability, chvQueueHint, chvVmDisplay } from "../api/chv";
|
||||
import DetailActions from "./detail/DetailActions";
|
||||
import BrowserPanel from "./detail/BrowserPanel";
|
||||
import ReviewPanel from "./detail/ReviewPanel";
|
||||
@@ -57,6 +58,8 @@ export default function TaskDetailPanel() {
|
||||
if (!t) return <Card id="nav-detail" title="任务详情"><div className="muted">点任务行查看详情</div></Card>;
|
||||
const [dur, durTitle] = durationOf(t);
|
||||
const [tok, tokTitle] = tokenOf(t);
|
||||
const chvAvail = chvAvailability(ctx.workers, ctx.workersSummary, ctx.workersError);
|
||||
const queueHint = chvQueueHint(t.executor, t.status, chvAvail);
|
||||
const fc = firstChar(t);
|
||||
const cost = ctx.costSummary;
|
||||
const modeText = t.plan ? "计划模式(只读)"
|
||||
@@ -111,6 +114,7 @@ export default function TaskDetailPanel() {
|
||||
const idle = last ? now - last : null;
|
||||
return (<Card title="LLM 进度">
|
||||
<div>状态 {t.status === "running" ? "进行中" : "排队中"} · 已用 {dur}{idle !== null ? " · 上次活动 " + fmtAge(idle) : ""}</div>
|
||||
{queueHint ? <div className="muted">{queueHint}</div> : null}
|
||||
{idle !== null && idle > 180
|
||||
? <div className="muted">超过 3 分钟无活动:可能在等上游模型响应(上游 503 时 worker 会重试后失败,失败后这里会显示失败原因;无活动不等于已断连,服务端没有“LLM 已连接”字段,不作此断言)</div>
|
||||
: <div className="muted">首轮无输出时属正常:模型第一轮回复到达前 trace 里没有消息;回复到达后会在会话区出现,成本里的 LLM 请求数随之增加</div>}
|
||||
@@ -126,7 +130,8 @@ export default function TaskDetailPanel() {
|
||||
<div><dt>任务</dt><dd>{t.id}</dd></div>
|
||||
<div><dt>turn</dt><dd>{shortId(t.turn_id)}</dd></div>
|
||||
<div><dt>root turn</dt><dd>{shortId(t.root_turn_id)}</dd></div>
|
||||
<div><dt>executor</dt><dd>{t.executor || ""}</dd></div>
|
||||
<div><dt>executor</dt><dd>{t.executor || ""}{t.executor === "cloud-hypervisor" ? "(实验执行器,默认关闭;仅展示,无 VM 控制按钮)" : ""}</dd></div>
|
||||
{t.executor === "cloud-hypervisor" ? <div><dt>VM 状态</dt><dd role="status" aria-label="VM 状态">{chvVmDisplay(t).text}</dd></div> : null}
|
||||
<div><dt>主机</dt><dd>{t.worktree_host || ""}</dd></div>
|
||||
<div><dt>模型</dt><dd>{t.model || "默认"}</dd></div>
|
||||
<div><dt>强度</dt><dd>{t.effort || "默认"}</dd></div>
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { useConsole } from "../context";
|
||||
import { shortId } from "../api/client";
|
||||
import { firstChar, durationOf, tokenOf } from "../api/taskMeta";
|
||||
import { chvAvailability, chvQueueHint, chvVmBadge } from "../api/chv";
|
||||
|
||||
// 任务列表行 meta:首字 / 总耗时 / token,计算见 api/taskMeta.ts,
|
||||
// 与任务详情共用同一口径。
|
||||
@@ -14,6 +15,7 @@ import { firstChar, durationOf, tokenOf } from "../api/taskMeta";
|
||||
// blue-grey background (#ddf4ff), not a bubble.
|
||||
export default function TaskListSection({ onNew }: { onNew: () => void }) {
|
||||
const ctx = useConsole();
|
||||
const chvAvail = chvAvailability(ctx.workers, ctx.workersSummary, ctx.workersError);
|
||||
return (
|
||||
<div>
|
||||
<div className="search-box">
|
||||
@@ -38,7 +40,7 @@ export default function TaskListSection({ onNew }: { onNew: () => void }) {
|
||||
{["queued", "running", "completed", "failed", "interrupted", "merged", "merge-failed"].map((s) => <option key={s} value={s}>{s}</option>)}
|
||||
</select>
|
||||
<select aria-label="执行器筛选" value={ctx.executorFilter} onChange={(e) => ctx.setExecutorFilter(e.target.value)}>
|
||||
<option value="">执行器:全部</option><option value="local">local</option><option value="docker">docker</option><option value="cloud">cloud</option>
|
||||
<option value="">执行器:全部</option><option value="local">local</option><option value="docker">docker</option><option value="cloud">cloud</option><option value="cloud-hypervisor">cloud-hypervisor(实验)</option>
|
||||
</select>
|
||||
</div>
|
||||
<div className="bar">
|
||||
@@ -75,6 +77,12 @@ export default function TaskListSection({ onNew }: { onNew: () => void }) {
|
||||
{(t.status === "failed" || t.status === "interrupted" || t.status === "merge-failed") && t.error_text ? (
|
||||
<div className="issue-meta" title={t.error_text}>{/503|Service Unavailable/.test(t.error_text) ? "上游 503:模型服务暂时不可用" : t.error_text.replace(/\s+/g, " ").slice(0, 60)}</div>
|
||||
) : null}
|
||||
{chvQueueHint(t.executor, t.status, chvAvail) ? (
|
||||
<div className="issue-meta" title={chvQueueHint(t.executor, t.status, chvAvail)}>{chvQueueHint(t.executor, t.status, chvAvail)}</div>
|
||||
) : null}
|
||||
{chvVmBadge(t) ? (
|
||||
<div className="issue-meta" title="后端只读字段 chv_vm_dead(worker 终局写回的 VM 死亡原因派生);无徽标时不代表 VM 活着">{chvVmBadge(t)}</div>
|
||||
) : null}
|
||||
</span>
|
||||
</li>
|
||||
);
|
||||
|
||||
@@ -1,12 +1,17 @@
|
||||
import { useConsole } from "../context";
|
||||
import { Card, Bar, DataListRaw } from "../components/ui";
|
||||
import { workerAgeText, shortId } from "../api/client";
|
||||
import { chvAvailability, chvAvailabilityText } from "../api/chv";
|
||||
|
||||
export default function WorkersSection() {
|
||||
const ctx = useConsole();
|
||||
const chvAvail = chvAvailability(ctx.workers, ctx.workersSummary, ctx.workersError);
|
||||
return (
|
||||
<Card id="nav-workers" title="Workers">
|
||||
<Bar><button onClick={ctx.refreshWorkers}>刷新</button></Bar>
|
||||
<div className="muted" aria-live="polite">
|
||||
{chvAvailabilityText(chvAvail)}
|
||||
</div>
|
||||
{ctx.workersError ? <div className="muted">{ctx.workersError}</div>
|
||||
: ctx.workers === null ? <div className="muted">{ctx.workersSummary || "未加载"}</div>
|
||||
: ctx.workers.length === 0 ? <div className="muted">无 worker 注册</div>
|
||||
|
||||
+26
-26
@@ -4,7 +4,7 @@
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Codex 命令台</title>
|
||||
<script type="module" crossorigin>(function(){const r=document.createElement("link").relList;if(r&&r.supports&&r.supports("modulepreload"))return;for(const g of document.querySelectorAll('link[rel="modulepreload"]'))f(g);new MutationObserver(g=>{for(const C of g)if(C.type==="childList")for(const A of C.addedNodes)A.tagName==="LINK"&&A.rel==="modulepreload"&&f(A)}).observe(document,{childList:!0,subtree:!0});function h(g){const C={};return g.integrity&&(C.integrity=g.integrity),g.referrerPolicy&&(C.referrerPolicy=g.referrerPolicy),g.crossOrigin==="use-credentials"?C.credentials="include":g.crossOrigin==="anonymous"?C.credentials="omit":C.credentials="same-origin",C}function f(g){if(g.ep)return;g.ep=!0;const C=h(g);fetch(g.href,C)}})();function Kh(i){return i&&i.__esModule&&Object.prototype.hasOwnProperty.call(i,"default")?i.default:i}var no={exports:{}},ui={};/**
|
||||
<script type="module" crossorigin>(function(){const r=document.createElement("link").relList;if(r&&r.supports&&r.supports("modulepreload"))return;for(const p of document.querySelectorAll('link[rel="modulepreload"]'))f(p);new MutationObserver(p=>{for(const N of p)if(N.type==="childList")for(const C of N.addedNodes)C.tagName==="LINK"&&C.rel==="modulepreload"&&f(C)}).observe(document,{childList:!0,subtree:!0});function h(p){const N={};return p.integrity&&(N.integrity=p.integrity),p.referrerPolicy&&(N.referrerPolicy=p.referrerPolicy),p.crossOrigin==="use-credentials"?N.credentials="include":p.crossOrigin==="anonymous"?N.credentials="omit":N.credentials="same-origin",N}function f(p){if(p.ep)return;p.ep=!0;const N=h(p);fetch(p.href,N)}})();function Ph(i){return i&&i.__esModule&&Object.prototype.hasOwnProperty.call(i,"default")?i.default:i}var co={exports:{}},ui={};/**
|
||||
* @license React
|
||||
* react-jsx-runtime.production.js
|
||||
*
|
||||
@@ -12,7 +12,7 @@
|
||||
*
|
||||
* This source code is licensed under the MIT license found in the
|
||||
* LICENSE file in the root directory of this source tree.
|
||||
*/var Rh;function ky(){if(Rh)return ui;Rh=1;var i=Symbol.for("react.transitional.element"),r=Symbol.for("react.fragment");function h(f,g,C){var A=null;if(C!==void 0&&(A=""+C),g.key!==void 0&&(A=""+g.key),"key"in g){C={};for(var S in g)S!=="key"&&(C[S]=g[S])}else C=g;return g=C.ref,{$$typeof:i,type:f,key:A,ref:g!==void 0?g:null,props:C}}return ui.Fragment=r,ui.jsx=h,ui.jsxs=h,ui}var Dh;function By(){return Dh||(Dh=1,no.exports=ky()),no.exports}var c=By(),uo={exports:{}},yt={};/**
|
||||
*/var Hh;function Vy(){if(Hh)return ui;Hh=1;var i=Symbol.for("react.transitional.element"),r=Symbol.for("react.fragment");function h(f,p,N){var C=null;if(N!==void 0&&(C=""+N),p.key!==void 0&&(C=""+p.key),"key"in p){N={};for(var j in p)j!=="key"&&(N[j]=p[j])}else N=p;return p=N.ref,{$$typeof:i,type:f,key:C,ref:p!==void 0?p:null,props:N}}return ui.Fragment=r,ui.jsx=h,ui.jsxs=h,ui}var Bh;function Qy(){return Bh||(Bh=1,co.exports=Vy()),co.exports}var c=Qy(),so={exports:{}},pt={};/**
|
||||
* @license React
|
||||
* react.production.js
|
||||
*
|
||||
@@ -20,7 +20,7 @@
|
||||
*
|
||||
* This source code is licensed under the MIT license found in the
|
||||
* LICENSE file in the root directory of this source tree.
|
||||
*/var Uh;function qy(){if(Uh)return yt;Uh=1;var i=Symbol.for("react.transitional.element"),r=Symbol.for("react.portal"),h=Symbol.for("react.fragment"),f=Symbol.for("react.strict_mode"),g=Symbol.for("react.profiler"),C=Symbol.for("react.consumer"),A=Symbol.for("react.context"),S=Symbol.for("react.forward_ref"),H=Symbol.for("react.suspense"),R=Symbol.for("react.memo"),Y=Symbol.for("react.lazy"),x=Symbol.for("react.activity"),k=Symbol.for("react.view_transition"),W=Symbol.iterator;function V(y){return y===null||typeof y!="object"?null:(y=W&&y[W]||y["@@iterator"],typeof y=="function"?y:null)}var I={isMounted:function(){return!1},enqueueForceUpdate:function(){},enqueueReplaceState:function(){},enqueueSetState:function(){}},X=Object.assign,ht={};function gt(y,D,ct){this.props=y,this.context=D,this.refs=ht,this.updater=ct||I}gt.prototype.isReactComponent={},gt.prototype.setState=function(y,D){if(typeof y!="object"&&typeof y!="function"&&y!=null)throw Error("takes an object of state variables to update or a function which returns an object of state variables.");this.updater.enqueueSetState(this,y,D,"setState")},gt.prototype.forceUpdate=function(y){this.updater.enqueueForceUpdate(this,y,"forceUpdate")};function xt(){}xt.prototype=gt.prototype;function Z(y,D,ct){this.props=y,this.context=D,this.refs=ht,this.updater=ct||I}var Tt=Z.prototype=new xt;Tt.constructor=Z,X(Tt,gt.prototype),Tt.isPureReactComponent=!0;var w=Array.isArray;function Q(){}var et={H:null,A:null,T:null,S:null},zt=Object.prototype.hasOwnProperty;function Bt(y,D,ct){var it=ct.ref;return{$$typeof:i,type:y,key:D,ref:it!==void 0?it:null,props:ct}}function vt(y,D){return Bt(y.type,D,y.props)}function wt(y){return typeof y=="object"&&y!==null&&y.$$typeof===i}function q(y){var D={"=":"=0",":":"=2"};return"$"+y.replace(/[=:]/g,function(ct){return D[ct]})}var ot=/\/+/g;function rt(y,D){return typeof y=="object"&&y!==null&&y.key!=null?q(""+y.key):D.toString(36)}function G(y){switch(y.status){case"fulfilled":return y.value;case"rejected":throw y.reason;default:switch(typeof y.status=="string"?y.then(Q,Q):(y.status="pending",y.then(function(D){y.status==="pending"&&(y.status="fulfilled",y.value=D)},function(D){y.status==="pending"&&(y.status="rejected",y.reason=D)})),y.status){case"fulfilled":return y.value;case"rejected":throw y.reason}}throw y}function N(y,D,ct,it,Nt){var _t=typeof y;(_t==="undefined"||_t==="boolean")&&(y=null);var Rt=!1;if(y===null)Rt=!0;else switch(_t){case"bigint":case"string":case"number":Rt=!0;break;case"object":switch(y.$$typeof){case i:case r:Rt=!0;break;case Y:return Rt=y._init,N(Rt(y._payload),D,ct,it,Nt)}}if(Rt)return Nt=Nt(y),Rt=it===""?"."+rt(y,0):it,w(Nt)?(ct="",Rt!=null&&(ct=Rt.replace(ot,"$&/")+"/"),N(Nt,D,ct,"",function(ce){return ce})):Nt!=null&&(wt(Nt)&&(Nt=vt(Nt,ct+(Nt.key==null||y&&y.key===Nt.key?"":(""+Nt.key).replace(ot,"$&/")+"/")+Rt)),D.push(Nt)),1;Rt=0;var P=it===""?".":it+":";if(w(y))for(var dt=0;dt<y.length;dt++)it=y[dt],_t=P+rt(it,dt),Rt+=N(it,D,ct,_t,Nt);else if(dt=V(y),typeof dt=="function")for(y=dt.call(y),dt=0;!(it=y.next()).done;)it=it.value,_t=P+rt(it,dt++),Rt+=N(it,D,ct,_t,Nt);else if(_t==="object"){if(typeof y.then=="function")return N(G(y),D,ct,it,Nt);throw D=String(y),Error("Objects are not valid as a React child (found: "+(D==="[object Object]"?"object with keys {"+Object.keys(y).join(", ")+"}":D)+"). If you meant to render a collection of children, use an array instead.")}return Rt}function ut(y,D,ct){if(y==null)return y;var it=[],Nt=0;return N(y,it,"","",function(_t){return D.call(ct,_t,Nt++)}),it}function St(y){if(y._status===-1){var D=y._result,ct=D();ct.then(function(it){(y._status===0||y._status===-1)&&(y._status=1,y._result=it,ct.status===void 0&&(ct.status="fulfilled",ct.value=it))},function(it){(y._status===0||y._status===-1)&&(y._status=2,y._result=it,ct.status===void 0&&(ct.status="rejected",ct.reason=it))}),y._status===-1&&(y._status=0,y._result=ct)}if(y._status===1)return y._result.default;throw y._result}var pt=typeof reportError=="function"?reportError:function(y){if(typeof window=="object"&&typeof window.ErrorEvent=="function"){var D=new window.ErrorEvent("error",{bubbles:!0,cancelable:!0,message:typeof y=="object"&&y!==null&&typeof y.message=="string"?String(y.message):String(y),error:y});if(!window.dispatchEvent(D))return}else if(typeof process=="object"&&typeof process.emit=="function"){process.emit("uncaughtException",y);return}console.error(y)};function ge(y){var D=et.T,ct={};ct.types=D!==null?D.types:null,et.T=ct;try{var it=y(),Nt=et.S;Nt!==null&&Nt(ct,it),typeof it=="object"&&it!==null&&typeof it.then=="function"&&it.then(Q,pt)}catch(_t){pt(_t)}finally{D!==null&&ct.types!==null&&(D.types=ct.types),et.T=D}}function we(y){var D=et.T;if(D!==null){var ct=D.types;ct===null?D.types=[y]:ct.indexOf(y)===-1&&ct.push(y)}else ge(we.bind(null,y))}var Ce={map:ut,forEach:function(y,D,ct){ut(y,function(){D.apply(this,arguments)},ct)},count:function(y){var D=0;return ut(y,function(){D++}),D},toArLine truncated
|
||||
*/var qh;function Xy(){if(qh)return pt;qh=1;var i=Symbol.for("react.transitional.element"),r=Symbol.for("react.portal"),h=Symbol.for("react.fragment"),f=Symbol.for("react.strict_mode"),p=Symbol.for("react.profiler"),N=Symbol.for("react.consumer"),C=Symbol.for("react.context"),j=Symbol.for("react.forward_ref"),z=Symbol.for("react.suspense"),R=Symbol.for("react.memo"),q=Symbol.for("react.lazy"),g=Symbol.for("react.activity"),k=Symbol.for("react.view_transition"),F=Symbol.iterator;function X(y){return y===null||typeof y!="object"?null:(y=F&&y[F]||y["@@iterator"],typeof y=="function"?y:null)}var J={isMounted:function(){return!1},enqueueForceUpdate:function(){},enqueueReplaceState:function(){},enqueueSetState:function(){}},Q=Object.assign,vt={};function St(y,D,nt){this.props=y,this.context=D,this.refs=vt,this.updater=nt||J}St.prototype.isReactComponent={},St.prototype.setState=function(y,D){if(typeof y!="object"&&typeof y!="function"&&y!=null)throw Error("takes an object of state variables to update or a function which returns an object of state variables.");this.updater.enqueueSetState(this,y,D,"setState")},St.prototype.forceUpdate=function(y){this.updater.enqueueForceUpdate(this,y,"forceUpdate")};function jt(){}jt.prototype=St.prototype;function K(y,D,nt){this.props=y,this.context=D,this.refs=vt,this.updater=nt||J}var Nt=K.prototype=new jt;Nt.constructor=K,Q(Nt,St.prototype),Nt.isPureReactComponent=!0;var w=Array.isArray;function V(){}var at={H:null,A:null,T:null,S:null},Dt=Object.prototype.hasOwnProperty;function qt(y,D,nt){var tt=nt.ref;return{$$typeof:i,type:y,key:D,ref:tt!==void 0?tt:null,props:nt}}function dt(y,D){return qt(y.type,D,y.props)}function Ut(y){return typeof y=="object"&&y!==null&&y.$$typeof===i}function H(y){var D={"=":"=0",":":"=2"};return"$"+y.replace(/[=:]/g,function(nt){return D[nt]})}var ot=/\/+/g;function yt(y,D){return typeof y=="object"&&y!==null&&y.key!=null?H(""+y.key):D.toString(36)}function G(y){switch(y.status){case"fulfilled":return y.value;case"rejected":throw y.reason;default:switch(typeof y.status=="string"?y.then(V,V):(y.status="pending",y.then(function(D){y.status==="pending"&&(y.status="fulfilled",y.value=D)},function(D){y.status==="pending"&&(y.status="rejected",y.reason=D)})),y.status){case"fulfilled":return y.value;case"rejected":throw y.reason}}throw y}function Y(y,D,nt,tt,Ct){var bt=typeof y;(bt==="undefined"||bt==="boolean")&&(y=null);var Tt=!1;if(y===null)Tt=!0;else switch(bt){case"bigint":case"string":case"number":Tt=!0;break;case"object":switch(y.$$typeof){case i:case r:Tt=!0;break;case q:return Tt=y._init,Y(Tt(y._payload),D,nt,tt,Ct)}}if(Tt)return Ct=Ct(y),Tt=tt===""?"."+yt(y,0):tt,w(Ct)?(nt="",Tt!=null&&(nt=Tt.replace(ot,"$&/")+"/"),Y(Ct,D,nt,"",function(re){return re})):Ct!=null&&(Ut(Ct)&&(Ct=dt(Ct,nt+(Ct.key==null||y&&y.key===Ct.key?"":(""+Ct.key).replace(ot,"$&/")+"/")+Tt)),D.push(Ct)),1;Tt=0;var et=tt===""?".":tt+":";if(w(y))for(var mt=0;mt<y.length;mt++)tt=y[mt],bt=et+yt(tt,mt),Tt+=Y(tt,D,nt,bt,Ct);else if(mt=X(y),typeof mt=="function")for(y=mt.call(y),mt=0;!(tt=y.next()).done;)tt=tt.value,bt=et+yt(tt,mt++),Tt+=Y(tt,D,nt,bt,Ct);else if(bt==="object"){if(typeof y.then=="function")return Y(G(y),D,nt,tt,Ct);throw D=String(y),Error("Objects are not valid as a React child (found: "+(D==="[object Object]"?"object with keys {"+Object.keys(y).join(", ")+"}":D)+"). If you meant to render a collection of children, use an array instead.")}return Tt}function st(y,D,nt){if(y==null)return y;var tt=[],Ct=0;return Y(y,tt,"","",function(bt){return D.call(nt,bt,Ct++)}),tt}function gt(y){if(y._status===-1){var D=y._result,nt=D();nt.then(function(tt){(y._status===0||y._status===-1)&&(y._status=1,y._result=tt,nt.status===void 0&&(nt.status="fulfilled",nt.value=tt))},function(tt){(y._status===0||y._status===-1)&&(y._status=2,y._result=tt,nt.status===void 0&&(nt.status="rejected",nt.reason=tt))}),y._status===-1&&(y._status=0,y._result=nt)}if(y._status===1)return y._result.default;throw y._result}var xt=typeof reportError=="function"?reportError:function(y){if(typeof window=="object"&&typeof window.ErrorEvent=="function"){var D=new window.ErrorEvent("error",{bubbles:!0,cancelable:!0,message:typeof y=="object"&&y!==null&&typeof y.message=="string"?String(y.message):String(y),error:y});if(!window.dispatchEvent(D))return}else if(typeof process=="object"&&typeof process.emit=="function"){process.emit("uncaughtException",y);return}console.error(y)};function Se(y){var D=at.T,nt={};nt.types=D!==null?D.types:null,at.T=nt;try{var tt=y(),Ct=at.S;Ct!==null&&Ct(nt,tt),typeof tt=="object"&&tt!==null&&typeof tt.then=="function"&&tt.then(V,xt)}catch(bt){xt(bt)}finally{D!==null&&nt.types!==null&&(D.types=nt.types),at.T=D}}function L(y){var D=at.T;if(D!==null){var nt=D.types;nt===null?D.types=[y]:nt.indexOf(y)===-1&&nt.push(y)}else Se(L.bind(null,y))}var se={map:st,forEach:function(y,D,nt){st(y,function(){D.apply(this,arguments)},nt)},count:function(y){var D=0;return st(y,function(){D++}),D},toALine truncated
|
||||
* @license React
|
||||
* scheduler.production.js
|
||||
*
|
||||
@@ -28,7 +28,7 @@
|
||||
*
|
||||
* This source code is licensed under the MIT license found in the
|
||||
* LICENSE file in the root directory of this source tree.
|
||||
*/var wh;function Gy(){return wh||(wh=1,(function(i){function r(G,N){var ut=G.length;G.push(N);t:for(;0<ut;){var St=ut-1>>>1,pt=G[St];if(0<g(pt,N))G[St]=N,G[ut]=pt,ut=St;else break t}}function h(G){return G.length===0?null:G[0]}function f(G){if(G.length===0)return null;var N=G[0],ut=G.pop();if(ut!==N){G[0]=ut;t:for(var St=0,pt=G.length,ge=pt>>>1;St<ge;){var we=2*(St+1)-1,Ce=G[we],y=we+1,D=G[y];if(0>g(Ce,ut))y<pt&&0>g(D,Ce)?(G[St]=D,G[y]=ut,St=y):(G[St]=Ce,G[we]=ut,St=we);else if(y<pt&&0>g(D,ut))G[St]=D,G[y]=ut,St=y;else break t}}return N}function g(G,N){var ut=G.sortIndex-N.sortIndex;return ut!==0?ut:G.id-N.id}if(i.unstable_now=void 0,typeof performance=="object"&&typeof performance.now=="function"){var C=performance;i.unstable_now=function(){return C.now()}}else{var A=Date,S=A.now();i.unstable_now=function(){return A.now()-S}}var H=[],R=[],Y=1,x=null,k=3,W=!1,V=!1,I=!1,X=!1,ht=typeof setTimeout=="function"?setTimeout:null,gt=typeof clearTimeout=="function"?clearTimeout:null,xt=typeof setImmediate<"u"?setImmediate:null;function Z(G){for(var N=h(R);N!==null;){if(N.callback===null)f(R);else if(N.startTime<=G)f(R),N.sortIndex=N.expirationTime,r(H,N);else break;N=h(R)}}function Tt(G){if(I=!1,Z(G),!V)if(h(H)!==null)V=!0,w||(w=!0,wt());else{var N=h(R);N!==null&&rt(Tt,N.startTime-G)}}var w=!1,Q=-1,et=5,zt=-1;function Bt(){return X?!0:!(i.unstable_now()-zt<et)}function vt(){if(X=!1,w){var G=i.unstable_now();zt=G;var N=!0;try{t:{V=!1,I&&(I=!1,gt(Q),Q=-1),W=!0;var ut=k;try{e:{for(Z(G),x=h(H);x!==null&&!(x.expirationTime>G&&Bt());){var St=x.callback;if(typeof St=="function"){x.callback=null,k=x.priorityLevel;var pt=St(x.expirationTime<=G);if(G=i.unstable_now(),typeof pt=="function"){x.callback=pt,Z(G),N=!0;break e}x===h(H)&&f(H),Z(G)}else f(H);x=h(H)}if(x!==null)N=!0;else{var ge=h(R);ge!==null&&rt(Tt,ge.startTime-G),N=!1}}break t}finally{x=null,k=ut,W=!1}N=void 0}}finally{N?wt():w=!1}}}var wt;if(typeof xt=="function")wt=function(){xt(vt)};else if(typeof MessageChannel<"u"){var q=new MessageChannel,ot=q.port2;q.port1.onmessage=vt,wt=function(){ot.postMessage(null)}}else wt=function(){ht(vt,0)};function rt(G,N){Q=ht(function(){G(i.unstable_now())},N)}i.unstable_IdlePriority=5,i.unstable_ImmediatePriority=1,i.unstable_LowPriority=4,i.unstable_NormalPriority=3,i.unstable_Profiling=null,i.unstable_UserBlockingPriority=2,i.unstable_cancelCallback=function(G){G.callback=null},i.unstable_forceFrameRate=function(G){0>G||125<G?console.error("forceFrameRate takes a positive int between 0 and 125, forcing frame rates higher than 125 fps is not supported"):et=0<G?Math.floor(1e3/G):5},i.unstable_getCurrentPriorityLevel=function(){return k},i.unstable_next=function(G){switch(k){case 1:case 2:case 3:var N=3;break;default:N=k}var ut=k;k=N;try{return G()}finally{k=ut}},i.unstable_requestPaint=function(){X=!0},i.unstable_runWithPriority=function(G,N){switch(G){case 1:case 2:case 3:case 4:case 5:break;default:G=3}var ut=k;k=G;try{return N()}finally{k=ut}},i.unstable_scheduleCallback=function(G,N,ut){var St=i.unstable_now();switch(typeof ut=="object"&&ut!==null?(ut=ut.delay,ut=typeof ut=="number"&&0<ut?St+ut:St):ut=St,G){case 1:var pt=-1;break;case 2:pt=250;break;case 5:pt=1073741823;break;case 4:pt=1e4;break;default:pt=5e3}return pt=ut+pt,G={id:Y++,callback:N,priorityLevel:G,startTime:ut,expirationTime:pt,sortIndex:-1},ut>St?(G.sortIndex=ut,r(R,G),h(H)===null&&G===h(R)&&(I?(gt(Q),Q=-1):I=!0,rt(Tt,ut-St))):(G.sortIndex=pt,r(H,G),V||W||(V=!0,w||(w=!0,wt()))),G},i.unstable_shouldYield=Bt,i.unstable_wrapCallback=function(G){var N=k;return function(){var ut=k;k=N;try{return G.apply(this,arguments)}finally{k=ut}}}})(so)),so}var kh;function Qy(){return kh||(kh=1,co.exports=Gy()),co.exports}var ro={exports:{}},He={};/**
|
||||
*/var Gh;function Zy(){return Gh||(Gh=1,(function(i){function r(G,Y){var st=G.length;G.push(Y);t:for(;0<st;){var gt=st-1>>>1,xt=G[gt];if(0<p(xt,Y))G[gt]=Y,G[st]=xt,st=gt;else break t}}function h(G){return G.length===0?null:G[0]}function f(G){if(G.length===0)return null;var Y=G[0],st=G.pop();if(st!==Y){G[0]=st;t:for(var gt=0,xt=G.length,Se=xt>>>1;gt<Se;){var L=2*(gt+1)-1,se=G[L],y=L+1,D=G[y];if(0>p(se,st))y<xt&&0>p(D,se)?(G[gt]=D,G[y]=st,gt=y):(G[gt]=se,G[L]=st,gt=L);else if(y<xt&&0>p(D,st))G[gt]=D,G[y]=st,gt=y;else break t}}return Y}function p(G,Y){var st=G.sortIndex-Y.sortIndex;return st!==0?st:G.id-Y.id}if(i.unstable_now=void 0,typeof performance=="object"&&typeof performance.now=="function"){var N=performance;i.unstable_now=function(){return N.now()}}else{var C=Date,j=C.now();i.unstable_now=function(){return C.now()-j}}var z=[],R=[],q=1,g=null,k=3,F=!1,X=!1,J=!1,Q=!1,vt=typeof setTimeout=="function"?setTimeout:null,St=typeof clearTimeout=="function"?clearTimeout:null,jt=typeof setImmediate<"u"?setImmediate:null;function K(G){for(var Y=h(R);Y!==null;){if(Y.callback===null)f(R);else if(Y.startTime<=G)f(R),Y.sortIndex=Y.expirationTime,r(z,Y);else break;Y=h(R)}}function Nt(G){if(J=!1,K(G),!X)if(h(z)!==null)X=!0,w||(w=!0,Ut());else{var Y=h(R);Y!==null&&yt(Nt,Y.startTime-G)}}var w=!1,V=-1,at=5,Dt=-1;function qt(){return Q?!0:!(i.unstable_now()-Dt<at)}function dt(){if(Q=!1,w){var G=i.unstable_now();Dt=G;var Y=!0;try{t:{X=!1,J&&(J=!1,St(V),V=-1),F=!0;var st=k;try{e:{for(K(G),g=h(z);g!==null&&!(g.expirationTime>G&&qt());){var gt=g.callback;if(typeof gt=="function"){g.callback=null,k=g.priorityLevel;var xt=gt(g.expirationTime<=G);if(G=i.unstable_now(),typeof xt=="function"){g.callback=xt,K(G),Y=!0;break e}g===h(z)&&f(z),K(G)}else f(z);g=h(z)}if(g!==null)Y=!0;else{var Se=h(R);Se!==null&&yt(Nt,Se.startTime-G),Y=!1}}break t}finally{g=null,k=st,F=!1}Y=void 0}}finally{Y?Ut():w=!1}}}var Ut;if(typeof jt=="function")Ut=function(){jt(dt)};else if(typeof MessageChannel<"u"){var H=new MessageChannel,ot=H.port2;H.port1.onmessage=dt,Ut=function(){ot.postMessage(null)}}else Ut=function(){vt(dt,0)};function yt(G,Y){V=vt(function(){G(i.unstable_now())},Y)}i.unstable_IdlePriority=5,i.unstable_ImmediatePriority=1,i.unstable_LowPriority=4,i.unstable_NormalPriority=3,i.unstable_Profiling=null,i.unstable_UserBlockingPriority=2,i.unstable_cancelCallback=function(G){G.callback=null},i.unstable_forceFrameRate=function(G){0>G||125<G?console.error("forceFrameRate takes a positive int between 0 and 125, forcing frame rates higher than 125 fps is not supported"):at=0<G?Math.floor(1e3/G):5},i.unstable_getCurrentPriorityLevel=function(){return k},i.unstable_next=function(G){switch(k){case 1:case 2:case 3:var Y=3;break;default:Y=k}var st=k;k=Y;try{return G()}finally{k=st}},i.unstable_requestPaint=function(){Q=!0},i.unstable_runWithPriority=function(G,Y){switch(G){case 1:case 2:case 3:case 4:case 5:break;default:G=3}var st=k;k=G;try{return Y()}finally{k=st}},i.unstable_scheduleCallback=function(G,Y,st){var gt=i.unstable_now();switch(typeof st=="object"&&st!==null?(st=st.delay,st=typeof st=="number"&&0<st?gt+st:gt):st=gt,G){case 1:var xt=-1;break;case 2:xt=250;break;case 5:xt=1073741823;break;case 4:xt=1e4;break;default:xt=5e3}return xt=st+xt,G={id:q++,callback:Y,priorityLevel:G,startTime:st,expirationTime:xt,sortIndex:-1},st>gt?(G.sortIndex=st,r(R,G),h(z)===null&&G===h(R)&&(J?(St(V),V=-1):J=!0,yt(Nt,st-gt))):(G.sortIndex=xt,r(z,G),X||F||(X=!0,w||(w=!0,Ut()))),G},i.unstable_shouldYield=qt,i.unstable_wrapCallback=function(G){var Y=k;return function(){var st=k;k=Y;try{return G.apply(this,arguments)}finally{k=st}}}})(fo)),fo}var Vh;function Ky(){return Vh||(Vh=1,oo.exports=Zy()),oo.exports}var mo={exports:{}},ke={};/**
|
||||
* @license React
|
||||
* react-dom.production.js
|
||||
*
|
||||
@@ -36,7 +36,7 @@
|
||||
*
|
||||
* This source code is licensed under the MIT license found in the
|
||||
* LICENSE file in the root directory of this source tree.
|
||||
*/var Bh;function Xy(){if(Bh)return He;Bh=1;var i=yo();function r(Y){var x="https://react.dev/errors/"+Y;if(1<arguments.length){x+="?args[]="+encodeURIComponent(arguments[1]);for(var k=2;k<arguments.length;k++)x+="&args[]="+encodeURIComponent(arguments[k])}return"Minified React error #"+Y+"; visit "+x+" for the full message or use the non-minified dev environment for full errors and additional helpful warnings."}function h(){}var f={d:{f:h,r:function(){throw Error(r(522))},D:h,C:h,L:h,m:h,X:h,S:h,M:h},p:0,findDOMNode:null},g=Symbol.for("react.portal"),C=Symbol.for("react.recoverable"),A=Symbol.for("react.optimistic_key");function S(Y,x,k){var W=3<arguments.length&&arguments[3]!==void 0?arguments[3]:null;return{$$typeof:g,key:W==null?null:W===A?A:""+W,children:Y,containerInfo:x,implementation:k}}var H=i.__CLIENT_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE;function R(Y,x){if(Y==="font")return"";if(typeof x=="string")return x==="use-credentials"?x:""}return He.__DOM_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE=f,He.browser=function(Y){return{$$typeof:C,_reason:Y}},He.createPortal=function(Y,x){var k=2<arguments.length&&arguments[2]!==void 0?arguments[2]:null;if(!x||x.nodeType!==1&&x.nodeType!==9&&x.nodeType!==11)throw Error(r(299));return S(Y,x,null,k)},He.flushSync=function(Y){var x=H.T,k=f.p;try{if(H.T=null,f.p=2,Y)return Y()}finally{H.T=x,f.p=k,f.d.f()}},He.preconnect=function(Y,x){typeof Y=="string"&&(x?(x=x.crossOrigin,x=typeof x=="string"?x==="use-credentials"?x:"":void 0):x=null,f.d.C(Y,x))},He.prefetchDNS=function(Y){typeof Y=="string"&&f.d.D(Y)},He.preinit=function(Y,x){if(typeof Y=="string"&&x&&typeof x.as=="string"){var k=x.as,W=R(k,x.crossOrigin),V=typeof x.integrity=="string"?x.integrity:void 0,I=typeof x.fetchPriority=="string"?x.fetchPriority:void 0;k==="style"?f.d.S(Y,typeof x.precedence=="string"?x.precedence:void 0,{crossOrigin:W,integrity:V,fetchPriority:I}):k==="script"&&f.d.X(Y,{crossOrigin:W,integrity:V,fetchPriority:I,nonce:typeof x.nonce=="string"?x.nonce:void 0})}},He.preinitModule=function(Y,x){if(typeof Y=="string")if(typeof x=="object"&&x!==null){if(x.as==null||x.as==="script"){var k=R(x.as,x.crossOrigin);f.d.M(Y,{crossOrigin:k,integrity:typeof x.integrity=="string"?x.integrity:void 0,nonce:typeof x.nonce=="string"?x.nonce:void 0,fetchPriority:typeof x.fetchPriority=="string"?x.fetchPriority:void 0})}}else x==null&&f.d.M(Y)},He.preload=function(Y,x){if(typeof Y=="string"&&typeof x=="object"&&x!==null&&typeof x.as=="string"){var k=x.as,W=R(k,x.crossOrigin);f.d.L(Y,k,{crossOrigin:W,integrity:typeof x.integrity=="string"?x.integrity:void 0,nonce:typeof x.nonce=="string"?x.nonce:void 0,type:typeof x.type=="string"?x.type:void 0,fetchPriority:typeof x.fetchPriority=="string"?x.fetchPriority:void 0,referrerPolicy:typeof x.referrerPolicy=="string"?x.referrerPolicy:void 0,imageSrcSet:typeof x.imageSrcSet=="string"?x.imageSrcSet:void 0,imageSizes:typeof x.imageSizes=="string"?x.imageSizes:void 0,media:typeof x.media=="string"?x.media:void 0})}},He.preloadModule=function(Y,x){if(typeof Y=="string")if(x){var k=R(x.as,x.crossOrigin);f.d.m(Y,{as:typeof x.as=="string"&&x.as!=="script"?x.as:void 0,crossOrigin:k,integrity:typeof x.integrity=="string"?x.integrity:void 0,nonce:typeof x.nonce=="string"?x.nonce:void 0,fetchPriority:typeof x.fetchPriority=="string"?x.fetchPriority:void 0})}else f.d.m(Y)},He.requestFormReset=function(Y){f.d.r(Y)},He.unstable_batchedUpdates=function(Y,x){return Y(x)},He.useFormState=function(Y,x,k){return H.H.useFormState(Y,x,k)},He.useFormStatus=function(){return H.H.useHostTransitionStatus()},He.version="19.3.1",He}var qh;function Ly(){if(qh)return ro.exports;qh=1;function i(){if(!(typeof __REACT_DEVTOOLS_GLOBAL_HOOK__>"u"||typeof __REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE!="function"))try{__REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE(i)}catch(r){console.error(r)}}return i(),ro.exports=Xy(),ro.exports}/**
|
||||
*/var Qh;function Jy(){if(Qh)return ke;Qh=1;var i=So();function r(q){var g="https://react.dev/errors/"+q;if(1<arguments.length){g+="?args[]="+encodeURIComponent(arguments[1]);for(var k=2;k<arguments.length;k++)g+="&args[]="+encodeURIComponent(arguments[k])}return"Minified React error #"+q+"; visit "+g+" for the full message or use the non-minified dev environment for full errors and additional helpful warnings."}function h(){}var f={d:{f:h,r:function(){throw Error(r(522))},D:h,C:h,L:h,m:h,X:h,S:h,M:h},p:0,findDOMNode:null},p=Symbol.for("react.portal"),N=Symbol.for("react.recoverable"),C=Symbol.for("react.optimistic_key");function j(q,g,k){var F=3<arguments.length&&arguments[3]!==void 0?arguments[3]:null;return{$$typeof:p,key:F==null?null:F===C?C:""+F,children:q,containerInfo:g,implementation:k}}var z=i.__CLIENT_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE;function R(q,g){if(q==="font")return"";if(typeof g=="string")return g==="use-credentials"?g:""}return ke.__DOM_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE=f,ke.browser=function(q){return{$$typeof:N,_reason:q}},ke.createPortal=function(q,g){var k=2<arguments.length&&arguments[2]!==void 0?arguments[2]:null;if(!g||g.nodeType!==1&&g.nodeType!==9&&g.nodeType!==11)throw Error(r(299));return j(q,g,null,k)},ke.flushSync=function(q){var g=z.T,k=f.p;try{if(z.T=null,f.p=2,q)return q()}finally{z.T=g,f.p=k,f.d.f()}},ke.preconnect=function(q,g){typeof q=="string"&&(g?(g=g.crossOrigin,g=typeof g=="string"?g==="use-credentials"?g:"":void 0):g=null,f.d.C(q,g))},ke.prefetchDNS=function(q){typeof q=="string"&&f.d.D(q)},ke.preinit=function(q,g){if(typeof q=="string"&&g&&typeof g.as=="string"){var k=g.as,F=R(k,g.crossOrigin),X=typeof g.integrity=="string"?g.integrity:void 0,J=typeof g.fetchPriority=="string"?g.fetchPriority:void 0;k==="style"?f.d.S(q,typeof g.precedence=="string"?g.precedence:void 0,{crossOrigin:F,integrity:X,fetchPriority:J}):k==="script"&&f.d.X(q,{crossOrigin:F,integrity:X,fetchPriority:J,nonce:typeof g.nonce=="string"?g.nonce:void 0})}},ke.preinitModule=function(q,g){if(typeof q=="string")if(typeof g=="object"&&g!==null){if(g.as==null||g.as==="script"){var k=R(g.as,g.crossOrigin);f.d.M(q,{crossOrigin:k,integrity:typeof g.integrity=="string"?g.integrity:void 0,nonce:typeof g.nonce=="string"?g.nonce:void 0,fetchPriority:typeof g.fetchPriority=="string"?g.fetchPriority:void 0})}}else g==null&&f.d.M(q)},ke.preload=function(q,g){if(typeof q=="string"&&typeof g=="object"&&g!==null&&typeof g.as=="string"){var k=g.as,F=R(k,g.crossOrigin);f.d.L(q,k,{crossOrigin:F,integrity:typeof g.integrity=="string"?g.integrity:void 0,nonce:typeof g.nonce=="string"?g.nonce:void 0,type:typeof g.type=="string"?g.type:void 0,fetchPriority:typeof g.fetchPriority=="string"?g.fetchPriority:void 0,referrerPolicy:typeof g.referrerPolicy=="string"?g.referrerPolicy:void 0,imageSrcSet:typeof g.imageSrcSet=="string"?g.imageSrcSet:void 0,imageSizes:typeof g.imageSizes=="string"?g.imageSizes:void 0,media:typeof g.media=="string"?g.media:void 0})}},ke.preloadModule=function(q,g){if(typeof q=="string")if(g){var k=R(g.as,g.crossOrigin);f.d.m(q,{as:typeof g.as=="string"&&g.as!=="script"?g.as:void 0,crossOrigin:k,integrity:typeof g.integrity=="string"?g.integrity:void 0,nonce:typeof g.nonce=="string"?g.nonce:void 0,fetchPriority:typeof g.fetchPriority=="string"?g.fetchPriority:void 0})}else f.d.m(q)},ke.requestFormReset=function(q){f.d.r(q)},ke.unstable_batchedUpdates=function(q,g){return q(g)},ke.useFormState=function(q,g,k){return z.H.useFormState(q,g,k)},ke.useFormStatus=function(){return z.H.useHostTransitionStatus()},ke.version="19.3.1",ke}var Xh;function Fy(){if(Xh)return mo.exports;Xh=1;function i(){if(!(typeof __REACT_DEVTOOLS_GLOBAL_HOOK__>"u"||typeof __REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE!="function"))try{__REACT_DEVTOOLS_GLOBAL_HOOK__.checkDCE(i)}catch(r){console.error(r)}}return i(),mo.exports=Jy(),mo.exports}/**
|
||||
* @license React
|
||||
* react-dom-client.production.js
|
||||
*
|
||||
@@ -44,37 +44,37 @@
|
||||
*
|
||||
* This source code is licensed under the MIT license found in the
|
||||
* LICENSE file in the root directory of this source tree.
|
||||
*/var Yh;function Vy(){if(Yh)return ii;Yh=1;var i=Qy(),r=yo(),h=Ly();function f(t){var e="https://react.dev/errors/"+t;if(1<arguments.length){e+="?args[]="+encodeURIComponent(arguments[1]);for(var l=2;l<arguments.length;l++)e+="&args[]="+encodeURIComponent(arguments[l])}return"Minified React error #"+t+"; visit "+e+" for the full message or use the non-minified dev environment for full errors and additional helpful warnings."}function g(t){return!(!t||t.nodeType!==1&&t.nodeType!==9&&t.nodeType!==11)}function C(t){for(var e=t,l=e;l&&!l.alternate;)e=l,(e.flags&4098)!==0&&(t=e.return),l=e.return;for(;e.return;)e=e.return;return e.tag===3?t:null}function A(t){if(t.tag===13){var e=t.memoizedState;if(e===null&&(t=t.alternate,t!==null&&(e=t.memoizedState)),e!==null)return e.dehydrated}return null}function S(t){if(t.tag===31){var e=t.memoizedState;if(e===null&&(t=t.alternate,t!==null&&(e=t.memoizedState)),e!==null)return e.dehydrated}return null}function H(t){if(C(t)!==t)throw Error(f(188))}function R(t){var e=t.alternate;if(!e){if(e=C(t),e===null)throw Error(f(188));return e!==t?null:t}for(var l=t,a=e;;){var n=l.return;if(n===null)break;var u=n.alternate;if(u===null){if(a=n.return,a!==null){l=a;continue}break}if(n.child===u.child){for(u=n.child;u;){if(u===l)return H(n),t;if(u===a)return H(n),e;u=u.sibling}throw Error(f(188))}if(l.return!==a.return)l=n,a=u;else{for(var s=!1,o=n.child;o;){if(o===l){s=!0,l=n,a=u;break}if(o===a){s=!0,a=n,l=u;break}o=o.sibling}if(!s){for(o=u.child;o;){if(o===l){s=!0,l=u,a=n;break}if(o===a){s=!0,a=u,l=n;break}o=o.sibling}if(!s)throw Error(f(189))}}if(l.alternate!==a)throw Error(f(190))}if(l.tag!==3)throw Error(f(188));return l.stateNode.current===l?t:e}function Y(t){var e=t.tag;if(e===5||e===26||e===27||e===6)return t;for(t=t.child;t!==null;){if(e=Y(t),e!==null)return e;t=t.sibling}return null}function x(t,e,l,a,n,u){for(;t!==null;){if((t.tag===5||t.tag===27||t.tag===6)&&l(t,a,n,u)||(t.tag!==22||t.memoizedState===null)&&(e||t.tag!==5&&t.tag!==27)&&x(t.child,e,l,a,n,u))return!0;t=t.sibling}return!1}function k(t){for(t=t.return;t!==null;){if(t.tag===3||t.tag===5||t.tag===27)return t;t=t.return}return null}function W(t){var e=!1;for(t=t.return;t!==null&&(t.tag===4&&(e=!0),!(t.tag===3||t.tag===5||t.tag===27));)t=t.return;return e}function V(t){var e=[null,null],l=k(t);return l===null||I(e,t,l.child,{foundSelf:!1}),e}function I(t,e,l,a){for(;l!==null;){if(l===e)a.foundSelf=!0;else if(l.tag===5||l.tag===27||l.tag===6){if(a.foundSelf)return t[1]=l,!0;t[0]=l}else if((l.tag!==22||l.memoizedState===null)&&I(t,e,l.child,a))return!0;l=l.sibling}return!1}function X(t){switch(t.tag){case 5:case 27:case 6:return t.stateNode;case 3:return t.stateNode.containerInfo;default:throw Error(f(559))}}var ht=null,gt=null;function xt(t,e,l){return t===l?!0:t===e?(ht=t,!0):!1}function Z(t,e,l){return t===l?(gt=t,!1):t===e?(gt!==null&&(ht=t),!0):!1}function Tt(t){if(t===null)return null;do t=t===null?null:t.return;while(t&&t.tag!==5&&t.tag!==27&&t.tag!==3);return t||null}function w(t,e,l){for(var a=0,n=t;n;n=l(n))a++;n=0;for(var u=e;u;u=l(u))n++;for(;0<a-n;)t=l(t),a--;for(;0<n-a;)e=l(e),n--;for(;a--;){if(t===e||e!==null&&t===e.alternate)return t;t=l(t),e=l(e)}return null}var Q=Object.assign,et=Symbol.for("react.element"),zt=Symbol.for("react.transitional.element"),Bt=Symbol.for("react.portal"),vt=Symbol.for("react.fragment"),wt=Symbol.for("react.strict_mode"),q=Symbol.for("react.profiler"),ot=Symbol.for("react.consumer"),rt=Symbol.for("react.context"),G=Symbol.for("react.forward_ref"),N=Symbol.for("react.suspense"),ut=Symbol.for("react.suspense_list"),St=Symbol.for("react.memo"),pt=Symbol.for("react.lazy"),ge=Symbol.for("react.activity"),we=Symbol.for("react.legacy_hidden"),Ce=Symbol.for("react.memo_cache_sentinel"),y=Symbol.for("react.view_transition"),D=Symbol.for("react.recoverable"),ct=Symbol.iterator;function it(t){return t===null||typeof t!="object"?null:(t=ct&&t[ct]||t["@@iterator"],typeof t=="function"?t:null)}var Nt=Symbol.for("react.client.reference");function _t(t){if(t==null)return null;if(typeof t=="function")return t.$$typeof===Nt?null:t.displayName||t.name||null;if(typeof t=="string")return t;switch(t){case vt:return"Fragment";case q:return"Profiler";case wt:return"StrictMode";case N:return"Suspense";case ut:return"SuspenseList";case ge:return"Activity";case y:return"ViewTransition"}if(typeof t=="object")switch(t.$$typeof){case Bt:return"Portal";case rt:return t.displayName||"Context";case ot:return(t._context.displayName||"Context")+".Consumer";case G:var e=t.render;return t=t.displayName,t||(t=e.displayName||e.name||"",t=t!==""?"ForwardRef("+t+")":"ForwardRef"),t;case St:return e=t.displayName||null,e!==null?e:_t(t.type)||"Memo";case pt:e=t._payload,t=t._init;try{return _t(t(e))}catch{}}return null}var Rt=Array.isArray,P=r.__CLIENT_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE,dt=h.__DOM_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE,ce={pending:!1,data:null,method:null,action:nLine truncated
|
||||
*/var Lh;function Wy(){if(Lh)return ii;Lh=1;var i=Ky(),r=So(),h=Fy();function f(t){var e="https://react.dev/errors/"+t;if(1<arguments.length){e+="?args[]="+encodeURIComponent(arguments[1]);for(var l=2;l<arguments.length;l++)e+="&args[]="+encodeURIComponent(arguments[l])}return"Minified React error #"+t+"; visit "+e+" for the full message or use the non-minified dev environment for full errors and additional helpful warnings."}function p(t){return!(!t||t.nodeType!==1&&t.nodeType!==9&&t.nodeType!==11)}function N(t){for(var e=t,l=e;l&&!l.alternate;)e=l,(e.flags&4098)!==0&&(t=e.return),l=e.return;for(;e.return;)e=e.return;return e.tag===3?t:null}function C(t){if(t.tag===13){var e=t.memoizedState;if(e===null&&(t=t.alternate,t!==null&&(e=t.memoizedState)),e!==null)return e.dehydrated}return null}function j(t){if(t.tag===31){var e=t.memoizedState;if(e===null&&(t=t.alternate,t!==null&&(e=t.memoizedState)),e!==null)return e.dehydrated}return null}function z(t){if(N(t)!==t)throw Error(f(188))}function R(t){var e=t.alternate;if(!e){if(e=N(t),e===null)throw Error(f(188));return e!==t?null:t}for(var l=t,a=e;;){var n=l.return;if(n===null)break;var u=n.alternate;if(u===null){if(a=n.return,a!==null){l=a;continue}break}if(n.child===u.child){for(u=n.child;u;){if(u===l)return z(n),t;if(u===a)return z(n),e;u=u.sibling}throw Error(f(188))}if(l.return!==a.return)l=n,a=u;else{for(var s=!1,o=n.child;o;){if(o===l){s=!0,l=n,a=u;break}if(o===a){s=!0,a=n,l=u;break}o=o.sibling}if(!s){for(o=u.child;o;){if(o===l){s=!0,l=u,a=n;break}if(o===a){s=!0,a=u,l=n;break}o=o.sibling}if(!s)throw Error(f(189))}}if(l.alternate!==a)throw Error(f(190))}if(l.tag!==3)throw Error(f(188));return l.stateNode.current===l?t:e}function q(t){var e=t.tag;if(e===5||e===26||e===27||e===6)return t;for(t=t.child;t!==null;){if(e=q(t),e!==null)return e;t=t.sibling}return null}function g(t,e,l,a,n,u){for(;t!==null;){if((t.tag===5||t.tag===27||t.tag===6)&&l(t,a,n,u)||(t.tag!==22||t.memoizedState===null)&&(e||t.tag!==5&&t.tag!==27)&&g(t.child,e,l,a,n,u))return!0;t=t.sibling}return!1}function k(t){for(t=t.return;t!==null;){if(t.tag===3||t.tag===5||t.tag===27)return t;t=t.return}return null}function F(t){var e=!1;for(t=t.return;t!==null&&(t.tag===4&&(e=!0),!(t.tag===3||t.tag===5||t.tag===27));)t=t.return;return e}function X(t){var e=[null,null],l=k(t);return l===null||J(e,t,l.child,{foundSelf:!1}),e}function J(t,e,l,a){for(;l!==null;){if(l===e)a.foundSelf=!0;else if(l.tag===5||l.tag===27||l.tag===6){if(a.foundSelf)return t[1]=l,!0;t[0]=l}else if((l.tag!==22||l.memoizedState===null)&&J(t,e,l.child,a))return!0;l=l.sibling}return!1}function Q(t){switch(t.tag){case 5:case 27:case 6:return t.stateNode;case 3:return t.stateNode.containerInfo;default:throw Error(f(559))}}var vt=null,St=null;function jt(t,e,l){return t===l?!0:t===e?(vt=t,!0):!1}function K(t,e,l){return t===l?(St=t,!1):t===e?(St!==null&&(vt=t),!0):!1}function Nt(t){if(t===null)return null;do t=t===null?null:t.return;while(t&&t.tag!==5&&t.tag!==27&&t.tag!==3);return t||null}function w(t,e,l){for(var a=0,n=t;n;n=l(n))a++;n=0;for(var u=e;u;u=l(u))n++;for(;0<a-n;)t=l(t),a--;for(;0<n-a;)e=l(e),n--;for(;a--;){if(t===e||e!==null&&t===e.alternate)return t;t=l(t),e=l(e)}return null}var V=Object.assign,at=Symbol.for("react.element"),Dt=Symbol.for("react.transitional.element"),qt=Symbol.for("react.portal"),dt=Symbol.for("react.fragment"),Ut=Symbol.for("react.strict_mode"),H=Symbol.for("react.profiler"),ot=Symbol.for("react.consumer"),yt=Symbol.for("react.context"),G=Symbol.for("react.forward_ref"),Y=Symbol.for("react.suspense"),st=Symbol.for("react.suspense_list"),gt=Symbol.for("react.memo"),xt=Symbol.for("react.lazy"),Se=Symbol.for("react.activity"),L=Symbol.for("react.legacy_hidden"),se=Symbol.for("react.memo_cache_sentinel"),y=Symbol.for("react.view_transition"),D=Symbol.for("react.recoverable"),nt=Symbol.iterator;function tt(t){return t===null||typeof t!="object"?null:(t=nt&&t[nt]||t["@@iterator"],typeof t=="function"?t:null)}var Ct=Symbol.for("react.client.reference");function bt(t){if(t==null)return null;if(typeof t=="function")return t.$$typeof===Ct?null:t.displayName||t.name||null;if(typeof t=="string")return t;switch(t){case dt:return"Fragment";case H:return"Profiler";case Ut:return"StrictMode";case Y:return"Suspense";case st:return"SuspenseList";case Se:return"Activity";case y:return"ViewTransition"}if(typeof t=="object")switch(t.$$typeof){case qt:return"Portal";case yt:return t.displayName||"Context";case ot:return(t._context.displayName||"Context")+".Consumer";case G:var e=t.render;return t=t.displayName,t||(t=e.displayName||e.name||"",t=t!==""?"ForwardRef("+t+")":"ForwardRef"),t;case gt:return e=t.displayName||null,e!==null?e:bt(t.type)||"Memo";case xt:e=t._payload,t=t._init;try{return bt(t(e))}catch{}}return null}var Tt=Array.isArray,et=r.__CLIENT_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE,mt=h.__DOM_INTERNALS_DO_NOT_USE_OR_WARN_USERS_THEY_CANNOT_UPGRADE,re={pending:!1,data:null,method:null,action:nLine truncated
|
||||
at`)?" (<anonymous>)":-1<l.stack.indexOf("@")?"@unknown:0:0":""}return`
|
||||
`+Be+t+se}var rl=!1;function Fl(t,e){if(!t||rl)return"";rl=!0;var l=Error.prepareStackTrace;Error.prepareStackTrace=void 0;try{var a={DetermineComponentFrameRoot:function(){try{if(e){var z=function(){throw Error()};if(Object.defineProperty(z.prototype,"props",{set:function(){throw Error()}}),typeof Reflect=="object"&&Reflect.construct){try{Reflect.construct(z,[])}catch(L){var p=L}Reflect.construct(t,[],z)}else{try{z.call()}catch(L){p=L}z=!1;try{var _=Object.getOwnPropertyDescriptor(t.prototype,"props");Object.defineProperty(t.prototype,"props",{configurable:!0,set:function(){throw Error()}}),z=!0,new t}finally{z&&(_!==void 0?Object.defineProperty(t.prototype,"props",_):delete t.prototype.props)}}}else{try{throw Error()}catch(L){p=L}(z=t())&&typeof z.catch=="function"&&z.catch(function(){})}}catch(L){if(L&&p&&typeof L.stack=="string")return[L.stack,p.stack]}return[null,null]}};a.DetermineComponentFrameRoot.displayName="DetermineComponentFrameRoot";var n=Object.getOwnPropertyDescriptor(a.DetermineComponentFrameRoot,"name");n&&n.configurable&&Object.defineProperty(a.DetermineComponentFrameRoot,"name",{value:"DetermineComponentFrameRoot"});var u=a.DetermineComponentFrameRoot(),s=u[0],o=u[1];if(s&&o){var m=s.split(`
|
||||
`),j=o.split(`
|
||||
`);for(n=a=0;a<m.length&&!m[a].includes("DetermineComponentFrameRoot");)a++;for(;n<j.length&&!j[n].includes("DetermineComponentFrameRoot");)n++;if(a===m.length||n===j.length)for(a=m.length-1,n=j.length-1;1<=a&&0<=n&&m[a]!==j[n];)n--;for(;1<=a&&0<=n;a--,n--)if(m[a]!==j[n]){if(a!==1||n!==1)do if(a--,n--,0>n||m[a]!==j[n]){var E=`
|
||||
`+m[a].replace(" at new "," at ");return t.displayName&&E.includes("<anonymous>")&&(E=E.replace("<anonymous>",t.displayName)),E}while(1<=a&&0<=n);break}}}finally{rl=!1,Error.prepareStackTrace=l}return(l=t?t.displayName||t.name:"")?re(l):""}function ma(t,e){switch(t.tag){case 26:case 27:case 5:return re(t.type);case 16:return re("Lazy");case 13:return t.child!==e&&e!==null?re("Suspense Fallback"):re("Suspense");case 19:return re("SuspenseList");case 0:case 15:return Fl(t.type,!1);case 11:return Fl(t.type.render,!1);case 1:return Fl(t.type,!0);case 31:return re("Activity");case 30:return re("ViewTransition");default:return""}}function Dl(t){try{var e="",l=null;do e+=ma(t,l),l=t,t=t.return;while(t);return e}catch(a){return`
|
||||
`+Be+t+oe}var rl=!1;function Fl(t,e){if(!t||rl)return"";rl=!0;var l=Error.prepareStackTrace;Error.prepareStackTrace=void 0;try{var a={DetermineComponentFrameRoot:function(){try{if(e){var O=function(){throw Error()};if(Object.defineProperty(O.prototype,"props",{set:function(){throw Error()}}),typeof Reflect=="object"&&Reflect.construct){try{Reflect.construct(O,[])}catch(Z){var S=Z}Reflect.construct(t,[],O)}else{try{O.call()}catch(Z){S=Z}O=!1;try{var _=Object.getOwnPropertyDescriptor(t.prototype,"props");Object.defineProperty(t.prototype,"props",{configurable:!0,set:function(){throw Error()}}),O=!0,new t}finally{O&&(_!==void 0?Object.defineProperty(t.prototype,"props",_):delete t.prototype.props)}}}else{try{throw Error()}catch(Z){S=Z}(O=t())&&typeof O.catch=="function"&&O.catch(function(){})}}catch(Z){if(Z&&S&&typeof Z.stack=="string")return[Z.stack,S.stack]}return[null,null]}};a.DetermineComponentFrameRoot.displayName="DetermineComponentFrameRoot";var n=Object.getOwnPropertyDescriptor(a.DetermineComponentFrameRoot,"name");n&&n.configurable&&Object.defineProperty(a.DetermineComponentFrameRoot,"name",{value:"DetermineComponentFrameRoot"});var u=a.DetermineComponentFrameRoot(),s=u[0],o=u[1];if(s&&o){var m=s.split(`
|
||||
`),b=o.split(`
|
||||
`);for(n=a=0;a<m.length&&!m[a].includes("DetermineComponentFrameRoot");)a++;for(;n<b.length&&!b[n].includes("DetermineComponentFrameRoot");)n++;if(a===m.length||n===b.length)for(a=m.length-1,n=b.length-1;1<=a&&0<=n&&m[a]!==b[n];)n--;for(;1<=a&&0<=n;a--,n--)if(m[a]!==b[n]){if(a!==1||n!==1)do if(a--,n--,0>n||m[a]!==b[n]){var E=`
|
||||
`+m[a].replace(" at new "," at ");return t.displayName&&E.includes("<anonymous>")&&(E=E.replace("<anonymous>",t.displayName)),E}while(1<=a&&0<=n);break}}}finally{rl=!1,Error.prepareStackTrace=l}return(l=t?t.displayName||t.name:"")?fe(l):""}function ma(t,e){switch(t.tag){case 26:case 27:case 5:return fe(t.type);case 16:return fe("Lazy");case 13:return t.child!==e&&e!==null?fe("Suspense Fallback"):fe("Suspense");case 19:return fe("SuspenseList");case 0:case 15:return Fl(t.type,!1);case 11:return Fl(t.type.render,!1);case 1:return Fl(t.type,!0);case 31:return fe("Activity");case 30:return fe("ViewTransition");default:return""}}function Dl(t){try{var e="",l=null;do e+=ma(t,l),l=t,t=t.return;while(t);return e}catch(a){return`
|
||||
Error generating stack: `+a.message+`
|
||||
`+a.stack}}var te=Object.prototype.hasOwnProperty,Pe=i.unstable_scheduleCallback,qe=i.unstable_cancelCallback,_l=i.unstable_shouldYield,ol=i.unstable_requestPaint,Wt=i.unstable_now,Ul=i.unstable_getCurrentPriorityLevel,mn=i.unstable_ImmediatePriority,Qa=i.unstable_UserBlockingPriority,Se=i.unstable_NormalPriority,Xa=i.unstable_LowPriority,ha=i.unstable_IdlePriority,su=i.log,ru=i.unstable_setDisableYieldValue,Wl=null,Ae=null;function fl(t){if(typeof su=="function"&&ru(t),Ae&&typeof Ae.setStrictMode=="function")try{Ae.setStrictMode(Wl,t)}catch{}}var Oe=Math.clz32?Math.clz32:du,ou=Math.log,fu=Math.LN2;function du(t){return t>>>=0,t===0?32:31-(ou(t)/fu|0)|0}var B=256,U=262144,$=4194304;function at(t){var e=t&42;if(e!==0)return e;switch(t&-t){case 1:return 1;case 2:return 2;case 4:return 4;case 8:return 8;case 16:return 16;case 32:return 32;case 64:return 64;case 128:return 128;case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:return t&-t;case 262144:case 524288:case 1048576:case 2097152:return t&3932160;case 4194304:case 8388608:case 16777216:case 33554432:return t&62914560;case 67108864:return 67108864;case 134217728:return 134217728;case 268435456:return 268435456;case 536870912:return 536870912;case 1073741824:return 0;default:return t}}function O(t,e,l){var a=t.pendingLanes;if(a===0)return 0;var n=0,u=t.suspendedLanes,s=t.pingedLanes;t=t.warmLanes;var o=a&134217727;return o!==0?(a=o&~u,a!==0?n=at(a):(s&=o,s!==0?n=at(s):l||(l=o&~t,l!==0&&(n=at(l))))):(o=a&~u,o!==0?n=at(o):s!==0?n=at(s):l||(l=a&~t,l!==0&&(n=at(l)))),n===0?0:e!==0&&e!==n&&(e&u)===0&&(u=n&-n,l=e&-e,u>=l||u===32&&(l&4194048)!==0)?e:n}function J(t,e){return(t.pendingLanes&~(t.suspendedLanes&~t.pingedLanes)&e)===0}function nt(t,e){(e&8)!==0&&(e|=e&32);var l=t.entangledLanes;if(l!==0)for(t=t.entanglements,l&=e;0<l;){var a=31-Oe(l),n=1<<a;e|=t[a],l&=~n}return e}function qt(t,e){switch(t){case 1:case 2:case 4:case 8:case 64:return e+250;case 16:case 32:case 128:case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:case 262144:case 524288:case 1048576:case 2097152:return e+5e3;case 4194304:case 8388608:case 16777216:case 33554432:return-1;case 67108864:case 134217728:case 268435456:case 536870912:case 1073741824:return-1;default:return-1}}function Dt(){var t=$;return $<<=1,($&62914560)===0&&($=4194304),t}function xe(t){for(var e=[],l=0;31>l;l++)e.push(t);return e}function be(t,e){t.pendingLanes|=e,e!==268435456&&(t.suspendedLanes=0,t.pingedLanes=0,t.warmLanes=0)}function Dc(t,e,l,a,n,u){var s=t.pendingLanes;t.pendingLanes=l,t.suspendedLanes=0,t.pingedLanes=0,t.warmLanes=0,t.expiredLanes&=l,t.entangledLanes&=l,t.errorRecoveryDisabledLanes&=l,t.shellSuspendCounter=0;var o=t.entanglements,m=t.expirationTimes,j=t.hiddenUpdates;for(l=s&~l;0<l;){var E=31-Oe(l),z=1<<E;o[E]=0,m[E]=-1;var p=j[E];if(p!==null)for(j[E]=null,E=0;E<p.length;E++){var _=p[E];_!==null&&(_.lane&=-536870913)}l&=~z}a!==0&&ci(t,a,0),u!==0&&n===0&&t.tag!==0&&(t.suspendedLanes|=u&~(s&~e))}function ci(t,e,l){t.pendingLanes|=e,t.suspendedLanes&=~e;var a=31-Oe(e);t.entangledLanes|=e,t.entanglements[a]=t.entanglements[a]|1073741824|l&261930}function hn(t,e){var l=t.entangledLanes|=e;for(t=t.entanglements;l;){var a=31-Oe(l),n=1<<a;n&e|t[a]&e&&(t[a]|=e),l&=~n}}function $l(t,e){var l=e&-e;return l=(l&42)!==0?1:Hl(l),(l&(t.suspendedLanes|e))!==0?0:l}function Hl(t){switch(t){case 2:t=1;break;case 8:t=4;break;case 32:t=16;break;case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:case 262144:case 524288:case 1048576:case 2097152:case 4194304:case 8388608:case 16777216:case 33554432:t=128;break;case 268435456:t=134217728;break;default:t=0}return t}function mu(t){return t&=-t,2<t?8<t?(t&134217727)!==0?32:268435456:8:2}function hu(){var t=dt.p;return t!==0?t:(t=window.event,t===void 0?32:Eh(t.type))}function Ze(t,e){var l=dt.p;try{return dt.p=t,e()}finally{dt.p=l}}var ae=Math.random().toString(36).slice(2),ve="__reactFiber$"+ae,Ke="__reactProps$"+ae,vn="__reactContainer$"+ae,So="__reactEvents$"+ae,Ph="__reactListeners$"+ae,tv="__reactHandles$"+ae,xo="__reactResources$"+ae,vu="__reactMarker$"+ae,si="__reactLoad$"+ae;function ri(t){delete t[ve],delete t[Ke],delete t[Ph],delete t[tv]}function La(t){var e;if(e=t[ve])return e;for(var l=t.parentNode;l;){if(e=l[vn]||l[ve]){if(l=e.alternate,e.child!==null||l!==null&&l.child!==null)for(t=uh(t);t!==null;){if(l=t[ve])return l;t=uh(t)}return e}t=l,l=t.parentNode}return null}function yn(t){if(t=t[ve]||t[vn]){var e=t.tag;if(e===5||e===6||e===13||e===31||e===26||e===27||e===3)return t}return null}function yu(t){var e=t.tag;if(e===5||e===26||e===27||e===6)return t.stateNode;throw Error(f(33))}function gn(t){var e=t[xo];return e||(e=t[xo]={hoistableStyles:new Map,hoistableScripts:new Map}),e}function je(t){t[vu]=!0}function bo(t){t[si]=void 0}var jo=new Set,To={};function Va(t,e){pn(t,e),pn(t+"Capture",e)}function pn(t,Line truncated
|
||||
`).replace(G0,"")}function qm(t,e){return e=Bm(e),Bm(t)===e}function Vt(t,e,l,a,n,u){switch(l){case"children":if(typeof a=="string")e==="body"||e==="textarea"&&a===""||xn(t,a);else if(typeof a=="number"||typeof a=="bigint")e!=="body"&&xn(t,""+a);else return;break;case"className":fi(t,"class",a);break;case"tabIndex":fi(t,"tabindex",a);break;case"dir":case"role":case"viewBox":case"width":case"height":fi(t,l,a);break;case"style":Do(t,a,u);return;case"data":if(e!=="object"){fi(t,"data",a);break}case"src":case"href":if(a===""&&(e!=="a"||l!=="href")){t.removeAttribute(l);break}if(a==null||typeof a=="function"||typeof a=="symbol"||typeof a=="boolean"){t.removeAttribute(l);break}a=di(a),t.setAttribute(l,a);break;case"action":case"formAction":if(typeof a=="function"){t.setAttribute(l,"javascript:throw new Error('A React form was unexpectedly submitted. If you called form.submit() manually, consider using form.requestSubmit() instead. If you\\'re trying to use event.stopPropagation() in a submit event handler, consider also calling event.preventDefault().')");break}else typeof u=="function"&&(l==="formAction"?(e!=="input"&&Vt(t,e,"name",n.name,n,null),Vt(t,e,"formEncType",n.formEncType,n,null),Vt(t,e,"formMethod",n.formMethod,n,null),Vt(t,e,"formTarget",n.formTarget,n,null)):(Vt(t,e,"encType",n.encType,n,null),Vt(t,e,"method",n.method,n,null),Vt(t,e,"target",n.target,n,null)));if(a==null||typeof a=="symbol"||typeof a=="boolean"){t.removeAttribute(l);break}a=di(a),t.setAttribute(l,a);break;case"onClick":a!=null&&(t.onclick=wl);return;case"onScroll":a!=null&&At("scroll",t);return;case"onScrollEnd":a!=null&&At("scrollend",t);return;case"dangerouslySetInnerHTML":if(a!=null){if(typeof a!="object"||!("__html"in a))throw Error(f(61));if(l=a.__html,l!=null){if(n.children!=null)throw Error(f(60));(u!=null?u.__html:void 0)!==l&&(t.innerHTML=l)}}break;case"multiple":t.multiple=a&&typeof a!="function"&&typeof a!="symbol";break;case"muted":t.muted=a&&typeof a!="function"&&typeof a!="symbol";break;case"suppressContentEditableWarning":case"suppressHydrationWarning":case"defaultValue":case"defaultChecked":case"innerHTML":case"ref":break;case"autoFocus":break;case"xlinkHref":if(a==null||typeof a=="function"||typeof a=="boolean"||typeof a=="symbol"){t.removeAttribute("xlink:href");break}l=di(a),t.setAttributeNS("http://www.w3.org/1999/xlink","xlink:href",l);break;case"contentEditable":case"spellCheck":case"draggable":case"value":case"autoReverse":case"externalResourcesRequired":case"focusable":case"preserveAlpha":a!=null&&typeof a!="function"&&typeof a!="symbol"?t.setAttribute(l,a):t.removeAttribute(l);break;case"inert":case"allowFullScreen":case"async":case"autoPlay":case"controls":case"credentialless":case"default":case"defer":case"disabled":case"disablePictureInPicture":case"disableRemotePlayback":case"formNoValidate":case"hidden":case"loop":case"noModule":case"noValidate":case"open":case"playsInline":case"readOnly":case"required":case"reversed":case"scoped":case"seamless":case"itemScope":a&&typeof a!="function"&&typeof a!="symbol"?t.setAttribute(l,""):t.removeAttribute(l);break;case"capture":case"download":a===!0?t.setAttribute(l,""):a!==!1&&a!=null&&typeof a!="function"&&typeof a!="symbol"?t.setAttribute(l,a):t.removeAttribute(l);break;case"cols":case"rows":case"size":case"span":a!=null&&typeof a!="function"&&typeof a!="symbol"&&!isNaN(a)&&1<=a?t.setAttribute(l,a):t.removeAttribute(l);break;case"rowSpan":case"start":a==null||typeof a=="function"||typeof a=="symbol"||isNaN(a)?t.removeAttribute(l):t.setAttribute(l,a);break;case"popover":At("beforetoggle",t),At("toggle",t),oi(t,"popover",a);break;case"xlinkActuate":Il(t,"http://www.w3.org/1999/xlink","xlink:actuate",a);break;case"xlinkArcrole":Il(t,"http://www.w3.org/1999/xlink","xlink:arcrole",a);break;case"xlinkRole":Il(t,"http://www.w3.org/1999/xlink","xlink:role",a);break;case"xlinkShow":Il(t,"http://www.w3.org/1999/xlink","xlink:show",a);break;case"xlinkTitle":Il(t,"http://www.w3.org/1999/xlink","xlink:title",a);break;case"xlinkType":Il(t,"http://www.w3.org/1999/xlink","xlink:type",a);break;case"xmlBase":Il(t,"http://www.w3.org/XML/1998/namespace","xml:base",a);break;case"xmlLang":Il(t,"http://www.w3.org/XML/1998/namespace","xml:lang",a);break;case"xmlSpace":Il(t,"http://www.w3.org/XML/1998/namespace","xml:space",a);break;case"is":oi(t,"is",a);break;case"innerText":case"textContent":return;default:if(!(2<l.length)||l[0]!=="o"&&l[0]!=="O"||l[1]!=="n"&&l[1]!=="N")l=iv.get(l)||l,oi(t,l,a);else return}kt=!0}function Hr(t,e,l,a,n,u){switch(l){case"style":Do(t,a,u);return;case"dangerouslySetInnerHTML":if(a!=null){if(typeof a!="object"||!("__html"in a))throw Error(f(61));if(l=a.__html,l!=null){if(n.children!=null)throw Error(f(60));(u!=null?u.__html:void 0)!==l&&(t.innerHTML=l)}}break;case"children":if(typeof a=="string")xn(t,a);else if(typeof a=="number"||typeof a=="bigint")xn(t,""+a);else return;break;case"onScroll":a!=null&&At("scroll",t);return;case"onScrollEnd":a!=null&&At("scroLine truncated
|
||||
`;qt&&(xe+=qt+`
|
||||
`),Dt&&(xe+=(Array.isArray(Dt)?Dt.join(`
|
||||
`):String(Dt))+`
|
||||
`),xe+=JSON.stringify(nt,null,2).slice(0,4e3),jl(xe),ce("已返回登录信息,见下方")}catch(nt){ce("发起失败:"+nt.message)}},[]),U=d.useCallback(async()=>{const O=V.trim();if(!O){W("请填要查配额的用户 id");return}try{const J=await tt(K.adminQuotaUser(O));W("用户 "+O.slice(0,8)+":本月用量 "+fo(J.used_usd_micros)+" / 配额 "+fo(J.quota_usd_micros)+(J.remaining_usd_micros!==null&&J.remaining_usd_micros!==void 0?"(剩余 "+fo(J.remaining_usd_micros)+")":"")+" | 月份 "+(J.month||""))}catch(J){W(oo(J))}},[V]),$=d.useCallback(async()=>{if(window.confirm("确定手动触发 sweep 吗?超阈值的陈旧 running 任务会被标为 interrupted。"))try{const J=(await tt(K.adminSweep,"POST",{})).swept||[];W("sweep 完成,清理 "+J.length+" 个任务"+(J.length?":"+J.join("、"):""))}catch(O){W(oo(O))}},[]),at=d.useCallback(async()=>{try{const O=await tt(K.adminBackup,"POST",{});W("备份完成:backend="+O.backend+" | 路径 "+O.path+" | 大小 "+O.size_bytes+" 字节 | 任务 "+O.task_count+" 行")}catch(O){W(oo(O))}},[]);return{keySaved:i,keyState:h,quotaText:g,healthText:A,adminGate:H,adminBody:Y,adminResult:k,adminUserId:V,setAdminUserId:I,projects:X,projectId:gt,setProjectId:xt,projectMsg:Z,githubRepos:w,workers:et,workersSummary:Bt,workersError:wt,members:ot,memberMsg:G,memberError:ut,skills:pt,apps:we,mcpServers:y,skillMsg:ct,skillError:Nt,mcpMsg:Rt,mcpOauthMsg:dt,mcpOauthBox:Ga,envs:Le,envCount:Gt,envMsg:Jl,envRepoLine:da,envRepoBar:zl,envRepoMsg:Tl,envSelect:se,setEnvSelect:re,models:rl,modelCustomVisible:ma,refreshHealth:qe,loadQuota:_l,probeAdmin:ol,refreshProjects:Wt,loadGithubRepos:Ul,loadRelayModels:mn,refreshWorkers:Qa,refreshEnvs:Se,refreshMembers:Xa,refreshSkills:ha,saveKeyInput:su,clearKeyInput:ru,createProject:Wl,createEnv:Ae,archiveEnv:fl,saveEnvRepo:Oe,inviteMember:ou,changeMemberRole:fu,reloadMcp:du,mcpOauth:B,adminQuota:U,adminSweep:$,adminBackup:at}}const mo=20;function ag(){const[i,r]=d.useState([]),[h,f]=d.useState(""),[g,C]=d.useState(""),[A,S]=d.useState(""),[H,R]=d.useState(""),[Y,x]=d.useState(0),[k,W]=d.useState(0),[V,I]=d.useState({}),X=d.useRef(0),ht=d.useRef({status:"",executor:"",q:""});ht.current={status:g,executor:A,q:H};const gt=d.useRef(""),xt=d.useCallback(async(ot,rt,G)=>{const N=ot!==void 0?ot:rt;if(!N){r([]),W(0),I({});return}gt.current=N;const ut=G!==void 0?G:X.current,St=ht.current;try{const pt=await tt(K.tasksPage(N,mo,ut*mo,St.status,St.executor,St.q));r(pt.tasks||[]),W(pt.total??(pt.tasks||[]).length),I(pt.status_counts||{}),f("")}catch(pt){f("加载失败:"+pt.message)}},[]),Z=d.useCallback(ot=>{const rt=Math.max(0,ot);X.current=rt,x(rt)},[]),Tt=d.useCallback(ot=>{Z(ot),xt(gt.current||void 0,void 0,ot)},[xt,Z]);d.useEffect(()=>{if(!gt.current)return;const ot=setTimeout(()=>{xt(gt.current,void 0,0)},300);return()=>clearTimeout(ot)},[g,A,H]);const w=d.useCallback(ot=>{C(ot),Z(0)},[Z]),Q=d.useCallback(ot=>{S(ot),Z(0)},[Z]),et=d.useCallback(ot=>{R(ot),Z(0)},[Z]),zt=d.useCallback(async ot=>{Z(0),await xt(ot,void 0,0)},[xt,Z]),Bt=Math.max(1,Math.ceil(k/mo)),vt=k===0?"共 0 个任务":"共 "+k+" 个任务,第 "+(Y+1)+" / "+Bt+" 页",wt=i,q=d.useCallback(async(ot,rt)=>{const G={...ot};rt.length&&(G.upload_id=await tg(rt));const N=await tt(K.tasksCreate,"POST",G);return"任务已建:"+Ne(N.id)},[]);return{tasks:i,taskError:h,statusFilter:g,setStatusFilter:w,executorFilter:A,setExecutorFilter:Q,searchText:H,setSearchText:et,filteredTasks:wt,taskCountText:vt,refreshTasks:xt,refreshTasksFirstPage:zt,page:Y,totalPages:Bt,taskTotal:k,taskStatusCounts:V,goToPage:Tt,dispatch:q}}function ng(i,r){const[h,f]=d.useState(null),[g,C]=d.useState(""),[A,S]=d.useState(""),[H,R]=d.useState(""),[Y,x]=d.useState(""),[k,W]=d.useState(void 0),[V,I]=d.useState(!1),[X,ht]=d.useState(""),[gt,xt]=d.useState([]),[Z,Tt]=d.useState([]),[w,Q]=d.useState(""),[et,zt]=d.useState([]),[Bt,vt]=d.useState(""),[wt,q]=d.useState(""),[ot,rt]=d.useState([]),[G,N]=d.useState(""),[ut,St]=d.useState(""),[pt,ge]=d.useState(""),[we,Ce]=d.useState(""),[y,D]=d.useState([]),[ct,it]=d.useState([]),[Nt,_t]=d.useState(""),[Rt,P]=d.useState([]),[dt,ce]=d.useState(""),[Ga,jl]=d.useState(""),[Le,le]=d.useState(""),[Gt,he]=d.useState(""),[Jl,pe]=d.useState([]),[da,Ve]=d.useState(""),[zl,Rl]=d.useState(null),Tl=d.useRef(null),Be=d.useRef(null),se=d.useRef(null);se.current=h;const re=d.useCallback(()=>{if(Tl.current){try{Tl.current.abort()}catch{}Tl.current=null}Be.current&&(clearTimeout(Be.current),Be.current=null)},[]),rl=d.useCallback(async B=>{try{const U=await tt(K.taskAgents(B));Tt(U.agents||[]),Q(U.note||"")}catch(U){Tt([]),Q("子代理链路加载失败:"+U.message)}},[]),Fl=d.useCallback(B=>{Be.current&&clearTimeout(Be.current),Be.current=setTimeout(()=>void rl(B),1e3)},[rl]),ma=d.useCallback(async B=>{if(!B.coordinator_parts){xt([]);return}try{const U=await tt(K.tasksByParent(B.project_id||"",B.id));xt(U.tasks||[])}catch{xt([])}},[]),Dl=d.useCallback(async B=>{try{Line truncated
|
||||
`+a.stack}}var ee=Object.prototype.hasOwnProperty,Pe=i.unstable_scheduleCallback,qe=i.unstable_cancelCallback,_l=i.unstable_shouldYield,ol=i.unstable_requestPaint,$t=i.unstable_now,Ul=i.unstable_getCurrentPriorityLevel,mn=i.unstable_ImmediatePriority,Va=i.unstable_UserBlockingPriority,be=i.unstable_NormalPriority,Qa=i.unstable_LowPriority,ha=i.unstable_IdlePriority,su=i.log,ru=i.unstable_setDisableYieldValue,Wl=null,Me=null;function fl(t){if(typeof su=="function"&&ru(t),Me&&typeof Me.setStrictMode=="function")try{Me.setStrictMode(Wl,t)}catch{}}var Oe=Math.clz32?Math.clz32:du,ou=Math.log,fu=Math.LN2;function du(t){return t>>>=0,t===0?32:31-(ou(t)/fu|0)|0}var B=256,U=262144,P=4194304;function it(t){var e=t&42;if(e!==0)return e;switch(t&-t){case 1:return 1;case 2:return 2;case 4:return 4;case 8:return 8;case 16:return 16;case 32:return 32;case 64:return 64;case 128:return 128;case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:return t&-t;case 262144:case 524288:case 1048576:case 2097152:return t&3932160;case 4194304:case 8388608:case 16777216:case 33554432:return t&62914560;case 67108864:return 67108864;case 134217728:return 134217728;case 268435456:return 268435456;case 536870912:return 536870912;case 1073741824:return 0;default:return t}}function A(t,e,l){var a=t.pendingLanes;if(a===0)return 0;var n=0,u=t.suspendedLanes,s=t.pingedLanes;t=t.warmLanes;var o=a&134217727;return o!==0?(a=o&~u,a!==0?n=it(a):(s&=o,s!==0?n=it(s):l||(l=o&~t,l!==0&&(n=it(l))))):(o=a&~u,o!==0?n=it(o):s!==0?n=it(s):l||(l=a&~t,l!==0&&(n=it(l)))),n===0?0:e!==0&&e!==n&&(e&u)===0&&(u=n&-n,l=e&-e,u>=l||u===32&&(l&4194048)!==0)?e:n}function $(t,e){return(t.pendingLanes&~(t.suspendedLanes&~t.pingedLanes)&e)===0}function ct(t,e){(e&8)!==0&&(e|=e&32);var l=t.entangledLanes;if(l!==0)for(t=t.entanglements,l&=e;0<l;){var a=31-Oe(l),n=1<<a;e|=t[a],l&=~n}return e}function Yt(t,e){switch(t){case 1:case 2:case 4:case 8:case 64:return e+250;case 16:case 32:case 128:case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:case 262144:case 524288:case 1048576:case 2097152:return e+5e3;case 4194304:case 8388608:case 16777216:case 33554432:return-1;case 67108864:case 134217728:case 268435456:case 536870912:case 1073741824:return-1;default:return-1}}function wt(){var t=P;return P<<=1,(P&62914560)===0&&(P=4194304),t}function je(t){for(var e=[],l=0;31>l;l++)e.push(t);return e}function Te(t,e){t.pendingLanes|=e,e!==268435456&&(t.suspendedLanes=0,t.pingedLanes=0,t.warmLanes=0)}function kc(t,e,l,a,n,u){var s=t.pendingLanes;t.pendingLanes=l,t.suspendedLanes=0,t.pingedLanes=0,t.warmLanes=0,t.expiredLanes&=l,t.entangledLanes&=l,t.errorRecoveryDisabledLanes&=l,t.shellSuspendCounter=0;var o=t.entanglements,m=t.expirationTimes,b=t.hiddenUpdates;for(l=s&~l;0<l;){var E=31-Oe(l),O=1<<E;o[E]=0,m[E]=-1;var S=b[E];if(S!==null)for(b[E]=null,E=0;E<S.length;E++){var _=S[E];_!==null&&(_.lane&=-536870913)}l&=~O}a!==0&&ri(t,a,0),u!==0&&n===0&&t.tag!==0&&(t.suspendedLanes|=u&~(s&~e))}function ri(t,e,l){t.pendingLanes|=e,t.suspendedLanes&=~e;var a=31-Oe(e);t.entangledLanes|=e,t.entanglements[a]=t.entanglements[a]|1073741824|l&261930}function hn(t,e){var l=t.entangledLanes|=e;for(t=t.entanglements;l;){var a=31-Oe(l),n=1<<a;n&e|t[a]&e&&(t[a]|=e),l&=~n}}function $l(t,e){var l=e&-e;return l=(l&42)!==0?1:wl(l),(l&(t.suspendedLanes|e))!==0?0:l}function wl(t){switch(t){case 2:t=1;break;case 8:t=4;break;case 32:t=16;break;case 256:case 512:case 1024:case 2048:case 4096:case 8192:case 16384:case 32768:case 65536:case 131072:case 262144:case 524288:case 1048576:case 2097152:case 4194304:case 8388608:case 16777216:case 33554432:t=128;break;case 268435456:t=134217728;break;default:t=0}return t}function mu(t){return t&=-t,2<t?8<t?(t&134217727)!==0?32:268435456:8:2}function hu(){var t=mt.p;return t!==0?t:(t=window.event,t===void 0?32:Oh(t.type))}function Ze(t,e){var l=mt.p;try{return mt.p=t,e()}finally{mt.p=l}}var ne=Math.random().toString(36).slice(2),ge="__reactFiber$"+ne,Ke="__reactProps$"+ne,vn="__reactContainer$"+ne,_o="__reactEvents$"+ne,nv="__reactListeners$"+ne,uv="__reactHandles$"+ne,Eo="__reactResources$"+ne,vu="__reactMarker$"+ne,oi="__reactLoad$"+ne;function fi(t){delete t[ge],delete t[Ke],delete t[nv],delete t[uv]}function Xa(t){var e;if(e=t[ge])return e;for(var l=t.parentNode;l;){if(e=l[vn]||l[ge]){if(l=e.alternate,e.child!==null||l!==null&&l.child!==null)for(t=oh(t);t!==null;){if(l=t[ge])return l;t=oh(t)}return e}t=l,l=t.parentNode}return null}function yn(t){if(t=t[ge]||t[vn]){var e=t.tag;if(e===5||e===6||e===13||e===31||e===26||e===27||e===3)return t}return null}function yu(t){var e=t.tag;if(e===5||e===26||e===27||e===6)return t.stateNode;throw Error(f(33))}function gn(t){var e=t[Eo];return e||(e=t[Eo]={hoistableStyles:new Map,hoistableScripts:new Map}),e}function _e(t){t[vu]=!0}function No(t){t[oi]=void 0}var Co=new Set,Ao={};function La(t,e){pn(t,e),pn(t+"Capture",e)}function pn(t,Line truncated
|
||||
`).replace(Z0,"")}function Xm(t,e){return e=Qm(e),Qm(t)===e}function Zt(t,e,l,a,n,u){switch(l){case"children":if(typeof a=="string")e==="body"||e==="textarea"&&a===""||xn(t,a);else if(typeof a=="number"||typeof a=="bigint")e!=="body"&&xn(t,""+a);else return;break;case"className":mi(t,"class",a);break;case"tabIndex":mi(t,"tabindex",a);break;case"dir":case"role":case"viewBox":case"width":case"height":mi(t,l,a);break;case"style":Bo(t,a,u);return;case"data":if(e!=="object"){mi(t,"data",a);break}case"src":case"href":if(a===""&&(e!=="a"||l!=="href")){t.removeAttribute(l);break}if(a==null||typeof a=="function"||typeof a=="symbol"||typeof a=="boolean"){t.removeAttribute(l);break}a=hi(a),t.setAttribute(l,a);break;case"action":case"formAction":if(typeof a=="function"){t.setAttribute(l,"javascript:throw new Error('A React form was unexpectedly submitted. If you called form.submit() manually, consider using form.requestSubmit() instead. If you\\'re trying to use event.stopPropagation() in a submit event handler, consider also calling event.preventDefault().')");break}else typeof u=="function"&&(l==="formAction"?(e!=="input"&&Zt(t,e,"name",n.name,n,null),Zt(t,e,"formEncType",n.formEncType,n,null),Zt(t,e,"formMethod",n.formMethod,n,null),Zt(t,e,"formTarget",n.formTarget,n,null)):(Zt(t,e,"encType",n.encType,n,null),Zt(t,e,"method",n.method,n,null),Zt(t,e,"target",n.target,n,null)));if(a==null||typeof a=="symbol"||typeof a=="boolean"){t.removeAttribute(l);break}a=hi(a),t.setAttribute(l,a);break;case"onClick":a!=null&&(t.onclick=kl);return;case"onScroll":a!=null&&Ot("scroll",t);return;case"onScrollEnd":a!=null&&Ot("scrollend",t);return;case"dangerouslySetInnerHTML":if(a!=null){if(typeof a!="object"||!("__html"in a))throw Error(f(61));if(l=a.__html,l!=null){if(n.children!=null)throw Error(f(60));(u!=null?u.__html:void 0)!==l&&(t.innerHTML=l)}}break;case"multiple":t.multiple=a&&typeof a!="function"&&typeof a!="symbol";break;case"muted":t.muted=a&&typeof a!="function"&&typeof a!="symbol";break;case"suppressContentEditableWarning":case"suppressHydrationWarning":case"defaultValue":case"defaultChecked":case"innerHTML":case"ref":break;case"autoFocus":break;case"xlinkHref":if(a==null||typeof a=="function"||typeof a=="boolean"||typeof a=="symbol"){t.removeAttribute("xlink:href");break}l=hi(a),t.setAttributeNS("http://www.w3.org/1999/xlink","xlink:href",l);break;case"contentEditable":case"spellCheck":case"draggable":case"value":case"autoReverse":case"externalResourcesRequired":case"focusable":case"preserveAlpha":a!=null&&typeof a!="function"&&typeof a!="symbol"?t.setAttribute(l,a):t.removeAttribute(l);break;case"inert":case"allowFullScreen":case"async":case"autoPlay":case"controls":case"credentialless":case"default":case"defer":case"disabled":case"disablePictureInPicture":case"disableRemotePlayback":case"formNoValidate":case"hidden":case"loop":case"noModule":case"noValidate":case"open":case"playsInline":case"readOnly":case"required":case"reversed":case"scoped":case"seamless":case"itemScope":a&&typeof a!="function"&&typeof a!="symbol"?t.setAttribute(l,""):t.removeAttribute(l);break;case"capture":case"download":a===!0?t.setAttribute(l,""):a!==!1&&a!=null&&typeof a!="function"&&typeof a!="symbol"?t.setAttribute(l,a):t.removeAttribute(l);break;case"cols":case"rows":case"size":case"span":a!=null&&typeof a!="function"&&typeof a!="symbol"&&!isNaN(a)&&1<=a?t.setAttribute(l,a):t.removeAttribute(l);break;case"rowSpan":case"start":a==null||typeof a=="function"||typeof a=="symbol"||isNaN(a)?t.removeAttribute(l):t.setAttribute(l,a);break;case"popover":Ot("beforetoggle",t),Ot("toggle",t),di(t,"popover",a);break;case"xlinkActuate":Il(t,"http://www.w3.org/1999/xlink","xlink:actuate",a);break;case"xlinkArcrole":Il(t,"http://www.w3.org/1999/xlink","xlink:arcrole",a);break;case"xlinkRole":Il(t,"http://www.w3.org/1999/xlink","xlink:role",a);break;case"xlinkShow":Il(t,"http://www.w3.org/1999/xlink","xlink:show",a);break;case"xlinkTitle":Il(t,"http://www.w3.org/1999/xlink","xlink:title",a);break;case"xlinkType":Il(t,"http://www.w3.org/1999/xlink","xlink:type",a);break;case"xmlBase":Il(t,"http://www.w3.org/XML/1998/namespace","xml:base",a);break;case"xmlLang":Il(t,"http://www.w3.org/XML/1998/namespace","xml:lang",a);break;case"xmlSpace":Il(t,"http://www.w3.org/XML/1998/namespace","xml:space",a);break;case"is":di(t,"is",a);break;case"innerText":case"textContent":return;default:if(!(2<l.length)||l[0]!=="o"&&l[0]!=="O"||l[1]!=="n"&&l[1]!=="N")l=fv.get(l)||l,di(t,l,a);else return}Bt=!0}function Br(t,e,l,a,n,u){switch(l){case"style":Bo(t,a,u);return;case"dangerouslySetInnerHTML":if(a!=null){if(typeof a!="object"||!("__html"in a))throw Error(f(61));if(l=a.__html,l!=null){if(n.children!=null)throw Error(f(60));(u!=null?u.__html:void 0)!==l&&(t.innerHTML=l)}}break;case"children":if(typeof a=="string")xn(t,a);else if(typeof a=="number"||typeof a=="bigint")xn(t,""+a);else return;break;case"onScroll":a!=null&&Ot("scroll",t);return;case"onScrollEnd":a!=null&&Ot("scroLine truncated
|
||||
`;Yt&&(je+=Yt+`
|
||||
`),wt&&(je+=(Array.isArray(wt)?wt.join(`
|
||||
`):String(wt))+`
|
||||
`),je+=JSON.stringify(ct,null,2).slice(0,4e3),jl(je),re("已返回登录信息,见下方")}catch(ct){re("发起失败:"+ct.message)}},[]),U=d.useCallback(async()=>{const A=X.trim();if(!A){F("请填要查配额的用户 id");return}try{const $=await lt(W.adminQuotaUser(A));F("用户 "+A.slice(0,8)+":本月用量 "+vo($.used_usd_micros)+" / 配额 "+vo($.quota_usd_micros)+($.remaining_usd_micros!==null&&$.remaining_usd_micros!==void 0?"(剩余 "+vo($.remaining_usd_micros)+")":"")+" | 月份 "+($.month||""))}catch($){F(ho($))}},[X]),P=d.useCallback(async()=>{if(window.confirm("确定手动触发 sweep 吗?超阈值的陈旧 running 任务会被标为 interrupted。"))try{const $=(await lt(W.adminSweep,"POST",{})).swept||[];F("sweep 完成,清理 "+$.length+" 个任务"+($.length?":"+$.join("、"):""))}catch(A){F(ho(A))}},[]),it=d.useCallback(async()=>{try{const A=await lt(W.adminBackup,"POST",{});F("备份完成:backend="+A.backend+" | 路径 "+A.path+" | 大小 "+A.size_bytes+" 字节 | 任务 "+A.task_count+" 行")}catch(A){F(ho(A))}},[]);return{keySaved:i,keyState:h,quotaText:p,healthText:C,adminGate:z,adminBody:q,adminResult:k,adminUserId:X,setAdminUserId:J,projects:Q,projectId:St,setProjectId:jt,projectMsg:K,githubRepos:w,workers:at,workersSummary:qt,workersError:Ut,members:ot,memberMsg:G,memberError:st,skills:xt,apps:L,mcpServers:y,skillMsg:nt,skillError:Ct,mcpMsg:Tt,mcpOauthMsg:mt,mcpOauthBox:Ga,envs:Xe,envCount:Vt,envMsg:Jl,envRepoLine:da,envRepoBar:zl,envRepoMsg:Tl,envSelect:oe,setEnvSelect:fe,models:rl,modelCustomVisible:ma,refreshHealth:qe,loadQuota:_l,probeAdmin:ol,refreshProjects:$t,loadGithubRepos:Ul,loadRelayModels:mn,refreshWorkers:Va,refreshEnvs:be,refreshMembers:Qa,refreshSkills:ha,saveKeyInput:su,clearKeyInput:ru,createProject:Wl,createEnv:Me,archiveEnv:fl,saveEnvRepo:Oe,inviteMember:ou,changeMemberRole:fu,reloadMcp:du,mcpOauth:B,adminQuota:U,adminSweep:P,adminBackup:it}}const yo=20;function sg(){const[i,r]=d.useState([]),[h,f]=d.useState(""),[p,N]=d.useState(""),[C,j]=d.useState(""),[z,R]=d.useState(""),[q,g]=d.useState(0),[k,F]=d.useState(0),[X,J]=d.useState({}),Q=d.useRef(0),vt=d.useRef({status:"",executor:"",q:""});vt.current={status:p,executor:C,q:z};const St=d.useRef(""),jt=d.useCallback(async(ot,yt,G)=>{const Y=ot!==void 0?ot:yt;if(!Y){r([]),F(0),J({});return}St.current=Y;const st=G!==void 0?G:Q.current,gt=vt.current;try{const xt=await lt(W.tasksPage(Y,yo,st*yo,gt.status,gt.executor,gt.q));r(xt.tasks||[]),F(xt.total??(xt.tasks||[]).length),J(xt.status_counts||{}),f("")}catch(xt){f("加载失败:"+xt.message)}},[]),K=d.useCallback(ot=>{const yt=Math.max(0,ot);Q.current=yt,g(yt)},[]),Nt=d.useCallback(ot=>{K(ot),jt(St.current||void 0,void 0,ot)},[jt,K]);d.useEffect(()=>{if(!St.current)return;const ot=setTimeout(()=>{jt(St.current,void 0,0)},300);return()=>clearTimeout(ot)},[p,C,z]);const w=d.useCallback(ot=>{N(ot),K(0)},[K]),V=d.useCallback(ot=>{j(ot),K(0)},[K]),at=d.useCallback(ot=>{R(ot),K(0)},[K]),Dt=d.useCallback(async ot=>{K(0),await jt(ot,void 0,0)},[jt,K]),qt=Math.max(1,Math.ceil(k/yo)),dt=k===0?"共 0 个任务":"共 "+k+" 个任务,第 "+(q+1)+" / "+qt+" 页",Ut=i,H=d.useCallback(async(ot,yt)=>{const G={...ot};yt.length&&(G.upload_id=await ug(yt));const Y=await lt(W.tasksCreate,"POST",G);return"任务已建:"+Ae(Y.id)},[]);return{tasks:i,taskError:h,statusFilter:p,setStatusFilter:w,executorFilter:C,setExecutorFilter:V,searchText:z,setSearchText:at,filteredTasks:Ut,taskCountText:dt,refreshTasks:jt,refreshTasksFirstPage:Dt,page:q,totalPages:qt,taskTotal:k,taskStatusCounts:X,goToPage:Nt,dispatch:H}}function rg(i,r){const[h,f]=d.useState(null),[p,N]=d.useState(""),[C,j]=d.useState(""),[z,R]=d.useState(""),[q,g]=d.useState(""),[k,F]=d.useState(void 0),[X,J]=d.useState(!1),[Q,vt]=d.useState(""),[St,jt]=d.useState([]),[K,Nt]=d.useState([]),[w,V]=d.useState(""),[at,Dt]=d.useState([]),[qt,dt]=d.useState(""),[Ut,H]=d.useState(""),[ot,yt]=d.useState([]),[G,Y]=d.useState(""),[st,gt]=d.useState(""),[xt,Se]=d.useState(""),[L,se]=d.useState(""),[y,D]=d.useState([]),[nt,tt]=d.useState([]),[Ct,bt]=d.useState(""),[Tt,et]=d.useState([]),[mt,re]=d.useState(""),[Ga,jl]=d.useState(""),[Xe,ae]=d.useState(""),[Vt,ye]=d.useState(""),[Jl,xe]=d.useState([]),[da,Le]=d.useState(""),[zl,Rl]=d.useState(null),Tl=d.useRef(null),Be=d.useRef(null),oe=d.useRef(null);oe.current=h;const fe=d.useCallback(()=>{if(Tl.current){try{Tl.current.abort()}catch{}Tl.current=null}Be.current&&(clearTimeout(Be.current),Be.current=null)},[]),rl=d.useCallback(async B=>{try{const U=await lt(W.taskAgents(B));Nt(U.agents||[]),V(U.note||"")}catch(U){Nt([]),V("子代理链路加载失败:"+U.message)}},[]),Fl=d.useCallback(B=>{Be.current&&clearTimeout(Be.current),Be.current=setTimeout(()=>void rl(B),1e3)},[rl]),ma=d.useCallback(async B=>{if(!B.coordinator_parts){jt([]);return}try{const U=await lt(W.tasksByParent(B.project_id||"",B.id));jt(U.tasks||[])}catch{jt([])}},[]),Dl=d.useCallback(async B=>{try{cLine truncated
|
||||
|
||||
`);hn=Hl.pop()||"";for(const mu of Hl){const hu=mu.split(`
|
||||
`);hn=wl.pop()||"";for(const mu of wl){const hu=mu.split(`
|
||||
`).filter(Ze=>Ze.indexOf("data: ")===0).map(Ze=>Ze.slice(6)).join(`
|
||||
`);if(hu)try{const Ze=JSON.parse(hu);if(Ze.type==="snapshot")at=Ze.entries||[],Object.keys(O).forEach(ae=>delete O[ae]),at.forEach((ae,ve)=>{ae.id&&(O[ae.id]=ve)}),J=Ze.note||"",rt([...at]),N(J),Fl(B);else if(Ze.type==="entry"){const ae=Ze.entry;ae.id&&ae.id in O?at[O[ae.id]]=ae:(ae.id&&(O[ae.id]=at.length),at.push(ae)),rt([...at]),Fl(B)}else Ze.type==="done"&&(nt=!0,qt=Ze.status,St(qt))}catch{}}if(nt)break}}catch(Dt){if(Dt&&Dt.name==="AbortError")return;N("链路流中断:"+Dt.message);return}nt&&r()},[re,Fl,r]),Se=d.useCallback(async B=>{re();try{const U=await tt(K.task(B));f(U);let $="";U.result_text&&($+=`=== result ===
|
||||
`);if(hu)try{const Ze=JSON.parse(hu);if(Ze.type==="snapshot")it=Ze.entries||[],Object.keys(A).forEach(ne=>delete A[ne]),it.forEach((ne,ge)=>{ne.id&&(A[ne.id]=ge)}),$=Ze.note||"",yt([...it]),Y($),Fl(B);else if(Ze.type==="entry"){const ne=Ze.entry;ne.id&&ne.id in A?it[A[ne.id]]=ne:(ne.id&&(A[ne.id]=it.length),it.push(ne)),yt([...it]),Fl(B)}else Ze.type==="done"&&(ct=!0,Yt=Ze.status,gt(Yt))}catch{}}if(ct)break}}catch(wt){if(wt&&wt.name==="AbortError")return;Y("链路流中断:"+wt.message);return}ct&&r()},[fe,Fl,r]),be=d.useCallback(async B=>{fe();try{const U=await lt(W.task(B));f(U);let P="";U.result_text&&(P+=`=== result ===
|
||||
`+U.result_text.slice(0,8e3)+`
|
||||
`),U.error_text&&($+=`=== error ===
|
||||
`),U.error_text&&(P+=`=== error ===
|
||||
`+U.error_text.slice(0,8e3)+`
|
||||
`),$||($="(暂无 result/error 输出)"),C($),x(""),W(void 0),Dl(B),I(!1),ht(""),ce(""),he(""),mn(U.status);const at=i(U.project_id),O=at==="maintainer"||at==="owner",J=at==="contributor"||O,nt=uu.includes(U.status),qt=[];["failed","interrupted","merge-failed"].includes(U.status)&&O||qt.push("重试:不可用——仅 failed/interrupted/merge-failed 可重试"+(O?"":",且仅 maintainer 可重试")),!nt&&J||qt.push("取消:不可用——"+(nt?"终态任务不可取消":"仅项目成员(contributor 及以上)可取消")),U.status==="completed"&&O||qt.push("合并:不可用——仅 completed 可合并"+(O?"":",且仅 maintainer 可合并")),U.status==="merged"&&O||qt.push("提 PR:不可用——仅 merged 可提 PR"+(O?"":",且仅 maintainer 可提 PR")),R(qt.join(";")),S(""),Qa(B,U.status),rl(B),ma(U),Pe(B),te(B),ol(B),Wt(B),Ul(B),qe(B),_l(B)}catch(U){f(null),C("加载失败:"+U.message)}},[re,Qa,rl,ma,Pe,te,ol,Wt,Ul,qe,_l,Dl,i]),Xa=d.useCallback(async()=>{const B=se.current;if(B){try{const U=await tt(K.task(B.id));f(U);let $="";U.result_text&&($+=`=== result ===
|
||||
`),P||(P="(暂无 result/error 输出)"),N(P),g(""),F(void 0),Dl(B),J(!1),vt(""),re(""),ye(""),mn(U.status);const it=i(U.project_id),A=it==="maintainer"||it==="owner",$=it==="contributor"||A,ct=uu.includes(U.status),Yt=[];["failed","interrupted","merge-failed"].includes(U.status)&&A||Yt.push("重试:不可用——仅 failed/interrupted/merge-failed 可重试"+(A?"":",且仅 maintainer 可重试")),!ct&&$||Yt.push("取消:不可用——"+(ct?"终态任务不可取消":"仅项目成员(contributor 及以上)可取消")),U.status==="completed"&&A||Yt.push("合并:不可用——仅 completed 可合并"+(A?"":",且仅 maintainer 可合并")),U.status==="merged"&&A||Yt.push("提 PR:不可用——仅 merged 可提 PR"+(A?"":",且仅 maintainer 可提 PR")),R(Yt.join(";")),j(""),Va(B,U.status),rl(B),ma(U),Pe(B),ee(B),ol(B),$t(B),Ul(B),qe(B),_l(B)}catch(U){f(null),N("加载失败:"+U.message)}},[fe,Va,rl,ma,Pe,ee,ol,$t,Ul,qe,_l,Dl,i]),Qa=d.useCallback(async()=>{const B=oe.current;if(B){try{const U=await lt(W.task(B.id));f(U);let P="";U.result_text&&(P+=`=== result ===
|
||||
`+U.result_text.slice(0,8e3)+`
|
||||
`),U.error_text&&($+=`=== error ===
|
||||
`),U.error_text&&(P+=`=== error ===
|
||||
`+U.error_text.slice(0,8e3)+`
|
||||
`),C($||"(暂无 result/error 输出)")}catch{}te(B.id),Pe(B.id),qe(B.id),_l(B.id),ma(B),rl(B.id),ol(B.id),Wt(B.id),Ul(B.id),Dl(B.id)}},[te,Pe,qe,_l,ma,rl,ol,Wt,Ul,Dl]),ha=d.useCallback(async B=>{const U=se.current;if(!U)return;const $={cancel:"确定取消该任务吗?",merge:"确定把 completed 任务标为 merged 吗?",pr:"确定为该任务提 PR 吗?(会在仓库真实建 PR)"};if(!($[B]&&!window.confirm($[B])))try{const at=await tt(K.taskAction(U.id,B),"POST",{});S(B+" 已提交:"+JSON.stringify(at).slice(0,300)),r(),await Se(U.id)}catch(at){S(B+" 失败:"+at.message)}},[Se,r]),su=d.useCallback(async()=>{const B=se.current;if(B)try{const U=await tt(K.taskCost(B.id));x(JSON.stringify(U,null,2).slice(0,4e3))}catch(U){x("查成本失败:"+U.message)}},[]),ru=d.useCallback(async B=>{const U=se.current;if(U){if(!B.trim()){ht("分支消息必填");return}if(!uu.includes(U.status)){ht("仅终态任务可分支");return}try{const $=await tt(K.taskFork(U.id),"POST",{message:B.trim()});ht("分支任务已建:"+$.id.slice(0,8)),r()}catch($){ht("分支失败:"+$.message)}}},[r]),Wl=d.useCallback(async B=>{const U=se.current;if(U)try{await tt(K.taskReviews(U.id),"POST",B),vt("评审已登记(requested)"),await qe(U.id)}catch($){vt("登记评审需求失败:"+$.message)}},[qe]),Ae=d.useCallback(B=>{const U=se.current;if(!U)return;const $=window.prompt("输入该条评审的 findings(将写回记录并标为 completed):");$!==null&&tt(K.taskReviewFindings(U.id,B),"POST",{findings:$}).then(()=>void qe(U.id)).catch(at=>vt("写 findings 失败:"+at.message))},[qe]),fl=d.useCallback((B,U)=>{const $=se.current;if(!$)return;const at=U==="approved"?"approve":"deny";tt(K.taskApprovalDecision($.id,B,at),"POST",{}).then(()=>{_t("已记录"+(U==="approved"?"允许":"拒绝")+"(当前版本只留回执,不拦截实时行为)"),_l($.id)}).catch(O=>_t("决定失败:"+O.message))},[_l]),Oe=d.useCallback(async(B,U,$)=>{const at=se.current;if(at){if(!U.trim()){ce("目标任务 id 不能为空");return}if(!$.trim()){ce("消息正文不能为空");return}try{const O=await tt(K.taskMessagesTo(U.trim()),"POST",{from_task_id:at.id,body:$.trim(),mode:B});ce("已发送("+O.delivery+")"),await ol(at.id)}catch(O){ce("发送失败:"+O.message)}}},[ol]),ou=d.useCallback((B,U)=>{const $=se.current;if(!$)return;if(!["completed","failed","interrupted"].includes($.status)){he("仅 completed/failed/interrupted 任务可操作线程(merged/merge-failed 不可改,服务端同样 409)");return}const at={};if(B==="rename"){if(!U.trim()){he("新线程名不能为空");return}at.name=U.trim()}tt(K.taskThreadAction($.id,B),"POST",at).then(O=>{he(B+" 已提交:"+JSON.stringify(O).slice(0,300)),Wt($.id),Ul($.id)}).catch(O=>he(B+" 失败:"+O.message))},[Wt,Ul]),fu=d.useCallback(async B=>{const U=se.current;if(!U)return;const $=B.trim();if(!$){Ve("消息不能为空");return}try{uu.includes(U.status)&&U.status!=="merged"?(await tt(K.taskContinue(U.id),"POST",{message:$}),Ve("已重新排队,同一会话继续"),r(),await Se(U.id)):["queued","running"].includes(U.status)?(await tt(K.taskFollowups(U.id),"POST",{body:$,mode:"direct"}),Ve("已直接发送"),await te(U.id),await Pe(U.id)):Ve("merged/merge-failed 任务不可继续,也不可发后续消息")}catch(at){Ve("发送失败:"+at.message)}},[Se,r,te,Pe]),du=d.useCallback(async()=>{const B=se.current;if(!B){Ce("先选一个任务");return}try{const U=await tt(K.taskTrace(B.id)),$=new Blob([JSON.stringify({task_id:B.id,status:U.status,note:U.note||"",count:(U.entries||[]).length,entries:U.entries||[]},null,2)],{type:"application/json"}),at=document.createElement("a");at.href=URL.createObjectURL($),at.download="trace-"+B.id.slice(0,8)+".json",document.body.appendChild(at),at.click(),at.remove(),setTimeout(()=>URL.revokeObjectURL(at.href),1e3),Ce("已下载完整链路 "+(U.entries||[]).length+" 条")}catch(U){Ce("下载失败:"+U.message)}},[]);return d.useEffect(()=>()=>re(),[re]),{currentTask:h,detailBody:g,actionMsg:A,actionReasons:H,costBox:Y,costSummary:k,forkVisible:V,setForkVisible:I,forkMsg:X,coordChildren:gt,agents:Z,agentsNote:w,reviews:et,reviewMsg:Bt,reviewError:wt,traceEntries:ot,traceNote:G,traceDone:ut,traceKind:pt,setTraceKind:ge,traceMsg:we,followups:y,approvals:ct,approvalMsg:Nt,peerMessages:Rt,peerMsg:dt,threadText:Ga,threadGateNote:Le,threadMsg:Gt,threadList:Jl,chatMsg:da,chatHistory:zl,showDetail:Se,refreshDetailLists:Xa,doAction:ha,taskCost:su,forkTask:ru,postReview:Wl,submitFindings:Ae,decideApproval:fl,postPeer:Oe,threadAction:ou,sendChatMessage:fu,downloadTrace:du,refreshReviews:()=>void qe(),refreshApprovals:()=>void _l()}}function ug(i){const[r,h]=d.useState([]),[f,g]=d.useState(""),[C,A]=d.useState(null),[S,H]=d.useState(""),R=d.useCallback(async V=>{const I=V!==void 0?V:i();if(!I){h([]);return}try{const X=await tt(K.projectAutomations(I));h(X.automations||[])}catch(X){g("自动化加载失败:"+X.message)}}Line truncated
|
||||
(链路摘要已截断至 300 字,完整回复见右侧任务结果)`:f,label:"助手"}}if(i.kind==="exec"||i.kind==="file"||i.kind==="tool"||i.kind==="call"){const f=i.kind==="exec"?"命令":i.kind==="file"?"文件":i.kind==="tool"?"工具":"调用";return{id:h,role:"tool",kind:i.kind,text:vo(i),label:f}}return i.kind==="plan"?{id:h,role:"status",kind:i.kind,text:i.text||""}:i.kind==="reasoning"?{id:h,role:"status",kind:i.kind,text:"思考:"+(i.summary||"")}:i.kind==="agent-start"?{id:h,role:"status",kind:i.kind,text:"子代理开始:"+(i.name||"")+(i.prompt?"("+i.prompt+")":"")}:i.kind==="agent-done"?{id:h,role:"status",kind:i.kind,text:"子代理结束:"+(i.name||"")+" "+(i.status||"")}:{id:h,role:"status",kind:i.kind,text:vo(i)}}function pg(){const i=Zt(),r=i.currentTask,h=d.useMemo(()=>{if(!r)return[];let H;if(i.chatHistory){H=i.chatHistory.messages.map(V=>({id:V.id,role:V.role,text:V.text,label:V.label,kind:V.kind}));const k=new Set(H.map(V=>V.id)),W=i.traceEntries.filter(V=>V.id&&!k.has(V.id)&&!String(V.id).startsWith("fu-")).map((V,I)=>Xh(V,I));H=[...H,...W]}else{const k=r.prompt?[{id:"task-prompt",role:"user",text:r.prompt}]:[],W=i.traceEntries.filter(I=>!String(I.id||"").startsWith("fu-")).map((I,X)=>Xh(I,X)),V=i.followups.map((I,X)=>({id:"followup-"+X,role:"user",text:String(I.body||""),label:"后续消息"}));H=[...k,...W,...V]}const R=i.traceKind,Y=R?H.filter(k=>k.role==="user"||!k.kind||k.kind===R):H,x=i.traceDone?[{id:"trace-done",role:"status",text:"链路已结束("+i.traceDone+")"}]:[];return[...Y,...x]},[r,i.chatHistory,i.traceEntries,i.followups,i.traceKind,i.traceDone]);if(!r)return c.jsx("div",{className:"muted",children:"点任务行查看详情后,这里显示继续会话"});const f=uu.includes(r.status),g=["queued","running"].includes(r.status),C=f||g,A=f?"终态任务:发送走继续接口(同一会话再跑一轮)":g?"运行中/排队中:发送走后续消息直接发送(跑完当前轮后优先执行)":"merged/merge-failed 任务不可继续,也不可发后续消息",S=i.chatHistory?i.chatHistory.legacy_continue_count??null:null;return c.jsxs("div",{className:"chat-column",children:[c.jsx("div",{className:"muted",children:"实时对话"}),c.jsxs(Ht,{children:[c.jsxs("select",{"aria-label":"链路类型",value:i.traceKind,onChange:H=>i.setTraceKind(H.target.value),children:[c.jsx("option",{value:"",children:"链路类型:全部"}),["exec","file","call","tool","message","reasoning","plan","agent-start","agent-done"].map(H=>c.jsx("option",{value:H,children:H==="exec"?"命令":H==="file"?"文件":H==="call"?"调用":H==="tool"?"工具":H==="message"?"消息":H==="reasoning"?"思考":H==="plan"?"计划":"子代理"},H))]}),c.jsx("button",{onClick:i.downloadTrace,children:"下载完整链路"}),c.jsx("span",{className:"muted",children:i.traceMsg})]}),i.traceNote?c.jsx("div",{children:c.jsx("span",{className:"err",children:i.traceNote})}):null,S!==null&&S>0?c.jsxs("div",{className:"muted",children:["此前有 ",S," 条继续消息发生在持久化之前,未入库,上面历史不完整;部署后新发的继续消息会正常显示。"]}):null,i.chatHistory&&i.chatHistory.trace_note?c.jsx("div",{className:"muted",children:i.chatHistory.trace_note}):null,c.jsx(yg,{messages:h,empty:"(暂无会话消息)"}),c.jsx(gg,{disabled:!C,placeholder:"消息",onSend:H=>void i.sendChatMessage(H)}),c.jsxs("div",{className:"muted",children:[A,"。",i.chatMsg]}),c.jsx("div",{className:"muted",children:"对话时间线:首条是本任务 prompt(用户);助手回复、命令/文件/工具/调用、计划/思考/子代理开始结束都在同一列里,不再另起一份链路名单。流式进行中时新条目实时追加;你在底部时自动跟随,往上翻时暂停跟随,点“回到最新”恢复。"})]})}function Sg(){return Zt().currentTask?c.jsx("div",{className:"conversation",children:c.jsx(pg,{})}):c.jsx("div",{className:"conversation",children:c.jsx("div",{className:"muted",children:"在任务列表里点一条任务,对话和详情就切到该任务;也可以在上面派发一个新任务。"})})}function xg(){const i=Zt(),r=i.currentTask,[h,f]=d.useState("");if(!r)return null;const g=uu.includes(r.status),C=["failed","interrupted","merge-failed"].includes(r.status);return c.jsxs(c.Fragment,{children:[c.jsxs(Ht,{children:[c.jsx("button",{disabled:!C,title:C?"":"仅 failed/interrupted/merge-failed 可重试",onClick:()=>i.doAction("retry"),children:"重试"}),c.jsx("button",{className:"danger",disabled:g,title:g?"终态任务不可取消":"",onClick:()=>i.doAction("cancel"),children:"取消"}),c.jsx("button",{disabled:r.status!=="completed",title:r.status!=="completed"?"仅 completed 可合并":"",onClick:()=>i.doAction("merge"),children:"合并(completed → merged)"}),c.jsx("button",{disabled:r.status!=="merged",title:r.status!=="merged"?"仅 merged 可提 PR":"",onClick:()=>i.doAction("pr"),children:"提 PR(须 mergedLine truncated
|
||||
`),N(P||"(暂无 result/error 输出)")}catch{}ee(B.id),Pe(B.id),qe(B.id),_l(B.id),ma(B),rl(B.id),ol(B.id),$t(B.id),Ul(B.id),Dl(B.id)}},[ee,Pe,qe,_l,ma,rl,ol,$t,Ul,Dl]),ha=d.useCallback(async B=>{const U=oe.current;if(!U)return;const P={cancel:"确定取消该任务吗?",merge:"确定把 completed 任务标为 merged 吗?",pr:"确定为该任务提 PR 吗?(会在仓库真实建 PR)"};if(!(P[B]&&!window.confirm(P[B])))try{const it=await lt(W.taskAction(U.id,B),"POST",{});j(B+" 已提交:"+JSON.stringify(it).slice(0,300)),r(),await be(U.id)}catch(it){j(B+" 失败:"+it.message)}},[be,r]),su=d.useCallback(async()=>{const B=oe.current;if(B)try{const U=await lt(W.taskCost(B.id));g(JSON.stringify(U,null,2).slice(0,4e3))}catch(U){g("查成本失败:"+U.message)}},[]),ru=d.useCallback(async B=>{const U=oe.current;if(U){if(!B.trim()){vt("分支消息必填");return}if(!uu.includes(U.status)){vt("仅终态任务可分支");return}try{const P=await lt(W.taskFork(U.id),"POST",{message:B.trim()});vt("分支任务已建:"+P.id.slice(0,8)),r()}catch(P){vt("分支失败:"+P.message)}}},[r]),Wl=d.useCallback(async B=>{const U=oe.current;if(U)try{await lt(W.taskReviews(U.id),"POST",B),dt("评审已登记(requested)"),await qe(U.id)}catch(P){dt("登记评审需求失败:"+P.message)}},[qe]),Me=d.useCallback(B=>{const U=oe.current;if(!U)return;const P=window.prompt("输入该条评审的 findings(将写回记录并标为 completed):");P!==null&<(W.taskReviewFindings(U.id,B),"POST",{findings:P}).then(()=>void qe(U.id)).catch(it=>dt("写 findings 失败:"+it.message))},[qe]),fl=d.useCallback((B,U)=>{const P=oe.current;if(!P)return;const it=U==="approved"?"approve":"deny";lt(W.taskApprovalDecision(P.id,B,it),"POST",{}).then(()=>{bt("已记录"+(U==="approved"?"允许":"拒绝")+"(当前版本只留回执,不拦截实时行为)"),_l(P.id)}).catch(A=>bt("决定失败:"+A.message))},[_l]),Oe=d.useCallback(async(B,U,P)=>{const it=oe.current;if(it){if(!U.trim()){re("目标任务 id 不能为空");return}if(!P.trim()){re("消息正文不能为空");return}try{const A=await lt(W.taskMessagesTo(U.trim()),"POST",{from_task_id:it.id,body:P.trim(),mode:B});re("已发送("+A.delivery+")"),await ol(it.id)}catch(A){re("发送失败:"+A.message)}}},[ol]),ou=d.useCallback((B,U)=>{const P=oe.current;if(!P)return;if(!["completed","failed","interrupted"].includes(P.status)){ye("仅 completed/failed/interrupted 任务可操作线程(merged/merge-failed 不可改,服务端同样 409)");return}const it={};if(B==="rename"){if(!U.trim()){ye("新线程名不能为空");return}it.name=U.trim()}lt(W.taskThreadAction(P.id,B),"POST",it).then(A=>{ye(B+" 已提交:"+JSON.stringify(A).slice(0,300)),$t(P.id),Ul(P.id)}).catch(A=>ye(B+" 失败:"+A.message))},[$t,Ul]),fu=d.useCallback(async B=>{const U=oe.current;if(!U)return;const P=B.trim();if(!P){Le("消息不能为空");return}try{uu.includes(U.status)&&U.status!=="merged"?(await lt(W.taskContinue(U.id),"POST",{message:P}),Le("已重新排队,同一会话继续"),r(),await be(U.id)):["queued","running"].includes(U.status)?(await lt(W.taskFollowups(U.id),"POST",{body:P,mode:"direct"}),Le("已直接发送"),await ee(U.id),await Pe(U.id)):Le("merged/merge-failed 任务不可继续,也不可发后续消息")}catch(it){Le("发送失败:"+it.message)}},[be,r,ee,Pe]),du=d.useCallback(async()=>{const B=oe.current;if(!B){se("先选一个任务");return}try{const U=await lt(W.taskTrace(B.id)),P=new Blob([JSON.stringify({task_id:B.id,status:U.status,note:U.note||"",count:(U.entries||[]).length,entries:U.entries||[]},null,2)],{type:"application/json"}),it=document.createElement("a");it.href=URL.createObjectURL(P),it.download="trace-"+B.id.slice(0,8)+".json",document.body.appendChild(it),it.click(),it.remove(),setTimeout(()=>URL.revokeObjectURL(it.href),1e3),se("已下载完整链路 "+(U.entries||[]).length+" 条")}catch(U){se("下载失败:"+U.message)}},[]);return d.useEffect(()=>()=>fe(),[fe]),{currentTask:h,detailBody:p,actionMsg:C,actionReasons:z,costBox:q,costSummary:k,forkVisible:X,setForkVisible:J,forkMsg:Q,coordChildren:St,agents:K,agentsNote:w,reviews:at,reviewMsg:qt,reviewError:Ut,traceEntries:ot,traceNote:G,traceDone:st,traceKind:xt,setTraceKind:Se,traceMsg:L,followups:y,approvals:nt,approvalMsg:Ct,peerMessages:Tt,peerMsg:mt,threadText:Ga,threadGateNote:Xe,threadMsg:Vt,threadList:Jl,chatMsg:da,chatHistory:zl,showDetail:be,refreshDetailLists:Qa,doAction:ha,taskCost:su,forkTask:ru,postReview:Wl,submitFindings:Me,decideApproval:fl,postPeer:Oe,threadAction:ou,sendChatMessage:fu,downloadTrace:du,refreshReviews:()=>void qe(),refreshApprovals:()=>void _l()}}function og(i){const[r,h]=d.useState([]),[f,p]=d.useState(""),[N,C]=d.useState(null),[j,z]=d.useState(""),R=d.useCallback(async X=>{const J=X!==void 0?X:i();if(!J){h([]);return}try{const Q=await lt(W.projectAutomations(J));h(Q.automations||[])}catch(Q){p("自动化加载失败:"+Q.message)}},Line truncated
|
||||
(链路摘要已截断至 300 字,完整回复见右侧任务结果)`:f,label:"助手"}}if(i.kind==="exec"||i.kind==="file"||i.kind==="tool"||i.kind==="call"){const f=i.kind==="exec"?"命令":i.kind==="file"?"文件":i.kind==="tool"?"工具":"调用";return{id:h,role:"tool",kind:i.kind,text:po(i),label:f}}return i.kind==="plan"?{id:h,role:"status",kind:i.kind,text:i.text||""}:i.kind==="reasoning"?{id:h,role:"status",kind:i.kind,text:"思考:"+(i.summary||"")}:i.kind==="agent-start"?{id:h,role:"status",kind:i.kind,text:"子代理开始:"+(i.name||"")+(i.prompt?"("+i.prompt+")":"")}:i.kind==="agent-done"?{id:h,role:"status",kind:i.kind,text:"子代理结束:"+(i.name||"")+" "+(i.status||"")}:{id:h,role:"status",kind:i.kind,text:po(i)}}function Cg(){const i=Kt(),r=i.currentTask,h=d.useMemo(()=>{if(!r)return[];let z;if(i.chatHistory){z=i.chatHistory.messages.map(X=>({id:X.id,role:X.role,text:X.text,label:X.label,kind:X.kind}));const k=new Set(z.map(X=>X.id)),F=i.traceEntries.filter(X=>X.id&&!k.has(X.id)&&!String(X.id).startsWith("fu-")).map((X,J)=>Fh(X,J));z=[...z,...F]}else{const k=r.prompt?[{id:"task-prompt",role:"user",text:r.prompt}]:[],F=i.traceEntries.filter(J=>!String(J.id||"").startsWith("fu-")).map((J,Q)=>Fh(J,Q)),X=i.followups.map((J,Q)=>({id:"followup-"+Q,role:"user",text:String(J.body||""),label:"后续消息"}));z=[...k,...F,...X]}const R=i.traceKind,q=R?z.filter(k=>k.role==="user"||!k.kind||k.kind===R):z,g=i.traceDone?[{id:"trace-done",role:"status",text:"链路已结束("+i.traceDone+")"}]:[];return[...q,...g]},[r,i.chatHistory,i.traceEntries,i.followups,i.traceKind,i.traceDone]);if(!r)return c.jsx("div",{className:"muted",children:"点任务行查看详情后,这里显示继续会话"});const f=uu.includes(r.status),p=["queued","running"].includes(r.status),N=f||p,C=f?"终态任务:发送走继续接口(同一会话再跑一轮)":p?"运行中/排队中:发送走后续消息直接发送(跑完当前轮后优先执行)":"merged/merge-failed 任务不可继续,也不可发后续消息",j=i.chatHistory?i.chatHistory.legacy_continue_count??null:null;return c.jsxs("div",{className:"chat-column",children:[c.jsx("div",{className:"muted",children:"实时对话"}),c.jsxs(Ht,{children:[c.jsxs("select",{"aria-label":"链路类型",value:i.traceKind,onChange:z=>i.setTraceKind(z.target.value),children:[c.jsx("option",{value:"",children:"链路类型:全部"}),["exec","file","call","tool","message","reasoning","plan","agent-start","agent-done"].map(z=>c.jsx("option",{value:z,children:z==="exec"?"命令":z==="file"?"文件":z==="call"?"调用":z==="tool"?"工具":z==="message"?"消息":z==="reasoning"?"思考":z==="plan"?"计划":"子代理"},z))]}),c.jsx("button",{onClick:i.downloadTrace,children:"下载完整链路"}),c.jsx("span",{className:"muted",children:i.traceMsg})]}),i.traceNote?c.jsx("div",{children:c.jsx("span",{className:"err",children:i.traceNote})}):null,j!==null&&j>0?c.jsxs("div",{className:"muted",children:["此前有 ",j," 条继续消息发生在持久化之前,未入库,上面历史不完整;部署后新发的继续消息会正常显示。"]}):null,i.chatHistory&&i.chatHistory.trace_note?c.jsx("div",{className:"muted",children:i.chatHistory.trace_note}):null,c.jsx(Eg,{messages:h,empty:"(暂无会话消息)"}),c.jsx(Ng,{disabled:!N,placeholder:"消息",onSend:z=>void i.sendChatMessage(z)}),c.jsxs("div",{className:"muted",children:[C,"。",i.chatMsg]}),c.jsx("div",{className:"muted",children:"对话时间线:首条是本任务 prompt(用户);助手回复、命令/文件/工具/调用、计划/思考/子代理开始结束都在同一列里,不再另起一份链路名单。流式进行中时新条目实时追加;你在底部时自动跟随,往上翻时暂停跟随,点“回到最新”恢复。"})]})}function Ag(){return Kt().currentTask?c.jsx("div",{className:"conversation",children:c.jsx(Cg,{})}):c.jsx("div",{className:"conversation",children:c.jsx("div",{className:"muted",children:"在任务列表里点一条任务,对话和详情就切到该任务;也可以在上面派发一个新任务。"})})}function Mg(){const i=Kt(),r=i.currentTask,[h,f]=d.useState("");if(!r)return null;const p=uu.includes(r.status),N=["failed","interrupted","merge-failed"].includes(r.status);return c.jsxs(c.Fragment,{children:[c.jsxs(Ht,{children:[c.jsx("button",{disabled:!N,title:N?"":"仅 failed/interrupted/merge-failed 可重试",onClick:()=>i.doAction("retry"),children:"重试"}),c.jsx("button",{className:"danger",disabled:p,title:p?"终态任务不可取消":"",onClick:()=>i.doAction("cancel"),children:"取消"}),c.jsx("button",{disabled:r.status!=="completed",title:r.status!=="completed"?"仅 completed 可合并":"",onClick:()=>i.doAction("merge"),children:"合并(completed → merged)"}),c.jsx("button",{disabled:r.status!=="merged",title:r.status!=="merged"?"仅 merged 可提 PR":"",onClick:()=>i.doAction("pr"),children:"提 PR(须 mergedLine truncated
|
||||
|
||||
`);for(;ot>=0;){const rt=vt.slice(0,ot);vt=vt.slice(ot+2);for(const G of rt.split(`
|
||||
`))if(G.startsWith("data: "))try{const N=JSON.parse(G.slice(6));N.waiting?S(N.task_status==="interrupted"?"任务已中断,等待续跑…":N.task_status==="paused"?"任务已暂停…":"等待会话…"):N.image&&(f("data:image/jpeg;base64,"+N.image),S("直播中"));const ut={};N.holder&&(ut.holder=N.holder),N.url&&(ut.url=N.url),N.title!==void 0&&(ut.title=N.title),Object.keys(ut).length>0&&C(St=>({...St||{},...ut}))}catch{}ot=vt.indexOf(`
|
||||
`);for(;ot>=0;){const yt=dt.slice(0,ot);dt=dt.slice(ot+2);for(const G of yt.split(`
|
||||
`))if(G.startsWith("data: "))try{const Y=JSON.parse(G.slice(6));Y.waiting?j(Y.task_status==="interrupted"?"任务已中断,等待续跑…":Y.task_status==="paused"?"任务已暂停…":"等待会话…"):Y.image&&(f("data:image/jpeg;base64,"+Y.image),j("直播中"));const st={};Y.holder&&(st.holder=Y.holder),Y.url&&(st.url=Y.url),Y.title!==void 0&&(st.title=Y.title),Object.keys(st).length>0&&N(gt=>({...gt||{},...st}))}catch{}ot=dt.indexOf(`
|
||||
|
||||
`)}}S("流已断开")}catch(et){if(et.name==="AbortError")return;R(et instanceof Error?et.message:String(et)),S("未开始")}finally{x(!1)}},[r]),X=d.useCallback(()=>{var w;(w=k.current)==null||w.abort(),x(!1),S("已停止")},[]),ht=d.useCallback(async w=>{if(r)try{const Q=await tt(K.taskBrowserControl(r.id),"POST",{holder:w});C(Q),R("")}catch(Q){R(Q instanceof Error?Q.message:String(Q))}},[r]),gt=d.useCallback(async w=>{if(!r||(g==null?void 0:g.holder)!=="user"||!W.current)return;const Q=W.current.getBoundingClientRect(),et=Math.round((w.clientX-Q.left)/Q.width*1280),zt=Math.round((w.clientY-Q.top)/Q.height*800);try{await tt(K.taskBrowserInput(r.id),"POST",{type:"mouse_click",x:et,y:zt})}catch(Bt){R(Bt instanceof Error?Bt.message:String(Bt))}},[r,g]),xt=d.useCallback(async w=>{if(!(!r||(g==null?void 0:g.holder)!=="user")){w.preventDefault();try{w.key.length===1?await tt(K.taskBrowserInput(r.id),"POST",{type:"type",text:w.key}):await tt(K.taskBrowserInput(r.id),"POST",{type:"key",key:w.key})}catch(Q){R(Q instanceof Error?Q.message:String(Q))}}},[r,g]);if(!r)return c.jsx("div",{className:"muted",children:"点任务行查看详情"});const Z=(g==null?void 0:g.holder)||"—",Tt=g!==null;return c.jsxs("div",{children:[c.jsxs("div",{style:{display:"flex",gap:8,flexWrap:"wrap",alignItems:"center"},children:[c.jsxs("span",{children:["状态 ",c.jsx("b",{children:A})]}),c.jsxs("span",{className:"muted",children:["会话 ",Tt?"活跃":"未激活(轮次结束即消失)"]}),c.jsxs("span",{className:"muted",children:["控制权 ",Z]}),g!=null&&g.url?c.jsx("span",{className:"muted",title:g.url,children:g.url.slice(0,80)}):null]}),c.jsxs("div",{style:{display:"flex",gap:8,flexWrap:"wrap",marginTop:8},children:[Y?c.jsx("button",{type:"button",onClick:X,children:"停止观看"}):c.jsx("button",{type:"button",onClick:()=>void I(),children:"开始观看"}),c.jsx("button",{type:"button",onClick:()=>void V(),children:"刷新状态"}),Z!=="user"?c.jsx("button",{type:"button",onClick:()=>void ht("user"),disabled:!Tt,children:"接管"}):c.jsx("button",{type:"button",onClick:()=>void ht("agent"),children:"交还给 Agent"})]}),H?c.jsx("div",{className:"muted",style:{marginTop:8},children:H}):null,Z==="user"?c.jsx("div",{className:"muted",style:{marginTop:8},children:"你已接管:点画面即点击,键盘输入直达页面。交还后 Agent 续跑前会先快照确认你的改动。"}):null,c.jsx("div",{style:{marginTop:8},children:h?c.jsx("img",{ref:W,src:h,alt:"浏览器实时画面",tabIndex:0,onClick:w=>void gt(w),onKeyDown:w=>void xt(w),style:{width:"100%",maxWidth:1280,cursor:Z==="user"?"crosshair":"default",border:"1px solid var(--border, #ccc)"}}):c.jsx("div",{className:"muted",children:Tt?"点“开始观看”接入画面流(接管后点画面即点击)":"浏览器会话未激活:任务本轮未开浏览器,或轮次已结束。"})})]})}function jg(){const i=Zt(),[r,h]=d.useState("uncommittedChanges"),[f,g]=d.useState(""),[C,A]=d.useState(""),[S,H]=d.useState("");return c.jsxs(c.Fragment,{children:[c.jsx("h3",{children:"评审队列"}),c.jsxs("select",{"aria-label":"评审目标",value:r,onChange:R=>h(R.target.value),children:[c.jsx("option",{value:"uncommittedChanges",children:"未提交改动(工作区)"}),c.jsx("option",{value:"baseBranch",children:"与分支的差异"}),c.jsx("option",{value:"commit",children:"指定提交"}),c.jsx("option",{value:"custom",children:"自定义评审指令"})]}),c.jsx("input",{className:"stacked",placeholder:"与分支的差异用:分支名(如 main)",style:{width:"100%"},value:f,onChange:R=>g(R.target.value)}),c.jsx("input",{className:"stacked",placeholder:"指定提交用:提交 sha",style:{width:"100%"},value:C,onChange:R=>A(R.target.value)}),c.jsx("textarea",{placeholder:"自定义评审指令用:评审指令正文",value:S,onChange:R=>H(R.target.value)}),c.jsxs(Ht,{children:[c.jsx("button",{className:"primary",onClick:()=>void i.postReview({target_type:r,branch:f.trim()||void 0,sha:C.trim()||void 0,instructions:S.trim()||void 0}),children:"登记评审需求"}),c.jsx("button",{onClick:i.refreshReviews,children:"刷新队列"}),c.jsx("span",{className:"muted",children:i.reviewMsg})]}),c.jsx("div",{className:"muted",children:"本基线只登记评审需求(requested),不自动跑评审;评审是否执行、结果如何,以评审队列里的记录为准。"}),i.reviewError?c.jsx("div",{className:"muted",children:i.reviewError}):i.reviews.length===0?c.jsx("div",{className:"muted",children:"点任务行查看详情后,点“刷新队列”查看评审记录"}):c.jsx(cu,{children:i.reviews.map(R=>{let Y=R.target_type||"";return R.branch&&(Y+=":"+R.branch),R.sha&&(Y+=":"+String(R.sha).slice(0,12)),R.instructions&&(Y+=":"+String(R.instructions).slice(0,80)),c.jsxs("li",{children:[c.jsxs("span",{className:"row-main",children:[c.jsx("b",{children:"评审"})," ",Ne(R.id)," | ",Y," | ",c.jsx("span",{className:"st-"+(R.status||""),children:RLine truncated
|
||||
`)}}j("流已断开")}catch(at){if(at.name==="AbortError")return;R(at instanceof Error?at.message:String(at)),j("未开始")}finally{g(!1)}},[r]),Q=d.useCallback(()=>{var w;(w=k.current)==null||w.abort(),g(!1),j("已停止")},[]),vt=d.useCallback(async w=>{if(r)try{const V=await lt(W.taskBrowserControl(r.id),"POST",{holder:w});N(V),R("")}catch(V){R(V instanceof Error?V.message:String(V))}},[r]),St=d.useCallback(async w=>{if(!r||(p==null?void 0:p.holder)!=="user"||!F.current)return;const V=F.current.getBoundingClientRect(),at=Math.round((w.clientX-V.left)/V.width*1280),Dt=Math.round((w.clientY-V.top)/V.height*800);try{await lt(W.taskBrowserInput(r.id),"POST",{type:"mouse_click",x:at,y:Dt})}catch(qt){R(qt instanceof Error?qt.message:String(qt))}},[r,p]),jt=d.useCallback(async w=>{if(!(!r||(p==null?void 0:p.holder)!=="user")){w.preventDefault();try{w.key.length===1?await lt(W.taskBrowserInput(r.id),"POST",{type:"type",text:w.key}):await lt(W.taskBrowserInput(r.id),"POST",{type:"key",key:w.key})}catch(V){R(V instanceof Error?V.message:String(V))}}},[r,p]);if(!r)return c.jsx("div",{className:"muted",children:"点任务行查看详情"});const K=(p==null?void 0:p.holder)||"—",Nt=p!==null;return c.jsxs("div",{children:[c.jsxs("div",{style:{display:"flex",gap:8,flexWrap:"wrap",alignItems:"center"},children:[c.jsxs("span",{children:["状态 ",c.jsx("b",{children:C})]}),c.jsxs("span",{className:"muted",children:["会话 ",Nt?"活跃":"未激活(轮次结束即消失)"]}),c.jsxs("span",{className:"muted",children:["控制权 ",K]}),p!=null&&p.url?c.jsx("span",{className:"muted",title:p.url,children:p.url.slice(0,80)}):null]}),c.jsxs("div",{style:{display:"flex",gap:8,flexWrap:"wrap",marginTop:8},children:[q?c.jsx("button",{type:"button",onClick:Q,children:"停止观看"}):c.jsx("button",{type:"button",onClick:()=>void J(),children:"开始观看"}),c.jsx("button",{type:"button",onClick:()=>void X(),children:"刷新状态"}),K!=="user"?c.jsx("button",{type:"button",onClick:()=>void vt("user"),disabled:!Nt,children:"接管"}):c.jsx("button",{type:"button",onClick:()=>void vt("agent"),children:"交还给 Agent"})]}),z?c.jsx("div",{className:"muted",style:{marginTop:8},children:z}):null,K==="user"?c.jsx("div",{className:"muted",style:{marginTop:8},children:"你已接管:点画面即点击,键盘输入直达页面。交还后 Agent 续跑前会先快照确认你的改动。"}):null,c.jsx("div",{style:{marginTop:8},children:h?c.jsx("img",{ref:F,src:h,alt:"浏览器实时画面",tabIndex:0,onClick:w=>void St(w),onKeyDown:w=>void jt(w),style:{width:"100%",maxWidth:1280,cursor:K==="user"?"crosshair":"default",border:"1px solid var(--border, #ccc)"}}):c.jsx("div",{className:"muted",children:Nt?"点“开始观看”接入画面流(接管后点画面即点击)":"浏览器会话未激活:任务本轮未开浏览器,或轮次已结束。"})})]})}function zg(){const i=Kt(),[r,h]=d.useState("uncommittedChanges"),[f,p]=d.useState(""),[N,C]=d.useState(""),[j,z]=d.useState("");return c.jsxs(c.Fragment,{children:[c.jsx("h3",{children:"评审队列"}),c.jsxs("select",{"aria-label":"评审目标",value:r,onChange:R=>h(R.target.value),children:[c.jsx("option",{value:"uncommittedChanges",children:"未提交改动(工作区)"}),c.jsx("option",{value:"baseBranch",children:"与分支的差异"}),c.jsx("option",{value:"commit",children:"指定提交"}),c.jsx("option",{value:"custom",children:"自定义评审指令"})]}),c.jsx("input",{className:"stacked",placeholder:"与分支的差异用:分支名(如 main)",style:{width:"100%"},value:f,onChange:R=>p(R.target.value)}),c.jsx("input",{className:"stacked",placeholder:"指定提交用:提交 sha",style:{width:"100%"},value:N,onChange:R=>C(R.target.value)}),c.jsx("textarea",{placeholder:"自定义评审指令用:评审指令正文",value:j,onChange:R=>z(R.target.value)}),c.jsxs(Ht,{children:[c.jsx("button",{className:"primary",onClick:()=>void i.postReview({target_type:r,branch:f.trim()||void 0,sha:N.trim()||void 0,instructions:j.trim()||void 0}),children:"登记评审需求"}),c.jsx("button",{onClick:i.refreshReviews,children:"刷新队列"}),c.jsx("span",{className:"muted",children:i.reviewMsg})]}),c.jsx("div",{className:"muted",children:"本基线只登记评审需求(requested),不自动跑评审;评审是否执行、结果如何,以评审队列里的记录为准。"}),i.reviewError?c.jsx("div",{className:"muted",children:i.reviewError}):i.reviews.length===0?c.jsx("div",{className:"muted",children:"点任务行查看详情后,点“刷新队列”查看评审记录"}):c.jsx(cu,{children:i.reviews.map(R=>{let q=R.target_type||"";return R.branch&&(q+=":"+R.branch),R.sha&&(q+=":"+String(R.sha).slice(0,12)),R.instructions&&(q+=":"+String(R.instructions).slice(0,80)),c.jsxs("li",{children:[c.jsxs("span",{className:"row-main",children:[c.jsx("b",{children:"评审"})," ",Ae(R.id)," | ",q," | ",c.jsx("span",{className:"st-"+(R.status||""),children:RLine truncated
|
||||
<style rel="stylesheet" crossorigin>:root{--text:#1f2328;--muted:#656d76;--subtle:#6e7781;--line:#d0d7de;--line-muted:#d8dee4;--paper:#ffffff;--card:#ffffff;--subtle-bg:#f6f8fa;--inset-bg:#f6f8fa;--accent:#0969da;--accent-emphasis:#0969da;--green:#2da44e;--green-hover:#2c974b;--success:#1a7f37;--danger:#d1242f;--danger-bg:#ffebe9;--attention:#9a6700;--done:#8250df;--selected-bg:#ddf4ff;--radius:6px;--shadow:0 1px 0 rgba(27,31,36,.04);--sans:-apple-system,BlinkMacSystemFont,"Segoe UI","Noto Sans",Helvetica,Arial,sans-serif;--mono:ui-monospace,SFMono-Regular,SFMono-Regular,Menlo,Consolas,"Liberation Mono",monospace;--ctl:32px}*{box-sizing:border-box}html,body,#root{height:100%}body{font-family:var(--sans);margin:0;color:var(--text);background:#fff;line-height:1.5;font-size:14px}a{color:var(--accent);text-underline-offset:3px}:focus-visible{outline:2px solid var(--accent);outline-offset:2px}@media(prefers-reduced-motion:reduce){*{scroll-behavior:auto!important;transition:none!important}}.muted{color:var(--muted);font-size:12px;line-height:1.5}.err{color:var(--danger)}.ok{color:var(--success)}button{cursor:pointer;font-weight:500;border-radius:var(--radius);border:1px solid var(--line);background:#fff;color:var(--text);box-shadow:var(--shadow)}button:hover{background:var(--subtle-bg)}button:disabled{opacity:.45;cursor:not-allowed;box-shadow:none}button.primary,button.seal{background:var(--green);border-color:#1b1f2426;color:#fff}button.primary:hover,button.seal:hover{background:var(--green-hover)}button.danger{color:var(--danger);border-color:var(--line);background:#fff}button.danger:hover{background:var(--danger-bg);color:var(--danger)}.icon-btn{width:32px;height:32px;padding:0;display:grid;place-items:center;border:1px solid var(--line);border-radius:var(--radius);background:#fff;color:var(--text);font-size:16px;line-height:1}.icon-btn:hover{background:var(--subtle-bg)}.dash-topbar{position:sticky;top:0;z-index:10;background:#24292f;color:#fff;border-bottom:1px solid rgba(255,255,255,.15);padding:10px 20px 8px}.dash-topbar .muted{color:#ffffffbf}.dash-topbar-row{display:flex;gap:14px;align-items:center;flex-wrap:wrap}.dash-brand{font-weight:600;font-size:16px;color:#fff}.dash-status{font-size:13px;font-weight:600;color:#ffffffbf}.dash-status.ok{color:#2da44e}.dash-stamp{margin-left:auto}.dash-nav{display:flex;gap:4px;flex-wrap:wrap;margin-top:8px}.dash-nav a{color:#fff;text-decoration:none;font-size:13px;font-weight:600;padding:6px 12px;border-radius:var(--radius)}.dash-nav a:hover{background:#ffffff1f;text-decoration:none}.tab-nav button{background:transparent;border:0;box-shadow:none;color:#fff;font-size:13px;font-weight:600;padding:6px 12px;border-radius:var(--radius)}.tab-nav button:hover{background:#ffffff1f}.tab-nav button.active{background:#ffffff2e}.dash-page-tabs{display:flex}.dash-page-tabs>.tab-content{grid-column:1 / -1;min-width:0;display:flex;flex-direction:column;gap:20px}.tab-workbench{display:grid;grid-template-columns:320px minmax(0,1fr);gap:20px;align-items:start}.tab-workbench>*{min-width:0}@media(max-width:899px){.tab-workbench{grid-template-columns:1fr}.tab-workbench #sec-dispatch{order:1}.tab-workbench #sec-tasks{order:2}.tab-workbench #sec-conv{order:3}}.dash-page{max-width:1180px;margin:0 auto;padding:20px;display:flex;flex-direction:column;gap:20px}.dash-page>*{min-width:0}.dash-section>*{min-width:0}.dash-section{scroll-margin-top:130px;display:flex;flex-direction:column;gap:16px}.dash-section.card{padding:16px}.dash-cards{display:grid;grid-template-columns:repeat(3,1fr);gap:12px}.dash-card{background:#fff;border:1px solid var(--line);border-radius:var(--radius);padding:16px;min-width:0}.dash-card-label{color:var(--muted);font-size:12px;font-weight:600}.dash-card-value{font-size:24px;font-weight:600;line-height:1.3;overflow-wrap:anywhere}.card{background:var(--card);border:1px solid var(--line);border-radius:var(--radius);box-shadow:none;padding:16px}.card h2{font-size:14px;margin:0 0 12px;font-weight:600}.card h3{font-size:14px;margin:24px 0 12px;padding-top:16px;border-top:1px solid var(--line);font-weight:600}.card h3:first-of-type{border-top:0;padding-top:0}.composer h2{font-size:16px}.composer textarea.prompt{min-height:148px;font-size:14px;padding:12px;border-radius:var(--radius)}.composer textarea.coord{font-family:var(--mono);font-size:13px}.bar{display:flex;gap:8px;flex-wrap:wrap;align-items:center;margin:12px 0}.bar>button,.bar>label{display:inline-flex;align-items:center;height:var(--ctl);min-height:var(--ctl);padding:0 16px;box-sizing:border-box;white-space:nowrap}.bar>input,.bar>select{height:var(--ctl);min-height:var(--ctl)}input,select,textarea,button{font:inherit;font-size:14px;padding:5px 12px;border-radius:var(--radius);border:1px solid var(--line);background:#fff;color:var(--text)}input:focus,select:focus,textarea:focus{outline:none;border-color:var(--accent);box-shadow:0 0 0 3px #0969da4d}select{-moz-appearance:none;appearance:none;-webkit-appearance:none;cursLine truncated
|
||||
</head>
|
||||
<body>
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
"""执行器基类(叶子模块,不依赖 cloud.sandbox 与 cloud.chv,
|
||||
避免循环导入)。
|
||||
|
||||
原来在 cloud/sandbox.py 里;拆分后 Local/Docker/Cloud
|
||||
执行器(sandbox.py 内)与 CloudHypervisorExecutor
|
||||
(cloud/chv/executor.py)共用这一个类对象,isinstance
|
||||
判断与旧路径 `from cloud.sandbox import Executor` 都不变
|
||||
(sandbox.py 会 re-export 它)。
|
||||
"""
|
||||
|
||||
|
||||
class Executor:
|
||||
"""执行器基类。
|
||||
|
||||
task 是 dict,含 id、project_id、repo、prompt、branch(可选)。
|
||||
execute(task) -> dict,至少含 executor、task_id、status、stdout。
|
||||
"""
|
||||
|
||||
def execute(self, task) -> dict:
|
||||
raise NotImplementedError("子类必须实现 execute(task)")
|
||||
|
||||
def goal_task(self, task) -> dict:
|
||||
"""目标循环(host.py goal):turn→验测→喂回重试。
|
||||
|
||||
子类实现;默认不支持(CloudExecutor 的通用 HTTP 协议里
|
||||
没有 goal/验测概念,调它会如实失败,不静默降级成普通
|
||||
dispatch)。结果 dict 除 execute() 的字段外另含:
|
||||
goal_met(True/False/None=无结论)、goal_error。
|
||||
"""
|
||||
raise NotImplementedError("该执行器不支持 goal 模式")
|
||||
+199
-123
@@ -11,6 +11,13 @@
|
||||
时不启动容器);超时与取消时 docker kill + rm -f,不留僵尸
|
||||
容器。网络白名单不在本类里假装实现(见 cloud/README.md)。
|
||||
- CloudExecutor:占位,未接通云端执行。
|
||||
- CloudHypervisorExecutor:生产级 v1,Cloud Hypervisor VM 内经
|
||||
SSH 调 host.py(已拆到 cloud/chv/,API 唤醒流程见该类
|
||||
docstring:preflight → vmm.ping → vm.create → vm.boot →
|
||||
SSH 就绪 → 跑任务 → finally 关 VM;任务内 LLM 绝不能
|
||||
发现/调用它,执行器选择只在任务创建前由 server 白名单/
|
||||
环境默认值/worker 认领决定)。本模块只留它的兼容
|
||||
re-export,旧导入路径仍可用。
|
||||
"""
|
||||
|
||||
import json
|
||||
@@ -18,8 +25,10 @@ import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
from cloud.marker_tools import extract_quota_status as _extract_quota_status
|
||||
@@ -68,25 +77,10 @@ def _full_stdout(text):
|
||||
_DOCKER_LOGGER = logging.getLogger("cloud.docker")
|
||||
|
||||
|
||||
class Executor:
|
||||
"""执行器基类。
|
||||
|
||||
task 是 dict,含 id、project_id、repo、prompt、branch(可选)。
|
||||
execute(task) -> dict,至少含 executor、task_id、status、stdout。
|
||||
"""
|
||||
|
||||
def execute(self, task) -> dict:
|
||||
raise NotImplementedError("子类必须实现 execute(task)")
|
||||
|
||||
def goal_task(self, task) -> dict:
|
||||
"""目标循环(host.py goal):turn→验测→喂回重试。
|
||||
|
||||
子类实现;默认不支持(CloudExecutor 的通用 HTTP 协议里
|
||||
没有 goal/验测概念,调它会如实失败,不静默降级成普通
|
||||
dispatch)。结果 dict 除 execute() 的字段外另含:
|
||||
goal_met(True/False/None=无结论)、goal_error。
|
||||
"""
|
||||
raise NotImplementedError("该执行器不支持 goal 模式")
|
||||
# 执行器基类已移到叶子模块 cloud/executor_base.py(供
|
||||
# cloud/chv/executor.py 共用,避免循环导入);这里 re-export,
|
||||
# 旧路径 `from cloud.sandbox import Executor` 仍可用。
|
||||
from cloud.executor_base import Executor # noqa: F401
|
||||
|
||||
|
||||
def _clean_usage(raw):
|
||||
@@ -314,6 +308,133 @@ def _approval_cli_args(approval):
|
||||
return []
|
||||
|
||||
|
||||
def _host_command_options(model=None, effective_model=None,
|
||||
model_provider=None, effort=None,
|
||||
approval=None, sandbox=None):
|
||||
"""host.py 顶层全局参数(Local/Docker/CHV 三执行器共享,
|
||||
本轨去重复):顺序固定为 --model → --model-provider →
|
||||
--effort → approval 参数 → --sandbox,且必须放在子命令
|
||||
之前。approval=full 自带 danger-full-access,显式 sandbox
|
||||
在后(argparse 里后出现的 --sandbox 覆盖前面的,见
|
||||
LocalExecutor._base_command 的旧注释)。"""
|
||||
cmd = []
|
||||
effective = model or effective_model
|
||||
if effective:
|
||||
cmd += ["--model", effective]
|
||||
if model_provider:
|
||||
cmd += ["--model-provider", model_provider]
|
||||
if effort:
|
||||
cmd += ["--effort", effort]
|
||||
cmd += _approval_cli_args(approval)
|
||||
if sandbox:
|
||||
cmd += ["--sandbox", sandbox]
|
||||
return cmd
|
||||
|
||||
|
||||
def _call_runner_with_timeout(runner, cmd, timeout=None):
|
||||
"""runner 调用共享 helper(CHV 与 Docker 的 env=None 分支
|
||||
共用):timeout 给定时优先传 timeout,假 runner 只接受
|
||||
cmd 时 TypeError 回退到只传 cmd(超时护栏丢了,但绝不
|
||||
因此静默换执行器;worker 层 task_timeout 仍是外层硬护栏)。
|
||||
Docker 的 env 分支与 Local 的 _run_local 语义不同(env
|
||||
丢了必须 loud fail,绝不静默降级),所以它们不走这里。"""
|
||||
if timeout is None:
|
||||
return runner(cmd)
|
||||
try:
|
||||
return runner(cmd, timeout=timeout)
|
||||
except TypeError:
|
||||
return runner(cmd)
|
||||
|
||||
|
||||
def parse_goal_result(task_id, result, executor_name):
|
||||
"""host.py goal 输出解析的正典实现(Local 与 CHV 共享,
|
||||
本轨去重复;Docker 的 goal_task 仍保留内联版,见下文)。
|
||||
|
||||
解析用完整 stdout(结论在末尾),存库的 stdout 才截尾;
|
||||
returncode 非 0 时:已打印 met/unmet 结论的仍判 failed,
|
||||
无结论的异常退出判 interrupted(断点在盘上,可续跑,
|
||||
绝不能一律 failed);host 打印了 interrupted 标记时也判
|
||||
interrupted。为 0 时无结论仍判 failed,绝不把“没打印
|
||||
结论”当成达标。host 任务已存在时(同任务 /retry 撞名,
|
||||
goal 没有 run 续跑语义)错误写明“goal 失败请用继续而
|
||||
非重试”。断点丢失拒绝(goal-resume 拒绝行)判 failed
|
||||
终态,不进自动恢复。
|
||||
|
||||
Docker 未收敛的原因(有意保留,不是不小心漏掉):
|
||||
Docker 的 goal-resume 变体故意没有“已存在”分支
|
||||
(goal-resume 本来就是操作已有 host 任务,不存在
|
||||
dispatch 撞名问题),且它的超时证据路径与解析绑在
|
||||
一起;硬收敛会改 Docker 生产路径的语义,本轨不动它。
|
||||
三执行器在通用输入下结论一致由 test_chv_optimize.py
|
||||
的矩阵测试钉住(CHV 与 Local 全等;Docker 的通用
|
||||
goal 路径由既有 docker 测试覆盖)。"""
|
||||
returncode = getattr(result, "returncode", 1)
|
||||
stdout_full = getattr(result, "stdout", "") or ""
|
||||
stderr = getattr(result, "stderr", "") or ""
|
||||
met, goal_error = _parse_goal_output(stdout_full)
|
||||
interrupted_reason = _goal_interrupted_reason(stdout_full)
|
||||
refusal = _goal_resume_refusal(stdout_full)
|
||||
combined = stdout_full + stderr
|
||||
perm_kind, perm_snippet = _classify_goal_error(
|
||||
stdout_full, stderr)
|
||||
if refusal is not None and met is None:
|
||||
# 断点丢失拒绝:终态 failed,不进自动恢复。
|
||||
met, goal_error = None, refusal
|
||||
status = "failed"
|
||||
elif "已存在" in combined:
|
||||
met, goal_error = None, (
|
||||
"host 任务已存在:goal 失败请用继续而非重试——"
|
||||
"同任务重试会撞已存在 host-state,worker 会先"
|
||||
"尝试断点续跑,断点续跑仍失败时请用继续(不要"
|
||||
"重排同一任务)")
|
||||
status = "failed"
|
||||
elif interrupted_reason is not None:
|
||||
# host 侧已记 interrupted:异常中断不是失败,云端跟
|
||||
# 着记 interrupted,任务可 /goal-resume 断点续跑。
|
||||
status = "interrupted"
|
||||
goal_error = (interrupted_reason
|
||||
or "host goal 异常中断,断点可续跑")
|
||||
elif (met is None and returncode != 0
|
||||
and perm_kind == "permanent"):
|
||||
# P0 三件其二:永久错误(断点 KeyError、配置缺失等)
|
||||
# 的无结论异常退出——同代码同断点重试必然再失败,
|
||||
# 判 failed 终态进死信,绝不判 interrupted 空转。
|
||||
# host 已明言 interrupted 的在上一分支,不受影响。
|
||||
status = "failed"
|
||||
goal_error = perm_snippet or "goal 永久错误"
|
||||
elif returncode != 0:
|
||||
if met is None:
|
||||
# 无结论的异常退出(进程被杀、连接断后恢复时抛错
|
||||
# 等):断点在盘上,记 interrupted 保住续跑活路,
|
||||
# 绝不能一律 failed——2026-10-28 长 goal 实弹教训。
|
||||
status = "interrupted"
|
||||
goal_error = ((stderr.strip()[-500:] if stderr.strip()
|
||||
else stdout_full.strip()[-500:])
|
||||
or "host goal 异常退出(无结论),断点可续跑")
|
||||
else:
|
||||
status = "failed"
|
||||
elif met is True:
|
||||
status = "completed"
|
||||
elif met is False:
|
||||
status = "failed"
|
||||
else:
|
||||
status = "failed"
|
||||
goal_error = goal_error or "host goal 输出无结论"
|
||||
return {
|
||||
"executor": executor_name,
|
||||
"task_id": task_id,
|
||||
"status": status,
|
||||
"stdout": _goal_stdout_tail(stdout_full),
|
||||
"stderr": stderr,
|
||||
"returncode": returncode,
|
||||
"host_status": ("goal-met" if met is True else
|
||||
"goal-unmet" if met is False else None),
|
||||
"goal_met": met,
|
||||
"goal_error": goal_error,
|
||||
"usage": _parse_goal_usage(stdout_full),
|
||||
}
|
||||
|
||||
|
||||
class LocalExecutor(Executor):
|
||||
"""本地执行器:调 host.py CLI 子进程做执行封装。
|
||||
|
||||
@@ -356,22 +477,17 @@ class LocalExecutor(Executor):
|
||||
if self.config_path is not None:
|
||||
cmd += ["--config", self.config_path]
|
||||
# G9:顶层全局参数,必须放在子命令之前(调用方在
|
||||
# _base_command 之后才追加子命令)。
|
||||
# _base_command 之后才追加子命令)。选项顺序与
|
||||
# Docker/CHV 共享 _host_command_options;显式沙箱放
|
||||
# 最后:argparse 里后出现的 --sandbox 覆盖前面的
|
||||
# (approval=full 自带 danger-full-access);server 侧
|
||||
# 已拒绝 plan+full 同存,这里是第二道防线。
|
||||
if getattr(self, "ignore_host_budget", False):
|
||||
cmd += ["--ignore-host-budget"]
|
||||
effective = model or self.model
|
||||
if effective:
|
||||
cmd += ["--model", effective]
|
||||
if model_provider:
|
||||
cmd += ["--model-provider", model_provider]
|
||||
if effort:
|
||||
cmd += ["--effort", effort]
|
||||
cmd += _approval_cli_args(approval)
|
||||
# 显式沙箱放最后:argparse 里后出现的 --sandbox 覆盖前面
|
||||
# 的(approval=full 自带 danger-full-access);server 侧
|
||||
# 已拒绝 plan+full 同存,这里是第二道防线。
|
||||
if sandbox:
|
||||
cmd += ["--sandbox", sandbox]
|
||||
cmd += _host_command_options(
|
||||
model=model, effective_model=self.model,
|
||||
model_provider=model_provider, effort=effort,
|
||||
approval=approval, sandbox=sandbox)
|
||||
return cmd
|
||||
|
||||
def _run_local(self, cmd, extra_env):
|
||||
@@ -711,81 +827,15 @@ class LocalExecutor(Executor):
|
||||
def _goal_result_from_runner(self, task_id, result):
|
||||
"""把 host.py goal 的 runner 返回值解析成 goal 结果 dict。
|
||||
|
||||
解析用完整 stdout(结论在末尾),存库的 stdout 才截尾;
|
||||
returncode 非 0 时:已打印 met/unmet 结论的仍判 failed,
|
||||
无结论的异常退出判 interrupted(断点在盘上,可续跑,
|
||||
绝不能一律 failed);host 打印了 interrupted 标记时也判
|
||||
interrupted。为 0 时无结论仍判 failed,绝不把“没打印
|
||||
结论”当成达标。host 任务已存在时(同任务 /retry 撞名,
|
||||
goal 没有 run 续跑语义)错误写明“goal 失败请用继续而
|
||||
非重试”(P1:重试在已存在 host-state 上必失败,断点续
|
||||
跑才是正确路径,见 worker 的回落逻辑)。
|
||||
直接委托给共享的 parse_goal_result(正典语义,见该
|
||||
函数 docstring):解析用完整 stdout(结论在末尾),
|
||||
存库的 stdout 才截尾;returncode 非 0 时已打印结论
|
||||
的仍判 failed,无结论的异常退出判 interrupted;为 0
|
||||
时无结论仍判 failed;“已存在”撞名与断点丢失拒绝判
|
||||
failed 终态。本方法只定 executor="local",语义与旧
|
||||
内联版逐行一致(旧版就是正典的来源)。
|
||||
"""
|
||||
returncode = getattr(result, "returncode", 1)
|
||||
stdout_full = getattr(result, "stdout", "") or ""
|
||||
stderr = getattr(result, "stderr", "") or ""
|
||||
met, goal_error = _parse_goal_output(stdout_full)
|
||||
interrupted_reason = _goal_interrupted_reason(stdout_full)
|
||||
refusal = _goal_resume_refusal(stdout_full)
|
||||
combined = stdout_full + stderr
|
||||
perm_kind, perm_snippet = _classify_goal_error(
|
||||
stdout_full, stderr)
|
||||
if refusal is not None and met is None:
|
||||
# 断点丢失拒绝:终态 failed,不进自动恢复(见 helper)。
|
||||
met, goal_error = None, refusal
|
||||
status = "failed"
|
||||
elif "已存在" in combined:
|
||||
met, goal_error = None, (
|
||||
"host 任务已存在:goal 失败请用继续而非重试——"
|
||||
"同任务重试会撞已存在 host-state,worker 会先"
|
||||
"尝试断点续跑,断点续跑仍失败时请用继续(不要"
|
||||
"重排同一任务)")
|
||||
status = "failed"
|
||||
elif interrupted_reason is not None:
|
||||
# host 侧已记 interrupted:异常中断不是失败,云端跟
|
||||
# 着记 interrupted,任务可 /goal-resume 断点续跑。
|
||||
status = "interrupted"
|
||||
goal_error = (interrupted_reason
|
||||
or "host goal 异常中断,断点可续跑")
|
||||
elif (met is None and returncode != 0
|
||||
and perm_kind == "permanent"):
|
||||
# P0 三件其二:永久错误(断点 KeyError、配置缺失等)
|
||||
# 的无结论异常退出——同代码同断点重试必然再失败,
|
||||
# 判 failed 终态进死信,绝不判 interrupted 空转。
|
||||
# host 已明言 interrupted 的在上一分支,不受影响。
|
||||
status = "failed"
|
||||
goal_error = perm_snippet or "goal 永久错误"
|
||||
elif returncode != 0:
|
||||
if met is None:
|
||||
# 无结论的异常退出(进程被杀、连接断后恢复时抛错
|
||||
# 等):断点在盘上,记 interrupted 保住续跑活路,
|
||||
# 绝不能一律 failed——2026-10-28 长 goal 实弹教训。
|
||||
status = "interrupted"
|
||||
goal_error = ((stderr.strip()[-500:] if stderr.strip()
|
||||
else stdout_full.strip()[-500:])
|
||||
or "host goal 异常退出(无结论),断点可续跑")
|
||||
else:
|
||||
status = "failed"
|
||||
elif met is True:
|
||||
status = "completed"
|
||||
elif met is False:
|
||||
status = "failed"
|
||||
else:
|
||||
status = "failed"
|
||||
goal_error = goal_error or "host goal 输出无结论"
|
||||
return {
|
||||
"executor": "local",
|
||||
"task_id": task_id,
|
||||
"status": status,
|
||||
"stdout": _goal_stdout_tail(stdout_full),
|
||||
"stderr": stderr,
|
||||
"returncode": returncode,
|
||||
"host_status": ("goal-met" if met is True else
|
||||
"goal-unmet" if met is False else None),
|
||||
"goal_met": met,
|
||||
"goal_error": goal_error,
|
||||
"usage": _parse_goal_usage(stdout_full),
|
||||
}
|
||||
return parse_goal_result(task_id, result, "local")
|
||||
|
||||
def popen_task(self, task, kind="dispatch", seed_dir=None,
|
||||
extra_env=None, model_provider=None):
|
||||
@@ -1179,18 +1229,13 @@ class DockerExecutor(Executor):
|
||||
cmd += ["--config",
|
||||
"/app/" + os.path.basename(self.config_path)]
|
||||
# G9:与 LocalExecutor 同款,顶层全局参数,放子命令之前。
|
||||
# 选项顺序与 Local/CHV 共享 _host_command_options。
|
||||
if getattr(self, "ignore_host_budget", False):
|
||||
cmd += ["--ignore-host-budget"]
|
||||
effective = model or self.model
|
||||
if effective:
|
||||
cmd += ["--model", effective]
|
||||
if model_provider:
|
||||
cmd += ["--model-provider", model_provider]
|
||||
if effort:
|
||||
cmd += ["--effort", effort]
|
||||
cmd += _approval_cli_args(approval)
|
||||
if sandbox:
|
||||
cmd += ["--sandbox", sandbox]
|
||||
cmd += _host_command_options(
|
||||
model=model, effective_model=self.model,
|
||||
model_provider=model_provider, effort=effort,
|
||||
approval=approval, sandbox=sandbox)
|
||||
cmd += list(args)
|
||||
return cmd
|
||||
|
||||
@@ -1206,12 +1251,9 @@ class DockerExecutor(Executor):
|
||||
丢掉密钥跑任务(那样容器内 key 缺失,任务必失败还难查)。
|
||||
"""
|
||||
if env is None:
|
||||
try:
|
||||
return self.runner(
|
||||
cmd,
|
||||
timeout=self.timeout if timeout is None else timeout)
|
||||
except TypeError:
|
||||
return self.runner(cmd)
|
||||
return _call_runner_with_timeout(
|
||||
self.runner, cmd,
|
||||
self.timeout if timeout is None else timeout)
|
||||
try:
|
||||
return self.runner(
|
||||
cmd,
|
||||
@@ -1554,6 +1596,19 @@ class DockerExecutor(Executor):
|
||||
"container": self.container_name(task),
|
||||
"goal_met": None, "goal_error": str(exc)[:500],
|
||||
}
|
||||
# feat-chv-fix(P2-2,分叉说明 + 同步检查清单):
|
||||
# 以下是 Docker goal 解析的内联版,故意不收敛进正典
|
||||
# parse_goal_result()(本文件约 349 行起):① Docker
|
||||
# 的 goal-resume 变体没有“已存在”分支(goal-resume
|
||||
# 操作已有 host 任务,无 dispatch 撞名问题);② 超时
|
||||
# 证据路径(_docker_timeout_evidence)与本解析绑在
|
||||
# 一起(见本方法上方的 TimeoutExpired 分支)。
|
||||
# 同步检查清单——改正典时必须同步核对这里,反之亦然:
|
||||
# ① 非 0 退出但已打印 met 结论判 failed;② 无结论异常
|
||||
# 退出判 interrupted(断点可续跑);③ 永久错误无结论判
|
||||
# failed;④ interrupted 标记判 interrupted;⑤ rc==0
|
||||
# 无结论判 failed;⑥ “已存在”撞名判 failed 并提示用
|
||||
# 继续而非重试。改一处漏一处即回归,见 test 矩阵。
|
||||
returncode = getattr(result, "returncode", 1)
|
||||
stdout_full = getattr(result, "stdout", "") or ""
|
||||
stderr = getattr(result, "stderr", "") or ""
|
||||
@@ -2146,3 +2201,24 @@ class CloudExecutor(Executor):
|
||||
|
||||
def run_task(self, task) -> dict:
|
||||
return self.execute(task)
|
||||
|
||||
|
||||
|
||||
|
||||
# -- Cloud Hypervisor 执行器(已拆到 cloud/chv/,这里只留兼容 re-export) --
|
||||
# #216 的 CHV 类(原 2233-3342 行)与 WarmPool(原 3342-3437 行)、
|
||||
# _parse_memory_bytes、_unix_socket_api 已分别移到 cloud/chv/
|
||||
# 的 executor.py / experimental_pool.py / config.py /
|
||||
# api_client.py;本模块不再保留它们的副本。生产接口旧导入
|
||||
# 路径(from cloud.sandbox import CloudHypervisorExecutor /
|
||||
# ChvFailure)仍可用,新代码优先从 cloud.chv import。
|
||||
# WarmPool 除外:本轨破环性清理(feat-chv-arch-maint-fe,
|
||||
# 见 cloud/chv/__init__.py),`from cloud.sandbox import
|
||||
# WarmPool` 一律 ImportError,实验用法只走
|
||||
# cloud.chv.experimental_pool。执行器方法内按需 import
|
||||
# 本模块共享 helper 的原因见 cloud/chv/executor.py 头注释。
|
||||
from cloud.chv import ChvFailure # noqa: F401
|
||||
from cloud.chv import CloudHypervisorExecutor # noqa: F401
|
||||
from cloud.chv import reconcile_orphans_for_executor # noqa: F401
|
||||
from cloud.chv.api_client import _unix_socket_api # noqa: F401
|
||||
from cloud.chv.config import _parse_memory_bytes # noqa: F401
|
||||
+54
-6
@@ -494,6 +494,9 @@ def _task_worktree_for_files(server, task):
|
||||
"""
|
||||
if task.get("executor") == "cloud":
|
||||
return None, (409, "cloud task worktrees are not on this host")
|
||||
if task.get("executor") == "cloud-hypervisor":
|
||||
return None, (409, "cloud-hypervisor task worktrees are "
|
||||
"inside the VM, not on this host")
|
||||
worktree_host = task.get("worktree_host")
|
||||
if worktree_host and worktree_host != socket.gethostname():
|
||||
return None, (409, "task worktree is on host %s;"
|
||||
@@ -1550,7 +1553,7 @@ def _default_executor():
|
||||
raw = raw.strip()
|
||||
if raw == "":
|
||||
return None
|
||||
if raw in ("local", "docker", "cloud"):
|
||||
if raw in ("local", "docker", "cloud", "cloud-hypervisor"):
|
||||
return raw
|
||||
return "docker"
|
||||
|
||||
@@ -1763,6 +1766,30 @@ def _row_get(row, name):
|
||||
return None
|
||||
|
||||
|
||||
def _chv_vm_dead_from_row(row):
|
||||
"""CHV 只读字段 chv_vm_dead 的派生规则(本轨命名抽出,
|
||||
行为与 feat-chv-arch95plus 逐字一致,见 ADR-11/14)。
|
||||
|
||||
来源链(缺一环字段就不出现,绝不编造):
|
||||
执行器 _vm_death_result 写 result["vm_dead"]=True →
|
||||
worker 终局仅当 status='interrupted' 时把原因写进
|
||||
tasks.interrupted_reason(非 goal 形态是完整原因文本,
|
||||
goal 形态无 interrupted_reason 字段时回落为纯文本
|
||||
“VM 中途死亡”)→ 本函数只在 executor 为
|
||||
cloud-hypervisor 且原因文本含“VM 中途死亡”时返回
|
||||
True,其它情形返回 None(调用方不写字段)。
|
||||
本字段是“已报告死亡”,不是 live 状态:server 进程
|
||||
没有 VM 登记表,VM 活着/死了的 live 状态编不出来;
|
||||
前端无该字段时必须显示“未报告死亡”,不得写成活着。
|
||||
"""
|
||||
if _row_get(row, "executor") != "cloud-hypervisor":
|
||||
return None
|
||||
reason = _row_get(row, "interrupted_reason")
|
||||
if reason and "VM 中途死亡" in str(reason):
|
||||
return True
|
||||
return None
|
||||
|
||||
|
||||
def _task_to_dict(row, attachments=None, total_tokens=None):
|
||||
depends_on = row["depends_on"]
|
||||
try:
|
||||
@@ -1823,6 +1850,14 @@ def _task_to_dict(row, attachments=None, total_tokens=None):
|
||||
# 列表接口由 _task_tokens_map 一次聚合传入;其它调用方
|
||||
# 不传时为 None(前端显示占位),绝不把“未算”写成 0。
|
||||
"total_tokens": total_tokens,
|
||||
# feat-chv-arch-maint-fe:只读字段经
|
||||
# _chv_vm_dead_from_row 派生(规则见该函数;本轨
|
||||
# 未新增第二个生命周期字段——server 持久化的 CHV
|
||||
# 生命周期事实只有 executor/status/interrupted_reason
|
||||
# 三项,chv_vm_dead 已是其完整派生,再编字段就是
|
||||
# 复制 interrupted_reason 或猜 live 状态,两者都不做)。
|
||||
**({"chv_vm_dead": True}
|
||||
if _chv_vm_dead_from_row(row) else {}),
|
||||
}
|
||||
|
||||
|
||||
@@ -2492,9 +2527,11 @@ class CloudControlPlaneHandler(BaseHTTPRequestHandler):
|
||||
self._send_json(400, {"error": "name too long"})
|
||||
return
|
||||
executor = body.get("executor")
|
||||
if executor is not None and executor not in ("local", "docker"):
|
||||
if executor is not None and executor not in (
|
||||
"local", "docker", "cloud-hypervisor"):
|
||||
self._send_json(400, {
|
||||
"error": "executor must be local or docker"})
|
||||
"error": "executor must be local, docker or "
|
||||
"cloud-hypervisor"})
|
||||
return
|
||||
try:
|
||||
branch = _validate_env_branch(body.get("branch"))
|
||||
@@ -4248,12 +4285,15 @@ class CloudControlPlaneHandler(BaseHTTPRequestHandler):
|
||||
self._send_json(400, {"error": "priority must be an integer"})
|
||||
return
|
||||
# executor 可选:None 表示任意 worker 可认领;指定时只能是
|
||||
# local/docker/cloud,非法值 400 且不建任务。
|
||||
# local/docker/cloud/cloud-hypervisor,非法值 400 且不建任务。
|
||||
# cloud-hypervisor 是创建层可见的调度选择(任务内 LLM
|
||||
# 绝不能自选它,见 sandbox.py 该类 docstring)。
|
||||
executor = body.get("executor")
|
||||
if executor is not None and executor not in (
|
||||
"local", "docker", "cloud"):
|
||||
"local", "docker", "cloud", "cloud-hypervisor"):
|
||||
self._send_json(400, {
|
||||
"error": "executor must be local, docker or cloud"})
|
||||
"error": "executor must be local, docker, cloud or "
|
||||
"cloud-hypervisor"})
|
||||
return
|
||||
# 任务级选项:model 不做白名单(中转目录会变,下拉只是
|
||||
# 建议值);effort 必须是 codex 的 effort 档位;approval
|
||||
@@ -4481,6 +4521,10 @@ class CloudControlPlaneHandler(BaseHTTPRequestHandler):
|
||||
self._send_json(400, {
|
||||
"error": "附件不支持 cloud 执行器"})
|
||||
return
|
||||
if executor == "cloud-hypervisor":
|
||||
self._send_json(400, {
|
||||
"error": "附件不支持 cloud-hypervisor 执行器"})
|
||||
return
|
||||
conn0 = self._db()
|
||||
try:
|
||||
brow = conn0.execute(
|
||||
@@ -5893,6 +5937,10 @@ class CloudControlPlaneHandler(BaseHTTPRequestHandler):
|
||||
if executor == "cloud":
|
||||
return (409, "thread ops are local-executor only;"
|
||||
" cloud task threads are not on this host")
|
||||
if executor == "cloud-hypervisor":
|
||||
return (409, "thread ops are local-executor only;"
|
||||
" cloud-hypervisor task threads live inside"
|
||||
" the VM, not on this host")
|
||||
worktree_host = task.get("worktree_host")
|
||||
if worktree_host and worktree_host != socket.gethostname():
|
||||
return (409, "task worktree is on host %s;"
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
"""CHV 架构 9.5 提升轨测试(feat-chv-arch95,堆叠于 #246)。
|
||||
|
||||
只用标准库与 mock,绝不启动真实 VM。本文件是“严格测试替身”
|
||||
的载体:替身不再无条件放行——fake API 校验 VmConfig 内容
|
||||
(任务盘 + seed 盘、net 条目),fake SSH 校验 known_hosts
|
||||
钉死与 StrictHostKeyChecking=yes,mock 再也不能把被测逻辑
|
||||
本身也 mock 掉。测试里只用 FAKE 占位密钥,绝无真实密钥。
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
DockerExecutor,
|
||||
parse_goal_result,
|
||||
)
|
||||
except ImportError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
DockerExecutor,
|
||||
parse_goal_result,
|
||||
)
|
||||
|
||||
|
||||
class FakeResult:
|
||||
def __init__(self, returncode=0, stdout="", stderr=""):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
class FakeProc:
|
||||
def __init__(self, alive=True):
|
||||
self._alive = alive
|
||||
|
||||
def poll(self):
|
||||
return None if self._alive else 1
|
||||
|
||||
def terminate(self):
|
||||
self._alive = False
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self._alive = False
|
||||
|
||||
|
||||
def _write(path, content=b"x"):
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(content)
|
||||
os.chmod(path, 0o755)
|
||||
return path
|
||||
|
||||
|
||||
class Arch95Case(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.binary = _write(os.path.join(self.dir, "cloud-hypervisor"))
|
||||
self.ch_remote = _write(os.path.join(self.dir, "ch-remote"))
|
||||
self.kernel = _write(os.path.join(self.dir, "vmlinux"))
|
||||
self.rootfs = _write(os.path.join(self.dir, "disk.raw"), b"R" * 100)
|
||||
self.ssh_key = _write(os.path.join(self.dir, "id_chv"))
|
||||
self.kvm = _write(os.path.join(self.dir, "kvm"))
|
||||
self.cpuinfo = os.path.join(self.dir, "cpuinfo")
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu vmx sse\n")
|
||||
self.state_dir = os.path.join(self.dir, "state")
|
||||
self.calls = []
|
||||
self.audit = []
|
||||
self.api_payloads = []
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh-keygen":
|
||||
priv = cmd[cmd.index("-f") + 1]
|
||||
with open(priv, "w") as fh:
|
||||
fh.write("-----BEGIN OPENSSH PRIVATE KEY-----\nFAKE\n")
|
||||
os.chmod(priv, 0o600)
|
||||
with open(priv + ".pub", "w") as fh:
|
||||
fh.write("ssh-ed25519 AAAAfake1 vm-key\n")
|
||||
return FakeResult(0, "", "")
|
||||
if cmd and cmd[0] == "genisoimage":
|
||||
out = cmd[cmd.index("-output") + 1]
|
||||
with open(out, "wb") as fh:
|
||||
fh.write(b"CIDATA" * 16)
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
def _strict_api(self, socket_path, method, path, payload=None):
|
||||
"""严格替身:vm.create 必须带任务盘 + seed 盘、net 无重复。"""
|
||||
if path == "/vm.create":
|
||||
self.api_payloads.append(payload)
|
||||
disks = [d.get("path") for d in payload.get("disks", [])]
|
||||
self.assertEqual(len(disks), 2, "VmConfig 必须恰好两盘")
|
||||
self.assertTrue(disks[0].endswith("disk.raw"),
|
||||
"第一盘必须是任务盘")
|
||||
self.assertTrue(disks[1].endswith("seed.iso"),
|
||||
"第二盘必须是 seed 盘,缺 seed 即假保证")
|
||||
self.assertTrue(payload.get("kernel", {}).get("path"))
|
||||
nets = payload.get("net", [])
|
||||
taps = [n.get("tap") for n in nets]
|
||||
self.assertEqual(len(taps), len(set(taps)), "net tap 重复")
|
||||
return (200, {})
|
||||
|
||||
def make_exec(self, api=None, runner=None, env_extra=None):
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
"CHV_SSH_FORWARD": "1"}
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
ex = CloudHypervisorExecutor(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, rootfs=self.rootfs,
|
||||
ssh_key=self.ssh_key, kvm_path=self.kvm,
|
||||
cpuinfo_path=self.cpuinfo, state_dir=self.state_dir,
|
||||
runner=runner or self._runner,
|
||||
launcher=lambda cmd: FakeProc(alive=True),
|
||||
api_client=api or self._strict_api,
|
||||
sleeper=lambda s: None, env=env)
|
||||
ex.audit_callback = lambda e, t, d, k=None: self.audit.append(
|
||||
(e, t.get("id") if isinstance(t, dict) else t, d, k))
|
||||
return ex
|
||||
|
||||
|
||||
class WarmPoolDecisionTest(Arch95Case):
|
||||
def test_pool_is_experimental_and_dispatch_is_cold_only(self):
|
||||
self.assertFalse(WarmPool.wired_to_dispatch)
|
||||
ex = self.make_exec()
|
||||
pool = WarmPool(ex)
|
||||
# 手工放一个 warm 条目:dispatch 必须不消耗它(冷启动路径)。
|
||||
pool._entries["warm-1"] = {"state": "warm",
|
||||
"task": {"id": "warm-1"},
|
||||
"vm": "chv-warm-1",
|
||||
"snapshot_url": None}
|
||||
ex.wake_vm({"id": "t1"})
|
||||
self.assertEqual(pool._entries["warm-1"]["state"], "warm")
|
||||
self.assertEqual(pool.warm_count(), 1)
|
||||
|
||||
|
||||
class StrictVmConfigTest(Arch95Case):
|
||||
def test_wake_vm_config_has_task_disk_and_seed_disk(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.wake_vm({"id": "t1"})
|
||||
self.assertEqual(out["state"], "running")
|
||||
self.assertEqual(len(self.api_payloads), 1)
|
||||
# known_hosts 钉死:文件存在且含预置公钥,SSH 命令带钉死选项。
|
||||
kh = ex.known_hosts_path({"id": "t1"})
|
||||
with open(kh) as fh:
|
||||
content = fh.read()
|
||||
self.assertIn("AAAAfake1", content)
|
||||
cmd = ex.build_ssh_command({"id": "t1"}, ["true"])
|
||||
self.assertIn("StrictHostKeyChecking=yes", " ".join(cmd))
|
||||
self.assertIn(kh, " ".join(cmd))
|
||||
|
||||
def test_strict_ssh_runner_rejects_missing_known_hosts(self):
|
||||
def strict_runner(cmd, timeout=None):
|
||||
if cmd and cmd[0] == "ssh":
|
||||
self.assertIn("StrictHostKeyChecking=yes", cmd)
|
||||
found = False
|
||||
for i, a in enumerate(cmd):
|
||||
if a.startswith("UserKnownHostsFile="):
|
||||
path = a.split("=", 1)[1]
|
||||
found = True
|
||||
elif a == "UserKnownHostsFile":
|
||||
path = cmd[i + 1]
|
||||
found = True
|
||||
else:
|
||||
continue
|
||||
with open(path) as fh:
|
||||
self.assertTrue(fh.read().strip(),
|
||||
"known_hosts 为空即假钉死")
|
||||
self.assertTrue(found, "SSH 命令缺 known_hosts 钉死选项")
|
||||
return self._runner(cmd, timeout=timeout)
|
||||
ex = self.make_exec(runner=strict_runner)
|
||||
ex.wake_vm({"id": "t1"})
|
||||
# 严格 runner 在 _run_guest 里再跑一次 status 风格调用:
|
||||
# 这里直接验证 build 出的命令能过严格 runner 的检查。
|
||||
cmd = ex.build_ssh_command({"id": "t1"}, ["true"])
|
||||
strict_runner(cmd)
|
||||
|
||||
|
||||
class ConfigUnifiedTest(Arch95Case):
|
||||
def test_preflight_unified_config_invalid(self):
|
||||
cases = [
|
||||
("ssh_port", 0), ("ssh_port", 70000),
|
||||
("max_vms", -1), ("disk_gb", 0),
|
||||
("api_timeout", 0), ("ready_timeout", -1),
|
||||
("ready_interval", 0), ("warm_pool_size", -1),
|
||||
("snapshot_max_count", -1),
|
||||
("snapshot_max_total_gb", -1),
|
||||
("snapshot_retention_days", -1),
|
||||
("host_key_mode", "bogus"),
|
||||
]
|
||||
for field, bad in cases:
|
||||
with self.subTest(field=field, bad=bad):
|
||||
ex = self.make_exec()
|
||||
setattr(ex, field, bad)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "config_invalid")
|
||||
|
||||
|
||||
class DockerSyncGuardTest(unittest.TestCase):
|
||||
"""Docker goal 内联版与正典 parse_goal_result 的同步守卫:
|
||||
7 个真实标记用例 + 已存在撞名用例,两者状态必须全等;
|
||||
任一实现改了分叉,测试即 fail(不再只靠文档清单)。"""
|
||||
|
||||
def _docker_status(self, stdout, returncode, stderr=""):
|
||||
old = os.environ.get("XCM_API_KEY")
|
||||
os.environ["XCM_API_KEY"] = "FAKE-TEST-KEY"
|
||||
try:
|
||||
ex = DockerExecutor(runner=lambda cmd, timeout=None:
|
||||
FakeResult(returncode, stdout, stderr))
|
||||
# to10:goal 解析是纯函数测试,绝不依赖本机 docker
|
||||
# 二进制(无 docker 的机器上 _check_docker 会直接炸)。
|
||||
ex._check_docker = lambda: None
|
||||
ex._call_runner = lambda cmd, timeout=None, env=None: \
|
||||
FakeResult(returncode, stdout, stderr)
|
||||
return ex.goal_task({"id": "t1", "goal": "do it",
|
||||
"goal_max_rounds": 3})["status"]
|
||||
finally:
|
||||
if old is None:
|
||||
del os.environ["XCM_API_KEY"]
|
||||
else:
|
||||
os.environ["XCM_API_KEY"] = old
|
||||
|
||||
def test_docker_and_canonical_goal_status_equal(self):
|
||||
cases = [
|
||||
("goal met after rounds\n", 0, ""),
|
||||
("goal unmet after rounds\nround 1\nverify_out=fail\n", 0, ""),
|
||||
("goal interrupted after rounds: boom\n", 1, ""),
|
||||
("", 1, "boom"),
|
||||
("KeyError: breakpoint\n", 1, ""),
|
||||
("", 0, ""),
|
||||
("goal-resume 拒绝:breakpoint lost\n", 1, ""),
|
||||
("任务已存在\n", 1, ""),
|
||||
]
|
||||
for stdout, rc, stderr in cases:
|
||||
with self.subTest(stdout=stdout[:30], rc=rc):
|
||||
canon = parse_goal_result(
|
||||
"t1", FakeResult(rc, stdout, stderr), "local")
|
||||
docker = self._docker_status(stdout, rc, stderr)
|
||||
self.assertEqual(docker, canon["status"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,180 @@
|
||||
"""CHV 架构 9.5+ 提升轨测试(feat-chv-arch95plus,堆叠于 #252)。
|
||||
|
||||
只用标准库与 mock,绝不启动真实 VM;测试里只用 FAKE
|
||||
占位密钥,绝无真实密钥。覆盖:长函数提取后的行为不变、
|
||||
WarmPool facade 不调实验接口、#215 对齐接口形状、审计
|
||||
失败计数不中断、只读字段 chv_vm_dead 的派生规则、
|
||||
lockfile 校验脚本的通过/漂移/空清单。
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from cloud.chv.compat215 import resolve_inside, tmpfs_isolated
|
||||
from cloud.chv.hardening import validate_snapshot_url
|
||||
from cloud.chv.orphans import reconcile_orphans_for_executor
|
||||
from cloud.sandbox import CloudHypervisorExecutor
|
||||
from cloud.server import _task_to_dict
|
||||
except ImportError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv.compat215 import resolve_inside, tmpfs_isolated
|
||||
from cloud.chv.hardening import validate_snapshot_url
|
||||
from cloud.chv.orphans import reconcile_orphans_for_executor
|
||||
from cloud.sandbox import CloudHypervisorExecutor
|
||||
from cloud.server import _task_to_dict
|
||||
|
||||
|
||||
def _row(**kw):
|
||||
base = {
|
||||
"id": "t1", "project_id": "p1", "prompt": "x",
|
||||
"status": "interrupted", "idempotency_key": None,
|
||||
"depends_on": "[]", "priority": 0, "created_by": "u",
|
||||
"created_at": 0, "updated_at": 0, "last_heartbeat": 0,
|
||||
}
|
||||
base.update(kw)
|
||||
return base
|
||||
|
||||
|
||||
class FakeExecutor:
|
||||
def __init__(self):
|
||||
self.acquire_called = False
|
||||
self.prewarm_called = False
|
||||
|
||||
def shutdown_vm(self, vm):
|
||||
return True
|
||||
|
||||
def _emit(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
|
||||
class Compat215Test(unittest.TestCase):
|
||||
def test_resolve_inside_blocks_escape_and_symlink(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
inside = os.path.join(d, "a.txt")
|
||||
with open(inside, "w") as fh:
|
||||
fh.write("x")
|
||||
self.assertEqual(resolve_inside(d, "a.txt"), inside)
|
||||
self.assertIsNone(resolve_inside(d, "../outside"))
|
||||
self.assertIsNone(resolve_inside(d, "/etc/passwd"))
|
||||
self.assertEqual(resolve_inside(d, ""), d)
|
||||
|
||||
def test_tmpfs_isolated_chv_and_unknown_fail_closed(self):
|
||||
isolated, _ = tmpfs_isolated("cloud-hypervisor", "/tmp")
|
||||
self.assertFalse(isolated)
|
||||
isolated, _ = tmpfs_isolated("unknown-kind", "/tmp")
|
||||
self.assertFalse(isolated)
|
||||
|
||||
def test_snapshot_url_uses_resolve_inside(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
state = os.path.join(d, "state")
|
||||
os.makedirs(state)
|
||||
path = os.path.join(state, "s.raw")
|
||||
with open(path, "w") as fh:
|
||||
fh.write("x")
|
||||
self.assertEqual(
|
||||
validate_snapshot_url("file://" + path, state), path)
|
||||
with self.assertRaises(Exception):
|
||||
validate_snapshot_url(
|
||||
"file:///etc/passwd", state)
|
||||
|
||||
def test_no_timeout_or_quota_map(self):
|
||||
import cloud.chv.compat215 as m
|
||||
self.assertFalse(hasattr(m, "resolve_module_timeout"))
|
||||
self.assertFalse(hasattr(m, "decide"))
|
||||
|
||||
|
||||
class PoolFacadeTest(unittest.TestCase):
|
||||
def test_facade_only_reconciles_never_acquires(self):
|
||||
# facade 直接读 registry 文件做清理(本轨不再构造
|
||||
# WarmPool):用假执行器 + 空 registry,返回 [] 不炸,
|
||||
# 且绝不调 acquire/prewarm(假执行器根本没有这两个方法)。
|
||||
ex = FakeExecutor()
|
||||
# facade 需要 state_dir 属性定 registry 路径;没有则返回 []
|
||||
ex.state_dir = tempfile.mkdtemp()
|
||||
ex.warm_pool_size = 0
|
||||
ex.snapshot_url = None
|
||||
result = reconcile_orphans_for_executor(ex)
|
||||
self.assertEqual(result, [])
|
||||
self.assertFalse(ex.acquire_called)
|
||||
self.assertFalse(ex.prewarm_called)
|
||||
|
||||
|
||||
class AuditCounterTest(unittest.TestCase):
|
||||
def test_emit_failure_counts_and_does_not_raise(self):
|
||||
ex = CloudHypervisorExecutor.__new__(CloudHypervisorExecutor)
|
||||
ex.audit_callback = lambda *a, **k: (_ for _ in ()).throw(
|
||||
RuntimeError("db down"))
|
||||
ex.audit_failure_count = 0
|
||||
# _emit 不抛异常(任务不因此中断),计数 +1
|
||||
ex._emit("vm_boot", {"id": "t1"}, "vm=chv-t1")
|
||||
self.assertEqual(ex.audit_failure_count, 1)
|
||||
ex._emit("vm_boot", {"id": "t1"}, "vm=chv-t1")
|
||||
self.assertEqual(ex.audit_failure_count, 2)
|
||||
|
||||
|
||||
class ChvVmDeadFieldTest(unittest.TestCase):
|
||||
def test_only_chv_with_vm_death_reason_returns_true(self):
|
||||
row = _row(executor="cloud-hypervisor",
|
||||
interrupted_reason="VM 中途死亡,任务未完成")
|
||||
d = _task_to_dict(row)
|
||||
self.assertTrue(d.get("chv_vm_dead"))
|
||||
row2 = _row(executor="cloud-hypervisor",
|
||||
interrupted_reason="quota")
|
||||
self.assertNotIn("chv_vm_dead", _task_to_dict(row2))
|
||||
row3 = _row(executor="docker",
|
||||
interrupted_reason="VM 中途死亡,任务未完成")
|
||||
self.assertNotIn("chv_vm_dead", _task_to_dict(row3))
|
||||
row4 = _row(executor="cloud-hypervisor",
|
||||
interrupted_reason=None)
|
||||
self.assertNotIn("chv_vm_dead", _task_to_dict(row4))
|
||||
|
||||
|
||||
class LockfileScriptTest(unittest.TestCase):
|
||||
def _run(self, data, extra=None):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
p = os.path.join(d, "BUILDINFO.json")
|
||||
with open(p, "w") as fh:
|
||||
json.dump(data, fh)
|
||||
script = str(Path(__file__).resolve().parents[2]
|
||||
/ "chv-guest" / "check_lockfile.py")
|
||||
cmd = [sys.executable, script, p] + (extra or [])
|
||||
return subprocess.run(cmd, capture_output=True, text=True)
|
||||
|
||||
def test_pass_and_empty_and_drift(self):
|
||||
good = {"tag": "t", "versions": {"dpkg": ["a 1"],
|
||||
"pip": ["b==2"]}}
|
||||
self.assertEqual(self._run(good, ["--tag", "t"]).returncode, 0)
|
||||
bad = {"tag": "t", "versions": {"dpkg": [], "pip": []}}
|
||||
self.assertNotEqual(self._run(bad).returncode, 0)
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
frozen = os.path.join(d, "frozen.json")
|
||||
with open(frozen, "w") as fh:
|
||||
json.dump({"dpkg": ["missing 9"], "pip": []}, fh)
|
||||
self.assertNotEqual(
|
||||
self._run(good, ["--frozen", frozen]).returncode, 0)
|
||||
|
||||
|
||||
class LongFunctionTest(unittest.TestCase):
|
||||
def test_no_chv_function_over_50_lines(self):
|
||||
import ast
|
||||
base = Path(__file__).resolve().parents[2] / "cloud" / "chv"
|
||||
over = []
|
||||
for f in base.glob("*.py"):
|
||||
tree = ast.parse(f.read_text())
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef):
|
||||
n = node.end_lineno - node.lineno + 1
|
||||
if n > 50:
|
||||
over.append("%s:%s:%d" % (f.name, node.name, n))
|
||||
self.assertEqual(over, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,480 @@
|
||||
"""CHV 架构/可维护性/前端(只读侧)推进轨测试。
|
||||
|
||||
本轨 feat-chv-arch-maint-fe,堆叠于 #254(feat-chv-
|
||||
arch95plus)。只用标准库与 mock,绝不启动真实 VM;
|
||||
测试里只用 FAKE 占位值,绝无真实密钥。覆盖:
|
||||
- WarmPool 破环性清理:旧 import 路径一律 ImportError,
|
||||
实验路径唯一,生产代码不 import 实验模块;
|
||||
- orphans facade:registry 语义(有效/非法/损坏/缺失/
|
||||
shutdown 抛错/_emit 抛错)与旧 WarmPool 逐项一致;
|
||||
- 配置片段:五片段键集互不相交、并集等于旧字段集、
|
||||
默认值与单项非法报错逐字一致、组装键冲突即报错;
|
||||
- ADR-12 模块依赖图:叶子模块只依赖标准库、禁 import
|
||||
清单、公开 API 全量类型标注且无 Any 逃逸;
|
||||
- #215 替换清单:每项 local 真实存在、#215 模块当前
|
||||
不存在、R3 永不替换;
|
||||
- 服务端只读字段:goal 回落形态(纯文本“VM 中途死亡”)
|
||||
同样派生 chv_vm_dead,非 VM 中断原因一律不派生。
|
||||
"""
|
||||
|
||||
import ast
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from cloud.chv import reconcile_orphans_for_executor
|
||||
from cloud.chv.compat215 import REPLACEMENT_POINTS
|
||||
from cloud.chv.compat215 import resolve_inside, tmpfs_isolated
|
||||
from cloud.chv.config import _assemble_chv_config
|
||||
from cloud.chv.config import _base_fragment
|
||||
from cloud.chv.config import _disk_fragment
|
||||
from cloud.chv.config import _network_fragment
|
||||
from cloud.chv.config import _quota_fragment
|
||||
from cloud.chv.config import _ssh_fragment
|
||||
from cloud.chv.config import _timeout_fragment
|
||||
from cloud.chv.config import build_chv_config
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.chv.orphans import _REGISTRY_FILENAME
|
||||
from cloud.chv.orphans import reconcile_orphans_for_executor as _facade
|
||||
from cloud.server import _chv_vm_dead_from_row, _task_to_dict
|
||||
except ImportError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv import reconcile_orphans_for_executor
|
||||
from cloud.chv.compat215 import REPLACEMENT_POINTS
|
||||
from cloud.chv.compat215 import resolve_inside, tmpfs_isolated
|
||||
from cloud.chv.config import _assemble_chv_config
|
||||
from cloud.chv.config import _base_fragment
|
||||
from cloud.chv.config import _disk_fragment
|
||||
from cloud.chv.config import _network_fragment
|
||||
from cloud.chv.config import _quota_fragment
|
||||
from cloud.chv.config import _ssh_fragment
|
||||
from cloud.chv.config import _timeout_fragment
|
||||
from cloud.chv.config import build_chv_config
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.chv.orphans import _REGISTRY_FILENAME
|
||||
from cloud.chv.orphans import reconcile_orphans_for_executor as _facade
|
||||
from cloud.server import _chv_vm_dead_from_row, _task_to_dict
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
CHV = ROOT / "cloud" / "chv"
|
||||
|
||||
EXPECTED_FIELDS = {
|
||||
"binary", "kernel", "rootfs", "cpus", "memory", "state_dir",
|
||||
"kvm_path", "cpuinfo_path", "ch_remote", "model",
|
||||
"ssh_key", "ssh_user", "ssh_port", "host_key_mode",
|
||||
"guest_host_py", "cmdline",
|
||||
"net", "net_json", "ssh_host", "ssh_forward",
|
||||
"egress_proxy", "allow_direct_egress",
|
||||
"max_vms", "warm_pool_size",
|
||||
"disk_mode", "disk_gb", "reuse_experiment",
|
||||
"single_tenant_project", "snapshot_max_count",
|
||||
"snapshot_max_total_gb", "snapshot_retention_days",
|
||||
"snapshot_dir", "allow_remote_snapshot", "snapshot_url",
|
||||
"task_timeout", "api_timeout", "ready_timeout", "ready_interval",
|
||||
}
|
||||
|
||||
|
||||
class FakeProdExecutor:
|
||||
"""生产 facade 替身:只有 facade 允许碰的属性。
|
||||
|
||||
acquire/prewarm/release 一律不存在——facade 若敢碰,
|
||||
__getattr__ 直接抛错,测试即 fail。"""
|
||||
|
||||
def __init__(self, state_dir):
|
||||
self.state_dir = state_dir
|
||||
self.shutdowns = []
|
||||
self.emits = []
|
||||
|
||||
def __getattr__(self, name):
|
||||
if name in ("acquire", "prewarm", "release"):
|
||||
raise AssertionError("facade 不得碰实验接口:%s" % name)
|
||||
raise AttributeError(name)
|
||||
|
||||
def shutdown_vm(self, vm):
|
||||
self.shutdowns.append(vm)
|
||||
return True
|
||||
|
||||
def _emit(self, event, task, detail):
|
||||
self.emits.append(event)
|
||||
|
||||
|
||||
def _write_registry(state_dir, data):
|
||||
path = os.path.join(state_dir, _REGISTRY_FILENAME)
|
||||
if isinstance(data, str):
|
||||
with open(path, "w") as fh:
|
||||
fh.write(data)
|
||||
else:
|
||||
with open(path, "w") as fh:
|
||||
json.dump(data, fh)
|
||||
return path
|
||||
|
||||
|
||||
class BreakingCleanupTest(unittest.TestCase):
|
||||
def test_old_paths_raise_import_error(self):
|
||||
with self.assertRaises(ImportError):
|
||||
from cloud.chv import WarmPool # noqa: F401
|
||||
with self.assertRaises(ImportError):
|
||||
from cloud.sandbox import WarmPool # noqa: F401
|
||||
with self.assertRaises(ImportError):
|
||||
import cloud.chv.pool # noqa: F401
|
||||
|
||||
def test_new_paths_and_identity(self):
|
||||
self.assertEqual(WarmPool.__module__,
|
||||
"cloud.chv.experimental_pool")
|
||||
self.assertIs(reconcile_orphans_for_executor, _facade)
|
||||
import cloud.sandbox as sb
|
||||
self.assertIs(sb.reconcile_orphans_for_executor, _facade)
|
||||
self.assertFalse(WarmPool.wired_to_dispatch)
|
||||
|
||||
def test_production_sources_never_import_experimental(self):
|
||||
# 钉的是 import 语句,不是文档提及:executor/__init__
|
||||
# 的 docstring 本来就要写清实验模块去向,文本匹配会
|
||||
# 误杀,故用 AST 只看 Import/ImportFrom 节点。
|
||||
for rel in ("cloud/worker.py", "cloud/chv/executor.py",
|
||||
"cloud/chv/__init__.py", "cloud/sandbox.py",
|
||||
"cloud/chv/orphans.py"):
|
||||
tree = ast.parse((ROOT / rel).read_text())
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Import):
|
||||
for a in node.names:
|
||||
self.assertNotIn(a.name, (
|
||||
"cloud.chv.pool",
|
||||
"cloud.chv.experimental_pool"), rel)
|
||||
if isinstance(node, ast.ImportFrom):
|
||||
self.assertNotIn(node.module, (
|
||||
"cloud.chv.pool",
|
||||
"cloud.chv.experimental_pool"), rel)
|
||||
if node.module in ("cloud.chv", "cloud.sandbox"):
|
||||
names = [a.name for a in node.names]
|
||||
self.assertNotIn("WarmPool", names, rel)
|
||||
|
||||
|
||||
class OrphansTest(unittest.TestCase):
|
||||
def test_valid_warm_and_assigned_cleaned_invalid_dropped(self):
|
||||
# feat-chv-p1fix(P1-02):非法条目不再静默丢弃,
|
||||
# 先发 invalid 审计并计数,再发 cleaned 审计。
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, {
|
||||
"a": {"state": "warm", "vm": "chv-a",
|
||||
"task_id": "a", "snapshot_url": None},
|
||||
"b": {"state": "assigned", "vm": "chv-b",
|
||||
"task_id": "b", "snapshot_url": None},
|
||||
"c": {"state": "warm", "task_id": "c"},
|
||||
"e": {"state": "running", "vm": "chv-e"},
|
||||
"f": "not-a-dict",
|
||||
})
|
||||
ex = FakeProdExecutor(d)
|
||||
self.assertEqual(
|
||||
reconcile_orphans_for_executor(ex),
|
||||
["chv-a", "chv-b"])
|
||||
self.assertEqual(ex.shutdowns, ["chv-a", "chv-b"])
|
||||
self.assertEqual(ex.emits, [
|
||||
"warm_pool_registry_invalid_entries",
|
||||
"warm_pool_reconcile"])
|
||||
with open(path) as fh:
|
||||
self.assertEqual(json.load(fh), {})
|
||||
|
||||
def test_invalid_only_returns_empty_and_drops_audited_entries(self):
|
||||
# feat-chv-p1fix(P1-02):仅非法条目时返回 [] 且
|
||||
# 不 shutdown;非法条目已审计计数,落盘时丢弃,
|
||||
# 不再每次重启重复审计同一批非法条目。
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, {
|
||||
"c": {"state": "warm", "task_id": "c"}})
|
||||
ex = FakeProdExecutor(d)
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex), [])
|
||||
self.assertEqual(ex.shutdowns, [])
|
||||
self.assertEqual(
|
||||
ex.emits, ["warm_pool_registry_invalid_entries"])
|
||||
with open(path) as fh:
|
||||
self.assertEqual(json.load(fh), {})
|
||||
|
||||
def test_corrupt_json_quarantined(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, "{not json")
|
||||
ex = FakeProdExecutor(d)
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex), [])
|
||||
self.assertFalse(os.path.exists(path))
|
||||
self.assertTrue(os.path.exists(path + ".corrupt"))
|
||||
|
||||
def test_nondict_top_level_quarantined(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, "[1, 2]")
|
||||
ex = FakeProdExecutor(d)
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex), [])
|
||||
self.assertTrue(os.path.exists(path + ".corrupt"))
|
||||
|
||||
def test_missing_file_and_no_state_dir(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
ex = FakeProdExecutor(d)
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex), [])
|
||||
|
||||
class NoDir:
|
||||
def shutdown_vm(self, vm):
|
||||
raise AssertionError("must not be called")
|
||||
|
||||
self.assertEqual(
|
||||
reconcile_orphans_for_executor(NoDir()), [])
|
||||
|
||||
def test_shutdown_failure_not_cleaned_and_retained(self):
|
||||
# feat-chv-p1fix(P1-01):shutdown 失败的条目不进
|
||||
# cleaned、不触发清库,落盘保留原始条目可重试;
|
||||
# _emit 失败仍 best-effort,不炸清理流程。
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, {
|
||||
"a": {"state": "warm", "vm": "chv-a"}})
|
||||
ex = FakeProdExecutor(d)
|
||||
|
||||
def boom(vm):
|
||||
raise RuntimeError("shutdown down")
|
||||
|
||||
ex.shutdown_vm = boom
|
||||
ex._emit = lambda *a, **k: (_ for _ in ()).throw(
|
||||
RuntimeError("audit down"))
|
||||
self.assertEqual(
|
||||
reconcile_orphans_for_executor(ex), [])
|
||||
with open(path) as fh:
|
||||
self.assertEqual(
|
||||
json.load(fh),
|
||||
{"a": {"state": "warm", "vm": "chv-a"}})
|
||||
|
||||
def test_experimental_pool_registry_roundtrip_unchanged(self):
|
||||
# 实验池自身的 registry 语义(载入即 orphan、reconcile
|
||||
# 清理)与上一轨一致,只是文件 IO 委托给 orphans。
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
ex = FakeProdExecutor(d)
|
||||
ex.warm_pool_size = 0
|
||||
ex.snapshot_url = None
|
||||
pool1 = WarmPool(ex)
|
||||
pool1._entries["chv-old"] = {
|
||||
"state": "warm", "vm": "chv-old",
|
||||
"task": {"id": "old"}, "snapshot_url": None}
|
||||
pool1._save_registry()
|
||||
pool2 = WarmPool(ex)
|
||||
self.assertEqual(pool2.orphan_count(), 1)
|
||||
self.assertEqual(pool2.warm_count(), 0)
|
||||
self.assertEqual(pool2.reconcile_orphans(), ["chv-old"])
|
||||
|
||||
|
||||
class ConfigFragmentsTest(unittest.TestCase):
|
||||
def test_fragment_keys_disjoint_and_union_is_field_set(self):
|
||||
frags = [_base_fragment({}, {}), _ssh_fragment({}, {}),
|
||||
_network_fragment({}, {}), _quota_fragment({}),
|
||||
_disk_fragment({}), _timeout_fragment({}, {})]
|
||||
seen = set()
|
||||
for frag in frags:
|
||||
self.assertFalse(seen & set(frag), "片段键重叠")
|
||||
seen |= set(frag)
|
||||
self.assertEqual(seen, EXPECTED_FIELDS)
|
||||
self.assertEqual(len(EXPECTED_FIELDS), 38)
|
||||
|
||||
def test_defaults_and_overrides(self):
|
||||
cfg = build_chv_config({}, {})
|
||||
self.assertEqual(cfg.binary, "cloud-hypervisor")
|
||||
self.assertEqual(cfg.ssh_port, 2222)
|
||||
self.assertEqual(cfg.ssh_user, "root")
|
||||
self.assertEqual(cfg.host_key_mode, "provisioned")
|
||||
self.assertEqual(cfg.max_vms, 4)
|
||||
self.assertEqual(cfg.disk_mode, "copy")
|
||||
self.assertEqual(cfg.disk_gb, 50.0)
|
||||
self.assertEqual(cfg.api_timeout, 30.0)
|
||||
self.assertEqual(cfg.ready_timeout, 120.0)
|
||||
self.assertEqual(cfg.ready_interval, 5.0)
|
||||
self.assertIsNone(cfg.task_timeout)
|
||||
self.assertIsNone(cfg.snapshot_url)
|
||||
cfg2 = build_chv_config(
|
||||
{"ssh_user": "u", "ssh_port": 2223},
|
||||
{"CHV_SSH_USER": "env-user", "CHV_MAX_VMS": "2",
|
||||
"CHV_EGRESS_PROXY": "http://p:1",
|
||||
"CHV_SNAPSHOT_URL": "file:///snap/base"})
|
||||
self.assertEqual(cfg2.ssh_user, "u")
|
||||
self.assertEqual(cfg2.ssh_port, 2223)
|
||||
self.assertEqual(cfg2.max_vms, 2)
|
||||
self.assertEqual(cfg2.egress_proxy, "http://p:1")
|
||||
self.assertEqual(cfg2.snapshot_url, "file:///snap/base")
|
||||
|
||||
def test_single_error_messages_unchanged(self):
|
||||
cases = [
|
||||
({"ssh_user": ""}, {}, "CHV_SSH_USER 不能为空字符串"),
|
||||
({}, {"CHV_SSH_PORT": "0"}, "CHV_SSH_PORT 必须在 1-65535 之间"),
|
||||
({}, {"CHV_SSH_PORT": "99999"},
|
||||
"CHV_SSH_PORT 必须在 1-65535 之间"),
|
||||
({}, {"CHV_DISK_MODE": "bad"},
|
||||
"CHV_DISK_MODE 只能是 copy 或 reuse"),
|
||||
({}, {"CHV_DISK_GB": "0"}, "CHV_DISK_GB 必须大于 0"),
|
||||
({}, {"CHV_SNAPSHOT_MAX_COUNT": "-1"},
|
||||
"CHV_SNAPSHOT_* 不能为负数"),
|
||||
({}, {"CHV_MAX_VMS": "x"}, "CHV_MAX_VMS 必须是整数"),
|
||||
({}, {"CHV_WARM_POOL_SIZE": "x"},
|
||||
"CHV_WARM_POOL_SIZE 必须是整数"),
|
||||
({}, {"CHV_API_TIMEOUT": "x"},
|
||||
"CHV_*_TIMEOUT/INTERVAL 必须是数字"),
|
||||
]
|
||||
for params, env, msg in cases:
|
||||
with self.assertRaises(ValueError) as cm:
|
||||
build_chv_config(params, env)
|
||||
self.assertIn(msg, str(cm.exception))
|
||||
|
||||
def test_assemble_key_conflict_raises(self):
|
||||
with self.assertRaises(ValueError):
|
||||
_assemble_chv_config([{"a": 1}, {"a": 2}])
|
||||
|
||||
|
||||
def _top_level_imports(path):
|
||||
tree = ast.parse(path.read_text())
|
||||
mods = set()
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.Import):
|
||||
mods.update(a.name for a in node.names)
|
||||
elif isinstance(node, ast.ImportFrom) and node.module:
|
||||
mods.add(node.module)
|
||||
return mods
|
||||
|
||||
|
||||
class DependencyGraphTest(unittest.TestCase):
|
||||
def test_leaf_modules_only_import_stdlib(self):
|
||||
for name in ("orphans", "config", "compat215", "errors"):
|
||||
mods = _top_level_imports(CHV / (name + ".py"))
|
||||
for mod in mods:
|
||||
top = mod.split(".")[0]
|
||||
self.assertFalse(top.startswith("cloud"),
|
||||
"%s 不得 import %s" % (name, mod))
|
||||
|
||||
def test_executor_never_imports_pool_or_orphans(self):
|
||||
# 同上:只看 AST import 节点,模块头 docstring 的
|
||||
# 分工说明不算 import。
|
||||
tree = ast.parse((CHV / "executor.py").read_text())
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.Import):
|
||||
for a in node.names:
|
||||
self.assertNotIn(a.name, (
|
||||
"cloud.chv.pool",
|
||||
"cloud.chv.experimental_pool",
|
||||
"cloud.chv.orphans"))
|
||||
if isinstance(node, ast.ImportFrom):
|
||||
self.assertNotIn(node.module, (
|
||||
"cloud.chv.pool",
|
||||
"cloud.chv.experimental_pool",
|
||||
"cloud.chv.orphans"))
|
||||
|
||||
def test_no_chv_function_over_50_lines(self):
|
||||
over = []
|
||||
for f in CHV.glob("*.py"):
|
||||
tree = ast.parse(f.read_text())
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.FunctionDef):
|
||||
n = node.end_lineno - node.lineno + 1
|
||||
if n > 50:
|
||||
over.append("%s:%s:%d" % (f.name, node.name, n))
|
||||
self.assertEqual(over, [])
|
||||
|
||||
|
||||
class PublicApiTypesTest(unittest.TestCase):
|
||||
def test_public_api_fully_annotated_and_no_any(self):
|
||||
import re
|
||||
unannotated = []
|
||||
for f in CHV.glob("*.py"):
|
||||
text = f.read_text()
|
||||
for line in text.splitlines():
|
||||
self.assertIsNone(re.search(r"\bAny\b", line),
|
||||
"%s 含 Any 逃逸:%s" % (f.name, line))
|
||||
tree = ast.parse(text)
|
||||
for node in tree.body:
|
||||
if isinstance(node, ast.FunctionDef) \
|
||||
and not node.name.startswith("_"):
|
||||
bad = [a.arg for a in node.args.args
|
||||
if a.annotation is None]
|
||||
if bad or node.returns is None:
|
||||
unannotated.append("%s:%s" % (f.name, node.name))
|
||||
if isinstance(node, ast.ClassDef):
|
||||
for m in node.body:
|
||||
if isinstance(m, ast.FunctionDef) and (
|
||||
not m.name.startswith("_")
|
||||
or m.name == "__init__"):
|
||||
bad = [a.arg for a in m.args.args
|
||||
if a.arg != "self"
|
||||
and a.annotation is None]
|
||||
bad += [a.arg for a in m.args.kwonlyargs
|
||||
if a.annotation is None]
|
||||
if bad or m.returns is None:
|
||||
unannotated.append(
|
||||
"%s:%s.%s" % (f.name, node.name,
|
||||
m.name))
|
||||
self.assertEqual(unannotated, [])
|
||||
|
||||
def test_corrected_annotations_match_runtime(self):
|
||||
from cloud.chv.disk import record_snapshot
|
||||
from cloud.chv.hostkeys import host_key_paths
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
entries, removed = record_snapshot(
|
||||
d, "vm1", "https://example.invalid/snap", 20.0)
|
||||
self.assertIsInstance(entries, list)
|
||||
self.assertIsInstance(removed, list)
|
||||
paths = host_key_paths(d, "vm1")
|
||||
self.assertIsInstance(paths, dict)
|
||||
self.assertIn("private", paths)
|
||||
|
||||
|
||||
class ReplacementChecklistTest(unittest.TestCase):
|
||||
def test_points_resolve_and_215_modules_absent(self):
|
||||
by_id = {p["id"]: p for p in REPLACEMENT_POINTS}
|
||||
self.assertEqual(set(by_id), {"R1", "R2", "R3"})
|
||||
self.assertIsNotNone(resolve_inside)
|
||||
self.assertIsNotNone(tmpfs_isolated)
|
||||
hardening = (ROOT / "cloud" / "chv" / "hardening.py").read_text()
|
||||
self.assertIn("resolve_inside", hardening)
|
||||
for mod in ("cloud.gates", "cloud.sandbox_policy",
|
||||
"cloud.timeouts", "cloud.quota"):
|
||||
self.assertIsNone(importlib.util.find_spec(mod), mod)
|
||||
import cloud.chv.compat215 as m
|
||||
self.assertFalse(hasattr(m, "resolve_module_timeout"))
|
||||
self.assertFalse(hasattr(m, "decide"))
|
||||
self.assertIsNone(by_id["R3"]["replacement"])
|
||||
self.assertIsNone(by_id["R2"]["caller"])
|
||||
|
||||
def test_tmpfs_isolated_has_no_production_caller(self):
|
||||
hits = []
|
||||
for f in (ROOT / "cloud").rglob("*.py"):
|
||||
if "tests" in f.parts:
|
||||
continue
|
||||
if "tmpfs_isolated" in f.read_text():
|
||||
hits.append(str(f.relative_to(ROOT)))
|
||||
self.assertEqual(hits, ["cloud/chv/compat215.py"])
|
||||
|
||||
|
||||
def _row(**kw):
|
||||
base = {
|
||||
"id": "t1", "project_id": "p1", "prompt": "x",
|
||||
"status": "interrupted", "idempotency_key": None,
|
||||
"depends_on": "[]", "priority": 0, "created_by": "u",
|
||||
"created_at": 0, "updated_at": 0, "last_heartbeat": 0,
|
||||
}
|
||||
base.update(kw)
|
||||
return base
|
||||
|
||||
|
||||
class ServerLifecycleTest(unittest.TestCase):
|
||||
def test_goal_fallback_form_also_derives_dead(self):
|
||||
row = _row(executor="cloud-hypervisor",
|
||||
interrupted_reason="VM 中途死亡")
|
||||
self.assertTrue(_chv_vm_dead_from_row(row))
|
||||
self.assertTrue(_task_to_dict(row).get("chv_vm_dead"))
|
||||
|
||||
def test_non_vm_reasons_never_derive(self):
|
||||
for reason in ("quota", "worker_shutdown", None, ""):
|
||||
row = _row(executor="cloud-hypervisor",
|
||||
interrupted_reason=reason)
|
||||
self.assertIsNone(_chv_vm_dead_from_row(row))
|
||||
self.assertNotIn("chv_vm_dead", _task_to_dict(row))
|
||||
row = _row(executor="docker",
|
||||
interrupted_reason="VM 中途死亡,任务未完成")
|
||||
self.assertIsNone(_chv_vm_dead_from_row(row))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,621 @@
|
||||
"""CHV 覆盖收尾轨(feat-chv-cover100)新增测试。
|
||||
|
||||
只补本轨实测基线(coverage --source=cloud/chv,跑
|
||||
test_chv*.py + test_cloud_hypervisor.py:1925 行、缺 80 行、
|
||||
96%)里仍未覆盖的行,生产代码零改动。每项都是行为断言:
|
||||
|
||||
- api_client:无冒号头行跳过、分片到达的体部、不可解析
|
||||
的状态行、带 payload 的 unix socket 调用、normalize 的
|
||||
dict/对象两分支(此前只覆 tuple 分支);
|
||||
- compat215:commonpath 跨盘符 ValueError 回 None;
|
||||
- config:磁盘 GB 非数字、快照数非数字、ready_interval 为 0
|
||||
回落 5 秒(旧语义);
|
||||
- disk:目录统计里文件消失、配额早退、disk_usage 失败与
|
||||
宿主余量不足、已存在磁盘不可读、拷贝失败、剪枝删文件
|
||||
失败、快照总量统计的文件大小回落与失败、file:// 快照
|
||||
文件不存在、总量帽删旧文件失败;
|
||||
- hardening:state_dir stat 失败、chmod 后仍宽松、socket
|
||||
死活的空/缺失/异常三态、空 net 配置、非 dict 条目、
|
||||
egress 空值与不可解析 URL;
|
||||
- hostkeys:无 runner 时 genisoimage/keygen 走 subprocess
|
||||
的失败路径、seed/key 私钥 chmod 失败、无 runcmd 时密钥
|
||||
环境条目仍追加(不丢);
|
||||
- experimental_pool(单独计分,非生产路径):registry
|
||||
坏条目计数、非 dict task 落盘、reconcile 跳过 warm/
|
||||
shutdown 失败/_emit 失败、prewarm 重名跳过与唤醒失败、
|
||||
acquire _emit 失败、release shutdown 失败。
|
||||
|
||||
全部用假 socket/假 runner/假执行器,绝不连真实网络、
|
||||
不起真实 VM、不改生产行为。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import socket
|
||||
import tempfile
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
from unittest import mock
|
||||
|
||||
from cloud.chv import api_client
|
||||
from cloud.chv import compat215
|
||||
from cloud.chv import config as chv_config
|
||||
from cloud.chv import disk as chv_disk
|
||||
from cloud.chv import experimental_pool
|
||||
from cloud.chv import hardening
|
||||
from cloud.chv import hostkeys
|
||||
from cloud.chv.errors import ChvFailure
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
|
||||
|
||||
class _FakeApiSocket:
|
||||
"""_unix_socket_api 用的假 socket:按 recv 序列供数据。"""
|
||||
|
||||
def __init__(self, chunks):
|
||||
self._chunks = list(chunks)
|
||||
self.sent = b""
|
||||
self.closed = False
|
||||
|
||||
def settimeout(self, _timeout):
|
||||
pass
|
||||
|
||||
def connect(self, _path):
|
||||
pass
|
||||
|
||||
def sendall(self, data):
|
||||
self.sent += data
|
||||
|
||||
def recv(self, _size):
|
||||
if not self._chunks:
|
||||
return b""
|
||||
return self._chunks.pop(0)
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
|
||||
|
||||
class ApiClientCover100Test(unittest.TestCase):
|
||||
def test_header_line_without_colon_skipped(self):
|
||||
self.assertEqual(
|
||||
api_client._parse_content_length(
|
||||
"HTTP/1.1 200 OK\r\nGARBAGE-LINE\r\nContent-Length: 2"),
|
||||
2)
|
||||
|
||||
def test_unix_api_with_payload_and_chunked_body(self):
|
||||
# 体部分两片到达:首片随头一起到(体部前缀),次片
|
||||
# 才补满 Content-Length——_read_body 的追加行必须跑到。
|
||||
fake = _FakeApiSocket([
|
||||
b"HTTP/1.1 200 OK\r\nContent-Length: 4\r\n\r\nab",
|
||||
b"cd",
|
||||
])
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
sock_path = os.path.join(tmp, "chv.sock")
|
||||
with open(sock_path, "w") as fh:
|
||||
fh.write("x") # 仅让 exists() 为真,不真建 socket
|
||||
with mock.patch.object(api_client.socket, "socket",
|
||||
lambda *a, **k: fake):
|
||||
status, body = api_client._unix_socket_api(
|
||||
sock_path, "PUT", "/vm.create",
|
||||
payload={"name": "vm1"})
|
||||
self.assertEqual(status, 200)
|
||||
self.assertEqual(body, "abcd")
|
||||
self.assertIn(b'"name": "vm1"', fake.sent)
|
||||
self.assertTrue(fake.closed)
|
||||
|
||||
def test_parse_status_non_numeric_is_zero(self):
|
||||
status, _body = api_client._parse_response(
|
||||
"HTTP/1.1 OK\r\n\r\n", b"{}")
|
||||
self.assertEqual(status, 0)
|
||||
|
||||
def test_parse_status_missing_token_is_zero(self):
|
||||
status, _body = api_client._parse_response("HTTP/\r\n\r\n", b"{}")
|
||||
self.assertEqual(status, 0)
|
||||
|
||||
def test_normalize_dict_branches(self):
|
||||
self.assertEqual(
|
||||
api_client.normalize_api_result(
|
||||
{"status": 201, "body": {"a": 1}}),
|
||||
(201, {"a": 1}))
|
||||
self.assertEqual(
|
||||
api_client.normalize_api_result({"status_code": 202}),
|
||||
(202, {"status_code": 202}))
|
||||
self.assertEqual(
|
||||
api_client.normalize_api_result({"body": "x"}),
|
||||
(0, "x"))
|
||||
|
||||
def test_normalize_object_branches(self):
|
||||
obj = SimpleNamespace(status=404)
|
||||
status, body = api_client.normalize_api_result(obj)
|
||||
self.assertEqual((status, body), (404, obj))
|
||||
obj2 = SimpleNamespace(status_code=503)
|
||||
status, _body = api_client.normalize_api_result(obj2)
|
||||
self.assertEqual(status, 503)
|
||||
obj3 = SimpleNamespace()
|
||||
status, _body = api_client.normalize_api_result(obj3)
|
||||
self.assertEqual(status, 0)
|
||||
|
||||
|
||||
class Compat215Cover100Test(unittest.TestCase):
|
||||
def test_commonpath_value_error_is_none(self):
|
||||
# 跨盘符(Windows)下 commonpath 抛 ValueError 时
|
||||
# 目录门必须回 None,而不是把异常漏给调用方。
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
with mock.patch.object(
|
||||
os.path, "commonpath",
|
||||
side_effect=ValueError("mixing drives")):
|
||||
self.assertIsNone(compat215.resolve_inside(tmp, "sub"))
|
||||
|
||||
|
||||
class ConfigCover100Test(unittest.TestCase):
|
||||
def test_disk_gb_non_numeric_rejected(self):
|
||||
with self.assertRaises(ValueError) as ctx:
|
||||
chv_config.build_chv_config({}, {"CHV_DISK_GB": "abc"})
|
||||
self.assertIn("CHV_DISK_GB 必须是数字", str(ctx.exception))
|
||||
|
||||
def test_snapshot_count_non_numeric_rejected(self):
|
||||
with self.assertRaises(ValueError) as ctx:
|
||||
chv_config.build_chv_config(
|
||||
{}, {"CHV_SNAPSHOT_MAX_COUNT": "x"})
|
||||
self.assertIn("CHV_SNAPSHOT_* 必须是数字", str(ctx.exception))
|
||||
|
||||
def test_ready_interval_zero_falls_back_to_five(self):
|
||||
cfg = chv_config.build_chv_config({"ready_interval": 0}, {})
|
||||
self.assertEqual(cfg.ready_interval, 5)
|
||||
|
||||
|
||||
class DiskCover100Test(unittest.TestCase):
|
||||
def _write(self, path, data):
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(data)
|
||||
|
||||
def test_dir_total_ignores_vanished_file(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
rootfs = os.path.join(tmp, "base.raw")
|
||||
self._write(rootfs, b"1234")
|
||||
state_dir = os.path.join(tmp, "state")
|
||||
victim = os.path.join(state_dir, "gone.raw")
|
||||
self._write(victim, b"zz")
|
||||
real_getsize = os.path.getsize
|
||||
|
||||
def getsize(path):
|
||||
if path == victim:
|
||||
raise OSError("walk 后文件消失")
|
||||
return real_getsize(path)
|
||||
|
||||
with mock.patch.object(os.path, "getsize",
|
||||
side_effect=getsize):
|
||||
# 不抛错:消失文件按 0 计,不卡住配额校验。
|
||||
self.assertIsNone(
|
||||
chv_disk.check_disk_quota(rootfs, 50.0, state_dir))
|
||||
|
||||
def test_quota_skipped_without_disk_gb_or_rootfs(self):
|
||||
self.assertIsNone(chv_disk.check_disk_quota("", None))
|
||||
self.assertIsNone(chv_disk.check_disk_quota("anything", None))
|
||||
self.assertIsNone(chv_disk.check_disk_quota("", 50.0))
|
||||
|
||||
def test_quota_disk_usage_failure_is_not_fatal(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
rootfs = os.path.join(tmp, "base.raw")
|
||||
self._write(rootfs, b"1234")
|
||||
with mock.patch.object(chv_disk.shutil, "disk_usage",
|
||||
side_effect=OSError("statfs 失败")):
|
||||
# 宿主余量读不到时不按“不足”误杀,只按已用量
|
||||
# 与镜像大小校验,本例两者都远小于配额。
|
||||
self.assertIsNone(
|
||||
chv_disk.check_disk_quota(rootfs, 50.0, tmp))
|
||||
|
||||
def test_quota_host_free_space_shortfall_rejected(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
rootfs = os.path.join(tmp, "base.raw")
|
||||
self._write(rootfs, b"1234")
|
||||
usage = SimpleNamespace(total=100, used=100, free=0)
|
||||
with mock.patch.object(chv_disk.shutil, "disk_usage",
|
||||
return_value=usage):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
chv_disk.check_disk_quota(rootfs, 50.0, tmp)
|
||||
self.assertEqual(ctx.exception.kind, "disk_quota_exceeded")
|
||||
|
||||
def test_provision_existing_disk_unreadable_rejected(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
rootfs = os.path.join(tmp, "base.raw")
|
||||
self._write(rootfs, b"1234")
|
||||
task_disk = os.path.join(tmp, "vm1", "disk.raw")
|
||||
self._write(task_disk, b"1234")
|
||||
with mock.patch.object(os.path, "getsize",
|
||||
side_effect=OSError("不可读")):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
chv_disk.provision_task_disk(tmp, "vm1", rootfs,
|
||||
"copy")
|
||||
self.assertEqual(ctx.exception.kind, "disk_quota_unverified")
|
||||
|
||||
def test_provision_copy_failure_rejected(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
rootfs = os.path.join(tmp, "base.raw")
|
||||
self._write(rootfs, b"1234")
|
||||
with mock.patch.object(chv_disk.shutil, "copyfile",
|
||||
side_effect=OSError("磁盘满")):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
chv_disk.provision_task_disk(tmp, "vm1", rootfs,
|
||||
"copy")
|
||||
self.assertEqual(ctx.exception.kind, "disk_quota_unverified")
|
||||
|
||||
def test_prune_delete_failure_is_best_effort(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
snap = os.path.join(tmp, "vm1", "old.snap")
|
||||
self._write(snap, b"old")
|
||||
chv_disk._save_snapshots(tmp, "vm1", [{
|
||||
"url": "file://" + snap, "path": snap, "size": 3,
|
||||
"created_at": 0}])
|
||||
with mock.patch.object(os, "remove",
|
||||
side_effect=OSError("删不掉")):
|
||||
kept, pruned = chv_disk.prune_snapshots(
|
||||
tmp, "vm1", retention_days=30, now=10 ** 9)
|
||||
self.assertEqual(kept, [])
|
||||
self.assertEqual(len(pruned), 1)
|
||||
self.assertTrue(os.path.exists(snap))
|
||||
|
||||
def test_snapshot_limits_uses_file_size_when_size_missing(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
snap = os.path.join(tmp, "vm1", "a.snap")
|
||||
self._write(snap, b"12345")
|
||||
chv_disk._save_snapshots(tmp, "vm1", [{
|
||||
"url": "file://" + snap, "path": snap,
|
||||
"created_at": 0}])
|
||||
kept = chv_disk.check_snapshot_limits(
|
||||
tmp, "vm1", max_count=10, max_total_gb=50.0,
|
||||
retention_days=None)
|
||||
self.assertEqual(len(kept), 1)
|
||||
|
||||
def test_snapshot_limits_ignores_unreadable_file_size(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
snap = os.path.join(tmp, "vm1", "a.snap")
|
||||
self._write(snap, b"12345")
|
||||
chv_disk._save_snapshots(tmp, "vm1", [{
|
||||
"url": "file://" + snap, "path": snap,
|
||||
"created_at": 0}])
|
||||
with mock.patch.object(os.path, "getsize",
|
||||
side_effect=OSError("不可读")):
|
||||
kept = chv_disk.check_snapshot_limits(
|
||||
tmp, "vm1", max_count=10, max_total_gb=50.0,
|
||||
retention_days=None)
|
||||
self.assertEqual(len(kept), 1)
|
||||
|
||||
def test_snapshot_limits_total_cap_via_file_size_rejected(self):
|
||||
# 总量统计走文件大小回落时同样触发总量帽:无 size
|
||||
# 字段的条目按实际文件大小计入,达到上限即拒绝。
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
snap = os.path.join(tmp, "vm1", "a.snap")
|
||||
self._write(snap, b"12345")
|
||||
chv_disk._save_snapshots(tmp, "vm1", [{
|
||||
"url": "file://" + snap, "path": snap,
|
||||
"created_at": 0}])
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
chv_disk.check_snapshot_limits(
|
||||
tmp, "vm1", max_count=10, max_total_gb=0.0,
|
||||
retention_days=None)
|
||||
self.assertEqual(ctx.exception.kind, "snapshot_quota_exceeded")
|
||||
|
||||
def test_record_file_snapshot_missing_file_size_none(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
entries, _removed = chv_disk.record_snapshot(
|
||||
tmp, "vm1", "file:///nonexistent-chv-snap", 50.0)
|
||||
self.assertEqual(len(entries), 1)
|
||||
self.assertIsNone(entries[0]["size"])
|
||||
self.assertEqual(entries[0]["path"], "/nonexistent-chv-snap")
|
||||
|
||||
def test_record_cap_removal_delete_failure_is_best_effort(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
old = os.path.join(tmp, "vm1", "old.snap")
|
||||
self._write(old, b"old")
|
||||
chv_disk._save_snapshots(tmp, "vm1", [{
|
||||
"url": "file://" + old, "path": old, "size": 10 ** 6,
|
||||
"created_at": 0}])
|
||||
real_remove = os.remove
|
||||
|
||||
def remove(path):
|
||||
if path == old:
|
||||
raise OSError("删不掉")
|
||||
return real_remove(path)
|
||||
|
||||
with mock.patch.object(os, "remove", side_effect=remove):
|
||||
entries, removed = chv_disk.record_snapshot(
|
||||
tmp, "vm1", "https://example.invalid/new", 0.0)
|
||||
self.assertEqual(len(removed), 1)
|
||||
self.assertEqual(removed[0]["path"], old)
|
||||
self.assertEqual(len(entries), 1)
|
||||
self.assertTrue(os.path.exists(old))
|
||||
|
||||
|
||||
class HardeningCover100Test(unittest.TestCase):
|
||||
def test_state_dir_stat_failure_rejected(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
state_dir = os.path.join(tmp, "state")
|
||||
real_stat = os.stat
|
||||
|
||||
def stat(path, *args, **kwargs):
|
||||
if path == state_dir:
|
||||
raise OSError("stat 失败")
|
||||
return real_stat(path, *args, **kwargs)
|
||||
|
||||
with mock.patch.object(os, "stat", side_effect=stat):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
hardening.ensure_secure_state_dir(state_dir)
|
||||
self.assertEqual(ctx.exception.kind, "state_dir_insecure")
|
||||
|
||||
def test_state_dir_still_loose_after_chmod_rejected(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
state_dir = os.path.join(tmp, "state")
|
||||
os.makedirs(state_dir)
|
||||
os.chmod(state_dir, 0o755)
|
||||
with mock.patch.object(os, "chmod",
|
||||
lambda *a, **k: None):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
hardening.ensure_secure_state_dir(state_dir)
|
||||
self.assertEqual(ctx.exception.kind, "state_dir_insecure")
|
||||
|
||||
def test_socket_is_live_empty_and_missing_is_dead(self):
|
||||
self.assertFalse(hardening.socket_is_live(""))
|
||||
self.assertFalse(hardening.socket_is_live(None))
|
||||
self.assertFalse(
|
||||
hardening.socket_is_live("/tmp/no-such-chv.sock"))
|
||||
|
||||
def test_socket_is_live_socket_error_is_dead(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = os.path.join(tmp, "dead.sock")
|
||||
with open(path, "w") as fh:
|
||||
fh.write("x")
|
||||
with mock.patch.object(hardening.socket, "socket",
|
||||
side_effect=OSError("socket 失败")):
|
||||
self.assertFalse(hardening.socket_is_live(path))
|
||||
|
||||
def test_parse_net_entries_empty_is_empty(self):
|
||||
self.assertEqual(hardening.parse_net_entries(""), [])
|
||||
self.assertEqual(hardening.parse_net_entries(None), [])
|
||||
|
||||
def test_parse_net_entries_non_dict_rejected(self):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
hardening.parse_net_entries('[1, 2]')
|
||||
self.assertEqual(ctx.exception.kind, "net_config_invalid")
|
||||
|
||||
def test_egress_proxy_empty_is_none(self):
|
||||
self.assertIsNone(hardening.validate_egress_proxy(""))
|
||||
self.assertIsNone(hardening.validate_egress_proxy(None))
|
||||
|
||||
def test_egress_proxy_unparsable_rejected(self):
|
||||
with mock.patch("urllib.parse.urlparse",
|
||||
side_effect=ValueError("bad url")):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
hardening.validate_egress_proxy("http://[::1")
|
||||
self.assertEqual(ctx.exception.kind, "config_invalid")
|
||||
|
||||
|
||||
class HostkeysCover100Test(unittest.TestCase):
|
||||
def test_build_seed_disk_subprocess_failure(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
out = os.path.join(tmp, "seed.iso")
|
||||
result = SimpleNamespace(returncode=3, stderr="boom")
|
||||
with mock.patch("subprocess.run", return_value=result):
|
||||
with self.assertRaises(RuntimeError) as ctx:
|
||||
hostkeys.build_seed_disk(tmp, out, runner=None)
|
||||
self.assertIn("genisoimage 失败", str(ctx.exception))
|
||||
|
||||
def test_build_seed_disk_chmod_failure(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
out = os.path.join(tmp, "seed.iso")
|
||||
|
||||
def runner(cmd):
|
||||
with open(out, "wb") as fh:
|
||||
fh.write(b"ISO")
|
||||
return SimpleNamespace(returncode=0)
|
||||
|
||||
with mock.patch.object(os, "chmod",
|
||||
side_effect=OSError("chmod 失败")):
|
||||
with self.assertRaises(RuntimeError) as ctx:
|
||||
hostkeys.build_seed_disk(tmp, out, runner=runner)
|
||||
self.assertIn("chmod 0600 失败", str(ctx.exception))
|
||||
|
||||
def test_write_seed_without_runcmd_still_appends_secret(self):
|
||||
# 防御分支:若 user-data 模板将来没有 runcmd 段,
|
||||
# 密钥环境条目必须追加而不是丢弃。
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
seed_dir = os.path.join(tmp, "seed")
|
||||
with mock.patch.object(
|
||||
hostkeys, "build_seed_user_data",
|
||||
return_value="#cloud-config\nwrite_files:\n"):
|
||||
hostkeys.write_seed(seed_dir, "vm1", "PRIV", "PUB",
|
||||
secret_env={"MY_KEY": "a b"})
|
||||
with open(os.path.join(seed_dir, "user-data")) as fh:
|
||||
content = fh.read()
|
||||
self.assertIn(hostkeys.GUEST_ENV_PATH, content)
|
||||
self.assertIn("MY_KEY=", content)
|
||||
|
||||
def test_keygen_subprocess_failure(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
result = SimpleNamespace(returncode=1, stderr="boom")
|
||||
with mock.patch("subprocess.run", return_value=result):
|
||||
with self.assertRaises(RuntimeError) as ctx:
|
||||
hostkeys.generate_host_keypair(tmp, "vm1",
|
||||
runner=None)
|
||||
self.assertIn("ssh-keygen 失败", str(ctx.exception))
|
||||
|
||||
def test_keygen_private_chmod_failure(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
paths = hostkeys.host_key_paths(tmp, "vm1")
|
||||
|
||||
def runner(cmd):
|
||||
os.makedirs(os.path.dirname(paths["private"]),
|
||||
exist_ok=True)
|
||||
for key in ("private", "public"):
|
||||
with open(paths[key], "w") as fh:
|
||||
fh.write("x")
|
||||
return SimpleNamespace(returncode=0)
|
||||
|
||||
with mock.patch.object(os, "chmod",
|
||||
side_effect=OSError("chmod 失败")):
|
||||
with self.assertRaises(RuntimeError) as ctx:
|
||||
hostkeys.generate_host_keypair(tmp, "vm1",
|
||||
runner=runner)
|
||||
self.assertIn("chmod 0600 失败", str(ctx.exception))
|
||||
|
||||
|
||||
class _PoolStub:
|
||||
"""WarmPool 用的假执行器:行为由属性开关控制。"""
|
||||
|
||||
def __init__(self, registry_path=None):
|
||||
self.state_dir = None
|
||||
self.warm_pool_size = 0
|
||||
self.snapshot_url = "file:///snap/base"
|
||||
self.registry_path = registry_path
|
||||
self.shutdown_calls = []
|
||||
self.shutdown_raises = False
|
||||
self.emit_raises = False
|
||||
self.woken = []
|
||||
self.wake_raises = False
|
||||
|
||||
def vm_name(self, task):
|
||||
raw = task.get("id") if isinstance(task, dict) else str(task)
|
||||
return raw if raw.startswith("chv-") else "chv-" + raw
|
||||
|
||||
def wake_vm(self, task):
|
||||
if self.wake_raises:
|
||||
raise RuntimeError("wake 失败")
|
||||
self.woken.append(task["id"])
|
||||
return {"vm": self.vm_name(task)}
|
||||
|
||||
def pause_vm(self, task):
|
||||
return True
|
||||
|
||||
def resume_vm(self, task):
|
||||
return True
|
||||
|
||||
def shutdown_vm(self, task_or_name):
|
||||
self.shutdown_calls.append(task_or_name)
|
||||
if self.shutdown_raises:
|
||||
raise RuntimeError("shutdown 失败")
|
||||
return True
|
||||
|
||||
def _emit(self, event, task, detail="", kind=None):
|
||||
if self.emit_raises:
|
||||
raise RuntimeError("audit 失败")
|
||||
|
||||
|
||||
class ExperimentalPoolCover100Test(unittest.TestCase):
|
||||
"""实验池覆盖(单独计分,不计入生产覆盖)。"""
|
||||
|
||||
def test_load_registry_invalid_entry_counted(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = os.path.join(tmp, "warm-pool.json")
|
||||
with open(path, "w") as fh:
|
||||
json.dump({
|
||||
"bad": "not-a-dict",
|
||||
"good": {"vm": "vm-good", "state": "warm",
|
||||
"task_id": "t1",
|
||||
"snapshot_url": None},
|
||||
}, fh)
|
||||
pool = WarmPool(_PoolStub(), registry_path=path)
|
||||
self.assertEqual(pool.registry_corrupt_entries, 1)
|
||||
self.assertEqual(pool.orphan_count(), 1)
|
||||
self.assertEqual(pool.warm_count(), 0)
|
||||
|
||||
def test_save_registry_skips_non_warm_entries(self):
|
||||
# orphan 是上一世待清理条目,落盘时不写回:重启后
|
||||
# 孤儿改由 socket 扫描兜底,不借 registry 复活。
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = os.path.join(tmp, "warm-pool.json")
|
||||
pool = WarmPool(_PoolStub(), registry_path=path)
|
||||
pool._entries["w"] = {"state": "warm", "vm": "w",
|
||||
"task": {"id": "w"},
|
||||
"snapshot_url": None}
|
||||
pool._entries["old"] = {"state": "orphan", "vm": "old",
|
||||
"task": {"id": "old"},
|
||||
"snapshot_url": None}
|
||||
pool._save_registry()
|
||||
with open(path) as fh:
|
||||
data = json.load(fh)
|
||||
self.assertEqual(list(data), ["w"])
|
||||
|
||||
def test_save_registry_non_dict_task_id_none(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
path = os.path.join(tmp, "warm-pool.json")
|
||||
pool = WarmPool(_PoolStub(), registry_path=path)
|
||||
pool._entries["w"] = {"state": "warm", "vm": "w",
|
||||
"task": "not-a-dict",
|
||||
"snapshot_url": None}
|
||||
pool._save_registry()
|
||||
with open(path) as fh:
|
||||
data = json.load(fh)
|
||||
self.assertIsNone(data["w"]["task_id"])
|
||||
|
||||
def test_reconcile_skips_warm_entries(self):
|
||||
pool = WarmPool(_PoolStub(), registry_path=None)
|
||||
pool._entries["warm-1"] = {"state": "warm", "vm": "vm-warm",
|
||||
"task": {"id": "warm-1"}}
|
||||
pool._entries["old-1"] = {"state": "orphan", "vm": "vm-old",
|
||||
"task": {"id": "old-1"}}
|
||||
cleaned = pool.reconcile_orphans()
|
||||
self.assertEqual(cleaned, ["vm-old"])
|
||||
self.assertIn("warm-1", pool._entries)
|
||||
self.assertNotIn("old-1", pool._entries)
|
||||
|
||||
def test_reconcile_shutdown_failure_not_cleaned(self):
|
||||
# feat-chv-p1fix(P1-01 同款语义,实验池):shutdown
|
||||
# 失败的条目不进 cleaned、不从内存条目里 pop,
|
||||
# 下次 reconcile 可重试。
|
||||
stub = _PoolStub()
|
||||
stub.shutdown_raises = True
|
||||
pool = WarmPool(stub, registry_path=None)
|
||||
pool._entries["old-1"] = {"state": "orphan", "vm": "vm-old",
|
||||
"task": {"id": "old-1"}}
|
||||
self.assertEqual(pool.reconcile_orphans(), [])
|
||||
self.assertIn("old-1", pool._entries)
|
||||
|
||||
def test_reconcile_emit_failure_is_best_effort(self):
|
||||
stub = _PoolStub()
|
||||
stub.emit_raises = True
|
||||
pool = WarmPool(stub, registry_path=None)
|
||||
pool._entries["old-1"] = {"state": "orphan", "vm": "vm-old",
|
||||
"task": {"id": "old-1"}}
|
||||
self.assertEqual(pool.reconcile_orphans(), ["vm-old"])
|
||||
|
||||
def test_prewarm_skips_colliding_name(self):
|
||||
stub = _PoolStub()
|
||||
pool = WarmPool(stub, size=2, registry_path=None)
|
||||
pool._entries["warm-1"] = {"state": "warm", "vm": "vm-1",
|
||||
"task": {"id": "warm-1"},
|
||||
"snapshot_url": pool.snapshot_url}
|
||||
added = pool.prewarm()
|
||||
self.assertEqual(added, ["warm-2"])
|
||||
self.assertEqual(stub.woken, ["warm-2"])
|
||||
self.assertEqual(pool.warm_count(), 2)
|
||||
|
||||
def test_prewarm_wake_failure_drops_entry_and_raises(self):
|
||||
stub = _PoolStub()
|
||||
stub.wake_raises = True
|
||||
pool = WarmPool(stub, size=1, registry_path=None)
|
||||
with self.assertRaises(RuntimeError):
|
||||
pool.prewarm()
|
||||
self.assertEqual(pool._entries, {})
|
||||
self.assertEqual(pool.warm_count(), 0)
|
||||
|
||||
def test_acquire_emit_failure_is_best_effort(self):
|
||||
stub = _PoolStub()
|
||||
stub.emit_raises = True
|
||||
pool = WarmPool(stub, size=1, registry_path=None)
|
||||
pool.prewarm()
|
||||
got = pool.acquire({"id": "t1"})
|
||||
self.assertEqual(got["mode"], "warm")
|
||||
|
||||
def test_release_shutdown_failure_still_reports_destroyed(self):
|
||||
stub = _PoolStub()
|
||||
stub.shutdown_raises = True
|
||||
pool = WarmPool(stub, size=1, registry_path=None)
|
||||
pool._entries["warm-1"] = {"state": "assigned", "vm": "vm-1",
|
||||
"task": {"id": "warm-1"}}
|
||||
self.assertTrue(pool.release({"id": "t1"}, pool_vm="warm-1"))
|
||||
self.assertNotIn("warm-1", pool._entries)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,646 @@
|
||||
"""CHV 审核修复轨测试(堆叠于 #230,base=feat-chv-gaps)。
|
||||
|
||||
只用标准库与 mock runner/launcher/api_client,绝不启动真实
|
||||
VM:当前环境没有可跑真 VM 的宿主。真实 VM 启动、seed 盘
|
||||
在真机上的 cloud-init 应用、SSH 可达实测一律不在本文件
|
||||
声称已验证;本文件只证明代码层面的接线(VmConfig 含 seed
|
||||
盘、restore 不回落、reuse 拦截、SSH 目标解析、配额/快照/
|
||||
API 完整性等)成立。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from unittest import mock
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
try:
|
||||
from cloud.chv.api_client import _unix_socket_api
|
||||
from cloud.chv.disk import check_disk_quota, provision_task_disk
|
||||
from cloud.chv.hostkeys import build_seed_disk, seed_disk_path
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
)
|
||||
from cloud.worker import Worker
|
||||
except ImportError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv.api_client import _unix_socket_api
|
||||
from cloud.chv.disk import check_disk_quota, provision_task_disk
|
||||
from cloud.chv.hostkeys import build_seed_disk, seed_disk_path
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
)
|
||||
from cloud.worker import Worker
|
||||
|
||||
|
||||
class FakeResult:
|
||||
def __init__(self, returncode=0, stdout="", stderr=""):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
class FakeProc:
|
||||
def __init__(self, alive=True):
|
||||
self._alive = alive
|
||||
|
||||
def poll(self):
|
||||
return None if self._alive else 1
|
||||
|
||||
def terminate(self):
|
||||
self._alive = False
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self._alive = False
|
||||
|
||||
|
||||
def _write(path, content=b"x"):
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(content)
|
||||
os.chmod(path, 0o755)
|
||||
return path
|
||||
|
||||
|
||||
class ChvFixCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.binary = _write(os.path.join(self.dir, "cloud-hypervisor"))
|
||||
self.ch_remote = _write(os.path.join(self.dir, "ch-remote"))
|
||||
self.kernel = _write(os.path.join(self.dir, "vmlinux"))
|
||||
self.rootfs = _write(os.path.join(self.dir, "disk.raw"),
|
||||
b"R" * 100)
|
||||
self.ssh_key = _write(os.path.join(self.dir, "id_chv"))
|
||||
self.kvm = _write(os.path.join(self.dir, "kvm"))
|
||||
self.cpuinfo = os.path.join(self.dir, "cpuinfo")
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu vmx sse\n")
|
||||
self.state_dir = os.path.join(self.dir, "state")
|
||||
self.calls = []
|
||||
self.api_calls = []
|
||||
self.api_payloads = {}
|
||||
self.audit = []
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _default_runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh":
|
||||
if cmd[-1] == "true":
|
||||
return FakeResult(0, "", "")
|
||||
if "goal" in cmd:
|
||||
return FakeResult(0, "goal met after rounds\n", "")
|
||||
return FakeResult(0, "", "")
|
||||
if cmd and cmd[0] == "ssh-keygen":
|
||||
priv = cmd[cmd.index("-f") + 1]
|
||||
with open(priv, "w") as fh:
|
||||
fh.write("-----BEGIN OPENSSH PRIVATE KEY-----\nFAKE\n")
|
||||
os.chmod(priv, 0o600)
|
||||
with open(priv + ".pub", "w") as fh:
|
||||
fh.write("ssh-ed25519 AAAAfake1 vm-key\n")
|
||||
return FakeResult(0, "", "")
|
||||
if cmd and cmd[0] == "genisoimage":
|
||||
out = cmd[cmd.index("-output") + 1]
|
||||
with open(out, "wb") as fh:
|
||||
fh.write(b"CIDATA" * 16)
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
def _default_api(self, socket_path, method, path, payload=None):
|
||||
self.api_calls.append((method, path))
|
||||
if path == "/vm.create":
|
||||
self.api_payloads["vm.create"] = payload
|
||||
return (200, {})
|
||||
|
||||
def make_exec(self, runner=None, env_extra=None, **overrides):
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
"CHV_SSH_FORWARD": "1"}
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
kwargs = dict(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, rootfs=self.rootfs,
|
||||
ssh_key=self.ssh_key, kvm_path=self.kvm,
|
||||
cpuinfo_path=self.cpuinfo, state_dir=self.state_dir,
|
||||
runner=runner or self._default_runner,
|
||||
launcher=lambda cmd: FakeProc(alive=True),
|
||||
api_client=self._default_api,
|
||||
sleeper=lambda s: None, env=env)
|
||||
kwargs.update(overrides)
|
||||
ex = CloudHypervisorExecutor(**kwargs)
|
||||
ex.audit_callback = lambda e, t, d, k=None: self.audit.append(
|
||||
(e, t.get("id") if isinstance(t, dict) else t, d, k))
|
||||
return ex
|
||||
|
||||
|
||||
# -- P0-1 seed 磁盘接入 -------------------------------------------------
|
||||
class SeedDiskTest(ChvFixCase):
|
||||
def test_vm_config_carries_seed_disk_readonly(self):
|
||||
ex = self.make_exec()
|
||||
# feat-chv-secfix(L2):磁盘路径须在 state_dir 内,
|
||||
# 旧 /tmp/x 路径不再合法,改用 state_dir 下路径。
|
||||
disk = os.path.join(self.state_dir, "x", "disk.raw")
|
||||
seed = os.path.join(self.state_dir, "x", "seed.iso")
|
||||
cfg = ex.build_vm_config(disk, seed_disk=seed)
|
||||
self.assertEqual(len(cfg["disks"]), 2)
|
||||
self.assertEqual(cfg["disks"][1]["path"], seed)
|
||||
self.assertTrue(cfg["disks"][1]["readonly"])
|
||||
cfg2 = ex.build_vm_config(disk)
|
||||
self.assertEqual(len(cfg2["disks"]), 1)
|
||||
|
||||
def test_vm_config_rejects_disk_outside_state_dir(self):
|
||||
ex = self.make_exec()
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.build_vm_config("/tmp/x/disk.raw")
|
||||
self.assertEqual(ctx.exception.kind, "disk_path_not_allowed")
|
||||
|
||||
def test_provisioned_wake_mounts_seed_disk(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_HOST_KEY_MODE": "provisioned"})
|
||||
out = ex.wake_vm({"id": "seed1"})
|
||||
self.assertEqual(out["state"], "running")
|
||||
payload = self.api_payloads["vm.create"]
|
||||
self.assertEqual(len(payload["disks"]), 2)
|
||||
seed_disk = payload["disks"][1]["path"]
|
||||
self.assertTrue(seed_disk.endswith("seed.iso"))
|
||||
self.assertTrue(payload["disks"][1]["readonly"])
|
||||
self.assertTrue(os.path.exists(seed_disk))
|
||||
self.assertIn("genisoimage", [c[0] for c in self.calls])
|
||||
|
||||
def test_seed_disk_build_failure_is_fail_fast(self):
|
||||
def bad_runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "genisoimage":
|
||||
return FakeResult(1, "", "no genisoimage")
|
||||
return self._default_runner(cmd, timeout)
|
||||
|
||||
ex = self.make_exec(
|
||||
runner=bad_runner,
|
||||
env_extra={"CHV_HOST_KEY_MODE": "provisioned"})
|
||||
with self.assertRaises(RuntimeError):
|
||||
ex.wake_vm({"id": "seedfail"})
|
||||
# 制盘失败在起 VMM 之前,不应走到 vm.create。
|
||||
self.assertNotIn("/vm.create",
|
||||
[p for _, p in self.api_calls])
|
||||
|
||||
def test_build_seed_disk_rejects_missing_product(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
build_seed_disk(self.dir, os.path.join(self.dir, "no.iso"),
|
||||
runner=lambda cmd: FakeResult(0, "", ""))
|
||||
|
||||
|
||||
# -- P0-2 restore 不回落 ------------------------------------------------
|
||||
class RestoreNoFallbackTest(ChvFixCase):
|
||||
def test_wait_guest_ready_provisioned_missing_known_hosts(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_HOST_KEY_MODE": "provisioned"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wait_guest_ready({"id": "nopin"})
|
||||
self.assertEqual(ctx.exception.kind, "host_key_not_pinned")
|
||||
self.assertFalse(any(c[0] == "ssh-keyscan" for c in self.calls))
|
||||
|
||||
def test_restore_provisioned_missing_known_hosts_fails_before_launch(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_HOST_KEY_MODE": "provisioned"})
|
||||
launches = []
|
||||
ex.launcher = lambda cmd: launches.append(cmd) or FakeProc(True)
|
||||
# feat-chv-secfix(L2):恢复源须在白名单目录内,
|
||||
# 本测试的意图是“起 VMM 前失败”,用 state_dir 下路径。
|
||||
snap_url = "file://" + os.path.join(self.state_dir, "snap", "x")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.restore_vm({"id": "r1"}, snap_url)
|
||||
self.assertEqual(ctx.exception.kind, "host_key_not_pinned")
|
||||
self.assertEqual(launches, [])
|
||||
|
||||
|
||||
# -- P0-3 reuse 拦截 ----------------------------------------------------
|
||||
class ReuseBlockTest(ChvFixCase):
|
||||
def test_reuse_blocked_without_experiment_flag(self):
|
||||
ex = self.make_exec(env_extra={"CHV_DISK_MODE": "reuse"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind,
|
||||
"disk_mode_reuse_blocked")
|
||||
|
||||
def test_reuse_passes_with_experiment_flag_and_is_audited(self):
|
||||
ex = self.make_exec(env_extra={
|
||||
"CHV_DISK_MODE": "reuse",
|
||||
"CHV_REUSE_SINGLE_TENANT_EXPERIMENT": "1",
|
||||
# feat-chv-secfix(M6):实验放行还须声明单租户项目。
|
||||
"CHV_SINGLE_TENANT_PROJECT": "p1"})
|
||||
self.assertTrue(ex.preflight())
|
||||
out = ex.wake_vm({"id": "reuse1"})
|
||||
self.assertEqual(out["state"], "running")
|
||||
self.assertTrue(any(e == "disk_reuse_experiment_allowed"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
|
||||
# -- P0-4 SSH 目标解析 --------------------------------------------------
|
||||
class SshTargetTest(ChvFixCase):
|
||||
def test_explicit_host_wins(self):
|
||||
ex = self.make_exec(ssh_host="10.9.8.7")
|
||||
self.assertEqual(ex.resolve_ssh_host(), "10.9.8.7")
|
||||
|
||||
def test_net_guest_ip_used_when_no_explicit_host(self):
|
||||
ex = self.make_exec(env_extra={
|
||||
"CHV_NET_JSON": '[{"tap": "", "ip": "192.168.100.5",'
|
||||
'"mask": "255.255.255.0"}]',
|
||||
"CHV_SSH_FORWARD": "0"})
|
||||
# env_extra 覆盖 forward=0 后仍应走 net ip,不报未声明。
|
||||
self.assertEqual(ex.resolve_ssh_host(), "192.168.100.5")
|
||||
cmd = ex.build_ssh_command({"id": "t"}, ["true"])
|
||||
self.assertIn("root@192.168.100.5", " ".join(cmd))
|
||||
|
||||
def test_undeclared_target_is_fail_fast(self):
|
||||
ex = self.make_exec(env_extra={"CHV_SSH_FORWARD": "0"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.resolve_ssh_host()
|
||||
self.assertEqual(ctx.exception.kind, "ssh_target_undeclared")
|
||||
|
||||
def test_forward_attestation_allows_loopback(self):
|
||||
ex = self.make_exec()
|
||||
self.assertEqual(ex.resolve_ssh_host(), "127.0.0.1")
|
||||
|
||||
|
||||
# -- P1-1 网关 extra_env ------------------------------------------------
|
||||
class GatewayExtraEnvTest(ChvFixCase):
|
||||
def test_chv_ssh_command_carries_extra_env(self):
|
||||
# feat-chv-secfix(H2):密钥值绝不再进 argv。
|
||||
# 非密钥 extra_env 仍走 env 前缀;密钥子集须 seed
|
||||
# 投递后才 source 客内文件(见 worker 网关测试)。
|
||||
ex = self.make_exec()
|
||||
cmd = ex.build_ssh_command({"id": "t"}, ["true"],
|
||||
extra_env={"SUB2API_BASE_URL":
|
||||
"http://gw/"})
|
||||
joined = " ".join(cmd)
|
||||
self.assertIn("SUB2API_BASE_URL=http://gw/", joined)
|
||||
self.assertIn("HTTPS_PROXY=http://10.0.0.1:8888", joined)
|
||||
with self.assertRaises(ChvFailure):
|
||||
ex.build_ssh_command({"id": "t"}, ["true"],
|
||||
extra_env={"SUB2API_API_KEY": "k1"})
|
||||
|
||||
def test_worker_gateway_guard_allows_chv(self):
|
||||
from cloud.server import init_db
|
||||
db_path = os.path.join(self.dir, "control.db")
|
||||
init_db(db_path)
|
||||
ex = self.make_exec()
|
||||
worker = Worker(db_path, "/fake/host.py", executor=ex,
|
||||
env={"XCM_API_KEY": "test-placeholder"})
|
||||
task = {"id": "gw1", "project_id": "p1", "prompt": "做目标",
|
||||
"goal_mode": 1,
|
||||
"_gateway_provider": "sub2api",
|
||||
"_gateway_env": {"SUB2API_API_KEY": "gwkey",
|
||||
"SUB2API_BASE_URL": "http://gw/"}}
|
||||
result, error = worker._run_attempt(task)
|
||||
self.assertIsNone(error)
|
||||
self.assertEqual(result["status"], "completed")
|
||||
ssh_calls = [c for c in self.calls if c and c[0] == "ssh"]
|
||||
# feat-chv-secfix(H2):密钥值不得出现在任何 ssh
|
||||
# argv 里;远程命令改为 source 客内 seed 密钥文件。
|
||||
self.assertFalse(any("SUB2API_API_KEY=gwkey" in " ".join(c)
|
||||
for c in ssh_calls))
|
||||
self.assertTrue(any("/etc/chv/guest-env" in " ".join(c)
|
||||
for c in ssh_calls))
|
||||
# 任务结束 stop 后 seed 目录与宿主私钥均已清(H3/H5;
|
||||
# seed 内容本身的断言见 test_chv_secfix 的直接
|
||||
# provision_host_key 测试,stop 前 seed 仍在)。
|
||||
self.assertFalse(os.path.exists(
|
||||
os.path.join(self.state_dir, "chv-gw1", "seed")))
|
||||
self.assertFalse(os.path.exists(
|
||||
os.path.join(self.state_dir, "chv-gw1",
|
||||
"ssh_host_ed25519_key")))
|
||||
|
||||
|
||||
# -- P1-2 取消/清理/看门狗 ----------------------------------------------
|
||||
class CancelCleanupTest(ChvFixCase):
|
||||
def test_worker_cancel_calls_cancel_vm(self):
|
||||
from cloud.server import init_db
|
||||
db_path = os.path.join(self.dir, "control.db")
|
||||
init_db(db_path)
|
||||
ex = self.make_exec()
|
||||
cancelled = []
|
||||
ex.cancel_vm = lambda name: cancelled.append(name) or True
|
||||
worker = Worker(db_path, "/fake/host.py", executor=ex,
|
||||
env={"XCM_API_KEY": "test-placeholder"})
|
||||
worker._running_set("t-cancel")
|
||||
import sqlite3, time
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.execute(
|
||||
"INSERT INTO tasks(id, project_id, prompt, status,"
|
||||
" idempotency_key, depends_on, priority, created_by,"
|
||||
" created_at, updated_at, last_heartbeat)"
|
||||
" VALUES(?, ?, ?, ?, NULL, ?, ?, 'tester', ?, ?, NULL)",
|
||||
("t-cancel", "p1", "做事", "running", "[]", 0,
|
||||
time.time(), time.time()))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
self.assertTrue(worker.cancel("t-cancel"))
|
||||
self.assertEqual(cancelled, ["t-cancel"])
|
||||
|
||||
def test_worker_cleanup_calls_cleanup_stale_vms(self):
|
||||
from cloud.server import init_db
|
||||
db_path = os.path.join(self.dir, "control.db")
|
||||
init_db(db_path)
|
||||
ex = self.make_exec()
|
||||
ex.cleanup_stale_vms = lambda: ["chv-stale"]
|
||||
worker = Worker(db_path, "/fake/host.py", executor=ex,
|
||||
env={"XCM_API_KEY": "test-placeholder"})
|
||||
self.assertIn("chv-stale", worker.cleanup_stale_containers())
|
||||
|
||||
def test_chv_cancel_watchdog_kills_vm_on_cancel(self):
|
||||
import threading
|
||||
from cloud.server import init_db
|
||||
db_path = os.path.join(self.dir, "control.db")
|
||||
init_db(db_path)
|
||||
ex = self.make_exec()
|
||||
killed = []
|
||||
ex.cancel_vm = lambda name: killed.append(name) or True
|
||||
worker = Worker(db_path, "/fake/host.py", executor=ex,
|
||||
env={"XCM_API_KEY": "test-placeholder"})
|
||||
worker._cancel_requested.add("t-wd")
|
||||
stop = threading.Event()
|
||||
worker._chv_cancel_watchdog("t-wd", "chv-t-wd", stop)
|
||||
self.assertEqual(killed, ["chv-t-wd"])
|
||||
|
||||
|
||||
# -- P1-3 WarmPool acquire resume ---------------------------------------
|
||||
class PoolAcquireResumeTest(unittest.TestCase):
|
||||
def test_acquire_resumes_paused_vm(self):
|
||||
class Stub:
|
||||
def __init__(self):
|
||||
self.resumed = []
|
||||
self.state_dir = None
|
||||
self.warm_pool_size = 1
|
||||
self.snapshot_url = "file:///snap/base"
|
||||
|
||||
def vm_name(self, task):
|
||||
raw = task.get("id") if isinstance(task, dict) \
|
||||
else str(task)
|
||||
return raw if raw.startswith("chv-") \
|
||||
else "chv-" + raw
|
||||
|
||||
def wake_vm(self, task):
|
||||
return {"vm": self.vm_name(task)}
|
||||
|
||||
def pause_vm(self, task):
|
||||
return True
|
||||
|
||||
def resume_vm(self, task):
|
||||
self.resumed.append(task["id"])
|
||||
return True
|
||||
|
||||
def shutdown_vm(self, task_or_name):
|
||||
return True
|
||||
|
||||
def _emit(self, event, task, detail="", kind=None):
|
||||
pass
|
||||
|
||||
stub = Stub()
|
||||
pool = WarmPool(stub, size=1, snapshot_url="file:///snap/base",
|
||||
registry_path=None)
|
||||
pool.prewarm()
|
||||
got = pool.acquire({"id": "t1"})
|
||||
self.assertEqual(got["mode"], "warm")
|
||||
self.assertEqual(stub.resumed, ["warm-1"])
|
||||
|
||||
def test_acquire_discards_vm_whose_resume_fails(self):
|
||||
class Stub:
|
||||
def __init__(self):
|
||||
self.state_dir = None
|
||||
self.warm_pool_size = 1
|
||||
self.snapshot_url = "file:///snap/base"
|
||||
|
||||
def vm_name(self, task):
|
||||
raw = task.get("id") if isinstance(task, dict) \
|
||||
else str(task)
|
||||
return raw if raw.startswith("chv-") \
|
||||
else "chv-" + raw
|
||||
|
||||
def wake_vm(self, task):
|
||||
return {"vm": self.vm_name(task)}
|
||||
|
||||
def pause_vm(self, task):
|
||||
return True
|
||||
|
||||
def resume_vm(self, task):
|
||||
raise RuntimeError("resume 失败")
|
||||
|
||||
def shutdown_vm(self, task_or_name):
|
||||
return True
|
||||
|
||||
def _emit(self, event, task, detail="", kind=None):
|
||||
pass
|
||||
|
||||
pool = WarmPool(Stub(), size=1, snapshot_url="file:///snap/base",
|
||||
registry_path=None)
|
||||
pool.prewarm()
|
||||
self.assertEqual(pool.acquire({"id": "t1"})["mode"], "cold")
|
||||
|
||||
|
||||
# -- P1-4 容量持久化计数 -------------------------------------------------
|
||||
class CapacityPersistentTest(ChvFixCase):
|
||||
def test_socket_counts_after_restart(self):
|
||||
# feat-chv-secfix(M7):只有真能连通的 socket 才
|
||||
# 计数——用真实监听 socket 代表活 VMM。
|
||||
ex = self.make_exec(env_extra={"CHV_MAX_VMS": "1"})
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
sock_path = os.path.join(self.state_dir, "chv-other.sock")
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(sock_path)
|
||||
srv.listen(1)
|
||||
try:
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm({"id": "new1"})
|
||||
self.assertEqual(ctx.exception.kind, "host_capacity_exceeded")
|
||||
finally:
|
||||
srv.close()
|
||||
|
||||
def test_dead_socket_file_does_not_block(self):
|
||||
# M7 的另一面:空文件/死 socket 不再占满容量。
|
||||
ex = self.make_exec(env_extra={"CHV_MAX_VMS": "1"})
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(os.path.join(self.state_dir, "chv-other.sock"),
|
||||
"w") as fh:
|
||||
fh.write("")
|
||||
out = ex.wake_vm({"id": "new1"})
|
||||
self.assertEqual(out["state"], "running")
|
||||
|
||||
def test_own_socket_does_not_block_rewake(self):
|
||||
ex = self.make_exec(env_extra={"CHV_MAX_VMS": "1"})
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(os.path.join(self.state_dir, "chv-t1.sock"),
|
||||
"w") as fh:
|
||||
fh.write("")
|
||||
# 同任务重复 wake:自己的残留 socket 不计入容量。
|
||||
out = ex.wake_vm({"id": "t1"})
|
||||
self.assertEqual(out["state"], "running")
|
||||
|
||||
|
||||
# -- P1-5 磁盘配额 -------------------------------------------------------
|
||||
class DiskQuotaTest(ChvFixCase):
|
||||
def test_unreadable_image_is_not_silently_skipped(self):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
check_disk_quota("/nonexistent/disk.raw", 1.0)
|
||||
self.assertEqual(ctx.exception.kind, "disk_quota_unverified")
|
||||
|
||||
def test_total_usage_over_quota(self):
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(os.path.join(self.state_dir, "old.raw"),
|
||||
"wb") as fh:
|
||||
fh.write(b"X" * 100)
|
||||
# 配额 150 字节:已用 100 + 本次 100 > 150。
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
check_disk_quota(self.rootfs, 150 / (1024 ** 3),
|
||||
state_dir=self.state_dir)
|
||||
self.assertEqual(ctx.exception.kind, "disk_quota_exceeded")
|
||||
|
||||
def test_existing_disk_size_mismatch_is_recopied(self):
|
||||
task_disk = provision_task_disk(self.state_dir, "chv-a",
|
||||
self.rootfs, "copy")
|
||||
with open(task_disk, "wb") as fh:
|
||||
fh.write(b"Z" * 10)
|
||||
task_disk2 = provision_task_disk(self.state_dir, "chv-a",
|
||||
self.rootfs, "copy")
|
||||
self.assertEqual(task_disk, task_disk2)
|
||||
with open(task_disk2, "rb") as fh:
|
||||
self.assertEqual(fh.read(), b"R" * 100)
|
||||
|
||||
|
||||
# -- P1-6 API 截断 -------------------------------------------------------
|
||||
class ApiTruncationTest(unittest.TestCase):
|
||||
def _serve(self, raw):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
sock_path = os.path.join(tmp.name, "vmm.sock")
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(sock_path)
|
||||
srv.listen(1)
|
||||
|
||||
def _run():
|
||||
conn, _ = srv.accept()
|
||||
try:
|
||||
conn.recv(65536)
|
||||
conn.sendall(raw)
|
||||
finally:
|
||||
conn.close()
|
||||
srv.close()
|
||||
|
||||
threading.Thread(target=_run, daemon=True).start()
|
||||
return tmp, sock_path
|
||||
|
||||
def test_truncated_body_raises(self):
|
||||
body = b'{"ok": true}'
|
||||
raw = (b"HTTP/1.1 200 OK\r\nContent-Length: 100\r\n\r\n"
|
||||
+ body)
|
||||
tmp, sock_path = self._serve(raw)
|
||||
with self.assertRaises(RuntimeError):
|
||||
_unix_socket_api(sock_path, "GET", "/vmm.ping")
|
||||
tmp.cleanup()
|
||||
|
||||
def test_chunked_rejected(self):
|
||||
raw = (b"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n"
|
||||
b"5\r\nhello\r\n0\r\n\r\n")
|
||||
tmp, sock_path = self._serve(raw)
|
||||
with self.assertRaises(RuntimeError):
|
||||
_unix_socket_api(sock_path, "GET", "/vmm.ping")
|
||||
tmp.cleanup()
|
||||
|
||||
def test_full_body_parses(self):
|
||||
body = b'{"ok": true}'
|
||||
raw = (b"HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n"
|
||||
% len(body)) + body
|
||||
tmp, sock_path = self._serve(raw)
|
||||
status, parsed = _unix_socket_api(sock_path, "GET", "/vmm.ping")
|
||||
self.assertEqual(status, 200)
|
||||
self.assertTrue(parsed["ok"])
|
||||
tmp.cleanup()
|
||||
|
||||
|
||||
# -- P1-7 快照落盘验证 ---------------------------------------------------
|
||||
class SnapshotPersistTest(ChvFixCase):
|
||||
def test_snapshot_missing_file_does_not_shutdown(self):
|
||||
ex = self.make_exec()
|
||||
# feat-chv-secfix(L2):快照须在白名单目录内,
|
||||
# 用 state_dir 下的不存在路径,仍验 snapshot_not_persisted。
|
||||
missing = os.path.join(self.state_dir, "missing.snap")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.hibernate_vm({"id": "h1"}, "file://" + missing)
|
||||
self.assertEqual(ctx.exception.kind, "snapshot_not_persisted")
|
||||
kinds = [c[-1] for c in self.calls if c[0] == self.ch_remote]
|
||||
self.assertIn("pause", kinds)
|
||||
self.assertNotIn("shutdown", kinds)
|
||||
|
||||
|
||||
# -- P1-8 execute 死端口归类 ---------------------------------------------
|
||||
class ExecuteDeadPortTest(ChvFixCase):
|
||||
def test_dead_socket_ssh_failure_is_vm_not_running(self):
|
||||
def raising_runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh":
|
||||
raise RuntimeError("connection refused")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=raising_runner)
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(os.path.join(self.state_dir, "chv-dead.sock"),
|
||||
"w") as fh:
|
||||
fh.write("")
|
||||
ex._vms["chv-dead"] = FakeProc(alive=False)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.execute({"id": "dead"})
|
||||
self.assertEqual(ctx.exception.kind, "vm_not_running")
|
||||
|
||||
|
||||
# -- P2 配置校验时机 / 文档口径 ------------------------------------------
|
||||
class ConfigTimingAndDocsTest(ChvFixCase):
|
||||
def test_preflight_rejects_bad_cpu_and_memory(self):
|
||||
ex = self.make_exec(cpus="abc")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "config_invalid")
|
||||
ex2 = self.make_exec(memory="not-a-size")
|
||||
with self.assertRaises(ChvFailure) as ctx2:
|
||||
ex2.preflight()
|
||||
self.assertEqual(ctx2.exception.kind, "config_invalid")
|
||||
|
||||
def test_stop_cleanup_failure_is_audited(self):
|
||||
ex = self.make_exec()
|
||||
ex.stop_vm({"id": "ck"})
|
||||
os.makedirs(os.path.join(self.state_dir, "chv-ck", "seed"),
|
||||
exist_ok=True)
|
||||
with mock.patch("cloud.chv.executor.shutil.rmtree",
|
||||
side_effect=OSError("rm failed")):
|
||||
ex.stop_vm({"id": "ck"})
|
||||
self.assertTrue(any(e == "vm_key_cleanup_failed"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
def test_readme_sections_in_order_and_buildinfo_honest(self):
|
||||
readme = Path(__file__).resolve().parents[2] \
|
||||
/ "chv-guest" / "README.md"
|
||||
text = readme.read_text()
|
||||
self.assertLess(text.index("## 5. Host key"),
|
||||
text.index("## 6. 与执行器的对接"))
|
||||
self.assertIn("contents: [清单逐项名]", text)
|
||||
self.assertIn("Node.js 不装", text)
|
||||
|
||||
def test_manual_vm_command_is_private(self):
|
||||
ex = self.make_exec()
|
||||
self.assertFalse(hasattr(ex, "build_vm_command"))
|
||||
cmd = ex._build_vm_command({"id": "t"})
|
||||
self.assertIn("--api-socket", cmd)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,465 @@
|
||||
"""CHV 补缺口轨测试(堆叠于 #221,base=feat-chv-optimize)。
|
||||
|
||||
只用标准库与 mock runner/launcher/api_client,绝不启动真实
|
||||
VM:当前环境没有可跑真 VM 的宿主(生产主机是 CVM、无
|
||||
/dev/kvm、无嵌套虚拟化;本 VM 容器内同样无 /dev/kvm)。
|
||||
真实 VM 启动、快照恢复、压测一律不在本文件声称已验证。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
try:
|
||||
from cloud.chv.hostkeys import (
|
||||
build_seed_user_data,
|
||||
generate_host_keypair,
|
||||
host_key_paths,
|
||||
known_hosts_line,
|
||||
write_seed,
|
||||
)
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
_parse_memory_bytes,
|
||||
)
|
||||
from cloud.worker import Worker, _call_goal_resume_breakpoint
|
||||
except ImportError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv.hostkeys import (
|
||||
build_seed_user_data,
|
||||
generate_host_keypair,
|
||||
host_key_paths,
|
||||
known_hosts_line,
|
||||
write_seed,
|
||||
)
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
_parse_memory_bytes,
|
||||
)
|
||||
from cloud.worker import Worker, _call_goal_resume_breakpoint
|
||||
|
||||
|
||||
class FakeResult:
|
||||
def __init__(self, returncode=0, stdout="", stderr=""):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
class FakeProc:
|
||||
def __init__(self, alive=True):
|
||||
self._alive = alive
|
||||
|
||||
def poll(self):
|
||||
return None if self._alive else 1
|
||||
|
||||
def terminate(self):
|
||||
self._alive = False
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self._alive = False
|
||||
|
||||
|
||||
def _write(path, content=b"x"):
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(content)
|
||||
os.chmod(path, 0o755)
|
||||
return path
|
||||
|
||||
|
||||
class ChvGapCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.binary = _write(os.path.join(self.dir, "cloud-hypervisor"))
|
||||
self.ch_remote = _write(os.path.join(self.dir, "ch-remote"))
|
||||
self.kernel = _write(os.path.join(self.dir, "vmlinux"))
|
||||
self.rootfs = _write(os.path.join(self.dir, "disk.raw"),
|
||||
b"R" * 100)
|
||||
self.ssh_key = _write(os.path.join(self.dir, "id_chv"))
|
||||
self.kvm = _write(os.path.join(self.dir, "kvm"))
|
||||
self.cpuinfo = os.path.join(self.dir, "cpuinfo")
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu vmx sse\n")
|
||||
self.state_dir = os.path.join(self.dir, "state")
|
||||
self.calls = []
|
||||
self.audit = []
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _default_runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh":
|
||||
if cmd[-1] == "true":
|
||||
return FakeResult(0, "", "")
|
||||
if "goal-resume" in cmd:
|
||||
return FakeResult(0, "goal met after rounds\n", "")
|
||||
if "goal" in cmd:
|
||||
return FakeResult(0, "goal met after rounds\n", "")
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
def _default_api(self, socket_path, method, path, payload=None):
|
||||
return (200, {})
|
||||
|
||||
def _default_launcher(self, cmd):
|
||||
return FakeProc(alive=True)
|
||||
|
||||
def make_exec(self, runner=None, env_extra=None, **overrides):
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
# 实验室宣称:mock runner 假装回环端口已转发到客机
|
||||
#(feat-chv-fix 起,执行器不再无条件假设回环能通)。
|
||||
# 实验室显式 opt-in keyscan:生产默认已是
|
||||
# provisioned;本套件旧 fake 不实现 ssh-keygen/
|
||||
# genisoimage,测的不是 host key 模式,故显式
|
||||
# 声明实验室路径,绝不把它写成生产默认。
|
||||
"CHV_HOST_KEY_MODE": "keyscan",
|
||||
"CHV_SSH_FORWARD": "1"}
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
kwargs = dict(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, rootfs=self.rootfs,
|
||||
ssh_key=self.ssh_key, kvm_path=self.kvm,
|
||||
cpuinfo_path=self.cpuinfo, state_dir=self.state_dir,
|
||||
runner=runner or self._default_runner,
|
||||
launcher=self._default_launcher,
|
||||
api_client=self._default_api,
|
||||
sleeper=lambda s: None, env=env)
|
||||
kwargs.update(overrides)
|
||||
ex = CloudHypervisorExecutor(**kwargs)
|
||||
ex.audit_callback = lambda e, t, d, k=None: self.audit.append(
|
||||
(e, t.get("id") if isinstance(t, dict) else t, d, k))
|
||||
return ex
|
||||
|
||||
|
||||
# -- (1) worker 断点接线 ---------------------------------------------
|
||||
class BreakpointWiringTest(ChvGapCase):
|
||||
def test_call_helper_passes_model_provider_when_accepted(self):
|
||||
seen = {}
|
||||
|
||||
class FakeChv:
|
||||
def goal_resume_breakpoint_task(self, task, model_provider=None):
|
||||
seen["provider"] = model_provider
|
||||
return {"status": "completed", "task_id": task["id"]}
|
||||
|
||||
out = _call_goal_resume_breakpoint(
|
||||
FakeChv(), {"id": "bp1"}, model_provider="prov-x")
|
||||
self.assertEqual(out["status"], "completed")
|
||||
self.assertEqual(seen["provider"], "prov-x")
|
||||
|
||||
def test_call_helper_single_arg_executor_still_works(self):
|
||||
class FakeOld:
|
||||
def goal_resume_breakpoint_task(self, task):
|
||||
return {"status": "completed", "task_id": task["id"]}
|
||||
|
||||
out = _call_goal_resume_breakpoint(
|
||||
FakeOld(), {"id": "bp2"}, model_provider="prov-x")
|
||||
self.assertEqual(out["status"], "completed")
|
||||
|
||||
def test_chv_breakpoint_accepts_model_provider(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.goal_resume_breakpoint_task(
|
||||
{"id": "g1", "prompt": "续跑"}, model_provider="prov-x")
|
||||
self.assertEqual(out["status"], "completed")
|
||||
self.assertTrue(out["goal_met"])
|
||||
# 客内命令带 --model-provider(网关 key 本身不进命令值)。
|
||||
ssh_calls = [c for c in self.calls if c and c[0] == "ssh"]
|
||||
self.assertTrue(any("--model-provider" in c for c in ssh_calls))
|
||||
|
||||
def test_worker_breakpoint_branch_calls_generic_executor(self):
|
||||
# feat-chv-fix(P2-3):不再用源码 grep 充数,改为行为
|
||||
# 断言——真实 Worker._run_attempt 走通用断点分支,调到
|
||||
# 非 Local 执行器的 goal_resume_breakpoint_task,并把
|
||||
# 结果原样返回(参数、守卫顺序、返回值都覆盖)。
|
||||
from cloud.server import init_db
|
||||
db_path = os.path.join(self.dir, "control.db")
|
||||
init_db(db_path)
|
||||
seen = {}
|
||||
|
||||
class FakeGeneric:
|
||||
def goal_resume_breakpoint_task(self, task,
|
||||
model_provider=None,
|
||||
extra_env=None):
|
||||
seen["provider"] = model_provider
|
||||
seen["task_id"] = task["id"]
|
||||
return {"executor": "generic", "task_id": task["id"],
|
||||
"status": "completed", "stdout": "",
|
||||
"stderr": "", "returncode": 0,
|
||||
"host_status": None, "goal_met": True,
|
||||
"goal_error": None}
|
||||
|
||||
worker = Worker(db_path, "/fake/host.py",
|
||||
executor=FakeGeneric(),
|
||||
env={"XCM_API_KEY": "test-placeholder"})
|
||||
task = {"id": "bp-behav", "project_id": "p1",
|
||||
"prompt": "续跑", "goal_mode": 1,
|
||||
"_goal_resume_breakpoint": True}
|
||||
result, error = worker._run_attempt(task)
|
||||
self.assertIsNone(error)
|
||||
self.assertEqual(result["status"], "completed")
|
||||
self.assertTrue(result["goal_met"])
|
||||
self.assertEqual(seen["task_id"], "bp-behav")
|
||||
self.assertNotIn("executor 不支持", str(result))
|
||||
|
||||
|
||||
# -- (2) reconcile_orphans 接线 ---------------------------------------
|
||||
class ReconcileWiringTest(ChvGapCase):
|
||||
def test_worker_startup_and_close_call_reconcile(self):
|
||||
from cloud.server import init_db
|
||||
db_path = os.path.join(self.dir, "control.db")
|
||||
init_db(db_path)
|
||||
ex = self.make_exec()
|
||||
# 手工造一个 orphan:先建池写 registry,再新开池载入。
|
||||
pool1 = WarmPool(ex)
|
||||
pool1._entries["chv-old"] = {
|
||||
"state": "warm", "vm": "chv-old",
|
||||
"task": {"id": "old"}, "snapshot_url": None}
|
||||
pool1._save_registry()
|
||||
self.assertTrue(
|
||||
os.path.exists(os.path.join(self.state_dir, "warm-pool.json")))
|
||||
# Worker 构造即 startup reconcile:用假 shutdown 记录。
|
||||
# 本轨 worker 不再持有 _warm_pool(生产走 orphans
|
||||
# facade);实验池只用于本测试造 registry 种子。
|
||||
shutdowns = []
|
||||
orig_shutdown = ex.shutdown_vm
|
||||
ex.shutdown_vm = lambda name: shutdowns.append(name) or True
|
||||
worker = Worker(db_path, "/fake/host.py", executor=ex,
|
||||
env={"XCM_API_KEY": "test-placeholder"})
|
||||
self.assertIn("chv-old", shutdowns)
|
||||
self.assertFalse(hasattr(worker, "_warm_pool"))
|
||||
with open(os.path.join(self.state_dir, "warm-pool.json")) as fh:
|
||||
self.assertEqual(json.load(fh), {})
|
||||
worker.close()
|
||||
# close 时再次 reconcile 不炸(池已空,返回 [] 即可)。
|
||||
self.assertEqual(worker._reconcile_warm_pool("shutdown"), [])
|
||||
|
||||
def test_reconcile_failure_does_not_break_close(self):
|
||||
from cloud.server import init_db
|
||||
import cloud.chv.orphans as orphans
|
||||
db_path = os.path.join(self.dir, "control.db")
|
||||
init_db(db_path)
|
||||
ex = self.make_exec()
|
||||
worker = Worker(db_path, "/fake/host.py", executor=ex,
|
||||
env={"XCM_API_KEY": "test-placeholder"})
|
||||
# facade 抛错时 close 仍不炸(worker 方法内每次现取
|
||||
# facade,猴子补丁模块属性即生效)。
|
||||
orig = orphans.reconcile_orphans_for_executor
|
||||
orphans.reconcile_orphans_for_executor = lambda ex: (
|
||||
(_ for _ in ()).throw(RuntimeError("boom")))
|
||||
try:
|
||||
worker.close() # 不抛即通过
|
||||
finally:
|
||||
orphans.reconcile_orphans_for_executor = orig
|
||||
|
||||
|
||||
# -- (3) SSH host key 预置 ---------------------------------------------
|
||||
class HostKeyProvisionTest(ChvGapCase):
|
||||
def _fake_keygen_runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh-keygen":
|
||||
priv = cmd[cmd.index("-f") + 1]
|
||||
with open(priv, "w") as fh:
|
||||
fh.write("-----BEGIN OPENSSH PRIVATE KEY-----\nFAKE\n")
|
||||
os.chmod(priv, 0o600)
|
||||
with open(priv + ".pub", "w") as fh:
|
||||
fh.write("ssh-ed25519 AAAAfake1 vm-key\n")
|
||||
return FakeResult(0, "", "")
|
||||
return self._default_runner(cmd, timeout)
|
||||
|
||||
def test_per_vm_keys_differ_and_known_hosts_pinned(self):
|
||||
ex = self.make_exec(runner=self._fake_keygen_runner)
|
||||
r1 = ex.provision_host_key({"id": "vm-a"})
|
||||
r2 = ex.provision_host_key({"id": "vm-b"})
|
||||
# 不同 VM 的私钥文件不同路径、known_hosts 不同文件。
|
||||
self.assertNotEqual(r1["paths"]["private"],
|
||||
r2["paths"]["private"])
|
||||
self.assertNotEqual(r1["known_hosts"], r2["known_hosts"])
|
||||
for r in (r1, r2):
|
||||
with open(r["known_hosts"]) as fh:
|
||||
line = fh.read().strip()
|
||||
self.assertTrue(line.startswith("[127.0.0.1]:"))
|
||||
self.assertIn("ssh-ed25519 AAAAfake1", line)
|
||||
self.assertNotIn("PRIVATE KEY", line)
|
||||
# 同一 VM 再次预置复用已有对(不重生成、不改 known_hosts)。
|
||||
with open(r1["known_hosts"]) as fh:
|
||||
before = fh.read()
|
||||
ex.provision_host_key({"id": "vm-a"})
|
||||
with open(r1["known_hosts"]) as fh:
|
||||
self.assertEqual(fh.read(), before)
|
||||
|
||||
def test_seed_contains_private_key_and_is_deleted_on_stop(self):
|
||||
ex = self.make_exec(runner=self._fake_keygen_runner)
|
||||
r = ex.provision_host_key({"id": "vm-c"})
|
||||
seed_user = os.path.join(r["seed_dir"], "user-data")
|
||||
self.assertTrue(os.path.exists(seed_user))
|
||||
with open(seed_user) as fh:
|
||||
seed_text = fh.read()
|
||||
self.assertIn("ssh_host_ed25519_key", seed_text)
|
||||
self.assertIn("ssh_deletekeys: false", seed_text)
|
||||
# stop 时 seed 目录与私钥删掉,known_hosts 公钥保留审计。
|
||||
ex.stop_vm({"id": "vm-c"})
|
||||
self.assertFalse(os.path.exists(r["seed_dir"]))
|
||||
self.assertFalse(os.path.exists(r["paths"]["private"]))
|
||||
self.assertTrue(os.path.exists(r["known_hosts"]))
|
||||
|
||||
def test_known_hosts_line_rejects_bad_key(self):
|
||||
with self.assertRaises(ValueError):
|
||||
known_hosts_line(2222, "not-a-key")
|
||||
self.assertEqual(
|
||||
known_hosts_line(2222, "ssh-ed25519 AAAAx comment"),
|
||||
"[127.0.0.1]:2222 ssh-ed25519 AAAAx")
|
||||
|
||||
def test_seed_user_data_shape(self):
|
||||
text = build_seed_user_data("PRIV\n", "ssh-ed25519 AAAAx\n")
|
||||
self.assertIn("/etc/ssh/ssh_host_ed25519_key", text)
|
||||
self.assertIn("ssh_deletekeys: false", text)
|
||||
with self.assertRaises(ValueError):
|
||||
build_seed_user_data("", "ssh-ed25519 AAAAx")
|
||||
|
||||
def test_keyscan_is_weak_fallback_only(self):
|
||||
# provisioned 模式下 wake 前必须预置;keyscan 不应被
|
||||
# 当成生产路径:配置值本身可解析,且文档/代码里标弱。
|
||||
ex = self.make_exec(env_extra={"CHV_HOST_KEY_MODE": "provisioned"})
|
||||
self.assertEqual(ex.host_key_mode, "provisioned")
|
||||
with self.assertRaises(ValueError):
|
||||
self.make_exec(env_extra={"CHV_HOST_KEY_MODE": "bogus"})
|
||||
|
||||
|
||||
# -- (4) egress tap 校验 -------------------------------------------------
|
||||
class EgressValidationTest(ChvGapCase):
|
||||
def test_direct_egress_without_flag_is_fail_closed(self):
|
||||
ex = self.make_exec()
|
||||
ex.egress_proxy = None
|
||||
ex.allow_direct_egress = False
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "egress_not_configured")
|
||||
|
||||
def test_direct_egress_with_flag_passes_and_is_audited(self):
|
||||
ex = self.make_exec(env_extra={"CHV_ALLOW_DIRECT_EGRESS": "1"})
|
||||
# preflight 本身不拦显式直连(守卫在 wake 的审计里)。
|
||||
self.assertTrue(ex.preflight())
|
||||
out = ex.goal_task({"id": "g1", "prompt": "做目标",
|
||||
"goal_mode": True})
|
||||
self.assertEqual(out["status"], "completed")
|
||||
self.assertTrue(any(e == "egress_direct_allowed"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
def test_bad_net_json_is_preflight_error(self):
|
||||
ex = self.make_exec(env_extra={"CHV_NET_JSON": "{not json"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "net_config_invalid")
|
||||
|
||||
|
||||
# -- (5) 构建脚本 --------------------------------------------------------
|
||||
class BuildScriptTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.bin_dir = os.path.join(self.dir, "bin")
|
||||
os.makedirs(os.path.join(self.bin_dir, "codex-resources"))
|
||||
for name in ("codex", "codex-code-mode-host"):
|
||||
p = os.path.join(self.bin_dir, name)
|
||||
with open(p, "w") as fh:
|
||||
fh.write("#!/bin/sh\n")
|
||||
os.chmod(p, 0o755)
|
||||
self.repo = os.path.join(self.dir, "repo")
|
||||
os.makedirs(os.path.join(self.repo, "cloud"))
|
||||
with open(os.path.join(self.repo, "host.py"), "w") as fh:
|
||||
fh.write("# host\n")
|
||||
self.script = str(Path(__file__).resolve().parents[2]
|
||||
/ "chv-guest" / "build.sh")
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _run(self, args, env_extra=None):
|
||||
env = dict(os.environ)
|
||||
env.update({"CODEX_BIN_DIR": self.bin_dir,
|
||||
"UBUNTU_VERSION": "24.04",
|
||||
"CODEX_VERSION": "0.164.0-alpha.9",
|
||||
"REPO_COMMIT": "test123",
|
||||
# feat-chv-secfix(M4):SHA256 必填,
|
||||
# 干跑只验“已设”,用 FAKE 占位。
|
||||
"UBUNTU_IMAGE_SHA256": "0" * 64,
|
||||
"REPO_DIR": self.repo})
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
return subprocess.run(["bash", self.script] + args,
|
||||
capture_output=True, text=True, env=env)
|
||||
|
||||
def test_dry_run_prints_real_steps_and_builds_nothing(self):
|
||||
out_dir = os.path.join(self.dir, "out")
|
||||
r = self._run(["--dry-run", out_dir])
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertIn("qemu-img convert", r.stdout)
|
||||
self.assertIn("virt-customize", r.stdout)
|
||||
self.assertIn("virt-get-kernel", r.stdout)
|
||||
self.assertIn("BUILDINFO", r.stdout)
|
||||
self.assertIn("chv-guest-24.04-0.164.0-alpha.9-test123", r.stdout)
|
||||
self.assertFalse(os.path.exists(os.path.join(out_dir, "disk.raw")))
|
||||
|
||||
def test_full_mode_without_kvm_fails_fast(self):
|
||||
# 本 VM 容器内无 /dev/kvm,全量模式必须明确失败,
|
||||
# 绝不声称已建成镜像。
|
||||
r = self._run([os.path.join(self.dir, "out2")])
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
self.assertIn("/dev/kvm", r.stderr + r.stdout)
|
||||
|
||||
def test_missing_binary_fails_fast(self):
|
||||
os.remove(os.path.join(self.bin_dir, "codex"))
|
||||
r = self._run(["--dry-run", os.path.join(self.dir, "out3")])
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
|
||||
|
||||
# -- (6) 中途死亡 → interrupted → 断点续跑闭环 ---------------------------
|
||||
class MidDeathLoopTest(ChvGapCase):
|
||||
def test_vm_death_is_interrupted_then_breakpoint_resumes(self):
|
||||
state = {"dead": False}
|
||||
|
||||
def runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh":
|
||||
if cmd[-1] == "true":
|
||||
return FakeResult(0, "", "")
|
||||
if "goal-resume" in cmd and state["dead"]:
|
||||
# 第二次断点调用:VM 已换新,健康返回。
|
||||
return FakeResult(0, "goal met after rounds\n", "")
|
||||
if "goal" in cmd:
|
||||
# 第一次 goal:模拟传输中断,并把 VMM 进程标死。
|
||||
state["dead"] = True
|
||||
# 找到登记的 VMM 进程并标死(wake 已登记)。
|
||||
for proc in list(ex._vms.values()):
|
||||
proc._alive = False
|
||||
raise RuntimeError("connection lost mid-task")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=runner)
|
||||
# 第一次 goal:中途死应判 interrupted,不是 failed 死信。
|
||||
out = ex.goal_task({"id": "g9", "prompt": "做目标",
|
||||
"goal_mode": True})
|
||||
self.assertEqual(out["status"], "interrupted")
|
||||
# 断点续跑:同一执行器的断点方法能走通并完成。
|
||||
state["dead"] = True # 标记已进入断点阶段(runner 按此分支)
|
||||
out2 = ex.goal_resume_breakpoint_task(
|
||||
{"id": "g9", "prompt": "续跑"})
|
||||
self.assertEqual(out2["status"], "completed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,606 @@
|
||||
"""Cloud Hypervisor 优化轨(feat-chv-optimize,堆叠于 #216 之上)测试。
|
||||
|
||||
只用标准库与 mock runner/launcher/api_client,绝不启动真实
|
||||
VM(与 test_cloud_hypervisor.py 同款边界)。覆盖:
|
||||
- 拆模块:cloud/chv/* 新路径与 cloud/sandbox 旧路径是同一对象
|
||||
- 去重复:Local 与 CHV 的 goal 解析在矩阵输入下结论全等;
|
||||
三执行器 host 顶层参数顺序全等;runner 超时 helper
|
||||
- 安全:每 VM known_hosts 钉死、ssh-keyscan 首次捕获(TOFU)、
|
||||
非 root 用户与端口显式配置与收紧校验
|
||||
- execute 语义:只读已运行 VM,未运行报 vm_not_running
|
||||
- 预热池:registry 持久化、重启载入只当孤儿、reconcile 清理、
|
||||
损坏 quarantine、一 次性不复用
|
||||
- 长任务:VM 中途死亡判 interrupted 而非 failed/抛异常
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from cloud.chv import ChvFailure
|
||||
from cloud.chv import CloudHypervisorExecutor
|
||||
from cloud.chv import reconcile_orphans_for_executor
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.chv.api_client import _unix_socket_api as _chv_api
|
||||
from cloud.chv.config import _parse_memory_bytes as _chv_mem
|
||||
from cloud.sandbox import ChvFailure as _sb_failure
|
||||
from cloud.sandbox import CloudHypervisorExecutor as _sb_exec
|
||||
from cloud.sandbox import DockerExecutor
|
||||
from cloud.sandbox import reconcile_orphans_for_executor as _sb_reconcile
|
||||
from cloud.sandbox import LocalExecutor
|
||||
from cloud.sandbox import _call_runner_with_timeout
|
||||
from cloud.sandbox import _parse_memory_bytes as _sb_mem
|
||||
from cloud.sandbox import _unix_socket_api as _sb_api
|
||||
except ImportError: # discover -s cloud/tests 时把项目根目录补进 sys.path
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv import ChvFailure
|
||||
from cloud.chv import CloudHypervisorExecutor
|
||||
from cloud.chv import reconcile_orphans_for_executor
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.chv.api_client import _unix_socket_api as _chv_api
|
||||
from cloud.chv.config import _parse_memory_bytes as _chv_mem
|
||||
from cloud.sandbox import ChvFailure as _sb_failure
|
||||
from cloud.sandbox import CloudHypervisorExecutor as _sb_exec
|
||||
from cloud.sandbox import DockerExecutor
|
||||
from cloud.sandbox import reconcile_orphans_for_executor as _sb_reconcile
|
||||
from cloud.sandbox import LocalExecutor
|
||||
from cloud.sandbox import _call_runner_with_timeout
|
||||
from cloud.sandbox import _parse_memory_bytes as _sb_mem
|
||||
from cloud.sandbox import _unix_socket_api as _sb_api
|
||||
|
||||
|
||||
class FakeResult:
|
||||
def __init__(self, returncode=0, stdout="", stderr=""):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
class FakeProc:
|
||||
def __init__(self, alive=True):
|
||||
self._alive = alive
|
||||
self.terminated = False
|
||||
|
||||
def poll(self):
|
||||
return None if self._alive else 1
|
||||
|
||||
def terminate(self):
|
||||
self.terminated = True
|
||||
self._alive = False
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self._alive = False
|
||||
|
||||
|
||||
def _write(path, content=b"x"):
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(content)
|
||||
os.chmod(path, 0o755)
|
||||
return path
|
||||
|
||||
|
||||
class ChvOptCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.binary = _write(os.path.join(self.dir, "cloud-hypervisor"))
|
||||
self.ch_remote = _write(os.path.join(self.dir, "ch-remote"))
|
||||
self.kernel = _write(os.path.join(self.dir, "vmlinux"))
|
||||
self.rootfs = _write(os.path.join(self.dir, "disk.raw"),
|
||||
b"R" * 100)
|
||||
self.ssh_key = _write(os.path.join(self.dir, "id_chv"))
|
||||
self.kvm = _write(os.path.join(self.dir, "kvm"))
|
||||
self.cpuinfo = os.path.join(self.dir, "cpuinfo")
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu vmx sse\n")
|
||||
self.state_dir = os.path.join(self.dir, "state")
|
||||
self.calls = []
|
||||
self.api_calls = []
|
||||
self.launches = []
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def make_exec(self, runner=None, api_client=None, launcher=None,
|
||||
env_extra=None, **overrides):
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
# 实验室宣称:mock runner 假装回环端口已转发到客机
|
||||
#(feat-chv-fix 起,执行器不再无条件假设回环能通)。
|
||||
# 实验室显式 opt-in keyscan:生产默认已是
|
||||
# provisioned;本套件旧 fake 不实现 ssh-keygen/
|
||||
# genisoimage,测的不是 host key 模式,故显式
|
||||
# 声明实验室路径,绝不把它写成生产默认。
|
||||
"CHV_HOST_KEY_MODE": "keyscan",
|
||||
"CHV_SSH_FORWARD": "1"}
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
if runner is None:
|
||||
runner = self._default_runner
|
||||
if api_client is None:
|
||||
api_client = self._default_api
|
||||
if launcher is None:
|
||||
launcher = self._default_launcher
|
||||
kwargs = dict(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, rootfs=self.rootfs,
|
||||
ssh_key=self.ssh_key, kvm_path=self.kvm,
|
||||
cpuinfo_path=self.cpuinfo, state_dir=self.state_dir,
|
||||
runner=runner, launcher=launcher, api_client=api_client,
|
||||
sleeper=lambda s: None, env=env)
|
||||
kwargs.update(overrides)
|
||||
return CloudHypervisorExecutor(**kwargs)
|
||||
|
||||
def _default_runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh":
|
||||
if cmd[-1] == "true":
|
||||
return FakeResult(0, "", "")
|
||||
if "dispatch" in cmd:
|
||||
return FakeResult(0, json.dumps(
|
||||
{"status": "completed", "text": "ok-dispatch"}), "")
|
||||
if "goal" in cmd:
|
||||
return FakeResult(0, "goal met after rounds\n", "")
|
||||
if "status" in cmd:
|
||||
return FakeResult(0, json.dumps(
|
||||
{"status": "completed", "text": "ok-status"}), "")
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
def _default_api(self, socket_path, method, path, payload=None):
|
||||
self.api_calls.append((method, path, payload))
|
||||
return (200, {})
|
||||
|
||||
def _default_launcher(self, cmd):
|
||||
self.launches.append(list(cmd))
|
||||
return FakeProc(alive=True)
|
||||
|
||||
def _task(self, task_id="abc123"):
|
||||
return {"id": task_id, "prompt": "做一个加法函数",
|
||||
"project_id": "proj1"}
|
||||
|
||||
|
||||
# -- 拆模块 ---------------------------------------------------------------
|
||||
class SplitTest(unittest.TestCase):
|
||||
def test_new_and_old_import_paths_are_same_objects(self):
|
||||
self.assertIs(CloudHypervisorExecutor, _sb_exec)
|
||||
self.assertIs(ChvFailure, _sb_failure)
|
||||
self.assertIs(_chv_mem, _sb_mem)
|
||||
self.assertIs(_chv_api, _sb_api)
|
||||
self.assertIs(reconcile_orphans_for_executor, _sb_reconcile)
|
||||
|
||||
def test_classes_live_in_chv_modules(self):
|
||||
self.assertEqual(CloudHypervisorExecutor.__module__,
|
||||
"cloud.chv.executor")
|
||||
self.assertEqual(WarmPool.__module__,
|
||||
"cloud.chv.experimental_pool")
|
||||
self.assertEqual(ChvFailure.__module__, "cloud.chv.errors")
|
||||
|
||||
def test_old_warm_pool_import_paths_are_gone(self):
|
||||
# 本轨破环性清理:旧路径一律 ImportError,实验路径可用。
|
||||
with self.assertRaises(ImportError):
|
||||
from cloud.chv import WarmPool # noqa: F401
|
||||
with self.assertRaises(ImportError):
|
||||
from cloud.sandbox import WarmPool # noqa: F401
|
||||
with self.assertRaises(ImportError):
|
||||
import cloud.chv.pool # noqa: F401
|
||||
|
||||
|
||||
# -- 去重复:goal 解析与 host 命令 ------------------------------------------
|
||||
class SharedGoalTest(unittest.TestCase):
|
||||
CASES = [
|
||||
# (returncode, stdout, stderr)
|
||||
(0, "goal met after rounds\n", ""),
|
||||
(0, "round 1 verify_out=bad\ngoal unmet after rounds\n", ""),
|
||||
(1, "goal interrupted after rounds: boom\n", ""),
|
||||
(1, "KeyError: 'x'\n", ""),
|
||||
(1, "some odd failure\n", ""),
|
||||
(0, "nothing conclusive\n", ""),
|
||||
(1, "goal-resume 拒绝:断点丢失\n", ""),
|
||||
(1, "任务已存在\n", ""),
|
||||
(1, "goal met after rounds\n", ""),
|
||||
(1, "goal unmet after rounds\n", ""),
|
||||
(1, "", "HTTP 503 Service Unavailable\n"),
|
||||
]
|
||||
|
||||
def test_local_and_chv_goal_conclusions_match(self):
|
||||
local = LocalExecutor("/tmp/host.py")
|
||||
chv = ChvOptCase()
|
||||
chv.setUp()
|
||||
try:
|
||||
ex = chv.make_exec()
|
||||
for rc, out, err in self.CASES:
|
||||
res = FakeResult(rc, out, err)
|
||||
a = local._goal_result_from_runner("t1", res)
|
||||
b = ex._goal_result("t1", res)
|
||||
self.assertEqual(
|
||||
(a["status"], a["goal_met"]),
|
||||
(b["status"], b["goal_met"]),
|
||||
"rc=%s stdout=%r" % (rc, out))
|
||||
self.assertEqual(a["executor"], "local")
|
||||
self.assertEqual(b["executor"], "cloud-hypervisor")
|
||||
finally:
|
||||
chv.tearDown()
|
||||
|
||||
def test_already_exists_message_points_to_continue(self):
|
||||
chv = ChvOptCase()
|
||||
chv.setUp()
|
||||
try:
|
||||
ex = chv.make_exec()
|
||||
out = ex._goal_result(
|
||||
"t1", FakeResult(1, "任务已存在\n", ""))
|
||||
self.assertEqual(out["status"], "failed")
|
||||
self.assertIn("继续", out["goal_error"])
|
||||
finally:
|
||||
chv.tearDown()
|
||||
|
||||
def test_host_command_options_identical_across_executors(self):
|
||||
chv = ChvOptCase()
|
||||
chv.setUp()
|
||||
try:
|
||||
ex = chv.make_exec()
|
||||
local = LocalExecutor("/tmp/host.py")
|
||||
docker = DockerExecutor()
|
||||
for kwargs in (
|
||||
{"model": "m1", "effort": "low", "approval": "ask"},
|
||||
{"model": "m1", "effort": "low", "approval": "full",
|
||||
"sandbox": "read-only",
|
||||
"model_provider": "sub2api"},
|
||||
):
|
||||
lcmd = local._base_command(**kwargs)
|
||||
dcmd = docker._inner_host_command("status", **kwargs)
|
||||
ccmd = ex._guest_host_command("status", **kwargs)
|
||||
self.assertEqual(lcmd[2:], dcmd[2:-1])
|
||||
self.assertEqual(lcmd[2:], ccmd[2:-1])
|
||||
finally:
|
||||
chv.tearDown()
|
||||
|
||||
def test_runner_timeout_helper(self):
|
||||
seen = []
|
||||
|
||||
def only_cmd(cmd):
|
||||
seen.append(("cmd-only", list(cmd)))
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
out = _call_runner_with_timeout(only_cmd, ["x"], timeout=5)
|
||||
self.assertEqual(out.returncode, 0)
|
||||
self.assertEqual(seen, [("cmd-only", ["x"])])
|
||||
|
||||
def with_timeout(cmd, timeout=None):
|
||||
seen.append(("with-timeout", timeout))
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
_call_runner_with_timeout(with_timeout, ["x"], timeout=7)
|
||||
self.assertEqual(seen[-1], ("with-timeout", 7))
|
||||
_call_runner_with_timeout(with_timeout, ["x"], timeout=None)
|
||||
self.assertEqual(seen[-1], ("with-timeout", None))
|
||||
|
||||
|
||||
# -- 安全:known_hosts 钉死 -------------------------------------------------
|
||||
class SshPinningTest(ChvOptCase):
|
||||
def test_known_hosts_path_is_per_vm(self):
|
||||
ex = self.make_exec()
|
||||
a = ex.known_hosts_path({"id": "t-a"})
|
||||
b = ex.known_hosts_path({"id": "t-b"})
|
||||
self.assertNotEqual(a, b)
|
||||
self.assertEqual(a, os.path.join(self.state_dir, "chv-t-a",
|
||||
"known_hosts"))
|
||||
|
||||
def test_capture_host_key_writes_pinned_file_0600(self):
|
||||
def keyscan(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
return FakeResult(
|
||||
0, "[127.0.0.1]:2222 ssh-ed25519 AAAAB3test\n"
|
||||
"evil.example.com ssh-rsa AAAAEvil\n"
|
||||
"# comment\n", "")
|
||||
ex = self.make_exec(runner=keyscan)
|
||||
self.assertTrue(ex.capture_host_key(self._task()))
|
||||
path = ex.known_hosts_path(self._task())
|
||||
with open(path) as fh:
|
||||
content = fh.read()
|
||||
self.assertIn("[127.0.0.1]:2222 ssh-ed25519 AAAAB3test",
|
||||
content)
|
||||
self.assertNotIn("evil.example.com", content)
|
||||
self.assertEqual(os.stat(path).st_mode & 0o777, 0o600)
|
||||
|
||||
def test_capture_host_key_empty_is_false_not_error(self):
|
||||
ex = self.make_exec(
|
||||
runner=lambda cmd, timeout=None: FakeResult(0, "", ""))
|
||||
self.assertFalse(ex.capture_host_key(self._task()))
|
||||
self.assertFalse(os.path.exists(
|
||||
ex.known_hosts_path(self._task())))
|
||||
|
||||
def test_wait_guest_ready_captures_before_ssh(self):
|
||||
def runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh-keyscan":
|
||||
return FakeResult(
|
||||
0, "[127.0.0.1]:2222 ssh-ed25519 AAAAB3test\n",
|
||||
"")
|
||||
if cmd[0] == "ssh" and cmd[-1] == "true":
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
ex = self.make_exec(runner=runner)
|
||||
self.assertTrue(ex.wait_guest_ready(self._task()))
|
||||
self.assertTrue(os.path.exists(
|
||||
ex.known_hosts_path(self._task())))
|
||||
keyscan_idx = next(i for i, c in enumerate(self.calls)
|
||||
if c[0] == "ssh-keyscan")
|
||||
ssh_idx = next(i for i, c in enumerate(self.calls)
|
||||
if c[0] == "ssh" and c[-1] == "true")
|
||||
self.assertLess(keyscan_idx, ssh_idx)
|
||||
|
||||
def test_ssh_user_port_explicit_config(self):
|
||||
ex = self.make_exec(ssh_user="codex", ssh_port=2201)
|
||||
cmd = ex.build_ssh_command(self._task(), ["true"])
|
||||
joined = " ".join(cmd)
|
||||
self.assertIn("codex@127.0.0.1", joined)
|
||||
self.assertNotIn("root@127.0.0.1", joined)
|
||||
self.assertIn("2201", joined)
|
||||
|
||||
def test_ssh_user_port_tightened_validation(self):
|
||||
with self.assertRaises(ValueError):
|
||||
self.make_exec(ssh_user="")
|
||||
with self.assertRaises(ValueError):
|
||||
self.make_exec(ssh_port=0)
|
||||
with self.assertRaises(ValueError):
|
||||
self.make_exec(ssh_port=70000)
|
||||
with self.assertRaises(ValueError):
|
||||
self.make_exec(ssh_port="abc")
|
||||
with self.assertRaises(ValueError):
|
||||
self.make_exec(env_extra={"CHV_SSH_PORT": "0"})
|
||||
|
||||
|
||||
# -- execute 语义 -------------------------------------------------------------
|
||||
class ExecuteGateTest(ChvOptCase):
|
||||
def test_execute_without_running_vm_is_classified(self):
|
||||
ex = self.make_exec()
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.execute(self._task())
|
||||
self.assertEqual(ctx.exception.kind, "vm_not_running")
|
||||
self.assertEqual(self.launches, [])
|
||||
self.assertFalse(any(c[0] == "ssh" for c in self.calls))
|
||||
|
||||
def test_execute_with_dead_proc_and_no_socket_is_classified(self):
|
||||
ex = self.make_exec()
|
||||
ex._vms["chv-abc123"] = FakeProc(alive=False)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.execute(self._task())
|
||||
self.assertEqual(ctx.exception.kind, "vm_not_running")
|
||||
|
||||
def test_execute_with_socket_but_no_registry_proceeds(self):
|
||||
ex = self.make_exec()
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(os.path.join(self.state_dir,
|
||||
"chv-abc123.sock"), "w") as fh:
|
||||
fh.write("")
|
||||
out = ex.execute(self._task())
|
||||
self.assertIn("ok-status", out["stdout"])
|
||||
self.assertEqual(self.launches, [])
|
||||
|
||||
|
||||
# -- 预热池持久化与孤儿清理 ----------------------------------------------------
|
||||
class StubExecutor:
|
||||
"""WarmPool 测试替身:只实现池子调用的鸭子接口。"""
|
||||
|
||||
def __init__(self, state_dir):
|
||||
self.state_dir = state_dir
|
||||
self.warm_pool_size = 2
|
||||
self.snapshot_url = "file:///snap/base"
|
||||
self.wake_calls = []
|
||||
self.pause_calls = []
|
||||
self.shutdown_calls = []
|
||||
self.audit = []
|
||||
|
||||
def vm_name(self, task):
|
||||
raw = task.get("id") if isinstance(task, dict) else str(task)
|
||||
return raw if raw.startswith("chv-") else "chv-" + raw
|
||||
|
||||
def wake_vm(self, task):
|
||||
self.wake_calls.append(task["id"])
|
||||
return {"vm": self.vm_name(task), "state": "running"}
|
||||
|
||||
def pause_vm(self, task):
|
||||
self.pause_calls.append(task["id"])
|
||||
return True
|
||||
|
||||
def shutdown_vm(self, task_or_name):
|
||||
name = task_or_name if isinstance(task_or_name, str) \
|
||||
else self.vm_name(task_or_name)
|
||||
self.shutdown_calls.append(name)
|
||||
return True
|
||||
|
||||
def _emit(self, event, task, detail="", kind=None):
|
||||
self.audit.append(event)
|
||||
|
||||
|
||||
class PoolRegistryTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.state_dir = os.path.join(self.tmp.name, "state")
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
self.registry = os.path.join(self.state_dir, "warm-pool.json")
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_persist_load_orphan_reconcile(self):
|
||||
stub = StubExecutor(self.state_dir)
|
||||
pool = WarmPool(stub, size=2, snapshot_url="file:///snap/base")
|
||||
added = pool.prewarm()
|
||||
self.assertEqual(len(added), 2)
|
||||
self.assertTrue(os.path.exists(self.registry))
|
||||
with open(self.registry) as fh:
|
||||
data = json.load(fh)
|
||||
self.assertEqual(len(data), 2)
|
||||
# 新池实例载入:上一世条目只当孤儿,绝不发给任务:
|
||||
stub2 = StubExecutor(self.state_dir)
|
||||
pool2 = WarmPool(stub2, size=2,
|
||||
snapshot_url="file:///snap/base")
|
||||
self.assertEqual(pool2.warm_count(), 0)
|
||||
self.assertEqual(pool2.orphan_count(), 2)
|
||||
self.assertEqual(pool2.acquire({"id": "t1"})["mode"], "cold")
|
||||
cleaned = pool2.reconcile_orphans()
|
||||
self.assertEqual(len(cleaned), 2)
|
||||
self.assertEqual(sorted(stub2.shutdown_calls), sorted(cleaned))
|
||||
self.assertEqual(pool2.orphan_count(), 0)
|
||||
with open(self.registry) as fh:
|
||||
self.assertEqual(json.load(fh), {})
|
||||
# 第三实例:无残留:
|
||||
pool3 = WarmPool(StubExecutor(self.state_dir), size=2,
|
||||
snapshot_url="file:///snap/base")
|
||||
self.assertEqual(pool3.orphan_count(), 0)
|
||||
|
||||
def test_corrupt_registry_quarantines_and_starts_empty(self):
|
||||
with open(self.registry, "w") as fh:
|
||||
fh.write("{not json")
|
||||
pool = WarmPool(StubExecutor(self.state_dir), size=2,
|
||||
snapshot_url="file:///snap/base")
|
||||
self.assertTrue(pool.registry_corrupt)
|
||||
self.assertEqual(pool.warm_count(), 0)
|
||||
self.assertEqual(pool.orphan_count(), 0)
|
||||
self.assertTrue(os.path.exists(self.registry + ".corrupt"))
|
||||
self.assertFalse(os.path.exists(self.registry))
|
||||
# 损坏后仍可正常预热(registry 重建):
|
||||
self.assertEqual(len(pool.prewarm()), 2)
|
||||
self.assertTrue(os.path.exists(self.registry))
|
||||
|
||||
def test_assigned_entry_on_load_is_never_served(self):
|
||||
with open(self.registry, "w") as fh:
|
||||
json.dump({"warm-1": {"state": "assigned",
|
||||
"vm": "chv-warm-1",
|
||||
"task_id": "old-task",
|
||||
"snapshot_url": "file:///snap/base"}},
|
||||
fh)
|
||||
pool = WarmPool(StubExecutor(self.state_dir), size=2,
|
||||
snapshot_url="file:///snap/base")
|
||||
self.assertEqual(pool.warm_count(), 0)
|
||||
self.assertEqual(pool.acquire({"id": "t1"})["mode"], "cold")
|
||||
self.assertEqual(pool.reconcile_orphans(), ["chv-warm-1"])
|
||||
|
||||
def test_release_never_rewarms_and_registry_clears(self):
|
||||
stub = StubExecutor(self.state_dir)
|
||||
pool = WarmPool(stub, size=1, snapshot_url="file:///snap/base")
|
||||
added = pool.prewarm()
|
||||
got = pool.acquire({"id": "t1"})
|
||||
self.assertEqual(got["mode"], "warm")
|
||||
self.assertTrue(pool.release({"id": "t1"}, pool_vm=added[0]))
|
||||
self.assertEqual(pool.warm_count(), 0)
|
||||
with open(self.registry) as fh:
|
||||
self.assertEqual(json.load(fh), {})
|
||||
|
||||
|
||||
# -- 长任务中途死亡 --------------------------------------------------------------
|
||||
class MidDeathTest(ChvOptCase):
|
||||
def _dispatch_with(self, runner):
|
||||
ex = self.make_exec(runner=runner)
|
||||
return ex.dispatch_task(self._task())
|
||||
|
||||
def test_failed_result_with_dead_vm_is_interrupted(self):
|
||||
proc_holder = []
|
||||
|
||||
def launcher(cmd):
|
||||
proc = FakeProc(alive=True)
|
||||
proc_holder.append(proc)
|
||||
self.launches.append(list(cmd))
|
||||
return proc
|
||||
|
||||
def runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh" and "dispatch" in cmd:
|
||||
proc_holder[0]._alive = False
|
||||
return FakeResult(1, "",
|
||||
"Connection closed by remote host")
|
||||
if cmd[0] == "ssh":
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=runner, launcher=launcher)
|
||||
out = ex.dispatch_task(self._task())
|
||||
self.assertEqual(out["status"], "interrupted")
|
||||
self.assertTrue(out["vm_dead"])
|
||||
|
||||
def test_failed_result_with_live_vm_stays_failed(self):
|
||||
def runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh" and "dispatch" in cmd:
|
||||
return FakeResult(1, "", "host dispatch failed")
|
||||
if cmd[0] == "ssh":
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=runner)
|
||||
out = ex.dispatch_task(self._task())
|
||||
self.assertEqual(out["status"], "failed")
|
||||
self.assertNotIn("vm_dead", out)
|
||||
|
||||
def test_exception_with_dead_vm_is_interrupted_not_raised(self):
|
||||
proc_holder = []
|
||||
|
||||
def launcher(cmd):
|
||||
proc = FakeProc(alive=True)
|
||||
proc_holder.append(proc)
|
||||
self.launches.append(list(cmd))
|
||||
return proc
|
||||
|
||||
def runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh" and "dispatch" in cmd:
|
||||
proc_holder[0]._alive = False
|
||||
raise RuntimeError("ssh transport broken")
|
||||
if cmd[0] == "ssh":
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=runner, launcher=launcher)
|
||||
out = ex.dispatch_task(self._task())
|
||||
self.assertEqual(out["status"], "interrupted")
|
||||
self.assertTrue(out["vm_dead"])
|
||||
|
||||
def test_exception_with_live_vm_propagates(self):
|
||||
def runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh" and "dispatch" in cmd:
|
||||
raise RuntimeError("ssh transport broken")
|
||||
if cmd[0] == "ssh":
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=runner)
|
||||
with self.assertRaises(RuntimeError):
|
||||
ex.dispatch_task(self._task())
|
||||
|
||||
def test_goal_exception_with_dead_vm_is_interrupted(self):
|
||||
proc_holder = []
|
||||
|
||||
def launcher(cmd):
|
||||
proc = FakeProc(alive=True)
|
||||
proc_holder.append(proc)
|
||||
self.launches.append(list(cmd))
|
||||
return proc
|
||||
|
||||
def runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh" and "goal" in cmd:
|
||||
proc_holder[0]._alive = False
|
||||
raise RuntimeError("ssh transport broken")
|
||||
if cmd[0] == "ssh":
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=runner, launcher=launcher)
|
||||
out = ex.goal_task({"id": "g1", "prompt": "做目标",
|
||||
"goal_mode": True})
|
||||
self.assertEqual(out["status"], "interrupted")
|
||||
self.assertIsNone(out["goal_met"])
|
||||
self.assertTrue(out["vm_dead"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,730 @@
|
||||
"""CHV 生产级审核 P1 修复轨(feat-chv-p1fix,堆叠于 #270 之上)测试。
|
||||
|
||||
逐项对应审核报告 §5 的 P1-01~P1-10(P1-10 是文档项,
|
||||
无代码测试;文档数字见 docs/chv-production-readiness.md
|
||||
§15 与 docs/chv-architecture.md 头范围):
|
||||
- P1-01:生产 reconcile 失败项不进 cleaned、不清库、
|
||||
落盘保留失败项原始条目、可重试;实验池同款语义。
|
||||
- P1-02:registry 损坏与非法条目发审计并计数,损坏不
|
||||
静默返回空。
|
||||
- P1-03:残留私钥删失败发审计、写脏标记,调用方可区分
|
||||
成功与失败(failed VM 不进 cleaned)。
|
||||
- P1-04:build_ssh_command 删除回环回落,构造失败即抛;
|
||||
显式回环(CHV_SSH_HOST / CHV_SSH_FORWARD=1)仍可用。
|
||||
- P1-05:state_dir flock(locking.StateLock)——线程内
|
||||
互斥、跨进程互斥、wake 全程串行化、purge 快照遍历。
|
||||
- P1-06:单条快照超总量帽时拒绝记录,且不删旧快照、
|
||||
不落盘新条目。
|
||||
- P1-07:CHV_CPU / CHV_MEM / CHV_TASK_TIMEOUT 的 >0
|
||||
校验(config 构造层 + preflight 事后改写兜底)。
|
||||
- P1-08:快照保留期剪枝删失败发审计;snapshot_vm 失败
|
||||
路径发 vm_snapshot_failed 审计。
|
||||
- P1-09:4 处三元 else 分支——str 形态 pause/resume/
|
||||
snapshot + 非 dict spill task。
|
||||
|
||||
全部用标准库与 mock runner/launcher/api_client,绝不启动
|
||||
真实 VM(本机无 /dev/kvm,真机验证仍是未验证项)。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
try:
|
||||
from cloud.chv import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
reconcile_orphans_for_executor,
|
||||
)
|
||||
from cloud.chv.config import build_chv_config
|
||||
from cloud.chv.disk import (
|
||||
prune_snapshots,
|
||||
record_snapshot,
|
||||
)
|
||||
from cloud.chv.locking import StateLock, lock_path_for_state_dir
|
||||
except ImportError: # discover -s cloud/tests 时把项目根目录补进 sys.path
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
reconcile_orphans_for_executor,
|
||||
)
|
||||
from cloud.chv.config import build_chv_config
|
||||
from cloud.chv.disk import (
|
||||
prune_snapshots,
|
||||
record_snapshot,
|
||||
)
|
||||
from cloud.chv.locking import StateLock, lock_path_for_state_dir
|
||||
|
||||
|
||||
class FakeResult:
|
||||
def __init__(self, returncode=0, stdout="", stderr=""):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
class FakeProc:
|
||||
def __init__(self, alive=True):
|
||||
self._alive = alive
|
||||
|
||||
def poll(self):
|
||||
return None if self._alive else 1
|
||||
|
||||
def terminate(self):
|
||||
self._alive = False
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self._alive = False
|
||||
|
||||
|
||||
def _write(path, content=b"x"):
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(content)
|
||||
os.chmod(path, 0o755)
|
||||
return path
|
||||
|
||||
|
||||
class P1FixCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.binary = _write(os.path.join(self.dir, "cloud-hypervisor"))
|
||||
self.ch_remote = _write(os.path.join(self.dir, "ch-remote"))
|
||||
self.kernel = _write(os.path.join(self.dir, "vmlinux"))
|
||||
self.rootfs = _write(os.path.join(self.dir, "disk.raw"),
|
||||
b"R" * 100)
|
||||
self.ssh_key = _write(os.path.join(self.dir, "id_chv"))
|
||||
self.kvm = _write(os.path.join(self.dir, "kvm"))
|
||||
self.cpuinfo = os.path.join(self.dir, "cpuinfo")
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu vmx sse\n")
|
||||
self.state_dir = os.path.join(self.dir, "state")
|
||||
self.calls = []
|
||||
self.audit = []
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def make_exec(self, runner=None, api_client=None, launcher=None,
|
||||
env_extra=None, **overrides):
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
"CHV_SSH_FORWARD": "1",
|
||||
"CHV_HOST_KEY_MODE": "keyscan"}
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
if runner is None:
|
||||
runner = self._default_runner
|
||||
if api_client is None:
|
||||
api_client = self._default_api
|
||||
if launcher is None:
|
||||
launcher = self._default_launcher
|
||||
kwargs = dict(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, rootfs=self.rootfs,
|
||||
ssh_key=self.ssh_key, kvm_path=self.kvm,
|
||||
cpuinfo_path=self.cpuinfo, state_dir=self.state_dir,
|
||||
runner=runner, launcher=launcher, api_client=api_client,
|
||||
sleeper=lambda s: None, env=env)
|
||||
kwargs.update(overrides)
|
||||
ex = CloudHypervisorExecutor(**kwargs)
|
||||
ex.audit_callback = lambda *a: self.audit.append(a)
|
||||
return ex
|
||||
|
||||
def _default_runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
def _default_api(self, socket_path, method, path, payload=None):
|
||||
return (200, {})
|
||||
|
||||
def _default_launcher(self, cmd):
|
||||
return FakeProc(alive=True)
|
||||
|
||||
def _task(self, task_id="abc123"):
|
||||
return {"id": task_id, "prompt": "x", "project_id": "proj1"}
|
||||
|
||||
def _audit_events(self):
|
||||
return [a[0] for a in self.audit]
|
||||
|
||||
|
||||
# -- P1-01 / P1-02:生产 reconcile --------------------------------------
|
||||
|
||||
class FakeProdExecutor:
|
||||
def __init__(self, state_dir, fail_vms=()):
|
||||
self.state_dir = state_dir
|
||||
self.fail_vms = set(fail_vms)
|
||||
self.shutdowns = []
|
||||
self.emits = []
|
||||
|
||||
def shutdown_vm(self, vm):
|
||||
self.shutdowns.append(vm)
|
||||
if vm in self.fail_vms:
|
||||
raise RuntimeError("shutdown down")
|
||||
|
||||
def _emit(self, event, task, detail):
|
||||
self.emits.append(event)
|
||||
|
||||
|
||||
def _write_registry(state_dir, data):
|
||||
path = os.path.join(state_dir, "warm-pool.json")
|
||||
os.makedirs(state_dir, exist_ok=True)
|
||||
if isinstance(data, str):
|
||||
with open(path, "w") as fh:
|
||||
fh.write(data)
|
||||
else:
|
||||
with open(path, "w") as fh:
|
||||
json.dump(data, fh)
|
||||
return path
|
||||
|
||||
|
||||
class ReconcileP1FixTest(unittest.TestCase):
|
||||
def test_failure_not_cleaned_registry_retained_and_retryable(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, {
|
||||
"a": {"state": "warm", "vm": "chv-good"},
|
||||
"b": {"state": "warm", "vm": "chv-bad"},
|
||||
})
|
||||
ex = FakeProdExecutor(d, fail_vms={"chv-bad"})
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex),
|
||||
["chv-good"])
|
||||
self.assertIn("warm_pool_reconcile_failed", ex.emits)
|
||||
with open(path) as fh:
|
||||
remaining = json.load(fh)
|
||||
self.assertEqual(
|
||||
remaining,
|
||||
{"b": {"state": "warm", "vm": "chv-bad"}})
|
||||
# 修复后重试:失败项可被清理,库最终为空。
|
||||
ex.fail_vms = set()
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex),
|
||||
["chv-bad"])
|
||||
with open(path) as fh:
|
||||
self.assertEqual(json.load(fh), {})
|
||||
|
||||
def test_corrupt_registry_audited_not_silent(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, "{not json")
|
||||
ex = FakeProdExecutor(d)
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex), [])
|
||||
self.assertIn("warm_pool_registry_corrupt", ex.emits)
|
||||
self.assertTrue(os.path.exists(path + ".corrupt"))
|
||||
|
||||
def test_invalid_entries_audited_and_counted(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = _write_registry(d, {
|
||||
"a": {"state": "warm", "vm": "chv-a"},
|
||||
"c": {"state": "warm"},
|
||||
"e": {"state": "running", "vm": "chv-e"},
|
||||
})
|
||||
ex = FakeProdExecutor(d)
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex),
|
||||
["chv-a"])
|
||||
self.assertIn("warm_pool_registry_invalid_entries",
|
||||
ex.emits)
|
||||
with open(path) as fh:
|
||||
self.assertEqual(json.load(fh), {})
|
||||
|
||||
|
||||
# -- P1-03:残留私钥删失败 ----------------------------------------------
|
||||
|
||||
class StaleSecretsP1FixTest(P1FixCase):
|
||||
def test_delete_failure_not_cleaned_audited_and_dirty_marked(self):
|
||||
from cloud.chv.hostkeys import dirty_marker_path
|
||||
ex = self.make_exec()
|
||||
vm_dir = os.path.join(self.state_dir, "chv-bad")
|
||||
os.makedirs(vm_dir, exist_ok=True)
|
||||
priv = os.path.join(vm_dir, "ssh_host_ed25519_key")
|
||||
with open(priv, "w") as fh:
|
||||
fh.write("PRIVATE")
|
||||
real_remove = os.remove
|
||||
|
||||
def remove(path):
|
||||
if path == priv:
|
||||
raise OSError("删不掉")
|
||||
return real_remove(path)
|
||||
|
||||
with mock.patch.object(os, "remove", side_effect=remove):
|
||||
cleaned = ex.cleanup_stale_secrets()
|
||||
self.assertNotIn("chv-bad", cleaned)
|
||||
self.assertIn("host_secret_residue_delete_failed",
|
||||
self._audit_events())
|
||||
self.assertTrue(os.path.exists(
|
||||
dirty_marker_path(self.state_dir, "chv-bad")))
|
||||
self.assertTrue(os.path.exists(priv))
|
||||
|
||||
def test_delete_success_cleaned(self):
|
||||
ex = self.make_exec()
|
||||
vm_dir = os.path.join(self.state_dir, "chv-ok")
|
||||
os.makedirs(vm_dir, exist_ok=True)
|
||||
with open(os.path.join(vm_dir, "ssh_host_ed25519_key"),
|
||||
"w") as fh:
|
||||
fh.write("PRIVATE")
|
||||
cleaned = ex.cleanup_stale_secrets()
|
||||
self.assertIn("chv-ok", cleaned)
|
||||
self.assertIn("host_secret_residue_cleaned",
|
||||
self._audit_events())
|
||||
|
||||
|
||||
# -- P1-04:SSH 回落删除 -------------------------------------------------
|
||||
|
||||
class SshFallbackP1FixTest(P1FixCase):
|
||||
def test_undeclared_build_raises_not_loopback(self):
|
||||
ex = self.make_exec(env_extra={"CHV_SSH_FORWARD": "0"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.build_ssh_command(self._task(), ["true"])
|
||||
self.assertEqual(ctx.exception.kind, "ssh_target_undeclared")
|
||||
|
||||
def test_explicit_loopback_still_allowed(self):
|
||||
ex = self.make_exec(env_extra={"CHV_SSH_HOST": "127.0.0.1"})
|
||||
cmd = ex.build_ssh_command(self._task(), ["true"])
|
||||
self.assertIn("root@127.0.0.1", " ".join(cmd))
|
||||
ex2 = self.make_exec(env_extra={"CHV_SSH_FORWARD": "1"})
|
||||
cmd2 = ex2.build_ssh_command(self._task(), ["true"])
|
||||
self.assertIn("root@127.0.0.1", " ".join(cmd2))
|
||||
|
||||
|
||||
# -- P1-05:flock ---------------------------------------------------------
|
||||
|
||||
class StateLockP1FixTest(unittest.TestCase):
|
||||
def test_thread_mutual_exclusion(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
state_dir = os.path.join(d, "state")
|
||||
lock_a = StateLock(state_dir)
|
||||
lock_b = StateLock(state_dir)
|
||||
acquired = []
|
||||
lock_a.acquire()
|
||||
try:
|
||||
def _take():
|
||||
# RLock 须由加锁线程释放,故获取与释放同线程。
|
||||
lock_b.acquire()
|
||||
acquired.append(True)
|
||||
lock_b.release()
|
||||
|
||||
t = threading.Thread(target=_take)
|
||||
t.start()
|
||||
time.sleep(0.3)
|
||||
self.assertEqual(acquired, [])
|
||||
self.assertTrue(t.is_alive())
|
||||
finally:
|
||||
lock_a.release()
|
||||
t.join(timeout=5)
|
||||
self.assertEqual(acquired, [True])
|
||||
self.assertTrue(os.path.exists(
|
||||
lock_path_for_state_dir(state_dir)))
|
||||
|
||||
def test_cross_process_mutual_exclusion(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
state_dir = os.path.join(d, "state")
|
||||
os.makedirs(state_dir, exist_ok=True)
|
||||
root = str(Path(__file__).resolve().parents[2])
|
||||
ready = os.path.join(state_dir, "child-locked")
|
||||
script = (
|
||||
"import sys, time; sys.path.insert(0, %r); "
|
||||
"from cloud.chv.locking import StateLock; "
|
||||
"lock = StateLock(%r); lock.acquire(); "
|
||||
"open(%r, 'w').write('locked'); "
|
||||
"time.sleep(10)" % (root, state_dir, ready))
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, "-c", script],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
deadline = time.time() + 10
|
||||
while not os.path.exists(ready) and time.time() < deadline:
|
||||
time.sleep(0.05)
|
||||
self.assertTrue(os.path.exists(ready))
|
||||
self.assertIsNone(proc.poll())
|
||||
lock = StateLock(state_dir)
|
||||
acquired = []
|
||||
|
||||
def _take():
|
||||
lock.acquire()
|
||||
acquired.append(True)
|
||||
lock.release()
|
||||
|
||||
t = threading.Thread(target=_take)
|
||||
t.start()
|
||||
time.sleep(0.5)
|
||||
# 子进程持锁期间,本进程的加锁必须阻塞。
|
||||
self.assertEqual(acquired, [])
|
||||
proc.terminate()
|
||||
proc.wait(timeout=5)
|
||||
t.join(timeout=5)
|
||||
self.assertEqual(acquired, [True])
|
||||
finally:
|
||||
if proc.poll() is None:
|
||||
proc.kill()
|
||||
proc.wait(timeout=5)
|
||||
|
||||
def test_purge_snapshot_traversal_under_concurrent_mutation(self):
|
||||
# 旧无锁 pop + 持锁遍历并发可致字典迭代异常;新实现
|
||||
# 快照键列表后逐项持锁删,并发变更加锁路径下不得抛错。
|
||||
with tempfile.TemporaryDirectory():
|
||||
case = P1FixCase()
|
||||
case.setUp()
|
||||
try:
|
||||
ex = case.make_exec()
|
||||
ex.state_dir = case.state_dir
|
||||
stop = threading.Event()
|
||||
|
||||
def mutate():
|
||||
i = 0
|
||||
while not stop.is_set():
|
||||
name = "chv-m%d" % (i % 5)
|
||||
with ex._vms_lock:
|
||||
ex._vms[name] = FakeProc(alive=False)
|
||||
i += 1
|
||||
|
||||
t = threading.Thread(target=mutate)
|
||||
t.start()
|
||||
try:
|
||||
for _ in range(50):
|
||||
ex._purge_dead_registry()
|
||||
finally:
|
||||
stop.set()
|
||||
t.join(timeout=5)
|
||||
finally:
|
||||
case.tearDown()
|
||||
|
||||
|
||||
class WakeSerializationP1FixTest(P1FixCase):
|
||||
def test_concurrent_wakes_serialized_by_state_lock(self):
|
||||
# 两路并发 wake:每路 api 往返睡 0.2 秒;串行则总耗时
|
||||
# 至少 0.35 秒,并行则约 0.2 秒。阈值留足余量防抖动。
|
||||
def sleeping_api(socket_path, method, path, payload=None):
|
||||
if path == "/vmm.ping":
|
||||
time.sleep(0.2)
|
||||
return (200, {})
|
||||
|
||||
ex1 = self.make_exec(api_client=sleeping_api)
|
||||
ex2 = self.make_exec(api_client=sleeping_api)
|
||||
results = []
|
||||
start = time.time()
|
||||
|
||||
def wake(ex, task_id):
|
||||
try:
|
||||
results.append(ex.wake_vm({"id": task_id})["state"])
|
||||
except Exception as exc: # noqa: BLE001 - 失败即测试失败
|
||||
results.append("error:%s" % exc)
|
||||
|
||||
t1 = threading.Thread(target=wake, args=(ex1, "s1"))
|
||||
t2 = threading.Thread(target=wake, args=(ex2, "s2"))
|
||||
t1.start()
|
||||
t2.start()
|
||||
t1.join(timeout=30)
|
||||
t2.join(timeout=30)
|
||||
elapsed = time.time() - start
|
||||
self.assertEqual(sorted(results), ["running", "running"])
|
||||
self.assertGreaterEqual(elapsed, 0.35)
|
||||
self.assertTrue(os.path.exists(
|
||||
lock_path_for_state_dir(self.state_dir)))
|
||||
|
||||
|
||||
# -- P1-06:单条快照超帽 ---------------------------------------------------
|
||||
|
||||
class SingleSnapshotCapP1FixTest(unittest.TestCase):
|
||||
def test_single_oversize_snapshot_rejected_without_side_effects(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
old = os.path.join(tmp, "vm1", "old.snap")
|
||||
os.makedirs(os.path.join(tmp, "vm1"), exist_ok=True)
|
||||
with open(old, "wb") as fh:
|
||||
fh.write(b"O" * 100)
|
||||
from cloud.chv.disk import _save_snapshots
|
||||
_save_snapshots(tmp, "vm1", [{
|
||||
"url": "file://" + old, "path": old, "size": 100,
|
||||
"created_at": 0}])
|
||||
new = os.path.join(tmp, "vm1", "new.snap")
|
||||
with open(new, "wb") as fh:
|
||||
fh.write(b"N" * 1000)
|
||||
# 总量帽 500 字节:新单条 1000 字节本身已超帽。
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
record_snapshot(tmp, "vm1", "file://" + new,
|
||||
500 / (1024 ** 3))
|
||||
self.assertEqual(ctx.exception.kind,
|
||||
"snapshot_quota_exceeded")
|
||||
# 拒绝无副作用:旧快照文件仍在、登记仍只有旧条目。
|
||||
self.assertTrue(os.path.exists(old))
|
||||
from cloud.chv.disk import _load_snapshots
|
||||
entries = _load_snapshots(tmp, "vm1")
|
||||
self.assertEqual(len(entries), 1)
|
||||
self.assertEqual(entries[0]["path"], old)
|
||||
|
||||
|
||||
# -- P1-07:>0 校验 ---------------------------------------------------------
|
||||
|
||||
class PositiveConfigP1FixTest(P1FixCase):
|
||||
def test_config_layer_rejects_non_positive(self):
|
||||
base_env = {}
|
||||
for env_extra, field in (
|
||||
({"CHV_CPU": "0"}, "cpu"),
|
||||
({"CHV_CPU": "-2"}, "cpu"),
|
||||
({"CHV_MEM": "0"}, "mem"),
|
||||
({"CHV_MEM": "0G"}, "mem"),
|
||||
({"CHV_TASK_TIMEOUT": "0"}, "timeout"),
|
||||
({"CHV_TASK_TIMEOUT": "-5"}, "timeout")):
|
||||
env = dict(base_env)
|
||||
env.update(env_extra)
|
||||
with self.assertRaises(ValueError, msg=field):
|
||||
build_chv_config({}, env)
|
||||
# 合法值与 None(不限)不受影响。
|
||||
cfg = build_chv_config({}, {"CHV_TASK_TIMEOUT": "30"})
|
||||
self.assertEqual(cfg.task_timeout, 30.0)
|
||||
cfg = build_chv_config({}, {})
|
||||
self.assertIsNone(cfg.task_timeout)
|
||||
|
||||
def test_preflight_rejects_tampered_attributes(self):
|
||||
for attr, value in (("cpus", "0"), ("cpus", "-1"),
|
||||
("memory", "0"), ("task_timeout", 0),
|
||||
("task_timeout", -3)):
|
||||
ex = self.make_exec()
|
||||
setattr(ex, attr, value)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "config_invalid")
|
||||
|
||||
|
||||
# -- P1-08:剪枝审计与快照失败审计 -----------------------------------------
|
||||
|
||||
class SnapshotAuditP1FixTest(P1FixCase):
|
||||
def test_retention_prune_delete_failure_audited(self):
|
||||
ex = self.make_exec()
|
||||
vm_dir = os.path.join(self.state_dir, "chv-abc123")
|
||||
os.makedirs(vm_dir, exist_ok=True)
|
||||
old = os.path.join(self.state_dir, "old.snap")
|
||||
with open(old, "wb") as fh:
|
||||
fh.write(b"O" * 10)
|
||||
with open(os.path.join(vm_dir, "snapshots.json"), "w") as fh:
|
||||
json.dump([{"url": "file://" + old, "path": old,
|
||||
"size": 10, "created_at": 0}], fh)
|
||||
new = os.path.join(self.state_dir, "new.snap")
|
||||
with open(new, "wb") as fh:
|
||||
fh.write(b"N" * 10)
|
||||
real_remove = os.remove
|
||||
|
||||
def remove(path):
|
||||
if path == old:
|
||||
raise OSError("删不掉")
|
||||
return real_remove(path)
|
||||
|
||||
with mock.patch.object(os, "remove", side_effect=remove):
|
||||
out = ex.snapshot_vm(self._task(), "file://" + new)
|
||||
self.assertEqual(out, "file://" + new)
|
||||
self.assertIn("vm_snapshot_prune_failed",
|
||||
self._audit_events())
|
||||
self.assertTrue(os.path.exists(old))
|
||||
|
||||
def test_snapshot_failure_audited(self):
|
||||
def failing_runner(cmd, timeout=None):
|
||||
if cmd and cmd[0] == "ch-remote":
|
||||
raise RuntimeError("ch-remote 挂了")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex = self.make_exec(runner=failing_runner)
|
||||
# 目标须在 state_dir 内才过白名单(ch-remote 失败在后)。
|
||||
target = "file://" + os.path.join(self.state_dir, "x.snap")
|
||||
with self.assertRaises(RuntimeError):
|
||||
ex.snapshot_vm(self._task(), target)
|
||||
self.assertIn("vm_snapshot_failed", self._audit_events())
|
||||
|
||||
|
||||
# -- P1-09:4 处三元 else 分支 ----------------------------------------------
|
||||
|
||||
class BranchCoverP1FixTest(P1FixCase):
|
||||
def test_pause_resume_snapshot_str_forms(self):
|
||||
ex = self.make_exec()
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
self.assertTrue(ex.pause_vm("chv-abc"))
|
||||
self.assertTrue(ex.resume_vm("chv-abc"))
|
||||
snap = os.path.join(self.state_dir, "snap.raw")
|
||||
with open(snap, "wb") as fh:
|
||||
fh.write(b"S" * 10)
|
||||
out = ex.snapshot_vm("chv-abc", "file://" + snap)
|
||||
self.assertEqual(out, "file://" + snap)
|
||||
self.assertIn("vm_pause", self._audit_events())
|
||||
self.assertIn("vm_resume", self._audit_events())
|
||||
self.assertIn("vm_snapshot", self._audit_events())
|
||||
|
||||
def test_spill_audit_non_dict_task(self):
|
||||
ex = self.make_exec()
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
ex._spill_audit("ev", "plain-task-id", "detail", None)
|
||||
path = os.path.join(self.state_dir, "chv-audit-spill.jsonl")
|
||||
with open(path) as fh:
|
||||
row = json.loads(fh.readline())
|
||||
self.assertEqual(row["task_id"], "plain-task-id")
|
||||
self.assertEqual(row["event"], "ev")
|
||||
|
||||
|
||||
# -- 覆盖补齐:新分支的行覆盖(诚实口径,不凑数改代码) --------
|
||||
|
||||
class CoverageCompletionP1FixTest(P1FixCase):
|
||||
def test_prune_events_no_file_and_no_delete(self):
|
||||
from cloud.chv.disk import _save_snapshots
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
_save_snapshots(tmp, "vm1", [
|
||||
{"url": "file:///gone.snap", "path": "/gone.snap",
|
||||
"size": 5, "created_at": 0},
|
||||
{"url": "https://example.invalid/x", "path": None,
|
||||
"size": 5, "created_at": 0}])
|
||||
events = []
|
||||
kept, pruned = prune_snapshots(
|
||||
tmp, "vm1", retention_days=30, now=10 ** 9,
|
||||
events=events)
|
||||
self.assertEqual(kept, [])
|
||||
self.assertEqual(len(pruned), 2)
|
||||
kinds = [e[0] for e in events]
|
||||
self.assertEqual(kinds, ["pruned_no_file", "pruned_no_file"])
|
||||
events2 = []
|
||||
_save_snapshots(tmp, "vm1", [
|
||||
{"url": "file:///gone2.snap", "path": "/gone2.snap",
|
||||
"size": 5, "created_at": 0}])
|
||||
kept2, _ = prune_snapshots(
|
||||
tmp, "vm1", retention_days=30, now=10 ** 9,
|
||||
delete_files=False, events=events2)
|
||||
self.assertEqual(kept2, [])
|
||||
self.assertEqual([e[0] for e in events2],
|
||||
["pruned_no_file"])
|
||||
|
||||
def test_record_without_cap_and_tighten_delete_failure_event(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
entries, removed = record_snapshot(
|
||||
tmp, "vm1", "https://example.invalid/new", None)
|
||||
self.assertEqual(len(entries), 1)
|
||||
self.assertEqual(removed, [])
|
||||
old = os.path.join(tmp, "vm1", "old.snap")
|
||||
os.makedirs(os.path.join(tmp, "vm1"), exist_ok=True)
|
||||
with open(old, "wb") as fh:
|
||||
fh.write(b"O" * 100)
|
||||
from cloud.chv.disk import _save_snapshots
|
||||
_save_snapshots(tmp, "vm1", [
|
||||
{"url": "file://" + old, "path": old, "size": 100,
|
||||
"created_at": 0}])
|
||||
real_remove = os.remove
|
||||
|
||||
def remove(path):
|
||||
if path == old:
|
||||
raise OSError("删不掉")
|
||||
return real_remove(path)
|
||||
|
||||
events = []
|
||||
with mock.patch.object(os, "remove", side_effect=remove):
|
||||
entries2, removed2 = record_snapshot(
|
||||
tmp, "vm1", "https://example.invalid/new2",
|
||||
0.0, events=events)
|
||||
self.assertEqual(len(removed2), 1)
|
||||
self.assertEqual([e[0] for e in events], ["delete_failed"])
|
||||
self.assertTrue(os.path.exists(old))
|
||||
|
||||
def test_tighten_no_file_entry_event(self):
|
||||
from cloud.chv.disk import _save_snapshots
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
_save_snapshots(tmp, "vm1", [
|
||||
{"url": "https://example.invalid/old", "path": None,
|
||||
"size": 100, "created_at": 0}])
|
||||
events = []
|
||||
entries, removed = record_snapshot(
|
||||
tmp, "vm1", "https://example.invalid/new",
|
||||
0.0, events=events)
|
||||
self.assertEqual(len(removed), 1)
|
||||
self.assertEqual([e[0] for e in events],
|
||||
["pruned_no_file"])
|
||||
|
||||
def test_state_lock_ctx_refreshes_on_state_dir_change(self):
|
||||
ex = self.make_exec()
|
||||
old_lock = ex._state_lock
|
||||
new_dir = os.path.join(self.dir, "state2")
|
||||
ex.state_dir = new_dir
|
||||
self.assertIsNot(ex._state_lock_ctx(), old_lock)
|
||||
self.assertEqual(ex._state_lock.state_dir, new_dir)
|
||||
|
||||
def test_gc_and_secrets_listdir_failure_returns_empty(self):
|
||||
ex = self.make_exec()
|
||||
ex.state_dir = os.path.join(self.dir, "missing-dir")
|
||||
self.assertEqual(ex.gc_stale_disks(),
|
||||
{"deleted": [], "failed": []})
|
||||
self.assertEqual(ex.cleanup_stale_secrets(), [])
|
||||
|
||||
def test_cleanup_listdir_failure_returns_empty(self):
|
||||
ex = self.make_exec()
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
real_listdir = os.listdir
|
||||
calls = {"n": 0}
|
||||
|
||||
def listdir(path):
|
||||
calls["n"] += 1
|
||||
if calls["n"] >= 3:
|
||||
raise OSError("listdir 挂了")
|
||||
return real_listdir(path)
|
||||
|
||||
with mock.patch.object(os, "listdir", side_effect=listdir):
|
||||
self.assertEqual(ex.cleanup_stale_vms(), [])
|
||||
|
||||
def test_orphan_entries_compat_wrapper_and_no_emit_executor(self):
|
||||
from cloud.chv.orphans import _orphan_entries
|
||||
data = {"a": {"state": "warm", "vm": "chv-a",
|
||||
"task_id": "a", "snapshot_url": None},
|
||||
"b": {"state": "running", "vm": "chv-b"}}
|
||||
self.assertEqual(_orphan_entries(data),
|
||||
[{"vm": "chv-a", "task_id": "a",
|
||||
"snapshot_url": None}])
|
||||
|
||||
class NoEmit:
|
||||
def __init__(self, state_dir):
|
||||
self.state_dir = state_dir
|
||||
self.shutdowns = []
|
||||
|
||||
def shutdown_vm(self, vm):
|
||||
self.shutdowns.append(vm)
|
||||
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
_write_registry(d, {
|
||||
"a": {"state": "warm", "vm": "chv-a"}})
|
||||
ex = NoEmit(d)
|
||||
self.assertEqual(reconcile_orphans_for_executor(ex),
|
||||
["chv-a"])
|
||||
|
||||
def test_state_lock_flock_failure_closes_and_releases(self):
|
||||
import fcntl
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
state_dir = os.path.join(d, "state")
|
||||
lock = StateLock(state_dir)
|
||||
with mock.patch.object(fcntl, "flock",
|
||||
side_effect=OSError("flock 挂了")):
|
||||
with self.assertRaises(OSError):
|
||||
lock.acquire()
|
||||
self.assertIsNone(lock._fd)
|
||||
# 失败后锁仍可用(RLock 已释放、fd 已关闭)。
|
||||
lock.acquire()
|
||||
lock.release()
|
||||
|
||||
def test_pool_reconcile_failed_emit_failure_swallowed(self):
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
stub = _PoolStubForP1Fix()
|
||||
pool = WarmPool(stub, registry_path=None)
|
||||
pool._entries["old-1"] = {"state": "orphan", "vm": "vm-old",
|
||||
"task": {"id": "old-1"}}
|
||||
self.assertEqual(pool.reconcile_orphans(), [])
|
||||
self.assertIn("old-1", pool._entries)
|
||||
|
||||
|
||||
class _PoolStubForP1Fix:
|
||||
"""实验池失败审计吞错测试用假执行器。"""
|
||||
|
||||
def shutdown_vm(self, task_or_name):
|
||||
raise RuntimeError("shutdown 失败")
|
||||
|
||||
def _emit(self, event, task, detail="", kind=None):
|
||||
raise RuntimeError("audit 失败")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,701 @@
|
||||
"""CHV 安全审计修复轨测试(feat-chv-secfix,堆叠于 #239)。
|
||||
|
||||
只用标准库与 mock runner/launcher/api_client,绝不启动真实
|
||||
VM:当前环境没有可跑真 VM 的宿主。本文件只证明代码层面
|
||||
的修复(密钥移出 argv、磁盘删除、默认值、校验、审计等)
|
||||
成立;真机否定测试(tap 防火墙实效、unset 代理后直连、
|
||||
IP spoof 后 SSH 拒绝等)一律列在就绪清单 §11,不在本
|
||||
文件声称已验证。测试里只用 FAKE 占位密钥,绝无真实密钥。
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
try:
|
||||
from cloud.chv.config import build_chv_config
|
||||
from cloud.chv.disk import (
|
||||
check_snapshot_limits,
|
||||
delete_task_disk,
|
||||
provision_task_disk,
|
||||
record_snapshot,
|
||||
)
|
||||
from cloud.chv.hardening import (
|
||||
ensure_secure_state_dir,
|
||||
socket_is_live,
|
||||
validate_net_entries,
|
||||
validate_snapshot_url,
|
||||
)
|
||||
from cloud.chv.hostkeys import (
|
||||
build_seed_disk,
|
||||
dirty_marker_path,
|
||||
write_seed,
|
||||
)
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.chv.ssh_transport import (
|
||||
build_ssh_command,
|
||||
split_secret_env,
|
||||
)
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
)
|
||||
except ImportError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv.config import build_chv_config
|
||||
from cloud.chv.disk import (
|
||||
check_snapshot_limits,
|
||||
delete_task_disk,
|
||||
provision_task_disk,
|
||||
record_snapshot,
|
||||
)
|
||||
from cloud.chv.hardening import (
|
||||
ensure_secure_state_dir,
|
||||
socket_is_live,
|
||||
validate_net_entries,
|
||||
validate_snapshot_url,
|
||||
)
|
||||
from cloud.chv.hostkeys import (
|
||||
build_seed_disk,
|
||||
dirty_marker_path,
|
||||
write_seed,
|
||||
)
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.chv.ssh_transport import (
|
||||
build_ssh_command,
|
||||
split_secret_env,
|
||||
)
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
)
|
||||
|
||||
|
||||
class FakeResult:
|
||||
def __init__(self, returncode=0, stdout="", stderr=""):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
class FakeProc:
|
||||
def __init__(self, alive=True):
|
||||
self._alive = alive
|
||||
|
||||
def poll(self):
|
||||
return None if self._alive else 1
|
||||
|
||||
def terminate(self):
|
||||
self._alive = False
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self._alive = False
|
||||
|
||||
|
||||
def _write(path, content=b"x"):
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(content)
|
||||
os.chmod(path, 0o755)
|
||||
return path
|
||||
|
||||
|
||||
class SecfixCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.binary = _write(os.path.join(self.dir, "cloud-hypervisor"))
|
||||
self.ch_remote = _write(os.path.join(self.dir, "ch-remote"))
|
||||
self.kernel = _write(os.path.join(self.dir, "vmlinux"))
|
||||
self.rootfs = _write(os.path.join(self.dir, "disk.raw"),
|
||||
b"R" * 100)
|
||||
self.ssh_key = _write(os.path.join(self.dir, "id_chv"))
|
||||
self.kvm = _write(os.path.join(self.dir, "kvm"))
|
||||
self.cpuinfo = os.path.join(self.dir, "cpuinfo")
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu vmx sse\n")
|
||||
self.state_dir = os.path.join(self.dir, "state")
|
||||
self.calls = []
|
||||
self.audit = []
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _default_runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh":
|
||||
if cmd[-1] == "true":
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
if cmd and cmd[0] == "ssh-keygen":
|
||||
priv = cmd[cmd.index("-f") + 1]
|
||||
with open(priv, "w") as fh:
|
||||
fh.write("-----BEGIN OPENSSH PRIVATE KEY-----\nFAKE\n")
|
||||
os.chmod(priv, 0o600)
|
||||
with open(priv + ".pub", "w") as fh:
|
||||
fh.write("ssh-ed25519 AAAAfake1 vm-key\n")
|
||||
return FakeResult(0, "", "")
|
||||
if cmd and cmd[0] == "genisoimage":
|
||||
out = cmd[cmd.index("-output") + 1]
|
||||
with open(out, "wb") as fh:
|
||||
fh.write(b"CIDATA" * 16)
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
def _default_api(self, socket_path, method, path, payload=None):
|
||||
return (200, {})
|
||||
|
||||
def make_exec(self, runner=None, env_extra=None, **overrides):
|
||||
# 生产默认:provisioned(不显式设 keyscan),回环仍
|
||||
# 是实验室宣称(mock runner 假装转发已通)。
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
"CHV_SSH_FORWARD": "1"}
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
kwargs = dict(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, rootfs=self.rootfs,
|
||||
ssh_key=self.ssh_key, kvm_path=self.kvm,
|
||||
cpuinfo_path=self.cpuinfo, state_dir=self.state_dir,
|
||||
runner=runner or self._default_runner,
|
||||
launcher=lambda cmd: FakeProc(alive=True),
|
||||
api_client=self._default_api,
|
||||
sleeper=lambda s: None, env=env)
|
||||
kwargs.update(overrides)
|
||||
ex = CloudHypervisorExecutor(**kwargs)
|
||||
ex.audit_callback = lambda e, t, d, k=None: self.audit.append(
|
||||
(e, t.get("id") if isinstance(t, dict) else t, d, k))
|
||||
return ex
|
||||
|
||||
|
||||
# -- H1 egress 代码门 + H7 tap 隔离 ------------------------------------
|
||||
class EgressAndNetTest(SecfixCase):
|
||||
def test_egress_proxy_non_http_rejected(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_EGRESS_PROXY": "ftp://10.0.0.1:21"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "config_invalid")
|
||||
|
||||
def test_egress_proxy_without_host_rejected(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_EGRESS_PROXY": "http:///path"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "config_invalid")
|
||||
|
||||
def test_net_unknown_field_rejected(self):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
validate_net_entries(
|
||||
[{"tap": "tap0", "mac": "aa:bb:cc:dd:ee:01",
|
||||
"ip": "10.0.0.2", "gateway": "10.0.0.1"}], [])
|
||||
self.assertEqual(ctx.exception.kind, "net_config_invalid")
|
||||
|
||||
def test_net_intra_config_dup_rejected(self):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
validate_net_entries(
|
||||
[{"tap": "tap0", "mac": "aa:bb:cc:dd:ee:01",
|
||||
"ip": "10.0.0.2"},
|
||||
{"tap": "tap0", "mac": "aa:bb:cc:dd:ee:02",
|
||||
"ip": "10.0.0.3"}], [])
|
||||
self.assertEqual(ctx.exception.kind, "net_config_invalid")
|
||||
|
||||
def test_net_cross_vm_conflict_refuses_second_wake(self):
|
||||
net = '[{"tap": "tap0", "mac": "aa:bb:cc:dd:ee:01",' \
|
||||
'"ip": "10.0.0.2", "mask": "255.255.255.0"}]'
|
||||
ex = self.make_exec(env_extra={"CHV_NET_JSON": net})
|
||||
out = ex.wake_vm({"id": "vm1"})
|
||||
self.assertEqual(out["state"], "running")
|
||||
# 第一台的 net.json 已登记;第二台同 tap 即拒绝,
|
||||
# 且未起 VMM(launcher 只被调过一次)。
|
||||
launches = []
|
||||
ex.launcher = lambda cmd: launches.append(cmd) or FakeProc(True)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm({"id": "vm2"})
|
||||
self.assertEqual(ctx.exception.kind, "net_tap_conflict")
|
||||
self.assertEqual(launches, [])
|
||||
self.assertTrue(any(e == "vm_failed"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
|
||||
# -- H2 密钥移出 argv ---------------------------------------------------
|
||||
class SecretDeliveryTest(SecfixCase):
|
||||
def test_secret_env_split_and_argv_refusal(self):
|
||||
secrets, plain = split_secret_env(
|
||||
{"SUB2API_API_KEY": "FAKE-KEY",
|
||||
"SUB2API_BASE_URL": "http://gw/"})
|
||||
self.assertEqual(secrets, {"SUB2API_API_KEY": "FAKE-KEY"})
|
||||
self.assertEqual(plain, {"SUB2API_BASE_URL": "http://gw/"})
|
||||
with self.assertRaises(ValueError):
|
||||
build_ssh_command(2222, "/k", "root", None, "/kh",
|
||||
["true"],
|
||||
extra_env={"SUB2API_API_KEY": "FAKE-KEY"})
|
||||
|
||||
def test_seed_writes_single_write_files_with_guest_env(self):
|
||||
seed_dir = os.path.join(self.dir, "seed")
|
||||
write_seed(seed_dir, "chv-t1", "FAKE-PRIV", "ssh-ed25519 AAAAx",
|
||||
secret_env={"SUB2API_API_KEY": "FAKE-KEY-123"})
|
||||
text = ""
|
||||
with open(os.path.join(seed_dir, "user-data")) as fh:
|
||||
text = fh.read()
|
||||
# 只能有一个 write_files 键(重复键会覆盖 host key 条目)。
|
||||
self.assertEqual(text.count("write_files:"), 1)
|
||||
self.assertIn("/etc/chv/guest-env", text)
|
||||
self.assertIn("SUB2API_API_KEY=", text)
|
||||
self.assertEqual(oct(os.stat(
|
||||
os.path.join(seed_dir, "user-data")).st_mode & 0o777),
|
||||
"0o600")
|
||||
|
||||
def test_keyscan_with_secret_fails_closed_before_launch(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_HOST_KEY_MODE": "keyscan"})
|
||||
launches = []
|
||||
ex.launcher = lambda cmd: launches.append(cmd) or FakeProc(True)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm({"id": "k1"},
|
||||
extra_env={"SUB2API_API_KEY": "FAKE-KEY"})
|
||||
self.assertEqual(ctx.exception.kind,
|
||||
"secret_env_requires_provisioned")
|
||||
self.assertEqual(launches, [])
|
||||
|
||||
def test_provisioned_wake_deletes_host_private_key(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.wake_vm({"id": "s1"},
|
||||
extra_env={"SUB2API_API_KEY": "FAKE-KEY"})
|
||||
self.assertEqual(out["state"], "running")
|
||||
vm_dir = os.path.join(self.state_dir, "chv-s1")
|
||||
# 宿主私钥已删(seed 盘建好后即删);seed.iso 在且 0600;
|
||||
# 密钥投递有审计(只含名,不含值)。
|
||||
self.assertFalse(os.path.exists(
|
||||
os.path.join(vm_dir, "ssh_host_ed25519_key")))
|
||||
seed_iso = os.path.join(vm_dir, "seed.iso")
|
||||
self.assertTrue(os.path.exists(seed_iso))
|
||||
self.assertEqual(oct(os.stat(seed_iso).st_mode & 0o777),
|
||||
"0o600")
|
||||
self.assertTrue(any(e == "host_key_generated"
|
||||
for e, _, _, _ in self.audit))
|
||||
self.assertFalse(any("FAKE-KEY" in (d or "")
|
||||
for _, _, d, _ in self.audit))
|
||||
|
||||
def test_seed_iso_build_sets_0600(self):
|
||||
seed_dir = os.path.join(self.dir, "seed2")
|
||||
os.makedirs(seed_dir, exist_ok=True)
|
||||
out = os.path.join(self.dir, "seed2.iso")
|
||||
build_seed_disk(seed_dir, out, runner=self._default_runner)
|
||||
self.assertEqual(oct(os.stat(out).st_mode & 0o777), "0o600")
|
||||
|
||||
|
||||
# -- H3 任务盘删除 + GC --------------------------------------------------
|
||||
class DiskDeletionTest(SecfixCase):
|
||||
def test_stop_deletes_task_disk_and_audits(self):
|
||||
ex = self.make_exec()
|
||||
disk = provision_task_disk(self.state_dir, "chv-d1",
|
||||
self.rootfs, "copy")
|
||||
self.assertTrue(os.path.exists(disk))
|
||||
ex.stop_vm({"id": "d1"})
|
||||
self.assertFalse(os.path.exists(disk))
|
||||
self.assertTrue(any(e == "vm_disk_deleted"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
def test_gc_deletes_orphan_disk_only(self):
|
||||
ex = self.make_exec()
|
||||
provision_task_disk(self.state_dir, "chv-orphan",
|
||||
self.rootfs, "copy")
|
||||
live_disk = provision_task_disk(self.state_dir, "chv-live",
|
||||
self.rootfs, "copy")
|
||||
# live VM 有 socket 文件(mock 占位,不代表活 VMM,
|
||||
# 但 GC 的 live 判定含 socket 存在——见 gc_stale_disks
|
||||
# 的 socket 判定,此处只验 orphan 被删、live 不动)。
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(os.path.join(self.state_dir, "chv-live.sock"),
|
||||
"w") as fh:
|
||||
fh.write("")
|
||||
result = ex.gc_stale_disks()
|
||||
self.assertIn("chv-orphan", result["deleted"])
|
||||
self.assertTrue(os.path.exists(live_disk))
|
||||
|
||||
def test_disk_delete_failure_audited_and_marks_dirty(self):
|
||||
ex = self.make_exec()
|
||||
provision_task_disk(self.state_dir, "chv-f1",
|
||||
self.rootfs, "copy")
|
||||
from unittest import mock
|
||||
with mock.patch("cloud.chv.executor.delete_task_disk",
|
||||
side_effect=OSError("disk busy")):
|
||||
ex.stop_vm({"id": "f1"})
|
||||
self.assertTrue(any(e == "vm_disk_delete_failed"
|
||||
for e, _, _, _ in self.audit))
|
||||
self.assertTrue(os.path.exists(
|
||||
dirty_marker_path(self.state_dir, "chv-f1")))
|
||||
|
||||
|
||||
# -- H4 默认值 ------------------------------------------------------------
|
||||
class DefaultsTest(unittest.TestCase):
|
||||
def test_production_defaults(self):
|
||||
cfg = build_chv_config({}, {})
|
||||
self.assertEqual(cfg.host_key_mode, "provisioned")
|
||||
self.assertNotEqual(cfg.state_dir, "/tmp/chv-vms")
|
||||
self.assertTrue(cfg.state_dir.startswith(os.path.expanduser("~")))
|
||||
self.assertEqual(cfg.disk_gb, 50.0)
|
||||
self.assertEqual(cfg.snapshot_max_count, 10)
|
||||
self.assertEqual(cfg.snapshot_max_total_gb, 20.0)
|
||||
self.assertEqual(cfg.snapshot_retention_days, 30.0)
|
||||
|
||||
def test_keyscan_still_explicitly_selectable(self):
|
||||
cfg = build_chv_config(
|
||||
{}, {"CHV_HOST_KEY_MODE": "keyscan"})
|
||||
self.assertEqual(cfg.host_key_mode, "keyscan")
|
||||
|
||||
def test_disk_gb_zero_or_negative_rejected(self):
|
||||
with self.assertRaises(ValueError):
|
||||
build_chv_config({}, {"CHV_DISK_GB": "0"})
|
||||
with self.assertRaises(ValueError):
|
||||
build_chv_config({}, {"CHV_DISK_GB": "-5"})
|
||||
|
||||
|
||||
# -- H5 崩溃残留扫描 -------------------------------------------------------
|
||||
class ResidueScanTest(SecfixCase):
|
||||
def test_cleanup_stale_secrets_removes_residue(self):
|
||||
ex = self.make_exec()
|
||||
vm_dir = os.path.join(self.state_dir, "chv-old")
|
||||
os.makedirs(os.path.join(vm_dir, "seed"), exist_ok=True)
|
||||
with open(os.path.join(vm_dir, "ssh_host_ed25519_key"),
|
||||
"w") as fh:
|
||||
fh.write("FAKE-PRIV")
|
||||
with open(os.path.join(vm_dir, "seed.iso"), "wb") as fh:
|
||||
fh.write(b"CIDATA")
|
||||
cleaned = ex.cleanup_stale_secrets()
|
||||
self.assertIn("chv-old", cleaned)
|
||||
self.assertFalse(os.path.exists(
|
||||
os.path.join(vm_dir, "ssh_host_ed25519_key")))
|
||||
self.assertFalse(os.path.exists(
|
||||
os.path.join(vm_dir, "seed.iso")))
|
||||
self.assertTrue(any(e == "host_secret_residue_cleaned"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
|
||||
# -- H6 state_dir 与 socket 权限 -------------------------------------------
|
||||
class StateDirTest(SecfixCase):
|
||||
def test_preflight_creates_state_dir_0700(self):
|
||||
ex = self.make_exec()
|
||||
self.assertTrue(ex.preflight())
|
||||
mode = os.stat(self.state_dir).st_mode & 0o777
|
||||
self.assertEqual(oct(mode), "0o700")
|
||||
|
||||
def test_ensure_secure_state_dir_fixes_loose_dir(self):
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
os.chmod(self.state_dir, 0o755)
|
||||
ensure_secure_state_dir(self.state_dir)
|
||||
self.assertEqual(
|
||||
oct(os.stat(self.state_dir).st_mode & 0o777), "0o700")
|
||||
|
||||
def test_group_writable_socket_fails_fast(self):
|
||||
ex = self.make_exec()
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
sock_path = os.path.join(self.state_dir, "chv-sock.sock")
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(sock_path)
|
||||
srv.listen(1)
|
||||
os.chmod(sock_path, 0o666)
|
||||
try:
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm({"id": "sock"})
|
||||
self.assertEqual(ctx.exception.kind,
|
||||
"socket_permissions_insecure")
|
||||
finally:
|
||||
srv.close()
|
||||
|
||||
def test_dead_socket_not_counted_live_socket_counted(self):
|
||||
ex = self.make_exec(env_extra={"CHV_MAX_VMS": "1"})
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(os.path.join(self.state_dir, "chv-dead.sock"),
|
||||
"w") as fh:
|
||||
fh.write("")
|
||||
self.assertEqual(ex._persistent_vm_count(), 0)
|
||||
self.assertFalse(socket_is_live(
|
||||
os.path.join(self.state_dir, "chv-dead.sock")))
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(os.path.join(self.state_dir, "chv-live.sock"))
|
||||
srv.listen(1)
|
||||
try:
|
||||
self.assertTrue(socket_is_live(
|
||||
os.path.join(self.state_dir, "chv-live.sock")))
|
||||
self.assertEqual(ex._persistent_vm_count(), 1)
|
||||
finally:
|
||||
srv.close()
|
||||
|
||||
|
||||
# -- M1 远程转义 ------------------------------------------------------------
|
||||
class RemoteQuotingTest(unittest.TestCase):
|
||||
def test_metachars_round_trip_through_remote_shell(self):
|
||||
inner = ["python3", "/app/host.py", "dispatch",
|
||||
"--task", "a; rm -rf / $(whoami) 'q' \"d\""]
|
||||
cmd = build_ssh_command(2222, "/k", "root", None, "/kh",
|
||||
inner, ssh_host="10.0.0.5")
|
||||
user_idx = cmd.index("root@10.0.0.5")
|
||||
remote = cmd[user_idx + 1:]
|
||||
# ssh 拼接后远程 shell 解析回来的恰好是本地 list。
|
||||
self.assertEqual(shlex.split(" ".join(remote)), inner)
|
||||
# 含元字符的 token 在 argv 里是被引用的单个元素,
|
||||
# 不是裸的 shell 元字符。
|
||||
task_token = [t for t in cmd if "whoami" in t]
|
||||
self.assertEqual(len(task_token), 1)
|
||||
self.assertTrue(task_token[0].startswith("'"))
|
||||
|
||||
|
||||
# -- M2 脏标记拒起 ------------------------------------------------------------
|
||||
class DirtyMarkerTest(SecfixCase):
|
||||
def test_dirty_marker_refuses_provision_and_wake(self):
|
||||
ex = self.make_exec()
|
||||
vm_dir = os.path.join(self.state_dir, "chv-dirty")
|
||||
os.makedirs(vm_dir, exist_ok=True)
|
||||
with open(dirty_marker_path(self.state_dir, "chv-dirty"),
|
||||
"w") as fh:
|
||||
fh.write("cleanup failed\n")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.provision_host_key({"id": "dirty"})
|
||||
self.assertEqual(ctx.exception.kind, "vm_dirty_refused")
|
||||
with self.assertRaises(ChvFailure) as ctx2:
|
||||
ex.wake_vm({"id": "dirty"})
|
||||
self.assertEqual(ctx2.exception.kind, "vm_dirty_refused")
|
||||
|
||||
|
||||
# -- M3 快照上限 ---------------------------------------------------------------
|
||||
class SnapshotLimitTest(SecfixCase):
|
||||
def _snap_file(self, name, size=64):
|
||||
path = os.path.join(self.state_dir, name)
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(b"S" * size)
|
||||
return "file://" + path
|
||||
|
||||
def test_snapshot_count_limit_refused_before_chremote(self):
|
||||
ex = self.make_exec(env_extra={"CHV_SNAPSHOT_MAX_COUNT": "1"})
|
||||
url1 = self._snap_file("s1.snap")
|
||||
record_snapshot(self.state_dir, "chv-q", url1, 20.0)
|
||||
calls_before = len(self.calls)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.snapshot_vm({"id": "q"}, self._snap_file("s2.snap"))
|
||||
self.assertEqual(ctx.exception.kind, "snapshot_quota_exceeded")
|
||||
# 上限在 ch-remote 之前拒绝,runner 未被调新命令。
|
||||
self.assertEqual(len(self.calls), calls_before)
|
||||
|
||||
def test_snapshot_retention_prunes_expired(self):
|
||||
import time
|
||||
ex = self.make_exec()
|
||||
old_url = self._snap_file("old.snap")
|
||||
entries = [{"url": old_url,
|
||||
"path": old_url[len("file://"):],
|
||||
"size": 64,
|
||||
"created_at": time.time() - 40 * 86400}]
|
||||
import json as _json
|
||||
vm_dir = os.path.join(self.state_dir, "chv-r")
|
||||
os.makedirs(vm_dir, exist_ok=True)
|
||||
with open(os.path.join(vm_dir, "snapshots.json"), "w") as fh:
|
||||
_json.dump(entries, fh)
|
||||
new_url = self._snap_file("new.snap")
|
||||
ex.snapshot_vm({"id": "r"}, new_url)
|
||||
self.assertFalse(os.path.exists(old_url[len("file://"):]))
|
||||
self.assertTrue(os.path.exists(new_url[len("file://"):]))
|
||||
|
||||
def test_snapshot_total_cap_prunes_oldest(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_SNAPSHOT_MAX_TOTAL_GB": "0.0000001"})
|
||||
# 总量帽约 107 字节:两块 64 字节快照登记后,旧的被剪。
|
||||
url1 = self._snap_file("t1.snap")
|
||||
record_snapshot(self.state_dir, "chv-t", url1, 20.0)
|
||||
url2 = self._snap_file("t2.snap")
|
||||
_entries, removed = record_snapshot(
|
||||
self.state_dir, "chv-t", url2, 0.0000001)
|
||||
self.assertEqual(len(removed), 1)
|
||||
self.assertFalse(os.path.exists(url1[len("file://"):]))
|
||||
|
||||
|
||||
# -- M4/M8 构建脚本 --------------------------------------------------------------
|
||||
class BuildScriptTest(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.bin_dir = os.path.join(self.dir, "bin")
|
||||
os.makedirs(os.path.join(self.bin_dir, "codex-resources"))
|
||||
for name in ("codex", "codex-code-mode-host"):
|
||||
p = os.path.join(self.bin_dir, name)
|
||||
with open(p, "w") as fh:
|
||||
fh.write("#!/bin/sh\n")
|
||||
os.chmod(p, 0o755)
|
||||
self.repo = os.path.join(self.dir, "repo")
|
||||
os.makedirs(os.path.join(self.repo, "cloud"))
|
||||
with open(os.path.join(self.repo, "host.py"), "w") as fh:
|
||||
fh.write("# host\n")
|
||||
self.script = str(Path(__file__).resolve().parents[2]
|
||||
/ "chv-guest" / "build.sh")
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def _run(self, args, env_extra=None):
|
||||
env = dict(os.environ)
|
||||
env.update({"CODEX_BIN_DIR": self.bin_dir,
|
||||
"UBUNTU_VERSION": "24.04",
|
||||
"CODEX_VERSION": "0.164.0-alpha.9",
|
||||
"REPO_COMMIT": "test123",
|
||||
"REPO_DIR": self.repo})
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
return subprocess.run(["bash", self.script] + args,
|
||||
capture_output=True, text=True, env=env)
|
||||
|
||||
def test_dry_run_requires_sha_and_prints_sshd_hardening(self):
|
||||
out_dir = os.path.join(self.dir, "out")
|
||||
r = self._run(["--dry-run", out_dir])
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
self.assertIn("UBUNTU_IMAGE_SHA256", r.stderr)
|
||||
r2 = self._run(["--dry-run", out_dir],
|
||||
env_extra={"UBUNTU_IMAGE_SHA256": "0" * 64})
|
||||
self.assertEqual(r2.returncode, 0, r2.stderr)
|
||||
self.assertIn("99-chv.conf", r2.stdout)
|
||||
self.assertIn("PasswordAuthentication no", r2.stdout)
|
||||
self.assertIn("sshd -t", r2.stdout)
|
||||
self.assertIn("virt-cat", r2.stdout)
|
||||
|
||||
|
||||
# -- M5 审计补齐 ------------------------------------------------------------------
|
||||
class AuditCoverageTest(SecfixCase):
|
||||
def test_host_key_mismatch_audited_and_fail_fast(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_HOST_KEY_MODE": "keyscan"})
|
||||
kh = ex.known_hosts_path({"id": "m1"})
|
||||
os.makedirs(os.path.dirname(kh), exist_ok=True)
|
||||
with open(kh, "w") as fh:
|
||||
fh.write("[127.0.0.1]:2222 ssh-ed25519 AAAAfake\n")
|
||||
|
||||
def bad_ssh(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh":
|
||||
return FakeResult(
|
||||
255, "",
|
||||
"Host key verification failed.\n"
|
||||
"REMOTE HOST IDENTIFICATION HAS CHANGED\n")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex.runner = bad_ssh
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wait_guest_ready({"id": "m1"})
|
||||
self.assertEqual(ctx.exception.kind, "host_key_mismatch")
|
||||
self.assertTrue(any(e == "host_key_mismatch"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
def test_preflight_failure_is_audited(self):
|
||||
ex = self.make_exec(binary="/tmp/no-such-chv-binary")
|
||||
with self.assertRaises(ChvFailure):
|
||||
ex.preflight()
|
||||
self.assertTrue(any(e == "preflight_failed"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
def test_audit_callback_failure_does_not_break_task(self):
|
||||
ex = self.make_exec()
|
||||
ex.audit_callback = lambda *args: (_ for _ in ()).throw(
|
||||
RuntimeError("audit store down"))
|
||||
# _emit 只记宿主 error 日志,不抛异常。
|
||||
ex._emit("vm_create", {"id": "t"}, "vm=chv-t")
|
||||
|
||||
def test_capture_success_is_audited(self):
|
||||
ex = self.make_exec(
|
||||
env_extra={"CHV_HOST_KEY_MODE": "keyscan"})
|
||||
|
||||
def keyscan_runner(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd and cmd[0] == "ssh-keyscan":
|
||||
return FakeResult(
|
||||
0, "[127.0.0.1]:2222 ssh-ed25519 AAAAfake1\n", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
ex.runner = keyscan_runner
|
||||
self.assertTrue(ex.capture_host_key({"id": "cap1"}))
|
||||
self.assertTrue(any(e == "host_key_captured"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
|
||||
# -- M6 reuse 租户声明 ---------------------------------------------------------------
|
||||
class ReuseTenantTest(SecfixCase):
|
||||
def test_reuse_without_tenant_declaration_refused(self):
|
||||
ex = self.make_exec(env_extra={
|
||||
"CHV_DISK_MODE": "reuse",
|
||||
"CHV_REUSE_SINGLE_TENANT_EXPERIMENT": "1"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "reuse_tenant_undeclared")
|
||||
|
||||
def test_reuse_with_declaration_passes_preflight(self):
|
||||
ex = self.make_exec(env_extra={
|
||||
"CHV_DISK_MODE": "reuse",
|
||||
"CHV_REUSE_SINGLE_TENANT_EXPERIMENT": "1",
|
||||
"CHV_SINGLE_TENANT_PROJECT": "p1"})
|
||||
self.assertTrue(ex.preflight())
|
||||
|
||||
|
||||
# -- L1/L2/L3/L4 -----------------------------------------------------------------------
|
||||
class LowItemsTest(SecfixCase):
|
||||
def test_task_disk_symlink_not_followed(self):
|
||||
target = os.path.join(self.dir, "victim.bin")
|
||||
with open(target, "wb") as fh:
|
||||
fh.write(b"VICTIM")
|
||||
vm_dir = os.path.join(self.state_dir, "chv-l1")
|
||||
os.makedirs(vm_dir, exist_ok=True)
|
||||
os.symlink(target, os.path.join(vm_dir, "disk.raw"))
|
||||
disk = provision_task_disk(self.state_dir, "chv-l1",
|
||||
self.rootfs, "copy")
|
||||
self.assertFalse(os.path.islink(disk))
|
||||
with open(disk, "rb") as fh:
|
||||
self.assertEqual(fh.read(), b"R" * 100)
|
||||
with open(target, "rb") as fh:
|
||||
self.assertEqual(fh.read(), b"VICTIM")
|
||||
|
||||
def test_snapshot_url_outside_allowed_dir_refused(self):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
validate_snapshot_url("file:///etc/passwd",
|
||||
self.state_dir)
|
||||
self.assertEqual(ctx.exception.kind,
|
||||
"snapshot_path_not_allowed")
|
||||
|
||||
def test_remote_snapshot_requires_explicit_flag(self):
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
validate_snapshot_url("s3://bucket/snap",
|
||||
self.state_dir)
|
||||
self.assertEqual(ctx.exception.kind,
|
||||
"snapshot_remote_not_allowed")
|
||||
self.assertIsNone(validate_snapshot_url(
|
||||
"s3://bucket/snap", self.state_dir, allow_remote=True))
|
||||
|
||||
def test_capacity_failure_is_audited(self):
|
||||
ex = self.make_exec(env_extra={"CHV_MAX_VMS": "0"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm({"id": "cap"})
|
||||
self.assertEqual(ctx.exception.kind, "host_capacity_exceeded")
|
||||
self.assertTrue(any(e == "vm_failed" and k == "host_capacity_exceeded"
|
||||
for e, _, _, k in self.audit))
|
||||
|
||||
def test_dual_net_vars_warn_but_pass(self):
|
||||
ex = self.make_exec(env_extra={
|
||||
"CHV_NET": "tap=tap0",
|
||||
"CHV_NET_JSON": '[{"tap": "tap0"}]'})
|
||||
self.assertTrue(ex.preflight())
|
||||
self.assertTrue(any(e == "net_config_dual"
|
||||
for e, _, _, _ in self.audit))
|
||||
|
||||
def test_warm_pool_acquire_doc_defines_task_disk_policy(self):
|
||||
# L5 是文档项(接线仍未做):acquire 文档必须写清
|
||||
# 预热盘与任务盘的对应关系,不假装已接。
|
||||
self.assertIn("provision_task_disk", WarmPool.acquire.__doc__)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,388 @@
|
||||
"""to10 轨新增测试:覆盖率缺口分支(非刷行)。
|
||||
|
||||
只测此前 coverage 报告里未覆盖的分支:api_client 的假
|
||||
socket 错误路径、hostkeys 的 keygen/seed 失败路径、
|
||||
orphans 的 registry 失败路径、keyscan 过滤、compat 的
|
||||
fail-closed 分支、config 的类型转换与 assemble 冲突。
|
||||
不删旧测试、不改分母;每项都是行为断言,不是 grep。
|
||||
"""
|
||||
|
||||
import os
|
||||
import socket
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from cloud.chv import api_client
|
||||
from cloud.chv import compat215
|
||||
from cloud.chv import config as chv_config
|
||||
from cloud.chv import hostkeys
|
||||
from cloud.chv import orphans
|
||||
from cloud.chv import ssh_transport
|
||||
|
||||
|
||||
class _FakeSocket:
|
||||
"""按 recv 调用序列供数据的假 socket。"""
|
||||
|
||||
def __init__(self, chunks, timeout_at=None):
|
||||
self._chunks = list(chunks)
|
||||
self._timeout_at = timeout_at
|
||||
self._calls = 0
|
||||
self.closed = False
|
||||
|
||||
def recv(self, _size):
|
||||
self._calls += 1
|
||||
if self._timeout_at is not None \
|
||||
and self._calls >= self._timeout_at:
|
||||
raise socket.timeout("fake timeout")
|
||||
if not self._chunks:
|
||||
return b""
|
||||
return self._chunks.pop(0)
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
|
||||
|
||||
class ApiClientTest(unittest.TestCase):
|
||||
def test_header_timeout_raises(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._read_header(_FakeSocket([], timeout_at=1))
|
||||
|
||||
def test_header_early_close_raises(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._read_header(_FakeSocket([]))
|
||||
|
||||
def test_chunked_rejected(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._parse_content_length(
|
||||
"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked")
|
||||
|
||||
def test_no_length_rejected(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._parse_content_length("HTTP/1.1 200 OK")
|
||||
|
||||
def test_illegal_length_rejected(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._parse_content_length(
|
||||
"HTTP/1.1 200 OK\r\nContent-Length: xx")
|
||||
|
||||
def test_body_timeout_truncation(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._read_body(_FakeSocket([], timeout_at=1),
|
||||
b"ab", 10)
|
||||
|
||||
def test_body_early_close_truncation(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._read_body(_FakeSocket([]), b"ab", 10)
|
||||
|
||||
def test_parse_non_http_status_zero(self):
|
||||
status, _body = api_client._parse_response("garbage", b"{}")
|
||||
self.assertEqual(status, 0)
|
||||
|
||||
def test_parse_non_json_body_is_text(self):
|
||||
_status, body = api_client._parse_response(
|
||||
"HTTP/1.1 200 OK", b"not-json")
|
||||
self.assertEqual(body, "not-json")
|
||||
|
||||
def test_parse_empty_body_is_empty_dict(self):
|
||||
_status, body = api_client._parse_response(
|
||||
"HTTP/1.1 200 OK", b"")
|
||||
self.assertEqual(body, {})
|
||||
|
||||
def test_unix_api_missing_socket_raises(self):
|
||||
with self.assertRaises(RuntimeError):
|
||||
api_client._unix_socket_api("/tmp/no-such-chv.sock",
|
||||
"GET", "/vm.info")
|
||||
|
||||
|
||||
class HostkeysTest(unittest.TestCase):
|
||||
def test_keygen_reuse_existing_pair(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
paths = hostkeys.host_key_paths(tmp, "vm1")
|
||||
os.makedirs(os.path.dirname(paths["private"]),
|
||||
exist_ok=True)
|
||||
for key in ("private", "public"):
|
||||
with open(paths[key], "w") as fh:
|
||||
fh.write("x")
|
||||
out = hostkeys.generate_host_keypair(
|
||||
tmp, "vm1", runner=lambda cmd: (_ for _ in ()).throw(
|
||||
AssertionError("reuse 不应调 keygen")))
|
||||
self.assertEqual(out, paths)
|
||||
|
||||
def test_keygen_nonzero_rc_raises(self):
|
||||
class _Res:
|
||||
returncode = 1
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
with self.assertRaises(RuntimeError):
|
||||
hostkeys.generate_host_keypair(
|
||||
tmp, "vm2", runner=lambda cmd: _Res())
|
||||
|
||||
def test_keygen_no_files_after_runner_raises(self):
|
||||
class _Res:
|
||||
returncode = 0
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
with self.assertRaises(RuntimeError):
|
||||
hostkeys.generate_host_keypair(
|
||||
tmp, "vm3", runner=lambda cmd: _Res())
|
||||
|
||||
def test_seed_disk_nonzero_rc_raises(self):
|
||||
class _Res:
|
||||
returncode = 1
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
with self.assertRaises(RuntimeError):
|
||||
hostkeys.build_seed_disk(
|
||||
tmp, os.path.join(tmp, "seed.iso"),
|
||||
runner=lambda cmd: _Res())
|
||||
|
||||
def test_seed_disk_missing_output_raises(self):
|
||||
class _Res:
|
||||
returncode = 0
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
with self.assertRaises(RuntimeError):
|
||||
hostkeys.build_seed_disk(
|
||||
tmp, os.path.join(tmp, "seed.iso"),
|
||||
runner=lambda cmd: _Res())
|
||||
|
||||
|
||||
class OrphansAndKeyscanTest(unittest.TestCase):
|
||||
def test_quarantine_failure_only_warns(self):
|
||||
with self.assertLogs("cloud.cloud_hypervisor",
|
||||
level="WARNING") as logs:
|
||||
orphans._quarantine_registry("/tmp/no-such-dir-x/f.json")
|
||||
self.assertTrue(any("quarantine_failed" in line
|
||||
for line in logs.output))
|
||||
|
||||
def test_write_registry_failure_only_warns(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
blocker = os.path.join(tmp, "blocker")
|
||||
with open(blocker, "w") as fh:
|
||||
fh.write("x")
|
||||
with self.assertLogs("cloud.cloud_hypervisor",
|
||||
level="WARNING") as logs:
|
||||
orphans._write_registry(
|
||||
os.path.join(blocker, "registry.json"), {})
|
||||
self.assertTrue(any("save_failed" in line
|
||||
for line in logs.output))
|
||||
|
||||
def test_keyscan_filters_short_and_non_ssh_lines(self):
|
||||
text = "\n".join([
|
||||
"# comment",
|
||||
"",
|
||||
"only-two",
|
||||
"127.0.0.1 ecdsa-sha2-nistp256 AAAA",
|
||||
"[127.0.0.1]:2222 ssh-ed25519 AAAA-good",
|
||||
"other-host ssh-ed25519 AAAA-other",
|
||||
])
|
||||
kept = ssh_transport.parse_keyscan_output(text)
|
||||
self.assertEqual(kept, ["[127.0.0.1]:2222 ssh-ed25519 AAAA-good"])
|
||||
|
||||
|
||||
class CompatAndConfigTest(unittest.TestCase):
|
||||
def test_tmpfs_unknown_executor_fail_closed(self):
|
||||
isolated, _reason = compat215.tmpfs_isolated("unknown-kind",
|
||||
"/tmp")
|
||||
self.assertFalse(isolated)
|
||||
|
||||
def test_resolve_inside_empty_rel_is_base(self):
|
||||
self.assertEqual(compat215.resolve_inside("/a/b", ""), "/a/b")
|
||||
|
||||
def test_resolve_inside_absolute_is_none(self):
|
||||
self.assertIsNone(compat215.resolve_inside("/a/b", "/etc/passwd"))
|
||||
|
||||
def test_to_int_bool_and_float_and_bad(self):
|
||||
with self.assertRaises(ValueError):
|
||||
chv_config._to_int(True)
|
||||
self.assertEqual(chv_config._to_int(3.9), 3)
|
||||
self.assertEqual(chv_config._to_int(" 7 "), 7)
|
||||
with self.assertRaises(ValueError):
|
||||
chv_config._to_int(object())
|
||||
|
||||
def test_to_float_bool_and_bad(self):
|
||||
with self.assertRaises(ValueError):
|
||||
chv_config._to_float(False)
|
||||
self.assertEqual(chv_config._to_float("2.5"), 2.5)
|
||||
with self.assertRaises(ValueError):
|
||||
chv_config._to_float(object())
|
||||
|
||||
def test_assemble_conflict_raises(self):
|
||||
with self.assertRaises(ValueError):
|
||||
chv_config._assemble_chv_config(
|
||||
[{"cpus": "2"}, {"cpus": "4"}])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
class WorkerAdapterTest(unittest.TestCase):
|
||||
"""to10:worker_adapter 的直接行为测试(worker 薄包装
|
||||
由旧测试 test_chv_fix/test_chv_gaps 覆盖,此处测本体)。"""
|
||||
|
||||
def test_call_breakpoint_prefers_signature_kwargs(self):
|
||||
from cloud.chv import worker_adapter
|
||||
seen = {}
|
||||
|
||||
class _Ex:
|
||||
def goal_resume_breakpoint_task(self, task,
|
||||
model_provider=None,
|
||||
extra_env=None):
|
||||
seen.update(task=task, model_provider=model_provider,
|
||||
extra_env=extra_env)
|
||||
return "ok"
|
||||
out = worker_adapter.call_goal_resume_breakpoint(
|
||||
_Ex(), {"id": "t"}, model_provider="m",
|
||||
extra_env={"A": "1"})
|
||||
self.assertEqual(out, "ok")
|
||||
self.assertEqual(seen["model_provider"], "m")
|
||||
self.assertEqual(seen["extra_env"], {"A": "1"})
|
||||
|
||||
def test_call_breakpoint_single_arg_executor(self):
|
||||
from cloud.chv import worker_adapter
|
||||
|
||||
class _Ex:
|
||||
def goal_resume_breakpoint_task(self, task):
|
||||
return "single"
|
||||
self.assertEqual(
|
||||
worker_adapter.call_goal_resume_breakpoint(_Ex(), {"id": "t"}),
|
||||
"single")
|
||||
|
||||
def test_audit_callback_extracts_project_and_swallows(self):
|
||||
from cloud.chv import worker_adapter
|
||||
calls = []
|
||||
cb = worker_adapter.make_chv_audit_callback(calls.append)
|
||||
# append 只接受 1 参,回调内部 except 应吞掉,不炸
|
||||
cb("booted", {"project_id": "p1"}, "detail")
|
||||
self.assertEqual(calls, [])
|
||||
got = []
|
||||
cb2 = worker_adapter.make_chv_audit_callback(
|
||||
lambda pid, ev, detail: got.append((pid, ev, detail)))
|
||||
cb2("booted", {"project_id": "p1"}, "detail")
|
||||
cb2("booted", "not-a-dict", "detail")
|
||||
self.assertEqual(got[0], ("p1", "chv_vm_booted", "detail"))
|
||||
self.assertEqual(got[1], (None, "chv_vm_booted", "detail"))
|
||||
|
||||
def test_reconcile_non_chv_returns_empty(self):
|
||||
from cloud.chv import worker_adapter
|
||||
self.assertEqual(
|
||||
worker_adapter.reconcile_warm_pool(object(), lambda *a: None,
|
||||
"startup"),
|
||||
[])
|
||||
|
||||
def test_watchdog_vm_name_error_returns_none(self):
|
||||
from cloud.chv import worker_adapter
|
||||
|
||||
class _Bad:
|
||||
def vm_name(self, task):
|
||||
raise RuntimeError("boom")
|
||||
self.assertIsNone(worker_adapter.watchdog_vm_name(_Bad(), {}))
|
||||
|
||||
def test_cancel_watchdog_loop_kills_on_flag(self):
|
||||
import threading
|
||||
from cloud.chv import worker_adapter
|
||||
killed = []
|
||||
stop = threading.Event()
|
||||
worker_adapter.cancel_watchdog_loop(
|
||||
"t1", "chv-t1", stop, {"t1"},
|
||||
lambda tid: None,
|
||||
lambda vm: killed.append(vm),
|
||||
None)
|
||||
# stop 未置位时循环每 1 秒轮询:flag 已命中,第一轮即
|
||||
# 关 VM 并返回;这里不依赖日志(logger 传 None 时
|
||||
# log_event 内部不炸——若炸则本项失败)。
|
||||
self.assertEqual(killed, ["chv-t1"])
|
||||
|
||||
def test_quota_watchdog_loop_marks_and_kills(self):
|
||||
import threading
|
||||
from cloud.chv import worker_adapter
|
||||
killed = []
|
||||
marked = []
|
||||
stop = threading.Event()
|
||||
worker_adapter.quota_watchdog_loop(
|
||||
"t2", "chv-t2", stop, lambda: True,
|
||||
marked.append,
|
||||
lambda vm: killed.append(vm),
|
||||
None, interval=0.01)
|
||||
self.assertEqual(marked, ["t2"])
|
||||
self.assertEqual(killed, ["chv-t2"])
|
||||
|
||||
|
||||
class AuditSpillTest(unittest.TestCase):
|
||||
"""to10:_emit 回调失败时落盘(ADR-14),不中断、不抛。"""
|
||||
|
||||
def test_spill_on_callback_failure(self):
|
||||
import json as _json
|
||||
from cloud.chv.executor import CloudHypervisorExecutor
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
ex = CloudHypervisorExecutor(
|
||||
binary="/bin/true", kernel="/tmp/k", rootfs="/tmp/r",
|
||||
state_dir=tmp, ssh_key="/tmp/key")
|
||||
calls = []
|
||||
|
||||
def _bad(event, task, detail, kind=None):
|
||||
raise RuntimeError("db down")
|
||||
ex.audit_callback = _bad
|
||||
ex._emit("booted", {"id": "t-spill"}, "vm=chv-t-spill")
|
||||
self.assertEqual(ex.audit_failure_count, 1)
|
||||
spill = os.path.join(tmp, "chv-audit-spill.jsonl")
|
||||
self.assertTrue(os.path.exists(spill))
|
||||
row = _json.loads(open(spill).readline())
|
||||
self.assertEqual(row["event"], "booted")
|
||||
self.assertEqual(row["task_id"], "t-spill")
|
||||
self.assertEqual(row["detail"], "vm=chv-t-spill")
|
||||
|
||||
def test_spill_rotation_keeps_old(self):
|
||||
from cloud.chv.executor import CloudHypervisorExecutor
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
ex = CloudHypervisorExecutor(
|
||||
binary="/bin/true", kernel="/tmp/k", rootfs="/tmp/r",
|
||||
state_dir=tmp, ssh_key="/tmp/key")
|
||||
spill = os.path.join(tmp, "chv-audit-spill.jsonl")
|
||||
with open(spill, "w") as fh:
|
||||
fh.write("x" * (1024 * 1024 + 10))
|
||||
ex._spill_audit("booted", {"id": "t2"}, "d", None)
|
||||
self.assertTrue(os.path.exists(spill + ".old"))
|
||||
self.assertTrue(os.path.getsize(spill) < 1024)
|
||||
|
||||
|
||||
class LockfileScriptTest(unittest.TestCase):
|
||||
"""to10:generate_lockfile.py 的 generate/verify(合成
|
||||
BUILDINFO,不碰真实客机清单,不编版本号)。"""
|
||||
|
||||
def _load_script(self):
|
||||
import importlib.util
|
||||
path = os.path.join(
|
||||
os.path.dirname(os.path.dirname(__file__)),
|
||||
"..", "chv-guest", "generate_lockfile.py")
|
||||
path = os.path.normpath(path)
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"chv_generate_lockfile", path)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
def test_generate_reads_real_dpkg_version(self):
|
||||
import json as _json
|
||||
mod = self._load_script()
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
bi = os.path.join(tmp, "bi.json")
|
||||
out = os.path.join(tmp, "lock.json")
|
||||
with open(bi, "w") as fh:
|
||||
_json.dump({"tag": "t", "contents": ["coreutils"]},
|
||||
fh)
|
||||
self.assertEqual(mod.generate(bi, out), 0)
|
||||
lock = _json.load(open(out))
|
||||
self.assertEqual(lock["tag"], "t")
|
||||
self.assertEqual(lock["unresolved"], [])
|
||||
self.assertTrue(lock["versions"]["dpkg"][0].startswith(
|
||||
"coreutils "))
|
||||
|
||||
def test_verify_fails_on_unresolved(self):
|
||||
import json as _json
|
||||
mod = self._load_script()
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
lock = os.path.join(tmp, "lock.json")
|
||||
with open(lock, "w") as fh:
|
||||
_json.dump({"tag": "t",
|
||||
"versions": {"dpkg": ["a 1.0"],
|
||||
"pip": ["b 2.0"]},
|
||||
"unresolved": ["zzz"]}, fh)
|
||||
self.assertEqual(mod.verify(lock, None), 1)
|
||||
@@ -0,0 +1,700 @@
|
||||
"""Cloud Hypervisor 执行器 v1 + 生产级唤醒/生命周期测试。
|
||||
|
||||
只用标准库与 mock runner/launcher/api_client,绝不启动真实
|
||||
VM:当前环境没有可跑真 VM 的宿主(生产主机是 CVM、无
|
||||
/dev/kvm、无嵌套虚拟化;本 VM 容器内同样无 /dev/kvm),
|
||||
所以本文件验证的是:preflight 分类、API 唤醒顺序、VmConfig、
|
||||
SSH 就绪、暂停/快照/休眠/关闭/清理、预热池、单任务单 VM
|
||||
隔离、容量门、磁盘门、worker/server/autoscaler 接线。
|
||||
真实 VM 启动、快照恢复、压测一律不在本文件声称已验证。
|
||||
"""
|
||||
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
try:
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
_parse_memory_bytes,
|
||||
)
|
||||
except ImportError: # discover -s cloud/tests 时把项目根目录补进 sys.path
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.chv.experimental_pool import WarmPool
|
||||
from cloud.sandbox import (
|
||||
ChvFailure,
|
||||
CloudHypervisorExecutor,
|
||||
_parse_memory_bytes,
|
||||
)
|
||||
|
||||
|
||||
class FakeResult:
|
||||
def __init__(self, returncode=0, stdout="", stderr=""):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
class FakeProc:
|
||||
"""假 VMM 进程:poll() 返回 None=活着,数字=已死。"""
|
||||
|
||||
def __init__(self, alive=True):
|
||||
self._alive = alive
|
||||
self.terminated = False
|
||||
self.killed = False
|
||||
|
||||
def poll(self):
|
||||
return None if self._alive else 1
|
||||
|
||||
def terminate(self):
|
||||
self.terminated = True
|
||||
self._alive = False
|
||||
|
||||
def wait(self, timeout=None):
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self.killed = True
|
||||
self._alive = False
|
||||
|
||||
|
||||
def _write(path, content=b"x"):
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(content)
|
||||
os.chmod(path, 0o755)
|
||||
return path
|
||||
|
||||
|
||||
class ChvCase(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.dir = self.tmp.name
|
||||
self.binary = _write(os.path.join(self.dir, "cloud-hypervisor"))
|
||||
self.ch_remote = _write(os.path.join(self.dir, "ch-remote"))
|
||||
self.kernel = _write(os.path.join(self.dir, "vmlinux"))
|
||||
self.rootfs = _write(os.path.join(self.dir, "disk.raw"),
|
||||
b"R" * 100)
|
||||
self.ssh_key = _write(os.path.join(self.dir, "id_chv"))
|
||||
self.kvm = _write(os.path.join(self.dir, "kvm"))
|
||||
self.cpuinfo = os.path.join(self.dir, "cpuinfo")
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu vmx sse\n")
|
||||
self.state_dir = os.path.join(self.dir, "state")
|
||||
self.calls = [] # runner 调用记录
|
||||
self.api_calls = [] # api_client 调用记录
|
||||
self.launches = [] # launcher 调用记录
|
||||
self.audit = [] # audit 事件记录
|
||||
|
||||
def tearDown(self):
|
||||
self.tmp.cleanup()
|
||||
|
||||
def make_exec(self, runner=None, api_client=None, launcher=None,
|
||||
env_extra=None, **overrides):
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
# 实验室宣称:mock runner 假装回环端口已转发到客机
|
||||
#(feat-chv-fix 起,执行器不再无条件假设回环能通)。
|
||||
# 实验室显式 opt-in keyscan:生产默认已是
|
||||
# provisioned;本套件旧 fake 不实现 ssh-keygen/
|
||||
# genisoimage,测的不是 host key 模式,故显式
|
||||
# 声明实验室路径,绝不把它写成生产默认。
|
||||
"CHV_HOST_KEY_MODE": "keyscan",
|
||||
"CHV_SSH_FORWARD": "1"}
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
if runner is None:
|
||||
runner = self._default_runner
|
||||
if api_client is None:
|
||||
api_client = self._default_api
|
||||
if launcher is None:
|
||||
launcher = self._default_launcher
|
||||
kwargs = dict(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, rootfs=self.rootfs,
|
||||
ssh_key=self.ssh_key, kvm_path=self.kvm,
|
||||
cpuinfo_path=self.cpuinfo, state_dir=self.state_dir,
|
||||
runner=runner, launcher=launcher, api_client=api_client,
|
||||
sleeper=lambda s: None, env=env)
|
||||
kwargs.update(overrides)
|
||||
ex = CloudHypervisorExecutor(**kwargs)
|
||||
ex.audit_callback = lambda e, t, d, k=None: self.audit.append(
|
||||
(e, t.get("id") if isinstance(t, dict) else t, d, k))
|
||||
return ex
|
||||
|
||||
def _default_runner(self, cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh":
|
||||
if cmd[-1] == "true":
|
||||
return FakeResult(0, "", "")
|
||||
if "dispatch" in cmd:
|
||||
return FakeResult(0, json.dumps(
|
||||
{"status": "completed", "text": "ok-dispatch"}), "")
|
||||
if "goal" in cmd or "goal-resume" in cmd:
|
||||
return FakeResult(0, "goal met after rounds\n", "")
|
||||
if "status" in cmd:
|
||||
return FakeResult(0, json.dumps(
|
||||
{"status": "completed", "text": "ok-status"}), "")
|
||||
return FakeResult(0, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
|
||||
def _default_api(self, socket_path, method, path, payload=None):
|
||||
self.api_calls.append((method, path, payload))
|
||||
return (200, {})
|
||||
|
||||
def _default_launcher(self, cmd):
|
||||
self.launches.append(list(cmd))
|
||||
return FakeProc(alive=True)
|
||||
|
||||
def _task(self, task_id="abc123"):
|
||||
return {"id": task_id, "prompt": "做一个加法函数",
|
||||
"project_id": "proj1"}
|
||||
|
||||
|
||||
# -- preflight ---------------------------------------------------------
|
||||
class PreflightTest(ChvCase):
|
||||
def test_all_present_passes(self):
|
||||
self.assertTrue(self.make_exec().preflight())
|
||||
|
||||
def test_missing_binary_reports_it(self):
|
||||
ex = self.make_exec(binary="/tmp/no-such-chv-binary")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "binary_missing")
|
||||
self.assertIn("cloud-hypervisor", str(ctx.exception))
|
||||
|
||||
def test_missing_ch_remote_reports_it(self):
|
||||
ex = self.make_exec(ch_remote="/tmp/no-such-ch-remote")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "binary_missing")
|
||||
self.assertIn("ch-remote", str(ctx.exception))
|
||||
|
||||
def test_missing_kvm_reports_it(self):
|
||||
ex = self.make_exec(kvm_path="/tmp/no-such-kvm")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "kvm_missing")
|
||||
self.assertIn("/tmp/no-such-kvm", str(ctx.exception))
|
||||
|
||||
def test_missing_cpu_flag_reports_it(self):
|
||||
with open(self.cpuinfo, "w") as fh:
|
||||
fh.write("flags : fpu sse\n")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
self.make_exec().preflight()
|
||||
self.assertEqual(ctx.exception.kind, "cpu_flag_missing")
|
||||
self.assertIn("vmx", str(ctx.exception))
|
||||
|
||||
def test_missing_kernel_reports_it(self):
|
||||
ex = self.make_exec(kernel="/tmp/no-such-vmlinux")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "kernel_missing")
|
||||
|
||||
def test_missing_image_reports_it(self):
|
||||
ex = self.make_exec(rootfs="/tmp/no-such-disk.raw")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "image_missing")
|
||||
|
||||
def test_missing_ssh_key_reports_it(self):
|
||||
ex = self.make_exec(ssh_key="/tmp/no-such-key")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "ssh_key_missing")
|
||||
|
||||
def test_missing_egress_is_fail_closed(self):
|
||||
ex = self.make_exec(env_extra={"CHV_EGRESS_PROXY": ""})
|
||||
# env_extra 空值会被 __init__ 的 or 链忽略,故改用显式 env:
|
||||
ex.egress_proxy = None
|
||||
ex.allow_direct_egress = False
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
self.assertEqual(ctx.exception.kind, "egress_not_configured")
|
||||
|
||||
def test_multiple_missing_lists_all(self):
|
||||
ex = self.make_exec(binary="/tmp/no-chv", kernel="/tmp/no-kern")
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.preflight()
|
||||
text = str(ctx.exception)
|
||||
self.assertIn("cloud-hypervisor", text)
|
||||
self.assertIn("内核", text)
|
||||
|
||||
def test_dispatch_missing_dependency_does_not_call_runner(self):
|
||||
ex = self.make_exec(binary="/tmp/no-chv")
|
||||
with self.assertRaises(ChvFailure):
|
||||
ex.dispatch_task(self._task())
|
||||
self.assertEqual(self.calls, [])
|
||||
|
||||
|
||||
# -- 配置与命令构造 -----------------------------------------------------
|
||||
class ConfigCommandTest(ChvCase):
|
||||
def test_vm_config_shape_and_memory_bytes(self):
|
||||
ex = self.make_exec()
|
||||
# feat-chv-secfix(L2):磁盘须在 state_dir 内。
|
||||
disk = os.path.join(self.state_dir, "x", "disk.raw")
|
||||
cfg = ex.build_vm_config(disk)
|
||||
self.assertEqual(cfg["cpus"], {"boot_vcpus": 2,
|
||||
"max_vcpus": 2})
|
||||
self.assertEqual(cfg["memory"]["size"], 2 * 1024 ** 3)
|
||||
self.assertEqual(cfg["kernel"]["path"], self.kernel)
|
||||
self.assertEqual(cfg["disks"], [{"path": disk}])
|
||||
self.assertEqual(cfg["cmdline"]["args"],
|
||||
"console=hvc0 root=/dev/vda1 rw")
|
||||
|
||||
def test_memory_parsing(self):
|
||||
self.assertEqual(_parse_memory_bytes("512M"), 512 * 1024 ** 2)
|
||||
self.assertEqual(_parse_memory_bytes("1024"), 1024)
|
||||
self.assertEqual(_parse_memory_bytes("1K"), 1024)
|
||||
with self.assertRaises(ValueError):
|
||||
_parse_memory_bytes("abc")
|
||||
|
||||
def test_rootfs_image_alias(self):
|
||||
ex = self.make_exec(rootfs=None, env_extra={})
|
||||
# rootfs=None 时 __init__ 仍从 env 取不到(env 无 CHV_*),
|
||||
# 故用 env 含 CHV_IMAGE 的构造验证别名:
|
||||
env = {"CHV_EGRESS_PROXY": "http://10.0.0.1:8888",
|
||||
"CHV_IMAGE": self.rootfs}
|
||||
ex2 = CloudHypervisorExecutor(
|
||||
binary=self.binary, ch_remote=self.ch_remote,
|
||||
kernel=self.kernel, ssh_key=self.ssh_key,
|
||||
kvm_path=self.kvm, cpuinfo_path=self.cpuinfo,
|
||||
state_dir=self.state_dir, env=env,
|
||||
sleeper=lambda s: None)
|
||||
self.assertEqual(ex2.rootfs, self.rootfs)
|
||||
|
||||
def test_manual_vm_command_shape(self):
|
||||
ex = self.make_exec()
|
||||
self.assertFalse(hasattr(ex, "build_vm_command"))
|
||||
cmd = ex._build_vm_command(self._task())
|
||||
self.assertEqual(cmd[0], self.binary)
|
||||
self.assertIn("--api-socket", cmd)
|
||||
self.assertIn("--cpus", cmd)
|
||||
self.assertIn("boot=2", cmd)
|
||||
self.assertIn("size=2G", cmd)
|
||||
self.assertIn("--kernel", cmd)
|
||||
self.assertIn("--disk", cmd)
|
||||
|
||||
def test_ssh_command_has_key_path_not_secret_value(self):
|
||||
ex = self.make_exec()
|
||||
cmd = ex.build_ssh_command(self._task(), ["true"])
|
||||
joined = " ".join(cmd)
|
||||
self.assertIn(self.ssh_key, joined)
|
||||
self.assertIn("BatchMode=yes", joined)
|
||||
self.assertNotIn("SECRET", joined)
|
||||
# egress 代理地址在 env 前缀里(不是密钥),且只在代理已设时出现:
|
||||
self.assertIn("HTTPS_PROXY=http://10.0.0.1:8888", joined)
|
||||
# 本轨安全默认变化(#216 的 no + /dev/null 已废除):
|
||||
# host key 钉死在每 VM 的 known_hosts 文件上:
|
||||
self.assertIn("StrictHostKeyChecking=yes", joined)
|
||||
self.assertNotIn("StrictHostKeyChecking=no", joined)
|
||||
self.assertNotIn("/dev/null", joined)
|
||||
self.assertIn(os.path.join(self.state_dir,
|
||||
"chv-abc123", "known_hosts"),
|
||||
joined)
|
||||
|
||||
def test_isolation_names_and_disks_differ(self):
|
||||
ex = self.make_exec()
|
||||
a, b = {"id": "task-a"}, {"id": "task-b"}
|
||||
self.assertNotEqual(ex.vm_name(a), ex.vm_name(b))
|
||||
self.assertNotEqual(ex.api_socket(a), ex.api_socket(b))
|
||||
self.assertNotEqual(ex._provision_task_disk(a),
|
||||
ex._provision_task_disk(b))
|
||||
# copy 模式:每任务磁盘是基础镜像的真实副本,内容一致、
|
||||
# 路径不同(两个任务绝不写同一文件):
|
||||
with open(ex._provision_task_disk(a), "rb") as fh:
|
||||
self.assertEqual(fh.read(), b"R" * 100)
|
||||
|
||||
def test_disk_quota_gate(self):
|
||||
ex = self.make_exec(env_extra={"CHV_DISK_GB": "0.00000001"})
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm(self._task())
|
||||
self.assertEqual(ctx.exception.kind, "disk_quota_exceeded")
|
||||
|
||||
def test_capacity_gate_rejects_not_fallback(self):
|
||||
ex = self.make_exec(env_extra={"CHV_MAX_VMS": "1"})
|
||||
ex._vms["chv-other"] = FakeProc(alive=True)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm(self._task())
|
||||
self.assertEqual(ctx.exception.kind, "host_capacity_exceeded")
|
||||
self.assertEqual(self.launches, [])
|
||||
|
||||
|
||||
# -- API 唤醒 -----------------------------------------------------------
|
||||
class WakeApiTest(ChvCase):
|
||||
def test_wake_uses_api_mode_not_direct_command(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.wake_vm(self._task())
|
||||
self.assertEqual(out["state"], "running")
|
||||
self.assertTrue(out["ready"])
|
||||
self.assertEqual(out["mode"], "cold")
|
||||
# launcher 只带 --api-socket,不带内核/磁盘(API 模式证明):
|
||||
self.assertEqual(len(self.launches), 1)
|
||||
launch = self.launches[0]
|
||||
self.assertIn("--api-socket", launch)
|
||||
self.assertNotIn("--kernel", launch)
|
||||
self.assertNotIn("--disk", launch)
|
||||
# API 调用顺序:ping → create → boot:
|
||||
kinds = [(m, p) for m, p, _ in self.api_calls]
|
||||
self.assertEqual(kinds[0], ("GET", "/vmm.ping"))
|
||||
self.assertEqual(kinds[1], ("PUT", "/vm.create"))
|
||||
self.assertEqual(kinds[2], ("PUT", "/vm.boot"))
|
||||
create_payload = self.api_calls[1][2]
|
||||
self.assertEqual(create_payload["kernel"]["path"], self.kernel)
|
||||
self.assertTrue(create_payload["disks"][0]["path"].endswith(
|
||||
"disk.raw"))
|
||||
events = [e for e, _, _, _ in self.audit]
|
||||
self.assertIn("vm_create", events)
|
||||
self.assertIn("vm_boot", events)
|
||||
self.assertIn("vm_ready", events)
|
||||
|
||||
def test_wake_api_timeout_kills_proc(self):
|
||||
def never_ok(socket_path, method, path, payload=None):
|
||||
self.api_calls.append((method, path, payload))
|
||||
return (500, {})
|
||||
ex = self.make_exec(api_client=never_ok, api_timeout=3)
|
||||
proc_holder = []
|
||||
|
||||
def launcher(cmd):
|
||||
proc = FakeProc(alive=True)
|
||||
proc_holder.append(proc)
|
||||
self.launches.append(list(cmd))
|
||||
return proc
|
||||
ex.launcher = launcher
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm(self._task())
|
||||
self.assertEqual(ctx.exception.kind, "api_timeout")
|
||||
self.assertTrue(proc_holder[0].terminated)
|
||||
self.assertNotIn("chv-abc123", ex._vms)
|
||||
|
||||
def test_guest_ready_timeout_shuts_down(self):
|
||||
def ssh_never(cmd, timeout=None):
|
||||
self.calls.append(list(cmd))
|
||||
if cmd[0] == "ssh":
|
||||
return FakeResult(1, "", "")
|
||||
return FakeResult(0, "", "")
|
||||
ex = self.make_exec(runner=ssh_never, ready_timeout=10,
|
||||
ready_interval=5)
|
||||
with self.assertRaises(ChvFailure) as ctx:
|
||||
ex.wake_vm(self._task())
|
||||
self.assertEqual(ctx.exception.kind, "guest_not_ready")
|
||||
# 超时后关 VM:runner 里有 ch-remote shutdown:
|
||||
self.assertTrue(any("shutdown" in c for c in self.calls))
|
||||
|
||||
|
||||
# -- 暂停 / 快照 / 休眠 / 关闭 / 清理 -----------------------------------
|
||||
class LifecycleTest(ChvCase):
|
||||
def test_pause_resume(self):
|
||||
ex = self.make_exec()
|
||||
self.assertTrue(ex.pause_vm(self._task()))
|
||||
self.assertTrue(ex.resume_vm(self._task()))
|
||||
kinds = [c[-1] for c in self.calls if c[0] == self.ch_remote]
|
||||
self.assertEqual(kinds, ["pause", "resume"])
|
||||
|
||||
def test_restore_then_explicit_resume_order(self):
|
||||
# feat-chv-secfix(L2/H4):本测试走实验室 keyscan
|
||||
# 路径 + 声明快照目录(生产默认 provisioned + 白名单
|
||||
# 目录,旧 file:///snap/1 不再直通)。
|
||||
ex = self.make_exec(env_extra={"CHV_SNAPSHOT_DIR": "/snap"})
|
||||
out = ex.restore_vm(self._task(), "file:///snap/1")
|
||||
self.assertEqual(out["mode"], "restored")
|
||||
self.assertTrue(out["ready"])
|
||||
kinds = [c[-1] if c[-1] != "restore" else c[-1]
|
||||
for c in self.calls if c[0] == self.ch_remote]
|
||||
# ch-remote 调用里 restore 必须在 resume 之前:
|
||||
flat = []
|
||||
for c in self.calls:
|
||||
if c[0] == self.ch_remote:
|
||||
flat.append(c[-1] if c[-1].startswith("source_url")
|
||||
else c[-1])
|
||||
self.assertIn("resume", flat)
|
||||
restore_idx = next(i for i, c in enumerate(self.calls)
|
||||
if c[0] == self.ch_remote
|
||||
and any(a.startswith("source_url")
|
||||
for a in c))
|
||||
resume_idx = next(i for i, c in enumerate(self.calls)
|
||||
if c[0] == self.ch_remote and c[-1] == "resume")
|
||||
self.assertLess(restore_idx, resume_idx)
|
||||
|
||||
def test_hibernate_is_pause_snapshot_shutdown(self):
|
||||
ex = self.make_exec()
|
||||
# feat-chv-fix(P1-7):file:// 快照须真实存在且非空,
|
||||
# 否则 hibernate 不 shutdown;用临时文件作快照目标。
|
||||
# feat-chv-secfix(L2):快照须在白名单目录内,放在
|
||||
# state_dir 下。
|
||||
snap = os.path.join(self.state_dir, "h1.snap")
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(snap, "wb") as fh:
|
||||
fh.write(b"S" * 64)
|
||||
snap_url = "file://" + snap
|
||||
out = ex.hibernate_vm(self._task(), snap_url)
|
||||
self.assertEqual(out["state"], "hibernated")
|
||||
self.assertEqual(out["snapshot"], snap_url)
|
||||
kinds = [c[-1] for c in self.calls if c[0] == self.ch_remote]
|
||||
self.assertEqual(kinds[0], "pause")
|
||||
self.assertTrue(any(a.startswith("source_url")
|
||||
for c in self.calls for a in c))
|
||||
self.assertIn("shutdown", kinds)
|
||||
# 从休眠唤醒 = restore + resume:
|
||||
ex2 = self.make_exec()
|
||||
out2 = ex2.wake_from_hibernate(self._task(), snap_url)
|
||||
self.assertEqual(out2["mode"], "restored")
|
||||
|
||||
def test_stop_dead_proc(self):
|
||||
ex = self.make_exec()
|
||||
proc = FakeProc(alive=False)
|
||||
ex._vms["chv-abc123"] = proc
|
||||
sock = os.path.join(self.state_dir, "chv-abc123.sock")
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(sock, "w") as _fh:
|
||||
_fh.write("")
|
||||
self.assertTrue(ex.stop_vm(self._task()))
|
||||
self.assertFalse(os.path.exists(sock))
|
||||
self.assertNotIn("chv-abc123", ex._vms)
|
||||
|
||||
def test_stop_already_closed(self):
|
||||
ex = self.make_exec()
|
||||
self.assertFalse(ex.stop_vm(self._task()))
|
||||
# 已关仍尽力调过一次 ch-remote shutdown:
|
||||
self.assertTrue(any("shutdown" in c for c in self.calls))
|
||||
|
||||
def test_stop_socket_residual(self):
|
||||
ex = self.make_exec()
|
||||
sock = os.path.join(self.state_dir, "chv-abc123.sock")
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
with open(sock, "w") as _fh:
|
||||
_fh.write("")
|
||||
self.assertTrue(ex.stop_vm(self._task()))
|
||||
self.assertFalse(os.path.exists(sock))
|
||||
|
||||
def test_cleanup_skips_live_and_non_chv(self):
|
||||
ex = self.make_exec()
|
||||
os.makedirs(self.state_dir, exist_ok=True)
|
||||
live = os.path.join(self.state_dir, "chv-live.sock")
|
||||
with open(live, "w") as _fh:
|
||||
_fh.write("")
|
||||
ex._vms["chv-live"] = FakeProc(alive=True)
|
||||
dead = os.path.join(self.state_dir, "chv-dead.sock")
|
||||
with open(dead, "w") as _fh:
|
||||
_fh.write("")
|
||||
other = os.path.join(self.state_dir, "other.sock")
|
||||
with open(other, "w") as _fh:
|
||||
_fh.write("")
|
||||
cleaned = ex.cleanup_stale_vms()
|
||||
self.assertEqual(cleaned, ["chv-dead"])
|
||||
self.assertTrue(os.path.exists(live))
|
||||
self.assertTrue(os.path.exists(other))
|
||||
self.assertFalse(os.path.exists(dead))
|
||||
|
||||
def test_cleanup_missing_state_dir(self):
|
||||
ex = self.make_exec()
|
||||
self.assertEqual(ex.cleanup_stale_vms(), [])
|
||||
|
||||
def test_recycle_preserves_checkpoints_not_temp(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.recycle_vm(self._task())
|
||||
self.assertEqual(out["state"], "recycled")
|
||||
self.assertIn("task_checkpoints", out["preserved"])
|
||||
self.assertIn("task_attempts", out["preserved"])
|
||||
self.assertIn("guest /tmp", out["lost"])
|
||||
|
||||
|
||||
# -- 预热池 --------------------------------------------------------------
|
||||
class WarmPoolTest(ChvCase):
|
||||
def test_prewarm_acquire_release(self):
|
||||
ex = self.make_exec()
|
||||
pool = WarmPool(ex, size=2, snapshot_url="file:///snap/base")
|
||||
added = pool.prewarm()
|
||||
self.assertEqual(len(added), 2)
|
||||
self.assertEqual(pool.warm_count(), 2)
|
||||
first = pool.acquire(self._task("t1"))
|
||||
self.assertEqual(first["mode"], "warm")
|
||||
second = pool.acquire(self._task("t2"))
|
||||
self.assertEqual(second["mode"], "warm")
|
||||
third = pool.acquire(self._task("t3"))
|
||||
self.assertEqual(third["mode"], "cold")
|
||||
# release 一律销毁,不放回池:
|
||||
self.assertTrue(pool.release(self._task("t1"),
|
||||
pool_vm=added[0]))
|
||||
self.assertEqual(pool.warm_count(), 0)
|
||||
|
||||
def test_prewarm_without_snapshot_is_empty(self):
|
||||
ex = self.make_exec()
|
||||
pool = WarmPool(ex, size=2, snapshot_url=None)
|
||||
self.assertEqual(pool.prewarm(), [])
|
||||
self.assertEqual(pool.acquire(self._task())["mode"], "cold")
|
||||
|
||||
|
||||
# -- dispatch / goal(wake → 跑 → finally 关) ---------------------------
|
||||
class DispatchTest(ChvCase):
|
||||
def test_dispatch_wakes_runs_shuts_down(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.dispatch_task(self._task())
|
||||
self.assertEqual(out["executor"], "cloud-hypervisor")
|
||||
self.assertEqual(out["status"], "completed")
|
||||
self.assertIn("ok-dispatch", out["stdout"])
|
||||
self.assertEqual(out["vm"], "chv-abc123")
|
||||
# finally 关 VM:ch-remote shutdown 在 dispatch 之后出现:
|
||||
shutdown_idx = next(i for i, c in enumerate(self.calls)
|
||||
if c[0] == self.ch_remote
|
||||
and c[-1] == "shutdown")
|
||||
ssh_idx = next(i for i, c in enumerate(self.calls)
|
||||
if c[0] == "ssh" and "dispatch" in c)
|
||||
self.assertLess(ssh_idx, shutdown_idx)
|
||||
self.assertNotIn("chv-abc123", ex._vms)
|
||||
|
||||
def test_goal_met(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.goal_task({"id": "g1", "prompt": "做目标",
|
||||
"goal_mode": True})
|
||||
self.assertEqual(out["status"], "completed")
|
||||
self.assertTrue(out["goal_met"])
|
||||
|
||||
def test_goal_resume_breakpoint(self):
|
||||
ex = self.make_exec()
|
||||
out = ex.goal_resume_breakpoint_task({"id": "g1",
|
||||
"prompt": "续跑"})
|
||||
self.assertEqual(out["status"], "completed")
|
||||
self.assertTrue(out["goal_met"])
|
||||
|
||||
def test_execute_is_ssh_status_only(self):
|
||||
ex = self.make_exec()
|
||||
# 本轨 execute 语义门禁:只读已运行 VM。先登记一个活
|
||||
# 的 VMM 进程表示 VM 已醒(未登记时 execute 报
|
||||
# vm_not_running,见 test_chv_optimize.py):
|
||||
ex._vms["chv-abc123"] = FakeProc(alive=True)
|
||||
out = ex.execute(self._task())
|
||||
self.assertEqual(out["executor"], "cloud-hypervisor")
|
||||
self.assertIn("ok-status", out["stdout"])
|
||||
# execute 不 wake(launcher 未调用):
|
||||
self.assertEqual(self.launches, [])
|
||||
|
||||
|
||||
# -- worker / autoscaler / server 接线 -----------------------------------
|
||||
class WiringTest(ChvCase):
|
||||
def test_worker_executor_name_and_cli_builder(self):
|
||||
from cloud import worker as worker_mod
|
||||
from cloud.sandbox import CloudHypervisorExecutor as CHV
|
||||
ex = self.make_exec()
|
||||
w = worker_mod.Worker.__new__(worker_mod.Worker)
|
||||
w.executor = ex
|
||||
self.assertEqual(w._executor_name(), "cloud-hypervisor")
|
||||
|
||||
class Args:
|
||||
executor = "cloud-hypervisor"
|
||||
built = worker_mod._build_cli_executor(Args(), None)
|
||||
self.assertIsInstance(built, CHV)
|
||||
|
||||
def test_autoscaler_passthrough(self):
|
||||
from cloud.autoscaler import Autoscaler
|
||||
scaler = Autoscaler("/tmp/no.db", [], executor="cloud-hypervisor")
|
||||
cmd = scaler._worker_command("w1")
|
||||
self.assertIn("--executor", cmd)
|
||||
self.assertEqual(cmd[cmd.index("--executor") + 1],
|
||||
"cloud-hypervisor")
|
||||
scaler2 = Autoscaler("/tmp/no.db",
|
||||
["--executor", "docker"],
|
||||
executor="cloud-hypervisor")
|
||||
cmd2 = scaler2._worker_command("w1")
|
||||
self.assertEqual(cmd2.count("--executor"), 1)
|
||||
self.assertEqual(cmd2[cmd2.index("--executor") + 1], "docker")
|
||||
|
||||
def test_server_default_executor(self):
|
||||
from cloud.server import _default_executor
|
||||
with mock.patch.dict(os.environ,
|
||||
{"CLOUD_DEFAULT_EXECUTOR":
|
||||
"cloud-hypervisor"}):
|
||||
self.assertEqual(_default_executor(), "cloud-hypervisor")
|
||||
with mock.patch.dict(os.environ,
|
||||
{"CLOUD_DEFAULT_EXECUTOR": "bogus"}):
|
||||
self.assertEqual(_default_executor(), "docker")
|
||||
|
||||
|
||||
class ServerWhitelistTest(unittest.TestCase):
|
||||
"""server 白名单端到端:真实 HTTP + 临时库 + 临时端口。"""
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.db_path = os.path.join(self.tmp.name, "control.db")
|
||||
try:
|
||||
from cloud.auth import AuthStore
|
||||
from cloud.server import create_server
|
||||
except ImportError:
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
from cloud.auth import AuthStore
|
||||
from cloud.server import create_server
|
||||
self.auth = AuthStore(self.db_path)
|
||||
self.owner_id, self.owner_key = self.auth.create_user("owner")
|
||||
old = os.environ.get("CODEX_GLOBAL_ADMINS")
|
||||
self._old_admins = old
|
||||
os.environ["CODEX_GLOBAL_ADMINS"] = self.owner_id
|
||||
self.server = create_server(self.db_path, "127.0.0.1", 0)
|
||||
self.port = self.server.server_address[1]
|
||||
self.thread = threading.Thread(
|
||||
target=self.server.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
status, self.project = self._req(
|
||||
"POST", "/projects", {"name": "demo"}, self.owner_key)
|
||||
self.assertEqual(status, 201, self.project)
|
||||
|
||||
def tearDown(self):
|
||||
self.server.shutdown()
|
||||
self.server.server_close()
|
||||
self.thread.join(timeout=5)
|
||||
try:
|
||||
self.auth.conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
self.tmp.cleanup()
|
||||
if self._old_admins is None:
|
||||
os.environ.pop("CODEX_GLOBAL_ADMINS", None)
|
||||
else:
|
||||
os.environ["CODEX_GLOBAL_ADMINS"] = self._old_admins
|
||||
|
||||
def _req(self, method, path, body=None, key=None):
|
||||
conn = http.client.HTTPConnection("127.0.0.1", self.port,
|
||||
timeout=10)
|
||||
hdrs = {}
|
||||
payload = None
|
||||
if body is not None:
|
||||
payload = json.dumps(body).encode("utf-8")
|
||||
hdrs["Content-Type"] = "application/json"
|
||||
if key is not None:
|
||||
hdrs["X-API-Key"] = key
|
||||
try:
|
||||
conn.request(method, path, body=payload, headers=hdrs)
|
||||
resp = conn.getresponse()
|
||||
data = resp.read()
|
||||
finally:
|
||||
conn.close()
|
||||
try:
|
||||
parsed = json.loads(data.decode("utf-8")) if data else None
|
||||
except ValueError:
|
||||
parsed = None
|
||||
return resp.status, parsed
|
||||
|
||||
def test_task_executor_cloud_hypervisor_accepted(self):
|
||||
status, task = self._req(
|
||||
"POST", "/tasks",
|
||||
{"project_id": self.project["id"], "prompt": "x",
|
||||
"executor": "cloud-hypervisor"}, self.owner_key)
|
||||
self.assertEqual(status, 201, task)
|
||||
self.assertEqual(task["executor"], "cloud-hypervisor")
|
||||
|
||||
def test_task_executor_bogus_rejected(self):
|
||||
status, body = self._req(
|
||||
"POST", "/tasks",
|
||||
{"project_id": self.project["id"], "prompt": "x",
|
||||
"executor": "bogus"}, self.owner_key)
|
||||
self.assertEqual(status, 400)
|
||||
self.assertIn("cloud-hypervisor", body["error"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+181
-19
@@ -195,7 +195,11 @@ from cloud.pr_exec import (claim_pr_job, execute_task_pr, finish_pr_job,
|
||||
mark_pr_job_needs_local_worker)
|
||||
from cloud.queue import budget_exceeded, ready_tasks
|
||||
from cloud.repos import RepoError, checkout_dir, resolve_repo
|
||||
from cloud.sandbox import CloudExecutor, DockerExecutor, LocalExecutor
|
||||
from cloud.sandbox import CloudExecutor, CloudHypervisorExecutor, \
|
||||
DockerExecutor, LocalExecutor
|
||||
# to10:CHV 强相关接线抽到 cloud.chv.worker_adapter,见 ADR-13;
|
||||
# worker 只留薄包装(方法名/签名/调用点不变)。
|
||||
from cloud.chv import worker_adapter as _chv_worker_adapter
|
||||
from cloud.uploads import upload_root as _upload_root
|
||||
from cloud.sandbox import _default_runner as _sandbox_default_runner
|
||||
from cloud.caps import effective_capacity, resolve_capacity
|
||||
@@ -331,6 +335,17 @@ def _merge_env(base, override):
|
||||
return merged
|
||||
|
||||
|
||||
def _call_goal_resume_breakpoint(executor, task, model_provider=None,
|
||||
extra_env=None):
|
||||
"""通用断点调用(to10:本体已移到
|
||||
cloud.chv.worker_adapter.call_goal_resume_breakpoint,
|
||||
此处只留薄包装——旧测试从 cloud.worker import 它,名
|
||||
字与签名不变,行为逐字不变)。"""
|
||||
return _chv_worker_adapter.call_goal_resume_breakpoint(
|
||||
executor, task, model_provider=model_provider,
|
||||
extra_env=extra_env)
|
||||
|
||||
|
||||
def _resolve_model(model=None, config_path=None):
|
||||
"""模型名:显式参数 > host 配置文件里的 model > host 默认值。"""
|
||||
if model:
|
||||
@@ -506,6 +521,31 @@ class Worker:
|
||||
self.executor.ignore_host_budget = self.ignore_host_budget
|
||||
except Exception:
|
||||
pass
|
||||
# CHV 审计转发:VM 生命周期事件(启动/暂停/恢复/快照/
|
||||
# 销毁/失败)经本回调进 AuthStore 审计(_audit 本身
|
||||
# best-effort,失败不影响任务)。时间线展示 VM 事件时
|
||||
# 读审计表即可,不另起一套 VM 事件表;成本/配额仍走
|
||||
# worker 现有 quota 语义,本回调绝不记账。
|
||||
if isinstance(self.executor, CloudHypervisorExecutor):
|
||||
# to10:回调本体已移到 worker_adapter.make_chv_audit_
|
||||
# callback(原 __init__ 闭包逐字搬运),此处只挂接。
|
||||
try:
|
||||
self.executor.audit_callback = \
|
||||
_chv_worker_adapter.make_chv_audit_callback(
|
||||
self._audit)
|
||||
except Exception:
|
||||
pass
|
||||
# CHV 残留清理:仅孤儿清理接线(feat-chv-arch95 定案,
|
||||
# 本轨分文件:生产入口 cloud.chv.orphans,绝不构造
|
||||
# WarmPool、不 import 实验模块)。worker 启动是显式
|
||||
# 入口,在这里调 facade;关闭时 close() 再调一次。
|
||||
# 调用失败不炸 worker 构造/关闭,但要进审计(_audit
|
||||
# 本身 best-effort)。本 worker 不再持有 _warm_pool。
|
||||
if isinstance(self.executor, CloudHypervisorExecutor):
|
||||
try:
|
||||
self._reconcile_warm_pool("startup")
|
||||
except Exception:
|
||||
pass
|
||||
self._cost_store = cost_store
|
||||
self._cancel_requested = set()
|
||||
# 真取消用:process_factory 可注入(单测用它返回一个假长跑
|
||||
@@ -740,8 +780,21 @@ class Worker:
|
||||
self._cost_store = CostStore(self.db_path)
|
||||
return self._cost_store
|
||||
|
||||
def _reconcile_warm_pool(self, when):
|
||||
"""显式调生产残留清理 facade,返回已清理 VM 名
|
||||
(to10:本体已移到 worker_adapter.reconcile_warm_
|
||||
pool,此处只留薄包装,旧测试调用点不变)。"""
|
||||
return _chv_worker_adapter.reconcile_warm_pool(
|
||||
getattr(self, "executor", None), self._audit, when)
|
||||
|
||||
def close(self):
|
||||
"""关闭惰性建的成本存储连接(注入的 cost_store 由调用方管)。"""
|
||||
"""关闭惰性建的成本存储连接(注入的 cost_store 由调用方管)。
|
||||
CHV worker 关闭时先显式调一次 warm-pool reconcile,
|
||||
失败不炸关闭流程。"""
|
||||
try:
|
||||
self._reconcile_warm_pool("shutdown")
|
||||
except Exception:
|
||||
pass
|
||||
if self._cost_store is not None:
|
||||
try:
|
||||
self._cost_store._conn.close()
|
||||
@@ -2276,6 +2329,12 @@ class Worker:
|
||||
self.executor.kill_container(entry["container"])
|
||||
except Exception:
|
||||
pass
|
||||
# feat-chv-fix(P1-2):CHV 取消真关 VM,不只改库状态。
|
||||
if hasattr(self.executor, "cancel_vm"):
|
||||
try:
|
||||
self.executor.cancel_vm(task_id)
|
||||
except Exception:
|
||||
pass
|
||||
self._running_clear(task_id)
|
||||
conn = _db_connect(self.db_path)
|
||||
try:
|
||||
@@ -2378,22 +2437,48 @@ class Worker:
|
||||
pass
|
||||
return
|
||||
|
||||
def _chv_cancel_watchdog(self, task_id, vm_name, stop_event):
|
||||
"""CHV 同步路径的取消看门狗(to10:循环本体已移到
|
||||
worker_adapter.cancel_watchdog_loop,此处只传回调)。"""
|
||||
_chv_worker_adapter.cancel_watchdog_loop(
|
||||
task_id, vm_name, stop_event, self._cancel_requested,
|
||||
self.get_task, getattr(self.executor, "cancel_vm", None),
|
||||
self.logger)
|
||||
|
||||
def _chv_quota_watchdog(self, task_id, vm_name, stop_event,
|
||||
quota_check, interval=30.0):
|
||||
"""CHV 同步路径的配额看门狗(to10:循环本体已移到
|
||||
worker_adapter.quota_watchdog_loop,此处只传回调)。"""
|
||||
_chv_worker_adapter.quota_watchdog_loop(
|
||||
task_id, vm_name, stop_event, quota_check,
|
||||
self._mark_quota_hit,
|
||||
getattr(self.executor, "cancel_vm", None),
|
||||
self.logger, interval=interval)
|
||||
|
||||
def cleanup_stale_containers(self):
|
||||
"""启动清理兜底:清掉上一轮残留的 codex- 容器。
|
||||
|
||||
只在执行器有 cleanup_stale_containers 时做(Docker);
|
||||
Docker 执行器清容器、CHV 执行器清残留 VM(feat-chv-fix);
|
||||
列出或删除失败只记日志,绝不抛异常炸掉 worker。
|
||||
返回已清理的容器名列表(非 Docker 执行器返回 [])。
|
||||
返回已清理的容器/VM 名列表(两者皆无的执行器返回 [])。
|
||||
"""
|
||||
cleaned = []
|
||||
cleanup = getattr(self.executor, "cleanup_stale_containers", None)
|
||||
if cleanup is None:
|
||||
return []
|
||||
try:
|
||||
cleaned = cleanup()
|
||||
except Exception as exc:
|
||||
log_event(self.logger, "worker_cleanup_failed",
|
||||
message="cleanup stale containers failed: %s" % exc)
|
||||
return []
|
||||
if cleanup is not None:
|
||||
try:
|
||||
cleaned = list(cleanup() or [])
|
||||
except Exception as exc:
|
||||
log_event(self.logger, "worker_cleanup_failed",
|
||||
message="cleanup stale containers failed: %s" % exc)
|
||||
# feat-chv-fix(P1-2):CHV 残留 VM 走通用清理路径,
|
||||
# 不再只靠 worker 启动/关闭的 WarmPool reconcile 兜底。
|
||||
chv_cleanup = getattr(self.executor, "cleanup_stale_vms", None)
|
||||
if chv_cleanup is not None:
|
||||
try:
|
||||
cleaned = cleaned + list(chv_cleanup() or [])
|
||||
except Exception as exc:
|
||||
log_event(self.logger, "worker_cleanup_failed",
|
||||
message="cleanup stale vms failed: %s" % exc)
|
||||
if cleaned:
|
||||
log_event(self.logger, "worker_cleanup",
|
||||
message="cleaned stale containers: %s"
|
||||
@@ -2409,12 +2494,15 @@ class Worker:
|
||||
# -- 缺 key 门 ------------------------------------------------------
|
||||
def _executor_name(self):
|
||||
"""执行器名:DockerExecutor -> docker,CloudExecutor -> cloud,
|
||||
其它 -> local。cloud 绝不能回落成 local,否则 cloud worker
|
||||
CloudHypervisorExecutor -> cloud-hypervisor,其它 -> local。
|
||||
cloud/cloud-hypervisor 绝不能回落成 local,否则该 worker
|
||||
会认领 local 池的任务,池过滤就失效了。"""
|
||||
if isinstance(self.executor, DockerExecutor):
|
||||
return "docker"
|
||||
if isinstance(self.executor, CloudExecutor):
|
||||
return "cloud"
|
||||
if isinstance(self.executor, CloudHypervisorExecutor):
|
||||
return "cloud-hypervisor"
|
||||
return "local"
|
||||
def key_available(self):
|
||||
"""只查存在性:名在环境里且值非空即 True;值本身绝不返回。"""
|
||||
@@ -4544,6 +4632,29 @@ class Worker:
|
||||
_watchdog_stop, quota_check),
|
||||
daemon=True)
|
||||
_quota_watchdog_thread.start()
|
||||
# feat-chv-fix(P1-2):CHV 同步路径同款看门狗(VM 名
|
||||
# 由任务 id 派生)。进程模式不重复起(_execute_via_
|
||||
# process 已轮询 quota_check)。
|
||||
if isinstance(self.executor, CloudHypervisorExecutor) \
|
||||
and hasattr(self.executor, "cancel_vm"):
|
||||
# to10:VM 名派生已移到 worker_adapter.watchdog_vm_
|
||||
# name(原 try/except 块逐字搬运)。
|
||||
_chv_vm_name = _chv_worker_adapter.watchdog_vm_name(
|
||||
self.executor, task)
|
||||
if _chv_vm_name:
|
||||
_watchdog_stop = threading.Event()
|
||||
_watchdog_thread = threading.Thread(
|
||||
target=self._chv_cancel_watchdog,
|
||||
args=(task_id, _chv_vm_name, _watchdog_stop),
|
||||
daemon=True)
|
||||
_watchdog_thread.start()
|
||||
if quota_check is not None and not self._use_popen():
|
||||
_quota_watchdog_thread = threading.Thread(
|
||||
target=self._chv_quota_watchdog,
|
||||
args=(task_id, _chv_vm_name,
|
||||
_watchdog_stop, quota_check),
|
||||
daemon=True)
|
||||
_quota_watchdog_thread.start()
|
||||
# 启动前分段日志(2026-10-29 任务 A):认领后 61 秒中断,
|
||||
# 但阻塞的 docker run 中途没有任何标记、容器 --rm 后输出
|
||||
# 也没了,事后完全无法定位卡在哪一段。派发开始/结束进
|
||||
@@ -4679,8 +4790,12 @@ class Worker:
|
||||
# Docker 路径上面已有 is_docker 守卫,不走这里。
|
||||
# 网关任务在非 Local 执行器上无法注入 key 与
|
||||
# provider,必须 loud fail,不静默跑错网关。
|
||||
# feat-chv-fix(P1-1):守卫收窄——CHV 已补
|
||||
# extra_env SSH 注入路径,不再被“非 Local 即
|
||||
# 拦”;无注入能力的执行器仍 loud fail。
|
||||
if gateway_provider and not isinstance(
|
||||
self.executor, LocalExecutor):
|
||||
self.executor,
|
||||
(LocalExecutor, CloudHypervisorExecutor)):
|
||||
raise RuntimeError(
|
||||
"sub2api 网关不支持该执行器:%s"
|
||||
% type(self.executor).__name__)
|
||||
@@ -4690,6 +4805,19 @@ class Worker:
|
||||
self.executor, LocalExecutor):
|
||||
result = self.executor.goal_resume_breakpoint_task(
|
||||
task, model_provider=gateway_provider)
|
||||
elif task.get("_goal_resume_breakpoint") and hasattr(
|
||||
self.executor,
|
||||
"goal_resume_breakpoint_task"):
|
||||
# CHV 补缺口轨:任何提供断点方法的执行器
|
||||
# 都可走断点(CHV 的 goal_resume_breakpoint_task
|
||||
# 已接线;Docker 走上面 is_docker 分支,不到
|
||||
# 这里)。网关任务在进入这里之前已被
|
||||
# “网关不支持该执行器”守卫 loud fail,
|
||||
# 故这里不放宽网关规则。
|
||||
result = _call_goal_resume_breakpoint(
|
||||
self.executor, task,
|
||||
model_provider=gateway_provider,
|
||||
extra_env=gateway_env)
|
||||
elif task.get("_goal_resume_breakpoint"):
|
||||
# 假执行器/CloudExecutor 没有断点方法时
|
||||
# 绝不回落成 goal(会撞已存在)或
|
||||
@@ -4718,6 +4846,11 @@ class Worker:
|
||||
task, seed_dir=seed_host_dir,
|
||||
extra_env=gateway_env,
|
||||
model_provider=gateway_provider)
|
||||
elif isinstance(
|
||||
self.executor, CloudHypervisorExecutor):
|
||||
result = self.executor.goal_task(
|
||||
task, extra_env=gateway_env,
|
||||
model_provider=gateway_provider)
|
||||
else:
|
||||
result = self.executor.goal_task(task)
|
||||
else:
|
||||
@@ -4726,6 +4859,11 @@ class Worker:
|
||||
task, seed_dir=seed_host_dir,
|
||||
extra_env=gateway_env,
|
||||
model_provider=gateway_provider)
|
||||
elif isinstance(
|
||||
self.executor, CloudHypervisorExecutor):
|
||||
result = self.executor.dispatch_task(
|
||||
task, extra_env=gateway_env,
|
||||
model_provider=gateway_provider)
|
||||
else:
|
||||
result = self.executor.dispatch_task(task)
|
||||
combined = (result.get("stdout", "") or "") + \
|
||||
@@ -4749,14 +4887,18 @@ class Worker:
|
||||
quota_check)
|
||||
else:
|
||||
if gateway_provider and not isinstance(
|
||||
self.executor, LocalExecutor):
|
||||
self.executor,
|
||||
(LocalExecutor,
|
||||
CloudHypervisorExecutor)):
|
||||
raise RuntimeError(
|
||||
"sub2api 网关不支持该执行器:%s"
|
||||
% type(self.executor).__name__)
|
||||
result = self.executor.run_task(
|
||||
task, extra_env=gateway_env,
|
||||
model_provider=gateway_provider) \
|
||||
if isinstance(self.executor, LocalExecutor) \
|
||||
if isinstance(self.executor,
|
||||
(LocalExecutor,
|
||||
CloudHypervisorExecutor)) \
|
||||
else self.executor.run_task(task)
|
||||
except Exception as exc: # runner 异常也如实写回 failed
|
||||
result = {"executor": "local", "task_id": task_id,
|
||||
@@ -6140,6 +6282,17 @@ class Worker:
|
||||
final_written = self._set_status_if_running(task_id, final)
|
||||
if quota_stop and final_written and final == "interrupted":
|
||||
self._mark_interrupted_reason(task_id, "quota")
|
||||
# feat-chv-arch95plus:VM 死亡写回——执行器 _run_with_vm
|
||||
# 已把 vm_dead/interrupted_reason 放进 result,本处把它
|
||||
# 写进 tasks.interrupted_reason 列,server 的只读字段
|
||||
# chv_vm_dead 才能据此触发;否则该字段在生产里永远
|
||||
# 不出现,前端 VM 死亡横幅就是死代码。只写原因文本,
|
||||
# 不改状态本身(_mark 只在 status='interrupted' 时写)。
|
||||
if final_written and final == "interrupted" \
|
||||
and (result or {}).get("vm_dead"):
|
||||
self._mark_interrupted_reason(
|
||||
task_id, (result or {}).get("interrupted_reason")
|
||||
or "VM 中途死亡")
|
||||
finished_at = time.time()
|
||||
# P0 安全批后:CAS 没改到行(任务已被取消/回收)时
|
||||
# 绝不写回结果,否则迟到的旧进程会覆盖新一轮的结果
|
||||
@@ -6756,7 +6909,12 @@ def _build_cli_executor(args, host_py):
|
||||
docker 时返回 DockerExecutor(env_key 名用 Worker 的解析规则,
|
||||
值本身绝不从这里传——DockerExecutor 只查存在性,-e 只传名),
|
||||
cloud 时返回 CloudExecutor(通用 HTTP runner,未配 endpoint
|
||||
时执行即明确失败,绝不降级)。"""
|
||||
时执行即明确失败,绝不降级),cloud-hypervisor 时返回
|
||||
CloudHypervisorExecutor(配置走 CHV_* 环境变量,缺依赖时
|
||||
执行即明确失败,绝不降级;任务内 LLM 不可自选它,见
|
||||
sandbox.py 该类 docstring)。"""
|
||||
if getattr(args, "executor", "local") == "cloud-hypervisor":
|
||||
return CloudHypervisorExecutor()
|
||||
if getattr(args, "executor", "local") == "cloud":
|
||||
return CloudExecutor()
|
||||
if getattr(args, "executor", "local") != "docker":
|
||||
@@ -6834,11 +6992,15 @@ def main(argv=None):
|
||||
"的 max_daily_turns 预算照常生效(双重门)")
|
||||
parser.add_argument("--max-tasks", type=int, default=None)
|
||||
parser.add_argument("--executor", default="local",
|
||||
choices=["local", "docker", "cloud"],
|
||||
choices=["local", "docker", "cloud",
|
||||
"cloud-hypervisor"],
|
||||
help="执行器:local 调 host.py 子进程;docker"
|
||||
" 调 DockerExecutor(无 docker 时任务"
|
||||
"明确失败,不静默降级);cloud 调通用"
|
||||
" HTTP runner(需 CLOUD_RUNNER_ENDPOINT)")
|
||||
" HTTP runner(需 CLOUD_RUNNER_ENDPOINT);"
|
||||
"cloud-hypervisor 调 CloudHypervisorExecutor"
|
||||
"(VM 内经 SSH 调 host.py,配置走 CHV_*"
|
||||
" 环境变量,缺依赖时任务明确失败)")
|
||||
parser.add_argument("--worker-id", default=None,
|
||||
help="多 worker 池身份(默认:主机名-进程号;"
|
||||
"生产部署应配稳定 id,否则重启后旧行变 stale)")
|
||||
|
||||
@@ -0,0 +1,357 @@
|
||||
# Cloud Hypervisor 执行器架构决策记录(ADR)
|
||||
|
||||
范围:cloud/chv/ 执行器链(#216 → #221 → #230 → #232 → #234 → #239 → #246 → #252 → #254 → feat-chv-arch-maint-fe → feat-chv-closeup → feat-chv-cover100 → 本轨 feat-chv-p1fix)。
|
||||
本文件只记录已落代码的决策,不记录未验证为已验证;真机、压测、动态安全
|
||||
仍见 docs/chv-production-readiness.md §11。
|
||||
|
||||
## ADR-1 SSH 目标:客机 IP 发现(已选) vs vsock / 纯 NAT(未选)
|
||||
|
||||
- 决策:显式 CHV_SSH_HOST > CHV_NET_JSON 首个带 ip 的 net 条目 >
|
||||
显式 CHV_SSH_FORWARD=1 宣称下的回环;三者皆无报
|
||||
ssh_target_undeclared,绝不无条件假设 127.0.0.1:2222 能通。
|
||||
- 未选 vsock:需客内 agent + 宿主代理两套新代码,当前客机镜像
|
||||
无此组件,写进代码仍是假保证。
|
||||
- 未选纯 NAT:DNAT 是宿主防火墙外部状态,执行器代码既不能
|
||||
创建也不能校验它,写进代码仍是假保证;宿主防火墙见就绪
|
||||
清单 §11.4 部署清单,代码门 + 防火墙缺一不可。
|
||||
|
||||
## ADR-2 网关 key 走 seed 投递(已选) vs argv / 宿主环境(已弃)
|
||||
|
||||
- 决策:provisioned 模式下密钥子集随 seed 写进客内
|
||||
/etc/chv/guest-env(0600),远程命令 source 它;传输层密钥
|
||||
进 argv 直接 ValueError;keyscan 模式有密钥即 fail-closed。
|
||||
- 放弃 argv:宿主 ps / /proc/<pid>/cmdline 可读;放弃宿主子进程
|
||||
环境:Docker 线 -e 只传名、值仍在 CLI 子进程环境,CHV 比它
|
||||
更严(值连宿主子进程环境都不进)。
|
||||
- 残留:seed 随 stop 删,私钥在 seed 盘建好后即删,
|
||||
cleanup_stale_secrets 扫崩溃残留;删失败 fail-fast 或脏标记拒起。
|
||||
|
||||
## ADR-3 磁盘 copy 默认,reuse 只作单租户实验(已选)
|
||||
|
||||
- 决策:CHV_DISK_MODE 默认 copy,每任务独立 disk.raw;
|
||||
reuse 须同时满足 CHV_REUSE_SINGLE_TENANT_EXPERIMENT=1 与
|
||||
CHV_SINGLE_TENANT_PROJECT 声明,且 wake 时审计放行。
|
||||
- 边界:代码无法验证数据库多租户状态,声明 + 审计即机制,
|
||||
不假装已校验;生产多租户下 reuse 一律不启用。
|
||||
|
||||
## ADR-4 restore 不重预置(纠正,见 #232 审计)
|
||||
|
||||
- 纠正:审计原说法“restore 须把 seed 接入 VmConfig”是错的——
|
||||
restore 不走 vm.create,无 VmConfig 可挂盘;快照内已固化原
|
||||
VM host key,重预置反而破坏钉死。
|
||||
- 正确不变量:provisioned 下复用已钉死 known_hosts、缺失
|
||||
fail-fast(host_key_not_pinned),绝不 keyscan 回落。
|
||||
|
||||
## ADR-5 WarmPool:实验性,生产 dispatch 永远冷启动(本轨定案)
|
||||
|
||||
- 决策(二选一的 B 项):把 WarmPool 从生产路径宣称里摘掉,
|
||||
明确标为实验/停用,不再留“半接线”悬空状态。
|
||||
- 原因:预热 VM 的磁盘是干净快照盘,不是任务盘;dispatch 前
|
||||
查池若要真正接线,必须在派任务前给已暂停 VM 重新供给任务
|
||||
盘(热换盘 / 二盘挂载 / host.py 磁盘语义三者都未实现),
|
||||
把预热盘直接当任务盘用会击穿任务盘隔离不变量。
|
||||
- 落地:WarmPool.wired_to_dispatch 恒为 False;_run_with_vm
|
||||
绝不查池,dispatch 永远冷启动;acquire 内 resume 只保留为
|
||||
实验接口;worker 启动/关闭仍调 reconcile_orphans()(上一世
|
||||
残留清理是池子保留的唯一生产价值)。
|
||||
- 重启条件:将来要重新接线,必须先实现任务盘重供给并补
|
||||
“任务盘来源”不变量测试,否则不得把 wired 改成 True。
|
||||
|
||||
## ADR-6 Docker goal 解析分叉:共享正典 + 同步守卫(本轨定案)
|
||||
|
||||
- 决策:Local/CHV 共用 parse_goal_result 正典;Docker goal_task
|
||||
内联版保留,但与正典的 8 个真实标记用例(met/unmet/
|
||||
interrupted 标记/无结论异常退出/永久错误/无结论 rc0/
|
||||
goal-resume 拒绝/已存在撞名)由 test_chv_arch95.py 的
|
||||
DockerSyncGuardTest 行为测试钉住状态全等,任一实现改了
|
||||
分叉测试即 fail,不再只靠文档清单。
|
||||
- 注意:Docker 的超时证据路径仍与内联解析绑在一起,那是
|
||||
有意保留的第二处差异,不在 8 用例的状态等价范围内。
|
||||
|
||||
## ADR-7 #215 未合并时的复用边界(不复制模块)
|
||||
|
||||
- 现状:#215(timeouts/gates/quota/sandbox_policy)未合并,
|
||||
本链 base 里无其模块,本轨不复制其代码。
|
||||
- 纠正(feat-chv-arch95plus,核对过 #215 远程分支真实
|
||||
函数后):上一轨 ADR-7 的“超时→timeouts、配额→quota”
|
||||
泛化映射是错的。#215 的 timeouts 收敛的是审批/队列
|
||||
超时(None/0=不过期),quota 是按月美元配额,与 CHV
|
||||
的 api/ready 秒数(0 无效)和 VM 数/磁盘 GB/快照数
|
||||
单位与语义都不同,硬套会改行为,故这两项无映射。
|
||||
- 真实映射(已落代码,见 cloud/chv/compat215.py):
|
||||
gates.resolve_inside(base, rel) 目录门 → CHV 快照/磁盘
|
||||
路径白名单已在 hardening 里使用,合并后改为 import;
|
||||
sandbox_policy.tmpfs_isolated → CHV 客机 /tmp 落客机
|
||||
磁盘、一律不隔离(fail-closed),合并后改为 import。
|
||||
其余 #215 函数(审批、控制门、合并状态门、worker 寻址)
|
||||
与 CHV 无交集,不映射。替换时须重跑本链测试,不得改语义。
|
||||
|
||||
## ADR-8 WarmPool 生产隔离:facade 而非类导入
|
||||
|
||||
- 决策:生产路径(worker 启动/关闭)只调
|
||||
cloud/chv/pool.py 的 reconcile_orphans_for_executor()
|
||||
facade,该函数内部才构造 WarmPool 并只调
|
||||
reconcile_orphans();生产 dispatch 绝不调 prewarm/
|
||||
acquire/release,executor.py 也不 import pool。
|
||||
WarmPool 类与 sandbox re-export 保留仅为兼容(旧测试
|
||||
import 路径),新代码不得经类入口进实验接口。
|
||||
- 测试钉住:facade 测试断言 acquire/prewarm 绝不被调;
|
||||
executor 不 import pool 由模块依赖检查覆盖。
|
||||
- 本轨更新(feat-chv-arch-maint-fe,取代上文“保留兼容”
|
||||
部分):CHV 尚未合并、尚无生产用户,“怕断 import”
|
||||
不是保留理由,兼容保留已删除。WarmPool 从 pool.py
|
||||
移到 experimental_pool.py,cloud/chv/__init__ 与
|
||||
cloud/sandbox.py 不再导出它,生产残留清理彻底分文件
|
||||
到 cloud/chv/orphans.py(facade 不再构造 WarmPool,
|
||||
直接读 registry 文件做清理)。旧 import 路径一律
|
||||
ImportError,见 ADR-12;合并前即生效的破环性清理,
|
||||
合并时须按 PR 描述的清单检查调用方。
|
||||
|
||||
## ADR-9 审计失败取舍:记日志 + 计数,不中断任务
|
||||
|
||||
- 选项 A(失败即中断任务):审计完整性最强,但 DB 短暂
|
||||
不可用会把一次审计失败放大成任务失败,且与 worker
|
||||
全局 best-effort 语义冲突,CHV 单独硬中断是不一致的。
|
||||
- 选项 B(失败记日志 + 计数 + 不中断任务):已选。
|
||||
executor._emit 失败时记宿主 error 日志并累加
|
||||
audit_failure_count(宿主可观测,非任务状态),任务
|
||||
继续;审计完整性本身没有保证,文档与就绪清单明写。
|
||||
- 边界:本决策只针对 VM 生命周期审计;VM 启动/关机等
|
||||
门禁失败仍按原有分类失败处理,不受本条影响。
|
||||
|
||||
## ADR-10 长函数抽不动的耦合点(本轨后已无 >50 行函数)
|
||||
|
||||
- 本轨后 cloud/chv/ 全部函数均 ≤50 行,上一轨的 7 个
|
||||
长函数全部拆完:stop_vm(进程清理/文件清理/脏标记三
|
||||
helper)、wait_guest_ready(provisioned 守卫/不匹配检查)、
|
||||
restore_vm(校验/启动段)、cleanup_stale_secrets(单 VM
|
||||
清理/活 VM 判定)、cleanup_stale_vms(死 socket 扫描/
|
||||
登记清理)、_unix_socket_api(读头/解析长度/读体/解析
|
||||
响应)、validate_net_entries(归一化/去重/冲突检查)、
|
||||
build_chv_config(组装段纯移动)。
|
||||
- 残留耦合点(已拆但仍需注意):stop_vm 的三块清理共享
|
||||
同一脏标记,任一块失败即整 VM 标记脏(有意,不拆成
|
||||
三个标记);_unix_socket_api 的读满语义仍由四个 helper
|
||||
共同保证,改任一 helper 须重跑截断/超时测试。
|
||||
|
||||
## ADR-12 模块职责图与禁 import 清单(本轨)
|
||||
|
||||
依赖方向(箭头 = 可以 import,叶子在下):
|
||||
- worker.py → orphans.py(生产残留清理,方法内 import)
|
||||
→ 标准库;worker 不 import experimental_pool。
|
||||
- executor.py → config/disk/hardening/hostkeys/
|
||||
ssh_transport/api_client/errors(方法内按需 import
|
||||
sandbox 共享 helper 除外,顶层绝不 import sandbox)。
|
||||
- experimental_pool.py → orphans.py(只借 registry 文件
|
||||
IO 与损坏语义,方向是实验 → 生产,绝不反向)。
|
||||
- orphans/config/compat215/errors → 只依赖标准库
|
||||
(叶子模块,测试用 AST 钉住)。
|
||||
- __init__.py / sandbox.py → 只 re-export 生产接口
|
||||
(CloudHypervisorExecutor、ChvFailure、
|
||||
reconcile_orphans_for_executor),不导出 WarmPool。
|
||||
- config.py → executor.py(executor 只经 build_chv_config
|
||||
取配置,不直接读 env 做二次解析)。
|
||||
|
||||
禁 import 清单(测试用 AST 逐项钉住,文档提及不算):
|
||||
- orphans/executor/worker/__init__/sandbox 一律不得
|
||||
import experimental_pool 或已删除的 cloud.chv.pool;
|
||||
- executor 一律不得 import orphans(执行器本体不做
|
||||
残留清理,那是 worker 启动/关闭的职责);
|
||||
- 任何生产模块不得 import cloud.gates /
|
||||
cloud.sandbox_policy / cloud.timeouts / cloud.quota
|
||||
(#215 未合并,见 compat215 REPLACEMENT_POINTS)。
|
||||
|
||||
## ADR-13 配置组装片段化(本轨,行为不变)
|
||||
|
||||
- build_chv_config 拆成六个片段:base(身份/资源基底)、
|
||||
ssh(密钥/用户/端口/host key 模式/客内路径/命令行)、
|
||||
network(net/net_json/ssh_host/ssh_forward/egress)、
|
||||
quota(max_vms/warm_pool_size)、disk(磁盘模式/大小/
|
||||
reuse 实验/快照上限与 URL)、timeout(任务/API/就绪
|
||||
超时)。组装只做 dict 合并,键冲突即 ValueError,
|
||||
绝不静默覆盖。
|
||||
- 移动说明(字段集合与取值不变,测试以 38 字段集钉住):
|
||||
egress 从旧 quota 片段移到 network(它是出口策略,
|
||||
不是 VM 配额);snapshot_url 从旧超时片段移到 disk;
|
||||
warm_pool_size 从旧超时片段移到 quota。唯一可观测
|
||||
的校验顺序变化:warm_pool_size 非法与磁盘非法同时
|
||||
出现时先报前者;单项非法时报错文案逐字一致。
|
||||
- 类型:cloud/chv/ 公开 API(非下划线函数 + 公开类
|
||||
__init__)全量标注,无 Any 逃逸;本轨顺手修正三处
|
||||
旧标注错误(record_snapshot 应为 tuple、host_key_paths
|
||||
应为 dict、build_seed_disk 应为 str)。私有 helper
|
||||
不在“公开 API”口径内,不硬标。
|
||||
|
||||
## ADR-14 前端只读侧扩展与验证方式(本轨)
|
||||
|
||||
- 本轨前端只做只读侧,不做控制按钮:后端无 VM 控制
|
||||
端点且安全审计 No-Go,绝不发明控制接口。
|
||||
- 列表行:此前 chv_vm_dead 只在详情展示,本轨列表行
|
||||
增加“VM 已报告死亡”徽标(字段已由同一 serializer
|
||||
返回,列表此前只是没渲染);无徽标时不代表 VM 活着。
|
||||
- 详情:此前只有 chv_vm_dead 时才出现 VM 行,本轨对
|
||||
所有 CHV 任务都显示 VM 行——已报告死亡给死亡文案,
|
||||
否则给“未报告 VM 死亡(后端无 live 状态字段,不代表
|
||||
VM 活着)”。非 CHV 任务不显示 VM 行;死亡文案同样
|
||||
先过执行器门,防止非 CHV 行误带字段时误显示。
|
||||
- 池管理与成本页:不新开页面。已有数据继续在已有
|
||||
位置展示:WorkersSection 的 CHV worker 在线数/共计、
|
||||
详情成本卡(token/LLM 请求数/美元明细,来自
|
||||
GET /tasks/{id}/cost)。没有数据的页面绝不新开。
|
||||
- 后端字段:本轨未新增第二个生命周期字段,理由写清——
|
||||
server 持久化的 CHV 生命周期事实只有 executor /
|
||||
status / interrupted_reason 三项,chv_vm_dead 已是其
|
||||
完整派生;再编一个字段要么是 interrupted_reason 的
|
||||
复制,要么是猜 live 状态,两者都不做。本轨后端改动
|
||||
是把派生抽成 _chv_vm_dead_from_row(行为逐字一致)
|
||||
并补 goal 回落形态(纯文本“VM 中途死亡”)的测试。
|
||||
- 前端测试:纯逻辑全覆盖——执行器选择谓词(精确值,
|
||||
不宽松匹配)、可用性(上一轨)、附件守卫谓词(CHV +
|
||||
带附件 → 后端 400,前端提示口径,后端是最终权威;
|
||||
派发页提交拦截已接该谓词)、只读状态文案与列表徽标。
|
||||
组件渲染不在覆盖内(无 testing-library 依赖,不临时
|
||||
加依赖冒充覆盖)。
|
||||
- 验证方式(仓库浏览器任务拒绝回环地址是已知限制,
|
||||
本轨绝不声称已做真机点击):① tsc + vite 构建通过;
|
||||
② npm run build:console 产物 console.html 经 grep 含
|
||||
新文案;③ Vitest 全过;④ 用户侧 live browser 手动
|
||||
检查清单(见 PR 描述):本地起 vite preview 或直接
|
||||
打开部署后的控制台,建一个 CHV 任务(无附件)进详情
|
||||
看 VM 行文案、带附件提交看拦截文案、列表行对已报告
|
||||
死亡任务看徽标;真机点击结果由用户侧回填,不在本轨
|
||||
回执里预填。
|
||||
|
||||
## ADR-11 只读 VM 字段边界:chv_vm_dead 不是 live 状态
|
||||
|
||||
- 决策:server 任务序列化只在 CHV 任务且 worker 已把
|
||||
VM 死亡写进 interrupted_reason 列时返回 chv_vm_dead=True,
|
||||
其它情形字段不存在;worker 终局时把执行器的 vm_dead
|
||||
原因写回该列(见 worker.py 终局块),否则该字段在
|
||||
生产里永远不出现。
|
||||
- 边界:该字段是“已报告死亡”,不是“VM 活着/死了”的
|
||||
live 状态——server 进程没有 VM 登记表,live 状态编不
|
||||
出来;前端无该字段时不显示 VM 状态,不猜测。VM 控制
|
||||
按钮仍不做(后端无控制端点且安全审计 No-Go)。
|
||||
|
||||
## ADR-13 CHV worker 适配模块(to10 轨)
|
||||
|
||||
- 决策:新建 `cloud/chv/worker_adapter.py`,worker.py 只留
|
||||
薄包装。抽入:断点调用(原顶层函数本体)、审计回调
|
||||
(原 `__init__` 闭包)、孤儿清理(原 `_reconcile_warm_pool`
|
||||
本体)、两条看门狗循环(原 `_chv_*_watchdog` 本体,self
|
||||
依赖全部改成回调参数)、看门狗 VM 名派生(原 `_run_attempt`
|
||||
try/except 块)。worker 方法名、签名、调用点、错误文案
|
||||
全部不变,旧测试继续走 worker 方法;适配器本身另有直接
|
||||
行为测试。适配器只依赖标准库 + chv 叶子模块 +
|
||||
`cloud.logging_config`,不 import `cloud.worker`,无循环;
|
||||
公共 API 全注解(#257 的类型测试扫 `cloud/chv/*.py`,
|
||||
新模块不注解会直接炸该测试——本轨已补齐)。
|
||||
- 明确不抽(硬抽无收益,只会撕通用流程):cancel() 里的
|
||||
`cancel_vm` 分支(与 Docker `kill_container` 分支相邻的
|
||||
通用取消流)、`cleanup_stale_containers` 的 CHV 分支
|
||||
(与 Docker 清理相邻的通用清理流)、网关守卫(Local /
|
||||
Docker / CHV 三分支共用的派发流)、`_executor_name` 的
|
||||
CHV 分支(通用执行器名映射)、`_run_attempt` 的看门狗
|
||||
线程启动(线程管理是 worker 通用逻辑,只抽了 VM 名派生)、
|
||||
VM 死亡写回(通用终局流)。以上六处留在 worker.py,
|
||||
不是遗漏,是评估后有理由的结论。
|
||||
- 附带清理:`inspect` 在 worker.py 的唯一用处随断点函数
|
||||
移走,import 一并删除(留着会触发 F401);worker.py
|
||||
基线已有的 I001(两处 import 排序)与 F821(playwright
|
||||
播种块的 `task`)与本轨无关,不顺手重排整个文件。
|
||||
|
||||
## ADR-14 审计失败落盘兜底(to10 轨)
|
||||
|
||||
- 决策:选“本地审计落盘文件”作为兜底,不选“只记日志 +
|
||||
计数”(已做,仍不够:日志会轮转、计数只剩数字,事后
|
||||
找不回哪次事件丢了)。实现:`executor._emit` 回调失败
|
||||
时,除记宿主 error 日志 + `audit_failure_count` +1 外,
|
||||
再把事件 append 到 `state_dir/chv-audit-spill.jsonl`
|
||||
(每行一 JSON:ts/event/kind/detail/task_id),文件超
|
||||
1MB 时轮转成 `.old` 再写。
|
||||
- 取舍(写清,不当成完整修复):① 落盘本身 best-effort,
|
||||
写失败只记日志,绝不中断任务;② 落盘不是 DB 替代,
|
||||
任务 timeline 查询看不到它,需宿主人工查阅;③ 轮转
|
||||
只留最新两代,更早的丢失事件仍会丢;④ detail 沿用
|
||||
`_emit` 契约(只含 VM 名/路径/URL/kind),绝不含密钥值,
|
||||
落盘文件随 state_dir 的 0700 权限保护。审计完整性本身
|
||||
仍无保证——保证需要 DB 事务级审计写入,那是 worker
|
||||
全局语义的改动,不在本轨硬做。
|
||||
- 不选的另一条路:“审计失败即中断任务”——ADR-9 已否决:
|
||||
审计是可观测性不是门禁,DB 短暂不可用时硬中断会把
|
||||
一次审计失败放大成任务失败,且与 worker 全局 best-effort
|
||||
语义冲突。
|
||||
|
||||
## ADR-15 前端只读 VM 展示四态与可用性统一文案(to10 轨)
|
||||
|
||||
- 决策:`chvVmDisplay(task)` 返回四态而非此前两态文案:
|
||||
dead(字段 present 且 true)、not-reported(CHV 任务但
|
||||
字段 absent,只表示未报告)、unknown(任务详情本身未
|
||||
加载)、not-applicable(非 CHV,不显示行)。旧入口
|
||||
`chvVmStatusText` 保留为 display 的 text 包装(unknown /
|
||||
not-applicable 回空串),已测调用点不变。详情页 VM 行
|
||||
改用 `chvVmDisplay(t).text` 并加 `role="status"` 与
|
||||
`aria-label="VM 状态"`。
|
||||
- 决策:`chvAvailabilityText(availability)` 统一新建任务 /
|
||||
Workers / 环境三处可用性文案。此前三处各写各的,Workers
|
||||
的未知分支恒写“仅全局管理员可见”(加载失败/未加载时
|
||||
也是这句,是文案 bug),本轨统一后按 reason 分三句,
|
||||
known 零在线沿用派发页的完整版(不拦提交、后端是最终
|
||||
权威)。无障碍打磨:可用性区与提交结果区加
|
||||
`aria-live="polite"`(结果区另加 `role="status"`),附件
|
||||
三个 input 加 `aria-label`(此前只有 title),禁用态本身
|
||||
已由 `disabled` 传达,不另造禁用文案。控制按钮仍不做
|
||||
(后端无控制端点且安全审计 No-Go);组件渲染仍不在
|
||||
测试覆盖内(无 testing-library 依赖,不临时加依赖冒充)。
|
||||
|
||||
## ADR-16 to10 轨的类型与 lint 口径(诚实标注)
|
||||
|
||||
- mypy:`--follow-imports=silent cloud/chv/`,只查 chv 本身
|
||||
(不拖入无关文件)。基线该口径 115 错 → 本轨 0 错;
|
||||
默认 follow-imports 口径基线 117 错 → 本轨 2 错,且
|
||||
2 错全在作用域外的 `cloud/marker_tools.py:142`(与 chv
|
||||
无关,不顺手修它),chv 自身在默认口径下也是 0 错。
|
||||
- ruff:作用域 F/E9/W/I(未用变量、语法错、空白、import
|
||||
排序),基线 8 错 → 本轨 0 错;默认全规则基线 208 错
|
||||
(含本仓库未采纳的风格意见,如百分号格式化、
|
||||
best-effort 清理的 try/except-pass),本轨不声称全规则
|
||||
通过。worker.py 不在 lint 作用域内:其基线 I001 两处与
|
||||
F821 一处与本轨无关,未重排;本轨只删了随抽取而变多余
|
||||
的 `inspect` import(否则新增 F401)。
|
||||
- 覆盖率:`coverage --source=cloud/chv` 对 CHV 套件(不含
|
||||
worker/server 全量),基线 85% → 本轨 87%(1826 行,
|
||||
缺 268 → 242);未覆盖大头仍是 executor 深层生命周期
|
||||
分支(需假 VMM 全链路)与 disk 配额分支,不靠删测试
|
||||
或改分母凑数。
|
||||
|
||||
## ADR-17 P1 修复轨语义收紧(feat-chv-p1fix,生产级审核 §5)
|
||||
|
||||
本轨只修审核报告 §5 的 P1 十项(P2 不动),决策均为
|
||||
“失败可见、猜测收紧、竞态加锁”,不放宽任何既有安全语义:
|
||||
|
||||
- 失败不算成功:孤儿清理失败项不进 cleaned、不清库、
|
||||
落盘保留失败项(P1-01);残留私钥删失败不进 cleaned、
|
||||
发审计并写脏标记(P1-03);单条超帽快照拒绝记录而不
|
||||
是悄悄保留(P1-06);快照失败与剪枝删失败均发审计
|
||||
(P1-08)。“尽力清理、不因单条失败卡住整批”的旧语义
|
||||
被修正为“尽力清理、失败项留痕可重试”——整批仍不卡,
|
||||
但失败不再被成功名单掩盖。
|
||||
- 损坏与非法不静默:registry 损坏与非法条目均发审计并
|
||||
计数(P1-02);非法条目落盘时丢弃(已审计,不再每次
|
||||
重启重复审计)。
|
||||
- 不猜目标:SSH 命令构造删除回环回落,失败即抛
|
||||
(P1-04);显式回环(CHV_SSH_HOST / CHV_SSH_FORWARD=1)
|
||||
仍可用,wake/wait 严格路径不受影响。
|
||||
- 竞态用文件锁:容量/net/磁盘 check→act 持
|
||||
`state_dir/chv-state.lock` 的 flock 全程(P1-05),
|
||||
拒绝“只加进程内锁就声称多 worker 已解决”的写法;
|
||||
串行 wake 是有意的代价,已运行 VM 的任务执行不持锁。
|
||||
- 非正配置在入口拦:CPU/内存/任务超时非正在构造层与
|
||||
preflight 层双重校验(P1-07);None 任务超时 = 不限
|
||||
的旧语义保留。
|
||||
- 覆盖口径诚实:行覆盖与分支覆盖分开报(P1-09);
|
||||
`__init__.py` 计入 coverage.py 分母(4 行);长函数
|
||||
50 行硬门禁靠提取维持,新增 helper 均有命名职责
|
||||
(`_snapshot_create`、`_tighten_snapshot_cap`、
|
||||
`_preflight_task_timeout_check`、`_emit_prune_events`、
|
||||
`_state_lock_ctx`),不是为凑行数而拆。
|
||||
@@ -0,0 +1,985 @@
|
||||
# Cloud Hypervisor 执行器生产就绪清单
|
||||
|
||||
> 状态图例:✅ 代码已完成 · 🧪 mock 测试已覆盖(无真 VM) ·
|
||||
> ⏳ 需 KVM 宿主实测 · 🏗️ 需镜像构建 · 📈 需压测 · 🔒 需安全审计
|
||||
>
|
||||
> 本清单是 PR 的一部分,不是事后补的口号。凡是标 ⏳/🏗️/📈/🔒
|
||||
> 的项,在本轨内一律不声称已验证、已建成、已通过。
|
||||
|
||||
## 0. 交付点卡在哪一关(必读,先于逐项清单)
|
||||
|
||||
- **CI 门禁(未过,交付点 = PR OPEN)**:CI 必须通过才能合并。
|
||||
此前全仓 CI 因 GitHub 账单问题 0 步骤失败;若账单未恢复、
|
||||
CI 仍失败,本 PR 只到 OPEN,不合并,更不为变绿改 CI 配置
|
||||
或绕过检查。
|
||||
- **真 VM 门禁(未过,执行器默认关闭)**:生产启用(把真实任务
|
||||
路由到 cloud-hypervisor、部署到生产主机)必须先有带 /dev/kvm
|
||||
的宿主并完成真实 VM 启动、快照恢复、任务端到端验证。
|
||||
当前事实:生产主机 124.222.165.170 本身就是 CVM
|
||||
(systemd-detect-virt=kvm,DMI product=CVM),/dev/kvm 不存在、
|
||||
无 vmx/svm 标志、无 kvm 模块、未装 cloud-hypervisor,无嵌套
|
||||
虚拟化透传;本 VM 容器内同样无 /dev/kvm。
|
||||
故真 VM、快照恢复、压测在本轨内一律不声称已验证;执行器
|
||||
默认关闭(server 默认仍是 docker,worker 默认仍是 local,
|
||||
CHV 只在任务创建时显式选 `executor=cloud-hypervisor` 且有
|
||||
CHV worker 认领时才跑),不进入生产任务路由。
|
||||
- **还差什么才能真正上生产**(按顺序):① 用户侧开通带 /dev/kvm
|
||||
的裸金属或开启嵌套虚拟化的实例;② 在该宿主上构建并发布
|
||||
chv-guest 镜像(见 §4);③ 在该宿主上跑通真实 VM 启动、
|
||||
快照恢复、任务端到端、预热池与压测;④ 安全审计通过(§6);
|
||||
⑤ CI 账单恢复且本 PR CI 变绿后,按当时 main 头复核 diff 再合并、
|
||||
再部署。缺任何一项,都不把 CHV 推上生产。
|
||||
|
||||
## 1. 单任务单 VM 隔离
|
||||
|
||||
- ✅🧪 一个任务一个 VM:dispatch/run/goal 均走 wake → 跑 → finally
|
||||
shutdown,同一任务重复 wake 先关旧 VM;VM 名、api-socket、
|
||||
每任务磁盘路径均按任务派生,互不相同(测试断言)。
|
||||
- ✅🧪 每任务磁盘独立:copy 模式复制基础镜像为每任务磁盘;
|
||||
reuse 模式仅单租户实验,preflight 拦截(须显式实验二次确认并审计放行,feat-chv-fix 起代码真正拦截;此前“代码与文档双写”的说法是错的,代码当时并不拦截)。
|
||||
- ✅🧪 租户无共享路径:执行器不挂载宿主根、宿主代码目录或含
|
||||
密钥目录;客机代码在镜像内 `/app`,不在宿主共享挂载上。
|
||||
- ✅🧪 密钥不进镜像/代码/prompt:SSH argv 只有 key 文件路径;
|
||||
模型 key 由客机 cloud-init 预置的环境变量名注入;测试断言
|
||||
命令中不含哨兵密钥值。feat-chv-secfix 起网关密钥不再走
|
||||
SSH env 前缀(旧路径值进 argv 已废除),改走 provisioned
|
||||
seed 盘客内文件(/etc/chv/guest-env,0600),keyscan 模式
|
||||
下密钥投递 fail-closed;审计 detail 里绝无密钥值。
|
||||
- ✅🧪 SSH host key 按 VM 钉死(优化轨,#216 的 no + /dev/null
|
||||
已废除):每 VM 在 state_dir/<vm>/known_hosts 有独立文件,
|
||||
首次启动经 ssh-keyscan 在解析出的 SSH 目标上捕获 key 写入该文件,
|
||||
此后 SSH 一律 StrictHostKeyChecking=yes + 该文件校验;
|
||||
非 root 客机用户(CHV_SSH_USER,显式空串直接拒绝)与端口
|
||||
(CHV_SSH_PORT,1-65535)为显式配置。mock 只验证捕获→
|
||||
写文件→钉死选项的管线,不验证首次密钥真实性。
|
||||
- ✅🧪 Host key 预置方案(补缺口轨,替代 TOFU 为生产路径):
|
||||
宿主在启动前为每 VM 生成一次性 ed25519 host key 对,
|
||||
公钥立即写进该 VM 的 known_hosts 钉死,私钥只进 per-VM
|
||||
cloud-init seed(随 VM 关闭/回收而删),客机镜像硬性
|
||||
要求见 chv-guest/README.md §5(cloud-init NoCloud、
|
||||
首启先应用 seed 再起 sshd、不得无条件重生成 host key)。
|
||||
ssh-keyscan 首次捕获只保留为弱回落(须显式设
|
||||
CHV_HOST_KEY_MODE=keyscan),生产默认值已是 provisioned
|
||||
(feat-chv-secfix,H4,此前默认值 keyscan 是反的);TOFU 首次
|
||||
真实性仍待预置 key 方案在真机上验证,安全审计时须单独立项确认。
|
||||
- 🔒 需安全审计:VM 逃逸面(virtio 设备、tap 桥接、SSH 配置)、
|
||||
客机 root 提权面、租户间东西向隔离的宿主网络配置,须由
|
||||
安全审计逐项确认,代码自查不能代替审计。
|
||||
|
||||
## 2. 快速启动:预热快照池 + 镜像缓存
|
||||
|
||||
- ✅🧪 快照/恢复接口:snapshot_vm / restore_vm(restore 后显式
|
||||
resume,见 §3),WarmPool 接口与状态机(cold → prewarming →
|
||||
warm → assigned → running → recycled/destroyed)。
|
||||
- ⚠️ 定案(feat-chv-arch95,ADR-5):WarmPool 是实验性、非生产
|
||||
路径。dispatch 永远冷启动,_run_with_vm 绝不查池;池空回落
|
||||
冷启动的说法不再作为生产特性宣称,acquire/resume 只保留为
|
||||
实验接口。
|
||||
- ✅🧪 预热条目一次性:acquire 消耗、release 一律销毁,绝不跨
|
||||
任务复用脏 VM。
|
||||
- ✅🧪 预热池 registry 与孤儿清理(优化轨):池状态落
|
||||
state_dir/warm-pool.json(原子写),worker 重启后载入的
|
||||
条目一律当孤儿、只能 reconcile_orphans() 清理,绝不重新
|
||||
发给任务;registry 损坏时 quarantine 成 .corrupt 并从空池
|
||||
开始;registry 之外的残留 socket 仍由 cleanup_stale_vms()
|
||||
扫描处理。
|
||||
- ⏳ 需 KVM 宿主实测:预热池的真实冷启动耗时、resume 耗时、
|
||||
池大小与命中率、快照来源一致性,均需真机数据,mock 数字
|
||||
不能当性能结论。
|
||||
- 📈 需压测:多 VM 并行预热/分配/销毁的宿主 CPU/内存/磁盘
|
||||
压力、快照存储增长,均未测。
|
||||
|
||||
## 3. 完整生命周期
|
||||
|
||||
- ✅🧪 create / boot:API 模式 wake_vm(vmm.ping → vm.create →
|
||||
vm.boot → SSH 就绪)。
|
||||
- ✅🧪 pause-resume(sleep-wake):pause_vm / resume_vm(ch-remote)。
|
||||
- ✅🧪 snapshot-restore:restore_vm(restore + 显式 resume +
|
||||
SSH 就绪),恢复后默认 paused 的坑已在代码与测试里钉住。
|
||||
- ✅🧪 hibernate:CHV 无原生 hibernate 端点,本轨用 pause +
|
||||
snapshot + shutdown 等价实现,文档与代码双写“等价实现”,
|
||||
不声称接了原生端点;唤醒走 restore_vm。
|
||||
- ✅🧪 shutdown-destroy / recycle:shutdown_vm(ch-remote
|
||||
shutdown + 杀进程 + 删 socket)、recycle_vm(保留
|
||||
task_checkpoints/task_attempts,丢失 guest /tmp 与未拷出
|
||||
的 worktree 临时物,绝不声称恢复临时文件)。
|
||||
feat-chv-secfix 起 stop 即删任务 disk.raw(保留期 0,
|
||||
需取证须在 shutdown 前经 SSH 拷出),残留盘由
|
||||
gc_stale_disks 扫描清理,删失败记审计 + 脏标记。
|
||||
- ✅🧪 execute 只读已运行 VM(优化轨):execute() 先查 VM
|
||||
是否在运行(登记活进程或 api-socket 存在),未运行报
|
||||
ChvFailure(kind=vm_not_running),不为一次 status 起 VM,
|
||||
也不向不存在的端口发 SSH 再拿连接失败冒充任务状态。
|
||||
- ✅🧪 长任务中途死亡判 interrupted(优化轨):客内命令抛
|
||||
传输异常、或非 goal 结果已 failed 且登记 VMM 进程已死时,
|
||||
_run_with_vm 改判 interrupted(断点可续跑)而非笼统
|
||||
failed;VM 还活着时的异常/失败原样处理。中途心跳未做:
|
||||
host.py 单次 SSH 长回合没有客内进度信号可采,另起一路
|
||||
SSH 轮询只能证明 VM 活着、不能证明任务在进展,评估后
|
||||
认为风险大于收益,宁可只做死亡分类。
|
||||
- ⏳ 需 KVM 宿主实测:上述全部生命周期在真机上的状态转换、
|
||||
快照恢复后 guest 状态一致性、休眠后文件/worktree 保留,
|
||||
均未在真机验证。
|
||||
|
||||
## 4. 完整开发环境(客机镜像)
|
||||
|
||||
- ✅ 客机镜像内容清单:`chv-guest/README.md` 按
|
||||
`cloud/docker/Dockerfile` 能力对齐(Python 3.12、git、
|
||||
pandoc/poppler/latex、python-docx/openpyxl/pptx/PyMuPDF/
|
||||
Pillow、codex 0.164.0-alpha.9 成套、Playwright + Chromium、
|
||||
openssh-server、cloud-init)。
|
||||
- ✅ 构建脚本实做(补缺口轨):`chv-guest/build.sh`(参数校验、
|
||||
KVM 宿主检查、下载 Ubuntu 云镜像、qemu-img 转 raw、
|
||||
virt-customize 装清单、拷入 /app 代码、virt-get-kernel、
|
||||
BUILDINFO.json 生成与 tag 校验;--dry-run 只打印命令)。
|
||||
- 🏗️ 需镜像构建:镜像本身未构建、未发布;BUILDINFO 版本核对、
|
||||
浏览器版本钉死、中文 LaTeX 单独一版,均待构建时做。
|
||||
- ⏳ 需 KVM 宿主实测:镜像在真 VM 里启动、codex/Playwright/
|
||||
git/文档工具链逐项跑通,均未验证。
|
||||
|
||||
## 5. 资源与成本
|
||||
|
||||
- ✅🧪 每 VM 配额:CPU/内存进 VmConfig(CHV_CPU/CHV_MEM),
|
||||
磁盘配额门(CHV_DISK_GB,默认 50 GB 总量预算、超限拒绝,
|
||||
feat-chv-secfix 起默认开启,此前未设即不查),快照上限
|
||||
(每 VM 数量默认 10、总量默认 20 GB、保留期默认 30 天,
|
||||
超量最旧优先剪枝),宿主容量门
|
||||
(CHV_MAX_VMS,超限明确拒绝排队,不降级;残留 socket
|
||||
须真连通才计数,死 socket 不再占满容量)。
|
||||
- ✅ 超时与最大轮次:SSH 单次超时(CHV_TASK_TIMEOUT)+
|
||||
worker task_timeout 外层护栏;goal 最大轮次 100(与 Local
|
||||
同款),执行器内部不重试(重试是 worker max_retries 语义)。
|
||||
- ✅ 成本复用:不另起一套计费;worker 现有 quota/成本语义
|
||||
(per-user USD 配额、quota 看门狗、成本写库)仍是唯一口径,
|
||||
执行器只把 host.py 用量原样返回。
|
||||
- ✅ 空闲休眠省成本:hibernate_vm(pause + snapshot +
|
||||
shutdown)+ WarmPool 释放即销毁,空闲 VM 不占 vCPU/内存
|
||||
(快照只占磁盘,见 §2 压测项)。
|
||||
- ✅ autoscaler:按 executor 过滤队列深度(已有)+ 本轨透传
|
||||
`--executor` 给拉起的 worker(否则 CHV 池永远饿死)。
|
||||
- 📈 需压测:配额下的真实资源占用、超限拒绝的排队行为、
|
||||
成本与配额在长任务下的实际表现,均未测。
|
||||
|
||||
## 6. 网络与治理
|
||||
|
||||
- ✅🧪 受控 egress:默认 fail-closed(CHV_EGRESS_PROXY 未设且
|
||||
未显式允许直连时 preflight 直接失败);客机命令加 env
|
||||
代理前缀,出口只走白名单代理(Tinyproxy 思路复用)。
|
||||
feat-chv-secfix 措辞纠正:env 前缀只是代码门,不是网络层
|
||||
隔离——客内 root 可 unset/裸 socket 绕过;真正的隔离是
|
||||
“代码门 + 宿主防火墙缺一不可”(见 §11 部署清单),
|
||||
代码不再把 fail-closed 写成已隔离。
|
||||
- ⏳🔒 需宿主网络配置 + 安全审计:tap 桥接是否真的只路由到
|
||||
白名单代理、VM 能否绕过代理直连外网,是宿主网络配置问题,
|
||||
执行器代码不能自证,必须在 KVM 宿主上实测并审计。
|
||||
代码已做:每 VM 独立 tap/mac/ip 校验(共享即拒绝并审计)、
|
||||
net 条目字段白名单、egress 代理地址校验。
|
||||
- ✅ 审批门:任务 approval 经 `_approval_cli_args` 进客内 host.py
|
||||
顶层参数,VM 内 agent 不能自改;delete/deploy/spend 类动作
|
||||
仍走 server 的 approval/merge_gate,不由 VM 内 agent 自批
|
||||
(本轨不改 approval/merge_gate 语义,只复用)。
|
||||
- ✅ 工具与系统调用范围收敛:VM 边界本身就是系统调用边界
|
||||
(客机独立内核);plan 任务走 read-only 沙箱参数;最大步数
|
||||
(goal 100 轮)、时间(task_timeout)、重试(worker 1 次)
|
||||
超限即停并审计(vm_failed 带 kind)。
|
||||
|
||||
## 7. 可观测与审计
|
||||
|
||||
- ✅🧪 VM 事件审计:create/boot/ready/pause/resume/snapshot/
|
||||
restore/shutdown/destroy/failed/recycled/warm_pool_acquire/
|
||||
release 均经 audit_callback 发出;worker 接线后转 AuthStore
|
||||
审计(_audit 本身 best-effort)。feat-chv-secfix 起补齐:
|
||||
host_key_generated/host_key_captured/host_key_mismatch、
|
||||
preflight_failed、wake 前置失败(容量/配额/net 冲突)、
|
||||
vm_disk_deleted/vm_disk_delete_failed/vm_disk_gc、
|
||||
host_secret_residue_cleaned、net_config_dual、快照剪枝;
|
||||
审计回调失败记宿主 error 日志,不再双重静默。
|
||||
- ✅🧪 失败分类:ChvFailure 带 kind(kvm_missing/binary_missing/
|
||||
kernel_missing/image_missing/ssh_key_missing/cpu_flag_missing/
|
||||
egress_not_configured/api_timeout/guest_not_ready/vm_create_failed/
|
||||
vm_boot_failed/host_capacity_exceeded/disk_quota_exceeded),
|
||||
不笼统 failed。
|
||||
- ✅ 断点对齐(补缺口轨已接线):VM 快照/休眠与现有
|
||||
task_checkpoints/task_attempts 语义对齐(recycle 保留两者,
|
||||
丢失临时文件);worker 通用分支对任何提供
|
||||
goal_resume_breakpoint_task 的执行器调用它(Local 仍走
|
||||
原路径,网关任务仍被 loud fail 守卫拦住);WarmPool
|
||||
reconcile_orphans 已由 worker 启动与关闭时显式调用,
|
||||
失败不炸主流程但进审计;VM 中途死亡改判 interrupted
|
||||
后走同一断点路径,不再落 failed 死信(见 §3 与新增测试)。
|
||||
- ⏳ 需 KVM 宿主实测:审计事件在真机任务 timeline 里的完整
|
||||
呈现、失败分类与真实故障的对应关系,均未验证。
|
||||
|
||||
## 8. 多 VM 并发与容量
|
||||
|
||||
- ✅🧪 worker 注册:`_executor_name()` 返回 cloud-hypervisor,
|
||||
worker register/heartbeat 已上报 executor + capacity(现有
|
||||
机制复用,无需新表)。
|
||||
- ✅🧪 多 VM 并行:`_vms` 登记 + 锁保护,wake 按任务名登记,
|
||||
不同任务互不覆盖;容量门在 wake 前检查。
|
||||
- ✅🧪 残留 VM 清理:cleanup_stale_vms 处理已死/已关/socket
|
||||
残留三种情况,只清 chv- 前缀 socket;feat-chv-secfix 起
|
||||
清理前先跑残留盘 GC 与残留密钥扫描(不依赖 socket 是否
|
||||
存在,覆盖 worker 崩溃后 socket 已删、文件仍在的情形)。
|
||||
- ✅🧪 容量不足明确拒绝:kind=host_capacity_exceeded,调用方
|
||||
排队/拒绝,绝不降级 Docker/Local(测试断言 launcher 未调用)。
|
||||
- 📈 需压测:多 VM 并行下的宿主容量、VM 启动并发上限、残留
|
||||
清理在高压下的表现,均未测。
|
||||
|
||||
## 9. 默认关闭与生产启用条件(重申)
|
||||
|
||||
- ✅ 默认关闭:server 默认执行器仍是 docker,worker 默认 local;
|
||||
CHV 只在显式选 executor 且有 CHV worker 时才跑;无 CHV_*
|
||||
配置时 preflight 直接失败。
|
||||
- ⏳ 生产启用前置(全部未完成):带 /dev/kvm 的宿主开通 →
|
||||
chv-guest 镜像构建发布 → 真实 VM 启动/快照恢复/任务端到端
|
||||
验证 → 压测 → 安全审计 → CI 变绿后按当时 main 头复核 diff
|
||||
再合并部署。缺一不可。
|
||||
|
||||
## 10. 审核修复轨 feat-chv-fix(堆叠于 #230 之上)
|
||||
|
||||
- ✅🧪 P0 已修:seed 磁盘(CIDATA ISO,第二块只读盘)挂进
|
||||
VmConfig;restore 不回落 keyscan(provisioned 下复用已钉
|
||||
死 known_hosts、缺失 fail-fast,快照内 host key 不重预置);
|
||||
reuse 由 preflight 拦截(显式实验二次确认 + 审计放行);
|
||||
SSH 目标选客机 IP 方案(显式 host > net 静态 ip > 显式
|
||||
转发宣称下的回环,三者皆无 fail-fast;vsock 与纯 NAT 方案
|
||||
未选,理由见 config.py)。
|
||||
- ✅🧪 P1 已修或有结论:网关 extra_env 经 SSH env 前缀注入,
|
||||
worker 守卫收窄到 Local/CHV;取消/清理/看门狗接 CHV 通用
|
||||
路径;WarmPool acquire 内 resume(本轨定案为实验性,见 ADR-5,
|
||||
不再是生产特性);容量门改持久化
|
||||
计数(socket 联合计数);磁盘配额改总量+余量、已存在磁盘
|
||||
校验重拷;API 客户端按 Content-Length 验完整性;快照
|
||||
file:// 目标验存在且非空再 shutdown;execute 死端口归类
|
||||
vm_not_running。
|
||||
- ⚠️ 仍未验证(不得声称已交付):CHV + 网关真实任务;真机 wake/boot/SSH(含 seed
|
||||
注入与 NAT/客机 IP 实测);快照恢复端到端;压测;安全审计;
|
||||
CI 账单恢复。另:P0-2 审计原说法“restore 须把 seed 接入
|
||||
VmConfig”已纠正——restore 不走 vm.create,seed 无从挂盘,
|
||||
正确不变量是复用钉死 known_hosts。
|
||||
- ✅🧪 P2 已做(范围已恢复为全做,上一条收窄令作废):
|
||||
test_chv_gaps.py 的 grep 文本断言改成真实 Worker._run_attempt
|
||||
行为断言;stop_vm 删除失败记 vm_key_cleanup_failed 审计
|
||||
(同名复用旧 key 的彻底根治在本轨之前列为遗留——
|
||||
feat-chv-secfix 起已根治:清理失败写脏标记,同名 VM
|
||||
拒绝启动,见 §11 M2);配置校验时机统一到 preflight(CPU/内存非法
|
||||
进 preflight,kind=config_invalid);build.sh 与 README 口径
|
||||
对齐(章节顺序 §5 host key → §6 对接、BUILDINFO contents
|
||||
纠正为名字列表无逐项版本、Node.js 不装写实);第二启动路径
|
||||
清理(build_vm_command 降级为私有 _build_vm_command,手动
|
||||
等价写法保留;Docker goal 内联解析顶部加指向正典
|
||||
parse_goal_result 的分叉说明 + 同步检查清单);长函数只做
|
||||
小提取(preflight 的 reuse/CPU/内存校验块抽成
|
||||
_preflight_config_checks,语义不变;config.py 字段解析与
|
||||
其余长函数深耦合,本轨如实停、不硬拆)。
|
||||
|
||||
## 11. 安全审计修复轨 feat-chv-secfix(堆叠于 #239 之上)
|
||||
|
||||
> 审计报告:`~/workspace/chv-security-audit/REPORT.md`(只读静态
|
||||
> 审计,verdict No-Go,Critical 0 / High 7 / Medium 8 / Low 5)。
|
||||
> 本节逐项给结论:代码已做 = 本轨代码 + mock 测试已覆盖;
|
||||
> 真机未验证 = 代码改了,但动态实效仍须 KVM 宿主验证,
|
||||
> 绝不把静态通过写成动态已验。审计 verdict 本身仍是 No-Go,
|
||||
> 直到 §11 末的真机清单全部通过——本轨只修代码层,不改 verdict。
|
||||
|
||||
### 11.1 High(全部代码已做、真机未验证)
|
||||
|
||||
- H1 egress 网络层强制:代码能做的全部做了——egress 代理
|
||||
地址校验(http/https + 主机名)、直连仍须显式允许 + 审计、
|
||||
每 VM 独立 tap/mac/ip 校验(共享即拒绝并审计)、net 条目
|
||||
字段白名单。代码做不到的宿主防火墙规则见 §11.4 部署清单,
|
||||
文档明写“代码门 + 宿主防火墙缺一不可”;env 前缀不再被
|
||||
写成已隔离。
|
||||
- H2 网关 key 移出 argv:密钥子集(SUB2API_API_KEY)在
|
||||
provisioned 模式下随 seed 写进客内 /etc/chv/guest-env
|
||||
(0600 root),远程命令 source 该文件;传输层对密钥名
|
||||
进 argv 直接 ValueError(fail-closed)。比 argv 安全的理由:
|
||||
值既不在宿主进程表(ps /proc cmdline)里,也不在宿主子
|
||||
进程环境(/proc environ)里,只活在 seed.iso(0600,随
|
||||
VM 销毁而删)与客机内——客内 agent 本来就有该 key,修
|
||||
的是“宿主暴露面”,不是“客内可见性”。与 Docker 线的差异:
|
||||
Docker 是“-e 只传名、值由 CLI 子进程环境提供”(值仍在
|
||||
宿主进程环境里),CHV 是“值只进 seed 客内文件”(宿主
|
||||
进程环境里也没有),比 Docker 线更严。keyscan 模式无
|
||||
seed 可投递,有密钥即 fail-closed(secret_env_requires_
|
||||
provisioned),不再将就跑。残留清理:seed 目录与 seed.iso
|
||||
随 stop 而删,崩溃残留由 cleanup_stale_secrets 扫描。
|
||||
- H3 任务盘删除:stop/recycle/cleanup 路径删 disk.raw
|
||||
(保留期 0,需取证须在 shutdown 前经 SSH 拷出),删失败
|
||||
记 vm_disk_delete_failed 审计 + 脏标记;gc_stale_disks
|
||||
扫描非 live VM 的残留盘并审计。
|
||||
- H4 默认值改 provisioned:CHV_HOST_KEY_MODE 默认
|
||||
provisioned,keyscan 只作显式 opt-in;旧测试的
|
||||
CHV_SSH_FORWARD=1 等实验室宣称已明确标注为实验室路径,
|
||||
不再写成生产默认。
|
||||
- H5 seed.iso 权限与私钥清理:seed.iso 生成即 0600;宿主
|
||||
私钥在 seed 盘建好后即删(只留公钥 + known_hosts),删
|
||||
失败 fail-fast(VM 尚未启动);私钥 chmod 失败不再静默;
|
||||
启动/清理时 cleanup_stale_secrets 扫描残留私钥与 seed。
|
||||
- H6 state_dir 出 /tmp:默认
|
||||
~/.local/share/codex-app-server/chv-vms,preflight 建目录
|
||||
并保证 0700(失败即 state_dir_insecure);api-socket 组/
|
||||
其它可写即 fail-fast(socket_permissions_insecure;仅禁
|
||||
写位,VMM 默认 0755 类权限不误杀——VMM 默认值代码控制
|
||||
不了,真机权限仍须按清单核对)。
|
||||
- H7 tap 隔离校验:NET_JSON 透传前校验 tap/mac/ip 跨 VM
|
||||
不重复(wake 成功后登记 net.json,stop 时释放预约),
|
||||
共用即拒绝并审计;东西向隔离的宿主防火墙部分见 §11.4,
|
||||
不写进代码假装已防。
|
||||
|
||||
### 11.2 Medium(每项有修复或有理由的结论)
|
||||
|
||||
- M1 SSH 远程命令转义:远程 token 全部 shlex.quote 后仍以
|
||||
列表返回(ssh 拼接后远程解析结果与本地 list 语义一致),
|
||||
prompt 里的 ;/$()/引号不再改写执行语义。边界写清:客内
|
||||
已是 root,本项防的是语义篡改与审计失真,不是客内提权。
|
||||
- M2 清理失败同名复用旧 key:彻底根治,选“清理失败即拒起”
|
||||
(脏标记 + provision/wake 双重拒绝),不再写“已审计即可”。
|
||||
- M3 磁盘配额默认与快照上限:CHV_DISK_GB 默认 50 GB
|
||||
(未设不再不查,0/负数拒绝);快照每 VM 数量默认 10、
|
||||
总量默认 20 GB(超帽最旧优先剪枝)、保留期默认 30 天。
|
||||
- M4 供应链版本钉死:已钉死——云镜像 SHA256 必填(干跑验
|
||||
“已设”、全量验哈希)、Ubuntu 版本与 codex 版本均为必填
|
||||
环境变量;BUILDINFO 新增 versions(构建时从客机读回的
|
||||
dpkg + pip freeze 实际版本,可审计、可复现比对)。未钉死
|
||||
的清单:apt/pip 事前精确版本 lockfile(须在构建宿主上
|
||||
生成一次、再固化进脚本,本轨无构建宿主、未编造版本号);
|
||||
Playwright Chromium 二进制版本同样只随 playwright pip
|
||||
版本记录,未单独钉死。
|
||||
- M5 审计补齐:host_key_generated/host_key_captured/
|
||||
host_key_mismatch(stderr 含钉死拒绝标记时立即 fail-fast,
|
||||
不轮询成普通超时)、preflight_failed、wake 前置失败
|
||||
(容量/配额/net 冲突)均独立审计;审计回调失败记宿主
|
||||
error 日志,不再双重静默。worker 层 _audit 本身仍
|
||||
best-effort(未改,见 §7 既定语义)。
|
||||
- M6 reuse 实验标志单租户校验:代码能校验的已校验——
|
||||
实验放行须同时声明 CHV_SINGLE_TENANT_PROJECT,否则
|
||||
preflight 拒绝(reuse_tenant_undeclared);不能校验的
|
||||
部分明写:代码无法验证数据库里是否真的只有一个租户,
|
||||
声明 + 审计就是本项的机制,不假装已校验。
|
||||
- M7 容量计数残留 socket 误伤:只有真能连通的 socket 才
|
||||
计数,空文件/死 socket 不计入 CHV_MAX_VMS;竞态窗口
|
||||
写清(刚启动未 listen 的 VMM 会被判死,容量门是护栏
|
||||
不是分布式锁)。
|
||||
- M8 sshd 加固进构建:build.sh 写入
|
||||
/etc/ssh/sshd_config.d/99-chv.conf(PasswordAuthentication no、
|
||||
PermitRootLogin prohibit-password、ChallengeResponseAuthentication no、
|
||||
KbdInteractiveAuthentication no、PubkeyAuthentication yes),
|
||||
并经 sshd -t + grep 断言;README 声称与构建一致。
|
||||
|
||||
### 11.3 Low(顺手做)
|
||||
|
||||
- L1 磁盘拷贝符号链接:目标是链接时先删链接本身再拷贝,
|
||||
不跟随写入目标文件(TOCTOU 窗口已在 disk.py 写清,
|
||||
state_dir 0700 是前提)。
|
||||
- L2 磁盘/快照路径白名单:task_disk/seed_disk 须在
|
||||
state_dir 内(reuse 的基础镜像本身除外);file:// 快照
|
||||
须在 state_dir 或 CHV_SNAPSHOT_DIR 内,远端协议须显式
|
||||
CHV_ALLOW_REMOTE_SNAPSHOT=1。
|
||||
- L3 失败审计:preflight/容量/配额/net 冲突失败均发
|
||||
vm_failed 审计(见 M5)。
|
||||
- L4 CHV_NET 与 CHV_NET_JSON 双变量:双设时发
|
||||
net_config_dual 警告审计(不硬拒,CHV_NET 是手动直接
|
||||
启动命令的透传值);NET_JSON 条目字段白名单
|
||||
(tap/mac/ip/mask),未知字段直接拒绝。
|
||||
- L5 WarmPool 预热盘与任务盘对应关系:定义已写进
|
||||
pool.acquire 文档——预热盘是干净快照盘,不是任务盘;
|
||||
本轨定案为实验性(ADR-5),不再作为接线待办。
|
||||
|
||||
### 11.4 宿主防火墙部署清单(代码做不到的部分,缺一不可)
|
||||
|
||||
- 每 VM 独立 tap + 独立 mac + 独立静态 ip(运维分配,
|
||||
代码在 wake 时校验冲突,但分配本身是运维责任);
|
||||
- tap 桥接默认 DROP,只放行到 egress 代理的流量
|
||||
(iptables/nftables:默认 DROP + 仅代理地址/端口 ACCEPT,
|
||||
不给 tap 直连外网的 NAT,不把 tap 桥接到宿主内网网段);
|
||||
- 东西向隔离:VM 间默认禁通(每 VM 独立 tap + 桥接隔离,
|
||||
或每 tap 独立网段 + 转发禁令),ARP spoof 场景下 SSH
|
||||
钉死仍是第二道门,不是唯一门;
|
||||
- 宿主进程隔离:worker 用户最小化、VMM api-socket 所在
|
||||
state_dir 0700 且仅 worker 用户可访、宿主启用 hidepid
|
||||
收紧 /proc 可见性、进程表采集不记录完整 cmdline;
|
||||
- 部署后核对:state_dir 权限、socket 实际权限、
|
||||
tap 规则实测(见 §11.5 否定测试)。
|
||||
|
||||
### 11.5 仍需真机验证的清单(本轨一律未做,不声称已验)
|
||||
|
||||
1. 否定测试:客内 unset 代理后 curl 外网必须不通;
|
||||
VM A 扫 VM B / spoof B 的 ip 后 SSH 必须被钉死拒绝;
|
||||
2. provisioned 首启时序:seed 先应用再起 sshd、不重生成
|
||||
host key;sshd -T 实际输出与密码登录关闭;
|
||||
3. 宿主 SSH argv 实测:真机 ps 下网关 key 不可见;
|
||||
4. 长跑/崩溃注入:残留盘总量、快照增长、残留 socket 与
|
||||
seed/私钥残留的实际表现、GC 覆盖;
|
||||
5. 压测:配额与容量门在并发下的实际表现;
|
||||
6. 渗透类:任何以“拿到宿主 shell / 读到别租户数据”为
|
||||
目标的真实攻击尝试——本轨一律未做,静态通过不等于
|
||||
已证明不可逃逸;
|
||||
7. 审计 verdict:以上全部通过后,才能把 verdict 从 No-Go
|
||||
改为“有条件启用(默认关闭 + 灰度单租户/受信项目)”,
|
||||
不直接全量多租户。
|
||||
|
||||
### 11.6 本轨明确不做
|
||||
|
||||
- VM 控制按钮、预热池管理页、成本/配额新页面(前端零改动);
|
||||
- 真机动态渗透、压测(无 KVM,做不了,只列清单);
|
||||
- WarmPool dispatch 前查池接线(已定案不做,见 ADR-5;重启须先
|
||||
实现任务盘重供给并补不变量测试)。
|
||||
|
||||
## 12. 架构 9.5+ 提升轨 feat-chv-arch95plus(堆叠于 #252 之上)
|
||||
|
||||
> 验证方式标注(本轨新增,逐条不混淆):静态 = 代码 + mock
|
||||
> 测试已覆盖;真机 = 须 KVM 宿主验证;压测 = 须并发负载验证。
|
||||
> 凡写“代码已做”而未标真机/压测的,一律只代表静态通过。
|
||||
|
||||
- 长函数:cloud/chv/ 全部函数 ≤50 行(静态,mock 测试覆盖
|
||||
提取后的 stop/wait/restore/清理/接收/校验/组装段)。
|
||||
- WarmPool 生产隔离:生产路径只调 reconcile facade(静态);
|
||||
预热收益仍是实验性,真机收益未验证(真机)。
|
||||
- #215 对齐:目录门与 tmpfs 口径已落代码(静态);超时与
|
||||
美元配额无映射,#215 合并后替换为 import 仍须重跑测试
|
||||
(合并后,静态)。
|
||||
- 审计失败:记日志 + 计数 + 不中断(静态);审计完整性在
|
||||
DB 故障下的实际表现仍须真机/故障注入验证(真机)。
|
||||
- 供应链:BUILDINFO 事后 lockfile 校验脚本已落(静态);
|
||||
事前 apt/pip 精确 lockfile 仍缺(须构建宿主,外部)。
|
||||
- 只读 VM 字段:chv_vm_dead 派生自 worker 已写回的
|
||||
interrupted_reason(静态,mock 测试覆盖);live VM 状态
|
||||
仍无字段,VM 控制按钮仍不做(后端无控制端点 + 安全 No-Go)。
|
||||
- 合并前 diff 复核清单(合并时逐项打勾,不在本轨内做):
|
||||
server 白名单、worker 断点分支、sandbox re-export、
|
||||
console.html 构建产物与 src 一致、#215 合并后的 import
|
||||
替换点。
|
||||
- 真机验证清单(沿用 §11.5,另加本轨项):VM 死亡横幅在
|
||||
真任务中断后是否出现、审计失败计数在 DB 故障时是否
|
||||
累加且任务不中断、快照白名单目录门在符号链接逃逸时
|
||||
是否拒绝。
|
||||
- 灰度启用清单(启用时逐项打勾):CI 变绿 → 按 #216→#252→
|
||||
本 PR 顺序合并 → KVM 宿主 + 防火墙清单 → 镜像构建发布
|
||||
→ §11.5 + 本节真机清单全过 → 默认关闭下单租户/受信项目
|
||||
灰度 → 全量多租户(须另行决策,不在本轨内默认开启)。
|
||||
|
||||
## 13. to10 提升轨 feat-chv-to10(堆叠于 #257 之上)
|
||||
|
||||
本节是准备工作,不是启用结论:代码侧的 to10 项已做,
|
||||
生产就绪与动态安全仍未到 10 分(见 §13.4 外部关卡)。
|
||||
|
||||
### 13.1 合并前 diff 复核清单(以当前 main 头预填)
|
||||
|
||||
- 当前 main 头(本轨核查时):`aebc6238a613066316ec18796b83ea8fc2d8835e`
|
||||
(Merge PR #201)。链上 10 个 PR 在核查时全部
|
||||
OPEN / MERGEABLE(#216 base main,#221→#257 依次堆叠,
|
||||
本 PR base 为 feat-chv-arch-maint-fe)。合并时必须重新
|
||||
核查:main 头若已前进,整条链须逐个 rebase 并重跑测试,
|
||||
绝不直接合并旧基底。
|
||||
- 逐项复核点(合并执行者逐项打勾,任一项不符即停):
|
||||
1. server 任务创建白名单含 `cloud-hypervisor`,环境创建
|
||||
白名单为 local/docker/cloud-hypervisor(不含 cloud),
|
||||
两处白名单差异仍在前端文案里写清。
|
||||
2. worker 断点分支:Local 原路径不变,CHV 经
|
||||
`worker_adapter.call_goal_resume_breakpoint`(薄包装名
|
||||
`_call_goal_resume_breakpoint` 仍在 worker.py,旧测试
|
||||
导入点不变)。
|
||||
3. sandbox re-export:`cloud/chv` 不再 export WarmPool
|
||||
(#257 已移实验模块),`cloud/sandbox.py` 的 CHV
|
||||
re-export 与 `cloud/chv/__init__.py` 一致。
|
||||
4. 前端产物:`cloud/console.html` 与 `dist/index.html`
|
||||
的 sha256 一致(本轨:`269820d1…f389`,353163 字节),
|
||||
产物内含新文案(“无法判断 VM 状态”1 处、
|
||||
“CHV worker 在线数未知”3 处);合并时重建后复核,
|
||||
不直接沿用本轨哈希。
|
||||
5. `package.json` 的 vitest 测试脚本与 lockfile 增量只含
|
||||
vitest 新条目,无旧包版本变动。
|
||||
6. 本轨新增文件:`cloud/chv/worker_adapter.py`、
|
||||
`cloud/tests/test_chv_to10.py`、
|
||||
`chv-guest/generate_lockfile.py`;本轨改动文件:
|
||||
`cloud/worker.py`(薄包装 + 删多余 `inspect` import)、
|
||||
`cloud/chv/executor.py`(审计落盘 + 两函数抽 helper)、
|
||||
`cloud/chv/config.py`、`disk.py`、`hardening.py`、
|
||||
`hostkeys.py`、`ssh_transport.py`、`experimental_pool.py`、
|
||||
`compat215.py`(类型注解 + ruff 整理)、前端 6 文件。
|
||||
7. CI:链上全部 PR 的 `verify` 仍是账单故障秒级失败模式
|
||||
(本轨核查时 #257 等均为 UNSTABLE,非代码失败);
|
||||
CI 变绿前不合并,绝不改 CI 配置绕过。
|
||||
8. 旧链 #199→#205→#207 与 #209–#215 仍全部 OPEN:
|
||||
合并前复核它们与本链的交集(server 白名单、worker
|
||||
断点分支、sandbox re-export),有冲突先解决冲突方,
|
||||
不静默覆盖。
|
||||
|
||||
### 13.2 真机验证清单(用例与验收标准细化)
|
||||
|
||||
以下用例全部待 KVM 宿主,验收标准是“通过才算”,不是
|
||||
“跑过就算”;任一用例未跑即记为未验证,不写进已验。
|
||||
|
||||
- V-1 冷启动:CHV worker 在 KVM 宿主上起 VM → guest
|
||||
SSH 就绪(钉死 known_hosts,provisioned 模式)→ 跑通
|
||||
一个真实任务(host.py 在客机内执行)→ 任务完成且 VM
|
||||
已销毁、任务盘已删。验收:任务状态 completed,宿主
|
||||
state_dir 无该任务残留盘与 seed。
|
||||
- V-2 seed 首启时序:provisioned 模式下客机 cloud-init
|
||||
在 sshd 启动前应用 seed(host key 不再生、客内密钥文件
|
||||
0600)。验收:`sshd -T` 显示 PasswordAuthentication no
|
||||
等加固项,known_hosts 钉死校验通过,非 TOFU 回落。
|
||||
- V-3 快照恢复:任务中途 hibernate(pause + snapshot +
|
||||
shutdown)→ restore 后显式 resume → 断点续跑完成。
|
||||
验收:恢复后 known_hosts 不变(不重预置),快照文件
|
||||
落盘非空且经 `snapshot_not_persisted` 校验。
|
||||
- V-4 网关真实任务:sub2api 网关任务在 CHV 上跑通,
|
||||
密钥只走 seed 客内文件,宿主 `ps` / `/proc/*/cmdline`
|
||||
里看不到密钥值。验收:任务完成 + 宿主进程表抽查无密钥。
|
||||
- V-5 取消与配额真杀:同步 SSH 阻塞中取消任务、配额超限
|
||||
时,看门狗调 `cancel_vm` 真关 VM。验收:VM 进程消失,
|
||||
任务状态 interrupted 且原因写回正确。
|
||||
- V-6 审计落盘:DB 审计回调失败时,`chv-audit-spill.jsonl`
|
||||
里有该事件行、`audit_failure_count` +1、任务不中断。
|
||||
验收:落盘行 event/task_id/detail 与丢失事件一致。
|
||||
- V-7 动态安全负向:客机 root unset 代理后直连(应被
|
||||
tap/防火墙挡住)、跨 VM 探活与 IP 冒用(SSH 钉死应拒)、
|
||||
崩溃注入后残留扫描(私钥/seed/盘应被清理或脏标记拒起)。
|
||||
验收:每一项都有“被挡住”的证据,不是有代码就过。
|
||||
- V-8 负载:4 VM 并发(`CHV_MAX_VMS` 默认值)下容量门
|
||||
计数含重启后残留 socket 扫描,死 socket 不占名额、活
|
||||
VM 不被超发。验收:并发任务全部完成或明确排队拒绝,
|
||||
无静默降级到 Docker/Local。
|
||||
- V-9 供应链清单:`generate_lockfile.py` 在构建宿主上跑
|
||||
出真实 lockfile,`--verify` 通过且 unresolved 为空;
|
||||
再用 `check_lockfile.py` 对 BUILDINFO.versions 复核。
|
||||
验收:lockfile 由构建宿主产出,不是本轨合成数据。
|
||||
|
||||
### 13.3 灰度启用清单
|
||||
|
||||
- 阶段 0(合并后、默认关闭):合并整条链,CHV 仍默认
|
||||
关闭,不进生产任务路由;先跑 §13.2 的 V-1~V-9。
|
||||
- 阶段 1(单租户可信项目):只对明确指定的单租户项目
|
||||
开 CHV(`CHV_REUSE_SINGLE_TENANT_EXPERIMENT` 与
|
||||
`CHV_SINGLE_TENANT_PROJECT` 的声明口径不变,复用
|
||||
实验仍需同时声明,两者缺一即 preflight 拒绝)。
|
||||
- 阶段 2(受限多租户):放开到受邀项目,监控 VM 残留盘、
|
||||
审计落盘、容量门与快照配额三项指标一周无异常。
|
||||
- 阶段 3(全量多租户):仅在阶段 2 指标干净 + 动态安全
|
||||
审计结论由 No-Go 改为有条件启用后,才可讨论;绝不
|
||||
从阶段 0 直跳全量。
|
||||
|
||||
### 13.4 外部关卡(本轨代码做不到,逐项列出)
|
||||
|
||||
1. CI 账单恢复变绿(硬门禁,合并前提)。
|
||||
2. 链上 10 个 PR 按顺序合并 + 本 PR 合并(需先过 §13.1)。
|
||||
3. 用户侧 provision KVM 宿主(裸金属或嵌套虚拟化实例,
|
||||
须见 `/dev/kvm`;生产宿主与本 VM 均无 KVM,无法替代)。
|
||||
4. 客机镜像真实构建与发布(`build.sh` 全量跑通 + V-9)。
|
||||
5. §13.2 真机验证全部通过(含 V-7 动态安全负向)。
|
||||
6. 负载测试(V-8)与安全审计结论更新(No-Go → 有条件启用)。
|
||||
7. 前端 live 点击验证(用户侧 live browser 检查清单见
|
||||
本 PR 描述;回环地址限制下浏览器任务无法代劳)。
|
||||
|
||||
## 14. 收尾轨 feat-chv-closeup(堆叠于 #264 之上)
|
||||
|
||||
本节是准备工作,不是启用结论。本轨只到 PR OPEN,不合并、
|
||||
不部署;生产就绪与动态安全的外部关卡(§13.4)原样未动。
|
||||
|
||||
### 14.1 本轨量化结果(实测,非上一轨回执转述)
|
||||
|
||||
(本节数字为 closeup 轨快照,历史数字保留不改;
|
||||
P1 修复轨后的最新数字见 §15,§14.1 不再是最新口径。)
|
||||
|
||||
- 后端:232 项 → 317 项(新增 `cloud/tests/test_chv_closeup.py`
|
||||
85 项),全过。命令(仓库根目录,cwd 须为根目录——
|
||||
在 `cloud/` 下跑会撞 `cloud/queue.py` 遮蔽 stdlib queue,
|
||||
mypy 同理):
|
||||
`python3 -m unittest cloud.tests.test_chv_arch95
|
||||
cloud.tests.test_chv_arch95plus cloud.tests.test_chv_arch_maint_fe
|
||||
cloud.tests.test_chv_fix cloud.tests.test_chv_gaps
|
||||
cloud.tests.test_chv_optimize cloud.tests.test_chv_secfix
|
||||
cloud.tests.test_chv_to10 cloud.tests.test_cloud_hypervisor
|
||||
cloud.tests.test_chv_closeup`
|
||||
- 覆盖率(同命令 + `coverage run --source=cloud/chv`):
|
||||
chv 总 87%(1925 行,缺 258)→ 96%(缺 80);
|
||||
`executor.py` 85%(缺 157)→ 100%(缺 0);
|
||||
`worker_adapter.py` 82%(缺 16)→ 100%(缺 0)。
|
||||
本轨基线与上一轨回执的行数口径略有出入(上一轨称
|
||||
1826 行/缺 242,本轨同命令实测 1925 行/缺 258),
|
||||
本轨前后对比只用本轨同一命令,不混用上一轨数字。
|
||||
- 剩余未覆 80 行(按模块,全部在 executor/worker_adapter
|
||||
之外,本轨未声称覆盖):
|
||||
`api_client.py` 11 行(socket 层 chunked/超时/异常分支,
|
||||
需 socket 级故障注入);`compat215.py` 2 行(旧兼容回落);
|
||||
`config.py` 5 行(非法 env 构造分支);`disk.py` 22 行
|
||||
(快照登记/剪枝边角与配额异常分支);`experimental_pool.py`
|
||||
16 行(实验池,非生产路径);`hardening.py` 11 行
|
||||
(真实 socket 权限与 net 校验边角);`hostkeys.py` 13 行
|
||||
(ssh-keygen/genisoimage 子进程失败分支)。以上均非死
|
||||
代码的断言——未逐行证伪,留待后续轨按模块补,不在本
|
||||
轨凑数。
|
||||
- mypy:silent 口径 `cloud/chv/` 0 错(不变);
|
||||
默认口径 `cloud/marker_tools.py` 仍 2 错(见 §14.3,
|
||||
本轨未改该文件)。
|
||||
- ruff:`--select F,E9,W,I` 口径 `cloud/chv/` +
|
||||
本轨新测试 0 错(不变;默认全规则基线不声称通过)。
|
||||
- 前端:Vitest 23 项 → 36 项(新增
|
||||
`src/api/chv.closeup.test.ts` 13 项),全过。
|
||||
|
||||
### 14.2 审计落盘文件运维 runbook(ADR-14 取舍不变,只补可操作性)
|
||||
|
||||
- 定位:`<state_dir>/chv-audit-spill.jsonl` 与轮转文件
|
||||
`<state_dir>/chv-audit-spill.jsonl.old`。`state_dir`
|
||||
默认 `~/.local/share/codex-app-server/chv-vms`,
|
||||
可由 `CHV_STATE_DIR` 改址;以部署的实际值为准,
|
||||
不要猜。目录本身 0700(见 §13.1 的 H6 门)。
|
||||
- 触发:只有 `_emit` 的审计回调抛错时才写一行;回调
|
||||
成功、无回调(只记日志)时不写。触发时宿主日志必有
|
||||
`chv_audit_failed`,且 `audit_failure_count` +1——
|
||||
落盘行是“哪次事件丢了”的唯一取证,计数只是数字。
|
||||
- 格式:JSONL,每行 `{ts, event, kind, detail, task_id}`。
|
||||
`detail` 沿用 `_emit` 契约,只含 VM 名/路径/URL/kind,
|
||||
绝不含密钥值;读文件时仍按敏感文件对待(0700 目录内)。
|
||||
- 轮转:当前文件 >1MB 时,先 `os.replace` 成 `.old`
|
||||
再写新行;只留最新两代,更早的丢失事件会丢——
|
||||
这是 ADR-14 已定的取舍,不是 bug,不要指望它当
|
||||
完整审计账本。
|
||||
- 读取(宿主侧,只读):
|
||||
`cat <state_dir>/chv-audit-spill.jsonl`;
|
||||
按事件过滤:`jq -c 'select(.event=="vm_failed")'
|
||||
<state_dir>/chv-audit-spill.jsonl`;
|
||||
两代一起看时先 `.old` 后当前文件(时间顺序)。
|
||||
- 清理:`.old` 可随时删(删的是已被轮转覆盖的旧代);
|
||||
当前 `.jsonl` 只有在已人工核对、且确认丢失事件已
|
||||
无需追查时才删——删了就再也找不回那批事件。
|
||||
落盘文件不是 DB 替代:任务 timeline 查询看不到它,
|
||||
不要把它当成审计表去对账;审计完整性本身仍无保证
|
||||
(见 ADR-14),保证需要 DB 事务级审计写入,那是
|
||||
worker 全局语义改动,不在本轨。
|
||||
- 落盘写失败:只记 `chv_audit_spill_failed` 日志,
|
||||
绝不中断任务;此时连落盘都没有,唯一信号就是这条
|
||||
日志与计数。
|
||||
|
||||
### 14.3 mypy 默认口径剩余 2 错评估(结论:本轨不修)
|
||||
|
||||
- 位置:`cloud/marker_tools.py:142`,`try: from zoneinfo
|
||||
import ZoneInfo / except Exception: ZoneInfo = None`。
|
||||
mypy 把 import 推断为 `type[ZoneInfo]`,再把 `None`
|
||||
赋值判为两错(misc + assignment)。
|
||||
- 仅标注修法(沙箱验证过,未进仓库):在 import 区后加
|
||||
`from typing import Any` 与 `ZoneInfo: Any`(标注语句
|
||||
本身不产生绑定),默认口径 2 错 → 0 错;运行时
|
||||
`ZoneInfo` 仍是 stdlib `zoneinfo.ZoneInfo` 同一对象,
|
||||
行为不变。验证在 `/tmp` 副本上做,不碰仓库文件。
|
||||
- 不修的理由(两条,任一条已足够):① 该文件属标记
|
||||
工具链(git 历史为标记工具提交),不在本链 §13.1
|
||||
文件清单内,本轨改它就是动其它链的文件,徒增与
|
||||
仍 OPEN 的审计/标记 PR 的 rebase 冲突面;② CHV 链
|
||||
自己的门是 silent 口径(`--follow-imports=silent
|
||||
cloud/chv/`),该口径已 0 错,默认口径的 2 错是
|
||||
作用域外既有问题。修法留给 marker_tools 的归属轨,
|
||||
本轨只把配方写在这里,不为清零越界改文件。
|
||||
|
||||
### 14.4 合并前检查脚本(只读)
|
||||
|
||||
- 脚本:`tools/chv_merge_precheck.py`,用法见文件头。
|
||||
只读:`gh pr view` / `gh api` 查询 + 本地 `git` 读
|
||||
remote,不 fetch、不 checkout、不 merge、不改 CI。
|
||||
退出码:0 就绪、1 未就绪、2 查询失败(结论不可信)。
|
||||
- 本轨实测(未含本轨新 PR,建好后须追加新 PR 号重跑):
|
||||
`python3 tools/chv_merge_precheck.py` → 退出码 1,
|
||||
结论 NOT-READY:11 个 PR 全部 OPEN / MERGEABLE,
|
||||
链衔接全部 OK,当前 main 头仍 `aebc6238`(与 §13.1
|
||||
一致),但 11 个 PR 的 CI 全部 `FAIL [verify:FAILURE]`、
|
||||
mergeState 全部 UNSTABLE。脚本不诊断失败原因,
|
||||
不把秒级失败改判——账单恢复变绿前结论保持 NOT-READY。
|
||||
|
||||
### 14.5 灰度三阶段可执行清单(准备工作,未执行、未验证)
|
||||
|
||||
通用停止条件(任一触发即停在该阶段,不进下一阶段):
|
||||
`vm_failed` 审计激增、脏标记(`dirty_marker`)出现、
|
||||
`chv-audit-spill.jsonl` 持续增长、残留盘/残留密钥扫描
|
||||
清出本阶段不应有的项、任一任务静默降级到 Docker/Local。
|
||||
通用回滚:停止向 CHV 路由新任务(改回 docker/local
|
||||
执行器选择),已起 VM 走取消/关闭路径(`cancel_vm` /
|
||||
`shutdown_vm`);代码回滚另议(revert 合并),不与
|
||||
“停路由”混为一谈。
|
||||
|
||||
- 阶段 0(合并后、默认关闭):
|
||||
1. 按 §14.4 脚本逐项核对,退出码 0 前不合并;合并
|
||||
顺序即 §14.4 的链序,不得跳号。
|
||||
2. 合并后确认 CHV 仍默认关闭(无生产任务路由到
|
||||
`cloud-hypervisor`,任务创建白名单外的执行器值
|
||||
仍被后端拒绝)。
|
||||
3. 在 KVM 宿主(须见 `/dev/kvm`,本 VM 与生产宿主
|
||||
均无 KVM,不能替代)上跑 §14.6 的 V-1~V-9,
|
||||
全部通过才可谈阶段 1;任一未跑即记未验证。
|
||||
回滚点:合并本身可 revert;未进生产路由前无任务影响。
|
||||
- 阶段 1(单租户可信项目):
|
||||
1. 只对明确指定的单租户项目派发 CHV 任务;复用
|
||||
实验仍需 `CHV_REUSE_SINGLE_TENANT_EXPERIMENT` 与
|
||||
`CHV_SINGLE_TENANT_PROJECT` 同时声明,缺一即
|
||||
preflight 拒绝(代码行为,不放宽)。
|
||||
2. 检查点:每任务 VM 销毁后 `state_dir` 无该任务
|
||||
残留盘与 seed;`chv-audit-spill.jsonl` 无新增;
|
||||
取消路径真关 VM(V-5 口径)。
|
||||
3. 停止条件见通用条件;回滚:该项目任务改回原
|
||||
执行器,CHV worker 停接新任务。
|
||||
- 阶段 2(受限多租户):
|
||||
1. 放开到受邀项目,连续观察一周:VM 残留盘、
|
||||
审计落盘、容量门(`CHV_MAX_VMS`,含重启后残留
|
||||
socket 扫描)、快照配额四项指标。
|
||||
2. 检查点:每周复核四项指标零异常 + §14.2 落盘
|
||||
文件复核;任一指标异常即退回阶段 1。
|
||||
3. 停止条件见通用条件;回滚:收紧到阶段 1 项目集。
|
||||
- 阶段 3(全量多租户):仅在阶段 2 指标干净 +
|
||||
动态安全审计结论由 No-Go 改为有条件启用后,才可
|
||||
讨论;绝不从阶段 0 直跳全量。本轨不给阶段 3 的
|
||||
时间表,不写成已就绪。
|
||||
|
||||
### 14.6 真机验证用例 V-1~V-9 验收检查表(全部未验证)
|
||||
|
||||
每行格式:用例 — 验收标准(§13.2 细化)— 证据栏(待填)。
|
||||
状态栏一律“未验证”,跑过才改,跑过≠通过。
|
||||
|
||||
- [ ] V-1 冷启动 — 任务 completed,宿主 state_dir 无该
|
||||
任务残留盘与 seed。证据:______ 状态:未验证
|
||||
- [ ] V-2 seed 首启时序 — `sshd -T` 显示
|
||||
PasswordAuthentication no 等加固项,known_hosts 钉死
|
||||
校验通过,非 TOFU 回落。证据:______ 状态:未验证
|
||||
- [ ] V-3 快照恢复 — 恢复后 known_hosts 不变,快照文件
|
||||
落盘非空且经 `snapshot_not_persisted` 校验。证据:______
|
||||
状态:未验证
|
||||
- [ ] V-4 网关真实任务 — 任务完成 + 宿主进程表抽查无密钥
|
||||
(密钥只走 seed 客内文件)。证据:______ 状态:未验证
|
||||
- [ ] V-5 取消与配额真杀 — VM 进程消失,任务 interrupted
|
||||
且原因写回正确。证据:______ 状态:未验证
|
||||
- [ ] V-6 审计落盘 — 落盘行 event/task_id/detail 与丢失
|
||||
事件一致,`audit_failure_count` +1,任务不中断。
|
||||
证据:______ 状态:未验证
|
||||
- [ ] V-7 动态安全负向 — 直连被 tap/防火墙挡、跨 VM 探活
|
||||
与 IP 冒用被 SSH 钉死拒、崩溃注入后残留被清理或脏
|
||||
标记拒起,每项都有“被挡住”的证据。证据:______
|
||||
状态:未验证
|
||||
- [ ] V-8 负载 — 4 VM 并发下容量门计数含重启后残留
|
||||
socket 扫描,死 socket 不占名额、活 VM 不被超发,
|
||||
无静默降级。证据:______ 状态:未验证
|
||||
- [ ] V-9 供应链清单 — 构建宿主产出真实 lockfile,
|
||||
`--verify` 通过且 unresolved 为空,再用
|
||||
`check_lockfile.py` 对 BUILDINFO.versions 复核。
|
||||
证据:______ 状态:未验证
|
||||
|
||||
### 14.7 本轨逐项评分(目标 10,无一项硬写 10)
|
||||
|
||||
(本节评分为 closeup 轨快照;P1 修复轨后的评分更新见
|
||||
§15.7,其中“80 行未覆”“317 项/36 项”扣分理由已过期,
|
||||
不再作为当前扣分依据。)
|
||||
|
||||
- 代码质量 9.8 → 9.8,未达 10:executor/worker_adapter
|
||||
覆盖到 100%,但 chv 仍有 80 行未覆(非 executor
|
||||
模块),且默认口径 mypy 余 2 错在作用域外。差:代码内。
|
||||
- 架构 9.7 → 9.7,未达 10:本轨未改架构,ADR-13/14
|
||||
取舍不变。差:外部(生产验证后才可复评)。
|
||||
- 可维护性 9.7 → 9.8,未达 10:新增合并检查脚本与
|
||||
§14 运维/灰度文档,落盘可操作性补齐;但跨链文件
|
||||
的 mypy 归属未落。差:代码内(小)。
|
||||
- 文档 9.8 → 9.9,未达 10:§14 补齐 runbook、灰度
|
||||
清单、V 检查表、评分表;仍缺真机证据填表。差:外部。
|
||||
- 测试 9.7 → 9.8,未达 10:后端 317 项、前端 36 项全过,
|
||||
executor 100%;但 chv 总覆盖 96% 未满,且真机用例
|
||||
0 项已跑。差:代码内(80 行)+ 外部(真机)。
|
||||
- 静态安全 9.6 → 9.6,未达 10:取舍不变,只补落盘
|
||||
运维;lockfile 仍待构建宿主产出,绝不编造。差:外部。
|
||||
- 前端 9.5 → 9.6,未达 10:只读四态与可用性文案边角
|
||||
全覆盖(显式 false、undefined、优先级、旧 bug 文案
|
||||
不复发、120 秒边界);控制按钮与新页面仍不做,
|
||||
live 点击未验。差:代码内(有意不做)+ 外部(live)。
|
||||
- 生产就绪 5.0 → 5.5,未达 10:仅因合并/灰度准备
|
||||
工作(可跑脚本 + 可执行清单)小幅上升;CI 仍
|
||||
UNSTABLE、未合并、未部署、真机 0 项。差:外部为主。
|
||||
- 动态安全 2.0 → 2.0,未达 10:本轨无任何动态验证,
|
||||
V-7 未跑。差:外部(KVM 宿主 + 审计结论)。
|
||||
## 15. P1 修复轨 feat-chv-p1fix(堆叠于 #270 之上)
|
||||
|
||||
本节是工程生产级审核(`~/workspace/chv-prod-audit/REPORT.md`,
|
||||
审核对象 #270,提交 `8c8d62f`)§5 的 P1-01~P1-10 修复轨。
|
||||
本轨只到 PR OPEN,不合并、不部署、不做合并试验(P0-01
|
||||
仍未做,见 §15.6);P2-01~P2-21 一律不动。
|
||||
|
||||
### 15.1 本轨量化结果(实测)
|
||||
|
||||
- 后端:362 项 → 389 项(新增 `cloud/tests/test_chv_p1fix.py`
|
||||
27 项),全过。命令(仓库根目录):
|
||||
`python3 -m unittest cloud.tests.test_chv_arch95
|
||||
cloud.tests.test_chv_arch95plus cloud.tests.test_chv_arch_maint_fe
|
||||
cloud.tests.test_chv_fix cloud.tests.test_chv_gaps
|
||||
cloud.tests.test_chv_optimize cloud.tests.test_chv_secfix
|
||||
cloud.tests.test_chv_to10 cloud.tests.test_cloud_hypervisor
|
||||
cloud.tests.test_chv_closeup cloud.tests.test_chv_cover100
|
||||
cloud.tests.test_chv_p1fix`
|
||||
- 覆盖率(同命令 + `coverage run --source=cloud/chv`,再
|
||||
`--append test_cloud_hypervisor`,`.coverage` 写 /tmp):
|
||||
行覆盖 100%(2097 行,缺 0);分支覆盖 99%
|
||||
(714 分支,28 处部分分支未全走——行 100% 不等于
|
||||
分支 100%,PR 描述按此口径写,不写裸“100%”)。
|
||||
`cloud/chv/__init__.py` 计入分母:4 行,100%(仅
|
||||
docstring + 3 个 re-export,coverage.py 口径;审核报告
|
||||
的 trace 口径把它排除在外,两口径差在此说明,不混用)。
|
||||
审核报告 P1-09 点名的 4 处三元 else 分支(str 形态
|
||||
pause/resume/snapshot + 非 dict spill task)已由
|
||||
`test_chv_p1fix.py` 的 BranchCover 测试覆盖。
|
||||
- mypy:silent 口径 `cloud/chv/` 0 错(不变);
|
||||
默认口径 `cloud/marker_tools.py` 仍 2 错(作用域外,
|
||||
本轨未改该文件)。
|
||||
- ruff:`--select F,E9,W,I` 口径 `cloud/chv/` +
|
||||
本轨新测试 0 错(不变;默认全规则基线不声称通过)。
|
||||
- 前端:Vitest 37 项不变(`chv.test.ts` 23 +
|
||||
`chv.closeup.test.ts` 14),本轨未改前端代码;三轮
|
||||
连跑 37/37/37,无抖动回归。
|
||||
- 全量:`python3 -m unittest discover -s cloud/tests`
|
||||
(仓库根目录)1819 项全过(skipped=8),392 秒;
|
||||
上一轨引用口径 1792 项的差额来自本轨新增 27 项。
|
||||
- 长函数:AST 扫描 `cloud/chv/`,>50 行函数 0 个
|
||||
(本轨新增的 snapshot_vm / record_snapshot /
|
||||
_preflight_numeric_checks 均经提取守住 50 行门禁,
|
||||
见 ADR-17)。
|
||||
- §13.1 的 main 头 `aebc6238` 与前端产物哈希
|
||||
`269820d1…f389` 仍是 to10 轨快照,本轨不更新它们:
|
||||
合并前必须按 §13.1 逐项重跑重核,绝不把旧快照写成
|
||||
新数字(审核报告 P1-10 的要求原样保留)。
|
||||
|
||||
### 15.2 P1 逐项修法(证据:文件与测试名)
|
||||
|
||||
- P1-01(orphans.py + experimental_pool.py):失败项不进
|
||||
cleaned、不触发清库;生产 reconcile 落盘只写回失败项
|
||||
原始条目并审计 `warm_pool_reconcile_failed`,下次可重试;
|
||||
实验池失败条目留在内存条目里(文件落盘语义属 P2-16,
|
||||
本轨不改)。同步改旧测试:`test_chv_arch_maint_fe.py`
|
||||
的 shutdown 失败用例与 `test_chv_cover100.py` 的
|
||||
`test_reconcile_shutdown_failure_still_cleans`(已改名
|
||||
`test_reconcile_shutdown_failure_not_cleaned`)——代码
|
||||
与测试同改,不存在“只改一侧”。
|
||||
新测试:`ReconcileP1FixTest.test_failure_not_cleaned_registry_retained_and_retryable`。
|
||||
- P1-02(orphans.py):registry 损坏发
|
||||
`warm_pool_registry_corrupt` 审计(quarantine 后仍审计,
|
||||
不静默返回空);非法条目发
|
||||
`warm_pool_registry_invalid_entries` 审计并计数,落盘
|
||||
时丢弃已审计条目(不再每次重启重复审计)。
|
||||
新测试:`test_corrupt_registry_audited_not_silent`、
|
||||
`test_invalid_entries_audited_and_counted`。
|
||||
- P1-03(executor.py `_clean_one_vm_secrets` /
|
||||
`cleanup_stale_secrets`):返回值由 2 元组改为
|
||||
(removed_priv, removed_seed, failed),删失败发
|
||||
`host_secret_residue_delete_failed` 审计并写脏标记
|
||||
(下次 provision 见标记即 `vm_dirty_refused` 拒起),
|
||||
删失败的 VM 不进 cleaned(即使部分删掉也不算已清理)。
|
||||
旧测试 `test_clean_one_vm_secrets_remove_failures` 已
|
||||
同步改 3 元组解包与 failed 断言。
|
||||
- P1-04(executor.py):删除 `build_ssh_command` 的回环
|
||||
回落(构造失败即抛 `ssh_target_undeclared`),不设
|
||||
dry-run 例外参数——干跑同样不许猜目标;同款回落
|
||||
`_write_provisioned_known_hosts` 一并删除(wake 路径
|
||||
在 provision 前已在 `_wake_prechecks` 里严格解析,
|
||||
不受影响;wait 路径的 `capture_host_key` 本来就
|
||||
解析失败返回 False,同样严格)。显式回环仍可用:
|
||||
`CHV_SSH_HOST=127.0.0.1` 或 `CHV_SSH_FORWARD=1`。
|
||||
旧测试 `test_build_ssh_command_fallback_and_secret_not_delivered`
|
||||
与 `test_write_provisioned_known_hosts_fallback` 已
|
||||
同步改为抛错断言。
|
||||
- P1-05(新增 `cloud/chv/locking.py` + executor.py):
|
||||
选文件锁方案(非单 worker 声明):`StateLock` =
|
||||
`flock`(`state_dir/chv-state.lock`,永不删除)+
|
||||
进程内 RLock 可重入(同一实例嵌套不自锁;多实例/
|
||||
多进程靠 flock 互斥)。持有范围:wake_vm 全程、
|
||||
restore_vm 全程、cleanup_stale_vms 全程、orphans
|
||||
reconcile 全程;`_purge_dead_registry` 改为锁内快照
|
||||
键列表、逐项锁内读/删,`_stop_dead_socket_vms` 的
|
||||
登记读持锁。串行 wake 是有意的代价(容量门只有
|
||||
“查完立即占位”才成立,提前释放会在 socket 未绑定
|
||||
前留出计数窗口);已运行 VM 的任务执行不持此锁。
|
||||
新测试:`StateLockP1FixTest`(线程互斥、跨进程互斥、
|
||||
purge 并发变更加锁)、`WakeSerializationP1FixTest`
|
||||
(两路并发 wake 实测串行化)。
|
||||
- P1-06(disk.py `record_snapshot`):单条快照本身大于
|
||||
总量帽时,在任何落盘/删旧动作之前直接拒绝
|
||||
(`snapshot_quota_exceeded`),不悄悄保留超大快照;
|
||||
大小未知(size 为 None,如远端目标)时无法判定,
|
||||
沿用旧口径。新测试:
|
||||
`SingleSnapshotCapP1FixTest`(拒绝无副作用:旧文件
|
||||
与旧登记均在)。
|
||||
- P1-07(config.py + executor.py preflight):构造层
|
||||
拒绝能解析出的 CPU/内存非正值与显式任务超时非正值
|
||||
(`CHV_CPU`/`CHV_MEM`/`CHV_TASK_TIMEOUT`);preflight
|
||||
再验属性事后改写(CPU 0/负、内存 0、任务超时 0/负
|
||||
一律 `config_invalid` 干净失败,不进 VmConfig)。
|
||||
`None` 任务超时 = 不限(旧语义保留);解析不出的
|
||||
格式仍留给 preflight(构造时机分工不变)。
|
||||
新测试:`PositiveConfigP1FixTest`。
|
||||
- P1-08(disk.py + executor.py `snapshot_vm`):剪枝
|
||||
事件外发(`prune_snapshots` / `check_snapshot_limits` /
|
||||
`record_snapshot` 均接受 `events` 列表:deleted /
|
||||
delete_failed / pruned_no_file);`snapshot_vm` 据此发
|
||||
`vm_snapshot_pruned` / `vm_snapshot_prune_failed`
|
||||
审计,失败路径(白名单/上限/ch-remote/落盘校验/
|
||||
登记拒绝)统一发 `vm_snapshot_failed` 审计再抛。
|
||||
新测试:`SnapshotAuditP1FixTest`。
|
||||
- P1-09:见 §15.1(行 100% / 分支 99% 分开写;
|
||||
`__init__.py` 计入分母说明;4 分支测试在
|
||||
`BranchCoverP1FixTest`)。
|
||||
- P1-10:本节即同步结果;§14.1/§14.7 已加“已被 §15
|
||||
取代”指针(历史数字保留不改);架构文档头范围与
|
||||
ADR-17 见 `docs/chv-architecture.md`;§13.1 快照数
|
||||
字按要求保留、合并前重跑(见 §15.1 末)。
|
||||
|
||||
### 15.3 安全语义未放宽清单
|
||||
|
||||
egress 默认 fail-closed、key seed 投递(provisioned 才
|
||||
投递、keyscan 下密钥 fail-closed)、磁盘删除(删失败
|
||||
审计 + 脏标记)、默认 provisioned、tap/mac/ip 校验、
|
||||
StrictHostKeyChecking 钉死、网关 loud fail 守卫——
|
||||
本轨一律未改动,只收紧(回落删除、失败可见性、
|
||||
check→act 加锁)。前端无改动,不存在绕后端白名单
|
||||
的新路径。
|
||||
|
||||
### 15.4 新增与改动文件
|
||||
|
||||
- 新增:`cloud/chv/locking.py`(StateLock)、
|
||||
`cloud/tests/test_chv_p1fix.py`(27 项)。
|
||||
- 改动:`cloud/chv/orphans.py`、`cloud/chv/disk.py`、
|
||||
`cloud/chv/executor.py`、`cloud/chv/experimental_pool.py`、
|
||||
`cloud/chv/config.py`;旧测试同步改:
|
||||
`cloud/tests/test_chv_arch_maint_fe.py`、
|
||||
`cloud/tests/test_chv_closeup.py`、
|
||||
`cloud/tests/test_chv_cover100.py`;
|
||||
文档:本节 + `docs/chv-architecture.md`(头范围 +
|
||||
ADR-17)。
|
||||
|
||||
### 15.5 逐项完成情况
|
||||
|
||||
P1-01~P1-10 全部修完(每项均有新测试或同步改的旧
|
||||
测试,见 §15.2;无“修不动”的项)。未修完的部分:
|
||||
无。但“修完”只等于代码层完成,不等于生产就绪,
|
||||
见 §15.6。
|
||||
|
||||
### 15.6 仍未验证项(原样保留,不冒充)
|
||||
|
||||
- 合并试验 P0-01 仍未做(本轨禁令):隔离演练区以
|
||||
当前远端 HEAD 为目标的三方合并 + 全量测试未跑,
|
||||
直接合入仍可能以链侧文件覆盖 main 新增功能。
|
||||
- CI 仍 UNSTABLE(账单秒级失败,本轨未改 CI 配置)。
|
||||
- KVM 宿主 provision、客机镜像真实构建与发布未做;
|
||||
真机验证 V-1~V-9 全部未跑。
|
||||
- 前端 live 点击验证未做;渲染测试未做。
|
||||
- 旧安全审计 No-Go 未复审、未推翻。
|
||||
|
||||
### 15.7 本轨逐项评分更新(目标 10,无一项硬写 10)
|
||||
|
||||
- 代码质量 9.8 → 9.8,未达 10:P1 语义缺陷已修,但
|
||||
分支覆盖 99% 未满、长函数门禁靠提取维持(边界脆弱)。
|
||||
差:代码内(小)。
|
||||
- 测试 9.8 → 9.9,未达 10:后端 389 项、前端 37 项
|
||||
全过,行覆盖 100% 恢复且口径诚实分开写;但真机
|
||||
用例 0 项已跑、分支 28 处部分。差:外部(真机)为主。
|
||||
- 文档 9.9 → 9.9,未达 10:§15 与 ADR-17 已同步,但
|
||||
§13.1 仍是旧快照、真机证据表仍空。差:外部。
|
||||
- 架构 9.7 → 9.7,未达 10:新增 flock 锁是并发语义
|
||||
收紧而非架构重构,ADR 取舍不变。差:外部。
|
||||
- 前端 9.6 → 9.6,未达 10:本轨未改前端。差:代码内
|
||||
(有意不做)+ 外部(live)。
|
||||
- 生产就绪 5.5 → 5.5,未达 10:P0-01 未做、CI 未绿、
|
||||
未合并、未部署、真机 0 项。差:外部为主。
|
||||
- 动态安全 2.0 → 2.0,未达 10:本轨无任何动态验证。
|
||||
差:外部。
|
||||
@@ -0,0 +1,144 @@
|
||||
#!/usr/bin/env python3
|
||||
"""CHV 合并前检查(只读,绝不合并、绝不改 CI、绝不写库)。
|
||||
|
||||
用途:合并 §13.1/§14 链上 PR 之前,逐个核对 base/head、
|
||||
mergeable、CI 状态、与 main 头的关系与堆叠顺序,输出
|
||||
诚实结论。CI 仍是 UNSTABLE 就如实报 NOT-READY,不把
|
||||
账单模式的秒级失败读成代码失败,也不把 UNSTABLE 读成绿。
|
||||
|
||||
用法:
|
||||
python3 tools/chv_merge_precheck.py [PR号 ...]
|
||||
缺省检查整条 CHV 链(不含本轨新 PR;新 PR 建好后把
|
||||
它的号追加到参数末尾再跑一次):
|
||||
python3 tools/chv_merge_precheck.py 216 221 230 232 234 \\
|
||||
239 246 252 254 257 264 <本轨新 PR 号>
|
||||
|
||||
依赖:gh(已登录)与 git(仅读本地 remote,不 fetch、
|
||||
不 checkout、不 merge)。gh 不可用或 API 失败时退出码 2。
|
||||
退出码:0 = 全部就绪(可按顺序合并);1 = 未就绪(见原因);
|
||||
2 = 工具/查询失败(结论不可信,不要据此合并)。
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
DEFAULT_STACK = [216, 221, 230, 232, 234, 239, 246, 252, 254, 257, 264]
|
||||
|
||||
|
||||
def _run(cmd):
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise RuntimeError("%s 失败:%s" % (
|
||||
" ".join(cmd), (proc.stderr or proc.stdout).strip()[:500]))
|
||||
return proc.stdout
|
||||
|
||||
|
||||
def _gh_json(args):
|
||||
return json.loads(_run(["gh"] + args))
|
||||
|
||||
|
||||
def _check_summary(rollup):
|
||||
"""把 statusCheckRollup 归纳成一句话,不猜失败原因。"""
|
||||
if not rollup:
|
||||
return "NO-CHECKS"
|
||||
states = []
|
||||
for item in rollup:
|
||||
name = item.get("name") or item.get("workflowName") or "?"
|
||||
conclusion = item.get("conclusion")
|
||||
status = item.get("status")
|
||||
states.append("%s:%s" % (name, conclusion or status))
|
||||
conclusions = {item.get("conclusion") for item in rollup}
|
||||
if "FAILURE" in conclusions or "CANCELLED" in conclusions:
|
||||
verdict = "FAIL"
|
||||
elif conclusions <= {"SUCCESS", "SKIPPED", "NEUTRAL", None} \
|
||||
and all(item.get("status") == "COMPLETED" for item in rollup):
|
||||
verdict = "GREEN"
|
||||
else:
|
||||
verdict = "PENDING"
|
||||
return "%s [%s]" % (verdict, "; ".join(states))
|
||||
|
||||
|
||||
def main(argv):
|
||||
try:
|
||||
prs = [int(a) for a in argv] or list(DEFAULT_STACK)
|
||||
except ValueError:
|
||||
print("参数须为 PR 号(整数)", file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
repo = _gh_json(["repo", "view", "--json", "nameWithOwner"])[
|
||||
"nameWithOwner"]
|
||||
main_head = _run(
|
||||
["gh", "api", "repos/%s/commits/main" % repo, "--jq",
|
||||
".sha"]).strip()
|
||||
except Exception as exc:
|
||||
print("查询仓库/main 头失败:%s" % exc, file=sys.stderr)
|
||||
return 2
|
||||
print("仓库:%s" % repo)
|
||||
print("当前 main 头:%s" % main_head)
|
||||
print("链(按合并顺序):%s" % " → ".join(str(p) for p in prs))
|
||||
print("")
|
||||
problems = []
|
||||
tool_failed = False
|
||||
prev_head_branch = "main"
|
||||
for number in prs:
|
||||
try:
|
||||
pr = _gh_json(["pr", "view", str(number), "--json",
|
||||
"number,title,state,baseRefName,headRefName,"
|
||||
"headRefOid,mergeable,mergeStateStatus,"
|
||||
"statusCheckRollup,additions,deletions,"
|
||||
"changedFiles"])
|
||||
except Exception as exc:
|
||||
print("#%d:查询失败:%s" % (number, exc))
|
||||
tool_failed = True
|
||||
continue
|
||||
chain_ok = pr["baseRefName"] == prev_head_branch
|
||||
ci = _check_summary(pr.get("statusCheckRollup") or [])
|
||||
ready = (pr["state"] == "OPEN" and pr["mergeable"] == "MERGEABLE"
|
||||
and pr["mergeStateStatus"] == "CLEAN" and chain_ok
|
||||
and ci.startswith("GREEN"))
|
||||
print("#%(number)d %(title)s" % pr)
|
||||
print(" base=%(baseRefName)s head=%(headRefName)s@%(headRefOid)s"
|
||||
% pr)
|
||||
print(" state=%(state)s mergeable=%(mergeable)s "
|
||||
"mergeState=%(mergeStateStatus)s" % pr)
|
||||
print(" diff: +%s/-%s, %s files" % (
|
||||
pr.get("additions"), pr.get("deletions"),
|
||||
pr.get("changedFiles")))
|
||||
print(" CI: %s" % ci)
|
||||
print(" 链衔接(base 应为上一 PR 的 head 分支 %s):%s" % (
|
||||
prev_head_branch, "OK" if chain_ok else "MISMATCH"))
|
||||
print(" 本项:%s" % ("READY" if ready else "NOT-READY"))
|
||||
if not ready:
|
||||
reasons = []
|
||||
if pr["state"] != "OPEN":
|
||||
reasons.append("state=%s" % pr["state"])
|
||||
if pr["mergeable"] != "MERGEABLE":
|
||||
reasons.append("mergeable=%s" % pr["mergeable"])
|
||||
if pr["mergeStateStatus"] != "CLEAN":
|
||||
reasons.append("mergeState=%s"
|
||||
% pr["mergeStateStatus"])
|
||||
if not chain_ok:
|
||||
reasons.append("base 衔接断裂")
|
||||
if not ci.startswith("GREEN"):
|
||||
reasons.append("CI 未绿(%s)" % ci)
|
||||
problems.append("#%d:%s" % (number, "; ".join(reasons)))
|
||||
prev_head_branch = pr["headRefName"]
|
||||
print("")
|
||||
if tool_failed:
|
||||
print("结论:查询失败,结论不可信,不要据此合并。")
|
||||
return 2
|
||||
if problems:
|
||||
print("结论:NOT-READY,禁止合并。原因:")
|
||||
for problem in problems:
|
||||
print(" - %s" % problem)
|
||||
print("注意:CI 的秒级 verify 失败此前被定性为账单模式,"
|
||||
"脚本不据此改判——账单恢复变绿前,结论保持 NOT-READY。")
|
||||
return 1
|
||||
print("结论:READY,可按上面顺序逐个合并(合并后仍默认关闭,"
|
||||
"再跑 §14 灰度清单)。")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
Reference in new issue
Block a user