deploy: add git-pull source for host deployment
Host can now fetch code directly from GitHub instead of receiving a VM-built tarball, bypassing the VM->host scp/kex failures and stale-tar risk. Design: VM sends only two small scripts (host_git_fetch.sh + host_deploy.sh, sha-verified via the existing transfer chain, now with a chunk_key fallback that does not need SSH_PASSWORD); host clones via a read-only deploy key (SSH only, HTTPS refused, key must exist with mode 0600), checks out the exact 40-char commit, verifies HEAD, builds a tar in the same format as the VM release (DEPLOYED_COMMIT + RELEASE.json, member names without ./ prefix so host_deploy.sh tar -O works), then calls the unchanged host_deploy.sh for staging swap, backup, health and rollback. Git fetch tolerates the known intermittent host->GitHub degradation: 120s timeout, 3 attempts, 10s sleep, live untouched until host_deploy runs. VM gates: clean tree, tests, commit must be on origin/main, SSH repo only, full sha only; --source tar remains the default. Tests: deploy suite OK, including 6 new host_git_fetch simulations (success with clean tar members, unknown commit, key permissions/missing, HTTPS refusal, short sha) and 4 new deploy.py git-mode helpers plus chunk_key fallback; git-mode dry-run verified against HEAD 630d14f.
This commit is contained in:
host committed
1 parent
630d14fe07
commit
e8e256f6e5
5 files changed
+542
-7
No files matched your search
@@ -0,0 +1,159 @@
|
||||
#!/bin/sh
|
||||
# 主机侧 git 取包脚本：deploy.py --source git 时由 VM 传过来执行。
|
||||
#
|
||||
# 为什么需要它（而不是在 live 里 git pull）：
|
||||
# live 是 tar 解包树、没有 .git，运行中服务 + 部分 checkout +
|
||||
# 脏文件会直接污染生产；正确形态永远是 staging 取包、校验、
|
||||
# 再交给已有的 host_deploy.sh 做换目录 + 备份 + 健康 + 回滚。
|
||||
# 本脚本只做取包与校验，换目录逻辑一律复用 host_deploy.sh，
|
||||
# 不复制、不漂移（host_deploy.sh 本身不改，tar 路径不受影响）。
|
||||
#
|
||||
# 调用（VM 侧 deploy.py 传过来，绝不由人工在主机上拼路径）：
|
||||
# sh host_git_fetch.sh <repo> <commit> [key_path] [host_deploy_script]
|
||||
# repo：只接受 SSH 形式，例如 git@github.com:openqj/codex-app-server.git；
|
||||
# HTTPS 需要 token，不在本模式内，直接拒绝（不要把 token
|
||||
# 拼进 URL 或写进本脚本）。
|
||||
# commit：40 位完整 sha，短 sha 拒绝（防歧义）；必须是已推到
|
||||
# GitHub 的提交，否则主机拉不到——VM 侧 deploy.py 在
|
||||
# 传脚本前先核 commit 在 origin/main 里，主机侧再核
|
||||
# checkout 后的 HEAD，任何一步不符都退出、live 不动。
|
||||
# key_path：主机上的 GitHub 只读 deploy key 私钥路径，默认
|
||||
# /root/.ssh/id_codex_github；权限必须是 0600，否则
|
||||
# ssh 会直接忽略该密钥，脚本先验权限再动手。
|
||||
# host_deploy_script：已传到主机的 host_deploy.sh 路径；本脚本
|
||||
# 用取到的代码在本地建一个与 VM 发布包同格式的 tar
|
||||
# （根目录含 DEPLOYED_COMMIT 与 RELEASE.json，成员名
|
||||
# 不带 ./ 前缀，否则 host_deploy.sh 的 tar -O
|
||||
# DEPLOYED_COMMIT 找不到成员），算 sha 后调它完成换目录。
|
||||
#
|
||||
# 间歇劣化容忍（2026-10-26、2026-10-29 实测：主机到 GitHub 的
|
||||
# 通路间歇劣化，15 秒超时后 2.8 秒又通，不是永久封禁）：
|
||||
# 每路取包用 timeout 限时（默认 120 秒），失败重试（默认 3 次，
|
||||
# 间隔 10 秒）；重试耗尽才退出，live 在此之前绝不动。
|
||||
# 取包失败的唯一后果是本次部署失败，旧代码原地不动；此时
|
||||
# 可改走 tar 路径（deploy.py 默认 --source tar）再推一次。
|
||||
#
|
||||
# 可覆盖的环境变量（仅用于本地仿真测试，生产用默认值）：
|
||||
# CODEX_GIT_WORKDIR：取包工作区根，默认 /tmp/codex-app-server-git
|
||||
# CODEX_GIT_TIMEOUT：单次 clone 超时秒数，默认 120
|
||||
# CODEX_GIT_ATTEMPTS：取包重试次数，默认 3
|
||||
# CODEX_GIT_SLEEP：重试间隔秒数，默认 10
|
||||
# 密钥：本脚本只读密钥文件路径，不读、不打印密钥内容；repo 与
|
||||
# commit 不是密钥，可以出现在日志里。
|
||||
set -eu
|
||||
|
||||
REPO=${1:-}
|
||||
COMMIT=${2:-}
|
||||
KEY_PATH=${3:-/root/.ssh/id_codex_github}
|
||||
HOST_DEPLOY=${4:-}
|
||||
if [ -z "$REPO" ] || [ -z "$COMMIT" ] || [ -z "$HOST_DEPLOY" ]; then
|
||||
echo "用法：sh host_git_fetch.sh <repo> <commit> [key_path] [host_deploy_script]" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# --- 门禁：repo 只接受 SSH 形式，commit 只接受 40 位完整 sha --------
|
||||
# 例外（仅仿真测试）：CODEX_GIT_ALLOW_LOCAL=1 且 repo 是本地目录时，
|
||||
# 允许本地克隆代替 GitHub——deploy.py 从不设这个变量，生产永远
|
||||
# 走 SSH 门禁；测试用它代替联网，避免单测依赖 GitHub 与 deploy key。
|
||||
if [ "${CODEX_GIT_ALLOW_LOCAL:-0}" = "1" ] && [ -d "$REPO" ]; then
|
||||
:
|
||||
else
|
||||
case "$REPO" in
|
||||
git@github.com:*|ssh://git@github.com/*) ;;
|
||||
*)
|
||||
echo "git 源只接受 SSH 形式（deploy key），HTTPS 需要 token，不在本模式内：" >&2
|
||||
echo " repo=$REPO" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
case "$COMMIT" in
|
||||
*[!0-9a-f]*|?????????????????????????????????????????*)
|
||||
echo "commit 必须是 40 位完整 sha，短 sha 或非法字符拒绝部署：" >&2
|
||||
echo " commit=$COMMIT" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
# 上面的 case 只能排除非法字符与超长，长度不足 40 位时再补一刀。
|
||||
if [ "${#COMMIT}" -ne 40 ]; then
|
||||
echo "commit 必须是 40 位完整 sha，长度不符拒绝部署：$COMMIT" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# --- 门禁：deploy key 必须存在且权限 0600 ------------------------------
|
||||
if [ ! -f "$KEY_PATH" ]; then
|
||||
echo "GitHub deploy key 不存在，拒绝部署（旧代码不动）：" >&2
|
||||
echo " key=$KEY_PATH（主机上生成密钥对后，把公钥以只读方式加到" >&2
|
||||
echo " GitHub 仓库 Settings > Deploy keys，私钥权限 chmod 600）" >&2
|
||||
exit 2
|
||||
fi
|
||||
KEY_MODE=$(stat -c %a "$KEY_PATH" 2>/dev/null || stat -f %Lp "$KEY_PATH")
|
||||
if [ "$KEY_MODE" != "600" ]; then
|
||||
echo "GitHub deploy key 权限必须是 0600，否则 ssh 会忽略它，拒绝部署：" >&2
|
||||
echo " key=$KEY_PATH mode=$KEY_MODE（chmod 600 $KEY_PATH 后重推）" >&2
|
||||
exit 2
|
||||
fi
|
||||
if [ ! -f "$HOST_DEPLOY" ]; then
|
||||
echo "host_deploy.sh 不存在，拒绝部署（旧代码不动）：$HOST_DEPLOY" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
WORKDIR=${CODEX_GIT_WORKDIR:-/tmp/codex-app-server-git}
|
||||
GIT_TIMEOUT=${CODEX_GIT_TIMEOUT:-120}
|
||||
GIT_ATTEMPTS=${CODEX_GIT_ATTEMPTS:-3}
|
||||
GIT_SLEEP=${CODEX_GIT_SLEEP:-10}
|
||||
GIT_DIR="$WORKDIR/repo"
|
||||
TARBALL="$WORKDIR/codex-app-server-$COMMIT.tar.gz"
|
||||
|
||||
mkdir -p "$WORKDIR"
|
||||
|
||||
# --- 取包：clone + checkout + HEAD 校验，重试耗尽才退出 --------------
|
||||
attempt=1
|
||||
FETCHED=0
|
||||
while [ "$attempt" -le "$GIT_ATTEMPTS" ]; do
|
||||
echo "git 取包尝试 $attempt/$GIT_ATTEMPTS：repo=$REPO commit=$COMMIT"
|
||||
rm -rf "$GIT_DIR"
|
||||
if timeout "$GIT_TIMEOUT" env \
|
||||
GIT_SSH_COMMAND="ssh -i $KEY_PATH -o BatchMode=yes -o StrictHostKeyChecking=accept-new -o ConnectTimeout=30" \
|
||||
git clone --quiet "$REPO" "$GIT_DIR" \
|
||||
&& git -C "$GIT_DIR" checkout --quiet "$COMMIT" \
|
||||
&& [ "$(git -C "$GIT_DIR" rev-parse HEAD)" = "$COMMIT" ]; then
|
||||
FETCHED=1
|
||||
break
|
||||
fi
|
||||
echo "git 取包尝试 $attempt 失败，$GIT_SLEEP 秒后重试（live 未动）" >&2
|
||||
rm -rf "$GIT_DIR"
|
||||
attempt=$((attempt + 1))
|
||||
if [ "$attempt" -le "$GIT_ATTEMPTS" ]; then
|
||||
sleep "$GIT_SLEEP"
|
||||
fi
|
||||
done
|
||||
if [ "$FETCHED" != "1" ]; then
|
||||
echo "git 取包重试耗尽，拒绝部署（旧代码不动）；可改走 tar 路径再推一次。" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- 本地建包：与 VM 发布包同格式，交给 host_deploy.sh 换目录 --------
|
||||
printf '%s\n' "$COMMIT" > "$GIT_DIR/DEPLOYED_COMMIT"
|
||||
# RELEASE.json 用 python3 生成（主机已有 python3，host_deploy 的
|
||||
# health 检查同样依赖它）；日期用 UTC，避免时区歧义。
|
||||
COMMIT="$COMMIT" REPO="$REPO" python3 -c '
|
||||
import json, os
|
||||
from datetime import datetime, timezone
|
||||
print(json.dumps({
|
||||
"commit": os.environ["COMMIT"],
|
||||
"repo": os.environ["REPO"],
|
||||
"source": "git",
|
||||
"built_at": datetime.now(timezone.utc).isoformat(),
|
||||
}, indent=2, sort_keys=True))
|
||||
' > "$GIT_DIR/RELEASE.json"
|
||||
# 成员名必须不带 ./ 前缀（见文件头注释），且绝不能把 .git 打进包。
|
||||
tar --exclude=.git --transform 's,^\./,,' \
|
||||
-czf "$TARBALL" -C "$GIT_DIR" .
|
||||
TARBALL_SHA=$(sha256sum "$TARBALL" | awk '{print $1}')
|
||||
echo "本地建包完成：tarball=$TARBALL sha=$TARBALL_SHA"
|
||||
# 取包目录不再需要，包与 host_deploy 的 staging 才是上线输入。
|
||||
rm -rf "$GIT_DIR"
|
||||
# host_deploy.sh 负责换目录、备份、健康、回滚，并打印 RELEASE_OK；
|
||||
# 它的退出码与输出原样透传，deploy.py 靠 RELEASE_OK + 二次核对验收。
|
||||
exec sh "$HOST_DEPLOY" "$TARBALL" "$TARBALL_SHA" "$COMMIT"
|
||||
Reference in new issue
Block a user