Project repo_url: worker can run tasks from any git repo
Projects gain repo_url (validated: https:// or git@ only, no userinfo/password, no file:// or local paths; old DBs get the column via idempotent ALTER in both SQLite and Postgres init). Worker resolves a project's repo_url to a local checkout under --repos-dir (default ./repos) before dispatch: clone if absent, verify remote origin matches and fetch if present, never force reset/clean, never push (push needs write credentials the control plane does not store). Resolve failure fails the task with a clear error, never completed. Docker workers need repos_dir under a same-path extra mount (e.g. /var/lib/codex-app-server/repos). Also: driver-missing test is now mocked instead of depending on the host really having no psycopg driver installed. Deploy package: Caddyfile template exposing only /webhooks/* and /oauth/* (tasks/projects/admin stay 403) plus a webhook-secret generator script that never prints the secret. Tests: host 27 pass, cloud 286 pass (8 skipped are environment-dependent real-Postgres tests).
This commit is contained in:
1 parent
71510a32d4
commit
bed3fc0dff
10 files changed
+788
-17
No files matched your search
+36
-5
@@ -533,9 +533,40 @@ interrupted 且终态任务 cancel 返回 400、心跳后 sweep 不扫刚心跳
|
||||
绝不抛异常),`Worker.cleanup_stale_containers()` 在 run_loop 启动
|
||||
与 CLI 启动时各调一次;取消/超时/失败路径上已有的容器清理保留
|
||||
(docker run `--rm`、超时 kill+rm、取消按名 kill)。
|
||||
- 仍未做(不要宣称已做):webhook 建的任务仍需 worker 认领执行,
|
||||
worker 的 repo 仍是配置里的 repo,不是任意 GitHub 仓库的自动克隆;
|
||||
`CloudExecutor` 仍是占位符;公网 https 入口仍需部署侧反向代理 +
|
||||
TLS;Docker 超时杀容器与真实 `pg_dump`、真实 GitHub webhook 送达
|
||||
仍需在真实环境验证(本轮只用 fake runner / 本地 HTTP 单测)。
|
||||
## 项目仓库(任意仓库任务)
|
||||
|
||||
- 项目级 `repo_url`:`POST /projects` 接受可选 `repo_url`,
|
||||
`GET /projects` 列表与 `POST /projects` 的 201 返回里都含它,
|
||||
不传时为 `None`(老行为不变)。校验与 worker 共用同一套
|
||||
`cloud.repos.validate_repo_url`:只接受 `https://` 开头(且不含
|
||||
userinfo,即不含用户名/密码)或 `git@` 开头(`git@host:path` 形);
|
||||
`file://`、`ssh://`、本地路径、带密码的 URL 一律 400 且不建项目。
|
||||
- 存储与迁移:`projects` 表新增 `repo_url TEXT` 可空列;SQLite 的
|
||||
`init_db` 建表时就含该列,老库用 `ALTER TABLE` 幂等补列(仿照
|
||||
`migrate_tasks_table` 的写法);Postgres 的 `POSTGRES_SCHEMA`
|
||||
同样加该列,`init_postgres_schema` 也对老库补列。
|
||||
- worker 派发:`run_one` 派发前查任务所属项目的 `repo_url`,非空时
|
||||
经 `cloud.repos.resolve_repo` 在 `--repos-dir`(默认 `./repos`)下
|
||||
解析:目录不存在时 `git clone`;已存在时校验它是 git 仓库且
|
||||
remote origin 与 `repo_url` 一致(不一致时报错,绝不强制改
|
||||
remote),一致时 `git fetch origin`(失败明确报错,不静默用旧代码;
|
||||
fetch 成功后不 checkout/pull/reset/clean,工作树由 host.py 的
|
||||
worktree 逻辑处理)。解析出的路径设为 `task["repo"]` 再交给执行器,
|
||||
Local 与 Docker 的 `dispatch_task` 都已读 `task["repo"]`,同一条路径;
|
||||
查不到项目或 `repo_url` 为空时走原有默认 repo,不报错;解析失败时
|
||||
任务置 `failed`,错误文本写清是仓库解析失败,绝不标 completed。
|
||||
- Docker 下的路径前提:容器内 `task["repo"]` 路径必须与宿主机一致,
|
||||
所以 `repos_dir` 必须放在 `--docker-extra-mounts` 已挂载的目录下
|
||||
(例如把 `/var/lib/codex-app-server/repos` 以同路径挂载进容器),
|
||||
否则容器内 repo 路径不存在——不要假装任意目录都行。生产部署示例:
|
||||
`python3 -m cloud.worker --db <库> --executor docker
|
||||
--repos-dir /var/lib/codex-app-server/repos
|
||||
--docker-extra-mounts /var/lib/codex-app-server:/var/lib/codex-app-server,…`。
|
||||
- 仍未做(不要宣称已做):推送到远端明确不在本轮范围——任务在本地
|
||||
检出上执行、本地 worktree 合并仍按 host.py 现有流程,`git push`
|
||||
到远端需要仓库写凭证,控制面不存、不推,所以任意仓库的自动 PR
|
||||
仍未实现;私有仓库 clone 会失败,错误文本写明“需要仓库访问凭证,
|
||||
控制面不存仓库凭证”,绝不尝试用任何 key 去 clone;webhook 建的
|
||||
任务仍需 worker 认领执行;`CloudExecutor` 仍是占位符;公网 https
|
||||
入口仍需部署侧反向代理 + TLS。
|
||||
|
||||
Reference in new issue
Block a user