Commit Graph
18 Commits
Author SHA1 Message Date
host dbcc8e641a feat: 任务列表显示 turn_id 与 root_turn_id
host.py dispatch 打印补 turn_id/root_turn_id;执行器解析(local/Docker)透传;worker 写回 tasks 新列(新值非空才覆盖);server 迁移与接口字段;控制台任务列表新增 turn/root turn 两列(短显+悬停全文);新单测 9 个。
2026-10-27 14:52:00 +08:00
host 970b181bab 租户配额与计费:按用户月度 USD 上限 + 建任务/分支/重试准入 402 + 管理/自查端点 + 控制台展示 2026-10-27 05:18:37 +08:00
host 3a4997d346 feat: 旧任务继续会话——终端任务带新消息 + 全部选项同一会话续跑
server:POST /tasks/{id}/continue(终端非 merged 任务重排队,
resume_message 列存本轮消息,不传选项沿用原值,plan/goal 冲突
与新建一致,goal_mode=1 时转新 goal 语义;CAS 防并发改状态,
审计 task_continued);tasks.resume_message 三处迁移 + PG 建表。
worker:认领后本轮 prompt 换成续跑消息(只换内存副本),
goal 续跑走 goal-resume 而不是 goal,跑完清 resume_message
(进程被杀时不清,下次认领重跑同一条消息)。
host:goal-resume --goal-text(旧任务接续 goal 模式:跳过状态
与记录守卫,旧线程上从第 1 轮开新目标循环,新语义不限轮次;
无线程时拒绝)。
sandbox:Local/Docker goal_resume_task + popen goal-resume kind;
附带 popen goal 命令改成新语义(无验收不传 --verify,轮次 0
原样传,此前 goal 任务到本地 worker 会直接报错)。
console:详情页终端非 merged 任务出继续会话表单(消息 +
模型/强度/权限/计划/goal,模型下拉复用中转列表)。
测试:新增 test_task_continue.py 11 个 + test_host_continue.py 2 个。
2026-10-27 04:40:11 +08:00
host 7c444a688b feat: 任务附件——server/worker/console 链路打通(+ popen goal 新语义补漏)
server:POST /uploads(multipart,413 预检、路径/数量/大小校验、
24h 过期包、used 防一包多用)+ 建任务绑定(同事务写附件行 +
包置 used,cloud 执行器 400)+ 列表/详情带 attachments +
取消删暂存 + 分支带走附件清单;db.py PG 建表 + init_db 补表。
worker:_task_seed_dir(无附件/目录已删静默 None)+ dispatch/
goal 三路(同步本地、popen、Docker 只读挂载 /tmp/codex-seed)
+ cloud 执行器兜底失败 + 终局删暂存。
console:文件/文件夹/图片三路输入,先上传拿 upload_id 再建任务,
列表 [附件×n]、详情列附件路径。
附带:popen 的 goal 命令此前强制要求 verify_command,goal 新语义
任务到本地 worker(走 popen)会直接报错,改成与同步 goal_task
一致(无验收不传 --verify,轮次 0 原样传)。
测试:新增 test_task_attachments.py 15 个。
2026-10-27 04:30:43 +08:00
host 3d45edc8b4 feat: 命令台云端环境——选、建、管 + 任务绑定环境
环境是项目仓库的命名基准检出(项目 + 仓库快照 + 分支 + 执行器),
不是官方 Codex 云端环境(无公开调用口,不宣称能建它)。
- DB:environments 表(creating/ready/failed/archived,部分唯一索引
  保证同项目未归档名唯一、归档名可重用)+ tasks.environment_id,
  server/worker/db 三处迁移同步(含 #67 的 goal_mode,不丢新语义)。
- API:POST/GET /environments、GET /environments/{id}、
  POST /environments/{id}/archive;任务创建选环境须本项目已 ready、
  执行器一致或继承、cloud 执行器拒绝。
- Worker:run_all 每轮开头准备 creating 环境(真实 git clone +
  checkout 分支,幂等、无 preparing 卡死态,准备期归档 CAS 不覆盖);
  环境任务派发时 task[repo] 指到环境检出(租户 Docker 容器内路径 +
  envs 整目录挂载),任务仍一任务一 worktree。
- Console:环境卡(列表/新建/归档,未 ready 置灰)+ 新建任务环境下拉
  + 任务列表 [环境] 标记;公网 nginx 须同批代理 /environments。
- 单测:新增 test_task_environments.py 14 个;本地实弹(真实 git
  clone octocat/Hello-World,不跑模型)项目→环境→任务→归档全通。
2026-10-27 04:07:54 +08:00
openqjandhost c86b450851 feat: goal 模式新语义——prompt 即目标,无验收命令,不限轮次,模型宣告完成 (#67)
console 打勾 goal 模式时不再有专用 goal 文本、验收命令、轮次
三个输入(已移除,换成说明行),只发 goal_mode=1;server 侧
goal_mode=1 时 goal 存 prompt、验收命令空、轮次存 0,带验收
命令或与计划模式同用时 400,旧的带验收命令任务照原样可用。

host.py goal 循环:verify 为空时不跑验测,轮次提示里写明
标记约定;模型在完整跑完的回合末尾单独一行输出“目标已
达成”即达标(超时回合的部分文本含标记也不算);--max-rounds
0 为不限轮次,goal-resume 续跑按 0 处理(不用 or 兜底成 3)。
刹车只有取消与预算耗尽(后者记 interrupted、可续跑)。

迁移与 worker:tasks 加 goal_mode 列(server 迁移、worker 列
补齐、PG schema 三处同步);worker 按 goal_mode 或
verify_command 判定 goal 路径,重试与后续消息排空对新语义
同样只跑一次/不排空;Local/Docker goal_task 的 --verify 可选、
--max-rounds 0 原样传。

Co-authored-by: host <host@test.local>
2026-10-27 03:54:48 +08:00
openqjandhost f4443dca4f feat: 计划模式 + 后续消息 + 会话分支接云端 (#65)
- 计划模式:tasks 加 plan 列;plan 与 approval=full / goal 冲突时
  400;执行器在 approval 参数后追加 --sandbox read-only(本地
  dispatch/run/goal/popen 与 Docker 内层命令全覆盖),host 任务
  存下 sandbox 后续轮次自动继承只读。
- 后续消息:新表 task_followups;POST/GET /tasks/{id}/followups
  (仅 queued/running 的非 goal 任务;direct 立即往链路追加
  phase=followup-direct 条目,本轮跑完后优先于排队执行;排队
  按 direct 优先、队内先来先服务逐条再跑一轮,同一 thread;
  一条失败即停、剩余 pending 留给重试入口;取消时回 pending)。
- 会话分支:POST /tasks/{id}/fork(仅终态任务;继承 executor/
  模型/强度/权限/计划,goal 字段置空);worker 先 fork 再跑首轮,
  首轮失败不自动重试、不重复分叉,失败后走重试入口重跑首轮。
- 修任务级模型被覆盖:worker 此前无条件写成主模型,控制台
  选的模型实际不生效;现为任务级 > worker 主模型 > 配置。
- Docker 租户准备抽成 _docker_task_context,供普通尝试与 fork 共用。
- console:新建任务计划勾选(含冲突拦截)、详情计划徽标与
  分支来源、分支按钮与表单、后续消息区、列表 [计划]/[分支] 标记。
- 测试:cloud/tests/test_task_links.py 17 个(server 校验与继承、
  worker 模型优先、只读命令、排空顺序与停排、goal 不排空、
  fork 单次性);全量 cloud 542、host 49、deploy 14。

Co-authored-by: host <host@test.local>
2026-10-27 03:17:51 +08:00
openqjandhost 8b2f3968dc feat: per-task model/effort/approval + relay models endpoint (#59)
Co-authored-by: host <host@test.local>
2026-10-27 01:44:53 +08:00
openqjandhost fc8e337d0f HA 阶段 1:worktree_host + 跨主机 /pr 队列化 (#50)
* 部署自动化:唯一发布链路 deploy.py + host_deploy.sh,sha/commit 双重验真,失败回滚

旧链路三次栽过的坑:通用 tar 路径导致旧包误部署、传输不验 sha、
主机脚本多份漂移、health 通过不等于新代码上线。本单把四点变硬门禁:
- 产物唯一命名 codex-app-server-<commit>.tar.gz + .sha256,远端唯一路径
- git archive 产物 + DEPLOYED_COMMIT 清单,默认拒绝脏工作区
- 传输链 scp -> scp_run -> chunk 分片,每路传完远端验 sha,不符删掉重来
- 主机侧唯一脚本:先验 sha/清单,staging 解包 + 编译检查,换目录上线,
  work/ 原样保留,health + 上线 commit 不符即回滚并退出非零
- deploy.py 成功后独立 ssh 再读一次 live 清单二次核对
- CI:host-verify 加 deploy/tests;release.yml 只在手动/tag + production
  environment 审批时部署,主干推送绝不自动部署生产
- deploy.sh 标记为旧链路弃用
- health 的 commit 字段已在父提交 HA 阶段 0 里随 server.py 落地,
  本分支整体包含它;commit 为空时只靠文件核对,门禁同样成立
测试:host 32、cloud 487(4 跳过)、deploy 9,全部通过;
host_deploy.sh 仿真覆盖成功/坏 sha/清单不符/health commit 不符回滚。
生产实机部署未做,等确认后再用新链路推。

* HA stage 1: worktree_host + queued cross-host PR jobs

---------

Co-authored-by: host <host@test.local>
2026-10-27 00:05:18 +08:00
host 63823fc2e3 逐任务真实成本:上游用量事件 + 价目表计价
此前每任务只记一条 estimated(字符数/4,金额为空),注释称
host 协议不暴露真实 token 数。本轮实测推翻了该前提:
- account/usage/read 在中继上是死的(要求 ChatGPT 账号登录),
  不是来源,已在 cost.py docstring 写明死胡同
- app-server 每上游补全发 thread/tokenUsage/updated,last 是该次
  补全的用量;中继 5 调用回合实测 last 之和等于最后 total
  (input/output/cached/reasoning 四路都对得上)

实现:host.py run_turn 从事件窗口按 threadId+turnId 汇总用量,
重试/goal 各轮累加,dispatch/run JSON 与 goal 用量行透传;
sandbox 三路解析(dispatch JSON 解析改用完整 stdout,顺带修
了长 text 下 failed 被误判 completed 的潜 bug);worker 每任务
记一条 actual(重试/fallback 用量累加进同一条),无用量时回落
estimated;cost.py 价目表(默认 OpenAI 挂牌价,env 可覆盖,
无价模型 usd 记 None、报表进 unpriced_actual_tokens 而非 0 美元),
cost_records 加 cached_input_tokens 列(SQLite/PG 两支迁移);
报表与按天行新增 cached_input_tokens/unpriced_actual_tokens。
测试:host 32、cloud 483 全过。
2026-10-26 18:41:27 +08:00
host 5ee9d291dd 云端 goal/merge/review 三事件:goal 循环 + PR 事件通知
此前 goal-unmet/merge-failed/review-requested 只有允许列表,
云端没有生产者(概念只在本地 host.py)。本轮给三者接真生产者:
- goal:POST /tasks 接受 goal/verify_command/goal_max_rounds
  (迁移补列,SQLite 与 Postgres 两支);带 verify_command 的
  任务走 host.py goal 循环(Local/Docker 执行器 goal_task,
  turn→验测→喂回重试,缺省 3 轮);达标 completed,未达标
  failed 且只发 goal-unmet、不重复发 failed;goal 模式下 worker
  级重试/fallback 关闭,同任务 /retry 会撞 host 任务名已存在
  (已在文档写明,须新建任务);CloudExecutor 无 goal 概念,
  如实失败不降级
- review-requested:POST /tasks/{id}/pr 建 PR 成功时发
  (payload 含 pr_url/pr_number)
- merge-failed:PR 落盘(push/建 PR)失败 502 时发(payload
  含 step/error),任务保持 merged、可重调 /pr;跳过时不发
测试:host 29、cloud 469 全过。
2026-10-26 18:11:12 +08:00
host 9fa80bef89 修 GitHub App 安装在 Postgres 上崩:适配层译 INSERT OR IGNORE,安装后增减仓库同步 2026-10-26 16:11:29 +08:00
host 26d9edac33 Perf: light worker poll query + tasks(status) index
Worker polls the DB every interval (production 30s) and every
poll ran SELECT * FROM tasks: full rows including result_text
and error_text (up to 4000 chars each) for every task ever,
growing with history. load_tasks now selects only the columns
pick_next actually needs (id/project/prompt/status/depends/
priority/timestamps/executor) and only queued/completed/merged
rows; running/failed rows are neither ready nor completed, so
excluding them cannot change pick_next. get_task still does
SELECT * for the single claimed task. Plus CREATE INDEX IF NOT
EXISTS idx_tasks_status on tasks(status) in both SQLite
init_db and the Postgres schema, so the poll filter stops being
a full table scan on old databases too. Test: load_tasks returns
only queued+completed out of queued/completed/running/failed.
2026-10-26 06:19:01 +08:00
host 306cdf0d0c Cloud dispatch pool + auto PR (opt-in, skip clearly)
Worker pool: workers table + register/heartbeat (heartbeat is
registration, no 404), GET /workers with alive (120s default),
POST /workers/heartbeat (X-API-Key, missing worker_id 400).
Tasks gain executor (local/docker/cloud, None = any worker);
worker claim filters by executor in pick_next and in the CAS
WHERE clause, CAS semantics unchanged; CloudExecutor _name
now correctly returns cloud, not local.
CloudExecutor: generic HTTP runner interface (endpoint from
explicit arg or CLOUD_RUNNER_ENDPOINT, auth header name only,
token value only from env into the request header, never into
logs/body/errors); no endpoint configured = explicit failure,
never silent downgrade. Vendor adapters (ECS/Cloud Run) are
out of scope, documented as such.
Auto PR: projects.auto_pr (0/1, create-time only, no PATCH
route exists); POST /tasks/{id}/pr (maintainer+ via
approve_merge) only for merged tasks (else 409); skip reasons
in order: auto_pr_disabled, no_worktree, no_remote, no_gh,
gh_not_authenticated — all return 200 with pr_skipped, task
stays merged, no push attempted; push/pr-create failure
returns 502 with step, task stays merged; success writes
pr_url/pr_number and audits pr_created. No default auto-push
from the worker. Remote URLs in errors are scrubbed of
userinfo; gh credentials are managed by gh itself, never
stored by the control plane.
Tests: host 27 pass, cloud 323 pass (8 skipped are
environment-dependent real-Postgres tests).
2026-10-26 04:57:37 +08:00
host bed3fc0dff Project repo_url: worker can run tasks from any git repo
Projects gain repo_url (validated: https:// or git@ only, no
userinfo/password, no file:// or local paths; old DBs get the
column via idempotent ALTER in both SQLite and Postgres init).
Worker resolves a project's repo_url to a local checkout under
--repos-dir (default ./repos) before dispatch: clone if absent,
verify remote origin matches and fetch if present, never force
reset/clean, never push (push needs write credentials the
control plane does not store). Resolve failure fails the task
with a clear error, never completed. Docker workers need
repos_dir under a same-path extra mount (e.g.
/var/lib/codex-app-server/repos).
Also: driver-missing test is now mocked instead of depending
on the host really having no psycopg driver installed.
Deploy package: Caddyfile template exposing only /webhooks/*
and /oauth/* (tasks/projects/admin stay 403) plus a
webhook-secret generator script that never prints the secret.
Tests: host 27 pass, cloud 286 pass (8 skipped are
environment-dependent real-Postgres tests).
2026-10-26 04:16:11 +08:00
host dc3f572b2b Postgres real backend: psycopg adapter, schema, migration
Replace the PostgresDatabase skeleton with a real implementation:
psycopg (v3) first, psycopg2 second, lazy load, missing driver
still raises PostgresDriverMissingError with no silent SQLite
fallback. A sqlite3-style adapter (PgConnection/PgCursor/PgRow)
translates ? placeholders to %s outside string literals, maps
UNIQUE/FK violations, lock errors and connection failures to
explicit Postgres error types, and handles INSERT OR REPLACE,
PRAGMA table_info, AUTOINCREMENT and FTS5 refusal. POSTGRES_SCHEMA
covers all business tables with tasks result columns; server
init_db uses it under CLOUD_DB_BACKEND=postgres, SQLite path
unchanged. migrate_sqlite_to_postgres.py copies rows with
ON CONFLICT DO NOTHING and reseeds identity sequences.
Tests: cloud 239 (235 pass, 4 skipped without a real server),
host 27 pass; real-server tests skip unless CLOUD_TEST_PG_DSN
is set.
2026-10-26 03:23:06 +08:00
host 07ef869267 cloud: worker dispatch, DB abstraction, packaging 0.1.0, repo CI workflow 2026-10-26 01:44:15 +08:00
host 076ba01c74 cloud: fill gaps (memory/KB/cost/models endpoints, WAL concurrency, OAuth scaffold, push subscriptions) 2026-10-26 01:32:57 +08:00