16 Commits
Author SHA1 Message Date
host 44e0f4f1d6 fix: audit batch 1 (B-A migration audit_loop INTEGER, B-C gateway failure clears running slot)
- B-A: migrations.py tasks backfill loop now types audit_loop as
  INTEGER (other 6 columns stay TEXT), matching db.py backfill,
  server/worker ensure lists; TEXT readback '0' is truthy and
  would silently enable the audit loop for audit_loop=0 rows.
- B-C: gateway preparation failure early-return now calls
  _running_clear(task_id) next to _cleanup_task_seed, matching
  missing-key/repo/capability/tool gates; previously the capacity
  slot leaked until worker exit (default capacity 1 blocks reclaim).
- Tests: cloud/tests/test_audit_batch1.py (3 tests): migration
  ALTER types, INTEGER 0 readback not enabled via real worker
  row mapping, gateway failure clears slot and capacity reclaim.
  _claim doc 'three gates' list left unchanged (full early-return
  gate enumeration not verified this batch; residual wording).
- No other audit items touched (I-1..I-14, N-1..N-12).
2026-11-04 18:02:22 +00:00
host f2005689fd fix: wire audit loop into worker completion path (pushback delivery, escalate, packet preamble)
Audit-loop wiring, worker completion path only: after the final
write-back (and only when the final is completed/failed, not quota,
not cancelled, not goal-converted, and the task carries audit_loop),
collect this attempt's deterministic gate results (execution,
goal_verify only when host reported goal_met, capability /
required_tools / gateway as passed-or-inapplicable markers, since a
pre-dispatch gate rejection early-returns before any runner round
and requeueing it would deterministically re-fail the same gate)
and call audit_loop_evaluate. Pushback stores the packet JSON in
tasks.audit_loop_packet and CAS-requeues the task; the next claim
prepends the packet preamble to prompt (and goal) with a data-only
separator, DB prompt untouched. Pass clears the packet. Escalate
sets failed (VALID_STATUSES has no needs_review, never invented)
with an escalation error and clears the packet; subjective
needs_review leaves the outcome untouched and only audits.
New tasks.audit_loop / audit_loop_packet columns follow the
attempts DDL pattern in worker ensure, server migrate, PG schema,
PG backfill, migrations backfill, and POST /tasks accepts
audit_loop (True/1 only, coordinator tasks rejected like v9).
Coordinator parent tasks return via _run_coordinator before the
attempt loop and coordinator children never inherit audit_loop,
so coordinator wiring is left as legacy, unchanged.
2026-11-04 18:02:22 +00:00
host 37b1aadc02 fix: bounded audit loop (deterministic gates only, 3 pushbacks, escalate)
Audit loop engine (cloud/audit_loop.py): Finding/PushbackPacket/LoopState pure functions; deterministic gates only, subjective -> needs_review; packet completeness enforced; standards-weakening -> escalate; regression and missing-gate checks enforce full re-run; MAX_PUSHBACKS=3, round>=3 -> escalate. Persistence in new single-row task_audit_loop_state (attempts DDL pattern: module CREATE_TABLE_SQL + db.py POSTGRES_SCHEMA + migrations list + server init + worker ensure); task_attempts NOT reused (attempt_no PK collision) and task_followups NOT reused (user messages). Worker.audit_loop_evaluate is the explicit opt-in hook (task audit_loop field only, no env switch, never called by run_one); escalate audits worker_audit_loop_escalated; corrupted/missing count fail-closes to escalate.
2026-11-04 18:02:21 +00:00
host 621c08d10f fix: second-audit gate findings (B1, I1-I8, M1-M10)
B1: gateway capability check now triggers on walking the sub2api
gateway, not on the gateway having filled a model; task-level
explicit models are audited (worker_capability_gateway_explicit)
but never blocked, preserving explicit priority; gateway-default
models still fail-fast under active.
I1: dry-run adds a gateway-model section, sourced only from
CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active
tier whose requires the gateway model fails or cannot verify
blocks, shadow only warns.
I2: shadow review gate now counts completed samples
(completed >= 20), same basis for sample_shortfall; the endpoint
attaches per-tier capability status (read-only policy load) and
flags capability-failing tiers in the recommendation.
I3: configured-but-broken routing policy logs a warning in
load_routing_policy (unconfigured stays silent); the worker sets
routing_policy_load_failed and audits
worker_routing_policy_load_failed per task.
I4: resume preamble wraps prior output/error as data with an
explicit do-not-execute constraint; position verified in run_one
(tree -> preamble -> project -> original) and the docs corrected
from the false 'closest to the original prompt' claim.
I5: decided model is written back with a CAS UPDATE
(model IS NULL OR model=''), so later attempts reuse it;
explicit models are never overwritten.
I6: agent_* columns added to worker._ensure_result_columns and
db.init_postgres_schema backfill; migrations backfills them for
tasks before insert (sender_task_id pattern); column-level tests
for agent_tool_approvals and task_attempts.
I7: coordinator requeue adds AND status=<snapshot> (rowcount 0
skips requeue/audit) and filters paused_by_user / user_cancel /
user_pause children, which are never revived or cleared.
I8: explicit-model tasks (including coordinator children that
inherited a parent model) audit a source=explicit worker_routing
row; evaluate_shadow counts it as per-tier explicit_n only,
never routed_n, never a dedup slot.
M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation,
shadow sort doc fixed, truncated recommendation notes truncation,
seed cleanup on gateway/quota/goal-cap early returns, tools-only
coordinator definitions rejected, preamble goal branch marked
effectively unreachable, DESIGN.md drift corrected (outside repo).
M4/M7/M9: documentation/new-field only (pass_rate_fully_measured,
whitespace-model shadow-drop note, bound-note race note); old
field semantics unchanged.
2026-11-04 18:02:21 +00:00
host 19fd326a73 feat: attempt ledger + resume preamble (v6)
Foundry 'platform restarts with original input' gap, after
reading the existing mechanisms: creation idempotency
(UNIQUE project_id+idempotency_key), goal checkpoints
(task_checkpoints per round, same-thread breakpoint resume
with no-progress fuse), merge/cost/notification CAS, and
coordinator child reuse already exist. The uncovered segment
is non-goal platform-interrupted tasks: auto_resume requeues
them fresh in the same worktree after clearing result/error/
interrupted_reason, so the new run gets only the original
prompt and cannot tell what the previous attempt had done.

This track adds, in worker/server layer only (no host.py or
codex thread change): task_attempts ledger, recorded by
auto_resume_interrupted in the same transaction before
clearing (attempt_no = old auto_resume_count + 1, kind
goal_breakpoint|platform_fresh, interrupted_reason,
checkpoint_round, result/error truncated to 2000 chars,
INSERT OR IGNORE keeps the first record on duplicate);
resume preamble injected in-memory only at claim for
platform_fresh reruns (previous attempt facts, truncated to
1000 chars, plus check-worktree-first and verify-before-
recreating-external-artifacts instructions; skipped for
coordinator/_resume/breakpoint tasks and when no record
exists, never fabricated); attempts surfaced read-only in
GET /tasks/{id}/timeline under existing project-read
permission. Fresh PG schema, migrations table list, and
server/worker ensure paths share one DDL.

Deliberately NOT claimed: the preamble is an instruction,
not a hard guarantee like creation idempotency or merge
CAS; the ledger does not track worktree files or external
artifacts (no worktree snapshot, executor-dependent paths;
hard dedup for those still rests on their existing gates);
manual retry paths do not go through auto_resume, so they
get no ledger record or preamble. Tests:
test_attempts.py (7 tests).
2026-11-04 18:02:10 +00:00
host b391fe9320 resident P0+P1: DNS-level SSRF pin, batch budget, daily cap/cooldown/retention, PG schema/migration, dual-connection deadlock isolation, retarget baseline reset, proxy and structured failures, metrics
- DNS 级 SSRF pin:解析后 pin IP 建连,防 DNS 重绑定
- 批量总预算与每日上限/抖动冷却/保留期
- PG schema 与迁移清单
- 双连接死锁隔离
- 改目标重置基线
- 显式代理配置与结构化失败原因
- /metrics resident 口径
2026-11-02 03:30:57 +00:00
host 3a06205d8b feat: 跨 worker 真 kill(kill_requests 意图表 + 收殓器)与自动化触发入闸
机制 A 移植自旧 PR #101(分支 feat-residual-b,头 895e004),
不合旧 PR:旧基线 6e3eb7b 已 CONFLICTING,本批重做到现 main
6d4f87e(#151 合并后)上。

- kill_requests 意图表:cloud/db.py POSTGRES_SCHEMA + migrations
  清单补表;server.ensure_kill_requests_table / request_kill
  (同一任务只留一条 pending,写失败返回 False);init_db 的
  SQLite 分支补建表(PG 分支由 POSTGRES_SCHEMA 全量建表覆盖)。
- 取消/sweep 写请求 best-effort:_cancel_task 在 commit 后调
  request_kill(失败不挡取消);sweep_stale_tasks 在同一事务
  内为被扫任务写 system 请求。
- worker 收殓器 _reap_kill_requests:owner 活着(心跳停滞未超
  300 秒,COALESCE(last_heartbeat, updated_at))绝不抢杀;只
  有 owner 已死且任务仍 interrupted 时,按任务派生容器名
  (executor.container_name,codex-<安全字符任务 id>)在本机
  真杀,绝不按前缀扫、绝不杀自由传入的名字;本机无容器时
  请求照常终结;任务行失踪或已不在 interrupted 时请求直接
  终结;单条异常只记日志。run loop 每轮调一次,放在 sweep
  块外(旧 PR 只在 sweep 块内,sweep 关掉时请求会积压)。
- 自动化入闸只补两条旁路:worker._coord_sync_children 建子任
  务前过 _quota_exceeded(超限跳过建任务、记审计,同一 worker
  内同一分工只记一次);automations.run_due_automation
  (claim 抢单 -> 配额门 -> 建任务 -> 记产出,超限记
  skipped_quota 不进失败重试)。triage/webhook/续跑/后续/
  分叉未动。
- 单测:旧 PR 三份单测原样移植(Worker/Server 签名与现 main
  兼容,未改用例语义)。
2026-10-30 01:32:35 +08:00
host 585242fc8f 长任务 P0 三件:断点进库、错误分类+熔断、Run/Step 时间线
断点进库:task_checkpoints 表(PG 新库 + 老库幂等补表 + 迁移清单
对齐);worker 在 goal 任务执行期间每 30 秒把磁盘 host-state 断点
镜像进库,终局再补一次;断点续跑认领前,若 host-state 条目丢失
但库里有断点,先按库断点重建最小条目再跑,库里也没有断点时不
造假、走原有拒绝路径。

错误分类+熔断:sandbox 四处解析加永久错误分支(KeyError、配置
缺失等无结论异常退出判 failed 死信,不再判 interrupted;host 已
明言 interrupted 的不受影响,未知错误不分类);worker 自动恢复
加退避(10 秒起指数退避、封顶 300 秒)与无进展熔断(库断点轮次
连续 3 次不变即标 auto_resume_stalled 等人工)。

时间线:task_steps 表 + host 每轮写 host-steps 步骤文件 + worker
导入;GET /tasks/{id}/timeline 一次返回 steps 与 checkpoints;
认领、暂停、自动恢复、终局都记步骤。旧任务返回空数组,不编造。

单测:cloud/tests/test_p0_three.py 14 项(分类、熔断、镜像重建、
步骤导入去重、timeline 接口)。
2026-10-29 15:13:01 +08:00
host cbc90f9d30 自动记忆:官方 memories 对齐的提取+注入闭环(确定性 v1)
开关默认关(generate/use),设置接口 GET/PUT /users/me/auto-memory-settings,任务级覆盖 POST /tasks/{id}/auto-memory-flags,手动触发 POST /users/me/auto-memory/run,日志 GET /users/me/auto-memory-log。

提取:worker run_loop 轮次边界扫 completed 任务,闲置/时长/长度/外部上下文/配额剩余百分比五道门,脱敏后写 project 级 identity_memories,同一任务去重、同一目标 supersede 合并;只写 project 级,不跨项目泄漏。

注入:worker run_one 首轮把最新各 5 条用户/项目记忆拼进内存 prompt 副本(goal 任务同步改内存 goal),库原 prompt 不动;续跑/断点/协调员不注入,注入失败 fail-open。

v1 诚实边界:extract/consolidation 模型只存设置不调模型;MCP/web 线程内调用暂无 tasks 表标记,外部上下文门只认任务间消息与协调员子任务;无控制台 UI。
2026-10-28 12:12:10 +08:00
host b93666fbe8 合并云端身份与记忆 PR#102(6e9d6dc)到最新 main ea5ab62 2026-10-28 11:58:49 +08:00
host cf08c4cf37 feat: 线程生命周期对齐 15 项(bindings/refs/快照/心跳绑定/移交落点)
对照 Codex app 第二批:GET /tasks/{id}/bindings(项目/
环境/主机/worktree/PR/附件/血缘总表)、/refs 外部引用
增删查、/worktree/archive 快照(branch/head/status/diff/
untracked 路径)与 restore(目录在才解归档、目录删了
409 附快照)、自动化 bind + heartbeat(走 thread/message
同款投递路,cron 绑定不唤醒)、移交 completed 后任务
worktree_host 跟随切换、pr/checks 响应补 head_sha、
thread/read 的 tool 条目补 server/status。

诚实边界见 docs/thread-lifecycle-align.md:已有 worktree
附加、自动化变化静默、建议器、服务端定时触发器未实现;
Memory 对位是待合的 PR #102,本批不重复实现。

三套件:cloud 948 OK(4 跳过)、host 91 OK、deploy 29 OK
2026-10-28 04:56:22 +08:00
host 755029f220 feat: 线程等待与状态读取 8 项(wait/read/message/handoff/worktree/pr-checks/automation/sleep)
对照 Codex app:wait_threads 首变+afterCursor+8上限、
read_thread 详情摘要、send_message_to_thread 复活失败任务、
handoff 登记与上报轮询、worktree 派生状态、PR checks JSON+
check-runs 日志与注释、automation GET/PATCH、clock.sleep
(API+host)。诚实边界见 docs/thread-wait-ops.md:无
worktree 阶段流、无 automation 变化静默、无独立测试结果
记录、handoff 服务端不执行迁移、wait 仅数据库级变化检测。

三套件:cloud 940 OK(4 跳过)、host 91 OK、deploy 29 OK
2026-10-28 04:33:20 +08:00
host 6e9d6dc5e8 云端身份与记忆:用户画像+分级记忆,不动 thread
- 新增 cloud/identity.py:user_profiles(显示名/时区/偏好,偏好禁密钥键)
  与 identity_memories(user 级仅本人可见、project 级项目成员可见,
  kind fact/preference/decision/convention,supersede 链+物理删除)。
- server 新增 /users/me/profile、/users/me/memories、/projects/{id}/
  identity-memories、/users/me/identity-context,项目记忆先过项目
  read/create_task 门;上下文只显式取用,不自动注入任务/thread。
- PG schema 与迁移清单同步新增两张 TEXT 主键表,序列清单不动;
  server 补 do_PUT 入口(/users/me/profile 首个 PUT 路由)。
- 新测试 11 个;三套件 cloud 937 OK(4 跳过)、host 89 OK、deploy 29 OK。
- 未动:host.py、worker、CODEX_HOME、旧 summaries 表、控制台 UI;
  未推、未建 PR、未部署。
2026-10-28 03:43:02 +08:00
host 64995fafab P2-ops: migration alignment, deploy test list, monitor alerts/single-host, backup/rollback closure, Caddy template
- migrations: MIGRATION_TABLES now equals POSTGRES_SCHEMA tables
  (uploads, task_attachments, task_followups, environments added);
  SEQUENCE_TABLES matches IDENTITY tables; task_followups
  sender_task_id column ensured before insert; UNMIGRATED_TABLES
  and migration_coverage() report tables with no PG schema instead
  of silently skipping them; CLI prints the coverage report.
- deploy: DEFAULT_TEST_SUITES gains deploy/tests (discover -s only,
  never package path); regression tests pin the default list.
- monitor: MONITOR_SINGLE_HOST=1 makes the server check single-host;
  default stays dual to avoid silent downgrade; server_mode is in
  the JSON. Optional fail-only alert outlet (webhook or mail,
  default off, alert body is pass/fail JSON only, no secrets).
- backup/rollback: README and restore-drill document what is
  recoverable (DB, code) vs not (worktrees, traces, uploads, host
  configs); rollback.sh shares the server-unit list and dual-port
  health success standard with host_deploy.sh (sync test guards it).
- public entry: Caddyfile uses native {$DOMAIN}/{$ACME_EMAIL}
  variables plus a caddy.service template; README documents the
  steps and the current BT-nginx production status.
2026-10-27 16:37:28 +00:00
host 7091a2d0d2 P0 security batch: true global admin, quota gates on all spending paths, default quota for quota-less users, fix-review author trust + docker/owner attribution, GitHub App setup issuer binding, real cancel kill, wider shell env exclusion
- CODEX_GLOBAL_ADMINS replaces maintainer-derived global admin;
  admin endpoints (quotas, sweep, backup, push dispatch) fail-closed
  403 when unset, with audit on allow and deny.
- continue/followups/messages/retry check both caller and original
  creator quotas (USD + unpriced token cap), 402 on either overrun.
- Quota-less users now get the #86 default quota instead of unlimited;
  init_db backfills and OAuth create/bind/existing-login branches
  INSERT OR IGNORE a default row; explicit NULL row stays unlimited.
  user_quotas added to migration tables.
- /fix-review: untrusted authors rejected before dedup claim, no task;
  trusted tasks force executor=docker, created_by=project owner, owner
  quota gate, prompt is a fixed template plus a 2000-char excerpt.
- GitHub App setup state only issued by global admins, state rows bind
  issued_by (ALTER-compatible for old DBs), callback rejects issuers
  no longer in the admin list without consuming the state.
- Cancel: popen poll checks DB status every second, Docker path runs a
  cancel watchdog that kills by container name, write-back is CAS-guarded
  so late results never overwrite a new status.
- SHELL_ENV_EXCLUDE adds *_TOKEN, *_AUTH*, *_CREDENTIAL*, *API_KEY*;
  comment states the residual risk: same-UID /proc environ remains
  readable, true isolation needs UID separation or hidepid.
- Tests: new cloud/tests/test_p0_security.py (17 cases); old tests
  updated to the new semantics (admin env, default quota, trusted
  fix-review author) without weakening any gate.
2026-10-27 15:35:23 +00:00
host dc3f572b2b Postgres real backend: psycopg adapter, schema, migration
Replace the PostgresDatabase skeleton with a real implementation:
psycopg (v3) first, psycopg2 second, lazy load, missing driver
still raises PostgresDriverMissingError with no silent SQLite
fallback. A sqlite3-style adapter (PgConnection/PgCursor/PgRow)
translates ? placeholders to %s outside string literals, maps
UNIQUE/FK violations, lock errors and connection failures to
explicit Postgres error types, and handles INSERT OR REPLACE,
PRAGMA table_info, AUTOINCREMENT and FTS5 refusal. POSTGRES_SCHEMA
covers all business tables with tasks result columns; server
init_db uses it under CLOUD_DB_BACKEND=postgres, SQLite path
unchanged. migrate_sqlite_to_postgres.py copies rows with
ON CONFLICT DO NOTHING and reseeds identity sequences.
Tests: cloud 239 (235 pass, 4 skipped without a real server),
host 27 pass; real-server tests skip unless CLOUD_TEST_PG_DSN
is set.
2026-10-26 03:23:06 +08:00