Foundry 'platform restarts with original input' gap, after
reading the existing mechanisms: creation idempotency
(UNIQUE project_id+idempotency_key), goal checkpoints
(task_checkpoints per round, same-thread breakpoint resume
with no-progress fuse), merge/cost/notification CAS, and
coordinator child reuse already exist. The uncovered segment
is non-goal platform-interrupted tasks: auto_resume requeues
them fresh in the same worktree after clearing result/error/
interrupted_reason, so the new run gets only the original
prompt and cannot tell what the previous attempt had done.
This track adds, in worker/server layer only (no host.py or
codex thread change): task_attempts ledger, recorded by
auto_resume_interrupted in the same transaction before
clearing (attempt_no = old auto_resume_count + 1, kind
goal_breakpoint|platform_fresh, interrupted_reason,
checkpoint_round, result/error truncated to 2000 chars,
INSERT OR IGNORE keeps the first record on duplicate);
resume preamble injected in-memory only at claim for
platform_fresh reruns (previous attempt facts, truncated to
1000 chars, plus check-worktree-first and verify-before-
recreating-external-artifacts instructions; skipped for
coordinator/_resume/breakpoint tasks and when no record
exists, never fabricated); attempts surfaced read-only in
GET /tasks/{id}/timeline under existing project-read
permission. Fresh PG schema, migrations table list, and
server/worker ensure paths share one DDL.
Deliberately NOT claimed: the preamble is an instruction,
not a hard guarantee like creation idempotency or merge
CAS; the ledger does not track worktree files or external
artifacts (no worktree snapshot, executor-dependent paths;
hard dedup for those still rests on their existing gates);
manual retry paths do not go through auto_resume, so they
get no ledger record or preamble. Tests:
test_attempts.py (7 tests).
170 lines
7.9 KiB
Python
170 lines
7.9 KiB
Python
"""尝试账本(v6,Foundry“平台以原始输入重启”对应的另一半)。
|
||
|
||
已有的、不在本轨重复造的机制(查代码确认):
|
||
- 任务创建幂等:tasks 的 UNIQUE(project_id, idempotency_key),
|
||
重复创建回读已有任务,并发抢建靠 IntegrityError 回读。
|
||
- goal 断点:task_checkpoints 每轮快照(worker 镜像线程 +
|
||
终局导入),goal 任务自动续跑走同一线程、同一 worktree、
|
||
从 goal_rounds_done 下一轮起;无进展熔断(断点轮次 3 次
|
||
不变即 stall)与退避、次数上限并存。
|
||
- 合并与通知的 CAS:终局写回只在 running 时成功才写结果、
|
||
记成本、发通知,重复终局不重复记账发通知;协调员子任
|
||
务按分工名复用,不重复建。
|
||
|
||
本轨补的缺口,恰恰是上述机制都没盖住的那一段:非 goal
|
||
任务被平台中断后,auto_resume_interrupted 会把它全新重
|
||
排——同一 worktree 里文件还在,但重排前会清掉 result_text /
|
||
error_text / interrupted_reason,新一轮 agent 只拿到原始
|
||
prompt,既不知道上一尝试做到哪一步、出了什么错,也无法
|
||
区分“worktree 里已有的文件是上一尝试的产物还是仓库原有
|
||
内容”,于是可能重复创建上一尝试已创建的东西。goal 任务
|
||
不受此影响(线程连续性自带历史),协调员子任务也不受此
|
||
影响(auto_resume 本来就不接 coordinator_orphan)。
|
||
|
||
本轨只做三件可执行的事,全部在 worker/server 层,不改
|
||
host.py、不改 codex 线程恢复:
|
||
① 尝试账本 task_attempts:auto_resume 把任务回 queued
|
||
之前,先把正结束的这次尝试记下来——attempt_no(= 旧
|
||
auto_resume_count + 1,即正结束尝试的序号)、kind
|
||
(goal_breakpoint 或 platform_fresh)、interrupted_reason、
|
||
checkpoint_round(库中断点最大轮次,非 goal 恒为 -1)、
|
||
result_text / error_text 各截断 2000 字。记账与回 queued
|
||
在同一事务里,记账失败不单独炸恢复循环(外层已兜底),
|
||
但同一 (task_id, attempt_no) 重复记时 INSERT OR IGNORE
|
||
保留第一次,不覆盖。
|
||
② 重跑前言:非 goal 平台全新重排的任务在认领时,worker
|
||
在内存副本的 prompt 前贴一段上次尝试说明(原 prompt
|
||
不落库,与项目/待办/记忆注入同款只改内存副本),内容
|
||
是账本里的事实——第几次尝试、中断原因、断点轮次、上一
|
||
尝试部分输出与错误(各截断 1000 字)——并明确要求先查
|
||
worktree 现状再动手、对 worktree 外的产物先核查再创建。
|
||
goal 断点续跑、_resume 任务、协调员任务一律跳过(它们
|
||
已有会话连续性或不跑模型回合);账本里查不到记录的老
|
||
任务原样返回,不编造。
|
||
③ 时间线读回:GET /tasks/{id}/timeline 在 steps /
|
||
checkpoints 之外加 attempts 数组,项目 read 权限复用
|
||
时间线原有校验,只读不改。
|
||
|
||
诚实边界(写进模块与交付,不宣称):
|
||
- 前言是指令不是保证:agent 仍可能重复创建,账本只能
|
||
降低重复概率,不能像创建幂等键、合并 CAS 那样硬保证
|
||
不重复。worktree 外的产物(分支、PR、发布等)账本不
|
||
追踪——它只记 DB 里已有的事实,不读 worktree、不调外
|
||
部 API;那些产物的硬去重仍靠各自已有的门(创建幂等键、
|
||
合并状态 CAS、分工名复用)。
|
||
- 本轨不做 worktree 文件快照:快照路径因执行器而异
|
||
(local / Docker 租户 / cloud 的 worktree 位置不同),
|
||
在 worker 层硬做快照会把执行器差异漏进账本;agent 在
|
||
本轮里自己查 worktree 现状是更可靠的口径。
|
||
- 手动重试(/retry、/continue 等)不经过 auto_resume,
|
||
故不记尝试账本、不贴前言——用户显式重试时由用户自己
|
||
在新消息里给上下文,本轨不替用户编造上一尝试说明。
|
||
"""
|
||
|
||
import time
|
||
|
||
ATTEMPT_TEXT_CHARS = 2000
|
||
PREAMBLE_TEXT_CHARS = 1000
|
||
|
||
CREATE_TABLE_SQL = (
|
||
"CREATE TABLE IF NOT EXISTS task_attempts("
|
||
"task_id TEXT, "
|
||
"attempt_no INTEGER, "
|
||
"kind TEXT, "
|
||
"interrupted_reason TEXT, "
|
||
"checkpoint_round INTEGER, "
|
||
"result_text TEXT, "
|
||
"error_text TEXT, "
|
||
"created_at DOUBLE PRECISION, "
|
||
"PRIMARY KEY(task_id, attempt_no))")
|
||
|
||
|
||
def ensure_attempts_table(conn):
|
||
"""幂等建尝试账本表(server/worker 两支共用同一 DDL)。"""
|
||
conn.execute(CREATE_TABLE_SQL)
|
||
|
||
|
||
def _trunc(text, limit):
|
||
if text is None:
|
||
return None
|
||
text = str(text)
|
||
return text[:limit] if len(text) > limit else text
|
||
|
||
|
||
def record_attempt(conn, task_id, attempt_no, kind,
|
||
interrupted_reason, checkpoint_round,
|
||
result_text, error_text):
|
||
"""记一次正结束的尝试;重复记同一序号时保留第一次。
|
||
|
||
调用方负责建表(ensure_attempts_table)与事务提交;
|
||
本函数只执行 INSERT OR IGNORE,不提交、不抛主键冲突。
|
||
"""
|
||
conn.execute(
|
||
"INSERT OR IGNORE INTO task_attempts(task_id, attempt_no,"
|
||
" kind, interrupted_reason, checkpoint_round, result_text,"
|
||
" error_text, created_at) VALUES(?, ?, ?, ?, ?, ?, ?, ?)",
|
||
(task_id, attempt_no, kind, interrupted_reason,
|
||
checkpoint_round, _trunc(result_text, ATTEMPT_TEXT_CHARS),
|
||
_trunc(error_text, ATTEMPT_TEXT_CHARS), time.time()))
|
||
|
||
|
||
def latest_attempt(conn, task_id):
|
||
"""最新一次尝试记录;表缺失或无记录时返回 None,绝不炸。"""
|
||
try:
|
||
row = conn.execute(
|
||
"SELECT task_id, attempt_no, kind, interrupted_reason,"
|
||
" checkpoint_round, result_text, error_text, created_at"
|
||
" FROM task_attempts WHERE task_id = ?"
|
||
" ORDER BY attempt_no DESC LIMIT 1",
|
||
(task_id,)).fetchone()
|
||
except Exception:
|
||
return None
|
||
if row is None:
|
||
return None
|
||
try:
|
||
return {
|
||
"task_id": row[0], "attempt_no": row[1], "kind": row[2],
|
||
"interrupted_reason": row[3], "checkpoint_round": row[4],
|
||
"result_text": row[5], "error_text": row[6],
|
||
"created_at": row[7],
|
||
}
|
||
except (IndexError, TypeError):
|
||
return None
|
||
|
||
|
||
def build_resume_preamble(attempt):
|
||
"""由尝试记录拼重跑前言;无记录或非全新重排时返回空串。
|
||
|
||
只用账本里的事实,不推测上一尝试做了哪些文件改动
|
||
(账本不记文件清单,见模块注释),核查动作交给本轮
|
||
agent 在 worktree 里做。
|
||
"""
|
||
if not isinstance(attempt, dict):
|
||
return ""
|
||
if attempt.get("kind") != "platform_fresh":
|
||
return ""
|
||
lines = ["[上次尝试说明:平台中断后重新开始]",
|
||
"这是第 %s 次尝试的重新开始,不是全新任务。"
|
||
% (attempt.get("attempt_no") or "?"),
|
||
"上一尝试中断原因:%s;断点轮次:%s(-1 表示无断点,"
|
||
"非 goal 任务恒为 -1)。"
|
||
% (attempt.get("interrupted_reason") or "未记录",
|
||
attempt.get("checkpoint_round")
|
||
if attempt.get("checkpoint_round") is not None
|
||
else "未知")]
|
||
prev_result = _trunc(attempt.get("result_text"),
|
||
PREAMBLE_TEXT_CHARS)
|
||
prev_error = _trunc(attempt.get("error_text"),
|
||
PREAMBLE_TEXT_CHARS)
|
||
if prev_result:
|
||
lines.append("上一尝试的部分输出:%s" % prev_result)
|
||
if prev_error:
|
||
lines.append("上一尝试的错误:%s" % prev_error)
|
||
lines.append(
|
||
"同一 worktree 里上一尝试的文件仍在:先检查 worktree "
|
||
"现状(例如文件列表与改动)再动手,确认哪些已做、哪些 "
|
||
"未做;不要重复创建上一尝试已创建的东西;对 worktree "
|
||
"外的产物(分支、PR、发布等)无法从 worktree 确认时,"
|
||
"先核查再创建,不要假设没创建过。")
|
||
return "\n".join(lines)
|