Files
codex-app-server/cloud/marker_tools.py
T
host 3b2fef95fa 任务内标记工具:task-create/task-poll/task-read 移植、wait-threads 接通、新增 task-cancel/task-retry
- 新模块 cloud/marker_tools.py:纯逻辑解析、校验、快照格式化、
  控制门禁、派生代数与直接上限计数
- sandbox 两处解析提取五种标记块;worker 在 5 处投递点与
  _deliver_peer_messages 同款位置执行,报告排成后续消息下一轮回父任务
- wait 每块 cap 到 8 个目标(与服务端 /wait 对齐,spawn 原先的 10 个收紧),
  每轮最多 2 块;取消/重试门禁:同项目、非自己、非 goal、
  派生链或同创建人,另查 create_task / approve_merge 权限,
  查不到时收紧到仅派生链内
- goal 与协调员任务不解析;单测 30 个,cloud 全量 1043 个、4 跳过、0 失败
2026-10-28 10:13:40 +00:00

535 lines
21 KiB
Python

"""任务内标记工具(marker tools):task-create / task-poll / task-read /
wait-threads / task-cancel / task-retry。
设计(与 peer_messages、task_spawn 同款原则,不另起一套):
- agent 不用联网、不用凭证:agent 只在回合输出里写标记块,
可信的 worker 在 completed 轮次后解析并执行——本地与
Docker 执行器都通(两者都经 sandbox 的结果解析提取标记块,
与 task-message 同款)。
- 异步语义:写块 → 回合结束 → worker 执行 → 报告拼成后续
消息、下一轮才回到父任务。不是同轮函数返回;轮询与等待
在 worker 执行期间循环查库、占 worker 槽位(与协调员等齐
同款代价),超时或父任务被取消时中止。
- 只收不发:goal 任务与协调员任务不解析这些标记块。本批
工具是给“普通任务当项目经理、自己派生员工”那条路用的,
不是给协调员 PM 用的——协调员任务本身不跑模型回合,等齐
循环也不会在中途排空后续消息做判断。
- 拒绝只记报告与审计、不炸任务、不改目标状态:非法 JSON、
超长、超上限、跨项目、自操作、已终态、goal、已合并、门禁
不过,全部按“跳过 + 报告里写清原因”处理。
四个工具的分工,不要混:
- task-create:新建一个平级主线任务(不是子代理,也不是
fork)。新任务 prompt 由标记块给,不继承父任务的历史;
环境基线继承父任务的 environment_id,但 worker 会给它独立
的 tenant/worktree,不动父任务的 cwd。父任务未提交的改动
不会自动带过去。
- task-poll / task-read:按任务 id 轮询阅读。task-poll 按
interval 查库直到终态或超时;task-read 是同款别名,
不传 wait_seconds 时立即读一次快照,传了则等这么久再读
一次。read-thread / read_thread 是同款别名。
- wait-threads:一次等多个目标任务,并行查库(每轮查一次
全部目标,不是逐个串行等超时相加),全部终态或超时后把
每目标快照一起排回父任务。
- task-cancel / task-retry:对已存在的任务做取消 / 重试。
取消复用 worker 现有的取消路径语义(取消后状态为
interrupted);重试与服务端 POST /tasks/{id}/retry 对齐
(仅 failed / interrupted / merge-failed 可重试,重试时
清空上一轮 result_text / error_text / actual_model、状态
回 queued、prompt 与模型等原样保留)。
上限口径(本模块最终定的值,文档与代码一致):
- 每轮创建、轮询、取消、重试各最多 5 块;wait 每轮最多
2 块,每块最多 8 个目标。8 这个数是刻意与服务端
POST /tasks/{id}/wait 的“一次最多 8 个线程”对齐的;
spawn 分支原先的每块 10 个在此收紧到 8 个,不要再用 10。
- 超时口径与服务端 /wait 不同,且是故意的:服务端 /wait
是给外部调用方持 HTTP 连接等的,上限 120 秒;标记 wait
是 worker 在回合后直连库循环查,不持 HTTP 连接,超时
钳制与 task-poll 同款(0-3600 秒,0 = 立即快照),占的
是 worker 槽位而不是 HTTP 连接。两者不要混用、不要互抄
上限。
- 派生护栏:创建 prompt 1-20000 字,name 可选 1-100 字
(仅用于派生报告,tasks 表没有 name 列,不落库);
model 不做白名单;effort 必须是 codex 的 effort 档位;
executor 必须是 local / docker / cloud,未指定时继承父任务;
派生深度上限 3 代、单个父任务直接派生上限 20 个。
门禁(task-cancel / task-retry,纯函数层 + worker 权限层):
- 纯函数层(本模块 check_control_gate):目标存在、同项目、
非自己、非 goal、状态符合 kind(取消要求非终态,重试
要求仅三态)、关系为派生链内或同创建人,二选一满足即可。
“同项目猜到 id 就能取消任何人的任务”在此被挡住。
- worker 权限层(worker.py _deliver_marker_tools):取消要求
发送方创建人有项目 create_task 权限(与服务端取消口径
同款),重试要求有 approve_merge 权限(与服务端重试口
径同款)。权限检查本身抛异常时收紧到仅派生链内可用,
同创建人-only 的目标一律拒绝,并把原因写进报告——绝不
在查不到权限时静默放宽。
本模块只依赖标准库,不 import cloud 其它模块,避免循环
import。DB 助手函数只用 ? 占位符,与 peer_messages 同款写法。
"""
import json
import re
import time
import uuid
MAX_CREATES_PER_TURN = 5
MAX_POLLS_PER_TURN = 5
MAX_WAITS_PER_TURN = 2
MAX_WAIT_TARGETS_PER_BLOCK = 8
MAX_CONTROLS_PER_TURN = 5
MAX_PROMPT_CHARS = 20000
MAX_NAME_CHARS = 100
MAX_SPAWN_DEPTH = 3
MAX_DIRECT_CHILDREN = 20
DEFAULT_POLL_INTERVAL_SECONDS = 5
DEFAULT_POLL_TIMEOUT_SECONDS = 300
MAX_POLL_TIMEOUT_SECONDS = 3600
DEFAULT_SNAPSHOT_CHARS = 2000
MAX_SNAPSHOT_CHARS = 8000
TERMINAL_STATUSES = ("completed", "failed", "interrupted", "merged",
"merge-failed")
RETRYABLE_STATUSES = ("failed", "interrupted", "merge-failed")
EFFORTS = ("low", "medium", "high", "xhigh", "max")
EXECUTORS = ("local", "docker", "cloud")
_CREATE_RE = re.compile(r"```task-create\s*\n(.*?)\n```", re.DOTALL)
_POLL_RE = re.compile(
r"```(task-poll|task_poll|task-read|task_read|read-thread|read_thread)"
r"\s*\n(.*?)\n```", re.DOTALL)
_WAIT_RE = re.compile(r"```(wait-threads|wait_threads)\s*\n(.*?)\n```",
re.DOTALL)
_CANCEL_RE = re.compile(r"```(task-cancel|task_cancel)\s*\n(.*?)\n```",
re.DOTALL)
_RETRY_RE = re.compile(r"```(task-retry|task_retry)\s*\n(.*?)\n```",
re.DOTALL)
def row_get(row, name):
"""与 peer_messages.row_get 同款容错取值。"""
try:
return row[name]
except (KeyError, IndexError, TypeError):
return None
def _as_int(value, default):
if isinstance(value, bool):
return default
if isinstance(value, int):
return value
if isinstance(value, float) and value.is_integer():
return int(value)
return default
def _clamp(value, low, high, default):
value = _as_int(value, default)
if value < low:
return low
if value > high:
return high
return value
def extract_task_creates(text):
"""提取 task-create 块;prompt 缺失/空串/超长、name 超长、
effort/executor 非法时整块跳过;每轮最多 5 块;正文里的围栏
会把块截断成非法 JSON 而被跳过(与 task-message 同款非贪婪
匹配)。"""
if not text:
return []
out = []
for match in _CREATE_RE.finditer(str(text)):
if len(out) >= MAX_CREATES_PER_TURN:
break
try:
payload = json.loads(match.group(1))
except (TypeError, ValueError):
continue
if not isinstance(payload, dict):
continue
prompt = payload.get("prompt")
if not isinstance(prompt, str) or not prompt.strip():
continue
prompt = prompt.strip()
if len(prompt) > MAX_PROMPT_CHARS:
continue
name = payload.get("name")
if name is not None:
if not isinstance(name, str) or not name.strip():
continue
name = name.strip()
if len(name) > MAX_NAME_CHARS:
continue
model = payload.get("model")
if model is not None:
if not isinstance(model, str) or not model.strip():
continue
model = model.strip()
effort = payload.get("effort")
if effort is not None and effort not in EFFORTS:
continue
executor = payload.get("executor")
if executor is not None and executor not in EXECUTORS:
continue
out.append({"name": name, "prompt": prompt, "model": model,
"effort": effort, "executor": executor})
return out
def extract_task_polls(text):
"""提取 task-poll / task-read(含 read-thread 别名)块。
task-read 不传 wait_seconds 时立即读一次快照(timeout=0);
传了则等这么久再读一次。interval 钳制 1-60 秒,timeout
钳制 0-3600 秒,chars 钳制 200-8000 字;各类块合计每轮
最多 5 块,多余的忽略。"""
if not text:
return []
out = []
for match in _POLL_RE.finditer(str(text)):
if len(out) >= MAX_POLLS_PER_TURN:
break
kind, raw = match.group(1), match.group(2)
try:
payload = json.loads(raw)
except (TypeError, ValueError):
continue
if not isinstance(payload, dict):
continue
task_id = payload.get("task_id") or payload.get("to") \
or payload.get("to_task_id")
if not isinstance(task_id, str) or not task_id.strip():
continue
chars = _clamp(payload.get("chars"), 200, MAX_SNAPSHOT_CHARS,
DEFAULT_SNAPSHOT_CHARS)
if kind in ("task-read", "task_read", "read-thread",
"read_thread"):
wait = payload.get("wait_seconds")
if wait is None:
interval = DEFAULT_POLL_INTERVAL_SECONDS
timeout = 0
else:
timeout = _clamp(wait, 0, MAX_POLL_TIMEOUT_SECONDS, 0)
interval = timeout if timeout > 0 \
else DEFAULT_POLL_INTERVAL_SECONDS
else:
interval = _clamp(payload.get("interval_seconds"), 1, 60,
DEFAULT_POLL_INTERVAL_SECONDS)
timeout = _clamp(payload.get("timeout_seconds"), 0,
MAX_POLL_TIMEOUT_SECONDS,
DEFAULT_POLL_TIMEOUT_SECONDS)
out.append({"task_id": task_id.strip(),
"interval_seconds": interval,
"timeout_seconds": timeout, "chars": chars})
return out
def extract_wait_threads(text):
"""提取 wait-threads / wait_threads 块:一次等多个目标。
块格式::
```wait-threads
{"task_ids": ["B", "C"], "interval_seconds": 5,
"timeout_seconds": 300, "chars": 2000}
```
task_ids 也接受 thread_ids / threads 两个别名。去重保序,
每块最多 8 个目标(与服务端 /wait 口径对齐,见模块头),
每轮最多 2 块;无有效 id 的块跳过。interval 钳制 1-60 秒,
timeout 钳制 0-3600 秒(worker 直连库口径,见模块头),
chars 钳制 200-8000 字。"""
if not text:
return []
out = []
for match in _WAIT_RE.finditer(str(text)):
if len(out) >= MAX_WAITS_PER_TURN:
break
try:
payload = json.loads(match.group(2))
except (TypeError, ValueError):
continue
if not isinstance(payload, dict):
continue
raw_ids = (payload.get("task_ids") or payload.get("thread_ids")
or payload.get("threads"))
if not isinstance(raw_ids, list):
continue
ids = []
for tid in raw_ids:
if isinstance(tid, str) and tid.strip() \
and tid.strip() not in ids:
ids.append(tid.strip())
if not ids:
continue
ids = ids[:MAX_WAIT_TARGETS_PER_BLOCK]
out.append({
"task_ids": ids,
"interval_seconds": _clamp(
payload.get("interval_seconds"), 1, 60,
DEFAULT_POLL_INTERVAL_SECONDS),
"timeout_seconds": _clamp(
payload.get("timeout_seconds"), 0,
MAX_POLL_TIMEOUT_SECONDS, DEFAULT_POLL_TIMEOUT_SECONDS),
"chars": _clamp(payload.get("chars"), 200,
MAX_SNAPSHOT_CHARS, DEFAULT_SNAPSHOT_CHARS),
})
return out
def _extract_controls(text, regex):
"""提取 task-cancel / task-retry 块的公共逻辑。
task_id 缺失/空串时整块跳过;to / to_task_id / thread_id
是同款别名;每种每轮最多 5 块,多余的忽略。"""
if not text:
return []
out = []
for match in regex.finditer(str(text)):
if len(out) >= MAX_CONTROLS_PER_TURN:
break
try:
payload = json.loads(match.group(2))
except (TypeError, ValueError):
continue
if not isinstance(payload, dict):
continue
task_id = (payload.get("task_id") or payload.get("to")
or payload.get("to_task_id")
or payload.get("thread_id"))
if not isinstance(task_id, str) or not task_id.strip():
continue
out.append({"task_id": task_id.strip()})
return out
def extract_task_cancels(text):
"""提取 task-cancel 块:{"task_id": "..."}。"""
return _extract_controls(text, _CANCEL_RE)
def extract_task_retries(text):
"""提取 task-retry 块:{"task_id": "..."}。"""
return _extract_controls(text, _RETRY_RE)
def parent_spawn_generation(conn, parent_id):
"""父任务自身的派生代数:普通任务 0,第一代派生任务 1,依此类推。
只数 parent_task_id 非空且 coordinator_part 为空的祖先链(即
派生链);协调员子任务(coordinator_part 非空)本身不是派生
任务,代数为 0,链到此为止。走 32 步仍未到头时按已数到的算,
绝不死循环。"""
generation = 0
cur = parent_id
for _ in range(32):
if not cur:
break
row = conn.execute(
"SELECT parent_task_id, coordinator_part FROM tasks"
" WHERE id = ?", (cur,)).fetchone()
if row is None:
break
parent = row_get(row, "parent_task_id")
part = row_get(row, "coordinator_part")
if parent is not None and part is None:
generation += 1
cur = parent
continue
break
return generation
def count_direct_spawn_children(conn, parent_id):
"""单个父任务的直接派生子任务数(不含协调员子任务)。"""
row = conn.execute(
"SELECT COUNT(*) FROM tasks WHERE parent_task_id = ?"
" AND coordinator_part IS NULL", (parent_id,)).fetchone()
try:
return int(row[0]) if row is not None else 0
except (TypeError, ValueError, IndexError):
return 0
def fetch_task_snapshot(conn, task_id):
"""取目标任务的快照行;不存在时回 None。
用 SELECT * 而不是列清单:老库/最小测试表缺列时 row_get
容错回 None,不炸;门禁要用的 created_by / parent_task_id /
coordinator_part / verify_command / goal_mode 一并带上。"""
row = conn.execute(
"SELECT * FROM tasks WHERE id = ?", (task_id,)).fetchone()
if row is None:
return None
return {"id": row_get(row, "id"),
"project_id": row_get(row, "project_id"),
"status": row_get(row, "status"),
"result_text": row_get(row, "result_text"),
"error_text": row_get(row, "error_text"),
"executor": row_get(row, "executor"),
"model": row_get(row, "model"),
"actual_model": row_get(row, "actual_model"),
"turn_id": row_get(row, "turn_id"),
"last_heartbeat": row_get(row, "last_heartbeat"),
"worktree_host": row_get(row, "worktree_host"),
"pr_url": row_get(row, "pr_url"),
"pr_number": row_get(row, "pr_number"),
"environment_id": row_get(row, "environment_id"),
"created_by": row_get(row, "created_by"),
"parent_task_id": row_get(row, "parent_task_id"),
"coordinator_part": row_get(row, "coordinator_part"),
"verify_command": row_get(row, "verify_command"),
"goal_mode": row_get(row, "goal_mode")}
def ancestry_relation(conn, sender, target):
"""发送方与目标的关系:descendant / same_creator / other。
descendant:沿目标的 parent_task_id 链(最多 32 步)能追溯
到发送方(直接派生、隔代派生都算;协调员子任务的链到
协调员为止,不会穿过协调员再往上算——因为协调员子任务的
coordinator_part 非空,链在此处按派生链定义截断,见
parent_spawn_generation 的同款处理;但本函数走的是
parent_task_id 原始链,目标本身若是协调员子任务、其父
是协调员时,发送方若恰好是该协调员也会算 descendant——
协调员任务不解析标记块,所以发送方不可能是协调员,
这一支在实际调用里到不了,不影响门禁)。
same_creator:链上追不到,但目标 created_by 与发送方
created_by 相同且非空。
other:两者都不是。"""
sender_id = (sender or {}).get("id")
target_parent = (target or {}).get("parent_task_id")
cur = target_parent
for _ in range(32):
if not cur:
break
if cur == sender_id:
return "descendant"
row = conn.execute(
"SELECT parent_task_id FROM tasks WHERE id = ?",
(cur,)).fetchone()
if row is None:
break
cur = row_get(row, "parent_task_id")
sender_creator = (sender or {}).get("created_by")
target_creator = (target or {}).get("created_by")
if sender_creator and target_creator \
and sender_creator == target_creator:
return "same_creator"
return "other"
def is_goal_task(task):
"""goal 任务判定:verify_command 非空或 goal_mode 为真。
与 worker run_one 的 goal 分支同款(verify_command 或
goal_mode);goal 字段本身是 goal_mode=1 时 prompt 的副本,
不单独当判定条件,避免把普通任务的 goal 文本误判。"""
if not task:
return False
return bool(task.get("verify_command") or task.get("goal_mode"))
def check_control_gate(target, sender, relation, kind):
"""task-cancel / task-retry 的纯函数门禁,返回 (ok, reason)。
kind 只能是 "cancel" 或 "retry",其它值一律拒绝。权限层
不在这里:worker 侧另查 create_task / approve_merge,见
模块头;本函数只管目标状态与关系,单测不需要 AuthStore。"""
if kind not in ("cancel", "retry"):
return False, "未知的控制操作"
if target is None:
return False, "目标任务不存在"
if (target.get("project_id") is not None
and sender.get("project_id") is not None
and target.get("project_id") != sender.get("project_id")):
return False, "跨项目任务不可操作"
if target.get("id") == sender.get("id"):
return False, "不能操作自己"
if is_goal_task(target):
return False, "goal 任务不可取消或重试(goal 循环自有轮次)"
status = target.get("status")
if kind == "cancel":
if status in TERMINAL_STATUSES:
return False, "目标已终态(%s),不可取消" % status
else:
if status not in RETRYABLE_STATUSES:
return False, ("仅 failed / interrupted / merge-failed"
" 可重试,当前状态为 %s" % status)
if relation not in ("descendant", "same_creator"):
return False, "目标不在发送方派生链上且创建人不同"
return True, ""
def _heartbeat_age_text(snapshot, now=None):
"""最后心跳距今秒数,无心跳时回空串(不编造)。"""
hb = snapshot.get("last_heartbeat")
if hb is None:
return ""
try:
age = int((now if now is None else now) - float(hb))
except (TypeError, ValueError):
return ""
if age < 0:
age = 0
return "%d 秒前" % age
def _progress_lines(snapshot, now=None):
"""快照的进度行:执行器、模型、工作树主机、心跳、turn。"""
lines = []
if snapshot.get("executor"):
lines.append("执行器:%s" % snapshot["executor"])
model = snapshot.get("actual_model") or snapshot.get("model")
if model:
lines.append("模型:%s" % model)
if snapshot.get("worktree_host"):
lines.append("工作树主机:%s" % snapshot["worktree_host"])
age = _heartbeat_age_text(snapshot, now=now)
if age:
lines.append("最后心跳:%s" % age)
if snapshot.get("turn_id"):
lines.append("回合:%s" % snapshot["turn_id"])
return lines
def format_snapshot(snapshot, chars, timed_out=False, cancelled=False):
"""把快照格式化成给父任务下一轮的报告正文(截断防上下文爆炸)。"""
task_id = snapshot["id"]
status = snapshot["status"] or "?"
lines = ["[任务 %s]" % task_id, "状态:%s" % status]
lines.extend(_progress_lines(snapshot))
if cancelled:
lines.append("说明:等待期间父任务被取消,轮询已中止。")
elif timed_out:
lines.append("说明:等待超时,目标任务仍未终态,可再轮询一次。")
elif status in TERMINAL_STATUSES:
result = (snapshot["result_text"] or "").strip()
error = (snapshot["error_text"] or "").strip()
if result:
lines.append("结果:%s" % result[:chars])
if error:
lines.append("错误:%s" % error[:chars])
if not result and not error:
lines.append("结果:(空)")
else:
lines.append("说明:目标任务仍在运行或排队,可再轮询一次。")
return "\n".join(lines)
def new_child_id():
return uuid.uuid4().hex
def marker_report_header():
return "[任务工具报告]"