fix: second-audit gate findings (B1, I1-I8, M1-M10)
B1: gateway capability check now triggers on walking the sub2api gateway, not on the gateway having filled a model; task-level explicit models are audited (worker_capability_gateway_explicit) but never blocked, preserving explicit priority; gateway-default models still fail-fast under active. I1: dry-run adds a gateway-model section, sourced only from CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active tier whose requires the gateway model fails or cannot verify blocks, shadow only warns. I2: shadow review gate now counts completed samples (completed >= 20), same basis for sample_shortfall; the endpoint attaches per-tier capability status (read-only policy load) and flags capability-failing tiers in the recommendation. I3: configured-but-broken routing policy logs a warning in load_routing_policy (unconfigured stays silent); the worker sets routing_policy_load_failed and audits worker_routing_policy_load_failed per task. I4: resume preamble wraps prior output/error as data with an explicit do-not-execute constraint; position verified in run_one (tree -> preamble -> project -> original) and the docs corrected from the false 'closest to the original prompt' claim. I5: decided model is written back with a CAS UPDATE (model IS NULL OR model=''), so later attempts reuse it; explicit models are never overwritten. I6: agent_* columns added to worker._ensure_result_columns and db.init_postgres_schema backfill; migrations backfills them for tasks before insert (sender_task_id pattern); column-level tests for agent_tool_approvals and task_attempts. I7: coordinator requeue adds AND status=<snapshot> (rowcount 0 skips requeue/audit) and filters paused_by_user / user_cancel / user_pause children, which are never revived or cleared. I8: explicit-model tasks (including coordinator children that inherited a parent model) audit a source=explicit worker_routing row; evaluate_shadow counts it as per-tier explicit_n only, never routed_n, never a dedup slot. M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation, shadow sort doc fixed, truncated recommendation notes truncation, seed cleanup on gateway/quota/goal-cap early returns, tools-only coordinator definitions rejected, preamble goal branch marked effectively unreachable, DESIGN.md drift corrected (outside repo). M4/M7/M9: documentation/new-field only (pass_rate_fully_measured, whitespace-model shadow-drop note, bound-note race note); old field semantics unchanged.
This commit is contained in:
1 parent
f9a77577c5
commit
621c08d10f
13 files changed
+907
-44
No files matched your search
+174
-17
@@ -538,15 +538,30 @@ class Worker:
|
||||
# 解析失败 / 未配时为 None,派发回落旧链,绝不因此拒绝任务。
|
||||
# 策略里的 fallback_models 在 v0 不启用真实重试(见 models.py),
|
||||
# 只有 CLI --fallback-models 能启用,策略文件 alone 不能烧钱。
|
||||
# Audit fix 3(I3):策略加载失败标志。True 只在
|
||||
# “配置了策略但解析不出可用策略”时置真:未配置
|
||||
# (无 env、无 dict)是正常状态,不置标志;显式
|
||||
# RoutingPolicy 对象不经解析,不置标志。run_one 里
|
||||
# 该标志为真时记一条 worker_routing_policy_load_failed
|
||||
# 审计,让“策略已死”在运行期可见;models 侧的警告
|
||||
# 日志见 load_routing_policy。
|
||||
self.routing_policy_load_failed = False
|
||||
if isinstance(routing_policy, RoutingPolicy):
|
||||
self.routing_policy = routing_policy
|
||||
elif isinstance(routing_policy, dict):
|
||||
self.routing_policy = parse_routing_policy(routing_policy)
|
||||
if self.routing_policy is None:
|
||||
self.routing_policy_load_failed = True
|
||||
elif routing_policy is None:
|
||||
_policy_configured = bool(
|
||||
self._env.get("CLOUD_ROUTING_POLICY_FILE")
|
||||
or self._env.get("CLOUD_ROUTING_POLICY_JSON"))
|
||||
try:
|
||||
self.routing_policy = load_routing_policy(self._env)
|
||||
except Exception:
|
||||
self.routing_policy = None
|
||||
if self.routing_policy is None and _policy_configured:
|
||||
self.routing_policy_load_failed = True
|
||||
else:
|
||||
raise ValueError("routing_policy 必须是 RoutingPolicy 或 dict")
|
||||
# 真实 fallback 换模型列表:默认空 = 关闭,只记事件不换模型
|
||||
@@ -1535,6 +1550,16 @@ class Worker:
|
||||
"parent_task_id": "TEXT",
|
||||
"coordinator_part": "TEXT",
|
||||
"goal_resume_pending": "INTEGER DEFAULT 0",
|
||||
# Audit fix 3(I6):agent 列此前不在 worker
|
||||
# 兜底清单里(文档称“与 migrate_tasks_table
|
||||
# 同列”并不属实),worker 直连老库时读带
|
||||
# agent 定义的任务行会缺列;补齐,与 server
|
||||
# 清单、db.py 回填清单三处一致。
|
||||
"agent_name": "TEXT",
|
||||
"agent_version": "TEXT",
|
||||
"agent_instructions": "TEXT",
|
||||
"agent_tools": "TEXT",
|
||||
"agent_tool_approvals": "TEXT",
|
||||
"paused_by_user": "INTEGER DEFAULT 0",
|
||||
"interrupted_reason": "TEXT",
|
||||
"auto_resume_count": "INTEGER DEFAULT 0",
|
||||
@@ -1976,9 +2001,17 @@ class Worker:
|
||||
会话连续性)、auto_resume_count 为 0 的首轮任务、账本
|
||||
里查不到记录的老任务、最新尝试不是 platform_fresh
|
||||
的任务(goal 断点续跑走线程连续性,不贴前言)。
|
||||
贴法与项目注入同款:前言 + 原 prompt,goal 任务若
|
||||
意外走到这里也同步贴 goal(执行器取 goal 优先于
|
||||
prompt);查账本失败时原样返回,不炸认领。
|
||||
贴法(Audit fix 3,I4,已核实 run_one 调用顺序):
|
||||
run_one 里项目注入在前、前言在后,前言以前缀方式
|
||||
贴在当前 prompt 最前端——此时 prompt 已含项目注入,
|
||||
其后目录树注入还会再前置一层,最终顺序是目录树
|
||||
→ 前言 → 项目上下文 → 原 prompt。旧文档“离原
|
||||
prompt 最近”的说法是错的:前言离原 prompt 最远、
|
||||
是模型最先看到的一段之一;位置是刻意的前置(重
|
||||
排说明应当先被看到),不是紧贴原 prompt。goal
|
||||
任务若意外走到这里也同步贴 goal(执行器取 goal
|
||||
优先于 prompt);该分支正常流程不可达(见下),
|
||||
只是防御性保留。查账本失败时原样返回,不炸认领。
|
||||
"""
|
||||
if task.get("coordinator_parts"):
|
||||
return task
|
||||
@@ -2001,6 +2034,13 @@ class Worker:
|
||||
new_task["prompt"] = preamble + "\n\n" + (
|
||||
task.get("prompt") or "")
|
||||
if task.get("goal_mode") or task.get("verify_command"):
|
||||
# Audit fix 3(M8):此分支正常流程不可达——
|
||||
# build_resume_preamble 只对 platform_fresh 出
|
||||
# 前言,而 auto_resume 把 goal 任务一律归为
|
||||
# goal_breakpoint;且 goal 断点任务在函数入口
|
||||
# 已被 _goal_resume_breakpoint 挡回。保留不删:
|
||||
# 万一未来有 platform_fresh 的 goal 尝试,goal
|
||||
# 字段不同步贴前言会静默丢说明。
|
||||
new_task["goal"] = preamble + "\n\n" + (
|
||||
task.get("goal") or task.get("prompt") or "")
|
||||
new_task["_resume_preamble_injected"] = True
|
||||
@@ -5113,17 +5153,32 @@ class Worker:
|
||||
conn.commit()
|
||||
except Exception:
|
||||
pass
|
||||
# Audit fix 3(I7):快照同时取 paused_by_user
|
||||
# 与 interrupted_reason,重排前才能过滤用户暂停/
|
||||
# 取消的子任务;列缺失的老行按 0/None 计,不因
|
||||
# 缺列把整个子任务从快照里丢掉。
|
||||
rows = conn.execute(
|
||||
"SELECT id, coordinator_part, status FROM tasks"
|
||||
"SELECT id, coordinator_part, status, paused_by_user,"
|
||||
" interrupted_reason FROM tasks"
|
||||
" WHERE parent_task_id = ?",
|
||||
(task["id"],)).fetchall()
|
||||
finally:
|
||||
conn.close()
|
||||
existing = {}
|
||||
for row in rows:
|
||||
try:
|
||||
_paused = row["paused_by_user"]
|
||||
except (KeyError, IndexError, TypeError):
|
||||
_paused = 0
|
||||
try:
|
||||
_reason = row["interrupted_reason"]
|
||||
except (KeyError, IndexError, TypeError):
|
||||
_reason = None
|
||||
try:
|
||||
existing[row["coordinator_part"]] = {
|
||||
"id": row["id"], "status": row["status"]}
|
||||
"id": row["id"], "status": row["status"],
|
||||
"paused_by_user": _paused,
|
||||
"interrupted_reason": _reason}
|
||||
except (KeyError, IndexError, TypeError):
|
||||
continue
|
||||
now = time.time()
|
||||
@@ -5144,16 +5199,35 @@ class Worker:
|
||||
task["project_id"],
|
||||
"coordinator_child_tolerated",
|
||||
"%s -> %s" % (name, prev["id"]))
|
||||
else:
|
||||
conn.execute(
|
||||
"UPDATE tasks SET status = 'queued',"
|
||||
" result_text = NULL, error_text = NULL,"
|
||||
" updated_at = ? WHERE id = ?",
|
||||
(now, prev["id"]))
|
||||
elif prev.get("paused_by_user") or \
|
||||
prev.get("interrupted_reason") in (
|
||||
"user_cancel", "user_pause"):
|
||||
# Audit fix 3(I7):用户暂停/取消
|
||||
# 的子任务不复活、不清结果。口径与
|
||||
# auto_resume 一致(paused_by_user
|
||||
# 由 /pause、/cancel 置 1),外加
|
||||
# reason 双保险,防某一字段陈旧。
|
||||
self._audit(
|
||||
task["project_id"],
|
||||
"coordinator_child_requeued",
|
||||
"coordinator_child_paused_skipped",
|
||||
"%s -> %s" % (name, prev["id"]))
|
||||
else:
|
||||
# Audit fix 3(I7):重排加状态
|
||||
# 条件。快照与更新之间子任务若被
|
||||
# 人工重试并跑完,无条件 UPDATE 会
|
||||
# 把已完成任务清结果重排;rowcount
|
||||
# 为 0 说明状态已变,跳过重排与审计。
|
||||
_cur = conn.execute(
|
||||
"UPDATE tasks SET status = 'queued',"
|
||||
" result_text = NULL, error_text = NULL,"
|
||||
" updated_at = ? WHERE id = ?"
|
||||
" AND status = ?",
|
||||
(now, prev["id"], prev["status"]))
|
||||
if _cur.rowcount:
|
||||
self._audit(
|
||||
task["project_id"],
|
||||
"coordinator_child_requeued",
|
||||
"%s -> %s" % (name, prev["id"]))
|
||||
children.append({"name": name, "id": prev["id"]})
|
||||
continue
|
||||
# 自动化入闸(创建时门):子任务认领时虽有配额门,
|
||||
@@ -5579,6 +5653,13 @@ class Worker:
|
||||
"error_text": row.get("error_text")})
|
||||
report, overall = _coord_build_report(merged, parts)
|
||||
if bound_exceeded:
|
||||
# Audit fix 3(M9,文档级):此注记在取消时
|
||||
# 记下,报告末尾无条件 append;取消与子任务
|
||||
# 完成竞态时子任务最终可能是 completed,注记
|
||||
# 不会纠正。这里不改语义(改“取消成功才注记”
|
||||
# 需在收口处再查一次子任务终态并处理竞态窗口,
|
||||
# 另起一轨);读报告时以子任务终态为准,注记
|
||||
# 只说明“曾触发超时取消”。
|
||||
report += ("\n\n有界委派超时:%s(已取消该子任务,"
|
||||
"其他分工继续等齐)" % "、".join(bound_exceeded))
|
||||
if orphaned:
|
||||
@@ -5830,11 +5911,23 @@ class Worker:
|
||||
# _apply_project_injection 的跳过规则;只改内存副本;
|
||||
# 最后贴,排在待办与记忆之前(最通用上下文在最前)。
|
||||
task = self._apply_project_injection(task)
|
||||
# 重跑前言:平台全新重排的任务在项目注入之后贴,离原
|
||||
# prompt 最近;只改内存副本,原 prompt 不落库。
|
||||
# 重跑前言:平台全新重排的任务在项目注入之后、
|
||||
# 以前缀贴在整个 prompt 最前端(Audit fix 3,I4:
|
||||
# 旧注释“离原 prompt 最近”是错的——前言与原 prompt
|
||||
# 之间隔着项目上下文,其后目录树注入还会再前置一层;
|
||||
# 位置是刻意的前置,不是紧贴原 prompt)。只改内存
|
||||
# 副本,原 prompt 不落库。
|
||||
task = self._apply_resume_preamble(task)
|
||||
self._audit(task["project_id"], "worker_claim",
|
||||
"%s executor=%s" % (task_id, self._executor_name()))
|
||||
# Audit fix 3(I3):策略已死时每任务记一条审计,
|
||||
# 否则运营方只能靠日志猜“策略为什么没生效”。
|
||||
# _audit 本身 best-effort,不影响派发。
|
||||
if getattr(self, "routing_policy_load_failed", False):
|
||||
self._audit(task["project_id"],
|
||||
"worker_routing_policy_load_failed",
|
||||
"%s project=%s" % (task_id,
|
||||
task["project_id"]))
|
||||
# log_event 只接受 task_id/project_id/status/message 四个
|
||||
# 字段,executor 只进审计 detail,不进日志调用(否则
|
||||
# TypeError 会炸掉整个 run_one)。
|
||||
@@ -5926,6 +6019,8 @@ class Worker:
|
||||
log_event(self.logger, "worker_quota_stop", task_id=task_id,
|
||||
project_id=task["project_id"],
|
||||
status="interrupted")
|
||||
# Audit fix 3(M5):配额门早返回也要清暂存。
|
||||
self._cleanup_task_seed(task)
|
||||
# 同缺 key 门:配额门在起进程之前返回,先清占位。
|
||||
self._running_clear(task_id)
|
||||
return {"dispatched": True, "task_id": task_id,
|
||||
@@ -5977,6 +6072,9 @@ class Worker:
|
||||
log_event(self.logger, "worker_goal_cap_rejected",
|
||||
task_id=task_id, project_id=task["project_id"],
|
||||
status="failed")
|
||||
# Audit fix 3(M5):goal 轮次门早返回也要
|
||||
# 清暂存,同缺 key 门。
|
||||
self._cleanup_task_seed(task)
|
||||
self._running_clear(task_id)
|
||||
return {"dispatched": True, "task_id": task_id,
|
||||
"host_name": self.executor._host_name(task)
|
||||
@@ -6020,10 +6118,17 @@ class Worker:
|
||||
# xcmapi 模型名去撞 sub2api)。xcmapi 下 _ensure_gateway
|
||||
# 直接返回 (None, None),不做任何事。注意:这里改的是
|
||||
# 原 task dict,随后 task = dict(task) 会原样拷走。
|
||||
# Audit fix 3(B1):触发条件是“走了 sub2api 网关”,
|
||||
# 不是“网关填了模型”。任务级显式模型此前因
|
||||
# _gateway_filled=False 全程跳过能力检查,是生产
|
||||
# 覆盖的一半缺口;显式优先原则不变——显式模型
|
||||
# 不满足/unknown 时只审计、不拦任务(见下)。
|
||||
_explicit_model = bool(task.get("model"))
|
||||
_gateway_filled = False
|
||||
gateway_provider = None
|
||||
try:
|
||||
gateway_provider, gateway_env = self._ensure_gateway(task)
|
||||
if not task.get("model") and gateway_provider == "sub2api":
|
||||
if not _explicit_model and gateway_provider == "sub2api":
|
||||
task["model"] = self._sub2api_default_model(gateway_env)
|
||||
_gateway_filled = True
|
||||
except Exception as exc:
|
||||
@@ -6040,6 +6145,9 @@ class Worker:
|
||||
status="failed")
|
||||
self._notify_task_finished(
|
||||
task_id, task["project_id"], "failed", error=error)
|
||||
# Audit fix 3(M5):网关失败早返回也要清暂存,
|
||||
# 与缺 key 门、仓库门同款,否则 bundle 目录残留。
|
||||
self._cleanup_task_seed(task)
|
||||
return {"dispatched": True, "task_id": task_id,
|
||||
"host_name": self.executor._host_name(task)
|
||||
if hasattr(self.executor, "_host_name") else task_id,
|
||||
@@ -6059,7 +6167,8 @@ class Worker:
|
||||
# active 下不满足即拦,不存在“先试 worker 默认”
|
||||
# 的中间态;运营方须先把网关模型写进能力目录,
|
||||
# 否则 unknown 同样拦(fail-closed,与 v8 一致)。
|
||||
if _gateway_filled and self.routing_policy is not None:
|
||||
if gateway_provider == "sub2api" \
|
||||
and self.routing_policy is not None:
|
||||
_gw_tier = classify_task(task)
|
||||
_gw_requires = self.routing_policy.tier_requirements.get(
|
||||
_gw_tier) or {}
|
||||
@@ -6077,7 +6186,20 @@ class Worker:
|
||||
"cannot verify"
|
||||
if _gw_unknown else "lacks",
|
||||
", ".join(_gw_missing)))
|
||||
if _gw_mode == "shadow":
|
||||
if _explicit_model:
|
||||
# 显式优先:不拦任务,但绝不能无声
|
||||
# 豁免——记独立审计,含缺项/unknown,
|
||||
# 影子解析器不认此 action,不污染样本。
|
||||
self._audit(
|
||||
task["project_id"],
|
||||
"worker_capability_gateway_explicit",
|
||||
"%s model=%s tier=%s mode=%s"
|
||||
" missing=%s unknown=%s" % (
|
||||
task_id, task.get("model"),
|
||||
_gw_tier, _gw_mode,
|
||||
",".join(_gw_missing) or "-",
|
||||
_gw_unknown))
|
||||
elif _gw_mode == "shadow":
|
||||
self._audit(
|
||||
task["project_id"],
|
||||
"worker_capability_gateway_shadow",
|
||||
@@ -6106,6 +6228,9 @@ class Worker:
|
||||
task_id=task_id,
|
||||
project_id=task["project_id"],
|
||||
status="failed")
|
||||
# Audit fix 3(M5):网关门拒绝
|
||||
# 早返回也要清暂存,同缺 key 门。
|
||||
self._cleanup_task_seed(task)
|
||||
self._running_clear(task_id)
|
||||
return {"dispatched": True,
|
||||
"task_id": task_id,
|
||||
@@ -6191,7 +6316,39 @@ class Worker:
|
||||
task_id, decision.tier,
|
||||
decision.chosen_model or "-",
|
||||
decision.reason))
|
||||
# Audit fix 3(I5):跨尝试复用靠这次写回,不是
|
||||
# 靠单次 run_one 内的内存副本。旧版只改 task
|
||||
# 副本,模型为空的任务每次派发都会重新决策,
|
||||
# 策略一改同一任务中途就换模型。CAS 式写回:
|
||||
# 只在库模型仍为空时写,绝不覆盖显式模型;
|
||||
# 写失败 best-effort,不炸派发(本轮内存模型
|
||||
# 照常生效,只是下次派发可能再次决策)。
|
||||
if decision.chosen_model:
|
||||
try:
|
||||
_wb_conn = _db_connect(self.db_path)
|
||||
try:
|
||||
_wb_conn.execute(
|
||||
"UPDATE tasks SET model = ? WHERE id = ?"
|
||||
" AND (model IS NULL OR model = '')",
|
||||
(decision.chosen_model, task_id))
|
||||
_wb_conn.commit()
|
||||
finally:
|
||||
_wb_conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
task["model"] = decision.chosen_model or self.model
|
||||
elif not _gateway_filled:
|
||||
# Audit fix 3(I8):显式模型任务(含协调员子任务
|
||||
# 继承的父模型)此前不产生任何 worker_routing 行,
|
||||
# 影子按档看不到这部分。记一条 source=explicit 的
|
||||
# 行:评估函数只计 per-tier explicit_n,不计
|
||||
# routed_n、不占 seen(见 routing_shadow.py),
|
||||
# 父模型语义不变,档模型绝不覆盖它。
|
||||
self._audit(task["project_id"], "worker_routing",
|
||||
"%s tier=%s source=explicit applied=false"
|
||||
" suggested=- chosen=%s" % (
|
||||
task_id, classify_task(task),
|
||||
task.get("model")))
|
||||
# agent 指令通道(v2):local/docker 由 sandbox 的
|
||||
# --developer-instructions 派发,prompt 保持干净;通用
|
||||
# cloud 执行器没有这个通道,只能在这里把 v1 的带标记
|
||||
|
||||
Reference in new issue
Block a user