fix: second-audit gate findings (B1, I1-I8, M1-M10)
B1: gateway capability check now triggers on walking the sub2api gateway, not on the gateway having filled a model; task-level explicit models are audited (worker_capability_gateway_explicit) but never blocked, preserving explicit priority; gateway-default models still fail-fast under active. I1: dry-run adds a gateway-model section, sourced only from CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active tier whose requires the gateway model fails or cannot verify blocks, shadow only warns. I2: shadow review gate now counts completed samples (completed >= 20), same basis for sample_shortfall; the endpoint attaches per-tier capability status (read-only policy load) and flags capability-failing tiers in the recommendation. I3: configured-but-broken routing policy logs a warning in load_routing_policy (unconfigured stays silent); the worker sets routing_policy_load_failed and audits worker_routing_policy_load_failed per task. I4: resume preamble wraps prior output/error as data with an explicit do-not-execute constraint; position verified in run_one (tree -> preamble -> project -> original) and the docs corrected from the false 'closest to the original prompt' claim. I5: decided model is written back with a CAS UPDATE (model IS NULL OR model=''), so later attempts reuse it; explicit models are never overwritten. I6: agent_* columns added to worker._ensure_result_columns and db.init_postgres_schema backfill; migrations backfills them for tasks before insert (sender_task_id pattern); column-level tests for agent_tool_approvals and task_attempts. I7: coordinator requeue adds AND status=<snapshot> (rowcount 0 skips requeue/audit) and filters paused_by_user / user_cancel / user_pause children, which are never revived or cleared. I8: explicit-model tasks (including coordinator children that inherited a parent model) audit a source=explicit worker_routing row; evaluate_shadow counts it as per-tier explicit_n only, never routed_n, never a dedup slot. M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation, shadow sort doc fixed, truncated recommendation notes truncation, seed cleanup on gateway/quota/goal-cap early returns, tools-only coordinator definitions rejected, preamble goal branch marked effectively unreachable, DESIGN.md drift corrected (outside repo). M4/M7/M9: documentation/new-field only (pass_rate_fully_measured, whitespace-model shadow-drop note, bound-note race note); old field semantics unchanged.
This commit is contained in:
1 parent
f9a77577c5
commit
621c08d10f
13 files changed
+907
-44
No files matched your search
+36
-12
@@ -93,6 +93,9 @@ def _empty_tier():
|
||||
"routed_n": 0,
|
||||
"shadow_n": 0,
|
||||
"policy_n": 0,
|
||||
# Audit fix 3(I8):显式模型任务(source=explicit)
|
||||
# 按档单独计数,不进 routed_n、不占 seen。
|
||||
"explicit_n": 0,
|
||||
"completed": 0,
|
||||
"failed": 0,
|
||||
"other_status": 0,
|
||||
@@ -111,12 +114,20 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
|
||||
explicit_n=0, min_samples=MIN_SAMPLES):
|
||||
"""Evaluate shadow evidence from already-fetched rows.
|
||||
|
||||
``audit_rows``: dicts with at least ``detail``. When rows
|
||||
carry ``created_at`` (and optionally ``id`` as a tiebreak),
|
||||
``audit_rows``: dicts with at least ``detail``. Rows with
|
||||
``source=explicit`` are counted per tier as ``explicit_n``
|
||||
only (never ``routed_n``, never dedup slots). The review
|
||||
gate is ``completed >= min_samples`` (Audit fix 3, I2):
|
||||
failed/unknown/other statuses do not count toward readiness.
|
||||
When rows carry ``created_at`` (and optionally ``id`` as a
|
||||
tiebreak),
|
||||
they are sorted earliest-first inside this function before
|
||||
dedup, so the first-seen sample for a task is its earliest
|
||||
routing audit regardless of the caller's row order. Rows
|
||||
without ``created_at`` keep their given order (stable sort).
|
||||
without ``created_at`` sort last (Audit fix 3, M2: the old
|
||||
docstring said they kept their given order, but the sort
|
||||
key already put them last); among themselves they keep
|
||||
stable order.
|
||||
The server endpoint passes rows already ordered, and it
|
||||
strips ``created_at`` before calling -- that path is
|
||||
unaffected, but a future caller passing raw rows no longer
|
||||
@@ -153,14 +164,23 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
|
||||
if parsed is None:
|
||||
continue
|
||||
task_id = parsed["task_id"]
|
||||
if task_id in seen:
|
||||
continue
|
||||
seen.add(task_id)
|
||||
tier = parsed["tier"]
|
||||
if tier not in tiers:
|
||||
# Unknown tier: do not invent a bucket, do not count it
|
||||
# as a sample for any known tier.
|
||||
# as a sample for any known tier, and do not let it
|
||||
# occupy the dedup slot (Audit fix 3, M1): a task
|
||||
# whose first row is an unknown tier must not lose
|
||||
# its later known-tier row to dedup.
|
||||
continue
|
||||
if parsed["source"] == "explicit":
|
||||
# Audit fix 3(I8):显式行只计 per-tier
|
||||
# explicit_n:不计 routed_n、不占 seen(同一任务
|
||||
# 日后若出现路由行仍可计为样本),不破坏去重。
|
||||
tiers[tier]["explicit_n"] += 1
|
||||
continue
|
||||
if task_id in seen:
|
||||
continue
|
||||
seen.add(task_id)
|
||||
bucket = tiers[tier]
|
||||
routed_task_ids.add(task_id)
|
||||
bucket["routed_n"] += 1
|
||||
@@ -195,12 +215,16 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
|
||||
models.get(parsed["suggested_model"], 0) + 1
|
||||
else:
|
||||
bucket["suggested_unknown_n"] += 1
|
||||
# Audit fix 3(I2):就绪门改按已完成任务数计。
|
||||
# 旧版按 routed_n(已路由数)计,20 个全失败的档
|
||||
# 同样算就绪,与设计稿“已完成任务”的口径不一致;
|
||||
# sample_shortfall 同口径,失败/未知/其它状态不算。
|
||||
for tier, bucket in tiers.items():
|
||||
bucket["gate"] = (
|
||||
"ready_for_review" if bucket["routed_n"] >= min_samples
|
||||
"ready_for_review" if bucket["completed"] >= min_samples
|
||||
else "not_ready")
|
||||
bucket["sample_shortfall"] = max(
|
||||
0, min_samples - bucket["routed_n"])
|
||||
0, min_samples - bucket["completed"])
|
||||
ready = [t for t in ACTIVATION_ORDER
|
||||
if tiers[t]["gate"] == "ready_for_review"]
|
||||
if ready:
|
||||
@@ -211,9 +235,9 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
|
||||
"automatic from this report." % ", ".join(ready))
|
||||
else:
|
||||
recommendation = (
|
||||
"Not ready for review: no tier has %d routed samples. "
|
||||
"Keep shadow mode running; do not activate any tier."
|
||||
% min_samples)
|
||||
"Not ready for review: no tier has %d completed "
|
||||
"samples. Keep shadow mode running; do not activate "
|
||||
"any tier." % min_samples)
|
||||
return {
|
||||
"min_samples": min_samples,
|
||||
"tiers": tiers,
|
||||
|
||||
Reference in new issue
Block a user