fix: second-audit gate findings (B1, I1-I8, M1-M10)
B1: gateway capability check now triggers on walking the sub2api gateway, not on the gateway having filled a model; task-level explicit models are audited (worker_capability_gateway_explicit) but never blocked, preserving explicit priority; gateway-default models still fail-fast under active. I1: dry-run adds a gateway-model section, sourced only from CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active tier whose requires the gateway model fails or cannot verify blocks, shadow only warns. I2: shadow review gate now counts completed samples (completed >= 20), same basis for sample_shortfall; the endpoint attaches per-tier capability status (read-only policy load) and flags capability-failing tiers in the recommendation. I3: configured-but-broken routing policy logs a warning in load_routing_policy (unconfigured stays silent); the worker sets routing_policy_load_failed and audits worker_routing_policy_load_failed per task. I4: resume preamble wraps prior output/error as data with an explicit do-not-execute constraint; position verified in run_one (tree -> preamble -> project -> original) and the docs corrected from the false 'closest to the original prompt' claim. I5: decided model is written back with a CAS UPDATE (model IS NULL OR model=''), so later attempts reuse it; explicit models are never overwritten. I6: agent_* columns added to worker._ensure_result_columns and db.init_postgres_schema backfill; migrations backfills them for tasks before insert (sender_task_id pattern); column-level tests for agent_tool_approvals and task_attempts. I7: coordinator requeue adds AND status=<snapshot> (rowcount 0 skips requeue/audit) and filters paused_by_user / user_cancel / user_pause children, which are never revived or cleared. I8: explicit-model tasks (including coordinator children that inherited a parent model) audit a source=explicit worker_routing row; evaluate_shadow counts it as per-tier explicit_n only, never routed_n, never a dedup slot. M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation, shadow sort doc fixed, truncated recommendation notes truncation, seed cleanup on gateway/quota/goal-cap early returns, tools-only coordinator definitions rejected, preamble goal branch marked effectively unreachable, DESIGN.md drift corrected (outside repo). M4/M7/M9: documentation/new-field only (pass_rate_fully_measured, whitespace-model shadow-drop note, bound-note race note); old field semantics unchanged.
This commit is contained in:
1 parent
f9a77577c5
commit
621c08d10f
13 files changed
+907
-44
No files matched your search
+13
-2
@@ -230,11 +230,22 @@ def _run_stats(scored):
|
||||
if s["score"] is not None]
|
||||
mean_score = (sum(measured_scores) / len(measured_scores)) \
|
||||
if measured_scores else None
|
||||
# Audit fix 3(M4,新增字段、旧字段语义不动):
|
||||
# pass_rate 含未完全测定任务(all_pass 在含 unknown
|
||||
# 检查时仍可为真),只看它会被误导;严格口径是
|
||||
# pass_rate_fully_measured:分母只含 fully_measured
|
||||
# 任务。旧 pass_rate 保留原口径,不改,避免已有机
|
||||
# 制的对比口径悄悄变。
|
||||
fully_n = sum(1 for s in scores if s["fully_measured"])
|
||||
fully_pass = sum(1 for s in scores
|
||||
if s["fully_measured"] and s["all_pass"])
|
||||
return {
|
||||
"n": n, "pass_rate": (passed_tasks / n) if n else None,
|
||||
"passed_tasks": passed_tasks, "mean_score": mean_score,
|
||||
"fully_measured_n": sum(1 for s in scores
|
||||
if s["fully_measured"]),
|
||||
"fully_measured_n": fully_n,
|
||||
"fully_measured_passed_tasks": fully_pass,
|
||||
"pass_rate_fully_measured": (fully_pass / fully_n)
|
||||
if fully_n else None,
|
||||
"unknown_tasks_n": sum(1 for s in scores if s["unknown"]),
|
||||
"unmatched_n": len(scored["unmatched"]),
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user