fix: audit batch 2 visibility (I-1 shadow dropped rows, I-2 partial policy failure, I-5 failure policy table)

- I-1: evaluate_shadow counts dropped rows by cause (unparseable /
  unknown_tier / explicit / duplicate) with the identity
  audit_rows_total == routed_task_n + dropped_total; the shadow
  endpoint passes created_at/id through and reports audit totals
  next to sample counts.
- I-2 (option b): parse logs a warning naming each tier dropped for
  an invalid mode/requires; load attaches partial_failures to the
  policy; the worker records a task-level
  worker_routing_policy_load_failed audit for partial failures
  (same action as full failure, detail says failure=partial);
  the shadow report carries the policy fingerprint, dropped tiers
  and partial failures (no task context on that path, so no
  task-level audit there).
- I-5: coordinator gate audit lookup failure now raises
  _CoordGateLookupFailed and the merge converts it to escalate
  (no requeue, no round consumed); persisted-model lookup failure
  and required-tools check failure each write a named audit row
  before falling through to dispatch/allow; audit-loop evaluate
  failure stays escalate (regression test).
- Test fixtures: coordinator suites build audit_log via AuthStore
  as production's first real audit does, since their stubbed
  _audit never creates the table.
This commit is contained in:
host committed 2026-11-04 18:02:22 +00:00
1 parent 44e0f4f1d6
commit a7f77c8a98
7 files changed
+648 -40

No files matched your search

+34 -7
View File
@@ -68,13 +68,11 @@ def parse_worker_routing_detail(detail):
Old format (no suggested/chosen) parses with both models None.
A ``-`` placeholder also means None, never a model named "-".
9.5-code fail-open 清单:解析失败的行返回 None 后在
evaluate_shadow 里直接跳过、不计数、不报错——样本无
声消失(例如模型名含空白会使整行匹配失败,见 M7)。
这是已知缺口:改返回类型只需同步评估函数的唯一生产
调用方(server 影子端点)与测试,本轨只在此处写明,
不改语义;运营方无法从报告里看到丢了多少行,只能靠
审计行总数与样本数对不上来发现。
Audit batch 2 (I-1): 解析语义本身不变(失败仍返回 None),
但 evaluate_shadow 不再静默跳过:不可解析的行计入
dropped_rows["unparseable"],报告里可见(例如模型名含
空白会使整行匹配失败,见 M7——行不再无声消失,只是
不计为样本)。
"""
if not isinstance(detail, str):
return None
@@ -144,10 +142,23 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
"unpriced_tokens": int}. ``explicit_n``: number of tasks
with a creation-time model in the same project (counted
separately, never as routed samples).
Audit batch 2 (I-1): every input row lands in exactly one
bucket — a routed sample, or ``dropped_rows``. Dropped
buckets are the real skip reasons in the loop below:
``unparseable`` (detail does not match the format),
``unknown_tier`` (tier is not a known tier), ``explicit``
(source=explicit rows, also counted per-tier as explicit_n,
never as samples), ``duplicate`` (a later row for an
already-counted task). Identity: ``audit_rows_total ==
routed_task_n + dropped_total`` — the caller can reconcile
the report against the audit rows it fetched instead of
guessing whether samples were lost.
"""
tiers = {tier: _empty_tier() for tier in TIERS}
seen = set()
routed_task_ids = set()
dropped_rows = {"unparseable": 0, "unknown_tier": 0,
"explicit": 0, "duplicate": 0}
rows = list(audit_rows or [])
if any(isinstance(r, dict) and r.get("created_at") is not None
for r in rows):
@@ -169,6 +180,7 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
parsed = parse_worker_routing_detail(
(row or {}).get("detail"))
if parsed is None:
dropped_rows["unparseable"] += 1
continue
task_id = parsed["task_id"]
tier = parsed["tier"]
@@ -178,14 +190,23 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
# occupy the dedup slot (Audit fix 3, M1): a task
# whose first row is an unknown tier must not lose
# its later known-tier row to dedup.
# Audit batch 2 (I-1): the row is counted as dropped,
# not silently skipped.
dropped_rows["unknown_tier"] += 1
continue
if parsed["source"] == "explicit":
# Audit fix 3(I8):显式行只计 per-tier
# explicit_n:不计 routed_n、不占 seen(同一任务
# 日后若出现路由行仍可计为样本),不破坏去重。
# Audit batch 2 (I-1): explicit rows are also
# counted in dropped_rows so the row identity
# reconciles; they are not "lost" — they are
# counted separately by design.
tiers[tier]["explicit_n"] += 1
dropped_rows["explicit"] += 1
continue
if task_id in seen:
dropped_rows["duplicate"] += 1
continue
seen.add(task_id)
bucket = tiers[tier]
@@ -250,6 +271,12 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
"tiers": tiers,
"explicit_n": int(explicit_n or 0),
"routed_task_n": len(routed_task_ids),
# Audit batch 2 (I-1): row reconciliation. Every input
# row is either a routed sample or in dropped_rows, so
# audit_rows_total == routed_task_n + dropped_total.
"audit_rows_total": len(rows),
"dropped_rows": dropped_rows,
"dropped_total": sum(dropped_rows.values()),
"counterfactual": "not_measured",
"counterfactual_note": (
"Shadow suggested models never ran, so this report cannot "