fix: audit batch 2 visibility (I-1 shadow dropped rows, I-2 partial policy failure, I-5 failure policy table)

- I-1: evaluate_shadow counts dropped rows by cause (unparseable /
  unknown_tier / explicit / duplicate) with the identity
  audit_rows_total == routed_task_n + dropped_total; the shadow
  endpoint passes created_at/id through and reports audit totals
  next to sample counts.
- I-2 (option b): parse logs a warning naming each tier dropped for
  an invalid mode/requires; load attaches partial_failures to the
  policy; the worker records a task-level
  worker_routing_policy_load_failed audit for partial failures
  (same action as full failure, detail says failure=partial);
  the shadow report carries the policy fingerprint, dropped tiers
  and partial failures (no task context on that path, so no
  task-level audit there).
- I-5: coordinator gate audit lookup failure now raises
  _CoordGateLookupFailed and the merge converts it to escalate
  (no requeue, no round consumed); persisted-model lookup failure
  and required-tools check failure each write a named audit row
  before falling through to dispatch/allow; audit-loop evaluate
  failure stays escalate (regression test).
- Test fixtures: coordinator suites build audit_log via AuthStore
  as production's first real audit does, since their stubbed
  _audit never creates the table.
This commit is contained in:
host committed 2026-11-04 18:02:22 +00:00
1 parent 44e0f4f1d6
commit a7f77c8a98
7 files changed
+648 -40

No files matched your search

+36 -14
View File
@@ -379,7 +379,16 @@ class RoutingPolicy:
def __init__(self, version=None, mode="shadow", tiers=None,
fallback_models=None, tier_modes=None,
model_capabilities=None, tier_requirements=None):
model_capabilities=None, tier_requirements=None,
dropped_tiers=None, partial_failures=None):
# Audit batch 2 (I-2): tiers the parser dropped for an
# invalid mode/requires, and source-level failures that
# still left a usable policy. Both default to [] so
# directly-constructed policies never raise AttributeError
# on these fields; parse/load fill them in. They are
# visibility only: they never change routing decisions.
self.dropped_tiers = list(dropped_tiers or [])
self.partial_failures = list(partial_failures or [])
self.version = version if isinstance(version, str) \
and version.strip() else None
self.mode = mode if mode in ROUTING_MODES else "off"
@@ -474,6 +483,12 @@ def parse_routing_policy(data):
# accidentally activate, and never kills the other tiers.
raw_tiers = data.get("tiers") if isinstance(data.get("tiers"),
dict) else {}
# Audit batch 2 (I-2): record every tier the fail-closed pops
# actually remove, with a warning naming the tier. A pop that
# finds the tier already absent (e.g. _clean_tiers dropped it
# for an empty model) is not a second drop and is not logged:
# the warning must name a tier this pop really removed.
dropped_tiers = []
tier_modes = {}
for tier, entry in raw_tiers.items():
if not isinstance(entry, dict) or "mode" not in entry:
@@ -481,8 +496,12 @@ def parse_routing_policy(data):
tier_mode = entry.get("mode")
if tier_mode in ROUTING_MODES:
tier_modes[tier] = tier_mode
else:
elif tier in tiers:
tiers.pop(tier, None)
dropped_tiers.append(tier)
_logger.warning(
"routing policy tier %r dropped: invalid mode %r",
tier, tier_mode)
if not tiers:
return None
# Per-tier requirements (v8): a tier entry dict may carry
@@ -496,7 +515,12 @@ def parse_routing_policy(data):
continue
cleaned = parse_tier_requires(entry.get("requires"))
if cleaned is None:
tiers.pop(tier, None)
if tier in tiers:
tiers.pop(tier, None)
dropped_tiers.append(tier)
_logger.warning(
"routing policy tier %r dropped: invalid requires",
tier)
elif cleaned:
tier_requirements[tier] = cleaned
if not tiers:
@@ -510,7 +534,8 @@ def parse_routing_policy(data):
tier_modes=tier_modes,
model_capabilities=data.get(
"model_capabilities"),
tier_requirements=tier_requirements)
tier_requirements=tier_requirements,
dropped_tiers=dropped_tiers)
def load_routing_policy(env=None):
@@ -573,17 +598,14 @@ def load_routing_policy(env=None):
" worker default", "; ".join(failures))
return None
if failures:
# 9.5-code fail-open 清单:部分失败是 fail-open——
# Audit batch 2 (I-2): 部分失败仍是 fail-open——
# 可用来源继续生效、失败来源被丢弃,派发不炸。
# 可见性只有这条警告日志:load_routing_policy 仍
# 返回一个 policy,worker 的
# routing_policy_load_failed 标志只在全失败时
# 置位,所以部分失败不会产生任务级审计;运营方
# 只能从 worker 日志看到它,不能从审计里查到。
# 这是已知缺口(非静默,但不如全失败可见),
# 本轨不改返回类型:生产调用点只有 2 处(server
# 影子端点经 _load_routing_policy、worker 派发),
# 真正的成本是同步改调用方与测试,而不是“牵动全部”。
# 但可见性不再只有这条警告日志:失败清单挂在返回
# 的 policy 上(partial_failures),worker 派发时
# 据此记任务级审计(与全失败同款 action),影子
# 报告里也附指纹与丢弃档;返回类型本身不变,
# 仍是 RoutingPolicy 或 None。
policy.partial_failures = list(failures)
_logger.warning(
"routing policy partially failed (%s); continuing with"
" the usable source", "; ".join(failures))