fix: audit findings (fallback dead code, silent fallback, registry re-parse, shadow sort, gateway capability)
- RoutingPolicy.fallback_models: move the cleaning loop into __init__ (it was unreachable after tier_mode's return, so the attribute never existed); still parsed but not applied. - Capability fallback: RoutingDecision gains capability_fallback; the worker audits worker_capability_fallback when it fires, so a tier-model rejection followed by a worker-default fallback is no longer invisible (source stays worker_default, no model_routing event, so shadow samples are unaffected). - Registry: resolve_registry_agent checks skills directly against the loaded definition instead of re-parsing a hand-built dict, so future fields cannot be silently dropped the way tool_approvals was in v9. - Shadow: evaluate_shadow sorts rows earliest-first by created_at (id tiebreak) when timestamps are present, so dedup no longer depends on caller row order; the server path (which strips created_at) is unchanged. - Gateway: when sub2api fills the default model, the worker now checks it against the tier's capability requirements with the same mode semantics as v8 -- shadow only audits, active fails fast with worker_capability_gate_rejected and never calls the runner. No worker-default fallback here: the gateway model is the task's gateway model, falling back to a worker default would run against the wrong gateway.
This commit is contained in:
1 parent
a51919f24b
commit
f9a77577c5
5 files changed
+392
-34
No files matched your search
+33
-8
@@ -111,18 +111,43 @@ def evaluate_shadow(audit_rows, tasks_by_id, costs_by_task,
|
||||
explicit_n=0, min_samples=MIN_SAMPLES):
|
||||
"""Evaluate shadow evidence from already-fetched rows.
|
||||
|
||||
``audit_rows``: dicts with at least ``detail`` (and optionally
|
||||
``created_at`` for earliest-first dedup; without it, first-seen
|
||||
wins). ``tasks_by_id``: task_id -> {"status": str}.
|
||||
``costs_by_task``: task_id -> {"total_tokens": int,
|
||||
"usd_micros": int|None, "unpriced_tokens": int}.
|
||||
``explicit_n``: number of tasks with a creation-time model in
|
||||
the same project (counted separately, never as routed samples).
|
||||
``audit_rows``: dicts with at least ``detail``. When rows
|
||||
carry ``created_at`` (and optionally ``id`` as a tiebreak),
|
||||
they are sorted earliest-first inside this function before
|
||||
dedup, so the first-seen sample for a task is its earliest
|
||||
routing audit regardless of the caller's row order. Rows
|
||||
without ``created_at`` keep their given order (stable sort).
|
||||
The server endpoint passes rows already ordered, and it
|
||||
strips ``created_at`` before calling -- that path is
|
||||
unaffected, but a future caller passing raw rows no longer
|
||||
silently changes dedup semantics. ``tasks_by_id``: task_id
|
||||
-> {"status": str}. ``costs_by_task``: task_id ->
|
||||
{"total_tokens": int, "usd_micros": int|None,
|
||||
"unpriced_tokens": int}. ``explicit_n``: number of tasks
|
||||
with a creation-time model in the same project (counted
|
||||
separately, never as routed samples).
|
||||
"""
|
||||
tiers = {tier: _empty_tier() for tier in TIERS}
|
||||
seen = set()
|
||||
routed_task_ids = set()
|
||||
for row in audit_rows or []:
|
||||
rows = list(audit_rows or [])
|
||||
if any(isinstance(r, dict) and r.get("created_at") is not None
|
||||
for r in rows):
|
||||
def _sort_key(row):
|
||||
created = row.get("created_at") if isinstance(row, dict) \
|
||||
else None
|
||||
row_id = row.get("id") if isinstance(row, dict) else None
|
||||
# Rows without a timestamp sort first only if they
|
||||
# were first already: use a sentinel that keeps
|
||||
# stable order among themselves, but never jumps
|
||||
# ahead of a timestamped row -- so put them last.
|
||||
return (created is None,
|
||||
created if isinstance(created, (int, float))
|
||||
else 0,
|
||||
row_id if isinstance(row_id, (int, float))
|
||||
else 0)
|
||||
rows = sorted(rows, key=_sort_key)
|
||||
for row in rows:
|
||||
parsed = parse_worker_routing_detail(
|
||||
(row or {}).get("detail"))
|
||||
if parsed is None:
|
||||
|
||||
Reference in new issue
Block a user