- I-1: evaluate_shadow counts dropped rows by cause (unparseable /
unknown_tier / explicit / duplicate) with the identity
audit_rows_total == routed_task_n + dropped_total; the shadow
endpoint passes created_at/id through and reports audit totals
next to sample counts.
- I-2 (option b): parse logs a warning naming each tier dropped for
an invalid mode/requires; load attaches partial_failures to the
policy; the worker records a task-level
worker_routing_policy_load_failed audit for partial failures
(same action as full failure, detail says failure=partial);
the shadow report carries the policy fingerprint, dropped tiers
and partial failures (no task context on that path, so no
task-level audit there).
- I-5: coordinator gate audit lookup failure now raises
_CoordGateLookupFailed and the merge converts it to escalate
(no requeue, no round consumed); persisted-model lookup failure
and required-tools check failure each write a named audit row
before falling through to dispatch/allow; audit-loop evaluate
failure stays escalate (regression test).
- Test fixtures: coordinator suites build audit_log via AuthStore
as production's first real audit does, since their stubbed
_audit never creates the table.
- M1: test_goal_branch_via_convert_queued_then_claim actually calls
server._goal_convert_queued on a queued auto-resumed task before
claiming, instead of only inserting the post-conversion state.
- M5: test_worker_routing_audit_rows_carry_no_fingerprint drives a
real run_one with an active policy and asserts worker_routing
audit rows exist and carry no fingerprint (negative control
verified: injecting fingerprint into the detail fails it).
- M2: 'all callers' wording tightened to the real counts
(load_routing_policy: 2 production call sites; evaluate_shadow: 1).
- M4: content_fingerprint docstring now states the fingerprint covers
the full normalized payload including fallback_models, which the
router deliberately does not apply (fail-safe direction).
No behaviour change; M3 DESIGN.md edits are outside the repo.
Parse: agent_tools/tool_approvals column parsing unified into agent_def shared helpers (parse None = undeclared; server keeps None, worker maps to []/{} at _task_from_row, dispatch behaviour unchanged). Goal: _apply_resume_preamble goal branch proven reachable via _goal_convert_queued on a queued auto-resumed task, kept with a test and corrected docs (not dead code). Fail-open: audit write failure and preamble lookup failure now log warnings (best-effort unchanged); partial policy failure and shadow parse drops documented as known gaps. M7: shadow endpoint explicit_capability_failures_truncated test. Fingerprint: RoutingPolicy.content_fingerprint (canonical JSON sha256, version excluded) surfaced only in dry-run output, never in audit rows. M1 wording: legacy gap stated as fixed stock persisting per-task until completion, not converging.
B1: gateway capability check now triggers on walking the sub2api
gateway, not on the gateway having filled a model; task-level
explicit models are audited (worker_capability_gateway_explicit)
but never blocked, preserving explicit priority; gateway-default
models still fail-fast under active.
I1: dry-run adds a gateway-model section, sourced only from
CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active
tier whose requires the gateway model fails or cannot verify
blocks, shadow only warns.
I2: shadow review gate now counts completed samples
(completed >= 20), same basis for sample_shortfall; the endpoint
attaches per-tier capability status (read-only policy load) and
flags capability-failing tiers in the recommendation.
I3: configured-but-broken routing policy logs a warning in
load_routing_policy (unconfigured stays silent); the worker sets
routing_policy_load_failed and audits
worker_routing_policy_load_failed per task.
I4: resume preamble wraps prior output/error as data with an
explicit do-not-execute constraint; position verified in run_one
(tree -> preamble -> project -> original) and the docs corrected
from the false 'closest to the original prompt' claim.
I5: decided model is written back with a CAS UPDATE
(model IS NULL OR model=''), so later attempts reuse it;
explicit models are never overwritten.
I6: agent_* columns added to worker._ensure_result_columns and
db.init_postgres_schema backfill; migrations backfills them for
tasks before insert (sender_task_id pattern); column-level tests
for agent_tool_approvals and task_attempts.
I7: coordinator requeue adds AND status=<snapshot> (rowcount 0
skips requeue/audit) and filters paused_by_user / user_cancel /
user_pause children, which are never revived or cleared.
I8: explicit-model tasks (including coordinator children that
inherited a parent model) audit a source=explicit worker_routing
row; evaluate_shadow counts it as per-tier explicit_n only,
never routed_n, never a dedup slot.
M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation,
shadow sort doc fixed, truncated recommendation notes truncation,
seed cleanup on gateway/quota/goal-cap early returns, tools-only
coordinator definitions rejected, preamble goal branch marked
effectively unreachable, DESIGN.md drift corrected (outside repo).
M4/M7/M9: documentation/new-field only (pass_rate_fully_measured,
whitespace-model shadow-drop note, bound-note race note); old
field semantics unchanged.
- RoutingPolicy.fallback_models: move the cleaning loop into
__init__ (it was unreachable after tier_mode's return, so the
attribute never existed); still parsed but not applied.
- Capability fallback: RoutingDecision gains capability_fallback;
the worker audits worker_capability_fallback when it fires, so
a tier-model rejection followed by a worker-default fallback
is no longer invisible (source stays worker_default, no
model_routing event, so shadow samples are unaffected).
- Registry: resolve_registry_agent checks skills directly
against the loaded definition instead of re-parsing a
hand-built dict, so future fields cannot be silently dropped
the way tool_approvals was in v9.
- Shadow: evaluate_shadow sorts rows earliest-first by
created_at (id tiebreak) when timestamps are present, so
dedup no longer depends on caller row order; the server
path (which strips created_at) is unchanged.
- Gateway: when sub2api fills the default model, the worker
now checks it against the tier's capability requirements
with the same mode semantics as v8 -- shadow only audits,
active fails fast with worker_capability_gate_rejected and
never calls the runner. No worker-default fallback here:
the gateway model is the task's gateway model, falling back
to a worker default would run against the wrong gateway.
New pure module cloud/routing_shadow.py: parse worker_routing
audit details (old format without suggested model parses with
suggested unknown, never guessed), evaluate per-tier routed
samples with one-task-one-sample dedup, explicit tasks counted
separately, status under the chosen model, token/USD with
unpriced tokens kept apart, and a min-20 gate per tier. The
report explicitly marks the counterfactual not_measured:
shadow suggested models never ran, so no suggested-vs-chosen
cost/quality comparison is possible. Worker audit detail now
includes suggested/chosen for future rows. GET /routing-shadow
serves the report (project read permission, truncated flag
over 10000 audit rows, cost table ensured so a fresh DB does
not 500).
Tests: test_routing_shadow.py (5 tests).
host