feat: routing shadow evaluation v0b (read-only, never auto-activates)

New pure module cloud/routing_shadow.py: parse worker_routing
audit details (old format without suggested model parses with
suggested unknown, never guessed), evaluate per-tier routed
samples with one-task-one-sample dedup, explicit tasks counted
separately, status under the chosen model, token/USD with
unpriced tokens kept apart, and a min-20 gate per tier. The
report explicitly marks the counterfactual not_measured:
shadow suggested models never ran, so no suggested-vs-chosen
cost/quality comparison is possible. Worker audit detail now
includes suggested/chosen for future rows. GET /routing-shadow
serves the report (project read permission, truncated flag
over 10000 audit rows, cost table ensured so a fresh DB does
not 500).

Tests: test_routing_shadow.py (5 tests).
This commit is contained in:
host committed 2026-11-04 18:01:44 +00:00
1 parent fd37505c48
commit bac2a39377
4 files changed
+481 -2

No files matched your search

+9 -2
View File
@@ -5920,10 +5920,17 @@ class Worker:
# 不需要再用一条事件重复说明。
if decision.source in ("policy", "shadow"):
record_routing({"id": task_id, "events": events}, decision)
# 影子评估(v0b)需要建议模型:旧格式只有
# tier/source/applied,评估时建议模型只能记
# unknown;新格式补 suggested/chosen,旧解析
# 器仍能读(见 cloud/routing_shadow.py)。
self._audit(task["project_id"], "worker_routing",
"%s tier=%s source=%s applied=%s" % (
"%s tier=%s source=%s applied=%s"
" suggested=%s chosen=%s" % (
task_id, decision.tier, decision.source,
decision.applied))
decision.applied,
decision.suggested_model or "-",
decision.chosen_model or "-"))
task["model"] = decision.chosen_model or self.model
# agent 指令通道(v2):local/docker 由 sandbox 的
# --developer-instructions 派发,prompt 保持干净;通用