fix: second-audit gate findings (B1, I1-I8, M1-M10)

B1: gateway capability check now triggers on walking the sub2api
gateway, not on the gateway having filled a model; task-level
explicit models are audited (worker_capability_gateway_explicit)
but never blocked, preserving explicit priority; gateway-default
models still fail-fast under active.
I1: dry-run adds a gateway-model section, sourced only from
CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active
tier whose requires the gateway model fails or cannot verify
blocks, shadow only warns.
I2: shadow review gate now counts completed samples
(completed >= 20), same basis for sample_shortfall; the endpoint
attaches per-tier capability status (read-only policy load) and
flags capability-failing tiers in the recommendation.
I3: configured-but-broken routing policy logs a warning in
load_routing_policy (unconfigured stays silent); the worker sets
routing_policy_load_failed and audits
worker_routing_policy_load_failed per task.
I4: resume preamble wraps prior output/error as data with an
explicit do-not-execute constraint; position verified in run_one
(tree -> preamble -> project -> original) and the docs corrected
from the false 'closest to the original prompt' claim.
I5: decided model is written back with a CAS UPDATE
(model IS NULL OR model=''), so later attempts reuse it;
explicit models are never overwritten.
I6: agent_* columns added to worker._ensure_result_columns and
db.init_postgres_schema backfill; migrations backfills them for
tasks before insert (sender_task_id pattern); column-level tests
for agent_tool_approvals and task_attempts.
I7: coordinator requeue adds AND status=<snapshot> (rowcount 0
skips requeue/audit) and filters paused_by_user / user_cancel /
user_pause children, which are never revived or cleared.
I8: explicit-model tasks (including coordinator children that
inherited a parent model) audit a source=explicit worker_routing
row; evaluate_shadow counts it as per-tier explicit_n only,
never routed_n, never a dedup slot.
M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation,
shadow sort doc fixed, truncated recommendation notes truncation,
seed cleanup on gateway/quota/goal-cap early returns, tools-only
coordinator definitions rejected, preamble goal branch marked
effectively unreachable, DESIGN.md drift corrected (outside repo).
M4/M7/M9: documentation/new-field only (pass_rate_fully_measured,
whitespace-model shadow-drop note, bound-note race note); old
field semantics unchanged.
This commit is contained in:
host committed 2026-11-04 18:02:21 +00:00
1 parent f9a77577c5
commit 621c08d10f
13 files changed
+907 -44

No files matched your search

+81 -4
View File
@@ -26,6 +26,15 @@ What it reports, per tier (plan / goal / default):
is a BLOCKER (it would block dispatch at the worker
capability gate); a shadow/off tier with the same problem
is a warning only.
* gateway model check (Audit fix 3, I1): the same requires,
checked against the sub2api gateway model -- the model
production actually dispatches, which is NOT the tier
model. The gateway model comes only from env
(CLOUD_SUB2API_DEFAULT_MODEL); this module never calls
the relay to list models. An active tier whose requires
the gateway model fails or cannot verify is a BLOCKER;
a shadow tier is a warning only. "运营方须先把网关模型
写进能力目录" is now a checkable rule, not a comment.
What it does NOT do, honestly:
* It does NOT validate model names against a provider or relay
@@ -87,18 +96,25 @@ def _tier_dropped_reason(raw_entry):
return "unrecognised tier entry (expected string or dict)"
def dry_run_policy_document(data, env_mode_override=None):
def dry_run_policy_document(data, env_mode_override=None,
gateway_model=None):
"""Dry-run one merged policy document (pure, no IO).
``data`` is the merged document as parse_routing_policy would
receive it. ``env_mode_override`` is the raw CLOUD_ROUTING_MODE
value (or None): a valid value overrides the global mode only,
an invalid value is ignored with a warning.
an invalid value is ignored with a warning. ``gateway_model``
is the sub2api gateway model from env only (or None when
unset); it is never fetched from the relay here.
"""
warnings = []
tiers_report = {}
dropped = []
unknown = []
_gw = {"model": gateway_model,
"source": ("CLOUD_SUB2API_DEFAULT_MODEL"
if gateway_model else "env_unset"),
"tiers": {}}
if not isinstance(data, dict):
return {
"valid": False, "blocked": True, "version": None,
@@ -108,6 +124,7 @@ def dry_run_policy_document(data, env_mode_override=None):
"warnings": ["policy document is not an object; "
"the loader would ignore it entirely"],
"would_activate": [], "would_shadow": [], "would_off": [],
"gateway": _gw,
}
raw_tiers = data.get("tiers")
if isinstance(raw_tiers, dict):
@@ -128,6 +145,7 @@ def dry_run_policy_document(data, env_mode_override=None):
"tiers": tiers_report, "dropped_tiers": dropped,
"unknown_tiers": unknown, "warnings": warnings,
"would_activate": [], "would_shadow": [], "would_off": [],
"gateway": _gw,
}
global_mode = policy.mode
global_source = "policy"
@@ -217,6 +235,57 @@ def dry_run_policy_document(data, env_mode_override=None):
else:
warnings.append(note + "; %s mode does not block "
"dispatch" % status)
# Audit fix 3(I1):网关模型检查——生产实际派发的
# 是网关模型,不是档模型;档模型干跑通过不代表
# 网关模型能过。只查 active/shadow 档(worker 的
# 网关检查只在这两种模式下跑,off 档不查);网关
# 模型未在 env 里设定时记 unknown(live 回落会去
# 继电器查首个模型,干跑绝不联网,所以只能是
# unknown,不能假装 ok)。
_gw_blocked = False
for tier in ROUTING_TIERS:
_tstatus = tiers_report[tier]["status"]
if _tstatus not in ("active", "shadow"):
_gw["tiers"][tier] = {
"status": "not_checked", "missing": [],
"unknown": False, "tier_mode": _tstatus}
continue
_req = policy.tier_requirements.get(tier) or {}
if not _req:
_gw["tiers"][tier] = {
"status": "not_applicable", "missing": [],
"unknown": False, "tier_mode": _tstatus}
continue
if gateway_model is None:
_gw["tiers"][tier] = {
"status": "unknown", "missing": [],
"unknown": True, "tier_mode": _tstatus}
_note = ("tier %r requires %s but the gateway model"
" is not set in CLOUD_SUB2API_DEFAULT_MODEL"
" (and this dry-run never queries the relay)"
" -- cannot verify it against the capability"
" directory" % (tier, sorted(_req)))
else:
_ok, _missing, _unknown = check_model_capabilities(
policy.model_capabilities, gateway_model, _req)
_gw["tiers"][tier] = {
"status": "ok" if _ok else (
"unknown" if _unknown else "fail"),
"missing": _missing, "unknown": _unknown,
"tier_mode": _tstatus}
_note = ("tier %r gateway model %r %s capability:"
" %s" % (tier, gateway_model,
"cannot verify" if _unknown else "lacks",
", ".join(_missing)))
if _gw["tiers"][tier]["status"] in ("fail", "unknown"):
if _tstatus == "active":
_gw_blocked = True
warnings.append(_note + "; applying this policy "
"would block %r tasks at the gateway"
" gate" % tier)
else:
warnings.append(_note + "; shadow mode does not"
" block dispatch")
would_activate = [t for t in ROUTING_TIERS
if tiers_report[t]["status"] == "active"]
would_shadow = [t for t in ROUTING_TIERS
@@ -239,7 +308,7 @@ def dry_run_policy_document(data, env_mode_override=None):
"tasks on next worker dispatch if this policy were "
"applied; this dry-run does not activate anything" %
", ".join(would_activate))
blocked = bool(dropped or unknown or cap_blocked)
blocked = bool(dropped or unknown or cap_blocked or _gw_blocked)
return {
"valid": True, "blocked": blocked,
"version": policy.version, "global_mode": global_mode,
@@ -248,6 +317,7 @@ def dry_run_policy_document(data, env_mode_override=None):
"unknown_tiers": unknown, "warnings": warnings,
"would_activate": would_activate,
"would_shadow": would_shadow, "would_off": would_off,
"gateway": _gw,
}
@@ -290,8 +360,10 @@ def dry_run_from_env(env=None):
tiers = dict(merged["tiers"])
tiers.update(override["tiers"])
merged["tiers"] = tiers
_gw_model = (env.get("CLOUD_SUB2API_DEFAULT_MODEL") or "").strip()
report = dry_run_policy_document(
merged, env_mode_override=env.get("CLOUD_ROUTING_MODE") or None)
merged, env_mode_override=env.get("CLOUD_ROUTING_MODE") or None,
gateway_model=_gw_model or None)
# Source-failure warnings belong before the document warnings,
# and they do not change blocked (a failed source alone is not
# a dropped tier; the document report already covers validity).
@@ -311,6 +383,9 @@ def main(argv=None):
parser.add_argument("--mode", default=None,
help="Global mode override (same as "
"CLOUD_ROUTING_MODE)")
parser.add_argument("--gateway-model", default=None,
help="Gateway model (same as "
"CLOUD_SUB2API_DEFAULT_MODEL)")
args = parser.parse_args(argv)
env = {}
if args.policy_file:
@@ -319,6 +394,8 @@ def main(argv=None):
env["CLOUD_ROUTING_POLICY_JSON"] = args.policy_json
if args.mode:
env["CLOUD_ROUTING_MODE"] = args.mode
if args.gateway_model:
env["CLOUD_SUB2API_DEFAULT_MODEL"] = args.gateway_model
report = dry_run_from_env(env)
print(json.dumps(report, indent=2, sort_keys=True))
return 1 if (not report["valid"] or report["blocked"]) else 0