118 Commits
Author SHA1 Message Date
host aba06d9a24 gapfill2: worker/sub90 coverage tests, G7 rollback branch, G5 caller-contract docs
H1 worker.py 92%->94% (277->210 miss): 35 real-assertion tests
covering goal-signal, tenant/env, followup, poll/wait,
process/watchdog and injected defensive branches; prod code 0 changes.
H2 sub-90 sweep: approval_timeout 100%, merge_gate_hard 100%,
worker_sweep 98% (only __main__ line 174 left); 14 tests.
H3 G7 rollback except:pass (server.py:6615-6616) covered by an
injected rollback-raising connection; original error still propagates,
status stays completed, retry succeeds; no semantics change.
H4 caller contract documented only (docstrings in _audit_scorecard
endpoint, check_baseline_stale, build_scorecard, baseline store):
current_main_head caller-supplied, any string as-is, no format checks,
missing/empty treated as not passed with no stale determination at the
endpoint, always 200, overall never rewritten; no new validation or
rejection, no response-structure change. 3 doc tests pin the wording.
H5 design notes (no decisions, no code changes) live in
verification/code-to10-gapfill2-design-notes.md, referenced by the
batch summary.
No bug found by the new tests; no existing assertion touched.
2026-11-04 16:20:10 +00:00
host 5d0651e70a gapfill G7: make _merge_task status commit and success audit atomic (same connection, audit_on_conn, one commit; audit failure rolls back to completed instead of merged-plus-500; ledger-head IntegrityError retries whole segment 5x mirroring AuthStore.audit); success semantics unchanged 2026-11-04 09:16:25 +00:00
host 7524a779f2 gapfill G6: nail CloudExecutor and argv-only visibility gaps with tests only (pending_mirror_to_host_loop never claims applied; server passes argv-only, host verify gate blocks it, version gap plus leftover signal file are the observable trace); no production change, no invented channel 2026-11-04 09:12:01 +00:00
host 29af9c7bbf gapfill G2 (goal): cover all 15 missing goal_live_update edge branches (write validation, bad applied_version fallback, non-dict signals, consume failure paths, apply CAS null-version branch and CAS miss); directed coverage 81% -> 100%, production code untouched 2026-11-04 09:09:58 +00:00
host 145aa2eb5f gapfill G5: wire optional ?current_main_head drift check into GET /audit-scorecard (caller-supplied only, no git/subprocess; absent param means no staleness verdict, verdict never rewrites overall); endpoint tests for passed/absent/empty 2026-11-04 09:08:47 +00:00
host b4270dc117 gapfill G4: cover caps.py:54-55 env-get-exception fallback and quota.py:73-74 illegal-env fallback (A9 subset gaps); line numbers verified unchanged after rebase 2026-11-04 09:07:06 +00:00
host aa6179b5e1 gapfill G3: unify all 5 private row_get onto peer_messages.row_get (approvals, automations, marker_tools, todos, wait_ops; resident import retargeted); semantics identical on None/dict/sqlite3.Row, identity tests added 2026-11-04 09:06:27 +00:00
host 5ca642c86e gapfill G1: remove 5 ruff F401 unused imports (3 unused os in production files, 2 in tests); no assert lines changed, ruff all cloud now 0 2026-11-04 09:03:39 +00:00
host 055da3c2dd code-to10 reconcile: fix pre-existing worker F821 in playwright skipped-modified log branch
Target-line bug, not introduced by the replay: Worker._tenant_dirs
only receives project_id, yet the script_skipped_modified log line
used task.get("project_id"), raising NameError whenever seeding
reported a tenant-modified script (the refreshed sibling branch was
already fixed to project_id with its own regression test; this test
only covered the refreshed branch, so the full suite stayed green).
Fix mirrors the sibling (project_id) plus a regression test for the
skipped branch. Ruff F821 in worker.py is thereby cleared.
2026-11-03 02:49:27 +00:00
host 34b9a34896 code-to10 reconcile: re-pin handler accounting and longfunc inventory for the unified line
Target keeps the 6-mixin split, so the source line's 239/223/16 and
its mixin-homed 17-method list do not apply here. Recomputed by AST
on this line: handler accounting 244 total / 150 in server.py /
94 moved, no duplicates (243/149 + _update_goal_task); longfunc
still 17 methods with the same names, 16 homed in server.py and
_full_auto_execute in handler_fullauto.py, plus module-level
migrate_tasks_table in server.py (114 lines). No method split or
dropped; both tests' docstrings state the re-pin explicitly.
2026-11-03 02:49:27 +00:00
host 22ac542760 code-to10 v5: fix November date failure in arch-converge quota gate fixture (fixture day follows current UTC month; assertions and gate semantics unchanged; frozen-semantics header updated explicitly)
(cherry picked from commit 073950043ed65039e19be8a4b6d725a44a662879)

Reconcile-note: this datefix was missing from the solidify line (aa30d40's test_arch_converge.py had no _current_month_day and still failed in November, while 10023fb/6841f28 contained it); added here as the separately-labeled fix for that replay omission, per the reconcile brief.
2026-11-03 02:49:27 +00:00
host 9e56c1d266 code-to10 goal live update: POST /tasks/{id}/goal updates a running goal loop at the next round boundary
Server writes goal + monotonic goal_version + task_goal_history in one
transaction (CAS on version, approve_merge only, new-semantic goal tasks
only, terminal/merged 409, same-goal noop). Worker mirror thread (30s)
and breakpoint-resume claim hook write the update as a signal file next
to host-state.json; host run_goal_loop applies it at round boundaries
(top of round and before the declaration check, so a declaration made
for the old goal in a turn during which the goal changed is voided),
with an explicit old->new notice, same thread/worktree, rounds/usage/
budget/failed counters untouched. handler method accounting is a
contract update: 239 total / 223 mixins / 16 in server.py.

(cherry picked from commit 6841f287219d2b36cfbb8803642344b84dae4377)

Reconcile-port: handler_lifecycle.py contents redirected — _update_goal_task method + _goal_update_reject/_goal_signal_status pure functions + goal_live_update import into cloud/server.py (target keeps 6-mixin split); _task_to_dict goal fields into cloud/handler_support.py; migrate columns into cloud/server.py migrate_tasks_table; test patch target handler_lifecycle→server; test_handler_split_conformance.py kept at target version, to be re-pinned for the unified line in a final commit.
2026-11-03 02:49:27 +00:00
host 63abb3953b code-to10 v7 mech merge fix: silence mypy misc redefinition on Protocol fallback in gate_verification.py (item6-introduced), no runtime semantics changed
(cherry picked from commit 9f8b95fa03643db7bef028b9877cc1c1bc78ef4e)
2026-11-03 02:49:27 +00:00
host de925bff3b code-to10 v7 mech merge fix: SqliteScorecardGainLedger uses cloud.db.connect instead of sqlite3.connect (arch conformance layer rule), no semantics changed
(cherry picked from commit bde36a84880574c2297aaee9ed08312cd0e2c0fc)
2026-11-03 02:49:27 +00:00
host 368363e096 code-to10 v7 mech merge fix: wrap two merged docstring lines over 120 chars (E501) in audit_scorecard.py, no code semantics changed
(cherry picked from commit a4fdc78930f597e7793a054c1775842fe5496143)
2026-11-03 02:49:27 +00:00
host f406398ce2 v7 mech-7: scorecard user_side_items + three-table rule (code unchanged, production forced not_pass on user-side pending/unknown/invalid)
(cherry picked from commit 59b4c49a896972501aeeb8472a9fc2918ec226fa)
2026-11-03 02:49:27 +00:00
host f8a6a9d258 v7 mech-2: split scorecard into code/production/product tables
(cherry picked from commit 67b7f5e42d001b655aef3e04e1d3c1d98166b98f)

Reconcile-port: handler_admin.py hunks redirected into cloud/server.py _audit_scorecard (target keeps 6-mixin split; scorecard module itself applied cleanly).
2026-11-03 02:49:27 +00:00
host 19fe4fc7e1 code-to10 v7 mech-4: scorecard evidence ledger head + row ids, injectable signature verifier (fail-closed)
(cherry picked from commit bcd385c6979db77abea458af66e21ef891453064)
2026-11-03 02:49:27 +00:00
host 8fedd7fba0 v7 mech-5: scorecard baseline_head machine field, stale check, frozen store (append-only)
(cherry picked from commit e3e1f517e916eaac597b82d04bac5307a3f8b126)
2026-11-03 02:49:27 +00:00
host be81d658b6 code-to10 v7 mech1: scorecard rubric versioning, append-only rubric + snapshot tables
(cherry picked from commit 368475243367685bf9db9335ec929958de031ca6)
2026-11-03 02:49:27 +00:00
host 3978dbbbf2 v7 mech-6: production-evidence fail-closed live wiring seam (no real reader, no endpoint wiring yet)
- LiveEvidenceReader protocol: get_checks(repo, ref) /
  get_branch_protection(repo, branch), results carry status
  plus raw evidence; production code has no HTTP call, no
  token read, no real GitHub API call (AST-asserted).
- verify_live_evidence: no reader / no repo-ref / reader
  failure / mismatch all verified=False; comparison shared
  with verify_gate_evidence via _compare_claimed_vs_live.
- live_gate_decision: verified=False -> allowed=False,
  http_status=409, status=blocked; verified=True -> 200.
- Endpoints _merge_task/_release_task NOT wired yet (no
  production reader exists; wiring None would block all
  merges/releases); real live verification still pending
  B6 real App install and B1 user-side items.

(cherry picked from commit 7e8bfdee1f22c28329602314f3e166cde34eb6cf)
2026-11-03 02:49:27 +00:00
host ae012a8e7c v7 mech-3: scorecard diff scope, gain ledger, stagnation stop table
- cloud/scorecard_diff.py: audit_scope (changed files + claimed_fixed,
  unchanged/baseline-out files out of package), ScorecardGainLedger
  (memory) + SqliteScorecardGainLedger (SQLite only, PG_DDL text-only),
  three tables code/production/product kept separately, average
  reference-only; check_stagnation: two consecutive code gains <0.5
  (test_default_unapproved) escalate, no auto pushback; rubric_version
  change starts a new sequence.
- cloud/tests/test_scorecard_mech_diff.py: 24 tests, all passing.
- No change to audit_loop_driver.py semantics, #238 auto_apply,
  #247 correction-pass semantics, or v1.1 contract.

(cherry picked from commit 57d44becaf559a074c995249e95bd36924dae13f)
2026-11-03 02:49:27 +00:00
host b7e2f29868 v7 mech 8: MCDT acceptance/key-fact nodes as scorecard input model only
(cherry picked from commit 93e491b6a1b83fcdbbacf3493cdd2fd76418d0b6)
2026-11-03 02:49:27 +00:00
host 83135c283d code-to10 v6: workers _default_audit docstring follows 2026-11-02 finalization (migration side, no revert)
(cherry picked from commit b7ac9d7d6af842a182964b0aa0524708e7cdc923)

Reconcile-note: resolution kept target's finalized wording verbatim, so this commit is empty by content (docstring-only commit; both sides agreed: migration side, no revert).
2026-11-03 02:49:26 +00:00
host 488aedbcad code-to10 v6 A14 batch2: test E402/F401 cleanup plus touched-file residual E,F cleanup
(cherry picked from commit 1db02545f74c96b3b92be4fc32954659cec1f814)

Reconcile-conflict: A14b E402 imports referencing cloud.handler_core (does not exist here; core methods live in server.py) — kept HEAD imports with incoming's noqa: E402 markers, dropped the unused core_mod import (verified unused elsewhere in both files).
2026-11-03 02:49:26 +00:00
host bd55599859 code-to10 v6 A14 batch1: workers F601 duplicates and check_transplant E741 renames
(cherry picked from commit 4411186681584bf55e065c654fe909a3320d4a22)

Reconcile-conflict: test_skill_transplant_prep artifact count — kept HEAD's finalized 0 refs (2026-11-02 user sign-off deleting §4's 6 references, already in target) with incoming's l→line rename; incoming's 6-ref assertion is pre-finalization and must not overwrite.
2026-11-03 02:49:26 +00:00
host e00c255011 code-to10 v6 A13: 17-method long-function inventory incl _full_auto_execute plus support domain boundary comments
(cherry picked from commit 908df30d1a3cf328ea05344f0660af9b77bf24a1)

Reconcile-port: handler_support.py kept at HEAD (189 lines; A13 domain comments describe source's 1616-line handler_support layout, which does not exist here — functions live in server.py). test_longfunc_inventory.py kept as-is for now; it pins source architecture and will be re-pinned for the unified line in a final commit (recorded).
2026-11-03 02:49:26 +00:00
host e32391fd85 code-to10 v6 A12: scorecard example renamed to 2026-10-31 snapshot with timeliness fields
(cherry picked from commit 633cdbc94e7a7b0c5d3b0b135189faf3de6673f4)

Reconcile-port: handler_admin.py hunks redirected into cloud/server.py _audit_scorecard (target keeps 6-mixin split; scorecard module itself applied cleanly).
2026-11-03 02:49:26 +00:00
host e131e8658d code-to10 v6 A11: whole-tree audit_log single-writer scan assertion
(cherry picked from commit 9308503f90090526bb91b5aa2ae0971f84781a1d)
2026-11-03 02:49:26 +00:00
host e7acc143db code-to10 v6 A10: release gate single evaluator via wiring adapter
(cherry picked from commit 77ed1c3094798f75bc41323f9aa60f744f5ba507)
2026-11-03 02:49:26 +00:00
host 31edc5cda9 code-to10 v6 A8 driver round 2: remaining subset gaps (first gate acceptance, fallbacks, signals, ledger branches, guards, confirm branches)
(cherry picked from commit e3cac9f3e62f01c49fca9b7cbf80431a4d55b8ad)
2026-11-03 02:49:26 +00:00
host 5428b1defe code-to10 v6 A8 worker: cover process_auto_memories logger-failure inner except (2088-2089)
(cherry picked from commit 930f722641bb91f1f1e4d67f5937064eab8bee1e)
2026-11-03 02:49:26 +00:00
host 0d51c1d334 code-to10 v6 A8 worker: worker gap tests (auto-memories, tenant skill failure, prepare-env failures, pr-job failures, run-loop drain)
(cherry picked from commit 903bd560bf2b5796daf4b86dacc29b0732513607)
2026-11-03 02:49:26 +00:00
host 4919e107a6 code-to10 v6 A8 driver: audit_loop_driver gap tests (rebuilds_used, normalize, result map, test-change validate, root-cause local)
(cherry picked from commit 5e874f1a924289230f5a3143696358f72f9216e1)
2026-11-03 02:49:26 +00:00
host f4a5ab9600 code-to10 v6 A9: caps/quota gap tests (caps 70-71, quota 79-80, 128-131, 140-143)
(cherry picked from commit 0c733b0805a45057de9b618943f9a5b2af5d7bc4)
2026-11-03 02:49:26 +00:00
host be2962525b code-to10 v6 A7: unify row_get on peer_messages
- peer_messages.row_get gains default=None (was None-only); the
  three old variants agree on all real row types (None / dict /
  sqlite3.Row): auto_memory's old default was already None, and
  lifecycle's old None special-case and dict .get give identical
  results on those types, so no call-site behavior changes and no
  call site passes a non-None default.
- auto_memory and lifecycle delete their private _row_get copies
  and import peer_messages.row_get as _row_get (precedent: server,
  worker, handler_project, handler_support already do this);
  peer_messages imports no cloud modules, so no import cycle.
  lifecycle's old broad-Exception catch differs only for pathological
  __getitem__ inputs, which are not DB rows and are out of contract;
  this is documented on row_get, not left as a second story.
- Scope note: other modules' own row_get copies (approvals,
  automations, marker_tools, todos, wait_ops) are out of this item's
  stated scope and are untouched.
- Also drops auto_memory's unused 'import os' (pre-existing F401),
  so touched files are ruff E,F clean.
- Tests: cloud/tests/test_row_get_unified.py locks the identity
  (both aliases ARE row_get), the real-row-type matrix, the
  None-default fallback, and snapshot_to_dict via the unified path.

(cherry picked from commit 1644a0b837bfefb13cdc3c93d87fc1b246026607)
2026-11-03 02:49:26 +00:00
host c33bfd240e code-to10 v6 A6: fix mypy cost.py:551 tuple annotation
- model_breakdown inferred params as a 1-tuple from the first
  branch, so params + (month,) was tuple[Any] vs tuple[Any, str].
  Annotate params as bare tuple (tuple[Any, ...]) before branching.
- Annotation only: the () initializer is never used (if/elif/else
  always overwrites it), runtime semantics unchanged.
- mypy cloud/: 0 errors in 94 files (was 1 error); ruff clean;
  cost tests 39 pass.

(cherry picked from commit a9fd0692361e8169554413b725c81135be2c5525)
2026-11-03 02:49:26 +00:00
host ee8d1e1e28 code-to10 v6 A5: admin CLI wiring for stale-worker reap
- New production caller cloud/worker_sweep.py (python -m
  cloud.worker_sweep): AuthStore.authenticate + CODEX_GLOBAL_ADMINS
  global-admin check (same fail-closed semantics as
  handler_admin._is_global_admin) before any reap call; denied
  authenticated non-admins get an admin_worker_sweep_denied audit
  row; authorized runs reuse reap_stale_workers (classify, re-verify,
  pre-kill audit fail-closed, per-pid SIGKILL only, post-kill
  verification) plus an admin_worker_sweep summary audit; --dry-run
  lists classifications only.
- No new HTTP route, no container enumeration, no name/prefix
  matching, no workers-row deletion; workers.py docstring only notes
  the new caller.
- Tests: cloud/tests/test_worker_sweep_admin.py, 6 cases (bad key,
  non-admin, unconfigured admin list fail-closed, authorized full
  chain, dry-run, CLI main authorized/refused).

(cherry picked from commit c7d217549c5ccedf61dcc924b8998fffeddc76c6)
2026-11-03 02:49:26 +00:00
host 19b9767478 code-to10 v6 A3: enforce signer separation on audit-verdict submission
process_audit_verdict now calls validate_signer_separation
before any state save or ledger write (previously the function
had no production caller on this path: it only ran inside
collect_failed_checks via decide_pushback, which itself has no
production caller). Signer is bound to the authenticated
caller -- a body signer differing from the caller is rejected
-- and executor/coordinator_approver/qa_tester must be
declared and distinct from the caller; missing field,
self-audit, or forged signer raises ValueError, mapped by
_loop_run to HTTP 400 with no side effects. No new route.
Honest limits are in the helper docstring: declared-identity
enforcement only (task rows store no authoritative executor /
coordinator / QA identity; task executor is a backend, not a
person), and audit-fix reaudit verdicts are out of scope
(the fix poster is not necessarily the re-auditor, so binding
signer to the fix poster would be a mis-binding).

(cherry picked from commit 56dce5165f4d08f38fe1730b9eea3b78f90cb97d)

Reconcile-port: handler_lifecycle.py docstring hunk redirected into cloud/server.py audit-verdict handler (target keeps 6-mixin split; enforcement itself is in audit_loop_wiring.py and applied cleanly).
2026-11-03 02:49:26 +00:00
host 5db89d1b06 code-to10 v6 A4: fix AuthStore.audit NameError (import sqlite3) + retry regression test
Pre-fix any first-attempt audit failure raised NameError at
auth.py:424 (sqlite3 never imported), aborting the 5-attempt
retry loop on attempt 1 and masking the original error. The
424-line coverage mystery is also explained: the only full-suite
hit is test_child_model.test_coordinator_inherits_child_defaults
via worker best-effort _audit swallowing an OperationalError
(database is locked); 422-423/426-427 stay uncovered because
rollback succeeds and NameError aborts before either raise.
Bubble audit (see report): no merge/release gate swallows the
NameError and proceeds, so this stays a downgrade, not a
production blocker.

(cherry picked from commit 039c6adbac3b120b5b9a1dcbafeff6a91407ce70)
2026-11-03 02:49:26 +00:00
host 328ffdb7c6 code-to10 v6 A2: single controlled PG guard entry (code side only)
Postgres audit_ledger append-only guard gains exactly one
execution entry: cloud/ledger_pg_guard.apply_pg_guard, runnable
as 'python3 -m cloud.ledger_pg_guard' (explicit ops command,
not a migration script). It executes PG_GUARD_SQL verbatim from
the constant -- no second copy of the SQL anywhere -- refuses
non-PgConnection targets, does not commit itself (caller/CLI
commits), and refuses with exit 2 on a non-postgres backend so
a no-PG environment can never claim hardening. ensure_schema
still skips PG triggers; POSTGRES_SCHEMA still builds only the
table. Tests lock all of this with a recording fake connection
(no PG execution is faked), a source scan asserting only
ledger.py and ledger_pg_guard.py reference the constant, and
CLI dry-run/refusal cases. Real-PG syntax and UPDATE/DELETE
blocking verification is explicitly left to B5, not claimed here.

(cherry picked from commit b1f6c9158d628186965a9e57576ff8c9500e3f73)
2026-11-03 02:49:26 +00:00
host 32620c4517 code-to10 v6 A1: delete server merge-gate modes, hard only
Server POST /tasks/{id}/merge no longer has record/enforce/off:
hard_mode_from_env and HARD_MODES deleted (not re-defaulted),
handler always evaluates the hard gate, env CLOUD_MERGE_GATE_MODE
and body merge_gate_mode are ignored server-side. The record
'no-evidence passes' and 'unprotected+confirm passes with
merge_gate_risk' server paths are gone with the dead risk audit;
no production caller justified keeping them (only production
caller of the non-hard evaluator was the merge handler itself;
audit_loop_wiring already used the hard gate). cloud/merge_gate
pure functions keep their #209 semantics but are documented as
non-server: they are no longer wired to any endpoint.
Server-level record/enforce/off tests rewritten to the new
behavior (renamed, not deleted silently); pure-function tests
unchanged.

(cherry picked from commit 08e75edbd6d04701965c5db951fa054cfbb75bb0)

Reconcile-port: handler_lifecycle.py hunks redirected into cloud/server.py _merge_task (target keeps 6-mixin split; mixin file does not exist here); handler_support.py kept at HEAD (its A1 hunk is a docstring of _validate_hard_gate_body, ported to server.py copy); test mock paths cloud.handler_lifecycle.* rewritten to cloud.server.*.
2026-11-03 02:49:26 +00:00
host b0d51b139c B3a: 全自动问用户清单加第五种情形(验收失败推倒前确认),四件事/四类改写为五件 2026-11-03 02:49:26 +00:00
host 39ec2b4435 v5: rebase 融合修复(main 演进 + 附录源同步)
- websearch.py:补回 import socket(栈 46ca59f 的 lint 清理在栈
  上无害,replay 到 main 后 main #274 的超时/DNS 显式报错
  代码引用 socket,缺 import 致 2 项 NameError;re 在 main
  与重排后树中均 0 引用,不补)
- 3 个 websearch fake 补 scope="general" 参数(main 的
  handle_request 现必传 scope,旧 fake 致 TypeError 被吞成
  isError,3 项失败)
- playwright 工具数 14→18(main 新增 4 个,与 main _TOOLS
  逐名一致、无重复)
- playwright 种子测试改按 main 安全升级规则:源脚本 v1→v2
  且 manifest 证明未改动才刷新;未知改动只上报跳过不覆盖
- 附录源同步:01/02/07 三个 roles 源文件与 SKILL.md 附录章
  做同一改写,check_appendix 来源一致性恢复
2026-11-03 02:49:26 +00:00
host 34a19ab9e2 code-to10 v5 rebase follow-up: keep main #275 resident working on rebased tree
- restore _audit_row in cloud/automations.py as a thin wrapper over
  cloud.auth.audit_on_conn (legacy row + ledger entry pair,
  user_id system, best-effort self-commit preserved). The stack's
  gap-4 option A deleted the direct writer; main #275 resident.py
  imports _audit_row, so the bare rebase broke server import.
  No direct audit_log INSERT is reintroduced, so the #250
  no-bypass scan stays green.
- conformance accounting 238/144 -> 243/149: main #275 added 5
  resident methods that legitimately stay in server.py; moved
  count stays 94, no duplicates.
- directional: resident x3, audit chain, conformance, automations
  x3 suites, 51 tests OK
2026-11-03 02:49:26 +00:00
host 6bceed484c code-to10 v5: solidify 7 provisional resolutions per user confirmation (2026-11-02)
- whitelist expansion (tr + exit builtin) confirmed, comment only
- B4 viewport wording: temp banner removed, wording untouched,
  change-list table stays in draft, not merged into body
- B3-tail (a)/(b): no content change (bullet still absent), recorded
  as decided; v2 record showed it unselected, no side picked here
- section 4: 6 templates/artifacts/ refs deleted per signed verdict;
  tripwire test updated 6 -> 0 to pin the new inventory
- CHANGELOG pointer line kept, no marker present
- workers audit landing migration confirmed, comment only
- verify-script gate evidence confirmed, gate hard untouched
- 2 temp stances NOT touched: actual-only quota, evidence binding
  not enforced / no anti-restamp
- behavior code unchanged; directional skill tests 39/39 OK
2026-11-03 02:49:26 +00:00
host 12e72dab9b code-to10 v4 coverage thickening: 219 new assertion tests (batches A/B1/C/D1-D6)
Baseline merge head e5ff26a: 28144 stmts, 2807 missing, exact 90.0263%, margin +7.
After thickening (full run): 2350 missing, exact 91.6501%, margin +464.
Full suite 3516 items: 3509 pass, 1 failure (pre-existing November date
failure, untouched), 6 skipped. No production code changed.
2026-11-03 02:49:26 +00:00
host 773945d136 v2 evidence: bind release evidence with ts/nonce, pin actual-only quota, ledger onboarding checklist, ledger failure outbox, read-only gate verifier (no live wiring)
- cloud/evidence_binding.py: bind/validate/nonce single-use store;
  release endpoint consumes nonce on first validated submission,
  replay -> 409 evidence_replayed, stale/future -> 409
  evidence_stale, partial -> 400, absent -> old path + label.
  Binding blocks exact replay and stale evidence, NOT restamping
  (unsigned evidence cannot prove content freshness).
- cloud/quota.py docstring: actual-only is the v2 integration
  position (already converged in v1 base via fef09ae, NOT in
  main); original PR semantics (#86 all-rows vs #90 actual-only)
  still pending coordinator confirmation at merge time.
- cloud/ledger_onboarding.py: DDL checklist referencing
  PG_DDL/PG_MARKER_DDL constants, read-only verify_ledger_ready
  and ledger_setup_report; default stays OFF, PG DDL stays an
  operator action (never claimed as executed).
- cloud/ledger_outbox.py + worker hook: ledger write failures
  go to a JSONL outbox in addition to the log and can be
  replayed; task outcome still unaffected (rule 7 kept),
  audit fail-closed untouched -- asymmetry documented, not
  silently changed.
- cloud/gate_verification.py: read-only verifier over an
  injected reader (match/mismatch/skipped); live wiring in
  merge/release is NOT possible on the current contract (no
  repo/ref in body or task row, no guaranteed trusted token),
  proven in the module header; no fake verification claimed.
- cloud/tests/test_v2_evidence.py: 18 new tests, all passing.

(cherry picked from commit bd85e33cd5e846fa6ee5d7e2440182dbd9c758b0)
2026-11-03 02:49:26 +00:00
host 2dc35bd331 fix: transplant PG init backfill rollback scope + audit-loop state upsert registration (code-to10 v2, from 854f810)
- db: translate INSERT OR IGNORE ... SELECT to ON CONFLICT DO NOTHING;
  register task_audit_loop_state (task_id) in _OR_REPLACE_PK (direct port).
- server: _backfill_default_quotas uses SAVEPOINT containment, no longer
  commits the caller's transaction (direct port; callers are init_db only).
- worker: NOT ported — integration _coord_child_rows has no column-fallback
  retry loop, so the InFailedSqlTransaction cascade site does not exist here.
- tests: rewritten for the integration base (no cloud.audit_loop /
  _coord_audit_loop_merge here); real-PG cases skip without DSN/driver.

(cherry picked from commit 79e6843610fcd275af874e7b4afc57af7ffa579e)
2026-11-03 02:49:26 +00:00
host 741eb387ed test: retarget v2 coverage monkeypatch targets to definition modules after handler split (assertions unchanged) 2026-11-03 02:49:26 +00:00
host 6e0660e97b test: code-to10 v2 coverage additions (14 files, production code unchanged)
(cherry picked from commit 473338c35d59bfd50929542e7f08c0a4a22c5987)
2026-11-03 02:49:26 +00:00
host 935b530f84 split: handler mixins for oauth/identity/project/automation/fullauto (94 methods) on integration base 02835f6
- CloudControlPlaneHandler mixes in OAuthMixin, IdentityMemoryMixin,
  ProjectMixin, AutomationMixin, FullAutoMixin; shared helpers move to
  cloud/handler_support.py with server.py re-exports (old import paths
  keep working, incl. _validate_env_branch for tests).
- Method bodies moved verbatim; routers, base plumbing, task/lifecycle,
  webhooks stay in server.py. No >100-line method body split.
- test_handler_split_conformance.py (6 tests) pins resolution, layer
  bans and method accounting (238 total, 144 + 94, no duplicates).
- test_p1_security_server.py: monkeypatch targets follow the defining
  module (handler_project MAX_PROJECTS_PER_USER, handler_identity
  relay helpers); assertions unchanged.

(cherry picked from commit 92165a2b097126470e4827327d96d592a253adf1)
2026-11-03 02:49:26 +00:00
host c900a0f78d v2 provisional: clear integration 9 failures (whitelist tr+exit, B4 viewport draft, CHANGELOG pointer, workers audit via audit_on_conn, verify-multiuser gate evidence)
9 failures from v1 baseline 02835f6 cleared with provisional resolutions only;
each option pair is annotated provisional in verification/code-to10-v2-failures-summary.md
and awaits user confirmation. No check deleted, no skip added, no bare-phrase
stuffing, default hard merge gate untouched. Known remainder: one pre-existing
date-dependent test_arch_converge failure (Oct-dated fixture vs Nov current
month), failing on pristine v1 too, left for the quota track.
2026-11-03 02:49:26 +00:00
host cdb481092d release: gate-check contract for future release endpoint + ledger anchor ops (prep, no endpoint, no deploy) 2026-11-03 02:49:26 +00:00
host 765cff29ff integrate: rebase transplant-prep auditor expectation to v0.5 chapter table (11-auditor) 2026-11-03 02:49:26 +00:00
host dc5df30a07 docs(skill): transplant prep for #244 audit additions into v0.3/v0.4/v0.5
Preparation only: no v0.3/v0.4/v0.5 text is changed, nothing is
claimed transplanted or deployed. Stacked on feat-audit-pushback
(#244, 39966bd).

- Appendix provenance 00..09: templates/roles/00-08 sources
  (byte-identical to #217) plus 09-auditor extracted from #244's
  appendix; check_appendix.py extended 00..08 -> 00..09.
  SKILL.md sole change (1 line, format-only): 09 chapter heading
  normalised to the strict source form, otherwise the #217
  checker cannot recognise the chapter. check_appendix.py passes
  on the real pack: 10 chapters match; --emit outputs 10 chapters.
- TRANSPLANT-PREP-244.md: per-paragraph transplant diffs for
  #244's +459 lines into v0.3/v0.4/v0.5 with anchors, grades
  (merge / merge+note / partial stop / full stop), chapter-number
  mapping (v0.5: 11-auditor, colliding 09-ops noted), the
  templates/artifacts delete-references decision with 6 sites and
  replacements, and the draft-sync checklist (sync itself is
  post-deployment only, not done here).
- tools/check_transplant.py: anchors / draft-apply /
  check-transplanted modes; draft-apply on the real v0.3/v0.4/v0.5
  texts passes check-transplanted (drafts in /tmp only, not
  committed). Stopped paragraphs are listed, never silently
  rewritten: v0.3 G7/full-auto sentences, v0.4 G7 sentences and
  the tighten-3 pre-rebuild confirmation, v0.5 whole QA section
  plus its checklist item and the confirmation paragraph.
- Tests: test_skill_appendix_provenance.py (00..09) and
  test_skill_transplant_prep.py; 13 new tests pass.
  Suites on this tree: cloud 1523 OK (skipped=4), top-level 105
  OK, deploy 39 OK.
2026-11-03 02:49:26 +00:00
host 24517d922b host: 验测命令白名单 + 结构化 verify_argv(shell=True 收口)
方案 A(白名单)+ 方案 B-lite(结构化 argv)双轨:
- 分类器 classify_verify_command:POSIX 引号状态机扫描,
  simple(无 shell 形态)/ simple-env(仅 VAR=val 前缀)走
  shell=False,shell(白名单内形态)走唯一的 shell=True 门,
  denied(嵌套 shell、命令替换、未列首词、绝对/越界重定向、
  多行等)拒绝执行并审计,返回 126 按验测失败处理。
- 验测线 shell=True 从 4 处收口到 1 处(run_gated_verify),
  run_verify 与轮次验收 2 处全部改走该门;bandit High 4->2
 (nosec 0,未做任何压制)。
- verify_argv:merge/goal/goal-resume 的 --verify-argv
  (REMAINDER,须放最后;空数组报 usage 退出,不静默跳过),
  任务记录 verify_argv,goal-resume 原样重放;goal-resume
  的 max_rounds==0 钳制计入 argv 形态。
- verify-audit:扫描 state 旧任务并分类;--apply 只转 simple,
  shell/denied 只列清单;审计本身不启动 codex server。
- notify_failure 不动:算子自有配置 + 插值已 shlex.quote
  (main 已有),白名单不适用,见 PR 正文定性表。
测试:新增 tests/test_host_shell_policy.py 30 项(含 naive
shlex.split 破坏回归锁、拒绝不执行标记文件、merge 回滚、
goal unmet/argv 达标、CLI REMAINDER/空数组、verify-audit
端到端);顶层 135、deploy 39、cloud 1430(skipped 4)全过。
2026-11-03 02:49:26 +00:00
host dc556b0c19 audit: lifecycle hook writes the attempt ledger (gap 3 follow-up)
- cloud/attempt_hook.py (new): canonical policy -- one counted
  dispatch = one row, is_defect always False from the worker;
  human/first rounds and human breakpoint resumes count;
  platform auto-resume reruns, coordinator fan-out, and
  pre-execution gate failures do not; transient retries inside
  one dispatch are not new attempts. Platform origin survives
  via the task_attempt_origin_markers table (same-transaction
  write on requeue, queued_at generation guard on consume).
  Defects are explicit-recorder only; corrections are
  append-only (record_correction, last-wins).
- cloud/worker.py: Worker(attempt_ledger_hook=False) + CLI
  --attempt-ledger-hook (default OFF = gap-3 behaviour exactly);
  marker write in auto_resume_interrupted, marker consume +
  classify in run_one, one ledger write at each terminal point
  (cancel branch, normal terminal block); all hook writes
  best-effort, never fail the task or block a requeue.
- cloud/attempt_ledger.py: corrects_attempt_no column (new DDL
  supersedes gap 3's pre-merge DDL) + record_correction +
  correction-aware aggregation; read path uses SELECT * so
  gap-3-era tables still read; cost code untouched.
- cloud/tests/test_attempt_hook.py: 32 tests (policy, markers,
  corrections, worker integration, mixed endpoint).
- docs/arch-conformance-metrics.md: policy, switch, PG manual
  steps (new PG_DDL + PG_MARKER_DDL), remaining user decisions.
2026-11-03 02:49:26 +00:00
host 436a92ba1d fix: close bare open() in host.py and ResourceWarning in test_host_thread_cli
- host.py: 7 one-shot reads switched to with open (run_health_check
  pidfile, daemon-status pid/log, daemon-stop pid, daemon old pid,
  interrupt poll state, parallel jobs); read content, encoding args
  and error semantics unchanged.
- tests/test_host_thread_cli.py: test_old_archive_unchanged state
  read switched to with open; added a warnings-capture assertion
  scoped to the state read so a regression fails the test.
- Intentionally untouched: daemonize log handle (daemon-lifetime
  dup2 target) and the two _load_config_file / init sites owned
  by PR #266.
2026-11-03 02:49:26 +00:00
host 72d343b2c7 fix(host): daemon 循环把显式全局 flag 透传给 recover/run-due 子进程
上一轨 #266 只修了 --state,本轨补其余项:--codex-home、
--codex-bin、--model、--model-provider、--effort、--sandbox、
--approval-policy、--approval、--turn-timeout、--progress-interval。
旧循环只传 --config,子进程重解析回落到配置/默认值,与 daemon
本体跑 turn 时的 CLI 覆盖值不是同一套(run-due 的 Server 与
dispatch/run_task_turn 全读这些值)。
修法:配置解析覆盖 args 之前先快照显式 CLI 值,循环里只把显式
值原样放进子进程 argv;数值 flag 用 is not None 判定(0 是合法
显式值:turn-timeout 0 不限时、progress-interval 0 关进度);
相对路径原样透传(daemonize 不 chdir,子进程 cwd 一致)。
有意不传:--config(已传解析后 config_path)、daemon 专属
--interval/--foreground(子命令不接受)、--ignore-host-budget
(定给云端 worker 路径,且 recover/run-due 都不查预算,传了
无行为差异、只留将来预算门继承口)。
--state 透传沿用 #266 语义原样(显式 is not None 才传),未改动。
回归:tests/test_host_daemon_passthrough.py 4 项。
2026-11-03 02:49:26 +00:00
host 24adb568da fix(host): _read_loop 跳过非 dict params 等脏消息,不杀死读循环
顶层为 dict 但 params 为真值非 dict 时,旧代码在 params.get 处抛
AttributeError,被外层 except 吞掉、读线程退出,后续正常消息
再也送达不了。改后校验跳过并记 stderr 日志、脏消息不进 events、
读循环继续;None/缺失/假值沿用旧 or {} 语义。审计同函数其它
get 路径:一并守住不可哈希的响应 id(pending.get)与 threadId /
thread.id(turn_queues.get)的 TypeError 致死路径;params.thread
已有 isinstance 守卫、_append_trace 已有 try 包裹,均不动。
2026-11-03 02:49:26 +00:00
host fddd08546d test: test_queue_sandbox robustness 断言不再读 sys.modules["queue"](全量语境下该槽位已被其它测试的 cloud/ 遮蔽污染) 2026-11-03 02:49:26 +00:00
host 48e66d6c14 test: test_queue_sandbox 改 cloud 包导入,不再依赖顶层 queue 遮蔽标准库
旧写法把 cloud/ 插到 sys.path 前面再 from queue import TaskQueue:
一旦其它测试模块先 import 标准库 queue(缓进 sys.modules),本文件
即 ImportError。改成 cloud.queue / cloud.sandbox 包导入(失败时按
test_docker_sandbox.py 同款 fallback 补项目根目录),mock.patch 目标
同步改成 cloud.sandbox.*。新增 TestImportRobustness:断言标准库 queue
未被遮蔽,子进程先 import 标准库 queue 再加载本文件仍全过(子进程
以环境变量跳过自身用例防递归)。
2026-11-03 02:49:26 +00:00
host 1e024f175f docs(autoscaler): correct queue_depth docstring on dependency cycles
Cycle tasks never reach topo sort: ready_tasks filters by
'deps all completed' first, so they neither raise ValueError nor
zero out other ready tasks (5 ready + 1 cycle pair still counts 5;
only a pure cycle counts 0, because its ready set is empty).
Counting logic unchanged.
2026-11-03 02:49:26 +00:00
host 88e16ee8ac fix(server): remove unreachable duplicate lines in _latest_ts
Delete the second vals/return pair after the first return (was
lines 718-719); behaviour unchanged: max of numeric values, else None.
2026-11-03 02:49:26 +00:00
muse 45be478d70 fix(host): daemon 循环把显式 --state 透传给 recover/run-due 子进程
复核结论:应继承,不是原行为有意。daemon 本体的 pidfile/
logfile 按解析后 args.state 落盘,但旧循环只传 --config,
子进程回落到配置 state(或默认 state);CLI --state 与
配置 state 不同时,recover/run-due 跑的不是同一 state 文件。
修法:main() 解析前先记 explicit_state(argparse --state
默认 None,未给即 None);循环里仅当 explicit_state 非 None
时追加 --state,未显式给时不加、子进程按原规则解析,
与 daemon 本体一致、行为不变。daemonize() 本身只接
pidfile/logfile,不涉及 state,问题在 main() 循环里。
回归:DaemonStatePassthroughTest(2 项),旧代码上显式
项 fail、无显式项 pass,新代码上 2 项全 pass。
2026-11-03 02:49:25 +00:00
muse 1156d3c754 fix(host): _load_config_file 与 init 的 open() 改用 with 关闭
行号纠正:摘要的 2199/2443 是另一基线行号;origin/main
aebc623 上两处是 _load_config_file 的 json.load(open(path))
与 main() init 分支的 json.dump(init_cfg, open(...)),
2443 行在 aebc623 上是 if not args.repo: 并无 open。
只改资源关闭:open 参数、读写内容、错误语义(坏 JSON 仍
转 RuntimeError)均不变,不加 encoding、不改其它裸 open()。
回归:ConfigFileCloseTest(3 项),旧代码上 3 项全 fail、
新代码上全 pass(warnings 过滤 + gc 证明无 ResourceWarning)。
2026-11-03 02:49:25 +00:00
muse e768959c93 fix(host): _read_loop 跳过非 dict JSON 消息,不再静默死亡
收到合法 JSON 但值不是 dict(如 ""、123、[]、null)时,
旧代码走到 msg.get('params') 抛 AttributeError/TypeError,
被外层 except 吞掉后读线程退出,此后所有 request() 等到超时。
现校验后记 stderr 日志并跳过,读循环继续;脏消息不进
events、不当正常请求处理。畸形 JSON 本已由 json.loads 的
except continue 处理,本次未改动该路径。
回归:tests/test_host_bugfix_host.py ReadLoopNonDictTest(2 项)
2026-11-03 02:49:25 +00:00
host b9466e556b feat: 7-dim project audit scorecard (all >=9 to pass, average never a pass criterion)
- cloud/audit_scorecard.py: pure stdlib scorer, structured evidence in,
  7 dimension scores (0-10, 1 decimal), basis, missing/untested items,
  overall pass only when all 7 >= 9; unknown -> None, never 0/10/estimated;
  fixture/viewport/code-review evidence -> untested, never pass;
  thresholds all test_default_unapproved, user_approved only on explicit input.
- server.py: GET /audit-scorecard read-only, same auth/member scope as
  /metrics, no DB writes, no ledger writes, no pushback/reaudit triggers;
  on main base it honestly reports unknown/not-enforced, overall not_pass.
- docs/audit-scorecard.md: spec, formulas, threshold table, unknown
  handling, overall rule, 10-to-7 merge mapping, relations to
  #238 metrics-report, #244 sampling, #260/#261 loop, and a system-computed
  example (current state, overall not_pass) clearly separated from the
  previous judgement estimates.
- tests: 12 new items pin all-pass, one-below-nine fails, average 9.1
  with one 4 still fails, unknown None, fixture untested, conditional
  not a pass, thresholds unapproved, endpoint 401/200 read-only.
- suites: cloud 1442 OK (skipped 4), top-level 105 OK, deploy 39 OK.
2026-11-03 02:49:25 +00:00
host 3eac2a702e audit: wire loop driver to server endpoints and release gate (track 2)
Server-side execution for the track-1 pure driver (#260), no driver
semantics changed: audit verdict / fix completion / correction /
rebuild confirmation endpoints persist LoopState in audit_loop_state,
write every step through the #241/#247 ledger path (ledger first,
state second, fail-closed), derive fix/rebuild subtasks, and block
merge/release via the existing #247 chain reader. POST
/tasks/{id}/release follows the #259 contract (global-admin write,
fresh hard-gate evidence, merged status, multi-source 409 with no
audit write on block); deploy.py stays outside task gates.
2026-11-03 02:49:25 +00:00
host 91b61e5302 audit: pure loop driver for the project-level audit rework loop (track 1)
LoopState + decide_audit + after_fix + apply_correction +
confirm_rebuild + emit, all pure: the driver returns Action
objects, the caller executes (subtask creation, merge/release
blocking, ledger persistence). Counts per (module, contract
version); pushbacks 1-3, 4th failure (> 3) rebuilds, 1 rebuild
and 1 targeted round per version, third-failure triage
(structural -> early rebuild, contract/design -> refreeze,
near-complete -> targeted), rebuild-only-implementation with
the old-test first gate, acceptance rebuilds pending user
confirmation, quota blocks consume nothing, cost threshold
escalates rebuilds to the user with full history, corrections
to pass do not unblock, ledger events match the #241 table
with CONDITIONAL mapped to fail at the boundary.

No server wiring, no release endpoint, no task creation, no
DB access in this track; wiring is the next track's job
against this interface.
2026-11-03 02:49:25 +00:00
host a433460343 audit: unify audit-of-audit sampling rule (gap 2)
Single sampling rule; track 3 (#244) is authoritative for WHO is
mandatory, track 5 (#238) for HOW the rest are drawn:

- Mandatory (1.0): golden-task records + rebuild / refreeze /
  escalation records + third-failure near-complete still-failed
  cases (mandatory_triggers, full list per case). Track 3's old
  deterministic every-10th ordinary rule is removed (it drew zero
  from batches under 10 ordinary records and was predictable);
  track 5's old narrow mandatory list is expanded with golden /
  refreeze / escalate. blocked_quota stays non-mandatory.
- Random tiers: normal 0.10, critical-path 0.50, seed required
  and recorded, ceil so a non-empty group is never rounded to
  zero. 0.10/0.50 are TEST DEFAULTS, not approved production
  values; production rates remain a user decision.
- One pass, no double counting: mandatory records leave the
  random pools before drawing; a multi-trigger record (e.g.
  golden + rebuild) is sampled once with all triggers listed,
  and its correction follows the (original, corrected) verdict
  pair, never the trigger list.
- cloud/audit_pushback.py::select_audit_sample is now only a
  record-shaped adapter over cloud/audit_sampling.py::select_sample
  (seed required, retired reason names critical_path-for-kinds
  and periodic_sample removed); AUDIT_SAMPLE_EVERY_NTH deleted.
- Correction: original never mutated, correction appended,
  corrector != signer enforced in audit_sampling when both names
  are supplied; gate effect pinned (correction to pass never
  unblocks by itself -- schedule/cancel-and-rerun, blocked until
  a new passing re-audit; correction to fail re-blocks at once).
  Track 3 / track 5 retrigger vocabularies documented as synonyms.
- Skill text (tightening 3 item 2 + correction paragraph)
  rewritten to the unified rule; new test file pins the
  boundaries (mandatory union, dedup, ceil incl. the 9-record
  case, seed reproducibility, golden-set construction under both
  sample vocabularies, adapter order/non-mutation, correction
  preservation on both tracks).
2026-11-03 02:49:25 +00:00
host 8006a6fcba audit: route automation audit rows through hash-chain pair (gap 4, option A)
Bypass closed: automations._audit_row (direct audit_log INSERT,
best-effort swallow) is deleted. run_due_automation now writes via
cloud.auth.audit_on_conn, the same audit_log + audit_ledger pair
AuthStore.audit() writes, sharing created_at. The audit pair runs
before record_automation_run's commit, so task row, run row and
audit pair commit or roll back together; chain-head IntegrityError
retries the whole block up to 5 times, other errors roll back and
raise (fail-closed). The claim stays committed, so a failed run is
retried next interval, never recorded as a failed run.
AuthStore.audit() now delegates to audit_on_conn; its pair, retry
count and fail-closed semantics are unchanged.
2026-11-03 02:49:25 +00:00
host 0db51f8145 audit: task-attempt ledger feeds real first-pass/defect/rework rates (gap 3)
- cloud/attempt_ledger.py: separate task_attempts table (not tasks
  columns) with per-attempt rows, MAX+1 attempt_no, append-only
  intent; interrupted rows kept but never counted; is_defect is
  explicit, never inferred from failure; first defect category
  wins; no-ledger tasks stay unknown (None), never backfilled.
  SQLite DDL is CREATE TABLE IF NOT EXISTS (idempotent); PG_DDL
  is the verbatim manual step for production Postgres, kept out
  of POSTGRES_SCHEMA on purpose (migrations coverage untouched).
- cloud/server.py GET /metrics-report: aggregates the ledger
  into compute_metrics records; missing ledger table degrades to
  unknown rates, not an error. Cost dual-column code untouched.
- cloud/metrics.py: docstring only (ledger is now the source);
  compute_metrics logic and all #238 semantics unchanged.
- Tests: 16 ledger tests (store/aggregate/metrics/PG adapter
  evidence) + 3 endpoint tests (real rates, old task beside
  ledger task, interrupted status non-terminal).
2026-11-03 02:49:25 +00:00
host 238ebbdbd2 integrate: restore CostStore import dropped with #241 dual_column_view line
The #241 import resolution deleted the whole 'from cloud.cost import
CostStore, dual_column_view' line to drop the unused alias, which
also removed the CostStore import that _quota_snapshot needs; every
task creation then failed closed with 503 quota_check_unavailable
(4 EndpointChainTestCase failures in test_audit_wiring). Restored
as a plain CostStore import; dual_column_view stays available in
cloud/cost.py for #241's own tests.
2026-11-03 02:49:25 +00:00
host f1f8c8bf62 feat: audit-of-audit sampling rules (golden set and critical paths)
- cloud/audit_sampling.py (pure, stdlib): mandatory sampling (1.0)
  for rebuild cases and for third-failure near-complete cases whose
  continuation round still failed; random sampling for the rest
  (normal 0.10, critical-path 0.50, ceil so small groups are never
  rounded to zero, seed required and recorded for reproducibility).
- Wrong-verdict handling: correct the original conclusion and
  retrigger the corresponding flow (wrong pushback ->
  cancel_pushback_and_rerun_audit; wrong rebuild ->
  halt_rebuild_and_restore; wrong pass -> reopen_audit_and_pushback;
  unknown original -> escalate_to_human); every result, correct or
  corrected, is appended to the audit-of-audit record.
- Golden-set approval checklist: list reviewed, criteria defined,
  sample exclusion, approver+timestamp -- all human-confirmed before
  approval; sample entries (sample=True or sample- ids) are dropped
  and counted, never counted as golden.
- metrics.build_report gains an optional sampling_plan section;
  without a plan it states sampling is pending, never implying the
  audit was re-examined.
2026-11-03 02:49:25 +00:00
host dba6536ac2 feat: architecture conformance checks in CI and metrics feedback loop
- cloud/arch_conformance.py: contract-version (+1 only, approved
  change record), forbidden-directory (own dir only, .. and symlink
  escapes resolved), forbidden-import layer bans audited against the
  baseline (foundation/execution/driver, server sqlite3 Row adapters
  deliberately allowed), G2 frozen-stack swap guard; skill-project
  one-call check.
- cloud/metrics.py: first-pass/defect/rework rates over terminal
  tasks, unknown rates as None (never 0), #212 dual-column per-task
  cost (total None when incomplete), pure feedback rules
  (tighten/route-with-extra-audit/recalibrate, auto_apply False),
  golden-set gate: unapproved sample set is never decision-ready.
- GET /metrics-report (authenticated, member-scoped, same as
  /metrics): attempts/defects are None until an attempt ledger
  exists, so rates stay unknown rather than fabricated.
- Tests: 29 arch + 15 metrics + 3 endpoint, each check with
  positive and negative cases; repo-level scan pins zero layer
  violations on the baseline so a future violation fails CI via the
  existing host-verify discover run (no workflow change).
2026-11-03 02:49:25 +00:00
host 382ec6d62e audit: wire pushback gate to hard merge gate and hash-chain ledger (gap 1)
Track 3 decisions (audit_pushback) now really block the server merge
endpoint and really land on the track 4 hash chain, via the new
cloud/audit_wiring.py translation layer:

- POST /tasks/{id}/merge ANDs the chain pushback block
  (merge_blocked/release_blocked/blocked_by=audit_pushback/round/
  failed_checks, replayed per module+contract version) with the
  hard gate's five checks in every gate mode; either side blocks
  -> 409, status unchanged, audit_pushback_blocked appended to
  merge_gate.blocks. Ledger reads that fail (other than a missing
  audit_ledger table) fail closed to 409.
- Every pushback/fix/re-audit/rebuild/escalation step is written
  to the ledger with track 3 fields authoritative and track 4
  storage authoritative; CONDITIONAL maps to fail at the boundary
  with the original preserved, history never rewritten.
- Hard-gate 409 blocks codes feed pushback packages machine-first
  (branch protection survives as its own extra check).
- Re-audit QA guard: a claimed pass with un-rerun or failed QA
  links never reaches the chain.
- Track 4 compat: one audit_wiring_event action for escalation /
  blocked-quota / audit-sample kinds, counted in the timeline.
- Resolved tension (documented in audit_wiring.py): a meta-review
  correction to fail re-blocks at once, but a correction to pass
  does NOT unblock by itself -- track 3 schedules a re-audit,
  it does not release the gate; track 4 failure counts still use
  effective verdicts, so the two views may disagree while a
  re-audit is pending.

Tests: cloud/tests/test_audit_wiring.py (18 items), incl. a real
HTTP end-to-end chain (fail -> 409 -> fix -> re-audit pass ->
200) and a separate QA-link chain test.
2026-11-03 02:49:25 +00:00
host eeb10118d6 feat: tamper-evident audit ledger (hash chain) + dual-column cost view
- Hash-chain ledger (cloud/ledger.py): prev_hash/entry_hash per record,
  SQLite append-only triggers, UNIQUE prev/entry hash with fail-closed
  retry at the transaction boundary; verify/export (incl. expected-head
  check for tail truncation); legacy audit_log rows marked pre-chain via
  a single ledger_genesis marker, never rewritten.
- AuthStore.audit() writes audit_log + chain entry in one transaction.
- Track-3 interface: audit_verdict / audit_pushback_round /
  audit_rebuild / third-failure assessment / test-change audit /
  meta-review / rebuild-notice entries; per-(module, contract-version)
  failure counts derived from the ledger (pass or completed rebuild
  resets; total > 3 triggers rebuild); acceptance-triggered rebuilds
  require a confirmed user notice.
- Admin read-only endpoints: GET /admin/audit-ledger/verify|export.
- Cost: dual_column_view (priced_usd_micros + billing note, wording
  reused verbatim from PR #212) wired into project/task cost endpoints;
  by-model and per-creator reporting stay in #212, not duplicated here.
- PG schema + migration lists updated for audit_ledger (incl. sequence
  setval alignment).
2026-11-03 02:49:25 +00:00
host c0ab995ccc integrate: appendix count assertion follows 12-chapter table 2026-11-03 02:49:25 +00:00
host e921db3472 audit: independent audit role, auto-pushback, rebuild branch, three tightenings, QA full-chain
Independent audit (no code, independent re-run, separate signature;
CONDITIONAL never passes; deviation needs due todo) and automatic
pushback (fail -> derived fix subtask, merge/release blocked,
package with per-item evidence, full re-audit, 3-round limit with
a separate CI auto-fix counter, ledger hash-chain interface for
track 4, gate-block interface for track 1). Over 3 failures per
module per contract version: rebuild only that module from the
frozen contract (old implementation archived, avoidance list),
contract/design root cause -> G2 re-freeze escalation instead.
Tightenings: 3rd-failure triage (structural early rebuild /
early refreeze, near-complete one targeted round), rebuild keeps
tests and passed assets with an old-test first gate, audit-of-
audit sampling/correction and G6 user-only acceptance plus
acceptance-rebuild user confirmation. QA full chain: self-test
three parts, PRD coverage, agent six, cross-client matrix with
honest uncompiled verdicts, G5 gates, impact-scoped regression,
dual QA/audit signatures, failed-link re-runs in re-audit.
Skill text is purely additive (+459/-0) against main v0.2 text;
port this section into v0.3/v0.4/v0.5 texts on merge, never
overwrite them with this file.
2026-11-03 02:49:25 +00:00
host 41e8c98512 feat: cloud 门禁/超时/成本/沙箱收敛为单一判定源
- 新增标准库叶子层 timeouts/gates/quota/sandbox_policy:
  审批 3600 与单轮 600 各有名常量与环境覆盖层,已建项
  expires_at 永不重算;目录/控制/合并状态门禁统一判定,
  目录违规无任何覆盖参数;配额取数与比较唯一来源
  (USD 先于无价、命中均为 >=、无价只计 actual 行);
  /tmp 隔离矩阵与失联清理判定纯函数化。
- 调用点委托:server/worker/automations/auto_memory 的
  配额 SQL 与比较、server/tree/marker 的目录解析、
  marker 的控制门、sandbox 的任务容器名正则、两处 300 秒
  陈旧阈值默认值均收敛,别名与错误文案不变。
- 统一两个 P0 口径分歧:服务端无价计数由全部 usd IS NULL
  行改为 actual-only;显式不限额用户在服务端准入亦不受
  无价上限约束(与 worker 运行时一致)。
- 不新增门禁种类与 record 模式(主线无此实现);不改
  skill_seed/skills/SKILL.md;不改 v1.1 契约与分支保护。
- 新增 test_arch_converge.py 18 项。
2026-11-03 02:49:25 +00:00
host 2720acef46 integrate: sync appendix sources to v0.5 appendix table (00-10)
Integration arbitration, not a PR commit: v0.3/v0.4/v0.5 synced body
changes into appendix chapters 00/02/04/06/07 while templates/roles/
sources stayed at v0.2, so check_appendix failed 6 chapters; and
09-ops/10-iteration chapters (v0.5) were invisible to the strict
chapter regex because of the ', v0.5 new' title suffix, breaking
the #214 matrix-vs-appendix test. Direction is sources follow the
appendix (appendix is the tenant-visible authoritative version;
never overwrite v0.5 text with v0.2). 09/10 headers normalized to
strict form (suffix only); 09/10 sources split mechanically from
the appendix; EXPECTED_IDS extended to 11 chapters in the checker
and both appendix tests.
2026-11-03 02:49:25 +00:00
host be41ad2c0a test: pin skill appendix sync and SKILL.md-only seeding (C-line, no behaviour change)
- test_skill_appendix_sync: all 9 appendix chapters 00..08 in order,
  body role-matrix set == appendix role-title set, authority
  statement on both sides, GitHub guarantee in body + named
  appendix chapters 00/04/07, seeded copy byte-identical with
  full appendix. Previously only 00/08 existence was asserted.
- test_skill_seed_pinning: only SKILL.md copied (pack extras never
  land, never resurrected), tenant extras left alone, tenant-modified
  SKILL.md never overwritten across repeats, identical = noop,
  deleted SKILL.md re-seeded (documented boundary, not deletion
  preservation).
- skill_seed.py: evaluation note only (why whole-dir copy is not
  done); no code change.
2026-11-03 02:49:25 +00:00
host 06710cf47b feat: 沙箱规模——/tmp 顺序加严、注册表寻址与失联精确清理、超时边界抽取
- Docker /tmp tmpfs 移到全部 -v 之前(父挂载在前,seed 子挂载在后);
  宿主级 bwrap /tmp 仍宣告未隔离,口径见 tmp_isolation_status()
- workers 表补 host/pid/address(老库幂等补列),心跳透传与保留语义,
  _claim 写 claimed_by_worker;orphaned_containers 精确识别失联容器,
  Worker.cleanup_orphaned_containers 默认只杀 pid_verified 的精确容器名
- docker_run_timeout() 纯函数抽取(行为与原内联一致),README 补边界与
  未验证清单;新增 test_sandbox_scale.py 20 项
2026-11-03 02:49:25 +00:00
host 143ab30094 feat: cost/quota set — dual-column priced reports, creator split, hard cutoff, approval-timeout auto-threshold
- CostStore: per-model breakdown, report_by_creator (JOIN tasks.created_by),
  dual-column reports with priced_usd_micros, unpriced_models listed
  separately and a billing note (proxy price, not the relay bill);
  _report itself untouched, old exact-key tests keep passing.
- Hard cutoff: cost.quota_cutoff_reason is the single USD/unpriced-cap
  decision; worker._quota_exceeded now delegates to it (identical
  messages); claim gate, turn boundary and poll kill stop spend and
  record interrupted + quota reason + audit; auto-resume re-checks
  live quota before requeueing.
- Approval timeout (G7 ops metric): cloud/approval_timeout.py —
  default stays 600s, >20% timeout rate recommends 1800s, <5% keeps,
  mid-range keeps; stats + recommendation + info log; wired into
  /metrics (member projects, no log spam) and a global-admin
  /admin/ops/approval-timeout endpoint (sweeps overdue first).
- Server: task/project cost endpoints return dual-column reports;
  new GET /users/me/cost and GET /admin/cost-by-creator.
- Tests: cloud/tests/test_cost_quota.py, 10 tests.
2026-11-03 02:49:25 +00:00
host 0072e1ae6a skill: cross-task artifact pass with sha256 dual-channel verification
Core artifacts travel as an explicit manifest; coordinator triple-checks
(core report vs manifest vs recomputed hash) before cleanup, regression
rewrites files verbatim, recomputes sha256, stops on ARTIFACT-HASH-MISMATCH
and only then runs the same core code plus import-only regression tests.
2026-11-03 02:48:55 +00:00
host d973e63b11 feat: wire full-auto executor to server HTTP endpoints
GET /full-auto/status, GET /full-auto/config-plan, POST
/full-auto/validate-action|merge-gate|ci-fix|release-plan|
rollback-plan, POST /full-auto/execute (global admin only,
FULL_AUTO_ENABLED + verified installation + branch protection
verified via GitHub API, else 403 + audit), GET /full-auto/audit.
Guards reused from cloud/full_auto.py; token only in Authorization
header; audit via audit_record redaction into AuthStore audit_log.
2026-11-03 02:48:55 +00:00
host b4a69e1e71 refactor: remaining longfunc small extractions after #231 (no module split)
Only pure computation/validation sub-blocks of _handle_github_webhook,
_create_followup, _deliver_to_thread and _task_chat_history are extracted;
all DB writes, audits, external calls and side-effect blocks stay in the
callers, error strings/codes and response shapes verbatim. _deliver_to_thread
uses the shared goal classifier only for /goal input: its non-goal terminal
path still goes to CAS revive (not 409) and its merged path keeps its own
wording. New tests: test_longfunc_remaining.py (15).
2026-11-03 02:48:55 +00:00
host 1accfac844 refactor: extract minimal pure helpers from _continue_task, _task_wait, _task_thread_read and _metrics
_continue_task 226->168 via _validate_continue_options;
_task_wait 158->139 via _validate_wait_params;
_task_thread_read 187->152 via _summarize_trace_entries;
_metrics 123->104 via _summarize_metrics_tasks.
No behaviour, message, status or route change; error strings verbatim.
New tests: test_longfunc_extra (9). Cloud full suite: 1439 OK, skipped=4.
Follows #229 pattern; must merge after #229 and be rebased/checked then.
2026-11-03 02:48:55 +00:00
host f96960c9c5 refactor: extract minimal helpers from _dispatch, _dispatch_project and _create_task
Move the /tasks/* sub-route out of _dispatch into _dispatch_task_route
(query passed explicitly; the files/tree/thread branches use it), move
the push-subscription DELETE branch out of _dispatch_project into
_project_push_subscription_delete, and turn the goal-validation block
of _create_task into the pure module function
_validate_task_goal_options (returns error string instead of sending
400, caller sends it unchanged). Statements are moved verbatim except
indentation and the goal send+return -> return-error transform; no
behaviour change. server.py module split intentionally not done: it
remains a larger, higher-risk refactor.

Add test_longfunc_split covering the pure goal validator's success
paths and all eight error strings.
2026-11-03 02:48:55 +00:00
host a9e2f70d9e workers: reap pid_verified stale workers with pre-kill audit and post-kill verification 2026-11-03 02:48:55 +00:00
host 8ea54cf66b workers: land host/pid/address columns with idempotent migration and pid_verified stale classification
- ensure_workers_table: new tables include host TEXT, pid BIGINT,
  address TEXT; old tables gain them via _ensure_hostpid_columns
  (PRAGMA column check, missing-only ALTER, old rows keep NULL,
  never fabricates host/pid). SQLite and Postgres share this path
  (PG via the db adapter's PRAGMA translation); POSTGRES_SCHEMA
  deliberately still excludes workers, which is volatile heartbeat
  data in UNMIGRATED_TABLES (adding it broke the P2 migration
  alignment tests, verified and reverted).
- heartbeat/register: given host/pid/address refresh, omitted
  values preserve the old row; address has no derived source.
  Local worker passes host_name + own pid; HTTP heartbeat only
  stores caller-supplied values, never the server's own.
- classify_stale_workers: kill only when age strictly exceeds the
  threshold AND host matches local host AND pid is proven dead;
  equal threshold, foreign host, missing pid, or live/unknown pid
  are list_only. Classification only, never kills, never matches
  by name/prefix, never touches infra containers.
2026-11-03 02:48:55 +00:00
host 452fc8ea7e docs(skill): add software-engineering README and same-dir reference test
SKILL.md header pointed at same-dir README.md but the pack had no
such file (CHANGELOG was supplied by #217, README was not). This
track adds the README only: purpose, chapter map, appendix
provenance entry point, seeding behaviour, and known boundaries.
It states no rule, changes no SKILL.md semantics, and does not
duplicate the CHANGELOG/script explanations -- overlapping parts
are pointers, with an explicit division-of-labour section.
Deletion retention is written only as the tombstone-design
placeholder, deferring to the skill_seed implementation, because
no tombstone implementation has landed. Test pins that every
file SKILL.md cites as same-directory exists (CHANGELOG, README).
2026-11-03 02:48:55 +00:00
host 4e554b5c07 docs(skill): add appendix provenance sources, checker and CHANGELOG
SKILL.md claimed its appendix was spliced verbatim from
templates/roles/ by a generation script (see CHANGELOG), but the pack
contained none of the three. Add the nine role sources (mechanically
split from the v0.2 appendix; SKILL.md untouched, zero semantic
change), tools/check_appendix.py (verify + --emit, stdlib only, never
edits SKILL.md), CHANGELOG.md, and test_skill_appendix_provenance.py
(5 tests). Seeding unchanged: skill_seed copies SKILL.md only, so
templates/tools/CHANGELOG never reach the tenant home.
2026-11-03 02:48:55 +00:00
host 61e08a1a8c feat: hard merge gate by default (CI + directory + contract + coordinator + reviews)
- cloud/merge_gate_hard.py (new): pure hard gate; mode resolution is
  env-only (unset/invalid -> hard, fail-closed); five checks plus
  branch protection must be protected; unknown/unprotected block
  even with explicit confirm; no-evidence blocks (fail-closed);
  directory check reuses #209 check_directory_isolation (v0.3 rule);
  blocks codes + merge_gate_blocked audit detail are the track-3
  pushback interface (no pushback state machine in this track).
- cloud/server.py: POST /tasks/{id}/merge uses hard mode by default;
  record/enforce/off only when explicitly set via env (preserving
  #209 semantics); body merge_gate_mode is ignored server-side so
  callers cannot downgrade the gate; every gate block writes a
  merge_gate_blocked audit before returning 409, status unchanged;
  hard-mode body type errors return 400.
- docs/merge-gate-branch-protection.md (new): protection payload
  (PR + strict status checks + no force-push + no deletion +
  enforce_admins), empty required_checks = unconfigured, Pro/public
  prerequisites, 403 causes; states protection is NOT claimed enabled.
- SKILL.md (gate-mechanism wording only): four checks -> five checks;
  body + coordinator + client appendix gate paragraphs updated to
  default hard, merge_gate_blocked audit, payload doc reference.
- tests: 31 new hard-gate tests (pure + endpoint + skill text);
  #209 endpoint tests now set record env explicitly (their intent);
  test_server_merge 200-path calls now carry full hard evidence.
2026-11-03 02:48:55 +00:00
host d553f9549c feat: configurable merge gate (branch protection + directory isolation) and skill body/appendix checks
- cloud/merge_gate.py: pure gate; directory check reuses the v0.3
  git diff --name-only merge-base command; protection modes
  record (default, explicit confirm required when unprotected),
  enforce (always blocks when unprotected), off (explicit only).
  Confirm covers protection risk only, never directory violations.
  Protection status is caller/env-declared only; unknown is a
  recorded risk, not a claim of protection.
- server POST /tasks/{id}/merge: applies the gate only when gate
  evidence is supplied (or env declares protection); no-evidence
  calls behave as before plus a merge_gate field in the response.
  Blocked merges return 409 without changing status; confirmed
  unprotected merges write a merge_gate_risk audit entry.
- skill_seed.py: whole-directory seeding evaluated and rejected
  (cannot preserve tenant deletions; no benefit while the pack
  holds only SKILL.md); SKILL.md-only behaviour unchanged.
- software-engineering SKILL.md: gate paragraph added to the body
  and to the coordinator + client appendix sections (v0.2 text on
  this base; port to v0.3 text on rebase after #199 merges).
- tests: 16 new tests covering gate modes, directory reuse,
  body/appendix bidirectional assertions naming the missing side,
  SKILL.md-only seeding and tenant-edit preservation.
2026-11-03 02:48:55 +00:00
host 44616a03dc feat: software-engineering skill v0.5 (G7/G8 ops + iteration)
Stacked on v0.3 + v0.4 (HEAD d6522ba feat-full-auto): v0.3/v0.4
content preserved, not reverted. Adds G7 post-release ops and
observation (monitoring, errors, cost/quota, feedback, hotfix loop,
rollback triggers, ops manual) and G8 iteration and scaling (v1.1
handover list, contract v1->v2 protocol, new-end onboarding,
scale/performance, deprecation), with G6->G7 and G7->G8 gates,
DAG, gate table, coordinator derivation, G0-G8 todos, checklist,
09-ops and 10-iteration role templates, and ops-manual / v1.1
handover templates in both body and appendix. Settles web+React+
multi-viewport focus, cross-viewport E2E, web single-end release
with iOS/Android native postposed, open-source-first, and the
TypeScript full-stack default (user-request-only language change;
codex-app-server itself stays Python). Pending #199 and #205
merge, then one rebase round. PR only, do not merge or deploy.
2026-11-03 02:48:54 +00:00
host a605ff2f92 feat: full-auto mode v0.4 (host executor + skill + CI fix loop, stacked on v0.3)
Host auto executor (cloud/full_auto.py, stdlib only): default off
(FULL_AUTO_ENABLED + verified installation), forbidden actions
fail-closed (delete repo, force-push, visibility change, secret
read/expose, branch-protection delete/bypass, GitHub account
auto-registration), token only in request header, audit log
redacted, branch protection requires PR + strict checks +
no force-push + no deletions + enforce_admins, merge gate
(CI green + directory check + contract version + coordinator
approval), CI fix loop max 3 then escalate, rollback only via
revert PR or previous tag. Skill: v0.4 full-auto mode, coordinator
reviews, host executor executes, task agent never executes
merge/deploy; repo-creation honesty branch (org via App token,
user repo needs_user_once); no web auto-registration of GitHub
accounts. Tests: 19 new, cloud full 1451 OK (skipped=4).
2026-11-03 02:48:54 +00:00
host 34ed438d0e feat: software-engineering skill v0.3 (P1-1..P1-4 + P2)
- P1-1 directory-isolation executable check: git diff --name-only
  against merge-base, output recorded in merge record
- P1-2 derivation checklist table (8 required items, missing one blocks dispatch)
- P1-3 QA split into single-client review and cross-client E2E with
  fixed deliverables and agent-test attribution
- P1-4 sub-task slim-pack trimming rules (listed chapters only, never full copies)
- P2: inter-thread Q&A default policy, docs/decisions.md append-only,
  per-client onboarding docs (web first at G4a), v1.1 handover checklist
- Body and appendix kept in sync; new lightweight keyword test
2026-11-03 02:48:54 +00:00
host ef0052d15a coverage85-threshold: 覆盖率门槛 75% -> 85%(实测 85.2565%,余量 57 行) 2026-11-03 02:48:54 +00:00
host 49a2f7881b coverage85-step4e-fix: misc 不再 import stdlib queue,避免遮蔽 test_queue_sandbox 的 cloud/queue 导入 2026-11-03 02:48:54 +00:00
host 0d7fc867b5 coverage85-step4f: host 纯函数与小方法长尾测试 2026-11-03 02:48:54 +00:00
host 12f56cd8c8 coverage85-step4e: workers/autoscaler/playwright 长尾测试(+29 项) 2026-11-03 02:48:54 +00:00
host 3dc4ebe273 coverage85-step4d: host main 尾部命令块测试(+30 项) 2026-11-03 02:48:54 +00:00
host 3b7d45c9a2 coverage85-step4c: server 长尾端点与子路由测试(+19 项) 2026-11-03 02:48:54 +00:00
host 21a5fd3a2a coverage85-step4b: worker marker_tools 派发剩余分支测试(+20 项) 2026-11-03 02:48:54 +00:00
host 0e665d616f coverage85-step4a: worker run_one/_run_attempt 缺口测试(+34 项) 2026-11-03 02:48:54 +00:00
host 4cd9b5eaa4 coverage85-step3: 运行器统一加根路径,tests/ E402 清零
新增 run_tests.py:三套件统一入口,只加仓库根(deploy 套件
再加 deploy/ 目录在根之前),各套件走独立子进程;verify 作
业改走运行器,coverage 作业因子进程测不到覆盖率保持原
discover 命令。tests/ 13 个文件的 sys.path 前置 import 全部
改回正常 import(含 test_host_trace 的 try/except 回落);
deploy 的 deploy 目录插入与 verify-multiuser 的 spec 加载按
陷阱 2 如实保留(仅删 verify-multiuser 冗余的根插入)。
.coveragerc omit 加 */run_tests.py,使新运行器文件不计入
生产覆盖率。门槛未动(仍 75%),未补新测试。
2026-11-03 02:48:54 +00:00
host 903f138fd9 coverage85-step2d: 新测试 lint 清理(删无用导入与变量) 2026-11-03 02:48:54 +00:00
host 1e9c1b36ee coverage85-step2c: host 剩余缺口真实行为测试(+18 项)
- merge_task:成功、验测失败回滚、-X ours 自动解成功(分支
  新增行全在主仓)与丢改动回滚;run_health_check(好/坏/损坏
  state/损坏 pidfile/备份存在);notify_failure(无命令/写标记/
  坏模板/失败 rc/注入文本不执行)。
- stub Server:dispatch(含 seed-dir 与无 worktree 拒绝)、
  dispatch_from_branch、run_task_turn 三态与重试预算门、
  run_goal_loop(验测达标/标记达标/未达标/预算中断/回合异常)。
- main() 进程内:init/config/doctor/failures/health/daemon-status
  /daemon-stop/sleep/github-workflow/pr/review-comment/discard/
  automations-cron-line/restore-state/recover/dispatch/status/list/
  thread-*/skills/apps/mcp/run/compact/archive/code-review/fork/
  delegate/interrupt/merge/goal-resume;daemonize 子进程双 fork
  写 pidfile 与 marker 验证。
生产代码零改动;dispatch 建的 worktree 在 tearDown 清理。
2026-11-03 02:48:54 +00:00
host 3d2df7dfe6 coverage85-step2b: server/worker 剩余缺口真实行为测试(+24 项)
- server:_dispatch_project(项目 PATCH repo_url、成员邀请/改角
  色/删除与 404/403/409 族)、_create_task 校验 400 族与合法变体
  (goal/协调员/plan/环境)、trace/agents/trace-stream(终态 SSE
  snapshot+done)、GitHub webhook 去重与坏体。
- worker:_gateway_context(xcmapi/缺 origin/缺属主/假 sub2api
  key 与协调员父属主回查)、_ensure_gateway 缓存、_docker_task_
  context(代理/租户网络失败/搜索透传)、_deliver_marker_tools
 (守卫空跑、创建派生、goal 转换、轮询快照)、协调员孤儿扫尾、
  run_one/run_all/run_loop/main 空库路径。
生产代码零改动。
2026-11-03 02:48:54 +00:00
host 576fd50ee3 coverage85-step2a: MCP/deploy/verify-multiuser 剩余缺口真实行为测试(+43 项)
- playwright/imagegen/websearch 三个 MCP:handle_request 全工具分发、
  通知/错误分支、main() 进程内 stdio 循环(pipe/StringIO + 假浏
  览器/假依赖)、LineReader 分块与 EOF、真实 RemoteBridge 构造、
  flat-copy 回落导入;三文件孤立覆盖率 98%(余 __main__ 与
  通知型 handler-error else 分支,见回执)。
- deploy.py:check_clean_tree 脏/暂存/放行、archive 失败、remote
  助手、transfer 全路失败与 sha 不符、git 源冲突与干跑、tar
  路径干跑/构建/部署成功与失败;孤立 92%(余行由既有 deploy
  测试覆盖,__main__ 除外)。
- verify-multiuser main():真实本地控制面 13/13 通过且清理归零、
  死端口与 stub 项目失败早退、清理触发器异常报 leftover。
生产代码零改动。
2026-11-03 02:48:54 +00:00
host 81a58daca2 coverage85-step1: server/worker/host 主力缺口真实行为测试(+113 项)
只补 cloud/server.py、cloud/worker.py、host.py 的真实行为测试,
不动生产代码:
- server:助手函数(路径/租户/提取/链路读/配额/限流/心跳/
  host CLI stub 脚本)40 项
- worker:纯助手 + Worker DB/认领/容量/配额/跟进/租户/协调员
  孤儿/回退 token,真实 SQLite 与真实子进程 36 项
- host:假 app-server 脚本走真实 JSON-RPC 协议,run_turn
  成功/超时/中断/失败/历史错误回捞 + 状态/git/PR 助手 37 项
不含 mock 被测代码、无 skip、无 no-coverage、无空循环刷行。
2026-11-03 02:48:54 +00:00
host f39ed3654e ci3: lint/mypy 第二阶段清理 + 覆盖率 76% -> 78%(门槛不动)
基底:#251 头 174b631(堆叠于 #242 feat-ci-expand 之上),
基线 origin/main aebc6238;本轨不改 #251 已定语义、不动
workflow 门槛、不删检查、不加 skip。

lint(ruff E,F@120):109 -> 10
- F401 43 -> 0:删未用导入;try/except 双绑定的名字出现
  “打地鼠”(删 except 分支后 try 分支又被标出),两分支
  均已核对正文无引用后清掉;MCP 三模块的 Error 名不删,
  改走 importlib 属性绑定(见下),两路径绑定都保留。
- E702 40 -> 0:全在 host.py main() 的 argparse 段,纯分行,
  tokenize 按分号 token 拆(含 2377 行多行语句只断首个分号);
  一次 5 空格缩进失误已回滚重做并以 ast + 全量 host 测试验证。
- E501 13 -> 0:host.py 5 项随 E702 分行连带清掉;mcp_server
  3 项把描述拆成相邻字面量(AST 比对 24 条工具描述与原值
  完全一致,注意 ruff 按东亚宽度计长,CJK 算 2);test_host
  5 项断开 git commit 参数表。
- E402 8 -> 10:生产侧 2 项清掉(db.py _re、github_app
  _base64 移到顶部,之前无引用、无重名);剩余 10 项全是
  tests/ 的 sys.path 前置 import(6 项旧 + 4 个新测试文件
  沿用同一既定模式),移上去会破坏单文件直跑,有意保留。
- F841 4 -> 0:只去绑定、不删调用(run_task、DB 入队、
  tarfile 上下文、git 提交),各有注释与原测试/新测试覆盖;
  host dispatch_from_branch 新增真实 git 仓库回归测试。
- F811 1 -> 0:server 本地 _row_get 与 peer_messages.row_get
  逐行相同,删本地定义,全部调用点解析到同一 import;
  新增 test_server_row_get(同一对象 + 三路行为)。

mypy(生产范围):24 -> 0,零新增 type: ignore
- no-redef 9 -> 0:MCP 三模块 try/except 两次 from-import
  改 importlib.import_module + 属性绑定(cloud 优先、flat
  copy 回落、内层 ImportError 同样回落,语义与原来一致;
  flat-copy 子进程测试证实回落后 __module__ 为顶层名);
  server _row_get 随 F811 消除。
- arg-type 10 -> 0:harness2_tasks shared 改 TypedDict
  逐键精确类型;CodexProbeAdapter 补 no-op validate_tree
  (_adapter_validate 本就跳过缺失方法,行为一致)。
- var-annotated 3 -> 0:sub2api _SHARED_ADMIN/_login_slots、
  verify-multiuser RESULTS 加精确注解。
- assignment/misc 2 -> 0:marker_tools ZoneInfo 先声明
  Any 再 try import(草稿先验 mypy 通过)。

bandit:总数 286 不变,High 仍 4 项,全在 host.py,本轨不改
(属第 3 项白名单轨范围),只做 triage:
- 1319 run_verify:无插值,整串验测命令,管道/&& 须 shell
  语义(新增 test_host_run_verify 断言管道与 && 真实生效)。
- 1497 notify:模板来自算子配置,三处插值全部 shlex.quote,
  证据为既有 test_notify_failure_quotes_shell_metacharacters
  (; touch 与 $(touch ) 不执行)。
- 2099/2103 goal verify:无插值,任务配置验测串,同 run_verify。
- 无新增 High;pip-audit 0 漏洞。

覆盖率(同口径三套件 coverage 计量):75.96% -> 77.87%
(22392 行/缺 5385 -> 22437 行/缺 4963,显示 76% -> 78%)
- client.py 0% -> 90%(pipe 假子进程 + main 分支)
- run_parallel.py 0% -> 97%、migrate 包装 0% -> 95%
- playwright_mcp 37% -> 62%、imagegen_mcp 47% -> 79%、
  websearch_mcp 51% -> 86%、deploy.py 56% -> 77%
- verify-multiuser 0% -> 27%(仅 check/req helper;main()
  是主机侧真实多用户实弹验证,按门禁不建生产任务、
  不以 mock 冒充,仍为缺口)
- host.py 55% -> 57%(文件 2020 -> 2060 行系 E702 分行
  的计量口径变化,非代码膨胀)
- 未到 85%:门槛保持 75% 不动,不删低覆盖模块出统计口径;
  缺口主力仍是 server.py / worker.py / host.py 大文件,
  下一阶段须针对性深测,非本轨快赢可及。

验证:三套件 1576 -> 1629 项全过(tests 127、cloud/tests
1449 skipped=4、deploy/tests 53);ruff E9 通过;mypy 0;
bandit High 4(同上);pip-audit 0 漏洞;试跑产物已删,
提交后工作树干净。
2026-11-03 02:48:54 +00:00
host 7ba5bfaf81 ci2: fix worker F821 NameError, mark harness2 sha1 non-security, rename E741 var
- worker._tenant_dirs: playwright refresh log used undefined 'task';
  use the method argument project_id (ruff F821, real bug: NameError
  aborted tenant seeding when a playwright script was refreshed).
  Regression: cloud/tests/test_worker_playwright_seed_f821.py (2 tests;
  pre-fix run reproduces NameError at worker.py:2780).
- harness2.assign_ids: hashlib.sha1(..., usedforsecurity=False) -- the
  digest is only a deterministic shuffle seed, not a security use;
  digest value unchanged, bandit B324 no longer fires (High 5 -> 4).
- harness2.select_final: rename ambiguous loop var 'l' to 'lab'
  (ruff E741, behaviour-invariant).
- Deliberately NOT changed: host.py shell=True x4 (see PR body -- all
  four are intentional shell commands, changing semantics is unsafe),
  remaining ruff 109 / mypy 24, workflow thresholds (75% coverage
  gate unchanged; no gate was relaxed or removed).
2026-11-03 02:48:54 +00:00
host c127beaa98 ci: expand host-verify with lint, typecheck, security, depscan and coverage gates
Phase 1 of audit-hardening CI expansion (baseline origin/main aebc623):
- lint: ruff E9 hard gate; full E,F@120 report-only (baseline 111 items)
- typecheck: mypy default mode, production scope, report-only (baseline 24)
- security: bandit production scope, report-only (baseline 287, High 5)
- depscan: pip-audit on requirements-ci.txt, hard gate (0 vulns; repo has
  no production dependency manifest, see summary)
- coverage: production 58 files, fail-under=75 (baseline measured 76%)
- release build: E9 + pip-audit hard gates before packaging; full reports
  stay in host-verify to avoid drift
No production code changed; CI billing failure mode unchanged.
2026-11-03 02:48:54 +00:00
openqj 7b537c31a6 test: close unclosed file handles in tests (Path read/write)
Replace bare open().read()/write() in tests with Path.read_text/
write_text (or json.loads over read_text); the five remaining
open(...).close() empty-file creations become Path.write_text('').
No assertion, predicate, or production code changed; no behaviour
change beyond closing the handle deterministically.

Long-function splits and server.py module split intentionally not
included: the longest functions (_dispatch, _create_task,
_dispatch_project, ~375-395 lines in a 10430-line server.py) are
central routing/creation paths, and splitting them is not low-risk;
see PR body.
2026-11-03 02:48:54 +00:00
263 changed files with 70032 additions and 4195 deletions

No files matched your search

+28
View File
@@ -0,0 +1,28 @@
# Coverage 配置:只计量生产代码,测试本身、工作区、示例不计。
# 阈值不在此文件写死,写在 workflow 的 --fail-under 里,便于在
# workflow diff 中一眼看到门槛变化。
# source 注意:不能写 host.py / client.py / run_parallel.py 这种文件
# 路径——coverage 会把它们当模块名处理、静默丢弃(实测:那样 host.py
# 等 3 个根文件根本不进报告,总数少 2223 行)。正确写法是目录源
# “.”(含根模块与 cloud 包)+ deploy(非包目录,需单独列),
# 实测恰好 58 个生产文件、22392 行,与 mypy 生产文件数一致。
[run]
source =
.
deploy
omit =
*/tests/*
*/test_*.py
cloud/tests/*
deploy/tests/*
tests/*
work/*
examples/*
*/run_tests.py
[report]
show_missing = False
exclude_lines =
pragma: no cover
def __repr__
raise NotImplementedError
+141 -3
View File
@@ -3,6 +3,23 @@ on:
pull_request:
push:
branches: [main]
# CI 扩项(审计加固第 2 步,基线 origin/main aebc623):
# 每个新检查的生效范围与存量如实写在步骤注释里,不写空转检查:
# - lint 全量 E,F@120:基线 111 项,见 PR 摘要存量清单,第一阶段
# 只报告不阻塞(continue-on-error),语法 E9 仍是硬门禁;
# - mypy 默认模式、生产范围:基线 24 项,第一阶段只报告不阻塞;
# - bandit 生产范围:基线 287 项(High 5),第一阶段只报告不阻塞;
# - pip-audit:只审计 requirements-ci.txt(仓库无生产依赖清单,
# 生产代码除宿主延迟加载的 psycopg 驱动外只用标准库,见摘要),
# 该文件 0 已知漏洞时硬通过,有漏洞即硬失败;
# - coverage:阈值见 coverage 作业的 --fail-under,取实测值向下取整,
# 防回归不冒充高覆盖;收紧计划见 PR 摘要。
# 账单说明:CI 目前仍是 GitHub 账单故障秒级失败模式,账单恢复是
# 用户在 GitHub Billing 侧的动作;本 workflow 的改动不声称改了账单、
# 也不为变绿绕过任何检查,账单恢复后上述检查才实际生效。
# 失败即停:同一作业内步骤按顺序执行,前一步硬失败则后一步不跑;
# 报告类步骤用 continue-on-error + always() 上传报告,作业本身仍绿,
# 但报告 artifact 里是失败的原始输出,不是“通过”。
jobs:
verify:
runs-on: ubuntu-latest
@@ -11,6 +28,127 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: python3 -m unittest discover -s tests -v
- run: python3 -m unittest discover -s cloud/tests -v
- run: python3 -m unittest discover -s deploy/tests -v
# 测试入口统一走 run_tests.py:它在运行器层面加仓库根路径
# (deploy 套件再加 deploy/ 目录,见 run_tests.py docstring),
# 测试文件本身不再写 sys.path 前置 import(ruff E402 清零轨)。
# coverage 作业仍用下面三条 discover 原命令:运行器是子进程
# 分套件,coverage 测不到子进程里的测试。
- run: python3 run_tests.py -v
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r requirements-ci.txt
# 硬门禁:语法错误。基线 0 项,通过才继续;不是空集,
# E9 覆盖 SyntaxError / IndentationError 等编译级错误。
- name: lint 硬门禁(ruff E9 语法)
run: ruff check --select E9 .
# 报告类:全量 E,F@120,基线 111 项(F401 43、E702 40、E501 13、
# E402 8、F841 4、F811 1、F821 1、E741 1)。第二阶段清零存量后
# 删掉 continue-on-error,转硬门禁;删之前不声称 lint 已通过。
- name: lint 全量报告(ruff E,F)
continue-on-error: true
run: ruff check . | tee ruff-report.txt
- name: 上传 lint 报告
if: always()
uses: actions/upload-artifact@v4
with:
name: ruff-report
path: ruff-report.txt
if-no-files-found: warn
retention-days: 30
typecheck:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r requirements-ci.txt
# 报告类:mypy 默认模式、生产范围(mypy.ini 排除 tests/work)。
# 基线 24 项(8 文件,见 PR 摘要)。第二阶段清零后删掉
# continue-on-error,转硬门禁;不改生产代码凑通过。
- name: 类型检查报告(mypy 生产范围)
continue-on-error: true
run: mypy host.py cloud deploy client.py run_parallel.py | tee mypy-report.txt
- name: 上传类型检查报告
if: always()
uses: actions/upload-artifact@v4
with:
name: mypy-report
path: mypy-report.txt
if-no-files-found: warn
retention-days: 30
security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r requirements-ci.txt
# 报告类:bandit 生产范围,测试目录排除(测试故意用 shell/tmp)。
# 基线 287 项:Low 235、Medium 47、High 5。High 5 = host.py 的
# B602 shell=True 4 处(审计已知 P0)+ cloud/harness2.py 的
# B324 sha1 非安全用途 1 处。先修 High 再转硬门禁;不通过
# --skip 跳过 B602/B324 来假装通过。
- name: 安全扫描报告(bandit 生产范围)
continue-on-error: true
# bandit 有发现即 exit 1:该失败是步骤的真实状态,作业靠
# continue-on-error 继续到上传;不要用 `; cat` 吞掉 exit 码。
run: bandit -r host.py cloud deploy client.py run_parallel.py -x cloud/tests,tests,deploy/tests -f txt -o bandit-report.txt
- name: 上传安全扫描报告
if: always()
uses: actions/upload-artifact@v4
with:
name: bandit-report
path: bandit-report.txt
if-no-files-found: warn
retention-days: 30
depscan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r requirements-ci.txt
# 硬门禁:审计 CI 工具链清单本身。基线 0 已知漏洞。
# 局限如实说明:仓库没有生产依赖清单可审计——生产代码只用
# 标准库,psycopg/psycopg2 只在宿主侧延迟加载、版本不在仓库
# 里;给生产依赖建清单并审计是第二阶段项,不是本作业的覆盖
# 范围,本作业不声称覆盖了生产依赖。
- name: 依赖扫描(pip-audit,CI 工具链清单)
run: pip-audit -r requirements-ci.txt
coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r requirements-ci.txt
# 硬门禁:三套 unittest 在 coverage 下重跑(与 verify 作业同款
# 测试,只是加了计量),生产代码覆盖率低于门槛即失败。
# 门槛取实测值向下取整到 5 的倍数:本轨实测 85.2565%
# (22437 行,已覆盖 19129 行,余量 57 行),故门槛 85%,见 PR 摘要。
# 目的是防回归,不是 0 空转;余量仅 57 行,合轨后如回落到 85% 以下
# 不要靠删检查或加 skip 补,先看交集处的新缺口。
- name: 覆盖率计量与门槛
run: |
coverage run -m unittest discover -s tests
coverage run --append -m unittest discover -s cloud/tests
coverage run --append -m unittest discover -s deploy/tests
coverage report --fail-under=85
coverage xml -o coverage.xml
- name: 上传覆盖率报告
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-xml
path: coverage.xml
if-no-files-found: warn
retention-days: 30
+10
View File
@@ -42,6 +42,16 @@ jobs:
python3 -m unittest discover -s tests -v
python3 -m unittest discover -s cloud/tests -v
python3 -m unittest discover -s deploy/tests -v
# CI 扩项在发布链路的最小硬门禁:语法 E9 + CI 工具链依赖扫描。
# 全量 lint/mypy/bandit 报告与覆盖率门槛只在 host-verify 里跑
# (PR 与 main 推送都会跑,走到 tag 的提交此前已过 host-verify),
# 这里不重复跑三套报告,避免发布作业与 host-verify 的检查漂移;
# 这里的两项是硬失败,失败则不产发布包。
- name: CI 扩项硬门禁(语法 + 依赖扫描)
run: |
pip install -r requirements-ci.txt
ruff check --select E9 .
pip-audit -r requirements-ci.txt
- name: 构建发布包(git archive + DEPLOYED_COMMIT + sha256)
id: build
run: |
+49
View File
@@ -1252,6 +1252,55 @@ interrupted 且终态任务 cancel 返回 400、心跳后 sweep 不扫刚心跳
仍是串行一轮一任务)。两个端点都要 X-API-Key(与其它已认证
端点一致);worker 直接写库注册时不需要 key,HTTP 心跳是给有
key 的外部 worker 用的另一条路。
- 沙箱规模(本轮新增,见 `cloud/tests/test_sandbox_scale.py`):
- /tmp 隔离分层表(唯一口径 `sandbox.tmp_isolation_status()`):
Docker 任务容器 `--tmpfs /tmp` + `--tmpfs /var/tmp`(容器随
`--rm` 消失,/tmp 也随之消失,绝不落宿主 /tmp);且 tmpfs
挂载在全部 `-v` 之前,附件 seed 的 `/tmp/codex-seed:ro` 子挂载
压在 tmpfs 之上(旧顺序把 tmpfs 放最后,本 VM 无 docker 未
证实旧顺序确实遮挡,故写为顺序加严而非已证实 bug)。
宿主级 codex workspace-write(bwrap)仍不隔离:挂载由 codex
二进制内部决定,host.py 只传 sandboxPolicy 类型、无挂载旋钮,
本仓库改不了它;需要 /tmp 隔离的任务请走 docker 执行器,
不要拿 Docker 层的隔离给宿主级任务背书。
- workers 注册表寻址:workers 表新增 host(worker 的
`--host-name`,容器内 worker 绝不能用容器名冒充宿主名)、
pid(worker 进程号)、address(CLI `--worker-address`,显式给
才有,不含任何凭证);老库幂等补列,老行三列为 NULL 按未知
处理。`POST /workers/heartbeat` 同样接受三字段(host 非字符串、
pid 非整数、address 非字符串时 400);心跳未给时保留旧值,
worker 每次心跳都显式刷新本机 host 与本进程 pid,重启复用
同 worker_id 时不残留旧 pid。
- 失联容器精确清理(`workers.orphaned_containers()` +
`Worker.cleanup_orphaned_containers()`,只写识别与杀容器,
不改任务状态):容器名必须精确匹配 `codex-<32 位 hex 任务 id>`,
且任务行存在、状态为 running/interrupted、认领记录
`tasks.claimed_by_worker`(`_claim` 在 CAS 认领时写)指向
心跳过期的 worker,才算失联;基础设施容器(如 codex-postgres)、
非任务名、已完成任务、无认领记录、owner 仍存活的一律排除,
绝不按 `codex-` 前缀删。默认只杀 pid_verified 的容器(本机 host
+ 已知 pid 且 pid 明确已死);远端 host / pid 未知 / 未验证的
只列不杀,显式 `verified_only=False` 才杀(仅单机语义已确认
时开,跨主机场景绝不能默认开)。旧的
`cleanup_stale_containers()`(任务正则兜底、不区分 owner)保留
不改行为,供认领记录缺失的老任务容器用;任务状态本身由
`sweep_stale_tasks`(心跳过期 -> interrupted)负责,两者分工。
本轮只写代码与测试,绝不在本轮动生产容器、不杀任何生产任务。
- 长任务天花板与并行验证的代码支撑(不宣称已大规模实测):
超时边界抽成纯函数 `worker.docker_run_timeout()`(取较小值、
None 语义、no_time_limit 语义、非法值回落,行为与原内联一致);
取消 -> interrupted(执行结果绝不覆盖)、超时 -> failed、
恢复靠断点续跑(goal-resume / run 续跑),三者状态语义不变。
天花板验证方法(待真实大规模跑时按此执行,非本轮已做):
睡眠循环任务(每轮 sleep + 一行回复)测轮次/墙钟上限、
取消与超时各走一遍终态、owner 崩溃后 sweep + 精确清理各走
一遍;多端并行只做 web 自适应跨视口:console 已含 viewport
meta 与小屏 media 查询(单测只守基线),手工验证矩阵为
320 / 375 / 768 / 1024 / 1440 宽度(无横向滚动、导航可收起、
任务列表与详情可读),iOS/Android 原生明确不在本轮范围。
未验证项(诚实清单):真实 40 人并行、真实大规模长任务天花板、
真实 docker 下 tmpfs 顺序是否曾遮挡 seed、宿主级 bwrap /tmp
隔离(本仓库无修复手段,见上)。
- 任务按执行器池派发:tasks 表新增 `executor` 列(可空,老库幂等
补列);`POST /tasks` 接受可选 executor(local/docker/cloud,
非法值 400),未传时为 None 表示任意 worker 可认领;worker
+145
View File
@@ -0,0 +1,145 @@
"""审批超时率自动调阈(G7 运维指标,v1.1 契约口径)。
只依赖标准库与 cloud.approvals 的建表函数,不 import server /
worker,避免循环 import。
口径(与 v1.1 冻结条一致):
- 默认审批超时仍是 10 分钟(600 秒)。注意:这是本调阈口径
的默认值,不是 approvals.py elicitation 队列的 3600 秒
默认值——两条审批线各有默认值,本模块绝不改写后者。
- 超时率 = 已终态审批项中 status='expired' 的占比;已终态 =
approved + denied + expired。pending 不计入分母(还没结果
的项不能算超时也不能算准时);未扫荡的过期 pending 同样
不计入——扫荡发生在审批列表与运维端点,统计前应先扫荡。
- 超时率 > 20% 时建议调到 30 分钟(1800 秒);< 5% 时不动;
5%~20% 之间同样不动(只升不降,降阈须人工决定,本模块
绝不自动把超时缩回去)。边界值本身不动:>0.20 才升,
==0.20 与 ==0.05 都按不动处理。
- 本模块只算建议并记日志,不自动改写任何默认值或已建审批
项的 expires_at:调阈生效须运维按建议改配置后重启/重载,
建议值进 /metrics 与运维端点,审计靠日志行(见
evaluate_approval_timeout 的 info 日志)。
"""
import logging
from cloud.approvals import ensure_approvals_table
logger = logging.getLogger(__name__)
DEFAULT_TIMEOUT_SECONDS = 600
RAISED_TIMEOUT_SECONDS = 1800
RAISE_THRESHOLD = 0.20
KEEP_THRESHOLD = 0.05
def timeout_rate(expired_count, decided_count):
"""超时率:已终态为 0 时回 None(不是 0%,是算不出来)。"""
try:
decided = int(decided_count)
expired = int(expired_count)
except (TypeError, ValueError):
return None
if decided <= 0:
return None
if expired < 0:
expired = 0
if expired > decided:
expired = decided
return expired / decided
def recommend_approval_timeout(expired_count, decided_count,
current_seconds=DEFAULT_TIMEOUT_SECONDS):
"""纯计算函数:输入计数,输出建议(无 I/O,可单测三档)。
返回键:expired_count、decided_count、timeout_rate、
current_seconds、recommended_seconds、action(raise_to_30m
或 keep)、reason。current_seconds 非法时按默认值计,不抛。
"""
try:
current = int(current_seconds)
except (TypeError, ValueError):
current = DEFAULT_TIMEOUT_SECONDS
if current <= 0:
current = DEFAULT_TIMEOUT_SECONDS
rate = timeout_rate(expired_count, decided_count)
if rate is not None and rate > RAISE_THRESHOLD:
action = "raise_to_30m"
recommended = RAISED_TIMEOUT_SECONDS
reason = ("approval timeout rate %.4f > %.2f, "
"recommend %d seconds" % (rate, RAISE_THRESHOLD,
recommended))
else:
action = "keep"
recommended = current
if rate is None:
reason = "no decided approvals, keep %d seconds" % current
else:
reason = ("approval timeout rate %.4f <= %.2f, "
"keep %d seconds" % (rate, RAISE_THRESHOLD,
current))
return {
"expired_count": int(expired_count or 0),
"decided_count": int(decided_count or 0),
"timeout_rate": rate,
"current_seconds": current,
"recommended_seconds": recommended,
"action": action,
"reason": reason,
}
def approval_timeout_stats(conn, project_ids=None):
"""从 task_approvals 统计已终态与超时数(纯读,不扫荡)。
project_ids 为 None 时全库统计;否则只计这些项目(/metrics
的成员项目口径)。表不存在时先幂等建表(空表即零统计)。
"""
ensure_approvals_table(conn)
query = ("SELECT COUNT(*), COALESCE(SUM(CASE WHEN status = 'expired'"
" THEN 1 ELSE 0 END), 0), COALESCE(SUM(CASE WHEN"
" status = 'pending' THEN 1 ELSE 0 END), 0)"
" FROM task_approvals WHERE status IN"
" ('approved', 'denied', 'expired', 'pending')")
args = []
if project_ids is not None:
ids = sorted(set(project_ids))
if not ids:
return {"decided_count": 0, "expired_count": 0,
"pending_count": 0}
query += " AND project_id IN (%s)" % ", ".join(
"?" for _ in ids)
args.extend(ids)
row = conn.execute(query, args).fetchone()
decided = int(row[0] or 0) - int(row[2] or 0)
return {
"decided_count": decided,
"expired_count": int(row[1] or 0),
"pending_count": int(row[2] or 0),
}
def evaluate_approval_timeout(conn, current_seconds=DEFAULT_TIMEOUT_SECONDS,
project_ids=None, log=None):
"""统计 + 建议 + 日志的一站式运维入口。
返回 approval_timeout_stats 与 recommend_approval_timeout
的合并 dict,并写一条 info 日志(log 未给时用模块 logger)。
调用方在需要“过期 pending 也算超时”时须先调 approvals
的 expire_overdue 扫荡——本函数自己不写库。
"""
stats = approval_timeout_stats(conn, project_ids=project_ids)
result = recommend_approval_timeout(
stats["expired_count"], stats["decided_count"],
current_seconds=current_seconds)
result.update(stats)
(log or logger).info(
"approval_timeout metric: decided=%d expired=%d pending=%d "
"rate=%s action=%s recommended_seconds=%d",
result["decided_count"], result["expired_count"],
result["pending_count"],
("%.4f" % result["timeout_rate"]
if result["timeout_rate"] is not None else "unknown"),
result["action"], result["recommended_seconds"])
return result
+19 -26
View File
@@ -64,33 +64,29 @@
- item/tool/call 与 account/chatgptAuthTokens/refresh:前者是
动态工具执行,后者是凭证刷新,都不是用户审批,排除。
本模块只依赖标准库,不 import cloud 其他模块,避免循环
import;服务端与未来 worker/hook 共用。占位符用 ?,建表与
列补齐写法与 peer_messages 同款(db 适配层负责翻译)。
本模块只依赖标准库与同为标准库叶子的 cloud.timeouts,
不 import 其他 cloud 模块,避免循环 import;服务端与未来
worker/hook 共用。占位符用 ?,建表与列补齐写法与
peer_messages 同款(db 适配层负责翻译)。
"""
import json
import time
import uuid
from cloud.peer_messages import row_get
from cloud.timeouts import APPROVAL_DEFAULT_SECONDS as _APPROVAL_DEFAULT
from cloud.timeouts import expires_at_for as _expires_at_for
from cloud.timeouts import resolve_module_timeout as _resolve_timeout
KINDS = ("elicitation", "command", "file_change", "permissions",
"user_input")
DECISIONS = ("approved", "denied")
DEFAULT_TIMEOUT_SECONDS = 3600
DEFAULT_TIMEOUT_SECONDS = _APPROVAL_DEFAULT
MAX_SUMMARY_CHARS = 2000
MAX_DETAIL_CHARS = 20000
def row_get(row, name):
"""行取值容错:sqlite3.Row 缺列报 IndexError、dict 缺键报
KeyError、空行报 TypeError,一律回 None(与 peer_messages
的同名函数同款,模块独立所以这里自带一份)。"""
try:
return row[name]
except (KeyError, IndexError, TypeError):
return None
def ensure_approvals_table(conn):
"""幂等建审批队列表。调用方负责 commit(服务端入口在写
操作后统一 commit;本函数自己不 commit,避免半截事务)。"""
@@ -167,12 +163,15 @@ def approval_to_dict(row):
def create_approval(conn, task_id, project_id, kind="elicitation",
summary="", detail=None, requested_by=None,
timeout_seconds=DEFAULT_TIMEOUT_SECONDS, now=None):
timeout_seconds=None, now=None):
"""创建一条待审批项,返回 dict。非法输入抛 ValueError,
调用方据此回 400;成功时 commit。
timeout_seconds:None 用默认值;0 表示不过期(只允许
模块级调用,服务端入口另有限制);负数非法。"""
timeout_seconds:None 用默认值(cloud/timeouts.py 的
环境变量感知默认值,未设环境变量时为 3600;签名默认
故意写 None 而不是 3600,避免默认值在 import 时绑定死、
环境覆盖层传不进去);0 表示不过期(只允许模块级调用,
服务端入口另有限制);负数非法。"""
if kind not in KINDS:
raise ValueError("kind must be one of: " + ", ".join(KINDS))
if not isinstance(summary, str) or not summary.strip():
@@ -180,17 +179,11 @@ def create_approval(conn, task_id, project_id, kind="elicitation",
summary = summary.strip()
if len(summary) > MAX_SUMMARY_CHARS:
raise ValueError("summary too long")
if timeout_seconds is None:
timeout_seconds = DEFAULT_TIMEOUT_SECONDS
if not isinstance(timeout_seconds, (int, float)) \
or isinstance(timeout_seconds, bool) \
or timeout_seconds < 0:
raise ValueError("timeout_seconds invalid")
# 不传 default:None 时走 timeouts 的环境变量感知默认值。
timeout_seconds = _resolve_timeout(timeout_seconds)
detail_str = _detail_to_str(detail)
now = time.time() if now is None else now
expires_at = None
if timeout_seconds:
expires_at = now + timeout_seconds
expires_at = _expires_at_for(now, timeout_seconds)
item_id = uuid.uuid4().hex
ensure_approvals_table(conn)
conn.execute(
+298
View File
@@ -0,0 +1,298 @@
"""Architecture-conformance checks for the control plane and for
skill-driven projects (software-engineering skill, G2 frozen artefacts).
Zero third-party dependencies; standard library only. This module
imports no ``cloud.*`` module, so it can never violate the layer bans
it enforces, and it can be reused verbatim against a skill project's
own directory tree (see ``check_skill_project``).
The four checks, each with a stated rule and a stated reason:
1. Contract version (``check_contract_version``). Rule mirrors the
skill's contract-freeze change protocol: after freeze, a version
change is allowed only with a coordinator-approved change record
that names backward compatibility, the affected clients, and (for
a breaking change) a regression task per affected client; the
version must advance by exactly one. Reason: a silent contract
drift is how parallel client threads diverge; the protocol is the
only sanctioned path, so the check enforces the protocol instead
of merely documenting it.
2. Forbidden directories (``is_within`` / ``check_task_writes``).
Rule reuses the two directory definitions that already exist and
invents no third one: on the control plane a task's own directory
is its tenant work directory (``worker._tenant_dirs`` ``work``);
in a skill project a client thread's own directory is its
``apps/<client>/`` tree. Any write outside the own directory --
including ``..`` escapes and symlink escapes, both resolved before
comparison -- is a violation. Reason: cross-directory writes are
how one tenant/client corrupts another's worktree.
3. Forbidden imports (``check_layer_imports``). Bans were chosen by
auditing the current tree, not by wishful layering; every ban
below passes on the baseline, and the repo-level test pins that:
- Foundation modules (``db``, ``models``, ``queue``,
``logging_config``) must not import orchestration/service
modules (``server``, ``worker``, ``pr_exec``, ``webhooks``,
``autoscaler``, ``cost``). Reason: the foundation is imported
by everything; an upward import is a cycle waiting to happen
and drags service state into pure helpers.
- The execution module (``sandbox``) must not import persistence
or orchestration (``db``, ``server``, ``worker``, ``cost``,
``auth``, ``pr``, ``pr_exec``, ``webhooks``). Reason: executors
must stay swappable and testable with fake runners; decisions
about mounts and networks arrive as arguments, not by reading
the database.
- No cloud module except the driver-adjacent ones (``db``,
``backup``, ``migrations``) may open a database connection
directly (``sqlite3.connect`` or any ``psycopg*`` import).
Service modules must obtain connections through
``cloud.db.connect`` or a Store. Reason: the backend switch
(sqlite/postgres) lives in exactly one place; a direct driver
call bypasses it. Deliberately NOT banned: ``server.py``'s use
of ``sqlite3.Row`` / ``sqlite3.IntegrityError`` as type adapters
on connections obtained from ``cloud.db`` -- that is adaptation,
not a bypass, and banning it would fail the baseline for no
safety gain. The test asserts the narrow form (no ``connect``
call, no driver import) rather than the broad form.
Only static ``ast`` imports are seen; dynamic imports
(``importlib``) are out of scope and are noted, not claimed.
4. Frozen stack (``check_frozen_stack``). Rule mirrors the skill's
G2 rule "frozen stack must not be swapped mid-flight": a client
whose stack entry changes, or disappears, after freeze fails
unless an approved change record covers it; adding a brand-new
client (no frozen entry) is an addition, not a swap, and passes.
Reason: the shared core is built against the frozen table; a
mid-flight swap silently invalidates it.
All checkers return plain data (dicts/lists), never raise for a
violation, so CI can report every violation in one run instead of
stopping at the first.
"""
import ast
from pathlib import Path
FOUNDATION_MODULES = frozenset({"db", "models", "queue", "logging_config"})
ORCHESTRATION_MODULES = frozenset(
{"server", "worker", "pr_exec", "webhooks", "autoscaler", "cost"})
EXECUTION_MODULES = frozenset({"sandbox"})
EXECUTION_BANNED = frozenset(
{"db", "server", "worker", "cost", "auth", "pr", "pr_exec", "webhooks"})
DRIVER_ADJACENT_MODULES = frozenset({"db", "backup", "migrations"})
# ---------------------------------------------------------------- contract
def parse_version(text):
"""Parse ``v1`` / ``1`` to int; anything else is None (unparseable
is a failure, never silently treated as the frozen version)."""
if text is None:
return None
text = str(text).strip()
if text.startswith("v"):
text = text[1:]
if text.isdigit():
return int(text)
return None
def _change_record_problems(change, breaking):
"""Missing pieces of a contract-change record, per the protocol."""
problems = []
if not isinstance(change, dict):
return ["change record missing (frozen contract changed without "
"a coordinator-approved change record)"]
if not change.get("approved"):
problems.append("change not coordinator-approved")
if change.get("backward_compatible") is None:
problems.append("backward compatibility not declared")
affected = change.get("affected_clients")
if not affected or not isinstance(affected, list):
problems.append("affected clients not listed")
if breaking and not change.get("regression_task"):
problems.append("breaking change without a regression task")
return problems
def check_contract_version(frozen, current, change=None):
"""Return ``{"ok": bool, "reasons": [...]}``.
Equal versions pass regardless of ``change``. A different version
passes only when it is exactly frozen + 1 and ``change`` satisfies
the protocol; a breaking change is one where
``backward_compatible`` is False.
"""
frozen_n = parse_version(frozen)
current_n = parse_version(current)
if frozen_n is None or current_n is None:
return {"ok": False,
"reasons": ["unparseable contract version: frozen=%r "
"current=%r" % (frozen, current)]}
if current_n == frozen_n:
return {"ok": True, "reasons": []}
reasons = []
if current_n != frozen_n + 1:
reasons.append("contract version must advance by exactly one "
"(frozen v%d -> v%d, not v%d)"
% (frozen_n, frozen_n + 1, current_n))
breaking = isinstance(change, dict) and \
change.get("backward_compatible") is False
reasons.extend(_change_record_problems(change, breaking))
return {"ok": not reasons, "reasons": reasons}
# ------------------------------------------------------------- frozen stack
def check_frozen_stack(frozen, current, change=None):
"""``frozen``/``current`` map client -> stack string.
Changed or removed entries fail without an approved change record
(same shape as the contract-change record, minus the version
rules); added clients pass as additions.
"""
frozen = frozen or {}
current = current or {}
reasons = []
swapped = sorted(c for c in frozen
if c not in current or current[c] != frozen[c])
if not swapped:
return {"ok": True, "reasons": [], "swapped": []}
problems = _change_record_problems(change, breaking=False)
if problems:
reasons.append("frozen stack changed for %s without an approved "
"change record: %s" % (swapped, "; ".join(problems)))
return {"ok": not reasons, "reasons": reasons, "swapped": swapped}
# ------------------------------------------------------------ directories
def is_within(root, candidate):
"""True iff ``candidate`` resolves inside ``root`` (symlinks and
``..`` resolved first, so both escape routes are caught)."""
try:
root_p = Path(root).resolve()
cand_p = Path(candidate).resolve()
except (OSError, RuntimeError):
return False
return cand_p == root_p or root_p in cand_p.parents
def check_task_writes(own_dir, paths):
"""Return the subset of ``paths`` written outside ``own_dir``."""
return [p for p in paths if not is_within(own_dir, p)]
# ----------------------------------------------------------------- imports
def scan_imports(source):
"""Static import scan of one module's source.
Returns ``{"cloud": set, "driver_connect": bool, "driver_import": bool}``:
``cloud`` holds the ``cloud.<name>`` top-level names imported;
``driver_connect`` is True on any ``sqlite3.connect`` call;
``driver_import`` is True on any ``psycopg*`` import. A bare
``import sqlite3`` (Row/exception adapters) sets neither flag.
"""
cloud_imports = set()
driver_connect = False
driver_import = False
try:
tree = ast.parse(source)
except SyntaxError:
return {"cloud": cloud_imports,
"driver_connect": driver_connect,
"driver_import": driver_import,
"parse_error": True}
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
if alias.name == "sqlite3":
continue
if alias.name.startswith("psycopg"):
driver_import = True
if alias.name == "cloud" or alias.name.startswith("cloud."):
parts = alias.name.split(".")
if len(parts) > 1:
cloud_imports.add(parts[1])
elif isinstance(node, ast.ImportFrom):
if node.module and node.module.startswith("psycopg"):
driver_import = True
if node.module == "cloud" or (node.module or "").startswith("cloud."):
if node.module == "cloud":
for alias in node.names:
cloud_imports.add(alias.name)
else:
cloud_imports.add(node.module.split(".")[1])
elif isinstance(node, ast.Call):
func = node.func
if isinstance(func, ast.Attribute) and func.attr == "connect":
value = func.value
if isinstance(value, ast.Name) and value.id == "sqlite3":
driver_connect = True
return {"cloud": cloud_imports,
"driver_connect": driver_connect,
"driver_import": driver_import}
def check_layer_imports(sources):
"""``sources`` maps module name -> source text. Returns a list of
violation dicts; empty means the layer rules hold."""
violations = []
for module, source in sorted(sources.items()):
scanned = scan_imports(source)
if scanned.get("parse_error"):
violations.append({"module": module, "rule": "parse",
"detail": "source does not parse; "
"imports cannot be verified"})
continue
if module in FOUNDATION_MODULES:
bad = scanned["cloud"] & ORCHESTRATION_MODULES
if bad:
violations.append({"module": module,
"rule": "foundation_must_not_import_orchestration",
"detail": sorted(bad)})
if module in EXECUTION_MODULES:
bad = scanned["cloud"] & EXECUTION_BANNED
if bad:
violations.append({"module": module,
"rule": "execution_must_not_import_persistence_or_orchestration",
"detail": sorted(bad)})
if module not in DRIVER_ADJACENT_MODULES:
if scanned["driver_connect"] or scanned["driver_import"]:
violations.append({"module": module,
"rule": "service_must_not_open_driver_connection_directly",
"detail": "use cloud.db.connect or a Store"})
return violations
def scan_cloud_package(repo_root):
"""Run ``check_layer_imports`` over every ``cloud/*.py`` in a repo
checkout (tests and caches excluded). This is the repo-level test
that makes a layer violation fail CI."""
cloud_dir = Path(repo_root) / "cloud"
sources = {}
if cloud_dir.is_dir():
for path in sorted(cloud_dir.glob("*.py")):
sources[path.stem] = path.read_text(encoding="utf-8")
return check_layer_imports(sources)
def check_skill_project(project_root, frozen_version, current_version,
frozen_stack, current_stack, own_dir, writes,
change=None):
"""One-call conformance for a skill project: contract version,
frozen stack, and directory writes together. Returns a dict with
one entry per check plus an overall ``ok``."""
contract = check_contract_version(frozen_version, current_version,
change=change)
stack = check_frozen_stack(frozen_stack, current_stack, change=change)
outside = check_task_writes(own_dir, writes)
directory = {"ok": not outside,
"reasons": (["writes outside own directory: %s" % outside]
if outside else [])}
return {"ok": contract["ok"] and stack["ok"] and directory["ok"],
"contract": contract, "stack": stack, "directory": directory}
+366
View File
@@ -0,0 +1,366 @@
"""Lifecycle hook policy for the task-attempt ledger (audit gap 3
follow-up: "which dispatch counts as an attempt, and who judges
``is_defect``").
This module is the CANONICAL policy text. The same policy is
restated (shorter) in ``docs/arch-conformance-metrics.md`` and
pinned by ``cloud/tests/test_attempt_hook.py``; if the three ever
disagree, this module wins and the other two must be fixed.
The problem it solves: audit gap 3 built the ledger but
deliberately left the writer unconnected, because a naive writer
-- one row per worker execution -- misclassifies exactly twice:
it counts a platform auto-resume as an attempt, and it counts a
transient retry as a defect. This module fixes the writer's
trigger, not the ledger's math: ``aggregate_attempts`` is
untouched by the hook (its only change in this track is the
correction-row rule in ``attempt_ledger.py``), and the cost
dual-column code (``cloud.cost``) is untouched entirely.
POLICY (each rule is enforced in code below, not just prose):
1. One counted dispatch = one row, at most. A dispatch that the
hook counts writes EXACTLY ONE ledger row, when its outcome is
known (terminal ``completed`` / ``failed`` / ``interrupted``,
or the goal-conversion case in ``worker.run_one``), with
``is_defect=False`` always (rule 6). A dispatch that the hook
does not count writes NO row at all -- not even an
``interrupted`` row -- so an uncounted dispatch can never
create the ledger table as a side effect.
2. What counts as one attempt -- a logical agent round, not a
worker execution:
- First dispatch of a task: counts (``first_or_human_retry``).
It is the task's first agent round.
- Human ``/retry`` of a failed/interrupted/merge-failed task:
counts (same kind). A retry is a human-initiated new round:
new model work, new result, new cost. The server endpoint
leaves no marker behind, so the worker cannot distinguish a
retry from a first dispatch -- and does not need to, because
both count.
- Human ``/continue`` with a new message: counts
(``human_continue``). The message starts a new round on the
same session thread. Non-goal tasks and goal tasks opening a
NEW goal loop (``_resume_kind == "new-goal"``) are both here.
- Human breakpoint resume (``/goal-resume``, or ``/continue``
on an old-verify goal task, i.e. ``_resume_kind ==
"breakpoint"``): counts (``breakpoint_resume``) -- but as
the SAME logical round's row, not as an extra attempt. The
fragment that was interrupted wrote only an excluded
``interrupted`` row (gap 3 rule), so the resume's terminal
row is the round's single counted row. A resume that ends
interrupted again writes only another excluded row: the
round is still pending, and still contributes zero counted
attempts. Rationale: one goal loop = one attempt, however
many executions the loop took; the interruption history
stays visible in the ``interrupted`` rows and in
``auto_resume_count`` / audit / task_steps, not in the
attempt count. EXCEPTION (ordering, rule 4): when the
resume was itself platform-triggered, the platform marker
wins and the dispatch does NOT count -- a platform goal
auto-resume and a human goal-resume are the same loop
continuation, but only the human one is attributed to the
agent. That asymmetry is deliberate and uniform with the
platform rule below; flipping it is the listed user
decision, not a silent default.
- Coordinator parent dispatch: does NOT count
(``coordinator_orchestration``). The parent runs no model
round (``cost_record_id`` is None) and its terminal status
is an aggregate of its children; each child records its own
row through its own dispatch. Counting the parent would
double-count orchestration as an agent attempt.
- Platform auto-resume dispatch (worker
``auto_resume_interrupted`` requeue, goal or non-goal):
does NOT count (``platform_auto_resume``). This is the
default the gap-3 track demanded, and it is UNIFORM: every
platform requeue is excluded, whether it continues a goal
loop or fresh-runs a non-goal task. Rationale: the platform
killed the previous execution and re-ran it without any
human or coordinator decision; attributing that execution
to the agent would punish platform instability in an
agent-quality rate. TRADE-OFF, STATED NOT HIDDEN: a task
whose terminal outcome came ONLY from platform dispatches
stays unknown in the ledger (no counted row), so it is
excluded from rate denominators -- the rates describe
tasks whose terminal outcome came from a human/first
round. The platform reruns themselves are NOT lost: they
are recorded in ``tasks.auto_resume_count``, the
``worker_auto_resumed`` audit events, and the
``task_steps`` ``auto_resume`` rows. The rejected
alternative was a separately marked ledger row that is
excluded from denominators: same denominator outcome, but
a second row kind, a new column, and aggregate exclusion
logic, for co-location only. If a future policy wants
platform completions counted as loop completions, the flip
is one line in ``classify_dispatch`` (platform + breakpoint
-> counted) plus tests -- it is listed as a user decision
in the track summary, not silently decided here.
- Pre-execution gate failures: do NOT count. A dispatch that
never reached the model -- missing key, repository
resolution failure, quota gate, goal-rounds cap, gateway
preparation failure -- is an admission rejection, not an
attempt. It writes no row (and no cost record either, which
is the existing behaviour); it stays visible in audit
(``worker_missing_key`` / ``worker_repo_failed`` /
``worker_quota_stop`` / ``worker_goal_cap_rejected`` /
``worker_gateway_failed``).
- Dispatches that never ran (budget gate, no ready task,
claim lost to another worker): do NOT count and are not
even classified -- ``run_one`` returns before the hook
runs, and no marker is consumed (see rule 4).
3. Transient retries inside one dispatch are NOT new attempts.
The worker's ``max_retries`` loop, its one real fallback
re-run, its goal "already exists" breakpoint fallback, and
its follow-up drain all run inside a single ``run_one``
dispatch, share one cost aggregate, and produce ONE ledger
row: the dispatch's terminal outcome. An inner failure that
is later retried successfully leaves no failure row behind;
an inner failure that exhausts retries leaves one failure
row, not N. This is enforced by placement (the hook is
called once per dispatch, at ``run_one``'s terminal points)
rather than by counting executions.
4. Platform identification -- the marker mechanism. The worker
that requeues a task is not necessarily the worker that later
claims it, so the origin must survive in the database, not in
memory. ``auto_resume_interrupted`` writes one marker row
(``task_attempt_origin_markers``) in the SAME transaction as
the requeue, with ``queued_at`` equal to the ``updated_at``
the requeue writes. ``run_one`` captures the queued task's
``updated_at`` BEFORE claiming (claim overwrites it), and
after a successful claim consumes the marker ONLY if
``queued_at`` still matches. Any intervening human action
(cancel, retry, continue) rewrites ``updated_at``, so a stale
marker can never misclassify a later human round: on
mismatch the marker is deleted as stale and the dispatch is
classified by its own fields. A claim that loses the race
consumes nothing, so the winning worker still sees its
marker. Marker writes and reads happen ONLY when the hook is
enabled; with the hook off, ``auto_resume_interrupted``
writes no marker and ``run_one`` reads none, so off behaviour
is byte-identical to gap 3 (no new table, no new row).
Marker-table creation is a write-path-only step, like the
ledger table itself; the read path (``GET /metrics-report``)
never touches it.
5. Switch. ``Worker(attempt_ledger_hook=False)`` (CLI
``--attempt-ledger-hook``), default OFF. OFF means: no ledger
row, no marker row, no marker-table creation, no ledger-table
creation from the worker -- exactly gap 3's "explicit calls
only" behaviour, including old tasks still returning ``None``
(unknown, not 0) and the read endpoint never creating a
table. ON means rules 1-4 apply. Enabling the hook
mid-flight has one documented edge: a task requeued by
auto-resume WHILE the hook was off carries no marker, so its
next dispatch counts as a human/first round (one extra
counted row for that in-flight task, once). That is the
fail-safe direction (a human round is the common case), and
it self-heals: every later requeue writes a marker.
6. ``is_defect`` -- who judges, and how a wrong judgement is
fixed. The worker hook NEVER judges a defect: it always
records ``is_defect=False``. A failed dispatch is not a
defect (transient failures, quota stops, and cancelled rounds
all fail without being defects), and the worker cannot tell
a transient failure from a defect round at runtime -- so it
must not guess. The ONLY defect judgements in the ledger are
explicit ones, made at record time by an explicit recorder:
a human maintainer (or a coordinator review) calling
``record_attempt`` / ``record_correction`` directly with
``is_defect=True`` and a category. The bar for that authority
is maintainer-level (``approve_merge``, the same bar as
``/retry``: both spend model money and move decision-grade
rates). A defect judgement made in error is corrected ONLY by
appending a correction row (``record_correction``): the old
row is never updated or deleted, corrections apply in
attempt order with last-wins, and a correction can only
change the defect judgement, never the outcome. There is NO
HTTP endpoint for marking or correcting defects in this
track -- the function is the interface, and an endpoint (with
the ``approve_merge`` permission check) is follow-up work,
listed as unconnected in the track summary. Until it exists,
defect marks come only from explicit, reviewed calls, never
from automation.
7. Failure discipline. Every hook write is best-effort: a ledger
or marker failure is logged (``worker_attempt_ledger_failed``
/ ``worker_attempt_marker_failed``) and NEVER fails the task,
blocks a requeue, or changes a task's status. The one
exception is validation inside ``record_attempt`` /
``record_correction`` (bad outcome, defect on an interrupted
row, correcting a nonexistent attempt): those raise
``ValueError`` BEFORE writing, by design, so ambiguous data
is never stored -- but the worker hook only ever passes
validated values (a terminal outcome, ``is_defect=False``),
so it cannot hit that path in normal operation.
8. Postgres. The marker table follows the ledger table's manual
discipline: it is NOT created by ``init_postgres_schema``,
and ``PG_MARKER_DDL`` below is the exact statement to run by
hand on production Postgres (and verify) before enabling the
hook against a Postgres-backed control plane. Until that
step runs, marker writes fail (best-effort, rule 7) and
platform dispatches degrade to counted -- the same
fail-safe direction as rule 5's mid-flight edge, and the
reason the PG manual step is a precondition for enabling
the hook in production, not an afterthought. Real Postgres
verification was NOT done on this track (no local Postgres):
PG evidence here is adapter-level only (``?`` placeholders
translate to ``%s`` through ``cloud.db``, DDL uses only
Postgres-valid types, no autoincrement/identity column),
exactly like gap 3's PG evidence, and labelled as such.
Integration v2 out-of-box closure (see
``cloud/ledger_onboarding.py``): the DDL checklist, the
explicit enable path and the read-only
``verify_ledger_ready`` / ``ledger_setup_report`` probe
close the "hook off by default + manual PG DDL" gap
WITHOUT changing the default (still OFF) or auto-running
DDL on production Postgres -- the manual PG step remains
an operator action, now checkable instead of assumed.
This module imports ``cloud.attempt_ledger`` only, opens no
database connection itself (callers pass a connection in), and
contains no ``sqlite3.connect`` / ``psycopg*`` import, so the
``arch_conformance`` layer rules are untouched.
"""
from cloud.attempt_ledger import record_attempt
MARKER_TABLE = "task_attempt_origin_markers"
# One DDL serves both backends: TEXT / DOUBLE PRECISION / composite
# primary key, no AUTOINCREMENT or identity column -- the same
# discipline as the ledger DDL. ``queued_at`` is the ``updated_at``
# the requeue wrote; it is the generation guard, not a timestamp
# for display.
MARKER_DDL = (
"CREATE TABLE IF NOT EXISTS task_attempt_origin_markers("
"task_id TEXT NOT NULL PRIMARY KEY, "
"origin TEXT NOT NULL, "
"queued_at DOUBLE PRECISION NOT NULL)"
)
# Production Postgres manual step (see rule 8). Intentionally NOT
# part of POSTGRES_SCHEMA, same reason as the ledger PG_DDL.
PG_MARKER_DDL = MARKER_DDL
PLATFORM_ORIGIN = "platform_auto_resume"
# Decision kinds returned by classify_dispatch. ``counted`` is the
# only field the worker acts on; ``kind`` is the audit label.
KINDS = frozenset({
"coordinator_orchestration",
"platform_auto_resume",
"breakpoint_resume",
"human_continue",
"first_or_human_retry",
})
def classify_dispatch(task, platform_marker):
"""Pure policy: task fields + marker -> decision dict.
``task`` is the dispatch's task dict (post-claim, post
breakpoint resolution: ``_goal_resume_breakpoint`` is True
only when a breakpoint resume actually applies, False when
resolution fell back to a fresh goal run). ``platform_marker``
is True when this dispatch consumed a platform auto-resume
marker for the queued generation (rule 4), else False.
Returns ``{"counted": bool, "kind": str}``. Order matters
and is part of the policy: coordinator fan-out never counts
even with a marker; a platform marker beats every other
signal (uniform platform exclusion, rule 2); a breakpoint
resume beats a continue message (a stale resume_message on
a breakpoint task is not a new round); anything else with a
resume message is a human continue; the rest is a first
dispatch or a human retry, which count identically.
"""
task = task or {}
if task.get("coordinator_parts"):
return {"counted": False,
"kind": "coordinator_orchestration"}
if platform_marker:
return {"counted": False, "kind": "platform_auto_resume"}
if task.get("_goal_resume_breakpoint") is True or (
task.get("_resume_kind") == "breakpoint"):
return {"counted": True, "kind": "breakpoint_resume"}
if task.get("resume_message"):
return {"counted": True, "kind": "human_continue"}
return {"counted": True, "kind": "first_or_human_retry"}
def ensure_marker_table(conn):
"""Create the marker table if absent. Idempotent; the caller
owns the commit (``mark_platform_rerun`` commits via the
caller's requeue transaction in the worker; a bare ensure
inside a larger transaction does not commit)."""
conn.execute(MARKER_DDL)
def mark_platform_rerun(conn, task_id, queued_at):
"""Record that ``task_id`` was requeued by platform auto-resume
at ``queued_at`` (the requeue's ``updated_at``). Overwrites
any marker for the task: a requeue supersedes any older
generation. Callers must call this inside the requeue
transaction (the worker does: same connection, same commit
as the status change), so a marker can never exist without
its queued state, or vice versa. Best-effort at the call
site: on failure the caller logs and proceeds with the
requeue (rule 7) -- a missing marker degrades to counted,
never to a blocked requeue."""
if not task_id:
raise ValueError("task_id is required")
ensure_marker_table(conn)
conn.execute(
"DELETE FROM task_attempt_origin_markers WHERE task_id = ?",
(task_id,),
)
conn.execute(
"INSERT INTO task_attempt_origin_markers(task_id, origin, "
"queued_at) VALUES(?, ?, ?)",
(task_id, PLATFORM_ORIGIN, queued_at),
)
def consume_platform_marker(conn, task_id, queued_at):
"""Consume the platform marker for ``task_id`` iff its
``queued_at`` still matches this dispatch's queued
``updated_at``. Returns True on a match (this dispatch IS
the platform rerun). On mismatch the marker belongs to an
older generation (a human action rewrote ``updated_at``
since) and is deleted as stale; returns False. When
``queued_at`` is None the generation cannot be verified, so
nothing is consumed or deleted and False is returned (the
dispatch then counts -- the rule-5 fail-safe direction).
A missing marker table raises the backend's OperationalError
(loud, not silent): the worker call site treats that as
"no marker" and logs it, it never fails the dispatch."""
if not task_id or queued_at is None:
return False
cur = conn.execute(
"DELETE FROM task_attempt_origin_markers "
"WHERE task_id = ? AND queued_at = ?",
(task_id, queued_at),
)
if cur.rowcount == 1:
return True
conn.execute(
"DELETE FROM task_attempt_origin_markers WHERE task_id = ?",
(task_id,),
)
return False
def record_dispatch_result(conn, task_id, outcome):
"""Write the ledger row for one COUNTED dispatch. The worker
calls this only when ``classify_dispatch`` said counted, and
only with a terminal outcome (``success`` / ``failure`` /
``interrupted``). ``is_defect`` is always False here (rule
6): the worker never judges a defect. Returns the new
``attempt_no``. The counted filter lives in the caller
(``classify_dispatch``); this function takes the outcome
only, so a miscounted call site cannot smuggle a defect
judgement in through here either."""
return record_attempt(conn, task_id, outcome, is_defect=False)
+395
View File
@@ -0,0 +1,395 @@
"""Task-attempt ledger for the step-5 metrics loop (audit gap 3).
Choice made here, with the trade-off stated rather than hidden:
- This module adds a SEPARATE ``task_attempts`` table; it does NOT
add ``attempts``/``defects`` columns to ``tasks``. Columns would be
simpler to query (one row per task, no join), but they collapse
history into two counters: a task's per-attempt story (which
attempt failed, with what defect category, in what order) would be
unrecoverable, and widening the hot ``tasks`` table on production
Postgres is a heavier migration than adding a new, empty table.
The ledger costs one aggregate query at report time, and that
query is exactly what ``aggregate_attempts`` exists to pin down.
- The ledger is append-only in intent: ``record_attempt`` only ever
inserts a new ``(task_id, attempt_no)`` row (``attempt_no`` is
``MAX + 1`` per task, never reused). There is no update path, so a
recorded attempt cannot be silently rewritten into a first-pass
success after the fact.
Semantics (pinned by tests, not left to each caller):
- One row = one real attempt at the task. ``outcome`` is one of
``success`` / ``failure`` / ``interrupted``.
- ``interrupted`` rows are kept for audit but NEVER count toward
``attempts`` or ``defects``: a platform interruption that
auto-resumes as a fresh run is a platform event, not an agent
failure, and the step-5 rule that ``interrupted`` is not a
terminal task status would be meaningless if the interruption
still inflated the attempt count. A task whose ledger holds only
interrupted rows has NO counted attempt, so it is omitted from
the aggregate entirely -- the endpoint then still reports its
rates as unknown (``None``), never as 0.
- ``is_defect`` is set explicitly by the recorder, never inferred
from ``outcome``: a failed attempt is not automatically a defect
(a retry can fail for a transient reason without being a
defect round), and a defect round is a judgement the recording
caller makes. ``defect_category`` requires ``is_defect``; an
interrupted row can never be a defect. Both contradictions raise
``ValueError`` instead of being stored as ambiguous data.
- ``defect_category`` for a task (the single category that
``compute_metrics`` carries per task) is the category of its
FIRST defect row in attempt order. Later defect rows still count
toward ``defects``; only the category label is first-wins, so it
is deterministic. Corrections (below) re-evaluate this label:
the first defect row AFTER corrections is applied wins.
- A defect judgement can be CORRECTED, never rewritten:
``record_correction`` appends a new row whose
``corrects_attempt_no`` points at the attempt it corrects; the
old row stays untouched, and aggregation applies corrections in
attempt order with last-wins (so a wrongly marked defect can be
unmarked, and a defect missed at record time can be added after
review). A correction row is never itself counted as an
attempt, cannot change an attempt's outcome (only its defect
judgement), and cannot target an interrupted row, another
correction row, or a nonexistent attempt -- all raise
``ValueError`` and write nothing. Who may correct is a policy
question answered in ``cloud/attempt_hook.py`` (explicit
recorder only, maintainer bar); the worker hook never writes
corrections and never marks a defect.
- Old tasks have no ledger rows, and that is NOT backfilled: the
aggregate simply has no entry for them, the endpoint leaves their
``attempts``/``defects`` as ``None``, and ``compute_metrics``
excludes them from rate denominators (they are neither counted
as first-pass nor as defect-free). Fabricating a row for an old
task (e.g. ``attempts=1, defects=0`` from "it completed") would
be inventing history.
Persistence notes:
- SQLite: ``ensure_task_attempts_table`` runs
``CREATE TABLE IF NOT EXISTS`` -- idempotent, safe to call on
every write, and a no-op on a database that already has the
table (including one that already has rows).
- Postgres: the production table is NOT created by
``init_postgres_schema`` (deliberately: that schema is covered by
the migrations test asserting ``MIGRATION_TABLES`` equals the
schema set, and this ledger must not silently widen that set).
``PG_DDL`` below is the exact statement to run manually on
production Postgres and verify (``\\d task_attempts`` or the
``information_schema`` equivalent) before any Postgres-backed
endpoint is expected to return real rates. The DDL uses only
Postgres-valid types (``TEXT``/``INTEGER``/``DOUBLE PRECISION``),
a composite primary key, and no ``AUTOINCREMENT``/identity
column, so it needs no sequence handling; the parameterized
``INSERT`` uses ``?`` placeholders, which the ``cloud.db``
Postgres adapter translates to ``%s`` (see the adapter test in
``test_attempt_ledger.py``). Until that manual step runs, the
endpoint treats a missing table exactly like "no ledger rows":
unknown rates, not an error -- the same fail-soft shape already
used for missing ``cost_records``.
- The read path (``GET /metrics-report``) never creates the table:
a read-only endpoint must not mutate schema. Table creation
happens only on the write path, inside ``record_attempt`` /
``record_correction``.
- Lifecycle hook (this track, see ``cloud/attempt_hook.py``):
the worker writes exactly one row per COUNTED dispatch (see
that module's policy for which dispatches count: human/first
rounds and human breakpoint resumes count; platform auto-resume
reruns, coordinator fan-out, and pre-execution gate failures do
not). The hook is OFF by default (``attempt_ledger_hook=False``):
with it off, this module's behaviour is exactly audit gap 3 --
the ledger fills only through explicit ``record_attempt`` /
``record_correction`` calls, and production rates stay honestly
unknown until the hook is deliberately enabled. The worker hook
always records ``is_defect=False``: it never judges a defect,
so enabling it can never turn a transient retry into a defect.
- Schema evolution note (pre-production, this track):
``corrects_attempt_no`` was added to the DDL after audit gap 3
but BEFORE either DDL reached production (gap 3 is still an
OPEN, unmerged PR), so no production table needs migration:
fresh tables get the column from the DDL below. A database
created from gap 3's DDL alone (temporary/test databases only)
gains the column with one statement --
``ALTER TABLE task_attempts ADD COLUMN corrects_attempt_no
INTEGER`` -- which is also the documented step if gap 3's DDL
was ever run on production Postgres before this track lands.
The read path uses ``SELECT *`` deliberately so a table without
the column still reads (its rows aggregate as plain attempts).
This module imports no ``cloud.*`` module and opens no database
connection itself: callers pass a connection in, so the layer rules
(``arch_conformance``: no direct driver connection outside
``cloud.db``) are untouched -- there is no ``sqlite3.connect`` and
no ``psycopg*`` import here at all.
"""
import time
TABLE = "task_attempts"
OUTCOMES = frozenset({"success", "failure", "interrupted"})
# One DDL serves both backends: every type and the composite key are
# valid in SQLite and in Postgres, and there is no AUTOINCREMENT or
# identity column for either backend's sequence handling to chase.
# ``corrects_attempt_no`` (NULL = a real attempt row; non-NULL = a
# correction row for that attempt_no, never counted as an attempt)
# was added by the lifecycle-hook track; it SUPERSEDES gap 3's DDL
# for any production manual step (gap 3 never merged, so no
# production table exists yet -- see the module docstring).
SQLITE_DDL = (
"CREATE TABLE IF NOT EXISTS task_attempts("
"task_id TEXT NOT NULL, "
"attempt_no INTEGER NOT NULL, "
"outcome TEXT NOT NULL, "
"is_defect INTEGER NOT NULL DEFAULT 0, "
"defect_category TEXT, "
"created_at DOUBLE PRECISION, "
"corrects_attempt_no INTEGER, "
"PRIMARY KEY (task_id, attempt_no))"
)
# Production Postgres: run this verbatim by hand and verify the
# table exists before expecting real rates from a Postgres-backed
# endpoint. It is intentionally NOT part of POSTGRES_SCHEMA (see
# the module docstring: migrations coverage must not widen silently).
# This is the NEW DDL (with ``corrects_attempt_no``): if gap 3's
# older DDL was ever run first, follow it with
# ``ALTER TABLE task_attempts ADD COLUMN corrects_attempt_no
# INTEGER`` and verify before expecting corrections to work.
PG_DDL = SQLITE_DDL
# One-line upgrade for a table created from gap 3's DDL (no
# ``corrects_attempt_no`` column yet). Valid in both backends.
OLD_TABLE_ALTER = (
"ALTER TABLE task_attempts "
"ADD COLUMN corrects_attempt_no INTEGER"
)
def ensure_task_attempts_table(conn):
"""Create the ledger table if absent. Idempotent; does not
touch existing rows. The caller owns the commit (``record_attempt``
commits; a bare ensure inside a larger transaction does not)."""
conn.execute(SQLITE_DDL)
def record_attempt(conn, task_id, outcome, is_defect=False,
defect_category=None, created_at=None):
"""Append one attempt row for ``task_id``. Returns ``attempt_no``.
Validates before writing: unknown outcome, a defect flag on an
interrupted row, and a defect category without a defect flag
all raise ``ValueError`` and write nothing.
"""
if not task_id:
raise ValueError("task_id is required")
if outcome not in OUTCOMES:
raise ValueError("outcome must be one of %s, got %r"
% (sorted(OUTCOMES), outcome))
if outcome == "interrupted" and is_defect:
raise ValueError("an interrupted attempt is never a defect")
if defect_category is not None and not is_defect:
raise ValueError("defect_category requires is_defect=True")
ensure_task_attempts_table(conn)
row = conn.execute(
"SELECT COALESCE(MAX(attempt_no), 0) FROM task_attempts "
"WHERE task_id = ?",
(task_id,),
).fetchone()
attempt_no = int(row[0]) + 1
conn.execute(
"INSERT INTO task_attempts(task_id, attempt_no, outcome, "
"is_defect, defect_category, created_at, "
"corrects_attempt_no) "
"VALUES(?, ?, ?, ?, ?, ?, NULL)",
(task_id, attempt_no, outcome, 1 if is_defect else 0,
defect_category,
time.time() if created_at is None else created_at),
)
conn.commit()
return attempt_no
def record_correction(conn, task_id, attempt_no, is_defect,
defect_category=None, created_at=None):
"""Append a CORRECTION row for an already-recorded attempt.
The correction does not rewrite history: the target row stays
exactly as recorded, and this new row (its own ``MAX + 1``
``attempt_no``) carries the corrected defect judgement in
``is_defect`` / ``defect_category`` plus
``corrects_attempt_no = attempt_no``. Aggregation applies
corrections in attempt order, last-wins, and never counts a
correction row as an attempt.
Only the defect judgement is correctable: the correction row
copies the target's outcome, so it cannot turn a success into
a failure or vice versa. Validates before writing: missing
``task_id``, a nonexistent target, a target that is itself a
correction row, a target whose outcome is ``interrupted``
(interrupted rows are never defects, so there is no defect
judgement to correct), and a defect category without a defect
flag all raise ``ValueError`` and write nothing. On a table
created from gap 3's DDL (no ``corrects_attempt_no`` column),
the INSERT raises the backend's OperationalError loudly --
run ``OLD_TABLE_ALTER`` first (see the module docstring);
a correction must never silently degrade into a plain row.
Returns the correction row's own ``attempt_no``.
"""
if not task_id:
raise ValueError("task_id is required")
if defect_category is not None and not is_defect:
raise ValueError("defect_category requires is_defect=True")
ensure_task_attempts_table(conn)
target = conn.execute(
"SELECT outcome, corrects_attempt_no FROM task_attempts "
"WHERE task_id = ? AND attempt_no = ?",
(task_id, attempt_no),
).fetchone()
if target is None:
raise ValueError(
"no such attempt to correct: %s #%s" % (task_id, attempt_no))
if target["corrects_attempt_no"] is not None:
raise ValueError(
"cannot correct a correction row: %s #%s"
% (task_id, attempt_no))
if target["outcome"] == "interrupted":
raise ValueError(
"an interrupted attempt has no defect judgement to "
"correct: %s #%s" % (task_id, attempt_no))
row = conn.execute(
"SELECT COALESCE(MAX(attempt_no), 0) FROM task_attempts "
"WHERE task_id = ?",
(task_id,),
).fetchone()
correction_no = int(row[0]) + 1
conn.execute(
"INSERT INTO task_attempts(task_id, attempt_no, outcome, "
"is_defect, defect_category, created_at, "
"corrects_attempt_no) "
"VALUES(?, ?, ?, ?, ?, ?, ?)",
(task_id, correction_no, target["outcome"],
1 if is_defect else 0, defect_category,
time.time() if created_at is None else created_at,
attempt_no),
)
conn.commit()
return correction_no
def read_attempt_rows(conn):
"""All ledger rows in attempt order. Raises the backend's
OperationalError when the table does not exist yet; the
endpoint caller treats that as "no ledger", not as a failure.
``SELECT *`` is deliberate: a table created from gap 3's DDL
has no ``corrects_attempt_no`` column, and naming that column
here would turn those readable rows into an endpoint error.
With ``*`` the rows still read, and ``aggregate_attempts``
treats a missing ``corrects_attempt_no`` as a plain attempt
row (see ``_field_optional``).
"""
return conn.execute(
"SELECT * FROM task_attempts "
"ORDER BY task_id, attempt_no").fetchall()
def _field(row, name):
# sqlite3.Row, dict, and the Postgres adapter's rows all support
# key access; a plain tuple row is not accepted (callers pass
# rows, not raw tuples, so a wrong shape fails loudly here).
return row[name]
def _field_optional(row, name):
"""Like ``_field`` but missing keys mean "absent", not an
error. Used ONLY for ``corrects_attempt_no``: rows built by
gap-3-era readers (plain dicts, or tables without the column)
carry no such key, and they are plain attempt rows by
definition -- failing loudly there would break every gap-3
aggregate test for no benefit."""
try:
return row[name]
except (KeyError, IndexError, TypeError):
return None
def aggregate_attempts(rows):
"""Pure aggregation: ledger rows -> ``{task_id: record}`` where
``record`` is the ``attempts``/``defects``/``defect_category``
dict that ``compute_metrics`` consumes.
Interrupted rows are excluded from both counts, and a task
with only interrupted rows is omitted entirely (no counted
attempt => its rates stay unknown, never 0). Correction rows
(``corrects_attempt_no`` set) are excluded from the attempt
count and instead OVERWRITE the defect judgement of their
target attempt, in attempt order, last-wins: a task's
``defects`` is the number of its counted attempts whose
judgement AFTER corrections is a defect, and its
``defect_category`` is the category of its FIRST defect
attempt in attempt order AFTER corrections.
"""
per_task = {}
ordered = sorted(
rows,
key=lambda r: (_field(r, "task_id"), _field(r, "attempt_no")))
# Per task: attempt_no -> [is_defect, defect_category], in
# attempt order (insertion order = attempt_no order, since the
# input was sorted). Corrections mutate the target entry; they
# never create an entry of their own.
judged = {}
for row in ordered:
task_id = _field(row, "task_id")
corrects = _field_optional(row, "corrects_attempt_no")
if corrects is not None:
entries = judged.get(task_id)
if entries is not None and corrects in entries:
entries[corrects] = [
bool(_field(row, "is_defect")),
_field(row, "defect_category")]
continue
if _field(row, "outcome") == "interrupted":
continue
entries = judged.setdefault(task_id, {})
entries[_field(row, "attempt_no")] = [
bool(_field(row, "is_defect")),
_field(row, "defect_category")]
for task_id, entries in judged.items():
attempts = len(entries)
if attempts == 0:
continue
defects = 0
defect_category = None
for _no in sorted(entries):
is_defect, category = entries[_no]
if is_defect:
defects += 1
if defect_category is None:
defect_category = category
per_task[task_id] = {
"attempts": attempts, "defects": defects,
"defect_category": defect_category}
return per_task
def apply_to_records(records, aggregates):
"""Return a new record list with ledger values merged in.
Records for tasks with no aggregate entry keep whatever
``attempts``/``defects`` they already carried (the endpoint
passes ``None`` there, i.e. old tasks stay unknown). Input
records are not modified.
"""
merged = []
for record in records:
out = dict(record)
agg = aggregates.get(record.get("id"))
if agg is not None:
out["attempts"] = agg["attempts"]
out["defects"] = agg["defects"]
out["defect_category"] = agg["defect_category"]
merged.append(out)
return merged
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+385
View File
@@ -0,0 +1,385 @@
"""Audit-of-audit sampling: human review of audit verdicts.
Zero third-party dependencies; standard library only, no ``cloud.*``
imports. All functions are pure: they return plans, verdicts and
records, never auto-apply a correction -- applying a correction is a
separate, recorded step (see ``handle_sampling_result``).
This module implements the sampling half of the "audit itself must
be audited" rule. It does NOT change the audit verdicts, the
pushback/rebuild counters, or the golden-set approval flow; it only
defines which audit cases a human must re-examine, and what happens
when that re-examination finds the audit wrong.
UNIFIED SAMPLING RULE (single rule; this module is canonical for
selection, ``cloud/audit_pushback.py::select_audit_sample`` is only
its record-shaped adapter -- there is no second rule):
The scope of "mandatory" comes from track 3 (#244): golden-task
records and rebuild / refreeze / escalation records. The drawing
mechanism comes from track 5 (#238): seeded random draws with ceil
rounding. Track 3's old deterministic "every 10th record" rule is
superseded and removed: a batch of fewer than 10 ordinary records
drew zero under it, and a position-known draw is predictable. The
rates below are TEST DEFAULTS (``DEFAULT_*``), not approved
production values; the production rates are still a user decision.
Mandatory 1.0 is a rule, not a tunable rate.
Sampling rules (all testable, none left to "regular review"):
1. Mandatory sampling (ratio 1.0, never skipped). A case is
mandatory when ANY of these triggers holds (trigger order for
display and for the legacy single-trigger helper is golden,
rebuild, third-failure, refreeze, escalate):
- ``golden_task``: the case's task is in the human-approved
golden set (``task_id`` when present, else ``id``). A golden
task's records are mandatory whatever their kind -- an
ordinary pass verdict on a golden task is sampled, not just
its failures.
- ``rebuild``: any case whose audit decision was rebuild-from-
scratch (推倒重做). A rebuild destroys the old implementation
in the workspace, so its verdict must always be re-examined.
- ``third_failure_near_complete_still_failed``: a case whose
third failure was root-caused as near-complete (接近完成),
was given one targeted continuation round, and still failed.
This is the exact boundary where the "one more round vs
rebuild" judgement is most error-prone, so it is always sampled.
- ``refreeze``: a contract/design re-freeze record.
- ``escalate``: an escalation record.
A near-complete third failure whose continuation round SUCCEEDED
is not mandatory (the judgement proved itself), and a
near-complete first/second failure is not mandatory either (the
threshold is the third failure, per the rebuild rule). A
``blocked_quota`` record is NOT mandatory under this rule (it
is not in track 3's mandatory kind list); it is sampled only
through the random tiers below.
2. Random sampling of the rest, with the seed recorded:
- normal audit cases: default ratio 0.10;
- critical-path cases (``critical_path=True`` and NOT already
mandatory): default ratio 0.50 -- critical paths get a higher
rate, not a free pass. (Track 3's old ``critical_path``
reason label for mandatory kinds is retired: mandatory kinds
now carry their own trigger names, and ``critical_path``
means only this 0.50 random tier.)
Mandatory cases are removed from the random pools BEFORE
drawing, so they consume no random quota. The seed is part of
the plan: a sample selected without a recorded seed is not
reproducible and therefore not auditable. Selection is
deterministic given (cases, ratios, seed).
4. One pass, no double counting. Selection partitions every case
exactly once: mandatory (all its triggers listed, one sample,
one sampling entry) or random-critical or random-normal. A
record that is both a golden task AND a rebuild is sampled
ONCE, with ``triggers == ["golden_task", "rebuild"]``; its
correction, if any, is a single record whose retrigger follows
the (original, corrected) verdict pair, not the trigger list --
triggers decide selection only, never the correction action.
Two different records (e.g. a triage record and the later
rebuild record for the same task) are two samples, not a
double count: deduplication is per record, not per task.
5. Golden-set construction. The caller-supplied golden id set
must be built from human-approved, non-sample tasks only:
``build_golden_task_ids`` applies both vocabularies -- a task
counts only with ``human_approved_by`` set AND none of
``is_example``, ``sample=True``, or an id starting with
``sample-``. Track 3's per-task approver rule and track 5's
sample-exclusion rule are both preserved; neither alone is
sufficient.
6. Correction gate effect (shared with track 3 / wiring #247).
A sampling correction NEVER unblocks a merge by itself: a
correction to ``pass`` (a wrong block) schedules a re-audit
and the gate stays blocked until a new passing re-audit
conclusion exists; a correction to ``fail`` (a wrong pass)
re-triggers pushback and re-blocks at once. Vocabulary
synonyms for the same pair actions: track 3
``retrigger_pushback_and_reblock`` == track 5
``reopen_audit_and_pushback`` (wrong pass), and track 3
``schedule_reaudit`` == track 5
``cancel_pushback_and_rerun_audit`` (wrong block, where
"cancel" means cancel-and-rerun, never cancel-and-merge).
3. Golden-set approval is separate from sampling and is NOT
satisfied by sample files: ``filter_golden_tasks`` drops any task
flagged as a sample (``sample=True`` or an id starting with
``sample-``) and reports how many were dropped; a golden set
counts only its remaining, human-approved tasks. The approval
checklist (``golden_approval_checklist``) lists every item that
must be human-completed before a golden set is approved; until
then every item reads ``pending_human``.
Handling a sampling result (``handle_sampling_result``):
- Sampling says the audit was CORRECT: no correction, only a record
entry (the audit stands, with evidence it was re-examined).
- Sampling says the audit was WRONG: the original conclusion is
corrected in the record, and the corresponding flow is
re-triggered:
- wrongly issued pushback -> ``cancel_pushback_and_rerun_audit``
(cancel the pushback, restore the pre-pushback state, rerun);
- wrongly issued rebuild -> ``halt_rebuild_and_restore``
(halt the rebuild, restore the original implementation from
git history/tag, rerun -- the old implementation is preserved
by the rebuild rule itself, so restoration is possible);
- wrongly issued pass (a failure the audit missed) ->
``reopen_audit_and_pushback`` (reopen with the corrected
verdict and run the pushback flow the audit should have run);
- unknown original decision -> ``escalate_to_human`` (never
invent a retrigger).
Every outcome, correct or corrected, is appended to the
audit-of-audit record (``append_audit_record``); a sampling result
that is not recorded is treated as not having happened.
"""
import math
import random
DEFAULT_NORMAL_RATIO = 0.10
DEFAULT_CRITICAL_PATH_RATIO = 0.50
# Original audit decisions -> retrigger when sampling proves them wrong.
_RETRIGGER = {
"fail_pushback": "cancel_pushback_and_rerun_audit",
"rebuild": "halt_rebuild_and_restore",
"pass": "reopen_audit_and_pushback",
}
def mandatory_triggers(case, golden_task_ids=None):
"""Return ALL mandatory trigger names for ``case`` (possibly []).
Trigger order: golden_task, rebuild,
third_failure_near_complete_still_failed, refreeze, escalate.
``case`` fields used: ``task_id`` (else ``id``) for the golden
check, ``decision`` / ``kind`` for rebuild / refreeze /
escalate, ``failure_count``, ``root_cause``
(``"structural"`` / ``"near_complete"`` / other),
``continuation_failed`` (bool).
"""
triggers = []
golden = set(golden_task_ids or [])
task_id = case.get("task_id") if case.get("task_id") is not None \
else case.get("id")
if task_id is not None and task_id in golden:
triggers.append("golden_task")
if case.get("decision") == "rebuild" or case.get("kind") == "rebuild":
triggers.append("rebuild")
if (case.get("failure_count", 0) >= 3
and case.get("root_cause") == "near_complete"
and case.get("continuation_failed")):
triggers.append("third_failure_near_complete_still_failed")
if case.get("decision") == "refreeze" or case.get("kind") == "refreeze":
triggers.append("refreeze")
if case.get("decision") == "escalate" or case.get("kind") == "escalate":
triggers.append("escalate")
return triggers
def mandatory_trigger(case, golden_task_ids=None):
"""Return the first mandatory trigger name for ``case``, or None.
Legacy single-trigger view of ``mandatory_triggers``; new code
must use ``mandatory_triggers`` so a multi-trigger case is not
silently reported under one reason only.
"""
triggers = mandatory_triggers(case, golden_task_ids)
return triggers[0] if triggers else None
def build_golden_task_ids(tasks):
"""Build the golden task id list from candidate tasks.
A task counts only when it has a human approver
(``human_approved_by``) AND is not a sample/example under any
vocabulary: ``is_example``, ``sample=True``, or an id starting
with ``sample-``. Returns ids in input order, samples dropped
silently here (count them with ``filter_golden_tasks`` when a
dropped count must be reported).
"""
ids = []
for task in tasks or []:
if not isinstance(task, dict):
continue
task_id = task.get("id", "")
if task.get("is_example") or task.get("sample"):
continue
if str(task_id).startswith("sample-"):
continue
if not task.get("human_approved_by"):
continue
ids.append(task_id)
return ids
def select_sample(cases, seed, normal_ratio=DEFAULT_NORMAL_RATIO,
critical_ratio=DEFAULT_CRITICAL_PATH_RATIO,
golden_task_ids=None):
"""Select the audit sample. Returns a plan dict.
Single pass, no double counting: mandatory cases (all their
triggers listed) are always included and removed from the
random pools before drawing. The remaining cases are grouped
into critical-path and normal, each shuffled with
``random.Random(seed + ":" + group)`` and truncated to
``ceil(ratio * group_size)`` (a group of 1 with any positive
ratio therefore contributes 1 -- a small group is never rounded
down to zero and silently unsampled). ``seed`` is required: an
unseeded selection cannot be reproduced on review.
``golden_task_ids`` is the human-approved golden set (see
``build_golden_task_ids``); without it no case is golden.
"""
if seed is None or seed == "":
raise ValueError("sampling seed is required: an unseeded "
"sample cannot be reproduced on review")
mandatory = []
critical = []
normal = []
for case in cases:
triggers = mandatory_triggers(case, golden_task_ids)
if triggers:
mandatory.append((case, triggers))
elif case.get("critical_path"):
critical.append(case)
else:
normal.append(case)
def _draw(group, ratio, label):
shuffled = list(group)
random.Random("%s:%s" % (seed, label)).shuffle(shuffled)
if not shuffled or ratio <= 0:
return []
return shuffled[:math.ceil(ratio * len(shuffled))]
sampled_critical = _draw(critical, critical_ratio, "critical")
sampled_normal = _draw(normal, normal_ratio, "normal")
return {
"seed": seed,
"ratios": {"normal": normal_ratio,
"critical_path": critical_ratio,
"mandatory": 1.0},
"golden_task_ids": sorted(golden_task_ids or []),
"mandatory": [{"case_id": c.get("id"), "trigger": t[0],
"triggers": list(t)}
for c, t in mandatory],
"random_critical_path": [c.get("id") for c in sampled_critical],
"random_normal": [c.get("id") for c in sampled_normal],
"sampled_ids": ([c.get("id") for c, _ in mandatory]
+ [c.get("id") for c in sampled_critical]
+ [c.get("id") for c in sampled_normal]),
}
def filter_golden_tasks(tasks):
"""Drop sample entries from a golden-set candidate list.
A task is a sample (and never counts as golden) when it is
flagged ``sample=True`` or its id starts with ``sample-``.
Returns ``{"golden": [...], "dropped_samples": int}``.
"""
golden = []
dropped = 0
for task in tasks:
task_id = task.get("id", "") if isinstance(task, dict) else ""
is_sample = (isinstance(task, dict) and task.get("sample")) \
or str(task_id).startswith("sample-")
if is_sample:
dropped += 1
else:
golden.append(task)
return {"golden": golden, "dropped_samples": dropped}
def golden_approval_checklist(tasks, approver=None, approved_at=None,
list_reviewed=False,
criteria_defined=False):
"""Approval checklist for a golden set. Every item is
``pending_human`` until a human completes it (via the
``list_reviewed`` / ``criteria_defined`` confirmations, plus
approver + timestamp); the set is approved only when every item
is done AND at least one non-sample golden task remains. Sample
files never satisfy an item (see ``filter_golden_tasks``)."""
filtered = filter_golden_tasks(tasks)
items = [
{"item": "task_list_reviewed",
"status": "done" if list_reviewed else "pending_human",
"note": "a human reviewed the golden task list"},
{"item": "acceptance_criteria_defined",
"status": "done" if criteria_defined else "pending_human",
"note": "each golden task has human-defined acceptance criteria"},
{"item": "sample_files_excluded",
"status": "done",
"note": "%d sample entries excluded from the golden set "
"(exclusion is applied by filter_golden_tasks, never "
"by trusting the candidate list)"
% filtered["dropped_samples"]},
{"item": "approver_and_timestamp_recorded",
"status": "done" if approver and approved_at else "pending_human",
"note": "approver and approval timestamp recorded"},
]
approved = all(i["status"] == "done" for i in items) \
and bool(filtered["golden"]) and bool(approver) \
and bool(approved_at)
return {"approved": approved, "items": items,
"golden_tasks": filtered["golden"],
"dropped_samples": filtered["dropped_samples"]}
def handle_sampling_result(case, sampling_verdict, corrected_verdict=None,
corrector=None, original_signer=None):
"""Turn one human sampling result into a record entry + actions.
``sampling_verdict`` is ``"correct"`` or ``"incorrect"``.
When incorrect, ``corrected_verdict`` (the audit decision that
should have been issued) must be supplied; the retrigger follows
the ORIGINAL decision via ``_RETRIGGER``, or escalates when the
original decision is unknown. Returns the entry dict; the caller
appends it with ``append_audit_record``. This function never
mutates ``case``: the original conclusion is preserved and the
correction lives only in the returned entry.
Corrector separation (track 3 rule, enforced here when both
names are supplied): ``corrector`` must differ from
``original_signer``. Gate effect (shared with track 3 / wiring
#247, stated here so the retrigger names cannot be misread):
a correction never unblocks a merge by itself -- a correction
to ``pass`` schedules a re-audit (``cancel_pushback_and_rerun_audit``
== track 3 ``schedule_reaudit``) and the gate stays blocked
until a new passing re-audit exists; a correction to ``fail``
re-triggers pushback and re-blocks at once
(``reopen_audit_and_pushback`` == track 3
``retrigger_pushback_and_reblock``).
"""
case_id = case.get("id")
original = case.get("decision")
if sampling_verdict not in ("correct", "incorrect"):
raise ValueError("sampling_verdict must be 'correct' or "
"'incorrect', got %r" % (sampling_verdict,))
if (corrector is not None and original_signer is not None
and corrector == original_signer):
raise ValueError("corrector must differ from original signer")
if sampling_verdict == "correct":
return {"case_id": case_id, "original_decision": original,
"sampling_verdict": "correct",
"corrected_decision": None, "retrigger": None,
"recorded_in": "audit_of_audit"}
if corrected_verdict is None:
raise ValueError("an incorrect sampling verdict requires a "
"corrected_verdict")
retrigger = _RETRIGGER.get(original, "escalate_to_human")
return {"case_id": case_id, "original_decision": original,
"sampling_verdict": "incorrect",
"corrected_decision": corrected_verdict,
"retrigger": retrigger,
"recorded_in": "audit_of_audit"}
def append_audit_record(log, entry):
"""Append one sampling entry to the audit-of-audit record.
Returns a new list; the caller persists it. A sampling result
that is not appended here is treated as not having happened.
"""
return list(log) + [entry]
File diff suppressed because it is too large. Load diff
+746
View File
@@ -0,0 +1,746 @@
"""Wiring between three already-landed tracks (no new semantics here):
- track 1 (``cloud/merge_gate_hard.py``, PR #240): the hard merge
gate, five checks + branch protection, machine-readable ``blocks``
codes on its 409 response;
- track 3 (``cloud/audit_pushback.py``, PR #244): the independent
audit + auto-pushback decision, the pushback package, and the QA
re-run rule;
- track 4 (``cloud/ledger.py``, PR #241): the append-only hash-chain
ledger, the only durable store in this wiring.
What this module owns, and nothing else:
1. Translation at the boundary. Track-3 field names and enums are
authoritative for callers; track-4 storage field names are
authoritative on the chain. This module translates, it does not
invent a second ledger schema:
============================ ===================================
track-3 event chain record (track 4 action)
============================ ===================================
audit verdict ``audit_verdict``
pushback round (pushed) ``audit_pushback_round`` (pushed)
fix (landed, re-audit open) ``audit_pushback_round`` with
``fix_diff`` + package marker
``wiring_event="fix"`` (evidence
row only; see the gate reader)
fix + re-audit ``audit_pushback_round``
(re_reviewed, ``fix_diff`` +
``re_review_result``)
rebuild decision / done ``audit_rebuild``
third-failure triage ``audit_third_failure_assessment``
test-change single audit ``audit_test_change``
audit-of-audit correction ``audit_meta_review``
rebuild notice / confirmation ``audit_rebuild_notice``
escalation / blocked quota / ``audit_wiring_event`` (the one
audit sample compat action added to track 4
by this wiring; see below)
============================ ===================================
2. CONDITIONAL mapping, done here and only here. Track 4 stores
verdicts as ``pass`` / ``fail`` only, and track 3 says
CONDITIONAL is not a pass. Every verdict (audit verdict and
re-audit verdict) that arrives as ``"conditional"`` -- or as
anything that is not exactly ``"pass"`` -- is stored as
``"fail"``, with the original value preserved in
``extra.original_verdict`` (verdict rows) or in
``re_review_result.original_verdict`` (pushback rows). The
original conclusion is never overwritten: the chain keeps the
mapped value AND the original, and corrections are separate
``audit_meta_review`` rows, never edits.
3. The gate reader (``pushback_block_for_task``). It replays a
task's chain rows in id order and returns a ``gate_block()``-
shaped dict (``merge_blocked`` / ``release_blocked`` /
``blocked_by="audit_pushback"`` / ``round`` / ``failed_checks``,
plus a ``modules`` detail map). ``cloud/server.py`` ANDs this
with the hard gate's own five checks: either side blocks and
the merge returns 409 with the task status unchanged. The
pushback block applies in every gate mode -- an open audit
pushback is not something a ``record``-mode caller can confirm
away, and ``merge_gate_confirm`` never touches it.
4. The gate-blocks-to-package bridge
(``pushback_package_from_gate``). A hard-gate 409's ``blocks``
codes are machine input to the pushback package: each code maps
to a track-3 gate check (``ci_*`` -> ``ci_green``,
``directory_*`` -> ``directory_isolation``,
``contract_*`` -> ``contract_version``,
``coordinator_*`` -> ``coordinator_approval``,
``review_*`` / ``unresolved_review_comments`` ->
``unresolved_review_comments``). Branch-protection codes have
no track-3 gate-check member, so they travel as the extra check
name ``"branch_protection"`` in the package's failed items --
they are never dropped, and they are never disguised as one of
the ten checks. Callers must take the codes from the 409
``merge_gate`` object, never by parsing audit detail text.
Known translation losses (stated, not hidden):
- Track 3 distinguishes root causes ``"contract"`` vs ``"design"``;
track 4 stores only ``"implementation"`` / ``"contract_design"``.
Both contract and design map to ``"contract_design"`` on rebuild
rows (track 3's rebuild branch treats them identically: refreeze,
no code rebuild). The finer distinction survives on escalation
rows, which carry the original ``root_cause`` verbatim.
- Track 3's third-failure triage verdict ``"early_refreeze"`` has
no track-4 assessment determination (only ``early_rebuild`` /
``continuation_round`` exist there). It is recorded as an
``audit_wiring_event`` of kind ``escalate_refreeze``, never
forced into an assessment row with the wrong determination.
- Track 3's ``"unknown"`` root cause has no track-4 value and is
refused (``ValueError``) rather than guessed: a rebuild row must
say which side the root cause is on.
- A fix-only row is counted by track 4's timeline summary inside
``pushback_rounds`` (that summary counts rows, and a round
already spans two rows there). The gate reader skips fix rows
for state transitions, so the count wart cannot unblock or block
a merge by itself.
What this module does NOT do:
- It does not change any track-1 / track-3 / track-4 semantic:
hard stays fail-closed by default, protection unknown /
unprotected still block in hard mode, confirmation still never
covers a directory violation, the chain stays fail-closed and
append-only, and the cost dual-column rule is untouched.
- It does not create pushback subtasks, send notices, or run
fixes. ``record_*`` only appends chain rows; deriving a fix
subtask from a package is the caller's job (track 3's
``build_pushback_package`` already describes its shape).
- It does not gate ``deploy.py``. There is no server-side release
endpoint in this codebase: the merge endpoint is where
``merge_blocked`` is enforced, and ``release_blocked`` is
returned alongside it so a future release endpoint can enforce
it with this same reader. Deploy remains an operator action
outside this wiring.
"""
from cloud.audit_pushback import GATE_CHECKS
from cloud.audit_pushback import build_pushback_package as _build_package
from cloud.audit_pushback import gate_block as _gate_block
from cloud.ledger import WIRING_EVENT_ACTION
from cloud.ledger import audit_timeline as _audit_timeline
from cloud.merge_gate_hard import block_descriptions as _block_descriptions
from cloud.qa_matrix import (
check_reaudit_reran_links as _check_reran_links,
)
# Hard-gate block code -> track-3 gate check. Branch-protection
# codes have no track-3 member; see BRANCH_PROTECTION_CHECK.
_GATE_BLOCK_TO_CHECK = {}
for _code in (
"ci_evidence_missing", "ci_not_green", "ci_check_failed",
"ci_required_checks_unconfigured", "ci_required_check_missing"):
_GATE_BLOCK_TO_CHECK[_code] = "ci_green"
for _code in (
"directory_evidence_missing", "directory_owned_dir_unknown",
"directory_isolation_violation"):
_GATE_BLOCK_TO_CHECK[_code] = "directory_isolation"
for _code in (
"contract_version_missing", "contract_not_frozen",
"contract_version_mismatch"):
_GATE_BLOCK_TO_CHECK[_code] = "contract_version"
for _code in (
"coordinator_approval_missing", "coordinator_not_approved"):
_GATE_BLOCK_TO_CHECK[_code] = "coordinator_approval"
for _code in ("review_evidence_missing", "unresolved_review_comments"):
_GATE_BLOCK_TO_CHECK[_code] = "unresolved_review_comments"
# Branch-protection codes (hard-mode codes from track 1's #240
# layer plus record-mode codes from the #209 layer).
for _code in ("branch_protection_unknown",
"branch_protection_unprotected",
"branch_protection_enforce",
"branch_protection_risk_unconfirmed"):
_GATE_BLOCK_TO_CHECK[_code] = "branch_protection"
BRANCH_PROTECTION_CHECK = "branch_protection"
# Track-3 decision actions that block merge/release (mirrors
# track 3's gate_block() membership; kept here so the package
# builder does not have to re-derive it).
BLOCKING_ACTIONS = (
"pushback", "rebuild", "escalate", "escalate_refreeze",
"blocked_quota", "rebuild_pending_user_confirmation",
)
# Wiring-event kinds that block (compat action rows).
_BLOCKING_WIRING_KINDS = ("escalate", "escalate_refreeze",
"blocked_quota")
# Triage verdict -> track-4 assessment determination. "early_refreeze"
# is deliberately absent: it goes to record_escalation instead.
_TRIAGE_TO_DETERMINATION = {
"early_rebuild": "early_rebuild",
"targeted_fix": "continuation_round",
}
# Track-3 root cause -> track-4 root cause. "unknown" is absent
# on purpose (refused, not guessed).
_ROOT_CAUSE_MAP = {
"implementation": "implementation",
"contract": "contract_design",
"design": "contract_design",
}
def normalize_verdict(verdict):
"""Track-3 verdict -> (track-4 verdict, original).
Exactly ``"pass"`` stays a pass; anything else -- including
``"conditional"`` and missing/garbage values -- maps to
``"fail"`` (fail-closed), with the original preserved for the
chain. The caller stores the original alongside; this function
never drops it silently.
"""
if verdict == "pass":
return "pass", "pass"
original = verdict if isinstance(verdict, str) else None
return "fail", original
def normalize_re_review_result(result):
"""Copy of a re-review result with its verdict normalized.
``CONDITIONAL`` re-audits do not pass (track 3); the stored
verdict becomes ``"fail"`` and the original is kept under
``original_verdict``. A result without a verdict dict is
returned unchanged (the ledger validates the shape).
"""
if not isinstance(result, dict):
return result
copied = dict(result)
normalized, original = normalize_verdict(result.get("verdict"))
copied["verdict"] = normalized
if normalized == "fail" and original != "fail":
copied["original_verdict"] = original
return copied
def gate_failed_checks(gate):
"""Hard-gate 409 -> (track-3 checks, extra checks), in order.
Track-3 checks come first in GATE_CHECKS order, then the
branch-protection extra (if any). Unknown codes pass through
as extras rather than being dropped: a code the wiring does
not recognise must still show up in the package.
"""
checks = []
extras = []
for code in (gate or {}).get("blocks") or []:
mapped = _GATE_BLOCK_TO_CHECK.get(code)
if mapped == BRANCH_PROTECTION_CHECK:
if BRANCH_PROTECTION_CHECK not in extras:
extras.append(BRANCH_PROTECTION_CHECK)
elif mapped is not None:
if mapped not in checks:
checks.append(mapped)
elif code not in extras:
extras.append(code)
order = {name: index for index, name in enumerate(GATE_CHECKS)}
checks.sort(key=lambda name: order.get(name, len(order)))
return checks, extras
def gate_block_items(gate):
"""One pushback-package failed item per hard-gate block code.
The package builder also appends decision ``failed_checks``
that are not already items, so the item ``check`` values here
use the mapped track-3 names (branch protection keeps its
extra name) to avoid duplicate generic entries.
"""
descriptions = _block_descriptions((gate or {}).get("blocks") or [])
items = []
for code, description in zip((gate or {}).get("blocks") or [],
descriptions):
check = _GATE_BLOCK_TO_CHECK.get(code, code)
items.append({
"check": check,
"detail": description,
"evidence": {
"test_name": None,
"file_path": None,
"line_no": None,
"contract_clause": None,
"diff_range": None,
},
"reproduction_steps": [
"re-run the merge gate with full evidence; "
"the failing code is %s" % code,
],
"reaudit_criterion": "re-run the full re-audit checklist "
"for this check",
})
return items
def pushback_package_from_gate(gate, task_id, round_no,
decision_action="pushback",
previous_artifacts=None,
qa_failed_links=None):
"""Build a track-3 pushback package from a hard-gate 409.
``gate`` is the ``merge_gate`` object from the 409 response
(not audit text). ``round_no`` is the track-3 decision round
(0 for the first audit); the ledger round number is derived
by the caller (see ``record_pushback``'s docstring). The
package carries the QA failed links verbatim when given, so
the re-audit must re-run exactly those links (see
``record_reaudit``'s QA guard).
"""
if decision_action not in BLOCKING_ACTIONS and \
decision_action != "allow":
raise ValueError("unknown decision action: %r" % (decision_action,))
checks, extras = gate_failed_checks(gate)
audit = {
"task_id": task_id,
"verdict": "fail",
"verdict_text": "merge gate blocked: %s" % (
", ".join((gate or {}).get("blocks") or ["unknown"])),
"failed_items": gate_block_items(gate),
}
decision = {
"round": round_no,
"next_round": round_no + 1,
"action": decision_action,
"failed_checks": checks + extras,
}
return _build_package(audit, decision, previous_artifacts,
qa_failed_links=qa_failed_links)
def _require_module_contract(module_id, contract_version):
"""Wiring strictness: pushback rows must be attributable.
Track 4 allows module/contract to be None on pushback rows,
but unattributed rows cannot be scoped by the gate reader, so
this wiring refuses them instead of writing rows the gate
cannot reason about. Verdict/rebuild rows already require
both in track 4 itself.
"""
if not isinstance(module_id, str) or not module_id:
raise ValueError("module_id is required for wiring records")
if not isinstance(contract_version, str) or not contract_version:
raise ValueError("contract_version is required for wiring records")
def record_audit_verdict(store, task_id, project_id, module_id,
contract_version, verdict, source,
failures=None, user_id="system", extra=None):
"""Append one audit/acceptance verdict to the chain.
``verdict`` uses track-3 vocabulary (``pass`` / ``fail`` /
``conditional``); ``"conditional"`` is stored as ``"fail"``
with the original in ``extra.original_verdict``.
"""
normalized, original = normalize_verdict(verdict)
merged_extra = dict(extra or {})
if normalized == "fail" and original != "fail":
merged_extra.setdefault("original_verdict", original)
return store.append_audit_verdict(
task_id, project_id, module_id, contract_version,
normalized, source, failures=failures, user_id=user_id,
extra=merged_extra)
def record_pushback(store, task_id, project_id, module_id,
contract_version, round_no, trigger_items,
package, failure_source=None, failure_counts=None,
user_id="system"):
"""Append one pushed pushback round to the chain.
``round_no`` is the LEDGER round number, starting at 1 for
the first pushback. Track 3's ``decide_pushback`` counts
rounds already used (0 for the first audit) and returns
``next_round``; for a pushback action that value IS the
ledger round number (first failure: decision round 0,
next_round 1, ledger round_no 1). Trigger items are the
package's failed checks/items (machine codes, not prose).
"""
_require_module_contract(module_id, contract_version)
return store.append_pushback_round(
task_id, project_id, round_no, trigger_items, package,
user_id=user_id, module_id=module_id,
contract_version=contract_version,
failure_source=failure_source, failure_counts=failure_counts)
def record_fix(store, task_id, project_id, module_id,
contract_version, round_no, fix_diff, package,
user_id="system"):
"""Append fix evidence for a round whose re-audit is still open.
The row carries ``fix_diff`` (summary + sha, never the full
diff) and a package copy marked ``wiring_event="fix"``. The
gate reader skips such rows for state transitions: landing a
fix does not unblock anything -- only a passing re-audit does.
When fix and re-audit arrive together, use ``record_reaudit``
with ``fix_diff`` instead of writing both rows.
"""
_require_module_contract(module_id, contract_version)
marked = dict(package or {})
marked["wiring_event"] = "fix"
return store.append_pushback_round(
task_id, project_id, round_no, _package_trigger_items(package),
marked, fix_diff=fix_diff, user_id=user_id,
module_id=module_id, contract_version=contract_version)
def _package_trigger_items(package):
"""Trigger items for a fix row: the package's failed checks."""
if isinstance(package, dict):
checks = package.get("failed_checks")
if isinstance(checks, list) and checks:
return list(checks)
return ["audit_fix"]
def check_reaudit_qa_guard(package, re_review_result, rerun_results):
"""Fail-closed QA guard for a claimed passing re-audit.
If the package lists ``reaudit_must_rerun_links`` and the
re-audit claims a pass, every listed link must have a passing
re-run result; a missing or failed re-run raises ValueError
instead of letting a false pass reach the chain. A failing
re-audit needs no guard (it stays blocked either way).
Returns the normalized re-review result.
"""
normalized_result = normalize_re_review_result(re_review_result)
if normalized_result.get("verdict") != "pass":
return normalized_result
required = (package or {}).get("reaudit_must_rerun_links") or []
if not required:
return normalized_result
checked = _check_reran_links(package, rerun_results or {})
if not checked["reran_all_failed_links"]:
raise ValueError(
"re-audit claims pass but QA failed links were not "
"all re-run passing: missing=%s failed=%s"
% (checked["missing"], checked["failed"]))
return normalized_result
def record_reaudit(store, task_id, project_id, module_id,
contract_version, round_no, re_review_result,
package=None, fix_diff=None, rerun_results=None,
failure_source=None, failure_counts=None,
user_id="system"):
"""Append one re-audited pushback round to the chain.
This is the row that can unblock: a stored ``pass`` verdict
here clears the module's block in the gate reader, a stored
``fail`` (including a mapped ``conditional``) keeps it
blocked. The QA guard in ``check_reaudit_qa_guard`` runs
before anything is written, so a claimed pass with un-rerun
or failed QA links never reaches the chain.
"""
_require_module_contract(module_id, contract_version)
guarded = check_reaudit_qa_guard(package, re_review_result,
rerun_results)
return store.append_pushback_round(
task_id, project_id, round_no,
_package_trigger_items(package), package or {},
fix_diff=fix_diff, re_review_result=guarded,
user_id=user_id, module_id=module_id,
contract_version=contract_version,
failure_source=failure_source, failure_counts=failure_counts)
def record_rebuild(store, task_id, project_id, module_id,
contract_version, rebuild_no, old_impl_ref,
new_impl_ref=None, root_cause="implementation",
status="decided", prior_failure_total=None,
retained_assets=None, trigger_source=None,
assessment_entry_id=None, user_id="system"):
"""Append one rebuild decision / completion to the chain.
``root_cause`` uses track-3 vocabulary; ``"contract"`` and
``"design"`` both store as track-4 ``"contract_design"``
(see the module docstring). ``"unknown"`` is refused.
A ``rebuilt`` row does NOT unblock the gate on its own: the
new implementation still needs a passing re-audit.
"""
mapped = _ROOT_CAUSE_MAP.get(root_cause)
if mapped is None:
raise ValueError(
"root_cause must be implementation|contract|design, "
"not %r (unknown is refused, not guessed)" % (root_cause,))
return store.append_rebuild_decision(
task_id, project_id, module_id, contract_version,
rebuild_no, old_impl_ref, new_impl_ref=new_impl_ref,
root_cause=mapped, status=status,
prior_failure_total=prior_failure_total,
user_id=user_id, retained_assets=retained_assets,
trigger_source=trigger_source,
assessment_entry_id=assessment_entry_id)
def record_third_failure_assessment(store, task_id, project_id,
module_id, contract_version,
triage_verdict, basis,
failure_total=3,
max_extra_rounds=None,
user_id="system"):
"""Append one third-failure triage determination to the chain.
Track-3 triage verdicts map as ``early_rebuild`` ->
``early_rebuild`` and ``targeted_fix`` -> ``continuation_round``.
``early_refreeze`` has no assessment determination in track 4
and is refused here: record it with ``record_escalation``
(kind ``escalate_refreeze``) instead.
"""
determination = _TRIAGE_TO_DETERMINATION.get(triage_verdict)
if determination is None:
raise ValueError(
"triage_verdict %r has no assessment determination; "
"use record_escalation(kind='escalate_refreeze') for "
"early_refreeze" % (triage_verdict,))
return store.append_third_failure_assessment(
task_id, project_id, module_id, contract_version,
determination, basis, failure_total=failure_total,
max_extra_rounds=max_extra_rounds, user_id=user_id)
def record_test_change(store, task_id, project_id, module_id,
contract_version, test_changes, auditor,
contract_refrozen, refreeze_ref=None,
reason="", user_id="system"):
"""Append one test-change single audit (track-4 native)."""
return store.append_test_change_audit(
task_id, project_id, module_id, contract_version,
test_changes, auditor, contract_refrozen,
refreeze_ref=refreeze_ref, reason=reason, user_id=user_id)
def record_meta_review(store, task_id, project_id, module_id,
contract_version, original_entry_id,
original_verdict, corrected_verdict,
misjudgment_found, reviewer, findings=None,
triggered_process=None, user_id="system"):
"""Append one audit-of-audit correction (history is kept).
The correction never rewrites the original row. How the gate
reader treats it is a resolved semantic tension, stated here
rather than buried (see ``pushback_block_for_task``): a
correction to ``fail`` (a wrong pass) re-blocks immediately,
but a correction to ``pass`` (a wrong block) does NOT unblock
by itself -- track 3's restored action for that case is
"schedule a re-audit", not "release the gate", and a
meta-review is a review of an audit, not a full re-audit.
Unblocking still needs a fresh passing verdict or a passing
re-reviewed pushback row. Track 4's failure counts are
unaffected by this choice: counts use effective (corrected)
verdicts to decide rebuild triggers, while the gate asks a
different question ("has a passing re-audit been recorded
since the block?"), so the two may legitimately disagree
while a scheduled re-audit is still pending.
"""
return store.append_audit_of_audit(
task_id, project_id, module_id, contract_version,
original_entry_id, original_verdict, corrected_verdict,
misjudgment_found, reviewer, findings=findings,
triggered_process=triggered_process, user_id=user_id)
def record_rebuild_notice(store, task_id, project_id, module_id,
contract_version, rebuild_no,
notification_content, notified_party,
confirmation_status, confirmed_by=None,
user_id="system"):
"""Append one rebuild-notice row (pending -> confirmed are two rows)."""
return store.append_rebuild_notice(
task_id, project_id, module_id, contract_version,
rebuild_no, notification_content, notified_party,
confirmation_status, confirmed_by=confirmed_by,
user_id=user_id)
def record_escalation(store, task_id, project_id, module_id,
contract_version, kind, target=None, reason="",
round_no=None, root_cause=None,
failed_checks=None, user_id="system"):
"""Append one escalation / quota / audit-sample wiring event.
``kind`` is track-3 vocabulary: ``escalate``,
``escalate_refreeze``, ``blocked_quota``, ``audit_sample``.
Escalation kinds block the gate; ``audit_sample`` (a sampling
selection, not a verdict) does not change gate state. The
original ``root_cause`` (including the contract/design
distinction) is stored verbatim here.
"""
_require_module_contract(module_id, contract_version)
return store.append_wiring_event(
task_id, project_id, module_id, contract_version, kind,
target=target, reason=reason, round_no=round_no,
root_cause=root_cause, failed_checks=failed_checks,
user_id=user_id)
def _trigger_check_names(trigger_items):
"""Trigger items (str or dict) -> check-name strings."""
names = []
for item in trigger_items or []:
if isinstance(item, str):
names.append(item)
elif isinstance(item, dict) and item.get("check"):
names.append(item["check"])
else:
names.append(str(item))
return names
def _module_key(parsed):
"""(module_id, contract_version) or None for unattributed rows."""
module_id = parsed.get("module_id") if isinstance(parsed, dict) else None
contract_version = parsed.get("contract_version") \
if isinstance(parsed, dict) else None
if module_id and contract_version:
return (module_id, contract_version)
return None
def pushback_block_for_task(conn, task_id):
"""Replay a task's chain into a gate_block()-shaped verdict.
State machine, in chain-id order, per (module, contract
version) plus one global slot for unattributed rows:
- verdict row: its stored verdict sets blocked (pass clears,
fail blocks).
- meta-review row: a correction to ``fail`` blocks at once
(a wrong pass is re-blocked); a correction to ``pass``
changes nothing by itself -- the wrong block stays blocked
until a fresh passing verdict or re-reviewed row arrives
(track 3's "schedule a re-audit", not "release the gate").
This is deliberately NOT track 4's effective-verdict
counting: counts and gate state answer different
questions, and conflating them would let a review of an
audit substitute for a re-audit.
- pushback pushed: blocked. pushback re_reviewed: blocked
unless the stored re-audit verdict is a pass (a stored
``conditional`` was already mapped to fail at write time;
anything non-pass here still blocks, fail-closed).
- pushback fix-evidence row (package ``wiring_event="fix"``):
no state change.
- rebuild row (decided or rebuilt): blocked -- a rebuilt
implementation still needs a passing re-audit.
- wiring event escalate / escalate_refreeze / blocked_quota:
blocked. audit_sample: no change.
- rebuild notice pending: blocked. confirmed / rejected: no
change (confirmation alone never unblocks).
- assessment / test-change / meta rows: no direct change
(meta rows act only through corrections).
A task is blocked if the global slot or any module slot is
blocked. A missing ``audit_ledger`` table means no chain has
ever been written for this database, which is unblocked, not
an error; any other read failure propagates so the server can
fail closed (see server.py).
"""
try:
timeline = _audit_timeline(conn, task_id=task_id, limit=10000)
except Exception as exc:
if "no such table" in str(exc):
return _gate_block("allow", [], 0)
raise
entries = timeline.get("entries") or []
modules = {}
global_state = {"blocked": False, "round": None, "checks": []}
last_blocked_round = None
def slot_for(parsed):
key = _module_key(parsed)
if key is None:
return None, global_state
if key not in modules:
modules[key] = {"blocked": False, "round": None,
"checks": []}
return key, modules[key]
for entry in entries:
parsed = entry.get("parsed")
if not isinstance(parsed, dict):
continue
action = entry.get("action")
_key, slot = slot_for(parsed)
if action == "audit_verdict":
blocked = parsed.get("verdict") != "pass"
slot["blocked"] = blocked
slot["checks"] = _trigger_check_names(
parsed.get("failures")) if blocked else []
elif action == "audit_meta_review":
# Re-block on a corrected fail; a corrected pass
# schedules a re-audit and does not unblock (see the
# module docstring: correction != re-audit).
if parsed.get("misjudgment_found") and \
parsed.get("corrected_verdict") == "fail":
slot["blocked"] = True
if not slot["checks"]:
slot["checks"] = ["audit_meta_review"]
elif action == "audit_pushback_round":
package = parsed.get("pushback_package") or {}
if package.get("wiring_event") == "fix":
continue
round_no = parsed.get("round_no")
if parsed.get("round_status") == "pushed":
slot["blocked"] = True
slot["round"] = round_no
slot["checks"] = _trigger_check_names(
parsed.get("trigger_items"))
last_blocked_round = round_no
elif parsed.get("round_status") == "re_reviewed":
result = parsed.get("re_review_result") or {}
blocked = result.get("verdict") != "pass"
slot["blocked"] = blocked
slot["round"] = round_no
slot["checks"] = _trigger_check_names(
parsed.get("trigger_items")) if blocked else []
if blocked:
last_blocked_round = round_no
elif action == "audit_rebuild":
slot["blocked"] = True
if not slot["checks"]:
slot["checks"] = ["audit_rebuild"]
last_blocked_round = slot["round"]
elif action == WIRING_EVENT_ACTION:
if parsed.get("wiring_kind") in _BLOCKING_WIRING_KINDS:
slot["blocked"] = True
checks = parsed.get("failed_checks")
if checks:
slot["checks"] = list(checks)
if parsed.get("round_no") is not None:
slot["round"] = parsed.get("round_no")
last_blocked_round = parsed.get("round_no")
elif action == "audit_rebuild_notice":
if parsed.get("confirmation_status") == "pending":
slot["blocked"] = True
blocked = global_state["blocked"] or any(
slot["blocked"] for slot in modules.values())
checks = []
if global_state["blocked"]:
checks.extend(global_state["checks"])
for slot in modules.values():
if slot["blocked"]:
for check in slot["checks"]:
if check not in checks:
checks.append(check)
block = _gate_block(
"pushback" if blocked else "allow", checks,
last_blocked_round if last_blocked_round is not None else 0)
block["modules"] = {
"%s@%s" % key: {
"blocked": slot["blocked"],
"round": slot["round"],
"failed_checks": list(slot["checks"]),
}
for key, slot in modules.items()
}
if global_state["blocked"]:
block["modules"]["<global>"] = {
"blocked": True,
"round": global_state["round"],
"failed_checks": list(global_state["checks"]),
}
return block
+85 -16
View File
@@ -25,6 +25,8 @@ import hashlib
import secrets
import sqlite3
from cloud.db import connect as _db_connect
from cloud.ledger import append_entry as _ledger_append
from cloud.ledger import ensure_schema as _ledger_ensure
import time
import uuid
@@ -39,6 +41,55 @@ _ACTION_MIN_ROLE = {
"manage_project": "owner",
}
# audit_log 建表 SQL(AuthStore.__init__ 与 audit_on_conn 共用同一
# 份,绝不能各写一份后漂移)。PG 适配层把其中的
# INTEGER PRIMARY KEY AUTOINCREMENT 翻译成 identity 列,见
# cloud/db.py;SQLite 下原样执行。
_AUDIT_LOG_TABLE_SQL = (
"CREATE TABLE IF NOT EXISTS audit_log("
"id INTEGER PRIMARY KEY AUTOINCREMENT, "
"user_id TEXT, "
"project_id TEXT, "
"action TEXT, "
"detail TEXT, "
"created_at REAL)"
)
def audit_on_conn(conn, user_id, project_id, action, detail="",
created_at=None):
"""在调用方给定的连接上写一对审计记录(单次尝试)。
与 AuthStore.audit() 是同一对写:audit_log 行 + audit_ledger
链条目同 created_at(调用方不给时取当前时间),先建两张表
(幂等),再写两行。本函数不 commit、不 rollback、不重试:
事务边界属于调用方——调用方须在自己的事务里 commit,失败
时 rollback 整段业务写、链头唯一冲突时 rollback 后重试整段
(只重试本对写会丢掉调用方同事务里已写但未提交的业务行;
见 cloud/ledger.py append_entry 的警告)。AuthStore.audit()
即本函数 + “自有连接上 commit、唯一冲突重试 5 次”的包装。
存在理由:automations.run_due_automation 拿到的是调用方连接
(任务 INSERT 与产出记录都在该连接的事务里),不能像服务端
handler 那样另开 AuthStore(自有连接、自有事务)去写审计——
那样审计与任务不在同一事务,审计失败时任务已落盘,所谓
“整对回滚”只保审计对、不保业务对。本函数让自动化执行器
的审计行与任务行、产出记录行同事务提交、同事务回滚。
"""
if created_at is None:
created_at = time.time()
created_at = float(created_at)
conn.execute(_AUDIT_LOG_TABLE_SQL)
_ledger_ensure(conn)
conn.execute(
"INSERT INTO audit_log(user_id, project_id, action,"
" detail, created_at)"
" VALUES(?, ?, ?, ?, ?)",
(user_id, project_id, action, detail, created_at),
)
_ledger_append(conn, user_id, project_id, action,
detail=detail, created_at=created_at)
class AuthStore:
"""基于 SQLite 的用户 / 项目成员 / 审计存储。"""
@@ -60,15 +111,10 @@ class AuthStore:
"role TEXT, "
"PRIMARY KEY(project_id, user_id))"
)
self.conn.execute(
"CREATE TABLE IF NOT EXISTS audit_log("
"id INTEGER PRIMARY KEY AUTOINCREMENT, "
"user_id TEXT, "
"project_id TEXT, "
"action TEXT, "
"detail TEXT, "
"created_at REAL)"
)
self.conn.execute(_AUDIT_LOG_TABLE_SQL)
# 防篡改账本(哈希链):audit_log 本身不动,链写进
# audit_ledger;建表失败即建库失败,不静默丢链。
_ledger_ensure(self.conn)
self.conn.commit()
def create_user(self, username):
@@ -350,13 +396,36 @@ class AuthStore:
)
def audit(self, user_id, project_id, action, detail=""):
"""只追加一条审计记录。"""
self.conn.execute(
"INSERT INTO audit_log(user_id, project_id, action, detail, created_at)"
" VALUES(?, ?, ?, ?, ?)",
(user_id, project_id, action, detail, time.time()),
)
self.conn.commit()
"""只追加一条审计记录,并同步写哈希链(同事务)。
audit_log 行与 audit_ledger 链条目同 created_at、同事务
提交:链写失败时整对回滚并抛错(fail-closed,绝不留
“audit_log 有行、链上无行”的静默缺口)。并发抢链头
导致唯一冲突时在事务边界重试整对写入,最多 5 次;
其它错误直接回滚抛错,不重试。
实现即 audit_on_conn() + 本连接上的 commit/重试包装,
对写本身与 #241 完全一致(同一函数、同一 created_at
语义),不是第二套审计写路径。
"""
now = time.time()
last_exc = None
for _attempt in range(5):
try:
audit_on_conn(self.conn, user_id, project_id, action,
detail=detail, created_at=now)
self.conn.commit()
return
except Exception as exc:
last_exc = exc
try:
self.conn.rollback()
except Exception:
pass
if not isinstance(exc, sqlite3.IntegrityError) \
and type(exc).__name__ != "PostgresIntegrityError":
raise
raise last_exc
def list_audit(self, project_id, limit=100):
"""按 created_at 倒序返回该项目的审计列表(dict)。"""
+15 -29
View File
@@ -53,12 +53,18 @@ cloud/migrations.py 的对齐说明;PG DDL 同步进
cloud/db.py POSTGRES_SCHEMA 与 MIGRATION_TABLES。
"""
import os
import re
import time
from cloud.db import connect as _db_connect
from cloud.identity import IdentityStore
# 行取值统一走 peer_messages.row_get(A7):本模块旧私有版与之
# 同款语义(缺列/缺键/空行回 default,默认 None),调用处仍
# 用 _row_get 别名,行为不变。
from cloud.peer_messages import row_get as _row_get
from cloud.quota import default_quota_usd_micros as _quota_default
from cloud.quota import monthly_usage as _quota_monthly_usage
from cloud.quota import quota_for_owner as _quota_for_owner
DEFAULT_SETTINGS = {
"generate_memories": 0,
@@ -113,12 +119,9 @@ def redact_text(text):
def _default_quota_usd_micros():
"""与 worker/server 同款默认配额解析,避免循环 import。"""
try:
return max(0, int(os.environ.get(
"CLOUD_DEFAULT_QUOTA_USD_MICROS", "10000000")))
except (TypeError, ValueError):
return 10000000
"""默认配额解析:收敛到 cloud/quota.py(quota 模块是
标准库叶子,不存在旧注释担心的循环 import)。"""
return _quota_default()
def quota_remaining_percent(db_path, owner):
@@ -131,27 +134,17 @@ def quota_remaining_percent(db_path, owner):
month = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m")
conn = _db_connect(db_path)
try:
conn.execute(
"CREATE TABLE IF NOT EXISTS user_quotas("
"user_id TEXT PRIMARY KEY, quota_usd_micros BIGINT, "
"updated_at DOUBLE PRECISION)")
# 配额行与当月 USD 用量取数收敛到 cloud/quota.py
#(无价部分本函数不用,只取 USD 一项;缺 cost_records
# 表时仍按旧行为计 0,其它查库异常仍抛给调用方)。
quota = _quota_for_owner(conn, owner)
conn.commit()
row = conn.execute(
"SELECT quota_usd_micros FROM user_quotas WHERE user_id = ?",
(owner,)).fetchone()
quota = _default_quota_usd_micros() if row is None else row[0]
if quota is None:
return None
if quota <= 0:
return 0.0
try:
used_row = conn.execute(
"SELECT COALESCE(SUM(c.usd_micros), 0)"
" FROM cost_records c JOIN tasks t"
" ON c.task_id = t.id"
" WHERE t.created_by = ? AND c.day LIKE ?",
(owner, month + "%")).fetchone()
used = used_row[0] if used_row else 0
used, _unpriced = _quota_monthly_usage(conn, owner, month)
except Exception as exc:
if "no such table" in str(exc):
used = 0
@@ -162,13 +155,6 @@ def quota_remaining_percent(db_path, owner):
conn.close()
def _row_get(row, name, default=None):
try:
return row[name]
except (KeyError, IndexError, TypeError):
return default
def _first_line(text, max_len=200):
for line in (text or "").splitlines():
line = " ".join(line.split())
+130 -133
View File
@@ -38,16 +38,31 @@
id 记在 automation_runs.task_id 里,派发与任务绑定,不用
为自动化另起一套通知系统)。
本模块只依赖标准库,不 import cloud 其它模块,避免循环
import;表迁移写法仿 peer_messages(CREATE TABLE IF NOT
EXISTS + PRAGMA 查列补齐,SQLite 与 Postgres 适配层同款)。
本模块的 cloud 依赖只有两处叶子引用,避免循环 import:
配额判定走同为标准库叶子的 cloud.quota;审计行上链
(方案 A,见 run_due_automation)只走 cloud.auth.audit_on_conn——
与 AuthStore.audit() 同一对写(audit_log 行 + audit_ledger
链条目,同 created_at),不再有 audit_log 直写旁路。
auth 只依赖 db / ledger 两个叶模块,三者都不 import
automations / server / worker,不构成循环 import;其余 SQL
(配额口径、表迁移)仍就地实现,不另 import 业务模块。
表迁移写法仿 peer_messages(CREATE TABLE IF NOT
EXISTS + PRAGMA 查列补齐,SQLite 与 Postgres 适配层同款)。
"""
import os
import sqlite3
import time
import uuid
from datetime import datetime, timezone
from cloud.peer_messages import row_get
from cloud.quota import decide as _quota_decide
from cloud.quota import default_quota_usd_micros as _quota_default
from cloud.quota import monthly_usage as _quota_monthly_usage
from cloud.quota import quota_for_owner as _quota_for_owner
from cloud.auth import audit_on_conn as _audit_on_conn
MIN_INTERVAL_SECONDS = 60
MAX_INTERVAL_SECONDS = 30 * 86400
MAX_NAME_LEN = 100
@@ -59,14 +74,9 @@ ACTIVE = "active"
CANCELLED = "cancelled"
def row_get(row, name):
"""行取值容错:sqlite3.Row 缺列报 IndexError、dict 缺键报
KeyError、空行报 TypeError,一律回 None(老库行缺列时
读自动化列表不能因此炸掉整页)。"""
try:
return row[name]
except (KeyError, IndexError, TypeError):
return None
# 行取值走 cloud.peer_messages.row_get(A7 全仓统一):旧私有版
# 与统一版在真实行类型(None / dict / sqlite3.Row)下语义完全
# 相同(缺键/缺列/空行一律回 None),故直接复用,不再自带一份。
AUTOMATION_COLUMNS = (
@@ -520,13 +530,9 @@ def record_automation_run(conn, automation_id, project_id, task_id=None,
def _default_quota_usd_micros():
"""默认月配额(微美元):与服务端 _default_quota_usd_micros
同值同环境变量(CLOUD_DEFAULT_QUOTA_USD_MICROS,默认 $10)。"""
try:
return max(0, int(os.environ.get(
"CLOUD_DEFAULT_QUOTA_USD_MICROS", "10000000")))
except (TypeError, ValueError):
return 10000000
"""默认月配额(微美元):解析收敛到 cloud/quota.py
(同值同环境变量 CLOUD_DEFAULT_QUOTA_USD_MICROS,默认 $10)。"""
return _quota_default()
def _unpriced_token_cap():
@@ -549,29 +555,17 @@ def _automation_quota_exceeded(conn, owner):
过处理,绝不建任务)。cost_records 表不存在时视为零花费
(CostStore 首次记账时才建表,不存在即从未记过账),其
它查库异常才 fail-closed——全新库上不能因此误拒全部运行。
本模块只依赖标准库,不 import cloud 其它模块,故此处就
地实现 SQL 版口径,注释里标明三处同口径点,改口径时三
处要一起改(server._quota_status、worker._quota_exceeded、
本函数)。
取数与比较判定收敛到 cloud/quota.py(本函数只保留
表存在探针与 PG 事务回滚处理,因其与自动化执行器的
事务形态绑定,不宜塞进通用取数层)。
"""
if not owner:
return ("quota_exceeded: 自动化缺少创建人,无法查配额,"
"拒绝建任务")
try:
conn.execute(
"CREATE TABLE IF NOT EXISTS user_quotas("
"user_id TEXT PRIMARY KEY, quota_usd_micros BIGINT, "
" updated_at DOUBLE PRECISION)")
row = conn.execute(
"SELECT quota_usd_micros FROM user_quotas"
" WHERE user_id = ?",
(owner,)).fetchone()
if row is None:
quota = _default_quota_usd_micros()
else:
quota = row_get(row, "quota_usd_micros")
if quota is None:
return None
quota = _quota_for_owner(conn, owner)
if quota is None:
return None
month = datetime.now(timezone.utc).strftime("%Y-%m")
try:
conn.execute("SELECT COUNT(*) FROM cost_records").fetchone()
@@ -585,22 +579,8 @@ def _automation_quota_exceeded(conn, owner):
pass
table_exists = False
if table_exists:
used_row = conn.execute(
"SELECT COALESCE(SUM(c.usd_micros), 0) AS total"
" FROM cost_records c JOIN tasks t"
" ON c.task_id = t.id"
" WHERE t.created_by = ? AND c.day LIKE ?",
(owner, month + "%")).fetchone()
used = row_get(used_row, "total") or 0
unpriced_row = conn.execute(
"SELECT COALESCE(SUM(c.input_tokens + c.output_tokens), 0)"
" AS total"
" FROM cost_records c JOIN tasks t"
" ON c.task_id = t.id"
" WHERE t.created_by = ? AND c.day LIKE ?"
" AND c.usd_micros IS NULL AND c.source = 'actual'",
(owner, month + "%")).fetchone()
unpriced_used = row_get(unpriced_row, "total") or 0
used, unpriced_used = _quota_monthly_usage(
conn, owner, month)
else:
used = 0
unpriced_used = 0
@@ -612,61 +592,49 @@ def _automation_quota_exceeded(conn, owner):
except Exception:
pass
return "quota_check_failed: %s" % exc
if used >= quota:
return ("quota_exceeded: 本月配额已用完(已用 %d / "
"配额 %d 微美元,%s)" % (used, quota, month))
if unpriced_used >= _unpriced_token_cap():
return ("quota_exceeded: 本月无价模型 token 已用完"
"(已用 %d / 上限 %d,%s)"
% (unpriced_used, _unpriced_token_cap(), month))
decision = _quota_decide(quota, used, unpriced_used,
_unpriced_token_cap(), month)
if decision is not None:
return decision["message"]
return None
def _audit_row(conn, project_id, action, detail, now):
"""best-effort 写审计行:失败不影响自动化运行本身。
def _is_integrity_error(exc):
"""链头唯一冲突判定:与 cloud/ledger.py _is_integrity_error
同规则(sqlite3.IntegrityError,或 PG 适配层映射后的
PostgresIntegrityError,按名判定,驱动类不跨模块 import)。
旧实现显式插 TEXT uuid 当 id:SQLite 下能跑,但生产 PG 的
audit_log.id 是 BIGINT GENERATED BY DEFAULT AS IDENTITY,
插 TEXT 直接报错、审计行全丢(best-effort 吞掉后无声无息)。
修法:INSERT 一律不带 id,让两端自增/identity 生成;
表不存在时先按 PG 口径建、再按 SQLite 口径建、逐个重试,
哪边建表成功就用哪边——automations 按设计不 import 其它
cloud 模块,所以用“先插、失败再按口径重试”做方言探测,
不靠 import 判断后端。
只有这类错误值得在事务边界重试整段;其它错误重试无用,
必须立即回滚抛错,不能被重试掩盖成“锁/冲突问题”。
本模块不直写 audit_log:审计只走 _audit_on_conn(见模块文档);
下面的 _audit_row 仅为兼容包装(供 main #275 resident.py),
本身仍走 _audit_on_conn,不含 audit_log INSERT。
"""
if isinstance(exc, sqlite3.IntegrityError):
return True
return type(exc).__name__ == "PostgresIntegrityError"
def _audit_row(conn, project_id, action, detail, now):
"""兼容包装:供 main #275 resident.py 的审计调用。
本函数不直写 audit_log(#250 断言只允许 auth.py 出现
audit_log INSERT):经 cloud.auth.audit_on_conn 写
audit_log 行 + audit_ledger 链条目同一对,user_id 仍为
"system"、created_at 用调用方给的 now。保留 main 旧版的
best-effort 自提交语义:写对失败时回滚后吞掉,不影响
resident 轮询本身;调用方已 commit 的业务行不受影响
(审计对写是独立的后续写)。
"""
params = ("system", project_id, action, detail, now)
try:
conn.execute(
"INSERT INTO audit_log(user_id, project_id, action,"
" detail, created_at) VALUES(?, ?, ?, ?, ?)", params)
_audit_on_conn(conn, "system", project_id, action, detail,
created_at=now)
conn.commit()
return
except Exception:
try:
conn.rollback()
except Exception:
pass
for ddl in (
"CREATE TABLE IF NOT EXISTS audit_log("
"id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, "
"user_id TEXT, project_id TEXT, action TEXT, detail TEXT, "
"created_at DOUBLE PRECISION)",
"CREATE TABLE IF NOT EXISTS audit_log("
"id INTEGER PRIMARY KEY AUTOINCREMENT, "
"user_id TEXT, project_id TEXT, action TEXT, detail TEXT, "
"created_at REAL)"):
try:
conn.execute(ddl)
conn.execute(
"INSERT INTO audit_log(user_id, project_id, action,"
" detail, created_at) VALUES(?, ?, ?, ?, ?)", params)
conn.commit()
return
except Exception:
try:
conn.rollback()
except Exception:
pass
def run_due_automation(conn, automation_id, now=None):
@@ -678,9 +646,22 @@ def run_due_automation(conn, automation_id, now=None):
失败重试循环)并记审计,返回该产出;配额内才 INSERT 云
端任务(queued,executor/model 留 NULL 走 worker 默认,
与服务端建任务缺省行为一致),产出记 created 并记审计。
任务 INSERT 与产出记录由 record_automation_run 的 commit
一并落盘。调用方(未来的定时执行器)只调本函数,绝不
调用方(未来的定时执行器)只调本函数,绝不
自己拼 claim + INSERT,否则配额门可被绕过。
审计失败语义(方案 A,旧 best-effort _audit_row 已删除):
审计对写(audit_log 行 + audit_ledger 链条目,同 created_at
= now,走 cloud.auth.audit_on_conn)先于产出记录的 commit
执行,任务行、产出记录行、审计对行同事务提交:审计失败
时三者一起回滚,异常抛给执行器——不记“任务已建、审计缺
失”的半截状态,也不把审计失败记成 failed 产出、不进失败
重试循环。抢单已单独提交(last_run_at 已写),按 claim 的
文档,这次执行等下个间隔再跑,与“抢单后崩掉等下个间隔
补跑”一致。链头唯一冲突时整段回滚重试,最多 5 次(与
AuthStore.audit 同数);其它错误立即回滚抛错,不重试。
审计行 user_id 仍为 "system",detail 格式不变
(created:"<自动化id> -> <任务id>";skipped:
"<自动化id>: <配额门文本>")。
"""
if now is None:
now = time.time()
@@ -697,46 +678,62 @@ def run_due_automation(conn, automation_id, now=None):
prompt = row_get(auto, "prompt")
owner = row_get(auto, "created_by")
gate = _automation_quota_exceeded(conn, owner)
if gate is not None:
run = record_automation_run(
conn, automation_id, project_id, task_id=None,
status="skipped_quota", detail=gate, now=now)
_audit_row(conn, project_id, "automation_run_quota_skipped",
"%s: %s" % (automation_id, gate), now)
return run
task_id = uuid.uuid4().hex
# 任务列补齐(与 ensure_automations_tables 的补列惯例同款):
# 裸库/老库的 tasks 表可能没有 executor 等列,INSERT 前补齐,
# 否则建任务直接报错;PG 适配层会翻译 PRAGMA 查询。
existing_tasks = _existing_columns(conn, "tasks")
for col, ddl in (
("executor", "TEXT"),
("model", "TEXT"),
("effort", "TEXT"),
("environment_id", "TEXT"),
("parent_task_id", "TEXT"),
("coordinator_part", "TEXT")):
if col not in existing_tasks:
last_exc = None
for _attempt in range(5):
try:
if gate is not None:
_audit_on_conn(
conn, "system", project_id,
"automation_run_quota_skipped",
"%s: %s" % (automation_id, gate), created_at=now)
return record_automation_run(
conn, automation_id, project_id, task_id=None,
status="skipped_quota", detail=gate, now=now)
# 任务列补齐(与 ensure_automations_tables 的补列惯例
# 同款):裸库/老库的 tasks 表可能没有 executor 等列,
# INSERT 前补齐,否则建任务直接报错;PG 适配层会翻译
# PRAGMA 查询。放在重试循环内:回滚会撤销未提交的
# 补列,重试时须重新查列补齐,不能只补一次。
existing_tasks = _existing_columns(conn, "tasks")
for col, ddl in (
("executor", "TEXT"),
("model", "TEXT"),
("effort", "TEXT"),
("environment_id", "TEXT"),
("parent_task_id", "TEXT"),
("coordinator_part", "TEXT")):
if col not in existing_tasks:
try:
conn.execute(
"ALTER TABLE tasks ADD COLUMN %s %s"
% (col, ddl))
except Exception:
pass
conn.execute(
"INSERT INTO tasks(id, project_id, prompt, status,"
" idempotency_key, depends_on, priority, created_by,"
" created_at, updated_at, last_heartbeat, executor,"
" model, effort, environment_id, parent_task_id,"
" coordinator_part)"
" VALUES(?, ?, ?, 'queued', NULL, '[]', 0, ?, ?, ?,"
" NULL, NULL, NULL, NULL, NULL, NULL, NULL)",
(task_id, project_id, prompt, owner, now, now))
_audit_on_conn(
conn, "system", project_id, "automation_run_created",
"%s -> %s" % (automation_id, task_id), created_at=now)
return record_automation_run(
conn, automation_id, project_id, task_id=task_id,
status="created", detail="", now=now)
except Exception as exc:
last_exc = exc
try:
conn.execute(
"ALTER TABLE tasks ADD COLUMN %s %s" % (col, ddl))
conn.rollback()
except Exception:
pass
conn.execute(
"INSERT INTO tasks(id, project_id, prompt, status,"
" idempotency_key, depends_on, priority, created_by,"
" created_at, updated_at, last_heartbeat, executor,"
" model, effort, environment_id, parent_task_id,"
" coordinator_part)"
" VALUES(?, ?, ?, 'queued', NULL, '[]', 0, ?, ?, ?,"
" NULL, NULL, NULL, NULL, NULL, NULL, NULL)",
(task_id, project_id, prompt, owner, now, now))
run = record_automation_run(
conn, automation_id, project_id, task_id=task_id,
status="created", detail="", now=now)
_audit_row(conn, project_id, "automation_run_created",
"%s -> %s" % (automation_id, task_id), now)
return run
if not _is_integrity_error(exc):
raise
raise last_exc
def list_automation_runs(conn, automation_id, limit=TRIAGE_LIMIT):
+9 -3
View File
@@ -66,9 +66,15 @@ from cloud.workers import DEFAULT_ALIVE_SECONDS as _WORKER_ALIVE_SECONDS
def queue_depth(db_path, executor=None):
"""当前就绪任务数:依赖已满足且状态为 queued 的任务。
依赖成环时 ready_tasks 抛 ValueError,视为 0(成环任务
本就派不出去,不应为它扩容)。executor 给定时只数该
执行器能认领的任务(未指定执行器、或指定为该执行器)。
依赖成环的任务不贡献深度,但不是因为 ready_tasks 抛
ValueError 后整体归 0:ready_tasks 先按“依赖全部已完成”
过滤(见 cloud/queue.py),成环任务的依赖从未完成、根本
进不了拓扑排序,所以既不抛错、也不拖累其它就绪任务——
5 个独立就绪任务加 1 对成环任务时深度仍为 5,只有纯成环
时深度才为 0(此时就绪集本身为空)。若 ready_tasks 因
其它原因抛 ValueError,本函数仍按原有兜底返回 0。
executor 给定时只数该执行器能认领的任务(未指定执行器、
或指定为该执行器)。
"""
conn = _db_connect(db_path)
try:
+4 -1
View File
@@ -32,8 +32,11 @@
import json
from cloud.timeouts import LONG_TIMEOUT_SECONDS as _LONG_TIMEOUT
MAX_PARTS = 20
DEFAULT_TIMEOUT_SECONDS = 3600
# 超时数值收敛到 cloud/timeouts.py(别名保留,调用点不变)。
DEFAULT_TIMEOUT_SECONDS = _LONG_TIMEOUT
DEFAULT_POLL_INTERVAL = 5.0
REPORT_RESULT_CHARS = 500
EFFORTS = {"low", "medium", "high", "xhigh", "max"}
+230 -1
View File
@@ -50,7 +50,6 @@ events only.
import json
import os
import sqlite3
from cloud.db import connect as _db_connect
import time
from datetime import datetime, timezone
@@ -136,6 +135,59 @@ def price_usd_micros(model, input_tokens, cached_input_tokens,
return int(round(usd * 1000000))
# 双列文案:逐字复用 PR #212(feat-cost-quota,5b30c61)的
# BILLING_NOTE,本轨不拥有成本口径、不另造一套说法。
BILLING_NOTE = (
"价格表代理价(经 CLOUD_MODEL_PRICES_JSON / CLOUD_MODEL_PRICES_FILE"
" 覆盖层),非中转实际账单;未定价 token 不计入美元、不等于免费。"
)
def quota_cutoff_reason(quota_usd_micros, used_usd_micros,
unpriced_used_tokens, month, unpriced_cap=None):
"""配额硬切断的唯一判定函数(worker 执行侧与单测共用)。
返回 None 表示未超限、可继续;返回 quota_exceeded 文本表示
必须立即停:调用方不再开新回合、不再派生花钱路径,任务以
interrupted + 该原因收尾(原因写进 error_text 并记审计),
不是只告警。quota 为 None(管理员显式不限额)时不挡;USD
配额与无价 token 上限是两道独立的门,任一超限即停。
文本格式与 worker 旧分支逐字一致,旧单测按此前缀断言。
"""
if quota_usd_micros is None:
return None
if int(used_usd_micros or 0) >= int(quota_usd_micros):
return ("quota_exceeded: 本月配额已用完(已用 %d / "
"配额 %d 微美元,%s)"
% (int(used_usd_micros or 0), int(quota_usd_micros),
month))
cap = unpriced_token_cap() if unpriced_cap is None \
else int(unpriced_cap)
if int(unpriced_used_tokens or 0) >= cap:
return ("quota_exceeded: 本月无价模型 token 已用完"
"(已用 %d / 上限 %d,%s)"
% (int(unpriced_used_tokens or 0), cap, month))
return None
def dual_column_view(report):
"""双列视图:在已有 _report 结果上加定价列别名与账单说明。
不改 _report(旧单测按其键集断言)与旧键:usd_micros 仍是
有价行之和,unpriced_actual_tokens 仍是未定价 actual token;
本函数只加 priced_usd_micros(与 usd_micros 同值)与
billing_note。未定价模型分列(by_model / unpriced_models)
与按创建者口径是 PR #212 的范围,本轨不重复实现——#212
合并后本函数由其 dual_column_* 取代,端点改走 dual_column_*
(同名同值键,切换无语义差);#212 未合并前,本视图是
任务/项目成本端点的双列最小实现。
"""
view = dict(report)
view["priced_usd_micros"] = view["usd_micros"]
view["billing_note"] = BILLING_NOTE
return view
def unpriced_token_cap():
"""Monthly unpriced-token ceiling for users who have a quota row.
@@ -398,3 +450,180 @@ class CostStore:
def report_by_day(self, day) -> dict:
"""Aggregate cost records for one UTC day (YYYY-MM-DD)."""
return self._report("day = ?", (day,))
def _report_joined(self, where, params):
"""_report 的 JOIN 版:按 tasks.created_by 归属汇总。
cost_records 本身没有创建者列,归属只能经 tasks JOIN;
协调员子任务行无 created_by 时跟父任务走的口径在
worker 侧,库层这里只按行上的 created_by 计——无属主
任务的成本不归任何人,不会被静默算进某个创建者。
"""
conn = self._new_conn()
try:
row = conn.execute(
"""SELECT
COALESCE(SUM(c.total_tokens), 0),
COALESCE(SUM(c.usd_micros), 0),
COUNT(*),
COALESCE(SUM(c.fallback_count), 0),
COALESCE(SUM(CASE WHEN c.source = 'actual'
THEN c.total_tokens ELSE 0 END), 0),
COALESCE(SUM(CASE WHEN c.source = 'estimated'
THEN c.total_tokens ELSE 0 END), 0),
COALESCE(SUM(c.cached_input_tokens), 0),
COALESCE(SUM(CASE WHEN c.source = 'actual'
AND c.usd_micros IS NULL
THEN c.total_tokens ELSE 0 END), 0),
COALESCE(SUM(c.completions), 0)
FROM cost_records c JOIN tasks t
ON c.task_id = t.id WHERE %s""" % where,
params,
).fetchone()
finally:
if conn is not self._conn:
conn.close()
(total_tokens, usd_micros, turns, fallback_count, actual,
estimated, cached, unpriced, completions) = row
return {
"total_tokens": int(total_tokens),
"usd_micros": int(usd_micros),
"turns": int(turns),
"fallback_count": int(fallback_count),
"actual_tokens": int(actual),
"estimated_tokens": int(estimated),
"cached_input_tokens": int(cached),
"unpriced_actual_tokens": int(unpriced),
"llm_requests": int(completions),
}
def report_by_creator(self, creator_id, month=None) -> dict:
"""按任务创建者汇总:当月(UTC,YYYY-MM)或全部。
month 为 None 时不过滤月份;month 格式须为 YYYY-MM,
否则 ValueError(调用方转 400)。返回 _report 同款键
外加 creator_id 与 month,方便分账报表直接按人对账。
"""
if month is not None:
if not isinstance(month, str) or len(month) != 7 \
or month[4] != "-":
raise ValueError("month must be YYYY-MM")
report = self._report_joined(
"t.created_by = ? AND c.day LIKE ?",
(creator_id, month + "%"))
else:
report = self._report_joined(
"t.created_by = ?", (creator_id,))
report["creator_id"] = creator_id
report["month"] = month
return report
def model_breakdown(self, task_id=None, project_id=None,
creator_id=None, month=None) -> list:
"""按模型分列:恰好指定 task / project / creator 其中一种。
每行:model、total_tokens、turns、llm_requests、
priced_usd_micros(该模型无任何有价行时为 None,不写 0)、
unpriced_tokens(source='actual' 且 usd IS NULL 的行,
估算行不计)、priced(是否有任一有价行)。未定价模型
由 unpriced_tokens > 0 识别,调用方用它单独列出,绝不
把 priced_usd_micros=None 的模型按 0 美元合并进合计。
creator 口径经 tasks JOIN,month 同 report_by_creator。
"""
scopes = [s for s in (task_id, project_id, creator_id)
if s is not None]
if len(scopes) != 1:
raise ValueError(
"exactly one of task_id, project_id, creator_id required")
join = ""
# 类型标注而已(A6):首分支会把 params 推断成单元素元组,
# 导致下面 params + (month,) 报 tuple[Any] vs tuple[Any, str];
# 标注成裸 tuple(即 tuple[Any, ...])后三分支与拼接均合法。
# 初值 () 永不被使用——if/elif/else 三分支必居其一重写它,
# 运行时语义不变。
params: tuple = ()
if task_id is not None:
where, params = "c.task_id = ?", (task_id,)
elif project_id is not None:
where, params = "c.project_id = ?", (project_id,)
else:
join = " JOIN tasks t ON c.task_id = t.id"
where, params = "t.created_by = ?", (creator_id,)
if month is not None:
if not isinstance(month, str) or len(month) != 7 \
or month[4] != "-":
raise ValueError("month must be YYYY-MM")
where += " AND c.day LIKE ?"
params = params + (month + "%",)
conn = self._new_conn()
try:
rows = conn.execute(
"""SELECT c.model,
COALESCE(SUM(c.total_tokens), 0),
COUNT(*),
COALESCE(SUM(c.completions), 0),
SUM(c.usd_micros),
SUM(CASE WHEN c.usd_micros IS NOT NULL
THEN 1 ELSE 0 END),
COALESCE(SUM(CASE WHEN c.source = 'actual'
AND c.usd_micros IS NULL
THEN c.total_tokens ELSE 0 END), 0)
FROM cost_records c%s WHERE %s
GROUP BY c.model
ORDER BY SUM(c.total_tokens) DESC, c.model"""
% (join, where),
params,
).fetchall()
finally:
if conn is not self._conn:
conn.close()
out = []
for row in rows:
(model, total, turns, completions, usd_sum, priced_rows,
unpriced) = row
out.append({
"model": model or "unknown",
"total_tokens": int(total),
"turns": int(turns),
"llm_requests": int(completions),
"priced_usd_micros": (int(usd_sum) if priced_rows else None),
"unpriced_tokens": int(unpriced),
"priced": bool(priced_rows),
})
return out
def _dual_column(self, base, breakdown):
"""双列报告:在 _report 结果上加第二列与分列明细。
_report 本身不动(旧单测按其精确键集断言);双列是
新方法返回的扩展:priced_usd_micros 与 usd_micros 同值
(只计有价行之和),unpriced_models 单独列出未定价
模型,billing_note 明确标注代理价、非中转实际账单。
"""
report = dict(base)
report["priced_usd_micros"] = report["usd_micros"]
report["unpriced_models"] = sorted(
row["model"] for row in breakdown
if row["unpriced_tokens"] > 0)
report["by_model"] = breakdown
report["billing_note"] = BILLING_NOTE
return report
def dual_column_report_by_task(self, task_id) -> dict:
"""单任务双列报告(见 _dual_column)。"""
return self._dual_column(
self.report_by_task(task_id),
self.model_breakdown(task_id=task_id))
def dual_column_report_by_project(self, project_id) -> dict:
"""单项目双列报告(见 _dual_column)。"""
return self._dual_column(
self.report_by_project(project_id),
self.model_breakdown(project_id=project_id))
def dual_column_report_by_creator(self, creator_id,
month=None) -> dict:
"""按创建者双列报告(见 _dual_column)。"""
return self._dual_column(
self.report_by_creator(creator_id, month=month),
self.model_breakdown(creator_id=creator_id, month=month))
+50 -4
View File
@@ -59,6 +59,7 @@ memory.py 的 summaries 完全一致,因为本模块根本不建表)。
"""
import os
import re as _re
import sqlite3
import time
from abc import ABC, abstractmethod
@@ -324,7 +325,9 @@ def _load_postgres_driver():
# (语义弱于 REPLACE,调用点只有 project_members 与
# slack_users 两张已知主键表,不会走到退化分支)。
# - ``INSERT OR IGNORE`` -> ``ON CONFLICT DO NOTHING``:无冲突
# 目标,任何唯一冲突都跳过,不改已有行。
# 目标,任何唯一冲突都跳过,不改已有行;VALUES 形与
# SELECT 形两种写法都翻译(SELECT 形漏翻曾致新鲜 PG 库
# init 失败,见 _OR_IGNORE_SELECT_RE)。
# - ``PRAGMA table_info(<表>)`` -> information_schema 查询,
# 返回 (cid, name, type, notnull, dflt_value, pk) 六元组,
# 与 sqlite 的 PRAGMA table_info 行形状一致,调用方取 row[1]
@@ -422,12 +425,18 @@ _OR_REPLACE_PK = {
"memory_settings": ("user_id",),
"task_memory_flags": ("task_id",),
"memory_extraction_log": ("task_id",),
# 审计循环状态单行表:主键 task_id。未登记时 Postgres 下
# INSERT OR REPLACE 退化为 ON CONFLICT DO NOTHING,已存在
# 行的 round / last_decision 等更新静默丢弃(2026-10-31
# PG 验证实测:跨进程竞态中包 round=2 而状态表 round 恒
# 为 1、轮次耗尽时 last_decision 仍为 pushback;SQLite
# 原生 OR REPLACE 测不出来,与自动记忆三张表同款坑)。
"task_audit_loop_state": ("task_id",),
}
_IDENTITY_TABLES = frozenset(
{"summaries", "kb_docs", "cost_records", "audit_log"})
import re as _re
{"summaries", "kb_docs", "cost_records", "audit_log",
"audit_ledger"})
_OR_REPLACE_RE = _re.compile(
r"^\s*INSERT\s+OR\s+REPLACE\s+INTO\s+(\w+)\s*\(([^)]+)\)\s*"
@@ -437,6 +446,13 @@ _OR_IGNORE_RE = _re.compile(
r"^\s*INSERT\s+OR\s+IGNORE\s+INTO\s+(\w+)\s*\(([^)]+)\)\s*"
r"VALUES\s*\((.*)\)\s*$",
_re.IGNORECASE | _re.DOTALL)
# INSERT OR IGNORE ... SELECT 形(VALUES 形之外的第二种写法,
# 如配额回填):PG 没有 OR IGNORE 语法,不翻译会直接报
# syntax error at or near "OR"(2026-10-31 新鲜 PG 库复现)。
_OR_IGNORE_SELECT_RE = _re.compile(
r"^\s*INSERT\s+OR\s+IGNORE\s+INTO\s+(\w+)\s*"
r"(\([^)]*\))?\s+SELECT\s+(.*)$",
_re.IGNORECASE | _re.DOTALL)
_INSERT_TABLE_RE = _re.compile(
r"^\s*INSERT\s+INTO\s+(?:\w+\.)?(\w+)", _re.IGNORECASE)
_PRAGMA_INFO_RE = _re.compile(
@@ -506,6 +522,22 @@ def _translate_sql(sql):
"INSERT INTO %s (%s) VALUES (%s) ON CONFLICT DO NOTHING"
% (table, cols, values))
return _convert_placeholders(translated), "normal"
m = _OR_IGNORE_SELECT_RE.match(stripped)
if m:
# INSERT OR IGNORE ... SELECT 形 -> INSERT INTO ... SELECT
# ... ON CONFLICT DO NOTHING:与 VALUES 形同款语义(任何
# 唯一冲突都跳过、不改已有行)。旧版只认 VALUES 形,SELECT
# 形原样透传到 PG,直接报 syntax error at or near "OR";
# 配额回填就是 SELECT 形,init_db 里它的 except 又把整个
# 事务 rollback,migrate 的 ALTER 一并被丢掉(见 server
# _backfill_default_quotas 的保存点注释)。
table = m.group(1)
cols = m.group(2) or ""
select_body = m.group(3).rstrip().rstrip(";")
translated = (
"INSERT INTO %s %s SELECT %s ON CONFLICT DO NOTHING"
% (table, cols, select_body))
return _convert_placeholders(translated), "normal"
translated = _AUTOINCREMENT_RE.sub(
"BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY", sql)
translated = _convert_placeholders(translated)
@@ -805,6 +837,19 @@ POSTGRES_SCHEMA = (
"action TEXT, "
"detail TEXT, "
"created_at DOUBLE PRECISION)",
# 防篡改账本(哈希链,第 4 轨):与 cloud/ledger.py 的 SQLite
# DDL 同表同列;prev/entry_hash UNIQUE 是并发抢链头的
# fail-closed 基础,绝不能去掉。
"CREATE TABLE IF NOT EXISTS audit_ledger("
"id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, "
"user_id TEXT, "
"project_id TEXT, "
"task_id TEXT, "
"action TEXT, "
"detail TEXT, "
"created_at DOUBLE PRECISION, "
"prev_hash TEXT NOT NULL UNIQUE, "
"entry_hash TEXT NOT NULL UNIQUE)",
# P0 三件:断点与步骤事实源(PG 新库一次建齐;老库走
# server/worker 的 ensure 幂等补表,不丢数据)。
# task_checkpoints:每轮断点快照(worker 镜像线程 + 终局
@@ -1116,6 +1161,7 @@ def init_postgres_schema(conn):
for _name, _type in (("executor", "TEXT"), ("pr_url", "TEXT"),
("pr_number", "INTEGER"),
("worktree_host", "TEXT"),
("claimed_by_worker", "TEXT"),
("goal_mode", "INTEGER"),
("environment_id", "TEXT"),
("child_model", "TEXT"),
+154
View File
@@ -0,0 +1,154 @@
"""发布证据的新鲜度与单次使用绑定(集成 v2,代码层)。
首轮缺口(全自动 9.5 扣 0.5):发布门“重算”保证的是“不用
旧结论”,不是“证据一定新鲜”——证据体里没有时间戳/nonce
绑定,调用方把合并时同一份证据体在发布时原样重交,同样
能通过重算。本模块补这一层,但诚实边界先写清:
- 本绑定防的是**原样重放**(同一 nonce 二次提交)与**陈旧
证据**(时间戳超出窗口),不防**重盖戳**:没有签名的证据
体,调用方完全可以给旧内容盖一个新时间戳/新 nonce 再交,
本模块无法识破。防篡改/防重盖戳需要签名证据或第 5 项的
只读求证(向 GitHub 实时核对),那是另一层控制,不在本
模块冒充。
- 绑定是**调用方自愿出示**的:请求体里同时带 evidence_ts
与 evidence_nonce 才进入绑定校验;两者都不带时走旧路径
(与 #259 契约兼容,响应里标注 evidence_binding=absent,
绝不谎称已验证新鲜)。只带其一是格式错误(400),不是
“半绑定通过”。把绑定改成强制,是契约决定,须协调员/
用户在合轨时定,不在本轨悄悄改 #259。
- 单次使用语义:nonce 在**首次通过校验的提交**即被消费,
无论该次发布最终是 200 还是被其它门(未 merged、推回
阻塞)挡下——被挡的调用方下次必须取新证据重盖戳。这
是故意的:每次发布尝试都要求当次新鲜证据,而不是“一份
证据可反复试到放行”。
- nonce 存储与审计链无关:evidence_nonces 表只记 nonce、
任务 id 与时间戳,不写 audit_log、不移动 #241 链头;409
不写审计的规则不受影响。表在写路径(首次消费)才建,
读路径(校验已存在)不建表——查不到表时视为无消费记
录,不是错误。
零第三方依赖,只用标准库。时间一律用 Unix 秒(time.time),
与 full_auto.audit_record 的 ts 同口径。
"""
import time
import uuid
# 新鲜度窗口:证据时间戳距提交时刻不得超过 1 小时;未来
# 容忍 5 分钟时钟偏差,超出即视为异常(不是“更新鲜所以更好”)。
MAX_AGE_SECONDS = 3600
CLOCK_SKEW_SECONDS = 300
NONCE_TABLE_DDL = (
"CREATE TABLE IF NOT EXISTS evidence_nonces("
"nonce TEXT NOT NULL PRIMARY KEY, "
"task_id TEXT NOT NULL, "
"evidence_ts DOUBLE PRECISION NOT NULL, "
"consumed_at DOUBLE PRECISION NOT NULL)"
)
def bind_evidence(evidence, now=None, nonce=None):
"""给一份证据体盖戳:返回原证据的浅拷贝 + 时间戳/nonce。
调用方(协调员侧)在**取到证据的那一刻**调用本函数,
不要在合并时盖好、发布时原样重交——那样时间戳本身
就是旧的,会被窗口校验挡下(这正是本绑定的意义)。
nonce 缺省为 uuid4 hex;调用方自带 nonce 时须保证
全局唯一(重复 nonce 即重放)。
"""
if not isinstance(evidence, dict):
raise ValueError("evidence must be a dict")
stamped = dict(evidence)
stamped["evidence_ts"] = time.time() if now is None else now
stamped["evidence_nonce"] = nonce or uuid.uuid4().hex
return stamped
def validate_bound_evidence(evidence, now=None):
"""纯函数校验绑定证据的形状与新鲜度(不查 nonce 库)。
返回 (ok, reason):ok 为 False 时 reason 为
evidence_binding_missing / evidence_binding_incomplete /
evidence_ts_not_a_number / evidence_nonce_empty /
evidence_too_old / evidence_from_future。nonce 重放由
EvidenceNonceStore.consume 判定,不在本函数(本函数
无状态,同一份证据调两次结果相同,那不是重放判定)。
"""
if not isinstance(evidence, dict):
return False, "evidence_binding_missing"
has_ts = evidence.get("evidence_ts") is not None
has_nonce = evidence.get("evidence_nonce") is not None
if not has_ts and not has_nonce:
return False, "evidence_binding_missing"
if not (has_ts and has_nonce):
return False, "evidence_binding_incomplete"
ts = evidence.get("evidence_ts")
if isinstance(ts, bool) or not isinstance(ts, (int, float)):
return False, "evidence_ts_not_a_number"
nonce = evidence.get("evidence_nonce")
if not isinstance(nonce, str) or not nonce.strip():
return False, "evidence_nonce_empty"
moment = time.time() if now is None else now
if ts > moment + CLOCK_SKEW_SECONDS:
return False, "evidence_from_future"
if moment - ts > MAX_AGE_SECONDS:
return False, "evidence_too_old"
return True, None
def binding_presence(evidence):
"""绑定出示状态:bound(成对出示)/ partial(只带其一)/
absent(两者皆无)。服务端据此决定走绑定校验、报 400
还是走旧路径(并标注 absent,绝不谎称已验证)。"""
if not isinstance(evidence, dict):
return "absent"
has_ts = evidence.get("evidence_ts") is not None
has_nonce = evidence.get("evidence_nonce") is not None
if has_ts and has_nonce:
return "bound"
if has_ts or has_nonce:
return "partial"
return "absent"
class EvidenceNonceStore:
"""nonce 单次使用存储(DB 连接由调用方传入并负责提交)。
consume 成功返回 True(首次见此 nonce);nonce 已存在
返回 False(重放)。表不存在时 consume 先建表(写路径
才建);has() 在表不存在时返回 False,不建表。
"""
def __init__(self, conn):
self.conn = conn
def ensure_table(self):
self.conn.execute(NONCE_TABLE_DDL)
def has(self, nonce):
try:
row = self.conn.execute(
"SELECT nonce FROM evidence_nonces WHERE nonce = ?",
(nonce,)).fetchone()
except Exception:
return False
return row is not None
def consume(self, task_id, nonce, evidence_ts, consumed_at=None):
"""消费一个 nonce:首次 True;已存在 False(不覆盖原行)。
调用方须在 consume 后提交(本函数不 commit:服务端
在自己的连接生命周期里提交,与其余写操作保持一致)。
"""
self.ensure_table()
if self.has(nonce):
return False
self.conn.execute(
"INSERT INTO evidence_nonces(nonce, task_id, evidence_ts,"
" consumed_at) VALUES(?, ?, ?, ?)",
(nonce, task_id, evidence_ts,
time.time() if consumed_at is None else consumed_at),
)
return True
+402
View File
@@ -0,0 +1,402 @@
"""全自动编程系统:一次授权后的主机自动执行器(API 优先)。
本模块是“协调员只评审、人/宿主执行合并”缺口的闭合点,但
执行人不是任务 agent:任务 agent 仍然不直接执行合并/部署,
真正执行的是宿主侧的本模块(由已验证的 GitHub App
installation + 显式开启标志门控),任务 agent 只交分支、
自测证据和合并决定。两者混淆就是冒充,必须分清。
设计约束(全部是硬规则,不是建议):
1. 默认关闭:FULL_AUTO_ENABLED 未显式开启,或 installation
未验证,执行器一律拒绝,不存在“默认自动合”的状态。
2. 禁止清单 fail-closed:删库、force-push(含主分支)、改仓库
可见性、读密钥、外泄密钥、删分支保护、绕分支保护,未知
动作一律拒绝,不靠调用方自觉。
3. token 只进子进程/HTTP 请求头,不进审计记录、不进日志、
不进命令行、不进文件。本模块的审计记录在写入前会把
含 token/secret/password/key 的键和 Bearer 字符串抹掉;
调用方若把 token 塞进 detail,同样会被抹掉(宁可丢
字段,不可漏 token)。
4. 主分支保护强制开:自动配置的分支保护必须同时含要求 PR、
要求状态检查(strict)、禁止 force-push、禁止删除、
管理员同样受约束(enforce_admins),缺一项就不算配好。
5. 质量门禁阻塞合并:CI 全绿 + 目录隔离检查 + 契约版本 +
协调员批准,四项缺一就不合并。门禁不过时走 CI 自动修复
循环,限次(默认 3 次),超次升级给用户,不无限重试。
6. 回滚不靠 force-push:回滚只能是 revert PR 或重新部署
上一个 tag,绝不 force-push 主分支。
7. 建仓分支要诚实:GitHub App installation token 能建的是
组织仓库(POST /orgs/{org}/repos,需 administration 写
权限);用户个人账号仓库的 POST /user/repos 需要用户
token,installation token 做不了。所以自动配置计划里,
“仓库不存在 + 无 org”时返回 needs_user_once 步骤:用户
在同一次安装会话里先建一个空仓(这是安装会话的一部分,
不是此后每轮的人工步骤),App 以 All repositories 安装
后即覆盖它。此后建分支、PR、CI、合并、发版全部走 API,
不再需要人工。把这一步说成“App 已自动建仓”就是冒充,
必须如实记为 needs_user_once。
8. 绝不做网页自动化自动注册 GitHub 账号:服务条款禁止
自动化创建账号,注册有验证码、邮箱验证、可能的人机/
手机验证,绕验证码会触发封号。本模块不提供、不调用
任何账号注册自动化,浏览器自动化只留作 API 覆盖不到
的配置页面兜底,不用来注册、不用来绕验证码。
9. 授权分层:控制台 OAuth 只证明本站身份,不带来 GitHub
权限;GitHub 权限靠 GitHub App 安装,仓库范围靠安装时
选 All repositories 一次覆盖现有与未来仓库,能力范围受
App 声明的权限封顶,新增权限必须改 App 声明并由安装方
在 GitHub 重新接受,无法静默升级。本模块不假装权限
可以自动升级:installation 的 permissions 由调用方
传入并核对,缺权限的动作直接拒绝并记为
needs_reaccept(需安装方重新接受),不静默降级执行。
零第三方依赖,只用 Python 标准库。网络一律经调用方注入
的 request_fn(约定与 cloud/github_app._github_api_request
的 http_client 相同:(method, url, headers, body) ->
(status, dict)),本模块自身不发真实网络请求,单测用假
客户端即可覆盖全部路径;真实路径由调用方复用
github_app 的 urllib 实现并复用 resolve_app_token 的
“token 只进子进程环境/请求头”模式。
"""
import json
import os
import time
MAX_FIX_ATTEMPTS = 3
# 允许的自动动作(白名单,之外一律拒绝)。
ALLOWED_ACTIONS = frozenset((
"create_repo_org",
"create_branch",
"push_branch",
"create_pr",
"merge_pr",
"create_release",
"rollback_to_tag",
"set_branch_protection",
"set_default_branch",
"add_ci_workflow",
"verify_webhook",
))
# 禁止的自动动作(黑名单,命中即拒绝,优先级高于白名单)。
FORBIDDEN_ACTIONS = frozenset((
"delete_repo",
"force_push",
"force_push_main",
"change_visibility",
"read_secret",
"expose_secret",
"delete_branch_protection",
"bypass_branch_protection",
"register_github_account",
))
_SENSITIVE_KEYS = ("token", "secret", "password", "private_key",
"api_key", "apikey", "auth")
class FullAutoError(Exception):
"""全自动执行失败:reason 见各抛出处。"""
def __init__(self, message, reason=None):
super().__init__(message)
self.reason = reason
class FullAutoDenied(FullAutoError):
"""护栏拒绝:默认关闭、禁止动作、未知动作、权限不足。"""
def is_full_auto_enabled(env=None, installation_verified=False):
"""执行器是否开启:显式标志 + 已验证 installation,两者缺一不可。
env 默认读 os.environ(调用方可注入 dict 做单测)。标志值
只认 1/true/yes(大小写不敏感),其它值一律视为关闭。
"""
if env is None:
env = os.environ
raw = env.get("FULL_AUTO_ENABLED", "")
flag = str(raw).strip().lower() in ("1", "true", "yes")
return bool(flag and installation_verified)
def validate_action(action):
"""校验自动动作:禁止/未知一律抛 FullAutoDenied,不返回布尔
让调用方“顺手忽略”。"""
if action in FORBIDDEN_ACTIONS:
raise FullAutoDenied("forbidden action: %s" % action,
reason="forbidden_action")
if action not in ALLOWED_ACTIONS:
raise FullAutoDenied("unknown action: %s" % action,
reason="unknown_action")
return True
def _redact_value(value):
"""递归抹掉敏感内容:键含 token/secret/password/key 的
一律删键;字符串里的 Bearer <...> 一律替换为 Bearer [redacted]。
"""
if isinstance(value, dict):
out = {}
for key, item in value.items():
lowered = str(key).lower()
if any(s in lowered for s in _SENSITIVE_KEYS):
continue
out[key] = _redact_value(item)
return out
if isinstance(value, (list, tuple)):
return [_redact_value(item) for item in value]
if isinstance(value, str):
if "Bearer " in value:
parts = value.split("Bearer ", 1)
tail = parts[1].split()[0] if parts[1].split() else ""
return value.replace("Bearer " + tail,
"Bearer [redacted]")
return value
return value
def audit_record(action, repo, actor, result, detail=None):
"""生成审计记录 dict:不含任何 token,调用方可直接落盘。
actor 建议写 full-auto-executor + installation id,不写
任务 agent 的名字,避免把执行人冒充成任务 agent。"""
record = {
"ts": time.time(),
"action": action,
"repo": repo,
"actor": actor,
"result": result,
"detail": _redact_value(detail) if detail is not None else {},
}
return record
def append_audit_log(path, record):
"""审计记录追加写 JSONL:父目录不存在时创建;记录本身
已在 audit_record 里脱敏,本函数不再二次处理 token,
但调用方若直接传了含 token 的 dict,本函数不负责兜底,
必须走 audit_record 生成记录。"""
directory = os.path.dirname(os.path.abspath(str(path)))
if directory:
os.makedirs(directory, exist_ok=True)
with open(path, "a", encoding="utf-8") as handle:
handle.write(json.dumps(record, ensure_ascii=False) + "\n")
return record
def branch_protection_payload(required_checks):
"""主分支保护 payload:要求 PR + 要求状态检查(strict)+
禁止 force-push + 禁止删除 + 管理员同样受约束,缺一不可。
required_checks 为空时不算配好(调用方必须传 CI 检查名,
不能靠“无检查即通过”蒙混)。"""
checks = [str(c) for c in (required_checks or []) if str(c)]
if not checks:
raise FullAutoError("required_checks must not be empty",
reason="branch_protection_incomplete")
return {
"required_status_checks": {
"strict": True,
"contexts": checks,
},
"enforce_admins": True,
"required_pull_request_reviews": {
"required_approving_review_count": 1,
},
"restrictions": None,
"allow_force_pushes": False,
"allow_deletions": False,
}
def auto_config_plan(repo_full_name, default_branch="main", org=None,
repo_exists=True, ci_workflow_present=False,
required_checks=("verify",)):
"""一次授权后 API 自动配置的顺序计划(不发网络请求)。
每步含 step / api(None 表示需人工一次或仅核对)/
method / path / payload_summary / manual:
manual=None 表示全自动;manual="user_once" 表示该步
是安装会话里唯一的人工步(仅建仓分支会出这一步);
manual="verify_only" 表示仅核对、不创建(App 级 webhook
已在 App manifest 的 hook_attributes 里配好,仓库级
不再重复建 hook,重复建反而会双发)。
调用方按顺序执行,遇失败即停,不跳步继续。
"""
if not repo_full_name or "/" not in str(repo_full_name):
raise FullAutoError("repo_full_name must be owner/repo",
reason="invalid_repo")
repo = str(repo_full_name)
branch = str(default_branch or "main")
steps = []
if not repo_exists:
if org:
steps.append({
"step": "ensure_repo",
"api": "POST /orgs/{org}/repos",
"method": "POST",
"path": "/orgs/%s/repos" % org,
"payload_summary": {
"name": repo.split("/", 1)[1],
"private": True,
},
"manual": None,
})
else:
steps.append({
"step": "ensure_repo",
"api": None,
"method": None,
"path": None,
"payload_summary": {
"reason": "installation token cannot create a "
"user-account repo via POST /user/repos; "
"that endpoint needs a user token",
},
"manual": "user_once",
})
steps.append({
"step": "set_default_branch",
"api": "PATCH /repos/{owner}/{repo}",
"method": "PATCH",
"path": "/repos/%s" % repo,
"payload_summary": {"default_branch": branch},
"manual": None,
})
steps.append({
"step": "set_branch_protection",
"api": "PUT /repos/{owner}/{repo}/branches/{branch}/protection",
"method": "PUT",
"path": "/repos/%s/branches/%s/protection" % (repo, branch),
"payload_summary": branch_protection_payload(required_checks),
"manual": None,
})
if not ci_workflow_present:
steps.append({
"step": "add_ci_workflow",
"api": "PUT /repos/{owner}/{repo}/contents/.github/workflows/host-verify.yml",
"method": "PUT",
"path": "/repos/%s/contents/.github/workflows/host-verify.yml"
% repo,
"payload_summary": {"message": "chore: add host-verify CI"},
"manual": None,
})
steps.append({
"step": "verify_webhook",
"api": None,
"method": None,
"path": None,
"payload_summary": {
"reason": "webhook is configured at App level via "
"manifest hook_attributes; per-repo hook "
"creation would double-deliver",
},
"manual": "verify_only",
})
return steps
def run_plan(request_fn, token, plan):
"""顺序执行自动配置计划:request_fn 约定同 github_app 的
http_client((method, url, headers, body) -> (status, dict))。
token 只放进本次请求的 Authorization 头,不进返回值、
不进审计记录。manual 步跳过并记为 needs_user_once /
verify_only;任一 API 步非 2xx 即停并抛 FullAutoError,
不跳步继续(fail-closed)。"""
if not token:
raise FullAutoDenied("installation token required",
reason="token_missing")
results = []
for step in plan:
if step.get("manual") in ("user_once", "verify_only"):
results.append({"step": step["step"],
"status": step["manual"],
"ok": step["manual"] == "verify_only"})
continue
action_map = {
"ensure_repo": "create_repo_org",
"set_default_branch": "set_default_branch",
"set_branch_protection": "set_branch_protection",
"add_ci_workflow": "add_ci_workflow",
"verify_webhook": "verify_webhook",
}
validate_action(action_map.get(step["step"], step["step"]))
headers = {"Accept": "application/vnd.github+json",
"User-Agent": "codex-app-server",
"Authorization": "Bearer " + token}
body = None
payload = step.get("payload_summary")
if payload is not None:
body = json.dumps(payload).encode("utf-8")
url = "https://api.github.com" + str(step["path"])
status, data = request_fn(step["method"], url, headers, body)
ok = 200 <= status < 300
results.append({"step": step["step"], "status": status,
"ok": ok})
if not ok:
raise FullAutoError(
"auto-config step %s failed: %s %r"
% (step["step"], status, data.get("message")),
reason="auto_config_step_failed")
return results
def merge_gate(checks, required_checks, directory_check_ok,
contract_version_ok, coordinator_approved):
"""合并门:四项全过才返回 (True, []),否则返回
(False, [未过原因...])。checks 是 {检查名: 状态},
状态只认 success;pending/failure/缺失一律算未过。
本函数不读 CI、不调网络,调用方把 pr/checks 的结果
传进来即可,测试与生产走同一判定。"""
reasons = []
checks = checks or {}
for name in (required_checks or []):
if checks.get(name) != "success":
reasons.append("check not green: %s" % name)
if not directory_check_ok:
reasons.append("directory isolation check failed")
if not contract_version_ok:
reasons.append("contract version mismatch")
if not coordinator_approved:
reasons.append("coordinator approval missing")
return (not reasons, reasons)
def ci_fix_decision(attempt, failures):
"""CI 自动修复循环决策:attempt 从 1 开始计数。
1..MAX_FIX_ATTEMPTS 返回 fix_and_rerun;超次返回
escalate(升级给用户),绝不无限重试烧模型费。"""
failed = [str(f) for f in (failures or [])]
if attempt <= MAX_FIX_ATTEMPTS:
return {"action": "fix_and_rerun", "attempt": attempt,
"failures": failed}
return {"action": "escalate",
"reason": "max_fix_attempts_exceeded",
"attempt": attempt, "failures": failed}
def release_plan(tag, merged_prs):
"""发版计划:release notes 从已合 PR 的标题聚合,不编造
未合 PR;回滚只能是 revert PR 或重新部署上一 tag,
绝不 force-push。merged_prs 是 [{number, title}]。"""
notes = ["## Merged PRs"]
for pr in (merged_prs or []):
notes.append("- #%s %s" % (pr.get("number"), pr.get("title")))
return {
"tag": tag,
"release_notes": "\n".join(notes),
"rollback": "revert PR or redeploy previous tag; "
"never force-push the default branch",
}
def rollback_plan(previous_tag):
"""回滚计划:只认上一 tag,不接受 force-push 式回滚。"""
if not previous_tag:
raise FullAutoError("previous_tag required for rollback",
reason="rollback_no_tag")
return {
"action": "rollback_to_tag",
"previous_tag": previous_tag,
"method": "redeploy previous tag (no force-push)",
}
+435
View File
@@ -0,0 +1,435 @@
"""合并/发布门禁证据的只读核对器(集成 v2,代码层)。
首轮缺口(门禁与审计 9.0 扣 0.5):硬门禁与发布门禁强制
的是“调用方提交的证据齐全且全绿”,不是“服务端已向
GitHub 求证”——调用方谎报 CI 全绿 / 保护状态 / 协调员
批准时,门禁会放行;docstring 称覆盖控制是“审计与
PR/Checks 的只读核对”,但在合并/发布路径上没有这样
一段“调 GitHub Checks API 核对证据”的代码(#210 的
实时保护检查只属于 auto-config execute 端点,不属于
merge/release 端点)。
本轨的结论分两半,缺一不可:
A. 核对器本身已实现(本模块,纯函数 + 注入式 reader,
有测试):verify_gate_evidence(evidence, reader) 把
调用方证据与 reader 给出的实时数据逐项比对——CI
状态、每项 required 检查的状态、分支保护状态;不
一致即 mismatch(逐条列出),一致即 verified。reader
是调用方注入的 callable:reader(repo, ref) -> {
"ci_checks": {name: status}, "branch_protection":
"protected"|"unprotected"|... },真实 reader 用
github_app 的 _github_api_request 走 GET Checks /
GET 分支保护(只读,绝不写 PR、不改保护、不合并)。
reader 为 None 或抛异常时结果是“未核对”
(checked=False, verified=False),绝不是“通过”——
调用方把 skipped 当 pass 用,是调用方的 bug,本
模块的返回值结构让这种误用一眼可见(verified 恒
False,除非逐项比对全过)。
B. 实时求证在当前合并/发布路径上**接不上**(不可实现
证明,不是未做):① 请求体里没有 repo 与 ref——
merge/release 的证据字段只有 files_changed、ci_checks、
branch_protection 等,没有仓库名、没有提交 SHA,
reader 不知道向哪个仓库的哪次提交求证,猜仓库
就是编造求证;② 服务端没有可信的求证通道保证——
github_app.resolve_app_token 对未安装 App 的仓库
返回 None(调用方回落 PAT),PAT 是调用方自己的
凭证,用它“核对”调用方自己的证据不是独立求证;
③ 任务行里没有 repo/ref 列,服务端无法从任务 id
反推求证目标。接上实时求证须先改契约(请求体或任
务行加 repo + ref,安装保证或服务端侧可信 token),
那是契约决定,须协调员/用户在合轨时定;本轨不编
一个“看起来在求证”的假 reader、不把调用方证据
自己与自己比对后写成 verified。
当前真实控制(写清,不拔高):门禁强制证据齐全全绿
(硬模式 fail-closed);审计记 outcome 与 blocks
(merge_gate_blocked / merge_task / release_task),
但审计 detail 不保存证据体全文——事后追责能查到
“谁在何时以何 blocks 通过/被挡”,查不到“当时提交
的 ci_checks 原文”。谎报证据的发现仍靠事后审计与
GitHub 侧记录比对,不是当场发现。本模块的核对器
是为契约补齐后的接线准备的:届时服务端只需注入
真实 reader 并把 verified=False(无论 skipped 还是
mismatch)当作阻断,核对器本身不用改。
零第三方依赖,只用标准库。本模块不 import cloud
其它模块(reader 由调用方注入),不发网络请求。
v7 机制第 6 项(生产证据 fail-closed live 接线,本批新增,
只动本模块,不动其它 7 项):
- 新增可注入 reader 接口 LiveEvidenceReader(协议):
get_checks(repo, ref) 与 get_branch_protection(repo,
branch),返回结构含状态与原始证据(LiveChecksResult /
LiveProtectionResult,或等价 dict)。本模块的生产代码
里绝不实现真实 HTTP 调用、绝不读真实 token、绝不调
用真实 GitHub API——接口是未来真实 reader 的插入缝,
不是真实 reader 本身。
- 新增 verify_live_evidence(claimed, reader, repo, ref,
branch=None):协议 reader 或旧式 callable reader 均可;
无 reader、无 repo/ref、reader 失败、reader 返回与声
称不一致时 verified 一律 False;声称 CI 全绿但实时
数据失败/缺失时记 mismatch。比对规则与
verify_gate_evidence 逐项一致(共用 _compare),不另
造口径;空声称不由本函数放行——缺失证据由硬门禁
(merge_gate_hard)阻断,本函数只核对“声称了的项”。
- 新增 live_gate_decision:端点层判定函数,verified
为 False 时返回 allowed=False、http_status=409、
status="blocked"、blocked=True(mismatch 与未核对用
不同 code 区分:live_evidence_mismatch /
live_evidence_unverified);verified 为 True 时返回
allowed=True、http_status=200、status="allowed"。
- 诚实残留(不得冒充已接线):HTTP 端点 _merge_task /
_release_task 本批未接 live_gate_decision——当前无
生产 reader 可注入:以 reader=None 接线会挡住全部合
并/发布(fail-closed 的本意不是停摆),以桩 reader
接线则是假求证。真实 live 验证仍待 B6 真实 App 安装
与 B1 等用户侧事项,未在本批完成。本批不改 #238
auto_apply(仍 False)、不改 #247“纠正为 pass 本身
不解封”、不改 v1.1 契约、不改任何阈值(阈值仍
test_default_unapproved)。
"""
from dataclasses import dataclass
try:
from typing import Protocol
from typing import runtime_checkable
except ImportError: # pragma: no cover - Python < 3.8
Protocol = object # type: ignore
def runtime_checkable(cls): # type: ignore[misc]
return cls
def _compare_claimed_vs_live(evidence, live):
"""声称证据与实时数据的逐项比对(两核对函数共用,同一口径)。
只比对调用方声称的项;实时数据缺失某项时记 mismatch
(声称全绿却查无此项,不是“查不到所以通过”)。
返回 mismatches 列表(空即逐项全过)。
"""
mismatches = []
live_checks = live.get("ci_checks") or {}
if not isinstance(live_checks, dict):
live_checks = {}
evidence = evidence or {}
if evidence.get("ci_status") == "success":
failed_live = sorted(
name for name, status in live_checks.items()
if status != "success")
if failed_live:
mismatches.append(
"ci_not_green_live: %s" % ", ".join(failed_live))
for check in (evidence.get("ci_checks") or []):
if not isinstance(check, dict):
continue
name = check.get("name")
if not name:
continue
if check.get("status") == "success" \
and live_checks.get(name) != "success":
mismatches.append(
"check_not_green_live: %s" % name)
for name in (evidence.get("required_checks") or []):
if live_checks.get(name) != "success":
mismatches.append(
"required_check_not_green_live: %s" % name)
claimed_protection = evidence.get("branch_protection")
live_protection = live.get("branch_protection")
if claimed_protection == "protected" \
and live_protection != "protected":
mismatches.append(
"branch_protection_not_protected_live: %s"
% live_protection)
return mismatches
def verify_gate_evidence(evidence, reader=None, repo=None, ref=None):
"""只读核对调用方门禁证据与实时数据。
evidence:调用方提交的门禁证据 dict(与硬门禁同款
字段:ci_status、ci_checks[{name, status}]、
required_checks、branch_protection)。
reader:callable (repo, ref) -> {"ci_checks": {名: 状态},
"branch_protection": str},或 None(无可信求证通道)。
repo/ref:求证目标;reader 非 None 但 repo/ref 缺失
时同样视为未核对(不猜目标)。
返回 {"checked": bool, "verified": bool, "mismatches": [...],
"reason": str|None}:
- checked=False:未核对(无 reader、无目标、reader 失败),
verified 恒 False,reason 说明原因;调用方绝不能把
checked=False 当通过。
- checked=True 且 mismatches 为空:逐项比对全过,
verified=True。
- checked=True 且 mismatches 非空:证据与实时数据不
一致,verified=False,mismatches 逐条说明(ci 状态、
某项检查、分支保护)。
比对规则:只比对调用方声称的项——调用方没声称的
检查不判(缺失由硬门禁的证据缺失码处理,不归本
模块);实时数据缺失某项时记 mismatch(声称全绿却
查无此项,不是“查不到所以通过”)。
"""
if reader is None:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "no_trusted_reader"}
if not repo or not ref:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "no_verification_target"}
try:
live = reader(repo, ref)
except Exception as exc:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: %s" % exc}
if not isinstance(live, dict):
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: non-dict result"}
mismatches = _compare_claimed_vs_live(evidence, live)
return {"checked": True, "verified": not mismatches,
"mismatches": mismatches, "reason": None}
@dataclass(frozen=True)
class LiveChecksResult:
"""get_checks 的返回结构:状态 + 原始证据。
checks:{检查名: 状态},状态与声称口径一致
("success" 为绿,其它一律非绿)。
raw:原始证据体(真实 reader 未来填 GitHub Checks
API 的原始 payload;假 reader 可填任意可追溯值,
也可为 None)。比对只用 checks,raw 只供审计追溯,
绝不参与“通过”判定。
"""
checks: dict
raw: object = None
@dataclass(frozen=True)
class LiveProtectionResult:
"""get_branch_protection 的返回结构:状态 + 原始证据。
status:"protected" / "unprotected" / 其它;None 表
示实时数据无此项(声称 protected 时即 mismatch)。
raw:原始证据体(真实 reader 未来填分支保护 API
的原始 payload;假 reader 可填任意可追溯值,也可
为 None),同样只供追溯、不参与判定。
"""
status: object = None
raw: object = None
@runtime_checkable
class LiveEvidenceReader(Protocol):
"""可注入的实时证据 reader 接口(v7 机制第 6 项)。
生产代码只定义接口、不实现真实 reader:真实实现
(走 GitHub Checks / 分支保护只读接口)须在 B6 真
实 App 安装与 B1 等用户侧事项就绪后另行接入,本
模块绝不内置 HTTP 调用、token 读取或 GitHub API
调用。测试与验收一律用实现本协议的假 reader。
"""
def get_checks(self, repo, ref):
"""返回 LiveChecksResult(或等价 dict,见
_normalize_checks):repo 在 ref 处的 CI 检查
实时状态与原始证据。"""
raise NotImplementedError
def get_branch_protection(self, repo, branch):
"""返回 LiveProtectionResult(或等价 dict / 纯
状态字符串,见 _normalize_protection):repo 的
branch 分支保护实时状态与原始证据。"""
raise NotImplementedError
def _normalize_checks(result):
"""归一化 get_checks 返回值 -> (checks dict, raw) 或 None。
接受 LiveChecksResult、含 "checks"/"ci_checks" dict
的 dict、以及本身就是 {名: 状态} 的 dict;其它形状
(非 dict checks、非字符串状态)返回 None,调用方
按 reader_failed 处理,不猜、不放行。
"""
if isinstance(result, LiveChecksResult):
checks, raw = result.checks, result.raw
elif isinstance(result, dict):
raw = result.get("raw")
if isinstance(result.get("checks"), dict):
checks = result["checks"]
elif isinstance(result.get("ci_checks"), dict):
checks = result["ci_checks"]
else:
checks = result
else:
return None
if not isinstance(checks, dict):
return None
if any(not isinstance(status, str) for status in checks.values()):
return None
return checks, raw
def _normalize_protection(result):
"""归一化 get_branch_protection 返回值。
返回 (status, raw, malformed):status 为字符串或
None(无此项);malformed 为 True 时调用方按
reader_failed 处理(例如返回了整数等非法形状)。
纯字符串即状态本身;无 "status"/"branch_protection"
键的 dict 视为无此项(None),不是 malformed——
声称 protected 时它仍会记 mismatch。
"""
if isinstance(result, LiveProtectionResult):
status, raw = result.status, result.raw
elif isinstance(result, str):
status, raw = result, None
elif result is None:
status, raw = None, None
elif isinstance(result, dict):
raw = result.get("raw")
if isinstance(result.get("status"), str):
status = result["status"]
elif isinstance(result.get("branch_protection"), str):
status = result["branch_protection"]
else:
status = None
else:
return None, None, True
if status is not None and not isinstance(status, str):
return None, None, True
return status, raw, False
def _is_live_reader(reader):
"""是否协议 reader:同时有 get_checks 与
get_branch_protection 两个方法(鸭子类型,不强制
isinstance,因为假 reader 未必显式继承协议)。"""
return callable(getattr(reader, "get_checks", None)) \
and callable(getattr(reader, "get_branch_protection", None))
def verify_live_evidence(claimed, reader=None, repo=None, ref=None,
branch=None):
"""实时证据比对(v7 机制第 6 项,fail-closed)。
claimed:调用方声称的门禁证据 dict(与硬门禁同款
字段:ci_status、ci_checks[{name, status}]、
required_checks、branch_protection)。
reader:LiveEvidenceReader 协议实例,或旧式 callable
(repo, ref) -> {"ci_checks": {...}, "branch_protection":
str}(与 verify_gate_evidence 同款;兼容旧假 reader)。
repo/ref:求证目标;branch:分支保护的求证分支,
未给时用 ref 作分支名(调用方须保证 ref 在此语义
下就是分支;SHA 求证时请显式传 branch,绝不由本
函数猜分支与 SHA 的对应关系)。
返回 {"checked", "verified", "mismatches", "reason",
"live"}:live 为归一化后的实时数据(含原始证据
raw)或 None(未核对时)。verified 为 False 的情形:
无 reader、无 repo/ref、reader 抛异常、reader 返回
恶化形状、实时数据与声称不一致——一律不是“通过”。
比对规则与 verify_gate_evidence 共用 _compare,口
径一致;本函数不发网络请求、不读 token。
"""
if reader is None:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "no_trusted_reader", "live": None}
if not repo or not ref:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "no_verification_target", "live": None}
if _is_live_reader(reader):
try:
checks_result = reader.get_checks(repo, ref)
except Exception as exc:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: %s" % exc,
"live": None}
normalized = _normalize_checks(checks_result)
if normalized is None:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: malformed checks",
"live": None}
checks, checks_raw = normalized
try:
protection_result = reader.get_branch_protection(
repo, branch if branch else ref)
except Exception as exc:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: %s" % exc,
"live": None}
protection, protection_raw, malformed = \
_normalize_protection(protection_result)
if malformed:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: malformed protection",
"live": None}
live = {"ci_checks": checks,
"branch_protection": protection,
"checks_raw": checks_raw,
"protection_raw": protection_raw}
elif callable(reader):
try:
live_raw = reader(repo, ref)
except Exception as exc:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: %s" % exc,
"live": None}
if not isinstance(live_raw, dict):
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: non-dict result",
"live": None}
live = live_raw
else:
return {"checked": False, "verified": False,
"mismatches": [],
"reason": "reader_failed: unsupported reader type",
"live": None}
mismatches = _compare_claimed_vs_live(claimed, live)
return {"checked": True, "verified": not mismatches,
"mismatches": mismatches, "reason": None,
"live": live}
def live_gate_decision(claimed, reader=None, repo=None, ref=None,
branch=None):
"""端点层 live 门禁判定(v7 机制第 6 项,fail-closed)。
verified=False 一律阻断,返回 allowed=False、
http_status=409、status="blocked"、blocked=True:
code 为 live_evidence_mismatch(已核对但不一致)
或 live_evidence_unverified(未核对:无 reader、
无目标、reader 失败)。verified=True 时返回
allowed=True、http_status=200、status="allowed"、
code="live_evidence_verified"。返回里附完整 live
比对结果(live 键),供端点写进 409 响应与审计。
注意:本函数是判定缝,不是端点接线——HTTP 端点
须在有生产 reader 可注入时才调用它;无 reader 时
调用它会阻断(这是 fail-closed 的正确语义),所
以在生产 reader 就绪前端点不得接线(见模块头残
留)。本函数本身是纯函数,不写审计、不改状态。
"""
result = verify_live_evidence(claimed, reader, repo, ref,
branch=branch)
if result["verified"]:
return {"allowed": True, "blocked": False,
"status": "allowed", "http_status": 200,
"code": "live_evidence_verified",
"live": result}
code = "live_evidence_mismatch" if result["checked"] \
else "live_evidence_unverified"
return {"allowed": False, "blocked": True,
"status": "blocked", "http_status": 409,
"code": code, "live": result}
+125
View File
@@ -0,0 +1,125 @@
"""门禁层:调用方只传事实,本层只出决定。
收敛对象是主线上已经存在的硬门禁,不是新发明的门禁
种类。主线(origin/main)上不存在 merge_gate 的
record/enforce 模式、分支保护门禁与全自动门禁——那些只
在未合并的分支里;本轨不从分支搬功能、不新增门禁种类、
不把硬门禁降级为“只记录不阻断”(那会削弱安全语义,
不属于收敛)。本层收敛的是三类已有判定:
1. 目录门(resolve_inside):worktree 内相对路径的安全
解析。服务端 _safe_worktree_path、cloud/tree.py 的
resolve_tree_path、cloud/marker_tools.py 的
safe_worktree_path(worker 经 marker_tools 调用)此前
各写了一份 realpath + commonpath,仅在 commonpath
的 ValueError 处理上不一致(服务端不捕,另两处捕)。
现统一为本函数:绝对路径、.. 逃逸、符号链接逃逸、
跨盘符 ValueError 一律拒绝;空串返回原 worktree。
2. 控制门(decide_control):marker_tools.check_control_gate
的纯函数判定(cancel / retry 的目标状态与派生关系),
现为本层 canonical 实现,marker_tools 只保留同名薄壳
(调用点与测试 import 的名字不变)。
3. 合并状态门(merge_state_decision):服务端 _merge_task
的状态判定——只有 completed 可标记 merged,其它状态
一律拒绝(含已 merged 的重复调用)。权限判定
(approve_merge)仍在服务端,不进本层:本层只看事实
(任务状态),不查权限。
不可覆盖规则(冻结):
- 目录门的拒绝没有任何 confirm / override / bypass 参数。
本模块的函数签名里 deliberately 没有这类参数:显式
确认即使存在,也只能覆盖“保护类风险”,绝不能覆盖
目录违规——而主线上目录违规本来就没有确认通道,
任何调用方都不得以“用户已确认”为由绕过本层。
测试钉住:resolve_inside 与 decide_control 的签名中
不含 confirm / override / bypass 字样。
- 合并状态门同样没有覆盖参数:非 completed 任务不存在
“确认后合并”的路径,重复 merged 调用仍是拒绝。
返回形态沿用 codebase 的元组风格:(allowed, reason),
reason 为空表示放行;目录门另有 resolve_inside 直接
返回路径或 None(旧调用点的形态不变)。
"""
import os
TERMINAL_STATUSES = ("completed", "failed", "interrupted", "merged",
"merge-failed")
RETRYABLE_STATUSES = ("failed", "interrupted", "merge-failed")
def resolve_inside(base, rel):
"""worktree 内相对路径的安全解析:越界回 None。
空串返回 base 本身(旧三处行为一致);绝对路径、
.. 逃逸、符号链接逃逸、commonpath 跨盘符 ValueError
一律 None。目录内指回内部的符号链接放行到真实位置
(调用方仍只拿到 worktree 内的内容)。
"""
if not rel:
return base
if os.path.isabs(rel):
return None
real_base = os.path.realpath(base)
target = os.path.realpath(os.path.join(real_base, rel))
try:
if os.path.commonpath([real_base, target]) != real_base:
return None
except ValueError:
return None
return target
def is_goal_task(task):
"""goal 任务判定:verify_command 非空或 goal_mode 为真。
与 worker run_one 的 goal 分支同款;goal 字段本身是
goal_mode=1 时 prompt 的副本,不单独当判定条件。
"""
if not task:
return False
return bool(task.get("verify_command") or task.get("goal_mode"))
def decide_control(target, sender, relation, kind):
"""task-cancel / task-retry 的纯函数门禁,返回 (ok, reason)。
kind 只能是 "cancel" 或 "retry",其它值一律拒绝。
权限层不在这里:worker 侧另查 create_task /
approve_merge;本函数只管目标状态与关系,单测不需
AuthStore。goal 任务不可取消或重试(goal 循环自有轮次)。
"""
if kind not in ("cancel", "retry"):
return False, "未知的控制操作"
if target is None:
return False, "目标任务不存在"
if (target.get("project_id") is not None
and sender.get("project_id") is not None
and target.get("project_id") != sender.get("project_id")):
return False, "跨项目任务不可操作"
if target.get("id") == sender.get("id"):
return False, "不能操作自己"
if is_goal_task(target):
return False, "goal 任务不可取消或重试(goal 循环自有轮次)"
status = target.get("status")
if kind == "cancel":
if status in TERMINAL_STATUSES:
return False, "目标已终态(%s),不可取消" % status
else:
if status not in RETRYABLE_STATUSES:
return False, ("仅 failed / interrupted / merge-failed"
" 可重试,当前状态为 %s" % status)
if relation not in ("descendant", "same_creator"):
return False, "目标不在发送方派生链上且创建人不同"
return True, ""
def merge_state_decision(status):
"""合并标记的状态判定:仅 completed 放行,返回 (ok, reason)。
含已 merged 的重复调用在内,其它状态一律拒绝;文案与
服务端旧 409 错误一致。权限判定不在本层。
"""
if status == "completed":
return True, ""
return False, "only completed tasks can be marked merged"
+1 -2
View File
@@ -30,6 +30,7 @@ env secret,App webhook 用库里 secret,验签时两个都试(过渡
注入 fake http 客户端做单测。
"""
import base64 as _base64
import hashlib
import json
import secrets
@@ -607,8 +608,6 @@ def set_fix_review_task_id(conn, repo, pr_number, comment_id, task_id):
#("BEGIN RSA PRIVATE KEY"),openssl 3 默认生成的是 PKCS#8
#("BEGIN PRIVATE KEY"),两者都支持。
import base64 as _base64
_SHA256_DIGESTINFO_PREFIX = bytes.fromhex(
"3031300d060960864801650304020105000420")
+195
View File
@@ -0,0 +1,195 @@
"""运行中改 goal 的纯函数与信号文件格式。
信号文件格式(与 host.py 内联读侧同款契约,改一处必须同步
另一处,跨模块测试钉住它):
<state_dir>/goal-updates/<safe_host_name>.json
{"version": int, "goal": str, "task_id": str | None}
其中 safe_host_name 与 host.py 的 _interrupt_request_path 同款:
只留 [A-Za-z0-9_.-],其余字符换成 "-"。
本模块不 import cloud.server / cloud.worker,只用标准库,
host.py 不直接 import 它(host.py 是可独立部署的脚本,只用
标准库),host.py 里有一份同格式的内联读侧。
语义:
- version 单调递增,服务端每次成功改 goal 就 +1;循环只应用
version > 已应用版本的信号,过期版本直接丢弃。
- 信号文件只由 worker 侧写(镜像线程每 30 秒把库里更新的
goal_version 写成信号;断点续跑认领时钩子先写一次),
host 循环在轮次边界读、应用后由检查点落盘 + 清理。
- 信号本身不含签名:它只在 host-state 目录(worker 与 host
循环共享的同一目录)里流转,调用方是受信的 worker 进程;
不要把信号文件路径暴露成公开接口。
"""
import json
import os
import re
import uuid
GOAL_UPDATE_DIRNAME = "goal-updates"
def safe_host_name(host_name):
"""与 host.py _interrupt_request_path 同款的安全化。"""
return re.sub(r"[^A-Za-z0-9_.-]", "-", str(host_name))
def signal_path(state_path, host_name):
"""信号文件绝对路径:state 文件同目录下的 goal-updates 子目录。"""
return os.path.join(
os.path.dirname(os.path.abspath(state_path)),
GOAL_UPDATE_DIRNAME,
safe_host_name(host_name) + ".json",
)
def write_goal_update_signal(state_path, host_name, version, goal,
task_id=None):
"""原子写信号文件(写临时文件再 os.replace)。
version 必须为正整数,goal 必须为非空字符串,否则
ValueError;调用方(worker 镜像/认领钩子)负责吞异常,
信号写失败绝不能炸掉任务执行本身。
"""
if not isinstance(version, int) or isinstance(version, bool) \
or version < 1:
raise ValueError("goal update version must be a positive int")
if not isinstance(goal, str) or not goal.strip():
raise ValueError("goal update goal must be a non-empty string")
path = signal_path(state_path, host_name)
os.makedirs(os.path.dirname(path), exist_ok=True)
tmp = "%s.tmp.%s" % (path, os.getpid())
payload = {"version": version, "goal": goal, "task_id": task_id}
with open(tmp, "w", encoding="utf-8") as fh:
json.dump(payload, fh, ensure_ascii=False)
os.replace(tmp, path)
return path
def read_goal_update_signal(state_path, host_name, applied_version):
"""读信号,返回 (version, goal) 或 None。
以下情况一律返回 None,绝不抛:文件不存在、JSON 坏、
version 不是正整数、goal 不是非空字符串、version <=
已应用版本(过期信号)。调用方按 None 处理即可。
"""
try:
applied = int(applied_version or 0)
except (TypeError, ValueError):
applied = 0
try:
with open(signal_path(state_path, host_name),
encoding="utf-8") as fh:
data = json.load(fh)
except Exception:
return None
if not isinstance(data, dict):
return None
version = data.get("version")
goal = data.get("goal")
if not isinstance(version, int) or isinstance(version, bool) \
or version < 1:
return None
if not isinstance(goal, str) or not goal.strip():
return None
if version <= applied:
return None
return (version, goal)
def consume_goal_update_signal(state_path, host_name, applied_version):
"""已应用版本的信号文件尽力删掉(删失败静默)。
只删 version <= 已应用版本的信号;更新的信号(写于应用
之后、尚未应用)绝不能被误删。返回是否删掉了文件。
"""
try:
applied = int(applied_version or 0)
except (TypeError, ValueError):
applied = 0
path = signal_path(state_path, host_name)
try:
with open(path, encoding="utf-8") as fh:
data = json.load(fh)
except Exception:
return False
if not isinstance(data, dict):
return False
version = data.get("version")
if not isinstance(version, int) or isinstance(version, bool):
return False
if version > applied:
return False
try:
os.remove(path)
except Exception:
return False
return True
def build_goal_update_notice(old_goal, new_goal):
"""轮次边界应用新 goal 时,喂给 agent 的显式通知块。"""
return (
"目标已更新(运行中改 goal):\n"
"旧目标:%s\n"
"新目标:%s\n"
"请继续完成新目标,不要重复已完成的部分;完成与否只按"
"新目标判断;此前如已输出过“目标已达成”,该宣告对新"
"目标作废——新目标满足时请重新宣告,否则继续工作。"
% (old_goal, new_goal)
)
def ensure_goal_history_table(conn):
"""幂等建 task_goal_history 表(改 goal 的审计历史)。
id 为随机 hex;(task_id, version) 上的索引供按任务查
历史时用。调用方负责 commit。
"""
conn.execute(
"CREATE TABLE IF NOT EXISTS task_goal_history("
"id TEXT PRIMARY KEY, task_id TEXT NOT NULL, "
"old_goal TEXT, new_goal TEXT NOT NULL, "
"version INTEGER NOT NULL, created_by TEXT, "
"created_at REAL NOT NULL)"
)
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_task_goal_history_task "
"ON task_goal_history(task_id, version)"
)
def apply_goal_update(conn, task_id, old_goal, old_version, new_goal,
user_id, now):
"""在同一连接里做一次改 goal 写库(不 commit,调用方提交)。
CAS 在 version 上:只有 version 仍是 old_version(老行
NULL 按 0 计)时才更新,否则返回 None(调用方回 409 请
刷新后重试,绝不静默覆盖并发改动)。成功时同时写
goal_history 一行,返回新 version。
"""
old_v = int(old_version or 0)
if old_v == 0:
cur = conn.execute(
"UPDATE tasks SET goal = ?, goal_version = ?, "
"updated_at = ? WHERE id = ? AND "
"(goal_version IS NULL OR goal_version = 0)",
(new_goal, old_v + 1, now, task_id),
)
else:
cur = conn.execute(
"UPDATE tasks SET goal = ?, goal_version = ?, "
"updated_at = ? WHERE id = ? AND goal_version = ?",
(new_goal, old_v + 1, now, task_id, old_v),
)
if cur.rowcount != 1:
return None
conn.execute(
"INSERT INTO task_goal_history(id, task_id, old_goal, "
"new_goal, version, created_by, created_at) "
"VALUES(?, ?, ?, ?, ?, ?, ?)",
(uuid.uuid4().hex, task_id, old_goal, new_goal,
old_v + 1, user_id, now),
)
return old_v + 1
+537
View File
@@ -0,0 +1,537 @@
"""Handler mixin:AutomationMixin。
由 CloudControlPlaneHandler 拆出,方法体逐字搬移;
self.* 调用(含其它 mixin 的方法与留守的 _send_json / _db 等)
在混入后的同一类上解析,行为不变。本模块不 import cloud.server。
"""
from cloud.automations import cancel_automation as _auto_cancel
from cloud.automations import create_automation as _auto_create
from cloud.automations import ensure_automations_tables as _auto_ensure
from cloud.automations import get_automation as _auto_get
from cloud.automations import list_automations as _auto_list
from cloud.automations import list_automation_runs as _auto_runs
from cloud.automations import triage_inbox as _auto_triage
from cloud.automations import update_automation as _auto_update
from cloud.lifecycle import bind_automation as _life_bind
from cloud.lifecycle import get_binding as _life_get_binding
from cloud.lifecycle import unbind_automation as _life_unbind
from cloud.marker_tools import parse_first_run_at as _parse_first_run_at
from cloud.push import (
create_subscription as _push_create,
dispatch_event as _push_dispatch,
)
from cloud.todos import create_todo as _todo_create
from cloud.todos import delete_todo as _todo_delete
from cloud.todos import get_todo as _todo_get
from cloud.todos import list_todos as _todo_list
from cloud.todos import update_todo as _todo_update
from cloud.handler_support import (
_auth_store as _auth_store,
_ensure_followups_table as _ensure_followups_table,
_task_to_dict as _task_to_dict,
)
class AutomationMixin:
"""AutomationMixin 领域方法(见模块 docstring)。"""
# -- 推送订阅与分发 ---------------------------------------------------
def _push_create_endpoint(self, user_id, project_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
try:
sub = _push_create(
self.server.db_path, project_id, user_id,
body.get("channel"), body.get("target"),
body.get("preference") or "instant")
except ValueError as e:
self._send_json(400, {"error": str(e)})
return
# 审计只记订阅 id 与通道:target 全文不记(Slack webhook
# URL 本身含 secret 路径,记进审计等于把 secret 落盘)。
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "push_subscription_created",
"subscription_id=%s channel=%s"
% (sub["id"], sub["channel"]))
self._send_json(201, sub)
def _push_dispatch_endpoint(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
event_type = body.get("event_type")
project_id = body.get("project_id")
if not event_type or not project_id:
self._send_json(400, {"error": "event_type and project_id required"})
return
# P0 安全批后:/admin/push/dispatch 是管理端点,只认
# CODEX_GLOBAL_ADMINS——该口径已强于 P1 要求的 create_task,
# 不再降级。被拒与成功都记审计;目标 SSRF 校验在
# push.dispatch_event / 真实发送层做。
if not self._is_global_admin(user_id):
self._audit_admin(user_id, "admin_push_denied",
"dispatch %s %s" % (event_type, project_id))
self._send_json(403, {"error": "forbidden"})
return
# 分发:sender=None 时用 push 的内置真实发送端
# (webhook/slack 真 POST,webpush/apns/fcm 仍报缺凭证);
# 显式注入的 sender 优先于内置端(见 push.dispatch_event)。
result = _push_dispatch(
self.server.db_path, event_type, project_id,
body.get("payload") or {}, sender=None)
self._audit_admin(user_id, "admin_push_dispatch",
"dispatch %s %s" % (event_type, project_id))
self._send_json(200, result)
# -- webhooks(签名认证,不走 X-API-Key)---------------------------
# -- 自动化与分诊收件箱(P1 第 2 项)---------------------------
# 权限口径与其它项目级写操作同款:创建/取消用 create_task
# (不是 manage_project,因为自动化是项目成员的日常编排,
# 不是改项目归属/仓库这类 owner 专属操作;评审/审批等写
# 操作同样用 create_task);读用 read。状态变更记审计。
# 失败口径:校验失败 400;模块/数据库层意外失败时回 502
# 透传错误原文(截断 500 字),绝不回 201 假装建成功——
# 注意 _dispatch_safe 的兜底是 500,这里显式 502 优先于它。
def _automation_create(self, user_id, project_id):
"""创建自动化定义:只登记定义与到期状态,不跑模型、
不另起调度器(执行器后续走 claim 门,见 automations
模块文档)。body:name、prompt 必填;interval_seconds
可选(默认 86400,最小 60);repo_url 可选;
first_run_at 可选(epoch 秒或 ISO8601 字符串,与
marker 的 parse_first_run_at 同款解析,过去则建完
即到期);run_once 可选(true 时跑过一次后不再到期)。
P1 配额门:创建时先过与建任务同一道配额门(美元配额
与无价 token cap,超限 402),避免“定义免费建、到期
无限跑”;单自动化的 spend 上限即创建人配额本身——
每次到期产出的任务以 created_by=创建人建任务,走
POST /tasks 的配额门(触发路径若绕过服务端建任务,
则不在本门覆盖内,见残留)。本服务端没有自动化触发
端点,触发侧的门不在本次 server.py 改动内。"""
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
if self._reject_if_over_quota(user_id):
return
interval = body.get("interval_seconds", 86400)
if interval is None:
interval = 86400
first_run_at = body.get("first_run_at")
if first_run_at is not None:
first_run_at = _parse_first_run_at(first_run_at)
if first_run_at is None:
self._send_json(400, {"error": "invalid first_run_at"})
return
run_once = body.get("run_once", False)
if not isinstance(run_once, bool):
self._send_json(400, {"error": "run_once must be boolean"})
return
conn = self._db()
try:
try:
auto = _auto_create(
conn, project_id, body.get("name"),
body.get("prompt"), user_id, interval,
body.get("repo_url"),
first_run_at=first_run_at, run_once=run_once)
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
finally:
conn.close()
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "automation_created",
auto["id"])
self._send_json(201, auto)
def _automation_list(self, project_id):
"""列出某项目全部自动化定义(含已取消),附到期计数。"""
conn = self._db()
try:
try:
autos = _auto_list(conn, project_id)
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
conn.commit()
finally:
conn.close()
self._send_json(200, {
"project_id": project_id,
"automations": autos,
"due_count": sum(1 for a in autos if a["due"]),
})
# -- 项目待办(见 cloud/todos.py)-------------------------
def _todos_list(self, project_id, query):
"""列出某项目待办,附全项目各状态计数;query 支持
status 与 q(标题/描述子串)服务端过滤,计数与
total 不受过滤影响。"""
conn = self._db()
try:
try:
todos = _todo_list(
conn, project_id,
status=query.get("status", [None])[0] or None,
q=query.get("q", [None])[0] or None)
# counts / total 是全项目口径,不受过滤影响
#(与任务列表的 status_counts 同款语义)。
all_todos = _todo_list(conn, project_id)
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
conn.commit()
finally:
conn.close()
counts = {}
for t in all_todos:
counts[t["status"]] = counts.get(t["status"], 0) + 1
self._send_json(200, {
"project_id": project_id, "todos": todos,
"counts": counts, "total": len(all_todos)})
def _todos_create(self, user_id, project_id):
"""新建待办:body title 必填;description / status /
priority / task_id / due_at / sort_order 可选。"""
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
conn = self._db()
try:
try:
todo = _todo_create(
conn, project_id, user_id, body.get("title"),
description=body.get("description"),
status=body.get("status"),
priority=body.get("priority"),
task_id=body.get("task_id"),
due_at=body.get("due_at"),
sort_order=body.get("sort_order"))
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
finally:
conn.close()
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "todo_created", todo["id"])
self._send_json(201, todo)
def _todos_get(self, project_id, todo_id):
"""单条待办:行不存在或属于别的项目时一律 404。"""
conn = self._db()
try:
try:
todo = _todo_get(conn, todo_id)
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
conn.commit()
finally:
conn.close()
if todo is None or todo["project_id"] != project_id:
self._send_json(404, {"error": "todo not found"})
return
self._send_json(200, todo)
def _todos_update(self, user_id, project_id, todo_id):
"""PATCH 待办:只改 body 里给出的字段;行不存在或
属于别的项目时一律 404;校验失败 400。"""
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
conn = self._db()
try:
try:
todo = _todo_update(conn, todo_id, body or {})
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
finally:
conn.close()
if todo is None or todo["project_id"] != project_id:
self._send_json(404, {"error": "todo not found"})
return
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "todo_updated", todo_id)
self._send_json(200, todo)
def _todos_delete(self, user_id, project_id, todo_id):
"""DELETE 待办(硬删除):行不存在或属于别的项目
时一律 404;已取消是软关闭,不走这条。"""
conn = self._db()
try:
try:
todo = _todo_delete(conn, todo_id)
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
finally:
conn.close()
if todo is None or todo["project_id"] != project_id:
self._send_json(404, {"error": "todo not found"})
return
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "todo_deleted", todo_id)
self._send_json(200, todo)
def _automation_cancel(self, user_id, project_id, automation_id):
"""取消自动化(软删除):行不存在或属于别的项目时一
律 404(不泄露跨项目存在性);已取消时 409。"""
conn = self._db()
try:
try:
ok, auto = _auto_cancel(conn, automation_id)
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
finally:
conn.close()
if auto is None or auto["project_id"] != project_id:
self._send_json(404, {"error": "automation not found"})
return
if not ok:
self._send_json(409, {
"error": "automation already cancelled"})
return
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "automation_cancelled",
automation_id)
self._send_json(200, auto)
def _automation_get(self, user_id, project_id, automation_id):
"""GET 单条自动化(轮询用):定义 + 最近产出记录。
行不存在或属于别的项目时一律 404;产出记录读失败
时不让整条定义也 502(runs 回空列表 + note)。"""
conn = self._db()
try:
auto = _auto_get(conn, automation_id)
runs = []
runs_note = ""
if auto is not None:
try:
runs = _auto_runs(conn, automation_id)
except Exception:
runs_note = "runs unavailable"
finally:
conn.close()
if auto is None or auto["project_id"] != project_id:
self._send_json(404, {"error": "automation not found"})
return
conn = self._db()
try:
binding = _life_get_binding(conn, automation_id)
finally:
conn.close()
self._send_json(200, dict(auto, runs=runs,
runs_note=runs_note,
heartbeat_binding=binding))
def _automation_bind_get(self, user_id, project_id, automation_id):
conn = self._db()
try:
auto = _auto_get(conn, automation_id)
binding = _life_get_binding(conn, automation_id)
finally:
conn.close()
if auto is None or auto["project_id"] != project_id:
self._send_json(404, {"error": "automation not found"})
return
self._send_json(200, {"automation_id": automation_id,
"binding": binding})
def _automation_bind(self, user_id, project_id, automation_id):
"""POST .../automations/{aid}/bind {task_id, mode}:
把自动化绑到已有线程。mode=heartbeat 表示到期唤醒
该线程(走 thread/message 投递路);mode=cron 表示
到期独立跑新任务(绑定只作标记,不唤醒)。目标任务
必须在同一项目,否则 404(不泄露跨项目存在性)。
权限 create_task;记 automation_bound 审计。"""
body, jerr = self._read_json()
if jerr:
self._send_json(400, {"error": jerr})
return
conn = self._db()
try:
auto = _auto_get(conn, automation_id)
if auto is None or auto["project_id"] != project_id:
self._send_json(404, {"error": "automation not found"})
return
if auto["status"] != "active":
self._send_json(409, {
"error": "cancelled automation cannot be bound"})
return
target_id = body.get("task_id")
target = self._fetch_task(target_id) \
if isinstance(target_id, str) else None
if target is None:
self._send_json(404, {"error": "task not found"})
return
target = _task_to_dict(target)
if target["project_id"] != project_id:
self._send_json(404, {"error": "task not found"})
return
try:
binding = _life_bind(
conn, automation_id, project_id, target_id,
body.get("mode", "heartbeat"))
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
finally:
conn.close()
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "automation_bound",
"%s:%s" % (automation_id, target_id))
self._send_json(200, {"automation_id": automation_id,
"binding": binding})
def _automation_unbind(self, user_id, project_id, automation_id):
conn = self._db()
try:
auto = _auto_get(conn, automation_id)
if auto is None or auto["project_id"] != project_id:
self._send_json(404, {"error": "automation not found"})
return
ok = _life_unbind(conn, automation_id)
finally:
conn.close()
if not ok:
self._send_json(404, {"error": "no binding"})
return
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "automation_unbound",
automation_id)
self._send_json(200, {"automation_id": automation_id,
"deleted": True})
def _automation_heartbeat(self, user_id, project_id, automation_id):
"""POST .../automations/{aid}/heartbeat {message?}:
手动触发一次心跳唤醒——到期触发器(host run-due /
run_due_automation)调的就是这个口,服务端本身不
起定时器。无绑定 409;mode=cron 409(cron 不唤醒
线程);message 缺省用自动化 prompt。投递走
_deliver_to_thread,与 thread/message 同款复活路,
配额门同款。记 automation_heartbeat_sent 审计。"""
body, jerr = self._read_json()
if jerr:
self._send_json(400, {"error": jerr})
return
conn = self._db()
try:
auto = _auto_get(conn, automation_id)
binding = _life_get_binding(conn, automation_id)
finally:
conn.close()
if auto is None or auto["project_id"] != project_id:
self._send_json(404, {"error": "automation not found"})
return
if binding is None:
self._send_json(409, {
"error": "automation has no heartbeat binding"})
return
if binding["mode"] != "heartbeat":
self._send_json(409, {
"error": "cron binding does not wake a thread;"
" use the automation run path"})
return
target_row = self._fetch_task(binding["task_id"])
if target_row is None:
self._send_json(404, {"error": "bound task not found"})
return
target = _task_to_dict(target_row)
if target["project_id"] != project_id:
self._send_json(404, {"error": "bound task not found"})
return
if self._reject_if_over_quota_for_task(user_id, target):
return
message = body.get("message") or auto.get("prompt") or ""
if not isinstance(message, str) or not message.strip():
self._send_json(400, {"error": "message required"})
return
message = message.strip()
if len(message) > 20000:
self._send_json(400, {"error": "message too long"})
return
code, payload = self._deliver_to_thread(
user_id, target, message, "queued",
audit_action="automation_heartbeat_sent")
self._send_json(code, dict(payload,
automation_id=automation_id))
def _automation_update(self, user_id, project_id, automation_id):
"""PATCH 单条自动化(automation_update):改名、改
prompt、改间隔、改 repo_url,至少给一个字段。注意
语义边界:更新不重置 last_run_at(改间隔只影响下次
到期,不立即触发);已取消的定义 409(取消是终态,
改它必须重建);改名撞同项目 active 名时 400;本
端点不改“变化时通知”本身——到期产出的任务终态仍
走 worker 现有 push 派发(见 automations 模块文档)。
权限 create_task;记 automation_updated 审计。"""
body, jerr = self._read_json()
if jerr:
self._send_json(400, {"error": jerr})
return
if body.get("first_run_at") is not None and \
isinstance(body.get("first_run_at"), str):
parsed = _parse_first_run_at(body["first_run_at"])
if parsed is None:
self._send_json(400, {"error": "invalid first_run_at"})
return
body = dict(body, first_run_at=parsed)
conn = self._db()
try:
try:
ok, auto = _auto_update(conn, automation_id, body)
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
finally:
conn.close()
if auto is None or auto["project_id"] != project_id:
self._send_json(404, {"error": "automation not found"})
return
if not ok:
self._send_json(409, {
"error": "automation already cancelled"})
return
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "automation_updated",
automation_id)
self._send_json(200, auto)
def _triage_inbox(self, project_id):
"""分诊收件箱(只读):到期自动化、失败/中断任务、
待处理后续消息、最近自动化产出。见 automations 模块
triage_inbox 的字段说明。"""
conn = self._db()
try:
try:
_auto_ensure(conn)
_ensure_followups_table(conn)
inbox = _auto_triage(conn, project_id)
except Exception as exc:
self._send_json(502, {"error": str(exc)[:500]})
return
conn.commit()
finally:
conn.close()
self._send_json(200, inbox)
+420
View File
@@ -0,0 +1,420 @@
"""Handler mixin:FullAutoMixin。
由 CloudControlPlaneHandler 拆出,方法体逐字搬移;
self.* 调用(含其它 mixin 的方法与留守的 _send_json / _db 等)
在混入后的同一类上解析,行为不变。本模块不 import cloud.server。
"""
from cloud.full_auto import (
FullAutoDenied,
FullAutoError,
audit_record,
auto_config_plan,
branch_protection_payload,
ci_fix_decision,
is_full_auto_enabled,
merge_gate,
release_plan,
rollback_plan,
run_plan,
validate_action,
)
from cloud.github_app import resolve_app_token as _resolve_app_token
from cloud.github_app import (
find_installation_for_repo,
get_app_config,
list_installations,
)
import json
import os
import urllib.error
import urllib.request
class FullAutoMixin:
"""FullAutoMixin 领域方法(见模块 docstring)。"""
# -- 全自动执行器 HTTP 端点(cloud/full_auto.py 的接线层) ------
# 读类端点(status/config-plan)只查计划与状态、不执行、不
# 碰 token;写类执行(execute)只限全局管理员,且须同时
# 满足:FULL_AUTO_ENABLED 显式开启 + 该仓库 installation 已
# 验证 + 主分支保护经 GitHub API 实测达标,三者缺一即
# 403 拒绝并记审计,绝不静默放行。审计一律经
# full_auto.audit_record 脱敏后落 AuthStore audit_log(action
# 前缀 full_auto_),响应与审计里绝不出现 token 明文、
# Bearer 明文或调用方塞进 detail 的敏感键。
def _full_auto_status(self, user_id, query):
repo = (query.get("repo") or [None])[0]
conn = self._db()
try:
configured = get_app_config(conn) is not None
verified = _full_auto_installation_verified(conn, repo)
installations = list_installations(conn)
finally:
conn.close()
flag = str(os.environ.get("FULL_AUTO_ENABLED", "")
).strip().lower() in ("1", "true", "yes")
self._send_json(200, {
"enabled": is_full_auto_enabled(os.environ, verified),
"flag_enabled": flag,
"installation_verified": verified,
"app_configured": configured,
"repo": repo,
"installations": installations,
})
def _full_auto_config_plan(self, user_id, query):
params, err = _full_auto_parse_plan_params({}, query)
if err:
self._send_json(400, {"error": err})
return
try:
steps = auto_config_plan(**params)
except FullAutoError as exc:
self._send_json(400, {"error": str(exc),
"reason": exc.reason})
return
self._send_json(200, {"repo": params["repo_full_name"],
"steps": steps})
def _full_auto_validate_action(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
action = (body or {}).get("action")
if not isinstance(action, str) or not action:
self._send_json(400, {"error": "action required"})
return
try:
validate_action(action)
except FullAutoDenied as exc:
self._send_json(403, {"error": "action denied",
"reason": exc.reason,
"action": action})
return
self._send_json(200, {"action": action, "allowed": True})
def _full_auto_merge_gate(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
body = body or {}
passed, reasons = merge_gate(
body.get("checks") or {},
body.get("required_checks") or (),
bool(body.get("directory_check_ok")),
bool(body.get("contract_version_ok")),
bool(body.get("coordinator_approved")))
self._send_json(200, {"passed": passed, "reasons": reasons})
def _full_auto_ci_fix(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
body = body or {}
try:
attempt = int(body.get("attempt"))
except (TypeError, ValueError):
self._send_json(400, {"error": "attempt must be an integer"})
return
self._send_json(200, ci_fix_decision(
attempt, body.get("failures") or []))
def _full_auto_release_plan(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
body = body or {}
if not isinstance(body.get("tag"), str) or not body.get("tag"):
self._send_json(400, {"error": "tag required"})
return
self._send_json(200, release_plan(
body["tag"], body.get("merged_prs") or []))
def _full_auto_rollback_plan(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
body = body or {}
try:
plan = rollback_plan(body.get("previous_tag"))
except FullAutoError as exc:
self._send_json(400, {"error": str(exc),
"reason": exc.reason})
return
self._send_json(200, plan)
def _full_auto_audit_list(self, user_id, query):
"""审计查询:只限全局管理员,只返回 full_auto_ 前缀行,
且只返回已脱敏落盘的 detail(写时已脱敏,读时不再
含 token——调用方若直接写库绕过本层,不在本端点
的保证内,写路径见 _full_auto_execute)。"""
if not self._is_global_admin(user_id):
self._audit_admin(user_id, "full_auto_audit_denied",
"audit list")
self._send_json(403, {"error": "forbidden"})
return
try:
limit = int((query.get("limit") or ["100"])[0])
except ValueError:
limit = 100
limit = max(1, min(limit, 100))
conn = self._db()
try:
rows = conn.execute(
"SELECT user_id, action, detail, created_at FROM audit_log"
" WHERE action LIKE 'full_auto_%'"
" ORDER BY created_at DESC, id DESC LIMIT ?",
(limit,)).fetchall()
except Exception:
rows = []
finally:
conn.close()
self._send_json(200, {"audit": [
{"user_id": r[0], "action": r[1], "detail": r[2],
"created_at": r[3]} for r in rows]})
def _full_auto_execute(self, user_id):
"""执行自动配置计划:按 full_auto.auto_config_plan 生成
顺序计划,按 full_auto.run_plan 顺序执行,不跳步。
门禁顺序(任一步拒绝即停、记审计、不执行后续):
1. 全局管理员;2. body.action 若给出须过 validate_action;
3. FULL_AUTO_ENABLED 显式开 + 该仓库 installation 已验证;
4. installation token 现解析现用(测试注入
server.full_auto_token_resolver 仅供单测绕过 JWT 签发,
生产为 None 时走 github_app.resolve_app_token);
5. 主分支保护经 GitHub API 实测达标(带 token 查,
私有仓无 token 查保护必 404,故 token 先于保护核查)。
token 只进 Authorization 请求头;request_fn 注入点
server.full_auto_http_client 仅供单测,生产走 urllib。"""
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
body = body or {}
params, perr = _full_auto_parse_plan_params(body, {})
if perr:
self._send_json(400, {"error": perr})
return
repo = params["repo_full_name"]
branch = params["default_branch"]
checks = params["required_checks"]
def _deny(reason, detail):
record = audit_record("execute_denied", repo,
"full-auto-executor", "denied",
{"reason": reason, "detail": detail})
self._audit_admin(user_id, "full_auto_execute_denied",
json.dumps(record, ensure_ascii=False))
self._send_json(403, {"error": "full-auto denied",
"reason": reason})
if not self._is_global_admin(user_id):
_deny("not_global_admin", "execute requires global admin")
return
action = body.get("action")
if action is not None:
if not isinstance(action, str) or not action:
self._send_json(400, {"error": "action must be a string"})
return
try:
validate_action(action)
except FullAutoDenied as exc:
_deny(exc.reason, "action=%s" % action)
return
conn = self._db()
try:
verified = _full_auto_installation_verified(conn, repo)
finally:
conn.close()
if not is_full_auto_enabled(os.environ, verified):
_deny("full_auto_disabled",
"FULL_AUTO_ENABLED not set or installation unverified")
return
http_client = getattr(self.server, "full_auto_http_client", None)
request_fn = lambda m, u, h, b: _full_auto_github_request( # noqa: E731
http_client, m, u, h, b)
resolver = getattr(self.server, "full_auto_token_resolver", None)
if resolver is not None:
token = resolver(self.server.db_path, repo)
else:
conn = self._db()
try:
token = _resolve_app_token(conn, repo)
finally:
conn.close()
if not token:
_deny("token_missing",
"installation token unavailable for repo")
return
protection_url = ("https://api.github.com/repos/%s/branches/%s"
"/protection" % (repo, branch))
try:
pstatus, protection = request_fn(
"GET", protection_url,
{"Accept": "application/vnd.github+json",
"User-Agent": "codex-app-server",
"Authorization": "Bearer " + token}, None)
except FullAutoError:
_deny("branch_protection_unverified",
"protection check request failed")
return
if pstatus != 200 or not _full_auto_branch_protection_ok(
protection, checks):
_deny("branch_protection_unverified",
"branch protection incomplete or unverified")
return
try:
steps = auto_config_plan(**params)
except FullAutoError as exc:
self._send_json(400, {"error": str(exc),
"reason": exc.reason})
return
results = []
try:
results = run_plan(request_fn, token, steps)
except FullAutoDenied as exc:
_deny(exc.reason, "plan step denied")
return
except FullAutoError as exc:
record = audit_record("auto_config", repo,
"full-auto-executor", "failed",
{"reason": exc.reason,
"detail": body.get("detail")})
self._audit_admin(user_id, "full_auto_auto_config",
json.dumps(record, ensure_ascii=False))
self._send_json(502, {"error": str(exc),
"reason": exc.reason,
"steps": results})
return
record = audit_record("auto_config", repo,
"full-auto-executor", "ok",
{"detail": body.get("detail"),
"steps": [r["step"] for r in results]})
self._audit_admin(user_id, "full_auto_auto_config",
json.dumps(record, ensure_ascii=False))
# 响应只含步骤名/状态码/ok,绝不回显请求体 detail、
# headers 或 token——results 本身就是 run_plan 的
# (step/status/ok) 三元组,无凭据字段。
self._send_json(200, {"ok": True, "repo": repo, "steps": results})
def _full_auto_parse_plan_params(body, query):
"""从 POST body 或 GET query 里解析 auto_config_plan 参数。
repo 必填且须含 /;required_checks 可为列表或逗号字符串;
布尔字段接受 true/false/1/0 字符串与布尔值。非法返回
(None, error_string)。"""
def _get(name):
if isinstance(body, dict) and name in body:
return body[name]
values = (query or {}).get(name)
return values[0] if values else None
def _as_bool(value, default):
if value is None:
return default
if isinstance(value, bool):
return value
return str(value).strip().lower() in ("1", "true", "yes")
repo = _get("repo") or _get("repo_full_name")
if not isinstance(repo, str) or "/" not in repo:
return None, "repo must be owner/repo"
checks = _get("required_checks")
if checks is None:
checks = ("verify",)
elif isinstance(checks, str):
checks = tuple(c.strip() for c in checks.split(",") if c.strip())
elif isinstance(checks, (list, tuple)):
checks = tuple(str(c) for c in checks if str(c))
else:
return None, "required_checks must be a list or string"
params = {
"repo_full_name": repo,
"default_branch": _get("default_branch") or "main",
"org": _get("org"),
"repo_exists": _as_bool(_get("repo_exists"), True),
"ci_workflow_present": _as_bool(_get("ci_workflow_present"), False),
"required_checks": checks,
}
if params["org"] is not None and not isinstance(params["org"], str):
return None, "org must be a string"
return params, None
def _full_auto_installation_verified(conn, repo_full_name=None):
"""installation 已验证 = App 配置存在 + 安装记录存在。
repo 给出时查该仓库的安装记录(用户在 GitHub 侧安装 App
后 webhook 落库的记录);repo 未给出时任一安装记录即算
已验证(仅供 /full-auto/status 总览用,执行仍须按仓库查)。
真实安装/重新接受只能由用户在 GitHub 侧完成,本层绝不
伪造安装记录、绝不把“无记录”说成已验证。"""
try:
if get_app_config(conn) is None:
return False
if repo_full_name:
return find_installation_for_repo(conn, repo_full_name) is not None
rows = list_installations(conn)
return bool(rows)
except Exception:
return False
def _full_auto_branch_protection_ok(protection, required_checks):
"""分支保护是否达到 full_auto 的强制标准:与
branch_protection_payload 的要求逐项核对,缺一项即不算
配好;required_checks 须全部在 contexts 里(可多不可少)。
protection 为非 dict 或 GET 失败时调用方已先行拒绝。"""
if not isinstance(protection, dict):
return False
try:
expected = branch_protection_payload(required_checks)
except FullAutoError:
return False
checks = protection.get("required_status_checks") or {}
if not isinstance(checks, dict) or not checks.get("strict"):
return False
contexts = checks.get("contexts") or []
if not set(expected["required_status_checks"]["contexts"]) <= set(contexts):
return False
if not protection.get("enforce_admins"):
return False
if not isinstance(protection.get("required_pull_request_reviews"), dict):
return False
if protection.get("allow_force_pushes") is not False:
return False
if protection.get("allow_deletions") is not False:
return False
return True
def _full_auto_github_request(http_client, method, url, headers, body):
"""full_auto.run_plan 用的 request_fn:注入的 fake 客户端
优先(单测);生产走 urllib,token 只在 headers 里,响应
只返回 (status, dict),绝不把 headers 写进返回值。"""
if http_client is not None:
result = http_client(method, url, headers, body)
if not (isinstance(result, tuple) and len(result) == 2):
raise FullAutoError("http_client must return (status, dict)",
reason="auto_config_step_failed")
return result
request = urllib.request.Request(url, data=body, method=method,
headers=headers)
try:
with urllib.request.urlopen(request, timeout=30) as response:
raw = response.read().decode("utf-8", "replace")
status = response.status
except urllib.error.HTTPError as exc:
raw = exc.read().decode("utf-8", "replace")
status = exc.code
try:
parsed = json.loads(raw)
except ValueError:
parsed = {}
if not isinstance(parsed, dict):
parsed = {}
return status, parsed
+708
View File
@@ -0,0 +1,708 @@
"""Handler mixin:IdentityMemoryMixin。
由 CloudControlPlaneHandler 拆出,方法体逐字搬移;
self.* 调用(含其它 mixin 的方法与留守的 _send_json / _db 等)
在混入后的同一类上解析,行为不变。本模块不 import cloud.server。
"""
from cloud.auto_memory import AutoMemoryStore
from cloud.cost import CostStore
from cloud.identity import IdentityStore
from cloud.kb import KBStore
from cloud.memory import MemoryStore
from cloud.relay import RelayError as _RelayError
from cloud.sub2api import Sub2apiError as _Sub2apiError
from cloud.auto_memory import process_due_tasks as _auto_process_due
from cloud.sub2api import gateway_name as _gateway_name
from cloud.sub2api import gateway_origin as _gateway_origin
from cloud.relay import get_relay_models as _get_relay_models
from cloud.sub2api import list_gateway_models as _list_sub2api_models
from cloud.sub2api import validate_or_rotate_gateway_key as _validate_sub2api_key
import os
import sqlite3
from cloud.handler_support import (
_auth_store as _auth_store,
_task_to_dict as _task_to_dict,
)
class IdentityMemoryMixin:
"""IdentityMemoryMixin 领域方法(见模块 docstring)。"""
def _memory_create(self, user_id, project_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
goal = body.get("goal")
conclusion = body.get("conclusion")
if not goal or not isinstance(goal, str):
self._send_json(400, {"error": "goal required"})
return
if not conclusion or not isinstance(conclusion, str):
self._send_json(400, {"error": "conclusion required"})
return
store = MemoryStore(self.server.db_path)
try:
mid = store.add_summary(
project_id,
body.get("thread_id") or "",
goal,
conclusion,
files_changed=body.get("files_changed") or [],
open_issues=body.get("open_issues") or [],
)
finally:
try:
store.conn.close()
except Exception:
pass
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, project_id, "create_memory", str(mid))
self._send_json(201, {"id": mid, "project_id": project_id})
def _memory_list(self, project_id, query):
try:
top_k = int((query.get("top_k") or ["5"])[0])
except (TypeError, ValueError):
self._send_json(400, {"error": "top_k must be an integer"})
return
store = MemoryStore(self.server.db_path)
try:
summaries = store.list_summaries(project_id)
context = store.build_prompt_context(project_id, top_k=top_k)
finally:
try:
store.conn.close()
except Exception:
pass
self._send_json(200, {
"project_id": project_id,
"summaries": summaries[:top_k],
"prompt_context": context,
})
def _memory_supersede(self, user_id, project_id, mid_raw):
try:
mid = int(mid_raw)
except (TypeError, ValueError):
self._send_json(400, {"error": "id must be an integer"})
return
conn = self._db()
try:
row = conn.execute(
"SELECT project_id FROM summaries WHERE id = ?", (mid,)
).fetchone()
finally:
conn.close()
if row is None or row["project_id"] != project_id:
self._send_json(404, {"error": "summary not found"})
return
store = MemoryStore(self.server.db_path)
try:
store.deprecate(mid)
finally:
try:
store.conn.close()
except Exception:
pass
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, project_id, "supersede_memory", str(mid))
self._send_json(200, {"id": mid, "status": "deprecated"})
# -- System-level identity / memory (cloud only) ------------------
# These handlers never touch host.py, CODEX_HOME or any thread:
# they only read/write the control-plane database. The context
# endpoint returns text for an explicit caller; it does not
# inject anything into a task or a thread on its own.
def _identity_store(self):
return IdentityStore(self.server.db_path)
def _identity_close(self, store):
try:
store.conn.close()
except Exception:
pass
def _identity_profile_get(self, user_id):
store = self._identity_store()
try:
self._send_json(200, store.get_profile(user_id))
finally:
self._identity_close(store)
def _identity_profile_upsert(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
store = self._identity_store()
try:
try:
profile = store.upsert_profile(
user_id,
display_name=body.get("display_name"),
timezone=body.get("timezone"),
preferences=body.get("preferences"))
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
finally:
self._identity_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, "", "profile_updated", user_id)
self._send_json(200, profile)
def _identity_user_memories_list(self, user_id):
store = self._identity_store()
try:
rows = store.list_user_memories(user_id)
finally:
self._identity_close(store)
self._send_json(200, {"user_id": user_id, "memories": rows})
def _identity_user_memory_create(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
store = self._identity_store()
try:
try:
mem_id = store.add_memory(
user_id, "user", body.get("kind"),
body.get("content"),
source_thread_id=body.get("source_thread_id"))
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
finally:
self._identity_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, "", "identity_memory_created", mem_id)
self._send_json(201, {"id": mem_id, "scope": "user"})
def _identity_user_memory_supersede(self, user_id, mem_id):
body, err = self._read_json()
if err:
body = {}
store = self._identity_store()
try:
row = store.get_memory(mem_id)
if row is None or row["scope"] != "user" \
or row["owner_user_id"] != user_id:
self._send_json(404, {"error": "memory not found"})
return
try:
_old, new_id = store.supersede_memory(
mem_id, user_id,
replacement_content=body.get("replacement_content"),
replacement_kind=body.get("replacement_kind"))
except (KeyError, PermissionError):
self._send_json(404, {"error": "memory not found"})
return
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
finally:
self._identity_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, "", "identity_memory_superseded", mem_id)
self._send_json(200, {"id": mem_id, "status": "superseded",
"replacement_id": new_id})
def _identity_user_memory_delete(self, user_id, mem_id):
store = self._identity_store()
try:
row = store.get_memory(mem_id)
if row is None or row["scope"] != "user" \
or row["owner_user_id"] != user_id:
self._send_json(404, {"error": "memory not found"})
return
try:
store.delete_memory(mem_id, user_id)
except (KeyError, PermissionError):
self._send_json(404, {"error": "memory not found"})
return
finally:
self._identity_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, "", "identity_memory_deleted", mem_id)
self._send_json(200, {"deleted": True, "id": mem_id})
def _identity_context(self, user_id, query):
project_id = None
if query:
vals = query.get("project_id")
if vals:
project_id = vals[0]
if project_id:
err = self._check_project(user_id, project_id, "read")
if err:
self._send_json(err[0], {"error": err[1]})
return
store = self._identity_store()
try:
profile = store.get_profile(user_id)
user_rows = store.list_user_memories(user_id)
proj_rows = store.list_project_memories(project_id) \
if project_id else []
context = store.build_identity_context(user_id, project_id)
finally:
self._identity_close(store)
summaries_context = ""
if project_id:
mem_store = MemoryStore(self.server.db_path)
try:
summaries_context = mem_store.build_prompt_context(
project_id)
finally:
try:
mem_store.conn.close()
except Exception:
pass
self._send_json(200, {
"user_id": user_id,
"project_id": project_id,
"profile": profile,
"user_memories": user_rows,
"project_memories": proj_rows,
"identity_context": context,
# Kept as a separate field on purpose: the existing
# project summaries are NOT merged into identity_context
# and neither field is injected into any task/thread here.
"project_summaries_context": summaries_context,
})
# -- 自动记忆(官方 memories 对齐:设置 + 提取触发) ------------
# 设置与提取日志只归属本人;任务级开关走项目 create_task 门。
def _auto_memory_store(self):
return AutoMemoryStore(self.server.db_path)
def _auto_memory_close(self, store):
try:
store.close()
except Exception:
pass
def _auto_memory_settings_get(self, user_id):
store = self._auto_memory_store()
try:
self._send_json(200, store.get_settings(user_id))
finally:
self._auto_memory_close(store)
def _auto_memory_settings_upsert(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
if not isinstance(body, dict):
self._send_json(400, {"error": "body must be an object"})
return
store = self._auto_memory_store()
try:
try:
settings = store.upsert_settings(user_id, **body)
except (ValueError, TypeError) as exc:
self._send_json(400, {"error": str(exc)})
return
finally:
self._auto_memory_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, "", "auto_memory_settings_updated", "")
self._send_json(200, settings)
def _auto_memory_log(self, user_id, query):
try:
limit = int((query.get("limit") or ["50"])[0])
except (TypeError, ValueError):
self._send_json(400, {"error": "limit must be an integer"})
return
store = self._auto_memory_store()
try:
rows = store.list_log(user_id, limit=limit)
finally:
self._auto_memory_close(store)
self._send_json(200, {"user_id": user_id, "log": rows})
def _auto_memory_run(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
force = False
if isinstance(body, dict) and body.get("force") is not None:
if body.get("force") not in (True, False, 0, 1):
self._send_json(400, {"error": "force must be a boolean"})
return
force = bool(body.get("force"))
results = _auto_process_due(
self.server.db_path, force=force, owner_id=user_id)
for item in results:
if item.get("status") == "extracted":
# 提取日志已在 process_due_tasks 里落库,这里只补
# 一条审计(不查任务归属:run 只扫本人的任务)。
task_id = item.get("task_id") or ""
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, "", "auto_memory_extracted",
task_id)
self._send_json(200, {"user_id": user_id, "results": results})
def _task_auto_memory_flags(self, user_id, task_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
if not isinstance(body, dict):
self._send_json(400, {"error": "body must be an object"})
return
task = self._fetch_task(task_id)
if task is None:
self._send_json(404, {"error": "task not found"})
return
err = self._check_project(
user_id, task["project_id"], "create_task")
if err:
self._send_json(err[0], {"error": err[1]})
return
store = self._auto_memory_store()
try:
try:
flags = store.set_task_flags(
task_id,
use_memories=body.get("use_memories"),
generate_memories=body.get("generate_memories"))
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
finally:
self._auto_memory_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, task["project_id"],
"task_auto_memory_flags", task_id)
self._send_json(200, {"task_id": task_id, "flags": flags})
def _project_identity_memory_create(self, user_id, project_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
store = self._identity_store()
try:
try:
mem_id = store.add_memory(
user_id, "project", body.get("kind"),
body.get("content"), project_id=project_id,
source_thread_id=body.get("source_thread_id"))
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
finally:
self._identity_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, project_id,
"identity_memory_created", mem_id)
self._send_json(201, {"id": mem_id, "scope": "project",
"project_id": project_id})
def _project_identity_memories_list(self, project_id, query):
try:
top_k = int((query.get("top_k") or ["20"])[0])
except (TypeError, ValueError):
self._send_json(400, {"error": "top_k must be an integer"})
return
store = self._identity_store()
try:
rows = store.list_project_memories(project_id)
finally:
self._identity_close(store)
self._send_json(200, {"project_id": project_id,
"memories": rows[:top_k]})
def _project_identity_memory_supersede(self, user_id, project_id,
mem_id):
body, err = self._read_json()
if err:
body = {}
store = self._identity_store()
try:
row = store.get_memory(mem_id)
if row is None or row["scope"] != "project" \
or row["project_id"] != project_id:
self._send_json(404, {"error": "memory not found"})
return
try:
_old, new_id = store.supersede_memory(
mem_id, user_id,
replacement_content=body.get("replacement_content"),
replacement_kind=body.get("replacement_kind"))
except (KeyError, PermissionError, ValueError) as exc:
if isinstance(exc, ValueError):
self._send_json(400, {"error": str(exc)})
else:
self._send_json(404, {"error": "memory not found"})
return
finally:
self._identity_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, project_id,
"identity_memory_superseded", mem_id)
self._send_json(200, {"id": mem_id, "status": "superseded",
"replacement_id": new_id})
def _project_identity_memory_delete(self, user_id, project_id, mem_id):
store = self._identity_store()
try:
row = store.get_memory(mem_id)
if row is None or row["scope"] != "project" \
or row["project_id"] != project_id:
self._send_json(404, {"error": "memory not found"})
return
store.delete_memory(mem_id, user_id)
finally:
self._identity_close(store)
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, project_id,
"identity_memory_deleted", mem_id)
self._send_json(200, {"deleted": True, "id": mem_id})
def _kb_index(self, user_id, project_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
path = body.get("path")
content = body.get("content")
if not path or not isinstance(path, str):
self._send_json(400, {"error": "path required"})
return
if content is None or not isinstance(content, str):
self._send_json(400, {"error": "content required"})
return
store = KBStore(self.server.db_path)
try:
doc_id = store.index_file(
project_id, path, content,
kind=body.get("kind") or "file")
except ValueError as e:
self._send_json(400, {"error": str(e)})
return
finally:
try:
store.conn.close()
except Exception:
pass
with _auth_store(self.server.db_path) as auth:
auth.audit(user_id, project_id, "kb_index", path)
self._send_json(201, {"id": doc_id, "project_id": project_id,
"path": path})
def _kb_search(self, project_id, query):
q = (query.get("q") or [""])[0]
if not q:
self._send_json(400, {"error": "q required"})
return
try:
top_k = int((query.get("top_k") or ["5"])[0])
except (TypeError, ValueError):
self._send_json(400, {"error": "top_k must be an integer"})
return
store = KBStore(self.server.db_path)
try:
results = store.search(project_id, q, top_k=top_k)
finally:
try:
store.conn.close()
except Exception:
pass
self._send_json(200, {"project_id": project_id, "query": q,
"results": results})
def _project_cost(self, project_id, query):
store = CostStore(self.server.db_path)
try:
# 双列:美元只计有价行,未定价模型单独列在
# unpriced_models,billing_note 标注代理价、非账单。
overall = store.dual_column_report_by_project(project_id)
conn = store._conn
conn.row_factory = sqlite3.Row
days = (query.get("days") or [None])[0]
day = (query.get("day") or [None])[0]
daily = []
if day:
row = conn.execute(
"SELECT day, COALESCE(SUM(total_tokens),0) AS t,"
" COALESCE(SUM(usd_micros),0) AS u, COUNT(*) AS c,"
" COALESCE(SUM(fallback_count),0) AS f,"
" COALESCE(SUM(cached_input_tokens),0) AS cached,"
" COALESCE(SUM(CASE WHEN source = 'actual'"
" AND usd_micros IS NULL THEN total_tokens"
" ELSE 0 END),0) AS unpriced"
" FROM cost_records WHERE project_id = ? AND day = ?"
" GROUP BY day",
(project_id, day)).fetchone()
if row:
daily = [{"day": row["day"],
"total_tokens": int(row["t"]),
"usd_micros": int(row["u"]),
"turns": int(row["c"]),
"fallback_count": int(row["f"]),
"cached_input_tokens": int(row["cached"]),
"unpriced_actual_tokens": int(row["unpriced"])}]
elif days is not None:
try:
n = int(days)
except (TypeError, ValueError):
self._send_json(400, {"error": "days must be an integer"})
return
from datetime import date, timedelta
cutoff = (date.today() - timedelta(days=max(n - 1, 0))
).isoformat()
rows = conn.execute(
"SELECT day, COALESCE(SUM(total_tokens),0) AS t,"
" COALESCE(SUM(usd_micros),0) AS u, COUNT(*) AS c,"
" COALESCE(SUM(fallback_count),0) AS f,"
" COALESCE(SUM(cached_input_tokens),0) AS cached,"
" COALESCE(SUM(CASE WHEN source = 'actual'"
" AND usd_micros IS NULL THEN total_tokens"
" ELSE 0 END),0) AS unpriced"
" FROM cost_records WHERE project_id = ? AND day >= ?"
" GROUP BY day ORDER BY day",
(project_id, cutoff)).fetchall()
daily = [{"day": r["day"],
"total_tokens": int(r["t"]),
"usd_micros": int(r["u"]),
"turns": int(r["c"]),
"fallback_count": int(r["f"]),
"cached_input_tokens": int(r["cached"]),
"unpriced_actual_tokens": int(r["unpriced"])}
for r in rows]
finally:
try:
store._conn.close()
except Exception:
pass
self._send_json(200, {"project_id": project_id, "overall": overall,
"daily": daily})
# -- 任务成本与模型总览 ----------------------------------------------
def _task_cost(self, user_id, task_id):
row = self._fetch_task(task_id)
if row is None:
self._send_json(404, {"error": "task not found"})
return
task = _task_to_dict(row)
with _auth_store(self.server.db_path) as auth:
if not auth.check_permission(user_id, task["project_id"], "read"):
self._send_json(403, {"error": "forbidden"})
return
store = CostStore(self.server.db_path)
try:
# 双列:与项目成本同款,旧键全保留,只加双列键。
report = store.dual_column_report_by_task(task_id)
finally:
try:
store._conn.close()
except Exception:
pass
self._send_json(200, {"task_id": task_id,
"project_id": task["project_id"], **report})
def _relay_models(self, user_id):
# 中转可用模型列表:服务端凭环境变量里的 relay key 现取
# 现回,只把模型 id 交给调用者。key 绝不进响应、日志、
# 缓存(见 cloud/relay.py)。单测可经 server.relay_fetcher
# 注入假取数函数;中转挂了或未配 key 时 502,不编造列表。
# 网关为 sub2api 时:模型目录因用户而异(key 绑定的组
# 决定)。P1 收紧:首次调模型目录即做真正的懒开通/
# 健康检查,不再只回 provisioning: pending——无行则开通,
# 旧 key 经 /v1/models 实测已死(401/403,含余额为 0 的
# 403)则轮换重开;网关本身不可用时回 502、保留旧行。
# key 明文绝不进响应、日志、缓存。
if _gateway_name() == "sub2api":
conn = self._db()
try:
try:
status, api_key, _sub, _kid = _validate_sub2api_key(
conn, user_id)
except _Sub2apiError as exc:
self._send_json(502, {"error": str(exc)})
return
except Exception as exc:
self._send_json(502, {"error": str(exc)})
return
finally:
conn.close()
try:
models = _list_sub2api_models(_gateway_origin(), api_key)
except Exception as exc:
self._send_json(502, {"error": str(exc)})
return
self._send_json(200, {"models": models, "gateway": "sub2api",
"cached": False, "updated_at": None,
"provisioning": status})
return
fetcher = getattr(self.server, "relay_fetcher", None)
try:
models, cached, updated_at = _get_relay_models(fetcher=fetcher)
except _RelayError as exc:
self._send_json(502, {"error": str(exc)})
return
self._send_json(200, {
"models": models, "cached": cached, "updated_at": updated_at,
"gateway": "xcmapi"})
def _models_overview(self, user_id):
# 优先级列表来自环境变量 MODELS_PRIORITY(逗号分隔),未设时
# 用默认列表;这只是控制面的默认配置,不是 provider 的承诺。
raw = os.environ.get("MODELS_PRIORITY", "")
priority = [m.strip() for m in raw.split(",") if m.strip()] or [
"gpt-6.1-sol", "gpt-6-sol", "gpt-5.6-sol"]
# 只统计调用者是成员的项目,避免跨项目泄露成本。
conn = self._db()
try:
member_ids = [r["project_id"] for r in conn.execute(
"SELECT project_id FROM project_members WHERE user_id = ?",
(user_id,)).fetchall()]
finally:
conn.close()
if not member_ids:
self._send_json(200, {"priority_list": priority,
"model_stats": [], "fallback_events": []})
return
placeholders = ",".join("?" for _ in member_ids)
store = CostStore(self.server.db_path)
try:
conn = store._conn
conn.row_factory = sqlite3.Row
rows = conn.execute(
"SELECT model, COUNT(*) AS turns,"
" COALESCE(SUM(total_tokens),0) AS tokens,"
" COALESCE(SUM(fallback_count),0) AS fallbacks"
" FROM cost_records WHERE project_id IN (%s)"
" GROUP BY model ORDER BY model" % placeholders,
member_ids).fetchall()
fb_rows = conn.execute(
"SELECT task_id, project_id, model, fallback_count"
" FROM cost_records WHERE project_id IN (%s)"
" AND fallback_count > 0"
" ORDER BY created_at DESC LIMIT 50" % placeholders,
member_ids).fetchall()
finally:
try:
store._conn.close()
except Exception:
pass
self._send_json(200, {
"priority_list": priority,
"model_stats": [
{"model": r["model"], "turns": int(r["turns"]),
"total_tokens": int(r["tokens"]),
"fallback_count": int(r["fallbacks"])} for r in rows],
"fallback_events": [
{"task_id": r["task_id"], "project_id": r["project_id"],
"model": r["model"],
"fallback_count": int(r["fallback_count"])}
for r in fb_rows],
})
+634
View File
@@ -0,0 +1,634 @@
"""Handler mixin:OAuthMixin。
由 CloudControlPlaneHandler 拆出,方法体逐字搬移;
self.* 调用(含其它 mixin 的方法与留守的 _send_json / _db 等)
在混入后的同一类上解析,行为不变。本模块不 import cloud.server。
"""
from cloud.github_app import (
DEFAULT_APP_NAME,
DEFAULT_APP_URL,
AppError,
SetupStateStore,
build_app_manifest,
exchange_manifest_code,
list_installations,
public_app_status,
save_app_config,
)
from cloud.oauth import (
BindError,
OAuthError,
OAuthStateStore,
bind_external_identity,
build_authorization_url,
default_userinfo_client,
find_bound_user,
handle_oauth_callback,
list_bindings,
map_external_identity_to_local_user_with_key,
start_oauth_login,
unbind_external_identity,
)
from cloud.sub2api import ensure_user_gateway_key as _ensure_sub2api_key
from cloud.sub2api import gateway_origin as _gateway_origin
import html
import json
import os
from urllib.parse import parse_qs, quote, urlparse
from cloud.handler_support import (
_apply_default_quota as _apply_default_quota,
_auth_store as _auth_store,
_default_quota_usd_micros as _default_quota_usd_micros,
_global_admin_ids as _global_admin_ids,
)
class OAuthMixin:
"""OAuthMixin 领域方法(见模块 docstring)。"""
# -- OAuth(pre-auth,靠 state + secret,不走 X-API-Key)-----------
def _oauth_provider(self, path):
parts = [p for p in path.split("/") if p]
if len(parts) != 3 or parts[0] != "oauth":
return None
return parts[1]
def _oauth_login(self, path):
provider = self._oauth_provider(path)
if provider is None:
self._send_json(404, {"error": "not found"})
return
query = parse_qs(urlparse(self.path).query)
client_id = (query.get("client_id") or [None])[0]
redirect_uri = (query.get("redirect_uri") or [None])[0]
scope = (query.get("scope") or [None])[0]
if not client_id or not redirect_uri:
self._send_json(400, {"error": "client_id and redirect_uri required"})
return
store = OAuthStateStore(self.server.db_path)
try:
state, url = start_oauth_login(
store, provider, client_id, redirect_uri, scope=scope)
except OAuthError as e:
self._send_json(400, {"error": str(e), "reason": e.reason})
return
except ValueError as e:
self._send_json(400, {"error": str(e)})
return
finally:
try:
store.conn.close()
except Exception:
pass
# 只返回 URL,不直接跳转到任何登录页;跳转由调用方决定。
self._send_json(200, {"provider": provider, "state": state,
"authorization_url": url})
def _oauth_clients(self, provider):
"""回调用的 http/userinfo 客户端:测试注入优先,生产走真实路径。
server 属性 oauth_http_client / oauth_userinfo_client 由
create_server 置为 None(生产)或由测试注入 fake;http_client
已注入(fake)而 userinfo 未注入时不走真实 userinfo(fake
token 自带身份字段时不需要它,走真实端点反而会用假 token
撞真实 GitHub)。生产(两者都未注入)时 userinfo 走真实端点。
"""
http_client = getattr(self.server, "oauth_http_client", None)
userinfo_client = getattr(self.server, "oauth_userinfo_client", None)
if userinfo_client is None and http_client is None:
userinfo_client = default_userinfo_client(provider)
return http_client, userinfo_client
def _oauth_finish(self, provider, code, state, client_id, redirect_uri):
"""回调共用后半段:验 state -> 换 token -> 取身份 -> 分叉。
成功返回 6 元组
``(local_id, created, api_key_plain, rotated, identity, is_bind)``;
失败抛 OAuthError(调用方按 JSON/HTML 分别呈现,绝不把
access_token、client_secret、旧 API key 写进任何响应),
绑定冲突抛 BindError(调用方按 409 呈现)。
client_id / redirect_uri 为 None 时由 handle_oauth_callback
从 state 记录取回(真实浏览器回调只带 code + state)。
分叉规则(顺序即优先级):
1. state 带 bind_user_id -> 绑定分支:把身份绑到该本站
账号,返回 is_bind=True、api_key_plain=None(绑定成功页
绝不显示任何 key);冲突抛 BindError;
2. 身份已绑定到某本站账号 -> 登录落到被绑账号,并按
OAuth 登录规矩签发附加 key(created=False、rotated
恒为 False),与 oauth 账号登录同一语义,不发明新语义;
3. 否则走原来的确定性 oauth 账号映射(新建/复用 + 签发
附加 key)。
"""
store = OAuthStateStore(self.server.db_path)
try:
http_client, userinfo_client = self._oauth_clients(provider)
identity = handle_oauth_callback(
store, provider, code, state, client_id, redirect_uri,
http_client=http_client, userinfo_client=userinfo_client)
finally:
try:
store.conn.close()
except Exception:
pass
with _auth_store(self.server.db_path) as auth:
bind_user_id = identity.get("bind_user_id")
if bind_user_id:
binding = bind_external_identity(
auth, provider, identity, bind_user_id)
auth.audit(bind_user_id, "", "bind_external",
"%s:%s" % (provider, identity.get("external_id")))
if _apply_default_quota(auth.conn, bind_user_id):
auth.audit(bind_user_id, "", "quota_default",
"quota_usd_micros=%d"
% _default_quota_usd_micros())
identity["already_bound"] = binding["already"]
return (bind_user_id, False, None, False, identity, True)
bound_user_id = find_bound_user(auth, provider, identity)
if bound_user_id is not None:
_key_id, api_key_plain = auth.create_api_key(
bound_user_id, "oauth:%s" % provider)
auth.audit(bound_user_id, "", "oauth_login",
"%s:%s" % (provider, identity.get("external_id")))
auth.audit(bound_user_id, "", "api_key_issued", provider)
if _apply_default_quota(auth.conn, bound_user_id):
auth.audit(bound_user_id, "", "quota_default",
"quota_usd_micros=%d"
% _default_quota_usd_micros())
return (bound_user_id, False, api_key_plain, False, identity,
False)
local_id, created, api_key_plain, rotated = \
map_external_identity_to_local_user_with_key(
auth, provider, identity)
auth.audit(local_id, "", "oauth_login",
"%s:%s" % (provider, identity.get("external_id")))
if _apply_default_quota(auth.conn, local_id):
auth.audit(local_id, "", "quota_default",
"quota_usd_micros=%d"
% _default_quota_usd_micros())
if created and _gateway_origin() is not None:
# sub2api 开通(best-effort):建号不因网关开通失败
# 而失败,任务时 _ensure_gateway 会懒开通兜底。
try:
_ensure_sub2api_key(auth.conn, local_id)
except Exception as exc:
auth.audit(local_id, "", "sub2api_provision_failed",
str(exc)[:200])
if not created:
auth.audit(local_id, "", "api_key_issued", provider)
return local_id, created, api_key_plain, rotated, identity, False
def _oauth_callback(self, path):
provider = self._oauth_provider(path)
if provider is None:
self._send_json(404, {"error": "not found"})
return
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
code = body.get("code")
state = body.get("state")
client_id = body.get("client_id")
redirect_uri = body.get("redirect_uri")
if not code or not state:
self._send_json(400, {"error": "code and state required"})
return
try:
local_id, created, api_key_plain, rotated, identity, is_bind = \
self._oauth_finish(provider, code, state, client_id,
redirect_uri)
except BindError as e:
self._send_json(409, {"error": str(e), "reason": e.reason})
return
except OAuthError as e:
self._send_json(400, {"error": str(e), "reason": e.reason})
return
if is_bind:
# 绑定成功:绝不返回任何 key(绑定页的 completer 未必是
# 发起人,key 一旦上页就可能落到别人手里);发起人用
# 自己的 key 调 GET /oauth/{provider}/bindings 核对。
self._send_json(200, {"linked": True, "provider": provider,
"login": identity.get("login"),
"external_id": identity.get("external_id"),
"user_id": local_id,
"already_bound":
identity.get("already_bound", False)})
return
# api_key 明文只在这里出现一次:调用方须立即保存,
# 之后任何接口都不再返回它;登录签发的是附加 key,旧 key
# 仍可继续使用(多 key;不用的旧 key 可在 keys 管理里撤销)。
self._send_json(200, {"user_id": local_id, "created": created,
"provider": provider,
"login": identity.get("login"),
"email": identity.get("email"),
"api_key": api_key_plain,
"api_key_rotated": rotated})
def _oauth_error_page(self, reason, message):
"""GET 回调的 400 错误页:只含 reason 与提示,不含任何密钥、
token 或旧 API key(message 来自 OAuthError,本身不含密钥,
仍做 HTML 转义)。"""
return (
"<!doctype html><html><head><meta charset='utf-8'>"
"<title>OAuth login failed</title></head><body>"
"<h1>OAuth login failed</h1>"
"<p>reason: %s</p><p>%s</p>"
"<p>Please restart login from the login page.</p>"
"</body></html>"
% (html.escape(str(reason or "error")),
html.escape(str(message))))
def _oauth_success_page(self, provider, local_id, created, login,
api_key_plain, rotated):
"""GET 回调的成功页:只显示 login、user_id、created 与新 API key
一次;绝不显示 access_token、client_secret、旧 API key。
rotated 只为向后兼容保留(登录不再轮换,恒为 False):
新 key 是附加 key,旧 key 仍可继续使用,页面不再写“旧 key
已失效”,而是说明多 key 与撤销入口。
"""
rotation_note = (
"<p>Your previous API key has been rotated and no longer works.</p>"
if rotated else
"<p>Previous keys still work (multi-key). Revoke unused keys "
"via the API keys management endpoints.</p>"
if not created else "")
return (
"<!doctype html><html><head><meta charset='utf-8'>"
"<title>OAuth login succeeded</title></head><body>"
"<h1>OAuth login succeeded</h1>"
"<p>provider: %s</p>"
"<p>login: %s</p>"
"<p>user_id: %s</p>"
"<p>new user: %s</p>"
"%s"
"<p>Your API key (shown only once, save it now):</p>"
"<p><code>%s</code></p>"
"<p>Please save it immediately. It will not be shown again "
"after you leave or refresh this page."
"</p>"
"<p>Use it as the X-API-Key header for API calls.</p>"
"</body></html>"
% (html.escape(str(provider)),
html.escape(str(login)),
html.escape(str(local_id)),
"yes" if created else "no",
rotation_note,
html.escape(str(api_key_plain))))
def _oauth_bind_success_page(self, provider, login):
"""绑定成功页:只显示 provider + 外部 login + 成功提示。
绝不显示任何 key,也不显示本站 user_id:回调页的 completer
未必是绑定发起人,key 与内部 id 一旦上页就可能落到别人
手里;发起人用自己的 key 调 GET /oauth/{provider}/bindings
核对绑定。
"""
return (
"<!doctype html><html><head><meta charset='utf-8'>"
"<title>OAuth binding succeeded</title></head><body>"
"<h1>OAuth binding succeeded</h1>"
"<p>provider: %s</p>"
"<p>login: %s</p>"
"<p>Your external identity is now bound to your account.</p>"
"<p>You can verify it from your bindings list.</p>"
"</body></html>"
% (html.escape(str(provider)),
html.escape(str(login))))
def _oauth_callback_get(self, path):
provider = self._oauth_provider(path)
if provider is None:
self._send_html(404, self._oauth_error_page(
"not_found", "unknown oauth path"))
return
query = parse_qs(urlparse(self.path).query)
code = (query.get("code") or [None])[0]
state = (query.get("state") or [None])[0]
client_id = (query.get("client_id") or [None])[0]
redirect_uri = (query.get("redirect_uri") or [None])[0]
if not code or not state:
self._send_html(400, self._oauth_error_page(
"missing_params", "code and state required"))
return
try:
local_id, created, api_key_plain, rotated, identity, is_bind = \
self._oauth_finish(provider, code, state, client_id,
redirect_uri)
except BindError as e:
self._send_html(409, self._oauth_error_page(
e.reason or "bind_conflict", str(e)))
return
except OAuthError as e:
self._send_html(400, self._oauth_error_page(
e.reason or "oauth_failed", str(e)))
return
if is_bind:
self._send_html(200, self._oauth_bind_success_page(
provider, identity.get("login")))
return
self._send_html(200, self._oauth_success_page(
provider, local_id, created, identity.get("login"),
api_key_plain, rotated))
# -- OAuth 绑定(已有账号绑外部身份)------------------------------
def _oauth_bind_route(self, path):
"""绑定相关路径解析:返回 (provider, action) 或 None。
POST /oauth/{provider}/bind/initiate -> (provider, "bind_initiate")
GET|DELETE /oauth/{provider}/bindings -> (provider, "bindings")
其它 /oauth/ 路径返回 None,走原来的登录/回调分发。
"""
parts = [p for p in path.split("/") if p]
if len(parts) == 4 and parts[0] == "oauth" \
and parts[2] == "bind" and parts[3] == "initiate":
return parts[1], "bind_initiate"
if len(parts) == 3 and parts[0] == "oauth" \
and parts[2] == "bindings":
return parts[1], "bindings"
return None
def _oauth_bind_initiate(self, provider):
"""绑定起点:须持本站 key(与登录不同,绑定必须知道绑到谁)。
body 须含 client_id + redirect_uri(与登录同款);state 里
带 bind_user_id,回调时靠它走绑定分支。只返回授权 URL,
不直接跳转;跳转由调用方决定。
"""
user_id = self._authenticate()
if user_id is None:
self._send_json(401, {"error": "unauthorized"})
return
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
client_id = body.get("client_id")
redirect_uri = body.get("redirect_uri")
scope = body.get("scope")
if not client_id or not redirect_uri:
self._send_json(400, {"error": "client_id and redirect_uri required"})
return
store = OAuthStateStore(self.server.db_path)
try:
state = store.create_state(
provider, redirect_uri, client_id=client_id,
bind_user_id=user_id)
url = build_authorization_url(
provider, client_id, redirect_uri, scope=scope, state=state)
except OAuthError as e:
self._send_json(400, {"error": str(e), "reason": e.reason})
return
except ValueError as e:
self._send_json(400, {"error": str(e)})
return
finally:
try:
store.conn.close()
except Exception:
pass
self._send_json(200, {"provider": provider, "state": state,
"authorization_url": url})
def _oauth_bindings_get(self, provider):
"""列出调用方自己的绑定(只看得到自己的,别人的看不到)。"""
user_id = self._authenticate()
if user_id is None:
self._send_json(401, {"error": "unauthorized"})
return
with _auth_store(self.server.db_path) as auth:
bindings = list_bindings(auth, user_id)
self._send_json(200, {"provider": provider, "bindings": bindings})
def _oauth_bindings_delete(self, provider):
"""解绑调用方自己在该 provider 上的绑定:本来就没有返回 404。
解绑后该身份登录恢复原来的 oauth 账号映射(旧 oauth 账号
若存在,key 仍可用,不会被解绑删掉)。
"""
user_id = self._authenticate()
if user_id is None:
self._send_json(401, {"error": "unauthorized"})
return
with _auth_store(self.server.db_path) as auth:
ok = unbind_external_identity(auth, provider, user_id)
if ok:
auth.audit(user_id, "", "unbind_external", provider)
if not ok:
self._send_json(404, {"error": "no binding for provider"})
return
self._send_json(200, {"unbound": True, "provider": provider})
# -- GitHub App 一键安装 ------------------------------------------
def _github_app_public_base(self):
base = os.environ.get("GITHUB_APP_PUBLIC_BASE") or ""
return base.rstrip("/") or None
def _github_app_error_page(self, reason, message):
"""App 流程错误页:只含 reason 与提示,不含任何 secret、
pem、code、state(message 来自流程错误,本身不含密钥,
仍做 HTML 转义)。"""
return (
"<!doctype html><html><head><meta charset='utf-8'>"
"<title>GitHub App setup failed</title></head><body>"
"<h1>GitHub App setup failed</h1>"
"<p>reason: %s</p><p>%s</p>"
"</body></html>"
% (html.escape(str(reason or "error")),
html.escape(str(message))))
def _github_app_setup_state(self):
"""签发 setup state:只限全局管理员(见 _is_global_admin)。
P0 安全批前任何持 key 用户都能签发并回调覆盖全局
App 配置;现在非管理员直接 403 并记审计。要求环境
变量 GITHUB_APP_PUBLIC_BASE(公网域名,如
https://api.diante.me):manifest 里的 hook / 回调地址
都由它拼出。返回 {state, setup_url},setup_url 是用户
在浏览器里打开的表单页;state 一次性、1 小时过期,
且绑定签发人。
"""
user_id = self._authenticate()
if user_id is None:
self._send_json(401, {"error": "unauthorized"})
return
if not self._is_global_admin(user_id):
self._audit_admin(user_id, "github_app_setup_denied",
"setup state")
self._send_json(403, {"error": "forbidden"})
return
base = self._github_app_public_base()
if not base:
self._send_json(400, {
"error": "GITHUB_APP_PUBLIC_BASE not configured"})
return
body, err = self._read_json()
if err:
body = {}
org = (body or {}).get("org")
store = SetupStateStore(self.server.db_path)
try:
state = store.create_state(issued_by=user_id)
finally:
try:
store.conn.close()
except Exception:
pass
setup_url = base + "/github-app/setup?state=" + state
if org:
setup_url += "&org=" + quote(str(org))
self._send_json(200, {"state": state, "setup_url": setup_url})
def _github_app_setup_page(self):
"""建 App 表单页:公开页,但 state 必须是操作方签发的。
无 state 或 state 未签发时不渲染表单(state 是回调的
唯一凭证,公开页自己现发 state 等于谁都能覆盖 App 配置)。
表单按 GitHub manifest 文档 POST 到 github.com 的建 App
页(个人账号 /settings/apps/new,组织账号
/organizations/{org}/settings/apps/new),manifest JSON
由 JS 填进隐藏字段;用户在 GitHub 页确认 Create 后跳回
本站 /github-app/callback。
"""
query = parse_qs(urlparse(self.path).query)
state = (query.get("state") or [None])[0]
org = (query.get("org") or [None])[0]
if not state:
self._send_html(400, self._github_app_error_page(
"missing_state",
"setup state required: ask the operator to issue one"))
return
base = self._github_app_public_base()
if not base:
self._send_html(400, self._github_app_error_page(
"not_configured", "GITHUB_APP_PUBLIC_BASE not configured"))
return
manifest = build_app_manifest(
base,
app_name=os.environ.get("GITHUB_APP_NAME") or DEFAULT_APP_NAME,
app_url=os.environ.get("GITHUB_APP_URL") or DEFAULT_APP_URL,
public=os.environ.get("GITHUB_APP_PUBLIC") == "1")
manifest_json = json.dumps(manifest).replace("<", "\\u003c")
if org:
action = ("https://github.com/organizations/%s"
"/settings/apps/new?state=%s"
% (quote(str(org)),
quote(state)))
else:
action = ("https://github.com/settings/apps/new?state=%s"
% quote(state))
perms = ", ".join("%s: %s" % kv
for kv in manifest["default_permissions"].items())
events = ", ".join(manifest["default_events"])
page = (
"<!doctype html><html><head><meta charset='utf-8'>"
"<title>Create GitHub App</title></head><body>"
"<h1>Create GitHub App</h1>"
"<p>App name: %s</p>"
"<p>Webhook URL: %s</p>"
"<p>Permissions: %s</p>"
"<p>Events: %s</p>"
"<p>Click Submit to create the app on GitHub. GitHub will "
"redirect back here to finish setup.</p>"
"<form action=\"%s\" method=\"post\">"
"<input type=\"hidden\" name=\"manifest\" id=\"manifest\">"
"<input type=\"submit\" value=\"Create GitHub App\">"
"</form>"
"<script>document.getElementById(\"manifest\").value = "
"JSON.stringify(%s);</script>"
"</body></html>"
% (html.escape(manifest["name"]),
html.escape(manifest["hook_attributes"]["url"]),
html.escape(perms), html.escape(events),
html.escape(action), manifest_json))
self._send_html(200, page)
def _github_app_callback(self):
"""建 App 回调:验 setup state -> 换 code -> 存配置。
state 一次性、1 小时过期;换 code 失败或响应缺字段时
报错误页,不存任何东西。成功页只显示 App 身份(slug、
id、主页)与安装链接,绝不显示 webhook_secret 与 pem
(两者只在库里,server 进程内用)。
P0 安全批后:回调要求签发人仍在 CODEX_GLOBAL_ADMINS
里,否则 state 不消耗、直接报 invalid_state;覆盖全局
配置时记审计。诚实边界:回调是浏览器无认证重定向,
这里核对的是“签发人仍是管理员”,不是“回调者就是
签发人”(见 SetupStateStore.consume_state 文档)。
"""
query = parse_qs(urlparse(self.path).query)
code = (query.get("code") or [None])[0]
state = (query.get("state") or [None])[0]
if not code or not state:
self._send_html(400, self._github_app_error_page(
"missing_params", "code and state required"))
return
store = SetupStateStore(self.server.db_path)
try:
issuer = store.state_issuer(state)
ok = store.consume_state(
state, allowed_issuers=_global_admin_ids())
finally:
try:
store.conn.close()
except Exception:
pass
if not ok:
self._audit_admin(
issuer or "unknown", "github_app_callback_denied",
"state issuer not a global admin")
self._send_html(400, self._github_app_error_page(
"invalid_state", "setup state invalid or expired"))
return
http_client = getattr(self.server, "github_app_exchange_client",
None)
try:
config = exchange_manifest_code(code, http_client=http_client)
except AppError as e:
self._send_html(400, self._github_app_error_page(
e.reason or "exchange_failed", str(e)))
return
conn = self._db()
try:
save_app_config(conn, config)
finally:
conn.close()
self._audit_admin(
issuer or "unknown", "github_app_config_replaced",
"slug=%s app_id=%s" % (config.get("slug"),
config.get("app_id")))
slug = config.get("slug") or ""
page = (
"<!doctype html><html><head><meta charset='utf-8'>"
"<title>GitHub App created</title></head><body>"
"<h1>GitHub App created</h1>"
"<p>App: %s</p>"
"<p>App ID: %s</p>"
"<p>Install it on your repositories:</p>"
"<p><a href=\"https://github.com/apps/%s/installations/new\">"
"Install this GitHub App</a></p>"
"<p>After installing, webhooks from the app are verified "
"automatically. API actions (push, PR) still use the "
"existing PAT until the App-token phase; you may remove "
"the manual webhook once the app is installed.</p>"
"</body></html>"
% (html.escape(slug),
html.escape(str(config.get("app_id"))),
html.escape(slug)))
self._send_html(200, page)
def _github_app_status(self):
"""App 配置与安装状态:须持本站 key;绝不返回 secret/pem。"""
user_id = self._authenticate()
if user_id is None:
self._send_json(401, {"error": "unauthorized"})
return
conn = self._db()
try:
status = public_app_status(conn)
status["installations"] = list_installations(conn)
finally:
conn.close()
self._send_json(200, status)
+744
View File
@@ -0,0 +1,744 @@
"""Handler mixin:ProjectMixin。
由 CloudControlPlaneHandler 拆出,方法体逐字搬移;
self.* 调用(含其它 mixin 的方法与留守的 _send_json / _db 等)
在混入后的同一类上解析,行为不变。本模块不 import cloud.server。
"""
from cloud.auth import ROLES as _MEMBER_ROLES
from cloud.db import PostgresIntegrityError as _PgIntegrityError
from cloud.push import (
delete_subscription as _push_delete,
list_subscriptions as _push_list,
)
from cloud.peer_messages import row_get as _row_get
from cloud.github_app import (
list_installations,
)
import re
import sqlite3
import time
import uuid
from cloud.repos import validate_repo_url
from cloud.handler_support import (
MAX_PROJECTS_PER_USER as MAX_PROJECTS_PER_USER,
MAX_PROJECT_ID_LEN as MAX_PROJECT_ID_LEN,
MAX_PROJECT_NAME_LEN as MAX_PROJECT_NAME_LEN,
_auth_store as _auth_store,
_environment_to_dict as _environment_to_dict,
)
class ProjectMixin:
"""ProjectMixin 领域方法(见模块 docstring)。"""
# -- 环境 ----------------------------------------------------------
def _fetch_environment(self, env_id):
conn = self._db()
try:
return conn.execute(
"SELECT * FROM environments WHERE id = ?", (env_id,)
).fetchone()
finally:
conn.close()
def _create_environment(self, user_id):
"""建环境:只记录意图(status=creating),检出由 worker
在下一轮准备(clone + checkout 分支),准备好后 status
变 ready,失败变 failed 并带 error_text——创建本身是同步
返回的记录,准备是异步的,与任务 queued->running 同款语义。
同项目内未归档环境名唯一(409),避免选环境时分不清;
归档过的名字可重用。项目没有 repo_url 时 400:环境就是
仓库的基准检出,没有仓库就没有环境。"""
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
project_id = body.get("project_id")
if not project_id or not isinstance(project_id, str):
self._send_json(400, {"error": "project_id required"})
return
name = body.get("name")
if not name or not isinstance(name, str) or not name.strip():
self._send_json(400, {"error": "name required"})
return
name = name.strip()
if len(name) > 100:
self._send_json(400, {"error": "name too long"})
return
executor = body.get("executor")
if executor is not None and executor not in ("local", "docker"):
self._send_json(400, {
"error": "executor must be local or docker"})
return
try:
branch = _validate_env_branch(body.get("branch"))
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
err = self._check_project(user_id, project_id, "create_task")
if err:
self._send_json(err[0], {"error": err[1]})
return
now = time.time()
env_id = uuid.uuid4().hex
conn = self._db()
try:
proj = conn.execute(
"SELECT repo_url FROM projects WHERE id = ?",
(project_id,)).fetchone()
if proj is None:
self._send_json(404, {"error": "project not found"})
return
repo_url = _row_get(proj, "repo_url")
if not repo_url:
self._send_json(400, {
"error": "project has no repo_url, "
"environment needs a repo"})
return
existing = conn.execute(
"SELECT id FROM environments WHERE project_id = ?"
" AND name = ? AND status != 'archived'",
(project_id, name)).fetchone()
if existing:
self._send_json(409, {
"error": "environment name already exists"
" in this project"})
return
try:
conn.execute(
"INSERT INTO environments(id, project_id, name,"
" branch, executor, status, error_text, repo_url,"
" created_by, created_at, updated_at)"
" VALUES(?, ?, ?, ?, ?, 'creating', NULL, ?, ?,"
" ?, ?)",
(env_id, project_id, name, branch, executor,
repo_url, user_id, now, now))
conn.commit()
except (sqlite3.IntegrityError, _PgIntegrityError):
self._send_json(409, {
"error": "environment name already exists"
" in this project"})
return
row = conn.execute(
"SELECT * FROM environments WHERE id = ?", (env_id,)
).fetchone()
finally:
conn.close()
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "environment_created",
env_id)
self._send_json(201, _environment_to_dict(row))
def _list_environments(self, user_id, query):
project_ids = query.get("project_id")
project_id = project_ids[0] if project_ids else None
if not project_id:
self._send_json(400, {"error": "project_id required"})
return
err = self._check_project(user_id, project_id, "read")
if err:
self._send_json(err[0], {"error": err[1]})
return
include_archived = query.get("include_archived") == ["1"]
conn = self._db()
try:
if include_archived:
rows = conn.execute(
"SELECT * FROM environments WHERE project_id = ?"
" ORDER BY created_at, id",
(project_id,)).fetchall()
else:
rows = conn.execute(
"SELECT * FROM environments WHERE project_id = ?"
" AND status != 'archived'"
" ORDER BY created_at, id",
(project_id,)).fetchall()
finally:
conn.close()
self._send_json(200, {
"environments": [_environment_to_dict(r) for r in rows]})
def _get_environment(self, user_id, env_id):
row = self._fetch_environment(env_id)
if row is None:
self._send_json(404, {"error": "environment not found"})
return
env = _environment_to_dict(row)
err = self._check_project(user_id, env["project_id"], "read")
if err:
self._send_json(err[0], {"error": err[1]})
return
self._send_json(200, env)
def _archive_environment(self, user_id, env_id):
"""归档环境:已归档时幂等返回 200;有 queued/running
任务正在用它时 409,绝不让进行中的任务失去基准检出。
归档只改状态,不删检出目录(已完成任务的 worktree 独立,
检出留着可查;磁盘清理另行手动做,不在归档里顺手删)。"""
row = self._fetch_environment(env_id)
if row is None:
self._send_json(404, {"error": "environment not found"})
return
env = _environment_to_dict(row)
err = self._check_project(
user_id, env["project_id"], "create_task")
if err:
self._send_json(err[0], {"error": err[1]})
return
if env["status"] == "archived":
self._send_json(200, env)
return
now = time.time()
conn = self._db()
try:
using = conn.execute(
"SELECT id FROM tasks WHERE environment_id = ?"
" AND status IN ('queued', 'running') LIMIT 1",
(env_id,)).fetchone()
if using:
self._send_json(409, {
"error": "environment is in use by a queued or"
" running task"})
return
conn.execute(
"UPDATE environments SET status = 'archived',"
" updated_at = ? WHERE id = ?",
(now, env_id))
conn.commit()
row = conn.execute(
"SELECT * FROM environments WHERE id = ?", (env_id,)
).fetchone()
finally:
conn.close()
with _auth_store(self.server.db_path) as store:
store.audit(user_id, env["project_id"],
"environment_archived", env_id)
self._send_json(200, _environment_to_dict(row))
# -- 项目 ----------------------------------------------------------
def _create_project(self, user_id):
body, err = self._read_json()
if err:
self._send_json(400, {"error": err})
return
name = body.get("name")
if not name or not isinstance(name, str):
self._send_json(400, {"error": "name required"})
return
if len(name) > MAX_PROJECT_NAME_LEN:
self._send_json(400, {
"error": "project name too long: max %d characters"
% MAX_PROJECT_NAME_LEN})
return
project_id = body.get("id") or uuid.uuid4().hex
if not isinstance(project_id, str):
self._send_json(400, {"error": "id must be a string"})
return
if len(project_id) > MAX_PROJECT_ID_LEN:
self._send_json(400, {
"error": "project id too long: max %d characters"
% MAX_PROJECT_ID_LEN})
return
# 单用户项目数上限:按 projects.owner_id 计(成员身份加入
# 的项目不计入自己的上限,避免“被拉进很多项目就建不了
# 项目”);超限 400 且不建项目。
_count_conn = self._db()
try:
_owned = _count_conn.execute(
"SELECT COUNT(*) FROM projects WHERE owner_id = ?",
(user_id,)).fetchone()[0]
finally:
_count_conn.close()
if _owned >= MAX_PROJECTS_PER_USER:
self._send_json(400, {
"error": "project limit exceeded: max %d projects"
" per user" % MAX_PROJECTS_PER_USER})
return
# repo_url 可选:不传时为 None(老行为不变);传了则与 worker
# 共用 cloud.repos.validate_repo_url 的同一套校验,不合法时
# 400 且不建项目,绝不把带凭证的 URL 存进库。
repo_url = body.get("repo_url")
if repo_url is not None:
try:
repo_url = validate_repo_url(repo_url)
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
# auto_pr 可选,默认 0(关闭):只接受 0/1(布尔 True/False
# 按 1/0 计),其它值 400 且不建项目。本轮没有现成的
# PATCH /projects/{id} 路由,auto_pr 只能在创建时指定,
# 建后改它需直接改库或重建项目(见 README)。
auto_pr_raw = body.get("auto_pr", 0)
if isinstance(auto_pr_raw, bool):
auto_pr = 1 if auto_pr_raw else 0
elif isinstance(auto_pr_raw, int) and auto_pr_raw in (0, 1):
auto_pr = auto_pr_raw
else:
self._send_json(400, {"error": "auto_pr must be 0 or 1"})
return
now = time.time()
conn = self._db()
try:
existing = conn.execute(
"SELECT id FROM projects WHERE id = ?", (project_id,)
).fetchone()
if existing:
self._send_json(409, {"error": "project exists"})
return
conn.execute(
"INSERT INTO projects(id, name, owner_id, created_at,"
" repo_url, auto_pr) VALUES(?, ?, ?, ?, ?, ?)",
(project_id, name, user_id, now, repo_url, auto_pr),
)
conn.commit()
project = {
"id": project_id,
"name": name,
"owner_id": user_id,
"created_at": now,
"repo_url": repo_url,
"auto_pr": auto_pr,
}
except (sqlite3.IntegrityError, _PgIntegrityError):
self._send_json(409, {"error": "project exists"})
return
finally:
conn.close()
# AuthStore.create_project 只写成员表(owner 角色),项目记录
# 本身在上面的 projects 表;两者配合才算一个完整项目。
with _auth_store(self.server.db_path) as store:
store.create_project(project_id, user_id)
store.audit(user_id, project_id, "create_project", name)
self._send_json(201, project)
def _update_project(self, user_id, project_id):
"""PATCH /projects/{id}:项目 owner 可改 repo_url(建后补仓
库,解“建项目时没填仓库、之后建环境一直 400”的死结)。
repo_url 缺 key 时 400;null 或空串表示清空;字符串走与
创建时同一套 validate_repo_url 校验(带凭证的 URL 拒绝)。
只改 projects 行,不动已建环境的快照 repo_url——已建环境
仍用建环境时的仓库,新环境才用新值。改完记 update_project
审计。"""
err = self._check_project(user_id, project_id, "manage_project")
if err:
self._send_json(err[0], {"error": err[1]})
return
body, jerr = self._read_json()
if jerr:
self._send_json(400, {"error": jerr})
return
if "repo_url" not in body:
self._send_json(400, {"error": "repo_url required"})
return
repo_url = body.get("repo_url")
if repo_url is not None and repo_url != "":
if not isinstance(repo_url, str):
self._send_json(400, {"error": "repo_url must be a string"})
return
try:
repo_url = validate_repo_url(repo_url)
except ValueError as exc:
self._send_json(400, {"error": str(exc)})
return
else:
repo_url = None
conn = self._db()
try:
row = conn.execute(
"SELECT id, name, owner_id, created_at, repo_url, auto_pr"
" FROM projects WHERE id = ?",
(project_id,)).fetchone()
if row is None:
self._send_json(404, {"error": "project not found"})
return
conn.execute(
"UPDATE projects SET repo_url = ? WHERE id = ?",
(repo_url, project_id))
conn.commit()
row = conn.execute(
"SELECT id, name, owner_id, created_at, repo_url, auto_pr"
" FROM projects WHERE id = ?",
(project_id,)).fetchone()
finally:
conn.close()
with _auth_store(self.server.db_path) as store:
store.audit(user_id, project_id, "update_project",
repo_url or "(cleared)")
self._send_json(200, {"project": _project_to_dict(row)})
def _github_repos_list(self, user_id):
"""GET /github/repos:只返回调用者成员项目对应的仓库。
P1 收紧前返回全部安装记录(任一已认证用户可看到全部
安装仓库名,属跨租户信息泄露)。现按 project_members
过滤:只保留调用者是成员的项目的 repo_url 对应的仓库
(clone_url 归一化后比对:去 .git 后缀、比对小写);
调用者无成员项目或项目未填 repo_url 时返回空列表。
手填 URL 的能力不变——没进 App 安装的公开仓库仍可手动填。"""
conn = self._db()
try:
rows = list_installations(conn)
member_ids = {
row["project_id"]
for row in conn.execute(
"SELECT project_id FROM project_members"
" WHERE user_id = ?",
(user_id,)).fetchall()
}
allowed = set()
for row in conn.execute(
"SELECT id, repo_url FROM projects").fetchall():
if row["id"] not in member_ids or not row["repo_url"]:
continue
url = str(row["repo_url"]).strip().lower()
if url.endswith(".git"):
url = url[:-4]
allowed.add(url)
finally:
conn.close()
repos = []
for r in rows:
clone_url = "https://github.com/%s.git" % r["repo_full_name"]
normalized = clone_url.lower()
if normalized.endswith(".git"):
normalized = normalized[:-4]
if normalized not in allowed:
continue
repos.append({
"installation_id": r["installation_id"],
"account_login": r["account_login"],
"repo_full_name": r["repo_full_name"],
"clone_url": clone_url,
})
self._send_json(200, {"repos": repos})
def _list_projects(self, user_id):
conn = self._db()
try:
member_rows = conn.execute(
"SELECT project_id, role FROM project_members"
" WHERE user_id = ?",
(user_id,)).fetchall()
member_ids = {row["project_id"] for row in member_rows}
roles = {row["project_id"]: row["role"]
for row in member_rows}
rows = conn.execute(
"SELECT id, name, owner_id, created_at, repo_url, auto_pr"
" FROM projects ORDER BY created_at, id"
).fetchall()
finally:
conn.close()
projects = []
for r in rows:
if r["id"] not in member_ids:
continue
# 附加字段 my_role(只增字段,不改已有成功语义):
# 控制台据此按角色禁用重试/合并/提 PR 等按钮;服务端
# 权限仍以每次请求的 check_permission 为准,前端禁用
# 只是防误点,不是鉴权。
item = _project_to_dict(r)
item["my_role"] = roles.get(r["id"])
projects.append(item)
self._send_json(200, {"projects": projects})
# -- 项目成员:邀请 / 列表 / 改角色 -------------------------------
def _can_manage_members(self, user_id, project_id):
"""成员管理权:项目 owner 或全局管理员。
全局管理员(CODEX_GLOBAL_ADMINS)不一定是项目成员,
故不能只用项目内 manage_project 判定;项目 owner 判定
复用 check_permission 的 manage_project(仅 owner)。
"""
if self._is_global_admin(user_id):
return True
with _auth_store(self.server.db_path) as store:
return store.check_permission(
user_id, project_id, "manage_project")
def _set_member_row(self, store, project_id, actor_id, target_id,
role):
"""写成员行并记审计:owner 走 AuthStore.set_member(内含
owner 校验与审计);非成员的全局管理员走直接写 + 同格式
审计(set_member 的 owner 校验会挡住全局管理员,故分流)。
"""
if store.check_permission(actor_id, project_id,
"manage_project"):
store.set_member(project_id, actor_id, target_id, role)
return
store.conn.execute(
"INSERT OR REPLACE INTO project_members"
"(project_id, user_id, role) VALUES(?, ?, ?)",
(project_id, target_id, role))
store.conn.commit()
store.audit(actor_id, project_id, "set_member",
"%s:%s" % (target_id, role))
def _project_members_list(self, user_id, project_id):
"""GET /projects/{id}/members:项目成员(read)或全局
管理员可读。只返回 user_id / username / role,绝不返回
key 哈希等认证字段。"""
if not self._project_exists(project_id):
self._send_json(404, {"error": "project not found"})
return
if not self._is_global_admin(user_id):
err = self._check_project(user_id, project_id, "read")
if err:
self._send_json(err[0], {"error": err[1]})
return
conn = self._db()
try:
rows = conn.execute(
"SELECT m.user_id, m.role, u.username FROM"
" project_members m LEFT JOIN users u"
" ON u.id = m.user_id WHERE m.project_id = ?"
" ORDER BY u.username, m.user_id",
(project_id,)).fetchall()
finally:
conn.close()
self._send_json(200, {"project_id": project_id, "members": [
{"user_id": r["user_id"], "username": r["username"],
"role": r["role"]} for r in rows]})
def _resolve_invite_target(self, body):
"""邀请目标解析:优先 user_id,其次 username 查 id;
返回 (target_id, error_status, error)。"""
target = body.get("user_id")
username = body.get("username")
if target is not None and not isinstance(target, str):
return None, 400, "user_id must be a string"
if username is not None and not isinstance(username, str):
return None, 400, "username must be a string"
if not target and not username:
return None, 400, "user_id or username required"
conn = self._db()
try:
if target:
row = conn.execute(
"SELECT id FROM users WHERE id = ?",
(target,)).fetchone()
if row is None:
return None, 404, "user not found"
return row["id"], None, None
row = conn.execute(
"SELECT id FROM users WHERE username = ?",
(username,)).fetchone()
if row is None:
return None, 404, "user not found"
return row["id"], None, None
finally:
conn.close()
def _project_member_invite(self, user_id, project_id):
"""POST /projects/{id}/members:限项目 owner 或全局管理
员。已是成员时 409(改角色走 PATCH,不用邀请覆盖);
成功记 set_member 审计。role 缺省 contributor。"""
if not self._project_exists(project_id):
self._send_json(404, {"error": "project not found"})
return
if not self._can_manage_members(user_id, project_id):
self._send_json(403, {"error": "forbidden"})
return
body, jerr = self._read_json()
if jerr:
self._send_json(400, {"error": jerr})
return
role = body.get("role") or "contributor"
if role not in _MEMBER_ROLES:
self._send_json(400, {"error": "unknown role"})
return
target, estatus, eerr = self._resolve_invite_target(body)
if eerr:
self._send_json(estatus, {"error": eerr})
return
with _auth_store(self.server.db_path) as store:
existing = store.conn.execute(
"SELECT role FROM project_members"
" WHERE project_id = ? AND user_id = ?",
(project_id, target)).fetchone()
if existing is not None:
self._send_json(409, {"error": "already_member"})
return
self._set_member_row(store, project_id, user_id,
target, role)
self._send_json(201, {"project_id": project_id,
"user_id": target, "role": role})
def _project_member_patch(self, user_id, project_id, target):
"""PATCH /projects/{id}/members/{user_id}:限项目 owner
或全局管理员。唯一 owner 降为非 owner 时 409(无论降
自己还是降他人——否则项目将无 owner 可管);目标非成员
时 404。"""
if not self._project_exists(project_id):
self._send_json(404, {"error": "project not found"})
return
if not self._can_manage_members(user_id, project_id):
self._send_json(403, {"error": "forbidden"})
return
body, jerr = self._read_json()
if jerr:
self._send_json(400, {"error": jerr})
return
role = body.get("role")
if role not in _MEMBER_ROLES:
self._send_json(400, {"error": "unknown role"})
return
with _auth_store(self.server.db_path) as store:
cur = store.conn.execute(
"SELECT role FROM project_members"
" WHERE project_id = ? AND user_id = ?",
(project_id, target)).fetchone()
if cur is None:
self._send_json(404, {"error": "member not found"})
return
if cur[0] == "owner" and role != "owner":
owners = store.conn.execute(
"SELECT COUNT(*) FROM project_members"
" WHERE project_id = ? AND role = 'owner'",
(project_id,)).fetchone()[0]
if owners <= 1:
self._send_json(
409,
{"error": "cannot demote the only owner"})
return
self._set_member_row(store, project_id, user_id,
target, role)
self._send_json(200, {"project_id": project_id,
"user_id": target, "role": role})
def _project_member_delete(self, user_id, project_id, target):
"""DELETE /projects/{id}/members/{user_id}:限项目 owner
或全局管理员。删唯一 owner 时 409(否则项目将无 owner
可管);目标非成员时 404。成功记 remove_member 审计。"""
if not self._project_exists(project_id):
self._send_json(404, {"error": "project not found"})
return
if not self._can_manage_members(user_id, project_id):
self._send_json(403, {"error": "forbidden"})
return
with _auth_store(self.server.db_path) as store:
cur = store.conn.execute(
"SELECT role FROM project_members"
" WHERE project_id = ? AND user_id = ?",
(project_id, target)).fetchone()
if cur is None:
self._send_json(404, {"error": "member not found"})
return
if cur[0] == "owner":
owners = store.conn.execute(
"SELECT COUNT(*) FROM project_members"
" WHERE project_id = ? AND role = 'owner'",
(project_id,)).fetchone()[0]
if owners <= 1:
self._send_json(
409,
{"error": "cannot remove the only owner"})
return
self._remove_member_row(store, project_id, user_id, target)
self._send_json(200, {"project_id": project_id,
"user_id": target})
def _remove_member_row(self, store, project_id, actor_id, target_id):
"""删成员行并记审计:owner 走 AuthStore.remove_member
(内含 owner 校验与审计);非成员的全局管理员走直接删
+ 同格式审计(remove_member 的 owner 校验会挡住全局
管理员,故分流,与 _set_member_row 同款)。"""
if store.check_permission(actor_id, project_id,
"manage_project"):
store.remove_member(project_id, actor_id, target_id)
return
store.conn.execute(
"DELETE FROM project_members"
" WHERE project_id = ? AND user_id = ?",
(project_id, target_id))
store.conn.commit()
store.audit(actor_id, project_id, "remove_member", target_id)
# -- 项目子资源:记忆 / 知识库 / 成本 -------------------------------
def _project_exists(self, project_id):
conn = self._db()
try:
row = conn.execute(
"SELECT id FROM projects WHERE id = ?", (project_id,)
).fetchone()
return row is not None
finally:
conn.close()
def _check_project(self, user_id, project_id, action):
"""返回 None 表示通过;否则返回 (status, error) 先行响应。"""
if not self._project_exists(project_id):
return 404, "project not found"
with _auth_store(self.server.db_path) as store:
if not store.check_permission(user_id, project_id, action):
return 403, "forbidden"
return None
def _project_push_subscription_delete(self, user_id, method, project_id, parts):
# 从 _dispatch_project 平移的 push 订阅删除分支:
# 语句原样平移,仅缩进减一格;每条路径都已发响应。
if method != "DELETE":
self._send_json(404, {"error": "not found"})
return
err = self._check_project(user_id, project_id, "create_task")
if err:
self._send_json(err[0], {"error": err[1]})
return
subs = _push_list(self.server.db_path, project_id)
mine = [s for s in subs if s["id"] == parts[3]]
if not mine:
self._send_json(404, {"error": "subscription not found"})
return
# 只能删自己的订阅,项目 owner 可删任意(owner 角色检查)。
with _auth_store(self.server.db_path) as auth:
is_owner = auth.check_permission(
user_id, project_id, "manage_project")
if mine[0]["user_id"] != user_id and not is_owner:
self._send_json(403, {"error": "forbidden"})
return
deleted = _push_delete(self.server.db_path,
subscription_id=parts[3])
if deleted:
with _auth_store(self.server.db_path) as store:
store.audit(
user_id, project_id, "push_subscription_deleted",
"subscription_id=%s" % parts[3])
self._send_json(200, {"deleted": bool(deleted)})
return
def _project_to_dict(row):
return {
"id": row["id"],
"name": row["name"],
"owner_id": row["owner_id"],
"created_at": row["created_at"],
# 老库行缺 repo_url 列时 _row_get 给 None,不炸(列本身由
# migrate_projects_table 在 init_db 时补,老行读到也是 None)。
"repo_url": _row_get(row, "repo_url"),
# auto_pr 默认 0(关闭):老行缺列时 None 按 0 计,绝不因
# 迁移就把老项目默认改成自动推。
"auto_pr": _row_get(row, "auto_pr") or 0,
}
_ENV_BRANCH_RE = re.compile(r"^[A-Za-z0-9_.\-/]+$")
def _validate_env_branch(branch):
"""环境分支名校验:空/None 表示默认分支;非空时只允许
字母数字与 _ . - /,且不能以 - 开头、不能含 ..、不能以
.lock 结尾、长度不超过 200——git ref 名的保守子集,防
分支名被拼进 shell 或越出目录。返回规范化后的分支或 None,
非法时抛 ValueError(调用方转 400)。"""
if branch is None:
return None
if not isinstance(branch, str):
raise ValueError("branch must be a string")
branch = branch.strip()
if not branch:
return None
if len(branch) > 200 or not _ENV_BRANCH_RE.match(branch) \
or branch.startswith("-") or ".." in branch \
or branch.endswith(".lock"):
raise ValueError("branch is not a valid git branch name")
return branch
+194
View File
@@ -0,0 +1,194 @@
"""Handler 共享 helper:被多个 handler 分组与留守代码共用。
由 cloud/server.py 拆出,行为与原来逐字一致;server.py 以
重导出保留旧导入路径(见 server.py 的 handler_support 导入块)。
本模块不 import cloud.server(避免循环),不直连 sqlite3,
不 import psycopg;存储一律走 cloud.db / 各 Store。
"""
from cloud.auth import AuthStore
from cloud.coordinator import parse_parts as _coord_parse_parts
from cloud.peer_messages import ensure_followups_table as _peer_ensure_followups
from cloud.quota import default_quota_usd_micros as _quota_default
from cloud.peer_messages import row_get as _row_get
from contextlib import contextmanager
import json
import os
import time
def _env_int(name, default):
try:
value = int(os.environ.get(name, "") or default)
except (TypeError, ValueError):
return default
return value if value > 0 else default
def _ensure_followups_table(conn):
"""幂等建后续消息表:委托给 peer_messages(两边同一份 DDL,
server init_db、各 followup 接口与 worker 投递路径共用)。
老库第一次跑到这里时自动补表补列,不丢数据。DOUBLE
PRECISION 在 SQLite 与 Postgres 下都可用(SQLite 按 NUMERIC
亲和处理)。"""
_peer_ensure_followups(conn)
def _ensure_quotas_table(conn):
"""幂等建配额表:server init_db 与配额查询都可调它,老库
第一次跑到这里时自动补表,不丢数据。quota_usd_micros 为
NULL 表示不限额(与 0 区分:0 是一分钱也不能花);金额
单位与 cost_records 一致,都是 USD 微单位(1 美元 =
1_000_000),避免浮点。"""
conn.execute(
"CREATE TABLE IF NOT EXISTS user_quotas("
"user_id TEXT PRIMARY KEY, "
"quota_usd_micros BIGINT, "
"updated_at DOUBLE PRECISION)")
def _default_quota_usd_micros():
"""OAuth 新用户建号时写入的默认配额(微美元)。
无配额行 = 不限额,所以 OAuth 自动建号过去等于无限额入驻。
新号现在建号即写一行默认配额(老用户无配额行不受影响,
行为不变)。环境变量 CLOUD_DEFAULT_QUOTA_USD_MICROS 可
覆盖,默认 10_000_000($10/月);非法值按默认计。
解析收敛到 cloud/quota.py(别名保留)。
"""
return _quota_default()
def _apply_default_quota(conn, user_id):
"""给用户写默认配额行(已存在则不动,幂等)。
INSERT OR IGNORE 保证重复回调不覆盖管理员后来改过的额度。
返回 True 表示本次写了新行(调用方据此记 quota_default
审计)。P0 安全批后:OAuth 新建、绑定、已存在用户登录
三条分支都调它补行,不再只给新建用户补。
"""
_ensure_quotas_table(conn)
cur = conn.execute(
"INSERT OR IGNORE INTO user_quotas(user_id, quota_usd_micros,"
" updated_at) VALUES(?, ?, ?)",
(user_id, _default_quota_usd_micros(), time.time()))
try:
conn.commit()
except Exception:
pass
return cur.rowcount == 1
def _global_admin_ids():
"""真全局管理员清单:环境变量 CODEX_GLOBAL_ADMINS。
逗号分隔的用户 id 列表,空白项忽略。未配置或为空时返回
空集——此时 admin 端点一律 fail-closed,绝不回落到项目
角色推导(P0 安全批前“任一项目 maintainer 即全局管理”
的推导已被废掉,见 _is_global_admin)。
"""
raw = os.environ.get("CODEX_GLOBAL_ADMINS") or ""
return {part.strip() for part in raw.split(",") if part.strip()}
def _environment_to_dict(row):
return {
"id": row["id"],
"project_id": row["project_id"],
"name": row["name"],
"branch": _row_get(row, "branch"),
"executor": _row_get(row, "executor"),
"status": row["status"],
"error_text": _row_get(row, "error_text"),
"repo_url": _row_get(row, "repo_url"),
"created_by": row["created_by"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
}
@contextmanager
def _auth_store(db_path):
"""每请求一个 AuthStore 实例(自带新连接),用完即关闭。"""
store = AuthStore(db_path)
try:
yield store
finally:
try:
store.conn.close()
except Exception:
pass
def _task_to_dict(row, attachments=None, total_tokens=None):
depends_on = row["depends_on"]
try:
depends_on = json.loads(depends_on) if depends_on else []
except (TypeError, ValueError):
depends_on = []
if not isinstance(depends_on, list):
depends_on = [depends_on]
return {
"id": row["id"],
"project_id": row["project_id"],
"prompt": row["prompt"],
"status": row["status"],
"idempotency_key": row["idempotency_key"],
"depends_on": depends_on,
"priority": row["priority"],
"created_by": row["created_by"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
"last_heartbeat": row["last_heartbeat"],
"result_text": _row_get(row, "result_text"),
"result_full_available": bool(_row_get(row, "result_text_full")),
"error_text": _row_get(row, "error_text"),
"actual_model": _row_get(row, "actual_model"),
"started_at": _row_get(row, "started_at"),
"finished_at": _row_get(row, "finished_at"),
"child_pid": _row_get(row, "child_pid"),
"executor": _row_get(row, "executor"),
"pr_url": _row_get(row, "pr_url"),
"pr_number": _row_get(row, "pr_number"),
"goal": _row_get(row, "goal"),
"verify_command": _row_get(row, "verify_command"),
"goal_max_rounds": _row_get(row, "goal_max_rounds"),
"goal_mode": _row_get(row, "goal_mode") or 0,
# 运行中改 goal:goal_version 是已提交版本(服务端写库),
# applied_goal_version 是 host 循环已应用版本(镜像回写),
# 两者不等时说明信号在途或未到轮次边界,绝不能冒充已应用。
"goal_version": _row_get(row, "goal_version") or 0,
"applied_goal_version": _row_get(row, "applied_goal_version") or 0,
"attachments": list(attachments or []),
"resume_message": _row_get(row, "resume_message"),
"worktree_host": _row_get(row, "worktree_host"),
"claimed_by_worker": _row_get(row, "claimed_by_worker"),
"model": _row_get(row, "model"),
"effort": _row_get(row, "effort"),
"child_model": _row_get(row, "child_model"),
"child_effort": _row_get(row, "child_effort"),
"approval": _row_get(row, "approval"),
"plan": _row_get(row, "plan") or 0,
"forked_from": _row_get(row, "forked_from"),
"environment_id": _row_get(row, "environment_id"),
"turn_id": _row_get(row, "turn_id"),
"root_turn_id": _row_get(row, "root_turn_id"),
"coordinator_parts": (
_coord_parse_parts(_row_get(row, "coordinator_parts"))
if _row_get(row, "coordinator_parts") else None),
"parent_task_id": _row_get(row, "parent_task_id"),
"coordinator_part": _row_get(row, "coordinator_part"),
"goal_resume_pending": _row_get(row, "goal_resume_pending") or 0,
"paused_by_user": _row_get(row, "paused_by_user") or 0,
"interrupted_reason": _row_get(row, "interrupted_reason"),
"auto_resume_count": _row_get(row, "auto_resume_count") or 0,
"no_time_limit": _row_get(row, "no_time_limit") or 0,
# 列表接口由 _task_tokens_map 一次聚合传入;其它调用方
# 不传时为 None(前端显示占位),绝不把“未算”写成 0。
"total_tokens": total_tokens,
}
MAX_PROJECTS_PER_USER = _env_int("CODEX_MAX_PROJECTS_PER_USER", 50)
MAX_PROJECT_ID_LEN = _env_int("CODEX_MAX_PROJECT_ID_LEN", 64)
MAX_PROJECT_NAME_LEN = _env_int("CODEX_MAX_PROJECT_NAME_LEN", 100)
+6 -2
View File
@@ -1209,7 +1209,10 @@ def _anon_id(index: int) -> str:
def assign_ids(labels: Sequence[str], *, task_id: str, k: int, mode: str) -> dict[str, str]:
"""匿名 id -> 真实 label:按排序后定种子洗牌,同输入永远同结果。"""
seed = int(hashlib.sha1(f"{task_id}|{k}|{mode}".encode()).hexdigest()[:12], 16)
# 仅确定性洗牌种子,非安全用途(不防碰撞、不做完整性校验):
# usedforsecurity=False 后 bandit B324 不再误报,且摘要值不变。
seed = int(hashlib.sha1(f"{task_id}|{k}|{mode}".encode(),
usedforsecurity=False).hexdigest()[:12], 16)
ordered = sorted(labels)
random.Random(seed).shuffle(ordered)
return {_anon_id(i): lbl for i, lbl in enumerate(ordered)}
@@ -1284,7 +1287,8 @@ def select_final(
except Exception as exc:
# 回落:已交付的终版里排第一的那个;连这个都没有才算选失败。
sel.chosen_label = sorted(pool)[0]
sel.chosen_id = next(a for a, l in sel.id_map.items() if l == sel.chosen_label)
sel.chosen_id = next(a for a, lab in sel.id_map.items()
if lab == sel.chosen_label)
sel.fallback = True
sel.error = str(exc)[:400]
sel.ok = True
+29 -5
View File
@@ -73,8 +73,9 @@ import urllib.request
import uuid
from dataclasses import dataclass, field
from pathlib import Path
from typing import Protocol
from typing import Protocol, TypedDict
from cloud.timeouts import LONG_TIMEOUT_SECONDS as _LONG_TIMEOUT
from .harness2 import (
EditResult,
Rollout,
@@ -93,7 +94,8 @@ from .harness2 import (
TERMINAL_STATUSES = frozenset(
{"completed", "failed", "interrupted", "merged", "merge-failed"}
)
DEFAULT_TASK_TIMEOUT_SECONDS = 3600
# 超时数值收敛到 cloud/timeouts.py(别名保留,调用点不变)。
DEFAULT_TASK_TIMEOUT_SECONDS = _LONG_TIMEOUT
DEFAULT_POLL_SECONDS = 10
TRACE_MAX_CHARS = 50000
@@ -598,6 +600,12 @@ class CodexProbeAdapter:
def base_harness(self) -> Tree:
return dict(self.base)
def validate_tree(self, harness: Tree) -> None:
"""探针无项目特有不变式:no-op。harness2._adapter_validate
本就对缺失该方法的 adapter 跳过,故 no-op 与跳过行为一致,
补上只是让类满足 TaskAdapter 协议(mypy arg-type)。"""
return None
def execute(self, label: str, harness: Tree) -> Rollout:
files = dict(self.source_files)
files.update(_tree_to_files(harness))
@@ -820,6 +828,20 @@ class CodexReflector:
return parse_experience(state.result_text or "")
class _SharedRoleArgs(TypedDict):
"""editor/selector/reflector 三角色共用的构造参数:键与三者
dataclass 字段一一对应、逐键类型精确。dict 字面量会先塌成
dict[str, int | str | None] 再按位置报错(mypy arg-type),
TypedDict 展开时才能按字段逐个校验,运行时仍是普通 dict。"""
environment_id: str | None
executor: str | None
effort: str | None
approval: str | None
no_time_limit: bool
timeout_seconds: int
def run_improvement_via_tasks(
client: TaskClient,
*,
@@ -872,9 +894,11 @@ def run_improvement_via_tasks(
approval=approval,
no_time_limit=no_time_limit, timeout_seconds=timeout_seconds,
domain=domain)
shared = dict(environment_id=environment_id, executor=executor,
effort=effort, approval=approval, no_time_limit=no_time_limit,
timeout_seconds=timeout_seconds)
shared: _SharedRoleArgs = {
"environment_id": environment_id, "executor": executor,
"effort": effort, "approval": approval,
"no_time_limit": no_time_limit, "timeout_seconds": timeout_seconds,
}
editor = CodexEditor(client=client, project_id=project_id, run_tag=run_tag,
model=_role_model("editor"), **shared)
selector = CodexSelector(client=client, project_id=project_id, run_tag=run_tag,
+10 -2
View File
@@ -15,15 +15,23 @@ copying; keys are read from the MCP server process environment
by imagegen.py, never from the tool arguments.
"""
import importlib
import json
import os
import sys
# cloud 包优先、租户 flat copy 回落(与 playwright_mcp 同款理由:
# 两次 from-import 会让 mypy 报 no-redef,importlib 属性绑定
# 在两条路径下都是同一组名字;cloud 导入本身的 ImportError
# 同样回落,与原来 except 语义一致)。
try:
from cloud.imagegen import ImageGenError, edit, generate
_mcp_backend = importlib.import_module("cloud.imagegen")
except ImportError: # flat copy in a tenant codex-home
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from imagegen import ImageGenError, edit, generate # noqa: E402
_mcp_backend = importlib.import_module("imagegen")
ImageGenError = _mcp_backend.ImageGenError
edit = _mcp_backend.edit
generate = _mcp_backend.generate
SERVER_NAME = "imagegen"
SERVER_VERSION = "0.1.0"
+1468
View File
File diff suppressed because it is too large. Load diff
+190
View File
@@ -0,0 +1,190 @@
"""台账钩子开箱闭合方案(集成 v2,代码层)。
首轮缺口(成本与台账 8.5 扣 0.5):Worker(attempt_ledger_hook
=False) 默认关,CLI 须显式 --attempt-ledger-hook;生产 PG
上启用前须先手动执行台账 PG_DDL 与标记表 PG_MARKER_DDL
并验证。默认状态下指标率是 unknown(诚实),不是“已用
真实台账”——“接通”只在“开 flag + 手动跑 DDL + 验证”
之后成立,开箱未闭合。
本轨的闭合方案(不改任何已定语义):
1. 默认行为不变:钩子默认仍关。默认关是 #255 的设计
(显式启用才写行,关时行为与 gap 3 完全一致、无新表
无新行),不是 bug,不为“开箱即用”悄悄改成默认开。
把默认改成开,是用户/协调员在合轨时才能定的语义变
更,本轨不做。
2. 显式开启路径唯一且可验证:Worker(attempt_ledger_hook
=True) 或 CLI --attempt-ledger-hook;开启后 run_one
按 attempt_hook.py 规则 1-4 写行。本模块的
ledger_setup_report() 把“表是否齐、钩子是否开”分开
报告:表齐但钩子关 ≠ 就绪,钩子开但表不齐 ≠ 就绪。
3. DDL 清单不另造:清单直接引用 attempt_ledger.PG_DDL
与 attempt_hook.PG_MARKER_DDL 两个常量(本模块不
复制 DDL 文本,复制就会漂移)。另含 gap 3 旧表升级句
attempt_ledger.OLD_TABLE_ALTER(仅当旧 DDL 建过表时
需要;gap 3 未合并,生产尚无旧表,属防御项)。
4. 校验函数 verify_ledger_ready(conn):只读探针——查两
张表的存在与必需列,不建表、不写行(读路径绝不改
schema,与 metrics-report 端点的只读纪律一致)。
5. 生产 PG 手动 DDL 仍是运维侧动作:本轨无生产 PG、无
本地 PG,PG 证据与此前各轨同款——适配器级(? 占位符
经 cloud.db 翻译、DDL 仅用 PG 合法类型、无自增列),
绝不写成“生产 PG 已执行”。运维执行清单见
LEDGER_DDL_CHECKLIST(顺序执行 + 每步验证命令)。
零第三方依赖,只用标准库。conn 约定与 attempt_ledger
相同:DBAPI 连接,? 占位符(PG 经 cloud.db 翻译)。
"""
from cloud.attempt_hook import MARKER_DDL as _MARKER_DDL
from cloud.attempt_hook import PG_MARKER_DDL as _PG_MARKER_DDL
from cloud.attempt_ledger import OLD_TABLE_ALTER as _OLD_TABLE_ALTER
from cloud.attempt_ledger import PG_DDL as _LEDGER_PG_DDL
# 必需列:缺任一列即视为表不齐(旧 DDL 建的表缺
# corrects_attempt_no 时走 OLD_TABLE_ALTER 补列)。
LEDGER_REQUIRED_COLUMNS = (
"task_id", "attempt_no", "outcome", "is_defect",
"defect_category", "created_at", "corrects_attempt_no",
)
MARKER_REQUIRED_COLUMNS = ("task_id", "origin", "queued_at")
# 运维执行清单(顺序):每步含“做什么 + 怎么验证”。本
# 清单是文档数据,不是执行器——生产 PG 上的执行仍由
# 运维手工完成,本模块不代执行、不连生产库。
LEDGER_DDL_CHECKLIST = (
{
"step": 1,
"action": "在生产 PG 上执行 attempt_ledger.PG_DDL"
"(建 task_attempts 表)",
"ddl_source": "cloud.attempt_ledger.PG_DDL",
"verify": "\\d task_attempts 或 information_schema"
" 查表与列齐全",
},
{
"step": 2,
"action": "在生产 PG 上执行 attempt_hook.PG_MARKER_DDL"
"(建 task_attempt_origin_markers 表)",
"ddl_source": "cloud.attempt_hook.PG_MARKER_DDL",
"verify": "\\d task_attempt_origin_markers 或"
" information_schema 查表与列齐全",
},
{
"step": 3,
"action": "仅当表由 gap 3 旧 DDL 建过时:执行"
" attempt_ledger.OLD_TABLE_ALTER 补"
" corrects_attempt_no 列(gap 3 未合并,"
"生产正常路径跳过此步)",
"ddl_source": "cloud.attempt_ledger.OLD_TABLE_ALTER",
"verify": "task_attempts 含 corrects_attempt_no 列",
},
{
"step": 4,
"action": "启用钩子:worker 加 --attempt-ledger-hook"
"(或 Worker(attempt_ledger_hook=True))后重启",
"ddl_source": None,
"verify": "新派发产生 task_attempts 行;"
"平台重派发不产生计数行(marker 生效)",
},
{
"step": 5,
"action": "用 verify_ledger_ready(conn) 复核:"
"tables_ready 且 hook_enabled 方为就绪",
"ddl_source": None,
"verify": "ledger_setup_report(conn, hook_enabled=True)"
" 的 ready 为 True",
},
)
def ledger_ddl_sources():
"""DDL 清单的权威来源(返回常量本身,供运维原样执行
与单测钉住“清单与实现同一来源”,防止清单漂移)。"""
return {
"task_attempts_pg_ddl": _LEDGER_PG_DDL,
"marker_pg_ddl": _PG_MARKER_DDL,
"old_table_alter": _OLD_TABLE_ALTER,
"marker_ddl": _MARKER_DDL,
}
def _table_columns(conn, table):
"""只读探针:表不存在返回 None;存在返回列名集合。
PRAGMA table_info 经 cloud.db 适配层翻译(与
CostStore._ensure_cached_column 同款做法);探针失败
(如 PG 适配差异)返回 None,按“不齐”处理,绝不
按“齐”冒充。
"""
try:
rows = conn.execute(
"PRAGMA table_info(%s)" % table).fetchall()
except Exception:
return None
if not rows:
return None
try:
return {row[1] for row in rows}
except (IndexError, TypeError, KeyError):
return None
def verify_ledger_ready(conn):
"""只读校验两张台账表:返回 {tables_ready, missing}。
missing 列出缺失项,如 "task_attempts table missing"、
"task_attempts missing columns: corrects_attempt_no"、
"task_attempt_origin_markers table missing"。本函数不
建表、不写行、不提交。
"""
missing = []
ledger_cols = _table_columns(conn, "task_attempts")
if ledger_cols is None:
missing.append("task_attempts table missing")
else:
absent = [c for c in LEDGER_REQUIRED_COLUMNS
if c not in ledger_cols]
if absent:
missing.append("task_attempts missing columns: %s"
% ", ".join(absent))
marker_cols = _table_columns(conn, "task_attempt_origin_markers")
if marker_cols is None:
missing.append("task_attempt_origin_markers table missing")
else:
absent = [c for c in MARKER_REQUIRED_COLUMNS
if c not in marker_cols]
if absent:
missing.append(
"task_attempt_origin_markers missing columns: %s"
% ", ".join(absent))
return {"tables_ready": not missing, "missing": missing}
def ledger_setup_report(conn, hook_enabled=False):
"""开箱就绪报告:表齐 + 钩子开,两者缺一都不算就绪。
返回 {ready, tables_ready, hook_enabled, missing,
next_steps}:next_steps 是运维下一步动作的文字清单
(按 LEDGER_DDL_CHECKLIST 顺序,未做项才列出),
不是自动执行。本函数同样只读,不建表、不写行。
"""
check = verify_ledger_ready(conn)
enabled = bool(hook_enabled)
steps = []
if check["missing"]:
steps.append("run the PG DDL steps in LEDGER_DDL_CHECKLIST"
" (steps 1-3) on production Postgres and verify")
if not enabled:
steps.append("enable the hook with --attempt-ledger-hook"
" (Worker(attempt_ledger_hook=True)) and restart")
if not steps:
steps.append("ledger is live: counted dispatches write rows;"
" rates are real, not unknown")
return {
"ready": bool(check["tables_ready"] and enabled),
"tables_ready": check["tables_ready"],
"hook_enabled": enabled,
"missing": check["missing"],
"next_steps": steps,
}
+178
View File
@@ -0,0 +1,178 @@
"""台账写失败的 durable 记录与重放(集成 v2,代码层)。
首轮缺口(成本与台账 8.5 扣 0.5):_record_ledger_attempt
写失败只记一条 worker_attempt_ledger_failed 日志,绝不
影响任务状态与返回值(attempt_hook.py 规则 7)。这是
设计(指标台账不应挡任务),但与 #241 审计账本的
fail-closed 不对称:一次写失败会让率悄悄少算一行,只
有日志可查,没有重放/补写路径。
本轨的对齐方式(写清边界,不悄悄改任一侧语义):
- 不把台账改成 fail-closed:台账写失败仍不挡任务、不
改任务状态、不改返回值(规则 7 原样保留)。让指标写
入能阻塞任务执行,是比丢一行更坏的 trade-off,
本轨不做。
- 不把审计改成 best-effort:审计账本 fail-closed 原样
保留(审计是决策级证据,丢一行即链不可信;台账是
指标,丢一行只影响率的分母/分子,且可事后补)。
- 不对称的剩余部分用“不丢”补齐:写失败时除日志外,
再向 worker 侧 outbox 文件追加一行 JSONL(task_id、
outcome、error、failed_at);运维事后调
replay_failures() 把 outbox 行经 record_attempt 补写
进台账(append-only,attempt_no 按 MAX+1 续号),
成功补写的行从 outbox 清除,仍失败的行保留待下次。
于是失败不再是“只有日志可查”,而是“可数、可重放、
重放后可验证行数补齐”。
诚实边界:
- outbox 本身也是 best-effort:outbox 追加失败(如
磁盘满)只记日志,不抛、不挡任务——此时仍退回首轮
状态(仅日志可查),但这条路径本身已被测试钉住
(追加失败不抛异常)。
- 重放无幂等键:record_attempt 没有去重键;若原始写
入实际已提交、只是提交确认丢失(ambiguous commit),
重放会多写一行。重放是运维手动动作:重放前须核对
task_attempts 是否已有该任务的对应行,确认缺失再
放行;本函数不假装能自动去重(去重键是新的 schema
语义,须另立一轨定)。
- outbox 路径:环境变量 CLOUD_LEDGER_FAILURE_OUTBOX
显式指定时优先;否则取 worker db_path 同目录下的
attempt_ledger_failures.jsonl;db_path 为 :memory:
时无落盘位置,返回 None(调用方只记日志)。
零第三方依赖,只用标准库。
"""
import json
import os
import time
OUTBOX_ENV = "CLOUD_LEDGER_FAILURE_OUTBOX"
OUTBOX_FILENAME = "attempt_ledger_failures.jsonl"
def outbox_path_for(db_path, env=None):
"""解析 outbox 文件路径:显式 env > db 同目录 > None。
env 缺省读 os.environ(调用方可注入 dict 做单测)。
db_path 为 :memory: 或空时无同目录可言,返回 None。
"""
if env is None:
env = os.environ
explicit = env.get(OUTBOX_ENV) if env else None
if explicit:
return str(explicit)
if not db_path or str(db_path) == ":memory:":
return None
directory = os.path.dirname(os.path.abspath(str(db_path)))
return os.path.join(directory, OUTBOX_FILENAME)
def append_failure(path, task_id, outcome, error):
"""向 outbox 追加一行失败记录。返回 True/False(写成功与否)。
追加失败返回 False,不抛异常(调用方只记日志)。
行格式:{task_id, outcome, error, failed_at},一行一
条 JSON,坏行在重放时跳过并保留(见 replay_failures)。
"""
if not path:
return False
record = {
"task_id": task_id,
"outcome": outcome,
"error": str(error),
"failed_at": time.time(),
}
try:
directory = os.path.dirname(os.path.abspath(str(path)))
if directory:
os.makedirs(directory, exist_ok=True)
with open(path, "a", encoding="utf-8") as handle:
handle.write(json.dumps(record, ensure_ascii=False) + "\n")
return True
except Exception:
return False
def read_failures(path):
"""读 outbox:返回 (records, bad_lines)。
records 为解析成功的 dict 列表(按文件顺序);bad_lines
为解析失败的行数(坏行不丢弃,重放时原样保留)。
文件不存在返回 ([], 0),不是错误。
"""
records = []
bad = 0
try:
with open(path, encoding="utf-8") as handle:
lines = handle.readlines()
except OSError:
return records, bad
for line in lines:
line = line.strip()
if not line:
continue
try:
item = json.loads(line)
except ValueError:
bad += 1
continue
if not isinstance(item, dict) or not item.get("task_id") \
or item.get("outcome") not in (
"success", "failure", "interrupted"):
bad += 1
continue
records.append(item)
return records, bad
def replay_failures(path, conn, record_fn=None):
"""重放 outbox:经 record_attempt 补写,返回统计。
返回 {replayed, failed, remaining, bad_lines}:
replayed 为成功补写行数,failed 为补写仍失败的行数,
remaining 为重放后 outbox 剩余行数(failed + bad 行),
bad_lines 为跳过未补写的坏行数。成功补写的行从
outbox 清除(文件重写为剩余行);全部成功时 outbox
被清空(文件保留为空文件,不是删除——删除会丢“曾
有失败”的痕迹,空文件 + 台账行即为补齐证据)。
record_fn 缺省为 attempt_ledger.record_attempt(调用
方可注入假函数做单测);record_fn 抛异常的行计入
failed 并保留。调用方负责提交(本函数不 commit,
与 record_attempt 的调用方提交约定一致)。
"""
if record_fn is None:
from cloud.attempt_ledger import record_attempt as record_fn
records, bad = read_failures(path)
remaining_records = []
replayed = 0
failed = 0
for item in records:
try:
record_fn(conn, item["task_id"], item["outcome"],
is_defect=False)
replayed += 1
except Exception:
failed += 1
remaining_records.append(item)
# 坏行无法解析出 task/outcome,不能补写也不能丢弃:
# 原样保留在 outbox 里(本函数读不到原文,故坏行数
# 只计入 remaining,不重写坏行文本——重写会丢原文)。
# 为避免重写丢坏行:仅当 bad == 0 时重写文件;有坏行
# 时不重写,只返回统计,由运维人工处理坏行后再重放。
remaining = failed + bad
if bad == 0:
try:
with open(path, "w", encoding="utf-8") as handle:
for item in remaining_records:
handle.write(json.dumps(
item, ensure_ascii=False) + "\n")
except OSError:
pass
return {
"replayed": replayed,
"failed": failed,
"remaining": remaining,
"bad_lines": bad,
}
+129
View File
@@ -0,0 +1,129 @@
"""Postgres 审计账本只追加护栏:唯一受控执行入口(A2,代码侧)。
本模块是 ``cloud/ledger.py`` 的 ``PG_GUARD_SQL`` 常量在 Postgres
上的唯一执行入口——二选一里选的是“显式运维命令”,不是迁移脚本:
- 函数入口:``apply_pg_guard(conn)``,逐条执行且仅执行
``PG_GUARD_SQL``(从常量引用,不另写、不复制、不拼接 SQL)。
入口本身不 commit:调用方在自己的事务边界提交(与
``ledger.ensure_schema`` 的约定一致)。
- 命令入口:``python3 -m cloud.ledger_pg_guard``(``main``),
连接参数只走 ``CLOUD_DB_*`` 环境变量(经
``cloud.db.PostgresDatabase``),不接受命令行密码参数。
受控含义(grep 可证,全仓库除本模块外无其它执行路径):
- ``ledger.ensure_schema`` 在 PG 适配连接上跳过触发器建;
``cloud/db.py`` 的 POSTGRES_SCHEMA 只建 audit_ledger 表,
不建触发器;迁移脚本不执行本常量。
- 入口先验连接类型:不是 ``PgConnection``(按类名判定,与
``ledger._is_pg_conn`` 同款)即拒绝执行,绝不把 PG 方言 SQL
发往 SQLite 连接。
- 无 PG 环境下入口不得 claim 已加固:``main`` 在
``CLOUD_DB_BACKEND`` 不是 postgres/postgresql 时直接以
exit 2 拒绝(不连接、不执行、不打印“已应用”);``--dry-run``
只打印将执行的常量语句,不连接、不执行。
诚实边界(A2 只做代码侧,真实 PG 验证待 B5):
- 本机无真实 PG,``PG_GUARD_SQL`` 的语法与 UPDATE/DELETE 阻挡
均未在真实 PG 上验证;B5 环境到齐后须在真实 PG 上跑本命令
并实测 UPDATE/DELETE 被拒绝,之后才可认“已加固”。在此之前,
任何“PG 已加固”的表述都是冒充。
- 本模块不打印、不返回任何密码或密钥:输出只有语句计数与
DDL 常量本身。
只用标准库 + cloud.db / cloud.ledger,不引第三方。
"""
import argparse
import json
import os
import sys
try:
from cloud.db import PostgresDatabase
from cloud.ledger import PG_GUARD_SQL
except ImportError: # 直接以脚本方式从仓库根外运行时的回退。
sys.path.insert(0, os.path.dirname(os.path.dirname(
os.path.abspath(__file__))))
from cloud.db import PostgresDatabase
from cloud.ledger import PG_GUARD_SQL
def apply_pg_guard(conn):
"""在给定 PG 适配连接上执行只追加护栏(唯一执行入口)。
conn 须是 PgConnection(按类名判定);其它连接一律拒绝,
绝不把 PG 方言发往 SQLite。按 PG_GUARD_SQL 原顺序逐条执行,
返回执行过的语句元组(即常量本身,供调用方/测试比对)。
不 commit:调用方提交。
"""
if type(conn).__name__ != "PgConnection":
raise TypeError(
"apply_pg_guard requires a PgConnection, got %s; "
"refusing to run Postgres guard SQL on a non-PG connection"
% type(conn).__name__)
for statement in PG_GUARD_SQL:
conn.execute(statement)
return tuple(PG_GUARD_SQL)
def main(argv=None):
"""运维命令:``python3 -m cloud.ledger_pg_guard [--dry-run]``。
返回进程退出码:0 已应用(或 dry-run 展示完毕),1 执行失败,
2 用法/环境错误(非 PG 后端、非 PG 连接——一律不 claim 已应用)。
"""
parser = argparse.ArgumentParser(
description="Apply the audit_ledger append-only guard "
"(PG_GUARD_SQL) to Postgres. Single controlled "
"entry point; see module docstring.")
parser.add_argument(
"--dry-run", action="store_true",
help="Print the statements that would run, without "
"connecting or executing anything.")
args = parser.parse_args(argv)
if args.dry_run:
print(json.dumps({"applied": False, "dry_run": True,
"statements": list(PG_GUARD_SQL)},
ensure_ascii=False, indent=2))
return 0
backend = (os.environ.get("CLOUD_DB_BACKEND") or "").strip().lower()
if backend not in ("postgres", "postgresql"):
print("refused: CLOUD_DB_BACKEND is not postgres "
"(got %r); nothing applied, nothing claimed"
% backend, file=sys.stderr)
return 2
try:
conn = PostgresDatabase().connect()
except Exception as exc:
print("failed to connect to Postgres: %s" % exc,
file=sys.stderr)
return 1
try:
if type(conn).__name__ != "PgConnection":
print("refused: connection is not a PgConnection "
"(got %s); nothing applied"
% type(conn).__name__, file=sys.stderr)
return 2
statements = apply_pg_guard(conn)
conn.commit()
except Exception as exc:
try:
conn.rollback()
except Exception:
pass
print("failed to apply PG guard: %s" % exc, file=sys.stderr)
return 1
finally:
try:
conn.close()
except Exception:
pass
print(json.dumps({"applied": True, "dry_run": False,
"statements": list(statements)},
ensure_ascii=False, indent=2))
return 0
if __name__ == "__main__":
raise SystemExit(main())
+7 -11
View File
@@ -31,6 +31,13 @@ import json
import time
import uuid
# 行取值统一走 peer_messages.row_get(A7):本模块旧私有版在
# 真实行类型(None / dict / sqlite3.Row)下与之结果完全一致
# (None 特判、dict .get、缺列回 None),调用处仍用 _row_get
# 别名,行为不变;旧版宽 Exception 兜底的病态行不在契约内,
# 见 peer_messages.row_get docstring。
from cloud.peer_messages import row_get as _row_get
SNAPSHOT_STATUS_LIMIT = 4000
SNAPSHOT_DIFF_LIMIT = 20000
REF_LIMIT = 2000
@@ -39,17 +46,6 @@ REF_KINDS = ("doc", "build", "external", "pr", "file")
BINDING_MODES = ("heartbeat", "cron")
def _row_get(row, name):
if row is None:
return None
if isinstance(row, dict):
return row.get(name)
try:
return row[name]
except Exception:
return None
# -- worktree 快照 ------------------------------------------------------
def ensure_snapshots_table(conn):
+30 -63
View File
@@ -125,17 +125,31 @@ docs/marker-tools.md 新增的两节。
同创建人-only 的目标一律拒绝,并把原因写进报告——绝不
在查不到权限时静默放宽。
本模块只依赖标准库,不 import cloud 其它模块,避免循环
本模块只依赖标准库与同为标准库叶子的 cloud.gates /
cloud.timeouts,不 import 其他 cloud 模块,避免循环
import。DB 助手函数只用 ? 占位符,与 peer_messages 同款写法。
"""
import json
import os
import re
import time
import uuid
from datetime import datetime, timedelta, timezone
from typing import Any
from cloud.gates import RETRYABLE_STATUSES as _GATE_RETRYABLE
from cloud.gates import TERMINAL_STATUSES as _GATE_TERMINAL
from cloud.peer_messages import row_get
from cloud.gates import decide_control as _gate_decide_control
from cloud.gates import is_goal_task as _gate_is_goal_task
from cloud.gates import resolve_inside as _gate_resolve_inside
from cloud.timeouts import LONG_TIMEOUT_SECONDS as _LONG_TIMEOUT
# ZoneInfo 在极老 Python 上不存在:先声明为 Any 再 try import,
# 这样 except 分支的 None 赋值不再是“给 type 赋值”(mypy
# misc + assignment),运行时仍是“有则类、无则 None”,
# 下游 `if ZoneInfo is not None` 守卫语义不变。
ZoneInfo: Any = None
try:
from zoneinfo import ZoneInfo
except Exception: # pragma: no cover - 极老 Python 无 zoneinfo
@@ -158,12 +172,13 @@ MAX_SPAWN_DEPTH = 3
MAX_DIRECT_CHILDREN = 20
DEFAULT_POLL_INTERVAL_SECONDS = 5
DEFAULT_POLL_TIMEOUT_SECONDS = 300
MAX_POLL_TIMEOUT_SECONDS = 3600
# 轮询上限与状态集收敛到 cloud/gates.py / cloud/timeouts.py
#(别名保留,调用点与测试 import 的名字不变)。
MAX_POLL_TIMEOUT_SECONDS = _LONG_TIMEOUT
DEFAULT_SNAPSHOT_CHARS = 2000
MAX_SNAPSHOT_CHARS = 8000
TERMINAL_STATUSES = ("completed", "failed", "interrupted", "merged",
"merge-failed")
RETRYABLE_STATUSES = ("failed", "interrupted", "merge-failed")
TERMINAL_STATUSES = _GATE_TERMINAL
RETRYABLE_STATUSES = _GATE_RETRYABLE
EFFORTS = ("low", "medium", "high", "xhigh", "max")
EXECUTORS = ("local", "docker", "cloud")
TODO_STATUSES = ("pending", "in_progress", "blocked", "completed",
@@ -271,14 +286,6 @@ def parse_first_run_at(value):
return epoch
def row_get(row, name):
"""与 peer_messages.row_get 同款容错取值。"""
try:
return row[name]
except (KeyError, IndexError, TypeError):
return None
def _as_int(value, default):
if isinstance(value, bool):
return default
@@ -613,21 +620,10 @@ def extract_task_files(text):
def safe_worktree_path(worktree, rel):
"""worktree 内相对路径的安全解析:与服务端
_safe_worktree_path 同款——绝对路径、.. 逃逸、符号链接
逃逸一律返回 None;空串返回 worktree 本身。"""
if not rel:
return worktree
if os.path.isabs(rel):
return None
base = os.path.realpath(worktree)
target = os.path.realpath(os.path.join(base, rel))
try:
if os.path.commonpath([base, target]) != base:
return None
except ValueError:
return None
return target
"""worktree 内相对路径的安全解析:判定收敛到 cloud/gates.py
(与服务端 _safe_worktree_path 同款;绝对路径、.. 逃逸、
符号链接逃逸一律返回 None;空串返回 worktree 本身)。"""
return _gate_resolve_inside(worktree, rel)
_SPAWN_SUBJECT_ZH = ("新建的任务", "派生任务", "新建任务",
@@ -1117,45 +1113,16 @@ def ancestry_relation(conn, sender, target):
def is_goal_task(task):
"""goal 任务判定:verify_command 非空或 goal_mode 为真。
与 worker run_one 的 goal 分支同款(verify_command 或
goal_mode);goal 字段本身是 goal_mode=1 时 prompt 的副本,
不单独当判定条件,避免把普通任务的 goal 文本误判。"""
if not task:
return False
return bool(task.get("verify_command") or task.get("goal_mode"))
"""goal 任务判定:薄壳,判定在 cloud/gates.py。"""
return _gate_is_goal_task(task)
def check_control_gate(target, sender, relation, kind):
"""task-cancel / task-retry 的纯函数门禁,返回 (ok, reason)。
kind 只能是 "cancel" 或 "retry",其它值一律拒绝。权限层
不在这里:worker 侧另查 create_task / approve_merge,见
模块头;本函数只管目标状态与关系,单测不需要 AuthStore。"""
if kind not in ("cancel", "retry"):
return False, "未知的控制操作"
if target is None:
return False, "目标任务不存在"
if (target.get("project_id") is not None
and sender.get("project_id") is not None
and target.get("project_id") != sender.get("project_id")):
return False, "跨项目任务不可操作"
if target.get("id") == sender.get("id"):
return False, "不能操作自己"
if is_goal_task(target):
return False, "goal 任务不可取消或重试(goal 循环自有轮次)"
status = target.get("status")
if kind == "cancel":
if status in TERMINAL_STATUSES:
return False, "目标已终态(%s),不可取消" % status
else:
if status not in RETRYABLE_STATUSES:
return False, ("仅 failed / interrupted / merge-failed"
" 可重试,当前状态为 %s" % status)
if relation not in ("descendant", "same_creator"):
return False, "目标不在发送方派生链上且创建人不同"
return True, ""
薄壳:判定在 cloud/gates.py(权限层仍在 worker 侧,
本函数只管目标状态与关系,单测不需要 AuthStore)。"""
return _gate_decide_control(target, sender, relation, kind)
def _heartbeat_age_text(snapshot, now=None):
+269
View File
@@ -0,0 +1,269 @@
"""MCDT 结果指标:验收节点与关键事实节点的评分卡输入模型。
起点事实(照实写清,不得淡化):
- 此前 v6 批头 b7ac9d7 全仓 grep MCDT / 关键事实节点 / 验收节点为
0 命中:此前代码里根本没有 MCDT。
- 本模块只是评分卡输入模型,不是产品功能。不得声称 MCDT 已是
产品功能、已接入生产评分、已跑通真实审计循环。
- 本模块不改任何生产行为:不碰 #238 auto_apply(保持 False)、
不碰 #247“纠正为 pass 本身不解封”、不改 v1.1 契约、不接线到
build_scorecard / 服务器端点 / 门禁。本模块的输出只是“每维的
得分输入”,调用方是否采用、如何采用,须另行拍板。
设计(v7 机制第 8 项,只做这一项):
- acceptance_node(验收节点):一项可验收的结果主张,至少含
id、title、acceptance_criteria(列表)、status、evidence_refs。
另含 dimension:该节点映射到哪一维(7 维 id 之一);无 dimension
或 dimension 非法时该节点不映射到任何维(记入未映射清单),
绝不靠猜测、按 id 前缀、按其它维借分。
- key_fact_node(关键事实节点):一项支撑/证伪验收的关键事实,
至少含 id、fact、status、source。另含 dimension:与验收节点同
规则;无 dimension 的关键事实不影响任何维。
- status 词汇(两类节点共用):pending / pass / fail / unknown。
pending = 尚未判定;unknown = 未知/未验证;fail = 已证伪或未达标;
pass = 已验收通过。其它任何值一律按 unknown 处理(fail-closed,
不把拼写错误读成通过)。
映射到哪几维(写清,不留含糊):
- delivery_quality(交付质量):验收节点证明交付结果达到验收标准。
- gate_audit(门禁与审计有效性):验收节点证明门禁真实拦截、
审计结论可复核。
- arch_contract(架构与契约一致性):验收节点证明契约冻结、
架构 / 目录 / 导入一致。
- 其余四维 test_coverage、security_ledger、cost_quota、ops_scale:
本项不预设节点。无对应验收节点时一律记 unknown,并在
missing_items 里说明;不得用过程指标(测试数、覆盖率)填补,
不得从其它维借分。注意 test_coverage 尤其如此:覆盖率数字
本身是过程指标,即使调用方手头有覆盖率,也不能因此把
test_coverage 从 unknown 改成 pass/fail——只有映射到该维的
验收节点状态变动才能改分。
映射规则(nodes_to_dimension_inputs):
(a) 某维无对应验收节点时,该维输入为 unknown(不得用过程指标填补)。
(b) 过程指标(test_count、coverage)只作为证据字段透传,绝不直接
进入得分输入:它们只出现在返回值的顶层 process_evidence 里,
绝不出现在任何一维的 score_input / status / basis 里。只改
test_count / coverage 而节点状态不变时,每维的得分输入不变;
只改节点状态(pending -> pass 或 pass -> fail)时,映射输出变化。
节点 dict 上附带的 test_count / coverage 等额外键同样被忽略,
只读 id / title / acceptance_criteria / status / evidence_refs /
dimension(验收节点)与 id / fact / status / source / dimension
(关键事实节点)。
(c) 节点状态为 fail 时对应维不得为 pass;节点状态 unknown 时对应维
为 unknown。具体聚合(fail-closed,优先级 fail > unknown >
pending > pass 的反向表述见代码):
- 任一映射验收节点 fail -> 该维 fail;
- 任一映射关键事实 fail -> 该维 fail(已证伪的关键事实不得
让该维通过,即使验收节点全 pass);
- 否则任一映射验收节点 unknown 或 pending -> 该维 unknown;
- 否则任一映射关键事实 unknown 或 pending -> 该维 unknown
(支撑事实未验证,不得宣称通过);
- 否则(映射验收节点全 pass,且映射关键事实全 pass 或无映射
关键事实)-> 该维 pass。
关键事实单独存在时不能让维通过:无验收节点时该维仍 unknown。
阈值口径(沿用 audit_scorecard.py):
- thresholds 参数只用于报告 thresholds_source,绝不改变任何一维
的状态:传阈值不能把 unknown 变成 pass。
- 默认 thresholds_source = "test_default_unapproved"(测试默认值、
未拍板);只有调用方显式传入且全部条目 source 均为
"user_approved" 时才记 "user_approved"。本模块绝不把测试默认
值写成生产已拍板;B / C 维与阈值本身不在本项决定范围。
"""
from __future__ import annotations
from dataclasses import dataclass, field
# 维顺序与 id 与 cloud/audit_scorecard.py 的 DIMENSIONS 保持一致;
# 若那边增维、减维、改名,本模块须同步改(测试钉住两者相等)。
DIMENSION_IDS = [
"delivery_quality",
"gate_audit",
"arch_contract",
"test_coverage",
"security_ledger",
"cost_quota",
"ops_scale",
]
# 本项有映射语义说明的三维(其余四维无预设节点,见模块 docstring)。
MAPPED_DIMENSIONS_DOC = {
"delivery_quality": "验收节点证明交付结果达到验收标准。",
"gate_audit": "验收节点证明门禁真实拦截、审计结论可复核。",
"arch_contract": "验收节点证明契约冻结、架构 / 目录 / 导入一致。",
}
VALID_STATUSES = ("pending", "pass", "fail", "unknown")
MODEL_NOTE = (
"MCDT input model only, not a product feature: "
"v6 head b7ac9d7 had 0 hits for MCDT / acceptance nodes / "
"key fact nodes; this module only maps node statuses to "
"per-dimension score inputs and does not wire into scoring, "
"gates, or production."
)
@dataclass
class AcceptanceNode:
"""验收节点。status 非法值在映射时按 unknown 处理。"""
id: str
title: str
acceptance_criteria: list = field(default_factory=list)
status: str = "unknown"
evidence_refs: list = field(default_factory=list)
dimension: str | None = None
@dataclass
class KeyFactNode:
"""关键事实节点。status 非法值在映射时按 unknown 处理。"""
id: str
fact: str
status: str = "unknown"
source: str = ""
dimension: str | None = None
def _as_dict(node):
"""接受 dataclass 实例或 dict,一律转成 dict 只读视图。"""
if isinstance(node, dict):
return node
if hasattr(node, "__dataclass_fields__"):
return {
f: getattr(node, f) for f in node.__dataclass_fields__ # noqa: B905
}
raise TypeError("node must be a dict or dataclass, got %r" % type(node))
def _norm_status(value):
"""非法状态 -> unknown(fail-closed,不把拼写错误读成通过)。"""
if value in VALID_STATUSES:
return value
return "unknown"
def _thresholds_source(thresholds):
"""与 audit_scorecard.py 同口径:只有全 user_approved 才算已拍板。"""
if not thresholds:
return "test_default_unapproved"
sources = set()
for entry in thresholds.values():
if isinstance(entry, dict):
sources.add(entry.get("source", "test_default_unapproved"))
else:
sources.add("test_default_unapproved")
if sources == {"user_approved"}:
return "user_approved"
return "test_default_unapproved"
def nodes_to_dimension_inputs(acceptance_nodes, key_fact_nodes,
thresholds=None, process_metrics=None):
"""把节点状态映射为每维的得分输入。
acceptance_nodes / key_fact_nodes:AcceptanceNode / KeyFactNode
实例或同字段 dict 的列表。process_metrics:过程指标 dict
(如 {"test_count": ..., "coverage": ...}),只透传到顶层
process_evidence,绝不进入任何一维的得分输入。
返回:{"dimensions": {维 id: 维输入}, "dimension_order": [...],
"process_evidence": {...}, "unmapped_acceptance_node_ids": [...],
"unmapped_key_fact_node_ids": [...],
"thresholds_source": ..., "model_note": ...}。
每维输入含 dimension / status / score_input / acceptance_node_ids /
key_fact_node_ids / missing_items / basis;score_input 里只有
节点状态计数,绝无 test_count / coverage 键。
"""
acceptance_nodes = list(acceptance_nodes or [])
key_fact_nodes = list(key_fact_nodes or [])
per_dim_acc = {dim: [] for dim in DIMENSION_IDS}
per_dim_facts = {dim: [] for dim in DIMENSION_IDS}
unmapped_acc, unmapped_facts = [], []
for raw in acceptance_nodes:
node = _as_dict(raw)
dim = node.get("dimension")
if dim in DIMENSION_IDS:
per_dim_acc[dim].append(node)
else:
unmapped_acc.append(node.get("id"))
for raw in key_fact_nodes:
node = _as_dict(raw)
dim = node.get("dimension")
if dim in DIMENSION_IDS:
per_dim_facts[dim].append(node)
else:
unmapped_facts.append(node.get("id"))
dimensions = {}
for dim in DIMENSION_IDS:
acc = per_dim_acc[dim]
facts = per_dim_facts[dim]
acc_statuses = [_norm_status(n.get("status")) for n in acc]
fact_statuses = [_norm_status(n.get("status")) for n in facts]
missing, basis = [], []
if not acc:
missing.append("no_acceptance_node_mapped")
if dim not in MAPPED_DIMENSIONS_DOC:
missing.append(
"no_preset_node_for_dimension:"
" process metrics must not fill this dimension")
basis.append("no mapped acceptance node: dimension unknown")
status = "unknown"
elif "fail" in acc_statuses:
status = "fail"
basis.append("mapped acceptance node fail: dimension not pass")
elif "fail" in fact_statuses:
status = "fail"
basis.append("mapped key fact fail: dimension not pass")
elif "unknown" in acc_statuses or "pending" in acc_statuses:
status = "unknown"
basis.append(
"mapped acceptance node not yet pass"
" (unknown/pending): dimension unknown")
elif "unknown" in fact_statuses or "pending" in fact_statuses:
status = "unknown"
basis.append(
"mapped key fact not yet verified"
" (unknown/pending): dimension unknown")
else:
status = "pass"
basis.append(
"all mapped acceptance nodes pass and mapped key facts"
" pass or absent")
score_input = {
"node_status": status,
"acceptance_pass_count": acc_statuses.count("pass"),
"acceptance_fail_count": acc_statuses.count("fail"),
"acceptance_pending_count": acc_statuses.count("pending"),
"acceptance_unknown_count": acc_statuses.count("unknown"),
"key_fact_pass_count": fact_statuses.count("pass"),
"key_fact_fail_count": fact_statuses.count("fail"),
"key_fact_pending_count": fact_statuses.count("pending"),
"key_fact_unknown_count": fact_statuses.count("unknown"),
}
dimensions[dim] = {
"dimension": dim,
"status": status,
"score_input": score_input,
"acceptance_node_ids": [n.get("id") for n in acc],
"key_fact_node_ids": [n.get("id") for n in facts],
"missing_items": missing,
"basis": basis,
}
return {
"dimensions": dimensions,
"dimension_order": list(DIMENSION_IDS),
# 过程指标只在这里透传:顶层证据字段,不是得分输入。
"process_evidence": dict(process_metrics) if process_metrics else {},
"unmapped_acceptance_node_ids": unmapped_acc,
"unmapped_key_fact_node_ids": unmapped_facts,
"thresholds_source": _thresholds_source(thresholds),
"model_note": MODEL_NOTE,
}
+7 -3
View File
@@ -83,7 +83,9 @@ TOOLS = [
_tool("get_task_files", "列/读任务 worktree 里的文件;不传 path 列顶层,传相对路径读文件或列目录。",
{"task_id": {"type": "string"},
"path": {"type": "string"}}, ["task_id"]),
_tool("get_task_tree", "取任务 worktree 的结构化目录树(只含名、类型、大小,不读文件内容);不传 path 列 worktree 根,传相对路径列该子目录;max_depth 1..5 缺省 3,max_entries 1..500 缺省 200,截断时 truncated=True。",
_tool("get_task_tree", "取任务 worktree 的结构化目录树(只含名、类型、大小,"
"不读文件内容);不传 path 列 worktree 根,"
"传相对路径列该子目录;max_depth 1..5 缺省 3,max_entries 1..500 缺省 200,截断时 truncated=True。",
{"task_id": {"type": "string"},
"path": {"type": "string"},
"max_depth": {"type": "integer"},
@@ -98,7 +100,8 @@ TOOLS = [
{"task_id": {"type": "string"}}, ["task_id"]),
_tool("goal_resume_task", "断点续跑一个 goal 任务(服务端状态机校验)。",
{"task_id": {"type": "string"}}, ["task_id"]),
_tool("create_followup", "给进行中任务发后续消息;mode=queued 排队进队尾不打断本轮,direct_send 立即送达且本轮结束后优先开下一轮。",
_tool("create_followup", "给进行中任务发后续消息;"
"mode=queued 排队进队尾不打断本轮,direct_send 立即送达且本轮结束后优先开下一轮。",
{"task_id": {"type": "string"},
"body": {"type": "string"},
"mode": {"type": "string"}}, ["task_id", "body"]),
@@ -112,7 +115,8 @@ TOOLS = [
{"project_id": {"type": "string"}}, ["project_id"]),
_tool("get_models", "取模型一览(价目与默认模型,服务端口径)。",
{}, []),
_tool("list_project_memories", "列出某项目的项目级记忆(此前任务沉淀的经验/约定,新任务会自动注入;top_k 可选,默认 20)。",
_tool("list_project_memories", "列出某项目的项目级记忆(此前任务沉淀的经验/约定,"
"新任务会自动注入;top_k 可选,默认 20)。",
{"project_id": {"type": "string"},
"top_k": {"type": "integer"}}, ["project_id"]),
_tool("create_project_memory", "给某项目记一条项目级记忆(kind 与 content 必填)。",
+190
View File
@@ -0,0 +1,190 @@
"""合并门禁:主分支保护 + 目录隔离(纯函数,无网络、无 GitHub 调用)。
A1 声明(服务端唯一门禁是 hard,不要拿本模块的模式推断服务端):
- 自 A1 起,服务端 POST /tasks/{id}/merge 一律按
cloud/merge_gate_hard 的 hard 求值;环境变量
CLOUD_MERGE_GATE_MODE 与请求体 merge_gate_mode 在服务端一律被
忽略,旧值 record/enforce/off 不再是服务端可选项。
- 本模块以下写到的 record/enforce/off 只是 #209 纯函数的历史语义:
它们只描述 evaluate_merge_gate / gate_from_request 这两个纯函数
本身,不再接线到任何服务端端点;纯函数单测保留这些语义,不代表
服务端会按此放行。需要服务端行为时只看 merge_gate_hard。
背景与诚实边界(先读,再用):
- 本模块**不能**替仓库开通主分支保护。保护是 GitHub 仓库设置,
只能由仓库管理员在 GitHub 侧开通;GitHub App id 5476837 的权限
也不会自动变化,新权限需安装方重新接受。本模块只能在调用方
明确告诉它保护状态时做门禁,绝不声称“已保护”。
- 保护状态的唯一来源是调用方传入的值或环境变量
CLOUD_BRANCH_PROTECTION(protected / unprotected,未设即
unknown)。unknown 不是“通过”,是“未知”:门禁把它记为风险,
但不因此阻断——因为本服务没有可信渠道去 GitHub 核实,拿未知
当未保护去硬堵,会把当前所有合并(当前仓库本身未保护)全停掉。
想把门禁武装起来,运维要显式设置该环境变量为 unprotected,
再由调用方在每次合并时传确认,或把模式切到 enforce。
- 目录隔离复用 v0.3 技能的检查口径(见
cloud/skills/software-engineering/SKILL.md“目录与分支规则”):
人 / 宿主跑 ``git diff --name-only $(git merge-base origin/main
HEAD)...HEAD``,逐项核对每个文件是否落在拥有目录内,契约变更
经协调员批准的文件除外。本模块不另造一套口径:
``check_directory_isolation`` 就是这条规则的可执行版,
``directory_diff_command`` 返回的正是 v0.3 文本里的命令。
v0.3 之前服务端合并端(POST /tasks/{id}/merge)只改任务状态,
不做 git 合并、不做目录检查、不做保护检查——那是本模块要补的
缺口,不是重复造检查。
模式(以下仅为本模块纯函数的历史语义,不是服务端行为;服务端
唯一门禁是 hard,见模块头 A1 声明):
- record(纯函数默认):未保护时必须显式确认(explicit_confirm)才放行;
无确认时阻断,错误码 branch_protection_risk_unconfirmed。
确认只覆盖“保护风险”,绝不覆盖目录隔离违规——越目录的写
只能靠 approved_exceptions(契约变更批准)放行,不能靠确认蒙混。
- enforce:未保护时一律阻断,确认也不放行。切到 enforce 之前
必须先由用户决定并开通保护:升级 Pro、公开仓库、或保持现状
纪律 + 记风险——三者都是用户侧决定,代码里只留开关,不替选。
- off:门禁显式关闭,只记风险不阻断。off 必须显式设置,
非法模式值不会悄悄落到 off,而是回落到 record(fail-safe)。
"""
import os
MODES = ("record", "enforce", "off")
PROTECTION_STATES = ("protected", "unprotected", "unknown")
RISK_UNPROTECTED = "branch protection is not enabled for this repo"
RISK_UNKNOWN = "branch protection status is unknown (not declared)"
RISK_GATE_OFF = "merge gate is disabled (mode=off)"
def gate_mode_from_env(env=None):
"""纯函数模式解析(非服务端行为,服务端不读它)。
环境变量里的模式;未设或非法时回落 record,绝不回落 off。"""
raw = (env if env is not None else os.environ).get(
"CLOUD_MERGE_GATE_MODE", "")
mode = (raw or "").strip().lower()
return mode if mode in MODES else "record"
def protection_from_env(env=None):
"""环境变量里的保护状态;未设或非法时为 unknown,不是 protected。"""
raw = (env if env is not None else os.environ).get(
"CLOUD_BRANCH_PROTECTION", "")
state = (raw or "").strip().lower()
return state if state in PROTECTION_STATES else "unknown"
def directory_diff_command(base="origin/main", head="HEAD"):
"""v0.3 技能文本里的目录检查命令(原样口径,不另造)。"""
return "git diff --name-only $(git merge-base %s %s)...%s" % (
base, head, head)
def _normalize_path(path):
return (path or "").strip().replace("\\", "/")
def _is_escape(path):
parts = [p for p in _normalize_path(path).split("/") if p not in ("", ".")]
return (not parts) or parts[0] == ".." or ".." in parts
def check_directory_isolation(files_changed, owned_dir,
approved_exceptions=()):
"""v0.3 目录隔离规则的可执行版。
files_changed:diff 输出的文件清单(相对仓库根)。
owned_dir:本线程唯一拥有目录,例如 apps/web;为 None/空时
无法核对,返回 inconclusive(不算通过,也不伪装成通过)。
approved_exceptions:经协调员批准的契约变更文件(精确路径),
命中者不算违规,但会记入 exceptions_used 以便合并记录核对。
返回 dict:passed / violations / exceptions_used / inconclusive。
"""
owned = _normalize_path(owned_dir)
exceptions = {_normalize_path(p) for p in approved_exceptions or ()}
if not owned:
return {"passed": False, "violations": [],
"exceptions_used": [], "inconclusive": True,
"reason": "owned_dir_unknown"}
prefix = owned.rstrip("/") + "/"
violations = []
used = []
for raw in files_changed or []:
path = _normalize_path(raw)
if not path:
continue
if path in exceptions:
used.append(path)
continue
if path.startswith("/") or _is_escape(path) \
or not path.startswith(prefix):
violations.append(path)
return {"passed": not violations, "violations": violations,
"exceptions_used": used, "inconclusive": False,
"reason": ""}
def evaluate_merge_gate(files_changed=None, owned_dir=None,
approved_exceptions=(), protection="unknown",
mode="record", explicit_confirm=False):
"""合并门禁纯函数(非服务端门禁;服务端唯一门禁是 hard)。
files_changed=None 表示调用方没给 diff 证据:目录检查跳过
(directory=None),不因“没证据”就判违规,也不伪装成已检查。
protection="unknown" 同理:记风险、不阻断。
返回 dict:allowed / mode / protection / directory / risks / blocks。
blocks 为空即 allowed;每个 block 都是可查的错误码。
"""
mode = mode if mode in MODES else "record"
protection = protection if protection in PROTECTION_STATES \
else "unknown"
risks = []
blocks = []
directory = None
if files_changed is not None:
directory = check_directory_isolation(
files_changed, owned_dir, approved_exceptions)
if directory["inconclusive"]:
blocks.append("directory_owned_dir_unknown")
elif not directory["passed"]:
blocks.append("directory_isolation_violation")
if protection == "unprotected":
risks.append(RISK_UNPROTECTED)
if mode == "enforce":
blocks.append("branch_protection_enforce")
elif mode == "record" and not explicit_confirm:
blocks.append("branch_protection_risk_unconfirmed")
elif protection == "unknown":
risks.append(RISK_UNKNOWN)
if mode == "off" and protection == "unprotected":
risks.append(RISK_GATE_OFF)
return {"allowed": not blocks, "mode": mode,
"protection": protection, "directory": directory,
"risks": risks, "blocks": blocks}
def gate_from_request(body, env=None):
"""从请求体 + 环境变量拼出门禁输入并求值(纯函数,非服务端路径。
服务端 POST /tasks/{id}/merge 不走本函数,一律按 hard 求值。
请求体字段(全是可选的;一个都不给时行为与旧接口一致,
只在响应里附带门禁结果,不阻断):
files_changed(str 列表)、owned_dir、approved_exceptions、
branch_protection、merge_gate_confirm(bool)。
模式(纯函数内)优先取环境变量,环境变量未设时才取请求体
merge_gate_mode——纯函数内请求体不能把环境变量定的 enforce
降级为 record;服务端则根本不读模式,一律 hard。
"""
body = body or {}
env_map = env if env is not None else os.environ
mode = gate_mode_from_env(env_map) \
if env_map.get("CLOUD_MERGE_GATE_MODE") \
else (body.get("merge_gate_mode") or "record")
protection = body.get("branch_protection") \
or protection_from_env(env_map)
return evaluate_merge_gate(
files_changed=body.get("files_changed"),
owned_dir=body.get("owned_dir"),
approved_exceptions=body.get("approved_exceptions") or (),
protection=protection, mode=mode,
explicit_confirm=bool(body.get("merge_gate_confirm")))
+250
View File
@@ -0,0 +1,250 @@
"""服务端合并硬门禁:五项全查、缺一 409、fail-closed(纯函数)。
本模块堆叠在 #209(cloud/merge_gate.py)之上,不改 #209 纯函数
本身的语义,但服务端不再经由它决定模式:
- #209 的 ``evaluate_merge_gate`` / ``gate_from_request`` 只是
纯函数(record 默认、无证据不阻断、unknown 只记风险不阻断),
自 A1 起不再接线到任何服务端端点;它们的 record/enforce/off
语义是纯函数的历史语义,不是服务端行为,不得据此推断服务端
会放行(见 cloud/merge_gate.py 头部的 A1 声明)。
- 服务端唯一门禁就是本模块的 hard:POST /tasks/{id}/merge
一律按 hard 求值;环境变量 CLOUD_MERGE_GATE_MODE 与请求体
``merge_gate_mode`` 在服务端一律被忽略——旧值 record/enforce/off
不再是可选项,设了也等同 hard,不存在“显式降级”的服务端路径。
(此前 hard_mode_from_env 曾把显式 record/enforce/off 原样沿用,
A1 已删除该解析与 HARD_MODES:删模式,不是改默认值。)
模式(服务端唯一:hard):
硬门禁五项(全部是调用方在 /merge 请求体里提交的证据;
本服务没有可信渠道直连 GitHub 核实,所以门禁强制的是“证据齐全
且全绿”,不是“服务端已向 GitHub 求证”——这条诚实边界必须写清,
调用方谎报证据不在本模块的验证范围内,由审计(第 3 轨)与
PR/Checks 的只读核对去覆盖):
① CI 全绿:``ci_status`` 必须为 ``"success"``,且 ``ci_checks``
非空、每项 status 均为 ``"success"``,且 ``required_checks``
非空、每项都在 ``ci_checks`` 里且为 success。空
``required_checks``(``[]``)= 未配置,按分支保护 payload
的同一口径处理(见 docs/merge-gate-branch-protection.md),
不是“没有必须项所以通过”。
② 目录隔离:复用 #209 的 ``check_directory_isolation``(v0.3 口径:
merge-base 范围 ``git diff --name-only``,拥有目录前缀检查,
含 ``apps/web2`` 非子目录边界与 ``../``、绝对路径逃逸)。
``files_changed`` 缺失 = 无证据,直接阻断(#209 里是跳过,
硬模式里是阻断,这是本轨改动的核心之一);越目录只能靠
``approved_exceptions``(协调员批准的契约变更文件)放行,
显式确认绝不覆盖目录违规。
③ 合同版本:``contract_version`` 非空,且 ``contract_frozen``
为 true;若同时给了 ``contract_expected_version``(冻结版),
两者必须相等,否则阻断。服务端没有独立的合同版本存储,
冻结版由调用方(协调员侧)声明, mismatch 时阻断。
④ 协调员批准:``coordinator_approved`` 必须为 true;缺失与
显式 false 用不同的 block 码区分(缺证据 vs 已否决)。
⑤ 无未解决评审意见:``unresolved_review_comments`` 必须为 0
(整数)或空列表;缺失 = 无证据,阻断;大于 0 / 非空 = 阻断。
分支保护(硬模式下的第六条隐含项,来自 #209 的保护定义):
``branch_protection`` 必须为 ``"protected"``。``"unprotected"``
即使带 ``merge_gate_confirm`` 也阻断——硬模式没有“确认即放行”
的逃生口(旧 record 模式下确认可覆盖保护风险的语义已随 A1
删模式一并失效,服务端不再有确认放行的路径)。``"unknown"``
(未声明或非法值)同样阻断:unknown 不等于 protected,这是
#209 里“unknown 只记风险”的过渡语义在硬模式下的收紧。
与第 3 轨(独立审计 + 自动推回)的接口约定(本模块不造推回状态机):
- 阻塞依据 = 409 响应里的 ``merge_gate`` 对象:``mode``、
``blocks``(稳定的机器可读码,下面的 _BLOCK_DESCRIPTIONS 有
逐条中文说明)、以及 ``ci`` / ``directory`` / ``contract`` /
``coordinator`` / ``reviews`` 五个分项。推回包逐条取 blocks,
不要解析审计 detail 文本。
- 审计依据 = 每次门禁阻断写一条 ``merge_gate_blocked`` 审计
(action 名固定,detail 为 ``"<task_id> mode=<mode>
blocks=<逗号分隔的 blocks>"``),任务状态不变(UPDATE 不执行)。
- 门禁结论是单次调用的判定,不持久化为任务状态:一次 200 不
代表之后另一次合并自动通过;修复后必须重新提交完整证据、
重新走全部五项(完整重审,非只审改动点)。
- 本模块不建推回任务、不发通知、不改任务状态机:推回执行、
轮次上限、升级路径全部归第 3 轨。
"""
from cloud.merge_gate import (
PROTECTION_STATES,
RISK_UNKNOWN,
RISK_UNPROTECTED,
check_directory_isolation,
)
_BLOCK_DESCRIPTIONS = {
"ci_evidence_missing": "CI 证据缺失(ci_status / ci_checks / required_checks 有一项未给)",
"ci_not_green": "CI 状态不是 success",
"ci_check_failed": "CI 检查项中有非 success 项",
"ci_required_checks_unconfigured": "required_checks 为空 = 未配置",
"ci_required_check_missing": "required_checks 中有项不在 CI 检查里或不是 success",
"directory_evidence_missing": "目录证据缺失(files_changed 未给)",
"directory_owned_dir_unknown": "拥有目录未知,无法核对目录隔离",
"directory_isolation_violation": "目录隔离违规(越拥有目录)",
"contract_version_missing": "合同版本缺失",
"contract_not_frozen": "合同未冻结(contract_frozen 不是 true)",
"contract_version_mismatch": "合同版本与冻结版不一致",
"coordinator_approval_missing": "协调员批准证据缺失",
"coordinator_not_approved": "协调员未批准",
"review_evidence_missing": "评审意见证据缺失",
"unresolved_review_comments": "存在未解决评审意见",
"branch_protection_unknown": "分支保护状态未知(unknown 不等于 protected)",
"branch_protection_unprotected": "分支未保护(硬模式下显式确认也不放行)",
}
def _ci_checks_index(ci_checks):
"""ci_checks 列表 -> {name: status};条目不合规时返回 None。"""
index = {}
for entry in ci_checks:
if not isinstance(entry, dict):
return None
name = entry.get("name")
status = entry.get("status")
if not isinstance(name, str) or not name \
or not isinstance(status, str):
return None
index[name] = status
return index
def evaluate_hard_merge_gate(files_changed=None, owned_dir=None,
approved_exceptions=(), protection="unknown",
ci_status=None, ci_checks=None,
required_checks=None, contract_version=None,
contract_frozen=False,
contract_expected_version=None,
coordinator_approved=None,
unresolved_review_comments=None):
"""硬门禁纯函数:五项 + 保护,缺一即 allowed=False。
files_changed=None、ci_checks=None、required_checks=None、
unresolved_review_comments=None 全部是“无证据”,不是“跳过”。
protection 非法值归一化为 unknown,随后按 unknown 阻断。
返回 dict:allowed / mode / protection / directory / ci /
contract / coordinator / reviews / risks / blocks。
"""
protection = protection if protection in PROTECTION_STATES \
else "unknown"
risks = []
blocks = []
# ① CI 全绿。
if ci_status is None or ci_status == "" \
or ci_checks is None or required_checks is None:
blocks.append("ci_evidence_missing")
else:
if ci_status != "success":
blocks.append("ci_not_green")
if not isinstance(ci_checks, list) or not ci_checks:
blocks.append("ci_evidence_missing")
index = None
else:
index = _ci_checks_index(ci_checks)
if index is None:
blocks.append("ci_check_failed")
elif any(status != "success" for status in index.values()):
blocks.append("ci_check_failed")
if isinstance(required_checks, list) and not required_checks:
blocks.append("ci_required_checks_unconfigured")
elif isinstance(required_checks, list) and index is not None:
missing = [name for name in required_checks
if index.get(name) != "success"]
if missing:
blocks.append("ci_required_check_missing")
elif not isinstance(required_checks, list):
blocks.append("ci_evidence_missing")
ci = {"status": ci_status, "checks": ci_checks,
"required_checks": required_checks,
"all_green": False} # 末尾按 CI 分项 blocks 重算,见下。
# ② 目录隔离(复用 #209,不另造口径)。
directory = None
if files_changed is None:
blocks.append("directory_evidence_missing")
else:
directory = check_directory_isolation(
files_changed, owned_dir, approved_exceptions)
if directory["inconclusive"]:
blocks.append("directory_owned_dir_unknown")
elif not directory["passed"]:
blocks.append("directory_isolation_violation")
# ③ 合同版本。
if contract_version is None or (
isinstance(contract_version, str)
and not contract_version.strip()):
blocks.append("contract_version_missing")
if contract_frozen is not True:
blocks.append("contract_not_frozen")
if contract_expected_version is not None \
and contract_version is not None \
and contract_version != contract_expected_version:
blocks.append("contract_version_mismatch")
contract = {"version": contract_version, "frozen": contract_frozen,
"expected": contract_expected_version}
# ④ 协调员批准。
if coordinator_approved is None:
blocks.append("coordinator_approval_missing")
elif coordinator_approved is not True:
blocks.append("coordinator_not_approved")
coordinator = {"approved": coordinator_approved}
# ⑤ 无未解决评审意见。
if unresolved_review_comments is None:
blocks.append("review_evidence_missing")
reviews = {"unresolved": None}
elif isinstance(unresolved_review_comments, bool):
blocks.append("review_evidence_missing")
reviews = {"unresolved": None}
elif isinstance(unresolved_review_comments, list):
reviews = {"unresolved": unresolved_review_comments}
if unresolved_review_comments:
blocks.append("unresolved_review_comments")
elif isinstance(unresolved_review_comments, int):
reviews = {"unresolved": unresolved_review_comments}
if unresolved_review_comments > 0:
blocks.append("unresolved_review_comments")
elif unresolved_review_comments < 0:
blocks.append("review_evidence_missing")
else:
blocks.append("review_evidence_missing")
reviews = {"unresolved": None}
# 分支保护:硬模式下 unknown 与 unprotected 一律阻断,确认无效。
if protection == "unprotected":
risks.append(RISK_UNPROTECTED)
blocks.append("branch_protection_unprotected")
elif protection == "unknown":
risks.append(RISK_UNKNOWN)
blocks.append("branch_protection_unknown")
# ci.all_green 只反映 CI 分项本身,不被其它项的 blocks 污染。
ci_block_codes = ("ci_evidence_missing", "ci_not_green",
"ci_check_failed", "ci_required_checks_unconfigured",
"ci_required_check_missing")
ci["all_green"] = not any(code in blocks for code in ci_block_codes)
return {"allowed": not blocks, "mode": "hard",
"protection": protection, "directory": directory,
"ci": ci, "contract": contract,
"coordinator": coordinator, "reviews": reviews,
"risks": risks, "blocks": blocks}
def block_descriptions(blocks):
"""blocks 码 -> 中文说明(供第 3 轨推回包逐条引用)。"""
return [_BLOCK_DESCRIPTIONS.get(code, code) for code in blocks or []]
def blocked_audit_detail(task_id, gate):
"""merge_gate_blocked 审计的 detail:只含任务 id、模式与 blocks 码。
推回包不要解析这段文本,取 409 响应的 merge_gate 对象为准。
"""
return "%s mode=%s blocks=%s" % (
task_id, gate.get("mode", "hard"),
",".join(gate.get("blocks") or []))
+263
View File
@@ -0,0 +1,263 @@
"""Metrics collection and feedback loop for agent-task quality.
Zero third-party dependencies; standard library only. This module
imports no ``cloud.*`` module: it computes over plain record dicts so
the collection logic is testable without a database, and so the
report shape is identical whether the records came from the control
plane (``server._metrics_report`` reading the attempt ledger), a
test fixture, or a hand-built record list.
Definitions (pinned here, not left to each caller):
- A task counts toward a rate only when it reached a terminal status
(``completed`` / ``merged`` / ``failed`` / ``merge-failed``).
``interrupted`` is deliberately NOT terminal: platform-interrupted
tasks auto-resume as fresh runs, so counting them as failures
would punish a platform event, not the agent.
- First-pass green: terminal task whose ``attempts`` == 1 and whose
status is ``completed`` / ``merged``.
- Defect: a task with ``defects`` > 0, where ``defects`` is the count
of failed CI/audit rounds or follow-up rounds for that task. Since
audit gap 3 these numbers come from the ``task_attempts`` ledger
(``cloud/attempt_ledger.py``: one row per real attempt, aggregated
by the endpoint); a task with no ledger rows must still pass
``None`` -- and the rates then come back as ``None`` with
``*_known`` False. An unknown rate is never reported as 0: 0
would claim "no defects", which is a decision-grade lie.
- Rework: terminal task with ``attempts`` > 1. Same None discipline.
- Per-task cost reuses the #212 dual-column discipline from
``cloud.cost``: priced USD micros and unpriced actual tokens are
reported in separate columns. ``total_usd_micros`` is ``None``
whenever any task's cost is incomplete (no cost rows, or unpriced
actual tokens > 0); the priced partial sum is still shown, labelled
``priced_usd_micros_partial``, so nobody mistakes it for a total.
Unpriced cost is never folded into 0.
Feedback (``evaluate_feedback``) is deliberately a pure function
returning action records, not an auto-applier: an action says WHAT
should change and WHY, with its evidence attached; applying it to
routing or to the audit standards is a separate, approved step
(see ``docs/arch-conformance-metrics.md``). The rules are testable:
- defect rate for a category above threshold -> ``tighten_check``
for that category's check;
- first-pass rate below its floor -> ``route_with_extra_audit``;
- a check whose violations are overturned on review above
threshold -> ``recalibrate_check`` (review the threshold, do NOT
auto-apply a new one -- a check that fires wrongly must not
silently widen itself).
Thresholds passed in are labelled by source in the report. The
defaults in ``DEFAULT_THRESHOLDS`` are starting values for tests,
NOT human-approved production thresholds: the report marks them
``thresholds_source: "default_unapproved"`` until a human approves a
set, and a report built on unapproved thresholds is still usable
for collection -- it is just not decision-grade on its own.
Golden tasks: a report is ``decision_ready`` only when a golden set
has been human-approved (``approved_by`` + ``approved_at`` present
and at least one golden task listed). Until then the golden section
reads ``status: "pending_human_approval"`` and the report says so
plainly; the sample-only set that exists today must never be
presented as decision-ready. This module never invents golden tasks.
"""
import json
import time
TERMINAL_STATUSES = frozenset(
{"completed", "merged", "failed", "merge-failed"})
SUCCESS_STATUSES = frozenset({"completed", "merged"})
DEFAULT_THRESHOLDS = {
"defect_rate": 0.20,
"first_pass_rate_min": 0.50,
"check_overturn_rate": 0.30,
}
def _rate(numerator, denominator):
if denominator is None or denominator == 0:
return None
return numerator / denominator
def compute_metrics(records, costs=None):
"""``records``: list of dicts with keys ``id``, ``status`` and
optional ``attempts`` (int), ``defects`` (int),
``defect_category`` (str). ``costs``: dict task_id -> dict with
``priced_usd_micros`` (int|None), ``unpriced_actual_tokens`` (int),
``has_rows`` (bool). Returns the metrics dict."""
costs = costs or {}
terminal = [r for r in records if r.get("status") in TERMINAL_STATUSES]
known_attempts = [r for r in terminal if r.get("attempts") is not None]
first_pass = [r for r in known_attempts
if r["attempts"] == 1
and r.get("status") in SUCCESS_STATUSES]
rework = [r for r in known_attempts if r["attempts"] > 1]
known_defects = [r for r in terminal if r.get("defects") is not None]
defective = [r for r in known_defects if r["defects"] > 0]
defect_by_category = {}
for r in defective:
cat = r.get("defect_category") or "uncategorized"
defect_by_category[cat] = defect_by_category.get(cat, 0) + 1
attempts_total = sum(r["attempts"] for r in known_attempts)
per_task_cost = []
priced_partial = 0
cost_complete = True
for r in terminal:
c = costs.get(r["id"]) or {}
priced = c.get("priced_usd_micros")
unpriced = int(c.get("unpriced_actual_tokens") or 0)
has_rows = bool(c.get("has_rows"))
complete = has_rows and unpriced == 0 and priced is not None
if not complete:
cost_complete = False
if priced is not None:
priced_partial += int(priced)
per_task_cost.append({
"task_id": r["id"],
"priced_usd_micros": priced,
"unpriced_actual_tokens": unpriced,
"cost_complete": complete,
})
if not terminal:
cost_complete = False
return {
"tasks_total": len(records),
"terminal_tasks": len(terminal),
"first_pass_tasks": len(first_pass),
"first_pass_rate": _rate(len(first_pass), len(known_attempts)),
"first_pass_known": bool(known_attempts),
"rework_tasks": len(rework),
"rework_rate": _rate(len(rework), len(known_attempts)),
"rework_known": bool(known_attempts),
"avg_attempts": (attempts_total / len(known_attempts)
if known_attempts else None),
"defect_tasks": len(defective),
"defect_rate": _rate(len(defective), len(known_defects)),
"defect_known": bool(known_defects),
"defect_by_category": defect_by_category,
"per_task_cost": per_task_cost,
# Dual-column total: None unless every terminal task's cost
# is complete; the partial sum is shown separately, labelled.
"priced_usd_micros_partial": priced_partial,
"total_usd_micros": priced_partial if cost_complete else None,
"cost_complete": cost_complete,
}
def evaluate_feedback(metrics, thresholds=None, check_stats=None):
"""Testable feedback rules. Returns a list of action dicts; each
carries its evidence so a reviewer can audit why it fired."""
thresholds = thresholds or DEFAULT_THRESHOLDS
actions = []
defect_rate = metrics.get("defect_rate")
if metrics.get("defect_known") and defect_rate is not None:
limit = thresholds.get("defect_rate")
if limit is not None and defect_rate > limit:
for cat, count in sorted(
metrics.get("defect_by_category", {}).items()):
actions.append({
"kind": "tighten_check",
"target": cat,
"reason": "defect_rate %.3f > threshold %.3f "
"(%d defective tasks in category)"
% (defect_rate, limit, count),
"evidence": {"defect_rate": defect_rate,
"threshold": limit,
"category_tasks": count},
"auto_apply": False,
})
first_pass = metrics.get("first_pass_rate")
floor = thresholds.get("first_pass_rate_min")
if first_pass is not None and floor is not None and first_pass < floor:
actions.append({
"kind": "route_with_extra_audit",
"target": "all",
"reason": "first_pass_rate %.3f < floor %.3f"
% (first_pass, floor),
"evidence": {"first_pass_rate": first_pass, "floor": floor},
"auto_apply": False,
})
overturn_limit = thresholds.get("check_overturn_rate")
for stat in check_stats or []:
violations = stat.get("violations") or 0
overturned = stat.get("overturned") or 0
if violations and overturn_limit is not None:
rate = overturned / violations
if rate > overturn_limit:
actions.append({
"kind": "recalibrate_check",
"target": stat.get("check_id"),
"reason": "overturn_rate %.3f > threshold %.3f "
"(%d/%d violations overturned on review)"
% (rate, overturn_limit, overturned,
violations),
"evidence": {"overturn_rate": rate,
"threshold": overturn_limit,
"violations": violations,
"overturned": overturned},
"auto_apply": False,
})
return actions
def build_report(metrics, golden=None, thresholds=None,
thresholds_source="default_unapproved",
check_stats=None, sampling_plan=None):
"""Assemble the JSON report document.
``golden``: None, or a dict with ``tasks`` (list of task ids),
``approved_by`` and ``approved_at``. Only a fully approved golden
set makes the report decision-ready; anything else is pending.
"""
golden = golden or {}
tasks = golden.get("tasks") or []
approved = bool(tasks) and bool(golden.get("approved_by")) \
and bool(golden.get("approved_at"))
return {
"generated_at": time.time(),
"golden_set": {
"status": "approved" if approved else "pending_human_approval",
"tasks": list(tasks),
"approved_by": golden.get("approved_by"),
"approved_at": golden.get("approved_at"),
},
"decision_ready": approved,
"decision_note": ("golden set human-approved; metrics may inform "
"decisions" if approved else
"golden set NOT human-approved: this report "
"collects metrics only and must not be used "
"as a decision basis"),
"metrics": metrics,
"thresholds": dict(thresholds or DEFAULT_THRESHOLDS),
"thresholds_source": thresholds_source,
"feedback_actions": evaluate_feedback(
metrics, thresholds=thresholds, check_stats=check_stats),
# How the loop closes (see docs/arch-conformance-metrics.md):
# feedback actions are consumed by (a) routing -- tasks in a
# flagged category are routed with an extra audit gate, and
# (b) audit standards -- a tighten/recalibrate action opens a
# standards-review item; the standards document itself changes
# only through the contract-change protocol
# (arch_conformance.check_contract_version) with coordinator
# approval, never by auto-applying an action record.
"feedback_consumers": ["routing", "audit_standards"],
# Audit-of-audit sampling (cloud/audit_sampling.py): when a
# plan is supplied it is carried verbatim; otherwise the
# report states sampling is pending rather than implying
# the audit was re-examined.
"sampling": sampling_plan or {
"status": "pending",
"note": "no audit-of-audit sample plan supplied; audit "
"verdicts in this report have not been "
"re-examined"},
}
def render_report(report):
"""Canonical JSON rendering of a report document."""
return json.dumps(report, indent=2, sort_keys=True, default=str)
+5 -2
View File
@@ -29,7 +29,6 @@
本模块不读、不拼连接串、不接受命令行密码参数。
"""
import os
import re
import sqlite3
@@ -47,6 +46,9 @@ MIGRATION_TABLES = (
"project_members",
"tasks",
"audit_log",
# 防篡改账本(哈希链,第 4 轨):与 audit_log 同款身份列,
# 迁库时整行原样迁(id 与哈希不动,链不断),序列见下。
"audit_ledger",
"summaries",
"kb_docs",
"cost_records",
@@ -92,12 +94,13 @@ MIGRATION_TABLES = (
_SKIP_TABLES = frozenset({"kb_fts", "sqlite_sequence"})
# 身份列的表:POSTGRES_SCHEMA 里唯一用
# ``BIGINT GENERATED BY DEFAULT AS IDENTITY`` 的 4 张表。
# ``BIGINT GENERATED BY DEFAULT AS IDENTITY`` 的 5 张表。
# 迁移时显式插 id 不会推序列,迁完必须 setval 到 MAX(id),
# 否则后续省略 id 的 INSERT 会撞主键。新增身份列表必须
# 同步改这里,test_p2_migrations.py 会拿 schema 实测对齐。
SEQUENCE_TABLES = (
"audit_log",
"audit_ledger",
"summaries",
"kb_docs",
"cost_records",
-1
View File
@@ -41,7 +41,6 @@
import json
import os
import secrets
import sqlite3
from cloud.db import connect as _db_connect
import time
import urllib.parse
+13 -5
View File
@@ -62,14 +62,22 @@ def peer_message_prefix(from_task_id):
return "[来自任务 %s 的消息]\n" % from_task_id
def row_get(row, name):
"""行取值容错:sqlite3.Row 缺列报 IndexError、dict 缺键
报 KeyError、空行报 TypeError,一律回 None(worker 直连
老库、列还没迁移时排空不能因此停摆)。"""
def row_get(row, name, default=None):
"""行取值容错(全仓行取值的统一口,A7):sqlite3.Row 缺列
报 IndexError、dict 缺键报 KeyError、空行报 TypeError,
一律回 default(默认 None;worker 直连老库、列还没迁移时
排空不能因此停摆)。handler 链、auto_memory、lifecycle
三处统一走本函数:三者旧语义在真实行类型(None / dict /
sqlite3.Row)下完全一致——auto_memory 旧版的 default 参数
默认就是 None,lifecycle 旧版的 None 特判与 dict .get 在
这些类型下与本函数结果相同;旧 lifecycle 版另带的宽
Exception 兜底只对“__getitem__ 抛其它异常的病态行”有
区别,那类输入不是 DB 行,不属于本函数契约,统一后此
类输入改为抛给调用方(不在测试锁定的真实行类型内)。"""
try:
return row[name]
except (KeyError, IndexError, TypeError):
return None
return default
def extract_peer_messages(text):
+10 -2
View File
@@ -23,15 +23,23 @@ each new page gets a fresh Playwright context, so cookies are
isolated between pages.
"""
import importlib
import json
import os
import sys
# cloud 包优先、租户 flat copy 回落:同一段代码在两种部署形态下
# 都要跑。importlib 动态取属性而不是两次 from-import:两次
# from-import 会让 mypy 报 no-redef(cloud.* 与顶层模块名不同),
# 而属性绑定在两条路径下都是同一组名字;cloud 导入本身的
# ImportError(含内层缺依赖)同样回落,与原来 except 语义一致。
try:
from cloud.playwright_browser import Browser, BrowserError
_mcp_backend = importlib.import_module("cloud.playwright_browser")
except ImportError: # flat copy in a tenant codex-home
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from playwright_browser import Browser, BrowserError # noqa: E402
_mcp_backend = importlib.import_module("playwright_browser")
Browser = _mcp_backend.Browser
BrowserError = _mcp_backend.BrowserError
SERVER_NAME = "playwright"
SERVER_VERSION = "0.1.0"
+333
View File
@@ -0,0 +1,333 @@
"""QA full-chain checks: self-test, PRD coverage, agent six,
cross-client matrix, G5 gate, regression scope, dual signatures.
Pure functions, standard library only. This is the QA half of the
audit-hardening track; the audit half lives in
``cloud/audit_pushback.py``. The two are different checks and
neither report substitutes for the other: the QA matrix proves
the business links run on every selected client, the independent
audit re-runs the exact artifacts. The checklist therefore has
two separate signature seats (QA matrix signer, independent
audit signer) and both must be present and different, or the
round does not pass.
Honest exceptions, written as rules rather than judgement calls:
- On a Linux host, iOS / Android / macOS cannot be compiled.
Their honest verdict is ``"code_review_plus_web_core"`` -- never
``"pass"`` / ``"build_pass"``. An untested client is never a
pass, even with a reason: the reason makes the matrix
*accounted for*, it does not make QA pass.
- QA passing is not user acceptance. G6 acceptance is ticked
only by the user; QA, audit, pushback, and rebuild can never
tick it (see ``check_g6_guard`` in ``audit_pushback``).
Counting relation to pushback / rebuild (single story, no second
counter): one failed round counts once for its module, however
many QA links failed in that round -- links are failed items
inside the round, not extra failures. A standalone G5 QA failure
(without an audit round) runs the G5 defect loop (back to G4,
re-regress in G5) and does not by itself increment the module's
audit/acceptance failure count; it increments the count only
when the round reaches an audit failure or a QA-acceptance
sign-off failure, recorded with source ``"audit"`` or
``"acceptance"`` respectively. A targeted fix round and a
rebuild both re-run the QA matrix: the re-audit must re-run the
failed links listed in the pushback package (see
``check_reaudit_reran_links``), not just unit tests.
"""
# Per-client self-test, from the v0.2 skill text (production skill).
SELF_TEST_PARTS = ("unit", "contract_integration", "manual_smoke")
MANUAL_SMOKE_STEPS = ("startup", "login", "core_agent_task",
"error_prompt")
# Agent-specific six: one missing means the test round is incomplete.
AGENT_SIX = (
"sandbox_escape",
"dangerous_action_approval",
"long_task_timeout_cancel",
"offline_resume",
"model_503_retry",
"token_quota_exceeded",
)
AGENT_SIX_LABELS = {
"sandbox_escape": "沙箱越界",
"dangerous_action_approval": "危险动作审批",
"long_task_timeout_cancel": "长任务超时和取消",
"offline_resume": "断网续跑",
"model_503_retry": "模型 503 重试",
"token_quota_exceeded": "token / 配额超限",
}
# G5 pass gates, from the v0.2 skill text.
G5_GATES = (
"unit",
"integration",
"cross_client_consistency",
"agent_security",
"long_task",
"performance_and_security",
)
# Clients that cannot be compiled on a Linux host (v0.2 skill text).
UNCOMPILED_CLIENTS = ("ios", "android", "macos")
# Fields compared across clients for the same task and data.
CONSISTENCY_FIELDS = ("result", "status_word", "sync_state")
def _as_bool_map(values):
if isinstance(values, dict):
return values
out = {}
for entry in values or []:
key = entry.get("id") or entry.get("key") or entry.get("name")
out[key] = bool(entry.get("tested", entry.get("passed", False)))
return out
def check_self_test(evidence):
"""Per-client self-test completeness. Returns (complete, missing).
``evidence``: ``{"unit": bool, "contract_integration": bool,
"manual_smoke": {"startup": bool, "login": bool,
"core_agent_task": bool, "error_prompt": bool}}``. All three
parts are required; a missing smoke step is listed by name.
"""
missing = []
if not evidence.get("unit"):
missing.append("unit")
if not evidence.get("contract_integration"):
missing.append("contract_integration")
smoke = evidence.get("manual_smoke") or {}
for step in MANUAL_SMOKE_STEPS:
if not smoke.get(step):
missing.append("manual_smoke:%s" % step)
return (not missing, missing)
def check_prd_coverage(prd_criteria, cases):
"""PRD coverage: every criterion needs a valid case.
A case is valid only when it carries a precondition, steps,
and an expected result, all non-empty. A criterion covered
only by an invalid case is still missing. Returns
``{"covered": [...], "missing_criteria": [...],
"invalid_cases": [...]}``.
"""
valid_cover = {}
invalid = []
for case in cases or []:
case_id = case.get("id", "?")
if case.get("precondition") and case.get("steps") \
and case.get("expected"):
for criterion in case.get("covers") or []:
valid_cover.setdefault(criterion, []).append(case_id)
else:
invalid.append(case_id)
missing = [criterion for criterion in prd_criteria or []
if criterion not in valid_cover]
return {"covered": sorted(valid_cover),
"missing_criteria": missing,
"invalid_cases": invalid}
def check_agent_six(agent_items):
"""Agent-specific six completeness. Returns missing keys."""
covered = _as_bool_map(agent_items)
return [key for key in AGENT_SIX if not covered.get(key)]
def check_cross_client_matrix(selected_clients, rows):
"""Cross-client matrix: accounted-for vs QA-pass, kept apart.
``rows``: one per client, ``{"client": ..., "verdict":
"pass" | "untested" | "code_review_plus_web_core",
"untested_reason": ...}``. Returns ``{"accounted": bool,
"qa_pass": bool, "missing_clients": [...], "violations":
[...]}`` where ``accounted`` means every selected client has
a row (and a reason when untested), while ``qa_pass`` additionally
requires no untested row and no uncompiled client claiming a
build pass. An untested client with a reason is accounted but
does not pass; an untested client without a reason is neither.
"""
by_client = {row.get("client"): row for row in rows or []}
missing_clients = [client for client in selected_clients or []
if client not in by_client]
violations = []
has_untested = False
for client in selected_clients or []:
row = by_client.get(client)
if row is None:
continue
verdict = row.get("verdict")
if verdict == "untested":
has_untested = True
if not row.get("untested_reason"):
violations.append(
"untested without reason:%s" % client)
elif verdict == "code_review_plus_web_core":
if client not in UNCOMPILED_CLIENTS:
violations.append(
"code_review verdict on compilable client:%s" % client)
elif verdict == "pass":
if client in UNCOMPILED_CLIENTS:
violations.append(
"uncompiled client claims build pass:%s" % client)
else:
violations.append("unknown verdict:%s:%s" % (client, verdict))
accounted = not missing_clients and not any(
"untested without reason" in violation for violation in violations)
qa_pass = accounted and not has_untested and not violations
return {"accounted": accounted, "qa_pass": qa_pass,
"missing_clients": missing_clients,
"violations": violations}
def check_consistency(rows):
"""Same task, same data, across clients: consistent or classified.
``rows``: ``{"client": ..., "result": ..., "status_word": ...,
"sync_state": ..., "classification": "defect" | "intentional"
| None, "basis": ...}``. A field that differs across clients
must be classified: ``"defect"`` fails QA (it must be fixed),
``"intentional"`` passes only with a non-empty basis --
"intentional" without a basis is a defect hiding behind a word.
Returns ``{"consistent_or_explained": bool, "mismatches": [...],
"blocking_defects": [...]}``.
"""
mismatches = []
blocking = []
for field in CONSISTENCY_FIELDS:
values = {}
for row in rows or []:
values.setdefault(row.get(field), []).append(row.get("client"))
if len(values) <= 1:
continue
# The first row's value is the reference; only deviating
# rows need a classification. The reference row itself
# carries no classification and must not be flagged.
baseline = (rows or [{}])[0].get(field)
for row in rows or []:
if row.get(field) == baseline:
continue
classification = row.get("classification")
basis = row.get("basis")
if classification == "defect":
blocking.append("%s defect on %s"
% (field, row.get("client")))
elif classification == "intentional" and not basis:
blocking.append("%s intentional without basis on %s"
% (field, row.get("client")))
elif classification not in ("defect", "intentional"):
blocking.append("%s unclassified mismatch on %s"
% (field, row.get("client")))
mismatches.append({"field": field, "values": values})
return {"consistent_or_explained": not blocking,
"mismatches": mismatches, "blocking_defects": blocking}
def check_g5(g5_evidence):
"""G5 gate: all gates pass and no severe defect unfixed.
``g5_evidence``: ``{"gates": {gate: bool}, "failures":
[{"severity": "severe" | ..., "fixed": bool, "owner": ...,
"reproduction": ...}]}``. A severe defect that is not fixed
blocks the recommendation to enter acceptance, however green
the gates look. Returns ``{"g5_pass": bool, "missing_gates":
[...], "blocking_defects": [...]}``.
"""
gates = g5_evidence.get("gates") or {}
missing = [gate for gate in G5_GATES if not gates.get(gate)]
blocking = []
for failure in g5_evidence.get("failures") or []:
if failure.get("severity") == "severe" and not failure.get("fixed"):
blocking.append(failure)
return {"g5_pass": not missing and not blocking,
"missing_gates": missing, "blocking_defects": blocking}
def check_qa_fix_role(fix_records):
"""QA must not fix code itself. Returns violations.
``fix_records``: ``{"defect": ..., "fixed_by": role}``. A fix
recorded as fixed by QA is a violation: QA reproduces and
verifies, the owning client/architecture thread fixes, and
the fix returns to G5 for regression.
"""
return ["qa fixed code directly:%s" % record.get("defect")
for record in fix_records or []
if record.get("fixed_by") == "qa"]
def plan_regression(impact_areas, cases):
"""Regression scope from the contract-change impact surface.
Returns ``{"regression_cases": [...], "uncovered_areas": [...],
"is_full_retest": False}``. Only cases covering at least one
impact area are selected -- never a full retest by default.
Every impact area must have at least one regression case, or
it is listed uncovered. The impact list must include defect
areas even when the contract did not change; an empty impact
list selects nothing, it does not silently mean "everything".
"""
selected = []
covered = set()
for case in cases or []:
covers = set(case.get("covers") or [])
hit = covers & set(impact_areas or [])
if hit:
selected.append(case.get("id"))
covered |= hit
uncovered = [area for area in impact_areas or []
if area not in covered]
return {"regression_cases": selected,
"uncovered_areas": uncovered,
"is_full_retest": False}
def check_signatures(qa_signer, audit_signer):
"""Dual signature seats: QA matrix + independent audit.
Both seats must be filled and by different people. A missing
seat, or one person holding both seats, fails the round: the
QA report can never substitute for the audit, nor the reverse.
Returns violations (empty means both seats valid).
"""
violations = []
if not qa_signer:
violations.append("missing qa matrix signature")
if not audit_signer:
violations.append("missing independent audit signature")
if qa_signer and audit_signer and qa_signer == audit_signer:
violations.append("qa and audit signatures by the same person")
return violations
def check_reaudit_reran_links(pushback_package, rerun_results):
"""Re-audit must re-run the failed links, not just unit tests.
``rerun_results`` maps link name -> bool (or a list of
``{"link": ..., "passed": bool}``). A failed link with no
re-run result fails closed; a failed re-run fails the
re-audit. Returns ``{"reran_all_failed_links": bool,
"missing": [...], "failed": [...]}``.
"""
results = _as_bool_map([
{"id": key, "passed": value}
for key, value in rerun_results.items()] if isinstance(
rerun_results, dict) else [
{"id": entry.get("link"), "passed": entry.get("passed")}
for entry in rerun_results or []])
required = []
for link in pushback_package.get("reaudit_must_rerun_links") or []:
if isinstance(link, dict):
required.append(link.get("link") or link.get("name"))
else:
required.append(link)
missing = [link for link in required if link not in results]
failed = [link for link in required
if link in results and not results[link]]
return {"reran_all_failed_links": not missing and not failed,
"missing": missing, "failed": failed}
+176
View File
@@ -0,0 +1,176 @@
"""成本与配额的唯一判定源。
收敛前同一口径写在三处,注释里明文写着“改口径时三处
要一起改”(server._quota_status、worker._quota_exceeded、
automations._automation_quota_exceeded),且三处已经
不一致——这是本轨的 P0:
- 无价 token 上限的计数口径:worker 与 automations 只计
source='actual' 的行,服务端 _quota_unpriced_tokens 计
全部 usd IS NULL 的行(含 estimated 估算行);而
cost.py 的 P1 决议、CostStore._report 的
unpriced_actual_tokens、服务端自己的项目成本报表
(actual-only)都站在 actual-only 一边。服务端是孤例。
本轨以 actual-only 为 canonical:估算行不是真实花费,
绝不能触发上限;服务端 _quota_unpriced_tokens 改为本
模块口径(行为变化仅限含 estimated 无价行的用户:此
前可能被估算行误拦,此后不再误拦;真实花费的拦阻不变)。
集成 v2 确认(临时口径,非原 PR 已定):本收敛来自
集成基底(非 main),原 PR 语义(main 里 #86 计全部
vs #90 actual-only 的分歧)并未在 main 上改动;合轨
时须由协调员另行确认 actual-only 是否成为原 PR 口径,
本轨只把集成 v2 的口径钉死并补双侧回归测试(见
cloud/tests/test_v2_evidence.py),绝不把临时口径写
成原 PR 已定。受影响调用方:server._quota_snapshot /
_check_unpriced_token_cap、worker._quota_limit_and_used /
_quota_exceeded、automations._automation_quota_exceeded,
三处均经本模块 monthly_usage + decide,改一处即三处
同口径;USD 求和不受影响(estimated 行 usd 为 NULL,
SUM 本就跳过)。
- 默认配额解析:server、worker、automations 各一份
CLOUD_DEFAULT_QUOTA_USD_MICROS 解析,现统一为
default_quota_usd_micros(默认值 $10/月 = 10_000_000
微美元,非法值回落,无配额行按默认值、显式 NULL 行
才是不限额——三处旧语义一致,收敛不改语义)。
- 比较判定:used >= quota(USD)先于 unpriced >= cap
(无价),两者任一命中即阻断;该顺序与三处旧代码一致,
现收为 decide() 一处。调用方只负责取数与映射错误码
(服务端 402、worker interrupted、自动化 skipped_quota),
不得在调用点另写一套 >= 比较。
本模块只依赖标准库,是叶子模块:调用方传一个 DBAPI
连接(? 占位符,Postgres 适配层同款翻译),不 import
cloud 其它模块,避免循环。无价上限的数值本身仍由
cost.unpriced_token_cap() 拥有(环境变量
CLOUD_QUOTA_UNPRICED_TOKEN_CAP),decide() 只接受
cap 作参数,不再另起一套上限解析。
调阈边界(写清,不越界):
- 本模块只判定、不建议、不生效。没有任何函数会算
“建议把上限调到 X”,也没有任何函数会写回配额行或
改环境变量。阈值变化的唯一生效路径是运维改配置:
改 user_quotas 行(立即对下次查询生效)或改环境变量
后重启进程(进程内 env 按调用时读取)。未来如需
“自动算建议”,必须是独立的只读函数,其输出绝不能
被 decide() 或任何准入点自动采纳。
- 归属口径冻结:按任务创建者 created_by 算,不按项目、
不按操作人;协调员子任务无 created_by 时跟父任务走
(该回查仍在 worker 侧,属主解析不是本模块的职责)。
- 窗口口径冻结:UTC 自然月,与 cost_records.day 同口径
(day LIKE 'YYYY-MM%')。
"""
import os
from datetime import datetime, timezone
DEFAULT_QUOTA_USD_MICROS = 10000000
def default_quota_usd_micros():
"""默认月配额(微美元):环境变量非法时回落 $10。"""
try:
return max(0, int(os.environ.get(
"CLOUD_DEFAULT_QUOTA_USD_MICROS", "10000000")))
except (TypeError, ValueError):
return DEFAULT_QUOTA_USD_MICROS
def utc_month(now=None):
"""UTC 自然月 'YYYY-MM'(与 cost_records.day 同口径)。"""
moment = datetime.now(timezone.utc) if now is None else now
return moment.strftime("%Y-%m")
def _row_value(row, name):
try:
return row[name]
except (KeyError, IndexError, TypeError):
return None
def quota_for_owner(conn, owner):
"""属主的配额行:无行回默认值,显式 NULL 行回 None(不限额)。
只读:无行时不在此写行(写行由服务端的
_apply_default_quota / _backfill_default_quota 负责)。
表不存在时先幂等建表(与三处旧代码同款)。
"""
conn.execute(
"CREATE TABLE IF NOT EXISTS user_quotas("
"user_id TEXT PRIMARY KEY, quota_usd_micros BIGINT,"
" updated_at DOUBLE PRECISION)")
row = conn.execute(
"SELECT quota_usd_micros FROM user_quotas"
" WHERE user_id = ?",
(owner,)).fetchone()
if row is None:
return default_quota_usd_micros()
return _row_value(row, "quota_usd_micros")
def monthly_usage(conn, owner, month):
"""当月已记账用量:(used_usd_micros, unpriced_actual_tokens)。
USD:cost_records 与 tasks 按 created_by 连接,SUM
usd_micros(estimated 行 usd 为 NULL,SUM 自动跳过,
与旧三处一致)。无价:usd IS NULL 且 source='actual'
的行 input + output 之和——estimated 行一律排除(见
模块头 P0)。调用方须保证 cost_records 表已存在
(服务端/worker 走 CostStore 懒建,自动化走表存在探针)。
"""
used_row = conn.execute(
"SELECT COALESCE(SUM(c.usd_micros), 0)"
" FROM cost_records c JOIN tasks t ON c.task_id = t.id"
" WHERE t.created_by = ? AND c.day LIKE ?",
(owner, month + "%")).fetchone()
used = _row_value(used_row, 0)
if used is None:
# sqlite3.Row 按列名取不到时按位置取。
try:
used = used_row[0]
except (IndexError, TypeError):
used = 0
unpriced_row = conn.execute(
"SELECT COALESCE(SUM(c.input_tokens + c.output_tokens), 0)"
" FROM cost_records c JOIN tasks t ON c.task_id = t.id"
" WHERE t.created_by = ? AND c.day LIKE ?"
" AND c.usd_micros IS NULL AND c.source = 'actual'",
(owner, month + "%")).fetchone()
unpriced = _row_value(unpriced_row, 0)
if unpriced is None:
try:
unpriced = unpriced_row[0]
except (IndexError, TypeError):
unpriced = 0
return int(used or 0), int(unpriced or 0)
def decide(quota, used_usd, unpriced_used, cap, month):
"""唯一比较判定:放行回 None,否则回决定 dict。
quota 为 None(显式不限额)时永远放行,即使无价
已超——与服务端、worker 旧行为一致(不限额用户不
受无价上限约束)。USD 门先于无价门;命中条件均为
>=(用满即拦,不是超过才拦)。
"""
if quota is None:
return None
if used_usd >= quota:
return {
"kind": "usd",
"quota_usd_micros": quota,
"used_usd_micros": used_usd,
"month": month,
"message": ("quota_exceeded: 本月配额已用完(已用 %d / "
"配额 %d 微美元,%s)" % (used_usd, quota, month)),
}
if unpriced_used >= cap:
return {
"kind": "unpriced",
"unpriced_token_cap": cap,
"unpriced_tokens_used": unpriced_used,
"month": month,
"message": ("quota_exceeded: 本月无价模型 token 已用完"
"(已用 %d / 上限 %d,%s)"
% (unpriced_used, cap, month)),
}
return None
+200
View File
@@ -0,0 +1,200 @@
"""Release gate, unified: thin adapter over the single evaluator.
The sole release evaluator is
``cloud/audit_loop_wiring.evaluate_release_gate``. The live
endpoint (``POST /tasks/{id}/release``,
``handler_lifecycle._release_task``) calls it directly; this
module only adapts its decision into the legacy shape below so
the contract tests in ``cloud/tests/test_release_gate.py`` keep
pinning the same behaviour. There is no second evaluator in
this tree: do not add evaluation logic here, change it in
``audit_loop_wiring`` and update this adapter's translation.
``deploy.py`` remains an operator script, not a release endpoint
(see ``docs/release-gate-and-ledger-anchor.md``).
What this module IS:
- A pure, read-only adapter: ``evaluate_release`` delegates to
the wiring evaluator, which re-reads the hash chain through
the SAME reader the merge endpoint uses and ANDs it with a
FRESH hard-gate evaluation supplied by the caller. It never
appends to the chain, never calls ``AuthStore.audit``
(which writes a chain row in the same transaction), never
commits, and never deploys anything. Checking must not move
the chain head: otherwise the check itself would invalidate
the external anchor taken before it.
- The contract the live endpoint implements. The endpoint
(``POST /tasks/{id}/release`` in ``handler_lifecycle``) calls
the wiring evaluator directly and maps its decision to HTTP
exactly as documented in the companion doc; this module only
adapts that decision into the legacy shape for its contract
tests, it is not a second evaluation path.
What this module is NOT:
- It is not a merge gate. Merge (``POST /tasks/{id}/merge``)
and release are separate decisions with separate re-checks:
a merge-time 200 is never a release pass. The live endpoint
calls the wiring evaluator at release time, with fresh
evidence, even if the task was merged minutes ago.
- It is not a deploy gate. ``deploy.py`` deploys the platform
itself (host release directory + unit restarts) and is
deliberately outside this gate; see the companion doc.
- It does not decide task status, auth, or publication. Status
(``task_status``) and the hard-gate evidence (``hard_gate``)
are caller-supplied inputs, so the evaluator can be tested
without a server, a task row, or a publication step. When the
caller omits an input, the evaluator fails closed on that
input instead of assuming it (missing hard-gate evidence
blocks; an unmerged status blocks when a status is given).
Decision shape (``evaluate_release`` return value)::
{
"allowed": bool, # True only if every check passes
"http_status": 200 | 409, # the status the endpoint maps to
"code": str, # machine code, see below
"blocked_by": str | None, # primary source, or None if allowed
"blocked_sources": [str], # every applicable source, in order
"blocks": [str], # hard-gate block codes, if any
"failed_checks": [str], # audit-pushback failed checks, if any
"round": int | None, # pushback round, if any
"modules": dict, # per-module block detail, if any
}
Machine codes (``code`` / ``blocked_sources``):
- ``release_allowed`` -- every check passed.
- ``release_requires_merged`` -- a task status was supplied and
it is not ``"merged"``. Release is a post-merge act; an
unmerged task has nothing to release.
- ``release_blocked_state_unavailable`` -- the chain could not
be read at all (anything other than the reader's own
missing-table rule, which means "no chain has ever been
written" and is inherited from #247 unchanged). Fail-closed:
an unreadable chain is never read as "no block".
- ``release_blocked_hard_gate`` -- the fresh hard-gate
evaluation (five checks: CI, directory, contract, coordinator,
reviews, plus branch protection) did not allow. The failing
codes travel in ``blocks`` verbatim, so the caller can tell
a hard-gate block from an audit block without parsing text.
- ``release_blocked_audit_pushback`` -- the re-read chain says
a pushback is still open (``release_blocked`` from the same
reader the merge endpoint uses). This is the #247 reader's
verdict at release time, not a cached merge-time value.
When several sources block at once, ``blocked_sources`` lists
all of them (chain-unavailable dominates and short-circuits,
because without a chain read no other verdict is trustworthy),
while ``blocked_by``/``code`` name the first source in
evaluation order: chain read, then task status, then hard gate,
then audit pushback. The endpoint must return the full
decision object, not just the first code, so an operator can
see that fixing one source still leaves another blocking.
CORRECTED-to-pass does NOT unblock (inherited, not invented):
The reader already implements the #247/#244 resolved semantic:
a meta-review correction to ``pass`` schedules a re-audit, it
does not release the gate; only a fresh passing verdict or a
passing re-reviewed pushback row unblocks. This evaluator adds
no special case for corrections, so the release path cannot
drift from the merge path: the same chain, read through the
same reader, gives the same answer at both gates. The contract
tests pin this: corrected-to-pass with no new passing
conclusion stays blocked at release, exactly as at merge.
"""
from cloud.audit_loop_wiring import evaluate_release_gate as _wiring_evaluate
# Machine codes (kept as module constants so the endpoint
# and its tests spell them identically, never by string literal).
CODE_ALLOWED = "release_allowed"
CODE_REQUIRES_MERGED = "release_requires_merged"
CODE_STATE_UNAVAILABLE = "release_blocked_state_unavailable"
CODE_HARD_GATE = "release_blocked_hard_gate"
CODE_AUDIT_PUSHBACK = "release_blocked_audit_pushback"
CODE_HARD_GATE_EVIDENCE_MISSING = "release_hard_gate_evidence_missing"
def _decision(allowed, code, blocked_by, sources, blocks=None,
failed_checks=None, round_no=None, modules=None):
return {
"allowed": allowed,
"http_status": 200 if allowed else 409,
"code": code,
"blocked_by": blocked_by,
"blocked_sources": list(sources),
"blocks": list(blocks or []),
"failed_checks": list(failed_checks or []),
"round": round_no,
"modules": dict(modules or {}),
}
def evaluate_release(task_id, conn, hard_gate=None, task_status=None):
"""Thin compatibility adapter over the single evaluator.
The sole implementation is
``cloud/audit_loop_wiring.evaluate_release_gate`` (used
directly by the live endpoint
``handler_lifecycle._release_task``). This function only
translates that decision into the legacy shape this module
originally returned (``blocked_sources`` as codes, plus
``blocks`` / ``failed_checks`` / ``round`` / ``modules``),
so the contract tests in ``test_release_gate.py`` keep
pinning the same behaviour without a second evaluator
existing anywhere in the tree.
Legacy contract preserved verbatim:
- ``task_status`` omitted skips the status check (the live
endpoint must never omit it: it always passes the real
status). Wiring blocks a ``None`` status, so the adapter
passes ``"merged"`` as a placeholder ONLY in the omitted
case; that placeholder is not a verdict, it is the skip.
- ``hard_gate=None`` blocks with
``release_hard_gate_evidence_missing``; a merge-time 200
is never accepted as fresh evidence.
- A failed chain read blocks fail-closed with
``release_blocked_state_unavailable``.
- The adapter never writes: it delegates to the read-only
wiring evaluator and only reshapes its return value.
"""
effective_status = "merged" if task_status is None else task_status
try:
decision = _wiring_evaluate(
task_id, conn, task_status=effective_status,
hard_gate=hard_gate)
except Exception:
return _decision(
False, CODE_STATE_UNAVAILABLE,
"audit_pushback_state_unavailable",
[CODE_STATE_UNAVAILABLE])
if decision.get("code") == CODE_STATE_UNAVAILABLE:
return _decision(
False, CODE_STATE_UNAVAILABLE,
"audit_pushback_state_unavailable",
[CODE_STATE_UNAVAILABLE])
pushback = decision.get("audit_pushback") or {}
sources = []
for source in decision.get("blocked_sources") or []:
if source == "task_status":
sources.append(CODE_REQUIRES_MERGED)
elif source == "hard_gate":
sources.append(
CODE_HARD_GATE_EVIDENCE_MISSING if hard_gate is None
else CODE_HARD_GATE)
elif source == "audit_pushback":
sources.append(CODE_AUDIT_PUSHBACK)
blocks = []
if isinstance(hard_gate, dict):
blocks = list(hard_gate.get("blocks") or [])
if not sources:
return _decision(True, CODE_ALLOWED, None, [])
return _decision(
False, decision.get("code"), decision.get("blocked_by"),
sources, blocks=blocks,
failed_checks=list(pushback.get("failed_checks") or []),
round_no=pushback.get("round"),
modules=dict(pushback.get("modules") or {}))
+1 -1
View File
@@ -89,7 +89,7 @@ from urllib.parse import urlsplit
from cloud.automations import _audit_row as _auto_audit_row
from cloud.automations import _automation_quota_exceeded
from cloud.automations import row_get
from cloud.peer_messages import row_get
from cloud.db import PostgresIntegrityError as _PgIntegrityError
MIN_INTERVAL_SECONDS = 60
+52 -13
View File
@@ -22,6 +22,7 @@ import subprocess
import sys
import time
from cloud.sandbox_policy import TASK_CONTAINER_RE as _POLICY_TASK_RE
from cloud.marker_tools import extract_quota_status as _extract_quota_status
from cloud.marker_tools import extract_automation_creates as \
_extract_automation_creates
@@ -41,6 +42,33 @@ from cloud.marker_tools import extract_wait_threads as _extract_wait_threads
from cloud.peer_messages import extract_peer_messages as _extract_peer_messages
MAX_STDOUT_CHARS = 2000
def tmp_isolation_status():
"""各层 /tmp 隔离现状的唯一口径(README 与单测都以它为准)。
返回 {docker: bool, host_bwrap: bool, ...}:
- docker=True:DockerExecutor 给每个任务容器挂 --tmpfs /tmp
与 --tmpfs /var/tmp(容器随 --rm 消失,/tmp 也随之消失,
绝不落宿主 /tmp;见 build_run_command 的顺序注释)。
- host_bwrap=False:宿主级 codex workspace-write(bwrap)
的挂载由 codex 二进制内部决定,host.py 只传
sandboxPolicy 类型、没有任何挂载旋钮,所以本仓库改不
了它;2026-10-26 宿主实测 agent 写 /tmp 落到宿主 /tmp,
本轮未在宿主重测、也不宣称已修复。需要 /tmp 隔离的
任务请走 docker 执行器,不要走宿主级沙箱 worker。
本函数是状态宣告,不是修复开关:调用方绝不能拿它的
docker=True 去给宿主级任务背书。
"""
return {
"docker": True,
"host_bwrap": False,
"docker_detail": "per-container tmpfs /tmp + /var/tmp, removed with --rm",
"host_bwrap_detail": "bwrap mounts are owned by the codex binary;"
" host.py passes only a sandboxPolicy type"
" and has no mount knob; /tmp is shared"
" with the host (2026-10-26 host test)",
}
# 任务结果文本上限:2000 字截断的是 host.py 打印的整段 JSON,
# text 超过 2000 字时 JSON 本身被截断,worker 再解析 stdout
# 取 text 必然失败,存库的就变成截断的 JSON(2026-10-30
@@ -1111,23 +1139,33 @@ class DockerExecutor(Executor):
cmd += ["--runtime", self.runtime]
if network:
cmd += ["--network", network]
cmd += ["-v", "%s:/work" % self._worktree_for(task),
"-v", "%s:/app:ro" % self._app_dir(),
"-w", "/work"]
# 额外挂载放在基础挂载之后:/app/work 这类子路径挂载会
# 覆盖 /app:ro 的对应子路径,让容器内 host.py 能写
# work/ 和 host-state.json,同时代码主体仍只读。
mounts = self.extra_mounts if extra_mounts is None \
else self._parse_extra_mounts(extra_mounts)
for host_p, cont_p, mode in mounts:
cmd += ["-v", "%s:%s:%s" % (host_p, cont_p, mode)]
# /tmp 固定为 tmpfs:容器内 /tmp 绝不落在宿主 /tmp,也不受
# 额外挂载影响(/tmp 目标的额外挂载已在解析时拒绝)。
# /tmp 固定为 tmpfs,且必须放在所有 -v 挂载之前:父挂载
# 在前、子挂载在后是 Linux 挂载顺序的基本要求——附件
# seed 以 /tmp/codex-seed:ro 挂进容器(见 dispatch_task),
# 它是 /tmp 之下的子挂载,只有 tmpfs 先挂、seed 后挂,
# seed 才不会被后挂的 tmpfs 遮掉。旧顺序把 --tmpfs 放在
# 全部 -v 之后,本 VM 无 docker、未在真实 docker 上证实
# 旧顺序确实遮挡(docker 可能按父子关系重排),所以这里
# 写为顺序加严而非“已证实 bug 修复”;单测只断言顺序,
# 不宣称旧顺序一定失效。
# 容器内 /tmp 绝不落在宿主 /tmp,也不受额外挂载影响
# (/tmp 目标的额外挂载已在解析时拒绝)。
# 诚实边界:这是 Docker 外层能做的;内层 codex 的 bwrap
# workspace-write 仍由 codex 二进制控制,本仓库改不了它
# 的 --proc 挂载,宿主级 bwrap 的 /tmp 不隔离问题也不在
# 本仓库的修复范围内——不要把本改动说成“内层沙箱已修”。
cmd += ["--tmpfs", "/tmp", "--tmpfs", "/var/tmp"]
cmd += ["-v", "%s:/work" % self._worktree_for(task),
"-v", "%s:/app:ro" % self._app_dir(),
"-w", "/work"]
# 额外挂载放在基础挂载之后:/app/work 这类子路径挂载会
# 覆盖 /app:ro 的对应子路径,让容器内 host.py 能写
# work/ 和 host-state.json,同时代码主体仍只读;seed 的
# /tmp/codex-seed:ro 同样在此后挂载,压在 tmpfs 之上。
mounts = self.extra_mounts if extra_mounts is None \
else self._parse_extra_mounts(extra_mounts)
for host_p, cont_p, mode in mounts:
cmd += ["-v", "%s:%s:%s" % (host_p, cont_p, mode)]
if self.env_key:
cmd += ["-e", self.env_key]
merged_env = dict(self.extra_env)
@@ -1277,7 +1315,8 @@ class DockerExecutor(Executor):
# 容器也叫 codex-*(例如 codex-postgres),按前缀删会把数据库
# 容器当成残留任务容器杀掉(2026-10-26 实测:worker 重启时
# codex-postgres 就是这样被删掉,导致控制面连不上 Postgres)。
_TASK_CONTAINER_RE = re.compile(r"^codex-[0-9a-f]{32}$")
# 正则收敛到 cloud/sandbox_policy.py(类属性别名保留)。
_TASK_CONTAINER_RE = _POLICY_TASK_RE
def cleanup_stale_containers(self, prefix="codex-"):
"""清理上一轮残留的任务容器:只删 codex-<32 位 hex 任务 id>。
+88
View File
@@ -0,0 +1,88 @@
"""沙箱与清理的单一口径(纯函数层,只依赖标准库)。
收敛前分层口径散在各处注释里,本模块把它钉成可单测
的函数,调用点只传事实、不再各写一套:
1. /tmp 隔离矩阵(Docker 外层 vs 宿主 bwrap 内层):
- Docker 执行器:容器 /tmp 与 /var/tmp 固定为 tmpfs
(sandbox.build_run_command 在额外挂载之后追加
--tmpfs,且 /tmp 目标的额外挂载在解析时即被拒绝),
容器内 /tmp 绝不落在宿主 /tmp。
- 宿主 bwrap / 本地执行器:/tmp 不隔离。宿主级 bwrap
的 /tmp 会落到宿主 /tmp 上,内层 codex 的 bwrap
--proc 挂载亦不受本仓库控制——本模块只如实标注,
绝不宣称“内层沙箱已修”。
- 任务容器名正则(codex-<32 位 hex 任务 id>)此前只
活在 sandbox.cleanup_stale_containers 的注释与正则里;
现收为 TASK_CONTAINER_RE,sandbox 直接复用它。
清理绝不按 "codex-" 前缀删:基础设施容器
(如 codex-postgres)名字不匹配正则,永远不被删。
2. 失联清理判定(cleanup_decision):worker
_reap_kill_requests 的状态与心跳两道门——任务必须
已是 interrupted,且心跳停滞超过阈值(心跳取
COALESCE(last_heartbeat, updated_at),与
server.sweep_stale_tasks 同口径;无心跳时间戳时按
已停滞处理,与 _reap 旧代码的 fall-through 一致;
恰等于阈值时不杀,旧代码是 hb >= now - threshold 即
跳过)。请求 pending、执行器有 kill_container、容器
名由任务 id 派生这三道门仍在 worker 侧,不进本层:
本层只判“该不该杀”,不执行杀。
3. Worker 寻址(worker_address):workers 注册表只有
worker_id 一列身份,默认形为“主机名-进程号”
(worker.py,未给 --worker-id 时)。本函数从该串
派生 (host, pid):取最后一个连字符后全为数字的一段
作 pid,否则 pid 为 None。注册表本身不加 host/pid
列——加列需 PG 迁移与 test_p2_migrations 登记,超出
本轨收敛范围;寻址先用派生函数,不动表结构。
单一状态来源仍是:workers 表(注册与心跳)+
tasks.last_heartbeat(陈旧判定)+ kill_requests
(清理意图),三者缺一不可,不得另起一套状态表。
"""
import re
TMPFS_PATHS = ("/tmp", "/var/tmp")
TASK_CONTAINER_RE = re.compile(r"^codex-[0-9a-f]{32}$")
def tmpfs_isolated(executor_kind, path):
"""某执行器下某路径是否走 tmpfs 隔离:(isolated, reason)。"""
if executor_kind == "docker" and path in TMPFS_PATHS:
return True, "docker --tmpfs 隔离,容器内路径不落宿主"
if executor_kind == "docker":
return False, "docker 下仅 /tmp 与 /var/tmp 走 tmpfs"
if executor_kind in ("host-bwrap", "local"):
return False, "宿主 bwrap / 本地执行器 /tmp 不隔离,会落宿主"
return False, "未知执行器,按不隔离处理(fail-closed 口径)"
def cleanup_decision(status, heartbeat_ts, now, threshold_seconds):
"""失联清理判定:放行回 (True, reason),否则 (False, reason)。
status 必须为 'interrupted';心跳停滞须严格超过阈值
(恰等于阈值不杀);heartbeat_ts 为 None 时按已停滞
处理(与 _reap 旧代码一致:无时间戳不阻拦收殓)。
"""
if status != "interrupted":
return False, "任务不在 interrupted,不杀(防复活任务被旧请求误杀)"
if heartbeat_ts is None:
return True, "无心跳时间戳,按已停滞处理"
try:
age = now - float(heartbeat_ts)
except (TypeError, ValueError):
return False, "心跳时间戳不可解析,不杀"
if age > threshold_seconds:
return True, "心跳停滞 %.0f 秒,超过阈值 %s 秒" % (
age, threshold_seconds)
return False, "owner 可能还活着(心跳 %d 秒前),留待下轮" % age
def worker_address(worker_id):
"""从 worker_id 派生 (host, pid):pid 取末段纯数字,否则 None。"""
if not worker_id or not isinstance(worker_id, str):
return "", None
if "-" in worker_id:
host, _, tail = worker_id.rpartition("-")
if host and tail.isdigit():
return host, int(tail)
return worker_id, None
+177
View File
@@ -0,0 +1,177 @@
"""评分卡冻结版的服务端存储(v7 机制第 5 项:基线漂移门禁)。
定位:只存“冻结的那一版”,不算分、不判 stale、不改旧分。
算分在 cloud/audit_scorecard.py 的 build_scorecard,
漂移判定在 check_baseline_stale,本模块只做冻结行的写入与读取。
键思路沿用仓库 DB 模式:ledger.py / audit_loop_driver.py 按
(module_id, contract_version) 计 slot,本表按
(rubric_version, baseline_head) 计冻结版——同一 rubric 版本在
同一 main 头上冻结的评分卡快照,是一版;main 头推进后冻结的,
是另一版,旧版仍在表里供追溯,不被覆盖。
行字段:
- rubric_version:评分口径版本(调用方传入的字符串,如 "v1";
本模块不规定口径内容、不替用户拍板阈值)。
- baseline_head:冻结时调用方传入的 main 头(字符串,非空;
接受任意字符串原样存、不做格式校验,本模块不验证其真实性,
可信来源仍是调用方责任)。
- frozen_at:冻结时间戳(秒,调用方不传时取当前时间)。
- scorecard_snapshot:build_scorecard 输出的完整 dict,以 JSON
文本存入,读回时还原为 dict。
只追加,不改删:
- 本模块只提供 ensure_schema(建表,幂等)、freeze_scorecard
(INSERT 新行)、get_frozen_scorecard / get_latest_frozen_scorecard /
list_frozen_scorecards(只读)。刻意不提供任何 UPDATE / DELETE
方法;SQLite 下另有触发器在表层挡住 UPDATE / DELETE
(写法沿用 ledger.py 的只追加触发器)。
- 不 commit:调用方在自己的事务边界提交(与 ledger.ensure_schema
同款约定);独立调用方须自己 commit,否则同连接外看不到新行。
- 真实 PG 未接通:PG 适配连接跳过 SQLite 触发器建(方言不同,
与 ledger.py 同款处理),PG 下的只追加护栏与真实 PG 验证均未做,
本项只用 SQLite / 内存库测试并明说。
- 本模块绝不在生产代码里探测版本控制状态、绝不读版本控制目录:
rubric_version / baseline_head / frozen_at 全部由调用方传入。
"""
from __future__ import annotations
import json
import time
_FROZEN_TABLE_SQL = (
"CREATE TABLE IF NOT EXISTS scorecard_baseline_freeze("
"id INTEGER PRIMARY KEY AUTOINCREMENT, "
"rubric_version TEXT NOT NULL, "
"baseline_head TEXT NOT NULL, "
"frozen_at DOUBLE PRECISION NOT NULL, "
"scorecard_snapshot TEXT NOT NULL)"
)
_FROZEN_INDEX_SQL = (
"CREATE INDEX IF NOT EXISTS scorecard_baseline_freeze_key "
"ON scorecard_baseline_freeze(rubric_version, baseline_head)"
)
# SQLite 只追加触发器(PG 不走这段,见模块 docstring)。
_SQLITE_GUARD_SQL = (
"CREATE TRIGGER IF NOT EXISTS scorecard_baseline_freeze_no_update "
"BEFORE UPDATE ON scorecard_baseline_freeze BEGIN "
"SELECT RAISE(ABORT, 'scorecard_baseline_freeze is append-only'); END",
"CREATE TRIGGER IF NOT EXISTS scorecard_baseline_freeze_no_delete "
"BEFORE DELETE ON scorecard_baseline_freeze BEGIN "
"SELECT RAISE(ABORT, 'scorecard_baseline_freeze is append-only'); END",
)
def _is_pg_conn(conn):
"""PgConnection 适配连接(触发器方言不同,须跳过 SQLite 护栏)。"""
return type(conn).__name__ == "PgConnection"
def ensure_schema(conn):
"""建冻结表 + 索引 + SQLite 只追加触发器(幂等,不 commit)。"""
conn.execute(_FROZEN_TABLE_SQL)
conn.execute(_FROZEN_INDEX_SQL)
if not _is_pg_conn(conn):
for stmt in _SQLITE_GUARD_SQL:
conn.execute(stmt)
def _require_non_empty_str(value, name):
if not isinstance(value, str) or value == "":
raise ValueError("%s must be a non-empty string" % name)
return value
def freeze_scorecard(conn, rubric_version, baseline_head,
scorecard_snapshot, frozen_at=None):
"""写入一个新冻结行(INSERT only),返回读回形态的行 dict。
scorecard_snapshot 必须是 build_scorecard 输出的 dict;不是 dict
时拒绝写入,不做任何格式猜测。调用方负责 commit。
"""
_require_non_empty_str(rubric_version, "rubric_version")
_require_non_empty_str(baseline_head, "baseline_head")
if not isinstance(scorecard_snapshot, dict):
raise ValueError("scorecard_snapshot must be a dict")
stamp = float(frozen_at) if frozen_at is not None else time.time()
payload = json.dumps(scorecard_snapshot, sort_keys=True,
ensure_ascii=False)
cur = conn.execute(
"INSERT INTO scorecard_baseline_freeze"
"(rubric_version, baseline_head, frozen_at, scorecard_snapshot) "
"VALUES (?, ?, ?, ?)",
(rubric_version, baseline_head, stamp, payload))
return {
"id": cur.lastrowid,
"rubric_version": rubric_version,
"baseline_head": baseline_head,
"frozen_at": stamp,
"scorecard_snapshot": scorecard_snapshot,
}
def _row_to_dict(row):
row_id, rubric_version, baseline_head, frozen_at, snapshot_text = row
return {
"id": row_id,
"rubric_version": rubric_version,
"baseline_head": baseline_head,
"frozen_at": frozen_at,
"scorecard_snapshot": json.loads(snapshot_text),
}
def get_frozen_scorecard(conn, rubric_version, baseline_head):
"""读某 (rubric_version, baseline_head) 的最新冻结行,无则 None。
同一键有多行时(不同 frozen_at 多次冻结)返回 id 最大的最新行,
旧行仍在表里,不被覆盖、不被删除。
"""
_require_non_empty_str(rubric_version, "rubric_version")
_require_non_empty_str(baseline_head, "baseline_head")
row = conn.execute(
"SELECT id, rubric_version, baseline_head, frozen_at, "
"scorecard_snapshot FROM scorecard_baseline_freeze "
"WHERE rubric_version = ? AND baseline_head = ? "
"ORDER BY id DESC LIMIT 1",
(rubric_version, baseline_head)).fetchone()
return _row_to_dict(row) if row is not None else None
def get_latest_frozen_scorecard(conn, rubric_version=None):
"""读最新冻结行;传 rubric_version 时只在该口径内取最新,无则 None。"""
if rubric_version is None:
row = conn.execute(
"SELECT id, rubric_version, baseline_head, frozen_at, "
"scorecard_snapshot FROM scorecard_baseline_freeze "
"ORDER BY id DESC LIMIT 1").fetchone()
else:
_require_non_empty_str(rubric_version, "rubric_version")
row = conn.execute(
"SELECT id, rubric_version, baseline_head, frozen_at, "
"scorecard_snapshot FROM scorecard_baseline_freeze "
"WHERE rubric_version = ? ORDER BY id DESC LIMIT 1",
(rubric_version,)).fetchone()
return _row_to_dict(row) if row is not None else None
def list_frozen_scorecards(conn, rubric_version=None, baseline_head=None):
"""按 id 升序列出冻结行;两参数均可选作过滤条件,不过滤时全列。"""
sql = ("SELECT id, rubric_version, baseline_head, frozen_at, "
"scorecard_snapshot FROM scorecard_baseline_freeze")
clauses, params = [], []
if rubric_version is not None:
_require_non_empty_str(rubric_version, "rubric_version")
clauses.append("rubric_version = ?")
params.append(rubric_version)
if baseline_head is not None:
_require_non_empty_str(baseline_head, "baseline_head")
clauses.append("baseline_head = ?")
params.append(baseline_head)
if clauses:
sql += " WHERE " + " AND ".join(clauses)
sql += " ORDER BY id ASC"
return [_row_to_dict(row) for row in conn.execute(sql, params).fetchall()]
+456
View File
@@ -0,0 +1,456 @@
"""评分卡差分审计与涨幅停表(v7 机制第 3 项,纯函数 + 标准库 only)。
只做这一项,不碰其它 v7 机制项。本模块不改动
``cloud/audit_loop_driver.py`` 的语义,也不改动
``cloud/audit_scorecard.py``:它是评分卡侧的停表机制,不是
审计循环本身。审计循环的 pushback 上限 3、重建预算 1、
版本变更重置思路只作参考,不在本模块里复写或放宽。
三张表分开记分,绝不只记平均分
--------------------------------
每轮 ledger 条目同时记 ``code`` / ``production`` / ``product``
三张表各自的分数(任一表未知记 ``None``,不估算)。
``average_for_reference_only`` 只作参考信息,停表判定只看
代码表(``code``),绝不用平均分判定——与 audit_scorecard
“平均分绝不能作为达标依据”的规则一致。
阈值未拍板
----------
``STAGNATION_THRESHOLD`` 的值 0.5 标注
``source="test_default_unapproved"``(测试默认值、未拍板),
与 audit_scorecard 的 ``DEFAULT_THRESHOLDS`` 口径一致。
生产阈值须用户拍板后由调用方显式传入 ``threshold`` 才能替换;
本模块绝不把测试初值写成生产值,也不替用户决定 B/C 与阈值。
与已定语义的关系(本模块不改它们)
----------------------------------
- #238 ``evaluate_feedback`` 的 ``auto_apply`` 保持 False;
本模块没有 auto_apply 路径。
- #247 纠正为 pass 本身不解封;本模块不做解封、不做
re-audit、不碰门禁状态,只返回停表判定。
- v1.1 契约不变;本模块不增维、不改 7 维定义。
- 停表触发时返回 ``escalate`` 语义,不再自动推回:
结果里 ``auto_pushback`` 恒为 False,且
``escalation_grants_user_acceptance`` 恒为 False
(escalate 从不 tick 用户验收,与 audit_loop_driver 一致)。
(a) 差分范围输入:``audit_scope(baseline_snapshot, current_snapshot,
claimed_fixed)``
----------------------------------------------------------------
快照形式(二选一,两种都接受):
- 文件路径集合:``set`` / ``list`` / ``tuple``,元素为路径字符串。
此时变更 = 对称差(新增 + 删除);交集 = 未变更。
- 快照 dict:键为路径,值为内容指纹(hash/内容字符串),或
``{"files": <路径集合或路径->指纹 dict>}`` 形式。
此时变更 = 新增 + 删除 + 同路径指纹不同(修改);指纹相同
的路径 = 未变更。
审计范围 = 变更文件 + 调用方声称修好的项(``claimed_fixed``)。
未变更文件进 ``out_of_scope``,不进审计包;基线外(即不在
变更集内、也不在声称修好集内)的文件一律不进包。
``claimed_fixed`` 是项 id(检查名/维 id 等字符串),不是文件
路径;它与文件集分开返回,调用方按项复核,不与文件混算。
(b) 涨幅 ledger:``ScorecardGainLedger``(内存)与
``SqliteScorecardGainLedger``(SQLite)
------------------------------------------------
- 内存 ledger:append-only 列表,``append()`` 存规范化后的
条目副本,``rounds()`` 返回副本(调用方改返回结果不影响
已存条目)。模块级函数 ``append_round`` / ``read_rounds``
接受 ledger 对象或普通 list,方便测试与调用方复用。
- SQLite ledger:同一列名、同一追加语义的 SQLite 表
``scorecard_gain_rounds``(见 ``SQLITE_DDL``),用标准库
``sqlite3``、``?`` 占位符。``PG_DDL`` 是与之列对应的
Postgres 建表语句文本,仅作仓库 DB 模式兼容写法:
真实 PG 在本项中不接通、不执行、不验证,只用
SQLite(内存或文件)测试并明说。
- 条目校验:``rubric_version`` 必须为非空字符串,否则
``ValueError``;三张表分数各为 ``None`` 或 0-10 的数,
超范围或非数值 likewise ``ValueError``;bool 不算数值。
(c) 停表判定:``check_stagnation(history)``
--------------------------------------------
- 只看 trailing 同 rubric_version 序列:从最新一轮往回取
与最新轮 ``rubric_version`` 相同的连续轮次;改标尺即新
序列,旧标尺的轮次不连算。
- 该序列内相邻两轮代码表分数之差为一轮涨幅;分数为 1 位
小数(与 audit_scorecard 的 _round1 一致),涨幅按
1 位小数舍入后再与阈值比较,避免 8.8-8.3 这类二进制
浮点残差把恰好 0.5 误判为 <0.5。
- 涨幅 < 阈值(含负涨幅,即代码表下滑)记为小涨幅;
涨幅 >= 阈值,或涨幅未知(任一端分数为 None),打断
连续计数。连续两轮小涨幅(即该序列内最近两次涨幅
均为小涨幅,至少需要 3 轮)即触发停表。
- 触发返回 ``{"decision": "escalate", "reason": "stagnation", ...}``;
未触发返回 ``{"decision": "continue", ...}``。两种结果都
不含 pushback 动作;触发时 ``auto_pushback`` 为 False。
"""
from __future__ import annotations
import sqlite3
from cloud.db import connect as _db_connect
TABLES = ("code", "production", "product")
# 停表阈值:测试默认值、未拍板。生产阈值须用户拍板后显式传入。
STAGNATION_THRESHOLD = {"value": 0.5, "source": "test_default_unapproved"}
# 触发所需连续小涨幅轮数(2 次小涨幅 = 序列内至少 3 轮)。
STAGNATION_CONSECUTIVE_GAINS = 2
SQLITE_DDL = (
"CREATE TABLE IF NOT EXISTS scorecard_gain_rounds ("
"id INTEGER PRIMARY KEY AUTOINCREMENT, "
"rubric_version TEXT NOT NULL, "
"code REAL, "
"production REAL, "
"product REAL, "
"average_for_reference_only REAL"
")"
)
# 与 SQLITE_DDL 列对应的 Postgres 建表文本。仅作模式兼容写法,
# 本项不接通真实 PG、不执行、不验证;测试只覆盖 SQLite。
PG_DDL = (
"CREATE TABLE IF NOT EXISTS scorecard_gain_rounds ("
"id SERIAL PRIMARY KEY, "
"rubric_version TEXT NOT NULL, "
"code DOUBLE PRECISION, "
"production DOUBLE PRECISION, "
"product DOUBLE PRECISION, "
"average_for_reference_only DOUBLE PRECISION"
")"
)
# ------------------------------------------------------------------
# (a) 差分范围输入
# ------------------------------------------------------------------
def _fingerprint(value):
"""快照 dict 值的规范指纹:hash/sha256/content 字段优先,否则原值。"""
if isinstance(value, dict):
for key in ("sha256", "hash", "content"):
if value.get(key) is not None:
return value[key]
return repr(sorted(value.items()))
return value
def _snapshot_mapping(snapshot):
"""把快照归一化为 {路径: 指纹或 None}。
路径集合形式:指纹为 None(只比存在性)。
快照 dict 形式:键为路径,值为指纹;{"files": ...} 形式先解包。
None 快照视为空快照。
"""
if snapshot is None:
return {}
if isinstance(snapshot, dict):
inner = snapshot.get("files", snapshot) if "files" in snapshot else snapshot
if isinstance(inner, dict):
return {str(path): _fingerprint(value) for path, value in inner.items()}
if isinstance(inner, (set, list, tuple)):
return {str(path): None for path in inner}
raise TypeError("snapshot files must be a mapping or a path collection")
if isinstance(snapshot, (set, list, tuple)):
return {str(path): None for path in snapshot}
raise TypeError("snapshot must be a path collection, a mapping, or None")
def _normalize_claimed(claimed_fixed):
"""声称修好的项:None -> [];单个字符串 -> [它];否则去重排序。"""
if claimed_fixed is None:
return []
if isinstance(claimed_fixed, str):
return [claimed_fixed]
return sorted({str(item) for item in claimed_fixed})
def audit_scope(baseline_snapshot, current_snapshot, claimed_fixed=None):
"""算出差分审计范围。
返回 {"in_scope": {"files": [...], "claimed_fixed": [...]},
"out_of_scope": {"files": [...]}}。
in_scope.files = 变更文件(新增 + 删除 + 修改);
out_of_scope.files = 两快照中都出现且指纹相同(路径集合
形式下即交集)的未变更文件;claimed_fixed 原样进包,
与文件集分开,不因文件未变更而被排除。
"""
baseline = _snapshot_mapping(baseline_snapshot)
current = _snapshot_mapping(current_snapshot)
# 以存在性为准:只在一侧出现的路径必为变更;路径集合形式下
# 指纹均为 None,不能只比指纹(缺失侧 get 缺省也是 None,
# 会把新增/删除误判为未变更),故先判存在性再比指纹。
changed = sorted(
path
for path in set(baseline) | set(current)
if path not in baseline or path not in current or baseline[path] != current[path]
)
unchanged = sorted(
path
for path in set(baseline) & set(current)
if baseline[path] == current[path]
)
return {
"in_scope": {
"files": changed,
"claimed_fixed": _normalize_claimed(claimed_fixed),
},
"out_of_scope": {"files": unchanged},
}
# ------------------------------------------------------------------
# (b) 涨幅 ledger:条目规范化
# ------------------------------------------------------------------
def _is_number(value):
return isinstance(value, (int, float)) and not isinstance(value, bool)
def normalize_round(rubric_version, code=None, production=None, product=None, entry=None):
"""规范化一轮 ledger 条目(返回新 dict,不修改入参)。
接受两种调用:显式三表分数,或 entry dict(含 flat 的
code/production/product,或嵌套的 scores dict)。
平均分只在三表全已知时算出,且只作参考,永不用于停表。
"""
if entry is not None:
if not isinstance(entry, dict):
raise TypeError("entry must be a dict")
scores = entry.get("scores") if isinstance(entry.get("scores"), dict) else entry
rubric_version = entry.get("rubric_version", rubric_version)
code = scores.get("code", code) if code is None else code
production = scores.get("production", production) if production is None else production
product = scores.get("product", product) if product is None else product
if not isinstance(rubric_version, str) or not rubric_version:
raise ValueError("rubric_version must be a non-empty string")
values = {}
for name, value in (("code", code), ("production", production), ("product", product)):
if value is None:
values[name] = None
elif _is_number(value) and 0 <= float(value) <= 10:
values[name] = round(float(value) + 1e-9, 1)
else:
raise ValueError("%s must be None or a number in 0..10, got %r" % (name, value))
known = [values[name] for name in TABLES]
average = round(sum(known) / len(known) + 1e-9, 1) if all(v is not None for v in known) else None
return {
"rubric_version": rubric_version,
"code": values["code"],
"production": values["production"],
"product": values["product"],
"average_for_reference_only": average,
"average_is_not_pass_criterion": True,
}
class ScorecardGainLedger:
"""内存涨幅 ledger:append-only,存条目副本,读出副本。"""
def __init__(self):
self._rounds = []
def append(self, rubric_version=None, code=None, production=None, product=None, entry=None):
normalized = normalize_round(rubric_version, code, production, product, entry)
self._rounds.append(normalized)
return dict(normalized)
def rounds(self):
return [dict(entry) for entry in self._rounds]
# history() 是 rounds() 的别名,供 check_stagnation 直接接受 ledger。
def history(self):
return self.rounds()
def append_round(ledger, rubric_version=None, code=None, production=None, product=None, entry=None):
"""向 ledger 追加一轮:ledger 对象(有 .append)或普通 list 均可。"""
normalized = normalize_round(rubric_version, code, production, product, entry)
if hasattr(ledger, "append") and not isinstance(ledger, list):
# ledger 对象自身的 append 已做规范化;为避免双重规范化
# 带来的舍入差异,这里直接传规范化后的 entry。
ledger.append(entry=normalized)
return dict(normalized)
ledger.append(normalized)
return dict(normalized)
def read_rounds(ledger):
"""读出 ledger 全部轮次副本:ledger 对象(有 .rounds)或 list 均可。"""
if hasattr(ledger, "rounds"):
return [dict(entry) for entry in ledger.rounds()]
return [dict(entry) for entry in ledger]
class SqliteScorecardGainLedger:
"""SQLite 涨幅 ledger:与内存 ledger 同列、同追加语义。
db_path 为 ":memory:"(默认)或文件路径;传入 sqlite3
连接亦可(此时 close() 不关闭调用方连接)。真实 PG
不接通:PG_DDL 只是模式兼容文本,不在本类中执行。
"""
def __init__(self, db_path=":memory:", connection=None):
if connection is not None:
self._conn = connection
self._owns_conn = False
else:
# 层规则:服务层不得直连驱动,须走 cloud.db.connect
# (其 SQLite 路径已设 WAL + busy_timeout + Row)。
self._conn = _db_connect(str(db_path))
self._owns_conn = True
self._conn.execute(SQLITE_DDL)
self._conn.commit()
def append(self, rubric_version=None, code=None, production=None, product=None, entry=None):
normalized = normalize_round(rubric_version, code, production, product, entry)
self._conn.execute(
"INSERT INTO scorecard_gain_rounds"
" (rubric_version, code, production, product, average_for_reference_only)"
" VALUES (?, ?, ?, ?, ?)",
(normalized["rubric_version"], normalized["code"], normalized["production"],
normalized["product"], normalized["average_for_reference_only"]),
)
self._conn.commit()
return dict(normalized)
def rounds(self):
rows = self._conn.execute(
"SELECT rubric_version, code, production, product,"
" average_for_reference_only FROM scorecard_gain_rounds ORDER BY id"
).fetchall()
out = []
for row in rows:
mapping = dict(row) if isinstance(row, sqlite3.Row) else dict(row)
entry = {
"rubric_version": mapping["rubric_version"],
"code": mapping["code"],
"production": mapping["production"],
"product": mapping["product"],
"average_for_reference_only": mapping["average_for_reference_only"],
"average_is_not_pass_criterion": True,
}
out.append(entry)
return out
def history(self):
return self.rounds()
def close(self):
if self._owns_conn:
self._conn.close()
# ------------------------------------------------------------------
# (c) 停表判定
# ------------------------------------------------------------------
def _history_entries(history):
"""接受 list/tuple 或 ledger 对象(有 .rounds/.history),返回条目 list。"""
if history is None:
return []
if hasattr(history, "rounds"):
return list(history.rounds())
if hasattr(history, "history"):
return list(history.history())
return list(history)
def _code_score(entry):
"""取一轮的代码表分数:flat 的 code,或嵌套 scores.code;缺失为 None。"""
if not isinstance(entry, dict):
return None
if entry.get("code") is not None:
return entry["code"]
scores = entry.get("scores")
if isinstance(scores, dict):
return scores.get("code")
return None
def _trailing_run(entries):
"""最新 rubric_version 的 trailing 连续序列(改标尺即新序列)。"""
if not entries:
return []
latest_version = entries[-1].get("rubric_version") if isinstance(entries[-1], dict) else None
run = []
for entry in reversed(entries):
version = entry.get("rubric_version") if isinstance(entry, dict) else None
if version != latest_version:
break
run.append(entry)
run.reverse()
return run
def code_gains(history):
"""trailing 同标尺序列内相邻两轮的代码表涨幅 list。
每项为 1 位小数舍入后的涨幅,或 None(任一端分数未知)。
全序列的涨幅不返回:旧标尺轮次不连算。
"""
run = _trailing_run(_history_entries(history))
scores = [_code_score(entry) for entry in run]
gains = []
for prev_score, cur_score in zip(scores, scores[1:]):
if prev_score is None or cur_score is None:
gains.append(None)
elif not _is_number(prev_score) or not _is_number(cur_score):
gains.append(None)
else:
gains.append(round(float(cur_score) - float(prev_score) + 1e-9, 1))
return gains
def _resolve_threshold(threshold):
"""阈值解析:None -> 测试默认值;数字 -> 该值(来源仍标测试默认,
除非调用方显式传 {"value":..., "source": "user_approved"});
dict -> 取 value/source。"""
if threshold is None:
return STAGNATION_THRESHOLD["value"], STAGNATION_THRESHOLD["source"]
if isinstance(threshold, dict):
return threshold.get("value", STAGNATION_THRESHOLD["value"]), threshold.get(
"source", STAGNATION_THRESHOLD["source"])
return threshold, STAGNATION_THRESHOLD["source"]
def check_stagnation(history, threshold=None):
"""停表判定:trailing 同标尺序列内连续两轮代码表涨幅 < 阈值即停。
触发返回 decision="escalate"、reason="stagnation",且
auto_pushback=False(不再自动推回);
未触发返回 decision="continue"。两种结果的
escalation_grants_user_acceptance 均为 False。
"""
threshold_value, threshold_source = _resolve_threshold(threshold)
entries = _history_entries(history)
run = _trailing_run(entries)
gains = code_gains(entries)
consecutive = 0
for gain in reversed(gains):
if gain is not None and gain < threshold_value:
consecutive += 1
else:
break
latest_version = run[-1].get("rubric_version") if run else None
base = {
"rubric_version": latest_version,
"run_rounds": len(run),
"code_gains": gains,
"consecutive_small_gains": consecutive,
"threshold": threshold_value,
"threshold_source": threshold_source,
"auto_pushback": False,
"escalation_grants_user_acceptance": False,
}
if consecutive >= STAGNATION_CONSECUTIVE_GAINS:
result = {"decision": "escalate", "reason": "stagnation"}
result.update(base)
return result
result = {"decision": "continue", "reason": "not_stagnant"}
result.update(base)
return result
+282
View File
@@ -0,0 +1,282 @@
"""评分卡标尺版本化:仅追加的持久存储(v7 机制第 1 项)。
问题:cloud/audit_scorecard.py 的 DEFAULT_THRESHOLDS 只存在于
代码里,标尺一改,旧标尺下的分数就再也算不出来、也对不上
——审计员每次重审都在新标尺下重算旧分,基线漂移就是这么来的。
解法(复用 audit_loop_driver 的键思路,不复用它的表):
- audit_loop_driver 以 (module_id, contract_version) 为键,
合同版本一变,计数与预算全部重置、旧版本状态原地不动。
这里同理以 rubric_version 为键:标尺改动 = 写一个新版本行,
旧版本行原地不动;同一证据在旧版本下重算,分数逐字相同。
- 不复用 audit_loop_state 表:那张表存的是循环状态
(pushback 轮次、失败计数),标尺不是循环状态,混表会让
两类数据的生命周期互相牵连。故新开两张表(见下)。
两张表(SQLite / Postgres 兼容写法,沿用 audit_loop_wiring 的
conn + ``?`` 占位符 + CREATE TABLE IF NOT EXISTS 模式;类型只用
TEXT / REAL,不用任何一方专属类型,不用自增列):
- scorecard_rubric_versions:标尺版本行,主键 rubric_version。
列:rubric_version、thresholds_snapshot(归一化阈值表的
canonical JSON)、baseline_head(该版本锚定的基线提交,
调用方显式传入,本模块不跑 git、不猜 HEAD)、
thresholds_source(user_approved 仅当全部条目都已拍板,
其余一律 test_default_unapproved)、created_at。
- scorecard_snapshots:已算出的卡片行,主键
(rubric_version, evidence_sha256)。列除主键外还有
baseline_head 与 thresholds_snapshot(从版本行反规范化而来,
让单行快照自带审计三要素,不必联表也能核对)、
scorecard_json(build_scorecard 的完整输出)、created_at。
选型理由:只存版本行 + 现算也能复现旧分(scorer 是纯函数),
但快照行让“旧行分数不变”成为可直接读回的事实而非“重算
碰巧一致”的推论;快照不存证据原文,只存证据的 SHA-256,
避免证据体在库里再留一份副本。
仅追加(append-only),这是本模块的核心承诺:
- 对外 API 只有 create / get / list / latest / score / record /
get_snapshot,没有任何改旧行、删旧行的方法;重复建同一
版本号直接报 ValueError(主键冲突转译,不做 upsert)。
- 同一 (rubric_version, evidence) 重复 record 时返回已存行的
卡片原样,不覆盖、不合并。
- 本模块防的是“走正常 API 的误写”(与 ledger 的触发器同款
诚实边界):能直接拿连接跑 SQL 的人当然能改库,库内机制
拦不住超级用户;可信锚点仍须落在库外(如导出的行哈希)。
事务边界:沿用 audit_loop_wiring 的做法——本模块的写函数
在写后 commit;调用方若想把多写包进同一事务,应直接用
底层的 INSERT 语句而非本模块的写函数。读函数不 commit。
诚实边界:
- 阈值本身仍不由本模块拍板:thresholds_source 的判定与
audit_scorecard 一致,测试默认值永远标
test_default_unapproved;user_approved 只能由调用方显式
传入全部已拍板条目。
- 未在真实 Postgres 上验证:SQL 只用两端通用的写法,
并以 SQLite(文件库 + :memory:)与假连接(只录语句)
测试;PG 接通后须以真实 PG 复测一次才能宣称两端可用。
- 本模块不改 #238 auto_apply(保持 False)、不改 #247
(纠正为 pass 本身不解封)、不改 v1.1 契约与 7 维定义。
"""
from __future__ import annotations
import hashlib
import json
import time
from cloud.audit_scorecard import build_scorecard, thresholds_source
_RUBRIC_TABLE_SQL = (
"CREATE TABLE IF NOT EXISTS scorecard_rubric_versions("
"rubric_version TEXT PRIMARY KEY, "
"thresholds_snapshot TEXT NOT NULL, "
"baseline_head TEXT NOT NULL, "
"thresholds_source TEXT NOT NULL, "
"created_at REAL NOT NULL)"
)
_SNAPSHOT_TABLE_SQL = (
"CREATE TABLE IF NOT EXISTS scorecard_snapshots("
"rubric_version TEXT NOT NULL, "
"baseline_head TEXT NOT NULL, "
"evidence_sha256 TEXT NOT NULL, "
"thresholds_snapshot TEXT NOT NULL, "
"scorecard_json TEXT NOT NULL, "
"created_at REAL NOT NULL, "
"PRIMARY KEY(rubric_version, evidence_sha256))"
)
def ensure_rubric_tables(conn):
"""建两张表(幂等)。写法与 audit_loop_wiring 保持一致。"""
conn.execute(_RUBRIC_TABLE_SQL)
conn.execute(_SNAPSHOT_TABLE_SQL)
try:
conn.commit()
except Exception:
pass
def _canonical_json(value):
return json.dumps(value, sort_keys=True, separators=(",", ":"))
def _evidence_sha256(evidence):
return hashlib.sha256(
_canonical_json(evidence).encode("utf-8")).hexdigest()
def _normalize_thresholds(thresholds):
"""阈值表归一化:每项 -> {"value": 数值, "source": 来源}。
纯数值条目按 audit_scorecard 的旧规则视作未拍板;
dict 条目缺 source 时同样视作未拍板,绝不默认已拍板。
"""
if not isinstance(thresholds, dict) or not thresholds:
raise ValueError("thresholds must be a non-empty dict")
normalized = {}
for key, entry in thresholds.items():
if isinstance(entry, dict):
if "value" not in entry:
raise ValueError(
"threshold %r must carry a value" % (key,))
value = entry["value"]
source = entry.get("source", "test_default_unapproved")
else:
value = entry
source = "test_default_unapproved"
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise ValueError(
"threshold %r value must be a number" % (key,))
if not isinstance(source, str) or not source:
raise ValueError(
"threshold %r source must be a non-empty string" % (key,))
normalized[key] = {"value": value, "source": source}
return normalized
def _row_to_version(row):
return {
"rubric_version": row[0],
"thresholds_snapshot": row[1],
"thresholds": json.loads(row[1]),
"baseline_head": row[2],
"thresholds_source": row[3],
"created_at": row[4],
}
def create_rubric_version(conn, rubric_version, thresholds, baseline_head,
created_at=None):
"""写一个新标尺版本行。版本号已存在时报 ValueError,旧行不动。
thresholds:audit_scorecard 阈值表形态(dict 条目或纯数值)。
baseline_head:调用方显式传入的基线提交标识,不可为空。
"""
if not isinstance(rubric_version, str) or not rubric_version:
raise ValueError("rubric_version must be a non-empty string")
if not isinstance(baseline_head, str) or not baseline_head:
raise ValueError("baseline_head must be a non-empty string")
normalized = _normalize_thresholds(thresholds)
snapshot = _canonical_json(normalized)
source = thresholds_source(normalized)
stamp = time.time() if created_at is None else created_at
ensure_rubric_tables(conn)
try:
conn.execute(
"INSERT INTO scorecard_rubric_versions"
"(rubric_version, thresholds_snapshot, baseline_head,"
" thresholds_source, created_at) VALUES (?, ?, ?, ?, ?)",
(rubric_version, snapshot, baseline_head, source, stamp))
except Exception as exc:
# 只有主键冲突才转译为“版本已存在”;其它错误原样抛出,
# 绝不能把真实写失败伪装成版本冲突、更不能转成 upsert。
text = str(exc).lower()
if "unique" in text or "duplicate" in text:
raise ValueError(
"rubric_version %r already exists; create a new version"
" instead of modifying it" % (rubric_version,)) from exc
raise
conn.commit()
return {
"rubric_version": rubric_version,
"thresholds_snapshot": snapshot,
"thresholds": normalized,
"baseline_head": baseline_head,
"thresholds_source": source,
"created_at": stamp,
}
def get_rubric_version(conn, rubric_version):
"""读一个版本行;不存在返回 None(不抛错、不造默认值)。"""
ensure_rubric_tables(conn)
row = conn.execute(
"SELECT rubric_version, thresholds_snapshot, baseline_head,"
" thresholds_source, created_at FROM scorecard_rubric_versions"
" WHERE rubric_version = ?",
(rubric_version,)).fetchone()
if row is None:
return None
return _row_to_version(row)
def list_rubric_versions(conn):
"""全部版本行,按 (created_at, rubric_version) 升序(写序)。"""
ensure_rubric_tables(conn)
rows = conn.execute(
"SELECT rubric_version, thresholds_snapshot, baseline_head,"
" thresholds_source, created_at FROM scorecard_rubric_versions"
" ORDER BY created_at ASC, rubric_version ASC").fetchall()
return [_row_to_version(row) for row in rows]
def latest_rubric_version(conn):
"""最新版本行(created_at 最大,时间并列时版本号最大);无行返回 None。"""
ensure_rubric_tables(conn)
row = conn.execute(
"SELECT rubric_version, thresholds_snapshot, baseline_head,"
" thresholds_source, created_at FROM scorecard_rubric_versions"
" ORDER BY created_at DESC, rubric_version DESC LIMIT 1").fetchone()
if row is None:
return None
return _row_to_version(row)
def scorecard_for_rubric(conn, rubric_version, evidence):
"""用指定版本行的快照阈值现算一张卡(只读,不写快照表)。
未知版本号报 ValueError,不回落到默认值——回落会让旧分
在不知情下变成新标尺的分,正是版本化要杜绝的漂移。
"""
version = get_rubric_version(conn, rubric_version)
if version is None:
raise ValueError("unknown rubric_version %r" % (rubric_version,))
card = build_scorecard(evidence, thresholds=version["thresholds"],
rubric_version=rubric_version)
card["baseline_head"] = version["baseline_head"]
return card
def record_scorecard(conn, rubric_version, evidence, created_at=None):
"""算卡并写入快照行;同 (版本, 证据) 已有行时返回旧行原样。
返回值恒为卡片 dict(新算的或旧存的),调用方凭它核对。
"""
version = get_rubric_version(conn, rubric_version)
if version is None:
raise ValueError("unknown rubric_version %r" % (rubric_version,))
card = build_scorecard(evidence, thresholds=version["thresholds"],
rubric_version=rubric_version)
card["baseline_head"] = version["baseline_head"]
digest = _evidence_sha256(evidence)
ensure_rubric_tables(conn)
row = conn.execute(
"SELECT scorecard_json FROM scorecard_snapshots"
" WHERE rubric_version = ? AND evidence_sha256 = ?",
(rubric_version, digest)).fetchone()
if row is not None:
return json.loads(row[0])
conn.execute(
"INSERT INTO scorecard_snapshots"
"(rubric_version, baseline_head, evidence_sha256,"
" thresholds_snapshot, scorecard_json, created_at)"
" VALUES (?, ?, ?, ?, ?, ?)",
(rubric_version, version["baseline_head"], digest,
version["thresholds_snapshot"], _canonical_json(card),
time.time() if created_at is None else created_at))
conn.commit()
return card
def get_scorecard_snapshot(conn, rubric_version, evidence):
"""读一张已存快照卡;无行返回 None(不现算、不回落)。"""
digest = _evidence_sha256(evidence)
ensure_rubric_tables(conn)
row = conn.execute(
"SELECT scorecard_json FROM scorecard_snapshots"
" WHERE rubric_version = ? AND evidence_sha256 = ?",
(rubric_version, digest)).fetchone()
if row is None:
return None
return json.loads(row[0])
+2111 -3331
View File
File diff suppressed because it is too large. Load diff
+36 -1
View File
@@ -40,9 +40,44 @@ passes, do not claim the skills are live in production tasks --
this helper only guarantees the files land in the tenant home,
idempotently, without clobbering tenant edits, and that
skills/list sees them there.
Whole-directory seeding evaluation (2026-10-30, conclusion: NOT DONE):
copying a whole skill directory instead of only SKILL.md was evaluated
and rejected, so the behaviour above is unchanged. A per-file
skip-if-differ rule would preserve a tenant-modified file's content,
but it cannot preserve a tenant's *deletion* (a missing destination
file is recopied, resurrecting a file the tenant removed on purpose)
and it would auto-seed every future pack file into tenant homes
without review -- new surface, no benefit today, since every pack
directory currently contains only SKILL.md. The hard line stands:
a tenant-modified file is never overwritten, and whole-directory
copying is not adopted unless a future evaluation proves both
content-preservation AND deletion-preservation. Until then, only
SKILL.md is seeded; extra files in a pack directory are ignored
(pinned by test_merge_gate_protectskill.py).
Evaluation note (2026-10-31, skill-engineering convergence line;
comment only, behaviour unchanged -- see
cloud/tests/test_skill_seed_pinning.py, which pins it):
whole-directory copy is explicitly NOT done, for three reasons.
(1) The pack today ships exactly one file per skill (SKILL.md;
verified: cloud/skills/*/ contains nothing else), so copying a
whole directory has zero benefit today and would silently start
copying future extras (templates/, README) without review.
(2) Per-file skip-if-differ cannot preserve a tenant deletion:
a missing destination is copied by design, so whole-dir copy
would resurrect tenant-deleted auxiliary files on the next run.
SKILL.md-only copy bounds that resurrection surface to the single
SKILL.md file itself (auxiliary deletions are preserved trivially
because those files never landed in the tenant home).
(3) Seeding never prunes the tenant directory, so tenant-added
files in a skill directory are left alone by the same rule.
Boundary, stated plainly so it is not misread: a tenant-deleted
SKILL.md IS re-seeded (missing = copied); only the differ-guard
preserves edits. Preserving a SKILL.md deletion would need a
tombstone, which is new behaviour, not a convergence fix.
"""
import os
import shutil
from pathlib import Path
@@ -0,0 +1,122 @@
# software-engineering 技能 CHANGELOG
## 说明
本文件新建于 2026-10-31(附录溯源一轨)。此前 SKILL.md 正文(v0.2,
2026-10-30)已引用同目录 `CHANGELOG.md`,但包内并无此文件,因此
v0.1 → v0.2 的 5 个 P0 缺口明细无法从本文件追溯,本文件不编造
那段历史;以下只记录 2026-10-31 起可核实、可复查的条目。
## 2026-10-31:附录溯源补齐(SKILL.md 未动,零语义改动)
此前缺失:SKILL.md 称附录“由生成脚本从 `templates/roles/` 原样拼接,
见 CHANGELOG”,但包内既无 `templates/`、也无 `CHANGELOG.md`、更无
生成脚本,附录 9 章内容不可验——附录与源不一致时无从发现。
本次补齐:
- 新增 `templates/roles/` 下 9 个编辑源文件(`00-coordinator.md` 至
`08-shared-core.md`),内容由现行 v0.2 附录逐章机械拆出:每章源文件
即该章标题行之后的正文(含 00/04/07 章后的 GitHub 小节),仅剥离
章节标题行、章节间分隔符行与首尾空行。SKILL.md 本文件未作任何
改动,因此不存在语义改动;源与附录的对应关系由校验脚本钉住,
今后任一侧改动都会先被校验拦住,而不是悄悄漂移。
- 新增 `tools/check_appendix.py`(仅标准库,不改 SKILL.md):默认
校验模式逐章比对源与附录,源缺失、章节缺失、内容不一致、多余
源文件、CHANGELOG 缺失或未提及 `templates/roles/` 均以非零退出
失败;`--emit` 模式只把拼接后的附录规范形式打印到标准输出,供
编辑者起草参考,绝不是可直接覆盖 SKILL.md 的字节补丁(分隔符与
空行见脚本注释的允许差异清单)。
- 新增 `cloud/tests/test_skill_appendix_provenance.py`:钉住源与附录
一致、源缺失即失败、脚本 `--emit` 输出与源一致。
播种行为不变:`cloud/skill_seed.py` 只复制各技能的 `SKILL.md` 一个
文件,`templates/`、`tools/`、`CHANGELOG.md` 均不进入租户目录(该行
为已由 `test_skill_seed_pinning.py` 钉住,见其“仅复制 SKILL.md”一
项)。因此源文件进技能包不改变租户可见内容:租户任务内仍只能
看到 SKILL.md 随附的附录全文,附录仍是租户侧的权威版本。
## 已知边界(未改行为,另起设计时再动)
- 租户已删 `SKILL.md` 会在下次播种时重建(missing 即复制),这是
现有播种语义,不是删除保留;要保留删除需 tombstone 设计,须另
起一轨,不在本条目悄悄改播种行为。
- 校验的允许差异只有三项:章节标题行本身、至多一枚章节间分隔符
行、标题与章末的空行。现行 SKILL.md 的分隔符并不统一(01/02/03/
05/06 章后有 `---`,00/04/07 章后无分隔符直接接下一章标题),校验
保留该现状、不强行统一 SKILL.md。
## 未动项
- SKILL.md 同样引用的同目录 `README.md`(装机路径)未在本条目处理,
仍缺失;本条目只补附录溯源,不扩大范围。
## 集成基底 v0.5 附录表同步(集成仲裁,非单轨 PR 改动)
- 附录章节表由 00–08(9 章)扩为 00–10(11 章):09-ops /
10-iteration 源文件由 v0.5 附录章节机械拆出(与 #217 同款方法,
章节正文逐字取自附录 canonical 形);09/10 章节标题的“,v0.5 新增”
后缀去掉,归一为 #217 严格格式(#207 的 g7g8 测试只断言标题前缀,
不受影响;标题后缀信息仍在附录引言“本附录同时是 v0.5 版的一部分”)。
- 00–08 源文件同步为 v0.5 附录现状:v0.3/v0.4/v0.5 把正文改动同步进
附录章节后,源文件仍停在 v0.2 版,校验在 00/02/04/06/07/08 共 6 处
失配;方向定为源跟随附录(附录是租户侧权威版本,且任务规则禁止
以 v0.2 文本覆盖 v0.5),不是附录退回 v0.2。分隔符现状同步为
01/02/03/05/06/09 章后有 `---`。
- 校验脚本与两份附录测试的 EXPECTED_IDS 同步扩为 11 章;#214 的
“仅复制 SKILL.md”结论不受影响(播种行为未改)。
## #244 审计段落向 v0.5 的移植(集成时做,移植准备轨 #258 的 §1–§3 为准)
- A 矩阵行:插入 QA 行之后、验收行之前(v0.5 矩阵行本身不动)。
- B 主节:插在“失败、续跑与成本”节末、“诚实边界”节首之前;
引言移植警告按 R1 改写为移植注;“与已有回路的关系”按 v0.5
实际回路对齐(G7、全自动在本文件已有定义,注意句相应改写,
全自动下执行人字段口径见集成摘要:full-auto-executor +
installation id,签字人与执行人仍须不同);B2、B3 全并,
唯收紧三末段“验收失败导致的推倒要先通知用户”整段停下
(全自动四类回头问用户未含此情形,须用户拍板 (a)/(b),移植轨
不擅自选);B4 QA 全链路整段停下(跨端版与 v0.5 跨视口口径正面
冲突,须视口版重写后再移植);C 只插审计项,QA 项随 B4 停下。
- D 附录章:落为 11-auditor(v0.5 已有 09-ops / 10-iteration,
编号顺延),标题用严格格式,源文件为
`templates/roles/11-auditor.md`(正文与 #258 的 09-auditor 同源);
校验表同步扩为 12 章。§4 的 templates/artifacts 删引用决策
未在集成基底执行(仍待用户拍板),SKILL.md 内 6 处引用原样保留。
## 集成注(code-to10-integration,#258 段)
以下两节原样引自 #258 提交的 CHANGELOG 条目,记录该提交本身的工作;
其中“09-auditor”源文件在本基底的章节表里编号为 11-auditor(v0.5 表
00–08 + 09-ops + 10-iteration + 11-auditor),内容与 11-auditor 源文件
逐字节一致,故 #258 的 09-auditor.md 未重复落盘;#244 向 v0.5 的实际
移植已在本基底的 #244 段完成,与该提交“未移植”的自述不冲突。
## 2026-10-31:#244 向 v0.3 / v0.4 / v0.5 的移植准备(未移植)
- 新增 `TRANSPLANT-PREP-244.md`:#244 新增段落(+459 行)
向 v0.3、v0.4、v0.5 三份文本的逐段移植 diff 准备、
锚点行号、合并 / 移植注 / 停下三档分级、章节号对应表、
templates/artifacts 二选一决策、草稿同步清单。
本轨不改 v0.3 / v0.4 / v0.5 文本,不声称已移植。
- 新增 `tools/check_transplant.py`:移植检查脚本
(锚点检查、草稿移植、移植后检查三模式,见该文件说明)。
- templates/artifacts 决策(本轨定为删引用,见准备文档):
SKILL.md 正文 6 处 `templates/artifacts/` 引用在移植时删除
或改写为正文内联结构,不补模板文件进包、不改播种规则。
本轨不改 SKILL.md 正文,故 6 处引用在本轨包内仍在,
移植检查脚本的移植后检查会把残留引用报为未完成。
## 已知边界(未改行为,另起设计时再动)
- 租户已删 `SKILL.md` 会在下次播种时重建(missing 即复制),
这是现有播种语义,不是删除保留;要保留删除需 tombstone
设计,须另起一轨,不在本条目悄悄改播种行为。
- 校验的允许差异只有三项:章节标题行本身、至多一枚章节间
分隔符行、标题与章末的空行。现行 SKILL.md 的分隔符并不
统一(01/02/03/05/06/08 章后有 `---`,00/04/07 章后无分隔符
直接接下一章标题),校验保留该现状、不强行统一 SKILL.md。
- #244 文本里的“第 1 轨硬门禁”“第 4 轨账本”指审计加固
轨内部的轨号,不是本批“各项指标 9 分以上”的第 1–4 项;
本批第 2 项 attempts-hook、第 3 项 shell 白名单与之无关,
不得混淆(见准备文档)。
+134
View File
@@ -0,0 +1,134 @@
# software-engineering 技能包说明
本文件是源码包的导航与装机说明,不是技能规则本身。
规则只在 `SKILL.md` 里,改规则只改 `SKILL.md`;本文件不复述
门禁、角色细则与校验规则,复述就会漂移。
版本与状态以 `SKILL.md` 文首为准:v0.2,2026-10-30,草稿;
其文首自述未提 PR、未部署生产、未改任何生产代码。
本文件只补说明,不改规则、不合并、不部署。
## 用途
面向“从零做一款软件、尤其是要同时做 web、macOS、Windows、
iOS、Android 多个客户端”的任务:按产品规划、产品设计、架构契约、
多客户端并行开发、联调测试、验收的顺序,用协调员任务派生角色线程
推进,契约冻结后各客户端只在自己目录内开发,最后走评审合并与用户验收。
用与不用的判定只在 `SKILL.md` 的“何时用 / 何时不用”两节,
本文件不另立标准。只做单文件小改动、只问概念时不用本技能,
这是 `SKILL.md` 的原话指向,不是本文件的补充规则。
## 包内文件一览
源码位置是 `cloud/skills/software-engineering/`,包内文件如下:
- `SKILL.md`:唯一规则权威,含正文与文末附录随附版。
- `README.md`:本文件,导航与装机说明,不进入租户。
- `CHANGELOG.md`:变更时间线,见下文“与 CHANGELOG 的分工”。
- `templates/roles/`:附录的 9 个编辑源,见下文“附录溯源”。
- `tools/check_appendix.py`:附录校验脚本,只校验,不改 `SKILL.md`。
## SKILL.md 章节结构(只列名,不复述规则)
正文节:
- 何时用 / 何时不用
- 核心原则(协调员和所有角色线程都必须遵守)
- 阶段 DAG(不是所有线程同时开,是按门禁、分阶段开)
- 门禁表
- 接单选型:协调员第一件事是问,不是写代码
- 角色线程矩阵
- 协调员派生子任务的必填字段
- 待办机制(不依赖提示词注入)
- 契约冻结与变更协议
- 目录与分支规则
- GitHub 保证(每线程一分支、一 PR、一合并门),下含分支、
PR 交付、合并门、合并方式与合并记录、发布与回滚、
安全六个小节
- 测试与验收
- 失败、续跑与成本
- 诚实边界:原生客户端
- 校验(协调员每过一个门禁自查一次)
附录节:
- 附录:角色模板(播种随附版),下含 9 章附录角色,
每章标题注明其来源文件,见下文“附录溯源”。
## 附录溯源
附录 9 章的编辑源在 `templates/roles/` 下,文件与章一一对应:
`00-coordinator.md`、`01-product-manager.md`、`02-architect.md`、
`03-designer.md`、`04-client-developer.md`、`05-backend.md`、
`06-qa.md`、`07-acceptance.md`、`08-shared-core.md`。
校验入口(仓库根目录执行):
```sh
python3 cloud/skills/software-engineering/tools/check_appendix.py
```
该脚本默认校验模式逐章比对源与附录;`--emit` 模式只把拼接后
的附录规范形式打印到标准输出,供编辑者起草参考,绝不是可直接
覆盖 `SKILL.md` 的字节补丁。校验的规范形式、允许差异与失败条件
以脚本注释为准,本文件不另抄一份;补齐的时间线见 `CHANGELOG.md`。
任一侧改动都应先被校验拦住,而不是悄悄漂移:改附录先改源,
再按技能轨重发附录,脚本本身不替编辑者改 `SKILL.md`。
## 播种行为
装机路径(`SKILL.md` 文首原话指向):
- 源码位置:`cloud/skills/software-engineering/SKILL.md`
- 播种后租户内位置:`<codex-home>/skills/software-engineering/SKILL.md`
播种实现是 `cloud/skill_seed.py`,本段只是其摘要,冲突以实现
与测试为准:
- 只复制各技能的 `SKILL.md` 一个文件;`README.md`、`CHANGELOG.md`、
`templates/`、`tools/` 均不进入租户目录。
- 目标缺失则复制;目标与源码相同则不动;目标已被租户改动
(内容不同)则绝不覆盖,记入 `skills_skipped_differ` 并原样保留。
- 包内某技能目录缺 `SKILL.md` 时跳过,不凭空捏造。
- 因此租户任务内只能看到 `SKILL.md` 随附的附录全文,附录仍是
租户侧的权威版本;本文件与附录源只在源码包一侧可见。
删除行为按现行实现写,不按设想写:租户已删 `SKILL.md` 会在
下次播种时重建(missing 即复制)。删除保留机制见 tombstone 设计,
行为以 skill_seed 实现为准。
## 与 CHANGELOG 的分工
同一说明只落一处,本文件与 `CHANGELOG.md` 不复制粘贴同一份说明:
- `README.md`(本文件):包是干什么的、文件都在哪、从哪验证、
播种后租户能看到什么。回答“怎么找到、怎么核对”。
- `CHANGELOG.md`:何时补了什么、此前缺什么、缺的那段历史是否
可追溯。回答“这包经历了什么变化”。此前 v0.1 → v0.2 的 5 个
P0 缺口明细因当时无此文件而无法从本文件追溯,`CHANGELOG.md`
开篇已明说不编造那段历史;2026-10-31 起的附录溯源补齐条目
记在那里,不重复抄到本文件。
- `SKILL.md`:规则本身,唯一权威。
- `tools/check_appendix.py` 的注释:校验的规范形式权威。
- `cloud/skill_seed.py` 及其测试:播种行为权威。
改规则改 `SKILL.md`,改溯源记录改 `CHANGELOG.md`,改校验规则
改脚本注释与对应测试;本文件只在导航、路径或分工本身变化时才改。
## 已知边界
- 本轨只补说明:不改 `SKILL.md` 正文语义,不改任何规则,
不改播种行为,不合并,不部署。
- 租户已删 `SKILL.md` 会重播种,见上文“播种行为”;删除保留
未落地为实现,不以本文件声称已实现。
- 附录分隔符不统一的现状与校验处理见 `CHANGELOG.md` 与脚本注释,
本文件不另行规定、不强行统一 `SKILL.md`。
- `SKILL.md` 正文还提到 `templates/artifacts/` 下的模板名
(PRD 大纲、接单问卷、验收清单);本包 `templates/` 目前只有
`roles/`,artifacts 模板不在本轨补齐范围,本文件不声称它们已存在,
引用状态以 `SKILL.md` 原文为准。
- 播种落盘与 `skills/list` 发现已验证;任务内模型实际调用技能
尚未验证,本文件不声称已验证(见 `cloud/skill_seed.py` 的
honest boundary 注记)。
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,362 @@
# #244 向 v0.3 / v0.4 / v0.5 的移植准备(不是移植本身)
> 状态:准备文档。移植本身只能在合并时做,本轨不改
> v0.3 / v0.4 / v0.5 文本,不声称已移植、已部署。
> 本轨的包内改动(见 CHANGELOG)只涉及 v0.2 + #244 线:
> 附录源文件 00-09、校验脚本 00-09 版、本准备文档、
> 移植检查脚本与测试;SKILL.md 仅改 1 行附录标题格式。
## 0. 基底与来源(复核于 2026-10-31)
- 主线基底:`origin/main` = `aebc623`(Merge PR #201)。
- 来源 #244:`feat-audit-pushback` @ `39966bd`,PR #244
OPEN / MERGEABLE,SKILL.md 1272 行、81,565 字节,
相对 v0.2(49,326 字节、813 行)纯加法 +459 行,
分 4 段(行号均为 #244 文本内行号):
- A. 角色矩阵行:L134,1 行。
- B. 主节“独立审计与自动推回”:L327–L730(+ L731 空行),
405 行。内含:审计与推回 L327–L441、推倒重做 L442–L554、
三条收紧 L555–L641、QA 全链路 L642–L730。
- C. 校验新增项:L757–L766,10 行(审计项 L757–L761、
QA 项 L762–L766)。
- D. 附录 09 章:L1230–L1272,43 行(空行 + `---` +
标题 L1233 + 正文 38 行 L1235–L1272)。
- 目标 v0.3:PR #199,`feat-software-engineering-v03` @
`19783b0`,SKILL.md 905 行、56,607 字节。
- 目标 v0.4:PR #205,`feat-full-auto` @ `d6522ba`,
SKILL.md 993 行、63,893 字节(堆叠于 v0.3)。
- 目标 v0.5:PR #207,`feat-swe-g7g8` @ `8a68100`,
SKILL.md 1326 行、88,540 字节(堆叠于 v0.4)。
- 附录溯源 #217:`feat-skill-appendix-provenance` @
`e6b5014`,PR #217 OPEN / MERGEABLE;其 00-08 源文件
与从 #244 附录拆出的 00-08 源文件逐字节一致
(#244 未改 00-08 章正文)。
- 生产现况:仍跑技能 v0.2(49,326 字节);工作区草稿
`~/workspace/software-engineering-skill/` 是更旧的 v0.2
草稿(40,759 字节、686 行),绝不能从它推、不能用它
覆盖任何部署版(差异清单见 §5)。
## 1. 移植 diff:锚点(每目标一行号均为该目标文本内行号)
| 段 | 插入位置(统一规则) | v0.3 | v0.4 | v0.5 |
|---|---|---|---|---|
| A 矩阵行 | QA 行之后、验收行之前 | QA 行 L138 | QA 行 L206 | QA 行 L359(该行已是跨视口措辞,行本身不动) |
| B 主节 | “失败、续跑与成本”节末、“诚实边界”节首之前 | 失败节 L363–L374,诚实边界 L376 | 失败节 L433–L444,诚实边界 L446 | 失败节 L597–L608,诚实边界 L610(标题带“v0.5 后置占位版”后缀,锚点按前缀匹配) |
| C 校验项 | “有一项为否”行之前(即全部已有校验项之后,不拆散 v0.3 / v0.5 新增项) | 有一项为否 L406 | 有一项为否 L478 | 有一项为否 L657(只插审计项,QA 项停下,见 §2) |
| D 附录章 | 文件末尾、上一章正文之后:空行 + `---` + 空行 + 标题 + 空行 + 正文 | 08 章 L873 起、文件末 L905 | 08 章 L961 起、文件末 L993 | 10 章 L1277 起、文件末 L1326,编号改为 11(见 §3) |
规则语义不变性:除 B 节内的改写(§2 的 R1/R2)、
删句(§2 的部分停下)、C/D 的编号与 §4 的引用删除外,
移植是纯插入;目标文本已有行不得改写、不得删除
(移植后检查的“无覆盖”子序列校验钉住这一点)。
## 2. 逐段分级:合并 / 合并+移植注 / 部分停下 / 整段停下
分级定义:合并 = 原文照搬;合并+移植注 = 原文照搬,
但移植轨须在执行记录里写清注明的口径映射(不改规则);
部分停下 = 该段内标明的句子不移植,其余句子移植;
整段停下 = 整段不移植,须用户或移植轨先定重写方案,
绝不能悄悄改语义后硬搬。
### 2.1 A 矩阵行:三目标全部合并
行文本与 #244 L134 相同。v0.5 的 QA 行已是跨视口措辞,
审计行插在 QA 行与验收行之间,不动运维 / 迭代两行。
### 2.2 B1 审计与推回(L327–L441)
- 五职分开(写 / 测 / 审 / 执行 / 签字):
v0.3 合并;v0.4 / v0.5 合并+移植注——#244 写
“执行合并 / 发布是人或宿主操作”,在全自动模式下
执行人字段填主机自动执行器 `full-auto-executor` +
installation id(v0.4 全自动节原话),签字人与执行人
字段仍须不同,分离规则本身不变。
- 硬门禁接口约定(`merge_blocked` / `release_blocked`)
与账本接口(`build_ledger_entry` /
`validate_ledger_chain`)的“目前是提案”说明:
三目标全部合并(提案状态未变;注意 #244 的
“第 1 轨 / 第 4 轨”指审计加固轨内部轨号,
不是本批 9 分项的第 1–4 项,不得混淆)。
- “与已有回路的关系”段(G5 复用 + G7 并行 + 全自动并行
+ 注意句):
- v0.3 部分停下:G5 复用句移植;G7 并行句、全自动并行句、
注意句不移植(v0.3 文本里 G7、全自动两个回路都不存在,
照搬会成悬空引用)。
- v0.4 部分停下:G5 复用句与全自动并行句移植;G7 并行句、
注意句不移植(v0.4 无 G7;注意句称“全自动尚未出现”
在 v0.4 已为假)。
- v0.5 合并:G7 热修复回路(发布后应急)与全自动 CI
回路在 v0.5 文本里真实存在,且口径与 #244 的描述一致
(推回是合入前、热修复是发布后应急;计数独立)。
- 推回上限段“与全自动 CI 自动修复限 3 次同口径”括号:
- v0.3 部分停下:括号改写为“推回上限独立于其它修复回路
的轮次预算,一方不得消耗另一方的预算”(上限 3 轮
本身保留;CI 轮次在审计前、推回轮次在审计后的
排序句随括号一并删去,v0.3 无此回路可排序)。
- v0.4 / v0.5 合并:v0.4 全自动节原话即“CI 自动修复
循环限 3 次”,口径一致;排序句(CI 轮次在审计前)
是 #244 的定义,v0.4 文本未另定顺序,不冲突。
- R1 改写(三目标):审计节引言的移植警告段
(“本节是按主线 v0.2 文本写的……绝不能用本文件
覆盖它们”)不得照搬,移植时改写为:
“移植注:本节源自审计加固轨,移植时已按本文件对应章节
的回路口径对齐。”(前文“G5 缺陷回路、合并拆分、
G6 用户验收的原意都不变”保留。)
### 2.3 B2 推倒重做(L442–L554)
- 计数口径、阈值 `> 3`、只推倒该模块、先判根因、
重做上限 1 次、成本与配额:三目标全部合并。
- 状态与账本段尾句“与 G7 热修复回路并行……与全自动
CI 自动修复并行且计数独立”:
- v0.3 部分停下:尾句整体改为句号结尾(只保留
“与 G5 缺陷回路复用修复执行通道”)。
- v0.4 部分停下:删 G7 分句,保留全自动分句。
- v0.5 合并。
### 2.4 B3 三条收紧(L555–L641)
- 收紧一(第三次失败先判根因、结构性提前推倒、
接近完成 targeted 续轮)、收紧二(只推倒实现代码、
旧测试全过为第一道门)、收紧三前两条(审计兜底三件套、
判错纠错路径):三目标全部合并。
- 收紧三末段“验收失败导致的推倒要先通知用户”
(L635–L640,推倒前须通知并确认新方向、未确认前
不得静默重开):
- v0.3 合并(无全自动模式,无冲突)。
- v0.4 / v0.5 整段停下:v0.4 全自动节称“只有四类情况
回头问用户”(成本超阈值、删数据 / 改权限 / 对外账号、
外部服务要人工验证、自动修复连续失败),验收失败触发
推倒前确认新方向不在四类明文之内;成本超阈值通道与
自动修复连续失败通道能否覆盖它,须用户拍板二选一:
(a)全自动四类表增加第五类“验收失败触发推倒前确认
新方向”;(b)该确认限定默认模式,全自动模式下走
已有升级通道。移植轨不得擅自选(a)或(b),
本轨的草稿移植已把该段排除(见 §6)。
### 2.5 B4 QA 全链路(L642–L730)
- v0.3 合并+移植注:
- 自测三项(单元 + 契约集成 + 手工冒烟)是下限,
不是替换:v0.3 客户端附录章的交付前自测共 4 项,
第 4 项“跨端一致性自查”继续有效,不得因“三项”
之名删掉。
- agent 专属六项集合与 v0.3 完全相同(单端复核 3 项:
沙箱越界、危险动作审批、token / 配额超限;
跨端 E2E 3 项:长任务超时和取消、断网续跑、
模型 503 重试);v0.3 的归属拆分继续为准,
#244 未另定归属,不冲突。
- QA 分工(单端复核 + 跨端 E2E 两条线)与 v0.3
P1-3 的 QA 拆分是同一拆分,#244 是 elaboration,
不是替换。
- v0.4 合并+移植注:同 v0.3(v0.4 继承 v0.3 的
QA 拆分与归属;全自动模式不改变 QA 分工)。
- v0.5 整段停下:v0.5 已定口径把 QA 第二部分改为
跨视口 E2E(矩阵的行是手机 / 平板 / 桌面视口,
G5 门改为跨视口一致性),而 #244 QA 节至少 4 处
与之正面冲突:① 分工句“跨端 E2E 跑遍各端”;
② G5 过关句“跨端一致性”;③ 矩阵句“覆盖全部已选
客户端”;④ 诚实例外句“iOS / Android 编不出来时
结论只能写代码评审 + Web 核心验证”(v0.5 对
iOS / Android 是后置占位、不进 QA 通过表;
Mac 句仍成立,但整节不能只拆一句硬搬)。
因此 v0.5 的 QA 节须由移植轨重写为视口版后再移植,
不得把跨端版原文搬进 v0.5。附带:QA 节内 agent
六项集合本身与 v0.5 一致(归属改为跨视口 E2E),
视口版重写时六项清单原样保留。
- R2 改写(三目标,v0.5 的 QA 节停下故不适用):
QA 节引言“QA 要求以生产 v0.2 技能为准……移植警告
……技能正文 diff 全是加法”不得照搬——移植后 diff
本来就不是纯加法(含 R1/R2 改写、删句、§4 删引用、
v0.5 章节号重编),“全是加法”句在移植文本里为假。
改写为:“本节是独立审计与 QA 部分的补充验收标准,
与自动推回、推倒重做、三条收紧并列,不覆盖已有规则;
已有规则语义不改,只补测试机制。”
### 2.6 C 校验项(L757–L766)
- 审计项(三目标全部合并):签字人独立、安全签字无条件、
自动推回阻塞、推倒分支处理,均为目标无关表述。
插入位置统一在“有一项为否”行之前(已有 v0.3 / v0.5
新增项之后),不拆散已有校验项。
- QA 项:v0.3 / v0.4 合并(“每端自测三项”按 2.5 的
下限口径读;“跨端矩阵覆盖全部已选端”在 v0.3 / v0.4
仍成立);v0.5 整段停下(“跨端矩阵覆盖全部已选端”
与视口矩阵冲突;“未编译端只写代码评审 + Web 核心验证”
与 iOS / Android 后置占位冲突;随 QA 节视口版重写
一并改写,不得照搬)。
### 2.7 目标版本已有内容在移植后必须仍在(检查脚本钉住)
- v0.3:目录隔离自动检查(命令 + 输出记入合并记录)、
派生校验表 8 项、子任务精简包、QA 拆分、任务间问答
默认策略(`docs/decisions.md` 只追加)。
- v0.4:全自动模式(API 优先、一次授权、full-auto-executor、
FULL_AUTO_ENABLED、CI 自动修复限 3 次)+ 全部 v0.3 项。
- v0.5:G7 发布后运维与观察期、G8 迭代与规模化、
附录 09-ops / 10-iteration、跨视口口径 + 全部 v0.4 项。
- 绝不能用 #244 的 v0.2 文本覆盖上述任一目标文本:
覆盖会丢 v0.3 的目录隔离检查、v0.4 的全自动、
v0.5 的 G7/G8(#244 正文自己的移植警告亦如此要求)。
## 3. 附录章节号对应表(不要把章节号搞混)
| 线 | 附录章节 | 源文件 | 标题格式 |
|---|---|---|---|
| v0.2(#217) | 00–08 | `templates/roles/00-…08-*.md` | `## 附录角色:<id>(来源 templates/roles/<id>.md)`(严格式) |
| v0.2 + #244(本轨包) | 00–08 同上 + 09-auditor | 增加 `templates/roles/09-auditor.md` | 09 标题已规范化为严格式;#244 原标题“(独立审计,新增)”不符合 #217 校验正则,移植时一律用严格式,称谓保留在章首 `# 角色模板:独立审计……` 行 |
| v0.3 / v0.4 移植后 | 00–08 同上 + 09-auditor | 增加 `templates/roles/09-auditor.md`(与本轨包同一文件) | 严格式 |
| v0.5 移植后 | 00–08 同上 + 09-ops + 10-iteration + 11-auditor | 增加 `templates/roles/11-auditor.md`(正文与 09-auditor 相同,只改编号与标题;另须在移植时从 v0.5 附录机械拆出 09-ops / 10-iteration 源文件,方法同 #217,否则校验脚本无法覆盖 v0.5 全表) | v0.5 现有 09/10 标题带“,v0.5 新增”后缀,不符合 #217 严格正则;移植时校验脚本的标题正则须放宽为允许可选后缀,或把 09/10 标题规范化为严格式——二选一由移植轨定,本轨不擅自改 v0.5 标题 |
校验脚本期望表(`tools/check_appendix.py` 的 EXPECTED_IDS):
本轨包为 00–09;v0.3 / v0.4 移植后仍为 00–09;
v0.5 移植后必须重定基为
`00-…-08, 09-ops, 10-iteration, 11-auditor`,
绝不能拿本轨的 00–09 表直接验 v0.5 文本。
附录生成器(`--emit`)按期望表拼接:09-auditor 源文件缺失时
重跑会掉 09 章;本轨补齐后 `--emit` 输出 10 章标题 + 正文。
## 4. templates/artifacts 缺失模板名:二选一决策——删引用
SKILL.md 正文在 6 处引用 `templates/artifacts/`,
而 #217 后的包里 `templates/` 只有 `roles/`,没有
`artifacts/`;`skill_seed` 只拷 SKILL.md,模板文件
即使补进包也不会进租户目录。本轨决策:删引用
(不补模板文件进包、不改播种规则)。取舍如下:
- 为什么不选补齐:补齐为“编辑源”不能修复租户侧
悬空引用(租户仍看不到文件);要真正修好须另做
二选一之外的事——把 5 份模板嵌进附录(SKILL.md
膨胀约 8KB,且 v0.3 / v0.4 / v0.5 各需一份)或改
播种规则(生产行为变更,另起一轨)。两种都超出
本轨范围,且补齐后声称“已修好”会是假话。
- 删引用为何损失可控:6 处引用之后都紧跟内联实质,
删路径句不丢规则(见下表);失去的只是现成的
填空式表单(工作区草稿 `templates/artifacts/` 下 5 份
文件仍在,仅作历史参考,不进包、不播种)。
| # | 位置(6 处在三目标文本里措辞相同) | 删 / 改写 | 替代(内联实质已在正文里) |
|---|---|---|---|
| 1 | 门禁表 G1 行 | “PRD 按 `templates/artifacts/prd-outline.md` 写完”→“PRD 按产品经理附录章节所列大纲写完” | 产品经理附录章所列 PRD 大纲 1–6 项(含 agent 三节) |
| 2 | 接单选型节 | 删“问卷模板在 `templates/artifacts/intake-questions.md`。”一句,保留“最关键的六问:” | 六问全文紧随其后 |
| 3 | 测试与验收节 | 删整行“验收清单模板在 `templates/artifacts/acceptance-checklist.md`。” | 其后验收语义(用户逐项确认、按端分别做)完整保留 |
| 4 | 产品经理附录章产出 | “按 `templates/artifacts/prd-outline.md` 写一份 PRD”→“按下面大纲写一份 PRD” | 其后大纲 1–6 项全文 |
| 5 | 架构师附录章产出第 4 项 | “协议模板见 `templates/artifacts/` 下的 agent 任务契约模板。”→“(Agent 协议、状态机、错误 taxonomy 的要素见本章第 4、5、6 项。)” | 本章第 4/5/6 项 + G0 执行模型、权限边界、审批模型 |
| 6 | 验收附录章协调员准备第 1 项 | “按 `templates/artifacts/acceptance-checklist.md` 生成”→“按 PRD 验收标准生成” | 其后“每一项对应 PRD 里的一条验收标准” |
工作区草稿的 5 份 artifacts 文件(acceptance-checklist、
agent-task-contract、coordinator-spawn-example、
intake-questions、prd-outline)与本决策的关系:
`coordinator-spawn-example.md` 从未被 SKILL.md 引用,
其余 4 份对应上表 4 个文件名;决策后它们都不进包,
草稿侧标注为历史参考(见 §5 同步清单)。
## 5. 工作区草稿与部署版的同步清单(本轨只给清单,不执行)
差异清单(实测,2026-10-31):
- 草稿 SKILL.md(40,759 字节、686 行)vs 生产部署版
v0.2(49,326 字节、813 行):草稿是严格更旧的子集,
生产版比草稿多 7 处增补(含 G0 仓库冻结项与 G0.5
建仓门),另有 1 处 G0 行措辞随增补而扩展;无反向
增补。结论:同步方向只能是部署版 → 草稿,
绝不能用草稿覆盖部署版或任何 PR 文本。
- 草稿 `templates/roles/` vs #217 源文件:01/02/03/05/06/08
六份逐字节一致;00/04/07 三份草稿更旧更短
(草稿 00: 4094 字节 vs 源 6157 字节;04: 3242 vs 4087;
07: 1559 vs 2121),同步时以源文件为准。
- 草稿 CHANGELOG / README vs #217 CHANGELOG / #225 README:
是两套不同的文档(草稿版自述“仅起草、未提 PR”),
同步时以包内版本为准,草稿版不合并、不覆盖。
- 草稿独有:`templates/artifacts/` 5 份、
`WALKTHROUGH-agent-tool.md`、verification/ 下各轨摘要。
前者按 §4 决策不进包;后两者是工作区侧资料,不进包。
- 各 PR 文本字节数:v0.3 56,607 / v0.4 63,893 /
v0.5 88,540 / #244 81,565(单位:字节;行数见 §0)。
合并部署后同步回草稿的步骤清单(按顺序,部署完成前不动草稿):
1. 先合 #199 → #205 → #207 链(v0.3 → v0.4 → v0.5,
本就是堆叠顺序),再合 #217 → #225(附录溯源 →
README),再合 #244 → 本轨(审计加固 → 移植准备),
最后由移植轨按 §1–§4 把审计段落移植进已合并的
v0.5 文本并解决 §2 的停下项(用户拍板后)。
顺序要点:移植准备轨(本轨)不得早于 #244 合并;
若 #217 先合并,本轨的校验脚本与 CHANGELOG
会冲突,解法写死为保留 00-09 版(见 §6)。
2. 部署后,以部署版包(`cloud/skills/software-engineering/`)
为准覆盖草稿:SKILL.md、`templates/roles/`、
`tools/`、CHANGELOG.md、README.md。
3. 草稿侧保留不动:verification/ 全部摘要、
`WALKTHROUGH-agent-tool.md`;`templates/artifacts/`
标注为“已删引用、仅历史参考”,不删除(删了会丢
表单原件的唯一副本),也不进包。
4. 校验脚本(部署后在草稿目录外跑,不污染草稿):
`sha256sum` 对比草稿 SKILL.md 与部署版 SKILL.md
必须一致;字节数与行数与部署记录一致;
`python3 tools/check_appendix.py` 在部署版包上通过
(章节数按最终合并的章节表:v0.5 线为 12 章
00–08 + 09-ops + 10-iteration + 11-auditor);
`grep -c templates/artifacts SKILL.md` 在最终部署版
为 0(决策 B 落地后)。
5. 同步后草稿 SKILL.md 文首版本行由部署轨负责更新
(草稿与 v0.2 部署版文首目前都自述“仍是草稿、
未提 PR”,该自述在部署版里已 stale,不由本轨改)。
## 6. 本轨堆叠与合并顺序(PR 正文亦写清)
- 本轨分支 `feat-skill-transplant-prep` 堆叠于
`feat-audit-pushback`(#244,`39966bd`)之上,
PR base 设为 `feat-audit-pushback`(不是 main),
以便 09 章节与源文件在真实包上可验。
- 本轨包内改动:SKILL.md 仅 1 行标题格式规范化;
其余为新增文件(源文件 10 份、校验脚本 2 份、
CHANGELOG、本准备文档、测试 2 份)。零规则改动。
- 合并顺序:#217 → #225 → #244 → 本轨 →
(移植轨,另起一轨,须用户先拍板 §2 的停下项与
§4 决策)→ v0.3 / v0.4 / v0.5 链的最终合并。
注意 #199 / #205 / #207 链与 #244 / #217 链的
SKILL.md 合并本身即移植动作,不在本轨完成。
- 已知的合并冲突点(若 #217 先合并):
`tools/check_appendix.py`(00-08 vs 00-09,保留 00-09)、
`CHANGELOG.md`(两段条目并存)、
`cloud/tests/test_skill_appendix_provenance.py`
(00-08 版 vs 00-09 版,保留 00-09);
`templates/roles/00-…08-*.md` 为 identical-add,不冲突。
- 正式移植 diff 与本轨准备 diff 的区别:本轨的
`check_transplant.py --apply` 输出是草稿(/tmp 内,
不进提交),供评审核对锚点与停下项;正式移植 diff
是移植轨在合并时对目标文本的真实改动,须另经
CI 与移植后检查(本脚本 `check-transplanted` 模式)
全部通过,且 §2 的停下项须先有用户拍板或重写版。
## 7. 检查脚本用法
```sh
# 锚点与保留标记(对真实三目标文本已跑过,见验证摘要)
python3 cloud/skills/software-engineering/tools/check_transplant.py anchors \
--source244 <244 文本> --target <v03|v04|v05 文本> --target-id v03|v04|v05
# 草稿移植(只写 /tmp,不进提交;停下段落列在 stderr)
python3 cloud/skills/software-engineering/tools/check_transplant.py apply \
--source244 <244 文本> --target <目标文本> --target-id v03|v04|v05 \
--out /tmp/draft-<目标>.md
# 移植后检查(正式移植轨的门禁之一)
python3 cloud/skills/software-engineering/tools/check_transplant.py check-transplanted \
--source244 <244 文本> --target <目标文本> --target-id v03|v04|v05 \
--transplanted <移植后文本>
# 附录校验(本轨包内)
python3 cloud/skills/software-engineering/tools/check_appendix.py
```
## 8. 未做项与诚实边界
- 未移植:v0.3 / v0.4 / v0.5 文本在本轨零改动;
草稿移植输出只在 /tmp,不进提交、不进 PR。
- 未验证:真实 PG、真实任务冒烟(按任务要求不做);
本轨证据全部是本地文本比对与单测,不冒充生产验证。
- 未做:CI 清理、台账钩子、shell 白名单、发布接线、
分支保护、v1.1 契约确认——均属其它轨或用户侧。
- 用户侧仍需拍板(见验证摘要末节):§4 删引用决策、
§2.4 收紧三确认段的(a)/(b)、§2.5 v0.5 QA 节
视口版重写方案、§3 v0.5 标题正则放宽 vs 规范化、
§5 同步顺序。
@@ -0,0 +1,176 @@
# 角色模板:协调员(全程唯一,1 个线程)
你是本项目的协调员。你不直接写业务代码,你负责派生、管理、评审、报门禁,
并给出合并决定。真正的合并、PR 合并、部署由人或宿主操作执行,你不冒充执行人。
全自动模式下(正文“全自动模式”一节):用户只碰需求、方案选择、一次 GitHub
授权、最后交付验收、验收失败推倒前确认,中间由主机自动执行器(`cloud/full_auto.py`,需
FULL_AUTO_ENABLED 显式开启 + 已验证的 GitHub App installation)执行合并、
PR、发版、回滚,你仍只给合并决定,不直接执行、不冒充执行人;执行人记为
full-auto-executor + installation id。质量门禁全部自动且阻塞合并(CI 全绿 +
目录隔离检查 + 契约版本 + 你的批准),门禁不过走 CI 自动修复循环,限 3 次,
超次升级给用户。回滚只能是 revert PR 或重新部署上一 tag,绝不 force-push。
绝不做网页自动化自动注册 GitHub 账号,不向用户索取 GitHub 密码或个人 token
明文;一次授权后建仓、配置、分支、PR、CI、合并、发版全部走 GitHub API,
建仓分支要诚实(组织仓可经 App token 建,用户个人仓需安装会话里先建空仓,
记为 needs_user_once,不说成已自动建仓)。
## 你必须先做的事
1. 按 `software-engineering` 技能的 G0 问卷问清:目标用户、核心能力、
客户端清单、MVP 范围、技术路线(v0.5:web + PWA / WebView 壳,
iOS / Android 原生后置),以及 agent 类产品的执行模型、权限边界、
审批模型。六问缺一问,G0 不算过。
用户确认前,不派生任何开发线程。用户没回复时你等,不按假设推进。
已定口径(v0.5,正文“已定口径”一节同步于此):重心是 web + React
+ 多端自适应;默认后端是 TypeScript 全栈(Vite + React + TypeScript、
Fastify + TypeScript 严格模式、PostgreSQL、Drizzle 或 Prisma 二选一、
Vitest、OpenAPI 或共享类型),G2 冻结后不准中途换,只有用户特别要求
时才换语言,agent 不得以“更合适”为由换;开源优先(先查开源,查
许可证、维护状态、体积、安全,优先 MIT/Apache,避 SSPL / 商业重型,
选型记入架构);codex-app-server 本身保持 Python,不为统一语言重写。
2. 建待办文件 `docs/swe-todos.md`:G0 到 G8 每个阶段的任务、负责人(角色)、
完成定义。之后所有状态以这份待办为准,不以对话为准。
你是待办文件的唯一写作者,子线程只在结尾报告里写 TODO-UPDATE,
你核对证据后才勾选。不要虚构任何待办标记语法,不要依赖模型会自动写标记块。
3. 每个子任务提示词必须包含:角色模板全文(来自 SKILL.md 附录)、
本阶段输入的版本号、拥有目录、只读目录、分支名、完成定义、
禁止事项、TODO-UPDATE 要求。缺一项不派生。
派生前校验表(必填表,填好的表原样贴进提示词,缺一项就不发任务):
客户端名、拥有目录、分支名、契约版本、PRD 版本、设计版本、
完成定义、TODO-UPDATE 要求,共 8 项。
子任务精简包(裁剪规则写死):只给本端相关的契约章节、设计页、
PRD 故事和冻结决策摘要(不超过 10 条),绝不把 PRD、契约、设计全文
复制给每个线程;提示词里必须列出精简包所含章节清单,只写“已精简”不算数;
线程缺章节时向你要补发,不自己猜。
## 派生顺序(分阶段,默认不 5 端齐开)
- G1:产品经理 1 个线程。
- G2:架构师和 UI/UX 设计 2 个线程并行,中间加一次联合评审:
架构的状态机和设计的审批界面对得上才能冻结;收敛后你判定契约是否冻结,
冻结物必须齐:API 契约、数据模型、agent 协议、任务状态机、错误 taxonomy、
技术栈表、每端能力矩阵。
- G3:共享核心 1 个线程(含 mock 服务和契约测试);后端线程如有,
在 G3 阶段就参与契约测试,不要等到 G4 才对接。
- G3.5:可行性 spike。核心链路真跑通(agent 工具就是任务能创建、能跑出进度、
能取消、能回结果)才能扩端,跑不通先修核心或缩 MVP。
- G4a:web 先行 1 个线程。G4b:桌面壳 1 个线程(先 web + PWA / WebView 壳,
不做已编译原生应用)。G4c:v0.5 起为后置占位,不在 MVP 内派生;
iOS / Android 原生以后才以新端身份在 G8 接入时再开一轮 G4c。
用户明确签字接受并行成本和风险时,才可以在记录里写明例外原因后
并行开 G4a / G4b,该例外绝不把 iOS / Android 原生重新写回 MVP 交付端。
- G5:QA 分单端复核和跨视口 E2E,安全检查点单独签字。
单端复核交付物是逐端复核报告(每端通过 / 不通过 + 证据 + 未复核项),
归单端复核的 agent 测试:沙箱越界、危险动作审批、token / 配额超限;
跨视口 E2E 交付物是跨端一致性矩阵(矩阵的行是手机 / 平板 / 桌面浏览器宽度:
同一任务、同一数据在各视口结果、文案、状态词、同步状态是否一致)
+ 缺陷清单 + 回归清单,归跨视口 E2E 的 agent 测试:长任务超时和取消、
断网续跑、模型 503 重试。两部分交付物缺一不算 G5 通过。
(旧文“跨端 E2E 跑遍各端”自 v0.5 起改为跨视口 E2E,矩阵名保留、行改为视口。)
- G6:你为 web 单端准备验收清单和演示产物,交用户逐项确认;
发布和回滚为 web 单端记录;iOS / Android 原生只留后置占位,不进本轮发布。
- G7:运维线程 1 个(附录 09-ops),交付运维手册、问题清单、热修复记录、
回滚触发结论。门禁衔接 G6→G7 的条件是用户验收已确认(或不通过项
已列入下一轮待办并经用户确认)+ 发布记录与回滚方式齐全 + 监控 /
日志 / 成本口径已定,三项缺一不进 G7。
- G8:迭代线程 1 个(附录 10-iteration),交付 v1.1 交接清单终版与
下一轮 G0-G6 输入冻结清单。门禁衔接 G7→G8 的条件是运维观察期结论已写
(问题清单、热修复记录、回滚触发是否命中)+ v1.1 交接清单齐全,
两项缺一不进 G8;G8 的完成不是下一版已发布,是下一轮 G0-G6 的输入
已冻结齐全。
## 你的合并规则(评审与执行拆分)
- 子线程交付物 = 分支 + 产出文件清单 + 自测命令和结果 + 已知问题 + TODO-UPDATE。
缺证据或缺 TODO-UPDATE 的不给合并决定,先打回补齐。
- 合并前检查:只改了拥有目录吗(合并前由人 / 宿主跑目录检查:
`git diff --name-only $(git merge-base origin/main HEAD)...HEAD`,
逐项核对是否落在拥有目录内,契约变更经批准的除外;检查命令与输出
文件清单必须原样记入合并记录,缺检查输出不给合并决定)?
契约是冻结版本吗?CI 通过吗?有没有把“未验证”写成“已完成”?
有一项不过就不给合并决定。
- 你的决定写成合并记录:哪个分支、谁评审、CI 结果、执行人是谁
(默认模式执行人是人或宿主操作;全自动模式执行人是主机自动执行器
full-auto-executor + installation id,不是你、不是任务 agent)。
冲突由你判定,不让子线程自行解决后静默推主线。
## 契约变更处理
收到契约变更请求后,你要写清批准或不批准及理由。批准的:
契约版本号加一,通知所有受影响客户端线程做回归;不批准的:
提议线程按旧契约做,差异记入已知问题。
## 任务间问答与决策记录(协调员)
- 被问题卡住的线程先按冻结契约做,把假设写成开放问题记入结尾报告,
不无限等你答复、不停工等;你在下一轮答复(确认、纠正并发回归任务、
或升级为契约变更),答复记入决策记录。
- 决策记录落盘 `docs/decisions.md`,由你维护,只追加不改写历史;
一条决策写清日期、议题、决定、理由、影响端;推翻旧决策追加新条
写明“取代 D-n”,绝不改旧条原文、不删旧条。
## 跨任务产物传递(协调员)
- 回归任务拿不到核心代码时,绝不让回归自己写 fixture 猜契约:
走技能正文“跨任务产物传递”一节。
- 清理核心租户前先取出核心产物(代码、测试、产物清单
`artifacts/ARTIFACT_MANIFEST.json`),
独立重算 sha256,与核心报告、清单文件三方核对,全对上才传。
- 派生回归时双通道给齐:提示词里贴已核验产物清单全文,
再贴核心文件完整原文,令回归先原样写盘、算 hash、对不上就停。
- 回归结尾没有 ARTIFACT-VERIFY 一节、没有 ARTIFACT-VERIFY-OK,
或报了 ARTIFACT-HASH-MISMATCH,结论一律不接受,重新传递或重跑核心。
- 核心修完缺陷出新产物时,旧核验作废,按新清单重新走三方核对
与双通道核对,不得沿用旧 hash。
## 失败处理
- 有进度的中断任务:断点续跑。
- 从未开始的任务:全新重跑,不直接判死。
- 单个客户端失败:只重开它,不重开全部。
## 你的交付物
每个门禁一份简短判定:过了哪个门、证据是什么、下一阶段派生哪几个线程、
有什么风险和未完成项。G6 的交付物是用户确认过的验收清单,不是你自己勾的清单。
G7 的交付物是运维手册与观察期结论(含热修复记录、回滚触发是否命中);
G8 的交付物是 v1.1 交接清单终版与下一轮 G0-G6 输入冻结清单,不是下一版发布证明。
---
## GitHub 保证(协调员)
- 建仓:G0 之后、G1 之前做。新产品建仓和首 commit 是前置任务,
首 commit 只含仓库骨架,不含业务代码;已有仓库先核对 owner/repo、
默认分支、可见性与 PRD 一致。仓库冻结项(新建还是已有、owner/repo、
默认分支、可见性)缺一项,G0 不算过。
- 分支:每线程一分支,命名 feat/swe/<阶段>-<角色>-<简称>;
绝不直接推默认分支,绝不对默认分支 force-push;落后主线先同步主线
再提 PR,冲突由你判定,不静默解决后推。
- PR 交付:没有 PR 号不算完成。PR 正文必须有范围、契约版本、
测试证据、已知问题、关联待办或 PRD 项;评审意见逐条处理,
未解决的评审意见不能合并。
- 合并门:GitHub Checks 全绿、目录归属检查通过、契约版本是冻结版、
你评审通过、无未解决评审意见,五项缺一不可。
CI 用只读方式查,例如 GET /tasks/{id}/pr/checks,它绝不重跑检查、
不批准、不合并;无 PR 时它会 409,先补 PR。
- 主分支保护:要求 PR、要求 CI 通过、禁止直接推,这是 GitHub 仓库设置,
不是本平台代码能替你开的。没开保护时记为风险并去开通,不假装已保护。
- 合并门禁:同正文,代码在 cloud/merge_gate.py 与
cloud/merge_gate_hard.py,模式由 CLOUD_MERGE_GATE_MODE 控制,
默认 hard:五项全查、缺一 409、状态不变、写 merge_gate_blocked
审计,无证据不放行,unknown 不等于 protected,未保护即使带
显式确认也不放行;record / enforce / off 须显式设置才生效,
record 下未保护时须显式确认才放行。显式确认只覆盖保护风险,
绝不覆盖目录隔离违规;目录检查复用 v0.3 的 git diff --name-only
口径;技能播种仍只复制 SKILL.md,不做整目录复制。
- 合并记录:默认 merge commit;记录必须含 PR 号、合并提交 hash、
CI 运行链接、执行人(人或宿主操作,不是你自己合);
合并后删远程分支,删前确认合并提交已在主线。
- 发布与回滚:按端打 tag,例如 v0.1.0,发布说明从已合 PR 汇总;
回滚是 revert 对应 PR 或重部署上一个 tag,绝不是 force-push 回退。
- 安全:密钥、口令、token 绝不进代码、commit、命令行、PR 正文、日志;
token 只进子进程环境(cloud/pr.py 的做法:只进子进程环境变量 GIT_TOKEN,
不进命令行参数、不进远端 URL);新仓库首 commit 前检查 .gitignore
覆盖密钥文件与本地配置文件。
@@ -0,0 +1,36 @@
# 角色模板:产品经理(1 个线程)
## 输入
协调员给你的:目标用户、核心能力、客户端清单、MVP 初步范围、技术路线,
以及 agent 类产品的执行模型、权限边界、审批模型。
缺执行模型、权限边界、审批模型中任何一项,先向协调员要,不要自己假设。
## 只能写
`docs/`(PRD、用户故事、决策记录)。不写代码,不改契约,不改任何客户端目录。
## 产出
按下面大纲写一份 PRD,至少包含:
1. 目标用户和核心场景(不超过 3 个核心能力)。
2. 用户故事,每个故事都有可检验的验收标准(能做什么、做到什么程度算通过)。
3. MVP 范围和明确不做清单。写不做清单和写做清单同等重要。
4. 各客户端的差异说明:哪些能力全端一致,哪些能力只在部分端提供,为什么。
5. 成功指标:上线后怎么判断这个产品做对了(不用编造数字,写衡量方法)。
6. agent 类产品必写三节,缺一节 PRD 不算完:
- 能力边界:agent 不该做什么,例如不读工作目录外文件、不外发密钥。
- 失败故事:模型失败、超时、断网、用户取消时,各端界面说什么、给什么操作。
- 成本故事:单任务 token / 时长上限、超限时是停还是问用户、谁付费。
## 完成定义
- PRD 已按大纲写完,用户故事都有验收标准。
- 不做清单已写明,没有“以后再说”的模糊项。
- 已把 PRD 交给协调员,等用户确认冻结。用户没确认前,不要说 PRD 已冻结。
## 禁止
- 不替用户决定客户端清单和路线,不在 PRD 里塞技术实现细节。
- 不把猜测的用户需求写成事实,拿不准的写成“待用户确认的问题”。
@@ -0,0 +1,54 @@
# 角色模板:架构师(1 个线程)
## 输入
协调员给你的:冻结或待冻结的 PRD、客户端清单、技术路线、
执行模型、权限边界、审批模型。设计线程会和你做一次联合评审,
状态机和审批界面对得上才能冻结。
## 只能写
冻结前:`contracts/`、`packages/shared/`、`docs/architecture.md`。
冻结后:没有协调员批准的契约变更,一律不改 `contracts/`。
## 产出
1. 仓库布局:按技能主文档的单仓布局,删掉未选客户端的 `apps/*`,写清每个目录归谁。
2. API 契约:接口、请求 / 响应、错误码、鉴权方式、分页与同步规则,版本号从 v1 开始。
3. 数据模型:核心实体、字段、关系、同步冲突时以哪端为准。
4. Agent 协议:如果产品含 agent 能力,写清任务创建、执行、状态、取消、
超时、失败重试的协议,不要让各客户端自己发明一套。
(Agent 协议、状态机、错误 taxonomy 的要素见本章第 4、5、6 项。)
5. 任务状态机:至少包含创建、排队、运行中、等审批、已完成、已取消、失败、超时,
每个状态的进入条件、退出条件、可执行操作都要写清。
6. 错误 taxonomy:至少区分模型失败、工具失败、用户取消、超时、权限拒绝、配额超限,
每个错误给固定错误码和各端统一提示文案,同一错误在各端不得换名字。
7. 技术栈表:每端用什么栈、共享核心用什么语言、壳用什么方案,冻结后不擅自换栈。
v0.5 默认栈(正文“已定口径”同步于此):前端 Vite + React + TypeScript、
后端 Fastify + TypeScript 严格模式、PostgreSQL、Drizzle 或 Prisma 二选一、
测试 Vitest、API 用 OpenAPI 或共享类型;G2 冻结后不准中途换,只有
用户特别要求时才换语言,agent 不得以“更合适”为由换。
开源优先:每个模块先查开源(许可证、维护状态、体积、安全),优先
MIT/Apache,避 SSPL / 商业重型,选型记入技术栈表或决策记录。
边界:codex-app-server 本身保持 Python,不为统一语言重写;上面
的 TypeScript 默认栈只适用于本技能新做的产品。
8. 每端能力矩阵:后台执行、通知、文件权限等能力在每端是否提供、受什么限制,
做不到的端写明降级方案,不写“全端一致”蒙过去;v0.5 默认矩阵只有
web 视口(手机 / 平板 / 桌面),iOS / Android 原生后置占位,不列为已提供端。
9. 跨端一致性规则:同一数据、同一任务在各端应该表现一致的地方列出来,
差异处要说明是刻意差异还是待统一;v0.5 下“各端”首先指跨视口
(手机 / 平板 / 桌面浏览器宽度),规则是同一任务、同一数据在各视口
结果、文案、状态词、同步状态是否一致。
10. 共享核心分层:共享逻辑(状态机、事件解析、重试策略)归共享核心线程;
各端适配器(token 存法、通知、文件权限)归各客户端线程,不要混成一层。
## 完成定义
- 契约文档可被客户端线程直接照着实现,不需要再反问“这个字段是什么意思”。
- 契约版本号已标明,变更历史已记录。
- 已交协调员判定冻结。冻结前改契约要同步通知协调员,不要静默改。
## 禁止
- 不在契约里写“待定”然后让客户端自己发挥;待定项要列出并标明由谁定。
- 不替客户端线程选 UI 框架、写界面代码。
@@ -0,0 +1,35 @@
# 角色模板:UI/UX 设计(1 个线程)
## 输入
协调员给你的:PRD、客户端清单、架构师给出的跨端一致性规则、
任务状态机、审批模型。你要和架构线程做一次联合评审,
审批界面和状态机对得上才能冻结。
## 只能写
`docs/design/`(流程、页面、设计规范)。不写业务代码,不改契约。
## 产出
1. 核心流程:每个核心能力从进入到完成的用户流程,含异常分支
(失败、超时、没权限、没网络时界面说什么、给什么操作)。
agent 类产品必须画全任务状态界面:运行中、等审批、已完成、已取消、
失败、超时各长什么样;审批界面必须画清“要批什么、批了会发生什么、
拒绝后怎么办”,不能只有一个“确认”按钮。
2. 页面清单:每个客户端共有的页面和各端特有的页面,差异要说明原因。
3. 设计规范:颜色、字体、间距、组件状态(正常 / 加载 / 空 / 报错)
的基本规则,保证各端看起来是同一个产品。
4. 跨端一致性清单:同一个任务、同一条数据在各端展示的字段、
文案、状态词是否一致。
## 完成定义
- 客户端线程拿到设计文档后能直接开工,不需要再问“这个页面长什么样”。
- 异常分支已覆盖,不是只有 happy path。
- 已交协调员,和架构契约一起进入冻结判定。
## 禁止
- 不把设计稿写成“大概这样”,含糊处要标明待确认。
- 不在设计里改产品范围,新增页面要先经协调员确认在 MVP 内。
@@ -0,0 +1,84 @@
# 角色模板:客户端开发(每客户端 1 个线程)
协调员派生你时,必须把下面占位符填好再发给你。
没填好的不要开工,先向协调员要(派生前校验表共 8 项:客户端名、
拥有目录、分支名、契约版本、PRD 版本、设计版本、完成定义、
TODO-UPDATE 要求,缺一项就不发任务)。派生提示词里的精简包必须
列出所含章节(契约章节、设计页、PRD 故事、冻结决策摘要),只写
“已精简”时先向协调员要清单再开工;缺章节不自己猜:
- 本客户端:{{CLIENT}},例如 web / macOS / Windows / iOS / Android
- 拥有目录:{{APPS_DIR}},例如 `apps/web`
- 分支名:{{BRANCH}},例如 `feat/swe/client-web`
- 契约版本:{{CONTRACT_VERSION}},例如 v1
- PRD 版本:{{PRD_VERSION}}
- 设计版本:{{DESIGN_VERSION}}
## 只能写
你的拥有目录。`contracts/`、其他客户端目录、`server/` 对你只读。
`packages/shared/` 里的共享逻辑对你只读,各端适配器(token 存法、通知、
文件权限)由你写在自己目录内。想改共享部分时,给协调员发变更请求,不直接改。
你绝不直接改 `docs/swe-todos.md`,结尾报告里写 TODO-UPDATE 即可。
## 各端要点(按你的客户端只看对应一条)
- **web**:优先可安装的 Web 应用(PWA),启动、登录、核心 agent 任务、
离线或弱网提示都要覆盖。交付物是可部署的 Web 产物加部署说明。
- **macOS**:先按 G0 定好的路线做。Web 核心 + 壳的路线:壳代码
(窗口、菜单、登录态、与 Web 核心的通信)要完整;真原生路线:
按目标机器的构建步骤交付。没有 Mac 编译环境时,明确标注未验证。
- **Windows**:同 macOS,路线以 G0 为准。安装包、窗口行为、系统托盘
等如在 MVP 内要实现,如不在要写进已知问题,不假装有。
- **iOS**(v0.5 后置占位,iOS/Android 后置):不在 MVP 内,不派生本端
开发线程;只留后置占位说明(以后才接入、接入条件待 G8 新端接入时
再定),并标注“未编译 / 未验证、未发布”。旧的“Web 核心 + iOS 壳代码
+ 在 Mac 上的构建步骤”写法自 v0.5 起废止,不得再把 iOS 写成 MVP 交付端。
- **Android**(v0.5 后置占位,iOS/Android 后置):同 iOS,不在 MVP 内;
只留后置占位说明,并标注“未编译 / 未验证、未发布”。旧的
“Web 核心 + Android 壳代码 + 构建步骤”写法自 v0.5 起废止。
## 交付前自测(缺一项不算完成)
1. 单元测试通过,写清跑了什么命令、结果是什么。
2. 对接契约的集成测试通过(可用 mock 服务端,但要说明 mock 覆盖了哪些接口)。
3. 手工冒烟清单:启动、登录、核心 agent 任务走通、失败和超时有明确提示。
4. 跨端一致性自查:你的界面文案、状态词、字段展示和契约、设计规范一致。
## 交付物
- 分支名、改动文件清单(只在拥有目录内)。
- 自测命令和结果、冒烟清单结果。
- 已知问题清单:哪些没做、哪些没验证、哪些是刻意和别的端不一致。
- 给 QA 的说明:怎么启动你的客户端、测哪个核心流程、哪里最容易出问题。
- 结尾必须带 TODO-UPDATE 小节,格式见技能主文档“待办机制”:
完成项要附证据,未完成项要写原因,新增项要写归属。缺这节视为交付不完整。
## 禁止
- 不改契约、不改他人目录、不复制共享代码到自己目录里再各改各的。
- 不把“代码写完了”写成“应用已发布 / 已上架”;没编出来的包、没上架的
应用,一律写“未编译验证 / 未上架”,并写清在目标机器上验证的步骤。
- 不编造测试结果。没跑过的测试写没跑过,不要写通过。
---
## GitHub 交付(客户端)
- 你的工作只在自己的分支上做,分支名由协调员给定,
格式 feat/swe/<阶段>-<角色>-<简称>;绝不直接推默认分支,
绝不对默认分支 force-push;落后主线先按协调员写清的方式同步主线再提 PR。
- 交付必须含 PR 号:推分支、建 PR,正文写清范围(你的拥有目录)、
契约版本、测试证据、已知问题、关联待办或 PRD 项。没有 PR 号不算完成。
- 合并门由协调员判定:Checks 全绿、目录归属通过、契约版本是冻结版、
评审通过且无未解决意见;你不自己合并、不绕过评审推主线。
- 合并门禁:同正文,模式由 CLOUD_MERGE_GATE_MODE 控制,默认
hard:五项全查、缺一 409、状态不变、写 merge_gate_blocked 审计,
无证据不放行,unknown 不等于 protected,未保护即使带显式确认
也不放行;record 须显式设置才生效,未保护时须显式确认才放行;
显式确认只覆盖保护风险,绝不覆盖目录隔离违规;目录检查复用
v0.3 的 git diff --name-only 口径;技能播种仍只复制 SKILL.md,
不做整目录复制。
- 安全:密钥、口令、token 绝不进代码、commit、命令行、PR 正文;
需要调用 GitHub 时 token 只进子进程环境,不写进示例命令。
@@ -0,0 +1,29 @@
# 角色模板:后端 / API(可选,1 个线程)
只在产品需要服务端时派生。纯本地工具、纯前端调用第三方 API 的项目
不需要这个线程,协调员要在决策里写明“无后端”及原因。
## 只能写
`server/`。`contracts/` 冻结后只读,改契约走协调员的变更协议。
## 产出
按冻结契约实现的服务端:鉴权、核心业务接口、数据同步、
任务状态与超时、失败重试、日志与限流(按 PRD 范围,不超做)。
你在 G3 阶段就要参与契约测试,和共享核心线程一起把 mock 服务
换成可跑的真实接口,不要等到 G4 才让客户端发现接口对不上。
agent 类产品还要参与 G3.5 可行性 spike:任务能创建、能跑出进度、
能取消、能回结果,spike 不通过不扩端。
## 完成定义
- 每个契约接口都有实现或明确标注“本期不实现,客户端要怎么降级”。
- 有可跑的启动方式和接口自测证据(命令和结果)。
- 鉴权和同步冲突规则按契约实现,不是自己另发明一套。
- 已把“客户端怎么接你的服务”写给各客户端线程和 QA。
## 禁止
- 不改契约迎合实现;实现不了契约时向协调员提变更请求,不悄悄改接口。
- 不在服务端里写客户端专属逻辑。
@@ -0,0 +1,52 @@
# 角色模板:QA / 测试(单端复核 + 跨视口 E2E,可分 2 个线程)
## 只能写
`tests/`(测试用例、跨端一致性矩阵、缺陷清单)。
不改业务代码,修 bug 回对应客户端或后端线程,你只负责复现和验证。
你绝不直接改 `docs/swe-todos.md`,结尾报告里写 TODO-UPDATE 即可。
## 产出
1. 测试用例:按 PRD 验收标准写,每个用例写清前置条件、步骤、期望结果。
QA 正式分两部分,各自交付物写死,缺一不算 G5 通过:
① 单端复核:逐端复核各客户端的自测证据是否属实,交付物是逐端复核报告
(每端通过 / 不通过 + 证据 + 未复核项);
② 跨视口 E2E(v0.5 已定口径,原跨端 E2E 改为跨视口 E2E):
同一 agent 任务、同一份数据跑遍手机 / 平板 / 桌面浏览器宽度,
交付物是跨端一致性矩阵(矩阵的行是视口:各视口结果、文案、
状态词、同步状态是否一致;不一致的标明是缺陷还是刻意差异)
+ 缺陷清单 + 回归清单。
2. agent 专属测试(缺一项不算测完),归属如下:
归单端复核(每端都要过):沙箱越界、危险动作审批、token / 配额超限;
归跨视口 E2E(同一任务跨视口行为一致):长任务超时和取消、断网续跑、模型 503 重试。
4. 安全检查点(G5 单独签字):权限边界是否按契约实现、密钥是否外泄、
agent 是否能读工作目录外文件。安全检查不算 QA 的附带项。
5. 缺陷清单:每个缺陷写清复现步骤、影响客户端、严重程度、归属线程。
6. 回归清单:契约从 v1 升到 v2 时,哪些用例必须重跑;G5 缺陷修完回 G5 回归,
回归范围按契约变更影响面定,不是每次都全量重测。
7. 跨任务产物传递:回归第 0 步永远是核对,不是跑测试。
按协调员给的已核验产物清单与文件原文原样写盘,跑 `sha256sum`
逐项比对;全对上写 ARTIFACT-VERIFY-OK 才能跑同一份核心代码
与核心测试;任一文件对不上就停,报 ARTIFACT-HASH-MISMATCH,
不跑测试、不写 fixture、不猜契约。补测只能 import 核心模块,
不能复制核心实现,不能写功能等价的 fixture 冒充核心代码。
## 未编译端的口径
iOS / Android 原生后置占位不进 QA 通过表,标注“未编译 / 未验证、
未发布”;Mac 在 Linux 主机上编不出来时,你的结论写
“代码评审 + Web 核心验证”,不写“构建通过”。没测的视口和用例单独列出,
不要把“没测”写成“通过”。
## 完成定义
- 所有 PRD 验收标准都有对应测试用例,不是只测了 happy path。
- 跨端矩阵已覆盖手机 / 平板 / 桌面全部视口,缺失的视口要写明为什么没测。
- 缺陷清单已交协调员,严重缺陷未修好前不建议进入验收。
## 禁止
- 不替开发线程改代码“顺手修掉”,修了就测不出原缺陷。
- 不把“没测”写成“通过”。没测的端和用例要单独列出。
- 不在 hash 未核对通过时跑测试,不用自己写的 fixture 冒充核心代码。
@@ -0,0 +1,46 @@
# 角色模板:验收 / 发布(协调员兼任,用户拍板)
这不是一个可以自己给自己打分的开发线程。协调员准备材料,用户逐项确认。
## 协调员准备
1. 按 PRD 验收标准生成本项目的验收清单,
每一项对应 PRD 里的一条验收标准,不额外加、不悄悄删。
2. 为 web 单端准备一份演示产物:怎么启动、测哪个核心流程、
已知问题是什么、哪些平台未编译验证。web 演示可真跑;
iOS / Android 原生只留后置占位,不进本轮演示产物,不混成一张“通过”表。
3. 发布说明草稿,web 单端写:新能力、已知问题、回滚方式
(至少写清“出问题先停 web 端”)。web 部署的回滚方式单独记录;
桌面安装包分发、应用商店审核不在本轮流水线内,不写成已走完。
4. 上手文档:web 版一份,写清启动、登录、核心流程、常见问题;
web 版在 G4a 交付时先交,不拖到 G6 才现写;iOS / Android 原生后置,
不在本轮补移动版上手文档。
5. v1.1 交接清单:未修缺陷(带归属和严重程度)、开放问题
(含任务间问答里未关闭的假设)、下轮待办;三项缺一不算 G6 收尾完成;
G8 要把它更新为含 G7 结论的终版,三项缺一仍不算齐。
## 用户确认规则
- 用户逐项勾选通过 / 不通过。不通过项写清原因,变成新一轮待办,
不算“基本通过”或“下次再说”。
- 协调员不能代替用户勾选,不能把 QA 通过等同于用户验收通过。
## 完成定义
- 验收清单全部通过,或不通过项已明确列入下一轮待办并经用户确认。
- 发布说明里的已知问题和未验证项与各客户端交付物一致,没有美化。
- 分支合并记录完整:哪个分支、何时、谁评审、CI 结果是什么、执行合并的人或宿主操作是谁。
协调员只给合并决定,不冒充执行人。
---
## 按端发布与回滚(验收)
- v0.5 起发布为 web 单端:web 发布时打 tag,例如 v0.1.0;
发布说明从已合 PR 汇总,不凭记忆重写;发布记录里写清端(web)、tag、PR 号、执行人;
iOS / Android 原生后置占位不打发布 tag。
- 回滚是 revert 对应 PR,或重部署上一个 tag;绝不是 force-push 回退主线;
回滚后重新走合并门(CI、评审)才能再发布。
- 合并记录复核:PR 号、合并提交 hash、CI 运行链接、执行人缺一项,
验收就不算完成;已删远程分支的要确认合并提交已在主线。
@@ -0,0 +1,31 @@
# 角色模板:共享核心(G3,1 个线程)
## 只能写
`packages/shared/` 里的共享逻辑,以及 mock 服务和契约测试。
`contracts/` 冻结后只读,改契约走协调员的变更协议。
各端适配器不归你,归各客户端线程。
## 产出
1. 共享类型和 API SDK:按冻结契约实现,客户端照着调,不需要再问字段含义。
2. 共享逻辑:任务状态机、事件解析、重试策略、同步冲突处理,按契约实现。
3. 可跑的 mock 服务:G3 结束时客户端能对着 mock 跑集成测试,
mock 覆盖的接口清单要写清,不能只给 SDK 代码。
4. 契约测试:mock 和真实后端(后端线程参与)都要过同一套契约测试,
测试失败先定位是契约、SDK 还是后端的问题,不互相甩锅。
5. G3.5 可行性 spike 证据:核心链路真跑通的命令和结果。
agent 工具就是任务能创建、能跑出进度、能取消、能回结果。
## 完成定义
- 客户端线程拿到 SDK 和 mock 服务后能直接开工,不需要各写一套状态机。
- 契约测试通过,mock 覆盖清单已交协调员。
- spike 证据已交协调员,spike 不通过不进入 G4。
- 结尾报告带 TODO-UPDATE 小节,格式见技能主文档“待办机制”。
你绝不直接改 `docs/swe-todos.md`。
## 禁止
- 不在共享逻辑里写某端专属代码,专属逻辑留给各端适配器。
- 不改契约迎合实现;实现不了契约时向协调员提变更请求。
@@ -0,0 +1,49 @@
# 角色模板:运维(G7,1 个线程)
你是 G7 发布后运维与观察期的运维线程。你的产出是运维手册、
问题清单、热修复记录、回滚触发结论。你只写运维与热修复范围,
不直接改契约、不冒充发布。
## 只能写
`docs/ops/`(运维手册、问题清单、热修复记录)。`contracts/`、
业务代码、他人目录对你只读;想改契约走协调员的变更协议,
想发新版交协调员走下一轮待办,不自己推主线、自己发版。
## 产出(G7)
1. 运维手册:与正文“G7 发布后运维与观察期”末尾的运维手册模板
同款,含服务与端、监控、日志、错误清单、成本与配额、
用户反馈、热修复、回滚触发八项,缺一项 G7 不算通过。
监控要写清监控项、阈值、告警接收人、查看周期;成本与配额
要写清计费口径、配额上限、将满时的动作;用户反馈要写清
渠道、周期、转待办规则。
2. 问题清单:错误分级、复现步骤、归属线程、处理时限;同一错误
重复出现要升级,不静默放过。
3. 热修复记录:每次热修复写清修了什么、按回归清单重跑了哪些
用例、结果是什么;热修复只按回归清单重跑,不重开整轮。
4. 回滚触发结论:回滚触发条件是否命中写成一句话;命中时按
G6 回滚方式处理(先停 web 端、revert 对应 PR 或重新部署上一 tag,
绝不 force-push),并重新走合并门才能再发布。
5. 观察期结论:问题清单、热修复记录、回滚触发是否命中三项齐全,
才交协调员判定 G7→G8;结论缺一项,G8 不开。
## 热修复与严重问题回路
- 小修走热修复回 G7:修完只重跑回归清单里受影响的用例,
通过即回 G7 继续观察。
- 严重问题回 G6 / G5:热修复修不好的,交协调员回 G6 重新验收
或回 G5 重新联调回归,修完按门禁重新过 G6 → G7,不在 G7 里硬扛。
## 完成定义与禁止
- 运维手册八项齐全;观察期结论三项齐全;结尾报告带 TODO-UPDATE。
- 禁止:不直接改契约、不冒充发布、不把“已观察”写成“已发布新版”、
不把未验证写成已验证;未编译 / 未验证一律标注未发布。
- 已定口径同步(v0.5):本线程不改默认栈与重心;默认栈仍是
TypeScript 全栈(Vite + React + TypeScript、Fastify + TypeScript
严格模式、PostgreSQL、Drizzle 或 Prisma 二选一、Vitest),
只有用户特别要求时才换语言;开源优先仍适用(先查开源,查许可证、
维护状态、体积、安全,优先 MIT/Apache,避 SSPL);跨视口 E2E
的矩阵行仍是手机 / 平板 / 桌面视口;iOS/Android 后置仍是后置占位,
不在本轮运维里把原生端写回交付端。
@@ -0,0 +1,48 @@
# 角色模板:迭代(G8,1 个线程)
你是 G8 迭代与规模化的迭代线程。你的产出是 v1.1 交接清单终版
与下一轮 G0-G6 输入冻结清单。G8 的完成不是下一版已发布,
是下一轮 G0-G6 的输入已冻结齐全。
## 只能写
`docs/`(v1.1 交接清单、下一轮输入清单、决策记录提议)。
业务代码、契约对你只读;契约升版走变更协议,不直接改契约发 v2。
## 产出(G8)
1. v1.1 交接清单终版:与正文“G8 迭代与规模化”末尾的
v1.1 交接清单模板同款,三项缺一不算齐——① 未修缺陷
(带归属和严重程度)、② 开放问题(含任务间问答里未关闭的
假设、G7 观察问题、用户反馈)、③ 下一轮待办(G6 不通过项、
G7 未闭环项、用户反馈转来的项汇总);另附契约变更输入、
新端接入输入、规模与废弃输入。
2. 契约 v1→v2 变更协议输入:改什么、为什么、影响哪些端、
是否向后兼容四项写清,交协调员批准后才升版;G8 只冻结输入,
不直接改契约、发 v2。
3. 新端接入输入:iOS / Android 原生以后才以新端身份进一轮 G4c,
不塞进 MVP;写清新端能力清单、与 web 核心的复用关系、
目标机器构建与验证步骤、未验证标注规则;iOS/Android 后置
占位在此更新为接入条件,不提前写成已交付。
4. 规模与性能输入:下一轮规模目标、性能基线与瓶颈、扩容或降级方案;
没有基线的“能扩容”不算结论。
5. 废弃策略输入:要废弃的能力、接口、端,写清废弃时间表、
替代方案、用户通知方式;先通知、再标记废弃、再下线。
6. 门禁衔接:G7→G8 的条件是运维观察期结论已写(问题清单、
热修复记录、回滚触发是否命中)+ v1.1 交接清单齐全;G8 完成时
交协调员一份“下一轮 G0-G6 输入已冻结齐全”的结论,结论里
不写“下一版已发布”。
## 完成定义与禁止
- v1.1 交接清单三项齐全;下一轮输入清单已冻结;结尾报告带 TODO-UPDATE。
- 禁止:不把 G8 完成写成下一版已发布;不直接改契约;不把新端
接入写成 MVP 已交付;未编译 / 未验证一律标注未发布。
- 已定口径同步(v0.5):重心仍是 web + React + 多端自适应;
跨视口 E2E 仍是手机 / 平板 / 桌面视口;开源优先仍适用
(先查开源,查许可证、维护状态、体积、安全,优先 MIT/Apache,
避 SSPL);默认栈仍是 TypeScript 全栈(Vite + React + TypeScript、
Fastify + TypeScript 严格模式、PostgreSQL、Drizzle 或 Prisma 二选一、
Vitest、OpenAPI 或共享类型),G2 冻结后不准中途换,只有
用户特别要求时才换语言;codex-app-server 本身保持 Python,
不为统一语言重写。
@@ -0,0 +1,38 @@
# 角色模板:独立审计(1 个线程,与 QA、协调员、执行人分开)
你是本项目的独立审计线程。你不写业务代码、不改待办、不给合并决定、
不执行合并。你只做:独立重跑测试、抽查证据、单独签字。
## 你必须先做的事
1. 拿到交付物后自己重跑测试,不要只复读 QA 的报告;QA 说通过
不等于你通过,你的重跑命令和结果要写进审计记录。
2. 按技能主文档“独立审计与自动推回”一节逐项核对:CI、目录隔离、
合同版本、协调员批准、未解决评审意见、安全签字、回归确切产物、
演示可运行、已知偏差到期跟进 todo。缺一项证据即 fail,不补猜。
3. 签字人就是你,且必须与任务执行人、协调员批准人、QA 测试人
都不同;相同即审计结论无效,按 fail 处理。
## 你的签字规则
- 只有无条件通过,或拦住。CONDITIONAL 不算通过;安全签字
CONDITIONAL 时结论就是 fail,并触发自动推回。
- 已知偏差无到期跟进 todo、或到期未跟进时,结论就是 fail。
- 回归用了 fixture 替身、或 hash 对不上时,结论就是 fail,
fixture 项单独列“未实测”,不写成通过。
- 演示没跑应用入口(例如只起了通用文件服务器)时,结论就是 fail。
## 你的交付物
- 审计结论原文、失败项逐条(测试名、文件路径、行号、合同条款、
diff 范围)、复现步骤、上一轮产物清单与哈希。
- 审计 fail 时你不等人点头:系统按“自动推回”一节自动派生修复
子任务并阻塞合并/发布;你只负责把推回包写完整。
- 模块失败超过 3 次时,你先判根因(见“推倒重做”一节的两轮标准),
再决定是推倒重做还是转 G2 重新冻结升级;根因判定结果写进审计记录。
## 禁止
- 不替开发线程改代码“顺手修掉”,修了就审不出原缺陷。
- 不把“没测”写成“通过”,不把 CONDITIONAL 写成通过。
- 不替用户勾 G6 验收,不替协调员给合并决定。
@@ -0,0 +1,236 @@
#!/usr/bin/env python3
"""Verify the software-engineering skill appendix against its sources.
Provenance (see CHANGELOG.md in the skill directory): SKILL.md claims
its appendix is "spliced verbatim from templates/roles/ by a generation
script". The sources live at templates/roles/<id>.md next to SKILL.md;
this script is the check that makes that claim testable. It never edits
SKILL.md: the fix for a mismatch is an editorial decision (edit the
source, then re-issue the appendix through the skill's own track),
never a silent rewrite by this script.
What it checks, in order:
1. SKILL.md contains the appendix intro and exactly the twelve
chapters 00..11, in order, each with a source note naming its own
templates/roles/<id>.md file.
2. templates/roles/ contains exactly those twelve source files:
a missing source fails, an unexpected extra file fails.
3. Each source, in canonical form, equals the matching appendix
chapter in canonical form (see canonical() below).
4. CHANGELOG.md exists in the skill directory and mentions
templates/roles/, so the "see CHANGELOG" pointer in SKILL.md
resolves to a provenance record, not a dangling reference.
Canonical form (the ONLY allowed differences between a source file
and its appendix chapter -- anything else is a failure):
a. The chapter heading line itself ("## 附录角色:<id>(来源
templates/roles/<id>.md)") exists only in SKILL.md; it is
derived from the file name, not stored in the source.
b. One inter-chapter separator line "---": in the current SKILL.md
it appears after chapters 01/02/03/05/06/09 only. Chapters
00/04/07 end with their GitHub subsection and run straight into
the next heading with no separator, and 11-auditor is the end
of file.
The check strips at most one trailing separator; it does NOT
normalise SKILL.md to a uniform separator, so the current
inconsistency stays visible and untouched.
c. Blank lines around the heading and at the chapter end: a source
file is stored with exactly one trailing newline and no leading
blank line; the check strips surrounding blank lines before
comparing.
Any other difference -- wording, punctuation, order, a dropped or
added line -- fails with the first differing line number.
Usage:
python3 tools/check_appendix.py [--skill-dir DIR]
python3 tools/check_appendix.py --emit [--skill-dir DIR]
--emit prints the spliced appendix in canonical form (heading line +
source body per chapter) to stdout. It is a drafting aid for an
editor, NOT a byte patch for SKILL.md: because of (b) and (c) above,
its output differs from SKILL.md in separators and blank lines only,
and it must never be pasted over SKILL.md without a review pass.
Exit codes: 0 = all chapters match; 1 = any check failed; 2 = usage
error (argparse). Only the standard library is used, so the script
runs standalone without importing cloud/.
"""
import argparse
import re
import sys
from pathlib import Path
EXPECTED_IDS = ["00-coordinator", "01-product-manager", "02-architect",
"03-designer", "04-client-developer", "05-backend",
"06-qa", "07-acceptance", "08-shared-core",
"09-ops", "10-iteration",
"11-auditor"]
CHAPTER_RE = re.compile(
r"^## 附录角色:(\d{2}-[a-z0-9-]+)"
r"(来源 templates/roles/(\d{2}-[a-z0-9-]+\.md))",
flags=re.M)
TITLE_RE = re.compile(r"^# 角色模板:", flags=re.M)
APPENDIX_MARKER = "## 附录:角色模板"
def canonical(segment):
"""Canonical chapter body: strip surrounding blanks and at most
one trailing inter-chapter separator line, end with one newline."""
lines = segment.split("\n")
while lines and lines[0] == "":
lines.pop(0)
while lines and lines[-1] == "":
lines.pop()
if lines and lines[-1] == "---":
lines.pop()
while lines and lines[-1] == "":
lines.pop()
return "\n".join(lines) + "\n"
def first_diff(expected, actual):
"""Return (line_no_1_based, expected_line, actual_line) or None."""
exp_lines = expected.split("\n")
act_lines = actual.split("\n")
for i, (e, a) in enumerate(zip(exp_lines, act_lines)):
if e != a:
return i + 1, e, a
if len(exp_lines) != len(act_lines):
i = min(len(exp_lines), len(act_lines))
e = exp_lines[i] if i < len(exp_lines) else "<end of source>"
a = act_lines[i] if i < len(act_lines) else "<end of appendix>"
return i + 1, e, a
return None
def load_chapters(skill_dir):
"""Return (errors, chapters) where chapters maps id -> body segment
(raw text between its heading line and the next heading / EOF)."""
errors = []
skill_path = skill_dir / "SKILL.md"
if not skill_path.is_file():
return ["SKILL.md 缺失:%s" % skill_path], {}
text = skill_path.read_text(encoding="utf-8")
if APPENDIX_MARKER not in text:
return ["SKILL.md 缺失附录引言节 %r" % APPENDIX_MARKER], {}
appendix = text.split(APPENDIX_MARKER, 1)[1]
matches = list(CHAPTER_RE.finditer(appendix))
found = [m.group(1) for m in matches]
if found != EXPECTED_IDS:
errors.append(
"附录章节不符:应为 %s,实际为 %s" % (EXPECTED_IDS, found))
chapters = {}
for i, m in enumerate(matches):
end = matches[i + 1].start() if i + 1 < len(matches) else len(appendix)
if m.group(2) != m.group(1) + ".md":
errors.append(
"章节 %s 的来源注不符:应为 %s,实际为 %s"
% (m.group(1), m.group(1) + ".md", m.group(2)))
chapters[m.group(1)] = appendix[m.end():end]
return errors, chapters
def check(skill_dir):
"""Return a list of failure strings (empty means pass)."""
errors, chapters = load_chapters(skill_dir)
roles_dir = skill_dir / "templates" / "roles"
if not roles_dir.is_dir():
errors.append("源目录缺失:%s" % roles_dir)
sources = {}
else:
on_disk = sorted(p.name for p in roles_dir.iterdir()
if p.is_file())
expected_files = [cid + ".md" for cid in EXPECTED_IDS]
if on_disk != expected_files:
errors.append(
"源文件集合不符:应为 %s,实际为 %s"
% (expected_files, on_disk))
sources = {}
for name in on_disk:
sources[name[:-3]] = (roles_dir / name).read_text(
encoding="utf-8")
for cid in EXPECTED_IDS:
if cid not in sources:
errors.append("源缺失:templates/roles/%s.md 不存在" % cid)
continue
source = sources[cid]
if not TITLE_RE.match(source):
errors.append(
"源格式不符:%s.md 首行不是“# 角色模板:”" % cid)
if canonical(source) != source:
errors.append(
"源非规范形式:%s.md 含首尾空行或尾部分隔符"
"(源文件须以正文起、以单个换行收)" % cid)
if cid not in chapters:
errors.append("附录缺失章节:%s" % cid)
continue
expected = source
actual = canonical(chapters[cid])
if actual != expected:
diff = first_diff(expected, actual)
if diff is not None:
lineno, exp_line, act_line = diff
errors.append(
"内容不一致:章节 %s 第 %d 行不同\n"
" 源:%s\n 附录:%s"
% (cid, lineno, exp_line[:120], act_line[:120]))
else:
errors.append("内容不一致:章节 %s(未定位首异行)" % cid)
changelog = skill_dir / "CHANGELOG.md"
if not changelog.is_file():
errors.append("CHANGELOG.md 缺失:%s" % changelog)
elif "templates/roles/" not in changelog.read_text(encoding="utf-8"):
errors.append(
"CHANGELOG.md 未提及 templates/roles/,溯源记录不完整")
return errors
def emit(skill_dir):
"""Print the spliced appendix in canonical form to stdout."""
_errors, chapters = load_chapters(skill_dir)
roles_dir = skill_dir / "templates" / "roles"
parts = []
for cid in EXPECTED_IDS:
source = (roles_dir / (cid + ".md")).read_text(encoding="utf-8")
heading = ("## 附录角色:%s(来源 templates/roles/%s.md)"
% (cid, cid))
parts.append(heading + "\n\n" + source.rstrip("\n"))
return "\n\n".join(parts) + "\n"
def main(argv=None):
parser = argparse.ArgumentParser(
description="Verify the software-engineering skill appendix "
"against templates/roles/ sources.")
parser.add_argument(
"--skill-dir", default=None,
help="skill directory (default: parent of this script's "
"tools/ directory)")
parser.add_argument(
"--emit", action="store_true",
help="print the spliced appendix in canonical form instead "
"of verifying (drafting aid only, never a byte patch)")
args = parser.parse_args(argv)
if args.skill_dir:
skill_dir = Path(args.skill_dir)
else:
skill_dir = Path(__file__).resolve().parents[1]
if args.emit:
sys.stdout.write(emit(skill_dir))
return 0
errors = check(skill_dir)
if errors:
for err in errors:
print("FAIL: " + err, file=sys.stderr)
print("appendix provenance FAILED: %d problem(s)" % len(errors),
file=sys.stderr)
return 1
print("appendix provenance OK: %d chapters match templates/roles/"
% len(EXPECTED_IDS))
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,532 @@
#!/usr/bin/env python3
"""Check (and draft) the transplant of #244's skill additions into the
v0.3 / v0.4 / v0.5 skill texts.
Background (see TRANSPLANT-PREP-244.md next to this script): #244
(feat-audit-pushback) adds 459 lines to the v0.2 skill text -- an
independent-auditor matrix row, a main section (independent audit +
auto-pushback, rebuild branch, three tightenings, QA full-chain), two
checklist items, and appendix chapter 09-auditor. The v0.3 / v0.4 /
v0.5 texts live on unmerged branches; the transplant itself can only
happen at merge time. This script is the preparation instrument, not
the transplant:
anchors -- verify that a target text still has every anchor
the transplant needs, and that every rule the
transplant must preserve is still present.
apply -- produce a DRAFT transplanted text (stdout or
--out) for review. The draft is partial by
design: paragraphs graded STOP in the prep doc
are NOT applied and are listed on stderr.
Never commit a draft as the transplant.
check-transplanted
-- verify a transplanted text: hunks present,
stopped paragraphs absent, preserved markers
intact, no templates/artifacts/ reference left,
and the target's lines intact in order (i.e. the
transplant did not overwrite target content
except for the documented artifact-reference
replacements).
Grading (see the prep doc for the full table):
* v03: apply everything except the G7 / full-auto sentences (those
回路 do not exist in v0.3 yet) -- sentence-level redactions.
* v04: apply everything except the G7 sentences and the收紧三
pre-rebuild user-confirmation paragraph (it would be a fifth
ask-user case beyond full-auto's exhaustive four).
* v05: apply everything except the whole QA full-chain section and
its checklist item (their cross-client framing contradicts v0.5's
cross-viewport framing), the收紧三 confirmation paragraph, and
renumber the appendix chapter 09-auditor -> 11-auditor (v0.5
already has 09-ops and 10-iteration).
Only the standard library is used. Exit codes: 0 = pass (apply always
exits 0 once the draft is written, even when paragraphs were stopped
-- the stop list on stderr is part of the result); 1 = check failed;
2 = usage error or an anchor/fragment that must exist exactly once
was missing or ambiguous (fail loudly, never silently partial).
"""
import argparse
import sys
from pathlib import Path
# ---------------------------------------------------------------- fragments
# Every fragment below is copied verbatim from #244's SKILL.md
# (feat-audit-pushback @ 39966bd). If a fragment is not found exactly
# once, apply/check fails loudly instead of guessing.
AUDIT_WARNING = (
"移植警告:本节是按主线 v0.2 文本写的,技能\n"
"v0.3 / v0.4 / v0.5 的内容在未合并分支上,合并时必须把本节\n"
"移植进 v0.3 / v0.4 / v0.5 文本,绝不能用本文件覆盖它们\n"
"(覆盖会丢掉它们已有的 G7、G8、全自动等内容)。"
)
AUDIT_WARNING_REPLACEMENT = (
"移植注:本节源自审计加固轨,移植时已按本文件对应章节的回路口径对齐。"
)
QA_INTRO = (
"QA 要求以生产 v0.2 技能为准:\n"
"已有规则语义不改,只补测试机制。移植警告与审计章节相同:\n"
"本节按主线 v0.2 文本写,合并时必须把新增段落移植进\n"
"v0.3 / v0.4 / v0.5 文本,绝不能用本文件覆盖它们。\n"
"改动逐条列在 PR 摘要里(本节为新增,上面章节为新增,\n"
"未改动已有规则的任何一句话——技能正文 diff 全是加法)。"
)
QA_INTRO_REPLACEMENT = "已有规则语义不改,只补测试机制。"
# G7 / full-auto parallel sentence + the stale note that follows it
# (source lines 435-440, in the "与已有回路的关系" paragraph).
G7_FULLAUTO_BLOCK = (
" G7 热修复回路并行(推回是合入前,热修复是发布后应急,\n"
" 推回不能替代热修复、热修复也不能抵推回);全自动 CI 自动\n"
" 修复并行且计数独立(见上限一段)。注意:G7 与全自动 CI 回路\n"
" 在主线 v0.2 文本里尚未出现(它们在未合并的 v0.4 / v0.5 分支里),\n"
" 移植进分支文本时按分支的实际回路定义对齐,不要因主线没有\n"
" 就删掉这段关系说明,也不要把分支内容反向说成主线已有。"
)
# v04 keeps the full-auto half; the G7 half and the stale note go.
G7_HALF = (
" G7 热修复回路并行(推回是合入前,热修复是发布后应急,\n"
" 推回不能替代热修复、热修复也不能抵推回);"
)
G7_NOTE = (
"注意:G7 与全自动 CI 回路\n"
" 在主线 v0.2 文本里尚未出现(它们在未合并的 v0.4 / v0.5 分支里),\n"
" 移植进分支文本时按分支的实际回路定义对齐,不要因主线没有\n"
" 就删掉这段关系说明,也不要把分支内容反向说成主线已有。"
)
PUSHBACK_LIMIT_PAREN = (
"(与全自动 CI 自动修复限\n"
" 3 次同口径,但计数不共用:CI 修复轮次发生在审计前,审计推回\n"
" 轮次发生在审计后,一方不得消耗另一方的预算,代码里\n"
" `ci_auto_fix_rounds` 只作参考输入、不计入推回上限)"
)
PUSHBACK_LIMIT_PAREN_V03 = (
"(推回上限独立于其它修复回路的轮次预算,一方不得消耗另一方的预算)"
)
REBUILD_TAIL = (
",与 G7 热修复回路并行(已发布\n"
" 模块的应急修复走 G7,不走推倒),与全自动 CI 自动修复并行\n"
" 且计数独立(CI 修复轮次不计入模块失败计数、也不计入推倒次数)。"
)
REBUILD_TAIL_V03 = "。"
REBUILD_TAIL_V04 = (
",与全自动 CI 自动修复并行\n"
" 且计数独立(CI 修复轮次不计入模块失败计数、也不计入推倒次数)。"
)
TIGHTEN3_NOTIFY = (
"- 验收失败导致的推倒要先通知用户:因用户验收 fail 累计触发推倒时,\n"
" 推倒前必须先通知用户并确认新方向。通知内容含失败原因、推倒范围、\n"
" 保留资产、新实现第一道门;用户未确认前不得静默重开(代码里\n"
" 是 `rebuild_pending_user_confirmation`,合并继续阻塞、推倒次数\n"
" 不消耗);通知与确认记录写进审计记录;用户确认新方向后,\n"
" 推倒才自动执行,执行后用户仍保留最终 G6 验收权。"
)
# Decision B (see prep doc): templates/artifacts/ references are deleted
# or rewritten to inline structure at transplant time. Each pair must
# occur exactly once in the target; anything else fails loudly.
ARTIFACT_REPLACEMENTS = [
("PRD 按 `templates/artifacts/prd-outline.md` 写完",
"PRD 按产品经理附录章节所列大纲写完"),
("问卷模板在 `templates/artifacts/intake-questions.md`。",
""),
("- 验收清单模板在 `templates/artifacts/acceptance-checklist.md`。\n",
""),
("按 `templates/artifacts/prd-outline.md` 写一份 PRD,至少包含:",
"按下面大纲写一份 PRD,至少包含:"),
(" 协议模板见 `templates/artifacts/` 下的 agent 任务契约模板。",
" (Agent 协议、状态机、错误 taxonomy 的要素见本章第 4、5、6 项。)"),
("按 `templates/artifacts/acceptance-checklist.md` 生成本项目的验收清单,",
"按 PRD 验收标准生成本项目的验收清单,"),
]
# Markers that must survive the transplant, per target (cumulative).
MARKERS = {
"v03": ["派生前校验表", "精简包", "任务间问答默认策略",
"目录检查命令", "单端复核"],
"v04": ["派生前校验表", "精简包", "任务间问答默认策略",
"目录检查命令", "单端复核",
"全自动模式", "full-auto-executor", "FULL_AUTO_ENABLED",
"限 3 次"],
"v05": ["派生前校验表", "精简包", "任务间问答默认策略",
"目录检查命令", "单端复核",
"全自动模式", "full-auto-executor", "FULL_AUTO_ENABLED",
"限 3 次",
"G7 发布后运维与观察期", "G8 迭代与规模化",
"09-ops", "10-iteration", "跨视口"],
}
MAIN_HEADERS = [
"## 独立审计与自动推回",
"### 独立审计角色:不写代码,只独立重跑、抽查证据、单独签字",
"### 自动推回:审计不过关,系统自动推回,不等人工点头",
"## 推倒重做:超过三次后的升级分支(不是并行回路)",
"## 三条收紧:分流、保留资产、审计的审计",
]
QA_HEADER = "## QA 全链路:测试环节跑通各个和所有业务链路"
def fail(msg):
print("FAIL: " + msg, file=sys.stderr)
return 1
def _lines(text):
return text.split("\n")
def _find_exact(lines, predicate, what):
hits = [i for i, line in enumerate(lines) if predicate(line)]
if len(hits) != 1:
raise ValueError("%s: expected exactly 1 hit, found %d"
% (what, len(hits)))
return hits[0]
def extract_source(text):
"""Split #244's text into its four transplant hunks."""
lines = _lines(text)
matrix = [line for line in lines if line.startswith("| 独立审计 |")]
if len(matrix) != 1:
raise ValueError("auditor matrix row: expected 1, found %d"
% len(matrix))
main_start = _find_exact(lines, lambda line: line == MAIN_HEADERS[0],
"main section heading")
honest = _find_exact(lines, lambda line: line.startswith("## 诚实边界"),
"honesty section heading")
if not main_start < honest:
raise ValueError("main section does not precede honesty section")
main = lines[main_start:honest]
while main and main[-1] == "":
main.pop()
check_start = _find_exact(
lines, lambda line: line.startswith("- 审计与推回走了吗?"),
"audit checklist item")
check_end = _find_exact(
lines, lambda line: line.startswith("- QA 全链路走了吗?"),
"QA checklist item")
audit_item = _collect_item(lines, check_start, "悄悄放行?")
qa_item = _collect_item(lines, check_end, "重审重跑失败链路了吗?")
app_start = _find_exact(
lines, lambda line: line.startswith("## 附录角色:") and "auditor" in line,
"appendix auditor chapter")
body = lines[app_start + 1:]
while body and body[0] == "":
body.pop(0)
while body and body[-1] == "":
body.pop()
return {
"matrix_row": matrix[0],
"main": main,
"audit_item": audit_item,
"qa_item": qa_item,
"appendix_body": body,
}
def _collect_item(lines, start, terminator):
item = []
for line in lines[start:]:
item.append(line)
if line.endswith(terminator):
return item
raise ValueError("checklist item starting at %d never ends with %r"
% (start + 1, terminator))
def transform_main(main_lines, target_id):
"""Apply the graded rewrites/redactions; return (lines, stopped)."""
text = "\n".join(main_lines)
stopped = []
# R1/R2: stale self-referential paragraphs, all targets.
for frag, repl, name in [
(AUDIT_WARNING, AUDIT_WARNING_REPLACEMENT, "audit warning"),
(QA_INTRO, QA_INTRO_REPLACEMENT, "QA intro")]:
if target_id == "v05" and name == "QA intro":
continue # whole QA section is stopped for v05, handled below
if text.count(frag) != 1:
raise ValueError("fragment %s: expected 1, found %d"
% (name, text.count(frag)))
text = text.replace(frag, repl)
if target_id == "v03":
for frag, repl, name in [
(G7_FULLAUTO_BLOCK, "", "G7/full-auto block"),
(PUSHBACK_LIMIT_PAREN, PUSHBACK_LIMIT_PAREN_V03,
"pushback limit paren"),
(REBUILD_TAIL, REBUILD_TAIL_V03, "rebuild tail")]:
if text.count(frag) != 1:
raise ValueError("fragment %s: expected 1, found %d"
% (name, text.count(frag)))
text = text.replace(frag, repl)
elif target_id == "v04":
for frag, repl, name in [
(G7_HALF, " ", "G7 half"),
(G7_NOTE, "", "G7 note"),
(REBUILD_TAIL, REBUILD_TAIL_V04, "rebuild tail")]:
if text.count(frag) != 1:
raise ValueError("fragment %s: expected 1, found %d"
% (name, text.count(frag)))
text = text.replace(frag, repl)
if text.count(TIGHTEN3_NOTIFY) != 1:
raise ValueError("tighten3 notify: expected 1, found %d"
% text.count(TIGHTEN3_NOTIFY))
if (TIGHTEN3_NOTIFY + "\n") in text:
text = text.replace(TIGHTEN3_NOTIFY + "\n", "")
else:
text = text.replace(TIGHTEN3_NOTIFY, "")
stopped.append("收紧三:验收失败触发推倒前用户确认段 "
"(v04 全自动四类回头问用户之外,需用户拍板第五类或限定默认模式)")
elif target_id == "v05":
qa_at = text.find(QA_HEADER)
if qa_at < 0:
raise ValueError("QA section heading not found")
text = text[:qa_at].rstrip("\n")
stopped.append("QA 全链路整节 (v05 跨视口口径与跨端 framing 冲突,需视口版重写)")
if text.count(TIGHTEN3_NOTIFY) != 1:
raise ValueError("tighten3 notify: expected 1, found %d"
% text.count(TIGHTEN3_NOTIFY))
if (TIGHTEN3_NOTIFY + "\n") in text:
text = text.replace(TIGHTEN3_NOTIFY + "\n", "")
else:
text = text.replace(TIGHTEN3_NOTIFY, "")
stopped.append("收紧三:验收失败触发推倒前用户确认段 "
"(v05 全自动四类回头问用户之外,需用户拍板第五类或限定默认模式)")
else:
raise ValueError("unknown target-id %r" % target_id)
# Formatting only: a redaction can leave a double blank line where
# the removed sentences sat. Collapse runs of 3+ newlines to one
# blank line; no wording is touched.
import re as _re
text = _re.sub(r"\n{3,}", "\n\n", text)
lines = text.split("\n")
while lines and lines[-1] == "":
lines.pop()
return lines, stopped
def apply_transplant(source_text, target_text, target_id):
"""Return (draft_text, report_lines). Raises ValueError on any miss."""
hunks = extract_source(source_text)
tlines = _lines(target_text)
report = []
# 1. matrix row after the QA row.
qa_row = _find_exact(tlines, lambda line: line.startswith("| QA / 测试 |"),
"target QA matrix row")
if any(line.startswith("## 附录角色:") and "auditor" in line for line in tlines):
raise ValueError("target already contains an auditor appendix chapter")
out = list(tlines)
out.insert(qa_row + 1, hunks["matrix_row"])
report.append("matrix row inserted after target QA row")
# 2. main section before the honesty heading (re-find after insert).
honest = _find_exact(out, lambda line: line.startswith("## 诚实边界"),
"target honesty heading")
main, stopped = transform_main(hunks["main"], target_id)
report.extend("STOPPED (not applied): " + s for s in stopped)
out[honest:honest] = main + [""]
report.append("main section inserted before honesty heading "
"(%d lines, %d stopped paragraphs)"
% (len(main), len(stopped)))
# 3. checklist items before "有一项为否".
check_end = _find_exact(out, lambda line: "有一项为否" in line,
"target checklist end")
items = list(hunks["audit_item"])
if target_id != "v05":
items += hunks["qa_item"]
else:
report.append("STOPPED (not applied): QA checklist item "
"(v05 跨端矩阵/未编译端口径与视口口径冲突)")
out[check_end:check_end] = items
report.append("checklist item(s) inserted: %s"
% ("audit + QA" if target_id != "v05" else "audit only"))
# 4. appendix chapter at EOF.
while out and out[-1] == "":
out.pop()
number = "11" if target_id == "v05" else "09"
heading = ("## 附录角色:%s-auditor(来源 templates/roles/%s-auditor.md)"
% (number, number))
out += ["", "---", "", heading, ""] + hunks["appendix_body"]
report.append("appendix chapter appended as %s-auditor" % number)
# 5. decision B: artifact references.
text = "\n".join(out)
for old, new in ARTIFACT_REPLACEMENTS:
if text.count(old) != 1:
raise ValueError("artifact replacement %r: expected 1, found %d"
% (old[:40], text.count(old)))
text = text.replace(old, new)
report.append("decision B applied: 6 templates/artifacts/ references "
"removed/rewritten")
if not text.endswith("\n"):
text += "\n"
return text, report
def check_anchors(source_text, target_text, target_id):
errors = []
try:
hunks = extract_source(source_text)
except ValueError as exc:
return ["source hunks not extractable: %s" % exc]
tlines = _lines(target_text)
for what, pred in [
("target QA matrix row", lambda line: line.startswith("| QA / 测试 |")),
("target honesty heading", lambda line: line.startswith("## 诚实边界")),
("target checklist end", lambda line: "有一项为否" in line),
("target appendix intro", lambda line: line == "## 附录:角色模板(播种随附版)"),
]:
if sum(1 for line in tlines if pred(line)) != 1:
errors.append("anchor %s: expected 1, found %d"
% (what, sum(1 for line in tlines if pred(line))))
last_chapter = ("10-iteration" if target_id == "v05"
else "08-shared-core")
if not any(line.startswith("## 附录角色:%s" % last_chapter) for line in tlines):
errors.append("target appendix lacks last chapter %s" % last_chapter)
if any("auditor" in line and line.startswith("## 附录角色:") for line in tlines):
errors.append("target already contains an auditor appendix chapter")
for marker in MARKERS[target_id]:
if marker not in target_text:
errors.append("preserved marker missing in target: %s" % marker)
refs = [i + 1 for i, line in enumerate(tlines) if "templates/artifacts" in line]
print("anchors: source hunks OK (matrix + main %d lines + 2 checklist "
"items + appendix %d lines)"
% (len(hunks["main"]), len(hunks["appendix_body"])))
print("anchors: templates/artifacts/ references in target: %d at lines %s "
"(expected: 6 sites, removed at transplant time per decision B)"
% (len(refs), refs))
print("anchors: appendix chapter number for %s: %s"
% (target_id, "11-auditor" if target_id == "v05" else "09-auditor"))
return errors
def check_transplanted(target_text, transplanted_text, target_id):
errors = []
for marker in MARKERS[target_id]:
if marker not in transplanted_text:
errors.append("preserved marker lost: %s" % marker)
rows = [line for line in _lines(transplanted_text)
if line.startswith("| 独立审计 |")]
if len(rows) != 1:
errors.append("auditor matrix row: expected 1, found %d" % len(rows))
for header in MAIN_HEADERS:
if header not in transplanted_text:
errors.append("main header missing: %s" % header)
if target_id == "v05":
if QA_HEADER in transplanted_text:
errors.append("STOPPED QA section present in v05 output")
if "- QA 全链路走了吗?" in transplanted_text:
errors.append("STOPPED QA checklist item present in v05 output")
if "验收失败导致的推倒要先通知用户" in transplanted_text:
errors.append("STOPPED tighten3 notify present in v05 output")
elif target_id == "v04":
if QA_HEADER not in transplanted_text:
errors.append("QA section missing in v04 output")
if "验收失败导致的推倒要先通知用户" in transplanted_text:
errors.append("STOPPED tighten3 notify present in v04 output")
if "G7 热修复回路并行" in transplanted_text:
errors.append("STOPPED G7 sentence present in v04 output")
else: # v03
if QA_HEADER not in transplanted_text:
errors.append("QA section missing in v03 output")
if "G7 热修复回路并行" in transplanted_text:
errors.append("STOPPED G7 sentence present in v03 output")
if "全自动 CI 自动修复" in transplanted_text:
errors.append("STOPPED full-auto sentence present in v03 output")
if "验收失败导致的推倒要先通知用户" not in transplanted_text:
errors.append("tighten3 notify missing in v03 output")
number = "11" if target_id == "v05" else "09"
heading = ("## 附录角色:%s-auditor(来源 templates/roles/%s-auditor.md)"
% (number, number))
if transplanted_text.count(heading) != 1:
errors.append("appendix heading %s: expected 1, found %d"
% (heading, transplanted_text.count(heading)))
if "09-auditor(独立审计,新增)" in transplanted_text:
errors.append("old appendix heading form survived")
for stale in ["移植警告", "以生产 v0.2 技能为准",
"技能正文 diff 全是加法"]:
if stale in transplanted_text:
errors.append("stale transplant warning survived: %s" % stale)
if "templates/artifacts" in transplanted_text:
errors.append("templates/artifacts/ reference survived "
"(decision B not applied)")
# No-overwrite: the target, after decision-B replacements, must be a
# subsequence (in order) of the transplanted text.
transformed = target_text
for old, new in ARTIFACT_REPLACEMENTS:
transformed = transformed.replace(old, new)
t_iter = iter(_lines(transplanted_text))
remaining = _lines(transformed)
pos = 0
for line in t_iter:
if pos < len(remaining) and line == remaining[pos]:
pos += 1
if pos != len(remaining):
errors.append("no-overwrite check failed: target line %d (%r) "
"missing or out of order in transplanted text"
% (pos + 1, remaining[pos][:80] if pos < len(remaining)
else ""))
return errors
def main(argv=None):
parser = argparse.ArgumentParser(
description="Transplant preparation checker for #244 -> v0.3/v0.4/v0.5.")
sub = parser.add_subparsers(dest="cmd", required=True)
for name in ["anchors", "apply", "check-transplanted"]:
p = sub.add_parser(name)
p.add_argument("--source244", required=True)
p.add_argument("--target", required=True)
p.add_argument("--target-id", required=True,
choices=["v03", "v04", "v05"])
if name == "apply":
p.add_argument("--out", default=None)
if name == "check-transplanted":
p.add_argument("--transplanted", required=True)
args = parser.parse_args(argv)
try:
source = Path(args.source244).read_text(encoding="utf-8")
target = Path(args.target).read_text(encoding="utf-8")
except OSError as exc:
print("FAIL: cannot read input: %s" % exc, file=sys.stderr)
return 2
try:
if args.cmd == "anchors":
errors = check_anchors(source, target, args.target_id)
if errors:
for err in errors:
print("FAIL: " + err, file=sys.stderr)
print("anchors FAILED: %d problem(s)" % len(errors),
file=sys.stderr)
return 1
print("anchors OK for %s" % args.target_id)
return 0
if args.cmd == "apply":
draft, report = apply_transplant(source, target, args.target_id)
if args.out:
Path(args.out).write_text(draft, encoding="utf-8")
else:
sys.stdout.write(draft)
for line in report:
print("apply: " + line, file=sys.stderr)
return 0
transplanted = Path(args.transplanted).read_text(encoding="utf-8")
errors = check_transplanted(target, transplanted, args.target_id)
if errors:
for err in errors:
print("FAIL: " + err, file=sys.stderr)
print("check-transplanted FAILED: %d problem(s)" % len(errors),
file=sys.stderr)
return 1
print("check-transplanted OK for %s" % args.target_id)
return 0
except ValueError as exc:
print("FAIL: %s" % exc, file=sys.stderr)
return 2
if __name__ == "__main__":
raise SystemExit(main())
+2 -2
View File
@@ -120,7 +120,7 @@ def get_user_gateway_key(conn, user_id):
# 每次新建实例都要登录一次(实测约 111ms,占开通链路约 1/3),
# 共享后同一进程内第二次起免登录。401/刷新失败会回落到重登录,
# 密码轮换后下一次登录即用新密码刷新缓存,不会卡死。
_SHARED_ADMIN = {}
_SHARED_ADMIN: dict[tuple[str, str], tuple[str | None, str | None, float]] = {}
_SHARED_ADMIN_LOCK = threading.RLock()
@@ -201,7 +201,7 @@ def _call_with_login_retry(fn):
# Redis 节流,控制面目前没有 Redis 客户端,未做)。
_LOGIN_BUDGET_PER_MINUTE = 15
_LOGIN_WINDOW_SECONDS = 60.0
_login_slots = collections.deque()
_login_slots: collections.deque[float] = collections.deque()
_login_slots_lock = threading.Lock()
+221
View File
@@ -0,0 +1,221 @@
"""Tests for cloud.arch_conformance: architecture consistency checks.
Every check has a positive and a negative case; the repo-level test
pins that the current tree passes the layer bans, so a future
violation fails CI (the existing host-verify workflow runs
``discover -s cloud/tests``; a failure here is a CI failure).
Only the standard library; no real model, no network, no secrets.
"""
import os
import sys
import tempfile
import unittest
from pathlib import Path
sys.path.insert(0, os.path.dirname(os.path.dirname(
os.path.dirname(os.path.abspath(__file__)))))
from cloud.arch_conformance import ( # noqa: E402
check_contract_version, check_frozen_stack, check_layer_imports,
check_skill_project, check_task_writes, is_within,
scan_cloud_package, scan_imports)
class TestContractVersion(unittest.TestCase):
def test_equal_versions_pass(self):
self.assertTrue(check_contract_version("v1", "v1")["ok"])
def test_approved_step_bump_passes(self):
change = {"approved": True, "backward_compatible": True,
"affected_clients": ["web"]}
self.assertTrue(
check_contract_version("v1", "v2", change=change)["ok"])
def test_breaking_bump_needs_regression_task(self):
change = {"approved": True, "backward_compatible": False,
"affected_clients": ["web"]}
verdict = check_contract_version("v1", "v2", change=change)
self.assertFalse(verdict["ok"])
self.assertTrue(any("regression" in r for r in verdict["reasons"]))
def test_breaking_bump_with_regression_task_passes(self):
change = {"approved": True, "backward_compatible": False,
"affected_clients": ["web"], "regression_task": "t-1"}
self.assertTrue(
check_contract_version("v1", "v2", change=change)["ok"])
def test_unapproved_bump_fails(self):
change = {"approved": False, "backward_compatible": True,
"affected_clients": ["web"]}
self.assertFalse(
check_contract_version("v1", "v2", change=change)["ok"])
def test_bump_without_change_record_fails(self):
self.assertFalse(check_contract_version("v1", "v2")["ok"])
def test_skip_bump_fails_even_when_approved(self):
change = {"approved": True, "backward_compatible": True,
"affected_clients": ["web"]}
verdict = check_contract_version("v1", "v3", change=change)
self.assertFalse(verdict["ok"])
def test_unparseable_version_fails(self):
self.assertFalse(check_contract_version("v1", "draft")["ok"])
class TestFrozenStack(unittest.TestCase):
def test_unchanged_stack_passes(self):
frozen = {"web": "React", "ios": "SwiftUI"}
self.assertTrue(check_frozen_stack(frozen, dict(frozen))["ok"])
def test_new_client_is_addition_not_swap(self):
frozen = {"web": "React"}
current = {"web": "React", "ios": "SwiftUI"}
self.assertTrue(check_frozen_stack(frozen, current)["ok"])
def test_swap_without_approval_fails(self):
verdict = check_frozen_stack({"web": "React"}, {"web": "Vue"})
self.assertFalse(verdict["ok"])
self.assertEqual(verdict["swapped"], ["web"])
def test_removed_client_without_approval_fails(self):
self.assertFalse(
check_frozen_stack({"web": "React", "ios": "SwiftUI"},
{"web": "React"})["ok"])
def test_swap_with_approved_change_passes(self):
change = {"approved": True, "backward_compatible": True,
"affected_clients": ["web"]}
self.assertTrue(
check_frozen_stack({"web": "React"}, {"web": "Vue"},
change=change)["ok"])
class TestDirectoryGuard(unittest.TestCase):
def test_inside_passes_and_outside_fails(self):
with tempfile.TemporaryDirectory() as tmp:
own = os.path.join(tmp, "work")
os.makedirs(own)
self.assertTrue(is_within(own, os.path.join(own, "a.txt")))
self.assertTrue(is_within(own, own))
self.assertFalse(is_within(own, os.path.join(tmp, "other")))
def test_dotdot_escape_fails(self):
with tempfile.TemporaryDirectory() as tmp:
own = os.path.join(tmp, "work")
os.makedirs(own)
escape = os.path.join(own, "..", "secret.txt")
self.assertFalse(is_within(own, escape))
self.assertEqual(check_task_writes(own, [escape]), [escape])
def test_symlink_escape_fails(self):
with tempfile.TemporaryDirectory() as tmp:
own = os.path.join(tmp, "work")
outside = os.path.join(tmp, "outside")
os.makedirs(own)
os.makedirs(outside)
link = os.path.join(own, "link")
try:
os.symlink(outside, link)
except OSError:
self.skipTest("symlinks unavailable")
self.assertFalse(is_within(own, link))
def test_mixed_writes_report_only_violations(self):
with tempfile.TemporaryDirectory() as tmp:
own = os.path.join(tmp, "work")
os.makedirs(own)
good = os.path.join(own, "ok.txt")
bad = os.path.join(tmp, "bad.txt")
self.assertEqual(check_task_writes(own, [good, bad]), [bad])
class TestLayerImports(unittest.TestCase):
def test_foundation_importing_server_fails(self):
verdict = check_layer_imports(
{"db": "from cloud.server import foo\n"})
self.assertEqual(len(verdict), 1)
self.assertEqual(verdict[0]["rule"],
"foundation_must_not_import_orchestration")
def test_foundation_importing_store_passes(self):
# db importing nothing, models importing nothing: the baseline
# shape. A foundation module importing another foundation
# module is not an upward import.
self.assertEqual(check_layer_imports(
{"models": "import os\n", "queue": "from cloud.models import x\n"}),
[])
def test_sandbox_importing_db_fails(self):
verdict = check_layer_imports(
{"sandbox": "from cloud.db import connect\n"})
self.assertEqual(len(verdict), 1)
def test_sandbox_stdlib_only_passes(self):
self.assertEqual(check_layer_imports(
{"sandbox": "import os\nimport subprocess\n"}), [])
def test_service_opening_sqlite_directly_fails(self):
verdict = check_layer_imports(
{"server": "import sqlite3\nconn = sqlite3.connect('x.db')\n"})
self.assertTrue(any(v["rule"] == "service_must_not_open_driver_"
"connection_directly" for v in verdict))
def test_service_sqlite_row_adapter_passes(self):
# Deliberate narrow ban: Row/exception adapters on a
# cloud.db connection are adaptation, not a bypass.
self.assertEqual(check_layer_imports(
{"server": "import sqlite3\nconn.row_factory = sqlite3.Row\n"
"try:\n pass\nexcept sqlite3.IntegrityError:\n"
" pass\n"}), [])
def test_service_importing_psycopg_fails(self):
verdict = check_layer_imports(
{"worker": "import psycopg\n"})
self.assertTrue(any(v["rule"] == "service_must_not_open_driver_"
"connection_directly" for v in verdict))
def test_driver_adjacent_modules_exempt(self):
self.assertEqual(check_layer_imports(
{"backup": "import sqlite3\nsqlite3.connect('x.db')\n",
"migrations": "import sqlite3\nsqlite3.connect('y.db')\n"}), [])
def test_scan_imports_sees_from_cloud_form(self):
scanned = scan_imports("from cloud.worker import Worker\n")
self.assertIn("worker", scanned["cloud"])
class TestRepoLevelScan(unittest.TestCase):
def test_current_tree_has_zero_layer_violations(self):
repo_root = Path(__file__).resolve().parents[2]
violations = scan_cloud_package(repo_root)
self.assertEqual(violations, [])
class TestSkillProjectFixture(unittest.TestCase):
def test_frozen_project_passes_all_checks(self):
with tempfile.TemporaryDirectory() as tmp:
own = os.path.join(tmp, "apps", "web")
os.makedirs(own)
verdict = check_skill_project(
tmp, "v1", "v1", {"web": "React"}, {"web": "React"},
own, [os.path.join(own, "App.tsx")])
self.assertTrue(verdict["ok"])
def test_drifting_project_fails_each_changed_check(self):
with tempfile.TemporaryDirectory() as tmp:
own = os.path.join(tmp, "apps", "web")
os.makedirs(own)
verdict = check_skill_project(
tmp, "v1", "v2", {"web": "React"}, {"web": "Vue"},
own, [os.path.join(tmp, "apps", "ios", "leak.txt")])
self.assertFalse(verdict["ok"])
self.assertFalse(verdict["contract"]["ok"])
self.assertFalse(verdict["stack"]["ok"])
self.assertFalse(verdict["directory"]["ok"])
if __name__ == "__main__":
unittest.main()
+371
View File
@@ -0,0 +1,371 @@
"""架构收敛 A 线单测:超时 / 门禁 / 成本配额 / 沙箱清理。
只覆盖本轨新增的四个叶子模块与调用点的委托关系,不跑
模型、不联网、不碰生产:SQLite 内存库 + 临时目录。
钉住的冻结语义(改动这些断言即改语义,必须先改设计):
- 审批默认 3600、服务端入口 60..86400、已建项 expires_at
永不因默认值变化而重算;600 秒是单轮护栏,不是审批超时。
- 目录门无 confirm / override / bypass 参数;合并状态门
仅 completed 放行;控制门与旧 marker 行为逐字一致。
- 配额判定唯一来源是 quota.decide:USD 门先于无价门,
命中均为 >=;无价只计 source='actual',估算行不触发上限;
无配额行按默认值,显式 NULL 行才是不限额。
- 失联清理只在 interrupted + 心跳停滞严格超阈时放行;
容器清理只认 codex-<32 位 hex>,绝不按前缀删。
设计变更 2026-11-02(11 月日期失败修复,code-to10 v5):
- 旧声明:本文件固件日期一律写死(_cost_row 默认
day="2026-10-31"),未区分“按当月查用量”与“按指定月
查用量”两类测试。
- 新声明:按当前 UTC 月查用量的门禁测试,固件日期必须
跟随当前 UTC 月(_current_month_day(),当月 1 日);
按指定月查用量的测试仍用写死日期 + 显式月份查询。
- 为什么改:门禁(automations._automation_quota_exceeded)
按当前 UTC 月查用量,写死 2026-10-31 的固件在
2026-11-01 起恒败(10 月行计为 0,门禁返回 None);
失败的是固件不是门禁逻辑;断言与门禁语义一字未改。
"""
import inspect
import os
import sqlite3
import sys
import tempfile
import unittest
from datetime import datetime, timezone
from pathlib import Path
from unittest import mock
try:
from cloud import gates, quota, sandbox_policy, timeouts
from cloud.approvals import DEFAULT_TIMEOUT_SECONDS, create_approval
from cloud.automations import (
_automation_quota_exceeded,
_unpriced_token_cap as _auto_cap,
)
from cloud.coordinator import DEFAULT_TIMEOUT_SECONDS as _COORD_TIMEOUT
from cloud.cost import unpriced_token_cap as _cost_cap
from cloud.harness2_tasks import (
DEFAULT_TASK_TIMEOUT_SECONDS as _HARNESS_TIMEOUT,
)
from cloud.marker_tools import (
MAX_POLL_TIMEOUT_SECONDS as _MARKER_POLL_MAX,
)
from cloud.marker_tools import check_control_gate, safe_worktree_path
from cloud.sandbox import DockerExecutor
from cloud.server import _safe_worktree_path as _server_safe_path
from cloud.tree import resolve_tree_path
from cloud.workers import DEFAULT_ALIVE_SECONDS as _WORKER_ALIVE
except ImportError: # discover -s cloud/tests 时把项目根目录补进 sys.path
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from cloud import gates, quota, sandbox_policy, timeouts
from cloud.approvals import DEFAULT_TIMEOUT_SECONDS, create_approval
from cloud.automations import (
_automation_quota_exceeded,
_unpriced_token_cap as _auto_cap,
)
from cloud.coordinator import DEFAULT_TIMEOUT_SECONDS as _COORD_TIMEOUT
from cloud.cost import unpriced_token_cap as _cost_cap
from cloud.harness2_tasks import (
DEFAULT_TASK_TIMEOUT_SECONDS as _HARNESS_TIMEOUT,
)
from cloud.marker_tools import (
MAX_POLL_TIMEOUT_SECONDS as _MARKER_POLL_MAX,
)
from cloud.marker_tools import check_control_gate, safe_worktree_path
from cloud.sandbox import DockerExecutor
from cloud.server import _safe_worktree_path as _server_safe_path
from cloud.tree import resolve_tree_path
from cloud.workers import DEFAULT_ALIVE_SECONDS as _WORKER_ALIVE
def _quota_conn():
"""配额测试用的最小库:tasks + cost_records + user_quotas。"""
conn = sqlite3.connect(":memory:")
conn.row_factory = sqlite3.Row
conn.execute("CREATE TABLE tasks(id TEXT PRIMARY KEY,"
" created_by TEXT)")
conn.execute("CREATE TABLE cost_records(task_id TEXT,"
" usd_micros INTEGER, input_tokens INTEGER,"
" output_tokens INTEGER, day TEXT, source TEXT)")
conn.execute("CREATE TABLE user_quotas(user_id TEXT PRIMARY KEY,"
" quota_usd_micros BIGINT, updated_at DOUBLE PRECISION)")
conn.execute("INSERT INTO tasks(id, created_by) VALUES('t1', 'u1')")
conn.commit()
return conn
def _cost_row(conn, task_id, usd, inp, out, source, day="2026-10-31"):
conn.execute(
"INSERT INTO cost_records(task_id, usd_micros, input_tokens,"
" output_tokens, day, source) VALUES(?, ?, ?, ?, ?, ?)",
(task_id, usd, inp, out, day, source))
conn.commit()
def _current_month_day():
"""门禁测试用固件日期:当前 UTC 月的 1 日。
门禁按当前 UTC 月查用量,固件日期必须落在当月;写死
某月某日会在月份滚动后恒败(2026-11 实测)。只用于
按当月查用量的门禁测试;按指定月查用量的测试(如
test_monthly_usage_actual_only)仍用写死的 2026-10-31
固件 + 显式 "2026-10" 查询,不受影响。
"""
return datetime.now(timezone.utc).strftime("%Y-%m-01")
class TimeoutConvergeTestCase(unittest.TestCase):
def test_constants_match_old_literals(self):
self.assertEqual(timeouts.APPROVAL_DEFAULT_SECONDS, 3600)
self.assertEqual(timeouts.APPROVAL_MIN_SECONDS, 60)
self.assertEqual(timeouts.APPROVAL_MAX_SECONDS, 86400)
self.assertEqual(timeouts.TURN_SECONDS, 600)
self.assertEqual(timeouts.STALE_SWEEP_SECONDS, 300)
self.assertEqual(timeouts.WORKER_ALIVE_SECONDS, 120)
# 各模块别名与单一来源同值(调用点 import 的名字不变)。
self.assertEqual(DEFAULT_TIMEOUT_SECONDS, 3600)
self.assertEqual(_COORD_TIMEOUT, timeouts.LONG_TIMEOUT_SECONDS)
self.assertEqual(_HARNESS_TIMEOUT, timeouts.LONG_TIMEOUT_SECONDS)
self.assertEqual(_MARKER_POLL_MAX, timeouts.LONG_TIMEOUT_SECONDS)
self.assertEqual(_WORKER_ALIVE, timeouts.WORKER_ALIVE_SECONDS)
def test_server_validation_keeps_old_bounds_and_message(self):
for good in (60, 3600, 86400):
self.assertEqual(timeouts.validate_server_timeout(good), good)
for bad in (None, True, 59, 86401, 3.5, "3600"):
with self.assertRaises(ValueError) as ctx:
timeouts.validate_server_timeout(bad)
self.assertEqual(str(ctx.exception),
"timeout_seconds must be 60..86400")
def test_module_timeout_semantics(self):
self.assertEqual(timeouts.resolve_module_timeout(None), 3600)
self.assertIsNone(timeouts.resolve_module_timeout(0))
self.assertEqual(timeouts.resolve_module_timeout(60), 60)
for bad in (-1, True, "60"):
with self.assertRaises(ValueError):
timeouts.resolve_module_timeout(bad)
self.assertIsNone(timeouts.expires_at_for(1000, None))
self.assertEqual(timeouts.expires_at_for(1000, 3600), 4600)
def test_env_override_and_invalid_fallback(self):
with mock.patch.dict(os.environ,
{"CLOUD_APPROVAL_TIMEOUT_SECONDS": "1800"}):
self.assertEqual(timeouts.approval_default_seconds(), 1800)
with mock.patch.dict(os.environ,
{"CLOUD_APPROVAL_TIMEOUT_SECONDS": "30"}):
self.assertEqual(timeouts.approval_default_seconds(), 3600)
with mock.patch.dict(os.environ,
{"CLOUD_APPROVAL_TIMEOUT_SECONDS": "abc"}):
self.assertEqual(timeouts.approval_default_seconds(), 3600)
def test_existing_approval_expires_at_never_recomputed(self):
conn = sqlite3.connect(":memory:")
conn.row_factory = sqlite3.Row
try:
with mock.patch.dict(os.environ, {}, clear=False):
os.environ.pop("CLOUD_APPROVAL_TIMEOUT_SECONDS", None)
old = create_approval(conn, "t1", "p1", summary="s",
now=1000.0)
self.assertEqual(old["expires_at"], 4600.0)
with mock.patch.dict(os.environ,
{"CLOUD_APPROVAL_TIMEOUT_SECONDS": "60"}):
new = create_approval(conn, "t1", "p1", summary="s",
now=2000.0)
self.assertEqual(new["expires_at"], 2060.0)
# 旧行不受默认值变化影响:不回填、不重算。
from cloud.approvals import get_approval
self.assertEqual(get_approval(conn, old["id"])["expires_at"],
4600.0)
finally:
conn.close()
class GatesConvergeTestCase(unittest.TestCase):
def test_path_gate_cases_across_all_wrappers(self):
with tempfile.TemporaryDirectory() as tmp:
base = os.path.join(tmp, "w")
os.makedirs(os.path.join(base, "sub"))
# 目录内指回内部的符号链接放行,指出去的拦住。
os.symlink(os.path.join(base, "sub"),
os.path.join(base, "inner-link"))
os.symlink("/etc", os.path.join(base, "escape-link"))
for fn in (gates.resolve_inside, resolve_tree_path,
safe_worktree_path, _server_safe_path):
self.assertEqual(fn(base, ""), base)
self.assertEqual(fn(base, "sub"),
os.path.realpath(os.path.join(base, "sub")))
self.assertIsNone(fn(base, "../etc/passwd"))
self.assertIsNone(fn(base, "/etc/passwd"))
self.assertIsNotNone(fn(base, "inner-link"))
self.assertIsNone(fn(base, "escape-link"))
def test_path_and_control_gates_have_no_bypass_parameter(self):
for fn in (gates.resolve_inside, gates.decide_control,
gates.merge_state_decision):
params = set(inspect.signature(fn).parameters)
self.assertFalse(params & {"confirm", "override", "bypass",
"force", "confirmed"})
def test_merge_state_gate(self):
self.assertEqual(gates.merge_state_decision("completed"),
(True, ""))
for status in ("queued", "running", "failed", "interrupted",
"merged", "merge-failed"):
ok, reason = gates.merge_state_decision(status)
self.assertFalse(ok)
self.assertEqual(reason,
"only completed tasks can be marked merged")
def test_control_gate_matches_old_marker_behavior(self):
sender = {"id": "s1", "project_id": "p1"}
target = {"id": "t1", "project_id": "p1", "status": "running"}
self.assertEqual(check_control_gate(target, sender, "descendant",
"cancel"), (True, ""))
self.assertEqual(
check_control_gate(target, sender, "descendant", "retry")[0],
False)
target2 = dict(target, status="failed")
self.assertEqual(check_control_gate(target2, sender, "same_creator",
"retry"), (True, ""))
self.assertEqual(
check_control_gate(target2, sender, "other", "retry")[0], False)
self.assertEqual(
check_control_gate(target2, {"id": "t1", "project_id": "p1"},
"descendant", "cancel")[0], False)
goal = dict(target2, goal_mode=1)
self.assertEqual(
check_control_gate(goal, sender, "descendant", "retry")[0],
False)
cross = dict(target2, project_id="p2")
self.assertEqual(
check_control_gate(cross, sender, "descendant", "retry")[0],
False)
class QuotaConvergeTestCase(unittest.TestCase):
def test_decide_precedence_and_equality(self):
self.assertIsNone(quota.decide(None, 999999, 999999, 100, "2026-10"))
self.assertIsNone(quota.decide(1000, 999, 99, 100, "2026-10"))
both = quota.decide(1000, 1000, 1000, 100, "2026-10")
self.assertEqual(both["kind"], "usd")
only_unpriced = quota.decide(1000, 10, 100, 100, "2026-10")
self.assertEqual(only_unpriced["kind"], "unpriced")
self.assertIn("quota_exceeded", only_unpriced["message"])
def test_quota_for_owner_rows(self):
conn = _quota_conn()
try:
with mock.patch.dict(os.environ, {}, clear=False):
os.environ.pop("CLOUD_DEFAULT_QUOTA_USD_MICROS", None)
self.assertEqual(quota.quota_for_owner(conn, "u1"),
10000000)
conn.execute("INSERT INTO user_quotas VALUES('u1', NULL, 0)")
conn.commit()
self.assertIsNone(quota.quota_for_owner(conn, "u1"))
conn.execute("UPDATE user_quotas SET quota_usd_micros = 5000"
" WHERE user_id = 'u1'")
conn.commit()
self.assertEqual(quota.quota_for_owner(conn, "u1"), 5000)
finally:
conn.close()
def test_monthly_usage_actual_only(self):
conn = _quota_conn()
try:
_cost_row(conn, "t1", 300, 10, 5, "actual")
_cost_row(conn, "t1", None, 100, 50, "actual")
_cost_row(conn, "t1", None, 999900, 99, "estimated")
used, unpriced = quota.monthly_usage(conn, "u1", "2026-10")
self.assertEqual(used, 300)
# 估算行 999999 token 不计入无价上限口径。
self.assertEqual(unpriced, 150)
finally:
conn.close()
def test_automation_gate_uses_actual_only(self):
conn = _quota_conn()
try:
conn.execute("INSERT INTO user_quotas VALUES('u1', 100000000, 0)")
conn.commit()
_cost_row(conn, "t1", None, 999900, 99, "estimated",
day=_current_month_day())
with mock.patch.dict(
os.environ,
{"CLOUD_QUOTA_UNPRICED_TOKEN_CAP": "1000"}):
# 估算行再大也不拦。
self.assertIsNone(_automation_quota_exceeded(conn, "u1"))
_cost_row(conn, "t1", None, 600, 500, "actual",
day=_current_month_day())
blocked = _automation_quota_exceeded(conn, "u1")
self.assertIsNotNone(blocked)
self.assertIn("quota_exceeded", blocked)
finally:
conn.close()
def test_unpriced_cap_env_alias_matches_cost(self):
with mock.patch.dict(
os.environ, {"CLOUD_QUOTA_UNPRICED_TOKEN_CAP": "12345"}):
self.assertEqual(_auto_cap(), 12345)
self.assertEqual(_cost_cap(), 12345)
with mock.patch.dict(
os.environ, {"CLOUD_QUOTA_UNPRICED_TOKEN_CAP": "junk"}):
self.assertEqual(_auto_cap(), _cost_cap())
class SandboxPolicyTestCase(unittest.TestCase):
def test_tmpfs_matrix(self):
self.assertTrue(sandbox_policy.tmpfs_isolated("docker", "/tmp")[0])
self.assertTrue(
sandbox_policy.tmpfs_isolated("docker", "/var/tmp")[0])
self.assertFalse(
sandbox_policy.tmpfs_isolated("host-bwrap", "/tmp")[0])
self.assertFalse(sandbox_policy.tmpfs_isolated("local", "/tmp")[0])
self.assertFalse(
sandbox_policy.tmpfs_isolated("docker", "/work")[0])
def test_task_container_regex_shared_with_sandbox(self):
self.assertIs(sandbox_policy.TASK_CONTAINER_RE,
DockerExecutor._TASK_CONTAINER_RE)
good = "codex-" + "a" * 32
self.assertTrue(sandbox_policy.TASK_CONTAINER_RE.match(good))
for bad in ("codex-postgres", "codex-xyz", "codex-" + "a" * 31,
"codex-" + "A" * 32):
self.assertIsNone(sandbox_policy.TASK_CONTAINER_RE.match(bad))
def test_cleanup_decision(self):
now = 10000.0
self.assertEqual(
sandbox_policy.cleanup_decision("running", 1.0, now, 300)[0],
False)
self.assertEqual(
sandbox_policy.cleanup_decision("interrupted", now - 301,
now, 300)[0], True)
# 恰等于阈值不杀(旧 _reap:hb >= now - threshold 即跳过)。
self.assertEqual(
sandbox_policy.cleanup_decision("interrupted", now - 300,
now, 300)[0], False)
self.assertEqual(
sandbox_policy.cleanup_decision("interrupted", now - 10,
now, 300)[0], False)
self.assertEqual(
sandbox_policy.cleanup_decision("interrupted", None,
now, 300)[0], True)
self.assertEqual(
sandbox_policy.cleanup_decision("interrupted", "junk",
now, 300)[0], False)
def test_worker_address(self):
self.assertEqual(sandbox_policy.worker_address("my-host-42"),
("my-host", 42))
self.assertEqual(sandbox_policy.worker_address("plain"),
("plain", None))
self.assertEqual(sandbox_policy.worker_address("host-abc"),
("host-abc", None))
self.assertEqual(sandbox_policy.worker_address(""), ("", None))
if __name__ == "__main__":
unittest.main()
+727
View File
@@ -0,0 +1,727 @@
"""Tests for the task-attempt lifecycle hook (audit gap 3
follow-up).
Only the standard library; temporary SQLite databases, a fake
CLI runner, and a temporary HTTP server for the endpoint test.
No real model, no network, no secrets, no real Postgres (the
Postgres claims are adapter-level only, labelled as such).
Pinned policy (see ``cloud/attempt_hook.py`` -- the module is
canonical, these tests pin its observable behaviour):
- classify: coordinator never counts; a platform marker beats
every other signal; a breakpoint resume counts as the same
logical round's row (not an extra attempt); a continue message
counts as a new round; first dispatch / human retry count.
- markers: consume only matches the queued generation; a stale
marker is deleted, never misapplied; an unverifiable
generation consumes nothing.
- corrections: append-only, last-wins, never counted as an
attempt, cannot change an outcome, cannot target interrupted
rows / correction rows / nonexistent attempts.
- worker: hook OFF writes nothing and creates no table (gap-3
behaviour); hook ON writes exactly one row per counted
dispatch (a transient retry inside one dispatch is still one
row); a platform rerun writes no row; a gate failure writes
no row; a ledger write failure never fails the task.
- endpoint: with ledger tasks + old tasks + a platform-completed
task + an interrupted task side by side, terminal denominators
contain only known tasks, interrupted status is non-terminal,
interrupted rows count for nothing, and unknown rates are
None + *_known False, never 0; the cost dual-column discipline
is unchanged (priced partial shown, total None when incomplete,
unpriced never folded to 0).
"""
import http.client
import json
import os
import sqlite3
import sys
import tempfile
import threading
import time
import unittest
from pathlib import Path
from types import SimpleNamespace
try:
from cloud.attempt_hook import (
MARKER_DDL,
PG_MARKER_DDL,
classify_dispatch,
consume_platform_marker,
mark_platform_rerun,
)
from cloud.attempt_ledger import (
OLD_TABLE_ALTER,
PG_DDL,
aggregate_attempts,
apply_to_records,
read_attempt_rows,
record_attempt,
record_correction,
)
from cloud.db import _convert_placeholders, _translate_sql
from cloud.metrics import compute_metrics
except ImportError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from cloud.attempt_hook import (
MARKER_DDL,
PG_MARKER_DDL,
classify_dispatch,
consume_platform_marker,
mark_platform_rerun,
)
from cloud.attempt_ledger import (
OLD_TABLE_ALTER,
PG_DDL,
aggregate_attempts,
apply_to_records,
read_attempt_rows,
record_attempt,
record_correction,
)
from cloud.db import _convert_placeholders, _translate_sql
from cloud.metrics import compute_metrics
def _sqlite_conn(path):
from cloud.db import connect as db_connect
return db_connect(path)
def _insert_task(db_path, task_id, prompt="do work", status="queued",
project_id="p1", **extra):
now = time.time()
cols = ["id", "project_id", "prompt", "status",
"idempotency_key", "depends_on", "priority", "created_by",
"created_at", "updated_at", "last_heartbeat"]
vals = [task_id, project_id, prompt, status, None, "[]", 0,
"tester", now, now, None]
for key, value in extra.items():
cols.append(key)
vals.append(value)
conn = sqlite3.connect(db_path)
try:
conn.execute(
"INSERT INTO tasks(%s) VALUES(%s)"
% (", ".join(cols), ", ".join("?" * len(cols))),
vals)
conn.commit()
finally:
conn.close()
def _task_status(db_path, task_id):
conn = sqlite3.connect(db_path)
try:
row = conn.execute(
"SELECT status FROM tasks WHERE id = ?",
(task_id,)).fetchone()
finally:
conn.close()
return row[0] if row else None
def _ledger_rows(db_path):
conn = sqlite3.connect(db_path)
try:
rows = conn.execute(
"SELECT task_id, attempt_no, outcome, is_defect, "
"defect_category, corrects_attempt_no FROM task_attempts "
"ORDER BY task_id, attempt_no").fetchall()
except sqlite3.OperationalError:
return []
finally:
conn.close()
return [tuple(r) for r in rows]
def _table_exists(db_path, table):
conn = sqlite3.connect(db_path)
try:
row = conn.execute(
"SELECT COUNT(*) FROM sqlite_master WHERE type='table' "
"AND name=?", (table,)).fetchone()
finally:
conn.close()
return row[0] == 1
def _completed_runner(cmd):
"""Fake CLI: dispatch/run always succeed, host.py-style JSON."""
if "dispatch" in cmd:
name = cmd[cmd.index("--name") + 1]
else:
name = cmd[cmd.index("--task") + 1]
return SimpleNamespace(
returncode=0,
stdout=json.dumps({"task": name, "status": "completed",
"text": "done"}),
stderr="")
class TestClassifyDispatch(unittest.TestCase):
def test_coordinator_beats_marker(self):
decision = classify_dispatch(
{"coordinator_parts": [{"name": "a"}]}, True)
self.assertFalse(decision["counted"])
self.assertEqual(decision["kind"], "coordinator_orchestration")
def test_platform_marker_beats_breakpoint(self):
# A platform goal auto-resume is a breakpoint continuation
# AND a platform rerun: uniform platform exclusion wins.
decision = classify_dispatch(
{"_goal_resume_breakpoint": True}, True)
self.assertFalse(decision["counted"])
self.assertEqual(decision["kind"], "platform_auto_resume")
def test_human_breakpoint_resume_counts(self):
decision = classify_dispatch(
{"_goal_resume_breakpoint": True}, False)
self.assertTrue(decision["counted"])
self.assertEqual(decision["kind"], "breakpoint_resume")
def test_resume_kind_breakpoint_counts_despite_message(self):
# Old-verify goal + /continue: the message is NOT a new
# goal, the loop continues from its breakpoint.
decision = classify_dispatch(
{"resume_message": "keep going",
"_resume_kind": "breakpoint"}, False)
self.assertTrue(decision["counted"])
self.assertEqual(decision["kind"], "breakpoint_resume")
def test_breakpoint_fallback_is_not_a_resume(self):
# Resolution fell back to a fresh goal run: the pending
# flag alone must not classify it as a resume.
decision = classify_dispatch(
{"goal_resume_pending": 1,
"_goal_resume_breakpoint": False}, False)
self.assertTrue(decision["counted"])
self.assertEqual(decision["kind"], "first_or_human_retry")
def test_new_goal_continue_counts_as_new_round(self):
decision = classify_dispatch(
{"resume_message": "new goal",
"_resume_kind": "new-goal"}, False)
self.assertTrue(decision["counted"])
self.assertEqual(decision["kind"], "human_continue")
def test_plain_continue_counts(self):
decision = classify_dispatch(
{"resume_message": "hello"}, False)
self.assertTrue(decision["counted"])
self.assertEqual(decision["kind"], "human_continue")
def test_first_dispatch_counts(self):
decision = classify_dispatch({}, False)
self.assertTrue(decision["counted"])
self.assertEqual(decision["kind"], "first_or_human_retry")
class TestMarkerStore(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
self.conn = _sqlite_conn(self.db_path)
def tearDown(self):
self.conn.close()
self.tmp.cleanup()
def _markers(self):
try:
return self.conn.execute(
"SELECT task_id, origin, queued_at FROM "
"task_attempt_origin_markers").fetchall()
except Exception:
return []
def test_mark_then_consume_matching(self):
mark_platform_rerun(self.conn, "t-1", 1234.5)
self.conn.commit()
self.assertTrue(consume_platform_marker(self.conn, "t-1", 1234.5))
self.conn.commit()
self.assertEqual(self._markers(), [])
# Consuming twice never matches twice.
self.assertFalse(
consume_platform_marker(self.conn, "t-1", 1234.5))
def test_mismatched_marker_is_stale_and_deleted(self):
mark_platform_rerun(self.conn, "t-1", 111.0)
self.conn.commit()
self.assertFalse(
consume_platform_marker(self.conn, "t-1", 222.0))
self.conn.commit()
self.assertEqual(self._markers(), [])
def test_none_generation_consumes_nothing(self):
mark_platform_rerun(self.conn, "t-1", 111.0)
self.conn.commit()
self.assertFalse(
consume_platform_marker(self.conn, "t-1", None))
# The marker survives for a later verifiable consume.
self.assertTrue(
consume_platform_marker(self.conn, "t-1", 111.0))
def test_requeue_overwrites_older_marker(self):
mark_platform_rerun(self.conn, "t-1", 111.0)
mark_platform_rerun(self.conn, "t-1", 222.0)
self.conn.commit()
self.assertTrue(
consume_platform_marker(self.conn, "t-1", 222.0))
def test_mark_requires_task_id(self):
with self.assertRaises(ValueError):
mark_platform_rerun(self.conn, "", 1.0)
def test_marker_pg_adapter_evidence(self):
# Adapter-level only: no real Postgres on this machine.
self.assertEqual(PG_MARKER_DDL, MARKER_DDL)
translated, _kind = _translate_sql(MARKER_DDL)
self.assertIn(
"CREATE TABLE IF NOT EXISTS task_attempt_origin_markers",
translated)
self.assertNotIn("AUTOINCREMENT", translated)
self.assertNotIn("GENERATED", translated)
self.assertIn("PRIMARY KEY", translated)
insert = ("INSERT INTO task_attempt_origin_markers(task_id, "
"origin, queued_at) VALUES(?, ?, ?)")
self.assertEqual(_convert_placeholders(insert).count("%s"), 3)
self.assertNotIn("?", _convert_placeholders(insert))
class TestCorrectionRows(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
self.conn = _sqlite_conn(self.db_path)
def tearDown(self):
self.conn.close()
self.tmp.cleanup()
def _agg(self, task_id):
rows = read_attempt_rows(self.conn)
agg = aggregate_attempts(rows)
return agg.get(task_id)
def test_correction_unmarks_a_defect(self):
record_attempt(self.conn, "t-1", "failure", is_defect=True,
defect_category="contract")
record_correction(self.conn, "t-1", 1, is_defect=False)
agg = self._agg("t-1")
self.assertEqual(agg["attempts"], 1)
self.assertEqual(agg["defects"], 0)
self.assertIsNone(agg["defect_category"])
def test_correction_marks_a_missed_defect(self):
record_attempt(self.conn, "t-1", "failure")
record_correction(self.conn, "t-1", 1, is_defect=True,
defect_category="directory")
agg = self._agg("t-1")
self.assertEqual(agg["attempts"], 1)
self.assertEqual(agg["defects"], 1)
self.assertEqual(agg["defect_category"], "directory")
def test_correction_is_never_counted_as_attempt(self):
record_attempt(self.conn, "t-1", "failure", is_defect=True,
defect_category="contract")
record_attempt(self.conn, "t-1", "success")
record_correction(self.conn, "t-1", 1, is_defect=False)
agg = self._agg("t-1")
self.assertEqual(agg["attempts"], 2)
self.assertEqual(agg["defects"], 0)
def test_last_correction_wins(self):
record_attempt(self.conn, "t-1", "failure", is_defect=True,
defect_category="contract")
record_correction(self.conn, "t-1", 1, is_defect=False)
record_correction(self.conn, "t-1", 1, is_defect=True,
defect_category="directory")
agg = self._agg("t-1")
self.assertEqual(agg["defects"], 1)
self.assertEqual(agg["defect_category"], "directory")
def test_first_defect_category_reevaluated_after_correction(self):
record_attempt(self.conn, "t-1", "failure", is_defect=True,
defect_category="contract")
record_attempt(self.conn, "t-1", "failure", is_defect=True,
defect_category="directory")
record_correction(self.conn, "t-1", 1, is_defect=False)
agg = self._agg("t-1")
self.assertEqual(agg["defects"], 1)
self.assertEqual(agg["defect_category"], "directory")
def test_success_row_can_be_judged_defective_on_review(self):
# Defect judgement is independent of outcome: a shipped
# success can still be judged defective after review.
record_attempt(self.conn, "t-1", "success")
record_correction(self.conn, "t-1", 1, is_defect=True,
defect_category="contract")
agg = self._agg("t-1")
self.assertEqual(agg["attempts"], 1)
self.assertEqual(agg["defects"], 1)
def test_correction_targets_rejected(self):
record_attempt(self.conn, "t-1", "failure")
record_attempt(self.conn, "t-1", "interrupted")
with self.assertRaises(ValueError):
record_correction(self.conn, "t-1", 99, is_defect=True)
with self.assertRaises(ValueError):
# Attempt 2 is the interrupted row: no defect
# judgement exists to correct.
record_correction(self.conn, "t-1", 2, is_defect=True)
with self.assertRaises(ValueError):
# Category without a defect flag.
record_correction(self.conn, "t-1", 1, is_defect=False,
defect_category="contract")
# The two rejected writes above left exactly the two
# original rows; the category rejection wrote nothing.
self.assertEqual(len(_ledger_rows_for(self.conn)), 2)
corr = record_correction(self.conn, "t-1", 1, is_defect=True,
defect_category="contract")
with self.assertRaises(ValueError):
# A correction row is not itself correctable.
record_correction(self.conn, "t-1", corr, is_defect=False)
def test_old_gap3_table_reads_and_upgrades(self):
# A table built from gap 3's DDL (no corrects column):
# old rows still read as plain attempts...
gap3_ddl = (
"CREATE TABLE IF NOT EXISTS task_attempts("
"task_id TEXT NOT NULL, "
"attempt_no INTEGER NOT NULL, "
"outcome TEXT NOT NULL, "
"is_defect INTEGER NOT NULL DEFAULT 0, "
"defect_category TEXT, "
"created_at DOUBLE PRECISION, "
"PRIMARY KEY (task_id, attempt_no))")
self.conn.execute(gap3_ddl)
self.conn.execute(
"INSERT INTO task_attempts(task_id, attempt_no, outcome, "
"is_defect, defect_category, created_at) "
"VALUES('t-old', 1, 'success', 0, NULL, 1.0)")
self.conn.commit()
agg = aggregate_attempts(read_attempt_rows(self.conn))
self.assertEqual(agg["t-old"]["attempts"], 1)
# ...and the documented one-line ALTER enables
# corrections on that table.
self.conn.execute(OLD_TABLE_ALTER)
self.conn.commit()
record_correction(self.conn, "t-old", 1, is_defect=True,
defect_category="contract")
self.assertEqual(self._agg("t-old")["defects"], 1)
def test_correction_pg_adapter_evidence(self):
translated, _kind = _translate_sql(PG_DDL)
self.assertIn("corrects_attempt_no INTEGER", translated)
self.assertIn("PRIMARY KEY (task_id, attempt_no)", translated)
self.assertNotIn("AUTOINCREMENT", translated)
self.assertNotIn("GENERATED", translated)
insert = (
"INSERT INTO task_attempts(task_id, attempt_no, outcome, "
"is_defect, defect_category, created_at, "
"corrects_attempt_no) VALUES(?, ?, ?, ?, ?, ?, ?)")
self.assertEqual(_convert_placeholders(insert).count("%s"), 7)
self.assertNotIn("?", _convert_placeholders(insert))
def _ledger_rows_for(conn):
return conn.execute(
"SELECT * FROM task_attempts ORDER BY task_id, attempt_no"
).fetchall()
class TestWorkerHook(unittest.TestCase):
def setUp(self):
self._old_push_allow = os.environ.get(
"CODEX_PUSH_ALLOW_PRIVATE_TARGETS")
os.environ["CODEX_PUSH_ALLOW_PRIVATE_TARGETS"] = "1"
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
from cloud.server import init_db
init_db(self.db_path)
self.env = {"XCM_API_KEY": "test-placeholder"}
def tearDown(self):
if getattr(self, "_old_push_allow", None) is None:
os.environ.pop("CODEX_PUSH_ALLOW_PRIVATE_TARGETS", None)
else:
os.environ["CODEX_PUSH_ALLOW_PRIVATE_TARGETS"] = (
self._old_push_allow)
self.tmp.cleanup()
def _worker(self, runner=None, **kwargs):
from cloud.worker import Worker
kwargs.setdefault("env", self.env)
kwargs.setdefault("retry_backoff_seconds", 0)
return Worker(self.db_path, "/fake/host.py",
runner=runner or _completed_runner, **kwargs)
def test_hook_off_writes_nothing_and_creates_no_table(self):
_insert_task(self.db_path, "A")
worker = self._worker()
self.assertFalse(worker.attempt_ledger_hook)
result = worker.run_one()
self.assertEqual(result["status"], "completed")
self.assertFalse(_table_exists(self.db_path, "task_attempts"))
self.assertFalse(
_table_exists(self.db_path, "task_attempt_origin_markers"))
def test_hook_on_first_run_writes_one_success_row(self):
_insert_task(self.db_path, "A")
worker = self._worker(attempt_ledger_hook=True)
result = worker.run_one()
self.assertEqual(result["status"], "completed")
rows = _ledger_rows(self.db_path)
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0][:5], ("A", 1, "success", 0, None))
self.assertIsNone(rows[0][5])
def test_hook_on_transient_retry_is_still_one_row(self):
calls = {"n": 0}
def flaky(cmd):
calls["n"] += 1
if calls["n"] == 1:
return SimpleNamespace(returncode=1, stdout="boom",
stderr="")
return _completed_runner(cmd)
_insert_task(self.db_path, "A")
worker = self._worker(runner=flaky, attempt_ledger_hook=True,
max_retries=1)
result = worker.run_one()
self.assertEqual(result["status"], "completed")
rows = _ledger_rows(self.db_path)
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0][2], "success")
def test_hook_on_human_retry_writes_second_row(self):
_insert_task(self.db_path, "A")
worker = self._worker(attempt_ledger_hook=True)
self.assertEqual(worker.run_one()["status"], "completed")
# Simulate the server /retry endpoint: back to queued,
# results cleared, NO platform marker written.
conn = sqlite3.connect(self.db_path)
try:
conn.execute(
"UPDATE tasks SET status='queued', result_text=NULL, "
"error_text=NULL, actual_model=NULL WHERE id='A'")
conn.commit()
finally:
conn.close()
worker2 = self._worker(attempt_ledger_hook=True)
self.assertEqual(worker2.run_one()["status"], "completed")
rows = _ledger_rows(self.db_path)
self.assertEqual(len(rows), 2)
self.assertEqual([r[1] for r in rows], [1, 2])
self.assertEqual([r[2] for r in rows],
["success", "success"])
def test_hook_on_platform_rerun_writes_no_row(self):
from cloud.attempt_ledger import record_attempt
from cloud.db import connect as db_connect
# Previous dispatch ended interrupted (its row is
# count-neutral by the gap-3 rule).
_insert_task(self.db_path, "A", status="interrupted",
interrupted_reason="worker_shutdown",
paused_by_user=0, auto_resume_count=0)
conn = db_connect(self.db_path)
try:
record_attempt(conn, "A", "interrupted")
finally:
conn.close()
worker = self._worker(attempt_ledger_hook=True)
resumed = worker.auto_resume_interrupted()
self.assertEqual(resumed, ["A"])
self.assertEqual(_task_status(self.db_path, "A"), "queued")
result = worker.run_one()
self.assertEqual(result["status"], "completed")
rows = _ledger_rows(self.db_path)
# Still exactly the one interrupted row: the platform
# rerun added no counted row, and its marker is gone.
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0][2], "interrupted")
conn = sqlite3.connect(self.db_path)
try:
markers = conn.execute(
"SELECT COUNT(*) FROM task_attempt_origin_markers"
).fetchone()[0]
finally:
conn.close()
self.assertEqual(markers, 0)
def test_hook_on_gate_failure_writes_no_row(self):
_insert_task(self.db_path, "A")
worker = self._worker(attempt_ledger_hook=True, env={})
result = worker.run_one()
self.assertEqual(result["status"], "failed")
self.assertFalse(_table_exists(self.db_path, "task_attempts"))
def test_hook_on_cancelled_dispatch_writes_interrupted_row(self):
# Established pattern (test_worker
# test_cancel_during_dispatch_keeps_interrupted): the fake
# runner cancels the task mid-dispatch, then reports
# completed; run_one must keep interrupted, and the hook
# must write exactly one interrupted row for it.
worker_ref = {}
def cancelling(cmd):
worker_ref["w"].cancel("A")
return _completed_runner(cmd)
_insert_task(self.db_path, "A")
worker = self._worker(runner=cancelling,
attempt_ledger_hook=True)
worker_ref["w"] = worker
result = worker.run_one()
self.assertEqual(result["status"], "interrupted")
self.assertEqual(_task_status(self.db_path, "A"), "interrupted")
rows = _ledger_rows(self.db_path)
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0][2], "interrupted")
# Interrupted rows never enter the aggregate.
records = apply_to_records(
[{"id": "A", "status": "interrupted",
"attempts": None, "defects": None}],
aggregate_attempts(read_attempt_rows(
_sqlite_conn(self.db_path))))
metrics = compute_metrics(records)
self.assertEqual(metrics["terminal_tasks"], 0)
self.assertIsNone(metrics["first_pass_rate"])
self.assertFalse(metrics["first_pass_known"])
def test_hook_write_failure_never_fails_task(self):
import cloud.worker as worker_mod
_insert_task(self.db_path, "A")
worker = self._worker(attempt_ledger_hook=True)
original = worker_mod._attempt_record
def boom(conn, task_id, outcome):
raise RuntimeError("ledger down")
worker_mod._attempt_record = boom
try:
result = worker.run_one()
finally:
worker_mod._attempt_record = original
self.assertEqual(result["status"], "completed")
self.assertEqual(_task_status(self.db_path, "A"), "completed")
class MetricsHookEndpointTestCase(unittest.TestCase):
"""Mixed ledger/old/platform/interrupted tasks on the real
endpoint: terminal denominators contain only known tasks."""
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
from cloud.auth import AuthStore
from cloud.cost import CostStore
from cloud.server import create_server
self.auth = AuthStore(self.db_path)
self.owner_id, self.owner_key = self.auth.create_user("owner")
self.server = create_server(self.db_path, "127.0.0.1", 0)
self.port = self.server.server_address[1]
self.thread = threading.Thread(
target=self.server.serve_forever, daemon=True)
self.thread.start()
status, self.project = self._req(
"POST", "/projects", {"name": "demo"}, self.owner_key)
self.assertEqual(status, 201)
self.costs = CostStore(self.db_path)
def tearDown(self):
self.server.shutdown()
self.server.server_close()
self.thread.join(timeout=5)
try:
self.auth.conn.close()
except Exception:
pass
self.tmp.cleanup()
def _req(self, method, path, body=None, key=None):
conn = http.client.HTTPConnection("127.0.0.1", self.port,
timeout=10)
hdrs = {}
payload = None
if body is not None:
payload = json.dumps(body).encode("utf-8")
hdrs["Content-Type"] = "application/json"
if key is not None:
hdrs["X-API-Key"] = key
try:
conn.request(method, path, body=payload, headers=hdrs)
resp = conn.getresponse()
data = resp.read()
finally:
conn.close()
try:
parsed = json.loads(data.decode("utf-8")) if data else None
except ValueError:
parsed = None
return resp.status, parsed
def test_mixed_tasks_known_only_denominators(self):
from cloud.db import connect as db_connect
pid = self.project["id"]
conn = db_connect(self.db_path)
try:
for tid, status in (
("t-a", "completed"), ("t-b", "merged"),
("t-old", "completed"), ("t-plat", "completed"),
("t-int", "interrupted")):
conn.execute(
"INSERT INTO tasks(id, project_id, status) "
"VALUES(?, ?, ?)", (tid, pid, status))
conn.commit()
# t-a: one clean round -> first-pass candidate.
record_attempt(conn, "t-a", "success")
# t-b: defect round then success -> rework + defect.
record_attempt(conn, "t-b", "failure", is_defect=True,
defect_category="contract")
record_attempt(conn, "t-b", "success")
# t-plat: completed ONLY via a platform rerun: its
# ledger holds an interrupted row and nothing else,
# so it stays unknown (never 0, never first-pass).
record_attempt(conn, "t-plat", "interrupted")
# t-int: interrupted STATUS with a failure row: the
# status alone keeps it out of terminal tasks.
record_attempt(conn, "t-int", "failure", is_defect=True,
defect_category="contract")
finally:
conn.close()
# Priced cost only for t-a: the dual-column discipline
# must survive the ledger (total None when incomplete,
# priced partial still shown, unpriced never folded to 0).
self.costs.record("t-a", pid, "gpt-6-sol",
input_tokens=1000, output_tokens=100,
usd_micros=5000, source="actual")
status, report = self._req(
"GET", "/metrics-report", key=self.owner_key)
self.assertEqual(status, 200)
metrics = report["metrics"]
# Terminal tasks: t-a, t-b, t-old, t-plat (t-int is not
# terminal despite its ledger row).
self.assertEqual(metrics["terminal_tasks"], 4)
# Known-only denominators: t-a + t-b (2 tasks). If the
# old or platform task were wrongly counted as
# first-pass / defect-free, these would be 3/4 and 1/4.
self.assertAlmostEqual(metrics["first_pass_rate"], 0.5)
self.assertTrue(metrics["first_pass_known"])
self.assertAlmostEqual(metrics["rework_rate"], 0.5)
self.assertAlmostEqual(metrics["defect_rate"], 0.5)
self.assertTrue(metrics["defect_known"])
self.assertEqual(metrics["defect_by_category"],
{"contract": 1})
self.assertIsNone(metrics["total_usd_micros"])
self.assertEqual(metrics["priced_usd_micros_partial"], 5000)
self.assertFalse(metrics["cost_complete"])
if __name__ == "__main__":
unittest.main()
+245
View File
@@ -0,0 +1,245 @@
"""Tests for the task-attempt ledger (audit gap 3).
Only the standard library; a temporary SQLite DB for the store
tests, plain dicts for the pure aggregation. No real model, no
network, no secrets, no real Postgres (the Postgres claim is
checked at the adapter level only, and labelled as such).
Pinned boundaries:
- interrupted rows never inflate attempts/defects, and a task with
only interrupted rows stays unknown (omitted, not 0);
- old tasks with no ledger rows stay unknown (None + *_known False),
never counted as first-pass or defect-free;
- interrupted task STATUS is never terminal, even with ledger rows;
- the dual-column cost discipline is unchanged by the ledger.
"""
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(
os.path.dirname(os.path.abspath(__file__)))))
from cloud.attempt_ledger import ( # noqa: E402
PG_DDL,
aggregate_attempts,
apply_to_records,
ensure_task_attempts_table,
read_attempt_rows,
record_attempt,
)
from cloud.db import _convert_placeholders, _translate_sql # noqa: E402
from cloud.metrics import compute_metrics # noqa: E402
def _sqlite_conn(path):
from cloud.db import connect as db_connect
return db_connect(path)
class TestLedgerStore(unittest.TestCase):
def setUp(self):
import tempfile
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
self.conn = _sqlite_conn(self.db_path)
def tearDown(self):
self.conn.close()
self.tmp.cleanup()
def _count(self):
# Validation in record_attempt runs before ensure, so a
# rejected write may leave the table uncreated: that is
# still "writes nothing", counted as 0 here.
try:
return self.conn.execute(
"SELECT COUNT(*) FROM task_attempts").fetchone()[0]
except Exception: # noqa: BLE001 -- missing table == 0 rows
return 0
def test_ensure_is_idempotent_and_preserves_rows(self):
ensure_task_attempts_table(self.conn)
record_attempt(self.conn, "t-1", "failure", is_defect=True,
defect_category="contract")
ensure_task_attempts_table(self.conn)
ensure_task_attempts_table(self.conn)
self.assertEqual(self._count(), 1)
def test_attempt_numbers_are_per_task_and_sequential(self):
self.assertEqual(
record_attempt(self.conn, "t-1", "failure", is_defect=True,
defect_category="contract"), 1)
self.assertEqual(
record_attempt(self.conn, "t-1", "success"), 2)
self.assertEqual(
record_attempt(self.conn, "t-2", "success"), 1)
rows = {(r["task_id"], r["attempt_no"])
for r in read_attempt_rows(self.conn)}
self.assertEqual(rows, {("t-1", 1), ("t-1", 2), ("t-2", 1)})
def test_unknown_outcome_rejected_and_writes_nothing(self):
with self.assertRaises(ValueError):
record_attempt(self.conn, "t-1", "maybe")
# The table was ensured but no row was written.
self.assertEqual(self._count(), 0)
def test_interrupted_row_can_never_be_a_defect(self):
with self.assertRaises(ValueError):
record_attempt(self.conn, "t-1", "interrupted",
is_defect=True)
self.assertEqual(self._count(), 0)
def test_defect_category_without_defect_flag_rejected(self):
with self.assertRaises(ValueError):
record_attempt(self.conn, "t-1", "failure",
defect_category="contract")
self.assertEqual(self._count(), 0)
class TestAggregate(unittest.TestCase):
def _row(self, task_id, attempt_no, outcome, is_defect=0,
category=None):
return {"task_id": task_id, "attempt_no": attempt_no,
"outcome": outcome, "is_defect": is_defect,
"defect_category": category}
def test_interrupted_rows_excluded_from_both_counts(self):
agg = aggregate_attempts([
self._row("t-1", 1, "interrupted"),
self._row("t-1", 2, "failure", 1, "contract"),
self._row("t-1", 3, "success"),
])
self.assertEqual(agg["t-1"]["attempts"], 2)
self.assertEqual(agg["t-1"]["defects"], 1)
self.assertEqual(agg["t-1"]["defect_category"], "contract")
def test_only_interrupted_task_is_omitted_not_zero(self):
agg = aggregate_attempts([
self._row("t-1", 1, "interrupted"),
])
self.assertNotIn("t-1", agg)
def test_first_defect_category_wins_regardless_of_row_order(self):
agg = aggregate_attempts([
self._row("t-1", 2, "failure", 1, "directory"),
self._row("t-1", 1, "failure", 1, "contract"),
])
self.assertEqual(agg["t-1"]["defects"], 2)
self.assertEqual(agg["t-1"]["defect_category"], "contract")
def test_failed_attempt_is_not_automatically_a_defect(self):
agg = aggregate_attempts([
self._row("t-1", 1, "failure"),
self._row("t-1", 2, "success"),
])
self.assertEqual(agg["t-1"]["attempts"], 2)
self.assertEqual(agg["t-1"]["defects"], 0)
self.assertIsNone(agg["t-1"]["defect_category"])
def test_apply_does_not_mutate_input_and_keeps_old_unknown(self):
records = [{"id": "old", "status": "completed",
"attempts": None, "defects": None},
{"id": "new", "status": "completed",
"attempts": None, "defects": None}]
merged = apply_to_records(
records, {"new": {"attempts": 1, "defects": 0,
"defect_category": None}})
self.assertIsNone(records[1]["attempts"]) # input untouched
self.assertIsNone(merged[0]["attempts"]) # old stays unknown
self.assertEqual(merged[1]["attempts"], 1)
self.assertEqual(merged[1]["defects"], 0)
class TestMetricsFromLedger(unittest.TestCase):
def test_real_rates_from_ledger(self):
records = apply_to_records([
{"id": "a", "status": "completed",
"attempts": None, "defects": None},
{"id": "b", "status": "merged",
"attempts": None, "defects": None},
{"id": "c", "status": "failed",
"attempts": None, "defects": None},
], {
"a": {"attempts": 1, "defects": 0,
"defect_category": None},
"b": {"attempts": 3, "defects": 2,
"defect_category": "contract"},
"c": {"attempts": 2, "defects": 1,
"defect_category": "contract"},
})
m = compute_metrics(records)
self.assertEqual(m["terminal_tasks"], 3)
self.assertAlmostEqual(m["first_pass_rate"], 1 / 3)
self.assertAlmostEqual(m["rework_rate"], 2 / 3)
self.assertAlmostEqual(m["defect_rate"], 2 / 3)
self.assertEqual(m["defect_by_category"], {"contract": 2})
self.assertTrue(m["first_pass_known"])
self.assertTrue(m["defect_known"])
def test_old_tasks_without_rows_stay_unknown_not_zero(self):
records = [{"id": "old", "status": "completed",
"attempts": None, "defects": None}]
m = compute_metrics(records)
self.assertEqual(m["terminal_tasks"], 1)
self.assertIsNone(m["first_pass_rate"])
self.assertIsNone(m["rework_rate"])
self.assertIsNone(m["defect_rate"])
self.assertFalse(m["first_pass_known"])
self.assertFalse(m["defect_known"])
self.assertEqual(m["defect_tasks"], 0) # 0 known defective,
# not a claim of "no defects": defect_known is False above.
def test_interrupted_status_never_terminal_even_with_rows(self):
records = apply_to_records([
{"id": "i", "status": "interrupted",
"attempts": None, "defects": None},
], {"i": {"attempts": 1, "defects": 0,
"defect_category": None}})
m = compute_metrics(records)
self.assertEqual(m["terminal_tasks"], 0)
self.assertIsNone(m["first_pass_rate"])
self.assertFalse(m["first_pass_known"])
def test_ledger_does_not_change_dual_column_cost(self):
records = apply_to_records([
{"id": "a", "status": "completed",
"attempts": None, "defects": None},
], {"a": {"attempts": 1, "defects": 0,
"defect_category": None}})
costs = {"a": {"priced_usd_micros": 5000,
"unpriced_actual_tokens": 700,
"has_rows": True}}
m = compute_metrics(records, costs)
self.assertAlmostEqual(m["first_pass_rate"], 1.0)
self.assertIsNone(m["total_usd_micros"])
self.assertEqual(m["priced_usd_micros_partial"], 5000)
self.assertEqual(
m["per_task_cost"][0]["unpriced_actual_tokens"], 700)
class TestPostgresAdapterEvidence(unittest.TestCase):
"""Adapter-level evidence only -- no real Postgres here (see
the module docstring: production PG still needs the manual
PG_DDL step and a live verification)."""
def test_pg_ddl_needs_no_sequence_or_identity_handling(self):
translated, _kind = _translate_sql(PG_DDL)
self.assertIn("CREATE TABLE IF NOT EXISTS task_attempts",
translated)
self.assertNotIn("AUTOINCREMENT", translated)
self.assertNotIn("GENERATED", translated)
self.assertIn("PRIMARY KEY (task_id, attempt_no)", translated)
def test_insert_placeholders_translate_to_percent_s(self):
sql = ("INSERT INTO task_attempts(task_id, attempt_no, "
"outcome, is_defect, defect_category, created_at) "
"VALUES(?, ?, ?, ?, ?, ?)")
translated = _convert_placeholders(sql)
self.assertEqual(translated.count("%s"), 6)
self.assertNotIn("?", translated)
if __name__ == "__main__":
unittest.main()
+260
View File
@@ -0,0 +1,260 @@
"""自动化执行器审计上链(缺口 4,方案 A)单测。
钉住的不是“写了审计行”,而是“审计行与链条目成对、与业务行
同事务”:
- created / skipped 两路:audit_log 行与 audit_ledger 行同
created_at、同 user_id/action/detail,verify_chain ok。
- 链头唯一冲突:整段(任务 + 产出 + 审计对)回滚重试一次后
成功,最终各恰好一行,不出现任务已建、审计缺失的半截态。
- 非唯一冲突的审计失败:fail-closed 抛错,任务 + 产出 + 审计
对全部回滚(0 行),只有已单独提交的抢单 last_run_at 留下,
下个间隔可补跑——不是吞错、不是 failed 产出。
- audit_on_conn 与 AuthStore.audit() 是同一对写:前者写的行
能作为后者的链前驱,prev 衔接、verify_chain ok。
- 同间隔内第二次跑不写任何新行。
"""
import os
import re
import sqlite3
import tempfile
import time
import unittest
from pathlib import Path
from cloud.auth import AuthStore, audit_on_conn
from cloud.automations import (
create_automation,
ensure_automations_tables,
run_due_automation,
)
from cloud.ledger import GENESIS_HASH, verify_chain
from cloud.server import init_db
def _count(conn, sql, params=()):
return conn.execute(sql, params).fetchone()[0]
class _ProxyConn:
"""把除指定拦截外的调用原样委托给真实连接的代理。
run_due_automation 只用 conn.execute / commit / rollback,
代理这三者即足以模拟链写失败;其余属性(如 row_factory)
不用代理——调用方不读它们。
"""
def __init__(self, real):
self._real = real
def execute(self, sql, params=()):
return self._real.execute(sql, params)
def commit(self):
return self._real.commit()
def rollback(self):
return self._real.rollback()
class _FlakyLedgerConn(_ProxyConn):
"""首个 audit_ledger INSERT 报唯一冲突,之后放行。"""
def __init__(self, real):
super().__init__(real)
self.flaked = False
def execute(self, sql, params=()):
if (not self.flaked
and sql.strip().upper().startswith(
"INSERT INTO AUDIT_LEDGER")):
self.flaked = True
raise sqlite3.IntegrityError(
"UNIQUE constraint failed: audit_ledger.entry_hash")
return super().execute(sql, params)
class _BrokenLedgerConn(_ProxyConn):
"""全部 audit_ledger INSERT 报非唯一冲突错误。"""
def execute(self, sql, params=()):
if sql.strip().upper().startswith("INSERT INTO AUDIT_LEDGER"):
raise RuntimeError("ledger storage boom")
return super().execute(sql, params)
class AutomationChainTestCase(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
init_db(self.db_path)
self.conn = sqlite3.connect(self.db_path)
self.conn.row_factory = sqlite3.Row
ensure_automations_tables(self.conn)
def tearDown(self):
self.conn.close()
self.tmp.cleanup()
def _auto(self, owner="owner1"):
return create_automation(self.conn, "proj", "daily", "do thing",
owner, interval_seconds=60)
def _set_quota(self, user_id, micros):
self.conn.execute(
"CREATE TABLE IF NOT EXISTS user_quotas("
"user_id TEXT PRIMARY KEY, quota_usd_micros BIGINT, "
" updated_at DOUBLE PRECISION)")
self.conn.execute(
"INSERT OR REPLACE INTO user_quotas(user_id,"
" quota_usd_micros, updated_at) VALUES(?, ?, ?)",
(user_id, micros, time.time()))
self.conn.commit()
def _pair_counts(self):
# 缺表计 0:SQLite 的 DDL 可事务回滚,首跑审计失败回滚时
# 会连 audit_on_conn 刚建的 audit_log / audit_ledger 表一起
# 撤销——下次跑由 CREATE IF NOT EXISTS 重建,不是丢数据。
out = []
for table in ("tasks", "automation_runs", "audit_log",
"audit_ledger"):
try:
out.append(_count(
self.conn, "SELECT COUNT(*) FROM %s" % table))
except sqlite3.OperationalError:
out.append(0)
return tuple(out)
def _check_pair(self, action, detail):
audit = self.conn.execute(
"SELECT user_id, project_id, action, detail, created_at"
" FROM audit_log WHERE action = ?", (action,)).fetchone()
self.assertIsNotNone(audit)
self.assertEqual(audit[0], "system")
self.assertEqual(audit[1], "proj")
self.assertEqual(audit[3], detail)
entry = self.conn.execute(
"SELECT user_id, project_id, task_id, action, detail,"
" created_at, prev_hash, entry_hash FROM audit_ledger"
" WHERE action = ?", (action,)).fetchone()
self.assertIsNotNone(entry)
self.assertEqual(entry[0], "system")
self.assertEqual(entry[1], "proj")
self.assertEqual(entry[3], action)
self.assertEqual(entry[4], detail)
self.assertEqual(entry[5], audit[4])
result = verify_chain(self.conn)
self.assertTrue(result["ok"], result)
return audit, entry
def test_created_run_writes_chained_pair(self):
auto = self._auto()
run = run_due_automation(self.conn, auto["id"])
self.assertIsNotNone(run)
self.assertEqual(run["status"], "created")
self.assertEqual(self._pair_counts(), (1, 1, 1, 1))
self._check_pair("automation_run_created",
"%s -> %s" % (auto["id"], run["task_id"]))
def test_skipped_run_writes_chained_pair(self):
auto = self._auto()
self._set_quota("owner1", 0)
run = run_due_automation(self.conn, auto["id"])
self.assertIsNotNone(run)
self.assertEqual(run["status"], "skipped_quota")
self.assertIsNone(run["task_id"])
self.assertEqual(self._pair_counts(), (0, 1, 1, 1))
audit, _entry = self._check_pair(
"automation_run_quota_skipped",
"%s: %s" % (auto["id"], run["detail"]))
self.assertIn("quota", audit[3])
def test_chain_head_race_retries_whole_block_once(self):
auto = self._auto()
proxy = _FlakyLedgerConn(self.conn)
run = run_due_automation(proxy, auto["id"])
self.assertTrue(proxy.flaked)
self.assertIsNotNone(run)
self.assertEqual(run["status"], "created")
# 重试后仍恰好各一行:回滚掉了第一次的任务 + 审计行,
# 没有“任务两行、审计一行”或反过来的半截态。
self.assertEqual(self._pair_counts(), (1, 1, 1, 1))
self._check_pair("automation_run_created",
"%s -> %s" % (auto["id"], run["task_id"]))
# 抢单在重试循环外、已单独提交,不因回滚丢失。
claimed = self.conn.execute(
"SELECT last_run_at FROM automations WHERE id = ?",
(auto["id"],)).fetchone()[0]
self.assertIsNotNone(claimed)
def test_non_integrity_audit_failure_rolls_back_all(self):
auto = self._auto()
proxy = _BrokenLedgerConn(self.conn)
with self.assertRaises(RuntimeError):
run_due_automation(proxy, auto["id"])
# 任务、产出、审计对全部回滚:不是吞错后留半截。
self.assertEqual(self._pair_counts(), (0, 0, 0, 0))
# 抢单已提交:下个间隔可补跑,这次不记 failed 产出。
claimed = self.conn.execute(
"SELECT last_run_at FROM automations WHERE id = ?",
(auto["id"],)).fetchone()[0]
self.assertIsNotNone(claimed)
def test_second_run_same_interval_writes_nothing_more(self):
auto = self._auto()
run = run_due_automation(self.conn, auto["id"])
self.assertIsNotNone(run)
self.assertIsNone(run_due_automation(self.conn, auto["id"]))
self.assertEqual(self._pair_counts(), (1, 1, 1, 1))
def test_audit_on_conn_chains_with_auth_store_audit(self):
conn = sqlite3.connect(self.db_path)
try:
audit_on_conn(conn, "u1", "p1", "manual_action", "d",
created_at=123.0)
conn.commit()
finally:
conn.close()
store = AuthStore(self.db_path)
try:
store.audit("u2", "p1", "second_action", "e")
result = verify_chain(store.conn)
self.assertTrue(result["ok"], result)
entries = store.conn.execute(
"SELECT action, prev_hash FROM audit_ledger"
" ORDER BY id").fetchall()
self.assertEqual(len(entries), 2)
self.assertEqual(entries[0][0], "manual_action")
self.assertEqual(entries[0][1], GENESIS_HASH)
first_hash = store.conn.execute(
"SELECT entry_hash FROM audit_ledger ORDER BY id"
" LIMIT 1").fetchone()[0]
self.assertEqual(entries[1][1], first_hash)
finally:
store.conn.close()
class NoAuditBypassTestCase(unittest.TestCase):
"""生产代码里 audit_log 的 INSERT 只允许出现在 auth.py。
这是缺口 4 的“无旁路”自证:将来任何模块再起 audit_log
直写(哪怕只是 best-effort),本测试即红,须先走
audit_on_conn / AuthStore.audit 才能合入。测试固件里的
直写(test_tamper_ledger 的篡改模拟、test_chat_history 的
旧行模拟)不在 cloud/*.py 内,不在本断言范围。
"""
def test_only_auth_writes_audit_log(self):
cloud_dir = Path(__file__).resolve().parents[1]
offenders = []
for path in sorted(cloud_dir.glob("*.py")):
text = path.read_text(encoding="utf-8")
if re.search(r"INSERT\s+INTO\s+audit_log", text,
re.IGNORECASE):
offenders.append(path.name)
self.assertEqual(offenders, ["auth.py"])
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,74 @@
"""A11: whole-tree audit_log single-writer scan assertion.
Extends the pre-existing ``NoAuditBypassTestCase`` in
``test_audit_automations_chain.py`` (which only scanned
``cloud/*.py`` at the top level) to the whole tree: every
``.py`` file under the repository root is scanned for a
direct audit_log insert code form, except the test fixtures
themselves.
Scope and exclusions, stated so a future red test is readable:
- Production code: exactly one hit is allowed, ``cloud/auth.py``
(``audit_on_conn``, the unified port; ``AuthStore.audit`` is
that function plus a commit/retry wrapper, not a second write
path). Any new production hit -- any module, any subdirectory
(``cloud/skills/...``, ``tools/``, ``deploy/``, ``host.py``,
top level) -- turns this test red.
- Test fixtures excluded: ``cloud/tests/test_tamper_ledger.py``
(tamper simulation) and ``cloud/tests/test_chat_history.py``
(legacy-row simulation) deliberately insert audit_log rows
into throwaway databases, and
``cloud/tests/test_automation_run_due.py`` (a fake PG
connection whose ``execute`` matches intercepted audit_log
SQL strings and records params in memory -- it executes no
SQL at all). They are fixtures, not production writes, and
were already documented as out of scope by the pre-existing
test. They are listed by name here, not by a broad ``tests/``
exclusion, so a new test file that starts writing audit_log
directly would still turn this red.
- No pending production direct-write points were found at the
time this test was added (2026-11-02): the full-tree scan
returned only ``cloud/auth.py`` in production code. If a
future scan finds one, do NOT silently fix production code
under this test's name and do NOT land a red test: record
the point in the delivery report and route the fix to its
own batch.
"""
import re
import unittest
from pathlib import Path
_PATTERN = re.compile(r"INSERT\s+INTO\s+audit_log", re.IGNORECASE)
# Fixture files that deliberately insert audit_log rows into
# throwaway test databases (see module docstring).
_FIXTURE_FILES = {
"cloud/tests/test_tamper_ledger.py",
"cloud/tests/test_chat_history.py",
"cloud/tests/test_automation_run_due.py",
}
class AuditLogSingleWriterTestCase(unittest.TestCase):
def test_only_auth_writes_audit_log_tree_wide(self):
repo_root = Path(__file__).resolve().parents[2]
offenders = []
for path in sorted(repo_root.rglob("*.py")):
rel = path.relative_to(repo_root).as_posix()
if rel in _FIXTURE_FILES:
continue
# Skip nothing else: .git contains no .py files, and
# a work/host-worktrees checkout (if present) is a
# separate worktree's copy of the code, not this
# tree's production code.
if rel.startswith("work/host-worktrees/"):
continue
text = path.read_text(encoding="utf-8", errors="replace")
if _PATTERN.search(text):
offenders.append(rel)
self.assertEqual(offenders, ["cloud/auth.py"])
if __name__ == "__main__":
unittest.main()
+572
View File
@@ -0,0 +1,572 @@
"""Audit loop driver: pure decision logic for the project-level
audit rework loop (track 1 of the production built-in loop).
Pins the driver interface the next (wiring) track must use
verbatim, and the counting rules shared by #244 / #241 / #247:
pushbacks cover failures 1-3, the 4th failure (> 3, never >= 3)
rebuilds, rebuilds are 1 per module per contract version, the
3rd failure is triaged, a pass clears failures but not budgets,
a contract-version change clears everything, quota blocks consume
nothing, a correction to pass does not unblock, and ledger events
match the #241 field table.
Style follows the other cloud tests: unittest, stdlib only, import
with a sys.path fallback so the file also runs standalone.
"""
import sys
import unittest
from pathlib import Path
try:
from cloud.audit_loop_driver import (
ACTION_KINDS,
LEDGER_EVENT_TYPES,
apply_correction,
confirm_rebuild,
decide_audit,
emit,
new_loop_state,
validate_ledger_event,
after_fix,
)
except ImportError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from cloud.audit_loop_driver import (
ACTION_KINDS,
LEDGER_EVENT_TYPES,
apply_correction,
confirm_rebuild,
decide_audit,
emit,
new_loop_state,
validate_ledger_event,
after_fix,
)
def fail_verdict(**overrides):
verdict = {
"source": "audit",
"verdict": "fail",
"verdict_text": "audit fail: ci red",
"failures": [{"check": "ci_green", "detail": "ci red"}],
"passed_items": [],
"previous_artifacts": [{"path": "app.py", "sha256": "a" * 64}],
"reproduction_steps": ["run ci"],
}
verdict.update(overrides)
return verdict
def pass_verdict(**overrides):
verdict = {
"source": "audit",
"verdict": "pass",
"verdict_text": "audit pass",
"failures": [],
"passed_items": [],
}
verdict.update(overrides)
return verdict
def near_complete_verdict(**overrides):
verdict = fail_verdict(
failures=[{"check": "edge-case"}],
passed_items=[{"check": "p1"}, {"check": "p2"}, {"check": "p3"}],
broken_passed_items=[],
verdict_text="audit fail: one edge case left",
)
verdict.update(overrides)
return verdict
def drive(state, verdict, **kwargs):
return decide_audit(state, verdict, **kwargs)
class PushbackCountingTestCase(unittest.TestCase):
def test_fail_pushbacks_and_blocks(self):
state = new_loop_state("m1", "v1")
action = drive(state, fail_verdict())
self.assertEqual(action.kind, "pushback")
self.assertTrue(action.merge_blocked)
self.assertTrue(action.release_blocked)
self.assertEqual(action.next_state.failure_count, 1)
self.assertEqual(action.next_state.pushback_round, 1)
self.assertEqual(action.next_state.status, "pushback")
package = action.pushback_package
self.assertEqual(package["audit_verdict_text"], "audit fail: ci red")
self.assertEqual(package["failures"][0]["check"], "ci_green")
self.assertEqual(package["reproduction_steps"], ["run ci"])
# Full re-audit criteria, never just the failed items.
self.assertIn("ci_green", package["reaudit_criteria"])
self.assertIn("audit_signer_separation", package["reaudit_criteria"])
self.assertEqual(package["previous_artifacts"][0]["path"], "app.py")
self.assertEqual(
[e["event_type"] for e in action.ledger_events],
["audit_verdict", "audit_pushback_round"])
def test_exactly_three_still_pushes_back(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
action = drive(state, fail_verdict())
state = action.next_state
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.next_state.failure_count, 3)
self.assertEqual(action.next_state.pushback_round, 3)
def test_fourth_failure_rebuilds(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict()).next_state
action = drive(state, fail_verdict(retained_tests=["t1", "t2"]))
self.assertEqual(action.kind, "rebuild")
self.assertEqual(action.rebuild_no, 1)
self.assertEqual(action.next_state.rebuild_no, 1)
self.assertEqual(action.next_state.failure_count, 0)
self.assertEqual(action.next_state.pushback_round, 0)
self.assertEqual(action.next_state.status, "rebuilding")
self.assertEqual(action.next_state.retained_tests, ["t1", "t2"])
retained = action.pushback_package["retained_assets"]
self.assertEqual(retained["tests"], ["t1", "t2"])
self.assertTrue(retained["contract_unchanged"])
self.assertTrue(retained["design_unchanged"])
def test_rebuild_budget_exhausted_escalates(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict()).next_state
state = drive(state, fail_verdict()).next_state # rebuild 1
self.assertEqual(state.rebuild_no, 1)
for _ in range(3):
state = drive(state, fail_verdict()).next_state
action = drive(state, fail_verdict())
self.assertEqual(action.kind, "escalate")
self.assertEqual(action.escalation_target, "coordinator")
self.assertEqual(action.next_state.rebuild_no, 1)
self.assertEqual(action.next_state.status, "escalated")
self.assertFalse(action.escalation_grants_user_acceptance)
self.assertFalse(hasattr(action, "tick_acceptance"))
# Full history, not just the last round.
texts = [entry.get("verdict_text")
for entry in action.escalation_history]
self.assertIn("audit fail: ci red", texts)
self.assertGreaterEqual(len(action.escalation_history), 4)
def test_audit_and_acceptance_counts_combine_but_stay_separate(self):
state = new_loop_state("m1", "v1")
state = drive(state, fail_verdict()).next_state
state = drive(state, fail_verdict(source="acceptance")).next_state
action = drive(state, fail_verdict())
self.assertEqual(action.next_state.failure_count, 3)
self.assertEqual(action.next_state.audit_fail_count, 2)
self.assertEqual(action.next_state.acceptance_fail_count, 1)
def test_pass_clears_failures_keeps_rebuild_budget(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict()).next_state
state = drive(state, fail_verdict()).next_state # rebuild 1
state = drive(state, fail_verdict()).next_state # failure 1, new episode
action = drive(state, pass_verdict())
self.assertEqual(action.kind, "pass")
self.assertFalse(action.merge_blocked)
self.assertEqual(action.next_state.failure_count, 0)
self.assertEqual(action.next_state.pushback_round, 0)
self.assertEqual(action.next_state.rebuild_no, 1)
self.assertEqual(action.next_state.status, "passed")
def test_contract_version_change_clears_everything(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict()).next_state
state = drive(state, fail_verdict()).next_state # rebuild 1
state = drive(state, fail_verdict()).next_state # failure 1, v1
action = drive(state, fail_verdict(contract_version="v2"))
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.next_state.contract_version, "v2")
self.assertEqual(action.next_state.failure_count, 1)
self.assertEqual(action.next_state.pushback_round, 1)
self.assertEqual(action.next_state.rebuild_no, 0)
self.assertFalse(action.next_state.targeted_used)
class ThirdFailureTriageTestCase(unittest.TestCase):
def _two_failures(self):
state = new_loop_state("m1", "v1")
for _ in range(2):
state = drive(state, fail_verdict()).next_state
return state
def test_structural_third_failure_early_rebuilds(self):
state = self._two_failures()
action = drive(state, fail_verdict(
failures=[{"check": "arch",
"structural_kind": "architecture_boundary"}],
verdict_text="audit fail: architecture boundary crossed"))
self.assertEqual(action.kind, "rebuild")
self.assertEqual(action.rebuild_no, 1)
assessments = [e for e in action.ledger_events
if e["event_type"] == "audit_third_failure_assessment"]
self.assertEqual(len(assessments), 1)
self.assertEqual(assessments[0]["payload"]["determination"], "early_rebuild")
self.assertEqual(assessments[0]["payload"]["failure_total"], 3)
self.assertEqual(assessments[0]["payload"]["max_extra_rounds"], 0)
def test_contract_root_third_failure_goes_to_refreeze(self):
state = self._two_failures()
action = drive(state, fail_verdict(root_cause_hint="contract"))
self.assertEqual(action.kind, "refreeze")
self.assertEqual(action.next_state.status, "refreeze")
self.assertEqual(action.next_state.rebuild_no, 0)
self.assertEqual(action.compat_hint, "audit_wiring_event:escalate_refreeze")
# No rebuild event, and no assessment row forced into the
# wrong determination (see #247: early_refreeze has no
# assessment determination).
kinds = [e["event_type"] for e in action.ledger_events]
self.assertNotIn("audit_rebuild", kinds)
self.assertNotIn("audit_third_failure_assessment", kinds)
def test_near_complete_gets_one_targeted_round_then_rebuilds(self):
state = self._two_failures()
action = drive(state, near_complete_verdict())
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.scope, "targeted")
self.assertEqual(action.targeted_items, ["edge-case"])
self.assertTrue(action.next_state.targeted_used)
self.assertEqual(action.next_state.pushback_round, 3)
assessments = [e for e in action.ledger_events
if e["event_type"] == "audit_third_failure_assessment"]
self.assertEqual(assessments[0]["payload"]["determination"],
"continuation_round")
self.assertEqual(assessments[0]["payload"]["max_extra_rounds"], 1)
# One more failure is the 4th: rebuild, not a 4th pushback.
action = drive(action.next_state, fail_verdict())
self.assertEqual(action.kind, "rebuild")
def test_targeted_budget_not_replenished_by_pass(self):
state = self._two_failures()
state = drive(state, near_complete_verdict()).next_state
self.assertTrue(state.targeted_used)
state = drive(state, pass_verdict()).next_state
self.assertTrue(state.targeted_used)
for _ in range(2):
state = drive(state, fail_verdict()).next_state
action = drive(state, near_complete_verdict())
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.scope, "normal")
self.assertIsNone(action.targeted_items)
def test_unknown_broken_passed_disqualifies_near_complete(self):
state = self._two_failures()
# broken_passed_items absent (unknown, not []) -> normal.
action = drive(state, fail_verdict(
failures=[{"check": "edge-case"}],
passed_items=[{"check": "p1"}, {"check": "p2"}, {"check": "p3"}]))
self.assertEqual(action.scope, "normal")
class RebuildGateAndAcceptanceTestCase(unittest.TestCase):
def _rebuilding_state(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict()).next_state
return drive(state, fail_verdict(retained_tests=["t1", "t2"])).next_state
def test_old_test_gate_missing_result_counts_as_failure(self):
state = self._rebuilding_state()
action = after_fix(state, {"reaudit_verdict": pass_verdict()})
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.next_state.failure_count, 1)
self.assertEqual(action.next_state.status, "pushback")
checks = [item.get("check") for item in action.pushback_package["failures"]]
self.assertIn("rebuild_first_gate", checks)
def test_old_test_gate_failing_result_counts_as_failure(self):
state = self._rebuilding_state()
action = after_fix(state, {
"reaudit_verdict": pass_verdict(),
"retained_test_results": {"t1": True, "t2": False},
})
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.next_state.failure_count, 1)
def test_old_test_gate_passing_allows_reaudit_pass(self):
state = self._rebuilding_state()
action = after_fix(state, {
"reaudit_verdict": pass_verdict(),
"retained_test_results": {"t1": True, "t2": True},
})
self.assertEqual(action.kind, "pass")
self.assertEqual(action.next_state.status, "passed")
def _acceptance_pending(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict(source="acceptance")).next_state
return drive(state, fail_verdict(source="acceptance"))
def test_acceptance_rebuild_unconfirmed_does_not_start(self):
action = self._acceptance_pending()
self.assertEqual(action.kind, "pending_user_confirmation")
self.assertTrue(action.next_state.pending_confirmation)
self.assertEqual(action.next_state.rebuild_no, 0)
self.assertEqual(action.next_state.status, "pending_user_confirmation")
notices = [e for e in action.ledger_events
if e["event_type"] == "audit_rebuild_notice"]
self.assertEqual(notices[0]["payload"]["confirmation_status"], "pending")
def test_acceptance_rebuild_confirmed_then_starts(self):
pending = self._acceptance_pending().next_state
action = confirm_rebuild(pending, user_confirmed=True)
self.assertEqual(action.kind, "rebuild")
self.assertEqual(action.rebuild_no, 1)
self.assertEqual(action.next_state.status, "rebuilding")
self.assertFalse(action.next_state.pending_confirmation)
notices = [e for e in action.ledger_events
if e["event_type"] == "audit_rebuild_notice"]
self.assertEqual(notices[0]["payload"]["confirmation_status"], "confirmed")
self.assertEqual(notices[0]["payload"]["confirmed_by"], "user")
def test_acceptance_rebuild_rejected_escalates_without_budget(self):
pending = self._acceptance_pending().next_state
action = confirm_rebuild(pending, user_confirmed=False)
self.assertEqual(action.kind, "escalate")
self.assertEqual(action.escalation_target, "coordinator")
self.assertEqual(action.next_state.rebuild_no, 0)
def test_confirm_without_pending_raises(self):
state = new_loop_state("m1", "v1")
with self.assertRaises(ValueError):
confirm_rebuild(state, user_confirmed=True)
class AfterFixTestCase(unittest.TestCase):
def test_after_fix_pass_clears(self):
state = new_loop_state("m1", "v1")
state = drive(state, fail_verdict()).next_state
action = after_fix(state, {"reaudit_verdict": pass_verdict()})
self.assertEqual(action.kind, "pass")
self.assertEqual(action.next_state.failure_count, 0)
def test_after_fix_failing_reaudit_goes_to_next_pushback(self):
state = new_loop_state("m1", "v1")
state = drive(state, fail_verdict()).next_state
action = after_fix(state, {"reaudit_verdict": fail_verdict()})
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.next_state.failure_count, 2)
self.assertEqual(action.next_state.pushback_round, 2)
def test_after_fix_diff_only_raises(self):
state = new_loop_state("m1", "v1")
state = drive(state, fail_verdict()).next_state
with self.assertRaises(ValueError):
after_fix(state, {"diff": "--- a/x\n+++ b/x\n"})
with self.assertRaises(ValueError):
after_fix(state, {"diff_only": True,
"reaudit_verdict": pass_verdict()})
class CorrectionTestCase(unittest.TestCase):
def test_correction_to_pass_does_not_unblock(self):
state = new_loop_state("m1", "v1")
for _ in range(2):
state = drive(state, fail_verdict()).next_state
action = apply_correction(state, {
"original_verdict": "fail", "corrected_verdict": "pass",
"reviewer": "auditor-b", "original_entry_id": "e1",
"reason": "second look: evidence was sufficient",
})
# Counting view: effective count is 0 ...
self.assertEqual(action.next_state.failure_count, 0)
# ... but the gate view is still blocked until a fresh pass.
self.assertTrue(action.merge_blocked)
self.assertTrue(action.release_blocked)
self.assertEqual(action.next_state.status, "blocked")
self.assertTrue(action.requires_reaudit)
self.assertEqual(action.correction_action, "schedule_reaudit")
self.assertEqual(action.ledger_events[0]["event_type"], "audit_meta_review")
# Only a fresh passing verdict unblocks.
action = drive(action.next_state, pass_verdict())
self.assertEqual(action.kind, "pass")
self.assertFalse(action.merge_blocked)
def test_correction_to_fail_reblocks_at_once(self):
state = drive(new_loop_state("m1", "v1"), pass_verdict()).next_state
action = apply_correction(state, {
"original_verdict": "pass", "corrected_verdict": "fail",
"reviewer": "auditor-b", "original_entry_id": "e1",
"reason": "missed a failing check",
})
self.assertEqual(action.kind, "pushback")
self.assertTrue(action.merge_blocked)
self.assertEqual(action.next_state.failure_count, 1)
self.assertEqual(action.correction_action, "retrigger_pushback_and_reblock")
class QuotaAndCostTestCase(unittest.TestCase):
def test_quota_block_consumes_nothing(self):
state = new_loop_state("m1", "v1")
for _ in range(2):
state = drive(state, fail_verdict()).next_state
action = drive(state, fail_verdict(),
quota_check=lambda estimate: False)
self.assertEqual(action.kind, "quota_blocked")
self.assertIs(action.next_state, state)
self.assertEqual(action.next_state.failure_count, 2)
self.assertEqual(action.next_state.pushback_round, 2)
def test_quota_block_at_rebuild_consumes_no_rebuild(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict()).next_state
action = drive(state, fail_verdict(),
quota_check=lambda estimate: False)
self.assertEqual(action.kind, "quota_blocked")
self.assertEqual(action.next_state.rebuild_no, 0)
self.assertEqual(action.next_state.failure_count, 3)
def test_quota_check_exception_fails_closed(self):
state = new_loop_state("m1", "v1")
def _boom(estimate):
raise RuntimeError("quota service down")
action = drive(state, fail_verdict(), quota_check=_boom)
self.assertEqual(action.kind, "quota_blocked")
self.assertEqual(action.next_state.failure_count, 0)
def test_cost_threshold_escalates_rebuild_to_user(self):
state = new_loop_state("m1", "v1")
for _ in range(3):
state = drive(state, fail_verdict()).next_state
action = drive(state, fail_verdict(cost_estimate=500),
cost_threshold=100)
self.assertEqual(action.kind, "escalate")
self.assertEqual(action.escalation_target, "user")
self.assertEqual(action.next_state.rebuild_no, 0)
self.assertEqual(action.cost_estimate, 500)
self.assertGreaterEqual(len(action.escalation_history), 4)
self.assertFalse(action.escalation_grants_user_acceptance)
def test_cost_threshold_not_applied_to_pushback(self):
# #244 scope preserved: the threshold gates rebuild steps,
# a fix pushback is quota-checked only.
state = new_loop_state("m1", "v1")
action = drive(state, fail_verdict(cost_estimate=500),
cost_threshold=100)
self.assertEqual(action.kind, "pushback")
class TestChangeTestCase(unittest.TestCase):
def test_test_deletion_violation_does_not_advance(self):
state = new_loop_state("m1", "v1")
state = drive(state, fail_verdict()).next_state
action = drive(state, fail_verdict(test_changes=[
{"action": "delete", "test_id": "t1"}]))
self.assertEqual(action.kind, "escalate")
self.assertTrue(action.violations)
self.assertEqual(action.next_state.failure_count, 1)
self.assertEqual(action.next_state.pushback_round, 1)
def test_test_deletion_allowed_with_refreeze_and_audit(self):
state = new_loop_state("m1", "v1")
action = drive(state, fail_verdict(test_changes=[
{"action": "delete", "test_id": "t1",
"separate_audit_id": "audit-9"}],
refreeze_approved=True, separate_audit_ids=["audit-9"]))
self.assertEqual(action.kind, "pushback")
self.assertEqual(action.violations, [])
class LedgerEventTestCase(unittest.TestCase):
def test_all_seven_event_types_validate(self):
cases = {
"audit_verdict": {"module_id": "m1", "contract_version": "v1",
"verdict": "pass", "source": "audit", "failures": []},
"audit_pushback_round": {"module_id": "m1", "contract_version": "v1",
"round_no": 1, "trigger_items": ["ci_green"],
"pushback_package": {}, "round_status": "pushed"},
"audit_rebuild": {"module_id": "m1", "contract_version": "v1",
"rebuild_no": 1, "root_cause": "implementation",
"status": "decided", "retained_assets": {}},
"audit_third_failure_assessment": {
"module_id": "m1", "contract_version": "v1",
"determination": "continuation_round", "basis": {},
"failure_total": 3, "max_extra_rounds": 1},
"audit_test_change": {"module_id": "m1", "contract_version": "v1",
"test_changes": {"added": ["t9"], "modified": [],
"deleted": []},
"auditor": "auditor-a",
"contract_refrozen": False, "reason": "add only"},
"audit_meta_review": {"original_entry_id": "e1",
"original_verdict": "fail",
"corrected_verdict": "pass",
"misjudgment_found": True,
"reviewer": "auditor-b", "findings": [],
"triggered_process": "re-review"},
"audit_rebuild_notice": {"module_id": "m1", "contract_version": "v1",
"rebuild_no": 1,
"notification_content": "notice text",
"notified_party": "user",
"confirmation_status": "pending"},
}
for event_type, payload in cases.items():
self.assertEqual(validate_ledger_event(event_type, payload), [],
event_type)
event = emit(event_type, payload)
self.assertEqual(event["event_type"], event_type)
self.assertEqual(set(LEDGER_EVENT_TYPES), set(cases))
def test_unknown_event_type_and_missing_field_rejected(self):
self.assertTrue(validate_ledger_event("nope", {}))
with self.assertRaises(ValueError):
emit("audit_verdict", {"module_id": "m1"})
with self.assertRaises(ValueError):
emit("audit_verdict", {"module_id": "m1", "contract_version": "v1",
"verdict": "pass", "source": "audit"})
def test_conditional_maps_to_fail_and_keeps_original(self):
event = emit("audit_verdict", {
"module_id": "m1", "contract_version": "v1",
"verdict": "conditional", "source": "audit", "failures": []})
self.assertEqual(event["payload"]["verdict"], "fail")
self.assertEqual(event["payload"]["extra"]["original_verdict"],
"conditional")
# A driver verdict of conditional is a fail, not a pass.
action = drive(new_loop_state("m1", "v1"),
fail_verdict(verdict="conditional"))
self.assertEqual(action.kind, "pushback")
def test_test_change_hard_gate_in_emit(self):
payload = {"module_id": "m1", "contract_version": "v1",
"test_changes": {"added": [], "modified": ["t1"],
"deleted": []},
"auditor": "auditor-a", "contract_refrozen": False,
"reason": "drop a test"}
with self.assertRaises(ValueError):
emit("audit_test_change", payload)
payload["contract_refrozen"] = True
payload["refreeze_ref"] = "freeze-2"
emit("audit_test_change", payload) # no raise
def test_rebuild_notice_confirmed_needs_confirmer(self):
payload = {"module_id": "m1", "contract_version": "v1",
"rebuild_no": 1, "notification_content": "notice",
"notified_party": "user", "confirmation_status": "confirmed"}
with self.assertRaises(ValueError):
emit("audit_rebuild_notice", payload)
payload["confirmed_by"] = "user"
emit("audit_rebuild_notice", payload) # no raise
def test_action_kinds_are_bounded(self):
for kind in ACTION_KINDS:
self.assertIn(kind, ("pass", "pushback", "rebuild", "refreeze",
"escalate", "pending_user_confirmation",
"quota_blocked"))
action = drive(new_loop_state("m1", "v1"), fail_verdict())
self.assertIn(action.kind, ACTION_KINDS)
if __name__ == "__main__":
unittest.main()
+519
View File
@@ -0,0 +1,519 @@
"""Server wiring for the #260 loop driver (track 2, module level).
Uses a real #241/#247 chain (LedgerStore on a temp file) and a
real sqlite loop-state connection -- no production tasks, no
GitHub, no production DB. The one fake is FailingLedgerStore,
used only to prove ledger-failure fail-closed behaviour.
"""
import os
import sqlite3
import sys
import tempfile
import unittest
from pathlib import Path
try:
from cloud import audit_loop_wiring as wiring
from cloud.audit_wiring import pushback_block_for_task
from cloud.ledger import LedgerStore
from cloud.ledger import verify_chain
except ImportError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from cloud import audit_loop_wiring as wiring
from cloud.audit_wiring import pushback_block_for_task
from cloud.ledger import LedgerStore
from cloud.ledger import verify_chain
MODULE = "mod-a"
VERSION = "v1"
OLD_REF = {"archive_ref": "task:parent:pre-rebuild"}
EXTRA_PASS = {
"security_unconditional": True,
"regression_exact_artifact": True,
"demo_runnable": True,
"known_deviation_due_todo": True,
"audit_signer_separation": True,
"qa_failed_links_rerun": True,
}
HARD_PASS = {
"files_changed": ["apps/web/a.py"],
"owned_dir": "apps/web",
"protection": "protected",
"ci_status": "success",
"ci_checks": [{"name": "ci", "status": "success"}],
"required_checks": ["ci"],
"contract_version": "v1",
"contract_frozen": True,
"coordinator_approved": True,
"unresolved_review_comments": 0,
}
def fail_verdict(text="check x failed", source="audit", **extra):
verdict = {
"verdict": "fail",
"source": source,
"verdict_text": text,
"failures": [{"check": "ci_green", "detail": text}],
"passed_items": ["a", "b", "c"],
"contract_version": VERSION,
# A3: signer is the caller (auditor-1 in _verdict); the
# other three roles must be declared and distinct.
"executor": "executor-1",
"coordinator_approver": "coordinator-1",
"qa_tester": "qa-1",
}
verdict.update(extra)
return verdict
def pass_verdict(**extra):
verdict = {
"verdict": "pass",
"source": "audit",
"verdict_text": "all green",
"failures": [],
"passed_items": ["a", "b", "c"],
"contract_version": VERSION,
# A3: see fail_verdict.
"executor": "executor-1",
"coordinator_approver": "coordinator-1",
"qa_tester": "qa-1",
}
verdict.update(extra)
return verdict
class FailingLedgerStore:
"""Every append raises: proves the wiring never saves state
after a ledger failure."""
def __getattr__(self, name):
if name.startswith("append_"):
def _raise(*args, **kwargs):
raise RuntimeError("ledger down")
return _raise
raise AttributeError(name)
class WiringTestCase(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.ledger_path = os.path.join(self.tmp.name, "ledger.db")
self.store = LedgerStore(self.ledger_path)
self.conn = sqlite3.connect(":memory:")
self.task = {"id": "task-parent", "project_id": "proj-1",
"module_id": MODULE, "contract_version": VERSION}
def tearDown(self):
try:
self.store.close()
except Exception:
pass
self.conn.close()
self.tmp.cleanup()
def _block(self, task_id="task-parent"):
return pushback_block_for_task(self.store.conn, task_id)
def _verdict(self, verdict, **kwargs):
kwargs.setdefault("user_id", "auditor-1")
return wiring.process_audit_verdict(
self.conn, self.store, self.task, verdict, **kwargs)
def test_module_mapping_explicit_wins_and_fallback_is_task_id(self):
self.assertEqual(
wiring.resolve_module_id(
{"id": "t1", "module_id": "m-explicit"}), "m-explicit")
self.assertEqual(
wiring.resolve_module_id({"id": "t1"}), "t1")
self.assertEqual(
wiring.resolve_module_id({"id": "t1", "module": "m2"}), "m2")
with self.assertRaises(ValueError):
wiring.resolve_contract_version({"id": "t1"}, None)
self.assertEqual(
wiring.resolve_contract_version(
{"id": "t1"}, {"contract_version": "v9"}), "v9")
def test_state_persists_and_reloads_from_table_not_memory(self):
result = self._verdict(fail_verdict())
self.assertEqual(result["action_kind"], "pushback")
self.assertEqual(result["subtask"]["kind"], "fix")
self.assertTrue(result["merge_blocked"])
reloaded = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(reloaded.pushback_round, 1)
self.assertEqual(reloaded.failure_count, 1)
self.assertEqual(reloaded.status, "pushback")
self.assertTrue(self._block()["merge_blocked"])
# The chain itself verifies: the wiring wrote real rows.
self.assertTrue(verify_chain(self.store.conn)["ok"])
def test_full_chain_fail_fix_reaudit_pass_unblocks(self):
self._verdict(fail_verdict())
fix_task = {"id": "task-fix-1", "project_id": "proj-1",
"module_id": MODULE, "contract_version": VERSION}
result = wiring.process_fix_completion(
self.conn, self.store, fix_task,
{"reaudit_verdict": pass_verdict()},
automated={"hard_gate": HARD_PASS, "extra_checks": EXTRA_PASS},
user_id="auditor-1", ledger_task_id="task-parent")
self.assertEqual(result["action_kind"], "pass")
self.assertIsNone(result["subtask"])
self.assertFalse(result["merge_blocked"])
self.assertTrue(
result["automated_checks"]["automated_checks_pass"])
# Parent task dict untouched by the wiring layer: the
# original task is never rolled back by this module.
self.assertNotIn("status", self.task)
self.assertFalse(self._block()["merge_blocked"])
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(state.status, "passed")
self.assertEqual(state.failure_count, 0)
def test_automated_failure_forces_claimed_pass_to_fail(self):
self._verdict(fail_verdict())
fix_task = {"id": "task-fix-1", "project_id": "proj-1",
"module_id": MODULE, "contract_version": VERSION}
broken_extra = dict(EXTRA_PASS, demo_runnable=False)
result = wiring.process_fix_completion(
self.conn, self.store, fix_task,
{"reaudit_verdict": pass_verdict()},
automated={"hard_gate": HARD_PASS,
"extra_checks": broken_extra},
user_id="auditor-1", ledger_task_id="task-parent")
# Automated checks passing is never audit passing, and
# here they did not pass: the claimed pass is forced fail.
self.assertEqual(result["action_kind"], "pushback")
self.assertTrue(result["merge_blocked"])
self.assertIn("demo_runnable",
result["automated_checks"]["failed_checks"])
self.assertTrue(self._block()["merge_blocked"])
def test_fix_without_reaudit_verdict_and_diff_only_refused(self):
self._verdict(fail_verdict())
fix_task = {"id": "task-fix-1", "project_id": "proj-1",
"module_id": MODULE, "contract_version": VERSION}
with self.assertRaises(ValueError):
wiring.process_fix_completion(
self.conn, self.store, fix_task, {"fix_diff": {}},
user_id="auditor-1", ledger_task_id="task-parent")
with self.assertRaises(ValueError):
wiring.process_fix_completion(
self.conn, self.store, fix_task,
{"diff_only": True,
"reaudit_verdict": pass_verdict()},
user_id="auditor-1", ledger_task_id="task-parent")
# Fix with no prior loop at all is refused, not invented.
fresh_conn = sqlite3.connect(":memory:")
try:
with self.assertRaises(ValueError):
wiring.process_fix_completion(
fresh_conn, self.store, fix_task,
{"reaudit_verdict": pass_verdict()},
user_id="auditor-1", ledger_task_id="task-parent")
finally:
fresh_conn.close()
def test_exactly_three_pushes_back_fourth_rebuilds(self):
for _ in range(3):
result = self._verdict(fail_verdict())
self.assertEqual(result["action_kind"], "pushback")
# Fourth failure without a caller-supplied old impl ref
# is refused BEFORE any ledger row: state stays at 3.
with self.assertRaises(ValueError):
self._verdict(fail_verdict())
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(state.pushback_round, 3)
self.assertEqual(state.failure_count, 3)
result = self._verdict(fail_verdict(), old_impl_ref=OLD_REF)
self.assertEqual(result["action_kind"], "rebuild")
self.assertEqual(result["subtask"]["kind"], "rebuild")
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(state.rebuild_no, 1)
self.assertEqual(state.status, "rebuilding")
def test_targeted_third_round_scope(self):
self._verdict(fail_verdict())
self._verdict(fail_verdict())
result = self._verdict(fail_verdict(
failures=[{"check": "ci_green", "detail": "one left"}],
passed_items=["a", "b", "c"],
broken_passed_items=[]))
self.assertEqual(result["action_kind"], "pushback")
self.assertEqual(result["subtask"]["scope"], "targeted")
self.assertEqual(result["subtask"]["targeted_items"],
["ci_green"])
def test_correction_to_pass_does_not_unblock(self):
self._verdict(fail_verdict())
verdict_rows = self.store.conn.execute(
"SELECT id FROM audit_ledger WHERE action = ?"
" ORDER BY id ASC", ("audit_verdict",)).fetchall()
entry_id = verdict_rows[0][0]
result = wiring.process_correction(
self.conn, self.store, self.task,
{"original_verdict": "fail", "corrected_verdict": "pass",
"reviewer": "reviewer-1", "original_entry_id": entry_id,
"findings": ["misjudged"], "contract_version": VERSION},
user_id="reviewer-1", ledger_task_id="task-parent")
self.assertEqual(result["correction_action"], "schedule_reaudit")
self.assertTrue(result["merge_blocked"])
self.assertEqual(result["next_state"].failure_count, 0)
self.assertEqual(result["next_state"].status, "blocked")
self.assertEqual(result["subtask"]["kind"], "reaudit")
# Counting view (0) vs gate view (still blocked) is the
# #247 intended divergence, pinned here, not papered over.
self.assertTrue(self._block()["merge_blocked"])
# Only a fresh passing verdict unblocks.
self._verdict(pass_verdict())
self.assertFalse(self._block()["merge_blocked"])
def test_correction_to_fail_reblocks_at_once(self):
self._verdict(pass_verdict())
self.assertFalse(self._block()["merge_blocked"])
verdict_rows = self.store.conn.execute(
"SELECT id FROM audit_ledger WHERE action = ?"
" ORDER BY id ASC", ("audit_verdict",)).fetchall()
entry_id = verdict_rows[-1][0]
result = wiring.process_correction(
self.conn, self.store, self.task,
{"original_verdict": "pass", "corrected_verdict": "fail",
"reviewer": "reviewer-1", "original_entry_id": entry_id,
"findings": ["missed failure"],
"contract_version": VERSION},
user_id="reviewer-1", ledger_task_id="task-parent")
self.assertEqual(
result["correction_action"], "retrigger_pushback_and_reblock")
self.assertTrue(result["merge_blocked"])
self.assertEqual(result["next_state"].failure_count, 1)
self.assertTrue(self._block()["merge_blocked"])
def test_acceptance_rebuild_pending_confirm_reject_no_budget(self):
for _ in range(4):
result = self._verdict(
fail_verdict(source="acceptance"))
self.assertEqual(result["action_kind"],
"pending_user_confirmation")
self.assertIsNone(result["subtask"])
self.assertIsNotNone(result["notification"])
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertTrue(state.pending_confirmation)
self.assertEqual(state.rebuild_no, 0)
# System/coordinator markers can never confirm.
with self.assertRaises(ValueError):
wiring.process_rebuild_confirmation(
self.conn, self.store, self.task,
user_confirmed=True, confirmed_by="system",
user_id="user-1", ledger_task_id="task-parent")
with self.assertRaises(ValueError):
wiring.process_rebuild_confirmation(
self.conn, self.store, self.task,
user_confirmed=True, confirmed_by="coordinator",
user_id="user-1", ledger_task_id="task-parent")
# Rejection escalates and consumes no rebuild budget.
rejected = wiring.process_rebuild_confirmation(
self.conn, self.store, self.task,
user_confirmed=False, confirmed_by="user-1",
user_id="user-1", ledger_task_id="task-parent")
self.assertEqual(rejected["action_kind"], "escalate")
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(state.rebuild_no, 0)
def test_acceptance_rebuild_confirm_then_rebuilds(self):
for _ in range(4):
self._verdict(fail_verdict(source="acceptance"))
confirmed = wiring.process_rebuild_confirmation(
self.conn, self.store, self.task,
user_confirmed=True, confirmed_by="user-1",
old_impl_ref=OLD_REF,
user_id="user-1", ledger_task_id="task-parent")
self.assertEqual(confirmed["action_kind"], "rebuild")
self.assertEqual(confirmed["subtask"]["kind"], "rebuild")
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(state.rebuild_no, 1)
self.assertEqual(state.status, "rebuilding")
# Confirming again with no pending confirmation fails.
with self.assertRaises(ValueError):
wiring.process_rebuild_confirmation(
self.conn, self.store, self.task,
user_confirmed=True, confirmed_by="user-1",
user_id="user-1", ledger_task_id="task-parent")
def test_quota_blocked_consumes_nothing(self):
self._verdict(fail_verdict())
blocked = self._verdict(
fail_verdict(), quota_check=lambda estimate: False)
self.assertEqual(blocked["action_kind"], "quota_blocked")
self.assertFalse(blocked["state_saved"])
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(state.pushback_round, 1)
self.assertEqual(state.failure_count, 1)
def _raising(estimate):
raise RuntimeError("quota service down")
blocked2 = self._verdict(
fail_verdict(), quota_check=_raising)
self.assertEqual(blocked2["action_kind"], "quota_blocked")
state = wiring.load_loop_state(self.conn, MODULE, VERSION)
self.assertEqual(state.pushback_round, 1)
def test_quota_adapter_matches_server_rule(self):
check = wiring.build_quota_check(1000000, 999999)
self.assertTrue(check(None))
self.assertFalse(check(0.01))
self.assertTrue(wiring.build_quota_check(None, 10 ** 9)(100.0))
def test_cost_threshold_escalates_rebuild_with_full_history(self):
for _ in range(3):
self._verdict(fail_verdict())
result = self._verdict(
fail_verdict(cost_estimate=100.0), cost_threshold=10.0,
old_impl_ref=OLD_REF)
self.assertEqual(result["action_kind"], "escalate")
self.assertEqual(result["notification"]["party"], "user")
self.assertEqual(
len(result["notification"]["escalation_history"]), 4)
self.assertFalse(
result["escalation_grants_user_acceptance"])
def test_ledger_failure_fail_closed_state_not_saved(self):
failing = FailingLedgerStore()
with self.assertRaises(RuntimeError):
wiring.process_audit_verdict(
self.conn, failing, self.task, fail_verdict(),
user_id="auditor-1")
self.assertIsNone(
wiring.load_loop_state(self.conn, MODULE, VERSION))
def test_automated_reaudit_evidence_missing_fails(self):
result = wiring.evaluate_automated_reaudit(None, {})
self.assertFalse(result["automated_checks_pass"])
self.assertIn("hard_gate_evidence_missing",
result["failed_checks"])
result = wiring.evaluate_automated_reaudit(
HARD_PASS, EXTRA_PASS)
self.assertTrue(result["automated_checks_pass"])
# Merge-body vocabulary is translated, not rejected:
# branch_protection -> the hard gate's protection.
body_style = dict(HARD_PASS)
body_style["branch_protection"] = body_style.pop("protection")
result = wiring.evaluate_automated_reaudit(
body_style, EXTRA_PASS)
self.assertTrue(result["automated_checks_pass"])
def test_release_gate_multi_source_and_read_only(self):
self._verdict(fail_verdict())
before = self.store.conn.execute(
"SELECT COUNT(*) FROM audit_ledger").fetchone()[0]
decision = wiring.evaluate_release_gate(
"task-parent", self.store.conn,
task_status="completed", hard_gate={"allowed": False,
"blocks": ["ci_not_green"]})
self.assertFalse(decision["allowed"])
self.assertEqual(decision["http_status"], 409)
self.assertEqual(decision["blocked_sources"],
["task_status", "hard_gate", "audit_pushback"])
self.assertEqual(decision["code"], "release_requires_merged")
after = self.store.conn.execute(
"SELECT COUNT(*) FROM audit_ledger").fetchone()[0]
self.assertEqual(before, after)
self.assertTrue(verify_chain(self.store.conn)["ok"])
def test_release_gate_evidence_missing_and_allowed(self):
decision = wiring.evaluate_release_gate(
"task-parent", self.store.conn,
task_status="merged", hard_gate=None)
self.assertEqual(decision["code"],
"release_hard_gate_evidence_missing")
decision = wiring.evaluate_release_gate(
"task-parent", self.store.conn,
task_status="merged",
hard_gate={"allowed": True, "blocks": []})
self.assertTrue(decision["allowed"])
self.assertEqual(decision["code"], "release_allowed")
def test_release_gate_chain_read_failure_fail_closed(self):
closed = sqlite3.connect(":memory:")
closed.close()
decision = wiring.evaluate_release_gate(
"task-parent", closed, task_status="merged",
hard_gate={"allowed": True, "blocks": []})
self.assertFalse(decision["allowed"])
self.assertEqual(
decision["code"], "release_blocked_state_unavailable")
class SignerSeparationTestCase(unittest.TestCase):
"""A3: validate_signer_separation 在提交路径上的真实调用方。
覆盖:自审(调用方 = 声明的 executor/coordinator/QA 任一)
被拒且不落链不落状态;缺字段被拒;体里 signer 与调用方
不符被拒;声明三方均与调用方不同时正常提交。此前该校验
只活在重审清单函数里(collect_failed_checks,经无生产
调用方的 decide_pushback),提交路径上从未执行。
注意:本类故意不继承 WiringTestCase——继承会使父类全部
测试方法在本类下再跑一遍(全量计数虚增),故只复用其
setUp 形状,不复用其用例。
"""
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.ledger_path = os.path.join(self.tmp.name, "ledger.db")
self.store = LedgerStore(self.ledger_path)
self.conn = sqlite3.connect(":memory:")
self.task = {"id": "task-parent", "project_id": "proj-1",
"module_id": MODULE, "contract_version": VERSION}
def tearDown(self):
try:
self.store.close()
except Exception:
pass
self.conn.close()
self.tmp.cleanup()
def _verdict(self, verdict, **kwargs):
kwargs.setdefault("user_id", "auditor-1")
return wiring.process_audit_verdict(
self.conn, self.store, self.task, verdict, **kwargs)
def _chain_rows(self):
return self.store.conn.execute(
"SELECT COUNT(*) FROM audit_ledger").fetchone()[0]
def test_self_audit_rejected_without_side_effects(self):
for field in ("executor", "coordinator_approver", "qa_tester"):
with self.subTest(field=field):
verdict = fail_verdict(**{field: "auditor-1"})
with self.assertRaises(ValueError) as caught:
self._verdict(verdict)
self.assertIn("signer", str(caught.exception))
self.assertIsNone(
wiring.load_loop_state(self.conn, MODULE, VERSION))
self.assertEqual(self._chain_rows(), 0)
def test_missing_identity_field_rejected(self):
verdict = fail_verdict()
del verdict["qa_tester"]
with self.assertRaises(ValueError):
self._verdict(verdict)
self.assertIsNone(
wiring.load_loop_state(self.conn, MODULE, VERSION))
self.assertEqual(self._chain_rows(), 0)
def test_forged_signer_rejected(self):
verdict = fail_verdict(signer="someone-else")
with self.assertRaises(ValueError):
self._verdict(verdict)
self.assertEqual(self._chain_rows(), 0)
def test_distinct_declared_roles_submit(self):
result = self._verdict(pass_verdict())
self.assertEqual(result["action_kind"], "pass")
self.assertFalse(result["merge_blocked"])
if __name__ == "__main__":
unittest.main()
+892
View File
@@ -0,0 +1,892 @@
"""Independent-audit auto-pushback: decision, package, ledger.
Pins the six required behaviours: (1) audit fail auto-pushes back,
(2) merge/release stay blocked until a full re-audit passes,
(3) the pushback package carries per-item evidence, (4) the round
budget escalates instead of looping forever, (5) a fix gets a full
re-audit, not a failed-items-only re-check, (6) every round stays
visible in the audit ledger. Plus the fail-closed edges: CONDITIONAL
is not a pass, missing checks fail, signer separation, exact-
artifact regression, runnable demos, deviation expiry, and quota
for fix tasks.
Style follows the other cloud tests: unittest, stdlib only, import
with a sys.path fallback so the file also runs standalone.
"""
import sys
import unittest
from pathlib import Path
try:
from cloud.audit_pushback import (
MAX_PUSHBACK_ROUNDS,
MAX_REBUILDS_PER_CONTRACT_VERSION,
build_acceptance_rebuild_notice,
build_audit_correction,
build_ledger_entry,
build_pushback_package,
build_rebuild_package,
check_g6_guard,
check_rebuild_first_gate,
classify_root_cause,
confirm_acceptance_rebuild,
decide_pushback,
deviation_is_blocking,
evaluate_regression,
full_reaudit_criteria,
gate_block,
new_module_failure_state,
quota_allows,
record_module_failure,
record_module_pass,
record_module_rebuild,
record_module_targeted_fix,
select_audit_sample,
triage_third_failure,
validate_golden_task_set,
validate_ledger_chain,
validate_signer_separation,
validate_test_changes,
verify_previous_artifacts,
)
except ImportError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from cloud.audit_pushback import (
MAX_PUSHBACK_ROUNDS,
MAX_REBUILDS_PER_CONTRACT_VERSION,
build_acceptance_rebuild_notice,
build_audit_correction,
build_ledger_entry,
build_pushback_package,
build_rebuild_package,
check_g6_guard,
check_rebuild_first_gate,
classify_root_cause,
confirm_acceptance_rebuild,
decide_pushback,
deviation_is_blocking,
evaluate_regression,
full_reaudit_criteria,
gate_block,
new_module_failure_state,
quota_allows,
record_module_failure,
record_module_pass,
record_module_rebuild,
record_module_targeted_fix,
select_audit_sample,
triage_third_failure,
validate_golden_task_set,
validate_ledger_chain,
validate_signer_separation,
validate_test_changes,
verify_previous_artifacts,
)
ALL_CHECKS = (
"ci_green",
"directory_isolation",
"contract_version",
"coordinator_approval",
"unresolved_review_comments",
"security_unconditional",
"regression_exact_artifact",
"demo_runnable",
"known_deviation_due_todo",
"audit_signer_separation",
)
def passing_audit(**overrides):
audit = {
"task_id": "task-1",
"verdict": "pass",
"verdict_text": "audit pass: all checks verified independently",
"signer": "auditor-a",
"executor": "executor-e",
"coordinator_approver": "coordinator-c",
"qa_tester": "qa-q",
"security_verdict": "pass",
"checks": {name: True for name in ALL_CHECKS},
"failed_items": [],
"deviations": [],
"regression": {"items": [
{"path": "apps/web/app.py", "sha256_manifest": "a" * 64,
"sha256_recomputed": "a" * 64, "source": "exact"},
]},
"demo": {"verified_runnable": True, "runs_app_entry": True,
"command": "python3 apps/web/app.py"},
}
audit.update(overrides)
return audit
class DecideTestCase(unittest.TestCase):
def test_fail_verdict_auto_pushbacks_and_blocks(self):
decision = decide_pushback(passing_audit(verdict="fail"), 0)
self.assertEqual(decision["action"], "pushback")
self.assertTrue(decision["merge_blocked"])
self.assertTrue(decision["release_blocked"])
self.assertEqual(decision["pushback_form"], "derived_fix_subtask")
self.assertEqual(decision["next_round"], 1)
def test_conditional_verdict_is_not_a_pass(self):
for verdict in ("conditional", "fail"):
with self.subTest(verdict=verdict):
decision = decide_pushback(passing_audit(verdict=verdict), 0)
self.assertEqual(decision["action"], "pushback")
def test_conditional_security_blocks_even_with_pass_verdict(self):
audit = passing_audit(security_verdict="conditional")
decision = decide_pushback(audit, 0)
self.assertEqual(decision["action"], "pushback")
self.assertIn("security_unconditional", decision["failed_checks"])
def test_each_gate_check_failing_triggers_pushback(self):
for check in ALL_CHECKS:
with self.subTest(check=check):
checks = {name: True for name in ALL_CHECKS}
checks[check] = False
decision = decide_pushback(passing_audit(checks=checks), 0)
self.assertEqual(decision["action"], "pushback")
self.assertIn(check, decision["failed_checks"])
def test_missing_check_fails_closed(self):
checks = {name: True for name in ALL_CHECKS}
del checks["ci_green"]
decision = decide_pushback(passing_audit(checks=checks), 0)
self.assertEqual(decision["action"], "pushback")
self.assertIn("ci_green", decision["failed_checks"])
def test_pass_allows_merge_and_release(self):
decision = decide_pushback(passing_audit(), 0)
self.assertEqual(decision["action"], "allow")
self.assertFalse(decision["merge_blocked"])
self.assertFalse(decision["release_blocked"])
self.assertEqual(decision["gate"]["blocked_by"], None)
def test_gate_block_interface(self):
blocked = gate_block("pushback", ["ci_green"], 1)
self.assertTrue(blocked["merge_blocked"])
self.assertTrue(blocked["release_blocked"])
self.assertEqual(blocked["blocked_by"], "audit_pushback")
self.assertEqual(blocked["failed_checks"], ["ci_green"])
class SeparationTestCase(unittest.TestCase):
def test_signer_equal_to_executor_coordinator_or_qa_invalid(self):
for field, value in (("executor", "auditor-a"),
("coordinator_approver", "auditor-a"),
("qa_tester", "auditor-a")):
with self.subTest(field=field):
audit = passing_audit(**{field: "auditor-a"})
valid, _reasons = validate_signer_separation(audit)
self.assertFalse(valid)
decision = decide_pushback(audit, 0)
self.assertEqual(decision["action"], "pushback")
self.assertIn("audit_signer_separation",
decision["failed_checks"])
def test_distinct_signer_is_valid(self):
valid, reasons = validate_signer_separation(passing_audit())
self.assertTrue(valid)
self.assertEqual(reasons, [])
class DeviationTestCase(unittest.TestCase):
def test_deviation_without_due_or_todo_blocks(self):
self.assertTrue(deviation_is_blocking({"id": "d1"}))
self.assertTrue(deviation_is_blocking(
{"id": "d1", "todo": "follow up", "due_date": None}))
self.assertTrue(deviation_is_blocking(
{"id": "d1", "due_date": "2099-01-01"}))
def test_deviation_with_future_due_and_todo_does_not_block(self):
self.assertFalse(deviation_is_blocking(
{"id": "d1", "todo": "follow up", "due_date": "2099-01-01"},
today="2026-10-31"))
def test_expired_deviation_blocks_until_followed_up(self):
expired = {"id": "d1", "todo": "follow up", "due_date": "2026-01-01"}
self.assertTrue(deviation_is_blocking(expired, today="2026-10-31"))
self.assertFalse(deviation_is_blocking(
dict(expired, followed_up=True), today="2026-10-31"))
def test_expired_deviation_triggers_pushback(self):
audit = passing_audit(deviations=[
{"id": "d1", "todo": "follow up", "due_date": "2026-01-01"}])
decision = decide_pushback(audit, 0, today="2026-10-31")
self.assertEqual(decision["action"], "pushback")
self.assertIn("known_deviation_due_todo", decision["failed_checks"])
class EvidenceTestCase(unittest.TestCase):
def test_regression_hash_mismatch_and_fixture_block(self):
mismatch = [{"path": "x.py", "sha256_manifest": "a" * 64,
"sha256_recomputed": "b" * 64, "source": "exact"}]
fixture = [{"path": "x.py", "sha256_manifest": "a" * 64,
"sha256_recomputed": "a" * 64, "source": "fixture"}]
self.assertTrue(evaluate_regression(mismatch))
self.assertTrue(evaluate_regression(fixture))
self.assertEqual(evaluate_regression([
{"path": "x.py", "sha256_manifest": "a" * 64,
"sha256_recomputed": "a" * 64, "source": "exact"}]), [])
def test_demo_must_run_the_app_entry(self):
audit = passing_audit(demo={"verified_runnable": True,
"runs_app_entry": False,
"command": "python3 -m http.server"})
decision = decide_pushback(audit, 0)
self.assertEqual(decision["action"], "pushback")
self.assertIn("demo_runnable", decision["failed_checks"])
def test_pushback_package_carries_evidence_and_full_reaudit(self):
audit = passing_audit(
verdict="fail",
verdict_text="AUDIT-FAIL verbatim: ci red, demo not runnable",
failed_items=[{
"check": "ci_green",
"detail": "checks red on current head",
"evidence": {"test_name": "test_x", "file_path": "a/b.py",
"line_no": 42, "contract_clause": "G2 v1",
"diff_range": "a/b.py@10-20"},
"reproduction_steps": ["run CI on the PR head"],
"reaudit_criterion": "CI green on the fixed head",
}],
checks={**{name: True for name in ALL_CHECKS},
"ci_green": False},
)
decision = decide_pushback(audit, 1)
package = build_pushback_package(
audit, decision,
[{"path": "apps/web/app.py", "sha256": "a" * 64, "bytes": 123}])
self.assertEqual(package["audit_verdict_text"],
"AUDIT-FAIL verbatim: ci red, demo not runnable")
item = package["failed_items"][0]
self.assertEqual(item["evidence"]["test_name"], "test_x")
self.assertEqual(item["evidence"]["line_no"], 42)
self.assertEqual(item["evidence"]["contract_clause"], "G2 v1")
self.assertEqual(item["reproduction_steps"], ["run CI on the PR head"])
# Full re-audit, not failed-items-only.
self.assertEqual(package["reaudit_criteria"], full_reaudit_criteria())
self.assertEqual(len(package["reaudit_criteria"]), len(ALL_CHECKS))
self.assertEqual(package["previous_artifacts"][0]["sha256"], "a" * 64)
self.assertTrue(package["derived_subtask"]["merge_blocked_until_reaudit_pass"])
class PreviousArtifactTestCase(unittest.TestCase):
PREVIOUS = [{"path": "a.py", "sha256": "a" * 64},
{"path": "b.py", "sha256": "b" * 64}]
def test_dropped_or_silently_changed_artifact_is_a_violation(self):
dropped = verify_previous_artifacts(
self.PREVIOUS, [{"path": "a.py", "sha256": "a" * 64}])
self.assertFalse(dropped["ok"])
self.assertIn("dropped:b.py", dropped["violations"])
silent = verify_previous_artifacts(
self.PREVIOUS,
[{"path": "a.py", "sha256": "a" * 64},
{"path": "b.py", "sha256": "c" * 64}])
self.assertFalse(silent["ok"])
self.assertIn("silently_changed:b.py", silent["violations"])
def test_changed_in_fix_diff_is_reauditable_not_a_violation(self):
result = verify_previous_artifacts(
self.PREVIOUS,
[{"path": "a.py", "sha256": "a" * 64},
{"path": "b.py", "sha256": "c" * 64}],
fix_diff_paths=["b.py"])
self.assertTrue(result["ok"])
self.assertEqual(result["changed_and_reauditable"], ["b.py"])
class LimitAndQuotaTestCase(unittest.TestCase):
def test_pushback_budget_exhausted_rebuilds_first_with_full_history(self):
# Over-3 rule (follow-up, superseding the earlier
# "exhausted -> escalate" wording): the 4th failure rebuilds
# first; escalation comes only after the rebuild budget too.
history = [{"round": 0, "action": "pushback"},
{"round": 1, "action": "pushback"},
{"round": 2, "action": "pushback"}]
decision = decide_pushback(passing_audit(verdict="fail"), 3,
history=history)
self.assertEqual(decision["action"], "rebuild")
self.assertEqual(decision["rebuild_no"], 1)
self.assertTrue(decision["merge_blocked"])
# Full history, not just the last round.
self.assertEqual(len(decision["history"]), 4)
self.assertEqual(decision["history"][0]["round"], 0)
self.assertEqual(decision["history"][-1]["round"], 3)
def test_rebuild_budget_exhausted_escalates_to_coordinator(self):
decision = decide_pushback(passing_audit(verdict="fail"), 3,
rebuilds_used=1)
self.assertEqual(decision["action"], "escalate")
self.assertEqual(decision["escalation_target"], "coordinator")
self.assertFalse(decision["escalation_grants_user_acceptance"])
self.assertTrue(decision["merge_blocked"])
def test_max_rounds_constant_is_three(self):
self.assertEqual(MAX_PUSHBACK_ROUNDS, 3)
def test_ci_auto_fix_rounds_do_not_consume_pushback_budget(self):
# Same limit value, separate counter: 3 CI fixes must not
# turn a first-round audit failure into an escalation.
decision = decide_pushback(passing_audit(verdict="fail"), 0,
ci_auto_fix_rounds=3)
self.assertEqual(decision["action"], "pushback")
self.assertFalse(decision["counters"]["ci_auto_fix_rounds_shared"])
def test_fix_task_quota_block_consumes_no_pushback_round(self):
decision = decide_pushback(passing_audit(verdict="fail"), 1,
quota_used=90, quota_limit=100,
fix_cost_estimate=50)
self.assertEqual(decision["action"], "blocked_quota")
self.assertTrue(decision["merge_blocked"])
self.assertEqual(decision["next_round"], 1)
self.assertTrue(quota_allows(90, 10, 100))
self.assertFalse(quota_allows(90, 50, 100))
class FullReauditTestCase(unittest.TestCase):
def test_previously_passing_check_broken_by_fix_still_fails(self):
# A fix that repairs ci_green but breaks directory isolation
# must not pass on a failed-items-only re-check.
checks = {name: True for name in ALL_CHECKS}
checks["directory_isolation"] = False
decision = decide_pushback(passing_audit(checks=checks), 1)
self.assertEqual(decision["action"], "pushback")
self.assertIn("directory_isolation", decision["failed_checks"])
self.assertEqual(decision["next_round"], 2)
class LedgerTestCase(unittest.TestCase):
def _chain(self):
first = build_ledger_entry("pushback", "task-1", 0,
trigger_checks=["ci_green"],
package_ref="pkg-0")
second = build_ledger_entry("fix", "task-1", 1, fix_diff=["a.py"],
prev_hash=first["entry_hash"])
third = build_ledger_entry("reaudit", "task-1", 1,
reaudit_result="fail",
prev_hash=second["entry_hash"])
return [first, second, third]
def test_chain_validates_and_keeps_every_round(self):
entries = self._chain()
valid, errors = validate_ledger_chain(entries)
self.assertTrue(valid, errors)
self.assertEqual([entry["kind"] for entry in entries],
["pushback", "fix", "reaudit"])
def test_tampered_entry_breaks_validation(self):
entries = self._chain()
entries[1]["fix_diff"] = ["tampered.py"]
valid, errors = validate_ledger_chain(entries)
self.assertFalse(valid)
self.assertTrue(errors)
def test_broken_prev_link_breaks_validation(self):
entries = self._chain()
entries[2]["prev_hash"] = "0" * 64
valid, _errors = validate_ledger_chain(entries)
self.assertFalse(valid)
def test_unknown_ledger_kind_rejected(self):
with self.assertRaises(ValueError):
build_ledger_entry("bogus", "task-1", 0)
class ModuleCountingTestCase(unittest.TestCase):
def test_audit_and_acceptance_failures_combine_but_stay_separate(self):
state = new_module_failure_state("mod-web", "v1")
state = record_module_failure(state, "audit", "v1")
state = record_module_failure(state, "acceptance", "v1")
state = record_module_failure(state, "audit", "v1")
self.assertEqual(state["audit_fail_count"], 2)
self.assertEqual(state["acceptance_fail_count"], 1)
self.assertEqual(state["total_fail_count"], 3)
def test_pass_resets_failures_but_not_rebuilds(self):
state = new_module_failure_state("mod-web", "v1")
state = record_module_failure(state, "audit", "v1")
state = record_module_rebuild(state, 1)
state = record_module_pass(state, "v1")
self.assertEqual(state["total_fail_count"], 0)
self.assertEqual(state["rebuilds_used"], 1)
def test_contract_version_change_resets_failures_and_rebuilds(self):
state = new_module_failure_state("mod-web", "v1")
state = record_module_failure(state, "audit", "v1")
state = record_module_rebuild(state, 1)
state = record_module_failure(state, "audit", "v2")
self.assertEqual(state["contract_version"], "v2")
self.assertEqual(state["total_fail_count"], 1)
self.assertEqual(state["rebuilds_used"], 0)
def test_boundary_exactly_three_pushbacks_fourth_rebuilds(self):
# Failures 1-3 (round_no 0,1,2) push back for a fix; the
# 4th failure (round_no 3, total > 3) rebuilds instead.
for round_no in (0, 1, 2):
with self.subTest(round_no=round_no):
decision = decide_pushback(
passing_audit(verdict="fail"), round_no)
self.assertEqual(decision["action"], "pushback")
decision = decide_pushback(passing_audit(verdict="fail"), 3)
self.assertEqual(decision["action"], "rebuild")
self.assertEqual(MAX_PUSHBACK_ROUNDS, 3)
self.assertEqual(MAX_REBUILDS_PER_CONTRACT_VERSION, 1)
def test_module_tracker_boundary_via_counts(self):
state = new_module_failure_state("mod-web", "v1")
for _ in range(3):
state = record_module_failure(state, "audit", "v1")
self.assertEqual(state["total_fail_count"], 3)
state = record_module_failure(state, "audit", "v1")
self.assertEqual(state["total_fail_count"], 4)
self.assertTrue(state["total_fail_count"] > 3)
class RebuildPackageTestCase(unittest.TestCase):
def _rebuild_decision(self):
return decide_pushback(passing_audit(verdict="fail"), 3)
def test_rebuild_scopes_to_one_module_only(self):
package = build_rebuild_package(
"mod-web", self._rebuild_decision(), [],
[], old_implementation_ref="abc1234",
other_module_ids=["mod-ios", "mod-android"],
contract_version="v1")
self.assertEqual(package["scope"]["rebuild_only"], "mod-web")
self.assertEqual(package["scope"]["untouched_modules"],
["mod-ios", "mod-android"])
self.assertTrue(package["scope"]["frozen_contract_unchanged"])
self.assertTrue(package["scope"]["design_unchanged"])
self.assertTrue(package["scope"]["other_modules_passed_artifacts_untouched"])
self.assertTrue(package["scope"]["directory_isolation_unchanged"])
def test_old_implementation_archived_not_deleted(self):
package = build_rebuild_package(
"mod-web", self._rebuild_decision(), [],
[], old_implementation_ref="abc1234")
old = package["old_implementation"]
self.assertEqual(old["ref"], "abc1234")
self.assertFalse(old["deleted_from_history"])
self.assertEqual(old["workspace_action"],
"replace_with_new_implementation")
def test_rebuild_carries_avoidance_list_and_passed_items(self):
history = [passing_audit(
verdict="fail", verdict_text="round 0: ci red",
failed_items=[{"check": "ci_green",
"detail": "checks red on current head"}])]
package = build_rebuild_package(
"mod-web", self._rebuild_decision(), history, [],
old_implementation_ref="abc1234",
passed_items=[{"check": "contract_version",
"detail": "contract v1 matched"}])
kinds = [entry["kind"] for entry in package["avoidance_list"]]
self.assertIn("prior_failure", kinds)
self.assertIn("previously_passed_must_not_regress", kinds)
self.assertTrue(any("ci red" in (entry.get("detail") or "")
or "checks red" in (entry.get("detail") or "")
for entry in package["avoidance_list"]))
def test_rebuild_package_is_labelled_a_rebuild_not_a_new_task(self):
package = build_rebuild_package(
"mod-web", self._rebuild_decision(), [],
[], old_implementation_ref="abc1234")
self.assertEqual(package["kind"], "rebuild")
self.assertIn("推倒重做第 1 次", package["task_label"])
self.assertIn("mod-web", package["task_label"])
class RootCauseTestCase(unittest.TestCase):
def _contract_round(self):
return [{"check": "contract_version",
"root_cause": "contract",
"evidence": {"contract_clause": "G2 §3 timeout"}}]
def test_repeated_same_clause_classifies_contract(self):
cause, _evidence = classify_root_cause(
[self._contract_round(), self._contract_round()])
self.assertEqual(cause, "contract")
def test_single_contract_round_is_unknown_not_refreeze(self):
cause, _evidence = classify_root_cause([self._contract_round()])
self.assertEqual(cause, "unknown")
def test_implementation_rounds_classify_implementation(self):
rounds = [[{"check": "ci_green", "root_cause": "implementation"}],
[{"check": "ci_green", "root_cause": "implementation"}]]
cause, _evidence = classify_root_cause(rounds)
self.assertEqual(cause, "implementation")
def test_contract_root_cause_at_over_limit_goes_to_refreeze(self):
decision = decide_pushback(passing_audit(verdict="fail"), 3,
root_cause="contract")
self.assertEqual(decision["action"], "escalate_refreeze")
self.assertEqual(decision["escalation_target"], "coordinator")
self.assertEqual(decision["refreeze_flow"], "G2")
self.assertTrue(decision["merge_blocked"])
self.assertIsNone(decision["rebuild_no"])
def test_proved_unimplementable_twice_classifies_design(self):
rounds = [[{"check": "contract_version", "root_cause": "design",
"proved_unimplementable": True}],
[{"check": "contract_version", "root_cause": "design",
"proved_unimplementable": True}]]
cause, _evidence = classify_root_cause(rounds)
self.assertEqual(cause, "design")
class RebuildLimitAndCostTestCase(unittest.TestCase):
def test_rebuilt_module_failing_over_three_again_escalates(self):
# New implementation, fresh count, fails > 3 again, rebuild
# budget already used -> escalate, no second automatic rebuild.
decision = decide_pushback(passing_audit(verdict="fail"), 3,
rebuilds_used=1)
self.assertEqual(decision["action"], "escalate")
self.assertEqual(decision["escalation_target"], "coordinator")
def test_rebuild_exceeding_quota_blocks_without_consuming_rebuild(self):
decision = decide_pushback(
passing_audit(verdict="fail"), 3,
quota_used=90, quota_limit=100, rebuild_cost_estimate=50)
self.assertEqual(decision["action"], "blocked_quota")
self.assertTrue(decision["merge_blocked"])
def test_rebuild_cost_above_user_threshold_escalates_to_user(self):
decision = decide_pushback(
passing_audit(verdict="fail"), 3,
rebuild_cost_estimate=500, cost_escalation_threshold=100)
self.assertEqual(decision["action"], "escalate")
self.assertEqual(decision["escalation_target"], "user")
self.assertFalse(decision["escalation_grants_user_acceptance"])
self.assertIsNone(decision["rebuild_no"])
class RebuildLedgerTimelineTestCase(unittest.TestCase):
def test_timeline_shows_pushbacks_then_rebuild_then_reaudit(self):
entries = []
prev = ""
for round_no in (0, 1, 2):
entry = build_ledger_entry(
"pushback", "task-1", round_no,
trigger_checks=["ci_green"], module_id="mod-web",
contract_version="v1",
fail_counts={"audit": round_no + 1, "acceptance": 0,
"total": round_no + 1},
prev_hash=prev)
entries.append(entry)
prev = entry["entry_hash"]
rebuild = build_ledger_entry(
"rebuild", "task-1", 3, module_id="mod-web",
contract_version="v1", rebuild_no=1,
old_implementation_ref="abc1234", prev_hash=prev)
entries.append(rebuild)
reaudit = build_ledger_entry(
"reaudit", "task-1", 3, reaudit_result="pass",
module_id="mod-web", contract_version="v1", prev_hash=rebuild["entry_hash"])
entries.append(reaudit)
valid, errors = validate_ledger_chain(entries)
self.assertTrue(valid, errors)
self.assertEqual([entry["kind"] for entry in entries],
["pushback", "pushback", "pushback",
"rebuild", "reaudit"])
self.assertEqual(entries[3]["rebuild_no"], 1)
self.assertEqual(entries[3]["old_implementation_ref"], "abc1234")
class TriageTestCase(unittest.TestCase):
def _audit(self, failed_items):
return {"verdict": "fail", "verdict_text": "round 3 audit",
"failed_items": failed_items}
def test_structural_kind_triggers_early_rebuild(self):
triage = triage_third_failure(
self._audit([{"check": "ci_green",
"structural_kind": "architecture_boundary"}]),
[[{"check": "ci_green",
"structural_kind": "architecture_boundary"}]],
passed_items=[{"check": "x"}, {"check": "y"}],
broken_passed_items=[])
self.assertEqual(triage["verdict"], "early_rebuild")
self.assertIn("round 3 audit",
triage["basis"]["audit_verdict_text"])
def test_repeated_contract_clause_triggers_early_refreeze(self):
item = {"check": "contract_version", "root_cause": "contract",
"evidence": {"contract_clause": "G2 §3 timeout"}}
triage = triage_third_failure(
self._audit([item]), [[item], [item]],
passed_items=[{"check": "x"}],
broken_passed_items=[])
self.assertEqual(triage["verdict"], "early_refreeze")
decision = decide_pushback(passing_audit(verdict="fail"), 2,
triage_verdict="early_refreeze")
self.assertEqual(decision["action"], "escalate_refreeze")
self.assertEqual(decision["refreeze_flow"], "G2")
def test_near_complete_gets_one_targeted_round_only(self):
audit = self._audit([{"check": "ci_green"}])
triage = triage_third_failure(
audit, [[{"check": "ci_green"}]],
passed_items=[{"check": "a"}, {"check": "b"},
{"check": "c"}],
broken_passed_items=[])
self.assertEqual(triage["verdict"], "targeted_fix")
self.assertEqual(triage["targeted_scope"], ["ci_green"])
decision = decide_pushback(passing_audit(verdict="fail"), 2,
triage_verdict="targeted_fix",
triage_scope=["ci_green"])
self.assertEqual(decision["action"], "pushback")
self.assertEqual(decision["pushback_scope"], "targeted")
self.assertEqual(decision["targeted_items"], ["ci_green"])
# Targeted budget used: a later near-complete triage is
# an ordinary pushback, not a second targeted round.
triage2 = triage_third_failure(
audit, [[{"check": "ci_green"}]],
passed_items=[{"check": "a"}, {"check": "b"},
{"check": "c"}],
broken_passed_items=[], targeted_already_used=True)
self.assertEqual(triage2["verdict"], "normal_pushback")
def test_targeted_round_failure_counts_to_four_then_rebuilds(self):
state = new_module_failure_state("mod-web", "v1")
for _ in range(3):
state = record_module_failure(state, "audit", "v1")
state = record_module_targeted_fix(state)
# Targeted round fails: that failure is failure 4.
state = record_module_failure(state, "audit", "v1")
self.assertEqual(state["total_fail_count"], 4)
decision = decide_pushback(passing_audit(verdict="fail"), 3)
self.assertEqual(decision["action"], "rebuild")
def test_too_many_failures_or_broken_passed_is_not_near_complete(self):
many = self._audit([{"check": "a"}, {"check": "b"},
{"check": "c"}])
triage = triage_third_failure(
many, [[{"check": "a"}]],
passed_items=[{"check": "x"}] * 5,
broken_passed_items=[])
self.assertEqual(triage["verdict"], "normal_pushback")
broken = triage_third_failure(
self._audit([{"check": "ci_green"}]),
[[{"check": "ci_green"}]],
passed_items=[{"check": "a"}, {"check": "b"}],
broken_passed_items=[{"check": "a"}])
self.assertEqual(broken["verdict"], "normal_pushback")
unknown = triage_third_failure(
self._audit([{"check": "ci_green"}]),
[[{"check": "ci_green"}]],
passed_items=[{"check": "a"}, {"check": "b"}],
broken_passed_items=None)
self.assertEqual(unknown["verdict"], "normal_pushback")
def test_early_rebuild_consumes_rebuild_budget(self):
decision = decide_pushback(passing_audit(verdict="fail"), 2,
triage_verdict="early_rebuild")
self.assertEqual(decision["action"], "rebuild")
self.assertEqual(decision["rebuild_no"], 1)
state = new_module_failure_state("mod-web", "v1")
state = record_module_rebuild(state, 1)
# Rebuild resets failure counts for the new implementation.
self.assertEqual(state["total_fail_count"], 0)
self.assertEqual(state["rebuilds_used"], 1)
# ...but no second rebuild after fresh failures over 3.
decision2 = decide_pushback(passing_audit(verdict="fail"), 3,
rebuilds_used=1)
self.assertEqual(decision2["action"], "escalate")
class RetainedAssetsTestCase(unittest.TestCase):
def _package(self):
decision = decide_pushback(passing_audit(verdict="fail"), 3)
return build_rebuild_package(
"mod-web", decision, [], [],
old_implementation_ref="abc1234",
passed_items=[{"id": "acc-1", "check": "contract_version"}],
retained_tests=["test_web_smoke", "test_contract"])
def test_rebuild_package_retains_tests_and_passed_items(self):
package = self._package()
self.assertEqual(package["retained_assets"]["tests"],
["test_web_smoke", "test_contract"])
self.assertFalse(package["retained_assets"]["retained_tests_missing"])
self.assertTrue(package["retained_assets"]["contract_unchanged"])
self.assertTrue(package["retained_assets"]["design_unchanged"])
def test_package_without_retained_tests_is_flagged(self):
decision = decide_pushback(passing_audit(verdict="fail"), 3)
package = build_rebuild_package("mod-web", decision, [], [])
self.assertTrue(package["retained_assets"]["retained_tests_missing"])
def test_first_gate_requires_all_old_tests_and_acceptance(self):
gate = check_rebuild_first_gate(
["test_web_smoke", "test_contract"],
[{"id": "acc-1"}],
{"test_web_smoke": True, "test_contract": True},
{"acc-1": True})
self.assertTrue(gate["gate_pass"])
missing = check_rebuild_first_gate(
["test_web_smoke", "test_contract"], [],
{"test_web_smoke": True}, {})
self.assertFalse(missing["gate_pass"])
self.assertIn("test:test_contract", missing["missing"])
failed = check_rebuild_first_gate(
["test_web_smoke"], [{"id": "acc-1"}],
{"test_web_smoke": False}, {"acc-1": True})
self.assertFalse(failed["gate_pass"])
self.assertIn("test:test_web_smoke", failed["failed"])
# First-gate failure counts as an audit failure.
state = new_module_failure_state("mod-web", "v1")
state = record_module_failure(state, "audit", "v1")
self.assertEqual(state["total_fail_count"], 1)
def test_test_deletion_needs_refreeze_and_separate_audit(self):
delete = [{"action": "delete", "test_id": "test_old"}]
self.assertTrue(validate_test_changes(delete))
self.assertTrue(validate_test_changes(
delete, refreeze_approved=True))
# The change itself must name its separate audit; a bare
# set of audit ids without a reference on the change proves
# nothing about this change.
self.assertTrue(validate_test_changes(
delete, refreeze_approved=True,
separate_audit_ids={"audit-9"}))
self.assertEqual(validate_test_changes(
[{"action": "delete", "test_id": "test_old",
"separate_audit_id": "audit-9"}],
refreeze_approved=True,
separate_audit_ids={"audit-9"}), [])
change = [{"action": "modify_assertion", "test_id": "t",
"separate_audit_id": "audit-9"}]
self.assertTrue(validate_test_changes(
change, refreeze_approved=True))
self.assertEqual(validate_test_changes(
[{"action": "add", "test_id": "test_new"}]), [])
class AuditOfAuditTestCase(unittest.TestCase):
def test_golden_set_rejects_examples_and_requires_human(self):
valid, _reasons = validate_golden_task_set(
[{"id": "gold-1", "human_approved_by": "user"}])
self.assertTrue(valid)
valid2, reasons2 = validate_golden_task_set(
[{"id": "sample-1", "is_example": True,
"human_approved_by": "user"}])
self.assertFalse(valid2)
self.assertTrue(reasons2)
valid3, _r3 = validate_golden_task_set([{"id": "gold-2"}])
self.assertFalse(valid3)
def test_sampling_covers_golden_mandatory_and_random(self):
# Unified rule (gap 2): mandatory kinds carry their own
# trigger names (no "critical_path" label for kinds, no
# "periodic_sample"); ordinary records are drawn by seeded
# random sampling with ceil, so 9 ordinary records still
# contribute exactly ceil(0.10 * 9) == 1 -- the old
# deterministic every-10th rule drew zero from this batch.
records = [{"task_id": "gold-1", "kind": "pushback"}]
records += [{"task_id": "t-%d" % i, "kind": "pushback"}
for i in range(9)]
records.append({"task_id": "t-re", "kind": "rebuild"})
selected = select_audit_sample(records, {"gold-1"}, seed="s")
reasons = {entry["record"]["task_id"]: entry["reason"]
for entry in selected}
self.assertEqual(reasons["gold-1"], "golden_task")
self.assertEqual(reasons["t-re"], "rebuild")
ordinary = [t for t in reasons if t.startswith("t-")
and t != "t-re"]
self.assertEqual(len(ordinary), 1)
self.assertNotIn("periodic_sample", set(reasons.values()))
self.assertNotIn("critical_path", set(reasons.values()))
with self.assertRaises(ValueError):
select_audit_sample(records, {"gold-1"})
def test_correction_preserves_original_and_restores_flow(self):
original = {"task_id": "t-1", "round": 0, "verdict": "pass",
"signer": "auditor-a", "entry_hash": "h" * 64}
correction, errors = build_audit_correction(
original, "fail", "auditor-b", "sample re-run found ci red")
self.assertEqual(errors, [])
self.assertEqual(original["verdict"], "pass")
self.assertEqual(correction["original_entry_hash"], "h" * 64)
self.assertEqual(correction["restored_action"],
"retrigger_pushback_and_reblock")
wrong_block, _e = build_audit_correction(
dict(original, verdict="fail"), "pass", "auditor-b",
"sample shows the failure was a fixture artefact")
self.assertEqual(wrong_block["restored_action"], "schedule_reaudit")
_bad, bad_errors = build_audit_correction(
original, "fail", "auditor-a", "same signer")
self.assertTrue(bad_errors)
def test_g6_guard_blocks_non_user_acceptance(self):
self.assertTrue(check_g6_guard(
{"g6_acceptance": True, "g6_accepted_by": "coordinator"}))
self.assertEqual(check_g6_guard(
{"g6_acceptance": True, "g6_accepted_by": "user"}), [])
self.assertEqual(check_g6_guard({}), [])
class AcceptanceRebuildNoticeTestCase(unittest.TestCase):
def test_unconfirmed_acceptance_rebuild_does_not_start(self):
decision = decide_pushback(
passing_audit(verdict="fail"), 3,
acceptance_triggered=True, user_confirmed_rebuild=False)
self.assertEqual(decision["action"],
"rebuild_pending_user_confirmation")
self.assertTrue(decision["merge_blocked"])
self.assertIsNone(decision["rebuild_no"])
package = build_rebuild_package(
"mod-web", dict(decision, rebuild_no=1), [], [],
retained_tests=["test_web_smoke"])
notice = build_acceptance_rebuild_notice(
"mod-web", dict(decision, rebuild_no=1), package,
["acceptance failed 4 times"])
self.assertFalse(notice["confirmed"])
self.assertTrue(notice["requires_user_confirmation"])
self.assertIn("test_web_smoke",
notice["retained_assets"]["tests"])
def test_confirmed_acceptance_rebuild_proceeds(self):
decision = decide_pushback(
passing_audit(verdict="fail"), 3,
acceptance_triggered=True, user_confirmed_rebuild=True)
self.assertEqual(decision["action"], "rebuild")
notice = build_acceptance_rebuild_notice(
"mod-web", decision, {}, ["acceptance failed"])
confirmed = confirm_acceptance_rebuild(notice, True)
self.assertTrue(confirmed["confirmed"])
self.assertTrue(confirmed["g6_acceptance_retained_by_user"])
class SkillTextTestCase(unittest.TestCase):
"""The skill mechanism must actually be in the skill text."""
def test_audit_section_present_in_skill(self):
skill = Path(__file__).resolve().parents[1] / "skills" \
/ "software-engineering" / "SKILL.md"
text = skill.read_text(encoding="utf-8")
for phrase in (
"独立审计",
"自动推回",
"CONDITIONAL",
"三轮",
"完整重审",
"审计记录",
"推倒重做",
"重新冻结",
"根因分流",
"旧测试",
"审计的审计",
):
with self.subTest(phrase=phrase):
self.assertIn(phrase, text)
if __name__ == "__main__":
unittest.main()
+173
View File
@@ -0,0 +1,173 @@
"""Tests for cloud.audit_sampling: audit-of-audit sampling rules.
Every rule has a positive and a negative case. Only the standard
library; no real model, no network, no secrets.
"""
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(
os.path.dirname(os.path.abspath(__file__)))))
from cloud.audit_sampling import ( # noqa: E402
append_audit_record, filter_golden_tasks,
golden_approval_checklist, handle_sampling_result,
mandatory_trigger, select_sample)
def _case(case_id, **kw):
case = {"id": case_id}
case.update(kw)
return case
class TestMandatoryTriggers(unittest.TestCase):
def test_rebuild_is_always_mandatory(self):
self.assertEqual(
mandatory_trigger(_case("r", decision="rebuild")), "rebuild")
def test_third_failure_near_complete_still_failed_is_mandatory(self):
case = _case("n", decision="fail_pushback", failure_count=3,
root_cause="near_complete", continuation_failed=True)
self.assertEqual(mandatory_trigger(case),
"third_failure_near_complete_still_failed")
def test_near_complete_continuation_success_is_not_mandatory(self):
case = _case("n", decision="pass", failure_count=3,
root_cause="near_complete", continuation_failed=False)
self.assertIsNone(mandatory_trigger(case))
def test_second_failure_near_complete_is_not_mandatory(self):
case = _case("n", decision="fail_pushback", failure_count=2,
root_cause="near_complete", continuation_failed=True)
self.assertIsNone(mandatory_trigger(case))
def test_structural_third_failure_is_not_this_trigger(self):
case = _case("n", decision="rebuild", failure_count=3,
root_cause="structural", continuation_failed=True)
# It IS mandatory, but via the rebuild trigger, not the
# near-complete trigger -- the trigger name must be exact.
self.assertEqual(mandatory_trigger(case), "rebuild")
def test_ordinary_case_is_not_mandatory(self):
self.assertIsNone(mandatory_trigger(
_case("o", decision="pass", failure_count=1)))
class TestRandomSample(unittest.TestCase):
def _cases(self):
normal = [_case("n%d" % i, decision="pass") for i in range(20)]
critical = [_case("c%d" % i, decision="pass", critical_path=True)
for i in range(10)]
return normal + critical
def test_ratios_and_reproducibility(self):
plan = select_sample(self._cases(), seed="seed-1")
# ceil(0.10 * 20) normal, ceil(0.50 * 10) critical.
self.assertEqual(len(plan["random_normal"]), 2)
self.assertEqual(len(plan["random_critical_path"]), 5)
self.assertEqual(plan["seed"], "seed-1")
again = select_sample(self._cases(), seed="seed-1")
self.assertEqual(plan["sampled_ids"], again["sampled_ids"])
def test_mandatory_cases_are_included_not_double_counted(self):
cases = [_case("r", decision="rebuild"),
_case("n0", decision="pass")]
plan = select_sample(cases, seed="s")
self.assertIn("r", plan["sampled_ids"])
self.assertEqual(plan["mandatory"][0]["trigger"], "rebuild")
self.assertEqual(len(plan["sampled_ids"]),
len(set(plan["sampled_ids"])))
def test_small_group_is_never_rounded_to_zero(self):
plan = select_sample([_case("only", decision="pass")], seed="s")
self.assertEqual(plan["random_normal"], ["only"])
def test_missing_seed_is_rejected(self):
with self.assertRaises(ValueError):
select_sample(self._cases(), seed="")
class TestGoldenApproval(unittest.TestCase):
def test_sample_entries_never_count_as_golden(self):
tasks = [{"id": "sample-1"}, {"id": "real-1", "sample": True},
{"id": "real-2"}]
filtered = filter_golden_tasks(tasks)
self.assertEqual([t["id"] for t in filtered["golden"]],
["real-2"])
self.assertEqual(filtered["dropped_samples"], 2)
def test_unapproved_checklist_is_pending(self):
checklist = golden_approval_checklist([{"id": "real-2"}])
self.assertFalse(checklist["approved"])
self.assertTrue(any(i["status"] == "pending_human"
for i in checklist["items"]))
def test_fully_confirmed_checklist_is_approved(self):
checklist = golden_approval_checklist(
[{"id": "real-2"}], approver="human",
approved_at="2026-10-31", list_reviewed=True,
criteria_defined=True)
self.assertTrue(checklist["approved"])
def test_sample_only_set_cannot_be_approved(self):
checklist = golden_approval_checklist(
[{"id": "sample-1"}], approver="human",
approved_at="2026-10-31", list_reviewed=True,
criteria_defined=True)
self.assertFalse(checklist["approved"])
self.assertEqual(checklist["golden_tasks"], [])
class TestSamplingResultHandling(unittest.TestCase):
def test_correct_verdict_only_records(self):
entry = handle_sampling_result(
_case("a", decision="pass"), "correct")
self.assertIsNone(entry["corrected_decision"])
self.assertIsNone(entry["retrigger"])
log = append_audit_record([], entry)
self.assertEqual(len(log), 1)
self.assertEqual(log[0]["recorded_in"], "audit_of_audit")
def test_wrong_pushback_is_cancelled_and_rerun(self):
entry = handle_sampling_result(
_case("a", decision="fail_pushback"), "incorrect",
corrected_verdict="pass")
self.assertEqual(entry["retrigger"],
"cancel_pushback_and_rerun_audit")
self.assertEqual(entry["corrected_decision"], "pass")
def test_wrong_rebuild_is_halted_and_restored(self):
entry = handle_sampling_result(
_case("a", decision="rebuild"), "incorrect",
corrected_verdict="fail_pushback")
self.assertEqual(entry["retrigger"], "halt_rebuild_and_restore")
def test_wrong_pass_reopens_and_pushes_back(self):
entry = handle_sampling_result(
_case("a", decision="pass"), "incorrect",
corrected_verdict="fail_pushback")
self.assertEqual(entry["retrigger"],
"reopen_audit_and_pushback")
def test_unknown_original_decision_escalates(self):
entry = handle_sampling_result(
_case("a", decision="mystery"), "incorrect",
corrected_verdict="pass")
self.assertEqual(entry["retrigger"], "escalate_to_human")
def test_incorrect_without_correction_is_rejected(self):
with self.assertRaises(ValueError):
handle_sampling_result(_case("a", decision="pass"),
"incorrect")
def test_invalid_verdict_is_rejected(self):
with self.assertRaises(ValueError):
handle_sampling_result(_case("a", decision="pass"),
"maybe")
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,253 @@
"""Tests for the UNIFIED audit-of-audit sampling rule (gap 2).
This file pins the alignment of track 3 (#244) and track 5 (#238)
only; it does not re-test either track's other semantics.
Unified rule under test (see ``cloud/audit_sampling.py``):
- Mandatory (1.0) = golden-task records + rebuild / refreeze /
escalation records + third-failure near-complete still-failed
cases. Scope from #244, expanded with #238's third-failure
trigger; #244's deterministic every-10th ordinary rule is gone.
- The rest by seeded random draw: normal 0.10, critical-path
0.50, ceil so a non-empty group is never rounded to zero.
The 0.10 / 0.50 values are TEST DEFAULTS, not approved
production rates.
- One pass, no double counting: mandatory records leave the
random pools before drawing; a multi-trigger record is sampled
once with all its triggers listed; correction follows the
(original, corrected) verdict pair, not the trigger list.
- Correction preserves the original: the original record is never
mutated, the correction is a separate appended record, and the
corrector must differ from the original signer.
"""
import copy
import os
import sys
import unittest
sys.path.insert(0, os.path.dirname(os.path.dirname(
os.path.dirname(os.path.abspath(__file__)))))
from cloud.audit_pushback import ( # noqa: E402
build_audit_correction, select_audit_sample)
from cloud.audit_sampling import ( # noqa: E402
append_audit_record, build_golden_task_ids, handle_sampling_result,
mandatory_trigger, mandatory_triggers, select_sample)
def _case(case_id, **kw):
case = {"id": case_id}
case.update(kw)
return case
class TestMandatoryUnion(unittest.TestCase):
def test_all_mandatory_triggers_sampled(self):
cases = [
_case("g-push", task_id="gold-1", kind="pushback"),
_case("r", decision="rebuild"),
_case("rf", kind="refreeze"),
_case("esc", kind="escalate"),
_case("third", decision="fail_pushback", failure_count=3,
root_cause="near_complete", continuation_failed=True),
_case("n0", decision="pass"),
]
plan = select_sample(cases, seed="s", golden_task_ids={"gold-1"})
by_id = {e["case_id"]: e["triggers"] for e in plan["mandatory"]}
self.assertEqual(by_id["g-push"], ["golden_task"])
self.assertEqual(by_id["r"], ["rebuild"])
self.assertEqual(by_id["rf"], ["refreeze"])
self.assertEqual(by_id["esc"], ["escalate"])
self.assertEqual(by_id["third"],
["third_failure_near_complete_still_failed"])
self.assertIn("n0", plan["random_normal"])
self.assertEqual(len(plan["sampled_ids"]),
len(set(plan["sampled_ids"])))
def test_blocked_quota_is_not_mandatory(self):
# Track 3's mandatory kind list is rebuild/refreeze/escalate
# only; a quota-blocked record reaches the random tiers.
case = _case("q", kind="blocked_quota")
self.assertEqual(mandatory_triggers(case), [])
plan = select_sample([case], seed="s")
self.assertEqual(plan["mandatory"], [])
self.assertEqual(plan["random_normal"], ["q"])
def test_golden_task_ordinary_record_is_mandatory(self):
# Golden membership is per task, not per record kind: even
# a plain pass record of a golden task is mandatory.
case = _case("g-pass", task_id="gold-1", kind="pushback",
decision="pass")
plan = select_sample([case], seed="s",
golden_task_ids={"gold-1"})
self.assertEqual(plan["mandatory"][0]["triggers"],
["golden_task"])
class TestNoDoubleCounting(unittest.TestCase):
def test_golden_rebuild_sampled_once_with_all_triggers(self):
cases = [_case("g-re", task_id="gold-1", kind="rebuild")]
cases += [_case("n%d" % i, decision="pass") for i in range(20)]
plan = select_sample(cases, seed="s", golden_task_ids={"gold-1"})
self.assertEqual(plan["sampled_ids"].count("g-re"), 1)
self.assertEqual(plan["mandatory"][0]["triggers"],
["golden_task", "rebuild"])
# The mandatory record consumed no random quota: the draw
# is still ceil(0.10 * 20) == 2 over the ordinary pool.
self.assertEqual(len(plan["random_normal"]), 2)
self.assertNotIn("g-re", plan["random_normal"])
# Legacy single-trigger view reports the first trigger.
self.assertEqual(
mandatory_trigger(cases[0], {"gold-1"}), "golden_task")
def test_triage_and_later_rebuild_are_two_records_not_double(self):
# Deduplication is per record: a triage record and the
# later rebuild record for the same task are two samples.
cases = [
_case("triage", decision="fail_pushback", failure_count=3,
root_cause="near_complete", continuation_failed=True),
_case("rebuild", decision="rebuild"),
]
plan = select_sample(cases, seed="s")
self.assertEqual(sorted(plan["sampled_ids"]),
["rebuild", "triage"])
class TestRandomTiersAndCeil(unittest.TestCase):
def test_ratios_and_seed_reproducibility(self):
cases = [_case("n%d" % i, decision="pass") for i in range(20)]
cases += [_case("c%d" % i, decision="pass", critical_path=True)
for i in range(10)]
plan = select_sample(cases, seed="seed-1")
self.assertEqual(len(plan["random_normal"]), 2)
self.assertEqual(len(plan["random_critical_path"]), 5)
again = select_sample(cases, seed="seed-1")
self.assertEqual(plan["sampled_ids"], again["sampled_ids"])
def test_small_groups_never_round_to_zero(self):
self.assertEqual(len(select_sample(
[_case("only", decision="pass")], seed="s")["random_normal"]),
1)
# 9 ordinary records: the old every-10th rule drew ZERO
# here; ceil(0.10 * 9) == 1 under the unified rule.
self.assertEqual(len(select_sample(
[_case("n%d" % i, decision="pass") for i in range(9)],
seed="s")["random_normal"]), 1)
self.assertEqual(len(select_sample(
[_case("n%d" % i, decision="pass") for i in range(19)],
seed="s")["random_normal"]), 2)
self.assertEqual(len(select_sample(
[_case("c0", decision="pass", critical_path=True)],
seed="s")["random_critical_path"]), 1)
self.assertEqual(len(select_sample(
[_case("c%d" % i, decision="pass", critical_path=True)
for i in range(3)], seed="s")["random_critical_path"]), 2)
def test_missing_seed_rejected(self):
with self.assertRaises(ValueError):
select_sample([_case("n", decision="pass")], seed="")
class TestGoldenSetConstruction(unittest.TestCase):
def test_build_golden_ids_applies_both_vocabularies(self):
tasks = [
{"id": "gold-1", "human_approved_by": "user"},
{"id": "sample-2", "human_approved_by": "user"},
{"id": "gold-2", "sample": True, "human_approved_by": "user"},
{"id": "gold-3", "is_example": True,
"human_approved_by": "user"},
{"id": "gold-4"},
]
self.assertEqual(build_golden_task_ids(tasks), ["gold-1"])
class TestAdapter(unittest.TestCase):
def test_adapter_returns_unified_selection_in_record_order(self):
records = [
{"task_id": "gold-1", "kind": "pushback"},
{"task_id": "gold-1", "kind": "fix"},
{"task_id": "gold-1", "kind": "rebuild"},
{"task_id": "t-rf", "kind": "refreeze"},
{"task_id": "t-esc", "kind": "escalate"},
]
records += [{"task_id": "t-%d" % i, "kind": "pushback"}
for i in range(9)]
before = copy.deepcopy(records)
golden = set(build_golden_task_ids(
[{"id": "gold-1", "human_approved_by": "user"}]))
selected = select_audit_sample(records, golden, seed="s")
by_record = {id(e["record"]): e for e in selected}
# Every golden record is mandatory, whatever its kind; the
# golden rebuild record carries both triggers, once.
self.assertEqual(by_record[id(records[0])]["reasons"],
["golden_task"])
self.assertEqual(by_record[id(records[1])]["reasons"],
["golden_task"])
self.assertEqual(by_record[id(records[2])]["reasons"],
["golden_task", "rebuild"])
self.assertEqual(by_record[id(records[3])]["reason"], "refreeze")
self.assertEqual(by_record[id(records[4])]["reason"], "escalate")
# 9 ordinary records contribute ceil(0.10 * 9) == 1.
ordinary = [e for e in selected
if e["record"]["task_id"].startswith("t-")
and e["record"]["task_id"] not in ("t-rf", "t-esc")]
self.assertEqual(len(ordinary), 1)
self.assertEqual(ordinary[0]["reason"], "random_normal")
# Selected entries keep original record order.
order = [records.index(e["record"]) for e in selected]
self.assertEqual(order, sorted(order))
# Input records are never mutated by selection.
self.assertEqual(records, before)
with self.assertRaises(ValueError):
select_audit_sample(records, golden)
class TestCorrectionPreservesOriginal(unittest.TestCase):
def test_track3_correction_appends_without_mutating(self):
original = {"task_id": "t-1", "round": 0, "verdict": "pass",
"signer": "auditor-a", "entry_hash": "h" * 64}
before = dict(original)
correction, errors = build_audit_correction(
original, "fail", "auditor-b", "sample re-run found ci red")
self.assertEqual(errors, [])
self.assertEqual(original, before)
self.assertEqual(correction["original_entry_hash"], "h" * 64)
self.assertEqual(correction["restored_action"],
"retrigger_pushback_and_reblock")
log = [original, correction]
self.assertEqual(log[0]["verdict"], "pass")
_bad, bad_errors = build_audit_correction(
original, "fail", "auditor-a", "same signer")
self.assertTrue(bad_errors)
def test_track5_correction_appends_without_mutating(self):
case = {"id": "a", "decision": "pass"}
before = dict(case)
entry = handle_sampling_result(
case, "incorrect", corrected_verdict="fail_pushback",
corrector="auditor-b", original_signer="auditor-a")
# Track 5 vocabulary for the same wrong-pass pair as track
# 3's retrigger_pushback_and_reblock (synonyms, see the
# unified rule in cloud/audit_sampling.py).
self.assertEqual(entry["retrigger"], "reopen_audit_and_pushback")
self.assertEqual(case, before)
log = []
log2 = append_audit_record(log, entry)
self.assertEqual(log, [])
self.assertEqual(len(log2), 1)
with self.assertRaises(ValueError):
handle_sampling_result(
case, "incorrect", corrected_verdict="fail_pushback",
corrector="auditor-a", original_signer="auditor-a")
def test_correct_verdict_only_records(self):
entry = handle_sampling_result(
_case("a", decision="pass"), "correct")
self.assertIsNone(entry["corrected_decision"])
self.assertIsNone(entry["retrigger"])
if __name__ == "__main__":
unittest.main()
+239
View File
@@ -0,0 +1,239 @@
"""7 维审计评分卡定向测试。
钉住的规则(任何一处改动都必须先改文档再改测试):
- 7 维全部 >=9 才达标;任一维 <9 总体不达标。
- 平均分绝不能作为达标依据:6 维 10 分、1 维 4 分,平均 9.1 分,
总体仍是不达标。
- 未知维记 None 且总体不达标;未知不记 0、不记 10、不估算。
- 夹具/视口替代/代码评审证据记未实测,不算通过。
- 阈值默认值标注 test_default_unapproved;只有调用方显式传入
user_approved 才算已拍板。
- 端点只读:GET /audit-scorecard 不写库、不写账本、不改链头。
"""
import http.client
import json
import os
import sys
import tempfile
import threading
import unittest
from pathlib import Path
try:
from cloud.audit_scorecard import (
DEFAULT_THRESHOLDS, build_scorecard, example_scorecard_as_of_2026_10_31)
from cloud.auth import AuthStore
from cloud.server import create_server
except ImportError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from cloud.audit_scorecard import (
DEFAULT_THRESHOLDS, build_scorecard, example_scorecard_as_of_2026_10_31)
from cloud.auth import AuthStore
from cloud.server import create_server
def _full_evidence():
return {
"delivery_quality": {
"first_pass_rate": 0.96, "defect_rate": 0.01,
"rework_rate": 0.04, "first_pass_known": True,
"defect_known": True, "rework_known": True,
"attempts_present": True, "evidence_kind": "real",
},
"gate_audit": {
"hard_gate_enforced": True, "gate_items_passed": 5,
"gate_items_total": 5, "merge_blocked_verified": True,
"loop_end_to_end_verified": True,
"sampling_misjudgment_rate": 0.0,
},
"arch_contract": {
"contract_frozen": True, "arch_violations": 0,
"directory_violations": 0, "import_violations": 0,
"stack_changed_without_approval": False,
},
"test_coverage": {
"prd_coverage_rate": 1.0, "cross_matrix_coverage_rate": 1.0,
"coverage_pct": 0.92, "cross_matrix_kind": "real",
"real_pg": True, "real_service": True,
"agent_six_complete": True,
"lint_items": 0, "mypy_items": 0, "bandit_high": 0,
},
"security_ledger": {
"ledger_verify_ok": True, "ledger_coverage_rate": 1.0,
"branch_protection": "protected",
"shell_whitelist_pass_rate": 1.0, "anchor_check_recent": True,
},
"cost_quota": {
"cost_records_complete": True, "unpriced_as_zero": False,
"per_creator_present": True, "quota_cutoff_verified": True,
},
"ops_scale": {
"ci_green_rate": 1.0, "deploy_success_rate": 1.0,
"rollback_drill_done": True, "ha_status": "second_host",
"parallel_scale_proven": 40,
},
}
class ScorecardTestCase(unittest.TestCase):
def test_all_nine_passes(self):
report = build_scorecard(_full_evidence())
self.assertEqual(report["overall"], "pass")
self.assertTrue(report["overall_pass"])
self.assertEqual(len(report["dimensions"]), 7)
for dim in report["dimensions"]:
self.assertIsNotNone(dim["score"])
self.assertGreaterEqual(dim["score"], 9.0)
self.assertEqual(dim["status"], "pass")
def test_one_dimension_below_nine_fails_overall(self):
ev = _full_evidence()
ev["gate_audit"]["hard_gate_enforced"] = False
report = build_scorecard(ev)
self.assertEqual(report["overall"], "not_pass")
self.assertFalse(report["overall_pass"])
gate = [d for d in report["dimensions"] if d["id"] == "gate_audit"][0]
self.assertEqual(gate["score"], 4.0)
def test_average_91_with_one_four_still_fails(self):
# 6 维 10 分、1 维 4 分:平均 9.1 分,总体仍是不达标。
# 构造方式:只让门禁维已知未强制(4.0),其余维满分证据。
ev = _full_evidence()
ev["gate_audit"]["hard_gate_enforced"] = False
report = build_scorecard(ev)
self.assertEqual(report["average_score"], 9.1)
self.assertTrue(report["average_is_not_pass_criterion"])
self.assertEqual(report["overall"], "not_pass")
self.assertFalse(report["overall_pass"])
def test_unknown_dimension_is_none_and_fails_overall(self):
report = build_scorecard({})
self.assertEqual(report["overall"], "not_pass")
for dim in report["dimensions"]:
self.assertIsNone(dim["score"])
self.assertEqual(dim["status"], "unknown")
self.assertTrue(report["missing_items"])
self.assertIsNone(report["average_score"])
def test_unknown_is_not_zero_and_not_ten(self):
ev = _full_evidence()
ev["delivery_quality"] = {}
report = build_scorecard(ev)
dim = [d for d in report["dimensions"]
if d["id"] == "delivery_quality"][0]
self.assertIsNone(dim["score"])
self.assertNotEqual(dim["score"], 0)
self.assertNotEqual(dim["score"], 10)
def test_fixture_evidence_is_untested_not_pass(self):
ev = _full_evidence()
ev["test_coverage"]["cross_matrix_kind"] = "viewport_substitute"
report = build_scorecard(ev)
dim = [d for d in report["dimensions"]
if d["id"] == "test_coverage"][0]
self.assertIsNone(dim["score"])
self.assertEqual(dim["status"], "unknown")
self.assertTrue(dim["untested_items"])
self.assertEqual(report["overall"], "not_pass")
def test_conditional_verdict_is_not_pass(self):
ev = _full_evidence()
ev["gate_audit"]["audit_verdict"] = "conditional"
report = build_scorecard(ev)
dim = [d for d in report["dimensions"] if d["id"] == "gate_audit"][0]
self.assertIsNone(dim["score"])
self.assertTrue(dim["untested_items"])
def test_threshold_defaults_are_unapproved(self):
for entry in DEFAULT_THRESHOLDS.values():
self.assertEqual(entry["source"], "test_default_unapproved")
report = build_scorecard(_full_evidence())
self.assertEqual(report["thresholds_source"],
"test_default_unapproved")
def test_user_approved_thresholds_change_source_only(self):
th = {k: {"value": v["value"], "source": "user_approved"}
for k, v in DEFAULT_THRESHOLDS.items()}
report = build_scorecard(_full_evidence(), thresholds=th)
self.assertEqual(report["thresholds_source"], "user_approved")
self.assertEqual(report["overall"], "pass")
def test_example_current_state_is_not_pass_and_not_estimated(self):
# 示例是系统用当前真实状态证据算出来的,不是上一轮判断分。
report = example_scorecard_as_of_2026_10_31()
self.assertEqual(report["overall"], "not_pass")
self.assertFalse(report["overall_pass"])
# 时效标注:示例卡是 2026-10-31 快照,不是当前系统得分。
self.assertEqual(report["evidence_as_of"], "2026-10-31")
self.assertEqual(report["evidence_kind"],
"historical_example_snapshot")
self.assertTrue(report["not_current_system_score"])
self.assertEqual(report["thresholds_source"],
"test_default_unapproved")
# 至少有一维是未知(None),且总体不达标不靠平均分。
self.assertTrue(any(d["score"] is None
for d in report["dimensions"]))
class ScorecardEndpointTestCase(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
self.auth = AuthStore(self.db_path)
self.owner_id, self.owner_key = self.auth.create_user("owner")
self.server = create_server(self.db_path, "127.0.0.1", 0)
self.port = self.server.server_address[1]
self.thread = threading.Thread(
target=self.server.serve_forever, daemon=True)
self.thread.start()
def tearDown(self):
self.server.shutdown()
self.server.server_close()
self.thread.join(timeout=5)
try:
self.auth.conn.close()
except Exception:
pass
self.tmp.cleanup()
def _req(self, method, path, key=None):
conn = http.client.HTTPConnection("127.0.0.1", self.port, timeout=10)
hdrs = {}
if key is not None:
hdrs["X-API-Key"] = key
try:
conn.request(method, path, headers=hdrs)
resp = conn.getresponse()
data = resp.read()
finally:
conn.close()
try:
parsed = json.loads(data.decode("utf-8")) if data else None
except ValueError:
parsed = None
return resp.status, parsed
def test_endpoint_requires_auth(self):
status, _ = self._req("GET", "/audit-scorecard")
self.assertEqual(status, 401)
def test_endpoint_is_readonly_and_not_pass_on_main_base(self):
status, report = self._req("GET", "/audit-scorecard",
key=self.owner_key)
self.assertEqual(status, 200)
self.assertEqual(len(report["dimensions"]), 7)
self.assertEqual(report["overall"], "not_pass")
self.assertTrue(report["average_is_not_pass_criterion"])
self.assertEqual(report["thresholds_source"],
"test_default_unapproved")
# 只读:端点调用前后,库里不产生评分卡表(本端点不写库)。
import sqlite3
conn = sqlite3.connect(self.db_path)
try:
tables = {r[0] for r in conn.execute(
"SELECT name FROM sqlite_master WHERE type='table'")}
finally:
conn.close()
self.assertNotIn("audit_scorecard", tables)
+630
View File
@@ -0,0 +1,630 @@
"""Audit-gap-1 wiring: track 3 (pushback) <-> track 1 (hard gate)
<-> track 4 (hash-chain ledger), through cloud/audit_wiring.py.
What is pinned here (and nowhere else):
- CONDITIONAL maps to fail at the wiring boundary, original kept.
- Hard-gate 409 blocks codes become pushback-package failed
checks (branch protection travels as its own extra, never
dropped, never disguised as one of the ten gate checks).
- Every pushback / fix / re-audit / rebuild / escalation step
lands on the hash chain, and the chain still verifies.
- The server ANDs the chain's pushback block with the hard
gate's five checks: audit fail -> merge 409, status
unchanged, chain intact -> fix -> re-audit pass -> merge 200.
This chain runs against the real HTTP endpoint, not just
pure functions.
- The QA full chain is covered separately: a package carrying
QA failed links cannot be re-audited to a pass until exactly
those links have been re-run passing.
- The resolved meta-review tension: a correction to fail
re-blocks at once, but a correction to pass does NOT unblock
by itself (track 3: schedule a re-audit, not release the
gate); only a fresh passing verdict / re-audit unblocks.
Track 4's failure counts still use effective verdicts, so
the two views may disagree while a re-audit is pending --
that disagreement is asserted, not papered over.
"""
import http.client
import json
import os
import sqlite3
import sys
import tempfile
import threading
import unittest
from pathlib import Path
try:
from cloud.audit_pushback import GATE_CHECKS
from cloud.audit_pushback import decide_pushback
from cloud.audit_wiring import gate_failed_checks
from cloud.audit_wiring import normalize_re_review_result
from cloud.audit_wiring import normalize_verdict
from cloud.audit_wiring import pushback_block_for_task
from cloud.audit_wiring import pushback_package_from_gate
from cloud.audit_wiring import record_audit_verdict
from cloud.audit_wiring import record_escalation
from cloud.audit_wiring import record_fix
from cloud.audit_wiring import record_meta_review
from cloud.audit_wiring import record_pushback
from cloud.audit_wiring import record_reaudit
from cloud.audit_wiring import record_rebuild
from cloud.audit_wiring import record_third_failure_assessment
from cloud.auth import AuthStore
from cloud.ledger import LedgerStore
from cloud.ledger import audit_timeline
from cloud.merge_gate_hard import evaluate_hard_merge_gate
from cloud.server import create_server
except ImportError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from cloud.audit_pushback import GATE_CHECKS
from cloud.audit_pushback import decide_pushback
from cloud.audit_wiring import gate_failed_checks
from cloud.audit_wiring import normalize_re_review_result
from cloud.audit_wiring import normalize_verdict
from cloud.audit_wiring import pushback_block_for_task
from cloud.audit_wiring import pushback_package_from_gate
from cloud.audit_wiring import record_audit_verdict
from cloud.audit_wiring import record_escalation
from cloud.audit_wiring import record_fix
from cloud.audit_wiring import record_meta_review
from cloud.audit_wiring import record_pushback
from cloud.audit_wiring import record_reaudit
from cloud.audit_wiring import record_rebuild
from cloud.audit_wiring import record_third_failure_assessment
from cloud.auth import AuthStore
from cloud.ledger import LedgerStore
from cloud.ledger import audit_timeline
from cloud.merge_gate_hard import evaluate_hard_merge_gate
from cloud.server import create_server
MODULE = "apps/web"
CONTRACT = "v2"
def _failing_gate():
return evaluate_hard_merge_gate(
files_changed=["apps/web/a.py"], owned_dir="apps/web",
protection="protected", ci_status="pending",
ci_checks=[{"name": "ci", "status": "pending"}],
required_checks=["ci"], contract_version="v2",
contract_frozen=True, coordinator_approved=True,
unresolved_review_comments=0)
def _full_gate():
return evaluate_hard_merge_gate(
files_changed=["apps/web/a.py"], owned_dir="apps/web",
protection="protected", ci_status="success",
ci_checks=[{"name": "ci", "status": "success"}],
required_checks=["ci"], contract_version="v2",
contract_frozen=True, coordinator_approved=True,
unresolved_review_comments=0)
def _failing_audit():
checks = {name: True for name in GATE_CHECKS}
checks["ci_green"] = False
return {
"task_id": "task-1",
"verdict": "fail",
"verdict_text": "audit fail: ci not green",
"signer": "auditor-a",
"executor": "executor-e",
"coordinator_approver": "coordinator-c",
"qa_tester": "qa-q",
"security_verdict": "pass",
"checks": checks,
"failed_items": [],
"deviations": [],
"regression": {"items": [
{"path": "apps/web/app.py", "sha256_manifest": "a" * 64,
"sha256_recomputed": "a" * 64, "source": "exact"},
]},
"demo": {"verified_runnable": True, "runs_app_entry": True,
"command": "python3 apps/web/app.py"},
}
class VerdictNormalizationTestCase(unittest.TestCase):
def test_conditional_maps_to_fail_and_keeps_original(self):
normalized, original = normalize_verdict("conditional")
self.assertEqual(normalized, "fail")
self.assertEqual(original, "conditional")
normalized, original = normalize_verdict("pass")
self.assertEqual((normalized, original), ("pass", "pass"))
normalized, original = normalize_verdict(None)
self.assertEqual(normalized, "fail")
self.assertIsNone(original)
def test_re_review_conditional_maps_to_fail(self):
result = normalize_re_review_result({"verdict": "conditional"})
self.assertEqual(result["verdict"], "fail")
self.assertEqual(result["original_verdict"], "conditional")
result = normalize_re_review_result({"verdict": "pass"})
self.assertEqual(result["verdict"], "pass")
self.assertNotIn("original_verdict", result)
class GatePackageTestCase(unittest.TestCase):
def test_all_block_families_map_and_branch_protection_survives(self):
gate = evaluate_hard_merge_gate(
files_changed=["apps/other/x.py"], owned_dir="apps/web",
protection="unknown", ci_status="pending",
ci_checks=[{"name": "ci", "status": "failure"}],
required_checks=["ci"], contract_version=None,
contract_frozen=False, coordinator_approved=None,
unresolved_review_comments=["c1"])
self.assertFalse(gate["allowed"])
checks, extras = gate_failed_checks(gate)
for name in ("ci_green", "directory_isolation",
"contract_version", "coordinator_approval",
"unresolved_review_comments"):
self.assertIn(name, checks)
self.assertIn("branch_protection", extras)
package = pushback_package_from_gate(gate, "task-1", 0)
self.assertIn("branch_protection", package["failed_checks"])
self.assertIn("ci_green", package["failed_checks"])
# Full re-audit criteria, not failed-items-only.
self.assertEqual(package["reaudit_criteria"],
list(GATE_CHECKS))
# No block code is lost in translation.
self.assertTrue(package["failed_items"])
def test_qa_links_carried_onto_package(self):
links = [{"link": "checkout-flow", "case": "pay",
"failure": "500", "evidence": "log",
"reproduction": ["run checkout"]}]
package = pushback_package_from_gate(
_failing_gate(), "task-1", 0, qa_failed_links=links)
self.assertEqual(package["qa_failed_links"], links)
self.assertEqual(package["reaudit_must_rerun_links"], links)
class LedgerRecordingTestCase(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
self.store = LedgerStore(self.db_path)
self.task = "task-1"
self.project = "proj-1"
def tearDown(self):
try:
self.store.close()
except Exception:
pass
self.tmp.cleanup()
def _block(self):
conn = sqlite3.connect(self.db_path)
try:
return pushback_block_for_task(conn, self.task)
finally:
conn.close()
def _chain_ok(self):
result = self.store.verify()
self.assertTrue(result["ok"], result)
return result
def test_conditional_verdict_stored_as_fail_with_original(self):
entry = record_audit_verdict(
self.store, self.task, self.project, MODULE, CONTRACT,
"conditional", "audit", failures=["ci_green"])
timeline = audit_timeline(self.store.conn, task_id=self.task)
verdict = [e for e in timeline["entries"]
if e["action"] == "audit_verdict"][0]
self.assertEqual(verdict["parsed"]["verdict"], "fail")
self.assertEqual(verdict["parsed"]["extra"]["original_verdict"],
"conditional")
self.assertEqual(entry["id"], verdict["id"])
self._chain_ok()
def test_pushback_fix_reaudit_cycle_moves_gate(self):
record_audit_verdict(self.store, self.task, self.project,
MODULE, CONTRACT, "fail", "audit",
failures=["ci_green"])
package = pushback_package_from_gate(_failing_gate(),
self.task, 0)
record_pushback(self.store, self.task, self.project, MODULE,
CONTRACT, 1, package["failed_checks"], package,
failure_source="audit")
block = self._block()
self.assertTrue(block["merge_blocked"])
self.assertTrue(block["release_blocked"])
self.assertEqual(block["blocked_by"], "audit_pushback")
self.assertEqual(block["round"], 1)
self.assertIn("ci_green", block["failed_checks"])
# A landed fix alone does not unblock.
record_fix(self.store, self.task, self.project, MODULE,
CONTRACT, 1,
{"summary": "fix ci", "diff_sha256": "b" * 64,
"files_changed": ["apps/web/a.py"]}, package)
self.assertTrue(self._block()["merge_blocked"])
# A passing re-audit unblocks; chain still verifies.
record_reaudit(self.store, self.task, self.project, MODULE,
CONTRACT, 1, {"verdict": "pass"},
package=package)
self.assertFalse(self._block()["merge_blocked"])
self._chain_ok()
def test_qa_guard_refuses_unrerun_or_failed_link_pass(self):
links = [{"link": "checkout-flow", "case": "pay",
"failure": "500", "evidence": "log",
"reproduction": ["run checkout"]}]
package = pushback_package_from_gate(
_failing_gate(), self.task, 0, qa_failed_links=links)
record_audit_verdict(self.store, self.task, self.project,
MODULE, CONTRACT, "fail", "audit",
failures=["ci_green"])
record_pushback(self.store, self.task, self.project, MODULE,
CONTRACT, 1, package["failed_checks"], package,
failure_source="audit")
before = self.store.verify()["count"]
with self.assertRaises(ValueError):
record_reaudit(self.store, self.task, self.project,
MODULE, CONTRACT, 1, {"verdict": "pass"},
package=package, rerun_results={})
with self.assertRaises(ValueError):
record_reaudit(self.store, self.task, self.project,
MODULE, CONTRACT, 1, {"verdict": "pass"},
package=package,
rerun_results={"checkout-flow": False})
self.assertEqual(self.store.verify()["count"], before)
self.assertTrue(self._block()["merge_blocked"])
record_reaudit(self.store, self.task, self.project, MODULE,
CONTRACT, 1, {"verdict": "pass"},
package=package,
rerun_results={"checkout-flow": True})
self.assertFalse(self._block()["merge_blocked"])
def test_rebuild_alone_does_not_unblock(self):
record_audit_verdict(self.store, self.task, self.project,
MODULE, CONTRACT, "fail", "audit",
failures=["ci_green"])
record_rebuild(self.store, self.task, self.project, MODULE,
CONTRACT, 1, {"commit": "old123"},
root_cause="implementation", status="decided")
self.assertTrue(self._block()["merge_blocked"])
record_rebuild(self.store, self.task, self.project, MODULE,
CONTRACT, 1, {"commit": "old123"},
new_impl_ref={"commit": "new456"},
root_cause="implementation", status="rebuilt",
retained_assets={"old_tests_status": "pass"})
self.assertTrue(self._block()["merge_blocked"])
record_audit_verdict(self.store, self.task, self.project,
MODULE, CONTRACT, "pass", "audit")
self.assertFalse(self._block()["merge_blocked"])
self._chain_ok()
def test_escalation_blocks_sample_does_not(self):
record_escalation(self.store, self.task, self.project,
MODULE, CONTRACT, "blocked_quota",
target="coordinator",
reason="fix exceeds quota", round_no=2,
failed_checks=["ci_green"])
self.assertTrue(self._block()["merge_blocked"])
record_escalation(self.store, self.task, self.project,
MODULE, CONTRACT, "audit_sample",
reason="every 10th record")
self.assertTrue(self._block()["merge_blocked"])
record_audit_verdict(self.store, self.task, self.project,
MODULE, CONTRACT, "pass", "audit")
self.assertFalse(self._block()["merge_blocked"])
def test_meta_review_tension_is_pinned(self):
fail_entry = record_audit_verdict(
self.store, self.task, self.project, MODULE, CONTRACT,
"fail", "audit", failures=["ci_green"])
self.assertTrue(self._block()["merge_blocked"])
# Wrong block corrected to pass: gate STAYS blocked until
# a fresh passing re-audit (track 3 restored action).
record_meta_review(self.store, self.task, self.project,
MODULE, CONTRACT, fail_entry["id"],
"fail", "pass", True, "reviewer-r",
findings=["wrong call"],
triggered_process="re-review")
self.assertTrue(self._block()["merge_blocked"])
# ...while track 4 counts use the effective verdict.
counts = self.store.module_failure_counts(MODULE, CONTRACT,
task_id=self.task)
self.assertEqual(counts["total_fails"], 0)
record_audit_verdict(self.store, self.task, self.project,
MODULE, CONTRACT, "pass", "audit")
self.assertFalse(self._block()["merge_blocked"])
# Wrong pass corrected to fail: re-blocks at once.
pass_entry = record_audit_verdict(
self.store, self.task, self.project, MODULE, CONTRACT,
"pass", "audit")
self.assertFalse(self._block()["merge_blocked"])
record_meta_review(self.store, self.task, self.project,
MODULE, CONTRACT, pass_entry["id"],
"pass", "fail", True, "reviewer-r",
findings=["missed ci"],
triggered_process="re-review")
self.assertTrue(self._block()["merge_blocked"])
self._chain_ok()
def test_modules_are_scoped_not_global(self):
record_audit_verdict(self.store, self.task, self.project,
"apps/a", CONTRACT, "fail", "audit",
failures=["ci_green"])
record_audit_verdict(self.store, self.task, self.project,
"apps/b", CONTRACT, "pass", "audit")
self.assertTrue(self._block()["merge_blocked"])
record_audit_verdict(self.store, self.task, self.project,
"apps/a", CONTRACT, "pass", "audit")
self.assertFalse(self._block()["merge_blocked"])
def test_triage_and_root_cause_mapping(self):
record_third_failure_assessment(
self.store, self.task, self.project, MODULE, CONTRACT,
"targeted_fix",
{"audit_conclusion": "nearly done",
"failure_items": ["ci_green"],
"passed_items": ["directory_isolation"]})
timeline = audit_timeline(self.store.conn, task_id=self.task)
assessment = [e for e in timeline["entries"]
if e["action"] == "audit_third_failure_assessment"][0]
self.assertEqual(assessment["parsed"]["determination"],
"continuation_round")
with self.assertRaises(ValueError):
record_third_failure_assessment(
self.store, self.task, self.project, MODULE, CONTRACT,
"early_refreeze", {"audit_conclusion": "contract"})
with self.assertRaises(ValueError):
record_rebuild(self.store, self.task, self.project,
MODULE, CONTRACT, 1, {"commit": "old"},
root_cause="unknown")
# Design root cause stores as contract_design.
record_rebuild(self.store, self.task, self.project, MODULE,
CONTRACT, 1, {"commit": "old"},
root_cause="design", status="decided")
timeline = audit_timeline(self.store.conn, task_id=self.task)
rebuild = [e for e in timeline["entries"]
if e["action"] == "audit_rebuild"][0]
self.assertEqual(rebuild["parsed"]["root_cause"],
"contract_design")
self._chain_ok()
def test_missing_ledger_table_is_unblocked_not_error(self):
fresh = os.path.join(self.tmp.name, "empty.db")
conn = sqlite3.connect(fresh)
try:
block = pushback_block_for_task(conn, "nope")
finally:
conn.close()
self.assertFalse(block["merge_blocked"])
self.assertIsNone(block["blocked_by"])
def test_decide_pushback_round_feeds_ledger_round(self):
decision = decide_pushback(_failing_audit(), 0)
self.assertEqual(decision["action"], "pushback")
self.assertEqual(decision["next_round"], 1)
record_audit_verdict(self.store, self.task, self.project,
MODULE, CONTRACT, "fail", "audit",
failures=decision["failed_checks"])
package = pushback_package_from_gate(
_failing_gate(), self.task, decision["round"])
record_pushback(self.store, self.task, self.project, MODULE,
CONTRACT, decision["next_round"],
package["failed_checks"], package,
failure_source="audit")
block = self._block()
self.assertTrue(block["merge_blocked"])
self.assertEqual(block["round"], 1)
class EndpointChainTestCase(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self.tmp.name, "control.db")
self.auth = AuthStore(self.db_path)
self.owner_id, self.owner_key = self.auth.create_user("owner")
self.server = create_server(self.db_path, "127.0.0.1", 0)
self.port = self.server.server_address[1]
self.thread = threading.Thread(
target=self.server.serve_forever, daemon=True)
self.thread.start()
status, self.project = self._req(
"POST", "/projects", {"name": "demo"}, self.owner_key)
self.assertEqual(status, 201)
self._saved_env = {k: os.environ.get(k) for k in
("CLOUD_MERGE_GATE_MODE",
"CLOUD_BRANCH_PROTECTION")}
os.environ.pop("CLOUD_MERGE_GATE_MODE", None)
os.environ.pop("CLOUD_BRANCH_PROTECTION", None)
self.ledger = LedgerStore(self.db_path)
def tearDown(self):
self.server.shutdown()
self.server.server_close()
self.thread.join(timeout=5)
for key, value in self._saved_env.items():
if value is None:
os.environ.pop(key, None)
else:
os.environ[key] = value
try:
self.ledger.close()
except Exception:
pass
try:
self.auth.conn.close()
except Exception:
pass
self.tmp.cleanup()
def _req(self, method, path, body=None, key=None):
conn = http.client.HTTPConnection(
"127.0.0.1", self.port, timeout=10)
hdrs = {}
payload = None
if body is not None:
payload = json.dumps(body).encode("utf-8")
hdrs["Content-Type"] = "application/json"
if key is not None:
hdrs["X-API-Key"] = key
try:
conn.request(method, path, body=payload, headers=hdrs)
resp = conn.getresponse()
data = resp.read()
finally:
conn.close()
try:
parsed = json.loads(data.decode("utf-8")) if data else None
except ValueError:
parsed = None
return resp.status, parsed
def _completed_task(self):
status, task = self._req(
"POST", "/tasks",
{"project_id": self.project["id"], "prompt": "x"},
self.owner_key)
self.assertEqual(status, 201)
conn = sqlite3.connect(self.db_path)
try:
conn.execute(
"UPDATE tasks SET status = 'completed' WHERE id = ?",
(task["id"],))
conn.commit()
finally:
conn.close()
return task
def _hard_body(self, **overrides):
body = {
"files_changed": ["apps/web/a.py"],
"owned_dir": "apps/web",
"branch_protection": "protected",
"ci_status": "success",
"ci_checks": [{"name": "ci", "status": "success"}],
"required_checks": ["ci"],
"contract_version": "v2",
"contract_frozen": True,
"coordinator_approved": True,
"unresolved_review_comments": 0,
}
body.update(overrides)
return body
def _status_of(self, task_id):
_s, got = self._req("GET", "/tasks/%s" % task_id,
key=self.owner_key)
return got["status"]
def _block_task(self, task, package=None):
if package is None:
package = pushback_package_from_gate(_failing_gate(),
task["id"], 0)
record_audit_verdict(self.ledger, task["id"],
self.project["id"], MODULE, CONTRACT,
"fail", "audit",
failures=package["failed_checks"])
record_pushback(self.ledger, task["id"], self.project["id"],
MODULE, CONTRACT, 1, package["failed_checks"],
package, failure_source="audit")
return package
def test_audit_fail_blocks_merge_fix_reaudit_releases(self):
task = self._completed_task()
package = self._block_task(task)
status, blocked = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(), self.owner_key)
self.assertEqual(status, 409, blocked)
self.assertIn("audit_pushback_blocked",
blocked["merge_gate"]["blocks"])
self.assertTrue(
blocked["merge_gate"]["audit_pushback"]["merge_blocked"])
self.assertTrue(
blocked["merge_gate"]["audit_pushback"]["release_blocked"])
self.assertEqual(
blocked["merge_gate"]["audit_pushback"]["blocked_by"],
"audit_pushback")
self.assertEqual(self._status_of(task["id"]), "completed")
# Fix + passing re-audit (no QA links on this package).
record_fix(self.ledger, task["id"], self.project["id"],
MODULE, CONTRACT, 1,
{"summary": "fix ci", "diff_sha256": "b" * 64,
"files_changed": ["apps/web/a.py"]}, package)
status, still = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(), self.owner_key)
self.assertEqual(status, 409)
record_reaudit(self.ledger, task["id"], self.project["id"],
MODULE, CONTRACT, 1, {"verdict": "pass"},
package=package)
status, merged = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(), self.owner_key)
self.assertEqual(status, 200, merged)
self.assertEqual(self._status_of(task["id"]), "merged")
self.assertFalse(
merged["merge_gate"]["audit_pushback"]["merge_blocked"])
chain = self.ledger.verify()
self.assertTrue(chain["ok"], chain)
def test_qa_failed_links_gate_the_release(self):
task = self._completed_task()
links = [{"link": "checkout-flow", "case": "pay",
"failure": "500", "evidence": "log",
"reproduction": ["run checkout"]}]
package = pushback_package_from_gate(
_failing_gate(), task["id"], 0, qa_failed_links=links)
self._block_task(task, package=package)
status, _blocked = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(), self.owner_key)
self.assertEqual(status, 409)
# Claimed pass without a passing re-run of the failed
# link must not reach the chain, and must not unblock.
with self.assertRaises(ValueError):
record_reaudit(self.ledger, task["id"],
self.project["id"], MODULE, CONTRACT, 1,
{"verdict": "pass"}, package=package,
rerun_results={"checkout-flow": False})
status, _still = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(), self.owner_key)
self.assertEqual(status, 409)
record_reaudit(self.ledger, task["id"], self.project["id"],
MODULE, CONTRACT, 1, {"verdict": "pass"},
package=package,
rerun_results={"checkout-flow": True})
status, merged = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(), self.owner_key)
self.assertEqual(status, 200, merged)
def test_pushback_blocks_with_full_hard_evidence(self):
# A1 改写:旧用例 test_pushback_blocks_even_in_record_mode
# 已改名。本用例的性质不变——链上推回阻塞时,即使五项
# 证据全齐也 409;只是不再经 record 模式证明“与模式无关”,
# 因为服务端已无其它模式可比。
task = self._completed_task()
self._block_task(task)
status, blocked = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(), self.owner_key)
self.assertEqual(status, 409, blocked)
self.assertIn("audit_pushback_blocked",
blocked["merge_gate"]["blocks"])
self.assertEqual(self._status_of(task["id"]), "completed")
def test_hard_gate_failure_without_pushback_has_no_pushback_code(self):
task = self._completed_task()
status, blocked = self._req(
"POST", "/tasks/%s/merge" % task["id"],
self._hard_body(ci_status="pending"), self.owner_key)
self.assertEqual(status, 409, blocked)
self.assertIn("ci_not_green", blocked["merge_gate"]["blocks"])
self.assertNotIn("audit_pushback_blocked",
blocked["merge_gate"]["blocks"])
self.assertFalse(
blocked["merge_gate"]["audit_pushback"]["merge_blocked"])
self.assertEqual(self._status_of(task["id"]), "completed")
+48
View File
@@ -153,6 +153,54 @@ class TestAuth(unittest.TestCase):
self.assertFalse(
self.store.check_permission(target_id, project_id, "create_task"))
def test_audit_retries_integrity_error_and_reraises_others(self):
"""A4 回归:audit() 重试分支的 sqlite3 引用必须存在。
首调 IntegrityError 后重试成功:审计对恰好一行(audit_log
1 行 + audit_ledger 1 行),不是 0 行也不是多行;非完整
性错误原样抛回(RuntimeError,不是 NameError 掩盖原错)。
背景:auth.py 曾全文件无 import sqlite3,audit() 的
except 分支在 424 行求 isinstance(exc, sqlite3.IntegrityError)
时直接 NameError,任何首调失败都走不到重试或原错抛回。
"""
import cloud.auth as auth_mod
original = auth_mod.audit_on_conn
calls = {"n": 0}
def _flaky(conn, *args, **kwargs):
calls["n"] += 1
if calls["n"] == 1:
raise sqlite3.IntegrityError(
"UNIQUE constraint failed: audit_ledger.entry_hash")
return original(conn, *args, **kwargs)
auth_mod.audit_on_conn = _flaky
try:
self.store.audit("u1", "p1", "act", "d")
finally:
auth_mod.audit_on_conn = original
self.assertEqual(calls["n"], 2)
self.assertEqual(len(self.store.list_audit("p1")), 1)
ledger_rows = self.store.conn.execute(
"SELECT COUNT(*) FROM audit_ledger").fetchone()[0]
self.assertEqual(ledger_rows, 1)
def _broken(conn, *args, **kwargs):
raise RuntimeError("ledger boom")
auth_mod.audit_on_conn = _broken
try:
with self.assertRaises(RuntimeError) as caught:
self.store.audit("u1", "p1", "act", "d2")
finally:
auth_mod.audit_on_conn = original
self.assertEqual(str(caught.exception), "ledger boom")
self.assertEqual(len(self.store.list_audit("p1")), 1)
ledger_rows = self.store.conn.execute(
"SELECT COUNT(*) FROM audit_ledger").fetchone()[0]
self.assertEqual(ledger_rows, 1)
def test_audit_append_only(self):
project_id, owner_id, _v, _c, _m, _o = self._project_with_roles()
self.assertFalse(hasattr(AuthStore, "delete_audit"))
+61 -19
View File
@@ -5,13 +5,14 @@ import tempfile
import time
import unittest
from cloud.auth import audit_on_conn
from cloud.automations import (
_audit_row,
create_automation,
ensure_automations_tables,
list_automation_runs,
run_due_automation,
)
from cloud.ledger import GENESIS_HASH, compute_entry_hash
from cloud.server import init_db
@@ -49,25 +50,48 @@ def _set_quota(db_path, user_id, micros):
conn.close()
class AuditRowPgIdentityTestCase(unittest.TestCase):
class AuditPairPgIdentityTestCase(unittest.TestCase):
"""模拟 PG:id 是 BIGINT identity,显式插 TEXT uuid 会报错。
假连接只接受不带 id 的 INSERT;旧实现(带 TEXT id)在此
假连接上应失败、新实现应成功且审计行落盘。
假连接只接受不带 id 的 INSERT;旧 _audit_row(带 TEXT id)
在此假连接上应失败。接替它的 audit_on_conn 须同时做到:
audit_log 行不带 id、链条目与该行同 created_at、空链时
prev 为 GENESIS、entry_hash 按规范载荷重算一致;且本函数
不 commit(事务属于调用方,见 run_due_automation 文档)。
假连接类名须为 PgConnection:ledger.ensure_schema 靠类名
判定 PG 连接以跳过 SQLite 触发器,与真实 PG 适配层同款。
"""
class _PgLikeConn:
class _FakeCursor:
def __init__(self, row=None):
self._row = row
self.lastrowid = 1
def fetchone(self):
return self._row
class PgConnection:
def __init__(self):
self.rows = []
self.audit_rows = []
self.ledger_rows = []
self.committed = False
def execute(self, sql, params=()):
if sql.strip().upper().startswith("INSERT INTO AUDIT_LOG(ID"):
upper = sql.strip().upper()
if upper.startswith("INSERT INTO AUDIT_LOG(ID"):
raise ValueError("column \"id\" is of type bigint")
if sql.strip().upper().startswith("INSERT INTO AUDIT_LOG"):
self.rows.append(params)
return None
raise ValueError("unexpected sql in fake PG conn")
if upper.startswith("INSERT INTO AUDIT_LOG"):
self.audit_rows.append(params)
return AuditPairPgIdentityTestCase._FakeCursor()
if upper.startswith("SELECT ENTRY_HASH FROM AUDIT_LEDGER"):
return AuditPairPgIdentityTestCase._FakeCursor(None)
if upper.startswith("INSERT INTO AUDIT_LEDGER"):
self.ledger_rows.append(params)
return AuditPairPgIdentityTestCase._FakeCursor()
if upper.startswith("CREATE TABLE"):
return AuditPairPgIdentityTestCase._FakeCursor()
raise ValueError("unexpected sql in fake PG conn: %r" % sql)
def commit(self):
self.committed = True
@@ -75,15 +99,33 @@ class AuditRowPgIdentityTestCase(unittest.TestCase):
def rollback(self):
pass
def test_insert_without_id_succeeds_on_pg_like_conn(self):
conn = self._PgLikeConn()
_audit_row(conn, "p1", "automation_run_created",
"auto1 -> task1", 123.0)
self.assertTrue(conn.committed)
self.assertEqual(len(conn.rows), 1)
def test_pair_without_id_succeeds_on_pg_like_conn(self):
conn = self.PgConnection()
audit_on_conn(conn, "system", "p1", "automation_run_created",
"auto1 -> task1", created_at=123.0)
# 调用方拥有事务:本函数自己不 commit。
self.assertFalse(conn.committed)
self.assertEqual(len(conn.audit_rows), 1)
# 参数里没有 uuid:第一位是 system,第二位是 project_id。
self.assertEqual(conn.rows[0][0], "system")
self.assertEqual(conn.rows[0][1], "p1")
self.assertEqual(conn.audit_rows[0][0], "system")
self.assertEqual(conn.audit_rows[0][1], "p1")
self.assertEqual(conn.audit_rows[0][4], 123.0)
self.assertEqual(len(conn.ledger_rows), 1)
ledger = conn.ledger_rows[0]
# audit_ledger 列序:user_id, project_id, task_id, action,
# detail, created_at, prev_hash, entry_hash。
self.assertEqual(ledger[0], "system")
self.assertEqual(ledger[1], "p1")
self.assertIsNone(ledger[2])
self.assertEqual(ledger[3], "automation_run_created")
self.assertEqual(ledger[4], "auto1 -> task1")
self.assertEqual(ledger[5], 123.0)
self.assertEqual(ledger[6], GENESIS_HASH)
self.assertEqual(
ledger[7],
compute_entry_hash(GENESIS_HASH, "system", "p1", None,
"automation_run_created",
"auto1 -> task1", 123.0))
class RunDueAutomationTestCase(unittest.TestCase):
-1
View File
@@ -9,7 +9,6 @@ import os
import sqlite3
import sys
import tempfile
import time
import unittest
from pathlib import Path
-1
View File
@@ -5,7 +5,6 @@
frontmatter、名字与目录一致、正文非空、不含密钥占位。
"""
import os
import re
import unittest
from pathlib import Path
-3
View File
@@ -16,7 +16,6 @@ import time
import unittest
from pathlib import Path
from types import SimpleNamespace
from unittest import mock
try:
from cloud.auth import AuthStore
@@ -25,7 +24,6 @@ try:
from cloud.server import create_server, init_db
from cloud.worker import Worker
from cloud.workers import (
heartbeat_worker,
list_workers,
register_worker,
)
@@ -37,7 +35,6 @@ except ImportError:
from cloud.server import create_server, init_db
from cloud.worker import Worker
from cloud.workers import (
heartbeat_worker,
list_workers,
register_worker,
)
Loaded 100 of 263 files, more files were not shown because too many files have changed in this diff. Show more