Only pure computation/validation sub-blocks of _handle_github_webhook,
_create_followup, _deliver_to_thread and _task_chat_history are extracted;
all DB writes, audits, external calls and side-effect blocks stay in the
callers, error strings/codes and response shapes verbatim. _deliver_to_thread
uses the shared goal classifier only for /goal input: its non-goal terminal
path still goes to CAS revive (not 409) and its merged path keeps its own
wording. New tests: test_longfunc_remaining.py (15).
_continue_task 226->168 via _validate_continue_options;
_task_wait 158->139 via _validate_wait_params;
_task_thread_read 187->152 via _summarize_trace_entries;
_metrics 123->104 via _summarize_metrics_tasks.
No behaviour, message, status or route change; error strings verbatim.
New tests: test_longfunc_extra (9). Cloud full suite: 1439 OK, skipped=4.
Follows #229 pattern; must merge after #229 and be rebased/checked then.
Move the /tasks/* sub-route out of _dispatch into _dispatch_task_route
(query passed explicitly; the files/tree/thread branches use it), move
the push-subscription DELETE branch out of _dispatch_project into
_project_push_subscription_delete, and turn the goal-validation block
of _create_task into the pure module function
_validate_task_goal_options (returns error string instead of sending
400, caller sends it unchanged). Statements are moved verbatim except
indentation and the goal send+return -> return-error transform; no
behaviour change. server.py module split intentionally not done: it
remains a larger, higher-risk refactor.
Add test_longfunc_split covering the pure goal validator's success
paths and all eight error strings.
- ensure_workers_table: new tables include host TEXT, pid BIGINT,
address TEXT; old tables gain them via _ensure_hostpid_columns
(PRAGMA column check, missing-only ALTER, old rows keep NULL,
never fabricates host/pid). SQLite and Postgres share this path
(PG via the db adapter's PRAGMA translation); POSTGRES_SCHEMA
deliberately still excludes workers, which is volatile heartbeat
data in UNMIGRATED_TABLES (adding it broke the P2 migration
alignment tests, verified and reverted).
- heartbeat/register: given host/pid/address refresh, omitted
values preserve the old row; address has no derived source.
Local worker passes host_name + own pid; HTTP heartbeat only
stores caller-supplied values, never the server's own.
- classify_stale_workers: kill only when age strictly exceeds the
threshold AND host matches local host AND pid is proven dead;
equal threshold, foreign host, missing pid, or live/unknown pid
are list_only. Classification only, never kills, never matches
by name/prefix, never touches infra containers.
SKILL.md header pointed at same-dir README.md but the pack had no
such file (CHANGELOG was supplied by #217, README was not). This
track adds the README only: purpose, chapter map, appendix
provenance entry point, seeding behaviour, and known boundaries.
It states no rule, changes no SKILL.md semantics, and does not
duplicate the CHANGELOG/script explanations -- overlapping parts
are pointers, with an explicit division-of-labour section.
Deletion retention is written only as the tombstone-design
placeholder, deferring to the skill_seed implementation, because
no tombstone implementation has landed. Test pins that every
file SKILL.md cites as same-directory exists (CHANGELOG, README).
SKILL.md claimed its appendix was spliced verbatim from
templates/roles/ by a generation script (see CHANGELOG), but the pack
contained none of the three. Add the nine role sources (mechanically
split from the v0.2 appendix; SKILL.md untouched, zero semantic
change), tools/check_appendix.py (verify + --emit, stdlib only, never
edits SKILL.md), CHANGELOG.md, and test_skill_appendix_provenance.py
(5 tests). Seeding unchanged: skill_seed copies SKILL.md only, so
templates/tools/CHANGELOG never reach the tenant home.
- cloud/merge_gate_hard.py (new): pure hard gate; mode resolution is
env-only (unset/invalid -> hard, fail-closed); five checks plus
branch protection must be protected; unknown/unprotected block
even with explicit confirm; no-evidence blocks (fail-closed);
directory check reuses #209 check_directory_isolation (v0.3 rule);
blocks codes + merge_gate_blocked audit detail are the track-3
pushback interface (no pushback state machine in this track).
- cloud/server.py: POST /tasks/{id}/merge uses hard mode by default;
record/enforce/off only when explicitly set via env (preserving
#209 semantics); body merge_gate_mode is ignored server-side so
callers cannot downgrade the gate; every gate block writes a
merge_gate_blocked audit before returning 409, status unchanged;
hard-mode body type errors return 400.
- docs/merge-gate-branch-protection.md (new): protection payload
(PR + strict status checks + no force-push + no deletion +
enforce_admins), empty required_checks = unconfigured, Pro/public
prerequisites, 403 causes; states protection is NOT claimed enabled.
- SKILL.md (gate-mechanism wording only): four checks -> five checks;
body + coordinator + client appendix gate paragraphs updated to
default hard, merge_gate_blocked audit, payload doc reference.
- tests: 31 new hard-gate tests (pure + endpoint + skill text);
#209 endpoint tests now set record env explicitly (their intent);
test_server_merge 200-path calls now carry full hard evidence.
- cloud/merge_gate.py: pure gate; directory check reuses the v0.3
git diff --name-only merge-base command; protection modes
record (default, explicit confirm required when unprotected),
enforce (always blocks when unprotected), off (explicit only).
Confirm covers protection risk only, never directory violations.
Protection status is caller/env-declared only; unknown is a
recorded risk, not a claim of protection.
- server POST /tasks/{id}/merge: applies the gate only when gate
evidence is supplied (or env declares protection); no-evidence
calls behave as before plus a merge_gate field in the response.
Blocked merges return 409 without changing status; confirmed
unprotected merges write a merge_gate_risk audit entry.
- skill_seed.py: whole-directory seeding evaluated and rejected
(cannot preserve tenant deletions; no benefit while the pack
holds only SKILL.md); SKILL.md-only behaviour unchanged.
- software-engineering SKILL.md: gate paragraph added to the body
and to the coordinator + client appendix sections (v0.2 text on
this base; port to v0.3 text on rebase after #199 merges).
- tests: 16 new tests covering gate modes, directory reuse,
body/appendix bidirectional assertions naming the missing side,
SKILL.md-only seeding and tenant-edit preservation.
Stacked on v0.3 + v0.4 (HEAD d6522ba feat-full-auto): v0.3/v0.4
content preserved, not reverted. Adds G7 post-release ops and
observation (monitoring, errors, cost/quota, feedback, hotfix loop,
rollback triggers, ops manual) and G8 iteration and scaling (v1.1
handover list, contract v1->v2 protocol, new-end onboarding,
scale/performance, deprecation), with G6->G7 and G7->G8 gates,
DAG, gate table, coordinator derivation, G0-G8 todos, checklist,
09-ops and 10-iteration role templates, and ops-manual / v1.1
handover templates in both body and appendix. Settles web+React+
multi-viewport focus, cross-viewport E2E, web single-end release
with iOS/Android native postposed, open-source-first, and the
TypeScript full-stack default (user-request-only language change;
codex-app-server itself stays Python). Pending #199 and #205
merge, then one rebase round. PR only, do not merge or deploy.
- P1-1 directory-isolation executable check: git diff --name-only
against merge-base, output recorded in merge record
- P1-2 derivation checklist table (8 required items, missing one blocks dispatch)
- P1-3 QA split into single-client review and cross-client E2E with
fixed deliverables and agent-test attribution
- P1-4 sub-task slim-pack trimming rules (listed chapters only, never full copies)
- P2: inter-thread Q&A default policy, docs/decisions.md append-only,
per-client onboarding docs (web first at G4a), v1.1 handover checklist
- Body and appendix kept in sync; new lightweight keyword test
- worker._tenant_dirs: playwright refresh log used undefined 'task';
use the method argument project_id (ruff F821, real bug: NameError
aborted tenant seeding when a playwright script was refreshed).
Regression: cloud/tests/test_worker_playwright_seed_f821.py (2 tests;
pre-fix run reproduces NameError at worker.py:2780).
- harness2.assign_ids: hashlib.sha1(..., usedforsecurity=False) -- the
digest is only a deterministic shuffle seed, not a security use;
digest value unchanged, bandit B324 no longer fires (High 5 -> 4).
- harness2.select_final: rename ambiguous loop var 'l' to 'lab'
(ruff E741, behaviour-invariant).
- Deliberately NOT changed: host.py shell=True x4 (see PR body -- all
four are intentional shell commands, changing semantics is unsafe),
remaining ruff 109 / mypy 24, workflow thresholds (75% coverage
gate unchanged; no gate was relaxed or removed).
Phase 1 of audit-hardening CI expansion (baseline origin/main aebc623):
- lint: ruff E9 hard gate; full E,F@120 report-only (baseline 111 items)
- typecheck: mypy default mode, production scope, report-only (baseline 24)
- security: bandit production scope, report-only (baseline 287, High 5)
- depscan: pip-audit on requirements-ci.txt, hard gate (0 vulns; repo has
no production dependency manifest, see summary)
- coverage: production 58 files, fail-under=75 (baseline measured 76%)
- release build: E9 + pip-audit hard gates before packaging; full reports
stay in host-verify to avoid drift
No production code changed; CI billing failure mode unchanged.
Replace bare open().read()/write() in tests with Path.read_text/
write_text (or json.loads over read_text); the five remaining
open(...).close() empty-file creations become Path.write_text('').
No assertion, predicate, or production code changed; no behaviour
change beyond closing the handle deterministically.
Long-function splits and server.py module split intentionally not
included: the longest functions (_dispatch, _create_task,
_dispatch_project, ~375-395 lines in a 10430-line server.py) are
central routing/creation paths, and splitting them is not low-risk;
see PR body.
Goal tasks pushed back by the audit loop lost the packet preamble on
the next round: claim-time _apply_audit_loop_packet pasted it into the
in-memory prompt/goal, but the next round collided with the existing
host task name, fell back to breakpoint goal-resume (no --goal-text),
and the host reused the stored goal, discarding the preamble.
Fix (minimal, direction (c) trimmed): host goal-resume gains
--resume-note, accepted only by the breakpoint branch and prepended
to the first turn prompt of this resume only -- never written back
to the stored goal, never touching rounds or verification. The worker
stashes the same preamble as the memory-only _audit_loop_resume_note
at claim time and forwards it on breakpoint resume (Local/Docker
goal_resume_task and the popen goal-resume command, each guarded to
ignore the note when goal_text is set for a new-goal resume).
Accounting untouched: audit 3-round cap, goal_max_rounds/quota, and
DB goal columns are unchanged; non-goal pushback is unchanged.
Tests: cloud/tests/test_audit_n12.py (9 tests) -- host first-turn-only
note + stored-goal/rounds, worker exists-fallback end-to-end with
--resume-note, non-goal regression, goal-column accounting, and
Local/Docker/popen command shapes. Full cloud suite: 1760 OK
(skipped=11), baseline 1751 + 9 new; tests/test_host.py: 62 OK.
- I-3: new models.evaluate_capability returning a CapabilityVerdict
(ok/missing/unknown from check, known/unknown_missing from split,
status/verb derivations); decide_routing, worker gateway check,
worker persisted recheck, dry-run tier+gateway checks and server
shadow endpoint all call it. Blocking rules stay at the call
sites (gateway has no worker-default fallback, recheck blocks
only on known, dry-run/shadow never block); all user-visible
audit details, dry-run notes and shadow notes are byte-identical.
- I-4: new models.merge_policy_documents shared by
load_routing_policy and dry_run_from_env; the old tiers-update
block is kept verbatim (it was a no-op after update(): JSON
tiers replace the whole map, file tiers absent from JSON are
dropped -- preserved, not fixed, as that would be a behaviour
change). dropped_tiers/partial_failures remain load-time
records, not merge products.
- Tests: test_audit_batch3.py, 13 tests (verdict matrix + parity
with check/split, decide/dry-run wording, merge boundaries +
300 seeded random pairs + loader/dry-run agreement).
Full suite: 1694 OK (skipped=11), baseline 1681 + 13, 0 failures.
- I-1: evaluate_shadow counts dropped rows by cause (unparseable /
unknown_tier / explicit / duplicate) with the identity
audit_rows_total == routed_task_n + dropped_total; the shadow
endpoint passes created_at/id through and reports audit totals
next to sample counts.
- I-2 (option b): parse logs a warning naming each tier dropped for
an invalid mode/requires; load attaches partial_failures to the
policy; the worker records a task-level
worker_routing_policy_load_failed audit for partial failures
(same action as full failure, detail says failure=partial);
the shadow report carries the policy fingerprint, dropped tiers
and partial failures (no task context on that path, so no
task-level audit there).
- I-5: coordinator gate audit lookup failure now raises
_CoordGateLookupFailed and the merge converts it to escalate
(no requeue, no round consumed); persisted-model lookup failure
and required-tools check failure each write a named audit row
before falling through to dispatch/allow; audit-loop evaluate
failure stays escalate (regression test).
- Test fixtures: coordinator suites build audit_log via AuthStore
as production's first real audit does, since their stubbed
_audit never creates the table.
- B-A: migrations.py tasks backfill loop now types audit_loop as
INTEGER (other 6 columns stay TEXT), matching db.py backfill,
server/worker ensure lists; TEXT readback '0' is truthy and
would silently enable the audit loop for audit_loop=0 rows.
- B-C: gateway preparation failure early-return now calls
_running_clear(task_id) next to _cleanup_task_seed, matching
missing-key/repo/capability/tool gates; previously the capacity
slot leaked until worker exit (default capacity 1 blocks reclaim).
- Tests: cloud/tests/test_audit_batch1.py (3 tests): migration
ALTER types, INTEGER 0 readback not enabled via real worker
row mapping, gateway failure clears slot and capacity reclaim.
_claim doc 'three gates' list left unchanged (full early-return
gate enumeration not verified this batch; residual wording).
- No other audit items touched (I-1..I-14, N-1..N-12).
- M1: test_goal_branch_via_convert_queued_then_claim actually calls
server._goal_convert_queued on a queued auto-resumed task before
claiming, instead of only inserting the post-conversion state.
- M5: test_worker_routing_audit_rows_carry_no_fingerprint drives a
real run_one with an active policy and asserts worker_routing
audit rows exist and carry no fingerprint (negative control
verified: injecting fingerprint into the detail fails it).
- M2: 'all callers' wording tightened to the real counts
(load_routing_policy: 2 production call sites; evaluate_shadow: 1).
- M4: content_fingerprint docstring now states the fingerprint covers
the full normalized payload including fallback_models, which the
router deliberately does not apply (fail-safe direction).
No behaviour change; M3 DESIGN.md edits are outside the repo.
- db: translate INSERT OR IGNORE ... SELECT to ON CONFLICT DO NOTHING
(VALUES-only regex left SELECT form untranslated -> PG syntax error);
register task_audit_loop_state (task_id) in _OR_REPLACE_PK so PG
INSERT OR REPLACE becomes a real upsert instead of DO NOTHING.
- server: _backfill_default_quotas uses SAVEPOINT containment and no
longer commits the caller's transaction, so a backfill failure cannot
roll back init_db's migrate ALTERs.
- worker: _coord_child_rows rolls back before column-fallback retries so
a missing-column failure on PG does not cascade into
InFailedSqlTransaction.
- tests: test_pg_init_fix.py (translation, SQLite backfill/upsert,
real-PG fresh-init + merge + upsert + backfill containment, skipped
without CLOUD_TEST_PG_DSN/driver).
Parent audit loop sits after all children are aggregated and
before the parent final state; the single-task _audit_loop_wire
is never applied to parents or children. Parent gates are
per-child gates child:<name> (completed only); execution
failures map to one deterministic finding per child; subjective
findings are split off before evaluate and only audited as
needs_review. Gate-rejected children (capability/tool/approval
audit rows, or distinctive error markers) bypass evaluate and
escalate directly without consuming a round; tolerated, paused
and quota-missing parts are excluded, never findings. Rounds
are counted only in the parent task_audit_loop_state; on
pushback the packet is stored on the parent and split per failed
child into that child's audit_loop_packet column, consumed once
by the existing packet-injection path at next claim; pass and
escalate clear parent and child packets. POST /tasks now allows
audit_loop on coordinator parents (the old 400 rationale is
gone); children still take no external switch.
Audit-loop wiring, worker completion path only: after the final
write-back (and only when the final is completed/failed, not quota,
not cancelled, not goal-converted, and the task carries audit_loop),
collect this attempt's deterministic gate results (execution,
goal_verify only when host reported goal_met, capability /
required_tools / gateway as passed-or-inapplicable markers, since a
pre-dispatch gate rejection early-returns before any runner round
and requeueing it would deterministically re-fail the same gate)
and call audit_loop_evaluate. Pushback stores the packet JSON in
tasks.audit_loop_packet and CAS-requeues the task; the next claim
prepends the packet preamble to prompt (and goal) with a data-only
separator, DB prompt untouched. Pass clears the packet. Escalate
sets failed (VALID_STATUSES has no needs_review, never invented)
with an escalation error and clears the packet; subjective
needs_review leaves the outcome untouched and only audits.
New tasks.audit_loop / audit_loop_packet columns follow the
attempts DDL pattern in worker ensure, server migrate, PG schema,
PG backfill, migrations backfill, and POST /tasks accepts
audit_loop (True/1 only, coordinator tasks rejected like v9).
Coordinator parent tasks return via _run_coordinator before the
attempt loop and coordinator children never inherit audit_loop,
so coordinator wiring is left as legacy, unchanged.
Audit loop engine (cloud/audit_loop.py): Finding/PushbackPacket/LoopState pure functions; deterministic gates only, subjective -> needs_review; packet completeness enforced; standards-weakening -> escalate; regression and missing-gate checks enforce full re-run; MAX_PUSHBACKS=3, round>=3 -> escalate. Persistence in new single-row task_audit_loop_state (attempts DDL pattern: module CREATE_TABLE_SQL + db.py POSTGRES_SCHEMA + migrations list + server init + worker ensure); task_attempts NOT reused (attempt_no PK collision) and task_followups NOT reused (user messages). Worker.audit_loop_evaluate is the explicit opt-in hook (task audit_loop field only, no env switch, never called by run_one); escalate audits worker_audit_loop_escalated; corrupted/missing count fail-closes to escalate.
Parse: agent_tools/tool_approvals column parsing unified into agent_def shared helpers (parse None = undeclared; server keeps None, worker maps to []/{} at _task_from_row, dispatch behaviour unchanged). Goal: _apply_resume_preamble goal branch proven reachable via _goal_convert_queued on a queued auto-resumed task, kept with a test and corrected docs (not dead code). Fail-open: audit write failure and preamble lookup failure now log warnings (best-effort unchanged); partial policy failure and shadow parse drops documented as known gaps. M7: shadow endpoint explicit_capability_failures_truncated test. Fingerprint: RoutingPolicy.content_fingerprint (canonical JSON sha256, version excluded) surfaced only in dry-run output, never in audit rows. M1 wording: legacy gap stated as fixed stock persisting per-task until completion, not converging.
N1-1: writeback scope verified too broad (any chosen, incl. shadow
applied=False, worker_default and worker_explicit) and narrowed to
source=policy + applied only; writeback now records a
worker_model_persisted audit row as the identity marker.
N1: persisted models are rechecked against current tier requires
before dispatch on later attempts (fixes4 split known/unknown):
active + known-missing fails fast (worker_capability_recheck_
rejected, no runner), unknown-only and shadow only audit.
User-explicit, worker-CLI-explicit and gateway-filled tasks skip
the recheck; legacy models are recognised by a policy applied
worker_routing row. Recheck never rewrites tasks.model and never
re-decides.
Explicit gateway capability now splits known/unknown: known-missing
explicit models fail-fast under active (same shape as the gateway
default gate), unknown explicit models stay audit-only, shadow never
blocks; explicit priority now protects which model the user chose,
not that the chosen model must run. Shadow report reads
worker_capability_gateway_explicit rows as explicit_capability_failures
with a field note distinguishing the three explicit counts.
N2: coordinator requeue CAS rowcount=0 branch covered via a proxy
connection test (no refactor needed). N3: shadow endpoint test on
the create_server + real HTTP skeleton (capability, truncated, and
explicit-failure fields). N4: creation-layer rejection tests for
tools/instructions/tool_approvals on goal/coordinator tasks.
N5: model writeback failure logs a warning (best-effort kept).
N1: writeback capability-recheck trade-off documented in comment
only; recheck itself is a later batch.
B1: gateway capability check now triggers on walking the sub2api
gateway, not on the gateway having filled a model; task-level
explicit models are audited (worker_capability_gateway_explicit)
but never blocked, preserving explicit priority; gateway-default
models still fail-fast under active.
I1: dry-run adds a gateway-model section, sourced only from
CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active
tier whose requires the gateway model fails or cannot verify
blocks, shadow only warns.
I2: shadow review gate now counts completed samples
(completed >= 20), same basis for sample_shortfall; the endpoint
attaches per-tier capability status (read-only policy load) and
flags capability-failing tiers in the recommendation.
I3: configured-but-broken routing policy logs a warning in
load_routing_policy (unconfigured stays silent); the worker sets
routing_policy_load_failed and audits
worker_routing_policy_load_failed per task.
I4: resume preamble wraps prior output/error as data with an
explicit do-not-execute constraint; position verified in run_one
(tree -> preamble -> project -> original) and the docs corrected
from the false 'closest to the original prompt' claim.
I5: decided model is written back with a CAS UPDATE
(model IS NULL OR model=''), so later attempts reuse it;
explicit models are never overwritten.
I6: agent_* columns added to worker._ensure_result_columns and
db.init_postgres_schema backfill; migrations backfills them for
tasks before insert (sender_task_id pattern); column-level tests
for agent_tool_approvals and task_attempts.
I7: coordinator requeue adds AND status=<snapshot> (rowcount 0
skips requeue/audit) and filters paused_by_user / user_cancel /
user_pause children, which are never revived or cleared.
I8: explicit-model tasks (including coordinator children that
inherited a parent model) audit a source=explicit worker_routing
row; evaluate_shadow counts it as per-tier explicit_n only,
never routed_n, never a dedup slot.
M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation,
shadow sort doc fixed, truncated recommendation notes truncation,
seed cleanup on gateway/quota/goal-cap early returns, tools-only
coordinator definitions rejected, preamble goal branch marked
effectively unreachable, DESIGN.md drift corrected (outside repo).
M4/M7/M9: documentation/new-field only (pass_rate_fully_measured,
whitespace-model shadow-drop note, bound-note race note); old
field semantics unchanged.
- RoutingPolicy.fallback_models: move the cleaning loop into
__init__ (it was unreachable after tier_mode's return, so the
attribute never existed); still parsed but not applied.
- Capability fallback: RoutingDecision gains capability_fallback;
the worker audits worker_capability_fallback when it fires, so
a tier-model rejection followed by a worker-default fallback
is no longer invisible (source stays worker_default, no
model_routing event, so shadow samples are unaffected).
- Registry: resolve_registry_agent checks skills directly
against the loaded definition instead of re-parsing a
hand-built dict, so future fields cannot be silently dropped
the way tool_approvals was in v9.
- Shadow: evaluate_shadow sorts rows earliest-first by
created_at (id tiebreak) when timestamps are present, so
dedup no longer depends on caller row order; the server
path (which strips created_at) is unchanged.
- Gateway: when sub2api fills the default model, the worker
now checks it against the tier's capability requirements
with the same mode semantics as v8 -- shadow only audits,
active fails fast with worker_capability_gate_rejected and
never calls the runner. No worker-default fallback here:
the gateway model is the task's gateway model, falling back
to a worker default would run against the wrong gateway.
host