Only pure computation/validation sub-blocks of _handle_github_webhook,
_create_followup, _deliver_to_thread and _task_chat_history are extracted;
all DB writes, audits, external calls and side-effect blocks stay in the
callers, error strings/codes and response shapes verbatim. _deliver_to_thread
uses the shared goal classifier only for /goal input: its non-goal terminal
path still goes to CAS revive (not 409) and its merged path keeps its own
wording. New tests: test_longfunc_remaining.py (15).
_continue_task 226->168 via _validate_continue_options;
_task_wait 158->139 via _validate_wait_params;
_task_thread_read 187->152 via _summarize_trace_entries;
_metrics 123->104 via _summarize_metrics_tasks.
No behaviour, message, status or route change; error strings verbatim.
New tests: test_longfunc_extra (9). Cloud full suite: 1439 OK, skipped=4.
Follows #229 pattern; must merge after #229 and be rebased/checked then.
Move the /tasks/* sub-route out of _dispatch into _dispatch_task_route
(query passed explicitly; the files/tree/thread branches use it), move
the push-subscription DELETE branch out of _dispatch_project into
_project_push_subscription_delete, and turn the goal-validation block
of _create_task into the pure module function
_validate_task_goal_options (returns error string instead of sending
400, caller sends it unchanged). Statements are moved verbatim except
indentation and the goal send+return -> return-error transform; no
behaviour change. server.py module split intentionally not done: it
remains a larger, higher-risk refactor.
Add test_longfunc_split covering the pure goal validator's success
paths and all eight error strings.
- ensure_workers_table: new tables include host TEXT, pid BIGINT,
address TEXT; old tables gain them via _ensure_hostpid_columns
(PRAGMA column check, missing-only ALTER, old rows keep NULL,
never fabricates host/pid). SQLite and Postgres share this path
(PG via the db adapter's PRAGMA translation); POSTGRES_SCHEMA
deliberately still excludes workers, which is volatile heartbeat
data in UNMIGRATED_TABLES (adding it broke the P2 migration
alignment tests, verified and reverted).
- heartbeat/register: given host/pid/address refresh, omitted
values preserve the old row; address has no derived source.
Local worker passes host_name + own pid; HTTP heartbeat only
stores caller-supplied values, never the server's own.
- classify_stale_workers: kill only when age strictly exceeds the
threshold AND host matches local host AND pid is proven dead;
equal threshold, foreign host, missing pid, or live/unknown pid
are list_only. Classification only, never kills, never matches
by name/prefix, never touches infra containers.
SKILL.md header pointed at same-dir README.md but the pack had no
such file (CHANGELOG was supplied by #217, README was not). This
track adds the README only: purpose, chapter map, appendix
provenance entry point, seeding behaviour, and known boundaries.
It states no rule, changes no SKILL.md semantics, and does not
duplicate the CHANGELOG/script explanations -- overlapping parts
are pointers, with an explicit division-of-labour section.
Deletion retention is written only as the tombstone-design
placeholder, deferring to the skill_seed implementation, because
no tombstone implementation has landed. Test pins that every
file SKILL.md cites as same-directory exists (CHANGELOG, README).
SKILL.md claimed its appendix was spliced verbatim from
templates/roles/ by a generation script (see CHANGELOG), but the pack
contained none of the three. Add the nine role sources (mechanically
split from the v0.2 appendix; SKILL.md untouched, zero semantic
change), tools/check_appendix.py (verify + --emit, stdlib only, never
edits SKILL.md), CHANGELOG.md, and test_skill_appendix_provenance.py
(5 tests). Seeding unchanged: skill_seed copies SKILL.md only, so
templates/tools/CHANGELOG never reach the tenant home.
- cloud/merge_gate_hard.py (new): pure hard gate; mode resolution is
env-only (unset/invalid -> hard, fail-closed); five checks plus
branch protection must be protected; unknown/unprotected block
even with explicit confirm; no-evidence blocks (fail-closed);
directory check reuses #209 check_directory_isolation (v0.3 rule);
blocks codes + merge_gate_blocked audit detail are the track-3
pushback interface (no pushback state machine in this track).
- cloud/server.py: POST /tasks/{id}/merge uses hard mode by default;
record/enforce/off only when explicitly set via env (preserving
#209 semantics); body merge_gate_mode is ignored server-side so
callers cannot downgrade the gate; every gate block writes a
merge_gate_blocked audit before returning 409, status unchanged;
hard-mode body type errors return 400.
- docs/merge-gate-branch-protection.md (new): protection payload
(PR + strict status checks + no force-push + no deletion +
enforce_admins), empty required_checks = unconfigured, Pro/public
prerequisites, 403 causes; states protection is NOT claimed enabled.
- SKILL.md (gate-mechanism wording only): four checks -> five checks;
body + coordinator + client appendix gate paragraphs updated to
default hard, merge_gate_blocked audit, payload doc reference.
- tests: 31 new hard-gate tests (pure + endpoint + skill text);
#209 endpoint tests now set record env explicitly (their intent);
test_server_merge 200-path calls now carry full hard evidence.
- cloud/merge_gate.py: pure gate; directory check reuses the v0.3
git diff --name-only merge-base command; protection modes
record (default, explicit confirm required when unprotected),
enforce (always blocks when unprotected), off (explicit only).
Confirm covers protection risk only, never directory violations.
Protection status is caller/env-declared only; unknown is a
recorded risk, not a claim of protection.
- server POST /tasks/{id}/merge: applies the gate only when gate
evidence is supplied (or env declares protection); no-evidence
calls behave as before plus a merge_gate field in the response.
Blocked merges return 409 without changing status; confirmed
unprotected merges write a merge_gate_risk audit entry.
- skill_seed.py: whole-directory seeding evaluated and rejected
(cannot preserve tenant deletions; no benefit while the pack
holds only SKILL.md); SKILL.md-only behaviour unchanged.
- software-engineering SKILL.md: gate paragraph added to the body
and to the coordinator + client appendix sections (v0.2 text on
this base; port to v0.3 text on rebase after #199 merges).
- tests: 16 new tests covering gate modes, directory reuse,
body/appendix bidirectional assertions naming the missing side,
SKILL.md-only seeding and tenant-edit preservation.
Stacked on v0.3 + v0.4 (HEAD d6522ba feat-full-auto): v0.3/v0.4
content preserved, not reverted. Adds G7 post-release ops and
observation (monitoring, errors, cost/quota, feedback, hotfix loop,
rollback triggers, ops manual) and G8 iteration and scaling (v1.1
handover list, contract v1->v2 protocol, new-end onboarding,
scale/performance, deprecation), with G6->G7 and G7->G8 gates,
DAG, gate table, coordinator derivation, G0-G8 todos, checklist,
09-ops and 10-iteration role templates, and ops-manual / v1.1
handover templates in both body and appendix. Settles web+React+
multi-viewport focus, cross-viewport E2E, web single-end release
with iOS/Android native postposed, open-source-first, and the
TypeScript full-stack default (user-request-only language change;
codex-app-server itself stays Python). Pending #199 and #205
merge, then one rebase round. PR only, do not merge or deploy.
- P1-1 directory-isolation executable check: git diff --name-only
against merge-base, output recorded in merge record
- P1-2 derivation checklist table (8 required items, missing one blocks dispatch)
- P1-3 QA split into single-client review and cross-client E2E with
fixed deliverables and agent-test attribution
- P1-4 sub-task slim-pack trimming rules (listed chapters only, never full copies)
- P2: inter-thread Q&A default policy, docs/decisions.md append-only,
per-client onboarding docs (web first at G4a), v1.1 handover checklist
- Body and appendix kept in sync; new lightweight keyword test
- worker._tenant_dirs: playwright refresh log used undefined 'task';
use the method argument project_id (ruff F821, real bug: NameError
aborted tenant seeding when a playwright script was refreshed).
Regression: cloud/tests/test_worker_playwright_seed_f821.py (2 tests;
pre-fix run reproduces NameError at worker.py:2780).
- harness2.assign_ids: hashlib.sha1(..., usedforsecurity=False) -- the
digest is only a deterministic shuffle seed, not a security use;
digest value unchanged, bandit B324 no longer fires (High 5 -> 4).
- harness2.select_final: rename ambiguous loop var 'l' to 'lab'
(ruff E741, behaviour-invariant).
- Deliberately NOT changed: host.py shell=True x4 (see PR body -- all
four are intentional shell commands, changing semantics is unsafe),
remaining ruff 109 / mypy 24, workflow thresholds (75% coverage
gate unchanged; no gate was relaxed or removed).
Phase 1 of audit-hardening CI expansion (baseline origin/main aebc623):
- lint: ruff E9 hard gate; full E,F@120 report-only (baseline 111 items)
- typecheck: mypy default mode, production scope, report-only (baseline 24)
- security: bandit production scope, report-only (baseline 287, High 5)
- depscan: pip-audit on requirements-ci.txt, hard gate (0 vulns; repo has
no production dependency manifest, see summary)
- coverage: production 58 files, fail-under=75 (baseline measured 76%)
- release build: E9 + pip-audit hard gates before packaging; full reports
stay in host-verify to avoid drift
No production code changed; CI billing failure mode unchanged.
Goal tasks pushed back by the audit loop lost the packet preamble on
the next round: claim-time _apply_audit_loop_packet pasted it into the
in-memory prompt/goal, but the next round collided with the existing
host task name, fell back to breakpoint goal-resume (no --goal-text),
and the host reused the stored goal, discarding the preamble.
Fix (minimal, direction (c) trimmed): host goal-resume gains
--resume-note, accepted only by the breakpoint branch and prepended
to the first turn prompt of this resume only -- never written back
to the stored goal, never touching rounds or verification. The worker
stashes the same preamble as the memory-only _audit_loop_resume_note
at claim time and forwards it on breakpoint resume (Local/Docker
goal_resume_task and the popen goal-resume command, each guarded to
ignore the note when goal_text is set for a new-goal resume).
Accounting untouched: audit 3-round cap, goal_max_rounds/quota, and
DB goal columns are unchanged; non-goal pushback is unchanged.
Tests: cloud/tests/test_audit_n12.py (9 tests) -- host first-turn-only
note + stored-goal/rounds, worker exists-fallback end-to-end with
--resume-note, non-goal regression, goal-column accounting, and
Local/Docker/popen command shapes. Full cloud suite: 1760 OK
(skipped=11), baseline 1751 + 9 new; tests/test_host.py: 62 OK.
- I-3: new models.evaluate_capability returning a CapabilityVerdict
(ok/missing/unknown from check, known/unknown_missing from split,
status/verb derivations); decide_routing, worker gateway check,
worker persisted recheck, dry-run tier+gateway checks and server
shadow endpoint all call it. Blocking rules stay at the call
sites (gateway has no worker-default fallback, recheck blocks
only on known, dry-run/shadow never block); all user-visible
audit details, dry-run notes and shadow notes are byte-identical.
- I-4: new models.merge_policy_documents shared by
load_routing_policy and dry_run_from_env; the old tiers-update
block is kept verbatim (it was a no-op after update(): JSON
tiers replace the whole map, file tiers absent from JSON are
dropped -- preserved, not fixed, as that would be a behaviour
change). dropped_tiers/partial_failures remain load-time
records, not merge products.
- Tests: test_audit_batch3.py, 13 tests (verdict matrix + parity
with check/split, decide/dry-run wording, merge boundaries +
300 seeded random pairs + loader/dry-run agreement).
Full suite: 1694 OK (skipped=11), baseline 1681 + 13, 0 failures.
- I-1: evaluate_shadow counts dropped rows by cause (unparseable /
unknown_tier / explicit / duplicate) with the identity
audit_rows_total == routed_task_n + dropped_total; the shadow
endpoint passes created_at/id through and reports audit totals
next to sample counts.
- I-2 (option b): parse logs a warning naming each tier dropped for
an invalid mode/requires; load attaches partial_failures to the
policy; the worker records a task-level
worker_routing_policy_load_failed audit for partial failures
(same action as full failure, detail says failure=partial);
the shadow report carries the policy fingerprint, dropped tiers
and partial failures (no task context on that path, so no
task-level audit there).
- I-5: coordinator gate audit lookup failure now raises
_CoordGateLookupFailed and the merge converts it to escalate
(no requeue, no round consumed); persisted-model lookup failure
and required-tools check failure each write a named audit row
before falling through to dispatch/allow; audit-loop evaluate
failure stays escalate (regression test).
- Test fixtures: coordinator suites build audit_log via AuthStore
as production's first real audit does, since their stubbed
_audit never creates the table.
- B-A: migrations.py tasks backfill loop now types audit_loop as
INTEGER (other 6 columns stay TEXT), matching db.py backfill,
server/worker ensure lists; TEXT readback '0' is truthy and
would silently enable the audit loop for audit_loop=0 rows.
- B-C: gateway preparation failure early-return now calls
_running_clear(task_id) next to _cleanup_task_seed, matching
missing-key/repo/capability/tool gates; previously the capacity
slot leaked until worker exit (default capacity 1 blocks reclaim).
- Tests: cloud/tests/test_audit_batch1.py (3 tests): migration
ALTER types, INTEGER 0 readback not enabled via real worker
row mapping, gateway failure clears slot and capacity reclaim.
_claim doc 'three gates' list left unchanged (full early-return
gate enumeration not verified this batch; residual wording).
- No other audit items touched (I-1..I-14, N-1..N-12).
- M1: test_goal_branch_via_convert_queued_then_claim actually calls
server._goal_convert_queued on a queued auto-resumed task before
claiming, instead of only inserting the post-conversion state.
- M5: test_worker_routing_audit_rows_carry_no_fingerprint drives a
real run_one with an active policy and asserts worker_routing
audit rows exist and carry no fingerprint (negative control
verified: injecting fingerprint into the detail fails it).
- M2: 'all callers' wording tightened to the real counts
(load_routing_policy: 2 production call sites; evaluate_shadow: 1).
- M4: content_fingerprint docstring now states the fingerprint covers
the full normalized payload including fallback_models, which the
router deliberately does not apply (fail-safe direction).
No behaviour change; M3 DESIGN.md edits are outside the repo.
- db: translate INSERT OR IGNORE ... SELECT to ON CONFLICT DO NOTHING
(VALUES-only regex left SELECT form untranslated -> PG syntax error);
register task_audit_loop_state (task_id) in _OR_REPLACE_PK so PG
INSERT OR REPLACE becomes a real upsert instead of DO NOTHING.
- server: _backfill_default_quotas uses SAVEPOINT containment and no
longer commits the caller's transaction, so a backfill failure cannot
roll back init_db's migrate ALTERs.
- worker: _coord_child_rows rolls back before column-fallback retries so
a missing-column failure on PG does not cascade into
InFailedSqlTransaction.
- tests: test_pg_init_fix.py (translation, SQLite backfill/upsert,
real-PG fresh-init + merge + upsert + backfill containment, skipped
without CLOUD_TEST_PG_DSN/driver).
Parent audit loop sits after all children are aggregated and
before the parent final state; the single-task _audit_loop_wire
is never applied to parents or children. Parent gates are
per-child gates child:<name> (completed only); execution
failures map to one deterministic finding per child; subjective
findings are split off before evaluate and only audited as
needs_review. Gate-rejected children (capability/tool/approval
audit rows, or distinctive error markers) bypass evaluate and
escalate directly without consuming a round; tolerated, paused
and quota-missing parts are excluded, never findings. Rounds
are counted only in the parent task_audit_loop_state; on
pushback the packet is stored on the parent and split per failed
child into that child's audit_loop_packet column, consumed once
by the existing packet-injection path at next claim; pass and
escalate clear parent and child packets. POST /tasks now allows
audit_loop on coordinator parents (the old 400 rationale is
gone); children still take no external switch.
Audit-loop wiring, worker completion path only: after the final
write-back (and only when the final is completed/failed, not quota,
not cancelled, not goal-converted, and the task carries audit_loop),
collect this attempt's deterministic gate results (execution,
goal_verify only when host reported goal_met, capability /
required_tools / gateway as passed-or-inapplicable markers, since a
pre-dispatch gate rejection early-returns before any runner round
and requeueing it would deterministically re-fail the same gate)
and call audit_loop_evaluate. Pushback stores the packet JSON in
tasks.audit_loop_packet and CAS-requeues the task; the next claim
prepends the packet preamble to prompt (and goal) with a data-only
separator, DB prompt untouched. Pass clears the packet. Escalate
sets failed (VALID_STATUSES has no needs_review, never invented)
with an escalation error and clears the packet; subjective
needs_review leaves the outcome untouched and only audits.
New tasks.audit_loop / audit_loop_packet columns follow the
attempts DDL pattern in worker ensure, server migrate, PG schema,
PG backfill, migrations backfill, and POST /tasks accepts
audit_loop (True/1 only, coordinator tasks rejected like v9).
Coordinator parent tasks return via _run_coordinator before the
attempt loop and coordinator children never inherit audit_loop,
so coordinator wiring is left as legacy, unchanged.
Audit loop engine (cloud/audit_loop.py): Finding/PushbackPacket/LoopState pure functions; deterministic gates only, subjective -> needs_review; packet completeness enforced; standards-weakening -> escalate; regression and missing-gate checks enforce full re-run; MAX_PUSHBACKS=3, round>=3 -> escalate. Persistence in new single-row task_audit_loop_state (attempts DDL pattern: module CREATE_TABLE_SQL + db.py POSTGRES_SCHEMA + migrations list + server init + worker ensure); task_attempts NOT reused (attempt_no PK collision) and task_followups NOT reused (user messages). Worker.audit_loop_evaluate is the explicit opt-in hook (task audit_loop field only, no env switch, never called by run_one); escalate audits worker_audit_loop_escalated; corrupted/missing count fail-closes to escalate.
Parse: agent_tools/tool_approvals column parsing unified into agent_def shared helpers (parse None = undeclared; server keeps None, worker maps to []/{} at _task_from_row, dispatch behaviour unchanged). Goal: _apply_resume_preamble goal branch proven reachable via _goal_convert_queued on a queued auto-resumed task, kept with a test and corrected docs (not dead code). Fail-open: audit write failure and preamble lookup failure now log warnings (best-effort unchanged); partial policy failure and shadow parse drops documented as known gaps. M7: shadow endpoint explicit_capability_failures_truncated test. Fingerprint: RoutingPolicy.content_fingerprint (canonical JSON sha256, version excluded) surfaced only in dry-run output, never in audit rows. M1 wording: legacy gap stated as fixed stock persisting per-task until completion, not converging.
N1-1: writeback scope verified too broad (any chosen, incl. shadow
applied=False, worker_default and worker_explicit) and narrowed to
source=policy + applied only; writeback now records a
worker_model_persisted audit row as the identity marker.
N1: persisted models are rechecked against current tier requires
before dispatch on later attempts (fixes4 split known/unknown):
active + known-missing fails fast (worker_capability_recheck_
rejected, no runner), unknown-only and shadow only audit.
User-explicit, worker-CLI-explicit and gateway-filled tasks skip
the recheck; legacy models are recognised by a policy applied
worker_routing row. Recheck never rewrites tasks.model and never
re-decides.
Explicit gateway capability now splits known/unknown: known-missing
explicit models fail-fast under active (same shape as the gateway
default gate), unknown explicit models stay audit-only, shadow never
blocks; explicit priority now protects which model the user chose,
not that the chosen model must run. Shadow report reads
worker_capability_gateway_explicit rows as explicit_capability_failures
with a field note distinguishing the three explicit counts.
N2: coordinator requeue CAS rowcount=0 branch covered via a proxy
connection test (no refactor needed). N3: shadow endpoint test on
the create_server + real HTTP skeleton (capability, truncated, and
explicit-failure fields). N4: creation-layer rejection tests for
tools/instructions/tool_approvals on goal/coordinator tasks.
N5: model writeback failure logs a warning (best-effort kept).
N1: writeback capability-recheck trade-off documented in comment
only; recheck itself is a later batch.
B1: gateway capability check now triggers on walking the sub2api
gateway, not on the gateway having filled a model; task-level
explicit models are audited (worker_capability_gateway_explicit)
but never blocked, preserving explicit priority; gateway-default
models still fail-fast under active.
I1: dry-run adds a gateway-model section, sourced only from
CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active
tier whose requires the gateway model fails or cannot verify
blocks, shadow only warns.
I2: shadow review gate now counts completed samples
(completed >= 20), same basis for sample_shortfall; the endpoint
attaches per-tier capability status (read-only policy load) and
flags capability-failing tiers in the recommendation.
I3: configured-but-broken routing policy logs a warning in
load_routing_policy (unconfigured stays silent); the worker sets
routing_policy_load_failed and audits
worker_routing_policy_load_failed per task.
I4: resume preamble wraps prior output/error as data with an
explicit do-not-execute constraint; position verified in run_one
(tree -> preamble -> project -> original) and the docs corrected
from the false 'closest to the original prompt' claim.
I5: decided model is written back with a CAS UPDATE
(model IS NULL OR model=''), so later attempts reuse it;
explicit models are never overwritten.
I6: agent_* columns added to worker._ensure_result_columns and
db.init_postgres_schema backfill; migrations backfills them for
tasks before insert (sender_task_id pattern); column-level tests
for agent_tool_approvals and task_attempts.
I7: coordinator requeue adds AND status=<snapshot> (rowcount 0
skips requeue/audit) and filters paused_by_user / user_cancel /
user_pause children, which are never revived or cleared.
I8: explicit-model tasks (including coordinator children that
inherited a parent model) audit a source=explicit worker_routing
row; evaluate_shadow counts it as per-tier explicit_n only,
never routed_n, never a dedup slot.
M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation,
shadow sort doc fixed, truncated recommendation notes truncation,
seed cleanup on gateway/quota/goal-cap early returns, tools-only
coordinator definitions rejected, preamble goal branch marked
effectively unreachable, DESIGN.md drift corrected (outside repo).
M4/M7/M9: documentation/new-field only (pass_rate_fully_measured,
whitespace-model shadow-drop note, bound-note race note); old
field semantics unchanged.
- RoutingPolicy.fallback_models: move the cleaning loop into
__init__ (it was unreachable after tier_mode's return, so the
attribute never existed); still parsed but not applied.
- Capability fallback: RoutingDecision gains capability_fallback;
the worker audits worker_capability_fallback when it fires, so
a tier-model rejection followed by a worker-default fallback
is no longer invisible (source stays worker_default, no
model_routing event, so shadow samples are unaffected).
- Registry: resolve_registry_agent checks skills directly
against the loaded definition instead of re-parsing a
hand-built dict, so future fields cannot be silently dropped
the way tool_approvals was in v9.
- Shadow: evaluate_shadow sorts rows earliest-first by
created_at (id tiebreak) when timestamps are present, so
dedup no longer depends on caller row order; the server
path (which strips created_at) is unchanged.
- Gateway: when sub2api fills the default model, the worker
now checks it against the tier's capability requirements
with the same mode semantics as v8 -- shadow only audits,
active fails fast with worker_capability_gate_rejected and
never calls the runner. No worker-default fallback here:
the gateway model is the task's gateway model, falling back
to a worker default would run against the wrong gateway.
Agent definitions gain tool_approvals: known tool -> ask/auto/deny.
Unknown tool/policy is a 400, and required+deny is a 400
contradiction. The declaration is persisted (tasks.agent_tool_
approvals, PG + migrate), echoed in GET and the creation response
(via to_summary), and resolved for registry definitions -- fixing
a latent drop where resolve_registry_agent re-parsed without new
fields. At dispatch the worker audits the declaration and fails
fast on a defensive required+deny re-check (v3 tools-gate shape,
runner never called); goal/coordinator tasks reject the field at
creation like instructions. effective_tool_approvals maps a
declaration + host global approval to the actual host answer,
with ask_honoured always False: host answers elicitation with
one global switch and carries no tool identity, and the approvals
queue is record-only with no runtime ask channel, so per-tool
interception and credential scope are explicitly NOT claimed.
Policy carries an operator-declared model_capabilities directory
and per-tier requires (tool_calling/reasoning/vision booleans,
min_context_tokens). decide_routing checks the policy tier model
in active mode: capable applies, incapable falls back to a capable
worker default, and if both fail or are unknown it returns
capability_blocked, which the worker turns into a failed task
(v3 tools-gate shape, runner never called). Shadow mode only
annotates the reason, never blocks. Explicit task and worker CLI
models bypass the check by design. Invalid requires drops only
that tier (fail-closed, like invalid tier mode). Dry-run reports
per-tier capability status and blocks on active tiers that fail
or cannot be verified.
Production code had no golden tasks, scorer, or rubric
(harness2.py states this explicitly); routing shadow gives
cost/status only and admits counterfactual not_measured,
so 'is the suggested model actually better' had no
mechanism. This track adds a pure module + CLI that scores
already-completed task records against human-curated golden
tasks and compares two runs, without running tasks, spending
model cost, touching the DB, or activating anything.
Golden tasks: JSON {id, prompt, tier, checks}; only four
deterministic check types (contains, not_contains,
status_completed, max_tokens, max_cost_usd, the last in
cost.py microdollar units); invalid definitions fail loudly.
Scoring: per-task score = passed / measured checks;
cost checks with missing/unpriced data are unknown, excluded
from the denominator and reported separately (never counted
as pass or fail); empty result_text is a string-check fail,
not unknown. all_pass means 'all measured checks pass' and
is reported alongside fully_measured_n so a pass rate is
never misread as 'all checks passed'. Comparison: per-run
stats, paired delta over golden tasks present in BOTH runs
(unpaired listed separately, missing listed, unmatched
records excluded with a warning), duplicate task_id rejected.
Paired n < 20 (same threshold as shadow) => ready_for_
decision false, no significance or p-values computed.
Deliberately NOT included: a task runner (golden tasks
still need a separate run-and-record step or manual task
creation), an LLM judge or quality score, and any routing
activation suggestion. A contains check can be gamed by
echoing, so golden sets must be human-curated with checks
that require actually doing the work; the example file in
eval/ is format-only, not a decision-ready set. Tests:
test_eval_loop.py (8 tests).
Foundry 'platform restarts with original input' gap, after
reading the existing mechanisms: creation idempotency
(UNIQUE project_id+idempotency_key), goal checkpoints
(task_checkpoints per round, same-thread breakpoint resume
with no-progress fuse), merge/cost/notification CAS, and
coordinator child reuse already exist. The uncovered segment
is non-goal platform-interrupted tasks: auto_resume requeues
them fresh in the same worktree after clearing result/error/
interrupted_reason, so the new run gets only the original
prompt and cannot tell what the previous attempt had done.
This track adds, in worker/server layer only (no host.py or
codex thread change): task_attempts ledger, recorded by
auto_resume_interrupted in the same transaction before
clearing (attempt_no = old auto_resume_count + 1, kind
goal_breakpoint|platform_fresh, interrupted_reason,
checkpoint_round, result/error truncated to 2000 chars,
INSERT OR IGNORE keeps the first record on duplicate);
resume preamble injected in-memory only at claim for
platform_fresh reruns (previous attempt facts, truncated to
1000 chars, plus check-worktree-first and verify-before-
recreating-external-artifacts instructions; skipped for
coordinator/_resume/breakpoint tasks and when no record
exists, never fabricated); attempts surfaced read-only in
GET /tasks/{id}/timeline under existing project-read
permission. Fresh PG schema, migrations table list, and
server/worker ensure paths share one DDL.
Deliberately NOT claimed: the preamble is an instruction,
not a hard guarantee like creation idempotency or merge
CAS; the ledger does not track worktree files or external
artifacts (no worktree snapshot, executor-dependent paths;
hard dedup for those still rests on their existing gates);
manual retry paths do not go through auto_resume, so they
get no ledger record or preamble. Tests:
test_attempts.py (7 tests).
Coordinator parts gain three enforced bounds, validated in
validate_parts and carried through the coordinator_parts JSON
(no schema change): timeout_seconds (1..86400, per-attempt
wall clock from coordinator start; the coordinator poll loop
cancels only that child with the same cancel mechanism as
the global timeout, other parts keep waiting, audit
coordinator_child_bound_exceeded), on_failure fail|tolerate
(tolerated failures do not fail the coordinator, are named
in the report head, and their failed children are not
requeued on coordinator re-run while interrupted children
still are), and result_chars (0..2000, default 500, enforced
in build_report; the 4000-char report writeback cap still
applies). evaluate_outcome is the single outcome rule used
by both build_report and the worker error text, so tolerated
failures never leak into the coordinator failure reason.
Unknown part names fail closed.
Deliberately NOT added, after reading the execution layer:
per-part token/cost budget (cost is recorded only on
completion, so a running child's spend is unobservable and
creator quota is monthly, not per part), per-part round cap
(normal tasks have no per-task round cap in the worker),
and write-scope isolation (sandbox is per-project worktree,
a scope string in a prompt would be an instruction, not a
guarantee). Tests: test_delegation_bounds.py (8 tests).
resolve_registry_agent in cloud/agent_def.py: agent_name may be
<name> or <name>@<version>, agent_version may also be given
separately; both given must agree or 400. Unpinned loads
<name>.json (default); pinned loads <name>.<version>.json and
requires the file's own version field to match, with no silent
fallback to the default file. Version strings are restricted
to alphanumerics/._- (no path separators, no ..); the
agents-dir traversal guard is a second line of defence.
Server: agent_version is only for agent_name (inline agent
with agent_version is 400); stored agent_name is the base
name, not the @ form. Snapshot fixture:
agents/research-brief.v1.json (same content as the default
v1 file). Tests: test_agent_version.py (7 tests).
New module cloud/routing_dryrun.py: dry-run a merged policy
document or an env mapping (file first, JSON overrides whole
tier entries, CLOUD_ROUTING_MODE overrides global mode only,
never explicit tier modes), reporting per-tier effective
model/mode/mode-source, dropped tiers with reasons, absent and
unknown tiers, and would_activate/would_shadow/would_off.
Dropped or unknown tiers block (CLI exit 1); warnings alone
(no version, fallback_models parsed but not applied, failed
file/JSON sources that the loader silently ignores) do not
block but are always printed. Model names are NOT validated
against a catalog (deliberately not whitelisted) and sample
counts are NOT checked here (that is the shadow report's job);
the CLI never writes env, files, or worker state.
Tests: test_routing_dryrun.py (8 tests).
RoutingPolicy gains tier_modes: a tier entry may be a string
(model, mode inherits global) or {model, mode}. The explicit
tier mode overrides the global mode in both directions, so plan
can be active while goal/default stay shadow, and one tier can
be rolled back to off without touching the others. An invalid
tier mode drops that tier entirely (fail-closed: a typo must
never accidentally activate a tier, and never kills other
tiers). Tier-off tasks return worker_default and are not
audited as policy/shadow, so they never inflate shadow sample
counts. Explicit task and worker-CLI models still win over any
tier mode.
Tests: test_routing_tier.py (5 tests).
New pure module cloud/routing_shadow.py: parse worker_routing
audit details (old format without suggested model parses with
suggested unknown, never guessed), evaluate per-tier routed
samples with one-task-one-sample dedup, explicit tasks counted
separately, status under the chosen model, token/USD with
unpriced tokens kept apart, and a min-20 gate per tier. The
report explicitly marks the counterfactual not_measured:
shadow suggested models never ran, so no suggested-vs-chosen
cost/quality comparison is possible. Worker audit detail now
includes suggested/chosen for future rows. GET /routing-shadow
serves the report (project read permission, truncated flag
over 10000 audit rows, cost table ensured so a fresh DB does
not 500).
Tests: test_routing_shadow.py (5 tests).
Agent definitions gain tools (websearch/imagegen/playwright,
unknown tool is a 400), stored as agent_tools JSON and returned
by GET /tasks. This is deliberately required-tools, not an
allowlist: create_thread has no tool allowlist and tenant homes
are per-project shared, so blocking other tools cannot be
claimed. Instead the worker fails fast before any model call
when a declared tool has no [mcp_servers.<tool>] section in
the tenant codex-home config.toml (checked after _tenant_dirs
seeding, so a section seeded earlier by another worker counts
as present). The gate runs only for tenant-isolated Docker
workers; local/cloud workers skip it, documented, not verified.
Tests: test_agent_tools.py (8 tests).
Registry: POST /tasks accepts agent_name, loading <name>.json
from repo-root agents/ (traversal-guarded, skill-pack re-validated);
agent and agent_name together is a 400. Persistence: tasks gains
agent_name/agent_version/agent_instructions (migrate + PG schema),
returned by GET /tasks; stored prompt stays clean. Channel: local
and docker dispatch pass --developer-instructions to host.py
dispatch -> create_thread(developerInstructions=...); run/fork do
not re-send. Cloud executor has no such channel, so the worker
falls back to the v1 marked header in the in-memory dispatch
prompt only (DB prompt stays clean). Goal/coordinator tasks
still reject instructions.
Tests: test_agent_def_v2.py (7 tests) + updated v1 suite.
Agent definition is a versioned JSON document bundling the task
fields this codebase already honours (model/effort/approval/
executor/plan), plus instructions and required skills. At task
creation, explicit body fields win per field over the definition;
unresolved fields stay NULL and follow the worker-default chain
(and rule routing v0 when model is left empty).
Instructions reach the model only as a marked header prepended
to the stored prompt, because the dispatch path has no
developerInstructions channel; goal and coordinator tasks reject
instructions. Skills are validated against the cloud/skills pack
at parse time; seeding itself stays worker-global, documented
as the v1 boundary. Routing policy is not part of a definition
in v1 (worker-global), and unknown fields are rejected so typos
cannot be silently dropped. No new tasks-table column in v1;
definition identity is echoed in the creation response and via
the prompt header.
Tests: test_agent_def.py (11 tests).
Pre-dispatch routing only fills in tasks with no explicit model,
classified by structural signals only (plan > goal > default).
Explicit task model and worker CLI --model win over policy;
policy via CLOUD_ROUTING_POLICY_JSON/FILE + CLOUD_ROUTING_MODE.
Shadow mode records a suggestion without changing dispatch.
Only policy/shadow decisions emit a model_routing event, so the
existing events contract (fallback tests) is unchanged.
Fallback rules, effort/approval/sandbox/quota and provider are
untouched. Tests: test_models_routing.py (20 tests).
Long-running website monitoring: new cloud/resident.py layer on top of
the automations claim/quota model (no task state-machine change).
States running/paused/ended; pause preserves the content-hash cursor
and stops polling, resume re-checks from the cursor, end is terminal
(resume after end is 409, a new watch is required). Polling is a cheap
fetch + hash comparison; only a real change creates a queued model
task. Quota exhaustion pauses (quota_exceeded) instead of ending and
keeps the cursor. Driver is the worker run_loop round boundary;
server exposes create/list/get/update/pause/resume/end with audit.
Console UI intentionally not wired this round (see verification
summary for deviations).
- search(..., allowed_hosts=[...]) extends (never replaces) the 15-domain
code list for that call only; scope=general + allowed_hosts raises,
empty/illegal values raise before any HTTP, dot-boundary matching,
extension values never in error strings or attempted entries.
- Native-filter parity: Tavily/Exa get the effective list, Perplexity
skips its native filter past its 20-domain cap (post-filter stays
authoritative), Brave/SearXNG/DuckDuckGo rely on post-filtering.
- MCP web_search gains allowed_hosts passthrough (only when supplied,
so older search_fn fakes keep working).
- JS-heavy evaluation documented in docs/web-search-mcp.md, mock-first:
direct fetch is shell-only by construction; provider extract can
supply rendered text in mock only; live SearXNG in this VM env
failed (remote closed connection) and paid-backend live effects are
unvalidated blockers, not claimed.
- Tests: test_code_search_ext (9) + test_js_heavy_extract (5 + 1 live
skipped); existing websearch/code-search suites still green.
(cherry picked from commit 9c26bbe848438e1f34bcb000700aeb4c284f4f23)
- click/click_ref: pre-click allowlist gate on resolvable anchor href
(same check as navigate; miss refuses before the click executes);
click/click_ref/click_at: post-click landing URL check.
click_at has no pre-click gate by design (coordinates name no
element); JS-driven navigation, evaluate, meta refresh and popups
are documented as unintercepable with the Tinyproxy 403 backstop.
- Tenant script upgrade is now safe: only proven-unmodified scripts
(manifest match, 2a569ac known-shipped SHA-256, missing, or
current) auto-merge; tenant-modified scripts are never overwritten,
are reported as skipped, and get one deduped audit record. The
manifest never learns a tenant hash (else the next upgrade would
mistake a modified file for untouched). Idempotent reruns.
- Activation candidate = deployed Tinyproxy programming set encoded
with leading-dot entries (plain entries would regress
docs.python.org); npmjs.org per the deployed filter, not npmjs.com.
Host activation itself is NOT performed this round: only the plan,
validation tests and rollback are delivered.
Tests: 28 new (click 13, upgrade 8, activation 7), all fake/no-network;
98 browser-related tests pass including prior suites.
(cherry picked from commit e4a93f5748ec815f163d0bc096ff4dc4b8c651f5)
- Multi-tab: browser_new_page / browser_list_pages /
browser_switch_page / browser_close_page in playwright_mcp.py;
each new page gets a fresh Playwright context, so cookies and
storage are isolated between pages. Single-page tools act on
the current page; switching to a missing index is an error,
not a crash; closing the last page is an error (use
browser_close).
- Navigation allowlist: BROWSER_ALLOWED_HOSTS (comma list,
leading-dot suffix match, case/port-insensitive) checked in
_check_allowed_url before launching Chromium; unset (or empty)
means no check, the old behaviour. file:// always allowed;
localhost/127.0.0.1 have no special treatment.
- Close cleanup: browser_close closes all pages and best-effort
removes cwd browser-live.png / browser-live.json; custom
screenshot files are untouched. list_pages needs no launch.
- Egress filter: 10 programming-source Tinyproxy BASIC rules
(python.org, nodejs.org, go.dev, pkg.go.dev, rust-lang.org,
developer.mozilla.org, crates.io, stackoverflow.com,
cve.mitre.org, nvd.nist.gov); already-present github.com /
pypi.org / npmjs.org not duplicated; searx.diante.me stays
exact-only, no diante.me wildcard.
- Allowlist worker wiring: _docker_task_context now passes a
set, non-empty BROWSER_ALLOWED_HOSTS from the worker env into
[mcp_servers.playwright.env] in the tenant config, so the MCP
process actually sees it (container -e alone does not reach
MCP subprocesses, 2026-10-30). Unset/empty writes nothing and
never overwrites; the value never goes into logs; it is not
added to the websearch env block or the container env.
ensure_playwright_mcp_env now returns False without rewriting
when the result would be identical (canonical join: one blank
line before the block), so a second prep leaves an unchanged
tenant config alone; other sections untouched.
- Skill + docs: browser-automation SKILL.md multi-tab/allowlist
section; docs/platform-browser.md comparison doc (new file).
- Tests (mock/fake only, no network, no real Chromium, no real
credentials): cloud.tests.test_playwright +
cloud.tests.test_websearch_worker, 48 tests OK, including 6
new wiring tests (switch off passes no variable, switch on
reaches MCP env verbatim, empty never written, old-tenant
merge idempotent + no-rewrite, ensure no-op when identical,
allowlist enforces only after wiring).
- NOT done (see PR body): no persistent profile, no credential
vault, no approval card, no CAPTCHA bypass, in-page link
navigation does not pass the allowlist, no real-Chromium live
verification, host tinyproxy filter not reloaded.
- search(..., scope=code) restricts to 15 programming domains
(official docs, GitHub, registries, Stack Overflow, CVE/NVD):
native include_domains/search_domain_filter where supported
(Tavily, Exa, Perplexity), post-filtering for all providers
(Brave, SearXNG, DuckDuckGo have no native filter).
- web_fetch now uses fetch_best: Tavily extract, then Exa
contents, then direct fetch; both-fail errors name both
failures, direct-fetch 403 under the egress whitelist is
fail-closed, not bypassed. Default general scope and direct
fetch_url behaviour unchanged.
- MCP web_search gains scope enum (general/code); docs updated.
- Tests all mock HTTP, no real keys.
Deploy restarts kill running tasks: the worker marks them
interrupted (worker_shutdown on graceful exit, sweep on stale
reclaim), and auto_resume_interrupted used to accept goal tasks
only, so every non-goal task killed by a deploy stayed
interrupted forever unless a human retried it (the 2026-10-30
demo task died this way).
Now worker_shutdown/sweep interruptions are re-queued for
non-goal tasks too: the platform killed the task, so the
platform re-queues it. The re-queue is a fresh run, not a
breakpoint resume: goal_resume_pending is forced to 0 and any
stale resume_message is cleared, so the new run cannot be
hijacked by the goal-resume path or an old continue message;
files in the same worktree persist. All existing guards stay:
user pause, coordinator orphans, quota (still goal-only),
backoff, the 3-strike stall guard (non-goal tasks have no
checkpoints, so each platform re-queue counts as no-progress),
and the auto_resume_max cap. Other non-goal reasons (NULL,
quota, user cancel) still do not auto-resume.
- AutomationPanel rows now show title, prompt content (160-char preview with expand/collapse), repo_url, last/next run, run_once badge and cancelled time.
- Active automations get an inline edit form (name/prompt/interval/repo) that PATCHes only changed fields via the existing automation_update endpoint; backend semantics unchanged (no last_run_at reset, cancelled stays terminal).
- Rebuilt cloud/console.html with tsc + vite (build green); automation python tests 18/18 OK.
Stale bridge state used to end the stream on the first stale read,
so every round gap and every worker restart killed the watch (the
2026-10-30 demo watch died this way when a 15:03 deploy restarted
the worker). Now a stale state switches to a waiting heartbeat
every 5s while the task is still alive (running/queued/paused/
interrupted), and frames resume when the next round's MCP starts
polling. The stream ends only on a terminal task state, a missing
task row, or 10 minutes of continuous staleness. The panel shows
a waiting status and keeps the last frame. Also force-close the
SSE connection on stream end: HTTP/1.1 keep-alive otherwise left
terminal streams hanging with no EOF.
- cloud/tree.py: structured worktree tree (dirs first, ignore
.git/node_modules, symlinks not followed, entry/depth
truncation flags, text render for injection)
- GET /tasks/{id}/tree with /files auth + worktree resolution,
max_depth 1..5, max_entries 1..500, escape 400, missing 404,
cloud/cross-host 409
- MCP get_task_tree translating to the new endpoint
- worker first-round injection of a repo-checkout top-level
snapshot (2 layers, 120 entries) after repo resolution,
labelled as checkout snapshot not worktree; skipped for
coordinator/resume/fork, silent when no host source
- docs/project-tree.md; tests for tree, endpoint, injection,
MCP translation
Add cloud/skills/software-engineering/SKILL.md (686 lines, v0.2, appendix with 9 role templates travels inside SKILL.md because skill_seed only copies SKILL.md, not templates/). Update skill pack tests: EXPECTED set and real-pack list now include software-engineering, plus an explicit seed test asserting it lands at skills/software-engineering/SKILL.md, contains the appendix, and tenant edits are never overwritten. One-line wording tweak in the architect template avoids the existing secret-scan false positive on the substring sk- inside agent-task-contract.md; deployed file sha256 bd0436399ac05c6131b9281dad00973e50d13540e4c56ad21f21e2b5209bae8e (40759 bytes, 686 lines).
host