New pure module cloud/routing_shadow.py: parse worker_routing
audit details (old format without suggested model parses with
suggested unknown, never guessed), evaluate per-tier routed
samples with one-task-one-sample dedup, explicit tasks counted
separately, status under the chosen model, token/USD with
unpriced tokens kept apart, and a min-20 gate per tier. The
report explicitly marks the counterfactual not_measured:
shadow suggested models never ran, so no suggested-vs-chosen
cost/quality comparison is possible. Worker audit detail now
includes suggested/chosen for future rows. GET /routing-shadow
serves the report (project read permission, truncated flag
over 10000 audit rows, cost table ensured so a fresh DB does
not 500).
Tests: test_routing_shadow.py (5 tests).
Agent definitions gain tools (websearch/imagegen/playwright,
unknown tool is a 400), stored as agent_tools JSON and returned
by GET /tasks. This is deliberately required-tools, not an
allowlist: create_thread has no tool allowlist and tenant homes
are per-project shared, so blocking other tools cannot be
claimed. Instead the worker fails fast before any model call
when a declared tool has no [mcp_servers.<tool>] section in
the tenant codex-home config.toml (checked after _tenant_dirs
seeding, so a section seeded earlier by another worker counts
as present). The gate runs only for tenant-isolated Docker
workers; local/cloud workers skip it, documented, not verified.
Tests: test_agent_tools.py (8 tests).
Registry: POST /tasks accepts agent_name, loading <name>.json
from repo-root agents/ (traversal-guarded, skill-pack re-validated);
agent and agent_name together is a 400. Persistence: tasks gains
agent_name/agent_version/agent_instructions (migrate + PG schema),
returned by GET /tasks; stored prompt stays clean. Channel: local
and docker dispatch pass --developer-instructions to host.py
dispatch -> create_thread(developerInstructions=...); run/fork do
not re-send. Cloud executor has no such channel, so the worker
falls back to the v1 marked header in the in-memory dispatch
prompt only (DB prompt stays clean). Goal/coordinator tasks
still reject instructions.
Tests: test_agent_def_v2.py (7 tests) + updated v1 suite.
Agent definition is a versioned JSON document bundling the task
fields this codebase already honours (model/effort/approval/
executor/plan), plus instructions and required skills. At task
creation, explicit body fields win per field over the definition;
unresolved fields stay NULL and follow the worker-default chain
(and rule routing v0 when model is left empty).
Instructions reach the model only as a marked header prepended
to the stored prompt, because the dispatch path has no
developerInstructions channel; goal and coordinator tasks reject
instructions. Skills are validated against the cloud/skills pack
at parse time; seeding itself stays worker-global, documented
as the v1 boundary. Routing policy is not part of a definition
in v1 (worker-global), and unknown fields are rejected so typos
cannot be silently dropped. No new tasks-table column in v1;
definition identity is echoed in the creation response and via
the prompt header.
Tests: test_agent_def.py (11 tests).
Pre-dispatch routing only fills in tasks with no explicit model,
classified by structural signals only (plan > goal > default).
Explicit task model and worker CLI --model win over policy;
policy via CLOUD_ROUTING_POLICY_JSON/FILE + CLOUD_ROUTING_MODE.
Shadow mode records a suggestion without changing dispatch.
Only policy/shadow decisions emit a model_routing event, so the
existing events contract (fallback tests) is unchanged.
Fallback rules, effort/approval/sandbox/quota and provider are
untouched. Tests: test_models_routing.py (20 tests).
Long-running website monitoring: new cloud/resident.py layer on top of
the automations claim/quota model (no task state-machine change).
States running/paused/ended; pause preserves the content-hash cursor
and stops polling, resume re-checks from the cursor, end is terminal
(resume after end is 409, a new watch is required). Polling is a cheap
fetch + hash comparison; only a real change creates a queued model
task. Quota exhaustion pauses (quota_exceeded) instead of ending and
keeps the cursor. Driver is the worker run_loop round boundary;
server exposes create/list/get/update/pause/resume/end with audit.
Console UI intentionally not wired this round (see verification
summary for deviations).
- search(..., allowed_hosts=[...]) extends (never replaces) the 15-domain
code list for that call only; scope=general + allowed_hosts raises,
empty/illegal values raise before any HTTP, dot-boundary matching,
extension values never in error strings or attempted entries.
- Native-filter parity: Tavily/Exa get the effective list, Perplexity
skips its native filter past its 20-domain cap (post-filter stays
authoritative), Brave/SearXNG/DuckDuckGo rely on post-filtering.
- MCP web_search gains allowed_hosts passthrough (only when supplied,
so older search_fn fakes keep working).
- JS-heavy evaluation documented in docs/web-search-mcp.md, mock-first:
direct fetch is shell-only by construction; provider extract can
supply rendered text in mock only; live SearXNG in this VM env
failed (remote closed connection) and paid-backend live effects are
unvalidated blockers, not claimed.
- Tests: test_code_search_ext (9) + test_js_heavy_extract (5 + 1 live
skipped); existing websearch/code-search suites still green.
(cherry picked from commit 9c26bbe848438e1f34bcb000700aeb4c284f4f23)
- click/click_ref: pre-click allowlist gate on resolvable anchor href
(same check as navigate; miss refuses before the click executes);
click/click_ref/click_at: post-click landing URL check.
click_at has no pre-click gate by design (coordinates name no
element); JS-driven navigation, evaluate, meta refresh and popups
are documented as unintercepable with the Tinyproxy 403 backstop.
- Tenant script upgrade is now safe: only proven-unmodified scripts
(manifest match, 2a569ac known-shipped SHA-256, missing, or
current) auto-merge; tenant-modified scripts are never overwritten,
are reported as skipped, and get one deduped audit record. The
manifest never learns a tenant hash (else the next upgrade would
mistake a modified file for untouched). Idempotent reruns.
- Activation candidate = deployed Tinyproxy programming set encoded
with leading-dot entries (plain entries would regress
docs.python.org); npmjs.org per the deployed filter, not npmjs.com.
Host activation itself is NOT performed this round: only the plan,
validation tests and rollback are delivered.
Tests: 28 new (click 13, upgrade 8, activation 7), all fake/no-network;
98 browser-related tests pass including prior suites.
(cherry picked from commit e4a93f5748ec815f163d0bc096ff4dc4b8c651f5)
- Multi-tab: browser_new_page / browser_list_pages /
browser_switch_page / browser_close_page in playwright_mcp.py;
each new page gets a fresh Playwright context, so cookies and
storage are isolated between pages. Single-page tools act on
the current page; switching to a missing index is an error,
not a crash; closing the last page is an error (use
browser_close).
- Navigation allowlist: BROWSER_ALLOWED_HOSTS (comma list,
leading-dot suffix match, case/port-insensitive) checked in
_check_allowed_url before launching Chromium; unset (or empty)
means no check, the old behaviour. file:// always allowed;
localhost/127.0.0.1 have no special treatment.
- Close cleanup: browser_close closes all pages and best-effort
removes cwd browser-live.png / browser-live.json; custom
screenshot files are untouched. list_pages needs no launch.
- Egress filter: 10 programming-source Tinyproxy BASIC rules
(python.org, nodejs.org, go.dev, pkg.go.dev, rust-lang.org,
developer.mozilla.org, crates.io, stackoverflow.com,
cve.mitre.org, nvd.nist.gov); already-present github.com /
pypi.org / npmjs.org not duplicated; searx.diante.me stays
exact-only, no diante.me wildcard.
- Allowlist worker wiring: _docker_task_context now passes a
set, non-empty BROWSER_ALLOWED_HOSTS from the worker env into
[mcp_servers.playwright.env] in the tenant config, so the MCP
process actually sees it (container -e alone does not reach
MCP subprocesses, 2026-10-30). Unset/empty writes nothing and
never overwrites; the value never goes into logs; it is not
added to the websearch env block or the container env.
ensure_playwright_mcp_env now returns False without rewriting
when the result would be identical (canonical join: one blank
line before the block), so a second prep leaves an unchanged
tenant config alone; other sections untouched.
- Skill + docs: browser-automation SKILL.md multi-tab/allowlist
section; docs/platform-browser.md comparison doc (new file).
- Tests (mock/fake only, no network, no real Chromium, no real
credentials): cloud.tests.test_playwright +
cloud.tests.test_websearch_worker, 48 tests OK, including 6
new wiring tests (switch off passes no variable, switch on
reaches MCP env verbatim, empty never written, old-tenant
merge idempotent + no-rewrite, ensure no-op when identical,
allowlist enforces only after wiring).
- NOT done (see PR body): no persistent profile, no credential
vault, no approval card, no CAPTCHA bypass, in-page link
navigation does not pass the allowlist, no real-Chromium live
verification, host tinyproxy filter not reloaded.
- search(..., scope=code) restricts to 15 programming domains
(official docs, GitHub, registries, Stack Overflow, CVE/NVD):
native include_domains/search_domain_filter where supported
(Tavily, Exa, Perplexity), post-filtering for all providers
(Brave, SearXNG, DuckDuckGo have no native filter).
- web_fetch now uses fetch_best: Tavily extract, then Exa
contents, then direct fetch; both-fail errors name both
failures, direct-fetch 403 under the egress whitelist is
fail-closed, not bypassed. Default general scope and direct
fetch_url behaviour unchanged.
- MCP web_search gains scope enum (general/code); docs updated.
- Tests all mock HTTP, no real keys.
Deploy restarts kill running tasks: the worker marks them
interrupted (worker_shutdown on graceful exit, sweep on stale
reclaim), and auto_resume_interrupted used to accept goal tasks
only, so every non-goal task killed by a deploy stayed
interrupted forever unless a human retried it (the 2026-10-30
demo task died this way).
Now worker_shutdown/sweep interruptions are re-queued for
non-goal tasks too: the platform killed the task, so the
platform re-queues it. The re-queue is a fresh run, not a
breakpoint resume: goal_resume_pending is forced to 0 and any
stale resume_message is cleared, so the new run cannot be
hijacked by the goal-resume path or an old continue message;
files in the same worktree persist. All existing guards stay:
user pause, coordinator orphans, quota (still goal-only),
backoff, the 3-strike stall guard (non-goal tasks have no
checkpoints, so each platform re-queue counts as no-progress),
and the auto_resume_max cap. Other non-goal reasons (NULL,
quota, user cancel) still do not auto-resume.
- AutomationPanel rows now show title, prompt content (160-char preview with expand/collapse), repo_url, last/next run, run_once badge and cancelled time.
- Active automations get an inline edit form (name/prompt/interval/repo) that PATCHes only changed fields via the existing automation_update endpoint; backend semantics unchanged (no last_run_at reset, cancelled stays terminal).
- Rebuilt cloud/console.html with tsc + vite (build green); automation python tests 18/18 OK.
Stale bridge state used to end the stream on the first stale read,
so every round gap and every worker restart killed the watch (the
2026-10-30 demo watch died this way when a 15:03 deploy restarted
the worker). Now a stale state switches to a waiting heartbeat
every 5s while the task is still alive (running/queued/paused/
interrupted), and frames resume when the next round's MCP starts
polling. The stream ends only on a terminal task state, a missing
task row, or 10 minutes of continuous staleness. The panel shows
a waiting status and keeps the last frame. Also force-close the
SSE connection on stream end: HTTP/1.1 keep-alive otherwise left
terminal streams hanging with no EOF.
- cloud/tree.py: structured worktree tree (dirs first, ignore
.git/node_modules, symlinks not followed, entry/depth
truncation flags, text render for injection)
- GET /tasks/{id}/tree with /files auth + worktree resolution,
max_depth 1..5, max_entries 1..500, escape 400, missing 404,
cloud/cross-host 409
- MCP get_task_tree translating to the new endpoint
- worker first-round injection of a repo-checkout top-level
snapshot (2 layers, 120 entries) after repo resolution,
labelled as checkout snapshot not worktree; skipped for
coordinator/resume/fork, silent when no host source
- docs/project-tree.md; tests for tree, endpoint, injection,
MCP translation
- G0/G0.5: repo freeze items (new vs existing, owner/repo, default branch,
visibility); new-repo creation and first commit are G1 prerequisites.
- One branch per thread (feat/swe/<phase>-<role>-<short>), never push main
directly, never force-push main; sync main before PR, coordinator
resolves conflicts, never silently.
- Delivery requires a PR with scope, contract version, test evidence,
known issues and linked todos/PRD items; no PR number means not done.
- Merge gate: Checks green, directory ownership pass, frozen contract
version, coordinator approval, no unresolved review comments.
Branch protection (require PR, require Checks, block direct push) is a
GitHub repo setting; if unset, coordinator records it as risk and
enables it, never pretends it is protected. GET /tasks/{id}/pr/checks
is read-only diagnostics, never reruns or approves.
- Merge commit by default; merge record holds PR number, merge commit
hash, CI run link and executor; delete remote branch after merge.
- Per-client release tags (e.g. v0.1.0) from merged PRs; rollback is
revert or redeploy previous tag, never force-push rollback.
- Security: tokens/keys never in code, commits or command lines;
token only into subprocess env (cloud/pr.py GIT_TOKEN pattern);
.gitignore covers secrets and local config.
- Main text and appendix (coordinator, client, acceptance) updated in
sync; appendix travels inside SKILL.md for seeding.
Add cloud/skills/software-engineering/SKILL.md (686 lines, v0.2, appendix with 9 role templates travels inside SKILL.md because skill_seed only copies SKILL.md, not templates/). Update skill pack tests: EXPECTED set and real-pack list now include software-engineering, plus an explicit seed test asserting it lands at skills/software-engineering/SKILL.md, contains the appendix, and tenant edits are never overwritten. One-line wording tweak in the architect template avoids the existing secret-scan false positive on the substring sk- inside agent-task-contract.md; deployed file sha256 bd0436399ac05c6131b9281dad00973e50d13540e4c56ad21f21e2b5209bae8e (40759 bytes, 686 lines).
host
admin