Commit Graph
237 Commits
Author SHA1 Message Date
host 017f76010f feat: coordinator audit-loop wiring at merge point
Parent audit loop sits after all children are aggregated and
before the parent final state; the single-task _audit_loop_wire
is never applied to parents or children. Parent gates are
per-child gates child:<name> (completed only); execution
failures map to one deterministic finding per child; subjective
findings are split off before evaluate and only audited as
needs_review. Gate-rejected children (capability/tool/approval
audit rows, or distinctive error markers) bypass evaluate and
escalate directly without consuming a round; tolerated, paused
and quota-missing parts are excluded, never findings. Rounds
are counted only in the parent task_audit_loop_state; on
pushback the packet is stored on the parent and split per failed
child into that child's audit_loop_packet column, consumed once
by the existing packet-injection path at next claim; pass and
escalate clear parent and child packets. POST /tasks now allows
audit_loop on coordinator parents (the old 400 rationale is
gone); children still take no external switch.
2026-11-04 18:02:22 +00:00
host f2005689fd fix: wire audit loop into worker completion path (pushback delivery, escalate, packet preamble)
Audit-loop wiring, worker completion path only: after the final
write-back (and only when the final is completed/failed, not quota,
not cancelled, not goal-converted, and the task carries audit_loop),
collect this attempt's deterministic gate results (execution,
goal_verify only when host reported goal_met, capability /
required_tools / gateway as passed-or-inapplicable markers, since a
pre-dispatch gate rejection early-returns before any runner round
and requeueing it would deterministically re-fail the same gate)
and call audit_loop_evaluate. Pushback stores the packet JSON in
tasks.audit_loop_packet and CAS-requeues the task; the next claim
prepends the packet preamble to prompt (and goal) with a data-only
separator, DB prompt untouched. Pass clears the packet. Escalate
sets failed (VALID_STATUSES has no needs_review, never invented)
with an escalation error and clears the packet; subjective
needs_review leaves the outcome untouched and only audits.
New tasks.audit_loop / audit_loop_packet columns follow the
attempts DDL pattern in worker ensure, server migrate, PG schema,
PG backfill, migrations backfill, and POST /tasks accepts
audit_loop (True/1 only, coordinator tasks rejected like v9).
Coordinator parent tasks return via _run_coordinator before the
attempt loop and coordinator children never inherit audit_loop,
so coordinator wiring is left as legacy, unchanged.
2026-11-04 18:02:22 +00:00
host 37b1aadc02 fix: bounded audit loop (deterministic gates only, 3 pushbacks, escalate)
Audit loop engine (cloud/audit_loop.py): Finding/PushbackPacket/LoopState pure functions; deterministic gates only, subjective -> needs_review; packet completeness enforced; standards-weakening -> escalate; regression and missing-gate checks enforce full re-run; MAX_PUSHBACKS=3, round>=3 -> escalate. Persistence in new single-row task_audit_loop_state (attempts DDL pattern: module CREATE_TABLE_SQL + db.py POSTGRES_SCHEMA + migrations list + server init + worker ensure); task_attempts NOT reused (attempt_no PK collision) and task_followups NOT reused (user messages). Worker.audit_loop_evaluate is the explicit opt-in hook (task audit_loop field only, no env switch, never called by run_one); escalate audits worker_audit_loop_escalated; corrupted/missing count fail-closes to escalate.
2026-11-04 18:02:21 +00:00
host 1b8b6e525d fix: 9.5-code reachable items (parse unification, goal branch, fail-open logs, M7 test, fingerprint)
Parse: agent_tools/tool_approvals column parsing unified into agent_def shared helpers (parse None = undeclared; server keeps None, worker maps to []/{} at _task_from_row, dispatch behaviour unchanged). Goal: _apply_resume_preamble goal branch proven reachable via _goal_convert_queued on a queued auto-resumed task, kept with a test and corrected docs (not dead code). Fail-open: audit write failure and preamble lookup failure now log warnings (best-effort unchanged); partial policy failure and shadow parse drops documented as known gaps. M7: shadow endpoint explicit_capability_failures_truncated test. Fingerprint: RoutingPolicy.content_fingerprint (canonical JSON sha256, version excluded) surfaced only in dry-run output, never in audit rows. M1 wording: legacy gap stated as fixed stock persisting per-task until completion, not converging.
2026-11-04 18:02:21 +00:00
host d95d678379 fix: N1 persisted-model recheck (recheck + block)
N1-1: writeback scope verified too broad (any chosen, incl. shadow
applied=False, worker_default and worker_explicit) and narrowed to
source=policy + applied only; writeback now records a
worker_model_persisted audit row as the identity marker.
N1: persisted models are rechecked against current tier requires
before dispatch on later attempts (fixes4 split known/unknown):
active + known-missing fails fast (worker_capability_recheck_
rejected, no runner), unknown-only and shadow only audit.
User-explicit, worker-CLI-explicit and gateway-filled tasks skip
the recheck; legacy models are recognised by a policy applied
worker_routing row. Recheck never rewrites tasks.model and never
re-decides.
2026-11-04 18:02:21 +00:00
host 18e3f1c82d fix: third-audit gate findings (explicit split, shadow explicit failures, N2-N5, N1 note)
Explicit gateway capability now splits known/unknown: known-missing
explicit models fail-fast under active (same shape as the gateway
default gate), unknown explicit models stay audit-only, shadow never
blocks; explicit priority now protects which model the user chose,
not that the chosen model must run. Shadow report reads
worker_capability_gateway_explicit rows as explicit_capability_failures
with a field note distinguishing the three explicit counts.
N2: coordinator requeue CAS rowcount=0 branch covered via a proxy
connection test (no refactor needed). N3: shadow endpoint test on
the create_server + real HTTP skeleton (capability, truncated, and
explicit-failure fields). N4: creation-layer rejection tests for
tools/instructions/tool_approvals on goal/coordinator tasks.
N5: model writeback failure logs a warning (best-effort kept).
N1: writeback capability-recheck trade-off documented in comment
only; recheck itself is a later batch.
2026-11-04 18:02:21 +00:00
host 621c08d10f fix: second-audit gate findings (B1, I1-I8, M1-M10)
B1: gateway capability check now triggers on walking the sub2api
gateway, not on the gateway having filled a model; task-level
explicit models are audited (worker_capability_gateway_explicit)
but never blocked, preserving explicit priority; gateway-default
models still fail-fast under active.
I1: dry-run adds a gateway-model section, sourced only from
CLOUD_SUB2API_DEFAULT_MODEL (never queries the relay); an active
tier whose requires the gateway model fails or cannot verify
blocks, shadow only warns.
I2: shadow review gate now counts completed samples
(completed >= 20), same basis for sample_shortfall; the endpoint
attaches per-tier capability status (read-only policy load) and
flags capability-failing tiers in the recommendation.
I3: configured-but-broken routing policy logs a warning in
load_routing_policy (unconfigured stays silent); the worker sets
routing_policy_load_failed and audits
worker_routing_policy_load_failed per task.
I4: resume preamble wraps prior output/error as data with an
explicit do-not-execute constraint; position verified in run_one
(tree -> preamble -> project -> original) and the docs corrected
from the false 'closest to the original prompt' claim.
I5: decided model is written back with a CAS UPDATE
(model IS NULL OR model=''), so later attempts reuse it;
explicit models are never overwritten.
I6: agent_* columns added to worker._ensure_result_columns and
db.init_postgres_schema backfill; migrations backfills them for
tasks before insert (sender_task_id pattern); column-level tests
for agent_tool_approvals and task_attempts.
I7: coordinator requeue adds AND status=<snapshot> (rowcount 0
skips requeue/audit) and filters paused_by_user / user_cancel /
user_pause children, which are never revived or cleared.
I8: explicit-model tasks (including coordinator children that
inherited a parent model) audit a source=explicit worker_routing
row; evaluate_shadow counts it as per-tier explicit_n only,
never routed_n, never a dedup slot.
M1/M2/M3/M5/M6/M8/M10: dedup slot moved after tier validation,
shadow sort doc fixed, truncated recommendation notes truncation,
seed cleanup on gateway/quota/goal-cap early returns, tools-only
coordinator definitions rejected, preamble goal branch marked
effectively unreachable, DESIGN.md drift corrected (outside repo).
M4/M7/M9: documentation/new-field only (pass_rate_fully_measured,
whitespace-model shadow-drop note, bound-note race note); old
field semantics unchanged.
2026-11-04 18:02:21 +00:00
host f9a77577c5 fix: audit findings (fallback dead code, silent fallback, registry re-parse, shadow sort, gateway capability)
- RoutingPolicy.fallback_models: move the cleaning loop into
  __init__ (it was unreachable after tier_mode's return, so the
  attribute never existed); still parsed but not applied.
- Capability fallback: RoutingDecision gains capability_fallback;
  the worker audits worker_capability_fallback when it fires, so
  a tier-model rejection followed by a worker-default fallback
  is no longer invisible (source stays worker_default, no
  model_routing event, so shadow samples are unaffected).
- Registry: resolve_registry_agent checks skills directly
  against the loaded definition instead of re-parsing a
  hand-built dict, so future fields cannot be silently dropped
  the way tool_approvals was in v9.
- Shadow: evaluate_shadow sorts rows earliest-first by
  created_at (id tiebreak) when timestamps are present, so
  dedup no longer depends on caller row order; the server
  path (which strips created_at) is unchanged.
- Gateway: when sub2api fills the default model, the worker
  now checks it against the tier's capability requirements
  with the same mode semantics as v8 -- shadow only audits,
  active fails fast with worker_capability_gate_rejected and
  never calls the runner. No worker-default fallback here:
  the gateway model is the task's gateway model, falling back
  to a worker default would run against the wrong gateway.
2026-11-04 18:02:21 +00:00
host a51919f24b feat: tool approval declarations (v9)
Agent definitions gain tool_approvals: known tool -> ask/auto/deny.
Unknown tool/policy is a 400, and required+deny is a 400
contradiction. The declaration is persisted (tasks.agent_tool_
approvals, PG + migrate), echoed in GET and the creation response
(via to_summary), and resolved for registry definitions -- fixing
a latent drop where resolve_registry_agent re-parsed without new
fields. At dispatch the worker audits the declaration and fails
fast on a defensive required+deny re-check (v3 tools-gate shape,
runner never called); goal/coordinator tasks reject the field at
creation like instructions. effective_tool_approvals maps a
declaration + host global approval to the actual host answer,
with ask_honoured always False: host answers elicitation with
one global switch and carries no tool identity, and the approvals
queue is record-only with no runtime ask channel, so per-tool
interception and credential scope are explicitly NOT claimed.
2026-11-04 18:02:20 +00:00
host cd33a929ca feat: capability-based routing (v8)
Policy carries an operator-declared model_capabilities directory
and per-tier requires (tool_calling/reasoning/vision booleans,
min_context_tokens). decide_routing checks the policy tier model
in active mode: capable applies, incapable falls back to a capable
worker default, and if both fail or are unknown it returns
capability_blocked, which the worker turns into a failed task
(v3 tools-gate shape, runner never called). Shadow mode only
annotates the reason, never blocks. Explicit task and worker CLI
models bypass the check by design. Invalid requires drops only
that tier (fail-closed, like invalid tier mode). Dry-run reports
per-tier capability status and blocks on active tiers that fail
or cannot be verified.
2026-11-04 18:02:20 +00:00
host 362dffba4f feat: evaluation loop (v7, deterministic scoring only)
Production code had no golden tasks, scorer, or rubric
(harness2.py states this explicitly); routing shadow gives
cost/status only and admits counterfactual not_measured,
so 'is the suggested model actually better' had no
mechanism. This track adds a pure module + CLI that scores
already-completed task records against human-curated golden
tasks and compares two runs, without running tasks, spending
model cost, touching the DB, or activating anything.

Golden tasks: JSON {id, prompt, tier, checks}; only four
deterministic check types (contains, not_contains,
status_completed, max_tokens, max_cost_usd, the last in
cost.py microdollar units); invalid definitions fail loudly.
Scoring: per-task score = passed / measured checks;
cost checks with missing/unpriced data are unknown, excluded
from the denominator and reported separately (never counted
as pass or fail); empty result_text is a string-check fail,
not unknown. all_pass means 'all measured checks pass' and
is reported alongside fully_measured_n so a pass rate is
never misread as 'all checks passed'. Comparison: per-run
stats, paired delta over golden tasks present in BOTH runs
(unpaired listed separately, missing listed, unmatched
records excluded with a warning), duplicate task_id rejected.
Paired n < 20 (same threshold as shadow) => ready_for_
decision false, no significance or p-values computed.

Deliberately NOT included: a task runner (golden tasks
still need a separate run-and-record step or manual task
creation), an LLM judge or quality score, and any routing
activation suggestion. A contains check can be gamed by
echoing, so golden sets must be human-curated with checks
that require actually doing the work; the example file in
eval/ is format-only, not a decision-ready set. Tests:
test_eval_loop.py (8 tests).
2026-11-04 18:02:20 +00:00
host 19fd326a73 feat: attempt ledger + resume preamble (v6)
Foundry 'platform restarts with original input' gap, after
reading the existing mechanisms: creation idempotency
(UNIQUE project_id+idempotency_key), goal checkpoints
(task_checkpoints per round, same-thread breakpoint resume
with no-progress fuse), merge/cost/notification CAS, and
coordinator child reuse already exist. The uncovered segment
is non-goal platform-interrupted tasks: auto_resume requeues
them fresh in the same worktree after clearing result/error/
interrupted_reason, so the new run gets only the original
prompt and cannot tell what the previous attempt had done.

This track adds, in worker/server layer only (no host.py or
codex thread change): task_attempts ledger, recorded by
auto_resume_interrupted in the same transaction before
clearing (attempt_no = old auto_resume_count + 1, kind
goal_breakpoint|platform_fresh, interrupted_reason,
checkpoint_round, result/error truncated to 2000 chars,
INSERT OR IGNORE keeps the first record on duplicate);
resume preamble injected in-memory only at claim for
platform_fresh reruns (previous attempt facts, truncated to
1000 chars, plus check-worktree-first and verify-before-
recreating-external-artifacts instructions; skipped for
coordinator/_resume/breakpoint tasks and when no record
exists, never fabricated); attempts surfaced read-only in
GET /tasks/{id}/timeline under existing project-read
permission. Fresh PG schema, migrations table list, and
server/worker ensure paths share one DDL.

Deliberately NOT claimed: the preamble is an instruction,
not a hard guarantee like creation idempotency or merge
CAS; the ledger does not track worktree files or external
artifacts (no worktree snapshot, executor-dependent paths;
hard dedup for those still rests on their existing gates);
manual retry paths do not go through auto_resume, so they
get no ledger record or preamble. Tests:
test_attempts.py (7 tests).
2026-11-04 18:02:10 +00:00
host c15c224570 feat: bounded subagent delegation (v5)
Coordinator parts gain three enforced bounds, validated in
validate_parts and carried through the coordinator_parts JSON
(no schema change): timeout_seconds (1..86400, per-attempt
wall clock from coordinator start; the coordinator poll loop
cancels only that child with the same cancel mechanism as
the global timeout, other parts keep waiting, audit
coordinator_child_bound_exceeded), on_failure fail|tolerate
(tolerated failures do not fail the coordinator, are named
in the report head, and their failed children are not
requeued on coordinator re-run while interrupted children
still are), and result_chars (0..2000, default 500, enforced
in build_report; the 4000-char report writeback cap still
applies). evaluate_outcome is the single outcome rule used
by both build_report and the worker error text, so tolerated
failures never leak into the coordinator failure reason.
Unknown part names fail closed.

Deliberately NOT added, after reading the execution layer:
per-part token/cost budget (cost is recorded only on
completion, so a running child's spend is unobservable and
creator quota is monthly, not per part), per-part round cap
(normal tasks have no per-task round cap in the worker),
and write-scope isolation (sandbox is per-project worktree,
a scope string in a prompt would be an instruction, not a
guarantee). Tests: test_delegation_bounds.py (8 tests).
2026-11-04 18:01:44 +00:00
host 7f60f12639 feat: agent version pinning (v4)
resolve_registry_agent in cloud/agent_def.py: agent_name may be
<name> or <name>@<version>, agent_version may also be given
separately; both given must agree or 400. Unpinned loads
<name>.json (default); pinned loads <name>.<version>.json and
requires the file's own version field to match, with no silent
fallback to the default file. Version strings are restricted
to alphanumerics/._- (no path separators, no ..); the
agents-dir traversal guard is a second line of defence.
Server: agent_version is only for agent_name (inline agent
with agent_version is 400); stored agent_name is the base
name, not the @ form. Snapshot fixture:
agents/research-brief.v1.json (same content as the default
v1 file). Tests: test_agent_version.py (7 tests).
2026-11-04 18:01:44 +00:00
host 8fe4ad2f38 feat: routing policy dry-run (v0d, never applies)
New module cloud/routing_dryrun.py: dry-run a merged policy
document or an env mapping (file first, JSON overrides whole
tier entries, CLOUD_ROUTING_MODE overrides global mode only,
never explicit tier modes), reporting per-tier effective
model/mode/mode-source, dropped tiers with reasons, absent and
unknown tiers, and would_activate/would_shadow/would_off.
Dropped or unknown tiers block (CLI exit 1); warnings alone
(no version, fallback_models parsed but not applied, failed
file/JSON sources that the loader silently ignores) do not
block but are always printed. Model names are NOT validated
against a catalog (deliberately not whitelisted) and sample
counts are NOT checked here (that is the shadow report's job);
the CLI never writes env, files, or worker state.

Tests: test_routing_dryrun.py (8 tests).
2026-11-04 18:01:44 +00:00
host 38357c472d feat: per-tier routing activation (v0c)
RoutingPolicy gains tier_modes: a tier entry may be a string
(model, mode inherits global) or {model, mode}. The explicit
tier mode overrides the global mode in both directions, so plan
can be active while goal/default stay shadow, and one tier can
be rolled back to off without touching the others. An invalid
tier mode drops that tier entirely (fail-closed: a typo must
never accidentally activate a tier, and never kills other
tiers). Tier-off tasks return worker_default and are not
audited as policy/shadow, so they never inflate shadow sample
counts. Explicit task and worker-CLI models still win over any
tier mode.

Tests: test_routing_tier.py (5 tests).
2026-11-04 18:01:44 +00:00
host bac2a39377 feat: routing shadow evaluation v0b (read-only, never auto-activates)
New pure module cloud/routing_shadow.py: parse worker_routing
audit details (old format without suggested model parses with
suggested unknown, never guessed), evaluate per-tier routed
samples with one-task-one-sample dedup, explicit tasks counted
separately, status under the chosen model, token/USD with
unpriced tokens kept apart, and a min-20 gate per tier. The
report explicitly marks the counterfactual not_measured:
shadow suggested models never ran, so no suggested-vs-chosen
cost/quality comparison is possible. Worker audit detail now
includes suggested/chosen for future rows. GET /routing-shadow
serves the report (project read permission, truncated flag
over 10000 audit rows, cost table ensured so a fresh DB does
not 500).

Tests: test_routing_shadow.py (5 tests).
2026-11-04 18:01:44 +00:00
host fd37505c48 feat: agent tools as required tools with tenant tool gate
Agent definitions gain tools (websearch/imagegen/playwright,
unknown tool is a 400), stored as agent_tools JSON and returned
by GET /tasks. This is deliberately required-tools, not an
allowlist: create_thread has no tool allowlist and tenant homes
are per-project shared, so blocking other tools cannot be
claimed. Instead the worker fails fast before any model call
when a declared tool has no [mcp_servers.<tool>] section in
the tenant codex-home config.toml (checked after _tenant_dirs
seeding, so a section seeded earlier by another worker counts
as present). The gate runs only for tenant-isolated Docker
workers; local/cloud workers skip it, documented, not verified.

Tests: test_agent_tools.py (8 tests).
2026-11-04 18:01:44 +00:00
host 1672ab9ff2 feat: agent definition v2 (registry, persistence, developerInstructions channel)
Registry: POST /tasks accepts agent_name, loading <name>.json
from repo-root agents/ (traversal-guarded, skill-pack re-validated);
agent and agent_name together is a 400. Persistence: tasks gains
agent_name/agent_version/agent_instructions (migrate + PG schema),
returned by GET /tasks; stored prompt stays clean. Channel: local
and docker dispatch pass --developer-instructions to host.py
dispatch -> create_thread(developerInstructions=...); run/fork do
not re-send. Cloud executor has no such channel, so the worker
falls back to the v1 marked header in the in-memory dispatch
prompt only (DB prompt stays clean). Goal/coordinator tasks
still reject instructions.

Tests: test_agent_def_v2.py (7 tests) + updated v1 suite.
2026-11-04 18:01:44 +00:00
host 7bb3c37a10 feat: declarative agent definition v1 (inline agent on task creation)
Agent definition is a versioned JSON document bundling the task
fields this codebase already honours (model/effort/approval/
executor/plan), plus instructions and required skills. At task
creation, explicit body fields win per field over the definition;
unresolved fields stay NULL and follow the worker-default chain
(and rule routing v0 when model is left empty).

Instructions reach the model only as a marked header prepended
to the stored prompt, because the dispatch path has no
developerInstructions channel; goal and coordinator tasks reject
instructions. Skills are validated against the cloud/skills pack
at parse time; seeding itself stays worker-global, documented
as the v1 boundary. Routing policy is not part of a definition
in v1 (worker-global), and unknown fields are rejected so typos
cannot be silently dropped. No new tasks-table column in v1;
definition identity is echoed in the creation response and via
the prompt header.

Tests: test_agent_def.py (11 tests).
2026-11-04 18:01:44 +00:00
host e364970b46 feat: rule routing v0 for model selection (shadow/active, fail-closed)
Pre-dispatch routing only fills in tasks with no explicit model,
classified by structural signals only (plan > goal > default).
Explicit task model and worker CLI --model win over policy;
policy via CLOUD_ROUTING_POLICY_JSON/FILE + CLOUD_ROUTING_MODE.
Shadow mode records a suggestion without changing dispatch.
Only policy/shadow decisions emit a model_routing event, so the
existing events contract (fallback tests) is unchanged.
Fallback rules, effort/approval/sandbox/quota and provider are
untouched. Tests: test_models_routing.py (20 tests).
2026-11-04 18:01:44 +00:00
muse fc20296847 CI: sleep BEFORE write_control in playwright race test (sleep after write does not change file mtime, so previous fix could not work; runner mtime lags wall clock ~1ms)
ci / test (push) Failing after 16m8s
2026-11-04 17:30:07 +01:00
muse c1cb3f3d82 CI: tolerate runner mtime lag and stream heartbeat race in 2 browser tests (test-only, no production code change; playwright: 0.05s sleep after write_control, runner mtime lags wall clock 0.2-1.0ms; stream: drain waiting heartbeats until EOF after terminal status)
ci / test (push) Failing after 16m9s
2026-11-04 17:11:23 +01:00
host b391fe9320 resident P0+P1: DNS-level SSRF pin, batch budget, daily cap/cooldown/retention, PG schema/migration, dual-connection deadlock isolation, retarget baseline reset, proxy and structured failures, metrics
- DNS 级 SSRF pin:解析后 pin IP 建连,防 DNS 重绑定
- 批量总预算与每日上限/抖动冷却/保留期
- PG schema 与迁移清单
- 双连接死锁隔离
- 改目标重置基线
- 显式代理配置与结构化失败原因
- /metrics resident 口径
2026-11-02 03:30:57 +00:00
host 6525bcafd0 audit: resident website loop architecture and code quality to 10/10
Fix F1 name race with partial unique index, F2 pause/end mid-poll race with guarded updates, F3 redirect/numeric-IP SSRF, F4 quota prefix check, F5 duplicate row_get, F6 audit ternary; S1 unused import, S2 backoff doc, S3 defensive tearDown. Directed 20 pass, full 1700 pass.
2026-11-01 19:58:03 +00:00
host 561b3891fb test: resident website loop end-to-end with real fetch 2026-11-01 19:42:40 +00:00
host 0722ca3fd8 feat: resident watch tasks with pause/resume/end
Long-running website monitoring: new cloud/resident.py layer on top of
the automations claim/quota model (no task state-machine change).
States running/paused/ended; pause preserves the content-hash cursor
and stops polling, resume re-checks from the cursor, end is terminal
(resume after end is 409, a new watch is required). Polling is a cheap
fetch + hash comparison; only a real change creates a queued model
task. Quota exhaustion pauses (quota_exceeded) instead of ending and
keeps the cursor. Driver is the worker run_loop round boundary;
server exposes create/list/get/update/pause/resume/end with audit.
Console UI intentionally not wired this round (see verification
summary for deviations).
2026-11-01 19:01:39 +00:00
host 3a56f83114 websearch: 默认 auto 付费冻结,SearXNG 优先 + 显式解冻开关
- 默认 auto 只走免费链 searxng -> duckduckgo;即使环境里有
  付费 key 也不调用 Perplexity/Exa/Tavily/Brave,SearXNG
  失败也不回落付费,默认使用零付费花费。
- 付费后端代码/测试/文档全部保留;显式 pin(tavily/exa/
  perplexity/brave)仍照常调用,是有意覆盖不是默认。
- 解冻开关 WEBSEARCH_ENABLE_PAID,仅精确值 "1" 才恢复旧链
  (perplexity/exa/tavily/brave/searxng/duckduckgo),默认关;
  开关经 worker search_env 透传进 MCP config env。
- HttpClient 超时/DNS 失败改报显式错误(timeout after Ns /
  URL error),绝不把超时写成“无结果”。
- SearXNG 默认参数核实:代码无 SEARXNG_URL 硬编码默认值;
  SEARXNG_ENGINES 默认 bing,360search 已在基线;WEBSEARCH_TIMEOUT
  默认 20 秒。主机 env 实测仅 SEARXNG_URL=http://searx.diante.me
  一行,故默认 pin 生效;文档 https 漂移改回 http(主机 nginx
  仅听 80、无 443 vhost,https 在 TLS 层即失败)。
- 旧测试更新原因:test_websearch_mcp 原断言默认 auto 为
  perplexity 优先,与新冻结语义直接冲突,故改为断言冻结后
  的 searxng -> duckduckgo 回退 + 不泄漏 key;旧 perplexity ->
  exa 回退断言移至解冻开关测试(WEBSEARCH_ENABLE_PAID=1 下)。
  未删断言、未加 skip、未放宽门禁。
- 文档另删 WEBSEARCH_PROVIDERS 说法:代码从未实现该变量。
- 未推、未建 PR、未合并、未部署、未改主机 env/nginx/filter/
  证书、未做付费后端 live 验证、未创建生产任务。
2026-11-01 14:15:54 +00:00
host e9fa43dd8b websearch: task-level allowed_hosts extension for scope=code + JS-heavy extract evaluation (mock-first)
- search(..., allowed_hosts=[...]) extends (never replaces) the 15-domain
  code list for that call only; scope=general + allowed_hosts raises,
  empty/illegal values raise before any HTTP, dot-boundary matching,
  extension values never in error strings or attempted entries.
- Native-filter parity: Tavily/Exa get the effective list, Perplexity
  skips its native filter past its 20-domain cap (post-filter stays
  authoritative), Brave/SearXNG/DuckDuckGo rely on post-filtering.
- MCP web_search gains allowed_hosts passthrough (only when supplied,
  so older search_fn fakes keep working).
- JS-heavy evaluation documented in docs/web-search-mcp.md, mock-first:
  direct fetch is shell-only by construction; provider extract can
  supply rendered text in mock only; live SearXNG in this VM env
  failed (remote closed connection) and paid-backend live effects are
  unvalidated blockers, not claimed.
- Tests: test_code_search_ext (9) + test_js_heavy_extract (5 + 1 live
  skipped); existing websearch/code-search suites still green.

(cherry picked from commit 9c26bbe848438e1f34bcb000700aeb4c284f4f23)
2026-11-01 09:06:28 +00:00
host 125b436ab3 browser: click-path allowlist, safe tenant script upgrade, allowlist activation plan
- click/click_ref: pre-click allowlist gate on resolvable anchor href
  (same check as navigate; miss refuses before the click executes);
  click/click_ref/click_at: post-click landing URL check.
  click_at has no pre-click gate by design (coordinates name no
  element); JS-driven navigation, evaluate, meta refresh and popups
  are documented as unintercepable with the Tinyproxy 403 backstop.
- Tenant script upgrade is now safe: only proven-unmodified scripts
  (manifest match, 2a569ac known-shipped SHA-256, missing, or
  current) auto-merge; tenant-modified scripts are never overwritten,
  are reported as skipped, and get one deduped audit record. The
  manifest never learns a tenant hash (else the next upgrade would
  mistake a modified file for untouched). Idempotent reruns.
- Activation candidate = deployed Tinyproxy programming set encoded
  with leading-dot entries (plain entries would regress
  docs.python.org); npmjs.org per the deployed filter, not npmjs.com.
  Host activation itself is NOT performed this round: only the plan,
  validation tests and rollback are delivered.
Tests: 28 new (click 13, upgrade 8, activation 7), all fake/no-network;
98 browser-related tests pass including prior suites.

(cherry picked from commit e4a93f5748ec815f163d0bc096ff4dc4b8c651f5)
2026-11-01 09:06:28 +00:00
openqj 2a569ac6e3 Merge pull request #272 from openqj/feat-platform-browser
feat: platform browser multi-tab, navigation allowlist, close cleanup, egress filter, allowlist worker wiring
2026-11-01 16:42:59 +08:00
host da7a9950fb feat: platform browser multi-tab, navigation allowlist, close cleanup, egress filter, allowlist worker wiring
- Multi-tab: browser_new_page / browser_list_pages /
  browser_switch_page / browser_close_page in playwright_mcp.py;
  each new page gets a fresh Playwright context, so cookies and
  storage are isolated between pages. Single-page tools act on
  the current page; switching to a missing index is an error,
  not a crash; closing the last page is an error (use
  browser_close).
- Navigation allowlist: BROWSER_ALLOWED_HOSTS (comma list,
  leading-dot suffix match, case/port-insensitive) checked in
  _check_allowed_url before launching Chromium; unset (or empty)
  means no check, the old behaviour. file:// always allowed;
  localhost/127.0.0.1 have no special treatment.
- Close cleanup: browser_close closes all pages and best-effort
  removes cwd browser-live.png / browser-live.json; custom
  screenshot files are untouched. list_pages needs no launch.
- Egress filter: 10 programming-source Tinyproxy BASIC rules
  (python.org, nodejs.org, go.dev, pkg.go.dev, rust-lang.org,
  developer.mozilla.org, crates.io, stackoverflow.com,
  cve.mitre.org, nvd.nist.gov); already-present github.com /
  pypi.org / npmjs.org not duplicated; searx.diante.me stays
  exact-only, no diante.me wildcard.
- Allowlist worker wiring: _docker_task_context now passes a
  set, non-empty BROWSER_ALLOWED_HOSTS from the worker env into
  [mcp_servers.playwright.env] in the tenant config, so the MCP
  process actually sees it (container -e alone does not reach
  MCP subprocesses, 2026-10-30). Unset/empty writes nothing and
  never overwrites; the value never goes into logs; it is not
  added to the websearch env block or the container env.
  ensure_playwright_mcp_env now returns False without rewriting
  when the result would be identical (canonical join: one blank
  line before the block), so a second prep leaves an unchanged
  tenant config alone; other sections untouched.
- Skill + docs: browser-automation SKILL.md multi-tab/allowlist
  section; docs/platform-browser.md comparison doc (new file).
- Tests (mock/fake only, no network, no real Chromium, no real
  credentials): cloud.tests.test_playwright +
  cloud.tests.test_websearch_worker, 48 tests OK, including 6
  new wiring tests (switch off passes no variable, switch on
  reaches MCP env verbatim, empty never written, old-tenant
  merge idempotent + no-rewrite, ensure no-op when identical,
  allowlist enforces only after wiring).
- NOT done (see PR body): no persistent profile, no credential
  vault, no approval card, no CAPTCHA bypass, in-page link
  navigation does not pass the allowlist, no real-Chromium live
  verification, host tinyproxy filter not reloaded.
2026-11-01 08:34:13 +00:00
host 397a46f850 websearch: add scope=code programming search + provider extract chain
- search(..., scope=code) restricts to 15 programming domains
  (official docs, GitHub, registries, Stack Overflow, CVE/NVD):
  native include_domains/search_domain_filter where supported
  (Tavily, Exa, Perplexity), post-filtering for all providers
  (Brave, SearXNG, DuckDuckGo have no native filter).
- web_fetch now uses fetch_best: Tavily extract, then Exa
  contents, then direct fetch; both-fail errors name both
  failures, direct-fetch 403 under the egress whitelist is
  fail-closed, not bypassed. Default general scope and direct
  fetch_url behaviour unchanged.
- MCP web_search gains scope enum (general/code); docs updated.
- Tests all mock HTTP, no real keys.
2026-11-01 08:16:48 +00:00
host 4428a721a6 feat: auto-resume platform-interrupted non-goal tasks
Deploy restarts kill running tasks: the worker marks them
interrupted (worker_shutdown on graceful exit, sweep on stale
reclaim), and auto_resume_interrupted used to accept goal tasks
only, so every non-goal task killed by a deploy stayed
interrupted forever unless a human retried it (the 2026-10-30
demo task died this way).

Now worker_shutdown/sweep interruptions are re-queued for
non-goal tasks too: the platform killed the task, so the
platform re-queues it. The re-queue is a fresh run, not a
breakpoint resume: goal_resume_pending is forced to 0 and any
stale resume_message is cleared, so the new run cannot be
hijacked by the goal-resume path or an old continue message;
files in the same worktree persist. All existing guards stay:
user pause, coordinator orphans, quota (still goal-only),
backoff, the 3-strike stall guard (non-goal tasks have no
checkpoints, so each platform re-queue counts as no-progress),
and the auto_resume_max cap. Other non-goal reasons (NULL,
quota, user cancel) still do not auto-resume.
2026-10-30 15:39:55 +08:00
openqj e2ce04ce4f Merge pull request #197 from openqj/feat-stream-waiting
feat: browser stream waits through round gaps instead of disconnecting
2026-10-30 15:29:17 +08:00
host 962bb50a1c feat: browser stream waits through round gaps instead of disconnecting
Stale bridge state used to end the stream on the first stale read,
so every round gap and every worker restart killed the watch (the
2026-10-30 demo watch died this way when a 15:03 deploy restarted
the worker). Now a stale state switches to a waiting heartbeat
every 5s while the task is still alive (running/queued/paused/
interrupted), and frames resume when the next round's MCP starts
polling. The stream ends only on a terminal task state, a missing
task row, or 10 minutes of continuous staleness. The panel shows
a waiting status and keeps the last frame. Also force-close the
SSE connection on stream end: HTTP/1.1 keep-alive otherwise left
terminal streams hanging with no EOF.
2026-10-30 15:14:37 +08:00
host b977a9dee8 feat: sync shared-home MCP scripts on control-plane startup
租户 config 的 mcp_servers.* 指向共享 home 路径,但宿主共享
home 里原先没有 MCP 脚本,宿主 app-server 启动 MCP 即文件
不存在、握手断掉,面板 mcp-servers-list 三服务器 toolsError
全是 handshake failed;14:43 的手工拷入能修一次,但下次
MCP 脚本改动后面板又会跑旧版。

本批只在服务端 main() 启动时同步一次:清单唯一定义在
cloud/shared_home_mcp.py(7 文件:playwright_mcp/browser/
remote、websearch_mcp/websearch、imagegen_mcp/imagegen,含
同目录兄弟模块),解析优先用 host-config-host.json 的
codex_home,兜底用 tenants_root 父目录/codex-home;缺失
拷入、不同刷新(原子替换)、相同不动;源缺失/目标不可写
只记 warning,绝不阻塞启动。create_server() 不做同步,
避免单测写向生产共享路径。

被否决的落点:worker _tenant_dirs(只在下个任务派发时跑,
面板要等下一个任务才更新,且不修共享路径本身);deploy
host_deploy.sh(会在 shell 里复制第二份清单,且重启不
部署时不刷新);面板请求时同步(把读请求变成写请求,
失败会把面板读数变成 502 风险)。
2026-10-30 07:08:51 +00:00
host ce7fb3afd6 feat: project directory tree + first-round snapshot
- cloud/tree.py: structured worktree tree (dirs first, ignore
  .git/node_modules, symlinks not followed, entry/depth
  truncation flags, text render for injection)
- GET /tasks/{id}/tree with /files auth + worktree resolution,
  max_depth 1..5, max_entries 1..500, escape 400, missing 404,
  cloud/cross-host 409
- MCP get_task_tree translating to the new endpoint
- worker first-round injection of a repo-checkout top-level
  snapshot (2 layers, 120 entries) after repo resolution,
  labelled as checkout snapshot not worktree; skipped for
  coordinator/resume/fork, silent when no host source
- docs/project-tree.md; tests for tree, endpoint, injection,
  MCP translation
2026-10-30 14:46:11 +08:00
openqj 85f280e5e7 Merge pull request #193 from openqj/feat-software-engineering-github
feat: software-engineering skill GitHub guarantees
2026-10-30 14:40:30 +08:00
openqj 5c1069334a feat: software-engineering skill GitHub guarantees
- G0/G0.5: repo freeze items (new vs existing, owner/repo, default branch,
  visibility); new-repo creation and first commit are G1 prerequisites.
- One branch per thread (feat/swe/<phase>-<role>-<short>), never push main
  directly, never force-push main; sync main before PR, coordinator
  resolves conflicts, never silently.
- Delivery requires a PR with scope, contract version, test evidence,
  known issues and linked todos/PRD items; no PR number means not done.
- Merge gate: Checks green, directory ownership pass, frozen contract
  version, coordinator approval, no unresolved review comments.
  Branch protection (require PR, require Checks, block direct push) is a
  GitHub repo setting; if unset, coordinator records it as risk and
  enables it, never pretends it is protected. GET /tasks/{id}/pr/checks
  is read-only diagnostics, never reruns or approves.
- Merge commit by default; merge record holds PR number, merge commit
  hash, CI run link and executor; delete remote branch after merge.
- Per-client release tags (e.g. v0.1.0) from merged PRs; rollback is
  revert or redeploy previous tag, never force-push rollback.
- Security: tokens/keys never in code, commits or command lines;
  token only into subprocess env (cloud/pr.py GIT_TOKEN pattern);
  .gitignore covers secrets and local config.
- Main text and appendix (coordinator, client, acceptance) updated in
  sync; appendix travels inside SKILL.md for seeding.
2026-10-30 06:33:25 +00:00
host 56b8f98b47 perf: Chromium 启动 flag 微调 + screencast 画质 q60->q50
Flag:加 --no-first-run、--no-default-browser-check、
--disable-default-apps、--disable-extensions、
--disable-background-networking、--disable-sync、
--disable-translate,只跳过首跑/装配活,不改渲染与工具
行为。实测(runsc 容器,launch + new_page,各 3 次交替):
旧中位数 0.889s、新 0.819s——只差 70ms 且在逐次噪声
(0.70-0.99s)里,几百毫秒的预期不成立;零成本顺手留着。
画质:screencast 与 eager 抓帧 q60 -> q50,分辨率
1280x800 不动,点击坐标系不受影响。实测同页文本+渐变:
62068 -> 56428 字节(-9%),不是经验法则的 30%;觉得糊
一句话改回 60。
2026-10-30 14:27:04 +08:00
openqj 878f8de7da Merge pull request #190 from openqj/feat-remote-perf2
perf: 小优化组 + 恢复浏览器页签
2026-10-30 14:17:03 +08:00
host cbf5e65a7a perf: 小优化组 + 恢复浏览器页签
1. MCP 空闲睡眠 50ms -> 10ms:#187 后单轮 drain 约 0.3ms,
   100 次/s 空闲迭代约 3% 单核,请求与远控输入拾取延迟
   从平均 25ms 降到 5ms。
2. stream 轮询 50ms -> 20ms。
3. stream payload 加 url/title;面板 stream 处理合并 url/
   title,5 秒 state 轮询只在未观看时跑(观看时 stream 已
   全量覆盖,轮询纯冗余;不观看时保留轮询,避免会话开始
   后页眉 stale)。
4. 恢复浏览器页签:#176(ccb5a86)改写 TaskDetailPanel
   页签时顺带删了 BrowserPanel 的挂载,导致线上控制台
   自 #176 起就没有浏览器页签(面板文件一直在,只是不
   被 import,构建产物因此不变)。本次把 import、页签
   与 Card 原样挂回;构建产物含“开始观看”标记可证。
console.html 由 npm run build:console 重打(342484 字节)。
console-redesign/ 为构建脚本审阅产物,不进仓库。
2026-10-30 14:09:02 +08:00
host 885bf38790 feat: add software-engineering skill v0.2
Add cloud/skills/software-engineering/SKILL.md (686 lines, v0.2, appendix with 9 role templates travels inside SKILL.md because skill_seed only copies SKILL.md, not templates/). Update skill pack tests: EXPECTED set and real-pack list now include software-engineering, plus an explicit seed test asserting it lands at skills/software-engineering/SKILL.md, contains the appendix, and tenant edits are never overwritten. One-line wording tweak in the architect template avoids the existing secret-scan false positive on the substring sk- inside agent-task-contract.md; deployed file sha256 bd0436399ac05c6131b9281dad00973e50d13540e4c56ad21f21e2b5209bae8e (40759 bytes, 686 lines).
2026-10-30 06:00:31 +00:00
host fafc0734c7 perf: screencast 隔帧改每帧,跟手优先
Page.startScreencast everyNthFrame 2 -> 1,画质 q60、尺寸
1280x800 不动。静态页不产生额外成本(帧只在重绘时来,
且 #187 起桥只写变化的帧字节);收益全部落在真正需要跟
手的时候:滚动、拖拽、动画期间帧率翻倍。测试断言同步。
2026-10-30 13:50:18 +08:00
host 910168126d perf: 桥变化才写、stdin 优先、stream 50ms 轮询
1. 桥写放大:旧代码每 ~50ms 轮询都无条件重写 state 和帧
   文件(约 40 次原子 temp+replace/s 走 runsc gofer,
   泵持久返回同一帧、updated_at 每轮都变,字节其实不变)。
   现帧字节不变不写;state 只在 url/title/started/holder/
   streaming 变化时写,或 0.5s 心跳一次(判死阈值 30s,
   绰绰有余);静态页写量从 ~40次/s 降到 ~2次/s。
2. MCP 主循环 stdin 优先:旧循环先 drain 再读 stdin,
   每次工具调用都先付一轮文件 IO。现 pending 请求先处理,
   空闲轮询 drain,忙时 200ms 保底 drain 一次防饿死。
3. stream 轮询 200ms → 50ms,画面延迟下限随之下降。
附回归测试:无变化轮询不重写、帧变化立即写、holder
变化绕过节流立即写。
2026-10-30 13:36:24 +08:00
host 2d840a6c37 fix: navigate 后立即推一帧,静态页开流不再等重绘
screencast 帧只在重绘时来,页面可能在 startScreencast 之前
就已画好——静态页于是开流后一直无画面,直到下一次重绘。
navigate 后用 CDP captureScreenshot 抓一张当前视口塞进帧
队列,泵在下一轮就把桥帧文件写出来;泵跳过空 session 的
ack(eager 帧没有可确认的 screencast 帧)。失败静默,
screencast 本身仍照常流动。
2026-10-30 13:14:38 +08:00
host ccb5a8638d feat: 任务详情 LLM 请求数(cost_records 存 completions) 2026-10-30 12:58:41 +08:00
openqj 2ff8975f43 Merge pull request #183 from openqj/feat-browser-filebridge
fix: 浏览器远控传输层改文件桥(Unix socket 在 runsc 下宿主不可见)
2026-10-30 07:35:46 +08:00
host 6bcdeb8da1 fix: 浏览器远控传输层改文件桥(Unix socket 在 runsc 下宿主不可见)
根因(2026-10-30 活体实证):#181 的 .browser-remote.sock 假设
宿主能连容器内 socket,但生产 Docker 任务走 runsc/gVisor,
其 gofer 挂载下容器内建的 socket 只在容器内可见,宿主目录
里只有普通文件(browser-live.png 一直能透传)。后果:server
代理对真实任务一律 503,会话根本没活过。

修法:传输层改四个普通文件(worktree 内,原子写 + 追加日志):
- .browser-remote-state.json:MCP 每轮重写,mtime 即存活信号,
  server 判 30 秒无心跳为会话死亡(503);
- .browser-remote-frame.jpg:最新 screencast 帧(base64 文本);
- .browser-remote-input.jsonl:server 追加,MCP 按偏移量消费
  完整行,执行时再判 holder,半行等下一轮;
- .browser-remote-control.json:{holder, seq},MCP 首轮即应用,
  接管可先于浏览器启动。
- holder 竞态:control 写下后 state 文件约 50ms 才刷新,server
  的 input/stream 改判 effective_holder(控制文件 mtime 新者胜),
  否则接管后立刻点会误 409。
- 线程契约与控制权模型不变;MCP 主循环仍是唯一触页线程。
- 播种刷新(#181)让老租户自动拿到新版脚本。

测试:socket 测试改文件桥测试(桥轮询/偏移量/半行/陈旧态/
effective_holder + server 代理全链),全量 1357 项通过(skipped 4)。
2026-10-30 07:28:32 +08:00