Commit Graph
278 Commits
Author SHA1 Message Date
host be41ad2c0a test: pin skill appendix sync and SKILL.md-only seeding (C-line, no behaviour change)
- test_skill_appendix_sync: all 9 appendix chapters 00..08 in order,
  body role-matrix set == appendix role-title set, authority
  statement on both sides, GitHub guarantee in body + named
  appendix chapters 00/04/07, seeded copy byte-identical with
  full appendix. Previously only 00/08 existence was asserted.
- test_skill_seed_pinning: only SKILL.md copied (pack extras never
  land, never resurrected), tenant extras left alone, tenant-modified
  SKILL.md never overwritten across repeats, identical = noop,
  deleted SKILL.md re-seeded (documented boundary, not deletion
  preservation).
- skill_seed.py: evaluation note only (why whole-dir copy is not
  done); no code change.
2026-11-03 02:49:25 +00:00
host 06710cf47b feat: 沙箱规模——/tmp 顺序加严、注册表寻址与失联精确清理、超时边界抽取
- Docker /tmp tmpfs 移到全部 -v 之前(父挂载在前,seed 子挂载在后);
  宿主级 bwrap /tmp 仍宣告未隔离,口径见 tmp_isolation_status()
- workers 表补 host/pid/address(老库幂等补列),心跳透传与保留语义,
  _claim 写 claimed_by_worker;orphaned_containers 精确识别失联容器,
  Worker.cleanup_orphaned_containers 默认只杀 pid_verified 的精确容器名
- docker_run_timeout() 纯函数抽取(行为与原内联一致),README 补边界与
  未验证清单;新增 test_sandbox_scale.py 20 项
2026-11-03 02:49:25 +00:00
host 143ab30094 feat: cost/quota set — dual-column priced reports, creator split, hard cutoff, approval-timeout auto-threshold
- CostStore: per-model breakdown, report_by_creator (JOIN tasks.created_by),
  dual-column reports with priced_usd_micros, unpriced_models listed
  separately and a billing note (proxy price, not the relay bill);
  _report itself untouched, old exact-key tests keep passing.
- Hard cutoff: cost.quota_cutoff_reason is the single USD/unpriced-cap
  decision; worker._quota_exceeded now delegates to it (identical
  messages); claim gate, turn boundary and poll kill stop spend and
  record interrupted + quota reason + audit; auto-resume re-checks
  live quota before requeueing.
- Approval timeout (G7 ops metric): cloud/approval_timeout.py —
  default stays 600s, >20% timeout rate recommends 1800s, <5% keeps,
  mid-range keeps; stats + recommendation + info log; wired into
  /metrics (member projects, no log spam) and a global-admin
  /admin/ops/approval-timeout endpoint (sweeps overdue first).
- Server: task/project cost endpoints return dual-column reports;
  new GET /users/me/cost and GET /admin/cost-by-creator.
- Tests: cloud/tests/test_cost_quota.py, 10 tests.
2026-11-03 02:49:25 +00:00
host 0072e1ae6a skill: cross-task artifact pass with sha256 dual-channel verification
Core artifacts travel as an explicit manifest; coordinator triple-checks
(core report vs manifest vs recomputed hash) before cleanup, regression
rewrites files verbatim, recomputes sha256, stops on ARTIFACT-HASH-MISMATCH
and only then runs the same core code plus import-only regression tests.
2026-11-03 02:48:55 +00:00
host d973e63b11 feat: wire full-auto executor to server HTTP endpoints
GET /full-auto/status, GET /full-auto/config-plan, POST
/full-auto/validate-action|merge-gate|ci-fix|release-plan|
rollback-plan, POST /full-auto/execute (global admin only,
FULL_AUTO_ENABLED + verified installation + branch protection
verified via GitHub API, else 403 + audit), GET /full-auto/audit.
Guards reused from cloud/full_auto.py; token only in Authorization
header; audit via audit_record redaction into AuthStore audit_log.
2026-11-03 02:48:55 +00:00
host b4a69e1e71 refactor: remaining longfunc small extractions after #231 (no module split)
Only pure computation/validation sub-blocks of _handle_github_webhook,
_create_followup, _deliver_to_thread and _task_chat_history are extracted;
all DB writes, audits, external calls and side-effect blocks stay in the
callers, error strings/codes and response shapes verbatim. _deliver_to_thread
uses the shared goal classifier only for /goal input: its non-goal terminal
path still goes to CAS revive (not 409) and its merged path keeps its own
wording. New tests: test_longfunc_remaining.py (15).
2026-11-03 02:48:55 +00:00
host 1accfac844 refactor: extract minimal pure helpers from _continue_task, _task_wait, _task_thread_read and _metrics
_continue_task 226->168 via _validate_continue_options;
_task_wait 158->139 via _validate_wait_params;
_task_thread_read 187->152 via _summarize_trace_entries;
_metrics 123->104 via _summarize_metrics_tasks.
No behaviour, message, status or route change; error strings verbatim.
New tests: test_longfunc_extra (9). Cloud full suite: 1439 OK, skipped=4.
Follows #229 pattern; must merge after #229 and be rebased/checked then.
2026-11-03 02:48:55 +00:00
host f96960c9c5 refactor: extract minimal helpers from _dispatch, _dispatch_project and _create_task
Move the /tasks/* sub-route out of _dispatch into _dispatch_task_route
(query passed explicitly; the files/tree/thread branches use it), move
the push-subscription DELETE branch out of _dispatch_project into
_project_push_subscription_delete, and turn the goal-validation block
of _create_task into the pure module function
_validate_task_goal_options (returns error string instead of sending
400, caller sends it unchanged). Statements are moved verbatim except
indentation and the goal send+return -> return-error transform; no
behaviour change. server.py module split intentionally not done: it
remains a larger, higher-risk refactor.

Add test_longfunc_split covering the pure goal validator's success
paths and all eight error strings.
2026-11-03 02:48:55 +00:00
host a9e2f70d9e workers: reap pid_verified stale workers with pre-kill audit and post-kill verification 2026-11-03 02:48:55 +00:00
host 8ea54cf66b workers: land host/pid/address columns with idempotent migration and pid_verified stale classification
- ensure_workers_table: new tables include host TEXT, pid BIGINT,
  address TEXT; old tables gain them via _ensure_hostpid_columns
  (PRAGMA column check, missing-only ALTER, old rows keep NULL,
  never fabricates host/pid). SQLite and Postgres share this path
  (PG via the db adapter's PRAGMA translation); POSTGRES_SCHEMA
  deliberately still excludes workers, which is volatile heartbeat
  data in UNMIGRATED_TABLES (adding it broke the P2 migration
  alignment tests, verified and reverted).
- heartbeat/register: given host/pid/address refresh, omitted
  values preserve the old row; address has no derived source.
  Local worker passes host_name + own pid; HTTP heartbeat only
  stores caller-supplied values, never the server's own.
- classify_stale_workers: kill only when age strictly exceeds the
  threshold AND host matches local host AND pid is proven dead;
  equal threshold, foreign host, missing pid, or live/unknown pid
  are list_only. Classification only, never kills, never matches
  by name/prefix, never touches infra containers.
2026-11-03 02:48:55 +00:00
host 452fc8ea7e docs(skill): add software-engineering README and same-dir reference test
SKILL.md header pointed at same-dir README.md but the pack had no
such file (CHANGELOG was supplied by #217, README was not). This
track adds the README only: purpose, chapter map, appendix
provenance entry point, seeding behaviour, and known boundaries.
It states no rule, changes no SKILL.md semantics, and does not
duplicate the CHANGELOG/script explanations -- overlapping parts
are pointers, with an explicit division-of-labour section.
Deletion retention is written only as the tombstone-design
placeholder, deferring to the skill_seed implementation, because
no tombstone implementation has landed. Test pins that every
file SKILL.md cites as same-directory exists (CHANGELOG, README).
2026-11-03 02:48:55 +00:00
host 4e554b5c07 docs(skill): add appendix provenance sources, checker and CHANGELOG
SKILL.md claimed its appendix was spliced verbatim from
templates/roles/ by a generation script (see CHANGELOG), but the pack
contained none of the three. Add the nine role sources (mechanically
split from the v0.2 appendix; SKILL.md untouched, zero semantic
change), tools/check_appendix.py (verify + --emit, stdlib only, never
edits SKILL.md), CHANGELOG.md, and test_skill_appendix_provenance.py
(5 tests). Seeding unchanged: skill_seed copies SKILL.md only, so
templates/tools/CHANGELOG never reach the tenant home.
2026-11-03 02:48:55 +00:00
host 61e08a1a8c feat: hard merge gate by default (CI + directory + contract + coordinator + reviews)
- cloud/merge_gate_hard.py (new): pure hard gate; mode resolution is
  env-only (unset/invalid -> hard, fail-closed); five checks plus
  branch protection must be protected; unknown/unprotected block
  even with explicit confirm; no-evidence blocks (fail-closed);
  directory check reuses #209 check_directory_isolation (v0.3 rule);
  blocks codes + merge_gate_blocked audit detail are the track-3
  pushback interface (no pushback state machine in this track).
- cloud/server.py: POST /tasks/{id}/merge uses hard mode by default;
  record/enforce/off only when explicitly set via env (preserving
  #209 semantics); body merge_gate_mode is ignored server-side so
  callers cannot downgrade the gate; every gate block writes a
  merge_gate_blocked audit before returning 409, status unchanged;
  hard-mode body type errors return 400.
- docs/merge-gate-branch-protection.md (new): protection payload
  (PR + strict status checks + no force-push + no deletion +
  enforce_admins), empty required_checks = unconfigured, Pro/public
  prerequisites, 403 causes; states protection is NOT claimed enabled.
- SKILL.md (gate-mechanism wording only): four checks -> five checks;
  body + coordinator + client appendix gate paragraphs updated to
  default hard, merge_gate_blocked audit, payload doc reference.
- tests: 31 new hard-gate tests (pure + endpoint + skill text);
  #209 endpoint tests now set record env explicitly (their intent);
  test_server_merge 200-path calls now carry full hard evidence.
2026-11-03 02:48:55 +00:00
host d553f9549c feat: configurable merge gate (branch protection + directory isolation) and skill body/appendix checks
- cloud/merge_gate.py: pure gate; directory check reuses the v0.3
  git diff --name-only merge-base command; protection modes
  record (default, explicit confirm required when unprotected),
  enforce (always blocks when unprotected), off (explicit only).
  Confirm covers protection risk only, never directory violations.
  Protection status is caller/env-declared only; unknown is a
  recorded risk, not a claim of protection.
- server POST /tasks/{id}/merge: applies the gate only when gate
  evidence is supplied (or env declares protection); no-evidence
  calls behave as before plus a merge_gate field in the response.
  Blocked merges return 409 without changing status; confirmed
  unprotected merges write a merge_gate_risk audit entry.
- skill_seed.py: whole-directory seeding evaluated and rejected
  (cannot preserve tenant deletions; no benefit while the pack
  holds only SKILL.md); SKILL.md-only behaviour unchanged.
- software-engineering SKILL.md: gate paragraph added to the body
  and to the coordinator + client appendix sections (v0.2 text on
  this base; port to v0.3 text on rebase after #199 merges).
- tests: 16 new tests covering gate modes, directory reuse,
  body/appendix bidirectional assertions naming the missing side,
  SKILL.md-only seeding and tenant-edit preservation.
2026-11-03 02:48:55 +00:00
host 44616a03dc feat: software-engineering skill v0.5 (G7/G8 ops + iteration)
Stacked on v0.3 + v0.4 (HEAD d6522ba feat-full-auto): v0.3/v0.4
content preserved, not reverted. Adds G7 post-release ops and
observation (monitoring, errors, cost/quota, feedback, hotfix loop,
rollback triggers, ops manual) and G8 iteration and scaling (v1.1
handover list, contract v1->v2 protocol, new-end onboarding,
scale/performance, deprecation), with G6->G7 and G7->G8 gates,
DAG, gate table, coordinator derivation, G0-G8 todos, checklist,
09-ops and 10-iteration role templates, and ops-manual / v1.1
handover templates in both body and appendix. Settles web+React+
multi-viewport focus, cross-viewport E2E, web single-end release
with iOS/Android native postposed, open-source-first, and the
TypeScript full-stack default (user-request-only language change;
codex-app-server itself stays Python). Pending #199 and #205
merge, then one rebase round. PR only, do not merge or deploy.
2026-11-03 02:48:54 +00:00
host a605ff2f92 feat: full-auto mode v0.4 (host executor + skill + CI fix loop, stacked on v0.3)
Host auto executor (cloud/full_auto.py, stdlib only): default off
(FULL_AUTO_ENABLED + verified installation), forbidden actions
fail-closed (delete repo, force-push, visibility change, secret
read/expose, branch-protection delete/bypass, GitHub account
auto-registration), token only in request header, audit log
redacted, branch protection requires PR + strict checks +
no force-push + no deletions + enforce_admins, merge gate
(CI green + directory check + contract version + coordinator
approval), CI fix loop max 3 then escalate, rollback only via
revert PR or previous tag. Skill: v0.4 full-auto mode, coordinator
reviews, host executor executes, task agent never executes
merge/deploy; repo-creation honesty branch (org via App token,
user repo needs_user_once); no web auto-registration of GitHub
accounts. Tests: 19 new, cloud full 1451 OK (skipped=4).
2026-11-03 02:48:54 +00:00
host 34ed438d0e feat: software-engineering skill v0.3 (P1-1..P1-4 + P2)
- P1-1 directory-isolation executable check: git diff --name-only
  against merge-base, output recorded in merge record
- P1-2 derivation checklist table (8 required items, missing one blocks dispatch)
- P1-3 QA split into single-client review and cross-client E2E with
  fixed deliverables and agent-test attribution
- P1-4 sub-task slim-pack trimming rules (listed chapters only, never full copies)
- P2: inter-thread Q&A default policy, docs/decisions.md append-only,
  per-client onboarding docs (web first at G4a), v1.1 handover checklist
- Body and appendix kept in sync; new lightweight keyword test
2026-11-03 02:48:54 +00:00
host 49a2f7881b coverage85-step4e-fix: misc 不再 import stdlib queue,避免遮蔽 test_queue_sandbox 的 cloud/queue 导入 2026-11-03 02:48:54 +00:00
host 12f56cd8c8 coverage85-step4e: workers/autoscaler/playwright 长尾测试(+29 项) 2026-11-03 02:48:54 +00:00
host 3b7d45c9a2 coverage85-step4c: server 长尾端点与子路由测试(+19 项) 2026-11-03 02:48:54 +00:00
host 21a5fd3a2a coverage85-step4b: worker marker_tools 派发剩余分支测试(+20 项) 2026-11-03 02:48:54 +00:00
host 0e665d616f coverage85-step4a: worker run_one/_run_attempt 缺口测试(+34 项) 2026-11-03 02:48:54 +00:00
host 903f138fd9 coverage85-step2d: 新测试 lint 清理(删无用导入与变量) 2026-11-03 02:48:54 +00:00
host 3d2df7dfe6 coverage85-step2b: server/worker 剩余缺口真实行为测试(+24 项)
- server:_dispatch_project(项目 PATCH repo_url、成员邀请/改角
  色/删除与 404/403/409 族)、_create_task 校验 400 族与合法变体
  (goal/协调员/plan/环境)、trace/agents/trace-stream(终态 SSE
  snapshot+done)、GitHub webhook 去重与坏体。
- worker:_gateway_context(xcmapi/缺 origin/缺属主/假 sub2api
  key 与协调员父属主回查)、_ensure_gateway 缓存、_docker_task_
  context(代理/租户网络失败/搜索透传)、_deliver_marker_tools
 (守卫空跑、创建派生、goal 转换、轮询快照)、协调员孤儿扫尾、
  run_one/run_all/run_loop/main 空库路径。
生产代码零改动。
2026-11-03 02:48:54 +00:00
host 576fd50ee3 coverage85-step2a: MCP/deploy/verify-multiuser 剩余缺口真实行为测试(+43 项)
- playwright/imagegen/websearch 三个 MCP:handle_request 全工具分发、
  通知/错误分支、main() 进程内 stdio 循环(pipe/StringIO + 假浏
  览器/假依赖)、LineReader 分块与 EOF、真实 RemoteBridge 构造、
  flat-copy 回落导入;三文件孤立覆盖率 98%(余 __main__ 与
  通知型 handler-error else 分支,见回执)。
- deploy.py:check_clean_tree 脏/暂存/放行、archive 失败、remote
  助手、transfer 全路失败与 sha 不符、git 源冲突与干跑、tar
  路径干跑/构建/部署成功与失败;孤立 92%(余行由既有 deploy
  测试覆盖,__main__ 除外)。
- verify-multiuser main():真实本地控制面 13/13 通过且清理归零、
  死端口与 stub 项目失败早退、清理触发器异常报 leftover。
生产代码零改动。
2026-11-03 02:48:54 +00:00
host 81a58daca2 coverage85-step1: server/worker/host 主力缺口真实行为测试(+113 项)
只补 cloud/server.py、cloud/worker.py、host.py 的真实行为测试,
不动生产代码:
- server:助手函数(路径/租户/提取/链路读/配额/限流/心跳/
  host CLI stub 脚本)40 项
- worker:纯助手 + Worker DB/认领/容量/配额/跟进/租户/协调员
  孤儿/回退 token,真实 SQLite 与真实子进程 36 项
- host:假 app-server 脚本走真实 JSON-RPC 协议,run_turn
  成功/超时/中断/失败/历史错误回捞 + 状态/git/PR 助手 37 项
不含 mock 被测代码、无 skip、无 no-coverage、无空循环刷行。
2026-11-03 02:48:54 +00:00
host f39ed3654e ci3: lint/mypy 第二阶段清理 + 覆盖率 76% -> 78%(门槛不动)
基底:#251 头 174b631(堆叠于 #242 feat-ci-expand 之上),
基线 origin/main aebc6238;本轨不改 #251 已定语义、不动
workflow 门槛、不删检查、不加 skip。

lint(ruff E,F@120):109 -> 10
- F401 43 -> 0:删未用导入;try/except 双绑定的名字出现
  “打地鼠”(删 except 分支后 try 分支又被标出),两分支
  均已核对正文无引用后清掉;MCP 三模块的 Error 名不删,
  改走 importlib 属性绑定(见下),两路径绑定都保留。
- E702 40 -> 0:全在 host.py main() 的 argparse 段,纯分行,
  tokenize 按分号 token 拆(含 2377 行多行语句只断首个分号);
  一次 5 空格缩进失误已回滚重做并以 ast + 全量 host 测试验证。
- E501 13 -> 0:host.py 5 项随 E702 分行连带清掉;mcp_server
  3 项把描述拆成相邻字面量(AST 比对 24 条工具描述与原值
  完全一致,注意 ruff 按东亚宽度计长,CJK 算 2);test_host
  5 项断开 git commit 参数表。
- E402 8 -> 10:生产侧 2 项清掉(db.py _re、github_app
  _base64 移到顶部,之前无引用、无重名);剩余 10 项全是
  tests/ 的 sys.path 前置 import(6 项旧 + 4 个新测试文件
  沿用同一既定模式),移上去会破坏单文件直跑,有意保留。
- F841 4 -> 0:只去绑定、不删调用(run_task、DB 入队、
  tarfile 上下文、git 提交),各有注释与原测试/新测试覆盖;
  host dispatch_from_branch 新增真实 git 仓库回归测试。
- F811 1 -> 0:server 本地 _row_get 与 peer_messages.row_get
  逐行相同,删本地定义,全部调用点解析到同一 import;
  新增 test_server_row_get(同一对象 + 三路行为)。

mypy(生产范围):24 -> 0,零新增 type: ignore
- no-redef 9 -> 0:MCP 三模块 try/except 两次 from-import
  改 importlib.import_module + 属性绑定(cloud 优先、flat
  copy 回落、内层 ImportError 同样回落,语义与原来一致;
  flat-copy 子进程测试证实回落后 __module__ 为顶层名);
  server _row_get 随 F811 消除。
- arg-type 10 -> 0:harness2_tasks shared 改 TypedDict
  逐键精确类型;CodexProbeAdapter 补 no-op validate_tree
  (_adapter_validate 本就跳过缺失方法,行为一致)。
- var-annotated 3 -> 0:sub2api _SHARED_ADMIN/_login_slots、
  verify-multiuser RESULTS 加精确注解。
- assignment/misc 2 -> 0:marker_tools ZoneInfo 先声明
  Any 再 try import(草稿先验 mypy 通过)。

bandit:总数 286 不变,High 仍 4 项,全在 host.py,本轨不改
(属第 3 项白名单轨范围),只做 triage:
- 1319 run_verify:无插值,整串验测命令,管道/&& 须 shell
  语义(新增 test_host_run_verify 断言管道与 && 真实生效)。
- 1497 notify:模板来自算子配置,三处插值全部 shlex.quote,
  证据为既有 test_notify_failure_quotes_shell_metacharacters
  (; touch 与 $(touch ) 不执行)。
- 2099/2103 goal verify:无插值,任务配置验测串,同 run_verify。
- 无新增 High;pip-audit 0 漏洞。

覆盖率(同口径三套件 coverage 计量):75.96% -> 77.87%
(22392 行/缺 5385 -> 22437 行/缺 4963,显示 76% -> 78%)
- client.py 0% -> 90%(pipe 假子进程 + main 分支)
- run_parallel.py 0% -> 97%、migrate 包装 0% -> 95%
- playwright_mcp 37% -> 62%、imagegen_mcp 47% -> 79%、
  websearch_mcp 51% -> 86%、deploy.py 56% -> 77%
- verify-multiuser 0% -> 27%(仅 check/req helper;main()
  是主机侧真实多用户实弹验证,按门禁不建生产任务、
  不以 mock 冒充,仍为缺口)
- host.py 55% -> 57%(文件 2020 -> 2060 行系 E702 分行
  的计量口径变化,非代码膨胀)
- 未到 85%:门槛保持 75% 不动,不删低覆盖模块出统计口径;
  缺口主力仍是 server.py / worker.py / host.py 大文件,
  下一阶段须针对性深测,非本轨快赢可及。

验证:三套件 1576 -> 1629 项全过(tests 127、cloud/tests
1449 skipped=4、deploy/tests 53);ruff E9 通过;mypy 0;
bandit High 4(同上);pip-audit 0 漏洞;试跑产物已删,
提交后工作树干净。
2026-11-03 02:48:54 +00:00
host 7ba5bfaf81 ci2: fix worker F821 NameError, mark harness2 sha1 non-security, rename E741 var
- worker._tenant_dirs: playwright refresh log used undefined 'task';
  use the method argument project_id (ruff F821, real bug: NameError
  aborted tenant seeding when a playwright script was refreshed).
  Regression: cloud/tests/test_worker_playwright_seed_f821.py (2 tests;
  pre-fix run reproduces NameError at worker.py:2780).
- harness2.assign_ids: hashlib.sha1(..., usedforsecurity=False) -- the
  digest is only a deterministic shuffle seed, not a security use;
  digest value unchanged, bandit B324 no longer fires (High 5 -> 4).
- harness2.select_final: rename ambiguous loop var 'l' to 'lab'
  (ruff E741, behaviour-invariant).
- Deliberately NOT changed: host.py shell=True x4 (see PR body -- all
  four are intentional shell commands, changing semantics is unsafe),
  remaining ruff 109 / mypy 24, workflow thresholds (75% coverage
  gate unchanged; no gate was relaxed or removed).
2026-11-03 02:48:54 +00:00
openqj 7b537c31a6 test: close unclosed file handles in tests (Path read/write)
Replace bare open().read()/write() in tests with Path.read_text/
write_text (or json.loads over read_text); the five remaining
open(...).close() empty-file creations become Path.write_text('').
No assertion, predicate, or production code changed; no behaviour
change beyond closing the handle deterministically.

Long-function splits and server.py module split intentionally not
included: the longest functions (_dispatch, _create_task,
_dispatch_project, ~375-395 lines in a 10430-line server.py) are
central routing/creation paths, and splitting them is not low-risk;
see PR body.
2026-11-03 02:48:54 +00:00
host b391fe9320 resident P0+P1: DNS-level SSRF pin, batch budget, daily cap/cooldown/retention, PG schema/migration, dual-connection deadlock isolation, retarget baseline reset, proxy and structured failures, metrics
- DNS 级 SSRF pin:解析后 pin IP 建连,防 DNS 重绑定
- 批量总预算与每日上限/抖动冷却/保留期
- PG schema 与迁移清单
- 双连接死锁隔离
- 改目标重置基线
- 显式代理配置与结构化失败原因
- /metrics resident 口径
2026-11-02 03:30:57 +00:00
host 6525bcafd0 audit: resident website loop architecture and code quality to 10/10
Fix F1 name race with partial unique index, F2 pause/end mid-poll race with guarded updates, F3 redirect/numeric-IP SSRF, F4 quota prefix check, F5 duplicate row_get, F6 audit ternary; S1 unused import, S2 backoff doc, S3 defensive tearDown. Directed 20 pass, full 1700 pass.
2026-11-01 19:58:03 +00:00
host 561b3891fb test: resident website loop end-to-end with real fetch 2026-11-01 19:42:40 +00:00
host 0722ca3fd8 feat: resident watch tasks with pause/resume/end
Long-running website monitoring: new cloud/resident.py layer on top of
the automations claim/quota model (no task state-machine change).
States running/paused/ended; pause preserves the content-hash cursor
and stops polling, resume re-checks from the cursor, end is terminal
(resume after end is 409, a new watch is required). Polling is a cheap
fetch + hash comparison; only a real change creates a queued model
task. Quota exhaustion pauses (quota_exceeded) instead of ending and
keeps the cursor. Driver is the worker run_loop round boundary;
server exposes create/list/get/update/pause/resume/end with audit.
Console UI intentionally not wired this round (see verification
summary for deviations).
2026-11-01 19:01:39 +00:00
host 3a56f83114 websearch: 默认 auto 付费冻结,SearXNG 优先 + 显式解冻开关
- 默认 auto 只走免费链 searxng -> duckduckgo;即使环境里有
  付费 key 也不调用 Perplexity/Exa/Tavily/Brave,SearXNG
  失败也不回落付费,默认使用零付费花费。
- 付费后端代码/测试/文档全部保留;显式 pin(tavily/exa/
  perplexity/brave)仍照常调用,是有意覆盖不是默认。
- 解冻开关 WEBSEARCH_ENABLE_PAID,仅精确值 "1" 才恢复旧链
  (perplexity/exa/tavily/brave/searxng/duckduckgo),默认关;
  开关经 worker search_env 透传进 MCP config env。
- HttpClient 超时/DNS 失败改报显式错误(timeout after Ns /
  URL error),绝不把超时写成“无结果”。
- SearXNG 默认参数核实:代码无 SEARXNG_URL 硬编码默认值;
  SEARXNG_ENGINES 默认 bing,360search 已在基线;WEBSEARCH_TIMEOUT
  默认 20 秒。主机 env 实测仅 SEARXNG_URL=http://searx.diante.me
  一行,故默认 pin 生效;文档 https 漂移改回 http(主机 nginx
  仅听 80、无 443 vhost,https 在 TLS 层即失败)。
- 旧测试更新原因:test_websearch_mcp 原断言默认 auto 为
  perplexity 优先,与新冻结语义直接冲突,故改为断言冻结后
  的 searxng -> duckduckgo 回退 + 不泄漏 key;旧 perplexity ->
  exa 回退断言移至解冻开关测试(WEBSEARCH_ENABLE_PAID=1 下)。
  未删断言、未加 skip、未放宽门禁。
- 文档另删 WEBSEARCH_PROVIDERS 说法:代码从未实现该变量。
- 未推、未建 PR、未合并、未部署、未改主机 env/nginx/filter/
  证书、未做付费后端 live 验证、未创建生产任务。
2026-11-01 14:15:54 +00:00
host e9fa43dd8b websearch: task-level allowed_hosts extension for scope=code + JS-heavy extract evaluation (mock-first)
- search(..., allowed_hosts=[...]) extends (never replaces) the 15-domain
  code list for that call only; scope=general + allowed_hosts raises,
  empty/illegal values raise before any HTTP, dot-boundary matching,
  extension values never in error strings or attempted entries.
- Native-filter parity: Tavily/Exa get the effective list, Perplexity
  skips its native filter past its 20-domain cap (post-filter stays
  authoritative), Brave/SearXNG/DuckDuckGo rely on post-filtering.
- MCP web_search gains allowed_hosts passthrough (only when supplied,
  so older search_fn fakes keep working).
- JS-heavy evaluation documented in docs/web-search-mcp.md, mock-first:
  direct fetch is shell-only by construction; provider extract can
  supply rendered text in mock only; live SearXNG in this VM env
  failed (remote closed connection) and paid-backend live effects are
  unvalidated blockers, not claimed.
- Tests: test_code_search_ext (9) + test_js_heavy_extract (5 + 1 live
  skipped); existing websearch/code-search suites still green.

(cherry picked from commit 9c26bbe848438e1f34bcb000700aeb4c284f4f23)
2026-11-01 09:06:28 +00:00
host 125b436ab3 browser: click-path allowlist, safe tenant script upgrade, allowlist activation plan
- click/click_ref: pre-click allowlist gate on resolvable anchor href
  (same check as navigate; miss refuses before the click executes);
  click/click_ref/click_at: post-click landing URL check.
  click_at has no pre-click gate by design (coordinates name no
  element); JS-driven navigation, evaluate, meta refresh and popups
  are documented as unintercepable with the Tinyproxy 403 backstop.
- Tenant script upgrade is now safe: only proven-unmodified scripts
  (manifest match, 2a569ac known-shipped SHA-256, missing, or
  current) auto-merge; tenant-modified scripts are never overwritten,
  are reported as skipped, and get one deduped audit record. The
  manifest never learns a tenant hash (else the next upgrade would
  mistake a modified file for untouched). Idempotent reruns.
- Activation candidate = deployed Tinyproxy programming set encoded
  with leading-dot entries (plain entries would regress
  docs.python.org); npmjs.org per the deployed filter, not npmjs.com.
  Host activation itself is NOT performed this round: only the plan,
  validation tests and rollback are delivered.
Tests: 28 new (click 13, upgrade 8, activation 7), all fake/no-network;
98 browser-related tests pass including prior suites.

(cherry picked from commit e4a93f5748ec815f163d0bc096ff4dc4b8c651f5)
2026-11-01 09:06:28 +00:00
openqj 2a569ac6e3 Merge pull request #272 from openqj/feat-platform-browser
feat: platform browser multi-tab, navigation allowlist, close cleanup, egress filter, allowlist worker wiring
2026-11-01 16:42:59 +08:00
host da7a9950fb feat: platform browser multi-tab, navigation allowlist, close cleanup, egress filter, allowlist worker wiring
- Multi-tab: browser_new_page / browser_list_pages /
  browser_switch_page / browser_close_page in playwright_mcp.py;
  each new page gets a fresh Playwright context, so cookies and
  storage are isolated between pages. Single-page tools act on
  the current page; switching to a missing index is an error,
  not a crash; closing the last page is an error (use
  browser_close).
- Navigation allowlist: BROWSER_ALLOWED_HOSTS (comma list,
  leading-dot suffix match, case/port-insensitive) checked in
  _check_allowed_url before launching Chromium; unset (or empty)
  means no check, the old behaviour. file:// always allowed;
  localhost/127.0.0.1 have no special treatment.
- Close cleanup: browser_close closes all pages and best-effort
  removes cwd browser-live.png / browser-live.json; custom
  screenshot files are untouched. list_pages needs no launch.
- Egress filter: 10 programming-source Tinyproxy BASIC rules
  (python.org, nodejs.org, go.dev, pkg.go.dev, rust-lang.org,
  developer.mozilla.org, crates.io, stackoverflow.com,
  cve.mitre.org, nvd.nist.gov); already-present github.com /
  pypi.org / npmjs.org not duplicated; searx.diante.me stays
  exact-only, no diante.me wildcard.
- Allowlist worker wiring: _docker_task_context now passes a
  set, non-empty BROWSER_ALLOWED_HOSTS from the worker env into
  [mcp_servers.playwright.env] in the tenant config, so the MCP
  process actually sees it (container -e alone does not reach
  MCP subprocesses, 2026-10-30). Unset/empty writes nothing and
  never overwrites; the value never goes into logs; it is not
  added to the websearch env block or the container env.
  ensure_playwright_mcp_env now returns False without rewriting
  when the result would be identical (canonical join: one blank
  line before the block), so a second prep leaves an unchanged
  tenant config alone; other sections untouched.
- Skill + docs: browser-automation SKILL.md multi-tab/allowlist
  section; docs/platform-browser.md comparison doc (new file).
- Tests (mock/fake only, no network, no real Chromium, no real
  credentials): cloud.tests.test_playwright +
  cloud.tests.test_websearch_worker, 48 tests OK, including 6
  new wiring tests (switch off passes no variable, switch on
  reaches MCP env verbatim, empty never written, old-tenant
  merge idempotent + no-rewrite, ensure no-op when identical,
  allowlist enforces only after wiring).
- NOT done (see PR body): no persistent profile, no credential
  vault, no approval card, no CAPTCHA bypass, in-page link
  navigation does not pass the allowlist, no real-Chromium live
  verification, host tinyproxy filter not reloaded.
2026-11-01 08:34:13 +00:00
host 397a46f850 websearch: add scope=code programming search + provider extract chain
- search(..., scope=code) restricts to 15 programming domains
  (official docs, GitHub, registries, Stack Overflow, CVE/NVD):
  native include_domains/search_domain_filter where supported
  (Tavily, Exa, Perplexity), post-filtering for all providers
  (Brave, SearXNG, DuckDuckGo have no native filter).
- web_fetch now uses fetch_best: Tavily extract, then Exa
  contents, then direct fetch; both-fail errors name both
  failures, direct-fetch 403 under the egress whitelist is
  fail-closed, not bypassed. Default general scope and direct
  fetch_url behaviour unchanged.
- MCP web_search gains scope enum (general/code); docs updated.
- Tests all mock HTTP, no real keys.
2026-11-01 08:16:48 +00:00
openqj 27f48fe1ed Merge pull request #200 from openqj/feat-console-project-nav2
console: 项目导航移到左侧栏,工作台不再承担项目模块
2026-11-01 15:27:54 +08:00
openqj aebc6238a6 Merge pull request #201 from openqj/feat-platform-auto-resume
feat: auto-resume platform-interrupted non-goal tasks
2026-10-30 15:51:43 +08:00
host 08484302ac console: 项目导航移到左侧栏,工作台不再承担项目模块
左侧栏 ProjectNav:项目完整列表(名 + 短 id + 我的角色,
选中高亮,点击切换)+ 创建项目(名/GitHub 仓库/repo_url)
+ 选中项目成员(邀请/改角色/新增移除);工作台删除
sec-projects 一节与任务列表里的项目下拉框,侧栏为唯一
项目选择器,自动选第一个项目逻辑不变。
薄模块补齐(均为已核实后端路由):成员移除 DELETE、
自动化详情(GET 单条 + 产出)与编辑 PATCH、分诊条目点进
任务/自动化详情、Workers 搜索与在线筛选、管理区设置配额
POST。主线新功能(Todos、Browser、taskMeta 等)原样保留,
未做任何回退。console 标记单测本地 53 项通过。
2026-10-30 07:43:54 +00:00
host 4428a721a6 feat: auto-resume platform-interrupted non-goal tasks
Deploy restarts kill running tasks: the worker marks them
interrupted (worker_shutdown on graceful exit, sweep on stale
reclaim), and auto_resume_interrupted used to accept goal tasks
only, so every non-goal task killed by a deploy stayed
interrupted forever unless a human retried it (the 2026-10-30
demo task died this way).

Now worker_shutdown/sweep interruptions are re-queued for
non-goal tasks too: the platform killed the task, so the
platform re-queues it. The re-queue is a fresh run, not a
breakpoint resume: goal_resume_pending is forced to 0 and any
stale resume_message is cleared, so the new run cannot be
hijacked by the goal-resume path or an old continue message;
files in the same worktree persist. All existing guards stay:
user pause, coordinator orphans, quota (still goal-only),
backoff, the 3-strike stall guard (non-goal tasks have no
checkpoints, so each platform re-queue counts as no-progress),
and the auto_resume_max cap. Other non-goal reasons (NULL,
quota, user cancel) still do not auto-resume.
2026-10-30 15:39:55 +08:00
host 2d644f901b feat(console): automation panel edit and richer list
- AutomationPanel rows now show title, prompt content (160-char preview with expand/collapse), repo_url, last/next run, run_once badge and cancelled time.

- Active automations get an inline edit form (name/prompt/interval/repo) that PATCHes only changed fields via the existing automation_update endpoint; backend semantics unchanged (no last_run_at reset, cancelled stays terminal).

- Rebuilt cloud/console.html with tsc + vite (build green); automation python tests 18/18 OK.
2026-10-30 15:31:21 +08:00
openqj e2ce04ce4f Merge pull request #197 from openqj/feat-stream-waiting
feat: browser stream waits through round gaps instead of disconnecting
2026-10-30 15:29:17 +08:00
host 962bb50a1c feat: browser stream waits through round gaps instead of disconnecting
Stale bridge state used to end the stream on the first stale read,
so every round gap and every worker restart killed the watch (the
2026-10-30 demo watch died this way when a 15:03 deploy restarted
the worker). Now a stale state switches to a waiting heartbeat
every 5s while the task is still alive (running/queued/paused/
interrupted), and frames resume when the next round's MCP starts
polling. The stream ends only on a terminal task state, a missing
task row, or 10 minutes of continuous staleness. The panel shows
a waiting status and keeps the last frame. Also force-close the
SSE connection on stream end: HTTP/1.1 keep-alive otherwise left
terminal streams hanging with no EOF.
2026-10-30 15:14:37 +08:00
host b977a9dee8 feat: sync shared-home MCP scripts on control-plane startup
租户 config 的 mcp_servers.* 指向共享 home 路径,但宿主共享
home 里原先没有 MCP 脚本,宿主 app-server 启动 MCP 即文件
不存在、握手断掉,面板 mcp-servers-list 三服务器 toolsError
全是 handshake failed;14:43 的手工拷入能修一次,但下次
MCP 脚本改动后面板又会跑旧版。

本批只在服务端 main() 启动时同步一次:清单唯一定义在
cloud/shared_home_mcp.py(7 文件:playwright_mcp/browser/
remote、websearch_mcp/websearch、imagegen_mcp/imagegen,含
同目录兄弟模块),解析优先用 host-config-host.json 的
codex_home,兜底用 tenants_root 父目录/codex-home;缺失
拷入、不同刷新(原子替换)、相同不动;源缺失/目标不可写
只记 warning,绝不阻塞启动。create_server() 不做同步,
避免单测写向生产共享路径。

被否决的落点:worker _tenant_dirs(只在下个任务派发时跑,
面板要等下一个任务才更新,且不修共享路径本身);deploy
host_deploy.sh(会在 shell 里复制第二份清单,且重启不
部署时不刷新);面板请求时同步(把读请求变成写请求,
失败会把面板读数变成 502 风险)。
2026-10-30 07:08:51 +00:00
openqj 671b4bedef Merge pull request #194 from openqj/feat-project-tree
feat: project directory tree + first-round snapshot
2026-10-30 15:00:11 +08:00
openqj 4f035f0449 Merge pull request #186 from openqj/feat-console-llm-status
前端:任务详情 LLM 进度与失败原因标识
2026-10-30 14:46:24 +08:00
host ce7fb3afd6 feat: project directory tree + first-round snapshot
- cloud/tree.py: structured worktree tree (dirs first, ignore
  .git/node_modules, symlinks not followed, entry/depth
  truncation flags, text render for injection)
- GET /tasks/{id}/tree with /files auth + worktree resolution,
  max_depth 1..5, max_entries 1..500, escape 400, missing 404,
  cloud/cross-host 409
- MCP get_task_tree translating to the new endpoint
- worker first-round injection of a repo-checkout top-level
  snapshot (2 layers, 120 entries) after repo resolution,
  labelled as checkout snapshot not worktree; skipped for
  coordinator/resume/fork, silent when no host source
- docs/project-tree.md; tests for tree, endpoint, injection,
  MCP translation
2026-10-30 14:46:11 +08:00