Commit Graph
283 Commits
Author SHA1 Message Date
host eeb10118d6 feat: tamper-evident audit ledger (hash chain) + dual-column cost view
- Hash-chain ledger (cloud/ledger.py): prev_hash/entry_hash per record,
  SQLite append-only triggers, UNIQUE prev/entry hash with fail-closed
  retry at the transaction boundary; verify/export (incl. expected-head
  check for tail truncation); legacy audit_log rows marked pre-chain via
  a single ledger_genesis marker, never rewritten.
- AuthStore.audit() writes audit_log + chain entry in one transaction.
- Track-3 interface: audit_verdict / audit_pushback_round /
  audit_rebuild / third-failure assessment / test-change audit /
  meta-review / rebuild-notice entries; per-(module, contract-version)
  failure counts derived from the ledger (pass or completed rebuild
  resets; total > 3 triggers rebuild); acceptance-triggered rebuilds
  require a confirmed user notice.
- Admin read-only endpoints: GET /admin/audit-ledger/verify|export.
- Cost: dual_column_view (priced_usd_micros + billing note, wording
  reused verbatim from PR #212) wired into project/task cost endpoints;
  by-model and per-creator reporting stay in #212, not duplicated here.
- PG schema + migration lists updated for audit_ledger (incl. sequence
  setval alignment).
2026-11-03 02:49:25 +00:00
host c0ab995ccc integrate: appendix count assertion follows 12-chapter table 2026-11-03 02:49:25 +00:00
host e921db3472 audit: independent audit role, auto-pushback, rebuild branch, three tightenings, QA full-chain
Independent audit (no code, independent re-run, separate signature;
CONDITIONAL never passes; deviation needs due todo) and automatic
pushback (fail -> derived fix subtask, merge/release blocked,
package with per-item evidence, full re-audit, 3-round limit with
a separate CI auto-fix counter, ledger hash-chain interface for
track 4, gate-block interface for track 1). Over 3 failures per
module per contract version: rebuild only that module from the
frozen contract (old implementation archived, avoidance list),
contract/design root cause -> G2 re-freeze escalation instead.
Tightenings: 3rd-failure triage (structural early rebuild /
early refreeze, near-complete one targeted round), rebuild keeps
tests and passed assets with an old-test first gate, audit-of-
audit sampling/correction and G6 user-only acceptance plus
acceptance-rebuild user confirmation. QA full chain: self-test
three parts, PRD coverage, agent six, cross-client matrix with
honest uncompiled verdicts, G5 gates, impact-scoped regression,
dual QA/audit signatures, failed-link re-runs in re-audit.
Skill text is purely additive (+459/-0) against main v0.2 text;
port this section into v0.3/v0.4/v0.5 texts on merge, never
overwrite them with this file.
2026-11-03 02:49:25 +00:00
host 41e8c98512 feat: cloud 门禁/超时/成本/沙箱收敛为单一判定源
- 新增标准库叶子层 timeouts/gates/quota/sandbox_policy:
  审批 3600 与单轮 600 各有名常量与环境覆盖层,已建项
  expires_at 永不重算;目录/控制/合并状态门禁统一判定,
  目录违规无任何覆盖参数;配额取数与比较唯一来源
  (USD 先于无价、命中均为 >=、无价只计 actual 行);
  /tmp 隔离矩阵与失联清理判定纯函数化。
- 调用点委托:server/worker/automations/auto_memory 的
  配额 SQL 与比较、server/tree/marker 的目录解析、
  marker 的控制门、sandbox 的任务容器名正则、两处 300 秒
  陈旧阈值默认值均收敛,别名与错误文案不变。
- 统一两个 P0 口径分歧:服务端无价计数由全部 usd IS NULL
  行改为 actual-only;显式不限额用户在服务端准入亦不受
  无价上限约束(与 worker 运行时一致)。
- 不新增门禁种类与 record 模式(主线无此实现);不改
  skill_seed/skills/SKILL.md;不改 v1.1 契约与分支保护。
- 新增 test_arch_converge.py 18 项。
2026-11-03 02:49:25 +00:00
host 2720acef46 integrate: sync appendix sources to v0.5 appendix table (00-10)
Integration arbitration, not a PR commit: v0.3/v0.4/v0.5 synced body
changes into appendix chapters 00/02/04/06/07 while templates/roles/
sources stayed at v0.2, so check_appendix failed 6 chapters; and
09-ops/10-iteration chapters (v0.5) were invisible to the strict
chapter regex because of the ', v0.5 new' title suffix, breaking
the #214 matrix-vs-appendix test. Direction is sources follow the
appendix (appendix is the tenant-visible authoritative version;
never overwrite v0.5 text with v0.2). 09/10 headers normalized to
strict form (suffix only); 09/10 sources split mechanically from
the appendix; EXPECTED_IDS extended to 11 chapters in the checker
and both appendix tests.
2026-11-03 02:49:25 +00:00
host be41ad2c0a test: pin skill appendix sync and SKILL.md-only seeding (C-line, no behaviour change)
- test_skill_appendix_sync: all 9 appendix chapters 00..08 in order,
  body role-matrix set == appendix role-title set, authority
  statement on both sides, GitHub guarantee in body + named
  appendix chapters 00/04/07, seeded copy byte-identical with
  full appendix. Previously only 00/08 existence was asserted.
- test_skill_seed_pinning: only SKILL.md copied (pack extras never
  land, never resurrected), tenant extras left alone, tenant-modified
  SKILL.md never overwritten across repeats, identical = noop,
  deleted SKILL.md re-seeded (documented boundary, not deletion
  preservation).
- skill_seed.py: evaluation note only (why whole-dir copy is not
  done); no code change.
2026-11-03 02:49:25 +00:00
host 06710cf47b feat: 沙箱规模——/tmp 顺序加严、注册表寻址与失联精确清理、超时边界抽取
- Docker /tmp tmpfs 移到全部 -v 之前(父挂载在前,seed 子挂载在后);
  宿主级 bwrap /tmp 仍宣告未隔离,口径见 tmp_isolation_status()
- workers 表补 host/pid/address(老库幂等补列),心跳透传与保留语义,
  _claim 写 claimed_by_worker;orphaned_containers 精确识别失联容器,
  Worker.cleanup_orphaned_containers 默认只杀 pid_verified 的精确容器名
- docker_run_timeout() 纯函数抽取(行为与原内联一致),README 补边界与
  未验证清单;新增 test_sandbox_scale.py 20 项
2026-11-03 02:49:25 +00:00
host 143ab30094 feat: cost/quota set — dual-column priced reports, creator split, hard cutoff, approval-timeout auto-threshold
- CostStore: per-model breakdown, report_by_creator (JOIN tasks.created_by),
  dual-column reports with priced_usd_micros, unpriced_models listed
  separately and a billing note (proxy price, not the relay bill);
  _report itself untouched, old exact-key tests keep passing.
- Hard cutoff: cost.quota_cutoff_reason is the single USD/unpriced-cap
  decision; worker._quota_exceeded now delegates to it (identical
  messages); claim gate, turn boundary and poll kill stop spend and
  record interrupted + quota reason + audit; auto-resume re-checks
  live quota before requeueing.
- Approval timeout (G7 ops metric): cloud/approval_timeout.py —
  default stays 600s, >20% timeout rate recommends 1800s, <5% keeps,
  mid-range keeps; stats + recommendation + info log; wired into
  /metrics (member projects, no log spam) and a global-admin
  /admin/ops/approval-timeout endpoint (sweeps overdue first).
- Server: task/project cost endpoints return dual-column reports;
  new GET /users/me/cost and GET /admin/cost-by-creator.
- Tests: cloud/tests/test_cost_quota.py, 10 tests.
2026-11-03 02:49:25 +00:00
host 0072e1ae6a skill: cross-task artifact pass with sha256 dual-channel verification
Core artifacts travel as an explicit manifest; coordinator triple-checks
(core report vs manifest vs recomputed hash) before cleanup, regression
rewrites files verbatim, recomputes sha256, stops on ARTIFACT-HASH-MISMATCH
and only then runs the same core code plus import-only regression tests.
2026-11-03 02:48:55 +00:00
host d973e63b11 feat: wire full-auto executor to server HTTP endpoints
GET /full-auto/status, GET /full-auto/config-plan, POST
/full-auto/validate-action|merge-gate|ci-fix|release-plan|
rollback-plan, POST /full-auto/execute (global admin only,
FULL_AUTO_ENABLED + verified installation + branch protection
verified via GitHub API, else 403 + audit), GET /full-auto/audit.
Guards reused from cloud/full_auto.py; token only in Authorization
header; audit via audit_record redaction into AuthStore audit_log.
2026-11-03 02:48:55 +00:00
host b4a69e1e71 refactor: remaining longfunc small extractions after #231 (no module split)
Only pure computation/validation sub-blocks of _handle_github_webhook,
_create_followup, _deliver_to_thread and _task_chat_history are extracted;
all DB writes, audits, external calls and side-effect blocks stay in the
callers, error strings/codes and response shapes verbatim. _deliver_to_thread
uses the shared goal classifier only for /goal input: its non-goal terminal
path still goes to CAS revive (not 409) and its merged path keeps its own
wording. New tests: test_longfunc_remaining.py (15).
2026-11-03 02:48:55 +00:00
host 1accfac844 refactor: extract minimal pure helpers from _continue_task, _task_wait, _task_thread_read and _metrics
_continue_task 226->168 via _validate_continue_options;
_task_wait 158->139 via _validate_wait_params;
_task_thread_read 187->152 via _summarize_trace_entries;
_metrics 123->104 via _summarize_metrics_tasks.
No behaviour, message, status or route change; error strings verbatim.
New tests: test_longfunc_extra (9). Cloud full suite: 1439 OK, skipped=4.
Follows #229 pattern; must merge after #229 and be rebased/checked then.
2026-11-03 02:48:55 +00:00
host f96960c9c5 refactor: extract minimal helpers from _dispatch, _dispatch_project and _create_task
Move the /tasks/* sub-route out of _dispatch into _dispatch_task_route
(query passed explicitly; the files/tree/thread branches use it), move
the push-subscription DELETE branch out of _dispatch_project into
_project_push_subscription_delete, and turn the goal-validation block
of _create_task into the pure module function
_validate_task_goal_options (returns error string instead of sending
400, caller sends it unchanged). Statements are moved verbatim except
indentation and the goal send+return -> return-error transform; no
behaviour change. server.py module split intentionally not done: it
remains a larger, higher-risk refactor.

Add test_longfunc_split covering the pure goal validator's success
paths and all eight error strings.
2026-11-03 02:48:55 +00:00
host a9e2f70d9e workers: reap pid_verified stale workers with pre-kill audit and post-kill verification 2026-11-03 02:48:55 +00:00
host 8ea54cf66b workers: land host/pid/address columns with idempotent migration and pid_verified stale classification
- ensure_workers_table: new tables include host TEXT, pid BIGINT,
  address TEXT; old tables gain them via _ensure_hostpid_columns
  (PRAGMA column check, missing-only ALTER, old rows keep NULL,
  never fabricates host/pid). SQLite and Postgres share this path
  (PG via the db adapter's PRAGMA translation); POSTGRES_SCHEMA
  deliberately still excludes workers, which is volatile heartbeat
  data in UNMIGRATED_TABLES (adding it broke the P2 migration
  alignment tests, verified and reverted).
- heartbeat/register: given host/pid/address refresh, omitted
  values preserve the old row; address has no derived source.
  Local worker passes host_name + own pid; HTTP heartbeat only
  stores caller-supplied values, never the server's own.
- classify_stale_workers: kill only when age strictly exceeds the
  threshold AND host matches local host AND pid is proven dead;
  equal threshold, foreign host, missing pid, or live/unknown pid
  are list_only. Classification only, never kills, never matches
  by name/prefix, never touches infra containers.
2026-11-03 02:48:55 +00:00
host 452fc8ea7e docs(skill): add software-engineering README and same-dir reference test
SKILL.md header pointed at same-dir README.md but the pack had no
such file (CHANGELOG was supplied by #217, README was not). This
track adds the README only: purpose, chapter map, appendix
provenance entry point, seeding behaviour, and known boundaries.
It states no rule, changes no SKILL.md semantics, and does not
duplicate the CHANGELOG/script explanations -- overlapping parts
are pointers, with an explicit division-of-labour section.
Deletion retention is written only as the tombstone-design
placeholder, deferring to the skill_seed implementation, because
no tombstone implementation has landed. Test pins that every
file SKILL.md cites as same-directory exists (CHANGELOG, README).
2026-11-03 02:48:55 +00:00
host 4e554b5c07 docs(skill): add appendix provenance sources, checker and CHANGELOG
SKILL.md claimed its appendix was spliced verbatim from
templates/roles/ by a generation script (see CHANGELOG), but the pack
contained none of the three. Add the nine role sources (mechanically
split from the v0.2 appendix; SKILL.md untouched, zero semantic
change), tools/check_appendix.py (verify + --emit, stdlib only, never
edits SKILL.md), CHANGELOG.md, and test_skill_appendix_provenance.py
(5 tests). Seeding unchanged: skill_seed copies SKILL.md only, so
templates/tools/CHANGELOG never reach the tenant home.
2026-11-03 02:48:55 +00:00
host 61e08a1a8c feat: hard merge gate by default (CI + directory + contract + coordinator + reviews)
- cloud/merge_gate_hard.py (new): pure hard gate; mode resolution is
  env-only (unset/invalid -> hard, fail-closed); five checks plus
  branch protection must be protected; unknown/unprotected block
  even with explicit confirm; no-evidence blocks (fail-closed);
  directory check reuses #209 check_directory_isolation (v0.3 rule);
  blocks codes + merge_gate_blocked audit detail are the track-3
  pushback interface (no pushback state machine in this track).
- cloud/server.py: POST /tasks/{id}/merge uses hard mode by default;
  record/enforce/off only when explicitly set via env (preserving
  #209 semantics); body merge_gate_mode is ignored server-side so
  callers cannot downgrade the gate; every gate block writes a
  merge_gate_blocked audit before returning 409, status unchanged;
  hard-mode body type errors return 400.
- docs/merge-gate-branch-protection.md (new): protection payload
  (PR + strict status checks + no force-push + no deletion +
  enforce_admins), empty required_checks = unconfigured, Pro/public
  prerequisites, 403 causes; states protection is NOT claimed enabled.
- SKILL.md (gate-mechanism wording only): four checks -> five checks;
  body + coordinator + client appendix gate paragraphs updated to
  default hard, merge_gate_blocked audit, payload doc reference.
- tests: 31 new hard-gate tests (pure + endpoint + skill text);
  #209 endpoint tests now set record env explicitly (their intent);
  test_server_merge 200-path calls now carry full hard evidence.
2026-11-03 02:48:55 +00:00
host d553f9549c feat: configurable merge gate (branch protection + directory isolation) and skill body/appendix checks
- cloud/merge_gate.py: pure gate; directory check reuses the v0.3
  git diff --name-only merge-base command; protection modes
  record (default, explicit confirm required when unprotected),
  enforce (always blocks when unprotected), off (explicit only).
  Confirm covers protection risk only, never directory violations.
  Protection status is caller/env-declared only; unknown is a
  recorded risk, not a claim of protection.
- server POST /tasks/{id}/merge: applies the gate only when gate
  evidence is supplied (or env declares protection); no-evidence
  calls behave as before plus a merge_gate field in the response.
  Blocked merges return 409 without changing status; confirmed
  unprotected merges write a merge_gate_risk audit entry.
- skill_seed.py: whole-directory seeding evaluated and rejected
  (cannot preserve tenant deletions; no benefit while the pack
  holds only SKILL.md); SKILL.md-only behaviour unchanged.
- software-engineering SKILL.md: gate paragraph added to the body
  and to the coordinator + client appendix sections (v0.2 text on
  this base; port to v0.3 text on rebase after #199 merges).
- tests: 16 new tests covering gate modes, directory reuse,
  body/appendix bidirectional assertions naming the missing side,
  SKILL.md-only seeding and tenant-edit preservation.
2026-11-03 02:48:55 +00:00
host 44616a03dc feat: software-engineering skill v0.5 (G7/G8 ops + iteration)
Stacked on v0.3 + v0.4 (HEAD d6522ba feat-full-auto): v0.3/v0.4
content preserved, not reverted. Adds G7 post-release ops and
observation (monitoring, errors, cost/quota, feedback, hotfix loop,
rollback triggers, ops manual) and G8 iteration and scaling (v1.1
handover list, contract v1->v2 protocol, new-end onboarding,
scale/performance, deprecation), with G6->G7 and G7->G8 gates,
DAG, gate table, coordinator derivation, G0-G8 todos, checklist,
09-ops and 10-iteration role templates, and ops-manual / v1.1
handover templates in both body and appendix. Settles web+React+
multi-viewport focus, cross-viewport E2E, web single-end release
with iOS/Android native postposed, open-source-first, and the
TypeScript full-stack default (user-request-only language change;
codex-app-server itself stays Python). Pending #199 and #205
merge, then one rebase round. PR only, do not merge or deploy.
2026-11-03 02:48:54 +00:00
host a605ff2f92 feat: full-auto mode v0.4 (host executor + skill + CI fix loop, stacked on v0.3)
Host auto executor (cloud/full_auto.py, stdlib only): default off
(FULL_AUTO_ENABLED + verified installation), forbidden actions
fail-closed (delete repo, force-push, visibility change, secret
read/expose, branch-protection delete/bypass, GitHub account
auto-registration), token only in request header, audit log
redacted, branch protection requires PR + strict checks +
no force-push + no deletions + enforce_admins, merge gate
(CI green + directory check + contract version + coordinator
approval), CI fix loop max 3 then escalate, rollback only via
revert PR or previous tag. Skill: v0.4 full-auto mode, coordinator
reviews, host executor executes, task agent never executes
merge/deploy; repo-creation honesty branch (org via App token,
user repo needs_user_once); no web auto-registration of GitHub
accounts. Tests: 19 new, cloud full 1451 OK (skipped=4).
2026-11-03 02:48:54 +00:00
host 34ed438d0e feat: software-engineering skill v0.3 (P1-1..P1-4 + P2)
- P1-1 directory-isolation executable check: git diff --name-only
  against merge-base, output recorded in merge record
- P1-2 derivation checklist table (8 required items, missing one blocks dispatch)
- P1-3 QA split into single-client review and cross-client E2E with
  fixed deliverables and agent-test attribution
- P1-4 sub-task slim-pack trimming rules (listed chapters only, never full copies)
- P2: inter-thread Q&A default policy, docs/decisions.md append-only,
  per-client onboarding docs (web first at G4a), v1.1 handover checklist
- Body and appendix kept in sync; new lightweight keyword test
2026-11-03 02:48:54 +00:00
host 49a2f7881b coverage85-step4e-fix: misc 不再 import stdlib queue,避免遮蔽 test_queue_sandbox 的 cloud/queue 导入 2026-11-03 02:48:54 +00:00
host 12f56cd8c8 coverage85-step4e: workers/autoscaler/playwright 长尾测试(+29 项) 2026-11-03 02:48:54 +00:00
host 3b7d45c9a2 coverage85-step4c: server 长尾端点与子路由测试(+19 项) 2026-11-03 02:48:54 +00:00
host 21a5fd3a2a coverage85-step4b: worker marker_tools 派发剩余分支测试(+20 项) 2026-11-03 02:48:54 +00:00
host 0e665d616f coverage85-step4a: worker run_one/_run_attempt 缺口测试(+34 项) 2026-11-03 02:48:54 +00:00
host 903f138fd9 coverage85-step2d: 新测试 lint 清理(删无用导入与变量) 2026-11-03 02:48:54 +00:00
host 3d2df7dfe6 coverage85-step2b: server/worker 剩余缺口真实行为测试(+24 项)
- server:_dispatch_project(项目 PATCH repo_url、成员邀请/改角
  色/删除与 404/403/409 族)、_create_task 校验 400 族与合法变体
  (goal/协调员/plan/环境)、trace/agents/trace-stream(终态 SSE
  snapshot+done)、GitHub webhook 去重与坏体。
- worker:_gateway_context(xcmapi/缺 origin/缺属主/假 sub2api
  key 与协调员父属主回查)、_ensure_gateway 缓存、_docker_task_
  context(代理/租户网络失败/搜索透传)、_deliver_marker_tools
 (守卫空跑、创建派生、goal 转换、轮询快照)、协调员孤儿扫尾、
  run_one/run_all/run_loop/main 空库路径。
生产代码零改动。
2026-11-03 02:48:54 +00:00
host 576fd50ee3 coverage85-step2a: MCP/deploy/verify-multiuser 剩余缺口真实行为测试(+43 项)
- playwright/imagegen/websearch 三个 MCP:handle_request 全工具分发、
  通知/错误分支、main() 进程内 stdio 循环(pipe/StringIO + 假浏
  览器/假依赖)、LineReader 分块与 EOF、真实 RemoteBridge 构造、
  flat-copy 回落导入;三文件孤立覆盖率 98%(余 __main__ 与
  通知型 handler-error else 分支,见回执)。
- deploy.py:check_clean_tree 脏/暂存/放行、archive 失败、remote
  助手、transfer 全路失败与 sha 不符、git 源冲突与干跑、tar
  路径干跑/构建/部署成功与失败;孤立 92%(余行由既有 deploy
  测试覆盖,__main__ 除外)。
- verify-multiuser main():真实本地控制面 13/13 通过且清理归零、
  死端口与 stub 项目失败早退、清理触发器异常报 leftover。
生产代码零改动。
2026-11-03 02:48:54 +00:00
host 81a58daca2 coverage85-step1: server/worker/host 主力缺口真实行为测试(+113 项)
只补 cloud/server.py、cloud/worker.py、host.py 的真实行为测试,
不动生产代码:
- server:助手函数(路径/租户/提取/链路读/配额/限流/心跳/
  host CLI stub 脚本)40 项
- worker:纯助手 + Worker DB/认领/容量/配额/跟进/租户/协调员
  孤儿/回退 token,真实 SQLite 与真实子进程 36 项
- host:假 app-server 脚本走真实 JSON-RPC 协议,run_turn
  成功/超时/中断/失败/历史错误回捞 + 状态/git/PR 助手 37 项
不含 mock 被测代码、无 skip、无 no-coverage、无空循环刷行。
2026-11-03 02:48:54 +00:00
host f39ed3654e ci3: lint/mypy 第二阶段清理 + 覆盖率 76% -> 78%(门槛不动)
基底:#251 头 174b631(堆叠于 #242 feat-ci-expand 之上),
基线 origin/main aebc6238;本轨不改 #251 已定语义、不动
workflow 门槛、不删检查、不加 skip。

lint(ruff E,F@120):109 -> 10
- F401 43 -> 0:删未用导入;try/except 双绑定的名字出现
  “打地鼠”(删 except 分支后 try 分支又被标出),两分支
  均已核对正文无引用后清掉;MCP 三模块的 Error 名不删,
  改走 importlib 属性绑定(见下),两路径绑定都保留。
- E702 40 -> 0:全在 host.py main() 的 argparse 段,纯分行,
  tokenize 按分号 token 拆(含 2377 行多行语句只断首个分号);
  一次 5 空格缩进失误已回滚重做并以 ast + 全量 host 测试验证。
- E501 13 -> 0:host.py 5 项随 E702 分行连带清掉;mcp_server
  3 项把描述拆成相邻字面量(AST 比对 24 条工具描述与原值
  完全一致,注意 ruff 按东亚宽度计长,CJK 算 2);test_host
  5 项断开 git commit 参数表。
- E402 8 -> 10:生产侧 2 项清掉(db.py _re、github_app
  _base64 移到顶部,之前无引用、无重名);剩余 10 项全是
  tests/ 的 sys.path 前置 import(6 项旧 + 4 个新测试文件
  沿用同一既定模式),移上去会破坏单文件直跑,有意保留。
- F841 4 -> 0:只去绑定、不删调用(run_task、DB 入队、
  tarfile 上下文、git 提交),各有注释与原测试/新测试覆盖;
  host dispatch_from_branch 新增真实 git 仓库回归测试。
- F811 1 -> 0:server 本地 _row_get 与 peer_messages.row_get
  逐行相同,删本地定义,全部调用点解析到同一 import;
  新增 test_server_row_get(同一对象 + 三路行为)。

mypy(生产范围):24 -> 0,零新增 type: ignore
- no-redef 9 -> 0:MCP 三模块 try/except 两次 from-import
  改 importlib.import_module + 属性绑定(cloud 优先、flat
  copy 回落、内层 ImportError 同样回落,语义与原来一致;
  flat-copy 子进程测试证实回落后 __module__ 为顶层名);
  server _row_get 随 F811 消除。
- arg-type 10 -> 0:harness2_tasks shared 改 TypedDict
  逐键精确类型;CodexProbeAdapter 补 no-op validate_tree
  (_adapter_validate 本就跳过缺失方法,行为一致)。
- var-annotated 3 -> 0:sub2api _SHARED_ADMIN/_login_slots、
  verify-multiuser RESULTS 加精确注解。
- assignment/misc 2 -> 0:marker_tools ZoneInfo 先声明
  Any 再 try import(草稿先验 mypy 通过)。

bandit:总数 286 不变,High 仍 4 项,全在 host.py,本轨不改
(属第 3 项白名单轨范围),只做 triage:
- 1319 run_verify:无插值,整串验测命令,管道/&& 须 shell
  语义(新增 test_host_run_verify 断言管道与 && 真实生效)。
- 1497 notify:模板来自算子配置,三处插值全部 shlex.quote,
  证据为既有 test_notify_failure_quotes_shell_metacharacters
  (; touch 与 $(touch ) 不执行)。
- 2099/2103 goal verify:无插值,任务配置验测串,同 run_verify。
- 无新增 High;pip-audit 0 漏洞。

覆盖率(同口径三套件 coverage 计量):75.96% -> 77.87%
(22392 行/缺 5385 -> 22437 行/缺 4963,显示 76% -> 78%)
- client.py 0% -> 90%(pipe 假子进程 + main 分支)
- run_parallel.py 0% -> 97%、migrate 包装 0% -> 95%
- playwright_mcp 37% -> 62%、imagegen_mcp 47% -> 79%、
  websearch_mcp 51% -> 86%、deploy.py 56% -> 77%
- verify-multiuser 0% -> 27%(仅 check/req helper;main()
  是主机侧真实多用户实弹验证,按门禁不建生产任务、
  不以 mock 冒充,仍为缺口)
- host.py 55% -> 57%(文件 2020 -> 2060 行系 E702 分行
  的计量口径变化,非代码膨胀)
- 未到 85%:门槛保持 75% 不动,不删低覆盖模块出统计口径;
  缺口主力仍是 server.py / worker.py / host.py 大文件,
  下一阶段须针对性深测,非本轨快赢可及。

验证:三套件 1576 -> 1629 项全过(tests 127、cloud/tests
1449 skipped=4、deploy/tests 53);ruff E9 通过;mypy 0;
bandit High 4(同上);pip-audit 0 漏洞;试跑产物已删,
提交后工作树干净。
2026-11-03 02:48:54 +00:00
host 7ba5bfaf81 ci2: fix worker F821 NameError, mark harness2 sha1 non-security, rename E741 var
- worker._tenant_dirs: playwright refresh log used undefined 'task';
  use the method argument project_id (ruff F821, real bug: NameError
  aborted tenant seeding when a playwright script was refreshed).
  Regression: cloud/tests/test_worker_playwright_seed_f821.py (2 tests;
  pre-fix run reproduces NameError at worker.py:2780).
- harness2.assign_ids: hashlib.sha1(..., usedforsecurity=False) -- the
  digest is only a deterministic shuffle seed, not a security use;
  digest value unchanged, bandit B324 no longer fires (High 5 -> 4).
- harness2.select_final: rename ambiguous loop var 'l' to 'lab'
  (ruff E741, behaviour-invariant).
- Deliberately NOT changed: host.py shell=True x4 (see PR body -- all
  four are intentional shell commands, changing semantics is unsafe),
  remaining ruff 109 / mypy 24, workflow thresholds (75% coverage
  gate unchanged; no gate was relaxed or removed).
2026-11-03 02:48:54 +00:00
openqj 7b537c31a6 test: close unclosed file handles in tests (Path read/write)
Replace bare open().read()/write() in tests with Path.read_text/
write_text (or json.loads over read_text); the five remaining
open(...).close() empty-file creations become Path.write_text('').
No assertion, predicate, or production code changed; no behaviour
change beyond closing the handle deterministically.

Long-function splits and server.py module split intentionally not
included: the longest functions (_dispatch, _create_task,
_dispatch_project, ~375-395 lines in a 10430-line server.py) are
central routing/creation paths, and splitting them is not low-risk;
see PR body.
2026-11-03 02:48:54 +00:00
host b391fe9320 resident P0+P1: DNS-level SSRF pin, batch budget, daily cap/cooldown/retention, PG schema/migration, dual-connection deadlock isolation, retarget baseline reset, proxy and structured failures, metrics
- DNS 级 SSRF pin:解析后 pin IP 建连,防 DNS 重绑定
- 批量总预算与每日上限/抖动冷却/保留期
- PG schema 与迁移清单
- 双连接死锁隔离
- 改目标重置基线
- 显式代理配置与结构化失败原因
- /metrics resident 口径
2026-11-02 03:30:57 +00:00
host 6525bcafd0 audit: resident website loop architecture and code quality to 10/10
Fix F1 name race with partial unique index, F2 pause/end mid-poll race with guarded updates, F3 redirect/numeric-IP SSRF, F4 quota prefix check, F5 duplicate row_get, F6 audit ternary; S1 unused import, S2 backoff doc, S3 defensive tearDown. Directed 20 pass, full 1700 pass.
2026-11-01 19:58:03 +00:00
host 561b3891fb test: resident website loop end-to-end with real fetch 2026-11-01 19:42:40 +00:00
host 0722ca3fd8 feat: resident watch tasks with pause/resume/end
Long-running website monitoring: new cloud/resident.py layer on top of
the automations claim/quota model (no task state-machine change).
States running/paused/ended; pause preserves the content-hash cursor
and stops polling, resume re-checks from the cursor, end is terminal
(resume after end is 409, a new watch is required). Polling is a cheap
fetch + hash comparison; only a real change creates a queued model
task. Quota exhaustion pauses (quota_exceeded) instead of ending and
keeps the cursor. Driver is the worker run_loop round boundary;
server exposes create/list/get/update/pause/resume/end with audit.
Console UI intentionally not wired this round (see verification
summary for deviations).
2026-11-01 19:01:39 +00:00
host 3a56f83114 websearch: 默认 auto 付费冻结,SearXNG 优先 + 显式解冻开关
- 默认 auto 只走免费链 searxng -> duckduckgo;即使环境里有
  付费 key 也不调用 Perplexity/Exa/Tavily/Brave,SearXNG
  失败也不回落付费,默认使用零付费花费。
- 付费后端代码/测试/文档全部保留;显式 pin(tavily/exa/
  perplexity/brave)仍照常调用,是有意覆盖不是默认。
- 解冻开关 WEBSEARCH_ENABLE_PAID,仅精确值 "1" 才恢复旧链
  (perplexity/exa/tavily/brave/searxng/duckduckgo),默认关;
  开关经 worker search_env 透传进 MCP config env。
- HttpClient 超时/DNS 失败改报显式错误(timeout after Ns /
  URL error),绝不把超时写成“无结果”。
- SearXNG 默认参数核实:代码无 SEARXNG_URL 硬编码默认值;
  SEARXNG_ENGINES 默认 bing,360search 已在基线;WEBSEARCH_TIMEOUT
  默认 20 秒。主机 env 实测仅 SEARXNG_URL=http://searx.diante.me
  一行,故默认 pin 生效;文档 https 漂移改回 http(主机 nginx
  仅听 80、无 443 vhost,https 在 TLS 层即失败)。
- 旧测试更新原因:test_websearch_mcp 原断言默认 auto 为
  perplexity 优先,与新冻结语义直接冲突,故改为断言冻结后
  的 searxng -> duckduckgo 回退 + 不泄漏 key;旧 perplexity ->
  exa 回退断言移至解冻开关测试(WEBSEARCH_ENABLE_PAID=1 下)。
  未删断言、未加 skip、未放宽门禁。
- 文档另删 WEBSEARCH_PROVIDERS 说法:代码从未实现该变量。
- 未推、未建 PR、未合并、未部署、未改主机 env/nginx/filter/
  证书、未做付费后端 live 验证、未创建生产任务。
2026-11-01 14:15:54 +00:00
host e9fa43dd8b websearch: task-level allowed_hosts extension for scope=code + JS-heavy extract evaluation (mock-first)
- search(..., allowed_hosts=[...]) extends (never replaces) the 15-domain
  code list for that call only; scope=general + allowed_hosts raises,
  empty/illegal values raise before any HTTP, dot-boundary matching,
  extension values never in error strings or attempted entries.
- Native-filter parity: Tavily/Exa get the effective list, Perplexity
  skips its native filter past its 20-domain cap (post-filter stays
  authoritative), Brave/SearXNG/DuckDuckGo rely on post-filtering.
- MCP web_search gains allowed_hosts passthrough (only when supplied,
  so older search_fn fakes keep working).
- JS-heavy evaluation documented in docs/web-search-mcp.md, mock-first:
  direct fetch is shell-only by construction; provider extract can
  supply rendered text in mock only; live SearXNG in this VM env
  failed (remote closed connection) and paid-backend live effects are
  unvalidated blockers, not claimed.
- Tests: test_code_search_ext (9) + test_js_heavy_extract (5 + 1 live
  skipped); existing websearch/code-search suites still green.

(cherry picked from commit 9c26bbe848438e1f34bcb000700aeb4c284f4f23)
2026-11-01 09:06:28 +00:00
host 125b436ab3 browser: click-path allowlist, safe tenant script upgrade, allowlist activation plan
- click/click_ref: pre-click allowlist gate on resolvable anchor href
  (same check as navigate; miss refuses before the click executes);
  click/click_ref/click_at: post-click landing URL check.
  click_at has no pre-click gate by design (coordinates name no
  element); JS-driven navigation, evaluate, meta refresh and popups
  are documented as unintercepable with the Tinyproxy 403 backstop.
- Tenant script upgrade is now safe: only proven-unmodified scripts
  (manifest match, 2a569ac known-shipped SHA-256, missing, or
  current) auto-merge; tenant-modified scripts are never overwritten,
  are reported as skipped, and get one deduped audit record. The
  manifest never learns a tenant hash (else the next upgrade would
  mistake a modified file for untouched). Idempotent reruns.
- Activation candidate = deployed Tinyproxy programming set encoded
  with leading-dot entries (plain entries would regress
  docs.python.org); npmjs.org per the deployed filter, not npmjs.com.
  Host activation itself is NOT performed this round: only the plan,
  validation tests and rollback are delivered.
Tests: 28 new (click 13, upgrade 8, activation 7), all fake/no-network;
98 browser-related tests pass including prior suites.

(cherry picked from commit e4a93f5748ec815f163d0bc096ff4dc4b8c651f5)
2026-11-01 09:06:28 +00:00
openqj 2a569ac6e3 Merge pull request #272 from openqj/feat-platform-browser
feat: platform browser multi-tab, navigation allowlist, close cleanup, egress filter, allowlist worker wiring
2026-11-01 16:42:59 +08:00
host da7a9950fb feat: platform browser multi-tab, navigation allowlist, close cleanup, egress filter, allowlist worker wiring
- Multi-tab: browser_new_page / browser_list_pages /
  browser_switch_page / browser_close_page in playwright_mcp.py;
  each new page gets a fresh Playwright context, so cookies and
  storage are isolated between pages. Single-page tools act on
  the current page; switching to a missing index is an error,
  not a crash; closing the last page is an error (use
  browser_close).
- Navigation allowlist: BROWSER_ALLOWED_HOSTS (comma list,
  leading-dot suffix match, case/port-insensitive) checked in
  _check_allowed_url before launching Chromium; unset (or empty)
  means no check, the old behaviour. file:// always allowed;
  localhost/127.0.0.1 have no special treatment.
- Close cleanup: browser_close closes all pages and best-effort
  removes cwd browser-live.png / browser-live.json; custom
  screenshot files are untouched. list_pages needs no launch.
- Egress filter: 10 programming-source Tinyproxy BASIC rules
  (python.org, nodejs.org, go.dev, pkg.go.dev, rust-lang.org,
  developer.mozilla.org, crates.io, stackoverflow.com,
  cve.mitre.org, nvd.nist.gov); already-present github.com /
  pypi.org / npmjs.org not duplicated; searx.diante.me stays
  exact-only, no diante.me wildcard.
- Allowlist worker wiring: _docker_task_context now passes a
  set, non-empty BROWSER_ALLOWED_HOSTS from the worker env into
  [mcp_servers.playwright.env] in the tenant config, so the MCP
  process actually sees it (container -e alone does not reach
  MCP subprocesses, 2026-10-30). Unset/empty writes nothing and
  never overwrites; the value never goes into logs; it is not
  added to the websearch env block or the container env.
  ensure_playwright_mcp_env now returns False without rewriting
  when the result would be identical (canonical join: one blank
  line before the block), so a second prep leaves an unchanged
  tenant config alone; other sections untouched.
- Skill + docs: browser-automation SKILL.md multi-tab/allowlist
  section; docs/platform-browser.md comparison doc (new file).
- Tests (mock/fake only, no network, no real Chromium, no real
  credentials): cloud.tests.test_playwright +
  cloud.tests.test_websearch_worker, 48 tests OK, including 6
  new wiring tests (switch off passes no variable, switch on
  reaches MCP env verbatim, empty never written, old-tenant
  merge idempotent + no-rewrite, ensure no-op when identical,
  allowlist enforces only after wiring).
- NOT done (see PR body): no persistent profile, no credential
  vault, no approval card, no CAPTCHA bypass, in-page link
  navigation does not pass the allowlist, no real-Chromium live
  verification, host tinyproxy filter not reloaded.
2026-11-01 08:34:13 +00:00
host 397a46f850 websearch: add scope=code programming search + provider extract chain
- search(..., scope=code) restricts to 15 programming domains
  (official docs, GitHub, registries, Stack Overflow, CVE/NVD):
  native include_domains/search_domain_filter where supported
  (Tavily, Exa, Perplexity), post-filtering for all providers
  (Brave, SearXNG, DuckDuckGo have no native filter).
- web_fetch now uses fetch_best: Tavily extract, then Exa
  contents, then direct fetch; both-fail errors name both
  failures, direct-fetch 403 under the egress whitelist is
  fail-closed, not bypassed. Default general scope and direct
  fetch_url behaviour unchanged.
- MCP web_search gains scope enum (general/code); docs updated.
- Tests all mock HTTP, no real keys.
2026-11-01 08:16:48 +00:00
openqj 27f48fe1ed Merge pull request #200 from openqj/feat-console-project-nav2
console: 项目导航移到左侧栏,工作台不再承担项目模块
2026-11-01 15:27:54 +08:00
openqj aebc6238a6 Merge pull request #201 from openqj/feat-platform-auto-resume
feat: auto-resume platform-interrupted non-goal tasks
2026-10-30 15:51:43 +08:00
host 08484302ac console: 项目导航移到左侧栏,工作台不再承担项目模块
左侧栏 ProjectNav:项目完整列表(名 + 短 id + 我的角色,
选中高亮,点击切换)+ 创建项目(名/GitHub 仓库/repo_url)
+ 选中项目成员(邀请/改角色/新增移除);工作台删除
sec-projects 一节与任务列表里的项目下拉框,侧栏为唯一
项目选择器,自动选第一个项目逻辑不变。
薄模块补齐(均为已核实后端路由):成员移除 DELETE、
自动化详情(GET 单条 + 产出)与编辑 PATCH、分诊条目点进
任务/自动化详情、Workers 搜索与在线筛选、管理区设置配额
POST。主线新功能(Todos、Browser、taskMeta 等)原样保留,
未做任何回退。console 标记单测本地 53 项通过。
2026-10-30 07:43:54 +00:00
host 4428a721a6 feat: auto-resume platform-interrupted non-goal tasks
Deploy restarts kill running tasks: the worker marks them
interrupted (worker_shutdown on graceful exit, sweep on stale
reclaim), and auto_resume_interrupted used to accept goal tasks
only, so every non-goal task killed by a deploy stayed
interrupted forever unless a human retried it (the 2026-10-30
demo task died this way).

Now worker_shutdown/sweep interruptions are re-queued for
non-goal tasks too: the platform killed the task, so the
platform re-queues it. The re-queue is a fresh run, not a
breakpoint resume: goal_resume_pending is forced to 0 and any
stale resume_message is cleared, so the new run cannot be
hijacked by the goal-resume path or an old continue message;
files in the same worktree persist. All existing guards stay:
user pause, coordinator orphans, quota (still goal-only),
backoff, the 3-strike stall guard (non-goal tasks have no
checkpoints, so each platform re-queue counts as no-progress),
and the auto_resume_max cap. Other non-goal reasons (NULL,
quota, user cancel) still do not auto-resume.
2026-10-30 15:39:55 +08:00
host 2d644f901b feat(console): automation panel edit and richer list
- AutomationPanel rows now show title, prompt content (160-char preview with expand/collapse), repo_url, last/next run, run_once badge and cancelled time.

- Active automations get an inline edit form (name/prompt/interval/repo) that PATCHes only changed fields via the existing automation_update endpoint; backend semantics unchanged (no last_run_at reset, cancelled stays terminal).

- Rebuilt cloud/console.html with tsc + vite (build green); automation python tests 18/18 OK.
2026-10-30 15:31:21 +08:00
openqj e2ce04ce4f Merge pull request #197 from openqj/feat-stream-waiting
feat: browser stream waits through round gaps instead of disconnecting
2026-10-30 15:29:17 +08:00