- worker._tenant_dirs: playwright refresh log used undefined 'task';
use the method argument project_id (ruff F821, real bug: NameError
aborted tenant seeding when a playwright script was refreshed).
Regression: cloud/tests/test_worker_playwright_seed_f821.py (2 tests;
pre-fix run reproduces NameError at worker.py:2780).
- harness2.assign_ids: hashlib.sha1(..., usedforsecurity=False) -- the
digest is only a deterministic shuffle seed, not a security use;
digest value unchanged, bandit B324 no longer fires (High 5 -> 4).
- harness2.select_final: rename ambiguous loop var 'l' to 'lab'
(ruff E741, behaviour-invariant).
- Deliberately NOT changed: host.py shell=True x4 (see PR body -- all
four are intentional shell commands, changing semantics is unsafe),
remaining ruff 109 / mypy 24, workflow thresholds (75% coverage
gate unchanged; no gate was relaxed or removed).
- Role models: run_improvement_via_tasks gains role_models
(probe/editor/selector/reflector) plus --model-<role> CLI flags;
missing/empty falls back to shared --model, both empty means
server default, so default behavior is unchanged.
- Sweep: run_sweep + examples/harness2_sweep.py manifest CLI with
shared library_root (library restore), per-run failure isolation
and sweep-report.json summary; preflight gates task creation.
- Deliverable assets: GET /tasks/{id}/files for local-executor
tasks (safe path resolution, UTF-8 text <=256KB, listing <=200;
docker/cloud/cross-host 409 with explicit reason, since tenant
Docker shares one /app/work per project and per-task attribution
would misattribute); probe rollout deliverable_files from the
endpoint plus DELIVERABLE FILE reply blocks (endpoint wins on
collision, bad blocks dropped), surfaced in
deliverables/<label>/files/ context and full_result summary.
- Docs: harness2.md comparison table updated for all three items.
- select_final trivial condition now k==1 and single-candidate pool,
regardless of whether a selector is provided (previously gated on
selector is None, so wired runs always paid for a useless selector
task at k=1)
- Budget gains max_sessions gate (probe + stage-attempt sessions;
fmt2 corrections and selector/reflector sessions exempt), the
backstop for unpriced models where the USD gate never binds
- validate_run_tag (charset, no leading '-', no s<N> seed prefix)
enforced at run_improvement_via_tasks and the CLI
- CLI --check preflight (base harness, source dir, prompt, tag,
control-plane /health probe) without creating tasks
- ModeResult.to_dict now includes rollout summaries so a lost probe
is visible in full_result instead of only on the server
- CLI plan counts the selector only when k>1
Port Google Research's Harness^2 method layer (model weights frozen,
only the surrounding harness is optimized): Propose-Probe-Compose with
INV/DOM/AGG editors, parallel/sequential modes, per-domain experience
library, anonymous-product selection, k<=3 hard cap and budget gate.
Execution goes only through the existing control-plane API (upload ->
task with idempotency key -> poll -> trace/cost -> cancel on timeout);
no new server endpoints, no codex flags composed, tasks inherit the
normal executor/sandbox/quota/audit path.
Wiring lessons from four real rounds against production (project
wwww, ledger task): HttpTaskClient retries transient disconnects and
502/503/429 (3 attempts); editors get a worked file-block example, an
explicit <no-change> marker and one feedback retry (.fmt2) instead of
silently accepting notes-only replies; run_task unwraps the server's
result_text JSON envelope (text field holds the real reply) — without
this, line-anchored file-block parsing could never see blocks the
agents were actually emitting. Final round (real4) fully green:
all stages landed, selection non-fallback, experience入库, probe
delivers the correct signed total 85.15.
Tests: 35 harness tests (17 method + 18 wiring incl. fake-control-plane
full round, idempotent replay, retry, format retry, envelope round).
Deliberate deviations from upstream are documented in docs/harness2.md
(no network-unrestricted/sandbox-disabled defaults, auto_apply off,
no benchmark/scorer, probe deliverable = final reply text).
host