Commit Graph
31 Commits
Author SHA1 Message Date
host bed3fc0dff Project repo_url: worker can run tasks from any git repo
Projects gain repo_url (validated: https:// or git@ only, no
userinfo/password, no file:// or local paths; old DBs get the
column via idempotent ALTER in both SQLite and Postgres init).
Worker resolves a project's repo_url to a local checkout under
--repos-dir (default ./repos) before dispatch: clone if absent,
verify remote origin matches and fetch if present, never force
reset/clean, never push (push needs write credentials the
control plane does not store). Resolve failure fails the task
with a clear error, never completed. Docker workers need
repos_dir under a same-path extra mount (e.g.
/var/lib/codex-app-server/repos).
Also: driver-missing test is now mocked instead of depending
on the host really having no psycopg driver installed.
Deploy package: Caddyfile template exposing only /webhooks/*
and /oauth/* (tasks/projects/admin stay 403) plus a
webhook-secret generator script that never prints the secret.
Tests: host 27 pass, cloud 286 pass (8 skipped are
environment-dependent real-Postgres tests).
2026-10-26 04:16:11 +08:00
openqj 71510a32d4 Merge pull request #6 from openqj/hotfix-cleanup-infra
Hotfix: stale-container cleanup must not delete infra containers
2026-10-26 04:02:54 +08:00
host 29dcae4513 Hotfix: stale-container cleanup must not delete infra containers
Worker startup cleanup previously deleted every container whose
name started with codex-, including the codex-postgres database
container. On the external host the first worker restart after
the prod-grade deploy deleted codex-postgres, leaving the
control plane unable to connect to Postgres (connection refused
on 127.0.0.1:5432, server auto-restart loop).
Fix: cleanup only deletes codex-<32-hex task id> containers,
which is exactly the name DockerExecutor gives task containers;
infrastructure names like codex-postgres never match and are
never deleted. Regression test covers postgres + non-hex names.
2026-10-26 03:59:54 +08:00
openqj 5a6d136ce4 Merge pull request #5 from openqj/prod-grade-batch
Production-grade: webhook execution, Postgres backup, rate limit, worker timeout/retry
2026-10-26 03:56:40 +08:00
host 33841355b5 Production-grade: webhook execution, Postgres backup, rate limit, worker timeout/retry
GitHub webhook: delivery-id dedup persisted in webhook_deliveries,
/fix-review creates a queued task with audit (no model run in the
webhook itself), check_run/check_suite parsing without mutating
task state (CI backfill still via pr-checks).
Control plane: Postgres pg_dump backup branch + POST /admin/backup
(maintainer+, password only via PGPASSWORD env), per-key/IP rate
limit (default 120/min, 429), health returns backend + db_ok,
metrics by_status always has all seven status keys.
Worker: per-task timeout (default 1800s, --task-timeout), timeout
is failed not interrupted, limited retries (default 1, backoff,
fallback failures not re-retried), stale codex- container cleanup
on loop/CLI start.
Tests: cloud 271 (267 pass, 4 skipped are pre-existing real-
Postgres tests), host 27 pass.
2026-10-26 03:55:07 +08:00
openqj 86160ad175 Merge pull request #4 from openqj/oauth-real-login
OAuth real browser login: GET callback, userinfo, API key issuance
2026-10-26 03:36:52 +08:00
host 0ed301b010 OAuth real browser login: GET callback, userinfo, API key issuance
Add GET /oauth/{provider}/callback (code+state from query, client_id
and redirect_uri recovered from the state record) alongside the
existing POST callback. Callback now fetches real userinfo
(GitHub api.github.com/user, Google openidconnect userinfo) and
issues a usable local API key on success: new users get the
create_user key, existing users rotate (old key dead, audit
oauth_login + api_key_rotated). Key plaintext appears only once
on the success page/response; failures are 400 HTML/JSON without
secrets, and a missing client_secret fails before any network call.
State remains one-time, 10 minutes, provider/redirect_uri matched.
Tests: 10 new OAuth real-login tests pass; host 27, cloud 249
(4 skipped are the pre-existing real-Postgres tests).
Real end-to-end still needs a registered OAuth App and a public
HTTPS callback entry (server still binds localhost only).
2026-10-26 03:35:40 +08:00
openqj fbb00b8b6f Merge pull request #3 from openqj/postgres-backend
Postgres real backend: psycopg adapter, schema, migration
2026-10-26 03:28:25 +08:00
host dc3f572b2b Postgres real backend: psycopg adapter, schema, migration
Replace the PostgresDatabase skeleton with a real implementation:
psycopg (v3) first, psycopg2 second, lazy load, missing driver
still raises PostgresDriverMissingError with no silent SQLite
fallback. A sqlite3-style adapter (PgConnection/PgCursor/PgRow)
translates ? placeholders to %s outside string literals, maps
UNIQUE/FK violations, lock errors and connection failures to
explicit Postgres error types, and handles INSERT OR REPLACE,
PRAGMA table_info, AUTOINCREMENT and FTS5 refusal. POSTGRES_SCHEMA
covers all business tables with tasks result columns; server
init_db uses it under CLOUD_DB_BACKEND=postgres, SQLite path
unchanged. migrate_sqlite_to_postgres.py copies rows with
ON CONFLICT DO NOTHING and reseeds identity sequences.
Tests: cloud 239 (235 pass, 4 skipped without a real server),
host 27 pass; real-server tests skip unless CLOUD_TEST_PG_DSN
is set.
2026-10-26 03:23:06 +08:00
openqj a19e77ee44 Merge pull request #2 from openqj/host/ci-cloud-tests
host: ci-cloud-tests
2026-10-26 03:09:08 +08:00
host cb7e67ba4a CI: also run cloud tests in host-verify workflow
Extend the verify job with a second step running cloud tests, so
CI covers both suites (host 27 + cloud 223, both passing locally
outside the model sandbox; the 40 socket failures reported inside
the sandbox were sandbox artifacts, not code failures).

Generator (host.py HOST_VERIFY_WORKFLOW) and generated workflow
were updated together; template matches the generated file.
2026-10-26 03:07:56 +08:00
openqj dfd6ec4013 Merge pull request #1 from openqj/chore/gitignore-state-backups
Ignore host-state backup files
2026-10-26 03:01:19 +08:00
host 19cf44551a Skip server-restart test when codex binary is absent
CI failed on the first real PR with FileNotFoundError: 'codex':
the test starts the real app-server, but GitHub runners have no
codex binary on PATH (locally it passes because the VM has one).
Skipping when shutil.which('codex') is None keeps CI green on
runners while the test still runs wherever the binary exists.
2026-10-26 03:00:30 +08:00
host 50bc03cd51 Make host tests independent of global git identity
CI failed on the first real PR because five test commits relied on
a global user.name/user.email, which GitHub runners do not set.
Pass -c user.name=Test -c user.email=test@example.com on every
test commit, matching the helper that already does this for the
other tests. host.py worktree commits inherit the repo config
set by _init_git_repo, so they needed no change.
2026-10-26 02:59:06 +08:00
host b290531369 Ignore host-state backup files
host-state-real.json.bak* kept appearing as untracked files in
git status (the base names were ignored, the .bak rotations were
not). Ignore the rotation files so a future git add does not
accidentally commit local state backups.
2026-10-26 02:49:58 +08:00
host 07a8eaaa73 Docker sandbox real test + cloud merge status endpoint
- DockerExecutor: extra mounts (host.py work/state, repo, codex-home
  at same paths), worker --docker-extra-mounts, container user
  configurable; Dockerfile copies codex-resources beside the codex
  binary and puts bwrap on PATH (previous /usr/local/lib layout
  made codex report bubblewrap unavailable inside the container).
- Real Docker task on external host: triple added in ~18s, merged
  inside a container, main repo now add+mul2+triple, --rm cleanup.
  Honest boundary: this host/kernel forbids unprivileged user
  namespaces, so bwrap fails on host and in containers; production
  mode here is Docker isolation + inner danger-full-access, not
  double sandboxing.
- Cloud merge status: POST /tasks/{id}/merge syncs status only
  (merge itself stays with host.py merge); completed -> merged,
  maintainer+ (approve_merge), other states and repeat calls 409,
  audit merge_task. Tests: cloud 223, host 27, all passing.
2026-10-26 02:46:10 +08:00
host 631580beab cloud: production gaps (worker loop + true cancel, task result columns + migration, DB interface migration, metrics + backup) 2026-10-26 01:53:33 +08:00
host 07ef869267 cloud: worker dispatch, DB abstraction, packaging 0.1.0, repo CI workflow 2026-10-26 01:44:15 +08:00
host 076ba01c74 cloud: fill gaps (memory/KB/cost/models endpoints, WAL concurrency, OAuth scaffold, push subscriptions) 2026-10-26 01:32:57 +08:00
host b74d99e959 cloud: production control plane (API, auth/RBAC, queue, sandbox, memory/KB, fallback/cost, webhooks) 2026-10-26 01:23:21 +08:00
host b97c34465f host: GitHub PR/CI line (github-workflow, pr, pr-checks, pr-merge with checks gate) 2026-10-26 01:12:04 +08:00
host 2f326f33f8 docs: cloud architecture design (remote sandbox, dispatch, GitHub/Slack/push, multi-user, memory/KB, fallback/cost) 2026-10-26 01:07:49 +08:00
host 240eaacf50 host: phase2 daemon + health + auto-recovery (restore, recover, double-fork) 2026-10-26 00:57:01 +08:00
host e9735d1fad host: fill second-class gaps (merge verify, deps, review comments, reconnect, budget, notify, state version) 2026-10-26 00:48:27 +08:00
host ed55c9651a docs: record real-project run 2026-10-26 2026-10-26 00:17:31 +08:00
host 73fdead337 host: real-backup 2026-10-26 00:16:54 +08:00
host 7f4e2edb3c host: real-tests 2026-10-26 00:16:54 +08:00
host 5731675153 host: real-backup 2026-10-26 00:16:54 +08:00
host 4ef3539bb9 host: real-tests 2026-10-26 00:16:54 +08:00
host d1e842b90f ignore local host configs 2026-10-26 00:11:46 +08:00
host b49b9cf1e7 initial real project snapshot 2026-10-26 00:11:40 +08:00