Startup race: concurrent init_db (cloud-server + worker) ran
CREATE TABLE / CREATE INDEX IF NOT EXISTS against a DB where the
object did not exist yet; the check-then-create is not atomic, so
both sessions created it and one died with UniqueViolation on
pg_type_typname_nsp_index (tables) or pg_class_relname_nsp_index
(indexes). Hit on the 37ba687 and 7983567 deploys (7983567 added
idx_tasks_parent); systemd restart healed it every time.
Fix (advisory lock, not catch-duplicate_object): init_postgres_schema
takes pg_advisory_xact_lock (key 20261108, released at its commit),
and server.init_db takes the session-level lock for the whole PG
branch so the _ensure_* objects outside POSTGRES_SCHEMA
(task_continue_messages, user_gateway_keys, task_attempts,
task_audit_loop_state) are covered too. Same idiom as the existing
RATE_LIMIT_SWEEP_ADVISORY_KEY sweep lock; key is distinct.
Tests: two new real-PG regression tests (skipped without
CLOUD_TEST_PG_DSN), 4 processes racing on an empty schema, for
init_postgres_schema and for full init_db; both fail on the
unfixed code and pass with the fix.
- db: translate INSERT OR IGNORE ... SELECT to ON CONFLICT DO NOTHING;
register task_audit_loop_state (task_id) in _OR_REPLACE_PK (direct port).
- server: _backfill_default_quotas uses SAVEPOINT containment, no longer
commits the caller's transaction (direct port; callers are init_db only).
- worker: NOT ported — integration _coord_child_rows has no column-fallback
retry loop, so the InFailedSqlTransaction cascade site does not exist here.
- tests: rewritten for the integration base (no cloud.audit_loop /
_coord_audit_loop_merge here); real-PG cases skip without DSN/driver.
(cherry picked from commit 79e6843610fcd275af874e7b4afc57af7ffa579e)
- db: translate INSERT OR IGNORE ... SELECT to ON CONFLICT DO NOTHING
(VALUES-only regex left SELECT form untranslated -> PG syntax error);
register task_audit_loop_state (task_id) in _OR_REPLACE_PK so PG
INSERT OR REPLACE becomes a real upsert instead of DO NOTHING.
- server: _backfill_default_quotas uses SAVEPOINT containment and no
longer commits the caller's transaction, so a backfill failure cannot
roll back init_db's migrate ALTERs.
- worker: _coord_child_rows rolls back before column-fallback retries so
a missing-column failure on PG does not cascade into
InFailedSqlTransaction.
- tests: test_pg_init_fix.py (translation, SQLite backfill/upsert,
real-PG fresh-init + merge + upsert + backfill containment, skipped
without CLOUD_TEST_PG_DSN/driver).