PG proof: fix task_attempts migration on real Postgres 16.15
ci / test (push) Failing after 33m23s

- attempt_ledger.ensure: relax gap-3 outcome NOT NULL on Postgres
  (ALTER COLUMN outcome DROP NOT NULL, probe-gated); without it
  every resume write on a gap-3-migrated PG table fails with
  NotNullViolation. SQLite keeps NOT NULL (no such ALTER).
- db.PgCursor: PRAGMA table_info fake rows had the notnull flag
  inverted (NO -> 0); now matches sqlite semantics (1 = NOT NULL).
  No caller read row[3] before, so this was latent.
- ledger_onboarding: stale 'PG path unverified' notes updated to
  point at the real-PG proof.
- tests: pragma_info notnull pin (fake driver) + _column_notnull
  probe pins (fresh + gap-3 SQLite).

Proof: 28/28 checks on real Postgres 16.15 (see verification/
codex-task-attempts-pg-proof-20261106.md).
This commit is contained in:
host committed 2026-11-05 21:11:56 +00:00
1 parent 698398dafb
commit 2fcde375a4
5 files changed
+104 -16

No files matched your search

+47 -7
View File
@@ -128,9 +128,16 @@ Persistence notes:
endpoint treats a missing table exactly like "no ledger rows": endpoint treats a missing table exactly like "no ledger rows":
unknown rates, not an error -- the same fail-soft shape already unknown rates, not an error -- the same fail-soft shape already
used for missing ``cost_records``. The Postgres upgrade path used for missing ``cost_records``. The Postgres upgrade path
for a legacy-shaped table (fresh ``PG_DDL`` vs. adding the for a legacy-shaped table was verified against a real
missing unified columns) has NOT been verified against a real Postgres 16.15 instance on 2026-11-06 (fresh ``PG_DDL``,
Postgres instance in this batch -- see the batch summary. three legacy shapes migrated in place, both writers, PK,
idempotent ensure; see
``verification/codex-task-attempts-pg-proof-20261106.md``).
One gap-3-only step is Postgres-only by nature: a gap-3
table's ``outcome`` was created NOT NULL, and ``ensure``
relaxes it with ``ALTER COLUMN outcome DROP NOT NULL``
(see ``ensure_task_attempts_table``); on SQLite that
statement is not valid and the column stays NOT NULL.
- The read path (``GET /metrics-report``) never creates the table: - The read path (``GET /metrics-report``) never creates the table:
a read-only endpoint must not mutate schema. Table creation a read-only endpoint must not mutate schema. Table creation
happens only on the write path, inside ``record_attempt`` / happens only on the write path, inside ``record_attempt`` /
@@ -217,9 +224,12 @@ SQLITE_DDL = TASK_ATTEMPTS_DDL
# endpoint. It is intentionally NOT part of POSTGRES_SCHEMA (see # endpoint. It is intentionally NOT part of POSTGRES_SCHEMA (see
# the module docstring: migrations coverage must not widen silently). # the module docstring: migrations coverage must not widen silently).
# For a legacy-shaped table already on Postgres, add whichever # For a legacy-shaped table already on Postgres, add whichever
# unified columns are missing (see UNIFIED_COLUMN_TYPES); # unified columns are missing (see UNIFIED_COLUMN_TYPES) and
# that Postgres upgrade path is NOT yet verified against a real # relax a NOT NULL outcome if the legacy shape declared one
# Postgres instance -- see the batch summary. # (see ensure_task_attempts_table); that Postgres upgrade path
# was verified against a real Postgres 16.15 instance on
# 2026-11-06 -- see verification/
# codex-task-attempts-pg-proof-20261106.md.
PG_DDL = TASK_ATTEMPTS_DDL PG_DDL = TASK_ATTEMPTS_DDL
# Unified column set, in DDL order. The migration in # Unified column set, in DDL order. The migration in
@@ -272,6 +282,17 @@ def _existing_columns(conn, table):
return set() return set()
def _column_notnull(conn, table):
"""列名 -> 是否 NOT NULL(PRAGMA table_info 的 row[3];
SQLite 与 PG 适配层含义一致:1 = NOT NULL,0 = 可空;
探针失败时回空 dict,调用方按“无需放宽”处理)。"""
try:
return {row[1]: bool(row[3]) for row in conn.execute(
"PRAGMA table_info(%s)" % table).fetchall()}
except Exception:
return {}
def ensure_task_attempts_table(conn): def ensure_task_attempts_table(conn):
"""Create the unified table if absent; migrate a legacy """Create the unified table if absent; migrate a legacy
table to it if present. Idempotent; does not touch existing table to it if present. Idempotent; does not touch existing
@@ -287,7 +308,12 @@ def ensure_task_attempts_table(conn):
is migrated in place by adding whichever unified columns it is migrated in place by adding whichever unified columns it
lacks (``UNIFIED_COLUMN_TYPES``); added columns are always lacks (``UNIFIED_COLUMN_TYPES``); added columns are always
nullable or defaulted, so existing rows are never blocked, nullable or defaulted, so existing rows are never blocked,
and no row is dropped or rewritten. This replaces the and no row is dropped or rewritten. A gap-3 table's
``outcome`` was created NOT NULL; ensure also relaxes it to
nullable on Postgres (``ALTER COLUMN outcome DROP NOT
NULL``), because the resume writer inserts outcome NULL --
without the relax every resume write on such a table fails.
This replaces the
rehearsal-era runtime column-completion workaround: the rehearsal-era runtime column-completion workaround: the
target here is the single canonical definition, and target here is the single canonical definition, and
``cloud.attempts`` / ``cloud.db`` share that same string ``cloud.attempts`` / ``cloud.db`` share that same string
@@ -303,6 +329,20 @@ def ensure_task_attempts_table(conn):
% (col, UNIFIED_COLUMN_TYPES[col])) % (col, UNIFIED_COLUMN_TYPES[col]))
except Exception: except Exception:
pass pass
# gap-3 形态的 outcome 是 NOT NULL,而统一 DDL 要求它可空
# (恢复写路不报 outcome,NOT NULL 会使它在 PG 上每次写
# 行都报 NotNullViolation)。只在探针报 NOT NULL 时放宽:
# Postgres 上是 ALTER COLUMN DROP NOT NULL(已是可空时
# 不会走到这里);SQLite 上该语句不合法,try/except 吞
# 掉——SQLite 的 gap-3 表仍保留 NOT NULL(放宽需重建表,
# 与“迁移不做重建式类型收紧”的边界一致,本批只修 PG 路径)。
if _column_notnull(conn, "task_attempts").get("outcome"):
try:
conn.execute(
"ALTER TABLE task_attempts "
"ALTER COLUMN outcome DROP NOT NULL")
except Exception:
pass
def record_attempt(conn, task_id, outcome, is_defect=False, def record_attempt(conn, task_id, outcome, is_defect=False,
+1 -1
View File
@@ -733,7 +733,7 @@ class PgCursor:
info_cur.close() info_cur.close()
self._fake_rows = [ self._fake_rows = [
PgRow((i, r[1], r[2], PgRow((i, r[1], r[2],
0 if r[3] == "NO" else 1, r[4], r[5]), 1 if r[3] == "NO" else 0, r[4], r[5]),
("cid", "name", "type", "notnull", ("cid", "name", "type", "notnull",
"dflt_value", "pk")) "dflt_value", "pk"))
for i, r in enumerate(raw)] for i, r in enumerate(raw)]
+11 -8
View File
@@ -23,15 +23,17 @@
与 attempt_hook.PG_MARKER_DDL 两个常量(本模块不 与 attempt_hook.PG_MARKER_DDL 两个常量(本模块不
复制 DDL 文本,复制就会漂移)。老形态表升级走 复制 DDL 文本,复制就会漂移)。老形态表升级走
attempt_ledger.ensure_task_attempts_table 的统一列补齐 attempt_ledger.ensure_task_attempts_table 的统一列补齐
(仅当老形态建过表时需要;属防御项,PG 真实库升级 (仅当老形态建过表时需要;属防御项,真实 PG 16.15
路径未实证)。 升级路径已于 2026-11-06 实证,见 verification/
codex-task-attempts-pg-proof-20261106.md)。
4. 校验函数 verify_ledger_ready(conn):只读探针——查两 4. 校验函数 verify_ledger_ready(conn):只读探针——查两
张表的存在与必需列,不建表、不写行(读路径绝不改 张表的存在与必需列,不建表、不写行(读路径绝不改
schema,与 metrics-report 端点的只读纪律一致)。 schema,与 metrics-report 端点的只读纪律一致)。
5. 生产 PG 手动 DDL 仍是运维侧动作:本轨无生产 PG、无 5. 生产 PG 手动 DDL 仍是运维侧动作:升级路径本身已在
本地 PG,PG 证据与此前各轨同款——适配器级(? 占位符 真实 PG 16.15 上实证(见 verification/
经 cloud.db 翻译、DDL 仅用 PG 合法类型、无自增列), codex-task-attempts-pg-proof-20261106.md),但“生产 PG
绝不写成“生产 PG 已执行”。运维执行清单见 已执行”仍须运维在生产库上跑过才算,绝不写成已执行。
运维执行清单见
LEDGER_DDL_CHECKLIST(顺序执行 + 每步验证命令)。 LEDGER_DDL_CHECKLIST(顺序执行 + 每步验证命令)。
零第三方依赖,只用标准库。conn 约定与 attempt_ledger 零第三方依赖,只用标准库。conn 约定与 attempt_ledger
@@ -79,8 +81,9 @@ LEDGER_DDL_CHECKLIST = (
"action": "仅当表已由老形态(旧恢复 DDL、gap 3 台账 DDL" "action": "仅当表已由老形态(旧恢复 DDL、gap 3 台账 DDL"
" 或演练形态)建过时:经 attempt_ledger." " 或演练形态)建过时:经 attempt_ledger."
"ensure_task_attempts_table 补齐缺失的统一列" "ensure_task_attempts_table 补齐缺失的统一列"
"(SQLite 已实测;PG 真实库升级路径未实证," "(SQLite 与真实 PG 16.15 均已实测,见 verification/"
"生产 PG 上执行前须先在真实 PG 上验证)", "codex-task-attempts-pg-proof-20261106.md;生产 PG"
" 上执行仍是运维侧动作)",
"ddl_source": "cloud.attempt_ledger.TASK_ATTEMPTS_DDL", "ddl_source": "cloud.attempt_ledger.TASK_ATTEMPTS_DDL",
"verify": "task_attempts 含统一 schema 全部 12 列", "verify": "task_attempts 含统一 schema 全部 12 列",
}, },
+19
View File
@@ -250,6 +250,25 @@ class AdapterFakeDriverTestCase(unittest.TestCase):
self.assertIn("%s", fake.conn.cursor_obj.executed[0][0]) self.assertIn("%s", fake.conn.cursor_obj.executed[0][0])
self.assertNotIn("?", fake.conn.cursor_obj.executed[0][0]) self.assertNotIn("?", fake.conn.cursor_obj.executed[0][0])
def test_pragma_info_notnull_flag_matches_sqlite(self):
# PG 实证(2026-11-06)抓到的真 bug:适配层把
# information_schema.is_nullable 翻反了(NO -> 0),
# 与 sqlite PRAGMA table_info 的 notnull 语义相反。
# 此前无调用方读 row[3],故只潜伏未炸;ensure 的
# outcome 放宽依赖该标志为真,此处钉住。
rows = [(1, "task_id", "text", "NO", None, 1),
(2, "outcome", "text", "YES", None, 0)]
fake = _FakeDriver(_FakeCursor(rows=rows))
db = _make_db(fake)
conn = db.connect()
info = conn.execute(
"PRAGMA table_info(task_attempts)").fetchall()
by_name = {row[1]: row for row in info}
self.assertEqual(by_name["task_id"][3], 1)
self.assertEqual(by_name["outcome"][3], 0)
self.assertEqual([row[1] for row in info],
["task_id", "outcome"])
def test_unique_violation_maps_to_integrity_error(self): def test_unique_violation_maps_to_integrity_error(self):
import sqlite3 import sqlite3
fake = _FakeDriver(_FakeCursor(fail_sequence=[_FakeUnique("dup")])) fake = _FakeDriver(_FakeCursor(fail_sequence=[_FakeUnique("dup")]))
@@ -30,6 +30,7 @@ from cloud.attempt_ledger import ( # noqa: E402
SQLITE_DDL, SQLITE_DDL,
TASK_ATTEMPTS_DDL, TASK_ATTEMPTS_DDL,
UNIFIED_COLUMNS, UNIFIED_COLUMNS,
_column_notnull,
aggregate_attempts, aggregate_attempts,
ensure_task_attempts_table, ensure_task_attempts_table,
read_attempt_rows, read_attempt_rows,
@@ -133,6 +134,31 @@ class TestFreshAndMigrations(unittest.TestCase):
self.assertEqual(self.conn.execute( self.assertEqual(self.conn.execute(
"SELECT COUNT(*) FROM task_attempts").fetchone()[0], 1) "SELECT COUNT(*) FROM task_attempts").fetchone()[0], 1)
def test_column_notnull_probe(self):
# PG 实证(2026-11-06):ensure 的 outcome 放宽依赖
# PRAGMA row[3] 的 notnull 语义;SQLite 侧钉住探针本身。
ensure_task_attempts_table(self.conn)
flags = _column_notnull(self.conn, "task_attempts")
self.assertFalse(flags["outcome"])
self.assertTrue(flags["task_id"])
self.assertTrue(flags["attempt_no"])
self.assertTrue(flags["is_defect"])
def test_column_notnull_probe_sees_gap3_outcome(self):
# gap-3 形态的 outcome 是 NOT NULL;探针必须报 True,
# 否则 PG 上的放宽不会触发,恢复写路仍会炸。
self.conn.execute(GAP3_DDL)
flags = _column_notnull(self.conn, "task_attempts")
self.assertTrue(flags["outcome"])
# SQLite 上放宽语句不合法、被 ensure 吞掉:迁移本身
# 不抛错、列集仍齐,但 outcome 保持 NOT NULL(需重建
# 表才能改,见批总结诚实边界;本批只修 PG 路径)。
ensure_task_attempts_table(self.conn)
self.assertEqual(set(_columns(self.conn)),
set(UNIFIED_COLUMNS))
self.assertTrue(
_column_notnull(self.conn, "task_attempts")["outcome"])
def test_old_shape_migrates_with_rows_intact(self): def test_old_shape_migrates_with_rows_intact(self):
self.conn.execute(OLD_DDL) self.conn.execute(OLD_DDL)
self.conn.execute( self.conn.execute(