PG proof: fix task_attempts migration on real Postgres 16.15
ci / test (push) Failing after 33m23s

- attempt_ledger.ensure: relax gap-3 outcome NOT NULL on Postgres
  (ALTER COLUMN outcome DROP NOT NULL, probe-gated); without it
  every resume write on a gap-3-migrated PG table fails with
  NotNullViolation. SQLite keeps NOT NULL (no such ALTER).
- db.PgCursor: PRAGMA table_info fake rows had the notnull flag
  inverted (NO -> 0); now matches sqlite semantics (1 = NOT NULL).
  No caller read row[3] before, so this was latent.
- ledger_onboarding: stale 'PG path unverified' notes updated to
  point at the real-PG proof.
- tests: pragma_info notnull pin (fake driver) + _column_notnull
  probe pins (fresh + gap-3 SQLite).

Proof: 28/28 checks on real Postgres 16.15 (see verification/
codex-task-attempts-pg-proof-20261106.md).
This commit is contained in:
host committed 2026-11-05 21:11:56 +00:00
1 parent 698398dafb
commit 2fcde375a4
5 files changed
+104 -16

No files matched your search

+47 -7
View File
@@ -128,9 +128,16 @@ Persistence notes:
endpoint treats a missing table exactly like "no ledger rows":
unknown rates, not an error -- the same fail-soft shape already
used for missing ``cost_records``. The Postgres upgrade path
for a legacy-shaped table (fresh ``PG_DDL`` vs. adding the
missing unified columns) has NOT been verified against a real
Postgres instance in this batch -- see the batch summary.
for a legacy-shaped table was verified against a real
Postgres 16.15 instance on 2026-11-06 (fresh ``PG_DDL``,
three legacy shapes migrated in place, both writers, PK,
idempotent ensure; see
``verification/codex-task-attempts-pg-proof-20261106.md``).
One gap-3-only step is Postgres-only by nature: a gap-3
table's ``outcome`` was created NOT NULL, and ``ensure``
relaxes it with ``ALTER COLUMN outcome DROP NOT NULL``
(see ``ensure_task_attempts_table``); on SQLite that
statement is not valid and the column stays NOT NULL.
- The read path (``GET /metrics-report``) never creates the table:
a read-only endpoint must not mutate schema. Table creation
happens only on the write path, inside ``record_attempt`` /
@@ -217,9 +224,12 @@ SQLITE_DDL = TASK_ATTEMPTS_DDL
# endpoint. It is intentionally NOT part of POSTGRES_SCHEMA (see
# the module docstring: migrations coverage must not widen silently).
# For a legacy-shaped table already on Postgres, add whichever
# unified columns are missing (see UNIFIED_COLUMN_TYPES);
# that Postgres upgrade path is NOT yet verified against a real
# Postgres instance -- see the batch summary.
# unified columns are missing (see UNIFIED_COLUMN_TYPES) and
# relax a NOT NULL outcome if the legacy shape declared one
# (see ensure_task_attempts_table); that Postgres upgrade path
# was verified against a real Postgres 16.15 instance on
# 2026-11-06 -- see verification/
# codex-task-attempts-pg-proof-20261106.md.
PG_DDL = TASK_ATTEMPTS_DDL
# Unified column set, in DDL order. The migration in
@@ -272,6 +282,17 @@ def _existing_columns(conn, table):
return set()
def _column_notnull(conn, table):
"""列名 -> 是否 NOT NULL(PRAGMA table_info 的 row[3];
SQLite 与 PG 适配层含义一致:1 = NOT NULL,0 = 可空;
探针失败时回空 dict,调用方按“无需放宽”处理)。"""
try:
return {row[1]: bool(row[3]) for row in conn.execute(
"PRAGMA table_info(%s)" % table).fetchall()}
except Exception:
return {}
def ensure_task_attempts_table(conn):
"""Create the unified table if absent; migrate a legacy
table to it if present. Idempotent; does not touch existing
@@ -287,7 +308,12 @@ def ensure_task_attempts_table(conn):
is migrated in place by adding whichever unified columns it
lacks (``UNIFIED_COLUMN_TYPES``); added columns are always
nullable or defaulted, so existing rows are never blocked,
and no row is dropped or rewritten. This replaces the
and no row is dropped or rewritten. A gap-3 table's
``outcome`` was created NOT NULL; ensure also relaxes it to
nullable on Postgres (``ALTER COLUMN outcome DROP NOT
NULL``), because the resume writer inserts outcome NULL --
without the relax every resume write on such a table fails.
This replaces the
rehearsal-era runtime column-completion workaround: the
target here is the single canonical definition, and
``cloud.attempts`` / ``cloud.db`` share that same string
@@ -303,6 +329,20 @@ def ensure_task_attempts_table(conn):
% (col, UNIFIED_COLUMN_TYPES[col]))
except Exception:
pass
# gap-3 形态的 outcome 是 NOT NULL,而统一 DDL 要求它可空
# (恢复写路不报 outcome,NOT NULL 会使它在 PG 上每次写
# 行都报 NotNullViolation)。只在探针报 NOT NULL 时放宽:
# Postgres 上是 ALTER COLUMN DROP NOT NULL(已是可空时
# 不会走到这里);SQLite 上该语句不合法,try/except 吞
# 掉——SQLite 的 gap-3 表仍保留 NOT NULL(放宽需重建表,
# 与“迁移不做重建式类型收紧”的边界一致,本批只修 PG 路径)。
if _column_notnull(conn, "task_attempts").get("outcome"):
try:
conn.execute(
"ALTER TABLE task_attempts "
"ALTER COLUMN outcome DROP NOT NULL")
except Exception:
pass
def record_attempt(conn, task_id, outcome, is_defect=False,
+1 -1
View File
@@ -733,7 +733,7 @@ class PgCursor:
info_cur.close()
self._fake_rows = [
PgRow((i, r[1], r[2],
0 if r[3] == "NO" else 1, r[4], r[5]),
1 if r[3] == "NO" else 0, r[4], r[5]),
("cid", "name", "type", "notnull",
"dflt_value", "pk"))
for i, r in enumerate(raw)]
+11 -8
View File
@@ -23,15 +23,17 @@
与 attempt_hook.PG_MARKER_DDL 两个常量(本模块不
复制 DDL 文本,复制就会漂移)。老形态表升级走
attempt_ledger.ensure_task_attempts_table 的统一列补齐
(仅当老形态建过表时需要;属防御项,PG 真实库升级
路径未实证)。
(仅当老形态建过表时需要;属防御项,真实 PG 16.15
升级路径已于 2026-11-06 实证,见 verification/
codex-task-attempts-pg-proof-20261106.md)。
4. 校验函数 verify_ledger_ready(conn):只读探针——查两
张表的存在与必需列,不建表、不写行(读路径绝不改
schema,与 metrics-report 端点的只读纪律一致)。
5. 生产 PG 手动 DDL 仍是运维侧动作:本轨无生产 PG、无
本地 PG,PG 证据与此前各轨同款——适配器级(? 占位符
经 cloud.db 翻译、DDL 仅用 PG 合法类型、无自增列),
绝不写成“生产 PG 已执行”。运维执行清单见
5. 生产 PG 手动 DDL 仍是运维侧动作:升级路径本身已在
真实 PG 16.15 上实证(见 verification/
codex-task-attempts-pg-proof-20261106.md),但“生产 PG
已执行”仍须运维在生产库上跑过才算,绝不写成已执行。
运维执行清单见
LEDGER_DDL_CHECKLIST(顺序执行 + 每步验证命令)。
零第三方依赖,只用标准库。conn 约定与 attempt_ledger
@@ -79,8 +81,9 @@ LEDGER_DDL_CHECKLIST = (
"action": "仅当表已由老形态(旧恢复 DDL、gap 3 台账 DDL"
" 或演练形态)建过时:经 attempt_ledger."
"ensure_task_attempts_table 补齐缺失的统一列"
"(SQLite 已实测;PG 真实库升级路径未实证,"
"生产 PG 上执行前须先在真实 PG 上验证)",
"(SQLite 与真实 PG 16.15 均已实测,见 verification/"
"codex-task-attempts-pg-proof-20261106.md;生产 PG"
" 上执行仍是运维侧动作)",
"ddl_source": "cloud.attempt_ledger.TASK_ATTEMPTS_DDL",
"verify": "task_attempts 含统一 schema 全部 12 列",
},
+19
View File
@@ -250,6 +250,25 @@ class AdapterFakeDriverTestCase(unittest.TestCase):
self.assertIn("%s", fake.conn.cursor_obj.executed[0][0])
self.assertNotIn("?", fake.conn.cursor_obj.executed[0][0])
def test_pragma_info_notnull_flag_matches_sqlite(self):
# PG 实证(2026-11-06)抓到的真 bug:适配层把
# information_schema.is_nullable 翻反了(NO -> 0),
# 与 sqlite PRAGMA table_info 的 notnull 语义相反。
# 此前无调用方读 row[3],故只潜伏未炸;ensure 的
# outcome 放宽依赖该标志为真,此处钉住。
rows = [(1, "task_id", "text", "NO", None, 1),
(2, "outcome", "text", "YES", None, 0)]
fake = _FakeDriver(_FakeCursor(rows=rows))
db = _make_db(fake)
conn = db.connect()
info = conn.execute(
"PRAGMA table_info(task_attempts)").fetchall()
by_name = {row[1]: row for row in info}
self.assertEqual(by_name["task_id"][3], 1)
self.assertEqual(by_name["outcome"][3], 0)
self.assertEqual([row[1] for row in info],
["task_id", "outcome"])
def test_unique_violation_maps_to_integrity_error(self):
import sqlite3
fake = _FakeDriver(_FakeCursor(fail_sequence=[_FakeUnique("dup")]))
@@ -30,6 +30,7 @@ from cloud.attempt_ledger import ( # noqa: E402
SQLITE_DDL,
TASK_ATTEMPTS_DDL,
UNIFIED_COLUMNS,
_column_notnull,
aggregate_attempts,
ensure_task_attempts_table,
read_attempt_rows,
@@ -133,6 +134,31 @@ class TestFreshAndMigrations(unittest.TestCase):
self.assertEqual(self.conn.execute(
"SELECT COUNT(*) FROM task_attempts").fetchone()[0], 1)
def test_column_notnull_probe(self):
# PG 实证(2026-11-06):ensure 的 outcome 放宽依赖
# PRAGMA row[3] 的 notnull 语义;SQLite 侧钉住探针本身。
ensure_task_attempts_table(self.conn)
flags = _column_notnull(self.conn, "task_attempts")
self.assertFalse(flags["outcome"])
self.assertTrue(flags["task_id"])
self.assertTrue(flags["attempt_no"])
self.assertTrue(flags["is_defect"])
def test_column_notnull_probe_sees_gap3_outcome(self):
# gap-3 形态的 outcome 是 NOT NULL;探针必须报 True,
# 否则 PG 上的放宽不会触发,恢复写路仍会炸。
self.conn.execute(GAP3_DDL)
flags = _column_notnull(self.conn, "task_attempts")
self.assertTrue(flags["outcome"])
# SQLite 上放宽语句不合法、被 ensure 吞掉:迁移本身
# 不抛错、列集仍齐,但 outcome 保持 NOT NULL(需重建
# 表才能改,见批总结诚实边界;本批只修 PG 路径)。
ensure_task_attempts_table(self.conn)
self.assertEqual(set(_columns(self.conn)),
set(UNIFIED_COLUMNS))
self.assertTrue(
_column_notnull(self.conn, "task_attempts")["outcome"])
def test_old_shape_migrates_with_rows_intact(self):
self.conn.execute(OLD_DDL)
self.conn.execute(