audit: unify audit-of-audit sampling rule (gap 2)
Single sampling rule; track 3 (#244) is authoritative for WHO is mandatory, track 5 (#238) for HOW the rest are drawn: - Mandatory (1.0): golden-task records + rebuild / refreeze / escalation records + third-failure near-complete still-failed cases (mandatory_triggers, full list per case). Track 3's old deterministic every-10th ordinary rule is removed (it drew zero from batches under 10 ordinary records and was predictable); track 5's old narrow mandatory list is expanded with golden / refreeze / escalate. blocked_quota stays non-mandatory. - Random tiers: normal 0.10, critical-path 0.50, seed required and recorded, ceil so a non-empty group is never rounded to zero. 0.10/0.50 are TEST DEFAULTS, not approved production values; production rates remain a user decision. - One pass, no double counting: mandatory records leave the random pools before drawing; a multi-trigger record (e.g. golden + rebuild) is sampled once with all triggers listed, and its correction follows the (original, corrected) verdict pair, never the trigger list. - cloud/audit_pushback.py::select_audit_sample is now only a record-shaped adapter over cloud/audit_sampling.py::select_sample (seed required, retired reason names critical_path-for-kinds and periodic_sample removed); AUDIT_SAMPLE_EVERY_NTH deleted. - Correction: original never mutated, correction appended, corrector != signer enforced in audit_sampling when both names are supplied; gate effect pinned (correction to pass never unblocks by itself -- schedule/cancel-and-rerun, blocked until a new passing re-audit; correction to fail re-blocks at once). Track 3 / track 5 retrigger vocabularies documented as synonyms. - Skill text (tightening 3 item 2 + correction paragraph) rewritten to the unified rule; new test file pins the boundaries (mandatory union, dedup, ceil incl. the 9-record case, seed reproducibility, golden-set construction under both sample vocabularies, adapter order/non-mutation, correction preservation on both tracks).
This commit is contained in:
1 parent
8006a6fcba
commit
a433460343
5 files changed
+546
-60
No files matched your search
+77
-25
@@ -212,9 +212,10 @@ Tightening 3 -- the audit itself is audited. (i) Independent
|
||||
re-runs use exact artifacts (#211 triple-hash), never reports
|
||||
alone. (ii) Golden tasks (human-approved only; example/sample
|
||||
files are rejected by ``validate_golden_task_set``) and
|
||||
rebuild/refreeze/escalation records are always sampled, and
|
||||
ordinary records are sampled every ``AUDIT_SAMPLE_EVERY_NTH``
|
||||
records, deterministically (``select_audit_sample``). (iii) G6
|
||||
rebuild/refreeze/escalation records are always sampled, and the
|
||||
rest are drawn by seeded random sampling (``select_audit_sample``,
|
||||
which only adapts ``cloud/audit_sampling.py::select_sample`` --
|
||||
the unified sampling rule lives there, not here). (iii) G6
|
||||
user acceptance is the final human gate and is never ticked by
|
||||
automation (``check_g6_guard``). A sampled audit found wrong is
|
||||
corrected by appending a correction record that preserves the
|
||||
@@ -282,10 +283,15 @@ MAX_TARGETED_FIX_ROUNDS_PER_CONTRACT_VERSION = 1
|
||||
# items, with no core-structure item and no broken passed item,
|
||||
# may qualify as "nearly done". More than this is not near-complete.
|
||||
NEAR_COMPLETE_MAX_FAILED_ITEMS = 2
|
||||
# Audit-of-audit sampling (tightening 3): golden tasks and
|
||||
# rebuild/refreeze/escalation records are always sampled; ordinary
|
||||
# records are sampled deterministically every Nth record.
|
||||
AUDIT_SAMPLE_EVERY_NTH = 10
|
||||
# Audit-of-audit sampling (tightening 3): the unified rule lives in
|
||||
# cloud/audit_sampling.py (mandatory = golden tasks + rebuild /
|
||||
# refreeze / escalation + third-failure near-complete still failed,
|
||||
# all at 1.0; the rest by seeded random draw, normal 0.10 /
|
||||
# critical-path 0.50, ceil so a small group is never rounded to
|
||||
# zero). The old deterministic "every 10th record" rule is
|
||||
# superseded and removed: it drew zero from batches under 10
|
||||
# ordinary records. The 0.10 / 0.50 ratios are TEST DEFAULTS, not
|
||||
# approved production values; production rates are a user decision.
|
||||
|
||||
# Gate checks, in the order they are reported. The first nine are the
|
||||
# hard trigger list from the audit-hardening brief; the tenth is the
|
||||
@@ -1151,27 +1157,66 @@ def validate_golden_task_set(tasks):
|
||||
return (not reasons, reasons)
|
||||
|
||||
|
||||
def select_audit_sample(records, golden_task_ids):
|
||||
def select_audit_sample(records, golden_task_ids, seed=None,
|
||||
normal_ratio=None, critical_ratio=None):
|
||||
"""Select audit records for audit-of-audit sampling (tightening 3).
|
||||
|
||||
Always sampled: golden tasks, and rebuild / refreeze /
|
||||
escalation records (the critical paths). Ordinary records are
|
||||
sampled deterministically every AUDIT_SAMPLE_EVERY_NTH record
|
||||
(by position, so the sample is reproducible and checkable).
|
||||
Returns a list of ``{"record": ..., "reason": ...}``.
|
||||
ADAPTER, not a second rule: this function translates records
|
||||
into ``cloud/audit_sampling.py`` cases, runs the unified
|
||||
``select_sample`` there, and translates the plan back into
|
||||
record order. The unified rule is: mandatory (1.0) = golden
|
||||
tasks + rebuild / refreeze / escalation records + third
|
||||
failure root-caused near-complete whose continuation round
|
||||
still failed; the rest by seeded random draw (normal default
|
||||
0.10, critical-path default 0.50, ceil so a non-empty group
|
||||
is never rounded to zero). ``seed`` is REQUIRED -- a missing
|
||||
seed raises ValueError, because an unseeded draw cannot be
|
||||
reproduced on review. Selection is a single pass: a mandatory
|
||||
record is removed from the random pools before drawing, so it
|
||||
consumes no random quota, and a multi-trigger record is
|
||||
returned ONCE with all its triggers in ``reasons`` (``reason``
|
||||
is the first trigger, in golden / rebuild / third-failure /
|
||||
refreeze / escalate order).
|
||||
|
||||
Retired reason names: track 3's old ``critical_path`` label
|
||||
for mandatory kinds and ``periodic_sample`` label for every
|
||||
10th record no longer exist -- mandatory kinds carry their own
|
||||
trigger names, and random draws carry ``random_critical_path``
|
||||
/ ``random_normal``. Input records are never mutated.
|
||||
|
||||
Returns a list of ``{"record": ..., "reason": ...,
|
||||
"reasons": [...]}`` in original record order.
|
||||
"""
|
||||
golden = set(golden_task_ids or [])
|
||||
from cloud.audit_sampling import select_sample as _select_sample
|
||||
|
||||
records = list(records or [])
|
||||
cases = []
|
||||
for index, record in enumerate(records):
|
||||
case = dict(record)
|
||||
# Unique per-record id for the plan; the golden check uses
|
||||
# task_id, so sharing a task_id across records is fine.
|
||||
case["id"] = "record-%d" % index
|
||||
cases.append(case)
|
||||
kwargs = {}
|
||||
if normal_ratio is not None:
|
||||
kwargs["normal_ratio"] = normal_ratio
|
||||
if critical_ratio is not None:
|
||||
kwargs["critical_ratio"] = critical_ratio
|
||||
plan = _select_sample(cases, seed, golden_task_ids=golden_task_ids,
|
||||
**kwargs)
|
||||
reasons_by_id = {}
|
||||
for entry in plan["mandatory"]:
|
||||
reasons_by_id[entry["case_id"]] = list(entry["triggers"])
|
||||
for case_id in plan["random_critical_path"]:
|
||||
reasons_by_id[case_id] = ["random_critical_path"]
|
||||
for case_id in plan["random_normal"]:
|
||||
reasons_by_id[case_id] = ["random_normal"]
|
||||
selected = []
|
||||
for index, record in enumerate(records or []):
|
||||
reason = None
|
||||
if record.get("task_id") in golden:
|
||||
reason = "golden_task"
|
||||
elif record.get("kind") in ("rebuild", "refreeze", "escalate"):
|
||||
reason = "critical_path"
|
||||
elif (index + 1) % AUDIT_SAMPLE_EVERY_NTH == 0:
|
||||
reason = "periodic_sample"
|
||||
if reason:
|
||||
selected.append({"record": record, "reason": reason})
|
||||
for index, record in enumerate(records):
|
||||
reasons = reasons_by_id.get("record-%d" % index)
|
||||
if reasons:
|
||||
selected.append({"record": record, "reason": reasons[0],
|
||||
"reasons": list(reasons)})
|
||||
return selected
|
||||
|
||||
|
||||
@@ -1188,7 +1233,14 @@ def build_audit_correction(original_entry, corrected_verdict, corrector,
|
||||
(should have blocked) re-triggers pushback and re-blocks;
|
||||
a wrong block (should have passed) schedules a re-audit --
|
||||
it does not jump straight to merge, because the correction
|
||||
itself is a review, not a full audit.
|
||||
itself is a review, not a full audit. This is the SAME pair
|
||||
rule as ``cloud/audit_sampling.py::handle_sampling_result``,
|
||||
only the vocabulary differs (there: wrong pass ->
|
||||
``reopen_audit_and_pushback``, wrong block ->
|
||||
``cancel_pushback_and_rerun_audit`` where "cancel" means
|
||||
cancel-and-rerun, never cancel-and-merge); the gate effect is
|
||||
identical and is pinned by the wiring track (#247): a
|
||||
correction to ``pass`` never unblocks by itself.
|
||||
"""
|
||||
errors = []
|
||||
signer = original_entry.get("signer")
|
||||
|
||||
Reference in new issue
Block a user