ci: expand host-verify with lint, typecheck, security, depscan and coverage gates
Phase 1 of audit-hardening CI expansion (baseline origin/main aebc623):
- lint: ruff E9 hard gate; full E,F@120 report-only (baseline 111 items)
- typecheck: mypy default mode, production scope, report-only (baseline 24)
- security: bandit production scope, report-only (baseline 287, High 5)
- depscan: pip-audit on requirements-ci.txt, hard gate (0 vulns; repo has
no production dependency manifest, see summary)
- coverage: production 58 files, fail-under=75 (baseline measured 76%)
- release build: E9 + pip-audit hard gates before packaging; full reports
stay in host-verify to avoid drift
No production code changed; CI billing failure mode unchanged.
This commit is contained in:
1 parent
7b537c31a6
commit
c127beaa98
6 files changed
+204
No files matched your search
+27
@@ -0,0 +1,27 @@
|
||||
# Coverage 配置:只计量生产代码,测试本身、工作区、示例不计。
|
||||
# 阈值不在此文件写死,写在 workflow 的 --fail-under 里,便于在
|
||||
# workflow diff 中一眼看到门槛变化。
|
||||
# source 注意:不能写 host.py / client.py / run_parallel.py 这种文件
|
||||
# 路径——coverage 会把它们当模块名处理、静默丢弃(实测:那样 host.py
|
||||
# 等 3 个根文件根本不进报告,总数少 2223 行)。正确写法是目录源
|
||||
# “.”(含根模块与 cloud 包)+ deploy(非包目录,需单独列),
|
||||
# 实测恰好 58 个生产文件、22392 行,与 mypy 生产文件数一致。
|
||||
[run]
|
||||
source =
|
||||
.
|
||||
deploy
|
||||
omit =
|
||||
*/tests/*
|
||||
*/test_*.py
|
||||
cloud/tests/*
|
||||
deploy/tests/*
|
||||
tests/*
|
||||
work/*
|
||||
examples/*
|
||||
|
||||
[report]
|
||||
show_missing = False
|
||||
exclude_lines =
|
||||
pragma: no cover
|
||||
def __repr__
|
||||
raise NotImplementedError
|
||||
@@ -3,6 +3,23 @@ on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
# CI 扩项(审计加固第 2 步,基线 origin/main aebc623):
|
||||
# 每个新检查的生效范围与存量如实写在步骤注释里,不写空转检查:
|
||||
# - lint 全量 E,F@120:基线 111 项,见 PR 摘要存量清单,第一阶段
|
||||
# 只报告不阻塞(continue-on-error),语法 E9 仍是硬门禁;
|
||||
# - mypy 默认模式、生产范围:基线 24 项,第一阶段只报告不阻塞;
|
||||
# - bandit 生产范围:基线 287 项(High 5),第一阶段只报告不阻塞;
|
||||
# - pip-audit:只审计 requirements-ci.txt(仓库无生产依赖清单,
|
||||
# 生产代码除宿主延迟加载的 psycopg 驱动外只用标准库,见摘要),
|
||||
# 该文件 0 已知漏洞时硬通过,有漏洞即硬失败;
|
||||
# - coverage:阈值见 coverage 作业的 --fail-under,取实测值向下取整,
|
||||
# 防回归不冒充高覆盖;收紧计划见 PR 摘要。
|
||||
# 账单说明:CI 目前仍是 GitHub 账单故障秒级失败模式,账单恢复是
|
||||
# 用户在 GitHub Billing 侧的动作;本 workflow 的改动不声称改了账单、
|
||||
# 也不为变绿绕过任何检查,账单恢复后上述检查才实际生效。
|
||||
# 失败即停:同一作业内步骤按顺序执行,前一步硬失败则后一步不跑;
|
||||
# 报告类步骤用 continue-on-error + always() 上传报告,作业本身仍绿,
|
||||
# 但报告 artifact 里是失败的原始输出,不是“通过”。
|
||||
jobs:
|
||||
verify:
|
||||
runs-on: ubuntu-latest
|
||||
@@ -14,3 +31,120 @@ jobs:
|
||||
- run: python3 -m unittest discover -s tests -v
|
||||
- run: python3 -m unittest discover -s cloud/tests -v
|
||||
- run: python3 -m unittest discover -s deploy/tests -v
|
||||
lint:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- run: pip install -r requirements-ci.txt
|
||||
# 硬门禁:语法错误。基线 0 项,通过才继续;不是空集,
|
||||
# E9 覆盖 SyntaxError / IndentationError 等编译级错误。
|
||||
- name: lint 硬门禁(ruff E9 语法)
|
||||
run: ruff check --select E9 .
|
||||
# 报告类:全量 E,F@120,基线 111 项(F401 43、E702 40、E501 13、
|
||||
# E402 8、F841 4、F811 1、F821 1、E741 1)。第二阶段清零存量后
|
||||
# 删掉 continue-on-error,转硬门禁;删之前不声称 lint 已通过。
|
||||
- name: lint 全量报告(ruff E,F)
|
||||
continue-on-error: true
|
||||
run: ruff check . | tee ruff-report.txt
|
||||
- name: 上传 lint 报告
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ruff-report
|
||||
path: ruff-report.txt
|
||||
if-no-files-found: warn
|
||||
retention-days: 30
|
||||
typecheck:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- run: pip install -r requirements-ci.txt
|
||||
# 报告类:mypy 默认模式、生产范围(mypy.ini 排除 tests/work)。
|
||||
# 基线 24 项(8 文件,见 PR 摘要)。第二阶段清零后删掉
|
||||
# continue-on-error,转硬门禁;不改生产代码凑通过。
|
||||
- name: 类型检查报告(mypy 生产范围)
|
||||
continue-on-error: true
|
||||
run: mypy host.py cloud deploy client.py run_parallel.py | tee mypy-report.txt
|
||||
- name: 上传类型检查报告
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: mypy-report
|
||||
path: mypy-report.txt
|
||||
if-no-files-found: warn
|
||||
retention-days: 30
|
||||
security:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- run: pip install -r requirements-ci.txt
|
||||
# 报告类:bandit 生产范围,测试目录排除(测试故意用 shell/tmp)。
|
||||
# 基线 287 项:Low 235、Medium 47、High 5。High 5 = host.py 的
|
||||
# B602 shell=True 4 处(审计已知 P0)+ cloud/harness2.py 的
|
||||
# B324 sha1 非安全用途 1 处。先修 High 再转硬门禁;不通过
|
||||
# --skip 跳过 B602/B324 来假装通过。
|
||||
- name: 安全扫描报告(bandit 生产范围)
|
||||
continue-on-error: true
|
||||
# bandit 有发现即 exit 1:该失败是步骤的真实状态,作业靠
|
||||
# continue-on-error 继续到上传;不要用 `; cat` 吞掉 exit 码。
|
||||
run: bandit -r host.py cloud deploy client.py run_parallel.py -x cloud/tests,tests,deploy/tests -f txt -o bandit-report.txt
|
||||
- name: 上传安全扫描报告
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: bandit-report
|
||||
path: bandit-report.txt
|
||||
if-no-files-found: warn
|
||||
retention-days: 30
|
||||
depscan:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- run: pip install -r requirements-ci.txt
|
||||
# 硬门禁:审计 CI 工具链清单本身。基线 0 已知漏洞。
|
||||
# 局限如实说明:仓库没有生产依赖清单可审计——生产代码只用
|
||||
# 标准库,psycopg/psycopg2 只在宿主侧延迟加载、版本不在仓库
|
||||
# 里;给生产依赖建清单并审计是第二阶段项,不是本作业的覆盖
|
||||
# 范围,本作业不声称覆盖了生产依赖。
|
||||
- name: 依赖扫描(pip-audit,CI 工具链清单)
|
||||
run: pip-audit -r requirements-ci.txt
|
||||
coverage:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- run: pip install -r requirements-ci.txt
|
||||
# 硬门禁:三套 unittest 在 coverage 下重跑(与 verify 作业同款
|
||||
# 测试,只是加了计量),生产代码覆盖率低于门槛即失败。
|
||||
# 门槛取基线实测值向下取整到 5 的倍数:基线实测 76%
|
||||
# (22392 行,生产 58 文件),故门槛 75%,见 PR 摘要。
|
||||
# 目的是防回归,不是 0 空转,也不是未经实测的 80%。
|
||||
- name: 覆盖率计量与门槛
|
||||
run: |
|
||||
coverage run -m unittest discover -s tests
|
||||
coverage run --append -m unittest discover -s cloud/tests
|
||||
coverage run --append -m unittest discover -s deploy/tests
|
||||
coverage report --fail-under=75
|
||||
coverage xml -o coverage.xml
|
||||
- name: 上传覆盖率报告
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: coverage-xml
|
||||
path: coverage.xml
|
||||
if-no-files-found: warn
|
||||
retention-days: 30
|
||||
@@ -42,6 +42,16 @@ jobs:
|
||||
python3 -m unittest discover -s tests -v
|
||||
python3 -m unittest discover -s cloud/tests -v
|
||||
python3 -m unittest discover -s deploy/tests -v
|
||||
# CI 扩项在发布链路的最小硬门禁:语法 E9 + CI 工具链依赖扫描。
|
||||
# 全量 lint/mypy/bandit 报告与覆盖率门槛只在 host-verify 里跑
|
||||
# (PR 与 main 推送都会跑,走到 tag 的提交此前已过 host-verify),
|
||||
# 这里不重复跑三套报告,避免发布作业与 host-verify 的检查漂移;
|
||||
# 这里的两项是硬失败,失败则不产发布包。
|
||||
- name: CI 扩项硬门禁(语法 + 依赖扫描)
|
||||
run: |
|
||||
pip install -r requirements-ci.txt
|
||||
ruff check --select E9 .
|
||||
pip-audit -r requirements-ci.txt
|
||||
- name: 构建发布包(git archive + DEPLOYED_COMMIT + sha256)
|
||||
id: build
|
||||
run: |
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# Mypy 配置:默认(非 strict)模式。
|
||||
# 代码库基本无类型注解,strict / disallow-untyped-defs 在基线下会
|
||||
# 淹没真实错误,故第一阶段只用默认模式 + 缺失第三方导入忽略。
|
||||
# ignore_missing_imports:psycopg 等驱动是宿主侧延迟加载,CI 环境
|
||||
# 不保证安装,缺导入不应算类型错误。
|
||||
# exclude:测试目录与 work/ 不进类型检查。测试大量用动态 mock /
|
||||
# 模块遮蔽(如 test_queue_sandbox 的 queue 遮蔽),属于测试脚手架
|
||||
# artifact,不是生产类型错误;生产范围另见 workflow 的 mypy 命令。
|
||||
[mypy]
|
||||
ignore_missing_imports = True
|
||||
show_error_codes = True
|
||||
exclude = (?x)(^|/)(tests|work)/|test_[^/]*\.py$
|
||||
@@ -0,0 +1,10 @@
|
||||
# CI 工具链固定版本(host-verify / release 共用)。
|
||||
# 只含 CI 检查工具,不含生产运行依赖:生产代码除宿主侧延迟加载的
|
||||
# psycopg/psycopg2 驱动外只用标准库,仓库内无 requirements 清单,
|
||||
# 因此 pip-audit 目前只能审计本文件(见 workflow 注释与 PR 摘要)。
|
||||
# 升级版本时须同步本地试跑结果,禁止只改版本号不验证。
|
||||
ruff==0.17.3
|
||||
bandit==1.9.4
|
||||
mypy==2.4.1
|
||||
coverage==7.17.0
|
||||
pip-audit==2.10.1
|
||||
@@ -0,0 +1,11 @@
|
||||
# Ruff 配置:第一阶段只开 E + F(pyflakes 正确性 + pycodestyle 基础)。
|
||||
# 不开全规则:全规则在基线 aebc623 下有 2332 项(含 UP031 printf 格式
|
||||
# 1113 项等风格 churn),一次性全开只会把 CI 变成常红噪音,不先收敛。
|
||||
# 行宽 120:88 下 E501 有 131 项、100 下 29 项、120 下 13 项;
|
||||
# 120 是现有代码最接近的、可在第二阶段再收紧到 100 的起点。
|
||||
# 排除 work/:它是运行时 worktree/任务工作区,不是源码。
|
||||
line-length = 120
|
||||
exclude = ["work"]
|
||||
|
||||
[lint]
|
||||
select = ["E", "F"]
|
||||
Reference in new issue
Block a user