对齐 Muse P4：浏览器完全对齐（CDP broker 单元外 + a11y-tree-only + 确定性分类器）
- chv-guest/chv-cdp-broker.py（新）：单元外 CDP broker，用户 chv-browser， 独家 chromium.launch(headless=True, args=LAUNCH_ARGS)；启动参数白名单 无 --remote-debugging-port、无 devtools；导航前 URL 校验 + 拦截名单； 快照恒带 untrusted_input 与 injection_flags；高风险表单在页时 open 即 ask 入队；观测到导航 POST 自动暂停；命中拦截规则的下载取消 + 暂停 + ask；全部回包与错误经 sanitize_error + response_leaks 检查。 - chv-guest/chv-browser.py（P2 版重构）：改为 broker 客户端，不再直连 Playwright（本文件无 playwright import），保留 URL 校验 + 快照截断 作为第二层防线；暂停状态与 broker 同步。 - chv-guest/chv_browser_guard.py（新）：纯 stdlib 共享规则——validate_url、 拦截名单加载/匹配、快照截断、错误消毒（去 HTML、截断 200 字符、凭据形 key=value 与 sk- 形 token 打码）、注入启发式（中英文各 6 条）、表单风险 扫描、下载扫描（可执行扩展名）、文件型 ask 队列（pending/allowed/denied， 与 P3 ApprovalQueue 同语义）、启动参数与回包泄露检查、脚本注入面 禁止清单（FORBIDDEN_PAGE_ATTRS/FORBIDDEN_OPS）。 - chv-guest/browser-blocklist.txt（新）：静态拦截名单；首版只有 RFC 2606 .test 自测条目，生产条目由运维构建时填入，无在线更新。 - chv-guest/build.sh/buildinfo.py：P2 三脚本 + P4 三文件纳入镜像与 buildinfo；新增 chv-cdp-broker.service 与 /run/chv/ask-queue tmpfiles。 - docs/chv-muse-p4-gap.md（新）：现状 vs Muse 官方要求差距表、改动、 诚实边界、P2 合入约定、真机待验证清单（round 12）。 - docs/chv-production-readiness.md：补 §15（P4 一节；编号合入时若与 P2 的 §15 撞，顺延）。 - cloud/tests/test_chv_muse_p4.py（新，36 项）：guard 单元、AST 钉住 禁止属性、假 Playwright broker 测试、假 broker agent 测试、真 Unix socket 联调、buildinfo/build 干跑断言；含否定测试（evaluate 被拒、 DOM 外泄被拒、DevTools 不可达、私有地址被拒、高风险表单被拦、 注入文本被标注）。 测试：P4 新测试 36/36；全量 cloud 套件 1903 项，两次连续全过 （skipped=8）。附记：第一次全量跑出 1 个失败但输出未保留、后两次 均未复现，疑为时序抖动，未追认。 真机验证未做：排队等 round-11/第三棒释放 100.65.51.33，清单见 gap 表 §5。本地提交，未推未合未部署。
This commit is contained in:
host committed
1 parent
06c3e172fe
commit
1c8fc3f7c8
11 files changed
+2878
-9
No files matched your search
@@ -0,0 +1,428 @@
|
||||
#!/usr/bin/env python3
|
||||
"""chv-cdp-broker：单元外 CDP broker（客机侧、单元外独立进程）。
|
||||
|
||||
对齐 Muse P4（见 docs/chv-muse-p4-gap.md）：Chromium 的所有权
|
||||
在这里，不在浏览器子代理手里。broker 独家持有 Playwright，
|
||||
CDP 连接只存在于本进程内；子代理（chv-browser.py）只经本
|
||||
socket 说话，绝不直连浏览器、绝不见 CDP 端点。
|
||||
|
||||
操作（Unix socket /run/chv/cdp-broker.sock，JSON 行）：
|
||||
open {url} -> 导航并返回 {title, url, a11y, truncated,
|
||||
untrusted_input, injection_flags, form_risk}
|
||||
snapshot {} -> 当前页同上
|
||||
screenshot {} -> 截图落盘 /run/chv/shots/<uuid>.png，返回 {path}
|
||||
close {} -> 关当前页（浏览器进程继续跑）
|
||||
pause {} -> 暂停（用户接管/填表/自动暂停的统一状态）
|
||||
resume {} -> 解除暂停
|
||||
status {} -> {paused, url, form_risk}
|
||||
list_asks {} -> 待审批 ask 列表
|
||||
resolve_ask {ask_id, decision} -> 写回 allow/denied
|
||||
ping {} -> {"ok": true}
|
||||
|
||||
暂停时除 ping/resume/status/list_asks 外一切操作拒绝——
|
||||
用户接管或凭证注入填表时，代理完全不能行动。
|
||||
|
||||
安全规则（纯函数在 chv_browser_guard.py，本文件只做接线）：
|
||||
导航前 validate_url + 静态拦截名单；快照恒带
|
||||
untrusted_input=true（外部页面文本永远是数据、不是指令），
|
||||
命中注入启发式带 injection_flags；高风险表单在页时 open
|
||||
导航即返回 ask（入队、执行停止）；观测到导航 POST 即自动
|
||||
暂停 + ask；下载命中可执行扩展名即取消 + 暂停 + ask。
|
||||
所有回包与日志过 sanitize_error；响应里绝无 ws 端点、
|
||||
DevTools 地址、原始 HTML。
|
||||
|
||||
Playwright 调用白名单：launch/goto/wait_for_load_state/title/
|
||||
url/accessibility.snapshot/screenshot/close + 只读监听
|
||||
on(request/download)。evaluate/add_script_tag/add_init_script/
|
||||
expose_function/set_content 等脚本注入面不在代码里出现，
|
||||
测试用 AST 扫本文件钉住。
|
||||
|
||||
只用标准库 + playwright（运行时，懒加载）。运行身份：系统用户
|
||||
chv-browser，socket 0600。
|
||||
"""
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import uuid
|
||||
|
||||
CDP_SOCK = "/run/chv/cdp-broker.sock"
|
||||
ASK_QUEUE_DIR = "/run/chv/ask-queue"
|
||||
BLOCKLIST_PATH = "/usr/local/share/chv/browser-blocklist.txt"
|
||||
SHOTS_DIR = "/run/chv/shots"
|
||||
PLAYWRIGHT_BROWSERS_PATH = "/usr/local/share/ms-playwright"
|
||||
MAX_REQUEST_BYTES = 65536
|
||||
NAV_TIMEOUT_MS = 30000
|
||||
|
||||
_GUARD = None
|
||||
|
||||
|
||||
def _load_guard():
|
||||
"""加载同目录的 chv_browser_guard（纯函数模块）。运行时
|
||||
安装在 /usr/local/bin/chv_browser_guard.py；测试时与本
|
||||
文件同在 chv-guest/。只在首次调用加载一次。"""
|
||||
global _GUARD
|
||||
if _GUARD is not None:
|
||||
return _GUARD
|
||||
here = os.path.dirname(os.path.abspath(__file__))
|
||||
path = os.path.join(here, "chv_browser_guard.py")
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"chv_browser_guard", path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
_GUARD = module
|
||||
return module
|
||||
|
||||
|
||||
def _load_playwright():
|
||||
"""懒加载 playwright；缺失时报明错（不静默假装有浏览器）。"""
|
||||
os.environ.setdefault("PLAYWRIGHT_BROWSERS_PATH",
|
||||
PLAYWRIGHT_BROWSERS_PATH)
|
||||
try:
|
||||
from playwright.sync_api import sync_playwright
|
||||
except ImportError as e:
|
||||
raise RuntimeError(
|
||||
"chv-cdp-broker: playwright 未装（%s）" % e)
|
||||
return sync_playwright
|
||||
|
||||
|
||||
class CdpBroker:
|
||||
"""CDP broker：独家持有 Chromium。playwright_factory 供单测
|
||||
注入假实现；正常运行时用 _load_playwright()。"""
|
||||
|
||||
def __init__(self, playwright_factory=None,
|
||||
blocklist_path=BLOCKLIST_PATH,
|
||||
ask_queue_dir=ASK_QUEUE_DIR, shots_dir=SHOTS_DIR):
|
||||
self._guard = _load_guard()
|
||||
self._factory = playwright_factory or _load_playwright
|
||||
self._blocklist_path = blocklist_path
|
||||
self._ask_queue_dir = ask_queue_dir
|
||||
self._shots_dir = shots_dir
|
||||
self._pw = None
|
||||
self._browser = None
|
||||
self._page = None
|
||||
self._paused = False
|
||||
self._form_risk = {"level": "none", "fields": []}
|
||||
self._current_url = ""
|
||||
|
||||
def _ensure_browser(self):
|
||||
if self._browser is not None:
|
||||
return
|
||||
ok, err = self._guard.check_launch_args(
|
||||
self._guard.LAUNCH_ARGS)
|
||||
if not ok:
|
||||
raise RuntimeError("启动参数不合规：%s" % err)
|
||||
self._pw = self._factory()
|
||||
entered = self._pw.__enter__()
|
||||
self._browser = entered.chromium.launch(
|
||||
headless=True, args=list(self._guard.LAUNCH_ARGS))
|
||||
|
||||
def _new_page(self):
|
||||
if self._page is not None:
|
||||
try:
|
||||
self._page.close()
|
||||
except Exception:
|
||||
pass
|
||||
self._page = self._browser.new_page()
|
||||
self._page.on("request", self._on_request)
|
||||
self._page.on("download", self._on_download)
|
||||
|
||||
def _on_request(self, request):
|
||||
"""只读监听：导航 POST（表单提交）即自动暂停 + 入队。
|
||||
非导航 POST（beacon/fetch）不在此处理——它不代表提交。"""
|
||||
try:
|
||||
method = request.method
|
||||
is_nav = request.is_navigation_request()
|
||||
except Exception:
|
||||
return
|
||||
if method != "POST" or not is_nav:
|
||||
return
|
||||
self._paused = True
|
||||
self._guard.enqueue_ask(
|
||||
self._ask_queue_dir, "form_submit_observed",
|
||||
self._current_url, "观测到导航 POST（疑似表单提交）")
|
||||
|
||||
def _on_download(self, download):
|
||||
"""只读监听：命中可执行扩展名即取消下载 + 暂停 + 入队。"""
|
||||
try:
|
||||
name = download.suggested_filename
|
||||
except Exception:
|
||||
return
|
||||
blocked, reason = self._guard.scan_download(name)
|
||||
if not blocked:
|
||||
return
|
||||
try:
|
||||
download.cancel()
|
||||
except Exception:
|
||||
pass
|
||||
self._paused = True
|
||||
self._guard.enqueue_ask(
|
||||
self._ask_queue_dir, "download_blocked",
|
||||
self._current_url, "下载命中拦截规则：%s" % reason)
|
||||
|
||||
def _paused_resp(self):
|
||||
return {"ok": False,
|
||||
"error": "浏览器已暂停（用户接管/填表/自动暂停）"}
|
||||
|
||||
def _ask_resp(self, ask_id, detail):
|
||||
return {"ok": False, "ask": True, "ask_id": ask_id,
|
||||
"error": "需审批：%s" % detail}
|
||||
|
||||
def open(self, url):
|
||||
if self._paused:
|
||||
return self._paused_resp()
|
||||
ok, err = self._guard.validate_url(url)
|
||||
if not ok:
|
||||
return {"ok": False, "error": err}
|
||||
host = self._guard.url_host(url)
|
||||
entries = self._guard.load_blocklist(self._blocklist_path)
|
||||
if self._guard.check_blocklist(host, entries):
|
||||
return {"ok": False, "error": "命中恶意站拦截名单：%s"
|
||||
% host}
|
||||
if self._form_risk.get("level") != "none":
|
||||
ask_id = self._guard.enqueue_ask(
|
||||
self._ask_queue_dir, "form_submit", url,
|
||||
"高风险表单在页（%s），导航须审批"
|
||||
% self._form_risk["level"])
|
||||
return self._ask_resp(ask_id, "高风险表单在页，导航须审批")
|
||||
try:
|
||||
self._ensure_browser()
|
||||
self._new_page()
|
||||
self._page.goto(url, timeout=NAV_TIMEOUT_MS)
|
||||
self._page.wait_for_load_state("domcontentloaded",
|
||||
timeout=NAV_TIMEOUT_MS)
|
||||
except Exception as e:
|
||||
return {"ok": False,
|
||||
"error": self._guard.sanitize_error(e)}
|
||||
return self.snapshot()
|
||||
|
||||
def snapshot(self):
|
||||
if self._paused:
|
||||
return self._paused_resp()
|
||||
if self._page is None:
|
||||
return {"ok": False, "error": "还没有打开的页面（先 open）"}
|
||||
try:
|
||||
title = self._page.title()
|
||||
url = self._page.url
|
||||
tree = self._page.accessibility.snapshot()
|
||||
except Exception as e:
|
||||
return {"ok": False,
|
||||
"error": self._guard.sanitize_error(e)}
|
||||
tree, truncated = self._guard.truncate_a11y(tree or {})
|
||||
flags = self._guard.detect_injection(tree)
|
||||
self._form_risk = self._guard.detect_form_risk(tree)
|
||||
self._current_url = url
|
||||
return {"ok": True, "title": title, "url": url,
|
||||
"a11y": tree, "truncated": truncated,
|
||||
"untrusted_input": True, "injection_flags": flags,
|
||||
"form_risk": self._form_risk}
|
||||
|
||||
def screenshot(self):
|
||||
if self._paused:
|
||||
return self._paused_resp()
|
||||
if self._page is None:
|
||||
return {"ok": False, "error": "还没有打开的页面（先 open）"}
|
||||
try:
|
||||
os.makedirs(self._shots_dir, exist_ok=True)
|
||||
path = os.path.join(self._shots_dir, "%s.png"
|
||||
% uuid.uuid4().hex)
|
||||
self._page.screenshot(path=path)
|
||||
except Exception as e:
|
||||
return {"ok": False,
|
||||
"error": self._guard.sanitize_error(e)}
|
||||
return {"ok": True, "path": path}
|
||||
|
||||
def close(self):
|
||||
if self._page is not None:
|
||||
try:
|
||||
self._page.close()
|
||||
except Exception:
|
||||
pass
|
||||
self._page = None
|
||||
self._form_risk = {"level": "none", "fields": []}
|
||||
self._current_url = ""
|
||||
return {"ok": True, "closed": True}
|
||||
|
||||
def pause(self):
|
||||
self._paused = True
|
||||
return {"ok": True, "paused": True}
|
||||
|
||||
def ping(self):
|
||||
return {"ok": True, "broker": True}
|
||||
|
||||
def resume(self):
|
||||
self._paused = False
|
||||
return {"ok": True, "paused": False}
|
||||
|
||||
def status(self):
|
||||
return {"ok": True, "paused": self._paused,
|
||||
"url": self._current_url,
|
||||
"form_risk": self._form_risk}
|
||||
|
||||
def list_asks(self):
|
||||
return {"ok": True,
|
||||
"asks": self._guard.list_asks(self._ask_queue_dir)}
|
||||
|
||||
def resolve_ask(self, ask_id, decision):
|
||||
ok, err = self._guard.resolve_ask(
|
||||
self._ask_queue_dir, ask_id, decision)
|
||||
if not ok:
|
||||
return {"ok": False, "error": err}
|
||||
if decision == "allowed":
|
||||
self._paused = False
|
||||
# 批准即放行这次操作：清掉快照缓存的表单风险（下一次
|
||||
# snapshot 会按新页面重算，不会丢检测）。
|
||||
self._form_risk = {"level": "none", "fields": []}
|
||||
else:
|
||||
self.close()
|
||||
self._paused = False
|
||||
return {"ok": True, "ask_id": ask_id,
|
||||
"decision": decision}
|
||||
|
||||
def shutdown(self):
|
||||
try:
|
||||
if self._browser is not None:
|
||||
self._browser.close()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if self._pw is not None:
|
||||
self._pw.__exit__(None, None, None)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def handle_request(broker, req):
|
||||
"""处理一个请求 dict，返回响应 dict（纯逻辑，可单测）。
|
||||
未知操作与禁用操作一律拒绝，不转发、不执行。"""
|
||||
guard = _load_guard()
|
||||
if not isinstance(req, dict):
|
||||
return {"ok": False, "error": "请求须是 JSON 对象"}
|
||||
op = req.get("op")
|
||||
if op in guard.FORBIDDEN_OPS:
|
||||
return {"ok": False,
|
||||
"error": "操作 %r 不暴露（无 JS/无原始 DOM/无交互）"
|
||||
% op}
|
||||
if op == "ping":
|
||||
return {"ok": True, "broker": True}
|
||||
if op == "open":
|
||||
return broker.open(req.get("url"))
|
||||
if op == "snapshot":
|
||||
return broker.snapshot()
|
||||
if op == "screenshot":
|
||||
return broker.screenshot()
|
||||
if op == "close":
|
||||
return broker.close()
|
||||
if op == "pause":
|
||||
return broker.pause()
|
||||
if op == "resume":
|
||||
return broker.resume()
|
||||
if op == "status":
|
||||
return broker.status()
|
||||
if op == "list_asks":
|
||||
return broker.list_asks()
|
||||
if op == "resolve_ask":
|
||||
return broker.resolve_ask(req.get("ask_id"),
|
||||
req.get("decision"))
|
||||
return {"ok": False,
|
||||
"error": "未知操作：%r（白名单：open/snapshot/screenshot/"
|
||||
"close/pause/resume/status/list_asks/resolve_ask/"
|
||||
"ping）" % (op,)}
|
||||
|
||||
|
||||
def _handle_conn(broker, conn):
|
||||
"""一个连接一请求一响应；回包先过泄露检查，不过即换通用错。"""
|
||||
guard = _load_guard()
|
||||
conn.settimeout(120)
|
||||
data = b""
|
||||
while len(data) < MAX_REQUEST_BYTES:
|
||||
chunk = conn.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
data += chunk
|
||||
if data.endswith(b"\n"):
|
||||
break
|
||||
try:
|
||||
req = json.loads(data.decode())
|
||||
except (ValueError, UnicodeDecodeError):
|
||||
resp = {"ok": False, "error": "请求不是合法 JSON"}
|
||||
else:
|
||||
try:
|
||||
resp = handle_request(broker, req)
|
||||
except Exception as e:
|
||||
resp = {"ok": False,
|
||||
"error": guard.sanitize_error(e)}
|
||||
if guard.response_leaks(resp):
|
||||
resp = {"ok": False, "error": "响应泄露检查未通过（已拦截）"}
|
||||
conn.sendall((json.dumps(resp, ensure_ascii=False) + "\n").encode())
|
||||
|
||||
|
||||
def serve(sock_path=CDP_SOCK, shots_dir=SHOTS_DIR,
|
||||
blocklist_path=BLOCKLIST_PATH, ask_queue_dir=ASK_QUEUE_DIR,
|
||||
playwright_factory=None):
|
||||
"""监听 Unix socket，逐连接处理。socket 0600。建不起来即抛错，
|
||||
不要静默带病跑。playwright_factory 只供测试注入假实现。"""
|
||||
if os.path.exists(sock_path):
|
||||
os.remove(sock_path)
|
||||
os.makedirs(os.path.dirname(sock_path), exist_ok=True)
|
||||
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
srv.bind(sock_path)
|
||||
srv.listen(16)
|
||||
os.chmod(sock_path, 0o600)
|
||||
print("chv-cdp-broker: 监听 %s" % sock_path,
|
||||
file=sys.stderr, flush=True)
|
||||
broker = CdpBroker(playwright_factory=playwright_factory,
|
||||
shots_dir=shots_dir,
|
||||
blocklist_path=blocklist_path,
|
||||
ask_queue_dir=ask_queue_dir)
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
conn, _ = srv.accept()
|
||||
except OSError:
|
||||
continue
|
||||
try:
|
||||
_handle_conn(broker, conn)
|
||||
except OSError as e:
|
||||
print("chv-cdp-broker: 连接处理异常：%s"
|
||||
% _load_guard().sanitize_error(e),
|
||||
file=sys.stderr, flush=True)
|
||||
finally:
|
||||
try:
|
||||
conn.close()
|
||||
except OSError:
|
||||
pass
|
||||
finally:
|
||||
broker.shutdown()
|
||||
|
||||
|
||||
def main(argv):
|
||||
sock_path = CDP_SOCK
|
||||
shots_dir = SHOTS_DIR
|
||||
blocklist_path = BLOCKLIST_PATH
|
||||
ask_queue_dir = ASK_QUEUE_DIR
|
||||
i = 0
|
||||
while i < len(argv):
|
||||
if argv[i] == "--socket" and i + 1 < len(argv):
|
||||
sock_path = argv[i + 1]
|
||||
i += 2
|
||||
elif argv[i] == "--shots-dir" and i + 1 < len(argv):
|
||||
shots_dir = argv[i + 1]
|
||||
i += 2
|
||||
elif argv[i] == "--blocklist" and i + 1 < len(argv):
|
||||
blocklist_path = argv[i + 1]
|
||||
i += 2
|
||||
elif argv[i] == "--ask-queue" and i + 1 < len(argv):
|
||||
ask_queue_dir = argv[i + 1]
|
||||
i += 2
|
||||
else:
|
||||
print("chv-cdp-broker: 未知参数：%s" % argv[i],
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
serve(sock_path, shots_dir, blocklist_path, ask_queue_dir)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv[1:]))
|
||||
Reference in new issue
Block a user