docker: pin tenant networks to 172.16.0.0/16 via explicit --subnet
ci / test (push) Successful in 35m1s
ci / test (push) Successful in 35m1s
Tenant networks were created bare (docker network create <name>), letting Docker's default pool hand out subnets outside the iptables 8889 allowlist (172.16.0.0/12). The smoke tenant landed on 192.168.32.0/20, so containers never reached the egress proxy and every model turn timed out at 600s twice. Fix (B): derive a deterministic /24 inside 172.16.0.0/16 from sha256(network name) with linear probing on IPAM overlap, max 32 probes. 172.16.0.0/16 is the one /16 in the allowlist Docker never auto-assigns (default pool starts at 172.17.0.0/16). No persistent state; capacity 256 tenant networks per host, explicit failure when exhausted. Pre-existing out-of-range networks are reused as-is and need manual removal (see README). Tests: 7 new (candidate properties incl. wrap-around, pinned argv, overlap probe, exhaustion, empty name).
This commit is contained in:
1 parent
d5642c6252
commit
4d3ea048a5
3 files changed
+163
-16
No files matched your search
+64
-13
@@ -13,6 +13,7 @@
|
||||
- CloudExecutor:占位,未接通云端执行。
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
@@ -43,6 +44,39 @@ from cloud.peer_messages import extract_peer_messages as _extract_peer_messages
|
||||
|
||||
MAX_STDOUT_CHARS = 2000
|
||||
|
||||
# 租户 Docker 网络子网划段(2026-11-07 B 方案)。
|
||||
#
|
||||
# 背景:宿主 iptables 只放行 172.16.0.0/12 与 127.0.0.1 访问
|
||||
# Tinyproxy(8889),容器内 codex 必须经该代理到模型网关。
|
||||
# 以前建网不传 --subnet,子网由 Docker 默认池分配,曾分到
|
||||
# 192.168.32.0/20(放行范围外),容器零次补全、单轮 600 秒
|
||||
# 超时两轮后任务失败。划段区选 172.16.0.0/16:它在放行范围
|
||||
# 内,且 Docker 默认池从 172.17.0.0/16 起分,永远不会自动分
|
||||
# 到 172.16 段,与现存网络、宿主 LAN、Tailscale 均不重叠
|
||||
# (2026-11-07 主机实测:10 个网络,172.16.0.0/16 空闲)。
|
||||
# 每个租户网占一个 /24(254 个地址足够);同一网络名每次
|
||||
# 算出的候选顺序相同(sha256 首字节起算 + 线性探测),不依
|
||||
# 赖任何持久化状态;候选子网已被别的租户网占用时顺延下一
|
||||
# 个。容量上限 256 个租户网/宿主(32 次探测封顶),满了就
|
||||
# 明确返回失败,不静默降级——扩容需另行决策(删残留旧网或
|
||||
# 扩大放行范围),不要在这里偷偷换段。
|
||||
TENANT_SUBNET_PREFIX = "172.16"
|
||||
TENANT_SUBNET_SLOTS = 256
|
||||
TENANT_SUBNET_MAX_PROBES = 32
|
||||
|
||||
|
||||
def tenant_subnet_candidates(name):
|
||||
"""按网络名确定性排出候选租户子网(/24,172.16.0.0/16 内)。
|
||||
|
||||
首候选由 sha256(网络名) 首字节决定,其余按槽位顺延;
|
||||
不同网络名一般落在不同槽,撞槽时靠 ensure_network 的
|
||||
overlap 探测顺延。纯函数,不碰 Docker、不读状态。
|
||||
"""
|
||||
seed = hashlib.sha256(name.encode("utf-8")).digest()[0]
|
||||
for i in range(TENANT_SUBNET_MAX_PROBES):
|
||||
slot = (seed + i) % TENANT_SUBNET_SLOTS
|
||||
yield "%s.%d.0/24" % (TENANT_SUBNET_PREFIX, slot)
|
||||
|
||||
|
||||
def tmp_isolation_status():
|
||||
"""各层 /tmp 隔离现状的唯一口径(README 与单测都以它为准)。
|
||||
@@ -1957,6 +1991,15 @@ class DockerExecutor(Executor):
|
||||
def ensure_network(self, name):
|
||||
"""确保 Docker 网络存在:不存在就创建,已存在就忽略。
|
||||
|
||||
创建时显式 --subnet 固定在 172.16.0.0/12 内划段(见
|
||||
tenant_subnet_candidates):宿主 iptables 只放行该范围
|
||||
访问 egress 代理,不传子网时 Docker 默认池可能分到范
|
||||
围外(如 192.168.32.0/20),容器连不上代理、任务空等
|
||||
600 秒超时。候选子网已被占用时按 overlap 报错顺延;
|
||||
已存在(同名)直接复用——注意:这意味着 2026-11-07
|
||||
之前建好的范围外旧网(如 codex-smoke-20261107)不会
|
||||
被自动纠正,须人工删除后重建,见部署清单。
|
||||
|
||||
返回 True 表示网络可用(创建成功或本来就存在),
|
||||
False 表示创建失败(调用方继续派发,docker run 时会
|
||||
因网络不存在而明确失败,不在这里静默降级默认网络——
|
||||
@@ -1966,19 +2009,27 @@ class DockerExecutor(Executor):
|
||||
"""
|
||||
if not name:
|
||||
return False
|
||||
try:
|
||||
result = self._call_runner(["docker", "network", "create", name],
|
||||
timeout=30)
|
||||
except Exception as exc:
|
||||
return "already exists" in str(exc)
|
||||
try:
|
||||
if getattr(result, "returncode", 1) == 0:
|
||||
return True
|
||||
text = (getattr(result, "stderr", "") or "") + \
|
||||
(getattr(result, "stdout", "") or "")
|
||||
return "already exists" in text
|
||||
except Exception:
|
||||
return False
|
||||
for subnet in tenant_subnet_candidates(name):
|
||||
try:
|
||||
result = self._call_runner(
|
||||
["docker", "network", "create",
|
||||
"--subnet", subnet, name],
|
||||
timeout=30)
|
||||
except Exception as exc:
|
||||
return "already exists" in str(exc)
|
||||
try:
|
||||
if getattr(result, "returncode", 1) == 0:
|
||||
return True
|
||||
text = (getattr(result, "stderr", "") or "") + \
|
||||
(getattr(result, "stdout", "") or "")
|
||||
if "already exists" in text:
|
||||
return True
|
||||
if "overlap" in text.lower():
|
||||
continue
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
return False
|
||||
|
||||
def execute(self, task) -> dict:
|
||||
"""只读执行:容器内调 host.py status,不直接跑模型回合。
|
||||
|
||||
Reference in new issue
Block a user