admin/codex-app-server
1
0 Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix: 浏览器远控传输层改文件桥（Unix socket 在 runsc 下宿主不可见）

Browse Source 
根因（2026-10-30 活体实证）：#181 的 .browser-remote.sock 假设
宿主能连容器内 socket，但生产 Docker 任务走 runsc/gVisor，
其 gofer 挂载下容器内建的 socket 只在容器内可见，宿主目录
里只有普通文件（browser-live.png 一直能透传）。后果：server
代理对真实任务一律 503，会话根本没活过。

修法：传输层改四个普通文件（worktree 内，原子写 + 追加日志）：
- .browser-remote-state.json：MCP 每轮重写，mtime 即存活信号，
  server 判 30 秒无心跳为会话死亡（503）；
- .browser-remote-frame.jpg：最新 screencast 帧（base64 文本）；
- .browser-remote-input.jsonl：server 追加，MCP 按偏移量消费
  完整行，执行时再判 holder，半行等下一轮；
- .browser-remote-control.json：{holder, seq}，MCP 首轮即应用，
  接管可先于浏览器启动。
- holder 竞态：control 写下后 state 文件约 50ms 才刷新，server
  的 input/stream 改判 effective_holder（控制文件 mtime 新者胜），
  否则接管后立刻点会误 409。
- 线程契约与控制权模型不变；MCP 主循环仍是唯一触页线程。
- 播种刷新（#181）让老租户自动拿到新版脚本。

测试：socket 测试改文件桥测试（桥轮询/偏移量/半行/陈旧态/
effective_holder + server 代理全链），全量 1357 项通过（skipped 4）。
This commit is contained in:
host committed 2026-10-30 07:28:32 +08:00
1 parent 3a5c5db9fb
commit 6bcdeb8da1
7 files changed
+628 -510
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

cloud/playwright_remote.py
+244 -170
View File
@@ -1,47 +1,60 @@
"""Remote-control HTTP service for the Playwright MCP server.
"""File-bridge remote control for the Playwright MCP server.
Runs INSIDE the MCP process (same thread-ownership problem as the
browser itself) and listens on a Unix socket in the task worktree
(``.browser-remote.sock`` by default). Why a socket, not a TCP
port: Docker task containers are per-turn and --rm, so a published
port would need a mapping nobody can observe mid-run (the worker's
runner blocks); a socket file in the mounted worktree is visible
to the host control plane with zero mapping, zero conflicts, and
dies with the turn.
Why files, not a Unix socket (corrected 2026-10-30): the first
version of this module ran an HTTP service on ``.browser-remote.sock``
in the worktree, assuming the host control plane could connect to
it. That assumption is false under runsc/gVisor, which is what
production Docker tasks run under: a socket created inside a
runsc container on a bind mount is visible inside the container
only -- the host never sees the file (verified live: the socket
listed inside the container, the host directory showed only the
regular file). Regular files DO propagate through the same mount
(browser-live.png has always been host-visible). So the bridge
is four regular files in the worktree, all written atomically
(temp file + os.replace) except the append-only input log:
Threading contract (measured 2026-10-30: Playwright sync calls
from any thread but the creating one raise "Cannot switch to a
different thread"):
- The HTTP thread NEVER calls page methods. It only reads plain
attributes (Browser.cached_info), the latest frame bytes, and
the holder flag, and it enqueues user inputs.
- The MCP main thread drains the input queue and pumps screencast
frames on every select() iteration (~50ms), so user input lands
within ~50ms even while the agent is idle waiting for its next
tool call, and CDP frame acks keep flowing (no ack = CDP stops
after the first frame).
- ``.browser-remote-state.json``: {url, title, started, holder,
streaming, updated_at} -- rewritten by the MCP drain loop on
every iteration, so its mtime is the session-liveness signal.
The server treats a state file older than STATE_STALE_SECONDS
as a dead session (503).
- ``.browser-remote-frame.jpg``: latest screencast frame, base64
text, rewritten only when a new frame arrives.
- ``.browser-remote-input.jsonl``: the server appends one JSON
object per line. The MCP tracks a byte offset and executes
complete lines only (a trailing partial line waits for the
next poll); a line is executed only if the holder is "user"
at execution time, otherwise it is dropped.
- ``.browser-remote-control.json``: {holder, seq} -- written by
the server on takeover/release; the MCP applies it when seq
changes. Writing control ahead of browser start is fine: the
MCP picks it up on its first poll.
Control model:
- holder "agent" (default): agent tools work, HTTP input is 409.
- holder "user": the user has taken over (console Takeover
button). Agent browser_* tools fail fast with a plain error so
the agent stops fighting the user; the user releases with
POST /control {"holder": "agent"} and the agent resumes on the
next continue message.
- The socket is chmod 0666: the real gates are the worktree
directory permissions and the control plane's task-owner auth
on its proxy endpoints, not the socket bits.
Threading contract (unchanged from the socket version, measured
2026-10-30: Playwright sync calls from any thread but the
creating one raise "Cannot switch to a different thread"):
only the MCP main thread touches the browser. It calls
RemoteBridge.poll() on every select() iteration (~50ms), so
control/input/frames flow even while the agent is idle, and CDP
frame acks keep flowing (no ack = CDP stops after the first
frame).
Control model (unchanged): holder "agent" by default; holder
"user" while the console user has taken over, during which
agent browser_* tools fail fast with a plain error.
"""
import collections
import json
import os
import socket
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import time
SOCKET_NAME = ".browser-remote.sock"
STREAM_FPS = 5
STATE_NAME = ".browser-remote-state.json"
FRAME_NAME = ".browser-remote-frame.jpg"
INPUT_NAME = ".browser-remote-input.jsonl"
CONTROL_NAME = ".browser-remote-control.json"
STATE_STALE_SECONDS = 30
class SimpleQueue:
@@ -70,52 +83,15 @@ class SimpleQueue:
return not self._deque
class RemoteState:
"""Shared between the MCP main thread and the HTTP thread."""
def __init__(self, browser):
self.browser = browser
self.holder = "agent"
self._lock = threading.Lock()
self.inputs = SimpleQueue()
self._frame_lock = threading.Lock()
self._frame = ""
def set_holder(self, holder):
if holder not in ("agent", "user"):
raise ValueError("holder must be 'agent' or 'user'")
with self._lock:
self.holder = holder
return self.holder
def get_holder(self):
with self._lock:
return self.holder
def enqueue_input(self, item):
if self.get_holder() != "user":
raise PermissionError(
"user does not hold control (holder=%s); "
"take over first" % self.get_holder())
self.inputs.put(item)
def set_frame(self, data):
with self._frame_lock:
self._frame = data or ""
def get_frame(self):
with self._frame_lock:
return self._frame
def state_dict(self):
info = self.browser.cached_info()
info["holder"] = self.get_holder()
info["streaming"] = bool(self.get_frame())
return info
def _atomic_write(path, data):
tmp = path + ".tmp"
with open(tmp, "wb") as fh:
fh.write(data)
os.replace(tmp, path)
def execute_input(browser, item):
"""Main-thread only: run one queued user input."""
"""Main-thread only: run one user input."""
kind = item.get("type")
if kind == "mouse_click":
return browser.click_at(item.get("x"), item.get("y"))
@@ -128,107 +104,205 @@ def execute_input(browser, item):
raise ValueError("unknown input type %r" % (kind,))
def _make_handler(state):
class Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
class RemoteBridge:
"""MCP-side file bridge. Main thread only (see module doc)."""
def log_message(self, *args): # noqa: D102 - quiet
def __init__(self, browser, workdir=None):
self.browser = browser
self.workdir = workdir or getattr(browser, "cwd", None) or "."
self.holder = "agent"
self._input_offset = 0
self._control_seq = None
self._frame_written = False
def _path(self, name):
return os.path.join(self.workdir, name)
def get_holder(self):
return self.holder
def set_holder(self, holder):
if holder not in ("agent", "user"):
raise ValueError("holder must be 'agent' or 'user'")
self.holder = holder
return self.holder
def state_dict(self):
info = self.browser.cached_info()
info["holder"] = self.holder
info["streaming"] = self._frame_written
info["updated_at"] = time.time()
return info
def _poll_control(self):
try:
with open(self._path(CONTROL_NAME), "rb") as fh:
raw = fh.read()
ctrl = json.loads(raw.decode("utf-8"))
except (OSError, ValueError):
return
seq = ctrl.get("seq")
if seq is None or seq == self._control_seq:
return
self._control_seq = seq
try:
self.set_holder(ctrl.get("holder"))
except ValueError:
pass
def _send(self, code, obj):
body = json.dumps(obj, ensure_ascii=False).encode("utf-8")
self.send_response(code)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def _read_json(self):
try:
length = int(self.headers.get("Content-Length") or 0)
except ValueError:
length = 0
raw = self.rfile.read(length) if length > 0 else b""
try:
return json.loads(raw.decode("utf-8")) if raw else {}
except ValueError:
return None
def do_GET(self): # noqa: N802
if self.path == "/state":
self._send(200, state.state_dict())
elif self.path == "/stream":
self._stream()
else:
self._send(404, {"error": "unknown path"})
def _poll_inputs(self):
try:
with open(self._path(INPUT_NAME), "rb") as fh:
fh.seek(self._input_offset)
data = fh.read()
except OSError:
return
def _stream(self):
import time
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Cache-Control", "no-cache")
self.end_headers()
if not data:
return
# Only complete lines (ending with \n) are consumed; a
# trailing partial line stays for the next poll.
lines = data.split(b"\n")
complete = lines[:-1]
consumed = sum(len(line) + 1 for line in complete)
self._input_offset += consumed
for raw in complete:
if not raw.strip():
continue
try:
while True:
frame = state.get_frame()
payload = json.dumps(
{"image": frame,
"holder": state.get_holder(),
"ts": int(time.time())},
ensure_ascii=False)
self.wfile.write(
("data: %s\n\n" % payload).encode("utf-8"))
self.wfile.flush()
time.sleep(1.0 / STREAM_FPS)
except (BrokenPipeError, ConnectionResetError):
item = json.loads(raw.decode("utf-8"))
except ValueError:
continue
if self.holder != "user":
continue
try:
execute_input(self.browser, item)
except Exception as exc: # noqa: BLE001 - input errors
print("playwright_remote: input error: %s" % exc)
def poll(self, browser=None):
"""One drain-loop iteration: control, inputs, frame, state."""
target = browser if browser is not None else self.browser
self._poll_control()
self._poll_inputs()
try:
frame = target.pump_screencast()
except Exception: # noqa: BLE001 - screencast best-effort
frame = ""
if frame:
try:
_atomic_write(self._path(FRAME_NAME),
frame.encode("utf-8"))
self._frame_written = True
except OSError:
pass
def do_POST(self): # noqa: N802
body = self._read_json()
if body is None:
self._send(400, {"error": "invalid json"})
return
if self.path == "/control":
try:
holder = state.set_holder(body.get("holder"))
except ValueError as exc:
self._send(400, {"error": str(exc)})
return
self._send(200, {"holder": holder})
elif self.path == "/input":
try:
state.enqueue_input(body)
except PermissionError as exc:
self._send(409, {"error": str(exc)})
return
self._send(200, {"queued": True})
else:
self._send(404, {"error": "unknown path"})
return
return Handler
class UnixHTTPServer(ThreadingHTTPServer):
address_family = socket.AF_UNIX
daemon_threads = True
def server_bind(self):
try:
os.unlink(self.server_address)
except OSError:
pass
super().server_bind()
try:
os.chmod(self.server_address, 0o666)
_atomic_write(
self._path(STATE_NAME),
json.dumps(self.state_dict(),
ensure_ascii=False).encode("utf-8"))
except OSError:
pass
def start_remote_server(state, socket_path):
"""Start the Unix-socket HTTP service on a daemon thread."""
server = UnixHTTPServer(socket_path, _make_handler(state))
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
return server
def bridge_paths(workdir):
"""Server-side helper: the four bridge file paths."""
return {
"state": os.path.join(workdir, STATE_NAME),
"frame": os.path.join(workdir, FRAME_NAME),
"input": os.path.join(workdir, INPUT_NAME),
"control": os.path.join(workdir, CONTROL_NAME),
}
def read_state(workdir):
"""Server-side: read + freshness-check the state file.
Returns (state_dict, None), or (None, (status, message)):
missing file -> 503 session not active; stale (no MCP
heartbeat for STATE_STALE_SECONDS) -> 503; unparseable ->
502 (atomic writes mean this should not happen).
"""
path = os.path.join(workdir, STATE_NAME)
try:
with open(path, "rb") as fh:
raw = fh.read()
except OSError:
return None, (503, "browser session not active")
try:
state = json.loads(raw.decode("utf-8"))
except ValueError:
return None, (502, "bad browser response")
updated = state.get("updated_at") or 0
if time.time() - updated > STATE_STALE_SECONDS:
return None, (503, "browser session not active")
return state, None
def effective_holder(workdir, state):
"""Server-side holder with the control race closed.
The MCP rewrites the state file every ~50ms, so right after
a takeover the state file still shows the old holder. The
control file is the newer truth when its mtime is at or
after the state's updated_at; otherwise the state file
(already refreshed by a poll) wins. Both agree once the
MCP has polled, so this only closes the ~50ms window.
"""
try:
mtime = os.path.getmtime(os.path.join(workdir, CONTROL_NAME))
with open(os.path.join(workdir, CONTROL_NAME),
"rb") as fh:
ctrl = json.loads(fh.read().decode("utf-8"))
except (OSError, ValueError):
return state.get("holder")
holder = ctrl.get("holder")
if holder not in ("agent", "user"):
return state.get("holder")
if mtime >= (state.get("updated_at") or 0):
return holder
return state.get("holder")
def validate_input(body):
"""Server-side input shape check. Returns (item, None) or
(None, (status, message)). Bounds mirror Browser.click_at /
mouse_move so a bad coordinate is a 400, not a queued no-op."""
if not isinstance(body, dict):
return None, (400, "input must be a JSON object")
kind = body.get("type")
if kind in ("mouse_click", "mouse_move"):
x, y = body.get("x"), body.get("y")
if (isinstance(x, bool) or isinstance(y, bool)
or not isinstance(x, int) or not isinstance(y, int)):
return None, (400, "x and y must be integers")
if not (0 <= x <= 1280 and 0 <= y <= 800):
return None, (400, "coordinates out of viewport")
return {"type": kind, "x": x, "y": y}, None
if kind == "key":
key = body.get("key")
if not isinstance(key, str) or not key:
return None, (400, "key must be a non-empty string")
return {"type": "key", "key": key}, None
if kind == "type":
text = body.get("text")
if not isinstance(text, str):
return None, (400, "text must be a string")
return {"type": "type", "text": text}, None
return None, (400, "unknown input type %r" % (kind,))
def append_input(workdir, item):
path = os.path.join(workdir, INPUT_NAME)
with open(path, "ab") as fh:
fh.write((json.dumps(item, ensure_ascii=False) + "\n")
.encode("utf-8"))
def write_control(workdir, holder):
if holder not in ("agent", "user"):
raise ValueError("holder must be 'agent' or 'user'")
_atomic_write(
os.path.join(workdir, CONTROL_NAME),
json.dumps({"holder": holder, "seq": time.time_ns()},
ensure_ascii=False).encode("utf-8"))
return holder