Fallback: extract HTTP code from failed-turn JSON; Docker codex_home points at real relay home

1. Worker _fallback_token correction: real host.py never returns
status='404'; a failed turn is status='failed' with the real
error (e.g. '404 model not found') in text/error. The old code
returned host_status verbatim, so 'failed' blocked every real
fallback and the old event-only branch did too. Now: if
host_status itself is a fallbackable code it is used; otherwise
the code is extracted from turn JSON text/error, stdout, stderr
or runner error, and 'failed' alone never counts as a signal.
Tests: failed-turn JSON with 404 in text falls back and
succeeds; with 401 it never falls back.
2. Docker config codex_home: /var/lib/codex-app-server/codex-home,
mounted at the same path into the container, is the only home
that holds the real relay config (xcmapi provider, base_url,
env_key). Pointing codex_home at a fresh empty dir made codex
POST to api.openai.com with no auth header and loop on connect
timeouts until host.py's 600s turn timeout. The /app/work
codex-home from PR #17 was necessary to get past initialize
but insufficient for a real turn; this replaces it.
This commit is contained in:
host committed 2026-10-26 06:07:07 +08:00
1 parent edc187bb70
commit 5827716a3a
3 files changed
+66 -18

No files matched your search

+27 -17
View File
@@ -917,26 +917,32 @@ class Worker:
return stdout.strip() or "host execution failed"
def _fallback_token(self, result, error):
"""一次失败尝试的可 fallback 信号(host_status 为空时的兜底)。
"""一次失败尝试的可 fallback 信号。
坏模型最常见的真实失败不在 turn 里,而在 thread/start:
host.py 的 create_thread 抛 RuntimeError、进程非 0 退出、
不打印 JSON,于是执行器结果的 host_status 为 None,旧逻辑
因此永远不 fallback(主机实测:坏模型任务直接 failed、
无 worker_fallback 审计)。此时从 stdout/stderr/runner 异常
文本里找显式 HTTP 状态码(404/429/5xx)或“model not found”
类文本,前者按码判断,后者按 404 计;找不到显式信号时
返回 None,绝不凭“host execution failed”这类通用文本猜测,
否则任何执行器故障都会被误当成模型故障换模型烧额度。
host_status 的真实含义是 host.py 回合状态(completed /
failed / timeout / interrupted),不是 HTTP 状态码——
真实世界里 host.py 从不返回 status=\"404\",只有单测里的
假 runner 才会这么写。所以这里不能把 host_status 原样
当 token:host_status 本身已是可 fallback 码(404/429/
5xx/timeout,含历史假 runner 形态)时直接返回;否则
(最常见:host_status=\"failed\")从回合 JSON 的 text/error
字段、stdout、stderr、runner 异常里找显式 HTTP 状态码
或“model not found”类文本(按 404 计),找到且可
fallback 才返回;找不到显式信号时返回 None,绝不凭
“host execution failed”这类通用文本猜测,否则任何执行
器故障都会被误当成模型故障换模型烧额度。thread/start
崩溃(host.py 非 0 退出、不打印 JSON)时 host_status
为 None,同样走这条提取路径。
"""
host_status = (result or {}).get("host_status")
if host_status is not None:
if host_status is not None and is_fallbackable(host_status):
return host_status
combined = " ".join([
combined_parts = [
(result or {}).get("stdout", "") or "",
(result or {}).get("stderr", "") or "",
error or "",
])
]
combined = " ".join(combined_parts)
if not combined:
return None
lowered = combined.lower()
@@ -1138,16 +1144,20 @@ class Worker:
if retry_count:
error = "[retries=%d] %s" % (retry_count, error)
# 未配 fallback_models 时的老行为:只记事件、不重试。
if self.router is not None and host_status is not None \
and is_fallbackable(host_status):
nxt = self.router.next_after(self.model, host_status)
# 注意用提取出的 token(真实回合失败时 host_status
# 只是 "failed",真正的 404 藏在回合 JSON 里),
# 不能直接用原始 host_status 判断。
_token = self._fallback_token(result, error)
if self.router is not None and _token is not None \
and is_fallbackable(_token):
nxt = self.router.next_after(self.model, _token)
if nxt is not None:
record = dict(task)
record["events"] = events
# record_fallback 本身会 append 到 events,
# 这里不要再 append 一次,否则事件会重复。
record_fallback(
record, self.model, nxt, str(host_status))
record, self.model, nxt, str(_token))
fallback_count = 1
self._audit(task["project_id"], "worker_fallback",
"%s -> %s" % (self.model, nxt))