逐任务真实成本:上游用量事件 + 价目表计价

此前每任务只记一条 estimated(字符数/4,金额为空),注释称
host 协议不暴露真实 token 数。本轮实测推翻了该前提:
- account/usage/read 在中继上是死的(要求 ChatGPT 账号登录),
  不是来源,已在 cost.py docstring 写明死胡同
- app-server 每上游补全发 thread/tokenUsage/updated,last 是该次
  补全的用量;中继 5 调用回合实测 last 之和等于最后 total
  (input/output/cached/reasoning 四路都对得上)

实现:host.py run_turn 从事件窗口按 threadId+turnId 汇总用量,
重试/goal 各轮累加,dispatch/run JSON 与 goal 用量行透传;
sandbox 三路解析(dispatch JSON 解析改用完整 stdout,顺带修
了长 text 下 failed 被误判 completed 的潜 bug);worker 每任务
记一条 actual(重试/fallback 用量累加进同一条),无用量时回落
estimated;cost.py 价目表(默认 OpenAI 挂牌价,env 可覆盖,
无价模型 usd 记 None、报表进 unpriced_actual_tokens 而非 0 美元),
cost_records 加 cached_input_tokens 列(SQLite/PG 两支迁移);
报表与按天行新增 cached_input_tokens/unpriced_actual_tokens。
测试:host 32、cloud 483 全过。
This commit is contained in:
host committed 2026-10-26 18:41:27 +08:00
1 parent 353d6c1aa8
commit 63823fc2e3
11 files changed
+709 -67

No files matched your search

+24 -4
View File
@@ -374,7 +374,7 @@ psycopg,写明“需安装 psycopg”),绝不静默回落到 SQLite,
| `POST /tasks/{id}/heartbeat` | 该项目 `create_task` | 更新 `last_heartbeat = now`;状态为 `queued` 则改为 `running`;返回任务 | | `POST /tasks/{id}/heartbeat` | 该项目 `create_task` | 更新 `last_heartbeat = now`;状态为 `queued` 则改为 `running`;返回任务 |
| `POST /admin/sweep` | 认证即可 | 调 `sweep_stale_tasks(db_path, threshold_seconds=300)`(body 可传 `threshold_seconds` 覆盖默认值);把 `running` 且 `last_heartbeat`(无心跳则用 `updated_at`)早于 `now - threshold` 的任务改为 `interrupted` 并记审计 `sweep_interrupted`;返回 `{"swept": [ids]}` | | `POST /admin/sweep` | 认证即可 | 调 `sweep_stale_tasks(db_path, threshold_seconds=300)`(body 可传 `threshold_seconds` 覆盖默认值);把 `running` 且 `last_heartbeat`(无心跳则用 `updated_at`)早于 `now - threshold` 的任务改为 `interrupted` 并记审计 `sweep_interrupted`;返回 `{"swept": [ids]}` |
| `POST /admin/backup` | maintainer 以上 | Postgres 时调 `backup_postgres`、SQLite 时调 `backup_db`(body 可传 `out_dir`,否则用 `CLOUD_BACKUP_DIR` 或默认 `cloud-backups`);返回 `{backend, path, size_bytes, task_count}`,绝不返回数据库密码或备份文件内容 | | `POST /admin/backup` | maintainer 以上 | Postgres 时调 `backup_postgres`、SQLite 时调 `backup_db`(body 可传 `out_dir`,否则用 `CLOUD_BACKUP_DIR` 或默认 `cloud-backups`);返回 `{backend, path, size_bytes, task_count}`,绝不返回数据库密码或备份文件内容 |
| `POST /tasks/{id}/cost` | 该任务所属项目 `read` | 成本聚合:`{task_id, project_id, total_tokens, usd_micros, turns, fallback_count, actual_tokens, estimated_tokens}`;真数与估算分开;不存在 `404`,越权 `403` | | `GET /tasks/{id}/cost` | 该任务所属项目 `read` | 成本聚合:`{task_id, project_id, total_tokens, usd_micros, turns, fallback_count, actual_tokens, estimated_tokens, cached_input_tokens, unpriced_actual_tokens}`;真数与估算分开,无价 actual 行进 `unpriced_actual_tokens`;不存在 `404`,越权 `403` |
| `POST /projects/{id}/memory` | 该项目 `create_task` | 写线程摘要 body `{thread_id?, goal, conclusion, files_changed?, open_issues?}`,返回 `201 {id}`,并记审计 `create_memory` | | `POST /projects/{id}/memory` | 该项目 `create_task` | 写线程摘要 body `{thread_id?, goal, conclusion, files_changed?, open_issues?}`,返回 `201 {id}`,并记审计 `create_memory` |
| `GET /projects/{id}/memory?top_k=5` | 该项目 `read` | 最近 active 摘要(默认 top_k=5)+ `prompt_context`(只含 top_k,不全量注入);不存在 `404`,越权 `403` | | `GET /projects/{id}/memory?top_k=5` | 该项目 `read` | 最近 active 摘要(默认 top_k=5)+ `prompt_context`(只含 top_k,不全量注入);不存在 `404`,越权 `403` |
| `POST /projects/{id}/memory/{mid}/supersede` | 该项目 `create_task` | 把摘要标废弃(deprecated,不是物理删除),并记审计 `supersede_memory`;摘要不属于该项目则 `404` | | `POST /projects/{id}/memory/{mid}/supersede` | 该项目 `create_task` | 把摘要标废弃(deprecated,不是物理删除),并记审计 `supersede_memory`;摘要不属于该项目则 `404` |
@@ -697,9 +697,29 @@ interrupted 且终态任务 cancel 返回 400、心跳后 sweep 不扫刚心跳
照样跑)。第二次的结果就是最终结果:成功则任务 completed, 照样跑)。第二次的结果就是最终结果:成功则任务 completed,
`actual_model` 写成 fallback 后的模型,成本记录 `fallback_count=1`; `actual_model` 写成 fallback 后的模型,成本记录 `fallback_count=1`;
第二次也失败则任务 failed,`error_text` 同时写清主模型失败原因与 第二次也失败则任务 failed,`error_text` 同时写清主模型失败原因与
fallback 模型失败原因。边界:真实 token 成本仍是估算(host 协议 fallback 模型失败原因。成本来源与 fallback 无关(见下文“真实成本”)。
不暴露真实 token 数),fallback 不改变成本来源,只改变实际跑的 - 真实成本(逐任务 actual):host.py 在每回合结束时从 app-server
模型与 `actual_model`/`fallback_count` 的记录。 的 `thread/tokenUsage/updated` 事件里汇总上游真实用量(每个上游
补全发一次,`last` 是该次补全的用量,同一 turn 内累加;中继实测
五次 `last` 之和等于最后一次的 `total`,input/output/cached/
reasoning 四路都对得上)。host.py dispatch/run 打印的 JSON 带
`usage`,goal 循环在结论行后另打一行 `usage input=… cached=…
output=… total=… completions=…`;执行器解析后 worker 每任务记
一条 `source="actual"`(重试/fallback 各次尝试的用量都累加进
同一条,不按尝试次数重复记行)。没带回用量时(老版本 server、
回合未产生补全)回落字符估算记 `estimated`,两者在报表里分开。
金额 = 实测 token × 价目表:默认是 OpenAI 挂牌价(gpt-6.1-sol
$2.00/$0.10/$10.00、gpt-6-sol $2.00/$0.20/$10.00,单位美元/百万
token,依次为输入/缓存输入/输出;缓存是输入的子集,不另加;
推理 token 已含在输出里,不重复算),中继的实际账单可能不同,
可用 `CLOUD_MODEL_PRICES_JSON` 或 `CLOUD_MODEL_PRICES_FILE`
按模型覆盖;无价模型记 `usd_micros=None`,报表里进
`unpriced_actual_tokens` 而不是 0 美元。`GET /tasks/{id}/cost` 与
`GET /projects/{id}/cost` 的聚合新增 `cached_input_tokens` 与
`unpriced_actual_tokens`;按天行同样新增这两列。注意死胡同:
app-server 的 `account/usage/read` 在中继上不可用(要求 ChatGPT
账号登录,中继 API key 调它报 `codex account authentication
required`),账户级用量不是本功能的来源,不要重走这条路。
- 容器清理兜底:`DockerExecutor.list_containers(prefix="codex-")`( - 容器清理兜底:`DockerExecutor.list_containers(prefix="codex-")`(
`docker ps -a --format {{.Names}}`,失败返回 `[]`)与 `docker ps -a --format {{.Names}}`,失败返回 `[]`)与
`cleanup_stale_containers()`(列出并逐个 kill+rm,单个失败只跳过, `cleanup_stale_containers()`(列出并逐个 kill+rm,单个失败只跳过,
+172 -37
View File
@@ -3,11 +3,30 @@
Zero third-party dependencies; standard library only. Zero third-party dependencies; standard library only.
Actual token counts come from the model/provider response when Actual token counts come from the model/provider response when
available; when they are not, token counts are estimates (see available: host.py collects the app-server's thread/tokenUsage/updated
``estimate_tokens``) and are stored separately as ``source="estimated"`` events during each turn (one per upstream completion, summed over the
so estimates are never mixed into actual totals. turn window) and the worker records them with ``source="actual"``.
When no usage events arrive (old server, turn produced no completion),
the worker falls back to ``estimate_tokens`` with
``source="estimated"``; the two are stored separately so estimates are
never mixed into actual totals.
USD is tokens times a price table, not the relay's bill: the defaults
are OpenAI list prices (see ``_DEFAULT_MODEL_PRICES``), and the relay
may charge differently. Override per model with
``CLOUD_MODEL_PRICES_JSON`` or ``CLOUD_MODEL_PRICES_FILE``; models
with no price record ``usd_micros=None`` and reports count their
tokens in ``unpriced_actual_tokens`` instead of pricing them at zero.
Dead end, documented so nobody retries it: the app-server's
account/usage/read method requires ChatGPT account authentication and
fails on a relay API key (2026-10-26 probe: ``codex account
authentication required to read token usage``). Account-level usage
is therefore not a source here; per-task actuals come from turn
events only.
""" """
import json
import os import os
import sqlite3 import sqlite3
from cloud.db import connect as _db_connect from cloud.db import connect as _db_connect
@@ -15,6 +34,101 @@ import time
from datetime import datetime, timezone from datetime import datetime, timezone
# OpenAI list prices, USD per 1M tokens, as (input, cached_input,
# output). gpt-6.1-sol $2.00/$0.10/$10.00 and gpt-6-sol
# $2.00/$0.20/$10.00, as reported at the 2026-09-30 launch.
# Again: list prices, not the relay's bill; override below.
_DEFAULT_MODEL_PRICES = {
"gpt-6.1-sol": (2.00, 0.10, 10.00),
"gpt-6-sol": (2.00, 0.20, 10.00),
}
def _coerce_rates(rates):
"""Accept [input, cached, output] or {input, cached_input, output}."""
try:
if isinstance(rates, dict):
return (float(rates["input"]), float(rates["cached_input"]),
float(rates["output"]))
return (float(rates[0]), float(rates[1]), float(rates[2]))
except (TypeError, ValueError, KeyError, IndexError):
return None
def model_prices():
"""Effective price table: defaults merged with env overrides.
``CLOUD_MODEL_PRICES_JSON`` (a JSON object) and
``CLOUD_MODEL_PRICES_FILE`` (path to a JSON file) both merge over
the defaults per model; unparsable content is ignored, never fatal.
"""
prices = dict(_DEFAULT_MODEL_PRICES)
raw = os.environ.get("CLOUD_MODEL_PRICES_JSON") or ""
if raw:
try:
override = json.loads(raw)
except ValueError:
override = None
if isinstance(override, dict):
for model, rates in override.items():
coerced = _coerce_rates(rates)
if coerced is not None:
prices[model] = coerced
path = os.environ.get("CLOUD_MODEL_PRICES_FILE") or ""
if path:
try:
with open(path) as f:
override = json.load(f)
except (OSError, ValueError):
override = None
if isinstance(override, dict):
for model, rates in override.items():
coerced = _coerce_rates(rates)
if coerced is not None:
prices[model] = coerced
return prices
def price_usd_micros(model, input_tokens, cached_input_tokens,
output_tokens):
"""USD micros for provider-measured tokens at the price table.
cached is a subset of input (prompt-cache reads): non-cached input
pays the input rate, cached pays the cached rate, output pays the
output rate. Reasoning tokens are already inside output_tokens
(provider total = input + output), never added twice. Returns None
when the model has no price entry (caller records usd as unknown).
"""
rates = model_prices().get(model)
if rates is None:
return None
input_rate, cached_rate, output_rate = rates
total_in = max(0, int(input_tokens or 0))
cached = max(0, min(int(cached_input_tokens or 0), total_in))
non_cached = total_in - cached
usd = (non_cached * input_rate + cached * cached_rate
+ max(0, int(output_tokens or 0)) * output_rate) / 1000000
return int(round(usd * 1000000))
_USAGE_SUM_KEYS = ("input_tokens", "cached_input_tokens",
"cache_write_input_tokens", "output_tokens",
"reasoning_output_tokens", "total_tokens", "completions")
def sum_usage(a, b):
"""Add two usage dicts (host.py shape); either side may be None."""
if not a:
return dict(b) if b else None
if not b:
return dict(a)
out = dict(a)
for k in _USAGE_SUM_KEYS:
if isinstance(b.get(k), int):
out[k] = out.get(k, 0) + b[k]
return out
def estimate_tokens(prompt_text, output_text) -> int: def estimate_tokens(prompt_text, output_text) -> int:
"""Estimate total tokens as (chars of prompt + output) // 4.""" """Estimate total tokens as (chars of prompt + output) // 4."""
prompt_text = prompt_text or "" prompt_text = prompt_text or ""
@@ -52,20 +166,14 @@ class CostStore:
_SOURCES = ("actual", "estimated") _SOURCES = ("actual", "estimated")
def __init__(self, db_path): _COST_DDL = """CREATE TABLE IF NOT EXISTS cost_records (
self.db_path = db_path
if db_path != ":memory:":
parent = os.path.dirname(os.path.abspath(db_path))
os.makedirs(parent, exist_ok=True)
self._conn = _db_connect(db_path)
self._conn.execute(
"""CREATE TABLE IF NOT EXISTS cost_records (
id INTEGER PRIMARY KEY AUTOINCREMENT, id INTEGER PRIMARY KEY AUTOINCREMENT,
task_id TEXT, task_id TEXT,
project_id TEXT, project_id TEXT,
model TEXT, model TEXT,
input_tokens INTEGER, input_tokens INTEGER,
output_tokens INTEGER, output_tokens INTEGER,
cached_input_tokens INTEGER DEFAULT 0,
total_tokens INTEGER, total_tokens INTEGER,
usd_micros INTEGER, usd_micros INTEGER,
source TEXT, source TEXT,
@@ -73,8 +181,36 @@ class CostStore:
day TEXT, day TEXT,
created_at REAL created_at REAL
)""" )"""
)
def _ensure_cached_column(self, conn):
"""老库补 cached_input_tokens 列(SQLite 与 Postgres 两支:
PRAGMA table_info 经 db.py 适配层翻译,缺列时 ALTER 补列;
并发下两个进程同时补列时第二个 ALTER 报错,直接吞掉)。
"""
try:
cols = {row[1] for row in conn.execute(
"PRAGMA table_info(cost_records)").fetchall()}
except Exception:
return
if "cached_input_tokens" not in cols:
try:
conn.execute(
"ALTER TABLE cost_records"
" ADD COLUMN cached_input_tokens INTEGER DEFAULT 0")
conn.commit()
except Exception:
pass
def __init__(self, db_path):
self.db_path = db_path
if db_path != ":memory:":
parent = os.path.dirname(os.path.abspath(db_path))
os.makedirs(parent, exist_ok=True)
self._conn = _db_connect(db_path)
self._conn.execute(self._COST_DDL)
self._conn.commit() self._conn.commit()
self._ensure_cached_column(self._conn)
def _new_conn(self): def _new_conn(self):
"""每次写/读开一个新连接,用完即关。 """每次写/读开一个新连接,用完即关。
@@ -92,32 +228,21 @@ class CostStore:
if self.db_path == ":memory:": if self.db_path == ":memory:":
return self._conn return self._conn
conn = _db_connect(self.db_path) conn = _db_connect(self.db_path)
conn.execute( conn.execute(self._COST_DDL)
"""CREATE TABLE IF NOT EXISTS cost_records (
id INTEGER PRIMARY KEY AUTOINCREMENT,
task_id TEXT,
project_id TEXT,
model TEXT,
input_tokens INTEGER,
output_tokens INTEGER,
total_tokens INTEGER,
usd_micros INTEGER,
source TEXT,
fallback_count INTEGER DEFAULT 0,
day TEXT,
created_at REAL
)"""
)
conn.commit() conn.commit()
self._ensure_cached_column(conn)
return conn return conn
def record(self, task_id, project_id, model, input_tokens=0, def record(self, task_id, project_id, model, input_tokens=0,
output_tokens=0, usd_micros=None, source="actual", output_tokens=0, cached_input_tokens=0, usd_micros=None,
fallback_count=0): source="actual", fallback_count=0):
"""Insert one cost record; returns its row id. """Insert one cost record; returns its row id.
``total_tokens`` is computed as input + output, and ``day`` is ``total_tokens`` is computed as input + output, and ``day`` is
the UTC date (YYYY-MM-DD) of ``created_at``. the UTC date (YYYY-MM-DD) of ``created_at``. cached is a
subset of input (prompt-cache reads), stored separately so
reports can show why a task was cheap; it is never added to
total on top of input.
""" """
if source not in self._SOURCES: if source not in self._SOURCES:
raise ValueError( raise ValueError(
@@ -125,6 +250,7 @@ class CostStore:
) )
input_tokens = int(input_tokens or 0) input_tokens = int(input_tokens or 0)
output_tokens = int(output_tokens or 0) output_tokens = int(output_tokens or 0)
cached_input_tokens = int(cached_input_tokens or 0)
total_tokens = input_tokens + output_tokens total_tokens = input_tokens + output_tokens
created_at = time.time() created_at = time.time()
day = datetime.fromtimestamp(created_at, tz=timezone.utc).strftime("%Y-%m-%d") day = datetime.fromtimestamp(created_at, tz=timezone.utc).strftime("%Y-%m-%d")
@@ -133,12 +259,12 @@ class CostStore:
cursor = conn.execute( cursor = conn.execute(
"""INSERT INTO cost_records """INSERT INTO cost_records
(task_id, project_id, model, input_tokens, output_tokens, (task_id, project_id, model, input_tokens, output_tokens,
total_tokens, usd_micros, source, fallback_count, day, cached_input_tokens, total_tokens, usd_micros, source,
created_at) fallback_count, day, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""", VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(task_id, project_id, model, input_tokens, output_tokens, (task_id, project_id, model, input_tokens, output_tokens,
total_tokens, usd_micros, source, int(fallback_count or 0), cached_input_tokens, total_tokens, usd_micros, source,
day, created_at), int(fallback_count or 0), day, created_at),
) )
conn.commit() conn.commit()
return cursor.lastrowid return cursor.lastrowid
@@ -175,6 +301,10 @@ class CostStore:
COALESCE(SUM(CASE WHEN source = 'actual' COALESCE(SUM(CASE WHEN source = 'actual'
THEN total_tokens ELSE 0 END), 0), THEN total_tokens ELSE 0 END), 0),
COALESCE(SUM(CASE WHEN source = 'estimated' COALESCE(SUM(CASE WHEN source = 'estimated'
THEN total_tokens ELSE 0 END), 0),
COALESCE(SUM(cached_input_tokens), 0),
COALESCE(SUM(CASE WHEN source = 'actual'
AND usd_micros IS NULL
THEN total_tokens ELSE 0 END), 0) THEN total_tokens ELSE 0 END), 0)
FROM cost_records WHERE %s""" % where, FROM cost_records WHERE %s""" % where,
params, params,
@@ -182,7 +312,8 @@ class CostStore:
finally: finally:
if conn is not self._conn: if conn is not self._conn:
conn.close() conn.close()
total_tokens, usd_micros, turns, fallback_count, actual, estimated = row (total_tokens, usd_micros, turns, fallback_count, actual,
estimated, cached, unpriced) = row
return { return {
"total_tokens": int(total_tokens), "total_tokens": int(total_tokens),
"usd_micros": int(usd_micros), "usd_micros": int(usd_micros),
@@ -190,6 +321,10 @@ class CostStore:
"fallback_count": int(fallback_count), "fallback_count": int(fallback_count),
"actual_tokens": int(actual), "actual_tokens": int(actual),
"estimated_tokens": int(estimated), "estimated_tokens": int(estimated),
"cached_input_tokens": int(cached),
# usd 求和只覆盖有价的行;无价 actual 行的 token 在此,
# 不会被静默算成 0 美元。
"unpriced_actual_tokens": int(unpriced),
} }
def report_by_task(self, task_id) -> dict: def report_by_task(self, task_id) -> dict:
+1
View File
@@ -859,6 +859,7 @@ POSTGRES_SCHEMA = (
"model TEXT, " "model TEXT, "
"input_tokens BIGINT, " "input_tokens BIGINT, "
"output_tokens BIGINT, " "output_tokens BIGINT, "
"cached_input_tokens BIGINT DEFAULT 0, "
"total_tokens BIGINT, " "total_tokens BIGINT, "
"usd_micros BIGINT, " "usd_micros BIGINT, "
"source TEXT, " "source TEXT, "
+59 -4
View File
@@ -44,6 +44,41 @@ class Executor:
raise NotImplementedError("该执行器不支持 goal 模式") raise NotImplementedError("该执行器不支持 goal 模式")
def _clean_usage(raw):
"""校验 host.py 带来的用量 dict:非 dict、非整数、或 input /
output / total 全为 0 时返回 None(调用方回落估算,绝不把
0 或脏数据当成真实用量记 actual)。"""
if not isinstance(raw, dict):
return None
keys = ("input_tokens", "cached_input_tokens",
"cache_write_input_tokens", "output_tokens",
"reasoning_output_tokens", "total_tokens", "completions")
try:
out = {k: int(raw.get(k) or 0) for k in keys}
except (TypeError, ValueError):
return None
if out["input_tokens"] <= 0 and out["output_tokens"] <= 0 \
and out["total_tokens"] <= 0:
return None
return out
_USAGE_LINE_RE = re.compile(
r"^usage input=(\d+) cached=(\d+) cache_write=(\d+) output=(\d+)"
r" reasoning=(\d+) total=(\d+) completions=(\d+)\s*$", re.M)
def _parse_goal_usage(stdout):
"""从 host.py goal 的用量行解析用量 dict,无用量行返回 None。"""
m = _USAGE_LINE_RE.search(stdout or "")
if not m:
return None
keys = ("input_tokens", "cached_input_tokens",
"cache_write_input_tokens", "output_tokens",
"reasoning_output_tokens", "total_tokens", "completions")
return {k: int(v) for k, v in zip(keys, m.groups())}
def _parse_goal_output(stdout): def _parse_goal_output(stdout):
"""解析 host.py goal 的 stdout,返回 (met, goal_error)。 """解析 host.py goal 的 stdout,返回 (met, goal_error)。
@@ -155,18 +190,28 @@ class LocalExecutor(Executor):
判断是否可 fallback),取不到 JSON 时按 completed 计。 判断是否可 fallback),取不到 JSON 时按 completed 计。
""" """
returncode = getattr(result, "returncode", 1) returncode = getattr(result, "returncode", 1)
stdout = _truncate_stdout(getattr(result, "stdout", "")) stdout_full = getattr(result, "stdout", "") or ""
stdout = _truncate_stdout(stdout_full)
status = "failed" status = "failed"
usage = None
if returncode == 0: if returncode == 0:
status = "completed" status = "completed"
# JSON 解析必须用完整 stdout:截头后的前 2000 字在
# text 很长时根本不是合法 JSON,旧代码会把 host 的
# failed 误判成 completed;usage 在 JSON 末尾,截头
# 后同样丢。存库的 stdout 仍用截头版,不变。
try: try:
parsed = json.loads(stdout) parsed = json.loads(stdout_full)
except (TypeError, ValueError): except (TypeError, ValueError):
parsed = None parsed = None
if isinstance(parsed, dict) and parsed.get("status"): if isinstance(parsed, dict) and parsed.get("status"):
status = ("completed" if parsed["status"] == "completed" status = ("completed" if parsed["status"] == "completed"
else "failed") else "failed")
host_status = parsed["status"] host_status = parsed["status"]
# host.py dispatch/run 打印的 JSON 里带 usage 时
# 原样透传给 worker 记 actual;脏数据由
# _clean_usage 挡掉,不进 actual。
usage = _clean_usage(parsed.get("usage"))
else: else:
host_status = None host_status = None
else: else:
@@ -182,6 +227,7 @@ class LocalExecutor(Executor):
# stderr,worker 判断 dispatch 续跑时需要它;execute() # stderr,worker 判断 dispatch 续跑时需要它;execute()
# 的旧结果 dict 不含此字段,保持不变。 # 的旧结果 dict 不含此字段,保持不变。
"stderr": _truncate_stdout(getattr(result, "stderr", "")), "stderr": _truncate_stdout(getattr(result, "stderr", "")),
"usage": usage,
} }
def _host_name(self, task): def _host_name(self, task):
@@ -309,6 +355,7 @@ class LocalExecutor(Executor):
"goal-unmet" if met is False else None), "goal-unmet" if met is False else None),
"goal_met": met, "goal_met": met,
"goal_error": goal_error, "goal_error": goal_error,
"usage": _parse_goal_usage(stdout_full),
} }
def popen_task(self, task, kind="dispatch"): def popen_task(self, task, kind="dispatch"):
@@ -852,6 +899,7 @@ class DockerExecutor(Executor):
"container": self.container_name(task), "container": self.container_name(task),
"goal_met": met, "goal_met": met,
"goal_error": goal_error, "goal_error": goal_error,
"usage": _parse_goal_usage(stdout_full),
} }
def network_gateway(self, network_name): def network_gateway(self, network_name):
@@ -935,19 +983,25 @@ def _parse_host_result(task_id, fake):
completed 外的状态均按 failed 计。 completed 外的状态均按 failed 计。
""" """
returncode = getattr(fake, "returncode", 1) returncode = getattr(fake, "returncode", 1)
stdout = _truncate_stdout(getattr(fake, "stdout", "")) stdout_full = getattr(fake, "stdout", "") or ""
stdout = _truncate_stdout(stdout_full)
status = "failed" status = "failed"
host_status = None host_status = None
usage = None
if returncode == 0: if returncode == 0:
status = "completed" status = "completed"
# 与 _result_from_runner 同理:JSON 解析用完整 stdout,
# 否则长 text 会把 host 的 failed 误判成 completed,
# 也会丢掉 JSON 末尾的 usage。
try: try:
parsed = json.loads(stdout) parsed = json.loads(stdout_full)
except (TypeError, ValueError): except (TypeError, ValueError):
parsed = None parsed = None
if isinstance(parsed, dict) and parsed.get("status"): if isinstance(parsed, dict) and parsed.get("status"):
host_status = parsed["status"] host_status = parsed["status"]
status = ("completed" if host_status == "completed" status = ("completed" if host_status == "completed"
else "failed") else "failed")
usage = _clean_usage(parsed.get("usage"))
return { return {
"executor": "docker", "executor": "docker",
"task_id": task_id, "task_id": task_id,
@@ -956,6 +1010,7 @@ def _parse_host_result(task_id, fake):
"returncode": returncode, "returncode": returncode,
"host_status": host_status, "host_status": host_status,
"stderr": _truncate_stdout(getattr(fake, "stderr", "")), "stderr": _truncate_stdout(getattr(fake, "stderr", "")),
"usage": usage,
} }
+17 -4
View File
@@ -1092,14 +1092,20 @@ class CloudControlPlaneHandler(BaseHTTPRequestHandler):
row = conn.execute( row = conn.execute(
"SELECT day, COALESCE(SUM(total_tokens),0) AS t," "SELECT day, COALESCE(SUM(total_tokens),0) AS t,"
" COALESCE(SUM(usd_micros),0) AS u, COUNT(*) AS c," " COALESCE(SUM(usd_micros),0) AS u, COUNT(*) AS c,"
" COALESCE(SUM(fallback_count),0) AS f" " COALESCE(SUM(fallback_count),0) AS f,"
" COALESCE(SUM(cached_input_tokens),0) AS cached,"
" COALESCE(SUM(CASE WHEN source = 'actual'"
" AND usd_micros IS NULL THEN total_tokens"
" ELSE 0 END),0) AS unpriced"
" FROM cost_records WHERE project_id = ? AND day = ?" " FROM cost_records WHERE project_id = ? AND day = ?"
" GROUP BY day", " GROUP BY day",
(project_id, day)).fetchone() (project_id, day)).fetchone()
if row: if row:
daily = [{"day": row["day"], "total_tokens": row["t"], daily = [{"day": row["day"], "total_tokens": row["t"],
"usd_micros": row["u"], "turns": row["c"], "usd_micros": row["u"], "turns": row["c"],
"fallback_count": row["f"]}] "fallback_count": row["f"],
"cached_input_tokens": row["cached"],
"unpriced_actual_tokens": row["unpriced"]}]
elif days is not None: elif days is not None:
try: try:
n = int(days) n = int(days)
@@ -1112,13 +1118,20 @@ class CloudControlPlaneHandler(BaseHTTPRequestHandler):
rows = conn.execute( rows = conn.execute(
"SELECT day, COALESCE(SUM(total_tokens),0) AS t," "SELECT day, COALESCE(SUM(total_tokens),0) AS t,"
" COALESCE(SUM(usd_micros),0) AS u, COUNT(*) AS c," " COALESCE(SUM(usd_micros),0) AS u, COUNT(*) AS c,"
" COALESCE(SUM(fallback_count),0) AS f" " COALESCE(SUM(fallback_count),0) AS f,"
" COALESCE(SUM(cached_input_tokens),0) AS cached,"
" COALESCE(SUM(CASE WHEN source = 'actual'"
" AND usd_micros IS NULL THEN total_tokens"
" ELSE 0 END),0) AS unpriced"
" FROM cost_records WHERE project_id = ? AND day >= ?" " FROM cost_records WHERE project_id = ? AND day >= ?"
" GROUP BY day ORDER BY day", " GROUP BY day ORDER BY day",
(project_id, cutoff)).fetchall() (project_id, cutoff)).fetchall()
daily = [{"day": r["day"], "total_tokens": r["t"], daily = [{"day": r["day"], "total_tokens": r["t"],
"usd_micros": r["u"], "turns": r["c"], "usd_micros": r["u"], "turns": r["c"],
"fallback_count": r["f"]} for r in rows] "fallback_count": r["f"],
"cached_input_tokens": r["cached"],
"unpriced_actual_tokens": r["unpriced"]}
for r in rows]
finally: finally:
try: try:
store._conn.close() store._conn.close()
+75 -1
View File
@@ -8,13 +8,15 @@ import os
import sys import sys
import unittest import unittest
from datetime import datetime, timezone from datetime import datetime, timezone
from unittest import mock
# cloud/ has no __init__.py by design (the coordinator creates it), so # cloud/ has no __init__.py by design (the coordinator creates it), so
# make the project root importable regardless of the discover top level. # make the project root importable regardless of the discover top level.
sys.path.insert(0, os.path.dirname(os.path.dirname( sys.path.insert(0, os.path.dirname(os.path.dirname(
os.path.dirname(os.path.abspath(__file__))))) os.path.dirname(os.path.abspath(__file__)))))
from cloud.cost import CostStore, estimate_tokens, fetch_account_usage from cloud.cost import (CostStore, estimate_tokens, fetch_account_usage,
price_usd_micros)
from cloud.models import FALLBACKABLE, ModelRouter, is_fallbackable, record_fallback from cloud.models import FALLBACKABLE, ModelRouter, is_fallbackable, record_fallback
@@ -192,8 +194,80 @@ class TestCostStore(unittest.TestCase):
"fallback_count": 0, "fallback_count": 0,
"actual_tokens": 0, "actual_tokens": 0,
"estimated_tokens": 0, "estimated_tokens": 0,
"cached_input_tokens": 0,
"unpriced_actual_tokens": 0,
}) })
class TestModelPrices(unittest.TestCase):
def test_list_prices(self):
# 2026-10-26 中继实测 5 调用回合:(58243-45960)*2
# + 45960*0.10 + 1138*10 = 40542 微美元。
self.assertEqual(
price_usd_micros("gpt-6.1-sol", 58243, 45960, 1138), 40542)
self.assertEqual(
price_usd_micros("gpt-6-sol", 1000, 200, 100),
int(round((800 * 2.00 + 200 * 0.20 + 100 * 10.00))))
def test_unknown_model_unpriced(self):
self.assertIsNone(price_usd_micros("gpt-5.6-sol", 100, 0, 100))
self.assertIsNone(price_usd_micros(None, 100, 0, 100))
def test_cached_clamped_to_input(self):
self.assertEqual(
price_usd_micros("gpt-6.1-sol", 100, 500, 100),
price_usd_micros("gpt-6.1-sol", 100, 100, 100))
def test_env_override(self):
with mock.patch.dict(os.environ, {
"CLOUD_MODEL_PRICES_JSON":
'{"gpt-6.1-sol": [4.00, 0.20, 20.00]}'}):
self.assertEqual(
price_usd_micros("gpt-6.1-sol", 1000, 0, 0), 4000)
with mock.patch.dict(os.environ, {
"CLOUD_MODEL_PRICES_JSON": 'not json'}):
# 坏覆盖不炸,回落默认表。
self.assertEqual(
price_usd_micros("gpt-6.1-sol", 1000, 0, 0), 2000)
class TestCachedColumn(unittest.TestCase):
def test_record_keeps_cached_and_unpriced(self):
store = CostStore(":memory:")
store.record("t1", "p1", "gpt-6.1-sol", input_tokens=1000,
output_tokens=100, cached_input_tokens=400,
usd_micros=2240, source="actual")
store.record("t2", "p1", "gpt-5.6-sol", input_tokens=500,
output_tokens=50, cached_input_tokens=0,
usd_micros=None, source="actual")
report = store.report_by_project("p1")
self.assertEqual(report["cached_input_tokens"], 400)
self.assertEqual(report["unpriced_actual_tokens"], 550)
self.assertEqual(report["usd_micros"], 2240)
def test_migration_adds_cached_column_to_old_table(self):
import sqlite3
import tempfile
path = os.path.join(tempfile.mkdtemp(), "c.db")
conn = sqlite3.connect(path)
conn.execute(
"CREATE TABLE cost_records (id INTEGER PRIMARY KEY"
" AUTOINCREMENT, task_id TEXT, project_id TEXT, model TEXT,"
" input_tokens INTEGER, output_tokens INTEGER,"
" total_tokens INTEGER, usd_micros INTEGER, source TEXT,"
" fallback_count INTEGER DEFAULT 0, day TEXT,"
" created_at REAL)")
conn.commit()
conn.close()
store = CostStore(path)
cols = {row[1] for row in store._conn.execute(
"PRAGMA table_info(cost_records)").fetchall()}
self.assertIn("cached_input_tokens", cols)
store.record("t1", "p1", "m", input_tokens=10, output_tokens=5,
cached_input_tokens=3, source="actual")
self.assertEqual(store.report_by_task("t1")["cached_input_tokens"],
3)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+95
View File
@@ -8,6 +8,7 @@
top-level 目录无关。 top-level 目录无关。
""" """
import json
import os import os
import sys import sys
import unittest import unittest
@@ -385,5 +386,99 @@ class TestGoalExecutors(unittest.TestCase):
self.assertIn("test -f x", seen[0]) self.assertIn("test -f x", seen[0])
class TestUsageParsing(unittest.TestCase):
def test_result_from_runner_usage(self):
usage = {"input_tokens": 1000, "cached_input_tokens": 400,
"cache_write_input_tokens": 0, "output_tokens": 100,
"reasoning_output_tokens": 20, "total_tokens": 1100,
"completions": 2}
fake = SimpleNamespace(
returncode=0,
stdout=json.dumps({"status": "completed", "text": "done",
"usage": usage}),
stderr="")
result = LocalExecutor("/fake/host.py",
runner=lambda cmd: fake
)._result_from_runner("t1", fake)
self.assertEqual(result["usage"], usage)
self.assertEqual(result["status"], "completed")
def test_result_from_runner_usage_survives_long_text(self):
# usage 在 JSON 末尾:旧的截头解析在 text 很长时会丢 usage、
# 还会把 failed 误判成 completed;解析必须用完整 stdout。
usage = {"input_tokens": 100, "cached_input_tokens": 0,
"cache_write_input_tokens": 0, "output_tokens": 10,
"reasoning_output_tokens": 0, "total_tokens": 110,
"completions": 1}
fake = SimpleNamespace(
returncode=0,
stdout=json.dumps({"status": "failed", "text": "x" * 5000,
"usage": usage}),
stderr="")
result = LocalExecutor("/fake/host.py",
runner=lambda cmd: fake
)._result_from_runner("t1", fake)
self.assertEqual(result["status"], "failed")
self.assertEqual(result["usage"], usage)
def test_result_from_runner_dirty_usage_dropped(self):
for bad in ({"input_tokens": "abc"},
{"input_tokens": 0, "output_tokens": 0,
"total_tokens": 0},
"not-a-dict"):
fake = SimpleNamespace(
returncode=0,
stdout=json.dumps({"status": "completed", "usage": bad}),
stderr="")
result = LocalExecutor("/fake/host.py",
runner=lambda cmd: fake
)._result_from_runner("t1", fake)
self.assertIsNone(result["usage"])
def test_goal_usage_line(self):
fake = SimpleNamespace(
returncode=0,
stdout="round 1: turn=completed verify_rc=0 verify_out=ok\n"
"goal met after rounds\n"
"usage input=100 cached=40 cache_write=0 output=50 "
"reasoning=10 total=150 completions=2\n",
stderr="")
result = LocalExecutor("/fake/host.py",
runner=lambda cmd: fake
)._goal_result_from_runner("t1", fake)
self.assertEqual(result["usage"], {
"input_tokens": 100, "cached_input_tokens": 40,
"cache_write_input_tokens": 0, "output_tokens": 50,
"reasoning_output_tokens": 10, "total_tokens": 150,
"completions": 2})
fake2 = SimpleNamespace(
returncode=0, stdout="goal met after rounds\n", stderr="")
result2 = LocalExecutor("/fake/host.py",
runner=lambda cmd: fake2
)._goal_result_from_runner("t1", fake2)
self.assertIsNone(result2["usage"])
def test_docker_goal_usage_line(self):
def fake_runner(cmd, timeout=None):
return SimpleNamespace(
returncode=0,
stdout="round 1: turn=completed verify_rc=0 verify_out=ok\n"
"goal met after rounds\n"
"usage input=100 cached=40 cache_write=0 output=50 "
"reasoning=10 total=150 completions=2\n",
stderr="")
with mock.patch("sandbox.shutil.which",
return_value="/usr/bin/docker"):
executor = DockerExecutor(
"/fake/host.py", runner=fake_runner,
env={"XCM_API_KEY": "placeholder-not-a-real-key"})
result = executor.goal_task(
{"id": "t1", "prompt": "do it",
"verify_command": "test -f x"})
self.assertEqual(result["usage"]["total_tokens"], 150)
self.assertEqual(result["usage"]["cached_input_tokens"], 40)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+69
View File
@@ -374,6 +374,75 @@ class WorkerTestCase(unittest.TestCase):
self.assertEqual(results[0]["status"], "failed") self.assertEqual(results[0]["status"], "failed")
self.assertIn("不支持 goal", results[0]["error"]) self.assertIn("不支持 goal", results[0]["error"])
# -- 成本:真实用量 --------------------------------------------------
def test_cost_actual_from_usage(self):
"""执行器带回用量时记 actual:token 全是实测值,usd 按
价目表算((1000-400)*2 + 400*0.10 + 100*10 = 2240 微美元)。"""
_insert_task(self.db_path, "A")
usage = {"input_tokens": 1000, "cached_input_tokens": 400,
"cache_write_input_tokens": 0, "output_tokens": 100,
"reasoning_output_tokens": 20, "total_tokens": 1100,
"completions": 1}
def runner(cmd):
return SimpleNamespace(
returncode=0,
stdout=json.dumps({"status": "completed", "text": "done",
"usage": usage}),
stderr="")
worker = self._worker(runner=runner, model="gpt-6.1-sol")
results = worker.run_all()
self.assertEqual([r["status"] for r in results], ["completed"])
from cloud.cost import CostStore
report = CostStore(self.db_path).report_by_task("A")
self.assertEqual(report["actual_tokens"], 1100)
self.assertEqual(report["cached_input_tokens"], 400)
self.assertEqual(report["estimated_tokens"], 0)
self.assertEqual(report["usd_micros"], 2240)
self.assertEqual(report["unpriced_actual_tokens"], 0)
def test_cost_actual_unpriced_model(self):
"""无价模型:token 照样记 actual,usd 记 None、报表里
进 unpriced_actual_tokens,绝不按 0 美元处理。"""
_insert_task(self.db_path, "A")
usage = {"input_tokens": 1000, "cached_input_tokens": 0,
"cache_write_input_tokens": 0, "output_tokens": 100,
"reasoning_output_tokens": 0, "total_tokens": 1100,
"completions": 1}
def runner(cmd):
return SimpleNamespace(
returncode=0,
stdout=json.dumps({"status": "completed", "text": "done",
"usage": usage}),
stderr="")
worker = self._worker(runner=runner, model="gpt-5.6-sol")
results = worker.run_all()
self.assertEqual([r["status"] for r in results], ["completed"])
from cloud.cost import CostStore
report = CostStore(self.db_path).report_by_task("A")
self.assertEqual(report["actual_tokens"], 1100)
self.assertEqual(report["usd_micros"], 0)
self.assertEqual(report["unpriced_actual_tokens"], 1100)
def test_cost_estimated_when_no_usage(self):
"""执行器没带回用量时回落字符估算记 estimated(旧行为)。"""
_insert_task(self.db_path, "A")
def runner(cmd):
return SimpleNamespace(returncode=0, stdout="plain text",
stderr="")
worker = self._worker(runner=runner, model="gpt-6.1-sol")
results = worker.run_all()
self.assertEqual([r["status"] for r in results], ["completed"])
from cloud.cost import CostStore
report = CostStore(self.db_path).report_by_task("A")
self.assertEqual(report["actual_tokens"], 0)
self.assertGreater(report["estimated_tokens"], 0)
# -- 取消 ------------------------------------------------------------ # -- 取消 ------------------------------------------------------------
def test_cancel_queued_task_marks_interrupted(self): def test_cancel_queued_task_marks_interrupted(self):
_insert_task(self.db_path, "A") _insert_task(self.db_path, "A")
+39 -14
View File
@@ -26,9 +26,12 @@
4. 写回:server 的 tasks 表没有 result/error 列,所以执行结果 4. 写回:server 的 tasks 表没有 result/error 列,所以执行结果
的持久化分三处,各复用已有结构、不另起字段名: 的持久化分三处,各复用已有结构、不另起字段名:
- tasks 表:最终状态 + updated_at(取消时为 interrupted); - tasks 表:最终状态 + updated_at(取消时为 interrupted);
- cost.py CostStore:每轮实际执行记一条 source="estimated" - cost.py CostStore:每轮实际执行记一条成本记录;执行器带回
的成本记录(host 协议不暴露真实 token 数,估算绝不冒充 上游真实用量(host.py 从 tokenUsage/updated 事件汇总)时记
actual;total_tokens 由 CostStore.record 按 input+output 算); source="actual"(input/output/cached 全是实测值,usd 按价目
表算、无价模型记 None);没带回用量时回落 cost.py 估算记
estimated,估算绝不冒充 actual;total_tokens 由
CostStore.record 按 input+output 算;
- models.py:可 fallback 的失败(404/429/5xx/timeout,401/400 - models.py:可 fallback 的失败(404/429/5xx/timeout,401/400
不算)。默认(未配 --fallback-models)时 worker 只记录 不算)。默认(未配 --fallback-models)时 worker 只记录
fallback 事件(record_fallback() 记 model_fallback 事件, fallback 事件(record_fallback() 记 model_fallback 事件,
@@ -122,7 +125,7 @@ import time
from types import SimpleNamespace from types import SimpleNamespace
from cloud.auth import AuthStore from cloud.auth import AuthStore
from cloud.cost import CostStore, estimate_tokens from cloud.cost import CostStore, estimate_tokens, price_usd_micros, sum_usage
from cloud.db import connect as _db_connect from cloud.db import connect as _db_connect
from cloud.logging_config import get_logger, log_event, setup_logging from cloud.logging_config import get_logger, log_event, setup_logging
from cloud.models import ModelRouter, is_fallbackable, record_fallback from cloud.models import ModelRouter, is_fallbackable, record_fallback
@@ -1455,8 +1458,12 @@ class Worker:
# fallback_models 时保持老行为(router 给出下一模型时只记事件 # fallback_models 时保持老行为(router 给出下一模型时只记事件
# 不重试);其它失败(含 worker 超时)按 max_retries 做同模型 # 不重试);其它失败(含 worker 超时)按 max_retries 做同模型
# 重试,退避等待后重跑一次 _run_attempt。 # 重试,退避等待后重跑一次 _run_attempt。
# 各次尝试的真实用量都要算进成本(失败尝试花的钱也是钱):
# 主尝试与 fallback 尝试的结果都先累加,最后只记一条汇总。
usage_total = None
for attempt in range(self.max_retries + 1): for attempt in range(self.max_retries + 1):
result, error = self._run_attempt(task) result, error = self._run_attempt(task)
usage_total = sum_usage(usage_total, (result or {}).get("usage"))
if self._was_cancelled(task_id) or \ if self._was_cancelled(task_id) or \
result.get("status") == "cancelled": result.get("status") == "cancelled":
break break
@@ -1493,6 +1500,8 @@ class Worker:
task = dict(task) task = dict(task)
task["model"] = fb task["model"] = fb
result, error = self._run_attempt(task) result, error = self._run_attempt(task)
usage_total = sum_usage(usage_total,
(result or {}).get("usage"))
break break
if self.router is not None and host_status_try is not None \ if self.router is not None and host_status_try is not None \
and is_fallbackable(host_status_try) and \ and is_fallbackable(host_status_try) and \
@@ -1585,16 +1594,32 @@ class Worker:
finished_at=finished_at) finished_at=finished_at)
with self._db_lock: with self._db_lock:
self.usage_turns += 1 self.usage_turns += 1
# 成本:host 协议不暴露真实 token 数,用 cost.py 的估算结构 # 成本:执行器带回上游真实用量时记 actual(input/output/
# 记 estimated,绝不标 actual;fallback_count 复用同一记录; # cached 全是实测值,usd 按价目表算、无价模型记 None,
# 模型记最后真正跑的那个(fallback 成功时就是 fallback 模型)。 # 报表里进 unpriced_actual_tokens 而不是 0 美元);
# cost 记录共用一条连接,并发下必须持锁写,否则两个线程 # 没带回用量时回落字符估算记 estimated,绝不冒充 actual。
# 共用同一 sqlite 连接会直接报错。 # fallback_count 复用同一记录;模型记最后真正跑的那个
total = estimate_tokens(task.get("prompt") or "", stdout) # (fallback 成功时就是 fallback 模型)。cost 记录共用
cost_id = self.costs.record( # 一条连接,并发下必须持锁写,否则两个线程共用同一
task_id, task["project_id"], actual_model, # sqlite 连接会直接报错。
input_tokens=total, output_tokens=0, usd_micros=None, if usage_total and (usage_total.get("total_tokens") or 0) > 0:
source="estimated", fallback_count=fallback_count) cost_id = self.costs.record(
task_id, task["project_id"], actual_model,
input_tokens=usage_total.get("input_tokens", 0),
output_tokens=usage_total.get("output_tokens", 0),
cached_input_tokens=usage_total.get(
"cached_input_tokens", 0),
usd_micros=price_usd_micros(
actual_model, usage_total.get("input_tokens", 0),
usage_total.get("cached_input_tokens", 0),
usage_total.get("output_tokens", 0)),
source="actual", fallback_count=fallback_count)
else:
total = estimate_tokens(task.get("prompt") or "", stdout)
cost_id = self.costs.record(
task_id, task["project_id"], actual_model,
input_tokens=total, output_tokens=0, usd_micros=None,
source="estimated", fallback_count=fallback_count)
if final_written: if final_written:
self._audit(task["project_id"], self._audit(task["project_id"],
"worker_completed" if final == "completed" "worker_completed" if final == "completed"
+79 -3
View File
@@ -59,6 +59,56 @@ HERE = os.path.dirname(os.path.abspath(__file__))
DEFAULT_STATE = os.path.join(HERE, "host-state.json") DEFAULT_STATE = os.path.join(HERE, "host-state.json")
def _sum_turn_usage(events, thread_id, turn_id):
"""从事件窗口里汇总一回合的上游真实用量。
app-server 在每个上游补全结束时发 thread/tokenUsage/updated,
params.tokenUsage.last 是该次补全的用量(2026-10-26 中继实测:
同一 turn 的多次 last 之和等于最后一次的 total,input/output/
cached/reasoning 四路都对得上)。按 threadId + turnId 过滤后
累加 last,得到本回合的真实用量;无匹配通知时返回 None(老
版本 server 或回合未产生补全时调用方回落估算,绝不把 0 当成
真实用量)。注意 totalTokens = inputTokens + outputTokens,
reasoning 包含在 output 里,累加时不另算。
"""
fields = (("input_tokens", "inputTokens"),
("cached_input_tokens", "cachedInputTokens"),
("cache_write_input_tokens", "cacheWriteInputTokens"),
("output_tokens", "outputTokens"),
("reasoning_output_tokens", "reasoningOutputTokens"),
("total_tokens", "totalTokens"))
total = None
for ev in events or []:
if ev.get("method") != "thread/tokenUsage/updated":
continue
p = ev.get("params") or {}
if p.get("threadId") != thread_id:
continue
if turn_id is not None and p.get("turnId") != turn_id:
continue
last = (p.get("tokenUsage") or {}).get("last") or {}
if total is None:
total = {k: 0 for k, _ in fields}
total["completions"] = 0
for key, wire in fields:
total[key] += int(last.get(wire) or 0)
total["completions"] += 1
return total
def _add_usage(a, b):
"""两份用量相加(任一为空时返回另一份;都为空返回 None)。"""
if not a:
return dict(b) if b else None
if not b:
return dict(a)
out = dict(a)
for k, v in b.items():
if isinstance(v, int):
out[k] = out.get(k, 0) + v
return out
def default_codex_bin(): def default_codex_bin():
if os.environ.get("CODEX_BIN"): if os.environ.get("CODEX_BIN"):
return os.environ["CODEX_BIN"] return os.environ["CODEX_BIN"]
@@ -298,7 +348,10 @@ class Server:
try: try:
done = q.get(timeout=timeout) done = q.get(timeout=timeout)
except queue.Empty: except queue.Empty:
return {"status": "timeout", "text": "", "turn_id": turn_id} with self.events_lock:
evs = list(self.events[start_events:])
return {"status": "timeout", "text": "", "turn_id": turn_id,
"usage": _sum_turn_usage(evs, thread_id, turn_id)}
turn = (done.get("params") or {}).get("turn") or {} turn = (done.get("params") or {}).get("turn") or {}
text = "" text = ""
for item in turn.get("items") or []: for item in turn.get("items") or []:
@@ -329,10 +382,16 @@ class Server:
# 没有原因,绝不因此把回合本身搞失败。 # 没有原因,绝不因此把回合本身搞失败。
error = self._turn_error_from_history( error = self._turn_error_from_history(
thread_id, turn.get("id") or turn_id) thread_id, turn.get("id") or turn_id)
# 真实用量:本回合窗口内的 tokenUsage/updated 通知累加。
# 失败/超时回合也一样记(已产生的补全就是已花的钱)。
with self.events_lock:
evs = list(self.events[start_events:])
usage = _sum_turn_usage(
evs, thread_id, turn.get("id") or turn_id)
return {"status": status, "text": text, return {"status": status, "text": text,
"turn_id": turn.get("id") or turn_id, "turn_id": turn.get("id") or turn_id,
"root_turn_id": turn.get("rootTurnId"), "root_turn_id": turn.get("rootTurnId"),
"error": error} "error": error, "usage": usage}
finally: finally:
self.turn_queues.pop(thread_id, None) self.turn_queues.pop(thread_id, None)
@@ -1261,6 +1320,7 @@ def run_task_turn_with_retry(server, state, task, prompt, model, effort,
每次重试前先查预算,超预算直接停,不烧钱。""" 每次重试前先查预算,超预算直接停,不烧钱。"""
max_retries = int(task.get("max_retries") or 0) max_retries = int(task.get("max_retries") or 0)
result = None result = None
usage = None
for attempt in range(max_retries + 1): for attempt in range(max_retries + 1):
if attempt > 0: if attempt > 0:
task["retry_count"] = attempt task["retry_count"] = attempt
@@ -1277,8 +1337,13 @@ def run_task_turn_with_retry(server, state, task, prompt, model, effort,
break break
result = run_task_turn(server, state, task, prompt_try, result = run_task_turn(server, state, task, prompt_try,
model, effort, sandbox) model, effort, sandbox)
# 各次尝试的真实用量都要算(失败尝试花的钱也是钱),
# 最后一次的结果里带汇总,不只带最后一次。
usage = _add_usage(usage, result.get("usage"))
if result["status"] == "completed": if result["status"] == "completed":
break break
if usage is not None and result is not None:
result = dict(result, usage=usage)
return result return result
@@ -1670,7 +1735,8 @@ def main():
print(json.dumps({"task": args.name, "thread_id": task["thread_id"], print(json.dumps({"task": args.name, "thread_id": task["thread_id"],
"cwd": task["cwd"], "status": task["status"], "cwd": task["cwd"], "status": task["status"],
"text": result.get("text", ""), "text": result.get("text", ""),
"error": result.get("error")}, "error": result.get("error"),
"usage": result.get("usage")},
indent=2, ensure_ascii=False)) indent=2, ensure_ascii=False))
elif args.cmd == "list": elif args.cmd == "list":
for name, t in state["tasks"].items(): for name, t in state["tasks"].items():
@@ -1847,6 +1913,7 @@ def main():
save_state(args.state, state) save_state(args.state, state)
ok = False ok = False
last_verify = "" last_verify = ""
goal_usage = None
for round_no in range(1, args.max_rounds + 1): for round_no in range(1, args.max_rounds + 1):
if round_no == 1: if round_no == 1:
prompt = args.task prompt = args.task
@@ -1860,6 +1927,7 @@ def main():
f"上一轮验收输出:\n{last_verify}") f"上一轮验收输出:\n{last_verify}")
result = run_task_turn(srv, state, task, prompt, result = run_task_turn(srv, state, task, prompt,
args.model, args.effort, args.sandbox) args.model, args.effort, args.sandbox)
goal_usage = _add_usage(goal_usage, result.get("usage"))
v = subprocess.run(args.verify, shell=True, cwd=task["cwd"], v = subprocess.run(args.verify, shell=True, cwd=task["cwd"],
capture_output=True, text=True) capture_output=True, text=True)
last_verify = (v.stdout + v.stderr).strip()[:1000] or \ last_verify = (v.stdout + v.stderr).strip()[:1000] or \
@@ -1875,6 +1943,14 @@ def main():
{"status": "goal-unmet", "text": last_verify[:200]}) {"status": "goal-unmet", "text": last_verify[:200]})
save_state(args.state, state) save_state(args.state, state)
print(f"goal {'met' if ok else 'unmet'} after rounds") print(f"goal {'met' if ok else 'unmet'} after rounds")
if goal_usage:
print(f"usage input={goal_usage['input_tokens']} "
f"cached={goal_usage['cached_input_tokens']} "
f"cache_write={goal_usage['cache_write_input_tokens']} "
f"output={goal_usage['output_tokens']} "
f"reasoning={goal_usage['reasoning_output_tokens']} "
f"total={goal_usage['total_tokens']} "
f"completions={goal_usage['completions']}")
elif args.cmd == "parallel": elif args.cmd == "parallel":
jobs = json.load(open(args.jobs)) jobs = json.load(open(args.jobs))
# 依赖 + 优先级:先拓扑排序,缺依赖/成环直接报错不跑; # 依赖 + 优先级:先拓扑排序,缺依赖/成环直接报错不跑;
+79
View File
@@ -572,6 +572,85 @@ class GithubPRTest(unittest.TestCase):
self.assertIn("remote", r.stdout + r.stderr) self.assertIn("remote", r.stdout + r.stderr)
class TurnUsageSumTest(unittest.TestCase):
def _note(self, thread_id, turn_id, last):
return {"method": "thread/tokenUsage/updated",
"params": {"threadId": thread_id, "turnId": turn_id,
"tokenUsage": {"last": last}}}
def test_sums_last_across_completions(self):
# 2026-10-26 中继实测前两次通知:last 之和等于第二次的 total。
evs = [
self._note("t1", "u1", {
"inputTokens": 10906, "cachedInputTokens": 9192,
"cacheWriteInputTokens": 0, "outputTokens": 346,
"reasoningOutputTokens": 256, "totalTokens": 11252}),
self._note("t1", "u1", {
"inputTokens": 11041, "cachedInputTokens": 9192,
"cacheWriteInputTokens": 0, "outputTokens": 230,
"reasoningOutputTokens": 161, "totalTokens": 11271}),
]
self.assertEqual(host._sum_turn_usage(evs, "t1", "u1"), {
"input_tokens": 21947, "cached_input_tokens": 18384,
"cache_write_input_tokens": 0, "output_tokens": 576,
"reasoning_output_tokens": 417, "total_tokens": 22523,
"completions": 2})
def test_filters_by_thread_and_turn(self):
evs = [
self._note("t1", "u1", {
"inputTokens": 10, "cachedInputTokens": 0,
"cacheWriteInputTokens": 0, "outputTokens": 5,
"reasoningOutputTokens": 0, "totalTokens": 15}),
self._note("t1", "u2", {
"inputTokens": 999, "cachedInputTokens": 0,
"cacheWriteInputTokens": 0, "outputTokens": 999,
"reasoningOutputTokens": 0, "totalTokens": 1998}),
self._note("t2", "u1", {
"inputTokens": 999, "cachedInputTokens": 0,
"cacheWriteInputTokens": 0, "outputTokens": 999,
"reasoningOutputTokens": 0, "totalTokens": 1998}),
{"method": "item/completed", "params": {}},
]
usage = host._sum_turn_usage(evs, "t1", "u1")
self.assertEqual(usage["input_tokens"], 10)
self.assertEqual(usage["completions"], 1)
self.assertIsNone(host._sum_turn_usage(evs, "t1", "u9"))
self.assertIsNone(host._sum_turn_usage([], "t1", "u1"))
def test_retry_sums_usage_across_attempts(self):
class FakeServer:
def __init__(self, results):
self.results = list(results)
def run_turn(self, thread_id, prompt, model=None, effort=None,
sandbox=None):
return self.results.pop(0)
task = {"name": "t", "thread_id": "x", "sandbox": None,
"max_retries": 1, "retry_count": 0, "turns": []}
state = {"tasks": {"t": task}, "automations": []}
srv = FakeServer([
{"status": "failed", "text": "", "usage": {
"input_tokens": 100, "cached_input_tokens": 0,
"cache_write_input_tokens": 0, "output_tokens": 10,
"reasoning_output_tokens": 0, "total_tokens": 110,
"completions": 1}},
{"status": "completed", "text": "ok", "usage": {
"input_tokens": 200, "cached_input_tokens": 50,
"cache_write_input_tokens": 0, "output_tokens": 20,
"reasoning_output_tokens": 5, "total_tokens": 220,
"completions": 1}},
])
result = host.run_task_turn_with_retry(
srv, state, task, "do it", "m", "low", cfg={})
self.assertEqual(result["status"], "completed")
self.assertEqual(result["usage"]["input_tokens"], 300)
self.assertEqual(result["usage"]["cached_input_tokens"], 50)
self.assertEqual(result["usage"]["total_tokens"], 330)
self.assertEqual(result["usage"]["completions"], 2)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()