websearch: 默认 auto 付费冻结，SearXNG 优先 + 显式解冻开关
- 默认 auto 只走免费链 searxng -> duckduckgo；即使环境里有 付费 key 也不调用 Perplexity/Exa/Tavily/Brave，SearXNG 失败也不回落付费，默认使用零付费花费。 - 付费后端代码/测试/文档全部保留；显式 pin（tavily/exa/ perplexity/brave）仍照常调用，是有意覆盖不是默认。 - 解冻开关 WEBSEARCH_ENABLE_PAID，仅精确值 "1" 才恢复旧链 （perplexity/exa/tavily/brave/searxng/duckduckgo），默认关； 开关经 worker search_env 透传进 MCP config env。 - HttpClient 超时/DNS 失败改报显式错误（timeout after Ns / URL error），绝不把超时写成“无结果”。 - SearXNG 默认参数核实：代码无 SEARXNG_URL 硬编码默认值； SEARXNG_ENGINES 默认 bing,360search 已在基线；WEBSEARCH_TIMEOUT 默认 20 秒。主机 env 实测仅 SEARXNG_URL=http://searx.diante.me 一行，故默认 pin 生效；文档 https 漂移改回 http（主机 nginx 仅听 80、无 443 vhost，https 在 TLS 层即失败）。 - 旧测试更新原因：test_websearch_mcp 原断言默认 auto 为 perplexity 优先，与新冻结语义直接冲突，故改为断言冻结后 的 searxng -> duckduckgo 回退 + 不泄漏 key；旧 perplexity -> exa 回退断言移至解冻开关测试（WEBSEARCH_ENABLE_PAID=1 下）。 未删断言、未加 skip、未放宽门禁。 - 文档另删 WEBSEARCH_PROVIDERS 说法：代码从未实现该变量。 - 未推、未建 PR、未合并、未部署、未改主机 env/nginx/filter/ 证书、未做付费后端 live 验证、未创建生产任务。
This commit is contained in:
host committed
1 parent
0602d06136
commit
3a56f83114
7 files changed
+492
-43
No files matched your search
+69
-5
@@ -21,6 +21,16 @@ Design notes (what is copied from oh-my-pi, and what is not):
|
||||
here is a small stdlib HTML parser: script/style stripped, text
|
||||
collapsed, long pages truncated head+tail like Hermes' deterministic
|
||||
budget (default 15000 chars, 75% head / 25% tail, ``[TRUNCATED]``).
|
||||
- FROZEN default chain (paid freeze): the default ``auto`` chain
|
||||
contains only the free backends, SearXNG then DuckDuckGo, even
|
||||
when paid keys happen to be set -- a SearXNG failure never falls
|
||||
back to a paid backend, so default use can never incur paid cost.
|
||||
The four paid backends (Perplexity, Exa, Tavily, Brave) keep their
|
||||
code, tests and docs, and still run when explicitly pinned
|
||||
(provider="tavily" etc.), which is a deliberate override, not a
|
||||
default. Only WEBSEARCH_ENABLE_PAID=1 (exact value "1", default
|
||||
off) restores the old default chain; unfreezing is an explicit act
|
||||
and may incur paid cost, never a silent default.
|
||||
|
||||
Security notes:
|
||||
- API keys come only from the process environment, never from
|
||||
@@ -57,6 +67,9 @@ from html.parser import HTMLParser
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_CHAIN",
|
||||
"DEFAULT_FREE_CHAIN",
|
||||
"PAID_PROVIDERS",
|
||||
"paid_search_enabled",
|
||||
"CODE_SEARCH_DOMAINS",
|
||||
"WebSearchError",
|
||||
"SearchResult",
|
||||
@@ -72,6 +85,12 @@ __all__ = [
|
||||
|
||||
DEFAULT_CHAIN = ("perplexity", "exa", "tavily", "brave", "searxng",
|
||||
"duckduckgo")
|
||||
# Paid backends, frozen out of the default auto chain (see the module
|
||||
# docstring). DEFAULT_CHAIN keeps the legacy full order so the
|
||||
# unfrozen chain is exactly the old one; DEFAULT_FREE_CHAIN is what
|
||||
# default auto actually walks while frozen.
|
||||
PAID_PROVIDERS = ("perplexity", "exa", "tavily", "brave")
|
||||
DEFAULT_FREE_CHAIN = ("searxng", "duckduckgo")
|
||||
DEFAULT_FETCH_CHAR_LIMIT = 15000
|
||||
|
||||
# Programming-only search scope (``search(..., scope="code")``).
|
||||
@@ -246,6 +265,27 @@ class HttpClient:
|
||||
raise WebSearchError(
|
||||
"HTTP %s" % exc.code) from _ProviderHttpError(
|
||||
exc.code, body)
|
||||
except (socket.timeout, TimeoutError) as exc:
|
||||
# A timeout is a timeout, never "no results": the caller
|
||||
# (search's attempted entry, named by provider) must be
|
||||
# able to tell which stage timed out, and a timed-out
|
||||
# SearXNG must not be rewritten as an empty answer.
|
||||
# HTTPError is a URLError subclass, so this branch must
|
||||
# stay after the HTTPError branch above.
|
||||
raise WebSearchError(
|
||||
"timeout after %ss waiting for HTTP response"
|
||||
% self.timeout) from exc
|
||||
except urllib.error.URLError as exc:
|
||||
reason = getattr(exc, "reason", None)
|
||||
if isinstance(reason, (socket.timeout, TimeoutError)):
|
||||
raise WebSearchError(
|
||||
"timeout after %ss waiting for HTTP response"
|
||||
% self.timeout) from exc
|
||||
# DNS / connection failures keep their own message;
|
||||
# they are not timeouts and must not be relabelled.
|
||||
raise WebSearchError(
|
||||
"URL error: %s" % reason if reason else "URL error"
|
||||
) from exc
|
||||
|
||||
def get_text(self, url, headers=None):
|
||||
status, body, _ctype = self.get(url, headers)
|
||||
@@ -534,11 +574,32 @@ _PROVIDERS = {
|
||||
}
|
||||
|
||||
|
||||
def available_providers(env=None):
|
||||
"""Providers usable right now: keyed ones with a key, plus free ones."""
|
||||
def paid_search_enabled(env=None):
|
||||
"""True only when the paid-unfreeze switch is explicitly on.
|
||||
|
||||
The switch is WEBSEARCH_ENABLE_PAID, and only the exact value
|
||||
"1" counts (same strict convention as WEBSEARCH_ALLOW_PRIVATE_URLS
|
||||
elsewhere in this module): any other value, including "true" or
|
||||
"yes", leaves the paid backends frozen out of default auto.
|
||||
Explicit pinning is unaffected by this switch either way.
|
||||
"""
|
||||
env = env if env is not None else os.environ
|
||||
return env.get("WEBSEARCH_ENABLE_PAID") == "1"
|
||||
|
||||
|
||||
def available_providers(env=None):
|
||||
"""Providers usable right now, under the paid-freeze rule.
|
||||
|
||||
Frozen (default): only the free chain -- SearXNG when its URL
|
||||
is set, plus DuckDuckGo -- even when paid keys are present.
|
||||
Unfrozen (WEBSEARCH_ENABLE_PAID=1): the legacy full chain,
|
||||
keyed backends only when their key is set, as before.
|
||||
"""
|
||||
env = env if env is not None else os.environ
|
||||
chain = (DEFAULT_CHAIN if paid_search_enabled(env)
|
||||
else DEFAULT_FREE_CHAIN)
|
||||
out = []
|
||||
for name in DEFAULT_CHAIN:
|
||||
for name in chain:
|
||||
if name in _FREE_PROVIDERS:
|
||||
if name == "searxng" and not env.get("SEARXNG_URL"):
|
||||
continue
|
||||
@@ -552,8 +613,11 @@ def resolve_provider_chain(provider="auto", env=None):
|
||||
"""Resolve the chain oh-my-pi style: auto walks, pinned does not.
|
||||
|
||||
``provider`` may be "auto" (default chain, filtered to what is
|
||||
configured), a single name (that provider only, even if it fails),
|
||||
or a comma-separated list (that order only). Unknown names raise
|
||||
configured -- frozen by default to the free chain, SearXNG then
|
||||
DuckDuckGo, unless WEBSEARCH_ENABLE_PAID=1 restores the legacy
|
||||
full chain), a single name (that provider only, even if it fails
|
||||
-- pinning a paid backend bypasses the freeze, deliberately), or
|
||||
a comma-separated list (that order only). Unknown names raise
|
||||
ValueError before any HTTP happens.
|
||||
"""
|
||||
env = env if env is not None else os.environ
|
||||
|
||||
Reference in new issue
Block a user