websearch: SearXNG 默认只查 bing,360search
默认 general 类别在 Docker 任务容器里要 20.0 秒才回 (wikipedia/wikidata/brave/duckduckgo/google 引擎全超时, 只有 bing+360search 回),撞 MCP 默认 20 秒超时;随后 DuckDuckGo 经代理也超时,整条链失败。代理本身无问题 (Tinyproxy 白名单早已放行 searx 与 duckduckgo)。 修法:_provider_searxng 默认 engines=bing,360search, SEARXNG_ENGINES 可覆盖、空值恢复旧行为;worker 搜索 env 透传补 SEARXNG_ENGINES。容器实测:钉住后 0.26 秒 回 3 条(首条北京天气),auto 链 0.15 秒成功;旧 20.0 秒。 测试:新增 test_websearch_searxng 3 项,MCP/worker 共 21 项全过。生产未部署。
This commit is contained in:
1 parent
79cc59e72e
commit
08d56b15ce
4 files changed
+92
-3
No files matched your search
@@ -0,0 +1,52 @@
|
|||||||
|
"""SearXNG provider: engines are pinned, not left to the default
|
||||||
|
general category (2026-10-30: default took 20s in a Docker task,
|
||||||
|
pinned bing,360search took 0.47s via the same egress proxy)."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||||
|
from cloud.websearch import _provider_searxng # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeHttp:
|
||||||
|
def __init__(self):
|
||||||
|
self.urls = []
|
||||||
|
|
||||||
|
def get(self, url, headers=None):
|
||||||
|
self.urls.append(url)
|
||||||
|
body = json.dumps({"results": [
|
||||||
|
{"title": "Beijing weather", "url": "https://example.test/w",
|
||||||
|
"content": "sunny"}]}).encode("utf-8")
|
||||||
|
return 200, body, "application/json"
|
||||||
|
|
||||||
|
|
||||||
|
class TestSearXNGEngines(unittest.TestCase):
|
||||||
|
def test_default_pins_bing_360search(self):
|
||||||
|
http = _FakeHttp()
|
||||||
|
results, _ = _provider_searxng(
|
||||||
|
"Beijing weather", 3, {"SEARXNG_URL": "http://searx.test"},
|
||||||
|
http)
|
||||||
|
self.assertEqual(len(results), 1)
|
||||||
|
self.assertIn("engines=bing%2C360search", http.urls[0])
|
||||||
|
|
||||||
|
def test_env_overrides_pin(self):
|
||||||
|
http = _FakeHttp()
|
||||||
|
_provider_searxng(
|
||||||
|
"q", 3, {"SEARXNG_URL": "http://searx.test",
|
||||||
|
"SEARXNG_ENGINES": "bing"}, http)
|
||||||
|
self.assertIn("engines=bing", http.urls[0])
|
||||||
|
self.assertNotIn("360search", http.urls[0])
|
||||||
|
|
||||||
|
def test_empty_env_restores_unpinned(self):
|
||||||
|
http = _FakeHttp()
|
||||||
|
_provider_searxng(
|
||||||
|
"q", 3, {"SEARXNG_URL": "http://searx.test",
|
||||||
|
"SEARXNG_ENGINES": ""}, http)
|
||||||
|
self.assertNotIn("engines=", http.urls[0])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
+16
-2
@@ -267,8 +267,22 @@ def _provider_searxng(query, limit, env, http):
|
|||||||
base = (env.get("SEARXNG_URL") or "").rstrip("/")
|
base = (env.get("SEARXNG_URL") or "").rstrip("/")
|
||||||
if not base:
|
if not base:
|
||||||
raise WebSearchError("searxng not configured (SEARXNG_URL unset)")
|
raise WebSearchError("searxng not configured (SEARXNG_URL unset)")
|
||||||
url = (base + "/search?"
|
# Engines are pinned, not left to SearXNG's default "general"
|
||||||
+ urllib.parse.urlencode({"q": query, "format": "json"}))
|
# category. Measured 2026-10-30 on the production host:
|
||||||
|
# default general search took 20.0s in a Docker task container
|
||||||
|
# (wikipedia / wikidata / brave / duckduckgo / google engines
|
||||||
|
# all timing out, only bing + 360search returning), which hits
|
||||||
|
# the MCP's 20s timeout and then falls through to DuckDuckGo,
|
||||||
|
# which itself times out via the egress proxy -- so the whole
|
||||||
|
# chain fails even though SearXNG can answer. Pinning to the
|
||||||
|
# two proven-fast engines cuts the same query to 0.47s via
|
||||||
|
# the same proxy. SEARXNG_ENGINES overrides the pin; an empty
|
||||||
|
# value restores the old unpinned behaviour.
|
||||||
|
params = {"q": query, "format": "json"}
|
||||||
|
engines = env.get("SEARXNG_ENGINES", "bing,360search")
|
||||||
|
if engines:
|
||||||
|
params["engines"] = engines
|
||||||
|
url = base + "/search?" + urllib.parse.urlencode(params)
|
||||||
_status, body, _ctype = http.get(url, _default_headers(env, "searxng"))
|
_status, body, _ctype = http.get(url, _default_headers(env, "searxng"))
|
||||||
data = json.loads(body.decode("utf-8", errors="replace"))
|
data = json.loads(body.decode("utf-8", errors="replace"))
|
||||||
results = []
|
results = []
|
||||||
|
|||||||
+2
-1
@@ -2816,7 +2816,8 @@ class Worker:
|
|||||||
# 走 extra_env(容器内进程本就需要它调 API)。
|
# 走 extra_env(容器内进程本就需要它调 API)。
|
||||||
search_env = {
|
search_env = {
|
||||||
k: os.environ[k]
|
k: os.environ[k]
|
||||||
for k in ("SEARXNG_URL", "PERPLEXITY_API_KEY", "EXA_API_KEY",
|
for k in ("SEARXNG_URL", "SEARXNG_ENGINES",
|
||||||
|
"PERPLEXITY_API_KEY", "EXA_API_KEY",
|
||||||
"TAVILY_API_KEY", "BRAVE_API_KEY")
|
"TAVILY_API_KEY", "BRAVE_API_KEY")
|
||||||
if os.environ.get(k)
|
if os.environ.get(k)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -104,6 +104,28 @@
|
|||||||
但 live-info 要真正查到天气/时间,还得先修 web_search
|
但 live-info 要真正查到天气/时间,还得先修 web_search
|
||||||
在 Docker 里的联网问题——那是另一批工作,不在本批
|
在 Docker 里的联网问题——那是另一批工作,不在本批
|
||||||
宣称范围内。
|
宣称范围内。
|
||||||
|
|
||||||
|
## web_search Docker 联网修法(2026-10-30,下一批)
|
||||||
|
|
||||||
|
- 定位(容器内直接跑 websearch.py,绕过模型):
|
||||||
|
SearXNG 默认 general 类别在容器里要 20.0 秒才回
|
||||||
|
(wikipedia / wikidata / brave / duckduckgo / google
|
||||||
|
引擎全部超时,只有 bing + 360search 回),正好撞
|
||||||
|
MCP 默认 20 秒超时;随后 DuckDuckGo 经 egress 代理
|
||||||
|
也超时(不是 DNS,是对方本身超时),于是整条链
|
||||||
|
失败。代理本身没问题:Tinyproxy 白名单早已放行
|
||||||
|
searx.diante.me 与 duckduckgo.com,容器经网关 8888
|
||||||
|
能到 SearXNG,只是太慢。
|
||||||
|
- 修法(本分支,未部署):`_provider_searxng` 默认
|
||||||
|
传 `engines=bing,360search`(`SEARXNG_ENGINES` 可
|
||||||
|
覆盖,空值恢复旧行为);worker 的搜索 env 透传
|
||||||
|
补上 `SEARXNG_ENGINES`。实测:同一容器、同一代理,
|
||||||
|
钉住引擎后 SearXNG 0.26 秒回 3 条(首条北京天气),
|
||||||
|
auto 链 0.15 秒走 SearXNG 成功;旧代码 20.0 秒。
|
||||||
|
- 测试:新增 `test_websearch_searxng.py` 3 项(默认
|
||||||
|
钉住、env 覆盖、空值恢复),连同 MCP / worker 测试
|
||||||
|
共 21 项全过。生产仍未部署此修法,live-info 端到
|
||||||
|
端再测另等用户定。
|
||||||
- 清理已核:第二次测试的用户/项目/任务、租户目录、
|
- 清理已核:第二次测试的用户/项目/任务、租户目录、
|
||||||
临时 worker 代码与临时 key 全部删除,用户剩余 0,
|
临时 worker 代码与临时 key 全部删除,用户剩余 0,
|
||||||
生产 health 仍 3696dad、worker_ok;第一次测试的
|
生产 health 仍 3696dad、worker_ok;第一次测试的
|
||||||
|
|||||||
Reference in new issue
Block a user