Perf: light worker poll query + tasks(status) index
Worker polls the DB every interval (production 30s) and every poll ran SELECT * FROM tasks: full rows including result_text and error_text (up to 4000 chars each) for every task ever, growing with history. load_tasks now selects only the columns pick_next actually needs (id/project/prompt/status/depends/ priority/timestamps/executor) and only queued/completed/merged rows; running/failed rows are neither ready nor completed, so excluding them cannot change pick_next. get_task still does SELECT * for the single claimed task. Plus CREATE INDEX IF NOT EXISTS idx_tasks_status on tasks(status) in both SQLite init_db and the Postgres schema, so the poll filter stops being a full table scan on old databases too. Test: load_tasks returns only queued+completed out of queued/completed/running/failed.
This commit is contained in:
1 parent
a1e492657a
commit
26d9edac33
4 files changed
+41
-1
No files matched your search
@@ -384,6 +384,23 @@ class FallbackRetryTestCase(unittest.TestCase):
|
||||
self.assertEqual(len(self.calls), 1)
|
||||
self.assertEqual(result["events"], [])
|
||||
|
||||
def test_load_tasks_only_picks_relevant_statuses(self):
|
||||
# load_tasks 只取 queued/completed/merged:running/failed
|
||||
# 的任务对 pick_next 既不是就绪也不是已完成,排除它们
|
||||
# 不影响结果,但省掉每轮 poll 全表 + 重字段的开销。
|
||||
for tid, status in [("Q", "queued"), ("C", "completed"),
|
||||
("R", "running"), ("F", "failed")]:
|
||||
_insert_task(self.db_path, tid)
|
||||
import sqlite3
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
conn.execute(
|
||||
"UPDATE tasks SET status = ? WHERE id = ?", (status, tid))
|
||||
conn.commit()
|
||||
conn.close()
|
||||
worker = self._worker(runner=lambda cmd: None)
|
||||
got = {t["id"] for t in worker.load_tasks()}
|
||||
self.assertEqual(got, {"Q", "C"})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user