Perf: make do_cancel quicker. (#8846)

### What problem does this PR solve?

### Type of change

- [x] Performance Improvement
This commit is contained in:
Kevin Hu
2025-07-15 14:35:00 +08:00
committed by GitHub
parent 5fa6f2f151
commit 24c41d2a61
3 changed files with 36 additions and 29 deletions

View File

@ -42,6 +42,7 @@ class RecursiveAbstractiveProcessing4TreeOrganizedRetrieval:
self._prompt = prompt
self._max_token = max_token
@timeout(60)
async def _chat(self, system, history, gen_conf):
response = get_llm_cache(self._llm_model.llm_name, system, history, gen_conf)
if response: