pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

reduce rerank batch size (#2801)

Browse Source 
### What problem does this PR solve?

### Type of change


- [x] Performance Improvement
This commit is contained in:
Kevin Hu
2024-10-11 11:29:19 +08:00
committed by GitHub
parent bae30e5cc4
commit 5e7c1fb23a
2 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
rag/llm/rerank_model.py
View File

@ -142,7 +142,7 @@ class YoudaoRerank(DefaultRerank):
token_count = 0
for _, t in pairs:
token_count += num_tokens_from_string(t)
batch_size = 32
batch_size = 8
res = []
for i in range(0, len(pairs), batch_size):
scores = self._model.compute_score(pairs[i:i + batch_size], max_length=self._model.max_length)