pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix rerank_model bug in chat and markdown bug (#4061)

Browse Source 
### What problem does this PR solve?

Fix rerank_model bug in chat and markdown bug
#4000
#3992
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
This commit is contained in:
liuhua
2024-12-17 16:03:37 +08:00
committed by GitHub
parent bcccaccc2b
commit 1d65299791
4 changed files with 15 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
rag/app/naive.py
View File

@ -169,11 +169,13 @@ class Markdown(MarkdownParser):
sections = []
tbls = []
for sec in remainder.split("\n"):
if num_tokens_from_string(sec) > 10 * self.chunk_token_num:
if num_tokens_from_string(sec) > 3 * self.chunk_token_num:
sections.append((sec[:int(len(sec) / 2)], ""))
sections.append((sec[int(len(sec) / 2):], ""))
else:
if sections and sections[-1][0].strip().find("#") == 0:
if sec.strip().find("#") == 0:
sections.append((sec, ""))
elif sections and sections[-1][0].strip().find("#") == 0:
sec_, _ = sections.pop(-1)
sections.append((sec_ + "\n" + sec, ""))
else: