pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

Feat: parsing supports jsonl or ldjson format (#9087)

Browse Source 
### What problem does this PR solve?

Supports jsonl or ldjson format. Feature request from
[discussion](https://github.com/orgs/infiniflow/discussions/8774).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
Yongteng Lei
2025-07-30 09:48:20 +08:00
committed by GitHub
parent ba563f8095
commit 39ef2ffba9
3 changed files with 64 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
rag/app/naive.py
View File

@ -499,7 +499,7 @@ def chunk(filename, binary=None, from_page=0, to_page=100000,
sections = [(_, "") for _ in sections if _]
callback(0.8, "Finish parsing.")
elif re.search(r"\.json$", filename, re.IGNORECASE):
elif re.search(r"\.(json|jsonl|ldjson)$", filename, re.IGNORECASE):
callback(0.1, "Start to parse.")
chunk_token_num = int(parser_config.get("chunk_token_num", 128))
sections = JsonParser(chunk_token_num)(binary)