Commit Graph

175 Commits

Author SHA1 Message Date
92cfbcb382 Fix: when parse markdown support extract image at local (#8906)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/8902
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-18 17:06:58 +08:00
e9b14142a5 Fix: fixed invalid save() arguments for slide thumbnails (#8851)
### What problem does this PR solve?

Fixed invalid save() arguments for slide thumbnails.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-15 17:19:45 +08:00
51a8604dcb Fix: fixed context loss caused by separating markdown tables from original text (#8844)
### What problem does this PR solve?

Fix context loss caused by separating markdown tables from original
text. #6871, #8804.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-15 13:03:01 +08:00
ce140f1393 Fix:Better Support Table Value Type (#8822)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/8782

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-14 17:51:26 +08:00
2b7adbd2d1 Fix: Improve Memory Usage For Presentation (#8792)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/8791


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-11 11:35:25 +08:00
f586dd0a96 Fix: docx parse error. (#8600)
### What problem does this PR solve?

docx parse error.

![image](https://github.com/user-attachments/assets/efbe6d1b-10c8-415e-b693-a86f73e1ffa6)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### What problem does this PR solve?

Some docx parse with naive cause error. `block.style.name` in Function
`__get_nearest_title` will be None in some case.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: wenxuan.zhang <wenxuan.zhang@chinacreator.com>
2025-07-01 17:38:11 +08:00
6b1221d2f6 Fix parser_config access for layout_recognize in presentation.py (#8492)
### What problem does this PR solve?
This PR addresses an issue in the presentation parser where the
`layout_recognize` configuration was incorrectly retrieved from
`kwargs.get("layout_recognize", "DeepDOC")`. Instead, it should be
sourced from the `parser_config` parameter, specifically
`parser_config.get("layout_recognize", "DeepDOC")`.

This mismatch could cause the parser to default to the "DeepDOC" layout
recognizer, ignoring any alternative recognition method specified in the
parser configuration. As a result, PDF document parsing might use an
incorrect recognition engine.

The fix ensures the presentation parser consistently uses the
`layout_recognize` setting from `parser_config`, aligning with the
configuration access patterns used elsewhere in the codebase.

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-26 11:54:43 +08:00
5256980ffb Fix: Solve the OOM issue when passing large PDF files while using QA chunking method. (#8464)
### What problem does this PR solve?

Using the QA chunking method with a large PDF (e.g., 300+ pages) may
lead to OOM in the ragflow-worker module.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-25 10:25:45 +08:00
d6a941ebf5 Fix the bug of long type value overflow (#8313)
### What problem does this PR solve?

This PR will fix the #8271 by extending int type to float type when
there is any value out of long type range in a column.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-24 18:18:30 +08:00
4a2ff633e0 Fix typo in code (#8327)
### What problem does this PR solve?

Fix typo in code

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-06-18 09:41:09 +08:00
baf32ee461 Display only the duplicate column names and corresponding original source. (#8138)
### What problem does this PR solve?
This PR aims to slove #8120 which request a better error display of
duplicate column names.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-10 10:16:38 +08:00
24625e0695 Fix: presentation of PDF using vlm. (#8133)
### What problem does this PR solve?

#8109

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-09 15:01:52 +08:00
bd4678bca6 Fix: Unnecessary truncation in markdown parser (#7972)
### What problem does this PR solve?

Fix unnecessary truncation in markdown parser. So that markdown can work
perfectly like
[this](https://github.com/infiniflow/ragflow/issues/7824#issuecomment-2921312576)
in #7824, supporting multiple special delimiters.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-30 15:04:21 +08:00
bfe97d896d Fix: docx get image exception. (#7636)
### What problem does this PR solve?

Close #7631

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-14 12:24:48 +08:00
321a280031 Feat: add image preview to retrieval test. (#7610)
### What problem does this PR solve?

#7608

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-13 14:30:36 +08:00
baa108f5cc Fix: markdown table conversion error (#7570)
### What problem does this PR solve?

Since `import markdown.markdown` has been changed to `import markdown`
in `rag/app/naive.py`, previous code for converting markdown tables
would call a markdown module instead of a callable function. This cause
error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-12 17:16:55 +08:00
5352bdf4da Error storing tag in Redis (#7541)
### What problem does this PR solve?

The parameter positions were incorrect and have been corrected to use
keyword argument passing

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-09 10:17:09 +08:00
1a5608d0f8 Fix: Add title_tks for Pictures (#7365)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/7362

append title_tks
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-28 13:35:34 +08:00
1662c7eda3 Feat: Markdown add image (#7124)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/6984

1. Markdown parser supports get pictures
2. For Native, when handling Markdown, it will handle images
3. improve merge and 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-04-25 18:35:28 +08:00
1b4016317e fix bug chunking:expected string or bytes-like object (#7116)
… bytes-like object

### What problem does this PR solve?
fix bug #6990 internal server error ehile chunking:expected string or
bytes-like object
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: unknown <taoshi.ln@chinatelecom.cn>
2025-04-18 14:42:36 +08:00
ed5f81b02e Fix: abnormal cell mergeing. (#6991)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-14 11:00:11 +08:00
5aae73c230 Make error messages during PPT processing clearer. (#6980)
### What problem does this PR solve?

Sometimes a slide may trigger a Proxy error (ArgumentException:
Parameter is not valid) due to issues in the original file, and this
error message can be confusing for users.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [x] Other (please describe):
2025-04-14 10:10:20 +08:00
14a3efd756 Fix: docx image exceptions. (#6839)
### What problem does this PR solve?

Close #6784

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-07 12:33:34 +08:00
ee5aa51d43 Fix: point in tag issue. (#6436)
### What problem does this PR solve?

#6414

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 10:45:29 +08:00
0e0ebaac5f Feat: Adds hierarchical title path tracking for tables in DOCX documents to improve context association (#6374)
### What problem does this PR solve?

Adds hierarchical title path tracking for tables in DOCX documents to
improve context association. Previously, extracted tables lacked
positional context within document structure.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-21 18:42:36 +08:00
95497b4aab Fix: adapt to old configurations. (#6321)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-20 14:50:59 +08:00
9611185eb4 Feat: add VLM-boosted DocX parser (#6307)
### What problem does this PR solve?

Add VLM-boosted DocX parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-20 11:24:44 +08:00
e4380843c4 Feat: add fallback for PDF figure parser (#6305)
### What problem does this PR solve?

Add fallback for PDF figure parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-20 10:48:38 +08:00
1d6760dd84 Feat: add VLM-boosted PDF parser (#6278)
### What problem does this PR solve?

Add VLM-boosted PDF parser if VLM is set.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-20 09:39:32 +08:00
5cf610af40 Feat: add vision LLM PDF parser (#6173)
### What problem does this PR solve?

Add vision LLM PDF parser

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-18 14:52:20 +08:00
1333d3c02a Fix: float transfer exception. (#6197)
### What problem does this PR solve?

#6177

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-18 11:13:44 +08:00
3a99c2b5f4 Refa: PARALLEL_DEVICES is a static parameter. (#6168)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2025-03-17 16:49:54 +08:00
bfa8d342b3 Fix: retrieval debug mode issue. (#6150)
### What problem does this PR solve?

#6139

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-17 13:07:13 +08:00
3e19044dee Feat: add OCR's muti-gpus and parallel processing support (#5972)
### What problem does this PR solve?

Add OCR's muti-gpus and parallel processing support

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

@yuzhichang I've tried to resolve the comments in #5697. OCR jobs can
now be done on both CPU and GPU. ( By the way, I've encountered a
“Generate embedding error” issue #5954 that might be due to my outdated
GPUs? idk. ) Please review it and give me suggestions.

GPU:

![gpu_ocr](https://github.com/user-attachments/assets/0ee2ecfb-a665-4e50-8bc7-15941b9cd80e)

![smi](https://github.com/user-attachments/assets/a2312f8c-cf24-443d-bf89-bec50503546d)

CPU:

![cpu_ocr](https://github.com/user-attachments/assets/1ba6bb0b-94df-41ea-be79-790096da4bf1)
2025-03-17 11:58:40 +08:00
4ff609b6a8 Fix: optimize OCR garbage identification to reduce unnecessary filtering (#6027)
### What problem does this PR solve?

Optimize OCR garbage identification to reduce unnecessary filtering.
#5713

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-13 18:48:32 +08:00
7cd37c37cd Feat: add CSV file parsing support (#5989)
### What problem does this PR solve?

Add CSV file parsing support #4552, #5849, #5870

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-12 19:20:50 +08:00
b0c21b00d9 Refactor: Optimize error handling and support parsing of XLS(EXCEL97—2003) files. (#5633)
Optimize error handling and support parsing of XLS(EXCEL97—2003) files.
2025-03-05 11:55:27 +08:00
b418ce5643 Fix table parser issue. (#5482)
### What problem does this PR solve?

#1475
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-28 16:09:12 +08:00
4f40f685d9 Code refactor (#5371)
### What problem does this PR solve?

#5173

### Type of change

- [x] Refactoring
2025-02-26 15:40:52 +08:00
c28bc41a96 Fix docx table issue. (#5117)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-19 12:40:06 +08:00
c24137bd11 Fix too long integer for Table. (#4651)
### What problem does this PR solve?

#4594

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-26 12:54:58 +08:00
9d717f0b6e Fix csv reader exception. (#4628)
### What problem does this PR solve?

#4552
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-24 14:47:19 +08:00
13f04b7cca Fix pdf applying Q&A issue. (#4599)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-23 12:30:46 +08:00
dd0ebbea35 Light GraphRAG (#4585)
### What problem does this PR solve?

#4543

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-22 19:43:14 +08:00
3894de895b Update comments (#4569)
### What problem does this PR solve?

Add license statement.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-01-21 20:52:28 +08:00
f556f0239c Fix dify retrieval issue. (#4473)
### What problem does this PR solve?

#4464
#4469 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-14 13:16:05 +08:00
e098fcf6ad Fix csv for TAG. (#4454)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-13 12:03:18 +08:00
c5da3cdd97 Tagging (#4426)
### What problem does this PR solve?

#4367

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-09 17:07:21 +08:00
50f209204e Synchronize with enterprise version (#4325)
### Type of change

- [x] Refactoring
2025-01-02 13:44:44 +08:00
8fb18f37f6 Code refactor. (#4291)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-12-30 18:38:51 +08:00