mirror of
https://github.com/infiniflow/ragflow.git
synced 2025-12-26 17:16:52 +08:00
Feature/mineru improvements (#11938)
我已在下面的评论中用中文重复说明。 ### What problem does this PR solve? ## Summary This PR enhances the MinerU document parser with additional configuration options, giving users more control over PDF parsing behavior and improving support for multilingual documents. ## Changes ### Backend (`deepdoc/parser/mineru_parser.py`) - Added configurable parsing options: - **Parse Method**: `auto`, `txt`, or `ocr` — allows users to choose the extraction strategy - **Formula Recognition**: Toggle for enabling/disabling formula extraction (useful to disable for Cyrillic documents where it may cause issues) - **Table Recognition**: Toggle for enabling/disabling table extraction - Added language code mapping (`LANGUAGE_TO_MINERU_MAP`) to translate RAGFlow language settings to MinerU-compatible language codes for better OCR accuracy - Improved parser configuration handling to pass these options through the processing pipeline ### Frontend (`web/`) - Created new `MinerUOptionsFormField` component that conditionally renders when MinerU is selected as the layout recognition engine - Added UI controls for: - Parse method selection (dropdown) - Formula recognition toggle (switch) - Table recognition toggle (switch) - Added i18n translations for English and Chinese - Integrated the options into both the dataset creation dialog and dataset settings page ### Integration - Updated `rag/app/naive.py` to forward MinerU options to the parser - Updated task service to handle the new configuration parameters ## Why MinerU is a powerful document parser, but the default settings don't work well for all document types. This PR allows users to: 1. Choose the best parsing method for their documents 2. Disable formula recognition for Cyrillic/non-Latin scripts where it causes issues 3. Control table extraction based on document needs 4. Benefit from automatic language detection for better OCR results ## Testing - [x] Tested MinerU parsing with different parse methods - [x] Verified UI renders correctly when MinerU is selected/deselected - [x] Confirmed settings persist correctly in dataset configuration ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) - [x] New Feature (non-breaking change which adds functionality) - [ ] Documentation Update - [x] Refactoring - [ ] Performance Improvement - [ ] Other (please describe): --------- Co-authored-by: user210 <user210@rt> Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
This commit is contained in:
@ -330,6 +330,15 @@ export default {
|
||||
reRankModelWaring: 'Re-rank model is very time consuming.',
|
||||
},
|
||||
knowledgeConfiguration: {
|
||||
mineruOptions: 'MinerU Options',
|
||||
mineruParseMethod: 'Parse Method',
|
||||
mineruParseMethodTip:
|
||||
'Method for parsing PDF: auto (automatic detection), txt (text extraction), ocr (optical character recognition)',
|
||||
mineruFormulaEnable: 'Formula Recognition',
|
||||
mineruFormulaEnableTip:
|
||||
'Enable formula recognition. Note: This may not work correctly for Cyrillic documents.',
|
||||
mineruTableEnable: 'Table Recognition',
|
||||
mineruTableEnableTip: 'Enable table recognition and extraction.',
|
||||
overlappedPercent: 'Overlapped percent(%)',
|
||||
generationScopeTip:
|
||||
'Determines whether RAPTOR is generated for the entire dataset or for a single file.',
|
||||
|
||||
@ -265,6 +265,15 @@ export default {
|
||||
theDocumentBeingParsedCannotBeDeleted: '正在解析的文档不能被删除',
|
||||
},
|
||||
knowledgeConfiguration: {
|
||||
mineruOptions: 'MinerU 选项',
|
||||
mineruParseMethod: '解析方法',
|
||||
mineruParseMethodTip:
|
||||
'PDF 解析方法:auto(自动检测)、txt(文本提取)、ocr(光学字符识别)',
|
||||
mineruFormulaEnable: '公式识别',
|
||||
mineruFormulaEnableTip:
|
||||
'启用公式识别。注意:对于西里尔文档可能无法正常工作。',
|
||||
mineruTableEnable: '表格识别',
|
||||
mineruTableEnableTip: '启用表格识别和提取。',
|
||||
generationScopeTip: '选择 RAPTOR 的生成范围:整个知识库或单个文件。',
|
||||
generationScope: '生成范围',
|
||||
scopeSingleFile: '单文件',
|
||||
|
||||
Reference in New Issue
Block a user