mirror of
https://github.com/infiniflow/ragflow.git
synced 2025-12-25 16:26:51 +08:00
Feature/mineru improvements (#11938)
我已在下面的评论中用中文重复说明。 ### What problem does this PR solve? ## Summary This PR enhances the MinerU document parser with additional configuration options, giving users more control over PDF parsing behavior and improving support for multilingual documents. ## Changes ### Backend (`deepdoc/parser/mineru_parser.py`) - Added configurable parsing options: - **Parse Method**: `auto`, `txt`, or `ocr` — allows users to choose the extraction strategy - **Formula Recognition**: Toggle for enabling/disabling formula extraction (useful to disable for Cyrillic documents where it may cause issues) - **Table Recognition**: Toggle for enabling/disabling table extraction - Added language code mapping (`LANGUAGE_TO_MINERU_MAP`) to translate RAGFlow language settings to MinerU-compatible language codes for better OCR accuracy - Improved parser configuration handling to pass these options through the processing pipeline ### Frontend (`web/`) - Created new `MinerUOptionsFormField` component that conditionally renders when MinerU is selected as the layout recognition engine - Added UI controls for: - Parse method selection (dropdown) - Formula recognition toggle (switch) - Table recognition toggle (switch) - Added i18n translations for English and Chinese - Integrated the options into both the dataset creation dialog and dataset settings page ### Integration - Updated `rag/app/naive.py` to forward MinerU options to the parser - Updated task service to handle the new configuration parameters ## Why MinerU is a powerful document parser, but the default settings don't work well for all document types. This PR allows users to: 1. Choose the best parsing method for their documents 2. Disable formula recognition for Cyrillic/non-Latin scripts where it causes issues 3. Control table extraction based on document needs 4. Benefit from automatic language detection for better OCR results ## Testing - [x] Tested MinerU parsing with different parse methods - [x] Verified UI renders correctly when MinerU is selected/deselected - [x] Confirmed settings persist correctly in dataset configuration ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) - [x] New Feature (non-breaking change which adds functionality) - [ ] Documentation Update - [x] Refactoring - [ ] Performance Improvement - [ ] Other (please describe): --------- Co-authored-by: user210 <user210@rt> Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
This commit is contained in:
@ -103,14 +103,22 @@ export const LanguageTranslationMap = {
|
||||
Chinese: 'zh',
|
||||
'Traditional Chinese': 'zh-TRADITIONAL',
|
||||
Russian: 'ru',
|
||||
Indonesia: 'id',
|
||||
Indonesian: 'id',
|
||||
Spanish: 'es',
|
||||
Vietnamese: 'vi',
|
||||
Japanese: 'ja',
|
||||
Korean: 'ko',
|
||||
'Portuguese BR': 'pt-br',
|
||||
German: 'de',
|
||||
French: 'fr',
|
||||
Italian: 'it',
|
||||
Tamil: 'ta',
|
||||
Telugu: 'te',
|
||||
Kannada: 'ka',
|
||||
Thai: 'th',
|
||||
Greek: 'el',
|
||||
Hindi: 'hi',
|
||||
Ukrainian: 'uk',
|
||||
};
|
||||
|
||||
export enum FileMimeType {
|
||||
|
||||
Reference in New Issue
Block a user