pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

Updates on parsing progress, including more detailed time cost inform… (#3402)

Browse Source 
### What problem does this PR solve?

#3401 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
Kevin Hu
2024-11-14 16:28:10 +08:00
committed by GitHub
parent 201bbef7c0
commit ab4384e011
2 changed files with 22 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
api/validation.py
View File

@ -32,7 +32,18 @@ def python_version_validation():
python_version_validation()
# Download nltk data
import nltk
nltk.download('wordnet', halt_on_error=False, quiet=True)
nltk.download('punkt_tab', halt_on_error=False, quiet=True)
def download_nltk_data():
import nltk
nltk.download('wordnet', halt_on_error=False, quiet=True)
nltk.download('punkt_tab', halt_on_error=False, quiet=True)
try:
from multiprocessing import Pool
pool = Pool(processes=1)
thr = pool.apply_async(download_nltk_data)
binary = thr.get(timeout=60)
except Exception as e:
print('\x1b[6;37;41m WARNING \x1b[0m' + "Downloading NLTK data failure.", flush=True)