pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2026-02-02 16:45:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

update xgboost and dep scripts for local build on MacOS (#7857)

Browse Source 
### What problem does this PR solve?

There are two main changes:

1. Update xgboost to 1.6.0 to build the project on MacOS with Apple
chips, this change refers to the issue:
https://github.com/infiniflow/ragflow/issues/5114.

2. When `use_china_mirrors` is set in `download_deps.py`, the names of
chrome files downloaded by the script will be different from the file
names used in Dockerfile, so I added the file name in `get_urls`
function to solve this problem.

I think it's better to add testing for Docker image
`infiniflow/ragflow_deps` to the test workflow, but since the workflow
is currently running on a self-hosted runner, I'm not sure how to modify
it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
This commit is contained in:
He Wang
2025-05-27 09:28:52 +08:00
committed by GitHub
parent 48294e624c
commit aaefc3f44c
4 changed files with 23 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
download_deps.py
View File

@ -11,12 +11,13 @@
# ///
from huggingface_hub import snapshot_download
from typing import Union
import nltk
import os
import urllib.request
import argparse
def get_urls(use_china_mirrors=False):
def get_urls(use_china_mirrors=False) -> Union[str, list[str]]:
if use_china_mirrors:
return [
"http://mirrors.tuna.tsinghua.edu.cn/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2_amd64.deb",
@ -24,8 +25,8 @@ def get_urls(use_china_mirrors=False):
"https://repo.huaweicloud.com/repository/maven/org/apache/tika/tika-server-standard/3.0.0/tika-server-standard-3.0.0.jar",
"https://repo.huaweicloud.com/repository/maven/org/apache/tika/tika-server-standard/3.0.0/tika-server-standard-3.0.0.jar.md5",
"https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken",
"https://storage.googleapis.com/chrome-for-testing-public/121.0.6167.85/linux64/chrome-linux64.zip",
"https://storage.googleapis.com/chrome-for-testing-public/121.0.6167.85/linux64/chromedriver-linux64.zip",
["https://storage.googleapis.com/chrome-for-testing-public/121.0.6167.85/linux64/chrome-linux64.zip", "chrome-linux64-121-0-6167-85"],
["https://storage.googleapis.com/chrome-for-testing-public/121.0.6167.85/linux64/chromedriver-linux64.zip", "chromedriver-linux64-121-0-6167-85"],
]
else:
return [
@ -60,10 +61,11 @@ if __name__ == "__main__":
urls = get_urls(args.china_mirrors)
for url in urls:
filename = url.split("/")[-1]
print(f"Downloading {url}...")
download_url = url[0] if isinstance(url, list) else url
filename = url[1] if isinstance(url, list) else url.split("/")[-1]
print(f"Downloading {filename} from {download_url}...")
if not os.path.exists(filename):
urllib.request.urlretrieve(url, filename)
urllib.request.urlretrieve(download_url, filename)
local_dir = os.path.abspath('nltk_data')
for data in ['wordnet', 'punkt', 'punkt_tab']: