pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

Feat: support vision llm for gpustack (#6636)

Browse Source 
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/6138

This PR is going to support vision llm for gpustack, modify url path
from `/v1-openai` to `/v1`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
Alex Chen
2025-03-31 15:33:52 +08:00
committed by GitHub
parent 7d9dd1e5d3
commit 46b5e32cd7
6 changed files with 22 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
rag/llm/cv_model.py
View File

@ -1028,4 +1028,14 @@ class AnthropicCV(Base):
except Exception as e:
yield ans + "\n**ERROR**: " + str(e)
yield total_tokens
yield total_tokens
class GPUStackCV(GptV4):
def __init__(self, key, model_name, lang="Chinese", base_url=""):
if not base_url:
raise ValueError("Local llm url cannot be None")
if base_url.split("/")[-1] != "v1":
base_url = os.path.join(base_url, "v1")
self.client = OpenAI(api_key=key, base_url=base_url)
self.model_name = model_name
self.lang = lang