Commit Graph

151 Commits

Author SHA1 Message Date
7ebc1f0943 Feat: add model provider DeepInfra (#9003)
### What problem does this PR solve?

Add model provider DeepInfra. This model list comes from our community. 

NOTE: most endpoints haven't been tested, but they should work as OpenAI
does.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-23 18:10:35 +08:00
0d7244e4a4 Fix: Adds newest Gemini models to fit google's standard API rate limits (#8970)
### What problem does this PR solve?

Adds configurations for gemini-2.5-flash and Gemini 2.5-pro models,
including tags, maximum token limits, and model types.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-23 10:18:04 +08:00
ed7bea060f Feat: add Kimi model series support (#8866)
### What problem does this PR solve?

Add Kimi model series support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-16 15:31:57 +08:00
a9abf9df48 Adds new Voyage embedding models (#8845)
### What problem does this PR solve?
This PR enhances the application's capabilities by adding support for
four new Voyage embedding models (voyage-3-large, voyage-3.5,
voyage-3.5-lite, and voyage-code-3) to the `llm_factories.json`
configuration file. These models expand the available options for text
embedding tasks, enabling improved processing of text data with a
maximum token limit of 32,000. This addition addresses the need for more
diverse and specialized embedding models to support various use cases
without altering existing functionality.

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2025-07-16 11:41:06 +08:00
1895667573 Feat: add xAI provider (#8781)
### What problem does this PR solve?

Add xAI provider (experimental feature, requires user feedback).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-11 10:35:23 +08:00
fffb7c0bba Fix: anthropic llm issue. (#8633)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-02 18:37:34 +08:00
695bfe34a2 fix opendal config 'oss_table' and 'max_allowed_packet' (#8611)
### What problem does this PR solve?

Fix the config option name of the opendal table name and setting of
'max_allowed_packet'.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: He Wang <wanghechn@qq.com>
2025-07-02 16:45:01 +08:00
aafeffa292 Feat: add gitee as LLM provider. (#8545)
### What problem does this PR solve?


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-06-30 09:22:31 +08:00
6aa0b0819d Fix: unify opendal config key from ‎schema to ‎scheme (#8232)
### What problem does this PR solve?

This PR resolves the inconsistency in the opendal configuration where
both ‎`schema` and ‎`scheme` were used as keys. The code and
configuration file now consistently use ‎`scheme`, which helps prevent
configuration errors and runtime issues. This change improves code
clarity and maintainability.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Additional context
- Updated both ‎`conf/service_conf.yaml` and
‎`rag/utils/opendal_conn.py` to use ‎`scheme` instead of ‎`schema`
- No breaking changes to other configuration fields
2025-06-13 14:56:51 +08:00
44287fb05f Oss support opendal(including mysql) (#8204)
### What problem does this PR solve?

#8074
Oss support opendal(including mysql)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-06-12 11:37:42 +08:00
5e30426916 Feat: add Qwen3-Embedding text-embedding-v4 (#8184)
### What problem does this PR solve?

Add Qwen3-Embedding text-embedding-v4.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-06-11 15:32:05 +08:00
37075eab98 Feat: add voyage-multimodal-3 (#7987)
### What problem does this PR solve?

Add voyage-multimodal-3.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-06-03 11:56:59 +08:00
6c9b8ec860 Refa: update gemini2.5 (#7822)
### What problem does this PR solve?

Update gemini2.5

### Type of change

- [x] Refactoring
2025-05-23 20:29:10 +08:00
50ff16e7a4 Feat: add claude4 models (#7809)
### What problem does this PR solve?

Add claude4 models.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-23 18:25:13 +08:00
e166f132b3 Feat: change default models (#7777)
### What problem does this PR solve?

change default models to buildin models
https://github.com/infiniflow/ragflow/issues/7774

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-23 18:21:25 +08:00
205974c359 Docs: Improve oauth configuration documentation and examples (#7675)
### What problem does this PR solve?

Improve oauth configuration documentation and examples.

- Related pull requests: 
  - #7379
  - #7553
  - #7587
- Related issues:
  -  #3495
### Type of change

- [x] Documentation Update
2025-05-16 14:17:39 +08:00
a8542508b7 Refa: Deprecate /github_callback in favor of /oauth/callback/<channel> for GitHub OAuth integration (#7587)
### What problem does this PR solve?

Deprecate `/github_callback` route in favor of
`/oauth/callback/<channel>` for GitHub OAuth integration:

- Added GitHub OAuth support in the authentication module
- Introduced `GithubOAuthClient` with methods to fetch and normalize
user info
  - Updated `CLIENT_TYPES` to include GitHub OAuth client
- Deprecated `/github_callback` route and suggested using the generic
`/oauth/callback/<channel>` route

---
- Related pull requests: 
  - #7379
  - #7553 

### Usage

- [Create a GitHub OAuth
App](https://github.com/settings/applications/new) to obtain the
`client_id` and `client_secret`, configure the authorization callback
url: `https://your-app.com/v1/user/oauth/callback/github`
- Edit `service_conf.yaml.template`:
  ```yaml
  # ...
  oauth:
    github:
      type: "github"
      icon: "github"
      display_name: "Github"
      client_id: "your_client_id"
      client_secret: "your_client_secret"
      redirect_uri: "https://your-app.com/v1/user/oauth/callback/github"
  # ...
  ```

### Type of change

- [x] Documentation Update
- [x] Refactoring (non-breaking change)
2025-05-15 14:39:37 +08:00
36e32dde1a Feat: update llm factories for SILICONFLOW (#7620)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Other (please describe): llm factories update
2025-05-14 19:46:27 +08:00
321a280031 Feat: add image preview to retrieval test. (#7610)
### What problem does this PR solve?

#7608

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-13 14:30:36 +08:00
e39ceb2bd1 Feat: add support for OpenAi gpt 4.1 series (#7540)
### What problem does this PR solve?

Adds support for the GPT-4.1 series from OpenAI.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-12 18:24:53 +08:00
6bd7d572ec Perf: Increase database connection pool size (#7559)
### What problem does this PR solve?

1. The MySQL instance is configured with max_connections=1000,
but our connection pool was limited to max_connections: 100.
This mismatch caused connection pool exhaustion during performance
testing.

2.  Increase stale_timeout to resolve #6548

### Type of change

- [x] Performance Improvement
2025-05-09 17:52:03 +08:00
e9053b6ed4 fix bug #7309 deepseek-ai/deepseek-vl2 model can not be select as a VL model to parse pdf image (#7312)
### What problem does this PR solve?
fix deepseek-ai/deepseek-vl2 model can not be select as a VL model to
parse pdf image . And add other vl models config from siliconflow
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: unknown <taoshi.ln@chinatelecom.cn>
2025-05-08 11:24:39 +08:00
e349635a3d Feat: Add /login/channels route and improve auth logic for frontend third-party login integration (#7521)
### What problem does this PR solve?

Add `/login/channels` route and improve auth logic to support frontend
integration with third-party login providers:

- Add `/login/channels` route to provide authentication channel list
with `display_name` and `icon`
- Optimize user info parsing logic by prioritizing `avatar_url` and
falling back to `picture`
- Simplify OIDC token validation by removing unnecessary `kid` checks
- Ensure `client_id` is safely cast to string during `audience`
validation
- Fix typo

---
- Related pull request: #7379 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2025-05-08 10:23:19 +08:00
093d280528 Feat: add Qwen3 and OpenAI o series (#7415)
### What problem does this PR solve?

Qwen3 and more LLMs.

Close #7296

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-04-29 18:26:29 +08:00
3a43043c8a Feat: Add support for OAuth2 and OpenID Connect (OIDC) authentication (#7379)
### What problem does this PR solve?

Add support for OAuth2 and OpenID Connect (OIDC) authentication,
allowing OAuth/OIDC authentication using the specified routes:
- `/login/<channel>`: Initiates the OAuth flow for the specified channel
- `/oauth/callback/<channel>`: Handles the OAuth callback after
successful authentication

The callback URL should be configured in your OAuth provider as:
```
https://your-app.com/oauth/callback/<channel>
```

For detailed instructions on configuring **service_conf.yaml.template**,
see: `./api/apps/auth/README.md#usage`.

- Related issues
#3495  

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2025-04-28 16:15:52 +08:00
23dcbc94ef feat: replace models of novita (#7360)
### What problem does this PR solve?

Replace models of novita

### Type of change

- [x] Other (please describe): Replace models of novita
2025-04-28 13:35:09 +08:00
67b087019c Update Groq AI Model Config (#7335)
With current config will get error "Fail to access model(gemma-7b-it)
using this api key"
Since the model has been removed, according to Groq official document:
https://console.groq.com/docs/models

### Type of change

- [ x] Bug Fix (non-breaking change which fixes an issue)
2025-04-27 17:05:25 +08:00
c8c3b756b0 Feat: Adds OpenSearch2.19.1 as the vector_database support (#7140)
### What problem does this PR solve?

This PR adds the support for latest OpenSearch2.19.1 as the store engine
& search engine option for RAGFlow.

### Main Benefit

1. OpenSearch2.19.1 is licensed under the [Apache v2.0 License] which is
much better than Elasticsearch
2. For search, OpenSearch2.19.1 supports full-text
search、vector_search、hybrid_search those are similar with Elasticsearch
on schema
3. For store, OpenSearch2.19.1 stores text、vector those are quite
simliar with Elasticsearch on schema

### Changes

- Support opensearch_python_connetor. I make a lot of adaptions since
the schema and api/method between ES and Opensearch differs in many
ways(especially the knn_search has a significant gap) :
rag/utils/opensearch_coon.py
- Support static config adaptions by changing:
conf/service_conf.yaml、api/settings.py、rag/settings.py
- Supprt some store&search schema changes between OpenSearch and ES:
conf/os_mapping.json
- Support OpenSearch python sdk : pyproject.toml
- Support docker config for OpenSearch2.19.1 :
docker/.env、docker/docker-compose-base.yml、docker/service_conf.yaml.template

### How to use
- I didn't change the priority that ES as the default doc/search engine.
Only if in docker/.env , we set DOC_ENGINE=${DOC_ENGINE:-opensearch}, it
will work.


### Others
Our team tested a lot of docs in our environment by using OpenSearch as
the vector database ,it works very well.
All the conifg for OpenSearch is necessary.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yongteng Lei <yongtengrey@outlook.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2025-04-24 16:03:31 +08:00
018ff4dd0a Refa: update llms (#7007)
### What problem does this PR solve?

Update LLM models

### Type of change

- [x] Refactoring
2025-04-15 09:19:07 +08:00
5b5558300a Feat: add gemini-2.5-pro-exp-03-25 (#6774)
### What problem does this PR solve?

#6733

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-04-03 10:48:58 +08:00
fc21dd0a4a Feat: add qwq-plus-latest (#6702)
### What problem does this PR solve?

#6697

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-04-01 11:06:03 +08:00
7d9dd1e5d3 Refa: remove default build-in rerank model. (#6682)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2025-03-31 15:33:19 +08:00
65a8cd1772 Fix knowledge_graph_kwd on infinity. Close #6476 and #6624 (#6651)
### What problem does this PR solve?

Fix knowledge_graph_kwd on infinity. Close #6476 and #6624

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-28 22:05:40 +08:00
d2043ff9f2 Fix: LmStudioChat issue. (#6591)
### What problem does this PR solve?

#6577

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-27 14:59:15 +08:00
735d9dd949 Feat: add "tools" to llm_factories.json (#6552)
### What problem does this PR solve?



### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Chenzy <chenzy901@gmail.com>
2025-03-26 17:31:18 +08:00
6bf26e2a81 Optimize graphrag again (#6513)
### What problem does this PR solve?

Removed set_entity and set_relation to avoid accessing doc engine during
graph computation.
Introduced GraphChange to avoid writing unchanged chunks.

### Type of change

- [x] Performance Improvement
2025-03-26 15:34:42 +08:00
85eb3775d6 Refa: update Anthropic models. (#6445)
### What problem does this PR solve?

#6421

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 12:34:57 +08:00
f16418ccf7 Feat: Add deepseek to llm_factories (#6051)
### What problem does this PR solve?

AWS Bedrock has made deepseek-r1 available on its serverless inference.

This adds the R1 serverless model for use via the bedrock model
abilities.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-14 10:35:44 +08:00
6e13922bdc Feat: Add qwq model support to Tongyi-Qianwen factory (#5981)
### What problem does this PR solve?

add qwq model support to Tongyi-Qianwen factory
https://github.com/infiniflow/ragflow/issues/5869

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


![image](https://github.com/user-attachments/assets/49f5c6a0-ecaf-41dd-a23a-2009f854d62c)


![image](https://github.com/user-attachments/assets/93ffa303-920e-4942-8188-bcd6b7209204)


![1741774779438](https://github.com/user-attachments/assets/25f2fd1d-8640-4df0-9a08-78ee9daaa8fe)


![image](https://github.com/user-attachments/assets/4763cf6c-1f76-43c4-80ee-74dfd666a184)

Co-authored-by: zhaozhicheng <zhicheng.zhao@fastonetech.com>
2025-03-12 18:54:15 +08:00
fa817a8ab3 Refa: SiliconFlow model list refresh. (#5825)
### What problem does this PR solve?

#5806

### Type of change


- [x] Refactoring
2025-03-10 12:51:12 +08:00
e05658685c Refa: update mistral model list. (#5818)
### What problem does this PR solve?

#5782

### Type of change

- [x] Refactoring
2025-03-10 11:22:06 +08:00
b8da2eeb69 Feat: support huggingface re-rank model. (#5684)
### What problem does this PR solve?

#5658

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-06 10:44:04 +08:00
76e8285904 use to_df replace to_pl when get infinity Result (#5604)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Performance Improvement

---------

Co-authored-by: wangwei <dwxiayi@163.com>
2025-03-05 09:35:40 +08:00
202acbd628 Perf: update novita.ai LLM library (#5574)
### What problem does this PR solve?

LLM library update

### Type of change

- [x] Other : config update
2025-03-04 11:35:25 +08:00
79bc9d97c9 Refa: better service conf (#5471)
### What problem does this PR solve?

This patch fix most of the issues like #4853 #5038 and so on

the root reason is that we need to add the hostname to the `/etc/hosts`
which is not wrote in main README
and the code side read `conf/service_conf.yaml` as settings 
and its hard for developers to debug, this patch fix it, or maybe can
discuss better solution here
 
### Type of change

- [x] Refactoring

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-02-28 14:28:00 +08:00
651422127c Feat: Accessing Alibaba Cloud OSS with Amazon S3 SDK (#5438)
Accessing Alibaba Cloud OSS with Amazon S3 SDK
2025-02-27 17:02:42 +08:00
cdcaae17c6 Feat: add VLLM (#5380)
### What problem does this PR solve?

Read to add VLMM.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-26 16:04:53 +08:00
7ce675030b Support downloading models from ModelScope Community. (#5073)
This PR supports downloading models from ModelScope. The main
modifications are as follows:
-New Feature (non-breaking change which adds functionality)
-Documentation Update

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-02-24 10:12:20 +08:00
4694604836 Specify img2text model by tag (#5063)
### What problem does this PR solve?

The current design is not well-suited for multimodal models, as each
model can only be configured for a single purpose—either chat or
Img2txt. To work around this limitation, we use model aliases such as
gpt-4o-mini and gpt-4o-mini-2024-07-18.

To fix this, this PR allows specifying the Img2txt model by tag instead
of model_type.

### Type of change
- [x] Refactoring
2025-02-18 11:14:48 +08:00
d2929e432e Feat: add LLM provider PPIO (#5013)
### What problem does this PR solve?

Add a LLM provider: PPIO

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2025-02-17 12:03:26 +08:00