Compare commits

...

49 Commits

Author SHA1 Message Date
8d8a5f73b6 Fix: meta data filter with AND logic operations. (#9687)
### What problem does this PR solve?

Close #9648

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 18:29:24 +08:00
d0fa66f4d5 Docs: update API endpoint paths (#9683)
### What problem does this PR solve?

- Update API endpoint paths in docs from `/v1/` to `/api/v1/` for
consistency

### Type of change

- [x] Documentation Update
2025-08-25 17:57:24 +08:00
9dd22e141b fix: validate chunk type before processing to prevent AttributeError (#9698)
### What problem does this PR solve?

This PR fixes a critical bug in the session listing endpoint where the
application crashes with an `AttributeError` when processing chunk data
that contains non-dictionary objects.

**Error before fix:**
```json
{
  "code": 100,
  "data": null,
  "message": "AttributeError(\"'str' object has no attribute 'get'\")"
}
```

**Root cause:**
The code assumes all items in the `chunks` array are dictionary objects
and directly calls the `.get()` method on them. However, in some cases,
the chunks array contains string objects or other non-dictionary types,
causing the application to crash when attempting to call `.get()` on a
string.

**Solution:**
Added type validation to ensure each chunk is a dictionary before
processing. Non-dictionary chunks are safely skipped, preventing the
crash while maintaining functionality for valid chunk data.

This fix improves the robustness of the session listing endpoint and
ensures users can retrieve their conversation sessions without
encountering server errors due to data format inconsistencies.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 17:57:01 +08:00
b6c1ca828e Refa: replace Chat Ollama implementation with LiteLLM (#9693)
### What problem does this PR solve?

replace Chat Ollama implementation with LiteLLM.

### Type of change

- [x] Refactoring
2025-08-25 17:56:31 +08:00
d367c7e226 Fix: Optimize dataset page layout and internationalization and default values for multi selection #3221 (#9695)
### What problem does this PR solve?

Fix: Optimize dataset page layout and internationalization and Fix
setting default values for multi selection drop-down boxes #3221

-Adjust the style and layout of each component on the dataset page
-Add and update multilingual translation content

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 17:29:15 +08:00
a3aa3f0d36 Refa: improve lightrag (#9690)
### What problem does this PR solve?

Improve lightrag.
#9647

### Type of change

- [x] Refactoring
2025-08-25 17:08:44 +08:00
7b8752fe24 fix: Create conversation sessions will lost prologue (#9666)
### What problem does this PR solve?

When create conversation,the prologue hasn't save in conversation.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 14:09:28 +08:00
5e2c33e5b0 Fix: grow reference list (#9674)
### What problem does this PR solve?

Fix Multiple conversations cause the reference list to grow indefinitely
due to Python's mutable default argument behavior.
Explicitly initialize reference as empty list when creating new sessions

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 14:08:15 +08:00
e40be8e541 Feat: Exclude operator_permission field from renaming chat fields #3221 (#9692)
### What problem does this PR solve?

Feat: Exclude operator_permission field from renaming chat fields #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-08-25 14:06:06 +08:00
23d0b564d3 Fix: Wrap VersionDialog in DropdownProvider for proper context (#9677)
### What problem does this PR solve?

The VersionDialog component was not receiving the correct context for
dropdown handling, causing improper behavior in its interactions.
This PR wraps VersionDialog in DropdownProvider to ensure it gets the
proper context and functions as expected.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 10:18:04 +08:00
ecaa9de843 Fix:[ERROR]'LLMBundle' object has no attribute 'language' (#9682)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/9672

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 10:17:10 +08:00
2f74727bb9 Fix: meta data error. (#9670)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 09:41:52 +08:00
adbb038a87 Fix: Place the invitation reminder icon in a separate file #9634 (#9662)
### What problem does this PR solve?

Fix: Place the invitation reminder icon in a separate file #9634
Fix: After receiving the agent message, pull the agent data to highlight
the edges passed #9538

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 20:08:55 +08:00
3947da10ae Fix: unexpected LLM parameters (#9661)
### What problem does this PR solve?

Remove unexpected LLM parameters.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 19:33:09 +08:00
4862be28ad Fix: Search app AI summary ERROR And The tag set cannot be selected #9649 #9652 (#9664)
### What problem does this PR solve?
Fix: Search app AI summary ERROR And The tag set cannot be selected
#9649 #9652
- Search app AI summary ERROR: 'dict' object has no attribute 'split'
#9649
- fix The tag set cannot be selected in the knowledge base. #9652
- Added custom parameter options to the LlmSettingFieldItems component
- Adjusted the document preview height to improve page layout
adaptability

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 19:32:32 +08:00
035e8ed0f7 Fix: code executor timeout (#9671)
### What problem does this PR solve?

Code executor timeout.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 19:31:49 +08:00
cc167ae619 Fix: Display the invited icon in the header #9634 (#9659)
### What problem does this PR solve?

Fix: Display the invited icon in the header #9634

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 15:05:56 +08:00
f8847e7bcd Fix: embedded search AI summary (#9658)
### What problem does this PR solve?

Fix search app AI summary ERROR: 'dict' object has no attribute 'split'.
#9649

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 12:55:29 +08:00
3baebd709b Refactoring: Agent completions API change response structure (#9631)
### What problem does this PR solve?

Resolve #9549 and #9436 , In v0.20.x,Agent completions API changed a
lot,such as without reference and so on

### Type of change

- [x] Refactoring
2025-08-22 12:04:15 +08:00
3e6a4b2628 Fix: Document Previewer is not working #9606 (#9656)
### What problem does this PR solve?
Fix: Document Previewer is not working #9606
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 12:03:51 +08:00
312635cb13 Refactor: based on async await to handle Redis when raptor (#9576)
### What problem does this PR solve?

based on async await to handle Redis when raptor

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2025-08-22 10:58:02 +08:00
756d454122 fix(sdk): add default empty dict for metadata_condition (#9640)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-08-22 10:57:48 +08:00
a4cab371fa Update fr.ts - RAPTOR Issue prompt (#9646)
Removed a line break causing problems with execution in Raptor.

### What problem does this PR solve?

When I activate Raptor without changing anything in French, I encounter
a problem that I don't have with the English version. I noticed in the
logs that there was an extra line break, so I suggest removing it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 09:54:49 +08:00
0d7e52338e Fix: Fixed an issue where knowledge base could not be shared #9634 (#9642)
### What problem does this PR solve?

Fix: Fixed an issue where knowledge base could not be shared #9634

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-22 09:34:11 +08:00
4110f7f5ce Fix: The buttons at the bottom of the dataset settings page are not visible on small screens #9638 (#9639)
### What problem does this PR solve?

Fix: The buttons at the bottom of the dataset settings page are not
visible on small screens #9638
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-21 19:25:14 +08:00
0af57ff772 fix(dataset, next-chats): Fix data form data acquisition logic And Optimize the chat settings interface and add language selection (#9629)
### What problem does this PR solve?

fix(dataset): data form data acquisition logic
fix(next-chats): Optimize the chat settings interface and add language
selection

- Replace form.formControl.trigger with form.trigger
- Use form.getValues() instead of form.formState.values
- Add language selection to support multiple languages
- Add default chat settings values
- Add new settings: icon, description, knowledge base ID, etc.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-08-21 16:57:46 +08:00
0bd58038a8 Fixes (web): Optimized search page style and functionality #3221 (#9627)
### What problem does this PR solve?

Fixes (web): Optimized search page style and functionality #3221

- Updated search page and view title styles
- Modified dataset list and multi-select control styles
- Optimized text field and button styles
- Updated filter button icons
- Adjusted metadata filter styles
- Added default descriptions for the smart assistant

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-21 16:57:14 +08:00
0cbcfcfedf Chore: Update infinity-sdk from 0.6.0.dev4 to 0.6.0.dev5 (#9628)
### What problem does this PR solve?

Bump infinity-sdk dependency to the latest development version
(0.6.0.dev5) in both pyproject.toml and uv.lock files to incorporate
recent changes and fixes from the SDK.

### Type of change

- [x] Other (please describe): Update deps
2025-08-21 16:56:57 +08:00
fbdde0259a Feat: Allow users to parse documents directly after uploading files #3221 (#9633)
### What problem does this PR solve?

Feat: Allow users to parse documents directly after uploading files
#3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-08-21 16:56:22 +08:00
d482173c9b Fix (style): Optimized Datasets color scheme and layout #3221 (#9620)
### What problem does this PR solve?


Fix (style): Optimized Datasets color scheme and layout #3221

- Updated background and text colors for multiple components

- Adjusted some layout structures, such as the paging position of
dataset tables

- Unified status icons and color mapping

- Optimized responsive layout to improve compatibility across different
screen sizes

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-21 12:14:56 +08:00
929dc97509 Fix: duplicated role... (#9622)
### What problem does this PR solve?

#9611
#9603 #9597

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-21 12:14:43 +08:00
30005c0203 Fix: Remove the file size and quantity restrictions of the upload control #9613 #9598 (#9618)
### What problem does this PR solve?

Fix: Remove the file size and quantity restrictions of the upload
control #9613 #9598

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-21 10:54:17 +08:00
382458ace7 Feat: advanced markdown parsing (#9607)
### What problem does this PR solve?

Using AST parsing to handle markdown more accurately, preventing
components from being cut off by chunking. #9564

<img width="1746" height="993" alt="image"
src="https://github.com/user-attachments/assets/4aaf4bf6-5714-4d48-a9cf-864f59633f7f"
/>

<img width="1739" height="982" alt="image"
src="https://github.com/user-attachments/assets/dc00233f-7a55-434f-bbb7-74ce7f57a6cf"
/>

<img width="559" height="100" alt="image"
src="https://github.com/user-attachments/assets/4a556b5b-d9c6-4544-a486-8ac342bd504e"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-08-21 09:36:18 +08:00
4080f6a54a Feature (web): Optimize dataset pages and segmented components #3221 (#9605)
### What problem does this PR solve?

Feature (web): Optimize dataset pages and segmented components #3221
-Add the activeClassName property to Segmented components to customize
the selected state style
-Update the icons and captions of the relevant components on the dataset
page
-Modify the parsing status column title of the dataset table
-Optimize the Segmented component style of the homepage application
section

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-21 09:32:04 +08:00
09570c7eef Feat: expand the capabilities of the MCP Server (#8707)
### What problem does this PR solve?

Expand the capabilities of the MCP Server. #8644.

Special thanks to @Drasek, this change is largely based on his original
implementation, it is super neat and well-structured to me. I basically
just integrated his code into the codebase with minimal modifications.

My main contribution is implementing a proper cache layer for dataset
and document metadata, using the LRU strategy with a 300s ± random 30s
TTL. The original code did not actually perform caching.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Caspar Armster <caspar@armster.de>
2025-08-20 19:30:25 +08:00
312f1a0477 Fix: enlarge raptor timeout limits. (#9600)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 17:29:15 +08:00
1ca226e43b Feat: Updated some colors according to the design draft #3221 (#9599)
### What problem does this PR solve?

Feat: Updated some colors according to the design draft #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-08-20 16:32:29 +08:00
830cda6a3a Fix (web): Optimize text display effect #3221 (#9594)
### What problem does this PR solve?

Fix (web): Optimize text display effect
-Add text ellipsis and overflow hidden classes to the HomeCard component
to achieve text overflow hiding and ellipsis effects
-Add text ellipsis and overflow hidden classes to the DatasetSidebar
component to improve the display of dataset names

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 15:42:21 +08:00
c66dbbe433 Fix: Fixed the issue where the save button at the bottom of the chat page could not be displayed on small screens #3221 (#9596)
### What problem does this PR solve?

Fix: Fixed the issue where the save button at the bottom of the chat
page could not be displayed on small screens #3221

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 15:42:09 +08:00
3b218b2dc0 fix:passing empty database array when updating assistant (#9570)
### What problem does this PR solve?

When the `dataset_ids` parameter is omitted in the **update assistant**
request, Passing an empty array `[]` triggers a misleading
message"Dataset use different embedding models", while omitting the
field does not.
To fix this, we:
- Provide a default empty list: `ids = req.get("dataset_ids", [])`.  
- Replace the `is not None` check with a truthy check: `if ids:`.

**Files changed**  
`api/apps/sdk/chat.py`  
- L153: `ids = req.get("dataset_ids")` → `ids = req.get("dataset_ids",
[])`
- L156: `if ids is not None:` → `if ids:`

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 13:40:05 +08:00
d58ef6127f Fix:KeyError: 'globals' KeyError: 'globals' (#9571)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/9545
add backward compatible logics

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 13:39:38 +08:00
55173c7201 Fix (web): Update the style of segmented controls and add metallic texture gradients (#9591)
### What problem does this PR solve?

Fix (web): Update the style of segmented controls and add metallic
texture gradients #3221
-Modified the selected state style of Segmented components, adding
metallic texture gradient and lower border
-Added a metallic gradient background image in tailwind.diag.js
-Added the -- metallic variable in tailwind.css to define metallic
texture colors

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 13:39:23 +08:00
f860bdf0ad Revert "Feat: reference should also be list after 0.20.x" (#9592)
Reverts infiniflow/ragflow#9582
2025-08-20 13:38:57 +08:00
997627861a Feat: reference should also be list after 0.20.x (#9582)
### What problem does this PR solve?

In 0.19.0 reference is list,and it should be a list,otherwise last
conversation's reference will be lost

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 13:38:14 +08:00
9f9d32d2cd Feat: Make the old page accessible via URL #3221 (#9589)
### What problem does this PR solve?

Feat: Make the old page accessible via URL #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-08-20 13:37:06 +08:00
d55f44601a Docs: Updated v0.20.3 release notes (#9583)
### What problem does this PR solve?
### Type of change

- [x] Documentation Update

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-08-20 10:52:50 +08:00
abb6359547 Docs: Update version references to v0.20.3 in READMEs and docs (#9581)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.20.2 to v0.20.3
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2025-08-20 10:45:44 +08:00
f55ff590d7 Fix: Fixed the issue where the model configuration page could not be scrolled #9572 (#9579)
### What problem does this PR solve?

Fix: Fixed the issue where the model configuration page could not be
scrolled #9572

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-20 10:30:08 +08:00
7d3bb3a2f9 Fix dataset card not responding to click events (#9574)
### What problem does this PR solve?

Fix home card not responding to click events

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-08-20 10:06:14 +08:00
142 changed files with 1934 additions and 691 deletions

View File

@ -22,7 +22,7 @@
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@ -190,7 +190,7 @@ releases! 🌟
> All Docker images are built for x86 platforms. We don't currently offer Docker images for ARM64.
> If you are on an ARM64 platform, follow [this guide](https://ragflow.io/docs/dev/build_docker_image) to build a Docker image compatible with your system.
> The command below downloads the `v0.20.2-slim` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.2-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` for the full edition `v0.20.2`.
> The command below downloads the `v0.20.3-slim` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.3-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` for the full edition `v0.20.3`.
```bash
$ cd ragflow/docker
@ -203,8 +203,8 @@ releases! 🌟
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
|-------------------|-----------------|-----------------------|--------------------------|
| v0.20.2 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.2-slim | &approx;2 | ❌ | Stable release |
| v0.20.3 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.3-slim | &approx;2 | ❌ | Stable release |
| nightly | &approx;9 | :heavy_check_mark: | _Unstable_ nightly build |
| nightly-slim | &approx;2 | ❌ | _Unstable_ nightly build |

View File

@ -22,7 +22,7 @@
<img alt="Lencana Daring" src="https://img.shields.io/badge/Online-Demo-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Rilis%20Terbaru" alt="Rilis Terbaru">
@ -181,7 +181,7 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
> Semua gambar Docker dibangun untuk platform x86. Saat ini, kami tidak menawarkan gambar Docker untuk ARM64.
> Jika Anda menggunakan platform ARM64, [silakan gunakan panduan ini untuk membangun gambar Docker yang kompatibel dengan sistem Anda](https://ragflow.io/docs/dev/build_docker_image).
> Perintah di bawah ini mengunduh edisi v0.20.2-slim dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.20.2-slim, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server. Misalnya, atur RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2 untuk edisi lengkap v0.20.2.
> Perintah di bawah ini mengunduh edisi v0.20.3-slim dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.20.3-slim, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server. Misalnya, atur RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3 untuk edisi lengkap v0.20.3.
```bash
$ cd ragflow/docker
@ -194,8 +194,8 @@ $ docker compose -f docker-compose.yml up -d
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
| ----------------- | --------------- | --------------------- | ------------------------ |
| v0.20.2 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.2-slim | &approx;2 | ❌ | Stable release |
| v0.20.3 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.3-slim | &approx;2 | ❌ | Stable release |
| nightly | &approx;9 | :heavy_check_mark: | _Unstable_ nightly build |
| nightly-slim | &approx;2 | ❌ | _Unstable_ nightly build |

View File

@ -22,7 +22,7 @@
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@ -160,7 +160,7 @@
> 現在、公式に提供されているすべての Docker イメージは x86 アーキテクチャ向けにビルドされており、ARM64 用の Docker イメージは提供されていません。
> ARM64 アーキテクチャのオペレーティングシステムを使用している場合は、[このドキュメント](https://ragflow.io/docs/dev/build_docker_image)を参照して Docker イメージを自分でビルドしてください。
> 以下のコマンドは、RAGFlow Docker イメージの v0.20.2-slim エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.20.2-slim とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。例えば、完全版 v0.20.2 をダウンロードするには、RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2 と設定します。
> 以下のコマンドは、RAGFlow Docker イメージの v0.20.3-slim エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.20.3-slim とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。例えば、完全版 v0.20.3 をダウンロードするには、RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3 と設定します。
```bash
$ cd ragflow/docker
@ -173,8 +173,8 @@
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
| ----------------- | --------------- | --------------------- | ------------------------ |
| v0.20.2 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.2-slim | &approx;2 | ❌ | Stable release |
| v0.20.3 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.3-slim | &approx;2 | ❌ | Stable release |
| nightly | &approx;9 | :heavy_check_mark: | _Unstable_ nightly build |
| nightly-slim | &approx;2 | ❌ | _Unstable_ nightly build |

View File

@ -22,7 +22,7 @@
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@ -160,7 +160,7 @@
> 모든 Docker 이미지는 x86 플랫폼을 위해 빌드되었습니다. 우리는 현재 ARM64 플랫폼을 위한 Docker 이미지를 제공하지 않습니다.
> ARM64 플랫폼을 사용 중이라면, [시스템과 호환되는 Docker 이미지를 빌드하려면 이 가이드를 사용해 주세요](https://ragflow.io/docs/dev/build_docker_image).
> 아래 명령어는 RAGFlow Docker 이미지의 v0.20.2-slim 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.20.2-slim과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오. 예를 들어, 전체 버전인 v0.20.2을 다운로드하려면 RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2로 설정합니다.
> 아래 명령어는 RAGFlow Docker 이미지의 v0.20.3-slim 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.20.3-slim과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오. 예를 들어, 전체 버전인 v0.20.3을 다운로드하려면 RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3로 설정합니다.
```bash
$ cd ragflow/docker
@ -173,8 +173,8 @@
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
| ----------------- | --------------- | --------------------- | ------------------------ |
| v0.20.2 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.2-slim | &approx;2 | ❌ | Stable release |
| v0.20.3 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.3-slim | &approx;2 | ❌ | Stable release |
| nightly | &approx;9 | :heavy_check_mark: | _Unstable_ nightly build |
| nightly-slim | &approx;2 | ❌ | _Unstable_ nightly build |

View File

@ -22,7 +22,7 @@
<img alt="Badge Estático" src="https://img.shields.io/badge/Online-Demo-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Última%20Relese" alt="Última Versão">
@ -180,7 +180,7 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
> Todas as imagens Docker são construídas para plataformas x86. Atualmente, não oferecemos imagens Docker para ARM64.
> Se você estiver usando uma plataforma ARM64, por favor, utilize [este guia](https://ragflow.io/docs/dev/build_docker_image) para construir uma imagem Docker compatível com o seu sistema.
> O comando abaixo baixa a edição `v0.20.2-slim` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.20.2-slim`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor. Por exemplo: defina `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` para a edição completa `v0.20.2`.
> O comando abaixo baixa a edição `v0.20.3-slim` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.20.3-slim`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor. Por exemplo: defina `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` para a edição completa `v0.20.3`.
```bash
$ cd ragflow/docker
@ -193,8 +193,8 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
| Tag da imagem RAGFlow | Tamanho da imagem (GB) | Possui modelos de incorporação? | Estável? |
| --------------------- | ---------------------- | ------------------------------- | ------------------------ |
| v0.20.2 | ~9 | :heavy_check_mark: | Lançamento estável |
| v0.20.2-slim | ~2 | ❌ | Lançamento estável |
| v0.20.3 | ~9 | :heavy_check_mark: | Lançamento estável |
| v0.20.3-slim | ~2 | ❌ | Lançamento estável |
| nightly | ~9 | :heavy_check_mark: | _Instável_ build noturno |
| nightly-slim | ~2 | ❌ | _Instável_ build noturno |

View File

@ -22,7 +22,7 @@
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@ -183,7 +183,7 @@
> 所有 Docker 映像檔都是為 x86 平台建置的。目前,我們不提供 ARM64 平台的 Docker 映像檔。
> 如果您使用的是 ARM64 平台,請使用 [這份指南](https://ragflow.io/docs/dev/build_docker_image) 來建置適合您系統的 Docker 映像檔。
> 執行以下指令會自動下載 RAGFlow slim Docker 映像 `v0.20.2-slim`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.20.2-slim` 的 Docker 映像,請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。例如,你可以透過設定 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` 來下載 RAGFlow 鏡像的 `v0.20.2` 完整發行版。
> 執行以下指令會自動下載 RAGFlow slim Docker 映像 `v0.20.3-slim`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.20.3-slim` 的 Docker 映像,請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。例如,你可以透過設定 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` 來下載 RAGFlow 鏡像的 `v0.20.3` 完整發行版。
```bash
$ cd ragflow/docker
@ -196,8 +196,8 @@
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
| ----------------- | --------------- | --------------------- | ------------------------ |
| v0.20.2 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.2-slim | &approx;2 | ❌ | Stable release |
| v0.20.3 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.3-slim | &approx;2 | ❌ | Stable release |
| nightly | &approx;9 | :heavy_check_mark: | _Unstable_ nightly build |
| nightly-slim | &approx;2 | ❌ | _Unstable_ nightly build |

View File

@ -22,7 +22,7 @@
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@ -183,7 +183,7 @@
> 请注意,目前官方提供的所有 Docker 镜像均基于 x86 架构构建,并不提供基于 ARM64 的 Docker 镜像。
> 如果你的操作系统是 ARM64 架构,请参考[这篇文档](https://ragflow.io/docs/dev/build_docker_image)自行构建 Docker 镜像。
> 运行以下命令会自动下载 RAGFlow slim Docker 镜像 `v0.20.2-slim`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.20.2-slim` 的 Docker 镜像,请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。比如,你可以通过设置 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` 来下载 RAGFlow 镜像的 `v0.20.2` 完整发行版。
> 运行以下命令会自动下载 RAGFlow slim Docker 镜像 `v0.20.3-slim`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.20.3-slim` 的 Docker 镜像,请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。比如,你可以通过设置 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` 来下载 RAGFlow 镜像的 `v0.20.3` 完整发行版。
```bash
$ cd ragflow/docker
@ -196,8 +196,8 @@
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
| ----------------- | --------------- | --------------------- | ------------------------ |
| v0.20.2 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.2-slim | &approx;2 | ❌ | Stable release |
| v0.20.3 | &approx;9 | :heavy_check_mark: | Stable release |
| v0.20.3-slim | &approx;2 | ❌ | Stable release |
| nightly | &approx;9 | :heavy_check_mark: | _Unstable_ nightly build |
| nightly-slim | &approx;2 | ❌ | _Unstable_ nightly build |

View File

@ -131,7 +131,16 @@ class Canvas:
self.path = self.dsl["path"]
self.history = self.dsl["history"]
self.globals = self.dsl["globals"]
if "globals" in self.dsl:
self.globals = self.dsl["globals"]
else:
self.globals = {
"sys.query": "",
"sys.user_id": "",
"sys.conversation_turns": 0,
"sys.files": []
}
self.retrieval = self.dsl["retrieval"]
self.memory = self.dsl.get("memory", [])
@ -417,7 +426,7 @@ class Canvas:
convs = []
if window_size <= 0:
return convs
for role, obj in self.history[window_size * -1:]:
for role, obj in self.history[window_size * -2:]:
if isinstance(obj, dict):
convs.append({"role": role, "content": obj.get("content", "")})
else:

View File

@ -36,7 +36,7 @@ _IS_RAW_CONF = "_is_raw_conf"
class ComponentParamBase(ABC):
def __init__(self):
self.message_history_window_size = 22
self.message_history_window_size = 13
self.inputs = {}
self.outputs = {}
self.description = ""

View File

@ -18,11 +18,8 @@ import logging
import os
import re
from typing import Any, Generator
import json_repair
from copy import deepcopy
from functools import partial
from api.db import LLMType
from api.db.services.llm_service import LLMBundle
from api.db.services.tenant_llm_service import TenantLLMService
@ -130,7 +127,7 @@ class LLM(ComponentBase):
args = {}
vars = self.get_input_elements() if not self._param.debug_inputs else self._param.debug_inputs
prompt = self._param.sys_prompt
sys_prompt = self._param.sys_prompt
for k, o in vars.items():
args[k] = o["value"]
if not isinstance(args[k], str):
@ -141,14 +138,18 @@ class LLM(ComponentBase):
self.set_input_value(k, args[k])
msg = self._canvas.get_history(self._param.message_history_window_size)[:-1]
msg.extend(deepcopy(self._param.prompts))
prompt = self.string_format(prompt, args)
for p in self._param.prompts:
if msg and msg[-1]["role"] == p["role"]:
continue
msg.append(p)
sys_prompt = self.string_format(sys_prompt, args)
for m in msg:
m["content"] = self.string_format(m["content"], args)
if self._param.cite and self._canvas.get_reference()["chunks"]:
prompt += citation_prompt()
sys_prompt += citation_prompt()
return prompt, msg
return sys_prompt, msg
def _generate(self, msg:list[dict], **kwargs) -> str:
if not self.imgs:

View File

@ -79,7 +79,7 @@ def main() -> dict:
return {
"result": fibonacci_recursive(100),
}
Here's a code example for Javascript(`main` function MUST be included and exported):
const axios = require('axios');
async function main(args) {
@ -156,7 +156,7 @@ class CodeExec(ToolBase, ABC):
self.set_output("_ERROR", "construct code request error: " + str(e))
try:
resp = requests.post(url=f"http://{settings.SANDBOX_HOST}:9385/run", json=code_req, timeout=10)
resp = requests.post(url=f"http://{settings.SANDBOX_HOST}:9385/run", json=code_req, timeout=os.environ.get("COMPONENT_EXEC_TIMEOUT", 10*60))
logging.info(f"http://{settings.SANDBOX_HOST}:9385/run", code_req, resp.status_code)
if resp.status_code != 200:
resp.raise_for_status()

View File

@ -150,10 +150,10 @@ def update(tenant_id, chat_id):
if not DialogService.query(tenant_id=tenant_id, id=chat_id, status=StatusEnum.VALID.value):
return get_error_data_result(message="You do not own the chat")
req = request.json
ids = req.get("dataset_ids")
ids = req.get("dataset_ids", [])
if "show_quotation" in req:
req["do_refer"] = req.pop("show_quotation")
if ids is not None:
if ids:
for kb_id in ids:
kbs = KnowledgebaseService.accessible(kb_id=kb_id, user_id=tenant_id)
if not kbs:

View File

@ -24,6 +24,7 @@ from api.db.services.llm_service import LLMBundle
from api import settings
from api.utils.api_utils import validate_request, build_error_result, apikey_required
from rag.app.tag import label_question
from api.db.services.dialog_service import meta_filter
@manager.route('/dify/retrieval', methods=['POST']) # noqa: F821
@ -37,18 +38,23 @@ def retrieval(tenant_id):
retrieval_setting = req.get("retrieval_setting", {})
similarity_threshold = float(retrieval_setting.get("score_threshold", 0.0))
top = int(retrieval_setting.get("top_k", 1024))
metadata_condition = req.get("metadata_condition",{})
metas = DocumentService.get_meta_by_kbs([kb_id])
doc_ids = []
try:
e, kb = KnowledgebaseService.get_by_id(kb_id)
if not e:
return build_error_result(message="Knowledgebase not found!", code=settings.RetCode.NOT_FOUND)
if kb.tenant_id != tenant_id:
return build_error_result(message="Knowledgebase not found!", code=settings.RetCode.NOT_FOUND)
embd_mdl = LLMBundle(kb.tenant_id, LLMType.EMBEDDING.value, llm_name=kb.embd_id)
print(metadata_condition)
print("after",convert_conditions(metadata_condition))
doc_ids.extend(meta_filter(metas, convert_conditions(metadata_condition)))
print("doc_ids",doc_ids)
if not doc_ids and metadata_condition is not None:
doc_ids = ['-999']
ranks = settings.retrievaler.retrieval(
question,
embd_mdl,
@ -59,6 +65,7 @@ def retrieval(tenant_id):
similarity_threshold=similarity_threshold,
vector_similarity_weight=0.3,
top=top,
doc_ids=doc_ids,
rank_feature=label_question(question, [kb])
)
@ -67,6 +74,7 @@ def retrieval(tenant_id):
[tenant_id],
[kb_id],
embd_mdl,
doc_ids,
LLMBundle(kb.tenant_id, LLMType.CHAT))
if ck["content_with_weight"]:
ranks["chunks"].insert(0, ck)
@ -93,3 +101,20 @@ def retrieval(tenant_id):
)
logging.exception(e)
return build_error_result(message=str(e), code=settings.RetCode.SERVER_ERROR)
def convert_conditions(metadata_condition):
if metadata_condition is None:
metadata_condition = {}
op_mapping = {
"is": "=",
"not is": ""
}
return [
{
"op": op_mapping.get(cond["comparison_operator"], cond["comparison_operator"]),
"key": cond["name"],
"value": cond["value"]
}
for cond in metadata_condition.get("conditions", [])
]

View File

@ -84,18 +84,10 @@ def create_agent_session(tenant_id, agent_id):
session_id=get_uuid()
canvas = Canvas(cvs.dsl, tenant_id, agent_id)
canvas.reset()
conv = {
"id": session_id,
"dialog_id": cvs.id,
"user_id": user_id,
"message": [],
"source": "agent",
"dsl": cvs.dsl
}
API4ConversationService.save(**conv)
cvs.dsl = json.loads(str(canvas))
conv = {"id": session_id, "dialog_id": cvs.id, "user_id": user_id, "message": [{"role": "assistant", "content": canvas.get_prologue()}], "source": "agent", "dsl": cvs.dsl}
API4ConversationService.save(**conv)
conv["agent_id"] = conv.pop("dialog_id")
return get_result(data=conv)
@ -450,37 +442,26 @@ def agents_completion_openai_compatibility(tenant_id, agent_id):
def agent_completions(tenant_id, agent_id):
req = request.json
ans = {}
if req.get("stream", True):
def generate():
for answer in agent_completion(tenant_id=tenant_id, agent_id=agent_id, **req):
if isinstance(answer, str):
try:
ans = json.loads(answer[5:]) # remove "data:"
except Exception:
continue
if ans.get("event") != "message":
continue
yield answer
yield "data:[DONE]\n\n"
resp = Response(generate(), mimetype="text/event-stream")
resp = Response(agent_completion(tenant_id=tenant_id, agent_id=agent_id, **req), mimetype="text/event-stream")
resp.headers.add_header("Cache-control", "no-cache")
resp.headers.add_header("Connection", "keep-alive")
resp.headers.add_header("X-Accel-Buffering", "no")
resp.headers.add_header("Content-Type", "text/event-stream; charset=utf-8")
return resp
result = {}
for answer in agent_completion(tenant_id=tenant_id, agent_id=agent_id, **req):
try:
ans = json.loads(answer[5:]) # remove "data:"
if not result:
result = ans.copy()
else:
result["data"]["answer"] += ans["data"]["answer"]
result["data"]["reference"] = ans["data"].get("reference", [])
except Exception as e:
return get_result(data=f"**ERROR**: {str(e)}")
return get_result(data=ans)
return get_error_data_result(str(e))
return result
@manager.route("/chats/<chat_id>/sessions", methods=["GET"]) # noqa: F821
@ -581,6 +562,9 @@ def list_agent_session(tenant_id, agent_id):
"chunks" in conv["reference"][chunk_num]):
chunks = conv["reference"][chunk_num]["chunks"]
for chunk in chunks:
# Ensure chunk is a dictionary before calling get method
if not isinstance(chunk, dict):
continue
new_chunk = {
"id": chunk.get("chunk_id", chunk.get("id")),
"content": chunk.get("content_with_weight", chunk.get("content")),
@ -909,7 +893,7 @@ def ask_about_embedded():
def stream():
nonlocal req, uid
try:
for ans in ask(req["question"], req["kb_ids"], uid, search_config):
for ans in ask(req["question"], req["kb_ids"], uid, search_config=search_config):
yield "data:" + json.dumps({"code": 0, "message": "", "data": ans}, ensure_ascii=False) + "\n\n"
except Exception as e:
yield "data:" + json.dumps({"code": 500, "message": str(e), "data": {"answer": "**ERROR**: " + str(e), "reference": []}}, ensure_ascii=False) + "\n\n"

View File

@ -134,6 +134,25 @@ class UserCanvasService(CommonService):
return False
return True
def structure_answer(conv, ans, message_id, session_id):
if not conv:
return ans
content = ""
if ans["event"] == "message":
if ans["data"].get("start_to_think") is True:
content = "<think>"
elif ans["data"].get("end_to_think") is True:
content = "</think>"
else:
content = ans["data"]["content"]
reference = ans["data"].get("reference")
result = {"id": message_id, "session_id": session_id, "answer": content}
if reference:
result["reference"] = [reference]
return result
def completion(tenant_id, agent_id, session_id=None, **kwargs):
query = kwargs.get("query", "") or kwargs.get("question", "")
files = kwargs.get("files", [])
@ -163,7 +182,8 @@ def completion(tenant_id, agent_id, session_id=None, **kwargs):
"user_id": user_id,
"message": [],
"source": "agent",
"dsl": cvs.dsl
"dsl": cvs.dsl,
"reference": []
}
API4ConversationService.save(**conv)
conv = API4Conversation(**conv)
@ -176,13 +196,14 @@ def completion(tenant_id, agent_id, session_id=None, **kwargs):
})
txt = ""
for ans in canvas.run(query=query, files=files, user_id=user_id, inputs=inputs):
ans["session_id"] = session_id
if ans["event"] == "message":
txt += ans["data"]["content"]
yield "data:" + json.dumps(ans, ensure_ascii=False) + "\n\n"
ans = structure_answer(conv, ans, message_id, session_id)
txt += ans["answer"]
if ans.get("answer") or ans.get("reference"):
yield "data:" + json.dumps({"code": 0, "data": ans},
ensure_ascii=False) + "\n\n"
conv.message.append({"role": "assistant", "content": txt, "created_at": time.time(), "id": message_id})
conv.reference = canvas.get_reference()
conv.reference.append(canvas.get_reference())
conv.errors = canvas.error
conv.dsl = str(canvas)
conv = conv.to_dict()
@ -211,11 +232,9 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
except Exception as e:
logging.exception(f"Agent OpenAI-Compatible completionOpenAI parse answer failed: {e}")
continue
if ans.get("event") != "message":
if not ans["data"]["answer"]:
continue
content_piece = ans["data"]["content"]
content_piece = ans["data"]["answer"]
completion_tokens += len(tiktokenenc.encode(content_piece))
yield "data: " + json.dumps(
@ -260,9 +279,9 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
):
if isinstance(ans, str):
ans = json.loads(ans[5:])
if ans.get("event") != "message":
if not ans["data"]["answer"]:
continue
all_content += ans["data"]["content"]
all_content += ans["data"]["answer"]
completion_tokens = len(tiktokenenc.encode(all_content))

View File

@ -256,10 +256,10 @@ def repair_bad_citation_formats(answer: str, kbinfos: dict, idx: set):
def meta_filter(metas: dict, filters: list[dict]):
doc_ids = []
doc_ids = set([])
def filter_out(v2docs, operator, value):
nonlocal doc_ids
ids = []
for input, docids in v2docs.items():
try:
input = float(input)
@ -284,16 +284,24 @@ def meta_filter(metas: dict, filters: list[dict]):
]:
try:
if all(conds):
doc_ids.extend(docids)
ids.extend(docids)
break
except Exception:
pass
return ids
for k, v2docs in metas.items():
for f in filters:
if k != f["key"]:
continue
filter_out(v2docs, f["op"], f["value"])
return doc_ids
ids = filter_out(v2docs, f["op"], f["value"])
if not doc_ids:
doc_ids = set(ids)
else:
doc_ids = doc_ids & set(ids)
if not doc_ids:
return []
return list(doc_ids)
def chat(dialog, messages, stream=True, **kwargs):

View File

@ -152,7 +152,7 @@ class LLMBundle(LLM4Tenant):
def describe_with_prompt(self, image, prompt):
if self.langfuse:
generation = self.language.start_generation(trace_context=self.trace_context, name="describe_with_prompt", metadata={"model": self.llm_name, "prompt": prompt})
generation = self.langfuse.start_generation(trace_context=self.trace_context, name="describe_with_prompt", metadata={"model": self.llm_name, "prompt": prompt})
txt, used_tokens = self.mdl.describe_with_prompt(image, prompt)
if not TenantLLMService.increase_usage(self.tenant_id, self.llm_type, used_tokens):

View File

@ -17,6 +17,7 @@ import asyncio
import functools
import json
import logging
import os
import queue
import random
import threading
@ -667,7 +668,10 @@ def timeout(seconds: float | int = None, attempts: int = 2, *, exception: Option
for a in range(attempts):
try:
result = result_queue.get(timeout=seconds)
if os.environ.get("ENABLE_TIMEOUT_ASSERTION"):
result = result_queue.get(timeout=seconds)
else:
result = result_queue.get()
if isinstance(result, Exception):
raise result
return result
@ -682,7 +686,10 @@ def timeout(seconds: float | int = None, attempts: int = 2, *, exception: Option
for a in range(attempts):
try:
with trio.fail_after(seconds):
if os.environ.get("ENABLE_TIMEOUT_ASSERTION"):
with trio.fail_after(seconds):
return await func(*args, **kwargs)
else:
return await func(*args, **kwargs)
except trio.TooSlowError:
if a < attempts - 1:

View File

@ -14,13 +14,15 @@
# limitations under the License.
#
from .pdf_parser import RAGFlowPdfParser as PdfParser, PlainParser
from .docx_parser import RAGFlowDocxParser as DocxParser
from .excel_parser import RAGFlowExcelParser as ExcelParser
from .ppt_parser import RAGFlowPptParser as PptParser
from .html_parser import RAGFlowHtmlParser as HtmlParser
from .json_parser import RAGFlowJsonParser as JsonParser
from .markdown_parser import MarkdownElementExtractor
from .markdown_parser import RAGFlowMarkdownParser as MarkdownParser
from .pdf_parser import PlainParser
from .pdf_parser import RAGFlowPdfParser as PdfParser
from .ppt_parser import RAGFlowPptParser as PptParser
from .txt_parser import RAGFlowTxtParser as TxtParser
__all__ = [
@ -33,4 +35,6 @@ __all__ = [
"JsonParser",
"MarkdownParser",
"TxtParser",
]
"MarkdownElementExtractor",
]

View File

@ -17,8 +17,10 @@
import re
import mistune
from markdown import markdown
class RAGFlowMarkdownParser:
def __init__(self, chunk_token_num=128):
self.chunk_token_num = int(chunk_token_num)
@ -35,40 +37,44 @@ class RAGFlowMarkdownParser:
table_list.append(raw_table)
if separate_tables:
# Skip this match (i.e., remove it)
new_text += working_text[last_end:match.start()] + "\n\n"
new_text += working_text[last_end : match.start()] + "\n\n"
else:
# Replace with rendered HTML
html_table = markdown(raw_table, extensions=['markdown.extensions.tables']) if render else raw_table
new_text += working_text[last_end:match.start()] + html_table + "\n\n"
html_table = markdown(raw_table, extensions=["markdown.extensions.tables"]) if render else raw_table
new_text += working_text[last_end : match.start()] + html_table + "\n\n"
last_end = match.end()
new_text += working_text[last_end:]
return new_text
if "|" in markdown_text: # for optimize performance
if "|" in markdown_text: # for optimize performance
# Standard Markdown table
border_table_pattern = re.compile(
r'''
r"""
(?:\n|^)
(?:\|.*?\|.*?\|.*?\n)
(?:\|(?:\s*[:-]+[-| :]*\s*)\|.*?\n)
(?:\|.*?\|.*?\|.*?\n)+
''', re.VERBOSE)
""",
re.VERBOSE,
)
working_text = replace_tables_with_rendered_html(border_table_pattern, tables)
# Borderless Markdown table
no_border_table_pattern = re.compile(
r'''
r"""
(?:\n|^)
(?:\S.*?\|.*?\n)
(?:(?:\s*[:-]+[-| :]*\s*).*?\n)
(?:\S.*?\|.*?\n)+
''', re.VERBOSE)
""",
re.VERBOSE,
)
working_text = replace_tables_with_rendered_html(no_border_table_pattern, tables)
if "<table>" in working_text.lower(): # for optimize performance
#HTML table extraction - handle possible html/body wrapper tags
if "<table>" in working_text.lower(): # for optimize performance
# HTML table extraction - handle possible html/body wrapper tags
html_table_pattern = re.compile(
r'''
r"""
(?:\n|^)
\s*
(?:
@ -83,9 +89,10 @@ class RAGFlowMarkdownParser:
)
\s*
(?=\n|$)
''',
re.VERBOSE | re.DOTALL | re.IGNORECASE
""",
re.VERBOSE | re.DOTALL | re.IGNORECASE,
)
def replace_html_tables():
nonlocal working_text
new_text = ""
@ -94,9 +101,9 @@ class RAGFlowMarkdownParser:
raw_table = match.group()
tables.append(raw_table)
if separate_tables:
new_text += working_text[last_end:match.start()] + "\n\n"
new_text += working_text[last_end : match.start()] + "\n\n"
else:
new_text += working_text[last_end:match.start()] + raw_table + "\n\n"
new_text += working_text[last_end : match.start()] + raw_table + "\n\n"
last_end = match.end()
new_text += working_text[last_end:]
working_text = new_text
@ -104,3 +111,163 @@ class RAGFlowMarkdownParser:
replace_html_tables()
return working_text, tables
class MarkdownElementExtractor:
def __init__(self, markdown_content):
self.markdown_content = markdown_content
self.lines = markdown_content.split("\n")
self.ast_parser = mistune.create_markdown(renderer="ast")
self.ast_nodes = self.ast_parser(markdown_content)
def extract_elements(self):
"""Extract individual elements (headers, code blocks, lists, etc.)"""
sections = []
i = 0
while i < len(self.lines):
line = self.lines[i]
if re.match(r"^#{1,6}\s+.*$", line):
# header
element = self._extract_header(i)
sections.append(element["content"])
i = element["end_line"] + 1
elif line.strip().startswith("```"):
# code block
element = self._extract_code_block(i)
sections.append(element["content"])
i = element["end_line"] + 1
elif re.match(r"^\s*[-*+]\s+.*$", line) or re.match(r"^\s*\d+\.\s+.*$", line):
# list block
element = self._extract_list_block(i)
sections.append(element["content"])
i = element["end_line"] + 1
elif line.strip().startswith(">"):
# blockquote
element = self._extract_blockquote(i)
sections.append(element["content"])
i = element["end_line"] + 1
elif line.strip():
# text block (paragraphs and inline elements until next block element)
element = self._extract_text_block(i)
sections.append(element["content"])
i = element["end_line"] + 1
else:
i += 1
sections = [section for section in sections if section.strip()]
return sections
def _extract_header(self, start_pos):
return {
"type": "header",
"content": self.lines[start_pos],
"start_line": start_pos,
"end_line": start_pos,
}
def _extract_code_block(self, start_pos):
end_pos = start_pos
content_lines = [self.lines[start_pos]]
# Find the end of the code block
for i in range(start_pos + 1, len(self.lines)):
content_lines.append(self.lines[i])
end_pos = i
if self.lines[i].strip().startswith("```"):
break
return {
"type": "code_block",
"content": "\n".join(content_lines),
"start_line": start_pos,
"end_line": end_pos,
}
def _extract_list_block(self, start_pos):
end_pos = start_pos
content_lines = []
i = start_pos
while i < len(self.lines):
line = self.lines[i]
# check if this line is a list item or continuation of a list
if (
re.match(r"^\s*[-*+]\s+.*$", line)
or re.match(r"^\s*\d+\.\s+.*$", line)
or (i > start_pos and not line.strip())
or (i > start_pos and re.match(r"^\s{2,}[-*+]\s+.*$", line))
or (i > start_pos and re.match(r"^\s{2,}\d+\.\s+.*$", line))
or (i > start_pos and re.match(r"^\s+\w+.*$", line))
):
content_lines.append(line)
end_pos = i
i += 1
else:
break
return {
"type": "list_block",
"content": "\n".join(content_lines),
"start_line": start_pos,
"end_line": end_pos,
}
def _extract_blockquote(self, start_pos):
end_pos = start_pos
content_lines = []
i = start_pos
while i < len(self.lines):
line = self.lines[i]
if line.strip().startswith(">") or (i > start_pos and not line.strip()):
content_lines.append(line)
end_pos = i
i += 1
else:
break
return {
"type": "blockquote",
"content": "\n".join(content_lines),
"start_line": start_pos,
"end_line": end_pos,
}
def _extract_text_block(self, start_pos):
"""Extract a text block (paragraphs, inline elements) until next block element"""
end_pos = start_pos
content_lines = [self.lines[start_pos]]
i = start_pos + 1
while i < len(self.lines):
line = self.lines[i]
# stop if we encounter a block element
if re.match(r"^#{1,6}\s+.*$", line) or line.strip().startswith("```") or re.match(r"^\s*[-*+]\s+.*$", line) or re.match(r"^\s*\d+\.\s+.*$", line) or line.strip().startswith(">"):
break
elif not line.strip():
# check if the next line is a block element
if i + 1 < len(self.lines) and (
re.match(r"^#{1,6}\s+.*$", self.lines[i + 1])
or self.lines[i + 1].strip().startswith("```")
or re.match(r"^\s*[-*+]\s+.*$", self.lines[i + 1])
or re.match(r"^\s*\d+\.\s+.*$", self.lines[i + 1])
or self.lines[i + 1].strip().startswith(">")
):
break
else:
content_lines.append(line)
end_pos = i
i += 1
else:
content_lines.append(line)
end_pos = i
i += 1
return {
"type": "text_block",
"content": "\n".join(content_lines),
"start_line": start_pos,
"end_line": end_pos,
}

View File

@ -94,7 +94,7 @@ SVR_HTTP_PORT=9380
# The RAGFlow Docker image to download.
# Defaults to the v0.20.1-slim edition, which is the RAGFlow Docker image without embedding models.
RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2-slim
RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3-slim
#
# To download the RAGFlow Docker image with embedding models, uncomment the following line instead:
# RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.1

View File

@ -79,8 +79,8 @@ The [.env](./.env) file contains important environment variables for Docker.
- `RAGFLOW-IMAGE`
The Docker image edition. Available editions:
- `infiniflow/ragflow:v0.20.2-slim` (default): The RAGFlow Docker image without embedding models.
- `infiniflow/ragflow:v0.20.2`: The RAGFlow Docker image with embedding models including:
- `infiniflow/ragflow:v0.20.3-slim` (default): The RAGFlow Docker image without embedding models.
- `infiniflow/ragflow:v0.20.3`: The RAGFlow Docker image with embedding models including:
- Built-in embedding models:
- `BAAI/bge-large-zh-v1.5`
- `maidalun1020/bce-embedding-base_v1`

View File

@ -99,8 +99,8 @@ RAGFlow utilizes MinIO as its object storage solution, leveraging its scalabilit
- `RAGFLOW-IMAGE`
The Docker image edition. Available editions:
- `infiniflow/ragflow:v0.20.2-slim` (default): The RAGFlow Docker image without embedding models.
- `infiniflow/ragflow:v0.20.2`: The RAGFlow Docker image with embedding models including:
- `infiniflow/ragflow:v0.20.3-slim` (default): The RAGFlow Docker image without embedding models.
- `infiniflow/ragflow:v0.20.3`: The RAGFlow Docker image with embedding models including:
- Built-in embedding models:
- `BAAI/bge-large-zh-v1.5`
- `maidalun1020/bce-embedding-base_v1`

View File

@ -77,7 +77,7 @@ After building the infiniflow/ragflow:nightly-slim image, you are ready to launc
1. Edit Docker Compose Configuration
Open the `docker/.env` file. Find the `RAGFLOW_IMAGE` setting and change the image reference from `infiniflow/ragflow:v0.20.2-slim` to `infiniflow/ragflow:nightly-slim` to use the pre-built image.
Open the `docker/.env` file. Find the `RAGFLOW_IMAGE` setting and change the image reference from `infiniflow/ragflow:v0.20.3-slim` to `infiniflow/ragflow:nightly-slim` to use the pre-built image.
2. Launch the Service

View File

@ -30,17 +30,17 @@ The "garbage in garbage out" status quo remains unchanged despite the fact that
Each RAGFlow release is available in two editions:
- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.2-slim`
- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.2`
- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.3-slim`
- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.3`
---
### Which embedding models can be deployed locally?
RAGFlow offers two Docker image editions, `v0.20.2-slim` and `v0.20.2`:
RAGFlow offers two Docker image editions, `v0.20.3-slim` and `v0.20.3`:
- `infiniflow/ragflow:v0.20.2-slim` (default): The RAGFlow Docker image without embedding models.
- `infiniflow/ragflow:v0.20.2`: The RAGFlow Docker image with embedding models including:
- `infiniflow/ragflow:v0.20.3-slim` (default): The RAGFlow Docker image without embedding models.
- `infiniflow/ragflow:v0.20.3`: The RAGFlow Docker image with embedding models including:
- Built-in embedding models:
- `BAAI/bge-large-zh-v1.5`
- `maidalun1020/bce-embedding-base_v1`

View File

@ -9,7 +9,7 @@ The component equipped with reasoning, tool usage, and multi-agent collaboration
---
An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.2 onwards, an **Agent** component is able to work independently and with the following capabilities:
An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.3 onwards, an **Agent** component is able to work independently and with the following capabilities:
- Autonomous reasoning with reflection and adjustment based on environmental feedback.
- Use of tools or subagents to complete tasks.

View File

@ -9,7 +9,7 @@ A component that retrieves information from specified datasets.
## Scenarios
A **Retrieval** component is essential in most RAG scenarios, where information is extracted from designated knowledge bases before being sent to the LLM for content generation. As of v0.20.2, a **Retrieval** component can operate either as a workflow component or as a tool of an **Agent**, enabling the Agent to control its invocation and search queries.
A **Retrieval** component is essential in most RAG scenarios, where information is extracted from designated knowledge bases before being sent to the LLM for content generation. As of v0.20.3, a **Retrieval** component can operate either as a workflow component or as a tool of an **Agent**, enabling the Agent to control its invocation and search queries.
## Configurations

View File

@ -48,7 +48,7 @@ You start an AI conversation by creating an assistant.
- If no target language is selected, the system will search only in the language of your query, which may cause relevant information in other languages to be missed.
- **Variable** refers to the variables (keys) to be used in the system prompt. `{knowledge}` is a reserved variable. Click **Add** to add more variables for the system prompt.
- If you are uncertain about the logic behind **Variable**, leave it *as-is*.
- As of v0.20.2, if you add custom variables here, the only way you can pass in their values is to call:
- As of v0.20.3, if you add custom variables here, the only way you can pass in their values is to call:
- HTTP method [Converse with chat assistant](../../references/http_api_reference.md#converse-with-chat-assistant), or
- Python method [Converse with chat assistant](../../references/python_api_reference.md#converse-with-chat-assistant).

View File

@ -128,7 +128,7 @@ See [Run retrieval test](./run_retrieval_test.md) for details.
## Search for knowledge base
As of RAGFlow v0.20.2, the search feature is still in a rudimentary form, supporting only knowledge base search by name.
As of RAGFlow v0.20.3, the search feature is still in a rudimentary form, supporting only knowledge base search by name.
![search knowledge base](https://github.com/infiniflow/ragflow/assets/93570324/836ae94c-2438-42be-879e-c7ad2a59693e)

View File

@ -87,4 +87,4 @@ RAGFlow's file management allows you to download an uploaded file:
![download_file](https://github.com/infiniflow/ragflow/assets/93570324/cf3b297f-7d9b-4522-bf5f-4f45743e4ed5)
> As of RAGFlow v0.20.2, bulk download is not supported, nor can you download an entire folder.
> As of RAGFlow v0.20.3, bulk download is not supported, nor can you download an entire folder.

View File

@ -18,7 +18,7 @@ RAGFlow ships with a built-in [Langfuse](https://langfuse.com) integration so th
Langfuse stores traces, spans and prompt payloads in a purpose-built observability backend and offers filtering and visualisations on top.
:::info NOTE
• RAGFlow **≥ 0.20.2** (contains the Langfuse connector)
• RAGFlow **≥ 0.20.3** (contains the Langfuse connector)
• A Langfuse workspace (cloud or self-hosted) with a _Project Public Key_ and _Secret Key_
:::

View File

@ -66,10 +66,10 @@ To upgrade RAGFlow, you must upgrade **both** your code **and** your Docker imag
git clone https://github.com/infiniflow/ragflow.git
```
2. Switch to the latest, officially published release, e.g., `v0.20.2`:
2. Switch to the latest, officially published release, e.g., `v0.20.3`:
```bash
git checkout -f v0.20.2
git checkout -f v0.20.3
```
3. Update **ragflow/docker/.env**:
@ -83,14 +83,14 @@ To upgrade RAGFlow, you must upgrade **both** your code **and** your Docker imag
<TabItem value="slim">
```bash
RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2-slim
RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3-slim
```
</TabItem>
<TabItem value="full">
```bash
RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2
RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3
```
</TabItem>
@ -114,10 +114,10 @@ No, you do not need to. Upgrading RAGFlow in itself will *not* remove your uploa
1. From an environment with Internet access, pull the required Docker image.
2. Save the Docker image to a **.tar** file.
```bash
docker save -o ragflow.v0.20.2.tar infiniflow/ragflow:v0.20.2
docker save -o ragflow.v0.20.3.tar infiniflow/ragflow:v0.20.3
```
3. Copy the **.tar** file to the target server.
4. Load the **.tar** file into Docker:
```bash
docker load -i ragflow.v0.20.2.tar
docker load -i ragflow.v0.20.3.tar
```

View File

@ -44,7 +44,7 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
`vm.max_map_count`. This value sets the maximum number of memory map areas a process may have. Its default value is 65530. While most applications require fewer than a thousand maps, reducing this value can result in abnormal behaviors, and the system will throw out-of-memory errors when a process reaches the limitation.
RAGFlow v0.20.2 uses Elasticsearch or [Infinity](https://github.com/infiniflow/infinity) for multiple recall. Setting the value of `vm.max_map_count` correctly is crucial to the proper functioning of the Elasticsearch component.
RAGFlow v0.20.3 uses Elasticsearch or [Infinity](https://github.com/infiniflow/infinity) for multiple recall. Setting the value of `vm.max_map_count` correctly is crucial to the proper functioning of the Elasticsearch component.
<Tabs
defaultValue="linux"
@ -184,13 +184,13 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
```bash
$ git clone https://github.com/infiniflow/ragflow.git
$ cd ragflow/docker
$ git checkout -f v0.20.2
$ git checkout -f v0.20.3
```
3. Use the pre-built Docker images and start up the server:
:::tip NOTE
The command below downloads the `v0.20.2-slim` edition of the RAGFlow Docker image. Refer to the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.2-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` for the full edition `v0.20.2`.
The command below downloads the `v0.20.3-slim` edition of the RAGFlow Docker image. Refer to the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.3-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` for the full edition `v0.20.3`.
:::
```bash
@ -207,8 +207,8 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
| RAGFlow image tag | Image size (GB) | Has embedding models and Python packages? | Stable? |
| ------------------- | --------------- | ----------------------------------------- | ------------------------ |
| `v0.20.2` | &approx;9 | :heavy_check_mark: | Stable release |
| `v0.20.2-slim` | &approx;2 | ❌ | Stable release |
| `v0.20.3` | &approx;9 | :heavy_check_mark: | Stable release |
| `v0.20.3-slim` | &approx;2 | ❌ | Stable release |
| `nightly` | &approx;9 | :heavy_check_mark: | *Unstable* nightly build |
| `nightly-slim` | &approx;2 | ❌ | *Unstable* nightly build |
@ -217,7 +217,7 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
```
:::danger IMPORTANT
The embedding models included in `v0.20.2` and `nightly` are:
The embedding models included in `v0.20.3` and `nightly` are:
- BAAI/bge-large-zh-v1.5
- maidalun1020/bce-embedding-base_v1

View File

@ -19,7 +19,7 @@ import TOCInline from '@theme/TOCInline';
### Cross-language search
Cross-language search (also known as cross-lingual retrieval) is a feature introduced in version 0.20.2. It enables users to submit queries in one language (for example, English) and retrieve relevant documents written in other languages such as Chinese or Spanish. This feature is enabled by the systems default chat model, which translates queries to ensure accurate matching of semantic meaning across languages.
Cross-language search (also known as cross-lingual retrieval) is a feature introduced in version 0.20.3. It enables users to submit queries in one language (for example, English) and retrieve relevant documents written in other languages such as Chinese or Spanish. This feature is enabled by the systems default chat model, which translates queries to ensure accurate matching of semantic meaning across languages.
By enabling cross-language search, users can effortlessly access a broader range of information regardless of language barriers, significantly enhancing the systems usability and inclusiveness.

View File

@ -3501,7 +3501,7 @@ Failure:
### Generate related questions
**POST** `/v1/sessions/related_questions`
**POST** `/api/v1/sessions/related_questions`
Generates five to ten alternative question strings from the user's original query to retrieve more relevant search results.
@ -3516,7 +3516,7 @@ The chat model autonomously determines the number of questions to generate based
#### Request
- Method: POST
- URL: `/v1/sessions/related_questions`
- URL: `/api/v1/sessions/related_questions`
- Headers:
- `'content-Type: application/json'`
- `'Authorization: Bearer <YOUR_LOGIN_TOKEN>'`
@ -3528,7 +3528,7 @@ The chat model autonomously determines the number of questions to generate based
```bash
curl --request POST \
--url http://{address}/v1/sessions/related_questions \
--url http://{address}/api/v1/sessions/related_questions \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <YOUR_LOGIN_TOKEN>' \
--data '

View File

@ -22,9 +22,9 @@ The embedding models included in a full edition are:
These two embedding models are optimized specifically for English and Chinese, so performance may be compromised if you use them to embed documents in other languages.
:::
## v0.20.2
## v0.20.3
Released on August 19, 2025.
Released on August 20, 2025.
### Improvements

View File

@ -15,6 +15,7 @@
#
import logging
import itertools
import os
import re
from dataclasses import dataclass
from typing import Any, Callable
@ -106,7 +107,8 @@ class EntityResolution(Extractor):
nonlocal remain_candidates_to_resolve, callback
async with semaphore:
try:
with trio.move_on_after(280) as cancel_scope:
enable_timeout_assertion = os.environ.get("ENABLE_TIMEOUT_ASSERTION")
with trio.move_on_after(280 if enable_timeout_assertion else 1000000000) as cancel_scope:
await self._resolve_candidate(candidate_batch, result_set, result_lock)
remain_candidates_to_resolve = remain_candidates_to_resolve - len(candidate_batch[1])
callback(msg=f"Resolved {len(candidate_batch[1])} pairs, {remain_candidates_to_resolve} are remained to resolve. ")
@ -169,7 +171,8 @@ class EntityResolution(Extractor):
logging.info(f"Created resolution prompt {len(text)} bytes for {len(candidate_resolution_i[1])} entity pairs of type {candidate_resolution_i[0]}")
async with chat_limiter:
try:
with trio.move_on_after(240) as cancel_scope:
enable_timeout_assertion = os.environ.get("ENABLE_TIMEOUT_ASSERTION")
with trio.move_on_after(280 if enable_timeout_assertion else 1000000000) as cancel_scope:
response = await trio.to_thread.run_sync(self._chat, text, [{"role": "user", "content": "Output:"}], {})
if cancel_scope.cancelled_caught:
logging.warning("_resolve_candidate._chat timeout, skipping...")

View File

@ -7,6 +7,7 @@ Reference:
import logging
import json
import os
import re
from typing import Callable
from dataclasses import dataclass
@ -51,6 +52,7 @@ class CommunityReportsExtractor(Extractor):
self._max_report_length = max_report_length or 1500
async def __call__(self, graph: nx.Graph, callback: Callable | None = None):
enable_timeout_assertion = os.environ.get("ENABLE_TIMEOUT_ASSERTION")
for node_degree in graph.degree:
graph.nodes[str(node_degree[0])]["rank"] = int(node_degree[1])
@ -92,7 +94,7 @@ class CommunityReportsExtractor(Extractor):
text = perform_variable_replacements(self._extraction_prompt, variables=prompt_variables)
async with chat_limiter:
try:
with trio.move_on_after(180) as cancel_scope:
with trio.move_on_after(180 if enable_timeout_assertion else 1000000000) as cancel_scope:
response = await trio.to_thread.run_sync( self._chat, text, [{"role": "user", "content": "Output:"}], {})
if cancel_scope.cancelled_caught:
logging.warning("extract_community_report._chat timeout, skipping...")

View File

@ -47,7 +47,7 @@ class Extractor:
self._language = language
self._entity_types = entity_types or DEFAULT_ENTITY_TYPES
@timeout(60*5)
@timeout(60*20)
def _chat(self, system, history, gen_conf={}):
hist = deepcopy(history)
conf = deepcopy(gen_conf)

View File

@ -15,6 +15,8 @@
#
import json
import logging
import os
import networkx as nx
import trio
@ -49,6 +51,7 @@ async def run_graphrag(
embedding_model,
callback,
):
enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
start = trio.current_time()
tenant_id, kb_id, doc_id = row["tenant_id"], str(row["kb_id"]), row["doc_id"]
chunks = []
@ -57,7 +60,7 @@ async def run_graphrag(
):
chunks.append(d["content_with_weight"])
with trio.fail_after(max(120, len(chunks)*120)):
with trio.fail_after(max(120, len(chunks)*60*10) if enable_timeout_assertion else 10000000000):
subgraph = await generate_subgraph(
LightKGExt
if "method" not in row["kb_parser_config"].get("graphrag", {}) or row["kb_parser_config"]["graphrag"]["method"] != "general"

View File

@ -130,7 +130,36 @@ Output:
PROMPTS[
"entiti_continue_extraction"
] = """MANY entities were missed in the last extraction. Add them below using the same format:
] = """
MANY entities and relationships were missed in the last extraction. Please find only the missing entities and relationships from previous text.
---Remember Steps---
1. Identify all entities. For each identified entity, extract the following information:
- entity_name: Name of the entity, use same language as input text. If English, capitalized the name
- entity_type: One of the following types: [{entity_types}]
- entity_description: Provide a comprehensive description of the entity's attributes and activities *based solely on the information present in the input text*. **Do not infer or hallucinate information not explicitly stated.** If the text provides insufficient information to create a comprehensive description, state "Description not available in text."
Format each entity as ("entity"{tuple_delimiter}<entity_name>{tuple_delimiter}<entity_type>{tuple_delimiter}<entity_description>)
2. From the entities identified in step 1, identify all pairs of (source_entity, target_entity) that are *clearly related* to each other.
For each pair of related entities, extract the following information:
- source_entity: name of the source entity, as identified in step 1
- target_entity: name of the target entity, as identified in step 1
- relationship_description: explanation as to why you think the source entity and the target entity are related to each other
- relationship_strength: a numeric score indicating strength of the relationship between the source entity and target entity
- relationship_keywords: one or more high-level key words that summarize the overarching nature of the relationship, focusing on concepts or themes rather than specific details
Format each relationship as ("relationship"{tuple_delimiter}<source_entity>{tuple_delimiter}<target_entity>{tuple_delimiter}<relationship_description>{tuple_delimiter}<relationship_keywords>{tuple_delimiter}<relationship_strength>)
3. Identify high-level key words that summarize the main concepts, themes, or topics of the entire text. These should capture the overarching ideas present in the document.
Format the content-level key words as ("content_keywords"{tuple_delimiter}<high_level_keywords>)
4. Return output in {language} as a single list of all the entities and relationships identified in steps 1 and 2. Use **{record_delimiter}** as the list delimiter.
5. When finished, output {completion_delimiter}
---Output---
Add new entities and relations below using the same format, and do not include entities and relations that have been previously extracted. :
"""
PROMPTS[
@ -252,4 +281,4 @@ When handling information with timestamps:
- List up to 5 most important reference sources at the end under "References", clearly indicating whether each source is from Knowledge Graph (KG) or Vector Data (VD)
Format: [KG/VD] Source content
Add sections and commentary to the response as appropriate for the length and format. If the provided information is insufficient to answer the question, clearly state that you don't know or cannot provide an answer in the same language as the user's question."""
Add sections and commentary to the response as appropriate for the length and format. If the provided information is insufficient to answer the question, clearly state that you don't know or cannot provide an answer in the same language as the user's question."""

View File

@ -307,6 +307,7 @@ def chunk_id(chunk):
async def graph_node_to_chunk(kb_id, embd_mdl, ent_name, meta, chunks):
global chat_limiter
enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
chunk = {
"id": get_uuid(),
"important_kwd": [ent_name],
@ -324,7 +325,7 @@ async def graph_node_to_chunk(kb_id, embd_mdl, ent_name, meta, chunks):
ebd = get_embed_cache(embd_mdl.llm_name, ent_name)
if ebd is None:
async with chat_limiter:
with trio.fail_after(3):
with trio.fail_after(3 if enable_timeout_assertion else 30000000):
ebd, _ = await trio.to_thread.run_sync(lambda: embd_mdl.encode([ent_name]))
ebd = ebd[0]
set_embed_cache(embd_mdl.llm_name, ent_name, ebd)
@ -362,6 +363,7 @@ def get_relation(tenant_id, kb_id, from_ent_name, to_ent_name, size=1):
async def graph_edge_to_chunk(kb_id, embd_mdl, from_ent_name, to_ent_name, meta, chunks):
enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
chunk = {
"id": get_uuid(),
"from_entity_kwd": from_ent_name,
@ -380,7 +382,7 @@ async def graph_edge_to_chunk(kb_id, embd_mdl, from_ent_name, to_ent_name, meta,
ebd = get_embed_cache(embd_mdl.llm_name, txt)
if ebd is None:
async with chat_limiter:
with trio.fail_after(3):
with trio.fail_after(3 if enable_timeout_assertion else 300000000):
ebd, _ = await trio.to_thread.run_sync(lambda: embd_mdl.encode([txt+f": {meta['description']}"]))
ebd = ebd[0]
set_embed_cache(embd_mdl.llm_name, txt, ebd)
@ -514,9 +516,10 @@ async def set_graph(tenant_id: str, kb_id: str, embd_mdl, graph: nx.Graph, chang
callback(msg=f"set_graph converted graph change to {len(chunks)} chunks in {now - start:.2f}s.")
start = now
enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
es_bulk_size = 4
for b in range(0, len(chunks), es_bulk_size):
with trio.fail_after(3):
with trio.fail_after(3 if enable_timeout_assertion else 30000000):
doc_store_result = await trio.to_thread.run_sync(lambda: settings.docStoreConn.insert(chunks[b:b + es_bulk_size], search.index_name(tenant_id), kb_id))
if b % 100 == es_bulk_size and callback:
callback(msg=f"Insert chunks: {b}/{len(chunks)}")

View File

@ -56,7 +56,7 @@ env:
ragflow:
image:
repository: infiniflow/ragflow
tag: v0.20.2-slim
tag: v0.20.3-slim
pullPolicy: IfNotPresent
pullSecrets: []
# Optional service configuration overrides

View File

@ -16,6 +16,9 @@
import json
import logging
import random
import time
from collections import OrderedDict
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from functools import wraps
@ -53,6 +56,13 @@ JSON_RESPONSE = True
class RAGFlowConnector:
_MAX_DATASET_CACHE = 32
_MAX_DOCUMENT_CACHE = 128
_CACHE_TTL = 300
_dataset_metadata_cache: OrderedDict[str, tuple[dict, float | int]] = OrderedDict() # "dataset_id" -> (metadata, expiry_ts)
_document_metadata_cache: OrderedDict[str, tuple[list[tuple[str, dict]], float | int]] = OrderedDict() # "dataset_id" -> ([(document_id, doc_metadata)], expiry_ts)
def __init__(self, base_url: str, version="v1"):
self.base_url = base_url
self.version = version
@ -72,6 +82,43 @@ class RAGFlowConnector:
res = requests.get(url=self.api_url + path, params=params, headers=self.authorization_header, json=json)
return res
def _is_cache_valid(self, ts):
return time.time() < ts
def _get_expiry_timestamp(self):
offset = random.randint(-30, 30)
return time.time() + self._CACHE_TTL + offset
def _get_cached_dataset_metadata(self, dataset_id):
entry = self._dataset_metadata_cache.get(dataset_id)
if entry:
data, ts = entry
if self._is_cache_valid(ts):
self._dataset_metadata_cache.move_to_end(dataset_id)
return data
return None
def _set_cached_dataset_metadata(self, dataset_id, metadata):
self._dataset_metadata_cache[dataset_id] = (metadata, self._get_expiry_timestamp())
self._dataset_metadata_cache.move_to_end(dataset_id)
if len(self._dataset_metadata_cache) > self._MAX_DATASET_CACHE:
self._dataset_metadata_cache.popitem(last=False)
def _get_cached_document_metadata_by_dataset(self, dataset_id):
entry = self._document_metadata_cache.get(dataset_id)
if entry:
data_list, ts = entry
if self._is_cache_valid(ts):
self._document_metadata_cache.move_to_end(dataset_id)
return {doc_id: doc_meta for doc_id, doc_meta in data_list}
return None
def _set_cached_document_metadata_by_dataset(self, dataset_id, doc_id_meta_list):
self._document_metadata_cache[dataset_id] = (doc_id_meta_list, self._get_expiry_timestamp())
self._document_metadata_cache.move_to_end(dataset_id)
if len(self._document_metadata_cache) > self._MAX_DOCUMENT_CACHE:
self._document_metadata_cache.popitem(last=False)
def list_datasets(self, page: int = 1, page_size: int = 1000, orderby: str = "create_time", desc: bool = True, id: str | None = None, name: str | None = None):
res = self._get("/datasets", {"page": page, "page_size": page_size, "orderby": orderby, "desc": desc, "id": id, "name": name})
if not res:
@ -87,10 +134,38 @@ class RAGFlowConnector:
return ""
def retrieval(
self, dataset_ids, document_ids=None, question="", page=1, page_size=30, similarity_threshold=0.2, vector_similarity_weight=0.3, top_k=1024, rerank_id: str | None = None, keyword: bool = False
self,
dataset_ids,
document_ids=None,
question="",
page=1,
page_size=30,
similarity_threshold=0.2,
vector_similarity_weight=0.3,
top_k=1024,
rerank_id: str | None = None,
keyword: bool = False,
force_refresh: bool = False,
):
if document_ids is None:
document_ids = []
# If no dataset_ids provided or empty list, get all available dataset IDs
if not dataset_ids:
dataset_list_str = self.list_datasets()
dataset_ids = []
# Parse the dataset list to extract IDs
if dataset_list_str:
for line in dataset_list_str.strip().split('\n'):
if line.strip():
try:
dataset_info = json.loads(line.strip())
dataset_ids.append(dataset_info["id"])
except (json.JSONDecodeError, KeyError):
# Skip malformed lines
continue
data_json = {
"page": page,
"page_size": page_size,
@ -110,12 +185,127 @@ class RAGFlowConnector:
res = res.json()
if res.get("code") == 0:
data = res["data"]
chunks = []
for chunk_data in res["data"].get("chunks"):
chunks.append(json.dumps(chunk_data, ensure_ascii=False))
return [types.TextContent(type="text", text="\n".join(chunks))]
# Cache document metadata and dataset information
document_cache, dataset_cache = self._get_document_metadata_cache(dataset_ids, force_refresh=force_refresh)
# Process chunks with enhanced field mapping including per-chunk metadata
for chunk_data in data.get("chunks", []):
enhanced_chunk = self._map_chunk_fields(chunk_data, dataset_cache, document_cache)
chunks.append(enhanced_chunk)
# Build structured response (no longer need response-level document_metadata)
response = {
"chunks": chunks,
"pagination": {
"page": data.get("page", page),
"page_size": data.get("page_size", page_size),
"total_chunks": data.get("total", len(chunks)),
"total_pages": (data.get("total", len(chunks)) + page_size - 1) // page_size,
},
"query_info": {
"question": question,
"similarity_threshold": similarity_threshold,
"vector_weight": vector_similarity_weight,
"keyword_search": keyword,
"dataset_count": len(dataset_ids),
},
}
return [types.TextContent(type="text", text=json.dumps(response, ensure_ascii=False))]
raise Exception([types.TextContent(type="text", text=res.get("message"))])
def _get_document_metadata_cache(self, dataset_ids, force_refresh=False):
"""Cache document metadata for all documents in the specified datasets"""
document_cache = {}
dataset_cache = {}
try:
for dataset_id in dataset_ids:
dataset_meta = None if force_refresh else self._get_cached_dataset_metadata(dataset_id)
if not dataset_meta:
# First get dataset info for name
dataset_res = self._get("/datasets", {"id": dataset_id, "page_size": 1})
if dataset_res and dataset_res.status_code == 200:
dataset_data = dataset_res.json()
if dataset_data.get("code") == 0 and dataset_data.get("data"):
dataset_info = dataset_data["data"][0]
dataset_meta = {"name": dataset_info.get("name", "Unknown"), "description": dataset_info.get("description", "")}
self._set_cached_dataset_metadata(dataset_id, dataset_meta)
if dataset_meta:
dataset_cache[dataset_id] = dataset_meta
docs = None if force_refresh else self._get_cached_document_metadata_by_dataset(dataset_id)
if docs is None:
docs_res = self._get(f"/datasets/{dataset_id}/documents")
docs_data = docs_res.json()
if docs_data.get("code") == 0 and docs_data.get("data", {}).get("docs"):
doc_id_meta_list = []
docs = {}
for doc in docs_data["data"]["docs"]:
doc_id = doc.get("id")
if not doc_id:
continue
doc_meta = {
"document_id": doc_id,
"name": doc.get("name", ""),
"location": doc.get("location", ""),
"type": doc.get("type", ""),
"size": doc.get("size"),
"chunk_count": doc.get("chunk_count"),
# "chunk_method": doc.get("chunk_method", ""),
"create_date": doc.get("create_date", ""),
"update_date": doc.get("update_date", ""),
# "process_begin_at": doc.get("process_begin_at", ""),
# "process_duration": doc.get("process_duration"),
# "progress": doc.get("progress"),
# "progress_msg": doc.get("progress_msg", ""),
# "status": doc.get("status", ""),
# "run": doc.get("run", ""),
"token_count": doc.get("token_count"),
# "source_type": doc.get("source_type", ""),
"thumbnail": doc.get("thumbnail", ""),
"dataset_id": doc.get("dataset_id", dataset_id),
"meta_fields": doc.get("meta_fields", {}),
# "parser_config": doc.get("parser_config", {})
}
doc_id_meta_list.append((doc_id, doc_meta))
docs[doc_id] = doc_meta
self._set_cached_document_metadata_by_dataset(dataset_id, doc_id_meta_list)
if docs:
document_cache.update(docs)
except Exception:
# Gracefully handle metadata cache failures
pass
return document_cache, dataset_cache
def _map_chunk_fields(self, chunk_data, dataset_cache, document_cache):
"""Preserve all original API fields and add per-chunk document metadata"""
# Start with ALL raw data from API (preserve everything like original version)
mapped = dict(chunk_data)
# Add dataset name enhancement
dataset_id = chunk_data.get("dataset_id") or chunk_data.get("kb_id")
if dataset_id and dataset_id in dataset_cache:
mapped["dataset_name"] = dataset_cache[dataset_id]["name"]
else:
mapped["dataset_name"] = "Unknown"
# Add document name convenience field
mapped["document_name"] = chunk_data.get("document_keyword", "")
# Add per-chunk document metadata
document_id = chunk_data.get("document_id")
if document_id and document_id in document_cache:
mapped["document_metadata"] = document_cache[document_id]
return mapped
class RAGFlowCtx:
def __init__(self, connector: RAGFlowConnector):
@ -195,7 +385,58 @@ async def list_tools(*, connector) -> list[types.Tool]:
"items": {"type": "string"},
"description": "Optional array of document IDs to search within."
},
"question": {"type": "string", "description": "The question or query to search for."},
"question": {
"type": "string",
"description": "The question or query to search for."
},
"page": {
"type": "integer",
"description": "Page number for pagination",
"default": 1,
"minimum": 1,
},
"page_size": {
"type": "integer",
"description": "Number of results to return per page (default: 10, max recommended: 50 to avoid token limits)",
"default": 10,
"minimum": 1,
"maximum": 100,
},
"similarity_threshold": {
"type": "number",
"description": "Minimum similarity threshold for results",
"default": 0.2,
"minimum": 0.0,
"maximum": 1.0,
},
"vector_similarity_weight": {
"type": "number",
"description": "Weight for vector similarity vs term similarity",
"default": 0.3,
"minimum": 0.0,
"maximum": 1.0,
},
"keyword": {
"type": "boolean",
"description": "Enable keyword-based search",
"default": False,
},
"top_k": {
"type": "integer",
"description": "Maximum results to consider before ranking",
"default": 1024,
"minimum": 1,
"maximum": 1024,
},
"rerank_id": {
"type": "string",
"description": "Optional reranking model identifier",
},
"force_refresh": {
"type": "boolean",
"description": "Set to true only if fresh dataset and document metadata is explicitly required. Otherwise, cached metadata is used (default: false).",
"default": False,
},
},
"required": ["question"],
},
@ -209,6 +450,16 @@ async def call_tool(name: str, arguments: dict, *, connector) -> list[types.Text
if name == "ragflow_retrieval":
document_ids = arguments.get("document_ids", [])
dataset_ids = arguments.get("dataset_ids", [])
question = arguments.get("question", "")
page = arguments.get("page", 1)
page_size = arguments.get("page_size", 10)
similarity_threshold = arguments.get("similarity_threshold", 0.2)
vector_similarity_weight = arguments.get("vector_similarity_weight", 0.3)
keyword = arguments.get("keyword", False)
top_k = arguments.get("top_k", 1024)
rerank_id = arguments.get("rerank_id")
force_refresh = arguments.get("force_refresh", False)
# If no dataset_ids provided or empty list, get all available dataset IDs
if not dataset_ids:
@ -229,7 +480,15 @@ async def call_tool(name: str, arguments: dict, *, connector) -> list[types.Text
return connector.retrieval(
dataset_ids=dataset_ids,
document_ids=document_ids,
question=arguments["question"],
question=question,
page=page,
page_size=page_size,
similarity_threshold=similarity_threshold,
vector_similarity_weight=vector_similarity_weight,
keyword=keyword,
top_k=top_k,
rerank_id=rerank_id,
force_refresh=force_refresh,
)
raise ValueError(f"Tool not found: {name}")

View File

@ -1,6 +1,6 @@
[project]
name = "ragflow"
version = "0.20.2"
version = "0.20.3"
description = "[RAGFlow](https://ragflow.io/) is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding. It offers a streamlined RAG workflow for businesses of any scale, combining LLM (Large Language Models) to provide truthful question-answering capabilities, backed by well-founded citations from various complex formatted data."
authors = [{ name = "Zhichang Yu", email = "yuzhichang@gmail.com" }]
license-files = ["LICENSE"]
@ -45,7 +45,7 @@ dependencies = [
"html-text==0.6.2",
"httpx[socks]==0.27.2",
"huggingface-hub>=0.25.0,<0.26.0",
"infinity-sdk==0.6.0-dev4",
"infinity-sdk==0.6.0.dev5",
"infinity-emb>=0.0.66,<0.0.67",
"itsdangerous==2.1.2",
"json-repair==0.35.0",

View File

@ -30,7 +30,7 @@ from tika import parser
from api.db import LLMType
from api.db.services.llm_service import LLMBundle
from deepdoc.parser import DocxParser, ExcelParser, HtmlParser, JsonParser, MarkdownParser, PdfParser, TxtParser
from deepdoc.parser import DocxParser, ExcelParser, HtmlParser, JsonParser, MarkdownElementExtractor, MarkdownParser, PdfParser, TxtParser
from deepdoc.parser.figure_parser import VisionFigureParser, vision_figure_parser_figure_data_wrapper
from deepdoc.parser.pdf_parser import PlainParser, VisionParser
from rag.nlp import concat_img, find_codec, naive_merge, naive_merge_with_images, naive_merge_docx, rag_tokenizer, tokenize_chunks, tokenize_chunks_with_images, tokenize_table
@ -289,7 +289,7 @@ class Pdf(PdfParser):
return [(b["text"], self._line_tag(b, zoomin)) for b in self.boxes], tbls, figures
else:
tbls = self._extract_table_figure(True, zoomin, True, True)
# self._naive_vertical_merge()
self._naive_vertical_merge()
self._concat_downward()
# self._filter_forpages()
logging.info("layouts cost: {}s".format(timer() - first_start))
@ -350,17 +350,14 @@ class Markdown(MarkdownParser):
else:
with open(filename, "r") as f:
txt = f.read()
remainder, tables = self.extract_tables_and_remainder(f'{txt}\n', separate_tables=separate_tables)
sections = []
extractor = MarkdownElementExtractor(txt)
element_sections = extractor.extract_elements()
sections = [(element, "") for element in element_sections]
tbls = []
for sec in remainder.split("\n"):
if sec.strip().find("#") == 0:
sections.append((sec, ""))
elif sections and sections[-1][0].strip().find("#") == 0:
sec_, _ = sections.pop(-1)
sections.append((sec_ + "\n" + sec, ""))
else:
sections.append((sec, ""))
for table in tables:
tbls.append(((None, markdown(table, extensions=['markdown.extensions.tables'])), ""))
return sections, tbls

View File

@ -36,6 +36,7 @@ class SupportedLiteLLMProvider(StrEnum):
Nvidia = "NVIDIA"
TogetherAI = "TogetherAI"
Anthropic = "Anthropic"
Ollama = "Ollama"
FACTORY_DEFAULT_BASE_URL = {
@ -59,6 +60,7 @@ LITELLM_PROVIDER_PREFIX = {
SupportedLiteLLMProvider.Nvidia: "nvidia_nim/",
SupportedLiteLLMProvider.TogetherAI: "together_ai/",
SupportedLiteLLMProvider.Anthropic: "", # don't need a prefix
SupportedLiteLLMProvider.Ollama: "ollama_chat/",
}
ChatModel = globals().get("ChatModel", {})

View File

@ -29,7 +29,6 @@ import json_repair
import litellm
import openai
import requests
from ollama import Client
from openai import OpenAI
from openai.lib.azure import AzureOpenAI
from strenum import StrEnum
@ -112,6 +111,32 @@ class Base(ABC):
def _clean_conf(self, gen_conf):
if "max_tokens" in gen_conf:
del gen_conf["max_tokens"]
allowed_conf = {
"temperature",
"max_completion_tokens",
"top_p",
"stream",
"stream_options",
"stop",
"n",
"presence_penalty",
"frequency_penalty",
"functions",
"function_call",
"logit_bias",
"user",
"response_format",
"seed",
"tools",
"tool_choice",
"logprobs",
"top_logprobs",
"extra_headers",
}
gen_conf = {k: v for k, v in gen_conf.items() if k in allowed_conf}
return gen_conf
def _chat(self, history, gen_conf, **kwargs):
@ -657,73 +682,6 @@ class ZhipuChat(Base):
return super().chat_streamly_with_tools(system, history, gen_conf)
class OllamaChat(Base):
_FACTORY_NAME = "Ollama"
def __init__(self, key, model_name, base_url=None, **kwargs):
super().__init__(key, model_name, base_url=base_url, **kwargs)
self.client = Client(host=base_url) if not key or key == "x" else Client(host=base_url, headers={"Authorization": f"Bearer {key}"})
self.model_name = model_name
self.keep_alive = kwargs.get("ollama_keep_alive", int(os.environ.get("OLLAMA_KEEP_ALIVE", -1)))
def _clean_conf(self, gen_conf):
options = {}
if "max_tokens" in gen_conf:
options["num_predict"] = gen_conf["max_tokens"]
for k in ["temperature", "top_p", "presence_penalty", "frequency_penalty"]:
if k not in gen_conf:
continue
options[k] = gen_conf[k]
return options
def _chat(self, history, gen_conf={}, **kwargs):
# Calculate context size
ctx_size = self._calculate_dynamic_ctx(history)
gen_conf["num_ctx"] = ctx_size
response = self.client.chat(model=self.model_name, messages=history, options=gen_conf, keep_alive=self.keep_alive)
ans = response["message"]["content"].strip()
token_count = response.get("eval_count", 0) + response.get("prompt_eval_count", 0)
return ans, token_count
def chat_streamly(self, system, history, gen_conf={}, **kwargs):
if system:
history.insert(0, {"role": "system", "content": system})
if "max_tokens" in gen_conf:
del gen_conf["max_tokens"]
try:
# Calculate context size
ctx_size = self._calculate_dynamic_ctx(history)
options = {"num_ctx": ctx_size}
if "temperature" in gen_conf:
options["temperature"] = gen_conf["temperature"]
if "max_tokens" in gen_conf:
options["num_predict"] = gen_conf["max_tokens"]
if "top_p" in gen_conf:
options["top_p"] = gen_conf["top_p"]
if "presence_penalty" in gen_conf:
options["presence_penalty"] = gen_conf["presence_penalty"]
if "frequency_penalty" in gen_conf:
options["frequency_penalty"] = gen_conf["frequency_penalty"]
ans = ""
try:
response = self.client.chat(model=self.model_name, messages=history, stream=True, options=options, keep_alive=self.keep_alive)
for resp in response:
if resp["done"]:
token_count = resp.get("prompt_eval_count", 0) + resp.get("eval_count", 0)
yield token_count
ans = resp["message"]["content"]
yield ans
except Exception as e:
yield ans + "\n**ERROR**: " + str(e)
yield 0
except Exception as e:
yield "**ERROR**: " + str(e)
yield 0
class LocalAIChat(Base):
_FACTORY_NAME = "LocalAI"
@ -1396,7 +1354,7 @@ class Ai302Chat(Base):
class LiteLLMBase(ABC):
_FACTORY_NAME = ["Tongyi-Qianwen", "Bedrock", "Moonshot", "xAI", "DeepInfra", "Groq", "Cohere", "Gemini", "DeepSeek", "NVIDIA", "TogetherAI", "Anthropic"]
_FACTORY_NAME = ["Tongyi-Qianwen", "Bedrock", "Moonshot", "xAI", "DeepInfra", "Groq", "Cohere", "Gemini", "DeepSeek", "NVIDIA", "TogetherAI", "Anthropic", "Ollama"]
def __init__(self, key, model_name, base_url=None, **kwargs):
self.timeout = int(os.environ.get("LM_TIMEOUT_SECONDS", 600))

View File

@ -114,6 +114,8 @@ def kb_prompt(kbinfos, max_tokens, hash_id=False):
docs = {d.id: d.meta_fields for d in docs}
def draw_node(k, line):
if line is not None and not isinstance(line, str):
line = str(line)
if not line:
return ""
return f"\n├── {k}: " + re.sub(r"\n+", " ", line, flags=re.DOTALL)

View File

@ -42,9 +42,12 @@ class RecursiveAbstractiveProcessing4TreeOrganizedRetrieval:
self._prompt = prompt
self._max_token = max_token
@timeout(60)
@timeout(60*20)
async def _chat(self, system, history, gen_conf):
response = get_llm_cache(self._llm_model.llm_name, system, history, gen_conf)
response = await trio.to_thread.run_sync(
lambda: get_llm_cache(self._llm_model.llm_name, system, history, gen_conf)
)
if response:
return response
response = await trio.to_thread.run_sync(
@ -53,19 +56,23 @@ class RecursiveAbstractiveProcessing4TreeOrganizedRetrieval:
response = re.sub(r"^.*</think>", "", response, flags=re.DOTALL)
if response.find("**ERROR**") >= 0:
raise Exception(response)
set_llm_cache(self._llm_model.llm_name, system, response, history, gen_conf)
await trio.to_thread.run_sync(
lambda: set_llm_cache(self._llm_model.llm_name, system, response, history, gen_conf)
)
return response
@timeout(2)
@timeout(20)
async def _embedding_encode(self, txt):
response = get_embed_cache(self._embd_model.llm_name, txt)
response = await trio.to_thread.run_sync(
lambda: get_embed_cache(self._embd_model.llm_name, txt)
)
if response is not None:
return response
embds, _ = await trio.to_thread.run_sync(lambda: self._embd_model.encode([txt]))
if len(embds) < 1 or len(embds[0]) < 1:
raise Exception("Embedding error: ")
embds = embds[0]
set_embed_cache(self._embd_model.llm_name, txt, embds)
await trio.to_thread.run_sync(lambda: set_embed_cache(self._embd_model.llm_name, txt, embds))
return embds
def _get_optimal_clusters(self, embeddings: np.ndarray, random_state: int):
@ -86,7 +93,7 @@ class RecursiveAbstractiveProcessing4TreeOrganizedRetrieval:
layers = [(0, len(chunks))]
start, end = 0, len(chunks)
@timeout(60)
@timeout(60*20)
async def summarize(ck_idx: list[int]):
nonlocal chunks
texts = [chunks[i][0] for i in ck_idx]

View File

@ -21,7 +21,7 @@ import sys
import threading
import time
from api.utils.api_utils import timeout, is_strong_enough
from api.utils.api_utils import timeout
from api.utils.log_utils import init_root_logger, get_project_base_directory
from graphrag.general.index import run_graphrag
from graphrag.utils import get_llm_cache, set_llm_cache, get_tags_from_cache, set_tags_to_cache
@ -478,8 +478,6 @@ async def embedding(docs, mdl, parser_config=None, callback=None):
@timeout(3600)
async def run_raptor(row, chat_mdl, embd_mdl, vector_size, callback=None):
# Pressure test for GraphRAG task
await is_strong_enough(chat_mdl, embd_mdl)
chunks = []
vctr_nm = "q_%d_vec"%vector_size
for d in settings.retrievaler.chunk_list(row["doc_id"], row["tenant_id"], [str(row["kb_id"])],
@ -553,7 +551,6 @@ async def do_handle_task(task):
try:
# bind embedding model
embedding_model = LLMBundle(task_tenant_id, LLMType.EMBEDDING, llm_name=task_embedding_id, lang=task_language)
await is_strong_enough(None, embedding_model)
vts, _ = embedding_model.encode(["ok"])
vector_size = len(vts[0])
except Exception as e:
@ -568,7 +565,6 @@ async def do_handle_task(task):
if task.get("task_type", "") == "raptor":
# bind LLM for raptor
chat_model = LLMBundle(task_tenant_id, LLMType.CHAT, llm_name=task_llm_id, lang=task_language)
await is_strong_enough(chat_model, None)
# run RAPTOR
async with kg_limiter:
chunks, token_count = await run_raptor(task, chat_model, embedding_model, vector_size, progress_callback)
@ -580,7 +576,6 @@ async def do_handle_task(task):
graphrag_conf = task["kb_parser_config"].get("graphrag", {})
start_ts = timer()
chat_model = LLMBundle(task_tenant_id, LLMType.CHAT, llm_name=task_llm_id, lang=task_language)
await is_strong_enough(chat_model, None)
with_resolution = graphrag_conf.get("resolution", False)
with_community = graphrag_conf.get("community", False)
async with kg_limiter:

View File

@ -22,7 +22,7 @@ from util import format_timeout_duration, parse_timeout_duration
from core.container import init_containers, teardown_containers
from core.logger import logger
TIMEOUT = 10
TIMEOUT = parse_timeout_duration(os.getenv("SANDBOX_TIMEOUT", "10s"))
@asynccontextmanager
@ -39,6 +39,5 @@ async def _lifespan(app: FastAPI):
def init():
TIMEOUT = parse_timeout_duration(os.getenv("SANDBOX_TIMEOUT"))
logger.info(f"Global timeout: {format_timeout_duration(TIMEOUT)}")
return _lifespan

View File

@ -1,6 +1,6 @@
[project]
name = "ragflow-sdk"
version = "0.20.2"
version = "0.20.3"
description = "Python client sdk of [RAGFlow](https://github.com/infiniflow/ragflow). RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding."
authors = [{ name = "Zhichang Yu", email = "yuzhichang@gmail.com" }]
license = { text = "Apache License, Version 2.0" }

2
sdk/python/uv.lock generated
View File

@ -342,7 +342,7 @@ wheels = [
[[package]]
name = "ragflow-sdk"
version = "0.20.2"
version = "0.20.3"
source = { virtual = "." }
dependencies = [
{ name = "beartype" },

8
uv.lock generated
View File

@ -2603,7 +2603,7 @@ wheels = [
[[package]]
name = "infinity-sdk"
version = "0.6.0.dev4"
version = "0.6.0.dev5"
source = { registry = "https://mirrors.aliyun.com/pypi/simple" }
dependencies = [
{ name = "numpy" },
@ -2620,7 +2620,7 @@ dependencies = [
{ name = "thrift" },
]
wheels = [
{ url = "https://mirrors.aliyun.com/pypi/packages/d4/cc/645ed8de15952940c7308a788036376583a5fc29fdcf3e4bc75b5ad0c881/infinity_sdk-0.6.0.dev4-py3-none-any.whl", hash = "sha256:f8f4bd8a44e3fae7b4228b5c9e9a16559b4905f50d2d7d0a3d18f39974613e7a" },
{ url = "https://mirrors.aliyun.com/pypi/packages/fe/a4/6079bf9790f16badc01e7b79a28c90bec407cfcaa8a2ed37e4a68120f87a/infinity_sdk-0.6.0.dev5-py3-none-any.whl", hash = "sha256:510ac408d5cd9d3d4df33c7c0877f55c5ae8a6019e465190c86d58012a319179" },
]
[[package]]
@ -5268,7 +5268,7 @@ wheels = [
[[package]]
name = "ragflow"
version = "0.20.2"
version = "0.20.3"
source = { virtual = "." }
dependencies = [
{ name = "akshare" },
@ -5471,7 +5471,7 @@ requires-dist = [
{ name = "httpx", extras = ["socks"], specifier = "==0.27.2" },
{ name = "huggingface-hub", specifier = ">=0.25.0,<0.26.0" },
{ name = "infinity-emb", specifier = ">=0.0.66,<0.0.67" },
{ name = "infinity-sdk", specifier = "==0.6.0.dev4" },
{ name = "infinity-sdk", specifier = "==0.6.0.dev5" },
{ name = "itsdangerous", specifier = "==2.1.2" },
{ name = "json-repair", specifier = "==0.35.0" },
{ name = "langfuse", specifier = ">=2.60.0" },

View File

@ -14,7 +14,7 @@ module.exports = {
'error',
{
'**/*.{jsx,tsx}': 'KEBAB_CASE',
'**/*.{js,ts}': 'KEBAB_CASE',
'**/*.{js,ts}': '[a-z0-9.-]*',
},
],
'check-file/folder-naming-convention': [

View File

@ -2,7 +2,6 @@ import { Toaster as Sonner } from '@/components/ui/sonner';
import { Toaster } from '@/components/ui/toaster';
import i18n from '@/locales/config';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { ReactQueryDevtools } from '@tanstack/react-query-devtools';
import { App, ConfigProvider, ConfigProviderProps, theme } from 'antd';
import pt_BR from 'antd/lib/locale/pt_BR';
import deDE from 'antd/locale/de_DE';
@ -85,7 +84,7 @@ function Root({ children }: React.PropsWithChildren) {
<Sonner position={'top-right'} expand richColors closeButton></Sonner>
<Toaster />
</ConfigProvider>
<ReactQueryDevtools buttonPosition={'top-left'} />
{/* <ReactQueryDevtools buttonPosition={'top-left'} initialIsOpen={false} /> */}
</>
);
}

View File

@ -8,47 +8,93 @@ import {
} from '@/components/ui/dialog';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
import { IModalProps } from '@/interfaces/common';
import { Dispatch, SetStateAction, useCallback, useState } from 'react';
import { zodResolver } from '@hookform/resolvers/zod';
import { TFunction } from 'i18next';
import { useForm } from 'react-hook-form';
import { useTranslation } from 'react-i18next';
import { z } from 'zod';
import { FileUploader } from '../file-uploader';
import { RAGFlowFormItem } from '../ragflow-form';
import { Form } from '../ui/form';
import { Switch } from '../ui/switch';
type UploaderTabsProps = {
setFiles: Dispatch<SetStateAction<File[]>>;
function buildUploadFormSchema(t: TFunction) {
const FormSchema = z.object({
parseOnCreation: z.boolean().optional(),
fileList: z
.array(z.instanceof(File))
.min(1, { message: t('fileManager.pleaseUploadAtLeastOneFile') }),
});
return FormSchema;
}
export type UploadFormSchemaType = z.infer<
ReturnType<typeof buildUploadFormSchema>
>;
const UploadFormId = 'UploadFormId';
type UploadFormProps = {
submit: (values?: UploadFormSchemaType) => void;
showParseOnCreation?: boolean;
};
export function UploaderTabs({ setFiles }: UploaderTabsProps) {
function UploadForm({ submit, showParseOnCreation }: UploadFormProps) {
const { t } = useTranslation();
const FormSchema = buildUploadFormSchema(t);
type UploadFormSchemaType = z.infer<typeof FormSchema>;
const form = useForm<UploadFormSchemaType>({
resolver: zodResolver(FormSchema),
defaultValues: {
parseOnCreation: false,
fileList: [],
},
});
return (
<Tabs defaultValue="account">
<TabsList className="grid w-full grid-cols-2 mb-4">
<TabsTrigger value="account">{t('fileManager.local')}</TabsTrigger>
<TabsTrigger value="password">{t('fileManager.s3')}</TabsTrigger>
</TabsList>
<TabsContent value="account">
<FileUploader
maxFileCount={8}
maxSize={8 * 1024 * 1024}
onValueChange={setFiles}
accept={{ '*': [] }}
/>
</TabsContent>
<TabsContent value="password">{t('common.comingSoon')}</TabsContent>
</Tabs>
<Form {...form}>
<form
onSubmit={form.handleSubmit(submit)}
id={UploadFormId}
className="space-y-4"
>
{showParseOnCreation && (
<RAGFlowFormItem
name="parseOnCreation"
label={t('fileManager.parseOnCreation')}
>
{(field) => (
<Switch
onCheckedChange={field.onChange}
checked={field.value}
></Switch>
)}
</RAGFlowFormItem>
)}
<RAGFlowFormItem name="fileList" label={t('fileManager.file')}>
{(field) => (
<FileUploader
value={field.value}
onValueChange={field.onChange}
accept={{ '*': [] }}
/>
)}
</RAGFlowFormItem>
</form>
</Form>
);
}
type FileUploadDialogProps = IModalProps<UploadFormSchemaType> &
Pick<UploadFormProps, 'showParseOnCreation'>;
export function FileUploadDialog({
hideModal,
onOk,
loading,
}: IModalProps<File[]>) {
showParseOnCreation = false,
}: FileUploadDialogProps) {
const { t } = useTranslation();
const [files, setFiles] = useState<File[]>([]);
const handleOk = useCallback(() => {
onOk?.(files);
}, [files, onOk]);
return (
<Dialog open onOpenChange={hideModal}>
@ -56,9 +102,21 @@ export function FileUploadDialog({
<DialogHeader>
<DialogTitle>{t('fileManager.uploadFile')}</DialogTitle>
</DialogHeader>
<UploaderTabs setFiles={setFiles}></UploaderTabs>
<Tabs defaultValue="account">
<TabsList className="grid w-full grid-cols-2 mb-4">
<TabsTrigger value="account">{t('fileManager.local')}</TabsTrigger>
<TabsTrigger value="password">{t('fileManager.s3')}</TabsTrigger>
</TabsList>
<TabsContent value="account">
<UploadForm
submit={onOk!}
showParseOnCreation={showParseOnCreation}
></UploadForm>
</TabsContent>
<TabsContent value="password">{t('common.comingSoon')}</TabsContent>
</Tabs>
<DialogFooter>
<ButtonLoading type="submit" onClick={handleOk} loading={loading}>
<ButtonLoading type="submit" loading={loading} form={UploadFormId}>
{t('common.save')}
</ButtonLoading>
</DialogFooter>

View File

@ -15,6 +15,7 @@ import { Progress } from '@/components/ui/progress';
import { ScrollArea } from '@/components/ui/scroll-area';
import { useControllableState } from '@/hooks/use-controllable-state';
import { cn, formatBytes } from '@/lib/utils';
import { useTranslation } from 'react-i18next';
function isFileWithPreview(file: File): file is File & { preview: string } {
return 'preview' in file && typeof file.preview === 'string';
@ -168,14 +169,14 @@ export function FileUploader(props: FileUploaderProps) {
accept = {
'image/*': [],
},
maxSize = 1024 * 1024 * 2,
maxFileCount = 1,
maxSize = 1024 * 1024 * 10000000,
maxFileCount = 100000000000,
multiple = false,
disabled = false,
className,
...dropzoneProps
} = props;
const { t } = useTranslation();
const [files, setFiles] = useControllableState({
prop: valueProp,
onChange: onValueChange,
@ -267,7 +268,7 @@ export function FileUploader(props: FileUploaderProps) {
<div
{...getRootProps()}
className={cn(
'group relative grid h-52 w-full cursor-pointer place-items-center rounded-lg border-2 border-dashed border-muted-foreground/25 px-5 py-2.5 text-center transition hover:bg-muted/25',
'group relative grid h-72 w-full cursor-pointer place-items-center rounded-lg border-2 border-dashed border-muted-foreground/25 px-5 py-2.5 text-center transition hover:bg-muted/25',
'ring-offset-background focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2',
isDragActive && 'border-muted-foreground/50',
isDisabled && 'pointer-events-none opacity-60',
@ -298,14 +299,15 @@ export function FileUploader(props: FileUploaderProps) {
</div>
<div className="flex flex-col gap-px">
<p className="font-medium text-muted-foreground">
Drag {`'n'`} drop files here, or click to select files
{t('knowledgeDetails.uploadTitle')}
</p>
<p className="text-sm text-muted-foreground/70">
You can upload
{t('knowledgeDetails.uploadDescription')}
{/* You can upload
{maxFileCount > 1
? ` ${maxFileCount === Infinity ? 'multiple' : maxFileCount}
files (up to ${formatBytes(maxSize)} each)`
: ` a file with ${formatBytes(maxSize)}`}
: ` a file with ${formatBytes(maxSize)}`} */}
</p>
</div>
</div>

View File

@ -1,6 +1,7 @@
import { RAGFlowAvatar } from '@/components/ragflow-avatar';
import { Card, CardContent } from '@/components/ui/card';
import { formatDate } from '@/utils/date';
import { ReactNode } from 'react';
interface IProps {
data: {
@ -11,8 +12,9 @@ interface IProps {
};
onClick?: () => void;
moreDropdown: React.ReactNode;
sharedBadge?: ReactNode;
}
export function HomeCard({ data, onClick, moreDropdown }: IProps) {
export function HomeCard({ data, onClick, moreDropdown, sharedBadge }: IProps) {
return (
<Card
className="bg-bg-card border-colors-outline-neutral-standard"
@ -31,7 +33,7 @@ export function HomeCard({ data, onClick, moreDropdown }: IProps) {
</div>
<div className="flex flex-col justify-between gap-1 flex-1 h-full w-[calc(100%-50px)]">
<section className="flex justify-between">
<div className="text-[20px] font-bold w-80% leading-5">
<div className="text-[20px] font-bold w-80% leading-5 text-ellipsis overflow-hidden">
{data.name}
</div>
{moreDropdown}
@ -41,10 +43,11 @@ export function HomeCard({ data, onClick, moreDropdown }: IProps) {
<div className="whitespace-nowrap overflow-hidden text-ellipsis">
{data.description}
</div>
<div>
<div className="flex justify-between items-center">
<p className="text-sm opacity-80">
{formatDate(data.update_time)}
</p>
{sharedBadge}
</div>
</section>
</div>

View File

@ -68,7 +68,7 @@ export function LayoutRecognizeFormField() {
<div className="flex items-center">
<FormLabel
tooltip={t('layoutRecognizeTip')}
className="text-sm text-muted-foreground whitespace-nowrap w-1/4"
className="text-sm text-muted-foreground whitespace-wrap w-1/4"
>
{t('layoutRecognize')}
</FormLabel>

View File

@ -1,5 +1,5 @@
import { cn } from '@/lib/utils';
import { ChevronDown } from 'lucide-react';
import { Funnel } from 'lucide-react';
import React, {
ChangeEventHandler,
PropsWithChildren,
@ -25,20 +25,20 @@ export const FilterButton = React.forwardRef<
>(({ count = 0, ...props }, ref) => {
return (
<Button variant="secondary" {...props} ref={ref}>
<span
{/* <span
className={cn({
'text-text-primary': count > 0,
'text-text-sub-title-invert': count === 0,
})}
>
Filter
</span>
</span> */}
{count > 0 && (
<span className="rounded-full bg-text-badge px-1 text-xs ">
{count}
</span>
)}
<ChevronDown />
<Funnel />
</Button>
);
});

View File

@ -2,6 +2,7 @@ import { LlmModelType } from '@/constants/knowledge';
import { useComposeLlmOptionsByModelTypes } from '@/hooks/llm-hooks';
import * as SelectPrimitive from '@radix-ui/react-select';
import { forwardRef, memo, useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { LlmSettingFieldItems } from '../llm-setting-items/next';
import { Popover, PopoverContent, PopoverTrigger } from '../ui/popover';
import { Select, SelectTrigger, SelectValue } from '../ui/select';
@ -20,6 +21,7 @@ const NextInnerLLMSelect = forwardRef<
React.ElementRef<typeof SelectPrimitive.Trigger>,
NextInnerLLMSelectProps
>(({ value, disabled, filter, showSpeech2TextModel = false }, ref) => {
const { t } = useTranslation();
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
const ttsModel = useMemo(() => {
@ -49,7 +51,7 @@ const NextInnerLLMSelect = forwardRef<
}}
ref={ref}
>
<SelectValue>
<SelectValue placeholder={t('common.pleaseSelect')}>
{
modelOptions
.flatMap((x) => x.options)

View File

@ -19,6 +19,7 @@ type SliderInputSwitchFormFieldProps = {
name: string;
label: string;
defaultValue?: number;
onChange?: (value: number) => void;
className?: string;
checkName: string;
};
@ -30,6 +31,7 @@ export function SliderInputSwitchFormField({
label,
name,
defaultValue,
onChange,
className,
checkName,
}: SliderInputSwitchFormFieldProps) {
@ -66,6 +68,10 @@ export function SliderInputSwitchFormField({
<FormControl>
<SingleFormSlider
{...field}
onChange={(value: number) => {
onChange?.(value);
field.onChange(value);
}}
max={max}
min={min}
step={step}
@ -80,6 +86,10 @@ export function SliderInputSwitchFormField({
min={min}
step={step}
{...field}
onChange={(value: number) => {
onChange?.(value);
field.onChange(value);
}}
></NumberInput>
</FormControl>
</div>

View File

@ -58,7 +58,10 @@ export function MetadataFilter({ prefix = '' }: MetadataFilterProps) {
name={methodName}
tooltip={t('metadataTip')}
>
<SelectWithSearch options={MetadataOptions} />
<SelectWithSearch
options={MetadataOptions}
triggerClassName="!bg-bg-input"
/>
</RAGFlowFormItem>
)}
{hasKnowledge && metadata === DatasetMetadata.Manual && (

View File

@ -28,6 +28,7 @@ import {
PopoverTrigger,
} from '@/components/ui/popover';
import { cn } from '@/lib/utils';
import { t } from 'i18next';
import { RAGFlowSelectOptionType } from '../ui/select';
import { Separator } from '../ui/separator';
@ -114,7 +115,9 @@ export const SelectWithSearch = forwardRef<
<span className="leading-none truncate">{selectLabel}</span>
</span>
) : (
<span className="text-muted-foreground">Select value</span>
<span className="text-muted-foreground">
{t('common.selectPlaceholder')}
</span>
)}
<div className="flex items-center justify-between">
{value && allowClear && (

View File

@ -3,7 +3,7 @@ import { DocumentParserType } from '@/constants/knowledge';
import { useTranslate } from '@/hooks/common-hooks';
import random from 'lodash/random';
import { Plus } from 'lucide-react';
import { useCallback, useEffect } from 'react';
import { useCallback } from 'react';
import { useFormContext, useWatch } from 'react-hook-form';
import { SliderInputFormField } from '../slider-input-form-field';
import { Button } from '../ui/button';
@ -57,15 +57,19 @@ const RaptorFormFields = () => {
const form = useFormContext();
const { t } = useTranslate('knowledgeConfiguration');
const useRaptor = useWatch({ name: UseRaptorField });
useEffect(() => {
if (useRaptor) {
form.setValue(MaxTokenField, 256);
form.setValue(ThresholdField, 0.1);
form.setValue(MaxCluster, 64);
form.setValue(RandomSeedField, 0);
form.setValue(Prompt, t('promptText'));
}
}, [form, useRaptor, t]);
const changeRaptor = useCallback(
(isUseRaptor: boolean) => {
if (isUseRaptor) {
form.setValue(MaxTokenField, 256);
form.setValue(ThresholdField, 0.1);
form.setValue(MaxCluster, 64);
form.setValue(RandomSeedField, 0);
form.setValue(Prompt, t('promptText'));
}
},
[form],
);
const handleGenerate = useCallback(() => {
form.setValue(RandomSeedField, random(10000));
@ -97,7 +101,10 @@ const RaptorFormFields = () => {
<FormControl>
<Switch
checked={field.value}
onCheckedChange={field.onChange}
onCheckedChange={(e) => {
changeRaptor(e);
field.onChange(e);
}}
></Switch>
</FormControl>
</div>
@ -127,7 +134,13 @@ const RaptorFormFields = () => {
</FormLabel>
<div className="w-3/4">
<FormControl>
<Textarea {...field} rows={8} />
<Textarea
{...field}
rows={8}
onChange={(e) => {
field.onChange(e?.target?.value);
}}
/>
</FormControl>
</div>
</div>

View File

@ -5,6 +5,7 @@ import {
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { cn } from '@/lib/utils';
import { ReactNode, cloneElement, isValidElement } from 'react';
import { ControllerRenderProps, useFormContext } from 'react-hook-form';
@ -13,6 +14,7 @@ type RAGFlowFormItemProps = {
label: ReactNode;
tooltip?: ReactNode;
children: ReactNode | ((field: ControllerRenderProps) => ReactNode);
horizontal?: boolean;
};
export function RAGFlowFormItem({
@ -20,6 +22,7 @@ export function RAGFlowFormItem({
label,
tooltip,
children,
horizontal = false,
}: RAGFlowFormItemProps) {
const form = useFormContext();
return (
@ -27,8 +30,14 @@ export function RAGFlowFormItem({
control={form.control}
name={name}
render={({ field }) => (
<FormItem>
<FormLabel tooltip={tooltip}>{label}</FormLabel>
<FormItem
className={cn({
'flex items-center': horizontal,
})}
>
<FormLabel tooltip={tooltip} className={cn({ 'w-1/4': horizontal })}>
{label}
</FormLabel>
<FormControl>
{typeof children === 'function'
? children(field)

View File

@ -8,9 +8,5 @@ export function SharedBadge({ children }: PropsWithChildren) {
return null;
}
return (
<span className="bg-text-secondary rounded-sm px-1 text-bg-base text-xs">
{children}
</span>
);
return <span className="bg-bg-card rounded-sm px-1 text-xs">{children}</span>;
}

View File

@ -17,7 +17,7 @@ const buttonVariants = cva(
outline:
'border bg-background shadow-xs hover:bg-accent hover:text-accent-foreground dark:bg-input/30 dark:border-input dark:hover:bg-input/50',
secondary:
'bg-secondary text-secondary-foreground shadow-xs hover:bg-secondary/80',
'bg-bg-input text-secondary-foreground shadow-xs hover:bg-bg-input/80',
ghost:
'hover:bg-accent hover:text-accent-foreground dark:hover:bg-accent/50',
link: 'text-primary underline-offset-4 hover:underline',

View File

@ -116,7 +116,10 @@ export { ExpandedInput, Input, SearchInput };
type NumberInputProps = { onChange?(value: number): void } & InputProps;
export const NumberInput = ({ onChange, ...props }: NumberInputProps) => {
export const NumberInput = React.forwardRef<
HTMLInputElement,
NumberInputProps & { value: Value; onChange(value: Value): void }
>(function NumberInput({ onChange, ...props }, ref) {
return (
<Input
type="number"
@ -125,6 +128,7 @@ export const NumberInput = ({ onChange, ...props }: NumberInputProps) => {
onChange?.(value === '' ? 0 : Number(value)); // convert to number
}}
{...props}
ref={ref}
></Input>
);
};
});

View File

@ -209,8 +209,16 @@ export const MultiSelect = React.forwardRef<
const [isAnimating, setIsAnimating] = React.useState(false);
React.useEffect(() => {
setSelectedValues(defaultValue);
}, [defaultValue]);
if (!selectedValues && props.value) {
setSelectedValues(props.value as string[]);
}
}, [props.value, selectedValues]);
React.useEffect(() => {
if (!selectedValues && !props.value && defaultValue) {
setSelectedValues(defaultValue);
}
}, [defaultValue, props.value, selectedValues]);
const flatOptions = React.useMemo(() => {
return options.flatMap((option) =>
@ -291,15 +299,18 @@ export const MultiSelect = React.forwardRef<
variant="secondary"
className={cn(
isAnimating ? 'animate-bounce' : '',
'px-1',
multiSelectVariants({ variant }),
)}
style={{ animationDuration: `${animation}s` }}
>
<div className="flex items-center gap-1">
<div className="flex justify-between items-center gap-1">
{IconComponent && (
<IconComponent className="h-4 w-4" />
)}
<div>{option?.label}</div>
<div className="max-w-28 text-ellipsis overflow-hidden">
{option?.label}
</div>
<XCircle
className="h-4 w-4 cursor-pointer"
onClick={(event) => {

View File

@ -12,13 +12,13 @@ const Progress = React.forwardRef<
<ProgressPrimitive.Root
ref={ref}
className={cn(
'relative h-4 w-full overflow-hidden rounded-full bg-secondary',
'relative h-4 w-full overflow-hidden rounded-full bg-bg-accent',
className,
)}
{...props}
>
<ProgressPrimitive.Indicator
className="h-full w-full flex-1 bg-primary transition-all"
className="h-full w-full flex-1 bg-accent-primary transition-all"
style={{ transform: `translateX(-${100 - (value || 0)}%)` }}
/>
</ProgressPrimitive.Root>

View File

@ -23,6 +23,7 @@ export interface SegmentedProps
prefixCls?: string;
direction?: 'ltr' | 'rtl';
motionName?: string;
activeClassName?: string;
}
export function Segmented({
@ -30,6 +31,7 @@ export function Segmented({
value,
onChange,
className,
activeClassName,
}: SegmentedProps) {
const [selectedValue, setSelectedValue] = React.useState<
SegmentedValue | undefined
@ -57,9 +59,12 @@ export function Segmented({
className={cn(
'inline-flex items-center px-6 py-2 text-base font-normal rounded-3xl cursor-pointer',
{
'bg-text-primary': selectedValue === actualValue,
'text-bg-base': selectedValue === actualValue,
'text-bg-base bg-metallic-gradient border-b-[#00BEB4] border-b-2':
selectedValue === actualValue,
},
activeClassName && selectedValue === actualValue
? activeClassName
: '',
)}
onClick={() => handleOnChange(actualValue)}
>

View File

@ -54,7 +54,7 @@ const Textarea = forwardRef<HTMLTextAreaElement, TextareaProps>(
return (
<textarea
className={cn(
'flex min-h-[80px] w-full bg-bg-card rounded-md border border-input px-3 py-2 text-base ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 md:text-sm overflow-hidden',
'flex min-h-[80px] w-full bg-bg-input rounded-md border border-input px-3 py-2 text-base ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 md:text-sm overflow-hidden',
className,
)}
rows={autoSize?.minRows ?? props.rows ?? undefined}

View File

@ -0,0 +1,4 @@
export enum PermissionRole {
Me = 'me',
Team = 'team',
}

View File

@ -77,7 +77,7 @@ export const useNavigatePage = () => {
}, [navigate]);
const navigateToSearch = useCallback(
(id: string) => {
(id: string) => () => {
navigate(`${Routes.Search}/${id}`);
},
[navigate],

View File

@ -1,4 +1,5 @@
import { useHandleFilterSubmit } from '@/components/list-filter-bar/use-handle-filter-submit';
import { ResponseType } from '@/interfaces/database/base';
import {
IDocumentInfo,
IDocumentInfoFilter,
@ -45,9 +46,9 @@ export const useUploadNextDocument = () => {
data,
isPending: loading,
mutateAsync,
} = useMutation({
} = useMutation<ResponseType<IDocumentInfo[]>, Error, File[]>({
mutationKey: [DocumentApiAction.UploadDocument],
mutationFn: async (fileList: File[]) => {
mutationFn: async (fileList) => {
const formData = new FormData();
formData.append('kb_id', id!);
fileList.forEach((file: any) => {

View File

@ -8,9 +8,13 @@ import {
} from '@/interfaces/database/llm';
import { buildLlmUuid } from '@/utils/llm-util';
export const enum LLMApiAction {
LlmList = 'llmList',
}
export const useFetchLlmList = (modelType?: LlmModelType) => {
const { data } = useQuery<IThirdAiModelCollection>({
queryKey: ['llmList'],
queryKey: [LLMApiAction.LlmList],
initialData: {},
queryFn: async () => {
const { data } = await userService.llm_list({ model_type: modelType });

View File

@ -0,0 +1,464 @@
import message from '@/components/ui/message';
import { LanguageTranslationMap } from '@/constants/common';
import { ResponseGetType } from '@/interfaces/database/base';
import { IToken } from '@/interfaces/database/chat';
import { ITenantInfo } from '@/interfaces/database/knowledge';
import { ILangfuseConfig } from '@/interfaces/database/system';
import {
ISystemStatus,
ITenant,
ITenantUser,
IUserInfo,
} from '@/interfaces/database/user-setting';
import { ISetLangfuseConfigRequestBody } from '@/interfaces/request/system';
import userService, {
addTenantUser,
agreeTenant,
deleteTenantUser,
listTenant,
listTenantUser,
} from '@/services/user-service';
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { Modal } from 'antd';
import DOMPurify from 'dompurify';
import { isEmpty } from 'lodash';
import { useCallback, useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { history } from 'umi';
export const enum UserSettingApiAction {
UserInfo = 'userInfo',
TenantInfo = 'tenantInfo',
SaveSetting = 'saveSetting',
FetchManualSystemTokenList = 'fetchManualSystemTokenList',
FetchSystemTokenList = 'fetchSystemTokenList',
RemoveSystemToken = 'removeSystemToken',
CreateSystemToken = 'createSystemToken',
ListTenantUser = 'listTenantUser',
AddTenantUser = 'addTenantUser',
DeleteTenantUser = 'deleteTenantUser',
ListTenant = 'listTenant',
AgreeTenant = 'agreeTenant',
SetLangfuseConfig = 'setLangfuseConfig',
DeleteLangfuseConfig = 'deleteLangfuseConfig',
FetchLangfuseConfig = 'fetchLangfuseConfig',
}
export const useFetchUserInfo = (): ResponseGetType<IUserInfo> => {
const { i18n } = useTranslation();
const { data, isFetching: loading } = useQuery({
queryKey: [UserSettingApiAction.UserInfo],
initialData: {},
gcTime: 0,
queryFn: async () => {
const { data } = await userService.user_info();
if (data.code === 0) {
i18n.changeLanguage(
LanguageTranslationMap[
data.data.language as keyof typeof LanguageTranslationMap
],
);
}
return data?.data ?? {};
},
});
return { data, loading };
};
export const useFetchTenantInfo = (
showEmptyModelWarn = false,
): ResponseGetType<ITenantInfo> => {
const { t } = useTranslation();
const { data, isFetching: loading } = useQuery({
queryKey: [UserSettingApiAction.TenantInfo],
initialData: {},
gcTime: 0,
queryFn: async () => {
const { data: res } = await userService.get_tenant_info();
if (res.code === 0) {
// llm_id is chat_id
// asr_id is speech2txt
const { data } = res;
if (
showEmptyModelWarn &&
(isEmpty(data.embd_id) || isEmpty(data.llm_id))
) {
Modal.warning({
title: t('common.warn'),
content: (
<div
dangerouslySetInnerHTML={{
__html: DOMPurify.sanitize(t('setting.modelProvidersWarn')),
}}
></div>
),
onOk() {
history.push('/user-setting/model');
},
});
}
data.chat_id = data.llm_id;
data.speech2text_id = data.asr_id;
return data;
}
return res;
},
});
return { data, loading };
};
export const useSelectParserList = (): Array<{
value: string;
label: string;
}> => {
const { data: tenantInfo } = useFetchTenantInfo(true);
const parserList = useMemo(() => {
const parserArray: Array<string> = tenantInfo?.parser_ids?.split(',') ?? [];
return parserArray.map((x) => {
const arr = x.split(':');
return { value: arr[0], label: arr[1] };
});
}, [tenantInfo]);
return parserList;
};
export const useSaveSetting = () => {
const queryClient = useQueryClient();
const { t } = useTranslation();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.SaveSetting],
mutationFn: async (
userInfo: { new_password: string } | Partial<IUserInfo>,
) => {
const { data } = await userService.setting(userInfo);
if (data.code === 0) {
message.success(t('message.modified'));
queryClient.invalidateQueries({ queryKey: ['userInfo'] });
}
return data?.code;
},
});
return { data, loading, saveSetting: mutateAsync };
};
export const useFetchSystemVersion = () => {
const [version, setVersion] = useState('');
const [loading, setLoading] = useState(false);
const fetchSystemVersion = useCallback(async () => {
try {
setLoading(true);
const { data } = await userService.getSystemVersion();
if (data.code === 0) {
setVersion(data.data);
setLoading(false);
}
} catch (error) {
setLoading(false);
}
}, []);
return { fetchSystemVersion, version, loading };
};
export const useFetchSystemStatus = () => {
const [systemStatus, setSystemStatus] = useState<ISystemStatus>(
{} as ISystemStatus,
);
const [loading, setLoading] = useState(false);
const fetchSystemStatus = useCallback(async () => {
setLoading(true);
const { data } = await userService.getSystemStatus();
if (data.code === 0) {
setSystemStatus(data.data);
setLoading(false);
}
}, []);
return {
systemStatus,
fetchSystemStatus,
loading,
};
};
export const useFetchManualSystemTokenList = () => {
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.FetchManualSystemTokenList],
mutationFn: async () => {
const { data } = await userService.listToken();
return data?.data ?? [];
},
});
return { data, loading, fetchSystemTokenList: mutateAsync };
};
export const useFetchSystemTokenList = () => {
const {
data,
isFetching: loading,
refetch,
} = useQuery<IToken[]>({
queryKey: [UserSettingApiAction.FetchSystemTokenList],
initialData: [],
gcTime: 0,
queryFn: async () => {
const { data } = await userService.listToken();
return data?.data ?? [];
},
});
return { data, loading, refetch };
};
export const useRemoveSystemToken = () => {
const queryClient = useQueryClient();
const { t } = useTranslation();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.RemoveSystemToken],
mutationFn: async (token: string) => {
const { data } = await userService.removeToken({}, token);
if (data.code === 0) {
message.success(t('message.deleted'));
queryClient.invalidateQueries({
queryKey: [UserSettingApiAction.FetchSystemTokenList],
});
}
return data?.data ?? [];
},
});
return { data, loading, removeToken: mutateAsync };
};
export const useCreateSystemToken = () => {
const queryClient = useQueryClient();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.CreateSystemToken],
mutationFn: async (params: Record<string, any>) => {
const { data } = await userService.createToken(params);
if (data.code === 0) {
queryClient.invalidateQueries({
queryKey: [UserSettingApiAction.FetchSystemTokenList],
});
}
return data?.data ?? [];
},
});
return { data, loading, createToken: mutateAsync };
};
export const useListTenantUser = () => {
const { data: tenantInfo } = useFetchTenantInfo();
const tenantId = tenantInfo.tenant_id;
const {
data,
isFetching: loading,
refetch,
} = useQuery<ITenantUser[]>({
queryKey: [UserSettingApiAction.ListTenantUser, tenantId],
initialData: [],
gcTime: 0,
enabled: !!tenantId,
queryFn: async () => {
const { data } = await listTenantUser(tenantId);
return data?.data ?? [];
},
});
return { data, loading, refetch };
};
export const useAddTenantUser = () => {
const { data: tenantInfo } = useFetchTenantInfo();
const queryClient = useQueryClient();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.AddTenantUser],
mutationFn: async (email: string) => {
const { data } = await addTenantUser(tenantInfo.tenant_id, email);
if (data.code === 0) {
queryClient.invalidateQueries({
queryKey: [UserSettingApiAction.ListTenantUser],
});
}
return data?.code;
},
});
return { data, loading, addTenantUser: mutateAsync };
};
export const useDeleteTenantUser = () => {
const { data: tenantInfo } = useFetchTenantInfo();
const queryClient = useQueryClient();
const { t } = useTranslation();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.DeleteTenantUser],
mutationFn: async ({
userId,
tenantId,
}: {
userId: string;
tenantId?: string;
}) => {
const { data } = await deleteTenantUser({
tenantId: tenantId ?? tenantInfo.tenant_id,
userId,
});
if (data.code === 0) {
message.success(t('message.deleted'));
queryClient.invalidateQueries({
queryKey: [UserSettingApiAction.ListTenantUser],
});
queryClient.invalidateQueries({
queryKey: [UserSettingApiAction.ListTenant],
});
}
return data?.data ?? [];
},
});
return { data, loading, deleteTenantUser: mutateAsync };
};
export const useListTenant = () => {
const { data: tenantInfo } = useFetchTenantInfo();
const tenantId = tenantInfo.tenant_id;
const {
data,
isFetching: loading,
refetch,
} = useQuery<ITenant[]>({
queryKey: [UserSettingApiAction.ListTenant, tenantId],
initialData: [],
gcTime: 0,
enabled: !!tenantId,
queryFn: async () => {
const { data } = await listTenant();
return data?.data ?? [];
},
});
return { data, loading, refetch };
};
export const useAgreeTenant = () => {
const queryClient = useQueryClient();
const { t } = useTranslation();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.AgreeTenant],
mutationFn: async (tenantId: string) => {
const { data } = await agreeTenant(tenantId);
if (data.code === 0) {
message.success(t('message.operated'));
queryClient.invalidateQueries({
queryKey: [UserSettingApiAction.ListTenant],
});
}
return data?.data ?? [];
},
});
return { data, loading, agreeTenant: mutateAsync };
};
export const useSetLangfuseConfig = () => {
const { t } = useTranslation();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.SetLangfuseConfig],
mutationFn: async (params: ISetLangfuseConfigRequestBody) => {
const { data } = await userService.setLangfuseConfig(params);
if (data.code === 0) {
message.success(t('message.operated'));
}
return data?.code;
},
});
return { data, loading, setLangfuseConfig: mutateAsync };
};
export const useDeleteLangfuseConfig = () => {
const { t } = useTranslation();
const {
data,
isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: [UserSettingApiAction.DeleteLangfuseConfig],
mutationFn: async () => {
const { data } = await userService.deleteLangfuseConfig();
if (data.code === 0) {
message.success(t('message.deleted'));
}
return data?.code;
},
});
return { data, loading, deleteLangfuseConfig: mutateAsync };
};
export const useFetchLangfuseConfig = () => {
const { data, isFetching: loading } = useQuery<ILangfuseConfig>({
queryKey: [UserSettingApiAction.FetchLangfuseConfig],
gcTime: 0,
queryFn: async () => {
const { data } = await userService.getLangfuseConfig();
return data?.data;
},
});
return { data, loading };
};

View File

@ -0,0 +1,28 @@
import { Button } from '@/components/ui/button';
import { useNavigateWithFromState } from '@/hooks/route-hook';
import { useListTenant } from '@/hooks/use-user-setting-request';
import { TenantRole } from '@/pages/user-setting/constants';
import { BellRing } from 'lucide-react';
import { useCallback, useMemo } from 'react';
export function BellButton() {
const { data } = useListTenant();
const navigate = useNavigateWithFromState();
const showBell = useMemo(() => {
return data.some((x) => x.role === TenantRole.Invite);
}, [data]);
const handleBellClick = useCallback(() => {
navigate('/user-setting/team');
}, [navigate]);
return showBell ? (
<Button variant={'ghost'} onClick={handleBellClick}>
<div className="relative">
<BellRing className="size-4 " />
<span className="absolute size-1 rounded -right-1 -top-1 bg-red-600"></span>
</div>
</Button>
) : null;
}

View File

@ -31,6 +31,7 @@ import {
import React, { useCallback, useMemo } from 'react';
import { useTranslation } from 'react-i18next';
import { useLocation } from 'umi';
import { BellButton } from './bell-button';
const handleDocHelpCLick = () => {
window.open('https://ragflow.io/docs/dev/category/guides', 'target');
@ -53,12 +54,6 @@ export function Header() {
changeLanguage(key);
};
// const { data } = useListTenant();
// const showBell = useMemo(() => {
// return data.some((x) => x.role === TenantRole.Invite);
// }, [data]);
const items = LanguageList.map((x) => ({
key: x,
label: <span>{LanguageMap[x as keyof typeof LanguageMap]}</span>,
@ -68,10 +63,6 @@ export function Header() {
setTheme(theme === ThemeEnum.Dark ? ThemeEnum.Light : ThemeEnum.Dark);
}, [setTheme, theme]);
// const handleBellClick = useCallback(() => {
// navigate('/user-setting/team');
// }, [navigate]);
const tagsData = useMemo(
() => [
{ path: Routes.Root, name: t('header.Root'), icon: House },
@ -160,6 +151,7 @@ export function Header() {
<Button variant={'ghost'} onClick={onThemeClick}>
{theme === 'light' ? <Sun /> : <Moon />}
</Button>
<BellButton></BellButton>
<div className="relative">
<RAGFlowAvatar
name={nickname}

View File

@ -1,6 +1,7 @@
export default {
translation: {
common: {
selectPlaceholder: 'select value',
delete: 'Delete',
deleteModalTitle: 'Are you sure to delete this item?',
ok: 'Yes',
@ -70,7 +71,7 @@ export default {
review: 'from 500+ reviews',
},
header: {
knowledgeBase: 'Knowledge Base',
knowledgeBase: 'Dataset',
chat: 'Chat',
register: 'Register',
signin: 'Sign in',
@ -86,7 +87,7 @@ export default {
knowledgeList: {
welcome: 'Welcome back',
description: 'Which knowledge bases will you use today?',
createKnowledgeBase: 'Create knowledge base',
createKnowledgeBase: 'Create Dataset',
name: 'Name',
namePlaceholder: 'Please input name!',
doc: 'Docs',
@ -94,6 +95,16 @@ export default {
noMoreData: `That's all. Nothing more.`,
},
knowledgeDetails: {
created: 'Created',
learnMore: 'Learn More',
general: 'General',
chunkMethodTab: 'Chunk Method',
testResults: 'Test Results',
testSetting: 'Test Setting',
retrievalTesting: 'Retrieval Testing',
retrievalTestingDescription:
'Conduct a retrieval test to check if RAGFlow can recover the intended content for the LLM.',
Parse: 'Parse',
dataset: 'Dataset',
testing: 'Retrieval testing',
files: 'files',
@ -479,6 +490,7 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
improvise: 'Improvise',
precise: 'Precise',
balance: 'Balance',
custom: 'Custom',
freedomTip: `A shortcut to 'Temperature', 'Top P', 'Presence penalty', and 'Frequency penalty' settings, indicating the freedom level of the model. This parameter has three options: Select 'Improvise' to produce more creative responses; select 'Precise' (default) to produce more conservative responses; 'Balance' is a middle ground between 'Improvise' and 'Precise'.`,
temperature: 'Temperature',
temperatureMessage: 'Temperature is required',
@ -845,6 +857,7 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
uploadLimit:
'Each file must not exceed 10MB, and the total number of files must not exceed 128.',
destinationFolder: 'Destination folder',
pleaseUploadAtLeastOneFile: 'Please upload at least one file',
},
flow: {
cite: 'Cite',
@ -1441,6 +1454,7 @@ This delimiter is used to split the input text into several text pieces echo of
showQueryMindmap: 'Show Query Mindmap',
embedApp: 'Embed App',
relatedSearch: 'Related Search',
descriptionValue: 'You are an intelligent assistant.',
okText: 'Save',
cancelText: 'Cancel',
},

View File

@ -240,9 +240,8 @@ export default {
promptTip:
'Décrivez la tâche attendue du LLM, ses réponses, ses exigences, etc. Utilisez `/` pour afficher les variables disponibles.',
promptMessage: 'Le prompt est requis',
promptText: `Veuillez résumer les paragraphes suivants. Attention aux chiffres, ne pas inventer. Paragraphes suivants : {cluster_content
}
Le contenu à résumer est ci-dessus.`,
promptText: `Veuillez résumer les paragraphes suivants. Attention aux chiffres, ne pas inventer. Paragraphes suivants : {cluster_content}
Le contenu à résumer est ci-dessus.`,
maxToken: 'Nombre maximal de tokens',
maxTokenTip: 'Nombre maximal de tokens générés par résumé.',
maxTokenMessage: 'Nombre maximal de tokens requis',

View File

@ -454,6 +454,7 @@ export default {
improvise: '即興創作',
precise: '精確',
balance: '平衡',
custom: '自定義',
freedomTip: `“精確”意味著法學碩士會保守並謹慎地回答你的問題。“即興發揮”意味著你希望法學碩士能夠自由地暢所欲言。“平衡”是謹慎與自由之間的平衡。`,
temperature: '溫度',
temperatureMessage: '溫度是必填項',

View File

@ -1,6 +1,7 @@
export default {
translation: {
common: {
selectPlaceholder: '请选择',
delete: '删除',
deleteModalTitle: '确定删除吗?',
ok: '是',
@ -86,6 +87,16 @@ export default {
noMoreData: '没有更多数据了',
},
knowledgeDetails: {
created: '创建于',
learnMore: '了解更多',
general: '通用',
chunkMethodTab: '切片方法',
testResults: '测试结果',
testSetting: '测试设置',
retrievalTesting: '知识检索测试',
retrievalTestingDescription:
'进行检索测试,检查 RAGFlow 是否能够为大语言模型LLM恢复预期的内容。',
Parse: '解析',
dataset: '数据集',
testing: '检索测试',
configuration: '配置',
@ -182,8 +193,8 @@ export default {
<b>元数据为:</b><br>
<code>
{
“作者”:“Alex Dowson”,
“日期”:“2024-11-12
"作者": "Alex Dowson",
"日期": "2024-11-12"
}
</code><br>
<b>提示将为:</b><br>
@ -477,6 +488,7 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
improvise: '即兴创作',
precise: '精确',
balance: '平衡',
custom: '自定义',
freedomTip: `“精确”意味着大语言模型会保守并谨慎地回答你的问题。 “即兴发挥”意味着你希望大语言模型能够自由地畅所欲言。 “平衡”是谨慎与自由之间的平衡。`,
temperature: '温度',
temperatureMessage: '温度是必填项',
@ -799,6 +811,7 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
fileError: '文件错误',
uploadLimit: '文件大小不能超过10M文件总数不超过128个',
destinationFolder: '目标文件夹',
pleaseUploadAtLeastOneFile: '请上传至少一个文件',
},
flow: {
flow: '工作流',
@ -1344,6 +1357,7 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
showQueryMindmap: '显示查询思维导图',
embedApp: '嵌入网站',
relatedSearch: '相关搜索',
descriptionValue: '你是一位智能助手。',
okText: '保存',
cancelText: '返回',
},

View File

@ -4,6 +4,7 @@ import {
useHandleMessageInputChange,
useSelectDerivedMessages,
} from '@/hooks/logic-hooks';
import { useFetchAgent } from '@/hooks/use-agent-request';
import {
IEventList,
IInputEvent,
@ -188,11 +189,7 @@ export const useSendAgentMessage = (
return answerList[0]?.message_id;
}, [answerList]);
useEffect(() => {
if (answerList[0]?.session_id) {
setSessionId(answerList[0]?.session_id);
}
}, [answerList]);
const { refetch } = useFetchAgent();
const { findReferenceByMessageId } = useFindMessageReference(answerList);
const prologue = useGetBeginNodePrologue();
@ -250,7 +247,7 @@ export const useSendAgentMessage = (
setValue(message.content);
removeLatestMessage();
} else {
// refetch(); // pull the message list after sending the message successfully
refetch(); // pull the message list after sending the message successfully
}
} catch (error) {
console.log('🚀 ~ useSendAgentMessage ~ error:', error);
@ -258,28 +255,30 @@ export const useSendAgentMessage = (
},
[
agentId,
sessionId,
send,
clearUploadResponseList,
inputs,
beginParams,
uploadResponseList,
sessionId,
send,
clearUploadResponseList,
setValue,
removeLatestMessage,
refetch,
],
);
const sendFormMessage = useCallback(
(body: { id?: string; inputs: Record<string, BeginQuery> }) => {
send({ ...body, session_id: sessionId });
async (body: { id?: string; inputs: Record<string, BeginQuery> }) => {
addNewestOneQuestion({
content: Object.entries(body.inputs)
.map(([key, val]) => `${key}: ${val.value}`)
.join('<br/>'),
role: MessageType.User,
});
await send({ ...body, session_id: sessionId });
refetch();
},
[addNewestOneQuestion, send, sessionId],
[addNewestOneQuestion, refetch, send, sessionId],
);
// reset session
@ -346,6 +345,12 @@ export const useSendAgentMessage = (
}
}, [addEventList, answerList, addEventListFun, messageId]);
useEffect(() => {
if (answerList[0]?.session_id) {
setSessionId(answerList[0]?.session_id);
}
}, [answerList]);
return {
value,
sendLoading: !done,

View File

@ -210,7 +210,9 @@ export default function Agent() {
></EmbedDialog>
)}
{versionDialogVisible && (
<VersionDialog hideModal={hideVersionDialog}></VersionDialog>
<DropdownProvider>
<VersionDialog hideModal={hideVersionDialog}></VersionDialog>
</DropdownProvider>
)}
{settingDialogVisible && (
<SettingDialog hideModal={hideSettingDialog}></SettingDialog>

View File

@ -63,7 +63,6 @@ export function UploadAgentForm({ hideModal, onOk }: IModalProps<any>) {
value={field.value}
onValueChange={field.onChange}
maxFileCount={1}
maxSize={4 * 1024 * 1024}
accept={{ '*.json': [FileMimeType.Json] }}
/>
</FormControl>

View File

@ -20,9 +20,7 @@ export function AgentCard({ data, showAgentRenameModal }: DatasetCardProps) {
<MoreButton></MoreButton>
</AgentDropdown>
}
onClick={() => {
navigateToAgent(data?.id);
}}
onClick={navigateToAgent(data?.id)}
/>
);
}

View File

@ -8,7 +8,7 @@ import classNames from 'classnames';
import { useCallback } from 'react';
import { ISegmentedContentProps } from '../interface';
import { DatasetMetadata } from '../constants';
import { DatasetMetadata } from '@/constants/chat';
import styles from './index.less';
import { MetadataFilterConditions } from './metadata-filter-conditions';

View File

@ -1,7 +1 @@
export const EmptyConversationId = 'empty';
export enum DatasetMetadata {
Disabled = 'disabled',
Automatic = 'automatic',
Manual = 'manual',
}

View File

@ -35,7 +35,7 @@
.documentPreview {
// width: 40%;
height: calc(100vh - 130px);
height: calc(100vh - 180px);
overflow: auto;
}

View File

@ -3,15 +3,15 @@ import { RunningStatus } from '@/constants/knowledge';
export const RunningStatusMap = {
[RunningStatus.UNSTART]: {
label: 'UNSTART',
color: 'cyan',
color: 'var(--accent-primary)',
},
[RunningStatus.RUNNING]: {
label: 'Parsing',
color: 'blue',
color: 'var(--team-member)',
},
[RunningStatus.CANCEL]: { label: 'CANCEL', color: 'orange' },
[RunningStatus.DONE]: { label: 'SUCCESS', color: 'blue' },
[RunningStatus.FAIL]: { label: 'FAIL', color: 'red' },
[RunningStatus.CANCEL]: { label: 'CANCEL', color: 'var(--state-warning)' },
[RunningStatus.DONE]: { label: 'SUCCESS', color: 'var(--state-success)' },
[RunningStatus.FAIL]: { label: 'FAIL', color: 'var(--state-error' },
};
export * from '@/constants/knowledge';

View File

@ -11,7 +11,7 @@ import { IDocumentInfo } from '@/interfaces/database/document';
import { formatFileSize } from '@/utils/common-util';
import { formatDate } from '@/utils/date';
import { downloadDocument } from '@/utils/file-util';
import { ArrowDownToLine, FolderPen, ScrollText, Trash2 } from 'lucide-react';
import { Download, Eye, PenLine, Trash2 } from 'lucide-react';
import { useCallback } from 'react';
import { UseRenameDocumentShowType } from './use-rename-document';
import { isParserRunning } from './utils';
@ -57,12 +57,12 @@ export function DatasetActionCell({
disabled={isRunning}
onClick={handleRename}
>
<FolderPen />
<PenLine />
</Button>
<HoverCard>
<HoverCardTrigger>
<Button variant="ghost" disabled={isRunning} size={'sm'}>
<ScrollText />
<Eye />
</Button>
</HoverCardTrigger>
<HoverCardContent className="w-[40vw] max-h-[40vh] overflow-auto">
@ -93,7 +93,7 @@ export function DatasetActionCell({
disabled={isRunning}
size={'sm'}
>
<ArrowDownToLine />
<Download />
</Button>
)}
<ConfirmDeleteDialog onOk={handleRemove}>

View File

@ -164,7 +164,7 @@ export function DatasetTable({
)}
</TableBody>
</Table>
<div className="flex items-center justify-end py-4">
<div className="flex items-center justify-end py-4 absolute bottom-3 right-3">
<div className="space-x-2">
<RAGFlowPagination
{...pick(pagination, 'current', 'pageSize')}

View File

@ -111,6 +111,7 @@ export default function Dataset() {
hideModal={hideDocumentUploadModal}
onOk={onDocumentUploadOk}
loading={documentUploadLoading}
showParseOnCreation
></FileUploadDialog>
)}
{createVisible && (

View File

@ -17,7 +17,7 @@ function Dot({ run }: { run: RunningStatus }) {
const runningStatus = RunningStatusMap[run];
return (
<span
className={'size-2 inline-block rounded'}
className={'size-1 inline-block rounded'}
style={{ backgroundColor: runningStatus.color }}
></span>
);
@ -89,7 +89,7 @@ export function ParsingCard({ record }: IProps) {
return (
<HoverCard>
<HoverCardTrigger asChild>
<Button variant={'ghost'} size={'sm'}>
<Button variant={'transparent'} className="border-none" size={'sm'}>
<Dot run={record.run}></Dot>
</Button>
</HoverCardTrigger>

View File

@ -14,7 +14,7 @@ import {
import { Progress } from '@/components/ui/progress';
import { Separator } from '@/components/ui/separator';
import { IDocumentInfo } from '@/interfaces/database/document';
import { CircleX, Play, RefreshCw } from 'lucide-react';
import { CircleX, RefreshCw } from 'lucide-react';
import { useCallback } from 'react';
import { useTranslation } from 'react-i18next';
import { RunningStatus } from './constant';
@ -24,11 +24,13 @@ import { useHandleRunDocumentByIds } from './use-run-document';
import { UseSaveMetaShowType } from './use-save-meta';
import { isParserRunning } from './utils';
const IconMap = {
[RunningStatus.UNSTART]: <Play />,
[RunningStatus.RUNNING]: <CircleX />,
[RunningStatus.CANCEL]: <RefreshCw />,
[RunningStatus.DONE]: <RefreshCw />,
[RunningStatus.FAIL]: <RefreshCw />,
[RunningStatus.UNSTART]: (
<div className="w-0 h-0 border-l-[10px] border-l-accent-primary border-t-8 border-r-4 border-b-8 border-transparent"></div>
),
[RunningStatus.RUNNING]: <CircleX size={14} color="var(--state-error)" />,
[RunningStatus.CANCEL]: <RefreshCw size={14} color="var(--accent-primary)" />,
[RunningStatus.DONE]: <RefreshCw size={14} color="var(--accent-primary)" />,
[RunningStatus.FAIL]: <RefreshCw size={14} color="var(--accent-primary)" />,
};
export function ParsingStatusCell({
@ -60,11 +62,11 @@ export function ParsingStatusCell({
}, [record, showSetMetaModal]);
return (
<section className="flex gap-2 items-center">
<div className="w-28 flex items-center justify-between">
<section className="flex gap-8 items-center">
<div className="w-fit flex items-center justify-between">
<DropdownMenu>
<DropdownMenuTrigger asChild>
<Button variant={'ghost'} size={'sm'}>
<Button variant={'transparent'} className="border-none" size={'sm'}>
{parser_id === 'naive' ? 'general' : parser_id}
</Button>
</DropdownMenuTrigger>
@ -77,7 +79,6 @@ export function ParsingStatusCell({
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
<Separator orientation="vertical" className="h-2.5" />
</div>
<ConfirmDeleteDialog
title={t(`knowledgeDetails.redo`, { chunkNum: chunk_num })}
@ -85,17 +86,17 @@ export function ParsingStatusCell({
onOk={handleOperationIconClick(true)}
onCancel={handleOperationIconClick(false)}
>
<Button
variant={'ghost'}
size={'sm'}
<div
className="cursor-pointer flex items-center gap-3"
onClick={
isZeroChunk || isRunning
? handleOperationIconClick(false)
: () => {}
}
>
<Separator orientation="vertical" className="h-2.5" />
{operationIcon}
</Button>
</div>
</ConfirmDeleteDialog>
{isParserRunning(run) ? (
<HoverCard>

View File

@ -65,7 +65,8 @@ export function useDatasetTableColumns({
header: ({ column }) => {
return (
<Button
variant="ghost"
variant="transparent"
className="border-none"
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
>
{t('name')}
@ -103,7 +104,8 @@ export function useDatasetTableColumns({
header: ({ column }) => {
return (
<Button
variant="ghost"
variant="transparent"
className="border-none"
onClick={() => column.toggleSorting(column.getIsSorted() === 'asc')}
>
{t('uploadDate')}
@ -141,7 +143,7 @@ export function useDatasetTableColumns({
},
{
accessorKey: 'run',
header: t('parsingStatus'),
header: t('Parse'),
// meta: { cellClassName: 'min-w-[20vw]' },
cell: ({ row }) => {
return (

Some files were not shown because too many files have changed in this diff Show More