Fix: search list permission (#9767)

### What problem does this PR solve?

Search list permission.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

README.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.4">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -190,7 +190,7 @@ releases! 🌟
 > All Docker images are built for x86 platforms. We don't currently offer Docker images for ARM64.
 > If you are on an ARM64 platform, follow [this guide](https://ragflow.io/docs/dev/build_docker_image) to build a Docker image compatible with your system.
 
-   > The command below downloads the `v0.20.2-slim` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.2-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` for the full edition `v0.20.2`.
+   > The command below downloads the `v0.20.4-slim` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.4-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4` for the full edition `v0.20.4`.
 
    ```bash
    $ cd ragflow/docker
@@ -203,8 +203,8 @@ releases! 🌟
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    |-------------------|-----------------|-----------------------|--------------------------|
-   | v0.20.2           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.2-slim      | &approx;2       | ❌                   | Stable release            |
+   | v0.20.4           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.4-slim      | &approx;2       | ❌                   | Stable release            |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                   | _Unstable_ nightly build  |
 

README_id.md

@@ -22,7 +22,7 @@
         <img alt="Lencana Daring" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.4">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Rilis%20Terbaru" alt="Rilis Terbaru">
@@ -181,7 +181,7 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
 > Semua gambar Docker dibangun untuk platform x86. Saat ini, kami tidak menawarkan gambar Docker untuk ARM64.
 > Jika Anda menggunakan platform ARM64, [silakan gunakan panduan ini untuk membangun gambar Docker yang kompatibel dengan sistem Anda](https://ragflow.io/docs/dev/build_docker_image).
 
-> Perintah di bawah ini mengunduh edisi v0.20.2-slim dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.20.2-slim, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server. Misalnya, atur RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2 untuk edisi lengkap v0.20.2.
+> Perintah di bawah ini mengunduh edisi v0.20.4-slim dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.20.4-slim, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server. Misalnya, atur RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4 untuk edisi lengkap v0.20.4.
 
 ```bash
 $ cd ragflow/docker
@@ -194,8 +194,8 @@ $ docker compose -f docker-compose.yml up -d
 
 | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
 | ----------------- | --------------- | --------------------- | ------------------------ |
-| v0.20.2           | &approx;9       | :heavy_check_mark:    | Stable release           |
-| v0.20.2-slim      | &approx;2       | ❌                    | Stable release           |
+| v0.20.4           | &approx;9       | :heavy_check_mark:    | Stable release           |
+| v0.20.4-slim      | &approx;2       | ❌                    | Stable release           |
 | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
 | nightly-slim      | &approx;2       | ❌                    | _Unstable_ nightly build |
 

README_ja.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.4">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -160,7 +160,7 @@
 > 現在、公式に提供されているすべての Docker イメージは x86 アーキテクチャ向けにビルドされており、ARM64 用の Docker イメージは提供されていません。
 > ARM64 アーキテクチャのオペレーティングシステムを使用している場合は、[このドキュメント](https://ragflow.io/docs/dev/build_docker_image)を参照して Docker イメージを自分でビルドしてください。
 
-   > 以下のコマンドは、RAGFlow Docker イメージの v0.20.2-slim エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.20.2-slim とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。例えば、完全版 v0.20.2 をダウンロードするには、RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2 と設定します。
+   > 以下のコマンドは、RAGFlow Docker イメージの v0.20.4-slim エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.20.4-slim とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。例えば、完全版 v0.20.4 をダウンロードするには、RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4 と設定します。
 
    ```bash
    $ cd ragflow/docker
@@ -173,8 +173,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.2           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.2-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.4           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.4-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 

README_ko.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.4">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -160,7 +160,7 @@
 > 모든 Docker 이미지는 x86 플랫폼을 위해 빌드되었습니다. 우리는 현재 ARM64 플랫폼을 위한 Docker 이미지를 제공하지 않습니다.
 > ARM64 플랫폼을 사용 중이라면, [시스템과 호환되는 Docker 이미지를 빌드하려면 이 가이드를 사용해 주세요](https://ragflow.io/docs/dev/build_docker_image).
 
-   > 아래 명령어는 RAGFlow Docker 이미지의 v0.20.2-slim 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.20.2-slim과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오. 예를 들어, 전체 버전인 v0.20.2을 다운로드하려면 RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2로 설정합니다.
+   > 아래 명령어는 RAGFlow Docker 이미지의 v0.20.4-slim 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.20.4-slim과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오. 예를 들어, 전체 버전인 v0.20.4을 다운로드하려면 RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4로 설정합니다.
 
    ```bash
    $ cd ragflow/docker
@@ -173,8 +173,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.2           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.2-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.4           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.4-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 

README_pt_br.md

@@ -22,7 +22,7 @@
         <img alt="Badge Estático" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.4">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Última%20Relese" alt="Última Versão">
@@ -180,7 +180,7 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
 > Todas as imagens Docker são construídas para plataformas x86. Atualmente, não oferecemos imagens Docker para ARM64.
 > Se você estiver usando uma plataforma ARM64, por favor, utilize [este guia](https://ragflow.io/docs/dev/build_docker_image) para construir uma imagem Docker compatível com o seu sistema.
 
-    > O comando abaixo baixa a edição `v0.20.2-slim` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.20.2-slim`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor. Por exemplo: defina `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` para a edição completa `v0.20.2`.
+    > O comando abaixo baixa a edição `v0.20.4-slim` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.20.4-slim`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor. Por exemplo: defina `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4` para a edição completa `v0.20.4`.
 
     ```bash
     $ cd ragflow/docker
@@ -193,8 +193,8 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
 
     | Tag da imagem RAGFlow | Tamanho da imagem (GB) | Possui modelos de incorporação? | Estável?                 |
     | --------------------- | ---------------------- | ------------------------------- | ------------------------ |
-    | v0.20.2               | ~9                     | :heavy_check_mark:              | Lançamento estável       |
-    | v0.20.2-slim          | ~2                     | ❌                              | Lançamento estável       |
+    | v0.20.4               | ~9                     | :heavy_check_mark:              | Lançamento estável       |
+    | v0.20.4-slim          | ~2                     | ❌                              | Lançamento estável       |
     | nightly               | ~9                     | :heavy_check_mark:              | _Instável_ build noturno |
     | nightly-slim          | ~2                     | ❌                               | _Instável_ build noturno |
 

README_tzh.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.4">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -183,7 +183,7 @@
 > 所有 Docker 映像檔都是為 x86 平台建置的。目前，我們不提供 ARM64 平台的 Docker 映像檔。
 > 如果您使用的是 ARM64 平台，請使用 [這份指南](https://ragflow.io/docs/dev/build_docker_image) 來建置適合您系統的 Docker 映像檔。
 
-   > 執行以下指令會自動下載 RAGFlow slim Docker 映像 `v0.20.2-slim`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.20.2-slim` 的 Docker 映像，請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。例如，你可以透過設定 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` 來下載 RAGFlow 鏡像的 `v0.20.2` 完整發行版。
+   > 執行以下指令會自動下載 RAGFlow slim Docker 映像 `v0.20.4-slim`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.20.4-slim` 的 Docker 映像，請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。例如，你可以透過設定 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4` 來下載 RAGFlow 鏡像的 `v0.20.4` 完整發行版。
 
    ```bash
    $ cd ragflow/docker
@@ -196,8 +196,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.2           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.2-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.4           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.4-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 

README_zh.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.2">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.4">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -183,7 +183,7 @@
 > 请注意，目前官方提供的所有 Docker 镜像均基于 x86 架构构建，并不提供基于 ARM64 的 Docker 镜像。
 > 如果你的操作系统是 ARM64 架构，请参考[这篇文档](https://ragflow.io/docs/dev/build_docker_image)自行构建 Docker 镜像。
 
-   > 运行以下命令会自动下载 RAGFlow slim Docker 镜像 `v0.20.2-slim`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.20.2-slim` 的 Docker 镜像，请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。比如，你可以通过设置 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` 来下载 RAGFlow 镜像的 `v0.20.2` 完整发行版。
+   > 运行以下命令会自动下载 RAGFlow slim Docker 镜像 `v0.20.4-slim`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.20.4-slim` 的 Docker 镜像，请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。比如，你可以通过设置 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4` 来下载 RAGFlow 镜像的 `v0.20.4` 完整发行版。
 
    ```bash
    $ cd ragflow/docker
@@ -196,8 +196,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.2           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.2-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.4           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.4-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 

agent/canvas.py

@@ -131,7 +131,16 @@ class Canvas:
 
         self.path = self.dsl["path"]
         self.history = self.dsl["history"]
-        self.globals = self.dsl["globals"]
+        if "globals" in self.dsl:
+            self.globals = self.dsl["globals"]
+        else:
+            self.globals = {
+            "sys.query": "",
+            "sys.user_id": "",
+            "sys.conversation_turns": 0,
+            "sys.files": []
+        }
+            
         self.retrieval = self.dsl["retrieval"]
         self.memory = self.dsl.get("memory", [])
 
@@ -417,7 +426,7 @@ class Canvas:
         convs = []
         if window_size <= 0:
             return convs
-        for role, obj in self.history[window_size * -1:]:
+        for role, obj in self.history[window_size * -2:]:
             if isinstance(obj, dict):
                 convs.append({"role": role, "content": obj.get("content", "")})
             else:
@@ -460,6 +469,9 @@ class Canvas:
     def get_prologue(self):
         return self.components["begin"]["obj"]._param.prologue
 
+    def get_mode(self):
+        return self.components["begin"]["obj"]._param.mode
+
     def set_global_param(self, **kwargs):
         self.globals.update(kwargs)
 

agent/component/base.py

@@ -36,7 +36,7 @@ _IS_RAW_CONF = "_is_raw_conf"
 
 class ComponentParamBase(ABC):
     def __init__(self):
-        self.message_history_window_size = 22
+        self.message_history_window_size = 13
         self.inputs = {}
         self.outputs = {}
         self.description = ""

agent/component/llm.py

@@ -18,11 +18,8 @@ import logging
 import os
 import re
 from typing import Any, Generator
-
 import json_repair
-from copy import deepcopy
 from functools import partial
-
 from api.db import LLMType
 from api.db.services.llm_service import LLMBundle
 from api.db.services.tenant_llm_service import TenantLLMService
@@ -130,7 +127,7 @@ class LLM(ComponentBase):
 
         args = {}
         vars = self.get_input_elements() if not self._param.debug_inputs else self._param.debug_inputs
-        prompt = self._param.sys_prompt
+        sys_prompt = self._param.sys_prompt
         for k, o in vars.items():
             args[k] = o["value"]
             if not isinstance(args[k], str):
@@ -141,14 +138,18 @@ class LLM(ComponentBase):
             self.set_input_value(k, args[k])
 
         msg = self._canvas.get_history(self._param.message_history_window_size)[:-1]
-        msg.extend(deepcopy(self._param.prompts))
-        prompt = self.string_format(prompt, args)
+        for p in self._param.prompts:
+            if msg and msg[-1]["role"] == p["role"]:
+                continue
+            msg.append(p)
+
+        sys_prompt = self.string_format(sys_prompt, args)
         for m in msg:
             m["content"] = self.string_format(m["content"], args)
         if self._param.cite and self._canvas.get_reference()["chunks"]:
-            prompt += citation_prompt()
+            sys_prompt += citation_prompt()
 
-        return prompt, msg
+        return sys_prompt, msg
 
     def _generate(self, msg:list[dict], **kwargs) -> str:
         if not self.imgs:

agent/tools/code_exec.py

@@ -79,7 +79,7 @@ def main() -> dict:
     return {
         "result": fibonacci_recursive(100),
     }
-    
+
 Here's a code example for Javascript(`main` function MUST be included and exported):
 const axios = require('axios');
 async function main(args) {
@@ -156,7 +156,7 @@ class CodeExec(ToolBase, ABC):
             self.set_output("_ERROR", "construct code request error: " + str(e))
 
         try:
-            resp = requests.post(url=f"http://{settings.SANDBOX_HOST}:9385/run", json=code_req, timeout=10)
+            resp = requests.post(url=f"http://{settings.SANDBOX_HOST}:9385/run", json=code_req, timeout=os.environ.get("COMPONENT_EXEC_TIMEOUT", 10*60))
             logging.info(f"http://{settings.SANDBOX_HOST}:9385/run", code_req, resp.status_code)
             if resp.status_code != 200:
                 resp.raise_for_status()

api/apps/sdk/chat.py

@@ -150,10 +150,10 @@ def update(tenant_id, chat_id):
     if not DialogService.query(tenant_id=tenant_id, id=chat_id, status=StatusEnum.VALID.value):
         return get_error_data_result(message="You do not own the chat")
     req = request.json
-    ids = req.get("dataset_ids")
+    ids = req.get("dataset_ids", [])
     if "show_quotation" in req:
         req["do_refer"] = req.pop("show_quotation")
-    if ids is not None:
+    if ids:
         for kb_id in ids:
             kbs = KnowledgebaseService.accessible(kb_id=kb_id, user_id=tenant_id)
             if not kbs:

api/apps/sdk/dify_retrieval.py

@@ -24,6 +24,7 @@ from api.db.services.llm_service import LLMBundle
 from api import settings
 from api.utils.api_utils import validate_request, build_error_result, apikey_required
 from rag.app.tag import label_question
+from api.db.services.dialog_service import meta_filter
 
 
 @manager.route('/dify/retrieval', methods=['POST'])  # noqa: F821
@@ -37,18 +38,23 @@ def retrieval(tenant_id):
     retrieval_setting = req.get("retrieval_setting", {})
     similarity_threshold = float(retrieval_setting.get("score_threshold", 0.0))
     top = int(retrieval_setting.get("top_k", 1024))
-
+    metadata_condition = req.get("metadata_condition",{})
+    metas = DocumentService.get_meta_by_kbs([kb_id])
+ 
+    doc_ids = []
     try:
 
         e, kb = KnowledgebaseService.get_by_id(kb_id)
         if not e:
             return build_error_result(message="Knowledgebase not found!", code=settings.RetCode.NOT_FOUND)
 
-        if kb.tenant_id != tenant_id:
-            return build_error_result(message="Knowledgebase not found!", code=settings.RetCode.NOT_FOUND)
-
         embd_mdl = LLMBundle(kb.tenant_id, LLMType.EMBEDDING.value, llm_name=kb.embd_id)
-
+        print(metadata_condition)
+        print("after",convert_conditions(metadata_condition))
+        doc_ids.extend(meta_filter(metas, convert_conditions(metadata_condition)))
+        print("doc_ids",doc_ids)
+        if not doc_ids and metadata_condition is not None:
+            doc_ids = ['-999']
         ranks = settings.retrievaler.retrieval(
             question,
             embd_mdl,
@@ -59,6 +65,7 @@ def retrieval(tenant_id):
             similarity_threshold=similarity_threshold,
             vector_similarity_weight=0.3,
             top=top,
+            doc_ids=doc_ids,
             rank_feature=label_question(question, [kb])
         )
 
@@ -67,6 +74,7 @@ def retrieval(tenant_id):
                                                    [tenant_id],
                                                    [kb_id],
                                                    embd_mdl,
+                                                   doc_ids,
                                                    LLMBundle(kb.tenant_id, LLMType.CHAT))
             if ck["content_with_weight"]:
                 ranks["chunks"].insert(0, ck)
@@ -93,3 +101,20 @@ def retrieval(tenant_id):
             )
         logging.exception(e)
         return build_error_result(message=str(e), code=settings.RetCode.SERVER_ERROR)
+
+def convert_conditions(metadata_condition):
+    if metadata_condition is None:
+        metadata_condition = {}
+    op_mapping = {
+        "is": "=",
+        "not is": "≠"
+    }
+    return [
+    {
+        "op": op_mapping.get(cond["comparison_operator"], cond["comparison_operator"]),
+        "key": cond["name"],
+        "value": cond["value"]
+    }
+    for cond in metadata_condition.get("conditions", [])
+]
+

api/apps/sdk/session.py

@@ -16,8 +16,10 @@
 import json
 import re
 import time
+
 import tiktoken
 from flask import Response, jsonify, request
+
 from agent.canvas import Canvas
 from api import settings
 from api.db import LLMType, StatusEnum
@@ -27,7 +29,8 @@ from api.db.services.canvas_service import UserCanvasService, completionOpenAI
 from api.db.services.canvas_service import completion as agent_completion
 from api.db.services.conversation_service import ConversationService, iframe_completion
 from api.db.services.conversation_service import completion as rag_completion
-from api.db.services.dialog_service import DialogService, ask, chat, gen_mindmap
+from api.db.services.dialog_service import DialogService, ask, chat, gen_mindmap, meta_filter
+from api.db.services.document_service import DocumentService
 from api.db.services.knowledgebase_service import KnowledgebaseService
 from api.db.services.llm_service import LLMBundle
 from api.db.services.search_service import SearchService
@@ -37,7 +40,7 @@ from api.utils.api_utils import check_duplicate_ids, get_data_openai, get_error_
 from rag.app.tag import label_question
 from rag.prompts import chunks_format
 from rag.prompts.prompt_template import load_prompt
-from rag.prompts.prompts import cross_languages, keyword_extraction
+from rag.prompts.prompts import cross_languages, gen_meta_filter, keyword_extraction
 
 
 @manager.route("/chats/<chat_id>/sessions", methods=["POST"])  # noqa: F821
@@ -81,21 +84,13 @@ def create_agent_session(tenant_id, agent_id):
     if not isinstance(cvs.dsl, str):
         cvs.dsl = json.dumps(cvs.dsl, ensure_ascii=False)
 
-    session_id=get_uuid()
+    session_id = get_uuid()
     canvas = Canvas(cvs.dsl, tenant_id, agent_id)
     canvas.reset()
-    conv = {
-        "id": session_id,
-        "dialog_id": cvs.id,
-        "user_id": user_id,
-        "message": [],
-        "source": "agent",
-        "dsl": cvs.dsl
-    }
-    API4ConversationService.save(**conv)
 
     cvs.dsl = json.loads(str(canvas))
     conv = {"id": session_id, "dialog_id": cvs.id, "user_id": user_id, "message": [{"role": "assistant", "content": canvas.get_prologue()}], "source": "agent", "dsl": cvs.dsl}
+    API4ConversationService.save(**conv)
     conv["agent_id"] = conv.pop("dialog_id")
     return get_result(data=conv)
 
@@ -461,13 +456,14 @@ def agent_completions(tenant_id, agent_id):
                     except Exception:
                         continue
 
-                if ans.get("event") != "message":
+                if ans.get("event") != "message" or not ans.get("data", {}).get("reference", None):
                     continue
 
                 yield answer
 
             yield "data:[DONE]\n\n"
 
+    if req.get("stream", True):
         resp = Response(generate(), mimetype="text/event-stream")
         resp.headers.add_header("Cache-control", "no-cache")
         resp.headers.add_header("Connection", "keep-alive")
@@ -475,9 +471,17 @@ def agent_completions(tenant_id, agent_id):
         resp.headers.add_header("Content-Type", "text/event-stream; charset=utf-8")
         return resp
 
+    full_content = ""
     for answer in agent_completion(tenant_id=tenant_id, agent_id=agent_id, **req):
         try:
-            ans = json.loads(answer[5:])  # remove "data:"
+            ans = json.loads(answer[5:])
+
+            if ans["event"] == "message":
+                full_content += ans["data"]["content"]
+
+            if ans.get("data", {}).get("reference", None):
+                ans["data"]["content"] = full_content
+                return get_result(data=ans)
         except Exception as e:
             return get_result(data=f"**ERROR**: {str(e)}")
     return get_result(data=ans)
@@ -575,12 +579,12 @@ def list_agent_session(tenant_id, agent_id):
                 if message_num != 0 and messages[message_num]["role"] != "user":
                     chunk_list = []
                     # Add boundary and type checks to prevent KeyError
-                    if (chunk_num < len(conv["reference"]) and
-                            conv["reference"][chunk_num] is not None and
-                            isinstance(conv["reference"][chunk_num], dict) and
-                            "chunks" in conv["reference"][chunk_num]):
+                    if chunk_num < len(conv["reference"]) and conv["reference"][chunk_num] is not None and isinstance(conv["reference"][chunk_num], dict) and "chunks" in conv["reference"][chunk_num]:
                         chunks = conv["reference"][chunk_num]["chunks"]
                         for chunk in chunks:
+                            # Ensure chunk is a dictionary before calling get method
+                            if not isinstance(chunk, dict):
+                                continue
                             new_chunk = {
                                 "id": chunk.get("chunk_id", chunk.get("id")),
                                 "content": chunk.get("content_with_weight", chunk.get("content")),
@@ -876,14 +880,7 @@ def begin_inputs(agent_id):
         return get_error_data_result(f"Can't find agent by ID: {agent_id}")
 
     canvas = Canvas(json.dumps(cvs.dsl), objs[0].tenant_id)
-    return get_result(
-        data={
-            "title": cvs.title,
-            "avatar": cvs.avatar,
-            "inputs": canvas.get_component_input_form("begin"),
-            "prologue": canvas.get_prologue()
-        }
-    )
+    return get_result(data={"title": cvs.title, "avatar": cvs.avatar, "inputs": canvas.get_component_input_form("begin"), "prologue": canvas.get_prologue(), "mode": canvas.get_mode()})
 
 
 @manager.route("/searchbots/ask", methods=["POST"])  # noqa: F821
@@ -909,7 +906,7 @@ def ask_about_embedded():
     def stream():
         nonlocal req, uid
         try:
-            for ans in ask(req["question"], req["kb_ids"], uid, search_config):
+            for ans in ask(req["question"], req["kb_ids"], uid, search_config=search_config):
                 yield "data:" + json.dumps({"code": 0, "message": "", "data": ans}, ensure_ascii=False) + "\n\n"
         except Exception as e:
             yield "data:" + json.dumps({"code": 500, "message": str(e), "data": {"answer": "**ERROR**: " + str(e), "reference": []}}, ensure_ascii=False) + "\n\n"
@@ -923,7 +920,7 @@ def ask_about_embedded():
     return resp
 
 
-@manager.route("/searchbots/retrieval_test", methods=['POST'])  # noqa: F821
+@manager.route("/searchbots/retrieval_test", methods=["POST"])  # noqa: F821
 @validate_request("kb_id", "question")
 def retrieval_test_embedded():
     token = request.headers.get("Authorization").split()
@@ -953,18 +950,30 @@ def retrieval_test_embedded():
     if not tenant_id:
         return get_error_data_result(message="permission denined.")
 
+    if req.get("search_id", ""):
+        search_config = SearchService.get_detail(req.get("search_id", "")).get("search_config", {})
+        meta_data_filter = search_config.get("meta_data_filter", {})
+        metas = DocumentService.get_meta_by_kbs(kb_ids)
+        if meta_data_filter.get("method") == "auto":
+            chat_mdl = LLMBundle(tenant_id, LLMType.CHAT, llm_name=search_config.get("chat_id", ""))
+            filters = gen_meta_filter(chat_mdl, metas, question)
+            doc_ids.extend(meta_filter(metas, filters))
+            if not doc_ids:
+                doc_ids = None
+        elif meta_data_filter.get("method") == "manual":
+            doc_ids.extend(meta_filter(metas, meta_data_filter["manual"]))
+            if not doc_ids:
+                doc_ids = None
+
     try:
         tenants = UserTenantService.query(user_id=tenant_id)
         for kb_id in kb_ids:
             for tenant in tenants:
-                if KnowledgebaseService.query(
-                        tenant_id=tenant.tenant_id, id=kb_id):
+                if KnowledgebaseService.query(tenant_id=tenant.tenant_id, id=kb_id):
                     tenant_ids.append(tenant.tenant_id)
                     break
             else:
-                return get_json_result(
-                    data=False, message='Only owner of knowledgebase authorized for this operation.',
-                    code=settings.RetCode.OPERATING_ERROR)
+                return get_json_result(data=False, message="Only owner of knowledgebase authorized for this operation.", code=settings.RetCode.OPERATING_ERROR)
 
         e, kb = KnowledgebaseService.get_by_id(kb_ids[0])
         if not e:
@@ -984,17 +993,11 @@ def retrieval_test_embedded():
             question += keyword_extraction(chat_mdl, question)
 
         labels = label_question(question, [kb])
-        ranks = settings.retrievaler.retrieval(question, embd_mdl, tenant_ids, kb_ids, page, size,
-                               similarity_threshold, vector_similarity_weight, top,
-                               doc_ids, rerank_mdl=rerank_mdl, highlight=req.get("highlight"),
-                               rank_feature=labels
-                               )
+        ranks = settings.retrievaler.retrieval(
+            question, embd_mdl, tenant_ids, kb_ids, page, size, similarity_threshold, vector_similarity_weight, top, doc_ids, rerank_mdl=rerank_mdl, highlight=req.get("highlight"), rank_feature=labels
+        )
         if use_kg:
-            ck = settings.kg_retrievaler.retrieval(question,
-                                                   tenant_ids,
-                                                   kb_ids,
-                                                   embd_mdl,
-                                                   LLMBundle(kb.tenant_id, LLMType.CHAT))
+            ck = settings.kg_retrievaler.retrieval(question, tenant_ids, kb_ids, embd_mdl, LLMBundle(kb.tenant_id, LLMType.CHAT))
             if ck["content_with_weight"]:
                 ranks["chunks"].insert(0, ck)
 
@@ -1005,8 +1008,7 @@ def retrieval_test_embedded():
         return get_json_result(data=ranks)
     except Exception as e:
         if str(e).find("not_found") > 0:
-            return get_json_result(data=False, message='No chunk found! Check the chunk status please!',
-                                   code=settings.RetCode.DATA_ERROR)
+            return get_json_result(data=False, message="No chunk found! Check the chunk status please!", code=settings.RetCode.DATA_ERROR)
         return server_error_response(e)
 
 

api/apps/search_app.py

@@ -155,8 +155,9 @@ def list_search_app():
     owner_ids = req.get("owner_ids", [])
     try:
         if not owner_ids:
-            tenants = TenantService.get_joined_tenants_by_user_id(current_user.id)
-            tenants = [m["tenant_id"] for m in tenants]
+            # tenants = TenantService.get_joined_tenants_by_user_id(current_user.id)
+            # tenants = [m["tenant_id"] for m in tenants]
+            tenants = []
             search_apps, total = SearchService.get_by_tenant_ids(tenants, current_user.id, page_number, items_per_page, orderby, desc, keywords)
         else:
             tenants = owner_ids

api/db/services/canvas_service.py

@@ -134,6 +134,7 @@ class UserCanvasService(CommonService):
             return False
         return True
 
+
 def completion(tenant_id, agent_id, session_id=None, **kwargs):
     query = kwargs.get("query", "") or kwargs.get("question", "")
     files = kwargs.get("files", [])
@@ -163,7 +164,8 @@ def completion(tenant_id, agent_id, session_id=None, **kwargs):
             "user_id": user_id,
             "message": [],
             "source": "agent",
-            "dsl": cvs.dsl
+            "dsl": cvs.dsl,
+            "reference": []
         }
         API4ConversationService.save(**conv)
         conv = API4Conversation(**conv)
@@ -211,10 +213,8 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
                     except Exception as e:
                         logging.exception(f"Agent OpenAI-Compatible completionOpenAI parse answer failed: {e}")
                         continue
-
-                if ans.get("event") != "message":
+                if ans.get("event") != "message" or not ans.get("data", {}).get("reference", None):
                     continue
-
                 content_piece = ans["data"]["content"]
                 completion_tokens += len(tiktokenenc.encode(content_piece))
 
@@ -260,7 +260,7 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
             ):
                 if isinstance(ans, str):
                     ans = json.loads(ans[5:])
-                if ans.get("event") != "message":
+                if ans.get("event") != "message" or not ans.get("data", {}).get("reference", None):
                     continue
                 all_content += ans["data"]["content"]
 

api/db/services/dialog_service.py

@@ -256,10 +256,10 @@ def repair_bad_citation_formats(answer: str, kbinfos: dict, idx: set):
 
 
 def meta_filter(metas: dict, filters: list[dict]):
-    doc_ids = []
+    doc_ids = set([])
 
     def filter_out(v2docs, operator, value):
-        nonlocal doc_ids
+        ids = []
         for input, docids in v2docs.items():
             try:
                 input = float(input)
@@ -284,16 +284,24 @@ def meta_filter(metas: dict, filters: list[dict]):
                 ]:
                 try:
                     if all(conds):
-                        doc_ids.extend(docids)
+                        ids.extend(docids)
+                        break
                 except Exception:
                     pass
+        return ids
 
     for k, v2docs in metas.items():
         for f in filters:
             if k != f["key"]:
                 continue
-            filter_out(v2docs, f["op"], f["value"])
-    return doc_ids
+            ids = filter_out(v2docs, f["op"], f["value"])
+            if not doc_ids:
+                doc_ids = set(ids)
+            else:
+                doc_ids = doc_ids & set(ids)
+            if not doc_ids:
+                return []
+    return list(doc_ids)
 
 
 def chat(dialog, messages, stream=True, **kwargs):

api/db/services/llm_service.py

@@ -152,7 +152,7 @@ class LLMBundle(LLM4Tenant):
 
     def describe_with_prompt(self, image, prompt):
         if self.langfuse:
-            generation = self.language.start_generation(trace_context=self.trace_context, name="describe_with_prompt", metadata={"model": self.llm_name, "prompt": prompt})
+            generation = self.langfuse.start_generation(trace_context=self.trace_context, name="describe_with_prompt", metadata={"model": self.llm_name, "prompt": prompt})
 
         txt, used_tokens = self.mdl.describe_with_prompt(image, prompt)
         if not TenantLLMService.increase_usage(self.tenant_id, self.llm_type, used_tokens):

api/utils/api_utils.py

@@ -17,6 +17,7 @@ import asyncio
 import functools
 import json
 import logging
+import os
 import queue
 import random
 import threading
@@ -353,7 +354,7 @@ def get_parser_config(chunk_method, parser_config):
     if not chunk_method:
         chunk_method = "naive"
 
-    # Define default configurations for each chunk method
+    # Define default configurations for each chunking method
     key_mapping = {
         "naive": {"chunk_token_num": 512, "delimiter": r"\n", "html4excel": False, "layout_recognize": "DeepDOC", "raptor": {"use_raptor": False}, "graphrag": {"use_graphrag": False}},
         "qa": {"raptor": {"use_raptor": False}, "graphrag": {"use_graphrag": False}},
@@ -667,7 +668,10 @@ def timeout(seconds: float | int = None, attempts: int = 2, *, exception: Option
 
             for a in range(attempts):
                 try:
-                    result = result_queue.get(timeout=seconds)
+                    if os.environ.get("ENABLE_TIMEOUT_ASSERTION"):
+                        result = result_queue.get(timeout=seconds)
+                    else:
+                        result = result_queue.get()
                     if isinstance(result, Exception):
                         raise result
                     return result
@@ -682,7 +686,10 @@ def timeout(seconds: float | int = None, attempts: int = 2, *, exception: Option
 
             for a in range(attempts):
                 try:
-                    with trio.fail_after(seconds):
+                    if os.environ.get("ENABLE_TIMEOUT_ASSERTION"):
+                        with trio.fail_after(seconds):
+                            return await func(*args, **kwargs)
+                    else:
                         return await func(*args, **kwargs)
                 except trio.TooSlowError:
                     if a < attempts - 1:

conf/llm_factories.json

@@ -532,23 +532,65 @@
             "tags": "LLM,TEXT EMBEDDING,SPEECH2TEXT,MODERATION",
             "status": "1",
             "llm": [
+                {
+                    "llm_name": "glm-4.5",
+                    "tags": "LLM,CHAT,128K",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-x",
+                    "tags": "LLM,CHAT,128k",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-air",
+                    "tags": "LLM,CHAT,128K",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-airx",
+                    "tags": "LLM,CHAT,128k",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-flash",
+                    "tags": "LLM,CHAT,128k",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5v",
+                    "tags": "LLM,IMAGE2TEXT,64,",
+                    "max_tokens": 64000,
+                    "model_type": "image2text",
+                    "is_tools": false
+                },
                 {
                     "llm_name": "glm-4-plus",
-                    "tags": "LLM,CHAT,",
+                    "tags": "LLM,CHAT,128K",
                     "max_tokens": 128000,
                     "model_type": "chat",
                     "is_tools": true
                 },
                 {
                     "llm_name": "glm-4-0520",
-                    "tags": "LLM,CHAT,",
+                    "tags": "LLM,CHAT,128K",
                     "max_tokens": 128000,
                     "model_type": "chat",
                     "is_tools": true
                 },
                 {
                     "llm_name": "glm-4",
-                    "tags": "LLM,CHAT,",
+                    "tags":"LLM,CHAT,128K",
                     "max_tokens": 128000,
                     "model_type": "chat",
                     "is_tools": true

deepdoc/parser/__init__.py

@@ -14,13 +14,15 @@
 #  limitations under the License.
 #
 
-from .pdf_parser import RAGFlowPdfParser as PdfParser, PlainParser
 from .docx_parser import RAGFlowDocxParser as DocxParser
 from .excel_parser import RAGFlowExcelParser as ExcelParser
-from .ppt_parser import RAGFlowPptParser as PptParser
 from .html_parser import RAGFlowHtmlParser as HtmlParser
 from .json_parser import RAGFlowJsonParser as JsonParser
+from .markdown_parser import MarkdownElementExtractor
 from .markdown_parser import RAGFlowMarkdownParser as MarkdownParser
+from .pdf_parser import PlainParser
+from .pdf_parser import RAGFlowPdfParser as PdfParser
+from .ppt_parser import RAGFlowPptParser as PptParser
 from .txt_parser import RAGFlowTxtParser as TxtParser
 
 __all__ = [
@@ -33,4 +35,6 @@ __all__ = [
     "JsonParser",
     "MarkdownParser",
     "TxtParser",
-]
+    "MarkdownElementExtractor",
+]
+

deepdoc/parser/html_parser.py

@@ -15,35 +15,200 @@
 #  limitations under the License.
 #
 
-from rag.nlp import find_codec
-import readability
-import html_text
+from rag.nlp import find_codec, rag_tokenizer
+import uuid
 import chardet
-
+from bs4 import BeautifulSoup, NavigableString, Tag, Comment
+import html
 
 def get_encoding(file):
     with open(file,'rb') as f:
         tmp = chardet.detect(f.read())
         return tmp['encoding']
 
+BLOCK_TAGS = [
+    "h1", "h2", "h3", "h4", "h5", "h6",
+    "p", "div", "article", "section", "aside",
+    "ul", "ol", "li",
+    "table", "pre", "code", "blockquote",
+    "figure", "figcaption"
+]
+TITLE_TAGS = {"h1": "#", "h2": "##", "h3": "###", "h4": "#####", "h5": "#####", "h6": "######"}
+
 
 class RAGFlowHtmlParser:
-    def __call__(self, fnm, binary=None):
+    def __call__(self, fnm, binary=None, chunk_token_num=None):
         if binary:
             encoding = find_codec(binary)
             txt = binary.decode(encoding, errors="ignore")
         else:
             with open(fnm, "r",encoding=get_encoding(fnm)) as f:
                 txt = f.read()
-        return self.parser_txt(txt)
+        return self.parser_txt(txt, chunk_token_num)
 
     @classmethod
-    def parser_txt(cls, txt):
+    def parser_txt(cls, txt, chunk_token_num):
         if not isinstance(txt, str):
             raise TypeError("txt type should be string!")
-        html_doc = readability.Document(txt)
-        title = html_doc.title()
-        content = html_text.extract_text(html_doc.summary(html_partial=True))
-        txt = f"{title}\n{content}"
-        sections = txt.split("\n")
+
+        temp_sections = []
+        soup = BeautifulSoup(txt, "html5lib")
+        # delete <style> tag
+        for style_tag in soup.find_all(["style", "script"]):
+            style_tag.decompose()
+        # delete <script> tag in <div>
+        for div_tag in soup.find_all("div"):
+            for script_tag in div_tag.find_all("script"):
+                script_tag.decompose()
+        # delete inline style
+        for tag in soup.find_all(True):
+            if 'style' in tag.attrs:
+                del tag.attrs['style']
+        # delete HTML comment
+        for comment in soup.find_all(string=lambda text: isinstance(text, Comment)):
+            comment.extract()
+
+        cls.read_text_recursively(soup.body, temp_sections, chunk_token_num=chunk_token_num)
+        block_txt_list, table_list = cls.merge_block_text(temp_sections)
+        sections = cls.chunk_block(block_txt_list, chunk_token_num=chunk_token_num)
+        for table in table_list:
+            sections.append(table.get("content", ""))
         return sections
+
+    @classmethod
+    def split_table(cls, html_table, chunk_token_num=512):
+        soup = BeautifulSoup(html_table, "html.parser")
+        rows = soup.find_all("tr")
+        tables = []
+        current_table = []
+        current_count = 0
+        table_str_list = []
+        for row in rows:
+            tks_str = rag_tokenizer.tokenize(str(row))
+            token_count = len(tks_str.split(" ")) if tks_str else 0
+            if current_count + token_count > chunk_token_num:
+                tables.append(current_table)
+                current_table = []
+                current_count = 0
+            current_table.append(row)
+            current_count += token_count
+        if current_table:
+            tables.append(current_table)
+
+        for table_rows in tables:
+            new_table = soup.new_tag("table")
+            for row in table_rows:
+                new_table.append(row)
+            table_str_list.append(str(new_table))
+
+        return table_str_list
+
+    @classmethod
+    def read_text_recursively(cls, element, parser_result, chunk_token_num=512, parent_name=None, block_id=None):
+        if isinstance(element, NavigableString):
+            content = element.strip()
+
+            def is_valid_html(content):
+                try:
+                    soup = BeautifulSoup(content, "html.parser")
+                    return bool(soup.find())
+                except Exception:
+                    return False
+
+            return_info = []
+            if content:
+                if is_valid_html(content):
+                    soup = BeautifulSoup(content, "html.parser")
+                    child_info = cls.read_text_recursively(soup, parser_result, chunk_token_num, element.name, block_id)
+                    parser_result.extend(child_info)
+                else:
+                    info = {"content": element.strip(), "tag_name": "inner_text", "metadata": {"block_id": block_id}}
+                    if parent_name:
+                        info["tag_name"] = parent_name
+                    return_info.append(info)
+            return return_info
+        elif isinstance(element, Tag):
+
+            if str.lower(element.name) == "table":
+                table_info_list = []
+                table_id = str(uuid.uuid1())
+                table_list = [html.unescape(str(element))]
+                for t in table_list:
+                    table_info_list.append({"content": t, "tag_name": "table",
+                                            "metadata": {"table_id": table_id, "index": table_list.index(t)}})
+                return table_info_list
+            else:
+                block_id = None
+                if str.lower(element.name) in BLOCK_TAGS:
+                    block_id = str(uuid.uuid1())
+                for child in element.children:
+                    child_info = cls.read_text_recursively(child, parser_result, chunk_token_num, element.name,
+                                                           block_id)
+                    parser_result.extend(child_info)
+        return []
+
+    @classmethod
+    def merge_block_text(cls, parser_result):
+        block_content = []
+        current_content = ""
+        table_info_list = []
+        lask_block_id = None
+        for item in parser_result:
+            content = item.get("content")
+            tag_name = item.get("tag_name")
+            title_flag = tag_name in TITLE_TAGS
+            block_id = item.get("metadata", {}).get("block_id")
+            if block_id:
+                if title_flag:
+                    content = f"{TITLE_TAGS[tag_name]} {content}"
+                if lask_block_id != block_id:
+                    if lask_block_id is not None:
+                        block_content.append(current_content)
+                    current_content = content
+                    lask_block_id = block_id
+                else:
+                    current_content += (" " if current_content else "") + content
+            else:
+                if tag_name == "table":
+                    table_info_list.append(item)
+                else:
+                    current_content += (" " if current_content else "" + content)
+        if current_content:
+            block_content.append(current_content)
+        return block_content, table_info_list
+
+    @classmethod
+    def chunk_block(cls, block_txt_list, chunk_token_num=512):
+        chunks = []
+        current_block = ""
+        current_token_count = 0
+
+        for block in block_txt_list:
+            tks_str = rag_tokenizer.tokenize(block)
+            block_token_count = len(tks_str.split(" ")) if tks_str else 0
+            if block_token_count > chunk_token_num:
+                if current_block:
+                    chunks.append(current_block)
+                start = 0
+                tokens = tks_str.split(" ")
+                while start < len(tokens):
+                    end = start + chunk_token_num
+                    split_tokens = tokens[start:end]
+                    chunks.append(" ".join(split_tokens))
+                    start = end
+                current_block = ""
+                current_token_count = 0
+            else:
+                if current_token_count + block_token_count <= chunk_token_num:
+                    current_block += ("\n" if current_block else "") + block
+                    current_token_count += block_token_count
+                else:
+                    chunks.append(current_block)
+                    current_block = block
+                    current_token_count = block_token_count
+
+        if current_block:
+            chunks.append(current_block)
+
+        return chunks
+

deepdoc/parser/markdown_parser.py

@@ -17,8 +17,10 @@
 
 import re
 
+import mistune
 from markdown import markdown
 
+
 class RAGFlowMarkdownParser:
     def __init__(self, chunk_token_num=128):
         self.chunk_token_num = int(chunk_token_num)
@@ -35,40 +37,44 @@ class RAGFlowMarkdownParser:
                 table_list.append(raw_table)
                 if separate_tables:
                     # Skip this match (i.e., remove it)
-                    new_text += working_text[last_end:match.start()] + "\n\n"
+                    new_text += working_text[last_end : match.start()] + "\n\n"
                 else:
                     # Replace with rendered HTML
-                    html_table = markdown(raw_table, extensions=['markdown.extensions.tables']) if render else raw_table
-                    new_text += working_text[last_end:match.start()] + html_table + "\n\n"
+                    html_table = markdown(raw_table, extensions=["markdown.extensions.tables"]) if render else raw_table
+                    new_text += working_text[last_end : match.start()] + html_table + "\n\n"
                 last_end = match.end()
             new_text += working_text[last_end:]
             return new_text
 
-        if "|" in markdown_text: # for optimize performance
+        if "|" in markdown_text:  # for optimize performance
             # Standard Markdown table
             border_table_pattern = re.compile(
-                r'''
+                r"""
                 (?:\n|^)
                 (?:\|.*?\|.*?\|.*?\n)
                 (?:\|(?:\s*[:-]+[-| :]*\s*)\|.*?\n)
                 (?:\|.*?\|.*?\|.*?\n)+
-            ''', re.VERBOSE)
+            """,
+                re.VERBOSE,
+            )
             working_text = replace_tables_with_rendered_html(border_table_pattern, tables)
 
             # Borderless Markdown table
             no_border_table_pattern = re.compile(
-                r'''
+                r"""
                 (?:\n|^)
                 (?:\S.*?\|.*?\n)
                 (?:(?:\s*[:-]+[-| :]*\s*).*?\n)
                 (?:\S.*?\|.*?\n)+
-                ''', re.VERBOSE)
+                """,
+                re.VERBOSE,
+            )
             working_text = replace_tables_with_rendered_html(no_border_table_pattern, tables)
 
-        if "<table>" in working_text.lower(): # for optimize performance
-            #HTML table extraction - handle possible html/body wrapper tags
+        if "<table>" in working_text.lower():  # for optimize performance
+            # HTML table extraction - handle possible html/body wrapper tags
             html_table_pattern = re.compile(
-            r'''
+                r"""
             (?:\n|^)
             \s*
             (?:
@@ -83,9 +89,10 @@ class RAGFlowMarkdownParser:
             )
             \s*
             (?=\n|$)
-            ''',
-            re.VERBOSE | re.DOTALL | re.IGNORECASE
+            """,
+                re.VERBOSE | re.DOTALL | re.IGNORECASE,
             )
+
             def replace_html_tables():
                 nonlocal working_text
                 new_text = ""
@@ -94,9 +101,9 @@ class RAGFlowMarkdownParser:
                     raw_table = match.group()
                     tables.append(raw_table)
                     if separate_tables:
-                        new_text += working_text[last_end:match.start()] + "\n\n"
+                        new_text += working_text[last_end : match.start()] + "\n\n"
                     else:
-                        new_text += working_text[last_end:match.start()] + raw_table + "\n\n"
+                        new_text += working_text[last_end : match.start()] + raw_table + "\n\n"
                     last_end = match.end()
                 new_text += working_text[last_end:]
                 working_text = new_text
@@ -104,3 +111,163 @@ class RAGFlowMarkdownParser:
             replace_html_tables()
 
         return working_text, tables
+
+
+class MarkdownElementExtractor:
+    def __init__(self, markdown_content):
+        self.markdown_content = markdown_content
+        self.lines = markdown_content.split("\n")
+        self.ast_parser = mistune.create_markdown(renderer="ast")
+        self.ast_nodes = self.ast_parser(markdown_content)
+
+    def extract_elements(self):
+        """Extract individual elements (headers, code blocks, lists, etc.)"""
+        sections = []
+
+        i = 0
+        while i < len(self.lines):
+            line = self.lines[i]
+
+            if re.match(r"^#{1,6}\s+.*$", line):
+                # header
+                element = self._extract_header(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif line.strip().startswith("```"):
+                # code block
+                element = self._extract_code_block(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif re.match(r"^\s*[-*+]\s+.*$", line) or re.match(r"^\s*\d+\.\s+.*$", line):
+                # list block
+                element = self._extract_list_block(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif line.strip().startswith(">"):
+                # blockquote
+                element = self._extract_blockquote(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif line.strip():
+                # text block (paragraphs and inline elements until next block element)
+                element = self._extract_text_block(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            else:
+                i += 1
+
+        sections = [section for section in sections if section.strip()]
+        return sections
+
+    def _extract_header(self, start_pos):
+        return {
+            "type": "header",
+            "content": self.lines[start_pos],
+            "start_line": start_pos,
+            "end_line": start_pos,
+        }
+
+    def _extract_code_block(self, start_pos):
+        end_pos = start_pos
+        content_lines = [self.lines[start_pos]]
+
+        # Find the end of the code block
+        for i in range(start_pos + 1, len(self.lines)):
+            content_lines.append(self.lines[i])
+            end_pos = i
+            if self.lines[i].strip().startswith("```"):
+                break
+
+        return {
+            "type": "code_block",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }
+
+    def _extract_list_block(self, start_pos):
+        end_pos = start_pos
+        content_lines = []
+
+        i = start_pos
+        while i < len(self.lines):
+            line = self.lines[i]
+            # check if this line is a list item or continuation of a list
+            if (
+                re.match(r"^\s*[-*+]\s+.*$", line)
+                or re.match(r"^\s*\d+\.\s+.*$", line)
+                or (i > start_pos and not line.strip())
+                or (i > start_pos and re.match(r"^\s{2,}[-*+]\s+.*$", line))
+                or (i > start_pos and re.match(r"^\s{2,}\d+\.\s+.*$", line))
+                or (i > start_pos and re.match(r"^\s+\w+.*$", line))
+            ):
+                content_lines.append(line)
+                end_pos = i
+                i += 1
+            else:
+                break
+
+        return {
+            "type": "list_block",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }
+
+    def _extract_blockquote(self, start_pos):
+        end_pos = start_pos
+        content_lines = []
+
+        i = start_pos
+        while i < len(self.lines):
+            line = self.lines[i]
+            if line.strip().startswith(">") or (i > start_pos and not line.strip()):
+                content_lines.append(line)
+                end_pos = i
+                i += 1
+            else:
+                break
+
+        return {
+            "type": "blockquote",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }
+
+    def _extract_text_block(self, start_pos):
+        """Extract a text block (paragraphs, inline elements) until next block element"""
+        end_pos = start_pos
+        content_lines = [self.lines[start_pos]]
+
+        i = start_pos + 1
+        while i < len(self.lines):
+            line = self.lines[i]
+            # stop if we encounter a block element
+            if re.match(r"^#{1,6}\s+.*$", line) or line.strip().startswith("```") or re.match(r"^\s*[-*+]\s+.*$", line) or re.match(r"^\s*\d+\.\s+.*$", line) or line.strip().startswith(">"):
+                break
+            elif not line.strip():
+                # check if the next line is a block element
+                if i + 1 < len(self.lines) and (
+                    re.match(r"^#{1,6}\s+.*$", self.lines[i + 1])
+                    or self.lines[i + 1].strip().startswith("```")
+                    or re.match(r"^\s*[-*+]\s+.*$", self.lines[i + 1])
+                    or re.match(r"^\s*\d+\.\s+.*$", self.lines[i + 1])
+                    or self.lines[i + 1].strip().startswith(">")
+                ):
+                    break
+                else:
+                    content_lines.append(line)
+                    end_pos = i
+                    i += 1
+            else:
+                content_lines.append(line)
+                end_pos = i
+                i += 1
+
+        return {
+            "type": "text_block",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }

docker/.env

@@ -93,13 +93,13 @@ REDIS_PASSWORD=infini_rag_flow
 SVR_HTTP_PORT=9380
 
 # The RAGFlow Docker image to download.
-# Defaults to the v0.20.1-slim edition, which is the RAGFlow Docker image without embedding models.
-RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2-slim
+# Defaults to the v0.20.4-slim edition, which is the RAGFlow Docker image without embedding models.
+RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4-slim
 #
 # To download the RAGFlow Docker image with embedding models, uncomment the following line instead:
-# RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.1
+# RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4
 #
-# The Docker image of the v0.20.1 edition includes built-in embedding models:
+# The Docker image of the v0.20.4 edition includes built-in embedding models:
 #   - BAAI/bge-large-zh-v1.5
 #   - maidalun1020/bce-embedding-base_v1
 #

docker/README.md

@@ -79,8 +79,8 @@ The [.env](./.env) file contains important environment variables for Docker.
 - `RAGFLOW-IMAGE`  
   The Docker image edition. Available editions:  
   
-  - `infiniflow/ragflow:v0.20.2-slim` (default): The RAGFlow Docker image without embedding models.  
-  - `infiniflow/ragflow:v0.20.2`: The RAGFlow Docker image with embedding models including:
+  - `infiniflow/ragflow:v0.20.4-slim` (default): The RAGFlow Docker image without embedding models.  
+  - `infiniflow/ragflow:v0.20.4`: The RAGFlow Docker image with embedding models including:
     - Built-in embedding models:
       - `BAAI/bge-large-zh-v1.5` 
       - `maidalun1020/bce-embedding-base_v1`

docs/configurations.md

@@ -99,8 +99,8 @@ RAGFlow utilizes MinIO as its object storage solution, leveraging its scalabilit
 - `RAGFLOW-IMAGE`  
   The Docker image edition. Available editions:  
   
-  - `infiniflow/ragflow:v0.20.2-slim` (default): The RAGFlow Docker image without embedding models.  
-  - `infiniflow/ragflow:v0.20.2`: The RAGFlow Docker image with embedding models including:
+  - `infiniflow/ragflow:v0.20.4-slim` (default): The RAGFlow Docker image without embedding models.  
+  - `infiniflow/ragflow:v0.20.4`: The RAGFlow Docker image with embedding models including:
     - Built-in embedding models:
       - `BAAI/bge-large-zh-v1.5` 
       - `maidalun1020/bce-embedding-base_v1`

docs/develop/acquire_ragflow_api_key.md

@@ -11,7 +11,7 @@ An API key is required for the RAGFlow server to authenticate your HTTP/Python o
 2. Click **API** to switch to the **API** page.
 3. Obtain a RAGFlow API key:
 
-![ragflow_api_key](https://github.com/user-attachments/assets/f461ed61-04c6-4faf-b3d8-6b5fa56be4e7)
+![ragflow_api_key](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/ragflow_api_key.jpg)
 
 :::tip NOTE
 See the [RAGFlow HTTP API reference](../references/http_api_reference.md) or the [RAGFlow Python API reference](../references/python_api_reference.md) for a complete reference of RAGFlow's HTTP or Python APIs.

docs/develop/build_docker_image.mdx

@@ -77,7 +77,7 @@ After building the infiniflow/ragflow:nightly-slim image, you are ready to launc
 
 1. Edit Docker Compose Configuration
 
-Open the `docker/.env` file. Find the `RAGFLOW_IMAGE` setting and change the image reference from `infiniflow/ragflow:v0.20.2-slim` to `infiniflow/ragflow:nightly-slim` to use the pre-built image.
+Open the `docker/.env` file. Find the `RAGFLOW_IMAGE` setting and change the image reference from `infiniflow/ragflow:v0.20.4-slim` to `infiniflow/ragflow:nightly-slim` to use the pre-built image.
 
 
 2. Launch the Service

docs/faq.mdx

@@ -30,17 +30,17 @@ The "garbage in garbage out" status quo remains unchanged despite the fact that
 
 Each RAGFlow release is available in two editions:
 
-- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.2-slim`
-- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.2`
+- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.4-slim`
+- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.4`
 
 ---
 
 ### Which embedding models can be deployed locally?
 
-RAGFlow offers two Docker image editions, `v0.20.2-slim` and `v0.20.2`:  
+RAGFlow offers two Docker image editions, `v0.20.4-slim` and `v0.20.4`:  
   
-- `infiniflow/ragflow:v0.20.2-slim` (default): The RAGFlow Docker image without embedding models.  
-- `infiniflow/ragflow:v0.20.2`: The RAGFlow Docker image with embedding models including:
+- `infiniflow/ragflow:v0.20.4-slim` (default): The RAGFlow Docker image without embedding models.  
+- `infiniflow/ragflow:v0.20.4`: The RAGFlow Docker image with embedding models including:
   - Built-in embedding models:
     - `BAAI/bge-large-zh-v1.5`
     - `maidalun1020/bce-embedding-base_v1`

docs/guides/agent/agent_component_reference/agent.mdx

@@ -9,7 +9,7 @@ The component equipped with reasoning, tool usage, and multi-agent collaboration
 
 ---
 
-An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.2 onwards, an **Agent** component is able to work independently and with the following capabilities:
+An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.4 onwards, an **Agent** component is able to work independently and with the following capabilities:
 
 - Autonomous reasoning with reflection and adjustment based on environmental feedback.
 - Use of tools or subagents to complete tasks.

docs/guides/agent/agent_component_reference/retrieval.mdx

@@ -9,7 +9,7 @@ A component that retrieves information from specified datasets.
 
 ## Scenarios
 
-A **Retrieval** component is essential in most RAG scenarios, where information is extracted from designated knowledge bases before being sent to the LLM for content generation. As of v0.20.2, a **Retrieval** component can operate either as a workflow component or as a tool of an **Agent**, enabling the Agent to control its invocation and search queries.
+A **Retrieval** component is essential in most RAG scenarios, where information is extracted from designated knowledge bases before being sent to the LLM for content generation. As of v0.20.4, a **Retrieval** component can operate either as a workflow component or as a tool of an **Agent**, enabling the Agent to control its invocation and search queries.
 
 ## Configurations
 

docs/guides/chat/start_chat.md

@@ -48,7 +48,7 @@ You start an AI conversation by creating an assistant.
      - If no target language is selected, the system will search only in the language of your query, which may cause relevant information in other languages to be missed.
    - **Variable** refers to the variables (keys) to be used in the system prompt. `{knowledge}` is a reserved variable. Click **Add** to add more variables for the system prompt.
       - If you are uncertain about the logic behind **Variable**, leave it *as-is*.
-      - As of v0.20.2, if you add custom variables here, the only way you can pass in their values is to call:
+      - As of v0.20.4, if you add custom variables here, the only way you can pass in their values is to call:
          - HTTP method [Converse with chat assistant](../../references/http_api_reference.md#converse-with-chat-assistant), or
          - Python method [Converse with chat assistant](../../references/python_api_reference.md#converse-with-chat-assistant).
 

docs/guides/dataset/configure_knowledge_base.md

@@ -128,7 +128,7 @@ See [Run retrieval test](./run_retrieval_test.md) for details.
 
 ## Search for knowledge base
 
-As of RAGFlow v0.20.2, the search feature is still in a rudimentary form, supporting only knowledge base search by name.
+As of RAGFlow v0.20.4, the search feature is still in a rudimentary form, supporting only knowledge base search by name.
 
 ![search knowledge base](https://github.com/infiniflow/ragflow/assets/93570324/836ae94c-2438-42be-879e-c7ad2a59693e)
 

docs/guides/manage_files.md

@@ -87,4 +87,4 @@ RAGFlow's file management allows you to download an uploaded file:
 
 ![download_file](https://github.com/infiniflow/ragflow/assets/93570324/cf3b297f-7d9b-4522-bf5f-4f45743e4ed5)
 
-> As of RAGFlow v0.20.2, bulk download is not supported, nor can you download an entire folder. 
+> As of RAGFlow v0.20.4, bulk download is not supported, nor can you download an entire folder. 

docs/guides/tracing.mdx

@@ -18,7 +18,7 @@ RAGFlow ships with a built-in [Langfuse](https://langfuse.com) integration so th
 Langfuse stores traces, spans and prompt payloads in a purpose-built observability backend and offers filtering and visualisations on top.  
 
 :::info NOTE
-• RAGFlow **≥ 0.20.2** (contains the Langfuse connector)  
+• RAGFlow **≥ 0.20.4** (contains the Langfuse connector)  
 • A Langfuse workspace (cloud or self-hosted) with a _Project Public Key_ and _Secret Key_
 :::
 

docs/guides/upgrade_ragflow.mdx

@@ -66,10 +66,10 @@ To upgrade RAGFlow, you must upgrade **both** your code **and** your Docker imag
    git clone https://github.com/infiniflow/ragflow.git
    ```
 
-2. Switch to the latest, officially published release, e.g., `v0.20.2`:
+2. Switch to the latest, officially published release, e.g., `v0.20.4`:
 
    ```bash
-   git checkout -f v0.20.2
+   git checkout -f v0.20.4
    ```
 
 3. Update **ragflow/docker/.env**:
@@ -83,14 +83,14 @@ To upgrade RAGFlow, you must upgrade **both** your code **and** your Docker imag
   <TabItem value="slim">
 
 ```bash
-RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2-slim
+RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4-slim
 ```
 
   </TabItem>
   <TabItem value="full">
 
 ```bash
-RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2
+RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4
 ```
 
   </TabItem>
@@ -114,10 +114,10 @@ No, you do not need to. Upgrading RAGFlow in itself will *not* remove your uploa
 1. From an environment with Internet access, pull the required Docker image.
 2. Save the Docker image to a **.tar** file.
    ```bash
-   docker save -o ragflow.v0.20.2.tar infiniflow/ragflow:v0.20.2
+   docker save -o ragflow.v0.20.4.tar infiniflow/ragflow:v0.20.4
    ```
 3. Copy the **.tar** file to the target server.
 4. Load the **.tar** file into Docker:
    ```bash
-   docker load -i ragflow.v0.20.2.tar
+   docker load -i ragflow.v0.20.4.tar
    ```

docs/quickstart.mdx

@@ -44,7 +44,7 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
 
    `vm.max_map_count`. This value sets the maximum number of memory map areas a process may have. Its default value is 65530. While most applications require fewer than a thousand maps, reducing this value can result in abnormal behaviors, and the system will throw out-of-memory errors when a process reaches the limitation.
 
-   RAGFlow v0.20.2 uses Elasticsearch or [Infinity](https://github.com/infiniflow/infinity) for multiple recall. Setting the value of `vm.max_map_count` correctly is crucial to the proper functioning of the Elasticsearch component.
+   RAGFlow v0.20.4 uses Elasticsearch or [Infinity](https://github.com/infiniflow/infinity) for multiple recall. Setting the value of `vm.max_map_count` correctly is crucial to the proper functioning of the Elasticsearch component.
 
 <Tabs
   defaultValue="linux"
@@ -184,13 +184,13 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
    ```bash
    $ git clone https://github.com/infiniflow/ragflow.git
    $ cd ragflow/docker
-   $ git checkout -f v0.20.2
+   $ git checkout -f v0.20.4
    ```
 
 3. Use the pre-built Docker images and start up the server:
 
    :::tip NOTE
-   The command below downloads the `v0.20.2-slim` edition of the RAGFlow Docker image. Refer to the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.2-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.2` for the full edition `v0.20.2`.
+   The command below downloads the `v0.20.4-slim` edition of the RAGFlow Docker image. Refer to the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.4-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.4` for the full edition `v0.20.4`.
    :::
 
    ```bash
@@ -207,8 +207,8 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
 
 | RAGFlow image tag   | Image size (GB) | Has embedding models and Python packages? | Stable?                  |
 | ------------------- | --------------- | ----------------------------------------- | ------------------------ |
-| `v0.20.2`           | &approx;9       | :heavy_check_mark:                        | Stable release           |
-| `v0.20.2-slim`      | &approx;2       | ❌                                        | Stable release           |
+| `v0.20.4`           | &approx;9       | :heavy_check_mark:                        | Stable release           |
+| `v0.20.4-slim`      | &approx;2       | ❌                                        | Stable release           |
 | `nightly`           | &approx;9       | :heavy_check_mark:                        | *Unstable* nightly build |
 | `nightly-slim`      | &approx;2       | ❌                                        | *Unstable* nightly build |
 
@@ -217,7 +217,7 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
 ```
 
 :::danger IMPORTANT
-The embedding models included in `v0.20.2` and `nightly` are:
+The embedding models included in `v0.20.4` and `nightly` are:
 
 - BAAI/bge-large-zh-v1.5
 - maidalun1020/bce-embedding-base_v1

docs/references/glossary.mdx

@@ -19,7 +19,7 @@ import TOCInline from '@theme/TOCInline';
 
 ### Cross-language search
 
-Cross-language search (also known as cross-lingual retrieval) is a feature introduced in version 0.20.2. It enables users to submit queries in one language (for example, English) and retrieve relevant documents written in other languages such as Chinese or Spanish. This feature is enabled by the system’s default chat model, which translates queries to ensure accurate matching of semantic meaning across languages.
+Cross-language search (also known as cross-lingual retrieval) is a feature introduced in version 0.20.4. It enables users to submit queries in one language (for example, English) and retrieve relevant documents written in other languages such as Chinese or Spanish. This feature is enabled by the system’s default chat model, which translates queries to ensure accurate matching of semantic meaning across languages.
 
 By enabling cross-language search, users can effortlessly access a broader range of information regardless of language barriers, significantly enhancing the system’s usability and inclusiveness.
 

docs/references/http_api_reference.md

@@ -5,7 +5,7 @@ slug: /http_api_reference
 
 # HTTP API
 
-A complete reference for RAGFlow's RESTful API. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](../develop/acquire_ragflow_api_key.md).
+A complete reference for RAGFlow's RESTful API. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).
 
 ---
 
@@ -383,7 +383,7 @@ curl --request POST \
     - `"layout_recognize"`: `string`
       - Defaults to `DeepDOC`
     - `"tag_kb_ids"`: `array<string>` refer to [Use tag set](https://ragflow.io/docs/dev/use_tag_sets)
-      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunk Method
+      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunking Method
     - `"task_page_size"`: `int` For PDF only.
       - Defaults to `12`
       - Minimum: `1`
@@ -604,7 +604,7 @@ curl --request PUT \
     - `"layout_recognize"`: `string`
       - Defaults to `DeepDOC`
     - `"tag_kb_ids"`: `array<string>` refer to [Use tag set](https://ragflow.io/docs/dev/use_tag_sets)
-      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunk Method
+      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunking Method
     - `"task_page_size"`: `int` For PDF only.
       - Defaults to `12`
       - Minimum: `1`
@@ -731,7 +731,7 @@ Failure:
 ```
  ---
 
-## Get dataset's knowledge graph
+### Get knowledge graph
 
 **GET** `/api/v1/datasets/{dataset_id}/knowledge_graph`
 
@@ -810,7 +810,7 @@ Failure:
 ```
 ---
 
-## Delete dataset's knowledge graph
+### Delete knowledge graph
 
 **DELETE** `/api/v1/datasets/{dataset_id}/knowledge_graph`
 
@@ -3501,7 +3501,7 @@ Failure:
 
 ### Generate related questions
 
-**POST** `/v1/sessions/related_questions`
+**POST** `/api/v1/sessions/related_questions`
 
 Generates five to ten alternative question strings from the user's original query to retrieve more relevant search results.
 
@@ -3516,7 +3516,7 @@ The chat model autonomously determines the number of questions to generate based
 #### Request
 
 - Method: POST
-- URL: `/v1/sessions/related_questions`
+- URL: `/api/v1/sessions/related_questions`
 - Headers:
   - `'content-Type: application/json'`
   - `'Authorization: Bearer <YOUR_LOGIN_TOKEN>'`
@@ -3528,7 +3528,7 @@ The chat model autonomously determines the number of questions to generate based
 
 ```bash
 curl --request POST \
-     --url http://{address}/v1/sessions/related_questions \
+     --url http://{address}/api/v1/sessions/related_questions \
      --header 'Content-Type: application/json' \
      --header 'Authorization: Bearer <YOUR_LOGIN_TOKEN>' \
      --data '

docs/references/python_api_reference.md

@@ -5,7 +5,7 @@ slug: /python_api_reference
 
 # Python API
 
-A complete reference for RAGFlow's Python APIs. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](../develop/acquire_ragflow_api_key.md).
+A complete reference for RAGFlow's Python APIs. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).
 
 :::tip NOTE
 Run the following command to download the Python SDK:

docs/release_notes.md

@@ -9,8 +9,8 @@ Key features, improvements and bug fixes in the latest releases.
 
 :::info
 Each RAGFlow release is available in two editions:
-- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.1-slim`
-- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.1`
+- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.4-slim`
+- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.4`
 :::
 
 :::danger IMPORTANT
@@ -22,9 +22,41 @@ The embedding models included in a full edition are:
 These two embedding models are optimized specifically for English and Chinese, so performance may be compromised if you use them to embed documents in other languages.
 :::
 
-## v0.20.2
+## v0.20.4
 
-Released on August 19, 2025.
+Released on August 27, 2025.
+
+### Improvements
+
+- Agent component: Completes Chinese localization for the Agent component.
+- Introduces the `ENABLE_TIMEOUT_ASSERTION` environment variable to enable or disable timeout assertions for file parsing tasks.
+- Dataset:
+  - Improves Markdown file parsing, with AST support to avoid unintended chunking.
+  - Enhances HTML parsing, supporting bs4-based HTML tag traversal.
+
+### Added models
+
+ZHIPU GLM-4.5
+
+### New Agent templates
+
+Ecommerce Customer Service Workflow: A template designed to handle enquiries about product features and multi-product comparisons using the internal knowledge base, as well as to manage installation appointment bookings.
+
+### Fixed issues
+
+- Dataset:  
+  - Unable to share resources with the team.
+  - Inappropriate restrictions on the number and size of uploaded files.
+- Chat:
+  - Unable to preview referenced files in responses.
+  - Unable to send out messages after file uploads.
+- An OAuth2 authentication failure.
+- A logical error in multi-conditioned metadata searches within a dataset.
+- Citations infinitely increased in multi-turn conversations.
+
+## v0.20.3
+
+Released on August 20, 2025.
 
 ### Improvements
 

graphrag/entity_resolution.py

@@ -15,6 +15,7 @@
 #
 import logging
 import itertools
+import os
 import re
 from dataclasses import dataclass
 from typing import Any, Callable
@@ -106,7 +107,8 @@ class EntityResolution(Extractor):
             nonlocal remain_candidates_to_resolve, callback
             async with semaphore:
                 try:
-                    with trio.move_on_after(280) as cancel_scope:
+                    enable_timeout_assertion = os.environ.get("ENABLE_TIMEOUT_ASSERTION")
+                    with trio.move_on_after(280 if enable_timeout_assertion else 1000000000) as cancel_scope:
                         await self._resolve_candidate(candidate_batch, result_set, result_lock)
                         remain_candidates_to_resolve = remain_candidates_to_resolve - len(candidate_batch[1])
                         callback(msg=f"Resolved {len(candidate_batch[1])} pairs, {remain_candidates_to_resolve} are remained to resolve. ")
@@ -169,7 +171,8 @@ class EntityResolution(Extractor):
         logging.info(f"Created resolution prompt {len(text)} bytes for {len(candidate_resolution_i[1])} entity pairs of type {candidate_resolution_i[0]}")
         async with chat_limiter:
             try:
-                with trio.move_on_after(240) as cancel_scope:
+                enable_timeout_assertion = os.environ.get("ENABLE_TIMEOUT_ASSERTION")
+                with trio.move_on_after(280 if enable_timeout_assertion else 1000000000) as cancel_scope:
                     response = await trio.to_thread.run_sync(self._chat, text, [{"role": "user", "content": "Output:"}], {})
                 if cancel_scope.cancelled_caught:
                     logging.warning("_resolve_candidate._chat timeout, skipping...")

graphrag/general/community_reports_extractor.py

@@ -7,6 +7,7 @@ Reference:
 
 import logging
 import json
+import os
 import re
 from typing import Callable
 from dataclasses import dataclass
@@ -51,6 +52,7 @@ class CommunityReportsExtractor(Extractor):
         self._max_report_length = max_report_length or 1500
 
     async def __call__(self, graph: nx.Graph, callback: Callable | None = None):
+        enable_timeout_assertion = os.environ.get("ENABLE_TIMEOUT_ASSERTION")
         for node_degree in graph.degree:
             graph.nodes[str(node_degree[0])]["rank"] = int(node_degree[1])
 
@@ -92,7 +94,7 @@ class CommunityReportsExtractor(Extractor):
             text = perform_variable_replacements(self._extraction_prompt, variables=prompt_variables)
             async with chat_limiter:
                 try:
-                    with trio.move_on_after(180) as cancel_scope:
+                    with trio.move_on_after(180 if enable_timeout_assertion else 1000000000) as cancel_scope:
                         response = await trio.to_thread.run_sync( self._chat, text, [{"role": "user", "content": "Output:"}], {})
                     if cancel_scope.cancelled_caught:
                         logging.warning("extract_community_report._chat timeout, skipping...")

graphrag/general/extractor.py

@@ -47,7 +47,7 @@ class Extractor:
         self._language = language
         self._entity_types = entity_types or DEFAULT_ENTITY_TYPES
 
-    @timeout(60*5)
+    @timeout(60*20)
     def _chat(self, system, history, gen_conf={}):
         hist = deepcopy(history)
         conf = deepcopy(gen_conf)

graphrag/general/index.py

@@ -15,6 +15,8 @@
 #
 import json
 import logging
+import os
+
 import networkx as nx
 import trio
 
@@ -49,6 +51,7 @@ async def run_graphrag(
     embedding_model,
     callback,
 ):
+    enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
     start = trio.current_time()
     tenant_id, kb_id, doc_id = row["tenant_id"], str(row["kb_id"]), row["doc_id"]
     chunks = []
@@ -57,7 +60,7 @@ async def run_graphrag(
     ):
         chunks.append(d["content_with_weight"])
 
-    with trio.fail_after(max(120, len(chunks)*120)):
+    with trio.fail_after(max(120, len(chunks)*60*10) if enable_timeout_assertion else 10000000000):
         subgraph = await generate_subgraph(
             LightKGExt
             if "method" not in row["kb_parser_config"].get("graphrag", {}) or row["kb_parser_config"]["graphrag"]["method"] != "general"

graphrag/light/graph_prompt.py

@@ -130,7 +130,36 @@ Output:
 
 PROMPTS[
     "entiti_continue_extraction"
-] = """MANY entities were missed in the last extraction.  Add them below using the same format:
+] = """
+MANY entities and relationships were missed in the last extraction. Please find only the missing entities and relationships from previous text.
+
+---Remember Steps---
+
+1. Identify all entities. For each identified entity, extract the following information:
+- entity_name: Name of the entity, use same language as input text. If English, capitalized the name
+- entity_type: One of the following types: [{entity_types}]
+- entity_description: Provide a comprehensive description of the entity's attributes and activities *based solely on the information present in the input text*. **Do not infer or hallucinate information not explicitly stated.** If the text provides insufficient information to create a comprehensive description, state "Description not available in text."
+Format each entity as ("entity"{tuple_delimiter}<entity_name>{tuple_delimiter}<entity_type>{tuple_delimiter}<entity_description>)
+
+2. From the entities identified in step 1, identify all pairs of (source_entity, target_entity) that are *clearly related* to each other.
+For each pair of related entities, extract the following information:
+- source_entity: name of the source entity, as identified in step 1
+- target_entity: name of the target entity, as identified in step 1
+- relationship_description: explanation as to why you think the source entity and the target entity are related to each other
+- relationship_strength: a numeric score indicating strength of the relationship between the source entity and target entity
+- relationship_keywords: one or more high-level key words that summarize the overarching nature of the relationship, focusing on concepts or themes rather than specific details
+Format each relationship as ("relationship"{tuple_delimiter}<source_entity>{tuple_delimiter}<target_entity>{tuple_delimiter}<relationship_description>{tuple_delimiter}<relationship_keywords>{tuple_delimiter}<relationship_strength>)
+
+3. Identify high-level key words that summarize the main concepts, themes, or topics of the entire text. These should capture the overarching ideas present in the document.
+Format the content-level key words as ("content_keywords"{tuple_delimiter}<high_level_keywords>)
+
+4. Return output in {language} as a single list of all the entities and relationships identified in steps 1 and 2. Use **{record_delimiter}** as the list delimiter.
+
+5. When finished, output {completion_delimiter}
+
+---Output---
+
+Add new entities and relations below using the same format, and do not include entities and relations that have been previously extracted. :
 """
 
 PROMPTS[
@@ -252,4 +281,4 @@ When handling information with timestamps:
 - List up to 5 most important reference sources at the end under "References", clearly indicating whether each source is from Knowledge Graph (KG) or Vector Data (VD)
   Format: [KG/VD] Source content
 
-Add sections and commentary to the response as appropriate for the length and format. If the provided information is insufficient to answer the question, clearly state that you don't know or cannot provide an answer in the same language as the user's question."""
+Add sections and commentary to the response as appropriate for the length and format. If the provided information is insufficient to answer the question, clearly state that you don't know or cannot provide an answer in the same language as the user's question."""

graphrag/utils.py

@@ -307,6 +307,7 @@ def chunk_id(chunk):
 
 async def graph_node_to_chunk(kb_id, embd_mdl, ent_name, meta, chunks):
     global chat_limiter
+    enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
     chunk = {
         "id": get_uuid(),
         "important_kwd": [ent_name],
@@ -324,7 +325,7 @@ async def graph_node_to_chunk(kb_id, embd_mdl, ent_name, meta, chunks):
     ebd = get_embed_cache(embd_mdl.llm_name, ent_name)
     if ebd is None:
         async with chat_limiter:
-            with trio.fail_after(3):
+            with trio.fail_after(3 if enable_timeout_assertion else 30000000):
                 ebd, _ = await trio.to_thread.run_sync(lambda: embd_mdl.encode([ent_name]))
         ebd = ebd[0]
         set_embed_cache(embd_mdl.llm_name, ent_name, ebd)
@@ -362,6 +363,7 @@ def get_relation(tenant_id, kb_id, from_ent_name, to_ent_name, size=1):
 
 
 async def graph_edge_to_chunk(kb_id, embd_mdl, from_ent_name, to_ent_name, meta, chunks):
+    enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
     chunk = {
         "id": get_uuid(),
         "from_entity_kwd": from_ent_name,
@@ -380,7 +382,7 @@ async def graph_edge_to_chunk(kb_id, embd_mdl, from_ent_name, to_ent_name, meta,
     ebd = get_embed_cache(embd_mdl.llm_name, txt)
     if ebd is None:
         async with chat_limiter:
-            with trio.fail_after(3):
+            with trio.fail_after(3 if enable_timeout_assertion else 300000000):
                 ebd, _ = await trio.to_thread.run_sync(lambda: embd_mdl.encode([txt+f": {meta['description']}"]))
         ebd = ebd[0]
         set_embed_cache(embd_mdl.llm_name, txt, ebd)
@@ -514,9 +516,10 @@ async def set_graph(tenant_id: str, kb_id: str, embd_mdl, graph: nx.Graph, chang
         callback(msg=f"set_graph converted graph change to {len(chunks)} chunks in {now - start:.2f}s.")
     start = now
 
+    enable_timeout_assertion=os.environ.get("ENABLE_TIMEOUT_ASSERTION")
     es_bulk_size = 4
     for b in range(0, len(chunks), es_bulk_size):
-        with trio.fail_after(3):
+        with trio.fail_after(3 if enable_timeout_assertion else 30000000):
             doc_store_result = await trio.to_thread.run_sync(lambda: settings.docStoreConn.insert(chunks[b:b + es_bulk_size], search.index_name(tenant_id), kb_id))
         if b % 100 == es_bulk_size and callback:
             callback(msg=f"Insert chunks: {b}/{len(chunks)}")

helm/values.yaml

@@ -56,7 +56,7 @@ env:
 ragflow:
   image:
     repository: infiniflow/ragflow
-    tag: v0.20.2-slim
+    tag: v0.20.4-slim
     pullPolicy: IfNotPresent
     pullSecrets: []
   # Optional service configuration overrides

mcp/server/server.py

@@ -16,6 +16,9 @@
 
 import json
 import logging
+import random
+import time
+from collections import OrderedDict
 from collections.abc import AsyncIterator
 from contextlib import asynccontextmanager
 from functools import wraps
@@ -53,6 +56,13 @@ JSON_RESPONSE = True
 
 
 class RAGFlowConnector:
+    _MAX_DATASET_CACHE = 32
+    _MAX_DOCUMENT_CACHE = 128
+    _CACHE_TTL = 300
+
+    _dataset_metadata_cache: OrderedDict[str, tuple[dict, float | int]] = OrderedDict()  # "dataset_id" -> (metadata, expiry_ts)
+    _document_metadata_cache: OrderedDict[str, tuple[list[tuple[str, dict]], float | int]] = OrderedDict()  # "dataset_id" -> ([(document_id, doc_metadata)], expiry_ts)
+
     def __init__(self, base_url: str, version="v1"):
         self.base_url = base_url
         self.version = version
@@ -72,6 +82,43 @@ class RAGFlowConnector:
         res = requests.get(url=self.api_url + path, params=params, headers=self.authorization_header, json=json)
         return res
 
+    def _is_cache_valid(self, ts):
+        return time.time() < ts
+
+    def _get_expiry_timestamp(self):
+        offset = random.randint(-30, 30)
+        return time.time() + self._CACHE_TTL + offset
+
+    def _get_cached_dataset_metadata(self, dataset_id):
+        entry = self._dataset_metadata_cache.get(dataset_id)
+        if entry:
+            data, ts = entry
+            if self._is_cache_valid(ts):
+                self._dataset_metadata_cache.move_to_end(dataset_id)
+                return data
+        return None
+
+    def _set_cached_dataset_metadata(self, dataset_id, metadata):
+        self._dataset_metadata_cache[dataset_id] = (metadata, self._get_expiry_timestamp())
+        self._dataset_metadata_cache.move_to_end(dataset_id)
+        if len(self._dataset_metadata_cache) > self._MAX_DATASET_CACHE:
+            self._dataset_metadata_cache.popitem(last=False)
+
+    def _get_cached_document_metadata_by_dataset(self, dataset_id):
+        entry = self._document_metadata_cache.get(dataset_id)
+        if entry:
+            data_list, ts = entry
+            if self._is_cache_valid(ts):
+                self._document_metadata_cache.move_to_end(dataset_id)
+                return {doc_id: doc_meta for doc_id, doc_meta in data_list}
+        return None
+
+    def _set_cached_document_metadata_by_dataset(self, dataset_id, doc_id_meta_list):
+        self._document_metadata_cache[dataset_id] = (doc_id_meta_list, self._get_expiry_timestamp())
+        self._document_metadata_cache.move_to_end(dataset_id)
+        if len(self._document_metadata_cache) > self._MAX_DOCUMENT_CACHE:
+            self._document_metadata_cache.popitem(last=False)
+
     def list_datasets(self, page: int = 1, page_size: int = 1000, orderby: str = "create_time", desc: bool = True, id: str | None = None, name: str | None = None):
         res = self._get("/datasets", {"page": page, "page_size": page_size, "orderby": orderby, "desc": desc, "id": id, "name": name})
         if not res:
@@ -87,10 +134,38 @@ class RAGFlowConnector:
         return ""
 
     def retrieval(
-        self, dataset_ids, document_ids=None, question="", page=1, page_size=30, similarity_threshold=0.2, vector_similarity_weight=0.3, top_k=1024, rerank_id: str | None = None, keyword: bool = False
+        self,
+        dataset_ids,
+        document_ids=None,
+        question="",
+        page=1,
+        page_size=30,
+        similarity_threshold=0.2,
+        vector_similarity_weight=0.3,
+        top_k=1024,
+        rerank_id: str | None = None,
+        keyword: bool = False,
+        force_refresh: bool = False,
     ):
         if document_ids is None:
             document_ids = []
+        
+        # If no dataset_ids provided or empty list, get all available dataset IDs
+        if not dataset_ids:
+            dataset_list_str = self.list_datasets()
+            dataset_ids = []
+            
+            # Parse the dataset list to extract IDs
+            if dataset_list_str:
+                for line in dataset_list_str.strip().split('\n'):
+                    if line.strip():
+                        try:
+                            dataset_info = json.loads(line.strip())
+                            dataset_ids.append(dataset_info["id"])
+                        except (json.JSONDecodeError, KeyError):
+                            # Skip malformed lines
+                            continue
+        
         data_json = {
             "page": page,
             "page_size": page_size,
@@ -110,12 +185,127 @@ class RAGFlowConnector:
 
         res = res.json()
         if res.get("code") == 0:
+            data = res["data"]
             chunks = []
-            for chunk_data in res["data"].get("chunks"):
-                chunks.append(json.dumps(chunk_data, ensure_ascii=False))
-            return [types.TextContent(type="text", text="\n".join(chunks))]
+
+            # Cache document metadata and dataset information
+            document_cache, dataset_cache = self._get_document_metadata_cache(dataset_ids, force_refresh=force_refresh)
+
+            # Process chunks with enhanced field mapping including per-chunk metadata
+            for chunk_data in data.get("chunks", []):
+                enhanced_chunk = self._map_chunk_fields(chunk_data, dataset_cache, document_cache)
+                chunks.append(enhanced_chunk)
+
+            # Build structured response (no longer need response-level document_metadata)
+            response = {
+                "chunks": chunks,
+                "pagination": {
+                    "page": data.get("page", page),
+                    "page_size": data.get("page_size", page_size),
+                    "total_chunks": data.get("total", len(chunks)),
+                    "total_pages": (data.get("total", len(chunks)) + page_size - 1) // page_size,
+                },
+                "query_info": {
+                    "question": question,
+                    "similarity_threshold": similarity_threshold,
+                    "vector_weight": vector_similarity_weight,
+                    "keyword_search": keyword,
+                    "dataset_count": len(dataset_ids),
+                },
+            }
+
+            return [types.TextContent(type="text", text=json.dumps(response, ensure_ascii=False))]
+
         raise Exception([types.TextContent(type="text", text=res.get("message"))])
 
+    def _get_document_metadata_cache(self, dataset_ids, force_refresh=False):
+        """Cache document metadata for all documents in the specified datasets"""
+        document_cache = {}
+        dataset_cache = {}
+
+        try:
+            for dataset_id in dataset_ids:
+                dataset_meta = None if force_refresh else self._get_cached_dataset_metadata(dataset_id)
+                if not dataset_meta:
+                    # First get dataset info for name
+                    dataset_res = self._get("/datasets", {"id": dataset_id, "page_size": 1})
+                    if dataset_res and dataset_res.status_code == 200:
+                        dataset_data = dataset_res.json()
+                        if dataset_data.get("code") == 0 and dataset_data.get("data"):
+                            dataset_info = dataset_data["data"][0]
+                            dataset_meta = {"name": dataset_info.get("name", "Unknown"), "description": dataset_info.get("description", "")}
+                            self._set_cached_dataset_metadata(dataset_id, dataset_meta)
+                if dataset_meta:
+                    dataset_cache[dataset_id] = dataset_meta
+
+                docs = None if force_refresh else self._get_cached_document_metadata_by_dataset(dataset_id)
+                if docs is None:
+                    docs_res = self._get(f"/datasets/{dataset_id}/documents")
+                    docs_data = docs_res.json()
+                    if docs_data.get("code") == 0 and docs_data.get("data", {}).get("docs"):
+                        doc_id_meta_list = []
+                        docs = {}
+                        for doc in docs_data["data"]["docs"]:
+                            doc_id = doc.get("id")
+                            if not doc_id:
+                                continue
+                            doc_meta = {
+                                "document_id": doc_id,
+                                "name": doc.get("name", ""),
+                                "location": doc.get("location", ""),
+                                "type": doc.get("type", ""),
+                                "size": doc.get("size"),
+                                "chunk_count": doc.get("chunk_count"),
+                                # "chunk_method": doc.get("chunk_method", ""),
+                                "create_date": doc.get("create_date", ""),
+                                "update_date": doc.get("update_date", ""),
+                                # "process_begin_at": doc.get("process_begin_at", ""),
+                                # "process_duration": doc.get("process_duration"),
+                                # "progress": doc.get("progress"),
+                                # "progress_msg": doc.get("progress_msg", ""),
+                                # "status": doc.get("status", ""),
+                                # "run": doc.get("run", ""),
+                                "token_count": doc.get("token_count"),
+                                # "source_type": doc.get("source_type", ""),
+                                "thumbnail": doc.get("thumbnail", ""),
+                                "dataset_id": doc.get("dataset_id", dataset_id),
+                                "meta_fields": doc.get("meta_fields", {}),
+                                # "parser_config": doc.get("parser_config", {})
+                            }
+                            doc_id_meta_list.append((doc_id, doc_meta))
+                            docs[doc_id] = doc_meta
+                        self._set_cached_document_metadata_by_dataset(dataset_id, doc_id_meta_list)
+                if docs:
+                    document_cache.update(docs)
+
+        except Exception:
+            # Gracefully handle metadata cache failures
+            pass
+
+        return document_cache, dataset_cache
+
+    def _map_chunk_fields(self, chunk_data, dataset_cache, document_cache):
+        """Preserve all original API fields and add per-chunk document metadata"""
+        # Start with ALL raw data from API (preserve everything like original version)
+        mapped = dict(chunk_data)
+
+        # Add dataset name enhancement
+        dataset_id = chunk_data.get("dataset_id") or chunk_data.get("kb_id")
+        if dataset_id and dataset_id in dataset_cache:
+            mapped["dataset_name"] = dataset_cache[dataset_id]["name"]
+        else:
+            mapped["dataset_name"] = "Unknown"
+
+        # Add document name convenience field
+        mapped["document_name"] = chunk_data.get("document_keyword", "")
+
+        # Add per-chunk document metadata
+        document_id = chunk_data.get("document_id")
+        if document_id and document_id in document_cache:
+            mapped["document_metadata"] = document_cache[document_id]
+
+        return mapped
+
 
 class RAGFlowCtx:
     def __init__(self, connector: RAGFlowConnector):
@@ -195,7 +385,58 @@ async def list_tools(*, connector) -> list[types.Tool]:
                         "items": {"type": "string"},
                         "description": "Optional array of document IDs to search within."
                     },
-                    "question": {"type": "string", "description": "The question or query to search for."},
+                    "question": {
+                        "type": "string",
+                        "description": "The question or query to search for."
+                    },
+                    "page": {
+                        "type": "integer",
+                        "description": "Page number for pagination",
+                        "default": 1,
+                        "minimum": 1,
+                    },
+                    "page_size": {
+                        "type": "integer",
+                        "description": "Number of results to return per page (default: 10, max recommended: 50 to avoid token limits)",
+                        "default": 10,
+                        "minimum": 1,
+                        "maximum": 100,
+                    },
+                    "similarity_threshold": {
+                        "type": "number",
+                        "description": "Minimum similarity threshold for results",
+                        "default": 0.2,
+                        "minimum": 0.0,
+                        "maximum": 1.0,
+                    },
+                    "vector_similarity_weight": {
+                        "type": "number",
+                        "description": "Weight for vector similarity vs term similarity",
+                        "default": 0.3,
+                        "minimum": 0.0,
+                        "maximum": 1.0,
+                    },
+                    "keyword": {
+                        "type": "boolean",
+                        "description": "Enable keyword-based search",
+                        "default": False,
+                    },
+                    "top_k": {
+                        "type": "integer",
+                        "description": "Maximum results to consider before ranking",
+                        "default": 1024,
+                        "minimum": 1,
+                        "maximum": 1024,
+                    },
+                    "rerank_id": {
+                        "type": "string",
+                        "description": "Optional reranking model identifier",
+                    },
+                    "force_refresh": {
+                        "type": "boolean",
+                        "description": "Set to true only if fresh dataset and document metadata is explicitly required. Otherwise, cached metadata is used (default: false).",
+                        "default": False,
+                    },
                 },
                 "required": ["question"],
             },
@@ -209,6 +450,16 @@ async def call_tool(name: str, arguments: dict, *, connector) -> list[types.Text
     if name == "ragflow_retrieval":
         document_ids = arguments.get("document_ids", [])
         dataset_ids = arguments.get("dataset_ids", [])
+        question = arguments.get("question", "")
+        page = arguments.get("page", 1)
+        page_size = arguments.get("page_size", 10)
+        similarity_threshold = arguments.get("similarity_threshold", 0.2)
+        vector_similarity_weight = arguments.get("vector_similarity_weight", 0.3)
+        keyword = arguments.get("keyword", False)
+        top_k = arguments.get("top_k", 1024)
+        rerank_id = arguments.get("rerank_id")
+        force_refresh = arguments.get("force_refresh", False)
+
         
         # If no dataset_ids provided or empty list, get all available dataset IDs
         if not dataset_ids:
@@ -229,7 +480,15 @@ async def call_tool(name: str, arguments: dict, *, connector) -> list[types.Text
         return connector.retrieval(
             dataset_ids=dataset_ids,
             document_ids=document_ids,
-            question=arguments["question"],
+            question=question,
+            page=page,
+            page_size=page_size,
+            similarity_threshold=similarity_threshold,
+            vector_similarity_weight=vector_similarity_weight,
+            keyword=keyword,
+            top_k=top_k,
+            rerank_id=rerank_id,
+            force_refresh=force_refresh,
         )
     raise ValueError(f"Tool not found: {name}")
 

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ragflow"
-version = "0.20.2"
+version = "0.20.4"
 description = "[RAGFlow](https://ragflow.io/) is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding. It offers a streamlined RAG workflow for businesses of any scale, combining LLM (Large Language Models) to provide truthful question-answering capabilities, backed by well-founded citations from various complex formatted data."
 authors = [{ name = "Zhichang Yu", email = "yuzhichang@gmail.com" }]
 license-files = ["LICENSE"]
@@ -45,7 +45,7 @@ dependencies = [
     "html-text==0.6.2",
     "httpx[socks]==0.27.2",
     "huggingface-hub>=0.25.0,<0.26.0",
-    "infinity-sdk==0.6.0-dev4",
+    "infinity-sdk==0.6.0.dev5",
     "infinity-emb>=0.0.66,<0.0.67",
     "itsdangerous==2.1.2",
     "json-repair==0.35.0",

rag/app/naive.py

@@ -30,7 +30,7 @@ from tika import parser
 
 from api.db import LLMType
 from api.db.services.llm_service import LLMBundle
-from deepdoc.parser import DocxParser, ExcelParser, HtmlParser, JsonParser, MarkdownParser, PdfParser, TxtParser
+from deepdoc.parser import DocxParser, ExcelParser, HtmlParser, JsonParser, MarkdownElementExtractor, MarkdownParser, PdfParser, TxtParser
 from deepdoc.parser.figure_parser import VisionFigureParser, vision_figure_parser_figure_data_wrapper
 from deepdoc.parser.pdf_parser import PlainParser, VisionParser
 from rag.nlp import concat_img, find_codec, naive_merge, naive_merge_with_images, naive_merge_docx, rag_tokenizer, tokenize_chunks, tokenize_chunks_with_images, tokenize_table
@@ -289,7 +289,7 @@ class Pdf(PdfParser):
             return [(b["text"], self._line_tag(b, zoomin)) for b in self.boxes], tbls, figures
         else:
             tbls = self._extract_table_figure(True, zoomin, True, True)
-            # self._naive_vertical_merge()
+            self._naive_vertical_merge()
             self._concat_downward()
             # self._filter_forpages()
             logging.info("layouts cost: {}s".format(timer() - first_start))
@@ -350,17 +350,14 @@ class Markdown(MarkdownParser):
         else:
             with open(filename, "r") as f:
                 txt = f.read()
+
         remainder, tables = self.extract_tables_and_remainder(f'{txt}\n', separate_tables=separate_tables)
-        sections = []
+
+        extractor = MarkdownElementExtractor(txt)
+        element_sections = extractor.extract_elements()
+        sections = [(element, "") for element in element_sections]
+
         tbls = []
-        for sec in remainder.split("\n"):
-            if sec.strip().find("#") == 0:
-                sections.append((sec, ""))
-            elif sections and sections[-1][0].strip().find("#") == 0:
-                sec_, _ = sections.pop(-1)
-                sections.append((sec_ + "\n" + sec, ""))
-            else:
-                sections.append((sec, ""))
         for table in tables:
             tbls.append(((None, markdown(table, extensions=['markdown.extensions.tables'])), ""))
         return sections, tbls
@@ -520,7 +517,8 @@ def chunk(filename, binary=None, from_page=0, to_page=100000,
 
     elif re.search(r"\.(htm|html)$", filename, re.IGNORECASE):
         callback(0.1, "Start to parse.")
-        sections = HtmlParser()(filename, binary)
+        chunk_token_num = int(parser_config.get("chunk_token_num", 128))
+        sections = HtmlParser()(filename, binary, chunk_token_num)
         sections = [(_, "") for _ in sections if _]
         callback(0.8, "Finish parsing.")
 

rag/llm/__init__.py

@@ -36,6 +36,7 @@ class SupportedLiteLLMProvider(StrEnum):
     Nvidia = "NVIDIA"
     TogetherAI = "TogetherAI"
     Anthropic = "Anthropic"
+    Ollama = "Ollama"
 
 
 FACTORY_DEFAULT_BASE_URL = {
@@ -59,6 +60,7 @@ LITELLM_PROVIDER_PREFIX = {
     SupportedLiteLLMProvider.Nvidia: "nvidia_nim/",
     SupportedLiteLLMProvider.TogetherAI: "together_ai/",
     SupportedLiteLLMProvider.Anthropic: "",  # don't need a prefix
+    SupportedLiteLLMProvider.Ollama: "ollama_chat/",
 }
 
 ChatModel = globals().get("ChatModel", {})

rag/llm/chat_model.py

@@ -29,7 +29,6 @@ import json_repair
 import litellm
 import openai
 import requests
-from ollama import Client
 from openai import OpenAI
 from openai.lib.azure import AzureOpenAI
 from strenum import StrEnum
@@ -112,6 +111,32 @@ class Base(ABC):
     def _clean_conf(self, gen_conf):
         if "max_tokens" in gen_conf:
             del gen_conf["max_tokens"]
+
+        allowed_conf = {
+            "temperature",
+            "max_completion_tokens",
+            "top_p",
+            "stream",
+            "stream_options",
+            "stop",
+            "n",
+            "presence_penalty",
+            "frequency_penalty",
+            "functions",
+            "function_call",
+            "logit_bias",
+            "user",
+            "response_format",
+            "seed",
+            "tools",
+            "tool_choice",
+            "logprobs",
+            "top_logprobs",
+            "extra_headers",
+        }
+
+        gen_conf = {k: v for k, v in gen_conf.items() if k in allowed_conf}
+
         return gen_conf
 
     def _chat(self, history, gen_conf, **kwargs):
@@ -657,73 +682,6 @@ class ZhipuChat(Base):
         return super().chat_streamly_with_tools(system, history, gen_conf)
 
 
-class OllamaChat(Base):
-    _FACTORY_NAME = "Ollama"
-
-    def __init__(self, key, model_name, base_url=None, **kwargs):
-        super().__init__(key, model_name, base_url=base_url, **kwargs)
-
-        self.client = Client(host=base_url) if not key or key == "x" else Client(host=base_url, headers={"Authorization": f"Bearer {key}"})
-        self.model_name = model_name
-        self.keep_alive = kwargs.get("ollama_keep_alive", int(os.environ.get("OLLAMA_KEEP_ALIVE", -1)))
-
-    def _clean_conf(self, gen_conf):
-        options = {}
-        if "max_tokens" in gen_conf:
-            options["num_predict"] = gen_conf["max_tokens"]
-        for k in ["temperature", "top_p", "presence_penalty", "frequency_penalty"]:
-            if k not in gen_conf:
-                continue
-            options[k] = gen_conf[k]
-        return options
-
-    def _chat(self, history, gen_conf={}, **kwargs):
-        # Calculate context size
-        ctx_size = self._calculate_dynamic_ctx(history)
-
-        gen_conf["num_ctx"] = ctx_size
-        response = self.client.chat(model=self.model_name, messages=history, options=gen_conf, keep_alive=self.keep_alive)
-        ans = response["message"]["content"].strip()
-        token_count = response.get("eval_count", 0) + response.get("prompt_eval_count", 0)
-        return ans, token_count
-
-    def chat_streamly(self, system, history, gen_conf={}, **kwargs):
-        if system:
-            history.insert(0, {"role": "system", "content": system})
-        if "max_tokens" in gen_conf:
-            del gen_conf["max_tokens"]
-        try:
-            # Calculate context size
-            ctx_size = self._calculate_dynamic_ctx(history)
-            options = {"num_ctx": ctx_size}
-            if "temperature" in gen_conf:
-                options["temperature"] = gen_conf["temperature"]
-            if "max_tokens" in gen_conf:
-                options["num_predict"] = gen_conf["max_tokens"]
-            if "top_p" in gen_conf:
-                options["top_p"] = gen_conf["top_p"]
-            if "presence_penalty" in gen_conf:
-                options["presence_penalty"] = gen_conf["presence_penalty"]
-            if "frequency_penalty" in gen_conf:
-                options["frequency_penalty"] = gen_conf["frequency_penalty"]
-
-            ans = ""
-            try:
-                response = self.client.chat(model=self.model_name, messages=history, stream=True, options=options, keep_alive=self.keep_alive)
-                for resp in response:
-                    if resp["done"]:
-                        token_count = resp.get("prompt_eval_count", 0) + resp.get("eval_count", 0)
-                        yield token_count
-                    ans = resp["message"]["content"]
-                    yield ans
-            except Exception as e:
-                yield ans + "\n**ERROR**: " + str(e)
-            yield 0
-        except Exception as e:
-            yield "**ERROR**: " + str(e)
-            yield 0
-
-
 class LocalAIChat(Base):
     _FACTORY_NAME = "LocalAI"
 
@@ -1396,7 +1354,7 @@ class Ai302Chat(Base):
 
 
 class LiteLLMBase(ABC):
-    _FACTORY_NAME = ["Tongyi-Qianwen", "Bedrock", "Moonshot", "xAI", "DeepInfra", "Groq", "Cohere", "Gemini", "DeepSeek", "NVIDIA", "TogetherAI", "Anthropic"]
+    _FACTORY_NAME = ["Tongyi-Qianwen", "Bedrock", "Moonshot", "xAI", "DeepInfra", "Groq", "Cohere", "Gemini", "DeepSeek", "NVIDIA", "TogetherAI", "Anthropic", "Ollama"]
 
     def __init__(self, key, model_name, base_url=None, **kwargs):
         self.timeout = int(os.environ.get("LM_TIMEOUT_SECONDS", 600))

rag/llm/rerank_model.py

@@ -44,14 +44,17 @@ class Base(ABC):
         raise NotImplementedError("Please implement encode method!")
 
     def total_token_count(self, resp):
-        try:
-            return resp.usage.total_tokens
-        except Exception:
-            pass
-        try:
-            return resp["usage"]["total_tokens"]
-        except Exception:
-            pass
+        if hasattr(resp, "usage") and hasattr(resp.usage, "total_tokens"):
+            try:
+                return resp.usage.total_tokens
+            except Exception:
+                pass
+
+        if 'usage' in resp and 'total_tokens' in resp['usage']:
+            try:
+                return resp["usage"]["total_tokens"]
+            except Exception:
+                pass
         return 0
 
 

rag/prompts/prompts.py

@@ -114,6 +114,8 @@ def kb_prompt(kbinfos, max_tokens, hash_id=False):
     docs = {d.id: d.meta_fields for d in docs}
 
     def draw_node(k, line):
+        if line is not None and not isinstance(line, str):
+            line = str(line)
         if not line:
             return ""
         return f"\n├── {k}: " + re.sub(r"\n+", " ", line, flags=re.DOTALL)

rag/raptor.py

@@ -42,9 +42,12 @@ class RecursiveAbstractiveProcessing4TreeOrganizedRetrieval:
         self._prompt = prompt
         self._max_token = max_token
 
-    @timeout(60)
+    @timeout(60*20)
     async def _chat(self, system, history, gen_conf):
-        response = get_llm_cache(self._llm_model.llm_name, system, history, gen_conf)
+        response = await trio.to_thread.run_sync(
+            lambda: get_llm_cache(self._llm_model.llm_name, system, history, gen_conf)
+        )
+
         if response:
             return response
         response = await trio.to_thread.run_sync(
@@ -53,19 +56,23 @@ class RecursiveAbstractiveProcessing4TreeOrganizedRetrieval:
         response = re.sub(r"^.*</think>", "", response, flags=re.DOTALL)
         if response.find("**ERROR**") >= 0:
             raise Exception(response)
-        set_llm_cache(self._llm_model.llm_name, system, response, history, gen_conf)
+        await trio.to_thread.run_sync(
+            lambda: set_llm_cache(self._llm_model.llm_name, system, response, history, gen_conf)
+        )
         return response
 
-    @timeout(2)
+    @timeout(20)
     async def _embedding_encode(self, txt):
-        response = get_embed_cache(self._embd_model.llm_name, txt)
+        response = await trio.to_thread.run_sync(
+            lambda: get_embed_cache(self._embd_model.llm_name, txt)
+        )
         if response is not None:
             return response
         embds, _ = await trio.to_thread.run_sync(lambda: self._embd_model.encode([txt]))
         if len(embds) < 1 or len(embds[0]) < 1:
             raise Exception("Embedding error: ")
         embds = embds[0]
-        set_embed_cache(self._embd_model.llm_name, txt, embds)
+        await trio.to_thread.run_sync(lambda: set_embed_cache(self._embd_model.llm_name, txt, embds))
         return embds
 
     def _get_optimal_clusters(self, embeddings: np.ndarray, random_state: int):
@@ -86,7 +93,7 @@ class RecursiveAbstractiveProcessing4TreeOrganizedRetrieval:
         layers = [(0, len(chunks))]
         start, end = 0, len(chunks)
 
-        @timeout(60)
+        @timeout(60*20)
         async def summarize(ck_idx: list[int]):
             nonlocal chunks
             texts = [chunks[i][0] for i in ck_idx]

rag/svr/task_executor.py

@@ -21,7 +21,7 @@ import sys
 import threading
 import time
 
-from api.utils.api_utils import timeout, is_strong_enough
+from api.utils.api_utils import timeout
 from api.utils.log_utils import init_root_logger, get_project_base_directory
 from graphrag.general.index import run_graphrag
 from graphrag.utils import get_llm_cache, set_llm_cache, get_tags_from_cache, set_tags_to_cache
@@ -478,8 +478,6 @@ async def embedding(docs, mdl, parser_config=None, callback=None):
 
 @timeout(3600)
 async def run_raptor(row, chat_mdl, embd_mdl, vector_size, callback=None):
-    # Pressure test for GraphRAG task
-    await is_strong_enough(chat_mdl, embd_mdl)
     chunks = []
     vctr_nm = "q_%d_vec"%vector_size
     for d in settings.retrievaler.chunk_list(row["doc_id"], row["tenant_id"], [str(row["kb_id"])],
@@ -553,7 +551,6 @@ async def do_handle_task(task):
     try:
         # bind embedding model
         embedding_model = LLMBundle(task_tenant_id, LLMType.EMBEDDING, llm_name=task_embedding_id, lang=task_language)
-        await is_strong_enough(None, embedding_model)
         vts, _ = embedding_model.encode(["ok"])
         vector_size = len(vts[0])
     except Exception as e:
@@ -568,7 +565,6 @@ async def do_handle_task(task):
     if task.get("task_type", "") == "raptor":
         # bind LLM for raptor
         chat_model = LLMBundle(task_tenant_id, LLMType.CHAT, llm_name=task_llm_id, lang=task_language)
-        await is_strong_enough(chat_model, None)
         # run RAPTOR
         async with kg_limiter:
             chunks, token_count = await run_raptor(task, chat_model, embedding_model, vector_size, progress_callback)
@@ -580,7 +576,6 @@ async def do_handle_task(task):
         graphrag_conf = task["kb_parser_config"].get("graphrag", {})
         start_ts = timer()
         chat_model = LLMBundle(task_tenant_id, LLMType.CHAT, llm_name=task_llm_id, lang=task_language)
-        await is_strong_enough(chat_model, None)
         with_resolution = graphrag_conf.get("resolution", False)
         with_community = graphrag_conf.get("community", False)
         async with kg_limiter:

sandbox/executor_manager/core/config.py

@@ -22,7 +22,7 @@ from util import format_timeout_duration, parse_timeout_duration
 from core.container import init_containers, teardown_containers
 from core.logger import logger
 
-TIMEOUT = 10
+TIMEOUT = parse_timeout_duration(os.getenv("SANDBOX_TIMEOUT", "10s"))
 
 
 @asynccontextmanager
@@ -39,6 +39,5 @@ async def _lifespan(app: FastAPI):
 
 
 def init():
-    TIMEOUT = parse_timeout_duration(os.getenv("SANDBOX_TIMEOUT"))
     logger.info(f"Global timeout: {format_timeout_duration(TIMEOUT)}")
     return _lifespan

sdk/python/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ragflow-sdk"
-version = "0.20.2"
+version = "0.20.4"
 description = "Python client sdk of [RAGFlow](https://github.com/infiniflow/ragflow). RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding."
 authors = [{ name = "Zhichang Yu", email = "yuzhichang@gmail.com" }]
 license = { text = "Apache License, Version 2.0" }

sdk/python/uv.lock

@@ -342,7 +342,7 @@ wheels = [
 
 [[package]]
 name = "ragflow-sdk"
-version = "0.20.2"
+version = "0.20.4"
 source = { virtual = "." }
 dependencies = [
     { name = "beartype" },

uv.lock

@@ -2603,7 +2603,7 @@ wheels = [
 
 [[package]]
 name = "infinity-sdk"
-version = "0.6.0.dev4"
+version = "0.6.0.dev5"
 source = { registry = "https://mirrors.aliyun.com/pypi/simple" }
 dependencies = [
     { name = "numpy" },
@@ -2620,7 +2620,7 @@ dependencies = [
     { name = "thrift" },
 ]
 wheels = [
-    { url = "https://mirrors.aliyun.com/pypi/packages/d4/cc/645ed8de15952940c7308a788036376583a5fc29fdcf3e4bc75b5ad0c881/infinity_sdk-0.6.0.dev4-py3-none-any.whl", hash = "sha256:f8f4bd8a44e3fae7b4228b5c9e9a16559b4905f50d2d7d0a3d18f39974613e7a" },
+    { url = "https://mirrors.aliyun.com/pypi/packages/fe/a4/6079bf9790f16badc01e7b79a28c90bec407cfcaa8a2ed37e4a68120f87a/infinity_sdk-0.6.0.dev5-py3-none-any.whl", hash = "sha256:510ac408d5cd9d3d4df33c7c0877f55c5ae8a6019e465190c86d58012a319179" },
 ]
 
 [[package]]
@@ -5268,7 +5268,7 @@ wheels = [
 
 [[package]]
 name = "ragflow"
-version = "0.20.2"
+version = "0.20.4"
 source = { virtual = "." }
 dependencies = [
     { name = "akshare" },
@@ -5471,7 +5471,7 @@ requires-dist = [
     { name = "httpx", extras = ["socks"], specifier = "==0.27.2" },
     { name = "huggingface-hub", specifier = ">=0.25.0,<0.26.0" },
     { name = "infinity-emb", specifier = ">=0.0.66,<0.0.67" },
-    { name = "infinity-sdk", specifier = "==0.6.0.dev4" },
+    { name = "infinity-sdk", specifier = "==0.6.0.dev5" },
     { name = "itsdangerous", specifier = "==2.1.2" },
     { name = "json-repair", specifier = "==0.35.0" },
     { name = "langfuse", specifier = ">=2.60.0" },

web/.eslintrc.js

@@ -14,7 +14,7 @@ module.exports = {
       'error',
       {
         '**/*.{jsx,tsx}': 'KEBAB_CASE',
-        '**/*.{js,ts}': 'KEBAB_CASE',
+        '**/*.{js,ts}': '[a-z0-9.-]*',
       },
     ],
     'check-file/folder-naming-convention': [

web/src/app.tsx

@@ -2,7 +2,6 @@ import { Toaster as Sonner } from '@/components/ui/sonner';
 import { Toaster } from '@/components/ui/toaster';
 import i18n from '@/locales/config';
 import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
-import { ReactQueryDevtools } from '@tanstack/react-query-devtools';
 import { App, ConfigProvider, ConfigProviderProps, theme } from 'antd';
 import pt_BR from 'antd/lib/locale/pt_BR';
 import deDE from 'antd/locale/de_DE';
@@ -85,7 +84,7 @@ function Root({ children }: React.PropsWithChildren) {
         <Sonner position={'top-right'} expand richColors closeButton></Sonner>
         <Toaster />
       </ConfigProvider>
-      <ReactQueryDevtools buttonPosition={'top-left'} />
+      {/* <ReactQueryDevtools buttonPosition={'top-left'} initialIsOpen={false} /> */}
     </>
   );
 }

web/src/components/file-upload-dialog/index.tsx

@@ -8,47 +8,93 @@ import {
 } from '@/components/ui/dialog';
 import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
 import { IModalProps } from '@/interfaces/common';
-import { Dispatch, SetStateAction, useCallback, useState } from 'react';
+import { zodResolver } from '@hookform/resolvers/zod';
+import { TFunction } from 'i18next';
+import { useForm } from 'react-hook-form';
 import { useTranslation } from 'react-i18next';
+import { z } from 'zod';
 import { FileUploader } from '../file-uploader';
+import { RAGFlowFormItem } from '../ragflow-form';
+import { Form } from '../ui/form';
+import { Switch } from '../ui/switch';
 
-type UploaderTabsProps = {
-  setFiles: Dispatch<SetStateAction<File[]>>;
+function buildUploadFormSchema(t: TFunction) {
+  const FormSchema = z.object({
+    parseOnCreation: z.boolean().optional(),
+    fileList: z
+      .array(z.instanceof(File))
+      .min(1, { message: t('fileManager.pleaseUploadAtLeastOneFile') }),
+  });
+
+  return FormSchema;
+}
+
+export type UploadFormSchemaType = z.infer<
+  ReturnType<typeof buildUploadFormSchema>
+>;
+
+const UploadFormId = 'UploadFormId';
+
+type UploadFormProps = {
+  submit: (values?: UploadFormSchemaType) => void;
+  showParseOnCreation?: boolean;
 };
-
-export function UploaderTabs({ setFiles }: UploaderTabsProps) {
+function UploadForm({ submit, showParseOnCreation }: UploadFormProps) {
   const { t } = useTranslation();
+  const FormSchema = buildUploadFormSchema(t);
+
+  type UploadFormSchemaType = z.infer<typeof FormSchema>;
+  const form = useForm<UploadFormSchemaType>({
+    resolver: zodResolver(FormSchema),
+    defaultValues: {
+      parseOnCreation: false,
+      fileList: [],
+    },
+  });
 
   return (
-    <Tabs defaultValue="account">
-      <TabsList className="grid w-full grid-cols-2 mb-4">
-        <TabsTrigger value="account">{t('fileManager.local')}</TabsTrigger>
-        <TabsTrigger value="password">{t('fileManager.s3')}</TabsTrigger>
-      </TabsList>
-      <TabsContent value="account">
-        <FileUploader
-          maxFileCount={8}
-          maxSize={8 * 1024 * 1024}
-          onValueChange={setFiles}
-          accept={{ '*': [] }}
-        />
-      </TabsContent>
-      <TabsContent value="password">{t('common.comingSoon')}</TabsContent>
-    </Tabs>
+    <Form {...form}>
+      <form
+        onSubmit={form.handleSubmit(submit)}
+        id={UploadFormId}
+        className="space-y-4"
+      >
+        {showParseOnCreation && (
+          <RAGFlowFormItem
+            name="parseOnCreation"
+            label={t('fileManager.parseOnCreation')}
+          >
+            {(field) => (
+              <Switch
+                onCheckedChange={field.onChange}
+                checked={field.value}
+              ></Switch>
+            )}
+          </RAGFlowFormItem>
+        )}
+        <RAGFlowFormItem name="fileList" label={t('fileManager.file')}>
+          {(field) => (
+            <FileUploader
+              value={field.value}
+              onValueChange={field.onChange}
+              accept={{ '*': [] }}
+            />
+          )}
+        </RAGFlowFormItem>
+      </form>
+    </Form>
   );
 }
 
+type FileUploadDialogProps = IModalProps<UploadFormSchemaType> &
+  Pick<UploadFormProps, 'showParseOnCreation'>;
 export function FileUploadDialog({
   hideModal,
   onOk,
   loading,
-}: IModalProps<File[]>) {
+  showParseOnCreation = false,
+}: FileUploadDialogProps) {
   const { t } = useTranslation();
-  const [files, setFiles] = useState<File[]>([]);
-
-  const handleOk = useCallback(() => {
-    onOk?.(files);
-  }, [files, onOk]);
 
   return (
     <Dialog open onOpenChange={hideModal}>
@@ -56,9 +102,21 @@ export function FileUploadDialog({
         <DialogHeader>
           <DialogTitle>{t('fileManager.uploadFile')}</DialogTitle>
         </DialogHeader>
-        <UploaderTabs setFiles={setFiles}></UploaderTabs>
+        <Tabs defaultValue="account">
+          <TabsList className="grid w-full grid-cols-2 mb-4">
+            <TabsTrigger value="account">{t('fileManager.local')}</TabsTrigger>
+            <TabsTrigger value="password">{t('fileManager.s3')}</TabsTrigger>
+          </TabsList>
+          <TabsContent value="account">
+            <UploadForm
+              submit={onOk!}
+              showParseOnCreation={showParseOnCreation}
+            ></UploadForm>
+          </TabsContent>
+          <TabsContent value="password">{t('common.comingSoon')}</TabsContent>
+        </Tabs>
         <DialogFooter>
-          <ButtonLoading type="submit" onClick={handleOk} loading={loading}>
+          <ButtonLoading type="submit" loading={loading} form={UploadFormId}>
             {t('common.save')}
           </ButtonLoading>
         </DialogFooter>

web/src/components/file-uploader.tsx

@@ -15,6 +15,7 @@ import { Progress } from '@/components/ui/progress';
 import { ScrollArea } from '@/components/ui/scroll-area';
 import { useControllableState } from '@/hooks/use-controllable-state';
 import { cn, formatBytes } from '@/lib/utils';
+import { useTranslation } from 'react-i18next';
 
 function isFileWithPreview(file: File): file is File & { preview: string } {
   return 'preview' in file && typeof file.preview === 'string';
@@ -157,6 +158,8 @@ interface FileUploaderProps extends React.HTMLAttributes<HTMLDivElement> {
    * @example disabled
    */
   disabled?: boolean;
+
+  description?: string;
 }
 
 export function FileUploader(props: FileUploaderProps) {
@@ -168,14 +171,15 @@ export function FileUploader(props: FileUploaderProps) {
     accept = {
       'image/*': [],
     },
-    maxSize = 1024 * 1024 * 2,
-    maxFileCount = 1,
+    maxSize = 1024 * 1024 * 10000000,
+    maxFileCount = 100000000000,
     multiple = false,
     disabled = false,
     className,
+    description,
     ...dropzoneProps
   } = props;
-
+  const { t } = useTranslation();
   const [files, setFiles] = useControllableState({
     prop: valueProp,
     onChange: onValueChange,
@@ -267,7 +271,7 @@ export function FileUploader(props: FileUploaderProps) {
           <div
             {...getRootProps()}
             className={cn(
-              'group relative grid h-52 w-full cursor-pointer place-items-center rounded-lg border-2 border-dashed border-muted-foreground/25 px-5 py-2.5 text-center transition hover:bg-muted/25',
+              'group relative grid h-72 w-full cursor-pointer place-items-center rounded-lg border-2 border-dashed border-muted-foreground/25 px-5 py-2.5 text-center transition hover:bg-muted/25',
               'ring-offset-background focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2',
               isDragActive && 'border-muted-foreground/50',
               isDisabled && 'pointer-events-none opacity-60',
@@ -298,14 +302,15 @@ export function FileUploader(props: FileUploaderProps) {
                 </div>
                 <div className="flex flex-col gap-px">
                   <p className="font-medium text-muted-foreground">
-                    Drag {`'n'`} drop files here, or click to select files
+                    {t('knowledgeDetails.uploadTitle')}
                   </p>
                   <p className="text-sm text-muted-foreground/70">
-                    You can upload
+                    {description || t('knowledgeDetails.uploadDescription')}
+                    {/* You can upload
                     {maxFileCount > 1
                       ? ` ${maxFileCount === Infinity ? 'multiple' : maxFileCount}
                       files (up to ${formatBytes(maxSize)} each)`
-                      : ` a file with ${formatBytes(maxSize)}`}
+                      : ` a file with ${formatBytes(maxSize)}`} */}
                   </p>
                 </div>
               </div>

web/src/components/home-card.tsx

@@ -1,6 +1,7 @@
 import { RAGFlowAvatar } from '@/components/ragflow-avatar';
 import { Card, CardContent } from '@/components/ui/card';
 import { formatDate } from '@/utils/date';
+import { ReactNode } from 'react';
 
 interface IProps {
   data: {
@@ -11,8 +12,9 @@ interface IProps {
   };
   onClick?: () => void;
   moreDropdown: React.ReactNode;
+  sharedBadge?: ReactNode;
 }
-export function HomeCard({ data, onClick, moreDropdown }: IProps) {
+export function HomeCard({ data, onClick, moreDropdown, sharedBadge }: IProps) {
   return (
     <Card
       className="bg-bg-card  border-colors-outline-neutral-standard"
@@ -31,7 +33,7 @@ export function HomeCard({ data, onClick, moreDropdown }: IProps) {
         </div>
         <div className="flex flex-col justify-between gap-1 flex-1 h-full w-[calc(100%-50px)]">
           <section className="flex justify-between">
-            <div className="text-[20px] font-bold w-80% leading-5">
+            <div className="text-[20px] font-bold w-80% leading-5 text-ellipsis overflow-hidden">
               {data.name}
             </div>
             {moreDropdown}
@@ -41,10 +43,11 @@ export function HomeCard({ data, onClick, moreDropdown }: IProps) {
             <div className="whitespace-nowrap overflow-hidden text-ellipsis">
               {data.description}
             </div>
-            <div>
+            <div className="flex justify-between items-center">
               <p className="text-sm opacity-80">
                 {formatDate(data.update_time)}
               </p>
+              {sharedBadge}
             </div>
           </section>
         </div>

web/src/components/large-model-form-field.tsx

@@ -12,6 +12,7 @@ import {
   FormMessage,
 } from '@/components/ui/form';
 import { LlmModelType } from '@/constants/knowledge';
+import { t } from 'i18next';
 import { Funnel } from 'lucide-react';
 import { useFormContext, useWatch } from 'react-hook-form';
 import { useTranslation } from 'react-i18next';
@@ -21,15 +22,15 @@ import { Button } from './ui/button';
 
 const ModelTypes = [
   {
-    title: 'All Models',
+    title: t('flow.allModels'),
     value: 'all',
   },
   {
-    title: 'Text-only Models',
+    title: t('flow.textOnlyModels'),
     value: LlmModelType.Chat,
   },
   {
-    title: 'Multimodal Models',
+    title: t('flow.multimodalModels'),
     value: LlmModelType.Image2text,
   },
 ];

web/src/components/layout-recognize-form-field.tsx

@@ -68,7 +68,7 @@ export function LayoutRecognizeFormField() {
             <div className="flex items-center">
               <FormLabel
                 tooltip={t('layoutRecognizeTip')}
-                className="text-sm text-muted-foreground whitespace-nowrap w-1/4"
+                className="text-sm text-muted-foreground whitespace-wrap w-1/4"
               >
                 {t('layoutRecognize')}
               </FormLabel>

web/src/components/list-filter-bar/index.tsx

@@ -1,5 +1,5 @@
 import { cn } from '@/lib/utils';
-import { ChevronDown } from 'lucide-react';
+import { Funnel } from 'lucide-react';
 import React, {
   ChangeEventHandler,
   PropsWithChildren,
@@ -25,20 +25,20 @@ export const FilterButton = React.forwardRef<
 >(({ count = 0, ...props }, ref) => {
   return (
     <Button variant="secondary" {...props} ref={ref}>
-      <span
+      {/* <span
         className={cn({
           'text-text-primary': count > 0,
           'text-text-sub-title-invert': count === 0,
         })}
       >
         Filter
-      </span>
+      </span> */}
       {count > 0 && (
         <span className="rounded-full bg-text-badge px-1 text-xs ">
           {count}
         </span>
       )}
-      <ChevronDown />
+      <Funnel />
     </Button>
   );
 });

web/src/components/llm-select/next.tsx

@@ -2,6 +2,7 @@ import { LlmModelType } from '@/constants/knowledge';
 import { useComposeLlmOptionsByModelTypes } from '@/hooks/llm-hooks';
 import * as SelectPrimitive from '@radix-ui/react-select';
 import { forwardRef, memo, useMemo, useState } from 'react';
+import { useTranslation } from 'react-i18next';
 import { LlmSettingFieldItems } from '../llm-setting-items/next';
 import { Popover, PopoverContent, PopoverTrigger } from '../ui/popover';
 import { Select, SelectTrigger, SelectValue } from '../ui/select';
@@ -20,6 +21,7 @@ const NextInnerLLMSelect = forwardRef<
   React.ElementRef<typeof SelectPrimitive.Trigger>,
   NextInnerLLMSelectProps
 >(({ value, disabled, filter, showSpeech2TextModel = false }, ref) => {
+  const { t } = useTranslation();
   const [isPopoverOpen, setIsPopoverOpen] = useState(false);
 
   const ttsModel = useMemo(() => {
@@ -49,7 +51,7 @@ const NextInnerLLMSelect = forwardRef<
             }}
             ref={ref}
           >
-            <SelectValue>
+            <SelectValue placeholder={t('common.pleaseSelect')}>
               {
                 modelOptions
                   .flatMap((x) => x.options)

web/src/components/llm-setting-items/slider.tsx

@@ -19,6 +19,7 @@ type SliderInputSwitchFormFieldProps = {
   name: string;
   label: string;
   defaultValue?: number;
+  onChange?: (value: number) => void;
   className?: string;
   checkName: string;
 };
@@ -30,6 +31,7 @@ export function SliderInputSwitchFormField({
   label,
   name,
   defaultValue,
+  onChange,
   className,
   checkName,
 }: SliderInputSwitchFormFieldProps) {
@@ -66,6 +68,10 @@ export function SliderInputSwitchFormField({
             <FormControl>
               <SingleFormSlider
                 {...field}
+                onChange={(value: number) => {
+                  onChange?.(value);
+                  field.onChange(value);
+                }}
                 max={max}
                 min={min}
                 step={step}
@@ -80,6 +86,10 @@ export function SliderInputSwitchFormField({
                 min={min}
                 step={step}
                 {...field}
+                onChange={(value: number) => {
+                  onChange?.(value);
+                  field.onChange(value);
+                }}
               ></NumberInput>
             </FormControl>
           </div>

web/src/components/message-input/next.tsx

@@ -34,6 +34,7 @@ interface IProps {
   createConversationBeforeUploadDocument?(message: string): Promise<any>;
   stopOutputMessage?(): void;
   onUpload?: NonNullable<FileUploadProps['onUpload']>;
+  removeFile?(file: File): void;
 }
 
 export function NextMessageInput({
@@ -47,6 +48,7 @@ export function NextMessageInput({
   onInputChange,
   stopOutputMessage,
   onPressEnter,
+  removeFile,
 }: IProps) {
   const [files, setFiles] = React.useState<File[]>([]);
 
@@ -77,6 +79,13 @@ export function NextMessageInput({
     [submit],
   );
 
+  const handleRemoveFile = React.useCallback(
+    (file: File) => () => {
+      removeFile?.(file);
+    },
+    [removeFile],
+  );
+
   return (
     <FileUpload
       value={files}
@@ -121,6 +130,7 @@ export function NextMessageInput({
                   variant="secondary"
                   size="icon"
                   className="-top-1 -right-1 absolute size-4 shrink-0 cursor-pointer rounded-full"
+                  onClick={handleRemoveFile(file)}
                 >
                   <X className="size-2.5" />
                 </Button>

web/src/components/metadata-filter/index.tsx

@@ -58,7 +58,10 @@ export function MetadataFilter({ prefix = '' }: MetadataFilterProps) {
           name={methodName}
           tooltip={t('metadataTip')}
         >
-          <SelectWithSearch options={MetadataOptions} />
+          <SelectWithSearch
+            options={MetadataOptions}
+            triggerClassName="!bg-bg-input"
+          />
         </RAGFlowFormItem>
       )}
       {hasKnowledge && metadata === DatasetMetadata.Manual && (

web/src/components/originui/select-with-search.tsx

@@ -28,6 +28,7 @@ import {
   PopoverTrigger,
 } from '@/components/ui/popover';
 import { cn } from '@/lib/utils';
+import { t } from 'i18next';
 import { RAGFlowSelectOptionType } from '../ui/select';
 import { Separator } from '../ui/separator';
 
@@ -114,7 +115,9 @@ export const SelectWithSearch = forwardRef<
                 <span className="leading-none truncate">{selectLabel}</span>
               </span>
             ) : (
-              <span className="text-muted-foreground">Select value</span>
+              <span className="text-muted-foreground">
+                {t('common.selectPlaceholder')}
+              </span>
             )}
             <div className="flex items-center justify-between">
               {value && allowClear && (
@@ -142,9 +145,9 @@ export const SelectWithSearch = forwardRef<
           align="start"
         >
           <Command>
-            <CommandInput placeholder="Search ..." />
+            <CommandInput placeholder={t('common.search') + '...'} />
             <CommandList>
-              <CommandEmpty>No data found.</CommandEmpty>
+              <CommandEmpty>{t('common.noDataFound')}</CommandEmpty>
               {options.map((group, idx) => {
                 if (group.options) {
                   return (

web/src/components/parse-configuration/raptor-form-fields.tsx

@@ -3,7 +3,7 @@ import { DocumentParserType } from '@/constants/knowledge';
 import { useTranslate } from '@/hooks/common-hooks';
 import random from 'lodash/random';
 import { Plus } from 'lucide-react';
-import { useCallback, useEffect } from 'react';
+import { useCallback } from 'react';
 import { useFormContext, useWatch } from 'react-hook-form';
 import { SliderInputFormField } from '../slider-input-form-field';
 import { Button } from '../ui/button';
@@ -57,15 +57,19 @@ const RaptorFormFields = () => {
   const form = useFormContext();
   const { t } = useTranslate('knowledgeConfiguration');
   const useRaptor = useWatch({ name: UseRaptorField });
-  useEffect(() => {
-    if (useRaptor) {
-      form.setValue(MaxTokenField, 256);
-      form.setValue(ThresholdField, 0.1);
-      form.setValue(MaxCluster, 64);
-      form.setValue(RandomSeedField, 0);
-      form.setValue(Prompt, t('promptText'));
-    }
-  }, [form, useRaptor, t]);
+
+  const changeRaptor = useCallback(
+    (isUseRaptor: boolean) => {
+      if (isUseRaptor) {
+        form.setValue(MaxTokenField, 256);
+        form.setValue(ThresholdField, 0.1);
+        form.setValue(MaxCluster, 64);
+        form.setValue(RandomSeedField, 0);
+        form.setValue(Prompt, t('promptText'));
+      }
+    },
+    [form],
+  );
 
   const handleGenerate = useCallback(() => {
     form.setValue(RandomSeedField, random(10000));
@@ -97,7 +101,10 @@ const RaptorFormFields = () => {
                   <FormControl>
                     <Switch
                       checked={field.value}
-                      onCheckedChange={field.onChange}
+                      onCheckedChange={(e) => {
+                        changeRaptor(e);
+                        field.onChange(e);
+                      }}
                     ></Switch>
                   </FormControl>
                 </div>
@@ -127,7 +134,13 @@ const RaptorFormFields = () => {
                     </FormLabel>
                     <div className="w-3/4">
                       <FormControl>
-                        <Textarea {...field} rows={8} />
+                        <Textarea
+                          {...field}
+                          rows={8}
+                          onChange={(e) => {
+                            field.onChange(e?.target?.value);
+                          }}
+                        />
                       </FormControl>
                     </div>
                   </div>

web/src/components/ragflow-form.tsx

@@ -5,6 +5,7 @@ import {
   FormLabel,
   FormMessage,
 } from '@/components/ui/form';
+import { cn } from '@/lib/utils';
 import { ReactNode, cloneElement, isValidElement } from 'react';
 import { ControllerRenderProps, useFormContext } from 'react-hook-form';
 
@@ -13,6 +14,7 @@ type RAGFlowFormItemProps = {
   label: ReactNode;
   tooltip?: ReactNode;
   children: ReactNode | ((field: ControllerRenderProps) => ReactNode);
+  horizontal?: boolean;
 };
 
 export function RAGFlowFormItem({
@@ -20,6 +22,7 @@ export function RAGFlowFormItem({
   label,
   tooltip,
   children,
+  horizontal = false,
 }: RAGFlowFormItemProps) {
   const form = useFormContext();
   return (
@@ -27,8 +30,14 @@ export function RAGFlowFormItem({
       control={form.control}
       name={name}
       render={({ field }) => (
-        <FormItem>
-          <FormLabel tooltip={tooltip}>{label}</FormLabel>
+        <FormItem
+          className={cn({
+            'flex items-center': horizontal,
+          })}
+        >
+          <FormLabel tooltip={tooltip} className={cn({ 'w-1/4': horizontal })}>
+            {label}
+          </FormLabel>
           <FormControl>
             {typeof children === 'function'
               ? children(field)

web/src/components/shared-badge.tsx

@@ -8,9 +8,5 @@ export function SharedBadge({ children }: PropsWithChildren) {
     return null;
   }
 
-  return (
-    <span className="bg-text-secondary rounded-sm px-1 text-bg-base text-xs">
-      {children}
-    </span>
-  );
+  return <span className="bg-bg-card rounded-sm px-1 text-xs">{children}</span>;
 }

web/src/components/table-skeleton.tsx

@@ -1,3 +1,4 @@
+import { t } from 'i18next';
 import { Loader2 } from 'lucide-react';
 import { PropsWithChildren } from 'react';
 import { TableCell, TableRow } from './ui/table';
@@ -28,5 +29,5 @@ export function TableSkeleton({
 }
 
 export function TableEmpty({ columnsLength }: { columnsLength: number }) {
-  return <Row columnsLength={columnsLength}>No results.</Row>;
+  return <Row columnsLength={columnsLength}>{t('common.noResults')}</Row>;
 }

web/src/components/ui/button.tsx

@@ -17,7 +17,7 @@ const buttonVariants = cva(
         outline:
           'border bg-background shadow-xs hover:bg-accent hover:text-accent-foreground dark:bg-input/30 dark:border-input dark:hover:bg-input/50',
         secondary:
-          'bg-secondary text-secondary-foreground shadow-xs hover:bg-secondary/80',
+          'bg-bg-input text-secondary-foreground shadow-xs hover:bg-bg-input/80',
         ghost:
           'hover:bg-accent hover:text-accent-foreground dark:hover:bg-accent/50',
         link: 'text-primary underline-offset-4 hover:underline',

web/src/components/ui/input.tsx

@@ -116,7 +116,10 @@ export { ExpandedInput, Input, SearchInput };
 
 type NumberInputProps = { onChange?(value: number): void } & InputProps;
 
-export const NumberInput = ({ onChange, ...props }: NumberInputProps) => {
+export const NumberInput = React.forwardRef<
+  HTMLInputElement,
+  NumberInputProps & { value: Value; onChange(value: Value): void }
+>(function NumberInput({ onChange, ...props }, ref) {
   return (
     <Input
       type="number"
@@ -125,6 +128,7 @@ export const NumberInput = ({ onChange, ...props }: NumberInputProps) => {
         onChange?.(value === '' ? 0 : Number(value)); // convert to number
       }}
       {...props}
+      ref={ref}
     ></Input>
   );
-};
+});

web/src/components/ui/multi-select.tsx

@@ -209,8 +209,16 @@ export const MultiSelect = React.forwardRef<
     const [isAnimating, setIsAnimating] = React.useState(false);
 
     React.useEffect(() => {
-      setSelectedValues(defaultValue);
-    }, [defaultValue]);
+      if (!selectedValues?.length && props.value) {
+        setSelectedValues(props.value as string[]);
+      }
+    }, [props.value, selectedValues]);
+
+    React.useEffect(() => {
+      if (!selectedValues?.length && !props.value && defaultValue) {
+        setSelectedValues(defaultValue);
+      }
+    }, [defaultValue, props.value, selectedValues]);
 
     const flatOptions = React.useMemo(() => {
       return options.flatMap((option) =>
@@ -291,15 +299,18 @@ export const MultiSelect = React.forwardRef<
                         variant="secondary"
                         className={cn(
                           isAnimating ? 'animate-bounce' : '',
+                          'px-1',
                           multiSelectVariants({ variant }),
                         )}
                         style={{ animationDuration: `${animation}s` }}
                       >
-                        <div className="flex items-center gap-1">
+                        <div className="flex justify-between items-center gap-1">
                           {IconComponent && (
                             <IconComponent className="h-4 w-4" />
                           )}
-                          <div>{option?.label}</div>
+                          <div className="max-w-28 text-ellipsis overflow-hidden">
+                            {option?.label}
+                          </div>
                           <XCircle
                             className="h-4 w-4 cursor-pointer"
                             onClick={(event) => {

web/src/components/ui/progress.tsx

@@ -12,13 +12,13 @@ const Progress = React.forwardRef<
   <ProgressPrimitive.Root
     ref={ref}
     className={cn(
-      'relative h-4 w-full overflow-hidden rounded-full bg-secondary',
+      'relative h-4 w-full overflow-hidden rounded-full bg-bg-accent',
       className,
     )}
     {...props}
   >
     <ProgressPrimitive.Indicator
-      className="h-full w-full flex-1 bg-primary transition-all"
+      className="h-full w-full flex-1 bg-accent-primary transition-all"
       style={{ transform: `translateX(-${100 - (value || 0)}%)` }}
     />
   </ProgressPrimitive.Root>

web/src/components/ui/segmented.tsx

@@ -23,6 +23,7 @@ export interface SegmentedProps
   prefixCls?: string;
   direction?: 'ltr' | 'rtl';
   motionName?: string;
+  activeClassName?: string;
 }
 
 export function Segmented({
@@ -30,6 +31,7 @@ export function Segmented({
   value,
   onChange,
   className,
+  activeClassName,
 }: SegmentedProps) {
   const [selectedValue, setSelectedValue] = React.useState<
     SegmentedValue | undefined
@@ -57,9 +59,12 @@ export function Segmented({
             className={cn(
               'inline-flex items-center px-6 py-2 text-base font-normal rounded-3xl cursor-pointer',
               {
-                'bg-text-primary': selectedValue === actualValue,
-                'text-bg-base': selectedValue === actualValue,
+                'text-bg-base bg-metallic-gradient border-b-[#00BEB4] border-b-2':
+                  selectedValue === actualValue,
               },
+              activeClassName && selectedValue === actualValue
+                ? activeClassName
+                : '',
             )}
             onClick={() => handleOnChange(actualValue)}
           >

web/src/components/ui/table.tsx

@@ -8,7 +8,7 @@ const Table = React.forwardRef<
 >(({ className, rootClassName, ...props }, ref) => (
   <div
     className={cn(
-      'relative w-full overflow-auto rounded-2xl bg-bg-card',
+      'relative w-full overflow-auto rounded-2xl bg-bg-card scrollbar-none',
       rootClassName,
     )}
   >
@@ -27,7 +27,7 @@ const TableHeader = React.forwardRef<
 >(({ className, ...props }, ref) => (
   <thead
     ref={ref}
-    className={cn('[&_tr]:border-b top-0 sticky', className)}
+    className={cn('[&_tr]:border-b top-0 sticky bg-bg-title z-10', className)}
     {...props}
   />
 ));
@@ -67,7 +67,7 @@ const TableRow = React.forwardRef<
   <tr
     ref={ref}
     className={cn(
-      'border-b transition-colors hover:bg-muted/50 data-[state=selected]:bg-muted',
+      'border-b border-border-button transition-colors hover:bg-bg-card data-[state=selected]:bg-bg-card',
       className,
     )}
     {...props}
@@ -82,7 +82,7 @@ const TableHead = React.forwardRef<
   <th
     ref={ref}
     className={cn(
-      'h-12 px-4 text-left align-middle font-normal text-text-secondary [&:has([role=checkbox])]:pr-0',
+      'h-12 px-4 text-left align-middle font-normal text-text-secondary [&:has([role=checkbox])]:pr-0 ',
       className,
     )}
     {...props}

web/src/components/ui/textarea.tsx

@@ -54,7 +54,7 @@ const Textarea = forwardRef<HTMLTextAreaElement, TextareaProps>(
     return (
       <textarea
         className={cn(
-          'flex min-h-[80px] w-full bg-bg-card rounded-md border border-input px-3 py-2 text-base ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 md:text-sm overflow-hidden',
+          'flex min-h-[80px] w-full bg-bg-input rounded-md border border-input px-3 py-2 text-base ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 md:text-sm overflow-hidden',
           className,
         )}
         rows={autoSize?.minRows ?? props.rows ?? undefined}

web/src/components/ui/tooltip.tsx

@@ -33,7 +33,7 @@ export { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger };
 export const FormTooltip = ({ tooltip }: { tooltip: React.ReactNode }) => {
   return (
     <Tooltip>
-      <TooltipTrigger>
+      <TooltipTrigger tabIndex={-1}>
         <Info className="size-3 ml-2" />
       </TooltipTrigger>
       <TooltipContent>

web/src/constants/permission.ts

@@ -0,0 +1,4 @@
+export enum PermissionRole {
+  Me = 'me',
+  Team = 'team',
+}

web/src/hooks/auth-hooks.ts

@@ -1,5 +1,5 @@
+import message from '@/components/ui/message';
 import authorizationUtil from '@/utils/authorization-util';
-import { message } from 'antd';
 import { useEffect, useMemo, useState } from 'react';
 import { useNavigate, useSearchParams } from 'umi';
 
@@ -28,7 +28,7 @@ export const useOAuthCallback = () => {
       authorizationUtil.setAuthorization(auth);
       newQueryParameters.delete('auth');
       setSearchParams(newQueryParameters);
-      navigate('/knowledge');
+      navigate('/');
     }
   }, [
     error,

web/src/hooks/logic-hooks/navigate-hooks.ts

@@ -77,7 +77,7 @@ export const useNavigatePage = () => {
   }, [navigate]);
 
   const navigateToSearch = useCallback(
-    (id: string) => {
+    (id: string) => () => {
       navigate(`${Routes.Search}/${id}`);
     },
     [navigate],

web/src/hooks/use-agent-request.ts

@@ -390,7 +390,7 @@ export const useUploadCanvasFileWithProgress = (
         files.forEach((file) => {
           onError(file, error as Error);
         });
-        message.error('error', error?.message);
+        message.error(error?.message);
       }
     },
   });

web/src/hooks/use-chat-request.ts

@@ -1,3 +1,4 @@
+import { FileUploadProps } from '@/components/file-upload';
 import message from '@/components/ui/message';
 import { ChatSearchParams } from '@/constants/chat';
 import {
@@ -14,7 +15,7 @@ import { buildMessageListWithUuid, getConversationId } from '@/utils/chat';
 import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
 import { useDebounce } from 'ahooks';
 import { has } from 'lodash';
-import { useCallback, useMemo } from 'react';
+import { useCallback, useMemo, useRef } from 'react';
 import { useTranslation } from 'react-i18next';
 import { useParams, useSearchParams } from 'umi';
 import {
@@ -395,9 +396,14 @@ export const useDeleteMessage = () => {
   return { data, loading, deleteMessage: mutateAsync };
 };
 
+type UploadParameters = Parameters<NonNullable<FileUploadProps['onUpload']>>;
+
+type X = { file: UploadParameters[0][0]; options: UploadParameters[1] };
+
 export function useUploadAndParseFile() {
   const { conversationId } = useGetChatSearchParams();
   const { t } = useTranslation();
+  const controller = useRef(new AbortController());
 
   const {
     data,
@@ -405,22 +411,48 @@ export function useUploadAndParseFile() {
     mutateAsync,
   } = useMutation({
     mutationKey: [ChatApiAction.UploadAndParse],
-    mutationFn: async (file: File) => {
-      const formData = new FormData();
-      formData.append('file', file);
-      formData.append('conversation_id', conversationId);
+    mutationFn: async ({
+      file,
+      options: { onProgress, onSuccess, onError },
+    }: X) => {
+      try {
+        const formData = new FormData();
+        formData.append('file', file);
+        formData.append('conversation_id', conversationId);
 
-      const { data } = await chatService.uploadAndParse(formData);
+        const { data } = await chatService.uploadAndParse(
+          {
+            signal: controller.current.signal,
+            data: formData,
+            onUploadProgress: ({ progress }) => {
+              onProgress(file, (progress || 0) * 100 - 1);
+            },
+          },
+          true,
+        );
 
-      if (data.code === 0) {
-        message.success(t(`message.uploaded`));
+        onProgress(file, 100);
+
+        if (data.code === 0) {
+          onSuccess(file);
+          message.success(t(`message.uploaded`));
+        } else {
+          onError(file, new Error(data.message));
+        }
+
+        return data;
+      } catch (error) {
+        onError(file, error as Error);
       }
-
-      return data;
     },
   });
 
-  return { data, loading, uploadAndParseFile: mutateAsync };
+  const cancel = useCallback(() => {
+    controller.current.abort();
+    controller.current = new AbortController();
+  }, [controller]);
+
+  return { data, loading, uploadAndParseFile: mutateAsync, cancel };
 }
 
 export const useFetchExternalChatInfo = () => {

web/src/hooks/use-document-request.ts

@@ -1,4 +1,5 @@
 import { useHandleFilterSubmit } from '@/components/list-filter-bar/use-handle-filter-submit';
+import { ResponseType } from '@/interfaces/database/base';
 import {
   IDocumentInfo,
   IDocumentInfoFilter,
@@ -45,9 +46,9 @@ export const useUploadNextDocument = () => {
     data,
     isPending: loading,
     mutateAsync,
-  } = useMutation({
+  } = useMutation<ResponseType<IDocumentInfo[]>, Error, File[]>({
     mutationKey: [DocumentApiAction.UploadDocument],
-    mutationFn: async (fileList: File[]) => {
+    mutationFn: async (fileList) => {
       const formData = new FormData();
       formData.append('kb_id', id!);
       fileList.forEach((file: any) => {

web/src/hooks/use-llm-request.ts

@@ -8,9 +8,13 @@ import {
 } from '@/interfaces/database/llm';
 import { buildLlmUuid } from '@/utils/llm-util';
 
+export const enum LLMApiAction {
+  LlmList = 'llmList',
+}
+
 export const useFetchLlmList = (modelType?: LlmModelType) => {
   const { data } = useQuery<IThirdAiModelCollection>({
-    queryKey: ['llmList'],
+    queryKey: [LLMApiAction.LlmList],
     initialData: {},
     queryFn: async () => {
       const { data } = await userService.llm_list({ model_type: modelType });

web/src/hooks/use-user-setting-request.tsx

@@ -0,0 +1,464 @@
+import message from '@/components/ui/message';
+import { LanguageTranslationMap } from '@/constants/common';
+import { ResponseGetType } from '@/interfaces/database/base';
+import { IToken } from '@/interfaces/database/chat';
+import { ITenantInfo } from '@/interfaces/database/knowledge';
+import { ILangfuseConfig } from '@/interfaces/database/system';
+import {
+  ISystemStatus,
+  ITenant,
+  ITenantUser,
+  IUserInfo,
+} from '@/interfaces/database/user-setting';
+import { ISetLangfuseConfigRequestBody } from '@/interfaces/request/system';
+import userService, {
+  addTenantUser,
+  agreeTenant,
+  deleteTenantUser,
+  listTenant,
+  listTenantUser,
+} from '@/services/user-service';
+import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
+import { Modal } from 'antd';
+import DOMPurify from 'dompurify';
+import { isEmpty } from 'lodash';
+import { useCallback, useMemo, useState } from 'react';
+import { useTranslation } from 'react-i18next';
+import { history } from 'umi';
+
+export const enum UserSettingApiAction {
+  UserInfo = 'userInfo',
+  TenantInfo = 'tenantInfo',
+  SaveSetting = 'saveSetting',
+  FetchManualSystemTokenList = 'fetchManualSystemTokenList',
+  FetchSystemTokenList = 'fetchSystemTokenList',
+  RemoveSystemToken = 'removeSystemToken',
+  CreateSystemToken = 'createSystemToken',
+  ListTenantUser = 'listTenantUser',
+  AddTenantUser = 'addTenantUser',
+  DeleteTenantUser = 'deleteTenantUser',
+  ListTenant = 'listTenant',
+  AgreeTenant = 'agreeTenant',
+  SetLangfuseConfig = 'setLangfuseConfig',
+  DeleteLangfuseConfig = 'deleteLangfuseConfig',
+  FetchLangfuseConfig = 'fetchLangfuseConfig',
+}
+
+export const useFetchUserInfo = (): ResponseGetType<IUserInfo> => {
+  const { i18n } = useTranslation();
+
+  const { data, isFetching: loading } = useQuery({
+    queryKey: [UserSettingApiAction.UserInfo],
+    initialData: {},
+    gcTime: 0,
+    queryFn: async () => {
+      const { data } = await userService.user_info();
+      if (data.code === 0) {
+        i18n.changeLanguage(
+          LanguageTranslationMap[
+            data.data.language as keyof typeof LanguageTranslationMap
+          ],
+        );
+      }
+      return data?.data ?? {};
+    },
+  });
+
+  return { data, loading };
+};
+
+export const useFetchTenantInfo = (
+  showEmptyModelWarn = false,
+): ResponseGetType<ITenantInfo> => {
+  const { t } = useTranslation();
+  const { data, isFetching: loading } = useQuery({
+    queryKey: [UserSettingApiAction.TenantInfo],
+    initialData: {},
+    gcTime: 0,
+    queryFn: async () => {
+      const { data: res } = await userService.get_tenant_info();
+      if (res.code === 0) {
+        // llm_id is chat_id
+        // asr_id is speech2txt
+        const { data } = res;
+        if (
+          showEmptyModelWarn &&
+          (isEmpty(data.embd_id) || isEmpty(data.llm_id))
+        ) {
+          Modal.warning({
+            title: t('common.warn'),
+            content: (
+              <div
+                dangerouslySetInnerHTML={{
+                  __html: DOMPurify.sanitize(t('setting.modelProvidersWarn')),
+                }}
+              ></div>
+            ),
+            onOk() {
+              history.push('/user-setting/model');
+            },
+          });
+        }
+        data.chat_id = data.llm_id;
+        data.speech2text_id = data.asr_id;
+
+        return data;
+      }
+
+      return res;
+    },
+  });
+
+  return { data, loading };
+};
+
+export const useSelectParserList = (): Array<{
+  value: string;
+  label: string;
+}> => {
+  const { data: tenantInfo } = useFetchTenantInfo(true);
+
+  const parserList = useMemo(() => {
+    const parserArray: Array<string> = tenantInfo?.parser_ids?.split(',') ?? [];
+    return parserArray.map((x) => {
+      const arr = x.split(':');
+      return { value: arr[0], label: arr[1] };
+    });
+  }, [tenantInfo]);
+
+  return parserList;
+};
+
+export const useSaveSetting = () => {
+  const queryClient = useQueryClient();
+  const { t } = useTranslation();
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.SaveSetting],
+    mutationFn: async (
+      userInfo: { new_password: string } | Partial<IUserInfo>,
+    ) => {
+      const { data } = await userService.setting(userInfo);
+      if (data.code === 0) {
+        message.success(t('message.modified'));
+        queryClient.invalidateQueries({ queryKey: ['userInfo'] });
+      }
+      return data?.code;
+    },
+  });
+
+  return { data, loading, saveSetting: mutateAsync };
+};
+
+export const useFetchSystemVersion = () => {
+  const [version, setVersion] = useState('');
+  const [loading, setLoading] = useState(false);
+
+  const fetchSystemVersion = useCallback(async () => {
+    try {
+      setLoading(true);
+      const { data } = await userService.getSystemVersion();
+      if (data.code === 0) {
+        setVersion(data.data);
+        setLoading(false);
+      }
+    } catch (error) {
+      setLoading(false);
+    }
+  }, []);
+
+  return { fetchSystemVersion, version, loading };
+};
+
+export const useFetchSystemStatus = () => {
+  const [systemStatus, setSystemStatus] = useState<ISystemStatus>(
+    {} as ISystemStatus,
+  );
+  const [loading, setLoading] = useState(false);
+
+  const fetchSystemStatus = useCallback(async () => {
+    setLoading(true);
+    const { data } = await userService.getSystemStatus();
+    if (data.code === 0) {
+      setSystemStatus(data.data);
+      setLoading(false);
+    }
+  }, []);
+
+  return {
+    systemStatus,
+    fetchSystemStatus,
+    loading,
+  };
+};
+
+export const useFetchManualSystemTokenList = () => {
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.FetchManualSystemTokenList],
+    mutationFn: async () => {
+      const { data } = await userService.listToken();
+
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, fetchSystemTokenList: mutateAsync };
+};
+
+export const useFetchSystemTokenList = () => {
+  const {
+    data,
+    isFetching: loading,
+    refetch,
+  } = useQuery<IToken[]>({
+    queryKey: [UserSettingApiAction.FetchSystemTokenList],
+    initialData: [],
+    gcTime: 0,
+    queryFn: async () => {
+      const { data } = await userService.listToken();
+
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, refetch };
+};
+
+export const useRemoveSystemToken = () => {
+  const queryClient = useQueryClient();
+  const { t } = useTranslation();
+
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.RemoveSystemToken],
+    mutationFn: async (token: string) => {
+      const { data } = await userService.removeToken({}, token);
+      if (data.code === 0) {
+        message.success(t('message.deleted'));
+        queryClient.invalidateQueries({
+          queryKey: [UserSettingApiAction.FetchSystemTokenList],
+        });
+      }
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, removeToken: mutateAsync };
+};
+
+export const useCreateSystemToken = () => {
+  const queryClient = useQueryClient();
+
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.CreateSystemToken],
+    mutationFn: async (params: Record<string, any>) => {
+      const { data } = await userService.createToken(params);
+      if (data.code === 0) {
+        queryClient.invalidateQueries({
+          queryKey: [UserSettingApiAction.FetchSystemTokenList],
+        });
+      }
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, createToken: mutateAsync };
+};
+
+export const useListTenantUser = () => {
+  const { data: tenantInfo } = useFetchTenantInfo();
+  const tenantId = tenantInfo.tenant_id;
+  const {
+    data,
+    isFetching: loading,
+    refetch,
+  } = useQuery<ITenantUser[]>({
+    queryKey: [UserSettingApiAction.ListTenantUser, tenantId],
+    initialData: [],
+    gcTime: 0,
+    enabled: !!tenantId,
+    queryFn: async () => {
+      const { data } = await listTenantUser(tenantId);
+
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, refetch };
+};
+
+export const useAddTenantUser = () => {
+  const { data: tenantInfo } = useFetchTenantInfo();
+  const queryClient = useQueryClient();
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.AddTenantUser],
+    mutationFn: async (email: string) => {
+      const { data } = await addTenantUser(tenantInfo.tenant_id, email);
+      if (data.code === 0) {
+        queryClient.invalidateQueries({
+          queryKey: [UserSettingApiAction.ListTenantUser],
+        });
+      }
+      return data?.code;
+    },
+  });
+
+  return { data, loading, addTenantUser: mutateAsync };
+};
+
+export const useDeleteTenantUser = () => {
+  const { data: tenantInfo } = useFetchTenantInfo();
+  const queryClient = useQueryClient();
+  const { t } = useTranslation();
+
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.DeleteTenantUser],
+    mutationFn: async ({
+      userId,
+      tenantId,
+    }: {
+      userId: string;
+      tenantId?: string;
+    }) => {
+      const { data } = await deleteTenantUser({
+        tenantId: tenantId ?? tenantInfo.tenant_id,
+        userId,
+      });
+      if (data.code === 0) {
+        message.success(t('message.deleted'));
+        queryClient.invalidateQueries({
+          queryKey: [UserSettingApiAction.ListTenantUser],
+        });
+        queryClient.invalidateQueries({
+          queryKey: [UserSettingApiAction.ListTenant],
+        });
+      }
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, deleteTenantUser: mutateAsync };
+};
+
+export const useListTenant = () => {
+  const { data: tenantInfo } = useFetchTenantInfo();
+  const tenantId = tenantInfo.tenant_id;
+  const {
+    data,
+    isFetching: loading,
+    refetch,
+  } = useQuery<ITenant[]>({
+    queryKey: [UserSettingApiAction.ListTenant, tenantId],
+    initialData: [],
+    gcTime: 0,
+    enabled: !!tenantId,
+    queryFn: async () => {
+      const { data } = await listTenant();
+
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, refetch };
+};
+
+export const useAgreeTenant = () => {
+  const queryClient = useQueryClient();
+  const { t } = useTranslation();
+
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.AgreeTenant],
+    mutationFn: async (tenantId: string) => {
+      const { data } = await agreeTenant(tenantId);
+      if (data.code === 0) {
+        message.success(t('message.operated'));
+        queryClient.invalidateQueries({
+          queryKey: [UserSettingApiAction.ListTenant],
+        });
+      }
+      return data?.data ?? [];
+    },
+  });
+
+  return { data, loading, agreeTenant: mutateAsync };
+};
+
+export const useSetLangfuseConfig = () => {
+  const { t } = useTranslation();
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.SetLangfuseConfig],
+    mutationFn: async (params: ISetLangfuseConfigRequestBody) => {
+      const { data } = await userService.setLangfuseConfig(params);
+      if (data.code === 0) {
+        message.success(t('message.operated'));
+      }
+      return data?.code;
+    },
+  });
+
+  return { data, loading, setLangfuseConfig: mutateAsync };
+};
+
+export const useDeleteLangfuseConfig = () => {
+  const { t } = useTranslation();
+  const {
+    data,
+    isPending: loading,
+    mutateAsync,
+  } = useMutation({
+    mutationKey: [UserSettingApiAction.DeleteLangfuseConfig],
+    mutationFn: async () => {
+      const { data } = await userService.deleteLangfuseConfig();
+      if (data.code === 0) {
+        message.success(t('message.deleted'));
+      }
+      return data?.code;
+    },
+  });
+
+  return { data, loading, deleteLangfuseConfig: mutateAsync };
+};
+
+export const useFetchLangfuseConfig = () => {
+  const { data, isFetching: loading } = useQuery<ILangfuseConfig>({
+    queryKey: [UserSettingApiAction.FetchLangfuseConfig],
+    gcTime: 0,
+    queryFn: async () => {
+      const { data } = await userService.getLangfuseConfig();
+
+      return data?.data;
+    },
+  });
+
+  return { data, loading };
+};

web/src/layouts/bell-button.tsx

@@ -0,0 +1,28 @@
+import { Button } from '@/components/ui/button';
+import { useNavigateWithFromState } from '@/hooks/route-hook';
+import { useListTenant } from '@/hooks/use-user-setting-request';
+import { TenantRole } from '@/pages/user-setting/constants';
+import { BellRing } from 'lucide-react';
+import { useCallback, useMemo } from 'react';
+
+export function BellButton() {
+  const { data } = useListTenant();
+  const navigate = useNavigateWithFromState();
+
+  const showBell = useMemo(() => {
+    return data.some((x) => x.role === TenantRole.Invite);
+  }, [data]);
+
+  const handleBellClick = useCallback(() => {
+    navigate('/user-setting/team');
+  }, [navigate]);
+
+  return showBell ? (
+    <Button variant={'ghost'} onClick={handleBellClick}>
+      <div className="relative">
+        <BellRing className="size-4 " />
+        <span className="absolute size-1 rounded -right-1 -top-1 bg-red-600"></span>
+      </div>
+    </Button>
+  ) : null;
+}

web/src/layouts/next-header.tsx

@@ -31,6 +31,7 @@ import {
 import React, { useCallback, useMemo } from 'react';
 import { useTranslation } from 'react-i18next';
 import { useLocation } from 'umi';
+import { BellButton } from './bell-button';
 
 const handleDocHelpCLick = () => {
   window.open('https://ragflow.io/docs/dev/category/guides', 'target');
@@ -53,12 +54,6 @@ export function Header() {
     changeLanguage(key);
   };
 
-  // const { data } = useListTenant();
-
-  // const showBell = useMemo(() => {
-  //   return data.some((x) => x.role === TenantRole.Invite);
-  // }, [data]);
-
   const items = LanguageList.map((x) => ({
     key: x,
     label: <span>{LanguageMap[x as keyof typeof LanguageMap]}</span>,
@@ -68,10 +63,6 @@ export function Header() {
     setTheme(theme === ThemeEnum.Dark ? ThemeEnum.Light : ThemeEnum.Dark);
   }, [setTheme, theme]);
 
-  // const handleBellClick = useCallback(() => {
-  //   navigate('/user-setting/team');
-  // }, [navigate]);
-
   const tagsData = useMemo(
     () => [
       { path: Routes.Root, name: t('header.Root'), icon: House },
@@ -160,6 +151,7 @@ export function Header() {
         <Button variant={'ghost'} onClick={onThemeClick}>
           {theme === 'light' ? <Sun /> : <Moon />}
         </Button>
+        <BellButton></BellButton>
         <div className="relative">
           <RAGFlowAvatar
             name={nickname}

web/src/locales/en.ts

@@ -1,6 +1,8 @@
 export default {
   translation: {
     common: {
+      noResults: 'No results.',
+      selectPlaceholder: 'select value',
       delete: 'Delete',
       deleteModalTitle: 'Are you sure to delete this item?',
       ok: 'Yes',
@@ -39,6 +41,9 @@ export default {
       previousPage: 'Previous',
       nextPage: 'Next',
       add: 'Add',
+      remove: 'Remove',
+      search: 'Search',
+      noDataFound: 'No data found.',
       promptPlaceholder: `Please input or use / to quickly insert variables.`,
       mcp: {
         namePlaceholder: 'My MCP Server',
@@ -70,7 +75,7 @@ export default {
       review: 'from 500+ reviews',
     },
     header: {
-      knowledgeBase: 'Knowledge Base',
+      knowledgeBase: 'Dataset',
       chat: 'Chat',
       register: 'Register',
       signin: 'Sign in',
@@ -86,7 +91,7 @@ export default {
     knowledgeList: {
       welcome: 'Welcome back',
       description: 'Which knowledge bases will you use today?',
-      createKnowledgeBase: 'Create knowledge base',
+      createKnowledgeBase: 'Create Dataset',
       name: 'Name',
       namePlaceholder: 'Please input name!',
       doc: 'Docs',
@@ -94,6 +99,16 @@ export default {
       noMoreData: `That's all. Nothing more.`,
     },
     knowledgeDetails: {
+      created: 'Created',
+      learnMore: 'Learn More',
+      general: 'General',
+      chunkMethodTab: 'Chunk Method',
+      testResults: 'Test Results',
+      testSetting: 'Test Setting',
+      retrievalTesting: 'Retrieval Testing',
+      retrievalTestingDescription:
+        'Conduct a retrieval test to check if RAGFlow can recover the intended content for the LLM.',
+      Parse: 'Parse',
       dataset: 'Dataset',
       testing: 'Retrieval testing',
       files: 'files',
@@ -181,7 +196,7 @@ export default {
       delimiterTip:
         'A delimiter or separator can consist of one or multiple special characters. If it is multiple characters, ensure they are enclosed in backticks( ``). For example, if you configure your delimiters like this: \\n`##`;, then your texts will be separated at line breaks, double hash symbols (##), and semicolons.',
       html4excel: 'Excel to HTML',
-      html4excelTip: `Use with the General chunking method. When disabled, spreadsheets (XLSX or XLS(Excel 97-2003)) in the knowledge base will be parsed into key-value pairs. When enabled, they will be parsed into HTML tables, splitting every 12 rows if the original table has more than 12 rows.`,
+      html4excelTip: `Use with the General chunking method. When disabled, spreadsheets (XLSX or XLS(Excel 97-2003)) in the knowledge base will be parsed into key-value pairs. When enabled, they will be parsed into HTML tables, splitting every 12 rows if the original table has more than 12 rows. See https://ragflow.io/docs/dev/enable_excel2html for details.`,
       autoKeywords: 'Auto-keyword',
       autoKeywordsTip: `Automatically extract N keywords for each chunk to increase their ranking for queries containing those keywords. Be aware that extra tokens will be consumed by the chat model specified in 'System model settings'. You can check or update the added keywords for a chunk from the chunk list. For details, see https://ragflow.io/docs/dev/autokeyword_autoquestion.`,
       autoQuestions: 'Auto-question',
@@ -479,6 +494,7 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
       improvise: 'Improvise',
       precise: 'Precise',
       balance: 'Balance',
+      custom: 'Custom',
       freedomTip: `A shortcut to 'Temperature', 'Top P', 'Presence penalty', and 'Frequency penalty' settings, indicating the freedom level of the model. This parameter has three options: Select 'Improvise' to produce more creative responses; select 'Precise' (default) to produce more conservative responses; 'Balance' is a middle ground between 'Improvise' and 'Precise'.`,
       temperature: 'Temperature',
       temperatureMessage: 'Temperature is required',
@@ -825,7 +841,7 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
     fileManager: {
       name: 'Name',
       uploadDate: 'Upload Date',
-      knowledgeBase: 'Knowledge Base',
+      knowledgeBase: 'Dataset',
       size: 'Size',
       action: 'Action',
       addToKnowledge: 'Link to Knowledge Base',
@@ -845,13 +861,90 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
       uploadLimit:
         'Each file must not exceed 10MB, and the total number of files must not exceed 128.',
       destinationFolder: 'Destination folder',
+      pleaseUploadAtLeastOneFile: 'Please upload at least one file',
     },
     flow: {
+      days: 'Days',
+      beginInput: 'Begin Input',
+      ref: 'Variable',
+      stockCode: 'Stock Code',
+      apiKeyPlaceholder:
+        'YOUR_API_KEY (obtained from https://serpapi.com/manage-api-key)',
+      flowStart: 'Start',
+      flowNum: 'Num',
+      test: 'Test',
+      extractDepth: 'Extract Depth',
+      format: 'Format',
+      basic: 'basic',
+      advanced: 'advanced',
+      general: 'general',
+      searchDepth: 'Search Depth',
+      tavilyTopic: 'Tavily Topic',
+      maxResults: 'Max Results',
+      includeAnswer: 'Include Answer',
+      includeRawContent: 'Include Raw Content',
+      includeImages: 'Include Images',
+      includeImageDescriptions: 'Include Image Descriptions',
+      includeDomains: 'Include Domains',
+      ExcludeDomains: 'Exclude Domains',
+      Days: 'Days',
+      comma: 'Comma',
+      semicolon: 'Semicolon',
+      period: 'Period',
+      lineBreak: 'Line Break',
+      tab: 'Tab',
+      space: 'Space',
+      delimiters: 'Delimiters',
+      merge: 'Merge',
+      split: 'Split',
+      script: 'Script',
+      iterationItemDescription:
+        'It represents the current element in the iteration, which can be referenced and manipulated in subsequent steps.',
+      guidingQuestion: 'Guidance Question',
+      onFailure: 'On Failure',
+      userPromptDefaultValue:
+        'This is the order you need to send to the agent.',
+      search: 'Search',
+      communication: 'Communication',
+      developer: 'Developer',
+      typeCommandOrsearch: 'Type a command or search...',
+      builtIn: 'Built-in',
+      ExceptionDefaultValue: 'Exception default value',
+      exceptionMethod: 'Exception method',
+      maxRounds: 'Max rounds',
+      delayEfterError: 'Delay after error',
+      maxRetries: 'Max retries',
+      advancedSettings: 'Advanced Settings',
+      addTools: 'Add Tools',
+      sysPromptDefultValue: `
+      <role>
+        You are {{agent_name}}, an AI assistant specialized in {{domain_or_task}}.
+      </role>
+      <instructions>
+        1. Understand the user’s request.  
+        2. Decompose it into logical subtasks.  
+        3. Execute each subtask step by step, reasoning transparently.  
+        4. Validate accuracy and consistency.  
+        5. Summarize the final result clearly.
+      </instructions>`,
+      singleLineText: 'Single-line text',
+      multimodalModels: 'Multimodal Models',
+      textOnlyModels: 'Text-only Models',
+      allModels: 'All Models',
+      codeExecDescription: 'Write your custom Python or Javascript logic.',
+      stringTransformDescription:
+        'Modifies text content. Currently supports: Splitting or concatenating text.',
+      foundation: 'Foundation',
+      tools: 'Tools',
+      dataManipulation: 'Data Manipulation',
+      flow: 'Flow',
+      dialog: 'Dialogue',
       cite: 'Cite',
       citeTip: 'citeTip',
       name: 'Name',
       nameMessage: 'Please input name',
       description: 'Description',
+      descriptionMessage: 'This is an agent for a specific task.',
       examples: 'Examples',
       to: 'To',
       msg: 'Messages',
@@ -1276,6 +1369,7 @@ This delimiter is used to split the input text into several text pieces echo of
       variableSettings: 'Variable settings',
       globalVariables: 'Global variables',
       systemPrompt: 'System prompt',
+      userPrompt: 'User prompt',
       addCategory: 'Add category',
       categoryName: 'Category name',
       nextStep: 'Next step',
@@ -1339,10 +1433,15 @@ This delimiter is used to split the input text into several text pieces echo of
       openingSwitchTip:
         'Your users will see this welcome message at the beginning.',
       modeTip: 'The mode defines how the workflow is initiated.',
+      mode: 'Mode',
+      conversational: 'conversational',
+      task: 'task',
       beginInputTip:
         'By defining input parameters, this content can be accessed by other components in subsequent processes.',
       query: 'Query variables',
+      queryTip: 'Select the variable you want to use',
       agent: 'Agent',
+      addAgent: 'Add Agent',
       agentDescription:
         'Builds agent components equipped with reasoning, tool usage, and multi-agent collaboration. ',
       maxRecords: 'Max records',
@@ -1441,6 +1540,7 @@ This delimiter is used to split the input text into several text pieces echo of
       showQueryMindmap: 'Show Query Mindmap',
       embedApp: 'Embed App',
       relatedSearch: 'Related Search',
+      descriptionValue: 'You are an intelligent assistant.',
       okText: 'Save',
       cancelText: 'Cancel',
     },

web/src/locales/fr.ts

@@ -240,9 +240,8 @@ export default {
       promptTip:
         'Décrivez la tâche attendue du LLM, ses réponses, ses exigences, etc. Utilisez `/` pour afficher les variables disponibles.',
       promptMessage: 'Le prompt est requis',
-      promptText: `Veuillez résumer les paragraphes suivants. Attention aux chiffres, ne pas inventer. Paragraphes suivants : {cluster_content
-      }
-    Le contenu à résumer est ci-dessus.`,
+      promptText: `Veuillez résumer les paragraphes suivants. Attention aux chiffres, ne pas inventer. Paragraphes suivants : {cluster_content}
+  Le contenu à résumer est ci-dessus.`,
       maxToken: 'Nombre maximal de tokens',
       maxTokenTip: 'Nombre maximal de tokens générés par résumé.',
       maxTokenMessage: 'Nombre maximal de tokens requis',

web/src/locales/zh-traditional.ts

@@ -69,7 +69,7 @@ export default {
       setting: '用戶設置',
       logout: '登出',
       fileManager: '文件管理',
-      flow: 'Agent',
+      flow: '智能體',
       search: '搜尋',
       welcome: '歡迎來到',
     },
@@ -454,6 +454,7 @@ export default {
       improvise: '即興創作',
       precise: '精確',
       balance: '平衡',
+      custom: '自定義',
       freedomTip: `“精確”意味著法學碩士會保守並謹慎地回答你的問題。“即興發揮”意味著你希望法學碩士能夠自由地暢所欲言。“平衡”是謹慎與自由之間的平衡。`,
       temperature: '溫度',
       temperatureMessage: '溫度是必填項',
@@ -763,6 +764,23 @@ export default {
       destinationFolder: '目標資料夾',
     },
     flow: {
+      line: '單行文本',
+      paragraph: '段落文字',
+      options: '選項',
+      file: '文件',
+      integer: '數字',
+      boolean: '布爾值',
+      multimodalModels: '多模態模型',
+      textOnlyModels: '進文本模型',
+      allModels: '所有模型',
+      codeExecDescription: '用 Python 或者 Javascript 編寫自定義邏輯',
+      stringTransformDescription:
+        '修改文本内容，目前支持文本分割、文本拼接操作',
+      foundation: '基礎',
+      tools: '工具',
+      dataManipulation: '數據操控',
+      flow: '流程',
+      dialog: '對話',
       cite: '引用',
       citeTip: 'citeTip',
       name: '名稱',
@@ -804,7 +822,7 @@ export default {
       promptText: `請總結以下段落。注意數字，不要胡編亂造。段落如下：
 {input}
 以上就是你需要總結的內容。`,
-      createGraph: '建立 Agent',
+      createGraph: '創建智能體',
       createFromTemplates: '從模板創建',
       retrieval: '知識檢索',
       generate: '生成回答',