Docs: Added v0.20.5 release notes. (#10014)

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update

.github/workflows/tests.yml

@@ -67,6 +67,7 @@ jobs:
 
       - name: Start ragflow:nightly-slim
         run: |
+          sudo docker compose -f docker/docker-compose.yml down --volumes --remove-orphans
           echo -e "\nRAGFLOW_IMAGE=infiniflow/ragflow:nightly-slim" >> docker/.env
           sudo docker compose -f docker/docker-compose.yml up -d
 

README.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.5">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -71,10 +71,7 @@
 
 ## 💡 What is RAGFlow?
 
-[RAGFlow](https://ragflow.io/) is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document
+[RAGFlow](https://ragflow.io/) is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs. It offers a streamlined RAG workflow adaptable to enterprises of any scale. Powered by a converged context engine and pre-built agent templates, RAGFlow enables developers to transform complex data into high-fidelity, production-ready AI systems with exceptional efficiency and precision.
-understanding. It offers a streamlined RAG workflow for businesses of any scale, combining LLM (Large Language Models)
-to provide truthful question-answering capabilities, backed by well-founded citations from various complex formatted
-data.
 
 ## 🎮 Demo
 
@@ -190,7 +187,7 @@ releases! 🌟
 > All Docker images are built for x86 platforms. We don't currently offer Docker images for ARM64.
 > If you are on an ARM64 platform, follow [this guide](https://ragflow.io/docs/dev/build_docker_image) to build a Docker image compatible with your system.
 
-   > The command below downloads the `v0.20.3-slim` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.3-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` for the full edition `v0.20.3`.
+   > The command below downloads the `v0.20.5-slim` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.5-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5` for the full edition `v0.20.5`.
 
    ```bash
    $ cd ragflow/docker
@@ -203,8 +200,8 @@ releases! 🌟
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    |-------------------|-----------------|-----------------------|--------------------------|
-   | v0.20.3           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.5           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.3-slim      | &approx;2       | ❌                   | Stable release            |
+   | v0.20.5-slim      | &approx;2       | ❌                   | Stable release            |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                   | _Unstable_ nightly build  |
 
@@ -307,7 +304,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
 
 ## 🔨 Launch service from source for development
 
-1. Install uv, or skip this step if it is already installed:
+1. Install `uv` and `pre-commit`, or skip this step if they are already installed:
 
    ```bash
    pipx install uv pre-commit
@@ -348,8 +345,10 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    sudo apt-get install libjemalloc-dev
    # centos
    sudo yum install jemalloc
+   # mac
+   sudo brew install jemalloc
    ```
-   
+
 6. Launch backend service:
 
    ```bash

README_id.md

@@ -22,7 +22,7 @@
         <img alt="Lencana Daring" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.5">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Rilis%20Terbaru" alt="Rilis Terbaru">
@@ -67,7 +67,7 @@
 
 ## 💡 Apa Itu RAGFlow?
 
-[RAGFlow](https://ragflow.io/) adalah mesin RAG (Retrieval-Augmented Generation) open-source berbasis pemahaman dokumen yang mendalam. Platform ini menyediakan alur kerja RAG yang efisien untuk bisnis dengan berbagai skala, menggabungkan LLM (Large Language Models) untuk menyediakan kemampuan tanya-jawab yang benar dan didukung oleh referensi dari data terstruktur kompleks.
+[RAGFlow](https://ragflow.io/) adalah mesin RAG (Retrieval-Augmented Generation) open-source terkemuka yang mengintegrasikan teknologi RAG mutakhir dengan kemampuan Agent untuk menciptakan lapisan kontekstual superior bagi LLM. Menyediakan alur kerja RAG yang efisien dan dapat diadaptasi untuk perusahaan segala skala. Didukung oleh mesin konteks terkonvergensi dan template Agent yang telah dipra-bangun, RAGFlow memungkinkan pengembang mengubah data kompleks menjadi sistem AI kesetiaan-tinggi dan siap-produksi dengan efisiensi dan presisi yang luar biasa.
 
 ## 🎮 Demo
 
@@ -181,7 +181,7 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
 > Semua gambar Docker dibangun untuk platform x86. Saat ini, kami tidak menawarkan gambar Docker untuk ARM64.
 > Jika Anda menggunakan platform ARM64, [silakan gunakan panduan ini untuk membangun gambar Docker yang kompatibel dengan sistem Anda](https://ragflow.io/docs/dev/build_docker_image).
 
-> Perintah di bawah ini mengunduh edisi v0.20.3-slim dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.20.3-slim, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server. Misalnya, atur RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3 untuk edisi lengkap v0.20.3.
+> Perintah di bawah ini mengunduh edisi v0.20.5-slim dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.20.5-slim, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server. Misalnya, atur RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5 untuk edisi lengkap v0.20.5.
 
 ```bash
 $ cd ragflow/docker
@@ -194,8 +194,8 @@ $ docker compose -f docker-compose.yml up -d
 
 | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
 | ----------------- | --------------- | --------------------- | ------------------------ |
-| v0.20.3           | &approx;9       | :heavy_check_mark:    | Stable release           |
+| v0.20.5           | &approx;9       | :heavy_check_mark:    | Stable release           |
-| v0.20.3-slim      | &approx;2       | ❌                    | Stable release           |
+| v0.20.5-slim      | &approx;2       | ❌                    | Stable release           |
 | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
 | nightly-slim      | &approx;2       | ❌                    | _Unstable_ nightly build |
 
@@ -271,7 +271,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
 
 ## 🔨 Menjalankan Aplikasi dari untuk Pengembangan
 
-1. Instal uv, atau lewati langkah ini jika sudah terinstal:
+1. Instal `uv` dan `pre-commit`, atau lewati langkah ini jika sudah terinstal:
 
    ```bash
    pipx install uv pre-commit
@@ -312,6 +312,8 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    sudo apt-get install libjemalloc-dev
    # centos
    sudo yum install jemalloc
+   # mac
+   sudo brew install jemalloc
    ```
 
 6. Jalankan aplikasi backend:

README_ja.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.5">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -47,7 +47,7 @@
 
 ## 💡 RAGFlow とは？
 
-[RAGFlow](https://ragflow.io/) は、深い文書理解に基づいたオープンソースの RAG (Retrieval-Augmented Generation) エンジンである。LLM（大規模言語モデル）を組み合わせることで、様々な複雑なフォーマットのデータから根拠のある引用に裏打ちされた、信頼できる質問応答機能を実現し、あらゆる規模のビジネスに適した RAG ワークフローを提供します。
+[RAGFlow](https://ragflow.io/) は、先進的なRAG（Retrieval-Augmented Generation）技術と Agent 機能を融合し、大規模言語モデル（LLM）に優れたコンテキスト層を構築する最先端のオープンソース RAG エンジンです。あらゆる規模の企業に対応可能な合理化された RAG ワークフローを提供し、統合型コンテキストエンジンと事前構築されたAgentテンプレートにより、開発者が複雑なデータを驚異的な効率性と精度で高精細なプロダクションレディAIシステムへ変換することを可能にします。
 
 ## 🎮 Demo
 
@@ -160,7 +160,7 @@
 > 現在、公式に提供されているすべての Docker イメージは x86 アーキテクチャ向けにビルドされており、ARM64 用の Docker イメージは提供されていません。
 > ARM64 アーキテクチャのオペレーティングシステムを使用している場合は、[このドキュメント](https://ragflow.io/docs/dev/build_docker_image)を参照して Docker イメージを自分でビルドしてください。
 
-   > 以下のコマンドは、RAGFlow Docker イメージの v0.20.3-slim エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.20.3-slim とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。例えば、完全版 v0.20.3 をダウンロードするには、RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3 と設定します。
+   > 以下のコマンドは、RAGFlow Docker イメージの v0.20.5-slim エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.20.5-slim とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。例えば、完全版 v0.20.5 をダウンロードするには、RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5 と設定します。
 
    ```bash
    $ cd ragflow/docker
@@ -173,8 +173,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.3           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.5           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.3-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.5-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 
@@ -266,7 +266,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
 
 ## 🔨 ソースコードからサービスを起動する方法
 
-1. uv をインストールする。すでにインストールされている場合は、このステップをスキップしてください:
+1. `uv` と `pre-commit` をインストールする。すでにインストールされている場合は、このステップをスキップしてください:
 
    ```bash
    pipx install uv pre-commit
@@ -301,12 +301,14 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    ```
 
 5. オペレーティングシステムにjemallocがない場合は、次のようにインストールします:
-   
+
    ```bash
    # ubuntu
    sudo apt-get install libjemalloc-dev
    # centos
    sudo yum install jemalloc
+   # mac
+   sudo brew install jemalloc
    ```
 
 6. バックエンドサービスを起動する:

README_ko.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.5">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -47,7 +47,7 @@
 
 ## 💡 RAGFlow란?
 
-[RAGFlow](https://ragflow.io/)는 심층 문서 이해에 기반한 오픈소스 RAG (Retrieval-Augmented Generation) 엔진입니다. 이 엔진은 대규모 언어 모델(LLM)과 결합하여 정확한 질문 응답 기능을 제공하며, 다양한 복잡한 형식의 데이터에서 신뢰할 수 있는 출처를 바탕으로 한 인용을 통해 이를 뒷받침합니다. RAGFlow는 규모에 상관없이 모든 기업에 최적화된 RAG 워크플로우를 제공합니다.
+[RAGFlow](https://ragflow.io/) 는 최첨단 RAG(Retrieval-Augmented Generation)와 Agent 기능을 융합하여 대규모 언어 모델(LLM)을 위한 우수한 컨텍스트 계층을 생성하는 선도적인 오픈소스 RAG 엔진입니다. 모든 규모의 기업에 적용 가능한 효율적인 RAG 워크플로를 제공하며, 통합 컨텍스트 엔진과 사전 구축된 Agent 템플릿을 통해 개발자들이 복잡한 데이터를 예외적인 효율성과 정밀도로 고급 구현도의 프로덕션 준비 완료 AI 시스템으로 변환할 수 있도록 지원합니다.
 
 ## 🎮 데모
 
@@ -160,7 +160,7 @@
 > 모든 Docker 이미지는 x86 플랫폼을 위해 빌드되었습니다. 우리는 현재 ARM64 플랫폼을 위한 Docker 이미지를 제공하지 않습니다.
 > ARM64 플랫폼을 사용 중이라면, [시스템과 호환되는 Docker 이미지를 빌드하려면 이 가이드를 사용해 주세요](https://ragflow.io/docs/dev/build_docker_image).
 
-   > 아래 명령어는 RAGFlow Docker 이미지의 v0.20.3-slim 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.20.3-slim과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오. 예를 들어, 전체 버전인 v0.20.3을 다운로드하려면 RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3로 설정합니다.
+   > 아래 명령어는 RAGFlow Docker 이미지의 v0.20.5-slim 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.20.5-slim과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오. 예를 들어, 전체 버전인 v0.20.5을 다운로드하려면 RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5로 설정합니다.
 
    ```bash
    $ cd ragflow/docker
@@ -173,8 +173,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.3           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.5           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.3-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.5-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 
@@ -265,7 +265,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
 
 ## 🔨 소스 코드로 서비스를 시작합니다.
 
-1. uv를 설치하거나 이미 설치된 경우 이 단계를 건너뜁니다:
+1. `uv` 와 `pre-commit` 을 설치하거나, 이미 설치된 경우 이 단계를 건너뜁니다:
 
    ```bash
    pipx install uv pre-commit
@@ -306,6 +306,8 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    sudo apt-get install libjemalloc-dev
    # centos
    sudo yum install jemalloc
+   # mac
+   sudo brew install jemalloc
    ```
 
 6. 백엔드 서비스를 시작합니다:
@@ -339,7 +341,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
    ```bash
    pkill -f "ragflow_server.py|task_executor.py"
    ```
-   
+
 
 ## 📚 문서
 

README_pt_br.md

@@ -22,7 +22,7 @@
         <img alt="Badge Estático" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.5">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Última%20Relese" alt="Última Versão">
@@ -67,7 +67,7 @@
 
 ## 💡 O que é o RAGFlow?
 
-[RAGFlow](https://ragflow.io/) é um mecanismo RAG (Geração Aumentada por Recuperação) de código aberto baseado em entendimento profundo de documentos. Ele oferece um fluxo de trabalho RAG simplificado para empresas de qualquer porte, combinando LLMs (Modelos de Linguagem de Grande Escala) para fornecer capacidades de perguntas e respostas verídicas, respaldadas por citações bem fundamentadas de diversos dados complexos formatados.
+[RAGFlow](https://ragflow.io/) é um mecanismo de RAG (Retrieval-Augmented Generation) open-source líder que fusiona tecnologias RAG de ponta com funcionalidades Agent para criar uma camada contextual superior para LLMs. Oferece um fluxo de trabalho RAG otimizado adaptável a empresas de qualquer escala. Alimentado por um motor de contexto convergente e modelos Agent pré-construídos, o RAGFlow permite que desenvolvedores transformem dados complexos em sistemas de IA de alta fidelidade e pronto para produção com excepcional eficiência e precisão.
 
 ## 🎮 Demo
 
@@ -180,7 +180,7 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
 > Todas as imagens Docker são construídas para plataformas x86. Atualmente, não oferecemos imagens Docker para ARM64.
 > Se você estiver usando uma plataforma ARM64, por favor, utilize [este guia](https://ragflow.io/docs/dev/build_docker_image) para construir uma imagem Docker compatível com o seu sistema.
 
-    > O comando abaixo baixa a edição `v0.20.3-slim` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.20.3-slim`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor. Por exemplo: defina `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` para a edição completa `v0.20.3`.
+    > O comando abaixo baixa a edição `v0.20.5-slim` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.20.5-slim`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor. Por exemplo: defina `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5` para a edição completa `v0.20.5`.
 
     ```bash
     $ cd ragflow/docker
@@ -193,8 +193,8 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
 
     | Tag da imagem RAGFlow | Tamanho da imagem (GB) | Possui modelos de incorporação? | Estável?                 |
     | --------------------- | ---------------------- | ------------------------------- | ------------------------ |
-    | v0.20.3               | ~9                     | :heavy_check_mark:              | Lançamento estável       |
+    | v0.20.5               | ~9                     | :heavy_check_mark:              | Lançamento estável       |
-    | v0.20.3-slim          | ~2                     | ❌                              | Lançamento estável       |
+    | v0.20.5-slim          | ~2                     | ❌                              | Lançamento estável       |
     | nightly               | ~9                     | :heavy_check_mark:              | _Instável_ build noturno |
     | nightly-slim          | ~2                     | ❌                               | _Instável_ build noturno |
 
@@ -289,7 +289,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
 
 ## 🔨 Lançar o serviço a partir do código-fonte para desenvolvimento
 
-1. Instale o `uv`, ou pule esta etapa se ele já estiver instalado:
+1. Instale o `uv` e o `pre-commit`, ou pule esta etapa se eles já estiverem instalados:
 
    ```bash
    pipx install uv pre-commit
@@ -330,6 +330,8 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
     sudo apt-get install libjemalloc-dev
     # centos
     sudo yum instalar jemalloc
+    # mac
+    sudo brew install jemalloc
     ```
 
 6. Lance o serviço de back-end:

README_tzh.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.5">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -70,7 +70,7 @@
 
 ## 💡 RAGFlow 是什麼？
 
-[RAGFlow](https://ragflow.io/) 是一款基於深度文件理解所建構的開源 RAG（Retrieval-Augmented Generation）引擎。 RAGFlow 可以為各種規模的企業及個人提供一套精簡的 RAG 工作流程，結合大語言模型（LLM）針對用戶各類不同的複雜格式數據提供可靠的問答以及有理有據的引用。
+[RAGFlow](https://ragflow.io/) 是一款領先的開源 RAG（Retrieval-Augmented Generation）引擎，通過融合前沿的 RAG 技術與 Agent 能力，為大型語言模型提供卓越的上下文層。它提供可適配任意規模企業的端到端 RAG 工作流，憑藉融合式上下文引擎與預置的 Agent 模板，助力開發者以極致效率與精度將複雜數據轉化為高可信、生產級的人工智能系統。
 
 ## 🎮 Demo 試用
 
@@ -183,7 +183,7 @@
 > 所有 Docker 映像檔都是為 x86 平台建置的。目前，我們不提供 ARM64 平台的 Docker 映像檔。
 > 如果您使用的是 ARM64 平台，請使用 [這份指南](https://ragflow.io/docs/dev/build_docker_image) 來建置適合您系統的 Docker 映像檔。
 
-   > 執行以下指令會自動下載 RAGFlow slim Docker 映像 `v0.20.3-slim`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.20.3-slim` 的 Docker 映像，請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。例如，你可以透過設定 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` 來下載 RAGFlow 鏡像的 `v0.20.3` 完整發行版。
+   > 執行以下指令會自動下載 RAGFlow slim Docker 映像 `v0.20.5-slim`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.20.5-slim` 的 Docker 映像，請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。例如，你可以透過設定 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5` 來下載 RAGFlow 鏡像的 `v0.20.5` 完整發行版。
 
    ```bash
    $ cd ragflow/docker
@@ -196,8 +196,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.3           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.5           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.3-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.5-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 
@@ -301,7 +301,7 @@ docker build --platform linux/amd64 --build-arg NEED_MIRROR=1 -f Dockerfile -t i
 
 ## 🔨 以原始碼啟動服務
 
-1. 安裝 uv。如已安裝，可跳過此步驟：
+1. 安裝 `uv` 和 `pre-commit`。如已安裝，可跳過此步驟：
 
    ```bash
    pipx install uv pre-commit
@@ -343,6 +343,8 @@ docker build --platform linux/amd64 --build-arg NEED_MIRROR=1 -f Dockerfile -t i
    sudo apt-get install libjemalloc-dev
    # centos
    sudo yum install jemalloc
+   # mac
+   sudo brew install jemalloc
    ```
 
 6. 啟動後端服務：

README_zh.md

@@ -22,7 +22,7 @@
         <img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
     </a>
     <a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
-        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.3">
+        <img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.20.5">
     </a>
     <a href="https://github.com/infiniflow/ragflow/releases/latest">
         <img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -70,7 +70,7 @@
 
 ## 💡 RAGFlow 是什么？
 
-[RAGFlow](https://ragflow.io/) 是一款基于深度文档理解构建的开源 RAG（Retrieval-Augmented Generation）引擎。RAGFlow 可以为各种规模的企业及个人提供一套精简的 RAG 工作流程，结合大语言模型（LLM）针对用户各类不同的复杂格式数据提供可靠的问答以及有理有据的引用。
+[RAGFlow](https://ragflow.io/) 是一款领先的开源检索增强生成（RAG）引擎，通过融合前沿的 RAG 技术与 Agent 能力，为大型语言模型提供卓越的上下文层。它提供可适配任意规模企业的端到端 RAG 工作流，凭借融合式上下文引擎与预置的 Agent 模板，助力开发者以极致效率与精度将复杂数据转化为高可信、生产级的人工智能系统。
 
 ## 🎮 Demo 试用
 
@@ -183,7 +183,7 @@
 > 请注意，目前官方提供的所有 Docker 镜像均基于 x86 架构构建，并不提供基于 ARM64 的 Docker 镜像。
 > 如果你的操作系统是 ARM64 架构，请参考[这篇文档](https://ragflow.io/docs/dev/build_docker_image)自行构建 Docker 镜像。
 
-   > 运行以下命令会自动下载 RAGFlow slim Docker 镜像 `v0.20.3-slim`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.20.3-slim` 的 Docker 镜像，请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。比如，你可以通过设置 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` 来下载 RAGFlow 镜像的 `v0.20.3` 完整发行版。
+   > 运行以下命令会自动下载 RAGFlow slim Docker 镜像 `v0.20.5-slim`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.20.5-slim` 的 Docker 镜像，请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。比如，你可以通过设置 `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5` 来下载 RAGFlow 镜像的 `v0.20.5` 完整发行版。
 
    ```bash
    $ cd ragflow/docker
@@ -196,8 +196,8 @@
 
    | RAGFlow image tag | Image size (GB) | Has embedding models? | Stable?                  |
    | ----------------- | --------------- | --------------------- | ------------------------ |
-   | v0.20.3           | &approx;9       | :heavy_check_mark:    | Stable release           |
+   | v0.20.5           | &approx;9       | :heavy_check_mark:    | Stable release           |
-   | v0.20.3-slim      | &approx;2       | ❌                    | Stable release           |
+   | v0.20.5-slim      | &approx;2       | ❌                    | Stable release           |
    | nightly           | &approx;9       | :heavy_check_mark:    | _Unstable_ nightly build |
    | nightly-slim      | &approx;2       | ❌                     | _Unstable_ nightly build |
 
@@ -301,7 +301,7 @@ docker build --platform linux/amd64 --build-arg NEED_MIRROR=1 -f Dockerfile -t i
 
 ## 🔨 以源代码启动服务
 
-1. 安装 uv。如已经安装，可跳过本步骤：
+1. 安装 `uv` 和 `pre-commit`。如已经安装，可跳过本步骤：
 
    ```bash
    pipx install uv pre-commit
@@ -342,6 +342,8 @@ docker build --platform linux/amd64 --build-arg NEED_MIRROR=1 -f Dockerfile -t i
    sudo apt-get install libjemalloc-dev
    # centos
    sudo yum install jemalloc
+   # mac
+   sudo brew install jemalloc
    ```
 
 6. 启动后端服务：

agent/canvas.py

@@ -16,6 +16,7 @@
 import base64
 import json
 import logging
+import re
 import time
 from concurrent.futures import ThreadPoolExecutor
 from copy import deepcopy
@@ -29,83 +30,52 @@ from api.utils import get_uuid, hash_str2int
 from rag.prompts.prompts import chunks_format
 from rag.utils.redis_conn import REDIS_CONN
 
-
+class Graph:
-class Canvas:
     """
-    dsl = {
+        dsl = {
-        "components": {
-            "begin": {
-                "obj":{
-                    "component_name": "Begin",
-                    "params": {},
-                },
-                "downstream": ["answer_0"],
-                "upstream": [],
-            },
-            "retrieval_0": {
-                "obj": {
-                    "component_name": "Retrieval",
-                    "params": {}
-                },
-                "downstream": ["generate_0"],
-                "upstream": ["answer_0"],
-            },
-            "generate_0": {
-                "obj": {
-                    "component_name": "Generate",
-                    "params": {}
-                },
-                "downstream": ["answer_0"],
-                "upstream": ["retrieval_0"],
-            }
-        },
-        "history": [],
-        "path": ["begin"],
-        "retrieval": {"chunks": [], "doc_aggs": []},
-        "globals": {
-            "sys.query": "",
-            "sys.user_id": tenant_id,
-            "sys.conversation_turns": 0,
-            "sys.files": []
-        }
-    }
-    """
-
-    def __init__(self, dsl: str, tenant_id=None, task_id=None):
-        self.path = []
-        self.history = []
-        self.components = {}
-        self.error = ""
-        self.globals = {
-            "sys.query": "",
-            "sys.user_id": tenant_id,
-            "sys.conversation_turns": 0,
-            "sys.files": []
-        }
-        self.dsl = json.loads(dsl) if dsl else {
             "components": {
                 "begin": {
-                    "obj": {
+                    "obj":{
                         "component_name": "Begin",
-                        "params": {
+                        "params": {},
-                            "prologue": "Hi there!"
-                        }
                     },
-                    "downstream": [],
+                    "downstream": ["answer_0"],
                     "upstream": [],
-                    "parent_id": ""
+                },
+                "retrieval_0": {
+                    "obj": {
+                        "component_name": "Retrieval",
+                        "params": {}
+                    },
+                    "downstream": ["generate_0"],
+                    "upstream": ["answer_0"],
+                },
+                "generate_0": {
+                    "obj": {
+                        "component_name": "Generate",
+                        "params": {}
+                    },
+                    "downstream": ["answer_0"],
+                    "upstream": ["retrieval_0"],
                 }
             },
             "history": [],
-            "path": [],
+            "path": ["begin"],
-            "retrieval": [],
+            "retrieval": {"chunks": [], "doc_aggs": []},
             "globals": {
                 "sys.query": "",
-                "sys.user_id": "",
+                "sys.user_id": tenant_id,
                 "sys.conversation_turns": 0,
                 "sys.files": []
             }
         }
+        """
+
+    def __init__(self, dsl: str, tenant_id=None, task_id=None):
+        self.path = []
+        self.components = {}
+        self.error = ""
+        self.dsl = json.loads(dsl)
         self._tenant_id = tenant_id
         self.task_id = task_id if task_id else get_uuid()
         self.load()
@@ -116,8 +86,6 @@ class Canvas:
         for k, cpn in self.components.items():
             cpn_nms.add(cpn["obj"]["component_name"])
 
-        assert "Begin" in cpn_nms, "There have to be an 'Begin' component."
-
         for k, cpn in self.components.items():
             cpn_nms.add(cpn["obj"]["component_name"])
             param = component_class(cpn["obj"]["component_name"] + "Param")()
@@ -130,18 +98,10 @@ class Canvas:
             cpn["obj"] = component_class(cpn["obj"]["component_name"])(self, k, param)
 
         self.path = self.dsl["path"]
-        self.history = self.dsl["history"]
-        self.globals = self.dsl["globals"]
-        self.retrieval = self.dsl["retrieval"]
-        self.memory = self.dsl.get("memory", [])
 
     def __str__(self):
         self.dsl["path"] = self.path
-        self.dsl["history"] = self.history
-        self.dsl["globals"] = self.globals
         self.dsl["task_id"] = self.task_id
-        self.dsl["retrieval"] = self.retrieval
-        self.dsl["memory"] = self.memory
         dsl = {
             "components": {}
         }
@@ -160,14 +120,79 @@ class Canvas:
                 dsl["components"][k][c] = deepcopy(cpn[c])
         return json.dumps(dsl, ensure_ascii=False)
 
-    def reset(self, mem=False):
+    def reset(self):
         self.path = []
+        for k, cpn in self.components.items():
+            self.components[k]["obj"].reset()
+        try:
+            REDIS_CONN.delete(f"{self.task_id}-logs")
+        except Exception as e:
+            logging.exception(e)
+
+    def get_component_name(self, cid):
+        for n in self.dsl.get("graph", {}).get("nodes", []):
+            if cid == n["id"]:
+                return n["data"]["name"]
+        return ""
+
+    def run(self, **kwargs):
+        raise NotImplementedError()
+
+    def get_component(self, cpn_id) -> Union[None, dict[str, Any]]:
+        return self.components.get(cpn_id)
+
+    def get_component_obj(self, cpn_id) -> ComponentBase:
+        return self.components.get(cpn_id)["obj"]
+
+    def get_component_type(self, cpn_id) -> str:
+        return self.components.get(cpn_id)["obj"].component_name
+
+    def get_component_input_form(self, cpn_id) -> dict:
+        return self.components.get(cpn_id)["obj"].get_input_form()
+
+    def get_tenant_id(self):
+        return self._tenant_id
+
+
+class Canvas(Graph):
+
+    def __init__(self, dsl: str, tenant_id=None, task_id=None):
+        self.globals = {
+            "sys.query": "",
+            "sys.user_id": tenant_id,
+            "sys.conversation_turns": 0,
+            "sys.files": []
+        }
+        super().__init__(dsl, tenant_id, task_id)
+
+    def load(self):
+        super().load()
+        self.history = self.dsl["history"]
+        if "globals" in self.dsl:
+            self.globals = self.dsl["globals"]
+        else:
+            self.globals = {
+            "sys.query": "",
+            "sys.user_id": "",
+            "sys.conversation_turns": 0,
+            "sys.files": []
+        }
+            
+        self.retrieval = self.dsl["retrieval"]
+        self.memory = self.dsl.get("memory", [])
+
+    def __str__(self):
+        self.dsl["history"] = self.history
+        self.dsl["retrieval"] = self.retrieval
+        self.dsl["memory"] = self.memory
+        return super().__str__()
+
+    def reset(self, mem=False):
+        super().reset()
         if not mem:
             self.history = []
             self.retrieval = []
             self.memory = []
-        for k, cpn in self.components.items():
-            self.components[k]["obj"].reset()
 
         for k in self.globals.keys():
             if isinstance(self.globals[k], str):
@@ -183,22 +208,13 @@ class Canvas:
             else:
                 self.globals[k] = None
 
-        try:
-            REDIS_CONN.delete(f"{self.task_id}-logs")
-        except Exception as e:
-            logging.exception(e)
-
-    def get_component_name(self, cid):
-        for n in self.dsl.get("graph", {}).get("nodes", []):
-            if cid == n["id"]:
-                return n["data"]["name"]
-        return ""
-
     def run(self, **kwargs):
         st = time.perf_counter()
         self.message_id = get_uuid()
         created_at = int(time.time())
         self.add_user_input(kwargs.get("query"))
+        for k, cpn in self.components.items():
+            self.components[k]["obj"].reset(True)
 
         for k in kwargs.keys():
             if k in ["query", "user_id", "files"] and kwargs[k]:
@@ -285,9 +301,11 @@ class Canvas:
                                 yield decorate("message", {"content": m})
                                 _m += m
                         cpn_obj.set_output("content", _m)
+                        cite = re.search(r"\[ID:[ 0-9]+\]", _m)
                     else:
                         yield decorate("message", {"content": cpn_obj.output("content")})
-                    yield decorate("message_end", {"reference": self.get_reference()})
+                        cite = re.search(r"\[ID:[ 0-9]+\]",  cpn_obj.output("content"))
+                    yield decorate("message_end", {"reference": self.get_reference() if cite else None})
 
                     while partials:
                         _cpn_obj = self.get_component_obj(partials[0])
@@ -377,18 +395,6 @@ class Canvas:
                        })
             self.history.append(("assistant", self.get_component_obj(self.path[-1]).output()))
 
-    def get_component(self, cpn_id) -> Union[None, dict[str, Any]]:
-        return self.components.get(cpn_id)
-
-    def get_component_obj(self, cpn_id) -> ComponentBase:
-        return self.components.get(cpn_id)["obj"]
-
-    def get_component_type(self, cpn_id) -> str:
-        return self.components.get(cpn_id)["obj"].component_name
-
-    def get_component_input_form(self, cpn_id) -> dict:
-        return self.components.get(cpn_id)["obj"].get_input_form()
-
     def is_reff(self, exp: str) -> bool:
         exp = exp.strip("{").strip("}")
         if exp.find("@") < 0:
@@ -410,14 +416,11 @@ class Canvas:
             raise Exception(f"Can't find variable: '{cpn_id}@{var_nm}'")
         return cpn["obj"].output(var_nm)
 
-    def get_tenant_id(self):
-        return self._tenant_id
-
     def get_history(self, window_size):
         convs = []
         if window_size <= 0:
             return convs
-        for role, obj in self.history[window_size * -1:]:
+        for role, obj in self.history[window_size * -2:]:
             if isinstance(obj, dict):
                 convs.append({"role": role, "content": obj.get("content", "")})
             else:
@@ -427,39 +430,12 @@ class Canvas:
     def add_user_input(self, question):
         self.history.append(("user", question))
 
-    def _find_loop(self, max_loops=6):
-        path = self.path[-1][::-1]
-        if len(path) < 2:
-            return False
-
-        for i in range(len(path)):
-            if path[i].lower().find("answer") == 0 or path[i].lower().find("iterationitem") == 0:
-                path = path[:i]
-                break
-
-        if len(path) < 2:
-            return False
-
-        for loc in range(2, len(path) // 2):
-            pat = ",".join(path[0:loc])
-            path_str = ",".join(path)
-            if len(pat) >= len(path_str):
-                return False
-            loop = max_loops
-            while path_str.find(pat) == 0 and loop >= 0:
-                loop -= 1
-                if len(pat)+1 >= len(path_str):
-                    return False
-                path_str = path_str[len(pat)+1:]
-            if loop < 0:
-                pat = " => ".join([p.split(":")[0] for p in path[0:loc]])
-                return pat + " => " + pat
-
-        return False
-
     def get_prologue(self):
         return self.components["begin"]["obj"]._param.prologue
 
+    def get_mode(self):
+        return self.components["begin"]["obj"]._param.mode
+
     def set_global_param(self, **kwargs):
         self.globals.update(kwargs)
 
@@ -508,7 +484,7 @@ class Canvas:
         except Exception as e:
             logging.exception(e)
 
-    def add_refernce(self, chunks: list[object], doc_infos: list[object]):
+    def add_reference(self, chunks: list[object], doc_infos: list[object]):
         if not self.retrieval:
             self.retrieval = [{"chunks": {}, "doc_aggs": {}}]
 

agent/component/__init__.py

@@ -50,8 +50,9 @@ del _package_path, _import_submodules, _extract_classes_from_module
 
 
 def component_class(class_name):
-    m = importlib.import_module("agent.component")
+    for mdl in ["agent.component", "agent.tools", "rag.flow"]:
-    try:
+        try:
-        return getattr(m, class_name)
+            return getattr(importlib.import_module(mdl), class_name)
-    except Exception:
+        except Exception:
-        return getattr(importlib.import_module("agent.tools"), class_name)
+            pass
+    assert False, f"Can't import {class_name}"

agent/component/agent_with_tools.py

@@ -155,18 +155,18 @@ class Agent(LLM, ToolBase):
         if not self.tools:
             return LLM._invoke(self, **kwargs)
 
-        prompt, msg = self._prepare_prompt_variables()
+        prompt, msg, user_defined_prompt = self._prepare_prompt_variables()
 
         downstreams = self._canvas.get_component(self._id)["downstream"] if self._canvas.get_component(self._id) else []
         ex = self.exception_handler()
         if any([self._canvas.get_component_obj(cid).component_name.lower()=="message" for cid in downstreams]) and not self._param.output_structure and not (ex and ex["goto"]):
-            self.set_output("content", partial(self.stream_output_with_tools, prompt, msg))
+            self.set_output("content", partial(self.stream_output_with_tools, prompt, msg, user_defined_prompt))
             return
 
         _, msg = message_fit_in([{"role": "system", "content": prompt}, *msg], int(self.chat_mdl.max_length * 0.97))
         use_tools = []
         ans = ""
-        for delta_ans, tk in self._react_with_tools_streamly(prompt, msg, use_tools):
+        for delta_ans, tk in self._react_with_tools_streamly(prompt, msg, use_tools, user_defined_prompt):
             ans += delta_ans
 
         if ans.find("**ERROR**") >= 0:
@@ -182,11 +182,11 @@ class Agent(LLM, ToolBase):
             self.set_output("use_tools", use_tools)
         return ans
 
-    def stream_output_with_tools(self, prompt, msg):
+    def stream_output_with_tools(self, prompt, msg, user_defined_prompt={}):
         _, msg = message_fit_in([{"role": "system", "content": prompt}, *msg], int(self.chat_mdl.max_length * 0.97))
         answer_without_toolcall = ""
         use_tools = []
-        for delta_ans,_ in self._react_with_tools_streamly(prompt, msg, use_tools):
+        for delta_ans,_ in self._react_with_tools_streamly(prompt, msg, use_tools, user_defined_prompt):
             if delta_ans.find("**ERROR**") >= 0:
                 if self.get_exception_default_value():
                     self.set_output("content", self.get_exception_default_value())
@@ -209,7 +209,7 @@ class Agent(LLM, ToolBase):
                                                   ]):
             yield delta_ans
 
-    def _react_with_tools_streamly(self, prompt, history: list[dict], use_tools):
+    def _react_with_tools_streamly(self, prompt, history: list[dict], use_tools, user_defined_prompt={}):
         token_count = 0
         tool_metas = self.tool_meta
         hist = deepcopy(history)
@@ -230,7 +230,7 @@ class Agent(LLM, ToolBase):
             #    last_calling,
             #    last_calling != name
             #]):
-            #    self.toolcall_session.get_tool_obj(name).add2system_prompt(f"The chat history with other agents are as following: \n" + self.get_useful_memory(user_request, str(args["user_prompt"])))
+            #    self.toolcall_session.get_tool_obj(name).add2system_prompt(f"The chat history with other agents are as following: \n" + self.get_useful_memory(user_request, str(args["user_prompt"]),user_defined_prompt))
             last_calling = name
             tool_response = self.toolcall_session.tool_call(name, args)
             use_tools.append({
@@ -239,7 +239,7 @@ class Agent(LLM, ToolBase):
                 "results": tool_response
             })
             # self.callback("add_memory", {}, "...")
-            #self.add_memory(hist[-2]["content"], hist[-1]["content"], name, args, str(tool_response))
+            #self.add_memory(hist[-2]["content"], hist[-1]["content"], name, args, str(tool_response), user_defined_prompt)
 
             return name, tool_response
 
@@ -279,10 +279,10 @@ class Agent(LLM, ToolBase):
                 hist.append({"role": "user", "content": content})
 
         st = timer()
-        task_desc = analyze_task(self.chat_mdl, prompt, user_request, tool_metas)
+        task_desc = analyze_task(self.chat_mdl, prompt, user_request, tool_metas, user_defined_prompt)
         self.callback("analyze_task", {}, task_desc, elapsed_time=timer()-st)
         for _ in range(self._param.max_rounds + 1):
-            response, tk = next_step(self.chat_mdl, hist, tool_metas, task_desc)
+            response, tk = next_step(self.chat_mdl, hist, tool_metas, task_desc, user_defined_prompt)
             # self.callback("next_step", {}, str(response)[:256]+"...")
             token_count += tk
             hist.append({"role": "assistant", "content": response})
@@ -307,7 +307,7 @@ class Agent(LLM, ToolBase):
                         thr.append(executor.submit(use_tool, name, args))
 
                     st = timer()
-                    reflection = reflect(self.chat_mdl, hist, [th.result() for th in thr])
+                    reflection = reflect(self.chat_mdl, hist, [th.result() for th in thr], user_defined_prompt)
                     append_user_content(hist, reflection)
                     self.callback("reflection", {}, str(reflection), elapsed_time=timer()-st)
 
@@ -334,10 +334,10 @@ Respond immediately with your final comprehensive answer.
         for txt, tkcnt in complete():
             yield txt, tkcnt
 
-    def get_useful_memory(self, goal: str, sub_goal:str, topn=3) -> str:
+    def get_useful_memory(self, goal: str, sub_goal:str, topn=3, user_defined_prompt:dict={}) -> str:
         # self.callback("get_useful_memory", {"topn": 3}, "...")
         mems = self._canvas.get_memory()
-        rank = rank_memories(self.chat_mdl, goal, sub_goal, [summ for (user, assist, summ) in mems])
+        rank = rank_memories(self.chat_mdl, goal, sub_goal, [summ for (user, assist, summ) in mems], user_defined_prompt)
         try:
             rank = json_repair.loads(re.sub(r"```.*", "", rank))[:topn]
             mems = [mems[r] for r in rank]

agent/component/base.py

@@ -16,7 +16,7 @@
 
 import re
 import time
-from abc import ABC, abstractmethod
+from abc import ABC
 import builtins
 import json
 import os
@@ -36,7 +36,7 @@ _IS_RAW_CONF = "_is_raw_conf"
 
 class ComponentParamBase(ABC):
     def __init__(self):
-        self.message_history_window_size = 22
+        self.message_history_window_size = 13
         self.inputs = {}
         self.outputs = {}
         self.description = ""
@@ -410,8 +410,8 @@ class ComponentBase(ABC):
         )
 
     def __init__(self, canvas, id, param: ComponentParamBase):
-        from agent.canvas import Canvas  # Local import to avoid cyclic dependency
+        from agent.canvas import Graph  # Local import to avoid cyclic dependency
-        assert isinstance(canvas, Canvas), "canvas must be an instance of Canvas"
+        assert isinstance(canvas, Graph), "canvas must be an instance of Canvas"
         self._canvas = canvas
         self._id = id
         self._param = param
@@ -448,9 +448,11 @@ class ComponentBase(ABC):
     def error(self):
         return self._param.outputs.get("_ERROR", {}).get("value")
 
-    def reset(self):
+    def reset(self, only_output=False):
         for k in self._param.outputs.keys():
             self._param.outputs[k]["value"] = None
+        if only_output:
+            return
         for k in self._param.inputs.keys():
             self._param.inputs[k]["value"] = None
         self._param.debug_inputs = {}
@@ -526,6 +528,10 @@ class ComponentBase(ABC):
         cpn_nms = self._canvas.get_component(self._id)['upstream']
         return cpn_nms
 
+    def get_downstream(self) -> List[str]:
+        cpn_nms = self._canvas.get_component(self._id)['downstream']
+        return cpn_nms
+
     @staticmethod
     def string_format(content: str, kv: dict[str, str]) -> str:
         for n, v in kv.items():
@@ -554,6 +560,5 @@ class ComponentBase(ABC):
     def set_exception_default_value(self):
         self.set_output("result", self.get_exception_default_value())
 
-    @abstractmethod
     def thoughts(self) -> str:
-        ...
+        raise NotImplementedError()

agent/component/llm.py

@@ -17,12 +17,10 @@ import json
 import logging
 import os
 import re
-from typing import Any, Generator
-
-import json_repair
 from copy import deepcopy
+from typing import Any, Generator
+import json_repair
 from functools import partial
-
 from api.db import LLMType
 from api.db.services.llm_service import LLMBundle
 from api.db.services.tenant_llm_service import TenantLLMService
@@ -130,7 +128,7 @@ class LLM(ComponentBase):
 
         args = {}
         vars = self.get_input_elements() if not self._param.debug_inputs else self._param.debug_inputs
-        prompt = self._param.sys_prompt
+        sys_prompt = self._param.sys_prompt
         for k, o in vars.items():
             args[k] = o["value"]
             if not isinstance(args[k], str):
@@ -141,14 +139,29 @@ class LLM(ComponentBase):
             self.set_input_value(k, args[k])
 
         msg = self._canvas.get_history(self._param.message_history_window_size)[:-1]
-        msg.extend(deepcopy(self._param.prompts))
+        for p in self._param.prompts:
-        prompt = self.string_format(prompt, args)
+            if msg and msg[-1]["role"] == p["role"]:
+                continue
+            msg.append(deepcopy(p))
+
+        sys_prompt = self.string_format(sys_prompt, args)
+        user_defined_prompt, sys_prompt = self._extract_prompts(sys_prompt)
         for m in msg:
             m["content"] = self.string_format(m["content"], args)
         if self._param.cite and self._canvas.get_reference()["chunks"]:
-            prompt += citation_prompt()
+            sys_prompt += citation_prompt(user_defined_prompt)
 
-        return prompt, msg
+        return sys_prompt, msg, user_defined_prompt
+
+    def _extract_prompts(self, sys_prompt):
+        pts = {}
+        for tag in ["TASK_ANALYSIS", "PLAN_GENERATION", "REFLECTION", "CONTEXT_SUMMARY", "CONTEXT_RANKING", "CITATION_GUIDELINES"]:
+            r = re.search(rf"<{tag}>(.*?)</{tag}>", sys_prompt, flags=re.DOTALL|re.IGNORECASE)
+            if not r:
+                continue
+            pts[tag.lower()] = r.group(1)
+            sys_prompt = re.sub(rf"<{tag}>(.*?)</{tag}>", "", sys_prompt, flags=re.DOTALL|re.IGNORECASE)
+        return pts, sys_prompt
 
     def _generate(self, msg:list[dict], **kwargs) -> str:
         if not self.imgs:
@@ -196,7 +209,7 @@ class LLM(ComponentBase):
             ans = re.sub(r"^.*```json", "", ans, flags=re.DOTALL)
             return re.sub(r"```\n*$", "", ans, flags=re.DOTALL)
 
-        prompt, msg = self._prepare_prompt_variables()
+        prompt, msg, _ = self._prepare_prompt_variables()
         error = ""
 
         if self._param.output_structure:
@@ -260,11 +273,11 @@ class LLM(ComponentBase):
             answer += ans
         self.set_output("content", answer)
 
-    def add_memory(self, user:str, assist:str, func_name: str, params: dict, results: str):
+    def add_memory(self, user:str, assist:str, func_name: str, params: dict, results: str, user_defined_prompt:dict={}):
-        summ = tool_call_summary(self.chat_mdl, func_name, params, results)
+        summ = tool_call_summary(self.chat_mdl, func_name, params, results, user_defined_prompt)
         logging.info(f"[MEMORY]: {summ}")
         self._canvas.add_memory(user, assist, summ)
 
     def thoughts(self) -> str:
-        _, msg = self._prepare_prompt_variables()
+        _, msg,_ = self._prepare_prompt_variables()
         return "⌛Give me a moment—starting from: \n\n" + re.sub(r"(User's query:|[\\]+)", '', msg[-1]['content'], flags=re.DOTALL) + "\n\nI’ll figure out our best next move."

agent/templates/choose_your_knowledge_base_agent.json

@@ -1,8 +1,12 @@
 {
     "id": 19,
-    "title": "Choose Your Knowledge Base Agent",
+    "title": {
-    "description": "Select your desired knowledge base from the dropdown menu. The Agent will only retrieve from the selected knowledge base and use this content  to generate responses.",
+		"en": "Choose Your Knowledge Base Agent",
-    "canvas_type": "Agent", 
+		"zh": "选择知识库智能体"},
+    "description": {
+		"en": "Select your desired knowledge base from the dropdown menu. The Agent will only retrieve from the selected knowledge base and use this content  to generate responses.",
+		"zh": "从下拉菜单中选择知识库，智能体将仅根据所选知识库内容生成回答。"},
+	"canvas_type": "Agent", 
     "dsl": {
 		"components": {
 			"Agent:BraveParksJoke": {

agent/templates/choose_your_knowledge_base_workflow.json

@@ -1,8 +1,12 @@
 {
     "id": 18,
-    "title": "Choose Your Knowledge Base Workflow",
+    "title": {
-    "description": "Select your desired knowledge base from the dropdown menu. The retrieval assistant will only use data from your selected knowledge base to generate responses.",
+		"en": "Choose Your Knowledge Base Workflow",
-    "canvas_type": "Other",
+		"zh": "选择知识库工作流"},
+    "description": {
+		"en": "Select your desired knowledge base from the dropdown menu. The retrieval assistant will only use data from your selected knowledge base to generate responses.",
+		"zh": "从下拉菜单中选择知识库，工作流将仅根据所选知识库内容生成回答。"},
+	"canvas_type": "Other",
     "dsl": {
 		"components": {
 			"Agent:ProudDingosShout": {

agent/templates/customer_review_analysis.json

@@ -1,9 +1,13 @@
 
 {
     "id": 11,
-    "title": "Customer Review Analysis",
+    "title": {
-    "description": "Automatically classify customer reviews using LLM (Large Language Model) and route them via email to the relevant departments.",
+		"en": "Customer Review Analysis",
-    "canvas_type": "Customer Support",
+		"zh": "客户评价分析"},
+    "description": {
+		"en": "Automatically classify customer reviews using LLM (Large Language Model) and route them via email to the relevant departments.",
+		"zh": "大模型将自动分类客户评价，并通过电子邮件将结果发送到相关部门。"},
+	"canvas_type": "Customer Support",
     "dsl": {
 		"components": {
 			"Categorize:FourTeamsFold": {

agent/templates/customer_support.json

@@ -1,8 +1,12 @@
 
 {
     "id": 10,
-    "title": "Customer Support",
+    "title": {
-    "description": "This is an intelligent customer service processing system workflow based on user intent classification. It uses LLM to identify user demand types and transfers them to the corresponding professional agent for processing.",
+        "en":"Customer Support",
+        "zh": "客户支持"},
+    "description": {
+        "en": "This is an intelligent customer service processing system workflow based on user intent classification. It uses LLM to identify user demand types and transfers them to the corresponding professional agent for processing.",
+        "zh": "工作流系统，用于智能客服场景。基于用户意图分类。使用大模型识别用户需求类型，并将需求转移给相应的智能体进行处理。"},
     "canvas_type": "Customer Support",
     "dsl": {
             "components": {

agent/templates/cv_analysis_and_candidate_evaluation.json

@@ -1,8 +1,12 @@
 
 {
     "id": 15,
-    "title": "CV Analysis and Candidate Evaluation",
+    "title": {
-    "description": "This is a workflow that helps companies evaluate resumes, HR uploads a job description first, then submits multiple resumes via the chat window for evaluation.",
+        "en": "CV Analysis and Candidate Evaluation",
+        "zh": "简历分析和候选人评估"},
+    "description": {
+        "en": "This is a workflow that helps companies evaluate resumes, HR uploads a job description first, then submits multiple resumes via the chat window for evaluation.",
+        "zh": "帮助公司评估简历的工作流。HR首先上传职位描述，通过聊天窗口提交多份简历进行评估。"},
     "canvas_type": "Other",
     "dsl": {
             "components": {

agent/templates/deep_research.json

@@ -1,8 +1,12 @@
       
 {
     "id": 1,
-    "title": "Deep Research",
+    "title": {
-    "description": "For professionals in sales, marketing, policy, or consulting, the Multi-Agent Deep Research Agent conducts structured, multi-step investigations across diverse sources and delivers consulting-style reports with clear citations.",
+        "en": "Deep Research",
+        "zh": "深度研究"},
+    "description": {
+        "en": "For professionals in sales, marketing, policy, or consulting, the Multi-Agent Deep Research Agent conducts structured, multi-step investigations across diverse sources and delivers consulting-style reports with clear citations.",
+        "zh": "专为销售、市场、政策或咨询领域的专业人士设计，多智能体的深度研究会结合多源信息进行结构化、多步骤地回答问题，并附带有清晰的引用。"},
     "canvas_type": "Recommended",
     "dsl": {
             "components": {

agent/templates/deep_search_r.json

@@ -1,8 +1,12 @@
 
 {
     "id": 6,
-    "title": "Deep Research",
+    "title": {
-    "description": "For professionals in sales, marketing, policy, or consulting, the Multi-Agent Deep Research Agent conducts structured, multi-step investigations across diverse sources and delivers consulting-style reports with clear citations.",
+        "en": "Deep Research",
+        "zh": "深度研究"},
+    "description": {
+        "en": "For professionals in sales, marketing, policy, or consulting, the Multi-Agent Deep Research Agent conducts structured, multi-step investigations across diverse sources and delivers consulting-style reports with clear citations.",
+        "zh": "专为销售、市场、政策或咨询领域的专业人士设计，多智能体的深度研究会结合多源信息进行结构化、多步骤地回答问题，并附带有清晰的引用。"},
     "canvas_type": "Agent",
     "dsl": {
             "components": {

agent/templates/generate_SEO_blog.json

@@ -1,7 +1,11 @@
 {
     "id": 8,
-    "title": "Generate SEO Blog",
+    "title": {
-    "description": "This is a multi-agent version of the SEO blog generation workflow. It simulates a small team of AI “writers”, where each agent plays a specialized role — just like a real editorial team.",
+        "en": "Generate SEO Blog",
+        "zh": "生成SEO博客"},
+    "description": {
+        "en": "This is a multi-agent version of the SEO blog generation workflow. It simulates a small team of AI “writers”, where each agent plays a specialized role — just like a real editorial team.",
+        "zh": "多智能体架构可根据简单的用户输入自动生成完整的SEO博客文章。模拟小型“作家”团队，其中每个智能体扮演一个专业角色——就像真正的编辑团队。"},
     "canvas_type": "Agent",
     "dsl": {
             "components": {

agent/templates/image_lingo.json

@@ -1,7 +1,11 @@
 {
     "id": 13,
-    "title": "ImageLingo",
+    "title": {
-    "description": "ImageLingo lets you snap any photo containing text—menus, signs, or documents—and instantly recognize and translate it into your language of choice using advanced AI-powered translation technology.",
+        "en": "ImageLingo",
+        "zh": "图片解析"},
+    "description": {
+        "en": "ImageLingo lets you snap any photo containing text—menus, signs, or documents—and instantly recognize and translate it into your language of choice using advanced AI-powered translation technology.",
+        "zh": "多模态大模型允许您拍摄任何包含文本的照片——菜单、标志或文档——立即识别并转换成您选择的语言。"},
     "canvas_type": "Consumer App",
     "dsl": {
             "components": {

agent/templates/knowledge_base_report.json

@@ -1,7 +1,11 @@
 {
     "id": 20,
-    "title": "Report Agent Using Knowledge Base",
+    "title": {
-    "description": "A report generation assistant using local knowledge base, with advanced capabilities in task planning, reasoning, and reflective analysis. Recommended for academic research paper Q&A",
+        "en": "Report Agent Using Knowledge Base",
+        "zh": "知识库检索智能体"},
+    "description": {
+        "en": "A report generation assistant using local knowledge base, with advanced capabilities in task planning, reasoning, and reflective analysis. Recommended for academic research paper Q&A",
+        "zh": "一个使用本地知识库的报告生成助手，具备高级能力，包括任务规划、推理和反思性分析。推荐用于学术研究论文问答。"},
     "canvas_type": "Agent",
     "dsl": {
         "components": {

agent/templates/knowledge_base_report_r.json

@@ -0,0 +1,331 @@
+{
+    "id": 21,
+    "title": {
+        "en": "Report Agent Using Knowledge Base", 
+        "zh": "知识库检索智能体"},
+    "description": {
+        "en": "A report generation assistant using local knowledge base, with advanced capabilities in task planning, reasoning, and reflective analysis. Recommended for academic research paper Q&A",
+        "zh": "一个使用本地知识库的报告生成助手，具备高级能力，包括任务规划、推理和反思性分析。推荐用于学术研究论文问答。"},
+    "canvas_type": "Recommended",
+    "dsl": {
+        "components": {
+            "Agent:NewPumasLick": {
+                "downstream": [
+                    "Message:OrangeYearsShine"
+                ],
+                "obj": {
+                    "component_name": "Agent",
+                    "params": {
+                        "delay_after_error": 1,
+                        "description": "",
+                        "exception_comment": "",
+                        "exception_default_value": "",
+                        "exception_goto": [],
+                        "exception_method": null,
+                        "frequencyPenaltyEnabled": false,
+                        "frequency_penalty": 0.5,
+                        "llm_id": "qwen3-235b-a22b-instruct-2507@Tongyi-Qianwen",
+                        "maxTokensEnabled": true,
+                        "max_retries": 3,
+                        "max_rounds": 3,
+                        "max_tokens": 128000,
+                        "mcp": [],
+                        "message_history_window_size": 12,
+                        "outputs": {
+                            "content": {
+                                "type": "string",
+                                "value": ""
+                            }
+                        },
+                        "parameter": "Precise",
+                        "presencePenaltyEnabled": false,
+                        "presence_penalty": 0.5,
+                        "prompts": [
+                            {
+                                "content": "# User Query\n {sys.query}",
+                                "role": "user"
+                            }
+                        ],
+                        "sys_prompt": "## Role & Task\nYou are a **\u201cKnowledge Base Retrieval Q\\&A Agent\u201d** whose goal is to break down the user\u2019s question into retrievable subtasks, and then produce a multi-source-verified, structured, and actionable research report using the internal knowledge base.\n## Execution Framework (Detailed Steps & Key Points)\n1. **Assessment & Decomposition**\n   * Actions:\n     * Automatically extract: main topic, subtopics, entities (people/organizations/products/technologies), time window, geographic/business scope.\n     * Output as a list: N facts/data points that must be collected (*N* ranges from 5\u201320 depending on question complexity).\n2. **Query Type Determination (Rule-Based)**\n   * Example rules:\n     * If the question involves a single issue but requests \u201cmethod comparison/multiple explanations\u201d \u2192 use **depth-first**.\n     * If the question can naturally be split into \u22653 independent sub-questions \u2192 use **breadth-first**.\n     * If the question can be answered by a single fact/specification/definition \u2192 use **simple query**.\n3. **Research Plan Formulation**\n   * Depth-first: define 3\u20135 perspectives (methodology/stakeholders/time dimension/technical route, etc.), assign search keywords, target document types, and output format for each perspective.\n   * Breadth-first: list subtasks, prioritize them, and assign search terms.\n   * Simple query: directly provide the search sentence and required fields.\n4. **Retrieval Execution**\n   * After retrieval: perform coverage check (does it contain the key facts?) and quality check (source diversity, authority, latest update time).\n   * If standards are not met, automatically loop: rewrite queries (synonyms/cross-domain terms) and retry \u22643 times, or flag as requiring external search.\n5. **Integration & Reasoning**\n   * Build the answer using a **fact\u2013evidence\u2013reasoning** chain. For each conclusion, attach 1\u20132 strongest pieces of evidence.\n---\n## Quality Gate Checklist (Verify at Each Stage)\n* **Stage 1 (Decomposition)**:\n  * [ ] Key concepts and expected outputs identified\n  * [ ] Required facts/data points listed\n* **Stage 2 (Retrieval)**:\n  * [ ] Meets quality standards (see above)\n  * [ ] If not met: execute query iteration\n* **Stage 3 (Generation)**:\n  * [ ] Each conclusion has at least one direct evidence source\n  * [ ] State assumptions/uncertainties\n  * [ ] Provide next-step suggestions or experiment/retrieval plans\n  * [ ] Final length and depth match user expectations (comply with word count/format if specified)\n---\n## Core Principles\n1. **Strict reliance on the knowledge base**: answers must be **fully bounded** by the content retrieved from the knowledge base.\n2. **No fabrication**: do not generate, infer, or create information that is not explicitly present in the knowledge base.\n3. **Accuracy first**: prefer incompleteness over inaccurate content.\n4. **Output format**:\n   * Hierarchically clear modular structure\n   * Logical grouping according to the MECE principle\n   * Professionally presented formatting\n   * Step-by-step cognitive guidance\n   * Reasonable use of headings and dividers for clarity\n   * *Italicize* key parameters\n   * **Bold** critical information\n5. **LaTeX formula requirements**:\n   * Inline formulas: start and end with `$`\n   * Block formulas: start and end with `$$`, each `$$` on its own line\n   * Block formula content must comply with LaTeX math syntax\n   * Verify formula correctness\n---\n## Additional Notes (Interaction & Failure Strategy)\n* If the knowledge base does not cover critical facts: explicitly inform the user (with sample wording)\n* For time-sensitive issues: enforce time filtering in the search request, and indicate the latest retrieval date in the answer.\n* Language requirement: answer in the user\u2019s preferred language\n",
+                        "temperature": "0.1",
+                        "temperatureEnabled": true,
+                        "tools": [
+                            {
+                                "component_name": "Retrieval",
+                                "name": "Retrieval",
+                                "params": {
+                                    "cross_languages": [],
+                                    "description": "",
+                                    "empty_response": "",
+                                    "kb_ids": [],
+                                    "keywords_similarity_weight": 0.7,
+                                    "outputs": {
+                                        "formalized_content": {
+                                            "type": "string",
+                                            "value": ""
+                                        }
+                                    },
+                                    "rerank_id": "",
+                                    "similarity_threshold": 0.2,
+                                    "top_k": 1024,
+                                    "top_n": 8,
+                                    "use_kg": false
+                                }
+                            }
+                        ],
+                        "topPEnabled": false,
+                        "top_p": 0.75,
+                        "user_prompt": "",
+                        "visual_files_var": ""
+                    }
+                },
+                "upstream": [
+                    "begin"
+                ]
+            },
+            "Message:OrangeYearsShine": {
+                "downstream": [],
+                "obj": {
+                    "component_name": "Message",
+                    "params": {
+                        "content": [
+                            "{Agent:NewPumasLick@content}"
+                        ]
+                    }
+                },
+                "upstream": [
+                    "Agent:NewPumasLick"
+                ]
+            },
+            "begin": {
+                "downstream": [
+                    "Agent:NewPumasLick"
+                ],
+                "obj": {
+                    "component_name": "Begin",
+                    "params": {
+                        "enablePrologue": true,
+                        "inputs": {},
+                        "mode": "conversational",
+                        "prologue": "\u4f60\u597d\uff01 \u6211\u662f\u4f60\u7684\u52a9\u7406\uff0c\u6709\u4ec0\u4e48\u53ef\u4ee5\u5e2e\u5230\u4f60\u7684\u5417\uff1f"
+                    }
+                },
+                "upstream": []
+            }
+        },
+        "globals": {
+            "sys.conversation_turns": 0,
+            "sys.files": [],
+            "sys.query": "",
+            "sys.user_id": ""
+        },
+        "graph": {
+            "edges": [
+                {
+                    "data": {
+                        "isHovered": false
+                    },
+                    "id": "xy-edge__beginstart-Agent:NewPumasLickend",
+                    "source": "begin",
+                    "sourceHandle": "start",
+                    "target": "Agent:NewPumasLick",
+                    "targetHandle": "end"
+                },
+                {
+                    "data": {
+                        "isHovered": false
+                    },
+                    "id": "xy-edge__Agent:NewPumasLickstart-Message:OrangeYearsShineend",
+                    "markerEnd": "logo",
+                    "source": "Agent:NewPumasLick",
+                    "sourceHandle": "start",
+                    "style": {
+                        "stroke": "rgba(91, 93, 106, 1)",
+                        "strokeWidth": 1
+                    },
+                    "target": "Message:OrangeYearsShine",
+                    "targetHandle": "end",
+                    "type": "buttonEdge",
+                    "zIndex": 1001
+                },
+                {
+                    "data": {
+                        "isHovered": false
+                    },
+                    "id": "xy-edge__Agent:NewPumasLicktool-Tool:AllBirdsNailend",
+                    "selected": false,
+                    "source": "Agent:NewPumasLick",
+                    "sourceHandle": "tool",
+                    "target": "Tool:AllBirdsNail",
+                    "targetHandle": "end"
+                }
+            ],
+            "nodes": [
+                {
+                    "data": {
+                        "form": {
+                            "enablePrologue": true,
+                            "inputs": {},
+                            "mode": "conversational",
+                            "prologue": "\u4f60\u597d\uff01 \u6211\u662f\u4f60\u7684\u52a9\u7406\uff0c\u6709\u4ec0\u4e48\u53ef\u4ee5\u5e2e\u5230\u4f60\u7684\u5417\uff1f"
+                        },
+                        "label": "Begin",
+                        "name": "begin"
+                    },
+                    "dragging": false,
+                    "id": "begin",
+                    "measured": {
+                        "height": 48,
+                        "width": 200
+                    },
+                    "position": {
+                        "x": -9.569875358221438,
+                        "y": 205.84018385864917
+                    },
+                    "selected": false,
+                    "sourcePosition": "left",
+                    "targetPosition": "right",
+                    "type": "beginNode"
+                },
+                {
+                    "data": {
+                        "form": {
+                            "content": [
+                                "{Agent:NewPumasLick@content}"
+                            ]
+                        },
+                        "label": "Message",
+                        "name": "Response"
+                    },
+                    "dragging": false,
+                    "id": "Message:OrangeYearsShine",
+                    "measured": {
+                        "height": 56,
+                        "width": 200
+                    },
+                    "position": {
+                        "x": 734.4061285881053,
+                        "y": 199.9706031723009
+                    },
+                    "selected": false,
+                    "sourcePosition": "right",
+                    "targetPosition": "left",
+                    "type": "messageNode"
+                },
+                {
+                    "data": {
+                        "form": {
+                            "delay_after_error": 1,
+                            "description": "",
+                            "exception_comment": "",
+                            "exception_default_value": "",
+                            "exception_goto": [],
+                            "exception_method": null,
+                            "frequencyPenaltyEnabled": false,
+                            "frequency_penalty": 0.5,
+                            "llm_id": "qwen3-235b-a22b-instruct-2507@Tongyi-Qianwen",
+                            "maxTokensEnabled": true,
+                            "max_retries": 3,
+                            "max_rounds": 3,
+                            "max_tokens": 128000,
+                            "mcp": [],
+                            "message_history_window_size": 12,
+                            "outputs": {
+                                "content": {
+                                    "type": "string",
+                                    "value": ""
+                                }
+                            },
+                            "parameter": "Precise",
+                            "presencePenaltyEnabled": false,
+                            "presence_penalty": 0.5,
+                            "prompts": [
+                                {
+                                    "content": "# User Query\n {sys.query}",
+                                    "role": "user"
+                                }
+                            ],
+                            "sys_prompt": "## Role & Task\nYou are a **\u201cKnowledge Base Retrieval Q\\&A Agent\u201d** whose goal is to break down the user\u2019s question into retrievable subtasks, and then produce a multi-source-verified, structured, and actionable research report using the internal knowledge base.\n## Execution Framework (Detailed Steps & Key Points)\n1. **Assessment & Decomposition**\n   * Actions:\n     * Automatically extract: main topic, subtopics, entities (people/organizations/products/technologies), time window, geographic/business scope.\n     * Output as a list: N facts/data points that must be collected (*N* ranges from 5\u201320 depending on question complexity).\n2. **Query Type Determination (Rule-Based)**\n   * Example rules:\n     * If the question involves a single issue but requests \u201cmethod comparison/multiple explanations\u201d \u2192 use **depth-first**.\n     * If the question can naturally be split into \u22653 independent sub-questions \u2192 use **breadth-first**.\n     * If the question can be answered by a single fact/specification/definition \u2192 use **simple query**.\n3. **Research Plan Formulation**\n   * Depth-first: define 3\u20135 perspectives (methodology/stakeholders/time dimension/technical route, etc.), assign search keywords, target document types, and output format for each perspective.\n   * Breadth-first: list subtasks, prioritize them, and assign search terms.\n   * Simple query: directly provide the search sentence and required fields.\n4. **Retrieval Execution**\n   * After retrieval: perform coverage check (does it contain the key facts?) and quality check (source diversity, authority, latest update time).\n   * If standards are not met, automatically loop: rewrite queries (synonyms/cross-domain terms) and retry \u22643 times, or flag as requiring external search.\n5. **Integration & Reasoning**\n   * Build the answer using a **fact\u2013evidence\u2013reasoning** chain. For each conclusion, attach 1\u20132 strongest pieces of evidence.\n---\n## Quality Gate Checklist (Verify at Each Stage)\n* **Stage 1 (Decomposition)**:\n  * [ ] Key concepts and expected outputs identified\n  * [ ] Required facts/data points listed\n* **Stage 2 (Retrieval)**:\n  * [ ] Meets quality standards (see above)\n  * [ ] If not met: execute query iteration\n* **Stage 3 (Generation)**:\n  * [ ] Each conclusion has at least one direct evidence source\n  * [ ] State assumptions/uncertainties\n  * [ ] Provide next-step suggestions or experiment/retrieval plans\n  * [ ] Final length and depth match user expectations (comply with word count/format if specified)\n---\n## Core Principles\n1. **Strict reliance on the knowledge base**: answers must be **fully bounded** by the content retrieved from the knowledge base.\n2. **No fabrication**: do not generate, infer, or create information that is not explicitly present in the knowledge base.\n3. **Accuracy first**: prefer incompleteness over inaccurate content.\n4. **Output format**:\n   * Hierarchically clear modular structure\n   * Logical grouping according to the MECE principle\n   * Professionally presented formatting\n   * Step-by-step cognitive guidance\n   * Reasonable use of headings and dividers for clarity\n   * *Italicize* key parameters\n   * **Bold** critical information\n5. **LaTeX formula requirements**:\n   * Inline formulas: start and end with `$`\n   * Block formulas: start and end with `$$`, each `$$` on its own line\n   * Block formula content must comply with LaTeX math syntax\n   * Verify formula correctness\n---\n## Additional Notes (Interaction & Failure Strategy)\n* If the knowledge base does not cover critical facts: explicitly inform the user (with sample wording)\n* For time-sensitive issues: enforce time filtering in the search request, and indicate the latest retrieval date in the answer.\n* Language requirement: answer in the user\u2019s preferred language\n",
+                            "temperature": "0.1",
+                            "temperatureEnabled": true,
+                            "tools": [
+                                {
+                                    "component_name": "Retrieval",
+                                    "name": "Retrieval",
+                                    "params": {
+                                        "cross_languages": [],
+                                        "description": "",
+                                        "empty_response": "",
+                                        "kb_ids": [],
+                                        "keywords_similarity_weight": 0.7,
+                                        "outputs": {
+                                            "formalized_content": {
+                                                "type": "string",
+                                                "value": ""
+                                            }
+                                        },
+                                        "rerank_id": "",
+                                        "similarity_threshold": 0.2,
+                                        "top_k": 1024,
+                                        "top_n": 8,
+                                        "use_kg": false
+                                    }
+                                }
+                            ],
+                            "topPEnabled": false,
+                            "top_p": 0.75,
+                            "user_prompt": "",
+                            "visual_files_var": ""
+                        },
+                        "label": "Agent",
+                        "name": "Knowledge Base Agent"
+                    },
+                    "dragging": false,
+                    "id": "Agent:NewPumasLick",
+                    "measured": {
+                        "height": 84,
+                        "width": 200
+                    },
+                    "position": {
+                        "x": 347.00048227952215,
+                        "y": 186.49109364794631
+                    },
+                    "selected": false,
+                    "sourcePosition": "right",
+                    "targetPosition": "left",
+                    "type": "agentNode"
+                },
+                {
+                    "data": {
+                        "form": {
+                            "description": "This is an agent for a specific task.",
+                            "user_prompt": "This is the order you need to send to the agent."
+                        },
+                        "label": "Tool",
+                        "name": "flow.tool_10"
+                    },
+                    "dragging": false,
+                    "id": "Tool:AllBirdsNail",
+                    "measured": {
+                        "height": 48,
+                        "width": 200
+                    },
+                    "position": {
+                        "x": 220.24819746977118,
+                        "y": 403.31576836482583
+                    },
+                    "selected": false,
+                    "sourcePosition": "right",
+                    "targetPosition": "left",
+                    "type": "toolNode"
+                }
+            ]
+        },
+        "history": [],
+        "memory": [],
+        "messages": [],
+        "path": [],
+        "retrieval": []
+    },
+    "avatar": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAAH0klEQVR4nO2ZC1BU1wGG/3uRp/IygG+DGK0GOjE1U6cxI4tT03Y0E+kENbaJbKpj60wzgNMwnTjuEtu0miGasY+0krI202kMVEnVxtoOLG00oVa0LajVBDcSEI0REFBgkZv/3GWXfdzdvctuHs7kmzmec9//d+45914XCXc4Xwjk1+59VJGGF7C5QAFSWBvgyWmWLl7IKiny6QNL173B5YjB84bOyrpKA4B1DLySdQpLKAiZGtZ7a/KMVoQJz6UfEZyhTWwaEBmssiLvCueu6BJg8EwFqGTTAC+uvNWC9w82sRWcux/JwaSHstjywcogRt4RG0KExwWG4QsVYCebKSwe3L5lR9OOWjyzfg2WL/0a1/jncO3b2FHxGnKeWYqo+Giu8UEMrWJKWBACPMY/DG+63txhvnKshUu+DF2/hayMDFRsL+VScDb++AVc6OjAuInxXPJl2tfnIikrzUyJMi7qQmLRhOEr2fOFbX/7P6STF7BqoWevfdij4NWGQfx+57OYO2sG1wSnsek8Nm15EU8sikF6ouelXz9ph7JwDqYt+5IIZaGEkauDIrH4wPBmhjexCSEws+VdVG1M4NIoj+2xYzBuJtavWcEl/VS8dggx/ZdQvcGzQwp+cxOXsu5RBQQMVkYJM4LA/Txh+ELFMWFVPARS5kFiabZdx8Olh7l17BzdvhzZmROhdJ3j6D/nIyBgOCMlLAgA9xmF4TMV4BSbrgnrLiBl5rOsRCRRbDUsBzQFiJjY91PCBj9w+yiP1lXWsTLAjc9YQGB9I8+Yx1oTiUWFvW9QgDo2PdASaDp/EQ8/sRnhcPTVcuTMncXwQQVESL9DidscaPW+QEtAICRu9PSxFTpJiePV8AI9AsTvXZBY/Pa+wJ9ApNApIILm8S5Y4QXXQwhYFH6csemDP4G3G5v579i5d04mknknQhDYS4HCrCVr/mC3D305KnbCEpvVIia5Onw6WaWw+KAl0Np+FUXbdiMcyoqfUoeRHoFrJ1uRtnBG1/9Mf/3LtElp+VwF2wcd7woJib1vUPwMH4GWQCQJJtBa/V9cPmFD8uQUpMdNGDhY8bNYrobh8acHu270/l0ImJWRt64Wn6WACN9z5gq2lXwPW8pfweT0icP/fH23vO9QLYq3/QKyLBmFQI3CUcT9NdESEEPItKsSN3r7MBaSJoxHWZERM6ZmMLy2gDP8/pd/og418dTL37hFSUpMUC5f+UiWZcnY9s5+ixCwUiCXx2iiJdDNx6f4pgkH8Q3lbxK7h8+enoHha1cRNdMp8axiHxo6+/5bVdk8DSROYIW1X7QEIom3wHD3gEf4vu1bVYEJZeWQ0zJQvmcfyiv2QZak6raG/QWfK4Ez9mTc5v8xPMJfuojoxXmIX/9DOMe+FCWbcHu4BJJ0YEwCx0824bFNW9HesB+CqYu+jepfPYcHF+aoPXS8sQl/+vU2bgmOU2C+qRc9/YrrPPbGBtzavd0nvCxLxui4pJrBm911PFwak4CYA80cj+JCAiGUzYkmxrSY4N2c3GLi6UEIFL/wRxxqkhmHnTEpDQcrfq6ea+hcE8bNy3GFzyq4H22HW1Kd4WMSkg1jmsSRpKj0Rzhy4gNUv/y8Gjrv8SJK3OWScA+fMn/ysVPPvTmeh6nh1TcxBUJ+jEaKYr7N36x7h+Edj0pB6+WrLokn87+BrTt/p4ZPzZ6MM7/8R2//h33vOcNzdwgBMwVMbGvySQmo4a0NqOZccU7YmGXLEfPQUlUid/XT6B8YdIU/99vjsPcOdEhDsfOd4QVCwKB8yp8SWuG1njbTl83DpMWz1PCKAswuWPDI0e8WebyAJBbxNdrF7cls+hBpAb3h3XtehL/3+4u7D35rQwpP4YFTwMJ91rHpQyQFQgmf9sAMNL9Ur4afv/FBjIuPVj+n4YVTwMD96tj0IVICoYYXv/q1VJ1Sl8UveQyaRwErvOB6B5SwKhqP00gI6A0vhsycJ7/KIzxhyHqGN0ADbnNAAYOicRfCFdAb/p50Gbfuc/wy5w1D5lOghk0fuG0USlgVr7sQjoDe8C8WxKGKPy2KjzlvAQb02/sCbh+FApngX1QUtyeSuwDi0hxFByV7L+LIf3r5kvpp4PBr07Hqvn71Y85bgOG6WS2ggA1+4D6eUKKQApVsqngI6KSkqh9HzsoM/3zg8Oz5VQ9E8wjf30YFDGdkeAsCwH18oYRZGXk7C4HuYxcwe6rjQsFovzaEvoFxqNkTOPzMjGikJso8wsF77XYkLx6dAwxWxvBmBIH7aUMJi8J3w0DnTVz7dyvX6KPzVBt+kL8cmzesRq9ps2Z48bRJmOIapS7E4zM2lXNt5CcU6ID7+ocSZkqY2NRN6ysnsHbJEpR8ZwV6t5Yg+iuLELf2KVd48VwXQf3BQGUMb4ZOuH9gKFEIYJfiNrEDcXZHHV4q3YRv5i7ikgM94RlETNgihrcgBHhccCiRCf7VhBK5rAPyr9I/Y/WKPEyfksH/9NjQ2dODhsYzwcLXsypkeBtCRGLRDUUMAMyKHxEx4dtrzyP97nQMygripiQiKi4aSbPvQmKW7+OXF69ntYvBa1iPCYklZEZECsGm4ja0Ops7EJsaj4SprlU+8IJiqIjAFga3Ikx4vvAYkTGALxyWFArlsnbBC9Sz6mI5zWKNRGh3JJY7mjte4GOz+r4tkRbxQQAAAABJRU5ErkJggg=="
+}

agent/templates/market_generate_seo_blog.json

@@ -1,7 +1,11 @@
 {
     "id": 12,
-    "title": "Generate SEO Blog",
+    "title": {
-    "description": "This workflow automatically generates a complete SEO-optimized blog article based on a simple user input. You don’t need any writing experience. Just provide a topic or short request — the system will handle the rest.",
+        "en": "Generate SEO Blog",
+        "zh": "生成SEO博客"},
+    "description": {
+        "en": "This workflow automatically generates a complete SEO-optimized blog article based on a simple user input. You don’t need any writing experience. Just provide a topic or short request — the system will handle the rest.",
+        "zh": "此工作流根据简单的用户输入自动生成完整的SEO博客文章。你无需任何写作经验，只需提供一个主题或简短请求，系统将处理其余部分。"},
     "canvas_type": "Marketing",
     "dsl": {
             "components": {

agent/templates/seo_blog.json

@@ -1,7 +1,11 @@
 {
     "id": 4,
-    "title": "Generate SEO Blog",
+    "title": {
-    "description": "This workflow automatically generates a complete SEO-optimized blog article based on a simple user input. You don’t need any writing experience. Just provide a topic or short request — the system will handle the rest.",
+        "en": "Generate SEO Blog",
+        "zh": "生成SEO博客"},
+    "description": {
+        "en": "This workflow automatically generates a complete SEO-optimized blog article based on a simple user input. You don’t need any writing experience. Just provide a topic or short request — the system will handle the rest.",
+        "zh": "此工作流根据简单的用户输入自动生成完整的SEO博客文章。你无需任何写作经验，只需提供一个主题或简短请求，系统将处理其余部分。"},
     "canvas_type": "Recommended",
     "dsl": {
             "components": {

agent/templates/sql_assistant.json

@@ -1,7 +1,11 @@
 {
     "id": 17,
-    "title": "SQL Assistant",
+    "title": {
-    "description": "SQL Assistant is an AI-powered tool that lets business users turn plain-English questions into fully formed SQL queries. Simply type your question (e.g., “Show me last quarter’s top 10 products by revenue”) and SQL Assistant generates the exact SQL, runs it against your database, and returns the results in seconds. ",
+        "en": "SQL Assistant",
+        "zh": "SQL助理"},
+    "description": {
+        "en": "SQL Assistant is an AI-powered tool that lets business users turn plain-English questions into fully formed SQL queries. Simply type your question (e.g., “Show me last quarter’s top 10 products by revenue”) and SQL Assistant generates the exact SQL, runs it against your database, and returns the results in seconds. ",
+        "zh": "用户能够将简单文本问题转化为完整的SQL查询并输出结果。只需输入您的问题（例如，“展示上个季度前十名按收入排序的产品”），SQL助理就会生成精确的SQL语句，对其运行您的数据库，并几秒钟内返回结果。"},
     "canvas_type": "Marketing",
     "dsl": {
             "components": {

agent/templates/technical_docs_qa.json

@@ -1,8 +1,12 @@
 
 {
     "id": 9,
-    "title": "Technical Docs QA",
+    "title": {
-    "description": "This is a document question-and-answer system based on a knowledge base. When a user asks a question, it retrieves relevant document content to provide accurate answers.",
+        "en": "Technical Docs QA",
+        "zh": "技术文档问答"},
+    "description": {
+        "en": "This is a document question-and-answer system based on a knowledge base. When a user asks a question, it retrieves relevant document content to provide accurate answers.",
+        "zh": "基于知识库的文档问答系统，当用户提出问题时，会检索相关本地文档并提供准确回答。"},
     "canvas_type": "Customer Support",
     "dsl": {
             "components": {

agent/templates/trip_planner.json

@@ -1,9 +1,13 @@
 
 {
     "id": 14,
-    "title": "Trip Planner",
+    "title": {
-    "description": "This smart trip planner utilizes LLM technology to automatically generate customized travel itineraries, with optional tool integration for enhanced reliability.",
+		"en": "Trip Planner",
-    "canvas_type": "Consumer App",
+		"zh": "旅行规划"},
+    "description": {
+		"en": "This smart trip planner utilizes LLM technology to automatically generate customized travel itineraries, with optional tool integration for enhanced reliability.",
+		"zh": "智能旅行规划将利用大模型自动生成定制化的旅行行程，附带可选工具集成，以增强可靠性。"},
+	"canvas_type": "Consumer App",
     "dsl": {
 		"components": {
 			"Agent:OddGuestsPump": {

agent/templates/web_search_assistant.json

@@ -1,9 +1,13 @@
 
 {
     "id": 16,
-    "title": "WebSearch Assistant",
+    "title": {
-    "description": "A chat assistant template that integrates information extracted from a knowledge base and web searches to respond to queries. Let's start by setting up your knowledge base in 'Retrieval'!",
+		"en": "WebSearch Assistant",
-    "canvas_type": "Other",
+		"zh": "网页搜索助手"},
+    "description": {
+		"en": "A chat assistant template that integrates information extracted from a knowledge base and web searches to respond to queries. Let's start by setting up your knowledge base in 'Retrieval'!",
+		"zh": "集成了从知识库和网络搜索中提取的信息回答用户问题。让我们从设置您的知识库开始检索！"},
+	"canvas_type": "Other",
     "dsl": {
 		"components": {
 			"Agent:SmartSchoolsCross": {

agent/tools/base.py

@@ -166,7 +166,7 @@ class ToolBase(ComponentBase):
                 "count": 1,
                 "url": url
             })
-        self._canvas.add_refernce(chunks, aggs)
+        self._canvas.add_reference(chunks, aggs)
         self.set_output("formalized_content", "\n".join(kb_prompt({"chunks": chunks, "doc_aggs": aggs}, 200000, True)))
 
     def thoughts(self) -> str:

agent/tools/code_exec.py

@@ -79,7 +79,7 @@ def main() -> dict:
     return {
         "result": fibonacci_recursive(100),
     }
-    
+
 Here's a code example for Javascript(`main` function MUST be included and exported):
 const axios = require('axios');
 async function main(args) {
@@ -156,7 +156,7 @@ class CodeExec(ToolBase, ABC):
             self.set_output("_ERROR", "construct code request error: " + str(e))
 
         try:
-            resp = requests.post(url=f"http://{settings.SANDBOX_HOST}:9385/run", json=code_req, timeout=10)
+            resp = requests.post(url=f"http://{settings.SANDBOX_HOST}:9385/run", json=code_req, timeout=os.environ.get("COMPONENT_EXEC_TIMEOUT", 10*60))
             logging.info(f"http://{settings.SANDBOX_HOST}:9385/run", code_req, resp.status_code)
             if resp.status_code != 200:
                 resp.raise_for_status()

agent/tools/crawler.py

@@ -16,9 +16,8 @@
 from abc import ABC
 import asyncio
 from crawl4ai import AsyncWebCrawler
-
 from agent.tools.base import ToolParamBase, ToolBase
-from api.utils.web_utils import is_valid_url
+
 
 
 class CrawlerParam(ToolParamBase):
@@ -39,6 +38,7 @@ class Crawler(ToolBase, ABC):
     component_name = "Crawler"
 
     def _run(self, history, **kwargs):
+        from api.utils.web_utils import is_valid_url
         ans = self.get_input()
         ans = " - ".join(ans["content"]) if "content" in ans else ""
         if not is_valid_url(ans):
@@ -64,5 +64,5 @@ class Crawler(ToolBase, ABC):
             elif self._param.extract_type == 'markdown':
                 return result.markdown
             elif self._param.extract_type == 'content':
-                result.extracted_content
+                return result.extracted_content
             return result.markdown

agent/tools/deepl.py

@@ -43,7 +43,7 @@ class DeepLParam(ComponentParamBase):
 
 
 class DeepL(ComponentBase, ABC):
-    component_name = "GitHub"
+    component_name = "DeepL"
 
     def _run(self, history, **kwargs):
         ans = self.get_input()

agent/tools/exesql.py

@@ -13,6 +13,7 @@
 #  See the License for the specific language governing permissions and
 #  limitations under the License.
 #
+import json
 import os
 import re
 from abc import ABC
@@ -93,8 +94,20 @@ class ExeSQL(ToolBase, ABC):
         sql = kwargs.get("sql")
         if not sql:
             raise Exception("SQL for `ExeSQL` MUST not be empty.")
-        sqls = sql.split(";")
 
+        vars = self.get_input_elements_from_text(sql)
+        args = {}
+        for k, o in vars.items():
+            args[k] = o["value"]
+            if not isinstance(args[k], str):
+                try:
+                    args[k] = json.dumps(args[k], ensure_ascii=False)
+                except Exception:
+                    args[k] = str(args[k])
+            self.set_input_value(k, args[k])
+        sql = self.string_format(sql, args)
+
+        sqls = sql.split(";")
         if self._param.db_type in ["mysql", "mariadb"]:
             db = pymysql.connect(db=self._param.database, user=self._param.username, host=self._param.host,
                                  port=self._param.port, password=self._param.password)

agent/tools/retrieval.py

@@ -163,7 +163,7 @@ class Retrieval(ToolBase, ABC):
             self.set_output("formalized_content", self._param.empty_response)
             return
 
-        self._canvas.add_refernce(kbinfos["chunks"], kbinfos["doc_aggs"])
+        self._canvas.add_reference(kbinfos["chunks"], kbinfos["doc_aggs"])
         form_cnt = "\n".join(kb_prompt(kbinfos, 200000, True))
         self.set_output("formalized_content", form_cnt)
         return form_cnt

agent/tools/searxng.py

@@ -0,0 +1,156 @@
+#
+#  Copyright 2024 The InfiniFlow Authors. All Rights Reserved.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+#
+import logging
+import os
+import time
+from abc import ABC
+import requests
+from agent.tools.base import ToolMeta, ToolParamBase, ToolBase
+from api.utils.api_utils import timeout
+
+
+class SearXNGParam(ToolParamBase):
+    """
+    Define the SearXNG component parameters.
+    """
+
+    def __init__(self):
+        self.meta: ToolMeta = {
+            "name": "searxng_search",
+            "description": "SearXNG is a privacy-focused metasearch engine that aggregates results from multiple search engines without tracking users. It provides comprehensive web search capabilities.",
+            "parameters": {
+                "query": {
+                    "type": "string",
+                    "description": "The search keywords to execute with SearXNG. The keywords should be the most important words/terms(includes synonyms) from the original request.",
+                    "default": "{sys.query}",
+                    "required": True
+                },
+                "searxng_url": {
+                    "type": "string",
+                    "description": "The base URL of your SearXNG instance (e.g., http://localhost:4000). This is required to connect to your SearXNG server.",
+                    "required": False,
+                    "default": ""
+                }
+            }
+        }
+        super().__init__()
+        self.top_n = 10
+        self.searxng_url = ""
+
+    def check(self):
+        # Keep validation lenient so opening try-run panel won't fail without URL.
+        # Coerce top_n to int if it comes as string from UI.
+        try:
+            if isinstance(self.top_n, str):
+                self.top_n = int(self.top_n.strip())
+        except Exception:
+            pass
+        self.check_positive_integer(self.top_n, "Top N")
+
+    def get_input_form(self) -> dict[str, dict]:
+        return {
+            "query": {
+                "name": "Query",
+                "type": "line"
+            },
+            "searxng_url": {
+                "name": "SearXNG URL",
+                "type": "line",
+                "placeholder": "http://localhost:4000"
+            }
+        }
+
+
+class SearXNG(ToolBase, ABC):
+    component_name = "SearXNG"
+
+    @timeout(os.environ.get("COMPONENT_EXEC_TIMEOUT", 12))
+    def _invoke(self, **kwargs):
+        # Gracefully handle try-run without inputs
+        query = kwargs.get("query")
+        if not query or not isinstance(query, str) or not query.strip():
+            self.set_output("formalized_content", "")
+            return ""
+
+        searxng_url = (kwargs.get("searxng_url") or getattr(self._param, "searxng_url", "") or "").strip()
+        # In try-run, if no URL configured, just return empty instead of raising
+        if not searxng_url:
+            self.set_output("formalized_content", "")
+            return ""
+
+        last_e = ""
+        for _ in range(self._param.max_retries+1):
+            try:
+                # 构建搜索参数
+                search_params = {
+                    'q': query,
+                    'format': 'json',
+                    'categories': 'general',
+                    'language': 'auto',
+                    'safesearch': 1,
+                    'pageno': 1
+                }
+
+                # 发送搜索请求
+                response = requests.get(
+                    f"{searxng_url}/search",
+                    params=search_params,
+                    timeout=10
+                )
+                response.raise_for_status()
+                
+                data = response.json()
+                
+                # 验证响应数据
+                if not data or not isinstance(data, dict):
+                    raise ValueError("Invalid response from SearXNG")
+                
+                results = data.get("results", [])
+                if not isinstance(results, list):
+                    raise ValueError("Invalid results format from SearXNG")
+                
+                # 限制结果数量
+                results = results[:self._param.top_n]
+                
+                # 处理搜索结果
+                self._retrieve_chunks(results,
+                                      get_title=lambda r: r.get("title", ""),
+                                      get_url=lambda r: r.get("url", ""),
+                                      get_content=lambda r: r.get("content", ""))
+                
+                self.set_output("json", results)
+                return self.output("formalized_content")
+
+            except requests.RequestException as e:
+                last_e = f"Network error: {e}"
+                logging.exception(f"SearXNG network error: {e}")
+                time.sleep(self._param.delay_after_error)
+            except Exception as e:
+                last_e = str(e)
+                logging.exception(f"SearXNG error: {e}")
+                time.sleep(self._param.delay_after_error)
+
+        if last_e:
+            self.set_output("_ERROR", last_e)
+            return f"SearXNG error: {last_e}"
+
+        assert False, self.output()
+
+    def thoughts(self) -> str:
+        return """
+Keywords: {} 
+Searching with SearXNG for relevant results...
+                """.format(self.get_input().get("query", "-_-!"))

api/apps/canvas_app.py

@@ -24,7 +24,7 @@ from flask import request, Response
 from flask_login import login_required, current_user
 
 from agent.component import LLM
-from api.db import FileType
+from api.db import CanvasCategory, FileType
 from api.db.services.canvas_service import CanvasTemplateService, UserCanvasService, API4ConversationService
 from api.db.services.document_service import DocumentService
 from api.db.services.file_service import FileService
@@ -45,14 +45,14 @@ from rag.utils.redis_conn import REDIS_CONN
 @manager.route('/templates', methods=['GET'])  # noqa: F821
 @login_required
 def templates():
-    return get_json_result(data=[c.to_dict() for c in CanvasTemplateService.get_all()])
+    return get_json_result(data=[c.to_dict() for c in CanvasTemplateService.query(canvas_category=CanvasCategory.Agent)])
 
 
 @manager.route('/list', methods=['GET'])  # noqa: F821
 @login_required
 def canvas_list():
     return get_json_result(data=sorted([c.to_dict() for c in \
-                                 UserCanvasService.query(user_id=current_user.id)], key=lambda x: x["update_time"]*-1)
+                                 UserCanvasService.query(user_id=current_user.id, canvas_category=CanvasCategory.Agent)], key=lambda x: x["update_time"]*-1)
                            )
 
 
@@ -79,7 +79,7 @@ def save():
     req["dsl"] = json.loads(req["dsl"])
     if "id" not in req:
         req["user_id"] = current_user.id
-        if UserCanvasService.query(user_id=current_user.id, title=req["title"].strip()):
+        if UserCanvasService.query(user_id=current_user.id, title=req["title"].strip(), canvas_category=CanvasCategory.Agent):
             return get_data_error_result(message=f"{req['title'].strip()} already exists.")
         req["id"] = get_uuid()
         if not UserCanvasService.save(**req):
@@ -91,7 +91,7 @@ def save():
                 code=RetCode.OPERATING_ERROR)
         UserCanvasService.update_by_id(req["id"], req)
     # save version
-    UserCanvasVersionService.insert( user_canvas_id=req["id"], dsl=req["dsl"], title="{0}_{1}".format(req["title"], time.strftime("%Y_%m_%d_%H_%M_%S")))
+    UserCanvasVersionService.insert(user_canvas_id=req["id"], dsl=req["dsl"], title="{0}_{1}".format(req["title"], time.strftime("%Y_%m_%d_%H_%M_%S")))
     UserCanvasVersionService.delete_all_versions(req["id"])
     return get_json_result(data=req)
 
@@ -395,7 +395,7 @@ def list_canvas():
         tenants = TenantService.get_joined_tenants_by_user_id(current_user.id)
         canvas, total = UserCanvasService.get_by_tenant_ids(
             [m["tenant_id"] for m in tenants], current_user.id, page_number,
-            items_per_page, orderby, desc, keywords)
+            items_per_page, orderby, desc, keywords, canvas_category=CanvasCategory.Agent)
         return get_json_result(data={"canvas": canvas, "total": total})
     except Exception as e:
         return server_error_response(e)
@@ -418,12 +418,10 @@ def setting():
         return get_data_error_result(message="canvas not found.")
     flow = flow.to_dict()
     flow["title"] = req["title"]
-    if req["description"]:
+
-        flow["description"] = req["description"]
+    for key in ["description", "permission", "avatar"]:
-    if req["permission"]:
+        if value := req.get(key):
-        flow["permission"] = req["permission"]
+            flow[key] = value
-    if req["avatar"]:
-        flow["avatar"] = req["avatar"]
 
     num= UserCanvasService.update_by_id(req["id"], flow)
     return get_json_result(data=num)
@@ -472,3 +470,16 @@ def sessions(canvas_id):
     except Exception as e:
         return server_error_response(e)
 
+
+@manager.route('/prompts', methods=['GET'])  # noqa: F821
+@login_required
+def prompts():
+    from rag.prompts.prompts import ANALYZE_TASK_SYSTEM, ANALYZE_TASK_USER, NEXT_STEP, REFLECT, CITATION_PROMPT_TEMPLATE
+    return get_json_result(data={
+        "task_analysis": ANALYZE_TASK_SYSTEM +"\n\n"+ ANALYZE_TASK_USER,
+        "plan_generation": NEXT_STEP,
+        "reflection": REFLECT,
+        #"context_summary": SUMMARY4MEMORY,
+        #"context_ranking": RANK_MEMORY,
+        "citation_guidelines": CITATION_PROMPT_TEMPLATE
+    })

api/apps/chunk_app.py

@@ -93,6 +93,7 @@ def list_chunk():
 def get():
     chunk_id = request.args["chunk_id"]
     try:
+        chunk = None
         tenants = UserTenantService.query(user_id=current_user.id)
         if not tenants:
             return get_data_error_result(message="Tenant not found!")
@@ -290,6 +291,10 @@ def retrieval_test():
     kb_ids = req["kb_id"]
     if isinstance(kb_ids, str):
         kb_ids = [kb_ids]
+    if not kb_ids:
+        return get_json_result(data=False, message='Please specify dataset firstly.',
+                               code=settings.RetCode.DATA_ERROR)
+
     doc_ids = req.get("doc_ids", [])
     use_kg = req.get("use_kg", False)
     top = int(req.get("top_k", 1024))

api/apps/conversation_app.py

@@ -400,6 +400,8 @@ def related_questions():
     chat_mdl = LLMBundle(current_user.id, LLMType.CHAT, chat_id)
 
     gen_conf = search_config.get("llm_setting", {"temperature": 0.9})
+    if "parameter" in gen_conf:
+        del gen_conf["parameter"]
     prompt = load_prompt("related_question")
     ans = chat_mdl.chat(
         prompt,

api/apps/dataflow_app.py

@@ -0,0 +1,353 @@
+#
+#  Copyright 2024 The InfiniFlow Authors. All Rights Reserved.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+#
+import json
+import re
+import sys
+import time
+from functools import partial
+
+import trio
+from flask import request
+from flask_login import current_user, login_required
+
+from agent.canvas import Canvas
+from agent.component import LLM
+from api.db import CanvasCategory, FileType
+from api.db.services.canvas_service import CanvasTemplateService, UserCanvasService
+from api.db.services.document_service import DocumentService
+from api.db.services.file_service import FileService
+from api.db.services.task_service import queue_dataflow
+from api.db.services.user_canvas_version import UserCanvasVersionService
+from api.db.services.user_service import TenantService
+from api.settings import RetCode
+from api.utils import get_uuid
+from api.utils.api_utils import get_data_error_result, get_json_result, server_error_response, validate_request
+from api.utils.file_utils import filename_type, read_potential_broken_pdf
+from rag.flow.pipeline import Pipeline
+
+
+@manager.route("/templates", methods=["GET"])  # noqa: F821
+@login_required
+def templates():
+    return get_json_result(data=[c.to_dict() for c in CanvasTemplateService.query(canvas_category=CanvasCategory.DataFlow)])
+
+
+@manager.route("/list", methods=["GET"])  # noqa: F821
+@login_required
+def canvas_list():
+    return get_json_result(data=sorted([c.to_dict() for c in UserCanvasService.query(user_id=current_user.id, canvas_category=CanvasCategory.DataFlow)], key=lambda x: x["update_time"] * -1))
+
+
+@manager.route("/rm", methods=["POST"])  # noqa: F821
+@validate_request("canvas_ids")
+@login_required
+def rm():
+    for i in request.json["canvas_ids"]:
+        if not UserCanvasService.accessible(i, current_user.id):
+            return get_json_result(data=False, message="Only owner of canvas authorized for this operation.", code=RetCode.OPERATING_ERROR)
+        UserCanvasService.delete_by_id(i)
+    return get_json_result(data=True)
+
+
+@manager.route("/set", methods=["POST"])  # noqa: F821
+@validate_request("dsl", "title")
+@login_required
+def save():
+    req = request.json
+    if not isinstance(req["dsl"], str):
+        req["dsl"] = json.dumps(req["dsl"], ensure_ascii=False)
+    req["dsl"] = json.loads(req["dsl"])
+    req["canvas_category"] = CanvasCategory.DataFlow
+    if "id" not in req:
+        req["user_id"] = current_user.id
+        if UserCanvasService.query(user_id=current_user.id, title=req["title"].strip(), canvas_category=CanvasCategory.DataFlow):
+            return get_data_error_result(message=f"{req['title'].strip()} already exists.")
+        req["id"] = get_uuid()
+
+        if not UserCanvasService.save(**req):
+            return get_data_error_result(message="Fail to save canvas.")
+    else:
+        if not UserCanvasService.accessible(req["id"], current_user.id):
+            return get_json_result(data=False, message="Only owner of canvas authorized for this operation.", code=RetCode.OPERATING_ERROR)
+        UserCanvasService.update_by_id(req["id"], req)
+    # save version
+    UserCanvasVersionService.insert(user_canvas_id=req["id"], dsl=req["dsl"], title="{0}_{1}".format(req["title"], time.strftime("%Y_%m_%d_%H_%M_%S")))
+    UserCanvasVersionService.delete_all_versions(req["id"])
+    return get_json_result(data=req)
+
+
+@manager.route("/get/<canvas_id>", methods=["GET"])  # noqa: F821
+@login_required
+def get(canvas_id):
+    if not UserCanvasService.accessible(canvas_id, current_user.id):
+        return get_data_error_result(message="canvas not found.")
+    e, c = UserCanvasService.get_by_tenant_id(canvas_id)
+    return get_json_result(data=c)
+
+
+@manager.route("/run", methods=["POST"])  # noqa: F821
+@validate_request("id")
+@login_required
+def run():
+    req = request.json
+    flow_id = req.get("id", "")
+    doc_id = req.get("doc_id", "")
+    if not all([flow_id, doc_id]):
+        return get_data_error_result(message="id and doc_id are required.")
+
+    if not DocumentService.get_by_id(doc_id):
+        return get_data_error_result(message=f"Document for {doc_id} not found.")
+
+    user_id = req.get("user_id", current_user.id)
+    if not UserCanvasService.accessible(flow_id, current_user.id):
+        return get_json_result(data=False, message="Only owner of canvas authorized for this operation.", code=RetCode.OPERATING_ERROR)
+
+    e, cvs = UserCanvasService.get_by_id(flow_id)
+    if not e:
+        return get_data_error_result(message="canvas not found.")
+
+    if not isinstance(cvs.dsl, str):
+        cvs.dsl = json.dumps(cvs.dsl, ensure_ascii=False)
+
+    task_id = get_uuid()
+
+    ok, error_message = queue_dataflow(dsl=cvs.dsl, tenant_id=user_id, doc_id=doc_id, task_id=task_id, flow_id=flow_id, priority=0)
+    if not ok:
+        return server_error_response(error_message)
+
+    return get_json_result(data={"task_id": task_id, "flow_id": flow_id})
+
+
+@manager.route("/reset", methods=["POST"])  # noqa: F821
+@validate_request("id")
+@login_required
+def reset():
+    req = request.json
+    flow_id = req.get("id", "")
+    if not flow_id:
+        return get_data_error_result(message="id is required.")
+
+    if not UserCanvasService.accessible(flow_id, current_user.id):
+        return get_json_result(data=False, message="Only owner of canvas authorized for this operation.", code=RetCode.OPERATING_ERROR)
+
+    task_id = req.get("task_id", "")
+
+    try:
+        e, user_canvas = UserCanvasService.get_by_id(req["id"])
+        if not e:
+            return get_data_error_result(message="canvas not found.")
+
+        dataflow = Pipeline(dsl=json.dumps(user_canvas.dsl), tenant_id=current_user.id, flow_id=flow_id, task_id=task_id)
+        dataflow.reset()
+        req["dsl"] = json.loads(str(dataflow))
+        UserCanvasService.update_by_id(req["id"], {"dsl": req["dsl"]})
+        return get_json_result(data=req["dsl"])
+    except Exception as e:
+        return server_error_response(e)
+
+
+@manager.route("/upload/<canvas_id>", methods=["POST"])  # noqa: F821
+def upload(canvas_id):
+    e, cvs = UserCanvasService.get_by_tenant_id(canvas_id)
+    if not e:
+        return get_data_error_result(message="canvas not found.")
+
+    user_id = cvs["user_id"]
+
+    def structured(filename, filetype, blob, content_type):
+        nonlocal user_id
+        if filetype == FileType.PDF.value:
+            blob = read_potential_broken_pdf(blob)
+
+        location = get_uuid()
+        FileService.put_blob(user_id, location, blob)
+
+        return {
+            "id": location,
+            "name": filename,
+            "size": sys.getsizeof(blob),
+            "extension": filename.split(".")[-1].lower(),
+            "mime_type": content_type,
+            "created_by": user_id,
+            "created_at": time.time(),
+            "preview_url": None,
+        }
+
+    if request.args.get("url"):
+        from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CrawlResult, DefaultMarkdownGenerator, PruningContentFilter
+
+        try:
+            url = request.args.get("url")
+            filename = re.sub(r"\?.*", "", url.split("/")[-1])
+
+            async def adownload():
+                browser_config = BrowserConfig(
+                    headless=True,
+                    verbose=False,
+                )
+                async with AsyncWebCrawler(config=browser_config) as crawler:
+                    crawler_config = CrawlerRunConfig(markdown_generator=DefaultMarkdownGenerator(content_filter=PruningContentFilter()), pdf=True, screenshot=False)
+                    result: CrawlResult = await crawler.arun(url=url, config=crawler_config)
+                    return result
+
+            page = trio.run(adownload())
+            if page.pdf:
+                if filename.split(".")[-1].lower() != "pdf":
+                    filename += ".pdf"
+                return get_json_result(data=structured(filename, "pdf", page.pdf, page.response_headers["content-type"]))
+
+            return get_json_result(data=structured(filename, "html", str(page.markdown).encode("utf-8"), page.response_headers["content-type"], user_id))
+
+        except Exception as e:
+            return server_error_response(e)
+
+    file = request.files["file"]
+    try:
+        DocumentService.check_doc_health(user_id, file.filename)
+        return get_json_result(data=structured(file.filename, filename_type(file.filename), file.read(), file.content_type))
+    except Exception as e:
+        return server_error_response(e)
+
+
+@manager.route("/input_form", methods=["GET"])  # noqa: F821
+@login_required
+def input_form():
+    flow_id = request.args.get("id")
+    cpn_id = request.args.get("component_id")
+    try:
+        e, user_canvas = UserCanvasService.get_by_id(flow_id)
+        if not e:
+            return get_data_error_result(message="canvas not found.")
+        if not UserCanvasService.query(user_id=current_user.id, id=flow_id):
+            return get_json_result(data=False, message="Only owner of canvas authorized for this operation.", code=RetCode.OPERATING_ERROR)
+
+        dataflow = Pipeline(dsl=json.dumps(user_canvas.dsl), tenant_id=current_user.id, flow_id=flow_id, task_id="")
+
+        return get_json_result(data=dataflow.get_component_input_form(cpn_id))
+    except Exception as e:
+        return server_error_response(e)
+
+
+@manager.route("/debug", methods=["POST"])  # noqa: F821
+@validate_request("id", "component_id", "params")
+@login_required
+def debug():
+    req = request.json
+    if not UserCanvasService.accessible(req["id"], current_user.id):
+        return get_json_result(data=False, message="Only owner of canvas authorized for this operation.", code=RetCode.OPERATING_ERROR)
+    try:
+        e, user_canvas = UserCanvasService.get_by_id(req["id"])
+        canvas = Canvas(json.dumps(user_canvas.dsl), current_user.id)
+        canvas.reset()
+        canvas.message_id = get_uuid()
+        component = canvas.get_component(req["component_id"])["obj"]
+        component.reset()
+
+        if isinstance(component, LLM):
+            component.set_debug_inputs(req["params"])
+        component.invoke(**{k: o["value"] for k, o in req["params"].items()})
+        outputs = component.output()
+        for k in outputs.keys():
+            if isinstance(outputs[k], partial):
+                txt = ""
+                for c in outputs[k]():
+                    txt += c
+                outputs[k] = txt
+        return get_json_result(data=outputs)
+    except Exception as e:
+        return server_error_response(e)
+
+
+# api get list version dsl of canvas
+@manager.route("/getlistversion/<canvas_id>", methods=["GET"])  # noqa: F821
+@login_required
+def getlistversion(canvas_id):
+    try:
+        list = sorted([c.to_dict() for c in UserCanvasVersionService.list_by_canvas_id(canvas_id)], key=lambda x: x["update_time"] * -1)
+        return get_json_result(data=list)
+    except Exception as e:
+        return get_data_error_result(message=f"Error getting history files: {e}")
+
+
+# api get version dsl of canvas
+@manager.route("/getversion/<version_id>", methods=["GET"])  # noqa: F821
+@login_required
+def getversion(version_id):
+    try:
+        e, version = UserCanvasVersionService.get_by_id(version_id)
+        if version:
+            return get_json_result(data=version.to_dict())
+    except Exception as e:
+        return get_json_result(data=f"Error getting history file: {e}")
+
+
+@manager.route("/listteam", methods=["GET"])  # noqa: F821
+@login_required
+def list_canvas():
+    keywords = request.args.get("keywords", "")
+    page_number = int(request.args.get("page", 1))
+    items_per_page = int(request.args.get("page_size", 150))
+    orderby = request.args.get("orderby", "create_time")
+    desc = request.args.get("desc", True)
+    try:
+        tenants = TenantService.get_joined_tenants_by_user_id(current_user.id)
+        canvas, total = UserCanvasService.get_by_tenant_ids(
+            [m["tenant_id"] for m in tenants], current_user.id, page_number, items_per_page, orderby, desc, keywords, canvas_category=CanvasCategory.DataFlow
+        )
+        return get_json_result(data={"canvas": canvas, "total": total})
+    except Exception as e:
+        return server_error_response(e)
+
+
+@manager.route("/setting", methods=["POST"])  # noqa: F821
+@validate_request("id", "title", "permission")
+@login_required
+def setting():
+    req = request.json
+    req["user_id"] = current_user.id
+
+    if not UserCanvasService.accessible(req["id"], current_user.id):
+        return get_json_result(data=False, message="Only owner of canvas authorized for this operation.", code=RetCode.OPERATING_ERROR)
+
+    e, flow = UserCanvasService.get_by_id(req["id"])
+    if not e:
+        return get_data_error_result(message="canvas not found.")
+    flow = flow.to_dict()
+    flow["title"] = req["title"]
+    for key in ("description", "permission", "avatar"):
+        if value := req.get(key):
+            flow[key] = value
+
+    num = UserCanvasService.update_by_id(req["id"], flow)
+    return get_json_result(data=num)
+
+
+@manager.route("/trace", methods=["GET"])  # noqa: F821
+def trace():
+    dataflow_id = request.args.get("dataflow_id")
+    task_id = request.args.get("task_id")
+    if not all([dataflow_id, task_id]):
+        return get_data_error_result(message="dataflow_id and task_id are required.")
+
+    e, dataflow_canvas = UserCanvasService.get_by_id(dataflow_id)
+    if not e:
+        return get_data_error_result(message="dataflow not found.")
+
+    dsl_str = json.dumps(dataflow_canvas.dsl, ensure_ascii=False)
+    dataflow = Pipeline(dsl=dsl_str, tenant_id=dataflow_canvas.user_id, flow_id=dataflow_id, task_id=task_id)
+    log = dataflow.fetch_logs()
+
+    return get_json_result(data=log)

api/apps/dialog_app.py

@@ -66,7 +66,7 @@ def set_dialog():
 
     if not is_create:
         if not req.get("kb_ids", []) and not prompt_config.get("tavily_api_key") and "{knowledge}" in prompt_config['system']:
-            return get_data_error_result(message="Please remove `{knowledge}` in system prompt since no knowledge base/Tavily used here.")
+            return get_data_error_result(message="Please remove `{knowledge}` in system prompt since no knowledge base / Tavily used here.")
 
         for p in prompt_config["parameters"]:
             if p["optional"]:

api/apps/document_app.py

@@ -456,8 +456,7 @@ def run():
                     cancel_all_task_of(id)
                 else:
                     return get_data_error_result(message="Cannot cancel a task that is not in RUNNING status")
-
+            if all([("delete" not in req or req["delete"]), str(req["run"]) == TaskStatus.RUNNING.value, str(doc.run) == TaskStatus.DONE.value]):
-            if str(req["run"]) == TaskStatus.RUNNING.value and str(doc.run) == TaskStatus.DONE.value:
                 DocumentService.clear_chunk_num_when_rerun(doc.id)
 
             DocumentService.update_by_id(id, info)
@@ -683,7 +682,7 @@ def set_meta():
         meta = json.loads(req["meta"])
         if not isinstance(meta, dict):
             return get_json_result(data=False, message="Only dictionary type supported.", code=settings.RetCode.ARGUMENT_ERROR)
-        for k,v in meta.items():
+        for k, v in meta.items():
             if not isinstance(v, str) and not isinstance(v, int) and not isinstance(v, float):
                 return get_json_result(data=False, message=f"The type is not supported: {v}", code=settings.RetCode.ARGUMENT_ERROR)
     except Exception as e:

api/apps/llm_app.py

@@ -243,7 +243,7 @@ def add_llm():
                 model_name=mdl_nm,
                 base_url=llm["api_base"]
             )
-            arr, tc = mdl.similarity("Hello~ Ragflower!", ["Hi, there!", "Ohh, my friend!"])
+            arr, tc = mdl.similarity("Hello~ RAGFlower!", ["Hi, there!", "Ohh, my friend!"])
             if len(arr) == 0:
                 raise Exception("Not known.")
         except KeyError:
@@ -271,7 +271,7 @@ def add_llm():
             key=llm["api_key"], model_name=mdl_nm, base_url=llm["api_base"]
         )
         try:
-            for resp in mdl.tts("Hello~ Ragflower!"):
+            for resp in mdl.tts("Hello~ RAGFlower!"):
                 pass
         except RuntimeError as e:
             msg += f"\nFail to access model({factory}/{mdl_nm})." + str(e)

api/apps/mcp_server_app.py

@@ -82,7 +82,7 @@ def create() -> Response:
 
     server_name = req.get("name", "")
     if not server_name or len(server_name.encode("utf-8")) > 255:
-        return get_data_error_result(message=f"Invaild MCP name or length is {len(server_name)} which is large than 255.")
+        return get_data_error_result(message=f"Invalid MCP name or length is {len(server_name)} which is large than 255.")
 
     e, _ = MCPServerService.get_by_name_and_tenant(name=server_name, tenant_id=current_user.id)
     if e:
@@ -90,7 +90,7 @@ def create() -> Response:
 
     url = req.get("url", "")
     if not url:
-        return get_data_error_result(message="Invaild url.")
+        return get_data_error_result(message="Invalid url.")
 
     headers = safe_json_parse(req.get("headers", {}))
     req["headers"] = headers
@@ -141,10 +141,10 @@ def update() -> Response:
         return get_data_error_result(message="Unsupported MCP server type.")
     server_name = req.get("name", mcp_server.name)
     if server_name and len(server_name.encode("utf-8")) > 255:
-        return get_data_error_result(message=f"Invaild MCP name or length is {len(server_name)} which is large than 255.")
+        return get_data_error_result(message=f"Invalid MCP name or length is {len(server_name)} which is large than 255.")
     url = req.get("url", mcp_server.url)
     if not url:
-        return get_data_error_result(message="Invaild url.")
+        return get_data_error_result(message="Invalid url.")
 
     headers = safe_json_parse(req.get("headers", mcp_server.headers))
     req["headers"] = headers
@@ -218,7 +218,7 @@ def import_multiple() -> Response:
                 continue
 
             if not server_name or len(server_name.encode("utf-8")) > 255:
-                results.append({"server": server_name, "success": False, "message": f"Invaild MCP name or length is {len(server_name)} which is large than 255."})
+                results.append({"server": server_name, "success": False, "message": f"Invalid MCP name or length is {len(server_name)} which is large than 255."})
                 continue
 
             base_name = server_name
@@ -409,7 +409,7 @@ def test_mcp() -> Response:
 
     url = req.get("url", "")
     if not url:
-        return get_data_error_result(message="Invaild MCP url.")
+        return get_data_error_result(message="Invalid MCP url.")
 
     server_type = req.get("server_type", "")
     if server_type not in VALID_MCP_SERVER_TYPES:

api/apps/sdk/chat.py

@@ -150,10 +150,10 @@ def update(tenant_id, chat_id):
     if not DialogService.query(tenant_id=tenant_id, id=chat_id, status=StatusEnum.VALID.value):
         return get_error_data_result(message="You do not own the chat")
     req = request.json
-    ids = req.get("dataset_ids")
+    ids = req.get("dataset_ids", [])
     if "show_quotation" in req:
         req["do_refer"] = req.pop("show_quotation")
-    if ids is not None:
+    if ids:
         for kb_id in ids:
             kbs = KnowledgebaseService.accessible(kb_id=kb_id, user_id=tenant_id)
             if not kbs:

api/apps/sdk/dify_retrieval.py

@@ -24,6 +24,7 @@ from api.db.services.llm_service import LLMBundle
 from api import settings
 from api.utils.api_utils import validate_request, build_error_result, apikey_required
 from rag.app.tag import label_question
+from api.db.services.dialog_service import meta_filter, convert_conditions
 
 
 @manager.route('/dify/retrieval', methods=['POST'])  # noqa: F821
@@ -37,18 +38,23 @@ def retrieval(tenant_id):
     retrieval_setting = req.get("retrieval_setting", {})
     similarity_threshold = float(retrieval_setting.get("score_threshold", 0.0))
     top = int(retrieval_setting.get("top_k", 1024))
-
+    metadata_condition = req.get("metadata_condition",{})
+    metas = DocumentService.get_meta_by_kbs([kb_id])
+ 
+    doc_ids = []
     try:
 
         e, kb = KnowledgebaseService.get_by_id(kb_id)
         if not e:
             return build_error_result(message="Knowledgebase not found!", code=settings.RetCode.NOT_FOUND)
 
-        if kb.tenant_id != tenant_id:
-            return build_error_result(message="Knowledgebase not found!", code=settings.RetCode.NOT_FOUND)
-
         embd_mdl = LLMBundle(kb.tenant_id, LLMType.EMBEDDING.value, llm_name=kb.embd_id)
-
+        print(metadata_condition)
+        print("after",convert_conditions(metadata_condition))
+        doc_ids.extend(meta_filter(metas, convert_conditions(metadata_condition)))
+        print("doc_ids",doc_ids)
+        if not doc_ids and metadata_condition is not None:
+            doc_ids = ['-999']
         ranks = settings.retrievaler.retrieval(
             question,
             embd_mdl,
@@ -59,6 +65,7 @@ def retrieval(tenant_id):
             similarity_threshold=similarity_threshold,
             vector_similarity_weight=0.3,
             top=top,
+            doc_ids=doc_ids,
             rank_feature=label_question(question, [kb])
         )
 
@@ -93,3 +100,5 @@ def retrieval(tenant_id):
             )
         logging.exception(e)
         return build_error_result(message=str(e), code=settings.RetCode.SERVER_ERROR)
+
+

api/apps/sdk/doc.py

@@ -35,6 +35,7 @@ from api.db.services.knowledgebase_service import KnowledgebaseService
 from api.db.services.llm_service import LLMBundle
 from api.db.services.tenant_llm_service import TenantLLMService
 from api.db.services.task_service import TaskService, queue_tasks
+from api.db.services.dialog_service import meta_filter, convert_conditions
 from api.utils.api_utils import check_duplicate_ids, construct_json_result, get_error_data_result, get_parser_config, get_result, server_error_response, token_required
 from rag.app.qa import beAdoc, rmPrefix
 from rag.app.tag import label_question
@@ -1350,6 +1351,9 @@ def retrieval_test(tenant_id):
             highlight:
               type: boolean
               description: Whether to highlight matched content.
+            metadata_condition:
+              type: object
+              description: metadata filter condition.
       - in: header
         name: Authorization
         type: string
@@ -1413,6 +1417,10 @@ def retrieval_test(tenant_id):
     for doc_id in doc_ids:
         if doc_id not in doc_ids_list:
             return get_error_data_result(f"The datasets don't own the document {doc_id}")
+    if not doc_ids:
+        metadata_condition = req.get("metadata_condition", {})
+        metas = DocumentService.get_meta_by_kbs(kb_ids)
+        doc_ids = meta_filter(metas, convert_conditions(metadata_condition))
     similarity_threshold = float(req.get("similarity_threshold", 0.2))
     vector_similarity_weight = float(req.get("vector_similarity_weight", 0.3))
     top = int(req.get("top_k", 1024))

api/apps/sdk/session.py

@@ -16,8 +16,10 @@
 import json
 import re
 import time
+
 import tiktoken
 from flask import Response, jsonify, request
+
 from agent.canvas import Canvas
 from api import settings
 from api.db import LLMType, StatusEnum
@@ -27,7 +29,8 @@ from api.db.services.canvas_service import UserCanvasService, completionOpenAI
 from api.db.services.canvas_service import completion as agent_completion
 from api.db.services.conversation_service import ConversationService, iframe_completion
 from api.db.services.conversation_service import completion as rag_completion
-from api.db.services.dialog_service import DialogService, ask, chat, gen_mindmap
+from api.db.services.dialog_service import DialogService, ask, chat, gen_mindmap, meta_filter
+from api.db.services.document_service import DocumentService
 from api.db.services.knowledgebase_service import KnowledgebaseService
 from api.db.services.llm_service import LLMBundle
 from api.db.services.search_service import SearchService
@@ -37,7 +40,7 @@ from api.utils.api_utils import check_duplicate_ids, get_data_openai, get_error_
 from rag.app.tag import label_question
 from rag.prompts import chunks_format
 from rag.prompts.prompt_template import load_prompt
-from rag.prompts.prompts import cross_languages, keyword_extraction
+from rag.prompts.prompts import cross_languages, gen_meta_filter, keyword_extraction
 
 
 @manager.route("/chats/<chat_id>/sessions", methods=["POST"])  # noqa: F821
@@ -81,21 +84,13 @@ def create_agent_session(tenant_id, agent_id):
     if not isinstance(cvs.dsl, str):
         cvs.dsl = json.dumps(cvs.dsl, ensure_ascii=False)
 
-    session_id=get_uuid()
+    session_id = get_uuid()
     canvas = Canvas(cvs.dsl, tenant_id, agent_id)
     canvas.reset()
-    conv = {
-        "id": session_id,
-        "dialog_id": cvs.id,
-        "user_id": user_id,
-        "message": [],
-        "source": "agent",
-        "dsl": cvs.dsl
-    }
-    API4ConversationService.save(**conv)
 
     cvs.dsl = json.loads(str(canvas))
     conv = {"id": session_id, "dialog_id": cvs.id, "user_id": user_id, "message": [{"role": "assistant", "content": canvas.get_prologue()}], "source": "agent", "dsl": cvs.dsl}
+    API4ConversationService.save(**conv)
     conv["agent_id"] = conv.pop("dialog_id")
     return get_result(data=conv)
 
@@ -419,7 +414,7 @@ def agents_completion_openai_compatibility(tenant_id, agent_id):
                 tenant_id,
                 agent_id,
                 question,
-                session_id=req.get("id", req.get("metadata", {}).get("id", "")),
+                session_id=req.get("session_id", req.get("id", "") or req.get("metadata", {}).get("id", "")),
                 stream=True,
                 **req,
             ),
@@ -437,7 +432,7 @@ def agents_completion_openai_compatibility(tenant_id, agent_id):
                 tenant_id,
                 agent_id,
                 question,
-                session_id=req.get("id", req.get("metadata", {}).get("id", "")),
+                session_id=req.get("session_id", req.get("id", "") or req.get("metadata", {}).get("id", "")),
                 stream=False,
                 **req,
             )
@@ -450,7 +445,6 @@ def agents_completion_openai_compatibility(tenant_id, agent_id):
 def agent_completions(tenant_id, agent_id):
     req = request.json
 
-    ans = {}
     if req.get("stream", True):
 
         def generate():
@@ -461,7 +455,7 @@ def agent_completions(tenant_id, agent_id):
                     except Exception:
                         continue
 
-                if ans.get("event") != "message":
+                if ans.get("event") not in ["message", "message_end"]:
                     continue
 
                 yield answer
@@ -475,12 +469,25 @@ def agent_completions(tenant_id, agent_id):
         resp.headers.add_header("Content-Type", "text/event-stream; charset=utf-8")
         return resp
 
+    full_content = ""
+    reference = {}
+    final_ans = ""
     for answer in agent_completion(tenant_id=tenant_id, agent_id=agent_id, **req):
         try:
-            ans = json.loads(answer[5:])  # remove "data:"
+            ans = json.loads(answer[5:])
+
+            if ans["event"] == "message":
+                full_content += ans["data"]["content"]
+
+            if ans.get("data", {}).get("reference", None):
+                reference.update(ans["data"]["reference"])
+
+            final_ans = ans
         except Exception as e:
             return get_result(data=f"**ERROR**: {str(e)}")
-    return get_result(data=ans)
+    final_ans["data"]["content"] = full_content
+    final_ans["data"]["reference"] = reference
+    return get_result(data=final_ans)
 
 
 @manager.route("/chats/<chat_id>/sessions", methods=["GET"])  # noqa: F821
@@ -575,12 +582,12 @@ def list_agent_session(tenant_id, agent_id):
                 if message_num != 0 and messages[message_num]["role"] != "user":
                     chunk_list = []
                     # Add boundary and type checks to prevent KeyError
-                    if (chunk_num < len(conv["reference"]) and
+                    if chunk_num < len(conv["reference"]) and conv["reference"][chunk_num] is not None and isinstance(conv["reference"][chunk_num], dict) and "chunks" in conv["reference"][chunk_num]:
-                            conv["reference"][chunk_num] is not None and
-                            isinstance(conv["reference"][chunk_num], dict) and
-                            "chunks" in conv["reference"][chunk_num]):
                         chunks = conv["reference"][chunk_num]["chunks"]
                         for chunk in chunks:
+                            # Ensure chunk is a dictionary before calling get method
+                            if not isinstance(chunk, dict):
+                                continue
                             new_chunk = {
                                 "id": chunk.get("chunk_id", chunk.get("id")),
                                 "content": chunk.get("content_with_weight", chunk.get("content")),
@@ -876,14 +883,7 @@ def begin_inputs(agent_id):
         return get_error_data_result(f"Can't find agent by ID: {agent_id}")
 
     canvas = Canvas(json.dumps(cvs.dsl), objs[0].tenant_id)
-    return get_result(
+    return get_result(data={"title": cvs.title, "avatar": cvs.avatar, "inputs": canvas.get_component_input_form("begin"), "prologue": canvas.get_prologue(), "mode": canvas.get_mode()})
-        data={
-            "title": cvs.title,
-            "avatar": cvs.avatar,
-            "inputs": canvas.get_component_input_form("begin"),
-            "prologue": canvas.get_prologue()
-        }
-    )
 
 
 @manager.route("/searchbots/ask", methods=["POST"])  # noqa: F821
@@ -909,7 +909,7 @@ def ask_about_embedded():
     def stream():
         nonlocal req, uid
         try:
-            for ans in ask(req["question"], req["kb_ids"], uid, search_config):
+            for ans in ask(req["question"], req["kb_ids"], uid, search_config=search_config):
                 yield "data:" + json.dumps({"code": 0, "message": "", "data": ans}, ensure_ascii=False) + "\n\n"
         except Exception as e:
             yield "data:" + json.dumps({"code": 500, "message": str(e), "data": {"answer": "**ERROR**: " + str(e), "reference": []}}, ensure_ascii=False) + "\n\n"
@@ -923,7 +923,7 @@ def ask_about_embedded():
     return resp
 
 
-@manager.route("/searchbots/retrieval_test", methods=['POST'])  # noqa: F821
+@manager.route("/searchbots/retrieval_test", methods=["POST"])  # noqa: F821
 @validate_request("kb_id", "question")
 def retrieval_test_embedded():
     token = request.headers.get("Authorization").split()
@@ -941,6 +941,9 @@ def retrieval_test_embedded():
     kb_ids = req["kb_id"]
     if isinstance(kb_ids, str):
         kb_ids = [kb_ids]
+    if not kb_ids:
+        return get_json_result(data=False, message='Please specify dataset firstly.',
+                               code=settings.RetCode.DATA_ERROR)
     doc_ids = req.get("doc_ids", [])
     similarity_threshold = float(req.get("similarity_threshold", 0.0))
     vector_similarity_weight = float(req.get("vector_similarity_weight", 0.3))
@@ -953,18 +956,30 @@ def retrieval_test_embedded():
     if not tenant_id:
         return get_error_data_result(message="permission denined.")
 
+    if req.get("search_id", ""):
+        search_config = SearchService.get_detail(req.get("search_id", "")).get("search_config", {})
+        meta_data_filter = search_config.get("meta_data_filter", {})
+        metas = DocumentService.get_meta_by_kbs(kb_ids)
+        if meta_data_filter.get("method") == "auto":
+            chat_mdl = LLMBundle(tenant_id, LLMType.CHAT, llm_name=search_config.get("chat_id", ""))
+            filters = gen_meta_filter(chat_mdl, metas, question)
+            doc_ids.extend(meta_filter(metas, filters))
+            if not doc_ids:
+                doc_ids = None
+        elif meta_data_filter.get("method") == "manual":
+            doc_ids.extend(meta_filter(metas, meta_data_filter["manual"]))
+            if not doc_ids:
+                doc_ids = None
+
     try:
         tenants = UserTenantService.query(user_id=tenant_id)
         for kb_id in kb_ids:
             for tenant in tenants:
-                if KnowledgebaseService.query(
+                if KnowledgebaseService.query(tenant_id=tenant.tenant_id, id=kb_id):
-                        tenant_id=tenant.tenant_id, id=kb_id):
                     tenant_ids.append(tenant.tenant_id)
                     break
             else:
-                return get_json_result(
+                return get_json_result(data=False, message="Only owner of knowledgebase authorized for this operation.", code=settings.RetCode.OPERATING_ERROR)
-                    data=False, message='Only owner of knowledgebase authorized for this operation.',
-                    code=settings.RetCode.OPERATING_ERROR)
 
         e, kb = KnowledgebaseService.get_by_id(kb_ids[0])
         if not e:
@@ -984,17 +999,11 @@ def retrieval_test_embedded():
             question += keyword_extraction(chat_mdl, question)
 
         labels = label_question(question, [kb])
-        ranks = settings.retrievaler.retrieval(question, embd_mdl, tenant_ids, kb_ids, page, size,
+        ranks = settings.retrievaler.retrieval(
-                               similarity_threshold, vector_similarity_weight, top,
+            question, embd_mdl, tenant_ids, kb_ids, page, size, similarity_threshold, vector_similarity_weight, top, doc_ids, rerank_mdl=rerank_mdl, highlight=req.get("highlight"), rank_feature=labels
-                               doc_ids, rerank_mdl=rerank_mdl, highlight=req.get("highlight"),
+        )
-                               rank_feature=labels
-                               )
         if use_kg:
-            ck = settings.kg_retrievaler.retrieval(question,
+            ck = settings.kg_retrievaler.retrieval(question, tenant_ids, kb_ids, embd_mdl, LLMBundle(kb.tenant_id, LLMType.CHAT))
-                                                   tenant_ids,
-                                                   kb_ids,
-                                                   embd_mdl,
-                                                   LLMBundle(kb.tenant_id, LLMType.CHAT))
             if ck["content_with_weight"]:
                 ranks["chunks"].insert(0, ck)
 
@@ -1005,8 +1014,7 @@ def retrieval_test_embedded():
         return get_json_result(data=ranks)
     except Exception as e:
         if str(e).find("not_found") > 0:
-            return get_json_result(data=False, message='No chunk found! Check the chunk status please!',
+            return get_json_result(data=False, message="No chunk found! Check the chunk status please!", code=settings.RetCode.DATA_ERROR)
-                                   code=settings.RetCode.DATA_ERROR)
         return server_error_response(e)
 
 

api/apps/search_app.py

@@ -43,7 +43,7 @@ def create():
         return get_data_error_result(message=f"Search name length is {len(search_name)} which is large than 255.")
     e, _ = TenantService.get_by_id(current_user.id)
     if not e:
-        return get_data_error_result(message="Authorizationd identity.")
+        return get_data_error_result(message="Authorized identity.")
 
     search_name = search_name.strip()
     search_name = duplicate_name(SearchService.query, name=search_name, tenant_id=current_user.id, status=StatusEnum.VALID.value)
@@ -78,7 +78,7 @@ def update():
     tenant_id = req["tenant_id"]
     e, _ = TenantService.get_by_id(tenant_id)
     if not e:
-        return get_data_error_result(message="Authorizationd identity.")
+        return get_data_error_result(message="Authorized identity.")
 
     search_id = req["search_id"]
     if not SearchService.accessible4deletion(search_id, current_user.id):
@@ -155,8 +155,9 @@ def list_search_app():
     owner_ids = req.get("owner_ids", [])
     try:
         if not owner_ids:
-            tenants = TenantService.get_joined_tenants_by_user_id(current_user.id)
+            # tenants = TenantService.get_joined_tenants_by_user_id(current_user.id)
-            tenants = [m["tenant_id"] for m in tenants]
+            # tenants = [m["tenant_id"] for m in tenants]
+            tenants = []
             search_apps, total = SearchService.get_by_tenant_ids(tenants, current_user.id, page_number, items_per_page, orderby, desc, keywords)
         else:
             tenants = owner_ids

api/db/__init__.py

@@ -74,8 +74,10 @@ class TaskStatus(StrEnum):
     DONE = "3"
     FAIL = "4"
 
+
 VALID_TASK_STATUS     = {TaskStatus.UNSTART, TaskStatus.RUNNING, TaskStatus.CANCEL, TaskStatus.DONE, TaskStatus.FAIL}
 
+
 class ParserType(StrEnum):
     PRESENTATION = "presentation"
     LAWS = "laws"
@@ -105,10 +107,19 @@ class CanvasType(StrEnum):
     DocBot = "docbot"
 
 
+class CanvasCategory(StrEnum):
+    Agent = "agent_canvas"
+    DataFlow = "dataflow_canvas"
+
+VALID_CAVAS_CATEGORIES = {CanvasCategory.Agent, CanvasCategory.DataFlow}
+
+
 class MCPServerType(StrEnum):
     SSE = "sse"
     STREAMABLE_HTTP = "streamable-http"
 
+
 VALID_MCP_SERVER_TYPES = {MCPServerType.SSE, MCPServerType.STREAMABLE_HTTP}
 
+
 KNOWLEDGEBASE_FOLDER_NAME=".knowledgebase"

api/db/db_models.py

@@ -245,22 +245,21 @@ class JsonSerializedField(SerializedField):
 
 class RetryingPooledMySQLDatabase(PooledMySQLDatabase):
     def __init__(self, *args, **kwargs):
-        self.max_retries = kwargs.pop('max_retries', 5)
+        self.max_retries = kwargs.pop("max_retries", 5)
-        self.retry_delay = kwargs.pop('retry_delay', 1)
+        self.retry_delay = kwargs.pop("retry_delay", 1)
         super().__init__(*args, **kwargs)
 
     def execute_sql(self, sql, params=None, commit=True):
         from peewee import OperationalError
+
         for attempt in range(self.max_retries + 1):
             try:
                 return super().execute_sql(sql, params, commit)
             except OperationalError as e:
                 if e.args[0] in (2013, 2006) and attempt < self.max_retries:
-                    logging.warning(
+                    logging.warning(f"Lost connection (attempt {attempt + 1}/{self.max_retries}): {e}")
-                        f"Lost connection (attempt {attempt+1}/{self.max_retries}): {e}"
-                    )
                     self._handle_connection_loss()
-                    time.sleep(self.retry_delay * (2 ** attempt))
+                    time.sleep(self.retry_delay * (2**attempt))
                 else:
                     logging.error(f"DB execution failure: {e}")
                     raise
@@ -272,16 +271,15 @@ class RetryingPooledMySQLDatabase(PooledMySQLDatabase):
 
     def begin(self):
         from peewee import OperationalError
+
         for attempt in range(self.max_retries + 1):
             try:
                 return super().begin()
             except OperationalError as e:
                 if e.args[0] in (2013, 2006) and attempt < self.max_retries:
-                    logging.warning(
+                    logging.warning(f"Lost connection during transaction (attempt {attempt + 1}/{self.max_retries})")
-                        f"Lost connection during transaction (attempt {attempt+1}/{self.max_retries})"
-                    )
                     self._handle_connection_loss()
-                    time.sleep(self.retry_delay * (2 ** attempt))
+                    time.sleep(self.retry_delay * (2**attempt))
                 else:
                     raise
 
@@ -815,6 +813,7 @@ class UserCanvas(DataBaseModel):
     permission = CharField(max_length=16, null=False, help_text="me|team", default="me", index=True)
     description = TextField(null=True, help_text="Canvas description")
     canvas_type = CharField(max_length=32, null=True, help_text="Canvas type", index=True)
+    canvas_category = CharField(max_length=32, null=False, default="agent_canvas", help_text="Canvas category: agent_canvas|dataflow_canvas", index=True)
     dsl = JSONField(null=True, default={})
 
     class Meta:
@@ -824,10 +823,10 @@ class UserCanvas(DataBaseModel):
 class CanvasTemplate(DataBaseModel):
     id = CharField(max_length=32, primary_key=True)
     avatar = TextField(null=True, help_text="avatar base64 string")
-    title = CharField(max_length=255, null=True, help_text="Canvas title")
+    title = JSONField(null=True, default=dict, help_text="Canvas title")
-
+    description = JSONField(null=True, default=dict, help_text="Canvas description")
-    description = TextField(null=True, help_text="Canvas description")
     canvas_type = CharField(max_length=32, null=True, help_text="Canvas type", index=True)
+    canvas_category = CharField(max_length=32, null=False, default="agent_canvas", help_text="Canvas category: agent_canvas|dataflow_canvas", index=True)
     dsl = JSONField(null=True, default={})
 
     class Meta:
@@ -1021,4 +1020,21 @@ def migrate_db():
         migrate(migrator.add_column("dialog", "meta_data_filter", JSONField(null=True, default={})))
     except Exception:
         pass
+
+    try:
+        migrate(migrator.alter_column_type("canvas_template", "title", JSONField(null=True, default=dict, help_text="Canvas title")))
+    except Exception:
+        pass
+    try:
+        migrate(migrator.alter_column_type("canvas_template", "description", JSONField(null=True, default=dict, help_text="Canvas description")))
+    except Exception:
+        pass
+    try:
+        migrate(migrator.add_column("user_canvas", "canvas_category", CharField(max_length=32, null=False, default="agent_canvas", help_text="agent_canvas|dataflow_canvas", index=True)))
+    except Exception:
+        pass
+    try:
+        migrate(migrator.add_column("canvas_template", "canvas_category", CharField(max_length=32, null=False, default="agent_canvas", help_text="agent_canvas|dataflow_canvas", index=True)))
+    except Exception:
+        pass
     logging.disable(logging.NOTSET)

api/db/services/canvas_service.py

@@ -18,7 +18,7 @@ import logging
 import time
 from uuid import uuid4
 from agent.canvas import Canvas
-from api.db import TenantPermission
+from api.db import CanvasCategory, TenantPermission
 from api.db.db_models import DB, CanvasTemplate, User, UserCanvas, API4Conversation
 from api.db.services.api_service import API4ConversationService
 from api.db.services.common_service import CommonService
@@ -31,6 +31,12 @@ from peewee import fn
 class CanvasTemplateService(CommonService):
     model = CanvasTemplate
 
+class DataFlowTemplateService(CommonService):
+    """
+    Alias of CanvasTemplateService
+    """
+    model = CanvasTemplate
+
 
 class UserCanvasService(CommonService):
     model = UserCanvas
@@ -38,13 +44,14 @@ class UserCanvasService(CommonService):
     @classmethod
     @DB.connection_context()
     def get_list(cls, tenant_id,
-                 page_number, items_per_page, orderby, desc, id, title):
+                 page_number, items_per_page, orderby, desc, id, title, canvas_category=CanvasCategory.Agent):
         agents = cls.model.select()
         if id:
             agents = agents.where(cls.model.id == id)
         if title:
             agents = agents.where(cls.model.title == title)
         agents = agents.where(cls.model.user_id == tenant_id)
+        agents = agents.where(cls.model.canvas_category == canvas_category)
         if desc:
             agents = agents.order_by(cls.model.getter_by(orderby).desc())
         else:
@@ -71,6 +78,7 @@ class UserCanvasService(CommonService):
                 cls.model.create_time,
                 cls.model.create_date,
                 cls.model.update_date,
+                cls.model.canvas_category,
                 User.nickname,
                 User.avatar.alias('tenant_avatar'),
             ]
@@ -87,7 +95,7 @@ class UserCanvasService(CommonService):
     @DB.connection_context()
     def get_by_tenant_ids(cls, joined_tenant_ids, user_id,
                           page_number, items_per_page,
-                          orderby, desc, keywords,
+                          orderby, desc, keywords, canvas_category=CanvasCategory.Agent,
                           ):
         fields = [
             cls.model.id,
@@ -98,7 +106,8 @@ class UserCanvasService(CommonService):
             cls.model.permission,
             User.nickname,
             User.avatar.alias('tenant_avatar'),
-            cls.model.update_time
+            cls.model.update_time,
+            cls.model.canvas_category,
         ]
         if keywords:
             agents = cls.model.select(*fields).join(User, on=(cls.model.user_id == User.id)).where(
@@ -113,6 +122,7 @@ class UserCanvasService(CommonService):
                                                                 TenantPermission.TEAM.value)) | (
                     cls.model.user_id == user_id))
             )
+        agents = agents.where(cls.model.canvas_category == canvas_category)
         if desc:
             agents = agents.order_by(cls.model.getter_by(orderby).desc())
         else:
@@ -134,6 +144,7 @@ class UserCanvasService(CommonService):
             return False
         return True
 
+
 def completion(tenant_id, agent_id, session_id=None, **kwargs):
     query = kwargs.get("query", "") or kwargs.get("question", "")
     files = kwargs.get("files", [])
@@ -163,7 +174,8 @@ def completion(tenant_id, agent_id, session_id=None, **kwargs):
             "user_id": user_id,
             "message": [],
             "source": "agent",
-            "dsl": cvs.dsl
+            "dsl": cvs.dsl,
+            "reference": []
         }
         API4ConversationService.save(**conv)
         conv = API4Conversation(**conv)
@@ -211,28 +223,33 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
                     except Exception as e:
                         logging.exception(f"Agent OpenAI-Compatible completionOpenAI parse answer failed: {e}")
                         continue
-
+                if ans.get("event") not in ["message", "message_end"]:
-                if ans.get("event") != "message":
                     continue
 
-                content_piece = ans["data"]["content"]
+                content_piece = ""
+                if ans["event"] == "message":
+                    content_piece = ans["data"]["content"]
+
                 completion_tokens += len(tiktokenenc.encode(content_piece))
 
-                yield "data: " + json.dumps(
+                openai_data = get_data_openai(
-                    get_data_openai(
                         id=session_id or str(uuid4()),
                         model=agent_id,
                         content=content_piece,
                         prompt_tokens=prompt_tokens,
                         completion_tokens=completion_tokens,
                         stream=True
-                    ),
+                    )
-                    ensure_ascii=False
+
-                ) + "\n\n"
+                if ans.get("data", {}).get("reference", None):
+                    openai_data["choices"][0]["delta"]["reference"] = ans["data"]["reference"]
+
+                yield "data: " + json.dumps(openai_data, ensure_ascii=False) + "\n\n"
 
             yield "data: [DONE]\n\n"
 
         except Exception as e:
+            logging.exception(e)
             yield "data: " + json.dumps(
                 get_data_openai(
                     id=session_id or str(uuid4()),
@@ -250,6 +267,7 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
     else:
         try:
             all_content = ""
+            reference = {}
             for ans in completion(
                 tenant_id=tenant_id,
                 agent_id=agent_id,
@@ -260,13 +278,18 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
             ):
                 if isinstance(ans, str):
                     ans = json.loads(ans[5:])
-                if ans.get("event") != "message":
+                if ans.get("event") not in ["message", "message_end"]:
                     continue
-                all_content += ans["data"]["content"]
+
+                if ans["event"] == "message":
+                    all_content += ans["data"]["content"]
+
+                if ans.get("data", {}).get("reference", None):
+                    reference.update(ans["data"]["reference"])
 
             completion_tokens = len(tiktokenenc.encode(all_content))
 
-            yield get_data_openai(
+            openai_data = get_data_openai(
                 id=session_id or str(uuid4()),
                 model=agent_id,
                 prompt_tokens=prompt_tokens,
@@ -276,7 +299,12 @@ def completionOpenAI(tenant_id, agent_id, question, session_id=None, stream=True
                 param=None
             )
 
+            if reference:
+                openai_data["choices"][0]["message"]["reference"] = reference
+
+            yield openai_data
         except Exception as e:
+            logging.exception(e)
             yield get_data_openai(
                 id=session_id or str(uuid4()),
                 model=agent_id,

api/db/services/dialog_service.py

@@ -21,11 +21,9 @@ from copy import deepcopy
 from datetime import datetime
 from functools import partial
 from timeit import default_timer as timer
-
 import trio
 from langfuse import Langfuse
 from peewee import fn
-
 from agentic_reasoning import DeepResearcher
 from api import settings
 from api.db import LLMType, ParserType, StatusEnum
@@ -255,11 +253,28 @@ def repair_bad_citation_formats(answer: str, kbinfos: dict, idx: set):
     return answer, idx
 
 
+def convert_conditions(metadata_condition):
+    if metadata_condition is None:
+        metadata_condition = {}
+    op_mapping = {
+        "is": "=",
+        "not is": "≠"
+    }
+    return [
+    {
+        "op": op_mapping.get(cond["comparison_operator"], cond["comparison_operator"]),
+        "key": cond["name"],
+        "value": cond["value"]
+    }
+    for cond in metadata_condition.get("conditions", [])
+]
+
+
 def meta_filter(metas: dict, filters: list[dict]):
-    doc_ids = []
+    doc_ids = set([])
 
     def filter_out(v2docs, operator, value):
-        nonlocal doc_ids
+        ids = []
         for input, docids in v2docs.items():
             try:
                 input = float(input)
@@ -284,16 +299,24 @@ def meta_filter(metas: dict, filters: list[dict]):
                 ]:
                 try:
                     if all(conds):
-                        doc_ids.extend(docids)
+                        ids.extend(docids)
+                        break
                 except Exception:
                     pass
+        return ids
 
     for k, v2docs in metas.items():
         for f in filters:
             if k != f["key"]:
                 continue
-            filter_out(v2docs, f["op"], f["value"])
+            ids = filter_out(v2docs, f["op"], f["value"])
-    return doc_ids
+            if not doc_ids:
+                doc_ids = set(ids)
+            else:
+                doc_ids = doc_ids & set(ids)
+            if not doc_ids:
+                return []
+    return list(doc_ids)
 
 
 def chat(dialog, messages, stream=True, **kwargs):
@@ -342,7 +365,7 @@ def chat(dialog, messages, stream=True, **kwargs):
     # try to use sql if field mapping is good to go
     if field_map:
         logging.debug("Use SQL to retrieval:{}".format(questions[-1]))
-        ans = use_sql(questions[-1], field_map, dialog.tenant_id, chat_mdl, prompt_config.get("quote", True))
+        ans = use_sql(questions[-1], field_map, dialog.tenant_id, chat_mdl, prompt_config.get("quote", True), dialog.kb_ids)
         if ans:
             yield ans
             return
@@ -570,7 +593,7 @@ def chat(dialog, messages, stream=True, **kwargs):
         yield res
 
 
-def use_sql(question, field_map, tenant_id, chat_mdl, quota=True):
+def use_sql(question, field_map, tenant_id, chat_mdl, quota=True, kb_ids=None):
     sys_prompt = "You are a Database Administrator. You need to check the fields of the following tables based on the user's list of questions and write the SQL corresponding to the last question."
     user_prompt = """
 Table name: {};
@@ -607,6 +630,13 @@ Please write the SQL, only SQL, without any other explanations or text.
                     flds.append(k)
                 sql = "select doc_id,docnm_kwd," + ",".join(flds) + sql[8:]
 
+        if kb_ids:
+            kb_filter = "(" + " OR ".join([f"kb_id = '{kb_id}'" for kb_id in kb_ids]) + ")"
+            if "where" not in sql.lower():
+                sql += f" WHERE {kb_filter}"
+            else:
+                sql += f" AND {kb_filter}"
+
         logging.debug(f"{question} get SQL(refined): {sql}")
         tried_times += 1
         return settings.retrievaler.sql_retrieval(sql, format="json"), sql
@@ -813,4 +843,4 @@ def gen_mindmap(question, kb_ids, tenant_id, search_config={}):
     )
     mindmap = MindMapExtractor(chat_mdl)
     mind_map = trio.run(mindmap, [c["content_with_weight"] for c in ranks["chunks"]])
-    return mind_map.output
+    return mind_map.output

api/db/services/llm_service.py

@@ -152,7 +152,7 @@ class LLMBundle(LLM4Tenant):
 
     def describe_with_prompt(self, image, prompt):
         if self.langfuse:
-            generation = self.language.start_generation(trace_context=self.trace_context, name="describe_with_prompt", metadata={"model": self.llm_name, "prompt": prompt})
+            generation = self.langfuse.start_generation(trace_context=self.trace_context, name="describe_with_prompt", metadata={"model": self.llm_name, "prompt": prompt})
 
         txt, used_tokens = self.mdl.describe_with_prompt(image, prompt)
         if not TenantLLMService.increase_usage(self.tenant_id, self.llm_type, used_tokens):

api/db/services/task_service.py

@@ -54,15 +54,15 @@ def trim_header_by_lines(text: str, max_length) -> str:
 
 class TaskService(CommonService):
     """Service class for managing document processing tasks.
-    
+
     This class extends CommonService to provide specialized functionality for document
     processing task management, including task creation, progress tracking, and chunk
     management. It handles various document types (PDF, Excel, etc.) and manages their
     processing lifecycle.
-    
+
     The class implements a robust task queue system with retry mechanisms and progress
     tracking, supporting both synchronous and asynchronous task execution.
-    
+
     Attributes:
         model: The Task model class for database operations.
     """
@@ -72,14 +72,14 @@ class TaskService(CommonService):
     @DB.connection_context()
     def get_task(cls, task_id):
         """Retrieve detailed task information by task ID.
-    
+
         This method fetches comprehensive task details including associated document,
         knowledge base, and tenant information. It also handles task retry logic and
         progress updates.
-    
+
         Args:
             task_id (str): The unique identifier of the task to retrieve.
-    
+
         Returns:
             dict: Task details dictionary containing all task information and related metadata.
                  Returns None if task is not found or has exceeded retry limit.
@@ -139,13 +139,13 @@ class TaskService(CommonService):
     @DB.connection_context()
     def get_tasks(cls, doc_id: str):
         """Retrieve all tasks associated with a document.
-    
+
         This method fetches all processing tasks for a given document, ordered by page
         number and creation time. It includes task progress and chunk information.
-    
+
         Args:
             doc_id (str): The unique identifier of the document.
-    
+
         Returns:
             list[dict]: List of task dictionaries containing task details.
                        Returns None if no tasks are found.
@@ -170,10 +170,10 @@ class TaskService(CommonService):
     @DB.connection_context()
     def update_chunk_ids(cls, id: str, chunk_ids: str):
         """Update the chunk IDs associated with a task.
-    
+
         This method updates the chunk_ids field of a task, which stores the IDs of
         processed document chunks in a space-separated string format.
-    
+
         Args:
             id (str): The unique identifier of the task.
             chunk_ids (str): Space-separated string of chunk identifiers.
@@ -184,11 +184,11 @@ class TaskService(CommonService):
     @DB.connection_context()
     def get_ongoing_doc_name(cls):
         """Get names of documents that are currently being processed.
-    
+
         This method retrieves information about documents that are in the processing state,
         including their locations and associated IDs. It uses database locking to ensure
         thread safety when accessing the task information.
-    
+
         Returns:
             list[tuple]: A list of tuples, each containing (parent_id/kb_id, location)
                         for documents currently being processed. Returns empty list if
@@ -238,14 +238,14 @@ class TaskService(CommonService):
     @DB.connection_context()
     def do_cancel(cls, id):
         """Check if a task should be cancelled based on its document status.
-    
+
         This method determines whether a task should be cancelled by checking the
         associated document's run status and progress. A task should be cancelled
         if its document is marked for cancellation or has negative progress.
-    
+
         Args:
             id (str): The unique identifier of the task to check.
-    
+
         Returns:
             bool: True if the task should be cancelled, False otherwise.
         """
@@ -311,18 +311,18 @@ class TaskService(CommonService):
 
 def queue_tasks(doc: dict, bucket: str, name: str, priority: int):
     """Create and queue document processing tasks.
-    
+
     This function creates processing tasks for a document based on its type and configuration.
     It handles different document types (PDF, Excel, etc.) differently and manages task
     chunking and configuration. It also implements task reuse optimization by checking
     for previously completed tasks.
-    
+
     Args:
         doc (dict): Document dictionary containing metadata and configuration.
         bucket (str): Storage bucket name where the document is stored.
         name (str): File name of the document.
         priority (int, optional): Priority level for task queueing (default is 0).
-    
+
     Note:
         - For PDF documents, tasks are created per page range based on configuration
         - For Excel documents, tasks are created per row range
@@ -410,19 +410,19 @@ def queue_tasks(doc: dict, bucket: str, name: str, priority: int):
 
 def reuse_prev_task_chunks(task: dict, prev_tasks: list[dict], chunking_config: dict):
     """Attempt to reuse chunks from previous tasks for optimization.
-    
+
     This function checks if chunks from previously completed tasks can be reused for
     the current task, which can significantly improve processing efficiency. It matches
     tasks based on page ranges and configuration digests.
-    
+
     Args:
         task (dict): Current task dictionary to potentially reuse chunks for.
         prev_tasks (list[dict]): List of previous task dictionaries to check for reuse.
         chunking_config (dict): Configuration dictionary for chunk processing.
-    
+
     Returns:
         int: Number of chunks successfully reused. Returns 0 if no chunks could be reused.
-    
+
     Note:
         Chunks can only be reused if:
         - A previous task exists with matching page range and configuration digest
@@ -470,3 +470,39 @@ def has_canceled(task_id):
     except Exception as e:
         logging.exception(e)
     return False
+
+
+def queue_dataflow(dsl:str, tenant_id:str, doc_id:str, task_id:str, flow_id:str, priority: int, callback=None) -> tuple[bool, str]:
+    """
+    Returns a tuple (success: bool, error_message: str).
+    """
+    _ = callback
+
+    task = dict(
+    id=get_uuid() if not task_id else task_id,
+    doc_id=doc_id,
+    from_page=0,
+    to_page=100000000,
+    task_type="dataflow",
+    priority=priority,
+    )
+
+    TaskService.model.delete().where(TaskService.model.id == task["id"]).execute()
+    bulk_insert_into_db(model=Task, data_source=[task], replace_on_conflict=True)
+
+    kb_id = DocumentService.get_knowledgebase_id(doc_id)
+    if not kb_id:
+        return False, f"Can't find KB of this document: {doc_id}"
+
+    task["kb_id"] = kb_id
+    task["tenant_id"] = tenant_id
+    task["task_type"] = "dataflow"
+    task["dsl"] = dsl
+    task["dataflow_id"] = get_uuid() if not flow_id else flow_id
+
+    if not REDIS_CONN.queue_product(
+        get_svr_queue_name(priority), message=task
+    ):
+        return False, "Can't access Redis. Please check the Redis' status."
+
+    return True, ""

api/db/services/user_service.py

@@ -133,6 +133,13 @@ class UserService(CommonService):
                 cls.model.update(user_dict).where(
                     cls.model.id == user_id).execute()
 
+    @classmethod
+    @DB.connection_context()
+    def is_admin(cls, user_id):
+        return cls.model.select().where(
+            cls.model.id == user_id,
+            cls.model.is_superuser == 1).count() > 0
+
 
 class TenantService(CommonService):
     """Service class for managing tenant-related database operations.

api/utils/api_utils.py

@@ -17,6 +17,7 @@ import asyncio
 import functools
 import json
 import logging
+import os
 import queue
 import random
 import threading
@@ -55,6 +56,30 @@ from rag.utils.mcp_tool_call_conn import MCPToolCallSession, close_multiple_mcp_
 
 requests.models.complexjson.dumps = functools.partial(json.dumps, cls=CustomJSONEncoder)
 
+def serialize_for_json(obj):
+    """
+    Recursively serialize objects to make them JSON serializable.
+    Handles ModelMetaclass and other non-serializable objects.
+    """
+    if hasattr(obj, '__dict__'):
+        # For objects with __dict__, try to serialize their attributes
+        try:
+            return {key: serialize_for_json(value) for key, value in obj.__dict__.items() 
+                   if not key.startswith('_')}
+        except (AttributeError, TypeError):
+            return str(obj)
+    elif hasattr(obj, '__name__'):
+        # For classes and metaclasses, return their name
+        return f"<{obj.__module__}.{obj.__name__}>" if hasattr(obj, '__module__') else f"<{obj.__name__}>"
+    elif isinstance(obj, (list, tuple)):
+        return [serialize_for_json(item) for item in obj]
+    elif isinstance(obj, dict):
+        return {key: serialize_for_json(value) for key, value in obj.items()}
+    elif isinstance(obj, (str, int, float, bool)) or obj is None:
+        return obj
+    else:
+        # Fallback: convert to string representation
+        return str(obj)
 
 def request(**kwargs):
     sess = requests.Session()
@@ -127,7 +152,11 @@ def server_error_response(e):
     except BaseException:
         pass
     if len(e.args) > 1:
-        return get_json_result(code=settings.RetCode.EXCEPTION_ERROR, message=repr(e.args[0]), data=e.args[1])
+        try:
+            serialized_data = serialize_for_json(e.args[1])
+            return get_json_result(code= settings.RetCode.EXCEPTION_ERROR, message=repr(e.args[0]), data=serialized_data)
+        except Exception:
+            return get_json_result(code=settings.RetCode.EXCEPTION_ERROR, message=repr(e.args[0]), data=None)
     if repr(e).find("index_not_found_exception") >= 0:
         return get_json_result(code=settings.RetCode.EXCEPTION_ERROR, message="No chunk found, please upload file and parse it.")
 
@@ -291,6 +320,8 @@ def construct_error_response(e):
 def token_required(func):
     @wraps(func)
     def decorated_function(*args, **kwargs):
+        if os.environ.get("DISABLE_SDK"):
+            return get_json_result(data=False, message="`Authorization` can't be empty")
         authorization_str = flask_request.headers.get("Authorization")
         if not authorization_str:
             return get_json_result(data=False, message="`Authorization` can't be empty")
@@ -353,7 +384,7 @@ def get_parser_config(chunk_method, parser_config):
     if not chunk_method:
         chunk_method = "naive"
 
-    # Define default configurations for each chunk method
+    # Define default configurations for each chunking method
     key_mapping = {
         "naive": {"chunk_token_num": 512, "delimiter": r"\n", "html4excel": False, "layout_recognize": "DeepDOC", "raptor": {"use_raptor": False}, "graphrag": {"use_graphrag": False}},
         "qa": {"raptor": {"use_raptor": False}, "graphrag": {"use_graphrag": False}},
@@ -667,7 +698,10 @@ def timeout(seconds: float | int = None, attempts: int = 2, *, exception: Option
 
             for a in range(attempts):
                 try:
-                    result = result_queue.get(timeout=seconds)
+                    if os.environ.get("ENABLE_TIMEOUT_ASSERTION"):
+                        result = result_queue.get(timeout=seconds)
+                    else:
+                        result = result_queue.get()
                     if isinstance(result, Exception):
                         raise result
                     return result
@@ -682,7 +716,10 @@ def timeout(seconds: float | int = None, attempts: int = 2, *, exception: Option
 
             for a in range(attempts):
                 try:
-                    with trio.fail_after(seconds):
+                    if os.environ.get("ENABLE_TIMEOUT_ASSERTION"):
+                        with trio.fail_after(seconds):
+                            return await func(*args, **kwargs)
+                    else:
                         return await func(*args, **kwargs)
                 except trio.TooSlowError:
                     if a < attempts - 1:

conf/llm_factories.json

@@ -302,6 +302,20 @@
                     "model_type": "chat",
                     "is_tools": true
                 },
+                {
+                    "llm_name": "qwen-plus-2025-07-28",
+                    "tags": "LLM,CHAT,132k",
+                    "max_tokens": 131072,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "qwen-plus-2025-07-14",
+                    "tags": "LLM,CHAT,132k",
+                    "max_tokens": 131072,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
                 {
                     "llm_name": "qwq-plus-latest",
                     "tags": "LLM,CHAT,132k",
@@ -309,6 +323,27 @@
                     "model_type": "chat",
                     "is_tools": true
                 },
+                {
+                    "llm_name": "qwen-flash",
+                    "tags": "LLM,CHAT,1M",
+                    "max_tokens": 1000000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "qwen-flash-2025-07-28",
+                    "tags": "LLM,CHAT,1M",
+                    "max_tokens": 1000000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "qwen3-max-preview",
+                    "tags": "LLM,CHAT,256k",
+                    "max_tokens": 256000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
                 {
                     "llm_name": "qwen3-coder-480b-a35b-instruct",
                     "tags": "LLM,CHAT,256k",
@@ -532,23 +567,65 @@
             "tags": "LLM,TEXT EMBEDDING,SPEECH2TEXT,MODERATION",
             "status": "1",
             "llm": [
+                {
+                    "llm_name": "glm-4.5",
+                    "tags": "LLM,CHAT,128K",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-x",
+                    "tags": "LLM,CHAT,128k",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-air",
+                    "tags": "LLM,CHAT,128K",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-airx",
+                    "tags": "LLM,CHAT,128k",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5-flash",
+                    "tags": "LLM,CHAT,128k",
+                    "max_tokens": 128000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "glm-4.5v",
+                    "tags": "LLM,IMAGE2TEXT,64,",
+                    "max_tokens": 64000,
+                    "model_type": "image2text",
+                    "is_tools": false
+                },
                 {
                     "llm_name": "glm-4-plus",
-                    "tags": "LLM,CHAT,",
+                    "tags": "LLM,CHAT,128K",
                     "max_tokens": 128000,
                     "model_type": "chat",
                     "is_tools": true
                 },
                 {
                     "llm_name": "glm-4-0520",
-                    "tags": "LLM,CHAT,",
+                    "tags": "LLM,CHAT,128K",
                     "max_tokens": 128000,
                     "model_type": "chat",
                     "is_tools": true
                 },
                 {
                     "llm_name": "glm-4",
-                    "tags": "LLM,CHAT,",
+                    "tags":"LLM,CHAT,128K",
                     "max_tokens": 128000,
                     "model_type": "chat",
                     "is_tools": true
@@ -678,6 +755,20 @@
                     "model_type": "chat",
                     "is_tools": true
                 },
+                {
+                    "llm_name": "kimi-k2-0905-preview",
+                    "tags": "LLM,CHAT,256k",
+                    "max_tokens": 262144,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "kimi-k2-turbo-preview",
+                    "tags": "LLM,CHAT,256k",
+                    "max_tokens": 262144,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
                 {
                     "llm_name": "kimi-latest",
                     "tags": "LLM,CHAT,8k,32k,128k",
@@ -2620,21 +2711,21 @@
             "status": "1",
             "llm": [
                 {
-                    "llm_name": "Qwen3-Embedding-8B",
+                    "llm_name": "Qwen/Qwen3-Embedding-8B",
                     "tags": "TEXT EMBEDDING,TEXT RE-RANK,32k",
                     "max_tokens": 32000,
                     "model_type": "embedding",
                     "is_tools": false
                 },
                 {
-                    "llm_name": "Qwen3-Embedding-4B",
+                    "llm_name": "Qwen/Qwen3-Embedding-4B",
                     "tags": "TEXT EMBEDDING,TEXT RE-RANK,32k",
                     "max_tokens": 32000,
                     "model_type": "embedding",
                     "is_tools": false
                 },
                 {
-                    "llm_name": "Qwen3-Embedding-0.6B",
+                    "llm_name": "Qwen/Qwen3-Embedding-0.6B",
                     "tags": "TEXT EMBEDDING,TEXT RE-RANK,32k",
                     "max_tokens": 32000,
                     "model_type": "embedding",
@@ -2717,6 +2808,20 @@
                     "model_type": "chat",
                     "is_tools": true
                 },
+                {
+                    "llm_name": "Pro/deepseek-ai/DeepSeek-V3.1",
+                    "tags": "LLM,CHAT,160k",
+                    "max_tokens": 160000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
+                {
+                    "llm_name": "deepseek-ai/DeepSeek-V3.1",
+                    "tags": "LLM,CHAT,160",
+                    "max_tokens": 160000,
+                    "model_type": "chat",
+                    "is_tools": true
+                },
                 {
                     "llm_name": "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
                     "tags": "LLM,CHAT,32k",
@@ -4371,6 +4476,21 @@
                     "is_tools": false
                 }
             ]
+        },
+        {
+            "name": "Meituan",
+            "logo": "",
+            "tags": "LLM",
+            "status": "1",
+            "llm": [
+                {
+                    "llm_name": "LongCat-Flash-Chat",
+                    "tags": "LLM,CHAT,8000",
+                    "max_tokens": 8000,
+                    "model_type": "chat",
+                    "is_tools": true
+                }
+            ]
         }
     ]
 }

deepdoc/parser/__init__.py

@@ -14,13 +14,15 @@
 #  limitations under the License.
 #
 
-from .pdf_parser import RAGFlowPdfParser as PdfParser, PlainParser
 from .docx_parser import RAGFlowDocxParser as DocxParser
 from .excel_parser import RAGFlowExcelParser as ExcelParser
-from .ppt_parser import RAGFlowPptParser as PptParser
 from .html_parser import RAGFlowHtmlParser as HtmlParser
 from .json_parser import RAGFlowJsonParser as JsonParser
+from .markdown_parser import MarkdownElementExtractor
 from .markdown_parser import RAGFlowMarkdownParser as MarkdownParser
+from .pdf_parser import PlainParser
+from .pdf_parser import RAGFlowPdfParser as PdfParser
+from .ppt_parser import RAGFlowPptParser as PptParser
 from .txt_parser import RAGFlowTxtParser as TxtParser
 
 __all__ = [
@@ -33,4 +35,6 @@ __all__ = [
     "JsonParser",
     "MarkdownParser",
     "TxtParser",
-]
+    "MarkdownElementExtractor",
+]
+

deepdoc/parser/excel_parser.py

@@ -124,13 +124,19 @@ class RAGFlowExcelParser:
                         if c.value is None:
                             tb += "<td></td>"
                         else:
-                            tb += f"<td>{c.value}</td>"
+                            tb += f"<td>{escape(_fmt(c.value))}</td>"
                     tb += "</tr>"
                 tb += "</table>\n"
                 tb_chunks.append(tb)
 
         return tb_chunks
 
+    def markdown(self, fnm):
+        import pandas as pd
+        file_like_object = BytesIO(fnm) if not isinstance(fnm, str) else fnm
+        df = pd.read_excel(file_like_object)
+        return df.to_markdown(index=False)
+
     def __call__(self, fnm):
         file_like_object = BytesIO(fnm) if not isinstance(fnm, str) else fnm
         wb = RAGFlowExcelParser._load_excel_to_workbook(file_like_object)

deepdoc/parser/html_parser.py

@@ -15,35 +15,200 @@
 #  limitations under the License.
 #
 
-from rag.nlp import find_codec
+from rag.nlp import find_codec, rag_tokenizer
-import readability
+import uuid
-import html_text
 import chardet
-
+from bs4 import BeautifulSoup, NavigableString, Tag, Comment
+import html
 
 def get_encoding(file):
     with open(file,'rb') as f:
         tmp = chardet.detect(f.read())
         return tmp['encoding']
 
+BLOCK_TAGS = [
+    "h1", "h2", "h3", "h4", "h5", "h6",
+    "p", "div", "article", "section", "aside",
+    "ul", "ol", "li",
+    "table", "pre", "code", "blockquote",
+    "figure", "figcaption"
+]
+TITLE_TAGS = {"h1": "#", "h2": "##", "h3": "###", "h4": "#####", "h5": "#####", "h6": "######"}
+
 
 class RAGFlowHtmlParser:
-    def __call__(self, fnm, binary=None):
+    def __call__(self, fnm, binary=None, chunk_token_num=None):
         if binary:
             encoding = find_codec(binary)
             txt = binary.decode(encoding, errors="ignore")
         else:
             with open(fnm, "r",encoding=get_encoding(fnm)) as f:
                 txt = f.read()
-        return self.parser_txt(txt)
+        return self.parser_txt(txt, chunk_token_num)
 
     @classmethod
-    def parser_txt(cls, txt):
+    def parser_txt(cls, txt, chunk_token_num):
         if not isinstance(txt, str):
             raise TypeError("txt type should be string!")
-        html_doc = readability.Document(txt)
+
-        title = html_doc.title()
+        temp_sections = []
-        content = html_text.extract_text(html_doc.summary(html_partial=True))
+        soup = BeautifulSoup(txt, "html5lib")
-        txt = f"{title}\n{content}"
+        # delete <style> tag
-        sections = txt.split("\n")
+        for style_tag in soup.find_all(["style", "script"]):
+            style_tag.decompose()
+        # delete <script> tag in <div>
+        for div_tag in soup.find_all("div"):
+            for script_tag in div_tag.find_all("script"):
+                script_tag.decompose()
+        # delete inline style
+        for tag in soup.find_all(True):
+            if 'style' in tag.attrs:
+                del tag.attrs['style']
+        # delete HTML comment
+        for comment in soup.find_all(string=lambda text: isinstance(text, Comment)):
+            comment.extract()
+
+        cls.read_text_recursively(soup.body, temp_sections, chunk_token_num=chunk_token_num)
+        block_txt_list, table_list = cls.merge_block_text(temp_sections)
+        sections = cls.chunk_block(block_txt_list, chunk_token_num=chunk_token_num)
+        for table in table_list:
+            sections.append(table.get("content", ""))
         return sections
+
+    @classmethod
+    def split_table(cls, html_table, chunk_token_num=512):
+        soup = BeautifulSoup(html_table, "html.parser")
+        rows = soup.find_all("tr")
+        tables = []
+        current_table = []
+        current_count = 0
+        table_str_list = []
+        for row in rows:
+            tks_str = rag_tokenizer.tokenize(str(row))
+            token_count = len(tks_str.split(" ")) if tks_str else 0
+            if current_count + token_count > chunk_token_num:
+                tables.append(current_table)
+                current_table = []
+                current_count = 0
+            current_table.append(row)
+            current_count += token_count
+        if current_table:
+            tables.append(current_table)
+
+        for table_rows in tables:
+            new_table = soup.new_tag("table")
+            for row in table_rows:
+                new_table.append(row)
+            table_str_list.append(str(new_table))
+
+        return table_str_list
+
+    @classmethod
+    def read_text_recursively(cls, element, parser_result, chunk_token_num=512, parent_name=None, block_id=None):
+        if isinstance(element, NavigableString):
+            content = element.strip()
+
+            def is_valid_html(content):
+                try:
+                    soup = BeautifulSoup(content, "html.parser")
+                    return bool(soup.find())
+                except Exception:
+                    return False
+
+            return_info = []
+            if content:
+                if is_valid_html(content):
+                    soup = BeautifulSoup(content, "html.parser")
+                    child_info = cls.read_text_recursively(soup, parser_result, chunk_token_num, element.name, block_id)
+                    parser_result.extend(child_info)
+                else:
+                    info = {"content": element.strip(), "tag_name": "inner_text", "metadata": {"block_id": block_id}}
+                    if parent_name:
+                        info["tag_name"] = parent_name
+                    return_info.append(info)
+            return return_info
+        elif isinstance(element, Tag):
+
+            if str.lower(element.name) == "table":
+                table_info_list = []
+                table_id = str(uuid.uuid1())
+                table_list = [html.unescape(str(element))]
+                for t in table_list:
+                    table_info_list.append({"content": t, "tag_name": "table",
+                                            "metadata": {"table_id": table_id, "index": table_list.index(t)}})
+                return table_info_list
+            else:
+                block_id = None
+                if str.lower(element.name) in BLOCK_TAGS:
+                    block_id = str(uuid.uuid1())
+                for child in element.children:
+                    child_info = cls.read_text_recursively(child, parser_result, chunk_token_num, element.name,
+                                                           block_id)
+                    parser_result.extend(child_info)
+        return []
+
+    @classmethod
+    def merge_block_text(cls, parser_result):
+        block_content = []
+        current_content = ""
+        table_info_list = []
+        lask_block_id = None
+        for item in parser_result:
+            content = item.get("content")
+            tag_name = item.get("tag_name")
+            title_flag = tag_name in TITLE_TAGS
+            block_id = item.get("metadata", {}).get("block_id")
+            if block_id:
+                if title_flag:
+                    content = f"{TITLE_TAGS[tag_name]} {content}"
+                if lask_block_id != block_id:
+                    if lask_block_id is not None:
+                        block_content.append(current_content)
+                    current_content = content
+                    lask_block_id = block_id
+                else:
+                    current_content += (" " if current_content else "") + content
+            else:
+                if tag_name == "table":
+                    table_info_list.append(item)
+                else:
+                    current_content += (" " if current_content else "" + content)
+        if current_content:
+            block_content.append(current_content)
+        return block_content, table_info_list
+
+    @classmethod
+    def chunk_block(cls, block_txt_list, chunk_token_num=512):
+        chunks = []
+        current_block = ""
+        current_token_count = 0
+
+        for block in block_txt_list:
+            tks_str = rag_tokenizer.tokenize(block)
+            block_token_count = len(tks_str.split(" ")) if tks_str else 0
+            if block_token_count > chunk_token_num:
+                if current_block:
+                    chunks.append(current_block)
+                start = 0
+                tokens = tks_str.split(" ")
+                while start < len(tokens):
+                    end = start + chunk_token_num
+                    split_tokens = tokens[start:end]
+                    chunks.append(" ".join(split_tokens))
+                    start = end
+                current_block = ""
+                current_token_count = 0
+            else:
+                if current_token_count + block_token_count <= chunk_token_num:
+                    current_block += ("\n" if current_block else "") + block
+                    current_token_count += block_token_count
+                else:
+                    chunks.append(current_block)
+                    current_block = block
+                    current_token_count = block_token_count
+
+        if current_block:
+            chunks.append(current_block)
+
+        return chunks
+

deepdoc/parser/markdown_parser.py

@@ -17,8 +17,10 @@
 
 import re
 
+import mistune
 from markdown import markdown
 
+
 class RAGFlowMarkdownParser:
     def __init__(self, chunk_token_num=128):
         self.chunk_token_num = int(chunk_token_num)
@@ -35,40 +37,44 @@ class RAGFlowMarkdownParser:
                 table_list.append(raw_table)
                 if separate_tables:
                     # Skip this match (i.e., remove it)
-                    new_text += working_text[last_end:match.start()] + "\n\n"
+                    new_text += working_text[last_end : match.start()] + "\n\n"
                 else:
                     # Replace with rendered HTML
-                    html_table = markdown(raw_table, extensions=['markdown.extensions.tables']) if render else raw_table
+                    html_table = markdown(raw_table, extensions=["markdown.extensions.tables"]) if render else raw_table
-                    new_text += working_text[last_end:match.start()] + html_table + "\n\n"
+                    new_text += working_text[last_end : match.start()] + html_table + "\n\n"
                 last_end = match.end()
             new_text += working_text[last_end:]
             return new_text
 
-        if "|" in markdown_text: # for optimize performance
+        if "|" in markdown_text:  # for optimize performance
             # Standard Markdown table
             border_table_pattern = re.compile(
-                r'''
+                r"""
                 (?:\n|^)
                 (?:\|.*?\|.*?\|.*?\n)
                 (?:\|(?:\s*[:-]+[-| :]*\s*)\|.*?\n)
                 (?:\|.*?\|.*?\|.*?\n)+
-            ''', re.VERBOSE)
+            """,
+                re.VERBOSE,
+            )
             working_text = replace_tables_with_rendered_html(border_table_pattern, tables)
 
             # Borderless Markdown table
             no_border_table_pattern = re.compile(
-                r'''
+                r"""
                 (?:\n|^)
                 (?:\S.*?\|.*?\n)
                 (?:(?:\s*[:-]+[-| :]*\s*).*?\n)
                 (?:\S.*?\|.*?\n)+
-                ''', re.VERBOSE)
+                """,
+                re.VERBOSE,
+            )
             working_text = replace_tables_with_rendered_html(no_border_table_pattern, tables)
 
-        if "<table>" in working_text.lower(): # for optimize performance
+        if "<table>" in working_text.lower():  # for optimize performance
-            #HTML table extraction - handle possible html/body wrapper tags
+            # HTML table extraction - handle possible html/body wrapper tags
             html_table_pattern = re.compile(
-            r'''
+                r"""
             (?:\n|^)
             \s*
             (?:
@@ -83,9 +89,10 @@ class RAGFlowMarkdownParser:
             )
             \s*
             (?=\n|$)
-            ''',
+            """,
-            re.VERBOSE | re.DOTALL | re.IGNORECASE
+                re.VERBOSE | re.DOTALL | re.IGNORECASE,
             )
+
             def replace_html_tables():
                 nonlocal working_text
                 new_text = ""
@@ -94,9 +101,9 @@ class RAGFlowMarkdownParser:
                     raw_table = match.group()
                     tables.append(raw_table)
                     if separate_tables:
-                        new_text += working_text[last_end:match.start()] + "\n\n"
+                        new_text += working_text[last_end : match.start()] + "\n\n"
                     else:
-                        new_text += working_text[last_end:match.start()] + raw_table + "\n\n"
+                        new_text += working_text[last_end : match.start()] + raw_table + "\n\n"
                     last_end = match.end()
                 new_text += working_text[last_end:]
                 working_text = new_text
@@ -104,3 +111,163 @@ class RAGFlowMarkdownParser:
             replace_html_tables()
 
         return working_text, tables
+
+
+class MarkdownElementExtractor:
+    def __init__(self, markdown_content):
+        self.markdown_content = markdown_content
+        self.lines = markdown_content.split("\n")
+        self.ast_parser = mistune.create_markdown(renderer="ast")
+        self.ast_nodes = self.ast_parser(markdown_content)
+
+    def extract_elements(self):
+        """Extract individual elements (headers, code blocks, lists, etc.)"""
+        sections = []
+
+        i = 0
+        while i < len(self.lines):
+            line = self.lines[i]
+
+            if re.match(r"^#{1,6}\s+.*$", line):
+                # header
+                element = self._extract_header(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif line.strip().startswith("```"):
+                # code block
+                element = self._extract_code_block(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif re.match(r"^\s*[-*+]\s+.*$", line) or re.match(r"^\s*\d+\.\s+.*$", line):
+                # list block
+                element = self._extract_list_block(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif line.strip().startswith(">"):
+                # blockquote
+                element = self._extract_blockquote(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            elif line.strip():
+                # text block (paragraphs and inline elements until next block element)
+                element = self._extract_text_block(i)
+                sections.append(element["content"])
+                i = element["end_line"] + 1
+            else:
+                i += 1
+
+        sections = [section for section in sections if section.strip()]
+        return sections
+
+    def _extract_header(self, start_pos):
+        return {
+            "type": "header",
+            "content": self.lines[start_pos],
+            "start_line": start_pos,
+            "end_line": start_pos,
+        }
+
+    def _extract_code_block(self, start_pos):
+        end_pos = start_pos
+        content_lines = [self.lines[start_pos]]
+
+        # Find the end of the code block
+        for i in range(start_pos + 1, len(self.lines)):
+            content_lines.append(self.lines[i])
+            end_pos = i
+            if self.lines[i].strip().startswith("```"):
+                break
+
+        return {
+            "type": "code_block",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }
+
+    def _extract_list_block(self, start_pos):
+        end_pos = start_pos
+        content_lines = []
+
+        i = start_pos
+        while i < len(self.lines):
+            line = self.lines[i]
+            # check if this line is a list item or continuation of a list
+            if (
+                re.match(r"^\s*[-*+]\s+.*$", line)
+                or re.match(r"^\s*\d+\.\s+.*$", line)
+                or (i > start_pos and not line.strip())
+                or (i > start_pos and re.match(r"^\s{2,}[-*+]\s+.*$", line))
+                or (i > start_pos and re.match(r"^\s{2,}\d+\.\s+.*$", line))
+                or (i > start_pos and re.match(r"^\s+\w+.*$", line))
+            ):
+                content_lines.append(line)
+                end_pos = i
+                i += 1
+            else:
+                break
+
+        return {
+            "type": "list_block",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }
+
+    def _extract_blockquote(self, start_pos):
+        end_pos = start_pos
+        content_lines = []
+
+        i = start_pos
+        while i < len(self.lines):
+            line = self.lines[i]
+            if line.strip().startswith(">") or (i > start_pos and not line.strip()):
+                content_lines.append(line)
+                end_pos = i
+                i += 1
+            else:
+                break
+
+        return {
+            "type": "blockquote",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }
+
+    def _extract_text_block(self, start_pos):
+        """Extract a text block (paragraphs, inline elements) until next block element"""
+        end_pos = start_pos
+        content_lines = [self.lines[start_pos]]
+
+        i = start_pos + 1
+        while i < len(self.lines):
+            line = self.lines[i]
+            # stop if we encounter a block element
+            if re.match(r"^#{1,6}\s+.*$", line) or line.strip().startswith("```") or re.match(r"^\s*[-*+]\s+.*$", line) or re.match(r"^\s*\d+\.\s+.*$", line) or line.strip().startswith(">"):
+                break
+            elif not line.strip():
+                # check if the next line is a block element
+                if i + 1 < len(self.lines) and (
+                    re.match(r"^#{1,6}\s+.*$", self.lines[i + 1])
+                    or self.lines[i + 1].strip().startswith("```")
+                    or re.match(r"^\s*[-*+]\s+.*$", self.lines[i + 1])
+                    or re.match(r"^\s*\d+\.\s+.*$", self.lines[i + 1])
+                    or self.lines[i + 1].strip().startswith(">")
+                ):
+                    break
+                else:
+                    content_lines.append(line)
+                    end_pos = i
+                    i += 1
+            else:
+                content_lines.append(line)
+                end_pos = i
+                i += 1
+
+        return {
+            "type": "text_block",
+            "content": "\n".join(content_lines),
+            "start_line": start_pos,
+            "end_line": end_pos,
+        }

deepdoc/parser/pdf_parser.py

@@ -93,6 +93,7 @@ class RAGFlowPdfParser:
                 model_dir, "updown_concat_xgb.model"))
 
         self.page_from = 0
+        self.column_num = 1
 
     def __char_width(self, c):
         return (c["x1"] - c["x0"]) // max(len(c["text"]), 1)
@@ -427,10 +428,18 @@ class RAGFlowPdfParser:
             i += 1
         self.boxes = bxs
 
-    def _naive_vertical_merge(self):
+    def _naive_vertical_merge(self, zoomin=3):
         bxs = Recognizer.sort_Y_firstly(
             self.boxes, np.median(
                 self.mean_height) / 3)
+
+        column_width = np.median([b["x1"] - b["x0"] for b in self.boxes])
+        self.column_num = int(self.page_images[0].size[0] / zoomin / column_width)
+        if column_width < self.page_images[0].size[0] / zoomin / self.column_num:
+            logging.info("Multi-column................... {} {}".format(column_width,
+                  self.page_images[0].size[0] / zoomin / self.column_num))
+            self.boxes = self.sort_X_by_page(self.boxes, column_width / self.column_num)
+
         i = 0
         while i + 1 < len(bxs):
             b = bxs[i]
@@ -1139,20 +1148,94 @@ class RAGFlowPdfParser:
             need_image, zoomin, return_html, False)
         return self.__filterout_scraps(deepcopy(self.boxes), zoomin), tbls
 
+    def parse_into_bboxes(self, fnm, callback=None, zoomin=3):
+        start = timer()
+        self.__images__(fnm, zoomin)
+        if callback:
+            callback(0.40, "OCR finished ({:.2f}s)".format(timer() - start))
+
+        start = timer()
+        self._layouts_rec(zoomin)
+        if callback:
+            callback(0.63, "Layout analysis ({:.2f}s)".format(timer() - start))
+
+        start = timer()
+        self._table_transformer_job(zoomin)
+        if callback:
+            callback(0.83, "Table analysis ({:.2f}s)".format(timer() - start))
+
+        start = timer()
+        self._text_merge()
+        self._concat_downward()
+        self._naive_vertical_merge(zoomin)
+        if callback:
+            callback(0.92, "Text merged ({:.2f}s)".format(timer() - start))
+
+        start = timer()
+        tbls, figs = self._extract_table_figure(True, zoomin, True, True, True)
+
+        def insert_table_figures(tbls_or_figs, layout_type):
+            def min_rectangle_distance(rect1, rect2):
+                import math
+                pn1, left1, right1, top1, bottom1 = rect1
+                pn2, left2, right2, top2, bottom2 = rect2
+                if (right1 >= left2 and right2 >= left1 and
+                        bottom1 >= top2 and bottom2 >= top1):
+                    return 0 + (pn1-pn2)*10000
+                if right1 < left2:
+                    dx = left2 - right1
+                elif right2 < left1:
+                    dx = left1 - right2
+                else:
+                    dx = 0
+                if bottom1 < top2:
+                    dy = top2 - bottom1
+                elif bottom2 < top1:
+                    dy = top1 - bottom2
+                else:
+                    dy = 0
+                return math.sqrt(dx*dx + dy*dy) + (pn1-pn2)*10000
+
+            for (img, txt), poss in tbls_or_figs:
+                bboxes = [(i, (b["page_number"], b["x0"], b["x1"], b["top"], b["bottom"])) for i, b in enumerate(self.boxes)]
+                dists = [(min_rectangle_distance((pn, left, right, top, bott), rect),i) for i, rect in bboxes for pn, left, right, top, bott in poss]
+                min_i = np.argmin(dists, axis=0)[0]
+                min_i, rect = bboxes[dists[min_i][-1]]
+                if isinstance(txt, list):
+                    txt = "\n".join(txt)
+                self.boxes.insert(min_i, {
+                    "page_number": rect[0], "x0": rect[1], "x1": rect[2], "top": rect[3], "bottom": rect[4], "layout_type": layout_type, "text": txt, "image": img
+                })
+
+        for b in self.boxes:
+            b["position_tag"] = self._line_tag(b, zoomin)
+            b["image"] = self.crop(b["position_tag"], zoomin)
+
+        insert_table_figures(tbls, "table")
+        insert_table_figures(figs, "figure")
+        if callback:
+            callback(1, "Structured ({:.2f}s)".format(timer() - start))
+        return deepcopy(self.boxes)
+
     @staticmethod
     def remove_tag(txt):
         return re.sub(r"@@[\t0-9.-]+?##", "", txt)
 
-    def crop(self, text, ZM=3, need_position=False):
+    @staticmethod
-        imgs = []
+    def extract_positions(txt):
         poss = []
-        for tag in re.findall(r"@@[0-9-]+\t[0-9.\t]+##", text):
+        for tag in re.findall(r"@@[0-9-]+\t[0-9.\t]+##", txt):
             pn, left, right, top, bottom = tag.strip(
                 "#").strip("@").split("\t")
             left, right, top, bottom = float(left), float(
                 right), float(top), float(bottom)
             poss.append(([int(p) - 1 for p in pn.split("-")],
                          left, right, top, bottom))
+        return poss
+
+    def crop(self, text, ZM=3, need_position=False):
+        imgs = []
+        poss = self.extract_positions(text)
         if not poss:
             if need_position:
                 return None, None
@@ -1296,8 +1379,8 @@ class VisionParser(RAGFlowPdfParser):
 
     def __call__(self, filename, from_page=0, to_page=100000, **kwargs):
         callback = kwargs.get("callback", lambda prog, msg: None)
-
+        zoomin = kwargs.get("zoomin", 3)
-        self.__images__(fnm=filename, zoomin=3, page_from=from_page, page_to=to_page, **kwargs)
+        self.__images__(fnm=filename, zoomin=zoomin, page_from=from_page, page_to=to_page, callback=callback)
 
         total_pdf_pages = self.total_page
 
@@ -1311,16 +1394,19 @@ class VisionParser(RAGFlowPdfParser):
             if pdf_page_num < start_page or pdf_page_num >= end_page:
                 continue
 
-            docs = picture_vision_llm_chunk(
+            text = picture_vision_llm_chunk(
                 binary=img_binary,
                 vision_model=self.vision_model,
                 prompt=vision_llm_describe_prompt(page=pdf_page_num+1),
                 callback=callback,
             )
+            if kwargs.get("callback"):
+                kwargs["callback"](idx*1./len(self.page_images), f"Processed: {idx+1}/{len(self.page_images)}")
 
-            if docs:
+            if text:
-                all_docs.append(docs)
+                width, height = self.page_images[idx].size
-        return [(doc, "") for doc in all_docs], []
+                all_docs.append((text, f"{pdf_page_num+1} 0 {width/zoomin} 0 {height/zoomin}"))
+        return all_docs, []
 
 
 if __name__ == "__main__":

deepdoc/vision/seeit.py

@@ -31,11 +31,11 @@ def save_results(image_list, results, labels, output_dir='output/', threshold=0.
         logging.debug("save result to: " + out_path)
 
 
-def draw_box(im, result, lables, threshold=0.5):
+def draw_box(im, result, labels, threshold=0.5):
     draw_thickness = min(im.size) // 320
     draw = ImageDraw.Draw(im)
-    color_list = get_color_map_list(len(lables))
+    color_list = get_color_map_list(len(labels))
-    clsid2color = {n.lower():color_list[i] for i,n in enumerate(lables)}
+    clsid2color = {n.lower():color_list[i] for i,n in enumerate(labels)}
     result = [r for r in result if r["score"] >= threshold]
 
     for dt in result:

docker/.env

@@ -93,13 +93,13 @@ REDIS_PASSWORD=infini_rag_flow
 SVR_HTTP_PORT=9380
 
 # The RAGFlow Docker image to download.
-# Defaults to the v0.20.1-slim edition, which is the RAGFlow Docker image without embedding models.
+# Defaults to the v0.20.5-slim edition, which is the RAGFlow Docker image without embedding models.
-RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3-slim
+RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5-slim
 #
 # To download the RAGFlow Docker image with embedding models, uncomment the following line instead:
-# RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.1
+# RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5
 #
-# The Docker image of the v0.20.1 edition includes built-in embedding models:
+# The Docker image of the v0.20.5 edition includes built-in embedding models:
 #   - BAAI/bge-large-zh-v1.5
 #   - maidalun1020/bce-embedding-base_v1
 #
@@ -115,7 +115,7 @@ RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3-slim
 # RAGFLOW_IMAGE=registry.cn-hangzhou.aliyuncs.com/infiniflow/ragflow:nightly
 
 # The local time zone.
-TIMEZONE='Asia/Shanghai'
+TIMEZONE=Asia/Shanghai
 
 # Uncomment the following line if you have limited access to huggingface.co:
 # HF_ENDPOINT=https://hf-mirror.com

docker/README.md

@@ -79,8 +79,8 @@ The [.env](./.env) file contains important environment variables for Docker.
 - `RAGFLOW-IMAGE`  
   The Docker image edition. Available editions:  
   
-  - `infiniflow/ragflow:v0.20.3-slim` (default): The RAGFlow Docker image without embedding models.  
+  - `infiniflow/ragflow:v0.20.5-slim` (default): The RAGFlow Docker image without embedding models.  
-  - `infiniflow/ragflow:v0.20.3`: The RAGFlow Docker image with embedding models including:
+  - `infiniflow/ragflow:v0.20.5`: The RAGFlow Docker image with embedding models including:
     - Built-in embedding models:
       - `BAAI/bge-large-zh-v1.5` 
       - `maidalun1020/bce-embedding-base_v1`

docs/configurations.md

@@ -99,8 +99,8 @@ RAGFlow utilizes MinIO as its object storage solution, leveraging its scalabilit
 - `RAGFLOW-IMAGE`  
   The Docker image edition. Available editions:  
   
-  - `infiniflow/ragflow:v0.20.3-slim` (default): The RAGFlow Docker image without embedding models.  
+  - `infiniflow/ragflow:v0.20.5-slim` (default): The RAGFlow Docker image without embedding models.  
-  - `infiniflow/ragflow:v0.20.3`: The RAGFlow Docker image with embedding models including:
+  - `infiniflow/ragflow:v0.20.5`: The RAGFlow Docker image with embedding models including:
     - Built-in embedding models:
       - `BAAI/bge-large-zh-v1.5` 
       - `maidalun1020/bce-embedding-base_v1`

docs/develop/acquire_ragflow_api_key.md

@@ -11,7 +11,7 @@ An API key is required for the RAGFlow server to authenticate your HTTP/Python o
 2. Click **API** to switch to the **API** page.
 3. Obtain a RAGFlow API key:
 
-![ragflow_api_key](https://github.com/user-attachments/assets/f461ed61-04c6-4faf-b3d8-6b5fa56be4e7)
+![ragflow_api_key](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/ragflow_api_key.jpg)
 
 :::tip NOTE
 See the [RAGFlow HTTP API reference](../references/http_api_reference.md) or the [RAGFlow Python API reference](../references/python_api_reference.md) for a complete reference of RAGFlow's HTTP or Python APIs.

docs/develop/build_docker_image.mdx

@@ -77,7 +77,7 @@ After building the infiniflow/ragflow:nightly-slim image, you are ready to launc
 
 1. Edit Docker Compose Configuration
 
-Open the `docker/.env` file. Find the `RAGFLOW_IMAGE` setting and change the image reference from `infiniflow/ragflow:v0.20.3-slim` to `infiniflow/ragflow:nightly-slim` to use the pre-built image.
+Open the `docker/.env` file. Find the `RAGFLOW_IMAGE` setting and change the image reference from `infiniflow/ragflow:v0.20.5-slim` to `infiniflow/ragflow:nightly-slim` to use the pre-built image.
 
 
 2. Launch the Service

docs/faq.mdx

@@ -30,17 +30,17 @@ The "garbage in garbage out" status quo remains unchanged despite the fact that
 
 Each RAGFlow release is available in two editions:
 
-- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.3-slim`
+- **Slim edition**: excludes built-in embedding models and is identified by a **-slim** suffix added to the version name. Example: `infiniflow/ragflow:v0.20.5-slim`
-- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.3`
+- **Full edition**: includes built-in embedding models and has no suffix added to the version name. Example: `infiniflow/ragflow:v0.20.5`
 
 ---
 
 ### Which embedding models can be deployed locally?
 
-RAGFlow offers two Docker image editions, `v0.20.3-slim` and `v0.20.3`:  
+RAGFlow offers two Docker image editions, `v0.20.5-slim` and `v0.20.5`:  
   
-- `infiniflow/ragflow:v0.20.3-slim` (default): The RAGFlow Docker image without embedding models.  
+- `infiniflow/ragflow:v0.20.5-slim` (default): The RAGFlow Docker image without embedding models.  
-- `infiniflow/ragflow:v0.20.3`: The RAGFlow Docker image with embedding models including:
+- `infiniflow/ragflow:v0.20.5`: The RAGFlow Docker image with embedding models including:
   - Built-in embedding models:
     - `BAAI/bge-large-zh-v1.5`
     - `maidalun1020/bce-embedding-base_v1`

docs/guides/agent/agent_component_reference/agent.mdx

@@ -9,7 +9,7 @@ The component equipped with reasoning, tool usage, and multi-agent collaboration
 
 ---
 
-An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.3 onwards, an **Agent** component is able to work independently and with the following capabilities:
+An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.5 onwards, an **Agent** component is able to work independently and with the following capabilities:
 
 - Autonomous reasoning with reflection and adjustment based on environmental feedback.
 - Use of tools or subagents to complete tasks.
@@ -18,6 +18,14 @@ An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.3 onwa
 
 An **Agent** component is essential when you need the LLM to assist with summarizing, translating, or controlling various tasks. 
 
+## Prerequisites
+
+1. Ensure you have a chat model properly configured:
+
+![Set default models](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/set_default_models.jpg)
+
+2. If your Agent involves dataset retrieval, ensure you [have properly configured your target knowledge base(s)](../../dataset/configure_knowledge_base.md).
+
 ## Configurations
 
 ### Model
@@ -57,13 +65,44 @@ Click the dropdown menu of **Model** to show the model configuration window.
 
 Typically, you use the system prompt to describe the task for the LLM, specify how it should respond, and outline other miscellaneous requirements. We do not plan to elaborate on this topic, as it can be as extensive as prompt engineering. However, please be aware that the system prompt is often used in conjunction with keys (variables), which serve as various data inputs for the LLM. 
 
-:::danger IMPORTANT
 An **Agent** component relies on keys (variables) to specify its data inputs. Its immediate upstream component is *not* necessarily its data input, and the arrows in the workflow indicate *only* the processing sequence. Keys in a **Agent** component are used in conjunction with the system prompt to specify data inputs for the LLM. Use a forward slash `/` or the **(x)** button to show the keys to use.
-:::
+
+#### Advanced usage
+
+From v0.20.5 onwards, four framework-level prompt blocks are available in the **System prompt** field. Type `/` or click **(x)** to view them; they appear under the **Framework** entry in the dropdown menu.
+
+- `task_analysis` prompt block
+  - This block is responsible for analyzing tasks — either a user task or a task assigned by the lead Agent when the **Agent** component is acting as a Sub-Agent.
+  - Reference design: [analyze_task_system.md](https://github.com/infiniflow/ragflow/blob/main/rag/prompts/analyze_task_system.md) and [analyze_task_user.md](https://github.com/infiniflow/ragflow/blob/main/rag/prompts/analyze_task_user.md)
+  - Available *only* when this **Agent** component is acting as a planner, with either tools or sub-Agents under it.
+  - Input variables:
+    - `agent_prompt`: The system prompt.
+    - `task`: The user prompt for either a lead Agent or a sub-Agent. The lead Agent's user prompt is defined by the user, while a sub-Agent's user prompt is defined by the lead Agent when delegating tasks.
+    - `tool_desc`: A description of the tools and sub_Agents that can be called.
+    - `context`: The operational context, which stores interactions between the Agent, tools, and sub-agents; initially empty.
+- `plan_generation` prompt block
+  - This block creates a plan for the **Agent** component to execute next, based on the task analysis results.
+  - Reference design: [next_step.md](https://github.com/infiniflow/ragflow/blob/main/rag/prompts/next_step.md)
+  - Available *only* when this **Agent** component is acting as a planner, with either tools or sub-Agents under it.
+  - Input variables:
+    - `task_analysis`: The analysis result of the current task.
+    - `desc`: A description of the tools or sub-Agents currently being called.
+    - `today`: The date of today.
+- `reflection` prompt block
+  - This block enables the **Agent** component to reflect, improving task accuracy and efficiency.
+  - Reference design: [reflect.md](https://github.com/infiniflow/ragflow/blob/main/rag/prompts/reflect.md)
+  - Available *only* when this **Agent** component is acting as a planner, with either tools or sub-Agents under it.
+  - Input variables:
+    - `goal`: The goal of the current task. It is the user prompt for either a lead Agent or a sub-Agent. The lead Agent's user prompt is defined by the user, while a sub-Agent's user prompt is defined by the lead Agent.
+    - `tool_calls`: The history of tool calling
+    - `call.name`：The name of the tool called.
+    - `call.result`：The result of tool calling
+- `citation_guidelines` prompt block
+  - Reference design: [citation_prompt.md](https://github.com/infiniflow/ragflow/blob/main/rag/prompts/citation_prompt.md)
 
 ### User prompt
 
-The user-defined prompt. Defaults to `sys.query`, the user query.
+The user-defined prompt. Defaults to `sys.query`, the user query. As a general rule, when using the **Agent** component as a standalone module (not as a planner), you usually need to specify the corresponding **Retrieval** component’s output variable (`formalized_content`) here as part of the input to the LLM.
 
 
 ### Tools 
@@ -100,4 +139,24 @@ Increasing this value will significantly extend your agent's response time.
 
 ### Output
 
-The global variable name for the output of the **Agent** component, which can be referenced by other components in the workflow.
+The global variable name for the output of the **Agent** component, which can be referenced by other components in the workflow.
+
+## Frequently asked questions
+
+### Why does it take so long for my Agent to respond?
+
+An Agent’s response time generally depends on two key factors: the LLM’s capabilities and the prompt, the latter reflecting task complexity. When using an Agent, you should always balance task demands with the LLM’s ability. See [How to balance task complexity with an Agent's performance and speed?](#how-to-balance-task-complexity-with-an-agents-performance-and-speed) for details.
+
+## Best practices
+
+### How to balance task complexity with an Agent’s performance and speed?
+
+- For simple tasks, such as retrieval, rewriting, formatting, or structured data extraction, use concise prompts, remove planning or reasoning instructions, enforce output length limits, and select smaller or Turbo-class models. This significantly reduces latency and cost with minimal impact on quality.
+
+- For complex tasks, like multi-step reasoning, cross-document synthesis, or tool-based workflows, maintain or enhance prompts that include planning, reflection, and verification steps.
+
+- In multi-Agent orchestration systems, delegate simple subtasks to sub-Agents using smaller, faster models, and reserve more powerful models for the lead Agent to handle complexity and uncertainty.
+
+:::tip KEY INSIGHT
+Focus on minimizing output tokens — through summarization, bullet points, or explicit length limits — as this has far greater impact on reducing latency than optimizing input size.
+:::

docs/guides/agent/agent_component_reference/code.mdx

@@ -13,6 +13,32 @@ A component that enables users to integrate Python or JavaScript codes into thei
 
 A **Code** component is essential when you need to integrate complex code logic (Python or JavaScript) into your Agent for dynamic data processing.
 
+## Prerequisites
+
+### 1. Ensure gVisor is properly installed
+
+We use gVisor to isolate code execution from the host system. Please follow [the official installation guide](https://gvisor.dev/docs/user_guide/install/) to install gVisor, ensuring your operating system is compatible before proceeding.
+
+### 2. Ensure Sandbox is properly installed
+
+RAGFlow Sandbox is a secure, pluggable code execution backend. It serves as the code executor for the **Code** component. Please follow the [instructions here](https://github.com/infiniflow/ragflow/tree/main/sandbox) to install RAGFlow Sandbox.
+
+:::tip NOTE
+If your RAGFlow Sandbox is not working, please be sure to consult the [Troubleshooting](#troubleshooting) section in this document. We assure you that it addresses 99.99% of the issues!
+:::
+
+### 3. (Optional) Install necessary dependencies
+
+If you need to import your own Python or JavaScript packages into Sandbox, please follow the commands provided in the [How to import my own Python or JavaScript packages into Sandbox?](#how-to-import-my-own-python-or-javascript-packages-into-sandbox) section to install the additional dependencies. 
+
+### 4. Enable Sandbox-specific settings in RAGFlow
+
+Ensure all Sandbox-specific settings are enabled in **ragflow/docker/.env**.
+
+### 5. Restart the service after making changes
+
+Any changes to the configuration or environment *require* a full service restart to take effect.
+
 ## Configurations 
 
 ### Input
@@ -55,4 +81,112 @@ You define the output variable(s) of the **Code** component here.
 
 The defined output variable(s) will be auto-populated here.
 
+## Troubleshooting
 
+### `HTTPConnectionPool(host='sandbox-executor-manager', port=9385): Read timed out.`
+
+**Root cause**  
+
+- You did not properly install gVisor and `runsc` was not recognized as a valid Docker runtime.
+- You did not pull the required base images for the runners and no runner was started.
+
+**Solution**
+
+For the gVisor issue:
+
+1. Install [gVisor](https://gvisor.dev/docs/user_guide/install/).
+2. Restart Docker.
+3. Run the following to double check:
+
+   ```bash
+   docker run --rm --runtime=runsc hello-world
+   ```
+
+For the base image issue, pull the required base images:
+
+```bash
+docker pull infiniflow/sandbox-base-nodejs:latest
+docker pull infiniflow/sandbox-base-python:latest
+```
+
+### `HTTPConnectionPool(host='none', port=9385): Max retries exceeded.`
+
+**Root cause**  
+
+`sandbox-executor-manager` is not mapped in `/etc/hosts`.
+
+**Solution**  
+
+Add a new entry to `/etc/hosts`:
+
+`127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager`
+
+### `Container pool is busy`
+
+**Root cause**  
+
+All runners are currently in use, executing tasks. 
+
+**Solution**  
+
+Please try again shortly or increase the pool size in the configuration to improve availability and reduce waiting times.
+
+
+## Frequently asked questions
+
+### How to import my own Python or JavaScript packages into Sandbox?
+
+To import your Python packages, update **sandbox_base_image/python/requirements.txt** to install the required dependencies. For example, to add the `openpyxl` package, proceed with the following command lines:
+
+```bash {4,6}
+(ragflow) ➜ ragflow/sandbox main ✓ pwd # make sure you are in the right directory
+/home/infiniflow/workspace/ragflow/sandbox
+
+(ragflow) ➜ ragflow/sandbox main ✓ echo "openpyxl" >> sandbox_base_image/python/requirements.txt # add the package to the requirements.txt file
+
+(ragflow) ➜ ragflow/sandbox main ✗ cat sandbox_base_image/python/requirements.txt # make sure the package is added
+numpy
+pandas
+requests
+openpyxl # here it is
+
+(ragflow) ➜ ragflow/sandbox main ✗ make # rebuild the docker image, this command will rebuild the iamge and start the service immediately. To build image only, using `make build` instead.
+
+(ragflow) ➜ ragflow/sandbox main ✗ docker exec -it sandbox_python_0 /bin/bash # entering container to check if the package is installed
+
+
+# in the container
+nobody@ffd8a7dd19da:/workspace$ python # launch python shell
+Python 3.11.13 (main, Aug 12 2025, 22:46:03) [GCC 12.2.0] on linux
+Type "help", "copyright", "credits" or "license" for more information.
+>>> import openpyxl # import the package to verify installation
+>>>
+# That's okay!
+```
+
+To import your JavaScript packages, navigate to `sandbox_base_image/nodejs` and use `npm` to install the required packages. For example, to add the `lodash` package, run the following commands:
+
+```bash
+(ragflow) ➜ ragflow/sandbox main ✓ pwd
+/home/infiniflow/workspace/ragflow/sandbox
+
+(ragflow) ➜ ragflow/sandbox main ✓ cd sandbox_base_image/nodejs
+
+(ragflow) ➜ ragflow/sandbox/sandbox_base_image/nodejs main ✓ npm install lodash 
+
+(ragflow) ➜ ragflow/sandbox/sandbox_base_image/nodejs main ✓ cd ../.. # go back to sandbox root directory
+
+(ragflow) ➜ ragflow/sandbox main ✗ make # rebuild the docker image, this command will rebuild the iamge and start the service immediately. To build image only, using `make build` instead.
+
+(ragflow) ➜ ragflow/sandbox main ✗ docker exec -it sandbox_nodejs_0 /bin/bash # entering container to check if the package is installed
+
+# in the container
+nobody@dd4bbcabef63:/workspace$ npm list lodash # verify via npm list
+/workspace
+`-- lodash@4.17.21 extraneous
+
+nobody@dd4bbcabef63:/workspace$ ls node_modules | grep lodash # or verify via listing node_modules
+lodash
+
+# That's okay!
+```

docs/guides/agent/agent_component_reference/retrieval.mdx

@@ -9,19 +9,70 @@ A component that retrieves information from specified datasets.
 
 ## Scenarios
 
-A **Retrieval** component is essential in most RAG scenarios, where information is extracted from designated knowledge bases before being sent to the LLM for content generation. As of v0.20.3, a **Retrieval** component can operate either as a workflow component or as a tool of an **Agent**, enabling the Agent to control its invocation and search queries.
+A **Retrieval** component is essential in most RAG scenarios, where information is extracted from designated knowledge bases before being sent to the LLM for content generation. A **Retrieval** component can operate either as a standalone workflow module or as a tool for an **Agent** component. In the latter role, the **Agent** component has autonomous control over when to invoke it for query and retrieval.
+
+The following screenshot shows a reference design using the **Retrieval** component, where the component serves as a tool for an **Agent** component. You can find it from the **Report Agent Using Knowledge Base** Agent template.
+
+![retrieval_reference_design](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/retrieval_reference_design.jpg)
+
+## Prerequisites
+
+Ensure you [have properly configured your target knowledge base(s)](../../dataset/configure_knowledge_base.md).
+
+## Quickstart
+
+### 1. Click on a **Retrieval** component to show its configuration panel  
+
+The corresponding configuration panel appears to the right of the canvas. Use this panel to define and fine-tune the **Retrieval** component's search behavior.
+
+### 2. Input query variable(s)
+
+The **Retrieval** component depends on query variables to specify its queries. 
+
+:::caution IMPORTANT
+- If you use the **Retrieval** component as a standalone workflow module, input query variables in the **Input Variables** text box.
+- If it is used as a tool for an **Agent** component, input the query variables in the **Agent** component's **User prompt** field.
+:::
+
+By default, you can use `sys.query`, which is the user query and the default output of the **Begin** component. All global variables defined before the **Retrieval** component can also be used as query statements. Use the `(x)` button or type `/` to show all the available query variables.
+
+### 3. Select knowledge base(s) to query
+
+You can specify one or multiple knowledge bases to retrieve data from. If selecting mutiple, ensure they use the same embedding model.
+
+### 4. Expand **Advanced Settings** to configure the retrieval method
+
+By default, a combination of weighted keyword similarity and weighted vector cosine similarity is used for retrieval. If a rerank model is selected, a combination of weighted keyword similarity and weighted reranking score will be used instead.
+
+As a starter, you can skip this step to stay with the default retrieval method.
+
+:::caution WARNING
+Using a rerank model will *significantly* increase the system's response time. If you must use a rerank model, ensure you use a SaaS reranker; if you prefer a locally deployed rerank model, ensure you start RAGFlow with **docker-compose-gpu.yml**.
+:::
+
+### 5. Enable cross-language search
+
+If your user query is different from the languages of the knowledge bases, you can select the target languages in the **Cross-language search** dropdown menu. The model will then translates queries to ensure accurate matching of semantic meaning across languages.
+
+
+### 6. Test retrieval results
+
+Click the **Run** button on the top of canvas to test the retrieval results.
+
+### 7. Choose the next component
+
+When necessary, click the **+** button on the **Retrieval** component to choose the next component in the worflow from the dropdown list.
+
 
 ## Configurations
 
-Click on a **Retrieval** component to open its configuration window.
-
 ### Query variables
 
 *Mandatory*
 
-Select the query source for retrieval.
+Select the query source for retrieval. Defaults to `sys.query`, which is the default output of the **Begin** component.
 
-The **Retrieval** component relies on query variables to specify its data inputs (queries). All global variables defined before the **Retrieval** component are available in the dropdown list.  
+The **Retrieval** component relies on query variables to specify its queries. All global variables defined before the **Retrieval** component can also be used as queries. Use the `(x)` button or type `/` to show all the available query variables.
 
 ### Knowledge bases 
 
@@ -72,8 +123,23 @@ Select one or more languages for cross‑language search. If no language is sele
 
 ### Use knowledge graph
 
+:::caution IMPORTANT
+Before enabling this feature, ensure you have properly [constructed a knowledge graph from each target knowledge base](../../dataset/construct_knowledge_graph.md).
+:::
+
 Whether to use knowledge graph(s) in the specified knowledge base(s) during retrieval for multi-hop question answering. When enabled, this would involve iterative searches across entity, relationship, and community report chunks, greatly increasing retrieval time.
 
 ### Output
 
 The global variable name for the output of the **Retrieval** component, which can be referenced by other components in the workflow.
+
+
+## Frequently asked questions
+
+### How to reduce response time?
+
+Go through the checklist below for best performance:
+
+- Leave the **Rerank model** field empty.
+- If you must use a rerank model, ensure you use a SaaS reranker; if you prefer a locally deployed rerank model, ensure you start RAGFlow with **docker-compose-gpu.yml**.
+- Disable **Use knowledge graph**.

docs/guides/agent/agent_introduction.md

@@ -9,12 +9,12 @@ Key concepts, basic operations, a quick view of the agent editor.
 
 ---
 
-## Key concepts
-
 :::danger DEPRECATED!
 A new version is coming soon.
 :::
 
+## Key concepts
+
 Agents and RAG are complementary techniques, each enhancing the other’s capabilities in business applications. RAGFlow v0.8.0 introduces an agent mechanism, featuring a no-code workflow editor on the front end and a comprehensive graph-based task orchestration framework on the back end. This mechanism is built on top of RAGFlow's existing RAG solutions and aims to orchestrate search technologies such as query intent classification, conversation leading, and query rewriting to:
 
 - Provide higher retrievals and,
@@ -33,55 +33,19 @@ Before proceeding, ensure that:
 
 Click the **Agent** tab in the middle top of the page to show the **Agent** page. As shown in the screenshot below, the cards on this page represent the created agents, which you can continue to edit.
 
-![agent_mainpage](https://github.com/user-attachments/assets/5c0bb123-8f4e-42ea-b250-43f640dc6814)
+![Agent_list](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/agent_list.jpg)
 
 We also provide templates catered to different business scenarios. You can either generate your agent from one of our agent templates or create one from scratch:
 
 1. Click **+ Create agent** to show the **agent template** page:
 
-   ![agent_templates](https://github.com/user-attachments/assets/73bd476c-4bab-4c8c-82f8-6b00fb2cd044)
+   ![agent_template](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/agent_template_list.jpg)
 
-2. To create an agent from scratch, click the **Blank** card. Alternatively, to create an agent from one of our templates, hover over the desired card, such as **General-purpose chatbot**, click **Use this template**, name your agent in the pop-up dialogue, and click **OK** to confirm.  
+2. To create an agent from scratch, click **Create Agent**. Alternatively, to create an agent from one of our templates, click the desired card, such as **Deep Research**, name your agent in the pop-up dialogue, and click **OK** to confirm.  
 
-   *You are now taken to the **no-code workflow editor** page. The left panel lists the components (operators): Above the dividing line are the RAG-specific components; below the line are tools. We are still working to expand the component list.*
+   *You are now taken to the **no-code workflow editor** page.*
 
-   ![workflow_editor](https://github.com/user-attachments/assets/47b4d5ce-b35a-4d6b-b483-ba495a75a65d)
+   ![add_component](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/add_component.jpg)
 
-3. General speaking, now you can do the following:
+3. Click the **+** button on the **Begin** component to select the desired components in your workflow.
-   - Drag and drop a desired component to your workflow,
+4. Click **Save** to apply changes to your agent.
-   - Select the knowledge base to use,
-   - Update settings of specific components,
-   - Update LLM settings
-   - Sets the input and output for a specific component, and more.
-4. Click **Save** to apply changes to your agent and **Run** to test it.
-
-## Components
-
-Please review the flowing description of the RAG-specific components before you proceed:
-
-| Component      | Description                                                                                                                                                                                              |
-|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| **Retrieval**  | A component that retrieves information from specified knowledge bases and returns 'Empty response' if no information is found. Ensure the correct knowledge bases are selected.                          |
-| **Generate**   | A component that prompts the LLM to generate responses. You must ensure the prompt is set correctly.                                                                                                     |
-| **Interact**   | A component that serves as the interface between human and the bot, receiving user inputs and displaying the agent's responses.                                                                          |
-| **Categorize** | A component that uses the LLM to classify user inputs into predefined categories. Ensure you specify the name, description, and examples for each category, along with the corresponding next component. |
-| **Message**    | A component that sends out a static message. If multiple messages are supplied, it randomly selects one to send. Ensure its downstream is **Interact**, the interface component.                         |
-| **Rewrite**    | A component that rewrites a user query from the **Interact** component, based on the context of previous dialogues.                                                                                      |
-| **Keyword**    | A component that extracts keywords from a user query, with TopN specifying the number of keywords to extract.                                                                                            |
-
-:::caution NOTE
-
-- Ensure **Rewrite**'s upstream component is **Relevant** and downstream component is **Retrieval**.
-- Ensure the downstream component of **Message** is **Interact**.
-- The downstream component of **Begin** is always **Interact**.
-
-:::
-
-## Basic operations
-
-| Operation                 | Description                                                                                                                              |
-|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
-| Add a component           | Drag and drop the desired component from the left panel onto the canvas.                                                                 |
-| Delete a component        | On the canvas, hover over the three dots (...) of the component to display the delete option, then select it to remove the component.    |
-| Copy a component          | On the canvas, hover over the three dots (...) of the component to display the copy option, then select it to make a copy the component. |
-| Update component settings | On the canvas, click the desired component to display the component settings.                                                            |

docs/guides/agent/embed_agent_into_webpage.md

@@ -10,4 +10,6 @@ You can use iframe to embed an agent into a third-party webpage.
 1. Before proceeding, you must [acquire an API key](../models/llm_api_key_setup.md); otherwise, an error message would appear.
 2. On the **Agent** page, click an intended agent to access its editing page.
 3. Click **Management > Embed into webpage** on the top right corner of the canvas to show the **iframe** window:
-4. Copy the iframe and embed it into a specific location on your webpage.
+4. Copy the iframe and embed it into your webpage.
+
+![Embed_agent](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/embed_agent_into_webpage.jpg)

docs/guides/agent/general_purpose_chatbot.md

@@ -1,109 +0,0 @@
----
-sidebar_position: 2
-slug: /general_purpose_chatbot
----
-
-# Create chatbot
-
-Create a general-purpose chatbot.
-
----
-
-:::danger DEPRECATED!
-A new version is coming soon.
-:::
-
-Chatbot is one of the most common AI scenarios. However, effectively understanding user queries and responding appropriately remains a challenge. RAGFlow's general-purpose chatbot agent is our attempt to tackle this longstanding issue.  
-
-This chatbot closely resembles the chatbot introduced in [Start an AI chat](../chat/start_chat.md), but with a key difference - it introduces a reflective mechanism that allows it to improve the retrieval from the target knowledge bases by rewriting the user's query.
-
-This document provides guides on creating such a chatbot using our chatbot template.
-
-## Prerequisites
-
-1. Ensure you have properly set the LLM to use. See the guides on [Configure your API key](../models/llm_api_key_setup.md) or [Deploy a local LLM](../models/deploy_local_llm.mdx) for more information.
-2. Ensure you have a knowledge base configured and the corresponding files properly parsed. See the guide on [Configure a knowledge base](../dataset/configure_knowledge_base.md) for more information.
-3. Make sure you have read the [Introduction to Agentic RAG](./agent_introduction.md).
-
-## Create a chatbot agent from template
-
-To create a general-purpose chatbot agent using our template:
-
-1. Click the **Agent** tab in the middle top of the page to show the **Agent** page.
-2. Click **+ Create agent** on the top right of the page to show the **agent template** page.
-3. On the **agent template** page, hover over the card on **General-purpose chatbot** and click **Use this template**.  
-   *You are now directed to the **no-code workflow editor** page.*
-
-   ![workflow_editor](https://github.com/user-attachments/assets/52e7dc62-4bf5-4fbb-ab73-4a6e252065f0)
-
-:::tip NOTE
-RAGFlow's no-code editor spares you the trouble of coding, making agent development effortless.
-:::
-
-## Understand each component in the template
-
-Here’s a breakdown of each component and its role and requirements in the chatbot template:
-
-- **Begin**
-  - Function: Sets an opening greeting for users.
-  - Purpose: Establishes a welcoming atmosphere and prepares the user for interaction.
-
-- **Interact**
-  - Function: Serves as the interface between human and the bot.
-  - Role: Acts as the downstream component of **Begin**.  
-
-- **Retrieval**
-  - Function: Retrieves information from specified knowledge base(s).
-  - Requirement: Must have `knowledgebases` set up to function.
-
-- **Relevant**
-  - Function: Assesses the relevance of the retrieved information from the **Retrieval** component to the user query.
-  - Process:  
-    - If relevant, it directs the data to the **Generate** component for final response generation.
-    - Otherwise, it triggers the **Rewrite** component to refine the user query and redo the retrival process.
-
-- **Generate**
-  - Function: Prompts the LLM to generate responses based on the retrieved information.  
-  - Note: The prompt settings allow you to control the way in which the LLM generates responses. Be sure to review the prompts and make necessary changes.
-
-- **Rewrite**:  
-  - Function: Refines a user query when no relevant information from the knowledge base is retrieved.  
-  - Usage: Often used in conjunction with **Relevant** and **Retrieval** to create a reflective/feedback loop.  
-
-## Configure your chatbot agent
-
-1. Click **Begin** to set an opening greeting:  
-   ![opener](https://github.com/user-attachments/assets/4416bc16-2a84-4f24-a19b-6dc8b1de0908)
-
-2. Click **Retrieval** to select the right knowledge base(s) and make any necessary adjustments:  
-   ![setting_knowledge_bases](https://github.com/user-attachments/assets/5f694820-5651-45bc-afd6-cf580ca0228d)
-
-3. Click **Generate** to configure the LLM's summarization behavior:  
-   3.1. Confirm the model.  
-   3.2. Review the prompt settings. If there are variables, ensure they match the correct component IDs:  
-   ![prompt_settings](https://github.com/user-attachments/assets/19e94ea7-7f62-4b73-b526-32fcfa62f1e9)
-
-4. Click **Relevant** to review or change its settings:  
-   *You may retain the current settings, but feel free to experiment with changes to understand how the agent operates.*
-   ![relevant_settings](https://github.com/user-attachments/assets/9ff7fdd8-7a69-4ee2-bfba-c7fb8029150f)
-
-5. Click **Rewrite** to select a different model for query rewriting or update the maximum loop times for query rewriting:  
-   ![choose_model](https://github.com/user-attachments/assets/2bac1d6c-c4f1-42ac-997b-102858c3f550)
-   ![loop_time](https://github.com/user-attachments/assets/09a4ce34-7aac-496f-aa59-d8aa33bf0b1f)
-
-:::danger NOTE
-Increasing the maximum loop times may significantly extend the time required to receive the final response.
-:::
-
-1. Update your workflow where you see necessary.
-
-2. Click to **Save** to apply your changes.  
-   *Your agent appears as one of the agent cards on the **Agent** page.*
-
-## Test your chatbot agent
-
-1. Find your chatbot agent on the **Agent** page:  
-   ![find_chatbot](https://github.com/user-attachments/assets/6e6382c6-9a86-4190-9fdd-e363b7f64ba9)
-
-2. Experiment with your questions to verify if this chatbot functions as intended:  
-   ![test_chatbot](https://github.com/user-attachments/assets/c074d3bd-4c39-4b05-a68b-1fd361f256b3)

docs/guides/ai_search.md

@@ -11,7 +11,9 @@ Conduct an AI search.
 
 An AI search is a single-turn AI conversation using a predefined retrieval strategy (a hybrid search of weighted keyword similarity and weighted vector similarity) and the system's default chat model. It does not involve advanced RAG strategies like knowledge graph, auto-keyword, or auto-question. The related chunks are listed below the chat model's response in descending order based on their similarity scores. 
 
-![](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/ai_search.jpg)
+![Create search app](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/create_search_app.jpg)
+
+![Search view](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/search_view.jpg)
 
 :::tip NOTE
 When debugging your chat assistant, you can use AI search as a reference to verify your model settings and retrieval strategy.
@@ -22,10 +24,8 @@ When debugging your chat assistant, you can use AI search as a reference to veri
 - Ensure that you have configured the system's default models on the **Model providers** page.
 - Ensure that the intended knowledge bases are properly configured and the intended documents have finished file parsing.
 
-
 ## Frequently asked questions
 
 ### Key difference between an AI search and an AI chat?
 
 A chat is a multi-turn AI conversation where you can define your retrieval strategy (a weighted reranking score can be used to replace the weighted vector similarity in a hybrid search) and choose your chat model. In an AI chat, you can configure advanced RAG strategies, such as knowledge graphs, auto-keyword, and auto-question, for your specific case. Retrieved chunks are not displayed along with the answer.
-

docs/guides/chat/implement_deep_research.md

@@ -15,13 +15,13 @@ From v0.17.0 onward, RAGFlow supports integrating agentic reasoning in an AI cha
 
 To activate this feature:
 
-1. Enable the **Reasoning** toggle under the **Prompt engine** tab of your chat assistant dialogue.
+1. Enable the **Reasoning** toggle in **Chat setting**.
 
-![Image](https://github.com/user-attachments/assets/4a1968d0-0128-4371-879f-77f3a70197f5)
+![chat_reasoning](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/chat_reasoning.jpg)
 
-2. Enter the correct Tavily API key under the **Assistant settings** tab of your chat assistant dialogue to leverage Tavily-based web search
+2. Enter the correct Tavily API key to leverage Tavily-based web search:
 
-![Image](https://github.com/user-attachments/assets/e8787532-7e72-49ef-8951-169ae544512f)
+![chat_tavily](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/chat_tavily.jpg)
 
 *The following is a screenshot of a conversation that integrates Deep Research:*
 

docs/guides/chat/set_chat_variables.md

@@ -9,7 +9,7 @@ Set variables to be used together with the system prompt for your LLM.
 
 ---
 
-When configuring the system prompt for a chat model, variables play an important role in enhancing flexibility and reusability. With variables, you can dynamically adjust the system prompt to be sent to your model. In the context of RAGFlow, if you have defined variables in the **Chat Configuration** dialogue, except for the system's reserved variable `{knowledge}`, you are required to pass in values for them from RAGFlow's [HTTP API](../../references/http_api_reference.md#converse-with-chat-assistant) or through its [Python SDK](../../references/python_api_reference.md#converse-with-chat-assistant).
+When configuring the system prompt for a chat model, variables play an important role in enhancing flexibility and reusability. With variables, you can dynamically adjust the system prompt to be sent to your model. In the context of RAGFlow, if you have defined variables in **Chat setting**, except for the system's reserved variable `{knowledge}`, you are required to pass in values for them from RAGFlow's [HTTP API](../../references/http_api_reference.md#converse-with-chat-assistant) or through its [Python SDK](../../references/python_api_reference.md#converse-with-chat-assistant).
 
 :::danger IMPORTANT
 In RAGFlow, variables are closely linked with the system prompt. When you add a variable in the **Variable** section, include it in the system prompt. Conversely, when deleting a variable, ensure it is removed from the system prompt; otherwise, an error would occur.
@@ -17,9 +17,7 @@ In RAGFlow, variables are closely linked with the system prompt. When you add a
 
 ## Where to set variables
 
-Hover your mouse over your chat assistant, click **Edit** to open its **Chat Configuration** dialogue, then click the **Prompt engine** tab. Here, you can work on your variables in the **System prompt** field and the **Variable** section:
+![set_variables](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/chat_variables.jpg)
-
-![set_variables](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/prompt_engine.jpg)
 
 ## 1. Manage variables
 
@@ -42,8 +40,6 @@ Besides `{knowledge}`, you can also define your own variables to pair with the s
 - **Disabled** (Default): The variable is mandatory and must be provided.
 - **Enabled**: The variable is optional and can be omitted if not needed.
 
-
-
 ## 2. Update system prompt
 
 After you add or remove variables in the **Variable** section, ensure your changes are reflected in the system prompt to avoid inconsistencies or errors. Here's an example:

docs/guides/chat/start_chat.md

@@ -48,7 +48,7 @@ You start an AI conversation by creating an assistant.
      - If no target language is selected, the system will search only in the language of your query, which may cause relevant information in other languages to be missed.
    - **Variable** refers to the variables (keys) to be used in the system prompt. `{knowledge}` is a reserved variable. Click **Add** to add more variables for the system prompt.
       - If you are uncertain about the logic behind **Variable**, leave it *as-is*.
-      - As of v0.20.3, if you add custom variables here, the only way you can pass in their values is to call:
+      - As of v0.20.5, if you add custom variables here, the only way you can pass in their values is to call:
          - HTTP method [Converse with chat assistant](../../references/http_api_reference.md#converse-with-chat-assistant), or
          - Python method [Converse with chat assistant](../../references/python_api_reference.md#converse-with-chat-assistant).
 
@@ -77,28 +77,24 @@ You start an AI conversation by creating an assistant.
 
 5. Now, let's start the show:
 
-   ![question1](https://github.com/user-attachments/assets/c4114a3d-74ff-40a3-9719-6b47c7b11ab1)
+   ![chat_thermal_solution](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/chat_thermal_solution.jpg)
 
 :::tip NOTE
 
 1. Click the light bulb icon above the answer to view the expanded system prompt:
 
-![](https://github.com/user-attachments/assets/515ab187-94e8-412a-82f2-aba52cd79e09)
+![prompt_display](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/prompt_display.jpg)
 
    *The light bulb icon is available only for the current dialogue.*
 
 2. Scroll down the expanded prompt to view the time consumed for each task:
 
-![enlighten](https://github.com/user-attachments/assets/fedfa2ee-21a7-451b-be66-20125619923c)
+![time_elapsed](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/time_elapsed.jpg)
 :::
 
 ## Update settings of an existing chat assistant
 
-Hover over an intended chat assistant **>** **Edit** to show the chat configuration dialogue:
+![chat_setting](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/chat_setting.jpg)
-
-![edit_chat](https://github.com/user-attachments/assets/5c514cf0-a959-4cfe-abad-5e42a0e23974)
-
-![chat_config](https://github.com/user-attachments/assets/1a4eaed2-5430-4585-8ab6-930549838c5b)
 
 ## Integrate chat capabilities into your application or webpage
 
@@ -113,6 +109,8 @@ You can use iframe to embed the created chat assistant into a third-party webpag
 1. Before proceeding, you must [acquire an API key](../models/llm_api_key_setup.md); otherwise, an error message would appear.
 2. Hover over an intended chat assistant **>** **Edit** to show the **iframe** window:
 
-   ![chat-embed](https://github.com/user-attachments/assets/13ea3021-31c4-4a14-9b32-328cd3318fb5)
+   ![chat-embed](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/embed_chat_into_webpage.jpg)
 
-3. Copy the iframe and embed it into a specific location on your webpage.
+3. Copy the iframe and embed it into your webpage.
+
+![chat-embed](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/embedded_chat_app.jpg)

docs/guides/dataset/configure_knowledge_base.md

@@ -16,7 +16,7 @@ Knowledge base, hallucination-free chat, and file management are the three pilla
 
 With multiple knowledge bases, you can build more flexible, diversified question answering. To create your first knowledge base:
 
-![create knowledge base](https://github.com/infiniflow/ragflow/assets/93570324/110541ed-6cea-4a03-a11c-414a0948ba80)
+![create knowledge base](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/create_knowledge_base.jpg)
 
 _Each time a knowledge base is created, a folder with the same name is generated in the **root/.knowledgebase** directory._
 
@@ -24,7 +24,7 @@ _Each time a knowledge base is created, a folder with the same name is generated
 
 The following screenshot shows the configuration page of a knowledge base. A proper configuration of your knowledge base is crucial for future AI chats. For example, choosing the wrong embedding model or chunking method would cause unexpected semantic loss or mismatched answers in chats. 
 
-![knowledge base configuration](https://github.com/infiniflow/ragflow/assets/93570324/384c671a-8b9c-468c-b1c9-1401128a9b65)
+![knowledge base configuration](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/configure_knowledge_base.jpg)
 
 This section covers the following topics:
 
@@ -56,7 +56,7 @@ RAGFlow offers multiple chunking template to facilitate chunking files of differ
 
 You can also change a file's chunking method on the **Datasets** page.
 
-![change chunking method](https://github.com/infiniflow/ragflow/assets/93570324/ac116353-2793-42b2-b181-65e7082bed42)
+![change chunking method](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/embedded_chat_app.jpg)
 
 ### Select embedding model
 
@@ -82,10 +82,8 @@ While uploading files directly to a knowledge base seems more convenient, we *hi
 
 File parsing is a crucial topic in knowledge base configuration. The meaning of file parsing in RAGFlow is twofold: chunking files based on file layout and building embedding and full-text (keyword) indexes on these chunks. After having selected the chunking method and embedding model, you can start parsing a file:
 
-![parse file](https://github.com/infiniflow/ragflow/assets/93570324/5311f166-6426-447f-aa1f-bd488f1cfc7b)
+![parse file](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/parse_file.jpg)
 
-- Click the play button next to **UNSTART** to start file parsing.
-- Click the red-cross icon and then refresh, if your file parsing stalls for a long time. 
 - As shown above, RAGFlow allows you to use a different chunking method for a particular file, offering flexibility beyond the default method. 
 - As shown above, RAGFlow allows you to enable or disable individual files, offering finer control over knowledge base-based AI chats. 
 
@@ -97,13 +95,13 @@ RAGFlow features visibility and explainability, allowing you to view the chunkin
 
    _You are taken to the **Chunk** page:_
 
-   ![chunks](https://github.com/infiniflow/ragflow/assets/93570324/0547fd0e-e71b-41f8-8e0e-31649c85fd3d)
+   ![chunks](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/file_chunks.jpg)
 
 2. Hover over each snapshot for a quick view of each chunk.
 
-3. Double-click the chunked texts to add keywords or make *manual* changes where necessary:
+3. Double-click the chunked texts to add keywords, questions, tags, or make *manual* changes where necessary:
 
-   ![update chunk](https://github.com/infiniflow/ragflow/assets/93570324/1d84b408-4e9f-46fd-9413-8c1059bf9c76)
+   ![update chunk](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/add_keyword_question.jpg)
 
 :::caution NOTE
 You can add keywords to a file chunk to increase its ranking for queries containing those keywords. This action increases its keyword weight and can improve its position in search list.  
@@ -113,7 +111,7 @@ You can add keywords to a file chunk to increase its ranking for queries contain
 
    _As you can tell from the following, RAGFlow responds with truthful citations._
 
-   ![retrieval test](https://github.com/infiniflow/ragflow/assets/93570324/c03f06f6-f41f-4b20-a97e-ae405d3a950c)
+   ![retrieval test](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/retrieval_test.jpg)
 
 ### Run retrieval testing
 
@@ -124,13 +122,11 @@ RAGFlow uses multiple recall of both full-text search and vector search in its c
 
 See [Run retrieval test](./run_retrieval_test.md) for details.
 
-![retrieval test](https://github.com/infiniflow/ragflow/assets/93570324/c03f06f6-f41f-4b20-a97e-ae405d3a950c)
-
 ## Search for knowledge base
 
-As of RAGFlow v0.20.3, the search feature is still in a rudimentary form, supporting only knowledge base search by name.
+As of RAGFlow v0.20.5, the search feature is still in a rudimentary form, supporting only knowledge base search by name.
 
-![search knowledge base](https://github.com/infiniflow/ragflow/assets/93570324/836ae94c-2438-42be-879e-c7ad2a59693e)
+![search knowledge base](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/search_datasets.jpg)
 
 ## Delete knowledge base
 
@@ -139,4 +135,4 @@ You are allowed to delete a knowledge base. Hover your mouse over the three dot
 - The files uploaded directly to the knowledge base are gone;  
 - The file references, which you created from within **File Management**, are gone, but the associated files still exist in **File Management**. 
 
-![delete knowledge base](https://github.com/infiniflow/ragflow/assets/93570324/fec7a508-6cfe-4bca-af90-81d3fdb94098)
+![delete knowledge base](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/delete_datasets.jpg)

docs/guides/dataset/construct_knowledge_graph.md

@@ -31,7 +31,7 @@ RAPTOR (Recursive Abstractive Processing for Tree Organized Retrieval) can also
 
 The system's default chat model is used to generate knowledge graph. Before proceeding, ensure that you have a chat model properly configured:
 
-![Image](https://github.com/user-attachments/assets/6bc34279-68c3-4d99-8d20-b7bd1dafc1c1)
+![Set default models](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/set_default_models.jpg)
 
 ## Configurations
 
@@ -74,7 +74,7 @@ In a knowledge graph, a community is a cluster of entities linked by relationshi
 3. Click **Knowledge graph** to view the details of the generated graph.
 4. To use the created knowledge graph, do either of the following:
    
-   - In your **Chat Configuration** dialogue, click the **Assistant settings** tab to add the corresponding knowledge base(s) and click the **Prompt engine** tab to switch on the **Use knowledge graph** toggle.
+   - In the **Chat setting** panel of your chat app, switch on the **Use knowledge graph** toggle.
    - If you are using an agent, click the **Retrieval** agent component to specify the knowledge base(s) and switch on the **Use knowledge graph** toggle.
 
 ## Frequently asked questions

docs/guides/dataset/enable_raptor.md

@@ -39,7 +39,7 @@ Knowledge graphs can also be used for multi-hop question-answering tasks. See [C
 
 The system's default chat model is used to summarize clustered content. Before proceeding, ensure that you have a chat model properly configured:
 
-![Image](https://github.com/user-attachments/assets/6bc34279-68c3-4d99-8d20-b7bd1dafc1c1)
+![Set default models](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/set_default_models.jpg)
 
 ## Configurations
 

docs/guides/dataset/set_metadata.md

@@ -13,13 +13,13 @@ On the **Dataset** page of your knowledge base, you can add metadata to any uplo
 
 For example, if you have a dataset of HTML files and want the LLM to cite the source URL when responding to your query, add a `"url"` parameter to each file's metadata.
 
-![Image](https://github.com/user-attachments/assets/78cb5035-e96c-43f9-82d7-8fef1b68c843)
+![Set metadata](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/set_metadata.jpg)
 
 :::tip NOTE
 Ensure that your metadata is in JSON format; otherwise, your updates will not be applied.
 :::
 
-![Image](https://github.com/user-attachments/assets/379cf2c5-4e37-4b79-8aeb-53bf8e01d326)
+![Input metadata](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/input_metadata.jpg)
 
 ## Frequently asked questions
 

docs/guides/manage_files.md

@@ -87,4 +87,4 @@ RAGFlow's file management allows you to download an uploaded file:
 
 ![download_file](https://github.com/infiniflow/ragflow/assets/93570324/cf3b297f-7d9b-4522-bf5f-4f45743e4ed5)
 
-> As of RAGFlow v0.20.3, bulk download is not supported, nor can you download an entire folder. 
+> As of RAGFlow v0.20.5, bulk download is not supported, nor can you download an entire folder. 

docs/guides/models/llm_api_key_setup.md

@@ -42,13 +42,7 @@ After logging into RAGFlow, configuring your model API key through the **service
 After logging into RAGFlow, you can *only* configure API Key on the **Model providers** page:
 
 1. Click on your logo on the top right of the page **>** **Model providers**.
-2. Find your model card under **Models to be added** and click **Add the model**:
+2. Find your model card under **Models to be added** and click **Add the model**.
-   ![add model](https://github.com/infiniflow/ragflow/assets/93570324/07e43f63-367c-4c9c-8ed3-8a3a24703f4e)
 3. Paste your model API key.
 4. Fill in your base URL if you use a proxy to connect to the remote service.
-5. Click **OK** to confirm your changes.
+5. Click **OK** to confirm your changes.
-
-:::note
-To update an existing model API key:
-![update api key](https://github.com/infiniflow/ragflow/assets/93570324/0bfba679-33f7-4f6b-9ed6-f0e6e4b228ad)
-:::

docs/guides/team/join_or_leave_team.md

@@ -26,20 +26,12 @@ You cannot invite users to a team unless you are its owner.
 
 ## Accept or decline team invite
 
-1. You will be notified when you receive an invitation to join a team:
+1. You will be notified on the top right corner of your system page when you receive an invitation to join a team.
-
-![team_notification](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/team_notification.jpg)
 
 2. Click on your avatar in the top right corner of the page, then select **Team** in the left-hand panel to access the **Team** page.
 
-![team](https://github.com/user-attachments/assets/0eac2503-26bc-4568-b3f2-bcd84069a07a)
-
 _On the **Team** page, you can view the information about members of your team and the teams you have joined._
 
-![accept_or_decline_team_invite](https://github.com/user-attachments/assets/6a2cb61f-03d5-4423-9ed1-71df97ff4114)
-
 _After accepting the team invite, you should be able to view and update the team owner's knowledge bases whose **Permissions** is set to **Team**._
 
-## Leave a joined team
+## Leave a joined team
-
-![leave_team](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/quit.jpg)

docs/guides/team/manage_team_members.md

@@ -29,14 +29,14 @@ By default, each RAGFlow user is assigned a single team named after their name.
 
 Click on your avatar in the top right corner of the page, then select **Team** in the left-hand panel to access the **Team** page.
 
-![team](https://github.com/user-attachments/assets/0eac2503-26bc-4568-b3f2-bcd84069a07a)
+![team_view](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/team_view.jpg)
 
 _On the **Team** page, you can view the information about members of your team and the teams you have joined._
 
 You are, by default, the owner of your own team and the only person permitted to invite users to join your team or remove team members.
 
-![invite_team_member](https://github.com/user-attachments/assets/d85b55c3-7e86-4f04-a414-ca18a9ee8963)
+![invite_user](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/invite_user.jpg)
 
 ## Remove team members
 
-![remove_members](https://github.com/user-attachments/assets/5c1a6ab5-8862-47a0-ad09-77fe88866508)
+![delete_invite](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/delete_invite.jpg)

docs/guides/team/share_agents.md

@@ -12,12 +12,8 @@ Share an Agent with your team members.
 When ready, you may share your Agents with your team members so that they can use them. Please note that your Agents are not shared automatically; you must manually enable sharing by selecting the corresponding **Permissions** radio button:
 
 1. Click the intended Agent to open its editing canvas. 
-2. Click **Settings** to show the **Agent settings** dialogue.
+2. Click **Management** > **Settings** to show the **Agent settings** dialogue.
 3. Change **Permissions** from **Only me** to **Team**.
 4. Click **Save** to apply your changes.
 
-![share_agent](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/share_agent.jpg)
+*When completed, your team members will see your shared Agents.*
-
-*When completed, your team members will see your shared Agents like this:*
-
-![shared_agent](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/shared_agent.jpg)

docs/guides/team/share_knowledge_bases.md

@@ -15,8 +15,4 @@ When ready, you may share your knowledge bases with your team members so that th
 2. Change **Permissions** from **Only me** to **Team**.
 3. Click **Save** to apply your changes.
 
-![share_knowledge_base](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/share_knowledge_base.jpg)
+*Once completed, your team members will see your shared knowledge bases.*
-
-*Once completed, your team members will see your shared knowledge bases like this:*
-
-![shared_knowledge_base](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/shared_knowledge_base.jpg)

docs/guides/tracing.mdx

@@ -18,7 +18,7 @@ RAGFlow ships with a built-in [Langfuse](https://langfuse.com) integration so th
 Langfuse stores traces, spans and prompt payloads in a purpose-built observability backend and offers filtering and visualisations on top.  
 
 :::info NOTE
-• RAGFlow **≥ 0.20.3** (contains the Langfuse connector)  
+• RAGFlow **≥ 0.20.5** (contains the Langfuse connector)  
 • A Langfuse workspace (cloud or self-hosted) with a _Project Public Key_ and _Secret Key_
 :::
 

docs/guides/upgrade_ragflow.mdx

@@ -66,10 +66,10 @@ To upgrade RAGFlow, you must upgrade **both** your code **and** your Docker imag
    git clone https://github.com/infiniflow/ragflow.git
    ```
 
-2. Switch to the latest, officially published release, e.g., `v0.20.3`:
+2. Switch to the latest, officially published release, e.g., `v0.20.5`:
 
    ```bash
-   git checkout -f v0.20.3
+   git checkout -f v0.20.5
    ```
 
 3. Update **ragflow/docker/.env**:
@@ -83,14 +83,14 @@ To upgrade RAGFlow, you must upgrade **both** your code **and** your Docker imag
   <TabItem value="slim">
 
 ```bash
-RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3-slim
+RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5-slim
 ```
 
   </TabItem>
   <TabItem value="full">
 
 ```bash
-RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3
+RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5
 ```
 
   </TabItem>
@@ -114,10 +114,10 @@ No, you do not need to. Upgrading RAGFlow in itself will *not* remove your uploa
 1. From an environment with Internet access, pull the required Docker image.
 2. Save the Docker image to a **.tar** file.
    ```bash
-   docker save -o ragflow.v0.20.3.tar infiniflow/ragflow:v0.20.3
+   docker save -o ragflow.v0.20.5.tar infiniflow/ragflow:v0.20.5
    ```
 3. Copy the **.tar** file to the target server.
 4. Load the **.tar** file into Docker:
    ```bash
-   docker load -i ragflow.v0.20.3.tar
+   docker load -i ragflow.v0.20.5.tar
    ```

docs/quickstart.mdx

@@ -44,7 +44,7 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
 
    `vm.max_map_count`. This value sets the maximum number of memory map areas a process may have. Its default value is 65530. While most applications require fewer than a thousand maps, reducing this value can result in abnormal behaviors, and the system will throw out-of-memory errors when a process reaches the limitation.
 
-   RAGFlow v0.20.3 uses Elasticsearch or [Infinity](https://github.com/infiniflow/infinity) for multiple recall. Setting the value of `vm.max_map_count` correctly is crucial to the proper functioning of the Elasticsearch component.
+   RAGFlow v0.20.5 uses Elasticsearch or [Infinity](https://github.com/infiniflow/infinity) for multiple recall. Setting the value of `vm.max_map_count` correctly is crucial to the proper functioning of the Elasticsearch component.
 
 <Tabs
   defaultValue="linux"
@@ -184,13 +184,13 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
    ```bash
    $ git clone https://github.com/infiniflow/ragflow.git
    $ cd ragflow/docker
-   $ git checkout -f v0.20.3
+   $ git checkout -f v0.20.5
    ```
 
 3. Use the pre-built Docker images and start up the server:
 
    :::tip NOTE
-   The command below downloads the `v0.20.3-slim` edition of the RAGFlow Docker image. Refer to the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.3-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.3` for the full edition `v0.20.3`.
+   The command below downloads the `v0.20.5-slim` edition of the RAGFlow Docker image. Refer to the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.20.5-slim`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server. For example: set `RAGFLOW_IMAGE=infiniflow/ragflow:v0.20.5` for the full edition `v0.20.5`.
    :::
 
    ```bash
@@ -207,8 +207,8 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
 
 | RAGFlow image tag   | Image size (GB) | Has embedding models and Python packages? | Stable?                  |
 | ------------------- | --------------- | ----------------------------------------- | ------------------------ |
-| `v0.20.3`           | &approx;9       | :heavy_check_mark:                        | Stable release           |
+| `v0.20.5`           | &approx;9       | :heavy_check_mark:                        | Stable release           |
-| `v0.20.3-slim`      | &approx;2       | ❌                                        | Stable release           |
+| `v0.20.5-slim`      | &approx;2       | ❌                                        | Stable release           |
 | `nightly`           | &approx;9       | :heavy_check_mark:                        | *Unstable* nightly build |
 | `nightly-slim`      | &approx;2       | ❌                                        | *Unstable* nightly build |
 
@@ -217,7 +217,7 @@ This section provides instructions on setting up the RAGFlow server on Linux. If
 ```
 
 :::danger IMPORTANT
-The embedding models included in `v0.20.3` and `nightly` are:
+The embedding models included in `v0.20.5` and `nightly` are:
 
 - BAAI/bge-large-zh-v1.5
 - maidalun1020/bce-embedding-base_v1
@@ -267,25 +267,16 @@ RAGFlow also supports deploying LLMs locally using Ollama, Xinference, or LocalA
 
 To add and configure an LLM: 
 
-1. Click on your logo on the top right of the page **>** **Model providers**:
+1. Click on your logo on the top right of the page **>** **Model providers**.
 
-   ![add llm](https://github.com/infiniflow/ragflow/assets/93570324/10635088-028b-4b3d-add9-5c5a6e626814)
+2. Click on the desired LLM and update the API key accordingly.
-
-2. Click on the desired LLM and update the API key accordingly (DeepSeek-V2 in this case):
-
-   ![update api key](https://github.com/infiniflow/ragflow/assets/93570324/4e5e13ef-a98d-42e6-bcb1-0c6045fc1666)
-
-   *Your added models appear as follows:* 
-
-   ![added available models](https://github.com/infiniflow/ragflow/assets/93570324/d08b80e4-f921-480a-b41d-11832489c916)
 
 3. Click **System Model Settings** to select the default models: 
 
    - Chat model, 
    - Embedding model, 
-   - Image-to-text model. 
+   - Image-to-text model,
-
+   - and more.
-   ![system model settings](https://github.com/infiniflow/ragflow/assets/93570324/cdcc1da5-4494-44cd-ad5b-1222ed6acc3f)
 
 > Some models, such as the image-to-text model **qwen-vl-max**, are subsidiary to a specific LLM. And you may need to update your API key to access these models. 
 
@@ -295,13 +286,13 @@ You are allowed to upload files to a knowledge base in RAGFlow and parse them in
 
 To create your first knowledge base:
 
-1. Click the **Knowledge Base** tab in the top middle of the page **>** **Create knowledge base**.
+1. Click the **Dataset** tab in the top middle of the page **>** **Create dataset**.
 
 2. Input the name of your knowledge base and click **OK** to confirm your changes.
 
    _You are taken to the **Configuration** page of your knowledge base._
 
-   ![knowledge base configuration](https://github.com/infiniflow/ragflow/assets/93570324/384c671a-8b9c-468c-b1c9-1401128a9b65)
+   ![knowledge base configuration](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/configure_knowledge_base.jpg)
 
 3. RAGFlow offers multiple chunk templates that cater to different document layouts and file formats. Select the embedding model and chunking method (template) for your knowledge base. 
 
@@ -315,9 +306,7 @@ Once you have selected an embedding model and used it to parse a file, you are n
 
 5. In the uploaded file entry, click the play button to start file parsing:
 
-   ![file parsing](https://github.com/infiniflow/ragflow/assets/93570324/19f273fa-0ab0-435e-bdf4-a47fb080a078)
+   ![parse file](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/parse_file.jpg)
-
-   _When the file parsing completes, its parsing status changes to **SUCCESS**._
 
 :::caution NOTE 
 - If your file parsing gets stuck at below 1%, see [this FAQ](./faq.mdx#why-does-my-document-parsing-stall-at-under-one-percent).
@@ -332,23 +321,23 @@ RAGFlow features visibility and explainability, allowing you to view the chunkin
 
    _You are taken to the **Chunk** page:_
 
-   ![chunks](https://github.com/infiniflow/ragflow/assets/93570324/0547fd0e-e71b-41f8-8e0e-31649c85fd3d)
+   ![chunks](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/file_chunks.jpg)
 
 2. Hover over each snapshot for a quick view of each chunk.
 
 3. Double click the chunked texts to add keywords or make *manual* changes where necessary:
 
-   ![update chunk](https://github.com/infiniflow/ragflow/assets/93570324/1d84b408-4e9f-46fd-9413-8c1059bf9c76)
+   ![update chunk](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/add_keyword_question.jpg)
 
 :::caution NOTE
-You can add keywords to a file chunk to improve its ranking for queries containing those keywords. This action increases its keyword weight and can improve its position in search list.
+You can add keywords or questions to a file chunk to improve its ranking for queries containing those keywords. This action increases its keyword weight and can improve its position in search list.
 :::
 
 4. In Retrieval testing, ask a quick question in **Test text** to double check if your configurations work:
 
    _As you can tell from the following, RAGFlow responds with truthful citations._
 
-   ![retrieval test](https://github.com/infiniflow/ragflow/assets/93570324/c03f06f6-f41f-4b20-a97e-ae405d3a950c)
+   ![retrieval test](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/retrieval_test.jpg)
 
 ## Set up an AI chat
 
@@ -370,9 +359,7 @@ Conversations in RAGFlow are based on a particular knowledge base or multiple kn
 
 5. Now, let's start the show:
 
-   ![question1](https://github.com/infiniflow/ragflow/assets/93570324/bb72dd67-b35e-4b2a-87e9-4e4edbd6e677)
+   ![chat_thermal_solution](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/chat_thermal_solution.jpg)
-
-   ![question2](https://github.com/infiniflow/ragflow/assets/93570324/7cc585ae-88d0-4aa2-817d-0370b2ad7230)
 
 
 :::tip NOTE

docs/references/glossary.mdx

@@ -19,7 +19,7 @@ import TOCInline from '@theme/TOCInline';
 
 ### Cross-language search
 
-Cross-language search (also known as cross-lingual retrieval) is a feature introduced in version 0.20.3. It enables users to submit queries in one language (for example, English) and retrieve relevant documents written in other languages such as Chinese or Spanish. This feature is enabled by the system’s default chat model, which translates queries to ensure accurate matching of semantic meaning across languages.
+Cross-language search (also known as cross-lingual retrieval) is a feature introduced in version 0.20.5. It enables users to submit queries in one language (for example, English) and retrieve relevant documents written in other languages such as Chinese or Spanish. This feature is enabled by the system’s default chat model, which translates queries to ensure accurate matching of semantic meaning across languages.
 
 By enabling cross-language search, users can effortlessly access a broader range of information regardless of language barriers, significantly enhancing the system’s usability and inclusiveness.
 

docs/references/http_api_reference.md

@@ -5,7 +5,7 @@ slug: /http_api_reference
 
 # HTTP API
 
-A complete reference for RAGFlow's RESTful API. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](../develop/acquire_ragflow_api_key.md).
+A complete reference for RAGFlow's RESTful API. Before proceeding, please ensure you [have your RAGFlow API key ready for authentication](https://ragflow.io/docs/dev/acquire_ragflow_api_key).
 
 ---
 
@@ -143,7 +143,6 @@ Non-stream:
 }
 ```
 
-
 Failure:
 
 ```json
@@ -200,19 +199,24 @@ curl --request POST \
 - `stream` (*Body parameter*) `boolean`  
   Whether to receive the response as a stream. Set this to `false` explicitly if you prefer to receive the entire response in one go instead of as a stream.
 
+- `session_id` (*Body parameter*) `string`  
+  Agent session id.
+
 #### Response
 
 Stream:
 
 ```json
+...
+
 data: {
-    "id": "5fa65c94-e316-4954-800a-06dfd5827052",
+    "id": "c39f6f9c83d911f0858253708ecb6573",
     "object": "chat.completion.chunk",
-    "model": "99ee29d6783511f09c921a6272e682d8",
+    "model": "d1f79142831f11f09cc51795b9eb07c0",
     "choices": [
         {
             "delta": {
-                "content": "Hello"
+                "content": " terminal"
             },
             "finish_reason": null,
             "index": 0
@@ -220,21 +224,83 @@ data: {
     ]
 }
 
-data: {"id": "518022d9-545b-4100-89ed-ecd9e46fa753", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": "!"}, "finish_reason": null, "index": 0}]}
+data: {
+    "id": "c39f6f9c83d911f0858253708ecb6573",
+    "object": "chat.completion.chunk",
+    "model": "d1f79142831f11f09cc51795b9eb07c0",
+    "choices": [
+        {
+            "delta": {
+                "content": "."
+            },
+            "finish_reason": null,
+            "index": 0
+        }
+    ]
+}
 
-data: {"id": "f37c4af0-8187-4c86-8186-048c3c6ffe4e", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": " How"}, "finish_reason": null, "index": 0}]}
+data: {
-
+    "id": "c39f6f9c83d911f0858253708ecb6573",
-data: {"id": "3ebc0fcb-0f85-4024-b4a5-3b03234a16df", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": " can"}, "finish_reason": null, "index": 0}]}
+    "object": "chat.completion.chunk",
-
+    "model": "d1f79142831f11f09cc51795b9eb07c0",
-data: {"id": "efa1f3cf-7bc4-47a4-8e53-cd696f290587", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": " I"}, "finish_reason": null, "index": 0}]}
+    "choices": [
-
+        {
-data: {"id": "2eb6f741-50a3-4d3d-8418-88be27895611", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": " assist"}, "finish_reason": null, "index": 0}]}
+            "delta": {
-
+                "content": "",
-data: {"id": "f1227e4f-bf8b-462c-8632-8f5269492ce9", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": " you"}, "finish_reason": null, "index": 0}]}
+                "reference": {
-
+                    "chunks": {
-data: {"id": "35b669d0-b2be-4c0c-88d8-17ff98592b21", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": " today"}, "finish_reason": null, "index": 0}]}
+                        "20": {
-
+                            "id": "4b8935ac0a22deb1",
-data: {"id": "f00d8a39-af60-4f32-924f-d64106a7fdf1", "object": "chat.completion.chunk", "model": "99ee29d6783511f09c921a6272e682d8", "choices": [{"delta": {"content": "?"}, "finish_reason": null, "index": 0}]}
+                            "content": "```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.",
+                            "document_id": "4bdd2ff65e1511f0907f09f583941b45",
+                            "document_name": "INSTALL22.md",
+                            "dataset_id": "456ce60c5e1511f0907f09f583941b45",
+                            "image_id": "",
+                            "positions": [
+                                [
+                                    12,
+                                    11,
+                                    11,
+                                    11,
+                                    11
+                                ]
+                            ],
+                            "url": null,
+                            "similarity": 0.5697155305154673,
+                            "vector_similarity": 0.7323851005515574,
+                            "term_similarity": 0.5000000005,
+                            "doc_type": ""
+                        }
+                    },
+                    "doc_aggs": {
+                        "INSTALL22.md": {
+                            "doc_name": "INSTALL22.md",
+                            "doc_id": "4bdd2ff65e1511f0907f09f583941b45",
+                            "count": 3
+                        },
+                        "INSTALL.md": {
+                            "doc_name": "INSTALL.md",
+                            "doc_id": "4bd7fdd85e1511f0907f09f583941b45",
+                            "count": 2
+                        },
+                        "INSTALL(1).md": {
+                            "doc_name": "INSTALL(1).md",
+                            "doc_id": "4bdfb42e5e1511f0907f09f583941b45",
+                            "count": 2
+                        },
+                        "INSTALL3.md": {
+                            "doc_name": "INSTALL3.md",
+                            "doc_id": "4bdab5825e1511f0907f09f583941b45",
+                            "count": 1
+                        }
+                    }
+                }
+            },
+            "finish_reason": null,
+            "index": 0
+        }
+    ]
+}
 
 data: [DONE]
 ```
@@ -249,30 +315,77 @@ Non-stream:
             "index": 0,
             "logprobs": null,
             "message": {
-                "content": "Hello! How can I assist you today?",
+                "content": "\nTo install Neovim, the process varies depending on your operating system:\n\n### For Windows:\n1. **Download from GitHub**: \n   - Visit the [Neovim releases page](https://github.com/neovim/neovim/releases)\n   - Download the latest Windows installer (nvim-win64.msi)\n   - Run the installer and follow the prompts\n\n2. **Using winget** (Windows Package Manager):\n...",
+                "reference": {
+                    "chunks": {
+                        "20": {
+                            "content": "```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.",
+                            "dataset_id": "456ce60c5e1511f0907f09f583941b45",
+                            "doc_type": "",
+                            "document_id": "4bdd2ff65e1511f0907f09f583941b45",
+                            "document_name": "INSTALL22.md",
+                            "id": "4b8935ac0a22deb1",
+                            "image_id": "",
+                            "positions": [
+                                [
+                                    12,
+                                    11,
+                                    11,
+                                    11,
+                                    11
+                                ]
+                            ],
+                            "similarity": 0.5697155305154673,
+                            "term_similarity": 0.5000000005,
+                            "url": null,
+                            "vector_similarity": 0.7323851005515574
+                        }
+                    },
+                    "doc_aggs": {
+                        "INSTALL(1).md": {
+                            "count": 2,
+                            "doc_id": "4bdfb42e5e1511f0907f09f583941b45",
+                            "doc_name": "INSTALL(1).md"
+                        },
+                        "INSTALL.md": {
+                            "count": 2,
+                            "doc_id": "4bd7fdd85e1511f0907f09f583941b45",
+                            "doc_name": "INSTALL.md"
+                        },
+                        "INSTALL22.md": {
+                            "count": 3,
+                            "doc_id": "4bdd2ff65e1511f0907f09f583941b45",
+                            "doc_name": "INSTALL22.md"
+                        },
+                        "INSTALL3.md": {
+                            "count": 1,
+                            "doc_id": "4bdab5825e1511f0907f09f583941b45",
+                            "doc_name": "INSTALL3.md"
+                        }
+                    }
+                },
                 "role": "assistant"
             }
         }
     ],
     "created": null,
-    "id": "17aa4ec5-6d36-40c6-9a96-1b069c216d59",
+    "id": "c39f6f9c83d911f0858253708ecb6573",
-    "model": "99ee29d6783511f09c921a6272e682d8",
+    "model": "d1f79142831f11f09cc51795b9eb07c0",
     "object": "chat.completion",
     "param": null,
     "usage": {
-        "completion_tokens": 9,
+        "completion_tokens": 415,
         "completion_tokens_details": {
             "accepted_prediction_tokens": 0,
             "reasoning_tokens": 0,
             "rejected_prediction_tokens": 0
         },
-        "prompt_tokens": 1,
+        "prompt_tokens": 6,
-        "total_tokens": 10
+        "total_tokens": 421
     }
 }
 ```
 
-
 Failure:
 
 ```json
@@ -383,7 +496,7 @@ curl --request POST \
     - `"layout_recognize"`: `string`
       - Defaults to `DeepDOC`
     - `"tag_kb_ids"`: `array<string>` refer to [Use tag set](https://ragflow.io/docs/dev/use_tag_sets)
-      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunk Method
+      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunking Method
     - `"task_page_size"`: `int` For PDF only.
       - Defaults to `12`
       - Minimum: `1`
@@ -604,7 +717,7 @@ curl --request PUT \
     - `"layout_recognize"`: `string`
       - Defaults to `DeepDOC`
     - `"tag_kb_ids"`: `array<string>` refer to [Use tag set](https://ragflow.io/docs/dev/use_tag_sets)
-      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunk Method
+      - Must include a list of dataset IDs, where each dataset is parsed using the ​​Tag Chunking Method
     - `"task_page_size"`: `int` For PDF only.
       - Defaults to `12`
       - Minimum: `1`
@@ -729,9 +842,10 @@ Failure:
     "message": "The dataset doesn't exist"
 }
 ```
+
  ---
 
-## Get dataset's knowledge graph
+### Get knowledge graph
 
 **GET** `/api/v1/datasets/{dataset_id}/knowledge_graph`
 
@@ -808,9 +922,10 @@ Failure:
     "message": "The dataset doesn't exist"
 }
 ```
+
 ---
 
-## Delete dataset's knowledge graph
+### Delete knowledge graph
 
 **DELETE** `/api/v1/datasets/{dataset_id}/knowledge_graph`
 
@@ -855,6 +970,7 @@ Failure:
     "message": "The dataset doesn't exist"
 }
 ```
+
 ---
 
 ## FILE MANAGEMENT WITHIN DATASET
@@ -1692,7 +1808,8 @@ Retrieves chunks from specified datasets.
   - `"rerank_id"`: `string`  
   - `"keyword"`: `boolean`  
   - `"highlight"`: `boolean`
-  - `"cross_languages"`: `list[string]`  
+  - `"cross_languages"`: `list[string]`
+  - `"metadata_condition"`: `object`
 
 ##### Request example
 
@@ -1739,7 +1856,8 @@ curl --request POST \
   - `false`: Disable highlighting of matched terms (default).
 - `"cross_languages"`: (*Body parameter*) `list[string]`  
   The languages that should be translated into, in order to achieve keywords retrievals in different languages.
-
+- `"metadata_condition"`: (*Body parameter*), `object`
+  The metadata condition for filtering chunks.
 #### Response
 
 Success:
@@ -3017,41 +3135,88 @@ success without `session_id` provided and with no variables specified in the **B
 Stream:
 
 ```json
-data:{
-    "event": "message",
-    "message_id": "eb0c0a5e783511f0b9b61a6272e682d8",
-    "created_at": 1755083342,
-    "task_id": "99ee29d6783511f09c921a6272e682d8",
-    "data": {
-        "content": "Hello"
-    },
-    "session_id": "eaf19a8e783511f0b9b61a6272e682d8"
-}
-
-data:{
-    "event": "message",
-    "message_id": "eb0c0a5e783511f0b9b61a6272e682d8",
-    "created_at": 1755083342,
-    "task_id": "99ee29d6783511f09c921a6272e682d8",
-    "data": {
-        "content": "!"
-    },
-    "session_id": "eaf19a8e783511f0b9b61a6272e682d8"
-}
-
-data:{
-    "event": "message",
-    "message_id": "eb0c0a5e783511f0b9b61a6272e682d8",
-    "created_at": 1755083342,
-    "task_id": "99ee29d6783511f09c921a6272e682d8",
-    "data": {
-        "content": " How"
-    },
-    "session_id": "eaf19a8e783511f0b9b61a6272e682d8"
-}
-
 ...
 
+data: {
+    "event": "message",
+    "message_id": "cecdcb0e83dc11f0858253708ecb6573",
+    "created_at": 1756364483,
+    "task_id": "d1f79142831f11f09cc51795b9eb07c0",
+    "data": {
+        "content": " themes"
+    },
+    "session_id": "cd097ca083dc11f0858253708ecb6573"
+}
+
+data: {
+    "event": "message",
+    "message_id": "cecdcb0e83dc11f0858253708ecb6573",
+    "created_at": 1756364483,
+    "task_id": "d1f79142831f11f09cc51795b9eb07c0",
+    "data": {
+        "content": "."
+    },
+    "session_id": "cd097ca083dc11f0858253708ecb6573"
+}
+
+data: {
+    "event": "message_end",
+    "message_id": "cecdcb0e83dc11f0858253708ecb6573",
+    "created_at": 1756364483,
+    "task_id": "d1f79142831f11f09cc51795b9eb07c0",
+    "data": {
+        "reference": {
+            "chunks": {
+                "20": {
+                    "id": "4b8935ac0a22deb1",
+                    "content": "```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.",
+                    "document_id": "4bdd2ff65e1511f0907f09f583941b45",
+                    "document_name": "INSTALL22.md",
+                    "dataset_id": "456ce60c5e1511f0907f09f583941b45",
+                    "image_id": "",
+                    "positions": [
+                        [
+                            12,
+                            11,
+                            11,
+                            11,
+                            11
+                        ]
+                    ],
+                    "url": null,
+                    "similarity": 0.5705525104787287,
+                    "vector_similarity": 0.7351750337624289,
+                    "term_similarity": 0.5000000005,
+                    "doc_type": ""
+                }
+            },
+            "doc_aggs": {
+                "INSTALL22.md": {
+                    "doc_name": "INSTALL22.md",
+                    "doc_id": "4bdd2ff65e1511f0907f09f583941b45",
+                    "count": 3
+                },
+                "INSTALL.md": {
+                    "doc_name": "INSTALL.md",
+                    "doc_id": "4bd7fdd85e1511f0907f09f583941b45",
+                    "count": 2
+                },
+                "INSTALL(1).md": {
+                    "doc_name": "INSTALL(1).md",
+                    "doc_id": "4bdfb42e5e1511f0907f09f583941b45",
+                    "count": 2
+                },
+                "INSTALL3.md": {
+                    "doc_name": "INSTALL3.md",
+                    "doc_id": "4bdab5825e1511f0907f09f583941b45",
+                    "count": 1
+                }
+            }
+        }
+    },
+    "session_id": "cd097ca083dc11f0858253708ecb6573"
+}
+
 data:[DONE]
 ```
 
@@ -3061,21 +3226,77 @@ Non-stream:
 {
     "code": 0,
     "data": {
-        "created_at": 1755083440,
+        "created_at": 1756363177,
         "data": {
-            "created_at": 547061.147866385,
+            "content": "\nTo install Neovim, the process varies depending on your operating system:\n\n### For macOS:\nUsing Homebrew:\n```bash\nbrew install neovim\n```\n\n### For Linux (Debian/Ubuntu):\n```bash\nsudo apt update\nsudo apt install neovim\n```\n\nFor other Linux distributions, you can use their respective package managers or build from source.\n\n### For Windows:\n1. Download the latest Windows installer from the official Neovim GitHub releases page\n2. Run the installer and follow the prompts\n3. Add Neovim to your PATH if not done automatically\n\n### From source (Unix-like systems):\n```bash\ngit clone https://github.com/neovim/neovim.git\ncd neovim\nmake CMAKE_BUILD_TYPE=Release\nsudo make install\n```\n\nAfter installation, you can verify it by running `nvim --version` in your terminal.",
-            "elapsed_time": 2.595433341921307,
+            "created_at": 18129.044975627,
-            "inputs": {},
+            "elapsed_time": 10.0157331670016,
+            "inputs": {
+                "var1": {
+                    "value": "I am var1"
+                },
+                "var2": {
+                    "value": "I am var2"
+                }
+            },
             "outputs": {
-                "_created_time": 547061.149137775,
+                "_created_time": 18129.502422278,
-                "_elapsed_time": 8.720310870558023e-05,
+                "_elapsed_time": 0.00013378599760471843,
-                "content": "Hello! How can I assist you today?"
+                "content": "\nTo install Neovim, the process varies depending on your operating system:\n\n### For macOS:\nUsing Homebrew:\n```bash\nbrew install neovim\n```\n\n### For Linux (Debian/Ubuntu):\n```bash\nsudo apt update\nsudo apt install neovim\n```\n\nFor other Linux distributions, you can use their respective package managers or build from source.\n\n### For Windows:\n1. Download the latest Windows installer from the official Neovim GitHub releases page\n2. Run the installer and follow the prompts\n3. Add Neovim to your PATH if not done automatically\n\n### From source (Unix-like systems):\n```bash\ngit clone https://github.com/neovim/neovim.git\ncd neovim\nmake CMAKE_BUILD_TYPE=Release\nsudo make install\n```\n\nAfter installation, you can verify it by running `nvim --version` in your terminal."
+            },
+            "reference": {
+                "chunks": {
+                    "20": {
+                        "content": "```cd /usr/ports/editors/neovim/ && make install```## Android[Termux](https://github.com/termux/termux-app) offers a Neovim package.",
+                        "dataset_id": "456ce60c5e1511f0907f09f583941b45",
+                        "doc_type": "",
+                        "document_id": "4bdd2ff65e1511f0907f09f583941b45",
+                        "document_name": "INSTALL22.md",
+                        "id": "4b8935ac0a22deb1",
+                        "image_id": "",
+                        "positions": [
+                            [
+                                12,
+                                11,
+                                11,
+                                11,
+                                11
+                            ]
+                        ],
+                        "similarity": 0.5705525104787287,
+                        "term_similarity": 0.5000000005,
+                        "url": null,
+                        "vector_similarity": 0.7351750337624289
+                    }
+                },
+                "doc_aggs": {
+                    "INSTALL(1).md": {
+                        "count": 2,
+                        "doc_id": "4bdfb42e5e1511f0907f09f583941b45",
+                        "doc_name": "INSTALL(1).md"
+                    },
+                    "INSTALL.md": {
+                        "count": 2,
+                        "doc_id": "4bd7fdd85e1511f0907f09f583941b45",
+                        "doc_name": "INSTALL.md"
+                    },
+                    "INSTALL22.md": {
+                        "count": 3,
+                        "doc_id": "4bdd2ff65e1511f0907f09f583941b45",
+                        "doc_name": "INSTALL22.md"
+                    },
+                    "INSTALL3.md": {
+                        "count": 1,
+                        "doc_id": "4bdab5825e1511f0907f09f583941b45",
+                        "doc_name": "INSTALL3.md"
+                    }
+                }
             }
         },
         "event": "workflow_finished",
-        "message_id": "25807f94783611f095171a6272e682d8",
+        "message_id": "c4692a2683d911f0858253708ecb6573",
-        "session_id": "25663198783611f095171a6272e682d8",
+        "session_id": "c39f6f9c83d911f0858253708ecb6573",
-        "task_id": "99ee29d6783511f09c921a6272e682d8"
+        "task_id": "d1f79142831f11f09cc51795b9eb07c0"
     }
 }
 ```
@@ -3501,7 +3722,7 @@ Failure:
 
 ### Generate related questions
 
-**POST** `/v1/sessions/related_questions`
+**POST** `/api/v1/sessions/related_questions`
 
 Generates five to ten alternative question strings from the user's original query to retrieve more relevant search results.
 
@@ -3516,7 +3737,7 @@ The chat model autonomously determines the number of questions to generate based
 #### Request
 
 - Method: POST
-- URL: `/v1/sessions/related_questions`
+- URL: `/api/v1/sessions/related_questions`
 - Headers:
   - `'content-Type: application/json'`
   - `'Authorization: Bearer <YOUR_LOGIN_TOKEN>'`
@@ -3528,7 +3749,7 @@ The chat model autonomously determines the number of questions to generate based
 
 ```bash
 curl --request POST \
-     --url http://{address}/v1/sessions/related_questions \
+     --url http://{address}/api/v1/sessions/related_questions \
      --header 'Content-Type: application/json' \
      --header 'Authorization: Bearer <YOUR_LOGIN_TOKEN>' \
      --data '
@@ -3592,7 +3813,7 @@ Lists agents.
 #### Request
 
 - Method: GET
-- URL: `/api/v1/agents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&name={agent_name}&id={agent_id}`
+- URL: `/api/v1/agents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&title={agent_name}&id={agent_id}`
 - Headers:
   - `'Authorization: Bearer <YOUR_API_KEY>'`
 
@@ -3600,7 +3821,7 @@ Lists agents.
 
 ```bash
 curl --request GET \
-     --url http://{address}/api/v1/agents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&name={agent_name}&id={agent_id} \
+     --url http://{address}/api/v1/agents?page={page}&page_size={page_size}&orderby={orderby}&desc={desc}&title={agent_name}&id={agent_id} \
      --header 'Authorization: Bearer <YOUR_API_KEY>'
 ```
 
@@ -3618,7 +3839,7 @@ curl --request GET \
   Indicates whether the retrieved agents should be sorted in descending order. Defaults to `true`.
 - `id`: (*Filter parameter*), `string`  
   The ID of the agent to retrieve.
-- `name`: (*Filter parameter*), `string`  
+- `title`: (*Filter parameter*), `string`  
   The name of the agent to retrieve.
 
 #### Response