Compare commits

...

14 Commits

Author SHA1 Message Date
960f47c4d4 Fix: When I click to interrupt the chat, the page reports an error #10553 (#10554)
### What problem does this PR solve?
Fix: When I click to interrupt the chat, the page reports an error
#10553

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 19:07:18 +08:00
51139de178 Fix: Switch the default theme from light mode to dark mode and improve some styles #9869 (#10552)
### What problem does this PR solve?

Fix: Switch the default theme from light mode to dark mode and improve
some styles #9869
-Update UI component styles such as input boxes, tables, and prompt
boxes
-Optimize login page layout and style details
-Revise some of the wording, such as uniformly changing "data flow" to
"pipeline"
-Adjust the parser to support the markdown type

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 19:06:50 +08:00
1f5167f1ca Feat: Adjust the style of note nodes #9869 (#10547)
### What problem does this PR solve?

Feat: Adjust the style of note nodes #9869

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-10-14 17:15:26 +08:00
578ea34b3e Feat: build ragflow-cli (#10544)
### What problem does this PR solve?

Build admin client.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-10-14 16:28:43 +08:00
5fb3d2f55c Fix: update parser id for change_parser. (#10545)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 15:49:05 +08:00
d99d1e3518 Feat: Merge splitter and hierarchicalMerger into one node #9869 (#10543)
### What problem does this PR solve?

Feat: Merge splitter and hierarchicalMerger into one node #9869

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-10-14 14:55:47 +08:00
5b387b68ba The 'cmd' module is introduced to make the CLI easy to use. (#10542)
…pdate comand

### What problem does this PR solve?

To make the CLI easy to use.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-14 14:53:00 +08:00
f92a45dcc4 Feat: let toc run asynchronizly... (#10513)
### What problem does this PR solve?

#10436 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-14 14:14:52 +08:00
c4b8e4845c Docs: The full edition has only two built-in embedding models (#10540)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2025-10-14 14:13:37 +08:00
87659dcd3a Fix: unexpected Auth return code (#10539)
### What problem does this PR solve?

Fix unexpected Auth return code.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 14:13:10 +08:00
6fd9508017 Docs: Updated parse_documents (#10536)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2025-10-14 13:40:56 +08:00
113851a692 Add 'status' field when list services (#10538)
### What problem does this PR solve?

```
admin> list services;
command: list services;
Listing all services
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+---------+
| extra                                                                                     | host      | id | name          | port  | service_type   | status  |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+---------+
| {}                                                                                        | 0.0.0.0   | 0  | ragflow_0     | 9380  | ragflow_server | Timeout |
| {'meta_type': 'mysql', 'password': 'infini_rag_flow', 'username': 'root'}                 | localhost | 1  | mysql         | 5455  | meta_data      | Alive   |
| {'password': 'infini_rag_flow', 'store_type': 'minio', 'user': 'rag_flow'}                | localhost | 2  | minio         | 9000  | file_store     | Alive   |
| {'password': 'infini_rag_flow', 'retrieval_type': 'elasticsearch', 'username': 'elastic'} | localhost | 3  | elasticsearch | 1200  | retrieval      | Alive   |
| {'db_name': 'default_db', 'retrieval_type': 'infinity'}                                   | localhost | 4  | infinity      | 23817 | retrieval      | Timeout |
| {'database': 1, 'mq_type': 'redis', 'password': 'infini_rag_flow'}                        | localhost | 5  | redis         | 6379  | message_queue  | Alive   |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+---------+
admin> 
Use '\q' to quit
admin> 
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-14 13:40:32 +08:00
66c69d10fe Fix: Update the parsing editor to support dynamic field names and optimize UI styles #9869 (#10535)
### What problem does this PR solve?

Fix: Update the parsing editor to support dynamic field names and
optimize UI styles #9869
-Modify the default background color of UI components such as Input and
Select to 'bg bg base'`
-Remove TagItems component reference from naive configuration page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 13:31:48 +08:00
781d49cd0e Feat: Display the configuration of data flow operators on the node #9869 (#10533)
### What problem does this PR solve?

Feat: Display the configuration of data flow operators on the node #9869

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-10-14 13:30:54 +08:00
76 changed files with 939 additions and 635 deletions

1
.gitignore vendored
View File

@ -149,6 +149,7 @@ out
# Nuxt.js build / generate output # Nuxt.js build / generate output
.nuxt .nuxt
dist dist
admin/release
# Gatsby files # Gatsby files
.cache/ .cache/

View File

@ -15,22 +15,48 @@ It consists of a server-side Service and a command-line client (CLI), both imple
- **Admin Service**: A backend service that interfaces with the RAGFlow system to execute administrative operations and monitor its status. - **Admin Service**: A backend service that interfaces with the RAGFlow system to execute administrative operations and monitor its status.
- **Admin CLI**: A command-line interface that allows users to connect to the Admin Service and issue commands for system management. - **Admin CLI**: A command-line interface that allows users to connect to the Admin Service and issue commands for system management.
### Starting the Admin Service ### Starting the Admin Service
#### Launching from source code
1. Before start Admin Service, please make sure RAGFlow system is already started. 1. Before start Admin Service, please make sure RAGFlow system is already started.
2. Run the service script: 2. Launch from source code:
```bash ```bash
python admin/admin_server.py python admin/admin_server.py
``` ```
The service will start and listen for incoming connections from the CLI on the configured port. The service will start and listen for incoming connections from the CLI on the configured port.
#### Using docker image
1. Before startup, please configure the `docker_compose.yml` file to enable admin server:
```bash
command:
- --enable-adminserver
```
2. Start the containers, the service will start and listen for incoming connections from the CLI on the configured port.
### Using the Admin CLI ### Using the Admin CLI
1. Ensure the Admin Service is running. 1. Ensure the Admin Service is running.
2. Launch the CLI client: 2. Install ragflow-cli.
```bash ```bash
python admin/admin_client.py -h 0.0.0.0 -p 9381 pip install ragflow-cli
```
3. Launch the CLI client:
```bash
ragflow-cli -h 0.0.0.0 -p 9381
```
Enter superuser's password to login. Default password is `admin`.
## Supported Commands ## Supported Commands
@ -42,12 +68,7 @@ Commands are case-insensitive and must be terminated with a semicolon (`;`).
- Lists all available services within the RAGFlow system. - Lists all available services within the RAGFlow system.
- `SHOW SERVICE <id>;` - `SHOW SERVICE <id>;`
- Shows detailed status information for the service identified by `<id>`. - Shows detailed status information for the service identified by `<id>`.
- `STARTUP SERVICE <id>;`
- Attempts to start the service identified by `<id>`.
- `SHUTDOWN SERVICE <id>;`
- Attempts to gracefully shut down the service identified by `<id>`.
- `RESTART SERVICE <id>;`
- Attempts to restart the service identified by `<id>`.
### User Management Commands ### User Management Commands
@ -55,10 +76,17 @@ Commands are case-insensitive and must be terminated with a semicolon (`;`).
- Lists all users known to the system. - Lists all users known to the system.
- `SHOW USER '<username>';` - `SHOW USER '<username>';`
- Shows details and permissions for the specified user. The username must be enclosed in single or double quotes. - Shows details and permissions for the specified user. The username must be enclosed in single or double quotes.
- `CREATE USER <username> <password>;`
- Create user by username and password. The username and password must be enclosed in single or double quotes.
- `DROP USER '<username>';` - `DROP USER '<username>';`
- Removes the specified user from the system. Use with caution. - Removes the specified user from the system. Use with caution.
- `ALTER USER PASSWORD '<username>' '<new_password>';` - `ALTER USER PASSWORD '<username>' '<new_password>';`
- Changes the password for the specified user. - Changes the password for the specified user.
- `ALTER USER ACTIVE <username> <on/off>;`
- Changes the user to active or inactive.
### Data and Agent Commands ### Data and Agent Commands

View File

@ -16,14 +16,14 @@
import argparse import argparse
import base64 import base64
from cmd import Cmd
from Cryptodome.PublicKey import RSA from Cryptodome.PublicKey import RSA
from Cryptodome.Cipher import PKCS1_v1_5 as Cipher_pkcs1_v1_5 from Cryptodome.Cipher import PKCS1_v1_5 as Cipher_pkcs1_v1_5
from typing import Dict, List, Any from typing import Dict, List, Any
from lark import Lark, Transformer, Tree from lark import Lark, Transformer, Tree, Token
import requests import requests
from requests.auth import HTTPBasicAuth from requests.auth import HTTPBasicAuth
from api.common.base64 import encode_to_base64
GRAMMAR = r""" GRAMMAR = r"""
start: command start: command
@ -100,7 +100,6 @@ NUMBER: /[0-9]+/
%ignore WS %ignore WS
""" """
class AdminTransformer(Transformer): class AdminTransformer(Transformer):
def start(self, items): def start(self, items):
@ -183,7 +182,6 @@ class AdminTransformer(Transformer):
def meta_args(self, items): def meta_args(self, items):
return items return items
def encrypt(input_string): def encrypt(input_string):
pub = '-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArq9XTUSeYr2+N1h3Afl/z8Dse/2yD0ZGrKwx+EEEcdsBLca9Ynmx3nIB5obmLlSfmskLpBo0UACBmB5rEjBp2Q2f3AG3Hjd4B+gNCG6BDaawuDlgANIhGnaTLrIqWrrcm4EMzJOnAOI1fgzJRsOOUEfaS318Eq9OVO3apEyCCt0lOQK6PuksduOjVxtltDav+guVAA068NrPYmRNabVKRNLJpL8w4D44sfth5RvZ3q9t+6RTArpEtc5sh5ChzvqPOzKGMXW83C95TxmXqpbK6olN4RevSfVjEAgCydH6HN6OhtOQEcnrU97r9H0iZOWwbw3pVrZiUkuRD1R56Wzs2wIDAQAB\n-----END PUBLIC KEY-----' pub = '-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArq9XTUSeYr2+N1h3Afl/z8Dse/2yD0ZGrKwx+EEEcdsBLca9Ynmx3nIB5obmLlSfmskLpBo0UACBmB5rEjBp2Q2f3AG3Hjd4B+gNCG6BDaawuDlgANIhGnaTLrIqWrrcm4EMzJOnAOI1fgzJRsOOUEfaS318Eq9OVO3apEyCCt0lOQK6PuksduOjVxtltDav+guVAA068NrPYmRNabVKRNLJpL8w4D44sfth5RvZ3q9t+6RTArpEtc5sh5ChzvqPOzKGMXW83C95TxmXqpbK6olN4RevSfVjEAgCydH6HN6OhtOQEcnrU97r9H0iZOWwbw3pVrZiUkuRD1R56Wzs2wIDAQAB\n-----END PUBLIC KEY-----'
pub_key = RSA.importKey(pub) pub_key = RSA.importKey(pub)
@ -191,13 +189,50 @@ def encrypt(input_string):
cipher_text = cipher.encrypt(base64.b64encode(input_string.encode('utf-8'))) cipher_text = cipher.encrypt(base64.b64encode(input_string.encode('utf-8')))
return base64.b64encode(cipher_text).decode("utf-8") return base64.b64encode(cipher_text).decode("utf-8")
def encode_to_base64(input_string):
base64_encoded = base64.b64encode(input_string.encode('utf-8'))
return base64_encoded.decode('utf-8')
class AdminCommandParser: class AdminCLI(Cmd):
def __init__(self): def __init__(self):
super().__init__()
self.parser = Lark(GRAMMAR, start='start', parser='lalr', transformer=AdminTransformer()) self.parser = Lark(GRAMMAR, start='start', parser='lalr', transformer=AdminTransformer())
self.command_history = [] self.command_history = []
self.is_interactive = False
self.admin_account = "admin@ragflow.io"
self.admin_password: str = "admin"
self.host: str = ""
self.port: int = 0
def parse_command(self, command_str: str) -> Dict[str, Any]: intro = r"""Type "\h" for help."""
prompt = "admin> "
def onecmd(self, command: str) -> bool:
try:
print(f"command: {command}")
result = self.parse_command(command)
self.execute_command(result)
if isinstance(result, Tree):
return False
if result.get('type') == 'meta' and result.get('command') in ['q', 'quit', 'exit']:
return True
except KeyboardInterrupt:
print("\nUse '\\q' to quit")
except EOFError:
print("\nGoodbye!")
return True
return False
def emptyline(self) -> bool:
return False
def default(self, line: str) -> bool:
return self.onecmd(line)
def parse_command(self, command_str: str) -> dict[str, str] | Tree[Token]:
if not command_str.strip(): if not command_str.strip():
return {'type': 'empty'} return {'type': 'empty'}
@ -209,16 +244,6 @@ class AdminCommandParser:
except Exception as e: except Exception as e:
return {'type': 'error', 'message': f'Parse error: {str(e)}'} return {'type': 'error', 'message': f'Parse error: {str(e)}'}
class AdminCLI:
def __init__(self):
self.parser = AdminCommandParser()
self.is_interactive = False
self.admin_account = "admin@ragflow.io"
self.admin_password: str = "admin"
self.host: str = ""
self.port: int = 0
def verify_admin(self, args): def verify_admin(self, args):
conn_info = self._parse_connection_args(args) conn_info = self._parse_connection_args(args)
@ -323,7 +348,7 @@ class AdminCLI:
continue continue
print(f"command: {command}") print(f"command: {command}")
result = self.parser.parse_command(command) result = self.parse_command(command)
self.execute_command(result) self.execute_command(result)
if isinstance(result, Tree): if isinstance(result, Tree):
@ -610,10 +635,17 @@ def main():
/_/ |_/_/ |_\____/_/ /_/\____/|__/|__/ /_/ |_\__,_/_/ /_/ /_/_/_/ /_/ /_/ |_/_/ |_\____/_/ /_/\____/|__/|__/ /_/ |_\__,_/_/ /_/ /_/_/_/ /_/
""") """)
if cli.verify_admin(sys.argv): if cli.verify_admin(sys.argv):
cli.run_interactive() cli.cmdloop()
else: else:
print(r"""
____ ___ ______________ ___ __ _
/ __ \/ | / ____/ ____/ /___ _ __ / | ____/ /___ ___ (_)___
/ /_/ / /| |/ / __/ /_ / / __ \ | /| / / / /| |/ __ / __ `__ \/ / __ \
/ _, _/ ___ / /_/ / __/ / / /_/ / |/ |/ / / ___ / /_/ / / / / / / / / / /
/_/ |_/_/ |_\____/_/ /_/\____/|__/|__/ /_/ |_\__,_/_/ /_/ /_/_/_/ /_/
""")
if cli.verify_admin(sys.argv): if cli.verify_admin(sys.argv):
cli.run_interactive() cli.cmdloop()
# cli.run_single_command(sys.argv[1:]) # cli.run_single_command(sys.argv[1:])

47
admin/build_cli_release.sh Executable file
View File

@ -0,0 +1,47 @@
#!/bin/bash
set -e
echo "🚀 Start building..."
echo "================================"
PROJECT_NAME="ragflow-cli"
RELEASE_DIR="release"
BUILD_DIR="dist"
SOURCE_DIR="src"
PACKAGE_DIR="ragflow_cli"
echo "🧹 Clean old build folder..."
rm -rf release/
echo "📁 Prepare source code..."
mkdir release/$PROJECT_NAME/$SOURCE_DIR -p
cp pyproject.toml release/$PROJECT_NAME/pyproject.toml
cp README.md release/$PROJECT_NAME/README.md
mkdir release/$PROJECT_NAME/$SOURCE_DIR/$PACKAGE_DIR -p
cp admin_client.py release/$PROJECT_NAME/$SOURCE_DIR/$PACKAGE_DIR/admin_client.py
if [ -d "release/$PROJECT_NAME/$SOURCE_DIR" ]; then
echo "✅ source dir: release/$PROJECT_NAME/$SOURCE_DIR"
else
echo "❌ source dir not exist: release/$PROJECT_NAME/$SOURCE_DIR"
exit 1
fi
echo "🔨 Make build file..."
cd release/$PROJECT_NAME
export PYTHONPATH=$(pwd)
python -m build
echo "✅ check build result..."
if [ -d "$BUILD_DIR" ]; then
echo "📦 Package generated:"
ls -la $BUILD_DIR/
else
echo "❌ Build Failed: $BUILD_DIR not exist."
exit 1
fi
echo "🎉 Build finished successfully!"

24
admin/pyproject.toml Normal file
View File

@ -0,0 +1,24 @@
[project]
name = "ragflow-cli"
version = "0.21.0.dev2"
description = "Admin Service's client of [RAGFlow](https://github.com/infiniflow/ragflow). The Admin Service provides user management and system monitoring. "
authors = [{ name = "Lynn", email = "lynn_inf@hotmail.com" }]
license = { text = "Apache License, Version 2.0" }
readme = "README.md"
requires-python = ">=3.10,<3.13"
dependencies = [
"requests>=2.30.0,<3.0.0",
"beartype>=0.18.5,<0.19.0",
"pycryptodomex>=3.10.0",
"lark>=1.1.0",
]
[dependency-groups]
test = [
"pytest>=8.3.5",
"requests>=2.32.3",
"requests-toolbelt>=1.0.0",
]
[project.scripts]
ragflow-cli = "ragflow_cli.admin_client:main"

View File

@ -177,8 +177,17 @@ class ServiceMgr:
def get_all_services(): def get_all_services():
result = [] result = []
configs = SERVICE_CONFIGS.configs configs = SERVICE_CONFIGS.configs
for config in configs: for service_id, config in enumerate(configs):
result.append(config.to_dict()) config_dict = config.to_dict()
try:
service_detail = ServiceMgr.get_service_details(service_id)
if service_detail['alive']:
config_dict['status'] = 'Alive'
else:
config_dict['status'] = 'Timeout'
except Exception:
config_dict['status'] = 'Timeout'
result.append(config_dict)
return result return result
@staticmethod @staticmethod

View File

@ -568,7 +568,7 @@ def change_parser():
def reset_doc(): def reset_doc():
nonlocal doc nonlocal doc
e = DocumentService.update_by_id(doc.id, {"parser_id": req["parser_id"], "progress": 0, "progress_msg": "", "run": TaskStatus.UNSTART.value}) e = DocumentService.update_by_id(doc.id, {"pipeline_id": req["pipeline_id"], "parser_id": req["parser_id"], "progress": 0, "progress_msg": "", "run": TaskStatus.UNSTART.value})
if not e: if not e:
return get_data_error_result(message="Document not found!") return get_data_error_result(message="Document not found!")
if doc.token_num > 0: if doc.token_num > 0:

View File

@ -397,9 +397,10 @@ class KnowledgebaseService(CommonService):
else: else:
kbs = kbs.order_by(cls.model.getter_by(orderby).asc()) kbs = kbs.order_by(cls.model.getter_by(orderby).asc())
total = kbs.count()
kbs = kbs.paginate(page_number, items_per_page) kbs = kbs.paginate(page_number, items_per_page)
return list(kbs.dicts()), kbs.count() return list(kbs.dicts()), total
@classmethod @classmethod
@DB.connection_context() @DB.connection_context()

View File

@ -151,10 +151,12 @@ def get_data_error_result(code=settings.RetCode.DATA_ERROR, message="Sorry! Data
def server_error_response(e): def server_error_response(e):
logging.exception(e) logging.exception(e)
try: try:
if e.code == 401: msg = repr(e).lower()
return get_json_result(code=401, message=repr(e)) if getattr(e, "code", None) == 401 or ("unauthorized" in msg) or ("401" in msg):
except BaseException: return get_json_result(code=settings.RetCode.UNAUTHORIZED, message=repr(e))
pass except Exception as ex:
logging.warning(f"error checking authorization: {ex}")
if len(e.args) > 1: if len(e.args) > 1:
try: try:
serialized_data = serialize_for_json(e.args[1]) serialized_data = serialize_for_json(e.args[1])

View File

@ -40,19 +40,9 @@ Each RAGFlow release is available in two editions:
RAGFlow offers two Docker image editions, `v0.20.5-slim` and `v0.20.5`: RAGFlow offers two Docker image editions, `v0.20.5-slim` and `v0.20.5`:
- `infiniflow/ragflow:v0.20.5-slim` (default): The RAGFlow Docker image without embedding models. - `infiniflow/ragflow:v0.20.5-slim` (default): The RAGFlow Docker image without embedding models.
- `infiniflow/ragflow:v0.20.5`: The RAGFlow Docker image with embedding models including: - `infiniflow/ragflow:v0.20.5`: The RAGFlow Docker image with the following built-in embedding models:
- Built-in embedding models:
- `BAAI/bge-large-zh-v1.5` - `BAAI/bge-large-zh-v1.5`
- `maidalun1020/bce-embedding-base_v1` - `maidalun1020/bce-embedding-base_v1`
- Embedding models that will be downloaded once you select them in the RAGFlow UI:
- `BAAI/bge-base-en-v1.5`
- `BAAI/bge-large-en-v1.5`
- `BAAI/bge-small-en-v1.5`
- `BAAI/bge-small-zh-v1.5`
- `jinaai/jina-embeddings-v2-base-en`
- `jinaai/jina-embeddings-v2-small-en`
- `nomic-ai/nomic-embed-text-v1.5`
- `sentence-transformers/all-MiniLM-L6-v2`
--- ---

View File

@ -12,33 +12,50 @@ The Admin CLI and Admin Service form a client-server architectural suite for RAG
## Starting the Admin Service ### Starting the Admin Service
#### Launching from source code
1. Before start Admin Service, please make sure RAGFlow system is already started. 1. Before start Admin Service, please make sure RAGFlow system is already started.
2. Switch to ragflow/ directory and run the service script:
```bash 2. Launch from source code:
source .venv/bin/activate
export PYTHONPATH=$(pwd)
python admin/admin_server.py
```
The service will start and listen for incoming connections from the CLI on the configured port. Default port is 9381. ```bash
python admin/admin_server.py
```
The service will start and listen for incoming connections from the CLI on the configured port.
#### Using docker image
1. Before startup, please configure the `docker_compose.yml` file to enable admin server:
```bash
command:
- --enable-adminserver
```
2. Start the containers, the service will start and listen for incoming connections from the CLI on the configured port.
## Using the Admin CLI ### Using the Admin CLI
1. Ensure the Admin Service is running. 1. Ensure the Admin Service is running.
2. Launch the CLI client:
```bash 2. Install ragflow-cli.
source .venv/bin/activate
export PYTHONPATH=$(pwd)
python admin/admin_client.py -h 0.0.0.0 -p 9381
```
Enter superuser's password to login. Default password is `admin`. ```bash
pip install ragflow-cli
```
3. Launch the CLI client:
```bash
ragflow-cli -h 0.0.0.0 -p 9381
```
Enter superuser's password to login. Default password is `admin`.
@ -121,16 +138,16 @@ Commands are case-insensitive and must be terminated with a semicolon(;).
admin> list services; admin> list services;
command: list services; command: list services;
Listing all services Listing all services
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+ +-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+---------+
| extra | host | id | name | port | service_type | | extra | host | id | name | port | service_type | status |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+ +-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+---------+
| {} | 0.0.0.0 | 0 | ragflow_0 | 9380 | ragflow_server | | {} | 0.0.0.0 | 0 | ragflow_0 | 9380 | ragflow_server | Timeout |
| {'meta_type': 'mysql', 'password': 'infini_rag_flow', 'username': 'root'} | localhost | 1 | mysql | 5455 | meta_data | | {'meta_type': 'mysql', 'password': 'infini_rag_flow', 'username': 'root'} | localhost | 1 | mysql | 5455 | meta_data | Alive |
| {'password': 'infini_rag_flow', 'store_type': 'minio', 'user': 'rag_flow'} | localhost | 2 | minio | 9000 | file_store | | {'password': 'infini_rag_flow', 'store_type': 'minio', 'user': 'rag_flow'} | localhost | 2 | minio | 9000 | file_store | Alive |
| {'password': 'infini_rag_flow', 'retrieval_type': 'elasticsearch', 'username': 'elastic'} | localhost | 3 | elasticsearch | 1200 | retrieval | | {'password': 'infini_rag_flow', 'retrieval_type': 'elasticsearch', 'username': 'elastic'} | localhost | 3 | elasticsearch | 1200 | retrieval | Alive |
| {'db_name': 'default_db', 'retrieval_type': 'infinity'} | localhost | 4 | infinity | 23817 | retrieval | | {'db_name': 'default_db', 'retrieval_type': 'infinity'} | localhost | 4 | infinity | 23817 | retrieval | Timeout |
| {'database': 1, 'mq_type': 'redis', 'password': 'infini_rag_flow'} | localhost | 5 | redis | 6379 | message_queue | | {'database': 1, 'mq_type': 'redis', 'password': 'infini_rag_flow'} | localhost | 5 | redis | 6379 | message_queue | Alive |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+ +-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+---------+
``` ```

View File

@ -704,10 +704,9 @@ print("Async bulk parsing initiated.")
DataSet.parse_documents(document_ids: list[str]) -> list[tuple[str, str, int, int]] DataSet.parse_documents(document_ids: list[str]) -> list[tuple[str, str, int, int]]
``` ```
Parses documents **synchronously** in the current dataset. *Asynchronously* parses documents in the current dataset.
This method wraps `async_parse_documents()` and automatically waits for all parsing tasks to complete.
It returns detailed parsing results, including the status and statistics for each document. This method encapsulates `async_parse_documents()`. It awaits the completion of all parsing tasks before returning detailed results, including the parsing status and statistics for each document. If a keyboard interruption occurs (e.g., `Ctrl+C`), all pending parsing tasks will be cancelled gracefully.
If interrupted by the user (e.g. `Ctrl+C`), all pending parsing jobs will be cancelled gracefully.
#### Parameters #### Parameters
@ -718,15 +717,16 @@ The IDs of the documents to parse.
#### Returns #### Returns
A list of tuples with detailed parsing results: A list of tuples with detailed parsing results:
```python ```python
[ [
(document_id: str, status: str, chunk_count: int, token_count: int), (document_id: str, status: str, chunk_count: int, token_count: int),
... ...
] ]
``` ```
- **status** — Final parsing state (`success`, `failed`, `cancelled`, etc.) - `status`: The final parsing state (e.g., `success`, `failed`, `cancelled`).
- **chunk_count** — Number of content chunks created for the document. - `chunk_count`: The number of content chunks created from the document.
- **token_count** — Total number of tokens processed. - `token_count`: The total number of tokens processed.
--- ---

View File

@ -580,7 +580,7 @@ Released on September 30, 2024.
### Compatibility changes ### Compatibility changes
From this release onwards, RAGFlow offers slim editions of its Docker images to improve the experience for users with limited Internet access. A slim edition of RAGFlow's Docker image does not include built-in BGE/BCE embedding models and has a size of about 1GB; a full edition of RAGFlow is approximately 9GB and includes both built-in embedding models and embedding models that will be downloaded once you select them in the RAGFlow UI. From this release onwards, RAGFlow offers slim editions of its Docker images to improve the experience for users with limited Internet access. A slim edition of RAGFlow's Docker image does not include built-in BGE/BCE embedding models and has a size of about 1GB; a full edition of RAGFlow is approximately 9GB and includes two built-in embedding models.
The default Docker image edition is `nightly-slim`. The following list clarifies the differences between various editions: The default Docker image edition is `nightly-slim`. The following list clarifies the differences between various editions:

View File

@ -166,7 +166,7 @@ class HierarchicalMerger(ProcessBase):
img = None img = None
for i in path: for i in path:
txt += lines[i] + "\n" txt += lines[i] + "\n"
concat_img(img, id2image(section_images[i], partial(STORAGE_IMPL.get))) concat_img(img, id2image(section_images[i], partial(STORAGE_IMPL.get, tenant_id=self._canvas._tenant_id)))
cks.append(txt) cks.append(txt)
images.append(img) images.append(img)
@ -180,7 +180,7 @@ class HierarchicalMerger(ProcessBase):
] ]
async with trio.open_nursery() as nursery: async with trio.open_nursery() as nursery:
for d in cks: for d in cks:
nursery.start_soon(image2id, d, partial(STORAGE_IMPL.put), get_uuid()) nursery.start_soon(image2id, d, partial(STORAGE_IMPL.put, tenant_id=self._canvas._tenant_id), get_uuid())
self.set_output("chunks", cks) self.set_output("chunks", cks)
self.callback(1, "Done.") self.callback(1, "Done.")

View File

@ -512,4 +512,4 @@ class Parser(ProcessBase):
outs = self.output() outs = self.output()
async with trio.open_nursery() as nursery: async with trio.open_nursery() as nursery:
for d in outs.get("json", []): for d in outs.get("json", []):
nursery.start_soon(image2id, d, partial(STORAGE_IMPL.put), get_uuid()) nursery.start_soon(image2id, d, partial(STORAGE_IMPL.put, tenant_id=self._canvas._tenant_id), get_uuid())

View File

@ -87,7 +87,7 @@ class Splitter(ProcessBase):
sections, section_images = [], [] sections, section_images = [], []
for o in from_upstream.json_result or []: for o in from_upstream.json_result or []:
sections.append((o.get("text", ""), o.get("position_tag", ""))) sections.append((o.get("text", ""), o.get("position_tag", "")))
section_images.append(id2image(o.get("img_id"), partial(STORAGE_IMPL.get))) section_images.append(id2image(o.get("img_id"), partial(STORAGE_IMPL.get, tenant_id=self._canvas._tenant_id)))
chunks, images = naive_merge_with_images( chunks, images = naive_merge_with_images(
sections, sections,
@ -106,6 +106,6 @@ class Splitter(ProcessBase):
] ]
async with trio.open_nursery() as nursery: async with trio.open_nursery() as nursery:
for d in cks: for d in cks:
nursery.start_soon(image2id, d, partial(STORAGE_IMPL.put), get_uuid()) nursery.start_soon(image2id, d, partial(STORAGE_IMPL.put, tenant_id=self._canvas._tenant_id), get_uuid())
self.set_output("chunks", cks) self.set_output("chunks", cks)
self.callback(1, "Done.") self.callback(1, "Done.")

View File

@ -680,8 +680,7 @@ async def gen_toc_from_text(txt_info: dict, chat_mdl, callback=None):
chat_mdl, chat_mdl,
gen_conf={"temperature": 0.0, "top_p": 0.9} gen_conf={"temperature": 0.0, "top_p": 0.9}
) )
print(ans, "::::::::::::::::::::::::::::::::::::", flush=True) txt_info["toc"] = ans if ans and not isinstance(ans, str) else []
txt_info["toc"] = ans if ans else []
if callback: if callback:
callback(msg="") callback(msg="")
except Exception as e: except Exception as e:
@ -729,8 +728,6 @@ async def run_toc_from_text(chunks, chat_mdl, callback=None):
for chunk in chunks_res: for chunk in chunks_res:
titles.extend(chunk.get("toc", [])) titles.extend(chunk.get("toc", []))
print(titles, ">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>")
# Filter out entries with title == -1 # Filter out entries with title == -1
prune = len(titles) > 512 prune = len(titles) > 512
max_len = 12 if prune else 22 max_len = 12 if prune else 22
@ -745,12 +742,16 @@ async def run_toc_from_text(chunks, chat_mdl, callback=None):
filtered.append(x) filtered.append(x)
logging.info(f"\n\nFiltered TOC sections:\n{filtered}") logging.info(f"\n\nFiltered TOC sections:\n{filtered}")
if not filtered:
return []
# Generate initial level (level/title) # Generate initial level (level/title)
raw_structure = [x.get("title", "") for x in filtered] raw_structure = [x.get("title", "") for x in filtered]
# Assign hierarchy levels using LLM # Assign hierarchy levels using LLM
toc_with_levels = assign_toc_levels(raw_structure, chat_mdl, {"temperature": 0.0, "top_p": 0.9}) toc_with_levels = assign_toc_levels(raw_structure, chat_mdl, {"temperature": 0.0, "top_p": 0.9})
if not toc_with_levels:
return []
# Merge structure and content (by index) # Merge structure and content (by index)
prune = len(toc_with_levels) > 512 prune = len(toc_with_levels) > 512
@ -779,7 +780,6 @@ def relevant_chunks_with_toc(query: str, toc:list[dict], chat_mdl, topn: int=6):
chat_mdl, chat_mdl,
gen_conf={"temperature": 0.0, "top_p": 0.9} gen_conf={"temperature": 0.0, "top_p": 0.9}
) )
print(ans, "::::::::::::::::::::::::::::::::::::", flush=True)
id2score = {} id2score = {}
for ti, sc in zip(toc, ans): for ti, sc in zip(toc, ans):
if not isinstance(sc, dict) or sc.get("score", -1) < 1: if not isinstance(sc, dict) or sc.get("score", -1) < 1:

View File

@ -12,7 +12,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import concurrent
# from beartype import BeartypeConf # from beartype import BeartypeConf
# from beartype.claw import beartype_all # <-- you didn't sign up for this # from beartype.claw import beartype_all # <-- you didn't sign up for this
# beartype_all(conf=BeartypeConf(violation_type=UserWarning)) # <-- emit warnings from all code # beartype_all(conf=BeartypeConf(violation_type=UserWarning)) # <-- emit warnings from all code
@ -317,7 +317,7 @@ async def build_chunks(task, progress_callback):
d["img_id"] = "" d["img_id"] = ""
docs.append(d) docs.append(d)
return return
await image2id(d, partial(STORAGE_IMPL.put), d["id"], task["kb_id"]) await image2id(d, partial(STORAGE_IMPL.put, tenant_id=task["tenant_id"]), d["id"], task["kb_id"])
docs.append(d) docs.append(d)
except Exception: except Exception:
logging.exception( logging.exception(
@ -370,38 +370,6 @@ async def build_chunks(task, progress_callback):
nursery.start_soon(doc_question_proposal, chat_mdl, d, task["parser_config"]["auto_questions"]) nursery.start_soon(doc_question_proposal, chat_mdl, d, task["parser_config"]["auto_questions"])
progress_callback(msg="Question generation {} chunks completed in {:.2f}s".format(len(docs), timer() - st)) progress_callback(msg="Question generation {} chunks completed in {:.2f}s".format(len(docs), timer() - st))
if task["parser_id"].lower() == "naive" and task["parser_config"].get("toc_extraction", False):
progress_callback(msg="Start to generate table of content ...")
chat_mdl = LLMBundle(task["tenant_id"], LLMType.CHAT, llm_name=task["llm_id"], lang=task["language"])
docs = sorted(docs, key=lambda d:(
d.get("page_num_int", 0)[0] if isinstance(d.get("page_num_int", 0), list) else d.get("page_num_int", 0),
d.get("top_int", 0)[0] if isinstance(d.get("top_int", 0), list) else d.get("top_int", 0)
))
toc: list[dict] = await run_toc_from_text([d["content_with_weight"] for d in docs], chat_mdl, progress_callback)
logging.info("------------ T O C -------------\n"+json.dumps(toc, ensure_ascii=False, indent=' '))
ii = 0
while ii < len(toc):
try:
idx = int(toc[ii]["chunk_id"])
del toc[ii]["chunk_id"]
toc[ii]["ids"] = [docs[idx]["id"]]
if ii == len(toc) -1:
break
for jj in range(idx+1, int(toc[ii+1]["chunk_id"])+1):
toc[ii]["ids"].append(docs[jj]["id"])
except Exception as e:
logging.exception(e)
ii += 1
if toc:
d = copy.deepcopy(docs[-1])
d["content_with_weight"] = json.dumps(toc, ensure_ascii=False)
d["toc_kwd"] = "toc"
d["available_int"] = 0
d["page_num_int"] = 100000000
d["id"] = xxhash.xxh64((d["content_with_weight"] + str(d["doc_id"])).encode("utf-8", "surrogatepass")).hexdigest()
docs.append(d)
if task["kb_parser_config"].get("tag_kb_ids", []): if task["kb_parser_config"].get("tag_kb_ids", []):
progress_callback(msg="Start to tag for every chunk ...") progress_callback(msg="Start to tag for every chunk ...")
kb_ids = task["kb_parser_config"]["tag_kb_ids"] kb_ids = task["kb_parser_config"]["tag_kb_ids"]
@ -451,6 +419,39 @@ async def build_chunks(task, progress_callback):
return docs return docs
def build_TOC(task, docs, progress_callback):
progress_callback(msg="Start to generate table of content ...")
chat_mdl = LLMBundle(task["tenant_id"], LLMType.CHAT, llm_name=task["llm_id"], lang=task["language"])
docs = sorted(docs, key=lambda d:(
d.get("page_num_int", 0)[0] if isinstance(d.get("page_num_int", 0), list) else d.get("page_num_int", 0),
d.get("top_int", 0)[0] if isinstance(d.get("top_int", 0), list) else d.get("top_int", 0)
))
toc: list[dict] = trio.run(run_toc_from_text, [d["content_with_weight"] for d in docs], chat_mdl, progress_callback)
logging.info("------------ T O C -------------\n"+json.dumps(toc, ensure_ascii=False, indent=' '))
ii = 0
while ii < len(toc):
try:
idx = int(toc[ii]["chunk_id"])
del toc[ii]["chunk_id"]
toc[ii]["ids"] = [docs[idx]["id"]]
if ii == len(toc) -1:
break
for jj in range(idx+1, int(toc[ii+1]["chunk_id"])+1):
toc[ii]["ids"].append(docs[jj]["id"])
except Exception as e:
logging.exception(e)
ii += 1
if toc:
d = copy.deepcopy(docs[-1])
d["content_with_weight"] = json.dumps(toc, ensure_ascii=False)
d["toc_kwd"] = "toc"
d["available_int"] = 0
d["page_num_int"] = 100000000
d["id"] = xxhash.xxh64((d["content_with_weight"] + str(d["doc_id"])).encode("utf-8", "surrogatepass")).hexdigest()
return d
def init_kb(row, vector_size: int): def init_kb(row, vector_size: int):
idxnm = search.index_name(row["tenant_id"]) idxnm = search.index_name(row["tenant_id"])
return settings.docStoreConn.createIdx(idxnm, row.get("kb_id", ""), vector_size) return settings.docStoreConn.createIdx(idxnm, row.get("kb_id", ""), vector_size)
@ -753,7 +754,7 @@ async def insert_es(task_id, task_tenant_id, task_dataset_id, chunks, progress_c
return True return True
@timeout(60*60*2, 1) @timeout(60*60*3, 1)
async def do_handle_task(task): async def do_handle_task(task):
task_type = task.get("task_type", "") task_type = task.get("task_type", "")
@ -773,6 +774,8 @@ async def do_handle_task(task):
task_document_name = task["name"] task_document_name = task["name"]
task_parser_config = task["parser_config"] task_parser_config = task["parser_config"]
task_start_ts = timer() task_start_ts = timer()
toc_thread = None
executor = concurrent.futures.ThreadPoolExecutor()
# prepare the progress callback function # prepare the progress callback function
progress_callback = partial(set_progress, task_id, task_from_page, task_to_page) progress_callback = partial(set_progress, task_id, task_from_page, task_to_page)
@ -905,8 +908,6 @@ async def do_handle_task(task):
if not chunks: if not chunks:
progress_callback(1., msg=f"No chunk built from {task_document_name}") progress_callback(1., msg=f"No chunk built from {task_document_name}")
return return
# TODO: exception handler
## set_progress(task["did"], -1, "ERROR: ")
progress_callback(msg="Generate {} chunks".format(len(chunks))) progress_callback(msg="Generate {} chunks".format(len(chunks)))
start_ts = timer() start_ts = timer()
try: try:
@ -920,6 +921,8 @@ async def do_handle_task(task):
progress_message = "Embedding chunks ({:.2f}s)".format(timer() - start_ts) progress_message = "Embedding chunks ({:.2f}s)".format(timer() - start_ts)
logging.info(progress_message) logging.info(progress_message)
progress_callback(msg=progress_message) progress_callback(msg=progress_message)
if task["parser_id"].lower() == "naive" and task["parser_config"].get("toc_extraction", False):
toc_thread = executor.submit(build_TOC,task, chunks, progress_callback)
chunk_count = len(set([chunk["id"] for chunk in chunks])) chunk_count = len(set([chunk["id"] for chunk in chunks]))
start_ts = timer() start_ts = timer()
@ -934,8 +937,17 @@ async def do_handle_task(task):
DocumentService.increment_chunk_num(task_doc_id, task_dataset_id, token_count, chunk_count, 0) DocumentService.increment_chunk_num(task_doc_id, task_dataset_id, token_count, chunk_count, 0)
time_cost = timer() - start_ts time_cost = timer() - start_ts
progress_callback(msg="Indexing done ({:.2f}s).".format(time_cost))
if toc_thread:
d = toc_thread.result()
if d:
e = await insert_es(task_id, task_tenant_id, task_dataset_id, [d], progress_callback)
if not e:
return
DocumentService.increment_chunk_num(task_doc_id, task_dataset_id, 0, 1, 0)
task_time_cost = timer() - task_start_ts task_time_cost = timer() - task_start_ts
progress_callback(prog=1.0, msg="Indexing done ({:.2f}s). Task done ({:.2f}s)".format(time_cost, task_time_cost)) progress_callback(prog=1.0, msg="Task done ({:.2f}s)".format(task_time_cost))
logging.info( logging.info(
"Chunk doc({}), page({}-{}), chunks({}), token({}), elapsed:{:.2f}".format(task_document_name, task_from_page, "Chunk doc({}), page({}-{}), chunks({}), token({}), elapsed:{:.2f}".format(task_document_name, task_from_page,
task_to_page, len(chunks), task_to_page, len(chunks),

View File

@ -60,7 +60,7 @@ class RAGFlowMinio:
) )
return r return r
def put(self, bucket, fnm, binary): def put(self, bucket, fnm, binary, tenant_id=None):
for _ in range(3): for _ in range(3):
try: try:
if not self.conn.bucket_exists(bucket): if not self.conn.bucket_exists(bucket):
@ -76,13 +76,13 @@ class RAGFlowMinio:
self.__open__() self.__open__()
time.sleep(1) time.sleep(1)
def rm(self, bucket, fnm): def rm(self, bucket, fnm, tenant_id=None):
try: try:
self.conn.remove_object(bucket, fnm) self.conn.remove_object(bucket, fnm)
except Exception: except Exception:
logging.exception(f"Fail to remove {bucket}/{fnm}:") logging.exception(f"Fail to remove {bucket}/{fnm}:")
def get(self, bucket, filename): def get(self, bucket, filename, tenant_id=None):
for _ in range(1): for _ in range(1):
try: try:
r = self.conn.get_object(bucket, filename) r = self.conn.get_object(bucket, filename)
@ -93,7 +93,7 @@ class RAGFlowMinio:
time.sleep(1) time.sleep(1)
return return
def obj_exist(self, bucket, filename): def obj_exist(self, bucket, filename, tenant_id=None):
try: try:
if not self.conn.bucket_exists(bucket): if not self.conn.bucket_exists(bucket):
return False return False
@ -121,7 +121,7 @@ class RAGFlowMinio:
logging.exception(f"bucket_exist {bucket} got exception") logging.exception(f"bucket_exist {bucket} got exception")
return False return False
def get_presigned_url(self, bucket, fnm, expires): def get_presigned_url(self, bucket, fnm, expires, tenant_id=None):
for _ in range(10): for _ in range(10):
try: try:
return self.conn.get_presigned_url("GET", bucket, fnm, expires) return self.conn.get_presigned_url("GET", bucket, fnm, expires)

View File

@ -104,7 +104,7 @@ const RootProvider = ({ children }: React.PropsWithChildren) => {
<TooltipProvider> <TooltipProvider>
<QueryClientProvider client={queryClient}> <QueryClientProvider client={queryClient}>
<ThemeProvider <ThemeProvider
defaultTheme={ThemeEnum.Light} defaultTheme={ThemeEnum.Dark}
storageKey="ragflow-ui-theme" storageKey="ragflow-ui-theme"
> >
<Root>{children}</Root> <Root>{children}</Root>

View File

@ -108,6 +108,7 @@ export function DataFlowSelect(props: IProps) {
{...field} {...field}
placeholder={t('dataFlowPlaceholder')} placeholder={t('dataFlowPlaceholder')}
options={options} options={options}
triggerClassName="!bg-bg-base"
/> />
)} )}
{isMult && ( {isMult && (

View File

@ -1,3 +1,4 @@
import { cn } from '@/lib/utils';
import { forwardRef } from 'react'; import { forwardRef } from 'react';
import { useFormContext } from 'react-hook-form'; import { useFormContext } from 'react-hook-form';
import { useTranslation } from 'react-i18next'; import { useTranslation } from 'react-i18next';
@ -36,6 +37,7 @@ export const DelimiterInput = forwardRef<HTMLInputElement, InputProps & IProps>(
maxLength={maxLength} maxLength={maxLength}
defaultValue={defaultValue} defaultValue={defaultValue}
ref={ref} ref={ref}
className={cn('bg-bg-base', props.className)}
{...props} {...props}
></Input> ></Input>
); );

View File

@ -54,7 +54,10 @@ function MarkdownContent({
const { setDocumentIds, data: fileThumbnails } = const { setDocumentIds, data: fileThumbnails } =
useFetchDocumentThumbnailsByIds(); useFetchDocumentThumbnailsByIds();
const contentWithCursor = useMemo(() => { const contentWithCursor = useMemo(() => {
let text = DOMPurify.sanitize(content); let text = DOMPurify.sanitize(content, {
ADD_TAGS: ['think', 'section'],
ADD_ATTR: ['class'],
});
// let text = content; // let text = content;
if (text === '') { if (text === '') {
text = t('chat.searching'); text = t('chat.searching');

View File

@ -21,7 +21,7 @@ const ThemeProviderContext = createContext<ThemeProviderState>(initialState);
export function ThemeProvider({ export function ThemeProvider({
children, children,
defaultTheme = ThemeEnum.Light, defaultTheme = ThemeEnum.Dark,
storageKey = 'vite-ui-theme', storageKey = 'vite-ui-theme',
...props ...props
}: ThemeProviderProps) { }: ThemeProviderProps) {

View File

@ -31,7 +31,7 @@ const Input = React.forwardRef<HTMLInputElement, InputProps>(
<input <input
type={type} type={type}
className={cn( className={cn(
'flex h-8 w-full rounded-md border border-input bg-bg-card px-2 py-2 text-sm ring-offset-background file:border-0 file:bg-transparent file:text-sm file:font-medium file:text-foreground placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50', 'flex h-8 w-full rounded-md border border-input bg-bg-base px-2 py-2 text-sm ring-offset-background file:border-0 file:bg-transparent file:text-sm file:font-medium file:text-foreground placeholder:text-text-disabled focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50',
className, className,
)} )}
ref={ref} ref={ref}
@ -65,7 +65,11 @@ const ExpandedInput = ({
{prefix} {prefix}
</span> </span>
<Input <Input
className={cn({ 'pr-8': !!suffix, 'pl-8': !!prefix }, className)} className={cn(
{ 'pr-8': !!suffix, 'pl-8': !!prefix },
'bg-bg-base',
className,
)}
{...props} {...props}
></Input> ></Input>
<span <span

View File

@ -291,7 +291,7 @@ export const RAGFlowSelect = forwardRef<
onReset={handleReset} onReset={handleReset}
allowClear={allowClear} allowClear={allowClear}
ref={ref} ref={ref}
className={triggerClassName} className={cn(triggerClassName, 'bg-bg-base')}
> >
<SelectValue placeholder={placeholder}>{label}</SelectValue> <SelectValue placeholder={placeholder}>{label}</SelectValue>
</SelectTrigger> </SelectTrigger>

View File

@ -8,7 +8,7 @@ const Table = React.forwardRef<
>(({ className, rootClassName, ...props }, ref) => ( >(({ className, rootClassName, ...props }, ref) => (
<div <div
className={cn( className={cn(
'relative w-full overflow-auto rounded-2xl bg-bg-card scrollbar-none', 'relative w-full overflow-auto rounded-2xl bg-bg-card scrollbar-auto',
rootClassName, rootClassName,
)} )}
> >

View File

@ -20,7 +20,7 @@ const TooltipContent = React.forwardRef<
ref={ref} ref={ref}
sideOffset={sideOffset} sideOffset={sideOffset}
className={cn( className={cn(
'z-50 overflow-auto scrollbar-auto rounded-md border bg-popover px-3 py-1.5 text-sm text-popover-foreground shadow-md animate-in fade-in-0 zoom-in-95 data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=closed]:zoom-out-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 max-w-[20vw]', 'z-50 overflow-auto scrollbar-auto rounded-md whitespace-pre-wrap border bg-popover px-3 py-1.5 text-sm text-popover-foreground shadow-md animate-in fade-in-0 zoom-in-95 data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=closed]:zoom-out-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 max-w-[30vw]',
className, className,
)} )}
{...props} {...props}
@ -41,9 +41,7 @@ export const FormTooltip = ({ tooltip }: { tooltip: React.ReactNode }) => {
> >
<Info className="size-3 ml-2" /> <Info className="size-3 ml-2" />
</TooltipTrigger> </TooltipTrigger>
<TooltipContent> <TooltipContent>{tooltip}</TooltipContent>
<p>{tooltip}</p>
</TooltipContent>
</Tooltip> </Tooltip>
); );
}; };

View File

@ -24,6 +24,12 @@ export const useNavigatePage = () => {
}, },
[navigate], [navigate],
); );
const navigateToDatasetOverview = useCallback(
(id: string) => () => {
navigate(`${Routes.DatasetBase}${Routes.DataSetOverview}/${id}`);
},
[navigate],
);
const navigateToDataFile = useCallback( const navigateToDataFile = useCallback(
(id: string) => () => { (id: string) => () => {
@ -160,6 +166,7 @@ export const useNavigatePage = () => {
return { return {
navigateToDatasetList, navigateToDatasetList,
navigateToDataset, navigateToDataset,
navigateToDatasetOverview,
navigateToHome, navigateToHome,
navigateToProfile, navigateToProfile,
navigateToChatList, navigateToChatList,

View File

@ -300,8 +300,8 @@ export default {
dataFlowPlaceholder: 'Please select a pipeline.', dataFlowPlaceholder: 'Please select a pipeline.',
buildItFromScratch: 'Build it from scratch', buildItFromScratch: 'Build it from scratch',
dataFlow: 'Pipeline', dataFlow: 'Pipeline',
parseType: 'Parse Type', parseType: 'Ingestion pipeline',
manualSetup: 'Manual Setup', manualSetup: 'Choose pipeline',
builtIn: 'Built-in', builtIn: 'Built-in',
titleDescription: titleDescription:
'Update your knowledge base configuration here, particularly the chunking method.', 'Update your knowledge base configuration here, particularly the chunking method.',
@ -477,7 +477,8 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
useGraphRagTip: useGraphRagTip:
'Construct a knowledge graph over file chunks of the current knowledge base to enhance multi-hop question-answering involving nested logic. See https://ragflow.io/docs/dev/construct_knowledge_graph for details.', 'Construct a knowledge graph over file chunks of the current knowledge base to enhance multi-hop question-answering involving nested logic. See https://ragflow.io/docs/dev/construct_knowledge_graph for details.',
graphRagMethod: 'Method', graphRagMethod: 'Method',
graphRagMethodTip: `Light: (Default) Use prompts provided by github.com/HKUDS/LightRAG to extract entities and relationships. This option consumes fewer tokens, less memory, and fewer computational resources.</br> graphRagMethodTip: `
Light: (Default) Use prompts provided by github.com/HKUDS/LightRAG to extract entities and relationships. This option consumes fewer tokens, less memory, and fewer computational resources.</br>
General: Use prompts provided by github.com/microsoft/graphrag to extract entities and relationships`, General: Use prompts provided by github.com/microsoft/graphrag to extract entities and relationships`,
resolution: 'Entity resolution', resolution: 'Entity resolution',
resolutionTip: `An entity deduplication switch. When enabled, the LLM will combine similar entities - e.g., '2025' and 'the year of 2025', or 'IT' and 'Information Technology' - to construct a more accurate graph`, resolutionTip: `An entity deduplication switch. When enabled, the LLM will combine similar entities - e.g., '2025' and 'the year of 2025', or 'IT' and 'Information Technology' - to construct a more accurate graph`,
@ -1672,6 +1673,7 @@ This delimiter is used to split the input text into several text pieces echo of
page: '{{page}} /Page', page: '{{page}} /Page',
}, },
dataflowParser: { dataflowParser: {
result: 'Result',
parseSummary: 'Parse Summary', parseSummary: 'Parse Summary',
parseSummaryTip: 'Parserdeepdoc', parseSummaryTip: 'Parserdeepdoc',
rerunFromCurrentStep: 'Rerun From Current Step', rerunFromCurrentStep: 'Rerun From Current Step',
@ -1736,10 +1738,10 @@ This delimiter is used to split the input text into several text pieces echo of
addParser: 'Add Parser', addParser: 'Add Parser',
hierarchy: 'Hierarchy', hierarchy: 'Hierarchy',
regularExpressions: 'Regular Expressions', regularExpressions: 'Regular Expressions',
overlappedPercent: 'Overlapped percent', overlappedPercent: 'Overlapped percent (%)',
searchMethod: 'Search method', searchMethod: 'Search method',
searchMethodTip: `Defines how the content can be searched — by full-text, embedding, or both. searchMethodTip: `Defines how the content can be searched — by full-text, embedding, or both.
The Tokenizer will store the content in the corresponding data structures for the selected methods.`, The Indexer will store the content in the corresponding data structures for the selected methods.`,
begin: 'File', begin: 'File',
parserMethod: 'Parsing method', parserMethod: 'Parsing method',
systemPrompt: 'System Prompt', systemPrompt: 'System Prompt',
@ -1748,11 +1750,11 @@ The Tokenizer will store the content in the corresponding data structures for th
exportJson: 'Export JSON', exportJson: 'Export JSON',
viewResult: 'View result', viewResult: 'View result',
running: 'Running', running: 'Running',
summary: 'Augmented Context', summary: 'Summary',
keywords: 'Keywords', keywords: 'Keywords',
questions: 'Questions', questions: 'Questions',
metadata: 'Metadata', metadata: 'Metadata',
fieldName: 'Result Destination', fieldName: 'Result destination',
prompts: { prompts: {
system: { system: {
keywords: `Role keywords: `Role
@ -1817,6 +1819,9 @@ Important structured information may include: names, dates, locations, events, k
imageParseMethodOptions: { imageParseMethodOptions: {
ocr: 'OCR', ocr: 'OCR',
}, },
note: 'Note',
noteDescription: 'Note',
notePlaceholder: 'Please enter a note',
}, },
datasetOverview: { datasetOverview: {
downloadTip: 'Files being downloaded from data sources. ', downloadTip: 'Files being downloaded from data sources. ',

View File

@ -268,25 +268,25 @@ export default {
<br/> <br/>
是否要继续? 是否要继续?
`, `,
extractRaptor: '从文档中提取Raptor', extractRaptor: '从文档中提取RAPTOR',
extractKnowledgeGraph: '从文档中提取知识图谱', extractKnowledgeGraph: '从文档中提取知识图谱',
filterPlaceholder: '请输入', filterPlaceholder: '请输入',
fileFilterTip: '', fileFilterTip: '',
fileFilter: '正则匹配表达式', fileFilter: '正则匹配表达式',
setDefaultTip: '', setDefaultTip: '',
setDefault: '设置默认', setDefault: '设置默认',
eidtLinkDataPipeline: '编辑数据流', eidtLinkDataPipeline: '编辑pipeline',
linkPipelineSetTip: '管理与此数据集的数据管道链接', linkPipelineSetTip: '管理与此数据集的数据管道链接',
default: '默认', default: '默认',
dataPipeline: '数据流', dataPipeline: 'pipeline',
linkDataPipeline: '关联数据流', linkDataPipeline: '关联pipeline',
enableAutoGenerate: '是否启用自动生成', enableAutoGenerate: '是否启用自动生成',
teamPlaceholder: '请选择团队', teamPlaceholder: '请选择团队',
dataFlowPlaceholder: '请选择数据流', dataFlowPlaceholder: '请选择pipeline',
buildItFromScratch: '去Scratch构建', buildItFromScratch: '去Scratch构建',
dataFlow: '数据流', dataFlow: 'pipeline',
parseType: '切片方法', parseType: 'Ingestion pipeline',
manualSetup: '手动设置', manualSetup: '选择pipeline',
builtIn: '内置', builtIn: '内置',
titleDescription: '在这里更新您的知识库详细信息,尤其是切片方法。', titleDescription: '在这里更新您的知识库详细信息,尤其是切片方法。',
name: '知识库名称', name: '知识库名称',
@ -1588,6 +1588,7 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
page: '{{page}}条/页', page: '{{page}}条/页',
}, },
dataflowParser: { dataflowParser: {
result: '结果',
parseSummary: '解析摘要', parseSummary: '解析摘要',
parseSummaryTip: '解析器: deepdoc', parseSummaryTip: '解析器: deepdoc',
rerunFromCurrentStep: '从当前步骤重新运行', rerunFromCurrentStep: '从当前步骤重新运行',
@ -1610,7 +1611,7 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
<p>要保留这些更改,请点击“重新运行”以重新运行当前阶段。</p> `, <p>要保留这些更改,请点击“重新运行”以重新运行当前阶段。</p> `,
changeStepModalConfirmText: '继续切换', changeStepModalConfirmText: '继续切换',
changeStepModalCancelText: '取消', changeStepModalCancelText: '取消',
unlinkPipelineModalTitle: '解绑数据流', unlinkPipelineModalTitle: '解绑pipeline',
unlinkPipelineModalContent: ` unlinkPipelineModalContent: `
<p>一旦取消链接,该数据集将不再连接到当前数据管道。</p> <p>一旦取消链接,该数据集将不再连接到当前数据管道。</p>
<p>正在解析的文件将继续解析,直到完成。</p> <p>正在解析的文件将继续解析,直到完成。</p>
@ -1641,7 +1642,7 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
addParser: '增加解析器', addParser: '增加解析器',
hierarchy: '层次结构', hierarchy: '层次结构',
regularExpressions: '正则表达式', regularExpressions: '正则表达式',
overlappedPercent: '重叠百分比', overlappedPercent: '重叠百分比%',
searchMethod: '搜索方法', searchMethod: '搜索方法',
searchMethodTip: `决定该数据集启用的搜索方式,可选择全文、向量,或两者兼有。 searchMethodTip: `决定该数据集启用的搜索方式,可选择全文、向量,或两者兼有。
Tokenizer 会根据所选方式将内容存储为对应的数据结构。`, Tokenizer 会根据所选方式将内容存储为对应的数据结构。`,
@ -1709,6 +1710,9 @@ Tokenizer 会根据所选方式将内容存储为对应的数据结构。`,
cancel: '取消', cancel: '取消',
filenameEmbeddingWeight: '文件名嵌入权重', filenameEmbeddingWeight: '文件名嵌入权重',
switchPromptMessage: '提示词将发生变化,请确认是否放弃已有提示词?', switchPromptMessage: '提示词将发生变化,请确认是否放弃已有提示词?',
note: '注释',
noteDescription: '注释',
notePlaceholder: '请输入注释',
}, },
datasetOverview: { datasetOverview: {
downloadTip: '正在从数据源下载文件。', downloadTip: '正在从数据源下载文件。',

View File

@ -18,7 +18,7 @@ const InnerNodeHeader = ({
wrapperClassName, wrapperClassName,
}: IProps) => { }: IProps) => {
return ( return (
<section className={cn(wrapperClassName, 'pb-4')}> <section className={cn(wrapperClassName, 'pb-2')}>
<div className={cn(className, 'flex gap-2.5')}> <div className={cn(className, 'flex gap-2.5')}>
<OperatorIcon name={label as Operator}></OperatorIcon> <OperatorIcon name={label as Operator}></OperatorIcon>
<span className="truncate text-center font-semibold text-sm"> <span className="truncate text-center font-semibold text-sm">

View File

@ -7,7 +7,7 @@ export function NodeWrapper({ children, className, selected }: IProps) {
return ( return (
<section <section
className={cn( className={cn(
'bg-text-title-invert p-2.5 rounded-sm w-[200px] text-xs group', 'bg-text-title-invert p-2.5 rounded-md w-[200px] text-xs group',
{ 'border border-accent-primary': selected }, { 'border border-accent-primary': selected },
className, className,
)} )}

View File

@ -28,7 +28,18 @@ const NameFormSchema = z.object({
name: z.string(), name: z.string(),
}); });
function NoteNode({ data, id, selected }: NodeProps<INoteNode>) { type NoteNodeProps = NodeProps<INoteNode> & {
useWatchNoteFormChange?: typeof useWatchFormChange;
useWatchNoteNameFormChange?: typeof useWatchNameFormChange;
};
function NoteNode({
data,
id,
selected,
useWatchNoteFormChange,
useWatchNoteNameFormChange,
}: NoteNodeProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const form = useForm<z.infer<typeof FormSchema>>({ const form = useForm<z.infer<typeof FormSchema>>({
@ -41,19 +52,19 @@ function NoteNode({ data, id, selected }: NodeProps<INoteNode>) {
defaultValues: { name: data.name }, defaultValues: { name: data.name },
}); });
useWatchFormChange(id, form); (useWatchNoteFormChange || useWatchFormChange)(id, form);
useWatchNameFormChange(id, nameForm); (useWatchNoteNameFormChange || useWatchNameFormChange)(id, nameForm);
return ( return (
<NodeWrapper <NodeWrapper
className="p-0 w-full h-full flex flex-col bg-bg-component" className="p-0 w-full h-full flex flex-col bg-bg-component border border-state-warning rounded-lg shadow-md pb-1"
selected={selected} selected={selected}
> >
<NodeResizeControl minWidth={190} minHeight={128} style={controlStyle}> <NodeResizeControl minWidth={190} minHeight={128} style={controlStyle}>
<ResizeIcon /> <ResizeIcon />
</NodeResizeControl> </NodeResizeControl>
<section className="p-2 flex gap-2 items-center note-drag-handle rounded-t"> <section className="px-2 py-1 flex gap-2 items-center note-drag-handle rounded-t border-t-2 border-state-warning">
<NotebookPen className="size-4" /> <NotebookPen className="size-4" />
<Form {...nameForm}> <Form {...nameForm}>
<form className="flex-1"> <form className="flex-1">
@ -67,7 +78,7 @@ function NoteNode({ data, id, selected }: NodeProps<INoteNode>) {
placeholder={t('flow.notePlaceholder')} placeholder={t('flow.notePlaceholder')}
{...field} {...field}
type="text" type="text"
className="bg-transparent border-none focus-visible:outline focus-visible:outline-text-sub-title" className="bg-transparent border-none focus-visible:outline focus-visible:outline-text-sub-title p-1"
/> />
</FormControl> </FormControl>
<FormMessage /> <FormMessage />
@ -78,7 +89,7 @@ function NoteNode({ data, id, selected }: NodeProps<INoteNode>) {
</Form> </Form>
</section> </section>
<Form {...form}> <Form {...form}>
<form className="flex-1 p-1"> <form className="flex-1 px-1 min-h-1">
<FormField <FormField
control={form.control} control={form.control}
name="text" name="text"
@ -87,7 +98,7 @@ function NoteNode({ data, id, selected }: NodeProps<INoteNode>) {
<FormControl> <FormControl>
<Textarea <Textarea
placeholder={t('flow.notePlaceholder')} placeholder={t('flow.notePlaceholder')}
className="resize-none rounded-none p-1 h-full overflow-auto bg-transparent focus-visible:ring-0 border-none" className="resize-none rounded-none p-1 py-0 overflow-auto bg-transparent focus-visible:ring-0 border-none text-text-secondary focus-visible:ring-offset-0 !text-xs"
{...field} {...field}
/> />
</FormControl> </FormControl>

View File

@ -6,7 +6,7 @@ export function ResizeIcon() {
height="14" height="14"
viewBox="0 0 24 24" viewBox="0 0 24 24"
strokeWidth="2" strokeWidth="2"
stroke="rgba(76, 164, 231, 1)" stroke="var(--text-disabled)"
fill="none" fill="none"
strokeLinecap="round" strokeLinecap="round"
strokeLinejoin="round" strokeLinejoin="round"

View File

@ -48,7 +48,11 @@ const MarkdownContent = ({
const { setDocumentIds, data: fileThumbnails } = const { setDocumentIds, data: fileThumbnails } =
useFetchDocumentThumbnailsByIds(); useFetchDocumentThumbnailsByIds();
const contentWithCursor = useMemo(() => { const contentWithCursor = useMemo(() => {
let text = DOMPurify.sanitize(content); let text = DOMPurify.sanitize(content, {
ADD_TAGS: ['think', 'section'],
ADD_ATTR: ['class'],
});
// let text = content; // let text = content;
if (text === '') { if (text === '') {
text = t('chat.searching'); text = t('chat.searching');

View File

@ -45,7 +45,6 @@ import { RagNode } from './node';
import { BeginNode } from './node/begin-node'; import { BeginNode } from './node/begin-node';
import { NextStepDropdown } from './node/dropdown/next-step-dropdown'; import { NextStepDropdown } from './node/dropdown/next-step-dropdown';
import { ExtractorNode } from './node/extractor-node'; import { ExtractorNode } from './node/extractor-node';
import { HierarchicalMergerNode } from './node/hierarchical-merger-node';
import NoteNode from './node/note-node'; import NoteNode from './node/note-node';
import ParserNode from './node/parser-node'; import ParserNode from './node/parser-node';
import { SplitterNode } from './node/splitter-node'; import { SplitterNode } from './node/splitter-node';
@ -58,7 +57,6 @@ export const nodeTypes: NodeTypes = {
parserNode: ParserNode, parserNode: ParserNode,
tokenizerNode: TokenizerNode, tokenizerNode: TokenizerNode,
splitterNode: SplitterNode, splitterNode: SplitterNode,
hierarchicalMergerNode: HierarchicalMergerNode,
contextNode: ExtractorNode, contextNode: ExtractorNode,
}; };

View File

@ -1,57 +1,12 @@
import { Button } from '@/components/ui/button'; import { cn } from '@/lib/utils';
import { import { PropsWithChildren } from 'react';
Card,
CardContent,
CardDescription,
CardFooter,
CardHeader,
CardTitle,
} from '@/components/ui/card';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@/components/ui/select';
export function CardWithForm() { type LabelCardProps = {
className?: string;
} & PropsWithChildren;
export function LabelCard({ children, className }: LabelCardProps) {
return ( return (
<Card className="w-[350px]"> <div className={cn('bg-bg-card rounded-sm p-1', className)}>{children}</div>
<CardHeader>
<CardTitle>Create project</CardTitle>
<CardDescription>Deploy your new project in one-click.</CardDescription>
</CardHeader>
<CardContent>
<form>
<div className="grid w-full items-center gap-4">
<div className="flex flex-col space-y-1.5">
<Label htmlFor="name">Name</Label>
<Input id="name" placeholder="Name of your project" />
</div>
<div className="flex flex-col space-y-1.5">
<Label htmlFor="framework">Framework</Label>
<Select>
<SelectTrigger id="framework">
<SelectValue placeholder="Select" />
</SelectTrigger>
<SelectContent position="popper">
<SelectItem value="next">Next.js</SelectItem>
<SelectItem value="sveltekit">SvelteKit</SelectItem>
<SelectItem value="astro">Astro</SelectItem>
<SelectItem value="nuxt">Nuxt.js</SelectItem>
</SelectContent>
</Select>
</div>
</div>
</form>
</CardContent>
<CardFooter className="flex justify-between">
<Button variant="outline">Cancel</Button>
<Button>Deploy</Button>
</CardFooter>
</Card>
); );
} }

View File

@ -1 +1,18 @@
export { RagNode as ExtractorNode } from './index'; import LLMLabel from '@/components/llm-select/llm-label';
import { IRagNode } from '@/interfaces/database/agent';
import { NodeProps } from '@xyflow/react';
import { get } from 'lodash';
import { LabelCard } from './card';
import { RagNode } from './index';
export function ExtractorNode({ ...props }: NodeProps<IRagNode>) {
const { data } = props;
return (
<RagNode {...props}>
<LabelCard>
<LLMLabel value={get(data, 'form.llm_id')}></LLMLabel>
</LabelCard>
</RagNode>
);
}

View File

@ -1 +0,0 @@
export { RagNode as HierarchicalMergerNode } from './index';

View File

@ -1,6 +1,6 @@
import { IRagNode } from '@/interfaces/database/flow'; import { IRagNode } from '@/interfaces/database/flow';
import { NodeProps, Position } from '@xyflow/react'; import { NodeProps, Position } from '@xyflow/react';
import { memo, useMemo } from 'react'; import { PropsWithChildren, memo, useMemo } from 'react';
import { NodeHandleId, SingleOperators } from '../../constant'; import { NodeHandleId, SingleOperators } from '../../constant';
import useGraphStore from '../../store'; import useGraphStore from '../../store';
import { CommonHandle } from './handle'; import { CommonHandle } from './handle';
@ -9,12 +9,14 @@ import NodeHeader from './node-header';
import { NodeWrapper } from './node-wrapper'; import { NodeWrapper } from './node-wrapper';
import { ToolBar } from './toolbar'; import { ToolBar } from './toolbar';
type RagNodeProps = NodeProps<IRagNode> & PropsWithChildren;
function InnerRagNode({ function InnerRagNode({
id, id,
data, data,
isConnectable = true, isConnectable = true,
selected, selected,
}: NodeProps<IRagNode>) { children,
}: RagNodeProps) {
const getOperatorTypeFromId = useGraphStore( const getOperatorTypeFromId = useGraphStore(
(state) => state.getOperatorTypeFromId, (state) => state.getOperatorTypeFromId,
); );
@ -45,6 +47,7 @@ function InnerRagNode({
isConnectableEnd={false} isConnectableEnd={false}
></CommonHandle> ></CommonHandle>
<NodeHeader id={id} name={data.name} label={data.label}></NodeHeader> <NodeHeader id={id} name={data.name} label={data.label}></NodeHeader>
{children}
</NodeWrapper> </NodeWrapper>
</ToolBar> </ToolBar>
); );

View File

@ -9,6 +9,7 @@ interface IProps {
gap?: number; gap?: number;
className?: string; className?: string;
wrapperClassName?: string; wrapperClassName?: string;
icon?: React.ReactNode;
} }
const InnerNodeHeader = ({ const InnerNodeHeader = ({
@ -16,11 +17,12 @@ const InnerNodeHeader = ({
name, name,
className, className,
wrapperClassName, wrapperClassName,
icon,
}: IProps) => { }: IProps) => {
return ( return (
<section className={cn(wrapperClassName, 'pb-4')}> <section className={cn(wrapperClassName, 'pb-4')}>
<div className={cn(className, 'flex gap-2.5')}> <div className={cn(className, 'flex gap-2.5')}>
<OperatorIcon name={label as Operator}></OperatorIcon> {icon || <OperatorIcon name={label as Operator}></OperatorIcon>}
<span className="truncate text-center font-semibold text-sm"> <span className="truncate text-center font-semibold text-sm">
{name} {name}
</span> </span>

View File

@ -1,103 +1,16 @@
import { NodeProps, NodeResizeControl } from '@xyflow/react';
import {
Form,
FormControl,
FormField,
FormItem,
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import { Textarea } from '@/components/ui/textarea';
import { INoteNode } from '@/interfaces/database/flow'; import { INoteNode } from '@/interfaces/database/flow';
import { zodResolver } from '@hookform/resolvers/zod'; import BaseNoteNode from '@/pages/agent/canvas/node/note-node';
import { NotebookPen } from 'lucide-react'; import { NodeProps } from '@xyflow/react';
import { memo } from 'react'; import { memo } from 'react';
import { useForm } from 'react-hook-form';
import { useTranslation } from 'react-i18next';
import { z } from 'zod';
import { NodeWrapper } from '../node-wrapper';
import { ResizeIcon, controlStyle } from '../resize-icon';
import { useWatchFormChange, useWatchNameFormChange } from './use-watch-change'; import { useWatchFormChange, useWatchNameFormChange } from './use-watch-change';
const FormSchema = z.object({ function NoteNode({ ...props }: NodeProps<INoteNode>) {
text: z.string(),
});
const NameFormSchema = z.object({
name: z.string(),
});
function NoteNode({ data, id, selected }: NodeProps<INoteNode>) {
const { t } = useTranslation();
const form = useForm<z.infer<typeof FormSchema>>({
resolver: zodResolver(FormSchema),
defaultValues: data.form,
});
const nameForm = useForm<z.infer<typeof NameFormSchema>>({
resolver: zodResolver(NameFormSchema),
defaultValues: { name: data.name },
});
useWatchFormChange(id, form);
useWatchNameFormChange(id, nameForm);
return ( return (
<NodeWrapper <BaseNoteNode
className="p-0 w-full h-full flex flex-col" {...props}
selected={selected} useWatchNoteFormChange={useWatchFormChange}
> useWatchNoteNameFormChange={useWatchNameFormChange}
<NodeResizeControl minWidth={190} minHeight={128} style={controlStyle}> ></BaseNoteNode>
<ResizeIcon />
</NodeResizeControl>
<section className="p-2 flex gap-2 bg-background-note items-center note-drag-handle rounded-t">
<NotebookPen className="size-4" />
<Form {...nameForm}>
<form className="flex-1">
<FormField
control={nameForm.control}
name="name"
render={({ field }) => (
<FormItem className="h-full">
<FormControl>
<Input
placeholder={t('flow.notePlaceholder')}
{...field}
type="text"
className="bg-transparent border-none focus-visible:outline focus-visible:outline-text-sub-title"
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</form>
</Form>
</section>
<Form {...form}>
<form className="flex-1 p-1">
<FormField
control={form.control}
name="text"
render={({ field }) => (
<FormItem className="h-full">
<FormControl>
<Textarea
placeholder={t('flow.notePlaceholder')}
className="resize-none rounded-none p-1 h-full overflow-auto bg-transparent focus-visible:ring-0 border-none"
{...field}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</form>
</Form>
</NodeWrapper>
); );
} }

View File

@ -1,4 +1,4 @@
import useGraphStore from '@/pages/agent/store'; import useGraphStore from '@/pages/data-flow/store';
import { useEffect } from 'react'; import { useEffect } from 'react';
import { UseFormReturn, useWatch } from 'react-hook-form'; import { UseFormReturn, useWatch } from 'react-hook-form';

View File

@ -1,7 +1,10 @@
import { IRagNode } from '@/interfaces/database/flow'; import { BaseNode } from '@/interfaces/database/agent';
import { NodeProps, Position } from '@xyflow/react'; import { NodeProps, Position } from '@xyflow/react';
import { memo } from 'react'; import { memo } from 'react';
import { useTranslation } from 'react-i18next';
import { NodeHandleId } from '../../constant'; import { NodeHandleId } from '../../constant';
import { ParserFormSchemaType } from '../../form/parser-form';
import { LabelCard } from './card';
import { CommonHandle } from './handle'; import { CommonHandle } from './handle';
import { LeftHandleStyle, RightHandleStyle } from './handle-icon'; import { LeftHandleStyle, RightHandleStyle } from './handle-icon';
import NodeHeader from './node-header'; import NodeHeader from './node-header';
@ -12,7 +15,8 @@ function ParserNode({
data, data,
isConnectable = true, isConnectable = true,
selected, selected,
}: NodeProps<IRagNode>) { }: NodeProps<BaseNode<ParserFormSchemaType>>) {
const { t } = useTranslation();
return ( return (
<NodeWrapper selected={selected}> <NodeWrapper selected={selected}>
<CommonHandle <CommonHandle
@ -33,6 +37,17 @@ function ParserNode({
isConnectableEnd={false} isConnectableEnd={false}
></CommonHandle> ></CommonHandle>
<NodeHeader id={id} name={data.name} label={data.label}></NodeHeader> <NodeHeader id={id} name={data.name} label={data.label}></NodeHeader>
<section className="space-y-2">
{data.form?.setups.map((x, idx) => (
<LabelCard
key={idx}
className="flex justify- flex-col text-text-primary gap-1"
>
<span className="text-text-secondary">Parser {idx + 1}</span>
{t(`dataflow.fileFormatOptions.${x.fileFormat}`)}
</LabelCard>
))}
</section>
</NodeWrapper> </NodeWrapper>
); );
} }

View File

@ -1 +1,52 @@
export { RagNode as SplitterNode } from './index'; import { IRagNode } from '@/interfaces/database/flow';
import { NodeProps, Position } from '@xyflow/react';
import { PropsWithChildren, memo } from 'react';
import { NodeHandleId, Operator } from '../../constant';
import OperatorIcon from '../../operator-icon';
import { LabelCard } from './card';
import { CommonHandle } from './handle';
import { LeftHandleStyle, RightHandleStyle } from './handle-icon';
import NodeHeader from './node-header';
import { NodeWrapper } from './node-wrapper';
import { ToolBar } from './toolbar';
type RagNodeProps = NodeProps<IRagNode> & PropsWithChildren;
function InnerSplitterNode({
id,
data,
isConnectable = true,
selected,
}: RagNodeProps) {
return (
<ToolBar selected={selected} id={id} label={data.label} showCopy={false}>
<NodeWrapper selected={selected}>
<CommonHandle
id={NodeHandleId.End}
type="target"
position={Position.Left}
isConnectable={isConnectable}
style={LeftHandleStyle}
nodeId={id}
></CommonHandle>
<CommonHandle
type="source"
position={Position.Right}
isConnectable={isConnectable}
id={NodeHandleId.Start}
style={RightHandleStyle}
nodeId={id}
isConnectableEnd={false}
></CommonHandle>
<NodeHeader
id={id}
name={'Chunker'}
label={data.label}
icon={<OperatorIcon name={Operator.Splitter}></OperatorIcon>}
></NodeHeader>
<LabelCard>{data.name}</LabelCard>
</NodeWrapper>
</ToolBar>
);
}
export const SplitterNode = memo(InnerSplitterNode);

View File

@ -1,7 +1,10 @@
import { IRagNode } from '@/interfaces/database/flow'; import { BaseNode } from '@/interfaces/database/agent';
import { NodeProps, Position } from '@xyflow/react'; import { NodeProps, Position } from '@xyflow/react';
import { memo } from 'react'; import { memo } from 'react';
import { useTranslation } from 'react-i18next';
import { NodeHandleId } from '../../constant'; import { NodeHandleId } from '../../constant';
import { TokenizerFormSchemaType } from '../../form/tokenizer-form';
import { LabelCard } from './card';
import { CommonHandle } from './handle'; import { CommonHandle } from './handle';
import { LeftHandleStyle } from './handle-icon'; import { LeftHandleStyle } from './handle-icon';
import NodeHeader from './node-header'; import NodeHeader from './node-header';
@ -13,7 +16,9 @@ function TokenizerNode({
data, data,
isConnectable = true, isConnectable = true,
selected, selected,
}: NodeProps<IRagNode>) { }: NodeProps<BaseNode<TokenizerFormSchemaType>>) {
const { t } = useTranslation();
return ( return (
<ToolBar <ToolBar
selected={selected} selected={selected}
@ -32,6 +37,16 @@ function TokenizerNode({
nodeId={id} nodeId={id}
></CommonHandle> ></CommonHandle>
<NodeHeader id={id} name={data.name} label={data.label}></NodeHeader> <NodeHeader id={id} name={data.name} label={data.label}></NodeHeader>
<LabelCard className="text-text-primary flex justify-between flex-col gap-1">
<span className="text-text-secondary">
{t('dataflow.searchMethod')}
</span>
<ul className="space-y-1">
{data.form?.search_method.map((x) => (
<li key={x}>{t(`dataflow.tokenizerSearchMethodOptions.${x}`)}</li>
))}
</ul>
</LabelCard>
</NodeWrapper> </NodeWrapper>
</ToolBar> </ToolBar>
); );

View File

@ -337,7 +337,7 @@ export const NodeMap = {
[Operator.Parser]: 'parserNode', [Operator.Parser]: 'parserNode',
[Operator.Tokenizer]: 'tokenizerNode', [Operator.Tokenizer]: 'tokenizerNode',
[Operator.Splitter]: 'splitterNode', [Operator.Splitter]: 'splitterNode',
[Operator.HierarchicalMerger]: 'hierarchicalMergerNode', [Operator.HierarchicalMerger]: 'splitterNode',
[Operator.Extractor]: 'contextNode', [Operator.Extractor]: 'contextNode',
}; };

View File

@ -58,7 +58,13 @@ const FormSheet = ({
<SheetTitle className="hidden"></SheetTitle> <SheetTitle className="hidden"></SheetTitle>
<section className="flex-col border-b py-2 px-5"> <section className="flex-col border-b py-2 px-5">
<div className="flex items-center gap-2 pb-3"> <div className="flex items-center gap-2 pb-3">
<OperatorIcon name={operatorName}></OperatorIcon> <OperatorIcon
name={
operatorName === Operator.HierarchicalMerger
? Operator.Splitter
: operatorName
}
></OperatorIcon>
<div className="flex items-center gap-1 flex-1"> <div className="flex items-center gap-1 flex-1">
<label htmlFor="">{t('flow.title')}</label> <label htmlFor="">{t('flow.title')}</label>
{node?.id === BeginId ? ( {node?.id === BeginId ? (

View File

@ -30,7 +30,6 @@ import { useWatchFormChange } from '../../hooks/use-watch-form-change';
import { INextOperatorForm } from '../../interface'; import { INextOperatorForm } from '../../interface';
import { buildOutputList } from '../../utils/build-output-list'; import { buildOutputList } from '../../utils/build-output-list';
import { Output } from '../components/output'; import { Output } from '../components/output';
import { OutputFormatFormField } from './common-form-fields';
import { EmailFormFields } from './email-form-fields'; import { EmailFormFields } from './email-form-fields';
import { ImageFormFields } from './image-form-fields'; import { ImageFormFields } from './image-form-fields';
import { PdfFormFields } from './pdf-form-fields'; import { PdfFormFields } from './pdf-form-fields';
@ -147,10 +146,10 @@ function ParserItem({
)} )}
</RAGFlowFormItem> </RAGFlowFormItem>
<Widget prefix={prefix} fileType={fileFormat as FileType}></Widget> <Widget prefix={prefix} fileType={fileFormat as FileType}></Widget>
<OutputFormatFormField {/* <OutputFormatFormField
prefix={prefix} prefix={prefix}
fileType={fileFormat as FileType} fileType={fileFormat as FileType}
/> /> */}
{index < fieldLength - 1 && <Separator />} {index < fieldLength - 1 && <Separator />}
</section> </section>
); );

View File

@ -26,7 +26,7 @@ export const FormSchema = z.object({
value: z.string().optional(), value: z.string().optional(),
}), }),
), ),
overlapped_percent: z.number(), // 0.0 - 0.3 overlapped_percent: z.number(), // 0.0 - 0.3 , 0% - 30%
}); });
export type SplitterFormSchemaType = z.infer<typeof FormSchema>; export type SplitterFormSchemaType = z.infer<typeof FormSchema>;
@ -58,9 +58,8 @@ const SplitterForm = ({ node }: INextOperatorForm) => {
></SliderInputFormField> ></SliderInputFormField>
<SliderInputFormField <SliderInputFormField
name="overlapped_percent" name="overlapped_percent"
max={0.3} max={30}
min={0} min={0}
step={0.01}
label={t('dataflow.overlappedPercent')} label={t('dataflow.overlappedPercent')}
></SliderInputFormField> ></SliderInputFormField>
<section> <section>

View File

@ -29,6 +29,8 @@ export const FormSchema = z.object({
fields: z.string(), fields: z.string(),
}); });
export type TokenizerFormSchemaType = z.infer<typeof FormSchema>;
const TokenizerForm = ({ node }: INextOperatorForm) => { const TokenizerForm = ({ node }: INextOperatorForm) => {
const { t } = useTranslation(); const { t } = useTranslation();
const defaultValues = useFormValues(initialTokenizerValues, node); const defaultValues = useFormValues(initialTokenizerValues, node);
@ -44,7 +46,7 @@ const TokenizerForm = ({ node }: INextOperatorForm) => {
'dataflow.tokenizerFieldsOptions', 'dataflow.tokenizerFieldsOptions',
); );
const form = useForm<z.infer<typeof FormSchema>>({ const form = useForm<TokenizerFormSchemaType>({
defaultValues, defaultValues,
resolver: zodResolver(FormSchema), resolver: zodResolver(FormSchema),
mode: 'onChange', mode: 'onChange',

View File

@ -131,6 +131,7 @@ function transformParserParams(params: ParserFormSchemaType) {
function transformSplitterParams(params: SplitterFormSchemaType) { function transformSplitterParams(params: SplitterFormSchemaType) {
return { return {
...params, ...params,
overlapped_percent: Number(params.overlapped_percent) / 100,
delimiters: transformObjectArrayToPureArray(params.delimiters, 'value'), delimiters: transformObjectArrayToPureArray(params.delimiters, 'value'),
}; };
} }

View File

@ -44,7 +44,7 @@ const FormatPreserveEditor = ({
/> />
)} )}
{['text', 'html'].includes(initialValue.key) && ( {['text', 'html', 'markdown'].includes(initialValue.key) && (
<ObjectContainer <ObjectContainer
isReadonly={isReadonly} isReadonly={isReadonly}
className={className} className={className}

View File

@ -1,6 +1,6 @@
import { CheckedState } from '@radix-ui/react-checkbox'; import { CheckedState } from '@radix-ui/react-checkbox';
import { ChunkTextMode } from '../../constant'; import { ChunkTextMode } from '../../constant';
import { IChunk } from '../../interface'; import { ComponentParams, IChunk } from '../../interface';
import { parserKeyMap } from './json-parser'; import { parserKeyMap } from './json-parser';
export interface FormatPreserveEditorProps { export interface FormatPreserveEditorProps {
@ -28,6 +28,7 @@ export type IJsonContainerProps = {
value: { value: {
[key: string]: string; [key: string]: string;
}[]; }[];
params: ComponentParams;
}; };
isChunck?: boolean; isChunck?: boolean;
handleCheck: (e: CheckedState, index: number) => void; handleCheck: (e: CheckedState, index: number) => void;
@ -52,6 +53,7 @@ export type IObjContainerProps = {
key: string; key: string;
type: string; type: string;
value: string; value: string;
params: ComponentParams;
}; };
isChunck?: boolean; isChunck?: boolean;
handleCheck: (e: CheckedState, index: number) => void; handleCheck: (e: CheckedState, index: number) => void;

View File

@ -1,6 +1,7 @@
import { Checkbox } from '@/components/ui/checkbox'; import { Checkbox } from '@/components/ui/checkbox';
import { cn } from '@/lib/utils'; import { cn } from '@/lib/utils';
import { useCallback, useEffect } from 'react'; import { isArray } from 'lodash';
import { useCallback, useEffect, useMemo } from 'react';
import { ChunkTextMode } from '../../constant'; import { ChunkTextMode } from '../../constant';
import styles from '../../index.less'; import styles from '../../index.less';
import { useParserInit } from './hook'; import { useParserInit } from './hook';
@ -33,7 +34,13 @@ export const ArrayContainer = (props: IJsonContainerProps) => {
editDivRef, editDivRef,
} = useParserInit({ initialValue }); } = useParserInit({ initialValue });
const parserKey = parserKeyMap[content.key as keyof typeof parserKeyMap]; const parserKey = useMemo(() => {
const key =
content.key === 'chunks' && content.params.field_name
? content.params.field_name
: parserKeyMap[content.key as keyof typeof parserKeyMap];
return key;
}, [content]);
const handleEdit = useCallback( const handleEdit = useCallback(
(e?: any, index?: number) => { (e?: any, index?: number) => {
@ -73,7 +80,8 @@ export const ArrayContainer = (props: IJsonContainerProps) => {
return ( return (
<> <>
{content.value?.map((item, index) => { {isArray(content.value) &&
content.value?.map((item, index) => {
if ( if (
item[parserKeyMap[content.key as keyof typeof parserKeyMap]] === '' item[parserKeyMap[content.key as keyof typeof parserKeyMap]] === ''
) { ) {
@ -128,7 +136,7 @@ export const ArrayContainer = (props: IJsonContainerProps) => {
} }
}} }}
> >
{item[parserKeyMap[content.key]]} {item[parserKey]}
</div> </div>
)} )}
</section> </section>

View File

@ -218,18 +218,11 @@ export const useTimelineDataFlow = (data: IPipelineFileLogDetail) => {
const nodes: Array<ITimelineNodeObj & { id: number | string }> = []; const nodes: Array<ITimelineNodeObj & { id: number | string }> = [];
console.log('time-->', data); console.log('time-->', data);
const times = data?.dsl?.components; const times = data?.dsl?.components;
const graphNodes = data?.dsl?.graph?.nodes;
if (times) { if (times) {
const getNode = ( const getNode = (key: string, index: number, type: TimelineNodeType) => {
key: string,
index: number,
type:
| TimelineNodeType.begin
| TimelineNodeType.parser
| TimelineNodeType.tokenizer
| TimelineNodeType.characterSplitter
| TimelineNodeType.titleSplitter,
) => {
const node = times[key].obj; const node = times[key].obj;
const graphNode = graphNodes?.find((item) => item.id === key);
const name = camelCase( const name = camelCase(
node.component_name, node.component_name,
) as keyof typeof TimelineNodeObj; ) as keyof typeof TimelineNodeObj;
@ -247,6 +240,7 @@ export const useTimelineDataFlow = (data: IPipelineFileLogDetail) => {
} }
const timeNode = { const timeNode = {
...TimelineNodeObj[name], ...TimelineNodeObj[name],
title: graphNode?.data?.name,
id: index, id: index,
className: 'w-32', className: 'w-32',
completed: false, completed: false,
@ -255,6 +249,13 @@ export const useTimelineDataFlow = (data: IPipelineFileLogDetail) => {
), ),
type: tempType, type: tempType,
detail: { value: times[key], key: key }, detail: { value: times[key], key: key },
} as ITimelineNodeObj & {
id: number | string;
className: string;
completed: boolean;
date: string;
type: TimelineNodeType;
detail: { value: IDslComponent; key: string };
}; };
console.log('timeNodetype-->', type); console.log('timeNodetype-->', type);
nodes.push(timeNode); nodes.push(timeNode);
@ -329,3 +330,30 @@ export function useFetchPipelineResult({
return { pipelineResult }; return { pipelineResult };
} }
export const useSummaryInfo = (
data: IPipelineFileLogDetail,
currentTimeNode: TimelineNode,
) => {
const summaryInfo = useMemo(() => {
if (currentTimeNode.type === TimelineNodeType.parser) {
const setups =
currentTimeNode.detail?.value?.obj?.params?.setups?.[
data.document_suffix
];
if (setups) {
const { output_format, parse_method } = setups;
const res = [];
if (parse_method) {
res.push(`${t('dataflow.parserMethod')}: ${parse_method}`);
}
if (output_format) {
res.push(`${t('dataflow.outputFormat')}: ${output_format}`);
}
return res.join(' ');
}
}
return '';
}, [data, currentTimeNode]);
return { summaryInfo };
};

View File

@ -9,6 +9,7 @@ import {
useGetPipelineResultSearchParams, useGetPipelineResultSearchParams,
useHandleChunkCardClick, useHandleChunkCardClick,
useRerunDataflow, useRerunDataflow,
useSummaryInfo,
useTimelineDataFlow, useTimelineDataFlow,
} from './hooks'; } from './hooks';
@ -61,7 +62,7 @@ const Chunk = () => {
); );
const { const {
navigateToDataset, navigateToDatasetOverview,
navigateToDatasetList, navigateToDatasetList,
navigateToAgents, navigateToAgents,
navigateToDataflow, navigateToDataflow,
@ -150,7 +151,7 @@ const Chunk = () => {
({} as TimelineNode) ({} as TimelineNode)
); );
}, [activeStepId, timelineNodes]); }, [activeStepId, timelineNodes]);
const { summaryInfo } = useSummaryInfo(dataset, currentTimeNode);
return ( return (
<> <>
<PageHeader> <PageHeader>
@ -175,7 +176,7 @@ const Chunk = () => {
<BreadcrumbLink <BreadcrumbLink
onClick={() => { onClick={() => {
if (knowledgeId) { if (knowledgeId) {
navigateToDataset(knowledgeId)(); navigateToDatasetOverview(knowledgeId)();
} }
if (agentId) { if (agentId) {
navigateToDataflow(agentId)(); navigateToDataflow(agentId)();
@ -220,7 +221,7 @@ const Chunk = () => {
></DocumentPreview> ></DocumentPreview>
</section> </section>
</div> </div>
<div className="h-dvh border-r -mt-3"></div> <div className="h-[calc(100vh-100px)] border-r -mt-3"></div>
<div className="w-3/5 h-full"> <div className="w-3/5 h-full">
{/* {currentTimeNode?.type === TimelineNodeType.splitter && ( {/* {currentTimeNode?.type === TimelineNodeType.splitter && (
<ChunkerContainer <ChunkerContainer
@ -246,6 +247,7 @@ const Chunk = () => {
key: string; key: string;
} }
} }
summaryInfo={summaryInfo}
clickChunk={handleChunkCardClick} clickChunk={handleChunkCardClick}
reRunFunc={handleReRunFunc} reRunFunc={handleReRunFunc}
/> />

View File

@ -1,6 +1,6 @@
import { PipelineResultSearchParams } from './constant'; import { PipelineResultSearchParams } from './constant';
interface ComponentParams { export interface ComponentParams {
debug_inputs: Record<string, any>; debug_inputs: Record<string, any>;
delay_after_error: number; delay_after_error: number;
description: string; description: string;
@ -8,6 +8,7 @@ interface ComponentParams {
exception_goto: any; exception_goto: any;
exception_method: any; exception_method: any;
inputs: Record<string, any>; inputs: Record<string, any>;
field_name: string;
max_retries: number; max_retries: number;
message_history_window_size: number; message_history_window_size: number;
outputs: { outputs: {
@ -30,6 +31,66 @@ export interface IDslComponent {
obj: ComponentObject; obj: ComponentObject;
upstream: Array<string>; upstream: Array<string>;
} }
interface NodeData {
label: string;
name: string;
form?: {
outputs?: Record<
string,
{
type: string;
value: string | Array<Record<string, any>> | number;
}
>;
setups?: Array<Record<string, any>>;
chunk_token_size?: number;
delimiters?: Array<{
value: string;
}>;
overlapped_percent?: number;
};
}
interface EdgeData {
isHovered: boolean;
}
interface Position {
x: number;
y: number;
}
interface Measured {
height: number;
width: number;
}
interface Node {
data: NodeData;
dragging: boolean;
id: string;
measured: Measured;
position: Position;
selected: boolean;
sourcePosition: string;
targetPosition: string;
type: string;
}
interface Edge {
data: EdgeData;
id: string;
source: string;
sourceHandle: string;
target: string;
targetHandle: string;
}
interface GraphData {
edges: Edge[];
nodes: Node[];
}
export interface IPipelineFileLogDetail { export interface IPipelineFileLogDetail {
avatar: string; avatar: string;
create_date: string; create_date: string;
@ -42,6 +103,7 @@ export interface IPipelineFileLogDetail {
components: { components: {
[key: string]: IDslComponent; [key: string]: IDslComponent;
}; };
graph: GraphData;
task_id: string; task_id: string;
path: Array<string>; path: Array<string>;
}; };

View File

@ -19,6 +19,7 @@ interface IProps {
data: { value: IDslComponent; key: string }; data: { value: IDslComponent; key: string };
reRunLoading: boolean; reRunLoading: boolean;
clickChunk: (chunk: IChunk) => void; clickChunk: (chunk: IChunk) => void;
summaryInfo: string;
reRunFunc: (data: { value: IDslComponent; key: string }) => void; reRunFunc: (data: { value: IDslComponent; key: string }) => void;
} }
const ParserContainer = (props: IProps) => { const ParserContainer = (props: IProps) => {
@ -31,6 +32,7 @@ const ParserContainer = (props: IProps) => {
reRunLoading, reRunLoading,
clickChunk, clickChunk,
isReadonly, isReadonly,
summaryInfo,
} = props; } = props;
const { t } = useTranslation(); const { t } = useTranslation();
const [selectedChunkIds, setSelectedChunkIds] = useState<string[]>([]); const [selectedChunkIds, setSelectedChunkIds] = useState<string[]>([]);
@ -46,6 +48,7 @@ const ParserContainer = (props: IProps) => {
key, key,
type, type,
value, value,
params: data?.value?.obj?.params,
}; };
}, [data]); }, [data]);
@ -130,7 +133,7 @@ const ParserContainer = (props: IProps) => {
const newText = [...initialText.value, { text: text || ' ' }]; const newText = [...initialText.value, { text: text || ' ' }];
setInitialText({ setInitialText({
...initialText, ...initialText,
value: newText, value: newText as any,
}); });
}, },
[initialText], [initialText],
@ -156,15 +159,16 @@ const ParserContainer = (props: IProps) => {
{t('dataflowParser.parseSummary')} {t('dataflowParser.parseSummary')}
</h2> </h2>
<div className="text-[12px] text-text-secondary italic "> <div className="text-[12px] text-text-secondary italic ">
{t('dataflowParser.parseSummaryTip')} {/* {t('dataflowParser.parseSummaryTip')} */}
{summaryInfo}
</div> </div>
</div> </div>
)} )}
{isChunck && ( {isChunck && (
<div> <div>
<h2 className="text-[16px]">{t('chunk.chunkResult')}</h2> <h2 className="text-[16px]">{t('dataflowParser.result')}</h2>
<div className="text-[12px] text-text-secondary italic"> <div className="text-[12px] text-text-secondary italic">
{t('chunk.chunkResultTip')} {/* {t('chunk.chunkResultTip')} */}
</div> </div>
</div> </div>
)} )}
@ -190,7 +194,7 @@ const ParserContainer = (props: IProps) => {
<div <div
className={cn( className={cn(
' border rounded-lg p-[20px] box-border w-[calc(100%-20px)] overflow-auto scrollbar-none', ' border rounded-lg p-[20px] box-border w-[calc(100%-20px)] overflow-auto scrollbar-auto',
{ {
'h-[calc(100vh-240px)]': isChunck, 'h-[calc(100vh-240px)]': isChunck,
'h-[calc(100vh-180px)]': !isChunck, 'h-[calc(100vh-180px)]': !isChunck,

View File

@ -10,5 +10,5 @@ export enum ProcessingType {
export const ProcessingTypeMap = { export const ProcessingTypeMap = {
[ProcessingType.knowledgeGraph]: 'Knowledge Graph', [ProcessingType.knowledgeGraph]: 'Knowledge Graph',
[ProcessingType.raptor]: 'Raptor', [ProcessingType.raptor]: 'RAPTOR',
}; };

View File

@ -109,7 +109,9 @@ export const getFileLogsTableColumns = (
name={row.original.pipeline_title} name={row.original.pipeline_title}
className="size-4" className="size-4"
/> />
{row.original.pipeline_title} {row.original.pipeline_title === 'naive'
? 'general'
: row.original.pipeline_title}
</div> </div>
), ),
}, },
@ -396,7 +398,7 @@ const FileLogsTable: FC<FileLogsTableProps> = ({
</TableRow> </TableRow>
))} ))}
</TableHeader> </TableHeader>
<TableBody className="relative"> <TableBody className="relative min-w-[1280px] overflow-auto">
{table.getRowModel().rows?.length ? ( {table.getRowModel().rows?.length ? (
table.getRowModel().rows.map((row) => ( table.getRowModel().rows.map((row) => (
<TableRow <TableRow

View File

@ -100,6 +100,7 @@ export function EmbeddingModelItem({ line = 1, isEdit = true }: IProps) {
options={embeddingModelOptions} options={embeddingModelOptions}
disabled={isEdit ? disabled : false} disabled={isEdit ? disabled : false}
placeholder={t('embeddingModelPlaceholder')} placeholder={t('embeddingModelPlaceholder')}
triggerClassName="!bg-bg-base"
/> />
</FormControl> </FormControl>
</div> </div>

View File

@ -6,7 +6,6 @@ import { DelimiterFormField } from '@/components/delimiter-form-field';
import { ExcelToHtmlFormField } from '@/components/excel-to-html-form-field'; import { ExcelToHtmlFormField } from '@/components/excel-to-html-form-field';
import { LayoutRecognizeFormField } from '@/components/layout-recognize-form-field'; import { LayoutRecognizeFormField } from '@/components/layout-recognize-form-field';
import { MaxTokenNumberFormField } from '@/components/max-token-number-from-field'; import { MaxTokenNumberFormField } from '@/components/max-token-number-from-field';
import { TagItems } from '../components/tag-item';
import { import {
ConfigurationFormContainer, ConfigurationFormContainer,
MainContainer, MainContainer,
@ -26,7 +25,7 @@ export function NaiveConfiguration() {
<AutoKeywordsFormField></AutoKeywordsFormField> <AutoKeywordsFormField></AutoKeywordsFormField>
<AutoQuestionsFormField></AutoQuestionsFormField> <AutoQuestionsFormField></AutoQuestionsFormField>
<ExcelToHtmlFormField></ExcelToHtmlFormField> <ExcelToHtmlFormField></ExcelToHtmlFormField>
<TagItems></TagItems> {/* <TagItems></TagItems> */}
</ConfigurationFormContainer> </ConfigurationFormContainer>
</MainContainer> </MainContainer>
); );

View File

@ -20,15 +20,15 @@ export function ApplicationCard({
}: ApplicationCardProps) { }: ApplicationCardProps) {
return ( return (
<Card className="w-[264px]" onClick={onClick}> <Card className="w-[264px]" onClick={onClick}>
<CardContent className="p-2.5 group flex justify-between"> <CardContent className="p-2.5 group flex justify-between w-full">
<div className="flex items-center gap-2.5"> <div className="flex items-center gap-2.5 w-full">
<RAGFlowAvatar <RAGFlowAvatar
className="size-14 rounded-lg" className="size-14 rounded-lg"
avatar={app.avatar} avatar={app.avatar}
name={app.title || 'CN'} name={app.title || 'CN'}
></RAGFlowAvatar> ></RAGFlowAvatar>
<div className="flex-1"> <div className="flex-1">
<h3 className="text-sm font-normal line-clamp-1 mb-1"> <h3 className="text-sm font-normal line-clamp-1 mb-1 text-ellipsis w-[180px] overflow-hidden">
{app.title} {app.title}
</h3> </h3>
<p className="text-xs font-normal text-text-secondary"> <p className="text-xs font-normal text-text-secondary">

View File

@ -134,7 +134,7 @@ export const BgSvg = () => {
)} )}
</div> </div>
<div <div
className={`w-full -mt-40`} className={`w-full -mt-48`}
style={{ height: aspectRatio['middle'] + 'px' }} style={{ height: aspectRatio['middle'] + 'px' }}
> >
{def( {def(
@ -144,7 +144,7 @@ export const BgSvg = () => {
)} )}
</div> </div>
<div <div
className={`w-full -mt-52`} className={`w-full -mt-72`}
style={{ height: aspectRatio['bottom'] + 'px' }} style={{ height: aspectRatio['bottom'] + 'px' }}
> >
{def( {def(

View File

@ -23,12 +23,12 @@ const FlipCard3D = (props: IProps) => {
className={`relative w-full h-full transition-transform transform-style-3d ${isFlipped ? 'rotate-y-180' : ''}`} className={`relative w-full h-full transition-transform transform-style-3d ${isFlipped ? 'rotate-y-180' : ''}`}
> >
{/* Front Face */} {/* Front Face */}
<div className="absolute inset-0 flex items-center justify-center bg-blue-500 text-white rounded-xl backface-hidden"> <div className="absolute inset-0 flex items-center justify-center backface-hidden">
{children} {children}
</div> </div>
{/* Back Face */} {/* Back Face */}
<div className="absolute inset-0 flex items-center justify-center bg-green-500 text-white rounded-xl backface-hidden rotate-y-180"> <div className="absolute inset-0 flex items-center justify-center backface-hidden rotate-y-180">
{children} {children}
</div> </div>
</div> </div>

View File

@ -42,6 +42,10 @@
////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////////////////////
.perspective-1000 { .perspective-1000 {
perspective: 1000px; perspective: 1000px;
overflow: hidden;
min-height: 680px;
display: flex;
align-items: center;
} }
.transform-style-3d { .transform-style-3d {
transform-style: preserve-3d; transform-style: preserve-3d;

View File

@ -24,6 +24,7 @@ import {
FormMessage, FormMessage,
} from '@/components/ui/form'; } from '@/components/ui/form';
import { Input } from '@/components/ui/input'; import { Input } from '@/components/ui/input';
import { cn } from '@/lib/utils';
import { zodResolver } from '@hookform/resolvers/zod'; import { zodResolver } from '@hookform/resolvers/zod';
import { Eye, EyeOff } from 'lucide-react'; import { Eye, EyeOff } from 'lucide-react';
import { useForm } from 'react-hook-form'; import { useForm } from 'react-hook-form';
@ -135,8 +136,7 @@ const Login = () => {
}; };
return ( return (
<div className="min-h-screen relative overflow-hidden"> <>
<BgSvg />
<Spotlight opcity={0.4} coverage={60} color={'rgb(128, 255, 248)'} /> <Spotlight opcity={0.4} coverage={60} color={'rgb(128, 255, 248)'} />
<Spotlight <Spotlight
opcity={0.3} opcity={0.3}
@ -152,11 +152,13 @@ const Login = () => {
Y={'-10%'} Y={'-10%'}
color={'rgb(128, 255, 248)'} color={'rgb(128, 255, 248)'}
/> />
<div className=" h-[inherit] relative overflow-auto">
<BgSvg />
{/* <SpotlightTopRight opcity={0.7} coverage={10} /> */} {/* <SpotlightTopRight opcity={0.7} coverage={10} /> */}
<div className="absolute top-3 flex flex-col items-center mb-12 w-full text-text-primary"> <div className="absolute top-3 flex flex-col items-center mb-12 w-full text-text-primary">
<div className="flex items-center mb-4 w-full pl-10 pt-10 "> <div className="flex items-center mb-4 w-full pl-10 pt-10 ">
<div className="w-12 h-12 p-2 rounded-lg border-2 border-border flex items-center justify-center mr-3"> <div className="w-12 h-12 p-2 rounded-lg bg-bg-base border-2 border-border flex items-center justify-center mr-3">
<img <img
src={'/logo.svg'} src={'/logo.svg'}
alt="logo" alt="logo"
@ -165,12 +167,15 @@ const Login = () => {
</div> </div>
<div className="text-xl font-bold self-center">RAGFlow</div> <div className="text-xl font-bold self-center">RAGFlow</div>
</div> </div>
<h1 className="text-2xl font-bold text-center mb-2">{t('title')}</h1> <h1 className="text-[36px] font-medium text-center mb-2">
<div className="mt-4 px-6 py-1 text-sm font-medium text-cyan-600 border border-accent-primary rounded-full hover:bg-cyan-50 transition-colors duration-200 border-glow relative overflow-hidden"> {t('title')}
</h1>
{/* border border-accent-primary rounded-full */}
{/* <div className="mt-4 px-6 py-1 text-sm font-medium text-cyan-600 hover:bg-cyan-50 transition-colors duration-200 border-glow relative overflow-hidden">
{t('start')} {t('start')}
</div> */}
</div> </div>
</div> <div className="relative z-10 flex flex-col items-center justify-center min-h-[1050px] px-4 sm:px-6 lg:px-8">
<div className="relative z-10 flex flex-col items-center justify-center min-h-screen px-4 sm:px-6 lg:px-8">
{/* Logo and Header */} {/* Logo and Header */}
{/* Login Form */} {/* Login Form */}
@ -181,10 +186,10 @@ const Login = () => {
{title === 'login' ? t('loginTitle') : t('signUpTitle')} {title === 'login' ? t('loginTitle') : t('signUpTitle')}
</h2> </h2>
</div> </div>
<div className="w-full max-w-md bg-bg-base backdrop-blur-sm rounded-2xl shadow-xl pt-14 pl-8 pr-8 pb-2 border border-border-button "> <div className=" w-full max-w-[540px] bg-bg-component backdrop-blur-sm rounded-2xl shadow-xl pt-14 pl-10 pr-10 pb-2 border border-border-button ">
<Form {...form}> <Form {...form}>
<form <form
className="flex flex-col gap-6 text-text-primary" className="flex flex-col gap-8 text-text-primary "
onSubmit={form.handleSubmit((data) => onCheck(data))} onSubmit={form.handleSubmit((data) => onCheck(data))}
> >
<FormField <FormField
@ -274,7 +279,14 @@ const Login = () => {
field.onChange(checked); field.onChange(checked);
}} }}
/> />
<FormLabel>{t('rememberMe')}</FormLabel> <FormLabel
className={cn(' hover:text-text-primary', {
'text-text-disabled': !field.value,
'text-text-primary': field.value,
})}
>
{t('rememberMe')}
</FormLabel>
</div> </div>
</FormControl> </FormControl>
<FormMessage /> <FormMessage />
@ -315,13 +327,13 @@ const Login = () => {
</Form> </Form>
{title === 'login' && registerEnabled && ( {title === 'login' && registerEnabled && (
<div className="mt-6 text-right"> <div className="mt-10 text-right">
<p className="text-text-disabled text-sm"> <p className="text-text-disabled text-sm">
{t('signInTip')} {t('signInTip')}
<Button <Button
variant={'transparent'} variant={'transparent'}
onClick={changeTitle} onClick={changeTitle}
className="text-cyan-600 hover:text-cyan-800 font-medium border-none transition-colors duration-200" className="text-accent-primary/90 hover:text-accent-primary hover:bg-transparent font-medium border-none transition-colors duration-200"
> >
{t('signUp')} {t('signUp')}
</Button> </Button>
@ -329,13 +341,13 @@ const Login = () => {
</div> </div>
)} )}
{title === 'register' && ( {title === 'register' && (
<div className="mt-6 text-right"> <div className="mt-10 text-right">
<p className="text-text-disabled text-sm"> <p className="text-text-disabled text-sm">
{t('signUpTip')} {t('signUpTip')}
<Button <Button
variant={'transparent'} variant={'transparent'}
onClick={changeTitle} onClick={changeTitle}
className="text-cyan-600 hover:text-cyan-800 font-medium border-none transition-colors duration-200" className="text-accent-primary/90 hover:text-accent-primary hover:bg-transparent font-medium border-none transition-colors duration-200"
> >
{t('login')} {t('login')}
</Button> </Button>
@ -347,6 +359,7 @@ const Login = () => {
</FlipCard3D> </FlipCard3D>
</div> </div>
</div> </div>
</>
); );
}; };

View File

@ -44,6 +44,7 @@ import { useAddChatBox } from '../use-add-box';
type MultipleChatBoxProps = { type MultipleChatBoxProps = {
controller: AbortController; controller: AbortController;
chatBoxIds: string[]; chatBoxIds: string[];
stopOutputMessage(): void;
} & Pick< } & Pick<
ReturnType<typeof useAddChatBox>, ReturnType<typeof useAddChatBox>,
'removeChatBox' | 'addChatBox' | 'chatBoxIds' 'removeChatBox' | 'addChatBox' | 'chatBoxIds'
@ -200,6 +201,7 @@ export function MultipleChatBox({
chatBoxIds, chatBoxIds,
removeChatBox, removeChatBox,
addChatBox, addChatBox,
stopOutputMessage,
}: MultipleChatBoxProps) { }: MultipleChatBoxProps) {
const { const {
value, value,
@ -207,7 +209,6 @@ export function MultipleChatBox({
messageRecord, messageRecord,
handleInputChange, handleInputChange,
handlePressEnter, handlePressEnter,
stopOutputMessage,
setFormRef, setFormRef,
handleUploadFile, handleUploadFile,
} = useSendMultipleChatMessage(controller, chatBoxIds); } = useSendMultipleChatMessage(controller, chatBoxIds);

View File

@ -20,9 +20,10 @@ import { buildMessageItemReference } from '../../utils';
interface IProps { interface IProps {
controller: AbortController; controller: AbortController;
stopOutputMessage(): void;
} }
export function SingleChatBox({ controller }: IProps) { export function SingleChatBox({ controller, stopOutputMessage }: IProps) {
const { const {
value, value,
scrollRef, scrollRef,
@ -34,7 +35,6 @@ export function SingleChatBox({ controller }: IProps) {
handlePressEnter, handlePressEnter,
regenerateMessage, regenerateMessage,
removeMessageById, removeMessageById,
stopOutputMessage,
handleUploadFile, handleUploadFile,
removeFile, removeFile,
} = useSendMessage(controller); } = useSendMessage(controller);

View File

@ -39,7 +39,7 @@ export default function Chat() {
const { t } = useTranslation(); const { t } = useTranslation();
const { data: conversation } = useFetchConversation(); const { data: conversation } = useFetchConversation();
const { handleConversationCardClick, controller } = const { handleConversationCardClick, controller, stopOutputMessage } =
useHandleClickConversationCard(); useHandleClickConversationCard();
const { visible: settingVisible, switchVisible: switchSettingVisible } = const { visible: settingVisible, switchVisible: switchSettingVisible } =
useSetModalState(true); useSetModalState(true);
@ -74,6 +74,7 @@ export default function Chat() {
controller={controller} controller={controller}
removeChatBox={removeChatBox} removeChatBox={removeChatBox}
addChatBox={addChatBox} addChatBox={addChatBox}
stopOutputMessage={stopOutputMessage}
></MultipleChatBox> ></MultipleChatBox>
</section> </section>
); );
@ -129,7 +130,10 @@ export default function Chat() {
</CardTitle> </CardTitle>
</CardHeader> </CardHeader>
<CardContent className="flex-1 p-0 min-h-0"> <CardContent className="flex-1 p-0 min-h-0">
<SingleChatBox controller={controller}></SingleChatBox> <SingleChatBox
controller={controller}
stopOutputMessage={stopOutputMessage}
></SingleChatBox>
</CardContent> </CardContent>
</Card> </Card>
{settingVisible && ( {settingVisible && (

View File

@ -5,16 +5,20 @@ export function useHandleClickConversationCard() {
const [controller, setController] = useState(new AbortController()); const [controller, setController] = useState(new AbortController());
const { handleClickConversation } = useClickConversationCard(); const { handleClickConversation } = useClickConversationCard();
const handleConversationCardClick = useCallback( const stopOutputMessage = useCallback(() => {
(conversationId: string, isNew: boolean) => {
handleClickConversation(conversationId, isNew ? 'true' : '');
setController((pre) => { setController((pre) => {
pre.abort(); pre.abort();
return new AbortController(); return new AbortController();
}); });
}, []);
const handleConversationCardClick = useCallback(
(conversationId: string, isNew: boolean) => {
handleClickConversation(conversationId, isNew ? 'true' : '');
stopOutputMessage();
}, },
[handleClickConversation], [handleClickConversation, stopOutputMessage],
); );
return { controller, handleConversationCardClick }; return { controller, handleConversationCardClick, stopOutputMessage };
} }

View File

@ -123,10 +123,6 @@ export const useSendMessage = (controller: AbortController) => {
[getConversationIsNew, handleUploadFile, setConversation], [getConversationIsNew, handleUploadFile, setConversation],
); );
const stopOutputMessage = useCallback(() => {
controller.abort();
}, [controller]);
const sendMessage = useCallback( const sendMessage = useCallback(
async ({ async ({
message, message,
@ -249,7 +245,6 @@ export const useSendMessage = (controller: AbortController) => {
messageContainerRef, messageContainerRef,
derivedMessages, derivedMessages,
removeMessageById, removeMessageById,
stopOutputMessage,
handleUploadFile: onUploadFile, handleUploadFile: onUploadFile,
isUploading, isUploading,
removeFile, removeFile,

View File

@ -35,10 +35,6 @@ export function useSendMultipleChatMessage(
const { setFormRef, getLLMConfigById, isLLMConfigEmpty } = const { setFormRef, getLLMConfigById, isLLMConfigEmpty } =
useBuildFormRefs(chatBoxIds); useBuildFormRefs(chatBoxIds);
const stopOutputMessage = useCallback(() => {
controller.abort();
}, [controller]);
const addNewestQuestion = useCallback( const addNewestQuestion = useCallback(
(message: Message, answer: string = '') => { (message: Message, answer: string = '') => {
setMessageRecord((pre) => { setMessageRecord((pre) => {
@ -236,7 +232,6 @@ export function useSendMultipleChatMessage(
sendMessage, sendMessage,
handleInputChange, handleInputChange,
handlePressEnter, handlePressEnter,
stopOutputMessage,
sendLoading: !allDone, sendLoading: !allDone,
setFormRef, setFormRef,
handleUploadFile, handleUploadFile,

View File

@ -64,7 +64,10 @@ const MarkdownContent = ({
const { setDocumentIds, data: fileThumbnails } = const { setDocumentIds, data: fileThumbnails } =
useFetchDocumentThumbnailsByIds(); useFetchDocumentThumbnailsByIds();
const contentWithCursor = useMemo(() => { const contentWithCursor = useMemo(() => {
let text = DOMPurify.sanitize(content); let text = DOMPurify.sanitize(content, {
ADD_TAGS: ['think', 'section'],
ADD_ATTR: ['class'],
});
// let text = content; // let text = content;
if (text === '') { if (text === '') {
text = t('chat.searching'); text = t('chat.searching');