Compare commits

...

15 Commits

Author SHA1 Message Date
d8ef22db68 Fix(dataset): Optimized the dataset configuration page UI #9869 (#10066)
### What problem does this PR solve?
fix(dataset): Optimized the dataset configuration page UI

- Added the DataPipelineSelect component for selecting data pipelines
- Restructured the layout and style of the dataset settings page
- Removed unnecessary components and code
- Optimized data pipeline configuration
- Adjusted the Create Dataset dialog box
- Updated the processing log modal style

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-09-12 16:01:37 +08:00
592f3b1555 Feat: Bind options to the parser operator form. #9869 (#10069)
### What problem does this PR solve?

Feat: Bind options to the parser operator form. #9869

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-09-12 16:01:24 +08:00
3404469e2a Feat: Dynamically increase the configuration of the parser operator #9869 (#10060)
### What problem does this PR solve?

Feat: Dynamically increase the configuration of the parser operator
#9869
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-09-12 10:14:26 +08:00
63d7382dc9 fix: Displays the dataset creation and settings page #9869 (#10052)
### What problem does this PR solve?

[_Briefly describe what this PR aims to solve. Include background
context that will help reviewers understand the purpose of the
PR._](fix: Displays the dataset creation and settings page #9869)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-09-11 17:25:07 +08:00
179091b1a4 Fix: In ragflow/rag/app /naive.py, if there are multiple images in one line, the other images will be lost (#9968)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/9966

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-09-11 11:08:31 +08:00
d14d92a900 Feat: Translate the parser operator #9869 (#10037)
### What problem does this PR solve?

Feat: Translate the parser operator #9869

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-09-11 11:07:26 +08:00
1936ad82d2 Refactor:Improve BytesIO usage for GeminiCV (#10042)
### What problem does this PR solve?
Improve BytesIO usage for GeminiCV

### Type of change
- [x] Refactoring
2025-09-11 11:07:15 +08:00
8a09f07186 feat: Added UI functions related to data-flow knowledge base #3221 (#10038)
### What problem does this PR solve?

feat: Added UI functions related to data-flow knowledge base #3221

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-09-11 09:51:18 +08:00
df8d31451b Feat: Import dsl from agent list page #9869 (#10033)
### What problem does this PR solve?

Feat: Import dsl from agent list page #9869

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-09-10 18:22:16 +08:00
fc95d113c3 Feat(config): Update service config template new defaults (#10029)
### What problem does this PR solve?

- Update default LLM configuration with BAAI and model details #9404
- Add SMTP configuration section #9479
- Add OpenDAL storage configuration option #8232

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-09-10 16:39:26 +08:00
7d14455fbe Feat: Add type card to create agent dialog #9869 (#10025)
### What problem does this PR solve?

Feat: Add type card to create agent dialog #9869
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-09-10 15:56:10 +08:00
bbe6ed3b90 Fix: Fixed the issue where newly added tool operators would disappear after editing the form #10013 (#10016)
### What problem does this PR solve?

Fix: Fixed the issue where newly added tool operators would disappear
after editing the form #10013

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-09-10 15:55:59 +08:00
127af4e45c Refactor:Improve BytesIO usage for image2base64 (#9997)
### What problem does this PR solve?

Improve BytesIO usage for image2base64

### Type of change

- [x] Refactoring
2025-09-10 15:55:33 +08:00
41cdba19ba Feat: dataflow supports markdown (#10003)
### What problem does this PR solve?

Dataflow supports markdown.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-09-10 13:31:02 +08:00
0d9c1f1c3c Feat: dataflow supports Spreadsheet and Word processor document (#9996)
### What problem does this PR solve?

Dataflow supports Spreadsheet and Word processor document

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-09-10 13:02:53 +08:00
116 changed files with 6135 additions and 600 deletions

View File

@ -22,10 +22,10 @@ from openpyxl import Workbook, load_workbook
from rag.nlp import find_codec
# copied from `/openpyxl/cell/cell.py`
ILLEGAL_CHARACTERS_RE = re.compile(r'[\000-\010]|[\013-\014]|[\016-\037]')
ILLEGAL_CHARACTERS_RE = re.compile(r"[\000-\010]|[\013-\014]|[\016-\037]")
class RAGFlowExcelParser:
@staticmethod
def _load_excel_to_workbook(file_like_object):
if isinstance(file_like_object, bytes):
@ -36,7 +36,7 @@ class RAGFlowExcelParser:
file_head = file_like_object.read(4)
file_like_object.seek(0)
if not (file_head.startswith(b'PK\x03\x04') or file_head.startswith(b'\xD0\xCF\x11\xE0')):
if not (file_head.startswith(b"PK\x03\x04") or file_head.startswith(b"\xd0\xcf\x11\xe0")):
logging.info("Not an Excel file, converting CSV to Excel Workbook")
try:
@ -48,7 +48,7 @@ class RAGFlowExcelParser:
raise Exception(f"Failed to parse CSV and convert to Excel Workbook: {e_csv}")
try:
return load_workbook(file_like_object,data_only= True)
return load_workbook(file_like_object, data_only=True)
except Exception as e:
logging.info(f"openpyxl load error: {e}, try pandas instead")
try:
@ -59,7 +59,7 @@ class RAGFlowExcelParser:
except Exception as ex:
logging.info(f"pandas with default engine load error: {ex}, try calamine instead")
file_like_object.seek(0)
df = pd.read_excel(file_like_object, engine='calamine')
df = pd.read_excel(file_like_object, engine="calamine")
return RAGFlowExcelParser._dataframe_to_workbook(df)
except Exception as e_pandas:
raise Exception(f"pandas.read_excel error: {e_pandas}, original openpyxl error: {e}")
@ -116,9 +116,7 @@ class RAGFlowExcelParser:
tb = ""
tb += f"<table><caption>{sheetname}</caption>"
tb += tb_rows_0
for r in list(
rows[1 + chunk_i * chunk_rows: min(1 + (chunk_i + 1) * chunk_rows, len(rows))]
):
for r in list(rows[1 + chunk_i * chunk_rows : min(1 + (chunk_i + 1) * chunk_rows, len(rows))]):
tb += "<tr>"
for i, c in enumerate(r):
if c.value is None:
@ -133,8 +131,16 @@ class RAGFlowExcelParser:
def markdown(self, fnm):
import pandas as pd
file_like_object = BytesIO(fnm) if not isinstance(fnm, str) else fnm
df = pd.read_excel(file_like_object)
try:
file_like_object.seek(0)
df = pd.read_excel(file_like_object)
except Exception as e:
logging.warning(f"Parse spreadsheet error: {e}, trying to interpret as CSV file")
file_like_object.seek(0)
df = pd.read_csv(file_like_object)
df = df.replace(r"^\s*$", "", regex=True)
return df.to_markdown(index=False)
def __call__(self, fnm):

View File

@ -29,7 +29,6 @@ redis:
db: 1
password: '${REDIS_PASSWORD:-infini_rag_flow}'
host: '${REDIS_HOST:-redis}:6379'
# postgres:
# name: '${POSTGRES_DBNAME:-rag_flow}'
# user: '${POSTGRES_USER:-rag_flow}'
@ -65,15 +64,26 @@ redis:
# secret: 'secret'
# tenant_id: 'tenant_id'
# container_name: 'container_name'
# The OSS object storage uses the MySQL configuration above by default. If you need to switch to another object storage service, please uncomment and configure the following parameters.
# opendal:
# scheme: 'mysql' # Storage type, such as s3, oss, azure, etc.
# config:
# oss_table: 'opendal_storage'
# user_default_llm:
# factory: 'Tongyi-Qianwen'
# api_key: 'sk-xxxxxxxxxxxxx'
# base_url: ''
# factory: 'BAAI'
# api_key: 'backup'
# base_url: 'backup_base_url'
# default_models:
# chat_model: 'qwen-plus'
# embedding_model: 'BAAI/bge-large-zh-v1.5@BAAI'
# rerank_model: ''
# asr_model: ''
# chat_model:
# name: 'qwen2.5-7b-instruct'
# factory: 'xxxx'
# api_key: 'xxxx'
# base_url: 'https://api.xx.com'
# embedding_model:
# name: 'bge-m3'
# rerank_model: 'bge-reranker-v2'
# asr_model:
# model: 'whisper-large-v3' # alias of name
# image2text_model: ''
# oauth:
# oauth2:
@ -109,3 +119,14 @@ redis:
# switch: false
# component: false
# dataset: false
# smtp:
# mail_server: ""
# mail_port: 465
# mail_use_ssl: true
# mail_use_tls: false
# mail_username: ""
# mail_password: ""
# mail_default_sender:
# - "RAGFlow" # display name
# - "" # sender email address
# mail_frontend_url: "https://your-frontend.example.com"

View File

@ -41,37 +41,43 @@ class Docx(DocxParser):
pass
def get_picture(self, document, paragraph):
img = paragraph._element.xpath('.//pic:pic')
if not img:
return None
img = img[0]
embed = img.xpath('.//a:blip/@r:embed')
if not embed:
return None
embed = embed[0]
try:
related_part = document.part.related_parts[embed]
image_blob = related_part.image.blob
except UnrecognizedImageError:
logging.info("Unrecognized image format. Skipping image.")
return None
except UnexpectedEndOfFileError:
logging.info("EOF was unexpectedly encountered while reading an image stream. Skipping image.")
return None
except InvalidImageStreamError:
logging.info("The recognized image stream appears to be corrupted. Skipping image.")
return None
except UnicodeDecodeError:
logging.info("The recognized image stream appears to be corrupted. Skipping image.")
return None
except Exception:
logging.info("The recognized image stream appears to be corrupted. Skipping image.")
return None
try:
image = Image.open(BytesIO(image_blob)).convert('RGB')
return image
except Exception:
imgs = paragraph._element.xpath('.//pic:pic')
if not imgs:
return None
res_img = None
for img in imgs:
embed = img.xpath('.//a:blip/@r:embed')
if not embed:
continue
embed = embed[0]
try:
related_part = document.part.related_parts[embed]
image_blob = related_part.image.blob
except UnrecognizedImageError:
logging.info("Unrecognized image format. Skipping image.")
continue
except UnexpectedEndOfFileError:
logging.info("EOF was unexpectedly encountered while reading an image stream. Skipping image.")
continue
except InvalidImageStreamError:
logging.info("The recognized image stream appears to be corrupted. Skipping image.")
continue
except UnicodeDecodeError:
logging.info("The recognized image stream appears to be corrupted. Skipping image.")
continue
except Exception:
logging.info("The recognized image stream appears to be corrupted. Skipping image.")
continue
try:
image = Image.open(BytesIO(image_blob)).convert('RGB')
if res_img is None:
res_img = image
else:
res_img = concat_img(res_img, image)
except Exception:
continue
return res_img
def __clean(self, line):
line = re.sub(r"\u3000", " ", line).strip()

View File

@ -73,11 +73,13 @@ class Chunker(ProcessBase):
def _general(self, from_upstream: ChunkerFromUpstream):
self.callback(random.randint(1, 5) / 100.0, "Start to chunk via `General`.")
if from_upstream.output_format in ["markdown", "text"]:
if from_upstream.output_format in ["markdown", "text", "html"]:
if from_upstream.output_format == "markdown":
payload = from_upstream.markdown_result
else: # == "text"
elif from_upstream.output_format == "text":
payload = from_upstream.text_result
else: # == "html"
payload = from_upstream.html_result
if not payload:
payload = ""
@ -90,6 +92,7 @@ class Chunker(ProcessBase):
)
return [{"text": c} for c in cks]
# json
sections, section_images = [], []
for o in from_upstream.json_result or []:
sections.append((o.get("text", ""), o.get("position_tag", "")))

View File

@ -29,7 +29,7 @@ class ChunkerFromUpstream(BaseModel):
json_result: list[dict[str, Any]] | None = Field(default=None, alias="json")
markdown_result: str | None = Field(default=None, alias="markdown")
text_result: str | None = Field(default=None, alias="text")
html_result: str | None = Field(default=None, alias="html")
html_result: list[str] | None = Field(default=None, alias="html")
model_config = ConfigDict(populate_by_name=True, extra="forbid")

View File

@ -12,6 +12,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
import random
import trio
@ -29,8 +30,18 @@ class ParserParam(ProcessParamBase):
def __init__(self):
super().__init__()
self.allowed_output_format = {
"pdf": ["json", "markdown"],
"excel": ["json", "markdown", "html"],
"pdf": [
"json",
"markdown",
],
"spreadsheet": [
"json",
"markdown",
"html",
],
"word": [
"json",
],
"ppt": [],
"image": [],
"email": [],
@ -44,12 +55,29 @@ class ParserParam(ProcessParamBase):
"parse_method": "deepdoc", # deepdoc/plain_text/vlm
"vlm_name": "",
"lang": "Chinese",
"suffix": ["pdf"],
"suffix": [
"pdf",
],
"output_format": "json",
},
"excel": {
"spreadsheet": {
"output_format": "html",
"suffix": ["xls", "xlsx", "csv"],
"suffix": [
"xls",
"xlsx",
"csv",
],
},
"word": {
"suffix": [
"doc",
"docx",
],
"output_format": "json",
},
"markdown": {
"suffix": ["md", "markdown"],
"output_format": "json",
},
"ppt": {},
"image": {
@ -76,10 +104,15 @@ class ParserParam(ProcessParamBase):
pdf_output_format = pdf_config.get("output_format", "")
self.check_valid_value(pdf_output_format, "PDF output format abnormal.", self.allowed_output_format["pdf"])
excel_config = self.setups.get("excel", "")
if excel_config:
excel_output_format = excel_config.get("output_format", "")
self.check_valid_value(excel_output_format, "Excel output format abnormal.", self.allowed_output_format["excel"])
spreadsheet_config = self.setups.get("spreadsheet", "")
if spreadsheet_config:
spreadsheet_output_format = spreadsheet_config.get("output_format", "")
self.check_valid_value(spreadsheet_output_format, "Spreadsheet output format abnormal.", self.allowed_output_format["spreadsheet"])
doc_config = self.setups.get("doc", "")
if doc_config:
doc_output_format = doc_config.get("output_format", "")
self.check_valid_value(doc_output_format, "Word processer document output format abnormal.", self.allowed_output_format["doc"])
image_config = self.setups.get("image", "")
if image_config:
@ -93,10 +126,13 @@ class ParserParam(ProcessParamBase):
class Parser(ProcessBase):
component_name = "Parser"
def _pdf(self, blob):
def _pdf(self, from_upstream: ParserFromUpstream):
self.callback(random.randint(1, 5) / 100.0, "Start to work on a PDF.")
blob = from_upstream.blob
conf = self._param.setups["pdf"]
self.set_output("output_format", conf["output_format"])
if conf.get("parse_method") == "deepdoc":
bboxes = RAGFlowPdfParser().parse_into_bboxes(blob, callback=self.callback)
elif conf.get("parse_method") == "plain_text":
@ -110,6 +146,7 @@ class Parser(ProcessBase):
for t, poss in lines:
pn, x0, x1, top, bott = poss.split(" ")
bboxes.append({"page_number": int(pn), "x0": float(x0), "x1": float(x1), "top": float(top), "bottom": float(bott), "text": t})
if conf.get("output_format") == "json":
self.set_output("json", bboxes)
if conf.get("output_format") == "markdown":
@ -123,23 +160,93 @@ class Parser(ProcessBase):
mkdn += b.get("text", "") + "\n"
self.set_output("markdown", mkdn)
def _excel(self, blob):
self.callback(random.randint(1, 5) / 100.0, "Start to work on a Excel.")
conf = self._param.setups["excel"]
def _spreadsheet(self, from_upstream: ParserFromUpstream):
self.callback(random.randint(1, 5) / 100.0, "Start to work on a Spreadsheet.")
blob = from_upstream.blob
conf = self._param.setups["spreadsheet"]
self.set_output("output_format", conf["output_format"])
excel_parser = ExcelParser()
print("spreadsheet {conf=}", flush=True)
spreadsheet_parser = ExcelParser()
if conf.get("output_format") == "html":
html = excel_parser.html(blob, 1000000000)
html = spreadsheet_parser.html(blob, 1000000000)
self.set_output("html", html)
elif conf.get("output_format") == "json":
self.set_output("json", [{"text": txt} for txt in excel_parser(blob) if txt])
self.set_output("json", [{"text": txt} for txt in spreadsheet_parser(blob) if txt])
elif conf.get("output_format") == "markdown":
self.set_output("markdown", excel_parser.markdown(blob))
self.set_output("markdown", spreadsheet_parser.markdown(blob))
def _word(self, from_upstream: ParserFromUpstream):
from tika import parser as word_parser
self.callback(random.randint(1, 5) / 100.0, "Start to work on a Word Processor Document")
blob = from_upstream.blob
name = from_upstream.name
conf = self._param.setups["word"]
self.set_output("output_format", conf["output_format"])
print("word {conf=}", flush=True)
doc_parsed = word_parser.from_buffer(blob)
sections = []
if doc_parsed.get("content"):
sections = doc_parsed["content"].split("\n")
sections = [{"text": section} for section in sections if section]
else:
logging.warning(f"tika.parser got empty content from {name}.")
# json
assert conf.get("output_format") == "json", "have to be json for doc"
if conf.get("output_format") == "json":
self.set_output("json", sections)
def _markdown(self, from_upstream: ParserFromUpstream):
from functools import reduce
from rag.app.naive import Markdown as naive_markdown_parser
from rag.nlp import concat_img
self.callback(random.randint(1, 5) / 100.0, "Start to work on a Word Processor Document")
blob = from_upstream.blob
name = from_upstream.name
conf = self._param.setups["markdown"]
self.set_output("output_format", conf["output_format"])
print("markdown {conf=}", flush=True)
markdown_parser = naive_markdown_parser()
sections, tables = markdown_parser(name, blob, separate_tables=False)
# json
assert conf.get("output_format") == "json", "have to be json for doc"
if conf.get("output_format") == "json":
json_results = []
for section_text, _ in sections:
json_result = {
"text": section_text,
}
images = markdown_parser.get_pictures(section_text) if section_text else None
if images:
# If multiple images found, combine them using concat_img
combined_image = reduce(concat_img, images) if len(images) > 1 else images[0]
json_result["image"] = combined_image
json_results.append(json_result)
self.set_output("json", json_results)
async def _invoke(self, **kwargs):
function_map = {
"pdf": self._pdf,
"excel": self._excel,
"markdown": self._markdown,
"spreadsheet": self._spreadsheet,
"word": self._word
}
try:
from_upstream = ParserFromUpstream.model_validate(kwargs)
@ -150,5 +257,5 @@ class Parser(ProcessBase):
for p_type, conf in self._param.setups.items():
if from_upstream.name.split(".")[-1].lower() not in conf.get("suffix", []):
continue
await trio.to_thread.run_sync(function_map[p_type], from_upstream.blob)
await trio.to_thread.run_sync(function_map[p_type], from_upstream)
break

View File

@ -23,16 +23,31 @@
],
"output_format": "json"
},
"excel": {
"output_format": "html",
"spreadsheet": {
"suffix": [
"xls",
"xlsx",
"csv"
]
],
"output_format": "html"
},
"word": {
"suffix": [
"doc",
"docx"
],
"output_format": "json"
},
"markdown": {
"suffix": [
"md",
"markdown"
],
"output_format": "json"
}
}
}
}
},
"downstream": ["Chunker:0"],
"upstream": ["Begin"]

View File

@ -31,7 +31,7 @@ class TokenizerFromUpstream(BaseModel):
json_result: list[dict[str, Any]] | None = Field(default=None, alias="json")
markdown_result: str | None = Field(default=None, alias="markdown")
text_result: str | None = Field(default=None, alias="text")
html_result: str | None = Field(default=None, alias="html")
html_result: list[str] | None = Field(default=None, alias="html")
model_config = ConfigDict(populate_by_name=True, extra="forbid")

View File

@ -117,11 +117,13 @@ class Tokenizer(ProcessBase):
ck["content_sm_ltks"] = rag_tokenizer.fine_grained_tokenize(ck["content_ltks"])
if i % 100 == 99:
self.callback(i * 1.0 / len(chunks) / parts)
elif from_upstream.output_format in ["markdown", "text"]:
elif from_upstream.output_format in ["markdown", "text", "html"]:
if from_upstream.output_format == "markdown":
payload = from_upstream.markdown_result
else: # == "text"
elif from_upstream.output_format == "text":
payload = from_upstream.text_result
else: # == "html"
payload = from_upstream.html_result
if not payload:
return ""

View File

@ -124,17 +124,19 @@ class Base(ABC):
mime = "image/jpeg"
b64 = base64.b64encode(data).decode("utf-8")
return f"data:{mime};base64,{b64}"
buffered = BytesIO()
fmt = "JPEG"
try:
image.save(buffered, format="JPEG")
except Exception:
buffered = BytesIO() # reset buffer before saving PNG
image.save(buffered, format="PNG")
fmt = "PNG"
data = buffered.getvalue()
b64 = base64.b64encode(data).decode("utf-8")
mime = f"image/{fmt.lower()}"
with BytesIO() as buffered:
fmt = "JPEG"
try:
image.save(buffered, format="JPEG")
except Exception:
# reset buffer before saving PNG
buffered.seek(0)
buffered.truncate()
image.save(buffered, format="PNG")
fmt = "PNG"
data = buffered.getvalue()
b64 = base64.b64encode(data).decode("utf-8")
mime = f"image/{fmt.lower()}"
return f"data:{mime};base64,{b64}"
def prompt(self, b64):
@ -519,24 +521,24 @@ class GeminiCV(Base):
else "Please describe the content of this picture, like where, when, who, what happen. If it has number data, please extract them out."
)
b64 = self.image2base64(image)
img = open(BytesIO(base64.b64decode(b64)))
input = [prompt, img]
res = self.model.generate_content(input)
img.close()
return res.text, res.usage_metadata.total_token_count
with BytesIO(base64.b64decode(b64)) as bio:
img = open(bio)
input = [prompt, img]
res = self.model.generate_content(input)
img.close()
return res.text, res.usage_metadata.total_token_count
def describe_with_prompt(self, image, prompt=None):
from PIL.Image import open
b64 = self.image2base64(image)
vision_prompt = prompt if prompt else vision_llm_describe_prompt()
img = open(BytesIO(base64.b64decode(b64)))
input = [vision_prompt, img]
res = self.model.generate_content(
input,
)
img.close()
return res.text, res.usage_metadata.total_token_count
with BytesIO(base64.b64decode(b64)) as bio:
img = open(bio)
input = [vision_prompt, img]
res = self.model.generate_content(input)
img.close()
return res.text, res.usage_metadata.total_token_count
def chat(self, system, history, gen_conf, images=[]):
generation_config = dict(temperature=gen_conf.get("temperature", 0.3), top_p=gen_conf.get("top_p", 0.7))

View File

@ -751,6 +751,8 @@ class SILICONFLOWEmbed(Base):
token_count = 0
for i in range(0, len(texts), batch_size):
texts_batch = texts[i : i + batch_size]
texts_batch = [" " if not text.strip() else text for text in texts_batch]
payload = {
"model": self.model_name,
"input": texts_batch,
@ -935,7 +937,7 @@ class GiteeEmbed(SILICONFLOWEmbed):
if not base_url:
base_url = "https://ai.gitee.com/v1/embeddings"
super().__init__(key, model_name, base_url)
class DeepInfraEmbed(OpenAIEmbed):
_FACTORY_NAME = "DeepInfra"
@ -951,4 +953,4 @@ class Ai302Embed(Base):
def __init__(self, key, model_name, base_url="https://api.302.ai/v1/embeddings"):
if not base_url:
base_url = "https://api.302.ai/v1/embeddings"
super().__init__(key, model_name, base_url)
super().__init__(key, model_name, base_url)

View File

@ -518,7 +518,7 @@ def hierarchical_merge(bull, sections, depth):
return res
def naive_merge(sections, chunk_token_num=128, delimiter="\n。;!?", overlapped_percent=0):
def naive_merge(sections: str | list, chunk_token_num=128, delimiter="\n。;!?", overlapped_percent=0):
from deepdoc.parser.pdf_parser import RAGFlowPdfParser
if not sections:
return []
@ -534,7 +534,7 @@ def naive_merge(sections, chunk_token_num=128, delimiter="\n。", overl
pos = ""
if tnum < 8:
pos = ""
# Ensure that the length of the merged chunk does not exceed chunk_token_num
# Ensure that the length of the merged chunk does not exceed chunk_token_num
if cks[-1] == "" or tk_nums[-1] > chunk_token_num * (100 - overlapped_percent)/100.:
if cks:
overlapped = RAGFlowPdfParser.remove_tag(cks[-1])
@ -638,10 +638,10 @@ def concat_img(img1, img2):
return img2
if not img1 and not img2:
return None
if img1 is img2:
return img1
if isinstance(img1, Image.Image) and isinstance(img2, Image.Image):
pixel_data1 = img1.tobytes()
pixel_data2 = img2.tobytes()

8
web/package-lock.json generated
View File

@ -66,7 +66,7 @@
"jsencrypt": "^3.3.2",
"lexical": "^0.23.1",
"lodash": "^4.17.21",
"lucide-react": "^0.508.0",
"lucide-react": "^0.542.0",
"mammoth": "^1.7.2",
"next-themes": "^0.4.6",
"openai-speech-stream-player": "^1.0.8",
@ -25113,9 +25113,9 @@
}
},
"node_modules/lucide-react": {
"version": "0.508.0",
"resolved": "https://registry.npmmirror.com/lucide-react/-/lucide-react-0.508.0.tgz",
"integrity": "sha512-gcP16PnexqtOFrTtv98kVsGzTfnbPekzZiQfByi2S89xfk7E/4uKE1USZqccIp58v42LqkO7MuwpCqshwSrJCg==",
"version": "0.542.0",
"resolved": "https://registry.npmmirror.com/lucide-react/-/lucide-react-0.542.0.tgz",
"integrity": "sha512-w3hD8/SQB7+lzU2r4VdFyzzOzKnUjTZIF/MQJGSSvni7Llewni4vuViRppfRAa2guOsY5k4jZyxw/i9DQHv+dw==",
"license": "ISC",
"peerDependencies": {
"react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0"

View File

@ -79,7 +79,7 @@
"jsencrypt": "^3.3.2",
"lexical": "^0.23.1",
"lodash": "^4.17.21",
"lucide-react": "^0.508.0",
"lucide-react": "^0.542.0",
"mammoth": "^1.7.2",
"next-themes": "^0.4.6",
"openai-speech-stream-player": "^1.0.8",

View File

@ -0,0 +1 @@
<?xml version="1.0" standalone="no"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1756884949583" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="11332" xmlns:xlink="http://www.w3.org/1999/xlink" width="200" height="200"><path d="M190.464 489.472h327.68v40.96h-327.68z" fill="#C7DCFE" p-id="11333"></path><path d="M482.34496 516.5056l111.26784-308.20352 38.54336 13.9264L520.86784 530.432z" fill="#C7DCFE" p-id="11334"></path><path d="M620.544 196.608m-122.88 0a122.88 122.88 0 1 0 245.76 0 122.88 122.88 0 1 0-245.76 0Z" fill="#8FB8FC" p-id="11335"></path><path d="M182.272 509.952m-122.88 0a122.88 122.88 0 1 0 245.76 0 122.88 122.88 0 1 0-245.76 0Z" fill="#C7DCFE" p-id="11336"></path><path d="M558.65344 520.9088l283.77088 163.84-20.48 35.47136-283.77088-163.84z" fill="#C7DCFE" p-id="11337"></path><path d="M841.728 686.08m-122.88 0a122.88 122.88 0 1 0 245.76 0 122.88 122.88 0 1 0-245.76 0Z" fill="#B3CEFE" p-id="11338"></path><path d="M448.67584 803.77856l49.60256-323.91168 40.48896 6.20544-49.60256 323.91168z" fill="#C7DCFE" p-id="11339"></path><path d="M512 530.432m-143.36 0a143.36 143.36 0 1 0 286.72 0 143.36 143.36 0 1 0-286.72 0Z" fill="#4185FF" p-id="11340"></path><path d="M462.848 843.776m-102.4 0a102.4 102.4 0 1 0 204.8 0 102.4 102.4 0 1 0-204.8 0Z" fill="#8FB8FC" p-id="11341"></path></svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 36 KiB

View File

@ -0,0 +1 @@
<?xml version="1.0" standalone="no"?><!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"><svg t="1757483419289" class="icon" viewBox="0 0 1024 1024" version="1.1" xmlns="http://www.w3.org/2000/svg" p-id="22299" xmlns:xlink="http://www.w3.org/1999/xlink" width="200" height="200"><path d="M998.765714 523.629714c13.824 0 25.014857 11.190857 25.014857 25.014857a475.282286 475.282286 0 0 1-875.593142 256.219429l-27.574858 55.149714a25.014857 25.014857 0 1 1-44.763428-22.454857l44.178286-88.283428a24.868571 24.868571 0 0 1 26.550857-25.526858 25.014857 25.014857 0 0 1 8.265143 0.804572l99.474285 24.868571a25.014857 25.014857 0 0 1-12.068571 48.566857l-46.372572-11.556571A425.252571 425.252571 0 0 0 973.750857 548.571429c0-13.897143 11.190857-25.014857 25.014857-25.014858zM430.957714 365.714286l6.729143 0.658285c2.633143 0.438857 285.549714 160.109714 285.549714 160.109715 20.114286 17.846857 7.314286 34.523429-6.582857 45.933714-1.828571 1.462857-194.779429 113.078857-249.929143 144.969143l-10.678857 6.217143-3.876571 2.194285c-16.676571 8.923429-39.497143 8.923429-47.250286-11.995428-0.877714-2.194286-2.267429-250.221714-2.56-303.396572L402.285714 400.457143l0.731429-0.512c0.731429-18.651429 8.265143-38.034286 34.669714-33.645714z m-15.945143-273.408a475.282286 475.282286 0 0 1 533.869715 200.045714l27.501714-55.149714a25.014857 25.014857 0 1 1 44.690286 22.454857l-44.105143 88.283428a24.868571 24.868571 0 0 1-26.624 25.526858 24.868571 24.868571 0 0 1-8.192-0.804572l-99.547429-24.868571a25.014857 25.014857 0 0 1 12.068572-48.566857l46.445714 11.629714A425.252571 425.252571 0 0 0 123.245714 548.571429a25.014857 25.014857 0 0 1-50.029714 0 475.282286 475.282286 0 0 1 341.796571-456.265143z" fill="#3BA05C" p-id="22300"></path></svg>

After

Width:  |  Height:  |  Size: 1.8 KiB

View File

@ -30,11 +30,13 @@ const options = Languages.map((x) => ({
type CrossLanguageItemProps = {
name?: string;
vertical?: boolean;
label?: string;
};
export const CrossLanguageFormField = ({
name = 'prompt_config.cross_languages',
vertical = true,
label,
}: CrossLanguageItemProps) => {
const { t } = useTranslation();
const form = useFormContext();
@ -53,7 +55,7 @@ export const CrossLanguageFormField = ({
})}
>
<FormLabel tooltip={t('chat.crossLanguageTip')}>
{t('chat.crossLanguage')}
{label || t('chat.crossLanguage')}
</FormLabel>
<FormControl>
<MultiSelect

View File

@ -0,0 +1,76 @@
import { useTranslate } from '@/hooks/common-hooks';
import { buildSelectOptions } from '@/utils/component-util';
import { ArrowUpRight } from 'lucide-react';
import { useFormContext } from 'react-hook-form';
import {
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '../ui/form';
import { RAGFlowSelect } from '../ui/select';
interface IProps {
toDataPipeline?: () => void;
formFieldName: string;
}
const data = [
{ id: '1', name: 'data-pipeline-1' },
{ id: '2', name: 'data-pipeline-2' },
{ id: '3', name: 'data-pipeline-3' },
{ id: '4', name: 'data-pipeline-4' },
];
export function DataFlowItem(props: IProps) {
const { toDataPipeline, formFieldName } = props;
const { t } = useTranslate('knowledgeConfiguration');
const form = useFormContext();
const toDataPipLine = () => {
// window.open('/data-pipeline');
toDataPipeline?.();
};
const options = buildSelectOptions(data, 'id', 'name');
return (
<FormField
control={form.control}
name={formFieldName}
render={({ field }) => (
<FormItem className=" items-center space-y-0 ">
<div className="flex flex-col gap-1">
<div className="flex gap-2 justify-between ">
<FormLabel
tooltip={t('dataFlowTip')}
className="text-sm text-text-primary whitespace-wrap "
>
{t('dataFlow')}
</FormLabel>
<div
className="text-sm flex text-text-primary cursor-pointer"
onClick={toDataPipLine}
>
{t('buildItFromScratch')}
<ArrowUpRight size={14} />
</div>
</div>
<div className="text-muted-foreground">
<FormControl>
<RAGFlowSelect
{...field}
placeholder={t('dataFlowPlaceholder')}
options={options}
/>
</FormControl>
</div>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
);
}

View File

@ -0,0 +1,57 @@
// src/pages/dataset/file-logs/file-status-badge.tsx
import { FC } from 'react';
interface StatusBadgeProps {
status: 'Success' | 'Failed' | 'Running' | 'Pending';
}
const FileStatusBadge: FC<StatusBadgeProps> = ({ status }) => {
const getStatusColor = () => {
// #3ba05c → rgb(59, 160, 92) // state-success
// #d8494b → rgb(216, 73, 75) // state-error
// #00beb4 → rgb(0, 190, 180) // accent-primary
// #faad14 → rgb(250, 173, 20) // state-warning
switch (status) {
case 'Success':
return `bg-[rgba(59,160,92,0.1)] text-state-success`;
case 'Failed':
return `bg-[rgba(216,73,75,0.1)] text-state-error`;
case 'Running':
return `bg-[rgba(0,190,180,0.1)] text-accent-primary`;
case 'Pending':
return `bg-[rgba(250,173,20,0.1)] text-state-warning`;
default:
return 'bg-gray-500/10 text-white';
}
};
const getBgStatusColor = () => {
// #3ba05c → rgb(59, 160, 92) // state-success
// #d8494b → rgb(216, 73, 75) // state-error
// #00beb4 → rgb(0, 190, 180) // accent-primary
// #faad14 → rgb(250, 173, 20) // state-warning
switch (status) {
case 'Success':
return `bg-[rgba(59,160,92,1)] text-state-success`;
case 'Failed':
return `bg-[rgba(216,73,75,1)] text-state-error`;
case 'Running':
return `bg-[rgba(0,190,180,1)] text-accent-primary`;
case 'Pending':
return `bg-[rgba(250,173,20,1)] text-state-warning`;
default:
return 'bg-gray-500/10 text-white';
}
};
return (
<span
className={`inline-flex items-center w-[75px] px-2 py-1 rounded-full text-xs font-medium ${getStatusColor(0.1)}`}
>
<div className={`w-1 h-1 mr-1 rounded-full ${getBgStatusColor()}`}></div>
{status}
</span>
);
};
export default FileStatusBadge;

View File

@ -1,6 +1,7 @@
import { LlmModelType } from '@/constants/knowledge';
import { useTranslate } from '@/hooks/common-hooks';
import { useSelectLlmOptionsByModelType } from '@/hooks/llm-hooks';
import { cn } from '@/lib/utils';
import { camelCase } from 'lodash';
import { useMemo } from 'react';
import { useFormContext } from 'react-hook-form';
@ -18,7 +19,13 @@ export const enum DocumentType {
PlainText = 'Plain Text',
}
export function LayoutRecognizeFormField() {
export function LayoutRecognizeFormField({
name = 'parser_config.layout_recognize',
horizontal = true,
}: {
name?: string;
horizontal?: boolean;
}) {
const form = useFormContext();
const { t } = useTranslate('knowledgeDetails');
@ -53,33 +60,32 @@ export function LayoutRecognizeFormField() {
return (
<FormField
control={form.control}
name="parser_config.layout_recognize"
name={name}
render={({ field }) => {
if (typeof field.value === 'undefined') {
// default value set
form.setValue(
'parser_config.layout_recognize',
form.formState.defaultValues?.parser_config?.layout_recognize ??
'DeepDOC',
);
}
return (
<FormItem className=" items-center space-y-0 ">
<div className="flex items-center">
<FormItem className={'items-center space-y-0 '}>
<div
className={cn('flex', {
'flex-col ': !horizontal,
'items-center': horizontal,
})}
>
<FormLabel
tooltip={t('layoutRecognizeTip')}
className="text-sm text-muted-foreground whitespace-wrap w-1/4"
className={cn('text-sm text-muted-foreground whitespace-wrap', {
['w-1/4']: horizontal,
})}
>
{t('layoutRecognize')}
</FormLabel>
<div className="w-3/4">
<div className={horizontal ? 'w-3/4' : 'w-full'}>
<FormControl>
<RAGFlowSelect {...field} options={options}></RAGFlowSelect>
</FormControl>
</div>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<div className={horizontal ? 'w-1/4' : 'w-full'}></div>
<FormMessage />
</div>
</FormItem>

View File

@ -0,0 +1,25 @@
import { LlmModelType } from '@/constants/knowledge';
import { useComposeLlmOptionsByModelTypes } from '@/hooks/llm-hooks';
import { useTranslation } from 'react-i18next';
import { SelectWithSearch } from '../originui/select-with-search';
import { RAGFlowFormItem } from '../ragflow-form';
type LLMFormFieldProps = {
options?: any[];
name?: string;
};
export function LLMFormField({ options, name }: LLMFormFieldProps) {
const { t } = useTranslation();
const modelOptions = useComposeLlmOptionsByModelTypes([
LlmModelType.Chat,
LlmModelType.Image2text,
]);
return (
<RAGFlowFormItem name={name || 'llm_id'} label={t('model')}>
<SelectWithSearch options={options || modelOptions}></SelectWithSearch>
</RAGFlowFormItem>
);
}

View File

@ -1,11 +1,9 @@
import { LlmModelType, ModelVariableType } from '@/constants/knowledge';
import { ModelVariableType } from '@/constants/knowledge';
import { useTranslate } from '@/hooks/common-hooks';
import { useComposeLlmOptionsByModelTypes } from '@/hooks/llm-hooks';
import { camelCase } from 'lodash';
import { useCallback } from 'react';
import { useFormContext } from 'react-hook-form';
import { z } from 'zod';
import { SelectWithSearch } from '../originui/select-with-search';
import {
FormControl,
FormField,
@ -20,6 +18,7 @@ import {
SelectTrigger,
SelectValue,
} from '../ui/select';
import { LLMFormField } from './llm-form-field';
import { SliderInputSwitchFormField } from './slider';
import { useHandleFreedomChange } from './use-watch-change';
@ -61,11 +60,6 @@ export function LlmSettingFieldItems({
const form = useFormContext();
const { t } = useTranslate('chat');
const modelOptions = useComposeLlmOptionsByModelTypes([
LlmModelType.Chat,
LlmModelType.Image2text,
]);
const getFieldWithPrefix = useCallback(
(name: string) => {
return prefix ? `${prefix}.${name}` : name;
@ -82,22 +76,7 @@ export function LlmSettingFieldItems({
return (
<div className="space-y-5">
<FormField
control={form.control}
name={'llm_id'}
render={({ field }) => (
<FormItem>
<FormLabel>{t('model')}</FormLabel>
<FormControl>
<SelectWithSearch
options={options || modelOptions}
{...field}
></SelectWithSearch>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<LLMFormField options={options}></LLMFormField>
<FormField
control={form.control}
name={'parameter'}

View File

@ -1,6 +1,7 @@
'use client';
import { cn } from '@/lib/utils';
import { parseColorToRGBA } from '@/utils/common-util';
import { Slot } from '@radix-ui/react-slot';
import * as React from 'react';
@ -197,7 +198,208 @@ function TimelineTitle({
);
}
interface TimelineIndicatorNodeProps {
nodeSize?: string | number;
iconColor?: string;
lineColor?: string;
textColor?: string;
indicatorBgColor?: string;
indicatorBorderColor?: string;
}
interface TimelineNode
extends Omit<
React.HTMLAttributes<HTMLDivElement>,
'id' | 'title' | 'content'
>,
TimelineIndicatorNodeProps {
id: string | number;
title?: React.ReactNode;
content?: React.ReactNode;
date?: React.ReactNode;
icon?: React.ReactNode;
completed?: boolean;
clickable?: boolean;
activeStyle?: TimelineIndicatorNodeProps;
}
interface CustomTimelineProps extends React.HTMLAttributes<HTMLDivElement> {
nodes: TimelineNode[];
activeStep?: number;
nodeSize?: string | number;
onStepChange?: (step: number, id: string | number) => void;
orientation?: 'horizontal' | 'vertical';
lineStyle?: 'solid' | 'dashed';
lineColor?: string;
indicatorColor?: string;
defaultValue?: number;
activeStyle?: TimelineIndicatorNodeProps;
}
const CustomTimeline = ({
nodes,
activeStep,
nodeSize = 12,
onStepChange,
orientation = 'horizontal',
lineStyle = 'solid',
lineColor = 'var(--text-secondary)',
indicatorColor = 'var(--accent-primary)',
defaultValue = 1,
className,
activeStyle,
...props
}: CustomTimelineProps) => {
const [internalActiveStep, setInternalActiveStep] =
React.useState(defaultValue);
const _lineColor = `rgb(${parseColorToRGBA(lineColor)})`;
console.log(lineColor, _lineColor);
const currentActiveStep = activeStep ?? internalActiveStep;
const handleStepChange = (step: number, id: string | number) => {
if (activeStep === undefined) {
setInternalActiveStep(step);
}
onStepChange?.(step, id);
};
const [r, g, b] = parseColorToRGBA(indicatorColor);
return (
<Timeline
value={currentActiveStep}
onValueChange={(step) => handleStepChange(step, nodes[step - 1]?.id)}
orientation={orientation}
className={className}
{...props}
>
{nodes.map((node, index) => {
const step = index + 1;
const isCompleted = node.completed ?? step <= currentActiveStep;
const isActive = step === currentActiveStep;
const isClickable = node.clickable ?? true;
const _activeStyle = node.activeStyle ?? (activeStyle || {});
const _nodeSizeTemp =
isActive && _activeStyle?.nodeSize
? _activeStyle?.nodeSize
: node.nodeSize ?? nodeSize;
const _nodeSize =
typeof _nodeSizeTemp === 'number'
? `${_nodeSizeTemp}px`
: _nodeSizeTemp;
console.log('icon-size', nodeSize, node.nodeSize, _nodeSize);
// const activeStyle = _activeStyle || {};
return (
<TimelineItem
key={node.id}
step={step}
className={cn(
node.className,
isClickable &&
'cursor-pointer hover:opacity-80 transition-opacity',
isCompleted && 'data-[completed]:data-completed/timeline-item',
isActive && 'relative z-10',
)}
onClick={() => isClickable && handleStepChange(step, node.id)}
>
<TimelineSeparator
className={cn(
'group-data-[orientation=horizontal]/timeline:-top-6 group-data-[orientation=horizontal]/timeline:h-0.1 group-data-[orientation=horizontal]/timeline:-translate-y-1/2',
'group-data-[orientation=vertical]/timeline:-left-6 group-data-[orientation=vertical]/timeline:w-0.1 group-data-[orientation=vertical]/timeline:-translate-x-1/2 ',
// `group-data-[orientation=horizontal]/timeline:w-[calc(100%-0.5rem-1rem)] group-data-[orientation=vertical]/timeline:h-[calc(100%-1rem-1rem)] group-data-[orientation=vertical]/timeline:translate-y-7 group-data-[orientation=horizontal]/timeline:translate-x-7`,
)}
style={{
border:
lineStyle === 'dashed'
? `1px dashed ${isActive ? _activeStyle.lineColor || _lineColor : _lineColor}`
: lineStyle === 'solid'
? `1px solid ${isActive ? _activeStyle.lineColor || _lineColor : _lineColor}`
: 'none',
backgroundColor: 'transparent',
width:
orientation === 'horizontal'
? `calc(100% - ${_nodeSize} - 2px - 0.1rem)`
: '1px',
height:
orientation === 'vertical'
? `calc(100% - ${_nodeSize} - 2px - 0.1rem)`
: '1px',
transform: `translate(${
orientation === 'horizontal' ? `${_nodeSize}` : '0'
}, ${orientation === 'vertical' ? `${_nodeSize}` : '0'})`,
}}
/>
<TimelineIndicator
className={cn(
'flex items-center justify-center p-1',
isCompleted && 'bg-primary border-primary',
!isCompleted && 'border-text-secondary bg-bg-base',
)}
style={{
width: _nodeSize,
height: _nodeSize,
borderColor: isActive
? _activeStyle.indicatorBorderColor || indicatorColor
: isCompleted
? indicatorColor
: '',
// backgroundColor: isActive
// ? _activeStyle.indicatorBgColor || indicatorColor
// : isCompleted
// ? indicatorColor
// : '',
backgroundColor: isActive
? _activeStyle.indicatorBgColor ||
`rgba(${r}, ${g}, ${b}, 0.1)`
: isCompleted
? `rgba(${r}, ${g}, ${b}, 0.1)`
: '',
}}
>
{node.icon && (
<div
className={cn(
'text-current',
`w-[${_nodeSize}] h-[${_nodeSize}]`,
isActive &&
`text-primary w-[${_activeStyle.nodeSize || _nodeSize}] h-[${_activeStyle.nodeSize || _nodeSize}]`,
)}
style={{
color: isActive ? _activeStyle.iconColor : undefined,
}}
>
{node.icon}
</div>
)}
</TimelineIndicator>
<TimelineHeader>
{node.date && <TimelineDate>{node.date}</TimelineDate>}
<TimelineTitle
className={cn(
'text-sm font-medium',
isActive && _activeStyle.textColor
? `text-${_activeStyle.textColor}`
: '',
)}
style={{
color: isActive ? _activeStyle.textColor : undefined,
}}
>
{node.title}
</TimelineTitle>
</TimelineHeader>
{node.content && <TimelineContent>{node.content}</TimelineContent>}
</TimelineItem>
);
})}
</Timeline>
);
};
CustomTimeline.displayName = 'CustomTimeline';
export {
CustomTimeline,
Timeline,
TimelineContent,
TimelineDate,
@ -206,4 +408,5 @@ export {
TimelineItem,
TimelineSeparator,
TimelineTitle,
type TimelineNode,
};

View File

@ -15,6 +15,8 @@ type RAGFlowFormItemProps = {
tooltip?: ReactNode;
children: ReactNode | ((field: ControllerRenderProps) => ReactNode);
horizontal?: boolean;
required?: boolean;
labelClassName?: string;
};
export function RAGFlowFormItem({
@ -23,6 +25,8 @@ export function RAGFlowFormItem({
tooltip,
children,
horizontal = false,
required = false,
labelClassName,
}: RAGFlowFormItemProps) {
const form = useFormContext();
return (
@ -35,7 +39,11 @@ export function RAGFlowFormItem({
'flex items-center': horizontal,
})}
>
<FormLabel tooltip={tooltip} className={cn({ 'w-1/4': horizontal })}>
<FormLabel
required={required}
tooltip={tooltip}
className={cn({ 'w-1/4': horizontal }, labelClassName)}
>
{label}
</FormLabel>
<FormControl>

View File

@ -3,10 +3,22 @@ import React from 'react';
interface SpotlightProps {
className?: string;
opcity?: number;
coverage?: number;
}
const Spotlight: React.FC<SpotlightProps> = ({ className }) => {
/**
*
* @param opcity 0~1 default 0.5
* @param coverage 0~100 default 60
* @returns
*/
const Spotlight: React.FC<SpotlightProps> = ({
className,
opcity = 0.5,
coverage = 60,
}) => {
const isDark = useIsDarkTheme();
const rgb = isDark ? '255, 255, 255' : '194, 221, 243';
return (
<div
className={`absolute inset-0 opacity-80 ${className} rounded-lg`}
@ -18,9 +30,7 @@ const Spotlight: React.FC<SpotlightProps> = ({ className }) => {
<div
className="absolute inset-0"
style={{
background: isDark
? 'radial-gradient(circle at 50% 190%, #fff4 0%, #fff0 60%)'
: 'radial-gradient(circle at 50% 190%, #E4F3FF 0%, #E4F3FF00 60%)',
background: `radial-gradient(circle at 50% 190%, rgba(${rgb},${opcity}) 0%, rgba(${rgb},0) ${coverage}%)`,
pointerEvents: 'none',
}}
></div>

View File

@ -38,7 +38,7 @@ const DialogContent = React.forwardRef<
<DialogPrimitive.Content
ref={ref}
className={cn(
'fixed left-[50%] top-[50%] z-50 grid w-full max-w-xl translate-x-[-50%] translate-y-[-50%] gap-4 border bg-colors-background-neutral-standard p-6 shadow-lg duration-200 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[state=closed]:slide-out-to-left-1/2 data-[state=closed]:slide-out-to-top-[48%] data-[state=open]:slide-in-from-left-1/2 data-[state=open]:slide-in-from-top-[48%] sm:rounded-lg',
'fixed left-[50%] top-[50%] z-50 grid w-full max-w-xl translate-x-[-50%] translate-y-[-50%] gap-4 border bg-bg-base p-6 shadow-lg duration-200 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0 data-[state=closed]:zoom-out-95 data-[state=open]:zoom-in-95 data-[state=closed]:slide-out-to-left-1/2 data-[state=closed]:slide-out-to-top-[48%] data-[state=open]:slide-in-from-left-1/2 data-[state=open]:slide-in-from-top-[48%] sm:rounded-lg',
className,
)}
{...props}

View File

@ -125,6 +125,16 @@ export const useNavigatePage = () => {
[navigate],
);
const navigateToDataflowResult = useCallback(
(id: string, knowledgeId?: string) => () => {
navigate(
// `${Routes.ParsedResult}/${id}?${QueryStringMap.KnowledgeId}=${knowledgeId}`,
`${Routes.DataflowResult}/${id}`,
);
},
[navigate],
);
return {
navigateToDatasetList,
navigateToDataset,
@ -144,5 +154,6 @@ export const useNavigatePage = () => {
navigateToFiles,
navigateToAgentList,
navigateToOldProfile,
navigateToDataflowResult,
};
};

View File

@ -24,9 +24,7 @@ import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
import { useDebounce } from 'ahooks';
import { get, set } from 'lodash';
import { useCallback, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { useParams, useSearchParams } from 'umi';
import { v4 as uuid } from 'uuid';
import {
useGetPaginationWithRouter,
useHandleSearchChange,
@ -80,7 +78,7 @@ export const EmptyDsl = {
component_name: 'Begin',
params: {},
},
downstream: ['Answer:China'], // other edge target is downstream, edge source is current node id
downstream: [], // other edge target is downstream, edge source is current node id
upstream: [], // edge source is upstream, edge target is current node id
},
},
@ -96,21 +94,11 @@ export const EmptyDsl = {
};
export const useFetchAgentTemplates = () => {
const { t } = useTranslation();
const { data } = useQuery<IFlowTemplate[]>({
queryKey: [AgentApiAction.FetchAgentTemplates],
initialData: [],
queryFn: async () => {
const { data } = await agentService.listTemplates();
if (Array.isArray(data?.data)) {
data.data.unshift({
id: uuid(),
title: t('flow.blank'),
description: t('flow.createFromNothing'),
dsl: EmptyDsl,
});
}
return data.data;
},

View File

@ -41,8 +41,8 @@ export interface DSL {
path?: string[];
answer?: any[];
graph?: IGraph;
messages: Message[];
reference: IReference[];
messages?: Message[];
reference?: IReference[];
globals: Record<string, any>;
retrieval: IReference[];
}

View File

@ -102,6 +102,28 @@ export default {
noMoreData: `That's all. Nothing more.`,
},
knowledgeDetails: {
generateKnowledgeGraph:
'This will extract entities and relationships from all your documents in this dataset. The process may take a while to complete.',
generateRaptor:
'This will extract entities and relationships from all your documents in this dataset. The process may take a while to complete.',
generate: 'Generate',
raptor: 'Raptor',
knowledgeGraph: 'Knowledge Graph',
processingType: 'Processing Type',
dataPipeline: 'Data Pipeline',
operations: 'Operations',
status: 'Status',
task: 'Task',
startDate: 'Start Date',
source: 'Source',
fileName: 'File Name',
datasetLogs: 'Dataset Logs',
fileLogs: 'File Logs',
overview: 'Overview',
success: 'Success',
failed: 'Failed',
completed: 'Completed',
processLog: 'Process Log',
created: 'Created',
learnMore: 'Learn More',
general: 'General',
@ -195,6 +217,7 @@ export default {
chunk: 'Chunk',
bulk: 'Bulk',
cancel: 'Cancel',
close: 'Close',
rerankModel: 'Rerank model',
rerankPlaceholder: 'Please select',
rerankTip: `Optional. If left empty, RAGFlow will use a combination of weighted keyword similarity and weighted vector cosine similarity; if a rerank model is selected, a weighted reranking score will replace the weighted vector cosine similarity. Please be aware that using a rerank model will significantly increase the system's response time. If you wish to use a rerank model, ensure you use a SaaS reranker; if you prefer a locally deployed rerank model, ensure you start RAGFlow with docker-compose-gpu.yml.`,
@ -238,6 +261,17 @@ export default {
reRankModelWaring: 'Re-rank model is very time consuming.',
},
knowledgeConfiguration: {
default: 'Default',
dataPipeline: 'Data Pipeline',
linkDataPipeline: 'Link Data Pipeline',
enableAutoGenerate: 'Enable Auto Generate',
teamPlaceholder: 'Please select a team.',
dataFlowPlaceholder: 'Please select a data flow.',
buildItFromScratch: 'Build it from scratch',
dataFlow: 'Data Flow',
parseType: 'Parse Type',
manualSetup: 'Manual Setup',
builtIn: 'Built-in',
titleDescription:
'Update your knowledge base configuration here, particularly the chunking method.',
name: 'Knowledge base name',
@ -934,7 +968,7 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
exceptionMethod: 'Exception method',
maxRounds: 'Max reflection rounds',
delayEfterError: 'Delay after error',
maxRetries: 'Max retries',
maxRetries: 'Max reflection rounds',
advancedSettings: 'Advanced Settings',
addTools: 'Add Tools',
sysPromptDefultValue: `
@ -1589,5 +1623,19 @@ This delimiter is used to split the input text into several text pieces echo of
total: 'Total {{total}}',
page: '{{page}} /Page',
},
dataflowParser: {
parseSummary: 'Parse Summary',
parseSummaryTip: 'Parserdeepdoc',
rerunFromCurrentStep: 'Rerun From Current Step',
rerunFromCurrentStepTip: 'Changes detected. Click to re-run.',
},
dataflow: {
parser: 'Parser',
parserDescription: 'Parser',
chunker: 'Chunker',
chunkerDescription: 'Chunker',
tokenizer: 'Tokenizer',
tokenizerDescription: 'Tokenizer',
},
},
};

View File

@ -94,6 +94,24 @@ export default {
noMoreData: '没有更多数据了',
},
knowledgeDetails: {
generate: '生成',
raptor: 'Raptor',
knowledgeGraph: '知识图谱',
processingType: '处理类型',
dataPipeline: '数据管道',
operations: '操作',
status: '状态',
task: '任务',
startDate: '开始时间',
source: '来源',
fileName: '文件名',
datasetLogs: '数据集日志',
fileLogs: '文件日志',
overview: '概览',
success: '成功',
failed: '失败',
completed: '已完成',
processLog: '处理进度日志',
created: '创建于',
learnMore: '了解更多',
general: '通用',
@ -183,6 +201,7 @@ export default {
chunk: '解析块',
bulk: '批量',
cancel: '取消',
close: '关闭',
rerankModel: 'Rerank模型',
rerankPlaceholder: '请选择',
rerankTip: `非必选项:若不选择 rerank 模型,系统将默认采用关键词相似度与向量余弦相似度相结合的混合查询方式;如果设置了 rerank 模型,则混合查询中的向量相似度部分将被 rerank 打分替代。请注意:采用 rerank 模型会非常耗时。如需选用 rerank 模型,建议使用 SaaS 的 rerank 模型服务;如果你倾向使用本地部署的 rerank 模型,请务必确保你使用 docker-compose-gpu.yml 启动 RAGFlow。`,
@ -227,6 +246,17 @@ export default {
theDocumentBeingParsedCannotBeDeleted: '正在解析的文档不能被删除',
},
knowledgeConfiguration: {
default: '默认',
dataPipeline: '数据流',
linkDataPipeline: '关联数据流',
enableAutoGenerate: '是否启用自动生成',
teamPlaceholder: '请选择团队',
dataFlowPlaceholder: '请选择数据流',
buildItFromScratch: '去Scratch构建',
dataFlow: '数据流',
parseType: '切片方法',
manualSetup: '手动设置',
builtIn: '内置',
titleDescription: '在这里更新您的知识库详细信息,尤其是切片方法。',
name: '知识库名称',
photo: '知识库图片',
@ -892,7 +922,7 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
exceptionMethod: '异常处理方法',
maxRounds: '最大反思轮数',
delayEfterError: '错误后延迟',
maxRetries: '最大重试次数',
maxRetries: '最大反思轮数',
advancedSettings: '高级设置',
addTools: '添加工具',
sysPromptDefultValue: `
@ -1501,5 +1531,19 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
total: '总共 {{total}} 条',
page: '{{page}}条/页',
},
dataflowParser: {
parseSummary: '解析摘要',
parseSummaryTip: '解析器: deepdoc',
rerunFromCurrentStep: '从当前步骤重新运行',
rerunFromCurrentStepTip: '已修改,点击重新运行。',
},
dataflow: {
parser: '解析器',
parserDescription: '解析器',
chunker: '分块器',
chunkerDescription: '分块器',
tokenizer: '分词器',
tokenizerDescription: '分词器',
},
},
};

View File

@ -40,6 +40,7 @@ import { useCacheChatLog } from '../hooks/use-cache-chat-log';
import { useMoveNote } from '../hooks/use-move-note';
import { useDropdownManager } from './context';
import Spotlight from '@/components/spotlight';
import {
useHideFormSheetOnNodeDeletion,
useShowDrawer,
@ -309,6 +310,7 @@ function AgentCanvas({ drawerVisible, hideDrawer }: IProps) {
onBeforeDelete={handleBeforeDelete}
>
<AgentBackground></AgentBackground>
<Spotlight className="z-0" opcity={0.7} coverage={70} />
<Controls position={'bottom-center'} orientation="horizontal">
<ControlButton>
<Tooltip>

View File

@ -57,13 +57,6 @@ const FormSchema = z.object({
// )
// .optional(),
message_history_window_size: z.coerce.number(),
tools: z
.array(
z.object({
component_name: z.string(),
}),
)
.optional(),
...LlmSettingSchema,
max_retries: z.coerce.number(),
delay_after_error: z.coerce.number().optional(),

View File

@ -1,15 +1,21 @@
import { useFetchModelId } from '@/hooks/logic-hooks';
import { RAGFlowNodeType } from '@/interfaces/database/flow';
import { get, isEmpty } from 'lodash';
import { get, isEmpty, omit } from 'lodash';
import { useMemo } from 'react';
import { initialAgentValues } from '../../constant';
// You need to exclude the mcp and tools fields that are not in the form,
// otherwise the form data update will reset the tools or mcp data to an array
function omitToolsAndMcp(values: Record<string, any>) {
return omit(values, ['mcp', 'tools']);
}
export function useValues(node?: RAGFlowNodeType) {
const llmId = useFetchModelId();
const defaultValues = useMemo(
() => ({
...initialAgentValues,
...omitToolsAndMcp(initialAgentValues),
llm_id: llmId,
prompts: '',
}),
@ -24,7 +30,7 @@ export function useValues(node?: RAGFlowNodeType) {
}
return {
...formData,
...omitToolsAndMcp(formData),
prompts: get(formData, 'prompts.0.content', ''),
};
}, [defaultValues, node?.data?.form]);

View File

@ -1,71 +1,17 @@
import { useToast } from '@/components/hooks/use-toast';
import { FileMimeType, Platform } from '@/constants/common';
import { useSetModalState } from '@/hooks/common-hooks';
import { useFetchAgent } from '@/hooks/use-agent-request';
import { IGraph } from '@/interfaces/database/flow';
import { downloadJsonFile } from '@/utils/file-util';
import { message } from 'antd';
import isEmpty from 'lodash/isEmpty';
import { useCallback } from 'react';
import { useTranslation } from 'react-i18next';
import { useBuildDslData } from './use-build-dsl';
import { useSetGraphInfo } from './use-set-graph';
export const useHandleExportOrImportJsonFile = () => {
export const useHandleExportJsonFile = () => {
const { buildDslData } = useBuildDslData();
const {
visible: fileUploadVisible,
hideModal: hideFileUploadModal,
showModal: showFileUploadModal,
} = useSetModalState();
const setGraphInfo = useSetGraphInfo();
const { data } = useFetchAgent();
const { t } = useTranslation();
const { toast } = useToast();
const onFileUploadOk = useCallback(
async ({
fileList,
platform,
}: {
fileList: File[];
platform: Platform;
}) => {
console.log('🚀 ~ useHandleExportOrImportJsonFile ~ platform:', platform);
if (fileList.length > 0) {
const file = fileList[0];
if (file.type !== FileMimeType.Json) {
toast({ title: t('flow.jsonUploadTypeErrorMessage') });
return;
}
const graphStr = await file.text();
const errorMessage = t('flow.jsonUploadContentErrorMessage');
try {
const graph = JSON.parse(graphStr);
if (graphStr && !isEmpty(graph) && Array.isArray(graph?.nodes)) {
setGraphInfo(graph ?? ({} as IGraph));
hideFileUploadModal();
} else {
message.error(errorMessage);
}
} catch (error) {
message.error(errorMessage);
}
}
},
[hideFileUploadModal, setGraphInfo, t, toast],
);
const handleExportJson = useCallback(() => {
downloadJsonFile(buildDslData().graph, `${data.title}.json`);
}, [buildDslData, data.title]);
return {
fileUploadVisible,
handleExportJson,
handleImportJson: showFileUploadModal,
hideFileUploadModal,
onFileUploadOk,
};
};

View File

@ -24,7 +24,6 @@ import { ReactFlowProvider } from '@xyflow/react';
import {
ChevronDown,
CirclePlay,
Download,
History,
LaptopMinimalCheck,
Logs,
@ -37,7 +36,7 @@ import { useTranslation } from 'react-i18next';
import { useParams } from 'umi';
import AgentCanvas from './canvas';
import { DropdownProvider } from './canvas/context';
import { useHandleExportOrImportJsonFile } from './hooks/use-export-json';
import { useHandleExportJsonFile } from './hooks/use-export-json';
import { useFetchDataOnMount } from './hooks/use-fetch-data';
import { useGetBeginNodeDataInputs } from './hooks/use-get-begin-query';
import {
@ -46,7 +45,6 @@ import {
useWatchAgentChange,
} from './hooks/use-save-graph';
import { SettingDialog } from './setting-dialog';
import { UploadAgentDialog } from './upload-agent-dialog';
import { useAgentHistoryManager } from './use-agent-history-manager';
import { VersionDialog } from './version-dialog';
@ -71,13 +69,8 @@ export default function Agent() {
} = useSetModalState();
const { t } = useTranslation();
useAgentHistoryManager();
const {
handleExportJson,
handleImportJson,
fileUploadVisible,
onFileUploadOk,
hideFileUploadModal,
} = useHandleExportOrImportJsonFile();
const { handleExportJson } = useHandleExportJsonFile();
const { saveGraph, loading } = useSaveGraph();
const { flowDetail: agentDetail } = useFetchDataOnMount();
const inputs = useGetBeginNodeDataInputs();
@ -158,11 +151,6 @@ export default function Agent() {
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<AgentDropdownMenuItem onClick={handleImportJson}>
<Download />
{t('flow.import')}
</AgentDropdownMenuItem>
<DropdownMenuSeparator />
<AgentDropdownMenuItem onClick={handleExportJson}>
<Upload />
{t('flow.export')}
@ -193,12 +181,6 @@ export default function Agent() {
></AgentCanvas>
</DropdownProvider>
</ReactFlowProvider>
{fileUploadVisible && (
<UploadAgentDialog
hideModal={hideFileUploadModal}
onOk={onFileUploadOk}
></UploadAgentDialog>
)}
{embedVisible && (
<EmbedDialog
visible={embedVisible}

View File

@ -27,9 +27,11 @@ export default function AgentTemplates() {
const [selectMenuItem, setSelectMenuItem] = useState<string>(
MenuItemKey.Recommended,
);
useEffect(() => {
setTemplateList(list);
}, [list]);
const {
visible: creatingVisible,
hideModal: hideCreatingModal,
@ -110,10 +112,9 @@ export default function AgentTemplates() {
<main className="flex-1 bg-text-title-invert/50 h-dvh">
<div className="grid gap-6 sm:grid-cols-1 md:grid-cols-2 lg:grid-cols-3 xl:grid-cols-4 2xl:grid-cols-5 max-h-[94vh] overflow-auto px-8 pt-8">
{tempListFilter?.map((x, index) => {
{tempListFilter?.map((x) => {
return (
<TemplateCard
isCreate={index === 0}
key={x.id}
data={x}
showModal={showModal}

View File

@ -0,0 +1,4 @@
export enum FlowType {
Agent = 'agent',
Flow = 'flow',
}

View File

@ -6,16 +6,18 @@ import {
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { IModalProps } from '@/interfaces/common';
import { TagRenameId } from '@/pages/add-knowledge/constant';
import { useTranslation } from 'react-i18next';
import { CreateAgentForm } from './create-agent-form';
import { CreateAgentForm, CreateAgentFormProps } from './create-agent-form';
type CreateAgentDialogProps = CreateAgentFormProps;
export function CreateAgentDialog({
hideModal,
onOk,
loading,
}: IModalProps<any>) {
shouldChooseAgent,
}: CreateAgentDialogProps) {
const { t } = useTranslation();
return (
@ -24,7 +26,11 @@ export function CreateAgentDialog({
<DialogHeader>
<DialogTitle>{t('flow.createGraph')}</DialogTitle>
</DialogHeader>
<CreateAgentForm hideModal={hideModal} onOk={onOk}></CreateAgentForm>
<CreateAgentForm
hideModal={hideModal}
onOk={onOk}
shouldChooseAgent={shouldChooseAgent}
></CreateAgentForm>
<DialogFooter>
<ButtonLoading type="submit" form={TagRenameId} loading={loading}>
{t('common.save')}

View File

@ -4,38 +4,94 @@ import { zodResolver } from '@hookform/resolvers/zod';
import { useForm } from 'react-hook-form';
import { z } from 'zod';
import {
Form,
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import { RAGFlowFormItem } from '@/components/ragflow-form';
import { Card, CardContent } from '@/components/ui/card';
import { Form } from '@/components/ui/form';
import { IModalProps } from '@/interfaces/common';
import { cn } from '@/lib/utils';
import { TagRenameId } from '@/pages/add-knowledge/constant';
import { BrainCircuit, Check, Route } from 'lucide-react';
import { useCallback } from 'react';
import { useTranslation } from 'react-i18next';
import { FlowType } from './constant';
import { NameFormField, NameFormSchema } from './name-form-field';
export function CreateAgentForm({ hideModal, onOk }: IModalProps<any>) {
export type CreateAgentFormProps = IModalProps<any> & {
shouldChooseAgent?: boolean;
};
type FlowTypeCardProps = {
value?: FlowType;
onChange?: (value: FlowType) => void;
};
function FlowTypeCards({ value, onChange }: FlowTypeCardProps) {
const handleChange = useCallback(
(value: FlowType) => () => {
onChange?.(value);
},
[onChange],
);
return (
<section className="flex gap-10">
{Object.values(FlowType).map((val) => {
const isActive = value === val;
return (
<Card
key={val}
className={cn('flex-1 rounded-lg border bg-transparent', {
'border-text-primary': isActive,
'border-border-default': !isActive,
})}
>
<CardContent
onClick={handleChange(val)}
className={cn(
'cursor-pointer p-5 text-text-secondary flex justify-between items-center',
{
'text-text-primary': isActive,
},
)}
>
<div className="flex gap-2">
{val === FlowType.Agent ? (
<BrainCircuit className="size-6" />
) : (
<Route className="size-6" />
)}
<p>{val}</p>
</div>
{isActive && <Check />}
</CardContent>
</Card>
);
})}
</section>
);
}
export const FormSchema = z.object({
...NameFormSchema,
tag: z.string().trim().optional(),
description: z.string().trim().optional(),
type: z.nativeEnum(FlowType).optional(),
});
export type FormSchemaType = z.infer<typeof FormSchema>;
export function CreateAgentForm({
hideModal,
onOk,
shouldChooseAgent = false,
}: CreateAgentFormProps) {
const { t } = useTranslation();
const FormSchema = z.object({
name: z
.string()
.min(1, {
message: t('common.namePlaceholder'),
})
.trim(),
tag: z.string().trim().optional(),
description: z.string().trim().optional(),
});
const form = useForm<z.infer<typeof FormSchema>>({
const form = useForm<FormSchemaType>({
resolver: zodResolver(FormSchema),
defaultValues: { name: '' },
defaultValues: { name: '', type: FlowType.Agent },
});
async function onSubmit(data: z.infer<typeof FormSchema>) {
async function onSubmit(data: FormSchemaType) {
const ret = await onOk?.(data);
if (ret) {
hideModal?.();
@ -49,57 +105,12 @@ export function CreateAgentForm({ hideModal, onOk }: IModalProps<any>) {
className="space-y-6"
id={TagRenameId}
>
<FormField
control={form.control}
name="name"
render={({ field }) => (
<FormItem>
<FormLabel>{t('common.name')}</FormLabel>
<FormControl>
<Input
placeholder={t('common.namePlaceholder')}
{...field}
autoComplete="off"
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
{/* <FormField
control={form.control}
name="tag"
render={({ field }) => (
<FormItem>
<FormLabel>{t('flow.tag')}</FormLabel>
<FormControl>
<Input
placeholder={t('flow.tagPlaceholder')}
{...field}
autoComplete="off"
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="description"
render={({ field }) => (
<FormItem>
<FormLabel>{t('flow.description')}</FormLabel>
<FormControl>
<Input
placeholder={t('flow.descriptionPlaceholder')}
{...field}
autoComplete="off"
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/> */}
{shouldChooseAgent && (
<RAGFlowFormItem required name="type" label={t('common.type')}>
<FlowTypeCards></FlowTypeCards>
</RAGFlowFormItem>
)}
<NameFormField></NameFormField>
</form>
</Form>
);

View File

@ -0,0 +1,42 @@
import { useSetModalState } from '@/hooks/common-hooks';
import { EmptyDsl, useSetAgent } from '@/hooks/use-agent-request';
import { DSL } from '@/interfaces/database/agent';
import { useCallback } from 'react';
import { FlowType } from '../constant';
import { FormSchemaType } from '../create-agent-form';
export function useCreateAgentOrPipeline() {
const { loading, setAgent } = useSetAgent();
const {
visible: creatingVisible,
hideModal: hideCreatingModal,
showModal: showCreatingModal,
} = useSetModalState();
const createAgent = useCallback(
async (name: string) => {
return setAgent({ title: name, dsl: EmptyDsl as DSL });
},
[setAgent],
);
const handleCreateAgentOrPipeline = useCallback(
async (data: FormSchemaType) => {
if (data.type === FlowType.Agent) {
const ret = await createAgent(data.name);
if (ret.code === 0) {
hideCreatingModal();
}
}
},
[createAgent, hideCreatingModal],
);
return {
loading,
creatingVisible,
hideCreatingModal,
showCreatingModal,
handleCreateAgentOrPipeline,
};
}

View File

@ -1,14 +1,24 @@
import ListFilterBar from '@/components/list-filter-bar';
import { RenameDialog } from '@/components/rename-dialog';
import { Button } from '@/components/ui/button';
import {
DropdownMenu,
DropdownMenuContent,
DropdownMenuItem,
DropdownMenuTrigger,
} from '@/components/ui/dropdown-menu';
import { RAGFlowPagination } from '@/components/ui/ragflow-pagination';
import { useNavigatePage } from '@/hooks/logic-hooks/navigate-hooks';
import { useFetchAgentListByPage } from '@/hooks/use-agent-request';
import { t } from 'i18next';
import { pick } from 'lodash';
import { Plus } from 'lucide-react';
import { Clipboard, ClipboardPlus, FileInput, Plus } from 'lucide-react';
import { useCallback } from 'react';
import { AgentCard } from './agent-card';
import { CreateAgentDialog } from './create-agent-dialog';
import { useCreateAgentOrPipeline } from './hooks/use-create-agent';
import { UploadAgentDialog } from './upload-agent-dialog';
import { useHandleImportJsonFile } from './use-import-json';
import { useRenameAgent } from './use-rename-agent';
export default function Agents() {
@ -25,6 +35,21 @@ export default function Agents() {
showAgentRenameModal,
} = useRenameAgent();
const {
creatingVisible,
hideCreatingModal,
showCreatingModal,
loading,
handleCreateAgentOrPipeline,
} = useCreateAgentOrPipeline();
const {
handleImportJson,
fileUploadVisible,
onFileUploadOk,
hideFileUploadModal,
} = useHandleImportJsonFile();
const handlePageChange = useCallback(
(page: number, pageSize?: number) => {
setPagination({ page, pageSize });
@ -41,10 +66,37 @@ export default function Agents() {
onSearchChange={handleInputChange}
icon="agent"
>
<Button onClick={navigateToAgentTemplates}>
<Plus className="mr-2 h-4 w-4" />
{t('flow.createGraph')}
</Button>
<DropdownMenu>
<DropdownMenuTrigger>
<Button>
<Plus className="mr-2 h-4 w-4" />
{t('flow.createGraph')}
</Button>
</DropdownMenuTrigger>
<DropdownMenuContent>
<DropdownMenuItem
justifyBetween={false}
onClick={showCreatingModal}
>
<Clipboard />
Create from Blank
</DropdownMenuItem>
<DropdownMenuItem
justifyBetween={false}
onClick={navigateToAgentTemplates}
>
<ClipboardPlus />
Create from Template
</DropdownMenuItem>
<DropdownMenuItem
justifyBetween={false}
onClick={handleImportJson}
>
<FileInput />
Import json file
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
</ListFilterBar>
</div>
<div className="flex-1 overflow-auto">
@ -75,6 +127,21 @@ export default function Agents() {
loading={agentRenameLoading}
></RenameDialog>
)}
{creatingVisible && (
<CreateAgentDialog
loading={loading}
visible={creatingVisible}
hideModal={hideCreatingModal}
shouldChooseAgent
onOk={handleCreateAgentOrPipeline}
></CreateAgentDialog>
)}
{fileUploadVisible && (
<UploadAgentDialog
hideModal={hideFileUploadModal}
onOk={onFileUploadOk}
></UploadAgentDialog>
)}
</section>
);
}

View File

@ -0,0 +1,28 @@
import { RAGFlowFormItem } from '@/components/ragflow-form';
import { Input } from '@/components/ui/input';
import i18n from '@/locales/config';
import { useTranslation } from 'react-i18next';
import { z } from 'zod';
export const NameFormSchema = {
name: z
.string()
.min(1, {
message: i18n.t('common.namePlaceholder'),
})
.trim(),
};
export function NameFormField() {
const { t } = useTranslation();
return (
<RAGFlowFormItem
name="name"
required
label={t('common.name')}
tooltip={t('flow.sqlStatementTip')}
>
<Input placeholder={t('common.namePlaceholder')} autoComplete="off" />
</RAGFlowFormItem>
);
}

View File

@ -3,7 +3,6 @@ import { Button } from '@/components/ui/button';
import { Card, CardContent } from '@/components/ui/card';
import { IFlowTemplate } from '@/interfaces/database/flow';
import i18n from '@/locales/config';
import { Plus } from 'lucide-react';
import { useCallback, useMemo } from 'react';
import { useTranslation } from 'react-i18next';
interface IProps {
@ -12,7 +11,7 @@ interface IProps {
showModal(record: IFlowTemplate): void;
}
export function TemplateCard({ data, showModal, isCreate = false }: IProps) {
export function TemplateCard({ data, showModal }: IProps) {
const { t } = useTranslation();
const handleClick = useCallback(() => {
@ -26,41 +25,24 @@ export function TemplateCard({ data, showModal, isCreate = false }: IProps) {
return (
<Card className="border-colors-outline-neutral-standard group relative min-h-40">
<CardContent className="p-4 ">
{isCreate && (
<div
className="flex flex-col justify-center items-center gap-4 mb-4 absolute top-0 right-0 left-0 bottom-0 cursor-pointer "
<div className="flex justify-start items-center gap-4 mb-4">
<RAGFlowAvatar
className="w-7 h-7"
avatar={data.avatar ? data.avatar : 'https://github.com/shadcn.png'}
name={data?.title[language] || 'CN'}
></RAGFlowAvatar>
<div className="text-[18px] font-bold ">{data?.title[language]}</div>
</div>
<p className="break-words">{data?.description[language]}</p>
<div className="group-hover:bg-gradient-to-t from-black/70 from-10% via-black/0 via-50% to-black/0 w-full h-full group-hover:block absolute top-0 left-0 hidden rounded-xl">
<Button
variant="default"
className="w-1/3 absolute bottom-4 right-4 left-4 justify-center text-center m-auto"
onClick={handleClick}
>
<Plus size={50} fontWeight={700} />
<div>{t('flow.createAgent')}</div>
</div>
)}
{!isCreate && (
<>
<div className="flex justify-start items-center gap-4 mb-4">
<RAGFlowAvatar
className="w-7 h-7"
avatar={
data.avatar ? data.avatar : 'https://github.com/shadcn.png'
}
name={data?.title[language] || 'CN'}
></RAGFlowAvatar>
<div className="text-[18px] font-bold ">
{data?.title[language]}
</div>
</div>
<p className="break-words">{data?.description[language]}</p>
<div className="group-hover:bg-gradient-to-t from-black/70 from-10% via-black/0 via-50% to-black/0 w-full h-full group-hover:block absolute top-0 left-0 hidden rounded-xl">
<Button
variant="default"
className="w-1/3 absolute bottom-4 right-4 left-4 justify-center text-center m-auto"
onClick={handleClick}
>
{t('flow.useTemplate')}
</Button>
</div>
</>
)}
{t('flow.useTemplate')}
</Button>
</div>
</CardContent>
</Card>
);

View File

@ -1,3 +1,4 @@
import { ButtonLoading } from '@/components/ui/button';
import {
Dialog,
DialogContent,
@ -5,7 +6,6 @@ import {
DialogHeader,
DialogTitle,
} from '@/components/ui/dialog';
import { LoadingButton } from '@/components/ui/loading-button';
import { IModalProps } from '@/interfaces/common';
import { TagRenameId } from '@/pages/add-knowledge/constant';
import { useTranslation } from 'react-i18next';
@ -26,9 +26,9 @@ export function UploadAgentDialog({
</DialogHeader>
<UploadAgentForm hideModal={hideModal} onOk={onOk}></UploadAgentForm>
<DialogFooter>
<LoadingButton type="submit" form={TagRenameId} loading={loading}>
<ButtonLoading type="submit" form={TagRenameId} loading={loading}>
{t('common.save')}
</LoadingButton>
</ButtonLoading>
</DialogFooter>
</DialogContent>
</Dialog>

View File

@ -13,32 +13,24 @@ import {
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { FileMimeType, Platform } from '@/constants/common';
import { FileMimeType } from '@/constants/common';
import { IModalProps } from '@/interfaces/common';
import { TagRenameId } from '@/pages/add-knowledge/constant';
import { useTranslation } from 'react-i18next';
import { NameFormField, NameFormSchema } from '../name-form-field';
// const options = Object.values(Platform).map((x) => ({ label: x, value: x }));
export const FormSchema = z.object({
fileList: z.array(z.instanceof(File)),
...NameFormSchema,
});
export type FormSchemaType = z.infer<typeof FormSchema>;
export function UploadAgentForm({ hideModal, onOk }: IModalProps<any>) {
const { t } = useTranslation();
const FormSchema = z.object({
platform: z
.string()
.min(1, {
message: t('common.namePlaceholder'),
})
.trim(),
fileList: z.array(z.instanceof(File)),
});
const form = useForm<z.infer<typeof FormSchema>>({
resolver: zodResolver(FormSchema),
defaultValues: { platform: Platform.RAGFlow },
defaultValues: { name: '' },
});
async function onSubmit(data: z.infer<typeof FormSchema>) {
console.log('🚀 ~ onSubmit ~ data:', data);
async function onSubmit(data: FormSchemaType) {
const ret = await onOk?.(data);
if (ret) {
hideModal?.();
@ -52,12 +44,13 @@ export function UploadAgentForm({ hideModal, onOk }: IModalProps<any>) {
className="space-y-6"
id={TagRenameId}
>
<NameFormField></NameFormField>
<FormField
control={form.control}
name="fileList"
render={({ field }) => (
<FormItem>
<FormLabel>{t('common.name')}</FormLabel>
<FormLabel required>DSL</FormLabel>
<FormControl>
<FileUploader
value={field.value}
@ -70,19 +63,6 @@ export function UploadAgentForm({ hideModal, onOk }: IModalProps<any>) {
</FormItem>
)}
/>
{/* <FormField
control={form.control}
name="platform"
render={({ field }) => (
<FormItem>
<FormLabel>{t('common.name')}</FormLabel>
<FormControl>
<RAGFlowSelect {...field} options={options} />
</FormControl>
<FormMessage />
</FormItem>
)}
/> */}
</form>
</Form>
);

View File

@ -0,0 +1,56 @@
import { useToast } from '@/components/hooks/use-toast';
import { FileMimeType } from '@/constants/common';
import { useSetModalState } from '@/hooks/common-hooks';
import { EmptyDsl, useSetAgent } from '@/hooks/use-agent-request';
import { message } from 'antd';
import isEmpty from 'lodash/isEmpty';
import { useCallback } from 'react';
import { useTranslation } from 'react-i18next';
import { FormSchemaType } from './upload-agent-dialog/upload-agent-form';
export const useHandleImportJsonFile = () => {
const {
visible: fileUploadVisible,
hideModal: hideFileUploadModal,
showModal: showFileUploadModal,
} = useSetModalState();
const { t } = useTranslation();
const { toast } = useToast();
const { loading, setAgent } = useSetAgent();
const onFileUploadOk = useCallback(
async ({ fileList, name }: FormSchemaType) => {
if (fileList.length > 0) {
const file = fileList[0];
if (file.type !== FileMimeType.Json) {
toast({ title: t('flow.jsonUploadTypeErrorMessage') });
return;
}
const graphStr = await file.text();
const errorMessage = t('flow.jsonUploadContentErrorMessage');
try {
const graph = JSON.parse(graphStr);
if (graphStr && !isEmpty(graph) && Array.isArray(graph?.nodes)) {
const dsl = { ...EmptyDsl, graph };
setAgent({ title: name, dsl });
hideFileUploadModal();
} else {
message.error(errorMessage);
}
} catch (error) {
message.error(errorMessage);
}
}
},
[hideFileUploadModal, setAgent, t, toast],
);
return {
fileUploadVisible,
handleImportJson: showFileUploadModal,
hideFileUploadModal,
onFileUploadOk,
loading,
};
};

View File

@ -17,9 +17,9 @@ import {
TooltipTrigger,
} from '@/components/ui/tooltip';
import { IModalProps } from '@/interfaces/common';
import { useGetNodeDescription, useGetNodeName } from '@/pages/data-flow/hooks';
import { Position } from '@xyflow/react';
import { t } from 'i18next';
import { lowerFirst } from 'lodash';
import {
PropsWithChildren,
createContext,
@ -28,7 +28,6 @@ import {
useEffect,
useRef,
} from 'react';
import { useTranslation } from 'react-i18next';
import { Operator } from '../../../constant';
import { AgentInstanceContext, HandleContext } from '../../../context';
import OperatorIcon from '../../../operator-icon';
@ -53,7 +52,9 @@ function OperatorItemList({
const handleContext = useContext(HandleContext);
const hideModal = useContext(HideModalContext);
const onNodeCreated = useContext(OnNodeCreatedContext);
const { t } = useTranslation();
const getNodeName = useGetNodeName();
const getNodeDescription = useGetNodeDescription();
const handleClick = (operator: Operator) => {
const contextData = handleContext || {
@ -84,7 +85,7 @@ function OperatorItemList({
const commonContent = (
<div className="hover:bg-background-card py-1 px-3 cursor-pointer rounded-sm flex gap-2 items-center justify-start">
<OperatorIcon name={operator} />
{t(`flow.${lowerFirst(operator)}`)}
{getNodeName(operator)}
</div>
);
@ -101,12 +102,12 @@ function OperatorItemList({
onSelect={() => hideModal?.()}
>
<OperatorIcon name={operator} />
{t(`flow.${lowerFirst(operator)}`)}
{getNodeName(operator)}
</DropdownMenuItem>
)}
</TooltipTrigger>
<TooltipContent side="right">
<p>{t(`flow.${lowerFirst(operator)}Description`)}</p>
<p>{getNodeDescription(operator)}</p>
</TooltipContent>
</Tooltip>
);

View File

@ -523,3 +523,15 @@ export enum AgentExceptionMethod {
Comment = 'comment',
Goto = 'goto',
}
export enum FileType {
PDF = 'pdf',
Spreadsheet = 'spreadsheet',
Image = 'image',
Email = 'email',
TextMarkdown = 'text&markdown',
Docx = 'docx',
PowerPoint = 'ppt',
Video = 'video',
Audio = 'audio',
}

View File

@ -5,13 +5,13 @@ import {
SheetHeader,
SheetTitle,
} from '@/components/ui/sheet';
import { useTranslate } from '@/hooks/common-hooks';
import { IModalProps } from '@/interfaces/common';
import { RAGFlowNodeType } from '@/interfaces/database/flow';
import { cn } from '@/lib/utils';
import { lowerFirst } from 'lodash';
import { Play, X } from 'lucide-react';
import { useMemo } from 'react';
import { useTranslation } from 'react-i18next';
import { BeginId, Operator } from '../constant';
import { AgentFormContext } from '../context';
import { RunTooltip } from '../flow-tooltip';
@ -60,7 +60,7 @@ const FormSheet = ({
);
}, [clickedToolId, operatorName]);
const { t } = useTranslate('flow');
const { t } = useTranslation();
return (
<Sheet open={visible} modal={false}>
@ -80,7 +80,7 @@ const FormSheet = ({
<div className="flex-1">MCP Config</div>
) : (
<div className="flex items-center gap-1 flex-1">
<label htmlFor="">{t('title')}</label>
<label htmlFor="">{t('flow.title')}</label>
{node?.id === BeginId ? (
<span>{t(BeginId)}</span>
) : (
@ -106,7 +106,7 @@ const FormSheet = ({
{isMcp || (
<span>
{t(
`${lowerFirst(operatorName === Operator.Tool ? clickedToolId : operatorName)}Description`,
`dataflow.${lowerFirst(operatorName === Operator.Tool ? clickedToolId : operatorName)}Description`,
)}
</span>
)}

View File

@ -0,0 +1,67 @@
import { LayoutRecognizeFormField } from '@/components/layout-recognize-form-field';
import { LLMFormField } from '@/components/llm-setting-items/llm-form-field';
import {
SelectWithSearch,
SelectWithSearchFlagOptionType,
} from '@/components/originui/select-with-search';
import { RAGFlowFormItem } from '@/components/ragflow-form';
import { buildOptions } from '@/utils/form';
import { FileType } from '../../constant';
import { OutputFormatMap } from './constant';
import { CommonProps } from './interface';
import { buildFieldNameWithPrefix } from './utils';
function buildOutputOptionsFormatMap() {
return Object.entries(OutputFormatMap).reduce<
Record<string, SelectWithSearchFlagOptionType[]>
>((pre, [key, value]) => {
pre[key] = buildOptions(value);
return pre;
}, {});
}
export type OutputFormatFormFieldProps = CommonProps & {
fileType: FileType;
};
export function OutputFormatFormField({
prefix,
fileType,
}: OutputFormatFormFieldProps) {
return (
<RAGFlowFormItem
name={buildFieldNameWithPrefix(`output_format`, prefix)}
label="output_format"
>
<SelectWithSearch
options={buildOutputOptionsFormatMap()[fileType]}
></SelectWithSearch>
</RAGFlowFormItem>
);
}
export function ParserMethodFormField({ prefix }: CommonProps) {
return (
<LayoutRecognizeFormField
name={buildFieldNameWithPrefix(`parse_method`, prefix)}
horizontal={false}
></LayoutRecognizeFormField>
);
return (
<RAGFlowFormItem
name={buildFieldNameWithPrefix(`parse_method`, prefix)}
label="parse_method"
>
<SelectWithSearch options={[]}></SelectWithSearch>
</RAGFlowFormItem>
);
}
export function LargeModelFormField({ prefix }: CommonProps) {
return (
<LLMFormField
name={buildFieldNameWithPrefix('llm_id', prefix)}
></LLMFormField>
);
}

View File

@ -0,0 +1,52 @@
import { FileType } from '../../constant';
export enum PdfOutputFormat {
Json = 'json',
Markdown = 'markdown',
}
export enum SpreadsheetOutputFormat {
Json = 'json',
Html = 'html',
}
export enum ImageOutputFormat {
Text = 'text',
}
export enum EmailOutputFormat {
Json = 'json',
Text = 'text',
}
export enum TextMarkdownOutputFormat {
Text = 'text',
}
export enum DocxOutputFormat {
Markdown = 'markdown',
}
export enum PptOutputFormat {
Json = 'json',
}
export enum VideoOutputFormat {
Json = 'json',
}
export enum AudioOutputFormat {
Text = 'text',
}
export const OutputFormatMap = {
[FileType.PDF]: PdfOutputFormat,
[FileType.Spreadsheet]: SpreadsheetOutputFormat,
[FileType.Image]: ImageOutputFormat,
[FileType.Email]: EmailOutputFormat,
[FileType.TextMarkdown]: TextMarkdownOutputFormat,
[FileType.Docx]: DocxOutputFormat,
[FileType.PowerPoint]: PptOutputFormat,
[FileType.Video]: VideoOutputFormat,
[FileType.Audio]: AudioOutputFormat,
};

View File

@ -0,0 +1,35 @@
import { SelectWithSearch } from '@/components/originui/select-with-search';
import { RAGFlowFormItem } from '@/components/ragflow-form';
import { buildOptions } from '@/utils/form';
import { FileType } from '../../constant';
import { OutputFormatFormField } from './common-form-fields';
import { CommonProps } from './interface';
import { buildFieldNameWithPrefix } from './utils';
const options = buildOptions([
'from',
'to',
'cc',
'bcc',
'date',
'subject',
'body',
'attachments',
]);
export function EmailFormFields({ prefix }: CommonProps) {
return (
<>
<RAGFlowFormItem
name={buildFieldNameWithPrefix(`fields`, prefix)}
label="fields"
>
<SelectWithSearch options={options}></SelectWithSearch>
</RAGFlowFormItem>
<OutputFormatFormField
prefix={prefix}
fileType={FileType.Email}
></OutputFormatFormField>
</>
);
}

View File

@ -0,0 +1,21 @@
import { FileType } from '../../constant';
import {
LargeModelFormField,
OutputFormatFormField,
ParserMethodFormField,
} from './common-form-fields';
import { CommonProps } from './interface';
export function ImageFormFields({ prefix }: CommonProps) {
return (
<>
<ParserMethodFormField prefix={prefix}></ParserMethodFormField>
{/* Multimodal Model */}
<LargeModelFormField prefix={prefix}></LargeModelFormField>
<OutputFormatFormField
prefix={prefix}
fileType={FileType.Image}
></OutputFormatFormField>
</>
);
}

View File

@ -1,135 +1,140 @@
import { FormContainer } from '@/components/form-container';
import NumberInput from '@/components/originui/number-input';
import { SelectWithSearch } from '@/components/originui/select-with-search';
import {
Form,
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { useTranslate } from '@/hooks/common-hooks';
import { RAGFlowFormItem } from '@/components/ragflow-form';
import { BlockButton, Button } from '@/components/ui/button';
import { Form } from '@/components/ui/form';
import { Separator } from '@/components/ui/separator';
import { cn } from '@/lib/utils';
import { buildOptions } from '@/utils/form';
import { zodResolver } from '@hookform/resolvers/zod';
import { memo } from 'react';
import { useForm, useFormContext } from 'react-hook-form';
import { useHover } from 'ahooks';
import { Trash2 } from 'lucide-react';
import { memo, useCallback, useRef } from 'react';
import {
UseFieldArrayRemove,
useFieldArray,
useForm,
useFormContext,
} from 'react-hook-form';
import { z } from 'zod';
import { initialParserValues } from '../../constant';
import { FileType, initialParserValues } from '../../constant';
import { useFormValues } from '../../hooks/use-form-values';
import { useWatchFormChange } from '../../hooks/use-watch-form-change';
import { INextOperatorForm } from '../../interface';
import { GoogleCountryOptions, GoogleLanguageOptions } from '../../options';
import { buildOutputList } from '../../utils/build-output-list';
import { ApiKeyField } from '../components/api-key-field';
import { FormWrapper } from '../components/form-wrapper';
import { Output } from '../components/output';
import { QueryVariable } from '../components/query-variable';
import { OutputFormatFormField } from './common-form-fields';
import { EmailFormFields } from './email-form-fields';
import { ImageFormFields } from './image-form-fields';
import { PdfFormFields } from './pdf-form-fields';
import { buildFieldNameWithPrefix } from './utils';
import { VideoFormFields } from './video-form-fields';
const outputList = buildOutputList(initialParserValues.outputs);
export const GoogleFormPartialSchema = {
api_key: z.string(),
country: z.string(),
language: z.string(),
const FileFormatOptions = buildOptions(FileType);
const FileFormatWidgetMap = {
[FileType.PDF]: PdfFormFields,
[FileType.Video]: VideoFormFields,
[FileType.Audio]: VideoFormFields,
[FileType.Email]: EmailFormFields,
[FileType.Image]: ImageFormFields,
};
export const FormSchema = z.object({
...GoogleFormPartialSchema,
q: z.string(),
start: z.number(),
num: z.number(),
});
type ParserItemProps = {
name: string;
index: number;
fieldLength: number;
remove: UseFieldArrayRemove;
};
export function GoogleFormWidgets() {
function ParserItem({ name, index, fieldLength, remove }: ParserItemProps) {
const form = useFormContext();
const { t } = useTranslate('flow');
const ref = useRef(null);
const isHovering = useHover(ref);
const prefix = `${name}.${index}`;
const fileFormat = form.getValues(`${name}.${index}.fileFormat`);
const Widget =
fileFormat && fileFormat in FileFormatWidgetMap
? FileFormatWidgetMap[fileFormat as keyof typeof FileFormatWidgetMap]
: OutputFormatFormField;
return (
<>
<FormField
control={form.control}
name={`country`}
render={({ field }) => (
<FormItem className="flex-1">
<FormLabel>{t('country')}</FormLabel>
<FormControl>
<SelectWithSearch
{...field}
options={GoogleCountryOptions}
></SelectWithSearch>
</FormControl>
<FormMessage />
</FormItem>
<section
className={cn('space-y-5 p-5 rounded-md', {
'bg-state-error-5': isHovering,
})}
>
<div className="flex justify-between items-center">
<span className="text-text-primary text-sm">Parser {index}</span>
{index > 0 && (
<Button variant={'ghost'} onClick={() => remove(index)} ref={ref}>
<Trash2 />
</Button>
)}
/>
<FormField
control={form.control}
name={`language`}
render={({ field }) => (
<FormItem className="flex-1">
<FormLabel>{t('language')}</FormLabel>
<FormControl>
<SelectWithSearch
{...field}
options={GoogleLanguageOptions}
></SelectWithSearch>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</>
</div>
<RAGFlowFormItem
name={buildFieldNameWithPrefix(`fileFormat`, prefix)}
label="File Formats"
>
<SelectWithSearch options={FileFormatOptions}></SelectWithSearch>
</RAGFlowFormItem>
<Widget prefix={prefix} fileType={fileFormat as FileType}></Widget>
{index < fieldLength - 1 && <Separator />}
</section>
);
}
export const FormSchema = z.object({
parser: z.array(
z.object({
fileFormat: z.string().optional(),
}),
),
});
const ParserForm = ({ node }: INextOperatorForm) => {
const { t } = useTranslate('flow');
const defaultValues = useFormValues(initialParserValues, node);
const form = useForm<z.infer<typeof FormSchema>>({
defaultValues,
resolver: zodResolver(FormSchema),
shouldUnregister: true,
});
const name = 'parser';
const { fields, remove, append } = useFieldArray({
name,
control: form.control,
});
const add = useCallback(() => {
append({
fileFormat: undefined,
});
}, [append]);
useWatchFormChange(node?.id, form);
return (
<Form {...form}>
<FormWrapper>
<FormContainer>
<QueryVariable name="q"></QueryVariable>
</FormContainer>
<FormContainer>
<ApiKeyField placeholder={t('apiKeyPlaceholder')}></ApiKeyField>
<FormField
control={form.control}
name={`start`}
render={({ field }) => (
<FormItem>
<FormLabel>{t('flowStart')}</FormLabel>
<FormControl>
<NumberInput {...field} className="w-full"></NumberInput>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name={`num`}
render={({ field }) => (
<FormItem>
<FormLabel>{t('flowNum')}</FormLabel>
<FormControl>
<NumberInput {...field} className="w-full"></NumberInput>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<GoogleFormWidgets></GoogleFormWidgets>
</FormContainer>
</FormWrapper>
<form className="px-5">
{fields.map((field, index) => {
return (
<ParserItem
key={field.id}
name={name}
index={index}
fieldLength={fields.length}
remove={remove}
></ParserItem>
);
})}
<BlockButton onClick={add} type="button">
Add Parser
</BlockButton>
</form>
<div className="p-5">
<Output list={outputList}></Output>
</div>

View File

@ -0,0 +1,3 @@
export type CommonProps = {
prefix: string;
};

View File

@ -0,0 +1,27 @@
import { CrossLanguageFormField } from '@/components/cross-language-form-field';
import { FileType } from '../../constant';
import {
LargeModelFormField,
OutputFormatFormField,
ParserMethodFormField,
} from './common-form-fields';
import { CommonProps } from './interface';
import { buildFieldNameWithPrefix } from './utils';
export function PdfFormFields({ prefix }: CommonProps) {
return (
<>
<ParserMethodFormField prefix={prefix}></ParserMethodFormField>
{/* Multimodal Model */}
<LargeModelFormField prefix={prefix}></LargeModelFormField>
<CrossLanguageFormField
name={buildFieldNameWithPrefix(`lang`, prefix)}
label="lang"
></CrossLanguageFormField>
<OutputFormatFormField
prefix={prefix}
fileType={FileType.Image}
></OutputFormatFormField>
</>
);
}

View File

@ -0,0 +1,3 @@
export function buildFieldNameWithPrefix(name: string, prefix: string) {
return `${prefix}.${name}`;
}

View File

@ -0,0 +1,21 @@
import {
LargeModelFormField,
OutputFormatFormField,
OutputFormatFormFieldProps,
} from './common-form-fields';
export function VideoFormFields({
prefix,
fileType,
}: OutputFormatFormFieldProps) {
return (
<>
{/* Multimodal Model */}
<LargeModelFormField prefix={prefix}></LargeModelFormField>
<OutputFormatFormField
prefix={prefix}
fileType={fileType}
></OutputFormatFormField>
</>
);
}

View File

@ -30,7 +30,16 @@ export const useGetNodeName = () => {
const { t } = useTranslation();
return (type: string) => {
const name = t(`flow.${lowerFirst(type)}`);
const name = t(`dataflow.${lowerFirst(type)}`);
return name;
};
};
export const useGetNodeDescription = () => {
const { t } = useTranslation();
return (type: string) => {
const name = t(`dataflow.${lowerFirst(type)}Description`);
return name;
};
};

View File

@ -108,7 +108,7 @@ export const useGetNodeName = () => {
const { t } = useTranslation();
return (type: string) => {
const name = t(`flow.${lowerFirst(type)}`);
const name = t(`dataflow.${lowerFirst(type)}`);
return name;
};
};

View File

@ -0,0 +1,234 @@
import message from '@/components/ui/message';
import {
RAGFlowPagination,
RAGFlowPaginationType,
} from '@/components/ui/ragflow-pagination';
import { Spin } from '@/components/ui/spin';
import {
useFetchNextChunkList,
useSwitchChunk,
} from '@/hooks/use-chunk-request';
import classNames from 'classnames';
import { useCallback, useEffect, useState } from 'react';
import { useTranslation } from 'react-i18next';
import ChunkCard from './components/chunk-card';
import CreatingModal from './components/chunk-creating-modal';
import ChunkResultBar from './components/chunk-result-bar';
import CheckboxSets from './components/chunk-result-bar/checkbox-sets';
import RerunButton from './components/rerun-button';
import {
useChangeChunkTextMode,
useDeleteChunkByIds,
useHandleChunkCardClick,
useUpdateChunk,
} from './hooks';
import styles from './index.less';
const ChunkerContainer = () => {
const [selectedChunkIds, setSelectedChunkIds] = useState<string[]>([]);
const [isChange, setIsChange] = useState(false);
const { t } = useTranslation();
const {
data: { documentInfo, data = [], total },
pagination,
loading,
searchString,
handleInputChange,
available,
handleSetAvailable,
} = useFetchNextChunkList();
const { handleChunkCardClick, selectedChunkId } = useHandleChunkCardClick();
const isPdf = documentInfo?.type === 'pdf';
const {
chunkUpdatingLoading,
onChunkUpdatingOk,
showChunkUpdatingModal,
hideChunkUpdatingModal,
chunkId,
chunkUpdatingVisible,
documentId,
} = useUpdateChunk();
const { removeChunk } = useDeleteChunkByIds();
const { changeChunkTextMode, textMode } = useChangeChunkTextMode();
const selectAllChunk = useCallback(
(checked: boolean) => {
setSelectedChunkIds(checked ? data.map((x) => x.chunk_id) : []);
},
[data],
);
const showSelectedChunkWarning = useCallback(() => {
message.warning(t('message.pleaseSelectChunk'));
}, [t]);
const { switchChunk } = useSwitchChunk();
const [chunkList, setChunkList] = useState(data);
useEffect(() => {
setChunkList(data);
}, [data]);
const onPaginationChange: RAGFlowPaginationType['onChange'] = (
page,
size,
) => {
setSelectedChunkIds([]);
pagination.onChange?.(page, size);
};
const handleSwitchChunk = useCallback(
async (available?: number, chunkIds?: string[]) => {
let ids = chunkIds;
if (!chunkIds) {
ids = selectedChunkIds;
if (selectedChunkIds.length === 0) {
showSelectedChunkWarning();
return;
}
}
const resCode: number = await switchChunk({
chunk_ids: ids,
available_int: available,
doc_id: documentId,
});
if (ids?.length && resCode === 0) {
chunkList.forEach((x: any) => {
if (ids.indexOf(x['chunk_id']) > -1) {
x['available_int'] = available;
}
});
setChunkList(chunkList);
}
},
[
switchChunk,
documentId,
selectedChunkIds,
showSelectedChunkWarning,
chunkList,
],
);
const handleSingleCheckboxClick = useCallback(
(chunkId: string, checked: boolean) => {
setSelectedChunkIds((previousIds) => {
const idx = previousIds.findIndex((x) => x === chunkId);
const nextIds = [...previousIds];
if (checked && idx === -1) {
nextIds.push(chunkId);
} else if (!checked && idx !== -1) {
nextIds.splice(idx, 1);
}
return nextIds;
});
},
[],
);
const handleRemoveChunk = useCallback(async () => {
if (selectedChunkIds.length > 0) {
const resCode: number = await removeChunk(selectedChunkIds, documentId);
if (resCode === 0) {
setSelectedChunkIds([]);
}
} else {
showSelectedChunkWarning();
}
}, [selectedChunkIds, documentId, removeChunk, showSelectedChunkWarning]);
const handleChunkEditSave = (e: any) => {
setIsChange(true);
onChunkUpdatingOk(e);
};
return (
<>
{isChange && (
<div className=" absolute top-2 right-6">
<RerunButton />
</div>
)}
<div
className={classNames(
{ [styles.pagePdfWrapper]: isPdf },
'flex flex-col w-3/5',
)}
>
<Spin spinning={loading} className={styles.spin} size="large">
<div className="h-[50px] flex flex-row justify-between items-end pb-[5px]">
<div>
<h2 className="text-[16px]">{t('chunk.chunkResult')}</h2>
<div className="text-[12px] text-text-secondary italic">
{t('chunk.chunkResultTip')}
</div>
</div>
<ChunkResultBar
handleInputChange={handleInputChange}
searchString={searchString}
changeChunkTextMode={changeChunkTextMode}
createChunk={showChunkUpdatingModal}
available={available}
selectAllChunk={selectAllChunk}
handleSetAvailable={handleSetAvailable}
/>
</div>
<div className=" rounded-[16px] box-border mb-2">
<div className="pt-[5px] pb-[5px]">
<CheckboxSets
selectAllChunk={selectAllChunk}
switchChunk={handleSwitchChunk}
removeChunk={handleRemoveChunk}
checked={selectedChunkIds.length === data.length}
selectedChunkIds={selectedChunkIds}
/>
</div>
<div className="h-[calc(100vh-280px)] overflow-y-auto pr-2 scrollbar-thin">
<div
className={classNames(
styles.chunkContainer,
{
[styles.chunkOtherContainer]: !isPdf,
},
'flex flex-col gap-4',
)}
>
{chunkList.map((item) => (
<ChunkCard
item={item}
key={item.chunk_id}
editChunk={showChunkUpdatingModal}
checked={selectedChunkIds.some((x) => x === item.chunk_id)}
handleCheckboxClick={handleSingleCheckboxClick}
switchChunk={handleSwitchChunk}
clickChunkCard={handleChunkCardClick}
selected={item.chunk_id === selectedChunkId}
textMode={textMode}
></ChunkCard>
))}
</div>
</div>
<div className={styles.pageFooter}>
<RAGFlowPagination
pageSize={pagination.pageSize}
current={pagination.current}
total={total}
onChange={(page, pageSize) => {
onPaginationChange(page, pageSize);
}}
></RAGFlowPagination>
</div>
</div>
</Spin>
</div>
{chunkUpdatingVisible && (
<CreatingModal
doc_id={documentId}
chunkId={chunkId}
hideModal={hideChunkUpdatingModal}
visible={chunkUpdatingVisible}
loading={chunkUpdatingLoading}
onOk={(e) => {
handleChunkEditSave(e);
}}
parserId={documentInfo.parser_id}
/>
)}
</>
);
};
export { ChunkerContainer };

View File

@ -0,0 +1,36 @@
.image {
width: 100px !important;
object-fit: contain;
}
.imagePreview {
max-width: 50vw;
max-height: 50vh;
object-fit: contain;
}
.content {
flex: 1;
.chunkText;
}
.contentEllipsis {
.multipleLineEllipsis(3);
}
.contentText {
word-break: break-all !important;
}
.chunkCard {
width: 100%;
padding: 18px 10px;
}
.cardSelected {
background-color: @selectedBackgroundColor;
}
.cardSelectedDark {
background-color: #ffffff2f;
}

View File

@ -0,0 +1,127 @@
import Image from '@/components/image';
import { useTheme } from '@/components/theme-provider';
import { Card } from '@/components/ui/card';
import { Checkbox } from '@/components/ui/checkbox';
import {
Popover,
PopoverContent,
PopoverTrigger,
} from '@/components/ui/popover';
import { Switch } from '@/components/ui/switch';
import { IChunk } from '@/interfaces/database/knowledge';
import { CheckedState } from '@radix-ui/react-checkbox';
import classNames from 'classnames';
import DOMPurify from 'dompurify';
import { useEffect, useState } from 'react';
import { ChunkTextMode } from '../../constant';
import styles from './index.less';
interface IProps {
item: IChunk;
checked: boolean;
switchChunk: (available?: number, chunkIds?: string[]) => void;
editChunk: (chunkId: string) => void;
handleCheckboxClick: (chunkId: string, checked: boolean) => void;
selected: boolean;
clickChunkCard: (chunkId: string) => void;
textMode: ChunkTextMode;
}
const ChunkCard = ({
item,
checked,
handleCheckboxClick,
editChunk,
switchChunk,
selected,
clickChunkCard,
textMode,
}: IProps) => {
const available = Number(item.available_int);
const [enabled, setEnabled] = useState(false);
const { theme } = useTheme();
const onChange = (checked: boolean) => {
setEnabled(checked);
switchChunk(available === 0 ? 1 : 0, [item.chunk_id]);
};
const handleCheck = (e: CheckedState) => {
handleCheckboxClick(item.chunk_id, e === 'indeterminate' ? false : e);
};
const handleContentDoubleClick = () => {
editChunk(item.chunk_id);
};
const handleContentClick = () => {
clickChunkCard(item.chunk_id);
};
useEffect(() => {
setEnabled(available === 1);
}, [available]);
const [open, setOpen] = useState<boolean>(false);
return (
<Card
className={classNames('rounded-lg w-full py-3 px-3', {
'bg-bg-title': selected,
'bg-bg-input': !selected,
})}
>
<div className="flex items-start justify-between gap-2">
<Checkbox onCheckedChange={handleCheck} checked={checked}></Checkbox>
{item.image_id && (
<Popover open={open}>
<PopoverTrigger
asChild
onMouseEnter={() => setOpen(true)}
onMouseLeave={() => setOpen(false)}
>
<div>
<Image id={item.image_id} className={styles.image}></Image>
</div>
</PopoverTrigger>
<PopoverContent
className="p-0"
align={'start'}
side={'right'}
sideOffset={-20}
>
<div>
<Image
id={item.image_id}
className={styles.imagePreview}
></Image>
</div>
</PopoverContent>
</Popover>
)}
<section
onDoubleClick={handleContentDoubleClick}
onClick={handleContentClick}
className={styles.content}
>
<div
dangerouslySetInnerHTML={{
__html: DOMPurify.sanitize(item.content_with_weight),
}}
className={classNames(styles.contentText, {
[styles.contentEllipsis]: textMode === ChunkTextMode.Ellipse,
})}
></div>
</section>
<div>
<Switch
checked={enabled}
onCheckedChange={onChange}
aria-readonly
className="!m-0"
/>
</div>
</div>
</Card>
);
};
export default ChunkCard;

View File

@ -0,0 +1,206 @@
import EditTag from '@/components/edit-tag';
import Divider from '@/components/ui/divider';
import {
Form,
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import {
HoverCard,
HoverCardContent,
HoverCardTrigger,
} from '@/components/ui/hover-card';
import { Modal } from '@/components/ui/modal/modal';
import Space from '@/components/ui/space';
import { Switch } from '@/components/ui/switch';
import { Textarea } from '@/components/ui/textarea';
import { useFetchChunk } from '@/hooks/chunk-hooks';
import { IModalProps } from '@/interfaces/common';
import { Trash2 } from 'lucide-react';
import React, { useCallback, useEffect, useState } from 'react';
import { FieldValues, FormProvider, useForm } from 'react-hook-form';
import { useTranslation } from 'react-i18next';
import { useDeleteChunkByIds } from '../../hooks';
import {
transformTagFeaturesArrayToObject,
transformTagFeaturesObjectToArray,
} from '../../utils';
import { TagFeatureItem } from './tag-feature-item';
interface kFProps {
doc_id: string;
chunkId: string | undefined;
parserId: string;
}
const ChunkCreatingModal: React.FC<IModalProps<any> & kFProps> = ({
doc_id,
chunkId,
hideModal,
onOk,
loading,
parserId,
}) => {
// const [form] = Form.useForm();
// const form = useFormContext();
const form = useForm<FieldValues>({
defaultValues: {
content_with_weight: '',
tag_kwd: [],
question_kwd: [],
important_kwd: [],
tag_feas: [],
},
});
const [checked, setChecked] = useState(false);
const { removeChunk } = useDeleteChunkByIds();
const { data } = useFetchChunk(chunkId);
const { t } = useTranslation();
const isTagParser = parserId === 'tag';
const onSubmit = useCallback(
(values: FieldValues) => {
onOk?.({
...values,
tag_feas: transformTagFeaturesArrayToObject(values.tag_feas),
available_int: checked ? 1 : 0,
});
},
[checked, onOk],
);
const handleOk = form.handleSubmit(onSubmit);
const handleRemove = useCallback(() => {
if (chunkId) {
return removeChunk([chunkId], doc_id);
}
}, [chunkId, doc_id, removeChunk]);
const handleCheck = useCallback(() => {
setChecked(!checked);
}, [checked]);
useEffect(() => {
if (data?.code === 0) {
const { available_int, tag_feas } = data.data;
form.reset({
...data.data,
tag_feas: transformTagFeaturesObjectToArray(tag_feas),
});
setChecked(available_int !== 0);
}
}, [data, form, chunkId]);
return (
<Modal
title={`${chunkId ? t('common.edit') : t('common.create')} ${t('chunk.chunk')}`}
open={true}
onOk={handleOk}
onCancel={hideModal}
confirmLoading={loading}
destroyOnClose
>
<Form {...form}>
<div className="flex flex-col gap-4">
<FormField
control={form.control}
name="content_with_weight"
render={({ field }) => (
<FormItem>
<FormLabel>{t('chunk.chunk')}</FormLabel>
<FormControl>
<Textarea {...field} autoSize={{ minRows: 4, maxRows: 10 }} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="important_kwd"
render={({ field }) => (
<FormItem>
<FormLabel>{t('chunk.keyword')}</FormLabel>
<FormControl>
<EditTag {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name="question_kwd"
render={({ field }) => (
<FormItem>
<FormLabel className="flex justify-start items-start">
<div className="flex items-center gap-0">
<span>{t('chunk.question')}</span>
<HoverCard>
<HoverCardTrigger asChild>
<span className="text-xs mt-[-3px] text-center scale-[90%] font-thin text-primary cursor-pointer rounded-full w-[16px] h-[16px] border-muted-foreground/50 border">
?
</span>
</HoverCardTrigger>
<HoverCardContent className="w-80" side="top">
{t('chunk.questionTip')}
</HoverCardContent>
</HoverCard>
</div>
</FormLabel>
<FormControl>
<EditTag {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
{isTagParser && (
<FormField
control={form.control}
name="tag_kwd"
render={({ field }) => (
<FormItem>
<FormLabel>{t('knowledgeConfiguration.tagName')}</FormLabel>
<FormControl>
<EditTag {...field} />
</FormControl>
<FormMessage />
</FormItem>
)}
/>
)}
{!isTagParser && (
<FormProvider {...form}>
<TagFeatureItem />
</FormProvider>
)}
</div>
</Form>
{chunkId && (
<section>
<Divider />
<Space size={'large'}>
<div className="flex items-center gap-2">
{t('chunk.enabled')}
<Switch checked={checked} onCheckedChange={handleCheck} />
</div>
<div className="flex items-center gap-1" onClick={handleRemove}>
<Trash2 size={16} /> {t('common.delete')}
</div>
</Space>
</section>
)}
</Modal>
);
};
export default ChunkCreatingModal;

View File

@ -0,0 +1,136 @@
import { SelectWithSearch } from '@/components/originui/select-with-search';
import { Button } from '@/components/ui/button';
import {
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { NumberInput } from '@/components/ui/input';
import { useFetchTagListByKnowledgeIds } from '@/hooks/knowledge-hooks';
import { useFetchKnowledgeBaseConfiguration } from '@/hooks/use-knowledge-request';
import { CircleMinus, Plus } from 'lucide-react';
import { useCallback, useEffect, useMemo } from 'react';
import { useFieldArray, useFormContext } from 'react-hook-form';
import { useTranslation } from 'react-i18next';
import { FormListItem } from '../../utils';
const FieldKey = 'tag_feas';
export const TagFeatureItem = () => {
const { t } = useTranslation();
const { setKnowledgeIds, list } = useFetchTagListByKnowledgeIds();
const { data: knowledgeConfiguration } = useFetchKnowledgeBaseConfiguration();
const form = useFormContext();
const tagKnowledgeIds = useMemo(() => {
return knowledgeConfiguration?.parser_config?.tag_kb_ids ?? [];
}, [knowledgeConfiguration?.parser_config?.tag_kb_ids]);
const options = useMemo(() => {
return list.map((x) => ({
value: x[0],
label: x[0],
}));
}, [list]);
const filterOptions = useCallback(
(index: number) => {
const tags: FormListItem[] = form.getValues(FieldKey) ?? [];
// Exclude it's own current data
const list = tags
.filter((x, idx) => x && index !== idx)
.map((x) => x.tag);
// Exclude the selected data from other options from one's own options.
const resultList = options.filter(
(x) => !list.some((y) => x.value === y),
);
return resultList;
},
[form, options],
);
useEffect(() => {
setKnowledgeIds(tagKnowledgeIds);
}, [setKnowledgeIds, tagKnowledgeIds]);
const { fields, append, remove } = useFieldArray({
control: form.control,
name: FieldKey,
});
return (
<FormField
control={form.control}
name={FieldKey as any}
render={() => (
<FormItem>
<FormLabel>{t('knowledgeConfiguration.tags')}</FormLabel>
<div>
{fields.map((item, name) => {
return (
<div key={item.id} className="flex gap-3 items-center mb-4">
<div className="flex flex-1 gap-8">
<FormField
control={form.control}
name={`${FieldKey}.${name}.tag` as any}
render={({ field }) => (
<FormItem className="w-2/3">
<FormControl className="w-full">
<div>
<SelectWithSearch
options={filterOptions(name)}
placeholder={t(
'knowledgeConfiguration.tagName',
)}
value={field.value}
onChange={field.onChange}
/>
</div>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name={`${FieldKey}.${name}.frequency`}
render={({ field }) => (
<FormItem>
<FormControl>
<NumberInput
value={field.value}
onChange={field.onChange}
placeholder={t(
'knowledgeConfiguration.frequency',
)}
max={10}
min={0}
/>
</FormControl>
<FormMessage />
</FormItem>
)}
/>
</div>
<CircleMinus
onClick={() => remove(name)}
className="text-red-500"
/>
</div>
);
})}
<Button
variant="dashed"
className="w-full flex items-center justify-center gap-2"
onClick={() => append({ tag: '', frequency: 0 })}
>
<Plus size={16} />
{t('knowledgeConfiguration.addTag')}
</Button>
</div>
</FormItem>
)}
/>
);
};

View File

@ -0,0 +1,85 @@
import { Checkbox } from '@/components/ui/checkbox';
import { Label } from '@/components/ui/label';
import { Ban, CircleCheck, Trash2 } from 'lucide-react';
import { useCallback, useMemo } from 'react';
import { useTranslation } from 'react-i18next';
type ICheckboxSetProps = {
selectAllChunk: (e: any) => void;
removeChunk: (e?: any) => void;
switchChunk: (available: number) => void;
checked: boolean;
selectedChunkIds: string[];
};
export default (props: ICheckboxSetProps) => {
const {
selectAllChunk,
removeChunk,
switchChunk,
checked,
selectedChunkIds,
} = props;
const { t } = useTranslation();
const handleSelectAllCheck = useCallback(
(e: any) => {
console.log('eee=', e);
selectAllChunk(e);
},
[selectAllChunk],
);
const handleDeleteClick = useCallback(() => {
removeChunk();
}, [removeChunk]);
const handleEnabledClick = useCallback(() => {
switchChunk(1);
}, [switchChunk]);
const handleDisabledClick = useCallback(() => {
switchChunk(0);
}, [switchChunk]);
const isSelected = useMemo(() => {
return selectedChunkIds?.length > 0;
}, [selectedChunkIds]);
return (
<div className="flex gap-[40px] py-4 px-2">
<div className="flex items-center gap-3 cursor-pointer text-muted-foreground hover:text-text-primary">
<Checkbox
id="all_chunks_checkbox"
onCheckedChange={handleSelectAllCheck}
checked={checked}
className=" data-[state=checked]:bg-text-primary data-[state=checked]:border-text-primary data-[state=checked]:text-bg-base border-muted-foreground text-muted-foreground hover:text-bg-base hover:border-text-primary "
/>
<Label htmlFor="all_chunks_checkbox">{t('chunk.selectAll')}</Label>
</div>
{isSelected && (
<>
<div
className="flex items-center cursor-pointer text-muted-foreground hover:text-text-primary"
onClick={handleEnabledClick}
>
<CircleCheck size={16} />
<span className="block ml-1">{t('chunk.enable')}</span>
</div>
<div
className="flex items-center cursor-pointer text-muted-foreground hover:text-text-primary"
onClick={handleDisabledClick}
>
<Ban size={16} />
<span className="block ml-1">{t('chunk.disable')}</span>
</div>
<div
className="flex items-center cursor-pointer text-red-400 hover:text-red-500"
onClick={handleDeleteClick}
>
<Trash2 size={16} />
<span className="block ml-1">{t('chunk.delete')}</span>
</div>
</>
)}
</div>
);
};

View File

@ -0,0 +1,108 @@
import { Input } from '@/components/originui/input';
import { Button } from '@/components/ui/button';
import {
Popover,
PopoverContent,
PopoverTrigger,
} from '@/components/ui/popover';
import { Radio } from '@/components/ui/radio';
import { useTranslate } from '@/hooks/common-hooks';
import { cn } from '@/lib/utils';
import { SearchOutlined } from '@ant-design/icons';
import { ListFilter, Plus } from 'lucide-react';
import { useState } from 'react';
import { ChunkTextMode } from '../../constant';
interface ChunkResultBarProps {
changeChunkTextMode: React.Dispatch<React.SetStateAction<string | number>>;
available: number | undefined;
selectAllChunk: (value: boolean) => void;
handleSetAvailable: (value: number | undefined) => void;
createChunk: () => void;
handleInputChange: (e: React.ChangeEvent<HTMLInputElement>) => void;
searchString: string;
}
export default ({
changeChunkTextMode,
available,
selectAllChunk,
handleSetAvailable,
createChunk,
handleInputChange,
searchString,
}: ChunkResultBarProps) => {
const { t } = useTranslate('chunk');
const [textSelectValue, setTextSelectValue] = useState<string | number>(
ChunkTextMode.Full,
);
const handleFilterChange = (e: string | number) => {
const value = e === -1 ? undefined : (e as number);
selectAllChunk(false);
handleSetAvailable(value);
};
const filterContent = (
<div className="w-[200px]">
<Radio.Group onChange={handleFilterChange} value={available}>
<div className="flex flex-col gap-2 p-4">
<Radio value={-1}>{t('all')}</Radio>
<Radio value={1}>{t('enabled')}</Radio>
<Radio value={0}>{t('disabled')}</Radio>
</div>
</Radio.Group>
</div>
);
const textSelectOptions = [
{ label: t(ChunkTextMode.Full), value: ChunkTextMode.Full },
{ label: t(ChunkTextMode.Ellipse), value: ChunkTextMode.Ellipse },
];
const changeTextSelectValue = (value: string | number) => {
setTextSelectValue(value);
changeChunkTextMode(value);
};
return (
<div className="flex gap-2">
<div className="flex items-center gap-1 bg-bg-card text-muted-foreground w-fit h-[35px] rounded-md p-1">
{textSelectOptions.map((option) => (
<div
key={option.value}
className={cn(
'flex items-center cursor-pointer px-4 py-1 rounded-md',
{
'text-primary bg-bg-base': option.value === textSelectValue,
'text-text-primary': option.value !== textSelectValue,
},
)}
onClick={() => changeTextSelectValue(option.value)}
>
{option.label}
</div>
))}
</div>
<Input
className="bg-bg-card text-muted-foreground"
style={{ width: 200 }}
placeholder={t('search')}
icon={<SearchOutlined />}
onChange={handleInputChange}
value={searchString}
/>
<Popover>
<PopoverTrigger asChild>
<Button className="bg-bg-card text-muted-foreground hover:bg-card">
<ListFilter />
</Button>
</PopoverTrigger>
<PopoverContent className="p-0 w-[200px]">
{filterContent}
</PopoverContent>
</Popover>
<Button
onClick={() => createChunk()}
variant={'secondary'}
className="bg-bg-card text-muted-foreground hover:bg-card"
>
<Plus size={44} />
</Button>
</div>
);
};

View File

@ -0,0 +1,221 @@
import { ReactComponent as FilterIcon } from '@/assets/filter.svg';
import { KnowledgeRouteKey } from '@/constants/knowledge';
import { IChunkListResult, useSelectChunkList } from '@/hooks/chunk-hooks';
import { useTranslate } from '@/hooks/common-hooks';
import { useKnowledgeBaseId } from '@/hooks/knowledge-hooks';
import {
ArrowLeftOutlined,
CheckCircleOutlined,
CloseCircleOutlined,
DeleteOutlined,
DownOutlined,
FilePdfOutlined,
PlusOutlined,
SearchOutlined,
} from '@ant-design/icons';
import {
Button,
Checkbox,
Flex,
Input,
Menu,
MenuProps,
Popover,
Radio,
RadioChangeEvent,
Segmented,
SegmentedProps,
Space,
Typography,
} from 'antd';
import { useCallback, useMemo, useState } from 'react';
import { Link } from 'umi';
import { ChunkTextMode } from '../../constant';
const { Text } = Typography;
interface IProps
extends Pick<
IChunkListResult,
'searchString' | 'handleInputChange' | 'available' | 'handleSetAvailable'
> {
checked: boolean;
selectAllChunk: (checked: boolean) => void;
createChunk: () => void;
removeChunk: () => void;
switchChunk: (available: number) => void;
changeChunkTextMode(mode: ChunkTextMode): void;
}
const ChunkToolBar = ({
selectAllChunk,
checked,
createChunk,
removeChunk,
switchChunk,
changeChunkTextMode,
available,
handleSetAvailable,
searchString,
handleInputChange,
}: IProps) => {
const data = useSelectChunkList();
const documentInfo = data?.documentInfo;
const knowledgeBaseId = useKnowledgeBaseId();
const [isShowSearchBox, setIsShowSearchBox] = useState(false);
const { t } = useTranslate('chunk');
const handleSelectAllCheck = useCallback(
(e: any) => {
selectAllChunk(e.target.checked);
},
[selectAllChunk],
);
const handleSearchIconClick = () => {
setIsShowSearchBox(true);
};
const handleSearchBlur = () => {
if (!searchString?.trim()) {
setIsShowSearchBox(false);
}
};
const handleDelete = useCallback(() => {
removeChunk();
}, [removeChunk]);
const handleEnabledClick = useCallback(() => {
switchChunk(1);
}, [switchChunk]);
const handleDisabledClick = useCallback(() => {
switchChunk(0);
}, [switchChunk]);
const items: MenuProps['items'] = useMemo(() => {
return [
{
key: '1',
label: (
<>
<Checkbox onChange={handleSelectAllCheck} checked={checked}>
<b>{t('selectAll')}</b>
</Checkbox>
</>
),
},
{ type: 'divider' },
{
key: '2',
label: (
<Space onClick={handleEnabledClick}>
<CheckCircleOutlined />
<b>{t('enabledSelected')}</b>
</Space>
),
},
{
key: '3',
label: (
<Space onClick={handleDisabledClick}>
<CloseCircleOutlined />
<b>{t('disabledSelected')}</b>
</Space>
),
},
{ type: 'divider' },
{
key: '4',
label: (
<Space onClick={handleDelete}>
<DeleteOutlined />
<b>{t('deleteSelected')}</b>
</Space>
),
},
];
}, [
checked,
handleSelectAllCheck,
handleDelete,
handleEnabledClick,
handleDisabledClick,
t,
]);
const content = (
<Menu style={{ width: 200 }} items={items} selectable={false} />
);
const handleFilterChange = (e: RadioChangeEvent) => {
selectAllChunk(false);
handleSetAvailable(e.target.value);
};
const filterContent = (
<Radio.Group onChange={handleFilterChange} value={available}>
<Space direction="vertical">
<Radio value={undefined}>{t('all')}</Radio>
<Radio value={1}>{t('enabled')}</Radio>
<Radio value={0}>{t('disabled')}</Radio>
</Space>
</Radio.Group>
);
return (
<Flex justify="space-between" align="center">
<Space size={'middle'}>
<Link
to={`/knowledge/${KnowledgeRouteKey.Dataset}?id=${knowledgeBaseId}`}
>
<ArrowLeftOutlined />
</Link>
<FilePdfOutlined />
<Text ellipsis={{ tooltip: documentInfo?.name }} style={{ width: 150 }}>
{documentInfo?.name}
</Text>
</Space>
<Space>
<Segmented
options={[
{ label: t(ChunkTextMode.Full), value: ChunkTextMode.Full },
{ label: t(ChunkTextMode.Ellipse), value: ChunkTextMode.Ellipse },
]}
onChange={changeChunkTextMode as SegmentedProps['onChange']}
/>
<Popover content={content} placement="bottom" arrow={false}>
<Button>
{t('bulk')}
<DownOutlined />
</Button>
</Popover>
{isShowSearchBox ? (
<Input
size="middle"
placeholder={t('search')}
prefix={<SearchOutlined />}
allowClear
onChange={handleInputChange}
onBlur={handleSearchBlur}
value={searchString}
/>
) : (
<Button icon={<SearchOutlined />} onClick={handleSearchIconClick} />
)}
<Popover content={filterContent} placement="bottom" arrow={false}>
<Button icon={<FilterIcon />} />
</Popover>
<Button
icon={<PlusOutlined />}
type="primary"
onClick={() => createChunk()}
/>
</Space>
</Flex>
);
};
export default ChunkToolBar;

View File

@ -0,0 +1,114 @@
import message from '@/components/ui/message';
import { Spin } from '@/components/ui/spin';
import request from '@/utils/request';
import classNames from 'classnames';
import React, { useEffect, useRef, useState } from 'react';
interface CSVData {
rows: string[][];
headers: string[];
}
interface FileViewerProps {
className?: string;
url: string;
}
const CSVFileViewer: React.FC<FileViewerProps> = ({ url }) => {
const [data, setData] = useState<CSVData | null>(null);
const [isLoading, setIsLoading] = useState<boolean>(true);
const containerRef = useRef<HTMLDivElement>(null);
// const url = useGetDocumentUrl();
const parseCSV = (csvText: string): CSVData => {
console.log('Parsing CSV data:', csvText);
const lines = csvText.split('\n');
const headers = lines[0].split(',').map((header) => header.trim());
const rows = lines
.slice(1)
.map((line) => line.split(',').map((cell) => cell.trim()));
return { headers, rows };
};
useEffect(() => {
const loadCSV = async () => {
try {
const res = await request(url, {
method: 'GET',
responseType: 'blob',
onError: () => {
message.error('file load failed');
setIsLoading(false);
},
});
// parse CSV file
const reader = new FileReader();
reader.readAsText(res.data);
reader.onload = () => {
const parsedData = parseCSV(reader.result as string);
console.log('file loaded successfully', reader.result);
setData(parsedData);
};
} catch (error) {
message.error('CSV file parse failed');
console.error('Error loading CSV file:', error);
} finally {
setIsLoading(false);
}
};
loadCSV();
return () => {
setData(null);
};
}, [url]);
return (
<div
ref={containerRef}
className={classNames(
'relative w-full h-full p-4 bg-background-paper border border-border-normal rounded-md',
'overflow-auto max-h-[80vh] p-2',
)}
>
{isLoading ? (
<div className="absolute inset-0 flex items-center justify-center">
<Spin />
</div>
) : data ? (
<table className="min-w-full divide-y divide-border-normal">
<thead className="bg-background-header-bar">
<tr>
{data.headers.map((header, index) => (
<th
key={`header-${index}`}
className="px-6 py-3 text-left text-sm font-medium text-text-primary"
>
{header}
</th>
))}
</tr>
</thead>
<tbody className="bg-background-paper divide-y divide-border-normal">
{data.rows.map((row, rowIndex) => (
<tr key={`row-${rowIndex}`}>
{row.map((cell, cellIndex) => (
<td
key={`cell-${rowIndex}-${cellIndex}`}
className="px-6 py-4 whitespace-nowrap text-sm text-text-secondary"
>
{cell || '-'}
</td>
))}
</tr>
))}
</tbody>
</table>
) : null}
</div>
);
};
export default CSVFileViewer;

View File

@ -0,0 +1,70 @@
import message from '@/components/ui/message';
import { Spin } from '@/components/ui/spin';
import request from '@/utils/request';
import classNames from 'classnames';
import mammoth from 'mammoth';
import { useEffect, useState } from 'react';
interface DocPreviewerProps {
className?: string;
url: string;
}
export const DocPreviewer: React.FC<DocPreviewerProps> = ({
className,
url,
}) => {
// const url = useGetDocumentUrl();
const [htmlContent, setHtmlContent] = useState<string>('');
const [loading, setLoading] = useState(false);
const fetchDocument = async () => {
setLoading(true);
const res = await request(url, {
method: 'GET',
responseType: 'blob',
onError: () => {
message.error('Document parsing failed');
console.error('Error loading document:', url);
},
});
try {
const arrayBuffer = await res.data.arrayBuffer();
const result = await mammoth.convertToHtml(
{ arrayBuffer },
{ includeDefaultStyleMap: true },
);
const styledContent = result.value
.replace(/<p>/g, '<p class="mb-2">')
.replace(/<h(\d)>/g, '<h$1 class="font-semibold mt-4 mb-2">');
setHtmlContent(styledContent);
} catch (err) {
message.error('Document parsing failed');
console.error('Error parsing document:', err);
}
setLoading(false);
};
useEffect(() => {
if (url) {
fetchDocument();
}
}, [url]);
return (
<div
className={classNames(
'relative w-full h-full p-4 bg-background-paper border border-border-normal rounded-md',
className,
)}
>
{loading && (
<div className="absolute inset-0 flex items-center justify-center">
<Spin />
</div>
)}
{!loading && <div dangerouslySetInnerHTML={{ __html: htmlContent }} />}
</div>
);
};

View File

@ -0,0 +1,21 @@
import { formatDate } from '@/utils/date';
import { formatBytes } from '@/utils/file-util';
type Props = {
size: number;
name: string;
create_date: string;
};
export default ({ size, name, create_date }: Props) => {
const sizeName = formatBytes(size);
const dateStr = formatDate(create_date);
return (
<div>
<h2 className="text-[16px]">{name}</h2>
<div className="text-text-secondary text-[12px] pt-[5px]">
Size{sizeName} Uploaded Time{dateStr}
</div>
</div>
);
};

View File

@ -0,0 +1,25 @@
import { useFetchExcel } from '@/pages/document-viewer/hooks';
import classNames from 'classnames';
interface ExcelCsvPreviewerProps {
className?: string;
url: string;
}
export const ExcelCsvPreviewer: React.FC<ExcelCsvPreviewerProps> = ({
className,
url,
}) => {
// const url = useGetDocumentUrl();
const { containerRef } = useFetchExcel(url);
return (
<div
ref={containerRef}
className={classNames(
'relative w-full h-full p-4 bg-background-paper border border-border-normal rounded-md excel-csv-previewer',
className,
)}
></div>
);
};

View File

@ -0,0 +1,55 @@
import { useGetKnowledgeSearchParams } from '@/hooks/route-hook';
import { api_host } from '@/utils/api';
import { useSize } from 'ahooks';
import { CustomTextRenderer } from 'node_modules/react-pdf/dist/esm/shared/types';
import { useCallback, useEffect, useMemo, useState } from 'react';
export const useDocumentResizeObserver = () => {
const [containerWidth, setContainerWidth] = useState<number>();
const [containerRef, setContainerRef] = useState<HTMLElement | null>(null);
const size = useSize(containerRef);
const onResize = useCallback((width?: number) => {
if (width) {
setContainerWidth(width);
}
}, []);
useEffect(() => {
onResize(size?.width);
}, [size?.width, onResize]);
return { containerWidth, setContainerRef };
};
function highlightPattern(text: string, pattern: string, pageNumber: number) {
if (pageNumber === 2) {
return `<mark>${text}</mark>`;
}
if (text.trim() !== '' && pattern.match(text)) {
// return pattern.replace(text, (value) => `<mark>${value}</mark>`);
return `<mark>${text}</mark>`;
}
return text.replace(pattern, (value) => `<mark>${value}</mark>`);
}
export const useHighlightText = (searchText: string = '') => {
const textRenderer: CustomTextRenderer = useCallback(
(textItem) => {
return highlightPattern(textItem.str, searchText, textItem.pageNumber);
},
[searchText],
);
return textRenderer;
};
export const useGetDocumentUrl = () => {
const { documentId } = useGetKnowledgeSearchParams();
const url = useMemo(() => {
return `${api_host}/document/get/${documentId}`;
}, [documentId]);
return url;
};

View File

@ -0,0 +1,73 @@
import message from '@/components/ui/message';
import { Spin } from '@/components/ui/spin';
import request from '@/utils/request';
import classNames from 'classnames';
import { useEffect, useState } from 'react';
interface ImagePreviewerProps {
className?: string;
url: string;
}
export const ImagePreviewer: React.FC<ImagePreviewerProps> = ({
className,
url,
}) => {
// const url = useGetDocumentUrl();
const [imageSrc, setImageSrc] = useState<string | null>(null);
const [isLoading, setIsLoading] = useState<boolean>(true);
const fetchImage = async () => {
setIsLoading(true);
const res = await request(url, {
method: 'GET',
responseType: 'blob',
onError: () => {
message.error('Failed to load image');
setIsLoading(false);
},
});
const objectUrl = URL.createObjectURL(res.data);
setImageSrc(objectUrl);
setIsLoading(false);
};
useEffect(() => {
if (url) {
fetchImage();
}
}, [url]);
useEffect(() => {
return () => {
if (imageSrc) {
URL.revokeObjectURL(imageSrc);
}
};
}, [imageSrc]);
return (
<div
className={classNames(
'relative w-full h-full p-4 bg-background-paper border border-border-normal rounded-md image-previewer',
className,
)}
>
{isLoading && (
<div className="absolute inset-0 flex items-center justify-center">
<Spin />
</div>
)}
{!isLoading && imageSrc && (
<div className="max-h-[80vh] overflow-auto p-2">
<img
src={imageSrc}
alt={'image'}
className="w-full h-auto max-w-full object-contain"
onLoad={() => URL.revokeObjectURL(imageSrc!)}
/>
</div>
)}
</div>
);
};

View File

@ -0,0 +1,13 @@
.documentContainer {
width: 100%;
// height: calc(100vh - 284px);
height: calc(100vh - 170px);
position: relative;
:global(.PdfHighlighter) {
overflow-x: hidden;
}
:global(.Highlight--scrolledTo .Highlight__part) {
overflow-x: hidden;
background-color: rgba(255, 226, 143, 1);
}
}

View File

@ -0,0 +1,68 @@
import { memo } from 'react';
import CSVFileViewer from './csv-preview';
import { DocPreviewer } from './doc-preview';
import { ExcelCsvPreviewer } from './excel-preview';
import { ImagePreviewer } from './image-preview';
import styles from './index.less';
import PdfPreviewer, { IProps } from './pdf-preview';
import { PptPreviewer } from './ppt-preview';
import { TxtPreviewer } from './txt-preview';
type PreviewProps = {
fileType: string;
className?: string;
url: string;
};
const Preview = ({
fileType,
className,
highlights,
setWidthAndHeight,
url,
}: PreviewProps & Partial<IProps>) => {
return (
<>
{fileType === 'pdf' && highlights && setWidthAndHeight && (
<section className={styles.documentPreview}>
<PdfPreviewer
highlights={highlights}
setWidthAndHeight={setWidthAndHeight}
url={url}
></PdfPreviewer>
</section>
)}
{['doc', 'docx'].indexOf(fileType) > -1 && (
<section>
<DocPreviewer className={className} url={url} />
</section>
)}
{['txt', 'md'].indexOf(fileType) > -1 && (
<section>
<TxtPreviewer className={className} url={url} />
</section>
)}
{['visual'].indexOf(fileType) > -1 && (
<section>
<ImagePreviewer className={className} url={url} />
</section>
)}
{['pptx'].indexOf(fileType) > -1 && (
<section>
<PptPreviewer className={className} url={url} />
</section>
)}
{['xlsx'].indexOf(fileType) > -1 && (
<section>
<ExcelCsvPreviewer className={className} url={url} />
</section>
)}
{['csv'].indexOf(fileType) > -1 && (
<section>
<CSVFileViewer className={className} url={url} />
</section>
)}
</>
);
};
export default memo(Preview);

View File

@ -0,0 +1,127 @@
import { memo, useEffect, useRef } from 'react';
import {
AreaHighlight,
Highlight,
IHighlight,
PdfHighlighter,
PdfLoader,
Popup,
} from 'react-pdf-highlighter';
import { useCatchDocumentError } from '@/components/pdf-previewer/hooks';
import { Spin } from '@/components/ui/spin';
import FileError from '@/pages/document-viewer/file-error';
import styles from './index.less';
export interface IProps {
highlights: IHighlight[];
setWidthAndHeight: (width: number, height: number) => void;
url: string;
}
const HighlightPopup = ({
comment,
}: {
comment: { text: string; emoji: string };
}) =>
comment.text ? (
<div className="Highlight__popup">
{comment.emoji} {comment.text}
</div>
) : null;
// TODO: merge with DocumentPreviewer
const PdfPreview = ({ highlights: state, setWidthAndHeight, url }: IProps) => {
// const url = useGetDocumentUrl();
const ref = useRef<(highlight: IHighlight) => void>(() => {});
const error = useCatchDocumentError(url);
const resetHash = () => {};
useEffect(() => {
if (state.length > 0) {
ref?.current(state[0]);
}
}, [state]);
return (
<div
className={`${styles.documentContainer} rounded-[10px] overflow-hidden `}
>
<PdfLoader
url={url}
beforeLoad={
<div className="absolute inset-0 flex items-center justify-center">
<Spin />
</div>
}
workerSrc="/pdfjs-dist/pdf.worker.min.js"
errorMessage={<FileError>{error}</FileError>}
>
{(pdfDocument) => {
pdfDocument.getPage(1).then((page) => {
const viewport = page.getViewport({ scale: 1 });
const width = viewport.width;
const height = viewport.height;
setWidthAndHeight(width, height);
});
return (
<PdfHighlighter
pdfDocument={pdfDocument}
enableAreaSelection={(event) => event.altKey}
onScrollChange={resetHash}
scrollRef={(scrollTo) => {
ref.current = scrollTo;
}}
onSelectionFinished={() => null}
highlightTransform={(
highlight,
index,
setTip,
hideTip,
viewportToScaled,
screenshot,
isScrolledTo,
) => {
const isTextHighlight = !Boolean(
highlight.content && highlight.content.image,
);
const component = isTextHighlight ? (
<Highlight
isScrolledTo={isScrolledTo}
position={highlight.position}
comment={highlight.comment}
/>
) : (
<AreaHighlight
isScrolledTo={isScrolledTo}
highlight={highlight}
onChange={() => {}}
/>
);
return (
<Popup
popupContent={<HighlightPopup {...highlight} />}
onMouseOver={(popupContent) =>
setTip(highlight, () => popupContent)
}
onMouseOut={hideTip}
key={index}
>
{component}
</Popup>
);
}}
highlights={state}
/>
);
}}
</PdfLoader>
</div>
);
};
export default memo(PdfPreview);

View File

@ -0,0 +1,70 @@
import message from '@/components/ui/message';
import request from '@/utils/request';
import classNames from 'classnames';
import { init } from 'pptx-preview';
import { useEffect, useRef } from 'react';
interface PptPreviewerProps {
className?: string;
url: string;
}
export const PptPreviewer: React.FC<PptPreviewerProps> = ({
className,
url,
}) => {
// const url = useGetDocumentUrl();
const wrapper = useRef<HTMLDivElement>(null);
const containerRef = useRef<HTMLDivElement>(null);
const fetchDocument = async () => {
const res = await request(url, {
method: 'GET',
responseType: 'blob',
onError: () => {
message.error('Document parsing failed');
console.error('Error loading document:', url);
},
});
console.log(res);
try {
const arrayBuffer = await res.data.arrayBuffer();
if (containerRef.current) {
let width = 500;
let height = 900;
if (containerRef.current) {
width = containerRef.current.clientWidth - 50;
height = containerRef.current.clientHeight - 50;
}
let pptxPrviewer = init(containerRef.current, {
width: width,
height: height,
});
pptxPrviewer.preview(arrayBuffer);
}
} catch (err) {
message.error('ppt parse failed');
}
};
useEffect(() => {
if (url) {
fetchDocument();
}
}, [url]);
return (
<div
ref={containerRef}
className={classNames(
'relative w-full h-full p-4 bg-background-paper border border-border-normal rounded-md ppt-previewer',
className,
)}
>
<div className="overflow-auto p-2">
<div className="flex flex-col gap-4">
<div ref={wrapper} />
</div>
</div>
</div>
);
};

View File

@ -0,0 +1,56 @@
import message from '@/components/ui/message';
import { Spin } from '@/components/ui/spin';
import request from '@/utils/request';
import classNames from 'classnames';
import { useEffect, useState } from 'react';
type TxtPreviewerProps = { className?: string; url: string };
export const TxtPreviewer = ({ className, url }: TxtPreviewerProps) => {
// const url = useGetDocumentUrl();
const [loading, setLoading] = useState(false);
const [data, setData] = useState<string>('');
const fetchTxt = async () => {
setLoading(true);
const res = await request(url, {
method: 'GET',
responseType: 'blob',
onError: (err: any) => {
message.error('Failed to load file');
console.error('Error loading file:', err);
},
});
// blob to string
const reader = new FileReader();
reader.readAsText(res.data);
reader.onload = () => {
setData(reader.result as string);
setLoading(false);
console.log('file loaded successfully', reader.result);
};
console.log('file data:', res);
};
useEffect(() => {
if (url) {
fetchTxt();
} else {
setLoading(false);
setData('');
}
}, [url]);
return (
<div
className={classNames(
'relative w-full h-full p-4 bg-background-paper border border-border-normal rounded-md',
className,
)}
>
{loading && (
<div className="absolute inset-0 flex items-center justify-center">
<Spin />
</div>
)}
{!loading && <pre className="whitespace-pre-wrap p-2 ">{data}</pre>}
</div>
);
};

View File

@ -0,0 +1,48 @@
import { Textarea } from '@/components/ui/textarea';
import { cn } from '@/lib/utils';
import { useState } from 'react';
interface FormatPreserveEditorProps {
initialValue: string;
onSave: (value: string) => void;
className?: string;
}
const FormatPreserveEditor = ({
initialValue,
onSave,
className,
}: FormatPreserveEditorProps) => {
const [content, setContent] = useState(initialValue);
const [isEditing, setIsEditing] = useState(false);
const handleEdit = () => setIsEditing(true);
const handleSave = () => {
onSave(content);
setIsEditing(false);
};
return (
<div className="editor-container">
{isEditing ? (
<Textarea
className={cn(
'w-full h-full bg-transparent text-text-secondary',
className,
)}
value={content}
onChange={(e) => setContent(e.target.value)}
onBlur={handleSave}
autoSize={{ maxRows: 100 }}
autoFocus
/>
) : (
<pre className="text-text-secondary" onClick={handleEdit}>
{content}
</pre>
)}
</div>
);
};
export default FormatPreserveEditor;

View File

@ -0,0 +1,29 @@
import SvgIcon from '@/components/svg-icon';
import { Button } from '@/components/ui/button';
import { CircleAlert } from 'lucide-react';
import { useTranslation } from 'react-i18next';
import { useRerunDataflow } from '../../hooks';
interface RerunButtonProps {
className?: string;
}
const RerunButton = (props: RerunButtonProps) => {
const { t } = useTranslation();
const { loading } = useRerunDataflow();
const clickFunc = () => {
console.log('click rerun button');
};
return (
<div className="flex flex-col gap-2">
<div className="text-xs text-text-primary flex items-center gap-1">
<CircleAlert color="#d29e2d" strokeWidth={1} size={12} />
{t('dataflowParser.rerunFromCurrentStepTip')}
</div>
<Button onClick={clickFunc} disabled={loading}>
<SvgIcon name="rerun" width={16} />
{t('dataflowParser.rerunFromCurrentStep')}
</Button>
</div>
);
};
export default RerunButton;

View File

@ -0,0 +1,82 @@
import { CustomTimeline, TimelineNode } from '@/components/originui/timeline';
import {
CheckLine,
FilePlayIcon,
Grid3x2,
ListPlus,
PlayIcon,
} from 'lucide-react';
import { useMemo } from 'react';
export const TimelineNodeObj = {
begin: {
id: 1,
title: 'Begin',
icon: <PlayIcon size={13} />,
clickable: false,
},
parser: { id: 2, title: 'Parser', icon: <FilePlayIcon size={13} /> },
chunker: { id: 3, title: 'Chunker', icon: <Grid3x2 size={13} /> },
indexer: {
id: 4,
title: 'Indexer',
icon: <ListPlus size={13} />,
clickable: false,
},
complete: {
id: 5,
title: 'Complete',
icon: <CheckLine size={13} />,
clickable: false,
},
};
export interface TimelineDataFlowProps {
activeId: number | string;
activeFunc: (id: number | string) => void;
}
const TimelineDataFlow = ({ activeFunc, activeId }: TimelineDataFlowProps) => {
// const [activeStep, setActiveStep] = useState(2);
const timelineNodes: TimelineNode[] = useMemo(() => {
const nodes: TimelineNode[] = [];
Object.keys(TimelineNodeObj).forEach((key) => {
nodes.push({
...TimelineNodeObj[key as keyof typeof TimelineNodeObj],
className: 'w-32',
completed: false,
});
});
return nodes;
}, []);
const activeStep = useMemo(() => {
const index = timelineNodes.findIndex((node) => node.id === activeId);
return index > -1 ? index + 1 : 0;
}, [activeId, timelineNodes]);
const handleStepChange = (step: number, id: string | number) => {
// setActiveStep(step);
activeFunc?.(id);
console.log(step, id);
};
return (
<div className="">
<div>
<CustomTimeline
nodes={timelineNodes as TimelineNode[]}
activeStep={activeStep}
onStepChange={handleStepChange}
orientation="horizontal"
lineStyle="solid"
nodeSize={24}
activeStyle={{
nodeSize: 30,
iconColor: 'var(--accent-primary)',
textColor: 'var(--accent-primary)',
}}
/>
</div>
</div>
);
};
export default TimelineDataFlow;

View File

@ -0,0 +1,4 @@
export enum ChunkTextMode {
Full = 'full',
Ellipse = 'ellipse',
}

View File

@ -0,0 +1,185 @@
import message from '@/components/ui/message';
import {
useCreateChunk,
useDeleteChunk,
useSelectChunkList,
} from '@/hooks/chunk-hooks';
import { useSetModalState, useShowDeleteConfirm } from '@/hooks/common-hooks';
import { useGetKnowledgeSearchParams } from '@/hooks/route-hook';
import { IChunk } from '@/interfaces/database/knowledge';
import { buildChunkHighlights } from '@/utils/document-util';
import { useMutation, useQueryClient } from '@tanstack/react-query';
import { useCallback, useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { IHighlight } from 'react-pdf-highlighter';
import { ChunkTextMode } from './constant';
export const useHandleChunkCardClick = () => {
const [selectedChunkId, setSelectedChunkId] = useState<string>('');
const handleChunkCardClick = useCallback((chunkId: string) => {
setSelectedChunkId(chunkId);
}, []);
return { handleChunkCardClick, selectedChunkId };
};
export const useGetSelectedChunk = (selectedChunkId: string) => {
const data = useSelectChunkList();
return (
data?.data?.find((x) => x.chunk_id === selectedChunkId) ?? ({} as IChunk)
);
};
export const useGetChunkHighlights = (selectedChunkId: string) => {
const [size, setSize] = useState({ width: 849, height: 1200 });
const selectedChunk: IChunk = useGetSelectedChunk(selectedChunkId);
const highlights: IHighlight[] = useMemo(() => {
return buildChunkHighlights(selectedChunk, size);
}, [selectedChunk, size]);
const setWidthAndHeight = useCallback((width: number, height: number) => {
setSize((pre) => {
if (pre.height !== height || pre.width !== width) {
return { height, width };
}
return pre;
});
}, []);
return { highlights, setWidthAndHeight };
};
// Switch chunk text to be fully displayed or ellipse
export const useChangeChunkTextMode = () => {
const [textMode, setTextMode] = useState<ChunkTextMode>(ChunkTextMode.Full);
const changeChunkTextMode = useCallback((mode: ChunkTextMode) => {
setTextMode(mode);
}, []);
return { textMode, changeChunkTextMode };
};
export const useDeleteChunkByIds = (): {
removeChunk: (chunkIds: string[], documentId: string) => Promise<number>;
} => {
const { deleteChunk } = useDeleteChunk();
const showDeleteConfirm = useShowDeleteConfirm();
const removeChunk = useCallback(
(chunkIds: string[], documentId: string) => () => {
return deleteChunk({ chunkIds, doc_id: documentId });
},
[deleteChunk],
);
const onRemoveChunk = useCallback(
(chunkIds: string[], documentId: string): Promise<number> => {
return showDeleteConfirm({ onOk: removeChunk(chunkIds, documentId) });
},
[removeChunk, showDeleteConfirm],
);
return {
removeChunk: onRemoveChunk,
};
};
export const useUpdateChunk = () => {
const [chunkId, setChunkId] = useState<string | undefined>('');
const {
visible: chunkUpdatingVisible,
hideModal: hideChunkUpdatingModal,
showModal,
} = useSetModalState();
const { createChunk, loading } = useCreateChunk();
const { documentId } = useGetKnowledgeSearchParams();
const onChunkUpdatingOk = useCallback(
async (params: IChunk) => {
const code = await createChunk({
...params,
doc_id: documentId,
chunk_id: chunkId,
});
if (code === 0) {
hideChunkUpdatingModal();
}
},
[createChunk, hideChunkUpdatingModal, chunkId, documentId],
);
const handleShowChunkUpdatingModal = useCallback(
async (id?: string) => {
setChunkId(id);
showModal();
},
[showModal],
);
return {
chunkUpdatingLoading: loading,
onChunkUpdatingOk,
chunkUpdatingVisible,
hideChunkUpdatingModal,
showChunkUpdatingModal: handleShowChunkUpdatingModal,
chunkId,
documentId,
};
};
export const useFetchParserList = () => {
const [loading, setLoading] = useState(false);
return {
loading,
};
};
export const useRerunDataflow = () => {
const [loading, setLoading] = useState(false);
return {
loading,
};
};
export const useFetchPaserText = () => {
const initialText =
'第一行文本\n\t第二行缩进文本\n第三行 多个空格 第一行文本\n\t第二行缩进文本\n第三行 ' +
'多个空格第一行文本\n\t第二行缩进文本\n第三行 多个空格第一行文本\n\t第二行缩进文本\n第三行 ' +
'多个空格第一行文本\n\t第二行缩进文本\n第三行 多个空格第一行文本\n\t第二行缩进文本\n第三行 ' +
'多个空格第一行文本\n\t第二行缩进文本\n第三行 多个空格第一行文本\n\t第二行缩进文本\n第三行 ' +
'多个空格第一行文本\n\t第二行缩进文本\n第三行 多个空格第一行文本\n\t第二行缩进文本\n第三行 ' +
'多个空格第一行文本\n\t第二行缩进文本\n第三行 多个空格第一行文本\n\t第二行缩进文本\n第三行 ' +
'多个空格第一行文本\n\t第二行缩进文本\n第三行 多个空格第一行文本\n\t第二行缩进文本\n第三行 多个空格';
const [loading, setLoading] = useState(false);
const [data, setData] = useState<string>(initialText);
const { t } = useTranslation();
const queryClient = useQueryClient();
const {
// data,
// isPending: loading,
mutateAsync,
} = useMutation({
mutationKey: ['createChunk'],
mutationFn: async (payload: any) => {
// let service = kbService.create_chunk;
// if (payload.chunk_id) {
// service = kbService.set_chunk;
// }
// const { data } = await service(payload);
// if (data.code === 0) {
message.success(t('message.created'));
setTimeout(() => {
queryClient.invalidateQueries({ queryKey: ['fetchChunkList'] });
}, 1000); // Delay to ensure the list is updated
// }
// return data?.code;
},
});
return { data, loading, rerun: mutateAsync };
};

View File

@ -0,0 +1,96 @@
.chunkPage {
padding: 24px;
padding-top: 2px;
display: flex;
// height: calc(100vh - 112px);
height: 100vh;
flex-direction: column;
.filter {
margin: 10px 0;
display: flex;
height: 32px;
justify-content: space-between;
}
.pagePdfWrapper {
width: 60%;
}
.pageWrapper {
width: 100%;
}
.pageContent {
flex: 1;
width: 100%;
padding-right: 12px;
overflow-y: auto;
.spin {
min-height: 400px;
}
}
.documentPreview {
// width: 40%;
height: calc(100vh - 180px);
overflow: auto;
}
.chunkContainer {
display: flex;
height: calc(100vh - 332px);
}
.chunkOtherContainer {
width: 100%;
}
.pageFooter {
padding-top: 10px;
padding-right: 10px;
height: 32px;
}
}
.container {
height: 100px;
display: flex;
flex-direction: column;
justify-content: space-between;
.content {
display: flex;
justify-content: space-between;
.context {
flex: 1;
// width: 207px;
height: 88px;
overflow: hidden;
}
}
.footer {
height: 20px;
.text {
margin-left: 10px;
}
}
}
.card {
:global {
.ant-card-body {
padding: 10px;
margin: 0;
}
margin-bottom: 10px;
}
cursor: pointer;
}

View File

@ -0,0 +1,121 @@
import { useFetchNextChunkList } from '@/hooks/use-chunk-request';
import { useMemo, useState } from 'react';
import { useTranslation } from 'react-i18next';
import DocumentPreview from './components/document-preview';
import { useGetChunkHighlights, useHandleChunkCardClick } from './hooks';
import DocumentHeader from './components/document-preview/document-header';
import { PageHeader } from '@/components/page-header';
import {
Breadcrumb,
BreadcrumbItem,
BreadcrumbLink,
BreadcrumbList,
BreadcrumbPage,
BreadcrumbSeparator,
} from '@/components/ui/breadcrumb';
import {
QueryStringMap,
useNavigatePage,
} from '@/hooks/logic-hooks/navigate-hooks';
import { useFetchKnowledgeBaseConfiguration } from '@/hooks/use-knowledge-request';
import { ChunkerContainer } from './chunker';
import { useGetDocumentUrl } from './components/document-preview/hooks';
import TimelineDataFlow, { TimelineNodeObj } from './components/time-line';
import styles from './index.less';
import ParserContainer from './parser';
const Chunk = () => {
const {
data: { documentInfo },
} = useFetchNextChunkList();
const { selectedChunkId } = useHandleChunkCardClick();
const [activeStepId, setActiveStepId] = useState<number | string>(0);
const { data: dataset } = useFetchKnowledgeBaseConfiguration();
const { t } = useTranslation();
const { navigateToDataset, getQueryString, navigateToDatasetList } =
useNavigatePage();
const fileUrl = useGetDocumentUrl();
const { highlights, setWidthAndHeight } =
useGetChunkHighlights(selectedChunkId);
const fileType = useMemo(() => {
switch (documentInfo?.type) {
case 'doc':
return documentInfo?.name.split('.').pop() || 'doc';
case 'visual':
case 'docx':
case 'txt':
case 'md':
case 'pdf':
return documentInfo?.type;
}
return 'unknown';
}, [documentInfo]);
const handleStepChange = (id: number | string) => {
setActiveStepId(id);
};
return (
<>
<PageHeader>
<Breadcrumb>
<BreadcrumbList>
<BreadcrumbItem>
<BreadcrumbLink onClick={navigateToDatasetList}>
{t('knowledgeDetails.dataset')}
</BreadcrumbLink>
</BreadcrumbItem>
<BreadcrumbSeparator />
<BreadcrumbItem>
<BreadcrumbLink
onClick={navigateToDataset(
getQueryString(QueryStringMap.id) as string,
)}
>
{dataset.name}
</BreadcrumbLink>
</BreadcrumbItem>
<BreadcrumbSeparator />
<BreadcrumbItem>
<BreadcrumbPage>{documentInfo?.name}</BreadcrumbPage>
</BreadcrumbItem>
</BreadcrumbList>
</Breadcrumb>
</PageHeader>
<div className=" absolute ml-[50%] translate-x-[-50%] top-4 flex justify-center">
<TimelineDataFlow
activeFunc={handleStepChange}
activeId={activeStepId}
/>
</div>
<div className={styles.chunkPage}>
<div className="flex flex-none gap-8 border border-border mt-[26px] p-3 rounded-lg h-[calc(100vh-100px)]">
<div className="w-2/5">
<div className="h-[50px] flex flex-col justify-end pb-[5px]">
<DocumentHeader {...documentInfo} />
</div>
<section className={styles.documentPreview}>
<DocumentPreview
className={styles.documentPreview}
fileType={fileType}
highlights={highlights}
setWidthAndHeight={setWidthAndHeight}
url={fileUrl}
></DocumentPreview>
</section>
</div>
<div className="h-dvh border-r -mt-3"></div>
{activeStepId === TimelineNodeObj.chunker.id && <ChunkerContainer />}
{activeStepId === TimelineNodeObj.parser.id && <ParserContainer />}
</div>
</div>
</>
);
};
export default Chunk;

View File

@ -0,0 +1,58 @@
import Spotlight from '@/components/spotlight';
import { Spin } from '@/components/ui/spin';
import classNames from 'classnames';
import { useState } from 'react';
import { useTranslation } from 'react-i18next';
import FormatPreserveEditor from './components/parse-editer';
import RerunButton from './components/rerun-button';
import { useFetchParserList, useFetchPaserText } from './hooks';
const ParserContainer = () => {
const { data: initialValue, rerun: onSave } = useFetchPaserText();
const { t } = useTranslation();
const { loading } = useFetchParserList();
const [initialText, setInitialText] = useState(initialValue);
const [isChange, setIsChange] = useState(false);
const handleSave = (newContent: string) => {
console.log('保存内容:', newContent);
if (newContent !== initialText) {
setIsChange(true);
onSave(newContent);
} else {
setIsChange(false);
}
// Here, the API is called to send newContent to the backend
};
return (
<>
{isChange && (
<div className=" absolute top-2 right-6">
<RerunButton />
</div>
)}
<div className={classNames('flex flex-col w-3/5')}>
<Spin spinning={loading} className="" size="large">
<div className="h-[50px] flex flex-col justify-end pb-[5px]">
<div>
<h2 className="text-[16px]">
{t('dataflowParser.parseSummary')}
</h2>
<div className="text-[12px] text-text-secondary italic ">
{t('dataflowParser.parseSummaryTip')}
</div>
</div>
</div>
<div className=" border rounded-lg p-[20px] box-border h-[calc(100vh-180px)] overflow-auto scrollbar-none">
<FormatPreserveEditor
initialValue={initialText}
onSave={handleSave}
className="!h-[calc(100vh-220px)]"
/>
<Spotlight opcity={0.6} coverage={60} />
</div>
</Spin>
</div>
</>
);
};
export default ParserContainer;

View File

@ -0,0 +1,24 @@
export type FormListItem = {
frequency: number;
tag: string;
};
export function transformTagFeaturesArrayToObject(
list: Array<FormListItem> = [],
) {
return list.reduce<Record<string, number>>((pre, cur) => {
pre[cur.tag] = cur.frequency;
return pre;
}, {});
}
export function transformTagFeaturesObjectToArray(
object: Record<string, number> = {},
) {
return Object.keys(object).reduce<Array<FormListItem>>((pre, key) => {
pre.push({ frequency: object[key], tag: key });
return pre;
}, []);
}

View File

@ -0,0 +1,9 @@
export enum LogTabs {
FILE_LOGS = 'fileLogs',
DATASET_LOGS = 'datasetLogs',
}
export enum processingType {
knowledgeGraph = 'knowledgeGraph',
raptor = 'raptor',
}

View File

@ -0,0 +1,91 @@
import { FilterButton } from '@/components/list-filter-bar';
import {
CheckboxFormMultipleProps,
FilterPopover,
} from '@/components/list-filter-bar/filter-popover';
import { Button } from '@/components/ui/button';
import { SearchInput } from '@/components/ui/input';
import { cn } from '@/lib/utils';
import { ChangeEventHandler, useMemo } from 'react';
import { useTranslation } from 'react-i18next';
import { LogTabs } from './dataset-common';
interface IProps {
searchString?: string;
onSearchChange?: ChangeEventHandler<HTMLInputElement>;
active?: (typeof LogTabs)[keyof typeof LogTabs];
setActive?: (active: (typeof LogTabs)[keyof typeof LogTabs]) => void;
}
const DatasetFilter = (
props: IProps & Omit<CheckboxFormMultipleProps, 'setOpen'>,
) => {
const {
searchString,
onSearchChange,
value,
onChange,
filters,
onOpenChange,
active = LogTabs.FILE_LOGS,
setActive,
...rest
} = props;
const { t } = useTranslation();
const filterCount = useMemo(() => {
return typeof value === 'object' && value !== null
? Object.values(value).reduce((pre, cur) => {
return pre + cur.length;
}, 0)
: 0;
}, [value]);
return (
<div className="flex items-center justify-between mb-4">
<div className="flex space-x-2 bg-bg-card p-1 rounded-md">
<Button
className={cn(
'px-4 py-2 rounded-md hover:text-text-primary hover:bg-bg-base',
{
'bg-bg-base text-text-primary': active === LogTabs.FILE_LOGS,
'bg-transparent text-text-secondary ':
active !== LogTabs.FILE_LOGS,
},
)}
onClick={() => setActive?.(LogTabs.FILE_LOGS)}
>
{t('knowledgeDetails.fileLogs')}
</Button>
<Button
className={cn(
'px-4 py-2 rounded-md hover:text-text-primary hover:bg-bg-base',
{
'bg-bg-base text-text-primary': active === LogTabs.DATASET_LOGS,
'bg-transparent text-text-secondary ':
active !== LogTabs.DATASET_LOGS,
},
)}
onClick={() => setActive?.(LogTabs.DATASET_LOGS)}
>
{t('knowledgeDetails.datasetLogs')}
</Button>
</div>
<div className="flex items-center space-x-2">
<FilterPopover
value={value}
onChange={onChange}
filters={filters}
onOpenChange={onOpenChange}
>
<FilterButton count={filterCount}></FilterButton>
</FilterPopover>
<SearchInput
value={searchString}
onChange={onSearchChange}
className="w-32"
></SearchInput>
</div>
</div>
);
};
export { DatasetFilter };

View File

@ -0,0 +1,156 @@
import {
CircleQuestionMark,
Cpu,
FileChartLine,
HardDriveDownload,
} from 'lucide-react';
import { FC, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { LogTabs } from './dataset-common';
import { DatasetFilter } from './dataset-filter';
import FileLogsTable from './overview-table';
interface StatCardProps {
title: string;
value: number;
icon: JSX.Element;
children?: JSX.Element;
}
const StatCard: FC<StatCardProps> = ({ title, value, children, icon }) => {
return (
<div className="bg-bg-card p-4 rounded-lg border border-border flex flex-col gap-2">
<div className="flex items-center justify-between">
<h3 className="flex items-center gap-1 text-sm font-medium text-text-secondary">
{title}
<CircleQuestionMark size={12} />
</h3>
{icon}
</div>
<div className="text-2xl font-bold text-text-primary">{value}</div>
<div className="h-12 w-full flex items-center">
<div className="flex-1">{children}</div>
</div>
</div>
);
};
interface CardFooterProcessProps {
total: number;
completed: number;
success: number;
failed: number;
}
const CardFooterProcess: FC<CardFooterProcessProps> = ({
total,
completed,
success,
failed,
}) => {
const { t } = useTranslation();
const successPrecentage = (success / total) * 100;
const failedPrecentage = (failed / total) * 100;
return (
<div className="flex items-center flex-col gap-2">
<div className="flex justify-between w-full text-sm text-text-secondary">
<div className="flex items-center gap-2">
<div className="flex items-center gap-1">
{success}
<span>{t('knowledgeDetails.success')}</span>
</div>
<div className="flex items-center gap-1">
{failed}
<span>{t('knowledgeDetails.failed')}</span>
</div>
</div>
<div className="flex items-center gap-1">
{completed}
<span>{t('knowledgeDetails.completed')}</span>
</div>
</div>
<div className="w-full flex rounded-full h-3 bg-bg-card text-sm font-bold text-text-primary">
<div
className=" rounded-full h-3 bg-accent-primary"
style={{ width: successPrecentage + '%' }}
></div>
<div
className=" rounded-full h-3 bg-state-error"
style={{ width: failedPrecentage + '%' }}
></div>
</div>
</div>
);
};
const FileLogsPage: FC = () => {
const { t } = useTranslation();
const [active, setActive] = useState<(typeof LogTabs)[keyof typeof LogTabs]>(
LogTabs.FILE_LOGS,
);
const mockData = Array(30)
.fill(0)
.map((_, i) => ({
id: i === 0 ? '952734' : `14`,
fileName: 'PRD for DealBees 1.2 (1).txt',
source: 'GitHub',
pipeline: i === 0 ? 'data demo for...' : i === 1 ? 'test' : 'kikis demo',
startDate: '14/03/2025 14:53:39',
task: i === 0 ? 'chunck' : 'Parser',
status:
i === 0
? 'Success'
: i === 1
? 'Failed'
: i === 2
? 'Running'
: 'Pending',
}));
const pagination = {
current: 1,
pageSize: 30,
total: 100,
};
const changeActiveLogs = (active: (typeof LogTabs)[keyof typeof LogTabs]) => {
setActive(active);
};
const handlePaginationChange = (page: number, pageSize: number) => {
console.log('Pagination changed:', { page, pageSize });
};
return (
<div className="p-5 min-w-[880px] border-border border rounded-lg mr-5">
{/* Stats Cards */}
<div className="grid grid-cols-3 md:grid-cols-3 gap-4 mb-6">
<StatCard title="Total Files" value={2827} icon={<FileChartLine />}>
<div>+7% from last week</div>
</StatCard>
<StatCard title="Downloading" value={28} icon={<HardDriveDownload />}>
<CardFooterProcess
total={100}
success={8}
failed={2}
completed={15}
/>
</StatCard>
<StatCard title="Processing" value={156} icon={<Cpu />}>
<CardFooterProcess total={20} success={8} failed={2} completed={15} />
</StatCard>
</div>
{/* Tabs & Search */}
<DatasetFilter active={active} setActive={changeActiveLogs} />
{/* Table */}
<FileLogsTable
data={mockData}
pagination={pagination}
setPagination={handlePaginationChange}
pageCount={10}
active={active}
/>
</div>
);
};
export default FileLogsPage;

View File

@ -0,0 +1,385 @@
import FileStatusBadge from '@/components/file-status-badge';
import { FileIcon } from '@/components/icon-font';
import { RAGFlowAvatar } from '@/components/ragflow-avatar';
import SvgIcon from '@/components/svg-icon';
import { Button } from '@/components/ui/button';
import { RAGFlowPagination } from '@/components/ui/ragflow-pagination';
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow,
} from '@/components/ui/table';
import { useTranslate } from '@/hooks/common-hooks';
import { useNavigatePage } from '@/hooks/logic-hooks/navigate-hooks';
import ProcessLogModal from '@/pages/datasets/process-log-modal';
import {
ColumnDef,
ColumnFiltersState,
SortingState,
flexRender,
getCoreRowModel,
getFilteredRowModel,
getPaginationRowModel,
getSortedRowModel,
useReactTable,
} from '@tanstack/react-table';
import { TFunction } from 'i18next';
import { ClipboardList, Eye } from 'lucide-react';
import { Dispatch, FC, SetStateAction, useMemo, useState } from 'react';
import { LogTabs, processingType } from './dataset-common';
interface DocumentLog {
id: string;
fileName: string;
source: string;
pipeline: string;
startDate: string;
task: string;
status: 'Success' | 'Failed' | 'Running' | 'Pending';
}
interface FileLogsTableProps {
data: DocumentLog[];
pageCount: number;
pagination: {
current: number;
pageSize: number;
total: number;
};
setPagination: (pagination: { page: number; pageSize: number }) => void;
loading?: boolean;
active: (typeof LogTabs)[keyof typeof LogTabs];
}
export const getFileLogsTableColumns = (
t: TFunction<'translation', string>,
setIsModalVisible: Dispatch<SetStateAction<boolean>>,
navigateToDataflowResult: (
id: string,
knowledgeId?: string | undefined,
) => () => void,
) => {
// const { t } = useTranslate('knowledgeDetails');
const columns: ColumnDef<DocumentLog>[] = [
{
id: 'select',
header: ({ table }) => (
<input
type="checkbox"
checked={table.getIsAllRowsSelected()}
onChange={table.getToggleAllRowsSelectedHandler()}
className="rounded bg-gray-900 text-blue-500 focus:ring-blue-500"
/>
),
cell: ({ row }) => (
<input
type="checkbox"
checked={row.getIsSelected()}
onChange={row.getToggleSelectedHandler()}
className="rounded border-gray-600 bg-gray-900 text-blue-500 focus:ring-blue-500"
/>
),
},
{
accessorKey: 'id',
header: 'ID',
cell: ({ row }) => (
<div className="text-text-primary">{row.original.id}</div>
),
},
{
accessorKey: 'fileName',
header: t('fileName'),
cell: ({ row }) => (
<div
className="flex items-center gap-2 text-text-primary"
onClick={navigateToDataflowResult(
row.original.id,
row.original.kb_id,
)}
>
<FileIcon name={row.original.fileName}></FileIcon>
{row.original.fileName}
</div>
),
},
{
accessorKey: 'source',
header: t('source'),
cell: ({ row }) => (
<div className="text-text-primary">{row.original.source}</div>
),
},
{
accessorKey: 'pipeline',
header: t('dataPipeline'),
cell: ({ row }) => (
<div className="flex items-center gap-2 text-text-primary">
<RAGFlowAvatar
avatar={null}
name={row.original.pipeline}
className="size-4"
/>
{row.original.pipeline}
</div>
),
},
{
accessorKey: 'startDate',
header: t('startDate'),
cell: ({ row }) => (
<div className="text-text-primary">{row.original.startDate}</div>
),
},
{
accessorKey: 'task',
header: t('task'),
cell: ({ row }) => (
<div className="text-text-primary">{row.original.task}</div>
),
},
{
accessorKey: 'status',
header: t('status'),
cell: ({ row }) => <FileStatusBadge status={row.original.status} />,
},
{
id: 'operations',
header: t('operations'),
cell: ({ row }) => (
<div className="flex justify-start space-x-2">
<Button
variant="ghost"
size="sm"
className="p-1"
onClick={() => {
setIsModalVisible(true);
}}
>
<Eye />
</Button>
<Button
variant="ghost"
size="sm"
className="p-1"
onClick={navigateToDataflowResult(row.original.id)}
>
<ClipboardList />
</Button>
</div>
),
},
];
return columns;
};
export const getDatasetLogsTableColumns = (
t: TFunction<'translation', string>,
setIsModalVisible: Dispatch<SetStateAction<boolean>>,
) => {
// const { t } = useTranslate('knowledgeDetails');
const columns: ColumnDef<DocumentLog>[] = [
{
id: 'select',
header: ({ table }) => (
<input
type="checkbox"
checked={table.getIsAllRowsSelected()}
onChange={table.getToggleAllRowsSelectedHandler()}
className="rounded bg-gray-900 text-blue-500 focus:ring-blue-500"
/>
),
cell: ({ row }) => (
<input
type="checkbox"
checked={row.getIsSelected()}
onChange={row.getToggleSelectedHandler()}
className="rounded border-gray-600 bg-gray-900 text-blue-500 focus:ring-blue-500"
/>
),
},
{
accessorKey: 'id',
header: 'ID',
cell: ({ row }) => (
<div className="text-text-primary">{row.original.id}</div>
),
},
{
accessorKey: 'startDate',
header: t('startDate'),
cell: ({ row }) => (
<div className="text-text-primary">{row.original.startDate}</div>
),
},
{
accessorKey: 'processingType',
header: t('processingType'),
cell: ({ row }) => (
<div className="flex items-center gap-2 text-text-primary">
{processingType.knowledgeGraph === row.original.processingType && (
<SvgIcon name={`data-flow/knowledgegraph`} width={24}></SvgIcon>
)}
{processingType.raptor === row.original.processingType && (
<SvgIcon name={`data-flow/raptor`} width={24}></SvgIcon>
)}
{row.original.processingType}
</div>
),
},
{
accessorKey: 'status',
header: t('status'),
cell: ({ row }) => <FileStatusBadge status={row.original.status} />,
},
{
id: 'operations',
header: t('operations'),
cell: ({ row }) => (
<div className="flex justify-start space-x-2">
<Button
variant="ghost"
size="sm"
className="p-1"
onClick={() => {
setIsModalVisible(true);
}}
>
<Eye />
</Button>
</div>
),
},
];
return columns;
};
const FileLogsTable: FC<FileLogsTableProps> = ({
data,
pagination,
setPagination,
loading,
active = LogTabs.FILE_LOGS,
}) => {
const [sorting, setSorting] = useState<SortingState>([]);
const [columnFilters, setColumnFilters] = useState<ColumnFiltersState>([]);
const [rowSelection, setRowSelection] = useState({});
const { t } = useTranslate('knowledgeDetails');
const [isModalVisible, setIsModalVisible] = useState(false);
const { navigateToDataflowResult } = useNavigatePage();
const columns = useMemo(() => {
console.log('columns', active);
return active === LogTabs.FILE_LOGS
? getFileLogsTableColumns(t, setIsModalVisible, navigateToDataflowResult)
: getDatasetLogsTableColumns(t, setIsModalVisible);
}, [active, t]);
const currentPagination = useMemo(
() => ({
pageIndex: (pagination.current || 1) - 1,
pageSize: pagination.pageSize || 10,
}),
[pagination],
);
const table = useReactTable({
data,
columns,
manualPagination: true,
getCoreRowModel: getCoreRowModel(),
getPaginationRowModel: getPaginationRowModel(),
getSortedRowModel: getSortedRowModel(),
getFilteredRowModel: getFilteredRowModel(),
onSortingChange: setSorting,
onColumnFiltersChange: setColumnFilters,
onRowSelectionChange: setRowSelection,
state: {
sorting,
columnFilters,
rowSelection,
pagination: currentPagination,
},
pageCount: pagination.total
? Math.ceil(pagination.total / pagination.pageSize)
: 0,
});
const taskInfo = {
taskId: '#9527',
fileName: 'PRD for DealBees 1.2 (1).text',
fileSize: '2.4G',
source: 'Github',
task: 'Parse',
state: 'Running',
startTime: '14/03/2025 14:53:39',
duration: '800',
details:
'\n17:43:21 Task has been received.\n17:43:25 Page(1~100000001): Start to parse.\n17:43:25 Page(1~100000001): Start to tag for every chunk ...\n17:43:45 Page(1~100000001): Tagging 2 chunks completed in 18.99s\n17:43:45 Page(1~100000001): Generate 2 chunks\n17:43:55 Page(1~100000001): Embedding chunks (10.60s)\n17:43:55 Page(1~100000001): Indexing done (0.07s). Task done (33.97s)\n17:43:58 created task raptor\n17:43:58 Task has been received.\n17:44:36 Cluster one layer: 2 -> 1\n17:44:36 Indexing done (0.05s). Task done (37.88s)\n17:44:40 created task graphrag\n17:44:41 Task has been received.\n17:50:57 Entities extraction of chunk 0 1/3 done, 25 nodes, 26 edges, 14893 tokens.\n17:56:01 [ERROR][Exception]: Operation timed out after 7200 seconds and 1 attempts.',
};
return (
<div className="w-full h-[calc(100vh-350px)]">
<Table rootClassName="max-h-[calc(100vh-380px)]">
<TableHeader>
{table.getHeaderGroups().map((headerGroup) => (
<TableRow key={headerGroup.id}>
{headerGroup.headers.map((header) => (
<TableHead key={header.id}>
{flexRender(
header.column.columnDef.header,
header.getContext(),
)}
</TableHead>
))}
</TableRow>
))}
</TableHeader>
<TableBody className="relative">
{table.getRowModel().rows.length ? (
table.getRowModel().rows.map((row) => (
<TableRow
key={row.id}
data-state={row.getIsSelected() && 'selected'}
className="group"
>
{row.getVisibleCells().map((cell) => (
<TableCell
key={cell.id}
className={cell.column.columnDef.meta?.cellClassName}
>
{flexRender(cell.column.columnDef.cell, cell.getContext())}
</TableCell>
))}
</TableRow>
))
) : (
<TableRow>
<TableCell colSpan={columns.length} className="h-24 text-center">
No results.
</TableCell>
</TableRow>
)}
</TableBody>
</Table>
<div className="flex items-center justify-end py-4 absolute bottom-3 right-12">
<div className="space-x-2">
<RAGFlowPagination
{...{ current: pagination.current, pageSize: pagination.pageSize }}
total={pagination.total}
onChange={(page, pageSize) => setPagination({ page, pageSize })}
/>
</div>
</div>
<ProcessLogModal
visible={isModalVisible}
onCancel={() => setIsModalVisible(false)}
taskInfo={taskInfo}
/>
</div>
);
};
export default FileLogsTable;

View File

@ -0,0 +1,79 @@
import { RAGFlowAvatar } from '@/components/ragflow-avatar';
import { Button } from '@/components/ui/button';
import { Link, Route, Settings2, Unlink } from 'lucide-react';
import { useTranslation } from 'react-i18next';
interface DataPipelineItemProps {
name: string;
avatar?: string;
isDefault?: boolean;
linked?: boolean;
}
const DataPipelineItem = (props: DataPipelineItemProps) => {
const { t } = useTranslation();
const { name, avatar, isDefault, linked } = props;
return (
<div className="flex items-center justify-between gap-1 px-2 rounded-lg border">
<div className="flex items-center gap-1">
<RAGFlowAvatar avatar={avatar} name={name} className="size-4" />
<div>{name}</div>
{isDefault && (
<div className="text-xs bg-text-secondary text-bg-base px-2 py-1 rounded-md">
{t('knowledgeConfiguration.default')}
</div>
)}
</div>
<div className="flex gap-1 items-center">
<Button variant={'transparent'} className="border-none">
<Settings2 />
</Button>
<Button variant={'transparent'} className="border-none">
{linked ? <Link /> : <Unlink />}
</Button>
</div>
</div>
);
};
const LinkDataPipeline = () => {
const { t } = useTranslation();
const testNode = [
{
name: 'Data Pipeline 1',
avatar: 'https://avatars.githubusercontent.com/u/10656201?v=4',
isDefault: true,
linked: true,
},
{
name: 'Data Pipeline 2',
avatar: 'https://avatars.githubusercontent.com/u/10656201?v=4',
linked: false,
},
];
return (
<div className="flex flex-col gap-2">
<section className="flex flex-col">
<div className="flex items-center gap-1 text-text-primary text-sm">
<Route className="size-4" />
{t('knowledgeConfiguration.dataPipeline')}
</div>
<div className="flex justify-between items-center">
<div className="text-center text-xs text-text-secondary">
Manage data pipeline linkage with this dataset
</div>
<Button variant={'transparent'}>
<Link />
<span className="text-xs text-text-primary">
{t('knowledgeConfiguration.linkDataPipeline')}
</span>
</Button>
</div>
</section>
<section className="flex flex-col gap-2">
{testNode.map((item) => (
<DataPipelineItem key={item.name} {...item} />
))}
</section>
</div>
);
};
export default LinkDataPipeline;

View File

@ -0,0 +1,155 @@
import { RAGFlowAvatar } from '@/components/ragflow-avatar';
import { SliderInputFormField } from '@/components/slider-input-form-field';
import {
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { MultiSelect } from '@/components/ui/multi-select';
import { useFetchKnowledgeList } from '@/hooks/knowledge-hooks';
import { Flex, Form, InputNumber, Select, Slider, Space } from 'antd';
import DOMPurify from 'dompurify';
import { useFormContext, useWatch } from 'react-hook-form';
import { useTranslation } from 'react-i18next';
export const TagSetItem = () => {
const { t } = useTranslation();
const form = useFormContext();
const { list: knowledgeList } = useFetchKnowledgeList(true);
const knowledgeOptions = knowledgeList
.filter((x) => x.parser_id === 'tag')
.map((x) => ({
label: x.name,
value: x.id,
icon: () => (
<Space>
<RAGFlowAvatar
name={x.name}
avatar={x.avatar}
className="size-4"
></RAGFlowAvatar>
</Space>
),
}));
return (
<FormField
control={form.control}
name="parser_config.tag_kb_ids"
render={({ field }) => (
<FormItem className=" items-center space-y-0 ">
<div className="flex items-center">
<FormLabel
className="text-sm text-muted-foreground whitespace-nowrap w-1/4"
tooltip={
<div
dangerouslySetInnerHTML={{
__html: DOMPurify.sanitize(
t('knowledgeConfiguration.tagSetTip'),
),
}}
></div>
}
>
{t('knowledgeConfiguration.tagSet')}
</FormLabel>
<div className="w-3/4">
<FormControl>
<MultiSelect
options={knowledgeOptions}
onValueChange={field.onChange}
placeholder={t('chat.knowledgeBasesMessage')}
variant="inverted"
maxCount={10}
{...field}
/>
</FormControl>
</div>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
);
return (
<Form.Item
label={t('knowledgeConfiguration.tagSet')}
name={['parser_config', 'tag_kb_ids']}
tooltip={
<div
dangerouslySetInnerHTML={{
__html: DOMPurify.sanitize(t('knowledgeConfiguration.tagSetTip')),
}}
></div>
}
rules={[
{
message: t('chat.knowledgeBasesMessage'),
type: 'array',
},
]}
>
<Select
mode="multiple"
options={knowledgeOptions}
placeholder={t('chat.knowledgeBasesMessage')}
></Select>
</Form.Item>
);
};
export const TopNTagsItem = () => {
const { t } = useTranslation();
return (
<SliderInputFormField
name={'parser_config.topn_tags'}
label={t('knowledgeConfiguration.topnTags')}
max={10}
min={1}
defaultValue={3}
></SliderInputFormField>
);
return (
<Form.Item label={t('knowledgeConfiguration.topnTags')}>
<Flex gap={20} align="center">
<Flex flex={1}>
<Form.Item
name={['parser_config', 'topn_tags']}
noStyle
initialValue={3}
>
<Slider max={10} min={1} style={{ width: '100%' }} />
</Form.Item>
</Flex>
<Form.Item name={['parser_config', 'topn_tags']} noStyle>
<InputNumber max={10} min={1} />
</Form.Item>
</Flex>
</Form.Item>
);
};
export function TagItems() {
const form = useFormContext();
const ids: string[] = useWatch({
control: form.control,
name: 'parser_config.tag_kb_ids',
});
return (
<>
<TagSetItem></TagSetItem>
{Array.isArray(ids) && ids.length > 0 && <TopNTagsItem></TopNTagsItem>}
</>
);
}

View File

@ -0,0 +1,14 @@
import { FormContainerProps } from '@/components/form-container';
import { cn } from '@/lib/utils';
import { PropsWithChildren } from 'react';
export function ConfigurationFormContainer({
children,
className,
}: FormContainerProps) {
return <section className={cn('space-y-4', className)}>{children}</section>;
}
export function MainContainer({ children }: PropsWithChildren) {
return <section className="space-y-5">{children}</section>;
}

View File

@ -0,0 +1,183 @@
import {
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { Radio } from '@/components/ui/radio';
import { RAGFlowSelect } from '@/components/ui/select';
import { Switch } from '@/components/ui/switch';
import { useTranslate } from '@/hooks/common-hooks';
import { cn } from '@/lib/utils';
import { useFormContext } from 'react-hook-form';
import {
useHasParsedDocument,
useSelectChunkMethodList,
useSelectEmbeddingModelOptions,
} from '../hooks';
export function ChunkMethodItem() {
const { t } = useTranslate('knowledgeConfiguration');
const form = useFormContext();
// const handleChunkMethodSelectChange = useHandleChunkMethodSelectChange(form);
const parserList = useSelectChunkMethodList();
return (
<FormField
control={form.control}
name={'parser_id'}
render={({ field }) => (
<FormItem className=" items-center space-y-0 ">
<div className="flex items-center">
<FormLabel
required
tooltip={t('chunkMethodTip')}
className="text-sm text-muted-foreground whitespace-wrap w-1/4"
>
{t('chunkMethod')}
</FormLabel>
<div className="w-3/4 ">
<FormControl>
<RAGFlowSelect
{...field}
options={parserList}
placeholder={t('chunkMethodPlaceholder')}
// onChange={handleChunkMethodSelectChange}
/>
</FormControl>
</div>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
);
}
export function EmbeddingModelItem({ line = 1 }: { line?: 1 | 2 }) {
const { t } = useTranslate('knowledgeConfiguration');
const form = useFormContext();
const embeddingModelOptions = useSelectEmbeddingModelOptions();
const disabled = useHasParsedDocument();
return (
<FormField
control={form.control}
name={'embd_id'}
render={({ field }) => (
<FormItem className={cn(' items-center space-y-0 ')}>
<div
className={cn('flex', {
' items-center': line === 1,
'flex-col gap-1': line === 2,
})}
>
<FormLabel
required
tooltip={t('embeddingModelTip')}
className={cn('text-sm whitespace-wrap ', {
'w-1/4': line === 1,
})}
>
{t('embeddingModel')}
</FormLabel>
<div
className={cn('text-muted-foreground', { 'w-3/4': line === 1 })}
>
<FormControl>
<RAGFlowSelect
{...field}
options={embeddingModelOptions}
disabled={disabled}
placeholder={t('embeddingModelPlaceholder')}
/>
</FormControl>
</div>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
);
}
export function ParseTypeItem() {
const { t } = useTranslate('knowledgeConfiguration');
const form = useFormContext();
return (
<FormField
control={form.control}
name={'parseType'}
render={({ field }) => (
<FormItem className=" items-center space-y-0 ">
<div className="">
<FormLabel
tooltip={t('parseTypeTip')}
className="text-sm whitespace-wrap "
>
{t('parseType')}
</FormLabel>
<div className="text-muted-foreground">
<FormControl>
<Radio.Group {...field}>
<div className="w-3/4 flex gap-2 justify-between text-muted-foreground">
<Radio value={1}>{t('builtIn')}</Radio>
<Radio value={2}>{t('manualSetup')}</Radio>
</div>
</Radio.Group>
</FormControl>
</div>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
);
}
export function EnableAutoGenerateItem() {
const { t } = useTranslate('knowledgeConfiguration');
const form = useFormContext();
return (
<FormField
control={form.control}
name={'enableAutoGenerate'}
render={({ field }) => (
<FormItem className=" items-center space-y-0 ">
<div className="flex items-center">
<FormLabel
tooltip={t('enableAutoGenerateTip')}
className="text-sm whitespace-wrap w-1/4"
>
{t('enableAutoGenerate')}
</FormLabel>
<div className="text-muted-foreground w-3/4">
<FormControl>
<Switch
checked={field.value}
onCheckedChange={field.onChange}
/>
</FormControl>
</div>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
);
}

View File

@ -0,0 +1,73 @@
import { z } from 'zod';
export const formSchema = z.object({
name: z.string().min(1, {
message: 'Username must be at least 2 characters.',
}),
description: z.string().min(2, {
message: 'Username must be at least 2 characters.',
}),
// avatar: z.instanceof(File),
avatar: z.any().nullish(),
permission: z.string().optional(),
parser_id: z.string(),
embd_id: z.string(),
parser_config: z
.object({
layout_recognize: z.string(),
chunk_token_num: z.number(),
delimiter: z.string(),
auto_keywords: z.number().optional(),
auto_questions: z.number().optional(),
html4excel: z.boolean(),
tag_kb_ids: z.array(z.string()).nullish(),
topn_tags: z.number().optional(),
raptor: z
.object({
use_raptor: z.boolean().optional(),
prompt: z.string().optional(),
max_token: z.number().optional(),
threshold: z.number().optional(),
max_cluster: z.number().optional(),
random_seed: z.number().optional(),
})
.refine(
(data) => {
if (data.use_raptor && !data.prompt) {
return false;
}
return true;
},
{
message: 'Prompt is required',
path: ['prompt'],
},
),
graphrag: z
.object({
use_graphrag: z.boolean().optional(),
entity_types: z.array(z.string()).optional(),
method: z.string().optional(),
resolution: z.boolean().optional(),
community: z.boolean().optional(),
})
.refine(
(data) => {
if (
data.use_graphrag &&
(!data.entity_types || data.entity_types.length === 0)
) {
return false;
}
return true;
},
{
message: 'Please enter Entity types',
path: ['entity_types'],
},
),
})
.optional(),
pagerank: z.number(),
// icon: z.array(z.instanceof(File)),
});

View File

@ -0,0 +1,92 @@
import { AvatarUpload } from '@/components/avatar-upload';
import {
FormControl,
FormField,
FormItem,
FormLabel,
FormMessage,
} from '@/components/ui/form';
import { Input } from '@/components/ui/input';
import { useFormContext } from 'react-hook-form';
import { useTranslation } from 'react-i18next';
import { EmbeddingModelItem } from './configuration/common-item';
import { PermissionFormField } from './permission-form-field';
export function GeneralForm() {
const form = useFormContext();
const { t } = useTranslation();
return (
<>
<FormField
control={form.control}
name="name"
render={({ field }) => (
<FormItem className="items-center space-y-0">
<div className="flex">
<FormLabel className="text-sm text-muted-foreground whitespace-nowrap w-1/4">
<span className="text-red-600">*</span>
{t('common.name')}
</FormLabel>
<FormControl className="w-3/4">
<Input {...field}></Input>
</FormControl>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
<FormField
control={form.control}
name="avatar"
render={({ field }) => (
<FormItem className="items-center space-y-0">
<div className="flex">
<FormLabel className="text-sm text-muted-foreground whitespace-nowrap w-1/4">
{t('setting.avatar')}
</FormLabel>
<FormControl className="w-3/4">
<AvatarUpload {...field}></AvatarUpload>
</FormControl>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
)}
/>
<FormField
control={form.control}
name="description"
render={({ field }) => {
// null initialize empty string
if (typeof field.value === 'object' && !field.value) {
form.setValue('description', ' ');
}
return (
<FormItem className="items-center space-y-0">
<div className="flex">
<FormLabel className="text-sm text-muted-foreground whitespace-nowrap w-1/4">
{t('flow.description')}
</FormLabel>
<FormControl className="w-3/4">
<Input {...field}></Input>
</FormControl>
</div>
<div className="flex pt-1">
<div className="w-1/4"></div>
<FormMessage />
</div>
</FormItem>
);
}}
/>
<PermissionFormField></PermissionFormField>
<EmbeddingModelItem></EmbeddingModelItem>
</>
);
}

Some files were not shown because too many files have changed in this diff Show More