pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

Initial draft of the begin component reference (#4272)

Browse Source 
### What problem does this PR solve?

### Type of change


- [x] Documentation Update
This commit is contained in:
writinwaters
2024-12-27 17:07:31 +08:00
committed by GitHub
parent 7773afa561
commit 8cdf10148d
11 changed files with 86 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/guides/agent/_category_.json Normal file
View File

@ -0,0 +1,8 @@
{
"label": "Agents",
"position": 3,
"link": {
"type": "generated-index",
"description": "RAGFlow v0.8.0 introduces an agent mechanism, featuring a no-code workflow editor on the front end and a comprehensive graph-based task orchestration framework on the backend."
}
}

84
docs/guides/agent/agent_introduction.md Normal file
View File

@ -0,0 +1,84 @@
---
sidebar_position: 1
slug: /agent_introduction
---
# Introduction to agents
Key concepts, basic operations, a quick view of the agent editor.
---
## Key concepts
Agents and RAG are complementary techniques, each enhancing the other’s capabilities in business applications. RAGFlow v0.8.0 introduces an agent mechanism, featuring a no-code workflow editor on the front end and a comprehensive graph-based task orchestration framework on the back end. This mechanism is built on top of RAGFlow's existing RAG solutions and aims to orchestrate search technologies such as query intent classification, conversation leading, and query rewriting to:
- Provide higher retrievals and,
- Accommodate more complex scenarios.
## Create an agent
:::tip NOTE
Before proceeding, ensure that:
1. You have properly set the LLM to use. See the guides on [Configure your API key](../llm_api_key_setup.md) or [Deploy a local LLM](../deploy_local_llm.mdx) for more information.
2. You have a knowledge base configured and the corresponding files properly parsed. See the guide on [Configure a knowledge base](../configure_knowledge_base.md) for more information.
:::
Click the **Agent** tab in the middle top of the page to show the **Agent** page. As shown in the screenshot below, the cards on this page represent the created agents, which you can continue to edit.
![agent_mainpage](https://github.com/user-attachments/assets/5c0bb123-8f4e-42ea-b250-43f640dc6814)
We also provide templates catered to different business scenarios. You can either generate your agent from one of our agent templates or create one from scratch:
1. Click **+ Create agent** to show the **agent template** page:
![agent_templates](https://github.com/user-attachments/assets/73bd476c-4bab-4c8c-82f8-6b00fb2cd044)
2. To create an agent from scratch, click the **Blank** card. Alternatively, to create an agent from one of our templates, hover over the desired card, such as **General-purpose chatbot**, click **Use this template**, name your agent in the pop-up dialogue, and click **OK** to confirm.
*You are now taken to the **no-code workflow editor** page. The left panel lists the components (operators): Above the dividing line are the RAG-specific components; below the line are tools. We are still working to expand the component list.*
![workflow_editor](https://github.com/user-attachments/assets/47b4d5ce-b35a-4d6b-b483-ba495a75a65d)
4. General speaking, now you can do the following:
- Drag and drop a desired component to your workflow,
- Select the knowledge base to use,
- Update settings of specific components,
- Update LLM settings
- Sets the input and output for a specific component, and more.
5. Click **Save** to apply changes to your agent and **Run** to test it.
## Components
Please review the flowing description of the RAG-specific components before you proceed:
| Component | Description |
| -------------- | ------------------------------------------------------------ |
| **Retrieval** | A component that retrieves information from specified knowledge bases and returns 'Empty response' if no information is found. Ensure the correct knowledge bases are selected. |
| **Generate** | A component that prompts the LLM to generate responses. You must ensure the prompt is set correctly. |
| **Interact** | A component that serves as the interface between human and the bot, receiving user inputs and displaying the agent's responses. |
| **Categorize** | A component that uses the LLM to classify user inputs into predefined categories. Ensure you specify the name, description, and examples for each category, along with the corresponding next component. |
| **Message** | A component that sends out a static message. If multiple messages are supplied, it randomly selects one to send. Ensure its downstream is **Interact**, the interface component. |
| **Relevant** | A component that uses the LLM to assess whether the upstream output is relevant to the user's latest query. Ensure you specify the next component for each judge result. |
| **Rewrite** | A component that refines a user query if it fails to retrieve relevant information from the knowledge base. It repeats this process until the predefined looping upper limit is reached. Ensure its upstream is **Relevant** and downstream is **Retrieval**. |
| **Keyword** | A component that extracts keywords from a user query, with TopN specifying the number of keywords to extract. |
:::caution NOTE
- Ensure **Rewrite**'s upstream component is **Relevant** and downstream component is **Retrieval**.
- Ensure the downstream component of **Message** is **Interact**.
- The downstream component of **Begin** is always **Interact**.
:::
## Basic operations
| Operation | Description |
| ------------------------- | ------------------------------------------------------------ |
| Add a component | Drag and drop the desired component from the left panel onto the canvas. |
| Delete a component | On the canvas, hover over the three dots (...) of the component to display the delete option, then select it to remove the component. |
| Copy a component | On the canvas, hover over the three dots (...) of the component to display the copy option, then select it to make a copy the component. |
| Update component settings | On the canvas, click the desired component to display the component settings. |

101
docs/guides/agent/general_purpose_chatbot.md Normal file
View File

@ -0,0 +1,101 @@
---
sidebar_position: 2
slug: /general_purpose_chatbot
---
# Create a general-purpose chatbot
Chatbot is one of the most common AI scenarios. However, effectively understanding user queries and responding appropriately remains a challenge. RAGFlow's general-purpose chatbot agent is our attempt to tackle this longstanding issue.
This chatbot closely resembles the chatbot introduced in [Start an AI chat](../start_chat.md), but with a key difference - it introduces a reflective mechanism that allows it to improve the retrieval from the target knowledge bases by rewriting the user's query.
This document provides guides on creating such a chatbot using our chatbot template.
## Prerequisites
1. Ensure you have properly set the LLM to use. See the guides on [Configure your API key](../llm_api_key_setup.md) or [Deploy a local LLM](../deploy_local_llm.mdx) for more information.
2. Ensure you have a knowledge base configured and the corresponding files properly parsed. See the guide on [Configure a knowledge base](../configure_knowledge_base.md) for more information.
3. Make sure you have read the [Introduction to Agentic RAG](./agent_introduction.md).
## Create a chatbot agent from template
To create a general-purpose chatbot agent using our template:
1. Click the **Agent** tab in the middle top of the page to show the **Agent** page.
2. Click **+ Create agent** on the top right of the page to show the **agent template** page.
3. On the **agent template** page, hover over the card on **General-purpose chatbot** and click **Use this template**.
*You are now directed to the **no-code workflow editor** page.*
![workflow_editor](https://github.com/user-attachments/assets/52e7dc62-4bf5-4fbb-ab73-4a6e252065f0)
:::tip NOTE
RAGFlow's no-code editor spares you the trouble of coding, making agent development effortless.
:::
## Understand each component in the template
Here’s a breakdown of each component and its role and requirements in the chatbot template:
- **Begin**
- Function: Sets an opening greeting for users.
- Purpose: Establishes a welcoming atmosphere and prepares the user for interaction.
- **Interact**
- Function: Serves as the interface between human and the bot.
- Role: Acts as the downstream component of **Begin**.
- **Retrieval**
- Function: Retrieves information from specified knowledge base(s).
- Requirement: Must have `knowledgebases` set up to function.
- **Relevant**
- Function: Assesses the relevance of the retrieved information from the **Retrieval** component to the user query.
- Process:
- If relevant, it directs the data to the **Generate** component for final response generation.
- Otherwise, it triggers the **Rewrite** component to refine the user query and redo the retrival process.
- **Generate**
- Function: Prompts the LLM to generate responses based on the retrieved information.
- Note: The prompt settings allow you to control the way in which the LLM generates responses. Be sure to review the prompts and make necessary changes.
- **Rewrite**:
- Function: Refines a user query when no relevant information from the knowledge base is retrieved.
- Usage: Often used in conjunction with **Relevant** and **Retrieval** to create a reflective/feedback loop.
## Configure your chatbot agent
1. Click **Begin** to set an opening greeting:
![opener](https://github.com/user-attachments/assets/4416bc16-2a84-4f24-a19b-6dc8b1de0908)
2. Click **Retrieval** to select the right knowledge base(s) and make any necessary adjustments:
![setting_knowledge_bases](https://github.com/user-attachments/assets/5f694820-5651-45bc-afd6-cf580ca0228d)
3. Click **Generate** to configure the LLM's summarization behavior:
3.1. Confirm the model.
3.2. Review the prompt settings. If there are variables, ensure they match the correct component IDs:
![prompt_settings](https://github.com/user-attachments/assets/19e94ea7-7f62-4b73-b526-32fcfa62f1e9)
4. Click **Relevant** to review or change its settings:
*You may retain the current settings, but feel free to experiment with changes to understand how the agent operates.*
![relevant_settings](https://github.com/user-attachments/assets/9ff7fdd8-7a69-4ee2-bfba-c7fb8029150f)
5. Click **Rewrite** to select a different model for query rewriting or update the maximum loop times for query rewriting:
![choose_model](https://github.com/user-attachments/assets/2bac1d6c-c4f1-42ac-997b-102858c3f550)
![loop_time](https://github.com/user-attachments/assets/09a4ce34-7aac-496f-aa59-d8aa33bf0b1f)
:::danger NOTE
Increasing the maximum loop times may significantly extend the time required to receive the final response.
:::
1. Update your workflow where you see necessary.
2. Click to **Save** to apply your changes.
*Your agent appears as one of the agent cards on the **Agent** page.*
## Test your chatbot agent
1. Find your chatbot agent on the **Agent** page:
![find_chatbot](https://github.com/user-attachments/assets/6e6382c6-9a86-4190-9fdd-e363b7f64ba9)
2. Experiment with your questions to verify if this chatbot functions as intended:
![test_chatbot](https://github.com/user-attachments/assets/c074d3bd-4c39-4b05-a68b-1fd361f256b3)

416
docs/guides/agent/text2sql_agent.md Normal file
View File

@ -0,0 +1,416 @@
---
sidebar_position: 3
slug: /text2sql_agent
---
# Create a Text2SQL agent
Build a Text2SQL agent leverging RAGFlow's RAG capabilities. Contributed by @TeslaZY.
## Scenario
The Text2SQL agent is designed to bridge the gap between Natural Language Processing (NLP) and Structured Query Language (SQL). Its key advantages are as follows:
- **Assisting non-technical users with SQL**: Not all users have a background in SQL or understand the structure of the tables involved in queries. With a Text2SQL agent, users can pose questions or request data in natural language without needing an in-depth knowledge of the database structure or SQL syntax.
- **Enhancing SQL development efficiency**: For those familiar with SQL, the Text2SQL agent streamlines the process by enabling users to construct complex queries quickly, without the need to code each part manually.
- **Minimizing errors**: Manually writing SQL queries can be error-prone, particularly for complex queries or for users not well-versed in the database structure. The Text2SQL agent can interpret natural language instructions and generate accurate SQL queries, thereby reducing potential syntax and logic errors.
- **Boosting data analysis capabilities**: In business intelligence and data analysis, swiftly gaining insights from data is critical. The Text2SQL agent facilitates extracting valuable information from databases more directly and conveniently, thus aiding in accelerating decision-making.
- **Automation and integration**: The Text2SQL agent can be integrated into larger systems to support automated workflows, such as automatic report generation and data monitoring. It can also integrate seamlessly with other services and technologies, offering richer application possibilities.
- **Support for multiple languages and varied expressions**: People can express the same idea in numerous ways. An effective Text2SQL system should be capable of understanding diverse expressions and accurately converting them into SQL queries.
In summary, the Text2SQL agent seeks to make database queries more intuitive and user-friendly while ensuring efficiency and accuracy. It caters to a broad spectrum of users, from completely non-technical individuals to seasoned data analysts and developers.
However, traditional Text2SQL solutions often require model fine-tuning, which can substantially escalate deployment and maintenance costs when implemented in enterprise environments alongside RAG or Agent components. RAGFlow’s RAG-based Text2SQL utilizes an existing (connected) large language model (LLM), allowing for seamless integration with other RAG/Agent components without the necessity for additional fine-tuned models.
## Recipe
A list of components required:
- Begin
- Interface
- Retrieval
- Generate
- ExeSQL
## Procedure
### Preparation of Data
#### Database Environment
Mysql-8.0.39
#### Database Table Creation Statements
```sql
SET NAMES utf8mb4;
-- ----------------------------
-- Table structure for Customers
-- ----------------------------
DROP TABLE IF EXISTS `Customers`;
CREATE TABLE `Customers` (
`CustomerID` int NOT NULL AUTO_INCREMENT,
`UserName` varchar(50) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`Email` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`PhoneNumber` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`CustomerID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- ----------------------------
-- Table structure for Products
-- ----------------------------
DROP TABLE IF EXISTS `Products`;
CREATE TABLE `Products` (
`ProductID` int NOT NULL AUTO_INCREMENT,
`ProductName` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`Description` text COLLATE utf8mb4_unicode_ci,
`Price` decimal(10,2) DEFAULT NULL,
`StockQuantity` int DEFAULT NULL,
PRIMARY KEY (`ProductID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- ----------------------------
-- Table structure for Orders
-- ----------------------------
DROP TABLE IF EXISTS `Orders`;
CREATE TABLE `Orders` (
`OrderID` int NOT NULL AUTO_INCREMENT,
`CustomerID` int DEFAULT NULL,
`OrderDate` date DEFAULT NULL,
`TotalPrice` decimal(10,2) DEFAULT NULL,
PRIMARY KEY (`OrderID`),
KEY `CustomerID` (`CustomerID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
-- ----------------------------
-- Table structure for OrderDetails
-- ----------------------------
DROP TABLE IF EXISTS `OrderDetails`;
CREATE TABLE `OrderDetails` (
`OrderDetailID` int NOT NULL AUTO_INCREMENT,
`OrderID` int DEFAULT NULL,
`ProductID` int DEFAULT NULL,
`UnitPrice` decimal(10,2) DEFAULT NULL,
`Quantity` int DEFAULT NULL,
`TotalPrice` decimal(10,2) DEFAULT NULL,
PRIMARY KEY (`OrderDetailID`),
KEY `OrderID` (`OrderID`),
KEY `ProductID` (`ProductID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
```
#### Generate Test Data
```sql
START TRANSACTION;
INSERT INTO Customers (UserName, Email, PhoneNumber) VALUES
('Alice', 'alice@example.com', '123456789'),
('Bob', 'bob@example.com', '987654321'),
('Charlie', 'charlie@example.com', '112233445'),
('Diana', 'diana@example.com', '555666777'),
('Eve', 'eve@example.com', '999888777'),
('Frank', 'frank@example.com', '123123123'),
('Grace', 'grace@example.com', '456456456'),
('Hugo', 'hugo@example.com', '789789789'),
('Ivy', 'ivy@example.com', '321321321'),
('Jack', 'jack@example.com', '654654654');
INSERT INTO Products (ProductName, Description, Price, StockQuantity) VALUES
('Laptop', 'High performance laptop', 1200.00, 50),
('Smartphone', 'Latest model smartphone', 800.00, 100),
('Tablet', 'Portable tablet device', 300.00, 75),
('Headphones', 'Noise-cancelling headphones', 150.00, 200),
('Camera', 'Professional camera', 600.00, 30),
('Monitor', '24-inch Full HD monitor', 200.00, 45),
('Keyboard', 'Mechanical keyboard', 100.00, 150),
('Mouse', 'Ergonomic gaming mouse', 50.00, 250),
('Speaker', 'Wireless Bluetooth speaker', 80.00, 120),
('Router', 'Wi-Fi router with high speed', 120.00, 90);
INSERT INTO Orders (CustomerID, OrderDate, TotalPrice) VALUES
(1, '2024-01-15', 0),
(2, '2024-02-01', 0),
(3, '2024-03-05', 0),
(4, '2024-04-10', 0),
(5, '2024-05-15', 0),
(6, '2024-06-20', 0),
(7, '2024-07-25', 0),
(8, '2024-08-30', 0),
(9, '2024-09-05', 0),
(10, '2024-10-10', 0),
(1, '2024-11-15', 0),
(2, '2024-12-01', 0),
(3, '2024-01-05', 0),
(4, '2024-02-10', 0),
(5, '2024-03-15', 0),
(6, '2024-04-20', 0),
(7, '2024-05-25', 0),
(8, '2024-06-30', 0),
(9, '2024-07-05', 0),
(10, '2024-08-10', 0);
INSERT INTO OrderDetails (OrderID, ProductID, UnitPrice, Quantity, TotalPrice) VALUES
(1, 1, (SELECT Price FROM Products WHERE ProductID = 1), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 1)),
(1, 2, (SELECT Price FROM Products WHERE ProductID = 2), 1, (SELECT Price FROM Products WHERE ProductID = 2)),
(2, 3, (SELECT Price FROM Products WHERE ProductID = 3), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 3)),
(2, 4, (SELECT Price FROM Products WHERE ProductID = 4), 1, (SELECT Price FROM Products WHERE ProductID = 4)),
(3, 5, (SELECT Price FROM Products WHERE ProductID = 5), 1, (SELECT Price FROM Products WHERE ProductID = 5)),
(3, 6, (SELECT Price FROM Products WHERE ProductID = 6), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 6)),
(4, 7, (SELECT Price FROM Products WHERE ProductID = 7), 5, (SELECT Price * 5 FROM Products WHERE ProductID = 7)),
(5, 8, (SELECT Price FROM Products WHERE ProductID = 8), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 8)),
(5, 9, (SELECT Price FROM Products WHERE ProductID = 9), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 9)),
(6, 10, (SELECT Price FROM Products WHERE ProductID = 10), 4, (SELECT Price * 4 FROM Products WHERE ProductID = 10)),
(7, 2, (SELECT Price FROM Products WHERE ProductID = 2), 4, (SELECT Price * 4 FROM Products WHERE ProductID = 2)),
(7, 8, (SELECT Price FROM Products WHERE ProductID = 8), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 8)),
(8, 1, (SELECT Price FROM Products WHERE ProductID = 1), 1, (SELECT Price FROM Products WHERE ProductID = 1)),
(8, 9, (SELECT Price FROM Products WHERE ProductID = 9), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 9)),
(8, 10, (SELECT Price FROM Products WHERE ProductID = 10), 5, (SELECT Price * 5 FROM Products WHERE ProductID = 10)),
(9, 3, (SELECT Price FROM Products WHERE ProductID = 3), 5, (SELECT Price * 5 FROM Products WHERE ProductID = 3)),
(9, 6, (SELECT Price FROM Products WHERE ProductID = 6), 1, (SELECT Price FROM Products WHERE ProductID = 6)),
(10, 4, (SELECT Price FROM Products WHERE ProductID = 4), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 4)),
(10, 7, (SELECT Price FROM Products WHERE ProductID = 7), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 7)),
(11, 5, (SELECT Price FROM Products WHERE ProductID = 5), 1, (SELECT Price FROM Products WHERE ProductID = 5)),
(11, 10, (SELECT Price FROM Products WHERE ProductID = 10), 4, (SELECT Price * 4 FROM Products WHERE ProductID = 10)),
(12, 1, (SELECT Price FROM Products WHERE ProductID = 1), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 1)),
(12, 8, (SELECT Price FROM Products WHERE ProductID = 8), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 8)),
(13, 2, (SELECT Price FROM Products WHERE ProductID = 2), 1, (SELECT Price FROM Products WHERE ProductID = 2)),
(13, 9, (SELECT Price FROM Products WHERE ProductID = 9), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 9)),
(14, 3, (SELECT Price FROM Products WHERE ProductID = 3), 4, (SELECT Price * 4 FROM Products WHERE ProductID = 3)),
(14, 6, (SELECT Price FROM Products WHERE ProductID = 6), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 6)),
(15, 4, (SELECT Price FROM Products WHERE ProductID = 4), 5, (SELECT Price * 5 FROM Products WHERE ProductID = 4)),
(15, 7, (SELECT Price FROM Products WHERE ProductID = 7), 1, (SELECT Price FROM Products WHERE ProductID = 7)),
(16, 5, (SELECT Price FROM Products WHERE ProductID = 5), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 5)),
(16, 10, (SELECT Price FROM Products WHERE ProductID = 10), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 10)),
(17, 1, (SELECT Price FROM Products WHERE ProductID = 1), 4, (SELECT Price * 4 FROM Products WHERE ProductID = 1)),
(17, 8, (SELECT Price FROM Products WHERE ProductID = 8), 1, (SELECT Price FROM Products WHERE ProductID = 8)),
(18, 2, (SELECT Price FROM Products WHERE ProductID = 2), 5, (SELECT Price * 5 FROM Products WHERE ProductID = 2)),
(18, 9, (SELECT Price FROM Products WHERE ProductID = 9), 2, (SELECT Price * 2 FROM Products WHERE ProductID = 9)),
(19, 3, (SELECT Price FROM Products WHERE ProductID = 3), 3, (SELECT Price * 3 FROM Products WHERE ProductID = 3)),
(19, 6, (SELECT Price FROM Products WHERE ProductID = 6), 4, (SELECT Price * 4 FROM Products WHERE ProductID = 6)),
(20, 4, (SELECT Price FROM Products WHERE ProductID = 4), 1, (SELECT Price FROM Products WHERE ProductID = 4)),
(20, 7, (SELECT Price FROM Products WHERE ProductID = 7), 5, (SELECT Price * 5 FROM Products WHERE ProductID = 7));
-- Update Orders Table's TotalPrice
UPDATE Orders o
JOIN (
SELECT OrderID, SUM(TotalPrice) as order_total
FROM OrderDetails
GROUP BY OrderID
) od ON o.OrderID = od.OrderID
SET o.TotalPrice = od.order_total;
COMMIT;
```
### Configure Knowledge Base
For RAGFlow’s RAG-based Text2SQL, the following knowledge bases are typically required:
- **DDL**: Database table creation statements.
- **DB_Description**: Detailed descriptions of tables and columns.
- **Q->SQL**: Natural language query descriptions along with corresponding SQL query examples (Question-Answer pairs).
However, in specialized query scenarios, user queries might include abbreviations or synonyms for domain-specific terms. If a user references a synonym for a domain-specific term, the system may fail to generate the correct SQL query. Therefore, it is advisable to incorporate a thesaurus for synonyms to assist the Agent in generating more accurate SQL queries.
- **TextSQL_Thesaurus**: A thesaurus covering domain-specific terms and their synonyms.
#### Configure DDL Knowledge Base
1. The content of the DDL text is as follows:
```sql
CREATE TABLE Customers (
CustomerID int NOT NULL AUTO_INCREMENT,
UserName varchar(50) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Email varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PhoneNumber varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (CustomerID)
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
CREATE TABLE Products (
ProductID int NOT NULL AUTO_INCREMENT,
ProductName varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Description text COLLATE utf8mb4_unicode_ci,
Price decimal(10,2) DEFAULT NULL,
StockQuantity int DEFAULT NULL,
PRIMARY KEY (ProductID)
) ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
CREATE TABLE Orders (
OrderID int NOT NULL AUTO_INCREMENT,
CustomerID int DEFAULT NULL,
OrderDate date DEFAULT NULL,
TotalPrice decimal(10,2) DEFAULT NULL,
PRIMARY KEY (OrderID),
KEY CustomerID (CustomerID)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
CREATE TABLE OrderDetails (
OrderDetailID int NOT NULL AUTO_INCREMENT,
OrderID int DEFAULT NULL,
ProductID int DEFAULT NULL,
UnitPrice decimal(10,2) DEFAULT NULL,
Quantity int DEFAULT NULL,
TotalPrice decimal(10,2) DEFAULT NULL,
PRIMARY KEY (OrderDetailID),
KEY OrderID (OrderID),
KEY ProductID (ProductID)
) ENGINE=InnoDB AUTO_INCREMENT=40 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
```
2. Set the chunk data for the DLL knowledge base
![DDL knowledge base](https://github.com/user-attachments/assets/2c073e1b-8fdd-443e-98ca-4fd36f9d93e0)
#### Configure DB_Description Knowledge Base
1. the content of the DB_Description text is as follows:
2.
```markdown
### Customers (Customer Information Table)
The Customers table records detailed information about different customers in the online store. Here is the meaning of each field within this table:
- CustomerID: A unique identifier for a customer, auto-incremented.
- UserName: The name used by the customer for logging into the online store or displayed on the site.
- Email: The email address of the customer, which can be used for account verification, password recovery, and order updates.
- PhoneNumber: The phone number of the customer, useful for contact purposes such as delivery notifications or customer service.
### Products (Product Information Table)
The Products table contains information about the products offered by the online store. Each field within this table represents:
- ProductID: A unique identifier for a product, auto-incremented.
- ProductName: The name of the product, such as laptop, smartphone, nounch, etc.
- Description: Detailed information about the product.
- Price: The selling price of the product, stored as a decimal value to accommodate currency formatting.
- StockQuantity: The quantity of the product available in stock.
### Orders (Order Information Table)
The Orders table tracks orders placed by customers. This table includes fields that denote:
- OrderID: A unique identifier for an order, auto-incremented.
- CustomerID: A foreign key that references the CustomerID in the Customers table, indicating which customer placed the order.
- OrderDate: The date when the order was placed.
- TotalPrice: The total price of all items in the order, calculated at the time of purchase.
### OrderDetails (Order Details Table)
The OrderDetails table provides detailed information about each item in an order. Fields within this table include:
- OrderDetailID: A unique identifier for each line item in an order, auto-incremented.
- OrderID: A foreign key that references the OrderID in the Orders table, linking the detail to a specific order.
- ProductID: A foreign key that references the ProductID in the Products table, specifying which product was ordered.
- UnitPrice: The price per unit of the product at the time of order.
- Quantity: The number of units of the product ordered.
- TotalPrice: The total price for this particular item in the order, calculated as UnitPrice * Quantity.
```
2. set the chunk data for the DB_Description knowledge base
![DB_Description knowledge base](https://github.com/user-attachments/assets/0e3f1cad-dd67-4d7c-ae2d-b31ca3be664d)
#### Configure Q->SQL Knowledge Base
1. Q->SQL Excel Document
[QA.xlsx](https://github.com/user-attachments/files/18258416/QA.xlsx)
2. Upload the Q->SQL Excel document to the Q->SQL knowledge base and set the chunk data as follows via parsing:
![Q->SQL knowledge base](https://github.com/user-attachments/assets/391f4395-1458-4f5b-8b55-517ec3a7b1dc)
#### Configure TextSQL_Thesaurus Knowledge Base
1. the content of the TextSQL_Thesaurus text is as follows:
```txt
###
Standard noun: StockQuantity
Synonyms: stock,stockpile,inventory
###
Standard noun: UserName
Synonyms: user name, user's name
###
Standard noun: Quantity
Synonyms: amount,number
###
Standard noun: Smartphone
Synonyms: phone, mobile phone, smart phone, mobilephone
###
Standard noun: ProductName
Synonyms: product name, product's name
###
Standard noun: tablet
Synonyms: pad,Pad
###
Standard noun: laptop
Synonyms: laptop computer,laptop pc
```
2. set the chunk data for the TextSQL_Thesaurus knowledge base
![TextSQL_Thesaurus knowledge base](https://github.com/user-attachments/assets/76e6f802-1381-4bbc-951f-50992fbeecd8)
### Build the Agent
1. Create an Agent using the Text2SQL Agent template.
2. Enter the configuration page of the Agent to start the setup process.
3. Create a Retrieval node and name it Thesaurus; create an ExeSQL node.
4. Configure the Q->SQL, DDL, DB_Description, and TextSQL_Thesaurus knowledge bases. Please refer to the following:
![Configure Retrieval node](https://github.com/user-attachments/assets/25d67b01-954e-4eb4-87f5-c54262cf9a3e)
5. Configure the Generate node, named LLM‘s prompt:
- Add this content to the prompt provided by the template to provide the thesaurus content to the LLM:
```plaintext
## You may use the following Thesaurus statements. For example, what I ask is from Synonyms, you must use Standard noun to generate SQL. Use responses to past questions also to guide you: {sql_thesaurus}.
```
- Ensure the mapping between keys and component IDs is configured correctly.
- The configuration result should look like this:
![](https://github.com/user-attachments/assets/04e30e2b-2029-4087-9671-f513dbb1f00d)
6. Configure the ExecSQL node, filling in the configuration information for the MySQL database.
![Configure the ExecSQL node](https://github.com/user-attachments/assets/a3173ae7-8a6a-45e1-b9dd-b0311990f48a)
7. Set an opener in the Begin component like:
```plaintext
Hi! I'm your electronic products online store business data analysis assistant. What can I do for you?
### Run and Test the Agent
1. click the Run button to start the agent.
2. input the question:
```
Help me summarize stock quantities for each product
```
3. click the send button to send the question to the agent.
4. The agent will respond with the following:
![](https://github.com/user-attachments/assets/6d31cf09-dd1e-4faa-a7eb-5af6e4ff1d50)
### Debug the Agent
Since version 0.15.0, ragflow has introduced step-by-step execution for Agent components/tools, providing a robust mechanism for debugging and testing. Let's explore how to perform a step run.
1. To enter Test Run mode, you can either click the triangle icon located above the component or access the component's detail page by clicking on the component itself. Once there, select the Test Run button in the upper right corner of the component details.
![](https://github.com/user-attachments/assets/9f5d5c3e-396d-418b-9c7a-fcd8ad5af418)
![](https://github.com/user-attachments/assets/0816a582-1d88-42a1-bcb1-8f76cb702503)
2. Enter a question that does not exist in the Q->SQL knowledge base but is similar in nature.
Click the Run button to receive the component's output.
```
Find all customers who has bought a mobile phone
```
![](https://github.com/user-attachments/assets/a6270188-72af-4be7-a192-efddb611f3a4)
3. As the image shows, no matching information was retrieved from the Q->SQL knowledge base, yet a similar question exists within the database. Adjust the Rerank model, "Similarity threshold," or "Keywords similarity weight" accordingly to return relevant content.
![](https://github.com/user-attachments/assets/0592c45b-9276-465d-93d3-2530b2fb81c0)
![](https://github.com/user-attachments/assets/9e72be3a-41af-4ef2-863d-03757ddfdde6)
4. Observe the inputs and outputs of the LLM node and ExeSQL node.
![](https://github.com/user-attachments/assets/55a2b2ec-3518-4fb5-abd5-1634cd485eac)
5. The agent now produces the correct SQL query result.
6. For a query about "mobile phone," the agent successfully generates the appropriate SQL query using "Smartphone." This showcases how the thesaurus guides the LLM in generating accurate SQL queries.
With this, you maybe appreciate the capabilities of Step Run. It undoubtedly assists in constructing more effective agents.
## Troubleshooting
### Total: 0 No record in the database!
1. Confirm if the sql is correct. If so, check the connection information of the database.
2. If the connection information is correct, maybe there is actually no data matching your query in the database.
## Considerations
In real production scenarios within vertical domains, several considerations are essential for effective Text2SQL implementation:
1. **Handling DDL and DB_Description**: Dealing with Data Definition Language (DDL) statements and database descriptions requires substantial debugging experience. It is crucial to discern which information is vital and which may be redundant, depending on the true business context. This includes determining the relevance of table attributes such as primary keys, foreign keys, indexes, and so forth.
2. **Maintaining Quality QA Data**: Ensuring a high standard for question-and-answer data significantly aids the LLM in generating more accurate SQL queries.
3. **Managing Domain-Specific Synonyms**: Professional domain synonyms can greatly impact the generation of SQL query conditions. Therefore, maintaining an extensive and up-to-date synonym library is critical to mitigate this challenge.
4. **Facilitating User Feedback**: Implementing a feedback mechanism within the Agent allows users to provide correct SQL queries. Administrators can then use this feedback to automatically generate corresponding QA data, reducing the need for manual maintenance.
In summary, achieving high-quality output from Text2SQL remains contingent upon high-quality input. Constructing robust question-and-answer datasets is at the core of optimizing RAGFlow's Text2SQL capabilities.