Feat: Support tool calling in Generate component (#7572)

### What problem does this PR solve?

Hello, our use case requires LLM agent to invoke some tools, so I made a
simple implementation here.

This PR does two things:

1. A simple plugin mechanism based on `pluginlib`:

This mechanism lives in the `plugin` directory. It will only load
plugins from `plugin/embedded_plugins` for now.

A sample plugin `bad_calculator.py` is placed in
`plugin/embedded_plugins/llm_tools`, it accepts two numbers `a` and `b`,
then give a wrong result `a + b + 100`.

In the future, it can load plugins from external location with little
code change.

Plugins are divided into different types. The only plugin type supported
in this PR is `llm_tools`, which must implement the `LLMToolPlugin`
class in the `plugin/llm_tool_plugin.py`.
More plugin types can be added in the future.

2. A tool selector in the `Generate` component:

Added a tool selector to select one or more tools for LLM:


![image](https://github.com/user-attachments/assets/74a21fdf-9333-4175-991b-43df6524c5dc)

And with the `bad_calculator` tool, it results this with the `qwen-max`
model:


![image](https://github.com/user-attachments/assets/93aff9c4-8550-414a-90a2-1a15a5249d94)


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>

plugin/README.md

@@ -0,0 +1,97 @@
+# Plugins
+
+This directory contains the plugin mechanism for RAGFlow.
+
+RAGFlow will load plugins from `embedded_plugins` subdirectory recursively.
+
+## Supported plugin types
+
+Currently, the only supported plugin type is `llm_tools`.
+
+- `llm_tools`: A tool for LLM to call.
+
+## How to add a plugin
+
+Add a LLM tool plugin is simple: create a plugin file, put a class inherits the `LLMToolPlugin` class in it, then implement the `get_metadata` and the `invoke` methods.
+
+- `get_metadata` method: This method returns a `LLMToolMetadata` object, which contains the description of this tool.
+The description will be provided to LLM, and the RAGFlow web frontend for displaying.
+
+- `invoke` method: This method accepts parameters generated by LLM, and return a `str` containing the tool execution result.
+All the execution logic of this tool should go into this method.
+
+When you start RAGFlow, you can see your plugin was loaded in the log:
+
+```
+2025-05-15 19:29:08,959 INFO     34670 Recursively importing plugins from path `/some-path/ragflow/plugin/embedded_plugins`
+2025-05-15 19:29:08,960 INFO     34670 Loaded llm_tools plugin BadCalculatorPlugin version 1.0.0
+```
+
+Or it may contain some errors for you to fix your plugin.
+
+### Demo
+
+We will demonstrate how to add a plugin with a calculator tool which will give wrong answers.
+
+First, create a plugin file `bad_calculator.py` under the `embedded_plugins/llm_tools` directory.
+
+Then, we create a `BadCalculatorPlugin` class, extending the `LLMToolPlugin` base class:
+
+```python
+class BadCalculatorPlugin(LLMToolPlugin):
+    _version_ = "1.0.0"
+```
+
+The `_version_` field is required, which specifies the version of the plugin.
+
+Our calculator has two numbers `a` and `b` as inputs, so we add a `invoke` method to our `BadCalculatorPlugin` class:
+
+```python
+def invoke(self, a: int, b: int) -> str:
+    return str(a + b + 100)
+```
+
+The `invoke` method will be called by LLM. It can have many parameters, but the return type must be a `str`.
+
+Finally, we have to add a `get_metadata` method, to tell LLM how to use our `bad_calculator`:
+
+```python
+@classmethod
+def get_metadata(cls) -> LLMToolMetadata:
+    return {
+        # Name of this tool, providing to LLM
+        "name": "bad_calculator",
+        # Display name of this tool, providing to RAGFlow frontend
+        "displayName": "$t:bad_calculator.name",
+        # Description of the usage of this tool, providing to LLM
+        "description": "A tool to calculate the sum of two numbers (will give wrong answer)",
+        # Description of this tool, providing to RAGFlow frontend
+        "displayDescription": "$t:bad_calculator.description",
+        # Parameters of this tool
+        "parameters": {
+            # The first parameter - a
+            "a": {
+                # Parameter type, options are: number, string, or whatever the LLM can recognise
+                "type": "number",
+                # Description of this parameter, providing to LLM
+                "description": "The first number",
+                # Description of this parameter, provding to RAGFlow frontend
+                "displayDescription": "$t:bad_calculator.params.a",
+                # Whether this parameter is required
+                "required": True
+            },
+            # The second parameter - b
+            "b": {
+                "type": "number",
+                "description": "The second number",
+                "displayDescription": "$t:bad_calculator.params.b",
+                "required": True
+            }
+        }
+```
+
+The `get_metadata` method is a `classmethod`. It will provide the description of this tool to LLM.
+
+The fields starts with `display` can use a special notation: `$t:xxx`, which will use the i18n mechanism in the RAGFlow frontend, getting text from the `llmTools` category. The frontend will display what you put here if you don't use this notation.
+
+Now our tool is ready. You can select it in the `Generate` component and try it out.

plugin/README_zh.md

@@ -0,0 +1,98 @@
+# 插件
+
+这个文件夹包含了RAGFlow的插件机制。
+
+RAGFlow将会从`embedded_plugins`子文件夹中递归加载所有的插件。
+
+## 支持的插件类型
+
+目前，唯一支持的插件类型是`llm_tools`。
+
+- `llm_tools`：用于供LLM进行调用的工具。
+
+## 如何添加一个插件
+
+添加一个LLM工具插件是很简单的：创建一个插件文件，向其中放一个继承自`LLMToolPlugin`的类，再实现它的`get_metadata`和`invoke`方法即可。
+
+- `get_metadata`方法：这个方法返回一个`LLMToolMetadata`对象，其中包含了对这个工具的描述。
+这些描述信息将被提供给LLM进行调用，和RAGFlow的Web前端用作展示。
+
+- `invoke`方法：这个方法接受LLM生成的参数，并且返回一个`str`对象，其中包含了这个工具的执行结果。
+这个工具的所有执行逻辑都应当放到这个方法里。
+
+当你启动RAGFlow时，你会在日志中看见你的插件被加载了：
+
+```
+2025-05-15 19:29:08,959 INFO     34670 Recursively importing plugins from path `/some-path/ragflow/plugin/embedded_plugins`
+2025-05-15 19:29:08,960 INFO     34670 Loaded llm_tools plugin BadCalculatorPlugin version 1.0.0
+```
+
+也可能会报错，这时就需要根据报错对你的插件进行修复。
+
+### 示例
+
+我们将会添加一个会给出错误答案的计算器工具，来演示添加插件的过程。
+
+首先，在`embedded_plugins/llm_tools`文件夹下创建一个插件文件`bad_calculator.py`。
+
+接下来，我们创建一个`BadCalculatorPlugin`类，继承基类`LLMToolPlugin`：
+
+```python
+class BadCalculatorPlugin(LLMToolPlugin):
+    _version_ = "1.0.0"
+```
+
+`_version_`字段是必填的，用于指定这个插件的版本号。
+
+我们的计算器拥有两个输入字段`a`和`b`，所以我们添加如下的`invoke`方法到`BadCalculatorPlugin`类中：
+
+```python
+def invoke(self, a: int, b: int) -> str:
+    return str(a + b + 100)
+```
+
+`invoke`方法将会被LLM所调用。这个方法可以有许多参数，但它必须返回一个`str`。
+
+最后，我们需要添加一个`get_metadata`方法，来告诉LLM怎样使用我们的`bad_calculator`工具：
+
+```python
+@classmethod
+def get_metadata(cls) -> LLMToolMetadata:
+    return {
+        # 这个工具的名称，会提供给LLM
+        "name": "bad_calculator",
+        # 这个工具的展示名称，会提供给RAGFlow的Web前端
+        "displayName": "$t:bad_calculator.name",
+        # 这个工具的用法描述，会提供给LLM
+        "description": "A tool to calculate the sum of two numbers (will give wrong answer)",
+        # 这个工具的描述，会提供给RAGFlow的Web前端
+        "displayDescription": "$t:bad_calculator.description",
+        # 这个工具的参数
+        "parameters": {
+            # 第一个参数 - a
+            "a": {
+                # 参数类型，选项为：number, string, 或者LLM可以识别的任何类型
+                "type": "number",
+                # 这个参数的描述，会提供给LLM
+                "description": "The first number",
+                # 这个参数的描述，会提供给RAGFlow的Web前端
+                "displayDescription": "$t:bad_calculator.params.a",
+                # 这个参数是否是必填的
+                "required": True
+            },
+            # 第二个参数 - b
+            "b": {
+                "type": "number",
+                "description": "The second number",
+                "displayDescription": "$t:bad_calculator.params.b",
+                "required": True
+            }
+        }
+```
+
+`get_metadata`方法是一个`classmethod`。它会把这个工具的描述提供给LLM。
+
+以`display`开头的字段可以使用一种特殊写法`$t:xxx`，这种写法将使用RAGFlow的国际化机制，从`llmTools`这个分类中获取文字。如果你不使用这种写法，那么前端将会显示此处的原始内容。
+
+现在，我们的工具已经做好了，你可以在`生成回答`组件中选择这个工具来尝试一下。
+

plugin/__init__.py

@@ -0,0 +1,3 @@
+from .plugin_manager import PluginManager
+
+GlobalPluginManager = PluginManager()

plugin/common.py

@@ -0,0 +1 @@
+PLUGIN_TYPE_LLM_TOOLS = "llm_tools"

plugin/embedded_plugins/llm_tools/bad_calculator.py

@@ -0,0 +1,37 @@
+import logging
+from plugin.llm_tool_plugin import LLMToolMetadata, LLMToolPlugin
+
+
+class BadCalculatorPlugin(LLMToolPlugin):
+    """
+    A sample LLM tool plugin, will add two numbers with 100.
+    It only present for demo purpose. Do not use it in production.
+    """
+    _version_ = "1.0.0"
+
+    @classmethod
+    def get_metadata(cls) -> LLMToolMetadata:
+        return {
+            "name": "bad_calculator",
+            "displayName": "$t:bad_calculator.name",
+            "description": "A tool to calculate the sum of two numbers (will give wrong answer)",
+            "displayDescription": "$t:bad_calculator.description",
+            "parameters": {
+                "a": {
+                    "type": "number",
+                    "description": "The first number",
+                    "displayDescription": "$t:bad_calculator.params.a",
+                    "required": True
+                },
+                "b": {
+                    "type": "number",
+                    "description": "The second number",
+                    "displayDescription": "$t:bad_calculator.params.b",
+                    "required": True
+                }
+            }
+        }
+
+    def invoke(self, a: int, b: int) -> str:
+        logging.info(f"Bad calculator tool was called with arguments {a} and {b}")
+        return str(a + b + 100)

plugin/llm_tool_plugin.py

@@ -0,0 +1,51 @@
+from typing import Any, TypedDict
+import pluginlib
+
+from .common import PLUGIN_TYPE_LLM_TOOLS
+
+
+class LLMToolParameter(TypedDict):
+    type: str
+    description: str
+    displayDescription: str
+    required: bool
+
+
+class LLMToolMetadata(TypedDict):
+    name: str
+    displayName: str
+    description: str
+    displayDescription: str
+    parameters: dict[str, LLMToolParameter]
+
+
+@pluginlib.Parent(PLUGIN_TYPE_LLM_TOOLS)
+class LLMToolPlugin:
+    @classmethod
+    @pluginlib.abstractmethod
+    def get_metadata(cls) -> LLMToolMetadata:
+        pass
+
+    def invoke(self, **kwargs) -> str:
+        raise NotImplementedError
+
+
+def llm_tool_metadata_to_openai_tool(llm_tool_metadata: LLMToolMetadata) -> dict[str, Any]:
+    return {
+        "type": "function",
+        "function": {
+            "name": llm_tool_metadata["name"],
+            "description": llm_tool_metadata["description"],
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    k: {
+                        "type": p["type"],
+                        "description": p["description"]
+                    }
+                    for k, p in llm_tool_metadata["parameters"].items()
+                },
+                "required": [k for k, p in llm_tool_metadata["parameters"].items() if p["required"]]
+            }
+        }
+    }

plugin/plugin_manager.py

@@ -0,0 +1,45 @@
+import logging
+import os
+from pathlib import Path
+import pluginlib
+
+from .common import PLUGIN_TYPE_LLM_TOOLS
+
+from .llm_tool_plugin import LLMToolPlugin
+
+
+class PluginManager:
+    _llm_tool_plugins: dict[str, LLMToolPlugin]
+
+    def __init__(self) -> None:
+        self._llm_tool_plugins = {}
+
+    def load_plugins(self) -> None:
+        loader = pluginlib.PluginLoader(
+            paths=[str(Path(os.path.dirname(__file__), "embedded_plugins"))]
+        )
+        
+        for type, plugins in loader.plugins.items():
+            for name, plugin in plugins.items():
+                logging.info(f"Loaded {type} plugin {name} version {plugin.version}")
+
+                if type == PLUGIN_TYPE_LLM_TOOLS:
+                    metadata = plugin.get_metadata()
+                    self._llm_tool_plugins[metadata["name"]] = plugin
+
+    def get_llm_tools(self) -> list[LLMToolPlugin]:
+        return list(self._llm_tool_plugins.values())
+
+    def get_llm_tool_by_name(self, name: str) -> LLMToolPlugin | None:
+        return self._llm_tool_plugins.get(name)
+
+    def get_llm_tools_by_names(self, tool_names: list[str]) -> list[LLMToolPlugin]:
+        results = []
+
+        for name in tool_names:
+            plugin = self._llm_tool_plugins.get(name)
+
+            if plugin is not None:
+                results.append(plugin)
+
+        return results