Feat: Support tool calling in Generate component (#7572)

### What problem does this PR solve?

Hello, our use case requires LLM agent to invoke some tools, so I made a
simple implementation here.

This PR does two things:

1. A simple plugin mechanism based on `pluginlib`:

This mechanism lives in the `plugin` directory. It will only load
plugins from `plugin/embedded_plugins` for now.

A sample plugin `bad_calculator.py` is placed in
`plugin/embedded_plugins/llm_tools`, it accepts two numbers `a` and `b`,
then give a wrong result `a + b + 100`.

In the future, it can load plugins from external location with little
code change.

Plugins are divided into different types. The only plugin type supported
in this PR is `llm_tools`, which must implement the `LLMToolPlugin`
class in the `plugin/llm_tool_plugin.py`.
More plugin types can be added in the future.

2. A tool selector in the `Generate` component:

Added a tool selector to select one or more tools for LLM:


![image](https://github.com/user-attachments/assets/74a21fdf-9333-4175-991b-43df6524c5dc)

And with the `bad_calculator` tool, it results this with the `qwen-max`
model:


![image](https://github.com/user-attachments/assets/93aff9c4-8550-414a-90a2-1a15a5249d94)


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
This commit is contained in:
Song Fuchang
2025-05-16 16:32:19 +08:00
committed by GitHub
parent cb26564d50
commit a1f06a4fdc
28 changed files with 625 additions and 61 deletions

View File

@ -454,6 +454,8 @@ This auto-tagging feature enhances retrieval by adding another layer of domain-s
model: 'Model',
modelTip: 'Large language chat model',
modelMessage: 'Please select!',
modelEnabledTools: 'Enabled tools',
modelEnabledToolsTip: 'Please select one or more tools for the chat model to use. It takes no effect for models not supporting tool call.',
freedom: 'Freedom',
improvise: 'Improvise',
precise: 'Precise',
@ -1267,5 +1269,15 @@ This delimiter is used to split the input text into several text pieces echo of
inputVariables: 'Input variables',
runningHintText: 'is running...🕞',
},
llmTools: {
bad_calculator: {
name: "Calculator",
description: "A tool to calculate the sum of two numbers (will give wrong answer)",
params: {
a: "The first number",
b: "The second number",
},
},
},
},
};

View File

@ -461,6 +461,8 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
model: '模型',
modelTip: '大语言聊天模型',
modelMessage: '请选择',
modelEnabledTools: '可用的工具',
modelEnabledToolsTip: '请选择一个或多个可供该模型所使用的工具。仅对支持工具调用的模型生效。',
freedom: '自由度',
improvise: '即兴创作',
precise: '精确',
@ -1231,5 +1233,15 @@ General实体和关系提取提示来自 GitHub - microsoft/graphrag基于
knowledge: 'knowledge',
chat: 'chat',
},
llmTools: {
bad_calculator: {
name: "计算器",
description: "用于计算两个数的和的工具(会给出错误答案)",
params: {
a: "第一个数",
b: "第二个数",
},
},
},
},
};