Doc: miscellaneous (#10641)

### What problem does this PR solve?

### Type of change

- [x] Documentation Update

docs/guides/agent/agent_component_reference/agent.mdx

@@ -9,7 +9,7 @@ The component equipped with reasoning, tool usage, and multi-agent collaboration
 
 ---
 
-An **Agent** component fine-tunes the LLM and sets its prompt. From v0.21.0 onwards, an **Agent** component is able to work independently and with the following capabilities:
+An **Agent** component fine-tunes the LLM and sets its prompt. From v0.20.5 onwards, an **Agent** component is able to work independently and with the following capabilities:
 
 - Autonomous reasoning with reflection and adjustment based on environmental feedback.
 - Use of tools or subagents to complete tasks.
@@ -24,7 +24,7 @@ An **Agent** component is essential when you need the LLM to assist with summari
 
 ![Set default models](https://raw.githubusercontent.com/infiniflow/ragflow-docs/main/images/set_default_models.jpg)
 
-2. If your Agent involves dataset retrieval, ensure you [have properly configured your target knowledge base(s)](../../dataset/configure_knowledge_base.md).
+2. If your Agent involves dataset retrieval, ensure you [have properly configured your target dataset(s)](../../dataset/configure_knowledge_base.md).
 
 ## Quickstart
 
@@ -113,7 +113,7 @@ Click the dropdown menu of **Model** to show the model configuration window.
 - **Model**: The chat model to use.  
   - Ensure you set the chat model correctly on the **Model providers** page.
   - You can use different models for different components to increase flexibility or improve overall performance.
-- **Freedom**: A shortcut to **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty** settings, indicating the freedom level of the model. From **Improvise**, **Precise**, to **Balance**, each preset configuration corresponds to a unique combination of **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**.   
+- **Creavity**: A shortcut to **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty** settings, indicating the freedom level of the model. From **Improvise**, **Precise**, to **Balance**, each preset configuration corresponds to a unique combination of **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**.   
   This parameter has three options:
   - **Improvise**: Produces more creative responses.
   - **Precise**: (Default) Produces more conservative responses.
@@ -133,10 +133,11 @@ Click the dropdown menu of **Model** to show the model configuration window.
   - A higher **frequency penalty** value results in the model being more conservative in its use of repeated tokens.
   - Defaults to 0.7.
 - **Max tokens**:  
+  This sets the maximum length of the model's output, measured in the number of tokens (words or pieces of words). It is disabled by default, allowing the model to determine the number of tokens in its responses.
 
 :::tip NOTE
 - It is not necessary to stick with the same model for all components. If a specific model is not performing well for a particular task, consider using a different one.
-- If you are uncertain about the mechanism behind **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**, simply choose one of the three options of **Preset configurations**.
+- If you are uncertain about the mechanism behind **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**, simply choose one of the three options of **Creavity**.
 :::
 
 ### System prompt
@@ -147,7 +148,7 @@ An **Agent** component relies on keys (variables) to specify its data inputs. It
 
 #### Advanced usage
 
-From v0.21.0 onwards, four framework-level prompt blocks are available in the **System prompt** field, enabling you to customize and *override* prompts at the framework level. Type `/` or click **(x)** to view them; they appear under the **Framework** entry in the dropdown menu.
+From v0.20.5 onwards, four framework-level prompt blocks are available in the **System prompt** field, enabling you to customize and *override* prompts at the framework level. Type `/` or click **(x)** to view them; they appear under the **Framework** entry in the dropdown menu.
 
 - `task_analysis` prompt block
   - This block is responsible for analyzing tasks — either a user task or a task assigned by the lead Agent when the **Agent** component is acting as a Sub-Agent.

docs/guides/agent/agent_component_reference/categorize.mdx

@@ -42,7 +42,7 @@ Click the dropdown menu of **Model** to show the model configuration window.
 - **Model**: The chat model to use.  
   - Ensure you set the chat model correctly on the **Model providers** page.
   - You can use different models for different components to increase flexibility or improve overall performance.
-- **Freedom**: A shortcut to **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty** settings, indicating the freedom level of the model. From **Improvise**, **Precise**, to **Balance**, each preset configuration corresponds to a unique combination of **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**.  
+- **Creavity**: A shortcut to **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty** settings, indicating the freedom level of the model. From **Improvise**, **Precise**, to **Balance**, each preset configuration corresponds to a unique combination of **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**.  
   This parameter has three options:  
   - **Improvise**: Produces more creative responses.
   - **Precise**: (Default) Produces more conservative responses.
@@ -61,10 +61,12 @@ Click the dropdown menu of **Model** to show the model configuration window.
 - **Frequency penalty**: Discourages the model from repeating the same words or phrases too frequently in the generated text.  
   - A higher **frequency penalty** value results in the model being more conservative in its use of repeated tokens.
   - Defaults to 0.7.
+- **Max tokens**:  
+  This sets the maximum length of the model's output, measured in the number of tokens (words or pieces of words). It is disabled by default, allowing the model to determine the number of tokens in its responses.
 
 :::tip NOTE
 - It is not necessary to stick with the same model for all components. If a specific model is not performing well for a particular task, consider using a different one.
-- If you are uncertain about the mechanism behind **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**, simply choose one of the three options of **Preset configurations**.
+- If you are uncertain about the mechanism behind **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**, simply choose one of the three options of **Creavity**.
 :::
 
 ### Message window size

docs/guides/agent/agent_component_reference/indexer.md

@@ -1,5 +1,5 @@
 ---
-sidebar_position: 30
+sidebar_position: 40
 slug: /indexer_component
 ---
 

docs/guides/agent/agent_component_reference/retrieval.mdx

@@ -87,9 +87,9 @@ RAGFlow employs a combination of weighted keyword similarity and weighted vector
 
 Defaults to 0.2.
 
-### Keyword similarity weight
+### Vector similarity weight
 
-This parameter sets the weight of keyword similarity in the combined similarity score. The total of the two weights must equal 1.0. Its default value is 0.7, which means the weight of vector similarity in the combined search is 1 - 0.7 = 0.3.
+This parameter sets the weight of vector similarity in the composite similarity score. The total of the two weights must equal 1.0. Its default value is 0.3, which means the weight of keyword similarity in a combined search is 1 - 0.3 = 0.7.
 
 ### Top N
 

docs/guides/agent/agent_component_reference/transformer.md

@@ -0,0 +1,80 @@
+---
+sidebar_position: 37
+slug: /transformer_component
+---
+
+# Transformer component
+
+A component that uses an LLM to extract insights from the chunks.
+
+---
+
+A **Transformer** component indexes chunks and configures their storage formats in the document engine. It *typically* precedes the **Indexer** in the ingestion pipeline, but you can also chain multiple **Transformer** components in sequence.
+
+## Scenario
+
+A **Transformer** component is essential when you need the LLM to extract new information, such as keywords, questions, metadata, and summaries, from the original chunks. 
+
+## Configurations
+
+### Model
+
+Click the dropdown menu of **Model** to show the model configuration window.
+
+- **Model**: The chat model to use.  
+  - Ensure you set the chat model correctly on the **Model providers** page.
+  - You can use different models for different components to increase flexibility or improve overall performance.
+- **Creavity**: A shortcut to **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty** settings, indicating the freedom level of the model. From **Improvise**, **Precise**, to **Balance**, each preset configuration corresponds to a unique combination of **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**.   
+  This parameter has three options:
+  - **Improvise**: Produces more creative responses.
+  - **Precise**: (Default) Produces more conservative responses.
+  - **Balance**: A middle ground between **Improvise** and **Precise**.
+- **Temperature**: The randomness level of the model's output.  
+  Defaults to 0.1.
+  - Lower values lead to more deterministic and predictable outputs.
+  - Higher values lead to more creative and varied outputs.
+  - A temperature of zero results in the same output for the same prompt.
+- **Top P**: Nucleus sampling.  
+  - Reduces the likelihood of generating repetitive or unnatural text by setting a threshold *P* and restricting the sampling to tokens with a cumulative probability exceeding *P*.
+  - Defaults to 0.3.
+- **Presence penalty**: Encourages the model to include a more diverse range of tokens in the response.  
+  - A higher **presence penalty** value results in the model being more likely to generate tokens not yet been included in the generated text.
+  - Defaults to 0.4.
+- **Frequency penalty**: Discourages the model from repeating the same words or phrases too frequently in the generated text.  
+  - A higher **frequency penalty** value results in the model being more conservative in its use of repeated tokens.
+  - Defaults to 0.7.
+- **Max tokens**:  
+  This sets the maximum length of the model's output, measured in the number of tokens (words or pieces of words). It is disabled by default, allowing the model to determine the number of tokens in its responses.
+
+:::tip NOTE
+- It is not necessary to stick with the same model for all components. If a specific model is not performing well for a particular task, consider using a different one.
+- If you are uncertain about the mechanism behind **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**, simply choose one of the three options of **Creativity**.
+:::
+
+### Result destination
+
+Select the type of output to be generated by the LLM:
+
+- Summary
+- Keywords
+- Questions
+- Metadata
+
+### System prompt
+
+Typically, you use the system prompt to describe the task for the LLM, specify how it should respond, and outline other miscellaneous requirements. We do not plan to elaborate on this topic, as it can be as extensive as prompt engineering. 
+
+:::tip NOTE
+The system prompt here automatically updates to match your selected **Result destination**.
+:::
+
+### User prompt
+
+The user-defined prompt. For example, you can type `/` or click **(x)** to insert variables of preceding components in the ingestion pipeline as the LLM's input.
+
+### Output
+
+The global variable name for the output of the **Transformer** component, which can be referenced by subsequent **Transformer** components in the ingestion pipeline.
+
+- Default: `chunks`
+- Type: `Array<Object>`

docs/guides/chat/start_chat.md

@@ -19,7 +19,7 @@ You start an AI conversation by creating an assistant.
 
    > RAGFlow offers you the flexibility of choosing a different chat model for each dialogue, while allowing you to set the default models in **System Model Settings**.
 
-2. Update **Assistant settings**:
+2. Update Assistant-specific settings:
 
    - **Assistant name** is the name of your chat assistant. Each assistant corresponds to a dialogue with a unique combination of datasets, prompts, hybrid search configurations, and large model settings.
    - **Empty response**:
@@ -28,12 +28,12 @@ You start an AI conversation by creating an assistant.
    - **Show quote**: This is a key feature of RAGFlow and enabled by default. RAGFlow does not work like a black box. Instead, it clearly shows the sources of information that its responses are based on.
    - Select the corresponding datasets. You can select one or multiple datasets, but ensure that they use the same embedding model, otherwise an error would occur.
 
-3. Update **Prompt engine**:
+3. Update Prompt-specific settings:
 
    - In **System**, you fill in the prompts for your LLM, you can also leave the default prompt as-is for the beginning.
    - **Similarity threshold** sets the similarity "bar" for each chunk of text. The default is 0.2. Text chunks with lower similarity scores are filtered out of the final response.
-   - **Keyword similarity weight** is set to 0.7 by default. RAGFlow uses a hybrid score system to evaluate the relevance of different text chunks. This value sets the weight assigned to the keyword similarity component in the hybrid score.
-     - If **Rerank model** is left empty, the hybrid score system uses keyword similarity and vector similarity, and the default weight assigned to the vector similarity component is 1-0.7=0.3.
+   - **Vector similarity weight** is set to 0.3 by default. RAGFlow uses a hybrid score system to evaluate the relevance of different text chunks. This value sets the weight assigned to the vector similarity component in the hybrid score.
+     - If **Rerank model** is left empty, the hybrid score system uses keyword similarity and vector similarity, and the default weight assigned to the keyword similarity component is 1-0.3=0.7.
      - If **Rerank model** is selected, the hybrid score system uses keyword similarity and reranker score, and the default weight assigned to the reranker score is 1-0.7=0.3.
    - **Top N** determines the *maximum* number of chunks to feed to the LLM. In other words, even if more chunks are retrieved, only the top N chunks are provided as input.
    - **Multi-turn optimization** enhances user queries using existing context in a multi-round conversation. It is enabled by default. When enabled, it will consume additional LLM tokens and significantly increase the time to generate answers.
@@ -52,10 +52,10 @@ You start an AI conversation by creating an assistant.
          - HTTP method [Converse with chat assistant](../../references/http_api_reference.md#converse-with-chat-assistant), or
          - Python method [Converse with chat assistant](../../references/python_api_reference.md#converse-with-chat-assistant).
 
-4. Update **Model Setting**:
+4. Update Model-specific Settings:
 
    - In **Model**: you select the chat model. Though you have selected the default chat model in **System Model Settings**, RAGFlow allows you to choose an alternative chat model for your dialogue.
-   - **Freedom**: A shortcut to **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty** settings, indicating the freedom level of the model. From **Improvise**, **Precise**, to **Balance**, each preset configuration corresponds to a unique combination of **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**.   
+   - **Creavity**: A shortcut to **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty** settings, indicating the freedom level of the model. From **Improvise**, **Precise**, to **Balance**, each preset configuration corresponds to a unique combination of **Temperature**, **Top P**, **Presence penalty**, and **Frequency penalty**.   
    This parameter has three options:
       - **Improvise**: Produces more creative responses.
       - **Precise**: (Default) Produces more conservative responses.

docs/guides/dataset/run_retrieval_test.md

@@ -29,9 +29,9 @@ In contrast, chunks created from [knowledge graph construction](./construct_know
 
 This sets the bar for retrieving chunks: chunks with similarities below the threshold will be filtered out. By default, the threshold is set to 0.2. This means that only chunks with hybrid similarity score of 20 or higher will be retrieved.
 
-### Keyword similarity weight
+### Vector similarity weight
 
-This sets the weight of keyword similarity in the combined similarity score, whether used with vector cosine similarity or a reranking score. By default, it is set to 0.7, making the weight of the other component 0.3 (1 - 0.7).
+This sets the weight of vector similarity in the composite similarity score, whether used with vector cosine similarity or a reranking score. By default, it is set to 0.3, making the weight of the other component 0.7 (1 - 0.3).
 
 ### Rerank model