mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-24 04:47:23 +08:00
Merge branch 'main' into docs-publish-v0
This commit is contained in:
@ -496,4 +496,29 @@ For more information, see [Google Vertex AI documentation](https://cloud.google.
|
||||
|--------|---------------|-----------------------------------------------------|
|
||||
| model | LanguageModel | An instance of ChatVertexAI configured with the specified parameters. |
|
||||
|
||||
## xAI
|
||||
|
||||
This component generates text using xAI models like [Grok](https://x.ai/grok).
|
||||
|
||||
For more information, see the [xAI documentation](https://x.ai/).
|
||||
|
||||
### Inputs
|
||||
|
||||
| Name | Type | Description |
|
||||
|----------------|---------------|-----------------------------------------------------------------|
|
||||
| max_tokens | Integer | Maximum number of tokens to generate. Set to `0` for unlimited. Range: `0-128000`. |
|
||||
| model_kwargs | Dictionary | Additional keyword arguments for the model. |
|
||||
| json_mode | Boolean | If `True`, outputs JSON regardless of passing a schema. |
|
||||
| model_name | String | The xAI model to use. Default: `grok-2-latest`. |
|
||||
| base_url | String | Base URL for API requests. Default: `https://api.x.ai/v1`. |
|
||||
| api_key | SecretString | Your xAI API key for authentication. |
|
||||
| temperature | Float | Controls randomness in the output. Range: `[0.0, 2.0]`. Default: `0.1`. |
|
||||
| seed | Integer | Controls reproducibility of the job. |
|
||||
|
||||
### Outputs
|
||||
|
||||
| Name | Type | Description |
|
||||
|-------|---------------|------------------------------------------------------------------|
|
||||
| model | LanguageModel | An instance of ChatOpenAI configured with the specified parameters. |
|
||||
|
||||
|
||||
|
||||
BIN
docs/docs/Integrations/Apify/apify_agent_flow.png
Normal file
BIN
docs/docs/Integrations/Apify/apify_agent_flow.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
BIN
docs/docs/Integrations/Apify/apify_agent_flow_simple.png
Normal file
BIN
docs/docs/Integrations/Apify/apify_agent_flow_simple.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 117 KiB |
BIN
docs/docs/Integrations/Apify/apify_flow_wcc.png
Normal file
BIN
docs/docs/Integrations/Apify/apify_flow_wcc.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 80 KiB |
63
docs/docs/Integrations/Apify/integrations-apify.md
Normal file
63
docs/docs/Integrations/Apify/integrations-apify.md
Normal file
@ -0,0 +1,63 @@
|
||||
---
|
||||
title: Apify
|
||||
slug: /integrations-apify
|
||||
---
|
||||
|
||||
# Integrate Apify with Langflow
|
||||
|
||||
[Apify](https://apify.com/) is a web scraping and data extraction platform. It provides an app store with more than three thousand ready-made cloud tools called Actors.
|
||||
|
||||
The Apify components allow you to run Apify Actors in your flow to accomplish tasks such as:
|
||||
|
||||
- Crawling websites and extracting text content
|
||||
- Scraping social media platforms like Instagram and Facebook
|
||||
- Extracting data from Google Maps
|
||||
- Inserting data into a PostgreSQL/MySQL/MSSQL database
|
||||
- Running various other automation tasks
|
||||
|
||||
More info about Apify:
|
||||
|
||||
- [Website](https://apify.com/)
|
||||
- [Apify Actor Store](https://apify.com/store)
|
||||
- [Actor Whitepaper](https://whitepaper.actor/)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need an **Apify API token**. You can create a free account on [Apify](https://apify.com/) and generate your API key in the Apify Console. [Get a Free API key here](https://docs.apify.com/platform/integrations/api).
|
||||
|
||||
Enter the key in the *Apify Token* field in all components that require the key.
|
||||
|
||||
## Example flows
|
||||
|
||||
### Extract website text content in Markdown format
|
||||
|
||||
Use the [Website Content Crawler Actor](https://apify.com/apify/website-content-crawler) to extract text content in Markdown format from a website and process it in your flow.
|
||||

|
||||
|
||||
### Analyze and process website content with an Agent
|
||||
|
||||
Building on the previous example, this flow not only extracts website content using the [Website Content Crawler Actor](https://apify.com/apify/website-content-crawler) but also processes and analyzes it with an agent. The agent takes the extracted data and transforms it into summaries, insights, or structured responses, making the information more actionable. Unlike simple extraction, this approach enables automated content understanding and contextual processing.
|
||||

|
||||
|
||||
### Search and analyze social media profiles with an Agent
|
||||
|
||||
Perform comprehensive social media research with multiple Apify Actors. Start with the [Google Search Results Scraper Actor](https://apify.com/apify/google-search-scraper) to find relevant social media profiles, then use the [TikTok Data Extractor Actor](https://apify.com/clockworks/free-tiktok-scraper) to gather data and videos. The agent streamlines the process by collecting links from Google and retrieving content from TikTok, enabling deeper analysis of a person, brand, or topic.
|
||||

|
||||
|
||||
## Components
|
||||
|
||||
### Apify Actors
|
||||
|
||||
This component allows you to run Apify Actors in your flow. It can be used manually by providing run input or integrated as a tool for an AI Agent. When used with an AI Agent, the agent can leverage the Apify Actors to perform various tasks.
|
||||
|
||||
- **Input**:
|
||||
- Apify Token: Your API key.
|
||||
- Actor: The Apify Actor to run. Example: `apify/website-content-crawler`.
|
||||
- Run Input: The JSON input for configuring the Actor run.
|
||||
|
||||
- **Output**:
|
||||
- Actor Run Result: The JSON response containing the output of the Actor run.
|
||||
|
||||
## How to use Apify Actors in Langflow
|
||||
|
||||
First, you need to pick an Actor that you want to use in your flow from the [Apify Actor Store](https://apify.com/store). Then, create the **Apify Actors** component and input your Apify API token and the Actor ID. You can find the Actor ID in the Apify Actor Store, for instance, the [Website Content Crawler](https://apify.com/apify/website-content-crawler) has Actor ID `apify/website-content-crawler`. Now you can either connect the **Tool** output to an AI Agent or configure the Run input JSON manually and run the component to retrieve data from the **Output Data**. Example Run input can be obtained from the Actor details page in the Apify Actor Store. See the **JSON Example** in the input schema section [here](https://apify.com/apify/website-content-crawler/input-schema).
|
||||
@ -100,6 +100,7 @@ module.exports = {
|
||||
type: "category",
|
||||
label: "Integrations",
|
||||
items: [
|
||||
"Integrations/Apify/integrations-apify",
|
||||
"Integrations/integrations-assemblyai",
|
||||
"Integrations/Composio/integrations-composio",
|
||||
"Integrations/integrations-langfuse",
|
||||
|
||||
@ -112,6 +112,7 @@ dependencies = [
|
||||
"scrapegraph-py>=1.12.0",
|
||||
"pydantic-ai>=0.0.19",
|
||||
"smolagents>=1.8.0",
|
||||
"apify-client>=1.8.1",
|
||||
]
|
||||
|
||||
[tool.uv.sources]
|
||||
|
||||
5
src/backend/base/langflow/components/apify/__init__.py
Normal file
5
src/backend/base/langflow/components/apify/__init__.py
Normal file
@ -0,0 +1,5 @@
|
||||
from .apify_actor import ApifyActorsComponent
|
||||
|
||||
__all__ = [
|
||||
"ApifyActorsComponent",
|
||||
]
|
||||
324
src/backend/base/langflow/components/apify/apify_actor.py
Normal file
324
src/backend/base/langflow/components/apify/apify_actor.py
Normal file
@ -0,0 +1,324 @@
|
||||
import json
|
||||
import string
|
||||
from typing import Any, cast
|
||||
|
||||
from apify_client import ApifyClient
|
||||
from langchain_community.document_loaders.apify_dataset import ApifyDatasetLoader
|
||||
from langchain_core.tools import BaseTool
|
||||
from pydantic import BaseModel, Field, field_serializer
|
||||
|
||||
from langflow.custom import Component
|
||||
from langflow.field_typing import Tool
|
||||
from langflow.inputs.inputs import BoolInput
|
||||
from langflow.io import MultilineInput, Output, SecretStrInput, StrInput
|
||||
from langflow.schema import Data
|
||||
|
||||
MAX_DESCRIPTION_LEN = 250
|
||||
|
||||
|
||||
class ApifyActorsComponent(Component):
|
||||
display_name = "Apify Actors"
|
||||
description = (
|
||||
"Use Apify Actors to extract data from hundreds of places fast. "
|
||||
"This component can be used in a flow to retrieve data or as a tool with an agent."
|
||||
)
|
||||
documentation: str = "http://docs.langflow.org/integrations-apify"
|
||||
icon = "Apify"
|
||||
name = "ApifyActors"
|
||||
|
||||
inputs = [
|
||||
SecretStrInput(
|
||||
name="apify_token",
|
||||
display_name="Apify Token",
|
||||
info="The API token for the Apify account.",
|
||||
required=True,
|
||||
password=True,
|
||||
),
|
||||
StrInput(
|
||||
name="actor_id",
|
||||
display_name="Actor",
|
||||
info=(
|
||||
"Actor name from Apify store to run. For example 'apify/website-content-crawler' "
|
||||
"to use the Website Content Crawler Actor."
|
||||
),
|
||||
required=True,
|
||||
),
|
||||
# multiline input is more pleasant to use than the nested dict input
|
||||
MultilineInput(
|
||||
name="run_input",
|
||||
display_name="Run input",
|
||||
info=(
|
||||
'The JSON input for the Actor run. For example for the "apify/website-content-crawler" Actor: '
|
||||
'{"startUrls":[{"url":"https://docs.apify.com/academy/web-scraping-for-beginners"}],"maxCrawlDepth":0}'
|
||||
),
|
||||
value="{}",
|
||||
required=True,
|
||||
),
|
||||
MultilineInput(
|
||||
name="dataset_fields",
|
||||
display_name="Output fields",
|
||||
info=(
|
||||
"Fields to extract from the dataset, split by commas. "
|
||||
"Other fields will be ignored. Dots in nested structures will be replaced by underscores. "
|
||||
"Sample input: 'text, metadata.title'. "
|
||||
"Sample output: {'text': 'page content here', 'metadata_title': 'page title here'}. "
|
||||
"For example, for the 'apify/website-content-crawler' Actor, you can extract the 'markdown' field, "
|
||||
"which is the content of the website in markdown format."
|
||||
),
|
||||
),
|
||||
BoolInput(
|
||||
name="flatten_dataset",
|
||||
display_name="Flatten output",
|
||||
info=(
|
||||
"The output dataset will be converted from a nested format to a flat structure. "
|
||||
"Dots in nested structure will be replaced by underscores. "
|
||||
"This is useful for further processing of the Data object. "
|
||||
"For example, {'a': {'b': 1}} will be flattened to {'a_b': 1}."
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
outputs = [
|
||||
Output(display_name="Output", name="output", type_=list[Data], method="run_model"),
|
||||
Output(display_name="Tool", name="tool", type_=Tool, method="build_tool"),
|
||||
]
|
||||
|
||||
def __init__(self, *args, **kwargs) -> None:
|
||||
super().__init__(*args, **kwargs)
|
||||
self._apify_client: ApifyClient | None = None
|
||||
|
||||
def run_model(self) -> list[Data]:
|
||||
"""Run the Actor and return node output."""
|
||||
_input = json.loads(self.run_input)
|
||||
fields = ApifyActorsComponent.parse_dataset_fields(self.dataset_fields) if self.dataset_fields else None
|
||||
res = self._run_actor(self.actor_id, _input, fields=fields)
|
||||
if self.flatten_dataset:
|
||||
res = [ApifyActorsComponent.flatten(item) for item in res]
|
||||
data = [Data(data=item) for item in res]
|
||||
|
||||
self.status = data
|
||||
return data
|
||||
|
||||
def build_tool(self) -> Tool:
|
||||
"""Build a tool for an agent that runs the Apify Actor."""
|
||||
actor_id = self.actor_id
|
||||
|
||||
build = self._get_actor_latest_build(actor_id)
|
||||
readme = build.get("readme", "")[:250] + "..."
|
||||
if not (input_schema_str := build.get("inputSchema")):
|
||||
msg = "Input schema not found"
|
||||
raise ValueError(msg)
|
||||
input_schema = json.loads(input_schema_str)
|
||||
properties, required = ApifyActorsComponent.get_actor_input_schema_from_build(input_schema)
|
||||
properties = {"run_input": properties}
|
||||
|
||||
# works from input schema
|
||||
_info = [
|
||||
(
|
||||
"JSON encoded as a string with input schema (STRICTLY FOLLOW JSON FORMAT AND SCHEMA):\n\n"
|
||||
f"{json.dumps(properties, separators=(',', ':'))}"
|
||||
)
|
||||
]
|
||||
if required:
|
||||
_info.append("\n\nRequired fields:\n" + "\n".join(required))
|
||||
|
||||
info = "".join(_info)
|
||||
|
||||
input_model_cls = ApifyActorsComponent.create_input_model_class(info)
|
||||
tool_cls = ApifyActorsComponent.create_tool_class(self, readme, input_model_cls, actor_id)
|
||||
|
||||
return cast("Tool", tool_cls())
|
||||
|
||||
@staticmethod
|
||||
def create_tool_class(
|
||||
parent: "ApifyActorsComponent", readme: str, input_model: type[BaseModel], actor_id: str
|
||||
) -> type[BaseTool]:
|
||||
"""Create a tool class that runs an Apify Actor."""
|
||||
|
||||
class ApifyActorRun(BaseTool):
|
||||
"""Tool that runs Apify Actors."""
|
||||
|
||||
name: str = f"apify_actor_{ApifyActorsComponent.actor_id_to_tool_name(actor_id)}"
|
||||
description: str = (
|
||||
"Run an Apify Actor with the given input. "
|
||||
"Here is a part of the currently loaded Actor README:\n\n"
|
||||
f"{readme}\n\n"
|
||||
)
|
||||
|
||||
args_schema: type[BaseModel] = input_model
|
||||
|
||||
@field_serializer("args_schema")
|
||||
def serialize_args_schema(self, args_schema):
|
||||
return args_schema.schema()
|
||||
|
||||
def _run(self, run_input: str | dict) -> str:
|
||||
"""Use the Apify Actor."""
|
||||
input_dict = json.loads(run_input) if isinstance(run_input, str) else run_input
|
||||
|
||||
# retrieve if nested, just in case
|
||||
input_dict = input_dict.get("run_input", input_dict)
|
||||
|
||||
res = parent._run_actor(actor_id, input_dict)
|
||||
return "\n\n".join([ApifyActorsComponent.dict_to_json_str(item) for item in res])
|
||||
|
||||
return ApifyActorRun
|
||||
|
||||
@staticmethod
|
||||
def create_input_model_class(description: str) -> type[BaseModel]:
|
||||
"""Create a Pydantic model class for the Actor input."""
|
||||
|
||||
class ActorInput(BaseModel):
|
||||
"""Input for the Apify Actor tool."""
|
||||
|
||||
run_input: str = Field(..., description=description)
|
||||
|
||||
return ActorInput
|
||||
|
||||
def _get_apify_client(self) -> ApifyClient:
|
||||
"""Get the Apify client.
|
||||
|
||||
Is created if not exists or token changes.
|
||||
"""
|
||||
if not self.apify_token:
|
||||
msg = "API token is required."
|
||||
raise ValueError(msg)
|
||||
# when token changes, create a new client
|
||||
if self._apify_client is None or self._apify_client.token != self.apify_token:
|
||||
self._apify_client = ApifyClient(self.apify_token)
|
||||
if httpx_client := self._apify_client.http_client.httpx_client:
|
||||
httpx_client.headers["user-agent"] += "; Origin/langflow"
|
||||
return self._apify_client
|
||||
|
||||
def _get_actor_latest_build(self, actor_id: str) -> dict:
|
||||
"""Get the latest build of an Actor from the default build tag."""
|
||||
client = self._get_apify_client()
|
||||
actor = client.actor(actor_id=actor_id)
|
||||
if not (actor_info := actor.get()):
|
||||
msg = f"Actor {actor_id} not found."
|
||||
raise ValueError(msg)
|
||||
|
||||
default_build_tag = actor_info.get("defaultRunOptions", {}).get("build")
|
||||
latest_build_id = actor_info.get("taggedBuilds", {}).get(default_build_tag, {}).get("buildId")
|
||||
|
||||
if (build := client.build(latest_build_id).get()) is None:
|
||||
msg = f"Build {latest_build_id} not found."
|
||||
raise ValueError(msg)
|
||||
|
||||
return build
|
||||
|
||||
@staticmethod
|
||||
def get_actor_input_schema_from_build(input_schema: dict) -> tuple[dict, list[str]]:
|
||||
"""Get the input schema from the Actor build.
|
||||
|
||||
Trim the description to 250 characters.
|
||||
"""
|
||||
properties = input_schema.get("properties", {})
|
||||
required = input_schema.get("required", [])
|
||||
|
||||
properties_out: dict = {}
|
||||
for item, meta in properties.items():
|
||||
properties_out[item] = {}
|
||||
if desc := meta.get("description"):
|
||||
properties_out[item]["description"] = (
|
||||
desc[:MAX_DESCRIPTION_LEN] + "..." if len(desc) > MAX_DESCRIPTION_LEN else desc
|
||||
)
|
||||
for key_name in ("type", "default", "prefill", "enum"):
|
||||
if value := meta.get(key_name):
|
||||
properties_out[item][key_name] = value
|
||||
|
||||
return properties_out, required
|
||||
|
||||
def _get_run_dataset_id(self, run_id: str) -> str:
|
||||
"""Get the dataset id from the run id."""
|
||||
client = self._get_apify_client()
|
||||
run = client.run(run_id=run_id)
|
||||
if (dataset := run.dataset().get()) is None:
|
||||
msg = "Dataset not found"
|
||||
raise ValueError(msg)
|
||||
if (did := dataset.get("id")) is None:
|
||||
msg = "Dataset id not found"
|
||||
raise ValueError(msg)
|
||||
return did
|
||||
|
||||
@staticmethod
|
||||
def dict_to_json_str(d: dict) -> str:
|
||||
"""Convert a dictionary to a JSON string."""
|
||||
return json.dumps(d, separators=(",", ":"), default=lambda _: "<n/a>")
|
||||
|
||||
@staticmethod
|
||||
def actor_id_to_tool_name(actor_id: str) -> str:
|
||||
"""Turn actor_id into a valid tool name.
|
||||
|
||||
Tool name must only contain letters, numbers, underscores, dashes,
|
||||
and cannot contain spaces.
|
||||
"""
|
||||
valid_chars = string.ascii_letters + string.digits + "_-"
|
||||
return "".join(char if char in valid_chars else "_" for char in actor_id)
|
||||
|
||||
def _run_actor(self, actor_id: str, run_input: dict, fields: list[str] | None = None) -> list[dict]:
|
||||
"""Run an Apify Actor and return the output dataset.
|
||||
|
||||
Args:
|
||||
actor_id: Actor name from Apify store to run.
|
||||
run_input: JSON input for the Actor.
|
||||
fields: List of fields to extract from the dataset. Other fields will be ignored.
|
||||
"""
|
||||
client = self._get_apify_client()
|
||||
if (details := client.actor(actor_id=actor_id).call(run_input=run_input, wait_secs=1)) is None:
|
||||
msg = "Actor run details not found"
|
||||
raise ValueError(msg)
|
||||
if (run_id := details.get("id")) is None:
|
||||
msg = "Run id not found"
|
||||
raise ValueError(msg)
|
||||
|
||||
if (run_client := client.run(run_id)) is None:
|
||||
msg = "Run client not found"
|
||||
raise ValueError(msg)
|
||||
|
||||
# stream logs
|
||||
with run_client.log().stream() as response:
|
||||
if response:
|
||||
for line in response.iter_lines():
|
||||
self.log(line)
|
||||
run_client.wait_for_finish()
|
||||
|
||||
dataset_id = self._get_run_dataset_id(run_id)
|
||||
|
||||
loader = ApifyDatasetLoader(
|
||||
dataset_id=dataset_id,
|
||||
dataset_mapping_function=lambda item: item
|
||||
if not fields
|
||||
else {k.replace(".", "_"): ApifyActorsComponent.get_nested_value(item, k) for k in fields},
|
||||
)
|
||||
return loader.load()
|
||||
|
||||
@staticmethod
|
||||
def get_nested_value(data: dict[str, Any], key: str) -> Any:
|
||||
"""Get a nested value from a dictionary."""
|
||||
keys = key.split(".")
|
||||
value = data
|
||||
for k in keys:
|
||||
if not isinstance(value, dict) or k not in value:
|
||||
return None
|
||||
value = value[k]
|
||||
return value
|
||||
|
||||
@staticmethod
|
||||
def parse_dataset_fields(dataset_fields: str) -> list[str]:
|
||||
"""Convert a string of comma-separated fields into a list of fields."""
|
||||
dataset_fields = dataset_fields.replace("'", "").replace('"', "").replace("`", "")
|
||||
return [field.strip() for field in dataset_fields.split(",")]
|
||||
|
||||
@staticmethod
|
||||
def flatten(d: dict) -> dict:
|
||||
"""Flatten a nested dictionary."""
|
||||
|
||||
def items():
|
||||
for key, value in d.items():
|
||||
if isinstance(value, dict):
|
||||
for subkey, subvalue in ApifyActorsComponent.flatten(value).items():
|
||||
yield key + "_" + subkey, subvalue
|
||||
else:
|
||||
yield key, value
|
||||
|
||||
return dict(items())
|
||||
@ -1,6 +1,11 @@
|
||||
from typing import Any
|
||||
|
||||
from langflow.base.io.chat import ChatComponent
|
||||
from langflow.inputs import BoolInput
|
||||
from langflow.io import DropdownInput, MessageInput, MessageTextInput, Output
|
||||
from langflow.inputs.inputs import HandleInput
|
||||
from langflow.io import DropdownInput, MessageTextInput, Output
|
||||
from langflow.schema.data import Data
|
||||
from langflow.schema.dataframe import DataFrame
|
||||
from langflow.schema.message import Message
|
||||
from langflow.schema.properties import Source
|
||||
from langflow.utils.constants import (
|
||||
@ -18,10 +23,12 @@ class ChatOutput(ChatComponent):
|
||||
minimized = True
|
||||
|
||||
inputs = [
|
||||
MessageInput(
|
||||
HandleInput(
|
||||
name="input_value",
|
||||
display_name="Text",
|
||||
info="Message to be passed as output.",
|
||||
input_types=["Data", "DataFrame", "Message"],
|
||||
required=True,
|
||||
),
|
||||
BoolInput(
|
||||
name="should_store_message",
|
||||
@ -76,6 +83,13 @@ class ChatOutput(ChatComponent):
|
||||
info="The text color of the name",
|
||||
advanced=True,
|
||||
),
|
||||
BoolInput(
|
||||
name="clean_data",
|
||||
display_name="Basic Clean Data",
|
||||
value=True,
|
||||
info="Whether to clean the data",
|
||||
advanced=True,
|
||||
),
|
||||
]
|
||||
outputs = [
|
||||
Output(
|
||||
@ -92,16 +106,35 @@ class ChatOutput(ChatComponent):
|
||||
if display_name:
|
||||
source_dict["display_name"] = display_name
|
||||
if source:
|
||||
source_dict["source"] = source
|
||||
# Handle case where source is a ChatOpenAI object
|
||||
if hasattr(source, "model_name"):
|
||||
source_dict["source"] = source.model_name
|
||||
elif hasattr(source, "model"):
|
||||
source_dict["source"] = str(source.model)
|
||||
else:
|
||||
source_dict["source"] = str(source)
|
||||
return Source(**source_dict)
|
||||
|
||||
async def message_response(self) -> Message:
|
||||
# First convert the input to string if needed
|
||||
text = self.convert_to_string()
|
||||
|
||||
# Get source properties
|
||||
source, icon, display_name, source_id = self.get_properties_from_source_component()
|
||||
background_color = self.background_color
|
||||
text_color = self.text_color
|
||||
if self.chat_icon:
|
||||
icon = self.chat_icon
|
||||
message = self.input_value if isinstance(self.input_value, Message) else Message(text=self.input_value)
|
||||
|
||||
# Create or use existing Message object
|
||||
if isinstance(self.input_value, Message):
|
||||
message = self.input_value
|
||||
# Update message properties
|
||||
message.text = text
|
||||
else:
|
||||
message = Message(text=text)
|
||||
|
||||
# Set message properties
|
||||
message.sender = self.sender
|
||||
message.sender_name = self.sender_name
|
||||
message.session_id = self.session_id
|
||||
@ -110,12 +143,54 @@ class ChatOutput(ChatComponent):
|
||||
message.properties.icon = icon
|
||||
message.properties.background_color = background_color
|
||||
message.properties.text_color = text_color
|
||||
if self.session_id and isinstance(message, Message) and self.should_store_message:
|
||||
stored_message = await self.send_message(
|
||||
message,
|
||||
)
|
||||
|
||||
# Store message if needed
|
||||
if self.session_id and self.should_store_message:
|
||||
stored_message = await self.send_message(message)
|
||||
self.message.value = stored_message
|
||||
message = stored_message
|
||||
|
||||
self.status = message
|
||||
return message
|
||||
|
||||
def _validate_input(self) -> None:
|
||||
"""Validate the input data and raise ValueError if invalid."""
|
||||
if self.input_value is None:
|
||||
msg = "Input data cannot be None"
|
||||
raise ValueError(msg)
|
||||
if not isinstance(self.input_value, Data | DataFrame | Message | str | list):
|
||||
msg = f"Expected Data or DataFrame or Message or str, got {type(self.input_value).__name__}"
|
||||
raise TypeError(msg)
|
||||
|
||||
def _safe_convert(self, data: Any) -> str:
|
||||
"""Safely convert input data to string."""
|
||||
try:
|
||||
if isinstance(data, str):
|
||||
return data
|
||||
if isinstance(data, Message):
|
||||
return data.get_text()
|
||||
if isinstance(data, Data):
|
||||
if data.get_text() is None:
|
||||
msg = "Empty Data object"
|
||||
raise ValueError(msg)
|
||||
return data.get_text()
|
||||
if isinstance(data, DataFrame):
|
||||
if self.clean_data:
|
||||
# Remove empty rows
|
||||
data = data.dropna(how="all")
|
||||
# Remove empty lines in each cell
|
||||
data = data.replace(r"^\s*$", "", regex=True)
|
||||
# Replace multiple newlines with a single newline
|
||||
data = data.replace(r"\n+", "\n", regex=True)
|
||||
return data.to_markdown(index=False)
|
||||
return str(data)
|
||||
except (ValueError, TypeError, AttributeError) as e:
|
||||
msg = f"Error converting data: {e!s}"
|
||||
raise ValueError(msg) from e
|
||||
|
||||
def convert_to_string(self) -> str:
|
||||
"""Convert input data to string with proper error handling."""
|
||||
self._validate_input()
|
||||
if isinstance(self.input_value, list):
|
||||
return "\n".join([self._safe_convert(item) for item in self.input_value])
|
||||
return self._safe_convert(self.input_value)
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@ -0,0 +1,13 @@
|
||||
# Apify Actors component tests
|
||||
|
||||
This component was tested manually with various Apify Actors. The component was tested with manual runs and with an AI Agent as a tool.
|
||||
|
||||
## Test cases
|
||||
|
||||
### Run Apify Actors manually
|
||||
Apify Actor input was manually configured and the component was run to retrieve data.
|
||||
When invalid input was provided, the component returned an error message with details.
|
||||
|
||||
### Run Apify Actors with AI Agent
|
||||
Multiple Apify Actors components with different Actors were connected to an AI Agent.
|
||||
The agent was given a task that required running multiple Apify Actors to complete.
|
||||
@ -0,0 +1,98 @@
|
||||
import pytest
|
||||
from langflow.components.outputs import ChatOutput
|
||||
from langflow.schema.data import Data
|
||||
from langflow.schema.dataframe import DataFrame
|
||||
from langflow.schema.message import Message
|
||||
from langflow.utils.constants import MESSAGE_SENDER_AI, MESSAGE_SENDER_NAME_AI
|
||||
|
||||
from tests.base import ComponentTestBaseWithClient
|
||||
|
||||
|
||||
@pytest.mark.usefixtures("client")
|
||||
class TestChatOutput(ComponentTestBaseWithClient):
|
||||
@pytest.fixture
|
||||
def component_class(self):
|
||||
return ChatOutput
|
||||
|
||||
@pytest.fixture
|
||||
def default_kwargs(self):
|
||||
return {
|
||||
"input_value": "Hello, how are you?",
|
||||
"should_store_message": True,
|
||||
"sender": MESSAGE_SENDER_AI,
|
||||
"sender_name": MESSAGE_SENDER_NAME_AI,
|
||||
"session_id": "test_session_123",
|
||||
"data_template": "{text}",
|
||||
"background_color": "#f0f0f0",
|
||||
"chat_icon": "🤖",
|
||||
"text_color": "#000000",
|
||||
"clean_data": True,
|
||||
}
|
||||
|
||||
@pytest.fixture
|
||||
def file_names_mapping(self):
|
||||
return [
|
||||
{"version": "1.0.19", "module": "outputs", "file_name": "ChatOutput"},
|
||||
{"version": "1.1.0", "module": "outputs", "file_name": "chat"},
|
||||
{"version": "1.1.1", "module": "outputs", "file_name": "chat"},
|
||||
]
|
||||
|
||||
async def test_process_string_input(self, component_class, default_kwargs):
|
||||
"""Test processing a simple string input."""
|
||||
component = component_class(**default_kwargs)
|
||||
input_text = "Hello, this is a test message"
|
||||
component.input_value = input_text
|
||||
result = await component.message_response()
|
||||
assert result.text == input_text
|
||||
assert result.sender == MESSAGE_SENDER_AI
|
||||
assert result.sender_name == MESSAGE_SENDER_NAME_AI
|
||||
|
||||
async def test_process_data_input(self, component_class, default_kwargs):
|
||||
"""Test processing a Data object input."""
|
||||
component = component_class(**default_kwargs)
|
||||
data = Data(text="Test data message")
|
||||
component.input_value = data
|
||||
result = await component.message_response()
|
||||
assert result.text == "Test data message"
|
||||
assert result.sender == MESSAGE_SENDER_AI
|
||||
|
||||
async def test_process_dataframe_input(self, component_class, default_kwargs):
|
||||
"""Test processing a DataFrame input."""
|
||||
component = component_class(**default_kwargs)
|
||||
sample_df = DataFrame(data={"col1": ["A", "B"], "col2": [1, 2]})
|
||||
component.input_value = sample_df
|
||||
result = await component.message_response()
|
||||
assert "col1" in result.text
|
||||
assert "col2" in result.text
|
||||
assert "A" in result.text
|
||||
assert "B" in result.text
|
||||
|
||||
async def test_process_message_input(self, component_class, default_kwargs):
|
||||
"""Test processing a Message object input."""
|
||||
component = component_class(**default_kwargs)
|
||||
message = Message(text="Test message content")
|
||||
component.input_value = message
|
||||
result = await component.message_response()
|
||||
assert result.text == "Test message content"
|
||||
assert result.sender == MESSAGE_SENDER_AI
|
||||
|
||||
async def test_process_list_input(self, component_class, default_kwargs):
|
||||
"""Test processing a list of inputs."""
|
||||
component = component_class(**default_kwargs)
|
||||
input_list = ["First message", Data(text="Second message"), Message(text="Third message")]
|
||||
component.input_value = input_list
|
||||
result = await component.message_response()
|
||||
assert "First message" in result.text
|
||||
assert "Second message" in result.text
|
||||
assert "Third message" in result.text
|
||||
|
||||
async def test_invalid_input(self, component_class, default_kwargs):
|
||||
"""Test handling of invalid input."""
|
||||
component = component_class(**default_kwargs)
|
||||
component.input_value = None
|
||||
with pytest.raises(ValueError, match="Input data cannot be None"):
|
||||
await component.message_response()
|
||||
|
||||
component.input_value = 123 # Invalid type
|
||||
with pytest.raises(TypeError, match="Expected Data or DataFrame or Message or str"):
|
||||
await component.message_response()
|
||||
@ -1,37 +1,7 @@
|
||||
import pytest
|
||||
from langflow.components.outputs import ChatOutput, TextOutputComponent
|
||||
from langflow.utils.constants import MESSAGE_SENDER_AI, MESSAGE_SENDER_NAME_AI
|
||||
from langflow.components.outputs import TextOutputComponent
|
||||
|
||||
from tests.base import ComponentTestBaseWithClient, ComponentTestBaseWithoutClient
|
||||
|
||||
|
||||
@pytest.mark.usefixtures("client")
|
||||
class TestChatOutput(ComponentTestBaseWithClient):
|
||||
@pytest.fixture
|
||||
def component_class(self):
|
||||
return ChatOutput
|
||||
|
||||
@pytest.fixture
|
||||
def default_kwargs(self):
|
||||
return {
|
||||
"input_value": "Hello, how are you?",
|
||||
"should_store_message": True,
|
||||
"sender": MESSAGE_SENDER_AI,
|
||||
"sender_name": MESSAGE_SENDER_NAME_AI,
|
||||
"session_id": "test_session_123",
|
||||
"data_template": "{text}",
|
||||
"background_color": "#f0f0f0",
|
||||
"chat_icon": "🤖",
|
||||
"text_color": "#000000",
|
||||
}
|
||||
|
||||
@pytest.fixture
|
||||
def file_names_mapping(self):
|
||||
return [
|
||||
{"version": "1.0.19", "module": "outputs", "file_name": "ChatOutput"},
|
||||
{"version": "1.1.0", "module": "outputs", "file_name": "chat"},
|
||||
{"version": "1.1.1", "module": "outputs", "file_name": "chat"},
|
||||
]
|
||||
from tests.base import ComponentTestBaseWithoutClient
|
||||
|
||||
|
||||
class TestTextOutputComponent(ComponentTestBaseWithoutClient):
|
||||
|
||||
27
src/frontend/src/icons/Apify/Apify.jsx
Normal file
27
src/frontend/src/icons/Apify/Apify.jsx
Normal file
@ -0,0 +1,27 @@
|
||||
const SvgApifyLogo = (props) => (
|
||||
<svg
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
xmlSpace="preserve"
|
||||
x="0"
|
||||
y="0"
|
||||
version="1.1"
|
||||
viewBox="0 0 1080 1080"
|
||||
width="1.1em"
|
||||
height="1.1em"
|
||||
{...props}
|
||||
>
|
||||
<path
|
||||
fill="#97d700"
|
||||
d="M189.7 149c-75.3 10.7-127.1 79.6-116.5 154.1l81 576.8L493 106.4z"
|
||||
></path>
|
||||
<path
|
||||
fill="#71c5e8"
|
||||
d="M1008 629.2 976 186c-5.7-75.3-71-131.4-145.6-126.4-2.8 0-6.4.7-9.2.7L690.4 78.7l287.7 645.7c21.3-26.3 32-60.4 29.9-95.2"
|
||||
></path>
|
||||
<path
|
||||
fill="#ff9013"
|
||||
d="M277 1019.9c23.4 2.8 46.9-.7 68.2-9.9l493.7-208.8L604.5 274z"
|
||||
></path>
|
||||
</svg>
|
||||
);
|
||||
export default SvgApifyLogo;
|
||||
14
src/frontend/src/icons/Apify/apify.svg
Normal file
14
src/frontend/src/icons/Apify/apify.svg
Normal file
@ -0,0 +1,14 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Generator: Adobe Illustrator 24.1.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
|
||||
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
|
||||
viewBox="0 0 1080 1080" style="enable-background:new 0 0 1080 1080;" xml:space="preserve">
|
||||
<style type="text/css">
|
||||
.st0{fill:#97D700;}
|
||||
.st1{fill:#71C5E8;}
|
||||
.st2{fill:#FF9013;}
|
||||
</style>
|
||||
<path class="st0" d="M189.7,149c-75.3,10.7-127.1,79.6-116.5,154.1l81,576.8L493,106.4L189.7,149z"/>
|
||||
<path class="st1" d="M1008,629.2l-32-443.2c-5.7-75.3-71-131.4-145.6-126.4c-2.8,0-6.4,0.7-9.2,0.7L690.4,78.7l287.7,645.7
|
||||
C999.4,698.1,1010.1,664,1008,629.2z"/>
|
||||
<path class="st2" d="M277,1019.9c23.4,2.8,46.9-0.7,68.2-9.9l493.7-208.8L604.5,274L277,1019.9z"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 812 B |
8
src/frontend/src/icons/Apify/index.tsx
Normal file
8
src/frontend/src/icons/Apify/index.tsx
Normal file
@ -0,0 +1,8 @@
|
||||
import React, { forwardRef } from "react";
|
||||
import SvgApifyLogo from "./Apify";
|
||||
|
||||
export const ApifyIcon = forwardRef<SVGSVGElement, React.PropsWithChildren<{}>>(
|
||||
(props, ref) => {
|
||||
return <SvgApifyLogo ref={ref} {...props} />;
|
||||
},
|
||||
);
|
||||
@ -240,6 +240,7 @@ import { AWSIcon } from "../icons/AWS";
|
||||
import { AgentQLIcon } from "../icons/AgentQL";
|
||||
import { AirbyteIcon } from "../icons/Airbyte";
|
||||
import { AnthropicIcon } from "../icons/Anthropic";
|
||||
import { ApifyIcon } from "../icons/Apify";
|
||||
import { ArXivIcon } from "../icons/ArXiv";
|
||||
import { ArizeIcon } from "../icons/Arize";
|
||||
import { AssemblyAIIcon } from "../icons/AssemblyAI";
|
||||
@ -318,6 +319,7 @@ import { MistralIcon } from "../icons/mistral";
|
||||
import { SupabaseIcon } from "../icons/supabase";
|
||||
import { XAIIcon } from "../icons/xAI";
|
||||
import { iconsType } from "../types/components";
|
||||
|
||||
export const BG_NOISE =
|
||||
"url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADIAAAAyCAMAAAAp4XiDAAAAUVBMVEWFhYWDg4N3d3dtbW17e3t1dXWBgYGHh4d5eXlzc3OLi4ubm5uVlZWPj4+NjY19fX2JiYl/f39ra2uRkZGZmZlpaWmXl5dvb29xcXGTk5NnZ2c8TV1mAAAAG3RSTlNAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEAvEOwtAAAFVklEQVR4XpWWB67c2BUFb3g557T/hRo9/WUMZHlgr4Bg8Z4qQgQJlHI4A8SzFVrapvmTF9O7dmYRFZ60YiBhJRCgh1FYhiLAmdvX0CzTOpNE77ME0Zty/nWWzchDtiqrmQDeuv3powQ5ta2eN0FY0InkqDD73lT9c9lEzwUNqgFHs9VQce3TVClFCQrSTfOiYkVJQBmpbq2L6iZavPnAPcoU0dSw0SUTqz/GtrGuXfbyyBniKykOWQWGqwwMA7QiYAxi+IlPdqo+hYHnUt5ZPfnsHJyNiDtnpJyayNBkF6cWoYGAMY92U2hXHF/C1M8uP/ZtYdiuj26UdAdQQSXQErwSOMzt/XWRWAz5GuSBIkwG1H3FabJ2OsUOUhGC6tK4EMtJO0ttC6IBD3kM0ve0tJwMdSfjZo+EEISaeTr9P3wYrGjXqyC1krcKdhMpxEnt5JetoulscpyzhXN5FRpuPHvbeQaKxFAEB6EN+cYN6xD7RYGpXpNndMmZgM5Dcs3YSNFDHUo2LGfZuukSWyUYirJAdYbF3MfqEKmjM+I2EfhA94iG3L7uKrR+GdWD73ydlIB+6hgref1QTlmgmbM3/LeX5GI1Ux1RWpgxpLuZ2+I+IjzZ8wqE4nilvQdkUdfhzI5QDWy+kw5Wgg2pGpeEVeCCA7b85BO3F9DzxB3cdqvBzWcmzbyMiqhzuYqtHRVG2y4x+KOlnyqla8AoWWpuBoYRxzXrfKuILl6SfiWCbjxoZJUaCBj1CjH7GIaDbc9kqBY3W/Rgjda1iqQcOJu2WW+76pZC9QG7M00dffe9hNnseupFL53r8F7YHSwJWUKP2q+k7RdsxyOB11n0xtOvnW4irMMFNV4H0uqwS5ExsmP9AxbDTc9JwgneAT5vTiUSm1E7BSflSt3bfa1tv8Di3R8n3Af7MNWzs49hmauE2wP+ttrq+AsWpFG2awvsuOqbipWHgtuvuaAE+A1Z/7gC9hesnr+7wqCwG8c5yAg3AL1fm8T9AZtp/bbJGwl1pNrE7RuOX7PeMRUERVaPpEs+yqeoSmuOlokqw49pgomjLeh7icHNlG19yjs6XXOMedYm5xH2YxpV2tc0Ro2jJfxC50ApuxGob7lMsxfTbeUv07TyYxpeLucEH1gNd4IKH2LAg5TdVhlCafZvpskfncCfx8pOhJzd76bJWeYFnFciwcYfubRc12Ip/ppIhA1/mSZ/RxjFDrJC5xifFjJpY2Xl5zXdguFqYyTR1zSp1Y9p+tktDYYSNflcxI0iyO4TPBdlRcpeqjK/piF5bklq77VSEaA+z8qmJTFzIWiitbnzR794USKBUaT0NTEsVjZqLaFVqJoPN9ODG70IPbfBHKK+/q/AWR0tJzYHRULOa4MP+W/HfGadZUbfw177G7j/OGbIs8TahLyynl4X4RinF793Oz+BU0saXtUHrVBFT/DnA3ctNPoGbs4hRIjTok8i+algT1lTHi4SxFvONKNrgQFAq2/gFnWMXgwffgYMJpiKYkmW3tTg3ZQ9Jq+f8XN+A5eeUKHWvJWJ2sgJ1Sop+wwhqFVijqWaJhwtD8MNlSBeWNNWTa5Z5kPZw5+LbVT99wqTdx29lMUH4OIG/D86ruKEauBjvH5xy6um/Sfj7ei6UUVk4AIl3MyD4MSSTOFgSwsH/QJWaQ5as7ZcmgBZkzjjU1UrQ74ci1gWBCSGHtuV1H2mhSnO3Wp/3fEV5a+4wz//6qy8JxjZsmxxy5+4w9CDNJY09T072iKG0EnOS0arEYgXqYnXcYHwjTtUNAcMelOd4xpkoqiTYICWFq0JSiPfPDQdnt+4/wuqcXY47QILbgAAAABJRU5ErkJggg==)";
|
||||
|
||||
@ -520,6 +522,8 @@ export const SIDEBAR_CATEGORIES = [
|
||||
];
|
||||
|
||||
export const SIDEBAR_BUNDLES = [
|
||||
// Add apify
|
||||
{ display_name: "Apify", name: "apify", icon: "Apify" },
|
||||
{ display_name: "LangChain", name: "langchain_utilities", icon: "LangChain" },
|
||||
{ display_name: "AgentQL", name: "agentql", icon: "AgentQL" },
|
||||
{ display_name: "AssemblyAI", name: "assemblyai", icon: "AssemblyAI" },
|
||||
@ -741,6 +745,7 @@ export const nodeIconsLucide: iconsType = {
|
||||
SearchAPI: SearchAPIIcon,
|
||||
Wikipedia: WikipediaIcon,
|
||||
Arize: ArizeIcon,
|
||||
Apify: ApifyIcon,
|
||||
|
||||
//Node Icons
|
||||
model_specs: FileSliders,
|
||||
|
||||
25
uv.lock
generated
25
uv.lock
generated
@ -228,6 +228,29 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/7a/4daaf3b6c08ad7ceffea4634ec206faeff697526421c20f07628c7372156/anyio-4.7.0-py3-none-any.whl", hash = "sha256:ea60c3723ab42ba6fff7e8ccb0488c898ec538ff4df1f1d5e642c3601d07e352", size = 93052 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "apify-client"
|
||||
version = "1.8.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "apify-shared" },
|
||||
{ name = "httpx" },
|
||||
{ name = "more-itertools" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/9c/d7/37d955bff90f6dfc353642e4b4075db3dd9abf8f5111ed35152be3dfc3da/apify_client-1.8.1.tar.gz", hash = "sha256:2be1be7879570655bddeebf126833efe94cabb95b3755592845e92c20c70c674", size = 48422 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/15/0b/42bb740ba9ff4cd540cc3c22687e3c07a823a5bdecb18a714de6df183a3a/apify_client-1.8.1-py3-none-any.whl", hash = "sha256:cfa6df3816c436204e37457fba28981a0ef6a7602cde372463f0f078eee64747", size = 73532 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "apify-shared"
|
||||
version = "1.2.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d5/12/3c6639116aba2b851b38bfcd5d81bb91f733f041c12c1d547028bd075a08/apify_shared-1.2.1.tar.gz", hash = "sha256:986557e2b01c584aa57258fb4af83d32ecb6979c0d804ccbfda7c0e79a2d00b1", size = 13597 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/31/ff/9d60edc7602587a2e83f86687c2a1794d6ec7711599dbd0fb075e12d6abd/apify_shared-1.2.1-py3-none-any.whl", hash = "sha256:cee136729c41c215796d8ca9f7b2e5736995d44f9471663d61827d16f592df9b", size = 12404 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "appdirs"
|
||||
version = "1.4.4"
|
||||
@ -4033,6 +4056,7 @@ source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "ag2" },
|
||||
{ name = "aiofile" },
|
||||
{ name = "apify-client" },
|
||||
{ name = "arize-phoenix-otel" },
|
||||
{ name = "assemblyai" },
|
||||
{ name = "astra-assistants", extra = ["tools"] },
|
||||
@ -4191,6 +4215,7 @@ dev = [
|
||||
requires-dist = [
|
||||
{ name = "ag2", specifier = ">=0.1.0" },
|
||||
{ name = "aiofile", specifier = ">=3.9.0,<4.0.0" },
|
||||
{ name = "apify-client", specifier = ">=1.8.1" },
|
||||
{ name = "arize-phoenix-otel", specifier = ">=0.6.1" },
|
||||
{ name = "assemblyai", specifier = "==0.35.1" },
|
||||
{ name = "astra-assistants", extras = ["tools"], specifier = "~=2.2.9" },
|
||||
|
||||
Reference in New Issue
Block a user