feat: Add langflow-stepflow package (#12015)

* feat: Add langflow-stepflow package

Introduces a new workspace package `src/langflow-stepflow/` that ports
the Stepflow integration from the Stepflow repository into the Langflow
codebase.

The package has two submodules:
- `translation/`: translates Langflow flow JSON to Stepflow flow
  definitions (ported from integrations/langflow/converter/)
- `worker/`: a Stepflow worker that executes individual Langflow
  components via lfx (ported from integrations/langflow/executor/)

Entry point: `python -m langflow_stepflow.worker` starts the HTTP worker
server for use by a Stepflow orchestrator.

No changes to existing Langflow code in this commit.

* fix: Address review feedback on langflow-stepflow package

- Widen Python version to >=3.10,<3.14 to match langflow-base
- Fix import sorting (ruff I001) and unused variable (ruff F841)
- Replace sk- prefixed test strings to avoid Gitleaks false positives
- Remove placeholder step reference resolution in component_tool
  (orchestrator resolves these before reaching the worker)
- Let exceptions propagate from component_tool_executor instead of
  returning error dicts that look like successful results
- Skip secret/password field defaults in tool input schemas
- Use LRU-style bounded cache (128) for compiled components
- Use asyncio.to_thread for sync component methods to avoid blocking
- Fix mutable NamedTuple default (list→tuple) in PlaceholderGraph
- Warn and skip deps with missing field mappings instead of silently
  falling back to "input" (which overwrites on multiple deps)
- Tighten _is_data_list to check __class_name__ == "Data" specifically
- Add comment explaining intentional teardown-before-init sequence
- Add integration test for example flows

* chore: Upgrade to Stepflow SDK 0.12.0 and adapt to lfx 0.3+ renames

- Bump stepflow-py and stepflow-orchestrator from >=0.10.0 to >=0.12.0
- Remove Python 3.11 environment markers (SDK now supports 3.10+)
- Replace server.run() with gRPC pull transport (run_grpc_worker)
  matching the upstream stepflow-langflow integration pattern
- Update Flow serialization from Pydantic model_dump to msgspec.to_builtins
- Remove Pydantic ValueExpr/actual_instance unwrapping (now plain dicts)
- Add _langflow_type_name() to map lfx 0.3+ class renames (JSON→Data,
  Table→DataFrame) back to canonical Langflow type names
- Fix DataFrame isinstance checks for lfx Table subclass
- Add missing README.md, tests/__init__.py, helpers package
- Add skip conditions for missing fixture files

* change to lfx schema

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* chore: Align langflow-stepflow ruff config with repo and format

- Set line-length = 120 to match root pyproject.toml (was 88, causing
  conflicts between pre-commit format hook and check hook)
- Run ruff format across all source and test files
- Add pragma: allowlist secret on test fixtures with fake API keys

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* chore: Rebuild component index

* [autofix.ci] apply automated fixes

* fix(langflow-stepflow): allow Python 3.14 in requires-python

Lift requires-python from <3.14 to <3.15 to match the rest of the
workspace and the lockfile. Without this, uv sync on Python 3.14
fails the Unit/LFX/Integration test jobs and the Docker build.

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: ogabrielluiz <gabriel@langflow.org>
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
This commit is contained in:
Ben Chambers
2026-05-18 14:47:20 -07:00
committed by GitHub
parent d1da768192
commit e08080507b
44 changed files with 10578 additions and 1768 deletions

View File

@ -96,6 +96,7 @@ members = [
"src/backend/base",
".",
"src/lfx",
"src/langflow-stepflow",
"src/sdk",
# langflow-extensions:bundle-members-start
"src/bundles/duckduckgo",

View File

@ -464,7 +464,7 @@
},
{
"name": "OpenDsStar",
"version": null
"version": "1.0.26"
}
],
"total_dependencies": 4

View File

@ -0,0 +1,3 @@
# langflow-stepflow
Stepflow execution backend for Langflow.

View File

@ -0,0 +1,94 @@
[project]
name = "langflow-stepflow"
version = "0.1.0"
description = "Stepflow execution backend for Langflow"
readme = "README.md"
license = {text = "Apache-2.0"}
authors = [
{name = "Langflow Contributors"}
]
requires-python = ">=3.10,<3.15"
keywords = [
"workflow",
"langflow",
"stepflow",
"ai",
"automation",
]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
]
dependencies = [
"stepflow-py>=0.12.0",
"stepflow-orchestrator>=0.12.0",
"pyyaml>=6.0",
"msgspec>=0.19.0",
"jsonschema>=4.17.0",
"pydantic>=2.0.0",
"lfx",
"pandas>=2.0.0",
"langchain_core>=0.1.0",
"nest-asyncio>=1.6.0",
"opensearch-py>=2.8.0",
]
[dependency-groups]
dev = [
"pytest>=7.4.0",
"pytest-asyncio>=0.26.0",
"pytest-cov>=4.1.0",
"pytest-mock>=3.14.1",
"ruff>=0.9.4",
"mypy==1.19.1",
"types-PyYAML>=6.0.0",
"pandas-stubs>=2.0.0",
"httpx>=0.25.0",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/langflow_stepflow"]
[tool.ruff]
target-version = "py310"
line-length = 120
src = ["src", "tests"]
[tool.ruff.lint]
select = ["E", "F", "I", "W", "UP", "B", "C4"]
[tool.ruff.format]
quote-style = "double"
indent-style = "space"
docstring-code-format = true
[tool.pytest.ini_options]
asyncio_default_fixture_loop_scope = "function"
testpaths = ["tests"]
pythonpath = ["."]
python_files = ["test_*.py", "*_test.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
"--strict-markers",
"--disable-warnings",
"-ra",
]
markers = [
"slow: marks tests as slow",
"integration: marks tests as integration tests",
"real_execution: marks tests that use real Langflow component execution",
]

View File

@ -0,0 +1,12 @@
"""Langflow Stepflow Integration.
A Python package for integrating Langflow workflows with Stepflow,
providing translation and execution capabilities.
"""
from .translation.translator import LangflowConverter
__version__ = "0.1.0"
__all__ = [
"LangflowConverter",
]

View File

@ -0,0 +1,25 @@
"""Custom exception types for Langflow integration."""
class LangflowIntegrationError(Exception):
"""Base exception for Langflow integration errors."""
pass
class ConversionError(LangflowIntegrationError):
"""Error during Langflow to Stepflow conversion."""
pass
class ValidationError(LangflowIntegrationError):
"""Error during workflow validation."""
pass
class ExecutionError(LangflowIntegrationError):
"""Error during component execution."""
pass

View File

@ -0,0 +1,5 @@
"""Langflow to Stepflow translation components."""
from .translator import LangflowConverter
__all__ = ["LangflowConverter"]

View File

@ -0,0 +1,75 @@
"""Dependency analysis for Langflow workflows."""
from typing import Any
class DependencyAnalyzer:
"""Analyzes Langflow edges to build step dependency graphs."""
def build_dependency_graph(self, edges: list[dict[str, Any]]) -> dict[str, list[str]]:
"""Build a dependency graph from Langflow edges.
Args:
edges: List of Langflow edge objects
Returns:
Dict mapping step IDs to their dependencies
"""
dependencies: dict[str, list[str]] = {}
for edge in edges:
source = edge.get("source")
target = edge.get("target")
if source and target:
if target not in dependencies:
dependencies[target] = []
dependencies[target].append(source)
return dependencies
def get_execution_order(self, dependencies: dict[str, list[str]]) -> list[str]:
"""Get topological execution order for nodes.
Args:
dependencies: Dependency graph
Returns:
List of node IDs in execution order
Raises:
ValueError: If circular dependencies are detected
"""
# Topological sort using Kahn's algorithm
in_degree: dict[str, int] = {}
all_nodes = set()
# Collect all nodes and calculate in-degrees
for target, sources in dependencies.items():
all_nodes.add(target)
in_degree[target] = in_degree.get(target, 0) + len(sources)
for source in sources:
all_nodes.add(source)
if source not in in_degree:
in_degree[source] = 0
# Find nodes with no incoming edges
queue = [node for node in all_nodes if in_degree[node] == 0]
result = []
while queue:
node = queue.pop(0)
result.append(node)
# Update in-degrees for dependent nodes
for target, sources in dependencies.items():
if node in sources:
in_degree[target] -= 1
if in_degree[target] == 0:
queue.append(target)
# Check for circular dependencies
if len(result) != len(all_nodes):
raise ValueError("Circular dependencies detected in workflow")
return result

View File

@ -0,0 +1,141 @@
"""Hash-to-module mappings for known Langflow core components.
This module contains mappings of code hashes to component module paths for
known Langflow core components. When a component's code_hash matches an entry
here and the module matches, we can use the core executor instead of compiling
custom code from a blob.
The hashes are derived from the Langflow workflow metadata and can be found in
the `metadata.code_hash` and `metadata.module` fields of each node.
"""
import logging
from dataclasses import dataclass
logger = logging.getLogger(__name__)
@dataclass
class KnownComponent:
"""A known core component mapping."""
code_hash: str
module: str # Full module path, e.g., "lfx.components.docling.DoclingInlineComponent"
description: str = ""
# Alternative module paths that are functionally equivalent
# (e.g., custom_components variants)
aliases: tuple[str, ...] = ()
# Mapping of code_hash -> KnownComponent
# These hashes are version-specific and may need updating when Langflow versions change
KNOWN_COMPONENTS: dict[str, KnownComponent] = {
# ChatInput component (lfx)
"f701f686b325": KnownComponent( # pragma: allowlist secret
code_hash="f701f686b325", # pragma: allowlist secret
module="lfx.components.input_output.chat.ChatInput",
description="Chat Input component",
),
# ChatOutput component (lfx)
"9647f4d2f4b4": KnownComponent(
code_hash="9647f4d2f4b4",
module="lfx.components.input_output.chat_output.ChatOutput",
description="Chat Output component",
),
# LanguageModelComponent (lfx)
"bb5f8714781b": KnownComponent(
code_hash="bb5f8714781b",
module="lfx.components.models.language_model.LanguageModelComponent",
description="Language Model component",
),
# Note: PromptComponent (langflow.components.prompts.prompt) is NOT included
# because the module doesn't exist in the lfx package - it must be compiled
# from custom code.
#
# Docling components (lfx)
# Note: Some workflows use custom_components.docling_serve which requires
# custom_code compilation. The lfx.components.docling.docling_remote module
# provides the standard DoclingRemoteComponent for connecting to docling-serve.
"d76b3853ceb4": KnownComponent( # pragma: allowlist secret
code_hash="d76b3853ceb4", # pragma: allowlist secret
module="lfx.components.docling.docling_inline.DoclingInlineComponent",
description="Docling inline document processing (local/sidecar)",
),
"26eeb513dded": KnownComponent(
code_hash="26eeb513dded",
module="lfx.components.docling.docling_remote.DoclingRemoteComponent",
description="Docling remote processing via docling-serve API",
),
# Custom "Docling Serve" variant used in production workflows
# Same class as DoclingRemoteComponent but with workflow-specific customizations
"5723576d00e5": KnownComponent(
code_hash="5723576d00e5",
module="lfx.components.docling.docling_remote.DoclingRemoteComponent",
description="Docling Serve (custom variant of DoclingRemoteComponent)",
aliases=("custom_components.docling_serve",),
),
"397fa38f89d7": KnownComponent(
code_hash="397fa38f89d7",
module="lfx.components.docling.chunk_docling_document.ChunkDoclingDocumentComponent",
description="Chunk DoclingDocument for RAG pipelines",
),
"4de16ddd37ac": KnownComponent(
code_hash="4de16ddd37ac",
module="lfx.components.docling.export_docling_document.ExportDoclingDocumentComponent",
description="Export DoclingDocument to markdown, html or other formats",
),
# EmbeddingModel component (lfx)
"0e2d6fe67a26": KnownComponent(
code_hash="0e2d6fe67a26",
module="lfx.components.models_and_agents.embedding_model.EmbeddingModelComponent",
description="Embedding Model - generate embeddings using OpenAI/Ollama",
aliases=("custom_components.embedding_model",),
),
# Add more known components here as needed
}
def lookup_known_component(code_hash: str, module: str | None = None) -> KnownComponent | None:
"""Look up a known component by code hash.
Args:
code_hash: The code_hash from workflow metadata
module: Optional module path to verify (safety check)
Returns:
KnownComponent if found and module matches (if provided), None otherwise
"""
known = KNOWN_COMPONENTS.get(code_hash)
if known is None:
return None
# If module provided, verify it matches (or is a known alias)
if module is not None and known.module != module:
# Check if module is a known alias
if module not in known.aliases:
# Hash collision or outdated mapping - log warning and return None
logger.warning(
f"Code hash {code_hash} matches known component {known.module} "
f"but workflow specifies {module}. Using custom_code executor."
)
return None
# Module is a known alias, proceed with the known component
logger.info(f"Code hash {code_hash} using alias {module} -> {known.module}")
return known
def module_to_path(module: str) -> str:
"""Convert module path to URL path segment.
Example: "lfx.components.docling.DoclingInlineComponent"
-> "lfx/components/docling/DoclingInlineComponent"
Args:
module: Full module path with dots
Returns:
URL path segment with slashes
"""
return module.replace(".", "/")

View File

@ -0,0 +1,516 @@
"""Process individual Langflow nodes into Stepflow steps."""
import datetime
import logging
from typing import Any
from stepflow_py.worker import FlowBuilder, Value
from ..exceptions import ConversionError
from .known_components import lookup_known_component, module_to_path
from .schema_mapper import SchemaMapper
logger = logging.getLogger(__name__)
def _sanitize_for_stepflow(obj: Any) -> Any:
"""Recursively convert non-JSON-serializable types to JSON-safe equivalents.
The Stepflow FlowBuilder only handles str, int, float, bool, None, list, dict,
and Value references. This converts datetime and other types to strings.
"""
if obj is None or isinstance(obj, (str, int, float, bool)):
return obj
if isinstance(obj, datetime.datetime):
return obj.isoformat()
if isinstance(obj, datetime.date):
return obj.isoformat()
if isinstance(obj, dict):
return {k: _sanitize_for_stepflow(v) for k, v in obj.items()}
if isinstance(obj, list):
return [_sanitize_for_stepflow(v) for v in obj]
return str(obj)
class NodeProcessor:
"""Processes individual Langflow nodes into Stepflow steps."""
def __init__(self):
"""Initialize node processor."""
self.schema_mapper = SchemaMapper()
self.variables: dict[str, dict[str, Any]] = {}
def reset(self):
"""Reset internal state for processing a new workflow."""
self.variables.clear()
def process_node(
self,
node: dict[str, Any],
dependencies: dict[str, list[str]],
builder: FlowBuilder,
node_output_refs: dict[str, Any],
field_mapping: dict[str, dict[str, str]],
output_mapping: dict[str, str],
) -> Any | None:
"""Process a Langflow node using flow builder architecture.
Args:
node: Langflow node object
dependencies: Dependency graph for all nodes
all_nodes: All nodes in the workflow
builder: FlowBuilder instance
node_output_refs: Mapping of node IDs to their output references
field_mapping: Mapping of target nodes to their input field names
from edges
output_mapping: Mapping of source node IDs to their selected output names
from edges
Returns:
Output reference for this node, or None if node should be skipped
"""
try:
node_id = node.get("id")
if not node_id:
raise ConversionError("Node missing ID")
# Check if this is a note or documentation node that should be skipped
node_type = node.get("type", "")
if node_type == "noteNode":
# This is a React Flow note node - skip it entirely
return None
node_data = node.get("data", {})
component_type = node_data.get("type", "")
# Skip nodes without a valid component type (documentation nodes, etc.)
if not component_type or component_type.strip() == "":
return None
# Generate step ID (clean up for Stepflow)
step_id = self._generate_step_id(node_id, component_type)
# Get node structure info for routing decisions
node_info = node_data.get("node", {})
template = node_info.get("template", {})
# Handle ChatInput/ChatOutput as I/O connection points (not processing
# steps)
if component_type == "ChatInput":
# ChatInput returns a reference to workflow input directly
return Value.input.add_path("message")
elif component_type == "ChatOutput":
# ChatOutput depends on another node - return that node's output
# reference
dependency_node_ids = dependencies.get(node_id, [])
if dependency_node_ids and dependency_node_ids[0] in node_output_refs:
return node_output_refs[dependency_node_ids[0]]
else:
# ChatOutput with no dependencies - return input passthrough
return Value.input.add_path("message")
# Component is a tool if it has tool_mode=True at the component level
is_tool_component = node_info.get("tool_mode", False)
if is_tool_component:
return self._create_tool_component_step(
node,
step_id,
builder,
dependencies,
node_output_refs,
field_mapping,
)
# For regular components, determine routing based on component type
custom_code = template.get("code", {}).get("value", "")
# Check for known core components via hash lookup
metadata = node_info.get("metadata", {})
code_hash = metadata.get("code_hash")
module = metadata.get("module")
known_component = None
if code_hash and module:
known_component = lookup_known_component(code_hash, module)
# Determine component path and inputs based on routing
if known_component:
# Known core component - use core executor (no blob needed)
component_path = f"/langflow/core/{module_to_path(known_component.module)}"
logger.debug(f"Routing {component_type} to core executor: {component_path}")
# Extract outputs and selected_output for core executor
outputs = node_info.get("outputs", [])
selected_output = output_mapping.get(node_id)
if not selected_output and outputs:
selected_output = outputs[0].get("name")
# Prepare template without code field
raw_template = {k: v for k, v in template.items() if k != "code"}
template_without_code = _sanitize_for_stepflow(raw_template)
step_input = {
"template": template_without_code,
"outputs": outputs,
"selected_output": selected_output,
"input": self._extract_runtime_inputs_for_builder(
node,
dependencies.get(node_id, []),
node_output_refs,
field_mapping,
),
}
elif custom_code:
# Component with custom code - use custom code executor
component_path = "/langflow/custom_code"
logger.debug(f"Routing {component_type} to custom_code executor")
# First create a blob step for the code
blob_data = self._prepare_udf_blob(node, component_type, output_mapping)
blob_step_id = f"{step_id}_blob"
blob_step_handle = builder.add_step(
id=blob_step_id,
component="/builtin/put_blob",
input_data={
"data": _sanitize_for_stepflow(blob_data),
"blob_type": "data",
},
must_execute=True,
)
# Create the custom code executor step that uses the blob
step_input = {
"blob_id": Value.step(blob_step_handle.id, "blob_id"),
"input": self._extract_runtime_inputs_for_builder(
node,
dependencies.get(node_id, []),
node_output_refs,
field_mapping,
),
}
else:
# All executable components should have custom code or be known
raise ConversionError(
f"Component {component_type} in node {node_id} has no custom code "
f"and is not a known core component."
)
# Add step to builder with proper ID and component path
step_id = self._generate_step_id(node_id, component_type)
step_handle = builder.add_step(
id=step_id,
component=component_path,
input_data=step_input,
must_execute=True,
)
# Return a reference to this step's output
return Value.step(step_handle.id, "result")
except Exception as e:
raise ConversionError(f"Error processing node {node.get('id', 'unknown')}: {e}") from e
def _generate_step_id(self, node_id: str, component_type: str) -> str:
"""Generate a clean step ID from node ID and type.
Args:
node_id: Original Langflow node ID
component_type: Component type name
Returns:
Clean step ID suitable for Stepflow
"""
# Always use the full node_id to ensure uniqueness
# For any node_id with a suffix, keep it to ensure uniqueness
# Preserve original case for case-sensitive comparisons
base_id = node_id
# Always use langflow prefix with the full base_id to guarantee uniqueness
return f"langflow_{base_id}"
def _prepare_udf_blob(
self,
node: dict[str, Any],
component_type: str,
output_mapping: dict[str, str],
) -> dict[str, Any]:
"""Prepare enhanced UDF blob data for component execution.
Args:
node: Langflow node object
component_type: Component type name
output_mapping: Mapping of node IDs to their selected output names
Returns:
Enhanced UDF blob data with complete component information
"""
node_data = node.get("data", {})
node_info = node_data.get("node", {})
# Extract component code
template = node_info.get("template", {})
code = template.get("code", {}).get("value", "")
if not code:
raise ConversionError(f"No code found for component {component_type}")
# Extract comprehensive component metadata
outputs = node_data.get("outputs", [])
node_outputs = node_info.get("outputs", [])
# Use node outputs if available (more complete), fallback to data outputs
final_outputs = node_outputs if node_outputs else outputs
# Determine selected output - use output mapping from edges if available
selected_output = None
node_id = node.get("id")
if output_mapping and node_id in output_mapping:
# Use the output specified in the edge
selected_output = output_mapping[node_id]
elif final_outputs:
# Fallback to the first output if no edge mapping found
selected_output = final_outputs[0].get("name")
# Extract additional component metadata from node_info
base_classes = node_info.get("base_classes", [])
display_name = node_info.get("display_name", component_type)
description = node_info.get("description", "")
documentation = node_info.get("documentation", "")
metadata = node_info.get("metadata", {})
# Extract field order for proper component initialization
field_order = node_info.get("field_order", [])
# Extract component icon and UI information
icon = node_info.get("icon", "")
# Prepare template (remove code field to avoid duplication)
prepared_template: dict[str, Any] = {}
for field_name, field_config in template.items():
if field_name != "code":
prepared_template[field_name] = field_config
# Return enhanced blob data with complete component information
blob_data = {
"code": code,
"template": prepared_template,
"component_type": component_type,
"outputs": final_outputs,
"selected_output": selected_output,
# Enhanced metadata for real component execution
"base_classes": base_classes,
"display_name": display_name,
"description": description,
"documentation": documentation,
"metadata": metadata,
"field_order": field_order,
"icon": icon,
}
# Enhanced blob created with component metadata
return blob_data
def _add_variable(self, name: str, type: str) -> Value:
"""Add variable to internal state."""
input_type = self.schema_mapper.langflow_to_json_schema[type]
if (existing := self.variables.get(name)) is not None:
assert input_type == existing["type"][0], (
f"Variable {name} has conflicting types: {existing['type'][0]} vs {input_type}"
)
else:
self.variables[name] = {
"type": [input_type, "null"],
"default": None,
"env_var": name,
}
# Use a literal null as default so that if the variable isn't provided at
# runtime, the value resolves to null unless populated from environment
# via --env-variables or populate_variables_from_env.
return Value.variable(name, default=Value.literal(None))
def _extract_runtime_inputs_for_builder(
self,
node: dict[str, Any],
dependency_node_ids: list[str],
node_output_refs: dict[str, Any],
field_mapping: dict[str, dict[str, str]],
) -> dict[str, Any]:
"""Extract runtime inputs for UDF components using flow builder architecture.
Args:
node: Langflow node object
dependency_node_ids: IDs of nodes this node depends on
node_output_refs: Mapping of node IDs to their output references
field_mapping: Mapping of target nodes to their input field names
from edges
Returns:
Dict of runtime inputs
"""
runtime_inputs: dict[str, Any] = {}
node_id = node.get("id")
assert node_id is not None, "Node ID should not be None"
# Use field mapping if available, otherwise fall back to generic input names
if (node_field_map := field_mapping.get(node_id, None)) is not None:
# Group inputs by field name to handle list fields like 'tools'
field_inputs: dict[str, Any] = {}
for dep_id in dependency_node_ids:
if dep_id in node_field_map and dep_id in node_output_refs:
field_name = node_field_map[dep_id]
# Handle list fields by collecting multiple inputs
if field_name not in field_inputs:
field_inputs[field_name] = []
field_inputs[field_name].append(node_output_refs[dep_id])
elif dep_id in node_output_refs:
# Fallback to generic name if no field mapping
runtime_inputs[f"input_{len(runtime_inputs)}"] = node_output_refs[dep_id]
# Convert to runtime inputs format
for field_name, inputs in field_inputs.items():
if len(inputs) == 1:
# Single input - use directly
runtime_inputs[field_name] = inputs[0]
else:
# Multiple inputs - create a list
runtime_inputs[field_name] = inputs
else:
# Fallback to old behavior for backwards compatibility
for i, dep_id in enumerate(dependency_node_ids):
if dep_id in node_output_refs:
runtime_inputs[f"input_{i}"] = node_output_refs[dep_id]
else:
# Fallback to workflow input if dependency not found
runtime_inputs[f"input_{i}"] = Value.input("$.message")
# Add session_id mapping for UDF components (like Memory) that need it
node_data = node.get("data", {})
node_info = node_data.get("node", {})
template = node_info.get("template", {})
node_id = node.get("id", "")
component_type = node_data.get("type", "")
# Check for session_id field that needs mapping
if "session_id" in template:
session_id_config = template["session_id"]
if isinstance(session_id_config, dict):
session_id_value = session_id_config.get("value")
if session_id_value == "" or session_id_value is None:
runtime_inputs["session_id"] = Value.input("$.session_id")
# Special case: Agent components need session_id even if not in template
# Agent uses self.graph.session_id for memory retrieval
if component_type == "Agent":
runtime_inputs["session_id"] = Value.input("$.session_id")
# Handle standalone File components with workflow input mapping
if not dependency_node_ids and component_type == "File":
# For standalone File components, map workflow file_path to path parameter
# Path parameter should match Langflow's FileInput expectations
# file_path should be a simple list of paths, not wrapped
runtime_inputs["path"] = Value.input("$.file_path")
# Handle load_from_database inputs by examining the node.
for field_name, field_config in template.items():
if not isinstance(field_config, dict):
continue
if field_config.get("load_from_db", False):
# This field should be loaded from the database.
runtime_inputs[field_name] = self._add_variable(field_config["value"], field_config["type"])
return runtime_inputs
def _create_tool_component_step(
self,
node: dict[str, Any],
step_id: str,
builder: FlowBuilder,
dependencies: dict[str, list[str]],
node_output_refs: dict[str, Any],
field_mapping: dict[str, dict[str, str]],
) -> Any:
"""Create a component_tool step for tool-mode components.
Args:
node: Langflow node object
step_id: Generated step ID
builder: FlowBuilder instance
dependencies: Dependency graph
node_output_refs: Node output references
field_mapping: Field mapping from edges
Returns:
Output reference for the tool wrapper step
"""
try:
node_id = node.get("id", "")
node_data = node.get("data", {})
component_type = node_data.get("type", "")
node_info = node_data.get("node", {})
# Extract component inputs from dependencies and field mapping
component_inputs = self._build_component_inputs(node_id, dependencies, node_output_refs, field_mapping)
# Create step that calls component_tool to create tool wrapper
step_handle = builder.add_step(
id=step_id,
component="/langflow/component_tool",
input_data={
"code": Value.literal(node_info), # Store entire component definition
"inputs": component_inputs, # Static inputs from workflow
"component_type": component_type,
},
must_execute=True,
)
# Return a reference to this step's output (the tool wrapper)
# The component_tool returns the tool wrapper under "result" field
return Value.step(step_handle.id, "result")
except Exception as e:
raise ConversionError(f"Error creating tool component step: {e}") from e
def _build_component_inputs(
self,
node_id: str,
dependencies: dict[str, list[str]],
node_output_refs: dict[str, Any],
field_mapping: dict[str, dict[str, str]],
) -> dict[str, Any]:
"""Build input dict for a component from its dependencies.
Args:
node_id: Target node ID
dependencies: Dependency graph
node_output_refs: Output references from other nodes
field_mapping: Field name mapping from edges
Returns:
Dict mapping input field names to their values/references
"""
inputs = {}
# Get dependencies for this node
deps = dependencies.get(node_id, [])
field_map = field_mapping.get(node_id, {})
for dep_node_id in deps:
if dep_node_id in node_output_refs:
field_name = field_map.get(dep_node_id)
if field_name is None:
logger.warning(
"No field mapping for dependency %s -> %s; edge may be missing targetHandle.fieldName",
dep_node_id,
node_id,
)
continue
# Map dependency output to input field
inputs[field_name] = node_output_refs[dep_node_id]
return inputs

View File

@ -0,0 +1,178 @@
"""Schema mapping between Langflow and Stepflow formats."""
from typing import Any
class SchemaMapper:
"""Maps schemas between Langflow and Stepflow formats."""
def __init__(self):
"""Initialize schema mapper with type mappings."""
self.langflow_to_json_schema = {
"str": "string",
"int": "integer",
"float": "number",
"bool": "boolean",
"list": "array",
"dict": "object",
"dropdown": "string",
"slider": "number",
"file": "string",
"code": "string",
"prompt": "string",
"multiline": "string",
}
# component_output_heuristics: Maps Langflow component types to their expected
# output data types. This is used as a fallback when component output metadata
# is not available in the node definition. For example, ChatInput components
# typically output Message objects, while VectorStore components output
# DataFrame objects. These heuristics help generate proper output schemas
# during conversion.
self.component_output_heuristics = {
"ChatInput": ["Message"],
"ChatOutput": ["Message"],
"LanguageModelComponent": ["Message"],
"PromptComponent": ["Message"],
"TextSplitter": ["Data"],
"DocumentLoader": ["Data"],
"VectorStore": ["DataFrame"],
"Embeddings": ["DataFrame"],
}
def extract_output_schema(self, node: dict[str, Any]) -> dict[str, Any]:
"""Extract output schema from a Langflow node.
Args:
node: Langflow node object
Returns:
JSON schema for the node's output
"""
node_data = node.get("data", {})
component_type = node_data.get("type", "")
# Method 1: Check node outputs metadata
outputs = node_data.get("outputs", [])
if outputs:
return self._convert_langflow_outputs_to_schema(outputs)
# Method 2: Check base classes
base_classes = node_data.get("base_classes", [])
if base_classes:
return self._convert_langflow_types_to_schema(base_classes)
# Method 3: Use component type heuristics
if component_type in self.component_output_heuristics:
types = self.component_output_heuristics[component_type]
return self._convert_langflow_types_to_schema(types)
# Fallback: generic object schema
return {"type": "object", "properties": {"result": {"type": "object"}}}
def extract_input_schema(self, node: dict[str, Any]) -> dict[str, Any]:
"""Extract input schema from a Langflow node template.
Args:
node: Langflow node object
Returns:
JSON schema for the node's inputs
"""
node_data = node.get("data", {})
template = node_data.get("node", {}).get("template", {})
properties = {}
required = []
for field_name, field_config in template.items():
if field_name.startswith("_") or not isinstance(field_config, dict):
continue
field_type = field_config.get("type", "str")
json_type = self.langflow_to_json_schema.get(field_type, "string")
field_info = field_config.get("info", "")
property = {
"type": json_type,
"description": field_info,
}
# Add enum for dropdown fields
if field_type == "dropdown" and "options" in field_config:
property["enum"] = field_config["options"]
# Add number constraints for sliders
if field_type == "slider":
if "range_spec" in field_config:
range_spec = field_config["range_spec"]
if "min" in range_spec:
property["minimum"] = range_spec["min"]
if "max" in range_spec:
property["maximum"] = range_spec["max"]
if field_config.get("password", False) or field_config.get("_input_type", "") == "SecretStrInput":
property["is_secret"] = True
if field_config.get("required", False):
required.append(field_name)
properties[field_name] = property
return {
"type": "object",
"properties": properties,
"required": required,
}
def _convert_langflow_outputs_to_schema(self, outputs: list[dict[str, Any]]) -> dict[str, Any]:
"""Convert Langflow outputs metadata to JSON schema."""
if not outputs:
return {"type": "object"}
# For now, use the first output
# TODO: Handle multiple outputs properly
first_output = outputs[0]
output_types = first_output.get("types", ["object"])
return self._convert_langflow_types_to_schema(output_types)
def _convert_langflow_types_to_schema(self, langflow_types: list[str]) -> dict[str, Any]:
"""Convert Langflow types to JSON schema.
Args:
langflow_types: List of Langflow type names
Returns:
JSON schema object
"""
if "Message" in langflow_types:
return {
"type": "object",
"properties": {
"text": {"type": "string"},
"sender": {"type": "string"},
"sender_name": {"type": "string"},
"type": {"type": "string", "const": "Message"},
},
}
elif "Data" in langflow_types:
return {
"type": "object",
"properties": {
"data": {"type": "object"},
"text_key": {"type": "string"},
"type": {"type": "string", "const": "Data"},
},
}
elif "DataFrame" in langflow_types:
return {
"type": "object",
"properties": {
"data": {"type": "array", "items": {"type": "object"}},
"type": {"type": "string", "const": "DataFrame"},
},
}
else:
# Generic object for unknown types
return {"type": "object", "properties": {"result": {"type": "object"}}}

View File

@ -0,0 +1,139 @@
"""Stepflow-level tweaks for Langflow component configuration.
This module provides utilities for applying tweaks to Stepflow workflows
that were translated from Langflow. Tweaks are applied by modifying the
input fields of Langflow UDF executor steps before execution.
Key features:
- Apply tweaks to translated Stepflow workflows at execution time
- Target UDF executor steps using original Langflow node IDs
- Modify step input fields directly (not blob data)
- Compatible with Langflow API tweak format
Note: Test utilities and environment variable integration are in
tests/helpers/tweaks_builder.py to keep this production code lean.
"""
import copy
from typing import Any
def apply_stepflow_tweaks_to_dict(
workflow_dict: dict[str, Any],
tweaks: dict[str, dict[str, Any]] | None = None,
) -> dict[str, Any]:
"""Apply tweaks to a Stepflow workflow dict.
Args:
workflow_dict: Stepflow workflow as dict (from YAML)
tweaks: Dict mapping langflow_node_id -> {field_name: new_value}
Returns:
Modified workflow dict with tweaks applied
Examples:
>>> tweaks = {
... "LanguageModelComponent-kBOja": {
... "api_key": "new_test_key", # pragma: allowlist secret
... "temperature": 0.8,
... }
... }
>>> modified_dict = apply_stepflow_tweaks_to_dict(workflow_dict, tweaks)
"""
if not tweaks:
return workflow_dict
# Deep copy to avoid mutating original
modified_dict = copy.deepcopy(workflow_dict)
for step_dict in modified_dict.get("steps", []):
step_id = step_dict.get("id", "")
component = step_dict.get("component", "")
# Check if this is a Langflow executor step (custom_code or core)
is_langflow_step = (
step_id.startswith("langflow_")
and not step_id.endswith("_blob")
and (component == "/langflow/custom_code" or component.startswith("/langflow/core/"))
)
if is_langflow_step:
# Extract Langflow node ID
langflow_node_id = step_id[len("langflow_") :]
if langflow_node_id in tweaks:
# Ensure step has input section
if "input" not in step_dict:
step_dict["input"] = {}
if "input" not in step_dict["input"]:
step_dict["input"]["input"] = {}
# Apply tweaks
for field_name, new_value in tweaks[langflow_node_id].items():
step_dict["input"]["input"][field_name] = new_value
return modified_dict
def convert_tweaks_to_overrides(
tweaks: dict[str, dict[str, Any]] | None = None,
) -> dict[str, dict[str, Any]] | None:
"""Convert Langflow tweaks format to Stepflow overrides format.
This function transforms tweaks from the early binding format (modifying flow)
to the late binding format (runtime overrides). Instead of modifying the flow
directly, we create overrides that are applied at execution time.
Args:
tweaks: Dict mapping langflow_node_id -> {field_name: new_value}
Returns:
Dict mapping step_id to merge_patch format with field overrides
or None if no tweaks provided
Examples:
>>> tweaks = {
... "LanguageModelComponent-kBOja": {
... "api_key": "new_test_key", # pragma: allowlist secret
... "temperature": 0.8,
... }
... }
>>> overrides = convert_tweaks_to_overrides(tweaks)
>>> print(overrides)
{
"langflow_LanguageModelComponent-kBOja": {
"$type": "merge_patch",
"value": {
"input": {
"api_key": "new_test_key", # pragma: allowlist secret
"temperature": 0.8,
}
}
}
}
"""
if not tweaks:
return None
overrides = {}
for langflow_node_id, field_tweaks in tweaks.items():
# Convert Langflow node ID to Stepflow step ID format
step_id = f"langflow_{langflow_node_id}"
# Create the override value structure that matches step.input structure
# We need to wrap the field tweaks in an "input.input" key to match how
# the current tweaks modify step.input["input"][field_name]
override_value = {"input": {"input": field_tweaks}}
# Create the override entry with merge patch type (snake_case for API)
overrides[step_id] = {"$type": "merge_patch", "value": override_value}
return overrides
# Export main functions for easy importing
__all__ = [
"apply_stepflow_tweaks_to_dict",
"convert_tweaks_to_overrides",
]

View File

@ -0,0 +1,416 @@
"""Main Langflow to Stepflow converter implementation."""
import json
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import yaml
from stepflow_py.worker import Flow, FlowBuilder, Value
from ..exceptions import ConversionError
from .dependency_analyzer import DependencyAnalyzer
from .node_processor import NodeProcessor
from .schema_mapper import SchemaMapper
@dataclass
class WorkflowAnalysis:
"""Typed analysis results from analyzing a Langflow workflow."""
node_count: int # Total number of nodes in the Langflow workflow
edge_count: int # Total number of connections/edges between nodes
# Map of component type names to their counts (e.g., {"ChatInput": 1, "OpenAI": 2})
component_types: dict[str, int]
dependencies: dict[str, list[str]] # Map of node IDs to lists of their dependency node IDs
potential_issues: list[str] # List of warnings or potential problems detected during analysis
class LangflowConverter:
"""Convert Langflow JSON workflows to Stepflow YAML workflows."""
def __init__(self):
"""Initialize the converter."""
self.dependency_analyzer = DependencyAnalyzer()
self.schema_mapper = SchemaMapper()
self.node_processor = NodeProcessor()
def convert_file(self, input_path: str | Path) -> str:
"""Convert a Langflow JSON file to Stepflow YAML.
Args:
input_path: Path to the Langflow JSON file
Returns:
Stepflow YAML as a string
Raises:
ConversionError: If conversion fails
ValidationError: If validation fails
"""
input_path = Path(input_path)
if not input_path.exists():
raise ConversionError(f"Input file not found: {input_path}")
try:
with open(input_path, encoding="utf-8") as f:
langflow_data = json.load(f)
except json.JSONDecodeError as e:
raise ConversionError(f"Invalid JSON in {input_path}: {e}") from e
except Exception as e:
raise ConversionError(f"Error reading {input_path}: {e}") from e
workflow = self.convert(langflow_data)
return self.to_yaml(workflow)
def convert(self, langflow_data: dict[str, Any]) -> Flow:
"""Convert Langflow data structure to Stepflow workflow.
Args:
langflow_data: Parsed Langflow JSON data
Returns:
Flow object
Raises:
ConversionError: If conversion fails
"""
self.node_processor.reset()
try:
# Extract main data structure
if "data" not in langflow_data:
raise ConversionError("Invalid Langflow JSON: missing 'data' key")
data = langflow_data["data"]
nodes = data.get("nodes", [])
edges = data.get("edges", [])
if not nodes:
raise ConversionError("No nodes found in Langflow workflow")
# Build dependency graph from edges
dependencies = self.dependency_analyzer.build_dependency_graph(edges)
# Get proper execution order using topological sort
execution_order = self.dependency_analyzer.get_execution_order(dependencies)
# Create field mapping from edges for proper UDF input handling
field_mapping = self._build_field_mapping_from_edges(edges)
# Create output mapping from edges to track which output is used from
# each component
output_mapping = self._build_output_mapping_from_edges(edges)
# Create node lookup for efficient processing
node_lookup = {node["id"]: node for node in nodes}
# Create FlowBuilder
builder = FlowBuilder(name=self._generate_workflow_name(langflow_data))
# Note: Skip setting input schema for now as Schema class doesn't
# support properties
# input_schema = self._generate_input_section(nodes)
# if input_schema:
# builder.set_input_schema(input_schema)
# Process nodes and collect output references
node_output_refs: dict[str, Any] = {} # node_id -> output reference
processed_nodes = set()
# First, process nodes in execution order
for node_id in execution_order:
if node_id in node_lookup:
output_ref = self.node_processor.process_node(
node_lookup[node_id],
dependencies,
builder,
node_output_refs,
field_mapping,
output_mapping,
)
if output_ref is not None:
node_output_refs[node_id] = output_ref
processed_nodes.add(node_id)
# Then, process any remaining nodes (nodes with no dependencies)
for node in nodes:
node_id = node["id"]
if node_id not in processed_nodes:
output_ref = self.node_processor.process_node(
node,
dependencies,
builder,
node_output_refs,
field_mapping,
output_mapping,
)
if output_ref is not None:
node_output_refs[node_id] = output_ref
# Set workflow output using incremental output building
self._build_flow_output(builder, nodes, dependencies, node_output_refs)
# Set variable schema
if self.node_processor.variables:
builder.set_variables_schema({"type": "object", "properties": self.node_processor.variables})
# Build and return the flow
flow = builder.build()
return flow
except ConversionError:
raise
except Exception as e:
raise ConversionError(f"Unexpected error during conversion: {e}") from e
def to_yaml(self, workflow: Flow) -> str:
"""Convert Flow to YAML string.
Args:
workflow: Flow object (msgspec Struct from stepflow_py.worker)
Returns:
YAML string
"""
try:
import msgspec
workflow_dict = msgspec.to_builtins(workflow)
# Generate clean YAML
return yaml.dump(
workflow_dict,
default_flow_style=False,
sort_keys=False,
allow_unicode=True,
width=120,
)
except Exception as e:
raise ConversionError(f"Error generating YAML: {e}") from e
def analyze(self, langflow_data: dict[str, Any]) -> WorkflowAnalysis:
"""Analyze Langflow workflow structure without conversion.
Args:
langflow_data: Parsed Langflow JSON data
Returns:
Typed analysis results
"""
try:
data = langflow_data.get("data", {})
nodes = data.get("nodes", [])
edges = data.get("edges", [])
# Basic statistics
analysis: dict[str, Any] = {
"node_count": len(nodes),
"edge_count": len(edges),
"component_types": {},
"dependencies": {},
"potential_issues": [],
}
# Analyze nodes
for node in nodes:
node_data = node.get("data", {})
component_type = node_data.get("type", "Unknown")
if component_type not in analysis["component_types"]:
analysis["component_types"][component_type] = 0
analysis["component_types"][component_type] += 1
# Check for potential issues
if not node.get("id"):
analysis["potential_issues"].append("Node missing ID")
if not node_data.get("node", {}).get("template"):
analysis["potential_issues"].append(f"Node {node.get('id', 'unknown')} missing template")
# Analyze dependencies
dependencies = self.dependency_analyzer.build_dependency_graph(edges)
return WorkflowAnalysis(
node_count=len(nodes),
edge_count=len(edges),
component_types=analysis["component_types"],
dependencies=dependencies,
potential_issues=analysis["potential_issues"],
)
except Exception as e:
raise ConversionError(f"Error analyzing workflow: {e}") from e
def _generate_workflow_name(self, langflow_data: dict[str, Any]) -> str:
"""Generate a workflow name from Langflow data."""
# Try to get name from various sources
if "name" in langflow_data:
name = langflow_data["name"]
return str(name) if name is not None else "Converted Langflow Workflow"
data = langflow_data.get("data", {})
if "name" in data:
name = data["name"]
return str(name) if name is not None else "Converted Langflow Workflow"
# Fallback to generic name
return "Converted Langflow Workflow"
def _build_field_mapping_from_edges(self, edges: list[dict[str, Any]]) -> dict[str, dict[str, str]]:
"""Build field mapping from edges for proper input handling.
Args:
edges: List of Langflow edges
Returns:
Dict mapping target_node_id -> {source_node_id -> target_field_name}
"""
field_mapping: dict[str, dict[str, str]] = {}
for edge in edges:
target_id = edge.get("target")
source_id = edge.get("source")
if not target_id or not source_id:
continue
# Get target field name from edge data
edge_data = edge.get("data", {})
target_handle = edge_data.get("targetHandle", {})
if isinstance(target_handle, dict):
field_name = target_handle.get("fieldName")
elif isinstance(target_handle, str):
# Sometimes targetHandle is a JSON string - handle this case
try:
import json
target_info = json.loads(target_handle.replace("œ", '"'))
field_name = target_info.get("fieldName")
except (json.JSONDecodeError, KeyError, TypeError, AttributeError):
field_name = None
else:
field_name = None
if field_name:
if target_id not in field_mapping:
field_mapping[target_id] = {}
field_mapping[target_id][source_id] = field_name
return field_mapping
def _build_output_mapping_from_edges(self, edges: list[dict[str, Any]]) -> dict[str, str]:
"""Build output mapping from edges to track output usage per component.
Args:
edges: List of Langflow edges
Returns:
Dict mapping source_node_id -> output_name
"""
output_mapping: dict[str, str] = {}
for edge in edges:
source_id = edge.get("source")
if not source_id:
continue
# Get source output name from edge data
edge_data = edge.get("data", {})
source_handle = edge_data.get("sourceHandle", {})
output_name = None
if isinstance(source_handle, dict):
output_name = source_handle.get("name")
elif isinstance(source_handle, str):
# Sometimes sourceHandle is a JSON string - handle this case
try:
import json
source_info = json.loads(source_handle.replace("œ", '"'))
output_name = source_info.get("name")
except (json.JSONDecodeError, KeyError, TypeError, AttributeError):
output_name = None
# Only store if we found an output name and don't have one already
# (first edge wins if component has multiple outgoing edges with
# different outputs)
if output_name and source_id not in output_mapping:
output_mapping[source_id] = output_name
return output_mapping
def _build_flow_output(
self,
builder: FlowBuilder,
nodes: list[dict[str, Any]],
dependencies: dict[str, list[str]],
node_output_refs: dict[str, Any],
) -> None:
"""Build workflow output using incremental output building API.
Args:
builder: FlowBuilder instance to add output fields to
nodes: Original Langflow nodes
dependencies: Dependency graph
node_output_refs: Mapping of node IDs to their output references
"""
# Look for ChatOutput nodes first
chat_output_nodes = [n for n in nodes if n.get("data", {}).get("type") == "ChatOutput"]
if chat_output_nodes:
# Use the first ChatOutput node
chat_output_node = chat_output_nodes[0]
node_id = chat_output_node["id"]
# Check if ChatOutput has dependencies
if node_id in dependencies and dependencies[node_id]:
# ChatOutput depends on another node - use that node's output
dep_node_id = dependencies[node_id][0]
if dep_node_id in node_output_refs:
builder.set_output(node_output_refs[dep_node_id])
return
# ChatOutput has no dependencies or dependencies not found - check if
# it's a simple passthrough
if chat_output_nodes and len(nodes) <= 2:
# Simple ChatInput -> ChatOutput workflow
chat_input_nodes = [n for n in nodes if n.get("data", {}).get("type") == "ChatInput"]
if chat_input_nodes:
builder.set_output(Value.input.add_path("message"))
return
# Find leaf nodes (nodes with no dependents)
dependent_nodes = set()
for deps in dependencies.values():
dependent_nodes.update(deps)
leaf_nodes = []
for node in nodes:
node_id = node["id"]
component_type = node.get("data", {}).get("type", "")
# Skip ChatInput/ChatOutput as they're handled specially
if component_type in ["ChatInput", "ChatOutput"]:
continue
if node_id not in dependent_nodes and node_id in node_output_refs:
leaf_nodes.append((node_id, component_type))
# Build structured output based on leaf nodes
if len(leaf_nodes) == 1:
# Single leaf node - use it directly
node_id, _ = leaf_nodes[0]
builder.set_output(node_output_refs[node_id])
elif len(leaf_nodes) > 1:
# Multiple leaf nodes - create structured output using incremental building
for node_id, component_type in leaf_nodes:
# Generate a clean field name from the component type
field_name = component_type.lower().replace("component", "").replace("_", "")
if not field_name:
field_name = node_id.lower().replace("-", "_")
builder.add_output_field(field_name, node_output_refs[node_id])
else:
# No leaf nodes found - fallback to direct input passthrough
builder.set_output(Value.input.add_path("message"))

View File

@ -0,0 +1,4 @@
"""Stepflow worker components for Langflow execution.
Run the worker server with: python -m langflow_stepflow.worker
"""

View File

@ -0,0 +1,110 @@
"""Standalone Stepflow component server for Langflow integration.
Run with: python -m langflow_stepflow.worker
"""
from typing import Any
from stepflow_py.worker import StepflowContext, StepflowServer
from langflow_stepflow.worker.component_tool import component_tool_executor
from langflow_stepflow.worker.core_executor import CoreExecutor
from langflow_stepflow.worker.custom_code_executor import CustomCodeExecutor
# Create server instance
server = StepflowServer()
# Create executors
custom_code_executor = CustomCodeExecutor()
core_executor = CoreExecutor()
@server.component(name="custom_code")
async def custom_code_component(input_data: dict[str, Any], context: StepflowContext) -> dict[str, Any]:
"""Execute a Langflow custom code component."""
return await custom_code_executor.execute(input_data, context)
@server.component(name="core/{*component}")
async def core_component(
input_data: dict[str, Any],
context: StepflowContext,
component: str,
) -> dict[str, Any]:
"""Execute a known core Langflow component by module path."""
return await core_executor.execute(component, input_data, context)
@server.component(name="component_tool")
async def component_tool_component(input_data: dict[str, Any], context: StepflowContext) -> dict[str, Any]:
"""Create tool wrappers from Langflow components."""
return await component_tool_executor(input_data, context)
def main():
"""Main entry point for the Langflow component server.
Logging is automatically configured by the SDK via setup_observability().
Configure via environment variables:
- STEPFLOW_TASKS_URL: TasksService gRPC address (default: localhost:7837)
- STEPFLOW_QUEUE_NAME: Queue name for gRPC transport (default: langflow)
- STEPFLOW_MAX_CONCURRENT: Max concurrent tasks (default: 4)
- STEPFLOW_LOG_LEVEL: Log level (DEBUG, INFO, WARNING, ERROR, default: INFO)
- STEPFLOW_LOG_DESTINATION: Log destination (stderr, file, otlp)
- STEPFLOW_OTLP_ENDPOINT: OTLP endpoint for tracing/logging
- STEPFLOW_SERVICE_NAME: Service name (default: stepflow-python)
"""
import argparse
import asyncio
import os
import nest_asyncio # type: ignore
from stepflow_py.worker.observability import setup_observability
setup_observability()
nest_asyncio.apply()
if os.environ.get("LANGFLOW_DATABASE_URL"):
from langflow.services.utils import initialize_services, teardown_services
# Teardown first to clear any stale state from a previous run,
# then initialize fresh services for this worker process.
asyncio.run(teardown_services())
asyncio.run(initialize_services())
parser = argparse.ArgumentParser(description="Langflow Stepflow Component Server")
parser.add_argument(
"--tasks-url",
type=str,
default=os.environ.get("STEPFLOW_TASKS_URL", "localhost:7837"),
help="TasksService gRPC address (env: STEPFLOW_TASKS_URL)",
)
parser.add_argument(
"--queue-name",
type=str,
default=os.environ.get("STEPFLOW_QUEUE_NAME", "langflow"),
help="Queue name for gRPC transport (env: STEPFLOW_QUEUE_NAME)",
)
parser.add_argument(
"--max-concurrent",
type=int,
default=int(os.environ.get("STEPFLOW_MAX_CONCURRENT", "4")),
help="Max concurrent tasks (env: STEPFLOW_MAX_CONCURRENT)",
)
args = parser.parse_args()
from stepflow_py.worker.grpc_worker import run_grpc_worker
asyncio.run(
run_grpc_worker(
server=server,
tasks_url=args.tasks_url,
queue_name=args.queue_name,
max_concurrent=args.max_concurrent,
)
)
if __name__ == "__main__":
main()

View File

@ -0,0 +1,340 @@
"""Base executor class with shared functionality for Langflow component execution."""
import inspect
from abc import ABC, abstractmethod
from collections.abc import AsyncIterator
from contextlib import AsyncExitStack, asynccontextmanager
from typing import Any
from stepflow_py.worker.observability import get_tracer
from ..exceptions import ExecutionError
from .handlers import (
BaseModelInputHandler,
BaseModelOutputHandler,
DataFrameOutputHandler,
InputHandler,
LangflowTypeInputHandler,
LangflowTypeOutputHandler,
OutputHandler,
ToolWrapperInputHandler,
)
class BaseExecutor(ABC):
"""Abstract base class for Langflow component executors.
Provides shared functionality for both core and custom code executors:
- Execution method determination
- Component input defaults application
- Input/output handler dispatch
- Common component execution flow
Subclasses must implement `_instantiate_component` to define how components
are loaded and instantiated.
"""
@abstractmethod
async def _instantiate_component(
self,
component_info: dict[str, Any],
) -> tuple[Any, str]:
"""Instantiate a component from component info.
This is the only method that differs between executors:
- CoreExecutor: imports module and instantiates class
- CustomCodeExecutor: compiles code from blob and instantiates class
Args:
component_info: Information needed to instantiate the component.
For CoreExecutor: contains module_name, class_name
For CustomCodeExecutor: contains component class and metadata
Returns:
Tuple of (component_instance, component_name) for execution
"""
pass
def _get_input_handlers(self) -> list[InputHandler]:
"""Return the input handlers to apply during parameter preparation.
Base implementation returns handlers for type deserialization.
Subclasses may override to add additional handlers (e.g.,
StringCoercion, DataFrameConversion).
"""
return [
LangflowTypeInputHandler(),
BaseModelInputHandler(),
ToolWrapperInputHandler(),
]
def _get_output_handlers(self) -> list[OutputHandler]:
"""Return the output handlers to apply during result serialization.
Handlers are tried in order during the recursive tree walk;
the first match wins for each value.
"""
return [
LangflowTypeOutputHandler(),
DataFrameOutputHandler(),
BaseModelOutputHandler(),
]
@asynccontextmanager
async def _handler_pipeline(
self,
parameters: dict[str, Any],
template: dict[str, Any],
) -> AsyncIterator[tuple[dict[str, Any], list[OutputHandler]]]:
"""Enter handler contexts, apply input handlers, yield for execution.
Uses ``AsyncExitStack`` to keep all input handler contexts alive
across input preparation, component execution, and output processing.
Yields:
Tuple of (prepared_parameters, output_handlers).
"""
input_handlers = self._get_input_handlers()
output_handlers = self._get_output_handlers()
async with AsyncExitStack() as stack:
result = dict(parameters)
for handler in input_handlers:
matched: dict[str, tuple[Any, dict[str, Any]]] = {}
for key, value in result.items():
field_def = template.get(key, {})
if not isinstance(field_def, dict):
field_def = {}
if handler.matches(template_field=field_def, value=value):
matched[key] = (value, field_def)
if not matched:
continue
ctx = await stack.enter_async_context(handler.activate())
updates = await handler.prepare(matched, ctx)
result.update(updates)
yield result, output_handlers
async def _apply_output_handlers(self, value: Any, handlers: list[OutputHandler]) -> Any:
"""Recursively walk a value tree, applying output handlers.
For each value, tries handlers in order (first match wins).
Dicts and lists without a handler match are recursed into.
Primitives pass through unchanged.
"""
for handler in handlers:
if handler.matches(value=value):
return await handler.process(value)
if isinstance(value, list):
return [await self._apply_output_handlers(item, handlers) for item in value]
if isinstance(value, dict):
return {k: await self._apply_output_handlers(v, handlers) for k, v in value.items()}
if isinstance(value, str | int | float | bool | type(None)):
return value
raise ValueError(
f"Cannot serialize object of type {type(value).__name__}. "
"Only BaseModel objects and simple types are supported."
)
async def _execute_component_instance(
self,
component_instance: Any,
component_name: str,
execution_method: str,
template: dict[str, Any],
runtime_inputs: dict[str, Any],
) -> Any:
"""Execute a component instance with the given parameters.
This is the shared execution flow used by both executors after
the component has been instantiated. Returns serialized output.
Args:
component_instance: Instantiated component object
component_name: Name for logging/tracing
execution_method: Method name to call on the component
template: Template containing field definitions
runtime_inputs: Runtime input values
Returns:
Serialized component execution result
"""
tracer = get_tracer(__name__)
# Prepare raw parameters from template and runtime inputs
raw_params = await self._prepare_component_parameters(template, runtime_inputs)
# Handler pipeline: input transform → execute → output transform
async with self._handler_pipeline(raw_params, template) as (
component_parameters,
output_handlers,
):
# Apply component input defaults
component_parameters = self._apply_component_input_defaults(component_instance, component_parameters)
# Set session_id and graph context
session_id = component_parameters.get("session_id", "default_session")
component_instance._session_id = session_id
self._setup_graph_context(component_instance, session_id)
# Configure component with parameters
if hasattr(component_instance, "set_attributes"):
component_instance._parameters = component_parameters
component_instance.set_attributes(component_parameters)
# Verify execution method exists
if not hasattr(component_instance, execution_method):
available = [m for m in dir(component_instance) if not m.startswith("_")]
raise ExecutionError(f"Method {execution_method} not found in {component_name}. Available: {available}")
method = getattr(component_instance, execution_method)
# Execute the method with tracing
with tracer.start_as_current_span(
f"execute_method:{execution_method}",
attributes={
"component_name": component_name,
"execution_method": execution_method,
},
):
try:
if inspect.iscoroutinefunction(method):
result = await method()
else:
result = method()
except Exception as e:
raise ExecutionError(
f"Error executing {execution_method} on "
f"{component_name} "
f"({type(component_instance).__name__}): {e}"
) from e
# Serialize output through handlers
return await self._apply_output_handlers(result, output_handlers)
async def _prepare_component_parameters(
self, template: dict[str, Any], runtime_inputs: dict[str, Any]
) -> dict[str, Any]:
"""Prepare component parameters from template and runtime inputs.
Extracts defaults from template fields and merges with raw runtime
inputs. No deserialization is performed here — input handlers
handle all type conversion.
Args:
template: Template containing field definitions
runtime_inputs: Runtime input values
Returns:
Merged parameters dictionary (raw values, not deserialized)
"""
parameters: dict[str, Any] = {}
# Extract default values from template
for key, field in template.items():
if isinstance(field, dict) and "value" in field:
# Skip handle inputs with empty values (connected steps
# provide them)
input_types = field.get("input_types", [])
value = field["value"]
if input_types and (value == "" or value is None):
continue
parameters[key] = value
elif isinstance(field, dict):
# Field without value - skip
pass
else:
# Direct value
parameters[key] = field
# Override with raw runtime inputs (no deserialization — handlers do it)
parameters.update(runtime_inputs)
return parameters
def _determine_execution_method(self, outputs: list[dict[str, Any]], selected_output: str | None) -> str | None:
"""Determine which method to call based on outputs configuration.
Args:
outputs: List of output definitions from the component
selected_output: The name of the selected output (if any)
Returns:
The method name to call, or None if not found
"""
if not outputs:
return None
# If selected_output is provided, find the matching output method
if selected_output:
for output in outputs:
if output.get("name") == selected_output:
method = output.get("method")
if isinstance(method, str) and method:
return method
# Default to first output's method
method = outputs[0].get("method")
if isinstance(method, str) and method:
return method
return None
def _apply_component_input_defaults(self, component_instance: Any, parameters: dict[str, Any]) -> dict[str, Any]:
"""Apply default values from component's input definitions.
Args:
component_instance: Instantiated component with inputs attribute
parameters: Current component parameters
Returns:
Parameters with defaults applied for missing values
"""
if not hasattr(component_instance, "inputs"):
return parameters
inputs = component_instance.inputs
if not inputs:
return parameters
result = dict(parameters)
for input_def in inputs:
field_name = getattr(input_def, "name", None)
if not field_name:
continue
# Only set default if not already in parameters
if field_name not in result:
default_value = getattr(input_def, "value", None)
if default_value is not None and default_value != "":
result[field_name] = default_value
return result
def _setup_graph_context(self, component_instance: Any, session_id: str) -> None:
"""Set up graph context for components that need it.
Some components (like Agent) access self.graph.session_id and
self.graph.vertices. This creates a minimal graph context object.
Args:
component_instance: Component instance to configure
session_id: Session ID for the graph context
"""
class GraphContext:
def __init__(self, session_id: str):
self.session_id = session_id
self.vertices: list[Any] = []
self.flow_id = None
# Use __dict__ to set graph even if it's a read-only property
component_instance.__dict__["graph"] = GraphContext(session_id)

View File

@ -0,0 +1,163 @@
"""Component for creating tool wrappers from Langflow components."""
from typing import Any
from stepflow_py.worker import StepflowContext
async def component_tool_executor(input_data: dict[str, Any], context: StepflowContext) -> dict[str, Any]:
"""Create a tool wrapper from Langflow component code and inputs.
This component takes a Langflow component's code/JSON and static inputs,
then creates a serializable tool wrapper that can be used by agents.
Step references in static_inputs are resolved by the Stepflow orchestrator
before this function is called.
Args:
input_data: Contains:
- code: Component JSON/code blob
- inputs: Static input values to apply to component
- component_type: Type of the component
- session_id: Session ID for the workflow execution
Returns:
Tool wrapper dict with component code, static inputs, tool metadata, and
session_id
"""
component_code = input_data.get("code", {})
static_inputs = input_data.get("inputs", {})
component_type = input_data.get("component_type", "unknown")
session_id = input_data.get("session_id", "default_session")
# Store component code as a blob and get the ID
code_blob_id = await context.put_blob(component_code)
# Extract tool metadata from component template
tool_metadata = _extract_tool_metadata(component_code)
# Extract tool input schema from component template
tool_input_schema = _extract_tool_input_schema(component_code)
# Create tool wrapper - now includes session_id for proper message tracking
tool_wrapper = {
"__tool_wrapper__": True,
"code_blob_id": code_blob_id,
"static_inputs": static_inputs,
"component_type": component_type,
"tool_metadata": tool_metadata,
"tool_input_schema": tool_input_schema,
"session_id": session_id,
}
return {"result": tool_wrapper}
def _extract_tool_metadata(component_code: dict[str, Any]) -> dict[str, Any]:
"""Extract tool metadata from component's tools_metadata field.
Args:
component_code: Component JSON containing template and tools_metadata
Returns:
Dict with tool name, description, and other metadata
"""
try:
template = component_code.get("template", {})
tools_metadata = template.get("tools_metadata", {}).get("value", [])
if tools_metadata and len(tools_metadata) > 0:
# Use first tool metadata entry
tool_meta = tools_metadata[0]
return {
"name": tool_meta.get("name", "unknown_tool"),
"description": tool_meta.get("description", ""),
"display_name": tool_meta.get("display_name", ""),
"display_description": tool_meta.get("display_description", ""),
}
else:
# Fallback to component info if no tools_metadata
return {
"name": component_code.get("display_name", "unknown_tool"),
"description": component_code.get("description", ""),
"display_name": component_code.get("display_name", ""),
"display_description": component_code.get("description", ""),
}
except Exception:
return {
"name": "unknown_tool",
"description": "",
"display_name": "Unknown Tool",
"display_description": "",
}
def _extract_tool_input_schema(component_code: dict[str, Any]) -> dict[str, Any]:
"""Extract tool input schema from component template fields with tool_mode=True.
Args:
component_code: Component JSON containing template
Returns:
JSONSchema-style dict describing tool input parameters
"""
try:
template = component_code.get("template", {})
properties = {}
required = []
# Find fields with tool_mode=True
for field_name, field_data in template.items():
if isinstance(field_data, dict) and field_data.get("tool_mode", False):
# This field should be exposed as a tool parameter
field_type = field_data.get("type", "str")
field_info = field_data.get("info", "")
field_required = field_data.get("required", False)
is_secret = field_type == "password" or field_data.get("input_types", []) == ["secret"]
prop: dict[str, Any] = {
"type": _map_langflow_type_to_json_schema(field_type),
"description": field_info,
}
if not is_secret:
prop["default"] = field_data.get("value", "")
# Convert to JSON Schema format
properties[field_name] = prop
if field_required:
required.append(field_name)
return {
"type": "object",
"properties": properties,
"required": required,
}
except Exception:
return {
"type": "object",
"properties": {},
"required": [],
}
def _map_langflow_type_to_json_schema(langflow_type: str) -> str:
"""Map Langflow field types to JSON Schema types.
Args:
langflow_type: Langflow field type (e.g., "str", "int", "bool")
Returns:
JSON Schema type string
"""
type_mapping = {
"str": "string",
"int": "integer",
"float": "number",
"bool": "boolean",
"list": "array",
"dict": "object",
}
return type_mapping.get(langflow_type, "string")

View File

@ -0,0 +1,124 @@
"""Core component executor for known Langflow components.
Executes known Langflow components by importing them directly by module path,
without needing to compile code from blob storage. This is more efficient for
components whose code hash matches a known version.
"""
import importlib
from typing import Any
from stepflow_py.worker import StepflowContext
from stepflow_py.worker.observability import get_tracer
from ..exceptions import ExecutionError
from .base_executor import BaseExecutor
class CoreExecutor(BaseExecutor):
"""Executes known Langflow components by importing them directly.
This executor imports components by their module path and instantiates them
directly, without needing to compile code from blob storage.
"""
async def _instantiate_component(
self,
component_info: dict[str, Any],
) -> tuple[Any, str]:
"""Instantiate a component by importing its module.
Args:
component_info: Contains 'module_name' and 'class_name'
Returns:
Tuple of (component_instance, class_name)
"""
module_name = component_info["module_name"]
class_name = component_info["class_name"]
tracer = get_tracer(__name__)
with tracer.start_as_current_span(
f"instantiate_component:{class_name}",
attributes={"class_name": class_name, "module_name": module_name},
):
# Import the module and get the class
try:
module = importlib.import_module(module_name)
component_class = getattr(module, class_name)
except ImportError as e:
raise ExecutionError(f"Failed to import module {module_name}: {e}") from e
except AttributeError as e:
raise ExecutionError(f"Class {class_name} not found in module {module_name}: {e}") from e
# Instantiate the component
try:
component_instance = component_class()
except Exception as e:
raise ExecutionError(f"Failed to instantiate {class_name}: {e}") from e
return component_instance, class_name
async def execute(
self,
component_path: str,
input_data: dict[str, Any],
context: StepflowContext,
) -> dict[str, Any]:
"""Execute a core Langflow component.
Args:
component_path: The component path suffix after /langflow/core/
(e.g., "lfx/components/docling/DoclingInlineComponent")
input_data: Component input containing template, outputs, runtime inputs
context: Stepflow context (may be needed for some operations)
Returns:
Component execution result
"""
tracer = get_tracer(__name__)
# Convert path to module path (slashes to dots)
module_path = component_path.replace("/", ".")
# Split into module and class name
parts = module_path.rsplit(".", 1)
if len(parts) != 2:
raise ExecutionError(f"Invalid component path: {component_path}")
module_name, class_name = parts
with tracer.start_as_current_span(
f"core_execute:{class_name}",
attributes={
"component_path": component_path,
"module_name": module_name,
"class_name": class_name,
},
):
# Extract execution parameters from input
template = input_data.get("template", {})
outputs = input_data.get("outputs", [])
selected_output = input_data.get("selected_output")
runtime_inputs = input_data.get("input", {})
# Determine execution method
execution_method = self._determine_execution_method(outputs, selected_output)
if not execution_method:
raise ExecutionError(f"No execution method found for {class_name}")
# Instantiate the component
component_instance, component_name = await self._instantiate_component(
{"module_name": module_name, "class_name": class_name}
)
# Execute using shared base class method (returns serialized)
result = await self._execute_component_instance(
component_instance=component_instance,
component_name=component_name,
execution_method=execution_method,
template=template,
runtime_inputs=runtime_inputs,
)
return {"result": result}

View File

@ -0,0 +1,395 @@
"""Custom code executor for Langflow components.
Executes Langflow components by compiling their code from blob storage.
Uses a pre-compilation approach to eliminate context calls during execution.
"""
import inspect
from collections import OrderedDict
from typing import Any
from stepflow_py.worker import StepflowContext
from stepflow_py.worker.observability import get_tracer
from ..exceptions import ExecutionError
from .base_executor import BaseExecutor
from .handlers import (
BaseModelInputHandler,
DataFrameConversionInputHandler,
InputHandler,
LangflowTypeInputHandler,
StringCoercionInputHandler,
ToolWrapperInputHandler,
)
_COMPILED_CACHE_MAX_SIZE = 128
class CustomCodeExecutor(BaseExecutor):
"""Executes Langflow custom code components by compiling code from blobs.
This executor compiles component code from blob storage and instantiates
the resulting classes. It uses a pre-compilation approach to eliminate
context calls during execution.
"""
def __init__(self):
"""Initialize custom code executor."""
super().__init__()
self.compiled_components: OrderedDict[str, Any] = OrderedDict()
def _get_input_handlers(self) -> list[InputHandler]:
"""Return input handlers for custom code execution.
Includes all base handlers plus StringCoercion and DataFrame
conversion needed for Langflow type transformations.
"""
return [
LangflowTypeInputHandler(),
BaseModelInputHandler(),
ToolWrapperInputHandler(),
DataFrameConversionInputHandler(),
StringCoercionInputHandler(),
]
async def _instantiate_component(
self,
component_info: dict[str, Any],
) -> tuple[Any, str]:
"""Instantiate a component from pre-compiled component info.
Args:
component_info: Contains 'class' (the component class) and
'component_type' (name for logging)
Returns:
Tuple of (component_instance, component_type)
"""
component_class = component_info["class"]
component_type = component_info.get("component_type", "Unknown")
tracer = get_tracer(__name__)
with tracer.start_as_current_span(
f"instantiate_component:{component_type}",
attributes={"component_type": component_type},
):
try:
component_instance = component_class()
except Exception as e:
raise ExecutionError(f"Failed to instantiate {component_type}: {e}") from e
return component_instance, component_type
async def execute(self, input_data: dict[str, Any], context: StepflowContext) -> dict[str, Any]:
"""Execute using the new pre-compilation approach.
This method first pre-compiles all components, then executes without context.
Args:
input_data: Component input containing blob_id and runtime inputs
context: Stepflow context for pre-compilation only
Returns:
Component execution result
"""
# Step 1: Pre-compile all components from blobs
await self._prepare_components(input_data, context)
# Step 2: Execute using pre-compiled components (no context needed)
return await self._execute_with_precompiled_components(input_data)
async def _prepare_components(self, input_data: dict[str, Any], context: StepflowContext) -> None:
"""Pre-compile all components from blob data to eliminate runtime context calls.
This method fetches all blob data containing component source code and
compiles them into executable component instances.
Args:
input_data: Component input containing blob IDs
context: Stepflow context for blob operations
(used only for pre-compilation)
"""
blob_ids_to_compile = self._extract_blob_ids(input_data)
tracer = get_tracer(__name__)
for blob_id in blob_ids_to_compile:
if blob_id not in self.compiled_components:
try:
# Trace blob retrieval
with tracer.start_as_current_span("blob_get", attributes={"blob_id": blob_id}):
blob_data = await context.get_blob(blob_id)
except Exception as e:
raise ExecutionError(
f"No component code found for blob {blob_id}. All components must have real Langflow code."
) from e
# Pass blob_id to compilation for better tracing
compiled_component = await self._compile_component(blob_data, blob_id)
self.compiled_components[blob_id] = compiled_component
if len(self.compiled_components) > _COMPILED_CACHE_MAX_SIZE:
self.compiled_components.popitem(last=False)
def _extract_blob_ids(self, input_data: dict[str, Any]) -> set[str]:
"""Extract all blob IDs that need to be compiled from input data."""
blob_ids = set()
# Main blob_id
if "blob_id" in input_data:
blob_ids.add(input_data["blob_id"])
return blob_ids
async def _compile_component(self, blob_data: dict[str, Any], blob_id: str | None = None) -> dict[str, Any]:
"""Compile a component from enhanced blob data into an executable definition."""
component_type = blob_data.get("component_type", "Unknown")
code = blob_data.get("code", "")
template = blob_data.get("template", {})
outputs = blob_data.get("outputs", [])
selected_output = blob_data.get("selected_output")
# Extract enhanced metadata from Phase 1 improvements
base_classes = blob_data.get("base_classes", [])
display_name = blob_data.get("display_name", component_type)
description = blob_data.get("description", "")
documentation = blob_data.get("documentation", "")
metadata = blob_data.get("metadata", {})
field_order = blob_data.get("field_order", [])
icon = blob_data.get("icon", "")
# Trace the compilation process
tracer = get_tracer(__name__)
with tracer.start_as_current_span(
f"compile_component:{component_type}",
attributes={
"component_type": component_type,
"display_name": display_name,
**({"blob_id": blob_id} if blob_id else {}),
},
):
# Patch PlaceholderGraph for lfx compatibility
self._patch_placeholder_graph()
if not code or not code.strip():
raise ExecutionError(
f"No code found for component {component_type}. All executable components should have custom code."
)
try:
from lfx.custom.eval import eval_custom_component_code
# Trace the actual code evaluation
with tracer.start_as_current_span(
"eval_custom_component_code",
attributes={
"component_type": component_type,
"code_length": len(code),
},
):
component_class = eval_custom_component_code(code)
except Exception as e:
raise ExecutionError(f"Failed to evaluate component code for {component_type}: {e}") from e
if not component_class:
raise ExecutionError(f"eval_custom_component_code returned None for {component_type}")
# Determine execution method with enhanced logic
execution_method = self._determine_execution_method(outputs, selected_output)
if not execution_method:
raise ExecutionError(f"No execution method found for {component_type}")
return {
"class": component_class,
"template": template,
"execution_method": execution_method,
"component_type": component_type,
# Enhanced metadata for better component execution
"base_classes": base_classes,
"display_name": display_name,
"description": description,
"documentation": documentation,
"metadata": metadata,
"field_order": field_order,
"icon": icon,
}
async def _execute_with_precompiled_components(self, input_data: dict[str, Any]) -> dict[str, Any]:
"""Execute using pre-compiled components - no context needed."""
return await self._execute_single_component_precompiled(input_data)
async def _execute_single_component_precompiled(self, input_data: dict[str, Any]) -> dict[str, Any]:
"""Execute a single component using pre-compiled data."""
blob_id = input_data.get("blob_id")
if not blob_id:
raise ExecutionError("No blob_id provided")
if blob_id not in self.compiled_components:
raise ExecutionError(f"Component {blob_id} not pre-compiled")
compiled_component = self.compiled_components[blob_id]
runtime_inputs = input_data.get("input", {})
# Execute the component (returns serialized output)
result = await self._execute_compiled_component(compiled_component, runtime_inputs)
return {"result": result}
async def _execute_compiled_component(
self, compiled_component: dict[str, Any], runtime_inputs: dict[str, Any]
) -> Any:
"""Execute a pre-compiled component instance.
Returns serialized output (output handlers handle serialization).
"""
component_class = compiled_component["class"]
template = compiled_component["template"]
execution_method = compiled_component["execution_method"]
component_type = compiled_component["component_type"]
display_name = compiled_component.get("display_name", component_type)
tracer = get_tracer(__name__)
# Create component instance with tracing
with tracer.start_as_current_span(
f"instantiate_component:{component_type}",
attributes={
"component_type": component_type,
"display_name": display_name,
},
):
try:
component_instance = component_class()
except Exception as e:
raise ExecutionError(f"Failed to instantiate {component_type}: {e}") from e
# Prepare raw parameters
raw_params = await self._prepare_component_parameters(template, runtime_inputs)
# Handler pipeline: input transform → execute → output transform
async with self._handler_pipeline(raw_params, template) as (
component_parameters,
output_handlers,
):
# Apply component input defaults before configuring
component_parameters = self._apply_component_input_defaults(component_instance, component_parameters)
session_id = component_parameters.get("session_id", "default_session")
component_instance._session_id = session_id
self._setup_graph_context(component_instance, session_id)
resolved_parameters = component_parameters
# Configure component
if hasattr(component_instance, "set_attributes"):
component_instance._parameters = resolved_parameters
component_instance.set_attributes(resolved_parameters)
# Execute component method with tracing
if not hasattr(component_instance, execution_method):
available = [m for m in dir(component_instance) if not m.startswith("_")]
raise ExecutionError(f"Method {execution_method} not found in {component_type}. Available: {available}")
method = getattr(component_instance, execution_method)
with tracer.start_as_current_span(
f"execute_method:{execution_method}",
attributes={
"component_type": component_type,
"display_name": display_name,
"execution_method": execution_method,
},
):
try:
if inspect.iscoroutinefunction(method):
result = await method()
else:
result = await self._execute_sync_method_safely(method, component_type)
# Handle OpenSearch client objects
try:
from opensearchpy import OpenSearch
if isinstance(result, OpenSearch):
return {
"status": "success",
"type": "OpenSearchVectorStore",
}
except ImportError:
pass
# Serialize output through handlers
return await self._apply_output_handlers(result, output_handlers)
except Exception as e:
raise ExecutionError(f"Failed to execute {execution_method}: {e}") from e
def _patch_placeholder_graph(self) -> None:
"""Patch PlaceholderGraph to add vertices attribute for lfx compatibility.
lfx 0.1.12+ Agent components access graph.vertices, but PlaceholderGraph
doesn't have this attribute by default. This patch adds it.
"""
from typing import NamedTuple
class EnhancedPlaceholderGraph(NamedTuple):
"""Enhanced PlaceholderGraph with vertices for lfx compatibility."""
flow_id: str | None = None
flow_name: str | None = None
user_id: str | None = None
session_id: str | None = None
context: dict | None = None
vertices: tuple = ()
try:
from lfx.custom.custom_component import (
component as lfx_component_module,
)
lfx_component_module.PlaceholderGraph = EnhancedPlaceholderGraph # type: ignore[assignment,misc]
except ImportError:
pass
def _apply_component_input_defaults(
self, component_instance: Any, component_parameters: dict[str, Any]
) -> dict[str, Any]:
"""Apply component input defaults for missing parameters.
Args:
component_instance: Instantiated component
component_parameters: Current parameters
Returns:
Parameters with defaults applied from component inputs definition
"""
if hasattr(component_instance, "inputs"):
for input_field in component_instance.inputs:
if hasattr(input_field, "name") and hasattr(input_field, "value"):
param_name = input_field.name
default_value = input_field.value
if param_name not in component_parameters and default_value is not None and default_value != "":
component_parameters[param_name] = default_value
return component_parameters
def _determine_execution_method(self, outputs: list, selected_output: str | None) -> str | None:
"""Determine execution method from outputs metadata."""
if selected_output:
for output in outputs:
if output.get("name") == selected_output:
method = output.get("method")
if isinstance(method, str) and method:
return method
# Fallback to first output's method
if outputs:
method = outputs[0].get("method")
if isinstance(method, str) and method:
return method
return None
async def _execute_sync_method_safely(self, method, component_type: str):
"""Execute sync method in a thread pool to avoid blocking the event loop."""
import asyncio
return await asyncio.to_thread(method)

View File

@ -0,0 +1,28 @@
"""Input and output handlers for Langflow type transformations.
Input handlers dispatch on template field metadata and/or runtime value content
to transform parameter values before component execution.
Output handlers dispatch on Python type to serialize execution results back to
JSON-compatible formats with type markers.
"""
from .base import InputHandler, OutputHandler
from .base_model import BaseModelInputHandler, BaseModelOutputHandler
from .dataframe import DataFrameConversionInputHandler, DataFrameOutputHandler
from .langflow_types import LangflowTypeInputHandler, LangflowTypeOutputHandler
from .string_coercion import StringCoercionInputHandler
from .tool_wrapper import ToolWrapperInputHandler
__all__ = [
"InputHandler",
"OutputHandler",
"BaseModelInputHandler",
"BaseModelOutputHandler",
"DataFrameConversionInputHandler",
"DataFrameOutputHandler",
"LangflowTypeInputHandler",
"LangflowTypeOutputHandler",
"StringCoercionInputHandler",
"ToolWrapperInputHandler",
]

View File

@ -0,0 +1,87 @@
"""Base classes for input and output handlers."""
from __future__ import annotations
from abc import ABC, abstractmethod
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import Any
class InputHandler(ABC):
"""Abstract base class for input-side value transformations.
Subclasses declare which fields they handle via ``matches()`` (using
template metadata and/or value content), optionally set up resources
via ``activate()``, and transform values in ``prepare()``.
"""
@abstractmethod
def matches(self, *, template_field: dict[str, Any], value: Any) -> bool:
"""Return True if this handler should process the given field.
Args:
template_field: Template field metadata (type, input_types, etc.).
value: The current runtime value for this field.
"""
...
@asynccontextmanager
async def activate(self) -> AsyncIterator[Any]:
"""Set up and tear down handler resources.
Only entered when at least one field matches. Yields a context value
that is passed to ``prepare()``. The default implementation yields
``None`` (no resources needed).
"""
yield None
@abstractmethod
async def prepare(self, fields: dict[str, tuple[Any, dict[str, Any]]], context: Any) -> dict[str, Any]:
"""Batch-process all matched fields.
Args:
fields: Mapping of ``{key: (value, template_field)}`` for every
parameter whose template field matched this handler.
context: The value yielded by ``activate()``.
Returns:
Mapping of ``{key: resolved_value}`` for fields whose values
changed. Fields omitted from the result keep their original value.
"""
...
class OutputHandler(ABC):
"""Abstract base class for output-side value transformations.
Subclasses declare which values they handle via ``matches()`` (using
Python type checks or value inspection) and transform values in
``process()``.
Output handlers are applied during a recursive tree walk of the
execution result. Each handler processes a single matched value;
the executor handles recursion into dicts/lists.
"""
@abstractmethod
def matches(self, *, value: Any) -> bool:
"""Return True if this handler should process the given value.
Args:
value: The Python object to potentially serialize/transform.
"""
...
@abstractmethod
async def process(self, value: Any) -> Any:
"""Transform a single matched value.
Args:
value: The matched Python object.
Returns:
The serialized/transformed value (must be JSON-serializable
or a container of further values for the tree walker).
"""
...

View File

@ -0,0 +1,190 @@
"""Handlers for Pydantic BaseModel serialization and deserialization.
Input: reconstruct BaseModel instances from dicts with ``__class_name__`` markers.
Output: serialize BaseModel instances with class metadata and SecretStr handling.
"""
from __future__ import annotations
import importlib
import logging
from typing import Any
from .base import InputHandler, OutputHandler
logger = logging.getLogger(__name__)
def _has_class_name_marker(value: Any) -> bool:
"""Check if a value (or list of values) contains __class_name__ markers."""
if isinstance(value, dict) and "__class_name__" in value:
return True
if isinstance(value, list):
return any(isinstance(item, dict) and "__class_name__" in item for item in value)
return False
def _deserialize_base_model(obj: dict[str, Any]) -> Any:
"""Deserialize a dict with __class_name__ marker back to a BaseModel instance."""
class_name = obj.get("__class_name__")
module_name = obj.get("__module_name__")
if not class_name or not module_name:
return obj
try:
module = importlib.import_module(module_name)
class_type = getattr(module, class_name)
obj_data = {k: v for k, v in obj.items() if k not in ("__class_name__", "__module_name__")}
return class_type(**obj_data)
except Exception:
logger.debug("Failed to reconstruct BaseModel %s.%s", module_name, class_name)
return obj
class BaseModelInputHandler(InputHandler):
"""Deserialize dicts with ``__class_name__`` markers to BaseModel instances.
Handles dynamic import and reconstruction of arbitrary Pydantic models.
Also handles list values containing marked dicts.
"""
def matches(self, *, template_field: dict[str, Any], value: Any) -> bool:
return _has_class_name_marker(value)
async def prepare(self, fields: dict[str, tuple[Any, dict[str, Any]]], context: Any) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, (value, _template_field) in fields.items():
if isinstance(value, dict) and "__class_name__" in value:
result[key] = _deserialize_base_model(value)
elif isinstance(value, list):
result[key] = [
_deserialize_base_model(item) if isinstance(item, dict) and "__class_name__" in item else item
for item in value
]
return result
def _is_secret_str_type(field_type: Any) -> bool:
"""Check if a field type is SecretStr or similar secret type."""
try:
if hasattr(field_type, "__origin__"):
if field_type.__origin__ is type(None) or str(field_type.__origin__) == "typing.Union":
if hasattr(field_type, "__args__"):
for arg in field_type.__args__:
if _is_secret_str_type(arg):
return True
type_name = getattr(field_type, "__name__", str(field_type))
return "SecretStr" in type_name or "Secret" in type_name
except Exception:
return False
def _looks_like_env_var_name(value: str) -> bool:
"""Check if a string looks like an environment variable name.
Matches strings that are all uppercase letters/digits/underscores and
contain at least one underscore, like ``OPENAI_API_KEY`` or
``AWS_SECRET_KEY``. The underscore requirement avoids matching short
names like ``PATH`` or ``HOME``.
"""
return value.isupper() and "_" in value
def _resolve_secret_value(secret_value: Any) -> Any:
"""Resolve a SecretStr value, attempting env var lookup if it looks like one.
Only attempts resolution when the value looks like an env var name
(all uppercase with underscores) to avoid accidentally matching
unrelated environment variables.
"""
if not isinstance(secret_value, str) or not secret_value:
return secret_value
if not _looks_like_env_var_name(secret_value):
return secret_value
import os
resolved = os.getenv(secret_value)
return resolved if resolved is not None else secret_value
def _handle_special_pydantic_types(obj: Any, serialized: dict[str, Any]) -> dict[str, Any]:
"""Handle SecretStr and other special Pydantic types during serialization."""
try:
if hasattr(obj, "model_fields"):
fields = obj.model_fields
for field_name, field_info in fields.items():
if hasattr(field_info, "annotation"):
field_type = field_info.annotation
if _is_secret_str_type(field_type):
field_value = getattr(obj, field_name, None)
if field_value is not None:
try:
secret_value = field_value.get_secret_value()
serialized[field_name] = _resolve_secret_value(secret_value)
except Exception:
pass
elif hasattr(obj, "__fields__"):
fields = obj.__fields__
for field_name, field_info in fields.items():
field_type = field_info.type_
if _is_secret_str_type(field_type):
field_value = getattr(obj, field_name, None)
if field_value is not None:
try:
secret_value = field_value.get_secret_value()
serialized[field_name] = _resolve_secret_value(secret_value)
except Exception:
pass
except Exception:
pass
return serialized
class BaseModelOutputHandler(OutputHandler):
"""Serialize Pydantic BaseModel instances with class metadata.
Produces dicts with ``__class_name__`` and ``__module_name__`` markers.
Includes special handling for SecretStr fields (resolves env vars).
"""
def matches(self, *, value: Any) -> bool:
try:
from pydantic import BaseModel
if not isinstance(value, BaseModel):
return False
# Don't match Langflow types (handled by LangflowTypeOutputHandler)
try:
from lfx.schema.data import Data
from lfx.schema.dataframe import DataFrame
from lfx.schema.message import Message
if isinstance(value, Message | Data | DataFrame):
return False
except ImportError:
pass
return True
except ImportError:
return False
async def process(self, value: Any) -> Any:
try:
serialized = value.model_dump(mode="json", warnings=False)
except Exception:
serialized = value.model_dump(mode="json")
serialized = _handle_special_pydantic_types(value, serialized)
serialized["__class_name__"] = value.__class__.__name__
serialized["__module_name__"] = value.__class__.__module__
return serialized

View File

@ -0,0 +1,78 @@
"""Handlers for DataFrame conversion (input) and serialization (output)."""
from __future__ import annotations
import logging
from typing import Any
from .base import InputHandler, OutputHandler
logger = logging.getLogger(__name__)
def _is_data_list(value: list[Any]) -> bool:
"""Check if a list contains Data-like objects suitable for DataFrame conversion."""
non_null = [item for item in value if item is not None]
if not non_null:
return False
return all(
(isinstance(item, dict) and ("text" in item or item.get("__class_name__") == "Data"))
or (hasattr(item, "__class__") and item.__class__.__name__ in ("Data", "JSON"))
for item in non_null
)
class DataFrameConversionInputHandler(InputHandler):
"""Convert lists of Data objects to DataFrames for DataFrame-typed fields.
Matches template fields whose ``input_types`` include ``"DataFrame"``
and whose runtime value is a non-empty list of Data-like objects.
"""
def matches(self, *, template_field: dict[str, Any], value: Any) -> bool:
if "DataFrame" not in template_field.get("input_types", []):
return False
return isinstance(value, list) and len(value) > 0
async def prepare(self, fields: dict[str, tuple[Any, dict[str, Any]]], context: Any) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, (value, _template_field) in fields.items():
if not isinstance(value, list) or not value:
continue
if not _is_data_list(value):
continue
try:
from lfx.schema.dataframe import DataFrame
result[key] = DataFrame(data=value)
except Exception:
logger.debug("DataFrame conversion failed for field %r", key)
return result
class DataFrameOutputHandler(OutputHandler):
"""Serialize DataFrame objects to split JSON format.
Uses ``orient="split"`` for ~45% size reduction compared to records format.
"""
def matches(self, *, value: Any) -> bool:
try:
import pandas as pd
return isinstance(value, pd.DataFrame)
except ImportError:
return False
async def process(self, value: Any) -> Any:
json_str = value.to_json(orient="split")
return {
"__langflow_type__": "DataFrame",
"json_data": json_str,
"text_key": getattr(value, "text_key", "text"),
"default_value": getattr(value, "default_value", ""),
}

View File

@ -0,0 +1,179 @@
"""Handlers for Langflow core types (Message, Data, DataFrame).
Input: deserialize dicts with ``__langflow_type__`` markers back to lfx objects.
Output: serialize Message/Data objects to dicts with ``__langflow_type__`` markers.
"""
from __future__ import annotations
import logging
from typing import Any
from .base import InputHandler, OutputHandler
logger = logging.getLogger(__name__)
# lfx 0.3+ renamed Data→JSON, DataFrame→Table. Map runtime class names
# back to the canonical Langflow type names used in serialized payloads.
_CLASS_TO_LANGFLOW_TYPE: dict[str, str] = {
"JSON": "Data",
"Table": "DataFrame",
}
def _langflow_type_name(value: Any) -> str:
"""Return the canonical Langflow type name for a value."""
cls_name = type(value).__name__
return _CLASS_TO_LANGFLOW_TYPE.get(cls_name, cls_name)
def _is_langflow_type_dict(value: Any) -> bool:
"""Check if a single value is a dict with a ``__langflow_type__`` marker."""
return isinstance(value, dict) and "__langflow_type__" in value
def _has_langflow_type_marker(value: Any) -> bool:
"""Check if a value (or list of values) contains __langflow_type__ markers."""
if _is_langflow_type_dict(value):
return True
if isinstance(value, list):
return any(_is_langflow_type_dict(item) for item in value)
return False
def _deserialize_single(obj: dict[str, Any]) -> Any:
"""Deserialize a single dict with __langflow_type__ marker."""
try:
from lfx.schema.data import Data
from lfx.schema.message import Message
except ImportError:
return obj
langflow_type = obj.get("__langflow_type__")
if not langflow_type:
return obj
obj_data = {k: v for k, v in obj.items() if k != "__langflow_type__"}
if langflow_type == "Message":
return Message(**obj_data)
elif langflow_type == "Data":
return Data(**obj_data)
elif langflow_type == "DataFrame":
return _deserialize_dataframe(obj_data, obj)
return obj
def _deserialize_dataframe(obj_data: dict[str, Any], raw: dict[str, Any]) -> Any:
"""Deserialize a DataFrame from split JSON format with recovery fallback."""
try:
import io
import json
import pandas as pd
from lfx.schema.dataframe import DataFrame
text_key = obj_data.get("text_key", "text")
default_value = obj_data.get("default_value", "")
json_str = obj_data.get("json_data")
if not json_str:
raise ValueError("DataFrame missing required json_data field")
json_io = io.StringIO(json_str if isinstance(json_str, str) else json.dumps(json_str))
pd_df = pd.read_json(json_io, orient="split")
data_list = pd_df.to_dict(orient="records")
# Replace NaN with None for JSON compliance
data_list = [{k: (None if pd.isna(v) else v) for k, v in record.items()} for record in data_list]
return DataFrame(
data=data_list,
text_key=text_key,
default_value=default_value,
)
except Exception:
# Recovery fallback: try manual reconstruction
return _recover_dataframe(raw)
def _recover_dataframe(raw_value: dict[str, Any]) -> Any:
"""Manual DataFrame recovery from split JSON format.
Called when primary deserialization fails. Attempts a more lenient
reconstruction from the raw serialized dict.
"""
try:
import io
import pandas as pd
from lfx.schema.dataframe import DataFrame as LfDataFrame
json_str = raw_value.get("json_data")
if not json_str or not isinstance(json_str, str):
return raw_value
pd_df = pd.read_json(io.StringIO(json_str), orient="split")
records = pd_df.to_dict(orient="records")
def clean_value(v: Any) -> Any:
try:
return None if pd.isna(v) else v
except (ValueError, TypeError):
return v
data_list = [{k: clean_value(v) for k, v in record.items()} for record in records]
return LfDataFrame(
data=data_list,
text_key=raw_value.get("text_key", "text"),
default_value=raw_value.get("default_value", ""),
)
except Exception:
return raw_value
class LangflowTypeInputHandler(InputHandler):
"""Deserialize dicts with ``__langflow_type__`` markers to lfx objects.
Handles Message, Data, and DataFrame deserialization. Also handles
list values containing marked dicts (recursive deserialization).
Includes DataFrame recovery logic for robust handling of edge cases.
"""
def matches(self, *, template_field: dict[str, Any], value: Any) -> bool:
return _has_langflow_type_marker(value)
async def prepare(self, fields: dict[str, tuple[Any, dict[str, Any]]], context: Any) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, (value, _template_field) in fields.items():
if _is_langflow_type_dict(value):
result[key] = _deserialize_single(value)
elif isinstance(value, list):
result[key] = [_deserialize_single(item) if _is_langflow_type_dict(item) else item for item in value]
return result
class LangflowTypeOutputHandler(OutputHandler):
"""Serialize Message and Data objects with ``__langflow_type__`` markers.
Matches ``isinstance(value, (Message, Data))`` and produces dicts with
``model_dump(mode="json")`` plus a ``__langflow_type__`` key.
"""
def matches(self, *, value: Any) -> bool:
try:
from lfx.schema.data import Data
from lfx.schema.message import Message
return isinstance(value, Message | Data)
except ImportError:
return False
async def process(self, value: Any) -> Any:
serialized = value.model_dump(mode="json")
serialized["__langflow_type__"] = _langflow_type_name(value)
return serialized

View File

@ -0,0 +1,33 @@
"""Input handler that coerces Langflow Message objects to plain strings."""
from __future__ import annotations
from typing import Any
from .base import InputHandler
class StringCoercionInputHandler(InputHandler):
"""Coerce Langflow Message objects to their ``.text`` for string fields.
Matches template fields with ``type == "str"`` whose runtime value is a
Langflow ``Message`` instance. Extracts the ``text`` attribute so the
component receives a plain string as expected.
This is needed because lfx components (1.6.4+) are stricter about type
validation and reject Message objects for string-typed inputs.
"""
def matches(self, *, template_field: dict[str, Any], value: Any) -> bool:
if template_field.get("type") != "str":
return False
return hasattr(value, "__class__") and value.__class__.__name__ == "Message" and hasattr(value, "text")
async def prepare(self, fields: dict[str, tuple[Any, dict[str, Any]]], context: Any) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, (value, _template_field) in fields.items():
if hasattr(value, "__class__") and value.__class__.__name__ == "Message" and hasattr(value, "text"):
result[key] = value.text
return result

View File

@ -0,0 +1,177 @@
"""Input handler for tool wrapper deserialization.
Converts serialized tool wrapper dicts into callable LangChain StructuredTool
objects.
"""
from __future__ import annotations
import ast
import operator
from typing import Any
from .base import InputHandler
def _execute_calculator_tool(expression: str) -> str:
"""Execute calculator tool by evaluating the mathematical expression."""
try:
tree = ast.parse(expression, mode="eval")
result = _eval_expr(tree.body)
formatted_result = f"{float(result):.6f}".rstrip("0").rstrip(".")
return formatted_result
except Exception as e:
return f"Calculator error: {str(e)}"
def _eval_expr(node: ast.AST) -> float:
"""Evaluate an AST node recursively (from CalculatorComponent)."""
OPERATORS = {
ast.Add: operator.add,
ast.Sub: operator.sub,
ast.Mult: operator.mul,
ast.Div: operator.truediv,
ast.Pow: operator.pow,
}
if isinstance(node, ast.Constant):
if isinstance(node.value, int | float):
return float(node.value)
raise TypeError(f"Unsupported constant type: {type(node.value).__name__}")
if isinstance(node, ast.Num): # For backwards compatibility
if isinstance(node.n, int | float):
return float(node.n)
raise TypeError(f"Unsupported number type: {type(node.n).__name__}")
if isinstance(node, ast.BinOp):
op_type = type(node.op)
if op_type not in OPERATORS:
raise TypeError(f"Unsupported binary operator: {op_type.__name__}")
left = _eval_expr(node.left)
right = _eval_expr(node.right)
result = OPERATORS[op_type](left, right)
return float(result)
raise TypeError(f"Unsupported operation or expression type: {type(node).__name__}")
def _create_tool_from_wrapper(tool_wrapper: dict[str, Any]) -> Any:
"""Create a LangChain StructuredTool from a tool wrapper."""
try:
from langchain_core.tools import StructuredTool
from pydantic import BaseModel, create_model
tool_metadata = tool_wrapper.get("tool_metadata", {})
tool_input_schema = tool_wrapper.get("tool_input_schema", {})
static_inputs = tool_wrapper.get("static_inputs", {})
component_type = tool_wrapper.get("component_type", "unknown")
session_id = tool_wrapper.get("session_id", "default_session")
component_code = tool_wrapper.get("component_code")
code_blob_id = tool_wrapper.get("code_blob_id")
if component_code is None and code_blob_id is None:
raise ValueError("Tool wrapper missing both component_code and code_blob_id")
properties = tool_input_schema.get("properties", {})
field_definitions: dict[str, tuple[type, Any]] = {}
for field_name, field_def in properties.items():
field_type: type = str
default_value = field_def.get("default", "")
field_definitions[field_name] = (field_type, default_value)
input_schema: type[BaseModel]
if field_definitions:
input_schema = create_model("ToolInputSchema", **field_definitions) # type: ignore[call-overload]
else:
class EmptySchema(BaseModel):
pass
input_schema = EmptySchema
def tool_func(**kwargs) -> dict[str, Any]:
"""Execute the tool by running the component."""
try:
if tool_metadata.get("name") == "evaluate_expression" and "expression" in kwargs:
result = _execute_calculator_tool(kwargs["expression"])
return {"result": result}
merged_inputs = {
**static_inputs,
**kwargs,
"session_id": session_id,
}
tool_name = tool_metadata.get("name", "unknown")
result_data = {
"result": (f"Tool {tool_name} executed with inputs: {merged_inputs}"),
"component_type": component_type,
"inputs": merged_inputs,
"status": "tool_wrapper_execution",
}
if code_blob_id:
result_data["code_blob_id"] = code_blob_id
elif component_code:
result_data["has_component_code"] = True
return result_data
except Exception as e:
return {
"error": f"Tool execution failed: {str(e)}",
"component_type": component_type,
}
return StructuredTool.from_function(
func=tool_func,
name=tool_metadata.get("name", "unknown_tool"),
description=tool_metadata.get("description", ""),
args_schema=input_schema,
)
except Exception as e:
class FailedToolWrapper:
def __init__(self, tool_wrapper, error):
self.tool_wrapper = tool_wrapper
self.error = error
self.name = tool_wrapper.get("tool_metadata", {}).get("name", "failed_tool")
def invoke(self, inputs):
return {"error": f"Tool creation failed: {self.error}"}
return FailedToolWrapper(tool_wrapper, str(e))
class ToolWrapperInputHandler(InputHandler):
"""Deserialize tool wrapper dicts into callable StructuredTool objects.
Matches dicts with ``__tool_wrapper__`` key and converts them to
LangChain StructuredTool instances.
"""
def matches(self, *, template_field: dict[str, Any], value: Any) -> bool:
if isinstance(value, dict) and value.get("__tool_wrapper__"):
return True
if isinstance(value, list):
return any(isinstance(item, dict) and item.get("__tool_wrapper__") for item in value)
return False
async def prepare(self, fields: dict[str, tuple[Any, dict[str, Any]]], context: Any) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, (value, _template_field) in fields.items():
if isinstance(value, dict) and value.get("__tool_wrapper__"):
result[key] = _create_tool_from_wrapper(value)
elif isinstance(value, list):
result[key] = [
_create_tool_from_wrapper(item) if isinstance(item, dict) and item.get("__tool_wrapper__") else item
for item in value
]
return result

View File

View File

@ -0,0 +1,98 @@
"""Pytest configuration and fixtures."""
from pathlib import Path
from typing import Any
import pytest
from langflow_stepflow.translation.translator import LangflowConverter
# Register pytest-asyncio plugin at top level for all tests
pytest_plugins = ["pytest_asyncio"]
@pytest.fixture
def fixtures_dir() -> Path:
"""Path to test fixtures directory."""
return Path(__file__).parent / "fixtures"
@pytest.fixture
def langflow_fixtures_dir(fixtures_dir: Path) -> Path:
"""Path to Langflow JSON fixtures."""
return fixtures_dir / "langflow"
@pytest.fixture
def stepflow_fixtures_dir(fixtures_dir: Path) -> Path:
"""Path to expected Stepflow YAML fixtures."""
return fixtures_dir / "stepflow"
@pytest.fixture
def simple_langflow_workflow() -> dict[str, Any]:
"""Simple Langflow workflow for testing."""
return {
"data": {
"nodes": [
{
"id": "ChatInput-1",
"data": {
"type": "ChatInput",
"node": {
"template": {
"input_value": {
"type": "str",
"value": "",
"info": "Message to be passed as input",
}
}
},
"outputs": [
{
"name": "message",
"method": "message_response",
"types": ["Message"],
}
],
},
},
{
"id": "ChatOutput-2",
"data": {
"type": "ChatOutput",
"node": {
"template": {
"input_value": {
"type": "str",
"value": "",
"info": "Message to be passed as output",
}
}
},
"outputs": [
{
"name": "message",
"method": "message_response",
"types": ["Message"],
}
],
},
},
],
"edges": [
{
"source": "ChatInput-1",
"target": "ChatOutput-2",
"source_handle": "message",
"target_handle": "input_value",
}
],
}
}
@pytest.fixture
def converter() -> LangflowConverter:
"""LangflowConverter instance for testing."""
return LangflowConverter()

View File

@ -0,0 +1,186 @@
# Copyright 2025 DataStax Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not
# use this file except in compliance with the License. You may obtain a copy of
# the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations under
# the License.
"""Test utilities for building tweaks with environment variable support.
This module provides testing-specific utilities for creating tweaks for
Langflow component testing. It handles environment variables, pytest
integration, and common test scenarios.
This is separate from the production stepflow_tweaks.py to keep the
production code lean and focused.
"""
import os
from typing import Any
class TweaksBuilder:
"""Builder utility for creating tweaks with environment variable support.
This class makes it easy to build tweaks for testing by providing convenient
methods to add tweaks from environment variables or direct values.
Examples:
>>> builder = TweaksBuilder()
>>> builder.add_env_tweak(
... "LanguageModelComponent-abc123", "api_key", "OPENAI_API_KEY"
... )
>>> builder.add_tweak("LanguageModelComponent-abc123", "temperature", 0.8)
>>> tweaks = builder.build_or_skip() # Auto-skips test if env vars missing
"""
def __init__(self):
"""Initialize empty tweaks builder."""
self.tweaks: dict[str, dict[str, Any]] = {}
self.missing_env_vars: list[str] = []
def add_tweak(self, component_id: str, field_name: str, value: Any) -> "TweaksBuilder":
"""Add a direct value tweak for a component field.
Args:
component_id: Langflow component ID (e.g., "LanguageModelComponent-abc123")
field_name: Field name to tweak (e.g., "api_key", "temperature")
value: Value to set for the field
Returns:
Self for method chaining
"""
if component_id not in self.tweaks:
self.tweaks[component_id] = {}
self.tweaks[component_id][field_name] = value
return self
def add_env_tweak(self, component_id: str, field_name: str, env_var: str) -> "TweaksBuilder":
"""Add a tweak from an environment variable.
Args:
component_id: Langflow component ID (e.g., "LanguageModelComponent-abc123")
field_name: Field name to tweak (e.g., "api_key", "temperature")
env_var: Environment variable name (e.g., "OPENAI_API_KEY")
Returns:
Self for method chaining
Note:
If the environment variable is not set, it will be reported later
"""
env_value = os.environ.get(env_var)
if env_value is not None:
self.add_tweak(component_id, field_name, env_value)
else:
self.missing_env_vars.append(env_var)
return self
def add_astradb_tweaks(
self,
component_id: str,
endpoint_env: str = "ASTRA_DB_API_ENDPOINT",
**kwargs: Any,
) -> "TweaksBuilder":
"""Add common AstraDB component tweaks.
Args:
component_id: Langflow component ID
token_env: Environment variable for application token
endpoint_env: Environment variable for API endpoint
**kwargs: Additional direct tweaks (e.g., database_name="test_db")
Returns:
Self for method chaining
"""
self.add_env_tweak(component_id, "api_endpoint", endpoint_env)
# Add default test values if not overridden
if "database_name" not in kwargs:
kwargs["database_name"] = "langflow-test"
if "collection_name" not in kwargs:
kwargs["collection_name"] = "test_collection"
for field_name, value in kwargs.items():
self.add_tweak(component_id, field_name, value)
return self
def build(self) -> dict[str, dict[str, Any]]:
"""Build the final tweaks dictionary.
Returns:
Dictionary ready to use with apply_stepflow_tweaks
Raises:
ValueError: If any required environment variables are missing
"""
if self.missing_env_vars:
missing_vars = ", ".join(self.missing_env_vars)
raise ValueError(
f"Missing required environment variables: {missing_vars}. "
"Please set these variables or use pytest.skip() to skip the test."
)
return dict(self.tweaks)
def build_or_skip(self) -> dict[str, dict[str, Any]]:
"""Build tweaks dictionary or skip test if environment variables are missing.
This is a convenience method for use in pytest tests that automatically
calls pytest.skip() if required environment variables are missing.
Returns:
Dictionary ready to use with apply_stepflow_tweaks
Raises:
pytest.skip: If any required environment variables are missing
"""
if self.missing_env_vars:
import pytest
missing_vars = ", ".join(self.missing_env_vars)
pytest.skip(f"Missing required environment variables: {missing_vars}")
return dict(self.tweaks)
# Generic helper functions
def create_openai_test_tweaks(*component_ids: str) -> dict[str, dict[str, Any]]:
"""Create tweaks for multiple OpenAI components.
Args:
*component_ids: Langflow component IDs that use OpenAI API key
Returns:
Tweaks dictionary for use with stepflow workflows
"""
builder = TweaksBuilder()
for component_id in component_ids:
builder.add_openai_tweaks(component_id)
return builder.build_or_skip()
def create_astradb_test_tweaks(*component_ids: str, **overrides: Any) -> dict[str, dict[str, Any]]:
"""Create tweaks for multiple AstraDB components.
Args:
*component_ids: Langflow component IDs that use AstraDB
**overrides: Override default values (e.g., database_name="custom_db")
Returns:
Tweaks dictionary for use with stepflow workflows
"""
builder = TweaksBuilder()
for component_id in component_ids:
builder.add_astradb_tweaks(component_id, **overrides)
return builder.build_or_skip()

View File

@ -0,0 +1,126 @@
"""Integration tests for example flows from the Stepflow documentation.
Tests the full end-to-end path used by the ``stepflow-langflow run`` CLI:
LangflowConverter -> StepflowClient -> orchestrator -> langflow worker.
The POC flow requires ``OPENAI_API_KEY`` for execution tests.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import Any
import pytest
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _find_poc_flow() -> Path:
"""Locate the POC flow JSON from the sibling stepflow repo's docs."""
# This file is at tests/integration/test_example_flows.py
here = Path(__file__).resolve()
# here.parents[4] = langflow repo root (e.g. /Users/.../langflow)
langflow_repo = here.parents[4]
candidate = langflow_repo.parent / "stepflow" / "docs" / "static" / "files" / "2025-09-langflow-poc-flow.json"
if candidate.exists():
return candidate
# Env var fallback
if env_path := os.environ.get("STEPFLOW_POC_FLOW"):
p = Path(env_path)
if p.exists():
return p
pytest.skip("POC flow not found. Set STEPFLOW_POC_FLOW or ensure sibling stepflow repo exists.")
def requires_openai() -> pytest.MarkDecorator:
"""Skip if OPENAI_API_KEY is not set."""
return pytest.mark.skipif(
not os.environ.get("OPENAI_API_KEY"),
reason="OPENAI_API_KEY not set",
)
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(scope="module")
def poc_flow_data() -> dict[str, Any]:
"""Load the POC flow JSON."""
path = _find_poc_flow()
with open(path) as f:
return json.load(f)
# ---------------------------------------------------------------------------
# Translation tests (no orchestrator required)
# ---------------------------------------------------------------------------
@pytest.mark.integration
def test_poc_flow_translation(converter, poc_flow_data):
"""The POC flow must translate to a valid Stepflow flow with correct
variable schema for OPENAI_API_KEY."""
flow = converter.convert(poc_flow_data)
assert flow is not None, "convert() returned None"
assert flow.steps, "translated flow has no steps"
for step in flow.steps:
assert step.component.startswith(("/langflow/", "/builtin/")), f"unexpected component prefix: {step.component}"
# Verify the variable schema includes OPENAI_API_KEY with env_var annotation
import msgspec
flow_dict = msgspec.to_builtins(flow)
var_props = flow_dict.get("schemas", {}).get("properties", {}).get("variables", {}).get("properties", {})
assert "OPENAI_API_KEY" in var_props, f"Expected OPENAI_API_KEY in variable schema, got: {list(var_props.keys())}"
assert var_props["OPENAI_API_KEY"].get("env_var") == "OPENAI_API_KEY", (
f"Expected env_var annotation, got: {var_props['OPENAI_API_KEY']}"
)
# ---------------------------------------------------------------------------
# Execution tests (require orchestrator + OPENAI_API_KEY)
# ---------------------------------------------------------------------------
@pytest.mark.integration
@pytest.mark.asyncio(loop_scope="module")
@requires_openai()
async def test_poc_flow_execution(runner, poc_flow_data):
"""The POC flow must execute end-to-end with variables populated from env.
Uses StepflowRunner which delegates to StepflowClient with
``populate_variables_from_env=True``, exercising the same code path
as ``stepflow-langflow run --local``.
"""
flow_data = poc_flow_data.get("data", poc_flow_data)
run_outputs, session_id = await runner.run(
flow_data=flow_data,
input_value=None,
session_id=None,
)
assert run_outputs, "Expected at least one RunOutputs"
assert session_id
result = run_outputs[0].outputs[0]
assert result is not None, "Expected a ResultData"
# The flow produces a blog post — verify we got non-trivial text
has_output = bool(result.messages) or result.results is not None
assert has_output, "Expected messages or results from POC flow"
if result.messages:
text = result.messages[0].message
assert text and len(text) > 100, f"Expected substantial text output, got: {text!r:.200}"

View File

@ -0,0 +1 @@
"""Unit tests."""

View File

@ -0,0 +1,180 @@
"""Unit tests for the BaseExecutor shared functionality."""
from typing import Any
from unittest.mock import MagicMock
import pytest
from langflow_stepflow.worker.base_executor import BaseExecutor
class ConcreteTestExecutor(BaseExecutor):
"""Concrete implementation of BaseExecutor for testing."""
async def _instantiate_component(
self,
component_info: dict[str, Any],
) -> tuple[Any, str]:
"""Test implementation that just returns the info as-is."""
return component_info.get("instance"), component_info.get("name", "test")
@pytest.fixture
def executor():
"""Create a ConcreteTestExecutor instance for testing base functionality."""
return ConcreteTestExecutor()
class TestBaseExecutorDetermineExecutionMethod:
"""Tests for _determine_execution_method in BaseExecutor."""
def test_with_selected_output_match(self, executor):
"""Test finding method for matching selected_output."""
outputs = [
{"name": "text", "method": "text_response"},
{"name": "message", "method": "build_message"},
]
result = executor._determine_execution_method(outputs, "message")
assert result == "build_message"
def test_fallback_to_first(self, executor):
"""Test fallback to first output's method."""
outputs = [
{"name": "default", "method": "default_method"},
{"name": "other", "method": "other_method"},
]
result = executor._determine_execution_method(outputs, None)
assert result == "default_method"
def test_empty_outputs(self, executor):
"""Test with empty outputs list."""
result = executor._determine_execution_method([], None)
assert result is None
def test_selected_not_found_fallback(self, executor):
"""Test fallback when selected_output doesn't match."""
outputs = [{"name": "text", "method": "text_method"}]
result = executor._determine_execution_method(outputs, "nonexistent")
assert result == "text_method"
class TestBaseExecutorApplyInputDefaults:
"""Tests for _apply_component_input_defaults in BaseExecutor."""
def test_no_inputs_attribute(self, executor):
"""Test with component that has no inputs attribute."""
component = MagicMock(spec=[]) # No inputs attribute
params = {"key": "value"}
result = executor._apply_component_input_defaults(component, params)
assert result == {"key": "value"}
def test_adds_missing_defaults(self, executor):
"""Test that defaults are added for missing parameters."""
input_def = MagicMock()
input_def.name = "temperature"
input_def.value = 0.7
component = MagicMock()
component.inputs = [input_def]
params = {"model": "gpt-4"}
result = executor._apply_component_input_defaults(component, params)
assert result == {"model": "gpt-4", "temperature": 0.7}
def test_preserves_existing_values(self, executor):
"""Test that existing params are not overwritten."""
input_def = MagicMock()
input_def.name = "temperature"
input_def.value = 0.7
component = MagicMock()
component.inputs = [input_def]
params = {"temperature": 0.9} # Already set
result = executor._apply_component_input_defaults(component, params)
assert result == {"temperature": 0.9} # Original preserved
class TestBaseExecutorApplyOutputHandlers:
"""Tests for _apply_output_handlers in BaseExecutor."""
@pytest.mark.asyncio
async def test_serializes_lfx_dataframe(self, executor):
"""Test that lfx DataFrames are serialized by the handler chain."""
from lfx.schema.dataframe import DataFrame
df = DataFrame(data=[{"text": "row1", "url": "http://example.com"}])
handlers = executor._get_output_handlers()
result = await executor._apply_output_handlers(df, handlers)
assert isinstance(result, dict)
assert result["__langflow_type__"] == "DataFrame"
assert "json_data" in result
@pytest.mark.asyncio
async def test_serializes_plain_pandas_dataframe(self, executor):
"""Plain pandas DataFrames must also serialize (issue #673).
Components compiled from flow JSON blobs may produce plain pandas
DataFrames instead of lfx DataFrames.
"""
import pandas as pd
pdf = pd.DataFrame([{"text": "row1", "url": "http://example.com"}])
handlers = executor._get_output_handlers()
result = await executor._apply_output_handlers(pdf, handlers)
assert isinstance(result, dict)
assert result["__langflow_type__"] == "DataFrame"
assert "json_data" in result
@pytest.mark.asyncio
async def test_serializes_message(self, executor):
"""Test that Messages are serialized correctly."""
from lfx.schema.message import Message
msg = Message(text="hello")
handlers = executor._get_output_handlers()
result = await executor._apply_output_handlers(msg, handlers)
assert isinstance(result, dict)
assert result["__langflow_type__"] == "Message"
assert result["text"] == "hello"
@pytest.mark.asyncio
async def test_passes_through_primitives(self, executor):
"""Test that primitive values pass through unchanged."""
handlers = executor._get_output_handlers()
assert await executor._apply_output_handlers("hello", handlers) == "hello"
assert await executor._apply_output_handlers(42, handlers) == 42
assert await executor._apply_output_handlers(None, handlers) is None
@pytest.mark.asyncio
async def test_recurses_into_dicts(self, executor):
"""Test that dicts are recursed into."""
handlers = executor._get_output_handlers()
result = await executor._apply_output_handlers({"key": "value", "count": 42}, handlers)
assert result == {"key": "value", "count": 42}
@pytest.mark.asyncio
async def test_raises_on_unknown_type(self, executor):
"""Test that unknown types raise ValueError."""
handlers = executor._get_output_handlers()
with pytest.raises(ValueError, match="Cannot serialize object of type"):
await executor._apply_output_handlers(object(), handlers)
class TestBaseExecutorSetupGraphContext:
"""Tests for _setup_graph_context in BaseExecutor."""
def test_sets_graph_context(self, executor):
"""Test that graph context is set correctly."""
component = MagicMock()
component.__dict__ = {}
executor._setup_graph_context(component, "test-session-id")
assert "graph" in component.__dict__
assert component.__dict__["graph"].session_id == "test-session-id"
assert component.__dict__["graph"].vertices == []
assert component.__dict__["graph"].flow_id is None

View File

@ -0,0 +1,289 @@
"""Unit tests for the CoreExecutor."""
from unittest.mock import AsyncMock, MagicMock
import pytest
from langflow_stepflow.exceptions import ExecutionError
from langflow_stepflow.worker.core_executor import CoreExecutor
@pytest.fixture
def executor():
"""Create a CoreExecutor instance."""
return CoreExecutor()
@pytest.fixture
def mock_context():
"""Create a mock StepflowContext."""
context = MagicMock()
context.get_blob = AsyncMock(return_value={})
context.put_blob = AsyncMock(return_value="blob_id")
return context
class TestCoreExecutorExecutionMethodDetermination:
"""Tests for _determine_execution_method inherited from base."""
def test_determine_execution_method_with_selected_output(self, executor):
"""Test determining method when selected_output matches."""
outputs = [
{"name": "text", "method": "text_response"},
{"name": "message", "method": "build_message"},
]
result = executor._determine_execution_method(outputs, "message")
assert result == "build_message"
def test_determine_execution_method_fallback_to_first(self, executor):
"""Test falling back to first output's method."""
outputs = [
{"name": "default", "method": "default_method"},
{"name": "other", "method": "other_method"},
]
result = executor._determine_execution_method(outputs, None)
assert result == "default_method"
def test_determine_execution_method_empty_outputs(self, executor):
"""Test with empty outputs list."""
result = executor._determine_execution_method([], None)
assert result is None
def test_determine_execution_method_selected_not_found(self, executor):
"""Test when selected_output doesn't match any output."""
outputs = [{"name": "text", "method": "text_method"}]
result = executor._determine_execution_method(outputs, "nonexistent")
# Should fall back to first output's method
assert result == "text_method"
class TestCoreExecutorInputDefaults:
"""Tests for _apply_component_input_defaults inherited from base."""
def test_apply_defaults_no_inputs(self, executor):
"""Test with component that has no inputs attribute."""
component = MagicMock(spec=[]) # No inputs attribute
params = {"key": "value"}
result = executor._apply_component_input_defaults(component, params)
assert result == {"key": "value"}
def test_apply_defaults_empty_inputs(self, executor):
"""Test with empty inputs list."""
component = MagicMock()
component.inputs = []
params = {"key": "value"}
result = executor._apply_component_input_defaults(component, params)
assert result == {"key": "value"}
def test_apply_defaults_adds_missing(self, executor):
"""Test that defaults are added for missing parameters."""
input_def = MagicMock()
input_def.name = "temperature"
input_def.value = 0.7
component = MagicMock()
component.inputs = [input_def]
params = {"model": "gpt-4"}
result = executor._apply_component_input_defaults(component, params)
assert result == {"model": "gpt-4", "temperature": 0.7}
def test_apply_defaults_preserves_existing(self, executor):
"""Test that existing params are not overwritten."""
input_def = MagicMock()
input_def.name = "temperature"
input_def.value = 0.7
component = MagicMock()
component.inputs = [input_def]
params = {"temperature": 0.9} # Already set
result = executor._apply_component_input_defaults(component, params)
assert result == {"temperature": 0.9} # Original value preserved
class TestCoreExecutorErrors:
"""Tests for error handling in CoreExecutor."""
@pytest.mark.asyncio
async def test_execute_invalid_path_no_dot(self, executor, mock_context):
"""Test error when path has no class name separator."""
with pytest.raises(ExecutionError, match="Invalid component path"):
await executor.execute("invalid", {}, mock_context)
@pytest.mark.asyncio
async def test_execute_module_not_found(self, executor, mock_context):
"""Test error when module doesn't exist."""
input_data = {
"template": {},
"outputs": [{"name": "result", "method": "run"}],
"input": {},
}
with pytest.raises(ExecutionError, match="Failed to import module"):
await executor.execute("nonexistent/module/path/ClassName", input_data, mock_context)
@pytest.mark.asyncio
async def test_execute_class_not_found(self, executor, mock_context):
"""Test error when class doesn't exist in module."""
input_data = {
"template": {},
"outputs": [{"name": "result", "method": "run"}],
"input": {},
}
# os module exists but NonExistentClass doesn't
with pytest.raises(ExecutionError, match="Class.*not found in module"):
await executor.execute("os/NonExistentClass", input_data, mock_context)
@pytest.mark.asyncio
async def test_execute_no_execution_method(self, executor, mock_context):
"""Test error when no execution method found."""
input_data = {
"template": {},
"outputs": [], # Empty outputs
"input": {},
}
# Use a real importable class
with pytest.raises(ExecutionError, match="No execution method found"):
await executor.execute("dataclasses/dataclass", input_data, mock_context)
class TestCoreExecutorWithRealComponent:
"""Integration tests with real Langflow components.
Note: Some Langflow components have complex dependencies (database, etc.)
These tests focus on verifying the executor can import and instantiate
components, rather than full execution which may require more setup.
"""
@pytest.mark.asyncio
async def test_import_prompt_component(self, executor, mock_context):
"""Test that PromptComponent can be imported and instantiated."""
import importlib
try:
module = importlib.import_module("langflow.components.prompts.prompt")
component_class = module.PromptComponent
# Verify we can instantiate
instance = component_class()
assert instance is not None
assert hasattr(instance, "build_prompt")
except ModuleNotFoundError:
# langflow.components.prompts may not be available in all environments
# Skip test if not available
pytest.skip("langflow.components.prompts not available")
@pytest.mark.asyncio
async def test_import_chat_input_component(self, executor, mock_context):
"""Test that ChatInput component can be imported and instantiated."""
import importlib
module = importlib.import_module("lfx.components.input_output.chat")
component_class = module.ChatInput
# Verify we can instantiate
instance = component_class()
assert instance is not None
assert hasattr(instance, "message_response")
@pytest.mark.asyncio
async def test_execute_with_mocked_method(self, executor, mock_context):
"""Test execution flow with a mocked component method."""
from unittest.mock import MagicMock, patch
input_data = {
"template": {
"template": {"value": "Hello {name}!"},
},
"outputs": [{"name": "prompt", "method": "build_prompt"}],
"selected_output": "prompt",
"input": {
"name": "World",
},
}
# Mock the component instantiation and method
mock_instance = MagicMock()
mock_instance.inputs = []
mock_instance.build_prompt = MagicMock(return_value="Hello World!")
with patch("importlib.import_module") as mock_import:
mock_module = MagicMock()
mock_module.TestComponent = MagicMock(return_value=mock_instance)
mock_import.return_value = mock_module
result = await executor.execute(
"test_module/TestComponent",
input_data,
mock_context,
)
assert "result" in result
# Method was called
mock_instance.build_prompt.assert_called_once()
class TestCoreExecutorPrepareParameters:
"""Tests for _prepare_component_parameters."""
@pytest.mark.asyncio
async def test_extract_values_from_template(self, executor):
"""Test extracting values from template structure."""
template = {
"param1": {"value": "value1"},
"param2": {"value": 42},
"param3": "direct_value",
}
runtime_inputs = {}
result = await executor._prepare_component_parameters(template, runtime_inputs)
assert result["param1"] == "value1"
assert result["param2"] == 42
assert result["param3"] == "direct_value"
@pytest.mark.asyncio
async def test_runtime_inputs_override_template(self, executor):
"""Test that runtime inputs override template values."""
template = {
"param1": {"value": "template_value"},
}
runtime_inputs = {
"param1": "runtime_value",
}
result = await executor._prepare_component_parameters(template, runtime_inputs)
assert result["param1"] == "runtime_value"
@pytest.mark.asyncio
async def test_empty_template_dict_skipped(self, executor):
"""Test that template fields without value are skipped."""
template = {
"param1": {"value": "has_value"},
"param2": {}, # No value key
"param3": {"type": "str"}, # No value key
}
runtime_inputs = {}
result = await executor._prepare_component_parameters(template, runtime_inputs)
assert result == {"param1": "has_value"}
@pytest.mark.asyncio
async def test_handle_inputs_with_empty_values_skipped(self, executor):
"""Test that handle inputs with empty values are skipped."""
template = {
"input_value": {
"value": "",
"input_types": ["Message"], # This is a handle input
},
"regular_param": {"value": "keep_this"},
}
runtime_inputs = {}
result = await executor._prepare_component_parameters(template, runtime_inputs)
# Handle input with empty value should be skipped
assert "input_value" not in result
assert result["regular_param"] == "keep_this"

View File

@ -0,0 +1,78 @@
"""Unit tests for DependencyAnalyzer."""
import pytest
from langflow_stepflow.translation.dependency_analyzer import (
DependencyAnalyzer,
)
class TestDependencyAnalyzer:
"""Test DependencyAnalyzer functionality."""
def test_build_empty_dependency_graph(self):
"""Test building dependency graph from empty edges."""
analyzer = DependencyAnalyzer()
dependencies = analyzer.build_dependency_graph([])
assert dependencies == {}
def test_build_simple_dependency_graph(self):
"""Test building dependency graph from simple edges."""
analyzer = DependencyAnalyzer()
edges = [
{"source": "A", "target": "B"},
{"source": "B", "target": "C"},
]
dependencies = analyzer.build_dependency_graph(edges)
assert dependencies == {"B": ["A"], "C": ["B"]}
def test_build_complex_dependency_graph(self):
"""Test building dependency graph with multiple sources."""
analyzer = DependencyAnalyzer()
edges = [
{"source": "A", "target": "C"},
{"source": "B", "target": "C"},
{"source": "C", "target": "D"},
]
dependencies = analyzer.build_dependency_graph(edges)
assert dependencies == {"C": ["A", "B"], "D": ["C"]}
def test_get_execution_order_simple(self):
"""Test execution order for simple dependency chain."""
analyzer = DependencyAnalyzer()
dependencies = {"B": ["A"], "C": ["B"]}
order = analyzer.get_execution_order(dependencies)
assert order == ["A", "B", "C"]
def test_get_execution_order_complex(self):
"""Test execution order for complex dependency graph."""
analyzer = DependencyAnalyzer()
dependencies = {"C": ["A", "B"], "D": ["C"], "E": ["B"]}
order = analyzer.get_execution_order(dependencies)
# A and B should come before C
assert order.index("A") < order.index("C")
assert order.index("B") < order.index("C")
# C should come before D
assert order.index("C") < order.index("D")
# B should come before E
assert order.index("B") < order.index("E")
# All nodes should be present
assert set(order) == {"A", "B", "C", "D", "E"}
def test_circular_dependency_detection(self):
"""Test detection of circular dependencies."""
analyzer = DependencyAnalyzer()
dependencies = {"A": ["B"], "B": ["C"], "C": ["A"]} # Circular!
with pytest.raises(ValueError, match="Circular dependencies detected"):
analyzer.get_execution_order(dependencies)

View File

@ -0,0 +1,312 @@
"""Unit tests for input/output handlers."""
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import Any
from unittest.mock import MagicMock
import pytest
from langflow_stepflow.worker.base_executor import BaseExecutor
from langflow_stepflow.worker.handlers import (
DataFrameConversionInputHandler,
InputHandler,
StringCoercionInputHandler,
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
class ConcreteTestExecutor(BaseExecutor):
"""Concrete BaseExecutor subclass for testing _handler_pipeline."""
async def _instantiate_component(
self,
component_info: dict[str, Any],
) -> tuple[Any, str]:
return component_info.get("instance"), component_info.get("name", "test")
# ---------------------------------------------------------------------------
# StringCoercionInputHandler
# ---------------------------------------------------------------------------
class TestStringCoercionInputHandler:
def test_matches_str_type_with_message_value(self):
handler = StringCoercionInputHandler()
msg = MagicMock()
msg.__class__ = type("Message", (), {})
msg.text = "hello"
assert handler.matches(template_field={"type": "str"}, value=msg) is True
def test_no_match_other_types(self):
handler = StringCoercionInputHandler()
assert handler.matches(template_field={"type": "int"}, value="hello") is False
assert handler.matches(template_field={"type": "file"}, value="hello") is False
assert handler.matches(template_field={}, value="hello") is False
def test_no_match_str_type_with_plain_string(self):
handler = StringCoercionInputHandler()
assert handler.matches(template_field={"type": "str"}, value="already a string") is False
@pytest.mark.asyncio
async def test_coerces_message_to_text(self):
handler = StringCoercionInputHandler()
msg = MagicMock()
msg.__class__ = type("Message", (), {})
msg.text = "hello world"
fields = {"input_value": (msg, {"type": "str"})}
result = await handler.prepare(fields, None)
assert result == {"input_value": "hello world"}
@pytest.mark.asyncio
async def test_passes_through_non_message_objects(self):
handler = StringCoercionInputHandler()
fields = {"input_value": (42, {"type": "str"})}
result = await handler.prepare(fields, None)
assert result == {}
# ---------------------------------------------------------------------------
# DataFrameConversionInputHandler
# ---------------------------------------------------------------------------
class TestDataFrameConversionInputHandler:
def test_matches_dataframe_in_input_types_with_list(self):
handler = DataFrameConversionInputHandler()
data_list = [{"text": "row1"}]
assert (
handler.matches(
template_field={"input_types": ["DataFrame", "Data"]},
value=data_list,
)
is True
)
def test_no_match_without_dataframe(self):
handler = DataFrameConversionInputHandler()
assert handler.matches(template_field={"input_types": ["Message"]}, value=[]) is False
assert handler.matches(template_field={}, value=[]) is False
def test_no_match_empty_list(self):
handler = DataFrameConversionInputHandler()
assert handler.matches(template_field={"input_types": ["DataFrame"]}, value=[]) is False
def test_no_match_non_list(self):
handler = DataFrameConversionInputHandler()
assert handler.matches(template_field={"input_types": ["DataFrame"]}, value="not a list") is False
@pytest.mark.asyncio
async def test_converts_data_list(self):
handler = DataFrameConversionInputHandler()
data_list = [
{"text": "row1", "value": 1},
{"text": "row2", "value": 2},
]
fields = {
"data_input": (
data_list,
{"input_types": ["DataFrame"]},
),
}
result = await handler.prepare(fields, None)
assert "data_input" in result
import pandas
assert isinstance(result["data_input"], pandas.DataFrame)
@pytest.mark.asyncio
async def test_skips_empty_list(self):
handler = DataFrameConversionInputHandler()
fields = {"data_input": ([], {"input_types": ["DataFrame"]})}
result = await handler.prepare(fields, None)
assert result == {}
@pytest.mark.asyncio
async def test_skips_non_list(self):
handler = DataFrameConversionInputHandler()
fields = {"data_input": ("not a list", {"input_types": ["DataFrame"]})}
result = await handler.prepare(fields, None)
assert result == {}
@pytest.mark.asyncio
async def test_skips_non_data_list(self):
handler = DataFrameConversionInputHandler()
fields = {
"data_input": (
[1, 2, 3], # not Data-like
{"input_types": ["DataFrame"]},
),
}
result = await handler.prepare(fields, None)
assert result == {}
@pytest.mark.asyncio
async def test_skips_all_none_list(self):
handler = DataFrameConversionInputHandler()
fields = {
"data_input": (
[None, None], # no non-null items
{"input_types": ["DataFrame"]},
),
}
result = await handler.prepare(fields, None)
assert result == {}
# ---------------------------------------------------------------------------
# _handler_pipeline
# ---------------------------------------------------------------------------
class TestHandlerPipeline:
@pytest.fixture
def executor(self):
return ConcreteTestExecutor()
@pytest.mark.asyncio
async def test_runs_handlers_in_order(self, executor):
"""Handlers should be applied sequentially, each seeing previous results."""
call_order: list[str] = []
class HandlerA(InputHandler):
def matches(self, *, template_field, value):
return template_field.get("handle_a", False)
async def prepare(self, fields, context):
call_order.append("A")
return {k: v + "_A" for k, (v, _) in fields.items()}
class HandlerB(InputHandler):
def matches(self, *, template_field, value):
return template_field.get("handle_b", False)
async def prepare(self, fields, context):
call_order.append("B")
return {k: v + "_B" for k, (v, _) in fields.items()}
parameters = {"x": "val"}
template = {"x": {"handle_a": True, "handle_b": True}}
executor._get_input_handlers = lambda: [HandlerA(), HandlerB()]
executor._get_output_handlers = lambda: []
async with executor._handler_pipeline(parameters, template) as (result, _):
assert call_order == ["A", "B"]
assert result["x"] == "val_A_B"
@pytest.mark.asyncio
async def test_skips_handlers_with_no_matches(self, executor):
"""activate() should not be called when no fields match."""
activated = []
class TrackingHandler(InputHandler):
def matches(self, *, template_field, value):
return False # never matches
@asynccontextmanager
async def activate(self) -> AsyncIterator[Any]:
activated.append(True)
yield None
async def prepare(self, fields, context):
return {}
parameters = {"x": "val"}
template = {"x": {"type": "str"}}
executor._get_input_handlers = lambda: [TrackingHandler()]
executor._get_output_handlers = lambda: []
async with executor._handler_pipeline(parameters, template) as (result, _):
assert activated == [] # activate was never called
assert result == {"x": "val"}
@pytest.mark.asyncio
async def test_passes_context_from_activate(self, executor):
"""Context yielded by activate() should be passed to prepare()."""
received_contexts: list[Any] = []
class ContextHandler(InputHandler):
def matches(self, *, template_field, value):
return True
@asynccontextmanager
async def activate(self) -> AsyncIterator[str]:
yield "my_context"
async def prepare(self, fields, context):
received_contexts.append(context)
return {}
parameters = {"x": "val"}
template = {"x": {"type": "str"}}
executor._get_input_handlers = lambda: [ContextHandler()]
executor._get_output_handlers = lambda: []
async with executor._handler_pipeline(parameters, template) as (_, __):
pass
assert received_contexts == ["my_context"]
@pytest.mark.asyncio
async def test_non_dict_template_fields_get_empty_metadata(self, executor):
"""Non-dict template entries are normalised to {} before matching.
Handlers that require specific metadata keys won't match them.
"""
class RequiresTypeHandler(InputHandler):
def matches(self, *, template_field, value):
return "type" in template_field
async def prepare(self, fields, context):
return dict.fromkeys(fields, "changed")
parameters = {"x": "val", "y": "val2"}
template = {"x": {"type": "str"}, "y": "direct_value"}
executor._get_input_handlers = lambda: [RequiresTypeHandler()]
executor._get_output_handlers = lambda: []
async with executor._handler_pipeline(parameters, template) as (result, _):
assert result["x"] == "changed"
assert result["y"] == "val2" # not matched (no "type" key)
@pytest.mark.asyncio
async def test_empty_handlers_is_noop(self, executor):
parameters = {"x": "val"}
template = {"x": {"type": "str"}}
executor._get_input_handlers = lambda: []
executor._get_output_handlers = lambda: []
async with executor._handler_pipeline(parameters, template) as (result, _):
assert result == {"x": "val"}
@pytest.mark.asyncio
async def test_handler_updates_are_merged(self, executor):
"""Only keys returned by prepare() should be updated."""
class SelectiveHandler(InputHandler):
def matches(self, *, template_field, value):
return True
async def prepare(self, fields, context):
# Only update "a", leave "b" alone
return {"a": "updated"}
parameters = {"a": "orig_a", "b": "orig_b"}
template = {"a": {"type": "str"}, "b": {"type": "str"}}
executor._get_input_handlers = lambda: [SelectiveHandler()]
executor._get_output_handlers = lambda: []
async with executor._handler_pipeline(parameters, template) as (result, _):
assert result["a"] == "updated"
assert result["b"] == "orig_b"

View File

@ -0,0 +1,633 @@
"""Unit tests for LangflowType input/output handlers and DataFrame output handler.
Tests serialization (output) and deserialization (input) of Langflow
Message, Data, and DataFrame types, including round-trip tests.
"""
import json
import pandas as pd
import pytest
from lfx.schema.data import Data
from lfx.schema.dataframe import DataFrame
from lfx.schema.message import Message
from langflow_stepflow.worker.handlers import (
DataFrameOutputHandler,
LangflowTypeInputHandler,
LangflowTypeOutputHandler,
)
# ---------------------------------------------------------------------------
# LangflowTypeOutputHandler — matches()
# ---------------------------------------------------------------------------
class TestLangflowTypeOutputHandlerMatches:
def setup_method(self):
self.handler = LangflowTypeOutputHandler()
def test_matches_message(self):
assert self.handler.matches(value=Message(text="hi")) is True
def test_matches_data(self):
assert self.handler.matches(value=Data(text="row")) is True
def test_does_not_match_dataframe(self):
df = DataFrame(data=[{"text": "a"}])
assert self.handler.matches(value=df) is False
def test_does_not_match_dict(self):
assert self.handler.matches(value={"text": "hi"}) is False
def test_does_not_match_string(self):
assert self.handler.matches(value="hello") is False
def test_does_not_match_none(self):
assert self.handler.matches(value=None) is False
def test_does_not_match_int(self):
assert self.handler.matches(value=42) is False
# ---------------------------------------------------------------------------
# LangflowTypeOutputHandler — process()
# ---------------------------------------------------------------------------
class TestLangflowTypeOutputHandlerProcess:
def setup_method(self):
self.handler = LangflowTypeOutputHandler()
@pytest.mark.asyncio
async def test_serialize_message(self):
msg = Message(text="hello world")
result = await self.handler.process(msg)
assert result["__langflow_type__"] == "Message"
assert result["text"] == "hello world"
assert isinstance(result, dict)
@pytest.mark.asyncio
async def test_serialize_message_preserves_fields(self):
msg = Message(text="test", sender="user", sender_name="Alice")
result = await self.handler.process(msg)
assert result["text"] == "test"
assert result["sender"] == "user"
assert result["sender_name"] == "Alice"
assert result["__langflow_type__"] == "Message"
@pytest.mark.asyncio
async def test_serialize_data(self):
data = Data(text="some data")
result = await self.handler.process(data)
assert result["__langflow_type__"] == "Data"
assert result["text"] == "some data"
assert isinstance(result, dict)
@pytest.mark.asyncio
async def test_serialize_data_with_extra_fields(self):
data = Data(data={"key": "value", "count": 42})
result = await self.handler.process(data)
assert result["__langflow_type__"] == "Data"
# Data.model_dump() flattens the data dict into top-level keys
assert result["key"] == "value"
assert result["count"] == 42
@pytest.mark.asyncio
async def test_serialize_message_empty_text(self):
msg = Message(text="")
result = await self.handler.process(msg)
assert result["__langflow_type__"] == "Message"
assert result["text"] == ""
@pytest.mark.asyncio
async def test_serialize_data_unicode(self):
data = Data(text="Unicode: \u00e9\u00e0\u00fc \u4e16\u754c \ud83c\udf0d")
result = await self.handler.process(data)
assert result["text"] == "Unicode: \u00e9\u00e0\u00fc \u4e16\u754c \ud83c\udf0d"
# ---------------------------------------------------------------------------
# DataFrameOutputHandler — matches()
# ---------------------------------------------------------------------------
class TestDataFrameOutputHandlerMatches:
def setup_method(self):
self.handler = DataFrameOutputHandler()
def test_matches_lfx_dataframe(self):
df = DataFrame(data=[{"text": "row1"}])
assert self.handler.matches(value=df) is True
def test_matches_plain_pandas_dataframe(self):
"""Plain pandas DataFrames should also match (issue #673)."""
pdf = pd.DataFrame([{"text": "row1"}])
assert self.handler.matches(value=pdf) is True
def test_does_not_match_message(self):
assert self.handler.matches(value=Message(text="hi")) is False
def test_does_not_match_data(self):
assert self.handler.matches(value=Data(text="hi")) is False
def test_does_not_match_dict(self):
assert self.handler.matches(value={"data": []}) is False
def test_does_not_match_list(self):
assert self.handler.matches(value=[1, 2, 3]) is False
# ---------------------------------------------------------------------------
# DataFrameOutputHandler — process()
# ---------------------------------------------------------------------------
class TestDataFrameOutputHandlerProcess:
def setup_method(self):
self.handler = DataFrameOutputHandler()
@pytest.mark.asyncio
async def test_serialize_dataframe(self):
df = DataFrame(data=[{"text": "row1", "value": 1}])
result = await self.handler.process(df)
assert result["__langflow_type__"] == "DataFrame"
assert "json_data" in result
assert result["text_key"] == "text"
assert result["default_value"] == ""
# Verify json_data is valid split-format JSON
parsed = json.loads(result["json_data"])
assert "columns" in parsed
assert "data" in parsed
@pytest.mark.asyncio
async def test_serialize_dataframe_multiple_rows(self):
data = [
{"text": "row1", "count": 10},
{"text": "row2", "count": 20},
{"text": "row3", "count": 30},
]
df = DataFrame(data=data)
result = await self.handler.process(df)
parsed = json.loads(result["json_data"])
assert len(parsed["data"]) == 3
@pytest.mark.asyncio
async def test_serialize_dataframe_preserves_text_key(self):
df = DataFrame(data=[{"content": "test"}], text_key="content")
result = await self.handler.process(df)
assert result["text_key"] == "content"
@pytest.mark.asyncio
async def test_serialize_dataframe_single_row(self):
df = DataFrame(data=[{"text": "only row"}])
result = await self.handler.process(df)
parsed = json.loads(result["json_data"])
assert len(parsed["data"]) == 1
@pytest.mark.asyncio
async def test_serialize_plain_pandas_dataframe(self):
"""Plain pandas DataFrames should serialize correctly (issue #673)."""
pdf = pd.DataFrame([{"text": "row1", "url": "http://example.com"}])
result = await self.handler.process(pdf)
assert result["__langflow_type__"] == "DataFrame"
assert "json_data" in result
# Plain pandas DataFrames don't have text_key/default_value attrs
assert result["text_key"] == "text"
assert result["default_value"] == ""
parsed = json.loads(result["json_data"])
assert "columns" in parsed
assert len(parsed["data"]) == 1
# ---------------------------------------------------------------------------
# LangflowTypeInputHandler — matches()
# ---------------------------------------------------------------------------
class TestLangflowTypeInputHandlerMatches:
def setup_method(self):
self.handler = LangflowTypeInputHandler()
def test_matches_dict_with_marker(self):
value = {"__langflow_type__": "Message", "text": "hi"}
assert self.handler.matches(template_field={}, value=value) is True
def test_matches_list_with_marked_items(self):
value = [
{"__langflow_type__": "Data", "text": "row1"},
{"__langflow_type__": "Data", "text": "row2"},
]
assert self.handler.matches(template_field={}, value=value) is True
def test_matches_mixed_list(self):
"""List with some marked items should still match."""
value = [
{"__langflow_type__": "Data", "text": "row1"},
"plain string",
]
assert self.handler.matches(template_field={}, value=value) is True
def test_no_match_plain_dict(self):
assert self.handler.matches(template_field={}, value={"text": "hi"}) is False
def test_no_match_empty_list(self):
assert self.handler.matches(template_field={}, value=[]) is False
def test_no_match_string(self):
assert self.handler.matches(template_field={}, value="hello") is False
def test_no_match_none(self):
assert self.handler.matches(template_field={}, value=None) is False
def test_no_match_list_without_markers(self):
value = [{"text": "row1"}, {"text": "row2"}]
assert self.handler.matches(template_field={}, value=value) is False
# ---------------------------------------------------------------------------
# LangflowTypeInputHandler — prepare() (Message)
# ---------------------------------------------------------------------------
class TestLangflowTypeInputHandlerMessage:
def setup_method(self):
self.handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_deserialize_message(self):
serialized = {"__langflow_type__": "Message", "text": "hello"}
fields = {"msg": (serialized, {})}
result = await self.handler.prepare(fields, None)
assert isinstance(result["msg"], Message)
assert result["msg"].text == "hello"
@pytest.mark.asyncio
async def test_deserialize_message_with_sender(self):
serialized = {
"__langflow_type__": "Message",
"text": "test",
"sender": "user",
"sender_name": "Alice",
}
fields = {"msg": (serialized, {})}
result = await self.handler.prepare(fields, None)
msg = result["msg"]
assert isinstance(msg, Message)
assert msg.text == "test"
assert msg.sender == "user"
assert msg.sender_name == "Alice"
@pytest.mark.asyncio
async def test_deserialize_message_empty_text(self):
serialized = {"__langflow_type__": "Message", "text": ""}
fields = {"msg": (serialized, {})}
result = await self.handler.prepare(fields, None)
assert isinstance(result["msg"], Message)
assert result["msg"].text == ""
# ---------------------------------------------------------------------------
# LangflowTypeInputHandler — prepare() (Data)
# ---------------------------------------------------------------------------
class TestLangflowTypeInputHandlerData:
def setup_method(self):
self.handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_deserialize_data(self):
serialized = {"__langflow_type__": "Data", "text": "some data"}
fields = {"data_field": (serialized, {})}
result = await self.handler.prepare(fields, None)
assert isinstance(result["data_field"], Data)
assert result["data_field"].text == "some data"
@pytest.mark.asyncio
async def test_deserialize_data_with_extra_fields(self):
serialized = {
"__langflow_type__": "Data",
"data": {"key": "value", "count": 42},
}
fields = {"data_field": (serialized, {})}
result = await self.handler.prepare(fields, None)
data = result["data_field"]
assert isinstance(data, Data)
assert data.data["key"] == "value"
# ---------------------------------------------------------------------------
# LangflowTypeInputHandler — prepare() (DataFrame)
# ---------------------------------------------------------------------------
class TestLangflowTypeInputHandlerDataFrame:
def setup_method(self):
self.handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_deserialize_dataframe(self):
# Build the split-JSON format that DataFrameOutputHandler produces
import pandas as pd
pd_df = pd.DataFrame([{"text": "row1", "count": 1}])
json_str = pd_df.to_json(orient="split")
serialized = {
"__langflow_type__": "DataFrame",
"json_data": json_str,
"text_key": "text",
"default_value": "",
}
fields = {"df_field": (serialized, {})}
result = await self.handler.prepare(fields, None)
df = result["df_field"]
assert isinstance(df, pd.DataFrame)
@pytest.mark.asyncio
async def test_deserialize_dataframe_with_none_values(self):
"""DataFrames with NaN/None values should be handled."""
import pandas as pd
pd_df = pd.DataFrame([{"text": "row1", "value": None}])
json_str = pd_df.to_json(orient="split")
serialized = {
"__langflow_type__": "DataFrame",
"json_data": json_str,
"text_key": "text",
"default_value": "",
}
fields = {"df_field": (serialized, {})}
result = await self.handler.prepare(fields, None)
assert isinstance(result["df_field"], pd.DataFrame)
@pytest.mark.asyncio
async def test_deserialize_dataframe_missing_json_data_falls_back(self):
"""Missing json_data should trigger recovery fallback."""
serialized = {
"__langflow_type__": "DataFrame",
"text_key": "text",
"default_value": "",
}
fields = {"df_field": (serialized, {})}
result = await self.handler.prepare(fields, None)
# Recovery also fails (no json_data at all) — returns raw dict
assert isinstance(result["df_field"], dict)
# ---------------------------------------------------------------------------
# LangflowTypeInputHandler — prepare() (list handling)
# ---------------------------------------------------------------------------
class TestLangflowTypeInputHandlerLists:
def setup_method(self):
self.handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_deserialize_list_of_messages(self):
value = [
{"__langflow_type__": "Message", "text": "msg1"},
{"__langflow_type__": "Message", "text": "msg2"},
]
fields = {"msgs": (value, {})}
result = await self.handler.prepare(fields, None)
assert len(result["msgs"]) == 2
assert isinstance(result["msgs"][0], Message)
assert isinstance(result["msgs"][1], Message)
assert result["msgs"][0].text == "msg1"
assert result["msgs"][1].text == "msg2"
@pytest.mark.asyncio
async def test_deserialize_list_of_data(self):
value = [
{"__langflow_type__": "Data", "text": "d1"},
{"__langflow_type__": "Data", "text": "d2"},
]
fields = {"items": (value, {})}
result = await self.handler.prepare(fields, None)
assert len(result["items"]) == 2
assert all(isinstance(item, Data) for item in result["items"])
@pytest.mark.asyncio
async def test_mixed_list_preserves_unmarked_items(self):
"""Non-marked items in a list should pass through unchanged."""
value = [
{"__langflow_type__": "Message", "text": "msg1"},
"plain string",
42,
None,
]
fields = {"items": (value, {})}
result = await self.handler.prepare(fields, None)
items = result["items"]
assert len(items) == 4
assert isinstance(items[0], Message)
assert items[1] == "plain string"
assert items[2] == 42
assert items[3] is None
@pytest.mark.asyncio
async def test_deserialize_multiple_fields(self):
"""Multiple fields should all be processed."""
fields = {
"msg": ({"__langflow_type__": "Message", "text": "hello"}, {}),
"data": ({"__langflow_type__": "Data", "text": "world"}, {}),
}
result = await self.handler.prepare(fields, None)
assert isinstance(result["msg"], Message)
assert isinstance(result["data"], Data)
# ---------------------------------------------------------------------------
# LangflowTypeInputHandler — unknown / edge cases
# ---------------------------------------------------------------------------
class TestLangflowTypeInputHandlerEdgeCases:
def setup_method(self):
self.handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_unknown_langflow_type_returns_dict(self):
"""Unknown __langflow_type__ values should return the raw dict."""
serialized = {"__langflow_type__": "UnknownType", "data": "test"}
fields = {"field": (serialized, {})}
result = await self.handler.prepare(fields, None)
assert isinstance(result["field"], dict)
assert result["field"]["__langflow_type__"] == "UnknownType"
@pytest.mark.asyncio
async def test_empty_langflow_type_returns_dict(self):
"""Empty __langflow_type__ value should return the raw dict."""
serialized = {"__langflow_type__": "", "text": "test"}
fields = {"field": (serialized, {})}
result = await self.handler.prepare(fields, None)
assert isinstance(result["field"], dict)
# ---------------------------------------------------------------------------
# Round-trip tests: serialize → deserialize
# ---------------------------------------------------------------------------
class TestLangflowTypeRoundTrip:
def setup_method(self):
self.output_handler = LangflowTypeOutputHandler()
self.input_handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_message_round_trip(self):
original = Message(text="hello world")
serialized = await self.output_handler.process(original)
fields = {"msg": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
restored = result["msg"]
assert isinstance(restored, Message)
assert restored.text == original.text
@pytest.mark.asyncio
async def test_message_with_metadata_round_trip(self):
original = Message(text="test", sender="user", sender_name="Bob")
serialized = await self.output_handler.process(original)
fields = {"msg": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
restored = result["msg"]
assert isinstance(restored, Message)
assert restored.text == "test"
assert restored.sender == "user"
assert restored.sender_name == "Bob"
@pytest.mark.asyncio
async def test_data_round_trip(self):
original = Data(text="some data", data={"key": "value"})
serialized = await self.output_handler.process(original)
fields = {"data": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
restored = result["data"]
assert isinstance(restored, Data)
assert restored.text == original.text
@pytest.mark.asyncio
async def test_list_of_messages_round_trip(self):
originals = [Message(text="msg1"), Message(text="msg2")]
serialized_list = [await self.output_handler.process(msg) for msg in originals]
fields = {"msgs": (serialized_list, {})}
result = await self.input_handler.prepare(fields, None)
restored = result["msgs"]
assert len(restored) == 2
assert all(isinstance(m, Message) for m in restored)
assert restored[0].text == "msg1"
assert restored[1].text == "msg2"
class TestDataFrameRoundTrip:
"""Round-trip tests for DataFrame: serialize with DataFrameOutputHandler,
deserialize with LangflowTypeInputHandler."""
def setup_method(self):
self.output_handler = DataFrameOutputHandler()
self.input_handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_dataframe_round_trip(self):
original = DataFrame(
data=[
{"text": "row1", "count": 10},
{"text": "row2", "count": 20},
]
)
serialized = await self.output_handler.process(original)
assert serialized["__langflow_type__"] == "DataFrame"
fields = {"df": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
restored = result["df"]
assert isinstance(restored, pd.DataFrame)
@pytest.mark.asyncio
async def test_dataframe_round_trip_preserves_text_key(self):
original = DataFrame(
data=[{"content": "test"}],
text_key="content",
)
serialized = await self.output_handler.process(original)
assert serialized["text_key"] == "content"
fields = {"df": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
restored = result["df"]
assert isinstance(restored, pd.DataFrame)
class TestPandasDataFrameRoundTrip:
"""Round-trip tests for plain pandas DataFrames (issue #673).
Components compiled from flow JSON blobs may produce plain pandas
DataFrames instead of lfx DataFrames, depending on the Langflow
version that exported the flow.
"""
def setup_method(self):
self.output_handler = DataFrameOutputHandler()
self.input_handler = LangflowTypeInputHandler()
@pytest.mark.asyncio
async def test_pandas_dataframe_round_trip(self):
original = pd.DataFrame(
[
{"text": "row1", "count": 10},
{"text": "row2", "count": 20},
]
)
serialized = await self.output_handler.process(original)
assert serialized["__langflow_type__"] == "DataFrame"
fields = {"df": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
restored = result["df"]
assert isinstance(restored, pd.DataFrame)

View File

@ -0,0 +1,113 @@
"""Unit tests for SchemaMapper."""
from langflow_stepflow.translation.schema_mapper import SchemaMapper
class TestSchemaMapper:
"""Test SchemaMapper functionality."""
def test_init(self):
"""Test initialization."""
mapper = SchemaMapper()
assert "str" in mapper.langflow_to_json_schema
assert mapper.langflow_to_json_schema["str"] == "string"
def test_extract_output_schema_from_outputs(self):
"""Test schema extraction from outputs metadata."""
mapper = SchemaMapper()
node = {
"data": {
"type": "ChatInput",
"outputs": [{"name": "message", "types": ["Message"]}],
}
}
schema = mapper.extract_output_schema(node)
assert schema["type"] == "object"
assert "text" in schema["properties"]
assert schema["properties"]["type"]["const"] == "Message"
def test_extract_output_schema_from_base_classes(self):
"""Test schema extraction from base classes."""
mapper = SchemaMapper()
node = {"data": {"type": "CustomComponent", "base_classes": ["Data"]}}
schema = mapper.extract_output_schema(node)
assert schema["type"] == "object"
assert "data" in schema["properties"]
assert schema["properties"]["type"]["const"] == "Data"
def test_extract_output_schema_from_heuristics(self):
"""Test schema extraction using component heuristics."""
mapper = SchemaMapper()
node = {"data": {"type": "ChatInput"}}
schema = mapper.extract_output_schema(node)
assert schema["type"] == "object"
assert "text" in schema["properties"]
assert schema["properties"]["type"]["const"] == "Message"
def test_extract_output_schema_fallback(self):
"""Test fallback schema for unknown components."""
mapper = SchemaMapper()
node = {"data": {"type": "UnknownComponent"}}
schema = mapper.extract_output_schema(node)
assert schema["type"] == "object"
assert "result" in schema["properties"]
def test_extract_input_schema_simple(self):
"""Test input schema extraction from template."""
mapper = SchemaMapper()
node = {
"data": {
"node": {
"template": {
"input_value": {
"type": "str",
"info": "Input message",
"required": True,
},
"temperature": {
"type": "float",
"info": "Temperature setting",
"required": False,
},
}
}
}
}
schema = mapper.extract_input_schema(node)
assert schema["type"] == "object"
assert len(schema["properties"]) == 2
assert schema["properties"]["input_value"]["type"] == "string"
assert schema["properties"]["temperature"]["type"] == "number"
assert schema["required"] == ["input_value"]
def test_extract_input_schema_dropdown(self):
"""Test input schema with dropdown field."""
mapper = SchemaMapper()
node = {
"data": {
"node": {
"template": {
"model": {
"type": "dropdown",
"options": ["gpt-3.5-turbo", "gpt-4"],
"info": "Model selection",
}
}
}
}
}
schema = mapper.extract_input_schema(node)
assert schema["properties"]["model"]["type"] == "string"
assert schema["properties"]["model"]["enum"] == ["gpt-3.5-turbo", "gpt-4"]

View File

@ -0,0 +1,340 @@
"""Tests for Stepflow-level tweaks functionality."""
import pytest
from langflow_stepflow.translation.stepflow_tweaks import (
apply_stepflow_tweaks_to_dict,
convert_tweaks_to_overrides,
)
from langflow_stepflow.translation.translator import LangflowConverter
from tests.helpers.tweaks_builder import TweaksBuilder
class TestStepflowTweaks:
"""Test Stepflow-level tweaks functionality."""
def test_convert_tweaks_to_overrides_basic(self):
"""Test basic tweaks to overrides conversion."""
tweaks = {
"LanguageModelComponent-kBOja": {
"api_key": "test-api-key", # pragma: allowlist secret
"temperature": 0.7,
}
}
overrides = convert_tweaks_to_overrides(tweaks)
assert overrides is not None
assert "langflow_LanguageModelComponent-kBOja" in overrides
step_override = overrides["langflow_LanguageModelComponent-kBOja"]
assert step_override["$type"] == "merge_patch"
assert "input" in step_override["value"]
input_overrides = step_override["value"]["input"]["input"]
assert input_overrides["api_key"] == "test-api-key" # pragma: allowlist secret
assert input_overrides["temperature"] == 0.7
def test_convert_tweaks_to_overrides_empty(self):
"""Test conversion with no tweaks returns None."""
assert convert_tweaks_to_overrides(None) is None
assert convert_tweaks_to_overrides({}) is None
def test_convert_tweaks_to_overrides_multiple_steps(self):
"""Test conversion with multiple steps."""
tweaks = {
"Step1": {"field1": "value1"},
"Step2": {"field2": "value2", "field3": 123},
}
overrides = convert_tweaks_to_overrides(tweaks)
assert len(overrides) == 2
assert "langflow_Step1" in overrides
assert "langflow_Step2" in overrides
# Check Step1
assert overrides["langflow_Step1"]["$type"] == "merge_patch"
step1_input = overrides["langflow_Step1"]["value"]["input"]["input"]
assert step1_input["field1"] == "value1"
# Check Step2
assert overrides["langflow_Step2"]["$type"] == "merge_patch"
step2_input = overrides["langflow_Step2"]["value"]["input"]["input"]
assert step2_input["field2"] == "value2"
assert step2_input["field3"] == 123
class TestStepflowTweaksIntegration:
"""Test integration with the full Langflow conversion process."""
@pytest.fixture
def basic_prompting_flow_dict(self):
"""Convert basic_prompting.json to a dict for testing."""
import json
from pathlib import Path
converter = LangflowConverter()
fixture_path = Path(__file__).parent.parent / "fixtures" / "langflow" / "basic_prompting.json"
if fixture_path.exists():
with open(fixture_path) as f:
langflow_data = json.load(f)
# Convert to Flow object and get dict representation
flow = converter.convert(langflow_data)
import msgspec
return msgspec.to_builtins(flow)
else:
pytest.skip("basic_prompting.json fixture not found")
def test_real_workflow_tweaks_application(self, basic_prompting_flow_dict):
"""Test tweaks application on a real converted workflow."""
tweaks = {
"LanguageModelComponent-kBOja": { # Must match actual component ID
"api_key": "integration_test_key", # pragma: allowlist secret
"temperature": 0.7,
"model_name": "gpt-4",
}
}
modified_dict = apply_stepflow_tweaks_to_dict(basic_prompting_flow_dict, tweaks)
# Find the LanguageModelComponent executor step (custom_code or core)
langflow_step = None
for step in modified_dict["steps"]:
if step["id"] == "langflow_LanguageModelComponent-kBOja" and (
step["component"] == "/langflow/custom_code" or step["component"].startswith("/langflow/core/")
):
langflow_step = step
break
assert langflow_step is not None, "LanguageModelComponent executor step not found"
# Verify tweaks were applied
input_section = langflow_step.get("input", {}).get("input", {})
assert input_section.get("api_key") == "integration_test_key" # pragma: allowlist secret
assert input_section.get("temperature") == 0.7
assert input_section.get("model_name") == "gpt-4"
def test_tweaks_preserve_existing_inputs(self, basic_prompting_flow_dict):
"""Test that tweaks preserve existing input values that aren't overwritten."""
tweaks = {
"LanguageModelComponent-kBOja": {
"api_key": "new_key", # pragma: allowlist secret
}
}
modified_dict = apply_stepflow_tweaks_to_dict(basic_prompting_flow_dict, tweaks)
# Find the step
for step in modified_dict["steps"]:
if step["id"] == "langflow_LanguageModelComponent-kBOja":
input_section = step.get("input", {}).get("input", {})
# New value should be applied
assert input_section.get("api_key") == "new_key"
# Other fields from the original workflow should still exist
# (The exact fields depend on the fixture content)
break
def test_empty_tweaks_returns_unchanged(self, basic_prompting_flow_dict):
"""Test that empty tweaks returns the workflow unchanged."""
import copy
original = copy.deepcopy(basic_prompting_flow_dict)
# Empty dict
result = apply_stepflow_tweaks_to_dict(basic_prompting_flow_dict, {})
assert result == original
# None
result = apply_stepflow_tweaks_to_dict(basic_prompting_flow_dict, None)
assert result == original
class TestTweaksBuilder:
"""Test the TweaksBuilder utility for creating tweaks."""
def test_basic_tweak_building(self):
"""Test basic tweak creation with direct values."""
builder = TweaksBuilder()
builder.add_tweak("Component-123", "api_key", "test_key") # pragma: allowlist secret
builder.add_tweak("Component-123", "temperature", 0.8)
builder.add_tweak("AnotherComponent-456", "model", "gpt-4")
tweaks = builder.build()
expected = {
"Component-123": {"api_key": "test_key", "temperature": 0.8}, # pragma: allowlist secret
"AnotherComponent-456": {"model": "gpt-4"},
}
assert tweaks == expected
def test_method_chaining(self):
"""Test that methods can be chained for fluent API."""
tweaks = (
TweaksBuilder()
.add_tweak("Component-123", "api_key", "test_key")
.add_tweak("Component-123", "temperature", 0.8)
.build()
)
expected = {"Component-123": {"api_key": "test_key", "temperature": 0.8}} # pragma: allowlist secret
assert tweaks == expected
def test_env_tweak_with_existing_variable(self, monkeypatch):
"""Test adding tweaks from environment variables that exist."""
monkeypatch.setenv("TEST_API_KEY", "env_test_key")
monkeypatch.setenv("TEST_TEMPERATURE", "0.9")
tweaks = (
TweaksBuilder()
.add_env_tweak("Component-123", "api_key", "TEST_API_KEY") # pragma: allowlist secret
.add_env_tweak("Component-123", "temperature", "TEST_TEMPERATURE")
.build()
)
expected = {
"Component-123": {
"api_key": "env_test_key", # pragma: allowlist secret
"temperature": "0.9", # Environment variables are strings
}
}
assert tweaks == expected
def test_env_tweak_with_missing_variable(self, monkeypatch):
"""Test that missing environment variables are tracked."""
# Ensure the env var doesn't exist
monkeypatch.delenv("MISSING_API_KEY", raising=False)
builder = TweaksBuilder()
builder.add_env_tweak("Component-123", "api_key", "MISSING_API_KEY")
# Should track missing var
assert "MISSING_API_KEY" in builder.missing_env_vars
# Should raise error on build
with pytest.raises(ValueError, match="Missing required environment variables: MISSING_API_KEY"):
builder.build()
def test_build_or_skip_with_missing_env_vars(self, monkeypatch):
"""Test that build_or_skip skips test when env vars are missing."""
monkeypatch.delenv("MISSING_VAR", raising=False)
builder = TweaksBuilder()
builder.add_env_tweak("Component-123", "field", "MISSING_VAR")
# Should call pytest.skip
with pytest.raises(
pytest.skip.Exception,
match="Missing required environment variables: MISSING_VAR",
):
builder.build_or_skip()
def test_build_or_skip_with_all_env_vars_present(self, monkeypatch):
"""Test that build_or_skip works normally when all env vars are present."""
monkeypatch.setenv("PRESENT_VAR", "test_value")
tweaks = TweaksBuilder().add_env_tweak("Component-123", "field", "PRESENT_VAR").build_or_skip()
expected = {"Component-123": {"field": "test_value"}}
assert tweaks == expected
def test_add_astradb_tweaks(self, monkeypatch):
"""Test the convenience method for AstraDB tweaks."""
monkeypatch.setenv("ASTRA_DB_API_ENDPOINT", "https://astra-endpoint.com")
tweaks = TweaksBuilder().add_astradb_tweaks("AstraDB-store-123").build()
expected = {
"AstraDB-store-123": {
"api_endpoint": "https://astra-endpoint.com",
"database_name": "langflow-test", # Default value
"collection_name": "test_collection", # Default value
}
}
assert tweaks == expected
def test_add_astradb_tweaks_with_overrides(self, monkeypatch):
"""Test AstraDB tweaks with overridden default values."""
monkeypatch.setenv("ASTRA_DB_API_ENDPOINT", "https://astra-endpoint.com")
tweaks = (
TweaksBuilder()
.add_astradb_tweaks(
"AstraDB-store-123",
database_name="custom_db",
collection_name="custom_collection",
extra_field="extra_value",
)
.build()
)
expected = {
"AstraDB-store-123": {
"api_endpoint": "https://astra-endpoint.com",
"database_name": "custom_db",
"collection_name": "custom_collection",
"extra_field": "extra_value",
}
}
assert tweaks == expected
def test_mixed_tweaks_and_env_vars(self, monkeypatch):
"""Test combining direct tweaks and environment variable tweaks."""
monkeypatch.setenv("API_KEY", "env_api_key")
tweaks = (
TweaksBuilder()
.add_env_tweak("Component-123", "api_key", "API_KEY")
.add_tweak("Component-123", "temperature", 0.5)
.add_tweak("Component-456", "model", "claude-3")
.build()
)
expected = {
"Component-123": {"api_key": "env_api_key", "temperature": 0.5}, # pragma: allowlist secret
"Component-456": {"model": "claude-3"},
}
assert tweaks == expected
def test_overwriting_tweaks(self):
"""Test that later tweaks overwrite earlier ones."""
tweaks = (
TweaksBuilder()
.add_tweak("Component-123", "temperature", 0.3)
.add_tweak("Component-123", "temperature", 0.7) # Should overwrite
.build()
)
expected = {"Component-123": {"temperature": 0.7}}
assert tweaks == expected
def test_empty_builder(self):
"""Test that empty builder produces empty tweaks."""
tweaks = TweaksBuilder().build()
assert tweaks == {}
def test_multiple_missing_env_vars(self, monkeypatch):
"""Test error handling with multiple missing environment variables."""
monkeypatch.delenv("MISSING_VAR_1", raising=False)
monkeypatch.delenv("MISSING_VAR_2", raising=False)
builder = TweaksBuilder()
builder.add_env_tweak("Component-123", "field1", "MISSING_VAR_1")
builder.add_env_tweak("Component-456", "field2", "MISSING_VAR_2")
with pytest.raises(ValueError) as exc_info:
builder.build()
error_msg = str(exc_info.value)
assert "MISSING_VAR_1" in error_msg
assert "MISSING_VAR_2" in error_msg

View File

@ -0,0 +1,273 @@
"""Unit tests for ToolWrapperInputHandler."""
import pytest
from langchain_core.tools import StructuredTool
from langflow_stepflow.worker.handlers.tool_wrapper import (
ToolWrapperInputHandler,
_create_tool_from_wrapper,
_execute_calculator_tool,
)
def _make_tool_wrapper(
*,
name: str = "test_tool",
description: str = "A test tool",
component_code: str | None = "print('hello')",
code_blob_id: str | None = None,
properties: dict | None = None,
static_inputs: dict | None = None,
component_type: str = "TestComponent",
session_id: str = "test_session",
) -> dict:
"""Create a tool wrapper dict for testing."""
wrapper: dict = {
"__tool_wrapper__": True,
"tool_metadata": {"name": name, "description": description},
"tool_input_schema": {"properties": properties or {}},
"static_inputs": static_inputs or {},
"component_type": component_type,
"session_id": session_id,
}
if component_code is not None:
wrapper["component_code"] = component_code
if code_blob_id is not None:
wrapper["code_blob_id"] = code_blob_id
return wrapper
# ---------------------------------------------------------------------------
# _execute_calculator_tool
# ---------------------------------------------------------------------------
class TestExecuteCalculatorTool:
def test_simple_addition(self):
assert _execute_calculator_tool("2 + 3") == "5"
def test_multiplication(self):
assert _execute_calculator_tool("6 * 7") == "42"
def test_division(self):
assert _execute_calculator_tool("10 / 4") == "2.5"
def test_power(self):
assert _execute_calculator_tool("2 ** 10") == "1024"
def test_complex_expression(self):
assert _execute_calculator_tool("(2 + 3) * 4") == "20"
def test_float_result(self):
assert _execute_calculator_tool("1 / 3") == "0.333333"
def test_invalid_expression(self):
result = _execute_calculator_tool("not_valid")
assert "Calculator error" in result
def test_division_by_zero(self):
result = _execute_calculator_tool("1 / 0")
assert "Calculator error" in result
# ---------------------------------------------------------------------------
# _create_tool_from_wrapper
# ---------------------------------------------------------------------------
class TestCreateToolFromWrapper:
def test_creates_structured_tool(self):
wrapper = _make_tool_wrapper(name="my_tool", description="does stuff")
tool = _create_tool_from_wrapper(wrapper)
assert isinstance(tool, StructuredTool)
assert tool.name == "my_tool"
assert tool.description == "does stuff"
def test_creates_tool_with_code_blob_id(self):
wrapper = _make_tool_wrapper(
component_code=None,
code_blob_id="abc123",
)
tool = _create_tool_from_wrapper(wrapper)
assert isinstance(tool, StructuredTool)
def test_tool_with_input_schema(self):
wrapper = _make_tool_wrapper(
properties={
"query": {"type": "string", "default": ""},
"limit": {"type": "integer", "default": "10"},
},
)
tool = _create_tool_from_wrapper(wrapper)
assert isinstance(tool, StructuredTool)
def test_calculator_tool_execution(self):
wrapper = _make_tool_wrapper(
name="evaluate_expression",
properties={"expression": {"type": "string", "default": ""}},
)
tool = _create_tool_from_wrapper(wrapper)
result = tool.invoke({"expression": "2 + 3"})
assert result == {"result": "5"}
def test_non_calculator_tool_execution(self):
wrapper = _make_tool_wrapper(
name="search_tool",
properties={"query": {"type": "string", "default": ""}},
static_inputs={"api_key": "test_key"}, # pragma: allowlist secret
component_type="SearchComponent",
session_id="sess_1",
)
tool = _create_tool_from_wrapper(wrapper)
result = tool.invoke({"query": "hello"})
assert result["component_type"] == "SearchComponent"
assert result["inputs"]["query"] == "hello"
assert result["inputs"]["api_key"] == "test_key" # pragma: allowlist secret
assert result["inputs"]["session_id"] == "sess_1"
assert result["status"] == "tool_wrapper_execution"
def test_tool_with_code_blob_id_in_result(self):
wrapper = _make_tool_wrapper(
name="blob_tool",
component_code=None,
code_blob_id="blob_abc",
)
tool = _create_tool_from_wrapper(wrapper)
result = tool.invoke({})
assert result["code_blob_id"] == "blob_abc"
def test_tool_with_component_code_in_result(self):
wrapper = _make_tool_wrapper(
name="code_tool",
component_code="def run(): pass",
)
tool = _create_tool_from_wrapper(wrapper)
result = tool.invoke({})
assert result["has_component_code"] is True
def test_missing_both_code_sources_returns_failed_wrapper(self):
wrapper = _make_tool_wrapper(component_code=None, code_blob_id=None)
tool = _create_tool_from_wrapper(wrapper)
# Should be a FailedToolWrapper, not a StructuredTool
assert not isinstance(tool, StructuredTool)
assert hasattr(tool, "name")
assert tool.name == "test_tool"
result = tool.invoke({})
assert "error" in result
assert "Tool creation failed" in result["error"]
# ---------------------------------------------------------------------------
# ToolWrapperInputHandler.matches
# ---------------------------------------------------------------------------
class TestToolWrapperInputHandlerMatches:
def setup_method(self):
self.handler = ToolWrapperInputHandler()
def test_matches_dict_with_tool_wrapper_marker(self):
wrapper = _make_tool_wrapper()
assert self.handler.matches(template_field={}, value=wrapper) is True
def test_matches_list_with_tool_wrapper(self):
wrapper = _make_tool_wrapper()
assert self.handler.matches(template_field={}, value=[wrapper]) is True
def test_matches_list_with_mixed_items(self):
wrapper = _make_tool_wrapper()
assert self.handler.matches(template_field={}, value=[wrapper, "other"]) is True
def test_no_match_plain_dict(self):
assert self.handler.matches(template_field={}, value={"key": "value"}) is False
def test_no_match_dict_with_false_marker(self):
assert self.handler.matches(template_field={}, value={"__tool_wrapper__": False}) is False
def test_no_match_string(self):
assert self.handler.matches(template_field={}, value="hello") is False
def test_no_match_empty_list(self):
assert self.handler.matches(template_field={}, value=[]) is False
def test_no_match_list_without_wrappers(self):
assert self.handler.matches(template_field={}, value=["a", "b", 42]) is False
def test_no_match_none(self):
assert self.handler.matches(template_field={}, value=None) is False
# ---------------------------------------------------------------------------
# ToolWrapperInputHandler.prepare
# ---------------------------------------------------------------------------
class TestToolWrapperInputHandlerPrepare:
def setup_method(self):
self.handler = ToolWrapperInputHandler()
@pytest.mark.asyncio
async def test_prepare_single_wrapper(self):
wrapper = _make_tool_wrapper(name="my_tool")
fields = {"tools": (wrapper, {})}
result = await self.handler.prepare(fields, None)
assert "tools" in result
assert isinstance(result["tools"], StructuredTool)
assert result["tools"].name == "my_tool"
@pytest.mark.asyncio
async def test_prepare_list_of_wrappers(self):
wrapper_a = _make_tool_wrapper(name="tool_a")
wrapper_b = _make_tool_wrapper(name="tool_b")
fields = {"tools": ([wrapper_a, wrapper_b], {})}
result = await self.handler.prepare(fields, None)
assert "tools" in result
assert len(result["tools"]) == 2
assert isinstance(result["tools"][0], StructuredTool)
assert isinstance(result["tools"][1], StructuredTool)
assert result["tools"][0].name == "tool_a"
assert result["tools"][1].name == "tool_b"
@pytest.mark.asyncio
async def test_prepare_mixed_list(self):
wrapper = _make_tool_wrapper(name="real_tool")
fields = {"tools": ([wrapper, "not_a_wrapper", 42], {})}
result = await self.handler.prepare(fields, None)
assert "tools" in result
assert len(result["tools"]) == 3
assert isinstance(result["tools"][0], StructuredTool)
assert result["tools"][1] == "not_a_wrapper"
assert result["tools"][2] == 42
@pytest.mark.asyncio
async def test_prepare_multiple_fields(self):
wrapper_a = _make_tool_wrapper(name="tool_a")
wrapper_b = _make_tool_wrapper(name="tool_b")
fields = {
"primary_tool": (wrapper_a, {}),
"secondary_tool": (wrapper_b, {}),
}
result = await self.handler.prepare(fields, None)
assert isinstance(result["primary_tool"], StructuredTool)
assert isinstance(result["secondary_tool"], StructuredTool)
assert result["primary_tool"].name == "tool_a"
assert result["secondary_tool"].name == "tool_b"
@pytest.mark.asyncio
async def test_prepare_failed_wrapper_in_list(self):
good_wrapper = _make_tool_wrapper(name="good_tool")
bad_wrapper = _make_tool_wrapper(name="bad_tool", component_code=None, code_blob_id=None)
fields = {"tools": ([good_wrapper, bad_wrapper], {})}
result = await self.handler.prepare(fields, None)
assert isinstance(result["tools"][0], StructuredTool)
# Bad wrapper becomes FailedToolWrapper
assert not isinstance(result["tools"][1], StructuredTool)
assert result["tools"][1].name == "bad_tool"

View File

@ -0,0 +1,411 @@
"""Unit tests for LangflowConverter."""
from typing import Any
import pytest
from langflow_stepflow.exceptions import ConversionError
from langflow_stepflow.translation.translator import LangflowConverter
def get_step_input_dict(step) -> dict:
"""Extract the input dict from a step.
With the msgspec-based SDK, step.input is already a plain dict.
"""
if step.input is None:
return {}
if isinstance(step.input, dict):
return step.input
import msgspec
return msgspec.to_builtins(step.input)
class TestLangflowConverter:
"""Test LangflowConverter functionality."""
def test_init_default(self):
"""Test default initialization."""
converter = LangflowConverter()
assert converter is not None
def test_convert_simple_workflow(self, converter: LangflowConverter, simple_langflow_workflow: dict[str, Any]):
"""Test conversion of simple workflow."""
workflow = converter.convert(simple_langflow_workflow)
assert workflow.name == "Converted Langflow Workflow"
# ChatInput and ChatOutput are I/O connection points, not processing steps
# They handle Message type conversion but don't create workflow steps
assert len(workflow.steps) == 0
# Verify the workflow structure is valid (Flow is a proper object)
assert workflow is not None
# Workflow should have output that references input (passthrough)
assert workflow.output is not None
def test_convert_empty_nodes(self, converter: LangflowConverter):
"""Test conversion with empty nodes list."""
workflow_data = {"data": {"nodes": [], "edges": []}}
with pytest.raises(ConversionError, match="No nodes found"):
converter.convert(workflow_data)
def test_convert_missing_data(self, converter: LangflowConverter):
"""Test conversion with missing data key."""
workflow_data = {"invalid": "structure"}
with pytest.raises(ConversionError, match="missing 'data' key"):
converter.convert(workflow_data)
def test_to_yaml(self, converter: LangflowConverter, simple_langflow_workflow: dict[str, Any]):
"""Test YAML generation."""
workflow = converter.convert(simple_langflow_workflow)
yaml_output = converter.to_yaml(workflow)
assert isinstance(yaml_output, str)
assert "name: Converted Langflow Workflow" in yaml_output
# Should have output section since ChatInput/ChatOutput create passthrough
assert "output:" in yaml_output
def test_analyze_workflow(self, converter: LangflowConverter, simple_langflow_workflow: dict[str, Any]):
"""Test workflow analysis."""
analysis = converter.analyze(simple_langflow_workflow)
assert analysis.node_count == 2
assert analysis.edge_count == 1
assert "ChatInput" in analysis.component_types
assert "ChatOutput" in analysis.component_types
def test_step_ordering_with_dependencies(self, converter: LangflowConverter):
"""Test that steps are ordered based on dependencies, not node order."""
# Create workflow with intentionally wrong node order
trivial_code = """
from langflow.custom.custom_component.component import Component
from langflow.io import Output
class TrivialComponent(Component):
outputs = [Output(display_name="Output", name="output", method="build")]
def build(self) -> str:
return "trivial result"
"""
workflow_data = {
"data": {
"nodes": [
{
"id": "output-node", # This depends on others but appears first
"data": {
"type": "TrivialOutput",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
{
"id": "input-node", # This has no dependencies but appears last
"data": {
"type": "TrivialInput",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
{
"id": "middle-node", # This depends on input but appears middle
"data": {
"type": "TrivialMiddle",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
],
"edges": [
{
"source": "input-node",
"target": "middle-node",
"data": {
"sourceHandle": {"name": "output"},
"targetHandle": {"fieldName": "input_0"},
},
},
{
"source": "middle-node",
"target": "output-node",
"data": {
"sourceHandle": {"name": "output"},
"targetHandle": {"fieldName": "input_0"},
},
},
],
}
}
workflow = converter.convert(workflow_data)
# Check that steps are reordered based on dependencies
step_ids = [step.id for step in workflow.steps]
# The Prompt (middle-node) should generate steps (blob + UDF executor)
# Step count may vary with routing improvements, focus on dependency ordering
middle_node_steps = [sid for sid in step_ids if "middle-node" in sid]
assert len(middle_node_steps) > 0, f"Expected middle-node steps in {step_ids}"
# Verify no forward references
for i, step in enumerate(workflow.steps):
if hasattr(step, "input") and step.input:
step_input = get_step_input_dict(step)
for _key, value in step_input.items():
if isinstance(value, dict) and "$step" in str(value):
from_step = value.get("$step", "")
if from_step:
# Find position of referenced step
try:
ref_pos = step_ids.index(from_step)
assert ref_pos < i, (
f"Step {step.id} at position {i} "
f"references {from_step} at position {ref_pos} "
"(forward reference)"
)
except ValueError:
pytest.fail(f"Step {step.id} references undefined step {from_step}")
def test_complex_dependency_ordering(self, converter: LangflowConverter):
"""Test topological sorting with complex dependencies like memory_chatbot."""
# Simulate the memory_chatbot structure with trivial custom code
trivial_code = """
from langflow.custom.custom_component.component import Component
from langflow.io import Output
class TrivialComponent(Component):
outputs = [Output(display_name="Output", name="output", method="build")]
def build(self) -> str:
return "trivial result"
"""
workflow_data = {
"data": {
"nodes": [
{
"id": "ChatInput-1",
"data": {
"type": "TrivialChatInput",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
{
"id": "ChatOutput-2",
"data": {
"type": "TrivialChatOutput",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
{
"id": "Memory-3",
"data": {
"type": "TrivialMemory",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
{
"id": "Prompt-4",
"data": {
"type": "TrivialPrompt",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
{
"id": "LLM-5",
"data": {
"type": "TrivialLLM",
"node": {
"template": {"code": {"value": trivial_code}},
"outputs": [{"name": "output", "method": "build"}],
},
},
"type": "genericNode",
},
],
"edges": [
{
"source": "Memory-3",
"target": "Prompt-4",
"data": {
"sourceHandle": {"name": "output"},
"targetHandle": {"fieldName": "input_0"},
},
},
{
"source": "Prompt-4",
"target": "LLM-5",
"data": {
"sourceHandle": {"name": "output"},
"targetHandle": {"fieldName": "input_0"},
},
},
{
"source": "ChatInput-1",
"target": "LLM-5",
"data": {
"sourceHandle": {"name": "output"},
"targetHandle": {"fieldName": "input_1"},
},
},
{
"source": "LLM-5",
"target": "ChatOutput-2",
"data": {
"sourceHandle": {"name": "output"},
"targetHandle": {"fieldName": "input_0"},
},
},
],
}
}
workflow = converter.convert(workflow_data)
step_ids = [step.id for step in workflow.steps]
# Find the main processing steps (not blob steps)
# Step count may vary with routing improvements, focus on dependency ordering
memory_steps = [sid for sid in step_ids if "Memory-3" in sid and "_blob" not in sid]
prompt_steps = [sid for sid in step_ids if "Prompt-4" in sid and "_blob" not in sid]
llm_steps = [sid for sid in step_ids if "LLM-5" in sid and "_blob" not in sid]
# Verify the main processing steps exist
assert len(memory_steps) > 0, f"Expected memory processing step in {step_ids}"
assert len(prompt_steps) > 0, f"Expected prompt processing step in {step_ids}"
assert len(llm_steps) > 0, f"Expected llm processing step in {step_ids}"
# Check dependency ordering between main processing steps
memory_pos = step_ids.index(memory_steps[0])
prompt_pos = step_ids.index(prompt_steps[0])
llm_pos = step_ids.index(llm_steps[0])
# Memory should come before Prompt (Memory -> Prompt)
assert memory_pos < prompt_pos, f"Memory should come before Prompt: {step_ids}"
# Prompt should come before LLM (Prompt -> LLM)
assert prompt_pos < llm_pos, f"Prompt should come before LLM: {step_ids}"
def test_component_routing_strategy_with_custom_code(self):
"""Test that components with custom code create custom blobs."""
converter = LangflowConverter()
# Create workflow with component that has custom code
workflow_data = {
"data": {
"nodes": [
{
"id": "custom-component",
"data": {
"type": "CustomComponent",
"node": {
"template": {
"code": {
"value": """
from langflow.custom.custom_component.component import Component
from langflow.io import Output
class CustomComponent(Component):
def custom_method(self):
return "Custom implementation"
"""
},
"param1": {"type": "str", "value": "test"},
},
"outputs": [{"name": "output", "method": "custom_method"}],
"base_classes": ["Component"],
"display_name": "Custom Component",
"metadata": {"module": "custom.module"},
},
"outputs": [{"name": "output", "method": "custom_method"}],
},
}
],
"edges": [],
}
}
workflow = converter.convert(workflow_data)
# Should create blob + custom_code steps for component with custom code
step_components = [step.component for step in workflow.steps]
assert "/builtin/put_blob" in step_components, "Should create blob step for custom code"
assert "/langflow/custom_code" in step_components, "Should route custom code to custom code executor"
# Find the blob step and verify it contains the custom code
blob_steps = [step for step in workflow.steps if step.component == "/builtin/put_blob"]
assert len(blob_steps) > 0, "Should have at least one blob step"
blob_step = blob_steps[0]
blob_input = get_step_input_dict(blob_step)
blob_data = blob_input.get("data", {})
assert "code" in blob_data, "Blob should contain component code"
code_value = blob_data["code"]
# Code may be a plain string or a $literal wrapper depending on SDK version
if isinstance(code_value, dict):
assert "CustomComponent" in str(code_value), "Should contain custom component class"
else:
assert "CustomComponent" in code_value, "Should contain custom component class"
def test_component_routing_strategy_rejects_incomplete_components(self):
"""Test that components without custom code are rejected (unified approach)."""
converter = LangflowConverter()
# Create workflow with component missing custom code (invalid scenario)
workflow_data = {
"data": {
"nodes": [
{
"id": "incomplete-component",
"data": {
"type": "IncompleteComponent",
"node": {
"template": {
"param1": {"type": "str", "value": "test"}
# No "code" field - this is incomplete/invalid
},
"outputs": [{"name": "output", "method": "build"}],
"base_classes": ["Component"],
"display_name": "Incomplete Component",
"metadata": {},
},
"outputs": [{"name": "output", "method": "build"}],
},
}
],
"edges": [],
}
}
# Should raise ConversionError for components without custom code
with pytest.raises(ConversionError, match="has no custom code"):
converter.convert(workflow_data)

View File

@ -0,0 +1,426 @@
"""Unit tests for BaseModel serialization and deserialization handlers."""
import os
from typing import Any
from unittest.mock import patch
import pytest
from pydantic import BaseModel, SecretStr
from langflow_stepflow.worker.base_executor import BaseExecutor
from langflow_stepflow.worker.handlers import (
BaseModelInputHandler,
BaseModelOutputHandler,
)
from langflow_stepflow.worker.handlers.base_model import (
_is_secret_str_type,
)
# Test BaseModel classes
class SimpleTestModel(BaseModel):
"""Simple test model without special types."""
name: str
value: int
enabled: bool = True
class SecretTestModel(BaseModel):
"""Test model with SecretStr fields."""
name: str
api_key: SecretStr
optional_secret: SecretStr | None = None
class MockOpenAIEmbeddings(BaseModel):
"""Mock OpenAI embeddings model to test real-world scenario."""
model: str = "text-embedding-3-small"
openai_api_key: SecretStr
chunk_size: int = 1000
max_retries: int = 3
dimensions: int | None = None
class ConcreteTestExecutor(BaseExecutor):
"""Concrete BaseExecutor subclass for tree walker tests."""
async def _instantiate_component(self, component_info: dict[str, Any]) -> tuple[Any, str]:
return component_info.get("instance"), component_info.get("name", "test")
class TestBaseModelOutputHandler:
"""Test cases for BaseModelOutputHandler serialization."""
def setup_method(self):
"""Set up test fixtures."""
self.handler = BaseModelOutputHandler()
def test_simple_types_not_matched(self):
"""Test that simple types are not matched by BaseModelOutputHandler."""
assert not self.handler.matches(value="test")
assert not self.handler.matches(value=42)
assert not self.handler.matches(value=True)
assert not self.handler.matches(value=[1, 2, 3])
assert not self.handler.matches(value={"key": "value"})
assert not self.handler.matches(value=None)
@pytest.mark.asyncio
async def test_simple_basemodel_serialization(self):
"""Test serialization of simple BaseModel without special types."""
model = SimpleTestModel(name="test", value=42, enabled=False)
serialized = await self.handler.process(model)
# Check that it includes the data
assert serialized["name"] == "test"
assert serialized["value"] == 42
assert serialized["enabled"] is False
# Check that it includes class metadata
assert serialized["__class_name__"] == "SimpleTestModel"
assert serialized["__module_name__"] == "tests.unit.test_type_converter"
@pytest.mark.asyncio
async def test_secret_str_serialization_with_actual_secret(self):
"""Test that SecretStr fields are properly serialized with actual values."""
secret_value = "test-api-key-12345" # pragma: allowlist secret
model = SecretTestModel(
name="test",
api_key=SecretStr(secret_value),
optional_secret=SecretStr("optional-secret-value"),
)
serialized = await self.handler.process(model)
# Check that secret values are properly extracted
assert serialized["name"] == "test"
assert serialized["api_key"] == secret_value # pragma: allowlist secret
assert serialized["optional_secret"] == "optional-secret-value" # pragma: allowlist secret
# Check class metadata
assert serialized["__class_name__"] == "SecretTestModel"
assert serialized["__module_name__"] == "tests.unit.test_type_converter"
@pytest.mark.asyncio
async def test_secret_str_serialization_with_env_var_resolution(self):
"""Test that SecretStr fields with environment variable names are resolved."""
# Set up environment variable
test_api_key = "actual-api-key-from-env" # pragma: allowlist secret
with patch.dict(os.environ, {"TEST_API_KEY": test_api_key}):
# Create model with environment variable name as secret
model = SecretTestModel(
name="test",
api_key=SecretStr("TEST_API_KEY"), # Environment variable name
)
serialized = await self.handler.process(model)
# Check that environment variable was resolved
assert serialized["api_key"] == test_api_key
assert serialized["name"] == "test"
@pytest.mark.asyncio
async def test_secret_str_serialization_no_env_var_fallback(self):
"""Test SecretStr serialization when environment variable doesn't exist."""
# Create model with non-existent environment variable name
model = SecretTestModel(name="test", api_key=SecretStr("NON_EXISTENT_ENV_VAR"))
serialized = await self.handler.process(model)
# Should keep the original value if env var doesn't exist
assert serialized["api_key"] == "NON_EXISTENT_ENV_VAR" # pragma: allowlist secret
assert serialized["name"] == "test"
@pytest.mark.asyncio
async def test_openai_embeddings_like_serialization(self):
"""Test serialization of OpenAI-like embeddings model."""
api_key = "real-openai-key-12345" # pragma: allowlist secret
model = MockOpenAIEmbeddings(
model="text-embedding-3-small",
openai_api_key=SecretStr(api_key),
chunk_size=500,
dimensions=1536,
)
serialized = await self.handler.process(model)
# Check all fields are properly serialized
assert serialized["model"] == "text-embedding-3-small"
assert serialized["openai_api_key"] == api_key # SecretStr unwrapped
assert serialized["chunk_size"] == 500
assert serialized["max_retries"] == 3 # Default value
assert serialized["dimensions"] == 1536
# Check class metadata for reconstruction
assert serialized["__class_name__"] == "MockOpenAIEmbeddings"
assert serialized["__module_name__"] == "tests.unit.test_type_converter"
@pytest.mark.asyncio
async def test_openai_api_key_env_var_resolution(self):
"""Test that OPENAI_API_KEY environment variable is properly resolved."""
real_api_key = "proj-real-openai-key-from-environment" # pragma: allowlist secret
with patch.dict(os.environ, {"OPENAI_API_KEY": real_api_key}):
model = MockOpenAIEmbeddings(openai_api_key=SecretStr("OPENAI_API_KEY"))
serialized = await self.handler.process(model)
# Verify the actual API key value is serialized, not the env var name
assert serialized["openai_api_key"] == real_api_key
@pytest.mark.asyncio
async def test_mixed_secret_and_regular_fields(self):
"""Test model with both secret and regular fields."""
api_key = "secret-key-value" # pragma: allowlist secret
model = SecretTestModel(
name="production-model",
api_key=SecretStr(api_key),
optional_secret=None, # Test None value
)
serialized = await self.handler.process(model)
# Regular field should be unchanged
assert serialized["name"] == "production-model"
# Secret field should be unwrapped
assert serialized["api_key"] == api_key
# None secret field should remain None
assert serialized["optional_secret"] is None
@pytest.mark.asyncio
@patch.dict(os.environ, {}, clear=True) # Clear environment
async def test_secret_str_with_no_env_vars(self):
"""Test SecretStr serialization when no environment variables are set."""
model = SecretTestModel(name="test", api_key=SecretStr("MISSING_API_KEY"))
serialized = await self.handler.process(model)
# Should keep original value if env var resolution fails
assert serialized["api_key"] == "MISSING_API_KEY" # pragma: allowlist secret
@pytest.mark.asyncio
async def test_serialized_data_structure_debugging(self):
"""Test to show what serialized data looks like for debugging."""
api_key = "debug-key-12345" # pragma: allowlist secret
model = MockOpenAIEmbeddings(model="test-model", openai_api_key=SecretStr(api_key), chunk_size=123)
serialized = await self.handler.process(model)
# Verify the structure
assert "openai_api_key" in serialized
assert serialized["openai_api_key"] == api_key
assert "__class_name__" in serialized
assert "__module_name__" in serialized
class TestBaseModelInputHandler:
"""Test cases for BaseModelInputHandler deserialization."""
def setup_method(self):
"""Set up test fixtures."""
self.input_handler = BaseModelInputHandler()
self.output_handler = BaseModelOutputHandler()
@pytest.mark.asyncio
async def test_simple_basemodel_deserialization(self):
"""Test deserialization of simple BaseModel."""
model = SimpleTestModel(name="test", value=42, enabled=False)
serialized = await self.output_handler.process(model)
fields = {"param": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
deserialized = result["param"]
assert isinstance(deserialized, SimpleTestModel)
assert deserialized.name == "test"
assert deserialized.value == 42
assert deserialized.enabled is False
@pytest.mark.asyncio
async def test_openai_embeddings_like_deserialization(self):
"""Test deserialization of OpenAI-like embeddings model."""
api_key = "real-openai-key-12345" # pragma: allowlist secret
model = MockOpenAIEmbeddings(
model="text-embedding-ada-002",
openai_api_key=SecretStr(api_key),
chunk_size=750,
)
# Serialize then deserialize
serialized = await self.output_handler.process(model)
fields = {"param": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
deserialized = result["param"]
assert isinstance(deserialized, MockOpenAIEmbeddings)
assert deserialized.model == "text-embedding-ada-002"
assert deserialized.chunk_size == 750
assert deserialized.max_retries == 3 # Default
# Check that SecretStr field is reconstructed properly
assert isinstance(deserialized.openai_api_key, SecretStr)
assert deserialized.openai_api_key.get_secret_value() == api_key
@pytest.mark.asyncio
async def test_openai_api_key_env_var_round_trip(self):
"""Test that OPENAI_API_KEY environment variable is properly resolved."""
real_api_key = "proj-real-openai-key-from-environment" # pragma: allowlist secret
with patch.dict(os.environ, {"OPENAI_API_KEY": real_api_key}):
model = MockOpenAIEmbeddings(openai_api_key=SecretStr("OPENAI_API_KEY"))
serialized = await self.output_handler.process(model)
# Test full round-trip
fields = {"param": (serialized, {})}
result = await self.input_handler.prepare(fields, None)
deserialized = result["param"]
assert isinstance(deserialized.openai_api_key, SecretStr)
assert deserialized.openai_api_key.get_secret_value() == real_api_key
def test_deserialization_without_class_metadata(self):
"""Test that regular dicts without markers don't match."""
regular_dict = {"name": "test", "value": 42}
assert not self.input_handler.matches(template_field={}, value=regular_dict)
@pytest.mark.asyncio
async def test_deserialization_with_invalid_class_metadata(self):
"""Test deserialization gracefully handles invalid class metadata."""
invalid_serialized = {
"name": "test",
"__class_name__": "NonExistentClass",
"__module_name__": "non.existent.module",
}
fields = {"param": (invalid_serialized, {})}
result = await self.input_handler.prepare(fields, None)
# Should return the dict unchanged if class can't be imported
assert result["param"] == invalid_serialized
class TestIsSecretStrType:
"""Test the _is_secret_str_type helper function."""
def test_direct_secret_str_detection(self):
assert _is_secret_str_type(SecretStr)
def test_optional_secret_str_detection(self):
assert _is_secret_str_type(SecretStr | None)
def test_non_secret_types(self):
assert not _is_secret_str_type(str)
assert not _is_secret_str_type(int)
assert not _is_secret_str_type(str | None)
class TestOutputTreeWalker:
"""Test the recursive output tree walker in BaseExecutor."""
def setup_method(self):
self.executor = ConcreteTestExecutor()
self.handlers = [BaseModelOutputHandler()]
@pytest.mark.asyncio
async def test_simple_types_pass_through(self):
"""Test that simple types pass through the tree walker unchanged."""
assert await self.executor._apply_output_handlers("test", self.handlers) == "test"
assert await self.executor._apply_output_handlers(42, self.handlers) == 42
assert await self.executor._apply_output_handlers(True, self.handlers) is True
assert await self.executor._apply_output_handlers([1, 2, 3], self.handlers) == [
1,
2,
3,
]
assert await self.executor._apply_output_handlers({"key": "value"}, self.handlers) == {"key": "value"}
assert await self.executor._apply_output_handlers(None, self.handlers) is None
@pytest.mark.asyncio
async def test_non_basemodel_object_error(self):
"""Test that non-serializable objects raise appropriate errors."""
class NonSerializableClass:
def __init__(self):
self.value = "test"
obj = NonSerializableClass()
with pytest.raises(ValueError, match="Cannot serialize object of type"):
await self.executor._apply_output_handlers(obj, self.handlers)
@pytest.mark.asyncio
async def test_langflow_types_still_work(self):
"""Test that existing Langflow type serialization still works."""
try:
from langflow.schema.message import Message
message = Message(text="test message")
from langflow_stepflow.worker.handlers import (
LangflowTypeOutputHandler,
)
handlers = [LangflowTypeOutputHandler(), BaseModelOutputHandler()]
serialized = await self.executor._apply_output_handlers(message, handlers)
# Should have Langflow type metadata, not BaseModel metadata
assert "__langflow_type__" in serialized
assert serialized["__langflow_type__"] == "Message"
except ImportError:
# Skip if Langflow not available
pytest.skip("Langflow not available for testing")
@pytest.mark.asyncio
async def test_real_openai_embeddings_serialization(self):
"""Test with actual OpenAI embeddings class if available."""
try:
from langchain_openai import OpenAIEmbeddings
from pydantic import SecretStr
# Test with a real OpenAIEmbeddings instance
api_key = "test-real-openai-key" # pragma: allowlist secret
embeddings = OpenAIEmbeddings(
model="text-embedding-3-small",
openai_api_key=api_key, # Becomes SecretStr automatically
chunk_size=500,
)
# Serialize
serialized = await self.executor._apply_output_handlers(embeddings, self.handlers)
# Check that API key is properly extracted
assert "openai_api_key" in serialized
assert serialized["openai_api_key"] == api_key
assert serialized["model"] == "text-embedding-3-small"
assert serialized["chunk_size"] == 500
# Check metadata for reconstruction
assert serialized["__class_name__"] == "OpenAIEmbeddings"
assert "langchain_openai" in serialized["__module_name__"]
# Test deserialization
input_handler = BaseModelInputHandler()
fields = {"param": (serialized, {})}
result = await input_handler.prepare(fields, None)
deserialized = result["param"]
assert isinstance(deserialized, OpenAIEmbeddings)
assert deserialized.model == "text-embedding-3-small"
assert deserialized.chunk_size == 500
# Check that SecretStr is properly reconstructed
assert hasattr(deserialized, "openai_api_key")
if isinstance(deserialized.openai_api_key, SecretStr):
assert deserialized.openai_api_key.get_secret_value() == api_key
else:
# In some versions, it might be a string after deserialization
assert deserialized.openai_api_key == api_key
except ImportError:
pytest.skip("langchain_openai not available for real OpenAI embeddings test")

File diff suppressed because it is too large Load Diff

4260
uv.lock generated

File diff suppressed because it is too large Load Diff