chore: upgrade langchain-classic to 1.0.7
- Update version constraint from ~=1.0.0 to ~=1.0.7
- Fixes issues present in version 1.0.4
- Update uv.lock with new dependency resolution
Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>
* Add UTM tracking to outbound watsonx Orchestrate links
Introduces a decorateWxoUrl utility that appends utm_source, utm_medium,
utm_campaign, and a per-link utm_content to IBM-hosted URLs using the URL
API. The utm_source is configurable via LANGFLOW_WXO_UTM_SOURCE with a
"langflow" default. Applied to the Sign-up and Find-your-credentials links
in step-provider and add-provider-modal.
* update tsts
* update tsts
Adds optional session_id support to flows exposed via the MCP
interface so MCP clients (Claude Desktop, Cursor, etc.) can persist
conversation history across tool calls.
- json_schema_from_flow now advertises session_id as an optional
string property on every generated tool schema, unless the flow
already defines its own session_id input.
- handle_call_tool reads session_id from the incoming arguments and
forwards it to simple_run_flow. When absent or blank, it falls
back to a generated UUID, preserving the prior per-call behavior.
Reported by Alejandro: MCP clients had no way to maintain chat
history because the server unconditionally generated a fresh
conversation id per call.
* feat: upgrade Docker images to Python 3.14 (experimental)
- Update pyproject.toml: requires-python from <3.14 to <3.15
- Update all 5 Dockerfiles to use Python 3.14:
- Builder stage: python3.12-trixie-slim → python3.14-trixie-slim
- Runtime stage: python:3.12-slim-trixie → python:3.14-slim-trixie
- src/backend/base/pyproject.toml
- docker/build_and_push_base.Dockerfile
- docker/build_and_push.Dockerfile
- docker/build_and_push_backend.Dockerfile
- docker/build_and_push_with_extras.Dockerfile
- docker/build_and_push_ep.Dockerfile
Python 3.14.5 was released on May 10, 2026. Upgrading to the latest
Python version should help reduce CVE vulnerabilities in Docker images.
This is an experimental change to test Python 3.14 compatibility:
- Docker images will use Python 3.14
- CI/CD workflows still test on Python 3.10-3.13
- Nightly build will validate if dependencies work with 3.14
- Can be reverted quickly if issues are found
- Monitor nightly build for failures
- If successful, update CI/CD workflows to add Python 3.14 to test matrix
- If failures occur, revert and investigate compatibility issues
* chore: support Python 3.14
Bump requires-python upper bound to <3.15 across langflow, langflow-base,
lfx, and langflow-sdk, and add 3.14 to CI test matrices so PRs are gated
on 3.14 compatibility.
Conditional pins for transitive deps without 3.14 wheels at the existing
caps:
- onnxruntime: >=1.26 on 3.14 (existing <1.24 cap retained for 3.10)
- faiss-cpu: >=1.13.2 on 3.14 (existing ==1.9.0.post1 retained for <3.14)
* chore: marker-gate IBM watsonx packages for Python 3.14
ibm-watsonx-ai 1.3.x cannot import on Python 3.14: its StrEnum subclasses
override __init__ in a way that conflicts with 3.14's reworked enum
__set_name__ path (TypeError on KnowledgeBaseFieldRole class creation).
Cap ibm-watsonx-ai, langchain-ibm, and the ibm-watsonx-orchestrate-*
extras at python_version<'3.14' until upstream adapts. Watsonx component
imports are already lazy, so this surfaces only at component-access time
on 3.14, where the existing ImportError handler in
lfx.components.ibm.__init__ degrades it gracefully.
test_model_utils.py imports ChatWatsonx at module top to verify
get_model_name resolves model_id; guard that import so the test
module skips on 3.14 instead of breaking collection.
* [autofix.ci] apply automated fixes
* chore: upgrade remaining Docker images to Python 3.14
build_and_push*.Dockerfile already moved to 3.14 in the merge from
feat/upgrade-python-3.14-docker-images. Apply the same bump to the
dev, devcontainer, and lfx Docker images so all in-repo Dockerfiles
share one Python version.
* [autofix.ci] apply automated fixes
* chore: gate 3.14-broken extras to python_version<'3.14'
cuga (and its transitive fastembed -> py-rust-stemmers source build),
altk/agent-lifecycle-toolkit (re-pulls ibm-watsonx-ai which was already
gated), langchain-pinecone, langwatch, ragstack-ai-knowledge-store, and
OpenDsStar all pin themselves below 3.14 upstream. Without a marker
guard, the workspace lock still tries to install them and a Docker
`uv sync` then attempts source builds that require Rust (CI Docker
test failure with py-rust-stemmers 0.1.5).
Add python_version<'3.14' (or <'3.13' for ragstack) to each pin so
those packages are simply omitted from the 3.14 install set until
upstream catches up. uv pip check is now clean on 3.14.2.
* test: skip test_altk_agent on Python 3.14
altk (agent-lifecycle-toolkit) is gated to python_version<'3.14'
upstream and now via the langflow-base [altk] extra marker. The test
imports altk at module top to exercise ALTKAgentComponent; guard the
import so collection skips on 3.14 instead of failing.
* test: skip remaining altk tests on Python 3.14
Three more test modules import lfx.base.agents.altk_* at module top,
which transitively imports altk. Add the same module-level skip guard
as test_altk_agent.py:
- test_altk_agent_logic.py
- test_altk_agent_tool_conversion.py
- test_conversation_context_ordering.py
* fix(calculator): replace removed ast.Num with ast.Constant for Python 3.14
ast.Num was deprecated in Python 3.8 in favor of ast.Constant and
removed entirely in 3.14. The calculator tool and its core walker
crashed on 3.14 with 'module ast has no attribute Num'.
Switch the isinstance checks to ast.Constant + isinstance(node.value,
(int, float)), and drop the now-dead ast.Num backwards-compat branch
in calculator_core.py (the ast.Constant branch already handles every
input it would have matched). Update the parser unit-test fixtures
to construct ast.Constant nodes.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* test: skip LangWatch HTTP instrumentation tests on Python 3.14
langwatch is gated to python_version<'3.14' upstream. The
TestLangWatchHttpInstrumentation class patches langwatch.setup and
langwatch.trace inside a fixture, which requires langwatch to be
importable; on 3.14 the test errors with ModuleNotFoundError.
Guard the class with a skipif on langwatch availability.
* test: allow IBM and altk components to be missing on Python 3.14
test_all_modules_importable enforces that every component in __all__
imports cleanly. On 3.14 the ibm-watsonx-ai, langchain-ibm, and altk
packages are gated upstream and intentionally not installed, so the
WatsonxAI, WatsonxEmbeddings, and ALTKAgent components fail to import
as designed.
- test_all_components_in_categories_importable: accept a known-gated
deny-list on 3.14 instead of failing.
- test_all_lfx_component_modules_directly_importable: extend the
existing 'missing optional dependency' allowlist with altk,
langchain_ibm, and ibm_watsonx_ai.
* fix(lfx): pass ensure_exists to user_cache_dir for Python 3.14
Python 3.14 tightened PurePath.__init__ to reject unknown keyword
arguments. The KeyedWorkerLockManager constructor was passing
ensure_exists=True to Path() instead of user_cache_dir(), which
silently worked on 3.10-3.13 but raises TypeError on 3.14 and also
meant the cache directory was never actually being created.
Move the kwarg to user_cache_dir() where it belongs and update the
two unit tests that asserted the buggy call shape.
* [autofix.ci] apply automated fixes
* test: accept either ZIP or JSON error path on Python 3.14
Python 3.14's zipfile.is_zipfile() now validates the central-directory
signature in addition to EOCD, so the garbage+EOCD payload no longer
passes the is_zipfile() dispatch in the /flows/upload/ route. The
endpoint still returns 400 with a descriptive detail (now from the JSON
branch); only the message wording differs, which the test was asserting
verbatim.
* fix(lfx): normalize cors_origins ['*'] back to '*' on Python 3.14
Pydantic-settings on Python 3.14 parses the env var '*' into ['*']
before the cors_origins field validator runs (the list[str] | str union
resolves differently than on 3.10-3.13). Collapse that back to the
bare-string wildcard so downstream consumers — including
warn_about_future_cors_changes and the test suite — see the same shape
on every supported Python version.
---------
Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit 1955f8fae5)
* feat: upgrade Docker images to Python 3.14 (experimental)
## Changes
- Update pyproject.toml: requires-python from <3.14 to <3.15
- Update all 5 Dockerfiles to use Python 3.14:
- Builder stage: python3.12-trixie-slim → python3.14-trixie-slim
- Runtime stage: python:3.12-slim-trixie → python:3.14-slim-trixie
## Files Updated
- src/backend/base/pyproject.toml
- docker/build_and_push_base.Dockerfile
- docker/build_and_push.Dockerfile
- docker/build_and_push_backend.Dockerfile
- docker/build_and_push_with_extras.Dockerfile
- docker/build_and_push_ep.Dockerfile
## Rationale
Python 3.14.5 was released on May 10, 2026. Upgrading to the latest
Python version should help reduce CVE vulnerabilities in Docker images.
## Testing Strategy
This is an experimental change to test Python 3.14 compatibility:
- Docker images will use Python 3.14
- CI/CD workflows still test on Python 3.10-3.13
- Nightly build will validate if dependencies work with 3.14
- Can be reverted quickly if issues are found
## Next Steps
- Monitor nightly build for failures
- If successful, update CI/CD workflows to add Python 3.14 to test matrix
- If failures occur, revert and investigate compatibility issues
* chore: support Python 3.14
Bump requires-python upper bound to <3.15 across langflow, langflow-base,
lfx, and langflow-sdk, and add 3.14 to CI test matrices so PRs are gated
on 3.14 compatibility.
Conditional pins for transitive deps without 3.14 wheels at the existing
caps:
- onnxruntime: >=1.26 on 3.14 (existing <1.24 cap retained for 3.10)
- faiss-cpu: >=1.13.2 on 3.14 (existing ==1.9.0.post1 retained for <3.14)
* chore: marker-gate IBM watsonx packages for Python 3.14
ibm-watsonx-ai 1.3.x cannot import on Python 3.14: its StrEnum subclasses
override __init__ in a way that conflicts with 3.14's reworked enum
__set_name__ path (TypeError on KnowledgeBaseFieldRole class creation).
Cap ibm-watsonx-ai, langchain-ibm, and the ibm-watsonx-orchestrate-*
extras at python_version<'3.14' until upstream adapts. Watsonx component
imports are already lazy, so this surfaces only at component-access time
on 3.14, where the existing ImportError handler in
lfx.components.ibm.__init__ degrades it gracefully.
test_model_utils.py imports ChatWatsonx at module top to verify
get_model_name resolves model_id; guard that import so the test
module skips on 3.14 instead of breaking collection.
* [autofix.ci] apply automated fixes
* chore: upgrade remaining Docker images to Python 3.14
build_and_push*.Dockerfile already moved to 3.14 in the merge from
feat/upgrade-python-3.14-docker-images. Apply the same bump to the
dev, devcontainer, and lfx Docker images so all in-repo Dockerfiles
share one Python version.
* [autofix.ci] apply automated fixes
* chore: gate 3.14-broken extras to python_version<'3.14'
cuga (and its transitive fastembed -> py-rust-stemmers source build),
altk/agent-lifecycle-toolkit (re-pulls ibm-watsonx-ai which was already
gated), langchain-pinecone, langwatch, ragstack-ai-knowledge-store, and
OpenDsStar all pin themselves below 3.14 upstream. Without a marker
guard, the workspace lock still tries to install them and a Docker
`uv sync` then attempts source builds that require Rust (CI Docker
test failure with py-rust-stemmers 0.1.5).
Add python_version<'3.14' (or <'3.13' for ragstack) to each pin so
those packages are simply omitted from the 3.14 install set until
upstream catches up. uv pip check is now clean on 3.14.2.
* test: skip test_altk_agent on Python 3.14
altk (agent-lifecycle-toolkit) is gated to python_version<'3.14'
upstream and now via the langflow-base [altk] extra marker. The test
imports altk at module top to exercise ALTKAgentComponent; guard the
import so collection skips on 3.14 instead of failing.
* test: skip remaining altk tests on Python 3.14
Three more test modules import lfx.base.agents.altk_* at module top,
which transitively imports altk. Add the same module-level skip guard
as test_altk_agent.py:
- test_altk_agent_logic.py
- test_altk_agent_tool_conversion.py
- test_conversation_context_ordering.py
* fix(calculator): replace removed ast.Num with ast.Constant for Python 3.14
ast.Num was deprecated in Python 3.8 in favor of ast.Constant and
removed entirely in 3.14. The calculator tool and its core walker
crashed on 3.14 with 'module ast has no attribute Num'.
Switch the isinstance checks to ast.Constant + isinstance(node.value,
(int, float)), and drop the now-dead ast.Num backwards-compat branch
in calculator_core.py (the ast.Constant branch already handles every
input it would have matched). Update the parser unit-test fixtures
to construct ast.Constant nodes.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* test: skip LangWatch HTTP instrumentation tests on Python 3.14
langwatch is gated to python_version<'3.14' upstream. The
TestLangWatchHttpInstrumentation class patches langwatch.setup and
langwatch.trace inside a fixture, which requires langwatch to be
importable; on 3.14 the test errors with ModuleNotFoundError.
Guard the class with a skipif on langwatch availability.
* test: allow IBM and altk components to be missing on Python 3.14
test_all_modules_importable enforces that every component in __all__
imports cleanly. On 3.14 the ibm-watsonx-ai, langchain-ibm, and altk
packages are gated upstream and intentionally not installed, so the
WatsonxAI, WatsonxEmbeddings, and ALTKAgent components fail to import
as designed.
- test_all_components_in_categories_importable: accept a known-gated
deny-list on 3.14 instead of failing.
- test_all_lfx_component_modules_directly_importable: extend the
existing 'missing optional dependency' allowlist with altk,
langchain_ibm, and ibm_watsonx_ai.
* fix(lfx): pass ensure_exists to user_cache_dir for Python 3.14
Python 3.14 tightened PurePath.__init__ to reject unknown keyword
arguments. The KeyedWorkerLockManager constructor was passing
ensure_exists=True to Path() instead of user_cache_dir(), which
silently worked on 3.10-3.13 but raises TypeError on 3.14 and also
meant the cache directory was never actually being created.
Move the kwarg to user_cache_dir() where it belongs and update the
two unit tests that asserted the buggy call shape.
* [autofix.ci] apply automated fixes
* test: accept either ZIP or JSON error path on Python 3.14
Python 3.14's zipfile.is_zipfile() now validates the central-directory
signature in addition to EOCD, so the garbage+EOCD payload no longer
passes the is_zipfile() dispatch in the /flows/upload/ route. The
endpoint still returns 400 with a descriptive detail (now from the JSON
branch); only the message wording differs, which the test was asserting
verbatim.
* fix(lfx): normalize cors_origins ['*'] back to '*' on Python 3.14
Pydantic-settings on Python 3.14 parses the env var '*' into ['*']
before the cors_origins field validator runs (the list[str] | str union
resolves differently than on 3.10-3.13). Collapse that back to the
bare-string wildcard so downstream consumers — including
warn_about_future_cors_changes and the test suite — see the same shape
on every supported Python version.
---------
Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix(knowledge-base): align Filter by metadata pickers with app dropdowns
The "+ Filter by metadata" popover used plain <Input> fields backed by a
native HTML <datalist> for key/value suggestions. The browser-native
dropdown looked nothing like the shadcn `Select` used right next to it
(e.g. "All sources") — QA flagged the inconsistency.
Replace each free-text input with a shadcn-style combobox built on
Popover + Command (cmdk):
- Outline button trigger with chevron, matching SelectTrigger.
- Suggestion list rendered as CommandItem rows with a check on the
current selection.
- When the typed query is not in the suggestions, a "Use \"foo\"" item
appears so users can still commit a custom key/value.
- Closing the picker clears the search query; selecting an item
commits and closes.
Validation, refetch-on-open, error messaging, and submit flow are
unchanged. Tests rewritten against the new combobox testids and a
jsdom shim for Element.prototype.scrollIntoView (cmdk calls it).
* refactor(knowledge-base): split ChunksMetadataFilter into focused modules
ChunksMetadataFilter was holding data fetching, derived option lists,
validation rules, popover orchestration, and the combobox UI all in one
file. Split along single-responsibility lines so each piece changes for
its own reason:
- MetadataCombobox.tsx — presentational Popover+Command combobox with
custom-entry fallback. Generic over options/testId; reusable.
- hooks/useChunksMetadataFilter.ts — wraps useGetKbMetadataKeys and
derives availableKeys / valueSuggestions / flags / refetch.
- metadataFilterValidation.ts — pure KEY_PATTERN + validateMetadataFilter
returning { ok, key, value } | { ok: false, error }. Testable in
isolation; no DOM.
- ChunksMetadataFilter.tsx — orchestration only: outer popover, form
state, submit. Composes the three above.
Adds focused unit tests for the validator (regex + happy path + empty
+ uppercase). Existing integration test for the filter itself is left
untouched.
* fix(knowledge-base): gate Next Step on metadata-pair validity
Custom Fields validation only fired on blur and lived in MetadataEditor's
local state, so an invalid key like "Year" still let the user advance to
Review & Build. StepReview rendered the raw pairs (showing "Year: 2020"
in the summary) but ``metadataPairsToFormValue`` silently filtered the
invalid row out at submit — the backend never received that metadata.
The UI was effectively lying.
Single source of truth in metadataValidation.ts:
- validateMetadataPair (per-row: empty, regex, value length)
- validateMetadataPairs (cross-row: duplicates, max count)
- filterValidMetadataPairs (clean trimmed set for display/submit)
Wired through:
- MetadataEditor derives errors from the prop list via the shared
validator, dropping the blur-only local state. Errors always
reflect the current input.
- useKnowledgeBaseForm.getValidationErrors now checks run-level and
per-file metadata. handleNext blocks while any row is invalid and
surfaces a single error key under validationErrors.metadata.
- StepConfiguration renders that error next to the Custom Fields
block.
- StepReview filters the summary through filterValidMetadataPairs so
it only shows pairs the backend will actually persist.
Adds focused unit tests for the new validator (per-row rules, dupes,
max-count, filter behavior).
* feat: implement Redis job queue and fakeredis support
- Refactored the job queue service to support Redis-backed management for cross-worker scaling.
- Added environment variables for configuration:
- `LANGFLOW_JOB_QUEUE_TYPE=redis`
- `LANGFLOW_REDIS_QUEUE_DB=1`
- Updated job ownership methods to be asynchronous for improved concurrency handling.
- Enhanced Redis cache service with namespacing via key prefixes.
- Introduced `fakeredis` for in-memory Redis simulation in testin>
- Added comprehensive unit tests for Redis job queue components.
* fix: run experimental warning for RedisCache usage only once
- Introduced a mechanism to emit a one-time warning for the RedisCache experimental feature during server runtime.
- The warning is logged only if no other worker has already emitted it, ensuring clarity for users regarding the experimental status of RedisCache.
- The implementation includes a temporary file check to prevent multiple warnings across different processes.
* docs: document environment variables for worker management
- Added documentation for LANGFLOW_GUNICORN_PRELOAD to explain preloading for better performance.
- Detailed the use of LANGFLOW_JOB_QUEUE_TYPE for specifying backends (e.g., Redis).
- Included LANGFLOW_REDIS_QUEUE_DB to define the database index for job queues.
- Updated the "High-Load Environments" guide with these optimal configurations.
* docs: updated 'High-load and multi-worker environments' section
* feat: enhance RedisJobQueueService with consumer wrapper management
- Introduced a caching mechanism for Redis stream consumers to optimize job data retrieval.
- Added methods to manage consumer wrappers, ensuring they are reused across sequential polls.
- Implemented cleanup logic to cancel and clear consumer wrappers during job cleanup and service stop.
- Expanded unit tests to verify consumer wrapper reuse and cleanup behavior.
* fix: ensure Redis keys are deleted during job cleanup even on cancellation
- Updated the cleanup_job method in RedisJobQueueService to guarantee Redis keys are removed even if the job cleanup is interrupted by a CancelledError.
- Added a new unit test to verify that Redis keys are deleted correctly when cleanup is called during task cancellation.
* fix: manage connection check task in RedisJobQueueService
- Added handling for the connection check task in the stop method to ensure it is properly cancelled and awaited if still running.
- This change improves resource management and prevents potential issues during service shutdown.
* fix: handle unpublished sentinel requeue on cancellation in RedisJobQueueService
- Updated the job processing logic to ensure that if a job is cancelled during the xadd operation, the unpublished sentinel is requeued instead of being dropped.
- Introduced a new unit test to verify this behavior, ensuring robustness in job handling during cancellations.
* fix: improve Redis job queue service with enhanced configuration and cleanup
- Added atexit cleanup to remove stale temporary files for RedisCache.
- Refactored Redis job queue service to use shared constants for stream prefixes, improving maintainability.
- Updated type hints for better clarity and consistency in RedisQueueWrapper and RedisJobQueueService.
- Enhanced error handling with configurable backoff for transient read failures.
* fix: enhance Redis job queue service with maxlen configuration for xadd
- Updated the xadd method in RedisJobQueueService to include maxlen and approximate parameters, improving stream management and preventing excessive memory usage.
* fix: enhance job ownership retrieval in RedisJobQueueService
- Updated the get_job_owner method to refresh the Redis key TTL on successful lookups, ensuring long-running jobs maintain their ownership anchor.
- Improved code clarity by extracting the owner key into a variable and adding detailed docstring explanations for better understanding of the TTL management.
* fix: improve job ownership handling in cleanup_job method
- Enhanced the cleanup_job method in RedisJobQueueService to accurately capture job ownership before deleting Redis keys, preventing potential data corruption in multi-worker scenarios.
- Added comments for clarity on ownership logic and its implications during job cleanup.
* fix: optimize TTL management in RedisJobQueueService
- Introduced periodic TTL refresh logic in the _bridge_to_redis method to enhance Redis stream management, reducing round-trips and improving throughput.
- Added constants for TTL refresh events and seconds to maintain clarity and configurability.
- Updated event handling to ensure TTL is refreshed appropriately based on event count and time elapsed.
* fix: improve event handling in flow response management
- Removed unnecessary error handling for missing event tasks in get_flow_events_response, allowing for smoother operation when no task exists.
- Updated create_flow_response to handle optional event_task parameter, ensuring proper cleanup during disconnections.
- Added unit tests to verify behavior when event tasks are missing, enhancing robustness in streaming scenarios.
* fix: enhance RedisQueueWrapper with startup grace period and stream observation
- Added a startup grace period to prevent premature end-of-stream signals when the producer has not yet issued its first XADD.
- Introduced a flag to track whether the stream has been observed, improving the handling of early polling scenarios.
- Updated logic to ensure proper handling of stream existence checks and logging for better debugging during job processing.
* fix: implement cleanup for old cross-worker job queues in RedisJobQueueService
- Added a new method to clean up done cross-worker consumer wrappers that are not owned by the current worker, ensuring proper resource management.
- Enhanced the existing cleanup logic to prevent memory leaks by explicitly pruning stale entries from the consumer wrappers dictionary.
- Improved logging to provide better visibility into the cleanup process for cross-worker jobs.
* fix: enhance job handling in JobQueueService with guarded task execution
- Updated the start_job method to accept a Coroutine type for task_coro, ensuring type safety.
- Introduced a new _guarded_task method to wrap job coroutines, guaranteeing that unhandled exceptions emit an error event and write a sentinel to the Redis Stream, improving reliability in job processing.
- Enhanced documentation to clarify the behavior of the new task handling mechanism and its implications for cross-worker consumers.
* fix: improve client disconnection handling in create_flow_response
- Added logging for scenarios where a client disconnects without an associated event_task, clarifying that the producer will continue running until the build completes.
- Documented the limitation regarding cross-worker passive disconnects and the need for a Redis side-channel for proper cancellation, enhancing observability in the logs.
* fix: enhance RedisQueueWrapper to manage initial read state and buffer behavior
- Introduced a flag to track the completion of the first XREAD call, ensuring proper buffer management during the initial read phase.
- Updated the empty method to reflect the state of the buffer accurately, preventing premature exits from the drain loop until the first read is complete.
- Improved documentation to clarify the behavior of the new flag and its impact on job processing.
* fix: clarify RedisQueueWrapper behavior in tests for buffer state and no-op operations
- Updated test documentation to explain the behavior of the empty() method in relation to the first XREAD completion, ensuring accurate understanding of buffer state during job processing.
- Adjusted assertions in tests to reflect the intended behavior of the RedisQueueWrapper, specifically regarding the no-op nature of put_nowait and its impact on the internal buffer.
* fix: update dependencies in pyproject.toml and uv.lock
- Added fakeredis dependency with a minimum version of 2.0.0 to both pyproject.toml and uv.lock files.
- Ensured proper formatting and comments in pyproject.toml for clarity on onnxruntime version constraints.
* fix: enhance RedisQueueWrapper and job queue handling for improved cancellation and buffer management
- Introduced a structural protocol `_CancellableQueue` to ensure queues can handle cancellation properly during client disconnects.
- Updated `RedisQueueWrapper` to implement this protocol, allowing for graceful cancellation of background tasks.
- Added a maximum size limit to the internal buffer to prevent unbounded memory usage and ensure backpressure on slow consumers.
- Implemented a done callback to handle unexpected fill task crashes, ensuring consumers are not left hanging indefinitely.
- Enhanced unit tests to verify compliance with the new protocol and the behavior of the buffer under various conditions.
* fix: RedisQueueWrapper sentinel delivery, error time-bound, and cancel response
- Deliver end-of-stream sentinel on fill-task cancellation (_on_fill_done now
handles both cancelled and exception paths so consumers are never left hanging)
- Add _error_start time-bound to xread and exists() error loops: after
_STARTUP_GRACE_S seconds of continuous Redis errors the sentinel is delivered
instead of retrying forever
- Advance _last_id cursor only after buffer.put() succeeds so cancellation mid-put
does not silently skip that message in the Redis cursor
- Return False from cancel_flow_build when event_task is None (cross-worker path)
so the HTTP response correctly reports success=False instead of false success
* [autofix.ci] apply automated fixes
---------
Co-authored-by: ogabrielluiz <gabriel@langflow.org>
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Memory base and Model Providers both rendered the plain "Brain" Lucide
icon, which QA flagged as confusing — the two sections felt
interchangeable at a glance.
Repoint each surface to its own variant per product/manager direction:
Model Providers / model trigger / settings nav: Brain → BrainCircuit
Memory base (sidebar nav, modal, details header, empty states):
Brain → BrainCog
BrainCircuit was already in use for Language Models / LLM Operations in
the component library, so the Model Providers surface now lines up with
that family. Memory keeps its own distinct glyph (cog) to read as
"managed knowledge" rather than "a model".
Test fixture in sidebarSegmentedNav.test.tsx updated. Provider-list
fixtures that happened to default to "Brain" for unrelated test data are
left as-is.
* Unserializable Object string support
* FlowCreate causing unhandled error fix
* [autofix.ci] apply automated fixes
* added typeerror to thecatch and improved testcases
* fix: added testcase
* fix: widen TraceRead I/O types and validate flows list items before import
TraceRead.input/output were typed as dict|None, causing a 500 when a
stored span contained the '[Unserializable Object]' sentinel string.
Widen to dict|str|None to match TraceSummaryRead.
The flows upload endpoint assumed data["flows"] was a list of dicts;
payloads like {"flows": "oops"} or {"flows": [1]} reached
normalize_code_for_import and raised AttributeError before the 422
handler ran. Add explicit isinstance checks for both the list container
and each item, returning 422 with a clear message on failure.
Regression tests added for both fixes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
* docs: improve sidebar and TOC readability + scroll progress
- Fix left nav sidebar hugging the left edge (remove margin-left)
- Increase sidebar font sizes and lift muted colors for legibility
- TOC right-side: white inactive items, pink active, gray for passed sections
- Add clientModule (tocProgress.js) to track scroll progress in TOC
- Add Redocusaurus API page styles to match site theme
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: fix sidebar/TOC height stability and active item reflow
- Force sidebarViewport to 100vh to prevent shrinking during scroll
- Replace font-weight on active TOC item with text-shadow to avoid reflow
* docs: left-anchor sidebar and increase fonts on large screens
* docs: constrain prev/next nav to match content max-width
* docs: fix sidebar jump on scroll by moving padding-top inside menu
* docs: fix sidebar-ad spacing with transparent padding on li
* docs: fix sidebarViewport height and MutationObserver leak
* docs: constrain doc content to 75vw and remove dead CSS comments
- Cap docMainContainer at 75vw
- Set pagination-nav to 100% width to match content column
- Remove commented-out .markdown and .ch-scrollycoding blocks
* docs: set docMainContainer max-width to 75% for better reading width
* docs: center main-wrapper on large screens and center docItemWrapper
* fix: revert lodash override to stable 4.17.21
lodash 4.18.0 is a broken release — template.js uses assignWith and
arrayEach without importing them, crashing npm start. Pin to 4.17.21
across all nested overrides.
* fix: prevent horizontal scroll on mobile docs pages
docsWrapper has flex:1 0 auto (flex-shrink:0), so it never shrinks below
the natural content width (~570px on iPhone). Cap it to 100vw and add
min-width:0 down the flex chain (docRoot, docMainContainer) so the
layout stays within viewport on narrow screens.
Also remove the overflow:visible override from ch-code-wrapper, which
was defeating the code block's own horizontal containment.
* feat: responsive navbar search and hide social icons on mobile
Shrink search bar as viewport narrows (160px at 1100px, 100px at 996px),
collapse to icon-only at 960px. Hide GitHub, X, and Discord icons below
996px where the hamburger menu takes over navigation.
* fix: images always respect container width above 996px
Use min(100%, 600px) so images are capped at 600px but never overflow
their container when the content area is narrower (e.g. sidebar open).
* fix: improve TOC progress scroll tracking and bottom-of-page detection
Adds scroll handler with RAF throttle, pauses/resumes MutationObserver
to avoid loops, and forces last TOC link active when scrolled to page bottom.
* fix: translate Portuguese comments to English in custom.css
* fix: sidebar viewport fills full screen height
* feat: sidebar section icon turns pink with ease transition when active
* fix: reduce TOC font size to 0.8rem
* fix: remove docMainContainer right padding and add 48px gap to content col
* fix: card header responsive alignment and title margin reset
* fix: darken card background and active sidebar item bg on light theme
* fix: revert TOC colors and add inline code border radius tweak
* fix: minimal scrollbar for sidebar and TOC — thumb only on hover
* fix: scrollbar fade-in/out transition at 0.25s ease
* feat: add rehype plugin for table column wrapping and centering
Inserts zero-width space after underscores in inline code cells so long
env-var names wrap at underscore boundaries. Also centers Format/Default
columns via a col-center class applied at build time.
* fix: use --ifm-heading-color for sidebar group labels in both themes
* refactor: split docs custom.css into themed partials
Breaks the 1,658-line monolithic custom.css into 8 focused files
(tokens, layout, navbar, sidebar, typography, components, code,
redocusaurus). custom.css is now a thin entry point with @imports.
* fix: align sidebar and navbar with shared --docs-gutter token
Introduces --docs-gutter: 1rem in tokens.css and applies it to both
navbar__inner horizontal padding and sidebar nav.menu padding-left,
keeping the Langflow logo and sidebar items visually aligned.
* fix: match sidebar ad background to page background color
Dark mode uses #111 (same as sidebar bg); light mode uses --ifm-background-color.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: restore langfuse feedback trace linking
* [autofix.ci] apply automated fixes
* fix: add run id to placeholder graph
* fix: gate langfuse feedback sync and surface failures
Gate the background task on tracing being enabled and Langfuse being
configured so we don't enqueue work that silently no-ops. Raise instead
of swallowing missing-credentials and import errors so background-task
failures hit logs. Add delete_feedback_score for the clear-feedback
(True/False -> None) transition that previously fell through.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* MB enhancement: Added agent pre-processing features to ingestion, can act as an LLM judge to gate ingestion based on system prompt.
* Update mb01b2c3d4e5_add_preprocessing_output.py
---------
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* Add Memory Base API: models, migrations, service, endpoints, and tests
Introduces Memory Base (MB) — a per-flow knowledge base that auto-captures
conversation history and ingests it into a Chroma vector store on configurable
thresholds.
Backend changes:
- MemoryBase + MemoryBaseSession DB models with full CRUD
- Three Alembic migrations: base tables, merge head, phase-2 fields
(embedding_model, preprocessing, preproc_model, preproc_instructions)
- MemoryBaseService: create/list/get/update/delete, session tracking,
pending-message cursor logic, mismatch detection, regenerate
- ingest_memory_task: async Chroma ingestion with cursor advance on success
- REST API (/api/v1/memories): CRUD, flush, sessions, mismatch, regenerate
- Flow output hook: on_flow_output() triggers auto-capture after each run
- Plumbing in build.py, endpoints.py, workflow.py to call on_flow_output
- deps.py: expose get_memory_base_service()
- kb_helpers.py: FS/metadata helpers used by MB service
Authored-By: Debojit Kaushik <kaushik.debojit@gmail.com>
Add dedupe_key idempotency enforcement for MB ingestion jobs
Centralizes idempotency into JobService.create_job() with a null-safe check,
removing the redundant pre-flight logic from MemoryBaseService.
Key changes:
- services/jobs/exceptions.py: new DuplicateJobError(RuntimeError) —
raised when a QUEUED/IN_PROGRESS/COMPLETED job with the same dedupe_key
exists; FAILED/CANCELLED are retryable and are excluded
- services/jobs/__init__.py: exports DuplicateJobError
- services/jobs/service.py: null-guarded dedup query inside create_job()
within the same session_scope as the insert (minimizes TOCTOU window)
- services/database/models/jobs/model.py: dedupe_key field -> index=True
- alembic/versions/36aa87831162: adds dedupe_key column + ix_job_dedupe_key
index to job table with checkfirst guards
- services/memory_base/service.py: updated key format to
"ingestion:{mb_id}:{session_id}:{first_msg_id}" for namespace isolation;
removed _has_non_retryable_job_for_dedupe_key and _has_active_job methods
and all call sites; DuplicateJobError catch in _maybe_trigger() for
silent skip on auto-capture; split regenerate() catch clauses
- api/v1/memories.py: explicit DuplicateJobError catch before RuntimeError
in flush_memory_base() for semantic clarity (both return 409)
Co-Authored-By: Debojit Kaushik <kaushik.debojit@gmail.com>
Checkpointing working version of MBs. TODO: User separation, Get messages endpoint, MB resumption midway through a chat for a session, tests.
Added messages endpoint for Memory Bases. Added pagination to sessions endpoint. Modifed messages model to include ingestion related attributes.
Added unit tests, fixed linting issues and formatting issues.
Aligned Workflows API, /run endpoint, playground to all work with Memory Bases. Created DB models asociated with tracking memory base state with sessions and jobs. Added tests, created unit tests for service and task files related to MemoryBases.
Improved concurrency of jobs, added (memory_base_id, session_id) locking to serialize jobs in case the job creation cadence moves ahead of ingestion jobs. Improved concurrency handling and moved the pending check to be live inside the ingestion job rather than a snapshot before triggering the job.
Consolidated all migrations related to memory_bases into one idempotent version and serialized all migrations form release along with memory_bases for cleanliness and maintainability.
Introduced advisory locking to address multi worker environment, added unique constraint to MB creation, added sanitization check to KB pathnames to avoid illegal directory creation.
Added partial write rollback for ChromaDB, aligned same session is used for each job to avoid dangling advisory locks.
* feat(kb): Knowledge Bases Infrastructure Overhaul
* refactor(kb): reuse Memory Base helpers to reduce duplication (#12878)
* refactor(kb): reuse Memory Base helpers to reduce duplication
Stacked on top of #12802 — consolidates the overlap between this KB
infra PR and the Memory Base work in #12417 so the two PRs land
cleanly together.
1. Chain migrations linearly
``72df732be86b_add_ingestion_run_table`` now points at
``mb00a1b2c3d4`` (the Memory Base head) instead of
``d306e5c17c41``. Both PRs had branched from the same release
ancestor, which would have required an Alembic merge migration at
integration time; with the stack in place, a single linear history
lands: ``d306e5c17c41 ← mb00a1b2c3d4 ← 72df732be86b ←
15fe9304bca7 ← e728126476a8``.
2. Delegate the path-traversal guard to the shared helper
``_validate_kb_path_containment`` now wraps
``kb_path_helpers.validate_kb_path`` instead of duplicating its
``is_relative_to`` check — the HTTPException translation and
structured log line stay here because they're route-specific, but
the traversal rule itself is defined once.
3. Reuse ``infer_embedding_provider`` during KB backfill
``knowledge_base_service.backfill_from_disk`` now falls back to
the pure-string inference helper when legacy metadata has
``embedding_model`` set but ``embedding_provider`` is
``"Unknown"``. Matches the inference the Memory Base service uses
and avoids shipping backfilled rows with
``embedding_provider="Unknown"`` when the model name is
recognizable.
4. Use ``dedupe_key`` for connector ingestions
``POST /knowledge_bases/{kb_name}/ingest/connector`` now builds a
stable SHA-256 key over (user, kb, source_type,
canonicalized-source-config) and passes it to
``JobService.create_job``. A double-click on "Ingest" or a retry
racing with an in-flight job now returns HTTP 409 instead of
spawning a duplicate run. Only the hash lands on the job row, so
credentials in ``source_config`` don't leak through ``dedupe_key``.
5. Extract ``KBIngestionHelper.write_documents_to_backend``
Backend-agnostic counterpart to
``write_documents_to_chroma`` (which is Chroma-only and used by
Memory Base). The inline retry/cancellation loop inside
``perform_ingestion`` collapses into a single helper call so
Mongo/Astra/Postgres/OpenSearch paths share identical batching,
cancellation, and exponential-backoff behaviour with the
file-upload/folder paths.
* Update templates
* Update .secrets.baseline
* Update component_index.json
* Update starter projects
* Update .secrets.baseline
* refactor(kb): migrate ingestion-source registry to AdapterRegistry (#12879)
Port the hand-rolled ``ingestion_sources/registry.py`` onto the
existing ``lfx.services.adapters.registry.AdapterRegistry`` so
third parties can publish ingestion sources as
pip-installable plugins without modifying core.
Plugin authors can now register sources two new ways in addition to
the in-tree ``register_source(SourceType.X, Y)`` call:
* Entry point group ``lfx.ingestion_source.adapters``
* ``[ingestion_source.adapters]`` in ``lfx.toml`` (or
``[tool.lfx.ingestion_source.adapters]`` in ``pyproject.toml``)
What stays the same:
* ``SourceType`` enum remains the source of truth for built-in
identifiers; API boundaries (``ConnectorIngestRequest``) still
validate against it.
* Every existing caller (``register_source`` / ``get_source_class``
/ ``create_source`` / ``registered_sources``) keeps its contract
— the six built-ins continue to self-register at import time via
``ingestion_sources/__init__``.
* ``create_source`` still returns a fresh instance per call. Sources
carry request-scoped ``user_id``/``source_config`` so
``AdapterRegistry.get_instance`` (which caches singletons) is
bypassed in favour of ``get_class``.
* Collision semantics preserved: re-registering a different class
under an existing key raises ``ValueError`` at import time.
* Error-message split preserved: unknown strings (call-site typo) →
``"Unknown ingestion source"``; known-but-unregistered →
``"not registered"``. API routes key HTTP 400 vs 500 off this.
New public helper: ``registered_source_keys()`` returns the full
set of registered keys (including third-party plugins) for callers
that want to surface plugins in a picker; ``registered_sources()``
still returns only the built-in ``SourceType`` enum members for
backward compatibility.
Tests:
* Existing 49 ingestion-source tests pass unmodified (behavior
parity with the old registry).
* 4 new tests cover the plugin surface: entry-point registration,
TOML registration, built-ins not shadowed by entry points, and
the Unknown-vs-not-registered error-message split.
* fix(kb): unstick remote-backend deletes + hydrate stale embedding metadata (#12880)
* Update component_index.json
* Template update
* fix(kb): delete opensearch metadata matches (#12873)
* scope: Chroma and OpenSearch backends
* Ruff cleanup
* format
* [autofix.ci] apply automated fixes
* test(kb): cover backend registry ingestion routing (#12870)
* [codex] Add Knowledge Backends settings (#12909)
feat: add knowledge backend settings
* feat(kb): New KB backend settings
* [autofix.ci] apply automated fixes
* Update component_index.json
* chore(logs): Improve the logging display in KB
* feat(kb): Full support for OpenSearch backend
* Update index.tsx
* test(kb): tighten ingestion run auth assertion (#12872)
* test(kb): tighten ingestion run auth assertion
* [autofix.ci] apply automated fixes
* fix(kb): surface job_id in run detail modal and report skipped runs accurately
Addresses two QA findings on PR #12872:
- BUG-1: the Ingestion Run Detail modal hid the run's job_id even
though the API returned it. Add a Run ID + Job ID block to the
modal so operators can correlate runs with the underlying job
service entry.
- BUG-2: a run that produced 0 successes but >=1 skipped item (e.g.
empty file / no extractable text) was finalized as SUCCEEDED and
surfaced via the cumulative-KB-chunks toast as "ingestion complete
- N chunks ready". Treat any run with skipped or failed items
(without a clean full success) as PARTIAL, and replace the toast
with a per-run breakdown (succeeded/skipped/failed + chunks
ingested by THIS run) shown as a notice instead of success.
Also adds a regression test for the skipped-only -> PARTIAL path in
KBIngestionHelper.perform_ingestion, and tightens types in the
existing KnowledgeBasesTab test (removes pre-existing any usage so
the staged biome no-any hook stays green).
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* test(kb): cover folder ingest allow-list wiring (#12869)
* test(kb): cover folder ingest allow-list wiring
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix(kb): allow clearing persisted separator (#12871)
* fix(kb): allow clearing persisted separator
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* Update component_index.json
* Update test_cometapi_component.py
* Revert "Update test_cometapi_component.py"
This reverts commit e789ef6bed.
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* feat: Add test connection option for backends
* Biome check
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* chore: rebake starter projects after release-1.10.0 merge
The Validate Starter Project Templates hook regenerated the embedded
component source in Knowledge Retrieval.json and Vector Store RAG.json
to match the merged retrieval.py (now imports path/metadata helpers
from _kb_paths).
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* feat(jobs): add job_metadata mirror for KB ingestion runs
Step 1 of unifying KB ingestion tracking onto the canonical 'job'
table per @dkaushik94's review feedback on PR 12935.
Adds a nullable 'job_metadata' JSON column on the 'job' table and
mirrors the 'ingestion_run' lifecycle data onto it: create_run,
mark_running and finalize_run all dual-write the same counters,
items list, status, and source config. The 'ingestion_run' table
remains the canonical source of truth for read paths in this PR;
that's deferred to the contract phase once the API/UI migrate.
Component-path ingestions (no parent Job row) skip the mirror
without raising. JobService gains an 'update_job_metadata' helper
for shallow-merge / replace semantics.
Tests: 9 new tests cover create/mark/finalize dual-write,
component-path no-op, ghost-job tolerance, and the JobService
helper's merge/replace/missing-row branches. Existing 10
ingestion-run endpoint tests still pass.
Migration: da7f6b9b638a (EXPAND, nullable column, fully reversible)
* chore: rebake starter projects after retrieval.py TYPE_CHECKING change
* feat(jobs): unify KB ingestion tracking onto job table; drop ingestion_run
Switches both reads and writes off the legacy 'ingestion_run' table
and onto 'Job.job_metadata'. The full sequence in this PR:
da7f6b9b638a (prev commit) — add Job.job_metadata column (EXPAND)
927dd37c3ea0 — backfill existing ingestion_run rows (MIGRATE)
fc7393328808 — drop the ingestion_run table (CONTRACT, reversible)
Reads/writes now go through 'Job.job_metadata'; the response shape
(IngestionRunInfo / IngestionRunDetail) is unchanged so no frontend
changes are needed. 76 tests pass.
* chore(jobs): collapse ingestion_run lifecycle migrations
The ingestion_run table never shipped to any tagged release —
verified by checking every tag from v1.9.x through v1.10.0.dev9. The
table only existed on feature branches (this one and
feat/kb-ingest-metadata). So the 'create table → backfill → drop
table' migration dance was theatre for a table no production user
ever had.
Collapses 5 migrations to 1 by deleting:
72df732be86b — add ingestion_run table
e728126476a8 — add kb_id FK to ingestion_run
927dd37c3ea0 — backfill ingestion_run → job.job_metadata
fc7393328808 — drop ingestion_run table
Repoints the chain so 15fe9304bca7 (knowledge_base) attaches directly
to mb00a1b2c3d4, and da7f6b9b638a (Job.job_metadata) attaches after
15fe9304bca7. Net schema change for production users: a single
nullable JSON column added to the job table.
Devs running this feature branch locally and carrying an
ingestion_run table in their dev DB will need to drop it manually or
nuke + re-migrate. There are no production users of this branch by
construction.
Tests: all 22 KB-related tests still pass.
* refactor(kb): drop unused VectorStoreBackend Protocol
The Protocol was exported as part of the public surface but nothing
in the codebase actually used it as a type annotation, isinstance
narrowing target, or registry constraint:
- Every concrete backend (Chroma/OpenSearch/Mongo/Astra/Postgres)
inherits from the BaseVectorStoreBackend ABC
- The registry signature requires type[BaseVectorStoreBackend]
- No call site used '\: VectorStoreBackend' or
'isinstance(x, VectorStoreBackend)'
- The only test reference (a runtime-check assertion) duplicated
the ABC isinstance check on the line above
Per @dkaushik94's review feedback: the Protocol was dead weight.
Folded its contract-level guidance ('lightweight to construct, heavy
resources released in teardown') into the ABC docstring so the
documentation isn't lost, then deleted the Protocol class and
trimmed exports.
Test surface: replaced the Protocol runtime-check assertion in
test_backends_registry.py with a BaseVectorStoreBackend isinstance
check (same intent, no Protocol needed). 9 registry tests pass.
* refactor(kb): standardize Job.asset_id on KnowledgeBaseRecord.id
Per @dkaushik94's review on list_runs_for_kb: switches the read-side
filter from a JSON-extract on Job.job_metadata.kb_name to an indexed
btree lookup on Job.asset_id. Required standardizing what asset_id
means for KB ingestion jobs.
Before — Job.asset_id was set to metadata['id'] (a UUID stored in
embedding_metadata.json), independent of the knowledge_base table's
primary key. Two id-spaces in flight, with legacy generation logic
inlined at three create_job sites.
After — Job.asset_id is set to KnowledgeBaseRecord.id everywhere.
When the KB has a DB record (the common case post-startup-backfill),
the row's primary key flows directly onto Job.asset_id. Legacy KBs
on disk only fall back to metadata['id'] (or a generated UUID) so
backwards-compat survives.
* New _resolve_kb_asset_id helper centralizes the resolution logic.
Three ingest endpoints (file upload / folder / connector) and the
cancel endpoint now go through it instead of inlining metadata['id']
reads with their own fallback dance.
* list_runs_for_kb resolves the kb_record up front and filters by
Job.user_id + asset_type='knowledge_base' + asset_id=kb_record.id
when present. Falls back to JSON-extract on kb_name only for KBs
without a DB record.
* Tests: _insert_run helper auto-provisions a KnowledgeBaseRecord so
the indexed path is exercised by default. New unit test pins the
indexed-path behavior: two Jobs with the same kb_name in metadata
but different asset_ids must not cross-contaminate the result set.
23 KB tests pass (13 service + 10 endpoint).
* refactor(kb): drop embedding_provider/model columns; model_selection is the single source
Per @dkaushik94's review: the flat embedding_provider /
embedding_model columns on KnowledgeBaseRecord were redundant
with model_selection, which already carried the same data inside
its dict. Drops the columns and routes everything through two new
helpers.
* New helpers get_embedding_provider(model_selection) and
get_embedding_model(model_selection) in
langflow.api.utils.knowledge_base_service. Both extract the
field from a model_selection payload (single-dict or list-wrapped),
with sensible fallbacks ("Unknown" / empty string).
* record_to_metadata_dict continues to emit the flat keys in the
legacy JSON shape (the API response and on-disk metadata.json
consumers still want them) — but they're now derived views over
model_selection via the helpers, not separate columns.
* KnowledgeBaseRecord model: removed the two columns. Migration
15fe9304bca7_add_knowledge_base_table.py edited to not create
them in the first place (branch has no users — same approach as the
ingestion_run lifecycle collapse).
* create_record signature drops embedding_provider /
embedding_model params. Three callers updated:
- knowledge_bases.py:create_knowledge_base folds the request's
flat fields into model_selection when the request didn't
carry one of its own.
- lfx/components/files_and_knowledge/ingestion.py:_persist_kb_record
passes model_selection directly.
- backfill_from_disk synthesizes model_selection from the
legacy on-disk flat fields (preserving the
infer_embedding_provider fallback for KBs that only have a
model name).
* On-disk embedding_metadata.json writes (in
lfx/components/files_and_knowledge/ingestion.py) and
KBAnalysisHelper.get_metadata reads of those flat keys are
unchanged — the file format keeps the flat fields for back-compat
with retrieval / memory_retrieval readers.
* Tests: ~10 sites updated to pass model_selection={...} instead
of the two flat params. test_backfill_inserts_missing_rows now
asserts on record.model_selection.get('provider').
API surface: CreateKnowledgeBaseRequest and KnowledgeBaseInfo
keep the flat fields (frontend back-compat is preserved). The
columns are gone, the helpers are the single source of truth.
31 tests pass (21 service + 10 endpoint).
* [autofix.ci] apply automated fixes
* fix(kb): standardize perform_ingestion on model_selection
CI surfaced a TypeError after the previous commit's global rename
swept up a perform_ingestion() test call too — that function still
took flat embedding_provider / embedding_model args while the test
was now passing model_selection.
Aligned the helper with the rest of the standardization: drops the
two flat params, accepts model_selection (dict | list) and derives
provider / model internally via the existing
get_embedding_provider / get_embedding_model helpers when calling
build_embeddings.
Updated all three call sites in knowledge_bases.py
(file-upload / folder / connector ingest paths) to pass
model_selection — synthesized from the legacy flat metadata fields
when the on-disk metadata predates model_selection. Reverted the one
test that the original sweep missed (test_perform_ingestion_rollback)
to the new shape.
4/4 TestPerformIngestionTask tests pass.
* Update Vector Store RAG.json
* feat(kb): user metadata on ingestion + retrieval (#12921)
* fix(frontend): rename KB upload labels and open ingest section by default
Rename "Configure Sources" heading to "Ingest Content" and the dropdown
trigger from "Add Sources" to "Add Files". Drop the "Hide Configuration"
footer toggle so the ingest section is always visible. Chunk size,
overlap, and separator inputs render disabled until at least one source
is added. Match the Add Files focus-visible ring to border-input so the
Radix-managed focus return no longer flashes a black outline.
* fix(models): mark Google + IBM embedding models deprecated
Flag the Google Generative AI and IBM WatsonX embedding model entries in
the unified model catalog as deprecated. They are not natively supported
by Knowledge Base ingestion, so users were attempting invalid
configurations. The KB upload picker and Model Providers modal now fetch
with include_deprecated=true and render a "Deprecated" badge so the
models stay visible but are clearly flagged as unsupported.
* fix(models): keep gemini-embedding-001 active
Only mark text-embedding-004 and embedding-001 as deprecated; the
gemini-embedding-001 model is still served by the Google Generative AI
v1beta endpoint, so leave it visible without the badge.
* feat(kb): user metadata on ingestion + retrieval
Surface the per-chunk user-metadata pipeline that already lives on the KB
infra branch. Adds the API + UI to collect tags at ingest time, persist
them on the ingestion_run row, filter the chunks browser by them, and
narrow KnowledgeBaseComponent retrieval through them.
Backend
- POST /{kb}/ingest gains optional metadata + per_file_metadata multipart
fields. POST /{kb}/ingest/folder gains the same fields on its JSON body.
- New parse_user_metadata / parse_per_file_metadata helpers enforce ≤16
keys, lowercase ^[a-z0-9_]{1,32}$ keys, ≤256-char values, ≤16-item
string arrays, and reject the reserved chunk-internal keys.
- FileUploadSource and FolderSource merge per-file overrides into each
IngestionItem.source_metadata; perform_ingestion now lets per-item
values win over run-level on key collision.
- ingestion_run grows a user_metadata JSON column (alembic
16a290ab1332, EXPAND, server_default '{}').
- GET /{kb}/chunks accepts repeating meta_<key>=<value> params; values
AND across keys, OR within each key. Sidesteps the global comma-split
query middleware that breaks JSON-blob query params.
- KnowledgeBaseComponent gains a metadata_filter input that post-filters
the similarity_search results client-side until per-backend
translators land.
Frontend
- New MetadataEditor renders inside the KB upload modal under Chunking
Settings for run-level tags, and inside FilesPanel as an expandable
per-file override.
- Chunks browser gets ChunksMetadataFilter (popover + chips) wired into
useGetKnowledgeBaseChunks via the new metadata_filter param map.
Tests
- 29 validator tests for the rule set, 4 ingestion-source tests for
per-file propagation + describe counts, integration tests for the
chunks endpoint AND/OR filter, 9 retrieval-side helper tests for
parse + match, 8 frontend MetadataEditor tests.
* [autofix.ci] apply automated fixes
* feat(kb): show metadata tags in upload review summary
Surface run-level tags + per-file override count in the Step 2 Review
summary so users can confirm what is about to ship onto each chunk
before clicking Create. Empty metadata stays invisible to keep the
summary tight on the common no-tag path.
* [autofix.ci] apply automated fixes
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* fix(kb): restore IngestionRun model after merge wipe
The merge from fix/kb-ingest-content-naming (based off release-1.10.0)
silently dropped the IngestionRun SQLModel + its registration in the
models package __init__. Without the model, alembic's autogenerate
guard saw the live DB column-set drift from the empty target_metadata
and raised AutogenerateDiffsDetected at startup, blocking the backend
from booting.
Restore the model files and re-add the import + __all__ entry so the
table is visible to SQLModel's MetaData on boot.
* [autofix.ci] apply automated fixes
* fix(kb): restore ingestion_run migrations after merge wipe
Same root cause as the previous fix: the merge from
fix/kb-ingest-content-naming dropped not just the IngestionRun model
but the three migrations that build the ingestion_run schema:
- 72df732be86b — create ingestion_run table
- e728126476a8 — add kb_id FK column
- 16a290ab1332 — add user_metadata JSON column
Without these, the model says the table exists while no migration
creates it, so test_no_phantom_migrations flags a phantom add_table
diff and the model/migration consistency CI step fails.
Restore the migration files from the original feature commit. Schema
is unchanged from that commit so this is byte-for-byte the state CI
last saw green on this branch.
* fix(alembic): collapse ingestion_run + job_metadata heads to a single tip
Two issues blocked test_no_phantom_migrations after the previous
restoration:
1. e728126476a8 (add kb_id to ingestion_run) chained off
15fe9304bca7 (creates knowledge_base) which left the
ingestion_run-creation migration 72df732be86b dangling as its own
head. Re-chain e728126476a8 onto 72df732be86b so the ingestion_run
chain is linear; the FK target dependency on knowledge_base is
declared via depends_on instead of down_revision.
2. After step 1 the graph still had two independent tips —
16a290ab1332 (ingestion_run.user_metadata) and da7f6b9b638a
(job.job_metadata) — neither referencing the other. Add an empty
merge migration 5238aab36810 unifying both so 'alembic upgrade
head' resolves to a single revision.
`alembic heads` now reports ['5238aab36810'] and
test_migration_execution passes locally (8 passed, 2 sqlite-only
skips).
* [autofix.ci] apply automated fixes
* fix(alembic): mark merge migration with EXPAND phase
The migration validator only accepts EXPAND / MIGRATE / CONTRACT.
Empty merge migrations are pure graph nodes (no DDL) so they belong
in EXPAND — they never drop or rewrite data.
* Update component_index.json
* fix(frontend): align KB upload modal test text with i18n strings
CI surfaced a TestingLibraryElementError — the test was looking for
'Ingest Content' / 'Add Files', but the merge from
feat/kb-v1-db-connectors switched those to i18n strings
('Configure Sources' / 'Add Sources' via t('knowledge.configureSources')
etc.).
Updates the two assertions and the surrounding test names to match
the current rendered copy. 71/71 KnowledgeBaseUploadModal tests pass.
* feat(kb): metadata input on flow KnowledgeIngestionComponent
Adds a Metadata advanced input accepting a JSON object. The decoded
dict is JSON-encoded once per run and stamped onto every chunk's
source_metadata key — same shape the API path writes — so the chunks
browser metadata filter and KnowledgeBaseComponent.metadata_filter
work uniformly across upload, folder, and flow-driven ingestion.
Malformed JSON or non-object payloads log a warning and skip the
metadata stamp rather than failing the run.
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* chore: rename knowledge backends to db providers
* Linting cleanup
* Update dbProviderConstants.ts
* fix: icon assets
* Update index.tsx
* fix: Ensure selection of default provider
* Update indices
* chore: Consolidate database migrations
* Update kb1a2b3c4d5e_add_knowledge_base_schema.py
* fix: Ingestions from component behave same
* Ruff fixes
* More ruff fixes
* Update Vector Store RAG.json
* [autofix.ci] apply automated fixes
* Starter project update
* fix: Address two P1s for folder access
* Regen other deps
* fix(knowledge-base): pin review modal height, poll active ingestion runs, rename labels (#12977)
* fix(knowledge-base): pin review modal height and poll active ingestion runs
- Lock the upload modal to a fixed `h-[690px]` on the Review step so the
envelope no longer resizes as users page through chunk previews of
varying length. Step 1 keeps its `min-h-*` flex behavior.
- Add drawer-scoped polling to the Ingestion Runs section: refetch every
5s while any run is non-terminal (`pending`/`running`), refetch on
drawer open, and stop entirely once all rows are terminal or the
drawer closes. Background tabs are skipped via
`refetchIntervalInBackground: false`.
* fix(knowledge-base): rename Configure Sources to Ingest Content and Add Sources to Add Files
Update the English locale to align the upload modal labels with the
agreed wording:
- `knowledge.configureSources` / `knowledge.helpConfigureSources`:
"Configure Sources" -> "Ingest Content"
- `knowledge.addSources` / `knowledge.addSourcesTitle` /
`knowledge.submitAddSources`: "Add Sources" -> "Add Files"
`knowledge.sourcesLabel` ("Sources") is intentionally left as-is. Other
locales remain untouched and will be picked up via the existing
translation flow.
* Update component_index.json
* Update KnowledgeBaseUploadModal.test.tsx
* Update component_index.json
* fix(knowledge-base): surface metadata on chunk cards and make filter self-documenting (#12993)
Closes the QA-flagged UX gap where the chunks-browser metadata feature
was effectively only usable by API users. Two fixes bundled:
- Chunk cards now render the source file name + user-supplied tags as
chips, so the metadata applied at ingestion time is visible without
hitting the API. Reserved ingestion-internal keys (chunk_index,
job_id, etc.) stay hidden via a frontend mirror of
KB_METADATA_RESERVED_KEYS.
- The "Filter by metadata" popover no longer asks the user to type
blind. A new GET /knowledge_bases/{kb}/metadata/keys endpoint scans
chunks server-side and returns distinct user keys + a sample of
values (capped at 50/key with a truncated flag). The popover binds
both inputs to a <datalist>, so users get native suggestions on
click while keeping free-text typing for keys not yet ingested.
Refetches on every popover open so freshly-ingested metadata shows
up without a hard page refresh.
Test coverage: 41 backend tests in TestKnowledgeBaseAPI (3 new for
the keys endpoint), 30 frontend tests in sourceChunksPage (file_name
rendering, user-tag chips, reserved-key hiding, array-valued tags,
malformed JSON, datalist population, validation, refetch-on-open).
* feat(knowledge-base): KB row settings shortcut + ingestion history in Update modal (#13012)
Two QA-flagged UX gaps on the Knowledge Bases tab:
- "Update Knowledge" was buried inside the row's overflow menu — promoted
to a one-click settings icon directly on the row, with the dropdown
retained for the rest of the row actions. Disabled while ingesting,
matching the dropdown rule.
- The "Update Knowledge" modal gave no visibility into what a KB
already contained, so users couldn't tell if a file had been ingested
before. New collapsible "Previously ingested" panel above the upload
fields shows each prior run with its user-typed source name, type,
date, status, and counts. Empty/loading/error states covered.
Backend additions
- IngestionRunInfo gains a ``source_name`` field, populated in
``_run_row_to_info`` from ``RunRow.source_config["source_name"]``.
Whitespace-only values collapse to ``None`` so the frontend renders
the type label as a clean fallback.
Frontend
- IngestionHistoryPanel uses ``useGetIngestionRuns`` with
``staleTime: 0`` + ``refetchOnMount: "always"`` so the list is fresh
immediately after an ingest, mirroring the cache fix from #12993.
- StepConfiguration receives ``kbName`` and renders the panel only when
``isAddSourcesMode && kbName``.
- Run row prefers ``source_name`` (e.g. "Test6") with the type label
("File Upload") as a small subtitle when both are present.
Tests
- Backend: new ``test_exposes_source_name_from_source_config`` covers
populated, missing, and whitespace-only cases.
- Frontend: 6 IngestionHistoryPanel tests (empty/loading/error/render
states, source_name preference, whitespace handling, collapse,
hook args). 3 new column tests for the row settings icon. Existing
column tests migrated to the new ``kb-row-actions-trigger`` testid.
* Update Vector Store RAG.json
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update IngestionHistoryPanel.tsx
* fix(memory-base): resolve embeddings from static registry, fix Google Gemini provider inference (#13023)
Memory Base retrieval failed with "Embedding model X for provider Y not
found." for every supported embedding model. Two interacting bugs:
1. KBIngestionHelper.build_embeddings looked the model up in the
user-filtered catalog returned by get_embedding_model_options. That
catalog is empty whenever the per-user credential lookup silently
returns an empty enabled-providers set — which happens when the call
bridges from an async event loop into a worker thread. Fall back to
the static EMBEDDING_PROVIDER_CLASS_MAPPING / EMBEDDING_PARAM_MAPPINGS
registry instead: the metadata persisted in embedding_metadata.json
is the source of truth, and the runtime API key is still resolved by
get_embeddings.
2. infer_embedding_provider mis-classified models/gemini-embedding-001
as "OpenAI" (no Google pattern matched the gemini-embedding name) and
the Google patterns returned "Google" instead of the canonical
"Google Generative AI" registered in EMBEDDING_PROVIDER_CLASS_MAPPING.
Look the model up in the unified catalog first, and update the
pattern table to use the canonical provider name plus the
"models/..." prefix Google uses.
Adds regression tests covering OpenAI text-embedding-3-*, Google
gemini-embedding-001, and the previously-empty user-filtered catalog
path.
* feat: Adding Chroma Cloud as Vector DB provider (#13030)
* Adding Chroma Cloud support for Knowledge Bases.
Added Chroma cloud support. Users can configure Chroma Cloud and switch between providers especially between ChromaDB local and Cloud.
* [autofix.ci] apply automated fixes
* Update Vector Store RAG.json
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix(kb): align OpenSearch KB default vector field with canvas component (#13009)
* fix(kb): align OpenSearch KB default vector field with canvas component
The KB OpenSearch backend defaulted to 'vector_field' while the
canvas OpenSearch component defaults to 'chunk_embedding'.
Combining KB ingestion with the canvas component on the same index
failed with "Field 'chunk_embedding' is not knn_vector type".
Switch the KB backend (and the corresponding frontend defaults /
fallbacks in the DB Providers settings page) to 'chunk_embedding'
so both layers agree out of the box. Operators who already
ingested into 'vector_field' can still override via backend_config
or the global variable.
* test(kb): pin OpenSearch backend default vector_field to 'chunk_embedding'
Adds unit coverage for ``OpenSearchBackend`` so the canvas / KB
field-name alignment can't silently regress:
* asserts ``DEFAULT_VECTOR_FIELD == 'chunk_embedding'``,
* asserts ``_build_vector_store`` passes ``vector_field='chunk_embedding'``
to the LangChain wrapper when no override is provided,
* asserts an explicit ``backend_config['vector_field']`` still wins,
* parametrized table for None / empty / truthy resolution.
* fix(kb): cancel in-flight ingestion jobs on KB delete (#13003)
* fix(kb): cancel in-flight ingestion jobs on KB delete
Without this, deleting a KB while an ingestion job is running would
leave the deleted KB reappearing in the list a few seconds later. The
background job kept polling `is_job_cancelled` (it never tripped) and
its open backend client would auto-recreate the KB directory on the
next chunk write — Chroma's PersistentClient creates its sqlite parent
on demand. The ingestion then rewrote `embedding_metadata.json` into
the recreated dir, and the list endpoint's disk-fallback path
re-discovered it as a valid KB.
Add `JobService.cancel_in_flight_jobs_by_asset` and call it from the
single + bulk KB delete endpoints before storage teardown. The
ingestion's existing cancelled-handler then runs cleanup, so no
further chunks are written.
* fix(kb): DB-first delete + .kb_deleted sentinel hides locked-disk dirs
Two related bugs around KB delete on Windows when Chroma still holds an
exclusive SQLite lock:
1. delete_storage() ran before the DB row delete, so a locked rmtree
turned into HTTP 500 and left the row dangling. The dir was
partially cleaned (non-locked files removed) but the row plus the
SQLite stub kept the KB visible to listings.
2. The disk-scan fallback in get_knowledge_bases() and the API list
endpoint had no way to tell a "really still here" dir from a
"we tried to delete this and rmtree was blocked" dir, so a deleted
KB kept reappearing in the canvas component dropdown.
Fix:
* KBStorageHelper.delete_storage() now drops a .kb_deleted sentinel
file inside any directory it could not physically remove. Replaces
the previous .deleted_<name>_<timestamp> rename fallback (which
failed under the same Windows lock conditions on the rename itself).
* delete_knowledge_base() and delete_knowledge_bases_bulk() reorder
to DB-first (Option A). When the DB delete succeeds and storage
cleanup fails, the endpoint returns 200 with a warning instead of
500: the user no longer sees the KB, and the operator gets a clear
follow-up message.
* lfx.base.knowledge_bases.knowledge_base_utils.get_knowledge_bases()
skips dirs carrying the sentinel. The API-side list endpoint's
disk-scan fallback does the same via KBStorageHelper.is_kb_dir_deleted.
* create_knowledge_base() detects sentinel-marked dirs and surfaces a
clearer 409 ("previously deleted; awaiting restart cleanup") instead
of the bare "already exists", and clears any stale sentinel after a
successful mkdir as a defensive no-op.
* Cross-package contract: the lfx layer cannot import from the langflow
API package, so it inlines the sentinel filename string. A unit
test in test_kb_storage_deletion.py asserts the inlined value matches
KB_DELETED_SENTINEL.
Tests: 88 KB-related tests pass (test_kb_storage_deletion.py +
test_knowledge_bases_api.py). Two existing tests updated to assert the
new 200-with-warning shape; the previous 500-on-storage-failure shape
was the buggy behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: Stay deleted when sentinel shows up
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: enforce session isolation in MemoryBase retrieval and purge chunks on session delete (#13037)
* fix: enforce session isolation in MemoryBase retrieval and purge chunks on session delete
Two related bugs in the MemoryBase feature (#12903):
1. Filter by Session: switch the Chroma where-clause to the canonical
`{"session_id": {"$eq": ...}}` form and force-coerce the toggle to
a real bool. Both shapes are accepted by chromadb today, but the
explicit form removes any ambiguity if a non-bool slips into the
attribute and would otherwise be truthy (e.g. the literal string
"false").
2. Ghost chunks on session delete: the message-session delete
endpoints only wiped MessageTable rows, leaving every embedded
chunk behind in Chroma. New sessions then pulled the deleted
data back through the retrieval component. Wire the delete
endpoints to a new `MemoryBaseService.purge_session_data` that
walks the user's Memory Bases, deletes chunks where
`session_id == X`, and clears the corresponding tracking rows
(MemoryBaseSession + MemoryBaseWorkflowRun). Chunk-delete
failures are logged but never abort the user-visible message
delete.
* Update ingestion.py
* fix: exclude component error messages from MemoryBase ingestion (#13039)
Component errors (e.g. an embedding API failure) were persisted to the
message table with `error=True` / `category='error'` and the MemoryBase
ingestion task pulled them in like any other message, embedding the error
text into Chroma. Retrieval then surfaced these chunks as context.
Filter both signals at the SQL fetch boundary so errors never enter the
document-build / Chroma write path. The cursor still advances past any
later non-error message, so skipped error rows are not reconsidered.
Includes regression tests covering the error flag, the error category,
and cross-session/cross-flow isolation.
* fix(kb): require OpenSearch username/password in DB Provider settings (#13005)
The DB Providers settings panel marked Username and Password as
optional, but the OpenSearch runtime components (canvas vector-store +
KB backend) default to basic auth and raise "Auth Mode is 'basic' but
username/password are missing" when the corresponding global variables
aren't set. Operators who filled in only the cluster URL hit a
confusing error far downstream from the settings screen.
Mark the credential fields required so the existing canSave / save
validation in DBProvidersPage forces them up-front. Operators with
auth-less clusters (sigv4 / upstream proxy) can still satisfy the
form with a placeholder value — OpenSearch ignores credentials when
auth is not enforced.
Also assert the asymmetric (one-field-only) case in tests, since the
backend's tuple-based http_auth silently treats partial creds as
no-auth.
* Update component_index.json
* Update component_index.json
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update kb1a2b3c4d5e_add_knowledge_base_schema.py
* [autofix.ci] apply automated fixes
* fix: templates
* fix: Session filtering for memory bases
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update component_index.json
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update test_memory_retrieval.py
* test: pin OpenSearch backend filter=None drop behavior
The OpenSearch backend's similarity_search override drops `filter` when
None/empty so LangChain's OpenSearchVectorSearch wrapper never injects
`"filter": null` into the k-NN query body. If that regresses, every KB
retrieval against OpenSearch fails with x_content_parse_exception and the
upstream flow loses both KB and any downstream context — which is the most
likely OpenSearch-only symptom shape behind Rafael's report.
These tests pin:
- filter=None: kwarg is dropped from both asimilarity_search and
asimilarity_search_with_score
- filter={}: also dropped (k-NN parser rejects empty objects)
- filter={truthy}: forwarded unchanged
* fix(knowledge-base): KB UI polish — drawer + row dropdown (#13069)
* fix(knowledge-base): keep drawer open when interacting with row dropdown
The KnowledgePage outside-click handler treats clicks on Radix portal
content (dropdown menus, popovers, dialogs, tooltips) as outside the
drawer and closes it. Because dropdown content lives on document.body,
clicking a menu item triggered closeDrawer, the layout reflowed (mr-80
toggled off), AG-Grid resized, and the row re-rendered—tearing down the
open dropdown before the click event reached the menu item. Result:
"View Chunks" and other row actions required two clicks.
Skip the dismissal when the mousedown target is inside any Radix popper
portal, menu, dialog, or tooltip.
* refactor(knowledge-base): rename row action to "Ingest Files" with FileUp icon
The row shortcut and dropdown item previously used a gear icon and the
label "Update Knowledge". The gear implies settings, but the only action
behind it is adding/ingesting files. Switch both the icon (Settings →
FileUp, RefreshCw → FileUp) and the label ("Update Knowledge" → "Ingest
Files") so the affordance matches the action.
Tooltip, aria-label, and dropdown item label updated. Tests adjusted
accordingly.
* fix(knowledge-base): stop clipping stacked Layers icon in Name column
The Name cell renderer applied "-mx-[3px]" to the row icon, pulling it
3px outside its layout box. For KBs with multiple source types the icon
becomes "Layers" (stacked), which has visible content right up to its
bounding box. With the negative margin, the icon's left edge crossed
AG-Grid's cell overflow boundary and got clipped.
Drop the negative margin, give the cell wrapper a 4px left padding for
breathing room, and tighten the icon→text gap from 4 to 3 so total
horizontal layout stays close to the original.
* fix(knowledge-base): give ingestion history rows room to breathe
Rows in the Previously Ingested panel packed the status badge, source
name, type label, and time onto a single line. Long source names + long
type labels (e.g. "Chroma Cloud Collection") clipped against the time
stamp and made the row hard to scan.
Restructure each row into three stacked sections:
1. status badge (left) · relative time (right)
2. source name (sm/medium) with the type label as a quiet subtitle
3. succeeded / failed / skipped counts · chunks
Row padding bumped from p-2 to p-3 and inter-section gap from gap-1 to
gap-2 to match the new hierarchy.
* refactor(knowledge-base): drop Source Files / Linked Flows placeholders
Both sections in the KB inspection drawer always rendered the same empty
state ("No source files available." / "No linked flows available.")
because neither field is wired up on the backend yet. Showing empty
placeholders adds noise without giving the user information.
Remove the two sections from the drawer. Update the mock + assertions
in KnowledgeBaseDrawer.test.tsx so the test stops claiming the removed
strings should render.
* fix(knowledge-base): give drawer ingestion runs the same room to breathe
The drawer's Ingestion Runs rows had the same cramped single-line layout
the modal panel had before the previous commit: status badge + source
type + time crammed onto one row, counts beneath. Match the new pattern
so the two surfaces feel consistent.
Each row now stacks:
1. status badge (left) · relative time (right)
2. source type label (sm/medium)
3. succeeded / failed / skipped counts · chunks · bytes
Row padding bumped p-2 → p-3 and inter-section gap gap-1 → gap-2.
---------
Co-authored-by: Eric Hare <ericrhare@gmail.com>
---------
Co-authored-by: Debojit Kaushik <kaushik.debojit@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: keval shah <kevalvirat@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: add apscheduler and cryptography for litellm proxy_server imports
When langflow-ide logs through litellm (e.g. to langfuse), litellm imports
its proxy server module, which requires `apscheduler` and `cryptography` at
runtime. The natural fix would be `litellm[proxy]`, but that extra pins
boto3>=1.40.76, which collides with our `aioboto3` transitives on
release-1.10.0. Adding the two missing modules directly keeps the install
working without taking on the rest of the proxy server runtime.
Fixes#12228
* test: guard apscheduler and cryptography in litellm extra
Asserts the `litellm` optional-dependency group keeps the runtime modules
needed for langfuse logging via litellm.proxy.proxy_server (#12228).
---------
Co-authored-by: Jah-yee Agent <agent@openclaw.ai>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: gracefully handle pyperclip failure in headless/remote environments (fixes#12341)
The langflow api-key command failed in Docker/SSH environments because pyperclip
could not find a clipboard mechanism. Wrap the clipboard copy in a try/except so
the API key is still displayed when clipboard access is unavailable.
* [autofix.ci] apply automated fixes
* test: cover pyperclip failure path in api_key_banner (#12341)
Regression guard ensuring the API key is still displayed when the
clipboard mechanism is unavailable (Docker/SSH/headless), plus a
drift guard for the happy-path clipboard hint.
* fix: log clipboard failure instead of silent pass (ruff S110)
Replaces the bare `except: pass` with a debug log so the failure is
diagnosable in headless environments without spamming users with
warnings about an expected condition.
---------
Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: strip reserved 'code' param before build() in component reuse path (#8610)
Root cause: When a custom component is reused (self.custom_component already exists),
get_params() returns params that still include the reserved 'code' field. Unlike the
instantiate_class path which pops 'code' before use, the reuse path passes it through
to build_custom_component, which calls build(**params) — causing TypeError for
components whose build() method does not accept **kwargs.
Made-with: Cursor
* test: add regression tests for custom component code parameter reuse
Verifies that TypeError no longer occurs when reusing custom components
with the code parameter in both instantiate and reuse paths.
Per CodeRabbit review on PR #12712.
Made-with: Cursor
* [autofix.ci] apply automated fixes
* fix: also strip 'code' from build params in langflow loading duplicate
Mirror the lfx fix in the parallel src/backend/base/langflow/interface/
initialize/loading.py copy. In production this build_custom_component is
dead code (only update_params_with_load_from_db_fields is imported from
this module), but keeping the duplicate in sync avoids future drift and
the same TypeError if any caller starts using it.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: use stable UUIDs for Qdrant document IDs to prevent duplicate ingestion (fixes#12641)
* fix: deterministically serialize Qdrant doc metadata for stable IDs
Use json.dumps with sort_keys=True and the project's custom_serializer
instead of str(sorted(metadata.items())). The previous form relied on
Python's repr for nested dicts/sets/datetimes/Decimals, which is unstable
across processes (PYTHONHASHSEED for sets) and does not recurse into
nested dict ordering.
* test: add unit tests for Qdrant stable-ID generation
Cover the four invariants of the deterministic UUID5 derivation:
- same content + metadata yields the same UUID across runs (uuid5)
- metadata insertion order is irrelevant (sort_keys=True)
- different content yields different UUIDs
- non-primitive metadata (datetime, Decimal) serializes via custom_serializer
---------
Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* deps: add opentelemetry http client instrumentation packages
Add opentelemetry-instrumentation-requests and opentelemetry-instrumentation-urllib3
to enable W3C TraceContext propagation on outgoing HTTP calls.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(tracing): propagate W3C TraceContext on outgoing HTTP calls
Instrument requests and urllib3 HTTP clients with the tracer provider
so that traceparent headers are injected on outgoing HTTP requests
(e.g., to LLM APIs). This enables end-to-end distributed tracing.
Both Arize Phoenix and LangWatch tracers now:
- Call _instrument_http_clients() during setup
- Call _uninstrument_http_clients() during cleanup
- Handle missing packages gracefully with debug logging
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(tracing): add tests for HTTP client trace context propagation
Verify that:
- RequestsInstrumentor and URLLib3Instrumentor are called during tracer setup
- Instrumentors are uninstrumented during cleanup
- traceparent headers are injected with valid W3C format on outgoing requests
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(tracing): address ruff linting issues in trace context propagation
- Use contextlib.suppress instead of try-except-pass in uninstrument methods
- Fix nested with statements using parenthesized context managers
- Fix unused arguments by prefixing with underscore
- Add timeout to requests.get calls
- Skip integration tests that mock at wrong layer (Session.send bypasses OTel)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(tracing): make HTTP client instrumentation process-scoped with ref-counting
The OpenTelemetry RequestsInstrumentor and URLLib3Instrumentor monkeypatch
globally, but tracers were calling instrument()/uninstrument() per instance.
If multiple graph runs overlapped, the first run to end() would uninstrument
globally and break context propagation for other active runs.
Changes:
- Add HTTPClientInstrumentationManager singleton with reference counting
- enable() increments count, instruments only on first call
- disable() decrements count, uninstruments only when count reaches zero
- Replace silent exception suppression with proper logging for debugging
Addresses PR review feedback from Qodo.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* initial draft
* update DS Star agent
* chore(deps): use OpenDsStar 1.0.1 from PyPI
* refactor: move OpenDsStar to namespace, update imports, remove agents shim
* update open ds star version
* improved logging
* improved logging
* fix: propagate source file path for DataFrame inputs in ingestion components
DataFrame from CSV/Excel reads now carries the source file path via
pandas attrs, so IngestionDescriber and FileContentRetriever can
resolve the original file instead of searching per-row data keys.
* feat: add files_ingestion component bundle and bump OpenDsStar to 1.0.8
Register files_ingestion as a sidebar bundle in the frontend and
component registry. Update OpenDsStar dependency from 1.0.6 to 1.0.8.
* Add file content retriever component and tests
* Fix: Add tool_mode=True to FileContentRetriever outputs to prevent execution during build phase
* Fix: Return empty DataFrame instead of raising error when file_path not provided during build
* Update file content retriever component
* refactor: rename IngestionDescriberComponent to FileDescriptionGeneratorComponent
- Renamed class from IngestionDescriberComponent to FileDescriptionGeneratorComponent
- Updated component name attribute and display name
- Renamed files: ingestion_describer.py -> file_description_generator.py
- Renamed test file: test_ingestion_describer.py -> test_file_description_generator.py
- Updated all imports and references
- Rebuilt component index
BREAKING CHANGE: Component class name changed. Existing flows using IngestionDescriberComponent will need to be updated.
* fix: improve FileContentRetriever tool descriptions for CodeAct agent
- Add explicit warnings that tools expect string file paths only
- Clarify that search results/Data objects should not be passed
- Change error handling to raise ValueError (agents can handle exceptions)
- Add build-phase safety (return empty results when no path provided)
- Update tests to expect exceptions instead of error messages
This fixes CodeAct agent failures where it was trying to pass search
results (list of Data objects) to retrieve_content_as_dataframe()
instead of extracting the file path string first.
* fix: File Content Retriever now finds DataFrames with file_path in columns
- Added caching for file maps to avoid rebuilding on each method call
- Enhanced _get_file_maps() to check both attrs['source_file_path'] and 'file_path' column
- Fixes agent issue where DataFrame from Read File component couldn't be found
- Added test for DataFrame with file_path in columns scenario
- All 30 tests pass (26 passed, 4 skipped)
* fix: use to_csv() instead of to_string() for DataFrame text representation
Fixes agent hanging issue when retrieving file content as DataFrame.
The problem was that to_string() creates a formatted table representation
that cannot be parsed as CSV. When agents called retrieve_content_as_dataframe(),
it would try to parse this formatted text with pd.read_csv(), causing a
ParserError and infinite retry loop.
Changed to use to_csv(index=False) which produces valid CSV format that can
be parsed back into a DataFrame when needed.
- Maintains optimal performance by returning original DataFrame from dataframe_map
- Only uses CSV text representation as fallback
- All 26 tests pass
- Works correctly for both text and tabular files
* add agent template
* improve agent template
* bump OpenDsStar to 1.0.10 and update ingestion components
Upgrade OpenDsStar dependency from 1.0.8 to 1.0.10 in both langflow-base
and lfx pyproject.toml. Update file content retriever, file description
generator, component tool, and custom component with related improvements.
Expand test coverage for file content retriever.
* fix FileContentRetriever multi-file data retrieval and add code_timeout
The FileContentRetriever was broken for the multi-file case (when Read File
outputs a DataFrame with file_path + text columns). Three bugs were fixed:
1. Multi-file DataFrame mapped to wrong data: When Read File produced a
summary DataFrame (one row per file, columns = [file_path, text]), the
retriever mapped each file_path to the entire 2-row summary DataFrame
instead of extracting per-file content. The agent would get 2 rows
instead of thousands of actual data rows. Fixed by iterating rows and
extracting each file's text into text_map.
2. CSV text now eagerly parsed into DataFrames: For .csv/.tsv files in
text_map that lack a pre-built DataFrame, _get_file_maps() now parses
the text into a DataFrame eagerly during initialization. This makes
retrieve_content_as_dataframe work for the multi-file case without
needing upstream structured DataFrames.
3. Maps built once, not per tool call: The file maps were rebuilt on every
tool call because component_tool.py deepcopy's the component, and the
maps were only built lazily during retrieval. Now the early-return paths
(empty file_path) eagerly call _get_file_maps() during component build
time, so the cached maps survive deepcopy. This is critical for
scalability with hundreds of large files.
Additional changes:
- Removed _is_likely_file_content heuristic that was incorrectly filtering
out valid file content
- Removed "Message" from input_types (was never handled)
- Removed redundant inner import of DataFrame
- Shortened error messages (cap available files to 5)
- Added warning log for unsupported input types
- Lazy CSV conversion: no longer eagerly converts structured DataFrames to
CSV text in _get_file_maps, only converts on demand in retrieve_content
- Added code_timeout input to OpenDsStarAgentComponent (default 60s, was
hardcoded 30s) to prevent timeouts with large datasets
- Added 19 unit tests covering all retrieval paths, caching, and the
multi-file bug reproduction
* disable as_dataframe tool from vector store to fix agent confusion
The Chroma vector store's as_dataframe output was exposed as an agent tool,
returning a 2-row file index DataFrame (file_path + text columns). The agent
mistook this for actual data and tried to query 'price' on it, causing every
query to fail with KeyError.
Set tool_mode=False on the as_dataframe output so only search_documents is
exposed as a tool from the vector store. The agent now follows the correct
workflow: search_documents → get file path → retrieve_content_as_dataframe.
* Revert "disable as_dataframe tool from vector store to fix agent confusion"
This reverts commit e21c5dc649.
* bump OpenDsStar to 1.0.15, improve ingestion component tool descriptions and caching
- Upgrade OpenDsStar dependency from 1.0.10 to 1.0.15
- Add __deepcopy__ to FileContentRetriever to preserve cached maps across tool invocations
- Add JSON format support for eager DataFrame parsing
- Improve tool docstrings on FileContentRetriever and VectorStore for better agent usage
- Enable tool_mode on VectorStore search and table outputs
* support Message input in FileContentRetriever and FileDescriptionGenerator
- Add file_path to Message metadata in base_file._extract_file_metadata
- Accept Message type in FileContentRetriever.file_data input and handle
it in _build_file_maps to extract file_path and text content
- Handle Message input in FileDescriptionGenerator to extract file_path
- Add tests for Message input: basic, missing file_path, and mixed types
* fix FileContentRetriever not appearing in component sidebar
- Remove forward reference to FileContentRetrieverComponent in __deepcopy__
that broke the exec-based template builder, silently preventing registration
- Remove unnecessary cast() call in __deepcopy__
- Regenerate component_index.json (362 -> 363 components)
- Update template JSON edge handles to include Message in input_types
* add persistent directory, real-time logs, tool descriptions, and Merge Flows component
FileContentRetriever:
- Add persistent directory support for saving/loading text and DataFrame maps to disk
- Split text storage into individual files (texts/) instead of single JSON
- Guard against None file_data during tool deepcopy calls
- Warn when maps are empty after building
- Make Persistent Directory a visible (non-advanced) input
FileDescriptionGenerator:
- Switch from subprocess.run to Popen with real-time stderr streaming
- Add configurable timeout input (default 1800s, was hardcoded 600s)
- Stream subprocess progress to UI via self.log()
Tool descriptions:
- Add info field to Output model (excluded from serialization to avoid template pollution)
- Fix stale tool metadata overwriting fresh descriptions in component.py
- Each tool output now gets its own description instead of generic component description
New components:
- Add Merge Flows component to trigger multiple upstream flows from a single node
Tests:
- Add persistence tests (round-trip, skip duplicates, add new files, corrupted JSON, regression)
- Update FileDescriptionGenerator tests for Popen-based subprocess
* update Structured Data Agent template from UI re-export
* update Structured Data Agent template
* update Structured Data Agent template
* update Structured Data Agent template
* update Structured Data Agent template
* update Structured Data Agent template
* fix chardet perf, pipe deadlock, auto-sync, Chroma errors, and ingestion failure detection
Performance:
- Fix chardet.detect() scanning entire files (800s for 1.5GB) — sample first 100KB only
- Default ascii detection to utf-8 (superset) to handle non-ASCII chars beyond sample
- Fix subprocess stdout pipe deadlock — drain both stdout and stderr concurrently via select
FileContentRetriever:
- Add auto-sync: remove persisted entries for files no longer in file_data input
- Clean up orphaned files from persistent directory on save
FileDescriptionGenerator:
- Fail with clear error when descriptions are not generated (grouped by error reason)
- Increase default timeout from 1800s to 3600s via _DEFAULT_TIMEOUT_SECONDS constant
- Remove fast-path code (moved to OpenDsStar)
Chroma:
- Add clear error messages for missing persist directory, corrupted DB, and initialization failures
- Recreate persist directory if deleted between runs
Other:
- Add opendsstar_cache to .gitignore
* Chore(release): merge release 1.9.0 into main (#12710)
* test: add upgrade migration check to ci (#12061)
* Add upgrade migration check to ci
* [autofix.ci] apply automated fixes
* Add fetch step
* ruff
* Add merge migration
* Revert "Add merge migration"
This reverts commit fd32424739.
backups
* coderabbit suggestions
1. Shell hardening in workflow - set -euo pipefail, full path grep, quoted variables
2. _WORKSPACE_ROOT extracted as module constant (also addresses Cristhianzl's review comment about parents[5] duplication)
3. git missing returns None instead of raising FileNotFoundError
4. # noqa: S603 added to subprocess.run (fixes the Ruff CI failure)
5. FK noise filtering now also compares target table/column, not just ondelete/onupdate
6. Removed redundant git fetch origin main step (fetch-depth: 0 already fetches all branches)
7. Deduplicated Alembic config creation in _get_main_branch_head (moved before the if branch)
8. Simplified dict type hints (removed unnecessary dict[tuple, object])
* test: improve migration tests from PR review feedback
- Narrow broad except clause to only wrap subprocess.run call
- Add specific error messages for multi-head and unresolvable revisions
- Remove redundant hardcoded schema test (covered by compare_metadata)
- Fix SQLite FK noise filter to skip ondelete/onupdate comparison
- Add downgrade verification to test_upgrade_from_main_branch
- Add test file and workflow to CI trigger paths
- Add prompt for follow-up PostgreSQL migration test PR
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* add engine check on downgrade
* [autofix.ci] apply automated fixes
* fix: harden CI error handling and test robustness
- Set validationPassed=false when validator crashes so CI fails instead of passing silently
- Wrap GitHub API calls in try-catch so comment-posting failures don't mask validation results
- Preserve git stderr in warnings for better CI debugging
- Add defensive handling for unexpected FK constraint shapes in SQLite noise filter
- Clean up SQLite WAL/SHM/journal companion files in test teardown
* Add explicit fetch to main
* ruff
* [autofix.ci] apply automated fixes
* Add sqlite filter tests and remove redundant fetch
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(deployments): unify payload passthrough from api to adapter (#12190)
* feat(deployments): unify dynamic payload passthrough across api and adapter
* use datatime.timezone for python3.10 compatibility
* use appropriate type vars in slots and sanitize error message
* tweaks to schemas
* use policy to avoid dump churn
* fix: allow clearing Max Tokens field with Backspace/Delete (#12198)
* fix: allow clearing Max Tokens field with Backspace/Delete
Empty string input was being converted to 0 via Number(""), which
triggered the min-value guard and snapped the field back to 1 before
onChange could propagate. Adding an early return for empty input lets
the field clear correctly, propagating null (no limit) downstream.
* test: add IntComponent tests for handleInputChange clearing behavior
Covers the regression where Backspace/Delete was blocked by the
min-value guard, and verifies that below-min values still clamp
correctly.
* fix: Resolve CodeQL false positives for path injection and URL substring sanitization (#12201)
* fix: Add explicit left/right DataFrame inputs for merge operations (#12177)
* add explicit left/right DataFrame inputs for merge operations
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* ruff style and checker fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix: add dict to allowlist preventing TableInput data loss (#12074)
* fix: add dict to allowlist preventing TableInput data loss
Co-Authored-By: DeyLak <DeyLak@users.noreply.github.com>
* test: add regression test for TableInput list[dict] preservation
Regression test for: https://github.com/langflow-ai/langflow/issues/12062
Co-Authored-By: DeyLak <DeyLak@users.noreply.github.com>
---------
Co-authored-by: DeyLak <DeyLak@users.noreply.github.com>
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
* chore: update pyproject versions 1.9.0
update pyproject versions 1.9.0
* fix: 1.9.0 nightly
1.9.0 nightly fix
* feat: add wxO deployment adapter (#12079)
* checkout wxo deployment adapter implementation
* checkout wxo deps and flow reqs impl
* clean up / minor refactor with updated tests
* major refactor: split up the implementation into folders and files
* clean up logic
* refactor and clean up of execution logic, improve type safety of "retry_create" helper, and remove dead code
* remove unused wxo service file
* remove "provider_name" arg and references
* fix: harden watsonx orchestrate deployment adapter for PR readiness
- Fix rollback AttributeError when agent_create_response is None
- Fix NoneType access on params in list() when called without params
- Fix inconsistent error types: use DeploymentNotFoundError in get/update
- Fix typo "occured" -> "occurred" in all error prefix messages
- Fix variable shadowing of fastapi.status in get_status()
- Fix pre-existing test bugs (wrong exception types, stale method refs)
- Fix e2e monkey-patching of non-existent service methods
- Add structured logging to create, delete, retry, and rollback flows
- Add jitter to retry backoff to avoid thundering herd
- Add __repr__ to WxOCredentials that fully masks the API key
- Extract hard-coded LLM model string to DEFAULT_WXO_AGENT_LLM constant
- Remove commented-out snapshot update code
- Expand test coverage from 24 to 75 tests covering retry logic,
service methods, client auth, utilities, execution/status helpers,
and artifact roundtrip
* fix(deployment): fix bugs and harden watsonx orchestrate adapter
- Fix update method silently discarding changes instead of sending them
to the WXO API
- Wrap all synchronous SDK calls in asyncio.to_thread to avoid blocking
the event loop
- Add error handling to get_status (was propagating raw exceptions)
- Use DeploymentType enum values instead of raw strings for
SUPPORTED_ADAPTER_DEPLOYMENT_TYPES
- Fix type annotations in list method (list -> set)
- Fix typo in comment (reccomend -> recommend)
- Remove dead code: extract_agent_connection_ids,
require_exclusive_resource, require_non_empty_string,
resolve_health_environment_id, fetch_agent_release_status,
normalize_release_status, resolve_lfx_runner_requirement,
_pin_requirement_name, sync_langflow_tool_connections,
create_langflow_flow_tool, resolve_snapshot_connections, and unused constants
- Make assert_create_resources_available and validate_connection async
- Make create_agent_run and get_agent_run async
- Add tests for list_types, get_status error handling, and update
side-effects
- Update existing tests for async function signatures
* actually remove the dead code from tools.py
* properly await "validate_connection"
* update e2e test file
* add more scenarios to e2e test runner
* refactor(watsonx-orchestrate): improve WxOClient encapsulation and error messages
Encapsulate SDK private method access and endpoint paths inside
WxOClient wrappers so callers never handle raw paths or touch internal
_get/_post methods. Route all private HTTP calls through a single
_base client auto-created in __post_init__, removing the externally
constructed `base` field.
Additionally fix double-period and redundant text in error messages
produced by the ErrorPrefix + handler pattern, and update E2E/unit
assertions to match the sanitised error output.
Changes:
- types.py: bake endpoint paths into wrapper signatures (post_run,
get_run, upload_tool_artifact, get_agents_raw); auto-create _base
via __post_init__; remove base from constructor
- client.py: drop BaseWXOClient import and base= constructor arg
- core/execution.py: pass run_id / query_suffix instead of raw paths
- core/tools.py: pass tool_id instead of raw path
- service.py: fix double-period in ErrorPrefix interpolation; remove
redundant restatements in generic except handlers
- tests: update _with_wxo_wrappers helper and test doubles to use
_base; update FakeBaseClient._get to accept params
- e2e: update rollback scenario detail_contains to match sanitised msg
* skip import and tests of wxo adapter if current env is running python 3.10
* clarify random prefix / retry behavior
* implement comments
* fix: align watsonx adapter with updated deployment schema
- Add `deployment_type` parameter to `get`, `update`, `redeploy`,
`duplicate`, `delete`, `get_status`, and `undeploy_deployment` in
WatsonxOrchestrateDeploymentService to match the updated
BaseDeploymentService ABC
- Update e2e script to use renamed `add_ids` field on
SnapshotDeploymentBindingUpdate
* remove non-interface method undeploy_deployment
* implement listing configs and snapshots
* add update implementation and improve http->deployment error translation and add tests
* improve exception handling
* checkout payload slot work
* custom payload schema for update
* new update implementation
* stop passing client cache to helpers
* add docs for ordereduniquests
* improve import patterns and document future risks for wxo dependencies
* remove global-variable prefixing of flows
* ref: harden typing, DRY helpers, and correctness fixes
- Replace `db: Any` with `db: AsyncSession` across client, config, and
update_helpers modules
- Extract `_ensure_dict` / `ensure_langflow_connections_binding` helpers
to eliminate repeated nested-dict safety logic in tools.py and
update_helpers.py, with documentation explaining why malformed API
payloads are silently corrected rather than rejected
- Use `PayloadSlot.parse()` for update payload validation instead of
standalone helper; remove `parse_provider_update_payload`
- Fix lambda late-binding with default-argument captures in service.py
- Use `zip(strict=True)` in update_helpers for defensive mismatch
detection
- Replace O(n²) duplicate detection with `collections.Counter` in
payloads.py
- Simplify `dedupe_list` to `list(dict.fromkeys(...))`
- Move type-annotation-only imports into TYPE_CHECKING blocks in
types.py and config.py
- Introduce `UPDATE_MAX_RETRIES` as a distinct constant from
`CREATE_MAX_RETRIES`
- Fix "watsonX" → "watsonx" casing in error messages
- Clarify `get_status` docstring and `validate_connection` error message
- Add TODO(deployments-cache) for client cache invalidation on
credential updates
* ref: drop client cache and adopt typed deployment context across the wxo adapter path, add request-context memoization with strict mixed-context guards, and lazily initialize wxo SDK clients. Add safer credential handling by storing only authenticators in WxOCredentials. Also improve provider error-detail extraction/messages and make the direct wxo E2E conflict scenario diagnostics and expectations more resilient.
* use explicit naming for context class; providerId
* fix error handling, retry safety, and exception diagnostics in wxo adapter
- Raise DeploymentError on empty API responses instead of fabricating
fake success in create_agent_run_result and get_agent_run
- Elevate rollback failure logging from WARNING to ERROR for alerting
- Apply retryable filter to retry_rollback to skip 401/403/409/422
- Preserve exception chains (from exc) instead of suppressing (from None)
across service.py, utils.py, execution.py, and update_helpers.py
- Broaden credential resolution catch to handle arbitrary DB exceptions
- Separate status code dispatch from string heuristics in
raise_for_status_and_detail to prevent misclassification
- Add tests for all behavioral changes
* [autofix.ci] apply automated fixes
* fix: harden wxO adapter safety, immutability, and error diagnostics
- Make WxOClient and WxOCredentials frozen dataclasses with eager SDK
client initialization to eliminate thread-safety races from
asyncio.to_thread workers and prevent post-construction mutation
- Move instance_url validation and normalization into type __post_init__
- Raise DeploymentError on missing run_id instead of returning partial
success with execution_id=None
- Preserve exception chains (from exc) instead of suppressing (from None)
across create, update, and delete service methods
- Add warning logs when _ensure_dict replaces non-dict binding values
and when _resolve_lfx_requirement falls back to minimum version
- Initialize derived_spec before try block to prevent potential NameError
- Log all ToolUploadBatchError errors before re-raising the first
- Change SUPPORTED_ADAPTER_DEPLOYMENT_TYPES from mutable set to frozenset
- Add tests for 409/422 error mapping in create, unsupported deployment
type rejection, empty update rejection, zip artifact extraction paths,
validate_connection negative paths, missing run_id, multiple deployment
ID rejection, exception chain preservation, and _ensure_dict warning
* [autofix.ci] apply automated fixes
* fix ruff errors and and todo for status method
* fix mypy errors
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix: 1.9.0 nightly (#12210)
fix 1.9.0 nightly
* fix(mcp): Add schema-driven type conversion (#11796)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* fix(mcp): Add schema-driven type conversion
- Add schema-driven type conversion (str→dict, str→int, etc.)
- normalize and unflatten tool arguments for MCP servers
- Unflatten flattened keys (e.g. params.search) into nested objects
* fix(mcp): handle dict type and array type in JSON schema for MCP tools
- Support "type": ["string", "null"] (JSON Schema array type)
- Normalize required to hashable elements (filter non-string entries)
- Add unit tests for create_input_schema_from_json_schema
* fix(mcp): map generic object type to dict for free-form params
When JSON schema has {"type": "object"} with no properties, treat it as
a free-form dict instead of building a nested Pydantic model. This allows
MCP servers expecting arbitrary key-value params to receive proper dicts,
and enables str→dict conversion via _normalize_arguments_for_mcp.
- Add conditional in parse_type: empty properties → dict, else nested model
- Add test_create_input_schema_generic_object_maps_to_dict
* fix(mcp): exclude None from tool arguments sent to MCP servers
* test_mcp: add more tests for datatypes
* fix(mcp): parse JSON strings for nested model params (e.g. foreman-mcp)
* fix(mcp): refactor _try_convert_value reducing repetition
* fix(mcp): add missing tests for remaining use cases
* fix(mcp): Fix mapping of None if expected is list, dict or str
* Update nightly_build.yml
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: prevent overwriting user-selected global variables in provider c… (#12217)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* fix: prevent overwriting user-selected global variables in provider config
Previously, the apply_provider_variable_config_to_build_config function would
automatically overwrite field values with environment variable keys whenever
an env var was present, even if the user had already selected a different
global variable.
This fix adds a check to only auto-set the environment variable if:
- The field is currently empty, OR
- The field is not already configured to load from the database
This preserves user selections while still providing automatic configuration
for new/empty fields.
Added comprehensive unit tests to verify:
- Auto-setting env vars for empty fields
- Preserving user-selected global variables
- Overwriting hardcoded values (expected behavior)
- Skipping when env var is not set
- Applying component metadata correctly
* [autofix.ci] apply automated fixes
* style: use dictionary comprehension instead of for-loop
Fixed PERF403 Ruff style warning by replacing for-loop with dictionary
comprehension in update_projects_components_with_latest_component_versions
* chore: retrigger CI build
* test: improve test coverage and clarity for provider config
- Renamed test_apply_provider_config_overwrites_hardcoded_value to
test_apply_provider_config_replaces_hardcoded_with_env_var for clarity
- Added test_apply_provider_config_idempotent_when_already_set to document
idempotent behavior when value already matches env var key
- Removed sensitive value from debug log message to prevent potential
exposure of API keys or credentials
These changes improve test coverage by documenting the no-op scenario
and enhance security by avoiding logging of potentially sensitive data.
* chore: retrigger CI build
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: Fixed CodeQL security scan about Incomplete URL substring sanitization on watsonx test suite (#12212)
* fix: Fixed CodeQL security scan about Incomplete URL substring sanitization
* fix coderabbitai comments
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* fix failing action
* docs: add search icon (#12216)
add-back-svg
* Revert "Merge branch 'main' into dev-fix-security-code-scan-watsonx"
This reverts commit 41eb034e11, reversing
changes made to 4e51f4d836.
* Revert "Merge branch 'main' into dev-fix-security-code-scan-watsonx"
This reverts commit 4e51f4d836, reversing
changes made to 530bddd0a4.
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
* fix: remove ibm-watsonx extra from complete installation (#12230)
* docs: lfx readme content (#11870)
* docs-add-lfx-content-to-readme
* github-link-syntax
* allowlist-blocklist
* docs: add a copy to markdown button to docusaurus theme (#12189)
* copy-page-component
* copy-theme-and-add-button
* docs: add versioning (#12218)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* initial-content
* cut-1.8-release-and-include-next-version
* stage-1.8.0-and-next
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
* docs: replace api build automation (#12214)
* remove-workflows-file-and-script
* tag-hidden-endpoints
* update-scripts-and-specs
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Debojit Kaushik <Kaushik.debojit@gmail.com>
* fix: prevent arbitrary file write via path traversal in files endpoint (#12227)
* fix(security): prevent arbitrary file write via path traversal in file uploads
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* docs: Add AI coding agent skills for code review, testing, and refactoring (#12241)
Add AI coding agent skills for code review, testing, and refactoring
* feat: Add Windows Playwright testing to nightly builds (#12221)
* feat: add Windows support for Playwright tests in nightly builds
- Add windows-latest as runner option in typescript_test.yml
- Update Playwright browser installation to be OS-aware (Windows doesn't support --with-deps)
- Add matrix strategy to nightly_build.yml to run tests on both Linux and Windows
- Update Slack notifications to indicate multi-platform testing
- Tests now run in parallel on ubuntu-latest and windows-latest
This enables catching Windows-specific regressions early in the nightly build process.
* fix: add shell: bash to all script steps for Windows compatibility
Windows runners default to PowerShell which doesn't understand bash syntax.
All steps with bash scripts now explicitly specify shell: bash to ensure
cross-platform compatibility.
* feat: make Windows Playwright tests non-blocking initially
Add continue-on-error for Windows tests to allow nightly builds to succeed
even if Windows-specific issues are found. This gives us visibility into
Windows bugs without blocking releases.
Windows tests will still run every night and report failures, but won't
block the build. Once Windows tests are stable, we can remove this flag.
* chore: fix white space
chore: fix white space
* fix: replace matrix strategy with separate jobs for Windows Playwright tests
- GitHub Actions reusable workflows don't support matrix strategy
- Split frontend-tests into frontend-tests-linux and frontend-tests-windows
- Windows tests are non-blocking (continue-on-error: true)
- Updated all job dependencies and Slack notification logic
- Addresses PR review comment about matrix limitation
* chore: trigger workflow validation refresh
---------
Co-authored-by: Adam Aghili <Adam.Aghili@ibm.com>
* fix: Avoid foreign key violation on span table with topological sort (#12232)
* fix: Foreign key violation on span table
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* address review comments
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: use deepcopy to prevent shared reference mutation in component updates (#12252)
* fix: use deepcopy to prevent shared reference mutation in component updates
Copies template data via deepcopy when updating project components to
prevent mutations from leaking between projects through all_types_dict.
Also fixes edge updates to use the copied project data and adds
cloneDeep in the frontend templatesGenerator.
* fix: deepcopy mutable FIELD_FORMAT_ATTRIBUTES and add coverage test
- Add deepcopy on field_dict[attr] assignment (line 192) to prevent
shared references for mutable attributes like input_types, options,
and fileTypes leaking back into all_types_dict
- Add test_update_components_does_not_mutate_field_format_attributes
to verify the fix covers the FIELD_FORMAT_ATTRIBUTES update path
* feat: add support for Langchain 1.0 (#11114)
* feat: upgrade to LangChain 1.0
- langchain ~=1.2.0
- langchain-core ~=1.2.3
- langchain-community ~=0.4.1
Updated all langchain-* integration packages to versions compatible with langchain-core 1.0+.
* feat(lfx): add langchain-classic dependency for legacy agent classes
LangChain 1.0 removed AgentExecutor and related classes to langchain-classic.
This adds the dependency to maintain backward compatibility.
* refactor(lfx): update imports for LangChain 1.0 compatibility
- Move AgentExecutor, agent creators from langchain to langchain_classic
- Move AsyncCallbackHandler from langchain.callbacks to langchain_core.callbacks
- Move Chain, BaseChatMemory from langchain to langchain_classic
- Update LANGCHAIN_IMPORT_STRING for code generation
* fix(lfx): make sqlalchemy import lazy in session_scope
LangChain 1.0 no longer includes sqlalchemy as a transitive dependency.
Move the import inside the function where it's used to avoid import errors
when sqlalchemy is not installed.
* chore: update uv.lock for langchain-classic
* feat: enable nv-ingest optional dependencies for langchain 1.0
- Uncomment nv-ingest-api and nv-ingest-client, update to >=26.1.0
(no longer has openai version conflict)
- Bump datasets from <4.0.0 to <5.0.0 to allow fsspec>=2025.5.1
required by nv-ingest
- Update mlx-vlm TODO comment with accurate blocking reason
* chore: update nv-ingest to 26.1.1
nv-ingest 26.1.1 removes the openai dependency, resolving the
conflict with langchain-openai>=1.0.0 (which requires openai>=1.109.1).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: enable mlx and mlx-vlm dependencies for langchain 1.0
opencv-python 4.13+ now supports numpy>=2, resolving the conflict
with langchain-aws>=1.0.0 (which requires numpy>=2.2 on Python 3.12+).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style: fix import order in callback.py
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: update imports to use langchain_classic for agent modules
* [autofix.ci] apply automated fixes
* fix: remove .item() calls in knowledge_bases.py
* fix(lfx): import BaseMemory from langchain_classic for langchain 1.0
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* refactor: update deprecated langchain imports for langchain 1.0
- langchain.callbacks.base -> langchain_core.callbacks.base
- langchain.tools -> langchain_core.tools
- langchain.schema -> langchain_core.messages/documents
- langchain.chains -> langchain_classic.chains
- langchain.retrievers -> langchain_classic.retrievers
- langchain.memory -> langchain_classic.memory
- langchain.globals -> langchain_core.globals
- langchain.docstore -> langchain_core.documents
- langchain.prompts -> langchain_core.prompts
Also simplified GoogleGenerativeAIEmbeddingsComponent to use native
langchain-google-genai 4.x which now supports output_dimensionality.
* [autofix.ci] apply automated fixes
* fix: add _to_int helper for pandas sum() compatibility across Python versions
* fix: update langfuse>=3.8.0 and fix cuga_agent.py (#11519)
* fix: update langfuse>=3.8.0 and fix cuga_agent.py
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* feat: implement LangFuseTracer for langfuse v3 API compatibility and add unit tests
* fix: upgrade cuga to 0.2.9 for langchain 1.0 compatibility
* fix: improve error handling and return value in get_langchain_callback method
* fix: update package versions for compatibility and improvements
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* fix: update langwatch dependency to version 0.10.0 for compatibility
* fix: update environment variable for Langfuse host to LANGFUSE_BASE_URL
* Update dependency versions in Youtube Analysis project
- Downgraded googleapiclient from 2.188.0 to 2.154.0
- Updated langchain_core from 1.2.7 to 1.2.9
- Updated fastapi from 0.128.1 to 0.128.5
- Downgraded youtube_transcript_api from 1.2.4 to 1.2.3
- Changed langchain_core version from 1.2.7 to 0.3.81
- Cleared input_types in model selection
# Conflicts:
# src/backend/base/langflow/initial_setup/starter_projects/Basic Prompt Chaining.json
# src/backend/base/langflow/initial_setup/starter_projects/Basic Prompting.json
# src/backend/base/langflow/initial_setup/starter_projects/Blog Writer.json
# src/backend/base/langflow/initial_setup/starter_projects/Custom Component Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/Document Q&A.json
# src/backend/base/langflow/initial_setup/starter_projects/Financial Report Parser.json
# src/backend/base/langflow/initial_setup/starter_projects/Hybrid Search RAG.json
# src/backend/base/langflow/initial_setup/starter_projects/Image Sentiment Analysis.json
# src/backend/base/langflow/initial_setup/starter_projects/Instagram Copywriter.json
# src/backend/base/langflow/initial_setup/starter_projects/Invoice Summarizer.json
# src/backend/base/langflow/initial_setup/starter_projects/Knowledge Retrieval.json
# src/backend/base/langflow/initial_setup/starter_projects/Market Research.json
# src/backend/base/langflow/initial_setup/starter_projects/Meeting Summary.json
# src/backend/base/langflow/initial_setup/starter_projects/Memory Chatbot.json
# src/backend/base/langflow/initial_setup/starter_projects/News Aggregator.json
# src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json
# src/backend/base/langflow/initial_setup/starter_projects/Pokédex Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Portfolio Website Code Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/Price Deal Finder.json
# src/backend/base/langflow/initial_setup/starter_projects/Research Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Research Translation Loop.json
# src/backend/base/langflow/initial_setup/starter_projects/SEO Keyword Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/SaaS Pricing.json
# src/backend/base/langflow/initial_setup/starter_projects/Search agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Sequential Tasks Agents.json
# src/backend/base/langflow/initial_setup/starter_projects/Simple Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Social Media Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Text Sentiment Analysis.json
# src/backend/base/langflow/initial_setup/starter_projects/Travel Planning Agents.json
# src/backend/base/langflow/initial_setup/starter_projects/Twitter Thread Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/Vector Store RAG.json
# src/backend/base/langflow/initial_setup/starter_projects/Youtube Analysis.json
* fix: handle InvalidRequestError during session rollback
* update projects
* ⚡️ Speed up method `LangFuseTracer.end` by 141% in PR #11114 (`feat/langchain-1.0`) (#11682)
Optimize LangFuseTracer.end
The optimized code achieves a **140% speedup** (8.23ms → 3.42ms) through two complementary optimizations:
## 1. Fast-path for Common Primitives in `serialize()`
**What changed:** Added an early-exit check that returns immutable primitives (`str`, `int`, `float`, `bool`) directly when no truncation or special handling is needed:
```python
if max_length is None and max_items is None and not to_str:
if isinstance(obj, (str, int, float, bool)):
return obj
```
**Why it's faster:**
- The profiler shows `_serialize_dispatcher()` consumed **81.5% of runtime** in the original code (40.4ms out of 49.6ms)
- This optimization reduced dispatcher calls from **8,040 to 1,013** (~87% reduction), as primitives now bypass the expensive pattern-matching dispatcher entirely
- The fast-path check itself is extremely cheap: just two quick conditionals and an `isinstance()` check against a tuple of built-in types
**When it helps:** This optimization is particularly effective for workloads with many primitive values in dictionaries and lists—which is exactly what the tracing use case provides (metadata dicts with strings, numbers, booleans).
## 2. Eliminate Redundant Serialization in `LangFuseTracer.end()`
**What changed:** Serialize `inputs`, `outputs`, and `metadata` once each, then reuse the results:
```python
inputs_ser = serialize(inputs)
outputs_ser = serialize(outputs)
metadata_ser = serialize(metadata) if metadata else None
```
**Why it's faster:**
- The original code called `serialize()` **6 times total** (3 for `.update()` + 3 for `.update_trace()`)
- The optimized version calls it **3 times**, then passes the cached results
- Profiler shows the time spent in `serialize()` calls dropped from **72.2ms to 31.2ms** (~57% reduction)
- This is pure elimination of redundant work—the same dictionaries were being serialized twice with identical results
**Impact on workloads:** The `test_end_multiple_iterations_calls_end_each_time` test (500 iterations) demonstrates this matters in hot paths. If `LangFuseTracer.end()` is called frequently during flow execution, avoiding duplicate serialization provides compounding benefits.
## Combined Effect
Both optimizations target the serialization bottleneck from different angles:
- The fast-path reduces the cost of *each* serialize call by ~75% for primitive-heavy data
- The caching reduces the *number* of serialize calls by 50%
Together, they deliver the observed 140% speedup, with the optimization being especially effective for the common case of metadata dictionaries containing mostly primitive types.
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
* refactor: reorder imports and simplify serialization logic for primitives
* [autofix.ci] apply automated fixes
* fix: update google dependency version to 0.4.0 in component_index.json
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix: pin z3-solver<4.15.7 to restore Linux wheels for Docker build
z3-solver 4.15.7 dropped manylinux wheels, causing the Docker build to
fail when trying to compile from source. Temporary pin until codeflash
is removed.
* [autofix.ci] apply automated fixes
* fix: update google dependency version to 0.4.0
* [autofix.ci] apply automated fixes
* fix: update langchain_core version to 1.2.17 in multiple starter project JSON files
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix: update deprecated langchain imports to langchain_classic for 1.0 compatibility
* fix: align langchain-chroma version in optional chroma dependency group
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* feat: fall back to langchain_classic for pre-1.0 imports in user components
Old flows using removed langchain imports (e.g. langchain.memory,
langchain.schema, langchain.chains) now resolve via langchain_classic
at two levels: module-level for entirely removed modules, and
attribute-level for removed attributes in modules that still exist
in langchain 1.0. New langchain 1.0 imports are never affected since
fallbacks only trigger on import failure.
* urllib parse module import bug
* Update component_index.json
* [autofix.ci] apply automated fixes
* chore: rebuild component index
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Harold Ship <harold.ship@gmail.com>
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: prevent CI injection via unsanitized GitHub context interpolation (#12224)
Pass github.event.pull_request.head.ref through env: instead of
interpolating it directly into run: shell steps. This prevents bash
from evaluating command substitutions embedded in malicious branch names
before input validation runs.
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
* feat: Refactor and Unify the ModelInput Selector Across Components (#12025)
* fix: Fixes Kubernetes deployment crash on runtime_port parsing (#11968) (#11975)
* feat: add runtime port validation for Kubernetes service discovery
* test: add unit tests for runtime port validation in Settings
* fix: improve runtime port validation to handle exceptions and edge cases
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
* fix(frontend): show delete option for default session when it has messages (#11969)
* feat: add documentation link to Guardrails component (#11978)
* feat: add documentation link to Guardrails component
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* feat: traces v0 (#11689) (#11983)
* feat: traces v0
v0 for traces includes:
- filters: status, token usage range and datatime
- accordian rows per trace
Could add:
- more filter options. Ecamples: session_id, trace_id and latency range
* fix: token range
* feat: create sidebar buttons for logs and trace
add sidebar buttons for logs and trace
remove lods canvas control
* fix: fix duplicate trace ID insertion
hopefully fix duplicate trace ID insertion on windows
* fix: update tests and alembic tables for uts
update tests and alembic tables for uts
* chore: add session_id
* chore: allo grouping by session_id and flow_id
* chore: update race input output
* chore: change run name to flow_name - flow_id
was flow_name - trace_id
now flow_name - flow_id
* facelift
* clean up and add testcases
* clean up and add testcases
* merge Alembic detected multiple heads
* [autofix.ci] apply automated fixes
* improve testcases
* remodel files
* chore: address gabriel simple changes
address gabriel simple changes in traces.py and native.py
* clean up and testcases
* chore: address OTel and PG status comments
https://github.com/langflow-ai/langflow/pull/11689#discussion_r2854630438https://github.com/langflow-ai/langflow/pull/11689#discussion_r2854630446
* chore: OTel span naming convention
model name is now set using name = f"{operation} {model_name}" if model_name else operation
* add traces
* feat: use uv sources for CPU-only PyTorch (#11884)
* feat: use uv sources for CPU-only PyTorch
Configure [tool.uv.sources] with pytorch-cpu index to avoid ~6GB CUDA
dependencies in Docker images. This replaces hardcoded wheel URLs with
a cleaner index-based approach.
- Add pytorch-cpu index with explicit = true
- Add torch/torchvision to [tool.uv.sources]
- Add explicit torch/torchvision deps to trigger source override
- Regenerate lockfile without nvidia/cuda/triton packages
- Add required-environments for multi-platform support
* fix: update regex to only replace name in [project] section
The previous regex matched all lines starting with `name = "..."`,
which incorrectly renamed the UV index `pytorch-cpu` to `langflow-nightly`
during nightly builds. This caused `uv lock` to fail with:
"Package torch references an undeclared index: pytorch-cpu"
The new regex specifically targets the name field within the [project]
section only, avoiding unintended replacements in other sections like
[[tool.uv.index]].
* style: fix ruff quote style
* fix: remove required-environments to fix Python 3.13 macOS x86_64 CI
The required-environments setting was causing hard failures when packages
like torch didn't have wheels for specific platform/Python combinations.
Without this setting, uv resolves optimistically and handles missing wheels
gracefully at runtime instead of failing during resolution.
---------
* LE-270: Hydration and Console Log error (#11628)
* LE-270: add fix hydration issues
* LE-270: fix disable field on max token on language model
---------
* test: add wait for selector in mcp server tests (#11883)
* Add wait for selector in mcp server tests
* [autofix.ci] apply automated fixes
* Add more awit for selectors
* [autofix.ci] apply automated fixes
---------
* fix: reduce visual lag in frontend (#11686)
* Reduce lag in frontend by batching react events and reducing minimval visual build time
* Cleanup
* [autofix.ci] apply automated fixes
* add tests and improve code read
* [autofix.ci] apply automated fixes
* Remove debug log
---------
* feat: lazy load imports for language model component (#11737)
* Lazy load imports for language model component
Ensures that only the necessary dependencies are required.
For example, if OpenAI provider is used, it will now only
import langchain_openai, rather than requiring langchain_anthropic,
langchain_ibm, etc.
* Add backwards-compat functions
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Add exception handling
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* comp index
* docs: azure default temperature (#11829)
* change-azure-openai-default-temperature-to-1.0
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* [autofix.ci] apply automated fixes
---------
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix unit test?
* add no-group dev to docker builds
* [autofix.ci] apply automated fixes
---------
* feat: generate requirements.txt from dependencies (#11810)
* Base script to generate requirements
Dymanically picks dependency for LanguageM Comp.
Requires separate change to remove eager loading.
* Lazy load imports for language model component
Ensures that only the necessary dependencies are required.
For example, if OpenAI provider is used, it will now only
import langchain_openai, rather than requiring langchain_anthropic,
langchain_ibm, etc.
* Add backwards-compat functions
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Add exception handling
* Add CLI command to create reqs
* correctly exclude langchain imports
* Add versions to reqs
* dynamically resolve provider imports for language model comp
* Lazy load imports for reqs, some ruff fixes
* Add dynamic resolves for embedding model comp
* Add install hints
* Add missing provider tests; add warnings in reqs script
* Add a few warnings and fix install hint
* update comments add logging
* Package hints, warnings, comments, tests
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Add alias for watsonx
* Fix anthropic for basic prompt, azure mapping
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* ruff
* [autofix.ci] apply automated fixes
* test formatting
* ruff
* [autofix.ci] apply automated fixes
---------
* fix: add handle to file input to be able to receive text (#11825)
* changed base file and file components to support muitiple files and files from messages
* update component index
* update input file component to clear value and show placeholder
* updated starter projects
* [autofix.ci] apply automated fixes
* updated base file, file and video file to share robust file verification method
* updated component index
* updated templates
* fix whitespaces
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* add file upload test for files fed through the handle
* [autofix.ci] apply automated fixes
* added tests and fixed things pointed out by revies
* update component index
* fixed test
* ruff fixes
* Update component_index.json
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* updated component index
* updated component index
* removed handle from file input
* Added functionality to use multiple files on the File Path, and to allow files on the langflow file system.
* [autofix.ci] apply automated fixes
* fixed lfx test
* build component index
---------
* docs: Add AGENTS.md development guide (#11922)
* add AGENTS.md rule to project
* change to agents-example
* remove agents.md
* add example description
* chore: address cris I1 comment
address cris I1 comment
* chore: address cris I5
address cris I5
* chore: address cris I6
address cris I6
* chore: address cris R7
address cris R7
* fix testcase
* chore: address cris R2
address cris R2
* restructure insight page into sidenav
* added header and total run node
* restructing branch
* chore: address gab otel model changes
address gab otel model changes will need no migration tables
* chore: update alembic migration tables
update alembic migration tables after model changes
* add empty state for gropu sessions
* remove invalid mock
* test: update and add backend tests
update and add backend tests
* chore: address backend code rabbit comments
address backend code rabbit comments
* chore: address code rabbit frontend comments
address code rabbit frontend comments
* chore: test_native_tracer minor fix address c1
test_native_tracer minor fix address c1
* chore: address C2 + C3
address C2 + C3
* chore: address H1-H5
address H1-H5
* test: update test_native_tracer
update test_native_tracer
* fixes
* chore: address M2
address m2
* chore: address M1
address M1
* dry changes, factorization
* chore: fix 422 spam and clean comments
fix 422 spam and clean comments
* chore: address M12
address M12
* chore: address M3
address M3
* chore: address M4
address M4
* chore: address M5
address M5
* chore: clean up for M7, M9, M11
clean up for M7, M9, M11
* chore: address L2,L4,L5,L6 + any test
address L2,L4,L5 and L6 + any test
* chore: alembic + comment clean up
alembic + comment clean up
* chore: remove depricated test_traces file
remove depricated test_traces file. test have all been moved to test_traces_api.py
* fix datetime
* chore: fix test_trace_api ge=0 is allowed now
fix test_trace_api ge=0 is allowed now
* chore: remove unused traces cost flow
remove unused traces cost flow
* fix traces test
* fix traces test
* fix traces test
* fix traces test
* fix traces test
* chore: address gabriels otel coment
address gabriels otel coment latest
---------
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com>
* fix(test): Fix superuser timeout test errors by replacing heavy clien… (#11982)
fix(test): Fix superuser timeout test errors by replacing heavy client fixture (#11972)
* fix super user timeout test error
* fix fixture db test
* remove canary test
* [autofix.ci] apply automated fixes
* flaky test
---------
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* refactor(components): Replace eager import with lazy loading in agentics module (#11974)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix: add ondelete=CASCADE to TraceBase.flow_id to match migration (#12002)
* fix: add ondelete=CASCADE to TraceBase.flow_id to match migration
The migration file creates the trace table's flow_id foreign key with
ondelete="CASCADE", but the model was missing this parameter. This
mismatch caused the migration validator to block startup.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: add defensive migration to ensure trace.flow_id has CASCADE
Adds a migration that ensures the trace.flow_id foreign key has
ondelete=CASCADE. While the original migration already creates it
with CASCADE, this provides a safety net for any databases that may
have gotten into an inconsistent state.
* fix: dynamically find FK constraint name in migration
The original migration did not name the FK constraint, so it gets an
auto-generated name that varies by database. This fix queries the
database to find the actual constraint name before dropping it.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: LE-456 - Update ButtonSendWrapper to handle building state and improve button functionality (#12000)
* fix: Update ButtonSendWrapper to handle building state and improve button functionality
* fix(frontend): rename stop button title to avoid Playwright selector conflict
The "Stop building" title caused getByRole('button', { name: 'Stop' })
to match two elements, breaking Playwright tests in shards 19, 20, 22, 25.
Renamed to "Cancel" to avoid the collision with the no-input stop button.
* Fix: pydantic fail because output is list, instead of a dict (#11987)
pydantic fail because output is list, instead of a dict
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
* refactor: Update guardrails icons (#12016)
* Update guardrails.py
Changing the heuristic threshold icons.
The field was using the default icons. I added icons related to the security theme.
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
* feat: Clean up the modelinput unification
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update test_embedding_model_component.py
* [autofix.ci] apply automated fixes
* Revert to main for other files
* More reversions
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Handle first run more elegantly in astra
* [autofix.ci] apply automated fixes
* Fix knowledge embedding dialog (#12071)
* fix: Handle message inputs when ingesting knowledge
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Update test_ingestion.py
* [autofix.ci] apply automated fixes
* fix: Unify the knowledge creation model selector
* Revert tracing
* Update ingestion.py
* Rebuild comp index
* [autofix.ci] apply automated fixes
* Update test_ingestion.py
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update test_ingestion.py
* Update component_index.json
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* Update comp index
* Update test_astradb_base_component.py
* Update Knowledge Ingestion.json
* [autofix.ci] apply automated fixes
* Fix broken tests
* Cleanup from claude
* [autofix.ci] apply automated fixes
* Fix failing tests
* Update test_unified_models.py
* [autofix.ci] apply automated fixes
* Update Nvidia Remix.json
* Refactor ingest
* Rebuild templates and component index
* Fix test
* [autofix.ci] apply automated fixes
* Update component_index.json
* [autofix.ci] apply automated fixes
* test: add update_build_config visibility tests and PR review fixes (#12114)
- Add update_build_config field-visibility tests to LanguageModelComponent,
ToolCallingAgentComponent, and BatchRunComponent covering Ollama, WatsonX,
OpenAI, and no-model-selected cases
- Remove 16 stale @pytest.mark.skip tests from test_agent_component.py
- Wire up validate_model_selection in agent.py for early input validation
- Document AstraDB intentional use of lower-level update_model_options_in_build_config
- Clarify model_kwargs info text to note provider-specific support
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Update embedding_model.py
* fix: address PR review recommendations for feat-unify-models++ (#12116)
- Fix 9 skipped tests in test_batch_run_component.py by replacing model
list with _MockLLM instances, following the existing pattern used by
test_with_config_failure_handling
- Fix test_agent_component.py: set component.model to a valid list before
calling get_agent_requirements() in the three max_tokens tests, since
validate_model_selection now requires a list-format model
- Replace os.environ direct reads in apply_provider_variable_config_to_build_config
with get_all_variables_for_provider() (DB-first, env fallback), and pass
user_id through from handle_model_input_update
- Add deprecated stubs for update_provider_fields_visibility, _update_watsonx_fields,
and _update_ollama_fields in model_config.py with DeprecationWarning pointing
to handle_model_input_update
- Fix typo: "deault" -> "default" in structured_output.py TODO comment
- Add 4 new KnowledgeIngestionComponent tests: new-format model_selection
metadata path, allow_duplicates=True, missing metadata file error, and
_build_embedding_metadata without API key
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Ruff errors
* Update test_ingestion.py
* Update component index
* Test updates
* Update component_index.json
* Update stable_hash_history.json
* Template updates
* Update batch_run.py
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update Youtube Analysis.json
* Fix tests
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Some cleanup and refactoring
* [autofix.ci] apply automated fixes
* Update Nvidia Remix.json
* Update Nvidia Remix.json
* Update unified_models.py
* Coderabbit AI review comments
* Component index update
* [autofix.ci] apply automated fixes
* Template updates
* [autofix.ci] apply automated fixes
* Template update
* [autofix.ci] apply automated fixes
* Review comments addressed
* [autofix.ci] apply automated fixes
* Update component_index.json
* Update stable_hash_history.json
* [autofix.ci] apply automated fixes
* Test updates
* Update test_ingestion.py
* Update test_ingestion.py
* Update test_ingestion.py
* [autofix.ci] apply automated fixes
* More clear tooltip text
* [autofix.ci] apply automated fixes
* Template updates
* Index and templates
* [autofix.ci] apply automated fixes
* Fix lambda build
* Template updates
* Rebuild comp index
* [autofix.ci] apply automated fixes
* Fix templates
* Fix failing test
* Update templates
* Update comp index
* [autofix.ci] apply automated fixes
* API key field in astra db
* Update starter
* Update comp index
* Starter proj update
* Add api key to field order
* Update test_unified_models.py
* Update test_unified_models.py
* [autofix.ci] apply automated fixes
* Update setup.py
* Update setup.py
* Update component_index.json
* [autofix.ci] apply automated fixes
* Return embedding models directly in KB
* [autofix.ci] apply automated fixes
* Update component_index.json
* fix: Refactor the unified models code
* Ruff checks
* Update flow_preparation.py
* [autofix.ci] apply automated fixes
* Update test_language_model_component.py
* fix: prevent overwriting user-selected global variables in provider c… (#12217)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* fix: prevent overwriting user-selected global variables in provider config
Previously, the apply_provider_variable_config_to_build_config function would
automatically overwrite field values with environment variable keys whenever
an env var was present, even if the user had already selected a different
global variable.
This fix adds a check to only auto-set the environment variable if:
- The field is currently empty, OR
- The field is not already configured to load from the database
This preserves user selections while still providing automatic configuration
for new/empty fields.
Added comprehensive unit tests to verify:
- Auto-setting env vars for empty fields
- Preserving user-selected global variables
- Overwriting hardcoded values (expected behavior)
- Skipping when env var is not set
- Applying component metadata correctly
* [autofix.ci] apply automated fixes
* style: use dictionary comprehension instead of for-loop
Fixed PERF403 Ruff style warning by replacing for-loop with dictionary
comprehension in update_projects_components_with_latest_component_versions
* chore: retrigger CI build
* test: improve test coverage and clarity for provider config
- Renamed test_apply_provider_config_overwrites_hardcoded_value to
test_apply_provider_config_replaces_hardcoded_with_env_var for clarity
- Added test_apply_provider_config_idempotent_when_already_set to document
idempotent behavior when value already matches env var key
- Removed sensitive value from debug log message to prevent potential
exposure of API keys or credentials
These changes improve test coverage by documenting the no-op scenario
and enhance security by avoiding logging of potentially sensitive data.
* chore: retrigger CI build
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* Update build_config.py
* [autofix.ci] apply automated fixes
* Update build_config.py
* Fix tests
* fix: Dropdown issue with field population
* Update test_unified_models.py
* Clean up key config
* [autofix.ci] apply automated fixes
* fix tests
* Fix tests
* fix: Update tests
* Update tests
* Update test_tool_calling_agent.py
* Update test_unified_models.py
* Update test_tool_calling_agent.py
* Update tests
* Google AI generative embeddings fixes
* [autofix.ci] apply automated fixes
* Merge release branch
* Template update
* Merge release branch
* [autofix.ci] apply automated fixes
* Update openai_constants.py
* Update openai_constants.py
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
Co-authored-by: keval shah <kevalvirat@gmail.com>
Co-authored-by: Antônio Alexandre Borges Lima <104531655+AntonioABLima@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com>
Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
Co-authored-by: Lucas Democh <ldgoularte@gmail.com>
Co-authored-by: Steve Haertel <stevehaertel@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
* fix: Wait for dynamic model fetch in Nvidia (#12229)
* fix: Wait for dynamic model fetch in Nvidia
* [autofix.ci] apply automated fixes
* Create test_nvidia_component.py
* Update test_nvidia_component.py
* [autofix.ci] apply automated fixes
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* Update test_nvidia_component.py
* Update component_index.json
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: protect image downloads by flow ownership (#12234)
* fix: protect image downloads by flow ownership
* test: add clarifying comments for image access review
---------
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
* feat: Add Windows Playwright test fixes to RC (#12265)
* feat: Add Windows Playwright tests to nightly builds
- Add windows-latest to typescript_test.yml runner options
- Add shell: bash to all script steps for cross-platform compatibility
- Split Playwright installation into OS-aware steps (Linux uses --with-deps, Windows/macOS/self-hosted don't)
- Fix artifact naming with OS prefix to prevent conflicts: blob-report-${{ runner.os }}-${{ matrix.shardIndex }}
- Split frontend-tests into separate Linux and Windows jobs in nightly_build.yml
- Add ref parameter to all test jobs to checkout code from release branch
- Add resolve-release-branch to needs dependencies
- Update Slack notifications to handle both Linux and Windows test results
- Windows tests are non-blocking (not checked in release-nightly-build condition)
- Update .secrets.baseline with new line number (263 -> 347) for LANGFLOW_ENG_SLACK_WEBHOOK_URL
Fixes LE-566
* fix: Use contains() for self-hosted runner detection
- Replace exact string equality (==, !=) with contains() for substring matching
- Fixes issue when inputs.runs-on is array format: '["self-hosted", "linux", "ARM64", ...]'
- Ensures self-hosted Linux runners correctly skip --with-deps flag
Addresses CodeRabbit feedback on PR #12264
* fix: Sanitize folder names for CodeQL (#12263)
* fix(deps): pin tar-fs to >=2.1.4 to fix symlink following vulnerability (#12078)
Adds override for tar-fs in package.json to ensure versions prior to
2.1.4 are never resolved. Addresses CVE in tar-fs <2.1.4 (PVR0686558)
where symlink validation bypass was possible with a crafted tarball.
* fix: Rebuild the embedding model in the nv template (#12275)
* fix: support ZIP file upload for flows and projects endpoints (#12253)
* feat: support ZIP file upload for flows and projects endpoints
Add ZIP upload support to both /flows/upload/ and /projects/upload/
endpoints, enabling round-trip download-then-upload workflows. Extract
shared ZIP parsing logic into a dedicated utility with zip bomb
protections (entry count and file size limits). Fix batch flow name
deduplication to avoid infinite loops and DB collisions. Add tests for
ZIP upload, empty ZIP rejection, and download-upload round-trip.
* fix: add type annotation to satisfy mypy union narrowing
* fix: address PR review for ZIP upload (#12253)
- Add BadZipFile handling in extract_flows_from_zip for defense-in-depth
- Wrap blocking Z…
* add OpenDsStar 1.0.20 dependency and update uv.lock
Add OpenDsStar==1.0.20 to both backend and lfx pyproject.toml.
Regenerate uv.lock to include OpenDsStar and its transitive deps.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix ruff lint errors in codeagents, file ingestion tests, and inspect script
* fix lfx tests: mock subprocess.Popen instead of subprocess.run
The FileDescriptionGenerator uses subprocess.Popen, not subprocess.run.
Tests were mocking the wrong function, causing real subprocess execution
that failed with ModuleNotFoundError for OpenDsStar in isolated lfx env.
* [autofix.ci] apply automated fixes
* fix lfx test mock data to match expected subprocess output format
Subprocess returns {"results": [...], "failed": [], "total": N},
not a bare list. Tests now use _wrap_results() helper.
* bump OpenDsStar to 1.0.21 to fix litellm dependency conflict
* fix tools metadata merge to preserve user-edited descriptions, remove orphan submodule
- Preserve user-edited description and display_description in tools
metadata merge (fixes edit-tools Playwright test)
- Remove orphaned OpenDsStar-fork submodule entry that caused CI
post-cleanup warnings
* smart tools description merge: update from code unless user customized
Compare old description to display_description — if they match, the user
never edited it and we update from code (picking up new Output.info
descriptions). If they differ, the user customized it and we preserve
their edit.
* [autofix.ci] apply automated fixes
* address CodeRabbit review: fix 4 major and 9 minor issues
Major fixes:
- Chroma: only suggest persist_directory when not in server mode
- FileContentRetriever: new input overwrites stale persisted data
- FileContentRetriever: preserve file_path on returned Message
- FileDescriptionGenerator: drain pipes in non-select fallback to prevent deadlock
Minor fixes:
- inspect_chroma_db: align docstring with actual 2000-char truncation
- test_file_content_retriever: fix empty/misplaced test functions
- test_file_description_generator: align duplicate paths test with actual behavior
- format.ts: fix NODE prefix regex pattern
- open_ds_star_agent: use get_running_loop(), remove duplicate cast import
- codeact_agent_smolagents: document timeout behavior, simplify import fallback
* bump OpenDsStar to 1.0.22
* Write File: make agent-friendly with tool_mode inputs, format control, and flexible input types
- Add tool_mode to input field (DataFrameInput) so agents can pass content directly
- Add file_format tool_mode input so agents can choose output format (csv, json, etc.)
- Handle raw pandas DataFrames from code agents (auto-wrap to Langflow DataFrame)
- Handle string inputs from chat agents (JSON → DataFrame, plain text → Message)
- Improve descriptions for agent discoverability
- Bump OpenDsStar to 1.0.24 (async tool invocation fix)
- Add 9 new tests covering all new features
* Write File: hide format dropdowns in tool mode, improve descriptions
- Hide local_format/aws_format/gdrive_format when tool_mode is active
- Agent uses unified file_format input instead
- Shorten field descriptions to pass ruff line length checks
* fix: add file_format field to News Aggregator starter project
The SaveToFile component now has a file_format input — update the
starter project template so field_order matches the component definition.
Update secrets baseline hash for shifted line.
* [autofix.ci] apply automated fixes
* rename Structured Data Agent starter project to Structured Data Analysis Agent
* fix: regenerate starter projects to include file_format field in SaveToFile node
The SaveToFileComponent added a new file_format input but the starter
project JSONs were not regenerated, causing field_order mismatches in
both Structured Data Analysis Agent and News Aggregator starter projects.
* feat: add html as a supported file format in SaveToFileComponent
Add html to LOCAL_MESSAGE_FORMAT_CHOICES, GDRIVE_FORMAT_CHOICES, and
the _save_message handler (reuses the txt write path). AWS already
supported html. Update the file_format tool-mode description to
include html.
* test: add test for saving Message as html format
* fix: regenerate starter projects after adding html format to SaveToFileComponent
* [autofix.ci] apply automated fixes
* fix(tests): skip assistant panel e2e test when OPENAI_API_KEY is missing
Without a configured model provider the assistant panel renders a
"no models" state instead of the input textarea and model selector,
causing the test to fail in CI environments without API keys.
* fix: add file_format field to SaveToFile in starter projects
The autofix bot incorrectly removed the file_format entry. Re-add it
to both field_order and template in News Aggregator and Structured
Data Analysis Agent starter projects. Update secrets baseline.
* fix: rebuild component index to include file_format in SaveToFile
The CI autofix kept removing file_format from starter projects because
the prebuilt component_index.json lacked the field. The update script
loads components from this index in production mode, so it regenerated
starter projects without file_format. Rebuild the index so it includes
the file_format input, which makes the autofix a no-op.
* fix: hide format dropdowns in SaveToFile when used as tool
The storage_location handler in update_build_config was showing format
dropdowns regardless of tool mode. Now it checks tools_metadata to
detect tool mode and keeps format dropdowns hidden, since the agent
uses the unified file_format string input instead.
* fix: remove debug script and replace stderr debug logging with proper logger
Remove scripts/inspect_chroma_db.py (debugging script). Replace all
_dbg() stderr writes in FileDescriptionGeneratorComponent with
standard logger calls at appropriate levels (debug/warning/info).
* fix: avoid double agent execution in OpenDsStarAgentRunnable.astream
astream was calling ainvoke after fully consuming the stream, which
re-executed the entire graph. Use the last streamed chunk as the final
result instead.
* fix: address PR review feedback from ogabrielluiz
- Mirror positional-to-kwargs mapping in async tool path (component_tool.py)
- Add final_answer guard to timeout suppression (codeact_agent_smolagents.py)
- Extract _build_graph_input() helper to deduplicate state dict x4 (open_ds_star_agent.py)
- Split ImportError from other exceptions in set_main_event_loop (open_ds_star_agent.py)
- Add sys.platform != 'win32' guard for select() on pipes (file_description_generator.py)
- Move ChromaError import before try block (chroma.py)
- Add logger.warning to __deepcopy__ fallback paths (component.py)
- Change accumulate_usage from additive to max-based merge (token_usage.py)
- Add async positional-arg test and update token usage tests
* increase opendsstar to 1.0.25 to fix pydantic dependency conflict
* vector stores: remove tool_mode=True for as_dataframe output
* increase OpenDsStar to 1.0.26
---------
Co-authored-by: Gal-Bloch <gal.bloch@ibm.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
Co-authored-by: Antônio Alexandre Borges Lima <104531655+AntonioABLima@users.noreply.github.com>
Co-authored-by: DeyLak <DeyLak@users.noreply.github.com>
Co-authored-by: Adam Aghili <Adam.Aghili@ibm.com>
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Florian Schüller <schuellerf@users.noreply.github.com>
Co-authored-by: Steve Haertel <stevehaertel@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
Co-authored-by: Mateusz Szewczyk <139469471+MateuszOssGit@users.noreply.github.com>
Co-authored-by: Debojit Kaushik <Kaushik.debojit@gmail.com>
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>
Co-authored-by: vjgit96 <vijay.katuri@ibm.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
Co-authored-by: Harold Ship <harold.ship@gmail.com>
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
Co-authored-by: keval shah <kevalvirat@gmail.com>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com>
Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com>
Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com>
Co-authored-by: Lucas Democh <ldgoularte@gmail.com>
Co-authored-by: Anderson Filho <115162146+andifilhohub@users.noreply.github.com>
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: himavarshagoutham <himavarshajan17@gmail.com>
Co-authored-by: Viktor Avelino <viktor.avelino@gmail.com>
Co-authored-by: Hamza Rashid <hzarashid@gmail.com>
Co-authored-by: Rodrigo Nader <rodrigonader@MacBook-Pro-de-Rodrigo.local>
Co-authored-by: ogabrielluiz <gabriel@langflow.org>
Co-authored-by: Tarcio <rodriguestarcio.adv@gmail.com>
Co-authored-by: Carlos Coelho <80289056+carlosrcoelho@users.noreply.github.com>
Co-authored-by: Alfio Gliozzo <54385608+gliozzo@users.noreply.github.com>
Co-authored-by: Alfio Gliozzo <gliozzo@us.ibm.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: manav2000 <manav2000@users.noreply.github.com>
Co-authored-by: Deon Sanchez <69873175+deon-sanchez@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: DAVID BOAZ <DAVIDBO@il.ibm.com>
Co-authored-by: Arek Mateusiak <severfire@users.noreply.github.com>
Co-authored-by: DevByteAI <abud6673@gmail.com>
Co-authored-by: Alice Reis <alicemarianareis@gmail.com>
Co-authored-by: Alex Kuligowski <alekuligowski@gmail.com>
Co-authored-by: a-effort <35465683+a-effort@users.noreply.github.com>
* fix: skip missing server file when delete_after_processing=True to avoid race condition (fixes#8887)
* [autofix.ci] apply automated fixes
* test(lfx): cover file_path branch for missing-file ValueError
Switch the missing-file ValueError test to set self.component.file_path
(via a Data object) instead of self.component.path, so the file_path
decision branch in load_files_base is exercised when
delete_server_file_after_processing=False and silent_errors=False.
* fix: cache parsed result so concurrent outputs don't drop server-file data
Previously the race-condition fix in load_files_base() returned an empty
list when a concurrent output call had already deleted the server file.
That avoided the spurious ValueError but silently dropped data — a File
component with multiple connected outputs could "succeed" while a
downstream node received an empty Data wrapper.
Cache the parsed Data list on the component instance after the first
successful processing, and reuse it when a subsequent call discovers the
file is gone. Each connected output now sees the same parsed content
instead of empty data, while still preserving the deletion semantics on
the source file.
Updates the regression test to assert the cached result is returned
(equal to the first call's output) rather than locking in the empty-list
behavior.
* fix: serialize load_files_base with per-instance lock and key cache by paths
The previous race-condition fix had two remaining gaps flagged in review:
1. It was not concurrency-safe. Two threads could both pass
_validate_and_resolve_paths before either deleted the source file;
the loser would then drop to empty data inside _filter_and_mark_files
when the file vanished mid-flight.
2. The post-hoc recovery ("if validation returned no files and any cache
exists, return it") was too broad — a legitimate empty input on a
later call could pick up a stale cached result for a different
server-file state.
Address both:
- Acquire a per-instance threading.Lock around the entire
validate/process/delete cycle so concurrent output methods on the
same component are serialized. The first caller does the work; the
others observe the cached parsed result.
- Cache the parsed Data list keyed by (local paths, server file paths,
markdown flag) so different inputs and different processing variants
don't share entries. Empty results are not cached, so a legitimate
empty input cannot return stale data.
- Set an explicit _validate_skipped_due_to_delete_race flag inside
_validate_and_resolve_paths so recovery only activates when validation
actually skipped a missing delete-after-processing file. Recovery
reuses any cached result for the same source paths (across markdown
variants) — preserving content beats dropping it when the file is
gone.
Adds a concurrency test that drives two threads through load_files_base
simultaneously and asserts both observe the same parsed data, plus a
test that an empty input does not return stale cached data.
* fix: eagerly init load_files_base lock in __init__ to close first-entry race
Lazy lock creation inside load_files_base() (hasattr-then-assign) is itself
racy: on the very first concurrent entry, two threads can each find the
attribute missing and create their own threading.Lock, bypassing
serialization for the exact race this PR is fixing.
Move the lock creation into __init__ so every instance has a single Lock
before any output method can run. Removes the lazy-init branch from
load_files_base().
---------
Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix(security): prevent arbitrary file read via public flow build endpoint (GHSA-rcjh-r59h-gq37)
The unauthenticated `POST /api/v1/build_public_tmp/{flow_id}/flow`
endpoint accepted a `files` array whose paths were resolved by
`LocalStorageService` and read off disk before being attached to the
LLM prompt. A path like `/etc/hosts` was split into `flow_id="/etc"`
and `file_name="hosts"`. `data_dir / "/etc"` collapsed to `/etc`
because joining an absolute segment resets the prefix, and the
containment check was anchored at that attacker-controlled folder
so it passed. The file contents were then echoed back through the
chat reply.
Fix layers:
1. Boundary validation in `build_public_tmp`: each `files` entry
must match `{source_flow_id}/{basename}` with no traversal,
backslash, or null bytes. Foreign flow_ids are rejected with
HTTP 400.
2. `LocalStorageService._validated_path` now also rejects
`flow_id` values containing path separators, traversal, or
null bytes, and anchors the containment check at the data
directory rather than the per-flow folder.
Adds regression tests for both layers covering `/etc/hosts`,
traversal, foreign-namespace and null-byte payloads, plus an
acceptance test that legitimate `{flow_id}/{filename}` references
still work for AUTO_LOGIN playground uploads.
* add identifier validation
---------
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
