mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-24 06:42:21 +08:00
d63f606159d76fa9e24eb6900a41a547cfcef2bd
941 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 80b105a923 |
Merge release-1.11.0 (via feat/v2-workflows-agui) into feat/v2-workflows-bg-default
Port the 6 background-execution settings and background_backend_is_scaled into 1.11.0's RuntimeSettings group mixin; union the BACKGROUND_EXECUTION_SERVICE / TELEMETRY_WRITER_SERVICE additions in schema.py and deps.py. |
|||
| fb64e45e75 | chore: bump version to 1.11.0 | |||
| 23a4884d83 |
test(background-execution): rename hard_proof marker to real_services
The hard_proof marker name was a vibe word that said nothing about what the tests need. Rename it to real_services everywhere: the pytest marker registration, the *_hard_proof.py test files, the Makefile target (real_services_tests), the -m selector in migration-validation.yml, and the CI job. real_services says what these tests require: real Postgres + Redis + worker subprocesses. (integration was already taken for the external-API suite under tests/integration.) |
|||
| 0d9f9112ef |
perf(telemetry): batched off-pool writer for transactions + vertex_builds (#13126)
* perf(telemetry): batched off-pool writer for transactions + vertex_builds
Adds TelemetryWriterService that buffers transaction and vertex_build rows
in memory and drains them in batched INSERTs via a dedicated AsyncEngine
(pool_size=1 for SQLite, 2 for Postgres, max_overflow=0). Retention is
amortized in a 60s sweeper instead of running on every insert.
Producers (log_transaction, log_vertex_build) enqueue instead of opening
a DB session, so telemetry traffic no longer competes with the
request-handling pool. Falls back to the legacy direct-write path via
LANGFLOW_TELEMETRY_WRITER_ENABLED=false.
Durability: in-flight rows spill to a diskcache.Deque per PID on shutdown
and are restored on next startup; orphan PID directories left by crashed
workers are adopted.
* perf(telemetry): tighten error handling and add coverage
Review feedback from pr-review-toolkit + silent-failure-hunter:
- Retention sweep snapshots the dirty-flow sets before commit and only
clears them after it lands; a crashed sweep no longer drops the flows on
the floor, so per-flow caps cannot drift unboundedly.
- Producer fall-through is no longer silent. transaction_service and
log_vertex_build each log a one-shot WARNING when telemetry_writer_enabled
is True but the writer is not running.
- Lifespan startup failure now logs ERROR instead of WARNING — if the user
opted into the writer and it didn't come up, that's an error.
- Shutdown drain timeout no longer suppressed silently: logs a WARNING with
pending row count + a hint to raise telemetry_writer_shutdown_drain_s.
- Writer's flush loop catches asyncio.CancelledError separately and
re-prepends the in-flight batch to the buffer so teardown's disk spill
catches it.
- After 6 consecutive batch failures the writer emits a loud ERROR with
buffer depths so operators see sustained data-loss risk.
Added tests:
- test_sanitization_survives_writer_round_trip
- test_retention_failure_preserves_dirty_flows
- test_in_flight_batch_returned_on_cancel
* perf(telemetry): address copilot review
- _restore_from_disk + _adopt_orphan_outboxes now route through _enqueue so a
large disk-spilled or orphan outbox can't bypass telemetry_writer_max_queue
and OOM the process. Oldest rows are dropped and counted via the existing
dropped_transactions / dropped_vertex_builds counters.
- chmod 0o700 the outbox root + per-PID directory so sanitized-but-still-
sensitive payloads aren't exposed cross-user on multi-tenant hosts.
Suppressed on platforms where chmod is a no-op (Windows).
- Added test_adopt_orphan_outboxes_honors_max_queue.
The remaining two copilot notes (private API access to DatabaseService and
diskcache.Cache.close) were also flagged by the in-tree review; tracking
separately. The "diskcache not declared" note is a false positive — the
dependency is at src/backend/base/pyproject.toml:76.
* perf(telemetry): address coderabbit review
- Sweeper hands off dirty sets via capture-and-clear so concurrent
flushes during a retention pass aren't wiped by the post-commit
subtract; failure path restores the snapshot.
- Stress README uses a concrete Postgres DSN matching the docker
example instead of an unset env var.
- Test PID-probe loops bounded via a shared helper with pytest.fail
fallback.
* perf(telemetry): swap diskcache outbox for stdlib sqlite (CVE-2025-69872)
Replaces the diskcache.Deque-backed spill outbox with a stdlib sqlite3
outbox (WAL mode, JSON payloads in TEXT). diskcache 5.6.3 has
CVE-2025-69872 (pickle deserialization RCE for an attacker with write
access to the cache dir), no fixed version released, and was never
declared as a dependency — import failed on Python 3.10.
The replacement is encapsulated in a small _Outbox helper that owns the
connection, schema, JSON codec, and lifecycle. The codec uses tagged
wrappers for datetime and UUID so SQLAlchemy's typed columns accept
restored payloads on the way back out.
Hardening informed by a survey of OTel collector, Fluent Bit, Vector,
Prometheus remote_write, Datadog agent, Logstash, and Filebeat:
- Per-PID outbox dirs now stamp an owner.json (hostname + Linux boot_id
or time()-monotonic() proxy). Adoption only proceeds when host+boot
match; cross-host or pre-owner-file dirs are logged and skipped so a
recycled PID after a container restart cannot pull in a stranger's
spill data.
- PRAGMA synchronous=FULL on the spill connection so the shutdown
commit hits the platter (NORMAL only fsyncs on WAL checkpoint, which
may never run if the process exits immediately after commit).
- Spill honors telemetry_writer_max_queue with drop-oldest, matching
the producer-side overflow policy so a backlogged buffer at shutdown
can neither stall teardown nor fill disk.
- append_all encodes payloads up front and only clears the deque after
the transaction commits — no more partial-drain on mid-flight SQLite
failure. drain collapses to a single DELETE FROM outbox after the
SELECT.
- Exception handlers at the spill/restore/adopt boundaries narrowed to
(sqlite3.Error, OSError) so genuinely unexpected exceptions
propagate rather than being silently logged.
Tests cover: cross-host orphan skipped, missing-owner orphan skipped,
spill cap drops oldest, and a realistic UUID+datetime payload
surviving spill → restore → SQLAlchemy INSERT.
* [autofix.ci] apply automated fixes
* fix(telemetry): name wait_for inner tasks so pyleak can filter
Integration tests using @pyleak_marker were flagging an inner asyncio
task created by the telemetry writer's `wait_for(Event.wait(), ...)`
loops. The wrapper task is auto-named (Task-N) and lands mid-await
across the test boundary, so pyleak counts it as leaked. The writer
itself shuts down cleanly via teardown; the "leak" is a pattern
mismatch with pyleak's per-test snapshot model, not a real lifecycle
bug.
Wrap Event.wait() in an explicitly named task (`telemetry-writer-tick`,
`telemetry-writer-backoff`, `telemetry-sweeper-tick`) via a small
_wait_or_shutdown helper, and extend pyleak_marker with a task_name_filter
that excludes `telemetry-*` from leak detection. CI was green on earlier
commits in this branch only because the diskcache import error
prevented the writer from starting at all — fixing the import surfaced
this latent pyleak collision.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* test(telemetry): disable writer in tests so reads see writes synchronously
* perf(telemetry): age out cross-host orphan outboxes on shared volumes
A pod on a shared PV (NFS/RWX) cannot adopt rows from a dead pod on a
different host because the dead pod's hostname doesn't match. Without a
janitor those directories accumulate forever. Add an owner-file mtime
heartbeat from the sweeper plus a prune pass that deletes cross-host
outboxes whose owner file hasn't been touched within
telemetry_writer_orphan_max_age_s (default 1h). Same-host orphans still
flow through the existing adoption path.
* perf(telemetry): add byte-aware flush + drop strategy
Today the writer bounds memory by row count only, so a worker logging fat
vertex_build artifacts can hold tens of MB per row and silently dwarf the
configured max_queue. Add a size_strategy switch ('count' | 'bytes' |
'either', default 'count' for parity) with two byte thresholds:
- batch_size_bytes (256KB) caps per-flush INSERT size when bytes apply
- max_queue_bytes (200MB) drops oldest by bytes when bytes apply
Parallel sizes deques mirror the payload buffers so accounting stays
consistent across drain, spill, restore, and the cancel/retry rollback.
Single rows above the byte budget are still emitted (no row is refused);
operators get dropped_*_bytes counters alongside dropped_*.
* test(telemetry): tighten byte-strategy assertions and cover end-to-end paths
Address gaps in the byte-aware strategy tests:
- pin exact dropped counts and remaining buffer state for the drop-by-bytes
case (was 'greater than zero')
- compute exact drain count from encoded size (was a 1-4 range)
- round-trip bytes through spill+restore to verify size deque is rebuilt
- round-trip bytes through orphan adoption for the same reason
- exercise the sweeper loop end-to-end to confirm heartbeat + prune are
wired in (previously only unit-tested in isolation)
Clarify in the settings docs that the byte caps measure encoded JSON size,
not Python in-memory footprint.
* ref: updates to telemetry writer PR (#13294)
* fix(telemetry-writer): harden error handling and add missing test coverage
Critical:
- teardown() now re-raises CancelledError after awaiting the writer task so the
asyncio cancellation chain propagates correctly on lifespan task kill
- suppress(OSError) on owner file write replaced with explicit error log so
operators know when disk-spilled rows will not be recoverable on restart
High / important:
- Escalation threshold check changed from == to >= so the error log fires on
every failure past the threshold, not just the 6th
- Dead `except OSError` on time.time() - time.monotonic() changed to
`except Exception` since time functions cannot raise OSError
- Removed redundant `from uuid import UUID` inside _run_retention_pass (already
imported at module top)
- Orphan directory cleanup: replaced blanket suppress(OSError) with per-child
suppress so ENOTEMPTY on an individual rmdir doesn't abort the whole loop and
leave the parent directory leaking silently; outer failure now logs at debug
Tests (3 new):
- test_retention_sweep_caps_vertex_builds_per_vertex: inserts 8 builds for a
single vertex with max_per_vertex=3 so the per-vertex DELETE subquery
actually executes (previous test used 8 distinct vertex IDs, bypassing it)
- test_either_strategy_trips_on_bytes_first: verifies bytes can be the first
trigger under 'either' strategy (previous test only covered the count-first path)
- test_writer_retries_on_batch_failure: injects 2 flush failures then success;
confirms failed_batches increments, rows are preserved in the buffer, and
flushed_rows reflects the final successful write
* ci: add stress-tests job to nightly build pipeline
Wires the stress-tests workflow into nightly so telemetry write stress
tests run automatically. Also gates release-nightly-build and
slack-notification on stress-tests result so a stress regression blocks
the nightly release and surfaces in Slack.
* ci: run stress tests in nightly without blocking release
Stress tests are informational for now — a failure is visible in the
workflow run and Slack but does not gate the nightly release or build.
* fix(telemetry-writer): address PR review on cancelled teardown and nightly stress tests
- Add the stress-tests.yml reusable workflow (was untracked, so the
nightly stress-tests job could never resolve)
- Notify Slack on stress-tests failure (add to slack-notification gate
and FAILED_JOB detection as non-blocking)
- Run sweeper cancel, disk spill, and engine dispose in a finally so a
cancelled teardown() still persists the in-memory buffer
- Add tests for the cancelled-teardown spill path and the >= escalation
threshold; drop the misleading wait-loop in the retry test
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
|
|||
| 1ab6251164 |
feat: sync langflow and lfx versions (#13176)
* feat(lfx): synchronize LFX onto Langflow major.minor version line (Phase 1) - Bump src/lfx/pyproject.toml from 0.5.0 to 1.10.0 - Tighten langflow-base lfx pin: ~=0.5.0 → ~=1.10.0 - Fix release.yml pre-release boundary from next-major to next-minor (<X.(Y+1).dev0) - Add minor-parity soft check to scripts/release-lfx.sh - Extend make patch to sync lfx version alongside langflow/base/frontend - Document LFX compatibility contract in RELEASE.md Contract: LFX X.Y.N is compatible with any Flow from Langflow X.Y.M. * feat(lfx/upgrade): add compatibility checker (Phase 2) * fix(lfx/upgrade): fix registry_code guard and types order-sensitivity in checker * feat(lfx/upgrade): implement safe-upgrade applier (Phase 2) * feat(lfx): add lfx upgrade command (Phase 2) * feat(lfx/run): add --upgrade-flow option (Phase 2) * feat(lfx/serve): add --upgrade-flow option (Phase 2) * update lfx pyproject version * [autofix.ci] apply automated fixes * test(lfx/upgrade): add v1.9.0 starter flow fixtures for upgrade integration tests * test(lfx/upgrade): add v1.9.0 starter flow fixtures and real-flow integration tests * [autofix.ci] apply automated fixes * fix(lfx/upgrade): use alias-aware registry lookup so renamed components are not falsely blocked * fix(lfx/upgrade): handle outer flow envelope and file-path inputs in upgrade checks * fix(lfx/upgrade): fail-fast on upgrade_flow errors; add regression tests for Bugs 1-3 * chore(tests): remove bug-number labels from test comments * [autofix.ci] apply automated fixes * fix(lfx/upgrade): address PR review comments - Reject .py files early in serve --upgrade-flow with a clear error message - Preserve outer flow envelope metadata (name, description, etc.) when lfx upgrade --write rewrites a file - Move Mapping import under TYPE_CHECKING to satisfy Ruff TC003 - Assert fixture flows and registry are non-empty so parametrized tests cannot pass vacuously - Remove unused capsys arg and replace print with sys.stderr.write (Ruff T201/ARG001) - Add upgrade_flow param to run command docstring * fix(lfx/upgrade): preserve envelope on run, apply nested upgrades (#13200) * fix(lfx/upgrade): preserve envelope on run, apply nested upgrades Three follow-ups on top of the upgrade tooling: - run --upgrade-flow: re-attach the inner graph to the outer envelope before handing the flow to aload_flow_from_json. Previously the upgrade path unwrapped {"data": ...} and passed the inner dict to the loader, which raised KeyError: 'data'. Adds happy-path tests for envelope and flat file inputs. - applier: recurse into one level of nested flow nodes (node.data.node.flow.data.nodes), matching the checker. Without this, outdated_safe nodes inside grouped components were reported but never written. Adds a regression test. - upgrade --write test: assert the envelope is preserved (name, description, endpoint_name, etc.) instead of the previous unwrapped-output expectation, so the test matches the actual fix in this PR. - Drive-by ruff cleanup: D417 docstring, SIM103, SIM108, E501, RUF059. * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix(lfx/upgrade): fix loader KeyError, outer-envelope checker gap, and nested-node applier Three bugs from ogabrielluiz's PR review: 1. run/base.py loader KeyError: file-path branch unwrapped the outer envelope for the checker but passed the inner dict to aload_flow_from_json, which does flow_graph["data"] unconditionally. Re-wrap as {"data": flow_dict} before the loader call. Applied consistently to all three input paths (--flow-json, --stdin, file-path). 2. run/base.py zero-node checker for --flow-json/--stdin: those paths stored the raw parsed JSON without unwrapping, so a caller passing an exported flow {"name":..., "data":{...}} caused the checker to see zero nodes and silently pass. Now unwrap with raw.get("data", raw) matching the file-path branch. 3. applier.py nested flows never upgraded: the early `continue` on top-level nodes not in safe_ids prevented the nested-node loop from running. Restructured so the nested check fires unconditionally for every top-level node, mirroring checker.py:160-165. * fix(lfx/run): unify outer-envelope unwrap across all --upgrade-flow input paths File-path reads now call raw.get("data", raw) matching --flow-json and --stdin, so the upgrade checker always sees the inner graph. This also removes the has_envelope/re-wrap machinery that was double-wrapping outer-envelope files when passing to aload_flow_from_json. Fix two test assertions that described the old double-wrapped shape. * fix(lfx/upgrade): inject registry into upgrade_command; fix Makefile lfx pin regex - upgrade_command now accepts an optional registry parameter so tests can pass the dict directly instead of mocking load_registry_from_index - Makefile sed regex broadened from \"lfx~=.*\" to match both ~= and >= forms so make patch works after release.yml rewrites the pin - Echo label changed to LFX (synced) to clarify the variable is the shared Langflow version, not a separate LFX-specific value * fix(make/patch): fix langflow-base sed regex and validation grep The dependency in pyproject.toml is "langflow-base[complete]>=X.Y.Z", not the "langflow-base==X" form the original regex expected, so make patch silently left the pin unchanged and the validation step always failed. The sed pattern now matches any extras/operator combination and rewrites to the canonical [complete]>= form; the grep uses -F so the [complete] brackets are treated literally. * test(lfx): add patch regex tests and symmetric safe-mode envelope tests test_patch_regexes.py — 15 tests covering the Python regexes embedded in the Makefile patch target. Exercises all three substitutions (langflow-base pin, lfx pin, version field) against every realistic pin format including the >=X.Y.Z,<dev0 form that release.yml writes. Would have caught the langflow-base==.* vs [complete]>= mismatch before manual testing. test_base.py — two new TestUpgradeFlowOption tests: test_upgrade_flow_safe_envelope_inline_json_loads_successfully test_upgrade_flow_safe_envelope_stdin_loads_successfully Symmetric to the existing file-path envelope test; verifies that --flow-json and --stdin with an outer-envelope flow also pass {"data": inner} to the loader after safe upgrades, not a double-wrapped {"data": outer_envelope}. * fix(lfx/upgrade): address review: compat checker, shared gate, fail-fast registry Checker correctness: - _outputs_are_compatible: drop cosmetic display_name from the breaking check; treat widened output types as safe (flow types must be a subset of registry types), only narrowing breaks downstream edges. - _input_types_contained: stop flagging widened input_types as breaking; keep narrowing as the only breaking case; fix misleading comments. - check_flow_compatibility now recurses fully into nested grouped components (symmetric with the applier) and accepts a pre-built registry lookup. CLI/run/serve: - New lfx.upgrade.cli_gate (UpgradeFlowMode enum, UpgradeFlowError, apply_upgrade_gate) shared by run_flow and serve_command so the two --upgrade-flow paths can't diverge. - run_flow: extract _materialize_flow_dict and route gating through the shared helper. - run/serve --upgrade-flow options typed as UpgradeFlowMode (check|safe choices). - lfx upgrade: load_registry_from_index fails fast when the bundled registry is empty/missing instead of silently marking every node blocked; ASCII-only report output; new --strict flag; build the registry lookup once and reuse it. Docs/tests: - RELEASE.md: migration note for the lfx 0.5.0 -> 1.10.0 version jump. - Regression tests for the checker fixes, the shared gate, fail-fast registry, --strict, and serve --upgrade-flow parity. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix(lfx): consolidate flow-envelope handling and fix version-ceiling regex Extract the outer-envelope unwrap/rewrap logic (previously hand-rolled as raw.get("data", raw) with subtly different rules across serve, run, and upgrade) into a single lfx.utils.flow_envelope module with split/merge helpers. This fixes a serve bug where an enveloped flow had its inner graph written bare to the temp file, making the loader's flow_graph["data"] raise KeyError. Also fix release.yml's major.minor extraction: the greedy sed grabbed the upper bound from a range pin (>=1.10.0,<1.11.dev0), drifting the version ceiling up one minor each release cycle. Anchor to the first version instead. * fix(lfx): upgrade-flow gate reads bundled component index, not empty cache The --upgrade-flow=check|safe gate on lfx run and lfx serve rejected every flow as 'blocked'. Both call sites passed component_cache.all_types_dict to apply_upgrade_gate, but that cache is populated lazily after services start, so at gate time it is empty -- an empty registry classifies every node as blocked. The standalone lfx upgrade command was unaffected because it reads the bundled _assets/component_index.json instead. Make the gate own registry loading: apply_upgrade_gate now defaults all_types_dict to None and loads the bundled index (the same source lfx upgrade uses) via a new _load_bundled_registry helper, raising UpgradeFlowError on a missing/empty index so a broken install fails loudly instead of silently blocking every component. Both call sites pass mode= and let the gate load the registry. Existing gate tests mocked component_cache with a populated registry, which is exactly what hid the bug; repoint them at the new _load_bundled_registry seam and add regression tests that do not mock the registry, including an end-to-end run_flow check against a real clean v1.9.0 starter flow. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai> |
|||
| d696cae935 |
feat(api/v2): durable background execution service (store + default backend)
Turns v2 mode:background into a durable, in-API background execution service behind a BackgroundExecutionService facade. Adds the store layer (result/error columns, job_events durable milestone log, execution_signals control, heartbeat/lease, 3 migrations), the default backend (bounded executor, runner, in-memory live bus, liveness-aware single-flight orphan sweep), the v2 endpoint rewiring, and the real-instance test harness. Needs no new infra; works on the SQLite single-process install. The redis-scaled worker backend is stacked on top in a follow-up PR. |
|||
| 71cd5ce376 |
feat: improve lfx serve production-readiness (#13121)
* docs: add design spec for lfx serve multi-flow startup and dynamic upload
* docs: add implementation plan for lfx serve multi-flow startup and dynamic upload
* feat(serve): add FlowRegistry for mutable in-process flow management
* feat(serve): add flow_id_from_content for content-based flow ID generation
* fix(test): update streaming tests to use FlowRegistry API
* test(serve): add upload endpoint tests
* feat(serve): add build_registry_from_directory and build_registry_from_paths helpers
* refactor(serve): remove dead _analyze_graph_structure and _generate_dynamic_run_description helpers
* feat(serve): accept directory and multiple file paths at startup; wire registry to serve_command
* [autofix.ci] apply automated fixes
* fix(serve): move traceback import to module level; remove planning docs from branch
* [autofix.ci] apply automated fixes
* fix(serve): deepcopy graph in stream_flow; use json.dumps for error stream payload
* test(serve): verify stream setup error payload is valid JSON with special characters
* [autofix.ci] apply automated fixes
* fix(serve): use load_flow_from_json directly in _load_graph_and_meta for clear error messages
* docs(serve): document multi-worker and persistence limitations on FlowRegistry
* feat(serve): allow starting server with no flows; accept full Langflow flow JSON on upload
- Remove the requirement for a flow path/JSON/stdin at startup; lfx serve
now starts with an empty FlowRegistry when no input is provided, ready
to accept flows via POST /flows/upload/
- UploadFlowRequest now accepts the full Langflow export JSON directly
(extra="allow", id field added) so callers can do -d @flow.json without
wrapping; load_flow_from_json receives the full dict and the flow's own
stable id is used when present
* fix(serve): harden upload endpoint and remove unused verbose_print param
- Pass body.model_dump(exclude={"replace"}) to load_flow_from_json instead of
body.model_dump(); keeps the top-level "data" key the function expects while
excluding the internal replace flag
- Set graph.flow_id after upload so it matches the registry key
- Return 409 Conflict on duplicate flow ID; add replace=true field to allow
explicit overwrite
- Remove unused verbose_print param from _load_graph_and_meta and update callers
- Clarify directory scan is top-level only (non-recursive) in CLI help text
- Add test_upload_full_export_as_body regression test: sends a complete Langflow
export JSON as the request body (curl -d @flow.json pattern) where body.data
has no nested "data" key — would have caught the KeyError('data') regression
* fix(serve): address PR review comments
- Restore .py script support: _load_graph_and_meta now dispatches to
load_graph_from_script for .py paths; CLI validation accepts .json and .py;
help text updated in both _running_commands.py and commands.py
- Fix flow ID collision in build_registry_from_paths: compute a shared common
root from all supplied paths so same-named files in different directories
get distinct UUIDv5 IDs instead of silently overwriting each other
- Harden FlowRegistry.add(): add overwrite=False parameter that raises
ValueError on duplicate IDs; startup paths keep default (collision = error),
upload endpoint passes overwrite=body.replace
- Add regression test: test_same_filename_in_different_dirs_gets_distinct_ids
- Update test_duplicate_add_replaces_graph to reflect new overwrite semantics
* test(serve): add unit tests for .py script path in serve command
- test_load_graph_and_meta_dispatches_to_script_loader_for_py: verifies
_load_graph_and_meta calls load_graph_from_script (not load_flow_from_json)
for .py paths, and that meta.title and relative_path are set correctly
- test_serve_command_accepts_py_file: end-to-end check that lfx serve my.py
reaches the server (exit_code == 0)
- test_serve_command_rejects_unsupported_extension: .txt and other unknown
extensions must exit non-zero with a helpful message
* [autofix.ci] apply automated fixes
* Add --no-env-fallback option to support scoped request_vars
* [autofix.ci] apply automated fixes
* feat(serve): filesystem-backed flow registry for multi-worker sharing
Adds a FlowStore abstraction with two implementations:
- NullFlowStore (default): in-memory only, existing behavior unchanged
- FilesystemFlowStore: persists each flow as {id}.json; writes are
atomic (tmp→rename); reads and deletes are TOCTOU-safe; path
traversal guard rejects unsafe flow IDs
FlowRegistry now delegates persistence to FlowStore:
- add(raw_json=...) writes to store when raw JSON is available
- get() falls through to store on cache miss, reconstructs + caches
- list_metas() merges in-memory and store-side entries
- remove() propagates deletion to store
- warm_from_store() pre-warms the in-memory cache at startup
New --flow-dir option on lfx serve activates the filesystem store.
All uvicorn workers pointing at the same directory serve the same
flows — use /tmp/lfx-flows for single-pod sharing or a PVC mount
for cross-pod sharing. Raw JSON is plumbed through _load_graph_and_meta
and both builder functions so startup-loaded .json flows are persisted
automatically; .py flows remain in-memory only.
* feat(serve): add --workers option for multi-worker uvicorn support
Adds --workers N to lfx serve, enabling multiple uvicorn processes to
share flows via --flow-dir.
serve_command is restructured from @syncify async to plain sync: all
async registry-building is extracted into _build_serve_registry() and
called via asyncio.run(), then uvicorn.run() handles the server.
For workers > 1, uvicorn requires an import string rather than an app
object. The factory create_serve_app() in serve_app.py reads
LFX_SERVE_FLOW_DIR and LFX_SERVE_NO_ENV_FALLBACK from env, pre-warms
from the shared FilesystemFlowStore, and returns a ready FastAPI app.
The parent sets those env vars unconditionally before dispatching.
For workers == 1, the pre-built app object is used directly (no change
to existing single-worker behavior). A warning is emitted when
--workers > 1 is used without --flow-dir since each worker would have
an isolated in-memory registry.
The startup panel flow count is now accurate for single-worker +
--flow-dir by calling warm_from_store() before printing.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* fix(serve): remove no_args_is_help, drop verbose_print param, harden _safe_path
- Remove no_args_is_help=True from serve command so lfx serve with no
arguments starts with an empty registry instead of showing help
- Remove unused verbose_print parameter from create_multi_serve_app and
update all callers (14 call sites across tests)
- Wrap _safe_path's relative_to() check in try/except for a clear error
message; restructure delete() to use try/except/else (TRY300)
- Add TYPE_CHECKING guard for FlowStore annotation in commands.py; wrap
long --workers help string; restore FBT003 noqa on no_env_fallback
- Fix test_env_fallback_skipped_when_flag_true to allow logger's getenv
calls while still asserting the credential variable is never looked up
- Fix ARG002/D205/RUF059 lint issues in test files
* [autofix.ci] apply automated fixes
* fix(serve): resolve merge conflicts and clean up test section comments
* ruff - test updates
* fix(serve): cross-worker delete propagation and HTTP 500 for execution failures
Cross-worker stale cache (multi-worker DELETE bug):
- Add FlowStore.is_persistent to distinguish real stores from NullFlowStore
- Track _store_sourced in FlowRegistry for flows written to a persistent store
- get() re-reads the store for store-sourced flows to detect cross-worker deletes;
evicts and returns None if the file is gone
- list_metas() skips store-sourced flows whose store key is no longer present
- Add _evict() helper to clean up _flows, _store_meta_cache, _store_keys, _store_sourced
HTTP status for execution failures:
- POST /flows/{id}/run now returns HTTP 500 (not 200) when execution raises or
when the flow returns success=false; body is still RunResponse for debug detail
- Updated OpenAPI responses declaration: 500 now maps to RunResponse model
* fix(serve): harden multi-worker registry correctness and safety
- Add FlowAlreadyRegisteredError (ValueError subclass) so upload_flow
catches only the specific race-condition exception rather than any
ValueError from unrelated internals.
- Add _SERVE_ENV_PREFIX constant; rebuild LFX_SERVE_* env var names from
it and add try/finally cleanup so those vars are always deleted after
uvicorn.run() returns in multi-worker mode.
- Reject .py startup files when --workers > 1: Python graphs cannot be
serialized to the filesystem store so workers would silently start with
empty registries for those flows.
- Remove warm_from_store() from build_registry_from_paths/directory; the
builders no longer pre-warm to avoid double-loading when the startup dir
and flow-dir overlap. serve_command still warms for single-worker mode.
- Fix FlowRegistry.__len__ to delegate to list_metas() so store-only
flows (not yet cache-loaded) are counted; this makes the startup panel
and /health flow_count accurate for multi-worker + pre-placed files.
- Fix FlowRegistry.remove() to delete both the primary store key file and
any alternate-keyed file (UUID or stem), ensuring cross-worker DELETE
propagation: after a delete, every worker's next request to that flow
triggers the per-request stale check, reads None from the store, and
returns 404 regardless of which key its cache entry uses.
- Document the per-request stale check in FlowRegistry docstring with
latency guidance for network volumes vs. local tmpfs.
- Add tests for all of the above, including a cross-worker DELETE
propagation scenario validated by a live 2-worker integration test.
* fix(serve): load startup flows in every uvicorn worker via LFX_SERVE_STARTUP_PATHS
Previously, lfx serve flow.json --workers 4 silently dropped startup flows:
the parent loaded them into a NullFlowStore registry that workers never
inherited. Each worker called create_serve_app() fresh and started empty,
returning flow_count: 0 from /health and 404 for the startup flow.
Fix: before calling uvicorn.run() in multi-worker mode, set
LFX_SERVE_STARTUP_PATHS to the JSON-encoded list of resolved startup file
paths (or the inline-JSON temp file path). create_serve_app() reads this env
var and re-loads the startup flows independently in each worker process using
a ThreadPoolExecutor to avoid RuntimeError from asyncio.run() being called
inside uvicorn's already-running event loop.
Startup flows loaded this way live in each worker's in-memory registry.
Uploaded flows still require --flow-dir for cross-worker sharing (existing
warning preserved). The LFX_SERVE_* env vars are cleaned up after
uvicorn.run() returns via the existing prefix-sweep finally block.
Tests added:
- test_serve_command_sets_startup_paths_env_for_multi_worker: verifies
LFX_SERVE_STARTUP_PATHS is set before uvicorn.run() and cleaned up after
- TestCreateServeAppFactory: unit tests for create_serve_app() in all three
modes (empty start, with startup paths, empty paths list)
* fix(serve): fix 3 worker/store consistency issues
Issue 1 — redundant store writes with --flow-dir + startup files + workers > 1:
When flow_dir is set the parent already persisted startup flows to the store.
create_serve_app() was re-loading and re-writing them in every worker.
Fix: create_serve_app() now skips LFX_SERVE_STARTUP_PATHS when flow_dir is
set and relies solely on warm_from_store(). Similarly, serve_command now only
populates LFX_SERVE_STARTUP_PATHS when flow_dir is NOT set.
Issue 2 — .py restriction was too broad:
The error blocked .py + --workers > 1 even without --flow-dir, where each
worker can safely re-execute the .py via LFX_SERVE_STARTUP_PATHS. The error
now only fires when --flow-dir is also set (workers would skip startup paths
and warm_from_store, missing the .py flow entirely since it can't be stored).
Issue 3 — parent built FastAPI app for multi-worker then discarded it:
create_multi_serve_app() was called before the workers > 1 / else branch,
building a full FastAPI app that was never served in multi-worker mode.
Moved into the else branch so it's only built for single-worker.
Tests:
- test_create_serve_app_with_flow_dir_skips_startup_paths_uses_store:
asserts build_registry_from_paths NOT called when flow_dir is set;
flow comes from store via warm_from_store instead.
- test_create_serve_app_without_flow_dir_loads_startup_paths_from_files:
asserts flow IS loaded when no flow_dir.
- test_serve_command_does_not_set_startup_paths_when_flow_dir_set:
asserts LFX_SERVE_STARTUP_PATHS is [] when flow_dir is set.
- test_serve_command_allows_py_with_multiple_workers_no_flow_dir:
asserts .py + workers>1 + no flow_dir is allowed and path appears in env.
* test(serve): remove duplicate test and fix mislabeled test name
- Remove test_create_serve_app_loads_startup_paths: functionally identical
to test_create_serve_app_without_flow_dir_loads_startup_paths_from_files
(same setup, same assertions, added in a prior session).
- Rename test_verify_api_key_header_takes_precedence to
test_verify_api_key_query_param_takes_precedence: the test passes the
correct key as the query_param argument and the wrong key as header_param,
then asserts the query param value is returned — query param has precedence,
not header. The old name was the opposite of what the test actually checks.
* [autofix.ci] apply automated fixes
* fix(serve): fix concurrent request isolation and startup UX
Three bug fixes discovered via live QA:
1. Component.__deepcopy__ produced shallow copies of _outputs_map,
_edges, and _components, causing concurrent requests to share the
same Output objects across graph copies. Output.cache=True by
default: the first request's output.value was served to all
subsequent concurrent requests, producing identical responses for
different inputs. Fixed by deep-copying all three fields with memo.
2. POST /flows/upload/ without an explicit id used uuid5(content),
which is deterministic — two uploads of any flow with the same
nodes/edges structure always generated the same UUID and the second
always returned 409. Fixed by using uuid4() (random).
3. Startup CLI emitted emoji and exclamation-point decoration in the
serve banner and verbose output. Replaced with plain text.
* [autofix.ci] apply automated fixes
* fix(serve): address PR review comments
- Move 409 conflict check before load_flow_from_json+prepare so callers
with an explicit id don't pay the parse cost for a flow that will be
rejected anyway
- Remove unreachable FlowAlreadyRegisteredError catch: no await exists
between registry.get() and registry.add(), so asyncio cannot
interleave another request in that window
- Update FlowMeta.id and UploadFlowResponse.id field descriptions from
stale "Deterministic UUIDv5" to plain "Flow identifier (UUID)"
* test(serve): fix upload duplicate/replace tests after uuid4 change
Both tests relied on uuid5(content) determinism to generate the same
flow ID from identical data. With uuid4, each upload gets a fresh ID,
so duplicate detection was never triggered. Tests now supply an
explicit id field to exercise the 409/replace paths directly.
* fix(serve): address code review suggestions
- Fix test_write_is_atomic to glob for any *.tmp files rather than
checking a hardcoded filename that could never exist
- Remove incorrect noqa: B904 from validate_id_is_uuid (not in an
except block); use `from None` instead
- Add TTL cache (1 s) for FlowStore.list_ids() in FlowRegistry to
avoid one filesystem glob per /health request; mutations invalidate
immediately, warm_from_store and delete-scan bypass it
- Document the UUID/stem dual-key alias invariant on _flows
- Add test verifying create_serve_app() propagates load errors from
the ThreadPoolExecutor startup path (fail fast, not silent empty registry)
* [autofix.ci] apply automated fixes
* fix: lfx serve no_env_fallback, deepcopy cycle, cross-worker delete (#13297)
fix: address concurrency and isolation issues in lfx serve
- Re-apply no_env_fallback to the per-request graph copy in the run and
stream endpoints; deepcopy() drops graph.context so the flag was lost.
- Register the copy in memo before recursive deepcopy calls in
Component.__deepcopy__ so reference cycles are preserved.
- Check the store directly in list_metas() so cross-worker deletes are
reflected immediately instead of after the TTL cache window.
- Remove the redundant upload-conflict test (already covered by
test_upload_duplicate_without_replace_returns_409).
Also mark test secrets with allowlist pragmas and fix two docstring
lint warnings in the touched test file.
* [autofix.ci] apply automated fixes
* feat(lfx): WXO/TRM request-scoped variables for lfx serve (child of #13121) (#13327)
* feat(lfx): WXO/TRM request-scoped variables for lfx serve global_vars
Align lfx serve with the TRM/WXO credential contract on top of #13121:
- Parse LANGFLOW_REQUEST_VARIABLES from global_vars payloads
- ContextVar-backed lookup in VariableService (no os.environ mutation)
- Activate request scope during execute_graph_with_capture
Child of add-workers-and-upload-lfx (PR #13121).
* [autofix.ci] apply automated fixes
* fix(lfx): harden request-scoped variable resolution + add isolation t… (#13347)
fix(lfx): harden request-scoped variable resolution + add isolation tests
Addresses PR review of WXO/TRM request-scoped variables for lfx serve:
- service: honor no_env_fallback in VariableService so model/KB credential
resolution no longer falls back to os.environ under --no-env-fallback;
correct the get_variable precedence docstring (5-step interleaved order)
- request_scope: add a per-request no_env_fallback ContextVar; use precise
Token type; document the None-vs-empty scope contract
- common: activate/reset the no_env_fallback scope from graph.context
- runtime_variables: log (instead of silently swallowing) a malformed or
non-object LANGFLOW_REQUEST_VARIABLES blob; anchor the override comment to
the TRM contract
Tests:
- add request-scope concurrency isolation + no-env-fallback suite
- add execute_graph_with_capture wiring test (graph.context -> ContextVars)
- give TestGraphExecution mock graphs a real context, fixing 11 tests that
broke once the helper began reading graph.context
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
* Update use of stem alias tracking to ensure proper deletes
* fix(lfx): enforce no_env_fallback across all credential resolution paths
Closes credential-isolation gaps in lfx serve and hardens request-scoped
variable handling:
- VariableService: skip the process-wide LANGFLOW_REQUEST_VARIABLES env read
when env fallback is disabled (previously leaked process credentials for an
empty-global_vars request despite --no-env-fallback).
- Unified models: gate os.environ reads in get_api_key_for_provider and the
connection-config fallbacks in instantiation.py (WatsonX URL/project-id,
Ollama base-url, OpenRouter headers) on the no-env-fallback flag.
- run_until_complete: propagate the caller's contextvars into the worker
thread so request-scoped vars and the flag survive the thread hop.
- Request variables: serialize structured (dict/list) values as valid JSON and
drop nulls instead of producing lossy Python reprs or a truthy "None".
- FlowRegistry: extract the duplicated store-alias scan shared by add/remove.
Adds tests for request-scope isolation, credential/env gating, and contextvar
propagation.
* fix(lfx): close no_env_fallback credential leaks and harden serve isolation
- credentials: honor no_env_fallback in get_all_variables_for_provider so served
flows no longer leak process env (was defeating the _env_if_allowed guards in
instantiation.py via pre-filled provider_vars)
- loading: apply the no_env_fallback contract to table load_from_db column
resolution (sibling of load_from_env_vars, previously still read os.getenv)
- variable service: resolve request-scoped vars (exact name + x-langflow-global-var
alias) before env fallback so a caller's per-request credential is never shadowed
by a same-named process env var
- runtime_variables: flatten the LANGFLOW_REQUEST_VARIABLES blob into graph context
for parity between the ContextVar and load_from_db resolution paths
- flow_store: clean up the partial temp file when an atomic write fails
- commands: on multi-worker shutdown remove only the serve env vars we set, not the
whole LFX_SERVE_* namespace
- components: add check_component_env_writes lint banning os.environ writes; stop the
mem0 component from writing the OpenAI key into the process environment
* fix(lfx): contain corrupt flow-store files so one bad file can't crash worker startup
FilesystemFlowStore.read() only caught FileNotFoundError, so a corrupted/partial
{id}.json in a shared store (PVC corruption, an external writer killed mid-write)
raised JSONDecodeError. warm_from_store() calls get() for every store id at startup
with no per-file guard, so a single bad file would crash every worker's startup and
500 GET /flows.
- read(): catch JSONDecodeError/OSError/UnicodeDecodeError, log, return None (treat
as absent) so warm_from_store/list_metas/get skip it and other flows keep serving.
- warm_from_store(): skip+log a flow that fails to reconstruct (corrupt JSON, or valid
JSON referencing a component not available in this build) instead of aborting startup.
- tests: corrupt-JSON read returns None; warm_from_store skips unloadable flows.
* fix test -- request wins over env now
* fix test. mock graph context correctly
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com>
|
|||
| f2d49e5098 | fix: tidy up docling extra reqs | |||
| 5120b7705a |
fix: Move Docling components into bundle (#13442)
* feat: move docling components to bundle * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * chore: wake CI * fix: trim docling chunking extra * Update test_endpoints.py * Update test_pilot_docling_upgrade.py * fix: show friendly component label in build banner for bundle nodes Extension-bundle components have namespaced node ids of the form ext:<bundle>:<ClassName>@<slot>-<uuid>. The flow-build status banner rendered that raw id, which overflowed the fixed-width (530px) container and collided with the elapsed-time counter. Collapse namespaced ids to <ComponentName>-<uuid> (matching the built-in ComponentName-UUID label) via a new getRunningNodeLabel helper; built-in node ids are returned unchanged. Covered by unit tests. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
|||
| e65ced99cc | Merge release-1.9.6 into release-1.10.0 | |||
| d70757976c |
chore: bump toml and package.json versions
bump toml and package.json versions |
|||
| 3e694305f9 |
fix: separate docling chunking dependencies (#13411)
* separate docling chunking * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix nitpicks * [autofix.ci] apply automated fixes * remove langchain docling dependency * add docling image description as an optional dep * [autofix.ci] apply automated fixes * fix line too long * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| dbe482c800 |
fix(ci): lockstep langflow-nightly and langflow-base-nightly versions (#13413)
* fix(ci): lockstep langflow-nightly and langflow-base-nightly versions
The nightly pipeline computed each package's .devN number independently
(pypi_nightly_tag.py queried each package's own PyPI history and
incremented separately), while langflow-nightly pins an exact
langflow-base-nightly[complete]==X.Y.Z.devN dependency. Because
langflow-nightly publishes on more nights than langflow-base-nightly,
the two counters drift (live: dev54 vs dev48). Each time base lags a
publish while main succeeds, the newest langflow-nightly pins a base
dev that does not exist yet, so 'uv pip install langflow-nightly'
becomes unsatisfiable and silently falls back to a weeks-old version.
Fix: version the two packages in lockstep. pypi_nightly_tag.py now
derives a single shared dev number from max(dev across BOTH packages'
PyPI histories) + 1 (restricted to the root base_version), so main and
base always get the identical tag and main's exact pin always
references a base version built and published in the same run. The tag
is computed in a single invocation ('both' mode) so the release and
base tags cannot drift across separate calls.
Also hardens tag computation: 404 / network / malformed responses for
either package now contribute nothing instead of crashing, a
base-version bump resets the dev counter cleanly, and non-dev/final
releases never advance it.
Adds scripts/ci/test_pypi_nightly_tag.py (unit tests) and a
ci-scripts-test.yml workflow to run scripts/ci tests on PRs.
* fix(ci): fail closed on non-404 PyPI errors in nightly tag computation
_all_dev_numbers() previously returned [] for ANY failure (network error,
non-404 HTTP status, malformed 200), which is unsafe: _shared_nightly_version()
takes max(dev)+1 across both packages, so a transient lookup failure on the
higher-versioned package silently LOWERS the next tag. E.g. if the langflow-nightly
lookup fails while langflow-base-nightly reports dev48, the script would emit
v1.10.0.dev49 even though langflow-nightly already published through dev54 — the
workflow then force-recreates an old git tag and fails publishing an already-existing
PyPI version.
Fail closed: only a true 404 (package has no releases) contributes []; network
errors, 5xx/non-404 statuses, and malformed 200s now raise so the nightly job aborts
before mutating tags. Adds tests for malformed/5xx/network failures and a direct
regression test for the higher-package-lookup-failure scenario.
* Update scripts/ci/test_pypi_nightly_tag.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update test_pypi_nightly_tag.py
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
|
|||
| 39bb3a12ca |
ci: isolate bundle installs from the core release gate; lazy-import ibm-db (#13402)
* ci: isolate bundle installs from the core release gate; lazy-import ibm-db
Two follow-ups to wiring lfx-ibm (and future native-dep bundles) into the
nightly/release pipeline.
1. cross-platform-test.yml: split the monolithic "Install packages from
wheel" step (in both the stable and experimental jobs) into three:
- core install (SDK/LFX + langflow-base) -- release gate
- per-bundle bundle install -- NON-BLOCKING (continue-on-error), reports
per-bundle pass/fail to the GitHub step summary
- langflow main install -- release gate
A bundle whose dependency has no wheel on a tested platform is now reported
as "failed (non-blocking)" instead of failing the whole nightly/release
publish. Local bundle wheels are still installed before main, so langflow
resolves its bundle deps from the freshly-built wheels rather than PyPI.
NOTE: a bundle that langflow hard-depends on is still pulled (and thus
gated) by the main install; to make a bundle optional on a platform, also
add a PEP 508 marker to its dependency line in langflow's pyproject.toml.
2. lfx-ibm: import ibm_db_dbi lazily inside DB2VS.__init__ (db2vs.py) instead
of at module top level, mirroring db2_vector.build_vector_store. ibm-db
ships no linux/aarch64 wheel, so the bundle now imports on that arch (only
the DB2 vector store is inert there; the watsonx components keep working).
Guard the driver-dependent test modules (test_db2vs, test_db2_vector) with
a module-level skip when ibm_db_dbi is unavailable, matching the existing
watsonx test pattern so they no longer error at collection on linux/aarch64.
Verified locally: 59 DB2 tests pass with ibm-db present; with ibm-db absent
the bundle imports cleanly, DB2 use raises a clean ImportError, and the
guarded test modules skip (9 passed, 2 skipped, 0 errors). actionlint adds no
new shellcheck findings vs HEAD; ruff check/format clean.
* fix: Remove long comments for bundle deps (#13403)
* [autofix.ci] apply automated fixes
* ci: revert bundle-install restructure to blocking; add ibm-db import regression test
Addresses two review findings on the lfx-ibm bundle changes.
1. Revert the cross-platform-test.yml bundle-install restructure. The
non-blocking per-bundle step could let the main install satisfy a hard-dep
bundle from PyPI (the index was not disabled), masking a broken local build,
and it never actually protected hard-dep bundles -- main re-gates them
anyway. Restore the original blocking, install-by-path step (local bundle
wheels install before main, so the gate is reliable and PyPI cannot
substitute). Document the real isolation lever for a genuinely
platform-optional bundle -- a PEP 508 marker on its dependency line in the
root pyproject -- in src/bundles/PORTING.md.
2. The module-level skip hid the importability regression the lazy import was
meant to guard. Add test_optional_dependency.py: it simulates ibm-db's
absence and asserts db2vs + the bundle package import and that DB2 use
raises a clean ImportError -- it runs on every platform, including the
x86_64 CI runners. Replace the module-level skips in test_db2vs.py /
test_db2_vector.py with class-level skipif on the DB2-driver test classes
only, so the helper tests and the module-level imports still run (and assert
importability) where ibm-db is absent.
Verified: with ibm-db present, 112 bundle tests pass. With ibm-db uninstalled
(the real linux/aarch64 case, find_spec -> None), 67 pass / 45 skip / 0 errors
-- the regression tests and helper tests run, only the driver-dependent classes
skip. actionlint reports no new findings; ruff check/format clean.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
|
|||
| f8ada15e2a |
feat(components): Add IBM DB2 Vector Store component (#13237)
* feat(ibm): add DB2 Vector Store component Add comprehensive IBM DB2 Vector Store integration for Langflow. Components Added: - DB2VectorStoreComponent: Main vector store component - DB2VS: LangChain-compatible vector store implementation - DB2 security validators: Input validation and sanitization Features: - Vector similarity search (Similarity, MMR, Similarity Score Threshold) - Secure connection handling with input validation - Metadata filtering for complex data types - Duplicate detection support - Batch document ingestion Test Coverage: - 41 unit tests (38 passing, 3 skipped) - Component integration tests - Security validation tests - DB2VS class tests Documentation: - Comprehensive developer guidelines * refactor(db2): simplify component to match Chroma patterns - Remove complex data ingestion (CSV, DataFrame, dict, Message handling) - Add _add_documents_to_vector_store() method matching Chroma - Add similarity_score_threshold search type - Simplify data ingestion from 220 to 45 lines (80% reduction) - Add 5 new tests (similarity_search, mmr_search, search_with_different_types, duplicate_handling, metadata_filtering) - All 41 tests passing (38 passed, 3 skipped) BREAKING CHANGE: Component now only accepts Data objects for ingestion. * ci: fix CI failures - update component index and starter projects - Regenerate component_index.json with DB2 component - Update all starter projects with new component registry - Fix ruff formatting issues * ci: trigger CI re-run to fix merge issue * ci: add merge conflict resolution for component_index.json in autofix workflow Handle merge conflicts in the Update Component Index job by: - Automatically resolving component_index.json conflicts using --theirs strategy - Failing if non-generated files have conflicts (requires manual resolution) - Completing the merge after resolving generated file conflicts This prevents the workflow from failing when component_index.json has conflicts during PR updates, as this file is auto-generated and can be safely regenerated. * fix: Remove DB2SQLComponent reference from IBM components - Removed DB2SQLComponent from __init__.py as the db2_sql.py module doesn't exist - This fixes the test_all_modules_importable test failure - Fixes CI failure in Unit Tests - Python 3.14 - Group 5 * revert: Remove unrelated starter project JSON changes * revert: Remove unrelated CI workflow and formatting changes * docs: add DB2 Vector Store .mdx documentation following Chroma DB structure * fix: address CodeRabbit review comments - add __init__.py docstring and implement score threshold filtering - Added docstring to src/backend/tests/unit/components/__init__.py to fix Ruff INP001 error - Added FloatInput score_threshold parameter (default 0.5) to DB2 Vector Store component - Implemented threshold filtering in similarity_score_threshold search mode - Filters results to only include documents with relevance scores >= threshold * chore: remove auto-generated and unnecessary files from PR - Remove uv.lock (dependency lock file - auto-generated) - Remove src/frontend/package-lock.json (frontend lock file - auto-generated) - Remove src/lfx/src/lfx/_assets/component_index.json (auto-generated by CI) - Remove .secrets.baseline (security baseline - auto-generated) - Remove starter_projects JSON (auto-generated example) - Remove tweaks_builder.py (unrelated test helper) Per DEVELOPMENT.md guidelines, these files should not be committed by contributors. * chore: restore auto-generated files to keep PR focused on DB2 component * fix(ci): update auto-generated files and fix docs build - Update component index with new IBM DB2 components (363 components, 97 modules) - Update frontend package-lock.json - Fix docs build by removing reference to missing image in bundles-db2.mdx Fixes CI failures: - Update Component Index - Update Starter Projects - Test Docs Build * feat(db2): add SSL/TLS support to DB2 Vector Store component - Add SSL/TLS encryption support for DB2 database connections - Add SSL certificate validation and download functionality - Support local certificate files (.crt, .pem, .cer) and URLs - Add optional certificate password support for encrypted keystores - Implement automatic cleanup of temporary downloaded certificates - Add comprehensive error handling and logging for SSL connections - Update component_index.json with new SSL configuration inputs - Update package-lock.json dependencies Security improvements: - Validate certificate paths and file permissions - Support system default CA certificates (recommended for IBM Cloud DB2) - Redact sensitive information in error messages - Clean up temporary files on connection failure This enhancement enables secure encrypted connections to DB2 databases, which is recommended for production environments. * [autofix.ci] apply automated fixes * refactor(db2): minimize metadata storage in DB2 vector store - Clear metadata before storage to reduce unnecessary data (file_path, filename, etc.) - Return only text content during retrieval for cleaner results - Optimize list comprehension for better performance (PERF401, RET504) * test: remove skipped tests from DB2 vector store test suite - Removed version compatibility tests for versions where component didn't exist - Changed file_names_mapping to return empty list for new component - Overrode base class version tests to prevent skips - Fixed linting issues (hardcoded passwords, nested with statements) - All 40 tests now pass with 0 skipped tests * fix(tests): update DB2 vector store test description format Fixes test_component_metadata assertion to match the actual component description format. The test was failing due to line wrapping differences in the multi-line description string. Fixes CI failure in PR #13163 * [autofix.ci] apply automated fixes * chore: trigger CI re-run to resolve flaky test failures * fix(tests): mark OpenAI-dependent test as api_key_required to prevent CI failures * restructure as new bundle package * [autofix.ci] apply automated fixes * fix(ibm): improve DB2 Vector Store component reliability and validation * fix: implement SSL/TLS toggle functionality for DB2 Vector Store - Add update_build_config method to dynamically show/hide SSL certificate fields - SSL certificate path and password fields now only visible when use_ssl is enabled - Improves UX by hiding irrelevant fields when SSL is disabled - Update component_index.json with new component configuration - All 38 existing tests pass --------- Co-authored-by: Dhruv Chaturvedi <dhruv_insights@mac.lkw-in.ibm.com> Co-authored-by: Dhruv Chaturvedi <dhruv_insights@Dhruvs-MacBook-Pro.local> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: priyanshu-krishnan1 <priyanshu.krishnan1@ibm.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| 8c7fe0ac3a |
chore: bump versions
bump package versions |
|||
| 0a5f25f32b |
chore: port Remove the diskcache dependency
port of https://github.com/langflow-ai/langflow/pull/12953 |
|||
| bf7e494260 | chore: add back docling-core extras | |||
| fca26ff79f | chore: address node and jquery issues | |||
| 7f8bd2d432 | chore: update versions and revert dockerfile npm | |||
| 0ba4c3725d | chore: mass dep update | |||
| ec3aea7ff8 |
fix(extensions): discover editable bundles; clean up ticket refs and reload UX (#13219)
* fix(extensions): discover editable installs via langflow.extensions entry-point
`lfx extension list` was silently dropping bundles installed via `uv pip
install -e` / `pip install -e`. Editable installs surface only
`dist-info/` entries in `dist.files`, so the wheel-shaped scan in
`_distribution_manifest_path` finds no `extension.json`. The bundle
pyproject.toml comment already promised an entry-point fallback for this
case, but it was never implemented in discovery.py.
Add `_distribution_manifest_path_via_entry_points` that resolves the
package via `importlib.util.find_spec` (no module body execution) and
locates the manifest under the resulting package directory. The fallback
runs only when `dist.files` produces no manifest, so wheel installs are
unaffected.
Tests:
- editable distribution with dist-info-only files is discovered
- entry-point pointing at an unimportable module yields no record
- wheel-install path never consults find_spec (guards against
re-importing every bundle package on startup)
* chore(extensions): drop internal ticket refs; relax reload --bundle and --all
Two related changes:
1. Strip internal `LE-NNNN` ticket references from the extension source,
bundles, tests, and public docs. The references were not actionable
outside the project and surfaced in user-visible CLI messages and the
author guide.
2. Relax `lfx extension reload` CLI ergonomics now that local discovery
(`discover_all_extensions`) gives us the install map without needing
an HTTP list endpoint:
- `lfx extension reload <ext_id>`: `--bundle` is now optional; when
omitted, the bundle name is resolved from local discovery. Explicit
`--bundle` still wins for cases where the local install isn't
visible to the running server.
- `lfx extension reload --all`: iterates over every locally-discovered
bundle, POSTs reload to each, renders per-bundle status, and exits 1
if any reload fails. Previously hard-errored as "not yet wired".
- `--all` is mutually exclusive with a positional id / `--bundle`
(exit 2 with a clear message).
- Missing both `extension_id` and `--all` exits 2 pointing at both.
Updated `test_reload_requires_explicit_bundle` (which enforced the old
"--bundle is mandatory" behavior) to cover the new resolution paths.
* docs(bundle-api): changelog entries for editable-install discovery + reload CLI
Satisfies the BUNDLE_API surface-change gate for the editable-install
discovery fallback (entry-point lookup in discovery.py) and the relaxed
`lfx extension reload` CLI (`--bundle` optional, `--all` implemented).
No additional behavior change in this commit -- it only documents the
two changes already shipped in this branch.
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Debojit Kaushik <Kaushik.debojit@gmail.com>
|
|||
| e08080507b |
feat: Add langflow-stepflow package (#12015)
* feat: Add langflow-stepflow package Introduces a new workspace package `src/langflow-stepflow/` that ports the Stepflow integration from the Stepflow repository into the Langflow codebase. The package has two submodules: - `translation/`: translates Langflow flow JSON to Stepflow flow definitions (ported from integrations/langflow/converter/) - `worker/`: a Stepflow worker that executes individual Langflow components via lfx (ported from integrations/langflow/executor/) Entry point: `python -m langflow_stepflow.worker` starts the HTTP worker server for use by a Stepflow orchestrator. No changes to existing Langflow code in this commit. * fix: Address review feedback on langflow-stepflow package - Widen Python version to >=3.10,<3.14 to match langflow-base - Fix import sorting (ruff I001) and unused variable (ruff F841) - Replace sk- prefixed test strings to avoid Gitleaks false positives - Remove placeholder step reference resolution in component_tool (orchestrator resolves these before reaching the worker) - Let exceptions propagate from component_tool_executor instead of returning error dicts that look like successful results - Skip secret/password field defaults in tool input schemas - Use LRU-style bounded cache (128) for compiled components - Use asyncio.to_thread for sync component methods to avoid blocking - Fix mutable NamedTuple default (list→tuple) in PlaceholderGraph - Warn and skip deps with missing field mappings instead of silently falling back to "input" (which overwrites on multiple deps) - Tighten _is_data_list to check __class_name__ == "Data" specifically - Add comment explaining intentional teardown-before-init sequence - Add integration test for example flows * chore: Upgrade to Stepflow SDK 0.12.0 and adapt to lfx 0.3+ renames - Bump stepflow-py and stepflow-orchestrator from >=0.10.0 to >=0.12.0 - Remove Python 3.11 environment markers (SDK now supports 3.10+) - Replace server.run() with gRPC pull transport (run_grpc_worker) matching the upstream stepflow-langflow integration pattern - Update Flow serialization from Pydantic model_dump to msgspec.to_builtins - Remove Pydantic ValueExpr/actual_instance unwrapping (now plain dicts) - Add _langflow_type_name() to map lfx 0.3+ class renames (JSON→Data, Table→DataFrame) back to canonical Langflow type names - Fix DataFrame isinstance checks for lfx Table subclass - Add missing README.md, tests/__init__.py, helpers package - Add skip conditions for missing fixture files * change to lfx schema * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * chore: Align langflow-stepflow ruff config with repo and format - Set line-length = 120 to match root pyproject.toml (was 88, causing conflicts between pre-commit format hook and check hook) - Run ruff format across all source and test files - Add pragma: allowlist secret on test fixtures with fake API keys * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * chore: Rebuild component index * [autofix.ci] apply automated fixes * fix(langflow-stepflow): allow Python 3.14 in requires-python Lift requires-python from <3.14 to <3.15 to match the rest of the workspace and the lockfile. Without this, uv sync on Python 3.14 fails the Unit/LFX/Integration test jobs and the Docker build. * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: ogabrielluiz <gabriel@langflow.org> Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com> |
|||
| cc009c9133 |
feat(lfx): Bundle Separation and LFX Extension Framework (#13043)
* feat(lfx): installed-package + seed-directory discovery for production install (LE-1022) Adds the read-only production install path for Modes A, B, and C of the Bundle Separation iteration. Manifest-shipping pip-installed distributions and seed-directory subdirectories are discovered at server startup and registered as Extensions at @official. * discovery.py: walks importlib.metadata.distributions() + the $LANGFLOW_SEED_DIR / /opt/langflow/bundles seed root; produces DiscoveredExtension records and typed errors for malformed manifests / configured-but-missing seed dirs. * registry.py: ExtensionRegistry service with the immutability invariant for installed and seed entries. Mutation verbs (uninstall, disable, enable, install, update_entry) all raise ExtensionImmutableError carrying the typed installed-extension-immutable / seed-directory-immutable code so the invariant is testable today; the CLI uninstall surface ships in B4. * lfx extension list: read-only inspector with text and JSON output for operators inspecting Mode B/C images. * Errors: four new typed codes (installed-extension-immutable, seed-directory-immutable, seed-directory-not-found, duplicate-extension-id) plus snapshot coverage in tests/unit/extension/test_errors.py. * Tests: 165 extension tests pass, including the LE-1022 acceptance cases -- three pip-installed wheels visible at @official, three seed bundles visible at @official, and the parametrized service-layer immutability check across every mutation verb. * Docs: docs/Deployment/deployment-extensions-production.mdx covers the Dockerfile template, k8s deployment notes, the bundle packaging convention (extension.json shipped via package-data), and troubleshooting for the typed error codes. * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (#12967) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * refactor(lfx): clarify broad except, log malformed bundle.json, expand test docstring - Document why _discovery.import_bundle_module catches BaseException (startup-time loader, must surface bad bundles as typed errors rather than abort). - Add debug-level logging when bundle.json is malformed or non-object so a stale-cache footgun is at least observable. - Expand test_skips_re_imported_class docstring so future maintainers don't accidentally weaken the __module__-equality guard if package-style relative imports get added later. Also drops the stale duplicate-distribution claim from the PR description; that code is correctly deferred to LE-1022 along with /all integration. * feat(lfx): wire Extension System into /all, pathsep-split LANGFLOW_COMPONENTS_PATH, emit duplicate-distribution Closes the four AC gaps the previous reviewer flagged on PR #12967: 1. /all integration: get_and_cache_all_types_dict now also calls a new import_extension_components() that loads installed Extensions via load_installed_extensions, loads inline bundles via discover_inline_bundles, and builds frontend-node templates with extension/bundle/extension_version fields stamped on. Failures are logged and skipped per bundle. 2. LANGFLOW_COMPONENTS_PATH is now split on os.pathsep so multi-entry env vars (e.g. /a:/b on POSIX) produce multiple components-path entries instead of one literal non-existent path. Empty segments and missing paths are skipped. 3. duplicate-distribution is a real producer: load_installed_extensions surfaces a typed warning on the winner LoadResult when two distributions share a canonical name, naming every involved manifest path. 4. Manifest-first precedence runtime wiring: new filter_component_entry_points loads each entry-point and only skips ones that resolve to a Component subclass on a manifest-shipping distribution. plugin_routes.load_plugin_routes now applies it so non-component entry-points (route registrars) keep loading per the AC's 'unaffected' promise. Added the previously-missing AC test for same-distribution component+non-component partition. Also makes _distribution_canonical_name defensive against MagicMock test seams. * fix(lfx): register extension components under namespaced ID; promote duplicate-distribution to error Addresses the latest review of PR #12967: P1: /all integration now keys the cache inner dict by LoadedComponent.namespaced_id (ext:<bundle>:<Class>@<slot>) rather than the bare class name. Templates also carry the namespaced_id as an explicit field so consumers that look at the value (not the key) still see the canonical address. This is the form the LE-1020 migration table will rewrite legacy class-name references to. P2: load_installed_extensions now appends duplicate-distribution to result.errors instead of result.warnings, so LoadResult.ok=False when two distributions share a canonical name. The winner's components still appear in result.components so flows already pinned to them keep working; only the conflict status changes. Updated test to assert errors + ok=False; added explicit assertion that the winner's components are still present. * fix(lfx): installed-distribution discovery accepts pyproject.toml manifest form Closes the latest review finding on PR #12967: the installed-distribution scan only looked for extension.json, ignoring distributions whose manifest lives in [tool.langflow.extension] inside pyproject.toml. The AC explicitly treats both as valid manifest forms. _distribution_manifest_path now: - Returns extension.json immediately when present (preserves precedence matching load_manifest's discovery order). - Falls back to pyproject.toml only when extension.json is absent AND the pyproject's [tool.langflow.extension] section is parseable. Validation reuses load_manifest itself so the rule lives in exactly one place; a stray pyproject.toml without the section is correctly ignored. Tests cover: pyproject-only discovery, pyproject-without-section ignored, extension.json wins on collision, end-to-end pyproject load at @official, and pyproject-form manifest-first entry-point suppression. * refactor(lfx): tighten loader invariants, surface silent skips, harden tests Addresses the latest review feedback on PR #12967: Type-level invariants (_types.py): - LoadedComponent.__post_init__ enforces that @extra components must NOT carry a distribution. The reverse (@official without distribution) is permitted because load_extension is also used for dev-mode loads against a working tree before pip install. - LoadResult docstring documents the partial-success contract: components may be non-empty when errors is non-empty (some files imported, others failed). Callers branching on ok get strict success. Silent-failure fixes (_orchestrator.py + settings/base.py): - inline-path-missing: a non-existent / non-dir LANGFLOW_COMPONENTS_PATH entry now produces a typed warning per skipped path so a typo no longer yields zero diagnostics. Settings-layer skip bumped from debug to warning for the same reason. - bundle-json-invalid: a malformed or non-object bundle.json now surfaces a typed warning instead of silently rewriting the user-declared id/version to derived values under the same bundle name. - no-component-subclass gating uses a call-local counter instead of result.errors so the diagnostic stays accurate when a future caller reuses a LoadResult (multi-bundle / batch wrapper scenarios). Test hardening: - test_re_imported_class_is_skipped_via_module_filter rewritten to actually exercise the __module__-equality check via sys.modules injection; previously passed via module-import-failed (relative-import failure), which would silently weaken if package registration changes. - test_user_declared_path_order_is_preserved: AC #8's multi-path order case (distinct bundles in [path_b, path_a]) was unasserted; added. - test_inline_module_import_failure_attributes_identity: AC #10's identity-on-partial-failure was covered for @official but not @extra; added. - test_uses_real_distributions_by_default tightened to assert ep placement (in kept, not in skipped) instead of exact-list equality, so a future Langflow-shipped manifest doesn't silently flip the assertion. bumped 64 -> 175 passing extension tests; 20 backend integration tests still pass. * fix(lfx): malformed pyproject manifests surface manifest-invalid instead of disappearing Closes the latest review finding on PR #12967: a pyproject.toml with a [tool.langflow.extension] section that has missing/invalid required fields was silently dropped because _pyproject_has_extension_section ran full schema validation via load_manifest and returned False on ValueError/TypeError. That conflated 'no section' with 'section malformed'. Fix: detect section presence only. _pyproject_has_extension_section now calls _read_pyproject_extension (TOML parse + key lookup, no schema check). Behavior: - Section absent or pyproject TOML unparseable -> False (treat as regular non-manifest package). - Section present and is a table (valid OR schema-invalid) -> True. - Section present but is not a table -> True; the author intended to declare an extension and load_extension will surface the typed error. This way a typo'd pyproject Extension produces a typed manifest-invalid LoadResult with extension_id attribution, and manifest-first precedence still suppresses its legacy component entry-points -- matching the 'typed load results on success/failure' contract for the supported pyproject manifest form. Tests: two new cases pin the behavior. test_malformed_pyproject_section_ surfaces_manifest_invalid asserts a typed load-failure result with distribution attribution; the second test pins manifest-first suppression for malformed pyproject distributions. * refactor(lfx): emit inline-path-unreadable, document reload contract, trim rot Closes the remaining nits on PR #12967: - inline-path-unreadable (new typed error code): a configured LANGFLOW_COMPONENTS_PATH entry that raises OSError on iterdir (typically permission-denied) now produces a typed LoadResult error carrying str(exc) instead of silently swallowing the message. - duplicate-inline-bundle hint trimmed: removed forward promise about hard-error-in-a-later-release; same actionability, no expiration. - discover_inline_bundles docstring trimmed from three paragraphs to two sentences (per CLAUDE.md anti-multi-paragraph rule). - _distribution_manifest_path docstring trimmed to one-liner. - installed_extension_roots dropped Used-by caller-narration list; manifest_owning_distributions docstring rewritten to call out the shadow-load risk for direct callers and point to load_installed_ extensions for the typed warning surface. - _discovery.import_bundle_module: replaced 'until that lands in a later milestone' rot with a single line ('Absolute imports only between bundle modules; relative imports unsupported.') AND added the single-load-per-process contract note documenting why LE-1018 reload must scrub registry/sys.modules before re-invoking the loader. - load_extension docstring grew a 'Single-load-per-process contract' block telling direct callers not to rely on this function for refresh. Tests: new test_unreadable_path_emits_inline_path_unreadable pins the OSError -> typed-error path. --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) (#12979) * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) Five-stage reload (parallel staging load -> validate -> swap under write lock -> cleanup -> emit) for installed Bundles in Mode A. In-flight flows keep the pre-swap class via existing references; new flows pick up the post-swap class atomically; concurrent reloads on the same Bundle are rejected with reload-in-progress. Adds: * lfx/extension/registry.py -- BundleRegistry with per-bundle reload-in-progress guard and components_index.json writer * lfx/extension/reload.py -- the five-stage pipeline; events emission is stubbed (TODO LE-1017) so the swap mechanics can ship before the events service lands * loader: optional module_namespace param so Stage 1 lands in __reload_staging__.<id> instead of the live _lfx_ext.* namespace * errors: four new typed reload codes (reload-in-progress, reload-bundle-not-installed, reload-bundle-name-mismatch, reload-source-missing) with branch templates and snapshot tests * HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by the existing get_current_active_user dependency, returns 409 with a typed body for the in-progress collision case * CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client against the dev server with text/json output and proper exit codes (--all is gated until LE-1019 lands the list endpoint) * tests: 16 reload-pipeline tests covering the AC matrix (rename round-trip, broken-bundle isolation, concurrent readers, in-flight flow, double-reload guard, bundle-name mismatch) plus 9 CLI client tests Mode A only. In Mode B/C bundle changes require a Docker image rebuild and the reload path is not exercised. The events emission in Stage 5 is intentionally stubbed -- the LE-1017 ticket will swap the body of _emit_bundle_reload_event in one place without touching the pipeline core. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [autofix.ci] apply automated fixes * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) (#12968) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) The two scaffolding CLIs an Extension author types: - `lfx extension init <target>` writes the basic single-Bundle template (manifest with $schema, README, .gitignore, one Component subclass + a pytest smoke test). AC #1: the generated extension validates clean against LE-1014. AC #2: the generated test file is a valid pytest module that exercises the component's build() method. AC #3: any --template other than 'basic' fails with a typed template-deferred-in-this-milestone error and a non-zero exit. Refuses to scaffold over a non-empty target dir. - `lfx extension dev <target>` validates the local extension, records its absolute path in <config_dir>/extensions/dev_extensions.json, prints reload instructions, and execs `langflow run` (or `python -m langflow` when langflow isn't on PATH). --skip-launch registers without launching (used by tests + external dev-server scripts); --skip-validate lets authors register a known-broken manifest to debug it under the loader. Stack: - Wave 0: error codes (extension-target-exists, extension-target-invalid, local-extension-missing) with format branches and snapshot tests. - Wave 1: lfx.extension.init_template -- pure-data scaffolder, no Typer dependency so the CLI is a thin shell over it. - Wave 1: lfx.extension.dev_registry -- atomic JSON state file under the langflow user-cache dir; helpers for register / list / unregister / load_dev_extensions / dev_extension_component_paths. - Wave 2: lfx.cli._extension_commands gains init/dev subcommands. - Wave 2: langflow.main lifespan hook reads the dev registry after bundle loading and extends components_path with each registered bundle dir, so the existing palette discovery picks up dev extensions. Missing paths surface as local-extension-missing warnings (AC #5) without aborting startup. Tests (84 new, 197 total in tests/unit/extension/): - test_init_template.py: AC scenarios + identifier derivation + deterministic file shape. - test_dev_registry.py: register/list/unregister round-trip, idempotent re-register refreshes timestamp, malformed state file treated as empty, missing-path warning, recovery when path reappears, env-var override precedence. - test_cli.py: AC #1 init->validate, AC #3 deferred templates, --skip-launch registers without launching, --skip-validate short-circuits the pre-flight pass. - test_errors.py: snapshot rows for all three new codes; the every-known-code-has-a-snapshot test enforces parity. LE-1018 (reload) reuses load_dev_extensions; LE-1022 (installed-pkg discovery) shares the components_path extension pattern. AC #4 ("boots Langflow with the extension visible in the palette within 5s") is delivered jointly by `extension dev` (registers + execs) and the lifespan hook (loads on startup). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1016 review feedback Fixes the four HIGH issues + the elevated LOW from the review pass: 1. dev_extension_component_paths now forwards EVERY warning, not just local-extension-missing. Previously a duplicate-component-name (or any future warning code) was silently dropped, hiding real signal from the lifespan hook's logs. 2. Defensive emit when a LoadResult has components but source_path is None. The current loader always sets source_path, but a future hand-built LoadResult could violate that contract; we now surface a typed local-extension-missing error rather than dropping the extension silently. 3. Replaced the fragile ``min(len(parts))`` bundle-root selection with a relative-to-source-path measurement. Handles deep-vs-shallow sibling extensions correctly without depending on absolute path depth. 4. Generated README now documents the langflow/lfx prerequisite under the Develop section so authors know `pytest` requires the lfx environment, not just Python. 5. Forced LANGFLOW_LAZY_LOAD_COMPONENTS=false unconditionally in the `extension dev` exec env (was setdefault, which let a developer's global lazy-loading export silently hide their dev components from the palette and miss AC #4's 5s budget). Plus three MEDIUMs: - sys.modules cleanup in test_generated_test_file_runs_against_generated_component so a later test importing the same dotted path doesn't pick up a stale module from a deleted tmp_path. Component instantiation moved inside the try block because Component.__init__ uses inspect.getsourcefile against self.__class__'s still-live module. - Added regex-drift test that pins the init_template patterns to match manifest.py's so a schema regex change can't quietly produce invalid scaffolded manifests. - Added two new dev_registry tests covering the forward-all-warnings contract and the source_path=None defense. Tests: 201 passing (4 new); ruff check + format clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * fix(lfx): align init template with renamed manifest API After merging feat/extension-loader into this branch, two surfaces fell out of sync with the renamed manifest schema: - init_template generates extension.json with `lfx: {bundle_api: [1]}`, but the validator now requires `lfx: {compat: ["1"]}` per LfxCompat. This made `test_basic_template_validates_clean` fail with manifest-invalid (`lfx.compat: Field required; lfx.bundle_api: Extra inputs are not permitted`). - test_init_template_regexes_match_manifest_schema reads `manifest_mod._BUNDLE_NAME_RE`, but that symbol was promoted to public `BUNDLE_NAME_RE` in |
|||
| 5e9f0c37bc |
Merge branch 'release-1.9.3' into release-1.10.0
# Conflicts: # .secrets.baseline # docs/package.json # pyproject.toml # src/backend/base/langflow/initial_setup/starter_projects/Vector Store RAG.json # src/backend/base/pyproject.toml # src/backend/tests/unit/components/files_and_knowledge/test_retrieval.py # src/frontend/package-lock.json # src/frontend/package.json # src/frontend/src/components/core/parameterRenderComponent/components/modelInputComponent/__tests__/ModelInputComponent.test.tsx # src/frontend/src/components/core/parameterRenderComponent/components/modelInputComponent/index.tsx # src/frontend/src/modals/modelProviderModal/hooks/useProviderConfiguration.ts # src/lfx/pyproject.toml # src/lfx/src/lfx/_assets/component_index.json # src/sdk/pyproject.toml # uv.lock |
|||
| d5ef588e42 | chore: bump versions to 1.9.3 | |||
| a80cf1c649 |
chore: support Python 3.14 (#13085)
* feat: upgrade Docker images to Python 3.14 (experimental)
- Update pyproject.toml: requires-python from <3.14 to <3.15
- Update all 5 Dockerfiles to use Python 3.14:
- Builder stage: python3.12-trixie-slim → python3.14-trixie-slim
- Runtime stage: python:3.12-slim-trixie → python:3.14-slim-trixie
- src/backend/base/pyproject.toml
- docker/build_and_push_base.Dockerfile
- docker/build_and_push.Dockerfile
- docker/build_and_push_backend.Dockerfile
- docker/build_and_push_with_extras.Dockerfile
- docker/build_and_push_ep.Dockerfile
Python 3.14.5 was released on May 10, 2026. Upgrading to the latest
Python version should help reduce CVE vulnerabilities in Docker images.
This is an experimental change to test Python 3.14 compatibility:
- Docker images will use Python 3.14
- CI/CD workflows still test on Python 3.10-3.13
- Nightly build will validate if dependencies work with 3.14
- Can be reverted quickly if issues are found
- Monitor nightly build for failures
- If successful, update CI/CD workflows to add Python 3.14 to test matrix
- If failures occur, revert and investigate compatibility issues
* chore: support Python 3.14
Bump requires-python upper bound to <3.15 across langflow, langflow-base,
lfx, and langflow-sdk, and add 3.14 to CI test matrices so PRs are gated
on 3.14 compatibility.
Conditional pins for transitive deps without 3.14 wheels at the existing
caps:
- onnxruntime: >=1.26 on 3.14 (existing <1.24 cap retained for 3.10)
- faiss-cpu: >=1.13.2 on 3.14 (existing ==1.9.0.post1 retained for <3.14)
* chore: marker-gate IBM watsonx packages for Python 3.14
ibm-watsonx-ai 1.3.x cannot import on Python 3.14: its StrEnum subclasses
override __init__ in a way that conflicts with 3.14's reworked enum
__set_name__ path (TypeError on KnowledgeBaseFieldRole class creation).
Cap ibm-watsonx-ai, langchain-ibm, and the ibm-watsonx-orchestrate-*
extras at python_version<'3.14' until upstream adapts. Watsonx component
imports are already lazy, so this surfaces only at component-access time
on 3.14, where the existing ImportError handler in
lfx.components.ibm.__init__ degrades it gracefully.
test_model_utils.py imports ChatWatsonx at module top to verify
get_model_name resolves model_id; guard that import so the test
module skips on 3.14 instead of breaking collection.
* [autofix.ci] apply automated fixes
* chore: upgrade remaining Docker images to Python 3.14
build_and_push*.Dockerfile already moved to 3.14 in the merge from
feat/upgrade-python-3.14-docker-images. Apply the same bump to the
dev, devcontainer, and lfx Docker images so all in-repo Dockerfiles
share one Python version.
* [autofix.ci] apply automated fixes
* chore: gate 3.14-broken extras to python_version<'3.14'
cuga (and its transitive fastembed -> py-rust-stemmers source build),
altk/agent-lifecycle-toolkit (re-pulls ibm-watsonx-ai which was already
gated), langchain-pinecone, langwatch, ragstack-ai-knowledge-store, and
OpenDsStar all pin themselves below 3.14 upstream. Without a marker
guard, the workspace lock still tries to install them and a Docker
`uv sync` then attempts source builds that require Rust (CI Docker
test failure with py-rust-stemmers 0.1.5).
Add python_version<'3.14' (or <'3.13' for ragstack) to each pin so
those packages are simply omitted from the 3.14 install set until
upstream catches up. uv pip check is now clean on 3.14.2.
* test: skip test_altk_agent on Python 3.14
altk (agent-lifecycle-toolkit) is gated to python_version<'3.14'
upstream and now via the langflow-base [altk] extra marker. The test
imports altk at module top to exercise ALTKAgentComponent; guard the
import so collection skips on 3.14 instead of failing.
* test: skip remaining altk tests on Python 3.14
Three more test modules import lfx.base.agents.altk_* at module top,
which transitively imports altk. Add the same module-level skip guard
as test_altk_agent.py:
- test_altk_agent_logic.py
- test_altk_agent_tool_conversion.py
- test_conversation_context_ordering.py
* fix(calculator): replace removed ast.Num with ast.Constant for Python 3.14
ast.Num was deprecated in Python 3.8 in favor of ast.Constant and
removed entirely in 3.14. The calculator tool and its core walker
crashed on 3.14 with 'module ast has no attribute Num'.
Switch the isinstance checks to ast.Constant + isinstance(node.value,
(int, float)), and drop the now-dead ast.Num backwards-compat branch
in calculator_core.py (the ast.Constant branch already handles every
input it would have matched). Update the parser unit-test fixtures
to construct ast.Constant nodes.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* test: skip LangWatch HTTP instrumentation tests on Python 3.14
langwatch is gated to python_version<'3.14' upstream. The
TestLangWatchHttpInstrumentation class patches langwatch.setup and
langwatch.trace inside a fixture, which requires langwatch to be
importable; on 3.14 the test errors with ModuleNotFoundError.
Guard the class with a skipif on langwatch availability.
* test: allow IBM and altk components to be missing on Python 3.14
test_all_modules_importable enforces that every component in __all__
imports cleanly. On 3.14 the ibm-watsonx-ai, langchain-ibm, and altk
packages are gated upstream and intentionally not installed, so the
WatsonxAI, WatsonxEmbeddings, and ALTKAgent components fail to import
as designed.
- test_all_components_in_categories_importable: accept a known-gated
deny-list on 3.14 instead of failing.
- test_all_lfx_component_modules_directly_importable: extend the
existing 'missing optional dependency' allowlist with altk,
langchain_ibm, and ibm_watsonx_ai.
* fix(lfx): pass ensure_exists to user_cache_dir for Python 3.14
Python 3.14 tightened PurePath.__init__ to reject unknown keyword
arguments. The KeyedWorkerLockManager constructor was passing
ensure_exists=True to Path() instead of user_cache_dir(), which
silently worked on 3.10-3.13 but raises TypeError on 3.14 and also
meant the cache directory was never actually being created.
Move the kwarg to user_cache_dir() where it belongs and update the
two unit tests that asserted the buggy call shape.
* [autofix.ci] apply automated fixes
* test: accept either ZIP or JSON error path on Python 3.14
Python 3.14's zipfile.is_zipfile() now validates the central-directory
signature in addition to EOCD, so the garbage+EOCD payload no longer
passes the is_zipfile() dispatch in the /flows/upload/ route. The
endpoint still returns 400 with a descriptive detail (now from the JSON
branch); only the message wording differs, which the test was asserting
verbatim.
* fix(lfx): normalize cors_origins ['*'] back to '*' on Python 3.14
Pydantic-settings on Python 3.14 parses the env var '*' into ['*']
before the cors_origins field validator runs (the list[str] | str union
resolves differently than on 3.10-3.13). Collapse that back to the
bare-string wildcard so downstream consumers — including
warn_about_future_cors_changes and the test suite — see the same shape
on every supported Python version.
---------
Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit
|
|||
| 1955f8fae5 |
chore: support Python 3.14 (#13085)
* feat: upgrade Docker images to Python 3.14 (experimental) ## Changes - Update pyproject.toml: requires-python from <3.14 to <3.15 - Update all 5 Dockerfiles to use Python 3.14: - Builder stage: python3.12-trixie-slim → python3.14-trixie-slim - Runtime stage: python:3.12-slim-trixie → python:3.14-slim-trixie ## Files Updated - src/backend/base/pyproject.toml - docker/build_and_push_base.Dockerfile - docker/build_and_push.Dockerfile - docker/build_and_push_backend.Dockerfile - docker/build_and_push_with_extras.Dockerfile - docker/build_and_push_ep.Dockerfile ## Rationale Python 3.14.5 was released on May 10, 2026. Upgrading to the latest Python version should help reduce CVE vulnerabilities in Docker images. ## Testing Strategy This is an experimental change to test Python 3.14 compatibility: - Docker images will use Python 3.14 - CI/CD workflows still test on Python 3.10-3.13 - Nightly build will validate if dependencies work with 3.14 - Can be reverted quickly if issues are found ## Next Steps - Monitor nightly build for failures - If successful, update CI/CD workflows to add Python 3.14 to test matrix - If failures occur, revert and investigate compatibility issues * chore: support Python 3.14 Bump requires-python upper bound to <3.15 across langflow, langflow-base, lfx, and langflow-sdk, and add 3.14 to CI test matrices so PRs are gated on 3.14 compatibility. Conditional pins for transitive deps without 3.14 wheels at the existing caps: - onnxruntime: >=1.26 on 3.14 (existing <1.24 cap retained for 3.10) - faiss-cpu: >=1.13.2 on 3.14 (existing ==1.9.0.post1 retained for <3.14) * chore: marker-gate IBM watsonx packages for Python 3.14 ibm-watsonx-ai 1.3.x cannot import on Python 3.14: its StrEnum subclasses override __init__ in a way that conflicts with 3.14's reworked enum __set_name__ path (TypeError on KnowledgeBaseFieldRole class creation). Cap ibm-watsonx-ai, langchain-ibm, and the ibm-watsonx-orchestrate-* extras at python_version<'3.14' until upstream adapts. Watsonx component imports are already lazy, so this surfaces only at component-access time on 3.14, where the existing ImportError handler in lfx.components.ibm.__init__ degrades it gracefully. test_model_utils.py imports ChatWatsonx at module top to verify get_model_name resolves model_id; guard that import so the test module skips on 3.14 instead of breaking collection. * [autofix.ci] apply automated fixes * chore: upgrade remaining Docker images to Python 3.14 build_and_push*.Dockerfile already moved to 3.14 in the merge from feat/upgrade-python-3.14-docker-images. Apply the same bump to the dev, devcontainer, and lfx Docker images so all in-repo Dockerfiles share one Python version. * [autofix.ci] apply automated fixes * chore: gate 3.14-broken extras to python_version<'3.14' cuga (and its transitive fastembed -> py-rust-stemmers source build), altk/agent-lifecycle-toolkit (re-pulls ibm-watsonx-ai which was already gated), langchain-pinecone, langwatch, ragstack-ai-knowledge-store, and OpenDsStar all pin themselves below 3.14 upstream. Without a marker guard, the workspace lock still tries to install them and a Docker `uv sync` then attempts source builds that require Rust (CI Docker test failure with py-rust-stemmers 0.1.5). Add python_version<'3.14' (or <'3.13' for ragstack) to each pin so those packages are simply omitted from the 3.14 install set until upstream catches up. uv pip check is now clean on 3.14.2. * test: skip test_altk_agent on Python 3.14 altk (agent-lifecycle-toolkit) is gated to python_version<'3.14' upstream and now via the langflow-base [altk] extra marker. The test imports altk at module top to exercise ALTKAgentComponent; guard the import so collection skips on 3.14 instead of failing. * test: skip remaining altk tests on Python 3.14 Three more test modules import lfx.base.agents.altk_* at module top, which transitively imports altk. Add the same module-level skip guard as test_altk_agent.py: - test_altk_agent_logic.py - test_altk_agent_tool_conversion.py - test_conversation_context_ordering.py * fix(calculator): replace removed ast.Num with ast.Constant for Python 3.14 ast.Num was deprecated in Python 3.8 in favor of ast.Constant and removed entirely in 3.14. The calculator tool and its core walker crashed on 3.14 with 'module ast has no attribute Num'. Switch the isinstance checks to ast.Constant + isinstance(node.value, (int, float)), and drop the now-dead ast.Num backwards-compat branch in calculator_core.py (the ast.Constant branch already handles every input it would have matched). Update the parser unit-test fixtures to construct ast.Constant nodes. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * test: skip LangWatch HTTP instrumentation tests on Python 3.14 langwatch is gated to python_version<'3.14' upstream. The TestLangWatchHttpInstrumentation class patches langwatch.setup and langwatch.trace inside a fixture, which requires langwatch to be importable; on 3.14 the test errors with ModuleNotFoundError. Guard the class with a skipif on langwatch availability. * test: allow IBM and altk components to be missing on Python 3.14 test_all_modules_importable enforces that every component in __all__ imports cleanly. On 3.14 the ibm-watsonx-ai, langchain-ibm, and altk packages are gated upstream and intentionally not installed, so the WatsonxAI, WatsonxEmbeddings, and ALTKAgent components fail to import as designed. - test_all_components_in_categories_importable: accept a known-gated deny-list on 3.14 instead of failing. - test_all_lfx_component_modules_directly_importable: extend the existing 'missing optional dependency' allowlist with altk, langchain_ibm, and ibm_watsonx_ai. * fix(lfx): pass ensure_exists to user_cache_dir for Python 3.14 Python 3.14 tightened PurePath.__init__ to reject unknown keyword arguments. The KeyedWorkerLockManager constructor was passing ensure_exists=True to Path() instead of user_cache_dir(), which silently worked on 3.10-3.13 but raises TypeError on 3.14 and also meant the cache directory was never actually being created. Move the kwarg to user_cache_dir() where it belongs and update the two unit tests that asserted the buggy call shape. * [autofix.ci] apply automated fixes * test: accept either ZIP or JSON error path on Python 3.14 Python 3.14's zipfile.is_zipfile() now validates the central-directory signature in addition to EOCD, so the garbage+EOCD payload no longer passes the is_zipfile() dispatch in the /flows/upload/ route. The endpoint still returns 400 with a descriptive detail (now from the JSON branch); only the message wording differs, which the test was asserting verbatim. * fix(lfx): normalize cors_origins ['*'] back to '*' on Python 3.14 Pydantic-settings on Python 3.14 parses the env var '*' into ['*'] before the cors_origins field validator runs (the list[str] | str union resolves differently than on 3.10-3.13). Collapse that back to the bare-string wildcard so downstream consumers — including warn_about_future_cors_changes and the test suite — see the same shape on every supported Python version. --------- Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| ea29bcc6b9 |
feat: Redis-backed job queue for multi-worker deployments (#12588)
* feat: implement Redis job queue and fakeredis support
- Refactored the job queue service to support Redis-backed management for cross-worker scaling.
- Added environment variables for configuration:
- `LANGFLOW_JOB_QUEUE_TYPE=redis`
- `LANGFLOW_REDIS_QUEUE_DB=1`
- Updated job ownership methods to be asynchronous for improved concurrency handling.
- Enhanced Redis cache service with namespacing via key prefixes.
- Introduced `fakeredis` for in-memory Redis simulation in testin>
- Added comprehensive unit tests for Redis job queue components.
* fix: run experimental warning for RedisCache usage only once
- Introduced a mechanism to emit a one-time warning for the RedisCache experimental feature during server runtime.
- The warning is logged only if no other worker has already emitted it, ensuring clarity for users regarding the experimental status of RedisCache.
- The implementation includes a temporary file check to prevent multiple warnings across different processes.
* docs: document environment variables for worker management
- Added documentation for LANGFLOW_GUNICORN_PRELOAD to explain preloading for better performance.
- Detailed the use of LANGFLOW_JOB_QUEUE_TYPE for specifying backends (e.g., Redis).
- Included LANGFLOW_REDIS_QUEUE_DB to define the database index for job queues.
- Updated the "High-Load Environments" guide with these optimal configurations.
* docs: updated 'High-load and multi-worker environments' section
* feat: enhance RedisJobQueueService with consumer wrapper management
- Introduced a caching mechanism for Redis stream consumers to optimize job data retrieval.
- Added methods to manage consumer wrappers, ensuring they are reused across sequential polls.
- Implemented cleanup logic to cancel and clear consumer wrappers during job cleanup and service stop.
- Expanded unit tests to verify consumer wrapper reuse and cleanup behavior.
* fix: ensure Redis keys are deleted during job cleanup even on cancellation
- Updated the cleanup_job method in RedisJobQueueService to guarantee Redis keys are removed even if the job cleanup is interrupted by a CancelledError.
- Added a new unit test to verify that Redis keys are deleted correctly when cleanup is called during task cancellation.
* fix: manage connection check task in RedisJobQueueService
- Added handling for the connection check task in the stop method to ensure it is properly cancelled and awaited if still running.
- This change improves resource management and prevents potential issues during service shutdown.
* fix: handle unpublished sentinel requeue on cancellation in RedisJobQueueService
- Updated the job processing logic to ensure that if a job is cancelled during the xadd operation, the unpublished sentinel is requeued instead of being dropped.
- Introduced a new unit test to verify this behavior, ensuring robustness in job handling during cancellations.
* fix: improve Redis job queue service with enhanced configuration and cleanup
- Added atexit cleanup to remove stale temporary files for RedisCache.
- Refactored Redis job queue service to use shared constants for stream prefixes, improving maintainability.
- Updated type hints for better clarity and consistency in RedisQueueWrapper and RedisJobQueueService.
- Enhanced error handling with configurable backoff for transient read failures.
* fix: enhance Redis job queue service with maxlen configuration for xadd
- Updated the xadd method in RedisJobQueueService to include maxlen and approximate parameters, improving stream management and preventing excessive memory usage.
* fix: enhance job ownership retrieval in RedisJobQueueService
- Updated the get_job_owner method to refresh the Redis key TTL on successful lookups, ensuring long-running jobs maintain their ownership anchor.
- Improved code clarity by extracting the owner key into a variable and adding detailed docstring explanations for better understanding of the TTL management.
* fix: improve job ownership handling in cleanup_job method
- Enhanced the cleanup_job method in RedisJobQueueService to accurately capture job ownership before deleting Redis keys, preventing potential data corruption in multi-worker scenarios.
- Added comments for clarity on ownership logic and its implications during job cleanup.
* fix: optimize TTL management in RedisJobQueueService
- Introduced periodic TTL refresh logic in the _bridge_to_redis method to enhance Redis stream management, reducing round-trips and improving throughput.
- Added constants for TTL refresh events and seconds to maintain clarity and configurability.
- Updated event handling to ensure TTL is refreshed appropriately based on event count and time elapsed.
* fix: improve event handling in flow response management
- Removed unnecessary error handling for missing event tasks in get_flow_events_response, allowing for smoother operation when no task exists.
- Updated create_flow_response to handle optional event_task parameter, ensuring proper cleanup during disconnections.
- Added unit tests to verify behavior when event tasks are missing, enhancing robustness in streaming scenarios.
* fix: enhance RedisQueueWrapper with startup grace period and stream observation
- Added a startup grace period to prevent premature end-of-stream signals when the producer has not yet issued its first XADD.
- Introduced a flag to track whether the stream has been observed, improving the handling of early polling scenarios.
- Updated logic to ensure proper handling of stream existence checks and logging for better debugging during job processing.
* fix: implement cleanup for old cross-worker job queues in RedisJobQueueService
- Added a new method to clean up done cross-worker consumer wrappers that are not owned by the current worker, ensuring proper resource management.
- Enhanced the existing cleanup logic to prevent memory leaks by explicitly pruning stale entries from the consumer wrappers dictionary.
- Improved logging to provide better visibility into the cleanup process for cross-worker jobs.
* fix: enhance job handling in JobQueueService with guarded task execution
- Updated the start_job method to accept a Coroutine type for task_coro, ensuring type safety.
- Introduced a new _guarded_task method to wrap job coroutines, guaranteeing that unhandled exceptions emit an error event and write a sentinel to the Redis Stream, improving reliability in job processing.
- Enhanced documentation to clarify the behavior of the new task handling mechanism and its implications for cross-worker consumers.
* fix: improve client disconnection handling in create_flow_response
- Added logging for scenarios where a client disconnects without an associated event_task, clarifying that the producer will continue running until the build completes.
- Documented the limitation regarding cross-worker passive disconnects and the need for a Redis side-channel for proper cancellation, enhancing observability in the logs.
* fix: enhance RedisQueueWrapper to manage initial read state and buffer behavior
- Introduced a flag to track the completion of the first XREAD call, ensuring proper buffer management during the initial read phase.
- Updated the empty method to reflect the state of the buffer accurately, preventing premature exits from the drain loop until the first read is complete.
- Improved documentation to clarify the behavior of the new flag and its impact on job processing.
* fix: clarify RedisQueueWrapper behavior in tests for buffer state and no-op operations
- Updated test documentation to explain the behavior of the empty() method in relation to the first XREAD completion, ensuring accurate understanding of buffer state during job processing.
- Adjusted assertions in tests to reflect the intended behavior of the RedisQueueWrapper, specifically regarding the no-op nature of put_nowait and its impact on the internal buffer.
* fix: update dependencies in pyproject.toml and uv.lock
- Added fakeredis dependency with a minimum version of 2.0.0 to both pyproject.toml and uv.lock files.
- Ensured proper formatting and comments in pyproject.toml for clarity on onnxruntime version constraints.
* fix: enhance RedisQueueWrapper and job queue handling for improved cancellation and buffer management
- Introduced a structural protocol `_CancellableQueue` to ensure queues can handle cancellation properly during client disconnects.
- Updated `RedisQueueWrapper` to implement this protocol, allowing for graceful cancellation of background tasks.
- Added a maximum size limit to the internal buffer to prevent unbounded memory usage and ensure backpressure on slow consumers.
- Implemented a done callback to handle unexpected fill task crashes, ensuring consumers are not left hanging indefinitely.
- Enhanced unit tests to verify compliance with the new protocol and the behavior of the buffer under various conditions.
* fix: RedisQueueWrapper sentinel delivery, error time-bound, and cancel response
- Deliver end-of-stream sentinel on fill-task cancellation (_on_fill_done now
handles both cancelled and exception paths so consumers are never left hanging)
- Add _error_start time-bound to xread and exists() error loops: after
_STARTUP_GRACE_S seconds of continuous Redis errors the sentinel is delivered
instead of retrying forever
- Advance _last_id cursor only after buffer.put() succeeds so cancellation mid-put
does not silently skip that message in the Redis cursor
- Return False from cancel_flow_build when event_task is None (cross-worker path)
so the HTTP response correctly reports success=False instead of false success
* [autofix.ci] apply automated fixes
---------
Co-authored-by: ogabrielluiz <gabriel@langflow.org>
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
|
|||
| b54bdf318d |
fix: update security dependencies (#13053)
* chore: update security dependencies - Update brace-expansion to ^5.0.5 in docs - Update picomatch to ^4.0.4 in docs - Update ip-address to ^10.1.1 in frontend - Update GitPython to >=3.1.48 in backend - Add protobuf constraint >=6.33.6,<7.0.0 in backend * chore: address additional CVEs and add security documentation - Update lodash from deprecated 4.18.0 to 4.17.21 - Add docs/SECURITY_OVERRIDES.md documenting all CVEs - Revert mem0ai 2.x upgrade due to compatibility issues * fix: update dependencies to address CVE vulnerabilities - Update langchain-core to >=1.3.3 (fixes CVE-2026-44843) - Update GitPython to >=3.1.50 (partially fixes CVE-2026-44243, CVE-2026-44244, GHSA-mv93-w799-cj2w) - Update pyarrow constraint to >=23.0.1,<24.0.0 (fixes CVE-2026-25087) - Add override-dependencies for transitive packages: - lxml >=6.1.0 (fixes CVE-2026-41066) - mako >=1.3.12 (fixes CVE-2026-44307) - urllib3 >=2.7.0 (fixes CVE-2026-44431, CVE-2026-44432) - python-liquid >=2.2.0 (fixes CVE-2026-45017) Total: 9 CVEs addressed Smoke tests: All imports successful, no breaking changes detected --------- Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> |
|||
| 6d2e255f5d |
Merge remote-tracking branch 'origin/release-1.9.2' into release-1.10.0
# Conflicts: # .secrets.baseline # pyproject.toml # src/backend/base/langflow/initial_setup/starter_projects/Knowledge Retrieval.json # src/backend/base/langflow/initial_setup/starter_projects/Vector Store RAG.json # src/backend/base/pyproject.toml # src/frontend/package-lock.json # src/frontend/package.json # src/lfx/pyproject.toml # src/lfx/src/lfx/_assets/component_index.json # src/sdk/pyproject.toml # uv.lock |
|||
| 5bf73714e3 | chore: Remove the diskcache dependency (#12953) | |||
| 9d94f9b7d0 |
fix: update playwright to 1.59.0 (#12947)
chore: update playwright to 1.59.0 - Update playwright dependency from 1.58.0 to 1.59.0 - Sync uv.lock with updated dependencies Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> |
|||
| 7d4d3e13a3 | chore: bump versions to 1.9.2 (#12898) | |||
| b9e81f6edd | Merge remote-tracking branch 'origin/main' into release-1.10.0 | |||
| 0732423f27 |
chore: security patch (#12725)
* chore: security patch security patch * chore: upgrade package-lock.json * chore: smolagents and transformer update * chore: redis upgrade * chore: litellm upgrade * fix: Pin click to avoid lower versions ## Root cause **litellm 1.83.5+** introduced an exact pin `click==8.1.8` in its `requires_dist` (upstream bug [BerriAI/litellm#26154](https://github.com/BerriAI/litellm/issues/26154) — "Pinning exact dependency versions breaks downstream consumers"). When this branch bumped litellm to 1.83.11, uv was forced to downgrade click from 8.3.2 → 8.1.8. Click 8.2+ captures stderr separately by default; 8.1 merges it — hence `ValueError: stderr not separately captured`. ## Fix - [pyproject.toml:155](pyproject.toml:155) — added `"click>=8.3.0"` to `[tool.uv] override-dependencies` with comment linking the upstream issue. - [src/backend/base/pyproject.toml:112](src/backend/base/pyproject.toml:112) — same override so `langflow-base` Docker builds pick it up too. - Regenerated `uv.lock` (click: 8.1.8 → 8.3.3). - Patched the `click` entry in `src/backend/base/uv.lock` to 8.3.3 (that lockfile is used by `docker/build_and_push_base.Dockerfile` via `uv sync --frozen`; uv treats backend/base as a workspace member so I updated its click block directly — no other references to 8.1.8 remain). - Reverted [src/lfx/tests/unit/cli/test_run_real_flows.py](src/lfx/tests/unit/cli/test_run_real_flows.py) back to its original `result.stderr` usage — now passes cleanly with click 8.3.3. All 19 tests in the file pass. * Update build_and_push_backend.Dockerfile The `uv pip install` command doesn't read `[tool.uv] override-dependencies` from the target packages' pyproject.toml — those overrides only apply during `uv lock`/`uv sync`. The fix is to pass the overrides explicitly via `--overrides`. Now let me verify it works by running the same install locally. Resolution succeeded — click==8.3.3, aiohttp==3.13.5, litellm==1.83.11 all coexist. Let me also test without `--overrides` to confirm I reproduce the original failure. Reproduced the Docker failure locally. Now test with the overrides file: Fix verified — resolution succeeds with the overrides file, failing without. Let me clean up and show the final Dockerfile change. ## Summary **Root cause:** The Dockerfile uses `uv pip install ./src/...` which **does not read** `[tool.uv] override-dependencies` from the target packages' `pyproject.toml` — those overrides only apply during `uv lock`/`uv sync`. On Linux, `cuga>=0.2.20` requires `aiohttp>=3.13.5`, but `litellm==1.83.11` exact-pins `aiohttp==3.13.3` (same upstream bug as the click pin, [BerriAI/litellm#26154](https://github.com/BerriAI/litellm/issues/26154)). Without an active override, resolution fails. **Fix:** Write a `/tmp/uv-overrides.txt` file mirroring the workspace's `override-dependencies` (litellm, python-dotenv, openai, aiohttp, click) and pass `--overrides /tmp/uv-overrides.txt` to `uv pip install`. Verified locally by reproducing the exact failure in an isolated tmp directory (to escape the workspace's pyproject.toml auto-apply), then confirming the overrides file resolves it: - aiohttp==3.13.5 ✓ - click==8.3.3 ✓ - litellm==1.83.11 ✓ - cuga==0.2.22 ✓ * Templates version update * Update .secrets.baseline * chore: update overrides * chore: bump version * chore: bump main version * chore: bump SDK version to 0.1.1 * chore: run uv lock and uv sync after SDK version bump * chore: read the overrides from a single source security patch * chore: add pip check toggle add pip check toggle * chore: litellm base uv.lock update * fix(chore): Pin litellm back to last working release * fix(chore): Pin to 1.83 and higher litellm * Update build_and_push_backend.Dockerfile * chore: update base uv.lock * fix: lazyload toolguard since its an optional extra * Update .secrets.baseline * Update component_index.json * Rebuild component index * Update component_index.json --------- Co-authored-by: Eric Hare <ericrhare@gmail.com> Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com> (cherry picked from commit 60a8f76c3fde17124057626c671ea8162872837a) |
|||
| b398f63a09 |
test: add missing test coverage for wxo deployments (#12689)
* test(deployments): add comprehensive test coverage for WXO deployment schemas, hooks, and components
Frontend (11 new/expanded test files, ~170 tests):
- use-connection-panel-state: tests for all handlers, ID sanitization, env var management
- step-review: tests for summary display, flows, connections, masking, edit mode
- global variable hooks: tests for useGetGlobalVariables auth guard and Zustand hydration
- use-patch-deployment: fix incorrect spec field assertion, test URL/body destructuring
- deployment-expanded-row, deployment-info-grid, deployment-flow-list: component tests
- flow-list-panel, version-panel, connection-search-list: panel component tests
- deployment-stepper-payload-builders: tests for buildConnectionPayloads,
buildDeploymentUpdatePayload, buildDeploymentPayload, buildProviderAccountPayload
Backend (2 expanded test files, ~21 new tests):
- test_deployments_response_mapping: tests for update/list/execution/config response shapes
- test_deployment_schemas: tests for pagination constraints, execution schemas, snapshot schemas
* Fix nested button issue; remove more unnecessary tests
* update fe tests
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* fix tests; filter sql warnings
* ruff
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
(cherry picked from commit
|
|||
| 9d84ec14d9 |
chore: version bump 1.10.0
version bump langflow: 1.10.0 langflow-base: 0.10.0 lfx: 0.5.0 langflow-sdk: 0.2.0 |
|||
| da516b7045 | Merge release branch | |||
| 71fbf5ff00 |
docs: cut version 1.9.0 (#12681)
* version-1.9.0-docs * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 19df4606ae |
chore: update deps (#12657)
* chore: update deps update deps due to sec vuln * chore: langchain-core>=1.2.28 langchain-core>=1.2.28 * chore: update pypdf pypdf 6.10 * chore: pyarrow, openai, litellm, mem0ai, toolguard picomatch, pyarrow, openai, litellm, mem0ai, toolguard * chore: override litellm * chore: match base uv.lock to root * chore: update tool.uv position update tool.uv position * chore: langflow-base[complete]>=0.9.0 langflow-base[complete]>=0.9.0 * chore: revert pyarrow revert pyarrow * chore: revert "lfx~=0.4.0", * chore: lfx white space * chore: remove allow-direct-references = true * chore: uv lock --upgrade after merge |
|||
| 95d4c94ef9 |
fix: upgrade playwright to 1.58.0 to address Chromium CVEs (#12668)
* fix: upgrade playwright to 1.58.0 to address Chromium CVEs - Add playwright>=1.58.0 to override-dependencies in pyproject.toml - Update uv.lock: playwright 1.49.0 -> 1.58.0, pyee 12.0.0 -> 13.0.1 - Fixes CVE-2026-2313, CVE-2026-2314, CVE-2026-2315, CVE-2026-2319, CVE-2026-2321, CVE-2026-2441, CVE-2026-2648, CVE-2026-2649 - Ensures Docker builds download updated Chromium with security patches * fix: update npm to latest version to address brace-expansion CVE-2026-33750 - Add npm update after Node.js installation in Dockerfile - Fixes CVE-2026-33750 in system npm's brace-expansion dependency - System npm had brace-expansion 2.0.2, update gets 5.0.5+ - Low risk change: npm is backward compatible, only affects CLI tool * revert: remove npm update from Dockerfile - npm update attempts were causing CI build failures - Bundled npm has issues but updating it is proving problematic - Focus on playwright CVE fix which is the primary concern - brace-expansion CVE-2026-33750 is lower priority (DoS only) --------- Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> |
|||
| 34886de048 |
fix: Updated Pillow minimum version (#12609)
* fix(frontend): enforce brace-expansion override * fix: enforce handlebars version >=4.7.9 in frontend overrides * fix: update Pillow minimum version to 12.1.1 --------- Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> |
|||
| 9cdbf1b23d |
Revert: policies component (#12564) (#12585)
* Revert "feat: boazdavid policies component added (#12564)"
This reverts commit
|
|||
| ed6c4eeb66 | fix: Change the order of deps for re parsing (#12581) | |||
| 64f84ab0ea |
feat: boazdavid policies component added (#12564)
* feat: @boazdavid policies component added * Update component_index.json * [autofix.ci] apply automated fixes * Update component_index.json * Update component_index.json --------- Co-authored-by: DAVID BOAZ <DAVIDBO@il.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 30f351766b |
feat: IBM Globalization Pipeline integration and i18n setup (#12226)
* feat: add IBM Globalization Pipeline integration and i18n setup
- Add GP REST API client with GP-HMAC authentication (scripts/gp/gp_client.py)
- Add upload/download scripts for syncing strings with GP
- Set up i18next + react-i18next with 7 language support (en, fr, ja, es, de, pt, zh-Hans)
- Add 48 UI strings in flat dot notation to en.json (from alerts_constants)
- Add GitHub Actions for automated upload on en.json changes and daily translation download
* [autofix.ci] apply automated fixes
* fix: resolve ruff linting issues in GP scripts
- Fix quote style, import ordering, docstring format
- Use Path instead of os.makedirs/open/os.path.join
- Add missing timeouts to requests calls
- Move noqa comments to correct lines for S501
* feat: migrate all frontend UI strings to i18n for GP localization
- Replace all hardcoded user-facing strings with t() calls across 80+ components, pages, modals, and utilities
- Add 326 translation keys to en.json covering errors, alerts, dialogs, chat, flow, settings, store, table, output, nav, auth, and more
- Add translated locale files for fr, es, de, pt, ja, zh-Hans (downloaded from Globalization Pipeline)
- Migrate alerts_constants.tsx and constants.ts string usages to react-i18next
- Support non-React files (stores, utils) via i18n.t() direct calls
* feat: migrate sidebar strings to i18n (Phase 4)
- Add 56 new translation keys for sidebar categories, nav items, labels, buttons, and empty states
- Convert SIDEBAR_CATEGORIES display_names to i18n key references in styleUtils.ts
- Update 13 sidebar components to use t() for all user-facing strings
- Refresh all 6 locale files from GP (382 total keys)
- Add BACKEND_STRINGS_STRATEGY.md documenting plan for component display_name i18n
* [autofix.ci] apply automated fixes
* fix: add react-i18next mock and fix sidebar nav tests for i18n migration
- Add global react-i18next mock in jest.setup.js using en.json lookups so
t(key) returns English strings instead of raw keys in test environment
- Fix sidebarSegmentedNav tests: update NAV_ITEMS expectations to use i18n
keys, and use en.json lookups for tooltip/accessibility label assertions
* feat: add LanguageSelector dropdown to globalization pipeline
Cherry-picked frontend-only changes from feat/gp-backend-i18n (6ab21559db, 3e274a84e3):
- Add LanguageSelector component with dropdown to switch UI language
- Wire LanguageSelector into AccountMenu header
- Persist language preference to localStorage in i18n.ts
- Send Accept-Language header via useCustomApiHeaders (reactive via useTranslation)
No backend files included.
* [autofix.ci] apply automated fixes
* ci: add GP_TEST environment to gp-download and gp-upload workflows
* temp: add feature branch push trigger for workflow testing
* temp: trigger gp-upload workflow test
* fix: use correct GP_test environment name in workflows
* fix: use vars for GP_INSTANCE and GP_BUNDLE
* [autofix.ci] apply automated fixes
* fix: correct environment name to GP-test (hyphen not underscore)
* chore: update translations from Globalization Pipeline [skip ci]
* [autofix.ci] apply automated fixes
* chore: update translations from Globalization Pipeline [skip ci]
* temp: trigger gp-upload workflow test
* ci: add download-gp-bundle job to nightly build pipeline
Downloads frontend translations from Globalization Pipeline after
create-nightly-tag and before frontend tests run, ensuring tests
always validate the latest translated locale files.
- Frontend tests (Linux/Windows) now depend on download-gp-bundle
- Backend unit tests are unaffected (no frontend locale dependency)
- GP failure blocks release-nightly-build (same severity as linux tests)
* chore: remove temporary dev files from gp scripts directory
Remove BACKEND_STRINGS_STRATEGY.md (internal planning doc), test_auth.py
(GP auth debugging script), and en.json (dev-time string snapshot).
Also add venv/ to .gitignore to prevent accidental commits of the local virtualenv.
* [autofix.ci] apply automated fixes
* chore: update translations from Globalization Pipeline [skip ci]
* [autofix.ci] apply automated fixes
* chore: update translations from Globalization Pipeline [skip ci]
* feat(i18n): move language selector from account menu to Settings > General
- Add LanguageForm card component in GeneralPage following the existing
Card pattern (ProfilePictureForm), with immediate language switching
- English marked as "(Recommended)" via i18n key
- Remove Language row from AccountMenu dropdown
- Add settings.languageTitle/Description/Recommended keys to en.json
for upload to Globalization Pipeline
* feat(i18n): refactor language constants and improve GP pipeline
- Extract SUPPORTED_LANGUAGES into src/frontend/src/constants/languages.ts
- Update LanguageForm to import from shared constants, add aria-label
- Add settings.languageSelectAriaLabel key to en.json
- Update LanguageSelector component to use shared SUPPORTED_LANGUAGES
- Fix gp_client and download_translations scripts
- Update GP upload/download/nightly GitHub Actions workflows
- Add GP script tests
* [autofix.ci] apply automated fixes
* ci(gp): trigger upload on release branches, download via dispatch only
- Upload now triggers on push to release-* branches (instead of main)
so translators get strings once they're finalized on the release branch
- Download removes the daily cron schedule; workflow_dispatch only
since the nightly build already handles scheduled runs
* chore: update translations from Globalization Pipeline
* fix(i18n): address PR review comments — tests, nightly build decoupling, GP test workflow
- Add pytest tests for gp_client.py (HMAC auth, HTTP errors, timeout)
- Add pytest tests for upload_strings.py and download_translations.py
- Add conftest.py to set sys.path so GP scripts can be imported from repo root
- Add scripts/gp/__init__.py to make gp a proper package for test imports
- Add gp-test.yml workflow to run GP script tests on PRs touching scripts/gp/**
- Decouple download-gp-bundle from nightly build chain:
- Add continue-on-error: true so GP failure never blocks the build
- Remove download-gp-bundle from needs/if of frontend-tests-linux,
frontend-tests-windows, and release-nightly-build
- Add warning annotation step that fires on GP download failure
* feat(i18n): lazy-load non-English locale files via Vite dynamic imports
- Remove 6 static locale imports from i18n.ts; only en.json is bundled
- Export loadLanguage() which dynamically imports a locale chunk on demand
and caches it via i18n.hasResourceBundle (no-op on repeat calls)
- Update index.tsx to await loadLanguage(detectedLang) before rendering,
preventing an English flash for non-English users
- Make handleChange async in LanguageForm and LanguageSelector to await
loadLanguage before calling i18n.changeLanguage
- Add i18n.test.ts covering loadLanguage: en no-op, new language load,
cache hit, and multiple independent languages
- Update LanguageForm.test.tsx to mock @/i18n and assert loadLanguage
is called before changeLanguage on language switch
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix: add ruff ignores for scripts/gp/tests
- Add per-file ignores for scripts/gp/tests/* to fix CI failures
- Ignore S101 (assert usage), TRY003, EM101 for test files
- Follows existing pattern for other test directories
* [autofix.ci] apply automated fixes
* fix(ci): run GP translation download before nightly tag creation
Previously download-gp-bundle ran after create-nightly-tag, so fresh
translations were committed to the branch after the tag was already
created. The nightly Docker image builds from the tag, meaning it never
included the day's translations.
Fix: move download-gp-bundle to run before create-nightly-tag so the
tag is created on a commit that already includes the latest translations.
* fix(ui): replace native select with Radix UI Select in LanguageForm
Swaps the native <select> element for the existing Radix UI Select
component to get consistent padding on both sides of the chevron,
removing reliance on browser-rendered chevron positioning.
* fix(ci): rearchitect GP translation download to use PRs instead of direct push
Replace direct branch commits in gp-download.yml and nightly_build.yml with
a PR-based flow using peter-evans/create-pull-request, respecting release
branch protection policies.
- gp-download.yml: dynamically resolves latest release-* branch, closes stale
translation PRs, opens a new PR via peter-evans/create-pull-request@v8, and
enables auto-merge (squash); scheduled at 23:00 UTC (1h before nightly)
- nightly_build.yml: remove download-gp-bundle job and its dependency from
create-nightly-tag — nightly build now uses translations already in the branch
* fix(ci): skip GP upload when triggered from an outdated release branch
Add a branch guard step that resolves the latest release-* branch at
runtime and skips the upload (with a notice annotation) if the triggering
branch is not the latest. Prevents stale en.json from an old release branch
overwriting current source strings in the Globalization Pipeline.
* fix(tests): fix LanguageForm test suite failures
- Add `...jest.requireActual` to @tanstack/react-query mock so QueryClient
constructor remains available (used in contexts/index.tsx at module load)
- Mock @/components/ui/select with a native <select> shim so test assertions
using selectOptions() and querySelectorAll("option") work correctly
* fix: add trailing newline to component_index.json
The build_component_index.py script was generating component_index.json
without a trailing newline, causing CI failures when autofix.ci tried to
add the missing newline. This fix ensures the generated JSON file always
ends with a newline character, following POSIX text file standards.
Fixes the 'No newline at end of file' error in the update-component-index
workflow job.
* [autofix.ci] apply automated fixes
* fix: add trailing newline to component_index.json
The build_component_index.py script was generating component_index.json
without a trailing newline, causing CI failures when autofix.ci tried to
add the missing newline. This fix ensures the generated JSON file always
ends with a newline character, following POSIX text file standards.
Fixes the 'No newline at end of file' error in the update-component-index
workflow job.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com>
|
|||
| b905ea1c4a |
fix: Added dependency updates for security and bug fixes (#12543)
* security: upgrade vulnerable dependencies with override enforcement - Add security overrides for orjson, gunicorn, pypdf, nltk, markdown, dynaconf, pillow - Update base pyproject.toml: pillow>=12.0.0, pypdf>=6.9.0 - Selective upgrade: only orjson (3.11.7->3.11.8) and pillow (11.3.0->12.2.0) - Resolves 7/8 flagged CVEs (diskcache awaiting upstream fix) - Defense-in-depth: TOML minimums + override enforcement * chore: update dependencies for security and bug fixes - Update aiohttp to >=3.13.4 (security fix) - Update pyarrow to >=19.0.1,!=22.0.0 (exclude buggy 22.0.0) - Update playwright to ^1.59.1 (from ^1.57.0) - Add picomatch >=4.0.4 to npm overrides - Add cuga upper bound <0.3.0 for stability All tests passing (212 suites, 3,104 tests) * chore: reapply dependency updates after merge - Add aiohttp>=3.13.4 to override-dependencies (security fix) - Maintain pyarrow>=19.0.1,!=22.0.0 (exclude buggy 22.0.0) - Maintain playwright^1.59.1 (updated from ^1.57.0) - Maintain picomatch>=4.0.4 in npm overrides - Maintain cuga>=0.2.10,<0.3.0 for stability * chore: add playwright to npm overrides - Add playwright ^1.59.1 to overrides to ensure consistent version across all dependencies * chore(deps): upgrade litellm to 1.80.0 - Updated litellm from >=1.60.2 to >=1.80.0 in langflow-base - Maintains compatibility with openai 1.x (requires >=1.99.5) - Includes security fixes and performance improvements - All existing security overrides maintained (aiohttp, pillow, picomatch) - pyarrow kept at current version for Google Vertex AI compatibility * chore: regenerate uv.lock after merge with release-1.9.0 - Resolved merge conflict in uv.lock - Regenerated lock file with litellm 1.80.0 - Includes latest component index updates from release-1.9.0 --------- Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> Co-authored-by: Adam Aghili <Adam.Aghili@ibm.com> Co-authored-by: Carlos Coelho <80289056+carlosrcoelho@users.noreply.github.com> |
|||
| f557b06c3b |
fix: upgrade vulnerable dependencies with override enforcement (#12526)
security: upgrade vulnerable dependencies with override enforcement - Add security overrides for orjson, gunicorn, pypdf, nltk, markdown, dynaconf, pillow - Update base pyproject.toml: pillow>=12.0.0, pypdf>=6.9.0 - Selective upgrade: only orjson (3.11.7->3.11.8) and pillow (11.3.0->12.2.0) - Resolves 7/8 flagged CVEs (diskcache awaiting upstream fix) - Defense-in-depth: TOML minimums + override enforcement Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> |
|||
| 3ee4bd57eb |
docs: test harness for api reference code samples (#12338)
* curl-examples-and-tests * move-curl-examples * docs-add-existing-js-and-python-examples * add-python-and-js-examples * use-code-snippet-not-code-block * remove-unused-ts-files * add-gitignore-for-docs-pycache * test-api-commands-with-langflow-env-var * add-fixtures-and-makefile-tests-pass * no-action * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * block-ruff-check-for-docs-examples * add-horizontal-scrolling-and-pin-copy-button --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
