Commit Graph

656 Commits

Author SHA1 Message Date
5cb0edde5a fix: keep nightly prerelease wheels explicit (#13815) 2026-06-24 13:14:56 -07:00
e7bf0da0da fix: include sdk wheel in bundle smoke test 2026-06-23 20:41:26 -07:00
3f4c1a0a55 fix: unblock nightly bundle install resolution 2026-06-23 20:37:27 -07:00
4a8b47e46a Merge remote-tracking branch 'origin/release-1.10.1' into release-1.11.0
# Conflicts:
#	.secrets.baseline
#	pyproject.toml
#	src/backend/base/langflow/initial_setup/starter_projects/Pokédex Agent.json
#	src/backend/base/langflow/initial_setup/starter_projects/Structured Data Analysis Agent.json
#	src/backend/base/langflow/services/auth/service.py
#	src/backend/base/langflow/services/job_queue/service.py
#	src/backend/base/pyproject.toml
#	src/backend/tests/unit/components/models_and_agents/test_cuga_agent.py
#	src/backend/tests/unit/test_redis_job_queue_service.py
#	src/bundles/lfx-bundles/src/lfx_bundles/deepseek/deepseek.py
#	src/bundles/lfx-bundles/src/lfx_bundles/glean/glean_search_api.py
#	src/bundles/lfx-bundles/src/lfx_bundles/homeassistant/home_assistant_control.py
#	src/bundles/lfx-bundles/src/lfx_bundles/homeassistant/list_home_assistant_states.py
#	src/bundles/lfx-bundles/src/lfx_bundles/huggingface/huggingface_inference_api.py
#	src/bundles/lfx-bundles/src/lfx_bundles/lmstudio/lmstudioembeddings.py
#	src/bundles/lfx-bundles/src/lfx_bundles/xai/xai.py
#	src/frontend/package-lock.json
#	src/frontend/package.json
#	src/lfx/pyproject.toml
#	src/lfx/src/lfx/_assets/component_index.json
#	src/lfx/src/lfx/components/files_and_knowledge/filesystem.py
#	src/lfx/tests/unit/components/tools/test_filesystem_deny_list.py
#	uv.lock
2026-06-23 17:22:13 -07:00
6453fa8a75 ci: throttle bundle publishes to PyPI 2026-06-23 16:45:34 -07:00
4c8fb6b954 fix: gate release rc setup for stable releases 2026-06-23 16:01:04 -07:00
7896b9f7f1 feat(bundles): metapackage split (Phase A) — engine-only lfx, lfx-bundles long tail, 5 graduated partner packages (#13563)
* feat(extension): add manifest-less lfx.bundles discovery + precedence tier

Foundation for the bundle metapackage split (1.11). Adds a third
@official-slot discovery source: a distribution declaring the
[project.entry-points."lfx.bundles"] group ships a package whose immediate
subdirectories are each a manifest-less bundle, folder-walked and registered
at @official with no extension.json (the langchain-community model).

- new loader/_bundles_root.py::load_lfx_bundles_extensions, mirroring
  discover_inline_bundles but sourcing roots from the lfx.bundles entry-point
  group via find_spec and reusing _load_bundle_directory
- discovery precedence becomes installed > seed > lfx_bundles > dev > inline
  so a manifest-shipping lfx-<provider> always shadows the same-named
  provider in the metapackage (lets a provider graduate with no lockstep
  release), emitting the existing bundle-shadowed warning
- new bundle-discovery-malformed warning code, emitted on warnings (never
  flips ok / aborts startup) for unresolvable declarations and invalid
  provider folder names
- exported via loader/__init__ and the lfx.extension PEP-562 lazy surface

10 new tests in test_load_lfx_bundles.py; full extension unit suite (449)
passes; ruff + format clean; mypy clean on the new module.

* docs(bundles): changelog entry for the lfx.bundles discovery surface

The BUNDLE_API.md changelog gate diffs each PR's branch against main, so
the entry covering this PR's surface additions (lfx.bundles discovery,
the lfx_bundles precedence tier, bundle-discovery-malformed) must live
on this branch, not only further up the stack. Text is verbatim from the
graduate-partners commit so the stacked merge dedupes trivially.

* fix(extension): reject plain-module lfx.bundles targets; correct validate wording

CodeRabbit review fixes on #13563:
- _spec_package_dir no longer falls back to a plain module's parent
  directory -- a single-file entry-point target now emits the
  bundle-discovery-malformed warning instead of folder-walking unrelated
  sibling directories as bundles (+ regression test)
- BUNDLE_API.md / loader docstring no longer say manifest-less providers
  are 'exempt from lfx extension validate'; they are not valid validator
  input (validate requires a manifest and reports manifest-not-found)

* fix(extension): review fixes — shadowed metapackage providers never import; typed manifest-less reload refusal

Review findings on #13563:

- A manifest-less provider whose bundle name is already claimed by a
  higher-precedence @official source (installed/seed) is now skipped
  WITHOUT importing: all @official sources share the
  _lfx_ext.official.<bundle>.* sys.modules namespace, so importing the
  losing copy overwrote the winner's live modules even though
  _resolve_bundle_shadowing dropped its components afterwards.  The
  orchestrator passes the claimed-name map (built by the new
  _claimed_official_bundles helper, mirroring the resolver's winner
  rule); the skipped copy still emits the typed bundle-shadowed warning.
  This collision is the expected post-graduation state, not an error.

- Manifest-less records now carry a manifestless provenance flag
  (additive field on LoadResult + BundleRecord).  The reload pipeline
  refuses them with the new typed reload-manifestless-unsupported code
  instead of routing through load_extension and failing with a
  misleading manifest-not-found.  Hot reload of metapackage providers
  is a process-restart concern (pip-installed content); a loader-based
  reload path can follow if QA wants it.

BUNDLE_API.md changelog entry extended for both surface changes.

* fix(extension): harden lfx.bundles discovery per review — broad find_spec guard, namespace portions, per-mode error codes

Review findings from ogabrielluiz on #13563:

- find_spec on a dotted declaration imports the PARENT package, whose
  __init__ can raise anything; a non-Import error escaped the old catch
  tuple and (through the palette cache's catch-all) wiped every source's
  components for the boot.  Catch Exception and degrade to the malformed
  sentinel; comment corrected.

- Namespace packages: walk ALL submodule_search_locations portions (one
  root per portion) instead of only locations[0], and dedupe resolved
  roots by path so duplicate entry-point declarations (or overlapping
  portions) never walk a directory twice and self-shadow.
  _spec_package_dir -> _spec_package_dirs.

- Split bundle-discovery-malformed into one code per failure mode,
  mirroring the inline tier: bundles-provider-name-invalid (rename the
  directory), bundles-root-unreadable (check permissions), keeping
  bundle-discovery-malformed for unresolvable declarations (fix the
  entry-point).  Same-tier duplicates get duplicate-lfx-bundles-provider
  instead of overloading bundle-shadowed, whose rendered template
  (format_extension_error renders templates, not ad-hoc messages) was
  wrong on both counts for that case.

- The claimed-bundles cross-source skip now emits bundle-shadowed on
  errors (matching _resolve_bundle_shadowing) so filtering by code never
  mixes severities; bundle-shadowed is already in the CLI warn-only set.

3 new regression tests (raising parent package, namespace portions,
duplicate entry points); BUNDLE_API changelog updated.

* feat(bundles): create lfx-bundles metapackage skeleton (#13564)

* feat(bundles): add lfx-bundles metapackage skeleton

Creates the manifest-less lfx-bundles distribution (the langchain-community
model) that the long-tail providers will move into. Empty skeleton for now;
the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider
folders + per-provider extras later.

- src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles
  entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a
  generated (currently empty) `all` extra; plus a bare lfx_bundles namespace
  package and a README documenting the model + install stories
- wired into the root workspace via the existing bundle marker blocks:
  dep lfx-bundles[all]>=1.0,<2.0, uv source, and member
- hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds
  it with zero workflow change

Verified: the wheel builds (entry_points.txt carries the lfx.bundles group);
load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty
skeleton registers zero providers with no error.

* fix(ci): nightly bundle rename follows [extras] refs and self-refs

Two gaps in update_bundle_versions.py around extras suffixes, both fatal
or silently wrong for the first nightly carrying the lfx-bundles
metapackage:

- update_root_pyproject_for_bundle's dep regex required the version
  specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0"
  (a MAIN dep) was left unrewritten while the workspace member was
  renamed to lfx-bundles-nightly; uv lock then tries to resolve stable
  lfx-bundles from PyPI, where it does not exist. The same gap silently
  left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the
  stable distribution. The regex now tolerates an [extras] group and
  carries it into the replacement.

- rename_bundle_pyproject skipped self-referencing extras, so the
  metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]"
  members after the rename, pulling the stable distribution (same
  lfx_bundles import package, install collision) once published. Self
  refs now follow the rename, idempotently.

Tests drive the real script module, mirroring test_bundle_lfx_pin.py.
This closes the PR-2 audit item flagged when the metapackage was
introduced. The canonical-pre-release cutover would retire the nightly
rename entirely; until it lands, the rename must be correct.

* fix(bundles): address review — drop orphaned nightly bundle-rename script, fix README install stories

- Delete scripts/ci/update_bundle_versions.py and its tests: the nightly
  bundle-rename track it served was retired by the canonical pre-release
  cutover (src/bundles/NIGHTLY.md) and no workflow or Makefile target on
  main or any release branch invokes it. Stale doc references in
  sync_bundle_lfx_pin.py and test_bundle_lfx_pin.py updated to match.
- README: split the install section into what works today (langflow, bare
  lfx) vs what arrives with the bulk move / engine-only split
  (lfx[bundles], per-provider lfx-bundles[<provider>] extras), and note
  the generated all extra is empty until the first provider tranche lands.

* ci(bundles): cross-bundle test matrix (lfx contract axis) (#13566)

* ci(bundles): add cross-bundle test matrix (lfx contract axis)

Tests every extracted bundle (the lfx-bundles metapackage + each graduated
lfx-<provider>) against the lfx contract surface it depends on -- overdue
since 1.10 left 4 independently-versioned bundles depending on lfx.

- .github/workflows/cross-bundle-test.yml: discovers bundles via the same
  src/bundles/*/pyproject.toml glob release.yml uses, then per
  (bundle x python 3.10/3.13): installs the in-repo lfx + the bundle, imports
  the declared entry-point package, asserts lfx.bundles discovery is
  error-free (for the metapackage), runs `lfx extension validate` for
  manifest bundles, and runs the bundle's own tests/.
- Triggers: pull_request (src/bundles, src/lfx), workflow_dispatch, a weekly
  schedule, and workflow_call.

The lfx-minor axis seeds with the in-repo lfx (the 1.10 line is unpublished);
when minors publish, add the version dimension (oldest+latest get full tests,
every supported minor gets contract-smoke) per the epic's cost-control shape.

Verified: the workflow YAML parses and the contract-smoke logic imports a
real bundle (lfx_arxiv) cleanly.

* fix(ci): cross-bundle smoke tolerates extras-less SDK degradation

Review findings on the cross-bundle matrix:

- CRITICAL: the contract smoke asserted zero discovery errors, but the CI
  venv installs lfx-bundles WITHOUT per-provider extras, so providers whose
  modules import their SDK at top level degrade with module-import-failed --
  the expected graceful-degradation contract. The smoke now fails only on
  structural codes and reports the degraded-module count. (Graduated partner
  bundles carry their deps directly, not as extras, so their steps are
  unaffected.)
- the scheduled run now actually delivers the exhaustive grid the header
  promised (all supported Pythons on schedule; oldest+latest on PRs)
- concurrency group with cancel-in-progress on PRs so stacked bundle PRs
  don't queue N-bundles x N-pythons jobs per push

* ci(bundles): tomli fallback for the py3.10 cross-bundle smoke

tomllib is stdlib only from Python 3.11, so the contract-smoke heredoc
failed with ModuleNotFoundError on every py3.10 matrix leg. Install the
tomli backport into the smoke venv and import it as a fallback.

* fix(test): coherent sys.modules restore in the ibm without_ibm_db fixture

The cross-bundle matrix's py3.10 leg exposed test-ordering pollution in
the ibm bundle suite: without_ibm_db monkeypatch-deleted three lfx_ibm
modules and re-imported them mid-test, but entries created during the
test survive teardown (delitem with raising=False records nothing for
keys it didn't find) and re-imported parents never regain the submodule
attribute bindings that mock.patch target resolution walks on Python
3.10 (3.11+ mock resolves via sys.modules and tolerates the
incoherence). Two fixture uses in sequence left the tree inconsistent
and 29 later watsonx tests failed with AttributeError on the package.

Snapshot, drop, and restore the entire lfx_ibm module tree wholesale
instead. Full suite now passes on both 3.10 and 3.13.

* feat(lfx): add lfx[bundles] extra and keep lfx engine-only (#13565)

* feat(bundles): add lfx-bundles metapackage skeleton

Creates the manifest-less lfx-bundles distribution (the langchain-community
model) that the long-tail providers will move into. Empty skeleton for now;
the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider
folders + per-provider extras later.

- src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles
  entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a
  generated (currently empty) `all` extra; plus a bare lfx_bundles namespace
  package and a README documenting the model + install stories
- wired into the root workspace via the existing bundle marker blocks:
  dep lfx-bundles[all]>=1.0,<2.0, uv source, and member
- hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds
  it with zero workflow change

Verified: the wheel builds (entry_points.txt carries the lfx.bundles group);
load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty
skeleton registers zero providers with no error.

* feat(lfx): add lfx[bundles] extra and keep lfx engine-only

lfx ships no bundles by default. The new optional extra pulls the long-tail
metapackage for users who want the provider components without the full
langflow server install.

- src/lfx/pyproject.toml: [project.optional-dependencies]
  bundles = ["lfx-bundles[all]>=1.0,<2.0"]; intentionally NO lfx[all]
- pip install lfx          -> engine only
  pip install "lfx[bundles]" -> engine + the lfx-bundles long tail
  (documented as a headless/serverless deployment footnote, not a headline)

Verified against the built lfx wheel METADATA: lfx-bundles appears only under
`extra == "bundles"`, never in core Requires-Dist, so the engine-only default
is preserved.

* fix(ci): nightly bundle rename follows [extras] refs and self-refs

Two gaps in update_bundle_versions.py around extras suffixes, both fatal
or silently wrong for the first nightly carrying the lfx-bundles
metapackage:

- update_root_pyproject_for_bundle's dep regex required the version
  specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0"
  (a MAIN dep) was left unrewritten while the workspace member was
  renamed to lfx-bundles-nightly; uv lock then tries to resolve stable
  lfx-bundles from PyPI, where it does not exist. The same gap silently
  left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the
  stable distribution. The regex now tolerates an [extras] group and
  carries it into the replacement.

- rename_bundle_pyproject skipped self-referencing extras, so the
  metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]"
  members after the rename, pulling the stable distribution (same
  lfx_bundles import package, install collision) once published. Self
  refs now follow the rename, idempotently.

Tests drive the real script module, mirroring test_bundle_lfx_pin.py.
This closes the PR-2 audit item flagged when the metapackage was
introduced. The canonical-pre-release cutover would retire the nightly
rename entirely; until it lands, the rename must be correct.

* ci(bundles): freeze lfx/components against new top-level providers (#13567)

* feat(bundles): add lfx-bundles metapackage skeleton

Creates the manifest-less lfx-bundles distribution (the langchain-community
model) that the long-tail providers will move into. Empty skeleton for now;
the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider
folders + per-provider extras later.

- src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles
  entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a
  generated (currently empty) `all` extra; plus a bare lfx_bundles namespace
  package and a README documenting the model + install stories
- wired into the root workspace via the existing bundle marker blocks:
  dep lfx-bundles[all]>=1.0,<2.0, uv source, and member
- hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds
  it with zero workflow change

Verified: the wheel builds (entry_points.txt carries the lfx.bundles group);
load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty
skeleton registers zero providers with no error.

* ci(bundles): freeze lfx/components against new top-level providers

After the metapackage split, new providers go to lfx-bundles (or a graduated
lfx-<provider>), never in-tree. Adds an additions-only CI gate.

- scripts/ci/check_components_frozen.py: stdlib gate comparing the live
  top-level dir listing of src/lfx/src/lfx/components/ against a committed
  baseline; fails on any directory not in the baseline. Removals are allowed
  so M4 shim cleanup (which shrinks the set) never trips it.
- scripts/ci/frozen_component_dirs.txt: the 111-dir baseline.
- .github/workflows/lint-py.yml: new freeze-components job runs the gate
  (stdlib-only, no uv sync).

Verified locally: passes on the baseline, fails (exit 1) on a simulated new
provider directory with an actionable message, passes again after cleanup.

* fix(ci): freeze gate counts only dirs shipping __init__.py

Review finding: a stray directory holding only __pycache__ bytecode (left
behind by a branch switch) tripped the additions-only gate locally. A
directory without __init__.py is not a provider; ignore it.

* fix(ci): nightly bundle rename follows [extras] refs and self-refs

Two gaps in update_bundle_versions.py around extras suffixes, both fatal
or silently wrong for the first nightly carrying the lfx-bundles
metapackage:

- update_root_pyproject_for_bundle's dep regex required the version
  specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0"
  (a MAIN dep) was left unrewritten while the workspace member was
  renamed to lfx-bundles-nightly; uv lock then tries to resolve stable
  lfx-bundles from PyPI, where it does not exist. The same gap silently
  left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the
  stable distribution. The regex now tolerates an [extras] group and
  carries it into the replacement.

- rename_bundle_pyproject skipped self-referencing extras, so the
  metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]"
  members after the rename, pulling the stable distribution (same
  lfx_bundles import package, install collision) once published. Self
  refs now follow the rename, idempotently.

Tests drive the real script module, mirroring test_bundle_lfx_pin.py.
This closes the PR-2 audit item flagged when the metapackage was
introduced. The canonical-pre-release cutover would retire the nightly
rename entirely; until it lands, the rename must be correct.

* feat(bundles): move 45 long-tail providers into lfx-bundles + graduate 5 partner packages (#13568)

* feat(bundles): add lfx-bundles metapackage skeleton

Creates the manifest-less lfx-bundles distribution (the langchain-community
model) that the long-tail providers will move into. Empty skeleton for now;
the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider
folders + per-provider extras later.

- src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles
  entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a
  generated (currently empty) `all` extra; plus a bare lfx_bundles namespace
  package and a README documenting the model + install stories
- wired into the root workspace via the existing bundle marker blocks:
  dep lfx-bundles[all]>=1.0,<2.0, uv source, and member
- hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds
  it with zero workflow change

Verified: the wheel builds (entry_points.txt carries the lfx.bundles group);
load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty
skeleton registers zero providers with no error.

* feat(bundles): consolidate first long-tail tranche into lfx-bundles

Adds scripts/migrate/consolidate_bundles.py (the inverse of port_bundle.py --
moves in-tree providers into the manifest-less lfx-bundles metapackage) and
runs it on a verified 5-provider tranche: tavily, exa, wikipedia, yahoosearch,
wolframalpha.

Per provider the script:
- moves src/lfx/src/lfx/components/<p>/ -> lfx_bundles/<p>/ (lowercase names);
- leaves a fail-soft import shim (first line `# lfx-bundles-shim`) so
  `from lfx.components.<p> import X` keeps working when lfx-bundles is
  installed, and raises an actionable ImportError otherwise;
- merges the provider's third-party deps into a PEP 685-normalized lfx-bundles
  extra and regenerates the `all` aggregate. Dep parity holds: `uv sync` is a
  no-op because those deps were already pulled via langflow-base[complete];
- appends the 4-entry migration block per Component class (28 entries) so saved
  flows referencing lfx.components.<p>.<Class> migrate to
  ext:<p>:<Class>@official.

To avoid double registration, the in-tree component walk
(_load_components_dynamically) now skips shimmed provider dirs, and
component_index.json is regenerated (355->348 components, 95->90 modules); the
moved providers load only at @official via lfx.bundles discovery.

Verified: discovery finds all 5 at @official with no errors; shims resolve;
`import lfx.components` still works; the index drops the 7 moved component
entries (residual `tools`-category name refs resolve via the shim); ruff clean.

First tranche proves the engine; the remaining long-tail scales by extending
PROVIDER_DEPS (each provider's deps verified individually -- the careful part).

* feat(bundles): consolidate 30-provider tranche 2 into lfx-bundles

Extends PROVIDER_DEPS with 30 individually-verified providers and runs the
consolidation: vector stores (chroma, clickhouse, couchbase, milvus, mongodb,
pgvector, pinecone, qdrant, supabase, upstash, weaviate), model providers
(groq, mistral, ollama, perplexity, sambanova), and tools/memory/data (apify,
assemblyai, confluence, firecrawl, git, glean, icosacomputing, mem0, needle,
scrapegraph, serpapi, unstructured, youtube, zep).

Dep verification (the careful part): every spec comes from langflow-base's
per-provider extras or its direct dependencies; langchain-community providers
carry the wrapper plus the SDK the wrapper lazy-imports (e.g. pgvector,
atlassian-python-api); requests is declared explicitly where imported (it is
only transitive in today's env); pinecone keeps its python_version<'3.14'
marker verbatim. Tranche excludes: providers with langflow imports (vlmrun),
provider-specific lfx.base dirs (composio/huggingface/langwatch), case-
sensitive names (FAISS/Notion), the openai-SDK family (azure/aiml/deepseek/
litellm/lmstudio/novita/openrouter/vllm/xai/cometapi -- cleaner after PR-8),
and the partner set.

Dep parity verified at the resolution level: the uv.lock diff is +220 lines of
lfx-bundles extras metadata with ZERO packages added or removed (`name =` diff
empty), so pip install langflow resolves the identical set.

Also: 192 append-only migration entries (48 classes x 4, zero bare-name
ambiguities); component_index.json regenerated 348->300 components / 90->60
modules (exactly the moved set, no stale standalone entries); mongodb_atlas.py
SLF001 per-file-ignore and the mem0/mongodb detect-secrets baseline entries
migrated to the new paths (lint/secrets exceptions travel with moved files);
35 bundles now discover at @official with 55 components; shims verified across
categories; 449 extension tests pass.

* feat(bundles): consolidate openai-SDK family tranche 3 into lfx-bundles

10 providers that ride the langchain-openai wrapper, deferred from tranche 2
until the partner graduation settled the openai-SDK dep story: aiml, azure,
cometapi, deepseek, litellm, lmstudio, novita, openrouter, vllm, xai.

Dep verification: every provider declares langchain-openai>=1.1.6; the openai
SDK is declared only where a component imports it directly (aiml, deepseek,
litellm, lmstudio, vllm, xai -- wrapper-transitive elsewhere); requests
declared where imported (cometapi, deepseek, novita, xai); lmstudio's lazy
NVIDIAEmbeddings path gets langchain-nvidia-ai-endpoints~=1.0.0. The litellm
component drives LiteLLM-served endpoints through the OpenAI client and does
NOT import the litellm package -- langflow-base's litellm extra stays put.
These providers use the lazy _dynamic_imports __init__ shape; it survives the
move unchanged (import_mod resolves via __spec__.parent and
lfx.components._importing remains a core helper).

Dep parity: uv.lock diff has zero package additions/removals (all specs
already resolved via langflow-base[complete]).

Also: 56 append-only migration entries (14 classes x 4, zero ambiguities);
component_index.json regenerated 300->286 components / 60->50 modules (exactly
the moved set); detect-secrets baseline re-keyed; 45 bundles now discover at
@official with 69 components; shims verified (azure/xai/litellm/deepseek);
ruff clean.

* fix(ci): nightly bundle rename follows [extras] refs and self-refs

Two gaps in update_bundle_versions.py around extras suffixes, both fatal
or silently wrong for the first nightly carrying the lfx-bundles
metapackage:

- update_root_pyproject_for_bundle's dep regex required the version
  specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0"
  (a MAIN dep) was left unrewritten while the workspace member was
  renamed to lfx-bundles-nightly; uv lock then tries to resolve stable
  lfx-bundles from PyPI, where it does not exist. The same gap silently
  left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the
  stable distribution. The regex now tolerates an [extras] group and
  carries it into the replacement.

- rename_bundle_pyproject skipped self-referencing extras, so the
  metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]"
  members after the rename, pulling the stable distribution (same
  lfx_bundles import package, install collision) once published. Self
  refs now follow the rename, idempotently.

Tests drive the real script module, mirroring test_bundle_lfx_pin.py.
This closes the PR-2 audit item flagged when the metapackage was
introduced. The canonical-pre-release cutover would retire the nightly
rename entirely; until it lands, the rename must be correct.

* feat(bundles): graduate partner set to standalone lfx-<provider> packages (#13573)

* feat(bundles): graduate partner set to standalone lfx-<provider> packages

Extracts the five partner/flagship providers into manifest-shipping
distributions: lfx-openai, lfx-anthropic, lfx-amazon, lfx-datastax,
lfx-cohere. Each ships extension.json (lfx.compat ["1"]), a
langflow.extensions entry-point, an lfx>=1.10.0,<2.0.0 pin, and is wired into
the root workspace marker blocks. Zero flow impact by the bundle-name
invariant: ext:<provider>:<Class>@official ids are unchanged.

Mechanics:
- adopts the five-phase scripts/migrate/port_bundle.py from
  feat/bundle-mass-extraction (handles shared-base moves, consumer rewrites,
  surgical index updates, migration-table appends) and runs it per partner
  with --migration-release 1.11.0
- datastax's shared lfx.base.datastax moves into lfx_datastax.base; its
  backend unit tests move to src/bundles/datastax/tests (74 pass); its ruff
  per-file-ignore and its check_component_env_writes ALLOWLIST entry travel;
  10 repo consumers rewritten (incl. the vector_store_rag starter project)
- amazon_bedrock_converse.py legacy langflow.* imports rewritten to the lfx.*
  equivalents (thin re-export aliases; behavior-identical) pre-extraction
- runtime deps pinned from langflow-base's extras / direct deps (wrapper-
  guaranteed SDKs stay transitive, parity-exact); --remove-base-extra dropped
  the openai/anthropic/cohere/aws extras from langflow-base[complete] in favor
  of the bundle pins; the astradb extra stays (also used outside datastax)
- in-tree dirs replaced with marker shims pointing at lfx_<p>.components.<p>
  (skipped by the in-tree walk; legacy from lfx.components.<p> imports keep
  working); lfx/components/__init__.py entries kept, consistent with the
  consolidated providers
- validate.py: accept classes whose base is a *derived* Component base
  (LCVectorStoreComponent / LCToolComponent / ...) -- they inherit the
  class-level outputs declaration and only override the output method, which
  the AST-only check could not see; bare Component subclasses keep the strict
  build/outputs requirement (+ regression test)
- BUNDLE_API.md changelog entries added for the branch's surface changes: the
  lfx.bundles manifest-less discovery group + precedence tier +
  bundle-discovery-malformed code (from the foundation PR) and the validator
  acceptance above
- 100 append-only migration entries (20 classes x 5 partners x 4 shapes);
  component_index.json surgically updated 300->280 components / 60->55
  modules, sha256 recomputed and verified
- detect-secrets baseline re-keyed for moved partner files

Dep parity: lock diff adds only the five lfx-* package names; no third-party
package added or removed. One benign transitive re-resolution: anthropic SDK
0.105.2 -> 0.109.0 (allowed by langchain-anthropic's range on both sides).

Verified: all five register at @official via the installed-manifest tier
(extension_id/distribution = lfx-<p>, 20 components); manifest-less
lfx_bundles unchanged (35 bundles, disjoint); all 5 shims import; engine-safe;
`lfx extension validate` passes for all five; 450 extension unit tests, 60
pilot-upgrade migration tests, 74 moved datastax tests pass; ruff clean.

* fix(bundles): shim lfx.base.datastax so stored-flow imports keep resolving

The datastax graduation moved lfx.base.datastax into lfx_datastax.base,
but saved flows and starter templates (Hybrid Search RAG,
TemplateAssistant) embed `from lfx.base.datastax.astradb_base import
AstraDBBaseComponent` inside their stored component code fields, which
is re-executed verbatim at flow build time. Without a shim that import
raises ModuleNotFoundError and the flows fail to build
(test-starter-projects red).

Mirror the lfx.components shim contract: module-aliasing to
lfx_datastax.base, narrow except that only translates a missing bundle
(transitive dep failures re-raise untouched), marker-tagged single-file
dir, removed at M4 together with the components shims.

* chore(bundles): bounded version ranges for curated lfx-* packages (#13576)

langflow's pyproject is the release-coordination point now that lfx and the
lfx-* bundles release independently. Every curated lfx-* dependency declares a
BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library
metadata create resolver conflicts for downstreams that depend on a different
version. Exact pins for reproducibility live in uv.lock and the release-build
manifests.

- the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles
  follow their own 0.1.x cadence)
- lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage
  contract)
- the three lfx-docling[...] optional-dependency refs bounded the same way
- policy documented inline in the bundle-deps marker block; the cross-bundle
  CI matrix verifies the ranges resolve together across the supported lfx axis

Verified: uv lock resolves with zero package/version changes (bounds are
metadata-only today); the nightly rename's dep regex already matches the
bounded form.

* test(bundles): lock the in-tree shim contract + breakage audit (#13577)

* chore(bundles): bounded version ranges for curated lfx-* packages

langflow's pyproject is the release-coordination point now that lfx and the
lfx-* bundles release independently. Every curated lfx-* dependency declares a
BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library
metadata create resolver conflicts for downstreams that depend on a different
version. Exact pins for reproducibility live in uv.lock and the release-build
manifests.

- the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles
  follow their own 0.1.x cadence)
- lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage
  contract)
- the three lfx-docling[...] optional-dependency refs bounded the same way
- policy documented inline in the bundle-deps marker block; the cross-bundle
  CI matrix verifies the ranges resolve together across the supported lfx axis

Verified: uv lock resolves with zero package/version changes (bounds are
metadata-only today); the nightly rename's dep regex already matches the
bounded form.

* test(bundles): lock the in-tree shim contract + breakage audit

Locks the five-point contract for the 50 lfx-bundles import shims under
lfx/components/ (45 metapackage + 5 graduated partners):

  1. first line is the `# lfx-bundles-shim` marker (keys the in-tree walk's
     double-registration skip);
  2. a shim dir is exactly one __init__.py -- no impls, no deps;
  3. sys.modules module-aliasing (submodule trees resolve);
  4. bundle missing -> actionable ModuleNotFoundError whose wording is part
     of the contract ("pip install lfx-bundles" / "pip install lfx-<p>");
  5. a missing *transitive* dep re-raises untouched.

Test shape (env-adaptive by design -- the lfx test env prunes the venv, so
nothing here assumes bundles are installed):
- source-level sweep of every real shim, parameterized (no imports);
- hermetic mechanism tests against a synthetic shim + synthetic target
  (alias-resolves / locked-message / transitive-reraise);
- the walk-skip detector and the marker sweep must agree exactly;
- one adaptive live test on lfx.components.tavily exercising whichever
  branch the running env is in.

106 tests pass in the pruned lfx env (where the live test exercises the
bare-engine locked-message branch).

Breakage audit (gh code search, recorded in the PR): ~20-25 public repos
reference lfx.components.*; the majority are langflow forks (vendored tree,
unaffected by packaging). Genuine library consumers exist and the moved
providers have real surface (openai: 11 external repos, datastax: 17) --
covered by the shims wherever bundles are co-installed (every
`pip install langflow`). Decision: deprecation warnings NOT warranted now;
revisit at M4 (shim removal), where one loud minor release before removal
is the right shape.

* test(bundles): extend the shim contract sweep to lfx.base shims

The datastax graduation moved lfx.base.datastax into lfx_datastax.base
and left a module-aliasing shim behind (stored flow code fields embed
the legacy import and are re-executed verbatim at build time). Sweep
lfx/base for marker shims with the same source-level contract as the
components sweep: one-file stub, module-aliasing to the bundle's base
subpackage, narrow name-checked except, locked install message.

* docs(bundles): install shapes, override rule, bundle_api_version (#13578)

Documents the deliberately small user-facing rule surface of the metapackage
split (1.11):

- extensions-overview.mdx: the two install stories (pip install langflow =
  everything, same as today; pip install lfx = engine only, bring your own
  bundles); the bundle-name-is-identity invariant; the current package table
  (lfx-bundles metapackage + the 5 partner packages + the 4 pilots); the
  legacy-import shim behavior with the locked ModuleNotFoundError wording and
  the migration recipe for direct lfx users (switch to lfx[bundles] or pin
  lfx-<provider> packages); the override rule ("ship a manifest -- a manifest
  always wins", with the bundle-shadowed warning); bundle_api_version as the
  single compat number (lfx.compat ["1"]; manifest-less packages ride their
  PEP 508 lfx pin) and the no-version-arithmetic rule.
- deployment-lfx-compatibility.mdx: lfx[bundles] as the headless/serverless
  deployment footnote (engine + lfx-bundles[all]; slimmer per-provider
  alternative; intentionally no lfx[all]).

Both files verified MDX-safe (no unbackticked angle brackets).

* fix(bundles): stack-review fixes — symlink containment, idempotency, docs accuracy (#13579)

* chore(bundles): bounded version ranges for curated lfx-* packages

langflow's pyproject is the release-coordination point now that lfx and the
lfx-* bundles release independently. Every curated lfx-* dependency declares a
BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library
metadata create resolver conflicts for downstreams that depend on a different
version. Exact pins for reproducibility live in uv.lock and the release-build
manifests.

- the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles
  follow their own 0.1.x cadence)
- lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage
  contract)
- the three lfx-docling[...] optional-dependency refs bounded the same way
- policy documented inline in the bundle-deps marker block; the cross-bundle
  CI matrix verifies the ranges resolve together across the supported lfx axis

Verified: uv lock resolves with zero package/version changes (bounds are
metadata-only today); the nightly rename's dep regex already matches the
bounded form.

* test(bundles): lock the in-tree shim contract + breakage audit

Locks the five-point contract for the 50 lfx-bundles import shims under
lfx/components/ (45 metapackage + 5 graduated partners):

  1. first line is the `# lfx-bundles-shim` marker (keys the in-tree walk's
     double-registration skip);
  2. a shim dir is exactly one __init__.py -- no impls, no deps;
  3. sys.modules module-aliasing (submodule trees resolve);
  4. bundle missing -> actionable ModuleNotFoundError whose wording is part
     of the contract ("pip install lfx-bundles" / "pip install lfx-<p>");
  5. a missing *transitive* dep re-raises untouched.

Test shape (env-adaptive by design -- the lfx test env prunes the venv, so
nothing here assumes bundles are installed):
- source-level sweep of every real shim, parameterized (no imports);
- hermetic mechanism tests against a synthetic shim + synthetic target
  (alias-resolves / locked-message / transitive-reraise);
- the walk-skip detector and the marker sweep must agree exactly;
- one adaptive live test on lfx.components.tavily exercising whichever
  branch the running env is in.

106 tests pass in the pruned lfx env (where the live test exercises the
bare-engine locked-message branch).

Breakage audit (gh code search, recorded in the PR): ~20-25 public repos
reference lfx.components.*; the majority are langflow forks (vendored tree,
unaffected by packaging). Genuine library consumers exist and the moved
providers have real surface (openai: 11 external repos, datastax: 17) --
covered by the shims wherever bundles are co-installed (every
`pip install langflow`). Decision: deprecation warnings NOT warranted now;
revisit at M4 (shim removal), where one loud minor release before removal
is the right shape.

* docs(bundles): install shapes, override rule, bundle_api_version

Documents the deliberately small user-facing rule surface of the metapackage
split (1.11):

- extensions-overview.mdx: the two install stories (pip install langflow =
  everything, same as today; pip install lfx = engine only, bring your own
  bundles); the bundle-name-is-identity invariant; the current package table
  (lfx-bundles metapackage + the 5 partner packages + the 4 pilots); the
  legacy-import shim behavior with the locked ModuleNotFoundError wording and
  the migration recipe for direct lfx users (switch to lfx[bundles] or pin
  lfx-<provider> packages); the override rule ("ship a manifest -- a manifest
  always wins", with the bundle-shadowed warning); bundle_api_version as the
  single compat number (lfx.compat ["1"]; manifest-less packages ride their
  PEP 508 lfx pin) and the no-version-arithmetic rule.
- deployment-lfx-compatibility.mdx: lfx[bundles] as the headless/serverless
  deployment footnote (engine + lfx-bundles[all]; slimmer per-provider
  alternative; intentionally no lfx[all]).

Both files verified MDX-safe (no unbackticked angle brackets).

* fix(bundles): review fixes — symlink containment, idempotency, docs accuracy

Fixes from the multi-agent stack review (kept as a separate PR so the
reviewed PRs' diffs stay frozen for QA's independent pass):

- _bundles_root.py: directory-level symlink containment — a provider dir
  that resolves outside the bundle root is skipped with a typed
  bundle-discovery-malformed warning, mirroring the seed-directory walk's
  is_within rule (+ regression test). Anchors the trust boundary to the
  installed package tree.
- consolidate_bundles.py: migration-append idempotency guard — entries are
  deduped on full content so a partially-failed earlier run cannot duplicate
  rows on re-run (table stays append-only).
- langflow-base pyproject: documented WHY the aws extra is deliberately
  retained after the lfx-amazon graduation (boto3 also backs the server's
  own S3 storage backend and lfx's S3 ingestion — review suggested removing
  it; that would regress S3 storage support).
- extensions-overview.mdx: note that `lfx extension list` shows
  manifest-shipping extensions only; manifest-less packages load at startup
  but are not listed.

93 loader+migration tests pass (incl. the new symlink test); ruff clean.

* refactor(bundles): stabilize import_mod as a public BUNDLE_API utility

The lazy-import helper that bundle packages call from their
__getattr__-based __init__.py files lived at lfx.components._importing -- an
internal path with no stability contract, imported by 35 separately-installed
bundle __init__ files (30 lfx-bundles providers + the 5 graduated partners).

- canonical home is now lfx.utils.lazy_import.import_mod (code unchanged);
  lfx.components._importing re-exports it so in-tree callers and any external
  code on the old path keep working
- all 35 bundle-side imports rewritten to the stable path
- BUNDLE_API.md: surface table entry + changelog (additive)
- contract tests: re-export identity, both call forms, the AttributeError
  conversion

Verified: identity holds across both paths; lazy __init__ loads work through
the new path for both bundle families; 110 tests pass; ruff clean.

* fix(bundles): tombstone the broken legacy ZepChatMemory build method (#13580)

build_message_history targeted the zep-python v1 SDK (ZepClient +
zep_python.langchain.ZepChatMessageHistory); both were removed in
zep-python 2.x and the zep extra pins 2.0.2, so the method has been
unable to run for as long as the pin has existed -- its ImportError
guard misleadingly told users to 'pip install zep-python' (already
installed). The component is legacy=True with helpers.Memory as its
designated replacement, so rather than hand-write a new integration
against the 2.x SDK, the method now raises a clear RuntimeError
pointing at the Message History component.

Flow identity is preserved: class/component name, display_name,
description, inputs and the memory output are byte-identical, so saved
flows keep loading, i18n locale keys are unchanged, and
migration_table.json needs no edits. New bundle tests pin the stub
contract (identity, actionable error, no zep_python import).

* test(bundles): extend the shim contract sweep to lfx.base shims

The datastax graduation moved lfx.base.datastax into lfx_datastax.base
and left a module-aliasing shim behind (stored flow code fields embed
the legacy import and are re-executed verbatim at build time). Sweep
lfx/base for marker shims with the same source-level contract as the
components sweep: one-file stub, module-aliasing to the bundle's base
subpackage, narrow name-checked except, locked install message.

* test(bundles): extras-drift guard for the lfx-bundles metapackage

Risk-2 of the metapackage split: the generated `all` extra and the
per-provider extras must never drift by hand-edit, or `pip install
langflow` silently loses a provider's deps. Guard the four invariants:
extras <-> provider dirs (PEP 685-normalized), `all` == the exact
self-ref set, normalized keys collision-free, and the metapackage
provider set disjoint from the graduated partner distributions.

* fix(extension): symlink-escaped providers reject with path-escape, matching the documented contract

The lfx.bundles provider containment check (stack-review symlink fix)
emitted bundle-discovery-malformed, whose template and hint describe a
broken entry-point declaration.  The changelog's path-safety entry
already documents symlink escapes as path-escape on every other
discovery path; use the same code here.  Also resolves the semantic
merge conflict with the per-mode code split (the removed
_malformed_error location kwarg).

* fix(tests): repoint lfx tests off moved providers; complete provider fallback map

CI fail-fast had been masking these: the LFX test job runs in an
engine-only env where openai/anthropic/chroma are now bundle-package
shims, so every test that used them as the example category failed on
all Python versions (3.14 just reported first), and the backend Group 2
leg failed collecting test_lfx_bundles_extras.py on Python 3.10.

- flow_requirements: complete _PROVIDER_PACKAGE_FALLBACKS for the nine
  moved model providers (OpenAI, Anthropic, Amazon Bedrock, Groq,
  Google Generative AI, SambaNova, IBM watsonx.ai, Ollama + existing
  Azure OpenAI). In engine-only installs MODEL_PROVIDERS_DICT registers
  only in-tree providers, so the dynamic source-inspection path cannot
  resolve moved providers; the static fallbacks keep lfx run/serve
  requirements inference working. A dict hit still wins.
- test_dynamic_imports / test_import_utils: use composio (still-in-tree
  lazy category with its SDK absent in the bare env) as the example
  category instead of openai/anthropic/chroma; patch import_module at
  its canonical home lfx.utils.lazy_import.
- flow-builder tests (build_flow_from_spec, flow_builder_tools,
  propose_field_edit): specs use LanguageModelComponent (in-tree)
  instead of OpenAIModel.
- test_lfx_bundles_extras: tomli fallback for Python 3.10 (tomllib is
  stdlib 3.11+; pytest guarantees tomli on <3.11).

Full lfx unit suite: 4550 passed. Backend extras+pin tests: 24 passed.

* fix(tests): repoint MCP client-server tests off graduated OpenAIModel key

Same fail-fast-masked class as 14dfa7f268, backend side: since the
partner graduation, extension components register in /api/v1/all under
their namespaced ids (ext:openai:OpenAIModelComponent@official), so the
bare 'OpenAIModel' key these tests hardcoded no longer exists on any
Python version (the 'py3.14-only' Group 2 failure was the only leg that
ran; 3.10 was cancelled by fail-fast). The MCP feature itself is fine —
search returns the namespaced ids and add_component accepts them.

- Use LanguageModelComponent (in-tree, stable bare key; SecretStr
  api_key, real_time_refresh fields, advanced 'stream', LanguageModel
  output) for the redaction/configure/search/describe/spec tests.
- configure_dynamic_field: model_name -> api_key (the refresh field on
  LanguageModelComponent).
- prompt-template-variables spec: drop the model node — server-side
  validation builds the graph and a model without an API key fails its
  build; the test's subject is the dynamic {var} fields.

Full file: 69 passed.

* Update test_mongodb_atlas.py

* Frontend tests

* Frontend test updates

* fix: restore loading.py and tableAutoCellRender to base-branch state

Commit 9df9475615 ('Frontend test updates') accidentally reverted both
files to their pre-#9902 state, removing the no_env_fallback contract in
load_from_env_vars / update_table_params_with_load_from_db_fields, the
defensive params.pop('code') in build_custom_component, and the
localValue/updateGlobalVariableCell handling in tableAutoCellRender.
The base branch's regression tests (test_loading_no_env_fallback.py,
test_loading_custom_component_code_param.py, tableAutoCellRender
index.test.tsx) were kept, so the reversion failed them.

* ci(bundles): make freeze gate reachable on PRs; review follow-ups

- Move the freeze-components job from lint-py.yml (workflow_call/
  workflow_dispatch only -- never invoked, so the gate could not block a
  PR) into extension-migration-checks.yml, which already triggers on
  pull_request for src/lfx/src/lfx/components/**. Add the gate's script
  and baseline to the path filter.
- Add timeout-minutes to the cross-bundle-test jobs so a hung install or
  test leg cannot occupy a runner for the 360-minute default.
- _discover_shimmed_component_dirs: read only the first line of each
  __init__.py instead of the whole file -- the shim marker contract is
  line 1 (enforced by test_shim_source_contract), and this prevents a
  non-shim file with the marker after leading blank lines from being
  misclassified and silently skipped from the in-tree walk.

* fix(bundles): restore legacy-name resolution for ext components; sync starter projects

Saved flows reference moved providers by their legacy palette identity --
either the bare class name (TavilySearchComponent) or the component's
name attribute (AstraDB, Chroma, needle, OllamaModel). After the provider
move neither resolved to a current template on the backend:

- lfx.utils.component_aliases now derives the bare class name (and its
  Component-stripped form) from ext:<bundle>:<Class>@<slot> keys,
  mirroring the frontend's getTemplateAliases. Ext templates carry
  name=None / _type='Component', so the key is the only source.
- _decorate_template_with_extension now stamps template['name'] with the
  component's legacy identity (name attr, falling back to class name).
  In-tree palette entries expose it as the dict key; ext entries are
  keyed by namespaced id, so without this both alias maps lose the only
  bridge from node types like 'AstraDB' to the current template.

Without these, the starter-project updater could not match moved-provider
nodes, so their embedded code stayed one whitespace change behind the
bundle sources -- every such flow showed the update-all banner, and
because tool-mode nodes' saved outputs ([component_as_tool]) differ from
the template's natural outputs the update was flagged breaking, opening
the confirmation modal instead of toasting 'successfully updated':
deterministic timeouts in the starter-project Playwright specs (shards
29/30/36/37). Regenerated the 13 affected starter projects with
scripts/ci/update_starter_projects.py (idempotent; code_hash hex strings
in the regenerated JSONs are detect-secrets false positives).

Also converts test_groq_integration.py to the captured-module
patch.object pattern (same module-identity trap as test_mongodb_atlas:
dotted-name @patch can land on a different materialization of the moved
module than the one that defined the class, leaving the real
get_groq_models in the instance's globals -- order-dependent failure in
unit-test Group 5).

* fix(bundles): keep importable module paths in starter projects

The starter-project updater syncs node metadata from the live templates
(NODE_FORMAT_ATTRIBUTES includes 'metadata'), and ext components report
their runtime sys.modules namespace there (_lfx_ext.official.<p>.<m>) --
a path that only exists inside a running extension loader. Persisting it
broke test_template_field_order_matches_component in both template jobs
('No module named _lfx_ext'): the test imports metadata.module to
instantiate the component, and the migration table likewise keys on the
legacy lfx.components.<provider> form that the bundle shims keep
importable.

_merge_node_metadata now preserves the node's stored module when the
live template's is a runtime _lfx_ext.* path, while still syncing the
rest of the metadata (e.g. dependencies). Starter projects regenerated
from the pre-pollution state with the fixed updater: the only diff vs
the previous regen is the 20 module lines reverting to the legacy
importable form. Regression tests added for both merge directions.

* feat: add Oracle integration (#13502)

* Add Oracle Integration

* [autofix.ci] apply automated fixes

* Fix styleUtils problem

* fix(oracledb): address review feedback

* fix(oracledb): apply formatting

* Fix component index

* refactor(oracledb): convert Oracle integration to lfx-oracle extension bundle

Port the in-tree lfx.components.oracledb provider into a standalone
Extension Bundle distribution (src/bundles/oracle, dist lfx-oracle,
package lfx_oracle, bundle name 'oracle') per src/bundles/PORTING.md:

- Move the four components + connection helper into the bundle; deps
  (oracledb, langchain-oracledb, langchain-community) move from the
  langflow-base[oracledb] extra into the bundle's pyproject
- Add migration-table entries mapping legacy bare class names and
  lfx.components.oracledb.* import paths to ext:oracle:<Class>@official
- Add test_pilot_oracle_upgrade.py integration coverage (mirrors ibm)
- Move backend unit tests into src/bundles/oracle/tests with bundle
  import paths; drop the empty ComponentTestBaseWithoutClient
  version-fixture scaffolding (component is new in 1.11.0)
- Declare explicit outputs on OracleVectorStoreComponent (mirrors
  LCVectorStoreComponent) so lfx extension validate passes
- Regenerate component index (oracledb category removed) and uv.lock;
  wire workspace member/dep/source in root pyproject
- Frontend sidebar entry renamed oracledb -> oracle; docs page renamed
  to bundles-oracle following the extracted-bundle convention

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>

* feat(lfx): fail-fast dependency preflight for `lfx run` (#13680)

Detect a flow's required third-party packages before the graph load and
fail fast with an actionable `pip install ...` message when any are
missing, instead of a deep ModuleNotFoundError mid-build. Addresses the
'export a flow, run it on a bare pip install lfx, it fails' confusion
introduced by the bundle split.

- flow_requirements: add find_missing_dependencies / _is_installed /
  format_missing_dependencies_error (best-effort, reuses the existing
  flow analyzer).
- run.base: _preflight_dependencies helper + check_dependencies gate in
  run_flow (JSON flows only; .py uses PEP 723; fail-open on analysis
  errors so a runnable flow is never blocked by the check itself).
- CLI: --check-dependencies/--no-check-dependencies on `lfx run`.
- tests: unit (find_missing_dependencies/format) + integration (run_flow
  + _preflight_dependencies).

* feat(bundles): move spider + toolguard provider components into lfx-bundles

Two provider components were still living inside otherwise-core directories
(file-level stragglers from the metapackage split). Both are portable
(lfx-only imports) and gated by dedicated optional SDKs, so they belong in
the lfx-bundles metapackage rather than lfx core:

- SpiderTool (langchain_utilities/spider.py) -> lfx_bundles/spider/, dep
  spider-client (`spider` extra).
- PoliciesComponent + policies/ submodule (models_and_agents) ->
  lfx_bundles/toolguard/, dep toolguard (`toolguard` extra).

Changes:
- Move the files into lfx_bundles/<name>/; rewrite policies' absolute
  self-imports (lfx.components.models_and_agents.policies -> lfx_bundles.toolguard.policies).
- Drop SpiderTool / PoliciesComponent from the core langchain_utilities /
  models_and_agents lazy-import registries.
- Add `spider` and `toolguard` extras to lfx-bundles pyproject + the generated
  `all` aggregate; uv.lock re-resolved (uv lock --check clean).
- Append migration entries (4 per class) mapping the historical
  lfx.components.* paths to ext🕷️SpiderTool@official /
  ext:toolguard:PoliciesComponent@official so saved flows migrate.
- Regenerate component_index.json (both classes drop out of core; 263
  components / 45 categories).
- Repoint the toolguard policies tests to lfx_bundles.toolguard (60 pass).

The toolguard deadlock-warmer (_warm_circular_imports) is unaffected: it warms
toolguard.runtime directly and runs single-threaded before the unified
component fan-out that loads bundle components.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* feat(bundles): consolidate cleanlab, twelvelabs, jigsawstack into lfx-bundles

First directory-level batch of the core-tail consolidation (validates the
consolidate_bundles.py path end-to-end after the spider/toolguard file-level
extractions). All three are flat, lfx-only, single-SDK providers:

- cleanlab    -> lfx_bundles/cleanlab    (cleanlab-tlm)   3 components
- twelvelabs  -> lfx_bundles/twelvelabs  (twelvelabs)     7 components
- jigsawstack -> lfx_bundles/jigsawstack (jigsawstack)   11 components

Via scripts/migrate/consolidate_bundles.py --apply: moves each dir into the
metapackage + leaves a fail-soft shim, merges per-provider extras and regen
`all`, appends the 4-entry migration block per class (84 entries, 0 ambiguous).
Component index regenerated (45->42 categories, 263->242 core components);
uv.lock re-resolved; twelvelabs test repointed to lfx_bundles.twelvelabs (passes).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(bundles): consolidate vector/datastore + community-wrapper providers into lfx-bundles

Second directory-level batch (8 providers, 12 components) via
consolidate_bundles.py --apply:

- baidu (qianfan + langchain-community), redis (redis + community),
  elastic (elasticsearch + langchain-elasticsearch + opensearch-py)
- bing, cloudflare, maritalk, searchapi, vectara (langchain-community REST wrappers)

Moves each dir into lfx-bundles + fail-soft shim, merges per-provider extras
and regen `all`, appends 4-entry migration block per class (48 entries, 0
ambiguous). Component index regenerated (42->34 categories, 242->230 core
components); uv.lock re-resolved.

Test fixups:
- Repoint baidu/elastic/searchapi backend tests to lfx_bundles.<provider>.
- test_dynamic_imports (lfx isolated suite) used the now-bundled searchapi as
  its fixture; bundles aren't installed in the engine-only lfx env, so switch
  it to the core langchain_utilities.FakeEmbeddingsComponent (same
  langchain_community optional-import path). 131 shim tests + 21 dynamic-import
  tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [autofix.ci] apply automated fixes

* feat(bundles): consolidate homeassistant, olivya, agentql REST providers into lfx-bundles

Third batch (3 providers, 4 components) via consolidate_bundles.py --apply.
These are thin REST wrappers with no dedicated vendor SDK:

- homeassistant (requests) -> lfx_bundles/homeassistant   2 components
- olivya (httpx, lfx core)  -> lfx_bundles/olivya          1 component
- agentql (httpx, lfx core) -> lfx_bundles/agentql         1 component

Moves + shims + per-provider extras + `all` regen; 16 migration entries (0
ambiguous). Component index 34->31 categories, 230->226 components; uv.lock
re-resolved. No tests reference these providers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(bundles): consolidate google, vertexai, altk, codeagents into lfx-bundles

Fourth directory-level batch (4 providers, 14 components) via
consolidate_bundles.py. google/vertexai (langchain-google-*), altk
(agent-lifecycle-toolkit), codeagents (smolagents + OpenDsStar); platform/py
markers copied verbatim from langflow-base extras. 56 migration entries (0
ambiguous); index 31->27 categories, 226->212 components. Backend tests
repointed; starter projects embedding moved components regenerated via the
validator. SKIP=detect-secrets: only flags are regenerated code_hash hex
(false positives).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(lfx): decouple dynamic-import unit tests from soon-to-be-bundled providers

test_dynamic_imports.py and test_import_utils.py used composio/nvidia/huggingface
as the "lazy category whose SDK is absent in the engine-only lfx env" fixture.
That couples the import-mechanism tests to specific providers — they have broken
each time a provider graduated to a bundle (openai/anthropic/chroma -> composio,
now composio et al.).

Switch to stable in-tree core categories:
- files_and_knowledge.KnowledgeComponent for the "missing-dep" path (its module
  imports langchain_chroma / langflow, absent in bare lfx) — works for both the
  import_mod direct calls and the lfx.components hierarchy access.
- helpers / processing for the "imports cleanly" and "module not found" paths.

No behavior change; 43 tests pass in the isolated lfx env. This unblocks moving
composio, huggingface, nvidia, cuga into lfx-bundles without breaking these tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(bundles): consolidate composio, huggingface, nvidia, cuga into lfx-bundles

Fifth batch (4 providers, 71 components) via consolidate_bundles.py --apply,
unblocked by the prior dynamic-import test decoupling:

- composio    -> lfx_bundles/composio    (composio + composio-langchain)  63 components
- huggingface -> lfx_bundles/huggingface (langchain-huggingface[marker], huggingface-hub, community)  2 components
- nvidia      -> lfx_bundles/nvidia      (langchain-nvidia-ai-endpoints, nv-ingest-client[py3.12], gassist[win32])  5 components
- cuga        -> lfx_bundles/cuga        (cuga; platform/py markers)  1 component

Markers copied verbatim from langflow-base extras (uv lock --check clean).
284 migration entries (0 ambiguous); component index 27->23 categories,
212->141 components.

Tests: relocate the dedicated lfx-isolated tests for the now-bundled components
to the backend suite (test_nvidia_component.py, test_cuga_session_id_fallback.py)
repointed to lfx_bundles.<provider>; repoint backend composio/huggingface/cuga
imports + a cuga patch target; pragma a fake test api_key. 43 decoupled lfx
import tests pass post-move. SKIP=detect-secrets: only flags are regenerated
component_index code_hash hex (false positives).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [autofix.ci] apply automated fixes

* feat(bundles): consolidate langwatch evaluator into lfx-bundles

langwatch -> lfx_bundles/langwatch (1 component). The LangWatchComponent is a
pure httpx REST wrapper (its only non-core touchpoint is lfx.base.langwatch.utils,
which is also httpx-based) — the langwatch SDK extra in langflow-base is for the
tracing service, not this component, so the bundle declares no extra deps.

Moves + shim + `all` regen; 4 migration entries (0 ambiguous). Component index
23->22 categories, 141->140 components. Backend langwatch test repointed
(16 passed). SKIP=detect-secrets: only flags are regenerated component_index
code_hash hex (false positives).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(bundles): consolidate FAISS, Notion into lfx-bundles (lowercase-slug support)

Uppercase source dirs need lowercase bundle slugs (BUNDLE_NAME_RE is
lowercase-only). Enhance consolidate_bundles.py with bundle_slug(provider) =
provider.lower(): the bundle dir, ext id, shim target, and extra key use the
lowercased slug, while the migration import_path keeps the historical
lfx.components.<Provider> casing so saved flows still resolve. Identity for
already-lowercase providers (no change to prior entries).

- FAISS  -> lfx_bundles/faiss  (faiss-cpu[py-split] + langchain-community)  1 component
- Notion -> lfx_bundles/notion (requests + Markdown)  8 components

test_bundle_shims.py: derive the expected bundle name by lowercasing the shim
dir name (FAISS shim aliases lfx_bundles.faiss) — 159 shim tests pass.
Relocate the lfx-isolated FAISS test to backend/vectorstores (repointed to
lfx_bundles.faiss.faiss). 36 migration entries (0 ambiguous); index 22->20
categories, 140->131 components. SKIP=detect-secrets: regenerated code_hash hex.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(bundles): fold vectorstores LocalDBComponent into the chroma bundle

The lone LocalDBComponent is Chroma-backed (eager `from langchain_chroma import
Chroma`), so it belongs in the existing `chroma` bundle rather than a new one —
its deps are already covered by the chroma extra (no new deps).

- Move vectorstores/local_db.py -> lfx_bundles/chroma/local_db.py; register
  LocalDBComponent in the chroma bundle's lazy registry.
- Replace lfx.components.vectorstores with a CROSS-BUNDLE shim aliasing
  lfx_bundles.chroma (dir != bundle).
- Migration: lfx.components.vectorstores[.local_db].LocalDBComponent ->
  ext:chroma:LocalDBComponent@official (4 entries, 0 ambiguous).
- test_bundle_shims.py: add a cross-bundle map {vectorstores: chroma} so the
  shim contract validates (161 shim tests pass).
- Repoint the backend LocalDB test (import + patch target) to lfx_bundles.chroma;
  test_all_modules_importable resolves it via the shim.

Component index 20->19 categories, 131->130 components. SKIP=detect-secrets:
regenerated code_hash hex (false positives).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [autofix.ci] apply automated fixes

* feat(bundles): remove unbundleable vlmrun + notdiamond components from lfx core

Per the core-tail audit, four providers couldn't be bundled. vlmrun (imports
`from langflow`, violating the lfx engine boundary) and notdiamond (its
`notdiamond` SDK isn't in the lockfile or any extra, so it can't run) have no
engine references, no starter flows, and no migration-table entries — they're
effectively dead in every distribution, so remove them outright.

(crewai and agentics are kept: crewai is woven into 3 starter-project example
flows, and the core Agent hard-depends on agentics.helpers.model_config — both
need deliberate upstream/refactor work, not deletion.)

- Delete src/lfx/.../components/{vlmrun,notdiamond}/ and drop them from the
  lfx.components top-level registry (TYPE_CHECKING / _dynamic_imports / __all__).
- Regenerate component index (19->17 categories, 130->128 components) and backend
  locales (orphan keys removed; extract_backend_strings --check OK).
- Repoint test_dynamic_imports.test_import_error_handling off the deleted
  notdiamond to the core processing category (43 lfx import tests pass).

SKIP=detect-secrets: only flags are regenerated component_index code_hash hex.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(lfx): tie flow-builder search 'returns all' to registry size

The hardcoded `count > 100` assertion predated the bundle-extraction
work, which deliberately moves non-legacy components out of lfx core
(now 66 non-legacy of 128 indexed). Assert the empty-query count equals
the full non-legacy registry instead — the durable 'no query == every
component the registry exposes' contract that survives future bundle
moves.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(bundles): deterministic component alias resolution + sidebar slug casing after bundle move

Moving agentql/composio and the long tail into lfx-bundles made two
shared-label resolutions order-dependent and left two sidebar slugs
mis-cased, surfacing as red CI on the starter-projects template tests
and the bundles-sidebar e2e.

- flatten_components_with_aliases: register every component's identity
  aliases (registry key / ext class name / name) before any component's
  display_name alias, so a component's own class identity always wins
  its key regardless of registry iteration order. A single setdefault
  pass previously let the Composio 'AgentQL'/'SerpAPI' wrappers
  (display_name) shadow the standalone components (class name) depending
  on load order, so update_starter_projects wrote the wrong
  metadata.module into the AgentQL starter node.
- Correct the AgentQL node module in News Aggregator.json and Price Deal
  Finder.json back to lfx.components.agentql.agentql_api.AgentQL.
- SIDEBAR_BUNDLES: lowercase the Notion/FAISS name slugs to match the
  lowercase bundle category names so both bundles render in the sidebar
  again (display_name and icons unchanged).
- Add order-independence regression tests for the identity/display tiers.

* test(weaviate): patch the class's own module, not a fixed string path

The Weaviate component moved into lfx-bundles, so weaviate.py is now
reachable under several module identities (the lfx.components.weaviate
shim, lfx_bundles.weaviate.weaviate, and the runtime _lfx_ext.* copy).
On Python 3.10, unittest.mock resolves a dotted patch target via a
getattr walk that follows the shim's package alias to a *different*
module object than the one the imported class actually lives in, so
patching 'lfx.components.weaviate.weaviate.WeaviateVectorStore' missed
the object the component body reads -> 'WeaviateVectorStore called 0
times' when an earlier test in the group forced the alternate identity.

Resolve the module from the imported class (sys.modules[Cls.__module__])
and patch.object it directly, bypassing dotted-string resolution. Same
fix for AuthApiKey. Order/identity-independent on all Python versions.

* test(ollama,perplexity): patch the class's own module, not a fixed string path

The Ollama and Perplexity components moved into lfx-bundles, so ollama.py
and perplexity.py are now reachable under several module identities (the
lfx.components.<x>.<x> shim, the canonical lfx_bundles.<x>.<x>, and the
runtime _lfx_ext.* ext-loader copy). On Python 3.10, unittest.mock
resolves a dotted patch target via a getattr walk that follows the shim's
package alias to a *different* module object than the one the imported
class / component body actually use, so patching e.g.
"lfx.components.ollama.ollama.ChatOllama" missed the object the body reads
-> "mock called 0 times" depending on suite order.

Resolve the module from the imported class (sys.modules[Cls.__module__])
and patch.object it directly, bypassing dotted-string resolution:
- perplexity: ChatPerplexity -> patch.object(_PERPLEXITY_MODULE, ...)
- ollama: ChatOllama / logger -> patch.object(_OLLAMA_MODULE, ...)
- ollama: ChatOllamaComponent._parse_json_response ->
  patch.object(ChatOllamaComponent, ...) (the imported class itself)
- ollama: httpx.AsyncClient.get/post -> patched on the global httpx
  (a single class object, identity-independent), dropping the shim prefix

Same fix as the Weaviate test (064069b6bb). Order/identity-independent on
all Python versions; the langchain_ollama / lfx.utils.util / socket
patches target stable modules and are left unchanged.

* fix(locales): restore en.json keys for still-shipping spider/policies components

The spider (SpiderTool) and toolguard (PoliciesComponent) components live in
lfx-bundles and still ship in the palette, but their 44 en.json keys were
pruned while all 6 translated locales (de/es/fr/ja/pt/zh-Hans) retained them.
The extractor walks only lfx.components, so bundle-only components with no
compat shim get dropped from en.json on regeneration. Left as-is, the next GP
upload/download cycle would strip ~264 existing translations (44 keys x 6
langs) for two components that are still in the palette.

Restore the 25 spidertool + 19 policies keys to re-align en.json with the
translated locales. (Durable fix — making the extractor walk src/bundles and/or
adding lfx.components shims for these two providers — is a follow-up, since
spider.py eager-imports its SDK and a shim alone won't make it walkable.)

* chore(deps): drop dead astradb/graph-retriever extras and langchain-aws from langflow-base

After datastax/amazon graduated to lfx-datastax/lfx-amazon, langflow-base still
declared astradb (langchain-astradb), graph-retriever (langchain-graph-retriever,
graph-retriever) and langchain-aws in its aws extra -- all pulled into the
complete aggregate -- though zero core .py files import them (the consuming
components moved to the bundles, which re-declare the same deps). This duplicated
the SDKs in every pip install langflow and was inconsistent with the
openai/anthropic/cohere extras already removed in this PR.

Remove the astradb and graph-retriever extras and their complete refs, and drop
langchain-aws from the aws extra (boto3 stays -- it backs langflow-base's own S3
storage, not just the amazon components). Regenerate uv.lock; closure unchanged,
the SDKs still resolve via lfx-datastax/lfx-amazon.

* ci(extension-migration): enforce component os.environ-write lint in CI

check_component_env_writes.py guards against components writing to process-global
os.environ (which bleeds one caller's credential into other requests on a warm
worker). It was wired only as a pre-commit hook, bypassable via git commit -n,
web-UI commits, or uninstalled hooks. Add it as a stdlib-only job in
extension-migration-checks.yml next to freeze-components, so the credential-bleed
guard is enforced in CI regardless of local pre-commit.

* test(bundles): cover migration-table completeness, preflight provider path, frozen gate

Three test-coverage gaps from the metapackage split, all green on the current tree:
- migration-table completeness: parametrized sweep asserting every lfx_bundles
  provider component class (196 across 72 providers) is reachable via
  migration_table.json as an ext:<provider>:<Class>@ target, so a provider moved
  into a bundle without a table entry fails CI instead of silently breaking saved
  flows. Includes a negative control.
- dependency preflight provider path: drives find_missing_dependencies and
  _preflight_dependencies through a node with a model/agent_llm provider field so
  the _PROVIDER_PACKAGE_FALLBACKS path (the only working provider-dep source in
  engine-only installs) is exercised.
- frozen-components gate: unit tests for check_components_frozen.py covering
  additions (fail), removals (allowed), and the init/pycache/dotdir and
  comment-parsing rules.

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* fix(locales): make spider/toolguard i18n keys survive regen via shims

The earlier restore of the 44 spider/policies en.json keys was reverted by the
gp-backend-check bot: it auto-regenerates en.json on every push to this PR (the
PR-level path filter matches the bundle moves under lfx/components) and the
bundle-blind extractor could not reproduce the keys.

Make them durable the way every other moved provider does:
- Add lfx.components.spider and lfx.components.toolguard '# lfx-bundles-shim'
  packages re-pointing to lfx_bundles.{spider,toolguard}, so the extractor (which
  walks lfx.components) emits SpiderTool/PoliciesComponent keys again.
- Lazy-import spider's SDK (from spider.spider import Spider moved into crawl())
  so SpiderTool is importable for extraction/registration without spider-client,
  matching the lazy pattern weaviate/zep already use (toolguard was already lazy).
- Add spider/toolguard to scripts/ci/frozen_component_dirs.txt so the
  additions-only freeze gate accepts the new shim dirs.
- Restore the 44 keys to en.json.

Verified the extractor now reproduces exactly the 44 keys (25 spidertool + 19
policies), so the bot keeps them instead of dropping them. Bundle components are
dynamic-only (not in component_index.json), so no index regen is needed.

* chore(bundles): declare astrapy in lfx-datastax; align lfx-ibm manifest version

- lfx-datastax imports astrapy directly (astrapy.DataAPIClient/Database in the
  astra components) but only pulled it transitively via langchain-astradb.
  Declare it explicitly, pinned to match langflow-base's astrapy extra
  (>=2.1.0,<3.0.0).
- lfx-ibm's extension.json reported version 0.1.0 while pyproject.toml is 0.1.1;
  the loader uses the manifest version, so bump it to 0.1.1 to match.
Regenerate uv.lock (closure unchanged; astrapy was already resolved
transitively).

* refactor(migrate): guard consolidate against subdir data loss; fix alias docstring

- consolidate_bundles.move_provider copies only top-level *.py but rmtree's the
  whole source tree, so a provider with a subpackage would have its subdir
  silently destroyed and never migration-mapped. Refuse with a clear error
  instead of half-moving (port_bundle.py handles nested layouts). No current
  PROVIDER_DEPS entry has a subdir, so this is a defensive guard.
- component_aliases: the order-independence docstrings overclaimed. Scope the
  guarantee to identity-beats-display (and real-key-beats-alias) and note that
  two components contributing the same identity alias remain first-wins by
  iteration order.

* ci(gp-backend): regen en.json on src/bundles/** edits

The gp-backend-check bot regenerates locales/en.json on PR changes, but its
path filter only watched src/lfx/src/lfx/components/**. Bundle component
strings reach core en.json through the lfx/components/<provider> import shims
(e.g. spider, toolguard), so editing a bundle's display_name/description/info
under src/bundles/** would not re-trigger the extractor and en.json could
silently drift from the translated locales. Add src/bundles/** to the trigger
so bundle-side string edits keep en.json aligned; regen is diff-gated, so
non-string bundle edits remain a no-op.

* fix(bundles): Policies in Models & Agents core; Spider in bundles sidebar

Policies (toolguard) component:
- Move PoliciesComponent + policies/ helpers out of the lfx-bundles
  consolidated metapackage back into lfx/components/models_and_agents core
  (restores the main/release-1.11.0 layout); register it in the category
  __init__ and restore the core-path tests.
- toolguard stays an optional langflow-base extra (langflow-base[toolguard],
  pulled in via [complete]); the component imports it lazily, so a bare lfx
  install still loads/inspects it.
- Remove the lfx-bundles toolguard provider (dir + extra + [all] entry), the
  lfx core import shim, and the frozen_component_dirs.txt entry; drop the 4
  ext:toolguard migration entries; regenerate component_index (128 -> 129)
  and uv.lock.

Spider:
- Add spider to SIDEBAR_BUNDLES so it renders in the Bundles section like
  arXiv instead of falling into the default categories (it was missing from
  the list while peer lfx-bundles providers were present).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(bundles): fix error-tier mirror claim, list-time discovery note, metapackage README

Addresses three documentation nits from the PR #13563 re-review:

- errors.py: the lfx.bundles diagnostics comment claimed a full mirror of the
  inline tier's split, but only the duplicate case matches. name-invalid and
  root-unreadable are warning-only here where their inline counterparts are
  hard errors (ok=False). Tighten the comment to match actual behavior.
- BUNDLE_API.md: note that manifest-less lfx.bundles providers are invisible to
  `lfx extension list` (discover_all_extensions walks only installed + seed;
  load_lfx_bundles_extensions runs in the startup component-loading path), so
  the precedence story isn't read as a list-time property.
- lfx-bundles/README.md: the bulk move has landed on this branch (71 providers,
  populated per-provider extras, generated `all`), so drop the "empty skeleton /
  available later" framing and document the final state.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(bundles): surface pip-install guidance for missing SDKs on the server run path

A flow loaded through the UI/API on an engine-only `lfx` (no bundles)
failed deep in the vertex build with a bare
`Error creating class. ModuleNotFoundError(No module named 'langchain_chroma')`.
The actionable install guidance only existed on the `lfx run` preflight
and the legacy import shim; the server run path
(simple_run_flow -> run_graph_internal -> graph.arun()) had neither.

Both vertex-build error formatters (lfx's `format_exception_message`, used
by graph.arun(), and langflow's, used by the build/chat endpoints) now
detect a plain ModuleNotFoundError in the cause chain and attach the same
`pip install <package>` guidance the CLI gives, mapping the missing import
to its PyPI distribution via the existing requirements resolver. Bundle-shim
"components moved to ..." messages are left untouched (their message does
not start with "No module named"), so graduated providers stay actionable.

* fix(bundles): surface curated shim message over its raw cause in exception formatter

get_causing_exception walked to the deepest __cause__, so a graduated
provider's curated "components moved to ..." ModuleNotFoundError (raised
`from` the raw "No module named 'lfx_<x>'") was unwrapped past it. The
formatter then regenerated bare "pip install lfx-<x>" guidance, dropping
the curated text — including the "or pip install langflow, which bundles
it" clause.

Both get_causing_exception copies (lfx + langflow) now stop at the first
ModuleNotFoundError whose message is not a raw "No module named ..." (the
curated shim error), and module_not_found_hint surfaces that curated
message verbatim. Plain SDK import errors are unchanged.

Adds regression tests reproducing the real 3-level cause chain on both
formatters; the old test used a truncated chain that hid this.

* fix(bundles): cover OpenRouter/IBM WatsonX in Orchestrate requirements detection (#13732)

fix(bundles): cover OpenRouter/IBM WatsonX in wxO requirements detection

The bundle split removed the langchain provider SDKs from lfx's own
dependency tree, which makes flow→requirements detection load-bearing for
Watsonx Orchestrate deployments: the runner installs only `lfx` + the
packages `generate_requirements_from_flow` emits, so a provider missing
from the resolution tables yields an empty requirements set and the
deployed flow fails to import its model class.

Two unified-selectable providers were unresolved:
- OpenRouter (runs on ChatOpenAI) was in no resolution table -> [].
- "IBM WatsonX" (catalog casing) only matched the legacy "IBM watsonx.ai"
  key; langchain-ibm landed only by an embedding-path coincidence.

Add both to _PROVIDER_PACKAGE_FALLBACKS (keyed by the exact catalog
strings) so every provider in MODEL_PROVIDER_METADATA resolves.

Tests:
- TestBundleSeparationOrchestrate in test_flow_requirements.py: catalog
  coverage guard, OpenRouter/IBM regressions, and split-contract guards
  (bundle dists and langchain imports must stay out of lfx's tree).
- End-to-end artifact tests in test_watsonx_orchestrate.py asserting a
  bundle-provider flow emits the langchain SDK (not lfx-openai/lfx-bundles)
  and that requirements.txt pins lfx as its first line.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Backend format

* Format again

* [autofix.ci] apply automated fixes

* fix(bundles): pin datastax ruff line-length to 120 to stop format tug-of-war

The datastax bundle's pyproject.toml declares a [tool.ruff.lint.per-file-ignores]
table, which makes ruff treat the directory as a standalone config root. It never
restated line-length, so ruff fell back to its default of 88 there. As a result
`make format_backend` (per-directory config discovery) wrapped lines to 88 while
the pre-commit ruff-format hook (run with --config pyproject.toml, forcing the
repo-wide 120) unwrapped them back, leaving the two formatters fighting forever.

Restate line-length = 120 so both config-resolution paths agree. Use the explicit
line-length rather than extend=../../../pyproject.toml to avoid pulling in the
root lint ruleset, which the bundle intentionally does not enforce.

* fix(bundles): relocate notdiamond + vlmrun into lfx-bundles

Both providers were dropped during the bulk long-tail move with no bundle
and no compatibility shim, removing them from the registry,
component_index, migration table and locales. That breaks saved flows
using the NotDiamond Router or VLM Run Transcription nodes (vlmrun ships
in langflow[complete]), contradicting the zero-user-facing-change goal.

Relocate both the same way as the other long-tail providers:
- move implementations into src/bundles/lfx-bundles/src/lfx_bundles/
- restore lfx.components.{notdiamond,vlmrun} marker shims
- re-register both in lfx/components/__init__.py
- add per-provider extras (vlmrun -> vlmrun[all]) and the 'all' aggregate
- add migration_table targets ext:{notdiamond,vlmrun}:<Class>@official
- restore the 32 en.json strings; refresh uv.lock

* docs(lfx): correct dependency-preflight scope to lfx run only

The preflight is wired into 'lfx run'; 'lfx serve' has its own
module-not-found hint at request time and does not call it.

---------

Co-authored-by: Elif Sema Balcioglu <elifsemabalcioglu@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-06-23 19:37:10 +00:00
f58914bc84 fix: release candidate version selection (#13776)
* fix: share release candidate version selection

* fix: review comments addressed
2026-06-22 15:35:10 -07:00
47760d22fb fix(ci): drop macOS Intel from py3.14 cross-platform set (onnxruntime x86_64 wheel gap) (#13773)
fix(ci): drop macOS Intel from py3.14 cross-platform set (onnxruntime x86_64 gap)

Follow-up to #13772. The new Python 3.14 experimental job on macOS Intel
(macos-latest-large / x86_64) fails at dependency resolution:

  No solution found ... onnxruntime>=1.24.1 has no wheels with a matching
  platform tag (macosx_*_x86_64) ... onnxruntime>=1.17.0,<=1.23.2 has no
  cp314 ABI ... lfx==...rc0 depends on markitdown -> magika -> onnxruntime ...
  unsatisfiable.

This is a permanent macOS x86_64 ecosystem gap, not the find-links bug:
- langflow pins `onnxruntime>=1.26; python_version>='3.14'` (pyproject.toml), and
- onnxruntime dropped macOS x86_64 wheels at 1.24, while the older onnxruntime
  that still ships macOS x86_64 wheels (<=1.23.2) has no cp314 wheels.

So macOS Intel + py3.14 can never resolve. macOS Intel resolves fine through
3.13 (the py<3.14 branch pins onnxruntime<1.24, which still has x86_64 wheels);
macOS arm64 has cp314 wheels and is unaffected. The job is non-blocking
(continue-on-error) so it never blocked releases, but it's a guaranteed,
no-signal failure burning the costly macos-latest-large runner every release.

Drop macOS Intel from the 3.14 experimental matrix (keep linux/windows/macOS-arm64)
and update the docs/comments to explain the x86_64 wheel gap.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 11:44:51 -07:00
b0d4fe0645 fix(ci): resolve pre-release cross-platform install failure; graduate py3.13, add py3.14 experimental (#13772)
fix(ci): resolve pre-release cross-platform install + graduate py3.13, add py3.14

The cross-platform install test failed *only* on "Pre-release" release runs,
in the Python 3.13 (Experimental) jobs, while normal releases passed.

Root cause: the experimental job's "Force reinstall local wheels to prevent
downgrades" step drops `--no-deps` when `pre_release=true` (to allow pre-release
dependency resolution) but never passed `--find-links` to the local wheel dirs.
uv then re-resolved langflow-base's `lfx>=X.Y.Zrc0,<X.(Y+1).dev0` constraint
against PyPI only, where the matching rc/dev wheel is not yet published, and
failed with "No solution found when resolving dependencies ... unsatisfiable".
Stable releases keep `--no-deps`, never re-resolve, and so never hit it.

Fix: build a `FIND_LINKS` array over the local wheel dirs (sdk/lfx/base/bundles)
and thread it into both force-reinstall `uv pip install` commands (Windows and
Unix). Harmless on the stable `--no-deps` path; required on the pre-release path.

Also:
- Graduate Python 3.13 from the experimental matrix to the stable (blocking)
  matrix on linux amd64, macOS arm64, and windows amd64. macOS Intel
  (macos-latest-large) stays at 3.12 only on the blocking matrix to limit use of
  the costly runner, matching the existing 3.10/3.12 design.
- Add Python 3.14 as the new experimental (non-blocking, continue-on-error) set,
  mirroring the platform coverage 3.13 previously had (incl. macOS Intel).
- Update the test-summary messages and cross-platform-test.md accordingly.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 11:06:40 -07:00
b033bfa22b feat(auth): external trusted JWT auth + JIT user mapping + access ceiling (#13293)
* feat(auth): external trusted JWT auth + JIT user mapping

Adds the OSS half of trusted external identity support:

- New EXTERNAL_AUTH_* AuthSettings (off by default) covering provider
  key, token transport (header/cookie), JWKS or trusted-decode, claim
  mapping, and a pluggable EXTERNAL_AUTH_IDENTITY_RESOLVER import path.
- New services/auth/external.py with JWT/JWKS validation, identity
  resolver protocol, and token extraction helpers.
- AuthService.get_or_create_user_from_claims +
  extract_user_info_from_claims implement the existing BaseAuthService
  JIT hook through SSOUserProfile - no new tables.
- _authenticate_with_token falls back to external resolution when the
  native JWT path fails, so Authorization-header callers transparently
  upgrade to external auth.
- Token extractors in services/auth/utils.py consult the configured
  external header/cookie after the native JWT path on session,
  WebSocket, SSE, and optional-user dependencies.
- /api/v1/session catches AuthenticationError so external-credential
  failures resolve to authenticated=False rather than 500.

Tests: 14 unit tests for external.py (JWT decode, claim mapping,
custom resolver) plus 3 integration tests in test_login.py exercising
the session endpoint JIT path (header + cookie + expired-token).

Co-Authored-By: phact <estevezsebastian@gmail.com>
Co-Authored-By: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Based-On: langflow-ai/langflow#13280

* Update src/backend/base/langflow/services/auth/external.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(auth): refetch JWKS once on unknown kid to survive IdP key rotation

Without this, a token signed by a newly rotated IdP key is rejected for
up to JWKS_CACHE_TTL_SECONDS (5 min) because the cached JWKS predates
the key. On a kid miss we now refetch the JWKS once, rate-limited to
one forced refresh per 30s per URL so attacker-supplied kids cannot
hammer the IdP's JWKS endpoint.

Adds JWKS-path tests (signature verification, rotation refetch,
rate-limited refresh) that were previously uncovered.

* feat(auth): add external access ceiling for trusted auth (#13637)

* feat(auth): add external access ceiling

* fix(auth): tolerate minimal external access settings

* fix(auth): address external-auth review (require JWKS audience, untangle authz/auth)

Resolves Gabriel's PR review feedback on #13293:

- Require EXTERNAL_AUTH_AUDIENCE on the JWKS verification path. Previously a
  JWKS-only config verified signature + exp but left aud/iss unbound, so a token
  the same IdP minted for a *different* relying party was accepted. decode now
  fails closed (before any network fetch) with an actionable message and binds
  aud; iss stays verified-when-set. Adds wrong-aud / missing-aud tests.

- Move the request-scoped action ceiling out of auth/external.py into a new
  authorization/access_ceiling.py so the authorization package no longer imports
  the auth layer (the only such import). Guards consult the authz-owned
  primitive; the auth layer only derives the ceiling and installs it. external.py
  re-exports the names for callers that derive/inspect the ceiling.

- _unique_external_username reuses _external_username_fallback instead of
  re-implementing the provider-digest formula.

- Read the non-optional EXTERNAL_AUTH_* settings as plain attributes (drop
  getattr(..., default), which re-hardcoded Field defaults) in service.py and
  api_key/crud.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(authz): pass API key context to authorization (#13639)

* chore: bump version to 1.10.1

* fix(lfx): restamp component index version after 1.10.1 fork bump (#13575)

Companion to the release-1.11.0 fix (#13574). The 1.10.1 fork bump left component_index.json's version at 1.10.0; _read_component_index fails closed on the exact-version mismatch, so the bundled registry loads as None and the upgrade-gate LFX tests fail on this branch's CI.

Surgical restamp: version -> 1.10.1, sha256 recomputed with the build script's exact hashing/serialization. Entries unchanged; 2-line diff. The three affected tests pass locally.

* fix(ci): rebalance backend test splits with measured durations + raise unit-test step timeout (1.10.1) (#13585)

fix(ci): rebalance backend test splits with measured durations + raise unit-test step timeout

Unit Tests - Python 3.12 - Group 3 has been failing at 98-99% on the
nightly: the job hits the nick-fields/retry 40-minute per-attempt timeout
and pytest is SIGTERM'd mid-test, so it looks like one flaky last test
when it is actually a deterministic timeout (and the internal retry plus
run-level retries can never succeed).

Root cause is twofold:

1. .test_durations was last regenerated ~May 2025 and covered only 2,219
   of ~9,500 current unit tests, so pytest-split weighted 77% of the
   suite at the 0.73s average. The expensive client-fixture tests
   (test_webhook.py, test_login.py - each pays a full create_app +
   lifespan boot per test, 60-120s late in a CI run) clustered into
   group 3's tail, making it ~8-10 minutes slower than its siblings
   (33:15 on 3.13, >40min on 3.12 which is additionally slowed by
   astrapy disabling SSL connection reuse on Python 3.12.0-11).
   The weekly store_pytest_durations workflow that should refresh the
   file has been dying at the 6-hour GitHub job limit every week (serial
   full-suite run no longer fits), so the file silently froze.

   This regenerates the file from real per-test wall-clock measured in
   the passing Python 3.13 nightly jobs (run 27246228762 attempt 1, all
   5 groups, parsed from the -vv xdist logs: per-worker start-to-start
   deltas). 9,508 tests now have measured durations; old entries are
   kept where no new measurement exists. Simulated least_duration
   split goes from one outlier group to 5 even groups (~24.6 min each)
   with the >50s tests spread 1-2 per group instead of 7 in one.

2. timeout_minutes 40 -> 50 gives headroom for matrix-cell variance
   (3.12 runs ~20% slower than 3.13) so a slightly slow cell degrades
   gracefully instead of burning 2x40min and failing the whole run.

Follow-ups (not in this PR): fix store_pytest_durations to run with
xdist or split groups so it fits the 6h limit; investigate the in-worker
degradation that makes client-fixture boots cost 8-12s early in a run
but 60-120s after ~30 minutes.

* fix(test): raise spawn-child join timeout in test_multi_process_visibility (1.10.1) (#13590)

fix(test): raise spawn-child join timeout in test_multi_process_visibility

The test spawns a child via multiprocessing spawn context, which
cold-imports the full langflow package (plus coverage's multiprocessing
hooks in CI) before appending a single event. The 10s join timeout is
routinely exceeded on a loaded CI runner sharing 4 vCPUs with a second
xdist worker: in nightly run 27253229568 (Unit Tests - Python 3.12 -
Group 5) the test failed all 12 executions (5 reruns x 2 step attempts),
each rerun exactly 10s apart - the join deadline, not a product bug.

Raise the liveness bound to 60s (join returns immediately when the
child exits, so the passing case is unaffected) and kill the child on
timeout so a hung spawn can't leak into later tests.

* fix(ci): anchor langflow-base version extraction in nightly docker build (1.10.1) (#13593)

fix(ci): anchor langflow-base version extraction in nightly docker build

The first nightly on the 1.11 workspace (tag v1.11.0.dev0, run
27253229568) failed in Build Nightly Base Package within seconds:
'Base version format is incorrect'. The extraction (uv tree, grep
langflow-base unanchored, awk field 3, first line) broke because uv
tree now prints the langflow-base workspace-root line
('langflow-base v1.11.0.dev0', two fields) before any dependency line
('langflow-base[complete] v1.11.0.dev0' under the langflow root, three
fields), so the first match yields an empty field 3 and the format
check exits 1. uv tree's stderr is discarded, which made the real
cause invisible in CI.

Anchor to the root line and take field 2 instead - the exact pattern
the langflow (main package) extraction in this same file already uses,
which is why the main-package builds passed on the same tag. Verified
both extractions print 1.11.0.dev0 against a local checkout of
v1.11.0.dev0.

* fix(test): gate models.dev background refresh out of tests (1.10.1) (#13598)

fix(test): gate models.dev background refresh out of tests

Integration tests failed twice in nightly run 27260425158 with pyleak
EventLoopBlockError - first Integration Tests 3.14, then 3.12 on the
rerun, each time in a different test. The blocking stack points at
refresh_models_dev_periodically: every app boot unconditionally starts
a lifespan task that immediately fetches https://models.dev/api.json,
so the request lands mid-test in whatever test happens to be running.
Under pyleak's asyncio debug instrumentation the fetch blocked the loop
0.797s against a 0.2s threshold. Whichever test draws the short straw
flakes - which is why it looked transient and moved between versions.

Add a LANGFLOW_MODELS_DEV_REFRESH env gate (default unchanged: enabled)
and disable it session-wide in the backend test conftest. Tests fall
back to the bundled static model lists, which is also deterministic.

Verified: with the gate set, app boot makes zero models.dev requests;
the previously failing integration test passes.

* fix(ci): allow pre-releases when pinning the nightly in migration validation (1.10.1) (#13601)

fix(ci): allow pre-releases when pinning the nightly in migration validation

Migration Test: pip/venv (stable -> nightly) failed deterministically on
nightly run 27260425158 (twice, including a rerun):

  hint: langflow-base was requested with a pre-release marker (e.g.,
  langflow-base==1.11.0.dev1), but pre-releases weren't enabled
  (try: --prerelease=allow)

This is the first nightly publishing as a canonical .devN pre-release
of the langflow distribution (nightly -> stable bundle cutover). The
'latest' branch of the upgrade step already passes --prerelease=allow,
but the pinned-version branches do not. uv implicitly allows the
pre-release for the directly requested ==X.Y.Z.devN pin, yet langflow's
metadata pins langflow-base==X.Y.Z.devN transitively, and uv rejects
transitive pre-releases unless they are enabled - so the install fails
after the stable uninstall, sinking the migration test.

Add --prerelease=allow to both pinned-version install lines.

* fix(ci): scope nightly migration-test pre-releases to the langflow stack (1.10.1) (#13605)

fix(ci): scope nightly migration-test pre-releases to the langflow stack

The previous fix (#13599) added a global --prerelease=allow, which let
UNRELATED dependencies resolve to alphas: on nightly run 27274206250
the stable->nightly upgrade pulled pydantic 2.14.0a1 + pydantic-yaml
1.6.1a1, and the pydantic alpha breaks langchain-core at import time
(RunnablePassthrough pydantic ValidationError), failing the nightly
boot right after a successful install. Clean installs were fine - only
this upgrade path resolved the alpha combo.

Scope pre-release eligibility to the langflow lockstep stack instead:
uv accepts a pre-release when the package's own requirement carries a
pre-release marker, but not via transitive pins, and the nightly chain
is langflow -> langflow-base -> lfx -> langflow-sdk with exact ==devN
pins. Request each directly; langflow-sdk versions independently
(0.2.0.devN), so an explicit .dev0 floor marks it eligible while lfx's
exact pin selects the version. The 'latest' branch gets the same
treatment via .dev0 floors on all four.

Verified by dry-run against PyPI: langflow/langflow-base/lfx at
1.11.0.dev2 + langflow-sdk 0.2.0.dev2 resolve with pydantic staying at
stable 2.13.4.

* fix(ci): create GitHub releases on the dispatched v-prefixed tag (1.10.1) (#13609)

fix(ci): create GitHub releases on the dispatched v-prefixed tag

The create_release job passed the bare version (v stripped) as the
release tag with no commit target, so when that tag did not exist
GitHub minted a new lightweight tag at the default-branch HEAD -- the
wrong commit, still carrying the previous version (main only adopts a
release's version via the post-release back-merge). Every release since
1.8.2 shipped a stray bare tag (1.8.2, 1.8.3, 1.9.0-1.9.6, 1.10.0)
pointing at a previous-version commit, and the GitHub release had to be
manually re-pointed to the real vX.Y.Z tag after each release.

- create_release now attaches the release to inputs.release_tag for
  stable releases; pre-releases keep their computed tag (e.g.
  1.10.0rc1) but it is minted at the release commit via 'commit:'.
- release-lfx.yml pins the minted lfx-v* tag to github.sha instead of
  the default branch.

The validate-tag-format guard (#12847) only blocks at dispatch time;
create_release was re-creating the very duplicates it guards against.

* fix(ci): resolve validate-version output in release-lfx changelog link (1.10.1) (#13611)

fix(ci): resolve validate-version output in release-lfx changelog link

The create-release job references
needs.validate-version.outputs.current_version in its generated release
notes (the Full Changelog compare link), but validate-version was not in
the job's needs array, so the expression evaluated empty and the link
rendered as compare/v...lfx-vX.Y.Z (broken base).

Add validate-version to the create-release needs. This adds no real
serialization: create-release already waits on release-lfx, which
transitively requires validate-version via run-tests, and the job's
always() if-condition is unaffected.

Flagged by actionlint: property "validate-version" is not defined in
object type {build-docker, release-lfx}.

* feat(auth): external trusted JWT auth + JIT user mapping

Adds the OSS half of trusted external identity support:

- New EXTERNAL_AUTH_* AuthSettings (off by default) covering provider
  key, token transport (header/cookie), JWKS or trusted-decode, claim
  mapping, and a pluggable EXTERNAL_AUTH_IDENTITY_RESOLVER import path.
- New services/auth/external.py with JWT/JWKS validation, identity
  resolver protocol, and token extraction helpers.
- AuthService.get_or_create_user_from_claims +
  extract_user_info_from_claims implement the existing BaseAuthService
  JIT hook through SSOUserProfile - no new tables.
- _authenticate_with_token falls back to external resolution when the
  native JWT path fails, so Authorization-header callers transparently
  upgrade to external auth.
- Token extractors in services/auth/utils.py consult the configured
  external header/cookie after the native JWT path on session,
  WebSocket, SSE, and optional-user dependencies.
- /api/v1/session catches AuthenticationError so external-credential
  failures resolve to authenticated=False rather than 500.

Tests: 14 unit tests for external.py (JWT decode, claim mapping,
custom resolver) plus 3 integration tests in test_login.py exercising
the session endpoint JIT path (header + cookie + expired-token).

Co-Authored-By: phact <estevezsebastian@gmail.com>
Co-Authored-By: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Based-On: langflow-ai/langflow#13280

* Update src/backend/base/langflow/services/auth/external.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(auth): refetch JWKS once on unknown kid to survive IdP key rotation

Without this, a token signed by a newly rotated IdP key is rejected for
up to JWKS_CACHE_TTL_SECONDS (5 min) because the cached JWKS predates
the key. On a kid miss we now refetch the JWKS once, rate-limited to
one forced refresh per 30s per URL so attacker-supplied kids cannot
hammer the IdP's JWKS endpoint.

Adds JWKS-path tests (signature verification, rotation refetch,
rate-limited refresh) that were previously uncovered.

* fix(components): follow HTTP redirects in URL component with per-hop SSRF revalidation (#13572)

* fix(components): follow HTTP redirects in URL component with per-hop SSRF revalidation

The URL component sent every request with follow_redirects=False, so any
site whose entered URL 301s to its canonical address (http->https or
www/non-www normalization) returned the redirect stub - e.g. a bare
"301 Moved Permanently / nginx" page - as the scraped content, or failed
outright when the redirect response had an empty body.

Redirects are now followed via a new advanced "Follow Redirects" input
(default on). When SSRF protection is enabled, hops are followed manually
and each Location target is re-validated with the same blocked-IP denylist
and DNS pinning as the initial request before any connection is made,
mirroring the API Request component; cross-host hops drop sensitive
headers and chains are capped at 20 redirects.

* fix(components): compare full origin when keeping credentials across redirects, crawl from post-redirect base

Addresses CodeRabbit review:
- _headers_for_redirect now keeps Authorization/Proxy-Authorization/Cookie
  only for same-origin hops (scheme, host, port) or a direct http->https
  upgrade on default ports, exactly the cases where httpx keeps the
  Authorization header. Applied to both the URL and API Request
  components (the helper was copied from the latter).
- _crawl_recursive resolves relative links and the prevent_outside check
  against the final post-redirect URL, and marks it visited, so depth>1
  crawls work on sites that 301 to their canonical address.

* chore: regenerate component index and starter projects for release-1.10.1 base

The autofix.ci jobs uploaded but never pushed the regeneration after the
rebase, leaving the 4 URL-component templates with a stale field_order
(missing follow_redirects) and the index without the new code hashes.
Generated with the same commands CI uses: LFX_DEV=1 make
build_component_index + scripts/ci/update_starter_projects.py.
Pokedex Agent / Structured Data Analysis Agent pick up the API Request
component's new code_hash from the origin-comparison fix.

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix(ci): make release Biome lint green — NUL-delimit file list + clear pre-existing lint errors (#13550)

The Lint Frontend job runs against `main` as the base. In a release
(workflow_dispatch) run it diffs the entire release branch (~915 files) and
re-lints nearly the whole frontend, exposing two issues that per-PR linting
never hits together:

1. xargs split starter-project spec paths containing spaces (e.g.
   "News Aggregator.spec.ts" -> "News" + "Aggregator.spec.ts"), producing
   `internalError/io: No such file or directory`. NUL-delimit the file list
   so spaces are preserved. (supersedes #13381)

2. The whole-branch diff surfaced 30 pre-existing Biome errors: 22
   noExplicitAny + 8 organizeImports. Resolved with real types where safe
   (freezeObject generic, ColDef defaults, messagesSorter field shape,
   VertexBuildTypeAPI, Record<string,string>, DragEvent<HTMLElement>,
   unknown for narrowed values) and justified biome-ignore for genuinely
   loose cases (polymorphic display values, test global stubs, captured
   unexported StreamCallbacks). Imports auto-sorted via biome.

Verified locally: biome check on the full release-vs-main file set is now 0
errors (was 30); tsc unchanged at its 303-error baseline (no new type errors).

* fix: limit public flow endpoint from displaying private flow streams (#13602)

* fix: limit public flow endpoint from displaying private flow streams

* fix: Address coderabbit reviews

---------

Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>

* fix: enforce the FileSystemTool credential deny-list (#13625)

* enforce the FileSystemTool credential deny-list

* security checks gh review

* [autofix.ci] apply automated fixes

* feat(authz): pass API key context to authorization

* refactor(authz): address review feedback on API-key auth context

- Add AuthCredentialContext.from_api_key_result() and use it at all six
  API-key projection sites (service.py x5, mcp_projects.py) so the caveat
  fields stay in sync and no site can silently drop one.
- authz_me builds the enforce context from the public
  current_auth_context_for_authz() helper instead of reaching into the
  private guards._auth_context.
- Clear request-local credential context at the top of verify_project_auth
  to match the service.py entrypoints, so the composer-token fast path can
  never inherit stale context.

---------

Co-authored-by: phact <estevezsebastian@gmail.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>

* [autofix.ci] apply automated fixes

* fix(auth): close access-ceiling bypasses and harden external JWT auth

Address review findings on the external trusted-auth + access-ceiling work:

- Enforce the access ceiling on execution/mutation paths that bypassed the
  ensure_*_permission chokepoint: MCP project tool-call (flow execute),
  flow version snapshot/activate/delete, v1 file upload/delete, and memory
  base CRUD/ingest.
- Move the external-user API-key block into the shared authenticate_api_key
  chokepoint so /run, v2 workflow, OpenAI-compat, WebSocket, webhook and MCP
  key auth all enforce it (was only on the JWT-fallback path).
- Require exp on both the JWKS and trusted-decode paths; reject non-https
  JWKS URLs (loopback http allowed for dev).
- Normalize EXTERNAL_AUTH_PROVIDER at the config boundary so the API-key floor
  cannot be silently disabled by an empty/whitespace value.
- Make a configured EXTERNAL_AUTH_ACCESS_CLAIM_MAPPING authoritative: an
  unmapped claim value falls to the default level instead of self-elevating
  via the built-in alias table.
- Stop nulling a stored SSOUserProfile.email when a later token omits email.
- Keep the external credential usable as a fallback when a stale/invalid
  native token is present (WebSocket, SSE, optional-user paths).
- Add delete to the editor access level (deploy stays admin-only).

Adds regression tests across auth, authz guards, api-key crud, and the newly
guarded routes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(mcp): give handle_call_tool flow stub the attrs ensure_flow_permission reads

The access-ceiling hardening added an ensure_flow_permission(EXECUTE) guard to
handle_call_tool that reads flow.user_id and flow.workspace_id, but the
_invoke_handle_call_tool flow stub only carried id/name/folder_id, raising
AttributeError. Set user_id to match the current user so the owner-override
path is exercised, and add workspace_id.

* fix(auth): close remaining ceiling gaps + external-cookie/filesystem/job-queue hardening

Synthesizes the external review (P1-P3) with the multi-agent re-review findings.

External auth:
- P1: regular HTTP (get_current_user) and /api/v1/session now extract the
  external credential separately and pass it as a fallback, so a stale/invalid
  native cookie can no longer shadow a valid external credential (previously only
  WS/SSE/optional paths did this).
- P2: clear the request-local external access ceiling at every auth entrypoint
  (api-key, raw access-token, webhook, ws api-key, MCP), not just
  authenticate_with_credentials, so a stale ceiling can never carry into a
  non-external path.
- Make a configured access-claim mapping authoritative even when it parses empty
  (all-invalid entries no longer re-enable alias self-elevation).
- A blocked external user's API key no longer increments usage counters.

Access-ceiling coverage (routes that previously escaped the deny-only cap):
- custom_component / custom_component_update (code instantiation) now enforce the
  ceiling directly (viewer denied; editor/admin/native users unchanged).
- Deprecated /upload/{flow_id}, update_project_mcp_settings, and the models.py
  default/enabled-model variable routes now call the appropriate guard.
- Memory-base guards resolve the base first and pass kb_id + real owner so plugin
  enforce runs for non-owners and audit rows carry the kb id.

Bundled hardening:
- P2: filesystem deny-list now denies a protected credential directory requested
  by basename (.ssh/.aws/.git) and as a glob entry, not just as a parent dir.
- P3: public-job marker write failures on the Redis backend now fail the build
  (503) instead of returning an un-shareable job id that 404s on other workers.

Adds regression tests across auth, authz route guards, api-key crud, external
auth, filesystem deny-list, memory bases, and the redis job queue.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* [autofix.ci] apply automated fixes

---------

Co-authored-by: phact <estevezsebastian@gmail.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
2026-06-17 22:58:43 +00:00
72f86c96eb ci: gate backend tests by Python changes (#13614)
* ci: gate backend tests by Python changes

* fix(ci): update test conditions to include docs-only path filter

* Update .github/workflows/ci.yml

Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>

---------

Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
2026-06-15 18:35:30 +00:00
415323ec6d docs: migrate code blocks to native Prism and restyle API reference pages (#13299)
* docs: migrate code blocks from CodeSnippet to native Prism

Replace the custom CodeSnippet component (@code-hike/lighter) with
Docusaurus's native @theme/CodeBlock across all MDX files (current and
versioned docs). Add bash to additionalLanguages and swizzle
prism-include-languages.js to add custom token highlighting for shell
commands and flags. Remove @code-hike/mdx dependency.

* docs: improve inline code styling

Darken inline code background, increase horizontal padding to 0.4em,
fix vertical alignment, and remove border in light mode.

* docs: address PR review — fix code slice regression and CSS/regex polish

- Inline RecursiveCharacterTextSplitter inputs and methods as literal
  code blocks in concepts-components.mdx (current + 1.8.0 + 1.9.0),
  restoring the focused slices lost when migrating from CodeSnippet
- Scope bash-plain Prism regex to unambiguous CLI subcommands only,
  removing generic bash builtins (run, add, get, set, start, stop, etc.)
- Merge duplicate .theme-code-block CSS rules into a single declaration

* fix(docs): prevent horizontal scroll on API docs pages

The Redoc two-column layout (sidebar 300px + api-content 1300px)
totals 1600px, expanding .main-wrapper beyond narrower viewports
because it has overflow:visible. Clips at .main-wrapper using the
html.plugin-redoc class that Docusaurus adds on API pages only.

* fix(docs): API docs sidebar and layout fixes

- Disable Redoc built-in search (disableSearch: true)
- Pin sidebar top to navbar height (top: 60px) so it never hides behind navbar
- Remove hardcoded #111 background from dark mode sidebar

* fix(docs): render markdown correctly in API docs descriptions

The _clean_descriptions function was converting newlines to <br> tags,
mixing HTML with Markdown. CommonMark stops parsing Markdown headings
(###) inside HTML blocks, causing them to appear as literal text in Redoc.

Replace the <br> conversion with a simple strip() so descriptions remain
pure Markdown and Redoc renders headings, lists, and code blocks correctly.

* feat(docs): align API docs colors with Langflow brand

- Set primaryColor to #F471B5 (Langflow pink)
- Add HTTP method badge colors matching Langflow palette
- Set schema.linesColor and requireLabelColor to brand pink
- Set inline code color to pink, headers to #e3e3e3
- Set sidebar background to #18181b (matches frontend dark bg)
- Set rightPanel background to #0d0d0f, codeBlock to #161618
- Refactor: move color config from CSS to theme.theme where safe
- Remove dead search input CSS (search disabled via disableSearch:true)
- Consolidate duplicate .menu-content rules

* wip(docs): API docs styling — colors, components, light/dark themes

* fix(docs): fix Response samples h3 title padding and refactor HTTP method button CSS

* feat(docs): add sidebar dark background, active item styles, and right panel color adjustments

* fix(docs): add sidebar borders and remove operation divider border

* fix(docs): fix expanded response background and align dark/light theme colors

* refactor(docs): standardize selectors, comments and remove duplicate rules in redocusaurus.css

* refactor(docs): apply PR review — CSS custom properties, version pin, dom version comments, fix overflow

* fix(docs): fix Redocly badge visibility covered by sidebar background

* fix(docs): extend sidebar border-right to Redocly badge area

* refactor(docs): replace hardcoded #ffffff label color with --redoc-text-label variable

* fix(docs): lighten inline code background in API docs dark theme

Redoc's default typography.code.backgroundColor (rgba(38, 50, 56, 0.05))
is nearly invisible over the dark background. Override it with
rgba(255, 255, 255, 0.05) in dark theme only, excluding pre > code so
code sample blocks stay unaffected.

* feat(docs): align docs primary pink with API spec brand color

Use #f471b5 (API spec primaryColor) as --ifm-color-primary in dark theme
and #e44fa0 (slightly darkened for contrast on white) in light theme.
Remove dark-theme pink overrides in sidebar.css (#ff6ad0 CTA and
hsla(329, 55%, 68%) active TOC link) — they compensated for the old
muted pink and are redundant now that the primary itself is bright.

* feat(docs): lighten dark theme text colors for better readability

Bump body text (#a8a8b0 -> #bcbcc4), headings (#cdcdd4 -> #dcdce2)
and sidebar menu (#8a8a92 -> #9c9ca4) one step brighter.

* chore(docs): add IBM Equal Access accessibility-checker setup

Add accessibility-checker as devDependency with aceconfig.js (policy
IBM_Accessibility, JSON reports in docs/a11y-results/, gitignored).
Scan with: npx achecker <url> against a built docs site.

* fix(docs): WCAG AA contrast and ARIA fixes across docs and API reference

Validated with IBM Equal Access scans (light theme): home, quickstart and
component pages at 0 violations; /api from 3382 down to 167 (all remaining
are Redoc-internal DOM: schema table headers, svg/select labels).

Docs site:
- Light primary pink #d11074 — passes 4.5:1 on white and inline-code bg
- Light Prism palette darkened per-token to pass 4.5:1 on #f9f9fd
- TabItem swizzled to give tabpanels an accessible name (aria-label)
- codeBlockA11y client module: scrollable code blocks get role=region +
  unique label; non-scrollable ones lose the needless tabindex

API reference (Redocusaurus):
- redocA11y client module: role=main on api-content, role=navigation on
  sidebar — fixes 1924 aria_content_in_landmark violations
- HTTP method badges and response chips darkened to pass with white text
- Light-theme overrides: accessible pink #cd1072 for links, required
  markers, constraint chips, schema tree lines; darker grays for utility
  buttons and type labels (incl. 0.7-opacity wrapper fix)
- Sidebar active/hover items use regular text color, method badges keep
  their own colors; expandable property names match non-expandable ones

* fix(docs): WCAG AA contrast fixes for dark theme

Validated with IBM Equal Access scans in dark mode (temporary
colorMode.defaultMode flip during scanning): home, quickstart and
component pages at 0 violations; /api matches light at 167 remaining
(all Redoc-internal DOM: table headers, svg/select labels).

- Code block comments/line numbers #4a5060 -> #798197 (4.56:1 on #18181a)
- Redoc dark sample tokens: boolean/null #e95c59, number #5392b8
- Status-code tabs: lift docusaurus-theme-redoc's #303846 !important
  selected-tab rule with a higher-specificity override
- oneOf variant buttons: dark text in both themes (their white/pink
  backgrounds are theme-independent)
- redocA11y client module: patch response chip colors (success green /
  error red) to dark-accessible variants when data-theme=dark — a single
  Redoc theme color cannot pass on both light and dark derived
  backgrounds, and status is only distinguishable by computed color

* fix(docs): resolve remaining Redoc-internal accessibility violations

Extend the redocA11y client module with semantic patches for Redoc DOM
the theme cannot reach (validated: 0 IBM Equal Access violations on all
scanned pages in both light and dark themes):

- Decorative svg chevrons/arrows: aria-hidden=true (svg_graphics_labelled)
- Content-type dropdowns: aria-label (input_label_exists)
- Schema field tables (2-col name|description layout, no <th> anywhere):
  role=presentation — content reads in DOM order; role=rowheader on <td>
  is invalid ARIA inside a native table (table_headers_exists/related)
- Semantic patches run on the next animation frame after DOM changes so
  Redoc's lazy-rendered operations are covered immediately; the heavier
  color patch stays debounced

* ci(docs): gate docs accessibility with IBM Equal Access scans

Add test-docs-accessibility job to docs_test.yml (rides the existing
docs/** path filter from ci.yml): build + scan 4 representative pages
(home, quickstart, component page, /api) in light theme, then flip
colorMode.defaultMode to dark, rebuild and scan again.

- scripts/a11y-ci.sh: serves the build and runs npx achecker per page
  with one retry to absorb Redoc lazy-render timing flakes; any real
  violation fails the job (failLevels: violation)
- aceconfig.js: pin ruleArchive to 19May2026 so IBM rule updates don't
  break CI without a deliberate bump

* ci(docs): fix Chrome sandbox launch on Ubuntu 24.04 runners

Ubuntu 24.04 restricts unprivileged user namespaces via AppArmor, which
prevents puppeteer's Chrome (used by the IBM checker) from starting its
sandbox. Re-enable them with the documented sysctl workaround instead of
weakening the browser with --no-sandbox.

* fix(docs): align response status code with description text

Redoc sets vertical-align: top and a smaller line-height on the status
code <strong> inside response buttons, leaving "200" visually higher
than "Successful Response". Align both to the shared text baseline.

* refactor(docs): apply PR review feedback

- redocA11y: match only real Redoc routes (/api, /api/workflow) — a bare
  startsWith("/api") also matched docs pages like /api-request and
  leaked one body MutationObserver per navigation
- codeBlockA11y: also observe the hidden attribute — Docusaurus tabs
  toggle panels via hidden (no childList mutation), so scrollable blocks
  inside an initially hidden tab were never re-evaluated for tabindex
- Extract Prism themes to src/prismThemes.js (docusaurus.config.js was
  past the 600-line red flag)
- concepts-components.mdx (current + 1.9.0 + 1.8.0): comment pointing
  hardcoded snippets to recursive_character.py to mitigate drift

Validated: clean build + IBM Equal Access scans 4/4 passing.

* replace-openapi-file-with-1.10

* migrate-prism-changes-to-1.10-version

* a11y-script-dont-block

---------

Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
2026-06-12 13:58:04 +00:00
d6d1692b70 fix(ci): make release Biome lint green — NUL-delimit file list + clear pre-existing lint errors (#13550)
The Lint Frontend job runs against `main` as the base. In a release
(workflow_dispatch) run it diffs the entire release branch (~915 files) and
re-lints nearly the whole frontend, exposing two issues that per-PR linting
never hits together:

1. xargs split starter-project spec paths containing spaces (e.g.
   "News Aggregator.spec.ts" -> "News" + "Aggregator.spec.ts"), producing
   `internalError/io: No such file or directory`. NUL-delimit the file list
   so spaces are preserved. (supersedes #13381)

2. The whole-branch diff surfaced 30 pre-existing Biome errors: 22
   noExplicitAny + 8 organizeImports. Resolved with real types where safe
   (freezeObject generic, ColDef defaults, messagesSorter field shape,
   VertexBuildTypeAPI, Record<string,string>, DragEvent<HTMLElement>,
   unknown for narrowed values) and justified biome-ignore for genuinely
   loose cases (polymorphic display values, test global stubs, captured
   unexported StreamCallbacks). Imports auto-sorted via biome.

Verified locally: biome check on the full release-vs-main file set is now 0
errors (was 30); tsc unchanged at its 303-error baseline (no new type errors).
2026-06-10 14:05:58 -07:00
29140c06f0 fix(ci): resolve validate-version output in release-lfx changelog link (1.11.0) (#13612)
fix(ci): resolve validate-version output in release-lfx changelog link

The create-release job references
needs.validate-version.outputs.current_version in its generated release
notes (the Full Changelog compare link), but validate-version was not in
the job's needs array, so the expression evaluated empty and the link
rendered as compare/v...lfx-vX.Y.Z (broken base).

Add validate-version to the create-release needs. This adds no real
serialization: create-release already waits on release-lfx, which
transitively requires validate-version via run-tests, and the job's
always() if-condition is unaffected.

Flagged by actionlint: property "validate-version" is not defined in
object type {build-docker, release-lfx}.
2026-06-10 08:42:28 -07:00
2df7c221f4 fix(ci): resolve validate-version output in release-lfx changelog link (1.10.1) (#13611)
fix(ci): resolve validate-version output in release-lfx changelog link

The create-release job references
needs.validate-version.outputs.current_version in its generated release
notes (the Full Changelog compare link), but validate-version was not in
the job's needs array, so the expression evaluated empty and the link
rendered as compare/v...lfx-vX.Y.Z (broken base).

Add validate-version to the create-release needs. This adds no real
serialization: create-release already waits on release-lfx, which
transitively requires validate-version via run-tests, and the job's
always() if-condition is unaffected.

Flagged by actionlint: property "validate-version" is not defined in
object type {build-docker, release-lfx}.
2026-06-10 08:42:21 -07:00
dea7b09257 fix(ci): create GitHub releases on the dispatched v-prefixed tag (1.11.0) (#13607)
fix(ci): create GitHub releases on the dispatched v-prefixed tag

The create_release job passed the bare version (v stripped) as the
release tag with no commit target, so when that tag did not exist
GitHub minted a new lightweight tag at the default-branch HEAD -- the
wrong commit, still carrying the previous version (main only adopts a
release's version via the post-release back-merge). Every release since
1.8.2 shipped a stray bare tag (1.8.2, 1.8.3, 1.9.0-1.9.6, 1.10.0)
pointing at a previous-version commit, and the GitHub release had to be
manually re-pointed to the real vX.Y.Z tag after each release.

- create_release now attaches the release to inputs.release_tag for
  stable releases; pre-releases keep their computed tag (e.g.
  1.10.0rc1) but it is minted at the release commit via 'commit:'.
- release-lfx.yml pins the minted lfx-v* tag to github.sha instead of
  the default branch.

The validate-tag-format guard (#12847) only blocks at dispatch time;
create_release was re-creating the very duplicates it guards against.
2026-06-10 08:41:41 -07:00
fb1edd99fd fix(ci): create GitHub releases on the dispatched v-prefixed tag (1.10.1) (#13609)
fix(ci): create GitHub releases on the dispatched v-prefixed tag

The create_release job passed the bare version (v stripped) as the
release tag with no commit target, so when that tag did not exist
GitHub minted a new lightweight tag at the default-branch HEAD -- the
wrong commit, still carrying the previous version (main only adopts a
release's version via the post-release back-merge). Every release since
1.8.2 shipped a stray bare tag (1.8.2, 1.8.3, 1.9.0-1.9.6, 1.10.0)
pointing at a previous-version commit, and the GitHub release had to be
manually re-pointed to the real vX.Y.Z tag after each release.

- create_release now attaches the release to inputs.release_tag for
  stable releases; pre-releases keep their computed tag (e.g.
  1.10.0rc1) but it is minted at the release commit via 'commit:'.
- release-lfx.yml pins the minted lfx-v* tag to github.sha instead of
  the default branch.

The validate-tag-format guard (#12847) only blocks at dispatch time;
create_release was re-creating the very duplicates it guards against.
2026-06-10 08:41:00 -07:00
1cf2bb4c16 fix(ci): scope nightly migration-test pre-releases to the langflow stack (1.10.1) (#13605)
fix(ci): scope nightly migration-test pre-releases to the langflow stack

The previous fix (#13599) added a global --prerelease=allow, which let
UNRELATED dependencies resolve to alphas: on nightly run 27274206250
the stable->nightly upgrade pulled pydantic 2.14.0a1 + pydantic-yaml
1.6.1a1, and the pydantic alpha breaks langchain-core at import time
(RunnablePassthrough pydantic ValidationError), failing the nightly
boot right after a successful install. Clean installs were fine - only
this upgrade path resolved the alpha combo.

Scope pre-release eligibility to the langflow lockstep stack instead:
uv accepts a pre-release when the package's own requirement carries a
pre-release marker, but not via transitive pins, and the nightly chain
is langflow -> langflow-base -> lfx -> langflow-sdk with exact ==devN
pins. Request each directly; langflow-sdk versions independently
(0.2.0.devN), so an explicit .dev0 floor marks it eligible while lfx's
exact pin selects the version. The 'latest' branch gets the same
treatment via .dev0 floors on all four.

Verified by dry-run against PyPI: langflow/langflow-base/lfx at
1.11.0.dev2 + langflow-sdk 0.2.0.dev2 resolve with pydantic staying at
stable 2.13.4.
2026-06-10 07:19:46 -07:00
84c3bbebcd fix(ci): scope nightly migration-test pre-releases to the langflow stack (1.11.0) (#13604)
fix(ci): scope nightly migration-test pre-releases to the langflow stack

The previous fix (#13599) added a global --prerelease=allow, which let
UNRELATED dependencies resolve to alphas: on nightly run 27274206250
the stable->nightly upgrade pulled pydantic 2.14.0a1 + pydantic-yaml
1.6.1a1, and the pydantic alpha breaks langchain-core at import time
(RunnablePassthrough pydantic ValidationError), failing the nightly
boot right after a successful install. Clean installs were fine - only
this upgrade path resolved the alpha combo.

Scope pre-release eligibility to the langflow lockstep stack instead:
uv accepts a pre-release when the package's own requirement carries a
pre-release marker, but not via transitive pins, and the nightly chain
is langflow -> langflow-base -> lfx -> langflow-sdk with exact ==devN
pins. Request each directly; langflow-sdk versions independently
(0.2.0.devN), so an explicit .dev0 floor marks it eligible while lfx's
exact pin selects the version. The 'latest' branch gets the same
treatment via .dev0 floors on all four.

Verified by dry-run against PyPI: langflow/langflow-base/lfx at
1.11.0.dev2 + langflow-sdk 0.2.0.dev2 resolve with pydantic staying at
stable 2.13.4.
2026-06-10 07:19:36 -07:00
481cdd3499 fix(ci): allow pre-releases when pinning the nightly in migration validation (1.10.1) (#13601)
fix(ci): allow pre-releases when pinning the nightly in migration validation

Migration Test: pip/venv (stable -> nightly) failed deterministically on
nightly run 27260425158 (twice, including a rerun):

  hint: langflow-base was requested with a pre-release marker (e.g.,
  langflow-base==1.11.0.dev1), but pre-releases weren't enabled
  (try: --prerelease=allow)

This is the first nightly publishing as a canonical .devN pre-release
of the langflow distribution (nightly -> stable bundle cutover). The
'latest' branch of the upgrade step already passes --prerelease=allow,
but the pinned-version branches do not. uv implicitly allows the
pre-release for the directly requested ==X.Y.Z.devN pin, yet langflow's
metadata pins langflow-base==X.Y.Z.devN transitively, and uv rejects
transitive pre-releases unless they are enabled - so the install fails
after the stable uninstall, sinking the migration test.

Add --prerelease=allow to both pinned-version install lines.
2026-06-10 04:50:33 -07:00
75e01157fe fix(ci): allow pre-releases when pinning the nightly in migration validation (1.11.0) (#13600)
fix(ci): allow pre-releases when pinning the nightly in migration validation

Migration Test: pip/venv (stable -> nightly) failed deterministically on
nightly run 27260425158 (twice, including a rerun):

  hint: langflow-base was requested with a pre-release marker (e.g.,
  langflow-base==1.11.0.dev1), but pre-releases weren't enabled
  (try: --prerelease=allow)

This is the first nightly publishing as a canonical .devN pre-release
of the langflow distribution (nightly -> stable bundle cutover). The
'latest' branch of the upgrade step already passes --prerelease=allow,
but the pinned-version branches do not. uv implicitly allows the
pre-release for the directly requested ==X.Y.Z.devN pin, yet langflow's
metadata pins langflow-base==X.Y.Z.devN transitively, and uv rejects
transitive pre-releases unless they are enabled - so the install fails
after the stable uninstall, sinking the migration test.

Add --prerelease=allow to both pinned-version install lines.
2026-06-10 04:50:26 -07:00
0b4e186f7b fix(ci): anchor langflow-base version extraction in nightly docker build (1.10.1) (#13593)
fix(ci): anchor langflow-base version extraction in nightly docker build

The first nightly on the 1.11 workspace (tag v1.11.0.dev0, run
27253229568) failed in Build Nightly Base Package within seconds:
'Base version format is incorrect'. The extraction (uv tree, grep
langflow-base unanchored, awk field 3, first line) broke because uv
tree now prints the langflow-base workspace-root line
('langflow-base v1.11.0.dev0', two fields) before any dependency line
('langflow-base[complete] v1.11.0.dev0' under the langflow root, three
fields), so the first match yields an empty field 3 and the format
check exits 1. uv tree's stderr is discarded, which made the real
cause invisible in CI.

Anchor to the root line and take field 2 instead - the exact pattern
the langflow (main package) extraction in this same file already uses,
which is why the main-package builds passed on the same tag. Verified
both extractions print 1.11.0.dev0 against a local checkout of
v1.11.0.dev0.
2026-06-10 00:26:02 -07:00
086e38898d fix(ci): anchor langflow-base version extraction in nightly docker build (1.11.0) (#13592)
fix(ci): anchor langflow-base version extraction in nightly docker build

The first nightly on the 1.11 workspace (tag v1.11.0.dev0, run
27253229568) failed in Build Nightly Base Package within seconds:
'Base version format is incorrect'. The extraction (uv tree, grep
langflow-base unanchored, awk field 3, first line) broke because uv
tree now prints the langflow-base workspace-root line
('langflow-base v1.11.0.dev0', two fields) before any dependency line
('langflow-base[complete] v1.11.0.dev0' under the langflow root, three
fields), so the first match yields an empty field 3 and the format
check exits 1. uv tree's stderr is discarded, which made the real
cause invisible in CI.

Anchor to the root line and take field 2 instead - the exact pattern
the langflow (main package) extraction in this same file already uses,
which is why the main-package builds passed on the same tag. Verified
both extractions print 1.11.0.dev0 against a local checkout of
v1.11.0.dev0.
2026-06-10 00:25:55 -07:00
f44f1ca546 fix(ci): rebalance backend test splits with measured durations + raise unit-test step timeout (1.10.1) (#13585)
fix(ci): rebalance backend test splits with measured durations + raise unit-test step timeout

Unit Tests - Python 3.12 - Group 3 has been failing at 98-99% on the
nightly: the job hits the nick-fields/retry 40-minute per-attempt timeout
and pytest is SIGTERM'd mid-test, so it looks like one flaky last test
when it is actually a deterministic timeout (and the internal retry plus
run-level retries can never succeed).

Root cause is twofold:

1. .test_durations was last regenerated ~May 2025 and covered only 2,219
   of ~9,500 current unit tests, so pytest-split weighted 77% of the
   suite at the 0.73s average. The expensive client-fixture tests
   (test_webhook.py, test_login.py - each pays a full create_app +
   lifespan boot per test, 60-120s late in a CI run) clustered into
   group 3's tail, making it ~8-10 minutes slower than its siblings
   (33:15 on 3.13, >40min on 3.12 which is additionally slowed by
   astrapy disabling SSL connection reuse on Python 3.12.0-11).
   The weekly store_pytest_durations workflow that should refresh the
   file has been dying at the 6-hour GitHub job limit every week (serial
   full-suite run no longer fits), so the file silently froze.

   This regenerates the file from real per-test wall-clock measured in
   the passing Python 3.13 nightly jobs (run 27246228762 attempt 1, all
   5 groups, parsed from the -vv xdist logs: per-worker start-to-start
   deltas). 9,508 tests now have measured durations; old entries are
   kept where no new measurement exists. Simulated least_duration
   split goes from one outlier group to 5 even groups (~24.6 min each)
   with the >50s tests spread 1-2 per group instead of 7 in one.

2. timeout_minutes 40 -> 50 gives headroom for matrix-cell variance
   (3.12 runs ~20% slower than 3.13) so a slightly slow cell degrades
   gracefully instead of burning 2x40min and failing the whole run.

Follow-ups (not in this PR): fix store_pytest_durations to run with
xdist or split groups so it fits the 6h limit; investigate the in-worker
degradation that makes client-fixture boots cost 8-12s early in a run
but 60-120s after ~30 minutes.
2026-06-09 21:28:45 -07:00
17447920f9 fix(ci): rebalance backend test splits with measured durations + raise unit-test step timeout (#13583)
Unit Tests - Python 3.12 - Group 3 has been failing at 98-99% on the
nightly: the job hits the nick-fields/retry 40-minute per-attempt timeout
and pytest is SIGTERM'd mid-test, so it looks like one flaky last test
when it is actually a deterministic timeout (and the internal retry plus
run-level retries can never succeed).

Root cause is twofold:

1. .test_durations was last regenerated ~May 2025 and covered only 2,219
   of ~9,500 current unit tests, so pytest-split weighted 77% of the
   suite at the 0.73s average. The expensive client-fixture tests
   (test_webhook.py, test_login.py - each pays a full create_app +
   lifespan boot per test, 60-120s late in a CI run) clustered into
   group 3's tail, making it ~8-10 minutes slower than its siblings
   (33:15 on 3.13, >40min on 3.12 which is additionally slowed by
   astrapy disabling SSL connection reuse on Python 3.12.0-11).
   The weekly store_pytest_durations workflow that should refresh the
   file has been dying at the 6-hour GitHub job limit every week (serial
   full-suite run no longer fits), so the file silently froze.

   This regenerates the file from real per-test wall-clock measured in
   the passing Python 3.13 nightly jobs (run 27246228762 attempt 1, all
   5 groups, parsed from the -vv xdist logs: per-worker start-to-start
   deltas). 9,508 tests now have measured durations; old entries are
   kept where no new measurement exists. Simulated least_duration
   split goes from one outlier group to 5 even groups (~24.6 min each)
   with the >50s tests spread 1-2 per group instead of 7 in one.

2. timeout_minutes 40 -> 50 gives headroom for matrix-cell variance
   (3.12 runs ~20% slower than 3.13) so a slightly slow cell degrades
   gracefully instead of burning 2x40min and failing the whole run.

Follow-ups (not in this PR): fix store_pytest_durations to run with
xdist or split groups so it fits the 6h limit; investigate the in-worker
degradation that makes client-fixture boots cost 8-12s early in a run
but 60-120s after ~30 minutes.
2026-06-09 21:28:02 -07:00
13a937c5b7 feat(ci): nightly → stable bundles via canonical pre-releases + decision record [gated] (#13528)
* docs: record nightly→stable bundle cutover plan (gated on lfx 1.10.0)

Add src/bundles/NIGHTLY.md documenting why langflow-nightly currently
renames the bundles (lfx and lfx-nightly ship the same lfx/ import, so a
stable bundle would co-install both and collide) and the deferred cutover
(Approach A: canonical pre-releases; B: lfx as a bundle extra), gated on
stable lfx 1.10.0 being published to PyPI.

Also expand two docstrings in scripts/ci/update_lfx_version.py to state
the deeper install-conflict reason, not just the resolve failure. No
behavior change.

* feat(ci): nightly Approach A — canonical pre-releases, drop nightly bundles [DRAFT/gated] (#13529)

feat(ci): nightly Approach A — canonical pre-releases, drop nightly bundles

DRAFT reference implementation of the nightly→stable-bundle cutover documented
in src/bundles/NIGHTLY.md. Publishes the nightly under CANONICAL package names as
.devN pre-releases instead of separate *-nightly distributions, so the stable
lfx-* bundles resolve against a single canonical lfx (no dual-lfx install
collision) and no nightly bundle packages are produced.

- tag scripts (pypi/lfx/sdk_nightly_tag.py): count .devN against the canonical
  PyPI histories instead of the *-nightly projects
- update scripts: stop renaming to *-nightly; set .devN versions; re-pin
  inter-package deps to exact canonical dev versions; delete the bundle
  rename/repin (update_lfx_dep_in_bundles, rename_bundles_for_nightly)
- release_nightly.yml: publish canonical pre-releases; remove bundle build,
  dist-nightly-bundles artifact, publish-nightly-bundles job + its gate; verify
  canonical names; main wheel glob dist/langflow-*.whl
- nightly_build.yml: drop the bundle git-add in the tag commit
- NIGHTLY.md: Approach A marked implemented + activation gate + A1/A2 follow-ups

Held as DRAFT: do not activate until stable lfx 1.10.0 is published AND the
nightly base is the next minor (release-1.11.0). A 1.10.0.devN core sorts below
1.10.0 and would fail the bundles' >=1.10.0 floor. Stacked on #13528.

(.secrets.baseline: incidental line-number shifts for the two workflows + prune
of pre-existing stale Pokédex Agent.json entries.)

* docs(ci): drop internal 'Approach A' label from nightly cutover comments

Comment- and docstring-only change across scripts/ci/* and the two nightly
workflows; no logic change. The two workflow edits stay single-line so
.secrets.baseline line numbers are unaffected. src/bundles/NIGHTLY.md keeps
its A/B decision-record framing intentionally.

* feat(ci): make nightly consumers work with canonical pre-releases

Follow-ups from the nightly cutover that are part of its blast radius (the
nightly now publishes canonical `.devN` pre-releases, not `*-nightly`
distributions):

- version.py: derive the "Nightly" label from the `.dev` version marker, since
  the canonical `langflow`/`langflow-base` distribution matches first in the
  lookup. Keeps the startup banner and telemetry `package` field identifying
  nightlies. Adds a canonical-dev test; updates the base-dev assertion.
- ci.yml check-nightly-status: query the canonical `langflow` project and pick
  the latest `.devN` release date instead of `langflow-nightly`'s `.info.version`.
- db-migration-validation.yml: install the nightly as `langflow[postgresql]==<dev>`
  (pre-release) instead of `langflow-nightly[...]`; verify via version("langflow").
- src/lfx/README.md: nightly install is `uv pip install --pre lfx`.
- NIGHTLY.md: rewrite the follow-ups section (these are addressed; Docker image,
  A2 meta-package, and website docs remain deferred by design).

The `langflowai/langflow-nightly` Docker image name is intentionally unchanged.

* fix(ci): correct nightly verify uv tree parsing + stale base-dep regex

Addresses review of #13528:

- release_nightly.yml LFX verify: `uv tree | grep lfx | head -n1` matches the
  bundle `lfx-ibm` first → 'Name lfx-ibm does not match lfx'. Root the tree with
  `uv tree --package lfx` so the first line is the lfx package itself.
- release_nightly.yml base verify: under canonical naming `langflow-base` prints
  as a top-level `langflow-base v<ver>` line, so the old $2/$3 field parse read
  name="v0.10.0" and version="". Use `uv tree --package langflow-base` and $1/$2.
- update_lf_base_dependency.py update_base_dep: regex only accepted ~=/==, so its
  CLI entry point couldn't match the current root dep `langflow-base[complete]>=0.10.0`.
  Add >= (parity with update_uv_dependency.py). The active nightly path uses
  update_uv_dependency.py and was unaffected.

* docs(nightly): point NIGHTLY.md status at the activation gate, not draft state

Per review of #13528: this file ships inside #13528 (#13529 was folded in), so
the 'stacked on prep #13528' + 'held as a draft' framing is stale and misleading.
Reword the status block to state the real guard is the activation gate (stable
lfx 1.10.0 published AND next-minor base), not merge/draft state. Also reword the
follow-ups heading 'decide before un-drafting' -> 'decide before activating'.
2026-06-09 13:23:55 -07:00
2524604a19 fix(ci): make Biome lint non-blocking for release workflow calls (correct implementation using allow-failure input)
- Revert invalid continue-on-error syntax on reusable workflow uses: job
- Add allow-failure input to ci.yml lint-frontend call
- Define allow-failure input parameter in lint-js.yml workflow_call
- Implement conditional exit-code swallow in lint-js.yml when allow-failure=true
- When release=true, Biome errors are non-blocking for release pipeline
- When release=false/unset, Biome failures block PR merge (existing ci_success gate)
2026-06-08 17:22:21 -04:00
801edf3bab Revert "fix(ci): make Biome lint non-blocking for release workflow calls (#13547)"
This reverts commit a1adb1aa6f.
2026-06-08 17:21:24 -04:00
a1adb1aa6f fix(ci): make Biome lint non-blocking for release workflow calls (#13547)
When ci.yml is called from release.yml (inputs.release == true), a
Biome failure was blocking all downstream publish and docker jobs via
needs.ci.result == 'success'. Biome was already excluded from the
ci_success gate for PR merges, but the reusable CI workflow itself
could still fail when the lint job errored.

Add continue-on-error: ${{ inputs.release == true }} to lint-frontend
so Biome failures are informational-only during release runs, while
remaining visible (and blocking ci_success) on regular PRs.
2026-06-08 17:04:51 -04:00
6610091697 fix(bundles): republish lfx-* at 0.1.1 with corrected pin + relax lfx floor for RC builds (#13542)
* fix(bundles): bump lfx-* bundles to 0.1.1 to republish with corrected lfx pin

The published 0.1.0 artifacts on PyPI carry stale lfx pins from before the lfx 0.5.0->1.10.0 version realignment:

- lfx-arxiv, lfx-duckduckgo: lfx>=0.5.0,<0.6.0 (hard-broken; the <0.6.0 cap can never resolve lfx 1.10.0)
- lfx-docling, lfx-ibm: lfx>=0.5.0 (uncapped floor/cap mismatch)

The source pin was already corrected to lfx>=1.10.0,<2.0.0 in #13516, but the bundles were never re-published. PyPI versions are immutable, so shipping the fix requires a version bump. Bump all four to 0.1.1 so the Release Bundles workflow cuts fresh wheels carrying the correct pin. Root pyproject keeps its >=0.1.0 floors (satisfied by 0.1.1); only the bundle dist versions and their uv.lock stamps change.

* fix(release): relax bundle lfx floor for pre-release builds + idempotent bundle publish

The RC pre-release run builds lfx as 1.10.0rc0, but bundles floor lfx at >=1.10.0. Under PEP 440 a pre-release sorts below the final, so 1.10.0rc0 fails >=1.10.0 and the cross-platform install test cannot resolve the bundle wheels against the RC lfx wheel.

build-base/build-main/build-lfx already rewrite their inter-package deps to the pre-release version when pre_release=true; build-bundles was the only release artifact missing that step. Add it: when pre_release=true, rewrite each bundle's lfx floor to the exact pre-release version, keeping the wide <2.0.0 BUNDLE_API cap. Stable source stays at >=1.10.0 -- only RC wheels are relaxed, at build time, so no source churn or re-tag.

Also make publish-bundles tolerate 'already exists' duplicate wheels on rerun, matching release_bundles.yml.
2026-06-08 13:18:56 -07:00
0d9f9112ef perf(telemetry): batched off-pool writer for transactions + vertex_builds (#13126)
* perf(telemetry): batched off-pool writer for transactions + vertex_builds

Adds TelemetryWriterService that buffers transaction and vertex_build rows
in memory and drains them in batched INSERTs via a dedicated AsyncEngine
(pool_size=1 for SQLite, 2 for Postgres, max_overflow=0). Retention is
amortized in a 60s sweeper instead of running on every insert.

Producers (log_transaction, log_vertex_build) enqueue instead of opening
a DB session, so telemetry traffic no longer competes with the
request-handling pool. Falls back to the legacy direct-write path via
LANGFLOW_TELEMETRY_WRITER_ENABLED=false.

Durability: in-flight rows spill to a diskcache.Deque per PID on shutdown
and are restored on next startup; orphan PID directories left by crashed
workers are adopted.

* perf(telemetry): tighten error handling and add coverage

Review feedback from pr-review-toolkit + silent-failure-hunter:

- Retention sweep snapshots the dirty-flow sets before commit and only
  clears them after it lands; a crashed sweep no longer drops the flows on
  the floor, so per-flow caps cannot drift unboundedly.
- Producer fall-through is no longer silent. transaction_service and
  log_vertex_build each log a one-shot WARNING when telemetry_writer_enabled
  is True but the writer is not running.
- Lifespan startup failure now logs ERROR instead of WARNING — if the user
  opted into the writer and it didn't come up, that's an error.
- Shutdown drain timeout no longer suppressed silently: logs a WARNING with
  pending row count + a hint to raise telemetry_writer_shutdown_drain_s.
- Writer's flush loop catches asyncio.CancelledError separately and
  re-prepends the in-flight batch to the buffer so teardown's disk spill
  catches it.
- After 6 consecutive batch failures the writer emits a loud ERROR with
  buffer depths so operators see sustained data-loss risk.

Added tests:
- test_sanitization_survives_writer_round_trip
- test_retention_failure_preserves_dirty_flows
- test_in_flight_batch_returned_on_cancel

* perf(telemetry): address copilot review

- _restore_from_disk + _adopt_orphan_outboxes now route through _enqueue so a
  large disk-spilled or orphan outbox can't bypass telemetry_writer_max_queue
  and OOM the process. Oldest rows are dropped and counted via the existing
  dropped_transactions / dropped_vertex_builds counters.
- chmod 0o700 the outbox root + per-PID directory so sanitized-but-still-
  sensitive payloads aren't exposed cross-user on multi-tenant hosts.
  Suppressed on platforms where chmod is a no-op (Windows).
- Added test_adopt_orphan_outboxes_honors_max_queue.

The remaining two copilot notes (private API access to DatabaseService and
diskcache.Cache.close) were also flagged by the in-tree review; tracking
separately. The "diskcache not declared" note is a false positive — the
dependency is at src/backend/base/pyproject.toml:76.

* perf(telemetry): address coderabbit review

- Sweeper hands off dirty sets via capture-and-clear so concurrent
  flushes during a retention pass aren't wiped by the post-commit
  subtract; failure path restores the snapshot.
- Stress README uses a concrete Postgres DSN matching the docker
  example instead of an unset env var.
- Test PID-probe loops bounded via a shared helper with pytest.fail
  fallback.

* perf(telemetry): swap diskcache outbox for stdlib sqlite (CVE-2025-69872)

Replaces the diskcache.Deque-backed spill outbox with a stdlib sqlite3
outbox (WAL mode, JSON payloads in TEXT). diskcache 5.6.3 has
CVE-2025-69872 (pickle deserialization RCE for an attacker with write
access to the cache dir), no fixed version released, and was never
declared as a dependency — import failed on Python 3.10.

The replacement is encapsulated in a small _Outbox helper that owns the
connection, schema, JSON codec, and lifecycle. The codec uses tagged
wrappers for datetime and UUID so SQLAlchemy's typed columns accept
restored payloads on the way back out.

Hardening informed by a survey of OTel collector, Fluent Bit, Vector,
Prometheus remote_write, Datadog agent, Logstash, and Filebeat:

- Per-PID outbox dirs now stamp an owner.json (hostname + Linux boot_id
  or time()-monotonic() proxy). Adoption only proceeds when host+boot
  match; cross-host or pre-owner-file dirs are logged and skipped so a
  recycled PID after a container restart cannot pull in a stranger's
  spill data.
- PRAGMA synchronous=FULL on the spill connection so the shutdown
  commit hits the platter (NORMAL only fsyncs on WAL checkpoint, which
  may never run if the process exits immediately after commit).
- Spill honors telemetry_writer_max_queue with drop-oldest, matching
  the producer-side overflow policy so a backlogged buffer at shutdown
  can neither stall teardown nor fill disk.
- append_all encodes payloads up front and only clears the deque after
  the transaction commits — no more partial-drain on mid-flight SQLite
  failure. drain collapses to a single DELETE FROM outbox after the
  SELECT.
- Exception handlers at the spill/restore/adopt boundaries narrowed to
  (sqlite3.Error, OSError) so genuinely unexpected exceptions
  propagate rather than being silently logged.

Tests cover: cross-host orphan skipped, missing-owner orphan skipped,
spill cap drops oldest, and a realistic UUID+datetime payload
surviving spill → restore → SQLAlchemy INSERT.

* [autofix.ci] apply automated fixes

* fix(telemetry): name wait_for inner tasks so pyleak can filter

Integration tests using @pyleak_marker were flagging an inner asyncio
task created by the telemetry writer's `wait_for(Event.wait(), ...)`
loops. The wrapper task is auto-named (Task-N) and lands mid-await
across the test boundary, so pyleak counts it as leaked. The writer
itself shuts down cleanly via teardown; the "leak" is a pattern
mismatch with pyleak's per-test snapshot model, not a real lifecycle
bug.

Wrap Event.wait() in an explicitly named task (`telemetry-writer-tick`,
`telemetry-writer-backoff`, `telemetry-sweeper-tick`) via a small
_wait_or_shutdown helper, and extend pyleak_marker with a task_name_filter
that excludes `telemetry-*` from leak detection. CI was green on earlier
commits in this branch only because the diskcache import error
prevented the writer from starting at all — fixing the import surfaced
this latent pyleak collision.

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* test(telemetry): disable writer in tests so reads see writes synchronously

* perf(telemetry): age out cross-host orphan outboxes on shared volumes

A pod on a shared PV (NFS/RWX) cannot adopt rows from a dead pod on a
different host because the dead pod's hostname doesn't match. Without a
janitor those directories accumulate forever. Add an owner-file mtime
heartbeat from the sweeper plus a prune pass that deletes cross-host
outboxes whose owner file hasn't been touched within
telemetry_writer_orphan_max_age_s (default 1h). Same-host orphans still
flow through the existing adoption path.

* perf(telemetry): add byte-aware flush + drop strategy

Today the writer bounds memory by row count only, so a worker logging fat
vertex_build artifacts can hold tens of MB per row and silently dwarf the
configured max_queue. Add a size_strategy switch ('count' | 'bytes' |
'either', default 'count' for parity) with two byte thresholds:

- batch_size_bytes (256KB) caps per-flush INSERT size when bytes apply
- max_queue_bytes (200MB) drops oldest by bytes when bytes apply

Parallel sizes deques mirror the payload buffers so accounting stays
consistent across drain, spill, restore, and the cancel/retry rollback.
Single rows above the byte budget are still emitted (no row is refused);
operators get dropped_*_bytes counters alongside dropped_*.

* test(telemetry): tighten byte-strategy assertions and cover end-to-end paths

Address gaps in the byte-aware strategy tests:
- pin exact dropped counts and remaining buffer state for the drop-by-bytes
  case (was 'greater than zero')
- compute exact drain count from encoded size (was a 1-4 range)
- round-trip bytes through spill+restore to verify size deque is rebuilt
- round-trip bytes through orphan adoption for the same reason
- exercise the sweeper loop end-to-end to confirm heartbeat + prune are
  wired in (previously only unit-tested in isolation)

Clarify in the settings docs that the byte caps measure encoded JSON size,
not Python in-memory footprint.

* ref: updates to telemetry writer PR (#13294)

* fix(telemetry-writer): harden error handling and add missing test coverage

Critical:
- teardown() now re-raises CancelledError after awaiting the writer task so the
  asyncio cancellation chain propagates correctly on lifespan task kill
- suppress(OSError) on owner file write replaced with explicit error log so
  operators know when disk-spilled rows will not be recoverable on restart

High / important:
- Escalation threshold check changed from == to >= so the error log fires on
  every failure past the threshold, not just the 6th
- Dead `except OSError` on time.time() - time.monotonic() changed to
  `except Exception` since time functions cannot raise OSError
- Removed redundant `from uuid import UUID` inside _run_retention_pass (already
  imported at module top)
- Orphan directory cleanup: replaced blanket suppress(OSError) with per-child
  suppress so ENOTEMPTY on an individual rmdir doesn't abort the whole loop and
  leave the parent directory leaking silently; outer failure now logs at debug

Tests (3 new):
- test_retention_sweep_caps_vertex_builds_per_vertex: inserts 8 builds for a
  single vertex with max_per_vertex=3 so the per-vertex DELETE subquery
  actually executes (previous test used 8 distinct vertex IDs, bypassing it)
- test_either_strategy_trips_on_bytes_first: verifies bytes can be the first
  trigger under 'either' strategy (previous test only covered the count-first path)
- test_writer_retries_on_batch_failure: injects 2 flush failures then success;
  confirms failed_batches increments, rows are preserved in the buffer, and
  flushed_rows reflects the final successful write

* ci: add stress-tests job to nightly build pipeline

Wires the stress-tests workflow into nightly so telemetry write stress
tests run automatically. Also gates release-nightly-build and
slack-notification on stress-tests result so a stress regression blocks
the nightly release and surfaces in Slack.

* ci: run stress tests in nightly without blocking release

Stress tests are informational for now — a failure is visible in the
workflow run and Slack but does not gate the nightly release or build.

* fix(telemetry-writer): address PR review on cancelled teardown and nightly stress tests

- Add the stress-tests.yml reusable workflow (was untracked, so the
  nightly stress-tests job could never resolve)
- Notify Slack on stress-tests failure (add to slack-notification gate
  and FAILED_JOB detection as non-blocking)
- Run sweeper cancel, disk spill, and engine dispose in a finally so a
  cancelled teardown() still persists the in-memory buffer
- Add tests for the cancelled-teardown spill path and the >= escalation
  threshold; drop the misleading wait-loop in the retry test

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
2026-06-05 17:32:47 +00:00
1ab6251164 feat: sync langflow and lfx versions (#13176)
* feat(lfx): synchronize LFX onto Langflow major.minor version line (Phase 1)

- Bump src/lfx/pyproject.toml from 0.5.0 to 1.10.0
- Tighten langflow-base lfx pin: ~=0.5.0 → ~=1.10.0
- Fix release.yml pre-release boundary from next-major to next-minor (<X.(Y+1).dev0)
- Add minor-parity soft check to scripts/release-lfx.sh
- Extend make patch to sync lfx version alongside langflow/base/frontend
- Document LFX compatibility contract in RELEASE.md

Contract: LFX X.Y.N is compatible with any Flow from Langflow X.Y.M.

* feat(lfx/upgrade): add compatibility checker (Phase 2)

* fix(lfx/upgrade): fix registry_code guard and types order-sensitivity in checker

* feat(lfx/upgrade): implement safe-upgrade applier (Phase 2)

* feat(lfx): add lfx upgrade command (Phase 2)

* feat(lfx/run): add --upgrade-flow option (Phase 2)

* feat(lfx/serve): add --upgrade-flow option (Phase 2)

* update lfx pyproject version

* [autofix.ci] apply automated fixes

* test(lfx/upgrade): add v1.9.0 starter flow fixtures for upgrade integration tests

* test(lfx/upgrade): add v1.9.0 starter flow fixtures and real-flow integration tests

* [autofix.ci] apply automated fixes

* fix(lfx/upgrade): use alias-aware registry lookup so renamed components are not falsely blocked

* fix(lfx/upgrade): handle outer flow envelope and file-path inputs in upgrade checks

* fix(lfx/upgrade): fail-fast on upgrade_flow errors; add regression tests for Bugs 1-3

* chore(tests): remove bug-number labels from test comments

* [autofix.ci] apply automated fixes

* fix(lfx/upgrade): address PR review comments

- Reject .py files early in serve --upgrade-flow with a clear error message
- Preserve outer flow envelope metadata (name, description, etc.) when lfx upgrade --write rewrites a file
- Move Mapping import under TYPE_CHECKING to satisfy Ruff TC003
- Assert fixture flows and registry are non-empty so parametrized tests cannot pass vacuously
- Remove unused capsys arg and replace print with sys.stderr.write (Ruff T201/ARG001)
- Add upgrade_flow param to run command docstring

* fix(lfx/upgrade): preserve envelope on run, apply nested upgrades (#13200)

* fix(lfx/upgrade): preserve envelope on run, apply nested upgrades

Three follow-ups on top of the upgrade tooling:

- run --upgrade-flow: re-attach the inner graph to the outer envelope before handing the flow to aload_flow_from_json. Previously the upgrade path unwrapped {"data": ...} and passed the inner dict to the loader, which raised KeyError: 'data'. Adds happy-path tests for envelope and flat file inputs.
- applier: recurse into one level of nested flow nodes (node.data.node.flow.data.nodes), matching the checker. Without this, outdated_safe nodes inside grouped components were reported but never written. Adds a regression test.
- upgrade --write test: assert the envelope is preserved (name, description, endpoint_name, etc.) instead of the previous unwrapped-output expectation, so the test matches the actual fix in this PR.
- Drive-by ruff cleanup: D417 docstring, SIM103, SIM108, E501, RUF059.

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* fix(lfx/upgrade): fix loader KeyError, outer-envelope checker gap, and nested-node applier

Three bugs from ogabrielluiz's PR review:

1. run/base.py loader KeyError: file-path branch unwrapped the outer envelope
   for the checker but passed the inner dict to aload_flow_from_json, which does
   flow_graph["data"] unconditionally. Re-wrap as {"data": flow_dict} before the
   loader call. Applied consistently to all three input paths (--flow-json,
   --stdin, file-path).

2. run/base.py zero-node checker for --flow-json/--stdin: those paths stored the
   raw parsed JSON without unwrapping, so a caller passing an exported flow
   {"name":..., "data":{...}} caused the checker to see zero nodes and silently
   pass. Now unwrap with raw.get("data", raw) matching the file-path branch.

3. applier.py nested flows never upgraded: the early `continue` on top-level
   nodes not in safe_ids prevented the nested-node loop from running. Restructured
   so the nested check fires unconditionally for every top-level node, mirroring
   checker.py:160-165.

* fix(lfx/run): unify outer-envelope unwrap across all --upgrade-flow input paths

File-path reads now call raw.get("data", raw) matching --flow-json and
--stdin, so the upgrade checker always sees the inner graph. This also
removes the has_envelope/re-wrap machinery that was double-wrapping
outer-envelope files when passing to aload_flow_from_json.

Fix two test assertions that described the old double-wrapped shape.

* fix(lfx/upgrade): inject registry into upgrade_command; fix Makefile lfx pin regex

- upgrade_command now accepts an optional registry parameter so tests
  can pass the dict directly instead of mocking load_registry_from_index
- Makefile sed regex broadened from \"lfx~=.*\" to match both ~= and >=
  forms so make patch works after release.yml rewrites the pin
- Echo label changed to LFX (synced) to clarify the variable is the
  shared Langflow version, not a separate LFX-specific value

* fix(make/patch): fix langflow-base sed regex and validation grep

The dependency in pyproject.toml is "langflow-base[complete]>=X.Y.Z",
not the "langflow-base==X" form the original regex expected, so make
patch silently left the pin unchanged and the validation step always
failed. The sed pattern now matches any extras/operator combination and
rewrites to the canonical [complete]>= form; the grep uses -F so the
[complete] brackets are treated literally.

* test(lfx): add patch regex tests and symmetric safe-mode envelope tests

test_patch_regexes.py — 15 tests covering the Python regexes embedded in
the Makefile patch target. Exercises all three substitutions (langflow-base
pin, lfx pin, version field) against every realistic pin format including
the >=X.Y.Z,<dev0 form that release.yml writes. Would have caught the
langflow-base==.* vs [complete]>= mismatch before manual testing.

test_base.py — two new TestUpgradeFlowOption tests:
  test_upgrade_flow_safe_envelope_inline_json_loads_successfully
  test_upgrade_flow_safe_envelope_stdin_loads_successfully
Symmetric to the existing file-path envelope test; verifies that --flow-json
and --stdin with an outer-envelope flow also pass {"data": inner} to the
loader after safe upgrades, not a double-wrapped {"data": outer_envelope}.

* fix(lfx/upgrade): address review: compat checker, shared gate, fail-fast registry

Checker correctness:
- _outputs_are_compatible: drop cosmetic display_name from the breaking check;
  treat widened output types as safe (flow types must be a subset of registry
  types), only narrowing breaks downstream edges.
- _input_types_contained: stop flagging widened input_types as breaking; keep
  narrowing as the only breaking case; fix misleading comments.
- check_flow_compatibility now recurses fully into nested grouped components
  (symmetric with the applier) and accepts a pre-built registry lookup.

CLI/run/serve:
- New lfx.upgrade.cli_gate (UpgradeFlowMode enum, UpgradeFlowError,
  apply_upgrade_gate) shared by run_flow and serve_command so the two
  --upgrade-flow paths can't diverge.
- run_flow: extract _materialize_flow_dict and route gating through the shared
  helper.
- run/serve --upgrade-flow options typed as UpgradeFlowMode (check|safe choices).
- lfx upgrade: load_registry_from_index fails fast when the bundled registry is
  empty/missing instead of silently marking every node blocked; ASCII-only report
  output; new --strict flag; build the registry lookup once and reuse it.

Docs/tests:
- RELEASE.md: migration note for the lfx 0.5.0 -> 1.10.0 version jump.
- Regression tests for the checker fixes, the shared gate, fail-fast registry,
  --strict, and serve --upgrade-flow parity.

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* fix(lfx): consolidate flow-envelope handling and fix version-ceiling regex

Extract the outer-envelope unwrap/rewrap logic (previously hand-rolled as
raw.get("data", raw) with subtly different rules across serve, run, and
upgrade) into a single lfx.utils.flow_envelope module with split/merge
helpers. This fixes a serve bug where an enveloped flow had its inner graph
written bare to the temp file, making the loader's flow_graph["data"] raise
KeyError.

Also fix release.yml's major.minor extraction: the greedy sed grabbed the
upper bound from a range pin (>=1.10.0,<1.11.dev0), drifting the version
ceiling up one minor each release cycle. Anchor to the first version instead.

* fix(lfx): upgrade-flow gate reads bundled component index, not empty cache

The --upgrade-flow=check|safe gate on lfx run and lfx serve rejected every
flow as 'blocked'. Both call sites passed component_cache.all_types_dict to
apply_upgrade_gate, but that cache is populated lazily after services start,
so at gate time it is empty -- an empty registry classifies every node as
blocked. The standalone lfx upgrade command was unaffected because it reads
the bundled _assets/component_index.json instead.

Make the gate own registry loading: apply_upgrade_gate now defaults
all_types_dict to None and loads the bundled index (the same source lfx
upgrade uses) via a new _load_bundled_registry helper, raising
UpgradeFlowError on a missing/empty index so a broken install fails loudly
instead of silently blocking every component. Both call sites pass mode=
and let the gate load the registry.

Existing gate tests mocked component_cache with a populated registry, which
is exactly what hid the bug; repoint them at the new _load_bundled_registry
seam and add regression tests that do not mock the registry, including an
end-to-end run_flow check against a real clean v1.9.0 starter flow.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
2026-06-05 01:30:59 +00:00
3e57126b91 fix: remove duplicate uv.lock from workspace member (#13326)
* fix: remove src/backend/base/uv.lock and Dockerfile references

- Deleted src/backend/base/uv.lock (monorepo should have only one uv.lock at root)
- Removed COPY ./src/backend/base/uv.lock lines from all Dockerfiles:
  - docker/build_and_push.Dockerfile
  - docker/build_and_push_base.Dockerfile
  - docker/build_and_push_ep.Dockerfile
  - docker/build_and_push_with_extras.Dockerfile
  - docker/dev.Dockerfile

Fixes LE-1093

* fix: remove stale base uv.lock regeneration paths
2026-05-30 13:05:34 +00:00
3ff3c576ca fix(ci): re-enable migration-pip-venv now that nightly bundles resolve (#13421)
The PyPI langflow-nightly is pip-installable again, so the temporary
`if: false` guard added on 2026-05-28 is no longer needed.

#13418 (release-1.10.0) and its forward-port #13419 (main) gave the
extension bundles their own nightly track: langflow-nightly now depends on
lfx-arxiv-nightly / lfx-duckduckgo-nightly / lfx-ibm-nightly, which pin
lfx-nightly==0.5.0.dev* instead of an unsatisfiable stable lfx>=0.5.0,<0.6.0.

Verified against the published dev57 wheels: a dry-run resolve of
langflow-nightly[postgresql] succeeds (549 packages, exit 0), pulling
lfx-nightly==0.5.0.dev57 via the *-nightly bundle variants.
2026-05-29 15:41:58 -07:00
9044b8f0c4 fix(ci): give extension bundles a nightly track so langflow-nightly resolves (#13418)
The nightly tagger renames lfx -> lfx-nightly, but the extension bundles
were published as stable lfx-<name> wheels pinning lfx>=0.5.0. So
langflow-nightly depended on the stable lfx-arxiv (etc.), which dragged in
lfx>=0.5.0 -- a version only published under the lfx-nightly name. Installing
the nightly therefore failed:

    Because only lfx<=0.4.5 is available and lfx-arxiv==0.1.0 depends on
    lfx>=0.5.0, ... your requirements are unsatisfiable.

Give the bundles their own nightly track, mirroring lfx/base/main:

- update_lfx_version.py now renames each src/bundles/* package to
  lfx-<name>-nightly, versions it <base>.dev<N> (sharing lfx's dev number),
  and repoints the root langflow deps + [tool.uv.sources] workspace entries
  at the -nightly names. Each bundle's own lfx dep was already repinned to
  lfx-nightly==<dev>. Only the [project] name/version change -- entry points
  and the import package stay as lfx_<name>, so extension discovery is intact.
- release_nightly.yml publishes the nightly bundle wheels to PyPI and gates
  publish-nightly-main on them, so langflow-nightly's exact == pins always
  reference bundles published in the same run.

No build/test/Docker changes needed: the bundles are already built and
committed under the nightly tag, the cross-platform test resolves them via
--find-links, and Docker builds from the regenerated workspace lock.
2026-05-29 13:37:41 -07:00
be7edcbda1 chore: unconstrained lfx version for bundles (#13415) 2026-05-29 10:54:38 -07:00
dbe482c800 fix(ci): lockstep langflow-nightly and langflow-base-nightly versions (#13413)
* fix(ci): lockstep langflow-nightly and langflow-base-nightly versions

The nightly pipeline computed each package's .devN number independently
(pypi_nightly_tag.py queried each package's own PyPI history and
incremented separately), while langflow-nightly pins an exact
langflow-base-nightly[complete]==X.Y.Z.devN dependency. Because
langflow-nightly publishes on more nights than langflow-base-nightly,
the two counters drift (live: dev54 vs dev48). Each time base lags a
publish while main succeeds, the newest langflow-nightly pins a base
dev that does not exist yet, so 'uv pip install langflow-nightly'
becomes unsatisfiable and silently falls back to a weeks-old version.

Fix: version the two packages in lockstep. pypi_nightly_tag.py now
derives a single shared dev number from max(dev across BOTH packages'
PyPI histories) + 1 (restricted to the root base_version), so main and
base always get the identical tag and main's exact pin always
references a base version built and published in the same run. The tag
is computed in a single invocation ('both' mode) so the release and
base tags cannot drift across separate calls.

Also hardens tag computation: 404 / network / malformed responses for
either package now contribute nothing instead of crashing, a
base-version bump resets the dev counter cleanly, and non-dev/final
releases never advance it.

Adds scripts/ci/test_pypi_nightly_tag.py (unit tests) and a
ci-scripts-test.yml workflow to run scripts/ci tests on PRs.

* fix(ci): fail closed on non-404 PyPI errors in nightly tag computation

_all_dev_numbers() previously returned [] for ANY failure (network error,
non-404 HTTP status, malformed 200), which is unsafe: _shared_nightly_version()
takes max(dev)+1 across both packages, so a transient lookup failure on the
higher-versioned package silently LOWERS the next tag. E.g. if the langflow-nightly
lookup fails while langflow-base-nightly reports dev48, the script would emit
v1.10.0.dev49 even though langflow-nightly already published through dev54 — the
workflow then force-recreates an old git tag and fails publishing an already-existing
PyPI version.

Fail closed: only a true 404 (package has no releases) contributes []; network
errors, 5xx/non-404 statuses, and malformed 200s now raise so the nightly job aborts
before mutating tags. Adds tests for malformed/5xx/network failures and a direct
regression test for the higher-package-lookup-failure scenario.

* Update scripts/ci/test_pypi_nightly_tag.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update test_pypi_nightly_tag.py

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-05-29 10:14:46 -07:00
58a75ffb0d feat(ci): reduce macOS Intel CI matrix and add macOS support documentation (#13328)
feat(ci): reduce macOS Intel CI matrix and add macOS support docs

- Remove Python 3.10 from macOS Intel stable CI matrix (keep 3.12 only)
- Add comprehensive macOS support documentation
- Document Python 3.14 support status (limited on Intel, no PyTorch)
- Reduces CI costs by ~$5-7 per run (~$1,800-2,500/year)

Implements LE-781 recommendations R2 and R3 from LE-265 investigation.

Related: LE-265, PR #12477
2026-05-29 16:20:10 +00:00
f400102088 ci: disable pip/venv DB migration test while langflow-nightly is pip-uninstallable
The PyPI langflow-nightly is currently pip-uninstallable: its core bundle
deps (lfx-arxiv, lfx-duckduckgo) pin stable lfx>=0.5.0,<0.6.0, but stable
lfx tops out at 0.4.4 (the 0.5.0 line ships only as lfx-nightly, a separate
package name that cannot satisfy an lfx pin). So the pip/venv migration job
fails at the install step.

The Docker Compose migration job is unaffected (it runs the prebuilt nightly
image) and stays enabled. Re-enable once either a stable lfx 0.5.x is
published to PyPI or nightly bundle variants pinning lfx-nightly are
published and langflow-nightly depends on them.
2026-05-28 17:22:58 -07:00
fb4dfd49fd ci: sync workflow fixes from main into release-1.10.0 (fix broken nightly) (#13400)
ci: sync workflow fixes from main into release-1.10.0

Brings the latest CI/workflow fixes from main into release-1.10.0 so the
nightly build is unblocked and the two branches share the same hardening.

Most important fix: release_nightly.yml's "Verify Nightly Name and Version"
step now uses the anchored grep '^langflow(-nightly)?[[:space:]]' instead of
'grep langflow | grep -v langflow-base | grep -v langflow-sdk'. The old
filter started matching two packages once the langflow-stepflow workspace
package was added, so the name became multi-line and the check failed
(this is what broke the nightly build).

Also carries main's fixes for cross-platform install
(--prerelease=if-necessary-or-explicit), docker build workflows,
db-migration-validation, and the backend test timeout bump (30 -> 40).
3-way merged from the common ancestor, so release-only changes are
preserved; no release-only feature workflows are modified.
2026-05-28 15:38:29 -07:00
cb7b838d45 feat: Add DB migration validation workflow for nightly builds (#13249)
* feat: Add DB migration validation workflow for nightly builds (LE-1259)

- Implements automated DB migration testing for nightly builds
- Tests two scenarios: pip/venv and Docker Compose migrations
- Validates migration from stable to nightly versions
- Verifies data persistence (witness flows) across migrations
- Integrated into nightly_build.yml workflow
- Includes Slack notifications for migration test results

This addresses the critical blocker identified by QA team for ensuring
safe database migrations in production deployments.

* docs: Add DB migration validation documentation

- Comprehensive guide for LE-1259 implementation
- Detailed test scenarios and execution details
- Environment configuration and success criteria
- Monitoring and troubleshooting guidelines
- Placed in docs/docs/Deployment/ for easy access

* fix: address PR review comments for DB migration validation

- Use actual nightly tag from create-nightly-tag output instead of hardcoded :latest
- Wire POSTGRES_VERSION env var into postgres service image tag
- Remove unnecessary checkout steps from both migration jobs
- Update step name from 'Create witness flow and credentials' to 'Create witness flow'
- Add -f flag to curl commands for fail-fast behavior
- Add flow creation verification with error handling
- Fix version extraction logic to properly test nightly build instead of PyPI latest
- Remove deprecated docker-compose version field
- Remove duplicate Slack notification job (consolidated in nightly_build.yml)

Addresses all 9 issues identified by @ogabrielluiz in PR review

* docs: remove implementation summary from user-facing docs

Per @ogabrielluiz review feedback, this file reads as an implementation
summary (Jira ticket, branch name, 'Next Steps', 'Files Changed: 2')
rather than user-facing documentation. The Deployment section is for
end-user docs, and this content is better suited for the PR description.

Also not added to sidebars.js, so would be an orphan page.

* fix(workflows): address 6 issues from Gabriel's second review of DB migration validation

Fixes all remaining issues identified in PR #13249 review:

1. Remove schedule trigger - only works on default branch, would cause duplicate runs
2. Fix postgres service image - hardcode to postgres:16 (env context not available in services)
3. Add curl fail-fast flags - use -fsSL for immediate failure on errors
4. Add flow ID verification - check witness data creation succeeded before proceeding
5. Remove orphaned Slack JSON - cleanup leftover from removed notify-results job
6. Fix version extraction - strip 'v' prefix for pip install (${VERSION#v})
7. Fix Docker image reference in nightly_build.yml - pass full image path with tag
8. Fix success notification - check migration validation didn't fail

All changes validated locally:
- YAML syntax validation passed
- Docker Compose config validated
- Tag manipulation logic tested (v prefix handling)
- Curl command structure verified
- Test credentials marked with pragma allowlist secret comments

Related: LE-1259, PR #13249
Depends on: PR #13212 (Docker volume permissions fix)

* fix(ci): address all PR review blockers for DB migration validation workflow

- Add auto_login token auth to all 4 API call sites (Create/Verify witness in both jobs)
- Add credential witness (POST /api/v1/variables/ with type: Credential) to prove
  encrypted-column migrations ran correctly
- Verify witness variable persists after upgrade in both jobs
- Strip 'v' prefix from nightly Docker tag before docker-compose sed replacement
- Fix langflow.__version__ crash - use importlib.metadata instead
- Remove deprecated 'version: 3.8' from inline docker-compose YAML
- Remove bogus LANGFLOW_SUPERUSER/LANGFLOW_SUPERUSER_PASSWORD env vars
- Remove LANGFLOW_SKIP_AUTH_AUTO_LOGIN (only works on nightly, not stable images)
- Fix shellcheck SC2016/SC2102 issues flagged by actionlint
- Mark test credential values with pragma allowlist secret (false positives)
- Update .secrets.baseline with known false positives in new workflow file

* fix(ci): remove duplicate db-migration-validation job after merge

* fix(ci): include default_fields in witness variable payloads

* fix(ci): handle compressed API responses in witness verification

* fix(ci): uninstall stable langflow before installing nightly in pip-venv job

langflow and langflow-nightly both own the same site-packages/langflow/
namespace via a shared langflow-base dependency. Without an explicit
uninstall, the stable copy wins at runtime and python -m langflow still
boots the old version - meaning no nightly migration is exercised and
the test produces a false positive.

Explicitly uninstall langflow + langflow-base before the nightly install
so the namespace is fully handed over to the nightly package.

---------

Co-authored-by: Eric Hare <ericrhare@gmail.com>
2026-05-28 14:26:31 -07:00
e4d5190be4 chore(ci): Fix 403 in DB migration validation witness flow (#13376)
fix auto login migration validation
2026-05-28 08:32:20 -03:00
6b2ca4bcd8 fix: restore qdrant deps and migration workflow 2026-05-27 20:09:57 -07:00
288bc8efe4 fix(ci): use local bundle wheels in nightly install tests (#13257) 2026-05-20 19:46:40 -07:00
5a8c17b22d ci: backport bundle/nightly CI fixes from main (#13206, #13207, #13208) (#13210)
* ci: stop publishing -nightly bundle variants; release bundles on demand (#13206)

Bundles (lfx-arxiv, lfx-duckduckgo) change infrequently enough that a
nightly cadence is overkill. Drop the rename-to-`-nightly` and bundle
publish lanes from the nightly pipeline and add a dedicated
workflow_dispatch-only release_bundles.yml for purposeful releases.

- nightly_build.yml: drop update_bundle_versions.py invocation and the
  src/bundles/*/pyproject.toml git-add guard.
- release_nightly.yml: remove build-nightly-bundles and
  publish-nightly-bundles jobs; untether test-cross-platform and
  publish-nightly-main from them.
- release_bundles.yml (new): build all src/bundles/* wheels, run a
  cross-OS install smoke test, then publish to PyPI under their stable
  names. Tolerates already-published versions so re-runs without a
  version bump are no-ops.

release.yml still owns bundle publishing as part of main releases.

Manual follow-up (PyPI admin): delete the lfx-arxiv-nightly and
lfx-duckduckgo-nightly projects from PyPI.

* ci(release_bundles): build lfx wheel locally for smoke test (#13207)

The release_bundles.yml smoke test installs bundle wheels into a fresh
venv, which makes uv resolve their `lfx>=X.Y,<Z` pin against PyPI. When
bundles are released ahead of a matching lfx (typical case — bundles
ship more often than lfx bumps land on PyPI), the resolve fails:

  Because only lfx<=0.4.3 is available and lfx-arxiv==0.1.0 depends on
  lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv==0.1.0 cannot be
  used.

Build the lfx wheel from the current ref in the build-bundles job and
install it alongside the bundle wheels in the smoke test. The lfx
wheel ships as a separate `dist-lfx-smoketest` artifact and is NOT
included in the publish-bundles step — only the `dist-bundles`
artifact gets pushed to PyPI.

* ci(nightly): re-pin bundle lfx deps to lfx-nightly during nightly build (#13208)

The nightly pipeline renames the workspace `lfx` package to `lfx-nightly`
in `src/lfx/pyproject.toml` and the root workspace dep, but leaves each
`src/bundles/*/pyproject.toml`'s `"lfx>=X.Y,<Z"` pin unchanged. With the
workspace `lfx` gone and PyPI still on lfx 0.4.3, the create-nightly-tag
job's `uv lock` fails:

  × No solution found when resolving dependencies for split [...]
  ╰─▶ Because only lfx<=0.4.3 is available and lfx-arxiv depends on
      lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv's requirements
      are unsatisfiable.

`update_lfx_version.py` now also rewrites the `lfx` (or already-rewritten
`lfx-nightly`) specifier in every `src/bundles/*/pyproject.toml` to
`lfx-nightly==<dev version>`, mirroring how `update_lf_base_dependency`
re-pins the lfx dep inside `langflow-base`. The lookahead in the regex
prevents matching sibling packages like `lfx-arxiv` / `lfx-duckduckgo`.

No-op when no bundle pyprojects exist (e.g. on main today), so this is
safe to merge ahead of the bundle extraction landing on main. Restored
the guarded `git add src/bundles/*/pyproject.toml` step so the modified
bundle pyprojects ride the same commit/tag as the rest of the nightly
version bumps.

Cherry-pick to release-1.10.0 to unblock nightlies cut from that branch.
2026-05-19 10:12:54 -07:00
c901602bc2 fix(ci): rename bundles to -nightly during nightly build (#13193)
fix: rename bundles for nightly build
2026-05-18 18:32:44 -07:00
cc009c9133 feat(lfx): Bundle Separation and LFX Extension Framework (#13043)
* feat(lfx): installed-package + seed-directory discovery for production install (LE-1022)

Adds the read-only production install path for Modes A, B, and C of the
Bundle Separation iteration. Manifest-shipping pip-installed
distributions and seed-directory subdirectories are discovered at server
startup and registered as Extensions at @official.

* discovery.py: walks importlib.metadata.distributions() + the
  $LANGFLOW_SEED_DIR / /opt/langflow/bundles seed root; produces
  DiscoveredExtension records and typed errors for malformed manifests
  / configured-but-missing seed dirs.
* registry.py: ExtensionRegistry service with the immutability
  invariant for installed and seed entries. Mutation verbs (uninstall,
  disable, enable, install, update_entry) all raise
  ExtensionImmutableError carrying the typed
  installed-extension-immutable / seed-directory-immutable code so the
  invariant is testable today; the CLI uninstall surface ships in B4.
* lfx extension list: read-only inspector with text and JSON output
  for operators inspecting Mode B/C images.
* Errors: four new typed codes
  (installed-extension-immutable, seed-directory-immutable,
  seed-directory-not-found, duplicate-extension-id) plus snapshot
  coverage in tests/unit/extension/test_errors.py.
* Tests: 165 extension tests pass, including the LE-1022 acceptance
  cases -- three pip-installed wheels visible at @official, three seed
  bundles visible at @official, and the parametrized service-layer
  immutability check across every mutation verb.
* Docs: docs/Deployment/deployment-extensions-production.mdx covers
  the Dockerfile template, k8s deployment notes, the bundle packaging
  convention (extension.json shipped via package-data), and
  troubleshooting for the typed error codes.

* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (#12967)

* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)

Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.

What's in lfx.extension:

- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
  `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
  `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
  None-only so non-null values produce a dedicated
  `field-deferred-in-this-milestone` error instead of a generic schema wall.
  `bundles` accepts a list but rejects length > 1 with
  `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
  re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
  artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
  per discriminant. Codes are registered in `ERROR_CODES`; an
  `ExtensionError` constructed with an unknown code raises at construction
  time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
  path-safety (no `..`, no absolute paths, no symlink escape), AST
  inspection of every `.py` (syntax, Component subclass present, build()
  declared, top-level `import *`, top-level I/O primitives), and an opt-in
  `--execute-imports` that runs each module in a subprocess with a
  temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
  vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.

Acceptance-criteria coverage in tests/unit/extension/:

- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
  rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
  with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
  file is never written); `--execute-imports` DOES execute it and the canary
  appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
  ERROR_CODES and the snapshot table are in lockstep so future additions
  cannot ship without a format branch and a snapshot.

Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).

* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader

tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).

Fixes ModuleNotFoundError seen in CI on the 3.10 job.

* Update src/lfx/tests/unit/extension/test_schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/extension/schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/cli/_extension_commands.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)

Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:

  - load_extension(root): one manifest, one Bundle, registered at @official.
    Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
  - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
    is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
    user-declared path order). First-wins on duplicate names; second emits
    duplicate-inline-bundle warning that names both paths.

Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.

New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.

Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lfx): address LE-1015 review feedback

- Drop the unproduced duplicate-distribution error code; LE-1022 will add
  it once startup-time discovery has a place to surface it.  Restores the
  invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
  absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
  was already existence-checked, and a concurrent removal in the narrow
  window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lfx): split extension loader into a small subpackage (LE-1015)

Addresses the file-size feedback from the LE-1015 review.  The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:

    loader/
      __init__.py     # re-exports the public surface
      _types.py       # SLOT constants, LoadedComponent, LoadResult
      _discovery.py   # filesystem walk + importlib.util orchestration
      _detection.py   # Component subclass identification (MRO heuristic)
      _orchestrator.py # load_extension, discover_inline_bundles
      _plugins.py     # manifest-first precedence over langflow.plugins

Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline.  No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass.  ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).

The test suite is split to mirror the package:

    tests/unit/extension/loader/
      conftest.py             # shared fixtures, FakeDist, autouse scrub
      test_load_extension.py  # @official slot, identity, failure modes
      test_inline_bundles.py  # LANGFLOW_COMPONENTS_PATH, @extra slot
      test_plugins.py         # manifest-first precedence helpers
      test_types.py           # LoadedComponent, LoadResult, code parity

Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: LangflowCompat -> LfxCompat

* fix: Remove references to ticket

* fix: encode maxItems constraint in schema

* fix: deferred fields and schema version

* Fix ruff errors

* fix: align loader tests with renamed manifest API and strip ticket refs

The merge of feat/extension-validate brought in the LfxCompat / compat
rename, but the loader test fixtures still used LangflowCompat / bundle_api
and failed at the manifest layer. Update conftest.py + test_load_extension.py
to the post-rename API.

Strip 17 LE-XXXX ticket references from loader source files, errors.py
loader-specific comments, and the four loader test docstrings, matching the
convention applied to LE-1014. Descriptive prose preserved.

Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the
loader's orchestrator no longer reaches across modules into a private name.

* refactor(lfx): clarify broad except, log malformed bundle.json, expand test docstring

- Document why _discovery.import_bundle_module catches BaseException (startup-time loader, must surface bad bundles as typed errors rather than abort).
- Add debug-level logging when bundle.json is malformed or non-object so a stale-cache footgun is at least observable.
- Expand test_skips_re_imported_class docstring so future maintainers don't accidentally weaken the __module__-equality guard if package-style relative imports get added later.

Also drops the stale duplicate-distribution claim from the PR description; that code is correctly deferred to LE-1022 along with /all integration.

* feat(lfx): wire Extension System into /all, pathsep-split LANGFLOW_COMPONENTS_PATH, emit duplicate-distribution

Closes the four AC gaps the previous reviewer flagged on PR #12967:

1. /all integration: get_and_cache_all_types_dict now also calls a new
   import_extension_components() that loads installed Extensions via
   load_installed_extensions, loads inline bundles via discover_inline_bundles,
   and builds frontend-node templates with extension/bundle/extension_version
   fields stamped on. Failures are logged and skipped per bundle.

2. LANGFLOW_COMPONENTS_PATH is now split on os.pathsep so multi-entry env vars
   (e.g. /a:/b on POSIX) produce multiple components-path entries instead of
   one literal non-existent path. Empty segments and missing paths are skipped.

3. duplicate-distribution is a real producer: load_installed_extensions
   surfaces a typed warning on the winner LoadResult when two distributions
   share a canonical name, naming every involved manifest path.

4. Manifest-first precedence runtime wiring: new filter_component_entry_points
   loads each entry-point and only skips ones that resolve to a Component
   subclass on a manifest-shipping distribution. plugin_routes.load_plugin_routes
   now applies it so non-component entry-points (route registrars) keep
   loading per the AC's 'unaffected' promise. Added the previously-missing
   AC test for same-distribution component+non-component partition.

Also makes _distribution_canonical_name defensive against MagicMock test seams.

* fix(lfx): register extension components under namespaced ID; promote duplicate-distribution to error

Addresses the latest review of PR #12967:

P1: /all integration now keys the cache inner dict by LoadedComponent.namespaced_id
(ext:<bundle>:<Class>@<slot>) rather than the bare class name. Templates also carry
the namespaced_id as an explicit field so consumers that look at the value (not the
key) still see the canonical address. This is the form the LE-1020 migration table
will rewrite legacy class-name references to.

P2: load_installed_extensions now appends duplicate-distribution to result.errors
instead of result.warnings, so LoadResult.ok=False when two distributions share a
canonical name. The winner's components still appear in result.components so flows
already pinned to them keep working; only the conflict status changes. Updated test
to assert errors + ok=False; added explicit assertion that the winner's components
are still present.

* fix(lfx): installed-distribution discovery accepts pyproject.toml manifest form

Closes the latest review finding on PR #12967: the installed-distribution
scan only looked for extension.json, ignoring distributions whose manifest
lives in [tool.langflow.extension] inside pyproject.toml. The AC explicitly
treats both as valid manifest forms.

_distribution_manifest_path now:
- Returns extension.json immediately when present (preserves precedence
  matching load_manifest's discovery order).
- Falls back to pyproject.toml only when extension.json is absent AND the
  pyproject's [tool.langflow.extension] section is parseable.

Validation reuses load_manifest itself so the rule lives in exactly one
place; a stray pyproject.toml without the section is correctly ignored.

Tests cover: pyproject-only discovery, pyproject-without-section ignored,
extension.json wins on collision, end-to-end pyproject load at @official,
and pyproject-form manifest-first entry-point suppression.

* refactor(lfx): tighten loader invariants, surface silent skips, harden tests

Addresses the latest review feedback on PR #12967:

Type-level invariants (_types.py):
- LoadedComponent.__post_init__ enforces that @extra components must NOT
  carry a distribution. The reverse (@official without distribution) is
  permitted because load_extension is also used for dev-mode loads
  against a working tree before pip install.
- LoadResult docstring documents the partial-success contract: components
  may be non-empty when errors is non-empty (some files imported, others
  failed). Callers branching on ok get strict success.

Silent-failure fixes (_orchestrator.py + settings/base.py):
- inline-path-missing: a non-existent / non-dir LANGFLOW_COMPONENTS_PATH
  entry now produces a typed warning per skipped path so a typo no
  longer yields zero diagnostics. Settings-layer skip bumped from debug
  to warning for the same reason.
- bundle-json-invalid: a malformed or non-object bundle.json now surfaces
  a typed warning instead of silently rewriting the user-declared
  id/version to derived values under the same bundle name.
- no-component-subclass gating uses a call-local counter instead of
  result.errors so the diagnostic stays accurate when a future caller
  reuses a LoadResult (multi-bundle / batch wrapper scenarios).

Test hardening:
- test_re_imported_class_is_skipped_via_module_filter rewritten to
  actually exercise the __module__-equality check via sys.modules
  injection; previously passed via module-import-failed (relative-import
  failure), which would silently weaken if package registration changes.
- test_user_declared_path_order_is_preserved: AC #8's multi-path order
  case (distinct bundles in [path_b, path_a]) was unasserted; added.
- test_inline_module_import_failure_attributes_identity: AC #10's
  identity-on-partial-failure was covered for @official but not @extra;
  added.
- test_uses_real_distributions_by_default tightened to assert ep
  placement (in kept, not in skipped) instead of exact-list equality, so
  a future Langflow-shipped manifest doesn't silently flip the assertion.

bumped 64 -> 175 passing extension tests; 20 backend integration tests
still pass.

* fix(lfx): malformed pyproject manifests surface manifest-invalid instead of disappearing

Closes the latest review finding on PR #12967: a pyproject.toml with a
[tool.langflow.extension] section that has missing/invalid required
fields was silently dropped because _pyproject_has_extension_section
ran full schema validation via load_manifest and returned False on
ValueError/TypeError. That conflated 'no section' with 'section
malformed'.

Fix: detect section presence only. _pyproject_has_extension_section now
calls _read_pyproject_extension (TOML parse + key lookup, no schema
check). Behavior:
- Section absent or pyproject TOML unparseable -> False (treat as
  regular non-manifest package).
- Section present and is a table (valid OR schema-invalid) -> True.
- Section present but is not a table -> True; the author intended to
  declare an extension and load_extension will surface the typed error.

This way a typo'd pyproject Extension produces a typed manifest-invalid
LoadResult with extension_id attribution, and manifest-first precedence
still suppresses its legacy component entry-points -- matching the
'typed load results on success/failure' contract for the supported
pyproject manifest form.

Tests: two new cases pin the behavior. test_malformed_pyproject_section_
surfaces_manifest_invalid asserts a typed load-failure result with
distribution attribution; the second test pins manifest-first
suppression for malformed pyproject distributions.

* refactor(lfx): emit inline-path-unreadable, document reload contract, trim rot

Closes the remaining nits on PR #12967:

- inline-path-unreadable (new typed error code): a configured
  LANGFLOW_COMPONENTS_PATH entry that raises OSError on iterdir
  (typically permission-denied) now produces a typed LoadResult error
  carrying str(exc) instead of silently swallowing the message.
- duplicate-inline-bundle hint trimmed: removed forward promise about
  hard-error-in-a-later-release; same actionability, no expiration.
- discover_inline_bundles docstring trimmed from three paragraphs to
  two sentences (per CLAUDE.md anti-multi-paragraph rule).
- _distribution_manifest_path docstring trimmed to one-liner.
- installed_extension_roots dropped Used-by caller-narration list;
  manifest_owning_distributions docstring rewritten to call out the
  shadow-load risk for direct callers and point to load_installed_
  extensions for the typed warning surface.
- _discovery.import_bundle_module: replaced 'until that lands in a
  later milestone' rot with a single line ('Absolute imports only
  between bundle modules; relative imports unsupported.') AND added
  the single-load-per-process contract note documenting why LE-1018
  reload must scrub registry/sys.modules before re-invoking the loader.
- load_extension docstring grew a 'Single-load-per-process contract'
  block telling direct callers not to rely on this function for refresh.

Tests: new test_unreadable_path_emits_inline_path_unreadable pins the
OSError -> typed-error path.

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) (#12979)

* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)

Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:

  - load_extension(root): one manifest, one Bundle, registered at @official.
    Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
  - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
    is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
    user-declared path order). First-wins on duplicate names; second emits
    duplicate-inline-bundle warning that names both paths.

Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.

New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.

Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lfx): address LE-1015 review feedback

- Drop the unproduced duplicate-distribution error code; LE-1022 will add
  it once startup-time discovery has a place to surface it.  Restores the
  invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
  absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
  was already existence-checked, and a concurrent removal in the narrow
  window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lfx): split extension loader into a small subpackage (LE-1015)

Addresses the file-size feedback from the LE-1015 review.  The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:

    loader/
      __init__.py     # re-exports the public surface
      _types.py       # SLOT constants, LoadedComponent, LoadResult
      _discovery.py   # filesystem walk + importlib.util orchestration
      _detection.py   # Component subclass identification (MRO heuristic)
      _orchestrator.py # load_extension, discover_inline_bundles
      _plugins.py     # manifest-first precedence over langflow.plugins

Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline.  No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass.  ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).

The test suite is split to mirror the package:

    tests/unit/extension/loader/
      conftest.py             # shared fixtures, FakeDist, autouse scrub
      test_load_extension.py  # @official slot, identity, failure modes
      test_inline_bundles.py  # LANGFLOW_COMPONENTS_PATH, @extra slot
      test_plugins.py         # manifest-first precedence helpers
      test_types.py           # LoadedComponent, LoadResult, code parity

Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018)

Five-stage reload (parallel staging load -> validate -> swap under write
lock -> cleanup -> emit) for installed Bundles in Mode A.  In-flight
flows keep the pre-swap class via existing references; new flows pick up
the post-swap class atomically; concurrent reloads on the same Bundle
are rejected with reload-in-progress.

Adds:

* lfx/extension/registry.py -- BundleRegistry with per-bundle
  reload-in-progress guard and components_index.json writer
* lfx/extension/reload.py -- the five-stage pipeline; events emission
  is stubbed (TODO LE-1017) so the swap mechanics can ship before the
  events service lands
* loader: optional module_namespace param so Stage 1 lands in
  __reload_staging__.<id> instead of the live _lfx_ext.* namespace
* errors: four new typed reload codes (reload-in-progress,
  reload-bundle-not-installed, reload-bundle-name-mismatch,
  reload-source-missing) with branch templates and snapshot tests
* HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by
  the existing get_current_active_user dependency, returns 409 with a
  typed body for the in-progress collision case
* CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client
  against the dev server with text/json output and proper exit codes
  (--all is gated until LE-1019 lands the list endpoint)
* tests: 16 reload-pipeline tests covering the AC matrix (rename
  round-trip, broken-bundle isolation, concurrent readers, in-flight
  flow, double-reload guard, bundle-name mismatch) plus 9 CLI client
  tests

Mode A only.  In Mode B/C bundle changes require a Docker image rebuild
and the reload path is not exercised.

The events emission in Stage 5 is intentionally stubbed -- the LE-1017
ticket will swap the body of _emit_bundle_reload_event in one place
without touching the pipeline core.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [autofix.ci] apply automated fixes

* feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) (#12968)

* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)

Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.

What's in lfx.extension:

- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
  `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
  `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
  None-only so non-null values produce a dedicated
  `field-deferred-in-this-milestone` error instead of a generic schema wall.
  `bundles` accepts a list but rejects length > 1 with
  `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
  re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
  artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
  per discriminant. Codes are registered in `ERROR_CODES`; an
  `ExtensionError` constructed with an unknown code raises at construction
  time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
  path-safety (no `..`, no absolute paths, no symlink escape), AST
  inspection of every `.py` (syntax, Component subclass present, build()
  declared, top-level `import *`, top-level I/O primitives), and an opt-in
  `--execute-imports` that runs each module in a subprocess with a
  temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
  vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.

Acceptance-criteria coverage in tests/unit/extension/:

- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
  rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
  with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
  file is never written); `--execute-imports` DOES execute it and the canary
  appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
  ERROR_CODES and the snapshot table are in lockstep so future additions
  cannot ship without a format branch and a snapshot.

Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).

* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader

tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).

Fixes ModuleNotFoundError seen in CI on the 3.10 job.

* Update src/lfx/tests/unit/extension/test_schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/extension/schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/cli/_extension_commands.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)

Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:

  - load_extension(root): one manifest, one Bundle, registered at @official.
    Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
  - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
    is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
    user-declared path order). First-wins on duplicate names; second emits
    duplicate-inline-bundle warning that names both paths.

Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.

New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.

Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lfx): address LE-1015 review feedback

- Drop the unproduced duplicate-distribution error code; LE-1022 will add
  it once startup-time discovery has a place to surface it.  Restores the
  invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
  absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
  was already existence-checked, and a concurrent removal in the narrow
  window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lfx): split extension loader into a small subpackage (LE-1015)

Addresses the file-size feedback from the LE-1015 review.  The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:

    loader/
      __init__.py     # re-exports the public surface
      _types.py       # SLOT constants, LoadedComponent, LoadResult
      _discovery.py   # filesystem walk + importlib.util orchestration
      _detection.py   # Component subclass identification (MRO heuristic)
      _orchestrator.py # load_extension, discover_inline_bundles
      _plugins.py     # manifest-first precedence over langflow.plugins

Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline.  No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass.  ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).

The test suite is split to mirror the package:

    tests/unit/extension/loader/
      conftest.py             # shared fixtures, FakeDist, autouse scrub
      test_load_extension.py  # @official slot, identity, failure modes
      test_inline_bundles.py  # LANGFLOW_COMPONENTS_PATH, @extra slot
      test_plugins.py         # manifest-first precedence helpers
      test_types.py           # LoadedComponent, LoadResult, code parity

Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016)

The two scaffolding CLIs an Extension author types:

  - `lfx extension init <target>` writes the basic single-Bundle
    template (manifest with $schema, README, .gitignore, one Component
    subclass + a pytest smoke test).  AC #1: the generated extension
    validates clean against LE-1014.  AC #2: the generated test file is
    a valid pytest module that exercises the component's build() method.
    AC #3: any --template other than 'basic' fails with a typed
    template-deferred-in-this-milestone error and a non-zero exit.
    Refuses to scaffold over a non-empty target dir.

  - `lfx extension dev <target>` validates the local extension, records
    its absolute path in <config_dir>/extensions/dev_extensions.json,
    prints reload instructions, and execs `langflow run` (or
    `python -m langflow` when langflow isn't on PATH).  --skip-launch
    registers without launching (used by tests + external dev-server
    scripts); --skip-validate lets authors register a known-broken
    manifest to debug it under the loader.

Stack:
  - Wave 0: error codes (extension-target-exists,
    extension-target-invalid, local-extension-missing) with format
    branches and snapshot tests.
  - Wave 1: lfx.extension.init_template -- pure-data scaffolder, no
    Typer dependency so the CLI is a thin shell over it.
  - Wave 1: lfx.extension.dev_registry -- atomic JSON state file under
    the langflow user-cache dir; helpers for register / list /
    unregister / load_dev_extensions / dev_extension_component_paths.
  - Wave 2: lfx.cli._extension_commands gains init/dev subcommands.
  - Wave 2: langflow.main lifespan hook reads the dev registry after
    bundle loading and extends components_path with each registered
    bundle dir, so the existing palette discovery picks up dev
    extensions.  Missing paths surface as local-extension-missing
    warnings (AC #5) without aborting startup.

Tests (84 new, 197 total in tests/unit/extension/):
  - test_init_template.py: AC scenarios + identifier derivation +
    deterministic file shape.
  - test_dev_registry.py: register/list/unregister round-trip,
    idempotent re-register refreshes timestamp, malformed state file
    treated as empty, missing-path warning, recovery when path
    reappears, env-var override precedence.
  - test_cli.py: AC #1 init->validate, AC #3 deferred templates,
    --skip-launch registers without launching, --skip-validate
    short-circuits the pre-flight pass.
  - test_errors.py: snapshot rows for all three new codes; the
    every-known-code-has-a-snapshot test enforces parity.

LE-1018 (reload) reuses load_dev_extensions; LE-1022 (installed-pkg
discovery) shares the components_path extension pattern.  AC #4 ("boots
Langflow with the extension visible in the palette within 5s") is
delivered jointly by `extension dev` (registers + execs) and the
lifespan hook (loads on startup).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lfx): address LE-1016 review feedback

Fixes the four HIGH issues + the elevated LOW from the review pass:

1. dev_extension_component_paths now forwards EVERY warning, not just
   local-extension-missing.  Previously a duplicate-component-name (or
   any future warning code) was silently dropped, hiding real signal
   from the lifespan hook's logs.

2. Defensive emit when a LoadResult has components but source_path is
   None.  The current loader always sets source_path, but a future
   hand-built LoadResult could violate that contract; we now surface a
   typed local-extension-missing error rather than dropping the
   extension silently.

3. Replaced the fragile ``min(len(parts))`` bundle-root selection with
   a relative-to-source-path measurement.  Handles deep-vs-shallow
   sibling extensions correctly without depending on absolute path
   depth.

4. Generated README now documents the langflow/lfx prerequisite under
   the Develop section so authors know `pytest` requires the lfx
   environment, not just Python.

5. Forced LANGFLOW_LAZY_LOAD_COMPONENTS=false unconditionally in the
   `extension dev` exec env (was setdefault, which let a developer's
   global lazy-loading export silently hide their dev components from
   the palette and miss AC #4's 5s budget).

Plus three MEDIUMs:

  - sys.modules cleanup in test_generated_test_file_runs_against_generated_component
    so a later test importing the same dotted path doesn't pick up a
    stale module from a deleted tmp_path.  Component instantiation
    moved inside the try block because Component.__init__ uses
    inspect.getsourcefile against self.__class__'s still-live module.
  - Added regex-drift test that pins the init_template patterns to
    match manifest.py's so a schema regex change can't quietly produce
    invalid scaffolded manifests.
  - Added two new dev_registry tests covering the forward-all-warnings
    contract and the source_path=None defense.

Tests: 201 passing (4 new); ruff check + format clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: LangflowCompat -> LfxCompat

* fix: Remove references to ticket

* fix: encode maxItems constraint in schema

* fix: deferred fields and schema version

* Fix ruff errors

* fix: align loader tests with renamed manifest API and strip ticket refs

The merge of feat/extension-validate brought in the LfxCompat / compat
rename, but the loader test fixtures still used LangflowCompat / bundle_api
and failed at the manifest layer. Update conftest.py + test_load_extension.py
to the post-rename API.

Strip 17 LE-XXXX ticket references from loader source files, errors.py
loader-specific comments, and the four loader test docstrings, matching the
convention applied to LE-1014. Descriptive prose preserved.

Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the
loader's orchestrator no longer reaches across modules into a private name.

* fix(lfx): align init template with renamed manifest API

After merging feat/extension-loader into this branch, two surfaces
fell out of sync with the renamed manifest schema:

- init_template generates extension.json with `lfx: {bundle_api: [1]}`,
  but the validator now requires `lfx: {compat: ["1"]}` per LfxCompat.
  This made `test_basic_template_validates_clean` fail with
  manifest-invalid (`lfx.compat: Field required; lfx.bundle_api: Extra
  inputs are not permitted`).
- test_init_template_regexes_match_manifest_schema reads
  `manifest_mod._BUNDLE_NAME_RE`, but that symbol was promoted to public
  `BUNDLE_NAME_RE` in c63f84a591. Update the test to reference the
  public name; init_template's local copy is still private since it
  also covers `_EXTENSION_ID_RE`, which remains private upstream.

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(lfx): append-only migration table + flow deserializer rewrite hook (#13024)

* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)

Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.

What's in lfx.extension:

- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
  `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
  `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
  None-only so non-null values produce a dedicated
  `field-deferred-in-this-milestone` error instead of a generic schema wall.
  `bundles` accepts a list but rejects length > 1 with
  `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
  re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
  artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
  per discriminant. Codes are registered in `ERROR_CODES`; an
  `ExtensionError` constructed with an unknown code raises at construction
  time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
  path-safety (no `..`, no absolute paths, no symlink escape), AST
  inspection of every `.py` (syntax, Component subclass present, build()
  declared, top-level `import *`, top-level I/O primitives), and an opt-in
  `--execute-imports` that runs each module in a subprocess with a
  temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
  vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.

Acceptance-criteria coverage in tests/unit/extension/:

- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
  rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
  with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
  file is never written); `--execute-imports` DOES execute it and the canary
  appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
  ERROR_CODES and the snapshot table are in lockstep so future additions
  cannot ship without a format branch and a snapshot.

Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).

* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader

tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).

Fixes ModuleNotFoundError seen in CI on the 3.10 job.

* Update src/lfx/tests/unit/extension/test_schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/extension/schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/cli/_extension_commands.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)

Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:

  - load_extension(root): one manifest, one Bundle, registered at @official.
    Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
  - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
    is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
    user-declared path order). First-wins on duplicate names; second emits
    duplicate-inline-bundle warning that names both paths.

Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.

New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.

Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lfx): address LE-1015 review feedback

- Drop the unproduced duplicate-distribution error code; LE-1022 will add
  it once startup-time discovery has a place to surface it.  Restores the
  invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
  absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
  was already existence-checked, and a concurrent removal in the narrow
  window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lfx): split extension loader into a small subpackage (LE-1015)

Addresses the file-size feedback from the LE-1015 review.  The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:

    loader/
      __init__.py     # re-exports the public surface
      _types.py       # SLOT constants, LoadedComponent, LoadResult
      _discovery.py   # filesystem walk + importlib.util orchestration
      _detection.py   # Component subclass identification (MRO heuristic)
      _orchestrator.py # load_extension, discover_inline_bundles
      _plugins.py     # manifest-first precedence over langflow.plugins

Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline.  No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass.  ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).

The test suite is split to mirror the package:

    tests/unit/extension/loader/
      conftest.py             # shared fixtures, FakeDist, autouse scrub
      test_load_extension.py  # @official slot, identity, failure modes
      test_inline_bundles.py  # LANGFLOW_COMPONENTS_PATH, @extra slot
      test_plugins.py         # manifest-first precedence helpers
      test_types.py           # LoadedComponent, LoadResult, code parity

Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: LangflowCompat -> LfxCompat

* fix: Remove references to ticket

* fix: encode maxItems constraint in schema

* fix: deferred fields and schema version

* Fix ruff errors

* fix: align loader tests with renamed manifest API and strip ticket refs

The merge of feat/extension-validate brought in the LfxCompat / compat
rename, but the loader test fixtures still used LangflowCompat / bundle_api
and failed at the manifest layer. Update conftest.py + test_load_extension.py
to the post-rename API.

Strip 17 LE-XXXX ticket references from loader source files, errors.py
loader-specific comments, and the four loader test docstrings, matching the
convention applied to LE-1014. Descriptive prose preserved.

Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the
loader's orchestrator no longer reaches across modules into a private name.

* feat(lfx): append-only migration table + flow deserializer rewrite hook

Adds the migration layer of the Extension System: an append-only JSON table
that maps three legacy component reference shapes (bare class name, old
import path, pre-Phase-A namespaced slot) to the post-Phase-A canonical
ext:<bundle>:<Class>@<slot> identifier, plus a deserializer hook that
rewrites a saved-flow payload in place against that table on load.

What landed:

  * lfx.extension.migration.schema -- Pydantic models for MigrationEntry +
    MigrationTable with per-entry validators (exactly one of bare/import/slot
    populated; canonical target shape) and table-level uniqueness check.
  * lfx.extension.migration.loader -- canonical in-repo path, threadsafe
    process-lifetime cache, typed errors on every failure mode.
  * lfx.extension.migration.rewrite -- node-by-node rewrite, idempotent on
    canonical refs, difflib-backed closest-match suggestion for unmapped
    references, cross-bucket ambiguity surfaces component-name-ambiguous
    instead of silently loading into the wrong bundle.
  * Wired into Graph.from_payload before validate_flow_for_current_settings
    so every saved-flow load goes through migration first.
  * scripts/migrate/check_migration_append_only.py -- CI guard that diffs
    the working-tree table against origin/main and rejects removals or
    target mutations; reordering and additions are allowed.
  * 34 new unit tests covering rewrite paths, loader failure modes, schema
    invariants, and the CI script behavior.

What is deliberately deferred:

  * flow-migrated event emission. The events pipeline is unavailable in this
    iteration; the wiring point in Graph.from_payload is marked TODO and the
    MigrationReport already carries every field a future emitter needs.

The shipped migration_table.json starts empty; entries land alongside the
pilot bundle extraction in a follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(frontend): palette Bundle reload action + loading + toasts (#13025)

* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)

Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.

What's in lfx.extension:

- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
  `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
  `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
  None-only so non-null values produce a dedicated
  `field-deferred-in-this-milestone` error instead of a generic schema wall.
  `bundles` accepts a list but rejects length > 1 with
  `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
  re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
  artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
  per discriminant. Codes are registered in `ERROR_CODES`; an
  `ExtensionError` constructed with an unknown code raises at construction
  time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
  path-safety (no `..`, no absolute paths, no symlink escape), AST
  inspection of every `.py` (syntax, Component subclass present, build()
  declared, top-level `import *`, top-level I/O primitives), and an opt-in
  `--execute-imports` that runs each module in a subprocess with a
  temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
  vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.

Acceptance-criteria coverage in tests/unit/extension/:

- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
  rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
  with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
  file is never written); `--execute-imports` DOES execute it and the canary
  appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
  ERROR_CODES and the snapshot table are in lockstep so future additions
  cannot ship without a format branch and a snapshot.

Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).

* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader

tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).

Fixes ModuleNotFoundError seen in CI on the 3.10 job.

* Update src/lfx/tests/unit/extension/test_schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/extension/schema.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/cli/_extension_commands.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)

Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:

  - load_extension(root): one manifest, one Bundle, registered at @official.
    Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
  - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
    is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
    user-declared path order). First-wins on duplicate names; second emits
    duplicate-inline-bundle warning that names both paths.

Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.

New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.

Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(lfx): address LE-1015 review feedback

- Drop the unproduced duplicate-distribution error code; LE-1022 will add
  it once startup-time discovery has a place to surface it.  Restores the
  invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
  absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
  was already existence-checked, and a concurrent removal in the narrow
  window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lfx): split extension loader into a small subpackage (LE-1015)

Addresses the file-size feedback from the LE-1015 review.  The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:

    loader/
      __init__.py     # re-exports the public surface
      _types.py       # SLOT constants, LoadedComponent, LoadResult
      _discovery.py   # filesystem walk + importlib.util orchestration
      _detection.py   # Component subclass identification (MRO heuristic)
      _orchestrator.py # load_extension, discover_inline_bundles
      _plugins.py     # manifest-first precedence over langflow.plugins

Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline.  No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass.  ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).

The test suite is split to mirror the package:

    tests/unit/extension/loader/
      conftest.py             # shared fixtures, FakeDist, autouse scrub
      test_load_extension.py  # @official slot, identity, failure modes
      test_inline_bundles.py  # LANGFLOW_COMPONENTS_PATH, @extra slot
      test_plugins.py         # manifest-first precedence helpers
      test_types.py           # LoadedComponent, LoadResult, code parity

Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018)

Five-stage reload (parallel staging load -> validate -> swap under write
lock -> cleanup -> emit) for installed Bundles in Mode A.  In-flight
flows keep the pre-swap class via existing references; new flows pick up
the post-swap class atomically; concurrent reloads on the same Bundle
are rejected with reload-in-progress.

Adds:

* lfx/extension/registry.py -- BundleRegistry with per-bundle
  reload-in-progress guard and components_index.json writer
* lfx/extension/reload.py -- the five-stage pipeline; events emission
  is stubbed (TODO LE-1017) so the swap mechanics can ship before the
  events service lands
* loader: optional module_namespace param so Stage 1 lands in
  __reload_staging__.<id> instead of the live _lfx_ext.* namespace
* errors: four new typed reload codes (reload-in-progress,
  reload-bundle-not-installed, reload-bundle-name-mismatch,
  reload-source-missing) with branch templates and snapshot tests
* HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by
  the existing get_current_active_user dependency, returns 409 with a
  typed body for the in-progress collision case
* CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client
  against the dev server with text/json output and proper exit codes
  (--all is gated until LE-1019 lands the list endpoint)
* tests: 16 reload-pipeline tests covering the AC matrix (rename
  round-trip, broken-bundle isolation, concurrent readers, in-flight
  flow, double-reload guard, bundle-name mismatch) plus 9 CLI client
  tests

Mode A only.  In Mode B/C bundle changes require a Docker image rebuild
and the reload path is not exercised.

The events emission in Stage 5 is intentionally stubbed -- the LE-1017
ticket will swap the body of _emit_bundle_reload_event in one place
without touching the pipeline core.

* feat(frontend): palette Bundle reload action + loading + toasts

Adds the frontend half of the bundle reload flow.  When the Bundle
header is right-clicked or its overflow ("⋮") icon clicked, a Reload
action fires POST /api/v1/extensions/{id}/bundles/{name}/reload and
surfaces the result via the existing alert-store toast system.

What landed (frontend only):

  * src/controllers/API/queries/extensions/ -- typed wire-format models
    (ReloadBundleResponse, ExtensionErrorPayload, ReloadInProgressDetail)
    and the useReloadBundle mutation hook.  The hook unwraps the 409
    `reload-in-progress` detail into a stable, parseable Error message so
    the UI can branch without reading status codes.
  * components/bundleHeaderActions.tsx -- new Select-based overflow menu
    next to the Bundle header chevron.  Three toast paths: success
    (green, with components +/- delta), structural failure (red, with
    typed errors and inline hints), reload-in-progress (notice).
    Loading state swaps the kebab icon for a spinning Loader2 while
    the request is in flight.  Renders nothing when no extension_id is
    on the bundle, so the static SIDEBAR_BUNDLES list is unaffected.
  * components/bundleItems.tsx -- wires the new actions in next to the
    chevron, plus a context-menu (right-click) capture that opens the
    same overflow trigger so keyboard / mouse / right-click all share
    one source of truth.
  * types/index.ts -- BundleItemProps.item gains an optional
    extension_id; took the opportunity to extract the SidebarBundle
    interface and tighten three pre-existing `any` types.
  * customization/feature-flags.ts -- ENABLE_EXTENSION_RELOAD gate, off
    by default until the bundle-list endpoint that populates extension_id
    per bundle ships.
  * controllers/API/helpers/constants.ts -- EXTENSIONS URL constant.
  * Tests: 7 component tests + 3 mutation-hook tests, all green.  Total
    sidebar + extensions test count: 479 passing.

Deferred:

  * Event-pipeline subscription is left as an inline TODO.  The mutation
    response carries enough information to drive the toasts on its own
    today; once the events service lands the toast wiring will move to
    a `bundle_reloaded` / `bundle_reload_failed` listener so multi-tab
    and multi-worker swaps surface exactly once.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: End to end bundle installation

* fix: Review comments addressed

* Update component_index.json

* Update component_index.json

* Update router.py

* fix(docker): copy src/bundles before uv sync so workspace bundles resolve

Each directory under ``src/bundles`` is a uv workspace member referenced by
``langflow-base`` (and the root project) as a path dependency.  The Docker
builders ran ``uv sync --no-install-project`` after copying only the
top-level pyproject.toml files, so resolution failed with
``Distribution not found at: file:///app/src/bundles/<name>``.

Copying the whole ``src/bundles`` tree (rather than enumerating each
bundle) means a new bundle dropped under that dir does not require a
Dockerfile edit.  The full ``./src`` copy a few lines later produces the
same final layer either way; this earlier copy just unblocks the
dependency-resolution sync.

Touched all builders that run a workspace-resolving uv sync:

  * docker/build_and_push.Dockerfile
  * docker/build_and_push_base.Dockerfile
  * docker/build_and_push_ep.Dockerfile
  * docker/build_and_push_with_extras.Dockerfile
  * docker/dev.Dockerfile (bind-mount instead of COPY)

* Revert "fix(docker): copy src/bundles before uv sync so workspace bundles resolve"

This reverts commit 5aa008a3cd.

* feat: DuckDuckGo as Extension in new Bundle System (#13044)

* feat: DuckDuckGo Extension for bundles

* fix ruff errors

* Update test_pilot_duckduckgo_upgrade.py

* test(lfx): handle non-empty canonical migration table + editable installs

- test_path_override_bypasses_cache: mirror the cached canonical table
  into the temp file instead of asserting it's empty.  Drift in
  migration_table.json (the duckduckgo entries shipped with the B1
  pilot) no longer breaks the cache-bypass invariant.

- test_lfx_duckduckgo_ships_manifest: editable installs (pip install -e)
  surface only dist-info entries in dist.files, so we cannot use that
  path to find extension.json in the workspace venv.  Detect editable
  mode via direct_url.json's PEP 660 marker and fall through to walking
  the source tree; non-editable wheel installs still exercise the
  dist.files path the loader uses at runtime.

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* fix(docker): copy src/bundles before uv sync so workspace bundles resolve

Each directory under ``src/bundles`` is a uv workspace member referenced by
``langflow-base`` (and the root project) as a path dependency.  The Docker
builders ran ``uv sync --no-install-project`` after copying only the
top-level pyproject.toml files, so resolution failed with
``Distribution not found at: file:///app/src/bundles/<name>``.

Copying the whole ``src/bundles`` tree (rather than enumerating each
bundle) means a new bundle dropped under that dir does not require a
Dockerfile edit.  The full ``./src`` copy a few lines later produces the
same final layer either way; this earlier copy just unblocks the
dependency-resolution sync.

Touched all builders that run a workspace-resolving uv sync:

  * docker/build_and_push.Dockerfile
  * docker/build_and_push_base.Dockerfile
  * docker/build_and_push_ep.Dockerfile
  * docker/build_and_push_with_extras.Dockerfile
  * docker/dev.Dockerfile (bind-mount instead of COPY)

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* fix: Review sweep

* fix: More review comments addressed

* Ruff check

* docs: add bundle porting guide

Step-by-step recipe for extracting a provider package from the in-tree
``src/lfx/src/lfx/components/<provider>/`` directory into a standalone
Extension Bundle distribution under ``src/bundles/<provider>/``.

The DuckDuckGo bundle is the reference; every section maps to a single
copy-pasteable change and a verification command.

Doc surfaces a forthcoming ``scripts/migrate/port_bundle.py`` automation
helper for the mechanical bits; that script lands on the next branch
together with the second-pilot port that validates the recipe end-to-end.

* fix(extension): accept Output(method=...) in build-method validator

The validator's static AST check looked for a literal ``build`` method
on every Component subclass and emitted ``build-method-missing`` when
absent.  Real Langflow components -- including the production
``DuckDuckGoSearchComponent`` and the in-tree ``ArXivComponent`` -- do
not declare a literal ``build``; they declare entry-points
declaratively via ``outputs = [Output(name=..., method=<name>)]`` and
the named method is what gets called.  Result: every clean port hit a
spurious ``build-method-missing`` error from ``lfx extension validate``.

Fix: extend ``_has_build_method`` to also accept any class that names a
method via ``Output(method="X")`` AND defines that method in the class
body.  The negative case ("Output(method=...) without a matching def")
still fires -- a typo'd method name would crash at runtime, so the
static check should keep flagging it.

Two new tests in ``test_validate.py`` lock the contract: positive case
mirroring duckduckgo / arxiv, negative case for typo'd method names.
All 26 validator tests pass.

PORTING.md updates fall out of running the recipe live against the
duckduckgo bundle on this branch:

  * Validate path is ``src/bundles/<bundle>/src/lfx_<bundle>``, not the
    bundle root (the manifest lives next to ``__init__.py``).
  * Index regen needs ``LFX_DEV=1`` to skip the prebuilt-index fast path
    and uses ``scripts/build_component_index.py``.
  * Drop the migration-table JSON from the ruff invocation (ruff treats
    it as Python and complains about the top-level expression).

* fix: Delete outdated components

* fix(palette): wire reload kebab via DropdownMenu, not Select

The reload kebab on the palette Bundle header was using Radix ``Select``
to back its overflow menu.  ``Select`` is for picking a value, not for
firing an action: ``onValueChange`` is gated by value-equality (so a
re-click of the only item is a no-op), and the popover-portal click
semantics interact poorly with the parent disclosure-trigger button.
The combined effect: clicking ⋮ → Reload opened the popover and closed
it, but never fired the network request, with no console error to point
at.

Switch to ``DropdownMenu`` (purpose-built action menu).
``DropdownMenuItem`` exposes ``onSelect`` which fires on every
activation -- the same callback path keyboard navigation uses -- so
clicking Reload reliably invokes the mutation.

Test mocks updated to drive the DropdownMenu primitives instead of
Select; all 8 bundleHeaderActions tests + the broader 437-test sidebar
suite still pass.

* fix(extension): route ddgs through the lockfile + drop bogus list endpoint hint

Two reviewer findings, both about reproducibility / correctness of
operator-facing surfaces.

[P2] ``docker/build_and_push_base.Dockerfile`` previously installed
``ddgs`` via an unpinned ``uv pip install ddgs`` after the workspace
sync.  That made the base image non-reproducible: a future ``ddgs``
release would silently drift from the tested lock state on every
rebuild.

Fix: route ddgs through the locked sync.

  * Restore the ``duckduckgo`` extra in ``src/backend/base/pyproject.toml``
    (``ddgs>=9.0.0``) as an internal "image-build sidecar".  Public
    consumers still install ``lfx-duckduckgo`` directly; the extra
    exists only to keep ``ddgs`` resolved in the lockfile.
  * The Dockerfile now passes ``--extra duckduckgo`` to ``uv sync
    --frozen``, picking up the locked ``ddgs==9.14.1``.
  * The follow-on bundle install keeps ``--no-deps`` so it does not
    duplicate the now-locked ``ddgs`` / ``lfx`` / ``langchain-community``.

[P3] ``langflow/api/v1/extensions.py`` returned a fix hint that pointed
operators at ``GET /api/v1/extensions`` -- a route that does not exist
in this PR (it lands with the LE-1019 list endpoint).  Replaced with a
reference to ``lfx extension list``, which is shipped here.

* fix(compat): bridge langflow.components.* dynamically via meta path finder

Commit 45552cd1df ("fix: Delete outdated components") removed the
physical shim files under ``src/backend/base/langflow/components/``
that forwarded saved-flow imports like
``from langflow.components.processing.converter import convert_to_dataframe``
or ``import langflow.components.knowledge_bases.retrieval`` to their
new ``lfx.components.*`` homes.  Without those shims, dotted imports
into ``langflow.components.<sub>.<leaf>`` failed at flow-load time --
the existing ``LangflowCompatibilityModule`` registers ``langflow.components``
itself in ``sys.modules`` but does not bridge submodules, so Python's
import machinery falls through to the now-empty langflow.components
directory and raises ``ModuleNotFoundError``.

Replace the deleted physical-shim stack with a single ``MetaPathFinder``
in ``langflow/__init__.py`` that dynamically resolves every
``langflow.components.<rest>`` import to ``lfx.components.<rest>`` and
registers the loaded lfx module in ``sys.modules`` under both names.
The langflow- and lfx-prefixed imports share a single underlying module
object, so class identity is preserved across the bridge -- ``isinstance``
checks against types resolved through either path keep working.

The finder also carries a small first-segment override map for the few
subpackages whose name diverged during the move; the only entry today is
``knowledge_bases`` -> ``files_and_knowledge``, matching the deleted
shim's intent.

Why a meta finder instead of restoring the physical shim files (option
1 from the scope analysis): the meta finder scales without per-bundle
maintenance.  Every future bundle extraction landed under ``lfx.components``
becomes reachable via the legacy ``langflow.components.<bundle>`` path
immediately; nobody has to remember to add a parallel langflow shim.

Six new unit tests in ``test_langflow_components_compat_shim.py`` lock
the contract: dotted submodule resolution, helpers re-export, the
``knowledge_bases`` override, class identity preservation, top-level
aliasing, and the "arbitrary extracted bundle" case that prevents
regression for future ports.

Verified pre-existing tests:
  * 17 dynamic-import integration tests pass.
  * The reload-route-guard suite still passes.
  * The Research Translation Loop starter-project test (which loads
    ArXivComponent) passes.

* Update index.tsx

* fix: Review pass on delivery

* fix: Second review sweep, bare names

* fix: Third review sweep

* Resolve dotted imports and attribute chains

Record import shape and flatten attribute chains so router references can be resolved across dotted imports and re-exports. Added ImportTarget dataclass, _attribute_chain helper, and switched IncludeCall/DecoratorRef to store attribute tuples. parse_file now records ImportTarget entries for imports and captures dotted parent/child chains for include_router and decorators. Replaced _resolve_var with _resolve_chain which handles "from" vs "module" imports, various import shapes (including import x.y and aliased imports), and prevents infinite recursion on re-export cycles.

* One more sweep

* Add seed-directory extension loading and docs

Introduce filesystem "seed directory" extension support and authoring docs. Adds three documentation pages (quickstart, manifest reference, author guide) and wires them into the docs sidebar. Implement load_seed_extensions to discover/load bundles from $LANGFLOW_SEED_DIR (default /opt/langflow/bundles), export it from loader/__init__ and extension package, and integrate it into the components import pipeline. Handle installed-vs-seed shadowing by preferring installed distributions and appending a typed seed-bundle-shadowed ExtensionError; add the corresponding error code and message. Update schema doc link and add unit/integration tests to cover seed loading, determinism, shadowing behavior, and migration-target resolution.

* Fix claims

* Fix docasaurus build

* Clean up the manual checklist

* Update test_pilot_duckduckgo_upgrade.py

* fix: Some wording issues with dogfooding

* fix: Comments addressed

* Update port_bundle.py

* Update component_index.json

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* fix: pytest testpaths glob and accurate scaffold output in port_bundle

testpaths now uses ``src/bundles/*/tests`` so future bundle tests are
picked up automatically without hardcoding each bundle at the root.
pytest expands the glob via ``glob.iglob(..., recursive=True)``.

Rewrote port_bundle.py's ``_render_migration_entries`` and
``_render_test_scaffold`` to match the actual conventions:

* Migration entries now use ``bare_class_name``/``import_path``/
  ``legacy_slot`` + ``target`` + ``added_in`` (the keys
  ``lfx.extension.migration.loader`` actually reads) instead of the
  invented ``from``/``to``/``release`` shape.
* Test scaffold now imports ``load_migration_table`` and
  ``migrate_flow_payload`` -- the APIs the real tests use -- instead of
  a fictitious ``resolve_legacy_id``.  Includes the ``migration_table``
  fixture, ``_saved_flow``/``_saved_flow_node`` helpers, and the
  distribution-importable + manifest-shipped checks that mirror
  ``test_pilot_duckduckgo_upgrade.py``.

Verified by rendering the scaffold for duckduckgo with synthetic plan
data, dropping it into ``src/lfx/tests/integration/extension/``, and
running pytest -- ``3 passed, 2 skipped`` (skips are the wheel-only
checks, by design when run inside the lfx isolation venv).

* [autofix.ci] apply automated fixes

* fix: Review comments

* Update test_discovery.py

* feat: Port Arxiv to the Extension Framework (#13047)

* feat(bundles): port arxiv as the second-pilot Bundle + porting helper

Validates ``src/bundles/PORTING.md`` end-to-end by following its recipe
against a clean candidate (``ArXivComponent``: no third-party runtime
deps, no langflow-base extra, no deactivated duplicate).  Touchpoints
exercised:

  * Bundle skeleton at ``src/bundles/arxiv/`` mirroring duckduckgo.
  * In-tree provider directory removed from ``src/lfx/src/lfx/components/``
    along with its three references in ``components/__init__.py``.
  * Workspace wiring: dep, ``[tool.uv.sources]``, ``[tool.uv.workspace]``
    members, lockfile.
  * Migration table: bare-name + two import-path forms + legacy_slot
    entry.
  * Component index regenerated via ``LFX_DEV=1`` (forces dynamic
    discovery; without it the script reproduces stale entries).
  * Integration test ``test_pilot_arxiv_upgrade.py`` mirroring the
    duckduckgo pilot suite; 5 tests pass against the workspace install.

PORTING.md updates fall out of running the recipe live:
  * Validate path is ``src/bundles/<bundle>/src/lfx_<bundle>``, not the
    bundle root (the manifest lives next to ``__init__.py``).
  * Index regen needs ``LFX_DEV=1`` to skip the prebuilt-index fast path.
  * Drop the migration-table JSON from the ruff invocation (ruff treats
    it as Python and complains about the top-level expression).

``scripts/migrate/port_bundle.py`` is the mechanical helper referenced
from § Automation: stdlib-only, dry-run by default, refuses on invalid
input, and intentionally leaves migration-table edits + integration-test
authoring to a human (release version + bare-name uniqueness require
judgement).  Three guard rails verified: invalid bundle name,
existing-target-bundle, missing in-tree provider.

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* Update Research Translation Loop.json

* Update .secrets.baseline

* fix: Move bundle test for arxiv

* Update PORTING.md

* Update component_index.json

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* Update component_index.json

* [autofix.ci] apply automated fixes

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* [autofix.ci] apply automated fixes

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* Update component_index.json

* Update component_index.json

* Update component_index.json

* fix: empty cache dict on reload

* Lint fixes

* Update test_components_cache_integration.py

* Update test_reload.py

* Hardening pass

* Update test_init_template.py

* fix: resolve cross-source bundle-name shadowing before registry population

The reload pipeline reads ``live.source_path`` from the registry on every
call.  Previously, the registry-population loop in
``import_extension_components`` only special-cased installed-shadows-seed;
for every other pair (seed/dev, seed/inline, dev/inline,
installed/dev, installed/inline) it silently overwrote earlier records
via last-wins iteration order.  A stale dev registration whose
``source_path`` pointed at a different filesystem location could clobber
the seed record's ``source_path``; reload would then walk the dev path
while the operator edited the seed copy on disk -- producing 200 OK with
empty deltas on every reload, including for syntactically broken edits.

Generalize the dedup pass to every (earlier, later) pair using the
explicit precedence ``installed > seed > dev > inline``, applied before
both registry population AND palette template construction so the two
read from the same winning source.  Add a new generic ``bundle-shadowed``
typed error code for the pairs the existing ``seed-bundle-shadowed`` did
not cover; keep ``seed-bundle-shadowed`` for the documented installed-
over-seed pair so existing CLI exit-code logic and snapshot tests stay
intact.  Both codes are warn-only in ``lfx extension list``.

Also add an INFO log line in reload Stage 1 with the resolved
``source_path`` so the next "200 OK with empty deltas" repro can be
triaged from the server log alone -- if the path logged is not the path
the operator was editing, this dedup is what to look at.

Includes a regression test
(``test_seed_bundle_shadows_dev_emits_generic_bundle_shadowed``) that
asserts both halves: seed wins in the BundleRegistry AND the registry's
``source_path`` is the seed path, not the stale dev path.

* Update component_index.json

* fix: widen lfx-* bundles' requires-python to <3.15 to match langflow root

The two pilot bundles (``lfx-arxiv``, ``lfx-duckduckgo``) were scaffolded
by ``scripts/migrate/port_bundle.py`` whose template hard-coded
``requires-python = ">=3.10,<3.14"``.  Because both bundles are uv
workspace members of the langflow root, that cap leaked into every
workspace-level resolve: ``cd src/lfx && uv sync`` (the path
``make lfx_tests`` takes) refuses to pick a Python 3.14 interpreter even
though lfx itself, langflow, and langflow-base all advertise
``>=3.10,<3.15``.  Hosts with 3.14 installed see:

    error: The requested interpreter resolved to Python 3.14.5, which
    is incompatible with the project's Python requirement: `>=3.10, <3.14`.

Fix at the source so future ``port_bundle.py`` runs do not regress this:
template emits ``<3.15``, and the two existing emitted pyprojects are
bumped in lockstep.  ``uv.lock`` regenerates with the wider range.

Verified ``make lfx_tests`` resolves cleanly on a host with both 3.13.12
and 3.14.5 installed; the focused extension slice runs to completion.

* Update __init__.py

* Update src/lfx/tests/unit/extension/migration/test_rewrite.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/extension/loader/_plugins.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update docs/docs/Deployment/deployment-extensions-production.mdx

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/lfx/src/lfx/extension/loader/_orchestrator.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update docs/docs/Develop/extensions-author-guide.mdx

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update docs/docs/Develop/extensions-manifest.mdx

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix: Coderabbit review suggestions

* Template updates

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* Tweaks to bundle hot reload

* [autofix.ci] apply automated fixes

* chore: Address review comments by @Cristhianzl

* [autofix.ci] apply automated fixes

* Update BUNDLE_API.md

* Update component_index.json

* Update component_index.json

* Update Structured Data Analysis Agent.json

* fix: Next round of review comments

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-05-18 17:36:56 +00:00
d017e156cd fix: add stream toggle back to agent (#13155)
* add stream toggle back to agent

* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]

* ci: stop transient Windows npm/cache flakes from failing frontend tests

The nightly frontend job fails almost every day on a single Windows
Playwright shard, never on actual test logic. With fail-fast:false over
~70 Windows shards each making several cache/artifact-service calls, a
transient GitHub Actions cache/artifact hiccup on any one shard flips
needs.setup-and-test.result to failure and reds the whole nightly
(observed steps: Setup Node.js Environment, Cache Playwright Browsers,
Upload Playwright Coverage Artifact).

- Decouple npm caching from actions/setup-node (its internal cache step
  can't be made non-fatal) and replace it with a standalone
  actions/cache@v5 step marked continue-on-error.
- Mark Cache Playwright Browsers and the artifact-upload steps
  continue-on-error so an infra blip can no longer fail a test shard.

Only a real test/browser failure can fail a shard now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* [autofix.ci] apply automated fixes

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-18 13:30:08 +00:00
bd35587957 chore: Add non-blocking test coverage advisor for PRs (#13148)
add checker to verify tests non blocker
2026-05-15 17:36:17 +00:00