mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-24 10:23:32 +08:00
feat/operations-component
1 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 7896b9f7f1 |
feat(bundles): metapackage split (Phase A) — engine-only lfx, lfx-bundles long tail, 5 graduated partner packages (#13563)
* feat(extension): add manifest-less lfx.bundles discovery + precedence tier Foundation for the bundle metapackage split (1.11). Adds a third @official-slot discovery source: a distribution declaring the [project.entry-points."lfx.bundles"] group ships a package whose immediate subdirectories are each a manifest-less bundle, folder-walked and registered at @official with no extension.json (the langchain-community model). - new loader/_bundles_root.py::load_lfx_bundles_extensions, mirroring discover_inline_bundles but sourcing roots from the lfx.bundles entry-point group via find_spec and reusing _load_bundle_directory - discovery precedence becomes installed > seed > lfx_bundles > dev > inline so a manifest-shipping lfx-<provider> always shadows the same-named provider in the metapackage (lets a provider graduate with no lockstep release), emitting the existing bundle-shadowed warning - new bundle-discovery-malformed warning code, emitted on warnings (never flips ok / aborts startup) for unresolvable declarations and invalid provider folder names - exported via loader/__init__ and the lfx.extension PEP-562 lazy surface 10 new tests in test_load_lfx_bundles.py; full extension unit suite (449) passes; ruff + format clean; mypy clean on the new module. * docs(bundles): changelog entry for the lfx.bundles discovery surface The BUNDLE_API.md changelog gate diffs each PR's branch against main, so the entry covering this PR's surface additions (lfx.bundles discovery, the lfx_bundles precedence tier, bundle-discovery-malformed) must live on this branch, not only further up the stack. Text is verbatim from the graduate-partners commit so the stacked merge dedupes trivially. * fix(extension): reject plain-module lfx.bundles targets; correct validate wording CodeRabbit review fixes on #13563: - _spec_package_dir no longer falls back to a plain module's parent directory -- a single-file entry-point target now emits the bundle-discovery-malformed warning instead of folder-walking unrelated sibling directories as bundles (+ regression test) - BUNDLE_API.md / loader docstring no longer say manifest-less providers are 'exempt from lfx extension validate'; they are not valid validator input (validate requires a manifest and reports manifest-not-found) * fix(extension): review fixes — shadowed metapackage providers never import; typed manifest-less reload refusal Review findings on #13563: - A manifest-less provider whose bundle name is already claimed by a higher-precedence @official source (installed/seed) is now skipped WITHOUT importing: all @official sources share the _lfx_ext.official.<bundle>.* sys.modules namespace, so importing the losing copy overwrote the winner's live modules even though _resolve_bundle_shadowing dropped its components afterwards. The orchestrator passes the claimed-name map (built by the new _claimed_official_bundles helper, mirroring the resolver's winner rule); the skipped copy still emits the typed bundle-shadowed warning. This collision is the expected post-graduation state, not an error. - Manifest-less records now carry a manifestless provenance flag (additive field on LoadResult + BundleRecord). The reload pipeline refuses them with the new typed reload-manifestless-unsupported code instead of routing through load_extension and failing with a misleading manifest-not-found. Hot reload of metapackage providers is a process-restart concern (pip-installed content); a loader-based reload path can follow if QA wants it. BUNDLE_API.md changelog entry extended for both surface changes. * fix(extension): harden lfx.bundles discovery per review — broad find_spec guard, namespace portions, per-mode error codes Review findings from ogabrielluiz on #13563: - find_spec on a dotted declaration imports the PARENT package, whose __init__ can raise anything; a non-Import error escaped the old catch tuple and (through the palette cache's catch-all) wiped every source's components for the boot. Catch Exception and degrade to the malformed sentinel; comment corrected. - Namespace packages: walk ALL submodule_search_locations portions (one root per portion) instead of only locations[0], and dedupe resolved roots by path so duplicate entry-point declarations (or overlapping portions) never walk a directory twice and self-shadow. _spec_package_dir -> _spec_package_dirs. - Split bundle-discovery-malformed into one code per failure mode, mirroring the inline tier: bundles-provider-name-invalid (rename the directory), bundles-root-unreadable (check permissions), keeping bundle-discovery-malformed for unresolvable declarations (fix the entry-point). Same-tier duplicates get duplicate-lfx-bundles-provider instead of overloading bundle-shadowed, whose rendered template (format_extension_error renders templates, not ad-hoc messages) was wrong on both counts for that case. - The claimed-bundles cross-source skip now emits bundle-shadowed on errors (matching _resolve_bundle_shadowing) so filtering by code never mixes severities; bundle-shadowed is already in the CLI warn-only set. 3 new regression tests (raising parent package, namespace portions, duplicate entry points); BUNDLE_API changelog updated. * feat(bundles): create lfx-bundles metapackage skeleton (#13564) * feat(bundles): add lfx-bundles metapackage skeleton Creates the manifest-less lfx-bundles distribution (the langchain-community model) that the long-tail providers will move into. Empty skeleton for now; the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider folders + per-provider extras later. - src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a generated (currently empty) `all` extra; plus a bare lfx_bundles namespace package and a README documenting the model + install stories - wired into the root workspace via the existing bundle marker blocks: dep lfx-bundles[all]>=1.0,<2.0, uv source, and member - hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds it with zero workflow change Verified: the wheel builds (entry_points.txt carries the lfx.bundles group); load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty skeleton registers zero providers with no error. * fix(ci): nightly bundle rename follows [extras] refs and self-refs Two gaps in update_bundle_versions.py around extras suffixes, both fatal or silently wrong for the first nightly carrying the lfx-bundles metapackage: - update_root_pyproject_for_bundle's dep regex required the version specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0" (a MAIN dep) was left unrewritten while the workspace member was renamed to lfx-bundles-nightly; uv lock then tries to resolve stable lfx-bundles from PyPI, where it does not exist. The same gap silently left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the stable distribution. The regex now tolerates an [extras] group and carries it into the replacement. - rename_bundle_pyproject skipped self-referencing extras, so the metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]" members after the rename, pulling the stable distribution (same lfx_bundles import package, install collision) once published. Self refs now follow the rename, idempotently. Tests drive the real script module, mirroring test_bundle_lfx_pin.py. This closes the PR-2 audit item flagged when the metapackage was introduced. The canonical-pre-release cutover would retire the nightly rename entirely; until it lands, the rename must be correct. * fix(bundles): address review — drop orphaned nightly bundle-rename script, fix README install stories - Delete scripts/ci/update_bundle_versions.py and its tests: the nightly bundle-rename track it served was retired by the canonical pre-release cutover (src/bundles/NIGHTLY.md) and no workflow or Makefile target on main or any release branch invokes it. Stale doc references in sync_bundle_lfx_pin.py and test_bundle_lfx_pin.py updated to match. - README: split the install section into what works today (langflow, bare lfx) vs what arrives with the bulk move / engine-only split (lfx[bundles], per-provider lfx-bundles[<provider>] extras), and note the generated all extra is empty until the first provider tranche lands. * ci(bundles): cross-bundle test matrix (lfx contract axis) (#13566) * ci(bundles): add cross-bundle test matrix (lfx contract axis) Tests every extracted bundle (the lfx-bundles metapackage + each graduated lfx-<provider>) against the lfx contract surface it depends on -- overdue since 1.10 left 4 independently-versioned bundles depending on lfx. - .github/workflows/cross-bundle-test.yml: discovers bundles via the same src/bundles/*/pyproject.toml glob release.yml uses, then per (bundle x python 3.10/3.13): installs the in-repo lfx + the bundle, imports the declared entry-point package, asserts lfx.bundles discovery is error-free (for the metapackage), runs `lfx extension validate` for manifest bundles, and runs the bundle's own tests/. - Triggers: pull_request (src/bundles, src/lfx), workflow_dispatch, a weekly schedule, and workflow_call. The lfx-minor axis seeds with the in-repo lfx (the 1.10 line is unpublished); when minors publish, add the version dimension (oldest+latest get full tests, every supported minor gets contract-smoke) per the epic's cost-control shape. Verified: the workflow YAML parses and the contract-smoke logic imports a real bundle (lfx_arxiv) cleanly. * fix(ci): cross-bundle smoke tolerates extras-less SDK degradation Review findings on the cross-bundle matrix: - CRITICAL: the contract smoke asserted zero discovery errors, but the CI venv installs lfx-bundles WITHOUT per-provider extras, so providers whose modules import their SDK at top level degrade with module-import-failed -- the expected graceful-degradation contract. The smoke now fails only on structural codes and reports the degraded-module count. (Graduated partner bundles carry their deps directly, not as extras, so their steps are unaffected.) - the scheduled run now actually delivers the exhaustive grid the header promised (all supported Pythons on schedule; oldest+latest on PRs) - concurrency group with cancel-in-progress on PRs so stacked bundle PRs don't queue N-bundles x N-pythons jobs per push * ci(bundles): tomli fallback for the py3.10 cross-bundle smoke tomllib is stdlib only from Python 3.11, so the contract-smoke heredoc failed with ModuleNotFoundError on every py3.10 matrix leg. Install the tomli backport into the smoke venv and import it as a fallback. * fix(test): coherent sys.modules restore in the ibm without_ibm_db fixture The cross-bundle matrix's py3.10 leg exposed test-ordering pollution in the ibm bundle suite: without_ibm_db monkeypatch-deleted three lfx_ibm modules and re-imported them mid-test, but entries created during the test survive teardown (delitem with raising=False records nothing for keys it didn't find) and re-imported parents never regain the submodule attribute bindings that mock.patch target resolution walks on Python 3.10 (3.11+ mock resolves via sys.modules and tolerates the incoherence). Two fixture uses in sequence left the tree inconsistent and 29 later watsonx tests failed with AttributeError on the package. Snapshot, drop, and restore the entire lfx_ibm module tree wholesale instead. Full suite now passes on both 3.10 and 3.13. * feat(lfx): add lfx[bundles] extra and keep lfx engine-only (#13565) * feat(bundles): add lfx-bundles metapackage skeleton Creates the manifest-less lfx-bundles distribution (the langchain-community model) that the long-tail providers will move into. Empty skeleton for now; the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider folders + per-provider extras later. - src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a generated (currently empty) `all` extra; plus a bare lfx_bundles namespace package and a README documenting the model + install stories - wired into the root workspace via the existing bundle marker blocks: dep lfx-bundles[all]>=1.0,<2.0, uv source, and member - hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds it with zero workflow change Verified: the wheel builds (entry_points.txt carries the lfx.bundles group); load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty skeleton registers zero providers with no error. * feat(lfx): add lfx[bundles] extra and keep lfx engine-only lfx ships no bundles by default. The new optional extra pulls the long-tail metapackage for users who want the provider components without the full langflow server install. - src/lfx/pyproject.toml: [project.optional-dependencies] bundles = ["lfx-bundles[all]>=1.0,<2.0"]; intentionally NO lfx[all] - pip install lfx -> engine only pip install "lfx[bundles]" -> engine + the lfx-bundles long tail (documented as a headless/serverless deployment footnote, not a headline) Verified against the built lfx wheel METADATA: lfx-bundles appears only under `extra == "bundles"`, never in core Requires-Dist, so the engine-only default is preserved. * fix(ci): nightly bundle rename follows [extras] refs and self-refs Two gaps in update_bundle_versions.py around extras suffixes, both fatal or silently wrong for the first nightly carrying the lfx-bundles metapackage: - update_root_pyproject_for_bundle's dep regex required the version specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0" (a MAIN dep) was left unrewritten while the workspace member was renamed to lfx-bundles-nightly; uv lock then tries to resolve stable lfx-bundles from PyPI, where it does not exist. The same gap silently left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the stable distribution. The regex now tolerates an [extras] group and carries it into the replacement. - rename_bundle_pyproject skipped self-referencing extras, so the metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]" members after the rename, pulling the stable distribution (same lfx_bundles import package, install collision) once published. Self refs now follow the rename, idempotently. Tests drive the real script module, mirroring test_bundle_lfx_pin.py. This closes the PR-2 audit item flagged when the metapackage was introduced. The canonical-pre-release cutover would retire the nightly rename entirely; until it lands, the rename must be correct. * ci(bundles): freeze lfx/components against new top-level providers (#13567) * feat(bundles): add lfx-bundles metapackage skeleton Creates the manifest-less lfx-bundles distribution (the langchain-community model) that the long-tail providers will move into. Empty skeleton for now; the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider folders + per-provider extras later. - src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a generated (currently empty) `all` extra; plus a bare lfx_bundles namespace package and a README documenting the model + install stories - wired into the root workspace via the existing bundle marker blocks: dep lfx-bundles[all]>=1.0,<2.0, uv source, and member - hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds it with zero workflow change Verified: the wheel builds (entry_points.txt carries the lfx.bundles group); load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty skeleton registers zero providers with no error. * ci(bundles): freeze lfx/components against new top-level providers After the metapackage split, new providers go to lfx-bundles (or a graduated lfx-<provider>), never in-tree. Adds an additions-only CI gate. - scripts/ci/check_components_frozen.py: stdlib gate comparing the live top-level dir listing of src/lfx/src/lfx/components/ against a committed baseline; fails on any directory not in the baseline. Removals are allowed so M4 shim cleanup (which shrinks the set) never trips it. - scripts/ci/frozen_component_dirs.txt: the 111-dir baseline. - .github/workflows/lint-py.yml: new freeze-components job runs the gate (stdlib-only, no uv sync). Verified locally: passes on the baseline, fails (exit 1) on a simulated new provider directory with an actionable message, passes again after cleanup. * fix(ci): freeze gate counts only dirs shipping __init__.py Review finding: a stray directory holding only __pycache__ bytecode (left behind by a branch switch) tripped the additions-only gate locally. A directory without __init__.py is not a provider; ignore it. * fix(ci): nightly bundle rename follows [extras] refs and self-refs Two gaps in update_bundle_versions.py around extras suffixes, both fatal or silently wrong for the first nightly carrying the lfx-bundles metapackage: - update_root_pyproject_for_bundle's dep regex required the version specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0" (a MAIN dep) was left unrewritten while the workspace member was renamed to lfx-bundles-nightly; uv lock then tries to resolve stable lfx-bundles from PyPI, where it does not exist. The same gap silently left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the stable distribution. The regex now tolerates an [extras] group and carries it into the replacement. - rename_bundle_pyproject skipped self-referencing extras, so the metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]" members after the rename, pulling the stable distribution (same lfx_bundles import package, install collision) once published. Self refs now follow the rename, idempotently. Tests drive the real script module, mirroring test_bundle_lfx_pin.py. This closes the PR-2 audit item flagged when the metapackage was introduced. The canonical-pre-release cutover would retire the nightly rename entirely; until it lands, the rename must be correct. * feat(bundles): move 45 long-tail providers into lfx-bundles + graduate 5 partner packages (#13568) * feat(bundles): add lfx-bundles metapackage skeleton Creates the manifest-less lfx-bundles distribution (the langchain-community model) that the long-tail providers will move into. Empty skeleton for now; the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider folders + per-provider extras later. - src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a generated (currently empty) `all` extra; plus a bare lfx_bundles namespace package and a README documenting the model + install stories - wired into the root workspace via the existing bundle marker blocks: dep lfx-bundles[all]>=1.0,<2.0, uv source, and member - hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds it with zero workflow change Verified: the wheel builds (entry_points.txt carries the lfx.bundles group); load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty skeleton registers zero providers with no error. * feat(bundles): consolidate first long-tail tranche into lfx-bundles Adds scripts/migrate/consolidate_bundles.py (the inverse of port_bundle.py -- moves in-tree providers into the manifest-less lfx-bundles metapackage) and runs it on a verified 5-provider tranche: tavily, exa, wikipedia, yahoosearch, wolframalpha. Per provider the script: - moves src/lfx/src/lfx/components/<p>/ -> lfx_bundles/<p>/ (lowercase names); - leaves a fail-soft import shim (first line `# lfx-bundles-shim`) so `from lfx.components.<p> import X` keeps working when lfx-bundles is installed, and raises an actionable ImportError otherwise; - merges the provider's third-party deps into a PEP 685-normalized lfx-bundles extra and regenerates the `all` aggregate. Dep parity holds: `uv sync` is a no-op because those deps were already pulled via langflow-base[complete]; - appends the 4-entry migration block per Component class (28 entries) so saved flows referencing lfx.components.<p>.<Class> migrate to ext:<p>:<Class>@official. To avoid double registration, the in-tree component walk (_load_components_dynamically) now skips shimmed provider dirs, and component_index.json is regenerated (355->348 components, 95->90 modules); the moved providers load only at @official via lfx.bundles discovery. Verified: discovery finds all 5 at @official with no errors; shims resolve; `import lfx.components` still works; the index drops the 7 moved component entries (residual `tools`-category name refs resolve via the shim); ruff clean. First tranche proves the engine; the remaining long-tail scales by extending PROVIDER_DEPS (each provider's deps verified individually -- the careful part). * feat(bundles): consolidate 30-provider tranche 2 into lfx-bundles Extends PROVIDER_DEPS with 30 individually-verified providers and runs the consolidation: vector stores (chroma, clickhouse, couchbase, milvus, mongodb, pgvector, pinecone, qdrant, supabase, upstash, weaviate), model providers (groq, mistral, ollama, perplexity, sambanova), and tools/memory/data (apify, assemblyai, confluence, firecrawl, git, glean, icosacomputing, mem0, needle, scrapegraph, serpapi, unstructured, youtube, zep). Dep verification (the careful part): every spec comes from langflow-base's per-provider extras or its direct dependencies; langchain-community providers carry the wrapper plus the SDK the wrapper lazy-imports (e.g. pgvector, atlassian-python-api); requests is declared explicitly where imported (it is only transitive in today's env); pinecone keeps its python_version<'3.14' marker verbatim. Tranche excludes: providers with langflow imports (vlmrun), provider-specific lfx.base dirs (composio/huggingface/langwatch), case- sensitive names (FAISS/Notion), the openai-SDK family (azure/aiml/deepseek/ litellm/lmstudio/novita/openrouter/vllm/xai/cometapi -- cleaner after PR-8), and the partner set. Dep parity verified at the resolution level: the uv.lock diff is +220 lines of lfx-bundles extras metadata with ZERO packages added or removed (`name =` diff empty), so pip install langflow resolves the identical set. Also: 192 append-only migration entries (48 classes x 4, zero bare-name ambiguities); component_index.json regenerated 348->300 components / 90->60 modules (exactly the moved set, no stale standalone entries); mongodb_atlas.py SLF001 per-file-ignore and the mem0/mongodb detect-secrets baseline entries migrated to the new paths (lint/secrets exceptions travel with moved files); 35 bundles now discover at @official with 55 components; shims verified across categories; 449 extension tests pass. * feat(bundles): consolidate openai-SDK family tranche 3 into lfx-bundles 10 providers that ride the langchain-openai wrapper, deferred from tranche 2 until the partner graduation settled the openai-SDK dep story: aiml, azure, cometapi, deepseek, litellm, lmstudio, novita, openrouter, vllm, xai. Dep verification: every provider declares langchain-openai>=1.1.6; the openai SDK is declared only where a component imports it directly (aiml, deepseek, litellm, lmstudio, vllm, xai -- wrapper-transitive elsewhere); requests declared where imported (cometapi, deepseek, novita, xai); lmstudio's lazy NVIDIAEmbeddings path gets langchain-nvidia-ai-endpoints~=1.0.0. The litellm component drives LiteLLM-served endpoints through the OpenAI client and does NOT import the litellm package -- langflow-base's litellm extra stays put. These providers use the lazy _dynamic_imports __init__ shape; it survives the move unchanged (import_mod resolves via __spec__.parent and lfx.components._importing remains a core helper). Dep parity: uv.lock diff has zero package additions/removals (all specs already resolved via langflow-base[complete]). Also: 56 append-only migration entries (14 classes x 4, zero ambiguities); component_index.json regenerated 300->286 components / 60->50 modules (exactly the moved set); detect-secrets baseline re-keyed; 45 bundles now discover at @official with 69 components; shims verified (azure/xai/litellm/deepseek); ruff clean. * fix(ci): nightly bundle rename follows [extras] refs and self-refs Two gaps in update_bundle_versions.py around extras suffixes, both fatal or silently wrong for the first nightly carrying the lfx-bundles metapackage: - update_root_pyproject_for_bundle's dep regex required the version specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0" (a MAIN dep) was left unrewritten while the workspace member was renamed to lfx-bundles-nightly; uv lock then tries to resolve stable lfx-bundles from PyPI, where it does not exist. The same gap silently left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the stable distribution. The regex now tolerates an [extras] group and carries it into the replacement. - rename_bundle_pyproject skipped self-referencing extras, so the metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]" members after the rename, pulling the stable distribution (same lfx_bundles import package, install collision) once published. Self refs now follow the rename, idempotently. Tests drive the real script module, mirroring test_bundle_lfx_pin.py. This closes the PR-2 audit item flagged when the metapackage was introduced. The canonical-pre-release cutover would retire the nightly rename entirely; until it lands, the rename must be correct. * feat(bundles): graduate partner set to standalone lfx-<provider> packages (#13573) * feat(bundles): graduate partner set to standalone lfx-<provider> packages Extracts the five partner/flagship providers into manifest-shipping distributions: lfx-openai, lfx-anthropic, lfx-amazon, lfx-datastax, lfx-cohere. Each ships extension.json (lfx.compat ["1"]), a langflow.extensions entry-point, an lfx>=1.10.0,<2.0.0 pin, and is wired into the root workspace marker blocks. Zero flow impact by the bundle-name invariant: ext:<provider>:<Class>@official ids are unchanged. Mechanics: - adopts the five-phase scripts/migrate/port_bundle.py from feat/bundle-mass-extraction (handles shared-base moves, consumer rewrites, surgical index updates, migration-table appends) and runs it per partner with --migration-release 1.11.0 - datastax's shared lfx.base.datastax moves into lfx_datastax.base; its backend unit tests move to src/bundles/datastax/tests (74 pass); its ruff per-file-ignore and its check_component_env_writes ALLOWLIST entry travel; 10 repo consumers rewritten (incl. the vector_store_rag starter project) - amazon_bedrock_converse.py legacy langflow.* imports rewritten to the lfx.* equivalents (thin re-export aliases; behavior-identical) pre-extraction - runtime deps pinned from langflow-base's extras / direct deps (wrapper- guaranteed SDKs stay transitive, parity-exact); --remove-base-extra dropped the openai/anthropic/cohere/aws extras from langflow-base[complete] in favor of the bundle pins; the astradb extra stays (also used outside datastax) - in-tree dirs replaced with marker shims pointing at lfx_<p>.components.<p> (skipped by the in-tree walk; legacy from lfx.components.<p> imports keep working); lfx/components/__init__.py entries kept, consistent with the consolidated providers - validate.py: accept classes whose base is a *derived* Component base (LCVectorStoreComponent / LCToolComponent / ...) -- they inherit the class-level outputs declaration and only override the output method, which the AST-only check could not see; bare Component subclasses keep the strict build/outputs requirement (+ regression test) - BUNDLE_API.md changelog entries added for the branch's surface changes: the lfx.bundles manifest-less discovery group + precedence tier + bundle-discovery-malformed code (from the foundation PR) and the validator acceptance above - 100 append-only migration entries (20 classes x 5 partners x 4 shapes); component_index.json surgically updated 300->280 components / 60->55 modules, sha256 recomputed and verified - detect-secrets baseline re-keyed for moved partner files Dep parity: lock diff adds only the five lfx-* package names; no third-party package added or removed. One benign transitive re-resolution: anthropic SDK 0.105.2 -> 0.109.0 (allowed by langchain-anthropic's range on both sides). Verified: all five register at @official via the installed-manifest tier (extension_id/distribution = lfx-<p>, 20 components); manifest-less lfx_bundles unchanged (35 bundles, disjoint); all 5 shims import; engine-safe; `lfx extension validate` passes for all five; 450 extension unit tests, 60 pilot-upgrade migration tests, 74 moved datastax tests pass; ruff clean. * fix(bundles): shim lfx.base.datastax so stored-flow imports keep resolving The datastax graduation moved lfx.base.datastax into lfx_datastax.base, but saved flows and starter templates (Hybrid Search RAG, TemplateAssistant) embed `from lfx.base.datastax.astradb_base import AstraDBBaseComponent` inside their stored component code fields, which is re-executed verbatim at flow build time. Without a shim that import raises ModuleNotFoundError and the flows fail to build (test-starter-projects red). Mirror the lfx.components shim contract: module-aliasing to lfx_datastax.base, narrow except that only translates a missing bundle (transitive dep failures re-raise untouched), marker-tagged single-file dir, removed at M4 together with the components shims. * chore(bundles): bounded version ranges for curated lfx-* packages (#13576) langflow's pyproject is the release-coordination point now that lfx and the lfx-* bundles release independently. Every curated lfx-* dependency declares a BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library metadata create resolver conflicts for downstreams that depend on a different version. Exact pins for reproducibility live in uv.lock and the release-build manifests. - the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles follow their own 0.1.x cadence) - lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage contract) - the three lfx-docling[...] optional-dependency refs bounded the same way - policy documented inline in the bundle-deps marker block; the cross-bundle CI matrix verifies the ranges resolve together across the supported lfx axis Verified: uv lock resolves with zero package/version changes (bounds are metadata-only today); the nightly rename's dep regex already matches the bounded form. * test(bundles): lock the in-tree shim contract + breakage audit (#13577) * chore(bundles): bounded version ranges for curated lfx-* packages langflow's pyproject is the release-coordination point now that lfx and the lfx-* bundles release independently. Every curated lfx-* dependency declares a BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library metadata create resolver conflicts for downstreams that depend on a different version. Exact pins for reproducibility live in uv.lock and the release-build manifests. - the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles follow their own 0.1.x cadence) - lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage contract) - the three lfx-docling[...] optional-dependency refs bounded the same way - policy documented inline in the bundle-deps marker block; the cross-bundle CI matrix verifies the ranges resolve together across the supported lfx axis Verified: uv lock resolves with zero package/version changes (bounds are metadata-only today); the nightly rename's dep regex already matches the bounded form. * test(bundles): lock the in-tree shim contract + breakage audit Locks the five-point contract for the 50 lfx-bundles import shims under lfx/components/ (45 metapackage + 5 graduated partners): 1. first line is the `# lfx-bundles-shim` marker (keys the in-tree walk's double-registration skip); 2. a shim dir is exactly one __init__.py -- no impls, no deps; 3. sys.modules module-aliasing (submodule trees resolve); 4. bundle missing -> actionable ModuleNotFoundError whose wording is part of the contract ("pip install lfx-bundles" / "pip install lfx-<p>"); 5. a missing *transitive* dep re-raises untouched. Test shape (env-adaptive by design -- the lfx test env prunes the venv, so nothing here assumes bundles are installed): - source-level sweep of every real shim, parameterized (no imports); - hermetic mechanism tests against a synthetic shim + synthetic target (alias-resolves / locked-message / transitive-reraise); - the walk-skip detector and the marker sweep must agree exactly; - one adaptive live test on lfx.components.tavily exercising whichever branch the running env is in. 106 tests pass in the pruned lfx env (where the live test exercises the bare-engine locked-message branch). Breakage audit (gh code search, recorded in the PR): ~20-25 public repos reference lfx.components.*; the majority are langflow forks (vendored tree, unaffected by packaging). Genuine library consumers exist and the moved providers have real surface (openai: 11 external repos, datastax: 17) -- covered by the shims wherever bundles are co-installed (every `pip install langflow`). Decision: deprecation warnings NOT warranted now; revisit at M4 (shim removal), where one loud minor release before removal is the right shape. * test(bundles): extend the shim contract sweep to lfx.base shims The datastax graduation moved lfx.base.datastax into lfx_datastax.base and left a module-aliasing shim behind (stored flow code fields embed the legacy import and are re-executed verbatim at build time). Sweep lfx/base for marker shims with the same source-level contract as the components sweep: one-file stub, module-aliasing to the bundle's base subpackage, narrow name-checked except, locked install message. * docs(bundles): install shapes, override rule, bundle_api_version (#13578) Documents the deliberately small user-facing rule surface of the metapackage split (1.11): - extensions-overview.mdx: the two install stories (pip install langflow = everything, same as today; pip install lfx = engine only, bring your own bundles); the bundle-name-is-identity invariant; the current package table (lfx-bundles metapackage + the 5 partner packages + the 4 pilots); the legacy-import shim behavior with the locked ModuleNotFoundError wording and the migration recipe for direct lfx users (switch to lfx[bundles] or pin lfx-<provider> packages); the override rule ("ship a manifest -- a manifest always wins", with the bundle-shadowed warning); bundle_api_version as the single compat number (lfx.compat ["1"]; manifest-less packages ride their PEP 508 lfx pin) and the no-version-arithmetic rule. - deployment-lfx-compatibility.mdx: lfx[bundles] as the headless/serverless deployment footnote (engine + lfx-bundles[all]; slimmer per-provider alternative; intentionally no lfx[all]). Both files verified MDX-safe (no unbackticked angle brackets). * fix(bundles): stack-review fixes — symlink containment, idempotency, docs accuracy (#13579) * chore(bundles): bounded version ranges for curated lfx-* packages langflow's pyproject is the release-coordination point now that lfx and the lfx-* bundles release independently. Every curated lfx-* dependency declares a BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library metadata create resolver conflicts for downstreams that depend on a different version. Exact pins for reproducibility live in uv.lock and the release-build manifests. - the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles follow their own 0.1.x cadence) - lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage contract) - the three lfx-docling[...] optional-dependency refs bounded the same way - policy documented inline in the bundle-deps marker block; the cross-bundle CI matrix verifies the ranges resolve together across the supported lfx axis Verified: uv lock resolves with zero package/version changes (bounds are metadata-only today); the nightly rename's dep regex already matches the bounded form. * test(bundles): lock the in-tree shim contract + breakage audit Locks the five-point contract for the 50 lfx-bundles import shims under lfx/components/ (45 metapackage + 5 graduated partners): 1. first line is the `# lfx-bundles-shim` marker (keys the in-tree walk's double-registration skip); 2. a shim dir is exactly one __init__.py -- no impls, no deps; 3. sys.modules module-aliasing (submodule trees resolve); 4. bundle missing -> actionable ModuleNotFoundError whose wording is part of the contract ("pip install lfx-bundles" / "pip install lfx-<p>"); 5. a missing *transitive* dep re-raises untouched. Test shape (env-adaptive by design -- the lfx test env prunes the venv, so nothing here assumes bundles are installed): - source-level sweep of every real shim, parameterized (no imports); - hermetic mechanism tests against a synthetic shim + synthetic target (alias-resolves / locked-message / transitive-reraise); - the walk-skip detector and the marker sweep must agree exactly; - one adaptive live test on lfx.components.tavily exercising whichever branch the running env is in. 106 tests pass in the pruned lfx env (where the live test exercises the bare-engine locked-message branch). Breakage audit (gh code search, recorded in the PR): ~20-25 public repos reference lfx.components.*; the majority are langflow forks (vendored tree, unaffected by packaging). Genuine library consumers exist and the moved providers have real surface (openai: 11 external repos, datastax: 17) -- covered by the shims wherever bundles are co-installed (every `pip install langflow`). Decision: deprecation warnings NOT warranted now; revisit at M4 (shim removal), where one loud minor release before removal is the right shape. * docs(bundles): install shapes, override rule, bundle_api_version Documents the deliberately small user-facing rule surface of the metapackage split (1.11): - extensions-overview.mdx: the two install stories (pip install langflow = everything, same as today; pip install lfx = engine only, bring your own bundles); the bundle-name-is-identity invariant; the current package table (lfx-bundles metapackage + the 5 partner packages + the 4 pilots); the legacy-import shim behavior with the locked ModuleNotFoundError wording and the migration recipe for direct lfx users (switch to lfx[bundles] or pin lfx-<provider> packages); the override rule ("ship a manifest -- a manifest always wins", with the bundle-shadowed warning); bundle_api_version as the single compat number (lfx.compat ["1"]; manifest-less packages ride their PEP 508 lfx pin) and the no-version-arithmetic rule. - deployment-lfx-compatibility.mdx: lfx[bundles] as the headless/serverless deployment footnote (engine + lfx-bundles[all]; slimmer per-provider alternative; intentionally no lfx[all]). Both files verified MDX-safe (no unbackticked angle brackets). * fix(bundles): review fixes — symlink containment, idempotency, docs accuracy Fixes from the multi-agent stack review (kept as a separate PR so the reviewed PRs' diffs stay frozen for QA's independent pass): - _bundles_root.py: directory-level symlink containment — a provider dir that resolves outside the bundle root is skipped with a typed bundle-discovery-malformed warning, mirroring the seed-directory walk's is_within rule (+ regression test). Anchors the trust boundary to the installed package tree. - consolidate_bundles.py: migration-append idempotency guard — entries are deduped on full content so a partially-failed earlier run cannot duplicate rows on re-run (table stays append-only). - langflow-base pyproject: documented WHY the aws extra is deliberately retained after the lfx-amazon graduation (boto3 also backs the server's own S3 storage backend and lfx's S3 ingestion — review suggested removing it; that would regress S3 storage support). - extensions-overview.mdx: note that `lfx extension list` shows manifest-shipping extensions only; manifest-less packages load at startup but are not listed. 93 loader+migration tests pass (incl. the new symlink test); ruff clean. * refactor(bundles): stabilize import_mod as a public BUNDLE_API utility The lazy-import helper that bundle packages call from their __getattr__-based __init__.py files lived at lfx.components._importing -- an internal path with no stability contract, imported by 35 separately-installed bundle __init__ files (30 lfx-bundles providers + the 5 graduated partners). - canonical home is now lfx.utils.lazy_import.import_mod (code unchanged); lfx.components._importing re-exports it so in-tree callers and any external code on the old path keep working - all 35 bundle-side imports rewritten to the stable path - BUNDLE_API.md: surface table entry + changelog (additive) - contract tests: re-export identity, both call forms, the AttributeError conversion Verified: identity holds across both paths; lazy __init__ loads work through the new path for both bundle families; 110 tests pass; ruff clean. * fix(bundles): tombstone the broken legacy ZepChatMemory build method (#13580) build_message_history targeted the zep-python v1 SDK (ZepClient + zep_python.langchain.ZepChatMessageHistory); both were removed in zep-python 2.x and the zep extra pins 2.0.2, so the method has been unable to run for as long as the pin has existed -- its ImportError guard misleadingly told users to 'pip install zep-python' (already installed). The component is legacy=True with helpers.Memory as its designated replacement, so rather than hand-write a new integration against the 2.x SDK, the method now raises a clear RuntimeError pointing at the Message History component. Flow identity is preserved: class/component name, display_name, description, inputs and the memory output are byte-identical, so saved flows keep loading, i18n locale keys are unchanged, and migration_table.json needs no edits. New bundle tests pin the stub contract (identity, actionable error, no zep_python import). * test(bundles): extend the shim contract sweep to lfx.base shims The datastax graduation moved lfx.base.datastax into lfx_datastax.base and left a module-aliasing shim behind (stored flow code fields embed the legacy import and are re-executed verbatim at build time). Sweep lfx/base for marker shims with the same source-level contract as the components sweep: one-file stub, module-aliasing to the bundle's base subpackage, narrow name-checked except, locked install message. * test(bundles): extras-drift guard for the lfx-bundles metapackage Risk-2 of the metapackage split: the generated `all` extra and the per-provider extras must never drift by hand-edit, or `pip install langflow` silently loses a provider's deps. Guard the four invariants: extras <-> provider dirs (PEP 685-normalized), `all` == the exact self-ref set, normalized keys collision-free, and the metapackage provider set disjoint from the graduated partner distributions. * fix(extension): symlink-escaped providers reject with path-escape, matching the documented contract The lfx.bundles provider containment check (stack-review symlink fix) emitted bundle-discovery-malformed, whose template and hint describe a broken entry-point declaration. The changelog's path-safety entry already documents symlink escapes as path-escape on every other discovery path; use the same code here. Also resolves the semantic merge conflict with the per-mode code split (the removed _malformed_error location kwarg). * fix(tests): repoint lfx tests off moved providers; complete provider fallback map CI fail-fast had been masking these: the LFX test job runs in an engine-only env where openai/anthropic/chroma are now bundle-package shims, so every test that used them as the example category failed on all Python versions (3.14 just reported first), and the backend Group 2 leg failed collecting test_lfx_bundles_extras.py on Python 3.10. - flow_requirements: complete _PROVIDER_PACKAGE_FALLBACKS for the nine moved model providers (OpenAI, Anthropic, Amazon Bedrock, Groq, Google Generative AI, SambaNova, IBM watsonx.ai, Ollama + existing Azure OpenAI). In engine-only installs MODEL_PROVIDERS_DICT registers only in-tree providers, so the dynamic source-inspection path cannot resolve moved providers; the static fallbacks keep lfx run/serve requirements inference working. A dict hit still wins. - test_dynamic_imports / test_import_utils: use composio (still-in-tree lazy category with its SDK absent in the bare env) as the example category instead of openai/anthropic/chroma; patch import_module at its canonical home lfx.utils.lazy_import. - flow-builder tests (build_flow_from_spec, flow_builder_tools, propose_field_edit): specs use LanguageModelComponent (in-tree) instead of OpenAIModel. - test_lfx_bundles_extras: tomli fallback for Python 3.10 (tomllib is stdlib 3.11+; pytest guarantees tomli on <3.11). Full lfx unit suite: 4550 passed. Backend extras+pin tests: 24 passed. * fix(tests): repoint MCP client-server tests off graduated OpenAIModel key Same fail-fast-masked class as |
