mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-24 12:26:01 +08:00
064069b6bbb4e9e92e02891b9bfb3ed2fdc0586f
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| f95fb0603d |
feat(bundles): consolidate FAISS, Notion into lfx-bundles (lowercase-slug support)
Uppercase source dirs need lowercase bundle slugs (BUNDLE_NAME_RE is lowercase-only). Enhance consolidate_bundles.py with bundle_slug(provider) = provider.lower(): the bundle dir, ext id, shim target, and extra key use the lowercased slug, while the migration import_path keeps the historical lfx.components.<Provider> casing so saved flows still resolve. Identity for already-lowercase providers (no change to prior entries). - FAISS -> lfx_bundles/faiss (faiss-cpu[py-split] + langchain-community) 1 component - Notion -> lfx_bundles/notion (requests + Markdown) 8 components test_bundle_shims.py: derive the expected bundle name by lowercasing the shim dir name (FAISS shim aliases lfx_bundles.faiss) — 159 shim tests pass. Relocate the lfx-isolated FAISS test to backend/vectorstores (repointed to lfx_bundles.faiss.faiss). 36 migration entries (0 ambiguous); index 22->20 categories, 140->131 components. SKIP=detect-secrets: regenerated code_hash hex. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| 594e4db2fc |
feat(bundles): consolidate langwatch evaluator into lfx-bundles
langwatch -> lfx_bundles/langwatch (1 component). The LangWatchComponent is a pure httpx REST wrapper (its only non-core touchpoint is lfx.base.langwatch.utils, which is also httpx-based) — the langwatch SDK extra in langflow-base is for the tracing service, not this component, so the bundle declares no extra deps. Moves + shim + `all` regen; 4 migration entries (0 ambiguous). Component index 23->22 categories, 141->140 components. Backend langwatch test repointed (16 passed). SKIP=detect-secrets: only flags are regenerated component_index code_hash hex (false positives). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| fee3f38865 |
feat(bundles): consolidate composio, huggingface, nvidia, cuga into lfx-bundles
Fifth batch (4 providers, 71 components) via consolidate_bundles.py --apply, unblocked by the prior dynamic-import test decoupling: - composio -> lfx_bundles/composio (composio + composio-langchain) 63 components - huggingface -> lfx_bundles/huggingface (langchain-huggingface[marker], huggingface-hub, community) 2 components - nvidia -> lfx_bundles/nvidia (langchain-nvidia-ai-endpoints, nv-ingest-client[py3.12], gassist[win32]) 5 components - cuga -> lfx_bundles/cuga (cuga; platform/py markers) 1 component Markers copied verbatim from langflow-base extras (uv lock --check clean). 284 migration entries (0 ambiguous); component index 27->23 categories, 212->141 components. Tests: relocate the dedicated lfx-isolated tests for the now-bundled components to the backend suite (test_nvidia_component.py, test_cuga_session_id_fallback.py) repointed to lfx_bundles.<provider>; repoint backend composio/huggingface/cuga imports + a cuga patch target; pragma a fake test api_key. 43 decoupled lfx import tests pass post-move. SKIP=detect-secrets: only flags are regenerated component_index code_hash hex (false positives). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| 5b8dd8a2cc |
feat(bundles): consolidate google, vertexai, altk, codeagents into lfx-bundles
Fourth directory-level batch (4 providers, 14 components) via consolidate_bundles.py. google/vertexai (langchain-google-*), altk (agent-lifecycle-toolkit), codeagents (smolagents + OpenDsStar); platform/py markers copied verbatim from langflow-base extras. 56 migration entries (0 ambiguous); index 31->27 categories, 226->212 components. Backend tests repointed; starter projects embedding moved components regenerated via the validator. SKIP=detect-secrets: only flags are regenerated code_hash hex (false positives). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| c10e40984d |
feat(bundles): consolidate homeassistant, olivya, agentql REST providers into lfx-bundles
Third batch (3 providers, 4 components) via consolidate_bundles.py --apply. These are thin REST wrappers with no dedicated vendor SDK: - homeassistant (requests) -> lfx_bundles/homeassistant 2 components - olivya (httpx, lfx core) -> lfx_bundles/olivya 1 component - agentql (httpx, lfx core) -> lfx_bundles/agentql 1 component Moves + shims + per-provider extras + `all` regen; 16 migration entries (0 ambiguous). Component index 34->31 categories, 230->226 components; uv.lock re-resolved. No tests reference these providers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| 11dacc45a7 |
feat(bundles): consolidate vector/datastore + community-wrapper providers into lfx-bundles
Second directory-level batch (8 providers, 12 components) via consolidate_bundles.py --apply: - baidu (qianfan + langchain-community), redis (redis + community), elastic (elasticsearch + langchain-elasticsearch + opensearch-py) - bing, cloudflare, maritalk, searchapi, vectara (langchain-community REST wrappers) Moves each dir into lfx-bundles + fail-soft shim, merges per-provider extras and regen `all`, appends 4-entry migration block per class (48 entries, 0 ambiguous). Component index regenerated (42->34 categories, 242->230 core components); uv.lock re-resolved. Test fixups: - Repoint baidu/elastic/searchapi backend tests to lfx_bundles.<provider>. - test_dynamic_imports (lfx isolated suite) used the now-bundled searchapi as its fixture; bundles aren't installed in the engine-only lfx env, so switch it to the core langchain_utilities.FakeEmbeddingsComponent (same langchain_community optional-import path). 131 shim tests + 21 dynamic-import tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| a082afbcda |
feat(bundles): consolidate cleanlab, twelvelabs, jigsawstack into lfx-bundles
First directory-level batch of the core-tail consolidation (validates the consolidate_bundles.py path end-to-end after the spider/toolguard file-level extractions). All three are flat, lfx-only, single-SDK providers: - cleanlab -> lfx_bundles/cleanlab (cleanlab-tlm) 3 components - twelvelabs -> lfx_bundles/twelvelabs (twelvelabs) 7 components - jigsawstack -> lfx_bundles/jigsawstack (jigsawstack) 11 components Via scripts/migrate/consolidate_bundles.py --apply: moves each dir into the metapackage + leaves a fail-soft shim, merges per-provider extras and regen `all`, appends the 4-entry migration block per class (84 entries, 0 ambiguous). Component index regenerated (45->42 categories, 263->242 core components); uv.lock re-resolved; twelvelabs test repointed to lfx_bundles.twelvelabs (passes). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
|||
| 7139d07033 |
feat(bundles): move 45 long-tail providers into lfx-bundles + graduate 5 partner packages (#13568)
* feat(bundles): add lfx-bundles metapackage skeleton
Creates the manifest-less lfx-bundles distribution (the langchain-community
model) that the long-tail providers will move into. Empty skeleton for now;
the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider
folders + per-provider extras later.
- src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles
entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a
generated (currently empty) `all` extra; plus a bare lfx_bundles namespace
package and a README documenting the model + install stories
- wired into the root workspace via the existing bundle marker blocks:
dep lfx-bundles[all]>=1.0,<2.0, uv source, and member
- hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds
it with zero workflow change
Verified: the wheel builds (entry_points.txt carries the lfx.bundles group);
load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty
skeleton registers zero providers with no error.
* feat(bundles): consolidate first long-tail tranche into lfx-bundles
Adds scripts/migrate/consolidate_bundles.py (the inverse of port_bundle.py --
moves in-tree providers into the manifest-less lfx-bundles metapackage) and
runs it on a verified 5-provider tranche: tavily, exa, wikipedia, yahoosearch,
wolframalpha.
Per provider the script:
- moves src/lfx/src/lfx/components/<p>/ -> lfx_bundles/<p>/ (lowercase names);
- leaves a fail-soft import shim (first line `# lfx-bundles-shim`) so
`from lfx.components.<p> import X` keeps working when lfx-bundles is
installed, and raises an actionable ImportError otherwise;
- merges the provider's third-party deps into a PEP 685-normalized lfx-bundles
extra and regenerates the `all` aggregate. Dep parity holds: `uv sync` is a
no-op because those deps were already pulled via langflow-base[complete];
- appends the 4-entry migration block per Component class (28 entries) so saved
flows referencing lfx.components.<p>.<Class> migrate to
ext:<p>:<Class>@official.
To avoid double registration, the in-tree component walk
(_load_components_dynamically) now skips shimmed provider dirs, and
component_index.json is regenerated (355->348 components, 95->90 modules); the
moved providers load only at @official via lfx.bundles discovery.
Verified: discovery finds all 5 at @official with no errors; shims resolve;
`import lfx.components` still works; the index drops the 7 moved component
entries (residual `tools`-category name refs resolve via the shim); ruff clean.
First tranche proves the engine; the remaining long-tail scales by extending
PROVIDER_DEPS (each provider's deps verified individually -- the careful part).
* feat(bundles): consolidate 30-provider tranche 2 into lfx-bundles
Extends PROVIDER_DEPS with 30 individually-verified providers and runs the
consolidation: vector stores (chroma, clickhouse, couchbase, milvus, mongodb,
pgvector, pinecone, qdrant, supabase, upstash, weaviate), model providers
(groq, mistral, ollama, perplexity, sambanova), and tools/memory/data (apify,
assemblyai, confluence, firecrawl, git, glean, icosacomputing, mem0, needle,
scrapegraph, serpapi, unstructured, youtube, zep).
Dep verification (the careful part): every spec comes from langflow-base's
per-provider extras or its direct dependencies; langchain-community providers
carry the wrapper plus the SDK the wrapper lazy-imports (e.g. pgvector,
atlassian-python-api); requests is declared explicitly where imported (it is
only transitive in today's env); pinecone keeps its python_version<'3.14'
marker verbatim. Tranche excludes: providers with langflow imports (vlmrun),
provider-specific lfx.base dirs (composio/huggingface/langwatch), case-
sensitive names (FAISS/Notion), the openai-SDK family (azure/aiml/deepseek/
litellm/lmstudio/novita/openrouter/vllm/xai/cometapi -- cleaner after PR-8),
and the partner set.
Dep parity verified at the resolution level: the uv.lock diff is +220 lines of
lfx-bundles extras metadata with ZERO packages added or removed (`name =` diff
empty), so pip install langflow resolves the identical set.
Also: 192 append-only migration entries (48 classes x 4, zero bare-name
ambiguities); component_index.json regenerated 348->300 components / 90->60
modules (exactly the moved set, no stale standalone entries); mongodb_atlas.py
SLF001 per-file-ignore and the mem0/mongodb detect-secrets baseline entries
migrated to the new paths (lint/secrets exceptions travel with moved files);
35 bundles now discover at @official with 55 components; shims verified across
categories; 449 extension tests pass.
* feat(bundles): consolidate openai-SDK family tranche 3 into lfx-bundles
10 providers that ride the langchain-openai wrapper, deferred from tranche 2
until the partner graduation settled the openai-SDK dep story: aiml, azure,
cometapi, deepseek, litellm, lmstudio, novita, openrouter, vllm, xai.
Dep verification: every provider declares langchain-openai>=1.1.6; the openai
SDK is declared only where a component imports it directly (aiml, deepseek,
litellm, lmstudio, vllm, xai -- wrapper-transitive elsewhere); requests
declared where imported (cometapi, deepseek, novita, xai); lmstudio's lazy
NVIDIAEmbeddings path gets langchain-nvidia-ai-endpoints~=1.0.0. The litellm
component drives LiteLLM-served endpoints through the OpenAI client and does
NOT import the litellm package -- langflow-base's litellm extra stays put.
These providers use the lazy _dynamic_imports __init__ shape; it survives the
move unchanged (import_mod resolves via __spec__.parent and
lfx.components._importing remains a core helper).
Dep parity: uv.lock diff has zero package additions/removals (all specs
already resolved via langflow-base[complete]).
Also: 56 append-only migration entries (14 classes x 4, zero ambiguities);
component_index.json regenerated 300->286 components / 60->50 modules (exactly
the moved set); detect-secrets baseline re-keyed; 45 bundles now discover at
@official with 69 components; shims verified (azure/xai/litellm/deepseek);
ruff clean.
* fix(ci): nightly bundle rename follows [extras] refs and self-refs
Two gaps in update_bundle_versions.py around extras suffixes, both fatal
or silently wrong for the first nightly carrying the lfx-bundles
metapackage:
- update_root_pyproject_for_bundle's dep regex required the version
specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0"
(a MAIN dep) was left unrewritten while the workspace member was
renamed to lfx-bundles-nightly; uv lock then tries to resolve stable
lfx-bundles from PyPI, where it does not exist. The same gap silently
left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the
stable distribution. The regex now tolerates an [extras] group and
carries it into the replacement.
- rename_bundle_pyproject skipped self-referencing extras, so the
metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]"
members after the rename, pulling the stable distribution (same
lfx_bundles import package, install collision) once published. Self
refs now follow the rename, idempotently.
Tests drive the real script module, mirroring test_bundle_lfx_pin.py.
This closes the PR-2 audit item flagged when the metapackage was
introduced. The canonical-pre-release cutover would retire the nightly
rename entirely; until it lands, the rename must be correct.
* feat(bundles): graduate partner set to standalone lfx-<provider> packages (#13573)
* feat(bundles): graduate partner set to standalone lfx-<provider> packages
Extracts the five partner/flagship providers into manifest-shipping
distributions: lfx-openai, lfx-anthropic, lfx-amazon, lfx-datastax,
lfx-cohere. Each ships extension.json (lfx.compat ["1"]), a
langflow.extensions entry-point, an lfx>=1.10.0,<2.0.0 pin, and is wired into
the root workspace marker blocks. Zero flow impact by the bundle-name
invariant: ext:<provider>:<Class>@official ids are unchanged.
Mechanics:
- adopts the five-phase scripts/migrate/port_bundle.py from
feat/bundle-mass-extraction (handles shared-base moves, consumer rewrites,
surgical index updates, migration-table appends) and runs it per partner
with --migration-release 1.11.0
- datastax's shared lfx.base.datastax moves into lfx_datastax.base; its
backend unit tests move to src/bundles/datastax/tests (74 pass); its ruff
per-file-ignore and its check_component_env_writes ALLOWLIST entry travel;
10 repo consumers rewritten (incl. the vector_store_rag starter project)
- amazon_bedrock_converse.py legacy langflow.* imports rewritten to the lfx.*
equivalents (thin re-export aliases; behavior-identical) pre-extraction
- runtime deps pinned from langflow-base's extras / direct deps (wrapper-
guaranteed SDKs stay transitive, parity-exact); --remove-base-extra dropped
the openai/anthropic/cohere/aws extras from langflow-base[complete] in favor
of the bundle pins; the astradb extra stays (also used outside datastax)
- in-tree dirs replaced with marker shims pointing at lfx_<p>.components.<p>
(skipped by the in-tree walk; legacy from lfx.components.<p> imports keep
working); lfx/components/__init__.py entries kept, consistent with the
consolidated providers
- validate.py: accept classes whose base is a *derived* Component base
(LCVectorStoreComponent / LCToolComponent / ...) -- they inherit the
class-level outputs declaration and only override the output method, which
the AST-only check could not see; bare Component subclasses keep the strict
build/outputs requirement (+ regression test)
- BUNDLE_API.md changelog entries added for the branch's surface changes: the
lfx.bundles manifest-less discovery group + precedence tier +
bundle-discovery-malformed code (from the foundation PR) and the validator
acceptance above
- 100 append-only migration entries (20 classes x 5 partners x 4 shapes);
component_index.json surgically updated 300->280 components / 60->55
modules, sha256 recomputed and verified
- detect-secrets baseline re-keyed for moved partner files
Dep parity: lock diff adds only the five lfx-* package names; no third-party
package added or removed. One benign transitive re-resolution: anthropic SDK
0.105.2 -> 0.109.0 (allowed by langchain-anthropic's range on both sides).
Verified: all five register at @official via the installed-manifest tier
(extension_id/distribution = lfx-<p>, 20 components); manifest-less
lfx_bundles unchanged (35 bundles, disjoint); all 5 shims import; engine-safe;
`lfx extension validate` passes for all five; 450 extension unit tests, 60
pilot-upgrade migration tests, 74 moved datastax tests pass; ruff clean.
* fix(bundles): shim lfx.base.datastax so stored-flow imports keep resolving
The datastax graduation moved lfx.base.datastax into lfx_datastax.base,
but saved flows and starter templates (Hybrid Search RAG,
TemplateAssistant) embed `from lfx.base.datastax.astradb_base import
AstraDBBaseComponent` inside their stored component code fields, which
is re-executed verbatim at flow build time. Without a shim that import
raises ModuleNotFoundError and the flows fail to build
(test-starter-projects red).
Mirror the lfx.components shim contract: module-aliasing to
lfx_datastax.base, narrow except that only translates a missing bundle
(transitive dep failures re-raise untouched), marker-tagged single-file
dir, removed at M4 together with the components shims.
* chore(bundles): bounded version ranges for curated lfx-* packages (#13576)
langflow's pyproject is the release-coordination point now that lfx and the
lfx-* bundles release independently. Every curated lfx-* dependency declares a
BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library
metadata create resolver conflicts for downstreams that depend on a different
version. Exact pins for reproducibility live in uv.lock and the release-build
manifests.
- the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles
follow their own 0.1.x cadence)
- lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage
contract)
- the three lfx-docling[...] optional-dependency refs bounded the same way
- policy documented inline in the bundle-deps marker block; the cross-bundle
CI matrix verifies the ranges resolve together across the supported lfx axis
Verified: uv lock resolves with zero package/version changes (bounds are
metadata-only today); the nightly rename's dep regex already matches the
bounded form.
* test(bundles): lock the in-tree shim contract + breakage audit (#13577)
* chore(bundles): bounded version ranges for curated lfx-* packages
langflow's pyproject is the release-coordination point now that lfx and the
lfx-* bundles release independently. Every curated lfx-* dependency declares a
BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library
metadata create resolver conflicts for downstreams that depend on a different
version. Exact pins for reproducibility live in uv.lock and the release-build
manifests.
- the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles
follow their own 0.1.x cadence)
- lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage
contract)
- the three lfx-docling[...] optional-dependency refs bounded the same way
- policy documented inline in the bundle-deps marker block; the cross-bundle
CI matrix verifies the ranges resolve together across the supported lfx axis
Verified: uv lock resolves with zero package/version changes (bounds are
metadata-only today); the nightly rename's dep regex already matches the
bounded form.
* test(bundles): lock the in-tree shim contract + breakage audit
Locks the five-point contract for the 50 lfx-bundles import shims under
lfx/components/ (45 metapackage + 5 graduated partners):
1. first line is the `# lfx-bundles-shim` marker (keys the in-tree walk's
double-registration skip);
2. a shim dir is exactly one __init__.py -- no impls, no deps;
3. sys.modules module-aliasing (submodule trees resolve);
4. bundle missing -> actionable ModuleNotFoundError whose wording is part
of the contract ("pip install lfx-bundles" / "pip install lfx-<p>");
5. a missing *transitive* dep re-raises untouched.
Test shape (env-adaptive by design -- the lfx test env prunes the venv, so
nothing here assumes bundles are installed):
- source-level sweep of every real shim, parameterized (no imports);
- hermetic mechanism tests against a synthetic shim + synthetic target
(alias-resolves / locked-message / transitive-reraise);
- the walk-skip detector and the marker sweep must agree exactly;
- one adaptive live test on lfx.components.tavily exercising whichever
branch the running env is in.
106 tests pass in the pruned lfx env (where the live test exercises the
bare-engine locked-message branch).
Breakage audit (gh code search, recorded in the PR): ~20-25 public repos
reference lfx.components.*; the majority are langflow forks (vendored tree,
unaffected by packaging). Genuine library consumers exist and the moved
providers have real surface (openai: 11 external repos, datastax: 17) --
covered by the shims wherever bundles are co-installed (every
`pip install langflow`). Decision: deprecation warnings NOT warranted now;
revisit at M4 (shim removal), where one loud minor release before removal
is the right shape.
* test(bundles): extend the shim contract sweep to lfx.base shims
The datastax graduation moved lfx.base.datastax into lfx_datastax.base
and left a module-aliasing shim behind (stored flow code fields embed
the legacy import and are re-executed verbatim at build time). Sweep
lfx/base for marker shims with the same source-level contract as the
components sweep: one-file stub, module-aliasing to the bundle's base
subpackage, narrow name-checked except, locked install message.
* docs(bundles): install shapes, override rule, bundle_api_version (#13578)
Documents the deliberately small user-facing rule surface of the metapackage
split (1.11):
- extensions-overview.mdx: the two install stories (pip install langflow =
everything, same as today; pip install lfx = engine only, bring your own
bundles); the bundle-name-is-identity invariant; the current package table
(lfx-bundles metapackage + the 5 partner packages + the 4 pilots); the
legacy-import shim behavior with the locked ModuleNotFoundError wording and
the migration recipe for direct lfx users (switch to lfx[bundles] or pin
lfx-<provider> packages); the override rule ("ship a manifest -- a manifest
always wins", with the bundle-shadowed warning); bundle_api_version as the
single compat number (lfx.compat ["1"]; manifest-less packages ride their
PEP 508 lfx pin) and the no-version-arithmetic rule.
- deployment-lfx-compatibility.mdx: lfx[bundles] as the headless/serverless
deployment footnote (engine + lfx-bundles[all]; slimmer per-provider
alternative; intentionally no lfx[all]).
Both files verified MDX-safe (no unbackticked angle brackets).
* fix(bundles): stack-review fixes — symlink containment, idempotency, docs accuracy (#13579)
* chore(bundles): bounded version ranges for curated lfx-* packages
langflow's pyproject is the release-coordination point now that lfx and the
lfx-* bundles release independently. Every curated lfx-* dependency declares a
BOUNDED range (>=A,<B), never an exact pin -- exact pins in reusable library
metadata create resolver conflicts for downstreams that depend on a different
version. Exact pins for reproducibility live in uv.lock and the release-build
manifests.
- the 4 pilots + 5 graduated partners: >=0.1.0 -> >=0.1.0,<1.0.0 (bundles
follow their own 0.1.x cadence)
- lfx-bundles[all]>=1.0,<2.0 unchanged (versions with the metapackage
contract)
- the three lfx-docling[...] optional-dependency refs bounded the same way
- policy documented inline in the bundle-deps marker block; the cross-bundle
CI matrix verifies the ranges resolve together across the supported lfx axis
Verified: uv lock resolves with zero package/version changes (bounds are
metadata-only today); the nightly rename's dep regex already matches the
bounded form.
* test(bundles): lock the in-tree shim contract + breakage audit
Locks the five-point contract for the 50 lfx-bundles import shims under
lfx/components/ (45 metapackage + 5 graduated partners):
1. first line is the `# lfx-bundles-shim` marker (keys the in-tree walk's
double-registration skip);
2. a shim dir is exactly one __init__.py -- no impls, no deps;
3. sys.modules module-aliasing (submodule trees resolve);
4. bundle missing -> actionable ModuleNotFoundError whose wording is part
of the contract ("pip install lfx-bundles" / "pip install lfx-<p>");
5. a missing *transitive* dep re-raises untouched.
Test shape (env-adaptive by design -- the lfx test env prunes the venv, so
nothing here assumes bundles are installed):
- source-level sweep of every real shim, parameterized (no imports);
- hermetic mechanism tests against a synthetic shim + synthetic target
(alias-resolves / locked-message / transitive-reraise);
- the walk-skip detector and the marker sweep must agree exactly;
- one adaptive live test on lfx.components.tavily exercising whichever
branch the running env is in.
106 tests pass in the pruned lfx env (where the live test exercises the
bare-engine locked-message branch).
Breakage audit (gh code search, recorded in the PR): ~20-25 public repos
reference lfx.components.*; the majority are langflow forks (vendored tree,
unaffected by packaging). Genuine library consumers exist and the moved
providers have real surface (openai: 11 external repos, datastax: 17) --
covered by the shims wherever bundles are co-installed (every
`pip install langflow`). Decision: deprecation warnings NOT warranted now;
revisit at M4 (shim removal), where one loud minor release before removal
is the right shape.
* docs(bundles): install shapes, override rule, bundle_api_version
Documents the deliberately small user-facing rule surface of the metapackage
split (1.11):
- extensions-overview.mdx: the two install stories (pip install langflow =
everything, same as today; pip install lfx = engine only, bring your own
bundles); the bundle-name-is-identity invariant; the current package table
(lfx-bundles metapackage + the 5 partner packages + the 4 pilots); the
legacy-import shim behavior with the locked ModuleNotFoundError wording and
the migration recipe for direct lfx users (switch to lfx[bundles] or pin
lfx-<provider> packages); the override rule ("ship a manifest -- a manifest
always wins", with the bundle-shadowed warning); bundle_api_version as the
single compat number (lfx.compat ["1"]; manifest-less packages ride their
PEP 508 lfx pin) and the no-version-arithmetic rule.
- deployment-lfx-compatibility.mdx: lfx[bundles] as the headless/serverless
deployment footnote (engine + lfx-bundles[all]; slimmer per-provider
alternative; intentionally no lfx[all]).
Both files verified MDX-safe (no unbackticked angle brackets).
* fix(bundles): review fixes — symlink containment, idempotency, docs accuracy
Fixes from the multi-agent stack review (kept as a separate PR so the
reviewed PRs' diffs stay frozen for QA's independent pass):
- _bundles_root.py: directory-level symlink containment — a provider dir
that resolves outside the bundle root is skipped with a typed
bundle-discovery-malformed warning, mirroring the seed-directory walk's
is_within rule (+ regression test). Anchors the trust boundary to the
installed package tree.
- consolidate_bundles.py: migration-append idempotency guard — entries are
deduped on full content so a partially-failed earlier run cannot duplicate
rows on re-run (table stays append-only).
- langflow-base pyproject: documented WHY the aws extra is deliberately
retained after the lfx-amazon graduation (boto3 also backs the server's
own S3 storage backend and lfx's S3 ingestion — review suggested removing
it; that would regress S3 storage support).
- extensions-overview.mdx: note that `lfx extension list` shows
manifest-shipping extensions only; manifest-less packages load at startup
but are not listed.
93 loader+migration tests pass (incl. the new symlink test); ruff clean.
* refactor(bundles): stabilize import_mod as a public BUNDLE_API utility
The lazy-import helper that bundle packages call from their
__getattr__-based __init__.py files lived at lfx.components._importing -- an
internal path with no stability contract, imported by 35 separately-installed
bundle __init__ files (30 lfx-bundles providers + the 5 graduated partners).
- canonical home is now lfx.utils.lazy_import.import_mod (code unchanged);
lfx.components._importing re-exports it so in-tree callers and any external
code on the old path keep working
- all 35 bundle-side imports rewritten to the stable path
- BUNDLE_API.md: surface table entry + changelog (additive)
- contract tests: re-export identity, both call forms, the AttributeError
conversion
Verified: identity holds across both paths; lazy __init__ loads work through
the new path for both bundle families; 110 tests pass; ruff clean.
* fix(bundles): tombstone the broken legacy ZepChatMemory build method (#13580)
build_message_history targeted the zep-python v1 SDK (ZepClient +
zep_python.langchain.ZepChatMessageHistory); both were removed in
zep-python 2.x and the zep extra pins 2.0.2, so the method has been
unable to run for as long as the pin has existed -- its ImportError
guard misleadingly told users to 'pip install zep-python' (already
installed). The component is legacy=True with helpers.Memory as its
designated replacement, so rather than hand-write a new integration
against the 2.x SDK, the method now raises a clear RuntimeError
pointing at the Message History component.
Flow identity is preserved: class/component name, display_name,
description, inputs and the memory output are byte-identical, so saved
flows keep loading, i18n locale keys are unchanged, and
migration_table.json needs no edits. New bundle tests pin the stub
contract (identity, actionable error, no zep_python import).
* test(bundles): extend the shim contract sweep to lfx.base shims
The datastax graduation moved lfx.base.datastax into lfx_datastax.base
and left a module-aliasing shim behind (stored flow code fields embed
the legacy import and are re-executed verbatim at build time). Sweep
lfx/base for marker shims with the same source-level contract as the
components sweep: one-file stub, module-aliasing to the bundle's base
subpackage, narrow name-checked except, locked install message.
* test(bundles): extras-drift guard for the lfx-bundles metapackage
Risk-2 of the metapackage split: the generated `all` extra and the
per-provider extras must never drift by hand-edit, or `pip install
langflow` silently loses a provider's deps. Guard the four invariants:
extras <-> provider dirs (PEP 685-normalized), `all` == the exact
self-ref set, normalized keys collision-free, and the metapackage
provider set disjoint from the graduated partner distributions.
* fix(extension): symlink-escaped providers reject with path-escape, matching the documented contract
The lfx.bundles provider containment check (stack-review symlink fix)
emitted bundle-discovery-malformed, whose template and hint describe a
broken entry-point declaration. The changelog's path-safety entry
already documents symlink escapes as path-escape on every other
discovery path; use the same code here. Also resolves the semantic
merge conflict with the per-mode code split (the removed
_malformed_error location kwarg).
* fix(tests): repoint lfx tests off moved providers; complete provider fallback map
CI fail-fast had been masking these: the LFX test job runs in an
engine-only env where openai/anthropic/chroma are now bundle-package
shims, so every test that used them as the example category failed on
all Python versions (3.14 just reported first), and the backend Group 2
leg failed collecting test_lfx_bundles_extras.py on Python 3.10.
- flow_requirements: complete _PROVIDER_PACKAGE_FALLBACKS for the nine
moved model providers (OpenAI, Anthropic, Amazon Bedrock, Groq,
Google Generative AI, SambaNova, IBM watsonx.ai, Ollama + existing
Azure OpenAI). In engine-only installs MODEL_PROVIDERS_DICT registers
only in-tree providers, so the dynamic source-inspection path cannot
resolve moved providers; the static fallbacks keep lfx run/serve
requirements inference working. A dict hit still wins.
- test_dynamic_imports / test_import_utils: use composio (still-in-tree
lazy category with its SDK absent in the bare env) as the example
category instead of openai/anthropic/chroma; patch import_module at
its canonical home lfx.utils.lazy_import.
- flow-builder tests (build_flow_from_spec, flow_builder_tools,
propose_field_edit): specs use LanguageModelComponent (in-tree)
instead of OpenAIModel.
- test_lfx_bundles_extras: tomli fallback for Python 3.10 (tomllib is
stdlib 3.11+; pytest guarantees tomli on <3.11).
Full lfx unit suite: 4550 passed. Backend extras+pin tests: 24 passed.
* fix(tests): repoint MCP client-server tests off graduated OpenAIModel key
Same fail-fast-masked class as
|
|||
| ee659ca38c |
fix(bundles): floor lfx pin at the minor line's .dev0 so nightlies resolve
The first 1.11.0.dev0 nightly failed its "Test Langflow Main CLI" step: the fork bump's sync_bundle_lfx_pin.py re-synced every bundle floor to lfx>=1.11.0, and PEP 440 sorts the nightly's 1.11.0.dev0 BELOW 1.11.0, so the workspace-built bundles (whose metadata shadows the satisfiable PyPI lfx-arxiv 0.1.1 during `uv pip install dist/*.whl`) reject the branch's own lfx while langflow-base pins it exactly — unresolvable. Floor at lfx>=X.Y.0.dev0,<(X+1).0.0 instead: X.Y.0.dev0 is the lowest version PEP 440 admits in the minor line, so every devN / rcN / final satisfies the floor while older lines and the next major stay excluded. This closes the NIGHTLY.md activation-gate hole structurally — future minor forks re-sync to a floor their own nightlies already satisfy. - sync_bundle_lfx_pin.py: lfx_floor_spec emits the .dev0 floor - port_bundle.py: mirrored _current_lfx_floor kept in step - bundle pyprojects restamped via the script (arxiv/docling/duckduckgo/ibm) - test_bundle_lfx_pin.py expectations updated (20/20 passing) - NIGHTLY.md gate section annotated with the post-activation fix uv.lock is unaffected (workspace lfx is recorded as an editable source with no specifier — which is also why uv sync/lock passed in the same job). The release.yml RC floor-relax sed still matches the new form; now redundant but harmless. No bundle version bump needed: published 0.1.1 floors >=1.10.0.rc0, satisfiable by the whole 1.11 line. |
|||
| db8935c233 |
fix(bundles): sync lfx pin to the 1.10.0 line after version realignment (#13516)
The LFX 0.5.0 -> 1.10.0 realignment (#13176) left every src/bundles/* package and the port_bundle.py generator flooring lfx>=0.5.0. That silently permits resolving against the now-dead 0.5.x line: a bundle built against 1.10.0's BUNDLE_API would ImportError there, and the BUNDLE_API_VERSION check (both "1") would not catch it. RELEASE.md flags exactly this — the jump "affects downstream pins, and neither pip nor uv will flag it." - Bump the 4 bundle pyprojects (arxiv, docling, duckduckgo, ibm) to "lfx>=1.10.0,<2.0.0": floored at the current major.minor line, capped below the next lfx major. Fine-grained API compat stays enforced via extension.json's lfx.compat against BUNDLE_API_VERSION, not the cap. - Add scripts/ci/sync_bundle_lfx_pin.py and call it from `make patch` so future releases keep bundle floors in step. Idempotent: a no-op on patch releases, moves the floor on minor/major bumps. Leaves docling self-refs and the nightly lfx-nightly== form untouched. - port_bundle.py now derives the floor from src/lfx/pyproject.toml, so newly-ported bundles are born at the current line. - Update src/bundles/PORTING.md and add test_bundle_lfx_pin.py (20 tests). |
|||
| be7edcbda1 | chore: unconstrained lfx version for bundles (#13415) | |||
| cc009c9133 |
feat(lfx): Bundle Separation and LFX Extension Framework (#13043)
* feat(lfx): installed-package + seed-directory discovery for production install (LE-1022) Adds the read-only production install path for Modes A, B, and C of the Bundle Separation iteration. Manifest-shipping pip-installed distributions and seed-directory subdirectories are discovered at server startup and registered as Extensions at @official. * discovery.py: walks importlib.metadata.distributions() + the $LANGFLOW_SEED_DIR / /opt/langflow/bundles seed root; produces DiscoveredExtension records and typed errors for malformed manifests / configured-but-missing seed dirs. * registry.py: ExtensionRegistry service with the immutability invariant for installed and seed entries. Mutation verbs (uninstall, disable, enable, install, update_entry) all raise ExtensionImmutableError carrying the typed installed-extension-immutable / seed-directory-immutable code so the invariant is testable today; the CLI uninstall surface ships in B4. * lfx extension list: read-only inspector with text and JSON output for operators inspecting Mode B/C images. * Errors: four new typed codes (installed-extension-immutable, seed-directory-immutable, seed-directory-not-found, duplicate-extension-id) plus snapshot coverage in tests/unit/extension/test_errors.py. * Tests: 165 extension tests pass, including the LE-1022 acceptance cases -- three pip-installed wheels visible at @official, three seed bundles visible at @official, and the parametrized service-layer immutability check across every mutation verb. * Docs: docs/Deployment/deployment-extensions-production.mdx covers the Dockerfile template, k8s deployment notes, the bundle packaging convention (extension.json shipped via package-data), and troubleshooting for the typed error codes. * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (#12967) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * refactor(lfx): clarify broad except, log malformed bundle.json, expand test docstring - Document why _discovery.import_bundle_module catches BaseException (startup-time loader, must surface bad bundles as typed errors rather than abort). - Add debug-level logging when bundle.json is malformed or non-object so a stale-cache footgun is at least observable. - Expand test_skips_re_imported_class docstring so future maintainers don't accidentally weaken the __module__-equality guard if package-style relative imports get added later. Also drops the stale duplicate-distribution claim from the PR description; that code is correctly deferred to LE-1022 along with /all integration. * feat(lfx): wire Extension System into /all, pathsep-split LANGFLOW_COMPONENTS_PATH, emit duplicate-distribution Closes the four AC gaps the previous reviewer flagged on PR #12967: 1. /all integration: get_and_cache_all_types_dict now also calls a new import_extension_components() that loads installed Extensions via load_installed_extensions, loads inline bundles via discover_inline_bundles, and builds frontend-node templates with extension/bundle/extension_version fields stamped on. Failures are logged and skipped per bundle. 2. LANGFLOW_COMPONENTS_PATH is now split on os.pathsep so multi-entry env vars (e.g. /a:/b on POSIX) produce multiple components-path entries instead of one literal non-existent path. Empty segments and missing paths are skipped. 3. duplicate-distribution is a real producer: load_installed_extensions surfaces a typed warning on the winner LoadResult when two distributions share a canonical name, naming every involved manifest path. 4. Manifest-first precedence runtime wiring: new filter_component_entry_points loads each entry-point and only skips ones that resolve to a Component subclass on a manifest-shipping distribution. plugin_routes.load_plugin_routes now applies it so non-component entry-points (route registrars) keep loading per the AC's 'unaffected' promise. Added the previously-missing AC test for same-distribution component+non-component partition. Also makes _distribution_canonical_name defensive against MagicMock test seams. * fix(lfx): register extension components under namespaced ID; promote duplicate-distribution to error Addresses the latest review of PR #12967: P1: /all integration now keys the cache inner dict by LoadedComponent.namespaced_id (ext:<bundle>:<Class>@<slot>) rather than the bare class name. Templates also carry the namespaced_id as an explicit field so consumers that look at the value (not the key) still see the canonical address. This is the form the LE-1020 migration table will rewrite legacy class-name references to. P2: load_installed_extensions now appends duplicate-distribution to result.errors instead of result.warnings, so LoadResult.ok=False when two distributions share a canonical name. The winner's components still appear in result.components so flows already pinned to them keep working; only the conflict status changes. Updated test to assert errors + ok=False; added explicit assertion that the winner's components are still present. * fix(lfx): installed-distribution discovery accepts pyproject.toml manifest form Closes the latest review finding on PR #12967: the installed-distribution scan only looked for extension.json, ignoring distributions whose manifest lives in [tool.langflow.extension] inside pyproject.toml. The AC explicitly treats both as valid manifest forms. _distribution_manifest_path now: - Returns extension.json immediately when present (preserves precedence matching load_manifest's discovery order). - Falls back to pyproject.toml only when extension.json is absent AND the pyproject's [tool.langflow.extension] section is parseable. Validation reuses load_manifest itself so the rule lives in exactly one place; a stray pyproject.toml without the section is correctly ignored. Tests cover: pyproject-only discovery, pyproject-without-section ignored, extension.json wins on collision, end-to-end pyproject load at @official, and pyproject-form manifest-first entry-point suppression. * refactor(lfx): tighten loader invariants, surface silent skips, harden tests Addresses the latest review feedback on PR #12967: Type-level invariants (_types.py): - LoadedComponent.__post_init__ enforces that @extra components must NOT carry a distribution. The reverse (@official without distribution) is permitted because load_extension is also used for dev-mode loads against a working tree before pip install. - LoadResult docstring documents the partial-success contract: components may be non-empty when errors is non-empty (some files imported, others failed). Callers branching on ok get strict success. Silent-failure fixes (_orchestrator.py + settings/base.py): - inline-path-missing: a non-existent / non-dir LANGFLOW_COMPONENTS_PATH entry now produces a typed warning per skipped path so a typo no longer yields zero diagnostics. Settings-layer skip bumped from debug to warning for the same reason. - bundle-json-invalid: a malformed or non-object bundle.json now surfaces a typed warning instead of silently rewriting the user-declared id/version to derived values under the same bundle name. - no-component-subclass gating uses a call-local counter instead of result.errors so the diagnostic stays accurate when a future caller reuses a LoadResult (multi-bundle / batch wrapper scenarios). Test hardening: - test_re_imported_class_is_skipped_via_module_filter rewritten to actually exercise the __module__-equality check via sys.modules injection; previously passed via module-import-failed (relative-import failure), which would silently weaken if package registration changes. - test_user_declared_path_order_is_preserved: AC #8's multi-path order case (distinct bundles in [path_b, path_a]) was unasserted; added. - test_inline_module_import_failure_attributes_identity: AC #10's identity-on-partial-failure was covered for @official but not @extra; added. - test_uses_real_distributions_by_default tightened to assert ep placement (in kept, not in skipped) instead of exact-list equality, so a future Langflow-shipped manifest doesn't silently flip the assertion. bumped 64 -> 175 passing extension tests; 20 backend integration tests still pass. * fix(lfx): malformed pyproject manifests surface manifest-invalid instead of disappearing Closes the latest review finding on PR #12967: a pyproject.toml with a [tool.langflow.extension] section that has missing/invalid required fields was silently dropped because _pyproject_has_extension_section ran full schema validation via load_manifest and returned False on ValueError/TypeError. That conflated 'no section' with 'section malformed'. Fix: detect section presence only. _pyproject_has_extension_section now calls _read_pyproject_extension (TOML parse + key lookup, no schema check). Behavior: - Section absent or pyproject TOML unparseable -> False (treat as regular non-manifest package). - Section present and is a table (valid OR schema-invalid) -> True. - Section present but is not a table -> True; the author intended to declare an extension and load_extension will surface the typed error. This way a typo'd pyproject Extension produces a typed manifest-invalid LoadResult with extension_id attribution, and manifest-first precedence still suppresses its legacy component entry-points -- matching the 'typed load results on success/failure' contract for the supported pyproject manifest form. Tests: two new cases pin the behavior. test_malformed_pyproject_section_ surfaces_manifest_invalid asserts a typed load-failure result with distribution attribution; the second test pins manifest-first suppression for malformed pyproject distributions. * refactor(lfx): emit inline-path-unreadable, document reload contract, trim rot Closes the remaining nits on PR #12967: - inline-path-unreadable (new typed error code): a configured LANGFLOW_COMPONENTS_PATH entry that raises OSError on iterdir (typically permission-denied) now produces a typed LoadResult error carrying str(exc) instead of silently swallowing the message. - duplicate-inline-bundle hint trimmed: removed forward promise about hard-error-in-a-later-release; same actionability, no expiration. - discover_inline_bundles docstring trimmed from three paragraphs to two sentences (per CLAUDE.md anti-multi-paragraph rule). - _distribution_manifest_path docstring trimmed to one-liner. - installed_extension_roots dropped Used-by caller-narration list; manifest_owning_distributions docstring rewritten to call out the shadow-load risk for direct callers and point to load_installed_ extensions for the typed warning surface. - _discovery.import_bundle_module: replaced 'until that lands in a later milestone' rot with a single line ('Absolute imports only between bundle modules; relative imports unsupported.') AND added the single-load-per-process contract note documenting why LE-1018 reload must scrub registry/sys.modules before re-invoking the loader. - load_extension docstring grew a 'Single-load-per-process contract' block telling direct callers not to rely on this function for refresh. Tests: new test_unreadable_path_emits_inline_path_unreadable pins the OSError -> typed-error path. --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) (#12979) * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) Five-stage reload (parallel staging load -> validate -> swap under write lock -> cleanup -> emit) for installed Bundles in Mode A. In-flight flows keep the pre-swap class via existing references; new flows pick up the post-swap class atomically; concurrent reloads on the same Bundle are rejected with reload-in-progress. Adds: * lfx/extension/registry.py -- BundleRegistry with per-bundle reload-in-progress guard and components_index.json writer * lfx/extension/reload.py -- the five-stage pipeline; events emission is stubbed (TODO LE-1017) so the swap mechanics can ship before the events service lands * loader: optional module_namespace param so Stage 1 lands in __reload_staging__.<id> instead of the live _lfx_ext.* namespace * errors: four new typed reload codes (reload-in-progress, reload-bundle-not-installed, reload-bundle-name-mismatch, reload-source-missing) with branch templates and snapshot tests * HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by the existing get_current_active_user dependency, returns 409 with a typed body for the in-progress collision case * CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client against the dev server with text/json output and proper exit codes (--all is gated until LE-1019 lands the list endpoint) * tests: 16 reload-pipeline tests covering the AC matrix (rename round-trip, broken-bundle isolation, concurrent readers, in-flight flow, double-reload guard, bundle-name mismatch) plus 9 CLI client tests Mode A only. In Mode B/C bundle changes require a Docker image rebuild and the reload path is not exercised. The events emission in Stage 5 is intentionally stubbed -- the LE-1017 ticket will swap the body of _emit_bundle_reload_event in one place without touching the pipeline core. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [autofix.ci] apply automated fixes * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) (#12968) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) The two scaffolding CLIs an Extension author types: - `lfx extension init <target>` writes the basic single-Bundle template (manifest with $schema, README, .gitignore, one Component subclass + a pytest smoke test). AC #1: the generated extension validates clean against LE-1014. AC #2: the generated test file is a valid pytest module that exercises the component's build() method. AC #3: any --template other than 'basic' fails with a typed template-deferred-in-this-milestone error and a non-zero exit. Refuses to scaffold over a non-empty target dir. - `lfx extension dev <target>` validates the local extension, records its absolute path in <config_dir>/extensions/dev_extensions.json, prints reload instructions, and execs `langflow run` (or `python -m langflow` when langflow isn't on PATH). --skip-launch registers without launching (used by tests + external dev-server scripts); --skip-validate lets authors register a known-broken manifest to debug it under the loader. Stack: - Wave 0: error codes (extension-target-exists, extension-target-invalid, local-extension-missing) with format branches and snapshot tests. - Wave 1: lfx.extension.init_template -- pure-data scaffolder, no Typer dependency so the CLI is a thin shell over it. - Wave 1: lfx.extension.dev_registry -- atomic JSON state file under the langflow user-cache dir; helpers for register / list / unregister / load_dev_extensions / dev_extension_component_paths. - Wave 2: lfx.cli._extension_commands gains init/dev subcommands. - Wave 2: langflow.main lifespan hook reads the dev registry after bundle loading and extends components_path with each registered bundle dir, so the existing palette discovery picks up dev extensions. Missing paths surface as local-extension-missing warnings (AC #5) without aborting startup. Tests (84 new, 197 total in tests/unit/extension/): - test_init_template.py: AC scenarios + identifier derivation + deterministic file shape. - test_dev_registry.py: register/list/unregister round-trip, idempotent re-register refreshes timestamp, malformed state file treated as empty, missing-path warning, recovery when path reappears, env-var override precedence. - test_cli.py: AC #1 init->validate, AC #3 deferred templates, --skip-launch registers without launching, --skip-validate short-circuits the pre-flight pass. - test_errors.py: snapshot rows for all three new codes; the every-known-code-has-a-snapshot test enforces parity. LE-1018 (reload) reuses load_dev_extensions; LE-1022 (installed-pkg discovery) shares the components_path extension pattern. AC #4 ("boots Langflow with the extension visible in the palette within 5s") is delivered jointly by `extension dev` (registers + execs) and the lifespan hook (loads on startup). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1016 review feedback Fixes the four HIGH issues + the elevated LOW from the review pass: 1. dev_extension_component_paths now forwards EVERY warning, not just local-extension-missing. Previously a duplicate-component-name (or any future warning code) was silently dropped, hiding real signal from the lifespan hook's logs. 2. Defensive emit when a LoadResult has components but source_path is None. The current loader always sets source_path, but a future hand-built LoadResult could violate that contract; we now surface a typed local-extension-missing error rather than dropping the extension silently. 3. Replaced the fragile ``min(len(parts))`` bundle-root selection with a relative-to-source-path measurement. Handles deep-vs-shallow sibling extensions correctly without depending on absolute path depth. 4. Generated README now documents the langflow/lfx prerequisite under the Develop section so authors know `pytest` requires the lfx environment, not just Python. 5. Forced LANGFLOW_LAZY_LOAD_COMPONENTS=false unconditionally in the `extension dev` exec env (was setdefault, which let a developer's global lazy-loading export silently hide their dev components from the palette and miss AC #4's 5s budget). Plus three MEDIUMs: - sys.modules cleanup in test_generated_test_file_runs_against_generated_component so a later test importing the same dotted path doesn't pick up a stale module from a deleted tmp_path. Component instantiation moved inside the try block because Component.__init__ uses inspect.getsourcefile against self.__class__'s still-live module. - Added regex-drift test that pins the init_template patterns to match manifest.py's so a schema regex change can't quietly produce invalid scaffolded manifests. - Added two new dev_registry tests covering the forward-all-warnings contract and the source_path=None defense. Tests: 201 passing (4 new); ruff check + format clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * fix(lfx): align init template with renamed manifest API After merging feat/extension-loader into this branch, two surfaces fell out of sync with the renamed manifest schema: - init_template generates extension.json with `lfx: {bundle_api: [1]}`, but the validator now requires `lfx: {compat: ["1"]}` per LfxCompat. This made `test_basic_template_validates_clean` fail with manifest-invalid (`lfx.compat: Field required; lfx.bundle_api: Extra inputs are not permitted`). - test_init_template_regexes_match_manifest_schema reads `manifest_mod._BUNDLE_NAME_RE`, but that symbol was promoted to public `BUNDLE_NAME_RE` in |
