11 Commits

Author	SHA1	Message	Date
vjgit96	3e57126b91	fix: remove duplicate uv.lock from workspace member (#13326) ... * fix: remove src/backend/base/uv.lock and Dockerfile references - Deleted src/backend/base/uv.lock (monorepo should have only one uv.lock at root) - Removed COPY ./src/backend/base/uv.lock lines from all Dockerfiles: - docker/build_and_push.Dockerfile - docker/build_and_push_base.Dockerfile - docker/build_and_push_ep.Dockerfile - docker/build_and_push_with_extras.Dockerfile - docker/dev.Dockerfile Fixes LE-1093 * fix: remove stale base uv.lock regeneration paths	2026-05-30 13:05:34 +00:00
Eric Hare	cc009c9133	feat(lfx): Bundle Separation and LFX Extension Framework (#13043) ... * feat(lfx): installed-package + seed-directory discovery for production install (LE-1022) Adds the read-only production install path for Modes A, B, and C of the Bundle Separation iteration. Manifest-shipping pip-installed distributions and seed-directory subdirectories are discovered at server startup and registered as Extensions at @official. * discovery.py: walks importlib.metadata.distributions() + the $LANGFLOW_SEED_DIR / /opt/langflow/bundles seed root; produces DiscoveredExtension records and typed errors for malformed manifests / configured-but-missing seed dirs. * registry.py: ExtensionRegistry service with the immutability invariant for installed and seed entries. Mutation verbs (uninstall, disable, enable, install, update_entry) all raise ExtensionImmutableError carrying the typed installed-extension-immutable / seed-directory-immutable code so the invariant is testable today; the CLI uninstall surface ships in B4. * lfx extension list: read-only inspector with text and JSON output for operators inspecting Mode B/C images. * Errors: four new typed codes (installed-extension-immutable, seed-directory-immutable, seed-directory-not-found, duplicate-extension-id) plus snapshot coverage in tests/unit/extension/test_errors.py. * Tests: 165 extension tests pass, including the LE-1022 acceptance cases -- three pip-installed wheels visible at @official, three seed bundles visible at @official, and the parametrized service-layer immutability check across every mutation verb. * Docs: docs/Deployment/deployment-extensions-production.mdx covers the Dockerfile template, k8s deployment notes, the bundle packaging convention (extension.json shipped via package-data), and troubleshooting for the typed error codes. * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (#12967) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * refactor(lfx): clarify broad except, log malformed bundle.json, expand test docstring - Document why _discovery.import_bundle_module catches BaseException (startup-time loader, must surface bad bundles as typed errors rather than abort). - Add debug-level logging when bundle.json is malformed or non-object so a stale-cache footgun is at least observable. - Expand test_skips_re_imported_class docstring so future maintainers don't accidentally weaken the __module__-equality guard if package-style relative imports get added later. Also drops the stale duplicate-distribution claim from the PR description; that code is correctly deferred to LE-1022 along with /all integration. * feat(lfx): wire Extension System into /all, pathsep-split LANGFLOW_COMPONENTS_PATH, emit duplicate-distribution Closes the four AC gaps the previous reviewer flagged on PR #12967: 1. /all integration: get_and_cache_all_types_dict now also calls a new import_extension_components() that loads installed Extensions via load_installed_extensions, loads inline bundles via discover_inline_bundles, and builds frontend-node templates with extension/bundle/extension_version fields stamped on. Failures are logged and skipped per bundle. 2. LANGFLOW_COMPONENTS_PATH is now split on os.pathsep so multi-entry env vars (e.g. /a:/b on POSIX) produce multiple components-path entries instead of one literal non-existent path. Empty segments and missing paths are skipped. 3. duplicate-distribution is a real producer: load_installed_extensions surfaces a typed warning on the winner LoadResult when two distributions share a canonical name, naming every involved manifest path. 4. Manifest-first precedence runtime wiring: new filter_component_entry_points loads each entry-point and only skips ones that resolve to a Component subclass on a manifest-shipping distribution. plugin_routes.load_plugin_routes now applies it so non-component entry-points (route registrars) keep loading per the AC's 'unaffected' promise. Added the previously-missing AC test for same-distribution component+non-component partition. Also makes _distribution_canonical_name defensive against MagicMock test seams. * fix(lfx): register extension components under namespaced ID; promote duplicate-distribution to error Addresses the latest review of PR #12967: P1: /all integration now keys the cache inner dict by LoadedComponent.namespaced_id (ext:<bundle>:<Class>@<slot>) rather than the bare class name. Templates also carry the namespaced_id as an explicit field so consumers that look at the value (not the key) still see the canonical address. This is the form the LE-1020 migration table will rewrite legacy class-name references to. P2: load_installed_extensions now appends duplicate-distribution to result.errors instead of result.warnings, so LoadResult.ok=False when two distributions share a canonical name. The winner's components still appear in result.components so flows already pinned to them keep working; only the conflict status changes. Updated test to assert errors + ok=False; added explicit assertion that the winner's components are still present. * fix(lfx): installed-distribution discovery accepts pyproject.toml manifest form Closes the latest review finding on PR #12967: the installed-distribution scan only looked for extension.json, ignoring distributions whose manifest lives in [tool.langflow.extension] inside pyproject.toml. The AC explicitly treats both as valid manifest forms. _distribution_manifest_path now: - Returns extension.json immediately when present (preserves precedence matching load_manifest's discovery order). - Falls back to pyproject.toml only when extension.json is absent AND the pyproject's [tool.langflow.extension] section is parseable. Validation reuses load_manifest itself so the rule lives in exactly one place; a stray pyproject.toml without the section is correctly ignored. Tests cover: pyproject-only discovery, pyproject-without-section ignored, extension.json wins on collision, end-to-end pyproject load at @official, and pyproject-form manifest-first entry-point suppression. * refactor(lfx): tighten loader invariants, surface silent skips, harden tests Addresses the latest review feedback on PR #12967: Type-level invariants (_types.py): - LoadedComponent.__post_init__ enforces that @extra components must NOT carry a distribution. The reverse (@official without distribution) is permitted because load_extension is also used for dev-mode loads against a working tree before pip install. - LoadResult docstring documents the partial-success contract: components may be non-empty when errors is non-empty (some files imported, others failed). Callers branching on ok get strict success. Silent-failure fixes (_orchestrator.py + settings/base.py): - inline-path-missing: a non-existent / non-dir LANGFLOW_COMPONENTS_PATH entry now produces a typed warning per skipped path so a typo no longer yields zero diagnostics. Settings-layer skip bumped from debug to warning for the same reason. - bundle-json-invalid: a malformed or non-object bundle.json now surfaces a typed warning instead of silently rewriting the user-declared id/version to derived values under the same bundle name. - no-component-subclass gating uses a call-local counter instead of result.errors so the diagnostic stays accurate when a future caller reuses a LoadResult (multi-bundle / batch wrapper scenarios). Test hardening: - test_re_imported_class_is_skipped_via_module_filter rewritten to actually exercise the __module__-equality check via sys.modules injection; previously passed via module-import-failed (relative-import failure), which would silently weaken if package registration changes. - test_user_declared_path_order_is_preserved: AC #8's multi-path order case (distinct bundles in [path_b, path_a]) was unasserted; added. - test_inline_module_import_failure_attributes_identity: AC #10's identity-on-partial-failure was covered for @official but not @extra; added. - test_uses_real_distributions_by_default tightened to assert ep placement (in kept, not in skipped) instead of exact-list equality, so a future Langflow-shipped manifest doesn't silently flip the assertion. bumped 64 -> 175 passing extension tests; 20 backend integration tests still pass. * fix(lfx): malformed pyproject manifests surface manifest-invalid instead of disappearing Closes the latest review finding on PR #12967: a pyproject.toml with a [tool.langflow.extension] section that has missing/invalid required fields was silently dropped because _pyproject_has_extension_section ran full schema validation via load_manifest and returned False on ValueError/TypeError. That conflated 'no section' with 'section malformed'. Fix: detect section presence only. _pyproject_has_extension_section now calls _read_pyproject_extension (TOML parse + key lookup, no schema check). Behavior: - Section absent or pyproject TOML unparseable -> False (treat as regular non-manifest package). - Section present and is a table (valid OR schema-invalid) -> True. - Section present but is not a table -> True; the author intended to declare an extension and load_extension will surface the typed error. This way a typo'd pyproject Extension produces a typed manifest-invalid LoadResult with extension_id attribution, and manifest-first precedence still suppresses its legacy component entry-points -- matching the 'typed load results on success/failure' contract for the supported pyproject manifest form. Tests: two new cases pin the behavior. test_malformed_pyproject_section_ surfaces_manifest_invalid asserts a typed load-failure result with distribution attribution; the second test pins manifest-first suppression for malformed pyproject distributions. * refactor(lfx): emit inline-path-unreadable, document reload contract, trim rot Closes the remaining nits on PR #12967: - inline-path-unreadable (new typed error code): a configured LANGFLOW_COMPONENTS_PATH entry that raises OSError on iterdir (typically permission-denied) now produces a typed LoadResult error carrying str(exc) instead of silently swallowing the message. - duplicate-inline-bundle hint trimmed: removed forward promise about hard-error-in-a-later-release; same actionability, no expiration. - discover_inline_bundles docstring trimmed from three paragraphs to two sentences (per CLAUDE.md anti-multi-paragraph rule). - _distribution_manifest_path docstring trimmed to one-liner. - installed_extension_roots dropped Used-by caller-narration list; manifest_owning_distributions docstring rewritten to call out the shadow-load risk for direct callers and point to load_installed_ extensions for the typed warning surface. - _discovery.import_bundle_module: replaced 'until that lands in a later milestone' rot with a single line ('Absolute imports only between bundle modules; relative imports unsupported.') AND added the single-load-per-process contract note documenting why LE-1018 reload must scrub registry/sys.modules before re-invoking the loader. - load_extension docstring grew a 'Single-load-per-process contract' block telling direct callers not to rely on this function for refresh. Tests: new test_unreadable_path_emits_inline_path_unreadable pins the OSError -> typed-error path. --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) (#12979) * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) Five-stage reload (parallel staging load -> validate -> swap under write lock -> cleanup -> emit) for installed Bundles in Mode A. In-flight flows keep the pre-swap class via existing references; new flows pick up the post-swap class atomically; concurrent reloads on the same Bundle are rejected with reload-in-progress. Adds: * lfx/extension/registry.py -- BundleRegistry with per-bundle reload-in-progress guard and components_index.json writer * lfx/extension/reload.py -- the five-stage pipeline; events emission is stubbed (TODO LE-1017) so the swap mechanics can ship before the events service lands * loader: optional module_namespace param so Stage 1 lands in __reload_staging__.<id> instead of the live _lfx_ext.* namespace * errors: four new typed reload codes (reload-in-progress, reload-bundle-not-installed, reload-bundle-name-mismatch, reload-source-missing) with branch templates and snapshot tests * HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by the existing get_current_active_user dependency, returns 409 with a typed body for the in-progress collision case * CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client against the dev server with text/json output and proper exit codes (--all is gated until LE-1019 lands the list endpoint) * tests: 16 reload-pipeline tests covering the AC matrix (rename round-trip, broken-bundle isolation, concurrent readers, in-flight flow, double-reload guard, bundle-name mismatch) plus 9 CLI client tests Mode A only. In Mode B/C bundle changes require a Docker image rebuild and the reload path is not exercised. The events emission in Stage 5 is intentionally stubbed -- the LE-1017 ticket will swap the body of _emit_bundle_reload_event in one place without touching the pipeline core. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [autofix.ci] apply automated fixes * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) (#12968) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) The two scaffolding CLIs an Extension author types: - `lfx extension init <target>` writes the basic single-Bundle template (manifest with $schema, README, .gitignore, one Component subclass + a pytest smoke test). AC #1: the generated extension validates clean against LE-1014. AC #2: the generated test file is a valid pytest module that exercises the component's build() method. AC #3: any --template other than 'basic' fails with a typed template-deferred-in-this-milestone error and a non-zero exit. Refuses to scaffold over a non-empty target dir. - `lfx extension dev <target>` validates the local extension, records its absolute path in <config_dir>/extensions/dev_extensions.json, prints reload instructions, and execs `langflow run` (or `python -m langflow` when langflow isn't on PATH). --skip-launch registers without launching (used by tests + external dev-server scripts); --skip-validate lets authors register a known-broken manifest to debug it under the loader. Stack: - Wave 0: error codes (extension-target-exists, extension-target-invalid, local-extension-missing) with format branches and snapshot tests. - Wave 1: lfx.extension.init_template -- pure-data scaffolder, no Typer dependency so the CLI is a thin shell over it. - Wave 1: lfx.extension.dev_registry -- atomic JSON state file under the langflow user-cache dir; helpers for register / list / unregister / load_dev_extensions / dev_extension_component_paths. - Wave 2: lfx.cli._extension_commands gains init/dev subcommands. - Wave 2: langflow.main lifespan hook reads the dev registry after bundle loading and extends components_path with each registered bundle dir, so the existing palette discovery picks up dev extensions. Missing paths surface as local-extension-missing warnings (AC #5) without aborting startup. Tests (84 new, 197 total in tests/unit/extension/): - test_init_template.py: AC scenarios + identifier derivation + deterministic file shape. - test_dev_registry.py: register/list/unregister round-trip, idempotent re-register refreshes timestamp, malformed state file treated as empty, missing-path warning, recovery when path reappears, env-var override precedence. - test_cli.py: AC #1 init->validate, AC #3 deferred templates, --skip-launch registers without launching, --skip-validate short-circuits the pre-flight pass. - test_errors.py: snapshot rows for all three new codes; the every-known-code-has-a-snapshot test enforces parity. LE-1018 (reload) reuses load_dev_extensions; LE-1022 (installed-pkg discovery) shares the components_path extension pattern. AC #4 ("boots Langflow with the extension visible in the palette within 5s") is delivered jointly by `extension dev` (registers + execs) and the lifespan hook (loads on startup). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1016 review feedback Fixes the four HIGH issues + the elevated LOW from the review pass: 1. dev_extension_component_paths now forwards EVERY warning, not just local-extension-missing. Previously a duplicate-component-name (or any future warning code) was silently dropped, hiding real signal from the lifespan hook's logs. 2. Defensive emit when a LoadResult has components but source_path is None. The current loader always sets source_path, but a future hand-built LoadResult could violate that contract; we now surface a typed local-extension-missing error rather than dropping the extension silently. 3. Replaced the fragile ``min(len(parts))`` bundle-root selection with a relative-to-source-path measurement. Handles deep-vs-shallow sibling extensions correctly without depending on absolute path depth. 4. Generated README now documents the langflow/lfx prerequisite under the Develop section so authors know `pytest` requires the lfx environment, not just Python. 5. Forced LANGFLOW_LAZY_LOAD_COMPONENTS=false unconditionally in the `extension dev` exec env (was setdefault, which let a developer's global lazy-loading export silently hide their dev components from the palette and miss AC #4's 5s budget). Plus three MEDIUMs: - sys.modules cleanup in test_generated_test_file_runs_against_generated_component so a later test importing the same dotted path doesn't pick up a stale module from a deleted tmp_path. Component instantiation moved inside the try block because Component.__init__ uses inspect.getsourcefile against self.__class__'s still-live module. - Added regex-drift test that pins the init_template patterns to match manifest.py's so a schema regex change can't quietly produce invalid scaffolded manifests. - Added two new dev_registry tests covering the forward-all-warnings contract and the source_path=None defense. Tests: 201 passing (4 new); ruff check + format clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * fix(lfx): align init template with renamed manifest API After merging feat/extension-loader into this branch, two surfaces fell out of sync with the renamed manifest schema: - init_template generates extension.json with `lfx: {bundle_api: [1]}`, but the validator now requires `lfx: {compat: ["1"]}` per LfxCompat. This made `test_basic_template_validates_clean` fail with manifest-invalid (`lfx.compat: Field required; lfx.bundle_api: Extra inputs are not permitted`). - test_init_template_regexes_match_manifest_schema reads `manifest_mod._BUNDLE_NAME_RE`, but that symbol was promoted to public `BUNDLE_NAME_RE` in c63f84a591. Update the test to reference the public name; init_template's local copy is still private since it also covers `_EXTENSION_ID_RE`, which remains private upstream. --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): append-only migration table + flow deserializer rewrite hook (#13024) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * feat(lfx): append-only migration table + flow deserializer rewrite hook Adds the migration layer of the Extension System: an append-only JSON table that maps three legacy component reference shapes (bare class name, old import path, pre-Phase-A namespaced slot) to the post-Phase-A canonical ext:<bundle>:<Class>@<slot> identifier, plus a deserializer hook that rewrites a saved-flow payload in place against that table on load. What landed: * lfx.extension.migration.schema -- Pydantic models for MigrationEntry + MigrationTable with per-entry validators (exactly one of bare/import/slot populated; canonical target shape) and table-level uniqueness check. * lfx.extension.migration.loader -- canonical in-repo path, threadsafe process-lifetime cache, typed errors on every failure mode. * lfx.extension.migration.rewrite -- node-by-node rewrite, idempotent on canonical refs, difflib-backed closest-match suggestion for unmapped references, cross-bucket ambiguity surfaces component-name-ambiguous instead of silently loading into the wrong bundle. * Wired into Graph.from_payload before validate_flow_for_current_settings so every saved-flow load goes through migration first. * scripts/migrate/check_migration_append_only.py -- CI guard that diffs the working-tree table against origin/main and rejects removals or target mutations; reordering and additions are allowed. * 34 new unit tests covering rewrite paths, loader failure modes, schema invariants, and the CI script behavior. What is deliberately deferred: * flow-migrated event emission. The events pipeline is unavailable in this iteration; the wiring point in Graph.from_payload is marked TODO and the MigrationReport already carries every field a future emitter needs. The shipped migration_table.json starts empty; entries land alongside the pilot bundle extraction in a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(frontend): palette Bundle reload action + loading + toasts (#13025) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) Five-stage reload (parallel staging load -> validate -> swap under write lock -> cleanup -> emit) for installed Bundles in Mode A. In-flight flows keep the pre-swap class via existing references; new flows pick up the post-swap class atomically; concurrent reloads on the same Bundle are rejected with reload-in-progress. Adds: * lfx/extension/registry.py -- BundleRegistry with per-bundle reload-in-progress guard and components_index.json writer * lfx/extension/reload.py -- the five-stage pipeline; events emission is stubbed (TODO LE-1017) so the swap mechanics can ship before the events service lands * loader: optional module_namespace param so Stage 1 lands in __reload_staging__.<id> instead of the live _lfx_ext.* namespace * errors: four new typed reload codes (reload-in-progress, reload-bundle-not-installed, reload-bundle-name-mismatch, reload-source-missing) with branch templates and snapshot tests * HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by the existing get_current_active_user dependency, returns 409 with a typed body for the in-progress collision case * CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client against the dev server with text/json output and proper exit codes (--all is gated until LE-1019 lands the list endpoint) * tests: 16 reload-pipeline tests covering the AC matrix (rename round-trip, broken-bundle isolation, concurrent readers, in-flight flow, double-reload guard, bundle-name mismatch) plus 9 CLI client tests Mode A only. In Mode B/C bundle changes require a Docker image rebuild and the reload path is not exercised. The events emission in Stage 5 is intentionally stubbed -- the LE-1017 ticket will swap the body of _emit_bundle_reload_event in one place without touching the pipeline core. * feat(frontend): palette Bundle reload action + loading + toasts Adds the frontend half of the bundle reload flow. When the Bundle header is right-clicked or its overflow ("⋮") icon clicked, a Reload action fires POST /api/v1/extensions/{id}/bundles/{name}/reload and surfaces the result via the existing alert-store toast system. What landed (frontend only): * src/controllers/API/queries/extensions/ -- typed wire-format models (ReloadBundleResponse, ExtensionErrorPayload, ReloadInProgressDetail) and the useReloadBundle mutation hook. The hook unwraps the 409 `reload-in-progress` detail into a stable, parseable Error message so the UI can branch without reading status codes. * components/bundleHeaderActions.tsx -- new Select-based overflow menu next to the Bundle header chevron. Three toast paths: success (green, with components +/- delta), structural failure (red, with typed errors and inline hints), reload-in-progress (notice). Loading state swaps the kebab icon for a spinning Loader2 while the request is in flight. Renders nothing when no extension_id is on the bundle, so the static SIDEBAR_BUNDLES list is unaffected. * components/bundleItems.tsx -- wires the new actions in next to the chevron, plus a context-menu (right-click) capture that opens the same overflow trigger so keyboard / mouse / right-click all share one source of truth. * types/index.ts -- BundleItemProps.item gains an optional extension_id; took the opportunity to extract the SidebarBundle interface and tighten three pre-existing `any` types. * customization/feature-flags.ts -- ENABLE_EXTENSION_RELOAD gate, off by default until the bundle-list endpoint that populates extension_id per bundle ships. * controllers/API/helpers/constants.ts -- EXTENSIONS URL constant. * Tests: 7 component tests + 3 mutation-hook tests, all green. Total sidebar + extensions test count: 479 passing. Deferred: * Event-pipeline subscription is left as an inline TODO. The mutation response carries enough information to drive the toasts on its own today; once the events service lands the toast wiring will move to a `bundle_reloaded` / `bundle_reload_failed` listener so multi-tab and multi-worker swaps surface exactly once. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: End to end bundle installation * fix: Review comments addressed * Update component_index.json * Update component_index.json * Update router.py * fix(docker): copy src/bundles before uv sync so workspace bundles resolve Each directory under ``src/bundles`` is a uv workspace member referenced by ``langflow-base`` (and the root project) as a path dependency. The Docker builders ran ``uv sync --no-install-project`` after copying only the top-level pyproject.toml files, so resolution failed with ``Distribution not found at: file:///app/src/bundles/<name>``. Copying the whole ``src/bundles`` tree (rather than enumerating each bundle) means a new bundle dropped under that dir does not require a Dockerfile edit. The full ``./src`` copy a few lines later produces the same final layer either way; this earlier copy just unblocks the dependency-resolution sync. Touched all builders that run a workspace-resolving uv sync: * docker/build_and_push.Dockerfile * docker/build_and_push_base.Dockerfile * docker/build_and_push_ep.Dockerfile * docker/build_and_push_with_extras.Dockerfile * docker/dev.Dockerfile (bind-mount instead of COPY) * Revert "fix(docker): copy src/bundles before uv sync so workspace bundles resolve" This reverts commit 5aa008a3cd. * feat: DuckDuckGo as Extension in new Bundle System (#13044) * feat: DuckDuckGo Extension for bundles * fix ruff errors * Update test_pilot_duckduckgo_upgrade.py * test(lfx): handle non-empty canonical migration table + editable installs - test_path_override_bypasses_cache: mirror the cached canonical table into the temp file instead of asserting it's empty. Drift in migration_table.json (the duckduckgo entries shipped with the B1 pilot) no longer breaks the cache-bypass invariant. - test_lfx_duckduckgo_ships_manifest: editable installs (pip install -e) surface only dist-info entries in dist.files, so we cannot use that path to find extension.json in the workspace venv. Detect editable mode via direct_url.json's PEP 660 marker and fall through to walking the source tree; non-editable wheel installs still exercise the dist.files path the loader uses at runtime. * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * fix(docker): copy src/bundles before uv sync so workspace bundles resolve Each directory under ``src/bundles`` is a uv workspace member referenced by ``langflow-base`` (and the root project) as a path dependency. The Docker builders ran ``uv sync --no-install-project`` after copying only the top-level pyproject.toml files, so resolution failed with ``Distribution not found at: file:///app/src/bundles/<name>``. Copying the whole ``src/bundles`` tree (rather than enumerating each bundle) means a new bundle dropped under that dir does not require a Dockerfile edit. The full ``./src`` copy a few lines later produces the same final layer either way; this earlier copy just unblocks the dependency-resolution sync. Touched all builders that run a workspace-resolving uv sync: * docker/build_and_push.Dockerfile * docker/build_and_push_base.Dockerfile * docker/build_and_push_ep.Dockerfile * docker/build_and_push_with_extras.Dockerfile * docker/dev.Dockerfile (bind-mount instead of COPY) --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix: Review sweep * fix: More review comments addressed * Ruff check * docs: add bundle porting guide Step-by-step recipe for extracting a provider package from the in-tree ``src/lfx/src/lfx/components/<provider>/`` directory into a standalone Extension Bundle distribution under ``src/bundles/<provider>/``. The DuckDuckGo bundle is the reference; every section maps to a single copy-pasteable change and a verification command. Doc surfaces a forthcoming ``scripts/migrate/port_bundle.py`` automation helper for the mechanical bits; that script lands on the next branch together with the second-pilot port that validates the recipe end-to-end. * fix(extension): accept Output(method=...) in build-method validator The validator's static AST check looked for a literal ``build`` method on every Component subclass and emitted ``build-method-missing`` when absent. Real Langflow components -- including the production ``DuckDuckGoSearchComponent`` and the in-tree ``ArXivComponent`` -- do not declare a literal ``build``; they declare entry-points declaratively via ``outputs = [Output(name=..., method=<name>)]`` and the named method is what gets called. Result: every clean port hit a spurious ``build-method-missing`` error from ``lfx extension validate``. Fix: extend ``_has_build_method`` to also accept any class that names a method via ``Output(method="X")`` AND defines that method in the class body. The negative case ("Output(method=...) without a matching def") still fires -- a typo'd method name would crash at runtime, so the static check should keep flagging it. Two new tests in ``test_validate.py`` lock the contract: positive case mirroring duckduckgo / arxiv, negative case for typo'd method names. All 26 validator tests pass. PORTING.md updates fall out of running the recipe live against the duckduckgo bundle on this branch: * Validate path is ``src/bundles/<bundle>/src/lfx_<bundle>``, not the bundle root (the manifest lives next to ``__init__.py``). * Index regen needs ``LFX_DEV=1`` to skip the prebuilt-index fast path and uses ``scripts/build_component_index.py``. * Drop the migration-table JSON from the ruff invocation (ruff treats it as Python and complains about the top-level expression). * fix: Delete outdated components * fix(palette): wire reload kebab via DropdownMenu, not Select The reload kebab on the palette Bundle header was using Radix ``Select`` to back its overflow menu. ``Select`` is for picking a value, not for firing an action: ``onValueChange`` is gated by value-equality (so a re-click of the only item is a no-op), and the popover-portal click semantics interact poorly with the parent disclosure-trigger button. The combined effect: clicking ⋮ → Reload opened the popover and closed it, but never fired the network request, with no console error to point at. Switch to ``DropdownMenu`` (purpose-built action menu). ``DropdownMenuItem`` exposes ``onSelect`` which fires on every activation -- the same callback path keyboard navigation uses -- so clicking Reload reliably invokes the mutation. Test mocks updated to drive the DropdownMenu primitives instead of Select; all 8 bundleHeaderActions tests + the broader 437-test sidebar suite still pass. * fix(extension): route ddgs through the lockfile + drop bogus list endpoint hint Two reviewer findings, both about reproducibility / correctness of operator-facing surfaces. [P2] ``docker/build_and_push_base.Dockerfile`` previously installed ``ddgs`` via an unpinned ``uv pip install ddgs`` after the workspace sync. That made the base image non-reproducible: a future ``ddgs`` release would silently drift from the tested lock state on every rebuild. Fix: route ddgs through the locked sync. * Restore the ``duckduckgo`` extra in ``src/backend/base/pyproject.toml`` (``ddgs>=9.0.0``) as an internal "image-build sidecar". Public consumers still install ``lfx-duckduckgo`` directly; the extra exists only to keep ``ddgs`` resolved in the lockfile. * The Dockerfile now passes ``--extra duckduckgo`` to ``uv sync --frozen``, picking up the locked ``ddgs==9.14.1``. * The follow-on bundle install keeps ``--no-deps`` so it does not duplicate the now-locked ``ddgs`` / ``lfx`` / ``langchain-community``. [P3] ``langflow/api/v1/extensions.py`` returned a fix hint that pointed operators at ``GET /api/v1/extensions`` -- a route that does not exist in this PR (it lands with the LE-1019 list endpoint). Replaced with a reference to ``lfx extension list``, which is shipped here. * fix(compat): bridge langflow.components.* dynamically via meta path finder Commit 45552cd1df ("fix: Delete outdated components") removed the physical shim files under ``src/backend/base/langflow/components/`` that forwarded saved-flow imports like ``from langflow.components.processing.converter import convert_to_dataframe`` or ``import langflow.components.knowledge_bases.retrieval`` to their new ``lfx.components.*`` homes. Without those shims, dotted imports into ``langflow.components.<sub>.<leaf>`` failed at flow-load time -- the existing ``LangflowCompatibilityModule`` registers ``langflow.components`` itself in ``sys.modules`` but does not bridge submodules, so Python's import machinery falls through to the now-empty langflow.components directory and raises ``ModuleNotFoundError``. Replace the deleted physical-shim stack with a single ``MetaPathFinder`` in ``langflow/__init__.py`` that dynamically resolves every ``langflow.components.<rest>`` import to ``lfx.components.<rest>`` and registers the loaded lfx module in ``sys.modules`` under both names. The langflow- and lfx-prefixed imports share a single underlying module object, so class identity is preserved across the bridge -- ``isinstance`` checks against types resolved through either path keep working. The finder also carries a small first-segment override map for the few subpackages whose name diverged during the move; the only entry today is ``knowledge_bases`` -> ``files_and_knowledge``, matching the deleted shim's intent. Why a meta finder instead of restoring the physical shim files (option 1 from the scope analysis): the meta finder scales without per-bundle maintenance. Every future bundle extraction landed under ``lfx.components`` becomes reachable via the legacy ``langflow.components.<bundle>`` path immediately; nobody has to remember to add a parallel langflow shim. Six new unit tests in ``test_langflow_components_compat_shim.py`` lock the contract: dotted submodule resolution, helpers re-export, the ``knowledge_bases`` override, class identity preservation, top-level aliasing, and the "arbitrary extracted bundle" case that prevents regression for future ports. Verified pre-existing tests: * 17 dynamic-import integration tests pass. * The reload-route-guard suite still passes. * The Research Translation Loop starter-project test (which loads ArXivComponent) passes. * Update index.tsx * fix: Review pass on delivery * fix: Second review sweep, bare names * fix: Third review sweep * Resolve dotted imports and attribute chains Record import shape and flatten attribute chains so router references can be resolved across dotted imports and re-exports. Added ImportTarget dataclass, _attribute_chain helper, and switched IncludeCall/DecoratorRef to store attribute tuples. parse_file now records ImportTarget entries for imports and captures dotted parent/child chains for include_router and decorators. Replaced _resolve_var with _resolve_chain which handles "from" vs "module" imports, various import shapes (including import x.y and aliased imports), and prevents infinite recursion on re-export cycles. * One more sweep * Add seed-directory extension loading and docs Introduce filesystem "seed directory" extension support and authoring docs. Adds three documentation pages (quickstart, manifest reference, author guide) and wires them into the docs sidebar. Implement load_seed_extensions to discover/load bundles from $LANGFLOW_SEED_DIR (default /opt/langflow/bundles), export it from loader/__init__ and extension package, and integrate it into the components import pipeline. Handle installed-vs-seed shadowing by preferring installed distributions and appending a typed seed-bundle-shadowed ExtensionError; add the corresponding error code and message. Update schema doc link and add unit/integration tests to cover seed loading, determinism, shadowing behavior, and migration-target resolution. * Fix claims * Fix docasaurus build * Clean up the manual checklist * Update test_pilot_duckduckgo_upgrade.py * fix: Some wording issues with dogfooding * fix: Comments addressed * Update port_bundle.py * Update component_index.json * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * fix: pytest testpaths glob and accurate scaffold output in port_bundle testpaths now uses ``src/bundles/*/tests`` so future bundle tests are picked up automatically without hardcoding each bundle at the root. pytest expands the glob via ``glob.iglob(..., recursive=True)``. Rewrote port_bundle.py's ``_render_migration_entries`` and ``_render_test_scaffold`` to match the actual conventions: * Migration entries now use ``bare_class_name``/``import_path``/ ``legacy_slot`` + ``target`` + ``added_in`` (the keys ``lfx.extension.migration.loader`` actually reads) instead of the invented ``from``/``to``/``release`` shape. * Test scaffold now imports ``load_migration_table`` and ``migrate_flow_payload`` -- the APIs the real tests use -- instead of a fictitious ``resolve_legacy_id``. Includes the ``migration_table`` fixture, ``_saved_flow``/``_saved_flow_node`` helpers, and the distribution-importable + manifest-shipped checks that mirror ``test_pilot_duckduckgo_upgrade.py``. Verified by rendering the scaffold for duckduckgo with synthetic plan data, dropping it into ``src/lfx/tests/integration/extension/``, and running pytest -- ``3 passed, 2 skipped`` (skips are the wheel-only checks, by design when run inside the lfx isolation venv). * [autofix.ci] apply automated fixes * fix: Review comments * Update test_discovery.py * feat: Port Arxiv to the Extension Framework (#13047) * feat(bundles): port arxiv as the second-pilot Bundle + porting helper Validates ``src/bundles/PORTING.md`` end-to-end by following its recipe against a clean candidate (``ArXivComponent``: no third-party runtime deps, no langflow-base extra, no deactivated duplicate). Touchpoints exercised: * Bundle skeleton at ``src/bundles/arxiv/`` mirroring duckduckgo. * In-tree provider directory removed from ``src/lfx/src/lfx/components/`` along with its three references in ``components/__init__.py``. * Workspace wiring: dep, ``[tool.uv.sources]``, ``[tool.uv.workspace]`` members, lockfile. * Migration table: bare-name + two import-path forms + legacy_slot entry. * Component index regenerated via ``LFX_DEV=1`` (forces dynamic discovery; without it the script reproduces stale entries). * Integration test ``test_pilot_arxiv_upgrade.py`` mirroring the duckduckgo pilot suite; 5 tests pass against the workspace install. PORTING.md updates fall out of running the recipe live: * Validate path is ``src/bundles/<bundle>/src/lfx_<bundle>``, not the bundle root (the manifest lives next to ``__init__.py``). * Index regen needs ``LFX_DEV=1`` to skip the prebuilt-index fast path. * Drop the migration-table JSON from the ruff invocation (ruff treats it as Python and complains about the top-level expression). ``scripts/migrate/port_bundle.py`` is the mechanical helper referenced from § Automation: stdlib-only, dry-run by default, refuses on invalid input, and intentionally leaves migration-table edits + integration-test authoring to a human (release version + bare-name uniqueness require judgement). Three guard rails verified: invalid bundle name, existing-target-bundle, missing in-tree provider. * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * Update Research Translation Loop.json * Update .secrets.baseline * fix: Move bundle test for arxiv * Update PORTING.md * Update component_index.json * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * Update component_index.json * [autofix.ci] apply automated fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * [autofix.ci] apply automated fixes * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * Update component_index.json * Update component_index.json * Update component_index.json * fix: empty cache dict on reload * Lint fixes * Update test_components_cache_integration.py * Update test_reload.py * Hardening pass * Update test_init_template.py * fix: resolve cross-source bundle-name shadowing before registry population The reload pipeline reads ``live.source_path`` from the registry on every call. Previously, the registry-population loop in ``import_extension_components`` only special-cased installed-shadows-seed; for every other pair (seed/dev, seed/inline, dev/inline, installed/dev, installed/inline) it silently overwrote earlier records via last-wins iteration order. A stale dev registration whose ``source_path`` pointed at a different filesystem location could clobber the seed record's ``source_path``; reload would then walk the dev path while the operator edited the seed copy on disk -- producing 200 OK with empty deltas on every reload, including for syntactically broken edits. Generalize the dedup pass to every (earlier, later) pair using the explicit precedence ``installed > seed > dev > inline``, applied before both registry population AND palette template construction so the two read from the same winning source. Add a new generic ``bundle-shadowed`` typed error code for the pairs the existing ``seed-bundle-shadowed`` did not cover; keep ``seed-bundle-shadowed`` for the documented installed- over-seed pair so existing CLI exit-code logic and snapshot tests stay intact. Both codes are warn-only in ``lfx extension list``. Also add an INFO log line in reload Stage 1 with the resolved ``source_path`` so the next "200 OK with empty deltas" repro can be triaged from the server log alone -- if the path logged is not the path the operator was editing, this dedup is what to look at. Includes a regression test (``test_seed_bundle_shadows_dev_emits_generic_bundle_shadowed``) that asserts both halves: seed wins in the BundleRegistry AND the registry's ``source_path`` is the seed path, not the stale dev path. * Update component_index.json * fix: widen lfx-* bundles' requires-python to <3.15 to match langflow root The two pilot bundles (``lfx-arxiv``, ``lfx-duckduckgo``) were scaffolded by ``scripts/migrate/port_bundle.py`` whose template hard-coded ``requires-python = ">=3.10,<3.14"``. Because both bundles are uv workspace members of the langflow root, that cap leaked into every workspace-level resolve: ``cd src/lfx && uv sync`` (the path ``make lfx_tests`` takes) refuses to pick a Python 3.14 interpreter even though lfx itself, langflow, and langflow-base all advertise ``>=3.10,<3.15``. Hosts with 3.14 installed see: error: The requested interpreter resolved to Python 3.14.5, which is incompatible with the project's Python requirement: `>=3.10, <3.14`. Fix at the source so future ``port_bundle.py`` runs do not regress this: template emits ``<3.15``, and the two existing emitted pyprojects are bumped in lockstep. ``uv.lock`` regenerates with the wider range. Verified ``make lfx_tests`` resolves cleanly on a host with both 3.13.12 and 3.14.5 installed; the focused extension slice runs to completion. * Update __init__.py * Update src/lfx/tests/unit/extension/migration/test_rewrite.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/loader/_plugins.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update docs/docs/Deployment/deployment-extensions-production.mdx Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/loader/_orchestrator.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update docs/docs/Develop/extensions-author-guide.mdx Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update docs/docs/Develop/extensions-manifest.mdx Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: Coderabbit review suggestions * Template updates * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * Tweaks to bundle hot reload * [autofix.ci] apply automated fixes * chore: Address review comments by @Cristhianzl * [autofix.ci] apply automated fixes * Update BUNDLE_API.md * Update component_index.json * Update component_index.json * Update Structured Data Analysis Agent.json * fix: Next round of review comments * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>	2026-05-18 17:36:56 +00:00
Gabriel Luiz Freitas Almeida	1955f8fae5	chore: support Python 3.14 (#13085) ... * feat: upgrade Docker images to Python 3.14 (experimental) ## Changes - Update pyproject.toml: requires-python from <3.14 to <3.15 - Update all 5 Dockerfiles to use Python 3.14: - Builder stage: python3.12-trixie-slim → python3.14-trixie-slim - Runtime stage: python:3.12-slim-trixie → python:3.14-slim-trixie ## Files Updated - src/backend/base/pyproject.toml - docker/build_and_push_base.Dockerfile - docker/build_and_push.Dockerfile - docker/build_and_push_backend.Dockerfile - docker/build_and_push_with_extras.Dockerfile - docker/build_and_push_ep.Dockerfile ## Rationale Python 3.14.5 was released on May 10, 2026. Upgrading to the latest Python version should help reduce CVE vulnerabilities in Docker images. ## Testing Strategy This is an experimental change to test Python 3.14 compatibility: - Docker images will use Python 3.14 - CI/CD workflows still test on Python 3.10-3.13 - Nightly build will validate if dependencies work with 3.14 - Can be reverted quickly if issues are found ## Next Steps - Monitor nightly build for failures - If successful, update CI/CD workflows to add Python 3.14 to test matrix - If failures occur, revert and investigate compatibility issues * chore: support Python 3.14 Bump requires-python upper bound to <3.15 across langflow, langflow-base, lfx, and langflow-sdk, and add 3.14 to CI test matrices so PRs are gated on 3.14 compatibility. Conditional pins for transitive deps without 3.14 wheels at the existing caps: - onnxruntime: >=1.26 on 3.14 (existing <1.24 cap retained for 3.10) - faiss-cpu: >=1.13.2 on 3.14 (existing ==1.9.0.post1 retained for <3.14) * chore: marker-gate IBM watsonx packages for Python 3.14 ibm-watsonx-ai 1.3.x cannot import on Python 3.14: its StrEnum subclasses override __init__ in a way that conflicts with 3.14's reworked enum __set_name__ path (TypeError on KnowledgeBaseFieldRole class creation). Cap ibm-watsonx-ai, langchain-ibm, and the ibm-watsonx-orchestrate-* extras at python_version<'3.14' until upstream adapts. Watsonx component imports are already lazy, so this surfaces only at component-access time on 3.14, where the existing ImportError handler in lfx.components.ibm.__init__ degrades it gracefully. test_model_utils.py imports ChatWatsonx at module top to verify get_model_name resolves model_id; guard that import so the test module skips on 3.14 instead of breaking collection. * [autofix.ci] apply automated fixes * chore: upgrade remaining Docker images to Python 3.14 build_and_push*.Dockerfile already moved to 3.14 in the merge from feat/upgrade-python-3.14-docker-images. Apply the same bump to the dev, devcontainer, and lfx Docker images so all in-repo Dockerfiles share one Python version. * [autofix.ci] apply automated fixes * chore: gate 3.14-broken extras to python_version<'3.14' cuga (and its transitive fastembed -> py-rust-stemmers source build), altk/agent-lifecycle-toolkit (re-pulls ibm-watsonx-ai which was already gated), langchain-pinecone, langwatch, ragstack-ai-knowledge-store, and OpenDsStar all pin themselves below 3.14 upstream. Without a marker guard, the workspace lock still tries to install them and a Docker `uv sync` then attempts source builds that require Rust (CI Docker test failure with py-rust-stemmers 0.1.5). Add python_version<'3.14' (or <'3.13' for ragstack) to each pin so those packages are simply omitted from the 3.14 install set until upstream catches up. uv pip check is now clean on 3.14.2. * test: skip test_altk_agent on Python 3.14 altk (agent-lifecycle-toolkit) is gated to python_version<'3.14' upstream and now via the langflow-base [altk] extra marker. The test imports altk at module top to exercise ALTKAgentComponent; guard the import so collection skips on 3.14 instead of failing. * test: skip remaining altk tests on Python 3.14 Three more test modules import lfx.base.agents.altk_* at module top, which transitively imports altk. Add the same module-level skip guard as test_altk_agent.py: - test_altk_agent_logic.py - test_altk_agent_tool_conversion.py - test_conversation_context_ordering.py * fix(calculator): replace removed ast.Num with ast.Constant for Python 3.14 ast.Num was deprecated in Python 3.8 in favor of ast.Constant and removed entirely in 3.14. The calculator tool and its core walker crashed on 3.14 with 'module ast has no attribute Num'. Switch the isinstance checks to ast.Constant + isinstance(node.value, (int, float)), and drop the now-dead ast.Num backwards-compat branch in calculator_core.py (the ast.Constant branch already handles every input it would have matched). Update the parser unit-test fixtures to construct ast.Constant nodes. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * test: skip LangWatch HTTP instrumentation tests on Python 3.14 langwatch is gated to python_version<'3.14' upstream. The TestLangWatchHttpInstrumentation class patches langwatch.setup and langwatch.trace inside a fixture, which requires langwatch to be importable; on 3.14 the test errors with ModuleNotFoundError. Guard the class with a skipif on langwatch availability. * test: allow IBM and altk components to be missing on Python 3.14 test_all_modules_importable enforces that every component in __all__ imports cleanly. On 3.14 the ibm-watsonx-ai, langchain-ibm, and altk packages are gated upstream and intentionally not installed, so the WatsonxAI, WatsonxEmbeddings, and ALTKAgent components fail to import as designed. - test_all_components_in_categories_importable: accept a known-gated deny-list on 3.14 instead of failing. - test_all_lfx_component_modules_directly_importable: extend the existing 'missing optional dependency' allowlist with altk, langchain_ibm, and ibm_watsonx_ai. * fix(lfx): pass ensure_exists to user_cache_dir for Python 3.14 Python 3.14 tightened PurePath.__init__ to reject unknown keyword arguments. The KeyedWorkerLockManager constructor was passing ensure_exists=True to Path() instead of user_cache_dir(), which silently worked on 3.10-3.13 but raises TypeError on 3.14 and also meant the cache directory was never actually being created. Move the kwarg to user_cache_dir() where it belongs and update the two unit tests that asserted the buggy call shape. * [autofix.ci] apply automated fixes * test: accept either ZIP or JSON error path on Python 3.14 Python 3.14's zipfile.is_zipfile() now validates the central-directory signature in addition to EOCD, so the garbage+EOCD payload no longer passes the is_zipfile() dispatch in the /flows/upload/ route. The endpoint still returns 400 with a descriptive detail (now from the JSON branch); only the message wording differs, which the test was asserting verbatim. * fix(lfx): normalize cors_origins ['*'] back to '*' on Python 3.14 Pydantic-settings on Python 3.14 parses the env var '*' into ['*'] before the cors_origins field validator runs (the list[str] | str union resolves differently than on 3.10-3.13). Collapse that back to the bare-string wildcard so downstream consumers — including warn_about_future_cors_changes and the test suite — see the same shape on every supported Python version. --------- Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>	2026-05-13 22:23:39 +00:00
Eric Hare	e555e470d9	fix: Cherry-pick nightly SDK build fixes to main (#12491) ... * fix: Build and install the langflow-sdk for lfx (fixes nightly) (#12481) * fix: Build and install the langflow-sdk for lfx * Publish sdk as a nightly * Update ci.yml * Update python_test.yml * Update ci.yml * fix: Properly grep for the langflow version (#12486) * fix: Properly grep for the langflow version * Mount the sdk where needed * Skip the sdk * [autofix.ci] apply automated fixes * Update setup.py * fix(docker): Remove broken npm self-upgrade from Docker images (#12309) * fix: replace grep -oP with sed for Node.js version extraction in Docker builds (#12331) The grep -oP (PCRE regex) command fails in the python:3.12.12-slim-trixie Docker base image because PCRE support is not available in the slim variant. This replaces grep -oP with portable sed -nE in all 5 Dockerfiles and adds an empty version guard to fail fast with a clear error message instead of producing a broken download URL. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com> Co-authored-by: vjgit96 <vijay.katuri@ibm.com>	2026-04-03 14:25:33 -04:00
Gabriel Luiz Freitas Almeida	dfa4f98c21	ci: Add missing lfx metadata files to Docker build context (#9845)	2025-09-12 15:50:57 -03:00
Gabriel Luiz Freitas Almeida	84aa1d63b3	fix: update sqlalchemy dependencies in pyproject.toml (#6854)	2025-02-26 19:27:19 -03:00
Mattias Michaux	cdf85e55de	fix: add the git binary to the final step of the container images (#5863) ... Add the git binary to the runtime images	2025-01-22 15:07:34 +00:00
Gabriel Luiz Freitas Almeida	8ea4405010	refactor: run upgrade in dockerfiles to update dependencies (#5508) ... chore: Update Dockerfiles to use COPY instead of ADD run apt-get upgrade - Replaced ADD commands with COPY for better clarity and consistency across Dockerfiles. - Added apt-get upgrade and clean commands to reduce image size and ensure packages are up to date. - Updated user creation commands to include necessary cleanup steps. - Ensured all Dockerfiles follow a similar structure for maintainability.	2025-01-02 15:49:41 +00:00
Gabriel Luiz Freitas Almeida	43ad226ddb	chore: update docker images to use uv (#3916) ... * fix dev dependencies * fix dev dependencies * update lock * Add langchain-unstructured dependency to pyproject.toml * Switch to uv-based Docker image and streamline build process - Use `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image - Simplify environment variable setup and remove unnecessary ones - Install project dependencies using `uv sync` with cache mounts - Add and copy necessary files for the build process - Update npm installation and frontend build steps - Adjust entrypoint and runtime configurations - Remove redundant user setup and streamline CMD execution * Refactor Dockerfile to use multi-stage builds for optimized image size - Introduce a builder stage to install dependencies and build the frontend. - Use npm ci instead of npm install for more reliable builds. - Copy built frontend assets to the final image. - Transition to a runtime stage with a slimmer Python base image. - Ensure the final image only contains necessary runtime dependencies. * Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` and update Dockerfile - Replace `python:3.12.3-slim` with `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image. - Enable bytecode compilation and set link mode to copy. - Update dependency installation process using `uv sync`. - Simplify frontend build and copy process. - Add OCI labels for image metadata. - Update CMD to use `langflow-base` for running the application. * Update Dockerfile to use uv base image and optimize dependency installation - Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` - Combine apt-get commands and clean up to reduce image size - Replace Poetry with uv for dependency management and caching * Refactor Dockerfile to use multi-stage build for optimized image size - Introduce a builder stage using `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` - Add a runtime stage using `python:3.12.3-slim` - Copy the virtual environment from the builder stage to the runtime stage - Remove the `uv` entrypoint to avoid invoking it by default * Update Dockerfile: Fix indentation and remove redundant ENTRYPOINT reset * ci: update release workflows to use uv (#3919) * Update nightly build workflow to use 'uv' for dependency management and caching - Replace Poetry with 'uv' for dependency installation and project setup - Add steps to install 'uv' and set up caching for 'uv.lock' - Modify Python setup to use version specified in 'pyproject.toml' - Remove Node.js setup steps * Update release workflow to use uv for dependency management - Replace Poetry with uv for dependency installation and caching - Update Python setup to use version specified in pyproject.toml - Add steps to restore uv cache and install project dependencies using uv - Adjust publish steps to maintain functionality with new setup * Replace 'poetry publish' with 'uv publish' in Makefile for consistency * Update nightly build workflow to use 'uv' for dependency management and caching * Set up Node.js 20 in release_nightly workflow and update version verification logic * Update Makefile to use `uv sync --frozen` for backend dependencies installation * Update Makefile to pass arguments to 'uv build' and remove '--skip-existing' from publish commands - Modified 'build_langflow' target to accept arguments. - Removed '--skip-existing' from 'publish_base' and 'publish_base_testpypi' commands. - Added TODO comments for updating test-pypi repository usage. * Remove --skip-existing flag from uv publish commands and add TODO comments for test-pypi updates * new lock * Update CI workflow to remove version prefix and add build args - Removed 'v' prefix from version extraction in nightly release workflow. - Added '--no-sources' argument to the build command in the distribution step. * Update CI workflow to use 'uv' for versioning and publishing - Replace 'poetry' with 'uv' for version extraction in release checks * Update CI workflow to use UV_PUBLISH_TOKEN for PyPI publishing * Add verification step for 'langflow-nightly' name and version in CI workflow - Corrected awk commands for extracting 'langflow-base' name and version. - Added a new step to verify 'langflow-nightly' name and version against expected values. * feat: Update dev.Dockerfile to use 'uv' for dependency management and caching * update dockerfiles with --no-editable --------- Co-authored-by: phact <estevezsebastian@gmail.com> Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>	2024-09-30 22:49:25 +00:00
Ítalo Johnny	6cdc915460	build: update docker for dev environment (#3551) ... * build: update dockerfile for dev environment * chore: add script to centralize docker commands * build: create docker-compose.yml for services * chore: add makefile target for docker-compose	2024-08-26 22:05:08 +00:00
Gabriel Luiz Freitas Almeida	56f13c918b	Reorganize repo (#1971) ... * chore: Update dockerfile paths and branch name in render.yaml and release.yml * chore(readthedocs.yaml): remove .readthedocs.yaml file as it is no longer needed chore(base.Dockerfile): remove base.Dockerfile as it is no longer used in the project feat(cdk-docker-compose.yml): add cdk-docker-compose.yml file to set up docker-compose for backend and frontend services * move dockerignore * chore: Remove test-results/.last-run.json file * chore: Cache Node.js dependencies during workflow execution * chore: Remove npm cache from workflow and cache Node.js dependencies * chore: Update shardIndex and shardTotal values in typescript_test.yml workflow * chore: Update Playwright test command with shard and worker options	2024-05-27 06:55:32 -07:00