mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-24 15:19:46 +08:00
cb651c343a87f69eebd33da87cdcdb8e8ae168aa
17783 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| cb651c343a | fix: More review comments addressed | |||
| 1d7e366498 | fix: Review sweep | |||
| 42fc6ae507 |
feat: DuckDuckGo as Extension in new Bundle System (#13044)
* feat: DuckDuckGo Extension for bundles * fix ruff errors * Update test_pilot_duckduckgo_upgrade.py * test(lfx): handle non-empty canonical migration table + editable installs - test_path_override_bypasses_cache: mirror the cached canonical table into the temp file instead of asserting it's empty. Drift in migration_table.json (the duckduckgo entries shipped with the B1 pilot) no longer breaks the cache-bypass invariant. - test_lfx_duckduckgo_ships_manifest: editable installs (pip install -e) surface only dist-info entries in dist.files, so we cannot use that path to find extension.json in the workspace venv. Detect editable mode via direct_url.json's PEP 660 marker and fall through to walking the source tree; non-editable wheel installs still exercise the dist.files path the loader uses at runtime. * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * fix(docker): copy src/bundles before uv sync so workspace bundles resolve Each directory under ``src/bundles`` is a uv workspace member referenced by ``langflow-base`` (and the root project) as a path dependency. The Docker builders ran ``uv sync --no-install-project`` after copying only the top-level pyproject.toml files, so resolution failed with ``Distribution not found at: file:///app/src/bundles/<name>``. Copying the whole ``src/bundles`` tree (rather than enumerating each bundle) means a new bundle dropped under that dir does not require a Dockerfile edit. The full ``./src`` copy a few lines later produces the same final layer either way; this earlier copy just unblocks the dependency-resolution sync. Touched all builders that run a workspace-resolving uv sync: * docker/build_and_push.Dockerfile * docker/build_and_push_base.Dockerfile * docker/build_and_push_ep.Dockerfile * docker/build_and_push_with_extras.Dockerfile * docker/dev.Dockerfile (bind-mount instead of COPY) --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
|||
| 81be42add3 |
Revert "fix(docker): copy src/bundles before uv sync so workspace bundles resolve"
This reverts commit
|
|||
| 5aa008a3cd |
fix(docker): copy src/bundles before uv sync so workspace bundles resolve
Each directory under ``src/bundles`` is a uv workspace member referenced by ``langflow-base`` (and the root project) as a path dependency. The Docker builders ran ``uv sync --no-install-project`` after copying only the top-level pyproject.toml files, so resolution failed with ``Distribution not found at: file:///app/src/bundles/<name>``. Copying the whole ``src/bundles`` tree (rather than enumerating each bundle) means a new bundle dropped under that dir does not require a Dockerfile edit. The full ``./src`` copy a few lines later produces the same final layer either way; this earlier copy just unblocks the dependency-resolution sync. Touched all builders that run a workspace-resolving uv sync: * docker/build_and_push.Dockerfile * docker/build_and_push_base.Dockerfile * docker/build_and_push_ep.Dockerfile * docker/build_and_push_with_extras.Dockerfile * docker/dev.Dockerfile (bind-mount instead of COPY) |
|||
| 01e4a0fa96 | Update router.py | |||
| 80385ac835 | Update component_index.json | |||
| 5ac159ed81 | Update component_index.json | |||
| 65c14bf077 | fix: Review comments addressed | |||
| b92b3d6e77 | feat: End to end bundle installation | |||
| 8873fad536 |
feat(frontend): palette Bundle reload action + loading + toasts (#13025)
* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)
Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.
What's in lfx.extension:
- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
`extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
`routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
None-only so non-null values produce a dedicated
`field-deferred-in-this-milestone` error instead of a generic schema wall.
`bundles` accepts a list but rejects length > 1 with
`multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
per discriminant. Codes are registered in `ERROR_CODES`; an
`ExtensionError` constructed with an unknown code raises at construction
time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
path-safety (no `..`, no absolute paths, no symlink escape), AST
inspection of every `.py` (syntax, Component subclass present, build()
declared, top-level `import *`, top-level I/O primitives), and an opt-in
`--execute-imports` that runs each module in a subprocess with a
temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.
Acceptance-criteria coverage in tests/unit/extension/:
- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
file is never written); `--execute-imports` DOES execute it and the canary
appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
ERROR_CODES and the snapshot table are in lockstep so future additions
cannot ship without a format branch and a snapshot.
Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).
* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader
tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).
Fixes ModuleNotFoundError seen in CI on the 3.10 job.
* Update src/lfx/tests/unit/extension/test_schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/extension/schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/cli/_extension_commands.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)
Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:
- load_extension(root): one manifest, one Bundle, registered at @official.
Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
- discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
user-declared path order). First-wins on duplicate names; second emits
duplicate-inline-bundle warning that names both paths.
Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.
New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.
Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(lfx): address LE-1015 review feedback
- Drop the unproduced duplicate-distribution error code; LE-1022 will add
it once startup-time discovery has a place to surface it. Restores the
invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
was already existence-checked, and a concurrent removal in the narrow
window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(lfx): split extension loader into a small subpackage (LE-1015)
Addresses the file-size feedback from the LE-1015 review. The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:
loader/
__init__.py # re-exports the public surface
_types.py # SLOT constants, LoadedComponent, LoadResult
_discovery.py # filesystem walk + importlib.util orchestration
_detection.py # Component subclass identification (MRO heuristic)
_orchestrator.py # load_extension, discover_inline_bundles
_plugins.py # manifest-first precedence over langflow.plugins
Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline. No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass. ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).
The test suite is split to mirror the package:
tests/unit/extension/loader/
conftest.py # shared fixtures, FakeDist, autouse scrub
test_load_extension.py # @official slot, identity, failure modes
test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot
test_plugins.py # manifest-first precedence helpers
test_types.py # LoadedComponent, LoadResult, code parity
Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018)
Five-stage reload (parallel staging load -> validate -> swap under write
lock -> cleanup -> emit) for installed Bundles in Mode A. In-flight
flows keep the pre-swap class via existing references; new flows pick up
the post-swap class atomically; concurrent reloads on the same Bundle
are rejected with reload-in-progress.
Adds:
* lfx/extension/registry.py -- BundleRegistry with per-bundle
reload-in-progress guard and components_index.json writer
* lfx/extension/reload.py -- the five-stage pipeline; events emission
is stubbed (TODO LE-1017) so the swap mechanics can ship before the
events service lands
* loader: optional module_namespace param so Stage 1 lands in
__reload_staging__.<id> instead of the live _lfx_ext.* namespace
* errors: four new typed reload codes (reload-in-progress,
reload-bundle-not-installed, reload-bundle-name-mismatch,
reload-source-missing) with branch templates and snapshot tests
* HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by
the existing get_current_active_user dependency, returns 409 with a
typed body for the in-progress collision case
* CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client
against the dev server with text/json output and proper exit codes
(--all is gated until LE-1019 lands the list endpoint)
* tests: 16 reload-pipeline tests covering the AC matrix (rename
round-trip, broken-bundle isolation, concurrent readers, in-flight
flow, double-reload guard, bundle-name mismatch) plus 9 CLI client
tests
Mode A only. In Mode B/C bundle changes require a Docker image rebuild
and the reload path is not exercised.
The events emission in Stage 5 is intentionally stubbed -- the LE-1017
ticket will swap the body of _emit_bundle_reload_event in one place
without touching the pipeline core.
* feat(frontend): palette Bundle reload action + loading + toasts
Adds the frontend half of the bundle reload flow. When the Bundle
header is right-clicked or its overflow ("⋮") icon clicked, a Reload
action fires POST /api/v1/extensions/{id}/bundles/{name}/reload and
surfaces the result via the existing alert-store toast system.
What landed (frontend only):
* src/controllers/API/queries/extensions/ -- typed wire-format models
(ReloadBundleResponse, ExtensionErrorPayload, ReloadInProgressDetail)
and the useReloadBundle mutation hook. The hook unwraps the 409
`reload-in-progress` detail into a stable, parseable Error message so
the UI can branch without reading status codes.
* components/bundleHeaderActions.tsx -- new Select-based overflow menu
next to the Bundle header chevron. Three toast paths: success
(green, with components +/- delta), structural failure (red, with
typed errors and inline hints), reload-in-progress (notice).
Loading state swaps the kebab icon for a spinning Loader2 while
the request is in flight. Renders nothing when no extension_id is
on the bundle, so the static SIDEBAR_BUNDLES list is unaffected.
* components/bundleItems.tsx -- wires the new actions in next to the
chevron, plus a context-menu (right-click) capture that opens the
same overflow trigger so keyboard / mouse / right-click all share
one source of truth.
* types/index.ts -- BundleItemProps.item gains an optional
extension_id; took the opportunity to extract the SidebarBundle
interface and tighten three pre-existing `any` types.
* customization/feature-flags.ts -- ENABLE_EXTENSION_RELOAD gate, off
by default until the bundle-list endpoint that populates extension_id
per bundle ships.
* controllers/API/helpers/constants.ts -- EXTENSIONS URL constant.
* Tests: 7 component tests + 3 mutation-hook tests, all green. Total
sidebar + extensions test count: 479 passing.
Deferred:
* Event-pipeline subscription is left as an inline TODO. The mutation
response carries enough information to drive the toasts on its own
today; once the events service lands the toast wiring will move to
a `bundle_reloaded` / `bundle_reload_failed` listener so multi-tab
and multi-worker swaps surface exactly once.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 77e7a93553 |
feat(lfx): append-only migration table + flow deserializer rewrite hook (#13024)
* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)
Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.
What's in lfx.extension:
- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
`extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
`routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
None-only so non-null values produce a dedicated
`field-deferred-in-this-milestone` error instead of a generic schema wall.
`bundles` accepts a list but rejects length > 1 with
`multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
per discriminant. Codes are registered in `ERROR_CODES`; an
`ExtensionError` constructed with an unknown code raises at construction
time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
path-safety (no `..`, no absolute paths, no symlink escape), AST
inspection of every `.py` (syntax, Component subclass present, build()
declared, top-level `import *`, top-level I/O primitives), and an opt-in
`--execute-imports` that runs each module in a subprocess with a
temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.
Acceptance-criteria coverage in tests/unit/extension/:
- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
file is never written); `--execute-imports` DOES execute it and the canary
appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
ERROR_CODES and the snapshot table are in lockstep so future additions
cannot ship without a format branch and a snapshot.
Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).
* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader
tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).
Fixes ModuleNotFoundError seen in CI on the 3.10 job.
* Update src/lfx/tests/unit/extension/test_schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/extension/schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/cli/_extension_commands.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)
Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:
- load_extension(root): one manifest, one Bundle, registered at @official.
Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
- discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
user-declared path order). First-wins on duplicate names; second emits
duplicate-inline-bundle warning that names both paths.
Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.
New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.
Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(lfx): address LE-1015 review feedback
- Drop the unproduced duplicate-distribution error code; LE-1022 will add
it once startup-time discovery has a place to surface it. Restores the
invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
was already existence-checked, and a concurrent removal in the narrow
window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(lfx): split extension loader into a small subpackage (LE-1015)
Addresses the file-size feedback from the LE-1015 review. The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:
loader/
__init__.py # re-exports the public surface
_types.py # SLOT constants, LoadedComponent, LoadResult
_discovery.py # filesystem walk + importlib.util orchestration
_detection.py # Component subclass identification (MRO heuristic)
_orchestrator.py # load_extension, discover_inline_bundles
_plugins.py # manifest-first precedence over langflow.plugins
Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline. No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass. ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).
The test suite is split to mirror the package:
tests/unit/extension/loader/
conftest.py # shared fixtures, FakeDist, autouse scrub
test_load_extension.py # @official slot, identity, failure modes
test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot
test_plugins.py # manifest-first precedence helpers
test_types.py # LoadedComponent, LoadResult, code parity
Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: LangflowCompat -> LfxCompat
* fix: Remove references to ticket
* fix: encode maxItems constraint in schema
* fix: deferred fields and schema version
* Fix ruff errors
* fix: align loader tests with renamed manifest API and strip ticket refs
The merge of feat/extension-validate brought in the LfxCompat / compat
rename, but the loader test fixtures still used LangflowCompat / bundle_api
and failed at the manifest layer. Update conftest.py + test_load_extension.py
to the post-rename API.
Strip 17 LE-XXXX ticket references from loader source files, errors.py
loader-specific comments, and the four loader test docstrings, matching the
convention applied to LE-1014. Descriptive prose preserved.
Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the
loader's orchestrator no longer reaches across modules into a private name.
* feat(lfx): append-only migration table + flow deserializer rewrite hook
Adds the migration layer of the Extension System: an append-only JSON table
that maps three legacy component reference shapes (bare class name, old
import path, pre-Phase-A namespaced slot) to the post-Phase-A canonical
ext:<bundle>:<Class>@<slot> identifier, plus a deserializer hook that
rewrites a saved-flow payload in place against that table on load.
What landed:
* lfx.extension.migration.schema -- Pydantic models for MigrationEntry +
MigrationTable with per-entry validators (exactly one of bare/import/slot
populated; canonical target shape) and table-level uniqueness check.
* lfx.extension.migration.loader -- canonical in-repo path, threadsafe
process-lifetime cache, typed errors on every failure mode.
* lfx.extension.migration.rewrite -- node-by-node rewrite, idempotent on
canonical refs, difflib-backed closest-match suggestion for unmapped
references, cross-bucket ambiguity surfaces component-name-ambiguous
instead of silently loading into the wrong bundle.
* Wired into Graph.from_payload before validate_flow_for_current_settings
so every saved-flow load goes through migration first.
* scripts/migrate/check_migration_append_only.py -- CI guard that diffs
the working-tree table against origin/main and rejects removals or
target mutations; reordering and additions are allowed.
* 34 new unit tests covering rewrite paths, loader failure modes, schema
invariants, and the CI script behavior.
What is deliberately deferred:
* flow-migrated event emission. The events pipeline is unavailable in this
iteration; the wiring point in Graph.from_payload is marked TODO and the
MigrationReport already carries every field a future emitter needs.
The shipped migration_table.json starts empty; entries land alongside the
pilot bundle extraction in a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 6da7fb151e |
feat(lfx): add extension init and extension dev CLIs (LE-1016) (#12968)
* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)
Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.
What's in lfx.extension:
- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
`extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
`routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
None-only so non-null values produce a dedicated
`field-deferred-in-this-milestone` error instead of a generic schema wall.
`bundles` accepts a list but rejects length > 1 with
`multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
per discriminant. Codes are registered in `ERROR_CODES`; an
`ExtensionError` constructed with an unknown code raises at construction
time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
path-safety (no `..`, no absolute paths, no symlink escape), AST
inspection of every `.py` (syntax, Component subclass present, build()
declared, top-level `import *`, top-level I/O primitives), and an opt-in
`--execute-imports` that runs each module in a subprocess with a
temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.
Acceptance-criteria coverage in tests/unit/extension/:
- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
file is never written); `--execute-imports` DOES execute it and the canary
appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
ERROR_CODES and the snapshot table are in lockstep so future additions
cannot ship without a format branch and a snapshot.
Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).
* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader
tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).
Fixes ModuleNotFoundError seen in CI on the 3.10 job.
* Update src/lfx/tests/unit/extension/test_schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/extension/schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/cli/_extension_commands.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)
Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:
- load_extension(root): one manifest, one Bundle, registered at @official.
Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
- discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
user-declared path order). First-wins on duplicate names; second emits
duplicate-inline-bundle warning that names both paths.
Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.
New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.
Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(lfx): address LE-1015 review feedback
- Drop the unproduced duplicate-distribution error code; LE-1022 will add
it once startup-time discovery has a place to surface it. Restores the
invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
was already existence-checked, and a concurrent removal in the narrow
window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(lfx): split extension loader into a small subpackage (LE-1015)
Addresses the file-size feedback from the LE-1015 review. The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:
loader/
__init__.py # re-exports the public surface
_types.py # SLOT constants, LoadedComponent, LoadResult
_discovery.py # filesystem walk + importlib.util orchestration
_detection.py # Component subclass identification (MRO heuristic)
_orchestrator.py # load_extension, discover_inline_bundles
_plugins.py # manifest-first precedence over langflow.plugins
Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline. No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass. ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).
The test suite is split to mirror the package:
tests/unit/extension/loader/
conftest.py # shared fixtures, FakeDist, autouse scrub
test_load_extension.py # @official slot, identity, failure modes
test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot
test_plugins.py # manifest-first precedence helpers
test_types.py # LoadedComponent, LoadResult, code parity
Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016)
The two scaffolding CLIs an Extension author types:
- `lfx extension init <target>` writes the basic single-Bundle
template (manifest with $schema, README, .gitignore, one Component
subclass + a pytest smoke test). AC #1: the generated extension
validates clean against LE-1014. AC #2: the generated test file is
a valid pytest module that exercises the component's build() method.
AC #3: any --template other than 'basic' fails with a typed
template-deferred-in-this-milestone error and a non-zero exit.
Refuses to scaffold over a non-empty target dir.
- `lfx extension dev <target>` validates the local extension, records
its absolute path in <config_dir>/extensions/dev_extensions.json,
prints reload instructions, and execs `langflow run` (or
`python -m langflow` when langflow isn't on PATH). --skip-launch
registers without launching (used by tests + external dev-server
scripts); --skip-validate lets authors register a known-broken
manifest to debug it under the loader.
Stack:
- Wave 0: error codes (extension-target-exists,
extension-target-invalid, local-extension-missing) with format
branches and snapshot tests.
- Wave 1: lfx.extension.init_template -- pure-data scaffolder, no
Typer dependency so the CLI is a thin shell over it.
- Wave 1: lfx.extension.dev_registry -- atomic JSON state file under
the langflow user-cache dir; helpers for register / list /
unregister / load_dev_extensions / dev_extension_component_paths.
- Wave 2: lfx.cli._extension_commands gains init/dev subcommands.
- Wave 2: langflow.main lifespan hook reads the dev registry after
bundle loading and extends components_path with each registered
bundle dir, so the existing palette discovery picks up dev
extensions. Missing paths surface as local-extension-missing
warnings (AC #5) without aborting startup.
Tests (84 new, 197 total in tests/unit/extension/):
- test_init_template.py: AC scenarios + identifier derivation +
deterministic file shape.
- test_dev_registry.py: register/list/unregister round-trip,
idempotent re-register refreshes timestamp, malformed state file
treated as empty, missing-path warning, recovery when path
reappears, env-var override precedence.
- test_cli.py: AC #1 init->validate, AC #3 deferred templates,
--skip-launch registers without launching, --skip-validate
short-circuits the pre-flight pass.
- test_errors.py: snapshot rows for all three new codes; the
every-known-code-has-a-snapshot test enforces parity.
LE-1018 (reload) reuses load_dev_extensions; LE-1022 (installed-pkg
discovery) shares the components_path extension pattern. AC #4 ("boots
Langflow with the extension visible in the palette within 5s") is
delivered jointly by `extension dev` (registers + execs) and the
lifespan hook (loads on startup).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(lfx): address LE-1016 review feedback
Fixes the four HIGH issues + the elevated LOW from the review pass:
1. dev_extension_component_paths now forwards EVERY warning, not just
local-extension-missing. Previously a duplicate-component-name (or
any future warning code) was silently dropped, hiding real signal
from the lifespan hook's logs.
2. Defensive emit when a LoadResult has components but source_path is
None. The current loader always sets source_path, but a future
hand-built LoadResult could violate that contract; we now surface a
typed local-extension-missing error rather than dropping the
extension silently.
3. Replaced the fragile ``min(len(parts))`` bundle-root selection with
a relative-to-source-path measurement. Handles deep-vs-shallow
sibling extensions correctly without depending on absolute path
depth.
4. Generated README now documents the langflow/lfx prerequisite under
the Develop section so authors know `pytest` requires the lfx
environment, not just Python.
5. Forced LANGFLOW_LAZY_LOAD_COMPONENTS=false unconditionally in the
`extension dev` exec env (was setdefault, which let a developer's
global lazy-loading export silently hide their dev components from
the palette and miss AC #4's 5s budget).
Plus three MEDIUMs:
- sys.modules cleanup in test_generated_test_file_runs_against_generated_component
so a later test importing the same dotted path doesn't pick up a
stale module from a deleted tmp_path. Component instantiation
moved inside the try block because Component.__init__ uses
inspect.getsourcefile against self.__class__'s still-live module.
- Added regex-drift test that pins the init_template patterns to
match manifest.py's so a schema regex change can't quietly produce
invalid scaffolded manifests.
- Added two new dev_registry tests covering the forward-all-warnings
contract and the source_path=None defense.
Tests: 201 passing (4 new); ruff check + format clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: LangflowCompat -> LfxCompat
* fix: Remove references to ticket
* fix: encode maxItems constraint in schema
* fix: deferred fields and schema version
* Fix ruff errors
* fix: align loader tests with renamed manifest API and strip ticket refs
The merge of feat/extension-validate brought in the LfxCompat / compat
rename, but the loader test fixtures still used LangflowCompat / bundle_api
and failed at the manifest layer. Update conftest.py + test_load_extension.py
to the post-rename API.
Strip 17 LE-XXXX ticket references from loader source files, errors.py
loader-specific comments, and the four loader test docstrings, matching the
convention applied to LE-1014. Descriptive prose preserved.
Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the
loader's orchestrator no longer reaches across modules into a private name.
* fix(lfx): align init template with renamed manifest API
After merging feat/extension-loader into this branch, two surfaces
fell out of sync with the renamed manifest schema:
- init_template generates extension.json with `lfx: {bundle_api: [1]}`,
but the validator now requires `lfx: {compat: ["1"]}` per LfxCompat.
This made `test_basic_template_validates_clean` fail with
manifest-invalid (`lfx.compat: Field required; lfx.bundle_api: Extra
inputs are not permitted`).
- test_init_template_regexes_match_manifest_schema reads
`manifest_mod._BUNDLE_NAME_RE`, but that symbol was promoted to public
`BUNDLE_NAME_RE` in
|
|||
| 557993f4b4 | [autofix.ci] apply automated fixes | |||
| 882dc4978a |
feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) (#12979)
* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)
Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:
- load_extension(root): one manifest, one Bundle, registered at @official.
Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
- discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
user-declared path order). First-wins on duplicate names; second emits
duplicate-inline-bundle warning that names both paths.
Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.
New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.
Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(lfx): address LE-1015 review feedback
- Drop the unproduced duplicate-distribution error code; LE-1022 will add
it once startup-time discovery has a place to surface it. Restores the
invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
was already existence-checked, and a concurrent removal in the narrow
window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(lfx): split extension loader into a small subpackage (LE-1015)
Addresses the file-size feedback from the LE-1015 review. The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:
loader/
__init__.py # re-exports the public surface
_types.py # SLOT constants, LoadedComponent, LoadResult
_discovery.py # filesystem walk + importlib.util orchestration
_detection.py # Component subclass identification (MRO heuristic)
_orchestrator.py # load_extension, discover_inline_bundles
_plugins.py # manifest-first precedence over langflow.plugins
Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline. No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass. ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).
The test suite is split to mirror the package:
tests/unit/extension/loader/
conftest.py # shared fixtures, FakeDist, autouse scrub
test_load_extension.py # @official slot, identity, failure modes
test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot
test_plugins.py # manifest-first precedence helpers
test_types.py # LoadedComponent, LoadResult, code parity
Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018)
Five-stage reload (parallel staging load -> validate -> swap under write
lock -> cleanup -> emit) for installed Bundles in Mode A. In-flight
flows keep the pre-swap class via existing references; new flows pick up
the post-swap class atomically; concurrent reloads on the same Bundle
are rejected with reload-in-progress.
Adds:
* lfx/extension/registry.py -- BundleRegistry with per-bundle
reload-in-progress guard and components_index.json writer
* lfx/extension/reload.py -- the five-stage pipeline; events emission
is stubbed (TODO LE-1017) so the swap mechanics can ship before the
events service lands
* loader: optional module_namespace param so Stage 1 lands in
__reload_staging__.<id> instead of the live _lfx_ext.* namespace
* errors: four new typed reload codes (reload-in-progress,
reload-bundle-not-installed, reload-bundle-name-mismatch,
reload-source-missing) with branch templates and snapshot tests
* HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by
the existing get_current_active_user dependency, returns 409 with a
typed body for the in-progress collision case
* CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client
against the dev server with text/json output and proper exit codes
(--all is gated until LE-1019 lands the list endpoint)
* tests: 16 reload-pipeline tests covering the AC matrix (rename
round-trip, broken-bundle isolation, concurrent readers, in-flight
flow, double-reload guard, bundle-name mismatch) plus 9 CLI client
tests
Mode A only. In Mode B/C bundle changes require a Docker image rebuild
and the reload path is not exercised.
The events emission in Stage 5 is intentionally stubbed -- the LE-1017
ticket will swap the body of _emit_bundle_reload_event in one place
without touching the pipeline core.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 4c854c9c1e |
feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (#12967)
* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)
Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.
What's in lfx.extension:
- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
`extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
`routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
None-only so non-null values produce a dedicated
`field-deferred-in-this-milestone` error instead of a generic schema wall.
`bundles` accepts a list but rejects length > 1 with
`multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
per discriminant. Codes are registered in `ERROR_CODES`; an
`ExtensionError` constructed with an unknown code raises at construction
time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
path-safety (no `..`, no absolute paths, no symlink escape), AST
inspection of every `.py` (syntax, Component subclass present, build()
declared, top-level `import *`, top-level I/O primitives), and an opt-in
`--execute-imports` that runs each module in a subprocess with a
temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.
Acceptance-criteria coverage in tests/unit/extension/:
- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
file is never written); `--execute-imports` DOES execute it and the canary
appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
ERROR_CODES and the snapshot table are in lockstep so future additions
cannot ship without a format branch and a snapshot.
Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).
* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader
tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).
Fixes ModuleNotFoundError seen in CI on the 3.10 job.
* Update src/lfx/tests/unit/extension/test_schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/extension/schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/cli/_extension_commands.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)
Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:
- load_extension(root): one manifest, one Bundle, registered at @official.
Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
- discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
user-declared path order). First-wins on duplicate names; second emits
duplicate-inline-bundle warning that names both paths.
Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.
New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.
Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(lfx): address LE-1015 review feedback
- Drop the unproduced duplicate-distribution error code; LE-1022 will add
it once startup-time discovery has a place to surface it. Restores the
invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
was already existence-checked, and a concurrent removal in the narrow
window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(lfx): split extension loader into a small subpackage (LE-1015)
Addresses the file-size feedback from the LE-1015 review. The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:
loader/
__init__.py # re-exports the public surface
_types.py # SLOT constants, LoadedComponent, LoadResult
_discovery.py # filesystem walk + importlib.util orchestration
_detection.py # Component subclass identification (MRO heuristic)
_orchestrator.py # load_extension, discover_inline_bundles
_plugins.py # manifest-first precedence over langflow.plugins
Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline. No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass. ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).
The test suite is split to mirror the package:
tests/unit/extension/loader/
conftest.py # shared fixtures, FakeDist, autouse scrub
test_load_extension.py # @official slot, identity, failure modes
test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot
test_plugins.py # manifest-first precedence helpers
test_types.py # LoadedComponent, LoadResult, code parity
Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: LangflowCompat -> LfxCompat
* fix: Remove references to ticket
* fix: encode maxItems constraint in schema
* fix: deferred fields and schema version
* Fix ruff errors
* fix: align loader tests with renamed manifest API and strip ticket refs
The merge of feat/extension-validate brought in the LfxCompat / compat
rename, but the loader test fixtures still used LangflowCompat / bundle_api
and failed at the manifest layer. Update conftest.py + test_load_extension.py
to the post-rename API.
Strip 17 LE-XXXX ticket references from loader source files, errors.py
loader-specific comments, and the four loader test docstrings, matching the
convention applied to LE-1014. Descriptive prose preserved.
Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the
loader's orchestrator no longer reaches across modules into a private name.
* refactor(lfx): clarify broad except, log malformed bundle.json, expand test docstring
- Document why _discovery.import_bundle_module catches BaseException (startup-time loader, must surface bad bundles as typed errors rather than abort).
- Add debug-level logging when bundle.json is malformed or non-object so a stale-cache footgun is at least observable.
- Expand test_skips_re_imported_class docstring so future maintainers don't accidentally weaken the __module__-equality guard if package-style relative imports get added later.
Also drops the stale duplicate-distribution claim from the PR description; that code is correctly deferred to LE-1022 along with /all integration.
* feat(lfx): wire Extension System into /all, pathsep-split LANGFLOW_COMPONENTS_PATH, emit duplicate-distribution
Closes the four AC gaps the previous reviewer flagged on PR #12967:
1. /all integration: get_and_cache_all_types_dict now also calls a new
import_extension_components() that loads installed Extensions via
load_installed_extensions, loads inline bundles via discover_inline_bundles,
and builds frontend-node templates with extension/bundle/extension_version
fields stamped on. Failures are logged and skipped per bundle.
2. LANGFLOW_COMPONENTS_PATH is now split on os.pathsep so multi-entry env vars
(e.g. /a:/b on POSIX) produce multiple components-path entries instead of
one literal non-existent path. Empty segments and missing paths are skipped.
3. duplicate-distribution is a real producer: load_installed_extensions
surfaces a typed warning on the winner LoadResult when two distributions
share a canonical name, naming every involved manifest path.
4. Manifest-first precedence runtime wiring: new filter_component_entry_points
loads each entry-point and only skips ones that resolve to a Component
subclass on a manifest-shipping distribution. plugin_routes.load_plugin_routes
now applies it so non-component entry-points (route registrars) keep
loading per the AC's 'unaffected' promise. Added the previously-missing
AC test for same-distribution component+non-component partition.
Also makes _distribution_canonical_name defensive against MagicMock test seams.
* fix(lfx): register extension components under namespaced ID; promote duplicate-distribution to error
Addresses the latest review of PR #12967:
P1: /all integration now keys the cache inner dict by LoadedComponent.namespaced_id
(ext:<bundle>:<Class>@<slot>) rather than the bare class name. Templates also carry
the namespaced_id as an explicit field so consumers that look at the value (not the
key) still see the canonical address. This is the form the LE-1020 migration table
will rewrite legacy class-name references to.
P2: load_installed_extensions now appends duplicate-distribution to result.errors
instead of result.warnings, so LoadResult.ok=False when two distributions share a
canonical name. The winner's components still appear in result.components so flows
already pinned to them keep working; only the conflict status changes. Updated test
to assert errors + ok=False; added explicit assertion that the winner's components
are still present.
* fix(lfx): installed-distribution discovery accepts pyproject.toml manifest form
Closes the latest review finding on PR #12967: the installed-distribution
scan only looked for extension.json, ignoring distributions whose manifest
lives in [tool.langflow.extension] inside pyproject.toml. The AC explicitly
treats both as valid manifest forms.
_distribution_manifest_path now:
- Returns extension.json immediately when present (preserves precedence
matching load_manifest's discovery order).
- Falls back to pyproject.toml only when extension.json is absent AND the
pyproject's [tool.langflow.extension] section is parseable.
Validation reuses load_manifest itself so the rule lives in exactly one
place; a stray pyproject.toml without the section is correctly ignored.
Tests cover: pyproject-only discovery, pyproject-without-section ignored,
extension.json wins on collision, end-to-end pyproject load at @official,
and pyproject-form manifest-first entry-point suppression.
* refactor(lfx): tighten loader invariants, surface silent skips, harden tests
Addresses the latest review feedback on PR #12967:
Type-level invariants (_types.py):
- LoadedComponent.__post_init__ enforces that @extra components must NOT
carry a distribution. The reverse (@official without distribution) is
permitted because load_extension is also used for dev-mode loads
against a working tree before pip install.
- LoadResult docstring documents the partial-success contract: components
may be non-empty when errors is non-empty (some files imported, others
failed). Callers branching on ok get strict success.
Silent-failure fixes (_orchestrator.py + settings/base.py):
- inline-path-missing: a non-existent / non-dir LANGFLOW_COMPONENTS_PATH
entry now produces a typed warning per skipped path so a typo no
longer yields zero diagnostics. Settings-layer skip bumped from debug
to warning for the same reason.
- bundle-json-invalid: a malformed or non-object bundle.json now surfaces
a typed warning instead of silently rewriting the user-declared
id/version to derived values under the same bundle name.
- no-component-subclass gating uses a call-local counter instead of
result.errors so the diagnostic stays accurate when a future caller
reuses a LoadResult (multi-bundle / batch wrapper scenarios).
Test hardening:
- test_re_imported_class_is_skipped_via_module_filter rewritten to
actually exercise the __module__-equality check via sys.modules
injection; previously passed via module-import-failed (relative-import
failure), which would silently weaken if package registration changes.
- test_user_declared_path_order_is_preserved: AC #8's multi-path order
case (distinct bundles in [path_b, path_a]) was unasserted; added.
- test_inline_module_import_failure_attributes_identity: AC #10's
identity-on-partial-failure was covered for @official but not @extra;
added.
- test_uses_real_distributions_by_default tightened to assert ep
placement (in kept, not in skipped) instead of exact-list equality, so
a future Langflow-shipped manifest doesn't silently flip the assertion.
bumped 64 -> 175 passing extension tests; 20 backend integration tests
still pass.
* fix(lfx): malformed pyproject manifests surface manifest-invalid instead of disappearing
Closes the latest review finding on PR #12967: a pyproject.toml with a
[tool.langflow.extension] section that has missing/invalid required
fields was silently dropped because _pyproject_has_extension_section
ran full schema validation via load_manifest and returned False on
ValueError/TypeError. That conflated 'no section' with 'section
malformed'.
Fix: detect section presence only. _pyproject_has_extension_section now
calls _read_pyproject_extension (TOML parse + key lookup, no schema
check). Behavior:
- Section absent or pyproject TOML unparseable -> False (treat as
regular non-manifest package).
- Section present and is a table (valid OR schema-invalid) -> True.
- Section present but is not a table -> True; the author intended to
declare an extension and load_extension will surface the typed error.
This way a typo'd pyproject Extension produces a typed manifest-invalid
LoadResult with extension_id attribution, and manifest-first precedence
still suppresses its legacy component entry-points -- matching the
'typed load results on success/failure' contract for the supported
pyproject manifest form.
Tests: two new cases pin the behavior. test_malformed_pyproject_section_
surfaces_manifest_invalid asserts a typed load-failure result with
distribution attribution; the second test pins manifest-first
suppression for malformed pyproject distributions.
* refactor(lfx): emit inline-path-unreadable, document reload contract, trim rot
Closes the remaining nits on PR #12967:
- inline-path-unreadable (new typed error code): a configured
LANGFLOW_COMPONENTS_PATH entry that raises OSError on iterdir
(typically permission-denied) now produces a typed LoadResult error
carrying str(exc) instead of silently swallowing the message.
- duplicate-inline-bundle hint trimmed: removed forward promise about
hard-error-in-a-later-release; same actionability, no expiration.
- discover_inline_bundles docstring trimmed from three paragraphs to
two sentences (per CLAUDE.md anti-multi-paragraph rule).
- _distribution_manifest_path docstring trimmed to one-liner.
- installed_extension_roots dropped Used-by caller-narration list;
manifest_owning_distributions docstring rewritten to call out the
shadow-load risk for direct callers and point to load_installed_
extensions for the typed warning surface.
- _discovery.import_bundle_module: replaced 'until that lands in a
later milestone' rot with a single line ('Absolute imports only
between bundle modules; relative imports unsupported.') AND added
the single-load-per-process contract note documenting why LE-1018
reload must scrub registry/sys.modules before re-invoking the loader.
- load_extension docstring grew a 'Single-load-per-process contract'
block telling direct callers not to rely on this function for refresh.
Tests: new test_unreadable_path_emits_inline_path_unreadable pins the
OSError -> typed-error path.
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|||
| 0d0bb83401 | Merge branch 'release-1.10.0' into feat/extension-production-install | |||
| 8a1a1f367b |
feat(lfx): installed-package + seed-directory discovery for production install (LE-1022)
Adds the read-only production install path for Modes A, B, and C of the Bundle Separation iteration. Manifest-shipping pip-installed distributions and seed-directory subdirectories are discovered at server startup and registered as Extensions at @official. * discovery.py: walks importlib.metadata.distributions() + the $LANGFLOW_SEED_DIR / /opt/langflow/bundles seed root; produces DiscoveredExtension records and typed errors for malformed manifests / configured-but-missing seed dirs. * registry.py: ExtensionRegistry service with the immutability invariant for installed and seed entries. Mutation verbs (uninstall, disable, enable, install, update_entry) all raise ExtensionImmutableError carrying the typed installed-extension-immutable / seed-directory-immutable code so the invariant is testable today; the CLI uninstall surface ships in B4. * lfx extension list: read-only inspector with text and JSON output for operators inspecting Mode B/C images. * Errors: four new typed codes (installed-extension-immutable, seed-directory-immutable, seed-directory-not-found, duplicate-extension-id) plus snapshot coverage in tests/unit/extension/test_errors.py. * Tests: 165 extension tests pass, including the LE-1022 acceptance cases -- three pip-installed wheels visible at @official, three seed bundles visible at @official, and the parametrized service-layer immutability check across every mutation verb. * Docs: docs/Deployment/deployment-extensions-production.mdx covers the Dockerfile template, k8s deployment notes, the bundle packaging convention (extension.json shipped via package-data), and troubleshooting for the typed error codes. |
|||
| 3befb61940 |
fix: skip missing server file when delete_after_processing=True to avoid race condition (#12556)
* fix: skip missing server file when delete_after_processing=True to avoid race condition (fixes #8887) * [autofix.ci] apply automated fixes * test(lfx): cover file_path branch for missing-file ValueError Switch the missing-file ValueError test to set self.component.file_path (via a Data object) instead of self.component.path, so the file_path decision branch in load_files_base is exercised when delete_server_file_after_processing=False and silent_errors=False. * fix: cache parsed result so concurrent outputs don't drop server-file data Previously the race-condition fix in load_files_base() returned an empty list when a concurrent output call had already deleted the server file. That avoided the spurious ValueError but silently dropped data — a File component with multiple connected outputs could "succeed" while a downstream node received an empty Data wrapper. Cache the parsed Data list on the component instance after the first successful processing, and reuse it when a subsequent call discovers the file is gone. Each connected output now sees the same parsed content instead of empty data, while still preserving the deletion semantics on the source file. Updates the regression test to assert the cached result is returned (equal to the first call's output) rather than locking in the empty-list behavior. * fix: serialize load_files_base with per-instance lock and key cache by paths The previous race-condition fix had two remaining gaps flagged in review: 1. It was not concurrency-safe. Two threads could both pass _validate_and_resolve_paths before either deleted the source file; the loser would then drop to empty data inside _filter_and_mark_files when the file vanished mid-flight. 2. The post-hoc recovery ("if validation returned no files and any cache exists, return it") was too broad — a legitimate empty input on a later call could pick up a stale cached result for a different server-file state. Address both: - Acquire a per-instance threading.Lock around the entire validate/process/delete cycle so concurrent output methods on the same component are serialized. The first caller does the work; the others observe the cached parsed result. - Cache the parsed Data list keyed by (local paths, server file paths, markdown flag) so different inputs and different processing variants don't share entries. Empty results are not cached, so a legitimate empty input cannot return stale data. - Set an explicit _validate_skipped_due_to_delete_race flag inside _validate_and_resolve_paths so recovery only activates when validation actually skipped a missing delete-after-processing file. Recovery reuses any cached result for the same source paths (across markdown variants) — preserving content beats dropping it when the file is gone. Adds a concurrency test that drives two threads through load_files_base simultaneously and asserts both observe the same parsed data, plus a test that an empty input does not return stale cached data. * fix: eagerly init load_files_base lock in __init__ to close first-entry race Lazy lock creation inside load_files_base() (hasattr-then-assign) is itself racy: on the very first concurrent entry, two threads can each find the attribute missing and create their own threading.Lock, bypassing serialization for the exact race this PR is fixing. Move the lock creation into __init__ so every instance has a single Lock before any output method can run. Removes the lazy-init branch from load_files_base(). --------- Co-authored-by: octo-patch <octo-patch@github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| 104d941dbf |
refactor: refine MCP Server tab layout and alignment (#13032)
improve mcp tab layout |
|||
| ad43a7d0b6 |
feat(agents): structured default system prompt with runtime env injection (#12855)
* default system prompt feature for agent * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * ruff style and checker * [autofix.ci] apply automated fixes * fix tests CI * fix agent merge conflict * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * address QA suggestions * message ordering * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * add guardrailws * Ruff check * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| 1dd95e2902 |
feat(deployments): support multi-version flows (#12950)
* feat(deployments): support multi-version flows
* fix(deployments): use flows in attach step
* refactor(deployments): split review step
* feat(deployments): scope default tool names
* test(deployments): update stepper state tests
* test(deployments): update attach flow tests
* fix(deployments): preserve removed flow state
* fix(deployments): keep stepper provider mounted
* fix(deployments): sync attached flow review
* refactor(deployments): share scoped lookups
* refactor(deployments): type version selection
* refactor(deployments): split payload builders
* chore(deployments): standardize labels
* refactor(deployments): isolate wxo naming
* refactor(deployments): simplify attach checks
* chore(deployments): drop stale effect ignores
* refactor(deployments): remount edit stepper
* fix(tests): update tests for effectiveAttachmentKey rename and scoped tool names
- use-connection-panel-state.test.ts: rename effectiveFlowId → effectiveAttachmentKey
in baseParams() to match hook's refactored param name
- deployment-create.spec.ts: make SNAPSHOTS_DUPLICATE_MOCK route dynamic — echo back
requested names as existing tools so duplicate check works with scoped names
(getDefaultDeploymentToolName now generates "Flow {scope}-{id}" format)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
|
|||
| 9ba0976ff2 |
fix: remove stepper modal close button (#12986)
fix(deploy): remove stepper modal close button |
|||
| 07c3e67a71 |
fix(security): prevent arbitrary file read via public flow build endpoint (GHSA-rcjh-r59h-gq37) (#13029)
* fix(security): prevent arbitrary file read via public flow build endpoint (GHSA-rcjh-r59h-gq37)
The unauthenticated `POST /api/v1/build_public_tmp/{flow_id}/flow`
endpoint accepted a `files` array whose paths were resolved by
`LocalStorageService` and read off disk before being attached to the
LLM prompt. A path like `/etc/hosts` was split into `flow_id="/etc"`
and `file_name="hosts"`. `data_dir / "/etc"` collapsed to `/etc`
because joining an absolute segment resets the prefix, and the
containment check was anchored at that attacker-controlled folder
so it passed. The file contents were then echoed back through the
chat reply.
Fix layers:
1. Boundary validation in `build_public_tmp`: each `files` entry
must match `{source_flow_id}/{basename}` with no traversal,
backslash, or null bytes. Foreign flow_ids are rejected with
HTTP 400.
2. `LocalStorageService._validated_path` now also rejects
`flow_id` values containing path separators, traversal, or
null bytes, and anchors the containment check at the data
directory rather than the per-flow folder.
Adds regression tests for both layers covering `/etc/hosts`,
traversal, foreign-namespace and null-byte payloads, plus an
acceptance test that legitimate `{flow_id}/{filename}` references
still work for AUTO_LOGIN playground uploads.
* add identifier validation
---------
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
|
|||
| 683b5b2299 |
docs: update mdc file to reflect current usage (#13019)
docs-update-mdc-file-for-current-usage |
|||
| 7504eb4c72 |
fix: pin postgres image to bookworm in docker_example (#13027)
* fix: pin postgres image to bookworm in docker_example to prevent collation mismatch The postgres:16 tag silently moved its base from Debian Bookworm (glibc 2.36) to Trixie (glibc 2.41), causing a recurring collation version mismatch warning on existing langflow-postgres volumes. Pin to postgres:16-bookworm in both docker-compose files and update the README so existing data volumes keep matching the OS locale data. Refs: https://github.com/langflow-ai/langflow/issues/9608 * docs: pin postgres image to bookworm in current docs compose snippets Mirror the docker_example pin in the four current docs that publish copyable Compose snippets pairing postgres:16 with a persistent langflow-postgres volume. Prevents the same Bookworm-to-Trixie collation version mismatch warning when users follow docs instead of docker_example. Versioned historical docs are left as-is. Refs: https://github.com/langflow-ai/langflow/issues/9608 * fix: switch postgres pin from bookworm to trixie for OS consistency Aligns the pinned postgres base with the langflow runtime image, which moved to Debian Trixie in #12990. Keeping postgres on bookworm would have diverged the stack and locked the database to an aging glibc that will receive fewer security backports as bookworm ages into oldstable. The pin itself still solves the original bug — postgres:16 cannot silently roll its OS underneath an existing volume. Document the one-time REFRESH COLLATION VERSION step for users upgrading from a bookworm-initialized volume in docker_example/README.md. Refs: https://github.com/langflow-ai/langflow/issues/9608 |
|||
| b707c9a41d |
docs: opensearch connector feature (#12998)
* docs-add-opensearch-provider-and-adjust-kb-docs * docs-combine-kb-config-sections-and-update-partial * add-release-note * Apply suggestions from code review Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com> * docs-clarify-embedding-model-step --------- Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com> |
|||
| 406b372d7a |
chore(starter-projects): new Vector Store RAG template (#13018)
* chore(starter-projects): new Vector Store RAG template Update the RAG starter flow to align with the new knowledge base paradigm: removes the inline knowledge base creation sub-flow and points users to the dedicated Knowledge Ingestion UI for loading content into their vector store. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(starter-projects): tweak Vector Store RAG README description Drop the outdated reference to two sub-flows in the in-template README and refresh the saved metadata. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: autofix of starter project json --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| abbe917811 | test: fix flow state clean test on windows (#13013) | |||
| 1217ced9e1 |
Fix: Migration 1.9.2 to 1.10.0 back to 1.9.2 idempotency issue (#12991)
* Fixing migration 1.9.2 -> 1.10.0 -> 1.9.2 idempotent. * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| af8871cf61 |
fix: force pull latest base images in Docker nightly builds (#13014)
- Add pull: true to all docker build steps in nightly workflow - Forces Docker to pull latest python:3.12-slim-trixie instead of using cached layers - Ensures Debian Trixie base images are used instead of cached Bookworm layers - Resolves issue where Docker layer caching prevented CVE fixes from taking effect |
|||
| 30ac789ca1 |
fix(frontend): default event_delivery to streaming to match server config (#12978)
Frontend was forcing event_delivery=polling in the build query, overriding LANGFLOW_EVENT_DELIVERY=streaming on the server. Token-by-token streaming in the Playground never worked because the frontend always took the polling code path (customPollBuildEvents) instead of SSE (performStreamingRequest). Aligns three default fallbacks with the server doc default (STREAMING): - buildUtils.buildFlowVertices: query-param default - utilityStore.eventDelivery: initial store value (race window before /config) - use-get-config: fallback when /config omits event_delivery buildFlowVerticesWithFallback already retries with POLLING on STREAMING_NOT_SUPPORTED / ENDPOINT_NOT_AVAILABLE, so streaming-first is safe. Fixes #12291 |
|||
| 7c347a56ea |
docs: release note refactor (#12987)
* docs-1.10-release-notes-and-link-out-to-previous-versions * docs-remove-outdated-windows-upgrade-note |
|||
| 92e21615c6 | fix(tests): Stabilize traces and user-flow-cleanup e2e on Windows CI (#12997) | |||
| 0aea910dac |
fix(frontend): paginator empty state shows "0 items" instead of "1-0 of 0 items" (#12965)
* fix(frontend): paginator empty state shows "0 items" instead of "1-0 of 0 items" When totalRowsCount is 0 the paginator rendered "1-0 of 0 items" and "of 0 pages", with the next button enabled because pageIndex (1) was not equal to maxIndex (0). Hide the range when empty, floor maxIndex to 1, and disable both navigation buttons when there is nowhere to go. * Update legacy paginator tests to assert new empty-state copy |
|||
| deecfc68fa |
fix(mcp): make run_tool wait_for timeout configurable, raise floor to… (#12996)
fix(mcp): make run_tool wait_for timeout configurable, raise floor to 180s
The hardcoded `timeout=30.0` on `asyncio.wait_for(session.call_tool(...))`
in both `MCPSseClient.run_tool` and `MCPStreamableHttpClient.run_tool` was
truncating in-flight tool calls whose servers needed longer than 30s to
respond — most visibly large MCP composers whose tool responses include
sizeable catalogs.
Failure mode:
LLM → AgentExecutor → run_tool (1 call from the LLM)
├── attempt 1: get_session (probe ..._1 fails → swap ..._2)
│ → call_tool → 30s wait_for fires, TimeoutError raised
│ (server is still working — response ignored on arrival)
└── attempt 2: get_session (probe ..._2 fails → swap ..._3)
→ call_tool → 30s wait_for fires again
→ util.py dedup ("Repeated TimeoutError, not retrying")
→ ValueError("Maximum retries exceeded ...")
← AgentExecutor surfaces error → flow fails
The retry loop's TimeoutError dedup correctly stops looping once it sees
the same error twice, but every call against a slow server hit the same
ceiling and never had a chance to succeed. The session swaps between
attempts also produced excess HTTP churn against the server.
Both `run_tool` methods now resolve the timeout from
`get_settings_service().settings.mcp_server_timeout` (already used for
connection setup at lines 1617 / 1887, env: `LANGFLOW_MCP_SERVER_TIMEOUT`)
with a `max(setting, 180.0)` floor so default deployments don't end up
*shorter* than the previous hardcoded 30s — the Pydantic default for
`mcp_server_timeout` is 20.
Touched:
src/lfx/src/lfx/base/mcp/util.py
- L1677, L1689 (stdio run_tool)
- L1960, L1972 (streamable HTTP run_tool)
Unchanged on purpose:
- L1217, L1381 session-creation `timeout=30.0` (separate concern).
- L1060 `_validate_session_connectivity` 3.0s probe (separate
concern; addressed in follow-up if probe thrash persists).
Tuning:
- LANGFLOW_MCP_SERVER_TIMEOUT now governs both connection setup and
tool-call wait_for. The 180s floor clamps low values; raise the env
var above 180 to take effect.
Repro / verification:
- Trace run that previously failed at 67.08s with "Maximum retries
exceeded with repeated TimeoutError errors" against
`mcp-guardium_get_service_info` (47KB response payload) now
completes on attempt 1.
Co-authored-by: MANSURA HABIBA <MANSURAH@ie.ibm.com>
|
|||
| 4b46b57e5c | fix: Ensure direct-uvicorn startup in macOS | |||
| 78f82cae32 |
perf: Enhance Gunicorn preload functionality for Langflow (#12778)
* perf: Enhance Gunicorn preload functionality for Langflow
- Introduced a new preload module to optimize memory usage by running fork-safe initialization in the Gunicorn master process.
- Updated the lifespan management in `main.py` to check if the master has preloaded resources, allowing workers to inherit state and skip redundant setup.
- Adjusted the server loading process to accommodate the new preload logic, ensuring efficient resource management across worker processes.
* feat(cache): Implement teardown method for RedisCache to prevent socket leaks
- Added a `teardown` method to the `RedisCache` class to close the Redis connection, addressing potential socket leaks during process forking.
- Introduced unit tests to verify the functionality of the `teardown` method, ensuring it handles client closure and errors gracefully.
- Tests cover scenarios including normal closure, error handling during closure, and teardown with URL-based connections.
* feat(cache): Implement teardown method in RedisCache to prevent socket leaks
- Added a `teardown` method to the `RedisCache` class, ensuring proper closure of the Redis client connection before forking to avoid socket leaks.
- Created comprehensive unit tests to validate the functionality of the `teardown` method, covering various scenarios including normal operation and error handling.
- Updated existing tests to reflect the new teardown functionality and ensure RedisCache is recognized as an instance of `ExternalAsyncBaseCacheService`.
* refactor(preload): Simplify DB engine disposal logic in master preload
- Removed exception handling around the DB engine disposal to streamline the process, ensuring that the engine is disposed of without unnecessary error logging. This change enhances code clarity and maintains the intended functionality of resource management during the preload phase.
* refactor(preload): Streamline cache service teardown logic
- Simplified the cache service socket closure process in the master preload function to prevent sharing across forks. This change enhances code clarity by removing unnecessary exception handling while maintaining the intended functionality of resource management during the preload phase.
* feat(preload): Introduce per-step completion flags for improved state management
- Added completion flags in the preload state to track the status of various initialization steps, including profile picture copying, starter project creation, agentic global variable initialization, MCP server configuration, and flow loading.
- Updated the lifespan management in `main.py` to utilize these flags, allowing the system to skip redundant setup tasks if they have already been completed during the preload phase.
- This enhancement improves resource management and ensures that the application behaves correctly in a multi-worker environment.
* refactor(lifespan): Replace temp dir management with get_owned_temp_dirs function
- Updated the lifespan management in `main.py` to utilize the new `get_owned_temp_dirs` function, which encapsulates the logic for determining temp directory ownership based on the process type (master or worker).
- Removed the `is_master` check from the lifespan function, simplifying the code and enhancing clarity regarding temp directory cleanup responsibilities.
- Added the `get_owned_temp_dirs` function in `preload.py` to centralize temp directory ownership logic, ensuring that workers do not attempt to clean up directories owned by the master process.
* refactor(lifespan): Enhance initialization flow with conditional gates
- Introduced conditional gates in the `get_lifespan` function to manage the initialization of profile pictures, super users, bundles, component types, and starter projects based on their completion status.
- Improved logging to provide clearer insights into which steps are being skipped or executed, enhancing the overall clarity of the initialization process.
- Updated the preload logic in `preload.py` to ensure that agentic global variables and MCP server configuration are only initialized when necessary, maintaining efficient resource management in a multi-worker environment.
* fix: resolve double-call of initialize_auto_login_default_superuser and add preload tests
- Fix double-call issue: setup_superuser() now handles AUTO_LOGIN completely with file lock
- Add comprehensive unit tests for preload.py covering failure-fallback contract
- Simplify code by doing superuser initialization in initialize_services() (called early in both preload and worker startup)
- File lock protects multi-worker race conditions when preload is disabled
- Tests verify critical step failures propagate, best-effort steps continue on failure
Made-with: Cursor
* fix(preload): resolve missing import and agentic initialization conflicts
Fixed critical bugs introduced in
|
|||
| fae08ab66a |
feat: Added API support for BE endpoints (#12423)
* add memories * add memories * accessibility and bug fixes * fix types * fix types * [autofix.ci] apply automated fixes * add new testcases * [autofix.ci] apply automated fixes * remove auto capture * [autofix.ci] apply automated fixes * coderabbit fixes and remove KB from FE * added enpoint support * coderabbit fix * [autofix.ci] apply automated fixes * added session and memory messages endpoint * code rabbit fixes * [autofix.ci] apply automated fixes * merge session id fix * [autofix.ci] apply automated fixes * fix testcases * [autofix.ci] apply automated fixes * fix testcases * fix traces sidebar * memory query rewrite and testcase improvement * [autofix.ci] apply automated fixes * Improve seperation of concern for memory hook * [autofix.ci] apply automated fixes * handleToggleActive Signature Breaking Change * improve testcase * [autofix.ci] apply automated fixes * fix testcases * fix mcp testcase * disable embedded option trigger causing user to to be able to switch provider when option is zero * Peer review updates * [autofix.ci] apply automated fixes * added refresh logic and fixed prior issues from CR * [autofix.ci] apply automated fixes --------- Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local> |
|||
| 596e98b9b7 |
fix: Respect proxy environment variables in URL Component (#12989)
* fix: URLComponent ignores proxy in async mode (#12285) The URLComponent fails to connect for users behind corporate proxies because its asynchronous mode does not recognize standard system proxy environment variables. The component defaults to use_async=True, which initializes the RecursiveUrlLoader; the underlying async loader does not natively respect system proxy environment variables, so the component attempts a direct connection and fails in restricted network environments. Detect standard proxy environment variables (http_proxy, HTTP_PROXY, https_proxy, HTTPS_PROXY) in URLComponent._create_loader. If a proxy is detected and use_async is enabled, override use_async to False so the loader uses its synchronous implementation, which natively respects system proxies. Empty and whitespace-only values are correctly evaluated as no-proxy and do not trigger the fallback. Closes #10297 * fix(URLComponent): broaden proxy detection and harden tests Address review findings on the proxy fix: - Add ALL_PROXY and all_proxy to the detected env vars. ALL_PROXY is commonly set in corporate and container environments (curl, git, many Unix tools honour it), and is sometimes the only proxy var configured. - Replace the unused proxy_url string with a single boolean any() over the env var keys, since only the presence of a proxy is consulted. - Reorganize the proxy tests under TestURLComponentProxyHandling with a shared default-attributes helper, parametrize over all six env var spellings, and add coverage for multiple-simultaneous-proxies and the use_async=False path (which should not log a warning). * [autofix.ci] apply automated fixes * fix: remove no_proxy="*" macOS startup hack so corporate proxies work Two startup paths set `os.environ["no_proxy"] = "*"` on macOS, which disables proxy use for every HTTP client in the process and every child gunicorn worker (httpx, requests, urllib3 — and therefore the OpenAI, Anthropic, Groq, etc. SDKs that wrap them). For users behind corporate proxies on macOS this made every external LLM call unroutable, even with HTTPS_PROXY properly set. The override traces to commit history with no concrete justification — the only artifact is a Stack Overflow link about a uWSGI segfault, but Langflow uses gunicorn, not uWSGI. Bench-verified that gunicorn boots cleanly and serves requests on macOS without it (1 worker via LangflowApplication, OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES retained, HTTPS_PROXY survives intact in parent and child). Verification: - gunicorn worker spawns and serves /health (200 OK), exits cleanly - httpx, urllib, requests all resolve HTTPS_PROXY after the macOS init (previously: all three returned empty proxy maps because of no_proxy=*) - OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES still set — the actual fork-safety fix is preserved Closes the macOS half of #10297. Companion fix for the async URLComponent half is in #12285 / branch pr-12285-rebased. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Orjson update --------- Co-authored-by: Diogo Veiga <diogo.veiga@tecnico.ulisboa.pt> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| c4d7a247f4 |
fix: update Docker base images to latest Python 3.12 and Debian Trixie to resolve CVEs (#12990)
* fix: update Docker base images to latest Python 3.12 to resolve CVEs - Update all Dockerfiles from python:3.12.13-slim-trixie to python:3.12-slim-trixie - This ensures automatic security patches for Debian base image CVEs - Affects nightly builds (base, main, main-all) and release builds (backend, ep) - Using unpinned patch version (3.12 vs 3.12.13) follows security best practices * fix: update builder stage to Debian Trixie for consistency - Update all Dockerfiles from bookworm-slim to trixie-slim in builder stage - Ensures consistency between builder and runtime Debian versions - Eliminates CVEs in both build and runtime environments - Uses ghcr.io/astral-sh/uv:python3.12-trixie-slim for all builders |
|||
| 364fd5dd9a |
fix: Notifications overlap the bottom-center menubar (#12946)
fix: raise CanvasBanner above bottom-center menubar to prevent overlap Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local> |
|||
| 32308507e8 |
feat: Frontend memory (#12293)
* add memories * add memories * accessibility and bug fixes * fix types * fix types * [autofix.ci] apply automated fixes * add new testcases * [autofix.ci] apply automated fixes * remove auto capture * [autofix.ci] apply automated fixes * coderabbit fixes and remove KB from FE * fix testcases * [autofix.ci] apply automated fixes * fix testcases * fix mcp testcase --------- Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local> |
|||
| 017cb1f93c |
feat(assistant): Add Refresh List and Manage Model Providers to model selector (#12985)
* add model provider config on model selection * [autofix.ci] apply automated fixes * fix fe tests playwright --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| d04debcb2e |
feat: native structured output on AgentComponent with prompt fallback (#12935)
* initial commit for structural response * add schema preprocessing tests * fix agent duplication message * add playwright tests * [autofix.ci] apply automated fixes * chore: auto-bake note keys and regenerate backend locales/en.json [skip ci] * ruff fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
|||
| b8a2b126ca |
feat(lfx): extension manifest schema, validate CLI, and error formatter (LE-1014) (#12952)
* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> |
|||
| 10745ac3a5 |
fix: encode non-ASCII filenames in Content-Disposition headers (RFC 5987) (#12918)
* fix: encode non-ASCII filenames in Content-Disposition headers
HTTP headers only support ASCII/latin-1. Placing Chinese or other
non-ASCII characters directly in filename="..." caused a latin-1
serialization error (500) on download.
Fix all four download routes to use an ASCII-safe fallback in
filename= and the URL-encoded form in filename*=UTF-8'':
- api/v1/files: /download/{flow_id}/{file_name}
- api/v1/flows_helpers: /flows/download/ (ZIP)
- api/v2/files: /files/{file_id} and /files/batch/
Also fix the frontend filename extraction regex in both
use-get-download-flows.ts and use-get-download-files.ts to parse
RFC 5987 (filename*=UTF-8''<encoded>) with decodeURIComponent,
falling back to plain filename= for older responses.
Closes #8105
* test: add non-ASCII filename encoding tests for file download routes
Cover the RFC 5987 Content-Disposition encoding fix across all
affected download endpoints:
- api/v1/files: parametrized test with Chinese, Japanese, accented
Latin, and ASCII filenames verifying filename*=UTF-8'' encoding
- api/v2/files: same parametrized test for single-file download plus
a batch download test verifying the ZIP Content-Disposition header
- flows download: test with Chinese flow names verifying the ZIP
response uses a decodable RFC 5987 filename
Reproduces the regression from issue #8105 where downloading files
with non-ASCII names returned HTTP 500.
* fix: use safe="" in quote() for RFC 5987 filename encoding and tighten test assertion
Ensures forward slashes and other special characters are percent-encoded
in Content-Disposition filename* values. Also fixes weak in-operator
assertion in v2 test to check the correct direction.
* fix: centralize RFC 5987 Content-Disposition encoding and patch header injection
Extract build_content_disposition() helper in api/utils/core.py that:
- Escapes double-quotes in the ASCII fallback (prevents parameter smuggling
via filenames like evil"; x=injected)
- Uses quote(safe="") to percent-encode forward slashes
- Returns the dual-param format (filename= + filename*=) for broad client
compatibility
Replace the duplicated 3-line encoding blocks in v1/files.py, v2/files.py,
flows_helpers.py, and projects_files.py with a single call to the helper.
projects_files.py was missing from the original fix: it used quote() without
safe="" and emitted only filename*= with no ASCII fallback.
* test: strengthen Content-Disposition header assertions
- v2/test_files.py: expected_rfc5987 now includes the file extension so a
regression that strips the extension from the RFC 5987 value is detected
- test_database.py: strengthen the dual-flow download assertion to verify
both filename= and filename*= params are present, not just the prefix
- test_database.py: rename test_download_flows_non_ascii_content_disposition
to test_download_flows_content_disposition_dual_param and update the
docstring to accurately reflect that the ZIP filename is always ASCII
(timestamp-based); the test now verifies the dual-param format
* fix: sanitize control chars and escape backslash in build_content_disposition
Strip ASCII control characters (CR/LF/NUL and others in 0x00-0x1f/0x7f range)
from filename before encoding to prevent header injection. Also escape backslash
before double-quote in the ASCII fallback per RFC 6266 §4.1 quoted-pair grammar.
* test: add adversarial tests for build_content_disposition and strengthen v2 assertion
Add direct unit tests for the helper covering: quote escaping, backslash escaping,
CRLF/NUL/control char stripping, empty filename, and very long names. Replace
weak substring assertions in the v2 single-file download test with exact equality
against the stored filename returned by the upload response.
* refactor: extract Content-Disposition parser to shared frontend util
Extract the duplicated 14-line RFC 5987 parsing block from use-get-download-files
and use-get-download-flows into a single parseContentDispositionFilename util.
Wraps decodeURIComponent in try/catch to gracefully fall back to the legacy
filename= param on malformed percent-sequences. Adds Jest tests covering all
branches: RFC 5987 priority, legacy fallback, CJK decode, null header, and
malformed input.
* test: replace Portuguese test filename with English equivalent
Rename the accented Latin test case from the Portuguese
arquivo_com_acentuação.txt to naïve_résumé.txt so all test
data uses English while still exercising the same ï/é encoding path.
|
|||
| 52ae55b20a |
docs: deprecate voice to voice websocket endpoint (#12970)
* docs-deprecate-voice-to-voice-endpoint * docs-explain-voice-mode-vs-speech-to-text |
|||
| 73b30f588b |
fix(frontend): render node dropdowns above build/notification overlays (#12973)
* fix(frontend): render node dropdowns above build/notification overlays ReactFlow nodes apply `transform` which creates a local stacking context. The model selector and generic node dropdown previously rendered without a Portal, so their `z-50` popover content was trapped inside that context and got covered by the build status / error notification panel (also `z-50`, sibling of the canvas). Switch both `dropdownComponent` and `modelInputComponent` to always use the portalled `PopoverContent` and bump the popover layer to `z-[60]`. Mounting the popover into `document.body` escapes the node's stacking context, and the higher z-index keeps it above the build panel regardless of portal mount order. * fix(frontend): flip node dropdowns above build/notification overlays Prior commit moved the dropdown popovers to a Portal so they could escape the node's stacking context, which fixed the overlap with the build status / error notification panel but also pulled the popovers out of the React Flow viewport's CSS scale transform — the dropdowns no longer scaled with canvas zoom and looked oversized when zoomed out and too small when zoomed in. Switch to a non-portal-based fix: keep the popovers inline in the node (so they continue to scale with canvas zoom like the rest of the node) and let Radix flip them above the trigger when the build panel is on screen. - Subscribe to `useFlowStore` for `isBuilding` and `buildInfo` and derive a `showingBuildPanel` flag in `dropdownComponent` and `modelInputComponent`. - Restore the original `PopoverContentWithoutPortal` selector in both files (children / editNode / inspectionPanel keep portaled content). - Always pass `avoidCollisions` and a dynamic `collisionPadding.bottom` equal to `BUILD_PANEL_COLLISION_PADDING_PX` (160) when the panel is visible, otherwise `0`. Radix then auto-flips the popover above the trigger whenever the bottom region is reserved. - Move `BUILD_PANEL_COLLISION_PADDING_PX` into `constants/constants.ts` with a comment so the magic number is defined once and documented. |
|||
| 41083413c0 |
fix: Duplicate lines rendered in the tracing view (#12944)
flush border top Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local> |
