mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-25 00:59:07 +08:00
77e7a93553
* feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014)
Foundation for the Bundle Separation iteration. Defines what a valid
extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema),
ships the offline `lfx extension validate` command, and ships the single
`format_extension_error` function that every other extension-system module
will use to render structured errors.
What's in lfx.extension:
- `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with
`extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`,
`routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as
None-only so non-null values produce a dedicated
`field-deferred-in-this-milestone` error instead of a generic schema wall.
`bundles` accepts a list but rejects length > 1 with
`multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader
re-checks at install time in LE-1015).
- `schema.build_schema()` + `build_schema_json()` produce the publishable
artifact at schemas.langflow.org/extension/v1.json.
- `ExtensionError` typed envelope and `format_extension_error` -- one branch
per discriminant. Codes are registered in `ERROR_CODES`; an
`ExtensionError` constructed with an unknown code raises at construction
time, preventing producers from shipping without a matching renderer.
- `validate_extension` runs four passes: manifest discovery + schema,
path-safety (no `..`, no absolute paths, no symlink escape), AST
inspection of every `.py` (syntax, Component subclass present, build()
declared, top-level `import *`, top-level I/O primitives), and an opt-in
`--execute-imports` that runs each module in a subprocess with a
temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env
vars stripped.
- `lfx extension validate` and `lfx extension schema` typer subcommands.
Acceptance-criteria coverage in tests/unit/extension/:
- Round-trips every v0 manifest field; deferred fields rejected; multi-bundle
rejected with the dedicated discriminant.
- JSON Schema validates the v0 example and rejects 12 malformed manifests
with distinct error paths (>= 10 required by the ticket).
- Median default-validate runtime < 100ms on the basic template.
- Crafted side-effect bundle: default validate does NOT execute it (canary
file is never written); `--execute-imports` DOES execute it and the canary
appears, while LANGFLOW_* env vars are NOT inherited by the subprocess.
- Snapshot tests for every code in ERROR_CODES; a guard test verifies
ERROR_CODES and the snapshot table are in lockstep so future additions
cannot ship without a format branch and a snapshot.
Wiring: `lfx extension` is a sub-app under the Authoring help panel so
future tickets (LE-1016 init/dev, LE-1018 reload) can attach without
colliding with the existing `lfx validate` (which validates flow JSON, not
extensions).
* fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader
tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the
existing tomli runtime dependency as the 3.10 fallback (same API, so the
import alias keeps the rest of the module unchanged).
Fixes ModuleNotFoundError seen in CI on the 3.10 job.
* Update src/lfx/tests/unit/extension/test_schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/extension/schema.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update src/lfx/src/lfx/cli/_extension_commands.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015)
Introduces lfx.extension.loader: the runtime that turns an Extension on disk
into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in:
- load_extension(root): one manifest, one Bundle, registered at @official.
Re-checks multi-bundle at runtime (defense-in-depth vs. the schema).
- discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH
is a Bundle at @extra. Walk order is platform-independent (sorted dirs,
user-declared path order). First-wins on duplicate names; second emits
duplicate-inline-bundle warning that names both paths.
Manifest-first precedence helpers (installed_extension_roots,
manifest_owning_distributions, filter_plugin_entry_points) let callers of
the legacy langflow.plugins entry-point loader skip distributions that ship
a manifest, so component entry-points are not double-registered.
New typed error codes: module-import-failed, duplicate-component-name,
duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid.
Each ships with a format branch and a snapshot test.
Tests cover the AC: single-bundle happy path, multi-bundle rejection,
missing/empty/no-Component bundle, duplicate class names, deterministic
walk order, recursive discovery, inline-bundle first-wins + dot-dir skip
+ bundle.json metadata, manifest-first precedence partition, PEP-503
distribution-name canonicalization.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(lfx): address LE-1015 review feedback
- Drop the unproduced duplicate-distribution error code; LE-1022 will add
it once startup-time discovery has a place to surface it. Restores the
invariant that every code in ERROR_CODES has a producer.
- Clarify that intra-bundle relative imports are NOT supported in v0; only
absolute references between bundle modules work in this milestone.
- Use strict=False when re-resolving bundle_root in the walker; the path
was already existence-checked, and a concurrent removal in the narrow
window should not raise across the loader's public boundary.
- Hoist the json import out of _read_inline_bundle_json's body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(lfx): split extension loader into a small subpackage (LE-1015)
Addresses the file-size feedback from the LE-1015 review. The single
870-line loader.py becomes a flat package keyed off the four section
banners that already existed inline:
loader/
__init__.py # re-exports the public surface
_types.py # SLOT constants, LoadedComponent, LoadResult
_discovery.py # filesystem walk + importlib.util orchestration
_detection.py # Component subclass identification (MRO heuristic)
_orchestrator.py # load_extension, discover_inline_bundles
_plugins.py # manifest-first precedence over langflow.plugins
Largest file is now _orchestrator.py at 440 LOC (was 870); every file is
well under the 800-LOC project guideline. No behavior change: the public
import paths from lfx.extension are unchanged, all 38 loader tests still
pass. ``_canonicalize_distribution`` is now exported as
``canonicalize_distribution`` from ``loader._plugins`` (it's a stable
PEP-503 helper that downstream modules will reach for, so it loses the
private underscore).
The test suite is split to mirror the package:
tests/unit/extension/loader/
conftest.py # shared fixtures, FakeDist, autouse scrub
test_load_extension.py # @official slot, identity, failure modes
test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot
test_plugins.py # manifest-first precedence helpers
test_types.py # LoadedComponent, LoadResult, code parity
Each test file imports its own slice of the public API and pulls fixtures
from conftest, so a reader looking at one banner can read it in isolation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: LangflowCompat -> LfxCompat
* fix: Remove references to ticket
* fix: encode maxItems constraint in schema
* fix: deferred fields and schema version
* Fix ruff errors
* fix: align loader tests with renamed manifest API and strip ticket refs
The merge of feat/extension-validate brought in the LfxCompat / compat
rename, but the loader test fixtures still used LangflowCompat / bundle_api
and failed at the manifest layer. Update conftest.py + test_load_extension.py
to the post-rename API.
Strip 17 LE-XXXX ticket references from loader source files, errors.py
loader-specific comments, and the four loader test docstrings, matching the
convention applied to LE-1014. Descriptive prose preserved.
Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the
loader's orchestrator no longer reaches across modules into a private name.
* feat(lfx): append-only migration table + flow deserializer rewrite hook
Adds the migration layer of the Extension System: an append-only JSON table
that maps three legacy component reference shapes (bare class name, old
import path, pre-Phase-A namespaced slot) to the post-Phase-A canonical
ext:<bundle>:<Class>@<slot> identifier, plus a deserializer hook that
rewrites a saved-flow payload in place against that table on load.
What landed:
* lfx.extension.migration.schema -- Pydantic models for MigrationEntry +
MigrationTable with per-entry validators (exactly one of bare/import/slot
populated; canonical target shape) and table-level uniqueness check.
* lfx.extension.migration.loader -- canonical in-repo path, threadsafe
process-lifetime cache, typed errors on every failure mode.
* lfx.extension.migration.rewrite -- node-by-node rewrite, idempotent on
canonical refs, difflib-backed closest-match suggestion for unmapped
references, cross-bucket ambiguity surfaces component-name-ambiguous
instead of silently loading into the wrong bundle.
* Wired into Graph.from_payload before validate_flow_for_current_settings
so every saved-flow load goes through migration first.
* scripts/migrate/check_migration_append_only.py -- CI guard that diffs
the working-tree table against origin/main and rejects removals or
target mutations; reordering and additions are allowed.
* 34 new unit tests covering rewrite paths, loader failure modes, schema
invariants, and the CI script behavior.
What is deliberately deferred:
* flow-migrated event emission. The events pipeline is unavailable in this
iteration; the wiring point in Graph.from_payload is marked TODO and the
MigrationReport already carries every field a future emitter needs.
The shipped migration_table.json starts empty; entries land alongside the
pilot bundle extraction in a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>