Files
langflow/scripts/ci/check_components_frozen.py
Eric Hare 2b5faf2c8d ci(bundles): freeze lfx/components against new top-level providers (#13567)
* feat(bundles): add lfx-bundles metapackage skeleton

Creates the manifest-less lfx-bundles distribution (the langchain-community
model) that the long-tail providers will move into. Empty skeleton for now;
the bulk move (scripts/migrate/consolidate_bundles.py) populates the provider
folders + per-provider extras later.

- src/bundles/lfx-bundles: a single pyproject declaring the lfx.bundles
  entry-point (lfx_bundles = "lfx_bundles"), an lfx>=1.10.0,<2.0.0 pin, and a
  generated (currently empty) `all` extra; plus a bare lfx_bundles namespace
  package and a README documenting the model + install stories
- wired into the root workspace via the existing bundle marker blocks:
  dep lfx-bundles[all]>=1.0,<2.0, uv source, and member
- hyphen dir name so release.yml's src/bundles/*/pyproject.toml glob builds
  it with zero workflow change

Verified: the wheel builds (entry_points.txt carries the lfx.bundles group);
load_lfx_bundles_extensions (PR-1) discovers the entry point and the empty
skeleton registers zero providers with no error.

* ci(bundles): freeze lfx/components against new top-level providers

After the metapackage split, new providers go to lfx-bundles (or a graduated
lfx-<provider>), never in-tree. Adds an additions-only CI gate.

- scripts/ci/check_components_frozen.py: stdlib gate comparing the live
  top-level dir listing of src/lfx/src/lfx/components/ against a committed
  baseline; fails on any directory not in the baseline. Removals are allowed
  so M4 shim cleanup (which shrinks the set) never trips it.
- scripts/ci/frozen_component_dirs.txt: the 111-dir baseline.
- .github/workflows/lint-py.yml: new freeze-components job runs the gate
  (stdlib-only, no uv sync).

Verified locally: passes on the baseline, fails (exit 1) on a simulated new
provider directory with an actionable message, passes again after cleanup.

* fix(ci): freeze gate counts only dirs shipping __init__.py

Review finding: a stray directory holding only __pycache__ bytecode (left
behind by a branch switch) tripped the additions-only gate locally. A
directory without __init__.py is not a provider; ignore it.

* fix(ci): nightly bundle rename follows [extras] refs and self-refs

Two gaps in update_bundle_versions.py around extras suffixes, both fatal
or silently wrong for the first nightly carrying the lfx-bundles
metapackage:

- update_root_pyproject_for_bundle's dep regex required the version
  specifier immediately after the name, so "lfx-bundles[all]>=1.0,<2.0"
  (a MAIN dep) was left unrewritten while the workspace member was
  renamed to lfx-bundles-nightly; uv lock then tries to resolve stable
  lfx-bundles from PyPI, where it does not exist. The same gap silently
  left "lfx-docling[local]>=0.1.0" optional-dep refs pointing at the
  stable distribution. The regex now tolerates an [extras] group and
  carries it into the replacement.

- rename_bundle_pyproject skipped self-referencing extras, so the
  metapackage's generated `all` extra kept 45 "lfx-bundles[<provider>]"
  members after the rename, pulling the stable distribution (same
  lfx_bundles import package, install collision) once published. Self
  refs now follow the rename, idempotently.

Tests drive the real script module, mirroring test_bundle_lfx_pin.py.
This closes the PR-2 audit item flagged when the metapackage was
introduced. The canonical-pre-release cutover would retire the nightly
rename entirely; until it lands, the rename must be correct.
2026-06-11 08:24:48 -07:00

91 lines
3.5 KiB
Python
Executable File

#!/usr/bin/env python3
"""CI gate: freeze the top-level provider directories under ``lfx/components/``.
After the bundle metapackage split (1.11), no NEW top-level subdirectory may be
added to ``src/lfx/src/lfx/components/`` -- new providers go to the
``lfx-bundles`` metapackage (``src/bundles/lfx-bundles/src/lfx_bundles/<provider>/``)
or a graduated ``lfx-<provider>`` package, never in-tree.
This is an **additions-only** gate: it fails when the live listing contains a
top-level directory that is not in the committed baseline
(``frozen_component_dirs.txt``). Removals are allowed and never fail the gate --
the bulk move (PR-6) replaces each moved provider with a near-empty import
shim that keeps the directory present, and the M4 shim cleanup later removes
those dirs, which only shrinks the set.
Stdlib-only by design so it runs on a bare runner without ``uv sync``.
"""
from __future__ import annotations
import sys
from pathlib import Path
# scripts/ci/check_components_frozen.py -> repo root is two parents up.
REPO_ROOT = Path(__file__).resolve().parents[2]
COMPONENTS_DIR = REPO_ROOT / "src" / "lfx" / "src" / "lfx" / "components"
BASELINE_FILE = Path(__file__).resolve().parent / "frozen_component_dirs.txt"
# Directories that are package machinery, not providers; never counted.
_SKIP = {"__pycache__"}
def _current_dirs() -> set[str]:
"""Top-level provider directories present in the working tree.
A directory only counts as a provider when it ships an ``__init__.py`` --
a stray empty directory (or one holding only ``__pycache__`` bytecode left
behind by a branch switch) is not a provider and must not trip the gate.
"""
return {
entry.name
for entry in COMPONENTS_DIR.iterdir()
if entry.is_dir()
and entry.name not in _SKIP
and not entry.name.startswith(".")
and (entry / "__init__.py").is_file()
}
def _baseline_dirs() -> set[str]:
"""The frozen baseline set (blank lines and ``#`` comments ignored)."""
lines = BASELINE_FILE.read_text(encoding="utf-8").splitlines()
return {line.strip() for line in lines if line.strip() and not line.lstrip().startswith("#")}
def main() -> int:
if not COMPONENTS_DIR.is_dir():
print(f"::error:: components directory not found: {COMPONENTS_DIR}", file=sys.stderr)
return 1
if not BASELINE_FILE.is_file():
print(f"::error:: frozen baseline not found: {BASELINE_FILE}", file=sys.stderr)
return 1
current = _current_dirs()
baseline = _baseline_dirs()
new_dirs = sorted(current - baseline)
if new_dirs:
print(
"::error:: src/lfx/src/lfx/components/ is frozen: no new top-level provider "
"directory may be added in-tree.\n"
f" Offending new directories: {new_dirs}\n"
" New providers go to the lfx-bundles metapackage "
"(src/bundles/lfx-bundles/src/lfx_bundles/<provider>/) or a graduated "
"lfx-<provider> package -- not src/lfx/src/lfx/components/.\n"
f" If a directory is a deliberate exception, add it to {BASELINE_FILE.name}.",
file=sys.stderr,
)
return 1
removed = sorted(baseline - current)
if removed:
# Informational only: shim cleanup (M4) legitimately shrinks the set.
print(f"note: {len(removed)} baselined component dir(s) no longer present (allowed): {removed}")
print(f"OK: {len(current)} top-level component dir(s); no additions beyond the frozen baseline.")
return 0
if __name__ == "__main__":
raise SystemExit(main())