mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-24 07:34:10 +08:00
cloud-dev-toggle
96 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 5a8c17b22d |
ci: backport bundle/nightly CI fixes from main (#13206, #13207, #13208) (#13210)
* ci: stop publishing -nightly bundle variants; release bundles on demand (#13206) Bundles (lfx-arxiv, lfx-duckduckgo) change infrequently enough that a nightly cadence is overkill. Drop the rename-to-`-nightly` and bundle publish lanes from the nightly pipeline and add a dedicated workflow_dispatch-only release_bundles.yml for purposeful releases. - nightly_build.yml: drop update_bundle_versions.py invocation and the src/bundles/*/pyproject.toml git-add guard. - release_nightly.yml: remove build-nightly-bundles and publish-nightly-bundles jobs; untether test-cross-platform and publish-nightly-main from them. - release_bundles.yml (new): build all src/bundles/* wheels, run a cross-OS install smoke test, then publish to PyPI under their stable names. Tolerates already-published versions so re-runs without a version bump are no-ops. release.yml still owns bundle publishing as part of main releases. Manual follow-up (PyPI admin): delete the lfx-arxiv-nightly and lfx-duckduckgo-nightly projects from PyPI. * ci(release_bundles): build lfx wheel locally for smoke test (#13207) The release_bundles.yml smoke test installs bundle wheels into a fresh venv, which makes uv resolve their `lfx>=X.Y,<Z` pin against PyPI. When bundles are released ahead of a matching lfx (typical case — bundles ship more often than lfx bumps land on PyPI), the resolve fails: Because only lfx<=0.4.3 is available and lfx-arxiv==0.1.0 depends on lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv==0.1.0 cannot be used. Build the lfx wheel from the current ref in the build-bundles job and install it alongside the bundle wheels in the smoke test. The lfx wheel ships as a separate `dist-lfx-smoketest` artifact and is NOT included in the publish-bundles step — only the `dist-bundles` artifact gets pushed to PyPI. * ci(nightly): re-pin bundle lfx deps to lfx-nightly during nightly build (#13208) The nightly pipeline renames the workspace `lfx` package to `lfx-nightly` in `src/lfx/pyproject.toml` and the root workspace dep, but leaves each `src/bundles/*/pyproject.toml`'s `"lfx>=X.Y,<Z"` pin unchanged. With the workspace `lfx` gone and PyPI still on lfx 0.4.3, the create-nightly-tag job's `uv lock` fails: × No solution found when resolving dependencies for split [...] ╰─▶ Because only lfx<=0.4.3 is available and lfx-arxiv depends on lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv's requirements are unsatisfiable. `update_lfx_version.py` now also rewrites the `lfx` (or already-rewritten `lfx-nightly`) specifier in every `src/bundles/*/pyproject.toml` to `lfx-nightly==<dev version>`, mirroring how `update_lf_base_dependency` re-pins the lfx dep inside `langflow-base`. The lookahead in the regex prevents matching sibling packages like `lfx-arxiv` / `lfx-duckduckgo`. No-op when no bundle pyprojects exist (e.g. on main today), so this is safe to merge ahead of the bundle extraction landing on main. Restored the guarded `git add src/bundles/*/pyproject.toml` step so the modified bundle pyprojects ride the same commit/tag as the rest of the nightly version bumps. Cherry-pick to release-1.10.0 to unblock nightlies cut from that branch. |
|||
| 487a2402ba |
fix: Model handling for tool calling in agents and update IBM models (#13201)
fix: Model handling for tool calling in agents and update IBM models (#13191) * Model handling for tool calling in agents and update IBM models * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 2be85b167e |
feat: promote OpenRouter to a first-class unified model provider (#13185)
* feat: promote OpenRouter to a first-class unified model provider Adds OpenRouter to the unified Model Providers system so users can configure it once in Settings rather than per-flow. Reuses ChatOpenAI with base_url=https://openrouter.ai/api/v1, supports a live model catalog (fetched from /api/v1/models with the user's bearer key), and forwards the optional HTTP-Referer / X-Title attribution headers when the user provides a site URL or app name. Validation is a cheap GET against /api/v1/models that raises ValueError("Invalid OpenRouter API key") on a 401. The existing flow-level OpenRouterComponent is left untouched for backward compatibility. Implements the opinionated direction proposed in #12839 (single curated provider instead of a generic OpenAI-compatible field). * Update Structured Data Analysis Agent.json * [autofix.ci] apply automated fixes * fix(openrouter): address review feedback - Per-model tool_calling derived from OpenRouter's `supported_parameters`, not blanket-True. The Agent component calls `get_language_model_options(tool_calling=True)` so this fixes a real bug where non-tool-calling models (e.g. perceptron/perceptron-mk1) would have appeared in agent flows and failed at runtime. - Default-model picks intersect with the curated seed list instead of taking the first 5 alphabetical ids (which were ai21/jamba-large-1.7, aion-labs/aion-1.0, etc.). Falls back to first MIN_DEFAULT_MODELS when the seed list shares nothing with the live catalog (stale-seed safety). - Broadened the live-fetch except to include ValueError/TypeError (covers malformed JSON, non-list `data`, non-dict entries) and added defensive isinstance checks so a 200 with a bad payload degrades to `[]` instead of crashing on AttributeError. - Bumped the live-fetch failure log from `debug` to `warning` with URL + status code, so an empty live catalog leaves a server-side breadcrumb. - Wrapped the credentials probe in try/except → ValueError so a 5xx / timeout during `POST /validate-provider` surfaces as a user-facing 400 via api/v1/variable.py (which only catches ValueError), not an unhandled 500. - Tightened `os.environ.get(var["variable_key"])` (was `var.get("variable_key", "")` → `os.environ.get("")` masking misconfig). - Documented `is_header` / `header_name` and the settings-only (no-`component_metadata`) pattern in the MODEL_PROVIDER_METADATA header comment. - Updated seed slugs against the current OpenRouter catalog (anthropic/claude-3.7-sonnet was missing); curated list is now Claude Opus/Sonnet/Haiku 4.x + GPT-4o pair + Gemini 2.5 Pro + Llama 3.3. - Added OPENROUTER_API_KEY / OPENROUTER_SITE_URL / OPENROUTER_APP_NAME to VARIABLES_TO_GET_FROM_ENVIRONMENT so users with env vars set see the provider as configured in Settings → Model Providers (parity with the other providers, and makes the "Falls back to env" copy honest). - Tests: pinned `MIN_DEFAULT_MODELS` instead of magic 3; added cases for per-model tool_calling derivation, seed-intersection defaults, stale-seed fallback, httpx.HTTPStatusError, malformed payload, transient network → ValueError validation, env-var attribution headers, and env auto-import list. Replaced `patch.dict("sys.modules", ...)` with `patch.object(requests, "get", ...)` so the test stays correct if the lazy import is hoisted. * Update Structured Data Analysis Agent.json * feat(model-providers): search bar, models.dev metadata override, capability tags Three follow-ups to the OpenRouter PR, requested by @ogabrielluiz in Slack: 1. **Search bars in the model providers modal.** ProviderList accepts a `query` prop and filters case-insensitively on the provider name; an `<Input icon="Search">` in ModelProvidersContent drives it. Inside the per-provider panel, ModelSelection grows its own search input that filters both LLM and embedding lists and resets when the selected provider changes (so OpenRouter → Anthropic doesn't carry stale text). Empty-state copy in both lanes; new i18n keys. 2. **models.dev metadata override (source of truth for covered providers).** New `lfx.base.models.models_dev_catalog` module: * `fetch_models_dev_snapshot()` — httpx GET against https://models.dev/api.json, returns None on transport / status / payload errors with a warning log. * `load_models_dev_snapshot()` / `save_models_dev_snapshot()` — disk cache at `<user_cache_dir>/langflow/models_dev/snapshot.json` with atomic temp-file write. * `apply_models_dev_overrides()` — translates each covered provider's models.dev entries into `ModelMetadata` (tool_call → tool_calling, reasoning passthrough, modalities.input has "image" → vision, limit.context → context_window, cost.input/output → cost_per_million_in/out). Providers not covered (IBM WatsonX, Azure OpenAI, Groq) pass through unchanged. Live-fetched providers (Ollama, IBM WatsonX, OpenRouter) keep their behavior because `replace_with_live_models` runs downstream at read time. `provider_queries.get_models_detailed()` now consults the in-memory snapshot via `get_active_snapshot()` and is `cache_clear()`-friendly so snapshot installs are picked up on the next call. The non-blocking refresh task in `get_lifespan` hydrates from disk first, then attempts a live fetch on a 24h interval; startup never blocks on models.dev. 3. **Capability tags next to the model name.** ModelRow renders small Badge pills (tool / reasoning / vision / embedding / search / preview) driven by metadata flags, in a stable left-to-right order, with the existing Deprecated badge kept at the end. The embedding tag only surfaces in the `all` view because the dedicated embeddings section header already conveys the same info. Tests: - 14 new backend cases in `test_models_dev_catalog.py` (fetch / load / save / override / cache invalidation paths). - Search + capability-tag cases added to ProviderList and ModelSelection Jest suites (44 total, all green). - lfx tests/unit/base/models + tests/unit/inputs: 114 pass, no regressions. - ruff check clean on all touched files. Out of scope: manual UI refresh button, surfacing pricing in the UI, auto-adding providers from models.dev without a LangChain client. * fix(openrouter): derive reasoning from supported_parameters OpenRouter exposes "reasoning" in each model's supported_parameters array exactly like it does "tools" (verified against /api/v1/models — Claude Opus 4.x, o1/o3, DeepSeek R1 all carry it). The live fetcher was only deriving tool_calling; reasoning was left at the create_model_metadata default of False, so the new reasoning badge never lit up for OpenRouter models even though the rest of the wiring was in place. Now derives both flags from the same array. Test renamed to …_derives_tool_calling_and_reasoning_per_model and pins all four combinations (tools+reasoning, tools only, reasoning only, neither, plus the missing-supported_parameters case). * feat(model-providers): sort by recency, hide deprecated by default OpenAI's list looked jumbled because the catalog rendered in raw constants-file order — reasoning models interleaved with chat models, deprecated and preview rows scattered throughout. Three changes: 1. **Thread `created` (Unix epoch) through ModelMetadata.** New optional field; populated by: - OpenRouter live fetch (existing `created` field on every model; defensive int-coercion handles bad payload values). - models.dev override (parses `release_date` YYYY-MM-DD → UTC midnight epoch; invalid / missing → 0). Static `*_constants.py` lists leave it at 0 → "unknown" tier. 2. **Sort each provider's group in `get_unified_models_detailed`.** Stable sort by `(is_deprecated, -created_epoch)`: - Non-deprecated dated rows: newest first. - Untimed rows (current static fallback): stable sort keeps their hand-curated constants-file order intact within the same tier. - Deprecated rows: bottom. The "first 5 = default" stamping now runs AFTER the sort so the highlighted defaults reflect the new ordering. For OpenAI's static list this changes nothing today (constants are already newest-first); once models.dev hydrates it picks up genuinely-recent dates from `release_date`. 3. **Collapse deprecated into a `<details>` disclosure.** Each section renders active rows normally, then a `Show N deprecated model(s)` summary that's closed by default. Resets on provider change. The rows stay in the DOM so existing flows referencing deprecated models still resolve; they're just out of the way. `not_supported` is already filtered at the API layer (the modal doesn't pass `includeUnsupported`), so those rows remain hidden as before. Tests: - Backend: `created` propagation from `release_date` (+ malformed/empty cases), OpenRouter `created` passthrough (+ invalid-value degradation), and a full sort-order assertion covering [newest, mid, undated-a, undated-b, deprecated] ordering. - Frontend: deprecated disclosure default-closed, singular pluralization, no-disclosure-when-nothing-deprecated, reset on provider change via real user click → onToggle (the right pattern; the prior draft of this test mutated `details.open` directly and missed React's prop reconciliation). - 117 lfx tests + 48 frontend tests pass; ruff clean. * [autofix.ci] apply automated fixes * fix(models.dev override): preserve static deprecated flag + auto-deprecate dated snapshots Two issues surfaced in the OpenAI/Anthropic screenshots: 1. **gpt-3.5-turbo wasn't in the deprecated disclosure** even though openai_constants.py has it flagged ``deprecated=True``. models.dev exposes no deprecated/lifecycle field (verified: only ``release_date`` and ``last_updated``), so the override threw away our static curation. ``apply_models_dev_overrides`` now builds a provider→{deprecated model names} lookup from the static lists and forwards the flag through ``_translate_model_entry`` for any name that matched a static deprecated row. ``gpt-4.5-preview`` and ``gpt-4-turbo-preview`` ride the same path. 2. **Anthropic dated-snapshot ids** (``claude-opus-4-5-20251101``, ``claude-haiku-4-5-20251001``, etc.) crowded the picker alongside their moving aliases. These are pinned-date snapshots of the alias — useful for production pins, noisy as defaults. Same pattern exists for OpenAI (``gpt-4o-2024-05-13``). ``_translate_model_entry`` now matches ``-YYYYMMDD$`` (Anthropic) and ``-YYYY-MM-DD$`` (OpenAI) and auto-flags those rows deprecated so they collapse into the disclosure tier rather than dominating the list. The 4-digit-only ``-0314``-style suffixes (``gpt-4-0314``, ``gpt-4-0613``) are deliberately *not* matched — those are pre-dated-naming snapshots that need static curation if we want to mark them. Tests: - test_apply_overrides_marks_dated_snapshot_ids_deprecated pins: * alias rows stay active (claude-opus-4-5, gpt-4o) * Anthropic 8-digit and OpenAI hyphenated suffixes flip to deprecated * 4-digit-only suffixes don't false-positive - test_apply_overrides_preserves_static_deprecated_flag pins: * gpt-3.5-turbo / gpt-4.5-preview keep their static deprecation * a new model not in the static list (gpt-6) stays active * fix(models.dev override): classify embeddings + age-based auto-deprecation Two follow-ups from screenshot review on the OpenAI provider list. **Bug 1 — embedding models showed in the Language Models section.** models.dev mixes embeddings into a provider's ``models`` dict (e.g. ``text-embedding-3-large`` lives in the OpenAI block). The translator was defaulting every entry to ``model_type="llm"``, so embeddings polluted the LLM lane in the modal. The clean discriminator is the ``family`` field (``"text-embedding"`` for OpenAI's current embeddings, verified for ada-002 / 3-small / 3-large); a name-substring check ("embedding") covers providers that fill ``family`` inconsistently. **Bug 2 — old OpenAI models (gpt-4, gpt-4-turbo, text-embedding-ada-002) stayed active even though they're functionally legacy.** models.dev has no deprecation field, and OpenAI doesn't formally deprecate gpt-4 / gpt-4-turbo in their catalog, but at ~924 days from release they're clearly out of rotation. New ``_is_aged_out`` helper auto-deprecates models older than ~30 months (900 days), preferring ``release_date`` over ``last_updated`` (the latter tracks catalog-curator edits like typo fixes, not new model versions — gpt-4 has last_updated=2024-04 which would falsely keep it active). Threshold tuned conservatively against today's catalog: - 1250d ada-002 → deprecated - 924d gpt-4, gpt-4-turbo → deprecated - 844d text-embedding-3-large/small → ACTIVE (still OpenAI's current embeddings; threshold deliberately above this point) - 735d gpt-4o → active - 261d Claude 4.x → active ``now`` is injected through ``_translate_model_entry`` and ``apply_models_dev_overrides`` for testability so the unit tests don't depend on wall-clock time. Tests: - test_apply_overrides_marks_embedding_family_as_embeddings_model_type pins ``family="text-embedding"`` → embeddings, generic LLMs stay llm, and a name-substring case (``voyage-3-embedding``) still classifies correctly when family is missing. - test_apply_overrides_auto_deprecates_stale_models pins both sides of the 900-day line and the no-date case (insufficient signal → stays active). * fix(models.dev override): drop duplicate static groups + Google dated-preview + gemini-1.5 Three issues from screenshot review. **1. Duplicate embedding rows in the OpenAI section.** The override iterates ``static_lists`` and replaces the first OpenAI group (``OPENAI_MODELS_DETAILED``, LLMs) with the combined override (which holds both LLMs and embeddings from the single models.dev block). The second OpenAI group (``OPENAI_EMBEDDING_MODELS_DETAILED``) then fell through to the else branch and got appended unchanged, so every embedding row rendered twice. Fix: when a provider's override has already been applied, drop any subsequent static groups for the same provider — they would only re-introduce duplicates. **2. Google's dated-preview suffix wasn't auto-deprecated.** Each vendor uses a different shape for snapshot ids: | Vendor | Shape | Example | | --------- | -------------------- | ----------------------------------------- | | Anthropic | ``-YYYYMMDD`` | ``claude-opus-4-5-20251101`` | | OpenAI | ``-YYYY-MM-DD`` | ``gpt-4o-2024-05-13`` | | Google | ``-preview-MM-DD`` | ``gemini-2.5-pro-preview-05-06`` | | Google | ``-preview-MM-YYYY`` | ``gemini-2.5-flash-preview-09-2025`` | ``_DATED_SNAPSHOT_RE`` extended to match the Google variants. The non-dated ``-preview-tts``-style suffix stays active (tts is a real variant, not a snapshot). **3. gemini-1.5-* stayed active despite being long out of rotation.** At ~600-820 days these names sit under the 900-day age threshold, so the heuristic alone wouldn't catch them. Added explicit ``deprecated=True`` entries for ``gemini-1.5-pro`` / ``gemini-1.5-flash`` / ``gemini-1.5-flash-8b`` to ``google_generative_ai_constants``; the override's static-deprecation preservation flows them through. Tests: - test_apply_overrides_drops_subsequent_static_groups_for_overridden_provider pins the no-duplicates invariant. - test_apply_overrides_marks_dated_snapshot_ids_deprecated grows to cover the Google preview-MM-DD / preview-MM-YYYY patterns and the non-dated -preview-tts negative case. - test_apply_overrides_preserves_gemini_1_5_static_deprecation pins that the static curation actually reaches the override output (and includes a sanity assert on the static list itself, so we don't drift the test off of the constants file by accident). 57 lfx model tests pass; ruff clean. * [autofix.ci] apply automated fixes * fix(ollama): stop 10s frontend poll + parallelize /api/show + cache Closes #13137. Two layered fixes for the runaway Ollama model fetch: **Frontend — drop the 10-second refetchInterval.** The previous code polled ``/api/v1/models`` every 10s as long as the Ollama provider card was selected. Each backend call fanned out to ``GET /api/tags`` + ``POST /api/show`` per model serially, so with many local models the request overran the 10s interval and requests accumulated. The catalog already refreshes on credential save / disconnect via ``invalidateProviderQueries`` and the backend cache (below) keeps a on-demand window-focus refresh cheap; the timer was unnecessary. **Backend — parallelize ``/api/show`` and cache.** * ``get_ollama_models`` now fans out per-model capability probes via ``asyncio.gather``, so total latency is bounded by the slowest single request rather than ``N * avg``. Per-model failures are absorbed at debug-level — one broken model no longer poisons the whole catalog response. * In-process TTL cache keyed by ``(base_url, capability)`` with a 30s window in front of the fan-out. Overlapping ``/api/v1/models`` requests (which happen on every modal interaction, agent picker, embed picker, etc.) now share one upstream round-trip. Test-only ``_ollama_cache_clear`` helper for isolation. Tests: - test_get_ollama_models_returns_only_matching_capability — capability filter still works. - test_get_ollama_models_runs_show_requests_in_parallel — pins parallelism via an asyncio.Event that would deadlock under serial. - test_get_ollama_models_tolerates_single_show_failure — one bad model doesn't drop the whole list. - test_get_ollama_models_caches_result_for_ttl_window — second call within the TTL hits the cache (zero upstream calls). - test_get_ollama_models_cache_keyed_by_base_url_and_capability — different base URLs and capability filters stay isolated. - test_get_ollama_models_refetches_after_ttl_expires — TTL expiry triggers a fresh fetch. * Update Structured Data Analysis Agent.json * [autofix.ci] apply automated fixes * feat(model-input): declarative filters propagate to picker + sticky-default Agent's Language Model dropdown was still showing tool-incompatible models like ``gemini-3.1-flash-image-preview`` (``tool_call=false`` per models.dev). Root cause: the tool-calling constraint lived as a hardcoded lambda at the Agent component's call site, but ``update_model_options_in_build_config``'s sticky-default re-injection path didn't know about it — so a previously-saved selection that no longer passed the filter got re-injected with ``not_enabled_locally=True`` and kept appearing in the picker. Make the filter declarative so the helper, the cache key, and the sticky-default path all read it from the same source. * ``ModelInputMixin`` gains a ``filters: dict[str, Any] | None`` field. Any keys/values forward to ``get_unified_models_detailed`` and are matched against each model's metadata. Example: the Agent declares ``ModelInput(name="model", filters={"tool_calling": True})`` instead of passing a lambda at the call site. * ``get_language_model_options`` accepts ``filters: dict | None``; the legacy ``tool_calling`` kwarg is preserved and merged into the filter dict for backward compatibility. * ``handle_model_input_update`` reads ``filters`` from the build_config and, when no explicit ``get_options_func`` is supplied, builds one that applies them. The options-cache key prefix is namespaced by sorted filter pairs so different constraint sets don't share cached options. * The sticky-default block in ``update_model_options_in_build_config`` validates the saved selection against the same filters dict via ``_saved_model_passes_filters``. When the saved row doesn't satisfy every constraint, the row is dropped and the value cleared so the downstream auto-default picks a compatible model. Conservative fallback: an unknown model (not in the catalog) still passes the filter so today's "inject and let the user configure" UX is unchanged for genuinely-unknown selections. * Agent component: drop the inline lambda in ``update_build_config``; the filter now lives on the ModelInput declaration. * Replaces the ``cache_key_prefix`` sniff for tool-calling intent — prefix is now derived from filters, not pattern-matched. Tests cover the helpers, both sticky-default branches (drop vs inject), and the conservative fallback for unknown saved models. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix(google): tool_calling=False on image-output Gemini models Even after the declarative filters={"tool_calling": True} shipped on the Agent component's ModelInput, the picker still showed gemini-3.1-flash-image-preview and friends. Root cause: the static google_generative_ai_constants.py catalog defaulted every Gemini model to tool_calling=True. Image-generation models can't actually run with tools (verified in models.dev — these rows have tool_call: false and modalities.output includes image), so the static-only fallback (before the models.dev refresh hydrates) was wrong upstream of the filter and leaked them through. Four image-output models flipped to tool_calling=False in the static catalog: * gemini-2.5-flash-image * gemini-2.0-flash-preview-image-generation * gemini-3-pro-image-preview * gemini-3.1-flash-image-preview Models.dev's correct flag still wins via the override path once the snapshot hydrates; this fix matters most for cold starts and air-gapped deployments where only the bundled static lists are in play. Tests (test_static_catalog_tool_calling.py): - Parametrized over the four known image models; each must declare tool_calling=False in the static catalog (and remain listed — we need the static curation so the override preserves the flag by name match). - Heuristic regression guard: any Google model with "image" in its name that defaults to tool_calling=True will fail this test. Future image variants are caught before they reach the picker; legitimate exceptions opt in via an allowlist literal in the test. * [autofix.ci] apply automated fixes * fix(model-input): source filters from class declaration, not round-tripped build_config The Agent picker was STILL showing gemini-3.1-flash-image-preview even after the static catalog flag flip and the declarative filters work. Reproduced and traced to a saved-flow round-trip: flows persisted before the filters field shipped don't carry it in their stored template. The frontend POSTs that stale template back, my helper read build_config["model"]["filters"] (None), and the filter path short-circuited to the unfiltered branch. Fix: source filters from the ModelInput's class-level declaration (canonical, always current with the server build), falling back to build_config only when the component happens to expose no inputs attribute. The build_config is also patched in place when the declaration overrides it, so the next round-trip carries the active filter back to the frontend. * New helper ``_filters_from_component_inputs`` walks the component's inputs list and returns the matching ModelInput's filters dict (with None values dropped, same hygiene as the build_config path). * New ``_resolve_filters`` orchestrates: class declaration > build_config, with an in-place write back to build_config when the class wins. * Both call sites in ``handle_model_input_update`` and ``update_model_options_in_build_config`` now call ``_resolve_filters``; the old direct call to ``_filters_from_build_config`` is gone from the hot paths (kept exported because it's still useful as a defensive fallback when no component is available). Tests (4 new in test_build_config_sticky_default.py): - _filters_from_component_inputs reads class declaration / returns empty when no matching input. - _resolve_filters prefers class over build_config and patches build_config in place. - _resolve_filters falls back to build_config when component has no inputs attribute (defensive fallback for non-standard callers). * fix(frontend): honor ModelInput.filters in dropdown augment loop The Agent picker kept showing tool-incompatible models like gemini-3.1-flash-image-preview even after every backend filter and static-catalog fix landed. Root cause was on the frontend, in the ModelInputComponent's groupedOptions memo: - The first loop iterates over the backend-supplied options (which are filtered). - The augment loop then iterates over every enabled model in useGetModelProviders × useGetEnabledModels — both of which are filter-unaware — and pushes the union into the dropdown. So even though the backend correctly excluded gemini-3.1-flash-image-preview from options, the augment loop happily re-added it because providersData listed it and enabledModelsData flagged it as enabled. Fix: read filters off nodeClass.template.model.filters (the same declaration that drives the backend filter) and apply the same metadata match to every option that flows through groupedOptions. Two checkpoints: 1. Defensive pass in the existing-options loop. The backend should have filtered already, but stale saved flows could surface a build_config that hasn't been corrected yet — same self-healing story as the backend _resolve_filters helper. 2. New pass in the augment loop, after the model_type check. Every enabled-model candidate must satisfy the filter or it's skipped. Conservative behavior matches the backend: when a filter key (e.g. tool_calling) is declared but the candidate's metadata doesn't carry that key at all, the candidate fails the check — better drop an undeclared row than surface one that crashes at run time. Tests (4 new): hides tool-incompatible augmented models, drops saved options that fail the filter, conservative drop on missing metadata key (with a known-good sibling so the assertion isn't paper-thin), no-op when no filters are declared. Suite-local beforeEach restores the default hook mocks since jest.clearAllMocks only resets call history, not implementations. * Fix tests * Update .secrets.baseline * Fix starter projects * Update .secrets.baseline * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Update component_index.json * Update Structured Data Analysis Agent.json * [autofix.ci] apply automated fixes * fix frontend tests * fix remaining CI errors --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| c901602bc2 |
fix(ci): rename bundles to -nightly during nightly build (#13193)
fix: rename bundles for nightly build |
|||
| 23f2edf327 |
chore: Clean up the startup warnings for Python 3.14 (#13156)
* fix: backport policies ToolGuard lazy imports (#13144) fix: backport policies toolguard lazy imports * chore: Clean up the startup warnings for Python 3.14 * Update base.py * Update component_index.json * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Update component_index.json * [autofix.ci] apply automated fixes * Update Structured Data Analysis Agent.json * [autofix.ci] apply automated fixes * Update component_index.json * [autofix.ci] apply automated fixes * Update component_index.json * Update starter projects * Update .secrets.baseline * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| d1da768192 |
feat(knowledge): merge ingestion + retrieval into mode-driven Knowledge component (#13120)
* feat(knowledge): merge ingestion + retrieval into mode-driven Knowledge component
Merge KnowledgeIngestionComponent and KnowledgeBaseComponent into a single
KnowledgeComponent driven by a TabInput mode selector ("Ingest" / "Retrieve"),
mirroring the PoliciesComponent pattern.
- New: src/lfx/src/lfx/components/files_and_knowledge/knowledge.py holds the
full merged component. update_build_config and update_outputs hide the
inputs and the output that do not apply to the current mode.
- Legacy KnowledgeIngestionComponent / KnowledgeBaseComponent become thin
subclasses pinned to their respective modes with class-level inputs and
outputs aligned to the legacy shape, so saved flows continue to load.
- Output names ("dataframe_output", "retrieve_data") preserved for back-compat.
- New tests cover mode-switch input/output visibility and subclass pinning;
existing test_ingestion / test_retrieval suites rerouted to patch the new
knowledge module path and continue to pass.
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* chore(knowledge): drop emoji from mode labels, mark legacy subclasses
- Mode tab labels: "📥 Ingest" / "🔍 Retrieve" → "Ingest" / "Retrieve" so the
selector blends with the rest of Langflow's TabInput palette (see MemoryComponent
for the same convention).
- Set `legacy = True` on KnowledgeIngestionComponent and KnowledgeBaseComponent
so the back-compat shims stay loadable by saved flows but no longer clutter
the new-component palette — users should reach for the merged Knowledge
component instead.
* [autofix.ci] apply automated fixes
* fix(knowledge): declare both outputs at class level so retrieve mode dispatches correctly
Saved flows in retrieve mode were hitting
``'NoneType' object has no attribute 'to_dataframe'`` because the merged
``KnowledgeComponent`` only declared ``dataframe_output`` (build_kb_info)
at the class level. The runtime resolves outputs from the class-level
``outputs`` list at __init__ time, so ``retrieve_data`` was never in
``_outputs_map``; the runtime fell back to ``build_kb_info``, which ran
``convert_to_dataframe(self.input_df)`` against a hidden, None-valued
input.
Fix: declare both outputs at the class level (TypeConverterComponent +
MemoryComponent already use this pattern). ``update_outputs`` keeps
filtering the canvas-visible output by mode, but the runtime can now
dispatch to either method based on which one the saved flow has wired up.
Also wire ``update_frontend_node`` to re-run the mode-driven output
filter on canvas load so saved flows render the correct output even
before the user clicks the tab.
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* fix(knowledge): dispatch by mode in output methods to survive stale edges and legacy labels
Two failure modes were still triggering the
``'NoneType' object has no attribute 'to_dataframe'`` crash even with both
outputs declared at the class level:
1. User drops the Knowledge component (default INGEST), wires the
``dataframe_output`` edge, then switches to Retrieve. The saved edge
still points at ``build_kb_info``; the runtime calls it against a
None ``input_df`` and crashes inside ``convert_to_dataframe``.
2. Flows saved with the original emoji mode labels ("📥 Ingest" /
"🔍 Retrieve") carry those strings on disk. The post-emoji-removal
equality checks (``mode == MODE_RETRIEVE``) failed, so even though
the user was conceptually in retrieve mode, the component fell
through the ingest path.
Both methods now check the mode at entry and delegate to the other one
when the wired output doesn't match. Mode matching is lenient
(substring-based) so the legacy emoji labels still resolve.
``update_build_config`` and ``update_outputs`` also normalize legacy
labels onto the current canonical values so visibility toggles correctly
for flows pulled from old saves.
* [autofix.ci] apply automated fixes
* feat(knowledge): point legacy shims at the merged component and migrate starters
- Add ``replacement = ["files_and_knowledge.Knowledge"]`` to both
``KnowledgeIngestionComponent`` and ``KnowledgeBaseComponent`` so the
canvas legacy-component banner offers the merged component as the
one-click upgrade target.
- Rewrite the ``KnowledgeBase`` nodes in the two starter projects that
shipped them (``Knowledge Retrieval`` and ``Vector Store RAG``) to use
the new ``Knowledge`` component with mode pre-set to ``Retrieve``. All
edges are re-pointed at the new node id, the embedded module path is
updated to ``lfx.components.files_and_knowledge.knowledge.KnowledgeComponent``,
and the mode value is normalised away from the old emoji label.
* [autofix.ci] apply automated fixes
* Update knowledge.py
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update component_index.json
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update src/lfx/src/lfx/components/files_and_knowledge/knowledge.py
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix: Restore knowledge creation functionality
* Template update
* Update component_index.json
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix: Linting errors
* Update knowledge.py
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* fix: Address @rafaelgiln comments
* Update test_knowledge.py
* Update component_index.json
* Template updates
* Update .secrets.baseline
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update Structured Data Analysis Agent.json
* Update component_index.json
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* Fix edge connection with parser
* Update Structured Data Analysis Agent.json
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
|
|||
| cc009c9133 |
feat(lfx): Bundle Separation and LFX Extension Framework (#13043)
* feat(lfx): installed-package + seed-directory discovery for production install (LE-1022) Adds the read-only production install path for Modes A, B, and C of the Bundle Separation iteration. Manifest-shipping pip-installed distributions and seed-directory subdirectories are discovered at server startup and registered as Extensions at @official. * discovery.py: walks importlib.metadata.distributions() + the $LANGFLOW_SEED_DIR / /opt/langflow/bundles seed root; produces DiscoveredExtension records and typed errors for malformed manifests / configured-but-missing seed dirs. * registry.py: ExtensionRegistry service with the immutability invariant for installed and seed entries. Mutation verbs (uninstall, disable, enable, install, update_entry) all raise ExtensionImmutableError carrying the typed installed-extension-immutable / seed-directory-immutable code so the invariant is testable today; the CLI uninstall surface ships in B4. * lfx extension list: read-only inspector with text and JSON output for operators inspecting Mode B/C images. * Errors: four new typed codes (installed-extension-immutable, seed-directory-immutable, seed-directory-not-found, duplicate-extension-id) plus snapshot coverage in tests/unit/extension/test_errors.py. * Tests: 165 extension tests pass, including the LE-1022 acceptance cases -- three pip-installed wheels visible at @official, three seed bundles visible at @official, and the parametrized service-layer immutability check across every mutation verb. * Docs: docs/Deployment/deployment-extensions-production.mdx covers the Dockerfile template, k8s deployment notes, the bundle packaging convention (extension.json shipped via package-data), and troubleshooting for the typed error codes. * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (#12967) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * refactor(lfx): clarify broad except, log malformed bundle.json, expand test docstring - Document why _discovery.import_bundle_module catches BaseException (startup-time loader, must surface bad bundles as typed errors rather than abort). - Add debug-level logging when bundle.json is malformed or non-object so a stale-cache footgun is at least observable. - Expand test_skips_re_imported_class docstring so future maintainers don't accidentally weaken the __module__-equality guard if package-style relative imports get added later. Also drops the stale duplicate-distribution claim from the PR description; that code is correctly deferred to LE-1022 along with /all integration. * feat(lfx): wire Extension System into /all, pathsep-split LANGFLOW_COMPONENTS_PATH, emit duplicate-distribution Closes the four AC gaps the previous reviewer flagged on PR #12967: 1. /all integration: get_and_cache_all_types_dict now also calls a new import_extension_components() that loads installed Extensions via load_installed_extensions, loads inline bundles via discover_inline_bundles, and builds frontend-node templates with extension/bundle/extension_version fields stamped on. Failures are logged and skipped per bundle. 2. LANGFLOW_COMPONENTS_PATH is now split on os.pathsep so multi-entry env vars (e.g. /a:/b on POSIX) produce multiple components-path entries instead of one literal non-existent path. Empty segments and missing paths are skipped. 3. duplicate-distribution is a real producer: load_installed_extensions surfaces a typed warning on the winner LoadResult when two distributions share a canonical name, naming every involved manifest path. 4. Manifest-first precedence runtime wiring: new filter_component_entry_points loads each entry-point and only skips ones that resolve to a Component subclass on a manifest-shipping distribution. plugin_routes.load_plugin_routes now applies it so non-component entry-points (route registrars) keep loading per the AC's 'unaffected' promise. Added the previously-missing AC test for same-distribution component+non-component partition. Also makes _distribution_canonical_name defensive against MagicMock test seams. * fix(lfx): register extension components under namespaced ID; promote duplicate-distribution to error Addresses the latest review of PR #12967: P1: /all integration now keys the cache inner dict by LoadedComponent.namespaced_id (ext:<bundle>:<Class>@<slot>) rather than the bare class name. Templates also carry the namespaced_id as an explicit field so consumers that look at the value (not the key) still see the canonical address. This is the form the LE-1020 migration table will rewrite legacy class-name references to. P2: load_installed_extensions now appends duplicate-distribution to result.errors instead of result.warnings, so LoadResult.ok=False when two distributions share a canonical name. The winner's components still appear in result.components so flows already pinned to them keep working; only the conflict status changes. Updated test to assert errors + ok=False; added explicit assertion that the winner's components are still present. * fix(lfx): installed-distribution discovery accepts pyproject.toml manifest form Closes the latest review finding on PR #12967: the installed-distribution scan only looked for extension.json, ignoring distributions whose manifest lives in [tool.langflow.extension] inside pyproject.toml. The AC explicitly treats both as valid manifest forms. _distribution_manifest_path now: - Returns extension.json immediately when present (preserves precedence matching load_manifest's discovery order). - Falls back to pyproject.toml only when extension.json is absent AND the pyproject's [tool.langflow.extension] section is parseable. Validation reuses load_manifest itself so the rule lives in exactly one place; a stray pyproject.toml without the section is correctly ignored. Tests cover: pyproject-only discovery, pyproject-without-section ignored, extension.json wins on collision, end-to-end pyproject load at @official, and pyproject-form manifest-first entry-point suppression. * refactor(lfx): tighten loader invariants, surface silent skips, harden tests Addresses the latest review feedback on PR #12967: Type-level invariants (_types.py): - LoadedComponent.__post_init__ enforces that @extra components must NOT carry a distribution. The reverse (@official without distribution) is permitted because load_extension is also used for dev-mode loads against a working tree before pip install. - LoadResult docstring documents the partial-success contract: components may be non-empty when errors is non-empty (some files imported, others failed). Callers branching on ok get strict success. Silent-failure fixes (_orchestrator.py + settings/base.py): - inline-path-missing: a non-existent / non-dir LANGFLOW_COMPONENTS_PATH entry now produces a typed warning per skipped path so a typo no longer yields zero diagnostics. Settings-layer skip bumped from debug to warning for the same reason. - bundle-json-invalid: a malformed or non-object bundle.json now surfaces a typed warning instead of silently rewriting the user-declared id/version to derived values under the same bundle name. - no-component-subclass gating uses a call-local counter instead of result.errors so the diagnostic stays accurate when a future caller reuses a LoadResult (multi-bundle / batch wrapper scenarios). Test hardening: - test_re_imported_class_is_skipped_via_module_filter rewritten to actually exercise the __module__-equality check via sys.modules injection; previously passed via module-import-failed (relative-import failure), which would silently weaken if package registration changes. - test_user_declared_path_order_is_preserved: AC #8's multi-path order case (distinct bundles in [path_b, path_a]) was unasserted; added. - test_inline_module_import_failure_attributes_identity: AC #10's identity-on-partial-failure was covered for @official but not @extra; added. - test_uses_real_distributions_by_default tightened to assert ep placement (in kept, not in skipped) instead of exact-list equality, so a future Langflow-shipped manifest doesn't silently flip the assertion. bumped 64 -> 175 passing extension tests; 20 backend integration tests still pass. * fix(lfx): malformed pyproject manifests surface manifest-invalid instead of disappearing Closes the latest review finding on PR #12967: a pyproject.toml with a [tool.langflow.extension] section that has missing/invalid required fields was silently dropped because _pyproject_has_extension_section ran full schema validation via load_manifest and returned False on ValueError/TypeError. That conflated 'no section' with 'section malformed'. Fix: detect section presence only. _pyproject_has_extension_section now calls _read_pyproject_extension (TOML parse + key lookup, no schema check). Behavior: - Section absent or pyproject TOML unparseable -> False (treat as regular non-manifest package). - Section present and is a table (valid OR schema-invalid) -> True. - Section present but is not a table -> True; the author intended to declare an extension and load_extension will surface the typed error. This way a typo'd pyproject Extension produces a typed manifest-invalid LoadResult with extension_id attribution, and manifest-first precedence still suppresses its legacy component entry-points -- matching the 'typed load results on success/failure' contract for the supported pyproject manifest form. Tests: two new cases pin the behavior. test_malformed_pyproject_section_ surfaces_manifest_invalid asserts a typed load-failure result with distribution attribution; the second test pins manifest-first suppression for malformed pyproject distributions. * refactor(lfx): emit inline-path-unreadable, document reload contract, trim rot Closes the remaining nits on PR #12967: - inline-path-unreadable (new typed error code): a configured LANGFLOW_COMPONENTS_PATH entry that raises OSError on iterdir (typically permission-denied) now produces a typed LoadResult error carrying str(exc) instead of silently swallowing the message. - duplicate-inline-bundle hint trimmed: removed forward promise about hard-error-in-a-later-release; same actionability, no expiration. - discover_inline_bundles docstring trimmed from three paragraphs to two sentences (per CLAUDE.md anti-multi-paragraph rule). - _distribution_manifest_path docstring trimmed to one-liner. - installed_extension_roots dropped Used-by caller-narration list; manifest_owning_distributions docstring rewritten to call out the shadow-load risk for direct callers and point to load_installed_ extensions for the typed warning surface. - _discovery.import_bundle_module: replaced 'until that lands in a later milestone' rot with a single line ('Absolute imports only between bundle modules; relative imports unsupported.') AND added the single-load-per-process contract note documenting why LE-1018 reload must scrub registry/sys.modules before re-invoking the loader. - load_extension docstring grew a 'Single-load-per-process contract' block telling direct callers not to rely on this function for refresh. Tests: new test_unreadable_path_emits_inline_path_unreadable pins the OSError -> typed-error path. --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) (#12979) * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): atomic-swap Bundle reload pipeline + endpoint + CLI (LE-1018) Five-stage reload (parallel staging load -> validate -> swap under write lock -> cleanup -> emit) for installed Bundles in Mode A. In-flight flows keep the pre-swap class via existing references; new flows pick up the post-swap class atomically; concurrent reloads on the same Bundle are rejected with reload-in-progress. Adds: * lfx/extension/registry.py -- BundleRegistry with per-bundle reload-in-progress guard and components_index.json writer * lfx/extension/reload.py -- the five-stage pipeline; events emission is stubbed (TODO LE-1017) so the swap mechanics can ship before the events service lands * loader: optional module_namespace param so Stage 1 lands in __reload_staging__.<id> instead of the live _lfx_ext.* namespace * errors: four new typed reload codes (reload-in-progress, reload-bundle-not-installed, reload-bundle-name-mismatch, reload-source-missing) with branch templates and snapshot tests * HTTP: POST /api/v1/extensions/{id}/bundles/{name}/reload, gated by the existing get_current_active_user dependency, returns 409 with a typed body for the in-progress collision case * CLI: lfx extension reload <id> [--bundle <name>] -- HTTP client against the dev server with text/json output and proper exit codes (--all is gated until LE-1019 lands the list endpoint) * tests: 16 reload-pipeline tests covering the AC matrix (rename round-trip, broken-bundle isolation, concurrent readers, in-flight flow, double-reload guard, bundle-name mismatch) plus 9 CLI client tests Mode A only. In Mode B/C bundle changes require a Docker image rebuild and the reload path is not exercised. The events emission in Stage 5 is intentionally stubbed -- the LE-1017 ticket will swap the body of _emit_bundle_reload_event in one place without touching the pipeline core. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [autofix.ci] apply automated fixes * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) (#12968) * feat(lfx): add extension manifest schema, validate CLI, and error formatter (LE-1014) Foundation for the Bundle Separation iteration. Defines what a valid extension.json looks like (Pydantic models + Draft 2020-12 JSON Schema), ships the offline `lfx extension validate` command, and ships the single `format_extension_error` function that every other extension-system module will use to render structured errors. What's in lfx.extension: - `ExtensionManifest` / `BundleRef` / `LangflowCompat` Pydantic models with `extra="forbid"` so unknown fields fail loudly. Deferred fields (`services`, `routes`, `hooks`, `starter_projects`, `userConfig`) are reserved as None-only so non-null values produce a dedicated `field-deferred-in-this-milestone` error instead of a generic schema wall. `bundles` accepts a list but rejects length > 1 with `multi-bundle-deferred-in-this-milestone` (validator-enforced; the loader re-checks at install time in LE-1015). - `schema.build_schema()` + `build_schema_json()` produce the publishable artifact at schemas.langflow.org/extension/v1.json. - `ExtensionError` typed envelope and `format_extension_error` -- one branch per discriminant. Codes are registered in `ERROR_CODES`; an `ExtensionError` constructed with an unknown code raises at construction time, preventing producers from shipping without a matching renderer. - `validate_extension` runs four passes: manifest discovery + schema, path-safety (no `..`, no absolute paths, no symlink escape), AST inspection of every `.py` (syntax, Component subclass present, build() declared, top-level `import *`, top-level I/O primitives), and an opt-in `--execute-imports` that runs each module in a subprocess with a temporary HOME / TMPDIR / LANGFLOW_CONFIG_DIR and LANGFLOW_*/LFX_* env vars stripped. - `lfx extension validate` and `lfx extension schema` typer subcommands. Acceptance-criteria coverage in tests/unit/extension/: - Round-trips every v0 manifest field; deferred fields rejected; multi-bundle rejected with the dedicated discriminant. - JSON Schema validates the v0 example and rejects 12 malformed manifests with distinct error paths (>= 10 required by the ticket). - Median default-validate runtime < 100ms on the basic template. - Crafted side-effect bundle: default validate does NOT execute it (canary file is never written); `--execute-imports` DOES execute it and the canary appears, while LANGFLOW_* env vars are NOT inherited by the subprocess. - Snapshot tests for every code in ERROR_CODES; a guard test verifies ERROR_CODES and the snapshot table are in lockstep so future additions cannot ship without a format branch and a snapshot. Wiring: `lfx extension` is a sub-app under the Authoring help panel so future tickets (LE-1016 init/dev, LE-1018 reload) can attach without colliding with the existing `lfx validate` (which validates flow JSON, not extensions). * fix(lfx): fall back to tomli on Python 3.10 in extension manifest loader tomllib is stdlib only on 3.11+, but lfx supports 3.10-3.13. Use the existing tomli runtime dependency as the 3.10 fallback (same API, so the import alias keeps the rest of the module unchanged). Fixes ModuleNotFoundError seen in CI on the 3.10 job. * Update src/lfx/tests/unit/extension/test_schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/extension/schema.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update src/lfx/src/lfx/cli/_extension_commands.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat(lfx): add single-Bundle loader and LANGFLOW_COMPONENTS_PATH discovery (LE-1015) Introduces lfx.extension.loader: the runtime that turns an Extension on disk into LoadedComponent records keyed by ext:<bundle>:<Class>@<slot>. Two paths in: - load_extension(root): one manifest, one Bundle, registered at @official. Re-checks multi-bundle at runtime (defense-in-depth vs. the schema). - discover_inline_bundles(paths): each subfolder of LANGFLOW_COMPONENTS_PATH is a Bundle at @extra. Walk order is platform-independent (sorted dirs, user-declared path order). First-wins on duplicate names; second emits duplicate-inline-bundle warning that names both paths. Manifest-first precedence helpers (installed_extension_roots, manifest_owning_distributions, filter_plugin_entry_points) let callers of the legacy langflow.plugins entry-point loader skip distributions that ship a manifest, so component entry-points are not double-registered. New typed error codes: module-import-failed, duplicate-component-name, duplicate-distribution, duplicate-inline-bundle, inline-bundle-name-invalid. Each ships with a format branch and a snapshot test. Tests cover the AC: single-bundle happy path, multi-bundle rejection, missing/empty/no-Component bundle, duplicate class names, deterministic walk order, recursive discovery, inline-bundle first-wins + dot-dir skip + bundle.json metadata, manifest-first precedence partition, PEP-503 distribution-name canonicalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1015 review feedback - Drop the unproduced duplicate-distribution error code; LE-1022 will add it once startup-time discovery has a place to surface it. Restores the invariant that every code in ERROR_CODES has a producer. - Clarify that intra-bundle relative imports are NOT supported in v0; only absolute references between bundle modules work in this milestone. - Use strict=False when re-resolving bundle_root in the walker; the path was already existence-checked, and a concurrent removal in the narrow window should not raise across the loader's public boundary. - Hoist the json import out of _read_inline_bundle_json's body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lfx): split extension loader into a small subpackage (LE-1015) Addresses the file-size feedback from the LE-1015 review. The single 870-line loader.py becomes a flat package keyed off the four section banners that already existed inline: loader/ __init__.py # re-exports the public surface _types.py # SLOT constants, LoadedComponent, LoadResult _discovery.py # filesystem walk + importlib.util orchestration _detection.py # Component subclass identification (MRO heuristic) _orchestrator.py # load_extension, discover_inline_bundles _plugins.py # manifest-first precedence over langflow.plugins Largest file is now _orchestrator.py at 440 LOC (was 870); every file is well under the 800-LOC project guideline. No behavior change: the public import paths from lfx.extension are unchanged, all 38 loader tests still pass. ``_canonicalize_distribution`` is now exported as ``canonicalize_distribution`` from ``loader._plugins`` (it's a stable PEP-503 helper that downstream modules will reach for, so it loses the private underscore). The test suite is split to mirror the package: tests/unit/extension/loader/ conftest.py # shared fixtures, FakeDist, autouse scrub test_load_extension.py # @official slot, identity, failure modes test_inline_bundles.py # LANGFLOW_COMPONENTS_PATH, @extra slot test_plugins.py # manifest-first precedence helpers test_types.py # LoadedComponent, LoadResult, code parity Each test file imports its own slice of the public API and pulls fixtures from conftest, so a reader looking at one banner can read it in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lfx): add `extension init` and `extension dev` CLIs (LE-1016) The two scaffolding CLIs an Extension author types: - `lfx extension init <target>` writes the basic single-Bundle template (manifest with $schema, README, .gitignore, one Component subclass + a pytest smoke test). AC #1: the generated extension validates clean against LE-1014. AC #2: the generated test file is a valid pytest module that exercises the component's build() method. AC #3: any --template other than 'basic' fails with a typed template-deferred-in-this-milestone error and a non-zero exit. Refuses to scaffold over a non-empty target dir. - `lfx extension dev <target>` validates the local extension, records its absolute path in <config_dir>/extensions/dev_extensions.json, prints reload instructions, and execs `langflow run` (or `python -m langflow` when langflow isn't on PATH). --skip-launch registers without launching (used by tests + external dev-server scripts); --skip-validate lets authors register a known-broken manifest to debug it under the loader. Stack: - Wave 0: error codes (extension-target-exists, extension-target-invalid, local-extension-missing) with format branches and snapshot tests. - Wave 1: lfx.extension.init_template -- pure-data scaffolder, no Typer dependency so the CLI is a thin shell over it. - Wave 1: lfx.extension.dev_registry -- atomic JSON state file under the langflow user-cache dir; helpers for register / list / unregister / load_dev_extensions / dev_extension_component_paths. - Wave 2: lfx.cli._extension_commands gains init/dev subcommands. - Wave 2: langflow.main lifespan hook reads the dev registry after bundle loading and extends components_path with each registered bundle dir, so the existing palette discovery picks up dev extensions. Missing paths surface as local-extension-missing warnings (AC #5) without aborting startup. Tests (84 new, 197 total in tests/unit/extension/): - test_init_template.py: AC scenarios + identifier derivation + deterministic file shape. - test_dev_registry.py: register/list/unregister round-trip, idempotent re-register refreshes timestamp, malformed state file treated as empty, missing-path warning, recovery when path reappears, env-var override precedence. - test_cli.py: AC #1 init->validate, AC #3 deferred templates, --skip-launch registers without launching, --skip-validate short-circuits the pre-flight pass. - test_errors.py: snapshot rows for all three new codes; the every-known-code-has-a-snapshot test enforces parity. LE-1018 (reload) reuses load_dev_extensions; LE-1022 (installed-pkg discovery) shares the components_path extension pattern. AC #4 ("boots Langflow with the extension visible in the palette within 5s") is delivered jointly by `extension dev` (registers + execs) and the lifespan hook (loads on startup). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lfx): address LE-1016 review feedback Fixes the four HIGH issues + the elevated LOW from the review pass: 1. dev_extension_component_paths now forwards EVERY warning, not just local-extension-missing. Previously a duplicate-component-name (or any future warning code) was silently dropped, hiding real signal from the lifespan hook's logs. 2. Defensive emit when a LoadResult has components but source_path is None. The current loader always sets source_path, but a future hand-built LoadResult could violate that contract; we now surface a typed local-extension-missing error rather than dropping the extension silently. 3. Replaced the fragile ``min(len(parts))`` bundle-root selection with a relative-to-source-path measurement. Handles deep-vs-shallow sibling extensions correctly without depending on absolute path depth. 4. Generated README now documents the langflow/lfx prerequisite under the Develop section so authors know `pytest` requires the lfx environment, not just Python. 5. Forced LANGFLOW_LAZY_LOAD_COMPONENTS=false unconditionally in the `extension dev` exec env (was setdefault, which let a developer's global lazy-loading export silently hide their dev components from the palette and miss AC #4's 5s budget). Plus three MEDIUMs: - sys.modules cleanup in test_generated_test_file_runs_against_generated_component so a later test importing the same dotted path doesn't pick up a stale module from a deleted tmp_path. Component instantiation moved inside the try block because Component.__init__ uses inspect.getsourcefile against self.__class__'s still-live module. - Added regex-drift test that pins the init_template patterns to match manifest.py's so a schema regex change can't quietly produce invalid scaffolded manifests. - Added two new dev_registry tests covering the forward-all-warnings contract and the source_path=None defense. Tests: 201 passing (4 new); ruff check + format clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: LangflowCompat -> LfxCompat * fix: Remove references to ticket * fix: encode maxItems constraint in schema * fix: deferred fields and schema version * Fix ruff errors * fix: align loader tests with renamed manifest API and strip ticket refs The merge of feat/extension-validate brought in the LfxCompat / compat rename, but the loader test fixtures still used LangflowCompat / bundle_api and failed at the manifest layer. Update conftest.py + test_load_extension.py to the post-rename API. Strip 17 LE-XXXX ticket references from loader source files, errors.py loader-specific comments, and the four loader test docstrings, matching the convention applied to LE-1014. Descriptive prose preserved. Promote _BUNDLE_NAME_RE to BUNDLE_NAME_RE on the manifest module so the loader's orchestrator no longer reaches across modules into a private name. * fix(lfx): align init template with renamed manifest API After merging feat/extension-loader into this branch, two surfaces fell out of sync with the renamed manifest schema: - init_template generates extension.json with `lfx: {bundle_api: [1]}`, but the validator now requires `lfx: {compat: ["1"]}` per LfxCompat. This made `test_basic_template_validates_clean` fail with manifest-invalid (`lfx.compat: Field required; lfx.bundle_api: Extra inputs are not permitted`). - test_init_template_regexes_match_manifest_schema reads `manifest_mod._BUNDLE_NAME_RE`, but that symbol was promoted to public `BUNDLE_NAME_RE` in |
|||
| 5e9f0c37bc |
Merge branch 'release-1.9.3' into release-1.10.0
# Conflicts: # .secrets.baseline # docs/package.json # pyproject.toml # src/backend/base/langflow/initial_setup/starter_projects/Vector Store RAG.json # src/backend/base/pyproject.toml # src/backend/tests/unit/components/files_and_knowledge/test_retrieval.py # src/frontend/package-lock.json # src/frontend/package.json # src/frontend/src/components/core/parameterRenderComponent/components/modelInputComponent/__tests__/ModelInputComponent.test.tsx # src/frontend/src/components/core/parameterRenderComponent/components/modelInputComponent/index.tsx # src/frontend/src/modals/modelProviderModal/hooks/useProviderConfiguration.ts # src/lfx/pyproject.toml # src/lfx/src/lfx/_assets/component_index.json # src/sdk/pyproject.toml # uv.lock |
|||
| cf488e545c | fix: Gate more tests on package presence | |||
| a80cf1c649 |
chore: support Python 3.14 (#13085)
* feat: upgrade Docker images to Python 3.14 (experimental)
- Update pyproject.toml: requires-python from <3.14 to <3.15
- Update all 5 Dockerfiles to use Python 3.14:
- Builder stage: python3.12-trixie-slim → python3.14-trixie-slim
- Runtime stage: python:3.12-slim-trixie → python:3.14-slim-trixie
- src/backend/base/pyproject.toml
- docker/build_and_push_base.Dockerfile
- docker/build_and_push.Dockerfile
- docker/build_and_push_backend.Dockerfile
- docker/build_and_push_with_extras.Dockerfile
- docker/build_and_push_ep.Dockerfile
Python 3.14.5 was released on May 10, 2026. Upgrading to the latest
Python version should help reduce CVE vulnerabilities in Docker images.
This is an experimental change to test Python 3.14 compatibility:
- Docker images will use Python 3.14
- CI/CD workflows still test on Python 3.10-3.13
- Nightly build will validate if dependencies work with 3.14
- Can be reverted quickly if issues are found
- Monitor nightly build for failures
- If successful, update CI/CD workflows to add Python 3.14 to test matrix
- If failures occur, revert and investigate compatibility issues
* chore: support Python 3.14
Bump requires-python upper bound to <3.15 across langflow, langflow-base,
lfx, and langflow-sdk, and add 3.14 to CI test matrices so PRs are gated
on 3.14 compatibility.
Conditional pins for transitive deps without 3.14 wheels at the existing
caps:
- onnxruntime: >=1.26 on 3.14 (existing <1.24 cap retained for 3.10)
- faiss-cpu: >=1.13.2 on 3.14 (existing ==1.9.0.post1 retained for <3.14)
* chore: marker-gate IBM watsonx packages for Python 3.14
ibm-watsonx-ai 1.3.x cannot import on Python 3.14: its StrEnum subclasses
override __init__ in a way that conflicts with 3.14's reworked enum
__set_name__ path (TypeError on KnowledgeBaseFieldRole class creation).
Cap ibm-watsonx-ai, langchain-ibm, and the ibm-watsonx-orchestrate-*
extras at python_version<'3.14' until upstream adapts. Watsonx component
imports are already lazy, so this surfaces only at component-access time
on 3.14, where the existing ImportError handler in
lfx.components.ibm.__init__ degrades it gracefully.
test_model_utils.py imports ChatWatsonx at module top to verify
get_model_name resolves model_id; guard that import so the test
module skips on 3.14 instead of breaking collection.
* [autofix.ci] apply automated fixes
* chore: upgrade remaining Docker images to Python 3.14
build_and_push*.Dockerfile already moved to 3.14 in the merge from
feat/upgrade-python-3.14-docker-images. Apply the same bump to the
dev, devcontainer, and lfx Docker images so all in-repo Dockerfiles
share one Python version.
* [autofix.ci] apply automated fixes
* chore: gate 3.14-broken extras to python_version<'3.14'
cuga (and its transitive fastembed -> py-rust-stemmers source build),
altk/agent-lifecycle-toolkit (re-pulls ibm-watsonx-ai which was already
gated), langchain-pinecone, langwatch, ragstack-ai-knowledge-store, and
OpenDsStar all pin themselves below 3.14 upstream. Without a marker
guard, the workspace lock still tries to install them and a Docker
`uv sync` then attempts source builds that require Rust (CI Docker
test failure with py-rust-stemmers 0.1.5).
Add python_version<'3.14' (or <'3.13' for ragstack) to each pin so
those packages are simply omitted from the 3.14 install set until
upstream catches up. uv pip check is now clean on 3.14.2.
* test: skip test_altk_agent on Python 3.14
altk (agent-lifecycle-toolkit) is gated to python_version<'3.14'
upstream and now via the langflow-base [altk] extra marker. The test
imports altk at module top to exercise ALTKAgentComponent; guard the
import so collection skips on 3.14 instead of failing.
* test: skip remaining altk tests on Python 3.14
Three more test modules import lfx.base.agents.altk_* at module top,
which transitively imports altk. Add the same module-level skip guard
as test_altk_agent.py:
- test_altk_agent_logic.py
- test_altk_agent_tool_conversion.py
- test_conversation_context_ordering.py
* fix(calculator): replace removed ast.Num with ast.Constant for Python 3.14
ast.Num was deprecated in Python 3.8 in favor of ast.Constant and
removed entirely in 3.14. The calculator tool and its core walker
crashed on 3.14 with 'module ast has no attribute Num'.
Switch the isinstance checks to ast.Constant + isinstance(node.value,
(int, float)), and drop the now-dead ast.Num backwards-compat branch
in calculator_core.py (the ast.Constant branch already handles every
input it would have matched). Update the parser unit-test fixtures
to construct ast.Constant nodes.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* test: skip LangWatch HTTP instrumentation tests on Python 3.14
langwatch is gated to python_version<'3.14' upstream. The
TestLangWatchHttpInstrumentation class patches langwatch.setup and
langwatch.trace inside a fixture, which requires langwatch to be
importable; on 3.14 the test errors with ModuleNotFoundError.
Guard the class with a skipif on langwatch availability.
* test: allow IBM and altk components to be missing on Python 3.14
test_all_modules_importable enforces that every component in __all__
imports cleanly. On 3.14 the ibm-watsonx-ai, langchain-ibm, and altk
packages are gated upstream and intentionally not installed, so the
WatsonxAI, WatsonxEmbeddings, and ALTKAgent components fail to import
as designed.
- test_all_components_in_categories_importable: accept a known-gated
deny-list on 3.14 instead of failing.
- test_all_lfx_component_modules_directly_importable: extend the
existing 'missing optional dependency' allowlist with altk,
langchain_ibm, and ibm_watsonx_ai.
* fix(lfx): pass ensure_exists to user_cache_dir for Python 3.14
Python 3.14 tightened PurePath.__init__ to reject unknown keyword
arguments. The KeyedWorkerLockManager constructor was passing
ensure_exists=True to Path() instead of user_cache_dir(), which
silently worked on 3.10-3.13 but raises TypeError on 3.14 and also
meant the cache directory was never actually being created.
Move the kwarg to user_cache_dir() where it belongs and update the
two unit tests that asserted the buggy call shape.
* [autofix.ci] apply automated fixes
* test: accept either ZIP or JSON error path on Python 3.14
Python 3.14's zipfile.is_zipfile() now validates the central-directory
signature in addition to EOCD, so the garbage+EOCD payload no longer
passes the is_zipfile() dispatch in the /flows/upload/ route. The
endpoint still returns 400 with a descriptive detail (now from the JSON
branch); only the message wording differs, which the test was asserting
verbatim.
* fix(lfx): normalize cors_origins ['*'] back to '*' on Python 3.14
Pydantic-settings on Python 3.14 parses the env var '*' into ['*']
before the cors_origins field validator runs (the list[str] | str union
resolves differently than on 3.10-3.13). Collapse that back to the
bare-string wildcard so downstream consumers — including
warn_about_future_cors_changes and the test suite — see the same shape
on every supported Python version.
---------
Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit
|
|||
| 1955f8fae5 |
chore: support Python 3.14 (#13085)
* feat: upgrade Docker images to Python 3.14 (experimental) ## Changes - Update pyproject.toml: requires-python from <3.14 to <3.15 - Update all 5 Dockerfiles to use Python 3.14: - Builder stage: python3.12-trixie-slim → python3.14-trixie-slim - Runtime stage: python:3.12-slim-trixie → python:3.14-slim-trixie ## Files Updated - src/backend/base/pyproject.toml - docker/build_and_push_base.Dockerfile - docker/build_and_push.Dockerfile - docker/build_and_push_backend.Dockerfile - docker/build_and_push_with_extras.Dockerfile - docker/build_and_push_ep.Dockerfile ## Rationale Python 3.14.5 was released on May 10, 2026. Upgrading to the latest Python version should help reduce CVE vulnerabilities in Docker images. ## Testing Strategy This is an experimental change to test Python 3.14 compatibility: - Docker images will use Python 3.14 - CI/CD workflows still test on Python 3.10-3.13 - Nightly build will validate if dependencies work with 3.14 - Can be reverted quickly if issues are found ## Next Steps - Monitor nightly build for failures - If successful, update CI/CD workflows to add Python 3.14 to test matrix - If failures occur, revert and investigate compatibility issues * chore: support Python 3.14 Bump requires-python upper bound to <3.15 across langflow, langflow-base, lfx, and langflow-sdk, and add 3.14 to CI test matrices so PRs are gated on 3.14 compatibility. Conditional pins for transitive deps without 3.14 wheels at the existing caps: - onnxruntime: >=1.26 on 3.14 (existing <1.24 cap retained for 3.10) - faiss-cpu: >=1.13.2 on 3.14 (existing ==1.9.0.post1 retained for <3.14) * chore: marker-gate IBM watsonx packages for Python 3.14 ibm-watsonx-ai 1.3.x cannot import on Python 3.14: its StrEnum subclasses override __init__ in a way that conflicts with 3.14's reworked enum __set_name__ path (TypeError on KnowledgeBaseFieldRole class creation). Cap ibm-watsonx-ai, langchain-ibm, and the ibm-watsonx-orchestrate-* extras at python_version<'3.14' until upstream adapts. Watsonx component imports are already lazy, so this surfaces only at component-access time on 3.14, where the existing ImportError handler in lfx.components.ibm.__init__ degrades it gracefully. test_model_utils.py imports ChatWatsonx at module top to verify get_model_name resolves model_id; guard that import so the test module skips on 3.14 instead of breaking collection. * [autofix.ci] apply automated fixes * chore: upgrade remaining Docker images to Python 3.14 build_and_push*.Dockerfile already moved to 3.14 in the merge from feat/upgrade-python-3.14-docker-images. Apply the same bump to the dev, devcontainer, and lfx Docker images so all in-repo Dockerfiles share one Python version. * [autofix.ci] apply automated fixes * chore: gate 3.14-broken extras to python_version<'3.14' cuga (and its transitive fastembed -> py-rust-stemmers source build), altk/agent-lifecycle-toolkit (re-pulls ibm-watsonx-ai which was already gated), langchain-pinecone, langwatch, ragstack-ai-knowledge-store, and OpenDsStar all pin themselves below 3.14 upstream. Without a marker guard, the workspace lock still tries to install them and a Docker `uv sync` then attempts source builds that require Rust (CI Docker test failure with py-rust-stemmers 0.1.5). Add python_version<'3.14' (or <'3.13' for ragstack) to each pin so those packages are simply omitted from the 3.14 install set until upstream catches up. uv pip check is now clean on 3.14.2. * test: skip test_altk_agent on Python 3.14 altk (agent-lifecycle-toolkit) is gated to python_version<'3.14' upstream and now via the langflow-base [altk] extra marker. The test imports altk at module top to exercise ALTKAgentComponent; guard the import so collection skips on 3.14 instead of failing. * test: skip remaining altk tests on Python 3.14 Three more test modules import lfx.base.agents.altk_* at module top, which transitively imports altk. Add the same module-level skip guard as test_altk_agent.py: - test_altk_agent_logic.py - test_altk_agent_tool_conversion.py - test_conversation_context_ordering.py * fix(calculator): replace removed ast.Num with ast.Constant for Python 3.14 ast.Num was deprecated in Python 3.8 in favor of ast.Constant and removed entirely in 3.14. The calculator tool and its core walker crashed on 3.14 with 'module ast has no attribute Num'. Switch the isinstance checks to ast.Constant + isinstance(node.value, (int, float)), and drop the now-dead ast.Num backwards-compat branch in calculator_core.py (the ast.Constant branch already handles every input it would have matched). Update the parser unit-test fixtures to construct ast.Constant nodes. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * test: skip LangWatch HTTP instrumentation tests on Python 3.14 langwatch is gated to python_version<'3.14' upstream. The TestLangWatchHttpInstrumentation class patches langwatch.setup and langwatch.trace inside a fixture, which requires langwatch to be importable; on 3.14 the test errors with ModuleNotFoundError. Guard the class with a skipif on langwatch availability. * test: allow IBM and altk components to be missing on Python 3.14 test_all_modules_importable enforces that every component in __all__ imports cleanly. On 3.14 the ibm-watsonx-ai, langchain-ibm, and altk packages are gated upstream and intentionally not installed, so the WatsonxAI, WatsonxEmbeddings, and ALTKAgent components fail to import as designed. - test_all_components_in_categories_importable: accept a known-gated deny-list on 3.14 instead of failing. - test_all_lfx_component_modules_directly_importable: extend the existing 'missing optional dependency' allowlist with altk, langchain_ibm, and ibm_watsonx_ai. * fix(lfx): pass ensure_exists to user_cache_dir for Python 3.14 Python 3.14 tightened PurePath.__init__ to reject unknown keyword arguments. The KeyedWorkerLockManager constructor was passing ensure_exists=True to Path() instead of user_cache_dir(), which silently worked on 3.10-3.13 but raises TypeError on 3.14 and also meant the cache directory was never actually being created. Move the kwarg to user_cache_dir() where it belongs and update the two unit tests that asserted the buggy call shape. * [autofix.ci] apply automated fixes * test: accept either ZIP or JSON error path on Python 3.14 Python 3.14's zipfile.is_zipfile() now validates the central-directory signature in addition to EOCD, so the garbage+EOCD payload no longer passes the is_zipfile() dispatch in the /flows/upload/ route. The endpoint still returns 400 with a descriptive detail (now from the JSON branch); only the message wording differs, which the test was asserting verbatim. * fix(lfx): normalize cors_origins ['*'] back to '*' on Python 3.14 Pydantic-settings on Python 3.14 parses the env var '*' into ['*'] before the cors_origins field validator runs (the list[str] | str union resolves differently than on 3.10-3.13). Collapse that back to the bare-string wildcard so downstream consumers — including warn_about_future_cors_changes and the test suite — see the same shape on every supported Python version. --------- Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 0b93705a8d |
fix: restore Langfuse feedback trace linking (#13081)
* fix: restore langfuse feedback trace linking * [autofix.ci] apply automated fixes * fix: add run id to placeholder graph * fix: gate langfuse feedback sync and surface failures Gate the background task on tracing being enabled and Langfuse being configured so we don't enqueue work that silently no-ops. Raise instead of swallowing missing-credentials and import errors so background-task failures hit logs. Add delete_feedback_score for the clear-feedback (True/False -> None) transition that previously fell through. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| da798a525d |
feat(components): Add Code Agents (CodeAct agent and OpenDsStar agent) and file ingestion components with supporting fixes (#12713)
* initial draft * update DS Star agent * chore(deps): use OpenDsStar 1.0.1 from PyPI * refactor: move OpenDsStar to namespace, update imports, remove agents shim * update open ds star version * improved logging * improved logging * fix: propagate source file path for DataFrame inputs in ingestion components DataFrame from CSV/Excel reads now carries the source file path via pandas attrs, so IngestionDescriber and FileContentRetriever can resolve the original file instead of searching per-row data keys. * feat: add files_ingestion component bundle and bump OpenDsStar to 1.0.8 Register files_ingestion as a sidebar bundle in the frontend and component registry. Update OpenDsStar dependency from 1.0.6 to 1.0.8. * Add file content retriever component and tests * Fix: Add tool_mode=True to FileContentRetriever outputs to prevent execution during build phase * Fix: Return empty DataFrame instead of raising error when file_path not provided during build * Update file content retriever component * refactor: rename IngestionDescriberComponent to FileDescriptionGeneratorComponent - Renamed class from IngestionDescriberComponent to FileDescriptionGeneratorComponent - Updated component name attribute and display name - Renamed files: ingestion_describer.py -> file_description_generator.py - Renamed test file: test_ingestion_describer.py -> test_file_description_generator.py - Updated all imports and references - Rebuilt component index BREAKING CHANGE: Component class name changed. Existing flows using IngestionDescriberComponent will need to be updated. * fix: improve FileContentRetriever tool descriptions for CodeAct agent - Add explicit warnings that tools expect string file paths only - Clarify that search results/Data objects should not be passed - Change error handling to raise ValueError (agents can handle exceptions) - Add build-phase safety (return empty results when no path provided) - Update tests to expect exceptions instead of error messages This fixes CodeAct agent failures where it was trying to pass search results (list of Data objects) to retrieve_content_as_dataframe() instead of extracting the file path string first. * fix: File Content Retriever now finds DataFrames with file_path in columns - Added caching for file maps to avoid rebuilding on each method call - Enhanced _get_file_maps() to check both attrs['source_file_path'] and 'file_path' column - Fixes agent issue where DataFrame from Read File component couldn't be found - Added test for DataFrame with file_path in columns scenario - All 30 tests pass (26 passed, 4 skipped) * fix: use to_csv() instead of to_string() for DataFrame text representation Fixes agent hanging issue when retrieving file content as DataFrame. The problem was that to_string() creates a formatted table representation that cannot be parsed as CSV. When agents called retrieve_content_as_dataframe(), it would try to parse this formatted text with pd.read_csv(), causing a ParserError and infinite retry loop. Changed to use to_csv(index=False) which produces valid CSV format that can be parsed back into a DataFrame when needed. - Maintains optimal performance by returning original DataFrame from dataframe_map - Only uses CSV text representation as fallback - All 26 tests pass - Works correctly for both text and tabular files * add agent template * improve agent template * bump OpenDsStar to 1.0.10 and update ingestion components Upgrade OpenDsStar dependency from 1.0.8 to 1.0.10 in both langflow-base and lfx pyproject.toml. Update file content retriever, file description generator, component tool, and custom component with related improvements. Expand test coverage for file content retriever. * fix FileContentRetriever multi-file data retrieval and add code_timeout The FileContentRetriever was broken for the multi-file case (when Read File outputs a DataFrame with file_path + text columns). Three bugs were fixed: 1. Multi-file DataFrame mapped to wrong data: When Read File produced a summary DataFrame (one row per file, columns = [file_path, text]), the retriever mapped each file_path to the entire 2-row summary DataFrame instead of extracting per-file content. The agent would get 2 rows instead of thousands of actual data rows. Fixed by iterating rows and extracting each file's text into text_map. 2. CSV text now eagerly parsed into DataFrames: For .csv/.tsv files in text_map that lack a pre-built DataFrame, _get_file_maps() now parses the text into a DataFrame eagerly during initialization. This makes retrieve_content_as_dataframe work for the multi-file case without needing upstream structured DataFrames. 3. Maps built once, not per tool call: The file maps were rebuilt on every tool call because component_tool.py deepcopy's the component, and the maps were only built lazily during retrieval. Now the early-return paths (empty file_path) eagerly call _get_file_maps() during component build time, so the cached maps survive deepcopy. This is critical for scalability with hundreds of large files. Additional changes: - Removed _is_likely_file_content heuristic that was incorrectly filtering out valid file content - Removed "Message" from input_types (was never handled) - Removed redundant inner import of DataFrame - Shortened error messages (cap available files to 5) - Added warning log for unsupported input types - Lazy CSV conversion: no longer eagerly converts structured DataFrames to CSV text in _get_file_maps, only converts on demand in retrieve_content - Added code_timeout input to OpenDsStarAgentComponent (default 60s, was hardcoded 30s) to prevent timeouts with large datasets - Added 19 unit tests covering all retrieval paths, caching, and the multi-file bug reproduction * disable as_dataframe tool from vector store to fix agent confusion The Chroma vector store's as_dataframe output was exposed as an agent tool, returning a 2-row file index DataFrame (file_path + text columns). The agent mistook this for actual data and tried to query 'price' on it, causing every query to fail with KeyError. Set tool_mode=False on the as_dataframe output so only search_documents is exposed as a tool from the vector store. The agent now follows the correct workflow: search_documents → get file path → retrieve_content_as_dataframe. * Revert "disable as_dataframe tool from vector store to fix agent confusion" This reverts commit |
|||
| eb5f83f950 |
fix: pin postgres image to bookworm in docker_example (#13027)
* fix: pin postgres image to bookworm in docker_example to prevent collation mismatch
The postgres:16 tag silently moved its base from Debian Bookworm
(glibc 2.36) to Trixie (glibc 2.41), causing a recurring collation
version mismatch warning on existing langflow-postgres volumes.
Pin to postgres:16-bookworm in both docker-compose files and update
the README so existing data volumes keep matching the OS locale data.
Refs: https://github.com/langflow-ai/langflow/issues/9608
* docs: pin postgres image to bookworm in current docs compose snippets
Mirror the docker_example pin in the four current docs that publish
copyable Compose snippets pairing postgres:16 with a persistent
langflow-postgres volume. Prevents the same Bookworm-to-Trixie
collation version mismatch warning when users follow docs instead
of docker_example.
Versioned historical docs are left as-is.
Refs: https://github.com/langflow-ai/langflow/issues/9608
* fix: switch postgres pin from bookworm to trixie for OS consistency
Aligns the pinned postgres base with the langflow runtime image, which
moved to Debian Trixie in #12990. Keeping postgres on bookworm would
have diverged the stack and locked the database to an aging glibc that
will receive fewer security backports as bookworm ages into oldstable.
The pin itself still solves the original bug — postgres:16 cannot
silently roll its OS underneath an existing volume.
Document the one-time REFRESH COLLATION VERSION step for users
upgrading from a bookworm-initialized volume in docker_example/README.md.
Refs: https://github.com/langflow-ai/langflow/issues/9608
(cherry picked from commit
|
|||
| 7504eb4c72 |
fix: pin postgres image to bookworm in docker_example (#13027)
* fix: pin postgres image to bookworm in docker_example to prevent collation mismatch The postgres:16 tag silently moved its base from Debian Bookworm (glibc 2.36) to Trixie (glibc 2.41), causing a recurring collation version mismatch warning on existing langflow-postgres volumes. Pin to postgres:16-bookworm in both docker-compose files and update the README so existing data volumes keep matching the OS locale data. Refs: https://github.com/langflow-ai/langflow/issues/9608 * docs: pin postgres image to bookworm in current docs compose snippets Mirror the docker_example pin in the four current docs that publish copyable Compose snippets pairing postgres:16 with a persistent langflow-postgres volume. Prevents the same Bookworm-to-Trixie collation version mismatch warning when users follow docs instead of docker_example. Versioned historical docs are left as-is. Refs: https://github.com/langflow-ai/langflow/issues/9608 * fix: switch postgres pin from bookworm to trixie for OS consistency Aligns the pinned postgres base with the langflow runtime image, which moved to Debian Trixie in #12990. Keeping postgres on bookworm would have diverged the stack and locked the database to an aging glibc that will receive fewer security backports as bookworm ages into oldstable. The pin itself still solves the original bug — postgres:16 cannot silently roll its OS underneath an existing volume. Document the one-time REFRESH COLLATION VERSION step for users upgrading from a bookworm-initialized volume in docker_example/README.md. Refs: https://github.com/langflow-ai/langflow/issues/9608 |
|||
| 78f82cae32 |
perf: Enhance Gunicorn preload functionality for Langflow (#12778)
* perf: Enhance Gunicorn preload functionality for Langflow
- Introduced a new preload module to optimize memory usage by running fork-safe initialization in the Gunicorn master process.
- Updated the lifespan management in `main.py` to check if the master has preloaded resources, allowing workers to inherit state and skip redundant setup.
- Adjusted the server loading process to accommodate the new preload logic, ensuring efficient resource management across worker processes.
* feat(cache): Implement teardown method for RedisCache to prevent socket leaks
- Added a `teardown` method to the `RedisCache` class to close the Redis connection, addressing potential socket leaks during process forking.
- Introduced unit tests to verify the functionality of the `teardown` method, ensuring it handles client closure and errors gracefully.
- Tests cover scenarios including normal closure, error handling during closure, and teardown with URL-based connections.
* feat(cache): Implement teardown method in RedisCache to prevent socket leaks
- Added a `teardown` method to the `RedisCache` class, ensuring proper closure of the Redis client connection before forking to avoid socket leaks.
- Created comprehensive unit tests to validate the functionality of the `teardown` method, covering various scenarios including normal operation and error handling.
- Updated existing tests to reflect the new teardown functionality and ensure RedisCache is recognized as an instance of `ExternalAsyncBaseCacheService`.
* refactor(preload): Simplify DB engine disposal logic in master preload
- Removed exception handling around the DB engine disposal to streamline the process, ensuring that the engine is disposed of without unnecessary error logging. This change enhances code clarity and maintains the intended functionality of resource management during the preload phase.
* refactor(preload): Streamline cache service teardown logic
- Simplified the cache service socket closure process in the master preload function to prevent sharing across forks. This change enhances code clarity by removing unnecessary exception handling while maintaining the intended functionality of resource management during the preload phase.
* feat(preload): Introduce per-step completion flags for improved state management
- Added completion flags in the preload state to track the status of various initialization steps, including profile picture copying, starter project creation, agentic global variable initialization, MCP server configuration, and flow loading.
- Updated the lifespan management in `main.py` to utilize these flags, allowing the system to skip redundant setup tasks if they have already been completed during the preload phase.
- This enhancement improves resource management and ensures that the application behaves correctly in a multi-worker environment.
* refactor(lifespan): Replace temp dir management with get_owned_temp_dirs function
- Updated the lifespan management in `main.py` to utilize the new `get_owned_temp_dirs` function, which encapsulates the logic for determining temp directory ownership based on the process type (master or worker).
- Removed the `is_master` check from the lifespan function, simplifying the code and enhancing clarity regarding temp directory cleanup responsibilities.
- Added the `get_owned_temp_dirs` function in `preload.py` to centralize temp directory ownership logic, ensuring that workers do not attempt to clean up directories owned by the master process.
* refactor(lifespan): Enhance initialization flow with conditional gates
- Introduced conditional gates in the `get_lifespan` function to manage the initialization of profile pictures, super users, bundles, component types, and starter projects based on their completion status.
- Improved logging to provide clearer insights into which steps are being skipped or executed, enhancing the overall clarity of the initialization process.
- Updated the preload logic in `preload.py` to ensure that agentic global variables and MCP server configuration are only initialized when necessary, maintaining efficient resource management in a multi-worker environment.
* fix: resolve double-call of initialize_auto_login_default_superuser and add preload tests
- Fix double-call issue: setup_superuser() now handles AUTO_LOGIN completely with file lock
- Add comprehensive unit tests for preload.py covering failure-fallback contract
- Simplify code by doing superuser initialization in initialize_services() (called early in both preload and worker startup)
- File lock protects multi-worker race conditions when preload is disabled
- Tests verify critical step failures propagate, best-effort steps continue on failure
Made-with: Cursor
* fix(preload): resolve missing import and agentic initialization conflicts
Fixed critical bugs introduced in
|
|||
| 596e98b9b7 |
fix: Respect proxy environment variables in URL Component (#12989)
* fix: URLComponent ignores proxy in async mode (#12285) The URLComponent fails to connect for users behind corporate proxies because its asynchronous mode does not recognize standard system proxy environment variables. The component defaults to use_async=True, which initializes the RecursiveUrlLoader; the underlying async loader does not natively respect system proxy environment variables, so the component attempts a direct connection and fails in restricted network environments. Detect standard proxy environment variables (http_proxy, HTTP_PROXY, https_proxy, HTTPS_PROXY) in URLComponent._create_loader. If a proxy is detected and use_async is enabled, override use_async to False so the loader uses its synchronous implementation, which natively respects system proxies. Empty and whitespace-only values are correctly evaluated as no-proxy and do not trigger the fallback. Closes #10297 * fix(URLComponent): broaden proxy detection and harden tests Address review findings on the proxy fix: - Add ALL_PROXY and all_proxy to the detected env vars. ALL_PROXY is commonly set in corporate and container environments (curl, git, many Unix tools honour it), and is sometimes the only proxy var configured. - Replace the unused proxy_url string with a single boolean any() over the env var keys, since only the presence of a proxy is consulted. - Reorganize the proxy tests under TestURLComponentProxyHandling with a shared default-attributes helper, parametrize over all six env var spellings, and add coverage for multiple-simultaneous-proxies and the use_async=False path (which should not log a warning). * [autofix.ci] apply automated fixes * fix: remove no_proxy="*" macOS startup hack so corporate proxies work Two startup paths set `os.environ["no_proxy"] = "*"` on macOS, which disables proxy use for every HTTP client in the process and every child gunicorn worker (httpx, requests, urllib3 — and therefore the OpenAI, Anthropic, Groq, etc. SDKs that wrap them). For users behind corporate proxies on macOS this made every external LLM call unroutable, even with HTTPS_PROXY properly set. The override traces to commit history with no concrete justification — the only artifact is a Stack Overflow link about a uWSGI segfault, but Langflow uses gunicorn, not uWSGI. Bench-verified that gunicorn boots cleanly and serves requests on macOS without it (1 worker via LangflowApplication, OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES retained, HTTPS_PROXY survives intact in parent and child). Verification: - gunicorn worker spawns and serves /health (200 OK), exits cleanly - httpx, urllib, requests all resolve HTTPS_PROXY after the macOS init (previously: all three returned empty proxy maps because of no_proxy=*) - OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES still set — the actual fork-safety fix is preserved Closes the macOS half of #10297. Companion fix for the async URLComponent half is in #12285 / branch pr-12285-rebased. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Orjson update --------- Co-authored-by: Diogo Veiga <diogo.veiga@tecnico.ulisboa.pt> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 6d2e255f5d |
Merge remote-tracking branch 'origin/release-1.9.2' into release-1.10.0
# Conflicts: # .secrets.baseline # pyproject.toml # src/backend/base/langflow/initial_setup/starter_projects/Knowledge Retrieval.json # src/backend/base/langflow/initial_setup/starter_projects/Vector Store RAG.json # src/backend/base/pyproject.toml # src/frontend/package-lock.json # src/frontend/package.json # src/lfx/pyproject.toml # src/lfx/src/lfx/_assets/component_index.json # src/sdk/pyproject.toml # uv.lock |
|||
| af8cd0219e |
fix: pass Google embedding dimensions (#12876)
fix: pass google embedding dimensions |
|||
| 7a84f8b254 |
fix(mcp): enforce auth on /streamable for auth_type=oauth projects (#12756)
* fix(mcp): enforce authentication on /streamable for auth_type=oauth projects Previously, projects configured with auth_type=oauth did not enforce authentication on the /streamable (and /sse) endpoints when MCP Composer was enabled. verify_project_auth only enforced API key authentication for auth_type=apikey; for oauth it fell through to the superuser fallback, returning HTTP 200 with full initialize/tools/list payloads to unauthenticated requests. This treats auth_type=oauth the same as auth_type=apikey on the direct langflow MCP transport endpoints: requests must present a valid API key or they are rejected with HTTP 401. End users of OAuth-configured projects should continue to connect via the configured MCP Composer OAuth endpoint (which proxies to these endpoints with backend credentials). * fix(mcp): preserve composer loopback forwarding on OAuth auth enforcement Address review feedback on PR #12756. Requiring an API key for every OAuth request would have broken the intended MCP Composer proxy path: the composer subprocess is started with only the Langflow endpoint URL and OAuth env vars and does not currently inject a backend x-api-key when it forwards authenticated OAuth traffic, so hardening /streamable and /sse would turn every legitimate composer-forwarded request into a 401. Keep the enforcement against direct unauthenticated external access, but trust requests whose direct TCP peer is a loopback address (the on-host MCP Composer subprocess). Remote callers still need a valid x-api-key to hit the project transport endpoints when auth_type=oauth. Add coverage for the loopback passthrough and the _is_loopback_client helper, and gate the rejection tests on a patched non-loopback client so they exercise the remote path even though httpx AsyncClient reports 127.0.0.1 by default. * fix(mcp): drop loopback trust shortcut; always require API key for OAuth transport Follow-up to PR #12756. The loopback-based passthrough from the prior commit is unsafe in deployments where Langflow sits behind a same-host reverse proxy or sidecar: the proxy becomes the direct TCP peer, so external unauthenticated traffic satisfies the loopback check and falls through to the superuser, reopening the original bypass. There is no composer-specific identity on the wire today — mcp-composer forwards without any Langflow credential — so loopback address cannot distinguish the trusted subprocess from another loopback peer. Require a valid x-api-key on /streamable and /sse for every OAuth project regardless of source. Until mcp-composer can forward a project-scoped backend credential, the composer proxy path will return 401; this is the intended secure behavior. Remove the _is_loopback_client helper, the client_host plumbing into verify_project_auth, and the associated tests. Drop the force_non_loopback_client fixture; the rejection tests now exercise the real code path directly. * fix(mcp): clarify OAuth 401 detail about the current API-key requirement The previous message asked callers to connect through MCP Composer, but the composer proxy path currently returns 401 itself (see PR #12756 discussion) because mcp-composer cannot forward a backend credential yet. Describe the actual working path — use an x-api-key — and note that composer-based access is pending credential forwarding support. * fix: No timeout when connect to oauth project * Secrets baseline update * fix: Standardize paths on Windows |
|||
| fc7f6c4c6c |
fix: Handle videos with no comments in YouTube Comments component (#10633)
* fix: Handle videos with no comments in YouTube Comments component - Define column order once to avoid code duplication - Create empty DataFrame with proper schema when no comments exist - Prevents KeyError when trying to reorder columns on empty DataFrame - Returns consistent DataFrame structure regardless of comment count * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * [autofix.ci] apply automated fixes * Update component_index.json --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| b9e81f6edd | Merge remote-tracking branch 'origin/main' into release-1.10.0 | |||
| 0732423f27 |
chore: security patch (#12725)
* chore: security patch security patch * chore: upgrade package-lock.json * chore: smolagents and transformer update * chore: redis upgrade * chore: litellm upgrade * fix: Pin click to avoid lower versions ## Root cause **litellm 1.83.5+** introduced an exact pin `click==8.1.8` in its `requires_dist` (upstream bug [BerriAI/litellm#26154](https://github.com/BerriAI/litellm/issues/26154) — "Pinning exact dependency versions breaks downstream consumers"). When this branch bumped litellm to 1.83.11, uv was forced to downgrade click from 8.3.2 → 8.1.8. Click 8.2+ captures stderr separately by default; 8.1 merges it — hence `ValueError: stderr not separately captured`. ## Fix - [pyproject.toml:155](pyproject.toml:155) — added `"click>=8.3.0"` to `[tool.uv] override-dependencies` with comment linking the upstream issue. - [src/backend/base/pyproject.toml:112](src/backend/base/pyproject.toml:112) — same override so `langflow-base` Docker builds pick it up too. - Regenerated `uv.lock` (click: 8.1.8 → 8.3.3). - Patched the `click` entry in `src/backend/base/uv.lock` to 8.3.3 (that lockfile is used by `docker/build_and_push_base.Dockerfile` via `uv sync --frozen`; uv treats backend/base as a workspace member so I updated its click block directly — no other references to 8.1.8 remain). - Reverted [src/lfx/tests/unit/cli/test_run_real_flows.py](src/lfx/tests/unit/cli/test_run_real_flows.py) back to its original `result.stderr` usage — now passes cleanly with click 8.3.3. All 19 tests in the file pass. * Update build_and_push_backend.Dockerfile The `uv pip install` command doesn't read `[tool.uv] override-dependencies` from the target packages' pyproject.toml — those overrides only apply during `uv lock`/`uv sync`. The fix is to pass the overrides explicitly via `--overrides`. Now let me verify it works by running the same install locally. Resolution succeeded — click==8.3.3, aiohttp==3.13.5, litellm==1.83.11 all coexist. Let me also test without `--overrides` to confirm I reproduce the original failure. Reproduced the Docker failure locally. Now test with the overrides file: Fix verified — resolution succeeds with the overrides file, failing without. Let me clean up and show the final Dockerfile change. ## Summary **Root cause:** The Dockerfile uses `uv pip install ./src/...` which **does not read** `[tool.uv] override-dependencies` from the target packages' `pyproject.toml` — those overrides only apply during `uv lock`/`uv sync`. On Linux, `cuga>=0.2.20` requires `aiohttp>=3.13.5`, but `litellm==1.83.11` exact-pins `aiohttp==3.13.3` (same upstream bug as the click pin, [BerriAI/litellm#26154](https://github.com/BerriAI/litellm/issues/26154)). Without an active override, resolution fails. **Fix:** Write a `/tmp/uv-overrides.txt` file mirroring the workspace's `override-dependencies` (litellm, python-dotenv, openai, aiohttp, click) and pass `--overrides /tmp/uv-overrides.txt` to `uv pip install`. Verified locally by reproducing the exact failure in an isolated tmp directory (to escape the workspace's pyproject.toml auto-apply), then confirming the overrides file resolves it: - aiohttp==3.13.5 ✓ - click==8.3.3 ✓ - litellm==1.83.11 ✓ - cuga==0.2.22 ✓ * Templates version update * Update .secrets.baseline * chore: update overrides * chore: bump version * chore: bump main version * chore: bump SDK version to 0.1.1 * chore: run uv lock and uv sync after SDK version bump * chore: read the overrides from a single source security patch * chore: add pip check toggle add pip check toggle * chore: litellm base uv.lock update * fix(chore): Pin litellm back to last working release * fix(chore): Pin to 1.83 and higher litellm * Update build_and_push_backend.Dockerfile * chore: update base uv.lock * fix: lazyload toolguard since its an optional extra * Update .secrets.baseline * Update component_index.json * Rebuild component index * Update component_index.json --------- Co-authored-by: Eric Hare <ericrhare@gmail.com> Co-authored-by: vijay kumar katuri <vijay.katuri@ibm.com> (cherry picked from commit 60a8f76c3fde17124057626c671ea8162872837a) |
|||
| dbc87e7704 |
fix: db guards for deployments (#12339)
* checkout api handlers
* add missing table
* update to use "version" terminology for flows instead of outdated "history" verbage
* Fix provider account id mapping
* recover a little todo comment
* Add flow version migration and minor exception handling, etc
1. deployments.py — Added AuthenticationError import + handling (401) in all 9 error handler blocks. Added update_deployment_db import + call to persist name changes to local DB after adapter update succeeds.
2. flow_version/exceptions.py — Added FlowVersionDeployedError for blocking deletion of deployed versions.
3. flow_version/crud.py —
- Added has_deployment_attachments() helper that checks if a flow version has any deployment attachments
- delete_flow_version_entry() now raises FlowVersionDeployedError if the version is attached to deployments
- Pruning now excludes deployed versions via a NOT IN subquery on FlowVersionDeplottachment, preventing provider-side snapshot orphaning
4. flow_version/__init__.py — Exports FlowVersionDeployedError
5. flow_version.py (API route) — Imported FlowVersionDeployedError, added it to _translate_version_error as 409 Conflict
6. models/__init__.py — Registered FlowVersionDeploymentAttachment for alembic/SQLModel metadata detection
7. Migration c0d2ce43b315 — Creates flow_version_deployment_attachment table with all columns, FKs (CASCADE), and indexes. Single head, chains off fc7f696a57bf.
* [autofix.ci] apply automated fixes
* address bugs and inconsistencies
* rename column to "provider_snapshot_id"
* harden deployment API: fail loudly on invalid state instead of silently passing
Replace silent fallbacks, warning logs, and dropped values with hard
failures (422/500/502) across the deployment orchestration layer. Adds
logging to bare exception handlers and wraps materialize_snapshots in
the adapter error handler.
* [autofix.ci] apply automated fixes
* add materialize_snapshots to deployment service protocol
Promotes materialize_snapshots from duck-typed getattr usage to a
first-class method on DeploymentServiceProtocol, BaseDeploymentService,
and the no-op DeploymentService stub. Introduces MaterializeSnapshotsResult
schema for typed return values.
Updates deployments.py to call the method through the protocol instead of
getattr, giving static analysis coverage over the contract.
Documents the snapshot abstraction across BaseFlowArtifact, SnapshotItem,
MaterializeSnapshotsResult, and FlowVersionDeploymentAttachment.provider_snapshot_id
— explaining that snapshots are immutable, provider-owned copies of flow data
with opaque provider-assigned identifiers (e.g. wxO tool ID, K8s ConfigMap
name, S3 key).
* deployment sync: extract helpers, server-side type filter, orphan detection
- Extract _fetch_provider_resource_keys helper for provider validation
- Rename _sync_page_with_provider → _list_deployments_synced
- Match resource keys by provider ID only (not name)
- Restore server-side deployment_type filter with guard against
false deletions (skip rows whose type doesn't match the filter
instead of deleting them)
- Add orphan/divergence logging for post-create, post-update,
post-duplicate DB write failures
- Return 422 on invalid UUID in update remove list (was silently ignored)
- Handle NotImplementedError → 501 from provider adapters
- Convert attachment IntegrityError to ValueError with descriptive message
- Add tests for sync helpers (15 cases)
* rebase on release-1.9.0 and align with lfx/services
* refactor(deployments): align provider mapper routing and WXO update payload mapping
Align deployment mapper resolution with adapter-type/provider-key routing and
refactor PATCH update handling to use mapper resolve/shape contracts end-to-end.
Map Langflow flow_version_id references at the API boundary into provider update
operations for Watsonx bind/unbind/remove paths, with expanded mapper tests.
* patch down-revision
* first pass with formalized boundary rules
* fix(deployments): harden watsonx payload boundary contracts
Enforce fail-fast payload slot parsing for required adapter results, split execution create/status slot contracts, and route execution-create mapping through deployment mappers.
Require watsonx flow artifact source_ref and move update reconciliation output to provider_result to keep mapper/adapter boundaries explicit and typed.
* refactor(deployments): modularize watsonx orchestrate create/update flow
Extract create/update logic into dedicated core modules with shared helpers to tighten deployment boundary contracts.
Align backend/lfx payload schema mapping and expand e2e/unit coverage for response mapping and update schema behavior.
* api impl for wxo-specific create payload
* further refactoring. add rollback of existing tools (undo new app bindings) in the create path
* add todo in execution.py
* refactor(deployments): replace snapshot_id/reference_id with source_ref-correlated tool refs
Introduce WatsonxToolRefBinding to correlate source_ref (flow version id)
with provider tool_id across all operation types. This replaces the prior
reference_id and snapshot_id fields with a unified structure that carries
provenance through create, bind, unbind, and remove_tool operations.
Key changes:
- Flatten API operation payloads: hoist flow_version_id onto operations,
remove nested WatsonxApiUpdateToolReference wrapper
- Replace tools.existing_ids with inline tool_id_with_ref on bind operations
- Rename WatsonxCreateSnapshotBinding to WatsonxToolRefBinding (input) and
WatsonxResultToolRefBinding (output, with created flag)
- Add created_app_ids to update results for connection tracking
- Raise HTTPException on contract violations in _to_api_tool_app_bindings
instead of silently dropping unmappable bindings
- Add schema-level validation for conflicting source_ref on same tool_id
- E2E: cache tool_id→source_ref from create results, use helpers to build
refs with distinct source_ref vs tool_id values
* Enforce DeploymentType enum and add description column
Make deployment_type a required column backed by a SQLAlchemy
TypeDecorator that validates on write (rejects None and invalid strings)
and coerces to DeploymentType on read. Add nullable description column
to the deployment model and surface it through the API.
Key changes:
- Add _DeploymentTypeColumn TypeDecorator for enum round-trip fidelity
- Make deployment_type non-optional in Deployment model, DeploymentRead,
CRUD functions, and API layer
- Add description (Text, nullable) to Deployment model and fold its
migration into the existing c0d2ce43b315 revision
- Remove _resolve_deployment_type helper — TypeDecorator handles coercion
- Remove DeploymentType fallbacks and backward-compat shims from API
endpoints, base mapper, and watsonx orchestrate mapper
- Document cross-package coupling: DeploymentType is owned by lfx but
persisted by langflow; member values must never be removed
- Fix pre-existing bug: provider_account_id → deployment_provider_account_id
in get_deployment_status endpoint
Note: deployment_type is nullable=True at the DB level to satisfy the
EXPAND-phase migration validator; NOT NULL is enforced at the application
layer by the _DeploymentTypeColumn TypeDecorator.
* refactor(deployments): extract route helpers, harden sync and error handling
Move bulk of deployment route logic into mappers/helpers layer to slim
down deployments.py and enforce clearer boundary between routes, mappers,
and adapters (documented in DEPLOYMENT_BOUNDARY_RULES.md).
Key changes:
- Extract ~700 lines from deployments.py into helpers.py (pagination,
adapter/mapper resolution, attachment management, snapshot sync,
rollback, response shaping)
- Add read-path snapshot-level sync: get_deployment and
list_deployments_synced verify provider_snapshot_ids against the
provider and prune stale attachments, with graceful fallback on error
- Add compensating rollback for create (rollback_provider_create) and
update (rollback_provider_update) when DB commit fails after provider
mutation, using mapper-driven payload reconstruction
- Introduce handle_adapter_errors() context manager centralising
DeploymentServiceError → HTTP status mapping via
http_status_for_deployment_error; sanitise 500 detail to avoid
leaking internals
- Add DeploymentNotConfiguredError → 503 mapping
- Add util_snapshot_ids_to_verify and resolve_rollback_update to base
mapper with WxO overrides for provider-specific snapshot ID extraction
and put_tools-based rollback payloads
- Add put_tools field to WatsonxDeploymentUpdatePayload for full tool
list replacement; early-return in build_provider_update_plan and
validate_operation_references when put_tools is set
- Extract verify_tools_by_ids into core/tools.py helper
- Harden resource_name_prefix with strip_whitespace + min_length=1
- Deduplicate snapshot_ids before provider calls
- Add deterministic order_by(created_at) to attachment CRUD queries
- Add exc_info=True to all best-effort rollback/compensate error logs
- Add session.rollback() in get_deployment snapshot sync error path
- Warn when list_snapshots receives both deployment_ids and snapshot_ids
- Add E2E scenarios for empty snapshot list, mixed snapshot IDs, tools
endpoint, and deployment re-list after update
Tests:
- Add test_deployment_route_handlers.py covering stale-row delete +
commit, non-404 adapter errors, handle_adapter_errors wiring,
snapshot sync (happy path, skip, error fallback), project-scoped
flow version validation for create and update
- Expand test_deployment_sync.py with snapshot-phase tests, rollback
tests, pagination guard, and project-scoped validation
- Add deployment_type assertion to response mapping test
- Add DeploymentNotConfiguredError and bare DeploymentServiceError cases
to exception mapping tests
- Add put_tools schema and update plan tests
* remove pompous performance commentary
* fix(deployments): remove upsert behavior and fail fast on duplicate name
The create_deployment endpoint previously performed a
get_deployment_by_resource_key lookup before inserting the DB row,
silently tolerating duplicates. Since the provider adapter returns a
fresh resource ID on every create, this lookup could never legitimately
match — and if it did, it would mask a data inconsistency bug.
Changes:
- Remove the get-or-create (upsert) pattern; go straight to
create_deployment_db and let the unique constraint surface conflicts.
- Add deployment_name_exists CRUD function and an early 409 guard so
duplicate names are rejected before any provider call, avoiding
costly provider-side rollback for a locally-checkable condition.
- Update existing route-handler tests to reflect the removed lookup.
- Add tests for deployment_name_exists and the 409 duplicate-name path.
* feat: convert provider_key and deployment_type columns to DB-level enums
Replace plain string columns with SQLAlchemy Enum types backed by
Postgres/SQLite enum constraints, enforcing valid values at the DB
layer rather than only in application code.
Migration follows expand-contract pattern (add enum column, backfill,
drop old string column, rename) with index ops outside batch context
to avoid SQLite column-lookup issues. Upgrade and downgrade are fully
atomic.
- Add DeploymentProviderKey enum as single source of truth for
provider identifiers; remove magic strings and _DeploymentTypeColumn
TypeDecorator
- Make provider_key immutable after creation (remove from update
request schema and API handler)
- Fix pre-existing test gap: add missing deployment_type argument to
all TestDeploymentCRUD create_deployment() calls
* feat(flow-versions): add deployment awareness and sync-on-read
Add is_deployed field to flow version responses and provider-verified
sync to the list/get read paths, matching the existing deployment
endpoint sync pattern.
API changes:
- list_flow_versions returns is_deployed per entry and accepts optional
deployment_ids filter to scope by specific deployments
- get_single_flow_version returns is_deployed on the full response
- Both endpoints now use write sessions and run best-effort
snapshot-level sync before returning results
Sync-on-read (helpers.sync_flow_version_attachments):
- Queries all attachments for a flow's versions joined with deployment
and provider account info in a single query
- Groups by provider, resolves adapter/mapper per group, verifies
provider_snapshot_ids via list_snapshots, and prunes stale attachment
rows through the existing sync_attachment_snapshot_ids helper
- Per-provider error handling so one provider outage doesn't block the
response
New CRUD: list_attachments_for_flow_with_provider_info joins
FlowVersionDeploymentAttachment -> Deployment -> DeploymentProviderAccount
to avoid N+1 queries during sync grouping.
Tests: 50 passing (11 new) covering is_deployed indicator, deployment_ids
filtering, stale attachment pruning on list/get, and sync failure
resilience.
* Add user id authentication to a few missing endpoints
* [autofix.ci] apply automated fixes
* Update tests
Removes some mock tests that did nothing
Adds sqlite for true testing of db accessors
Reduces Mock objects usage
* refactor(deployments): enforce ownership boundaries on execution responses
Move provider-owned execution identifiers (execution_id, agent_id,
status, timestamps, errors) out of the top-level API response and into
provider_data, keeping only Langflow-owned fields (deployment_id) at the
top level. This prevents future collisions if Langflow introduces its
own execution tracking.
Key changes:
- Remove execution_id from _ExecutionResponseBase; provider's opaque
run identifier now lives exclusively inside provider_data
- Rename WatsonxExecutionResultData → WatsonxAgentExecutionResultData
(adapter layer) and split the API-layer class into a private base
(_WatsonxApiAgentExecutionResultBase) with dedicated
WatsonxApiAgentExecutionCreateResultData and
WatsonxApiAgentExecutionStatusResultData subclasses
- Translate WXO run_id → execution_id at the adapter boundary
(create_agent_run_result / get_agent_run).
- Collapse util_execution_id + util_execution_deployment_resource_key
into a single util_resource_key_from_execution that trusts the
adapter-provided result.deployment_id directly
- Remove build_orchestrate_runs_query and extra payload fields
(thread_id, llm_params, guardrails, etc.) unused in MVP
- Simplify WxOClient.post_run signature (drop query_suffix)
- Exclude provider_data from flow tool artifact to avoid unexpected
top-level keys in the WxO tool runtime
- Document ownership boundary rules in DEPLOYMENT_BOUNDARY_RULES.md §14
- Add E2E polling for terminal execution status, input format variants,
and missing-deployment negative test
- Expand unit tests for renamed schemas, field mapping, passthrough
validation, and simplified payload builder
* feat: add name column to deployment_provider_account
Add a required, user-chosen display name to provider accounts
(e.g. "staging", "prod") that is unique within a given provider_key.
Includes model, CRUD, API schema/route, mapper, migration with
backfill, and tests.
* fix: replace hand-written migration with Alembic-generated revision
Replace the manually authored migration (b4e6f8a2c1d3) with an
Alembic-generated one (8255e9fc18d9) for the deployment_provider_account
name column.
Fix SQLite compatibility: sa.func.concat() generates a concat() function
call which does not exist in SQLite. Use sa.literal().concat() instead,
which produces the || operator and works on both PostgreSQL and SQLite.
* remove bogus unverified math from migration file
* feat: verify provider credentials before account creation
Add a verify_credentials step to the provider account creation flow
that validates API keys against the provider before persisting them.
This prevents storing invalid or revoked credentials and gives users
immediate feedback.
Key changes:
- Add verify_credentials to the deployment adapter interface (base,
service, protocol) with WXO implementation that obtains a token
via the IBM authenticator
- Add SSRF-hardened URL validation for provider_url (HTTPS-only,
private IP blocklist, localhost rejection, normalization)
- Introduce ValidatedUrl/ValidatedUrlOptional annotated types in
the API schema layer
- Refactor raise_for_status_and_detail to accept an optional cause
parameter for explicit exception chain control
- Use ResourceNotFoundError (parent) instead of DeploymentNotFoundError
in raise_for_status_and_detail for provider-agnostic 404 mapping
- Narrow get_authenticator return type to concrete union
* use whitelist only for valid urls
* feat: BREAKING: move credentials into provider_data and centralize update logic in mapper
- Replace top-level `api_key: SecretStr` with opaque `provider_data: dict` in API schemas;
mapper extracts credentials via `resolve_credential_fields` for DB storage
- Add `resolve_provider_account_update` to base mapper so routes delegate
full update-kwargs assembly (including cross-field logic) to the mapper
- WXO mapper override re-derives `provider_tenant_id` when `provider_url` changes
- Add tenant extraction utilities and `validate_tenant_url_consistency` in
`deployment_provider_account/utils.py` as single source of truth
- Add `model_validator` on `DeploymentProviderAccount` for defense-in-depth
tenant/URL consistency checks
- Rename `DEPLOYMENT_BOUNDARY_RULES.md` → `RULES.md`; document DB-direction
mapper contract, credential flow, and update assembly
- Update all tests for new `provider_data` shape, mapper update methods,
tenant extraction, and model-level consistency validation
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix(deployments): Harden provider account validation and WXO rollback
Use provider_url when resolving WXO credentials and scope provider account names per user within each provider. Re-verify provider account updates before persisting them and return 4xx responses for account conflicts instead of surfacing 500s.
Block deleting provider accounts that still own deployments and extend create rollback so failed WXO writes clean up provider-side resources. Add route, schema, mapper, sync, CRUD, and WXO tests to cover the new behavior.
* [autofix.ci] apply automated fixes
* update url validator docs; remove outdated reference to private url blacklist logic
* fix(deployments): Isolate snapshot sync writes
Use nested transactions around best-effort snapshot cleanup so provider sync failures cannot leak partial attachment deletes into the outer request transaction.
Persist explicit description clears during deployment PATCH requests, and add regression tests for both deployment sync paths and the route handler update flow.
* fix(deployments): Harden delete cleanup and wxO create tests
Treat missing provider agents as stale local cleanup so delete can
finish instead of leaving orphaned deployment rows behind.
Commit local delete operations eagerly, retry once on commit failure,
and move wxO create-path tests toward fake client objects so the real
service logic is exercised without external calls.
* new head
* fix(deployments): stop prefixing wxo raw connection app_ids
Preserve caller app_ids for newly created wxo connections while keeping lf_ prefixing for tool/deployment naming, centralize resource_name_prefix validation, and update mapper/service schema tests and docs to reflect the new behavior.
* fix(deployments): Harden provider account cleanup
Reconcile stale deployment rows before provider-account deletion so
out-of-band provider removals do not leave account cleanup blocked.
* fix: restore py310 compatibility and align payload slot tests
Import Self from typing_extensions in the deployment provider account model to keep backend imports working on Python 3.10, and update payload formalization tests to assert strict rejection of cross-model BaseModel inputs.
* fix(deployments): resolve mypy typing regressions across deployment flows
Normalize SQLModel query expressions for typed SQL operators, make deployment-related model IDs non-nullable where appropriate, and tighten provider mapper/update typing guards. Update deployment tests to align with stricter type contracts.
* fix(deployments): restore py310 test compatibility and migration idempotency
Replace Python 3.11-only UTC imports with timezone.utc in deployment-related tests and the watsonx e2e helper, and make the provider-account name migration safe to re-run when the column and unique constraint are created in separate steps.
* fix(deployments): restore provider account route ownership helpers
Reuse the shared provider-account lookup in update/delete routes so route tests keep the expected ownership path, and use the explicit-field helper when deciding whether to reverify credentials.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* fix(deployments): enforce deployment guard constraints with clean API errors
Add DB trigger guards to block deleting deployed flow versions, deleting projects with deployments, and moving deployed flows across projects. Translate trigger violations to HTTP 409 with sanitized details, add best-effort pre-operation deployment/snapshot sync to reduce stale-state false positives, and cover guard parsing/delete flush behavior with tests.
* checkout latest crud and helpers files
* make helper for try catch sync logic
* add gaurd to prevent moving deployments across projects
* prevent moving deployments across provider accounts
* add live sqlite tests
* recover lost changes
* recover feature flag setup from release-1.9.0
* fix: address review findings for deployment guard branch
- Add comment in delete_user explaining why best-effort provider sync
is intentionally skipped (too expensive; DB trigger is authoritative)
- Add docstring to _clean_guard_detail for maintainability
- Add happy-path trigger tests verifying operations succeed when no
deployment conflicts exist (flow version delete, project delete,
flow move, deployment rename)
- Fix list_attachments_for_flow_with_provider_info callers still using
the old flow_id= parameter (now flow_ids=)
- Fix pre-existing broken test fixture (missing name in
DeploymentProviderAccount)
* feat: add cross-project attachment guard and harden test suite
Add a new DB trigger (prevent_cross_project_attachment) that blocks
attaching a flow version to a deployment in a different project, for
both PostgreSQL and SQLite.
Refactor trigger tests into a unified dual-dialect test file
(test_deployment_guard_triggers.py) that runs against both SQLite
(always) and PostgreSQL (when TEST_DEPLOYMENT_GUARD_PG_URL is set).
Validated with both asyncpg and psycopg drivers.
Additional improvements:
- test_downgrade_removes_triggers now asserts the guard is active
before downgrade, preventing false-positive passes
- Endpoint test uses statement inspection instead of fragile
call-count gating
- Add exception parser coverage for CROSS_PROJECT_ATTACHMENT,
implicit __context__ chains, and cyclic exception chains
- Add endpoint tests for cascade_delete_flow, update_flow,
delete_multiple_flows, and global exception handler
* fix(deployments): make provider account identity immutable
Add DB deployment guard triggers to prevent updating provider account identity fields
(provider_key, provider_tenant_id, provider_url) once created, and preserve guard
error translation in the provider-account PATCH route. Extend trigger and route
tests to cover blocked updates and same-value no-op updates.
* refactor: extract sync logic into dedicated module and fix sync correctness issues
- Move deployment/attachment sync functions from helpers.py to new sync.py
module, reducing helpers.py size and improving separation of concerns
- Fix fetch_provider_resource_keys to use deployment_ids instead of
provider_params (pre-existing bug: wrong adapter parameter)
- Add empty-list guard to fetch_provider_resource_keys to avoid
unnecessary provider calls
- Wrap snapshot sync in savepoint for proper fault isolation
- Update imports across flows.py, projects.py, flow_version.py,
deployments.py to reference new sync module
- Update unit tests to target refactored module paths and add coverage
for bug fixes
* refactor: implement binding-aware deployment sync and optimistic guard retries
Add provider binding contract and mapper extraction path:
- add ProviderSnapshotBinding in deployments/contracts.py
- add BaseDeploymentMapper.extract_snapshot_bindings() default no-op
- implement Watsonx mapper extract_snapshot_bindings() with strict provider_data parsing via PayloadSlot (snapshot_ids required)
Replace attachment snapshot verification loop with binding-aware set-based cleanup:
- add delete_unbound_attachments(user_id, deployment_ids, bindings) in flow_version_deployment_attachment/crud.py
- uses CTE/anti-join to delete local attachments not present in provider bindings
- when bindings are empty, delete all attachments for provided deployment_ids
- add count_attachments_by_deployment_ids() grouped recount helper
- update list_deployments_synced() in deployments/helpers.py to:
- collect bindings from fetch_provider_resource_keys() provider_view during batch sync
- run one savepoint-wrapped delete_unbound_attachments() for accepted deployments
- run one grouped recount and patch attached_count from DB
- remove list_attachments_by_deployment_ids + snapshot-provider phase
- enforce two sync rounds max (initial pass + one refill)
Consolidate provider-side sync plumbing:
- change fetch_provider_resource_keys() in deployments/sync.py to return (known_resource_keys, provider_view)
- document tuple contents in function docstring
- remove now-unused sync_provider_attachment_snapshots() helper
- keep fetch_provider_snapshot_keys()/sync_attachment_snapshot_ids() for single-deployment read path
Scope read sync by provider account and conditionally expose deployment status:
- update deployment CRUD listing helpers to accept provider_account_id filter
- in flow_version API, add deployment_provider_account_id query param
- only perform deployment sync/is_deployed calculation when provider id is explicitly passed
- make FlowVersion.is_deployed optional (default None) so field can be omitted when not requested
Switch mutating APIs to optimistic catch-sync-retry pattern:
- add _retry_on_deployment_guard() in flows.py and projects.py
- replace pre-sync mutation flow with guarded retry on DeploymentGuardError
- for project create/update, retry only guarded flow-move DB operations
- preserve side-effect safety and improve stale-state recovery behavior
Tests:
- add unit tests for deployment guard retry helper (test_deployment_guard_retry.py)
- add unit tests for delete_unbound_attachments behavior, including empty bindings delete-all case (test_flow_version_deployment_attachment_crud.py)
- update deployment sync tests for tuple return, binding extraction, consolidated phase-2 cleanup/recount, and two-round refill behavior
- update flow_version tests for provider-scoped sync and conditional is_deployed field presence
* fix: extract tool_ids from provider data
* dont use stale rolled-back OORM Flow instance
* refactor(deployment): demote delete triggers and standardize guard handling
- Edit the existing deployment guard migration to remove DB delete triggers for flow version and project deletion in both PostgreSQL and SQLite.
- Keep core DB invariants (flow move, deployment project move, deployment provider account move, provider account identity immutability, and cross-project attachment).
- Add a new DB invariant trigger to enforce deployment.resource_key immutability.
- Rewrite trigger error text to technical/operator-focused language and remove ambiguous “detach” wording.
- Add migration comments documenting the current global provider identity guard and the future provider-specific migration path.
- Add app-level guard helpers in deployment/guards.py:
- check_flow_has_deployed_versions
- check_project_has_deployments
- Wire check_flow_has_deployed_versions into cascade_delete_flow before destructive deletes.
- Wire check_project_has_deployments into delete_project before deleting the folder row.
- Keep explicit DeploymentGuardError passthrough in cascade_delete_flow and retain parse fallback for non-guard exceptions.
- Refactor DeploymentGuardError to carry:
- code
- technical_detail
- detail (API-friendly message)
- Add friendly guard-detail mapping by guard code.
- Update parse_deployment_guard_error to parse DEPLOYMENT_GUARD:<CODE>:<DETAIL> and preserve raw technical detail from chained exceptions.
- Simplify delete_user to DB-cascade delete + flush only.
- Remove deployment guard translation/parsing logic from delete_user.
- Keep an explicit endpoint comment documenting the intentional no-provider-teardown trade-off.
- Update flow_version deletion message to: “Remove its deployment attachment rows first.”
- Update tests to match the new guard contract and behavior:
- Remove stale delete_user guard-translation endpoint test.
- Update delete endpoint tests to assert app-level guard behavior and new friendly details.
- Update trigger tests to reflect removed delete triggers and added resource_key trigger coverage.
- Update trigger downgrade assertions for the revised trigger set.
- Expand exception tests for code mapping, technical-detail preservation, unknown-code fallback, and new guard codes.
- Add unit tests for deployment guard helper functions.
* surface better msg when project deletion attempted
* refactor(deployment): rename flow guard code and align guard messages
Rename FLOW_VERSION_DEPLOYED to FLOW_HAS_DEPLOYED_VERSIONS across deployment guard checks, project-level remapping, and tests.
Update friendly guard wording for flow/project/move/cross-project cases, and convert pytest match patterns to raw escaped regex strings to satisfy Ruff.
* add docs to migration file
* update docs in migration
* update docs
* refactor(flow-version): scope deployment status to list endpoint
Keep deployment status provider-scoped on version listing only, remove single-version sync/status behavior, and omit is_deployed from snapshot creation responses while aligning docs and tests.
* refactor(deployment): centralize guard retry helpers and align sync error handling
Move flow/project deployment-guard retry wrappers into deployment sync helpers while preserving upstream route logic, and align provider sync error mapping with deployment domain exceptions. Tighten wxo snapshot binding validation and update docs/tests for guard messaging and sync behavior.
* refactor(deployment): enforce snapshot ID invariants and improve delete error UX
- add shared non-empty string guard (`require_non_empty`) in database utils and use it in deployment attachment CRUD write paths
- enforce required `provider_snapshot_id` on attachment model and add Pydantic + SQLAlchemy validators as safety nets for non-CRUD writes
- introduce deployment attachment key schemas (`DeploymentAttachmentKey`, `DeploymentAttachmentKeyBatch`) with deduplication support
- add batch stale-row cleanup (`delete_deployment_attachments_by_keys`) using a VALUES CTE to delete exact deployment/flow-version pairs
- remove mapper-level `util_snapshot_ids_to_verify` hook from base + watsonx mapper and replace with sync utilities:
- `extract_verified_snapshot_ids`
- `extract_verified_provider_snapshot_ids`
- update sync and helper flows to use verified snapshot IDs, stricter provider ID validation, and safer corrected-count handling
- align provider ID handling so both deployment and snapshot sync paths reject blank provider IDs consistently
- tighten adapter/provider-key validation and enforce required snapshot binding during attach flow-version operations
- update deployment routes/helpers to new sync signatures and behavior
- remove legacy null/blank snapshot safety-net filters from attachment queries/delete-unbound reconciliation (intentional invariant enforcement)
- add broad backend test coverage for:
- rejecting empty/blank snapshot IDs on create/update
- exact-key batch deletion behavior
- cartesian cross-delete regression prevention
- updated route/sync expectations after helper refactor
- improve flow delete UX in the frontend list view by surfacing backend `detail` via `getAxiosErrorMessage` instead of a generic "Please try again"
* update logic and remove stale comment
* fix(deployment): handle orphaned deployment attachments and enforce SQLite FK constraints
Enable PRAGMA foreign_keys=ON for SQLite to enforce referential integrity
that was previously silently ignored, and add application-level cleanup for
orphaned FlowVersionDeploymentAttachment rows left behind by environments
where cascades never fired.
- Enable foreign_keys pragma in lfx SQLite settings
- Update deployment guards to detect and prune orphan attachments (missing
deployment parent) before blocking on live ones, with structured logging
- Update flow_version and deployment CRUD queries to join on Deployment so
is_deployed status and attachment counts reflect only live attachments
- Document has_deployment_attachments write side-effect (orphan pruning)
- Wrap pre-sync orphan cleanup in try/except so failures don't abort the
sync process
- Explicitly delete spans and traces in cascade_delete_flow to avoid FK
violations from the span.trace_id constraint (which lacks ON DELETE
CASCADE in the DDL)
- Add in-memory SQLite integration tests covering guard pruning, orphan
cleanup, version pruning, deployment counts, and is_deployed status
- Add unit tests for sync exception-handling paths
* refactor(deployment): replace DB trigger guards with ORM-layer preflight checks
Remove the unshipped trigger migration (97c9a98c9c01) and enforce
deployment invariants at the ORM/service layer instead:
- Add orm_guards.py with preflight checks for all six guard codes
(FLOW_DEPLOYED_IN_PROJECT, DEPLOYMENT_PROJECT_MOVE, DEPLOYMENT_TYPE_UPDATE,
DEPLOYMENT_RESOURCE_KEY_UPDATE, DEPLOYMENT_PROVIDER_ACCOUNT_MOVE,
DEPLOYMENT_PROVIDER_ACCOUNT_IDENTITY_UPDATE, CROSS_PROJECT_ATTACHMENT)
- Wire guards as pure prefix blocks into deployment, provider-account,
and attachment CRUD functions — existing write logic unchanged
- Add preflight validation before bulk update(Flow) statements in
projects.py and folder/utils.py — existing statements unchanged
- Add ensure_flow_move_allowed to flows_helpers.py for ORM-instance
update paths (_patch_flow, _update_existing_flow, _validate_and_assign_folder)
- Optimize batch flow-move guard to one query per source folder
- Simplify parse_deployment_guard_error to isinstance-only chain walk
now that guards are raised explicitly as DeploymentGuardError
- Document retry helper contract: operations must enforce guards internally
- Replace trigger-centric tests with ORM/service-level guard tests
- Add project API guard integration tests for flow-move blocking
* fix(deployment): prune attachment rows before deployment deletes
Explicitly delete flow-version deployment attachments before deployment row deletes
to prevent orphaned attachments when FK cascades are disabled (e.g. SQLite
foreign_keys=OFF). Keep delete-by-resource-key and delete-by-id behavior intact
while documenting guard None-scope behavior and adding coverage for missing-row
and FK-disabled cleanup paths.
* recover lost tests
* misc
* remove deployment guard error extraction from lfx session scope
* extract re-raise logic into a helper
* fix: stabilize deployment guard retries and guard diagnostics
- scope WxO provider-client ContextVar state to each deployment provider scope via provider_clients_memoization_scope
- compose WxO scope lifecycle into deployment_provider_scope and document forward multi-adapter dispatch options
- add successful provider resource-key sync debug logging and remove stray print-based sync error debug
- log DeploymentGuardError details during flow cascade delete and in deployment guard exception re-raise helper
- fix folder defaulting guard precheck query to use Flow.folder_id.is_(None) so NULL-folder flows are included
- deduplicate require_non_empty by importing from langflow.services.database.utils in deployment mappers
- remove deprecated duplicate helper module at api/v1/mappers/deployments/util.py and keep package export wired to canonical helper
- add regression test for default-folder guard precheck covering NULL-folder flow handling
* add tests for provider scope context
* fix: improve deployment guard logging and async guard handling
Log DeploymentGuardError directly in flow/project endpoint guard catch blocks with operation context, and use the async guard helper for generic exception paths that need guard parsing. Add tests for async guard helper logging and raise behavior.
* less verbose name for scope
* harden delete_unbound_attachments by passing provider account id to it to scope deletion
* fix: return dense deployment attachment counts and simplify snapshot sync
Make `count_attachments_by_deployment_ids` return a dense mapping for all
requested deployment ids, including zero for ids with no countable attachments.
Update deployment listing code to rely on direct key access for corrected counts,
remove redundant `verified_snapshot_ids` plumbing from snapshot sync call sites,
keep local snapshot id integrity checks in sync logic, and align backend tests
plus log messaging with the new count contract.
* guard context scope
* test: expand deployment guard and sync regression coverage
Add comprehensive regression coverage for deployment guard and deployment-sync behavior across database and API test suites.
- verify deployment deletes remove linked attachment rows
- cover batched move guards and immutable deployment/provider fields
- validate cascade flow deletion and user-scoped orphan cleanup behavior
- exercise retry helper edge cases for flow/project guard retries
- align sync helper tests with stricter provider-id/error handling and dedupe/early-return paths
- add API 409 guard-path tests for deployed flow/project delete and move scenarios
* fix: align flow-version list semantics with provider-scoped deployment status
- remove `deployment_ids` handling from `GET /flows/{flow_id}/versions/` and keep plain list mode as versions-only
- enforce provider-account ownership for `deployment_provider_id` via `get_owned_provider_account_or_404` (404 on unknown/foreign ids)
- add explicit feature-flag guard for provider-scoped mode:
`Cannot use deployment_provider_id: the wxo_deployments feature flag is disabled`
- remove `has_providers` / `count_provider_accounts` branching and drive behavior directly from request params
- replace `get_flow_version_list` with `get_flow_versions_with_provider_status` and scope `is_deployed` by:
`Deployment.deployment_provider_account_id == provider_account_id`
- keep best-effort `sync_flow_version_attachments` before provider-scoped reads
- update API tests:
- delete deployment_ids endpoint tests
- add provider-scoped status regression coverage (true under provider A, false under provider B)
- add provider-id feature-flag rejection test (400)
- add unknown and foreign provider-id ownership tests (404)
- simplify feature-disabled plain-list test to assert `is_deployed` remains omitted
- migrate CRUD/in-memory tests off removed `get_flow_version_list` to `get_flow_versions_with_provider_status`
- update stale flow export TODO comments referencing the removed function
* fix: batch stale deployment cleanup per provider sync group
Replace per-deployment stale deletes in deployment sync with a single batched delete per provider group.
Add a bulk deployment-delete CRUD helper and regression coverage for grouped stale-delete behavior plus multi-deployment attachment cleanup.
* test: expand deployment guard and provider account update coverage
Add direct ORM guard coverage for noop immutable updates and attachment project matching. Update in-memory deployment tests to cover provider account API key rotation and align stale fixtures with current model constraints.
* fix: make deployment GET sync binding-aware and harden guard/prune behavior
- Align SQLite defaults in `lfx/services/settings/base.py`:
- remove forced `foreign_keys=ON` pragma from default sqlite pragmas
- Replace deployment GET snapshot-id verification with binding-aware reconciliation in `api/v1/deployments.py`:
- resolve and use the concrete deployment mapper in GET
- call mapper-provided `extract_snapshot_bindings_for_get(...)`
- prune detached links via `delete_unbound_attachments(...)` inside `session.begin_nested()`
- fall back safely to unverified attachment counts when sync is unsupported or fails
- change provider_data response shaping to `mapper.shape_deployment_get_data(...)` to hide internal fields
- Extend mapper contracts in `api/v1/mappers/deployments/base.py`:
- add `extract_snapshot_bindings_for_get(...)` (explicitly raises `NotImplementedError` by default)
- add `shape_deployment_get_data(...)` (explicitly raises `NotImplementedError` by default)
- make GET sync/shaping behavior explicit per provider instead of accidental passthrough
- Implement wxO-specific GET sync/shaping in `watsonx_orchestrate/mapper.py`:
- parse/validate `provider_data.tool_ids` for binding extraction
- emit `ProviderSnapshotBinding(resource_key, snapshot_id)` for each tool id
- validate/shape GET `provider_data` to expose only `{"llm": ...}` to API clients
- raise clear 500/value errors for malformed adapter payload contracts
- Update wxO adapter GET payload in `watsonx_orchestrate/service.py`:
- always include `tool_ids` and derived `environment`
- include `llm` when available
- remove `or None` fallback so mapper receives consistent structure
- Harden flow-version pruning in `services/database/models/flow_version/crud.py`:
- resolve concrete version IDs to prune first
- delete `FlowVersionDeploymentAttachment` children before deleting `FlowVersion` rows
- prevent stale doubly-orphan attachment rows when SQLite runs with foreign keys disabled
- Simplify guard exception semantics in `deployment/exceptions.py`:
- stop walking chained exceptions; only treat explicit `DeploymentGuardError` instances as guard failures
- add optional remap hook to `araise_if_deployment_guard_error_or_skip(...)`
- add `remap_flow_guard_for_project_delete(...)` to convert flow guard code to project guard code where needed
- Remove duplicated per-route guard catch blocks and centralize behavior:
- `api/utils/flow_utils.py`, `api/v1/flows.py`, and `api/v1/projects.py` now rely on shared async guard helper
- project delete path now remaps `FLOW_HAS_DEPLOYED_VERSIONS` into `PROJECT_HAS_DEPLOYMENTS` via shared remap function
- Keep multi-delete flow route behavior consistent in `api/v1/flows.py`:
- run guard remap/log helper in broad exception path before generic logging and 500 handling
- Update deployment exception tests in `test_exceptions.py`:
- remove chain-parsing expectations
- validate direct guard-raise semantics
- add remap function coverage
- Expand flow version pruning tests:
- `test_crud.py`: verify delete ordering (attachments first, versions second) and no-op behavior when nothing is pruned
- `test_in_memory.py`: add broad regression class proving orphan attachment cleanup, live deployment preservation, cross-flow/user isolation, and repeated-cycle convergence
- Expand deployment API/mapper/sync test coverage:
- `test_deployment_mapper_base.py`: assert new base mapper GET shaper raises until implemented
- `test_deployment_route_handlers.py`: migrate assertions to binding-aware sync behavior, unsupported-provider fallback, sync-failure fallback, and provider_data sanitization
- `test_deployment_sync.py`: add wxO mapper tests for `extract_snapshot_bindings_for_get(...)` and GET data shaping requirements
- `test_watsonx_orchestrate.py`: validate wxO GET provider_data now includes `tool_ids`/`environment` defaults and feature-flag-scoped provider context behavior
* remove redunant description
* remove agent environment from payload for get()
* fix(deployment): prune orphan attachments before reading in flow guard
The cascade flow delete path called `check_flow_has_deployed_versions`,
which issued a SELECT before any write. On SQLite that left the
connection holding only a SHARED lock; the subsequent cascade DELETEs
then had to upgrade SHARED -> RESERVED, which doesn't busy-wait when
another connection already holds RESERVED, surfacing intermittently
in CI as `OperationalError: database is locked`.
Reorder the guard to issue a single DELETE-with-scalar-subquery for
orphan attachments first, so the connection acquires the writer lock
up front. The live-attachment SELECT runs afterwards on the same
session. As a side benefit, orphan pruning now also runs when live
attachments coexist (previously the early raise short-circuited it,
though SAVEPOINT rollback in the retry wrapper still discards the
prune in that failure path).
Log the raw driver-reported `rowcount` verbatim on every prune for
debuggability, and update guard tests for the new DELETE-then-SELECT
ordering.
* add log that pruning might be rolled back
* fix(deployment): set provider scope when reconciling before provider-account delete
The DELETE /api/v1/deployments/providers/{id} handler called
list_deployments_synced without entering deployment_provider_scope, so the
WxO adapter raised CredentialResolutionError ("Deployment account context is
not available for adapter resolution"). Reconciliation was silently dropped
and the stale local count blocked every delete with a 409.
Wrap the reconciliation call in deployment_provider_scope(provider_account.id)
to match every other adapter call site in this module, so stale local
deployment rows can actually be pruned and the provider account can be
removed when the provider no longer owns any resources.
Add three regression tests in TestProviderAccountRoutes:
- assert deployment_provider_scope is active (with the right provider id)
during reconciliation, and that the context does not leak afterwards
- assert that a CredentialResolutionError from reconciliation falls back to
the local count and returns 409 without deleting the provider row
- assert that reconciliation is skipped entirely when no local deployments
exist for the provider
---------
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
(cherry picked from commit
|
|||
| 242c6c70b5 |
fix: MCP Tools component now works in LFX standalone mode (#12760)
* fix: MCP Tools component now works in LFX standalone mode
When serving a flow via `lfx` (the standalone flow server) without the
full Langflow package installed, `MCPToolsComponent.update_tool_list()`
unconditionally tried to import `langflow.api.v2.mcp.get_server` and
open a DB session, raising `ImportError` before the existing fallback
to `server_config_from_value` in `resolve_mcp_config()` could run.
Treat the ImportError as the expected signal of standalone mode: skip
the DB lookup and use the server config embedded in the flow JSON.
The DB path is unchanged when Langflow is available.
* [autofix.ci] apply automated fixes
* fix: narrow LFX standalone fallback to missing Langflow modules only
Previously the fallback to the flow-embedded MCP server config triggered
on any ImportError from the Langflow imports, which would silently swallow
transitive dependency failures (e.g. sqlmodel failing to import inside
langflow.services.database.models.user.crud). That is a real bug in the
full Langflow stack — not LFX standalone mode — and silently using the
stale flow-embedded config would hide it while also bypassing the DB
config that is supposed to take precedence when Langflow is available.
Narrow the fallback to ModuleNotFoundError whose .name is one of the
Langflow modules we are trying to import; re-raise everything else so it
surfaces normally.
* Update component_index.json
* [autofix.ci] apply automated fixes
* fix: address review feedback on LFX standalone fallback
- Expand comment on `except ModuleNotFoundError` narrowing to explicitly
document the ImportError-vs-ModuleNotFoundError distinction.
- Bump fallback log from adebug to ainfo (mode switch is material).
- Simplify `getattr(e, "name", None) or ""` to `e.name or ""`.
- Add TODO(legacy-cleanup) marker flagging 800-line file debt.
- Rename fallback tests to `should_[expected]_when_[condition]` convention.
- Add regression tests: plain ImportError surfaces; lfx.services.deps
missing surfaces (locks in prefix-check precedence).
- Document why `patch("builtins.__import__")` is necessary in tests.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
|
|||
| 3ea28ed902 |
fix: propagate x-api-key and authorization headers to nested MCP calls (#12541)
* fix: propagate x-api-key and authorization headers to nested MCP calls (fixes #12529) When Langflow is used as an MCP server containing flows with nested MCP components, authentication headers like x-api-key were silently dropped because extract_global_variables_from_headers() only captured headers with the X-LANGFLOW-GLOBAL-VAR-* prefix. Add _AUTH_HEADERS_TO_PROPAGATE to also capture x-api-key and authorization under their lowercase header names. These values are stored in the request_variables context, making them available for resolution in nested MCP server configs. Users can now reference them in their server headers config as {x-api-key: x-api-key} to propagate the incoming key. * fix: remove duplicate verify_public_flow_and_get_user from core.py The duplicate function in core.py referenced undefined names (uuid, session_scope) and shadowed the canonical implementation in flow_utils.py, which is the one re-exported from __init__.py and has the fuller signature including authenticated_user_id. Removing the dead copy resolves the F821 ruff errors. * Tighten scope of fix * Update .secrets.baseline * Clean up test locations * Update .secrets.baseline --------- Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| f748738c74 |
fix: MCP Auth Error on restart / swapping auth (#12715)
* fix: MCP Auth Error on restart / swapping auth * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Make this more robust * fix: Propagate the api key in PATCH --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| da516b7045 | Merge release branch | |||
| 38d142a723 |
fix: Upgrade cuga to 0.2.20 to resolve playwright dependency conflict (#12703)
* fix: upgrade playwright to 1.58.0 to address Chromium CVEs - Add playwright>=1.58.0 to override-dependencies in pyproject.toml - Update uv.lock: playwright 1.49.0 -> 1.58.0, pyee 12.0.0 -> 13.0.1 - Fixes CVE-2026-2313, CVE-2026-2314, CVE-2026-2315, CVE-2026-2319, CVE-2026-2321, CVE-2026-2441, CVE-2026-2648, CVE-2026-2649 - Ensures Docker builds download updated Chromium with security patches * fix: update npm to latest version to address brace-expansion CVE-2026-33750 - Add npm update after Node.js installation in Dockerfile - Fixes CVE-2026-33750 in system npm's brace-expansion dependency - System npm had brace-expansion 2.0.2, update gets 5.0.5+ - Low risk change: npm is backward compatible, only affects CLI tool * revert: remove npm update from Dockerfile - npm update attempts were causing CI build failures - Bundled npm has issues but updating it is proving problematic - Focus on playwright CVE fix which is the primary concern - brace-expansion CVE-2026-33750 is lower priority (DoS only) * chore: sync uv.lock files sync uv.lock files * fix(mcp): dedupe edges in connect_components (#12701) * fix(mcp): make add_connection idempotent to avoid duplicate edges connect_components used to append a new edge unconditionally. Because the edge id is deterministic from source/target/handles, calling it for a pair the flow already had wired up (UI-then-MCP, batch retry, or just a repeat call) produced a second edge with the same id, double-wiring the flow at runtime. Before appending, scan the existing edges for one with the same id and return that instead. Different outputs/inputs between the same pair still produce distinct ids and remain supported. * test(mcp): cover dedupe against UI-saved edges, broaden match key Older Langflow UIs saved edges with an `xy-edge__` id prefix instead of the current `reactflow__edge-`, so an id-based dedup would miss the UI-then-MCP case for any flow that came from an older version. Switch the existence check to a structural one (source, target, sourceHandle name, targetHandle fieldName) so the same logical connection dedupes regardless of id format. Add a fixture-driven test that loads MemoryChatbotNoLLM.json (an xy-edge-prefixed flow) and replays each connection through add_connection, asserting the edge count does not grow. * fix(mcp): validate_flow fast-fails and reports partial errors (#12697) * fix(mcp): validate_flow fast-fails and reports partial errors validate_flow polled /monitor/builds for up to 30 seconds waiting for every component to finish before reporting errors. When a component fails early (for example a missing required field), downstream components never run, so the loop waited out the full window and returned just "Build timed out: N/M components completed" with no actionable context. - Short-circuit as soon as any completed build reports valid: false; return those errors immediately instead of polling on. - On timeout, include the errors from the builds that did complete plus a component_count so the caller can see progress. - Extract _collect_build_errors so the poll loop and timeout branch share the same error shape. * fix(mcp): stream validate_flow build inline instead of polling The previous implementation triggered an async build and polled /monitor/builds, which depended on FastAPI BackgroundTasks firing the log_vertex_build calls after the trigger request had returned. Under ASGI test transport these tasks never run, so /monitor/builds stayed empty and validate_flow timed out with component_count=0. Switch to event_delivery=direct so the build streams its events back inside the same request: - Drive the build via client.stream_post and aggregate per-vertex results from end_vertex events. - Fast-fail on the first vertex with valid=false, since downstream vertices depend on it and would not produce useful information. - Surface top-level error events as a single flow-level error. - Replace _collect_build_errors with _extract_vertex_error, which reads the structured error payload from the end_vertex outputs. Update the lfx unit tests to use the streaming shape and tighten the backend integration test to assert real success now that the build actually runs end to end under ASGI. * fix(graph): make end_all_traces_in_context Python 3.10 compatible The implementation called asyncio.create_task(coro, context=context), but the context= keyword was added to create_task in Python 3.11. On 3.10 it raised TypeError. The bug was latent because nothing in the test suite previously consumed a streaming build response far enough for Starlette to dispatch the post-response BackgroundTasks where this code lives. The validate_flow streaming change exposes it. On 3.10, route the create_task call through context.run so the new Task copies the captured context as its current context, matching the isolation the 3.11 path provides via the context= kwarg. Add a regression test that asserts end_all_traces sees the value of a contextvar set before the context was captured, even after the caller mutates that var. * fix: failing wxo list llm test (#12700) patch service layer and update failing test * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Try to fix the missing typer import --------- Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> Co-authored-by: Adam Aghili <Adam.Aghili@ibm.com> Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai> Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| 7b856e7d22 |
fix(loop): iterate when only item is connected and render item as a table (#12669)
* fix(loop): drive iteration from item_output when done is not connected Extract the loop body execution into an idempotent `_iterate` helper that both `item_output` and `done_output` invoke. On release-1.9.0, `_should_process_output` only runs outputs whose name is in the vertex's outgoing edge source names, so `done_output` never ran when nothing was connected to `done` and the subgraph iteration never happened. `_iterate` caches aggregated results and cached exceptions in ctx so repeat calls (when both outputs are wired) run the subgraph exactly once and surface the same failure on re-entry. `item_output` is now async, calls `_iterate` before stopping the item branch, and returns a Data payload listing the dispatched items so the inspector shows real content instead of an empty string. Add regression tests covering the item-only topology, the classic both-outputs-connected topology, empty input, cached error re-raise, and per-run ctx isolation. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix(loop): emit item output as DataFrame so UI renders it as a table Wrapping the iterated rows in a `Data` payload with `count` / `items` keys caused the Item inspector to render as JSON instead of as a tabular view. Return the rows as a DataFrame, mirroring the Mock Data component pattern, so the inspector uses its table renderer. * feat(loop): emit summary logs for each run Add start, complete, skipped, and error log entries from `_iterate` so the component's Logs tab shows a per-run summary with row counts and total elapsed time. Failures include the elapsed duration and the error message. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix(loop): address review comments - Refresh Research Translation Loop starter flow's saved item-output schema (JSON -> Table) to match item_output now returning a DataFrame. - Clarify the test_ctx_isolation_across_runs docstring: production rebuilds the graph per request, so ctx is effectively per-run; reusing a single Graph instance across runs is intentionally not exercised. - Update _iterate's docstring to describe the actual ctx scope rather than claiming graph-instance-level reset semantics that the framework does not provide. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix(loop): stabilize chat output inspection test * fix(loop): align starter project with item output contract --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 9029c4b61e |
fix: Updates the CI workflow to handle known dependency conflicts. (#12691)
* fix: upgrade playwright to 1.58.0 to address Chromium CVEs - Add playwright>=1.58.0 to override-dependencies in pyproject.toml - Update uv.lock: playwright 1.49.0 -> 1.58.0, pyee 12.0.0 -> 13.0.1 - Fixes CVE-2026-2313, CVE-2026-2314, CVE-2026-2315, CVE-2026-2319, CVE-2026-2321, CVE-2026-2441, CVE-2026-2648, CVE-2026-2649 - Ensures Docker builds download updated Chromium with security patches * fix: update npm to latest version to address brace-expansion CVE-2026-33750 - Add npm update after Node.js installation in Dockerfile - Fixes CVE-2026-33750 in system npm's brace-expansion dependency - System npm had brace-expansion 2.0.2, update gets 5.0.5+ - Low risk change: npm is backward compatible, only affects CLI tool * revert: remove npm update from Dockerfile - npm update attempts were causing CI build failures - Bundled npm has issues but updating it is proving problematic - Focus on playwright CVE fix which is the primary concern - brace-expansion CVE-2026-33750 is lower priority (DoS only) * CI: Filter cuga/playwright dependency conflict in release workflow - Filter cuga/playwright conflict (we override playwright>=1.58.0 for CVE fixes) - Still fails CI if other genuine dependency issues are detected - Applied to both base and main package build steps --------- Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> |
|||
| 2fa3d1c759 |
feat: add langflow-sdk support to release workflow (#12679)
* feat: add langflow-sdk build and publish support to release workflow Add support for building and publishing the langflow-sdk package in the release workflow, mirroring the nightly build support added in #12491. Changes: - Add `release_sdk` workflow input for controlling SDK release - Add `determine-sdk-version` job with first-release PyPI handling - Add `build-sdk` job with version verification and import testing - Add `publish-sdk` job that publishes SDK to PyPI before LFX - Update `build-lfx` to download SDK artifact and test both wheels together - Update `build-base` and `build-main` to use SDK wheel as find-links - Pass SDK artifact to cross-platform tests - Add validation: releasing LFX requires releasing SDK - Add pre-release version handling for SDK and LFX's SDK dependency * Update release.yml * Update release.yml * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
|||
| 26c415056c |
feat: add SHA-256 hash-based API key lookup (#12597)
* feat: add SHA-256 hash-based API key lookup Replace the O(n) full table scan in check_key with an indexed SHA-256 hash column for O(1) lookup. Legacy keys without a hash fall back to decrypt-and-compare and get their hash backfilled on first successful match. - Add api_key_hash column to ApiKey model (nullable, indexed) - Hash stored at key creation time - Migration adds column and backfills hashes for existing keys - Fail closed when multiple keys share the same hash - Remove unused fernet_obj parameter from decrypt_api_key * fix(migration): skip hashing duplicate-plaintext API keys Two-pass backfill: decrypt all rows, group by plaintext, then hash only unique-plaintext rows. Duplicate-plaintext groups are left with NULL hash so the runtime fast-path lookup cannot return multiple matches and fail closed. The runtime slow-path matches and backfills exactly one row per group on first use; remaining NULL rows become harmless orphans. Logs counts only (no key IDs or plaintexts) to avoid leaking operational info via deployment logs. Extracts _backfill_hashes helper for testing; adds 9 unit tests. --------- Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| cfaba3ede7 |
fix: Remove redundant api key field in KB Ingest (#12624)
* fix: Remove redundant api key field in KB Ingest * Update component_index.json |
|||
| 499815a630 |
feat: add policies component for tool protection via ToolGuard (#12592)
* feat: add policies component for tool protection via ToolGuard Reintroduce the policies component from #12564 (originally by @boazdavid). Adds policy-based tool protection system with business policy enforcement using ToolGuard, including guard code generation from policy definitions, support for multiple language models, and enhanced tool metadata. Adds toolguard>=0.2.4 dependency and click>=8.3.2 override. * [autofix.ci] apply automated fixes * Update dependencies for new toolguard * Update component_index.json * Update uv.lock --------- Co-authored-by: DAVID BOAZ <DAVIDBO@il.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| cde9f23001 |
feat: deployment page and stepper UI with watsonx Orchestrate integration (#12303)
* feat: deployment page list
* feat: add deployment stepper modal with context-based state management
Implement a multi-step deployment creation flow with 4 steps (Provider,
Type, Attach Flows, Review). State is centralized in a scoped
DeploymentStepperContext to avoid prop-drilling across step components.
Includes bug fixes for version re-selection and connection pre-selection.
* feat: replace mock flows and versions with real API data in deployment stepper
Fetch real flows from the API (scoped to current folder) and versions
per-flow lazily when selected. Enrich selectedVersionByFlow context to
store versionTag alongside versionId so the review step can display it
without re-fetching. Remove MOCK_FLOWS_WITH_VERSIONS from mock-data.
* refactor: improve deployment stepper code quality and conventions
Rename all new deployment files to kebab-case per project conventions,
fix context re-render issues with useMemo/useCallback, replace raw
Tailwind colors with design tokens, add missing data-testid and ARIA
attributes, fix Deploy button disabled on review step, and move shared
FlowTabType to a dedicated types file.
* refactor: align deployment provider types and UI to backend contracts
- Rename ProviderInstance -> ProviderAccount to match backend naming
- Update ProviderAccount fields to match DeploymentProviderAccountGetResponse (provider_key, provider_url, provider_tenant_id, created_at, updated_at)
- Update ProviderCredentials fields to match DeploymentProviderAccountCreateRequest (provider_key, provider_url, api_key)
- Rename all "instance" UI terminology to "environment" (tabs, labels, components)
- Auto-select watsonx Orchestrate on mount as the only supported provider
- Remove Kubernetes from mock providers; update mock data to use watsonx_orchestrate only
* feat: add react-query hooks for deployments and provider accounts with mock data
- Add useGetProviderAccounts hook (GET /deployments/providers) replacing direct MOCK_PROVIDER_INSTANCES import in step-provider.tsx
- Add useGetDeployments hook (GET /deployments) replacing direct MOCK_DEPLOYMENTS import in deployments-page.tsx
- Replace fake setTimeout loading state with hook isLoading state
- Register DEPLOYMENTS and DEPLOYMENT_PROVIDER_ACCOUNTS URL constants
- Real API calls are commented out with TODO markers for easy swap when backend is ready
* chore: remove stale biome-ignore suppression in step-attach-flows
* fix: replace button role=radio with semantic input type=radio in ProviderCard
* fix: replace button role=radio with semantic radio inputs across deployment stepper
- Extract RadioSelectItem component (label + sr-only input + checkbox indicator)
used by version, connection, and environment selectors
- Fix step-type.tsx type cards with label/input pattern (matching ProviderCard)
- Add role="radiogroup" wrapper to EnvironmentList
- Fix envVars key: use stable crypto.randomUUID() id instead of array index
* feat: add name field to provider accounts, multi-select connections, and real API call
- Add `name` field to ProviderAccount, ProviderCredentials, and mock data
- Switch connection selection from single to multi-select (CheckboxSelectItem)
- Allow creating new connections inline from the attach flows step
- Lift connections state up to DeploymentStepperContext
- Move ConnectionItem type from step-attach-flows to shared types.ts
- Wire up real API call in useGetProviderAccounts (remove mock)
* feat: wire deploy button and populate deployments page with real API data
- Add usePostDeployment and usePostProviderAccount mutation hooks
- Wire Deploy button in stepper modal: creates provider account if needed,
then POSTs to /api/v1/deployments with WXO provider_data shape
- Fix environment_variables payload: wrap values as { value, source: "raw" }
to satisfy EnvVarValueSpec schema
- Fix connection app_id: prefix with conn_ so WXO name validation passes
(names must start with a letter)
- Multi-select connections with checkbox UI; persist connections in context
so they survive back/forward navigation
- Update Deployment type to match API response shape; remove mock fields
(url, status, health, lastModifiedBy)
- Wire useGetDeployments to real API; load provider ID from useGetProviderAccounts
- Update deployments table to display real fields: name, type, attached_count,
provider name, updated_at
* refactor: extract DeploymentsContent component to eliminate nested ternary
* feat: add Test Deployment chat modal for deployed agents
Implements a chat interface to test deployments directly from the UI.
Wires up two entry points: the stepper modal "Test" button (inline
transition) and the deployments table play button (standalone dialog).
- Add usePostDeploymentExecution and useGetDeploymentExecution hooks
hitting POST/GET /api/v1/deployments/executions
- Build test-deployment-modal: ChatHeader, ChatMessages, ChatInput,
ChatMessageBubble with user/bot avatars, loading dots, tool traces
- useDeploymentChat hook: recursive setTimeout polling (max 30 × 1.5s),
thread_id persistence for multi-turn, watsonx response parsing
(response_type/type text, wxo_thread_id extraction)
- Stepper modal transitions inline to TestDeploymentContent on Test click
- Deployments table play button opens standalone TestDeploymentModal
* feat: add syntax highlighting to chat code blocks using SimplifiedCodeTabComponent
* feat: add action menu and icon-based type badge to deployments table
- Add dropdown action menu (Duplicate, Update, Delete) to each row
- Replace left-border type badge with icon-based badge (Bot for Agent, Plug for MCP)
* refactor: remove connected status from provider card in stepper
* feat: auto-detect flow env vars in deployment connection form
- Add POST /deployments/variables/detections backend endpoint that scans
flow version data for credential fields (load_from_db=True globals and
password=True fallbacks) and returns detected variable names
- Derive meaningful env var names from the model field's category when no
global variable is linked (e.g. OPENAI_API_KEY from category "OpenAI")
- Add DetectEnvVarsRequest/DetectedEnvVar/DetectEnvVarsResponse schemas
- Add usePostDetectDeploymentEnvVars frontend mutation hook
- Pre-populate Create Connection env var rows with detected keys/values
when attaching a flow version; global variable selections render as tags
via InputComponent with global variable picker support
* feat: add empty state and smart default tab for available connections
- Default to "Create Connection" tab when no connections exist
- Show empty state with icon, description, and shortcut link when
the Available Connections tab has no items
* feat: redesign review step with two-column layout and env vars section
Match new design reference with Deployment/Attached Flows columns
and a masked Configuration section showing env variable keys.
* feat: integrate delete deployment with loading state and fix test modal flow
- Add useDeleteDeployment hook (DELETE /deployments/{id}, refetches list)
- Show spinner + faded row while deletion is in progress; fix race where
deleteTarget was cleared before isPending resolved by using separate deletingId state
- Redesign StepDeployStatus with animated spinner, ping ring, and success checkmark
- After deploy, "Test" closes the stepper and opens the standalone TestDeploymentModal
(consistent UI, correct providerId, chat reset on close)
- Prevent closing the stepper modal while deployment is in progress
* refactor: address PR review concerns for deployment UI
- Split step-attach-flows.tsx (656→274 lines) into FlowListPanel,
VersionPanel, and ConnectionPanel sub-components
- Extract Watsonx parser utilities into watsonx-result-parsers.ts,
reducing use-deployment-chat.ts from 384 to 277 lines
- Surface detectEnvVars errors via setErrorData instead of silently
resetting state
- Add EnvVarEntry named type to types.ts, replacing inline shape
repeated across two files
- Move import json to module level in deployments.py (PEP 8)
- Fix URL construction in useGetDeploymentExecution to use axios
params option, consistent with other hooks
- Remove commented-out MOCK_CONNECTIONS dead code
- Add TODO comment to hardcoded "watsonx-orchestrate" provider key
* refactor: apply React best practices and remove mock data from deployment UI
- Fix barrel imports: import directly from source files in deployments-page,
step-provider, and step-attach-flows
- Replace useEffect auto-select with derived effectiveFlowId in step-attach-flows
- Fix async state init bug for environmentTab using useRef guard pattern
- Fix stale closure in handleAddEnvVar with functional setState
- Wrap useState initial value in lazy initializer for envVars
- Wrap all 8 handlers in useCallback; wrap panel components in memo()
- Remove mock-data.ts; move PROVIDERS constant inline to step-provider
- Drop MOCK_CONNECTIONS (was empty array) from context
- Simplify step-provider UI: remove provider selection radio group since
only one provider exists; show watsonx Orchestrate as a static display
* feat: add Deploy button to canvas toolbar
Adds a primary-colored Deploy button at the far right of the canvas toolbar.
Clicking it saves the flow, creates a version snapshot, and opens the
deployment stepper modal with the current flow and version pre-selected in
the Attach Flows step, including auto-detection of environment variable keys.
* chore: disable ENABLE_DEPLOYMENTS feature flag
* fix: forward LANGFLOW_FEATURE_WXO_DEPLOYMENTS env var to frontend
The feature flag was reading from import.meta.env but the variable
was never injected by Vite's define config, so the deployments
feature was always disabled regardless of the .env value.
* feat: implement providers tab with environment list
Replace the "coming soon" placeholder in the Providers sub-tab with a
real table showing existing provider accounts (name, URL, provider key,
created date) fetched from the API, including loading skeleton and
empty state.
* feat: add provider creation modal with tab-aware action button
Create AddProviderModal with name, API key, and URL fields matching
the deploy modal. The top-right button now switches between
"New Deployment" and "New Environment" based on the active sub-tab.
* feat: implement provider account deletion with confirmation modal
Add useDeleteProviderAccount hook, wire it through ProvidersContent
and ProvidersTable with loading/disabled row state on delete, and
reuse DeleteConfirmationModal for the confirmation flow.
* fix: correct provider_key to watsonx-orchestrate and add API key visibility toggle
Fix the provider_key value sent to the API. Add eye/eye-off toggle
to the API Key input in both the add provider modal and the deploy
stepper's provider step.
* feat: navigate to deployments tab and open test modal after canvas deploy
After a successful deployment from the canvas deploy button, clicking
"Test" navigates to the deployments tab and auto-opens the test modal.
Also adds eye toggle to the deploy stepper's API Key field.
* Add llm config to deployment creation workflow
* [autofix.ci] apply automated fixes
* feat: add useGetDeploymentLlms query hook
Add missing query hook for the GET /deployments/llms endpoint,
resolving the broken import in step-type.tsx.
* Create provider env inline of step
Ensures the list llm call has an authed provider account to use
* feat: add extensibility for wxo tools in deployments api (#12425)
* feat(deployments): surface tool_ids in WXO API for explicit tool control
- Fix bind to reuse existing WXO tools via attachment lookup instead of
always creating new ones
- Add tool_id-based operations (bind_tool, unbind_tool, remove_tool_by_id)
alongside existing flow_version_id operations
- Add PATCH /deployments/snapshots/{provider_snapshot_id} endpoint to
update snapshot content with a new flow version (blast-radius bounded
to Langflow-tracked tools only)
- Add update_snapshot method to WXO adapter
- Enrich flow version list with deployment info (tool_id, tool_name,
app_ids) via FlowVersionReadWithDeployments API schema
- Surface tool_id in WatsonxApiToolAppBinding responses; flow_version_id
becomes optional for tool-id-based operations
* Revert "feat: Add Langflow Assistant chat panel for component generation (#11636)"
This reverts commit
|
|||
| abd772f780 |
fix: Build and install the langflow-sdk for lfx (fixes nightly) (#12481)
* fix: Build and install the langflow-sdk for lfx * Publish sdk as a nightly * Update ci.yml * Update python_test.yml * Update ci.yml |
|||
| 4bdc87dcc1 |
fix: Import and Statistics fixes for Knowledge Bases (#12446)
* fix: Access the appropriate attribute for chroma * Fix display of chunk metadata * [autofix.ci] apply automated fixes * Add some unit tests * Update ingestion.py * [autofix.ci] apply automated fixes * Review updates * Update component_index.json * Fix bug with ingestion * [autofix.ci] apply automated fixes * Update test_ingestion.py * Update test_ingestion.py --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Carlos Coelho <80289056+carlosrcoelho@users.noreply.github.com> |
|||
| 45325f6376 |
feat: Langflow SDK and Flow DevOps API Toolkit (#12245)
* feat(sdk): add langflow-sdk Python client package * feat(lfx): add Flow DevOps CLI toolkit * feat(api): add flow upsert, export normalization, ZIP upload * chore: CI coverage merge, Docker fixes, dependency updates * Address CQ5 review / import bug * Update test_status_command.py * Update test_status_command.py * Fix more tests * Review edits * Review edits * Fix tests * Follow up review updates * Refactor common client code * Update templates and comp index * [autofix.ci] apply automated fixes * Update test_database.py * Updates from review comments * Fix push interface to match the rest * Update push.py * Update push.py --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| c43fbc95ce |
feat: LE-374 token usage tracking for LLM and Agent components (#11891)
* feat: add token usage tracking for LLM and Agent components
Track input/output/total tokens across LLM providers (OpenAI, Anthropic,
Ollama) and display them on both node badges and chat messages.
Backend: thread-safe callback handler for agent token accumulation,
usage_metadata extraction for Ollama/LangChain standard, pipeline
integration from component through vertex to API response.
Frontend: token count formatting utility, Coins icon badge on nodes
with tooltip breakdown, chat message status with token display.
* feat: accumulate token usage across serial LLMs on chat messages
Add upstream token usage accumulation so chat messages display the
total tokens from all LLMs in the pipeline, not just the last one.
Output vertex node badges hide token counts since the accumulated
total is shown on the chat message instead.
* chore: add CLAUDE.local.md to .gitignore
* chore: update starter project templates for token usage tracking
* fix: enable token usage tracking for streaming LLM responses
Enable stream_usage=True on OpenAI and Anthropic model constructors so
the API includes token counts in streaming chunks.
Fix _handle_stream to propagate the AIMessage back to _get_chat_result
when not connected to a chat output, so usage can be extracted from the
invoke fallback path.
Accumulate usage across multiple streaming chunks instead of overwriting,
since Anthropic splits input/output tokens across separate events.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* refactor: centralize token usage extraction into shared module
Extract duplicated token usage logic from Component, LCModelComponent,
TokenUsageCallbackHandler, and Vertex into a shared lfx.schema.token_usage
module. Replace loose dict typing with the existing Usage Pydantic model
throughout the token tracking pipeline. Declare _token_usage on Component
__init__ instead of dynamically injecting it.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* feat: add validation for token_usage field in ResultDataResponse
* feat: enable stream_usage in OpenAI model tests
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* refactor: consolidate token usage extraction into single source of truth
Eliminate ~75 lines of duplicated LLMResult token extraction logic between
the token usage feature (TokenUsageCallbackHandler) and the traces feature
(NativeCallbackHandler) by adding a shared extract_usage_from_llm_result()
function. Also fix missing usage property mapping in chat history hook so
token counts display correctly in playground messages.
* feat: add token usage tracking to all LLM components
Add token usage extraction to the 7 remaining components that make LLM
calls but weren't tracking token consumption:
- Smart Router: direct extract_usage_from_message after invoke
- Guardrails: accumulate_usage across multiple guardrail checks
- Batch Run: accumulate_usage across batch responses
- Smart Transform: extract after ainvoke (already done in prior commit)
- Structured Output: via token_usage_callback on get_chat_result
- LLM Selector: direct extract for judge + callback for selected model
- NotDiamond: via token_usage_callback on get_chat_result
Also adds a backward-compatible token_usage_callback parameter to
get_chat_result() so components using that shared helper can capture
the AIMessage before it's reduced to .content.
* [autofix.ci] apply automated fixes
* fix: update mock_get_chat_result signatures to accept token_usage_callback
The structured output test mocks define explicit parameter lists for
get_chat_result but were missing the new token_usage_callback kwarg,
causing CI failure. Add **kwargs to all mock definitions.
* fix: address PR review findings for token usage UI and data flow
- Remove bare "bg" Tailwind class and replace hard-coded bg-neutral-700
with semantic bg-success-background on success tooltip (C2/C3)
- Propagate usage properties regardless of source.id presence so agent
inner messages still show token counts (H9)
- Make PropertiesType.source optional to match the new data flow
- Restore "in" preposition in "Finished in X.Xs" chat message (M10)
- Fix misleading "optional dependency" comment in native_callback.py (C1)
* fix: structured output token usage not captured due to config key mismatch
get_chat_result() reads "get_langchain_callbacks" as a callable, but
structured output was passing "callbacks" as a list — the token handler
was silently dropped. Fix by matching the expected key names and
injecting TokenUsageCallbackHandler via the LangChain callback chain
instead of the token_usage_callback parameter (which doesn't fire for
structured output chains that return Pydantic models, not AIMessages).
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* chore: update component index
* test: add E2E tests for token usage tracking
* [autofix.ci] apply automated fixes
* test: add missing unit tests from PR review
Adds the 4 recommended test scenarios identified in Cristhianzl's review
of PR #11891 (token usage tracking):
- TestStreamingTokenAccumulation: verifies extract_usage_from_chunk() +
accumulate_usage() correctly accumulates across multiple streaming chunks
(OpenAI, Anthropic, and usage_metadata formats)
- TestChatOutputTokenUsageAccumulation: verifies message_response() sets
upstream token usage on the message and updates the stored message when
applicable
- TestAgentTokenCallbackWiring: verifies TokenUsageCallbackHandler is wired
into run_agent() callbacks and its result is stored on _token_usage
- TestResultDataResponseTokenUsageValidator: verifies the field_validator
converts Usage Pydantic models to dicts and passes through None/dict values
* [autofix.ci] apply automated fixes
* Revert "[autofix.ci] apply automated fixes"
This reverts commit
|
|||
| ccc7ffa714 |
fix: Improve the sub-process handling of the Docling Worker (#12296)
* fix: Don't reprocess files in Advanced Mode * Preserve metadata in docling processing * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Preserve metadata in docling processing * Preserve metadata in docling processing * Update docling_inline.py * [autofix.ci] apply automated fixes * Preserve metadata in docling processing * Emit log while worker is active * [autofix.ci] apply automated fixes * Update test_file_component_image_processing.py * Update test_file_component_image_processing.py * Update component_index.json * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * Update component_index.json * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Merge release branch * Secrets baseline update * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix: Handle markdowns with large buffer output * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 86dd7dfc31 |
fix: resolve code scanning alerts for URL sanitization and insecure randomness (#12362)
Use urlparse for proper hostname validation instead of substring check (LE-719). Replace Math.random() with crypto.randomUUID() in test fixtures (LE-718). Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com> |
|||
| 42832d07a5 |
feat: add core deployment implementation (#12108)
* checkout api handlers
* add missing table
* update to use "version" terminology for flows instead of outdated "history" verbage
* Fix provider account id mapping
* recover a little todo comment
* Add flow version migration and minor exception handling, etc
1. deployments.py — Added AuthenticationError import + handling (401) in all 9 error handler blocks. Added update_deployment_db import + call to persist name changes to local DB after adapter update succeeds.
2. flow_version/exceptions.py — Added FlowVersionDeployedError for blocking deletion of deployed versions.
3. flow_version/crud.py —
- Added has_deployment_attachments() helper that checks if a flow version has any deployment attachments
- delete_flow_version_entry() now raises FlowVersionDeployedError if the version is attached to deployments
- Pruning now excludes deployed versions via a NOT IN subquery on FlowVersionDeplottachment, preventing provider-side snapshot orphaning
4. flow_version/__init__.py — Exports FlowVersionDeployedError
5. flow_version.py (API route) — Imported FlowVersionDeployedError, added it to _translate_version_error as 409 Conflict
6. models/__init__.py — Registered FlowVersionDeploymentAttachment for alembic/SQLModel metadata detection
7. Migration c0d2ce43b315 — Creates flow_version_deployment_attachment table with all columns, FKs (CASCADE), and indexes. Single head, chains off fc7f696a57bf.
* [autofix.ci] apply automated fixes
* address bugs and inconsistencies
* rename column to "provider_snapshot_id"
* harden deployment API: fail loudly on invalid state instead of silently passing
Replace silent fallbacks, warning logs, and dropped values with hard
failures (422/500/502) across the deployment orchestration layer. Adds
logging to bare exception handlers and wraps materialize_snapshots in
the adapter error handler.
* [autofix.ci] apply automated fixes
* add materialize_snapshots to deployment service protocol
Promotes materialize_snapshots from duck-typed getattr usage to a
first-class method on DeploymentServiceProtocol, BaseDeploymentService,
and the no-op DeploymentService stub. Introduces MaterializeSnapshotsResult
schema for typed return values.
Updates deployments.py to call the method through the protocol instead of
getattr, giving static analysis coverage over the contract.
Documents the snapshot abstraction across BaseFlowArtifact, SnapshotItem,
MaterializeSnapshotsResult, and FlowVersionDeploymentAttachment.provider_snapshot_id
— explaining that snapshots are immutable, provider-owned copies of flow data
with opaque provider-assigned identifiers (e.g. wxO tool ID, K8s ConfigMap
name, S3 key).
* deployment sync: extract helpers, server-side type filter, orphan detection
- Extract _fetch_provider_resource_keys helper for provider validation
- Rename _sync_page_with_provider → _list_deployments_synced
- Match resource keys by provider ID only (not name)
- Restore server-side deployment_type filter with guard against
false deletions (skip rows whose type doesn't match the filter
instead of deleting them)
- Add orphan/divergence logging for post-create, post-update,
post-duplicate DB write failures
- Return 422 on invalid UUID in update remove list (was silently ignored)
- Handle NotImplementedError → 501 from provider adapters
- Convert attachment IntegrityError to ValueError with descriptive message
- Add tests for sync helpers (15 cases)
* rebase on release-1.9.0 and align with lfx/services
* refactor(deployments): align provider mapper routing and WXO update payload mapping
Align deployment mapper resolution with adapter-type/provider-key routing and
refactor PATCH update handling to use mapper resolve/shape contracts end-to-end.
Map Langflow flow_version_id references at the API boundary into provider update
operations for Watsonx bind/unbind/remove paths, with expanded mapper tests.
* patch down-revision
* first pass with formalized boundary rules
* fix(deployments): harden watsonx payload boundary contracts
Enforce fail-fast payload slot parsing for required adapter results, split execution create/status slot contracts, and route execution-create mapping through deployment mappers.
Require watsonx flow artifact source_ref and move update reconciliation output to provider_result to keep mapper/adapter boundaries explicit and typed.
* refactor(deployments): modularize watsonx orchestrate create/update flow
Extract create/update logic into dedicated core modules with shared helpers to tighten deployment boundary contracts.
Align backend/lfx payload schema mapping and expand e2e/unit coverage for response mapping and update schema behavior.
* api impl for wxo-specific create payload
* further refactoring. add rollback of existing tools (undo new app bindings) in the create path
* add todo in execution.py
* refactor(deployments): replace snapshot_id/reference_id with source_ref-correlated tool refs
Introduce WatsonxToolRefBinding to correlate source_ref (flow version id)
with provider tool_id across all operation types. This replaces the prior
reference_id and snapshot_id fields with a unified structure that carries
provenance through create, bind, unbind, and remove_tool operations.
Key changes:
- Flatten API operation payloads: hoist flow_version_id onto operations,
remove nested WatsonxApiUpdateToolReference wrapper
- Replace tools.existing_ids with inline tool_id_with_ref on bind operations
- Rename WatsonxCreateSnapshotBinding to WatsonxToolRefBinding (input) and
WatsonxResultToolRefBinding (output, with created flag)
- Add created_app_ids to update results for connection tracking
- Raise HTTPException on contract violations in _to_api_tool_app_bindings
instead of silently dropping unmappable bindings
- Add schema-level validation for conflicting source_ref on same tool_id
- E2E: cache tool_id→source_ref from create results, use helpers to build
refs with distinct source_ref vs tool_id values
* Enforce DeploymentType enum and add description column
Make deployment_type a required column backed by a SQLAlchemy
TypeDecorator that validates on write (rejects None and invalid strings)
and coerces to DeploymentType on read. Add nullable description column
to the deployment model and surface it through the API.
Key changes:
- Add _DeploymentTypeColumn TypeDecorator for enum round-trip fidelity
- Make deployment_type non-optional in Deployment model, DeploymentRead,
CRUD functions, and API layer
- Add description (Text, nullable) to Deployment model and fold its
migration into the existing c0d2ce43b315 revision
- Remove _resolve_deployment_type helper — TypeDecorator handles coercion
- Remove DeploymentType fallbacks and backward-compat shims from API
endpoints, base mapper, and watsonx orchestrate mapper
- Document cross-package coupling: DeploymentType is owned by lfx but
persisted by langflow; member values must never be removed
- Fix pre-existing bug: provider_account_id → deployment_provider_account_id
in get_deployment_status endpoint
Note: deployment_type is nullable=True at the DB level to satisfy the
EXPAND-phase migration validator; NOT NULL is enforced at the application
layer by the _DeploymentTypeColumn TypeDecorator.
* refactor(deployments): extract route helpers, harden sync and error handling
Move bulk of deployment route logic into mappers/helpers layer to slim
down deployments.py and enforce clearer boundary between routes, mappers,
and adapters (documented in DEPLOYMENT_BOUNDARY_RULES.md).
Key changes:
- Extract ~700 lines from deployments.py into helpers.py (pagination,
adapter/mapper resolution, attachment management, snapshot sync,
rollback, response shaping)
- Add read-path snapshot-level sync: get_deployment and
list_deployments_synced verify provider_snapshot_ids against the
provider and prune stale attachments, with graceful fallback on error
- Add compensating rollback for create (rollback_provider_create) and
update (rollback_provider_update) when DB commit fails after provider
mutation, using mapper-driven payload reconstruction
- Introduce handle_adapter_errors() context manager centralising
DeploymentServiceError → HTTP status mapping via
http_status_for_deployment_error; sanitise 500 detail to avoid
leaking internals
- Add DeploymentNotConfiguredError → 503 mapping
- Add util_snapshot_ids_to_verify and resolve_rollback_update to base
mapper with WxO overrides for provider-specific snapshot ID extraction
and put_tools-based rollback payloads
- Add put_tools field to WatsonxDeploymentUpdatePayload for full tool
list replacement; early-return in build_provider_update_plan and
validate_operation_references when put_tools is set
- Extract verify_tools_by_ids into core/tools.py helper
- Harden resource_name_prefix with strip_whitespace + min_length=1
- Deduplicate snapshot_ids before provider calls
- Add deterministic order_by(created_at) to attachment CRUD queries
- Add exc_info=True to all best-effort rollback/compensate error logs
- Add session.rollback() in get_deployment snapshot sync error path
- Warn when list_snapshots receives both deployment_ids and snapshot_ids
- Add E2E scenarios for empty snapshot list, mixed snapshot IDs, tools
endpoint, and deployment re-list after update
Tests:
- Add test_deployment_route_handlers.py covering stale-row delete +
commit, non-404 adapter errors, handle_adapter_errors wiring,
snapshot sync (happy path, skip, error fallback), project-scoped
flow version validation for create and update
- Expand test_deployment_sync.py with snapshot-phase tests, rollback
tests, pagination guard, and project-scoped validation
- Add deployment_type assertion to response mapping test
- Add DeploymentNotConfiguredError and bare DeploymentServiceError cases
to exception mapping tests
- Add put_tools schema and update plan tests
* remove pompous performance commentary
* fix(deployments): remove upsert behavior and fail fast on duplicate name
The create_deployment endpoint previously performed a
get_deployment_by_resource_key lookup before inserting the DB row,
silently tolerating duplicates. Since the provider adapter returns a
fresh resource ID on every create, this lookup could never legitimately
match — and if it did, it would mask a data inconsistency bug.
Changes:
- Remove the get-or-create (upsert) pattern; go straight to
create_deployment_db and let the unique constraint surface conflicts.
- Add deployment_name_exists CRUD function and an early 409 guard so
duplicate names are rejected before any provider call, avoiding
costly provider-side rollback for a locally-checkable condition.
- Update existing route-handler tests to reflect the removed lookup.
- Add tests for deployment_name_exists and the 409 duplicate-name path.
* feat: convert provider_key and deployment_type columns to DB-level enums
Replace plain string columns with SQLAlchemy Enum types backed by
Postgres/SQLite enum constraints, enforcing valid values at the DB
layer rather than only in application code.
Migration follows expand-contract pattern (add enum column, backfill,
drop old string column, rename) with index ops outside batch context
to avoid SQLite column-lookup issues. Upgrade and downgrade are fully
atomic.
- Add DeploymentProviderKey enum as single source of truth for
provider identifiers; remove magic strings and _DeploymentTypeColumn
TypeDecorator
- Make provider_key immutable after creation (remove from update
request schema and API handler)
- Fix pre-existing test gap: add missing deployment_type argument to
all TestDeploymentCRUD create_deployment() calls
* feat(flow-versions): add deployment awareness and sync-on-read
Add is_deployed field to flow version responses and provider-verified
sync to the list/get read paths, matching the existing deployment
endpoint sync pattern.
API changes:
- list_flow_versions returns is_deployed per entry and accepts optional
deployment_ids filter to scope by specific deployments
- get_single_flow_version returns is_deployed on the full response
- Both endpoints now use write sessions and run best-effort
snapshot-level sync before returning results
Sync-on-read (helpers.sync_flow_version_attachments):
- Queries all attachments for a flow's versions joined with deployment
and provider account info in a single query
- Groups by provider, resolves adapter/mapper per group, verifies
provider_snapshot_ids via list_snapshots, and prunes stale attachment
rows through the existing sync_attachment_snapshot_ids helper
- Per-provider error handling so one provider outage doesn't block the
response
New CRUD: list_attachments_for_flow_with_provider_info joins
FlowVersionDeploymentAttachment -> Deployment -> DeploymentProviderAccount
to avoid N+1 queries during sync grouping.
Tests: 50 passing (11 new) covering is_deployed indicator, deployment_ids
filtering, stale attachment pruning on list/get, and sync failure
resilience.
* Add user id authentication to a few missing endpoints
* [autofix.ci] apply automated fixes
* Update tests
Removes some mock tests that did nothing
Adds sqlite for true testing of db accessors
Reduces Mock objects usage
* refactor(deployments): enforce ownership boundaries on execution responses
Move provider-owned execution identifiers (execution_id, agent_id,
status, timestamps, errors) out of the top-level API response and into
provider_data, keeping only Langflow-owned fields (deployment_id) at the
top level. This prevents future collisions if Langflow introduces its
own execution tracking.
Key changes:
- Remove execution_id from _ExecutionResponseBase; provider's opaque
run identifier now lives exclusively inside provider_data
- Rename WatsonxExecutionResultData → WatsonxAgentExecutionResultData
(adapter layer) and split the API-layer class into a private base
(_WatsonxApiAgentExecutionResultBase) with dedicated
WatsonxApiAgentExecutionCreateResultData and
WatsonxApiAgentExecutionStatusResultData subclasses
- Translate WXO run_id → execution_id at the adapter boundary
(create_agent_run_result / get_agent_run).
- Collapse util_execution_id + util_execution_deployment_resource_key
into a single util_resource_key_from_execution that trusts the
adapter-provided result.deployment_id directly
- Remove build_orchestrate_runs_query and extra payload fields
(thread_id, llm_params, guardrails, etc.) unused in MVP
- Simplify WxOClient.post_run signature (drop query_suffix)
- Exclude provider_data from flow tool artifact to avoid unexpected
top-level keys in the WxO tool runtime
- Document ownership boundary rules in DEPLOYMENT_BOUNDARY_RULES.md §14
- Add E2E polling for terminal execution status, input format variants,
and missing-deployment negative test
- Expand unit tests for renamed schemas, field mapping, passthrough
validation, and simplified payload builder
* feat: add name column to deployment_provider_account
Add a required, user-chosen display name to provider accounts
(e.g. "staging", "prod") that is unique within a given provider_key.
Includes model, CRUD, API schema/route, mapper, migration with
backfill, and tests.
* fix: replace hand-written migration with Alembic-generated revision
Replace the manually authored migration (b4e6f8a2c1d3) with an
Alembic-generated one (8255e9fc18d9) for the deployment_provider_account
name column.
Fix SQLite compatibility: sa.func.concat() generates a concat() function
call which does not exist in SQLite. Use sa.literal().concat() instead,
which produces the || operator and works on both PostgreSQL and SQLite.
* remove bogus unverified math from migration file
* feat: verify provider credentials before account creation
Add a verify_credentials step to the provider account creation flow
that validates API keys against the provider before persisting them.
This prevents storing invalid or revoked credentials and gives users
immediate feedback.
Key changes:
- Add verify_credentials to the deployment adapter interface (base,
service, protocol) with WXO implementation that obtains a token
via the IBM authenticator
- Add SSRF-hardened URL validation for provider_url (HTTPS-only,
private IP blocklist, localhost rejection, normalization)
- Introduce ValidatedUrl/ValidatedUrlOptional annotated types in
the API schema layer
- Refactor raise_for_status_and_detail to accept an optional cause
parameter for explicit exception chain control
- Use ResourceNotFoundError (parent) instead of DeploymentNotFoundError
in raise_for_status_and_detail for provider-agnostic 404 mapping
- Narrow get_authenticator return type to concrete union
* use whitelist only for valid urls
* feat: BREAKING: move credentials into provider_data and centralize update logic in mapper
- Replace top-level `api_key: SecretStr` with opaque `provider_data: dict` in API schemas;
mapper extracts credentials via `resolve_credential_fields` for DB storage
- Add `resolve_provider_account_update` to base mapper so routes delegate
full update-kwargs assembly (including cross-field logic) to the mapper
- WXO mapper override re-derives `provider_tenant_id` when `provider_url` changes
- Add tenant extraction utilities and `validate_tenant_url_consistency` in
`deployment_provider_account/utils.py` as single source of truth
- Add `model_validator` on `DeploymentProviderAccount` for defense-in-depth
tenant/URL consistency checks
- Rename `DEPLOYMENT_BOUNDARY_RULES.md` → `RULES.md`; document DB-direction
mapper contract, credential flow, and update assembly
- Update all tests for new `provider_data` shape, mapper update methods,
tenant extraction, and model-level consistency validation
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix(deployments): Harden provider account validation and WXO rollback
Use provider_url when resolving WXO credentials and scope provider account names per user within each provider. Re-verify provider account updates before persisting them and return 4xx responses for account conflicts instead of surfacing 500s.
Block deleting provider accounts that still own deployments and extend create rollback so failed WXO writes clean up provider-side resources. Add route, schema, mapper, sync, CRUD, and WXO tests to cover the new behavior.
* [autofix.ci] apply automated fixes
* update url validator docs; remove outdated reference to private url blacklist logic
* fix(deployments): Isolate snapshot sync writes
Use nested transactions around best-effort snapshot cleanup so provider sync failures cannot leak partial attachment deletes into the outer request transaction.
Persist explicit description clears during deployment PATCH requests, and add regression tests for both deployment sync paths and the route handler update flow.
* fix(deployments): Harden delete cleanup and wxO create tests
Treat missing provider agents as stale local cleanup so delete can
finish instead of leaving orphaned deployment rows behind.
Commit local delete operations eagerly, retry once on commit failure,
and move wxO create-path tests toward fake client objects so the real
service logic is exercised without external calls.
* new head
* fix(deployments): stop prefixing wxo raw connection app_ids
Preserve caller app_ids for newly created wxo connections while keeping lf_ prefixing for tool/deployment naming, centralize resource_name_prefix validation, and update mapper/service schema tests and docs to reflect the new behavior.
* fix(deployments): Harden provider account cleanup
Reconcile stale deployment rows before provider-account deletion so
out-of-band provider removals do not leave account cleanup blocked.
* fix: restore py310 compatibility and align payload slot tests
Import Self from typing_extensions in the deployment provider account model to keep backend imports working on Python 3.10, and update payload formalization tests to assert strict rejection of cross-model BaseModel inputs.
* fix(deployments): resolve mypy typing regressions across deployment flows
Normalize SQLModel query expressions for typed SQL operators, make deployment-related model IDs non-nullable where appropriate, and tighten provider mapper/update typing guards. Update deployment tests to align with stricter type contracts.
* fix(deployments): restore py310 test compatibility and migration idempotency
Replace Python 3.11-only UTC imports with timezone.utc in deployment-related tests and the watsonx e2e helper, and make the provider-account name migration safe to re-run when the column and unique constraint are created in separate steps.
* fix(deployments): restore provider account route ownership helpers
Reuse the shared provider-account lookup in update/delete routes so route tests keep the expected ownership path, and use the explicit-field helper when deciding whether to reverify credentials.
* pass name to create_provider_account helper in tests
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
---------
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
|
|||
| ae02b4b691 |
fix: prevent overwriting user-selected global variables in provider c… (#12329)
* fix: prevent overwriting user-selected global variables in provider c… (#12217) * fix: nightly now properly gets 1.9.0 branch (#12215) before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$' * docs: add search icon (#12216) add-back-svg * fix: prevent overwriting user-selected global variables in provider config Previously, the apply_provider_variable_config_to_build_config function would automatically overwrite field values with environment variable keys whenever an env var was present, even if the user had already selected a different global variable. This fix adds a check to only auto-set the environment variable if: - The field is currently empty, OR - The field is not already configured to load from the database This preserves user selections while still providing automatic configuration for new/empty fields. Added comprehensive unit tests to verify: - Auto-setting env vars for empty fields - Preserving user-selected global variables - Overwriting hardcoded values (expected behavior) - Skipping when env var is not set - Applying component metadata correctly * [autofix.ci] apply automated fixes * style: use dictionary comprehension instead of for-loop Fixed PERF403 Ruff style warning by replacing for-loop with dictionary comprehension in update_projects_components_with_latest_component_versions * chore: retrigger CI build * test: improve test coverage and clarity for provider config - Renamed test_apply_provider_config_overwrites_hardcoded_value to test_apply_provider_config_replaces_hardcoded_with_env_var for clarity - Added test_apply_provider_config_idempotent_when_already_set to document idempotent behavior when value already matches env var key - Removed sensitive value from debug log message to prevent potential exposure of API keys or credentials These changes improve test coverage by documenting the no-op scenario and enhance security by avoiding logging of potentially sensitive data. * chore: retrigger CI build --------- Co-Authored-By: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com> Co-Authored-By: Mendon Kissling <59585235+mendonk@users.noreply.github.com> Co-Authored-By: Steve Haertel <shaertel@ca.ibm.com> Co-Authored-By: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-Authored-By: Eric Hare <ericrhare@gmail.com> * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Update test_unified_models.py --------- Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com> Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com> Co-authored-by: Steve Haertel <shaertel@ca.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> |
|||
| 15c803a7bc | fix: Sanitize folder names for CodeQL (#12263) | |||
| 198fab1dc7 |
feat: Add Windows Playwright test fixes to RC (#12265)
* feat: Add Windows Playwright tests to nightly builds
- Add windows-latest to typescript_test.yml runner options
- Add shell: bash to all script steps for cross-platform compatibility
- Split Playwright installation into OS-aware steps (Linux uses --with-deps, Windows/macOS/self-hosted don't)
- Fix artifact naming with OS prefix to prevent conflicts: blob-report-${{ runner.os }}-${{ matrix.shardIndex }}
- Split frontend-tests into separate Linux and Windows jobs in nightly_build.yml
- Add ref parameter to all test jobs to checkout code from release branch
- Add resolve-release-branch to needs dependencies
- Update Slack notifications to handle both Linux and Windows test results
- Windows tests are non-blocking (not checked in release-nightly-build condition)
- Update .secrets.baseline with new line number (263 -> 347) for LANGFLOW_ENG_SLACK_WEBHOOK_URL
Fixes LE-566
* fix: Use contains() for self-hosted runner detection
- Replace exact string equality (==, !=) with contains() for substring matching
- Fixes issue when inputs.runs-on is array format: '["self-hosted", "linux", "ARM64", ...]'
- Ensures self-hosted Linux runners correctly skip --with-deps flag
Addresses CodeRabbit feedback on PR #12264
|
|||
| 69da593d56 |
feat: Refactor and Unify the ModelInput Selector Across Components (#12025)
* fix: Fixes Kubernetes deployment crash on runtime_port parsing (#11968) (#11975) * feat: add runtime port validation for Kubernetes service discovery * test: add unit tests for runtime port validation in Settings * fix: improve runtime port validation to handle exceptions and edge cases Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai> * fix(frontend): show delete option for default session when it has messages (#11969) * feat: add documentation link to Guardrails component (#11978) * feat: add documentation link to Guardrails component * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * feat: traces v0 (#11689) (#11983) * feat: traces v0 v0 for traces includes: - filters: status, token usage range and datatime - accordian rows per trace Could add: - more filter options. Ecamples: session_id, trace_id and latency range * fix: token range * feat: create sidebar buttons for logs and trace add sidebar buttons for logs and trace remove lods canvas control * fix: fix duplicate trace ID insertion hopefully fix duplicate trace ID insertion on windows * fix: update tests and alembic tables for uts update tests and alembic tables for uts * chore: add session_id * chore: allo grouping by session_id and flow_id * chore: update race input output * chore: change run name to flow_name - flow_id was flow_name - trace_id now flow_name - flow_id * facelift * clean up and add testcases * clean up and add testcases * merge Alembic detected multiple heads * [autofix.ci] apply automated fixes * improve testcases * remodel files * chore: address gabriel simple changes address gabriel simple changes in traces.py and native.py * clean up and testcases * chore: address OTel and PG status comments https://github.com/langflow-ai/langflow/pull/11689#discussion_r2854630438 https://github.com/langflow-ai/langflow/pull/11689#discussion_r2854630446 * chore: OTel span naming convention model name is now set using name = f"{operation} {model_name}" if model_name else operation * add traces * feat: use uv sources for CPU-only PyTorch (#11884) * feat: use uv sources for CPU-only PyTorch Configure [tool.uv.sources] with pytorch-cpu index to avoid ~6GB CUDA dependencies in Docker images. This replaces hardcoded wheel URLs with a cleaner index-based approach. - Add pytorch-cpu index with explicit = true - Add torch/torchvision to [tool.uv.sources] - Add explicit torch/torchvision deps to trigger source override - Regenerate lockfile without nvidia/cuda/triton packages - Add required-environments for multi-platform support * fix: update regex to only replace name in [project] section The previous regex matched all lines starting with `name = "..."`, which incorrectly renamed the UV index `pytorch-cpu` to `langflow-nightly` during nightly builds. This caused `uv lock` to fail with: "Package torch references an undeclared index: pytorch-cpu" The new regex specifically targets the name field within the [project] section only, avoiding unintended replacements in other sections like [[tool.uv.index]]. * style: fix ruff quote style * fix: remove required-environments to fix Python 3.13 macOS x86_64 CI The required-environments setting was causing hard failures when packages like torch didn't have wheels for specific platform/Python combinations. Without this setting, uv resolves optimistically and handles missing wheels gracefully at runtime instead of failing during resolution. --------- * LE-270: Hydration and Console Log error (#11628) * LE-270: add fix hydration issues * LE-270: fix disable field on max token on language model --------- * test: add wait for selector in mcp server tests (#11883) * Add wait for selector in mcp server tests * [autofix.ci] apply automated fixes * Add more awit for selectors * [autofix.ci] apply automated fixes --------- * fix: reduce visual lag in frontend (#11686) * Reduce lag in frontend by batching react events and reducing minimval visual build time * Cleanup * [autofix.ci] apply automated fixes * add tests and improve code read * [autofix.ci] apply automated fixes * Remove debug log --------- * feat: lazy load imports for language model component (#11737) * Lazy load imports for language model component Ensures that only the necessary dependencies are required. For example, if OpenAI provider is used, it will now only import langchain_openai, rather than requiring langchain_anthropic, langchain_ibm, etc. * Add backwards-compat functions * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Add exception handling * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * comp index * docs: azure default temperature (#11829) * change-azure-openai-default-temperature-to-1.0 * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * [autofix.ci] apply automated fixes --------- * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix unit test? * add no-group dev to docker builds * [autofix.ci] apply automated fixes --------- * feat: generate requirements.txt from dependencies (#11810) * Base script to generate requirements Dymanically picks dependency for LanguageM Comp. Requires separate change to remove eager loading. * Lazy load imports for language model component Ensures that only the necessary dependencies are required. For example, if OpenAI provider is used, it will now only import langchain_openai, rather than requiring langchain_anthropic, langchain_ibm, etc. * Add backwards-compat functions * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Add exception handling * Add CLI command to create reqs * correctly exclude langchain imports * Add versions to reqs * dynamically resolve provider imports for language model comp * Lazy load imports for reqs, some ruff fixes * Add dynamic resolves for embedding model comp * Add install hints * Add missing provider tests; add warnings in reqs script * Add a few warnings and fix install hint * update comments add logging * Package hints, warnings, comments, tests * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Add alias for watsonx * Fix anthropic for basic prompt, azure mapping * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * ruff * [autofix.ci] apply automated fixes * test formatting * ruff * [autofix.ci] apply automated fixes --------- * fix: add handle to file input to be able to receive text (#11825) * changed base file and file components to support muitiple files and files from messages * update component index * update input file component to clear value and show placeholder * updated starter projects * [autofix.ci] apply automated fixes * updated base file, file and video file to share robust file verification method * updated component index * updated templates * fix whitespaces * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * add file upload test for files fed through the handle * [autofix.ci] apply automated fixes * added tests and fixed things pointed out by revies * update component index * fixed test * ruff fixes * Update component_index.json * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * updated component index * updated component index * removed handle from file input * Added functionality to use multiple files on the File Path, and to allow files on the langflow file system. * [autofix.ci] apply automated fixes * fixed lfx test * build component index --------- * docs: Add AGENTS.md development guide (#11922) * add AGENTS.md rule to project * change to agents-example * remove agents.md * add example description * chore: address cris I1 comment address cris I1 comment * chore: address cris I5 address cris I5 * chore: address cris I6 address cris I6 * chore: address cris R7 address cris R7 * fix testcase * chore: address cris R2 address cris R2 * restructure insight page into sidenav * added header and total run node * restructing branch * chore: address gab otel model changes address gab otel model changes will need no migration tables * chore: update alembic migration tables update alembic migration tables after model changes * add empty state for gropu sessions * remove invalid mock * test: update and add backend tests update and add backend tests * chore: address backend code rabbit comments address backend code rabbit comments * chore: address code rabbit frontend comments address code rabbit frontend comments * chore: test_native_tracer minor fix address c1 test_native_tracer minor fix address c1 * chore: address C2 + C3 address C2 + C3 * chore: address H1-H5 address H1-H5 * test: update test_native_tracer update test_native_tracer * fixes * chore: address M2 address m2 * chore: address M1 address M1 * dry changes, factorization * chore: fix 422 spam and clean comments fix 422 spam and clean comments * chore: address M12 address M12 * chore: address M3 address M3 * chore: address M4 address M4 * chore: address M5 address M5 * chore: clean up for M7, M9, M11 clean up for M7, M9, M11 * chore: address L2,L4,L5,L6 + any test address L2,L4,L5 and L6 + any test * chore: alembic + comment clean up alembic + comment clean up * chore: remove depricated test_traces file remove depricated test_traces file. test have all been moved to test_traces_api.py * fix datetime * chore: fix test_trace_api ge=0 is allowed now fix test_trace_api ge=0 is allowed now * chore: remove unused traces cost flow remove unused traces cost flow * fix traces test * fix traces test * fix traces test * fix traces test * fix traces test * chore: address gabriels otel coment address gabriels otel coment latest --------- Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local> Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com> Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com> Co-authored-by: cristhianzl <cristhian.lousa@gmail.com> Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com> Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Co-authored-by: Edwin Jose <edwin.jose@datastax.com> Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com> * fix(test): Fix superuser timeout test errors by replacing heavy clien… (#11982) fix(test): Fix superuser timeout test errors by replacing heavy client fixture (#11972) * fix super user timeout test error * fix fixture db test * remove canary test * [autofix.ci] apply automated fixes * flaky test --------- Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * refactor(components): Replace eager import with lazy loading in agentics module (#11974) Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: add ondelete=CASCADE to TraceBase.flow_id to match migration (#12002) * fix: add ondelete=CASCADE to TraceBase.flow_id to match migration The migration file creates the trace table's flow_id foreign key with ondelete="CASCADE", but the model was missing this parameter. This mismatch caused the migration validator to block startup. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add defensive migration to ensure trace.flow_id has CASCADE Adds a migration that ensures the trace.flow_id foreign key has ondelete=CASCADE. While the original migration already creates it with CASCADE, this provides a safety net for any databases that may have gotten into an inconsistent state. * fix: dynamically find FK constraint name in migration The original migration did not name the FK constraint, so it gets an auto-generated name that varies by database. This fix queries the database to find the actual constraint name before dropping it. --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix: LE-456 - Update ButtonSendWrapper to handle building state and improve button functionality (#12000) * fix: Update ButtonSendWrapper to handle building state and improve button functionality * fix(frontend): rename stop button title to avoid Playwright selector conflict The "Stop building" title caused getByRole('button', { name: 'Stop' }) to match two elements, breaking Playwright tests in shards 19, 20, 22, 25. Renamed to "Cancel" to avoid the collision with the no-input stop button. * Fix: pydantic fail because output is list, instead of a dict (#11987) pydantic fail because output is list, instead of a dict Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local> * refactor: Update guardrails icons (#12016) * Update guardrails.py Changing the heuristic threshold icons. The field was using the default icons. I added icons related to the security theme. * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com> * feat: Clean up the modelinput unification * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Update test_embedding_model_component.py * [autofix.ci] apply automated fixes * Revert to main for other files * More reversions * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Handle first run more elegantly in astra * [autofix.ci] apply automated fixes * Fix knowledge embedding dialog (#12071) * fix: Handle message inputs when ingesting knowledge * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Update test_ingestion.py * [autofix.ci] apply automated fixes * fix: Unify the knowledge creation model selector * Revert tracing * Update ingestion.py * Rebuild comp index * [autofix.ci] apply automated fixes * Update test_ingestion.py * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Update test_ingestion.py * Update component_index.json * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * [autofix.ci] apply automated fixes * Update comp index * Update test_astradb_base_component.py * Update Knowledge Ingestion.json * [autofix.ci] apply automated fixes * Fix broken tests * Cleanup from claude * [autofix.ci] apply automated fixes * Fix failing tests * Update test_unified_models.py * [autofix.ci] apply automated fixes * Update Nvidia Remix.json * Refactor ingest * Rebuild templates and component index * Fix test * [autofix.ci] apply automated fixes * Update component_index.json * [autofix.ci] apply automated fixes * test: add update_build_config visibility tests and PR review fixes (#12114) - Add update_build_config field-visibility tests to LanguageModelComponent, ToolCallingAgentComponent, and BatchRunComponent covering Ollama, WatsonX, OpenAI, and no-model-selected cases - Remove 16 stale @pytest.mark.skip tests from test_agent_component.py - Wire up validate_model_selection in agent.py for early input validation - Document AstraDB intentional use of lower-level update_model_options_in_build_config - Clarify model_kwargs info text to note provider-specific support Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * Update embedding_model.py * fix: address PR review recommendations for feat-unify-models++ (#12116) - Fix 9 skipped tests in test_batch_run_component.py by replacing model list with _MockLLM instances, following the existing pattern used by test_with_config_failure_handling - Fix test_agent_component.py: set component.model to a valid list before calling get_agent_requirements() in the three max_tokens tests, since validate_model_selection now requires a list-format model - Replace os.environ direct reads in apply_provider_variable_config_to_build_config with get_all_variables_for_provider() (DB-first, env fallback), and pass user_id through from handle_model_input_update - Add deprecated stubs for update_provider_fields_visibility, _update_watsonx_fields, and _update_ollama_fields in model_config.py with DeprecationWarning pointing to handle_model_input_update - Fix typo: "deault" -> "default" in structured_output.py TODO comment - Add 4 new KnowledgeIngestionComponent tests: new-format model_selection metadata path, allow_duplicates=True, missing metadata file error, and _build_embedding_metadata without API key Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * Ruff errors * Update test_ingestion.py * Update component index * Test updates * Update component_index.json * Update stable_hash_history.json * Template updates * Update batch_run.py * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * Update Youtube Analysis.json * Fix tests * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * Some cleanup and refactoring * [autofix.ci] apply automated fixes * Update Nvidia Remix.json * Update Nvidia Remix.json * Update unified_models.py * Coderabbit AI review comments * Component index update * [autofix.ci] apply automated fixes * Template updates * [autofix.ci] apply automated fixes * Template update * [autofix.ci] apply automated fixes * Review comments addressed * [autofix.ci] apply automated fixes * Update component_index.json * Update stable_hash_history.json * [autofix.ci] apply automated fixes * Test updates * Update test_ingestion.py * Update test_ingestion.py * Update test_ingestion.py * [autofix.ci] apply automated fixes * More clear tooltip text * [autofix.ci] apply automated fixes * Template updates * Index and templates * [autofix.ci] apply automated fixes * Fix lambda build * Template updates * Rebuild comp index * [autofix.ci] apply automated fixes * Fix templates * Fix failing test * Update templates * Update comp index * [autofix.ci] apply automated fixes * API key field in astra db * Update starter * Update comp index * Starter proj update * Add api key to field order * Update test_unified_models.py * Update test_unified_models.py * [autofix.ci] apply automated fixes * Update setup.py * Update setup.py * Update component_index.json * [autofix.ci] apply automated fixes * Return embedding models directly in KB * [autofix.ci] apply automated fixes * Update component_index.json * fix: Refactor the unified models code * Ruff checks * Update flow_preparation.py * [autofix.ci] apply automated fixes * Update test_language_model_component.py * fix: prevent overwriting user-selected global variables in provider c… (#12217) * fix: nightly now properly gets 1.9.0 branch (#12215) before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$' * docs: add search icon (#12216) add-back-svg * fix: prevent overwriting user-selected global variables in provider config Previously, the apply_provider_variable_config_to_build_config function would automatically overwrite field values with environment variable keys whenever an env var was present, even if the user had already selected a different global variable. This fix adds a check to only auto-set the environment variable if: - The field is currently empty, OR - The field is not already configured to load from the database This preserves user selections while still providing automatic configuration for new/empty fields. Added comprehensive unit tests to verify: - Auto-setting env vars for empty fields - Preserving user-selected global variables - Overwriting hardcoded values (expected behavior) - Skipping when env var is not set - Applying component metadata correctly * [autofix.ci] apply automated fixes * style: use dictionary comprehension instead of for-loop Fixed PERF403 Ruff style warning by replacing for-loop with dictionary comprehension in update_projects_components_with_latest_component_versions * chore: retrigger CI build * test: improve test coverage and clarity for provider config - Renamed test_apply_provider_config_overwrites_hardcoded_value to test_apply_provider_config_replaces_hardcoded_with_env_var for clarity - Added test_apply_provider_config_idempotent_when_already_set to document idempotent behavior when value already matches env var key - Removed sensitive value from debug log message to prevent potential exposure of API keys or credentials These changes improve test coverage by documenting the no-op scenario and enhance security by avoiding logging of potentially sensitive data. * chore: retrigger CI build --------- Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com> Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com> Co-authored-by: Steve Haertel <shaertel@ca.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> * Update build_config.py * [autofix.ci] apply automated fixes * Update build_config.py * Fix tests * fix: Dropdown issue with field population * Update test_unified_models.py * Clean up key config * [autofix.ci] apply automated fixes * fix tests * Fix tests * fix: Update tests * Update tests * Update test_tool_calling_agent.py * Update test_unified_models.py * Update test_tool_calling_agent.py * Update tests * Google AI generative embeddings fixes * [autofix.ci] apply automated fixes * Merge release branch * Template update * Merge release branch * [autofix.ci] apply automated fixes * Update openai_constants.py * Update openai_constants.py --------- Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com> Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai> Co-authored-by: keval shah <kevalvirat@gmail.com> Co-authored-by: Antônio Alexandre Borges Lima <104531655+AntonioABLima@users.noreply.github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local> Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com> Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com> Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com> Co-authored-by: cristhianzl <cristhian.lousa@gmail.com> Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com> Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Co-authored-by: Edwin Jose <edwin.jose@datastax.com> Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com> Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com> Co-authored-by: Lucas Democh <ldgoularte@gmail.com> Co-authored-by: Steve Haertel <stevehaertel@users.noreply.github.com> Co-authored-by: Steve Haertel <shaertel@ca.ibm.com> |
|||
| 7d4ffbcbf5 |
feat: add support for Langchain 1.0 (#11114)
* feat: upgrade to LangChain 1.0 - langchain ~=1.2.0 - langchain-core ~=1.2.3 - langchain-community ~=0.4.1 Updated all langchain-* integration packages to versions compatible with langchain-core 1.0+. * feat(lfx): add langchain-classic dependency for legacy agent classes LangChain 1.0 removed AgentExecutor and related classes to langchain-classic. This adds the dependency to maintain backward compatibility. * refactor(lfx): update imports for LangChain 1.0 compatibility - Move AgentExecutor, agent creators from langchain to langchain_classic - Move AsyncCallbackHandler from langchain.callbacks to langchain_core.callbacks - Move Chain, BaseChatMemory from langchain to langchain_classic - Update LANGCHAIN_IMPORT_STRING for code generation * fix(lfx): make sqlalchemy import lazy in session_scope LangChain 1.0 no longer includes sqlalchemy as a transitive dependency. Move the import inside the function where it's used to avoid import errors when sqlalchemy is not installed. * chore: update uv.lock for langchain-classic * feat: enable nv-ingest optional dependencies for langchain 1.0 - Uncomment nv-ingest-api and nv-ingest-client, update to >=26.1.0 (no longer has openai version conflict) - Bump datasets from <4.0.0 to <5.0.0 to allow fsspec>=2025.5.1 required by nv-ingest - Update mlx-vlm TODO comment with accurate blocking reason * chore: update nv-ingest to 26.1.1 nv-ingest 26.1.1 removes the openai dependency, resolving the conflict with langchain-openai>=1.0.0 (which requires openai>=1.109.1). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: enable mlx and mlx-vlm dependencies for langchain 1.0 opencv-python 4.13+ now supports numpy>=2, resolving the conflict with langchain-aws>=1.0.0 (which requires numpy>=2.2 on Python 3.12+). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: fix import order in callback.py Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: update imports to use langchain_classic for agent modules * [autofix.ci] apply automated fixes * fix: remove .item() calls in knowledge_bases.py * fix(lfx): import BaseMemory from langchain_classic for langchain 1.0 * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * refactor: update deprecated langchain imports for langchain 1.0 - langchain.callbacks.base -> langchain_core.callbacks.base - langchain.tools -> langchain_core.tools - langchain.schema -> langchain_core.messages/documents - langchain.chains -> langchain_classic.chains - langchain.retrievers -> langchain_classic.retrievers - langchain.memory -> langchain_classic.memory - langchain.globals -> langchain_core.globals - langchain.docstore -> langchain_core.documents - langchain.prompts -> langchain_core.prompts Also simplified GoogleGenerativeAIEmbeddingsComponent to use native langchain-google-genai 4.x which now supports output_dimensionality. * [autofix.ci] apply automated fixes * fix: add _to_int helper for pandas sum() compatibility across Python versions * fix: update langfuse>=3.8.0 and fix cuga_agent.py (#11519) * fix: update langfuse>=3.8.0 and fix cuga_agent.py * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * feat: implement LangFuseTracer for langfuse v3 API compatibility and add unit tests * fix: upgrade cuga to 0.2.9 for langchain 1.0 compatibility * fix: improve error handling and return value in get_langchain_callback method * fix: update package versions for compatibility and improvements * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * [autofix.ci] apply automated fixes (attempt 3/3) * fix: update langwatch dependency to version 0.10.0 for compatibility * fix: update environment variable for Langfuse host to LANGFUSE_BASE_URL * Update dependency versions in Youtube Analysis project - Downgraded googleapiclient from 2.188.0 to 2.154.0 - Updated langchain_core from 1.2.7 to 1.2.9 - Updated fastapi from 0.128.1 to 0.128.5 - Downgraded youtube_transcript_api from 1.2.4 to 1.2.3 - Changed langchain_core version from 1.2.7 to 0.3.81 - Cleared input_types in model selection # Conflicts: # src/backend/base/langflow/initial_setup/starter_projects/Basic Prompt Chaining.json # src/backend/base/langflow/initial_setup/starter_projects/Basic Prompting.json # src/backend/base/langflow/initial_setup/starter_projects/Blog Writer.json # src/backend/base/langflow/initial_setup/starter_projects/Custom Component Generator.json # src/backend/base/langflow/initial_setup/starter_projects/Document Q&A.json # src/backend/base/langflow/initial_setup/starter_projects/Financial Report Parser.json # src/backend/base/langflow/initial_setup/starter_projects/Hybrid Search RAG.json # src/backend/base/langflow/initial_setup/starter_projects/Image Sentiment Analysis.json # src/backend/base/langflow/initial_setup/starter_projects/Instagram Copywriter.json # src/backend/base/langflow/initial_setup/starter_projects/Invoice Summarizer.json # src/backend/base/langflow/initial_setup/starter_projects/Knowledge Retrieval.json # src/backend/base/langflow/initial_setup/starter_projects/Market Research.json # src/backend/base/langflow/initial_setup/starter_projects/Meeting Summary.json # src/backend/base/langflow/initial_setup/starter_projects/Memory Chatbot.json # src/backend/base/langflow/initial_setup/starter_projects/News Aggregator.json # src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json # src/backend/base/langflow/initial_setup/starter_projects/Pokédex Agent.json # src/backend/base/langflow/initial_setup/starter_projects/Portfolio Website Code Generator.json # src/backend/base/langflow/initial_setup/starter_projects/Price Deal Finder.json # src/backend/base/langflow/initial_setup/starter_projects/Research Agent.json # src/backend/base/langflow/initial_setup/starter_projects/Research Translation Loop.json # src/backend/base/langflow/initial_setup/starter_projects/SEO Keyword Generator.json # src/backend/base/langflow/initial_setup/starter_projects/SaaS Pricing.json # src/backend/base/langflow/initial_setup/starter_projects/Search agent.json # src/backend/base/langflow/initial_setup/starter_projects/Sequential Tasks Agents.json # src/backend/base/langflow/initial_setup/starter_projects/Simple Agent.json # src/backend/base/langflow/initial_setup/starter_projects/Social Media Agent.json # src/backend/base/langflow/initial_setup/starter_projects/Text Sentiment Analysis.json # src/backend/base/langflow/initial_setup/starter_projects/Travel Planning Agents.json # src/backend/base/langflow/initial_setup/starter_projects/Twitter Thread Generator.json # src/backend/base/langflow/initial_setup/starter_projects/Vector Store RAG.json # src/backend/base/langflow/initial_setup/starter_projects/Youtube Analysis.json * fix: handle InvalidRequestError during session rollback * update projects * ⚡️ Speed up method `LangFuseTracer.end` by 141% in PR #11114 (`feat/langchain-1.0`) (#11682) Optimize LangFuseTracer.end The optimized code achieves a **140% speedup** (8.23ms → 3.42ms) through two complementary optimizations: ## 1. Fast-path for Common Primitives in `serialize()` **What changed:** Added an early-exit check that returns immutable primitives (`str`, `int`, `float`, `bool`) directly when no truncation or special handling is needed: ```python if max_length is None and max_items is None and not to_str: if isinstance(obj, (str, int, float, bool)): return obj ``` **Why it's faster:** - The profiler shows `_serialize_dispatcher()` consumed **81.5% of runtime** in the original code (40.4ms out of 49.6ms) - This optimization reduced dispatcher calls from **8,040 to 1,013** (~87% reduction), as primitives now bypass the expensive pattern-matching dispatcher entirely - The fast-path check itself is extremely cheap: just two quick conditionals and an `isinstance()` check against a tuple of built-in types **When it helps:** This optimization is particularly effective for workloads with many primitive values in dictionaries and lists—which is exactly what the tracing use case provides (metadata dicts with strings, numbers, booleans). ## 2. Eliminate Redundant Serialization in `LangFuseTracer.end()` **What changed:** Serialize `inputs`, `outputs`, and `metadata` once each, then reuse the results: ```python inputs_ser = serialize(inputs) outputs_ser = serialize(outputs) metadata_ser = serialize(metadata) if metadata else None ``` **Why it's faster:** - The original code called `serialize()` **6 times total** (3 for `.update()` + 3 for `.update_trace()`) - The optimized version calls it **3 times**, then passes the cached results - Profiler shows the time spent in `serialize()` calls dropped from **72.2ms to 31.2ms** (~57% reduction) - This is pure elimination of redundant work—the same dictionaries were being serialized twice with identical results **Impact on workloads:** The `test_end_multiple_iterations_calls_end_each_time` test (500 iterations) demonstrates this matters in hot paths. If `LangFuseTracer.end()` is called frequently during flow execution, avoiding duplicate serialization provides compounding benefits. ## Combined Effect Both optimizations target the serialization bottleneck from different angles: - The fast-path reduces the cost of *each* serialize call by ~75% for primitive-heavy data - The caching reduces the *number* of serialize calls by 50% Together, they deliver the observed 140% speedup, with the optimization being especially effective for the common case of metadata dictionaries containing mostly primitive types. Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com> * refactor: reorder imports and simplify serialization logic for primitives * [autofix.ci] apply automated fixes * fix: update google dependency version to 0.4.0 in component_index.json * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix: pin z3-solver<4.15.7 to restore Linux wheels for Docker build z3-solver 4.15.7 dropped manylinux wheels, causing the Docker build to fail when trying to compile from source. Temporary pin until codeflash is removed. * [autofix.ci] apply automated fixes * fix: update google dependency version to 0.4.0 * [autofix.ci] apply automated fixes * fix: update langchain_core version to 1.2.17 in multiple starter project JSON files * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * fix: update deprecated langchain imports to langchain_classic for 1.0 compatibility * fix: align langchain-chroma version in optional chroma dependency group * fix: nightly now properly gets 1.9.0 branch (#12215) before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$' * docs: add search icon (#12216) add-back-svg * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * feat: fall back to langchain_classic for pre-1.0 imports in user components Old flows using removed langchain imports (e.g. langchain.memory, langchain.schema, langchain.chains) now resolve via langchain_classic at two levels: module-level for entirely removed modules, and attribute-level for removed attributes in modules that still exist in langchain 1.0. New langchain 1.0 imports are never affected since fallbacks only trigger on import failure. * urllib parse module import bug * Update component_index.json * [autofix.ci] apply automated fixes * chore: rebuild component index * [autofix.ci] apply automated fixes --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Harold Ship <harold.ship@gmail.com> Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com> Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com> Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
|||
| f46ff2ab72 |
feat: Add Windows Playwright testing to nightly builds (#12221)
* feat: add Windows support for Playwright tests in nightly builds - Add windows-latest as runner option in typescript_test.yml - Update Playwright browser installation to be OS-aware (Windows doesn't support --with-deps) - Add matrix strategy to nightly_build.yml to run tests on both Linux and Windows - Update Slack notifications to indicate multi-platform testing - Tests now run in parallel on ubuntu-latest and windows-latest This enables catching Windows-specific regressions early in the nightly build process. * fix: add shell: bash to all script steps for Windows compatibility Windows runners default to PowerShell which doesn't understand bash syntax. All steps with bash scripts now explicitly specify shell: bash to ensure cross-platform compatibility. * feat: make Windows Playwright tests non-blocking initially Add continue-on-error for Windows tests to allow nightly builds to succeed even if Windows-specific issues are found. This gives us visibility into Windows bugs without blocking releases. Windows tests will still run every night and report failures, but won't block the build. Once Windows tests are stable, we can remove this flag. * chore: fix white space chore: fix white space * fix: replace matrix strategy with separate jobs for Windows Playwright tests - GitHub Actions reusable workflows don't support matrix strategy - Split frontend-tests into frontend-tests-linux and frontend-tests-windows - Windows tests are non-blocking (continue-on-error: true) - Updated all job dependencies and Slack notification logic - Addresses PR review comment about matrix limitation * chore: trigger workflow validation refresh --------- Co-authored-by: Adam Aghili <Adam.Aghili@ibm.com> |
|||
| 288719d4ff |
fix: prevent overwriting user-selected global variables in provider c… (#12217)
* fix: nightly now properly gets 1.9.0 branch (#12215) before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$' * docs: add search icon (#12216) add-back-svg * fix: prevent overwriting user-selected global variables in provider config Previously, the apply_provider_variable_config_to_build_config function would automatically overwrite field values with environment variable keys whenever an env var was present, even if the user had already selected a different global variable. This fix adds a check to only auto-set the environment variable if: - The field is currently empty, OR - The field is not already configured to load from the database This preserves user selections while still providing automatic configuration for new/empty fields. Added comprehensive unit tests to verify: - Auto-setting env vars for empty fields - Preserving user-selected global variables - Overwriting hardcoded values (expected behavior) - Skipping when env var is not set - Applying component metadata correctly * [autofix.ci] apply automated fixes * style: use dictionary comprehension instead of for-loop Fixed PERF403 Ruff style warning by replacing for-loop with dictionary comprehension in update_projects_components_with_latest_component_versions * chore: retrigger CI build * test: improve test coverage and clarity for provider config - Renamed test_apply_provider_config_overwrites_hardcoded_value to test_apply_provider_config_replaces_hardcoded_with_env_var for clarity - Added test_apply_provider_config_idempotent_when_already_set to document idempotent behavior when value already matches env var key - Removed sensitive value from debug log message to prevent potential exposure of API keys or credentials These changes improve test coverage by documenting the no-op scenario and enhance security by avoiding logging of potentially sensitive data. * chore: retrigger CI build --------- Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com> Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com> Co-authored-by: Steve Haertel <shaertel@ca.ibm.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Hare <ericrhare@gmail.com> |
