* feat: Add DB migration validation workflow for nightly builds (LE-1259)
- Implements automated DB migration testing for nightly builds
- Tests two scenarios: pip/venv and Docker Compose migrations
- Validates migration from stable to nightly versions
- Verifies data persistence (witness flows) across migrations
- Integrated into nightly_build.yml workflow
- Includes Slack notifications for migration test results
This addresses the critical blocker identified by QA team for ensuring
safe database migrations in production deployments.
* docs: Add DB migration validation documentation
- Comprehensive guide for LE-1259 implementation
- Detailed test scenarios and execution details
- Environment configuration and success criteria
- Monitoring and troubleshooting guidelines
- Placed in docs/docs/Deployment/ for easy access
* fix: address PR review comments for DB migration validation
- Use actual nightly tag from create-nightly-tag output instead of hardcoded :latest
- Wire POSTGRES_VERSION env var into postgres service image tag
- Remove unnecessary checkout steps from both migration jobs
- Update step name from 'Create witness flow and credentials' to 'Create witness flow'
- Add -f flag to curl commands for fail-fast behavior
- Add flow creation verification with error handling
- Fix version extraction logic to properly test nightly build instead of PyPI latest
- Remove deprecated docker-compose version field
- Remove duplicate Slack notification job (consolidated in nightly_build.yml)
Addresses all 9 issues identified by @ogabrielluiz in PR review
* docs: remove implementation summary from user-facing docs
Per @ogabrielluiz review feedback, this file reads as an implementation
summary (Jira ticket, branch name, 'Next Steps', 'Files Changed: 2')
rather than user-facing documentation. The Deployment section is for
end-user docs, and this content is better suited for the PR description.
Also not added to sidebars.js, so would be an orphan page.
* fix(workflows): address 6 issues from Gabriel's second review of DB migration validation
Fixes all remaining issues identified in PR #13249 review:
1. Remove schedule trigger - only works on default branch, would cause duplicate runs
2. Fix postgres service image - hardcode to postgres:16 (env context not available in services)
3. Add curl fail-fast flags - use -fsSL for immediate failure on errors
4. Add flow ID verification - check witness data creation succeeded before proceeding
5. Remove orphaned Slack JSON - cleanup leftover from removed notify-results job
6. Fix version extraction - strip 'v' prefix for pip install (${VERSION#v})
7. Fix Docker image reference in nightly_build.yml - pass full image path with tag
8. Fix success notification - check migration validation didn't fail
All changes validated locally:
- YAML syntax validation passed
- Docker Compose config validated
- Tag manipulation logic tested (v prefix handling)
- Curl command structure verified
- Test credentials marked with pragma allowlist secret comments
Related: LE-1259, PR #13249
Depends on: PR #13212 (Docker volume permissions fix)
* refactor(settings): split monolithic Settings into per-domain group mixins
Move the ~70-field Settings class from one 755-line base.py into 13 cohesive
BaseModel mixins under lfx/services/settings/groups/ (paths, server, database,
cache, storage, mcp, telemetry, observability, security, components, ui,
runtime, variables). Settings now composes them via multiple inheritance.
Inheritance order is chosen so cross-group validators see their dependencies
in info.data: PathSettings rightmost (config_dir before database_url),
ServerSettings just left of it (workers before event_delivery).
No env var or call-site changes. BASE_COMPONENTS_PATH re-export preserved for
tests that import it transitively.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* docs(settings): fix wrong mcp_server_timeout docstring and idle-timeout comment
mcp_server_timeout's docstring was copy-pasted from a database setting and
mentioned 'lock to released' / 'database connection'. Replace with text that
describes the actual field.
mcp_session_idle_timeout's comment said 'Defaults to 5 minutes' but 400s is
~6.7 minutes. Drop the misleading 'minutes' claim and keep the value.
* test(settings): add structural safety tests for the group composition
Adds 26 tests to guard the refactor:
- All 105 fields that lived on the monolithic Settings still exist on the
composed class. A missing group in the inheritance list trips this loudly.
- A sampling of critical scalar and dict defaults (host, port, workers,
cache_type, sqlite_pragmas, db_connection_settings, etc.) are byte-for-byte
unchanged.
- Cross-group validator dependencies still resolve via info.data:
workers > 1 forces event_delivery=direct (ServerSettings -> RuntimeSettings)
and database_url falls back to a sqlite path under config_dir without
raising 'config_dir not set' (PathSettings -> DatabaseSettings).
- A parametrized sweep verifies a representative set of LANGFLOW_* env vars
still populate their fields.
- Back-compat exports (CustomSource, is_list_of_any, yaml helpers,
BASE_COMPONENTS_PATH) are still importable from settings.base.
- update_settings handles scalars and list-with-no-duplicates correctly.
- save_settings_to_yaml round-trips without error.
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
fix(ci): narrow --prerelease=allow to wheels in cross-platform install tests
Nightly cross-platform install tests were failing on Python 3.12/3.13 with
a pydantic ValidationError at langchain_core import time:
File ".../langchain_core/runnables/passthrough.py", line 349, in <module>
_graph_passthrough: RunnablePassthrough = RunnablePassthrough()
pydantic_core._pydantic_core.ValidationError: 1 validation error for
RunnablePassthrough
name
Field required ...
Cause: --prerelease=allow was pulling pydantic 2.14.0a1 as a transitive
dep, which is incompatible with langchain-core 1.4.0's top-level
RunnablePassthrough() initialization.
We need pre-releases enabled for the nightly langflow* .dev wheels we
pass on the command line, but not for transitive deps. Switch to
--prerelease=if-necessary-or-explicit, which keeps the explicit .dev
wheel installs working while resolving stable pydantic for everything
else.
The nightly pipeline renames the workspace `lfx` package to `lfx-nightly`
in `src/lfx/pyproject.toml` and the root workspace dep, but leaves each
`src/bundles/*/pyproject.toml`'s `"lfx>=X.Y,<Z"` pin unchanged. With the
workspace `lfx` gone and PyPI still on lfx 0.4.3, the create-nightly-tag
job's `uv lock` fails:
× No solution found when resolving dependencies for split [...]
╰─▶ Because only lfx<=0.4.3 is available and lfx-arxiv depends on
lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv's requirements
are unsatisfiable.
`update_lfx_version.py` now also rewrites the `lfx` (or already-rewritten
`lfx-nightly`) specifier in every `src/bundles/*/pyproject.toml` to
`lfx-nightly==<dev version>`, mirroring how `update_lf_base_dependency`
re-pins the lfx dep inside `langflow-base`. The lookahead in the regex
prevents matching sibling packages like `lfx-arxiv` / `lfx-duckduckgo`.
No-op when no bundle pyprojects exist (e.g. on main today), so this is
safe to merge ahead of the bundle extraction landing on main. Restored
the guarded `git add src/bundles/*/pyproject.toml` step so the modified
bundle pyprojects ride the same commit/tag as the rest of the nightly
version bumps.
Cherry-pick to release-1.10.0 to unblock nightlies cut from that branch.
The release_bundles.yml smoke test installs bundle wheels into a fresh
venv, which makes uv resolve their `lfx>=X.Y,<Z` pin against PyPI. When
bundles are released ahead of a matching lfx (typical case — bundles
ship more often than lfx bumps land on PyPI), the resolve fails:
Because only lfx<=0.4.3 is available and lfx-arxiv==0.1.0 depends on
lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv==0.1.0 cannot be
used.
Build the lfx wheel from the current ref in the build-bundles job and
install it alongside the bundle wheels in the smoke test. The lfx
wheel ships as a separate `dist-lfx-smoketest` artifact and is NOT
included in the publish-bundles step — only the `dist-bundles`
artifact gets pushed to PyPI.
Bundles (lfx-arxiv, lfx-duckduckgo) change infrequently enough that a
nightly cadence is overkill. Drop the rename-to-`-nightly` and bundle
publish lanes from the nightly pipeline and add a dedicated
workflow_dispatch-only release_bundles.yml for purposeful releases.
- nightly_build.yml: drop update_bundle_versions.py invocation and the
src/bundles/*/pyproject.toml git-add guard.
- release_nightly.yml: remove build-nightly-bundles and
publish-nightly-bundles jobs; untether test-cross-platform and
publish-nightly-main from them.
- release_bundles.yml (new): build all src/bundles/* wheels, run a
cross-OS install smoke test, then publish to PyPI under their stable
names. Tolerates already-published versions so re-runs without a
version bump are no-ops.
release.yml still owns bundle publishing as part of main releases.
Manual follow-up (PyPI admin): delete the lfx-arxiv-nightly and
lfx-duckduckgo-nightly projects from PyPI.
* chore: bump backend unit test timeout from 30 to 40 minutes
* ci: anchor langflow version grep so new workspace packages can't break it
The pattern `uv tree | grep 'langflow' | grep -v 'langflow-base' | grep -v 'langflow-sdk'`
was used in 8 places to pluck the root `langflow` (or `langflow-nightly`)
line out of `uv tree`. It relied on enumerating every other workspace
package that starts with `langflow-`. As soon as a new one was added
(`langflow-stepflow`, via #12015) the filter let two lines through and
the downstream `awk` / `cut` produced a multi-line string, e.g.:
Name langflow-stepflow
langflow-nightly does not match langflow-nightly. Exiting the workflow.
Replace the enumeration with an anchored regex
`grep -E '^langflow(-nightly)?[[:space:]]'` so only the exact `langflow`
or `langflow-nightly` root line matches, regardless of how many other
`langflow-*` workspace packages exist now or in the future.
* ci: build, test, and publish src/bundles/* wheels
Langflow main pins exact versions of every bundle (e.g.
`lfx-arxiv-nightly==0.1.0.dev38`, `lfx-arxiv==0.1.0`), but neither the
nightly workflow nor the release workflow built or published those
bundle wheels. So when cross-platform tests (or downstream consumers)
ran `pip install langflow-nightly`, the resolver couldn't find
`lfx-arxiv-nightly` / `lfx-duckduckgo-nightly` on PyPI and bailed:
Because lfx-arxiv-nightly was not found in the package registry and
langflow-nightly==1.10.0.dev38 depends on lfx-arxiv-nightly==0.1.0.dev38,
we can conclude that langflow-nightly==1.10.0.dev38 cannot be used.
Add a build → test → publish lane for every package under
`src/bundles/*/` in both workflows, plus a matching bundle install step
in cross-platform-test.yml.
`release_nightly.yml`
- `build-nightly-bundles`: iterates `src/bundles/*/pyproject.toml`,
builds each wheel, uploads as `dist-nightly-bundles`.
- `test-cross-platform` now passes `bundles-artifact-name` so the
cross-platform test installs bundles before installing main.
- `publish-nightly-bundles`: publishes each bundle wheel to PyPI after
cross-platform passes and lfx is already published.
- `publish-nightly-main` now waits on `publish-nightly-bundles` so main
reaches PyPI only after its bundle deps do.
`release.yml`
- New `release_bundles` input (false by default).
- `validate-dependencies` rejects `release_package_main=true` without
`release_bundles=true`, mirroring the existing lfx/sdk gate.
- `build-bundles`, `publish-bundles` mirror the nightly jobs.
- `publish-main` now needs `publish-bundles`.
`cross-platform-test.yml`
- New optional `bundles-artifact-name` input; download step is gated on
it being set.
- `build-if-needed` (workflow_dispatch path) also builds bundles and
publishes the artifact, so adhoc runs stay self-contained.
- Each of the four "Install main package from wheel" steps gets a
preceding "Install bundle packages from wheel" step that runs only
when the artifact is present, leaving non-bundle callers
(e.g. pre-bundle release branches) unaffected.
* ci: drop mapfile from bundle install (macOS Bash 3.2 compat)
`mapfile` is a Bash 4+ builtin and the macOS runners use the system
Bash (3.2). The "Install bundle packages from wheel (Unix)" step
failed on `Install & Run - macos amd64 3.12` with:
mapfile: command not found
find: stdout: Broken pipe
Replace with the `shopt -s nullglob` + glob-array pattern already used
in the bundle build/publish steps, which works on Bash 3.2. All
artifact wheels live at the top level of ./bundles-dist, so the simple
glob is equivalent to the previous `find` (which also wasn't recursing
anywhere useful).
---------
Co-authored-by: Claude <noreply@anthropic.com>
Splits monolithic agent instructions into a slim AGENTS.md hub plus
docs/agents/ topic files (philosophy, architecture, components, contracts,
testing, anti-patterns) so AI coding agents have a single source of truth
for how the project thinks. Removes the .cursor/rules/ directory; AGENTS.md
is now the only agent-facing doc.
Fixes the long-standing pointer to the empty src/backend/base/langflow/components/
stubs (real components live in src/lfx/src/lfx/components/) and corrects
the v2 API description from 'future' to its actual mounted state.
