Files
langflow/.github/workflows/release_nightly.yml
Eric Hare 5a8c17b22d ci: backport bundle/nightly CI fixes from main (#13206, #13207, #13208) (#13210)
* ci: stop publishing -nightly bundle variants; release bundles on demand (#13206)

Bundles (lfx-arxiv, lfx-duckduckgo) change infrequently enough that a
nightly cadence is overkill. Drop the rename-to-`-nightly` and bundle
publish lanes from the nightly pipeline and add a dedicated
workflow_dispatch-only release_bundles.yml for purposeful releases.

- nightly_build.yml: drop update_bundle_versions.py invocation and the
  src/bundles/*/pyproject.toml git-add guard.
- release_nightly.yml: remove build-nightly-bundles and
  publish-nightly-bundles jobs; untether test-cross-platform and
  publish-nightly-main from them.
- release_bundles.yml (new): build all src/bundles/* wheels, run a
  cross-OS install smoke test, then publish to PyPI under their stable
  names. Tolerates already-published versions so re-runs without a
  version bump are no-ops.

release.yml still owns bundle publishing as part of main releases.

Manual follow-up (PyPI admin): delete the lfx-arxiv-nightly and
lfx-duckduckgo-nightly projects from PyPI.

* ci(release_bundles): build lfx wheel locally for smoke test (#13207)

The release_bundles.yml smoke test installs bundle wheels into a fresh
venv, which makes uv resolve their `lfx>=X.Y,<Z` pin against PyPI. When
bundles are released ahead of a matching lfx (typical case — bundles
ship more often than lfx bumps land on PyPI), the resolve fails:

  Because only lfx<=0.4.3 is available and lfx-arxiv==0.1.0 depends on
  lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv==0.1.0 cannot be
  used.

Build the lfx wheel from the current ref in the build-bundles job and
install it alongside the bundle wheels in the smoke test. The lfx
wheel ships as a separate `dist-lfx-smoketest` artifact and is NOT
included in the publish-bundles step — only the `dist-bundles`
artifact gets pushed to PyPI.

* ci(nightly): re-pin bundle lfx deps to lfx-nightly during nightly build (#13208)

The nightly pipeline renames the workspace `lfx` package to `lfx-nightly`
in `src/lfx/pyproject.toml` and the root workspace dep, but leaves each
`src/bundles/*/pyproject.toml`'s `"lfx>=X.Y,<Z"` pin unchanged. With the
workspace `lfx` gone and PyPI still on lfx 0.4.3, the create-nightly-tag
job's `uv lock` fails:

  × No solution found when resolving dependencies for split [...]
  ╰─▶ Because only lfx<=0.4.3 is available and lfx-arxiv depends on
      lfx>=0.5.0,<0.6.0, we can conclude that lfx-arxiv's requirements
      are unsatisfiable.

`update_lfx_version.py` now also rewrites the `lfx` (or already-rewritten
`lfx-nightly`) specifier in every `src/bundles/*/pyproject.toml` to
`lfx-nightly==<dev version>`, mirroring how `update_lf_base_dependency`
re-pins the lfx dep inside `langflow-base`. The lookahead in the regex
prevents matching sibling packages like `lfx-arxiv` / `lfx-duckduckgo`.

No-op when no bundle pyprojects exist (e.g. on main today), so this is
safe to merge ahead of the bundle extraction landing on main. Restored
the guarded `git add src/bundles/*/pyproject.toml` step so the modified
bundle pyprojects ride the same commit/tag as the rest of the nightly
version bumps.

Cherry-pick to release-1.10.0 to unblock nightlies cut from that branch.
2026-05-19 10:12:54 -07:00

509 lines
18 KiB
YAML

name: Langflow Nightly Build
run-name: Langflow Nightly Release by @${{ github.actor }}
on:
workflow_call:
inputs:
build_docker_base:
description: "Build Docker Image for Langflow Nightly Base"
required: true
type: boolean
default: false
build_docker_main:
description: "Build Docker Image for Langflow Nightly"
required: true
type: boolean
default: false
build_docker_ep:
description: "Build Docker Image for Langflow Nightly with Entrypoint"
required: false
type: boolean
default: false
build_lfx:
description: "Build and release LFX package"
required: false
type: boolean
default: false
nightly_tag_release:
description: "Tag for the nightly main build"
required: true
type: string
nightly_tag_base:
description: "Tag for the nightly base build"
required: true
type: string
nightly_tag_lfx:
description: "Tag for the nightly LFX build"
required: false
type: string
nightly_tag_sdk:
description: "Tag for the nightly SDK build"
required: false
type: string
push_to_registry:
description: "Whether to push images to registries. Set to false for testing builds without publishing."
required: true
type: boolean
default: false
env:
POETRY_VERSION: "1.8.3"
PYTHON_VERSION: "3.13"
jobs:
build-nightly-lfx:
name: Build LFX Nightly
if: ${{ inputs.build_lfx }}
runs-on: ubuntu-latest
outputs:
version: ${{ steps.verify.outputs.version }}
defaults:
run:
shell: bash
steps:
- name: Check out the code at a specific ref
uses: actions/checkout@v6
with:
ref: ${{ inputs.nightly_tag_release }}
persist-credentials: true
- name: "Setup Environment"
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
python-version: ${{ env.PYTHON_VERSION }}
prune-cache: false
- name: Install LFX dependencies
run: cd src/lfx && uv sync
- name: Verify SDK Nightly Name and Version
if: ${{ inputs.build_lfx }}
run: |
name=$(grep '^name = "' src/sdk/pyproject.toml | head -n 1 | sed 's/.*"\(.*\)".*/\1/')
version=$(grep '^version = "' src/sdk/pyproject.toml | head -n 1 | sed 's/.*"\(.*\)".*/\1/')
if [ "$name" != "langflow-sdk-nightly" ]; then
echo "Name $name does not match langflow-sdk-nightly. Exiting the workflow."
exit 1
fi
if [ "v$version" != "${{ inputs.nightly_tag_sdk }}" ]; then
echo "Version v$version does not match nightly tag ${{ inputs.nightly_tag_sdk }}. Exiting the workflow."
exit 1
fi
- name: Verify Nightly Name and Version
id: verify
run: |
cd src/lfx
name=$(uv tree | grep 'lfx' | head -n 1 | awk '{print $1}')
version=$(uv tree | grep 'lfx' | head -n 1 | awk '{print $2}')
if [ "$name" != "lfx-nightly" ]; then
echo "Name $name does not match lfx-nightly. Exiting the workflow."
exit 1
fi
if [ "$version" != "${{ inputs.nightly_tag_lfx }}" ]; then
echo "Version $version does not match nightly tag ${{ inputs.nightly_tag_lfx }}. Exiting the workflow."
exit 1
fi
# Strip the leading `v` from the version
version=$(echo $version | sed 's/^v//')
echo "version=$version" >> $GITHUB_OUTPUT
- name: Build langflow-sdk for distribution
run: |
make sdk_build args="--wheel"
- name: Build LFX for distribution
run: |
cd src/lfx
rm -rf dist/
uv build --wheel --out-dir dist
- name: Test LFX CLI
run: |
cd src/lfx
# Use a clean venv and install both wheels together so uv can resolve
# the local SDK dependency without falling back to PyPI.
uv venv test-env --seed
uv pip install --python ./test-env/bin/python --force-reinstall ../sdk/dist/*.whl dist/*.whl
./test-env/bin/lfx --help
echo "LFX CLI test completed successfully"
# PyPI publishing moved to after cross-platform testing
- name: Upload langflow-sdk Artifact
uses: actions/upload-artifact@v6
with:
name: dist-nightly-sdk
path: src/sdk/dist
- name: Upload LFX Artifact
uses: actions/upload-artifact@v6
with:
name: dist-nightly-lfx
path: src/lfx/dist
build-nightly-base:
name: Build Langflow Nightly Base
needs: [build-nightly-lfx]
if: ${{ always() && (needs.build-nightly-lfx.result == 'success' || inputs.build_lfx == false) }}
runs-on: ubuntu-latest
defaults:
run:
shell: bash
outputs:
version: ${{ steps.verify.outputs.version }}
skipped: ${{ steps.verify.outputs.skipped }}
steps:
- name: Check out the code at a specific ref
uses: actions/checkout@v6
with:
ref: ${{ inputs.nightly_tag_release }}
persist-credentials: true
- name: "Setup Environment"
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
python-version: ${{ env.PYTHON_VERSION }}
prune-cache: false
- name: Download LFX artifact
if: inputs.build_lfx
uses: actions/download-artifact@v7
with:
name: dist-nightly-lfx
path: lfx-dist
- name: Install the project
run: uv sync
- name: Force reinstall LFX from built wheel
if: inputs.build_lfx
run: |
echo "Installing LFX from built wheel to test the actual distribution package..."
# While workspace resolution installs from source, we want to test the exact
# wheel that will be published to PyPI to catch any packaging issues
uv pip install --force-reinstall --no-deps lfx-dist/*.whl
- name: Verify Nightly Name and Version
id: verify
run: |
name=$(uv tree | grep 'langflow-base' | awk '{print $2}' | head -n 1)
version=$(uv tree | grep 'langflow-base' | awk '{print $3}' | head -n 1)
# Strip extras from package name (e.g., "langflow-base-nightly[complete]" -> "langflow-base-nightly")
name_without_extras=$(echo $name | sed 's/\[.*\]//')
if [ "$name_without_extras" != "langflow-base-nightly" ]; then
echo "Name $name_without_extras does not match langflow-base-nightly. Exiting the workflow."
exit 1
fi
if [ "$version" != "${{ inputs.nightly_tag_base }}" ]; then
echo "Version $version does not match nightly tag ${{ inputs.nightly_tag_base }}. Exiting the workflow."
exit 1
fi
# Strip the leading `v` from the version
version=$(echo $version | sed 's/^v//')
echo "version=$version" >> $GITHUB_OUTPUT
- name: Build Langflow Base for distribution
run: |
rm -rf src/backend/base/dist
rm -rf dist
make build base=true args="--no-sources --wheel"
- name: Test Langflow Base CLI
run: |
# TODO: Unsure why the whl is not built in src/backend/base/dist
mkdir src/backend/base/dist
mv dist/*.whl src/backend/base/dist/
# Install with [complete] extra to test all optional dependencies
WHEEL_FILE=$(ls src/backend/base/dist/*.whl)
uv pip install "${WHEEL_FILE}[complete]"
uv run python -m langflow run --host localhost --port 7860 --backend-only &
SERVER_PID=$!
# Wait for the server to start
timeout 120 bash -c 'until curl -f http://localhost:7860/api/v1/auto_login; do sleep 2; done' || (echo "Server did not start in time" && kill $SERVER_PID && exit 1)
# Terminate the server
kill $SERVER_PID || (echo "Failed to terminate the server" && exit 1)
sleep 20 # give the server some time to terminate
# Check if the server is still running
if kill -0 $SERVER_PID 2>/dev/null; then
echo "Failed to terminate the server"
exit 0
else
echo "Server terminated successfully"
fi
# PyPI publishing moved to after cross-platform testing
- name: Upload Artifact
uses: actions/upload-artifact@v6
with:
name: dist-nightly-base
path: src/backend/base/dist
build-nightly-main:
name: Build Langflow Nightly Main
needs: [build-nightly-base]
runs-on: ubuntu-latest
outputs:
version: ${{ steps.verify.outputs.version }}
defaults:
run:
shell: bash
steps:
- name: Check out the code at a specific ref
uses: actions/checkout@v6
with:
ref: ${{ inputs.nightly_tag_release}}
persist-credentials: true
- name: "Setup Environment"
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-dependency-glob: "uv.lock"
python-version: ${{ env.PYTHON_VERSION }}
prune-cache: false
- name: Download LFX artifact
if: inputs.build_lfx
uses: actions/download-artifact@v7
with:
name: dist-nightly-lfx
path: lfx-dist
- name: Download base artifact
uses: actions/download-artifact@v7
with:
name: dist-nightly-base
path: base-dist
- name: Install the project
run: uv sync
- name: Force reinstall packages from built wheels
run: |
# While workspace resolution installs from source, we want to test the exact
# wheels that will be published to PyPI to catch any packaging issues
if [ "${{ inputs.build_lfx }}" == "true" ]; then
echo "Installing LFX from built wheel..."
uv pip install --force-reinstall --no-deps lfx-dist/*.whl
fi
echo "Installing langflow-base from built wheel..."
uv pip install --force-reinstall --no-deps base-dist/*.whl
- name: Verify Nightly Name and Version
id: verify
run: |
name=$(uv tree | grep 'langflow' | grep -v 'langflow-base' | grep -v 'langflow-sdk' | awk '{print $1}')
version=$(uv tree | grep 'langflow' | grep -v 'langflow-base' | grep -v 'langflow-sdk' | awk '{print $2}')
if [ "$name" != "langflow-nightly" ]; then
echo "Name $name does not match langflow-nightly. Exiting the workflow."
exit 1
fi
if [ "$version" != "${{ inputs.nightly_tag_release }}" ]; then
echo "Version $version does not match nightly tag ${{ inputs.nightly_tag_release }}. Exiting the workflow."
exit 1
fi
# Strip the leading `v` from the version
version=$(echo $version | sed 's/^v//')
echo "version=$version" >> $GITHUB_OUTPUT
- name: Wait for PyPI Propagation
if: needs.build-nightly-base.outputs.skipped == 'false'
run: sleep 300 # wait for 5 minutes to ensure PyPI propagation of base
- name: Build Langflow Main for distribution
run: make build main=true args="--no-sources --wheel"
- name: Test Langflow Main CLI
run: |
uv pip install dist/*.whl
uv run python -m langflow run --host localhost --port 7860 --backend-only &
SERVER_PID=$!
# Wait for the server to start
timeout 120 bash -c 'until curl -f http://localhost:7860/health_check; do sleep 2; done' || (echo "Server did not start in time" && kill $SERVER_PID && exit 1)
# Terminate the server
kill $SERVER_PID || (echo "Failed to terminate the server" && exit 1)
sleep 20 # give the server some time to terminate
# Check if the server is still running
if kill -0 $SERVER_PID 2>/dev/null; then
echo "Failed to terminate the server"
exit 0
else
echo "Server terminated successfully"
fi
# PyPI publishing moved to after cross-platform testing
- name: Upload Artifact
uses: actions/upload-artifact@v6
with:
name: dist-nightly-main
path: dist
test-cross-platform:
name: Test Cross-Platform Installation
needs: [build-nightly-lfx, build-nightly-base, build-nightly-main]
uses: ./.github/workflows/cross-platform-test.yml
with:
base-artifact-name: "dist-nightly-base"
main-artifact-name: "dist-nightly-main"
lfx-artifact-name: "dist-nightly-lfx"
sdk-artifact-name: "dist-nightly-sdk"
publish-nightly-lfx:
name: Publish LFX Nightly to PyPI
needs: [build-nightly-lfx, test-cross-platform, publish-nightly-sdk]
if: ${{ inputs.build_lfx }}
runs-on: ubuntu-latest
steps:
- name: Check out the code
uses: actions/checkout@v6
with:
ref: ${{ inputs.nightly_tag_release }}
persist-credentials: true
- name: Download LFX artifact
uses: actions/download-artifact@v7
with:
name: dist-nightly-lfx
path: src/lfx/dist
- name: Setup Environment
uses: astral-sh/setup-uv@v6
with:
enable-cache: false
python-version: "3.13"
- name: Publish LFX to PyPI
env:
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
run: |
make lfx_publish
publish-nightly-sdk:
name: Publish SDK Nightly to PyPI
needs: [build-nightly-lfx, test-cross-platform]
if: ${{ inputs.build_lfx }}
runs-on: ubuntu-latest
steps:
- name: Check out the code
uses: actions/checkout@v6
with:
ref: ${{ inputs.nightly_tag_release }}
persist-credentials: true
- name: Download langflow-sdk artifact
uses: actions/download-artifact@v7
with:
name: dist-nightly-sdk
path: src/sdk/dist
- name: Setup Environment
uses: astral-sh/setup-uv@v6
with:
enable-cache: false
python-version: "3.13"
- name: Publish SDK to PyPI
env:
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
run: |
make sdk_publish
publish-nightly-base:
name: Publish Langflow Base Nightly to PyPI
needs: [build-nightly-base, test-cross-platform, publish-nightly-lfx]
if: ${{ always() && needs.build-nightly-base.result == 'success' && needs.test-cross-platform.result == 'success' && (needs.publish-nightly-lfx.result == 'success' || inputs.build_lfx == false) }}
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
ref: ${{ inputs.nightly_tag_release }}
persist-credentials: true
- name: Download base artifact
uses: actions/download-artifact@v7
with:
name: dist-nightly-base
path: src/backend/base/dist
- name: Setup Environment
uses: astral-sh/setup-uv@v6
with:
enable-cache: false
python-version: "3.13"
- name: Publish base to PyPI
if: needs.build-nightly-base.outputs.skipped == 'false'
env:
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
run: |
make publish base=true
publish-nightly-main:
name: Publish Langflow Main Nightly to PyPI
needs: [build-nightly-main, test-cross-platform, publish-nightly-base]
if: ${{ always() && needs.build-nightly-main.result == 'success' && needs.test-cross-platform.result == 'success' && needs.publish-nightly-base.result == 'success' }}
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
ref: ${{ inputs.nightly_tag_release }}
persist-credentials: true
- name: Download main artifact
uses: actions/download-artifact@v7
with:
name: dist-nightly-main
path: dist
- name: Setup Environment
uses: astral-sh/setup-uv@v6
with:
enable-cache: false
python-version: "3.13"
- name: Publish to PyPI
env:
POETRY_PYPI_TOKEN_PYPI: ${{ secrets.PYPI_API_TOKEN }}
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
run: |
make publish main=true
call_docker_build_base:
name: Call Docker Build Workflow for Langflow Base
if: ${{ always() && inputs.build_docker_base }}
needs: [build-nightly-base, build-nightly-main]
uses: ./.github/workflows/docker-nightly-build.yml
with:
ref: ${{ inputs.nightly_tag_release }}
release_type: nightly-base
push_to_registry: ${{ inputs.push_to_registry }}
secrets: inherit
call_docker_build_main:
name: Call Docker Build Workflow for Langflow
if: ${{ always() && inputs.build_docker_main }}
needs: [build-nightly-main, call_docker_build_base]
uses: ./.github/workflows/docker-nightly-build.yml
with:
ref: ${{ inputs.nightly_tag_release }}
release_type: nightly-main
push_to_registry: ${{ inputs.push_to_registry }}
secrets: inherit
# TODO: Uncomment this when our runner can fit the builds that contain pytorch (and other large dependencies)
# call_docker_build_main_all:
# name: Call Docker Build Workflow for langflow-all
# if: always() && ${{ inputs.build_docker_main == 'true' }}
# needs: [build-nightly-main]
# uses: ./.github/workflows/docker-nightly-build.yml
# with:
# ref: ${{ inputs.nightly_tag_release }}
# release_type: nightly-main-all
# main_version: ${{ inputs.nightly_tag_release }}
# secrets: inherit
# call_docker_build_main_ep:
# name: Call Docker Build Workflow for Langflow with Entrypoint
# if: ${{ always() && inputs.build_docker_ep }}
# needs: [build-nightly-main, call_docker_build_main]
# uses: ./.github/workflows/docker-build-v2.yml
# with:
# ref: ${{ inputs.nightly_tag_release }}
# release_type: main-ep
# push_to_registry: ${{ inputs.push_to_registry }}
# secrets: inherit