mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-24 06:10:49 +08:00
b033bfa22b
* feat(auth): external trusted JWT auth + JIT user mapping Adds the OSS half of trusted external identity support: - New EXTERNAL_AUTH_* AuthSettings (off by default) covering provider key, token transport (header/cookie), JWKS or trusted-decode, claim mapping, and a pluggable EXTERNAL_AUTH_IDENTITY_RESOLVER import path. - New services/auth/external.py with JWT/JWKS validation, identity resolver protocol, and token extraction helpers. - AuthService.get_or_create_user_from_claims + extract_user_info_from_claims implement the existing BaseAuthService JIT hook through SSOUserProfile - no new tables. - _authenticate_with_token falls back to external resolution when the native JWT path fails, so Authorization-header callers transparently upgrade to external auth. - Token extractors in services/auth/utils.py consult the configured external header/cookie after the native JWT path on session, WebSocket, SSE, and optional-user dependencies. - /api/v1/session catches AuthenticationError so external-credential failures resolve to authenticated=False rather than 500. Tests: 14 unit tests for external.py (JWT decode, claim mapping, custom resolver) plus 3 integration tests in test_login.py exercising the session endpoint JIT path (header + cookie + expired-token). Co-Authored-By: phact <estevezsebastian@gmail.com> Co-Authored-By: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Based-On: langflow-ai/langflow#13280 * Update src/backend/base/langflow/services/auth/external.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(auth): refetch JWKS once on unknown kid to survive IdP key rotation Without this, a token signed by a newly rotated IdP key is rejected for up to JWKS_CACHE_TTL_SECONDS (5 min) because the cached JWKS predates the key. On a kid miss we now refetch the JWKS once, rate-limited to one forced refresh per 30s per URL so attacker-supplied kids cannot hammer the IdP's JWKS endpoint. Adds JWKS-path tests (signature verification, rotation refetch, rate-limited refresh) that were previously uncovered. * feat(auth): add external access ceiling for trusted auth (#13637) * feat(auth): add external access ceiling * fix(auth): tolerate minimal external access settings * fix(auth): address external-auth review (require JWKS audience, untangle authz/auth) Resolves Gabriel's PR review feedback on #13293: - Require EXTERNAL_AUTH_AUDIENCE on the JWKS verification path. Previously a JWKS-only config verified signature + exp but left aud/iss unbound, so a token the same IdP minted for a *different* relying party was accepted. decode now fails closed (before any network fetch) with an actionable message and binds aud; iss stays verified-when-set. Adds wrong-aud / missing-aud tests. - Move the request-scoped action ceiling out of auth/external.py into a new authorization/access_ceiling.py so the authorization package no longer imports the auth layer (the only such import). Guards consult the authz-owned primitive; the auth layer only derives the ceiling and installs it. external.py re-exports the names for callers that derive/inspect the ceiling. - _unique_external_username reuses _external_username_fallback instead of re-implementing the provider-digest formula. - Read the non-optional EXTERNAL_AUTH_* settings as plain attributes (drop getattr(..., default), which re-hardcoded Field defaults) in service.py and api_key/crud.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(authz): pass API key context to authorization (#13639) * chore: bump version to 1.10.1 * fix(lfx): restamp component index version after 1.10.1 fork bump (#13575) Companion to the release-1.11.0 fix (#13574). The 1.10.1 fork bump left component_index.json's version at 1.10.0; _read_component_index fails closed on the exact-version mismatch, so the bundled registry loads as None and the upgrade-gate LFX tests fail on this branch's CI. Surgical restamp: version -> 1.10.1, sha256 recomputed with the build script's exact hashing/serialization. Entries unchanged; 2-line diff. The three affected tests pass locally. * fix(ci): rebalance backend test splits with measured durations + raise unit-test step timeout (1.10.1) (#13585) fix(ci): rebalance backend test splits with measured durations + raise unit-test step timeout Unit Tests - Python 3.12 - Group 3 has been failing at 98-99% on the nightly: the job hits the nick-fields/retry 40-minute per-attempt timeout and pytest is SIGTERM'd mid-test, so it looks like one flaky last test when it is actually a deterministic timeout (and the internal retry plus run-level retries can never succeed). Root cause is twofold: 1. .test_durations was last regenerated ~May 2025 and covered only 2,219 of ~9,500 current unit tests, so pytest-split weighted 77% of the suite at the 0.73s average. The expensive client-fixture tests (test_webhook.py, test_login.py - each pays a full create_app + lifespan boot per test, 60-120s late in a CI run) clustered into group 3's tail, making it ~8-10 minutes slower than its siblings (33:15 on 3.13, >40min on 3.12 which is additionally slowed by astrapy disabling SSL connection reuse on Python 3.12.0-11). The weekly store_pytest_durations workflow that should refresh the file has been dying at the 6-hour GitHub job limit every week (serial full-suite run no longer fits), so the file silently froze. This regenerates the file from real per-test wall-clock measured in the passing Python 3.13 nightly jobs (run 27246228762 attempt 1, all 5 groups, parsed from the -vv xdist logs: per-worker start-to-start deltas). 9,508 tests now have measured durations; old entries are kept where no new measurement exists. Simulated least_duration split goes from one outlier group to 5 even groups (~24.6 min each) with the >50s tests spread 1-2 per group instead of 7 in one. 2. timeout_minutes 40 -> 50 gives headroom for matrix-cell variance (3.12 runs ~20% slower than 3.13) so a slightly slow cell degrades gracefully instead of burning 2x40min and failing the whole run. Follow-ups (not in this PR): fix store_pytest_durations to run with xdist or split groups so it fits the 6h limit; investigate the in-worker degradation that makes client-fixture boots cost 8-12s early in a run but 60-120s after ~30 minutes. * fix(test): raise spawn-child join timeout in test_multi_process_visibility (1.10.1) (#13590) fix(test): raise spawn-child join timeout in test_multi_process_visibility The test spawns a child via multiprocessing spawn context, which cold-imports the full langflow package (plus coverage's multiprocessing hooks in CI) before appending a single event. The 10s join timeout is routinely exceeded on a loaded CI runner sharing 4 vCPUs with a second xdist worker: in nightly run 27253229568 (Unit Tests - Python 3.12 - Group 5) the test failed all 12 executions (5 reruns x 2 step attempts), each rerun exactly 10s apart - the join deadline, not a product bug. Raise the liveness bound to 60s (join returns immediately when the child exits, so the passing case is unaffected) and kill the child on timeout so a hung spawn can't leak into later tests. * fix(ci): anchor langflow-base version extraction in nightly docker build (1.10.1) (#13593) fix(ci): anchor langflow-base version extraction in nightly docker build The first nightly on the 1.11 workspace (tag v1.11.0.dev0, run 27253229568) failed in Build Nightly Base Package within seconds: 'Base version format is incorrect'. The extraction (uv tree, grep langflow-base unanchored, awk field 3, first line) broke because uv tree now prints the langflow-base workspace-root line ('langflow-base v1.11.0.dev0', two fields) before any dependency line ('langflow-base[complete] v1.11.0.dev0' under the langflow root, three fields), so the first match yields an empty field 3 and the format check exits 1. uv tree's stderr is discarded, which made the real cause invisible in CI. Anchor to the root line and take field 2 instead - the exact pattern the langflow (main package) extraction in this same file already uses, which is why the main-package builds passed on the same tag. Verified both extractions print 1.11.0.dev0 against a local checkout of v1.11.0.dev0. * fix(test): gate models.dev background refresh out of tests (1.10.1) (#13598) fix(test): gate models.dev background refresh out of tests Integration tests failed twice in nightly run 27260425158 with pyleak EventLoopBlockError - first Integration Tests 3.14, then 3.12 on the rerun, each time in a different test. The blocking stack points at refresh_models_dev_periodically: every app boot unconditionally starts a lifespan task that immediately fetches https://models.dev/api.json, so the request lands mid-test in whatever test happens to be running. Under pyleak's asyncio debug instrumentation the fetch blocked the loop 0.797s against a 0.2s threshold. Whichever test draws the short straw flakes - which is why it looked transient and moved between versions. Add a LANGFLOW_MODELS_DEV_REFRESH env gate (default unchanged: enabled) and disable it session-wide in the backend test conftest. Tests fall back to the bundled static model lists, which is also deterministic. Verified: with the gate set, app boot makes zero models.dev requests; the previously failing integration test passes. * fix(ci): allow pre-releases when pinning the nightly in migration validation (1.10.1) (#13601) fix(ci): allow pre-releases when pinning the nightly in migration validation Migration Test: pip/venv (stable -> nightly) failed deterministically on nightly run 27260425158 (twice, including a rerun): hint: langflow-base was requested with a pre-release marker (e.g., langflow-base==1.11.0.dev1), but pre-releases weren't enabled (try: --prerelease=allow) This is the first nightly publishing as a canonical .devN pre-release of the langflow distribution (nightly -> stable bundle cutover). The 'latest' branch of the upgrade step already passes --prerelease=allow, but the pinned-version branches do not. uv implicitly allows the pre-release for the directly requested ==X.Y.Z.devN pin, yet langflow's metadata pins langflow-base==X.Y.Z.devN transitively, and uv rejects transitive pre-releases unless they are enabled - so the install fails after the stable uninstall, sinking the migration test. Add --prerelease=allow to both pinned-version install lines. * fix(ci): scope nightly migration-test pre-releases to the langflow stack (1.10.1) (#13605) fix(ci): scope nightly migration-test pre-releases to the langflow stack The previous fix (#13599) added a global --prerelease=allow, which let UNRELATED dependencies resolve to alphas: on nightly run 27274206250 the stable->nightly upgrade pulled pydantic 2.14.0a1 + pydantic-yaml 1.6.1a1, and the pydantic alpha breaks langchain-core at import time (RunnablePassthrough pydantic ValidationError), failing the nightly boot right after a successful install. Clean installs were fine - only this upgrade path resolved the alpha combo. Scope pre-release eligibility to the langflow lockstep stack instead: uv accepts a pre-release when the package's own requirement carries a pre-release marker, but not via transitive pins, and the nightly chain is langflow -> langflow-base -> lfx -> langflow-sdk with exact ==devN pins. Request each directly; langflow-sdk versions independently (0.2.0.devN), so an explicit .dev0 floor marks it eligible while lfx's exact pin selects the version. The 'latest' branch gets the same treatment via .dev0 floors on all four. Verified by dry-run against PyPI: langflow/langflow-base/lfx at 1.11.0.dev2 + langflow-sdk 0.2.0.dev2 resolve with pydantic staying at stable 2.13.4. * fix(ci): create GitHub releases on the dispatched v-prefixed tag (1.10.1) (#13609) fix(ci): create GitHub releases on the dispatched v-prefixed tag The create_release job passed the bare version (v stripped) as the release tag with no commit target, so when that tag did not exist GitHub minted a new lightweight tag at the default-branch HEAD -- the wrong commit, still carrying the previous version (main only adopts a release's version via the post-release back-merge). Every release since 1.8.2 shipped a stray bare tag (1.8.2, 1.8.3, 1.9.0-1.9.6, 1.10.0) pointing at a previous-version commit, and the GitHub release had to be manually re-pointed to the real vX.Y.Z tag after each release. - create_release now attaches the release to inputs.release_tag for stable releases; pre-releases keep their computed tag (e.g. 1.10.0rc1) but it is minted at the release commit via 'commit:'. - release-lfx.yml pins the minted lfx-v* tag to github.sha instead of the default branch. The validate-tag-format guard (#12847) only blocks at dispatch time; create_release was re-creating the very duplicates it guards against. * fix(ci): resolve validate-version output in release-lfx changelog link (1.10.1) (#13611) fix(ci): resolve validate-version output in release-lfx changelog link The create-release job references needs.validate-version.outputs.current_version in its generated release notes (the Full Changelog compare link), but validate-version was not in the job's needs array, so the expression evaluated empty and the link rendered as compare/v...lfx-vX.Y.Z (broken base). Add validate-version to the create-release needs. This adds no real serialization: create-release already waits on release-lfx, which transitively requires validate-version via run-tests, and the job's always() if-condition is unaffected. Flagged by actionlint: property "validate-version" is not defined in object type {build-docker, release-lfx}. * feat(auth): external trusted JWT auth + JIT user mapping Adds the OSS half of trusted external identity support: - New EXTERNAL_AUTH_* AuthSettings (off by default) covering provider key, token transport (header/cookie), JWKS or trusted-decode, claim mapping, and a pluggable EXTERNAL_AUTH_IDENTITY_RESOLVER import path. - New services/auth/external.py with JWT/JWKS validation, identity resolver protocol, and token extraction helpers. - AuthService.get_or_create_user_from_claims + extract_user_info_from_claims implement the existing BaseAuthService JIT hook through SSOUserProfile - no new tables. - _authenticate_with_token falls back to external resolution when the native JWT path fails, so Authorization-header callers transparently upgrade to external auth. - Token extractors in services/auth/utils.py consult the configured external header/cookie after the native JWT path on session, WebSocket, SSE, and optional-user dependencies. - /api/v1/session catches AuthenticationError so external-credential failures resolve to authenticated=False rather than 500. Tests: 14 unit tests for external.py (JWT decode, claim mapping, custom resolver) plus 3 integration tests in test_login.py exercising the session endpoint JIT path (header + cookie + expired-token). Co-Authored-By: phact <estevezsebastian@gmail.com> Co-Authored-By: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Based-On: langflow-ai/langflow#13280 * Update src/backend/base/langflow/services/auth/external.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(auth): refetch JWKS once on unknown kid to survive IdP key rotation Without this, a token signed by a newly rotated IdP key is rejected for up to JWKS_CACHE_TTL_SECONDS (5 min) because the cached JWKS predates the key. On a kid miss we now refetch the JWKS once, rate-limited to one forced refresh per 30s per URL so attacker-supplied kids cannot hammer the IdP's JWKS endpoint. Adds JWKS-path tests (signature verification, rotation refetch, rate-limited refresh) that were previously uncovered. * fix(components): follow HTTP redirects in URL component with per-hop SSRF revalidation (#13572) * fix(components): follow HTTP redirects in URL component with per-hop SSRF revalidation The URL component sent every request with follow_redirects=False, so any site whose entered URL 301s to its canonical address (http->https or www/non-www normalization) returned the redirect stub - e.g. a bare "301 Moved Permanently / nginx" page - as the scraped content, or failed outright when the redirect response had an empty body. Redirects are now followed via a new advanced "Follow Redirects" input (default on). When SSRF protection is enabled, hops are followed manually and each Location target is re-validated with the same blocked-IP denylist and DNS pinning as the initial request before any connection is made, mirroring the API Request component; cross-host hops drop sensitive headers and chains are capped at 20 redirects. * fix(components): compare full origin when keeping credentials across redirects, crawl from post-redirect base Addresses CodeRabbit review: - _headers_for_redirect now keeps Authorization/Proxy-Authorization/Cookie only for same-origin hops (scheme, host, port) or a direct http->https upgrade on default ports, exactly the cases where httpx keeps the Authorization header. Applied to both the URL and API Request components (the helper was copied from the latter). - _crawl_recursive resolves relative links and the prevent_outside check against the final post-redirect URL, and marks it visited, so depth>1 crawls work on sites that 301 to their canonical address. * chore: regenerate component index and starter projects for release-1.10.1 base The autofix.ci jobs uploaded but never pushed the regeneration after the rebase, leaving the 4 URL-component templates with a stale field_order (missing follow_redirects) and the index without the new code hashes. Generated with the same commands CI uses: LFX_DEV=1 make build_component_index + scripts/ci/update_starter_projects.py. Pokedex Agent / Structured Data Analysis Agent pick up the API Request component's new code_hash from the origin-comparison fix. * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix(ci): make release Biome lint green — NUL-delimit file list + clear pre-existing lint errors (#13550) The Lint Frontend job runs against `main` as the base. In a release (workflow_dispatch) run it diffs the entire release branch (~915 files) and re-lints nearly the whole frontend, exposing two issues that per-PR linting never hits together: 1. xargs split starter-project spec paths containing spaces (e.g. "News Aggregator.spec.ts" -> "News" + "Aggregator.spec.ts"), producing `internalError/io: No such file or directory`. NUL-delimit the file list so spaces are preserved. (supersedes #13381) 2. The whole-branch diff surfaced 30 pre-existing Biome errors: 22 noExplicitAny + 8 organizeImports. Resolved with real types where safe (freezeObject generic, ColDef defaults, messagesSorter field shape, VertexBuildTypeAPI, Record<string,string>, DragEvent<HTMLElement>, unknown for narrowed values) and justified biome-ignore for genuinely loose cases (polymorphic display values, test global stubs, captured unexported StreamCallbacks). Imports auto-sorted via biome. Verified locally: biome check on the full release-vs-main file set is now 0 errors (was 30); tsc unchanged at its 303-error baseline (no new type errors). * fix: limit public flow endpoint from displaying private flow streams (#13602) * fix: limit public flow endpoint from displaying private flow streams * fix: Address coderabbit reviews --------- Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> * fix: enforce the FileSystemTool credential deny-list (#13625) * enforce the FileSystemTool credential deny-list * security checks gh review * [autofix.ci] apply automated fixes * feat(authz): pass API key context to authorization * refactor(authz): address review feedback on API-key auth context - Add AuthCredentialContext.from_api_key_result() and use it at all six API-key projection sites (service.py x5, mcp_projects.py) so the caveat fields stay in sync and no site can silently drop one. - authz_me builds the enforce context from the public current_auth_context_for_authz() helper instead of reaching into the private guards._auth_context. - Clear request-local credential context at the top of verify_project_auth to match the service.py entrypoints, so the composer-token fast path can never inherit stale context. --------- Co-authored-by: phact <estevezsebastian@gmail.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com> Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com> * [autofix.ci] apply automated fixes * fix(auth): close access-ceiling bypasses and harden external JWT auth Address review findings on the external trusted-auth + access-ceiling work: - Enforce the access ceiling on execution/mutation paths that bypassed the ensure_*_permission chokepoint: MCP project tool-call (flow execute), flow version snapshot/activate/delete, v1 file upload/delete, and memory base CRUD/ingest. - Move the external-user API-key block into the shared authenticate_api_key chokepoint so /run, v2 workflow, OpenAI-compat, WebSocket, webhook and MCP key auth all enforce it (was only on the JWT-fallback path). - Require exp on both the JWKS and trusted-decode paths; reject non-https JWKS URLs (loopback http allowed for dev). - Normalize EXTERNAL_AUTH_PROVIDER at the config boundary so the API-key floor cannot be silently disabled by an empty/whitespace value. - Make a configured EXTERNAL_AUTH_ACCESS_CLAIM_MAPPING authoritative: an unmapped claim value falls to the default level instead of self-elevating via the built-in alias table. - Stop nulling a stored SSOUserProfile.email when a later token omits email. - Keep the external credential usable as a fallback when a stale/invalid native token is present (WebSocket, SSE, optional-user paths). - Add delete to the editor access level (deploy stays admin-only). Adds regression tests across auth, authz guards, api-key crud, and the newly guarded routes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(mcp): give handle_call_tool flow stub the attrs ensure_flow_permission reads The access-ceiling hardening added an ensure_flow_permission(EXECUTE) guard to handle_call_tool that reads flow.user_id and flow.workspace_id, but the _invoke_handle_call_tool flow stub only carried id/name/folder_id, raising AttributeError. Set user_id to match the current user so the owner-override path is exercised, and add workspace_id. * fix(auth): close remaining ceiling gaps + external-cookie/filesystem/job-queue hardening Synthesizes the external review (P1-P3) with the multi-agent re-review findings. External auth: - P1: regular HTTP (get_current_user) and /api/v1/session now extract the external credential separately and pass it as a fallback, so a stale/invalid native cookie can no longer shadow a valid external credential (previously only WS/SSE/optional paths did this). - P2: clear the request-local external access ceiling at every auth entrypoint (api-key, raw access-token, webhook, ws api-key, MCP), not just authenticate_with_credentials, so a stale ceiling can never carry into a non-external path. - Make a configured access-claim mapping authoritative even when it parses empty (all-invalid entries no longer re-enable alias self-elevation). - A blocked external user's API key no longer increments usage counters. Access-ceiling coverage (routes that previously escaped the deny-only cap): - custom_component / custom_component_update (code instantiation) now enforce the ceiling directly (viewer denied; editor/admin/native users unchanged). - Deprecated /upload/{flow_id}, update_project_mcp_settings, and the models.py default/enabled-model variable routes now call the appropriate guard. - Memory-base guards resolve the base first and pass kb_id + real owner so plugin enforce runs for non-owners and audit rows carry the kb id. Bundled hardening: - P2: filesystem deny-list now denies a protected credential directory requested by basename (.ssh/.aws/.git) and as a glob entry, not just as a parent dir. - P3: public-job marker write failures on the Redis backend now fail the build (503) instead of returning an un-shareable job id that 404s on other workers. Adds regression tests across auth, authz route guards, api-key crud, external auth, filesystem deny-list, memory bases, and the redis job queue. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [autofix.ci] apply automated fixes --------- Co-authored-by: phact <estevezsebastian@gmail.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com> Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local> Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>