Commit Graph

18042 Commits

Author SHA1 Message Date
d06020fc87 fix(background-execution): await in-flight job tasks on executor stop
stop() cancelled in-flight tasks but returned without awaiting them, so a job's
shielded terminal reconcile (a DB write) could race a closing engine and leave a
'Task was destroyed but it is pending' warning. Cancel and gather the in-flight
job tasks first, then tear down the workers. Cancelling the job tasks directly
(rather than relying on cancellation propagating through the worker's await)
avoids the worker sitting in cancellation limbo behind a job that swallows its
cancel on the user-stop path, which otherwise stalls teardown.
2026-06-04 04:37:34 -03:00
de86276094 fix(background-execution): stamp STOP signals consumed when acted on
unconsumed_signals filters consumed_at IS NULL but nothing stamped it, so STOP
rows lingered: the table grew and a re-enqueued job self-cancelled off a stale
STOP. Add JobService.consume_signals and stamp the STOP in the runner's
_reconcile_stop, the single point where a stop is finalized.
2026-06-04 04:05:28 -03:00
d76e86416f fix(background-execution): claim QUEUED jobs atomically in the startup sweep
Two uvicorn workers booting against the same DB both re-enqueued the same QUEUED
row, so a non-idempotent flow executed twice. Add JobService.claim_queued_job, a
conditional UPDATE ... WHERE status='QUEUED' that only one racer can win
(rowcount==1), and gate sweep re-enqueue on it. Works on SQLite and Postgres.
2026-06-04 04:03:29 -03:00
9f109320c8 fix(background-execution): enforce background_job_timeout on runs
The background_job_timeout setting was read by nobody, so a runaway run never
timed out. Bound the runner's drive with asyncio.wait_for(timeout=...) when the
setting is configured; execute_with_status already maps asyncio.TimeoutError to
TIMED_OUT, so the run ends TIMED_OUT with a terminal event. None keeps the prior
unbounded behaviour. The facade passes settings.background_job_timeout through.
2026-06-04 04:01:02 -03:00
c5e968a56c fix(background-execution): answer reattach to terminal jobs from durable status
events() decided 'is the job finished' from the process-local live bus, whose
_closed marker is empty after a restart. A reattach to an already-terminal job
replayed durable rows then blocked forever on the live tail. Key the decision off
the persisted job status instead: when terminal, replay the durable log and
return. Re-frame durable rows through the same format_sse_event formatter the
live path uses (with id=str(seq)) so replayed and live frames are byte-compatible
and Last-Event-ID resumes correctly.
2026-06-04 03:59:04 -03:00
49c6b382a7 fix(background-execution): make worker cancel-handling Python 3.10 safe
The worker used asyncio.Task.cancelling() to tell a job-cancel apart from a
worker-cancel, but that API is Python 3.11+. On 3.10 the first job cancellation
raised AttributeError inside the cancel handler and permanently killed the
worker, hanging later jobs QUEUED. Replace it with an explicit _closed flag set
by stop(): a CancelledError while shutting down re-raises to exit the worker,
otherwise it is a per-job cancel that is swallowed so the pool keeps serving.
2026-06-04 03:55:10 -03:00
dcf7ed890d test(lfx): include idempotency_key in WorkflowRunRequest round-trip body 2026-06-04 03:44:37 -03:00
1b2e748491 test(workflows): adapt status-failed mock for additive error_detail; make resume contract deterministic 2026-06-04 03:35:18 -03:00
461cc2f1cb test(workflows): guard dead in-memory background machinery stays deleted 2026-06-04 03:18:08 -03:00
d3cf825754 test(workflows): pin v2 background wire contract is additive-only (links + SSE id) 2026-06-04 03:16:48 -03:00
25062985dd feat(workflows): add optional idempotency_key to v2 background submit 2026-06-04 03:14:39 -03:00
89bf7bd010 fix(workflows): surface durable result/error on v2 status for background jobs 2026-06-04 03:09:26 -03:00
b183fef831 feat(workflows): persist submit request so re-enqueued QUEUED jobs replay original inputs 2026-06-04 03:05:29 -03:00
c6c6ab53e9 feat(workflows): add events re-attach link to WorkflowJobResponse 2026-06-04 02:59:16 -03:00
fa5d9c8855 fix(jobs): retry append_event on seq collision and sqlite lock (gap-free under concurrency) 2026-06-04 02:42:07 -03:00
cf26d6bc51 fix(workflows): make stop deterministically win over racing completion 2026-06-04 02:30:51 -03:00
beee5329a9 test(workflows): hard-proof durable background path on sqlite+postgres; stabilize stop e2e 2026-06-04 02:17:50 -03:00
b90b190965 feat(workflows): sweep orphaned jobs on startup (requeue QUEUED, fail IN_PROGRESS) 2026-06-04 02:04:52 -03:00
f20c90726a feat(workflows): route background mode through BackgroundExecutionService facade 2026-06-04 02:00:20 -03:00
a7d536c437 feat(workflows): add BackgroundExecutionService facade, factory, and dep 2026-06-04 01:23:34 -03:00
e8279f67d3 feat(workflows): add background job runner driving adapter to durable state 2026-06-04 00:39:51 -03:00
be46df481e feat(workflows): add in-memory live bus with durable replay reattach 2026-06-04 00:18:32 -03:00
c1839d2a2b feat(workflows): add bounded in-process executor for background jobs 2026-06-04 00:16:12 -03:00
eb8e1f4063 feat(settings): add background_max_concurrency and background_job_timeout 2026-06-04 00:03:23 -03:00
5c24ef0cd1 feat(workflows): classify adapter frames as durable vs ephemeral 2026-06-04 00:02:02 -03:00
82e500c19a test(jobs): hard-proof JobService store methods on real sqlite and postgres 2026-06-03 23:26:17 -03:00
a133842419 feat(jobs): add sweep_orphans startup reconcile to JobService 2026-06-03 23:19:14 -03:00
a42bc1538d feat(jobs): add write_signal and unconsumed_signals to JobService 2026-06-03 23:17:21 -03:00
c573e4bfee feat(jobs): add append_event (per-job seq) and read_events(after_seq) to JobService 2026-06-03 23:15:46 -03:00
a064694c1b feat(jobs): add set_result and set_error durable writers to JobService 2026-06-03 23:13:55 -03:00
dca1a72801 feat(jobs): add execution_signals table, ExecutionSignal model, SignalType enum 2026-06-03 23:12:14 -03:00
5d4d9eb50d feat(jobs): add job_events table and JobEvent model with unique (job_id, seq) 2026-06-03 23:09:45 -03:00
27945c9ffc feat(jobs): add durable result and error columns to job table 2026-06-03 23:07:24 -03:00
bb1b66f3e7 fix(jobs): order get_jobs_by_flow_id by created_timestamp not created_at 2026-06-03 23:04:14 -03:00
3c39a64724 test(background): real-instance test harness (hard_proof marker, sqlite/postgres/redis fixtures, make + CI) 2026-06-03 22:59:37 -03:00
a1b9cb76f2 feat(api/v2): emit per-output events on the langflow stream
Give v2-workflows sync and the langflow stream protocol one parser. The
stream now emits a normalized "output" event per terminal output carrying
an OutputEvent (the ComponentOutput shape sync returns in outputs[id], plus
component_id). A shared build_component_output() backs both the sync
converter and the adapter, and the build loop ships authoritative vertex
metadata as an additive output_meta key on end_vertex (existing consumers
read build_data and ignore it).

This is access-pattern parity (one parser, same fields, same terminal set),
not byte-identical content: the stream reuses the v1 build path whose
display serialization differs from sync's run_graph output.
2026-06-03 21:05:23 -03:00
f3c06efb73 feat(api/v2): add request-side output selection (output_ids)
Let a sync caller name the output(s) they want via output_ids so
output.text resolves deterministically (reason=single) on multi-output
flows instead of going null. Selection is steer-only: it picks the
answer among the named outputs without filtering the outputs map.

Invalid ids are rejected with 422 before the flow runs (and before any
job row is created), so a typo costs no compute. Resolution considers
selected outputs that actually fired, so branching flows resolve to
whichever candidate ran.
2026-06-03 16:23:23 -03:00
37d8a78913 feat(api/v2): structured output with resolution reason on v2 response
Replace the flat output_text shortcut with an `output` object carrying the
resolved text answer plus a `reason` that explains why it resolved that way
(single/multiple/none/non_string/failed), so a null answer is always
diagnosable instead of silently None. `reason` follows the LLM-domain
finish_reason/stop_reason convention, distinct from the lifecycle status.

Also add `display_name` to each ComponentOutput (the stable component id
stays the dict key) and a computed `has_errors` flag derived from errors.
2026-06-03 15:08:53 -03:00
461e593dce test(api/v2): cover output_text and session_id on the v2 workflow response
Pin the sync-response shortcuts on the v2 workflows endpoint:
- output_text surfaces the lone ChatOutput/TextOutput text and stays None for
  non-output message nodes, data-only flows, and multi-text flows
- session_id echoes the resolved session; the error response exposes neither
- each outputs entry exposes only {type, status, content, metadata}, with the
  component id carried by the dict key

Also drop the component_id kwarg the converter passed to ComponentOutput, which
has no such field and silently dropped it.
2026-06-02 22:20:07 -03:00
95f9102adb feat(api): add output_text and session_id to v2 workflow response
The synchronous /api/v2/workflows response keyed every result under its
component id, so reading the answer meant knowing an id you can't predict.
Surface two additive fields:

- output_text: the flow's single text answer (ChatOutput/TextOutput). None
  when the flow has zero or multiple text outputs, so callers read outputs
  rather than the shortcut guessing which channel is the answer.
- session_id: echoes the resolved session so chat/memory callers can
  continue the same thread (v1 /run returned this; v2 had dropped it).

outputs is unchanged, so this is non-breaking.
2026-06-01 21:05:26 -03:00
88560cb66b feat: native v2 workflows endpoint with pluggable stream protocols
Rebased onto release-1.10.0. The base independently rebuilt the v2
workflows backend (RBAC, body globals, share-aware fetch); keep our
forward design and conform its auth to that work:

1. Auth: keep get_current_user_for_workflow (session-or-API-key authN
   that does not hold a DB connection during the inline run, avoiding
   the SQLite lock contention api_key_security would cause) and enforce
   the base's RBAC on top: ensure_flow_permission(EXECUTE) before run,
   (READ) before status reconstruct, with widen_for_shares fetch.
2. Port the base's request-body globals onto the v2 WorkflowRunRequest.
   The X-LANGFLOW-GLOBAL-VAR-* headers stay supported (the Responses API
   passes globals that way); body globals win on conflict. Converters
   echo the effective globals via effective_globals.
3. Public endpoint keeps the v1 build_public_tmp posture
   (access_type==PUBLIC, run-as-owner); RBAC applies to the
   authenticated endpoint only.
4. Preserve the base's post-build KB-cache invalidation in the AG-UI
   build path.

The endpoint, AG-UI bridge, pluggable stream adapters, public endpoint,
and re-attach are unchanged.
2026-06-01 15:39:54 -03:00
7e321059db docs: redis worker queue (#13386)
* docs: remove 1.8 env vars

* docs: link out to deployment guide

* docs: redis queue

* fix-links-and-combine-env-vars-table

* docs: cleanup

* peer-review

* peer-review

* Apply suggestions from code review

Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>

* move-prereqs

---------

Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
2026-06-01 16:35:25 +00:00
73e49b13a0 test: Skip starter-projects shard4 on Windows CI (#13439)
skip shard 4 for windows
2026-06-01 09:56:11 -03:00
3e57126b91 fix: remove duplicate uv.lock from workspace member (#13326)
* fix: remove src/backend/base/uv.lock and Dockerfile references

- Deleted src/backend/base/uv.lock (monorepo should have only one uv.lock at root)
- Removed COPY ./src/backend/base/uv.lock lines from all Dockerfiles:
  - docker/build_and_push.Dockerfile
  - docker/build_and_push_base.Dockerfile
  - docker/build_and_push_ep.Dockerfile
  - docker/build_and_push_with_extras.Dockerfile
  - docker/dev.Dockerfile

Fixes LE-1093

* fix: remove stale base uv.lock regeneration paths
2026-05-30 13:05:34 +00:00
8179ceac3c feat: Hide UI elements in embedded mode (#13099)
* feat(LE-906): Config/settings plumbing

- Add 7 new boolean settings: custom_component_admin_only, embedded_mode,
  hide_logout_button, hide_new_project_button, hide_new_flow_button,
  hide_starter_projects, mcp_servers_locked (all default: false)
- All settings configurable via LANGFLOW_* env vars
- Expose all flags through ConfigResponse schema
- Fix hide_getting_started_progress to read from settings not env
- Add frontend type definitions for all 7 flags
- Initialize utility store with all new flags
- Implement hydration logic for new flags in config hook

* [autofix.ci] apply automated fixes (attempt 2/3)

* fix(LE-906): address PR1 review feedback

- replace internal ICA wording with neutral embedded-mode language
- add settings field for hide_getting_started_progress
- map hide_getting_started_progress from settings in ConfigResponse
- add regression test for embedded_mode hide-flag cascade
- revert unrelated component_index drift from PR scope

* fix(LE-906): address remaining PR1 Gabriel comments

* docs/test(LE-906): clarify embedded_mode scope for QA

- Clarify in settings docs that embedded_mode cascades UI hide flags only
- Keep mcp_servers_locked and custom_component_admin_only as explicit opt-ins
- Add config test assertions that security flags do not auto-cascade

* feat(LE-906): Embedded UI visibility changes

- Hide logout button from account menu when hideLogoutButton flag enabled
- Hide new flow button from header when hideNewFlowButton flag enabled
- Hide new project button from sidebar when hideNewProjectButton flag enabled
- Hide starter projects tab from templates modal when hideStarterProjects enabled
- All UI gates read from utility store (flags populated from server config)
- Supports embedded/iframe mode integration where standalone UI elements are hidden
- Pure frontend changes: no backend dependencies, no breaking changes
- Respects embedded_mode umbrella flag when individual hide flags set

* fix(LE-906): restore overwritten header components and keep embedded UI gates

* [autofix.ci] apply automated fixes

* feat(LE-906): hide new flow entry points

* fix(LE-906): address PR4 review comments from Gabriel

* fix(LE-906): resolve PR4 biome lint failures

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-30 12:03:58 +00:00
f05aaaf543 feat: Gate custom component creation for admins only (#13098)
* feat(LE-906): Custom component admin gating + fix field refresh side effect

- Add custom_component_admin_only gate to POST /custom_component endpoint
- Only admin users can CREATE new custom component code when flag enabled
- Remove admin gate from POST /custom_component/update endpoint
  * This endpoint handles field metadata refresh (e.g., model provider changes)
  * Normal users need access to refresh available models/options
  * Gate on creation prevents harmful code, not on field refresh operations
- Use hardened feature-flag checks with getattr(..., False) is True pattern
- Fixes side effect where normal users couldn't refresh field metadata
  (e.g., couldn't change LLM models when provider config changes)
- Gate applies only to creation of NEW custom components, not metadata updates

* fix(LE-906): address PR3 review feedback

- remove stale placeholder commentary from custom_component/update
- enforce admin-only restriction for truly custom code updates
- keep known-template refresh/update path available for non-admin users
- add endpoint tests for allow/block behavior under admin-only mode
- revert unrelated component_index drift from PR scope

* feat(LE-906): tighten custom component admin gating

* fix(LE-906): tighten custom-component admin gate carveouts

* [autofix.ci] apply automated fixes

* fix(LE-906): address PR3 review comments from Gabriel

- Rename create-side known-template refresh test for accurate intent
- Strengthen superuser coverage with novel-code create case and update case
- Revert unrelated starter project dependency version drift (OpenDsStar)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-30 03:54:14 +00:00
61ad11e530 feat: MCP lock enforcement + UI/tests (#13097)
* feat(LE-906): MCP lock enforcement + UI/tests

- Add mcp_servers_locked gate to POST /{project_id}/install endpoint
- Extend lock check to PATCH /{project_id} endpoint for auth settings updates
- Non-superuser requests blocked with 403 when flag is enabled
- Add hardened feature-flag checks using getattr(..., False) is True pattern
  to prevent MagicMock truthiness in tests
- Update AddMcpServerModal to show locked message when mcp_servers_locked=true
- Fix JSX structure in modal (missing fragment wrapper)
- All MCP modifications now gated when mcp_servers_locked flag enabled

* fix(LE-906): address PR2 review feedback

- add is_mcp_servers_locked helper with MagicMock-safe semantics and rationale
- add regression tests for explicit-true vs MagicMock placeholder behavior
- add missing mcp.modal.lockedTitle/lockedDescription i18n keys across locales
- remove debug leftover test_page.html
- revert unrelated component_index drift from PR scope

* fix(LE-906): enforce MCP lock in v2 servers

* fix(LE-906): address PR2 review comments from Gabriel

- Remove duplicate is_mcp_servers_locked helper from v1/mcp_projects.py;
  import from api/v2/mcp instead so unit tests protect production code
- Retarget test imports to langflow.api.v2.mcp.is_mcp_servers_locked
- Add test_v2_mcp_servers_unlocked_allows_non_superuser_add_patch_delete
  to cover the flag-off case (non-superuser can CRUD when gate is off)

* fix(LE-906): make MCP lock configurable in PR2

- Declare mcp_servers_locked in Settings so LANGFLOW_MCP_SERVERS_LOCKED is honored
- Add regression test to assert Settings exposes and reads mcp lock env var

* chore(ci): trigger PR2 CI
2026-05-30 03:22:07 +00:00
6049fe061c feat: Add embedded mode configuration flags (#13095)
* feat(LE-906): Config/settings plumbing

- Add 7 new boolean settings: custom_component_admin_only, embedded_mode,
  hide_logout_button, hide_new_project_button, hide_new_flow_button,
  hide_starter_projects, mcp_servers_locked (all default: false)
- All settings configurable via LANGFLOW_* env vars
- Expose all flags through ConfigResponse schema
- Fix hide_getting_started_progress to read from settings not env
- Add frontend type definitions for all 7 flags
- Initialize utility store with all new flags
- Implement hydration logic for new flags in config hook

* [autofix.ci] apply automated fixes (attempt 2/3)

* fix(LE-906): address PR1 review feedback

- replace internal ICA wording with neutral embedded-mode language
- add settings field for hide_getting_started_progress
- map hide_getting_started_progress from settings in ConfigResponse
- add regression test for embedded_mode hide-flag cascade
- revert unrelated component_index drift from PR scope

* fix(LE-906): address remaining PR1 Gabriel comments

* docs/test(LE-906): clarify embedded_mode scope for QA

- Clarify in settings docs that embedded_mode cascades UI hide flags only
- Keep mcp_servers_locked and custom_component_admin_only as explicit opt-ins
- Add config test assertions that security flags do not auto-cascade

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-29 22:58:36 +00:00
3ff3c576ca fix(ci): re-enable migration-pip-venv now that nightly bundles resolve (#13421)
The PyPI langflow-nightly is pip-installable again, so the temporary
`if: false` guard added on 2026-05-28 is no longer needed.

#13418 (release-1.10.0) and its forward-port #13419 (main) gave the
extension bundles their own nightly track: langflow-nightly now depends on
lfx-arxiv-nightly / lfx-duckduckgo-nightly / lfx-ibm-nightly, which pin
lfx-nightly==0.5.0.dev* instead of an unsatisfiable stable lfx>=0.5.0,<0.6.0.

Verified against the published dev57 wheels: a dry-run resolve of
langflow-nightly[postgresql] succeeds (549 packages, exit 0), pulling
lfx-nightly==0.5.0.dev57 via the *-nightly bundle variants.
2026-05-29 15:41:58 -07:00
2e41b7f3e2 fix(i18n): default to English regardless of browser locale (#13324)
Previously, on first load with no saved language preference, the app
used navigator.language as the fallback, causing users with Portuguese
(pt-PT / pt-BR) or other non-English browser locales to see the UI in
that language automatically. The expected behavior is that English is
always the default; users can change the language explicitly via
settings.

Co-authored-by: Antônio Alexandre Borges Lima <104531655+AntonioABLima@users.noreply.github.com>
2026-05-29 21:12:02 +00:00