* MB enhancement: Added agent pre-processing features to ingestion, can act as an LLM judge to gate ingestion based on system prompt.
* Update mb01b2c3d4e5_add_preprocessing_output.py
---------
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* Add Memory Base API: models, migrations, service, endpoints, and tests
Introduces Memory Base (MB) — a per-flow knowledge base that auto-captures
conversation history and ingests it into a Chroma vector store on configurable
thresholds.
Backend changes:
- MemoryBase + MemoryBaseSession DB models with full CRUD
- Three Alembic migrations: base tables, merge head, phase-2 fields
(embedding_model, preprocessing, preproc_model, preproc_instructions)
- MemoryBaseService: create/list/get/update/delete, session tracking,
pending-message cursor logic, mismatch detection, regenerate
- ingest_memory_task: async Chroma ingestion with cursor advance on success
- REST API (/api/v1/memories): CRUD, flush, sessions, mismatch, regenerate
- Flow output hook: on_flow_output() triggers auto-capture after each run
- Plumbing in build.py, endpoints.py, workflow.py to call on_flow_output
- deps.py: expose get_memory_base_service()
- kb_helpers.py: FS/metadata helpers used by MB service
Authored-By: Debojit Kaushik <kaushik.debojit@gmail.com>
Add dedupe_key idempotency enforcement for MB ingestion jobs
Centralizes idempotency into JobService.create_job() with a null-safe check,
removing the redundant pre-flight logic from MemoryBaseService.
Key changes:
- services/jobs/exceptions.py: new DuplicateJobError(RuntimeError) —
raised when a QUEUED/IN_PROGRESS/COMPLETED job with the same dedupe_key
exists; FAILED/CANCELLED are retryable and are excluded
- services/jobs/__init__.py: exports DuplicateJobError
- services/jobs/service.py: null-guarded dedup query inside create_job()
within the same session_scope as the insert (minimizes TOCTOU window)
- services/database/models/jobs/model.py: dedupe_key field -> index=True
- alembic/versions/36aa87831162: adds dedupe_key column + ix_job_dedupe_key
index to job table with checkfirst guards
- services/memory_base/service.py: updated key format to
"ingestion:{mb_id}:{session_id}:{first_msg_id}" for namespace isolation;
removed _has_non_retryable_job_for_dedupe_key and _has_active_job methods
and all call sites; DuplicateJobError catch in _maybe_trigger() for
silent skip on auto-capture; split regenerate() catch clauses
- api/v1/memories.py: explicit DuplicateJobError catch before RuntimeError
in flush_memory_base() for semantic clarity (both return 409)
Co-Authored-By: Debojit Kaushik <kaushik.debojit@gmail.com>
Checkpointing working version of MBs. TODO: User separation, Get messages endpoint, MB resumption midway through a chat for a session, tests.
Added messages endpoint for Memory Bases. Added pagination to sessions endpoint. Modifed messages model to include ingestion related attributes.
Added unit tests, fixed linting issues and formatting issues.
Aligned Workflows API, /run endpoint, playground to all work with Memory Bases. Created DB models asociated with tracking memory base state with sessions and jobs. Added tests, created unit tests for service and task files related to MemoryBases.
Improved concurrency of jobs, added (memory_base_id, session_id) locking to serialize jobs in case the job creation cadence moves ahead of ingestion jobs. Improved concurrency handling and moved the pending check to be live inside the ingestion job rather than a snapshot before triggering the job.
Consolidated all migrations related to memory_bases into one idempotent version and serialized all migrations form release along with memory_bases for cleanliness and maintainability.
Introduced advisory locking to address multi worker environment, added unique constraint to MB creation, added sanitization check to KB pathnames to avoid illegal directory creation.
Added partial write rollback for ChromaDB, aligned same session is used for each job to avoid dangling advisory locks.
* feat(kb): Knowledge Bases Infrastructure Overhaul
* refactor(kb): reuse Memory Base helpers to reduce duplication (#12878)
* refactor(kb): reuse Memory Base helpers to reduce duplication
Stacked on top of #12802 — consolidates the overlap between this KB
infra PR and the Memory Base work in #12417 so the two PRs land
cleanly together.
1. Chain migrations linearly
``72df732be86b_add_ingestion_run_table`` now points at
``mb00a1b2c3d4`` (the Memory Base head) instead of
``d306e5c17c41``. Both PRs had branched from the same release
ancestor, which would have required an Alembic merge migration at
integration time; with the stack in place, a single linear history
lands: ``d306e5c17c41 ← mb00a1b2c3d4 ← 72df732be86b ←
15fe9304bca7 ← e728126476a8``.
2. Delegate the path-traversal guard to the shared helper
``_validate_kb_path_containment`` now wraps
``kb_path_helpers.validate_kb_path`` instead of duplicating its
``is_relative_to`` check — the HTTPException translation and
structured log line stay here because they're route-specific, but
the traversal rule itself is defined once.
3. Reuse ``infer_embedding_provider`` during KB backfill
``knowledge_base_service.backfill_from_disk`` now falls back to
the pure-string inference helper when legacy metadata has
``embedding_model`` set but ``embedding_provider`` is
``"Unknown"``. Matches the inference the Memory Base service uses
and avoids shipping backfilled rows with
``embedding_provider="Unknown"`` when the model name is
recognizable.
4. Use ``dedupe_key`` for connector ingestions
``POST /knowledge_bases/{kb_name}/ingest/connector`` now builds a
stable SHA-256 key over (user, kb, source_type,
canonicalized-source-config) and passes it to
``JobService.create_job``. A double-click on "Ingest" or a retry
racing with an in-flight job now returns HTTP 409 instead of
spawning a duplicate run. Only the hash lands on the job row, so
credentials in ``source_config`` don't leak through ``dedupe_key``.
5. Extract ``KBIngestionHelper.write_documents_to_backend``
Backend-agnostic counterpart to
``write_documents_to_chroma`` (which is Chroma-only and used by
Memory Base). The inline retry/cancellation loop inside
``perform_ingestion`` collapses into a single helper call so
Mongo/Astra/Postgres/OpenSearch paths share identical batching,
cancellation, and exponential-backoff behaviour with the
file-upload/folder paths.
* Update templates
* Update .secrets.baseline
* Update component_index.json
* Update starter projects
* Update .secrets.baseline
* refactor(kb): migrate ingestion-source registry to AdapterRegistry (#12879)
Port the hand-rolled ``ingestion_sources/registry.py`` onto the
existing ``lfx.services.adapters.registry.AdapterRegistry`` so
third parties can publish ingestion sources as
pip-installable plugins without modifying core.
Plugin authors can now register sources two new ways in addition to
the in-tree ``register_source(SourceType.X, Y)`` call:
* Entry point group ``lfx.ingestion_source.adapters``
* ``[ingestion_source.adapters]`` in ``lfx.toml`` (or
``[tool.lfx.ingestion_source.adapters]`` in ``pyproject.toml``)
What stays the same:
* ``SourceType`` enum remains the source of truth for built-in
identifiers; API boundaries (``ConnectorIngestRequest``) still
validate against it.
* Every existing caller (``register_source`` / ``get_source_class``
/ ``create_source`` / ``registered_sources``) keeps its contract
— the six built-ins continue to self-register at import time via
``ingestion_sources/__init__``.
* ``create_source`` still returns a fresh instance per call. Sources
carry request-scoped ``user_id``/``source_config`` so
``AdapterRegistry.get_instance`` (which caches singletons) is
bypassed in favour of ``get_class``.
* Collision semantics preserved: re-registering a different class
under an existing key raises ``ValueError`` at import time.
* Error-message split preserved: unknown strings (call-site typo) →
``"Unknown ingestion source"``; known-but-unregistered →
``"not registered"``. API routes key HTTP 400 vs 500 off this.
New public helper: ``registered_source_keys()`` returns the full
set of registered keys (including third-party plugins) for callers
that want to surface plugins in a picker; ``registered_sources()``
still returns only the built-in ``SourceType`` enum members for
backward compatibility.
Tests:
* Existing 49 ingestion-source tests pass unmodified (behavior
parity with the old registry).
* 4 new tests cover the plugin surface: entry-point registration,
TOML registration, built-ins not shadowed by entry points, and
the Unknown-vs-not-registered error-message split.
* fix(kb): unstick remote-backend deletes + hydrate stale embedding metadata (#12880)
* Update component_index.json
* Template update
* fix(kb): delete opensearch metadata matches (#12873)
* scope: Chroma and OpenSearch backends
* Ruff cleanup
* format
* [autofix.ci] apply automated fixes
* test(kb): cover backend registry ingestion routing (#12870)
* [codex] Add Knowledge Backends settings (#12909)
feat: add knowledge backend settings
* feat(kb): New KB backend settings
* [autofix.ci] apply automated fixes
* Update component_index.json
* chore(logs): Improve the logging display in KB
* feat(kb): Full support for OpenSearch backend
* Update index.tsx
* test(kb): tighten ingestion run auth assertion (#12872)
* test(kb): tighten ingestion run auth assertion
* [autofix.ci] apply automated fixes
* fix(kb): surface job_id in run detail modal and report skipped runs accurately
Addresses two QA findings on PR #12872:
- BUG-1: the Ingestion Run Detail modal hid the run's job_id even
though the API returned it. Add a Run ID + Job ID block to the
modal so operators can correlate runs with the underlying job
service entry.
- BUG-2: a run that produced 0 successes but >=1 skipped item (e.g.
empty file / no extractable text) was finalized as SUCCEEDED and
surfaced via the cumulative-KB-chunks toast as "ingestion complete
- N chunks ready". Treat any run with skipped or failed items
(without a clean full success) as PARTIAL, and replace the toast
with a per-run breakdown (succeeded/skipped/failed + chunks
ingested by THIS run) shown as a notice instead of success.
Also adds a regression test for the skipped-only -> PARTIAL path in
KBIngestionHelper.perform_ingestion, and tightens types in the
existing KnowledgeBasesTab test (removes pre-existing any usage so
the staged biome no-any hook stays green).
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* test(kb): cover folder ingest allow-list wiring (#12869)
* test(kb): cover folder ingest allow-list wiring
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix(kb): allow clearing persisted separator (#12871)
* fix(kb): allow clearing persisted separator
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* Update component_index.json
* Update test_cometapi_component.py
* Revert "Update test_cometapi_component.py"
This reverts commit e789ef6bed.
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* feat: Add test connection option for backends
* Biome check
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* chore: rebake starter projects after release-1.10.0 merge
The Validate Starter Project Templates hook regenerated the embedded
component source in Knowledge Retrieval.json and Vector Store RAG.json
to match the merged retrieval.py (now imports path/metadata helpers
from _kb_paths).
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* feat(jobs): add job_metadata mirror for KB ingestion runs
Step 1 of unifying KB ingestion tracking onto the canonical 'job'
table per @dkaushik94's review feedback on PR 12935.
Adds a nullable 'job_metadata' JSON column on the 'job' table and
mirrors the 'ingestion_run' lifecycle data onto it: create_run,
mark_running and finalize_run all dual-write the same counters,
items list, status, and source config. The 'ingestion_run' table
remains the canonical source of truth for read paths in this PR;
that's deferred to the contract phase once the API/UI migrate.
Component-path ingestions (no parent Job row) skip the mirror
without raising. JobService gains an 'update_job_metadata' helper
for shallow-merge / replace semantics.
Tests: 9 new tests cover create/mark/finalize dual-write,
component-path no-op, ghost-job tolerance, and the JobService
helper's merge/replace/missing-row branches. Existing 10
ingestion-run endpoint tests still pass.
Migration: da7f6b9b638a (EXPAND, nullable column, fully reversible)
* chore: rebake starter projects after retrieval.py TYPE_CHECKING change
* feat(jobs): unify KB ingestion tracking onto job table; drop ingestion_run
Switches both reads and writes off the legacy 'ingestion_run' table
and onto 'Job.job_metadata'. The full sequence in this PR:
da7f6b9b638a (prev commit) — add Job.job_metadata column (EXPAND)
927dd37c3ea0 — backfill existing ingestion_run rows (MIGRATE)
fc7393328808 — drop the ingestion_run table (CONTRACT, reversible)
Reads/writes now go through 'Job.job_metadata'; the response shape
(IngestionRunInfo / IngestionRunDetail) is unchanged so no frontend
changes are needed. 76 tests pass.
* chore(jobs): collapse ingestion_run lifecycle migrations
The ingestion_run table never shipped to any tagged release —
verified by checking every tag from v1.9.x through v1.10.0.dev9. The
table only existed on feature branches (this one and
feat/kb-ingest-metadata). So the 'create table → backfill → drop
table' migration dance was theatre for a table no production user
ever had.
Collapses 5 migrations to 1 by deleting:
72df732be86b — add ingestion_run table
e728126476a8 — add kb_id FK to ingestion_run
927dd37c3ea0 — backfill ingestion_run → job.job_metadata
fc7393328808 — drop ingestion_run table
Repoints the chain so 15fe9304bca7 (knowledge_base) attaches directly
to mb00a1b2c3d4, and da7f6b9b638a (Job.job_metadata) attaches after
15fe9304bca7. Net schema change for production users: a single
nullable JSON column added to the job table.
Devs running this feature branch locally and carrying an
ingestion_run table in their dev DB will need to drop it manually or
nuke + re-migrate. There are no production users of this branch by
construction.
Tests: all 22 KB-related tests still pass.
* refactor(kb): drop unused VectorStoreBackend Protocol
The Protocol was exported as part of the public surface but nothing
in the codebase actually used it as a type annotation, isinstance
narrowing target, or registry constraint:
- Every concrete backend (Chroma/OpenSearch/Mongo/Astra/Postgres)
inherits from the BaseVectorStoreBackend ABC
- The registry signature requires type[BaseVectorStoreBackend]
- No call site used '\: VectorStoreBackend' or
'isinstance(x, VectorStoreBackend)'
- The only test reference (a runtime-check assertion) duplicated
the ABC isinstance check on the line above
Per @dkaushik94's review feedback: the Protocol was dead weight.
Folded its contract-level guidance ('lightweight to construct, heavy
resources released in teardown') into the ABC docstring so the
documentation isn't lost, then deleted the Protocol class and
trimmed exports.
Test surface: replaced the Protocol runtime-check assertion in
test_backends_registry.py with a BaseVectorStoreBackend isinstance
check (same intent, no Protocol needed). 9 registry tests pass.
* refactor(kb): standardize Job.asset_id on KnowledgeBaseRecord.id
Per @dkaushik94's review on list_runs_for_kb: switches the read-side
filter from a JSON-extract on Job.job_metadata.kb_name to an indexed
btree lookup on Job.asset_id. Required standardizing what asset_id
means for KB ingestion jobs.
Before — Job.asset_id was set to metadata['id'] (a UUID stored in
embedding_metadata.json), independent of the knowledge_base table's
primary key. Two id-spaces in flight, with legacy generation logic
inlined at three create_job sites.
After — Job.asset_id is set to KnowledgeBaseRecord.id everywhere.
When the KB has a DB record (the common case post-startup-backfill),
the row's primary key flows directly onto Job.asset_id. Legacy KBs
on disk only fall back to metadata['id'] (or a generated UUID) so
backwards-compat survives.
* New _resolve_kb_asset_id helper centralizes the resolution logic.
Three ingest endpoints (file upload / folder / connector) and the
cancel endpoint now go through it instead of inlining metadata['id']
reads with their own fallback dance.
* list_runs_for_kb resolves the kb_record up front and filters by
Job.user_id + asset_type='knowledge_base' + asset_id=kb_record.id
when present. Falls back to JSON-extract on kb_name only for KBs
without a DB record.
* Tests: _insert_run helper auto-provisions a KnowledgeBaseRecord so
the indexed path is exercised by default. New unit test pins the
indexed-path behavior: two Jobs with the same kb_name in metadata
but different asset_ids must not cross-contaminate the result set.
23 KB tests pass (13 service + 10 endpoint).
* refactor(kb): drop embedding_provider/model columns; model_selection is the single source
Per @dkaushik94's review: the flat embedding_provider /
embedding_model columns on KnowledgeBaseRecord were redundant
with model_selection, which already carried the same data inside
its dict. Drops the columns and routes everything through two new
helpers.
* New helpers get_embedding_provider(model_selection) and
get_embedding_model(model_selection) in
langflow.api.utils.knowledge_base_service. Both extract the
field from a model_selection payload (single-dict or list-wrapped),
with sensible fallbacks ("Unknown" / empty string).
* record_to_metadata_dict continues to emit the flat keys in the
legacy JSON shape (the API response and on-disk metadata.json
consumers still want them) — but they're now derived views over
model_selection via the helpers, not separate columns.
* KnowledgeBaseRecord model: removed the two columns. Migration
15fe9304bca7_add_knowledge_base_table.py edited to not create
them in the first place (branch has no users — same approach as the
ingestion_run lifecycle collapse).
* create_record signature drops embedding_provider /
embedding_model params. Three callers updated:
- knowledge_bases.py:create_knowledge_base folds the request's
flat fields into model_selection when the request didn't
carry one of its own.
- lfx/components/files_and_knowledge/ingestion.py:_persist_kb_record
passes model_selection directly.
- backfill_from_disk synthesizes model_selection from the
legacy on-disk flat fields (preserving the
infer_embedding_provider fallback for KBs that only have a
model name).
* On-disk embedding_metadata.json writes (in
lfx/components/files_and_knowledge/ingestion.py) and
KBAnalysisHelper.get_metadata reads of those flat keys are
unchanged — the file format keeps the flat fields for back-compat
with retrieval / memory_retrieval readers.
* Tests: ~10 sites updated to pass model_selection={...} instead
of the two flat params. test_backfill_inserts_missing_rows now
asserts on record.model_selection.get('provider').
API surface: CreateKnowledgeBaseRequest and KnowledgeBaseInfo
keep the flat fields (frontend back-compat is preserved). The
columns are gone, the helpers are the single source of truth.
31 tests pass (21 service + 10 endpoint).
* [autofix.ci] apply automated fixes
* fix(kb): standardize perform_ingestion on model_selection
CI surfaced a TypeError after the previous commit's global rename
swept up a perform_ingestion() test call too — that function still
took flat embedding_provider / embedding_model args while the test
was now passing model_selection.
Aligned the helper with the rest of the standardization: drops the
two flat params, accepts model_selection (dict | list) and derives
provider / model internally via the existing
get_embedding_provider / get_embedding_model helpers when calling
build_embeddings.
Updated all three call sites in knowledge_bases.py
(file-upload / folder / connector ingest paths) to pass
model_selection — synthesized from the legacy flat metadata fields
when the on-disk metadata predates model_selection. Reverted the one
test that the original sweep missed (test_perform_ingestion_rollback)
to the new shape.
4/4 TestPerformIngestionTask tests pass.
* Update Vector Store RAG.json
* feat(kb): user metadata on ingestion + retrieval (#12921)
* fix(frontend): rename KB upload labels and open ingest section by default
Rename "Configure Sources" heading to "Ingest Content" and the dropdown
trigger from "Add Sources" to "Add Files". Drop the "Hide Configuration"
footer toggle so the ingest section is always visible. Chunk size,
overlap, and separator inputs render disabled until at least one source
is added. Match the Add Files focus-visible ring to border-input so the
Radix-managed focus return no longer flashes a black outline.
* fix(models): mark Google + IBM embedding models deprecated
Flag the Google Generative AI and IBM WatsonX embedding model entries in
the unified model catalog as deprecated. They are not natively supported
by Knowledge Base ingestion, so users were attempting invalid
configurations. The KB upload picker and Model Providers modal now fetch
with include_deprecated=true and render a "Deprecated" badge so the
models stay visible but are clearly flagged as unsupported.
* fix(models): keep gemini-embedding-001 active
Only mark text-embedding-004 and embedding-001 as deprecated; the
gemini-embedding-001 model is still served by the Google Generative AI
v1beta endpoint, so leave it visible without the badge.
* feat(kb): user metadata on ingestion + retrieval
Surface the per-chunk user-metadata pipeline that already lives on the KB
infra branch. Adds the API + UI to collect tags at ingest time, persist
them on the ingestion_run row, filter the chunks browser by them, and
narrow KnowledgeBaseComponent retrieval through them.
Backend
- POST /{kb}/ingest gains optional metadata + per_file_metadata multipart
fields. POST /{kb}/ingest/folder gains the same fields on its JSON body.
- New parse_user_metadata / parse_per_file_metadata helpers enforce ≤16
keys, lowercase ^[a-z0-9_]{1,32}$ keys, ≤256-char values, ≤16-item
string arrays, and reject the reserved chunk-internal keys.
- FileUploadSource and FolderSource merge per-file overrides into each
IngestionItem.source_metadata; perform_ingestion now lets per-item
values win over run-level on key collision.
- ingestion_run grows a user_metadata JSON column (alembic
16a290ab1332, EXPAND, server_default '{}').
- GET /{kb}/chunks accepts repeating meta_<key>=<value> params; values
AND across keys, OR within each key. Sidesteps the global comma-split
query middleware that breaks JSON-blob query params.
- KnowledgeBaseComponent gains a metadata_filter input that post-filters
the similarity_search results client-side until per-backend
translators land.
Frontend
- New MetadataEditor renders inside the KB upload modal under Chunking
Settings for run-level tags, and inside FilesPanel as an expandable
per-file override.
- Chunks browser gets ChunksMetadataFilter (popover + chips) wired into
useGetKnowledgeBaseChunks via the new metadata_filter param map.
Tests
- 29 validator tests for the rule set, 4 ingestion-source tests for
per-file propagation + describe counts, integration tests for the
chunks endpoint AND/OR filter, 9 retrieval-side helper tests for
parse + match, 8 frontend MetadataEditor tests.
* [autofix.ci] apply automated fixes
* feat(kb): show metadata tags in upload review summary
Surface run-level tags + per-file override count in the Step 2 Review
summary so users can confirm what is about to ship onto each chunk
before clicking Create. Empty metadata stays invisible to keep the
summary tight on the common no-tag path.
* [autofix.ci] apply automated fixes
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* fix(kb): restore IngestionRun model after merge wipe
The merge from fix/kb-ingest-content-naming (based off release-1.10.0)
silently dropped the IngestionRun SQLModel + its registration in the
models package __init__. Without the model, alembic's autogenerate
guard saw the live DB column-set drift from the empty target_metadata
and raised AutogenerateDiffsDetected at startup, blocking the backend
from booting.
Restore the model files and re-add the import + __all__ entry so the
table is visible to SQLModel's MetaData on boot.
* [autofix.ci] apply automated fixes
* fix(kb): restore ingestion_run migrations after merge wipe
Same root cause as the previous fix: the merge from
fix/kb-ingest-content-naming dropped not just the IngestionRun model
but the three migrations that build the ingestion_run schema:
- 72df732be86b — create ingestion_run table
- e728126476a8 — add kb_id FK column
- 16a290ab1332 — add user_metadata JSON column
Without these, the model says the table exists while no migration
creates it, so test_no_phantom_migrations flags a phantom add_table
diff and the model/migration consistency CI step fails.
Restore the migration files from the original feature commit. Schema
is unchanged from that commit so this is byte-for-byte the state CI
last saw green on this branch.
* fix(alembic): collapse ingestion_run + job_metadata heads to a single tip
Two issues blocked test_no_phantom_migrations after the previous
restoration:
1. e728126476a8 (add kb_id to ingestion_run) chained off
15fe9304bca7 (creates knowledge_base) which left the
ingestion_run-creation migration 72df732be86b dangling as its own
head. Re-chain e728126476a8 onto 72df732be86b so the ingestion_run
chain is linear; the FK target dependency on knowledge_base is
declared via depends_on instead of down_revision.
2. After step 1 the graph still had two independent tips —
16a290ab1332 (ingestion_run.user_metadata) and da7f6b9b638a
(job.job_metadata) — neither referencing the other. Add an empty
merge migration 5238aab36810 unifying both so 'alembic upgrade
head' resolves to a single revision.
`alembic heads` now reports ['5238aab36810'] and
test_migration_execution passes locally (8 passed, 2 sqlite-only
skips).
* [autofix.ci] apply automated fixes
* fix(alembic): mark merge migration with EXPAND phase
The migration validator only accepts EXPAND / MIGRATE / CONTRACT.
Empty merge migrations are pure graph nodes (no DDL) so they belong
in EXPAND — they never drop or rewrite data.
* Update component_index.json
* fix(frontend): align KB upload modal test text with i18n strings
CI surfaced a TestingLibraryElementError — the test was looking for
'Ingest Content' / 'Add Files', but the merge from
feat/kb-v1-db-connectors switched those to i18n strings
('Configure Sources' / 'Add Sources' via t('knowledge.configureSources')
etc.).
Updates the two assertions and the surrounding test names to match
the current rendered copy. 71/71 KnowledgeBaseUploadModal tests pass.
* feat(kb): metadata input on flow KnowledgeIngestionComponent
Adds a Metadata advanced input accepting a JSON object. The decoded
dict is JSON-encoded once per run and stamped onto every chunk's
source_metadata key — same shape the API path writes — so the chunks
browser metadata filter and KnowledgeBaseComponent.metadata_filter
work uniformly across upload, folder, and flow-driven ingestion.
Malformed JSON or non-object payloads log a warning and skip the
metadata stamp rather than failing the run.
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* chore: rename knowledge backends to db providers
* Linting cleanup
* Update dbProviderConstants.ts
* fix: icon assets
* Update index.tsx
* fix: Ensure selection of default provider
* Update indices
* chore: Consolidate database migrations
* Update kb1a2b3c4d5e_add_knowledge_base_schema.py
* fix: Ingestions from component behave same
* Ruff fixes
* More ruff fixes
* Update Vector Store RAG.json
* [autofix.ci] apply automated fixes
* Starter project update
* fix: Address two P1s for folder access
* Regen other deps
* fix(knowledge-base): pin review modal height, poll active ingestion runs, rename labels (#12977)
* fix(knowledge-base): pin review modal height and poll active ingestion runs
- Lock the upload modal to a fixed `h-[690px]` on the Review step so the
envelope no longer resizes as users page through chunk previews of
varying length. Step 1 keeps its `min-h-*` flex behavior.
- Add drawer-scoped polling to the Ingestion Runs section: refetch every
5s while any run is non-terminal (`pending`/`running`), refetch on
drawer open, and stop entirely once all rows are terminal or the
drawer closes. Background tabs are skipped via
`refetchIntervalInBackground: false`.
* fix(knowledge-base): rename Configure Sources to Ingest Content and Add Sources to Add Files
Update the English locale to align the upload modal labels with the
agreed wording:
- `knowledge.configureSources` / `knowledge.helpConfigureSources`:
"Configure Sources" -> "Ingest Content"
- `knowledge.addSources` / `knowledge.addSourcesTitle` /
`knowledge.submitAddSources`: "Add Sources" -> "Add Files"
`knowledge.sourcesLabel` ("Sources") is intentionally left as-is. Other
locales remain untouched and will be picked up via the existing
translation flow.
* Update component_index.json
* Update KnowledgeBaseUploadModal.test.tsx
* Update component_index.json
* fix(knowledge-base): surface metadata on chunk cards and make filter self-documenting (#12993)
Closes the QA-flagged UX gap where the chunks-browser metadata feature
was effectively only usable by API users. Two fixes bundled:
- Chunk cards now render the source file name + user-supplied tags as
chips, so the metadata applied at ingestion time is visible without
hitting the API. Reserved ingestion-internal keys (chunk_index,
job_id, etc.) stay hidden via a frontend mirror of
KB_METADATA_RESERVED_KEYS.
- The "Filter by metadata" popover no longer asks the user to type
blind. A new GET /knowledge_bases/{kb}/metadata/keys endpoint scans
chunks server-side and returns distinct user keys + a sample of
values (capped at 50/key with a truncated flag). The popover binds
both inputs to a <datalist>, so users get native suggestions on
click while keeping free-text typing for keys not yet ingested.
Refetches on every popover open so freshly-ingested metadata shows
up without a hard page refresh.
Test coverage: 41 backend tests in TestKnowledgeBaseAPI (3 new for
the keys endpoint), 30 frontend tests in sourceChunksPage (file_name
rendering, user-tag chips, reserved-key hiding, array-valued tags,
malformed JSON, datalist population, validation, refetch-on-open).
* feat(knowledge-base): KB row settings shortcut + ingestion history in Update modal (#13012)
Two QA-flagged UX gaps on the Knowledge Bases tab:
- "Update Knowledge" was buried inside the row's overflow menu — promoted
to a one-click settings icon directly on the row, with the dropdown
retained for the rest of the row actions. Disabled while ingesting,
matching the dropdown rule.
- The "Update Knowledge" modal gave no visibility into what a KB
already contained, so users couldn't tell if a file had been ingested
before. New collapsible "Previously ingested" panel above the upload
fields shows each prior run with its user-typed source name, type,
date, status, and counts. Empty/loading/error states covered.
Backend additions
- IngestionRunInfo gains a ``source_name`` field, populated in
``_run_row_to_info`` from ``RunRow.source_config["source_name"]``.
Whitespace-only values collapse to ``None`` so the frontend renders
the type label as a clean fallback.
Frontend
- IngestionHistoryPanel uses ``useGetIngestionRuns`` with
``staleTime: 0`` + ``refetchOnMount: "always"`` so the list is fresh
immediately after an ingest, mirroring the cache fix from #12993.
- StepConfiguration receives ``kbName`` and renders the panel only when
``isAddSourcesMode && kbName``.
- Run row prefers ``source_name`` (e.g. "Test6") with the type label
("File Upload") as a small subtitle when both are present.
Tests
- Backend: new ``test_exposes_source_name_from_source_config`` covers
populated, missing, and whitespace-only cases.
- Frontend: 6 IngestionHistoryPanel tests (empty/loading/error/render
states, source_name preference, whitespace handling, collapse,
hook args). 3 new column tests for the row settings icon. Existing
column tests migrated to the new ``kb-row-actions-trigger`` testid.
* Update Vector Store RAG.json
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update IngestionHistoryPanel.tsx
* fix(memory-base): resolve embeddings from static registry, fix Google Gemini provider inference (#13023)
Memory Base retrieval failed with "Embedding model X for provider Y not
found." for every supported embedding model. Two interacting bugs:
1. KBIngestionHelper.build_embeddings looked the model up in the
user-filtered catalog returned by get_embedding_model_options. That
catalog is empty whenever the per-user credential lookup silently
returns an empty enabled-providers set — which happens when the call
bridges from an async event loop into a worker thread. Fall back to
the static EMBEDDING_PROVIDER_CLASS_MAPPING / EMBEDDING_PARAM_MAPPINGS
registry instead: the metadata persisted in embedding_metadata.json
is the source of truth, and the runtime API key is still resolved by
get_embeddings.
2. infer_embedding_provider mis-classified models/gemini-embedding-001
as "OpenAI" (no Google pattern matched the gemini-embedding name) and
the Google patterns returned "Google" instead of the canonical
"Google Generative AI" registered in EMBEDDING_PROVIDER_CLASS_MAPPING.
Look the model up in the unified catalog first, and update the
pattern table to use the canonical provider name plus the
"models/..." prefix Google uses.
Adds regression tests covering OpenAI text-embedding-3-*, Google
gemini-embedding-001, and the previously-empty user-filtered catalog
path.
* feat: Adding Chroma Cloud as Vector DB provider (#13030)
* Adding Chroma Cloud support for Knowledge Bases.
Added Chroma cloud support. Users can configure Chroma Cloud and switch between providers especially between ChromaDB local and Cloud.
* [autofix.ci] apply automated fixes
* Update Vector Store RAG.json
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix(kb): align OpenSearch KB default vector field with canvas component (#13009)
* fix(kb): align OpenSearch KB default vector field with canvas component
The KB OpenSearch backend defaulted to 'vector_field' while the
canvas OpenSearch component defaults to 'chunk_embedding'.
Combining KB ingestion with the canvas component on the same index
failed with "Field 'chunk_embedding' is not knn_vector type".
Switch the KB backend (and the corresponding frontend defaults /
fallbacks in the DB Providers settings page) to 'chunk_embedding'
so both layers agree out of the box. Operators who already
ingested into 'vector_field' can still override via backend_config
or the global variable.
* test(kb): pin OpenSearch backend default vector_field to 'chunk_embedding'
Adds unit coverage for ``OpenSearchBackend`` so the canvas / KB
field-name alignment can't silently regress:
* asserts ``DEFAULT_VECTOR_FIELD == 'chunk_embedding'``,
* asserts ``_build_vector_store`` passes ``vector_field='chunk_embedding'``
to the LangChain wrapper when no override is provided,
* asserts an explicit ``backend_config['vector_field']`` still wins,
* parametrized table for None / empty / truthy resolution.
* fix(kb): cancel in-flight ingestion jobs on KB delete (#13003)
* fix(kb): cancel in-flight ingestion jobs on KB delete
Without this, deleting a KB while an ingestion job is running would
leave the deleted KB reappearing in the list a few seconds later. The
background job kept polling `is_job_cancelled` (it never tripped) and
its open backend client would auto-recreate the KB directory on the
next chunk write — Chroma's PersistentClient creates its sqlite parent
on demand. The ingestion then rewrote `embedding_metadata.json` into
the recreated dir, and the list endpoint's disk-fallback path
re-discovered it as a valid KB.
Add `JobService.cancel_in_flight_jobs_by_asset` and call it from the
single + bulk KB delete endpoints before storage teardown. The
ingestion's existing cancelled-handler then runs cleanup, so no
further chunks are written.
* fix(kb): DB-first delete + .kb_deleted sentinel hides locked-disk dirs
Two related bugs around KB delete on Windows when Chroma still holds an
exclusive SQLite lock:
1. delete_storage() ran before the DB row delete, so a locked rmtree
turned into HTTP 500 and left the row dangling. The dir was
partially cleaned (non-locked files removed) but the row plus the
SQLite stub kept the KB visible to listings.
2. The disk-scan fallback in get_knowledge_bases() and the API list
endpoint had no way to tell a "really still here" dir from a
"we tried to delete this and rmtree was blocked" dir, so a deleted
KB kept reappearing in the canvas component dropdown.
Fix:
* KBStorageHelper.delete_storage() now drops a .kb_deleted sentinel
file inside any directory it could not physically remove. Replaces
the previous .deleted_<name>_<timestamp> rename fallback (which
failed under the same Windows lock conditions on the rename itself).
* delete_knowledge_base() and delete_knowledge_bases_bulk() reorder
to DB-first (Option A). When the DB delete succeeds and storage
cleanup fails, the endpoint returns 200 with a warning instead of
500: the user no longer sees the KB, and the operator gets a clear
follow-up message.
* lfx.base.knowledge_bases.knowledge_base_utils.get_knowledge_bases()
skips dirs carrying the sentinel. The API-side list endpoint's
disk-scan fallback does the same via KBStorageHelper.is_kb_dir_deleted.
* create_knowledge_base() detects sentinel-marked dirs and surfaces a
clearer 409 ("previously deleted; awaiting restart cleanup") instead
of the bare "already exists", and clears any stale sentinel after a
successful mkdir as a defensive no-op.
* Cross-package contract: the lfx layer cannot import from the langflow
API package, so it inlines the sentinel filename string. A unit
test in test_kb_storage_deletion.py asserts the inlined value matches
KB_DELETED_SENTINEL.
Tests: 88 KB-related tests pass (test_kb_storage_deletion.py +
test_knowledge_bases_api.py). Two existing tests updated to assert the
new 200-with-warning shape; the previous 500-on-storage-failure shape
was the buggy behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: Stay deleted when sentinel shows up
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: enforce session isolation in MemoryBase retrieval and purge chunks on session delete (#13037)
* fix: enforce session isolation in MemoryBase retrieval and purge chunks on session delete
Two related bugs in the MemoryBase feature (#12903):
1. Filter by Session: switch the Chroma where-clause to the canonical
`{"session_id": {"$eq": ...}}` form and force-coerce the toggle to
a real bool. Both shapes are accepted by chromadb today, but the
explicit form removes any ambiguity if a non-bool slips into the
attribute and would otherwise be truthy (e.g. the literal string
"false").
2. Ghost chunks on session delete: the message-session delete
endpoints only wiped MessageTable rows, leaving every embedded
chunk behind in Chroma. New sessions then pulled the deleted
data back through the retrieval component. Wire the delete
endpoints to a new `MemoryBaseService.purge_session_data` that
walks the user's Memory Bases, deletes chunks where
`session_id == X`, and clears the corresponding tracking rows
(MemoryBaseSession + MemoryBaseWorkflowRun). Chunk-delete
failures are logged but never abort the user-visible message
delete.
* Update ingestion.py
* fix: exclude component error messages from MemoryBase ingestion (#13039)
Component errors (e.g. an embedding API failure) were persisted to the
message table with `error=True` / `category='error'` and the MemoryBase
ingestion task pulled them in like any other message, embedding the error
text into Chroma. Retrieval then surfaced these chunks as context.
Filter both signals at the SQL fetch boundary so errors never enter the
document-build / Chroma write path. The cursor still advances past any
later non-error message, so skipped error rows are not reconsidered.
Includes regression tests covering the error flag, the error category,
and cross-session/cross-flow isolation.
* fix(kb): require OpenSearch username/password in DB Provider settings (#13005)
The DB Providers settings panel marked Username and Password as
optional, but the OpenSearch runtime components (canvas vector-store +
KB backend) default to basic auth and raise "Auth Mode is 'basic' but
username/password are missing" when the corresponding global variables
aren't set. Operators who filled in only the cluster URL hit a
confusing error far downstream from the settings screen.
Mark the credential fields required so the existing canSave / save
validation in DBProvidersPage forces them up-front. Operators with
auth-less clusters (sigv4 / upstream proxy) can still satisfy the
form with a placeholder value — OpenSearch ignores credentials when
auth is not enforced.
Also assert the asymmetric (one-field-only) case in tests, since the
backend's tuple-based http_auth silently treats partial creds as
no-auth.
* Update component_index.json
* Update component_index.json
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update kb1a2b3c4d5e_add_knowledge_base_schema.py
* [autofix.ci] apply automated fixes
* fix: templates
* fix: Session filtering for memory bases
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update component_index.json
* chore: auto-bake note keys and regenerate backend locales/en.json [skip ci]
* Update test_memory_retrieval.py
* test: pin OpenSearch backend filter=None drop behavior
The OpenSearch backend's similarity_search override drops `filter` when
None/empty so LangChain's OpenSearchVectorSearch wrapper never injects
`"filter": null` into the k-NN query body. If that regresses, every KB
retrieval against OpenSearch fails with x_content_parse_exception and the
upstream flow loses both KB and any downstream context — which is the most
likely OpenSearch-only symptom shape behind Rafael's report.
These tests pin:
- filter=None: kwarg is dropped from both asimilarity_search and
asimilarity_search_with_score
- filter={}: also dropped (k-NN parser rejects empty objects)
- filter={truthy}: forwarded unchanged
* fix(knowledge-base): KB UI polish — drawer + row dropdown (#13069)
* fix(knowledge-base): keep drawer open when interacting with row dropdown
The KnowledgePage outside-click handler treats clicks on Radix portal
content (dropdown menus, popovers, dialogs, tooltips) as outside the
drawer and closes it. Because dropdown content lives on document.body,
clicking a menu item triggered closeDrawer, the layout reflowed (mr-80
toggled off), AG-Grid resized, and the row re-rendered—tearing down the
open dropdown before the click event reached the menu item. Result:
"View Chunks" and other row actions required two clicks.
Skip the dismissal when the mousedown target is inside any Radix popper
portal, menu, dialog, or tooltip.
* refactor(knowledge-base): rename row action to "Ingest Files" with FileUp icon
The row shortcut and dropdown item previously used a gear icon and the
label "Update Knowledge". The gear implies settings, but the only action
behind it is adding/ingesting files. Switch both the icon (Settings →
FileUp, RefreshCw → FileUp) and the label ("Update Knowledge" → "Ingest
Files") so the affordance matches the action.
Tooltip, aria-label, and dropdown item label updated. Tests adjusted
accordingly.
* fix(knowledge-base): stop clipping stacked Layers icon in Name column
The Name cell renderer applied "-mx-[3px]" to the row icon, pulling it
3px outside its layout box. For KBs with multiple source types the icon
becomes "Layers" (stacked), which has visible content right up to its
bounding box. With the negative margin, the icon's left edge crossed
AG-Grid's cell overflow boundary and got clipped.
Drop the negative margin, give the cell wrapper a 4px left padding for
breathing room, and tighten the icon→text gap from 4 to 3 so total
horizontal layout stays close to the original.
* fix(knowledge-base): give ingestion history rows room to breathe
Rows in the Previously Ingested panel packed the status badge, source
name, type label, and time onto a single line. Long source names + long
type labels (e.g. "Chroma Cloud Collection") clipped against the time
stamp and made the row hard to scan.
Restructure each row into three stacked sections:
1. status badge (left) · relative time (right)
2. source name (sm/medium) with the type label as a quiet subtitle
3. succeeded / failed / skipped counts · chunks
Row padding bumped from p-2 to p-3 and inter-section gap from gap-1 to
gap-2 to match the new hierarchy.
* refactor(knowledge-base): drop Source Files / Linked Flows placeholders
Both sections in the KB inspection drawer always rendered the same empty
state ("No source files available." / "No linked flows available.")
because neither field is wired up on the backend yet. Showing empty
placeholders adds noise without giving the user information.
Remove the two sections from the drawer. Update the mock + assertions
in KnowledgeBaseDrawer.test.tsx so the test stops claiming the removed
strings should render.
* fix(knowledge-base): give drawer ingestion runs the same room to breathe
The drawer's Ingestion Runs rows had the same cramped single-line layout
the modal panel had before the previous commit: status badge + source
type + time crammed onto one row, counts beneath. Match the new pattern
so the two surfaces feel consistent.
Each row now stacks:
1. status badge (left) · relative time (right)
2. source type label (sm/medium)
3. succeeded / failed / skipped counts · chunks · bytes
Row padding bumped p-2 → p-3 and inter-section gap gap-1 → gap-2.
---------
Co-authored-by: Eric Hare <ericrhare@gmail.com>
---------
Co-authored-by: Debojit Kaushik <kaushik.debojit@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: keval shah <kevalvirat@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: add apscheduler and cryptography for litellm proxy_server imports
When langflow-ide logs through litellm (e.g. to langfuse), litellm imports
its proxy server module, which requires `apscheduler` and `cryptography` at
runtime. The natural fix would be `litellm[proxy]`, but that extra pins
boto3>=1.40.76, which collides with our `aioboto3` transitives on
release-1.10.0. Adding the two missing modules directly keeps the install
working without taking on the rest of the proxy server runtime.
Fixes#12228
* test: guard apscheduler and cryptography in litellm extra
Asserts the `litellm` optional-dependency group keeps the runtime modules
needed for langfuse logging via litellm.proxy.proxy_server (#12228).
---------
Co-authored-by: Jah-yee Agent <agent@openclaw.ai>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: gracefully handle pyperclip failure in headless/remote environments (fixes#12341)
The langflow api-key command failed in Docker/SSH environments because pyperclip
could not find a clipboard mechanism. Wrap the clipboard copy in a try/except so
the API key is still displayed when clipboard access is unavailable.
* [autofix.ci] apply automated fixes
* test: cover pyperclip failure path in api_key_banner (#12341)
Regression guard ensuring the API key is still displayed when the
clipboard mechanism is unavailable (Docker/SSH/headless), plus a
drift guard for the happy-path clipboard hint.
* fix: log clipboard failure instead of silent pass (ruff S110)
Replaces the bare `except: pass` with a debug log so the failure is
diagnosable in headless environments without spamming users with
warnings about an expected condition.
---------
Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: strip reserved 'code' param before build() in component reuse path (#8610)
Root cause: When a custom component is reused (self.custom_component already exists),
get_params() returns params that still include the reserved 'code' field. Unlike the
instantiate_class path which pops 'code' before use, the reuse path passes it through
to build_custom_component, which calls build(**params) — causing TypeError for
components whose build() method does not accept **kwargs.
Made-with: Cursor
* test: add regression tests for custom component code parameter reuse
Verifies that TypeError no longer occurs when reusing custom components
with the code parameter in both instantiate and reuse paths.
Per CodeRabbit review on PR #12712.
Made-with: Cursor
* [autofix.ci] apply automated fixes
* fix: also strip 'code' from build params in langflow loading duplicate
Mirror the lfx fix in the parallel src/backend/base/langflow/interface/
initialize/loading.py copy. In production this build_custom_component is
dead code (only update_params_with_load_from_db_fields is imported from
this module), but keeping the duplicate in sync avoids future drift and
the same TypeError if any caller starts using it.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: use stable UUIDs for Qdrant document IDs to prevent duplicate ingestion (fixes#12641)
* fix: deterministically serialize Qdrant doc metadata for stable IDs
Use json.dumps with sort_keys=True and the project's custom_serializer
instead of str(sorted(metadata.items())). The previous form relied on
Python's repr for nested dicts/sets/datetimes/Decimals, which is unstable
across processes (PYTHONHASHSEED for sets) and does not recurse into
nested dict ordering.
* test: add unit tests for Qdrant stable-ID generation
Cover the four invariants of the deterministic UUID5 derivation:
- same content + metadata yields the same UUID across runs (uuid5)
- metadata insertion order is irrelevant (sort_keys=True)
- different content yields different UUIDs
- non-primitive metadata (datetime, Decimal) serializes via custom_serializer
---------
Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* deps: add opentelemetry http client instrumentation packages
Add opentelemetry-instrumentation-requests and opentelemetry-instrumentation-urllib3
to enable W3C TraceContext propagation on outgoing HTTP calls.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat(tracing): propagate W3C TraceContext on outgoing HTTP calls
Instrument requests and urllib3 HTTP clients with the tracer provider
so that traceparent headers are injected on outgoing HTTP requests
(e.g., to LLM APIs). This enables end-to-end distributed tracing.
Both Arize Phoenix and LangWatch tracers now:
- Call _instrument_http_clients() during setup
- Call _uninstrument_http_clients() during cleanup
- Handle missing packages gracefully with debug logging
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(tracing): add tests for HTTP client trace context propagation
Verify that:
- RequestsInstrumentor and URLLib3Instrumentor are called during tracer setup
- Instrumentors are uninstrumented during cleanup
- traceparent headers are injected with valid W3C format on outgoing requests
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(tracing): address ruff linting issues in trace context propagation
- Use contextlib.suppress instead of try-except-pass in uninstrument methods
- Fix nested with statements using parenthesized context managers
- Fix unused arguments by prefixing with underscore
- Add timeout to requests.get calls
- Skip integration tests that mock at wrong layer (Session.send bypasses OTel)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(tracing): make HTTP client instrumentation process-scoped with ref-counting
The OpenTelemetry RequestsInstrumentor and URLLib3Instrumentor monkeypatch
globally, but tracers were calling instrument()/uninstrument() per instance.
If multiple graph runs overlapped, the first run to end() would uninstrument
globally and break context propagation for other active runs.
Changes:
- Add HTTPClientInstrumentationManager singleton with reference counting
- enable() increments count, instruments only on first call
- disable() decrements count, uninstruments only when count reaches zero
- Replace silent exception suppression with proper logging for debugging
Addresses PR review feedback from Qodo.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* initial draft
* update DS Star agent
* chore(deps): use OpenDsStar 1.0.1 from PyPI
* refactor: move OpenDsStar to namespace, update imports, remove agents shim
* update open ds star version
* improved logging
* improved logging
* fix: propagate source file path for DataFrame inputs in ingestion components
DataFrame from CSV/Excel reads now carries the source file path via
pandas attrs, so IngestionDescriber and FileContentRetriever can
resolve the original file instead of searching per-row data keys.
* feat: add files_ingestion component bundle and bump OpenDsStar to 1.0.8
Register files_ingestion as a sidebar bundle in the frontend and
component registry. Update OpenDsStar dependency from 1.0.6 to 1.0.8.
* Add file content retriever component and tests
* Fix: Add tool_mode=True to FileContentRetriever outputs to prevent execution during build phase
* Fix: Return empty DataFrame instead of raising error when file_path not provided during build
* Update file content retriever component
* refactor: rename IngestionDescriberComponent to FileDescriptionGeneratorComponent
- Renamed class from IngestionDescriberComponent to FileDescriptionGeneratorComponent
- Updated component name attribute and display name
- Renamed files: ingestion_describer.py -> file_description_generator.py
- Renamed test file: test_ingestion_describer.py -> test_file_description_generator.py
- Updated all imports and references
- Rebuilt component index
BREAKING CHANGE: Component class name changed. Existing flows using IngestionDescriberComponent will need to be updated.
* fix: improve FileContentRetriever tool descriptions for CodeAct agent
- Add explicit warnings that tools expect string file paths only
- Clarify that search results/Data objects should not be passed
- Change error handling to raise ValueError (agents can handle exceptions)
- Add build-phase safety (return empty results when no path provided)
- Update tests to expect exceptions instead of error messages
This fixes CodeAct agent failures where it was trying to pass search
results (list of Data objects) to retrieve_content_as_dataframe()
instead of extracting the file path string first.
* fix: File Content Retriever now finds DataFrames with file_path in columns
- Added caching for file maps to avoid rebuilding on each method call
- Enhanced _get_file_maps() to check both attrs['source_file_path'] and 'file_path' column
- Fixes agent issue where DataFrame from Read File component couldn't be found
- Added test for DataFrame with file_path in columns scenario
- All 30 tests pass (26 passed, 4 skipped)
* fix: use to_csv() instead of to_string() for DataFrame text representation
Fixes agent hanging issue when retrieving file content as DataFrame.
The problem was that to_string() creates a formatted table representation
that cannot be parsed as CSV. When agents called retrieve_content_as_dataframe(),
it would try to parse this formatted text with pd.read_csv(), causing a
ParserError and infinite retry loop.
Changed to use to_csv(index=False) which produces valid CSV format that can
be parsed back into a DataFrame when needed.
- Maintains optimal performance by returning original DataFrame from dataframe_map
- Only uses CSV text representation as fallback
- All 26 tests pass
- Works correctly for both text and tabular files
* add agent template
* improve agent template
* bump OpenDsStar to 1.0.10 and update ingestion components
Upgrade OpenDsStar dependency from 1.0.8 to 1.0.10 in both langflow-base
and lfx pyproject.toml. Update file content retriever, file description
generator, component tool, and custom component with related improvements.
Expand test coverage for file content retriever.
* fix FileContentRetriever multi-file data retrieval and add code_timeout
The FileContentRetriever was broken for the multi-file case (when Read File
outputs a DataFrame with file_path + text columns). Three bugs were fixed:
1. Multi-file DataFrame mapped to wrong data: When Read File produced a
summary DataFrame (one row per file, columns = [file_path, text]), the
retriever mapped each file_path to the entire 2-row summary DataFrame
instead of extracting per-file content. The agent would get 2 rows
instead of thousands of actual data rows. Fixed by iterating rows and
extracting each file's text into text_map.
2. CSV text now eagerly parsed into DataFrames: For .csv/.tsv files in
text_map that lack a pre-built DataFrame, _get_file_maps() now parses
the text into a DataFrame eagerly during initialization. This makes
retrieve_content_as_dataframe work for the multi-file case without
needing upstream structured DataFrames.
3. Maps built once, not per tool call: The file maps were rebuilt on every
tool call because component_tool.py deepcopy's the component, and the
maps were only built lazily during retrieval. Now the early-return paths
(empty file_path) eagerly call _get_file_maps() during component build
time, so the cached maps survive deepcopy. This is critical for
scalability with hundreds of large files.
Additional changes:
- Removed _is_likely_file_content heuristic that was incorrectly filtering
out valid file content
- Removed "Message" from input_types (was never handled)
- Removed redundant inner import of DataFrame
- Shortened error messages (cap available files to 5)
- Added warning log for unsupported input types
- Lazy CSV conversion: no longer eagerly converts structured DataFrames to
CSV text in _get_file_maps, only converts on demand in retrieve_content
- Added code_timeout input to OpenDsStarAgentComponent (default 60s, was
hardcoded 30s) to prevent timeouts with large datasets
- Added 19 unit tests covering all retrieval paths, caching, and the
multi-file bug reproduction
* disable as_dataframe tool from vector store to fix agent confusion
The Chroma vector store's as_dataframe output was exposed as an agent tool,
returning a 2-row file index DataFrame (file_path + text columns). The agent
mistook this for actual data and tried to query 'price' on it, causing every
query to fail with KeyError.
Set tool_mode=False on the as_dataframe output so only search_documents is
exposed as a tool from the vector store. The agent now follows the correct
workflow: search_documents → get file path → retrieve_content_as_dataframe.
* Revert "disable as_dataframe tool from vector store to fix agent confusion"
This reverts commit e21c5dc649.
* bump OpenDsStar to 1.0.15, improve ingestion component tool descriptions and caching
- Upgrade OpenDsStar dependency from 1.0.10 to 1.0.15
- Add __deepcopy__ to FileContentRetriever to preserve cached maps across tool invocations
- Add JSON format support for eager DataFrame parsing
- Improve tool docstrings on FileContentRetriever and VectorStore for better agent usage
- Enable tool_mode on VectorStore search and table outputs
* support Message input in FileContentRetriever and FileDescriptionGenerator
- Add file_path to Message metadata in base_file._extract_file_metadata
- Accept Message type in FileContentRetriever.file_data input and handle
it in _build_file_maps to extract file_path and text content
- Handle Message input in FileDescriptionGenerator to extract file_path
- Add tests for Message input: basic, missing file_path, and mixed types
* fix FileContentRetriever not appearing in component sidebar
- Remove forward reference to FileContentRetrieverComponent in __deepcopy__
that broke the exec-based template builder, silently preventing registration
- Remove unnecessary cast() call in __deepcopy__
- Regenerate component_index.json (362 -> 363 components)
- Update template JSON edge handles to include Message in input_types
* add persistent directory, real-time logs, tool descriptions, and Merge Flows component
FileContentRetriever:
- Add persistent directory support for saving/loading text and DataFrame maps to disk
- Split text storage into individual files (texts/) instead of single JSON
- Guard against None file_data during tool deepcopy calls
- Warn when maps are empty after building
- Make Persistent Directory a visible (non-advanced) input
FileDescriptionGenerator:
- Switch from subprocess.run to Popen with real-time stderr streaming
- Add configurable timeout input (default 1800s, was hardcoded 600s)
- Stream subprocess progress to UI via self.log()
Tool descriptions:
- Add info field to Output model (excluded from serialization to avoid template pollution)
- Fix stale tool metadata overwriting fresh descriptions in component.py
- Each tool output now gets its own description instead of generic component description
New components:
- Add Merge Flows component to trigger multiple upstream flows from a single node
Tests:
- Add persistence tests (round-trip, skip duplicates, add new files, corrupted JSON, regression)
- Update FileDescriptionGenerator tests for Popen-based subprocess
* update Structured Data Agent template from UI re-export
* update Structured Data Agent template
* update Structured Data Agent template
* update Structured Data Agent template
* update Structured Data Agent template
* update Structured Data Agent template
* fix chardet perf, pipe deadlock, auto-sync, Chroma errors, and ingestion failure detection
Performance:
- Fix chardet.detect() scanning entire files (800s for 1.5GB) — sample first 100KB only
- Default ascii detection to utf-8 (superset) to handle non-ASCII chars beyond sample
- Fix subprocess stdout pipe deadlock — drain both stdout and stderr concurrently via select
FileContentRetriever:
- Add auto-sync: remove persisted entries for files no longer in file_data input
- Clean up orphaned files from persistent directory on save
FileDescriptionGenerator:
- Fail with clear error when descriptions are not generated (grouped by error reason)
- Increase default timeout from 1800s to 3600s via _DEFAULT_TIMEOUT_SECONDS constant
- Remove fast-path code (moved to OpenDsStar)
Chroma:
- Add clear error messages for missing persist directory, corrupted DB, and initialization failures
- Recreate persist directory if deleted between runs
Other:
- Add opendsstar_cache to .gitignore
* Chore(release): merge release 1.9.0 into main (#12710)
* test: add upgrade migration check to ci (#12061)
* Add upgrade migration check to ci
* [autofix.ci] apply automated fixes
* Add fetch step
* ruff
* Add merge migration
* Revert "Add merge migration"
This reverts commit fd32424739.
backups
* coderabbit suggestions
1. Shell hardening in workflow - set -euo pipefail, full path grep, quoted variables
2. _WORKSPACE_ROOT extracted as module constant (also addresses Cristhianzl's review comment about parents[5] duplication)
3. git missing returns None instead of raising FileNotFoundError
4. # noqa: S603 added to subprocess.run (fixes the Ruff CI failure)
5. FK noise filtering now also compares target table/column, not just ondelete/onupdate
6. Removed redundant git fetch origin main step (fetch-depth: 0 already fetches all branches)
7. Deduplicated Alembic config creation in _get_main_branch_head (moved before the if branch)
8. Simplified dict type hints (removed unnecessary dict[tuple, object])
* test: improve migration tests from PR review feedback
- Narrow broad except clause to only wrap subprocess.run call
- Add specific error messages for multi-head and unresolvable revisions
- Remove redundant hardcoded schema test (covered by compare_metadata)
- Fix SQLite FK noise filter to skip ondelete/onupdate comparison
- Add downgrade verification to test_upgrade_from_main_branch
- Add test file and workflow to CI trigger paths
- Add prompt for follow-up PostgreSQL migration test PR
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* add engine check on downgrade
* [autofix.ci] apply automated fixes
* fix: harden CI error handling and test robustness
- Set validationPassed=false when validator crashes so CI fails instead of passing silently
- Wrap GitHub API calls in try-catch so comment-posting failures don't mask validation results
- Preserve git stderr in warnings for better CI debugging
- Add defensive handling for unexpected FK constraint shapes in SQLite noise filter
- Clean up SQLite WAL/SHM/journal companion files in test teardown
* Add explicit fetch to main
* ruff
* [autofix.ci] apply automated fixes
* Add sqlite filter tests and remove redundant fetch
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(deployments): unify payload passthrough from api to adapter (#12190)
* feat(deployments): unify dynamic payload passthrough across api and adapter
* use datatime.timezone for python3.10 compatibility
* use appropriate type vars in slots and sanitize error message
* tweaks to schemas
* use policy to avoid dump churn
* fix: allow clearing Max Tokens field with Backspace/Delete (#12198)
* fix: allow clearing Max Tokens field with Backspace/Delete
Empty string input was being converted to 0 via Number(""), which
triggered the min-value guard and snapped the field back to 1 before
onChange could propagate. Adding an early return for empty input lets
the field clear correctly, propagating null (no limit) downstream.
* test: add IntComponent tests for handleInputChange clearing behavior
Covers the regression where Backspace/Delete was blocked by the
min-value guard, and verifies that below-min values still clamp
correctly.
* fix: Resolve CodeQL false positives for path injection and URL substring sanitization (#12201)
* fix: Add explicit left/right DataFrame inputs for merge operations (#12177)
* add explicit left/right DataFrame inputs for merge operations
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* ruff style and checker fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix: add dict to allowlist preventing TableInput data loss (#12074)
* fix: add dict to allowlist preventing TableInput data loss
Co-Authored-By: DeyLak <DeyLak@users.noreply.github.com>
* test: add regression test for TableInput list[dict] preservation
Regression test for: https://github.com/langflow-ai/langflow/issues/12062
Co-Authored-By: DeyLak <DeyLak@users.noreply.github.com>
---------
Co-authored-by: DeyLak <DeyLak@users.noreply.github.com>
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
* chore: update pyproject versions 1.9.0
update pyproject versions 1.9.0
* fix: 1.9.0 nightly
1.9.0 nightly fix
* feat: add wxO deployment adapter (#12079)
* checkout wxo deployment adapter implementation
* checkout wxo deps and flow reqs impl
* clean up / minor refactor with updated tests
* major refactor: split up the implementation into folders and files
* clean up logic
* refactor and clean up of execution logic, improve type safety of "retry_create" helper, and remove dead code
* remove unused wxo service file
* remove "provider_name" arg and references
* fix: harden watsonx orchestrate deployment adapter for PR readiness
- Fix rollback AttributeError when agent_create_response is None
- Fix NoneType access on params in list() when called without params
- Fix inconsistent error types: use DeploymentNotFoundError in get/update
- Fix typo "occured" -> "occurred" in all error prefix messages
- Fix variable shadowing of fastapi.status in get_status()
- Fix pre-existing test bugs (wrong exception types, stale method refs)
- Fix e2e monkey-patching of non-existent service methods
- Add structured logging to create, delete, retry, and rollback flows
- Add jitter to retry backoff to avoid thundering herd
- Add __repr__ to WxOCredentials that fully masks the API key
- Extract hard-coded LLM model string to DEFAULT_WXO_AGENT_LLM constant
- Remove commented-out snapshot update code
- Expand test coverage from 24 to 75 tests covering retry logic,
service methods, client auth, utilities, execution/status helpers,
and artifact roundtrip
* fix(deployment): fix bugs and harden watsonx orchestrate adapter
- Fix update method silently discarding changes instead of sending them
to the WXO API
- Wrap all synchronous SDK calls in asyncio.to_thread to avoid blocking
the event loop
- Add error handling to get_status (was propagating raw exceptions)
- Use DeploymentType enum values instead of raw strings for
SUPPORTED_ADAPTER_DEPLOYMENT_TYPES
- Fix type annotations in list method (list -> set)
- Fix typo in comment (reccomend -> recommend)
- Remove dead code: extract_agent_connection_ids,
require_exclusive_resource, require_non_empty_string,
resolve_health_environment_id, fetch_agent_release_status,
normalize_release_status, resolve_lfx_runner_requirement,
_pin_requirement_name, sync_langflow_tool_connections,
create_langflow_flow_tool, resolve_snapshot_connections, and unused constants
- Make assert_create_resources_available and validate_connection async
- Make create_agent_run and get_agent_run async
- Add tests for list_types, get_status error handling, and update
side-effects
- Update existing tests for async function signatures
* actually remove the dead code from tools.py
* properly await "validate_connection"
* update e2e test file
* add more scenarios to e2e test runner
* refactor(watsonx-orchestrate): improve WxOClient encapsulation and error messages
Encapsulate SDK private method access and endpoint paths inside
WxOClient wrappers so callers never handle raw paths or touch internal
_get/_post methods. Route all private HTTP calls through a single
_base client auto-created in __post_init__, removing the externally
constructed `base` field.
Additionally fix double-period and redundant text in error messages
produced by the ErrorPrefix + handler pattern, and update E2E/unit
assertions to match the sanitised error output.
Changes:
- types.py: bake endpoint paths into wrapper signatures (post_run,
get_run, upload_tool_artifact, get_agents_raw); auto-create _base
via __post_init__; remove base from constructor
- client.py: drop BaseWXOClient import and base= constructor arg
- core/execution.py: pass run_id / query_suffix instead of raw paths
- core/tools.py: pass tool_id instead of raw path
- service.py: fix double-period in ErrorPrefix interpolation; remove
redundant restatements in generic except handlers
- tests: update _with_wxo_wrappers helper and test doubles to use
_base; update FakeBaseClient._get to accept params
- e2e: update rollback scenario detail_contains to match sanitised msg
* skip import and tests of wxo adapter if current env is running python 3.10
* clarify random prefix / retry behavior
* implement comments
* fix: align watsonx adapter with updated deployment schema
- Add `deployment_type` parameter to `get`, `update`, `redeploy`,
`duplicate`, `delete`, `get_status`, and `undeploy_deployment` in
WatsonxOrchestrateDeploymentService to match the updated
BaseDeploymentService ABC
- Update e2e script to use renamed `add_ids` field on
SnapshotDeploymentBindingUpdate
* remove non-interface method undeploy_deployment
* implement listing configs and snapshots
* add update implementation and improve http->deployment error translation and add tests
* improve exception handling
* checkout payload slot work
* custom payload schema for update
* new update implementation
* stop passing client cache to helpers
* add docs for ordereduniquests
* improve import patterns and document future risks for wxo dependencies
* remove global-variable prefixing of flows
* ref: harden typing, DRY helpers, and correctness fixes
- Replace `db: Any` with `db: AsyncSession` across client, config, and
update_helpers modules
- Extract `_ensure_dict` / `ensure_langflow_connections_binding` helpers
to eliminate repeated nested-dict safety logic in tools.py and
update_helpers.py, with documentation explaining why malformed API
payloads are silently corrected rather than rejected
- Use `PayloadSlot.parse()` for update payload validation instead of
standalone helper; remove `parse_provider_update_payload`
- Fix lambda late-binding with default-argument captures in service.py
- Use `zip(strict=True)` in update_helpers for defensive mismatch
detection
- Replace O(n²) duplicate detection with `collections.Counter` in
payloads.py
- Simplify `dedupe_list` to `list(dict.fromkeys(...))`
- Move type-annotation-only imports into TYPE_CHECKING blocks in
types.py and config.py
- Introduce `UPDATE_MAX_RETRIES` as a distinct constant from
`CREATE_MAX_RETRIES`
- Fix "watsonX" → "watsonx" casing in error messages
- Clarify `get_status` docstring and `validate_connection` error message
- Add TODO(deployments-cache) for client cache invalidation on
credential updates
* ref: drop client cache and adopt typed deployment context across the wxo adapter path, add request-context memoization with strict mixed-context guards, and lazily initialize wxo SDK clients. Add safer credential handling by storing only authenticators in WxOCredentials. Also improve provider error-detail extraction/messages and make the direct wxo E2E conflict scenario diagnostics and expectations more resilient.
* use explicit naming for context class; providerId
* fix error handling, retry safety, and exception diagnostics in wxo adapter
- Raise DeploymentError on empty API responses instead of fabricating
fake success in create_agent_run_result and get_agent_run
- Elevate rollback failure logging from WARNING to ERROR for alerting
- Apply retryable filter to retry_rollback to skip 401/403/409/422
- Preserve exception chains (from exc) instead of suppressing (from None)
across service.py, utils.py, execution.py, and update_helpers.py
- Broaden credential resolution catch to handle arbitrary DB exceptions
- Separate status code dispatch from string heuristics in
raise_for_status_and_detail to prevent misclassification
- Add tests for all behavioral changes
* [autofix.ci] apply automated fixes
* fix: harden wxO adapter safety, immutability, and error diagnostics
- Make WxOClient and WxOCredentials frozen dataclasses with eager SDK
client initialization to eliminate thread-safety races from
asyncio.to_thread workers and prevent post-construction mutation
- Move instance_url validation and normalization into type __post_init__
- Raise DeploymentError on missing run_id instead of returning partial
success with execution_id=None
- Preserve exception chains (from exc) instead of suppressing (from None)
across create, update, and delete service methods
- Add warning logs when _ensure_dict replaces non-dict binding values
and when _resolve_lfx_requirement falls back to minimum version
- Initialize derived_spec before try block to prevent potential NameError
- Log all ToolUploadBatchError errors before re-raising the first
- Change SUPPORTED_ADAPTER_DEPLOYMENT_TYPES from mutable set to frozenset
- Add tests for 409/422 error mapping in create, unsupported deployment
type rejection, empty update rejection, zip artifact extraction paths,
validate_connection negative paths, missing run_id, multiple deployment
ID rejection, exception chain preservation, and _ensure_dict warning
* [autofix.ci] apply automated fixes
* fix ruff errors and and todo for status method
* fix mypy errors
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix: 1.9.0 nightly (#12210)
fix 1.9.0 nightly
* fix(mcp): Add schema-driven type conversion (#11796)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* fix(mcp): Add schema-driven type conversion
- Add schema-driven type conversion (str→dict, str→int, etc.)
- normalize and unflatten tool arguments for MCP servers
- Unflatten flattened keys (e.g. params.search) into nested objects
* fix(mcp): handle dict type and array type in JSON schema for MCP tools
- Support "type": ["string", "null"] (JSON Schema array type)
- Normalize required to hashable elements (filter non-string entries)
- Add unit tests for create_input_schema_from_json_schema
* fix(mcp): map generic object type to dict for free-form params
When JSON schema has {"type": "object"} with no properties, treat it as
a free-form dict instead of building a nested Pydantic model. This allows
MCP servers expecting arbitrary key-value params to receive proper dicts,
and enables str→dict conversion via _normalize_arguments_for_mcp.
- Add conditional in parse_type: empty properties → dict, else nested model
- Add test_create_input_schema_generic_object_maps_to_dict
* fix(mcp): exclude None from tool arguments sent to MCP servers
* test_mcp: add more tests for datatypes
* fix(mcp): parse JSON strings for nested model params (e.g. foreman-mcp)
* fix(mcp): refactor _try_convert_value reducing repetition
* fix(mcp): add missing tests for remaining use cases
* fix(mcp): Fix mapping of None if expected is list, dict or str
* Update nightly_build.yml
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: prevent overwriting user-selected global variables in provider c… (#12217)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* fix: prevent overwriting user-selected global variables in provider config
Previously, the apply_provider_variable_config_to_build_config function would
automatically overwrite field values with environment variable keys whenever
an env var was present, even if the user had already selected a different
global variable.
This fix adds a check to only auto-set the environment variable if:
- The field is currently empty, OR
- The field is not already configured to load from the database
This preserves user selections while still providing automatic configuration
for new/empty fields.
Added comprehensive unit tests to verify:
- Auto-setting env vars for empty fields
- Preserving user-selected global variables
- Overwriting hardcoded values (expected behavior)
- Skipping when env var is not set
- Applying component metadata correctly
* [autofix.ci] apply automated fixes
* style: use dictionary comprehension instead of for-loop
Fixed PERF403 Ruff style warning by replacing for-loop with dictionary
comprehension in update_projects_components_with_latest_component_versions
* chore: retrigger CI build
* test: improve test coverage and clarity for provider config
- Renamed test_apply_provider_config_overwrites_hardcoded_value to
test_apply_provider_config_replaces_hardcoded_with_env_var for clarity
- Added test_apply_provider_config_idempotent_when_already_set to document
idempotent behavior when value already matches env var key
- Removed sensitive value from debug log message to prevent potential
exposure of API keys or credentials
These changes improve test coverage by documenting the no-op scenario
and enhance security by avoiding logging of potentially sensitive data.
* chore: retrigger CI build
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: Fixed CodeQL security scan about Incomplete URL substring sanitization on watsonx test suite (#12212)
* fix: Fixed CodeQL security scan about Incomplete URL substring sanitization
* fix coderabbitai comments
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* fix failing action
* docs: add search icon (#12216)
add-back-svg
* Revert "Merge branch 'main' into dev-fix-security-code-scan-watsonx"
This reverts commit 41eb034e11, reversing
changes made to 4e51f4d836.
* Revert "Merge branch 'main' into dev-fix-security-code-scan-watsonx"
This reverts commit 4e51f4d836, reversing
changes made to 530bddd0a4.
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
* fix: remove ibm-watsonx extra from complete installation (#12230)
* docs: lfx readme content (#11870)
* docs-add-lfx-content-to-readme
* github-link-syntax
* allowlist-blocklist
* docs: add a copy to markdown button to docusaurus theme (#12189)
* copy-page-component
* copy-theme-and-add-button
* docs: add versioning (#12218)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* initial-content
* cut-1.8-release-and-include-next-version
* stage-1.8.0-and-next
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
* docs: replace api build automation (#12214)
* remove-workflows-file-and-script
* tag-hidden-endpoints
* update-scripts-and-specs
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Debojit Kaushik <Kaushik.debojit@gmail.com>
* fix: prevent arbitrary file write via path traversal in files endpoint (#12227)
* fix(security): prevent arbitrary file write via path traversal in file uploads
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* docs: Add AI coding agent skills for code review, testing, and refactoring (#12241)
Add AI coding agent skills for code review, testing, and refactoring
* feat: Add Windows Playwright testing to nightly builds (#12221)
* feat: add Windows support for Playwright tests in nightly builds
- Add windows-latest as runner option in typescript_test.yml
- Update Playwright browser installation to be OS-aware (Windows doesn't support --with-deps)
- Add matrix strategy to nightly_build.yml to run tests on both Linux and Windows
- Update Slack notifications to indicate multi-platform testing
- Tests now run in parallel on ubuntu-latest and windows-latest
This enables catching Windows-specific regressions early in the nightly build process.
* fix: add shell: bash to all script steps for Windows compatibility
Windows runners default to PowerShell which doesn't understand bash syntax.
All steps with bash scripts now explicitly specify shell: bash to ensure
cross-platform compatibility.
* feat: make Windows Playwright tests non-blocking initially
Add continue-on-error for Windows tests to allow nightly builds to succeed
even if Windows-specific issues are found. This gives us visibility into
Windows bugs without blocking releases.
Windows tests will still run every night and report failures, but won't
block the build. Once Windows tests are stable, we can remove this flag.
* chore: fix white space
chore: fix white space
* fix: replace matrix strategy with separate jobs for Windows Playwright tests
- GitHub Actions reusable workflows don't support matrix strategy
- Split frontend-tests into frontend-tests-linux and frontend-tests-windows
- Windows tests are non-blocking (continue-on-error: true)
- Updated all job dependencies and Slack notification logic
- Addresses PR review comment about matrix limitation
* chore: trigger workflow validation refresh
---------
Co-authored-by: Adam Aghili <Adam.Aghili@ibm.com>
* fix: Avoid foreign key violation on span table with topological sort (#12232)
* fix: Foreign key violation on span table
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* address review comments
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: use deepcopy to prevent shared reference mutation in component updates (#12252)
* fix: use deepcopy to prevent shared reference mutation in component updates
Copies template data via deepcopy when updating project components to
prevent mutations from leaking between projects through all_types_dict.
Also fixes edge updates to use the copied project data and adds
cloneDeep in the frontend templatesGenerator.
* fix: deepcopy mutable FIELD_FORMAT_ATTRIBUTES and add coverage test
- Add deepcopy on field_dict[attr] assignment (line 192) to prevent
shared references for mutable attributes like input_types, options,
and fileTypes leaking back into all_types_dict
- Add test_update_components_does_not_mutate_field_format_attributes
to verify the fix covers the FIELD_FORMAT_ATTRIBUTES update path
* feat: add support for Langchain 1.0 (#11114)
* feat: upgrade to LangChain 1.0
- langchain ~=1.2.0
- langchain-core ~=1.2.3
- langchain-community ~=0.4.1
Updated all langchain-* integration packages to versions compatible with langchain-core 1.0+.
* feat(lfx): add langchain-classic dependency for legacy agent classes
LangChain 1.0 removed AgentExecutor and related classes to langchain-classic.
This adds the dependency to maintain backward compatibility.
* refactor(lfx): update imports for LangChain 1.0 compatibility
- Move AgentExecutor, agent creators from langchain to langchain_classic
- Move AsyncCallbackHandler from langchain.callbacks to langchain_core.callbacks
- Move Chain, BaseChatMemory from langchain to langchain_classic
- Update LANGCHAIN_IMPORT_STRING for code generation
* fix(lfx): make sqlalchemy import lazy in session_scope
LangChain 1.0 no longer includes sqlalchemy as a transitive dependency.
Move the import inside the function where it's used to avoid import errors
when sqlalchemy is not installed.
* chore: update uv.lock for langchain-classic
* feat: enable nv-ingest optional dependencies for langchain 1.0
- Uncomment nv-ingest-api and nv-ingest-client, update to >=26.1.0
(no longer has openai version conflict)
- Bump datasets from <4.0.0 to <5.0.0 to allow fsspec>=2025.5.1
required by nv-ingest
- Update mlx-vlm TODO comment with accurate blocking reason
* chore: update nv-ingest to 26.1.1
nv-ingest 26.1.1 removes the openai dependency, resolving the
conflict with langchain-openai>=1.0.0 (which requires openai>=1.109.1).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: enable mlx and mlx-vlm dependencies for langchain 1.0
opencv-python 4.13+ now supports numpy>=2, resolving the conflict
with langchain-aws>=1.0.0 (which requires numpy>=2.2 on Python 3.12+).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style: fix import order in callback.py
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: update imports to use langchain_classic for agent modules
* [autofix.ci] apply automated fixes
* fix: remove .item() calls in knowledge_bases.py
* fix(lfx): import BaseMemory from langchain_classic for langchain 1.0
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* refactor: update deprecated langchain imports for langchain 1.0
- langchain.callbacks.base -> langchain_core.callbacks.base
- langchain.tools -> langchain_core.tools
- langchain.schema -> langchain_core.messages/documents
- langchain.chains -> langchain_classic.chains
- langchain.retrievers -> langchain_classic.retrievers
- langchain.memory -> langchain_classic.memory
- langchain.globals -> langchain_core.globals
- langchain.docstore -> langchain_core.documents
- langchain.prompts -> langchain_core.prompts
Also simplified GoogleGenerativeAIEmbeddingsComponent to use native
langchain-google-genai 4.x which now supports output_dimensionality.
* [autofix.ci] apply automated fixes
* fix: add _to_int helper for pandas sum() compatibility across Python versions
* fix: update langfuse>=3.8.0 and fix cuga_agent.py (#11519)
* fix: update langfuse>=3.8.0 and fix cuga_agent.py
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* feat: implement LangFuseTracer for langfuse v3 API compatibility and add unit tests
* fix: upgrade cuga to 0.2.9 for langchain 1.0 compatibility
* fix: improve error handling and return value in get_langchain_callback method
* fix: update package versions for compatibility and improvements
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* fix: update langwatch dependency to version 0.10.0 for compatibility
* fix: update environment variable for Langfuse host to LANGFUSE_BASE_URL
* Update dependency versions in Youtube Analysis project
- Downgraded googleapiclient from 2.188.0 to 2.154.0
- Updated langchain_core from 1.2.7 to 1.2.9
- Updated fastapi from 0.128.1 to 0.128.5
- Downgraded youtube_transcript_api from 1.2.4 to 1.2.3
- Changed langchain_core version from 1.2.7 to 0.3.81
- Cleared input_types in model selection
# Conflicts:
# src/backend/base/langflow/initial_setup/starter_projects/Basic Prompt Chaining.json
# src/backend/base/langflow/initial_setup/starter_projects/Basic Prompting.json
# src/backend/base/langflow/initial_setup/starter_projects/Blog Writer.json
# src/backend/base/langflow/initial_setup/starter_projects/Custom Component Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/Document Q&A.json
# src/backend/base/langflow/initial_setup/starter_projects/Financial Report Parser.json
# src/backend/base/langflow/initial_setup/starter_projects/Hybrid Search RAG.json
# src/backend/base/langflow/initial_setup/starter_projects/Image Sentiment Analysis.json
# src/backend/base/langflow/initial_setup/starter_projects/Instagram Copywriter.json
# src/backend/base/langflow/initial_setup/starter_projects/Invoice Summarizer.json
# src/backend/base/langflow/initial_setup/starter_projects/Knowledge Retrieval.json
# src/backend/base/langflow/initial_setup/starter_projects/Market Research.json
# src/backend/base/langflow/initial_setup/starter_projects/Meeting Summary.json
# src/backend/base/langflow/initial_setup/starter_projects/Memory Chatbot.json
# src/backend/base/langflow/initial_setup/starter_projects/News Aggregator.json
# src/backend/base/langflow/initial_setup/starter_projects/Nvidia Remix.json
# src/backend/base/langflow/initial_setup/starter_projects/Pokédex Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Portfolio Website Code Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/Price Deal Finder.json
# src/backend/base/langflow/initial_setup/starter_projects/Research Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Research Translation Loop.json
# src/backend/base/langflow/initial_setup/starter_projects/SEO Keyword Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/SaaS Pricing.json
# src/backend/base/langflow/initial_setup/starter_projects/Search agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Sequential Tasks Agents.json
# src/backend/base/langflow/initial_setup/starter_projects/Simple Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Social Media Agent.json
# src/backend/base/langflow/initial_setup/starter_projects/Text Sentiment Analysis.json
# src/backend/base/langflow/initial_setup/starter_projects/Travel Planning Agents.json
# src/backend/base/langflow/initial_setup/starter_projects/Twitter Thread Generator.json
# src/backend/base/langflow/initial_setup/starter_projects/Vector Store RAG.json
# src/backend/base/langflow/initial_setup/starter_projects/Youtube Analysis.json
* fix: handle InvalidRequestError during session rollback
* update projects
* ⚡️ Speed up method `LangFuseTracer.end` by 141% in PR #11114 (`feat/langchain-1.0`) (#11682)
Optimize LangFuseTracer.end
The optimized code achieves a **140% speedup** (8.23ms → 3.42ms) through two complementary optimizations:
## 1. Fast-path for Common Primitives in `serialize()`
**What changed:** Added an early-exit check that returns immutable primitives (`str`, `int`, `float`, `bool`) directly when no truncation or special handling is needed:
```python
if max_length is None and max_items is None and not to_str:
if isinstance(obj, (str, int, float, bool)):
return obj
```
**Why it's faster:**
- The profiler shows `_serialize_dispatcher()` consumed **81.5% of runtime** in the original code (40.4ms out of 49.6ms)
- This optimization reduced dispatcher calls from **8,040 to 1,013** (~87% reduction), as primitives now bypass the expensive pattern-matching dispatcher entirely
- The fast-path check itself is extremely cheap: just two quick conditionals and an `isinstance()` check against a tuple of built-in types
**When it helps:** This optimization is particularly effective for workloads with many primitive values in dictionaries and lists—which is exactly what the tracing use case provides (metadata dicts with strings, numbers, booleans).
## 2. Eliminate Redundant Serialization in `LangFuseTracer.end()`
**What changed:** Serialize `inputs`, `outputs`, and `metadata` once each, then reuse the results:
```python
inputs_ser = serialize(inputs)
outputs_ser = serialize(outputs)
metadata_ser = serialize(metadata) if metadata else None
```
**Why it's faster:**
- The original code called `serialize()` **6 times total** (3 for `.update()` + 3 for `.update_trace()`)
- The optimized version calls it **3 times**, then passes the cached results
- Profiler shows the time spent in `serialize()` calls dropped from **72.2ms to 31.2ms** (~57% reduction)
- This is pure elimination of redundant work—the same dictionaries were being serialized twice with identical results
**Impact on workloads:** The `test_end_multiple_iterations_calls_end_each_time` test (500 iterations) demonstrates this matters in hot paths. If `LangFuseTracer.end()` is called frequently during flow execution, avoiding duplicate serialization provides compounding benefits.
## Combined Effect
Both optimizations target the serialization bottleneck from different angles:
- The fast-path reduces the cost of *each* serialize call by ~75% for primitive-heavy data
- The caching reduces the *number* of serialize calls by 50%
Together, they deliver the observed 140% speedup, with the optimization being especially effective for the common case of metadata dictionaries containing mostly primitive types.
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
* refactor: reorder imports and simplify serialization logic for primitives
* [autofix.ci] apply automated fixes
* fix: update google dependency version to 0.4.0 in component_index.json
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix: pin z3-solver<4.15.7 to restore Linux wheels for Docker build
z3-solver 4.15.7 dropped manylinux wheels, causing the Docker build to
fail when trying to compile from source. Temporary pin until codeflash
is removed.
* [autofix.ci] apply automated fixes
* fix: update google dependency version to 0.4.0
* [autofix.ci] apply automated fixes
* fix: update langchain_core version to 1.2.17 in multiple starter project JSON files
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix: update deprecated langchain imports to langchain_classic for 1.0 compatibility
* fix: align langchain-chroma version in optional chroma dependency group
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* feat: fall back to langchain_classic for pre-1.0 imports in user components
Old flows using removed langchain imports (e.g. langchain.memory,
langchain.schema, langchain.chains) now resolve via langchain_classic
at two levels: module-level for entirely removed modules, and
attribute-level for removed attributes in modules that still exist
in langchain 1.0. New langchain 1.0 imports are never affected since
fallbacks only trigger on import failure.
* urllib parse module import bug
* Update component_index.json
* [autofix.ci] apply automated fixes
* chore: rebuild component index
* [autofix.ci] apply automated fixes
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Harold Ship <harold.ship@gmail.com>
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: prevent CI injection via unsanitized GitHub context interpolation (#12224)
Pass github.event.pull_request.head.ref through env: instead of
interpolating it directly into run: shell steps. This prevents bash
from evaluating command substitutions embedded in malicious branch names
before input validation runs.
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
* feat: Refactor and Unify the ModelInput Selector Across Components (#12025)
* fix: Fixes Kubernetes deployment crash on runtime_port parsing (#11968) (#11975)
* feat: add runtime port validation for Kubernetes service discovery
* test: add unit tests for runtime port validation in Settings
* fix: improve runtime port validation to handle exceptions and edge cases
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
* fix(frontend): show delete option for default session when it has messages (#11969)
* feat: add documentation link to Guardrails component (#11978)
* feat: add documentation link to Guardrails component
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* feat: traces v0 (#11689) (#11983)
* feat: traces v0
v0 for traces includes:
- filters: status, token usage range and datatime
- accordian rows per trace
Could add:
- more filter options. Ecamples: session_id, trace_id and latency range
* fix: token range
* feat: create sidebar buttons for logs and trace
add sidebar buttons for logs and trace
remove lods canvas control
* fix: fix duplicate trace ID insertion
hopefully fix duplicate trace ID insertion on windows
* fix: update tests and alembic tables for uts
update tests and alembic tables for uts
* chore: add session_id
* chore: allo grouping by session_id and flow_id
* chore: update race input output
* chore: change run name to flow_name - flow_id
was flow_name - trace_id
now flow_name - flow_id
* facelift
* clean up and add testcases
* clean up and add testcases
* merge Alembic detected multiple heads
* [autofix.ci] apply automated fixes
* improve testcases
* remodel files
* chore: address gabriel simple changes
address gabriel simple changes in traces.py and native.py
* clean up and testcases
* chore: address OTel and PG status comments
https://github.com/langflow-ai/langflow/pull/11689#discussion_r2854630438https://github.com/langflow-ai/langflow/pull/11689#discussion_r2854630446
* chore: OTel span naming convention
model name is now set using name = f"{operation} {model_name}" if model_name else operation
* add traces
* feat: use uv sources for CPU-only PyTorch (#11884)
* feat: use uv sources for CPU-only PyTorch
Configure [tool.uv.sources] with pytorch-cpu index to avoid ~6GB CUDA
dependencies in Docker images. This replaces hardcoded wheel URLs with
a cleaner index-based approach.
- Add pytorch-cpu index with explicit = true
- Add torch/torchvision to [tool.uv.sources]
- Add explicit torch/torchvision deps to trigger source override
- Regenerate lockfile without nvidia/cuda/triton packages
- Add required-environments for multi-platform support
* fix: update regex to only replace name in [project] section
The previous regex matched all lines starting with `name = "..."`,
which incorrectly renamed the UV index `pytorch-cpu` to `langflow-nightly`
during nightly builds. This caused `uv lock` to fail with:
"Package torch references an undeclared index: pytorch-cpu"
The new regex specifically targets the name field within the [project]
section only, avoiding unintended replacements in other sections like
[[tool.uv.index]].
* style: fix ruff quote style
* fix: remove required-environments to fix Python 3.13 macOS x86_64 CI
The required-environments setting was causing hard failures when packages
like torch didn't have wheels for specific platform/Python combinations.
Without this setting, uv resolves optimistically and handles missing wheels
gracefully at runtime instead of failing during resolution.
---------
* LE-270: Hydration and Console Log error (#11628)
* LE-270: add fix hydration issues
* LE-270: fix disable field on max token on language model
---------
* test: add wait for selector in mcp server tests (#11883)
* Add wait for selector in mcp server tests
* [autofix.ci] apply automated fixes
* Add more awit for selectors
* [autofix.ci] apply automated fixes
---------
* fix: reduce visual lag in frontend (#11686)
* Reduce lag in frontend by batching react events and reducing minimval visual build time
* Cleanup
* [autofix.ci] apply automated fixes
* add tests and improve code read
* [autofix.ci] apply automated fixes
* Remove debug log
---------
* feat: lazy load imports for language model component (#11737)
* Lazy load imports for language model component
Ensures that only the necessary dependencies are required.
For example, if OpenAI provider is used, it will now only
import langchain_openai, rather than requiring langchain_anthropic,
langchain_ibm, etc.
* Add backwards-compat functions
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Add exception handling
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* comp index
* docs: azure default temperature (#11829)
* change-azure-openai-default-temperature-to-1.0
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* [autofix.ci] apply automated fixes
---------
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix unit test?
* add no-group dev to docker builds
* [autofix.ci] apply automated fixes
---------
* feat: generate requirements.txt from dependencies (#11810)
* Base script to generate requirements
Dymanically picks dependency for LanguageM Comp.
Requires separate change to remove eager loading.
* Lazy load imports for language model component
Ensures that only the necessary dependencies are required.
For example, if OpenAI provider is used, it will now only
import langchain_openai, rather than requiring langchain_anthropic,
langchain_ibm, etc.
* Add backwards-compat functions
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Add exception handling
* Add CLI command to create reqs
* correctly exclude langchain imports
* Add versions to reqs
* dynamically resolve provider imports for language model comp
* Lazy load imports for reqs, some ruff fixes
* Add dynamic resolves for embedding model comp
* Add install hints
* Add missing provider tests; add warnings in reqs script
* Add a few warnings and fix install hint
* update comments add logging
* Package hints, warnings, comments, tests
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Add alias for watsonx
* Fix anthropic for basic prompt, azure mapping
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* ruff
* [autofix.ci] apply automated fixes
* test formatting
* ruff
* [autofix.ci] apply automated fixes
---------
* fix: add handle to file input to be able to receive text (#11825)
* changed base file and file components to support muitiple files and files from messages
* update component index
* update input file component to clear value and show placeholder
* updated starter projects
* [autofix.ci] apply automated fixes
* updated base file, file and video file to share robust file verification method
* updated component index
* updated templates
* fix whitespaces
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* add file upload test for files fed through the handle
* [autofix.ci] apply automated fixes
* added tests and fixed things pointed out by revies
* update component index
* fixed test
* ruff fixes
* Update component_index.json
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* updated component index
* updated component index
* removed handle from file input
* Added functionality to use multiple files on the File Path, and to allow files on the langflow file system.
* [autofix.ci] apply automated fixes
* fixed lfx test
* build component index
---------
* docs: Add AGENTS.md development guide (#11922)
* add AGENTS.md rule to project
* change to agents-example
* remove agents.md
* add example description
* chore: address cris I1 comment
address cris I1 comment
* chore: address cris I5
address cris I5
* chore: address cris I6
address cris I6
* chore: address cris R7
address cris R7
* fix testcase
* chore: address cris R2
address cris R2
* restructure insight page into sidenav
* added header and total run node
* restructing branch
* chore: address gab otel model changes
address gab otel model changes will need no migration tables
* chore: update alembic migration tables
update alembic migration tables after model changes
* add empty state for gropu sessions
* remove invalid mock
* test: update and add backend tests
update and add backend tests
* chore: address backend code rabbit comments
address backend code rabbit comments
* chore: address code rabbit frontend comments
address code rabbit frontend comments
* chore: test_native_tracer minor fix address c1
test_native_tracer minor fix address c1
* chore: address C2 + C3
address C2 + C3
* chore: address H1-H5
address H1-H5
* test: update test_native_tracer
update test_native_tracer
* fixes
* chore: address M2
address m2
* chore: address M1
address M1
* dry changes, factorization
* chore: fix 422 spam and clean comments
fix 422 spam and clean comments
* chore: address M12
address M12
* chore: address M3
address M3
* chore: address M4
address M4
* chore: address M5
address M5
* chore: clean up for M7, M9, M11
clean up for M7, M9, M11
* chore: address L2,L4,L5,L6 + any test
address L2,L4,L5 and L6 + any test
* chore: alembic + comment clean up
alembic + comment clean up
* chore: remove depricated test_traces file
remove depricated test_traces file. test have all been moved to test_traces_api.py
* fix datetime
* chore: fix test_trace_api ge=0 is allowed now
fix test_trace_api ge=0 is allowed now
* chore: remove unused traces cost flow
remove unused traces cost flow
* fix traces test
* fix traces test
* fix traces test
* fix traces test
* fix traces test
* chore: address gabriels otel coment
address gabriels otel coment latest
---------
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com>
* fix(test): Fix superuser timeout test errors by replacing heavy clien… (#11982)
fix(test): Fix superuser timeout test errors by replacing heavy client fixture (#11972)
* fix super user timeout test error
* fix fixture db test
* remove canary test
* [autofix.ci] apply automated fixes
* flaky test
---------
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* refactor(components): Replace eager import with lazy loading in agentics module (#11974)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix: add ondelete=CASCADE to TraceBase.flow_id to match migration (#12002)
* fix: add ondelete=CASCADE to TraceBase.flow_id to match migration
The migration file creates the trace table's flow_id foreign key with
ondelete="CASCADE", but the model was missing this parameter. This
mismatch caused the migration validator to block startup.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: add defensive migration to ensure trace.flow_id has CASCADE
Adds a migration that ensures the trace.flow_id foreign key has
ondelete=CASCADE. While the original migration already creates it
with CASCADE, this provides a safety net for any databases that may
have gotten into an inconsistent state.
* fix: dynamically find FK constraint name in migration
The original migration did not name the FK constraint, so it gets an
auto-generated name that varies by database. This fix queries the
database to find the actual constraint name before dropping it.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: LE-456 - Update ButtonSendWrapper to handle building state and improve button functionality (#12000)
* fix: Update ButtonSendWrapper to handle building state and improve button functionality
* fix(frontend): rename stop button title to avoid Playwright selector conflict
The "Stop building" title caused getByRole('button', { name: 'Stop' })
to match two elements, breaking Playwright tests in shards 19, 20, 22, 25.
Renamed to "Cancel" to avoid the collision with the no-input stop button.
* Fix: pydantic fail because output is list, instead of a dict (#11987)
pydantic fail because output is list, instead of a dict
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
* refactor: Update guardrails icons (#12016)
* Update guardrails.py
Changing the heuristic threshold icons.
The field was using the default icons. I added icons related to the security theme.
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
* feat: Clean up the modelinput unification
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update test_embedding_model_component.py
* [autofix.ci] apply automated fixes
* Revert to main for other files
* More reversions
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Handle first run more elegantly in astra
* [autofix.ci] apply automated fixes
* Fix knowledge embedding dialog (#12071)
* fix: Handle message inputs when ingesting knowledge
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Update test_ingestion.py
* [autofix.ci] apply automated fixes
* fix: Unify the knowledge creation model selector
* Revert tracing
* Update ingestion.py
* Rebuild comp index
* [autofix.ci] apply automated fixes
* Update test_ingestion.py
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update test_ingestion.py
* Update component_index.json
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* Update comp index
* Update test_astradb_base_component.py
* Update Knowledge Ingestion.json
* [autofix.ci] apply automated fixes
* Fix broken tests
* Cleanup from claude
* [autofix.ci] apply automated fixes
* Fix failing tests
* Update test_unified_models.py
* [autofix.ci] apply automated fixes
* Update Nvidia Remix.json
* Refactor ingest
* Rebuild templates and component index
* Fix test
* [autofix.ci] apply automated fixes
* Update component_index.json
* [autofix.ci] apply automated fixes
* test: add update_build_config visibility tests and PR review fixes (#12114)
- Add update_build_config field-visibility tests to LanguageModelComponent,
ToolCallingAgentComponent, and BatchRunComponent covering Ollama, WatsonX,
OpenAI, and no-model-selected cases
- Remove 16 stale @pytest.mark.skip tests from test_agent_component.py
- Wire up validate_model_selection in agent.py for early input validation
- Document AstraDB intentional use of lower-level update_model_options_in_build_config
- Clarify model_kwargs info text to note provider-specific support
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Update embedding_model.py
* fix: address PR review recommendations for feat-unify-models++ (#12116)
- Fix 9 skipped tests in test_batch_run_component.py by replacing model
list with _MockLLM instances, following the existing pattern used by
test_with_config_failure_handling
- Fix test_agent_component.py: set component.model to a valid list before
calling get_agent_requirements() in the three max_tokens tests, since
validate_model_selection now requires a list-format model
- Replace os.environ direct reads in apply_provider_variable_config_to_build_config
with get_all_variables_for_provider() (DB-first, env fallback), and pass
user_id through from handle_model_input_update
- Add deprecated stubs for update_provider_fields_visibility, _update_watsonx_fields,
and _update_ollama_fields in model_config.py with DeprecationWarning pointing
to handle_model_input_update
- Fix typo: "deault" -> "default" in structured_output.py TODO comment
- Add 4 new KnowledgeIngestionComponent tests: new-format model_selection
metadata path, allow_duplicates=True, missing metadata file error, and
_build_embedding_metadata without API key
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Ruff errors
* Update test_ingestion.py
* Update component index
* Test updates
* Update component_index.json
* Update stable_hash_history.json
* Template updates
* Update batch_run.py
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* Update Youtube Analysis.json
* Fix tests
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Some cleanup and refactoring
* [autofix.ci] apply automated fixes
* Update Nvidia Remix.json
* Update Nvidia Remix.json
* Update unified_models.py
* Coderabbit AI review comments
* Component index update
* [autofix.ci] apply automated fixes
* Template updates
* [autofix.ci] apply automated fixes
* Template update
* [autofix.ci] apply automated fixes
* Review comments addressed
* [autofix.ci] apply automated fixes
* Update component_index.json
* Update stable_hash_history.json
* [autofix.ci] apply automated fixes
* Test updates
* Update test_ingestion.py
* Update test_ingestion.py
* Update test_ingestion.py
* [autofix.ci] apply automated fixes
* More clear tooltip text
* [autofix.ci] apply automated fixes
* Template updates
* Index and templates
* [autofix.ci] apply automated fixes
* Fix lambda build
* Template updates
* Rebuild comp index
* [autofix.ci] apply automated fixes
* Fix templates
* Fix failing test
* Update templates
* Update comp index
* [autofix.ci] apply automated fixes
* API key field in astra db
* Update starter
* Update comp index
* Starter proj update
* Add api key to field order
* Update test_unified_models.py
* Update test_unified_models.py
* [autofix.ci] apply automated fixes
* Update setup.py
* Update setup.py
* Update component_index.json
* [autofix.ci] apply automated fixes
* Return embedding models directly in KB
* [autofix.ci] apply automated fixes
* Update component_index.json
* fix: Refactor the unified models code
* Ruff checks
* Update flow_preparation.py
* [autofix.ci] apply automated fixes
* Update test_language_model_component.py
* fix: prevent overwriting user-selected global variables in provider c… (#12217)
* fix: nightly now properly gets 1.9.0 branch (#12215)
before it was attempting to pull release-notes as letters are alphanumerically after numbers when we sort -V then grab tail
now we only look at branch names that follow the pattern '^release-[0-9]+\.[0-9]+\.[0-9]+$'
* docs: add search icon (#12216)
add-back-svg
* fix: prevent overwriting user-selected global variables in provider config
Previously, the apply_provider_variable_config_to_build_config function would
automatically overwrite field values with environment variable keys whenever
an env var was present, even if the user had already selected a different
global variable.
This fix adds a check to only auto-set the environment variable if:
- The field is currently empty, OR
- The field is not already configured to load from the database
This preserves user selections while still providing automatic configuration
for new/empty fields.
Added comprehensive unit tests to verify:
- Auto-setting env vars for empty fields
- Preserving user-selected global variables
- Overwriting hardcoded values (expected behavior)
- Skipping when env var is not set
- Applying component metadata correctly
* [autofix.ci] apply automated fixes
* style: use dictionary comprehension instead of for-loop
Fixed PERF403 Ruff style warning by replacing for-loop with dictionary
comprehension in update_projects_components_with_latest_component_versions
* chore: retrigger CI build
* test: improve test coverage and clarity for provider config
- Renamed test_apply_provider_config_overwrites_hardcoded_value to
test_apply_provider_config_replaces_hardcoded_with_env_var for clarity
- Added test_apply_provider_config_idempotent_when_already_set to document
idempotent behavior when value already matches env var key
- Removed sensitive value from debug log message to prevent potential
exposure of API keys or credentials
These changes improve test coverage by documenting the no-op scenario
and enhance security by avoiding logging of potentially sensitive data.
* chore: retrigger CI build
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* Update build_config.py
* [autofix.ci] apply automated fixes
* Update build_config.py
* Fix tests
* fix: Dropdown issue with field population
* Update test_unified_models.py
* Clean up key config
* [autofix.ci] apply automated fixes
* fix tests
* Fix tests
* fix: Update tests
* Update tests
* Update test_tool_calling_agent.py
* Update test_unified_models.py
* Update test_tool_calling_agent.py
* Update tests
* Google AI generative embeddings fixes
* [autofix.ci] apply automated fixes
* Merge release branch
* Template update
* Merge release branch
* [autofix.ci] apply automated fixes
* Update openai_constants.py
* Update openai_constants.py
---------
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
Co-authored-by: keval shah <kevalvirat@gmail.com>
Co-authored-by: Antônio Alexandre Borges Lima <104531655+AntonioABLima@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com>
Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
Co-authored-by: Lucas Democh <ldgoularte@gmail.com>
Co-authored-by: Steve Haertel <stevehaertel@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
* fix: Wait for dynamic model fetch in Nvidia (#12229)
* fix: Wait for dynamic model fetch in Nvidia
* [autofix.ci] apply automated fixes
* Create test_nvidia_component.py
* Update test_nvidia_component.py
* [autofix.ci] apply automated fixes
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* Update test_nvidia_component.py
* Update component_index.json
* [autofix.ci] apply automated fixes
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: protect image downloads by flow ownership (#12234)
* fix: protect image downloads by flow ownership
* test: add clarifying comments for image access review
---------
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
* feat: Add Windows Playwright test fixes to RC (#12265)
* feat: Add Windows Playwright tests to nightly builds
- Add windows-latest to typescript_test.yml runner options
- Add shell: bash to all script steps for cross-platform compatibility
- Split Playwright installation into OS-aware steps (Linux uses --with-deps, Windows/macOS/self-hosted don't)
- Fix artifact naming with OS prefix to prevent conflicts: blob-report-${{ runner.os }}-${{ matrix.shardIndex }}
- Split frontend-tests into separate Linux and Windows jobs in nightly_build.yml
- Add ref parameter to all test jobs to checkout code from release branch
- Add resolve-release-branch to needs dependencies
- Update Slack notifications to handle both Linux and Windows test results
- Windows tests are non-blocking (not checked in release-nightly-build condition)
- Update .secrets.baseline with new line number (263 -> 347) for LANGFLOW_ENG_SLACK_WEBHOOK_URL
Fixes LE-566
* fix: Use contains() for self-hosted runner detection
- Replace exact string equality (==, !=) with contains() for substring matching
- Fixes issue when inputs.runs-on is array format: '["self-hosted", "linux", "ARM64", ...]'
- Ensures self-hosted Linux runners correctly skip --with-deps flag
Addresses CodeRabbit feedback on PR #12264
* fix: Sanitize folder names for CodeQL (#12263)
* fix(deps): pin tar-fs to >=2.1.4 to fix symlink following vulnerability (#12078)
Adds override for tar-fs in package.json to ensure versions prior to
2.1.4 are never resolved. Addresses CVE in tar-fs <2.1.4 (PVR0686558)
where symlink validation bypass was possible with a crafted tarball.
* fix: Rebuild the embedding model in the nv template (#12275)
* fix: support ZIP file upload for flows and projects endpoints (#12253)
* feat: support ZIP file upload for flows and projects endpoints
Add ZIP upload support to both /flows/upload/ and /projects/upload/
endpoints, enabling round-trip download-then-upload workflows. Extract
shared ZIP parsing logic into a dedicated utility with zip bomb
protections (entry count and file size limits). Fix batch flow name
deduplication to avoid infinite loops and DB collisions. Add tests for
ZIP upload, empty ZIP rejection, and download-upload round-trip.
* fix: add type annotation to satisfy mypy union narrowing
* fix: address PR review for ZIP upload (#12253)
- Add BadZipFile handling in extract_flows_from_zip for defense-in-depth
- Wrap blocking Z…
* add OpenDsStar 1.0.20 dependency and update uv.lock
Add OpenDsStar==1.0.20 to both backend and lfx pyproject.toml.
Regenerate uv.lock to include OpenDsStar and its transitive deps.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* fix ruff lint errors in codeagents, file ingestion tests, and inspect script
* fix lfx tests: mock subprocess.Popen instead of subprocess.run
The FileDescriptionGenerator uses subprocess.Popen, not subprocess.run.
Tests were mocking the wrong function, causing real subprocess execution
that failed with ModuleNotFoundError for OpenDsStar in isolated lfx env.
* [autofix.ci] apply automated fixes
* fix lfx test mock data to match expected subprocess output format
Subprocess returns {"results": [...], "failed": [], "total": N},
not a bare list. Tests now use _wrap_results() helper.
* bump OpenDsStar to 1.0.21 to fix litellm dependency conflict
* fix tools metadata merge to preserve user-edited descriptions, remove orphan submodule
- Preserve user-edited description and display_description in tools
metadata merge (fixes edit-tools Playwright test)
- Remove orphaned OpenDsStar-fork submodule entry that caused CI
post-cleanup warnings
* smart tools description merge: update from code unless user customized
Compare old description to display_description — if they match, the user
never edited it and we update from code (picking up new Output.info
descriptions). If they differ, the user customized it and we preserve
their edit.
* [autofix.ci] apply automated fixes
* address CodeRabbit review: fix 4 major and 9 minor issues
Major fixes:
- Chroma: only suggest persist_directory when not in server mode
- FileContentRetriever: new input overwrites stale persisted data
- FileContentRetriever: preserve file_path on returned Message
- FileDescriptionGenerator: drain pipes in non-select fallback to prevent deadlock
Minor fixes:
- inspect_chroma_db: align docstring with actual 2000-char truncation
- test_file_content_retriever: fix empty/misplaced test functions
- test_file_description_generator: align duplicate paths test with actual behavior
- format.ts: fix NODE prefix regex pattern
- open_ds_star_agent: use get_running_loop(), remove duplicate cast import
- codeact_agent_smolagents: document timeout behavior, simplify import fallback
* bump OpenDsStar to 1.0.22
* Write File: make agent-friendly with tool_mode inputs, format control, and flexible input types
- Add tool_mode to input field (DataFrameInput) so agents can pass content directly
- Add file_format tool_mode input so agents can choose output format (csv, json, etc.)
- Handle raw pandas DataFrames from code agents (auto-wrap to Langflow DataFrame)
- Handle string inputs from chat agents (JSON → DataFrame, plain text → Message)
- Improve descriptions for agent discoverability
- Bump OpenDsStar to 1.0.24 (async tool invocation fix)
- Add 9 new tests covering all new features
* Write File: hide format dropdowns in tool mode, improve descriptions
- Hide local_format/aws_format/gdrive_format when tool_mode is active
- Agent uses unified file_format input instead
- Shorten field descriptions to pass ruff line length checks
* fix: add file_format field to News Aggregator starter project
The SaveToFile component now has a file_format input — update the
starter project template so field_order matches the component definition.
Update secrets baseline hash for shifted line.
* [autofix.ci] apply automated fixes
* rename Structured Data Agent starter project to Structured Data Analysis Agent
* fix: regenerate starter projects to include file_format field in SaveToFile node
The SaveToFileComponent added a new file_format input but the starter
project JSONs were not regenerated, causing field_order mismatches in
both Structured Data Analysis Agent and News Aggregator starter projects.
* feat: add html as a supported file format in SaveToFileComponent
Add html to LOCAL_MESSAGE_FORMAT_CHOICES, GDRIVE_FORMAT_CHOICES, and
the _save_message handler (reuses the txt write path). AWS already
supported html. Update the file_format tool-mode description to
include html.
* test: add test for saving Message as html format
* fix: regenerate starter projects after adding html format to SaveToFileComponent
* [autofix.ci] apply automated fixes
* fix(tests): skip assistant panel e2e test when OPENAI_API_KEY is missing
Without a configured model provider the assistant panel renders a
"no models" state instead of the input textarea and model selector,
causing the test to fail in CI environments without API keys.
* fix: add file_format field to SaveToFile in starter projects
The autofix bot incorrectly removed the file_format entry. Re-add it
to both field_order and template in News Aggregator and Structured
Data Analysis Agent starter projects. Update secrets baseline.
* fix: rebuild component index to include file_format in SaveToFile
The CI autofix kept removing file_format from starter projects because
the prebuilt component_index.json lacked the field. The update script
loads components from this index in production mode, so it regenerated
starter projects without file_format. Rebuild the index so it includes
the file_format input, which makes the autofix a no-op.
* fix: hide format dropdowns in SaveToFile when used as tool
The storage_location handler in update_build_config was showing format
dropdowns regardless of tool mode. Now it checks tools_metadata to
detect tool mode and keeps format dropdowns hidden, since the agent
uses the unified file_format string input instead.
* fix: remove debug script and replace stderr debug logging with proper logger
Remove scripts/inspect_chroma_db.py (debugging script). Replace all
_dbg() stderr writes in FileDescriptionGeneratorComponent with
standard logger calls at appropriate levels (debug/warning/info).
* fix: avoid double agent execution in OpenDsStarAgentRunnable.astream
astream was calling ainvoke after fully consuming the stream, which
re-executed the entire graph. Use the last streamed chunk as the final
result instead.
* fix: address PR review feedback from ogabrielluiz
- Mirror positional-to-kwargs mapping in async tool path (component_tool.py)
- Add final_answer guard to timeout suppression (codeact_agent_smolagents.py)
- Extract _build_graph_input() helper to deduplicate state dict x4 (open_ds_star_agent.py)
- Split ImportError from other exceptions in set_main_event_loop (open_ds_star_agent.py)
- Add sys.platform != 'win32' guard for select() on pipes (file_description_generator.py)
- Move ChromaError import before try block (chroma.py)
- Add logger.warning to __deepcopy__ fallback paths (component.py)
- Change accumulate_usage from additive to max-based merge (token_usage.py)
- Add async positional-arg test and update token usage tests
* increase opendsstar to 1.0.25 to fix pydantic dependency conflict
* vector stores: remove tool_mode=True for as_dataframe output
* increase OpenDsStar to 1.0.26
---------
Co-authored-by: Gal-Bloch <gal.bloch@ibm.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
Co-authored-by: Jordan Frazier <122494242+jordanrfrazier@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Hamza Rashid <74062092+HzaRashid@users.noreply.github.com>
Co-authored-by: Viktor Avelino <64113566+viktoravelino@users.noreply.github.com>
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
Co-authored-by: Antônio Alexandre Borges Lima <104531655+AntonioABLima@users.noreply.github.com>
Co-authored-by: DeyLak <DeyLak@users.noreply.github.com>
Co-authored-by: Adam Aghili <Adam.Aghili@ibm.com>
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: Adam-Aghili <149833988+Adam-Aghili@users.noreply.github.com>
Co-authored-by: Florian Schüller <schuellerf@users.noreply.github.com>
Co-authored-by: Steve Haertel <stevehaertel@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
Co-authored-by: Steve Haertel <shaertel@ca.ibm.com>
Co-authored-by: Mateusz Szewczyk <139469471+MateuszOssGit@users.noreply.github.com>
Co-authored-by: Debojit Kaushik <Kaushik.debojit@gmail.com>
Co-authored-by: Janardan Singh Kavia <janardankavia@ibm.com>
Co-authored-by: Janardan S Kavia <janardanskavia@Janardans-MacBook-Pro.local>
Co-authored-by: vjgit96 <vijay.katuri@ibm.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@logspace.ai>
Co-authored-by: Harold Ship <harold.ship@gmail.com>
Co-authored-by: codeflash-ai[bot] <148906541+codeflash-ai[bot]@users.noreply.github.com>
Co-authored-by: keval shah <kevalvirat@gmail.com>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@Olayinkas-MacBook-Pro.local>
Co-authored-by: Olayinka Adelakun <olayinkaadelakun@mac.war.can.ibm.com>
Co-authored-by: Ram Gopal Srikar Katakam <44802869+RamGopalSrikar@users.noreply.github.com>
Co-authored-by: olayinkaadelakun <olayinka.adelakun@ibm.com>
Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: Himavarsha <40851462+HimavarshaVS@users.noreply.github.com>
Co-authored-by: Lucas Democh <ldgoularte@gmail.com>
Co-authored-by: Anderson Filho <115162146+andifilhohub@users.noreply.github.com>
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: himavarshagoutham <himavarshajan17@gmail.com>
Co-authored-by: Viktor Avelino <viktor.avelino@gmail.com>
Co-authored-by: Hamza Rashid <hzarashid@gmail.com>
Co-authored-by: Rodrigo Nader <rodrigonader@MacBook-Pro-de-Rodrigo.local>
Co-authored-by: ogabrielluiz <gabriel@langflow.org>
Co-authored-by: Tarcio <rodriguestarcio.adv@gmail.com>
Co-authored-by: Carlos Coelho <80289056+carlosrcoelho@users.noreply.github.com>
Co-authored-by: Alfio Gliozzo <54385608+gliozzo@users.noreply.github.com>
Co-authored-by: Alfio Gliozzo <gliozzo@us.ibm.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: manav2000 <manav2000@users.noreply.github.com>
Co-authored-by: Deon Sanchez <69873175+deon-sanchez@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: DAVID BOAZ <DAVIDBO@il.ibm.com>
Co-authored-by: Arek Mateusiak <severfire@users.noreply.github.com>
Co-authored-by: DevByteAI <abud6673@gmail.com>
Co-authored-by: Alice Reis <alicemarianareis@gmail.com>
Co-authored-by: Alex Kuligowski <alekuligowski@gmail.com>
Co-authored-by: a-effort <35465683+a-effort@users.noreply.github.com>
* fix: skip missing server file when delete_after_processing=True to avoid race condition (fixes#8887)
* [autofix.ci] apply automated fixes
* test(lfx): cover file_path branch for missing-file ValueError
Switch the missing-file ValueError test to set self.component.file_path
(via a Data object) instead of self.component.path, so the file_path
decision branch in load_files_base is exercised when
delete_server_file_after_processing=False and silent_errors=False.
* fix: cache parsed result so concurrent outputs don't drop server-file data
Previously the race-condition fix in load_files_base() returned an empty
list when a concurrent output call had already deleted the server file.
That avoided the spurious ValueError but silently dropped data — a File
component with multiple connected outputs could "succeed" while a
downstream node received an empty Data wrapper.
Cache the parsed Data list on the component instance after the first
successful processing, and reuse it when a subsequent call discovers the
file is gone. Each connected output now sees the same parsed content
instead of empty data, while still preserving the deletion semantics on
the source file.
Updates the regression test to assert the cached result is returned
(equal to the first call's output) rather than locking in the empty-list
behavior.
* fix: serialize load_files_base with per-instance lock and key cache by paths
The previous race-condition fix had two remaining gaps flagged in review:
1. It was not concurrency-safe. Two threads could both pass
_validate_and_resolve_paths before either deleted the source file;
the loser would then drop to empty data inside _filter_and_mark_files
when the file vanished mid-flight.
2. The post-hoc recovery ("if validation returned no files and any cache
exists, return it") was too broad — a legitimate empty input on a
later call could pick up a stale cached result for a different
server-file state.
Address both:
- Acquire a per-instance threading.Lock around the entire
validate/process/delete cycle so concurrent output methods on the
same component are serialized. The first caller does the work; the
others observe the cached parsed result.
- Cache the parsed Data list keyed by (local paths, server file paths,
markdown flag) so different inputs and different processing variants
don't share entries. Empty results are not cached, so a legitimate
empty input cannot return stale data.
- Set an explicit _validate_skipped_due_to_delete_race flag inside
_validate_and_resolve_paths so recovery only activates when validation
actually skipped a missing delete-after-processing file. Recovery
reuses any cached result for the same source paths (across markdown
variants) — preserving content beats dropping it when the file is
gone.
Adds a concurrency test that drives two threads through load_files_base
simultaneously and asserts both observe the same parsed data, plus a
test that an empty input does not return stale cached data.
* fix: eagerly init load_files_base lock in __init__ to close first-entry race
Lazy lock creation inside load_files_base() (hasattr-then-assign) is itself
racy: on the very first concurrent entry, two threads can each find the
attribute missing and create their own threading.Lock, bypassing
serialization for the exact race this PR is fixing.
Move the lock creation into __init__ so every instance has a single Lock
before any output method can run. Removes the lazy-init branch from
load_files_base().
---------
Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix(security): prevent arbitrary file read via public flow build endpoint (GHSA-rcjh-r59h-gq37)
The unauthenticated `POST /api/v1/build_public_tmp/{flow_id}/flow`
endpoint accepted a `files` array whose paths were resolved by
`LocalStorageService` and read off disk before being attached to the
LLM prompt. A path like `/etc/hosts` was split into `flow_id="/etc"`
and `file_name="hosts"`. `data_dir / "/etc"` collapsed to `/etc`
because joining an absolute segment resets the prefix, and the
containment check was anchored at that attacker-controlled folder
so it passed. The file contents were then echoed back through the
chat reply.
Fix layers:
1. Boundary validation in `build_public_tmp`: each `files` entry
must match `{source_flow_id}/{basename}` with no traversal,
backslash, or null bytes. Foreign flow_ids are rejected with
HTTP 400.
2. `LocalStorageService._validated_path` now also rejects
`flow_id` values containing path separators, traversal, or
null bytes, and anchors the containment check at the data
directory rather than the per-flow folder.
Adds regression tests for both layers covering `/etc/hosts`,
traversal, foreign-namespace and null-byte payloads, plus an
acceptance test that legitimate `{flow_id}/{filename}` references
still work for AUTO_LOGIN playground uploads.
* add identifier validation
---------
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
* fix: pin postgres image to bookworm in docker_example to prevent collation mismatch
The postgres:16 tag silently moved its base from Debian Bookworm
(glibc 2.36) to Trixie (glibc 2.41), causing a recurring collation
version mismatch warning on existing langflow-postgres volumes.
Pin to postgres:16-bookworm in both docker-compose files and update
the README so existing data volumes keep matching the OS locale data.
Refs: https://github.com/langflow-ai/langflow/issues/9608
* docs: pin postgres image to bookworm in current docs compose snippets
Mirror the docker_example pin in the four current docs that publish
copyable Compose snippets pairing postgres:16 with a persistent
langflow-postgres volume. Prevents the same Bookworm-to-Trixie
collation version mismatch warning when users follow docs instead
of docker_example.
Versioned historical docs are left as-is.
Refs: https://github.com/langflow-ai/langflow/issues/9608
* fix: switch postgres pin from bookworm to trixie for OS consistency
Aligns the pinned postgres base with the langflow runtime image, which
moved to Debian Trixie in #12990. Keeping postgres on bookworm would
have diverged the stack and locked the database to an aging glibc that
will receive fewer security backports as bookworm ages into oldstable.
The pin itself still solves the original bug — postgres:16 cannot
silently roll its OS underneath an existing volume.
Document the one-time REFRESH COLLATION VERSION step for users
upgrading from a bookworm-initialized volume in docker_example/README.md.
Refs: https://github.com/langflow-ai/langflow/issues/9608
* chore(starter-projects): new Vector Store RAG template
Update the RAG starter flow to align with the new knowledge base
paradigm: removes the inline knowledge base creation sub-flow and
points users to the dedicated Knowledge Ingestion UI for loading
content into their vector store.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(starter-projects): tweak Vector Store RAG README description
Drop the outdated reference to two sub-flows in the in-template
README and refresh the saved metadata.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: autofix of starter project json
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
- Add pull: true to all docker build steps in nightly workflow
- Forces Docker to pull latest python:3.12-slim-trixie instead of using cached layers
- Ensures Debian Trixie base images are used instead of cached Bookworm layers
- Resolves issue where Docker layer caching prevented CVE fixes from taking effect
Frontend was forcing event_delivery=polling in the build query, overriding
LANGFLOW_EVENT_DELIVERY=streaming on the server. Token-by-token streaming
in the Playground never worked because the frontend always took the polling
code path (customPollBuildEvents) instead of SSE (performStreamingRequest).
Aligns three default fallbacks with the server doc default (STREAMING):
- buildUtils.buildFlowVertices: query-param default
- utilityStore.eventDelivery: initial store value (race window before /config)
- use-get-config: fallback when /config omits event_delivery
buildFlowVerticesWithFallback already retries with POLLING on
STREAMING_NOT_SUPPORTED / ENDPOINT_NOT_AVAILABLE, so streaming-first is safe.
Fixes#12291
* fix(frontend): paginator empty state shows "0 items" instead of "1-0 of 0 items"
When totalRowsCount is 0 the paginator rendered "1-0 of 0 items" and
"of 0 pages", with the next button enabled because pageIndex (1) was
not equal to maxIndex (0). Hide the range when empty, floor maxIndex
to 1, and disable both navigation buttons when there is nowhere to go.
* Update legacy paginator tests to assert new empty-state copy
fix(mcp): make run_tool wait_for timeout configurable, raise floor to 180s
The hardcoded `timeout=30.0` on `asyncio.wait_for(session.call_tool(...))`
in both `MCPSseClient.run_tool` and `MCPStreamableHttpClient.run_tool` was
truncating in-flight tool calls whose servers needed longer than 30s to
respond — most visibly large MCP composers whose tool responses include
sizeable catalogs.
Failure mode:
LLM → AgentExecutor → run_tool (1 call from the LLM)
├── attempt 1: get_session (probe ..._1 fails → swap ..._2)
│ → call_tool → 30s wait_for fires, TimeoutError raised
│ (server is still working — response ignored on arrival)
└── attempt 2: get_session (probe ..._2 fails → swap ..._3)
→ call_tool → 30s wait_for fires again
→ util.py dedup ("Repeated TimeoutError, not retrying")
→ ValueError("Maximum retries exceeded ...")
← AgentExecutor surfaces error → flow fails
The retry loop's TimeoutError dedup correctly stops looping once it sees
the same error twice, but every call against a slow server hit the same
ceiling and never had a chance to succeed. The session swaps between
attempts also produced excess HTTP churn against the server.
Both `run_tool` methods now resolve the timeout from
`get_settings_service().settings.mcp_server_timeout` (already used for
connection setup at lines 1617 / 1887, env: `LANGFLOW_MCP_SERVER_TIMEOUT`)
with a `max(setting, 180.0)` floor so default deployments don't end up
*shorter* than the previous hardcoded 30s — the Pydantic default for
`mcp_server_timeout` is 20.
Touched:
src/lfx/src/lfx/base/mcp/util.py
- L1677, L1689 (stdio run_tool)
- L1960, L1972 (streamable HTTP run_tool)
Unchanged on purpose:
- L1217, L1381 session-creation `timeout=30.0` (separate concern).
- L1060 `_validate_session_connectivity` 3.0s probe (separate
concern; addressed in follow-up if probe thrash persists).
Tuning:
- LANGFLOW_MCP_SERVER_TIMEOUT now governs both connection setup and
tool-call wait_for. The 180s floor clamps low values; raise the env
var above 180 to take effect.
Repro / verification:
- Trace run that previously failed at 67.08s with "Maximum retries
exceeded with repeated TimeoutError errors" against
`mcp-guardium_get_service_info` (47KB response payload) now
completes on attempt 1.
Co-authored-by: MANSURA HABIBA <MANSURAH@ie.ibm.com>
* perf: Enhance Gunicorn preload functionality for Langflow
- Introduced a new preload module to optimize memory usage by running fork-safe initialization in the Gunicorn master process.
- Updated the lifespan management in `main.py` to check if the master has preloaded resources, allowing workers to inherit state and skip redundant setup.
- Adjusted the server loading process to accommodate the new preload logic, ensuring efficient resource management across worker processes.
* feat(cache): Implement teardown method for RedisCache to prevent socket leaks
- Added a `teardown` method to the `RedisCache` class to close the Redis connection, addressing potential socket leaks during process forking.
- Introduced unit tests to verify the functionality of the `teardown` method, ensuring it handles client closure and errors gracefully.
- Tests cover scenarios including normal closure, error handling during closure, and teardown with URL-based connections.
* feat(cache): Implement teardown method in RedisCache to prevent socket leaks
- Added a `teardown` method to the `RedisCache` class, ensuring proper closure of the Redis client connection before forking to avoid socket leaks.
- Created comprehensive unit tests to validate the functionality of the `teardown` method, covering various scenarios including normal operation and error handling.
- Updated existing tests to reflect the new teardown functionality and ensure RedisCache is recognized as an instance of `ExternalAsyncBaseCacheService`.
* refactor(preload): Simplify DB engine disposal logic in master preload
- Removed exception handling around the DB engine disposal to streamline the process, ensuring that the engine is disposed of without unnecessary error logging. This change enhances code clarity and maintains the intended functionality of resource management during the preload phase.
* refactor(preload): Streamline cache service teardown logic
- Simplified the cache service socket closure process in the master preload function to prevent sharing across forks. This change enhances code clarity by removing unnecessary exception handling while maintaining the intended functionality of resource management during the preload phase.
* feat(preload): Introduce per-step completion flags for improved state management
- Added completion flags in the preload state to track the status of various initialization steps, including profile picture copying, starter project creation, agentic global variable initialization, MCP server configuration, and flow loading.
- Updated the lifespan management in `main.py` to utilize these flags, allowing the system to skip redundant setup tasks if they have already been completed during the preload phase.
- This enhancement improves resource management and ensures that the application behaves correctly in a multi-worker environment.
* refactor(lifespan): Replace temp dir management with get_owned_temp_dirs function
- Updated the lifespan management in `main.py` to utilize the new `get_owned_temp_dirs` function, which encapsulates the logic for determining temp directory ownership based on the process type (master or worker).
- Removed the `is_master` check from the lifespan function, simplifying the code and enhancing clarity regarding temp directory cleanup responsibilities.
- Added the `get_owned_temp_dirs` function in `preload.py` to centralize temp directory ownership logic, ensuring that workers do not attempt to clean up directories owned by the master process.
* refactor(lifespan): Enhance initialization flow with conditional gates
- Introduced conditional gates in the `get_lifespan` function to manage the initialization of profile pictures, super users, bundles, component types, and starter projects based on their completion status.
- Improved logging to provide clearer insights into which steps are being skipped or executed, enhancing the overall clarity of the initialization process.
- Updated the preload logic in `preload.py` to ensure that agentic global variables and MCP server configuration are only initialized when necessary, maintaining efficient resource management in a multi-worker environment.
* fix: resolve double-call of initialize_auto_login_default_superuser and add preload tests
- Fix double-call issue: setup_superuser() now handles AUTO_LOGIN completely with file lock
- Add comprehensive unit tests for preload.py covering failure-fallback contract
- Simplify code by doing superuser initialization in initialize_services() (called early in both preload and worker startup)
- File lock protects multi-worker race conditions when preload is disabled
- Tests verify critical step failures propagate, best-effort steps continue on failure
Made-with: Cursor
* fix(preload): resolve missing import and agentic initialization conflicts
Fixed critical bugs introduced in c0e81a5401 that caused preload failures:
1. Missing import: Added DEFAULT_SUPERUSER_PASSWORD to module-level imports
- Was only imported inside AUTO_LOGIN block but used when AUTO_LOGIN=false
- Caused NameError that crashed preload with "session scope error"
2. Removed agentic variable initialization from setup_superuser()
- Prevents double-initialization conflict with preload's dedicated step
- initialize_agentic_global_variables() in preload handles all users
3. Made teardown_superuser() more robust
- Now skips deletion instead of raising errors on FK constraints
- Prevents startup failures when default superuser has associated flows
Resolves: "An error occurred during the session scope" preload error
Resolves: Ghost thread warnings from incomplete initialization
Made-with: Cursor
* refactor(preload): Enhance state management and initialization logic
- Updated the `_PreloadState` class to include `bundles_loaded` and `types_cached` flags for better tracking of initialization steps.
- Modified the `get_lifespan` function to utilize the new state flags, improving the conditional logic for loading bundles and caching component types.
- Implemented a `reset` method in `_PreloadState` to ensure consistent state restoration after preload failures, enhancing reliability in multi-worker environments.
- Simplified the teardown process in `ExternalAsyncBaseCacheService` by making `teardown` an abstract method, allowing direct calls without fallback checks.
This refactor improves clarity and efficiency in the preload and lifespan management processes.
* refactor(lifespan): Improve component types caching and starter project creation logic
- Enhanced the `get_lifespan` function to utilize a local handle for component types when the cache is inherited, ensuring consistency in multi-worker environments.
- Added logging for scenarios where the component types cache is empty, allowing for cache rebuilding instead of skipping starter project creation.
- Streamlined the starter project creation process with improved error handling and logging, ensuring clarity on failures and skipped operations.
This refactor enhances the reliability and clarity of the initialization process in the application.
* fix(tests): Update teardown_superuser test to preserve default superuser if never logged
- Renamed the test function to clarify its purpose.
- Modified the test logic to ensure that the default superuser is not removed during teardown if it has never been logged in, preventing foreign key errors.
- Added assertions to verify that the superuser remains intact and its properties are correctly set after teardown.
This change enhances the reliability of the superuser management in the test suite.
* feat: Implement orphaned MCP server config migration and add unit tests
- Introduced `migrate_orphaned_mcp_servers_config` function to recover MCP server config files that become orphaned after a database reset while preserving the config directory.
- Added logic to handle scenarios with multiple orphaned files, ensuring safe migration and logging for manual recovery.
- Created comprehensive unit tests to validate the migration functionality, covering cases for single orphan recovery, multiple orphans, and scenarios with no orphans.
- Enhanced the setup of the superuser to include the migration process, improving the robustness of user configuration management.
This update enhances the application's ability to recover user configurations in containerized environments.
* fix(teardown): Improve superuser removal logic and error handling
- Updated the `teardown_superuser` function to remove the default superuser when AUTO_LOGIN is disabled and the user has never logged in, enhancing user management.
- Improved error handling to raise a `RuntimeError` if the removal fails, ensuring that issues are logged and not silently ignored.
- Adjusted the `DatabaseService` teardown process to reflect the updated superuser removal logic and ensure proper resource disposal.
This change enhances the reliability of the superuser management during application shutdown.
* fix(setup_superuser): Enhance error handling for AUTO_LOGIN timeout scenario
- Updated the `setup_superuser` function to provide clearer logging and error messages when the AUTO_LOGIN lock times out and no default superuser exists.
- Introduced a check to verify the existence of the default superuser in the database, raising a `RuntimeError` with a descriptive message if it is not found.
- Added unit tests to validate the behavior of the setup process under timeout conditions, ensuring that the application fails gracefully when required superuser credentials are missing.
This change improves the robustness of the superuser initialization process and enhances error visibility during startup.
* fix(preload): Update error handling to log exceptions with traceback for best-effort steps
- Modified the error handling in the `_run_master_preload` function to log exceptions with traceback instead of warnings for best-effort steps, ensuring better visibility into failures.
- Updated unit tests to reflect the change in logging behavior, verifying that exceptions are correctly logged during preload operations.
This change enhances the clarity of error reporting during the preload process, aiding in debugging and monitoring.
* fix(preload): Ensure cleanup of temporary directories during reset
- Updated the `reset` method in the `_PreloadState` class to call `cleanup()` on each `TemporaryDirectory` before clearing the `temp_dirs` list, preventing on-disk directory leaks during failed preloads.
- Added a unit test to verify that `cleanup()` is called for all temporary directories during the reset process, ensuring proper resource management.
This change enhances the reliability of the preload process by ensuring that temporary resources are properly cleaned up.
* refactor(preload): Introduce PreloadStep enumeration and streamline step completion logic
- Added a `PreloadStep` enumeration to define ordered preload phases, enhancing clarity and maintainability of the preload process.
- Replaced direct state attribute checks with the `is_step_complete` function to improve readability and encapsulate step completion logic.
- Updated the `_run_master_preload` function to utilize `mark_step_complete` for recording successful completion of preload steps, enforcing prerequisite ordering.
- Enhanced unit tests to validate the new step completion logic and ensure proper functionality of the preload process.
This refactor improves the structure and reliability of the preload mechanism, making it easier to manage and understand the initialization flow.
* refactor(superuser): Remove redundant checks for superuser credentials in setup functions
- Eliminated unnecessary validation for `DEFAULT_SUPERUSER` and `DEFAULT_SUPERUSER_PASSWORD` in both `initialize_auto_login_default_superuser` and `setup_superuser` functions, simplifying the initialization logic.
- This change streamlines the superuser setup process, assuming that the necessary credentials are managed externally, thus enhancing code clarity and maintainability.
* refactor(preload): Streamline error handling and step completion in preload process
- Introduced a `_best_effort` function to encapsulate error handling for preload steps, improving code readability and maintainability.
- Updated the `_run_master_preload` function to utilize the new error handling mechanism for various steps, ensuring consistent logging and state management.
- Enhanced the documentation in the preload module to clarify the handling of fork-unsafe resources and the overall preload process.
This refactor enhances the robustness and clarity of the preload mechanism, making it easier to manage and understand the initialization flow.
* refactor(preload): Enhance error handling and resource cleanup in preload process
- Wrapped post-initialization work in a try/finally block to ensure the DB engine and cache service are always disposed of, preventing resource leaks in forked workers.
- Updated logging to provide clearer visibility into the preload steps, including error handling for best-effort operations.
- Improved the structure of the `_run_master_preload` function by consolidating related operations and ensuring consistent cleanup.
This refactor enhances the robustness and clarity of the preload mechanism, ensuring safe resource management during initialization.
* [autofix.ci] apply automated fixes
* test(setup_superuser): remove default superuser before lock-timeout no-superuser case
The `initialized_services` fixture starts with `AUTO_LOGIN=false`, which
runs `setup_superuser` through the credentials-fallback path and creates
the default superuser. The "raises_when_no_superuser" test then mocked
the lock to time out, but the existence check found that pre-created
user and returned `AUTO_LOGIN_LOCK_TIMEOUT_SUPERUSER_PRESENT` instead of
raising `RuntimeError`. Delete the default superuser before mocking the
lock so the no-superuser branch is actually exercised.
---------
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
* fix: URLComponent ignores proxy in async mode (#12285)
The URLComponent fails to connect for users behind corporate proxies
because its asynchronous mode does not recognize standard system proxy
environment variables. The component defaults to use_async=True, which
initializes the RecursiveUrlLoader; the underlying async loader does
not natively respect system proxy environment variables, so the
component attempts a direct connection and fails in restricted network
environments.
Detect standard proxy environment variables (http_proxy, HTTP_PROXY,
https_proxy, HTTPS_PROXY) in URLComponent._create_loader. If a proxy
is detected and use_async is enabled, override use_async to False so
the loader uses its synchronous implementation, which natively
respects system proxies. Empty and whitespace-only values are
correctly evaluated as no-proxy and do not trigger the fallback.
Closes#10297
* fix(URLComponent): broaden proxy detection and harden tests
Address review findings on the proxy fix:
- Add ALL_PROXY and all_proxy to the detected env vars. ALL_PROXY is
commonly set in corporate and container environments (curl, git,
many Unix tools honour it), and is sometimes the only proxy var
configured.
- Replace the unused proxy_url string with a single boolean any() over
the env var keys, since only the presence of a proxy is consulted.
- Reorganize the proxy tests under TestURLComponentProxyHandling with
a shared default-attributes helper, parametrize over all six env var
spellings, and add coverage for multiple-simultaneous-proxies and
the use_async=False path (which should not log a warning).
* [autofix.ci] apply automated fixes
* fix: remove no_proxy="*" macOS startup hack so corporate proxies work
Two startup paths set `os.environ["no_proxy"] = "*"` on macOS, which
disables proxy use for every HTTP client in the process and every child
gunicorn worker (httpx, requests, urllib3 — and therefore the OpenAI,
Anthropic, Groq, etc. SDKs that wrap them). For users behind corporate
proxies on macOS this made every external LLM call unroutable, even with
HTTPS_PROXY properly set.
The override traces to commit history with no concrete justification —
the only artifact is a Stack Overflow link about a uWSGI segfault, but
Langflow uses gunicorn, not uWSGI. Bench-verified that gunicorn boots
cleanly and serves requests on macOS without it (1 worker via
LangflowApplication, OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES retained,
HTTPS_PROXY survives intact in parent and child).
Verification:
- gunicorn worker spawns and serves /health (200 OK), exits cleanly
- httpx, urllib, requests all resolve HTTPS_PROXY after the macOS init
(previously: all three returned empty proxy maps because of no_proxy=*)
- OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES still set — the actual
fork-safety fix is preserved
Closes the macOS half of #10297. Companion fix for the async URLComponent
half is in #12285 / branch pr-12285-rebased.
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Orjson update
---------
Co-authored-by: Diogo Veiga <diogo.veiga@tecnico.ulisboa.pt>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix: update Docker base images to latest Python 3.12 to resolve CVEs
- Update all Dockerfiles from python:3.12.13-slim-trixie to python:3.12-slim-trixie
- This ensures automatic security patches for Debian base image CVEs
- Affects nightly builds (base, main, main-all) and release builds (backend, ep)
- Using unpinned patch version (3.12 vs 3.12.13) follows security best practices
* fix: update builder stage to Debian Trixie for consistency
- Update all Dockerfiles from bookworm-slim to trixie-slim in builder stage
- Ensures consistency between builder and runtime Debian versions
- Eliminates CVEs in both build and runtime environments
- Uses ghcr.io/astral-sh/uv:python3.12-trixie-slim for all builders
