Debojit Kaushik 5d947b11c7 feat: Add Memory Base APIs models migrations service endpoints and tests (#12417)
* Add Memory Base API: models, migrations, service, endpoints, and tests

Introduces Memory Base (MB) — a per-flow knowledge base that auto-captures
conversation history and ingests it into a Chroma vector store on configurable
thresholds.

Backend changes:
- MemoryBase + MemoryBaseSession DB models with full CRUD
- Three Alembic migrations: base tables, merge head, phase-2 fields
  (embedding_model, preprocessing, preproc_model, preproc_instructions)
- MemoryBaseService: create/list/get/update/delete, session tracking,
  pending-message cursor logic, mismatch detection, regenerate
- ingest_memory_task: async Chroma ingestion with cursor advance on success
- REST API (/api/v1/memories): CRUD, flush, sessions, mismatch, regenerate
- Flow output hook: on_flow_output() triggers auto-capture after each run
- Plumbing in build.py, endpoints.py, workflow.py to call on_flow_output
- deps.py: expose get_memory_base_service()
- kb_helpers.py: FS/metadata helpers used by MB service

Authored-By: Debojit Kaushik <kaushik.debojit@gmail.com>

Add dedupe_key idempotency enforcement for MB ingestion jobs

Centralizes idempotency into JobService.create_job() with a null-safe check,
removing the redundant pre-flight logic from MemoryBaseService.

Key changes:
- services/jobs/exceptions.py: new DuplicateJobError(RuntimeError) —
  raised when a QUEUED/IN_PROGRESS/COMPLETED job with the same dedupe_key
  exists; FAILED/CANCELLED are retryable and are excluded
- services/jobs/__init__.py: exports DuplicateJobError
- services/jobs/service.py: null-guarded dedup query inside create_job()
  within the same session_scope as the insert (minimizes TOCTOU window)
- services/database/models/jobs/model.py: dedupe_key field -> index=True
- alembic/versions/36aa87831162: adds dedupe_key column + ix_job_dedupe_key
  index to job table with checkfirst guards
- services/memory_base/service.py: updated key format to
  "ingestion:{mb_id}:{session_id}:{first_msg_id}" for namespace isolation;
  removed _has_non_retryable_job_for_dedupe_key and _has_active_job methods
  and all call sites; DuplicateJobError catch in _maybe_trigger() for
  silent skip on auto-capture; split regenerate() catch clauses
- api/v1/memories.py: explicit DuplicateJobError catch before RuntimeError
  in flush_memory_base() for semantic clarity (both return 409)

Co-Authored-By: Debojit Kaushik <kaushik.debojit@gmail.com>

Checkpointing working version of MBs. TODO: User separation, Get messages endpoint, MB resumption midway through a chat for a session, tests.

Added messages endpoint for Memory Bases. Added pagination to sessions endpoint. Modifed messages model to include ingestion related attributes.

Added unit tests, fixed linting issues and formatting issues.

Aligned Workflows API, /run endpoint, playground to all work with Memory Bases. Created DB models asociated with tracking memory base state with sessions and jobs. Added tests, created unit tests for service and task files related to MemoryBases.

Improved concurrency of jobs, added (memory_base_id, session_id) locking to serialize jobs in case the job creation cadence moves ahead of ingestion jobs. Improved concurrency handling and moved the pending check to be live inside the ingestion job rather than a snapshot before triggering the job.

Consolidated all migrations related to memory_bases into one idempotent version and serialized all migrations form release along with memory_bases for cleanliness and maintainability.

Introduced advisory locking to address multi worker environment, added unique constraint to MB creation, added sanitization check to KB pathnames to avoid illegal directory creation.

Added partial write rollback for ChromaDB, aligned same session is  used for each job to avoid dangling advisory locks.

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
2026-04-27 11:30:18 -07:00
2026-04-23 17:49:53 -07:00
2026-04-23 17:49:53 -07:00
2025-03-20 00:05:55 +00:00
2026-04-23 17:49:53 -07:00
2024-06-04 09:26:13 -03:00
2026-04-13 23:23:37 +00:00

Langflow logo

Release Notes PyPI - License PyPI - Downloads Twitter YouTube Channel Discord Server Ask DeepWiki

Langflow is a powerful platform for building and deploying AI-powered agents and workflows. It provides developers with both a visual authoring experience and built-in API and MCP servers that turn every workflow into a tool that can be integrated into applications built on any framework or stack. Langflow comes with batteries included and supports all major LLMs, vector databases and a growing library of AI tools.

Highlight features

  • Visual builder interface to quickly get started and iterate.
  • Source code access lets you customize any component using Python.
  • Interactive playground to immediately test and refine your flows with step-by-step control.
  • Multi-agent orchestration with conversation management and retrieval.
  • Deploy as an API or export as JSON for Python apps.
  • Deploy as an MCP server and turn your flows into tools for MCP clients.
  • Observability with LangSmith, LangFuse and other integrations.
  • Enterprise-ready security and scalability.

🖥️ Langflow Desktop

Langflow Desktop is the easiest way to get started with Langflow. All dependencies are included, so you don't need to manage Python environments or install packages manually. Available for Windows and macOS.

📥 Download Langflow Desktop

Quickstart

Requires Python 3.103.13 and uv (recommended package manager).

Install

From a fresh directory, run:

uv pip install langflow -U

The latest Langflow package is installed. For more information, see Install and run the Langflow OSS Python package.

Run

To start Langflow, run:

uv run langflow run

Langflow starts at http://127.0.0.1:7860.

That's it! You're ready to build with Langflow! 🎉

📦 Other install options

Run from source

If you've cloned this repository and want to contribute, run this command from the repository root:

make run_cli

For more information, see DEVELOPMENT.md.

Docker

Start a Langflow container with default settings:

docker run -p 7860:7860 langflowai/langflow:latest

Langflow is available at http://localhost:7860/. For configuration options, see the Docker deployment guide.

🛡️ Security

For security information, see our Security Policy.

🚀 Deployment

Langflow is completely open source and you can deploy it to all major deployment clouds. To learn how to deploy Langflow, see our Langflow deployment guides.

Stay up-to-date

Star Langflow on GitHub to be instantly notified of new releases.

Star Langflow

👋 Contribute

We welcome contributions from developers of all levels. If you'd like to contribute, please check our contributing guidelines and help make Langflow more accessible.


Star History Chart

❤️ Contributors

langflow contributors

Description
Langflow is a powerful tool for building and deploying AI-powered agents and workflows.
Readme MIT 2.3 GiB
Languages
Python 64.5%
TypeScript 23.4%
JavaScript 11.4%
CSS 0.3%
Makefile 0.2%
Other 0.1%