mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-25 00:31:49 +08:00
* langflow-webhook-auth-enable
* add-not-contains-filter-operator
* does-not-contains-operator
* less-redundant-explanation
* docs: add jq and path selection to data operations (#10083)
add-jq-and-path-to-data-operations
* smart transform historical names
* change back to smart transform
* jq expression capitalization/package name
* small edit for clarity of not contains operator
* read/write file component name changes
* docs: add smart router component (#10097)
* init
* add-to-release-notes
* remove-dynamic-output-as-parameter
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: screenshot audit (#10166)
* remove-unused
* agent-examples
* main-ui-screenshots
* components-screenshots
* combine-web-search-components
* simple-agent-flow-in-playground
* round-screenshots
* my-projects
* combine-data-components
* docs: component paths updates for lfx (#10130)
* contributing-bundles-path
* api-monitor-example
* concepts-components-page
* contribute-components-path
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: auto-add projects as MCP servers (#10096)
* add-mcp-auto-auth-as-default-behavior
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
* docs: amazon bedrock converse (#10289)
* use-bedrock-converse
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* Update docs/docs/Components/bundles-amazon.mdx
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs 1.7 release: add mock data component (#10288)
* add-component-and-release-note
* Apply suggestion from @aimurphy
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: update custom component docs (#10323)
* add-partial
* update-lfx-component-paths
* move-partial
* completed-quickstart
* clean up intro
* try-docker-with-custom-mount
* up-to-typed-annotations
* typed-annotations
* dynamic-fields
* end-of-file
* bundles-naming
* chore: update component index
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* docs: add cometapi back for 1.7 release (#10445)
* add-comet-bundle-back-for-1.7
* add-comet-to-release-notes
* docs: add back docling remote vlm for release 1.7 (#10489)
* add-back-docling-vlm-content
* add-release-note
* docs: ALTK component (#10511)
* broken-anchor
* sidebar-and-page
* add-release-note
* add-context-on-output
* docs: SSRF warning (#10573)
* add-ssrf-protection-env-var
* api-request-component
* Update docs/docs/Components/components-data.mdx
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* move-note-to-table
* release-note
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: dynamic create data component (#10517)
* add-dynamic-create-data-component-and-release-note
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* clarify-message-types
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: cuga component bundle (#10589)
* initlal-content
* cuga-specific-component-connections
* cleanup
* use-the-same-name
* add-lite-mode-remove-api-flag-and-mode
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* public-or-private-internet
* agent-doesnt-check-urls
* peer-review
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: remove docling vlm component from 1.7 release branch (#10630)
remove-vlm-component
* docs: rename component categories and make all components single pages (#10648)
* docs: OpenAPI spec version upgraded from 1.6.5 to 1.6.8 (#10627)
Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com>
Co-authored-by: Mendon Kissling <59585235+mendonk@users.noreply.github.com>
* up to models and agents
* sidebars
* fix-broken-links
* chore: Fix indentation on bundles-docling.mdx (#10640)
* sidebars
* redo-intros
* link-to-models
* data-components
* files-components-no-kb
* io-components
* helper-utility-components
* llm-ops-components
* logic-components
* processing-pages
* sidebars
* combine-legacy-components-into-one-page
* update-links
* remove-overview-pages-and-redirect
* make-mcp-tools-page
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* no-cap
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com>
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: combined web search component (#10664)
* combine-pages
* remove-rss-and-news-search-and-update-links
* remove-vlm-link
* leave-old-release-note-but-remove-link
* docs: add altk reflection component (#10660)
* add-new-component
* differentiate-components
* docs: mcp streamable http client (#10621)
* release note
* mcp-client-changes
* update-astra-example
* icons-and-copy
* order-of-names
* docs: add cuga decomposition strategy as advanced parameter (#10672)
* update-component-link
* init
* add-decomp-as-advanced-param
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* update-component-index
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: datastax bundles page (#10686)
* init
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: llm router changed to llm selector (#10663)
* update-component-name
* previous-name-and-release-note
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* [autofix.ci] apply automated fixes (attempt 3/3)
* docs: log alembic to stdout (#10711)
* docs-alembic-log-env-var
* cleanup
* remove-legacy-component-link
* docs: configure s3 for file storage backend (#10678)
* configure-file-storage-s3
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* clarify-s3-credentials
* add-storage-tags-and-cleanup-creds-seciton
* role-link-name
* fix-parse-error
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: allow rest tweaks to mcp tools component (#10833)
* typo
* tweak-mcp-tools-component
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* add-release-note
* docs: use mustache templates in prompts (#11262)
* mustache-templating
* syntax
* release-note
* peer-review
* docs: smart transform supports Message type (#11306)
* component-supports-message-type
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* peer-review
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: modular dependency imports for langflow-base (#11250)
* modular-base-dependencies
* syntax-and-clarification
* release-note
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* clarify-base-and-langflow
* component-index
* delete-component-index
* [autofix.ci] apply automated fixes
* set-agentic-experience
* potential-breaking-changes
* not-audio-package
* cleanup-and-syntax
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* docs: symmetric and asymmetric JWT (#11159)
* initial-content
* cleanup
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* docs-peer-review
* [autofix.ci] apply automated fixes
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* docs: add markdown output to url component (#11336)
* add-markdown-output-format
* raw-content
* Apply suggestions from code review
* docs: Add global variable support for MCP server headers (#11397)
* add-global-var-in-mcp-headers
* revert-curl-syntax-change
* remove-duplicate-tab
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* remove-code-block
* add-release-note
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* Update docs/docs/Develop/install-custom-dependencies.mdx
* Update docs/docs/Develop/jwt-authentication.mdx
* docs: global model provider feature (#11231)
* initial-changes-to-model-providers
* add-icon-for-model-partial
* syntax
* adding-custom-language-model
* release-note
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* peer-review
* use-anthropic-model-with-agent
* [autofix.ci] apply automated fixes
* design-changes
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* mustache-limitations
* release-note-for-jwt
* docs: playground refactor and screenshots (#11639)
* screenshots
* new-playground-and-icon
* release-note
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: component inspection panel (#11675)
* docs-component-inspection-panel
* cleanup
* docs: add tool shortlisting and remove web_apps from CUGA component (#11669)
docs-add-shortlist-tools-and-remove-webapps-parameters
* fix-details-tab-error
* docs: workflow API draft build (#11323)
* delete-unused-yaml-file
* initial-content
* add-python-and-ts-to-example-requests
* separate-pages
* test-spec-presentation
* hide-async-and-make-workflows-plural
* fix-broken-link
* add-changes-to-async
* use-workflow-spec-from-sdk-build
* make-setup-partial
* add-fetch-script-for-openapi-spec
* update-workflows-spec
* remove-stream-for-now
* remove-reconnect-to-stream
* consolidate-pages
* remove-force-boolean
* [autofix.ci] apply automated fixes
* docs: add guardrails component (#11674)
* docs-add-guardrails-component
* cleanup
* example-and-heuristic-check
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* [autofix.ci] apply automated fixes
* add-note-about-llm
* add-release-note
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* docs: pass env var to run command and endpoint as header (#11447)
* pass-env-var-to-lfx
* add-env-var-passing-to-run-endpoint
* add-python-and-js-commands
* docs: responses api token usage tracking (#11564)
* initlal-content
* add-release-note
* changes-for-accessing-advanced-parameters
* [autofix.ci] apply automated fixes
* [autofix.ci] apply automated fixes (attempt 2/3)
* small-playground-changes
* [autofix.ci] apply automated fixes
* Revert "docs: OpenAPI spec content updated without version change (#11787)"
This reverts commit a0d5618ac9.
* [autofix.ci] apply automated fixes
* docs: add LiteLLM proxy bundle (#11867)
* docs-add-litellm-proxy-component
* Update docs/docs/Components/bundles-lite-llm.mdx
* docs: 1.8 changes from QA (#11998)
* remove-rightside-playground
* tutorials
* image-size-update
* component-release-notes
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: pass API keys to args and not env (#11997)
* remove-rightside-playground
* tutorials
* image-size-update
* docs-troubleshoot-mcp-proxy-header-keys
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: knowledge bases (#11924)
* docs-add-back-kb-content
* update-with-release-candidate-branch
* fix-linking-error
* remove-advanced-flag
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* add-release-note
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: traces v0 (#12014)
* env-var-release-note-and-sidebars
* traces-and-database
* traces-ui-and-api-retrieval
* cleanup
* space
* move-section
* move-what-traces-capture-section
* docs: remove kb ingestion and rename kb retrieval (#12065)
remove-knowledge-ingestion-and-rename-knowledge-retrieval
* docs: add link to secret key rotation script (#12072)
* add-link-to-secret-key-rotation
* Apply suggestions from code review
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
---------
Co-authored-by: April I. Murphy <36110273+aimurphy@users.noreply.github.com>
* docs: openlayer follow-on (#12073)
* add-openlayer-to-sidebars-and-release-notes
* Update docs/docs/Support/release-notes.mdx
---------
Co-authored-by: April M <36110273+aimurphy@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
131 lines
6.4 KiB
Plaintext
131 lines
6.4 KiB
Plaintext
---
|
|
title: Manage vector data
|
|
slug: /knowledge
|
|
---
|
|
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
import Icon from "@site/src/components/icon";
|
|
import PartialGlobalModelProviders from '@site/docs/_partial-global-model-providers.mdx';
|
|
|
|
Vector data is critical to AI applications.
|
|
Langflow provides several components to help you store and retrieve vector data in your flows, including embedding models, vector stores, and knowledge bases.
|
|
|
|
## Embedding models
|
|
|
|
Embedding model components generate text embeddings using a specified Large Language Model (LLM).
|
|
|
|
There are two common use cases for these components:
|
|
|
|
* **Store vectors**: Generate embeddings for content written to a vector database.
|
|
* **Search vectors**: Generate an embedding from a query to run a similarity search.
|
|
|
|
In both cases the embedding model component is attached to a vector store component.
|
|
For more information, examples, and available options, see [Embedding model components](/components-embedding-models).
|
|
|
|
Alternatively, you can use [knowledge bases](#knowledge-bases), which include built-in support for several embedding models.
|
|
|
|
## Vector stores
|
|
|
|
Vector store components read and write to vector databases.
|
|
Typically, these components connect to remote databases, but some vector store components support local databases.
|
|
|
|
import PartialVectorRagBlurb from '@site/docs/_partial-vector-rag-blurb.mdx';
|
|
|
|
<PartialVectorRagBlurb />
|
|
|
|
<details>
|
|
<summary>Example: Vector search flow</summary>
|
|
|
|
import PartialVectorRagFlow from '@site/docs/_partial-vector-rag-flow.mdx';
|
|
|
|
<PartialVectorRagFlow />
|
|
|
|
</details>
|
|
|
|
## Knowledge bases
|
|
|
|
import PartialKbSummary from '@site/docs/_partial-kb-summary.mdx';
|
|
|
|
<PartialKbSummary />
|
|
|
|
### Knowledge base storage locations
|
|
|
|
Each knowledge base is a [ChromaDB](https://docs.trychroma.com/docs/overview/introduction) vector database.
|
|
Each database is stored in a separate directory that contains the following:
|
|
|
|
- **Vector embeddings**: Embeddings are stored using the Chroma vector database.
|
|
- **Metadata files**: Configuration and embedding model information.
|
|
- **Source data**: The original data used to create the knowledge base.
|
|
|
|
Knowledge bases are stored local to your Langflow instance.
|
|
The default storage location depends on your operating system and installation method:
|
|
|
|
- **Langflow Desktop**:
|
|
- **macOS**: `/Users/<username>/.langflow/knowledge_bases`
|
|
- **Windows**: `C:\Users\<name>\AppData\Roaming\com.LangflowDesktop\knowledge_bases`
|
|
- **Langflow OSS**:
|
|
- **macOS/Windows/Linux/WSL with `uv pip install`**: `<path_to_venv>/lib/python3.12/site-packages/langflow/knowledge_bases` (Python version can vary. Knowledge bases aren't shared between virtual environments.)
|
|
- **macOS/Windows/Linux/WSL with `git clone`**: `<path_to_clone>/src/backend/base/langflow/knowledge_bases`
|
|
|
|
If you set the `LANGFLOW_CONFIG_DIR` environment variable, the `knowledge_bases` subdirectory is created relative to that path.
|
|
|
|
To change the default `knowledge_bases` directory path, set the `LANGFLOW_KNOWLEDGE_BASES_DIR` environment variable:
|
|
|
|
```bash
|
|
export LANGFLOW_KNOWLEDGE_BASES_DIR="/path/to/parent/directory"
|
|
```
|
|
|
|
### Create a knowledge base
|
|
|
|
In this example, you'll create a knowledge base of chunked customer orders.
|
|
To follow along with this example, download [`customer-orders.csv`](/files/customer_orders.csv) to your local machine, or adapt the steps for your own structured data.
|
|
|
|
1. On the [**Projects** page](/concepts-flows#projects) page, click <Icon name="Library" aria-hidden="true"/>**Knowledge** below the list of projects to view and manage your knowledge bases.
|
|
|
|
2. To create a new knowledge base, click <Icon name="Plus" aria-hidden="true"/>**Add Knowledge**.
|
|
3. In the **Create Knowledge Base** pane, enter a name for your knowledge base, and select an embedding model.
|
|
<PartialGlobalModelProviders />
|
|
4. To configure sources for your knowledge base, click **Configure Sources**.
|
|
Optionally, to create an empty knowledge base, click **Create**.
|
|
5. In the **Configure Sources** pane, configure the sources for your knowledge base's data, and also how the embedded data will be chunked for vector search retrieval.
|
|
For this example, click <Icon name="Upload" aria-hidden="true"/>**Add Sources**, and then select the downloaded [`customer-orders.csv`](/files/customer_orders.csv) file from your local machine.
|
|
The default settings for **Chunk Size**, **Chunk Overlap**, and **Separator** are fine.
|
|
To continue, click **Next Step**.
|
|
6. The **Review & Build** pane allows you to preview your first chunk before you commit to spending tokens to embedall of the data into the knowledge base.
|
|
If the chunk isn't what you want to embed, click **Back** to configure your chunking strategy.
|
|
To embed this data, click **Create**.
|
|
7. Your data is embedded as a **Knowledge**.
|
|
When it is available to use, the **Status** changes to **Ready**.
|
|
|
|
To use the new knowledge base in a flow, see [Use the Knowledge Base component in a flow](/knowledge-base).
|
|
|
|
### Manage knowledge bases
|
|
|
|
On the [**Projects** page](/concepts-flows#projects) page, click <Icon name="Library" aria-hidden="true"/>**Knowledge** below the list of projects to view and manage your knowledge bases.
|
|
|
|
For each knowledge base, you can see the following information:
|
|
|
|
* Name
|
|
* Embedding model
|
|
* Size on disk
|
|
* Number of words, characters, and chunks
|
|
* The average length and size of chunks
|
|
* The knowledge base's status
|
|
|
|
Chunking behavior is determined by the embedding model, and the embedding model is set when you create the knowledge base.
|
|
If you need to change the embedding model, you must delete and recreate the knowledge base.
|
|
|
|
To update a knowledge base with , click <Icon name="EllipsisVertical" aria-hidden="true"/> **More**, and then select <Icon name="RefreshCW" aria-hidden="true"/> **Update Knowledge Base**.
|
|
|
|
To view a knowledge base's chunks, click <Icon name="EllipsisVertical" aria-hidden="true"/> **More**, and then select <Icon name="Layers" aria-hidden="true"/> **View Chunks**.
|
|
|
|
To delete a knowledge base, click <Icon name="EllipsisVertical" aria-hidden="true"/> **More**, and then click <Icon name="Trash2" aria-hidden="true"/> **Delete**.
|
|
If any flows use the deleted knowledge base, you must update them to use a different knowledge base.
|
|
|
|
For more information on using knowledge bases in a flow, see the [**Knowledge Base** component](/knowledge-base) documentation.
|
|
|
|
## See also
|
|
|
|
* [Use Langflow agents](/agents)
|
|
* [Language model components](/components-models) |