diff --git a/docs/docs/Develop/concepts-file-management.mdx b/docs/docs/Develop/concepts-file-management.mdx index b5c27e980b..a0c8df572f 100644 --- a/docs/docs/Develop/concepts-file-management.mdx +++ b/docs/docs/Develop/concepts-file-management.mdx @@ -7,8 +7,7 @@ import Icon from "@site/src/components/icon"; Each Langflow server has a file management system where you can store files that you want to use in your flows. -Files uploaded to Langflow file management are stored locally in your [Langflow configuration directory](/memory), and they are available to all of your flows. -Local storage is set by `LANGFLOW_STORAGE_TYPE`, which has only one allowed value (`local`). +Files uploaded to Langflow file management are stored in Langflow's [storage backend (local or AWS S3)](/concepts-file-management#configure-file-storage), and they are available to all of your flows. Uploading files to Langflow file management keeps your files in a central location, and allows you to reuse files across flows without repeated manual uploads. @@ -126,6 +125,81 @@ For more specialized image processing, browse [**Bundles**](/components-bundle-components). +## Configure file storage + +Langflow supports two storage backends for file management: + +* **Local storage**: Langflow's default storage backend. Files are stored locally in your [Langflow configuration directory](/memory). Set `LANGFLOW_STORAGE_TYPE=local` or leave it unset to use local storage. + +* **S3 storage**: Files are stored in an AWS S3 bucket. +Langflow uses the [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) library to interact with S3. + +To use S3 as your file storage backend, add the following configuration to your `.env` file: + +```text +# S3 Storage Configuration +LANGFLOW_STORAGE_TYPE=s3 +LANGFLOW_OBJECT_STORAGE_BUCKET_NAME=S3_BUCKET_NAME +LANGFLOW_OBJECT_STORAGE_PREFIX=S3_BUCKET_DIRECTORY + +# AWS Credentials (required for S3) +AWS_ACCESS_KEY_ID=S3_ACCESS_KEY +AWS_SECRET_ACCESS_KEY=S3_ACCESS_SECRET_KEY +AWS_DEFAULT_REGION=S3_REGION +``` + +Replace the following placeholders with the actual values for your S3 instance: + +* `S3_BUCKET_NAME`: The name of your S3 bucket. +* `S3_BUCKET_DIRECTORY`: An optional folder path within the bucket where files are stored, such as `s3://S3_BUCKET_NAME/S3_BUCKET_DIRECTORY`. +* `S3_ACCESS_KEY`: Your AWS Access Key ID. +* `S3_ACCESS_SECRET_KEY`: Your AWS Secret Access Key. +* `S3_REGION`: The AWS region where your bucket is located, such as `us-east-2`. + +Your AWS credentials must have the necessary permissions to perform the required S3 operations for your use case, such as reading, writing, and deleting files in S3. +This example policy allows basic CRUD operations on S3 objects. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "LangflowS3StorageAccess", + "Effect": "Allow", + "Action": [ + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject", + "s3:ListBucket", + "s3:PutObjectTagging", + ], + "Resource": [ + "arn:aws:s3:::S3_BUCKET_NAME", + "arn:aws:s3:::S3_BUCKET_NAME/S3_BUCKET_DIRECTORY/*" + ] + } + ] +} +``` + +Replace the following placeholders with the actual values for your IAM policy and S3 instance: + +* `S3_BUCKET_NAME`: The name of your S3 bucket. +* `S3_BUCKET_DIRECTORY`: An optional folder path within the bucket where files are stored, such as `s3://S3_BUCKET_NAME/S3_BUCKET_DIRECTORY`. + +For more information, see the [AWS documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html). + +## File storage environment variables {#file-storage-environment-variables} + +The following environment variables configure file storage backends for Langflow's file management system: + +| Variable | Format | Default | Description | +|----------|--------|---------|-------------| +| `LANGFLOW_STORAGE_TYPE` | String | `local` | Set the file storage backend. Supported values: `local` (files stored in the Langflow configuration directory) or `s3` (files stored in AWS S3). For S3 storage, you must also configure AWS credentials and bucket settings. | +| `LANGFLOW_OBJECT_STORAGE_BUCKET_NAME` | String | Not set | The name of the S3 bucket to use for file storage. Required when `LANGFLOW_STORAGE_TYPE=s3`. | +| `LANGFLOW_OBJECT_STORAGE_PREFIX` | String | Not set | Optional prefix/folder path within the S3 bucket where files will be stored. If not set, files are stored at the bucket root. | +| `LANGFLOW_OBJECT_STORAGE_TAGS` | JSON object | Not set | Optional S3 object tags applied to stored files when `LANGFLOW_STORAGE_TYPE=s3`. Ignored for local storage. Provided as a JSON map of string keys to string values, such as `{"env": "prod", "owner": "data-team"}`. | + ## See also * [Components reference](/concepts-components) \ No newline at end of file diff --git a/docs/docs/Develop/environment-variables.mdx b/docs/docs/Develop/environment-variables.mdx index eba40187c8..f996c6c15a 100644 --- a/docs/docs/Develop/environment-variables.mdx +++ b/docs/docs/Develop/environment-variables.mdx @@ -462,7 +462,9 @@ For more information about deploying Langflow servers, see [Langflow deployment ### Storage -See [Memory management options](/memory) and [Manage files](/concepts-file-management). +For file storage environment variables, see [File storage environment variables](/concepts-file-management#file-storage-environment-variables). + +For database environment variables, including PostgreSQL configuration, see [Memory management options](/memory#configure-external-memory). ### Telemetry diff --git a/docs/docs/Develop/memory.mdx b/docs/docs/Develop/memory.mdx index 963165d73d..e3c19f8e48 100644 --- a/docs/docs/Develop/memory.mdx +++ b/docs/docs/Develop/memory.mdx @@ -100,6 +100,8 @@ To fine-tune your database connection pool and timeout settings, you can set the Don't use the deprecated environment variables `LANGFLOW_DB_POOL_SIZE` or `LANGFLOW_DB_MAX_OVERFLOW`. Instead, use `pool_size` and `max_overflow` in `LANGFLOW_DB_CONNECTION_SETTINGS`. +* `LANGFLOW_MIGRATION_LOCK_NAMESPACE`: Optional namespace for PostgreSQL advisory locks used during database migrations. This is useful when running multiple Langflow instances that share the same PostgreSQL database. Each instance should use a unique namespace to avoid conflicts. If not set, Langflow uses a default namespace. This setting only applies when using PostgreSQL as your database backend. + ## Configure cache memory The default Langflow caching behavior is an asynchronous, in-memory cache: