Files
langflow/docs/sidebars.js
Gabriel Luiz Freitas Almeida e2d17a6d84 feat(logging): production-grade structured logs for Grafana/Loki (#13164)
* feat(logging): production-grade structured logs for Grafana/Loki

Make langflow and lfx log output viable for ingestion by Grafana/Loki and
other observability tools when run in JSON mode (LANGFLOW_LOG_ENV=container).

Core changes in src/lfx/src/lfx/log/logger.py:
- Preserve exceptions in JSON output via structlog.processors.ExceptionRenderer
  with ExceptionDictTransformer. Tracebacks now emit as a structured exception
  array (exc_type, exc_value, frames) instead of being dropped.
- show_locals defaults to OFF; opt-in via LANGFLOW_LOG_TRACE_LOCALS=true so
  frame locals can't leak API keys, env, or request bodies in shipped logs.
- Add service metadata (service / version / environment) from
  LANGFLOW_SERVICE_NAME / LANGFLOW_VERSION / LANGFLOW_ENVIRONMENT.
- Add logger name to every record so Grafana can filter by source.
- Add optional OpenTelemetry trace_id / span_id correlation. Import is
  resolved once at module load; runtime calls are wrapped so a flaky tracer
  SDK can never break logging.
- Add default-on PII redaction for password, token, api_key, authorization,
  cookie, etc. Walks nested dicts, lists, and tuples up to depth 4. Extra
  keys via LANGFLOW_LOG_REDACT_KEYS.
- Add per-logger level overrides via LANGFLOW_LOG_LEVELS="name=LEVEL,...".
  Malformed entries (typos like WARN instead of WARNING) raise UserWarning
  instead of silently dropping.
- Use ISO 8601 UTC timestamps.
- Install a stdlib InterceptHandler on the root logger in JSON modes so
  uvicorn, sqlalchemy, httpx, langchain, asyncio etc. emit a single unified
  JSON stream. Forwards exc_info and stack_info. emit() is wrapped to route
  any error through handleError so a malformed third-party log call cannot
  raise into the request path. Install is idempotent: re-running configure()
  updates the level instead of stacking handlers. Not installed in pretty
  mode so dev terminals don't get duplicate lines.
- Reset cached loggers at the start of configure() so modules that captured
  a logger before configure() ran pick up the new processor chain.
- Preserve the get_logger() name through PrintLogger so add_logger_name can
  attach it.

Tests in src/backend/tests/unit/test_logger.py:
- Cover structured tracebacks, PII redaction (top-level, nested, list,
  tuple, depth limit), logger name, stdlib intercept forwarding exc_info
  and stack_info, intercept idempotency, intercept-not-installed in pretty
  mode, malformed-args safety net, service info defaults and env overrides,
  malformed LANGFLOW_LOG_LEVELS warning, container_csv exception text,
  show_locals default off (verified by absence of the secret value, not
  just the key) and opt-in.

* docs(observability): Grafana + Loki reference stack and env-var docs

Adds a self-contained Loki + Promtail + Grafana docker-compose stack under
deploy/observability/grafana-loki/ with a pre-provisioned dashboard that
demonstrates the production logging features: structured tracebacks, PII
redaction, service/version/environment labels, and the stdlib intercept
path. Anyone running Langflow in JSON mode can point Promtail at their log
file and get a working board on first run.

Also documents the new env vars (LANGFLOW_SERVICE_NAME, LANGFLOW_VERSION,
LANGFLOW_ENVIRONMENT, LANGFLOW_LOG_LEVELS, LANGFLOW_LOG_REDACT_KEYS,
LANGFLOW_LOG_TRACE_LOCALS) in docs/docs/Develop/logging.mdx and adds a
new page docs/docs/Develop/observability-grafana-loki.mdx covering JSON
output shape, structured exceptions, stdlib routing, and OpenTelemetry
trace correlation. Registered in the Observability sidebar category.

* docs(observability): rename dashboard to 'Langflow Logs'

* docs(observability): fix broken link from logging guide to Grafana/Loki page

The logging guide linked to the new Grafana/Loki page with an absolute
path (/observability-grafana-loki). Since the page is new and only
exists in the next docs version, the absolute link resolved to a
non-existent root-version route and failed the Docusaurus broken-link
check. Use a version-aware relative .mdx link instead.

* docs(observability): clarify stdout requirement for unified JSON logs

The Grafana/Loki guide told users to set LANGFLOW_LOG_FILE, but the stdlib
intercept that routes uvicorn/sqlalchemy/httpx/langchain into the JSON stream
is only installed on the stdout path, so those library logs landed in the file
as plain text and the json parse stage could not label them. Point the guide at
stdout (redirected to the scraped file) and note the file-mode limitation.

Also document that the JSON format is platform-agnostic and works with any
JSON-ingesting backend including IBM Instana, whose OpenTelemetry-based Python
tracer (3.0+) correlates logs to traces via trace_id/span_id.

* fix(logging): render stdlib logs as redacted JSON in file mode

In JSON mode with LANGFLOW_LOG_FILE set, the stdlib intercept was skipped to
avoid a recursion loop, so third-party loggers (uvicorn, sqlalchemy, httpx,
asyncio) wrote plain text straight to the file. Those lines bypassed both JSON
rendering and PII redaction, so secrets in their structured fields landed in the
file verbatim and Loki could not parse or label them.

Route JSON file output through a structlog ProcessorFormatter on the rotating
handler: foreign stdlib records are enriched via foreign_pre_chain (ExtraAdder +
redaction) and rendered as JSON alongside application logs, while the
RotatingFileHandler keeps log rotation. The stdout path is unchanged. Also
forward stdlib extra fields through InterceptHandler so the stdout path redacts
them too, keeping both paths consistent.

Update the Grafana/Loki deploy guide to reflect that LANGFLOW_LOG_FILE now
produces a single redacted JSON stream.

* fix(logging): retrieval buffer captures the message text

add_serialized stored the rendered message under the 'message' key, but
SizedLogBuffer.write only read 'event'/'msg'/'text', so every entry returned by
the /logs and /logs-stream endpoints had an empty message. Read 'message' first
and keep the other keys as fallbacks for records written in other shapes.
2026-06-01 20:23:31 +00:00

618 lines
18 KiB
JavaScript

module.exports = {
docs: [
{
type: "html",
value: `<div class="sidebar-group-label">Build</div>`,
className: "sidebar-group-divider",
},
{
type: "category",
label: "Get started",
className: "sidebar-category-with-icon sidebar-icon-rocket",
items: [
{
type: "doc",
id: "Get-Started/about-langflow",
label: "About Langflow"
},
{
type: "doc",
id: "Get-Started/get-started-installation",
label: "Install Langflow"
},
{
type: "doc",
id: "Get-Started/get-started-quickstart",
label: "Quickstart"
},
{
type: "category",
label: "Tutorials",
items: [
"Tutorials/chat-with-rag",
"Tutorials/chat-with-files",
"Tutorials/agent",
"Tutorials/mcp-tutorial",
],
},
],
},
{
type: "category",
label: "Flows",
className: "sidebar-category-with-icon sidebar-icon-workflow",
items: [
{
type: "doc",
id: "Flows/concepts-overview",
label: "Use the visual editor"
},
{
type: "doc",
id: "Flows/concepts-flows",
label: "Build flows"
},
{
type: "category",
label: "Run flows",
items: [
{
type: "doc",
id: "Flows/concepts-publish",
label: "Trigger flows with the Langflow API"
},
{
type: "doc",
id: "Flows/webhook",
label: "Trigger flows with webhooks"
},
],
},
{
type: "doc",
id: "Flows/concepts-playground",
label: "Test flows"
},
{
type: "doc",
id: "Flows/concepts-flows-import",
label: "Import and export flows"
},
{
type: "doc",
id: "Flows/langflow-assistant",
label: "Build components with Langflow Assistant"
},
],
},
{
type: "category",
label: "Agents",
className: "sidebar-category-with-icon sidebar-icon-bot",
items: [
"Agents/agents",
"Agents/agents-tools",
],
},
{
type: "category",
label: "Model Context Protocol (MCP)",
className: "sidebar-category-with-icon sidebar-icon-plug",
items: [
"Agents/mcp-client",
"Agents/mcp-server",
"Agents/langflow-mcp-client",
"Agents/mcp-component-astra",
],
},
{
type: "html",
value: `<div class="sidebar-group-label">Develop & Deploy</div>`,
className: "sidebar-group-divider",
},
{
type: "category",
label: "Develop",
className: "sidebar-category-with-icon sidebar-icon-code",
items: [
"Develop/api-keys-and-authentication",
"Develop/jwt-authentication",
"Develop/install-custom-dependencies",
"Develop/configuration-global-variables",
"Develop/environment-variables",
{
type: "category",
label: "Storage and memory",
items: [
{
type: "doc",
id: "Develop/concepts-file-management",
label: "Manage files"
},
{
type: "doc",
id: "Develop/memory",
label: "Manage memory"
},
{
type: "doc",
id: "Develop/session-id",
label: "Use Session IDs"
},
"Develop/configuration-custom-database",
{
type: "doc",
id: "Develop/enterprise-database-guide",
label: "Database guide for enterprise administrators"
},
"Develop/knowledge",
"Develop/memory-bases",
],
},
{
type: "category",
label: "Observability",
items: [
"Develop/logging",
"Develop/observability-grafana-loki",
"Develop/traces",
{
type: "category",
label: "Monitoring",
items: [
"Develop/integrations-arize",
"Develop/integrations-langfuse",
"Develop/integrations-langsmith",
"Develop/integrations-langwatch",
"Develop/integrations-openlayer",
"Develop/integrations-opik",
"Develop/integrations-instana-traceloop",
],
},
"Develop/contributing-telemetry",
],
},
{
type: "doc",
id: "Develop/data-types",
label: "Use Langflow data types"
},
{
type: "doc",
id: "Develop/concepts-voice-mode",
label: "Use voice mode"
},
{
type: "doc",
id: "Develop/configuration-cli",
label: "Use the Langflow CLI"
},
{
type: "category",
label: "Bundle extensions",
items: [
"Develop/extensions-overview",
"Develop/extensions-quickstart",
"Develop/extensions-manifest",
],
},
],
},
{
type: "category",
label: "Deploy",
className: "sidebar-category-with-icon sidebar-icon-cloud",
items: [
{
type:"doc",
id: "Deployment/deployment-overview",
label: "Langflow deployment overview"
},
{
type: "doc",
id: "Deployment/deployment-public-server",
label: "Deploy a public Langflow server"
},
{
type: "doc",
id: "Deployment/deployment-nginx-ssl",
label: "Deploy Langflow with Nginx and SSL"
},
{
type: "category",
label: "Containerized deployments",
items: [
"Deployment/develop-application",
{
type: "doc",
id: "Deployment/deployment-docker",
label: "Langflow Docker images"
},
{
type: "doc",
id: "Deployment/deployment-caddyfile",
label: "Deploy Langflow on a remote server"
},
{
type: "category",
label: "Kubernetes",
items: [
{
type: "doc",
id: "Deployment/deployment-architecture",
label: "Deployment architecture"
},
{
type: "doc",
id: "Deployment/deployment-prod-best-practices",
label: "Best practices"
},
{
type: "doc",
id: "Deployment/deployment-kubernetes-dev",
label: "Deploy in development"
},
{
type: "doc",
id: "Deployment/deployment-kubernetes-prod",
label: "Deploy in production"
},
]
},
],
},
{
type: "category",
label: "Cloud platforms",
items: [
{
type: "doc",
id: "Deployment/deployment-gcp",
label: "Google Cloud Platform"
},
{
type: "doc",
id: "Deployment/deployment-hugging-face-spaces",
label: "Hugging Face Spaces"
},
{
type: "doc",
id: "Deployment/deployment-wxo",
label: "IBM watsonx Orchestrate"
},
{
type: "doc",
id: "Deployment/deployment-railway",
label: "Railway"
},
{
type: "doc",
id: "Deployment/deployment-render",
label: "Render"
},
]
},
{
type: "doc",
id: "Deployment/deployment-multi-worker",
label: "Deploy with multiple workers",
},
{
type: "doc",
id: "Deployment/deployment-block-custom-components",
label: "Block custom components",
},
{
type: "doc",
id: "Deployment/security",
label: "Security",
},
],
},
{
type: "html",
value: `<div class="sidebar-group-label">Reference</div>`,
className: "sidebar-group-divider",
},
{
type: "category",
label: "Components reference",
className: "sidebar-category-with-icon sidebar-icon-blocks",
items: [
"Components/concepts-components",
{
type: "category",
label: "Core components",
items: [
{
type: "category",
label: "Input / Output",
items: [
"Components/chat-input-and-output",
"Components/text-input-and-output",
"Components/webhook",
]
},
{
type: "category",
label: "Processing",
items: [
"Components/data-operations",
"Components/dataframe-operations",
"Components/dynamic-create-data",
"Components/parser",
"Components/split-text",
"Components/text-operations",
"Components/type-convert",
]
},
{
type: "category",
label: "Data Source",
items: [
"Components/api-request",
"Components/mock-data",
"Components/url",
"Components/web-search",
]
},
{
type: "category",
label: "Files and Knowledge",
items: [
"Components/file-system",
"Components/knowledge-base",
"Components/memory-base",
"Components/read-file",
"Components/write-file",
]
},
{
type: "category",
label: "Flow Controls",
items: [
"Components/if-else",
"Components/loop",
"Components/notify-and-listen",
"Components/run-flow",
]
},
{
type: "category",
label: "LLM Operations",
items: [
"Components/batch-run",
"Components/guardrails",
"Components/policies",
"Components/llm-selector",
"Components/smart-router",
"Components/smart-transform",
"Components/structured-output",
]
},
{
type: "category",
label: "Models and Agents",
items: [
"Components/components-models",
"Components/components-prompts",
"Components/components-agents",
"Components/mcp-tools",
"Components/components-embedding-models",
"Components/message-history",
]
},
{
type: "category",
label: "Utilities",
items: [
"Components/calculator",
"Components/current-date",
"Components/python-interpreter",
"Components/sql-database",
]
},
"Components/legacy-core-components",
],
},
{
type: "category",
label: "Bundles",
items: [
"Components/components-bundles",
"Components/bundles-agentics",
"Components/bundles-aiml",
"Components/bundles-altk",
"Components/bundles-amazon",
"Components/bundles-anthropic",
"Components/bundles-apify",
"Components/bundles-arxiv",
"Components/bundles-assemblyai",
"Components/bundles-azure",
"Components/bundles-baidu",
"Components/bundles-bing",
"Components/bundles-cassandra",
"Components/bundles-chroma",
"Components/bundles-cleanlab",
"Components/bundles-clickhouse",
"Components/bundles-cloudflare",
"Components/bundles-cohere",
"Components/bundles-cometapi",
"Components/bundles-composio",
"Components/bundles-couchbase",
"Components/bundles-cuga",
"Components/bundles-datastax",
"Components/bundles-deepseek",
"Components/bundles-docling",
"Components/bundles-duckduckgo",
"Components/bundles-elastic",
"Components/bundles-exa",
"Components/bundles-faiss",
"Components/bundles-glean",
"Components/bundles-google",
"Components/bundles-groq",
"Components/bundles-huggingface",
"Components/bundles-ibm",
"Components/bundles-icosacomputing",
"Components/bundles-langchain",
"Components/bundles-lite-llm",
"Components/bundles-lmstudio",
"Components/bundles-maritalk",
"Components/bundles-mem0",
"Components/bundles-milvus",
"Components/bundles-mistralai",
"Components/bundles-mongodb",
"Components/bundles-notion",
"Components/bundles-novita",
"Components/bundles-nvidia",
"Components/bundles-ollama",
"Components/bundles-openai",
"Components/bundles-openrouter",
"Components/bundles-perplexity",
"Components/bundles-pgvector",
"Components/bundles-pinecone",
"Components/bundles-qdrant",
"Components/bundles-redis",
"Components/bundles-sambanova",
"Components/bundles-searchapi",
"Components/bundles-serper",
"Components/bundles-supabase",
"Components/bundles-upstash",
"Components/bundles-vllm",
"Components/bundles-vectara",
"Components/bundles-vertexai",
"Components/bundles-weaviate",
"Components/bundles-wikipedia",
"Components/bundles-xai",
],
},
"Components/components-custom-components",
],
},
{
type: "category",
label: "API reference",
className: "sidebar-category-with-icon sidebar-icon-fileCode",
items: [
{
type: "doc",
id: "API-Reference/api-reference-api-examples",
label: "Get started with the Langflow API",
},
{
type: "doc",
id: "API-Reference/typescript-client",
label: "Use the TypeScript client"
},
{
type: "doc",
id: "API-Reference/flow-devops-sdk",
label: "Flow DevOps Toolkit SDK",
},
{
type: "doc",
id: "API-Reference/api-flows-run",
label: "Flow trigger endpoints",
},
{
type: "category",
label: "Developer API (Beta)",
items: [
"API-Reference/workflows-api",
{
type: "link",
label: "Workflow API specification (Beta)",
href: "/api/workflow",
},
],
},
"API-Reference/api-openai-responses",
"API-Reference/api-flows",
"API-Reference/api-files",
"API-Reference/api-projects",
"API-Reference/api-logs",
"API-Reference/api-monitor",
"API-Reference/api-build",
"API-Reference/api-users",
{
type: "link",
label: "Langflow API specification",
href: "/api",
},
],
},
{
type: "html",
value: `<div class="sidebar-group-label">Community</div>`,
className: "sidebar-group-divider",
},
{
type: "category",
label: "Contribute",
className: "sidebar-category-with-icon sidebar-icon-gitPR",
items: [
"Contributing/contributing-community",
"Contributing/contributing-how-to-contribute",
"Contributing/contributing-components",
"Contributing/contributing-bundles",
"Contributing/contributing-component-tests",
"Contributing/contributing-templates",
],
},
{
type: "category",
label: "Support",
className: "sidebar-category-with-icon sidebar-icon-helpCircle",
items: [
{
type: "doc",
id: "Support/troubleshooting",
label: "Troubleshoot",
},
{
type: "doc",
id: "Support/macos-support-matrix",
label: "macOS support",
},
{
type: "doc",
id: "Support/contributing-github-issues",
label: "Get help and request enhancements",
},
{
type: "doc",
id: "Support/luna-for-langflow",
label: "IBM Elite Support for Langflow",
},
{
type: "doc",
id: "Support/release-notes",
label: "Release notes",
},
],
},
{
type: "html",
className: "sidebar-ad",
value: `
<a href="https://www.langflow.org/desktop" target="_blank" rel="noopener noreferrer" class="menu__link">
<svg width="18" height="18" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg" aria-hidden="true">
<g clip-path="url(#clip0_1645_37)">
<path d="M12 17H20C21.1046 17 22 16.1046 22 15V13M12 17H4C2.89543 17 2 16.1046 2 15V5C2 3.89543 2.89543 3 4 3H10M12 17V21M8 21H12M12 21H16M11.75 10.2917H13.2083L16.125 7.375H17.5833L20.5 4.45833H21.9583M16.125 11.75H17.5833L20.5 8.83333H21.9583M11.75 5.91667H13.2083L16.125 3H17.5833" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
</g>
<defs>
<clipPath id="clip0_1645_37">
<rect width="24" height="24" fill="white"/>
</clipPath>
</defs>
</svg>
<div class="sidebar-ad-text-container">
<span class="sidebar-ad-text">Get started in minutes</span>
<span class="sidebar-ad-text sidebar-ad-text-gradient">Download Langflow Desktop</span>
</div>
</a>
`,
},
],
};