mirror of
https://github.com/langflow-ai/langflow.git
synced 2026-07-23 20:22:46 +08:00
e2d17a6d84
* feat(logging): production-grade structured logs for Grafana/Loki Make langflow and lfx log output viable for ingestion by Grafana/Loki and other observability tools when run in JSON mode (LANGFLOW_LOG_ENV=container). Core changes in src/lfx/src/lfx/log/logger.py: - Preserve exceptions in JSON output via structlog.processors.ExceptionRenderer with ExceptionDictTransformer. Tracebacks now emit as a structured exception array (exc_type, exc_value, frames) instead of being dropped. - show_locals defaults to OFF; opt-in via LANGFLOW_LOG_TRACE_LOCALS=true so frame locals can't leak API keys, env, or request bodies in shipped logs. - Add service metadata (service / version / environment) from LANGFLOW_SERVICE_NAME / LANGFLOW_VERSION / LANGFLOW_ENVIRONMENT. - Add logger name to every record so Grafana can filter by source. - Add optional OpenTelemetry trace_id / span_id correlation. Import is resolved once at module load; runtime calls are wrapped so a flaky tracer SDK can never break logging. - Add default-on PII redaction for password, token, api_key, authorization, cookie, etc. Walks nested dicts, lists, and tuples up to depth 4. Extra keys via LANGFLOW_LOG_REDACT_KEYS. - Add per-logger level overrides via LANGFLOW_LOG_LEVELS="name=LEVEL,...". Malformed entries (typos like WARN instead of WARNING) raise UserWarning instead of silently dropping. - Use ISO 8601 UTC timestamps. - Install a stdlib InterceptHandler on the root logger in JSON modes so uvicorn, sqlalchemy, httpx, langchain, asyncio etc. emit a single unified JSON stream. Forwards exc_info and stack_info. emit() is wrapped to route any error through handleError so a malformed third-party log call cannot raise into the request path. Install is idempotent: re-running configure() updates the level instead of stacking handlers. Not installed in pretty mode so dev terminals don't get duplicate lines. - Reset cached loggers at the start of configure() so modules that captured a logger before configure() ran pick up the new processor chain. - Preserve the get_logger() name through PrintLogger so add_logger_name can attach it. Tests in src/backend/tests/unit/test_logger.py: - Cover structured tracebacks, PII redaction (top-level, nested, list, tuple, depth limit), logger name, stdlib intercept forwarding exc_info and stack_info, intercept idempotency, intercept-not-installed in pretty mode, malformed-args safety net, service info defaults and env overrides, malformed LANGFLOW_LOG_LEVELS warning, container_csv exception text, show_locals default off (verified by absence of the secret value, not just the key) and opt-in. * docs(observability): Grafana + Loki reference stack and env-var docs Adds a self-contained Loki + Promtail + Grafana docker-compose stack under deploy/observability/grafana-loki/ with a pre-provisioned dashboard that demonstrates the production logging features: structured tracebacks, PII redaction, service/version/environment labels, and the stdlib intercept path. Anyone running Langflow in JSON mode can point Promtail at their log file and get a working board on first run. Also documents the new env vars (LANGFLOW_SERVICE_NAME, LANGFLOW_VERSION, LANGFLOW_ENVIRONMENT, LANGFLOW_LOG_LEVELS, LANGFLOW_LOG_REDACT_KEYS, LANGFLOW_LOG_TRACE_LOCALS) in docs/docs/Develop/logging.mdx and adds a new page docs/docs/Develop/observability-grafana-loki.mdx covering JSON output shape, structured exceptions, stdlib routing, and OpenTelemetry trace correlation. Registered in the Observability sidebar category. * docs(observability): rename dashboard to 'Langflow Logs' * docs(observability): fix broken link from logging guide to Grafana/Loki page The logging guide linked to the new Grafana/Loki page with an absolute path (/observability-grafana-loki). Since the page is new and only exists in the next docs version, the absolute link resolved to a non-existent root-version route and failed the Docusaurus broken-link check. Use a version-aware relative .mdx link instead. * docs(observability): clarify stdout requirement for unified JSON logs The Grafana/Loki guide told users to set LANGFLOW_LOG_FILE, but the stdlib intercept that routes uvicorn/sqlalchemy/httpx/langchain into the JSON stream is only installed on the stdout path, so those library logs landed in the file as plain text and the json parse stage could not label them. Point the guide at stdout (redirected to the scraped file) and note the file-mode limitation. Also document that the JSON format is platform-agnostic and works with any JSON-ingesting backend including IBM Instana, whose OpenTelemetry-based Python tracer (3.0+) correlates logs to traces via trace_id/span_id. * fix(logging): render stdlib logs as redacted JSON in file mode In JSON mode with LANGFLOW_LOG_FILE set, the stdlib intercept was skipped to avoid a recursion loop, so third-party loggers (uvicorn, sqlalchemy, httpx, asyncio) wrote plain text straight to the file. Those lines bypassed both JSON rendering and PII redaction, so secrets in their structured fields landed in the file verbatim and Loki could not parse or label them. Route JSON file output through a structlog ProcessorFormatter on the rotating handler: foreign stdlib records are enriched via foreign_pre_chain (ExtraAdder + redaction) and rendered as JSON alongside application logs, while the RotatingFileHandler keeps log rotation. The stdout path is unchanged. Also forward stdlib extra fields through InterceptHandler so the stdout path redacts them too, keeping both paths consistent. Update the Grafana/Loki deploy guide to reflect that LANGFLOW_LOG_FILE now produces a single redacted JSON stream. * fix(logging): retrieval buffer captures the message text add_serialized stored the rendered message under the 'message' key, but SizedLogBuffer.write only read 'event'/'msg'/'text', so every entry returned by the /logs and /logs-stream endpoints had an empty message. Read 'message' first and keep the other keys as fallbacks for records written in other shapes.
Run Langflow
Docker compose
To run Langflow with Docker compose, you need to have Docker and Docker compose installed on your machine. You can install Docker and Docker compose by following the instructions on the official Docker documentation.
The docker-compose file uses latest tag; it's recommended to pull the latest version of the images before running the docker-compose file.
docker compose pull
To start the Langflow services, run the following command:
docker compose up
After running the command, you can access the Langflow services at the following url: http://localhost:80.
Edit the .env file to change the port or other configurations.