Potential fix for
[https://github.com/infiniflow/ragflow/security/code-scanning/60](https://github.com/infiniflow/ragflow/security/code-scanning/60)
In general, the correct fix is to ensure that no sensitive data
(passwords, API keys, full connection strings with embedded credentials,
etc.) is ever written to logs. This can be done by (1) whitelisting only
clearly non-sensitive fields for logging, and/or (2) explicitly
scrubbing or masking any value that might contain credentials before
logging, and (3) not relying on later deletion from the dictionary to
protect against logging, since the log call already happened.
For this function, the best minimal fix is:
- Keep the idea of a safe key whitelist, but strengthen it so we are
absolutely sure we never log `password` or `connection_string`, even
indirectly.
- Avoid building the logged dict from the same potentially-tainted
`kwargs` object before we have removed sensitive keys, or relying solely
on key names that might change.
- Construct a separate, small log context that is obviously safe:
scheme, host, port, database, table, and possibly a boolean like
`has_password` instead of the password itself.
- Optionally, add a small helper to derive this safe log context, but
given the scope we can keep it inline.
Concretely in `rag/utils/opendal_conn.py`:
- Replace the current `SAFE_LOG_KEYS` / `loggable_kwargs` /
`logging.info(...)` block so that:
- We do not pass through arbitrary `kwargs` values by key filtering
alone.
- We instead build a new dict with explicitly chosen, non-sensitive
fields, e.g.:
```python
safe_log_info = {
"scheme": kwargs.get("scheme"),
"host": kwargs.get("host"),
"port": kwargs.get("port"),
"database": kwargs.get("database"),
"table": kwargs.get("table"),
"has_password": "password" in kwargs or "connection_string" in kwargs,
}
logging.info("Loaded OpenDAL configuration (non sensitive fields only):
%s", safe_log_info)
```
- This makes sure that neither the password nor a connection string
containing it is ever logged, while still retaining useful diagnostic
information.
- Keep the existing deletion of `password` and `connection_string` from
`kwargs` after logging, as an additional safety measure for any later
use of `kwargs`.
No new imports or external libraries are required; we only modify lines
45–56 of the shown snippet.
_Suggested fixes powered by Copilot Autofix. Review carefully before
merging._
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
### What problem does this PR solve?
change：
async issue and sensitive logging
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Refactor import modules.
### Type of change
- [x] Refactoring
---------
Signed-off-by: jinhai <haijin.chn@gmail.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
### What problem does this PR solve?
Add missing env var `MYSQL_MAX_PACKET` to service_conf.yaml.template,
and add default values to opendal config to fix npe.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Use `quote_plus` to escape password in opendal's mysql url to support
special characters like `#`.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Fix the config option name of the opendal table name and setting of
'max_allowed_packet'.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Signed-off-by: He Wang <wanghechn@qq.com>
**Context and Purpose:**
This PR automatically remediates a security vulnerability:
- **Description:** Detected possible formatted SQL query. Use
parameterized queries instead.
- **Rule ID:**
python.lang.security.audit.formatted-sql-query.formatted-sql-query
- **Severity:** HIGH
- **File:** rag/utils/opendal_conn.py
- **Lines Affected:** 98 - 98
This change is necessary to protect the application from potential
security risks associated with this vulnerability.
**Solution Implemented:**
The automated remediation process has applied the necessary changes to
the affected code in `rag/utils/opendal_conn.py` to resolve the
identified issue.
Please review the changes to ensure they are correct and integrate as
expected.
### What problem does this PR solve?
This PR fixes two issues in the OpenDAL storage connector:
1. The `health` method was missing, which prevented health checks on
the storage backend.
3. The initialization of the `opendal.Operator` object included a
redundant scheme parameter, causing unnecessary duplication and
potential confusion.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### Background
- The absence of a `health` method made it difficult to verify the
availability and reliability of the storage service.
- Initializing `opendal.Operator` with both `self._scheme` and
unpacked `**self._kwargs` could lead to errors or unexpected behavior
if the scheme was already included in the kwargs.
### What is changed and how it works?
- Adds a `health` method that writes a test file to verify storage
availability.
- Removes the duplicate scheme parameter from the `opendal.Operator`
initialization to ensure clarity and prevent conflicts.
before:
<img width="762" alt="企业微信截图_46be646f-2e99-4e5e-be67-b1483426e77c"
src="https://github.com/user-attachments/assets/acecbb8c-4810-457f-8342-6355148551ba"
/>
<img width="767" alt="image"
src="https://github.com/user-attachments/assets/147cd5a2-dde3-466b-a9c1-d1d4f0819e5d"
/>
after:
<img width="1123" alt="企业微信截图_09d62997-8908-4985-b89f-7a78b5da55ac"
src="https://github.com/user-attachments/assets/97dc88c9-0f4e-4d77-88b3-cd818e8da046"
/>
### What problem does this PR solve?
This PR resolves the inconsistency in the opendal configuration where
both `schema` and `scheme` were used as keys. The code and
configuration file now consistently use `scheme`, which helps prevent
configuration errors and runtime issues. This change improves code
clarity and maintainability.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### Additional context
- Updated both `conf/service_conf.yaml` and
`rag/utils/opendal_conn.py` to use `scheme` instead of `schema`
- No breaking changes to other configuration fields
### What problem does this PR solve?
#8074
Oss support opendal(including mysql)
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
