Commit Graph

1321 Commits

Author SHA1 Message Date
3beb85efa0 Feat: enhance metadata arranging. (#12745)
### What problem does this PR solve?
#11564

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-22 15:34:08 +08:00
bc7b864a6c top_k parameter ignored, always returned page_size results (#12753)
### What problem does this PR solve?
**Backend**
\rag\nlp\search.py
*Before the fix*
The top_k parameter was not applied to limit the total number of chunks,
and the rerank model also uses the exact whole valid_idx rather than
assigning valid_idx = valid_idx[:top] firstly.
*After the fix*
The top_k limit is applied to the total results before pagination, using
a default value of top = 1024 if top_k is not modified.

session.py
*Before the fix:*
When the frontend calls the retrieval API with `search_id`, the backend
only reads `meta_data_filter` from the saved `search_config`. The
`rerank_id`, `top_k`, `similarity_threshold`, and
`vector_similarity_weight` parameters are only taken from the direct
request body. Since the frontend doesn't pass these parameters
explicitly (it only passes `search_id`), they always fall back to
default values:
- `similarity_threshold` = 0.0
- `vector_similarity_weight` = 0.3
- `top_k` = 1024
- `rerank_id` = "" (no rerank)
This means user settings saved in the Search Settings page have no
effect on actual search results.

*After the fix:*
When a `search_id` is provided, the backend now reads all relevant
configuration from the saved `search_config`, including `rerank_id`,
`top_k`, `similarity_threshold`, and `vector_similarity_weight`. Request
parameters can still override these values if explicitly provided,
allowing flexibility. The rerank model is now properly instantiated
using the configured `rerank_id`, making the rerank feature actually
work.



**Frontend** 
\web\src\pages\next-search\search-setting.tsx
*Before the fix*
search-setting.tsx file, the top_k input box is only displayed when
rerank is enabled (wrapped in the rerankModelDisabled condition). If the
rerank switch is turned off, the top_k input field will be hidden, but
the form value will remain unchanged. In other words: - When rerank is
enabled, users can modify top_k (default 1024). - When rerank is
disabled, top_k retains the previous value, but it's not visible on the
interface. Therefore, the backend will always receive the top_k
parameter; it's just that the frontend UI binds this configuration item
to the rerank switch. When rerank is turned off, top_k will not
automatically reset to 1024, but will retain its original value.
*After the fix*
On the contrary, if we switch off the button rerank model, the value
top-k will be reset to 1024. By the way, If we use top-k in an
individual method, rather than put it into the method retrieval, we can
control it separately



Now all methods valid
Using rerank

<img width="2378" height="1565" alt="Screenshot 2026-01-21 190206"
src="https://github.com/user-attachments/assets/fa2b0df0-1334-4ca3-b169-da6c5fd59935"
/>

Not using rerank
<img width="2596" height="1559" alt="Screenshot 2026-01-21 190229"
src="https://github.com/user-attachments/assets/c5a80522-a0e1-40e7-b349-42fe86df3138"
/>




Before fixing they are the same

### Type of change
- Bug Fix (non-breaking change which fixes an issue)
2026-01-22 15:33:42 +08:00
6f3f69b62e Feat: API adds audio to text and text to speech functions (#12764)
### What problem does this PR solve?

API adds audio to text and text to speech functions

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-22 11:20:26 +08:00
89f438fe45 Add ping command to test ping API (#12757)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-22 00:18:29 +08:00
5836823187 Refactor:better handle list agent api desc param (#12733)
### What problem does this PR solve?
better handle list agent api desc param

### Type of change

- [x] Refactoring
2026-01-21 13:09:27 +08:00
960ecd3158 Feat: update and add new tests for web api apps (#12714)
### What problem does this PR solve?

This PR adds missing web API tests (system, search, KB, LLM, plugin,
connector). It also addresses a contract mismatch that was causing test
failures: metadata updates did not persist new keys (update‑only
behavior).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): Test coverage expansion and test helper
instrumentation
2026-01-20 19:12:15 +08:00
aee9860970 Make document change-status idempotent for Infinity doc store (#12717)
### What problem does this PR solve?

This PR makes the document change‑status endpoint idempotent under the
Infinity doc store. If a document already has the requested status, the
handler returns success without touching the engine, preventing
unnecessary updates and avoiding missing‑table errors while keeping
responses consistent.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 19:11:21 +08:00
32841549c1 Fix: Not within a request context (#12723)
### What problem does this PR solve?

ERROR    1819426 Unhandled exception during request
Traceback (most recent call last):
File
"/home/qinling/[github.com/infiniflow/ragflow/api/apps/document_app.py](http://github.com/infiniflow/ragflow/api/apps/document_app.py)",
line 639, in run
    return await thread_pool_exec(_run_sync)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/common/misc_utils.py](http://github.com/infiniflow/ragflow/common/misc_utils.py)",
line 132, in thread_pool_exec
return await loop.run_in_executor(_thread_pool_executor(), func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/futures.py", line 287, in __await__
    yield self  # This tells Task to wait for completion.
    ^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/tasks.py", line 385, in __wakeup
    future.result()
  File "/usr/lib/python3.12/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/api/apps/document_app.py](http://github.com/infiniflow/ragflow/api/apps/document_app.py)",
line 593, in _run_sync
if not DocumentService.accessible(doc_id,
[current_user.id](http://current_user.id/)):
                                              ^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 318, in __get__
    obj = instance._get_current_object()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 526, in _get_current_object
    return get_name(local())
                    ^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/api/apps/__init__.py](http://github.com/infiniflow/ragflow/api/apps/__init__.py)",
line 97, in _load_user
    authorization = request.headers.get("Authorization")
                    ^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 318, in __get__
    obj = instance._get_current_object()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 519, in _get_current_object
    raise RuntimeError(unbound_message) from None
RuntimeError: Not within a request context

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 16:56:41 +08:00
927db0b373 Refa: asyncio.to_thread to ThreadPoolExecutor to break thread limitat… (#12716)
### Type of change

- [x] Refactoring
2026-01-20 13:29:37 +08:00
120648ac81 fix: inaccurate error message when uploading multiple files containing an unsupported file type (#12711)
### What problem does this PR solve?

When uploading multiple files at once, if any of the files are of an
unsupported type and the blob is not removed, it triggers a
TypeError('Object of type bytes is not JSON serializable') exception.
This prevents the frontend from responding properly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 12:24:54 +08:00
b40d639fdb Add dataset with table parser type for Infinity and answer question in chat using SQL (#12541)
### What problem does this PR solve?

1) Create  dataset using table parser for infinity
2) Answer questions in chat using SQL

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-19 19:35:14 +08:00
4a7e40630b Refactor:memory delete will re-use super method (#12684)
### What problem does this PR solve?
memory delete will re-use super method

### Type of change

- [x] Refactoring
2026-01-19 12:45:37 +08:00
d6897b6054 Fix chat error (#12693)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-19 12:45:14 +08:00
0a8eb11c3d fix: Add proper error handling for database reconnection attempts (#12650)
## Problem
When database connection is lost, the reconnection logic had a bug: if
the first reconnect attempt failed, the second attempt was not wrapped
in error handling, causing unhandled exceptions.

## Solution
Added proper try-except blocks around the second reconnect attempt in
both MySQL and PostgreSQL database classes to ensure errors are properly
logged and handled.

## Changes
- Fixed `_handle_connection_loss()` in `RetryingPooledMySQLDatabase`
- Fixed `_handle_connection_loss()` in
`RetryingPooledPostgresqlDatabase`

Fixes #12294

---

Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=158349177

Co-authored-by: SID <158349177+0xsid0703@users.noreply.github.com>
2026-01-19 09:48:10 +08:00
38f0a92da9 Use RAGFlow CLI to replace RAGFlow Admin CLI (#12653)
### What problem does this PR solve?

```
$ python admin/client/ragflow_cli.py -t user -u aaa@aaa.com -p 9380

ragflow> list datasets;
ragflow> list default models;
ragflow> show version;

```


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-17 17:52:38 +08:00
4f036a881d Fix: Infinity keyword round-trip, highlight fallback, and KB update guards (#12660)
### What problem does this PR solve?

Fixes Infinity-specific API regressions: preserves ```important_kwd```
round‑trip for ```[""]```, restores required highlight key in retrieval
responses, and enforces Infinity guards for unsupported
```parser_id=tag``` and pagerank in ```/v1/kb/update```. Also removes a
slow/buggy pandas row-wise apply that was throwing ```ValueError``` and
causing flakiness.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 20:03:52 +08:00
2b20d0b3bb Fix : Web API tests by normalizing errors, validation, and uploads (#12620)
### What problem does this PR solve?

Fixes web API behavior mismatches that caused test failures by
normalizing error responses, tightening validations, correcting error
messages, and closing upload file handles.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 11:09:22 +08:00
cec06bfb5d Fix: empty chunk issue. (#12638)
#12570

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-15 17:46:21 +08:00
ac936005e6 fix: ensure deleted chunks are not returned in retrieval (#12520) (#12546)
## Summary
Fixes #12520 - Deleted chunks should not appear in retrieval/reference
results.

## Changes

### Core Fix
- **api/apps/chunk_app.py**: Include \doc_id\ in delete condition to
properly scope the delete operation

### Improved Error Handling
- **api/db/services/document_service.py**: Better separation of concerns
with individual try-catch blocks and proper logging for each cleanup
operation

### Doc Store Updates
- **rag/utils/es_conn.py**: Updated delete query construction to support
compound conditions
- **rag/utils/opensearch_conn.py**: Same updates for OpenSearch
compatibility

### Tests
- **test/testcases/.../test_retrieval_chunks.py**: Added
\TestDeletedChunksNotRetrievable\ class with regression tests
- **test/unit/test_delete_query_construction.py**: Unit tests for delete
query construction

## Testing
- Added regression tests that verify deleted chunks are not returned by
retrieval API
- Tests cover single chunk deletion and batch deletion scenarios
2026-01-15 14:45:55 +08:00
b40a7b2e7d Feat: Hash doc id to avoid duplicate name. (#12573)
### What problem does this PR solve?

Feat: Hash doc id to avoid duplicate name. 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-15 14:02:15 +08:00
9a10558f80 Refa: async retrieval process. (#12629)
### Type of change

- [x] Refactoring
- [x] Performance Improvement
2026-01-15 12:28:49 +08:00
SID
f82628c40c Fix: langfuse connection error handling #12621 (#12626)
## Description

Fixes connection error handling when langfuse service is unavailable.
The application now gracefully handles connection failures instead of
crashing.

## Changes

- Wrapped `langfuse.auth_check()` calls in try-except blocks in:
  - `api/db/services/dialog_service.py`
  - `api/db/services/tenant_llm_service.py`

## Problem

When langfuse service is unavailable or connection is refused,
`langfuse.auth_check()` throws `httpx.ConnectError: [Errno 111]
Connection refused`, causing the application to crash during document
parsing or dialog operations.

## Solution

Added try-except blocks around `langfuse.auth_check()` calls to catch
connection errors and gracefully skip langfuse tracing instead of
crashing. The application continues functioning normally even when
langfuse is unavailable.

## Related Issue

Fixes #12621

---

Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=158349177
2026-01-15 11:23:15 +08:00
15a8bb2e9c Fix: chunk list async issue. (#12615)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-14 17:32:07 +08:00
f72a35188d refactor: remove debug print statements (#12598)
### What problem does this PR solve?

This PR eliminates unnecessary debug print statements that were left in
hot paths of the codebase.

### Type of change

- [x] Refactoring
2026-01-14 10:05:34 +08:00
0795616b34 Align p3 HTTP/SDK tests with current backend behavior (#12563)
### What problem does this PR solve?

Updates pre-existing HTTP API and SDK tests to align with current
backend behavior (validation errors, 404s, and schema defaults). This
ensures p3 regression coverage is accurate without changing production
code.

### Type of change

- [x] Other (please describe): align p3 HTTP/SDK tests with current
backend behavior

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-13 19:22:47 +08:00
947e63ca14 Fixed typos and added pptx preview for frontend (#12577)
### What problem does this PR solve?
Previously, we added support for previewing PPT and PPTX files in the
backend. Now, we are adding it to the frontend, so when the slides in
the chat interface are referenced, they will no longer be blank.
### Type of change

- Bug Fix (non-breaking change which fixes an issue)
2026-01-13 17:02:36 +08:00
41c84fd78f Add MIME types for PPT and PPTX files (#12562)
Otherwise, slide files cannot be opened in Chat module

### What problem does this PR solve?

Backend Reason (API): In the api/utils/web_utils.py file of the backend,
the CONTENT_TYPE_MAP dictionary is missing ppt and pptx.
MIME type mapping. This means that when the frontend requests a PPTX
file, the backend cannot correctly inform the browser that it is a PPTX
file, resulting in the file being displayed incorrectly.
Type identification error.

### Type of change

-  Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-13 12:17:49 +08:00
44bada64c9 Feat: support tree structured deep-research policy. (#12559)
### What problem does this PR solve?

#12558
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-13 09:41:35 +08:00
a7dd3b7e9e Add time cost when start servers (#12552)
### What problem does this PR solve?

- API server
- Ingestion server
- Data sync server
- Admin server

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-12 12:48:23 +08:00
2e09db02f3 feat: add paddleocr parser (#12513)
### What problem does this PR solve?

Add PaddleOCR as a new PDF parser.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-09 17:48:45 +08:00
6abf55c048 Feat: support openapi (#12521)
### What problem does this PR solve?
Support OpenAPI interface description.

The issue of not supporting the Swagger interface after upgrading the
system framework from Flask to Quart has been resolved.

Resolved https://github.com/infiniflow/ragflow/issues/5264

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: puhaiyang <“761396462@qq.com”>
2026-01-09 17:48:20 +08:00
f9d4179bf2 Feat:memory sdk (#12538)
### What problem does this PR solve?

Move memory and message apis to /api, and add sdk support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-09 17:45:58 +08:00
f522391d1e Fix: "AttributeError(\"'list' object has no attribute 'get'\")" (#12518)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/12515

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-09 10:19:51 +08:00
14c250e3d7 Fix adding column error (#12503)
### What problem does this PR solve?

1. Fix redundant column adding
2. Refactor the code

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-08 16:44:53 +08:00
a093e616cf Fix: add multimodel models in chat api (#12496)
### What problem does this PR solve?

Fix: add multimodel models in chat api #11986
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-08 16:12:08 +08:00
696397ebba Fix: apply kb setting while re-parsing.... (#12501)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-08 16:11:50 +08:00
1996aa0dac Refactor: Enhance delta streaming in chat functions for improved reasoning and content handling (#12453)
### What problem does this PR solve?

change:
Enhance delta streaming in chat functions for improved reasoning and
content handling

### Type of change


- [x] Refactoring
2026-01-08 13:34:16 +08:00
f4e2783eb4 optimize doc id check: do not query db when doc id to validate is empty (#12500)
### What problem does this PR solve?
when a kb contains many documents, say 50000, and the retrieval is only
made against some kb without specifying any doc ids, the query for all
docs from the db is not necessary, and can be omitted to improve
performance.

### Type of change

- [x] Performance Improvement
2026-01-08 13:22:58 +08:00
2fd4a3134d Doc: memory http api (#12499)
### What problem does this PR solve?

Use task save function for add_message api, and added http API document.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Documentation Update
2026-01-08 12:54:10 +08:00
23a9544b73 Fix: toc async issue. (#12485)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 15:35:30 +08:00
011bbe9556 Feat: support context window for docx (#12455)
### What problem does this PR solve?

Feat: support context window for docx

#12303

Done:
- [x] naive.py
- [x] one.py

TODO:
- [ ] book.py
- [ ] manual.py

Fix: incorrect image position
Fix: incorrect chunk type tag

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-01-07 15:08:17 +08:00
07845be5bd Fix: display agent name for extract messages (#12480)
### What problem does this PR solve?

Display agent name for extract messages

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 13:19:54 +08:00
ca9645f39b Feat: adapt to , arglist (#12468)
### What problem does this PR solve?

Adapt to ',' joined arg list in get method url.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-07 09:59:08 +08:00
bdd9f3d4d1 Fix: try handle authorization as api-token (#12462)
### What problem does this PR solve?

Try handle authorization as api-token when jwt load failed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-06 19:25:42 +08:00
02e6870755 Refactor: import_test_cases use bulk_create (#12456)
### What problem does this PR solve?

import_test_cases use bulk_create

### Type of change

- [x] Refactoring
2026-01-06 11:39:07 +08:00
fada223249 Feat: process memory (#12445)
### What problem does this PR solve?

Add task status for raw message, and move extract message as a nested
property under raw message

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-05 17:58:32 +08:00
lif
81f9296d79 Fix: handle invalid img_id format in chunk update (#12422)
## Summary
- Fix ValueError when updating chunk with invalid/empty `img_id` format
- Add validation before splitting `img_id` by hyphen
- Use `split("-", 1)` to handle object names containing hyphens

## Test plan
- [x] Verify chunk update works with valid `img_id` (format:
`bucket-objectname`)
- [x] Verify chunk update doesn't crash with empty `img_id`
- [x] Verify chunk update doesn't crash when `img_id` has no hyphen
- [x] Verify ruff check passes

Fixes #12035

Signed-off-by: majiayu000 <1835304752@qq.com>
2026-01-05 11:27:19 +08:00
5ebe334a2f Refactor setting type (#12425)
### What problem does this PR solve?

Refactor setting type

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-04 20:26:12 +08:00
932496a8ec Fix:bug fix (#12423)
### What problem does this PR solve?
change: 
initialize webhook configuration in webhook function
remove debug print statement from airtable_connector
remove redundant uuid import in imap_connector

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-04 19:16:29 +08:00
ac9113b0ef feature: add system setting service (#12408)
### What problem does this PR solve?

#12409 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-04 14:21:39 +08:00