Commit Graph

2760 Commits

Author SHA1 Message Date
d17970ebd0 0321 chunkmethods (#6520)
### What problem does this PR solve?

#6061 

### Type of change


- [x] Documentation Update
2025-03-26 09:03:18 +08:00
bf483fdf02 Fix: describe parameter error. (#6519)
### What problem does this PR solve?
#6228

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-26 09:02:48 +08:00
b2b7ed8927 Fix: abnormal chunk id (#6506)
### What problem does this PR solve?

#6500

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-25 19:03:29 +08:00
0a79dfd5cf Test: Added test cases for List Chunks HTTP API (#6514)
### What problem does this PR solve?

cover [list
chunks](https://ragflow.io/docs/v0.17.2/http_api_reference#list-chunks)
endpoints

### Type of change

- [x] update test cases
2025-03-25 17:28:58 +08:00
1d73baf3d8 Feat: improve '/mv' '/list' API performance (#6502)
### What problem does this PR solve?

1. for /mv API use get by ids to avoid O(n) DB IO

2. for /list remove one useless call
### Type of change

- [x] Performance Improvement
2025-03-25 16:30:25 +08:00
f3ae4a3bae Fix: img_id errror. (#6504)
### What problem does this PR solve?

#6499

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-25 15:57:03 +08:00
814a210f5d Fix: failed to acquire lock exception with retry mechanism for postgres and mysql (#6483)
Added the with_retry decorator in db_models.py to add a retry mechanism
for database operations. Applied the retry mechanism to the lock and
unlock methods of the PostgresDatabaseLock and MysqlDatabaseLock classes
to enhance the reliability of lock operations.

### What problem does this PR solve?
resolve failed to acquire lock exception with retry mechanism

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
2025-03-25 15:09:56 +08:00
60c3a253ad Fix: api-key issue for xinference. (#6490)
### What problem does this PR solve?

#2792

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-25 15:01:13 +08:00
384b6549a6 Fix: remove doc status checking while creating an assistant. (#6486)
### What problem does this PR solve?

#6461

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-25 11:13:22 +08:00
b2ec39c59d Fix: Resolve FlowSetting not reading Title from .ts files (#6469)
### What problem does this PR solve?

Fix: Resolve FlowSetting not reading Title from .ts files

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-25 11:07:29 +08:00
095fc84cf2 Fix: claude max tokens. (#6484)
### What problem does this PR solve?

#6458

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-25 10:41:55 +08:00
542cf16292 Feat: add project_id and project_name to Langfuse API (#6481)
### What problem does this PR solve?

Enhance Langfuse API: add project_id and project_name

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-25 10:36:34 +08:00
27989eb9a5 Test: Add list chunk checkpoint for the add chunk API (#6482)
### What problem does this PR solve?

Add list chunk checkpoint for the add chunk API

### Type of change

- [x] update test cases
2025-03-25 10:36:21 +08:00
05997e8215 Remove thinking block from keyword node's result (#6474)
### What problem does this PR solve?

For now, if you use thinking model (deepseek-r1:32b with ollama server
in my case) in "Keyword" node, result contains all <think> block and so
node return not only keywords

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-03-25 10:22:41 +08:00
5d9afce12d Feat: improve the performance for '/upload' API (#6479)
### What problem does this PR solve?
improve the logic to fetch parent folder, remove the useless DB IO logic

### Type of change

- [x] Performance Improvement
2025-03-25 10:22:19 +08:00
ee6a0bd9db Refa: enhancement: enhance the prompt of related_question API (#6463)
### What problem does this PR solve?

Enhance the prompt of `related_question` API.

### Type of change

- [x] Enhancement
- [x] Documentation Update
2025-03-25 10:00:10 +08:00
b6f3242c6c Test: Update test cases to reduce execution time (#6470)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] update test cases
2025-03-25 09:17:05 +08:00
390086c6ab Fix: split process bug in graphrag extract (#6423)
### What problem does this PR solve?

1. miss completion delimiter.
2. miss bracket process.
3. doc_ids return by update_graph is a set, and insert operation in
extract_community need a list.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 21:41:20 +08:00
a40c5aea83 Miscellaneous UI updates (#6471)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2025-03-24 19:36:47 +08:00
f691b4ddd2 Feat: Improve "/convert" API's performance (#6465)
### What problem does this PR solve?

for batch requests based on get_by_ids to fetch all files first replace
the O(n) IO logic.

### Type of change


- [x] Performance Improvement
2025-03-24 19:08:22 +08:00
3c57a9986c Feat: Add LangfuseCard component. #6155 (#6468)
### What problem does this PR solve?

Feat: Add LangfuseCard component. #6155

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 19:07:55 +08:00
5e0a77df2b Feat: add Langfuse APIs (#6460)
### What problem does this PR solve?

Add Langfuse APIs

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 18:25:43 +08:00
66e557b6c0 Fix: Langfuse update model has no fields attribute (#6453)
### What problem does this PR solve?

Langfuse update model has no fields attribute

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 15:37:14 +08:00
200b6f55c6 Fix: NameError: free variable 'langfuse_generation' referenced before assignment in enclosing scope (#6451)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: lizheng@ssc-hn.com <lizheng@ssc-hn.com>
2025-03-24 15:14:36 +08:00
b77ce4e846 Feat: support api-key for Ollama. (#6448)
### What problem does this PR solve?

#6189

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 14:53:17 +08:00
85eb367ede Feat: add basic Langfuse support for LLM module (#6443)
### What problem does this PR solve?

#6155

Add basic Langfuse support for LLM module.

A trace example:

<img width="755" alt="image"
src="https://github.com/user-attachments/assets/25c1f852-5116-486c-a47f-6097187142ca"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 13:18:47 +08:00
0b63346a1a Test: Update test case for #6081 (#6446)
### What problem does this PR solve?

Update test case for #6081

### Type of change

- [x] Update test case
2025-03-24 13:18:12 +08:00
85eb3775d6 Refa: update Anthropic models. (#6445)
### What problem does this PR solve?

#6421

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 12:34:57 +08:00
e4c8d703b5 Test: Update test cases for PR #6194 #6259 #6376 (#6444)
### What problem does this PR solve?

PR #6194 #6259 #6376

### Type of change

- [x] Update test cases
2025-03-24 12:01:33 +08:00
60afb63d44 Feat: Add background-core-standard to tailwind.css #3221 (#6437)
### What problem does this PR solve?

Feat: Add background-core-standard to tailwind.css #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 10:51:46 +08:00
ee5aa51d43 Fix: point in tag issue. (#6436)
### What problem does this PR solve?

#6414

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 10:45:29 +08:00
a6aed0da46 Fix: rerank with YoudaoRerank issue. (#6396)
### What problem does this PR solve?

Fix rerank with YoudaoRerank issue,"'YoudaoRerank' object has no
attribute '_dynamic_batch_size'"


![17425412353825](https://github.com/user-attachments/assets/9ed304c7-317a-440e-acff-fe895fc20f07)


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-24 10:09:16 +08:00
d77380f024 Feat: support pic base bullet for PPT (#6406)
### What problem does this PR solve?

support pic base bullet for PPT

modify one mistake in document

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 09:31:31 +08:00
efc4796f01 Fix ratelimit errors during document parsing (#6413)
### What problem does this PR solve?

When using the online large model API knowledge base to extract
knowledge graphs, frequent Rate Limit Errors were triggered,
causing document parsing to fail. This commit fixes the issue by
optimizing API calls in the following way:
Added exponential backoff and jitter to the API call to reduce the
frequency of Rate Limit Errors.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-03-22 23:07:03 +08:00
d869e4d43f Fix: Preserve quotes while handling variable substitution withTemplate component. (#6410)
###Address Problem:
The original implementation used re.sub(r"(\\\"|\")", "", content) which
stripped all quotes from the processed content. While this worked for
simple Jinja2-rendered templates, it caused formatting issues when :
-Quotes were required in the final output (e.g., JSON, Python Code
strings)

###Solution:
    1. Selective JSON Serialization.
    2. Removed Global Quote Removal

### What problem does this PR solve?

This PR addresses an issue in template processing where all quotation
marks (" and \") were being removed from content, potentially corrupting
string formatting in rendered outputs. **In fact, extra quotes is
generated by json.dumps(v, ensure_ascii=False).**

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 19:44:03 +08:00
8eefc8b5fe Test: Added test cases for Add Chunk HTTP API (#6408)
### What problem does this PR solve?

cover [add
chunk](https://ragflow.io/docs/v0.17.2/http_api_reference#add-chunk)
endpoints

### Type of change

- [x] Add test cases
2025-03-21 19:16:30 +08:00
4091af4560 Fix: multiple top-level packages error in Python project (#6370)
### What problem does this PR solve?

This PR resolves the issue of multiple top-level packages being detected
in the Python project, which caused errors when using uv pip install.
The problem occurred because the project had multiple directories files
at the root level, leading to a flat-layout error.
To fix this, the pyproject.toml file was updated to explicitly list the
packages using the [tool.setuptools] section. This ensures that the
correct packages are included during installation, avoiding the
flat-layout error.
Type of change

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 18:44:49 +08:00
394d1a86f6 Fix: add chunk, empty question issue. (#6405)
### What problem does this PR solve?

#6404

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 18:44:12 +08:00
d88964f629 Feat: If the Transfer item is disabled, the item cannot be edited. #3221 (#6409)
### What problem does this PR solve?

Feat: If the Transfer item is disabled, the item cannot be edited. #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-21 18:42:52 +08:00
0e0ebaac5f Feat: Adds hierarchical title path tracking for tables in DOCX documents to improve context association (#6374)
### What problem does this PR solve?

Adds hierarchical title path tracking for tables in DOCX documents to
improve context association. Previously, extracted tables lacked
positional context within document structure.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-21 18:42:36 +08:00
8b7e53e643 Fix: miss calculate of token number. (#6401)
### What problem does this PR solve?

#6308

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 17:30:38 +08:00
979cdc3626 UI updates. (#6398)
### What problem does this PR solve?

Updated UI descriptions for delimiters and recommended chunk size

### Type of change

- [x] Documentation Update
2025-03-21 16:50:20 +08:00
a2a4bfe3e3 Fix: change ollama default num_ctx. (#6395)
### What problem does this PR solve?

#6163

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 16:22:03 +08:00
85480f6292 Fix: the error of Ollama embeddings interface returning "500 Internal Server Error" (#6350)
### What problem does this PR solve?

Fix the error where the Ollama embeddings interface returns a “500
Internal Server Error” when using models such as xiaobu-embedding-v2 for
embedding.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 15:25:48 +08:00
f537b6ca00 Fix: flow list translate to zh (#6371)
### What problem does this PR solve?

Add the Chinese translation of 'noMoreData' on the flow list page

### Type of change

- [x] Refactoring
2025-03-21 14:54:12 +08:00
b5471978b0 Fix: add chunk api, empty content issue (#6390)
### What problem does this PR solve?

#6387

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 14:05:59 +08:00
efdfb39a33 Feat: Add Duplicate ID Check and Update Deletion Logic (#6376)
- Introduce the `check_duplicate_ids` function in `dataset.py` and
`doc.py` to check for and handle duplicate IDs.
- Update the deletion operation to ensure that when deleting datasets
and documents, error messages regarding duplicate IDs can be returned.
- Implement the `check_duplicate_ids` function in `api_utils.py` to
return unique IDs and error messages for duplicate IDs.


### What problem does this PR solve?

Close https://github.com/infiniflow/ragflow/issues/6234

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-21 14:05:17 +08:00
7cc5603a82 Fix broken discord invitation links (#6388)
### Type of change

- [x] Documentation Update
2025-03-21 13:38:34 +08:00
9ed004e90d Refa: control the simi for entity resolution. (#6386)
### What problem does this PR solve?

#6352

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 13:16:34 +08:00
d83911b632 Fix: huggingface rerank model issue. (#6385)
### What problem does this PR solve?

#6348

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 12:43:32 +08:00