Commit Graph

2455 Commits

Author SHA1 Message Date
6ec6ca6971 Refactor graphrag to remove redis lock (#5828)
### What problem does this PR solve?

Refactor graphrag to remove redis lock

### Type of change

- [x] Refactoring
2025-03-10 15:15:06 +08:00
1163e9e409 Feat: When selecting a reordering model, give a prompt that it takes too long. #5834 (#5835)
### What problem does this PR solve?

Feat: When selecting a reordering model, give a prompt that it takes too
long. #5834

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-10 14:14:38 +08:00
15736c57c3 Fix: empty query issue. (#5830)
### What problem does this PR solve?

#5214

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-10 13:56:56 +08:00
fa817a8ab3 Refa: SiliconFlow model list refresh. (#5825)
### What problem does this PR solve?

#5806

### Type of change


- [x] Refactoring
2025-03-10 12:51:12 +08:00
8b99635eb3 Feat: Add TransferList component. #3221 (#5822)
### What problem does this PR solve?

Feat: Add TransferList component. #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-10 11:22:06 +08:00
1919780880 Refa: reduce default value of MAX_CONCURRENT_CHATS (#5821)
### What problem does this PR solve?

#5786

### Type of change

- [x] Refactoring
2025-03-10 11:22:06 +08:00
82f5d901c8 Refa: add model. (#5820)
### What problem does this PR solve?

#5783

### Type of change

- [x] Refactoring
2025-03-10 11:22:06 +08:00
dek
dc4d4342cd Fix: broken /api/v1/chats endpoint (#5785)
### What problem does this PR solve?

The `/api/v1/chats` API endpoint was broken, any GET request got the
following response:
```
{"code":100,"data":null,"message":"TypeError(\"'int' object is not callable\")"}
```

With this log ragflow-server side:

```
2025-03-07 14:36:26,297 ERROR    20 'int' object is not callable
Traceback (most recent call last):
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/ragflow/api/utils/api_utils.py", line 303, in decorated_function
    return func(*args, **kwargs)
  File "/ragflow/api/apps/sdk/chat.py", line 323, in list_chat
    logging.WARN(f"Don't exist the kb {kb_id}")
TypeError: 'int' object is not callable
2025-03-07 14:36:26,298 INFO     20 172.18.0.6 - - [07/Mar/2025 14:36:26] "GET /api/v1/chats HTTP/1.1" 200 -
``` 
This was caused by the incorrect use of `logging.WARN` as a method (it's
a loglevel object), instead of the correct `logging.warning()` method.

This PR fixes that, and also rewrites the message to be grammaticaly
correct.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-10 11:22:06 +08:00
e05658685c Refa: update mistral model list. (#5818)
### What problem does this PR solve?

#5782

### Type of change

- [x] Refactoring
2025-03-10 11:22:06 +08:00
b29539b442 Fix: CoHereRerank not respecting base_url when provided (#5784)
### What problem does this PR solve?

vLLM provider with a reranking model does not work : as vLLM uses under
the hood the [CoHereRerank
provider](https://github.com/infiniflow/ragflow/blob/v0.17.0/rag/llm/__init__.py#L250)
with a `base_url`, if this URL [is not passed to the Cohere
client](https://github.com/infiniflow/ragflow/blob/v0.17.0/rag/llm/rerank_model.py#L379-L382)
any attempt will endup on the Cohere SaaS (sending your private api key
in the process) instead of your vLLM instance.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-03-10 11:22:06 +08:00
b1a46d5adc Fix:when start with source code not in docker env report 'UnicodeDec… (#5802)
### What problem does this PR solve?

fix:when start with  source code not in docker env report
"UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 5:
illegal multibyte sequence" in windows

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: tangyu <1@1.com>
2025-03-10 11:22:06 +08:00
50c510d16b Fix: bugs mentioned by#5760 (#5778)
### What problem does this PR solve?

Fixed the issue of "stop deleting when encountering invalid dataset ID"

#5760

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-03-10 11:22:06 +08:00
8a84d1048c TEST: Added test cases for Delete Dataset HTTP API (#5770)
### What problem does this PR solve?

1. cover dataset deletion endpoints
2. format code with ruff

### Type of change

- [x] add testcases
- [ ] style
2025-03-07 17:44:51 +08:00
2ad852d8df Fix: truncate message issue. (#5776)
### What problem does this PR solve?

Close #5761
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 17:41:56 +08:00
ca39f5204d Initial draft of Implemnt deep research (#5774)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2025-03-07 17:06:49 +08:00
5b0e38060a Feat:Optimize the table extraction logic in the Markdown parser: (#5663)
Enhance the recognition of both borderless and bordered Markdown tables.
Add support for extracting HTML tables, including various scenarios with
nested HTML tags. Improve performance by using conditional checks to
reduce unnecessary regular expression matching.

### What problem does this PR solve?

Optimize the table extraction logic in the Markdown parser:
Enhance the recognition of both borderless and bordered Markdown tables.
Add support for extracting HTML tables, including various scenarios with
nested HTML tags.
Improve performance by using conditional checks to reduce unnecessary
regular expression matching.

### Type of change

- [x] Performance Improvement

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
2025-03-07 17:02:35 +08:00
66938e0b68 Feat(api): Add dsl parameters to control whether dsl fields are included (#5769)
1. **Issue**: When calling `list_agent_session` via the HTTP API, users
may only need to display conversation messages, and do not want to see
the associated dsl, which can be very large. Therefore, consider adding
a control option to determine whether the DSL should be returned, with
the default being to return it.

2. **Documentation Discrepancy**: In the HTTP API documentation, under
"List agent sessions," the "Response" section states that the "data"
field is a dictionary when "success" is returned. However, the actual
returned data is a list. This discrepancy has been corrected.
2025-03-07 16:58:00 +08:00
64c6cc4cf3 Fix: truncate message issue. (#5765)
### What problem does this PR solve?

Close #5761

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 16:33:25 +08:00
3418984848 Fix: meta fields updata issue, (#5764)
### What problem does this PR solve?

#4789

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 16:21:27 +08:00
3c79990934 Fix: Fixed the issue that files cannot be uploaded on the file management page. #5730 (#5763)
### What problem does this PR solve?

Fix: Fixed the issue that files cannot be uploaded on the file
management page. #5730

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 16:09:20 +08:00
da3f279495 Fix: add the validation for parser_config. (#5755)
### What problem does this PR solve?

#5719

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 15:34:34 +08:00
b1bbb9e210 Refa: make Rewrite component effective to relative data expression. (#5752)
### What problem does this PR solve?

#5716

### Type of change

- [x] Refactoring
2025-03-07 13:48:13 +08:00
0e3e129a83 Fix: Resolve inconsistency in APIToken dialog_id field definition (#5749)
The `dialog_id` field was inconsistently defined:
- In the `migrate_db()` function, it was set to `null=True`.
- In the model class, it was defined as `null=False`.

This inconsistency caused an issue during the initial deployment where
the database table did not allow `dialog_id` to be null. As a result,
calling `APITokenService.save(**obj)` in `system_app.py` raised the
following error:

```
peewee.IntegrityError: null value in column "dialog_id" violates not-null constraint
```

### What problem does this PR solve?

Error: peewee.IntegrityError: null value in column "dialog_id" violates
not-null constraint

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 13:26:08 +08:00
c87b58511e Fix: API empty field input. (#5748)
### What problem does this PR solve?

#5709

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 13:11:07 +08:00
8d61dcc8ab Fix: can not upload file close #5730 (#5742)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

close #5730 

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-03-07 11:24:23 +08:00
06b29d7da4 Fix: empty description (#5747)
### What problem does this PR solve?

#5705

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 11:23:37 +08:00
5229a76f68 Fix: Remove the document language parameter. #5640 (#5728)
### What problem does this PR solve?

Fix: Remove the document language parameter. #5686

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-07 11:03:58 +08:00
4f9504305a TEST: Added test cases for Create Dataset HTTP API (#5724)
### What problem does this PR solve?

1. add test cases
2. integrate workflows/tests.yml into CI pipeline

### Type of change

- [x] add testcases
2025-03-06 20:22:17 +08:00
27153dde85 Updated instructions in the UI (#5733)
### What problem does this PR solve?



### Type of change


- [x] Documentation Update
2025-03-06 19:56:15 +08:00
9fc7174612 Fix: too long context during KG issue. (#5723)
### What problem does this PR solve?

#5088

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-06 19:21:07 +08:00
8fb8374dfc Fix: delimiter issue. (#5720)
### What problem does this PR solve?

#5704

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-06 17:51:22 +08:00
ff35c140dc Refa: remove dataset language and validate dataset name length. (#5707)
### What problem does this PR solve?

#5686
#5702

### Type of change

- [x] Refactoring
2025-03-06 17:08:28 +08:00
df9b7b2fe9 Fix: rerank issue. (#5696)
### What problem does this PR solve?

#5673

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-06 15:05:19 +08:00
48f3f49e80 Fix: docs inconsistency. (#5695)
### What problem does this PR solve?

#5662

### Type of change

- [x] Documentation Update
2025-03-06 11:48:31 +08:00
94d7af00b8 Fix: Remove the max token parameter. #5640 #5646 (#5693)
### What problem does this PR solve?

Fix: Remove the max token parameter. #5640 #5646

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-06 11:44:27 +08:00
251ba7f058 Refa: remove max tokens since no one needs it. (#5690)
### What problem does this PR solve?

#5646 #5640

### Type of change

- [x] Refactoring
2025-03-06 11:29:40 +08:00
28296955f1 Minor: improve tips display (#5631)
### What problem does this PR solve?

1. Add the missing translations.  
![CleanShot 2025-03-05 at 10 29
32](https://github.com/user-attachments/assets/85e95372-07d9-47a1-82cf-6eb4d0e1c831)

2. Shorten overly long tips.  
![CleanShot 2025-03-05 at 10 34
49](https://github.com/user-attachments/assets/fae8ce4c-6495-4abf-958d-2febeb38b893)

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [x] Other (please describe):
2025-03-06 11:03:49 +08:00
1b2fc3cc9a Feat: Add rerank option to huggingface's model type drop-down box. #5658 (#5689)
### What problem does this PR solve?

Feat: Add rerank option to huggingface's model type drop-down box. #5658

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-06 11:03:08 +08:00
b8da2eeb69 Feat: support huggingface re-rank model. (#5684)
### What problem does this PR solve?

#5658

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-06 10:44:04 +08:00
5f62f0c9d7 Miscellaneous updates (#5670)
### What problem does this PR solve?

#5625 #5614 

### Type of change


- [x] Documentation Update
2025-03-06 09:55:27 +08:00
a54843cc65 Feat: Use react-hook-form to synchronize the data of the categorize form to the agent node. #3221 (#5665)
### What problem does this PR solve?

Feat: Use react-hook-form to synchronize the data of the categorize form
to the agent node. #3221

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-05 19:43:08 +08:00
4326873af6 refactor: no need to inherit in python3 clean the code (#5659)
### What problem does this PR solve?

As title

### Type of change


- [x] Refactoring

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-03-05 18:03:53 +08:00
a64f4539e7 Docs: updates. (#5661)
### What problem does this PR solve?



### Type of change

- [x] Documentation Update
2025-03-05 17:54:34 +08:00
ec68ab1c8c Fix: search citation issue. (#5657)
### What problem does this PR solve?
#5649
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 17:25:47 +08:00
e5041749a2 Fix: tavily search error. (#5653)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 17:03:05 +08:00
78b2e0be89 fix: issue #5600 (#5645)
fix: issue https://github.com/infiniflow/ragflow/issues/5600

### What problem does this PR solve?

close issue https://github.com/infiniflow/ragflow/issues/5600 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 16:50:37 +08:00
b6aded378d Feat: The parsing method is paper and needs to display Document parser. #5467 (#5652)
### What problem does this PR solve?

Feat: The parsing method is paper and needs to display Document parser.
#5467

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-03-05 16:25:34 +08:00
11e3f5e8b2 Feat: Upload file UI/UX enhancements (#5359)
### What problem does this PR solve?

Modifies the UX for uploading process on the website.

- Adds option to parse on creation the files
- Adds progress bar to display progress of chunk
- Adds per file feedback on uploading operation

#### Screenshots:

- Show files uploading:

![image](https://github.com/user-attachments/assets/a5693f42-8232-4d5c-a240-20ed343634a5)

- Errors on specific files

![image](https://github.com/user-attachments/assets/986a7f54-ab32-4634-89ab-a098fe1954aa)


### Type of change

- [X] New Feature (non-breaking change which adds functionality)
2025-03-05 15:20:32 +08:00
f65c3ae62b Refactored DocumentService.update_progress (#5642)
### What problem does this PR solve?

Refactored DocumentService.update_progress

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 14:48:03 +08:00
02c955babb Fix: parameter error. (#5641)
### What problem does this PR solve?

#5600

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 14:37:51 +08:00