Commit Graph

509 Commits

Author SHA1 Message Date
3ee9ca749d Fix the bug that prevented modifying dataset_ids (#3784)
### What problem does this PR solve?

Fix the bug that prevented modifying `dataset_ids`.
 #3772

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-12-02 11:40:20 +08:00
7058ac0041 Fix out of boundary. (#3786)
### What problem does this PR solve?

#3769
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-02 11:38:53 +08:00
e079656473 Update progress info and start welcome info (#3768)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-30 18:48:06 +08:00
d00297a763 Fix chunk creation using Infinity (#3763)
### What problem does this PR solve?

1. Store error type in Infinity
2. position list value read from Infinity isn't correct.

Fix issue: #3729

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-30 00:10:14 +08:00
381219aa41 Fixed increase_usage for builtin models (#3748)
### What problem does this PR solve?

Fixed increase_usage for builtin models. Close #1803

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-29 17:02:49 +08:00
b89f7c69ad Fix image_id absence issue (#3735)
### What problem does this PR solve?

#3731

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-29 10:37:09 +08:00
fcdc6ad085 Fix the issue where the agent interface cannot call the context (#3725)
### What problem does this PR solve?

Fix the context of the agent interface call to the context during web
testing, and change it to the context record of user chat
Remove duplicate messages and add them, which can be viewed in the
messages section of database api_4_comversation

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-11-29 10:36:48 +08:00
112ef42a19 Ensure thumbnail be smaller than 64K (#3722)
### What problem does this PR solve?

Ensure thumbnail be smaller than 64K. Close #1443
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-28 19:15:31 +08:00
91f1814a87 Fix error response (#3719)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2024-11-28 18:56:10 +08:00
9fcad0500d Add more web test cases (#3702)
### What problem does this PR solve?

Test cases about dataset

### Type of change

- [x] Other (please describe): test cases

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-28 15:46:35 +08:00
ec560cc99d Feat: Scrolling knowledge base list and set the number of entries per page to 30 #3695 (#3712)
### What problem does this PR solve?

Feat: Scrolling knowledge base list and set the number of entries per
page to 30 #3695

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-28 15:25:38 +08:00
bc701d7b4c Edit chunk shall update instead of insert it (#3709)
### What problem does this PR solve?

Edit chunk shall update instead of insert it. Close #3679 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-28 13:00:38 +08:00
9f57534843 Revert "Feat: Scrolling knowledge base list #3695" (#3708)
Reverts infiniflow/ragflow#3703
2024-11-28 11:44:23 +08:00
52b3492b18 Feat: Scrolling knowledge base list #3695 (#3703)
### What problem does this PR solve?

Feat: Scrolling knowledge base list #3695

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-11-28 10:51:30 +08:00
2249d5d413 Always open text file for write with UTF-8 (#3688)
### What problem does this PR solve?

Always open text file for write with UTF-8. Close #932 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-27 16:24:16 +08:00
b6f3f15f0b Fix KB list bugs and add web api test (#3649)
### What problem does this PR solve?

1. Read KB list path parameter, page_number and page_size, which type
isn't int
2. Add cases on create / list / delete datasets.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Test cases

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-26 18:21:15 +08:00
5337cad7e4 Check model id when set dialog. Close #849 (#3655)
### What problem does this PR solve?

Check model id when set dialog. Close #849

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-26 16:32:46 +08:00
5c59651bda Fix the bug causing garbled text (#3640)
### What problem does this PR solve?

Fix the bug causing garbled text #3613

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-26 12:06:56 +08:00
d8b150a34c Add support for folder deletion (#3635)
### What problem does this PR solve?

Add support for folder deletion.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-25 16:59:43 +08:00
ce6b4c0e05 Fix a bug in completions (#3632)
### What problem does this PR solve?

Fix a bug in completions

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-25 16:21:28 +08:00
86e48179a1 Fix bugs in api (#3624)
### What problem does this PR solve?

#3488

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-25 15:25:48 +08:00
08ead81dde Bump infinity to v0.5.0-dev5 (#3520)
### What problem does this PR solve?

Bump infinity to v0.5.0-dev5

### Type of change

- [x] Refactoring
2024-11-25 11:53:58 +08:00
646ac1f2b4 Improved image build instructions (#3580)
### What problem does this PR solve?

Improved arm64 image build instructions

### Type of change

- [x] Documentation Update
- [x] Refactoring
2024-11-22 20:24:32 +08:00
cc219ff648 Fix agent session API (#3589)
### What problem does this PR solve?

#3585
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-22 16:19:00 +08:00
d56f52eef8 Fix agent api invokation issue (#3581)
### What problem does this PR solve?

#3570 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-22 12:37:27 +08:00
9f3141804f Fix chunk enable/disable issue (#3579)
### What problem does this PR solve?

#3576

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-22 12:25:42 +08:00
ee50f78d99 Add component 'Template' (#3562)
### What problem does this PR solve?

#3560

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-21 18:26:22 +08:00
30f111edb3 Fixs for translation agent (#3557)
### What problem does this PR solve?

#3556 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-11-21 16:22:25 +08:00
c7c8b3812f Add test for document (#3548)
### What problem does this PR solve?

Add test for document

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-21 12:02:26 +08:00
d9c882399d Ensure LIGHTEN work (#3542)
### What problem does this PR solve?
#3531

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-21 09:56:28 +08:00
8930bfcff8 Fix bugs (#3535)
### What problem does this PR solve?

1. system monitor icon and text missing
2. Team knowledge base can't be search

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-20 20:55:42 +08:00
362db857d0 New: a new interpretor based on Andrew Ng theory. (#3532)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-20 20:55:22 +08:00
541272eb99 Fix: authorization issue (#3530)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-20 20:55:10 +08:00
5b44b99cfd Removed beartype (#3528)
### What problem does this PR solve?

The beartype configuration of
main(64f50992e0) is:
```
from beartype import BeartypeConf
from beartype.claw import beartype_all  # <-- you didn't sign up for this
beartype_all(conf=BeartypeConf(violation_type=UserWarning))    # <-- emit warnings from all code
```

ragflow_server failed at a third-party package:

```
(ragflow-py3.10) zhichyu@iris:~/github.com/infiniflow/ragflow$ rm -rf logs/* && bash docker/launch_backend_service.sh 
Starting task_executor.py for task 0 (Attempt 1)
Starting ragflow_server.py (Attempt 1)
Traceback (most recent call last):
  File "/home/zhichyu/github.com/infiniflow/ragflow/api/ragflow_server.py", line 22, in <module>
    from api.utils.log_utils import initRootLogger
  File "/home/zhichyu/github.com/infiniflow/ragflow/api/utils/__init__.py", line 25, in <module>
    import requests
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/__init__.py", line 15, in <module>
    from ._base_connection import _TYPE_BODY
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/_base_connection.py", line 5, in <module>
    from .util.connection import _TYPE_SOCKET_OPTIONS
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/util/__init__.py", line 4, in <module>
    from .connection import is_connection_dropped
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 7, in <module>
    from .timeout import _DEFAULT_TIMEOUT, _TYPE_TIMEOUT
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/util/timeout.py", line 20, in <module>
    _DEFAULT_TIMEOUT: Final[_TYPE_DEFAULT] = _TYPE_DEFAULT.token
NameError: name 'Final' is not defined
Traceback (most recent call last):
  File "/home/zhichyu/github.com/infiniflow/ragflow/rag/svr/task_executor.py", line 22, in <module>
    from api.utils.log_utils import initRootLogger
  File "/home/zhichyu/github.com/infiniflow/ragflow/api/utils/__init__.py", line 25, in <module>
    import requests
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/__init__.py", line 15, in <module>
    from ._base_connection import _TYPE_BODY
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/_base_connection.py", line 5, in <module>
    from .util.connection import _TYPE_SOCKET_OPTIONS
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/util/__init__.py", line 4, in <module>
    from .connection import is_connection_dropped
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/util/connection.py", line 7, in <module>
    from .timeout import _DEFAULT_TIMEOUT, _TYPE_TIMEOUT
  File "/home/zhichyu/github.com/infiniflow/ragflow/.venv/lib/python3.10/site-packages/urllib3/util/timeout.py", line 20, in <module>
    _DEFAULT_TIMEOUT: Final[_TYPE_DEFAULT] = _TYPE_DEFAULT.token
NameError: name 'Final' is not defined
```

This third-package is out of our control. I have to remove beartype
entirely.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-20 20:54:57 +08:00
6be7901df2 Warning instead of exception on type mismatch (#3523)
### What problem does this PR solve?

Warning instead of exception on type mismatch.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-20 20:54:42 +08:00
d02a2b131a Fix: potential risk (#3515)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-11-20 20:53:58 +08:00
e559cebcdc fix: keyerror issue (#3512)
### What problem does this PR solve?

#3511

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-20 20:53:20 +08:00
361cff34fc add input variables to begin component (#3498)
### What problem does this PR solve?

#3355 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-19 18:41:48 +08:00
d42362deb6 Add api for sessions and add max_tokens for tenant_llm (#3472)
### What problem does this PR solve?

Add api for sessions and add max_tokens for tenant_llm

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-19 14:51:33 +08:00
dec9b3e540 Fix logs. Use dict.pop instead of del. Close #3473 (#3484)
### What problem does this PR solve?

Fix logs. Use dict.pop instead of del.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-19 14:15:25 +08:00
4413683898 Introduced beartype (#3460)
### What problem does this PR solve?

Introduced [beartype](https://github.com/beartype/beartype) for runtime
type-checking.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 17:38:17 +08:00
4b3eeaa6ef Added LocalAI support for rerank models (#3446)
### What problem does this PR solve?

Hi there!
LocalAI added support of rerank models
https://localai.io/features/reranker/

I've implemented LocalAIRerank class (typically copied it from
OpenAI_APIRerank class).
Also, LocalAI model response with 500 error code if len of "documents"
is less than 2 in similarity check.
So I've added the second "document" on RERANK model connection check in
`api/apps/llm_app.py`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-18 12:05:52 +08:00
70cd5c1599 Remove unused code (#3448)
### What problem does this PR solve?

1. Remove unused code.
2. Move some codes from settings to constants

### Type of change

- [x] Refactoring

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-18 12:05:38 +08:00
f9643adc43 Adjusted heartbeat format (#3459)
### What problem does this PR solve?

Adjusted heartbeat format

### Type of change

- [x] Refactoring
2024-11-18 12:03:28 +08:00
a1d01a1b2f enlarge the default token length of RAPTOR summarization (#3454)
### What problem does this PR solve?

#3426

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 10:15:26 +08:00
4ed5ca2666 handle_task catch all exception (#3441)
### What problem does this PR solve?

handle_task catch all exception
Report heartbeats

### Type of change

- [x] Refactoring
2024-11-15 18:51:09 +08:00
1e90a1bf36 Move settings initialization after module init phase (#3438)
### What problem does this PR solve?

1. Module init won't connect database any more.
2. Config in settings need to be used with settings.CONFIG_NAME

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 17:30:56 +08:00
cb3b9d7ada refine the message of queuing a task (#3437)
### What problem does this PR solve?



### Type of change

- [x] Refactoring
2024-11-15 15:59:54 +08:00
ca9e97d2f2 Enlarge the term weight difference (#3435)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-11-15 15:41:50 +08:00
a854bc22d1 Rework task executor heartbeat (#3430)
### What problem does this PR solve?

Rework task executor heartbeat, and print in console.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-15 14:43:55 +08:00