Commit Graph

503 Commits

Author SHA1 Message Date
8dd45459be Add support for HTML file (#973)
### What problem does this PR solve?

Add support for HTML file

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-30 09:12:55 +08:00
dded365b8d Fix:After being idle for a while, new tasks need to be cancel and redo (#958)
### What problem does this PR solve?

After being idle for a while (When Redis Queue exceeds the
SVR_QUEUE_RETENTION(60*60) expiration time), new tasks need to be cancel
and redo.

When use xgroup_create to create a consumer group, set the ID to "$",
meaning that only messages added to the stream after the group is
created will be visible to new consumers. If the application scenario
requires processing messages that already exist in the queue, you might
need to change this ID to "0", so that the new consumer group can read
all messages from the beginning.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 09:03:11 +08:00
9fdd517af6 Update README.md (#978)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-29 20:22:41 +08:00
2604ded2e4 Update README.md (#976)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-29 20:02:16 +08:00
758eb03ccb fix jina adding issure and term weight refinement (#974)
### What problem does this PR solve?

#724 #162

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 19:38:57 +08:00
e0d05a3895 fix: if the conversation name is too long, it will overflow the current item. #607 (#972)
### What problem does this PR solve?

fix: if the conversation name is too long, it will overflow the current
item. #607

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-29 18:32:03 +08:00
614defec21 add rerank model (#969)
### What problem does this PR solve?

feat: add rerank models to the project #724 #162

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 16:50:02 +08:00
e1f0644deb feat: add jina (#967)
### What problem does this PR solve?
feat: add jina #650 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 16:48:52 +08:00
a135f9f5b6 feat: add rerank models to the project #724 #162 (#966)
### What problem does this PR solve?

Vector similarity weight is displayed incorrectly #965
feat: add rerank models to the project #724 #162
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-29 16:19:08 +08:00
daa4799385 limit the system context length of conversation messages. (#962)
### What problem does this PR solve?

#951 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-29 10:40:07 +08:00
495a6434ec feat: add FlowHeader and delete edge (#959)
### What problem does this PR solve?
feat: add FlowHeader and delete edge #918 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 10:01:39 +08:00
21aac545d9 Expanded the supported LLM list (#960)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-28 20:13:03 +08:00
0f317221b4 Update README (#956)
### What problem does this PR solve?

Update README due to support new LLMs.

### Type of change

- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-28 20:05:02 +08:00
a427672229 Fixed a docusaurus display issue (#954)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-05-28 17:26:13 +08:00
196f2b445f fix: fixed the issue of 404 error in the user settings page of the demo site (#948)
### What problem does this PR solve?

fix: fixed the issue of 404 error in the user settings page of the demo
site #947

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-28 11:22:56 +08:00
5041677f11 Add umap-learn, fasttext and volcengine in requirements_arm.txt (#945)
### What problem does this PR solve?

Complete the requirements for ARM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-28 11:13:48 +08:00
7eee193956 fix #917 #915 (#946)
### What problem does this PR solve?

#917 
#915

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-28 11:13:02 +08:00
9ffd7ae321 Added support for Baichuan LLM (#934)
### What problem does this PR solve?

- Added support for Baichuan LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: 海贼宅 <stu_xyx@163.com>
2024-05-28 09:09:37 +08:00
ec6ae744a1 minor editorial updates for clarity (#941)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-27 20:35:08 +08:00
d9bc093df1 feat: test buildNodesAndEdgesFromDSLComponents (#940)
### What problem does this PR solve?
 feat: test buildNodesAndEdgesFromDSLComponents #918

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-27 19:35:14 +08:00
571aaaff22 Add Dockerfile and requirements.txt for arm (#936)
### What problem does this PR solve?

#253 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-27 19:34:49 +08:00
GYH
7d8e03ec38 Update docnm_kwd to doc_name (#939)
### What problem does this PR solve?

Update docnm_kwd to doc_name 
#908 

### Type of change


- [x] Refactoring
2024-05-27 19:14:04 +08:00
65677f65c9 Updated RESTful API Reference (#908)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-27 18:34:16 +08:00
89d296feab Remove duplicated FROM. (#935)
### What problem does this PR solve?
Remove duplicated FROM in Dockerfile.cuda.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 17:16:47 +08:00
3ae8a87986 Expanded list of locally deployed embedding models (#930)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-27 14:01:52 +08:00
46454362d7 fix raptor bugs (#928)
### What problem does this PR solve?

#922 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 11:01:20 +08:00
55fb96131e feat: build react flow nodes and edges from mock data #918 (#919)
### What problem does this PR solve?
feat: build react flow nodes and edges from mock data #918

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-27 08:21:30 +08:00
20b57144b0 syntax error (#924)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 08:20:32 +08:00
9e3a0e4d03 The fasttext library is missing, and it is used in the operators.py file. (#925)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 08:18:47 +08:00
c0d71adaa2 Bug fix for volcengine (#909)
### What problem does this PR solve?
Bug fixes for the VolcEngine

- Bug fix for front-end configuration code of VolcEngine

- Bug fix for tokens counting logic of VolcEngine


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: 海贼宅 <stu_xyx@163.com>
2024-05-24 11:34:39 +08:00
735bdf06a4 Update README (#901)
### What problem does this PR solve?

Update README due to implement RAPTOR.

### Type of change

- [x] Documentation Update

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-24 08:30:08 +08:00
fe18627ebc Fix some syntax errors, re not import (#904)
re not import

### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-23 19:38:13 +08:00
4cda40c3ef feat: fixed issue with threshold translation #882 and add NodeContextMenu (#906)
### What problem does this PR solve?

feat: fixed issue with threshold translation #882
feat: add NodeContextMenu

### Type of change


- [ ] New Feature (non-breaking change which adds functionality)
2024-05-23 18:53:04 +08:00
GYH
1e5c5abe58 Update api_md document/rm (#894)
### What problem does this PR solve?

Update api_md document/rm
#717 

### Type of change

- [x] Documentation Update
2024-05-23 15:19:58 +08:00
6f99bbbb08 add raptor (#899)
### What problem does this PR solve?

#882 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 14:31:16 +08:00
3bbdf3b770 fixbug for computing 'not concating feature' (#896)
### What problem does this PR solve?

When pdfparser call `_naive_vertical_merge` method,there is a "not
concating feature " value by computing difference between `b` and `b_`'s
layoutno ,but actually is `b` and `b`. I think it's a bug, so fix it.
Please check again.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-23 14:29:42 +08:00
070b53f3bf feat: RAPTOR is not displayed when the parsing method is picture. (#897)
### What problem does this PR solve?

Implements RAPTOR for better chunking #882

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 14:13:09 +08:00
eb51ad73d6 Add support for VolcEngine - the current version supports SDK2 (#885)
- The main idea is to assemble **ak**, **sk**, and **ep_id** into a
dictionary and store it in the database **api_key** field
- I don’t know much about the front-end, so I learned from Ollama, which
may be redundant.

### Configuration method

- model name

- Format requirements: {"VolcEngine model name":"endpoint_id"}
    - For example: {"Skylark-pro-32K":"ep-xxxxxxxxx"}
    
- Volcano ACCESS_KEY
- Format requirements: VOLC_ACCESSKEY of the volcano engine
corresponding to the model

- Volcano SECRET_KEY
- Format requirements: VOLC_SECRETKEY of the volcano engine
corresponding to the model
    
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 11:15:29 +08:00
GYH
fbd0d74053 Add /api/document/rm function (#887)
### What problem does this PR solve?

Delete files from a knowledge base.

#717 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 10:32:56 +08:00
170186ee4d feat: remove the space before promptText (#886)
### What problem does this PR solve?

feat: remove the space before promptText #882 


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-22 18:36:55 +08:00
ed184ed87e Implements RAPTOR for better chunking #882 (#883)
### What problem does this PR solve?

Implements RAPTOR for better chunking #882

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-22 18:04:18 +08:00
GYH
43412571f7 Add api.md:/api/list_kb_docs/ description (#881)
### What problem does this PR solve?

Add api.md:/api/list_kb_docs/ description
#717 

### Type of change

- [x] Documentation Update
2024-05-22 17:37:11 +08:00
17489e6c6c fix import error (#877)
Fix import error for user_app.py

---------

Co-authored-by: yonghui li <yonghui.li@bondex.com.cn>
2024-05-22 16:14:53 +08:00
21453ffff0 fixed: The choices may be empty. (#876)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-22 15:29:07 +08:00
GYH
be13429d05 Add api/list_kb_docs function and modify api/list_chunks (#874)
### What problem does this PR solve?
#717 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-22 14:58:56 +08:00
5178daeeaf Fixed a format issue (#872)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-22 13:39:38 +08:00
d5b8d8e647 fixed a format issue for docusaurus publication (#871)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-22 12:45:34 +08:00
b62a20816e fix: display specific error message when previewing file error #868 (#869)
### What problem does this PR solve?

fix: display specific error message when previewing file error  #868


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-22 11:54:32 +08:00
3cae87a902 Reorganized docs for docusaurus publish (#860)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
2024-05-21 20:53:55 +08:00
1797f5ce31 fix: the site domain name in the Chat Bot API is hardcoded. #776 (#859)
### What problem does this PR solve?

fix: the site domain name in the Chat Bot API is hardcoded. #776

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-21 17:37:19 +08:00