Commit Graph

157 Commits

Author SHA1 Message Date
2023fdc13e fix file preview in file management (#1151)
### What problem does this PR solve?

fix file preview in file management

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-14 10:33:59 +08:00
e05395d2a7 fix multi-modual bug (#1127)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-12 14:21:35 +08:00
abcd3d2469 refactor (#1124)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-06-12 11:02:15 +08:00
68a698655a infinity: Update embedding_model.py (#1109)
### What problem does this PR solve?

I implemented infinity, a fast vector embeddings engine. 

### Type of change


- [x] Performance Improvement
- [X] Other (please describe):
2024-06-11 08:23:58 +08:00
923c3b8cac fix bug in api (#1088)
### What problem does this PR solve?

#1075 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-07 09:56:14 +08:00
722c342d56 fix: bug similarity() in YoudaoRerank (#1084)
### What problem does this PR solve?

bix fix

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-07 09:04:53 +08:00
195498daaa feat: Support Password Access for ElasticSearch (#1072)
### What problem does this PR solve?

Using password authentication to access ElasticSearch is essential,
especially in a production environment.

This PR will enable password access support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-06 13:19:26 +08:00
4454ba7a1e add self-rag (#1070)
### What problem does this PR solve?

#1069 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-06 11:13:39 +08:00
b8eedbdd86 refine rerank (#1056)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-04 17:27:00 +08:00
cc064040a2 refine API request data processing (#1031)
### What problem does this PR solve?

#1024 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-06-03 09:02:25 +08:00
c58a1c48eb Fix: bug #991 (#1013)
### What problem does this PR solve?

issue #991

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
2024-05-31 18:03:47 +08:00
260c68f60c Adding the Minimax model (#1009)
### What problem does this PR solve?

Added support for MiniMax LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: cecilia-uu <konghui1996@163.com>
2024-05-31 16:38:53 +08:00
GYH
b85c15cc96 Add file rag/svr/discord_svr.py (#1008)
### What problem does this PR solve?

File rag/svr/discord_svr.py is for discord bot.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-31 13:47:15 +08:00
9ed0e50f6b Update info (#1005)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-31 09:53:04 +08:00
b9bb11879f fix #994 (#1006)
### What problem does this PR solve?

#994 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-31 09:46:22 +08:00
dc7afe46fb fix bug 994 ,991 (#1004)
### What problem does this PR solve?

#994 
#991 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-31 09:24:24 +08:00
117a173fff fix tk_count undefine issue (#996)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 16:18:15 +08:00
77363a0875 fix bge rerank normalize issue (#988)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 12:55:17 +08:00
f077b57f8b set ollama keep_alive (#985)
### What problem does this PR solve?

#980 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 11:27:58 +08:00
0171082cc5 fix create dialog bug (#982)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 09:25:05 +08:00
8dd45459be Add support for HTML file (#973)
### What problem does this PR solve?

Add support for HTML file

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-30 09:12:55 +08:00
dded365b8d Fix:After being idle for a while, new tasks need to be cancel and redo (#958)
### What problem does this PR solve?

After being idle for a while (When Redis Queue exceeds the
SVR_QUEUE_RETENTION(60*60) expiration time), new tasks need to be cancel
and redo.

When use xgroup_create to create a consumer group, set the ID to "$",
meaning that only messages added to the stream after the group is
created will be visible to new consumers. If the application scenario
requires processing messages that already exist in the queue, you might
need to change this ID to "0", so that the new consumer group can read
all messages from the beginning.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-30 09:03:11 +08:00
758eb03ccb fix jina adding issure and term weight refinement (#974)
### What problem does this PR solve?

#724 #162

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 19:38:57 +08:00
614defec21 add rerank model (#969)
### What problem does this PR solve?

feat: add rerank models to the project #724 #162

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 16:50:02 +08:00
7eee193956 fix #917 #915 (#946)
### What problem does this PR solve?

#917 
#915

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-28 11:13:02 +08:00
9ffd7ae321 Added support for Baichuan LLM (#934)
### What problem does this PR solve?

- Added support for Baichuan LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: 海贼宅 <stu_xyx@163.com>
2024-05-28 09:09:37 +08:00
46454362d7 fix raptor bugs (#928)
### What problem does this PR solve?

#922 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-27 11:01:20 +08:00
c0d71adaa2 Bug fix for volcengine (#909)
### What problem does this PR solve?
Bug fixes for the VolcEngine

- Bug fix for front-end configuration code of VolcEngine

- Bug fix for tokens counting logic of VolcEngine


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: 海贼宅 <stu_xyx@163.com>
2024-05-24 11:34:39 +08:00
6f99bbbb08 add raptor (#899)
### What problem does this PR solve?

#882 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 14:31:16 +08:00
eb51ad73d6 Add support for VolcEngine - the current version supports SDK2 (#885)
- The main idea is to assemble **ak**, **sk**, and **ep_id** into a
dictionary and store it in the database **api_key** field
- I don’t know much about the front-end, so I learned from Ollama, which
may be redundant.

### Configuration method

- model name

- Format requirements: {"VolcEngine model name":"endpoint_id"}
    - For example: {"Skylark-pro-32K":"ep-xxxxxxxxx"}
    
- Volcano ACCESS_KEY
- Format requirements: VOLC_ACCESSKEY of the volcano engine
corresponding to the model

- Volcano SECRET_KEY
- Format requirements: VOLC_SECRETKEY of the volcano engine
corresponding to the model
    
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-23 11:15:29 +08:00
21453ffff0 fixed: The choices may be empty. (#876)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-22 15:29:07 +08:00
GYH
be13429d05 Add api/list_kb_docs function and modify api/list_chunks (#874)
### What problem does this PR solve?
#717 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-22 14:58:56 +08:00
a12fcf9156 fix minio helth bug (#850)
### What problem does this PR solve?

#643 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-20 19:35:30 +08:00
GYH
c27c02ea67 Split Excel file into different chunks (#847)
### What problem does this PR solve?


Split Excel into different chunk
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-20 18:35:15 +08:00
a7bd427116 add locally deployed llm (#841)
### What problem does this PR solve?


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-20 12:40:59 +08:00
2b36283712 fix english query bug (#840)
### What problem does this PR solve?

#834 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-20 12:23:51 +08:00
f6a599461f fix zhipuAI stream issue (#825)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 17:07:33 +08:00
d1614107e2 fix stream chat for ollama (#816)
### What problem does this PR solve?

#709

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-17 12:07:00 +08:00
95f809187e add stream chat (#811)
### What problem does this PR solve?

#709 
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-16 20:14:53 +08:00
6ff63ee2ba Support for code files parse (#789)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 16:34:28 +08:00
4d47b2b459 fix a string format error (#781)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-15 13:02:31 +08:00
aa1c915d6e support gpt-4o (#773)
### What problem does this PR solve?
#771 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 11:16:08 +08:00
77b1520b66 Refactor message output format (#772)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-15 10:48:42 +08:00
ffe5737f7d let index be batchly. (#733)
### What problem does this PR solve?

let index be batchly.

### Type of change


- [x] Refactoring
2024-05-11 19:47:53 +08:00
04a9e95161 let file in knowledgebases visible in file manager (#714)
### What problem does this PR solve?

Let file in knowledgebases visible in file manager.
#162 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-11 16:04:28 +08:00
648a2baaa9 fix disabled doc is still retreivalable (#695)
### What problem does this PR solve?

Fix that disabled doc is still retreivalable

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-09 15:32:24 +08:00
4153a36683 truncate text to fitin embedding model (#692)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-05-09 11:35:08 +08:00
c28f7b5d38 make sure the error will be recorded. (#672)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-05-08 13:58:41 +08:00
eb27a4309e add support for deepseek (#668)
### What problem does this PR solve?

#666 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-08 10:30:02 +08:00
a6e4b74d94 remove unused dependency (#664)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 19:46:17 +08:00