Commit Graph

474 Commits

Author SHA1 Message Date
dec9b3e540 Fix logs. Use dict.pop instead of del. Close #3473 (#3484)
### What problem does this PR solve?

Fix logs. Use dict.pop instead of del.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-19 14:15:25 +08:00
4b3eeaa6ef Added LocalAI support for rerank models (#3446)
### What problem does this PR solve?

Hi there!
LocalAI added support of rerank models
https://localai.io/features/reranker/

I've implemented LocalAIRerank class (typically copied it from
OpenAI_APIRerank class).
Also, LocalAI model response with 500 error code if len of "documents"
is less than 2 in similarity check.
So I've added the second "document" on RERANK model connection check in
`api/apps/llm_app.py`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-18 12:05:52 +08:00
70cd5c1599 Remove unused code (#3448)
### What problem does this PR solve?

1. Remove unused code.
2. Move some codes from settings to constants

### Type of change

- [x] Refactoring

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-18 12:05:38 +08:00
f9643adc43 Adjusted heartbeat format (#3459)
### What problem does this PR solve?

Adjusted heartbeat format

### Type of change

- [x] Refactoring
2024-11-18 12:03:28 +08:00
a1d01a1b2f enlarge the default token length of RAPTOR summarization (#3454)
### What problem does this PR solve?

#3426

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-18 10:15:26 +08:00
4ed5ca2666 handle_task catch all exception (#3441)
### What problem does this PR solve?

handle_task catch all exception
Report heartbeats

### Type of change

- [x] Refactoring
2024-11-15 18:51:09 +08:00
1e90a1bf36 Move settings initialization after module init phase (#3438)
### What problem does this PR solve?

1. Module init won't connect database any more.
2. Config in settings need to be used with settings.CONFIG_NAME

### Type of change

- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-15 17:30:56 +08:00
ca9e97d2f2 Enlarge the term weight difference (#3435)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2024-11-15 15:41:50 +08:00
e4c4fdabbd Update version display on web UI (#3405)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-14 17:51:21 +08:00
30f6421760 Use consistent log file names, introduced initLogger (#3403)
### What problem does this PR solve?

Use consistent log file names, introduced initLogger

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-11-14 17:13:48 +08:00
95d21e5d9f fix: standardize property name from 'chat' to 'chat_id' (#3383)
### What problem does this PR solve?

This PR addresses the inconsistency in property naming within the
codebase by renaming the 'chat' property to 'chat_id' in the session.py
file. This change aims to align the naming convention with other parts
of the application that refer to chat identifiers as 'chat_id', thereby
improving code clarity and maintainability.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-14 15:49:03 +08:00
4caf932808 fix bug about fetching knowledge graph (#3394)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-14 12:29:15 +08:00
83c6b1f308 set DLA active for KG (#3386)
### What problem does this PR solve?

### Type of change


- [x] Refactoring
2024-11-13 16:59:19 +08:00
7ab9715b0e refine the error message (#3382)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-11-13 16:20:59 +08:00
ccf189cb7f mv service_conf.yaml to conf/ and fix: add 'answer' as a parameter to 'generate' (#3379)
### What problem does this PR solve?
#3373

### Type of change

- [x] Refactoring
- [x] Bug fix
2024-11-13 15:56:40 +08:00
91332fa0f8 Refine english synonym (#3371)
### What problem does this PR solve?

#3361

### Type of change

- [x] Performance Improvement
2024-11-13 12:58:37 +08:00
a2a5631da4 Rework logging (#3358)
Unified all log files into one.

### What problem does this PR solve?

Unified all log files into one.

### Type of change

- [x] Refactoring
2024-11-12 17:35:13 +08:00
567a7563e7 Fix bugs in api and add examples (#3353)
### What problem does this PR solve?

Fix bugs in api.
Add simple examples for api.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-12 17:14:33 +08:00
f4c52371ab Integration with Infinity (#2894)
### What problem does this PR solve?

Integration with Infinity

- Replaced ELASTICSEARCH with dataStoreConn
- Renamed deleteByQuery with delete
- Renamed bulk to upsertBulk
- getHighlight, getAggregation
- Fix KGSearch.search
- Moved Dealer.sql_retrieval to es_conn.py


### Type of change

- [x] Refactoring
2024-11-12 14:59:41 +08:00
7c486ee3f9 Fix typo (#3319)
### What problem does this PR solve?

Fix typo

### Type of change

- [x] Refactoring
2024-11-11 09:36:39 +08:00
48ab6d7a45 Update authorization for team (#3262)
### What problem does this PR solve?

Update authorization for team.
#3253 #3233
### Type of change

- [x] Refactoring

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-07 19:26:03 +08:00
5038552ed9 fix: improve embedding model validation logic for dataset operations (#3235)
What problem does this PR solve?
When creating or updating datasets with custom embedding models (e.g.,
Ollama), the validation logic was too restrictive and prevented valid
models from being used. The previous implementation would reject valid
custom models if they weren't in the predefined list, even when they
existed in TenantLLMService.

Changes:
- Simplify and improve the embedding model validation flow in
create/update endpoints
- Check TenantLLMService for custom models before rejecting
- Make validation logic more consistent between create and update
operations

### What problem does this PR solve?

This fix allows users to successfully create and update datasets with
custom embedding models while maintaining proper validation checks.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: liuhua <10215101452@stu.ecnu.edu.cn>
2024-11-07 10:36:28 +08:00
f3aaa0d453 Add sdk for Agent API (#3220)
### What problem does this PR solve?

Add sdk for Agent API

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-06 18:03:45 +08:00
a7bf4ca8fc Fix bugs in API (#3204)
### What problem does this PR solve?

Fix bugs in API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-05 14:07:31 +08:00
185c6a0c71 Unified API response json schema (#3170)
### What problem does this PR solve?

Unified API response json schema

### Type of change

- [x] Refactoring
2024-11-05 11:02:31 +08:00
339639a9db add assistant to canvas chat history (#3201)
### What problem does this PR solve?

#3185

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-05 10:04:31 +08:00
cbca7dfce6 fix bugs in test (#3196)
### What problem does this PR solve?

fix bugs in test

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-11-04 20:03:14 +08:00
8305632852 add agent completion API (#3192)
### What problem does this PR solve?

#3105

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-04 17:20:16 +08:00
dd1146ec64 feat: docs for api endpoints to generate openapi specification (#3109)
### What problem does this PR solve?

**Added openapi specification for API routes. This creates swagger UI
similar to FastAPI to better use the API.**
Using python package `flasgger`

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

Not all routes are included since this is a work in progress.

Docs can be accessed on: `{host}:{port}/apidocs`
2024-11-04 15:35:36 +08:00
44ad9a6cd7 Add test for API (#3134)
### What problem does this PR solve?

Add test for API

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2024-11-01 22:59:17 +08:00
9aeb07d830 Add test for CI (#3114)
### What problem does this PR solve?

Add test for CI

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-31 14:07:23 +08:00
3fa570f49b fix: remove useless test code (#3122)
### What problem does this PR solve?

remove useless test code

### Type of change

- [X] Refactoring
2024-10-31 11:56:46 +08:00
18dfa2900c Fix bugs in API (#3103)
### What problem does this PR solve?

Fix bugs in API


- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-30 16:15:42 +08:00
c5a3146a8c fix: modify the response of metadata in Dify retrieval api (#3076)
### What problem does this PR solve?

Modify the response of metadata in Dify retrieval api

resolve   #2914

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-29 11:06:02 +08:00
d133cc043b remove file size check for sdk API (#3066)
### What problem does this PR solve?

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2024-10-28 16:13:40 +08:00
07bb2a6fd6 Turn resource to plural form (#3061)
### What problem does this PR solve?

Turn resource to plural form

### Type of change
- [x] Refactoring

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-28 15:06:18 +08:00
f93f485696 Turn resource to plural form (#3059)
### What problem does this PR solve?

Turn resource to plural form

### Type of change

- [x] Refactoring

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-28 14:15:36 +08:00
391b950be6 Fix non-null violation when inviting people to team (#3015)
### What problem does this PR solve?

Not sure why MySQL is inserting empty string in this case, but when I
use postgres I got `null value in column "invited_by" of relation
"user_tenant" violates not-null constraint`

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2024-10-25 18:39:09 +08:00
648f8e81d1 Fix issues in API (#3008)
### What problem does this PR solve?

Fix issues in API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-24 20:10:47 +08:00
161c7a231b Fix some issues in API and test (#3001)
### What problem does this PR solve?

Fix some issues in API and test

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-24 20:05:21 +08:00
50b425cf89 Test Cases (#2993)
### What problem does this PR solve?

Test Cases

### Type of change

- [x] Refactoring

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-23 22:58:27 +08:00
8714754afc Fix some issues in API (#2982)
### What problem does this PR solve?

Fix some issues in API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-23 12:02:18 +08:00
89d5b2414e fix SILICONFLOW rerank error (#2980)
### What problem does this PR solve?

#2977

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-23 10:12:39 +08:00
226bdd6e99 add auto keywords and auto-question (#2965)
### What problem does this PR solve?

#2687

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-22 13:12:49 +08:00
b2524eec49 fix sequence2txt error and usage total token issue (#2961)
### What problem does this PR solve?

#1363

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-22 11:38:37 +08:00
1935c3be1a Fix some issues in API (#2902)
### What problem does this PR solve?

Fix some issues in API

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-21 14:29:06 +08:00
c760f058df add owner check for team work (#2892)
### What problem does this PR solve?

#2834

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 13:48:57 +08:00
ceecac69e9 Delete useless files (#2889)
### What problem does this PR solve?

Delete useless files

### Type of change


- [x] Other (please describe):
Delete useless files

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-10-18 11:30:40 +08:00
e0c0bdeb0a add team tag to kb (#2890)
### What problem does this PR solve?
#2834

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-10-18 11:30:19 +08:00
6a60e26020 update dashscope (#2871)
### What problem does this PR solve?
#2857
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-10-17 09:52:31 +08:00