6ff63ee2ba
Support for code files parse ( #789 )
...
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 16:34:28 +08:00
d8c080ee52
fix bugs in searching file using keywords ( #780 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-15 12:51:57 +08:00
626ace8639
Updated document upload method ( #777 )
...
### What problem does this PR solve?
api_app.py
/document/upload
add two non mandatory parameters
parser_id:
[naive,qaresume,manual,table,paper,book,laws,presentation,picture,one]
run: 1
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 12:22:11 +08:00
aa1c915d6e
support gpt-4o ( #773 )
...
### What problem does this PR solve?
#771
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-15 11:16:08 +08:00
d7744f5870
Refactor method name ( #760 )
...
### What problem does this PR solve?
#757
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2024-05-14 14:48:15 +08:00
aa03dfa453
fix bug of get file ( #746 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-13 14:02:38 +08:00
69b7c61498
fix: typo in user_app.py ( #740 )
...
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Bug Fix (non-breaking change
- [x] Other (please describe): Fix typo
2024-05-13 09:25:45 +08:00
04a9e95161
let file in knowledgebases visible in file manager ( #714 )
...
### What problem does this PR solve?
Let file in knowledgebases visible in file manager.
#162
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-11 16:04:28 +08:00
7ddb2f19be
make sure to raise exception if redis is not there ( #674 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-05-08 15:20:45 +08:00
eb27a4309e
add support for deepseek ( #668 )
...
### What problem does this PR solve?
#666
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-05-08 10:30:02 +08:00
8d6d7f6887
fix task losting isssue ( #665 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 20:46:45 +08:00
a5aed2412f
fix bugs ( #662 )
...
### What problem does this PR solve?
Fix import error for task_service.py
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 16:41:56 +08:00
de839fc3f0
optimize srv broker and executor logic ( #630 )
...
### What problem does this PR solve?
Optimize task broker and executor for reduce memory usage and deployment
complexity.
### Type of change
- [x] Performance Improvement
- [x] Refactoring
### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com >
2024-05-07 11:43:33 +08:00
c6b6c748ae
fix file encoding detection bug ( #653 )
...
### What problem does this PR solve?
#651
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 10:01:24 +08:00
ca5acc151a
Refactor: Use TaskStatus enum for task status handling ( #646 )
...
### What problem does this PR solve?
This commit changes the status 'not started' from being hard-coded to
being maintained by the TaskStatus enum. This enhancement ensures
consistency across the codebase and improves maintainability.
### Type of change
- [x] Refactoring
2024-05-06 18:39:17 +08:00
5f03a4de11
remove redis ( #629 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-30 19:00:41 +08:00
cab274f560
remove PyMuPDF ( #618 )
...
### What problem does this PR solve?
#613
### Type of change
- [x] Other (please describe):
2024-04-30 12:38:09 +08:00
674b3aeafd
fix disable and enable llm setting in dialog ( #616 )
...
### What problem does this PR solve?
#614
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-30 11:04:14 +08:00
2af74cc494
refine docker layers ( #606 )
...
### What problem does this PR solve?
### Type of change
- [x] Performance Improvement
2024-04-29 17:57:40 +08:00
6874c6f3a7
refine document upload ( #602 )
...
### What problem does this PR solve?
#567
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-29 15:45:08 +08:00
8c07992b6c
refine code ( #595 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-28 19:13:33 +08:00
9d60a84958
refactor code ( #583 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-28 13:19:54 +08:00
038822f3bd
make cites in conversation API configurable ( #576 )
...
### What problem does this PR solve?
#566
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-28 11:56:17 +08:00
944776f207
fix bug about fetching file from minio ( #574 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-28 09:57:40 +08:00
ab06f502d7
fix bug of file management ( #565 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 19:59:21 +08:00
84b39c60f6
fix rename bug ( #562 )
...
### What problem does this PR solve?
fix rename file bugs
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 18:55:21 +08:00
f69ff39fa0
add file management feature ( #560 )
...
### What problem does this PR solve?
### Type of change
- [x] Documentation Update
2024-04-26 17:21:53 +08:00
b75d75e995
fix youdao bug ( #551 )
...
### What problem does this PR solve?
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-26 09:58:22 +08:00
26003b5076
Add upload file by knowledge base name API. ( #539 )
...
### What problem does this PR solve?
Add upload file by knowledge base name API.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
---------
Co-authored-by: chrysanthemum-boy <fannc@qq.com >
2024-04-25 15:10:19 +08:00
66f8d35632
Refactor ( #537 )
...
### What problem does this PR solve?
### Type of change
- [x] Refactoring
2024-04-25 14:14:28 +08:00
72384b191d
Add .doc file parser. ( #497 )
...
### What problem does this PR solve?
Add `.doc` file parser, using tika.
```
pip install tika
```
```
from tika import parser
from io import BytesIO
def extract_text_from_doc_bytes(doc_bytes):
file_like_object = BytesIO(doc_bytes)
parsed = parser.from_buffer(file_like_object)
return parsed["content"]
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: chrysanthemum-boy <fannc@qq.com >
2024-04-23 15:31:43 +08:00
b8e58fe27a
add redis to accelerate access of minio ( #482 )
...
### What problem does this PR solve?
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-22 14:11:09 +08:00
ed6081845a
Fit a lot of encodings for text file. ( #458 )
...
### What problem does this PR solve?
#384
### Type of change
- [x] Performance Improvement
2024-04-19 18:02:53 +08:00
453c29170f
make sure the models will not be load twice ( #422 )
...
### What problem does this PR solve?
#381
### Type of change
- [x] Refactoring
2024-04-18 09:37:23 +08:00
03f8b01b3b
fix bug for fasetembed ( #392 )
...
### What problem does this PR solve?
Issue link:#325
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-16 19:12:12 +08:00
890561703b
Add bce-embedding and fastembed ( #383 )
...
### What problem does this PR solve?
Issue link:#326
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-16 16:42:19 +08:00
c344486aa0
enlarge max file number per user limit ( #373 )
...
### What problem does this PR solve?
Issue link:#370
### Type of change
- [x] Refactoring
2024-04-16 10:00:52 +08:00
826ad6a33a
feat: FastEmbed embedding support ( #291 )
...
### Description
Following up on https://github.com/infiniflow/ragflow/pull/275 , this PR
adds support for FastEmbed model configurations.
The options are not exhaustive. You can find the full list
[here](https://qdrant.github.io/fastembed/examples/Supported_Models/ ).
P.S. I ran into OOM issues when building the image.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com >
2024-04-15 15:58:06 +08:00
c39b751600
conversation API backend update ( #360 )
...
### What problem does this PR solve?
Issue link:#345
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-15 14:43:44 +08:00
4608cccd05
add new model gpt-3-turbo ( #352 )
...
### What problem does this PR solve?
Issue link:#351
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-15 08:58:42 +08:00
d0ff779d3f
issue 341_Update __init__.py ( #344 )
...
### What problem does this PR solve?
Issue link:#341
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-13 12:01:21 +08:00
bc6828c45f
add quota config in dialog conf ( #339 )
...
### What problem does this PR solve?
Issue link:#337
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-12 14:58:03 +08:00
e20207101a
fix wrong log printting ( #330 )
...
### What problem does this PR solve?
Issue link:#325
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-12 09:08:08 +08:00
91068edf16
Support Xinference ( #320 )
...
### What problem does this PR solve?
Issue link:#299
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-11 18:22:25 +08:00
4fa768e733
fix bug #314 ( #317 )
...
### What problem does this PR solve?
Issue link:#314
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-11 16:25:28 +08:00
f6c7204002
refine log format ( #312 )
...
### What problem does this PR solve?
Issue link:#264
### Type of change
- [x] Documentation Update
- [x] Refactoring
2024-04-11 10:13:43 +08:00
0b2808f990
Update default language to English ( #309 )
...
### What problem does this PR solve?
Update default language to English
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2024-04-11 08:17:15 +08:00
653c759d8d
resolve issure to call ZH?IPUAI ( #277 )
...
### What problem does this PR solve?
Issue link:#265
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-09 16:16:10 +08:00
e1e693ec36
set database logger level ( #270 )
...
### What problem does this PR solve?
Issue link:#264
### Type of change
- [x] Performance Improvement
2024-04-09 09:47:02 +08:00
b6887a20f8
Support new feature about Ollama ( #262 )
...
### What problem does this PR solve?
Issue link:#221
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2024-04-08 19:59:31 +08:00
