Commit Graph

65 Commits

Author SHA1 Message Date
90975460af Add pdf support for QA parser (#1155)
### What problem does this PR solve?

Support extracting questions and answers from PDF files

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-14 15:12:39 +08:00
4454ba7a1e add self-rag (#1070)
### What problem does this PR solve?

#1069 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-06-06 11:13:39 +08:00
9ed0e50f6b Update info (#1005)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-05-31 09:53:04 +08:00
758eb03ccb fix jina adding issure and term weight refinement (#974)
### What problem does this PR solve?

#724 #162

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 19:38:57 +08:00
614defec21 add rerank model (#969)
### What problem does this PR solve?

feat: add rerank models to the project #724 #162

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-29 16:50:02 +08:00
7eee193956 fix #917 #915 (#946)
### What problem does this PR solve?

#917 
#915

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-28 11:13:02 +08:00
GYH
be13429d05 Add api/list_kb_docs function and modify api/list_chunks (#874)
### What problem does this PR solve?
#717 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-05-22 14:58:56 +08:00
2b36283712 fix english query bug (#840)
### What problem does this PR solve?

#834 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-20 12:23:51 +08:00
648a2baaa9 fix disabled doc is still retreivalable (#695)
### What problem does this PR solve?

Fix that disabled doc is still retreivalable

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-09 15:32:24 +08:00
de839fc3f0 optimize srv broker and executor logic (#630)
### What problem does this PR solve?

Optimize task broker and executor for reduce memory usage and deployment
complexity.

### Type of change
- [x] Performance Improvement
- [x] Refactoring

### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-05-07 11:43:33 +08:00
c6b6c748ae fix file encoding detection bug (#653)
### What problem does this PR solve?

#651 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-05-07 10:01:24 +08:00
7d3b68bb1e refine code (#626)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2024-04-30 17:53:28 +08:00
8c07992b6c refine code (#595)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-28 19:13:33 +08:00
9d60a84958 refactor code (#583)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2024-04-28 13:19:54 +08:00
f1c98aad6b Update version info (#564)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Documentation Update
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2024-04-26 20:07:26 +08:00
ed6081845a Fit a lot of encodings for text file. (#458)
### What problem does this PR solve?

#384

### Type of change

- [x] Performance Improvement
2024-04-19 18:02:53 +08:00
d4e0bfc8a5 fix gb2312 encoding issue (#394)
### What problem does this PR solve?

Issue link:#384
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-16 19:45:14 +08:00
890561703b Add bce-embedding and fastembed (#383)
### What problem does this PR solve?


Issue link:#326

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-04-16 16:42:19 +08:00
24c15daaed fix es exception (#298)
### What problem does this PR solve?

Issue link:#295
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-10 16:00:48 +08:00
923cbe488e fix #258 task_executor occupy cpu too much (#288)
### What problem does this PR solve?

Issue link:#285

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-10 10:11:22 +08:00
653c759d8d resolve issure to call ZH?IPUAI (#277)
### What problem does this PR solve?

Issue link:#265

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-09 16:16:10 +08:00
23b448cf96 fix docker compose issue (#238)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/226)]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-07 09:04:32 +08:00
392e515c3f fix bug about reload knowledgebase configuration reloading (#210)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/209)]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-04-03 11:00:50 +08:00
05298d5626 refine README (#169) 2024-03-29 10:48:29 +08:00
a5384446e3 let's load model from local (#163) 2024-03-28 16:10:47 +08:00
f3477202fe refine citation (#161) 2024-03-28 11:45:50 +08:00
37185466e2 READEME refined (#156) 2024-03-27 13:14:36 +08:00
fd7fcb5baf apply pep8 formalize (#155) 2024-03-27 11:33:46 +08:00
da21320b88 fix plainPdf bugs (#152) 2024-03-26 15:11:07 +08:00
71fe314955 refine page ranges (#147) 2024-03-25 13:11:57 +08:00
f6aee7f230 add use layout or not option (#145)
* add use layout or not option

* trival
2024-03-22 19:21:09 +08:00
6c6b144de2 refine manual parser (#140) 2024-03-21 18:17:32 +08:00
5875c8ba08 Add 'One' chunk method (#137) 2024-03-20 18:57:22 +08:00
6999598101 refine for English corpus (#135) 2024-03-20 16:56:16 +08:00
9a843667b3 fix github account login issue (#132) 2024-03-19 15:31:47 +08:00
9da671b951 refine manul parser (#131) 2024-03-19 12:26:04 +08:00
675a9f8d9a add dockerfile for cuda envirement. Refine table search strategy, (#123) 2024-03-14 19:45:29 +08:00
8f86ab9f7f refine pdf parser, add time zone to userinfo (#112) 2024-03-08 11:24:24 +08:00
436c52bbc5 refine presentation parser (#110) 2024-03-07 17:21:38 +08:00
2d7c9080f4 deal with stop reason being length problem (#109) 2024-03-07 16:12:01 +08:00
b89ac3c4be chage tas execution logic (#103) 2024-03-06 19:16:31 +08:00
d7c362f237 adjust hierarchical_merge strategy (#100) 2024-03-06 09:09:16 +08:00
602038ac49 fix task cancling bug (#98) 2024-03-05 16:33:47 +08:00
8a57f2afd5 change callback strategy, add timezone to docker (#96) 2024-03-05 12:08:41 +08:00
7bfaf0df29 fix position extraction bug (#93)
* fix position extraction bug

* remove delimiter for naive parser
2024-03-04 17:08:35 +08:00
685b4d8a95 fix table desc bugs, add positions to chunks (#91) 2024-03-04 14:42:26 +08:00
8a726fb04b solve task execution issues (#90) 2024-03-01 19:48:01 +08:00
1567e881de fix bug of inserting cites (#76) 2024-02-27 17:51:54 +08:00
4568a4b2cb refine admin initialization (#75) 2024-02-27 14:57:34 +08:00
7fd1eca582 init README of deepdoc, add picture processer. (#71)
* init README of deepdoc, add picture processer.

* add resume parsing
2024-02-23 18:28:12 +08:00