### What problem does this PR solve?
Optimize task broker and executor for reduce memory usage and deployment
complexity.
### Type of change
- [x] Performance Improvement
- [x] Refactoring
### Change Log
- Enhance redis utils for message queue(use stream)
- Modify task broker logic via message queue (1.get parse event from
message queue 2.use ThreadPoolExecutor async executor )
- Modify the table column name of document and task (process_duation ->
process_duration maybe just a spelling mistake)
- Reformat some code style(just what i see)
- Add requirement_dev.txt for developer
- Add redis container on docker compose
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Documentation Update
- [x] Refactoring
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- [x] Documentation Update
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
### What problem does this PR solve?
Since Kibana service is not currently being used, the associated volume
'kibanadata' has been commented out in the Docker Compose file. This
change helps to prevent the allocation of unnecessary resources and
simplifies the configuration.
### Type of change
- [x] Refactoring
unused Kibana volume storage
### What problem does this PR solve?
Update version to v0.3.1
### Type of change
- [x] Documentation Update
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
### What problem does this PR solve?
This PR aims to add support for running Ragflow on Docker with the
OpenCloudOS 9 distribution.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: edwardewang <edwardewang@tencent.com>
### What problem does this PR solve?
The docker-compose file can't config minio related port by .env file. So
I just add env `MINIO_CONSOLE_PORT=9001
MINIO_PORT=9000` to .env file.
### Type of change
- [x] Refactoring
### What problem does this PR solve?
update HF_ENDPOINT in docker-compose.yml
Issue link:#350
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
Issue link:#[[Link the issue
here](https://github.com/infiniflow/ragflow/issues/226)]
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
* add field progress msg into docinfo; add file processing procedure
* go through upload, create kb, add doc to kb
* smoke test for all API
* smoke test for all API
