### What problem does this PR solve?
Enhance the robustness of the code
### Type of change
- [x] Refactoring
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
fix code injection in https://github.com/infiniflow/ragflow/issues/1860,
developers can have a check to see if the fix works as expected.
### Type of change
Vulnerability Fix
The answer in the context carries reference markers and passes them to
the large model, which may include the markers in the new answer,
leading to abnormal reference points.
```
{'role': 'assistant', 'content': '设置在地下或半地下空间 ##0$$。'}
```
### What problem does this PR solve?
_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._
### Type of change
- Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
feat: Do not display arrow icons on leaf node of folders #1826
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
add support for eml file parser
#1363
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
#1771 add supprot for OpenAI-API-Compatible
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
support reset the user email from old to new
#1723
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
#1726 fix add local vision llm error when cannot download test pic
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
This pull request addresses an issue where the reference is an empty
array ([]) in specific cases, leading to errors in the application. When
the reference is empty, the code attempts to call the get method on a
list, resulting in the following error message:
``` json
{"retcode": 500, "retmsg": "'list' object has no attribute 'get'", "data": {"answer": "**ERROR**: 'list' object has no attribute 'get'", "reference": []}}
```
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
fix api reference empty bug
```
for chunk_i in answer['reference'].get('chunks',[]):
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'get'
```
```
return np.array([d["relevance_score"] for d in res["results"]]), res["meta"]["tokens"]["input_tokens"]+res["meta"]["tokens"]["output_tokens"]
~~~^^^^^^^^^^^
KeyError: 'results'
```
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
#1602
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
add password reset function by extending the Flask command. #1200
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
fix Redis no such key #1614
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Signed-off-by: seaver <zhudan187@qq.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
The encryption should first perform base64 encoding and then encrypt, to
maintain consistency with the frontend
#1620
### Type of change
- [x] Refactoring
### What problem does this PR solve?
This PR solves the problem that when you delete selected chunks, the
chunks can be deleted but Chunk Number doesn't change. Now you delete
one chunk, the Chunk Number is reduced by one, delete two chunks, the
Chunk Number is reduced by two...
#900
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Signed-off-by: seaver <zhudan187@qq.com>
### What problem does this PR solve?
#762
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
fix function create to slove the problem of creating a chunk and
increasing the chunk number by 2.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue) #900
---------
Signed-off-by: seaver <zhudan187@qq.com>
### What problem does this PR solve?
Aims to stop the process of parsing.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
#1353
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
#1467
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
#1432#1447
This PR adds support for the GROQ LLM (Large Language Model).
Groq is an AI solutions company delivering ultra-low latency inference
with the first-ever LPU™ Inference Engine. The Groq API enables
developers to integrate state-of-the-art LLMs, such as Llama-2 and
llama3-70b-8192, into low latency applications with the request limits
specified below. Learn more at [groq.com](https://groq.com/).
Supported Models
| ID | Requests per Minute | Requests per Day | Tokens per Minute |
|----------------------|---------------------|------------------|-------------------|
| gemma-7b-it | 30 | 14,400 | 15,000 |
| gemma2-9b-it | 30 | 14,400 | 15,000 |
| llama3-70b-8192 | 30 | 14,400 | 6,000 |
| llama3-8b-8192 | 30 | 14,400 | 30,000 |
| mixtral-8x7b-32768 | 30 | 14,400 | 5,000 |
---------
Co-authored-by: paresh0628 <paresh.tuvoc@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
support xinference rerank model
#1455
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Make the document start parsing.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
#1036
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
### What problem does this PR solve?
#308
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Co-authored-by: KevinHuSh <kevinhu.sh@gmail.com>
### What problem does this PR solve?
During knowledge Q&A conversations, the user-defined rerank model's
top_k parameter was not used
#1395
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
