mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-01-30 23:26:36 +08:00
docs: update docs icons (#12465)
### What problem does this PR solve? Update icons for docs. Trailing spaces are auto truncated by the editor, does not affect real content. ### Type of change - [x] Documentation Update
This commit is contained in:
@ -4,5 +4,8 @@
|
||||
"link": {
|
||||
"type": "generated-index",
|
||||
"description": "Guides on model settings."
|
||||
},
|
||||
"customProps": {
|
||||
"categoryIcon": "LucideBox"
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,6 +1,9 @@
|
||||
---
|
||||
sidebar_position: 2
|
||||
slug: /deploy_local_llm
|
||||
sidebar_custom_props: {
|
||||
categoryIcon: LucideMonitorCog
|
||||
}
|
||||
---
|
||||
|
||||
# Deploy local models
|
||||
@ -53,9 +56,9 @@ $ sudo docker exec ollama ollama pull llama3.2
|
||||
```
|
||||
|
||||
```bash
|
||||
$ sudo docker exec ollama ollama pull bge-m3
|
||||
> pulling daec91ffb5dd... 100% ▕████████████████▏ 1.2 GB
|
||||
> success
|
||||
$ sudo docker exec ollama ollama pull bge-m3
|
||||
> pulling daec91ffb5dd... 100% ▕████████████████▏ 1.2 GB
|
||||
> success
|
||||
```
|
||||
|
||||
### 2. Find Ollama URL and ensure it is accessible
|
||||
@ -105,7 +108,7 @@ Max retries exceeded with url: /api/chat (Caused by NewConnectionError('<urllib3
|
||||
### 5. Update System Model Settings
|
||||
|
||||
Click on your logo **>** **Model providers** **>** **System Model Settings** to update your model:
|
||||
|
||||
|
||||
- *You should now be able to find **llama3.2** from the dropdown list under **Chat model**, and **bge-m3** from the dropdown list under **Embedding model**.*
|
||||
|
||||
### 6. Update Chat Configuration
|
||||
@ -125,7 +128,7 @@ To deploy a local model, e.g., **Mistral**, using Xinference:
|
||||
|
||||
### 1. Check firewall settings
|
||||
|
||||
Ensure that your host machine's firewall allows inbound connections on port 9997.
|
||||
Ensure that your host machine's firewall allows inbound connections on port 9997.
|
||||
|
||||
### 2. Start an Xinference instance
|
||||
|
||||
@ -148,13 +151,13 @@ In RAGFlow, click on your logo on the top right of the page **>** **Model provid
|
||||
|
||||
### 5. Complete basic Xinference settings
|
||||
|
||||
Enter an accessible base URL, such as `http://<your-xinference-endpoint-domain>:9997/v1`.
|
||||
Enter an accessible base URL, such as `http://<your-xinference-endpoint-domain>:9997/v1`.
|
||||
> For rerank model, please use the `http://<your-xinference-endpoint-domain>:9997/v1/rerank` as the base URL.
|
||||
|
||||
### 6. Update System Model Settings
|
||||
|
||||
Click on your logo **>** **Model providers** **>** **System Model Settings** to update your model.
|
||||
|
||||
|
||||
*You should now be able to find **mistral** from the dropdown list under **Chat model**.*
|
||||
|
||||
### 7. Update Chat Configuration
|
||||
@ -170,7 +173,7 @@ To deploy a local model, e.g., **Qwen2**, using IPEX-LLM-accelerated Ollama:
|
||||
### 1. Check firewall settings
|
||||
|
||||
Ensure that your host machine's firewall allows inbound connections on port 11434. For example:
|
||||
|
||||
|
||||
```bash
|
||||
sudo ufw allow 11434/tcp
|
||||
```
|
||||
@ -179,7 +182,7 @@ sudo ufw allow 11434/tcp
|
||||
|
||||
#### 2.1 Install IPEX-LLM for Ollama
|
||||
|
||||
:::tip NOTE
|
||||
:::tip NOTE
|
||||
IPEX-LLM's supports Ollama on Linux and Windows systems.
|
||||
:::
|
||||
|
||||
@ -191,7 +194,7 @@ For detailed information about installing IPEX-LLM for Ollama, see [Run llama.cp
|
||||
|
||||
#### 2.2 Initialize Ollama
|
||||
|
||||
1. Activate the `llm-cpp` Conda environment and initialize Ollama:
|
||||
1. Activate the `llm-cpp` Conda environment and initialize Ollama:
|
||||
|
||||
<Tabs
|
||||
defaultValue="linux"
|
||||
@ -200,7 +203,7 @@ For detailed information about installing IPEX-LLM for Ollama, see [Run llama.cp
|
||||
{label: 'Windows', value: 'windows'},
|
||||
]}>
|
||||
<TabItem value="linux">
|
||||
|
||||
|
||||
```bash
|
||||
conda activate llm-cpp
|
||||
init-ollama
|
||||
@ -218,7 +221,7 @@ For detailed information about installing IPEX-LLM for Ollama, see [Run llama.cp
|
||||
</Tabs>
|
||||
|
||||
2. If the installed `ipex-llm[cpp]` requires an upgrade to the Ollama binary files, remove the old binary files and reinitialize Ollama using `init-ollama` (Linux) or `init-ollama.bat` (Windows).
|
||||
|
||||
|
||||
*A symbolic link to Ollama appears in your current directory, and you can use this executable file following standard Ollama commands.*
|
||||
|
||||
#### 2.3 Launch Ollama service
|
||||
@ -226,7 +229,7 @@ For detailed information about installing IPEX-LLM for Ollama, see [Run llama.cp
|
||||
1. Set the environment variable `OLLAMA_NUM_GPU` to `999` to ensure that all layers of your model run on the Intel GPU; otherwise, some layers may default to CPU.
|
||||
2. For optimal performance on Intel Arc™ A-Series Graphics with Linux OS (Kernel 6.2), set the following environment variable before launching the Ollama service:
|
||||
|
||||
```bash
|
||||
```bash
|
||||
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
|
||||
```
|
||||
3. Launch the Ollama service:
|
||||
@ -314,12 +317,12 @@ To enable IPEX-LLM accelerated Ollama in RAGFlow, you must also complete the con
|
||||
3. [Update System Model Settings](#6-update-system-model-settings)
|
||||
4. [Update Chat Configuration](#7-update-chat-configuration)
|
||||
|
||||
### 5. Deploy VLLM
|
||||
### 5. Deploy VLLM
|
||||
|
||||
ubuntu 22.04/24.04
|
||||
|
||||
```bash
|
||||
pip install vllm
|
||||
pip install vllm
|
||||
```
|
||||
### 5.1 RUN VLLM WITH BEST PRACTISE
|
||||
|
||||
|
||||
@ -1,6 +1,9 @@
|
||||
---
|
||||
sidebar_position: 1
|
||||
slug: /llm_api_key_setup
|
||||
sidebar_custom_props: {
|
||||
categoryIcon: LucideKey
|
||||
}
|
||||
---
|
||||
|
||||
# Configure model API key
|
||||
@ -30,7 +33,7 @@ You have two options for configuring your model API key:
|
||||
- Update `api_key` with yours.
|
||||
- Update `base_url` if you use a proxy to connect to the remote service.
|
||||
3. Reboot your system for your changes to take effect.
|
||||
4. Log into RAGFlow.
|
||||
4. Log into RAGFlow.
|
||||
_After logging into RAGFlow, you will find your chosen model appears under **Added models** on the **Model providers** page._
|
||||
|
||||
### Configure model API key after logging into RAGFlow
|
||||
|
||||
Reference in New Issue
Block a user