pengjunjie/ragflow
1
0
Fork 1
mirror of https://github.com/infiniflow/ragflow.git synced 2025-12-08 20:42:30 +08:00
Code Issues Packages Projects Releases Wiki Activity

added SVG for Groq model model providers (#1470)

Browse Source 
#1432  #1447 
This PR adds support for the GROQ LLM (Large Language Model).

Groq is an AI solutions company delivering ultra-low latency inference
with the first-ever LPU™ Inference Engine. The Groq API enables
developers to integrate state-of-the-art LLMs, such as Llama-2 and
llama3-70b-8192, into low latency applications with the request limits
specified below. Learn more at [groq.com](https://groq.com/).
Supported Models


| ID | Requests per Minute | Requests per Day | Tokens per Minute |

|----------------------|---------------------|------------------|-------------------|
| gemma-7b-it | 30 | 14,400 | 15,000 |
| gemma2-9b-it | 30 | 14,400 | 15,000 |
| llama3-70b-8192 | 30 | 14,400 | 6,000 |
| llama3-8b-8192 | 30 | 14,400 | 30,000 |
| mixtral-8x7b-32768 | 30 | 14,400 | 5,000 |

---------

Co-authored-by: paresh0628 <paresh.tuvoc@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
This commit is contained in:
paresh2806
2024-07-12 06:55:44 +05:30
committed by GitHub
parent 009e18f094
commit ddeac9ab3d
8 changed files with 118 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
api/db/init_data.py
View File

@ -180,6 +180,12 @@ factory_infos = [{
"logo": "",
"tags": "LLM,TEXT EMBEDDING,IMAGE2TEXT",
"status": "1",
},
{
"name": "Groq",
"logo": "",
"tags": "LLM",
"status": "1",
}
# {
# "name": "文心一言",
@ -933,6 +939,47 @@ def init_llm_factory():
"tags": "TEXT EMBEDDING",
"max_tokens": 2048,
"model_type": LLMType.EMBEDDING.value
},
# ------------------------ Groq -----------------------
{
"fid": factory_infos[18]["name"],
"llm_name": "gemma-7b-it",
"tags": "LLM,CHAT,15k",
"max_tokens": 8192,
"model_type": LLMType.CHAT.value
},
{
"fid": factory_infos[18]["name"],
"llm_name": "gemma2-9b-it",
"tags": "LLM,CHAT,15k",
"max_tokens": 8192,
"model_type": LLMType.CHAT.value
},
{
"fid": factory_infos[18]["name"],
"llm_name": "llama3-70b-8192",
"tags": "LLM,CHAT,6k",
"max_tokens": 8192,
"model_type": LLMType.CHAT.value
},
{
"fid": factory_infos[18]["name"],
"llm_name": "llama3-8b-8192",
"tags": "LLM,CHAT,30k",
"max_tokens": 8192,
"model_type": LLMType.CHAT.value
},
{
"fid": factory_infos[18]["name"],
"llm_name": "mixtral-8x7b-32768",
"tags": "LLM,CHAT,5k",
"max_tokens": 32768,
"model_type": LLMType.CHAT.value
}
]
for info in factory_infos: