mirror of
https://github.com/yangyaofei/dify-vllm-provider.git
synced 2026-07-23 12:31:55 +08:00
a300663fbd
## 实现过程 - manifest.yaml: 0.2.2 → 0.2.3 - README: reflect v0.2.3 - skills/release: add gh-primary/agent-browser-fallback preference
vLLM Dify Provider
Dify custom model provider for vLLM's OpenAI-Compatible Server, supporting extra parameters and thinking mode features.
Based on the official Dify OpenAI-API-compatible plugin, extended for vLLM OpenAI-Compatible Server.
Latest: v0.2.3
Fixes
- #34: Fixed thinking mode markup tags —
<think/>/</think/>→<think>/</think>, aligned 1:1 with dify-official-plugins openai_api_compatible - #31: Fixed
extra_bodyparameter delivery — JSON contents now merged into top-level request body instead of nested under"extra_body"key
Features (since v0.2.0)
- extra_body: Pass any vLLM extra parameters directly as JSON
- Thinking Mode:
enable_thinkingtoggle withcompatibility_mode(strict/extended) - Reasoning Effort: Support
reasoning_effort(none/low/medium/high), natively supported by vLLM - Compatibility Mode: Extended mode injects
chat_template_kwargs,thinking,enable_thinkingat top level - Structured Output: Support
response_format,json_schema,reasoning_format - Thinking Content Filter: Auto-filter
<think>...</think>when thinking is disabled - Thinking Content Cleanup: Strip thinking content from history before requests
- vLLM Reasoning Field: Priority read
reasoning(vLLM >= 0.17.1), fallback toreasoning_content
v0.2.0 Baseline
Breaking Change: Removed all legacy parameters, use
extra_bodyfor all extra parameter needs.
Usage
Add model
Same as OpenAI-API-compatible, select "Vllm" provider:
Configure extra_body
Pass extra parameters via the extra_body JSON text field:
Example:
{"chat_template_kwargs": {"enable_thinking": true}}