Inference Engines and Versions
The available minor versions, container images, and driver requirements of the inference engines built into the CSGHub platform are summarized below, for reference when creating frameworks and deploying instances.
Built-in Inference Engines and Versions
NVIDIA GPU
Choose the entry matching the CUDA version supported by the cluster driver.
| Engine | Version | Image | CUDA | Model format | Use case |
|---|---|---|---|---|---|
| vllm | v0.28.0 | opencsghq/vllm:v0.28.0 | 13.0 | safetensors | Text generation / embedding / reranking / speech |
| vllm | v0.9.2 | opencsghq/vllm:v0.9.2-cu118 | 11.8 | safetensors | Text generation |
| nvidia-vllm | 25.11-py3 | opencsghq/nvidia-vllm:25.11-py3 | 13.0 | safetensors | Text generation / embedding / reranking |
| sglang | v0.5.14 | opencsghq/sglang:v0.5.14-cu130 | 13.0 | safetensors | Text generation |
| nvidia-sglang | 25.12-py3 | opencsghq/nvidia-sglang:25.12-py3 | 13.0 | safetensors | Text generation |
| sglang-qwen3-guard-stream | v0.5.3rc0 | opencsghq/sglang:0.5.3rc0-qwen3-guard-stream | 12.8.1 | safetensors | Content moderation (Qwen3-Guard streaming) |
| tgi | 3.2.1 | opencsghq/tgi:3.2.1 | 12.4 | safetensors | Text generation |
| tei | 1.7 | opencsghq/tei:1.7 | 12.2 | safetensors | Embedding / reranking |
| llama.cpp | b9787 | opencsghq/llama.cpp:b9787 | 12.4 | gguf | Text generation |
| ktransformers | 0.2.1.post1 | opencsghq/ktransformers:0.2.1.post1 | 12.4 | gguf | Text generation |
| hf-inference-toolkit | 0.5.5 | opencsghq/hf-inference-toolkit:0.5.5 | 12.8 | safetensors | Text-to-image |
| diffusers | 0.39.0 | opencsghq/diffusers:0.39.0 | 12.8 | safetensors | Text-to-image / image editing |
| diffusers | 0.38.0 | opencsghq/diffusers:0.38.0 | 12.8 | safetensors | Text-to-image / image editing (legacy) |
| funasr | 1.3.9 | opencsghq/funasr:1.3.9 | 12.8 | pytorch | Speech recognition |
| paddleocr | 3.7.0 | opencsghq/paddleocr:3.7.0 | 12.8 | paddle_static | OCR |
| paddleocr-vl | 3.7.0 | opencsghq/paddleocr-vl:3.7.0 | 12.8 | safetensors | OCR (vision-language) |
| audiofly | 1.0 | opencsghq/audiofly:1.0 | 12.1 | pytorch | Text-to-audio |
| audio-fish | 1.5.1 | opencsghq/fish-speech:server-cuda | 12.6 | pytorch | Speech synthesis (fish-speech) |
| lightx2v | 0.0.1 | opencsghq/lightx2v:26011201-cu128 | 12.8 | safetensors | Video generation |
| longcat-video | 1.5 | opencsghq/longcat-video:1.5-cu124 | 12.4 | safetensors | Audio-driven avatar video |
CPU
No driver version requirement.
| Engine | Version | Image | Model format | Use case |
|---|---|---|---|---|
| vllm | v0.24.0 | opencsghq/vllm-cpu:v0.24.0 | safetensors | Text generation |
| tei | 1.7 | opencsghq/tei:cpu-1.7 | safetensors | Embedding / reranking |
| llama.cpp | b9787 | opencsghq/llama.cpp:b9787-cpu | gguf | Text generation |
| funasr | 1.3.9 | opencsghq/funasr-cpu:1.3.9 | pytorch | Speech recognition |
| paddleocr | 3.7.0 | opencsghq/paddleocr-cpu:3.7.0 | paddle_static | OCR |
| audio-fish | 1.5.1 | opencsghq/fish-speech:server-cpu | pytorch | Speech synthesis (fish-speech) |
AMD ROCm GPU
The driver version is the ROCm version the image is built for.
| Engine | Version | Image | ROCm | Model format | Use case |
|---|---|---|---|---|---|
| amd-vllm | 0.28.0 | opencsghq/amd-vllm:rocm7.2.1_vllm_0.28.0 | 7.2.1 | safetensors | Text generation / embedding / reranking |
| amd-tgi | 3.3.6 | opencsghq/tgi-rocm:rocm6.3.1_tgi_3.3.6 | 6.3.1 | safetensors | Text generation |
| amd-llama.cpp | b9787 | opencsghq/llama.cpp-rocm:rocm7.2.2-b9787 | 7.2.2 | gguf | Text generation |
| amd-funasr | 1.3.9 | opencsghq/funasr-rocm:1.3.9 | 7.2.2 | pytorch | Speech recognition |
| amd-diffusers | 0.39.0 | opencsghq/diffusers-rocm:0.39.0 | 7.2.2 | safetensors | Text-to-image / image editing |
| amd-diffusers | 0.38.0 | opencsghq/diffusers-rocm:0.38.0 | 7.2.2 | safetensors | Text-to-image / image editing (legacy) |
| amd-hf-inference-toolkit | 0.5.5 | opencsghq/hf-inference-toolkit-rocm:0.5.5 | 7.2.2 | safetensors | Text-to-image |
| amd-audiofly | 1.0 | opencsghq/audiofly-rocm:1.0 | 7.2.2 | pytorch | Text-to-audio |
| amd-longcat-video | 1.5 | opencsghq/longcat-video-rocm:1.5-rocm7.2.2 | 7.2.2 | safetensors | Audio-driven avatar video |
Huawei Ascend NPU
Driver versions are bundled in the images.
| Engine | Version | Image | Model format | Use case |
|---|---|---|---|---|
| ascend-vllm | v0.25.1rc | opencsghq/ascend-vllm:v0.25.1rc | safetensors | Text generation |
| mindie | 1.0.RC2 | opencsghq/mindie:2.0-csg-1.0.RC2 | safetensors | Text generation |
MetaX GPGPU
Driver versions are bundled in the images.
| Engine | Version | Image | Model format | Use case |
|---|---|---|---|---|
| vllm | v0.25.0 | opencsghq/metax-vllm:0.25.0 | safetensors | Text generation |
| sglang | v0.5.12 | opencsghq/metax-sglang:0.5.12 | safetensors | Text generation |
Hygon DCU
The driver version is the DTK version the image is built for.
| Engine | Version | Image | DTK | Model format | Use case |
|---|---|---|---|---|---|
| vllm | v0.8.5 | opencsghq/vllm:v0.8.5-dtk25.04 | 25.04 | safetensors | Text generation |
Kunlun GCU
See the table below for driver versions.
| Engine | Version | Image | Driver | Model format | Use case |
|---|---|---|---|---|---|
| tgi-gcu | 3.2.1 | opencsghq/tgi-gcu:tgi_25.02.22.1 | 25.02 | safetensors | Text generation |