Deploy Model Inference & Verify APIs

Launch an online LLM inference endpoint and verify OpenAI-compatible completions via cURL.

30–40 minsIntermediateAPI & SDK

Verifiable Outcome

A healthy online inference endpoint with streaming responses accessible via REST and SDKs.

Prerequisites

Compute instance matching model VRAM requirements (e.g. 16GB+ for 7B models or NPU)
An active Personal API Token generated in settings

Select Model & Provision Inference Instance

Choose an open model from the hub and select suitable GPU/NPU compute tiers.

Launch Endpoint & Monitor Health Checks

Wait for container deployment and weight loading until health checks report Ready.

Send cURL Test Request

Execute an OpenAI-compatible completion request with auth headers to verify streaming output.

curl -X POST https://api.opencsg.com/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [{"role": "user", "content": "Hello, introduce OpenCSG in one sentence"}],
    "stream": false
  }'

Inspect Latency & Token Metrics

Monitor time-to-first-token (TTFT), tokens/sec throughput, and hardware utilization.

Once verified, proceed to the next milestone.

Next Steps & Deep Dive

Connect your hands-on achievements back to platform documentation for production best practices.

Explore REST API & SDK Reference