Runner

1. Overview

The Runner is the core component of the CSGHub platform, responsible for executing computational tasks such as model training, inference, and task scheduling.

Through the Runner, communication with the main control end (CSGHub Server) is established, and user workloads are dynamically created and destroyed within the Kubernetes cluster.

This Chart provides a standardized deployment method via Helm, supporting flexible configuration, integration of external dependencies, and automated resource management.

2. Environment Requirements

Project	Requirement
Kubernetes Version	v1.33+
Helm Version	v3.12+
Network Requirements	Cluster nodes must be able to access the CSGHub Server and external image registries (if internal registries are disabled).
Permissions	Requires `cluster-admin` or the ability to create namespaces and RBAC resources (automatically created during deployment).

3. Deployment Steps

Add Helm Repository

helm repo add csghub https://charts.opencsg.com/csghub
helm repo update

Create Namespace (Optional)

kubectl create namespace csghub

Deploy Runner

Obtain the following information from the CSGHub main service:

domain: Provide a second-level domain used to expose the runner service. If the domain provided is example.com, the service will be exposed under runner.example.com.
externalUrl ⚠️ Required (standalone mode):
```
helm get notes csghub -n csghub | grep -A 6 'Access your CSGHub'
```
Get the corresponding URL for CSGHub.

hubAPIToken ⚠️ Required (standalone mode):

kubectl get cm csghub-core -o yaml -n csghub | grep 'API_TOKEN' | awk '{print $NF}'

originClusterID:

Unique identifier for the runner cluster. Mapped to STARHUB_SERVER_CLUSTER_ID in the ConfigMap.
region: A custom parameter used to identify the cluster region (e.g., cn-north).
registry:
```
helm get notes csghub -n csghub | grep -A 8 'Distribution Registry'
```
Obtain the <domain>, username, and password. Set insecure based on whether the externalUrl uses HTTPS.
objectStore:
```
helm get notes csghub -n csghub | grep -A 9 'Minio'
```
The above command provides endpoint, accessKey, and secretKey.

bucket, region, and pathStyle are fixed values. Set secure based on whether the externalUrl uses HTTPS.

Execute Deployment

💡 Tip:

Object storage and container registries can be directly integrated with external infrastructure.

Note: It is recommended to write custom configurations into a custom-values.yaml file for easier upgrades and version management.

helm install runner csghub/runner \
  --namespace csghub \
  --create-namespace \
  --set global.gateway.external.domain="example.com" \
  --set externalUrl="<csghub external_url>" \
  --set hubAPIToken="<csghub hub_api_token>" \
  --set runner.region="<region name>" \
  --set registry.registry="<csghub registry>" \
  --set registry.repository="csghub" \
  --set registry.username="<csghub registry username>" \
  --set registry.password="<csghub registry password>" \
  --set registry.insecure=<true|false> \
  --set objectStore.endpoint="<csghub minio>" \
  --set objectStore.accessKey="<csghub minio username>" \
  --set objectStore.secretKey="<csghub minio password>" \
  --set objectStore.bucket="csghub-registry" \
  --set objectStore.region="cn-north-1" \
  --set objectStore.secure=false \
  --set objectStore.pathStyle=true

Domestic deployment (use a domestic image registry to accelerate image pulling):

helm install runner csghub/runner \
  --namespace csghub \
  --create-namespace \
  --set global.gateway.external.domain="example.com" \
  --set externalUrl="<csghub external_url>" \
  --set hubAPIToken="<csghub hub_api_token>" \
  --set runner.region="<region name>" \
  --set registry.registry="<csghub registry>" \
  --set registry.repository="csghub" \
  --set registry.username="<csghub registry username>" \
  --set registry.password="<csghub registry password>" \
  --set registry.insecure=<true|false> \
  --set objectStore.endpoint="<csghub minio>" \
  --set objectStore.accessKey="<csghub minio username>" \
  --set objectStore.secretKey="<csghub minio password>" \
  --set objectStore.bucket="csghub-registry" \
  --set objectStore.region="cn-north-1" \
  --set objectStore.secure=false \
  --set objectStore.pathStyle=true \
  --set global.image.registry="registry.opencsg.com" \
  --set global.imageRegistry="registry.opencsg.com/opencsghq"

4. Configuration Details

v2.3.0 value restructuring: Some field paths have changed. See the change notes below.

4.1 Global Configuration

Parameter	Default Value	Description
`global.gateway.external.domain`	`csghub.example.com`	Base access domain for the platform
`global.gateway.tls.enabled`	`false`	Whether to enable TLS
`global.image.tag`	`v2.5.0`	Image version tag

4.2 Service Configuration

Parameter	Default Value	Description
`name`	`runner`	Name used to identify runner resources (includes exposed domain)
`runner.region`	`region-0`	Regional identifier for the Runner
`runner.interval`	`60`	Communication interval between Runner and Server (seconds)
`runner.namespace`	`spaces`	Default namespace for user workloads
`externalUrl`	`-`	⚠️ Required (standalone). CSGHub main service URL
`hubAPIToken`	`-`	⚠️ Required (standalone). CSGHub API authentication token
`originClusterID`	`""`	Unique identifier for the runner cluster, written to `STARHUB_SERVER_CLUSTER_ID` (optional)
`runner.mergingNamespace`	`disable`	Namespace merging mode (`multi`/`single`/`disable`)
`space.usePublicDomain`	`true`	Use public domain for app access (v2.3.0: path changed from `runner.usePublicDomain`)
`runner.allowCpuOnGpuNodes`	`false`	🆕 v2.3.0 new. Allow CPU workloads on GPU-labeled nodes

4.3 Package and Image Management

Parameter	Default Value	Description
`space.pipIndexUrl`	`https://pypi.tuna.tsinghua.edu.cn/simple/`	Custom pip source (v2.3.0: path changed from `runner.pipIndexUrl`)
`model.dockerRegBase`	`nil`	Model image registry (v2.3.0: renamed from `model.registry`)
`model.deployTimeoutInMin`	`60`	Model deploy timeout in minutes (v2.3.0: renamed from `model.deployTimeout`, unit changed from seconds to minutes)

4.4 GPU Configuration

Parameter	Default Value	Description
`runner.gpuModelLabel.typeLabel`	`nvidia.com/gpu.product`	GPU model label
`runner.gpuModelLabel.capacityLabel`	`nvidia.com/gpu`	GPU capacity label
`runner.gpuModelLabel.memLabel`	`nvidia.com/gpu.memory`	GPU memory label
`runner.gpuModelLabel.xpuType`	`GPU`	XPU type identifier

4.5 Knative Serving Configuration

💡 Tip: Since v1.12.0, the following parameters are deprecated and used only for backward compatibility.

Parameter	Default Value	Description
`knative.serving.domain`	`example.com`	Knative service domain suffix
`knative.serving.services`	`[]`	Legacy service configuration (deprecated)
`knative.serving.autoscaler.enableScaleToZero`	`true`	Enable KSVC instance auto-shutdown
`knative.serving.autoscaler.scaleToZeroPodRetentionPeriod`	`60m`	Tolerance time for KSVC instance shutdown

4.6 RBAC Configuration

Parameter	Default Value	Description
`rbac.create`	`true`	Automatically create SA, Role, and RoleBinding
`rbac.serviceAccountName`	`""`	Custom SA name; when left empty it is derived as `<release>-<name>` (e.g. `helm install runner` produces `runner-runner`)

4.7 Logging and Monitoring

Parameter	Default Value	Description
`logging.level`	`info`	Log level (`info`/`debug`/`error`)
`logcollector.enabled`	`false`	Enable log collector
`logcollector.loki.address`	`""`	Loki service address
`tempo.address`	`""`	Tempo tracing address

Note: Loki is not exposed by default on the CSGHub side. You must set loki.gateway.enabled=true in CSGHub. Tempo support for external exposure is coming soon.

4.8 Persistent Storage

Parameter	Default	Description
`runner.storageClassName`	`nil`	Specify a Kubernetes StorageClass for Space persistent volumes, e.g. `local-path`, `nfs-client`

4.9 External Resource Configuration

Registry:

registry:
  registry: "registry.example.com"
  repository: "csghub"
  username: "user"
  password: "pass"
  insecure: false

Object Store:

objectStore:
  endpoint: "https://minio.example.com"
  accessKey: "admin"
  secretKey: "password"
  bucket: "csghub-registry"
  region: "us-east-1"
  secure: true
  pathStyle: true

💡 You can also reference an existing Secret to supply the external registry / object store connection info (no need to put credentials in values.yaml). The Secret must be created in the release namespace beforehand.

Registry: global.registry.existingSecret (honored when global.registry.enabled=false), or the service-level registry.existingSecret (takes precedence over the global one). Secret keys (base64-encoded): REGISTRY_HOST (required), REGISTRY_REPOSITORY, REGISTRY_USERNAME, REGISTRY_PASSWORD

Object store: global.objectStore.existingSecret (honored when global.objectStore.enabled=false), or the service-level objectStore.existingSecret (takes precedence over the global one). Secret keys: OBJECT_STORE_ENDPOINT (required), OBJECT_STORE_ACCESS_KEY, OBJECT_STORE_SECRET_KEY, OBJECT_STORE_REGION, OBJECT_STORE_BUCKET

4.10 Resource and Scheduling Configuration

Parameter	Default Value	Description
`resources`	`{}`	Pod requests/limits configuration
`nodeSelector`	`{}`	Node selector
`tolerations`	`[]`	Tolerations configuration
`affinity`	`{}`	Affinity configuration

5. Verify Deployment

After deployment, verify the status using:

kubectl get pods -n csghub
kubectl get svc -n csghub

To view Runner logs:

kubectl logs -f deploy/runner-runner -n csghub

6. Upgrade and Uninstallation

6.1 Upgrade Chart

Pre-upgrade Operations

When upgrading to v2.4.2 or higher, you must update the agent-sandbox related CRDs first.

Since Helm does not automatically update the CRDs (CustomResourceDefinitions) required by dependencies during an upgrade, you must manually update the relevant CRDs before upgrading.

Run the following script:

curl -sSL https://charts.opencsg.com/repository/scripts/crds_install.sh | bash

By default, this script updates the CRDs of all components. To update the CRDs of a specific component only, use the CRD_SERVICES variable. For example:
curl -sSL https://charts.opencsg.com/repository/scripts/crds_install.sh | CRD_SERVICES=agent-sandbox bash
Currently supported values for CRD_SERVICES: agent-sandbox, gateway-helm.

What this script does:

Updates the agent-sandbox related CRDs.
Ensures relevant components function correctly after upgrading to v2.4.2.

Execute Upgrade

helm upgrade runner csghub/runner -n csghub -f custom-values.yaml

6.2 Uninstall Chart

helm uninstall runner -n csghub

7. FAQ

Issue	Solution
Runner cannot connect to Server	Verify if `externalUrl` and `hubAPIToken` are correct.
Knative service not installed	Ensure `autoConfigure: true` and the cluster has necessary permissions.
GPU tasks failed to schedule	Check if node labels and GPU drivers are correctly installed.
Image pull failure	Verify registry access permissions and `image.pullSecrets` configuration.

1. Overview​

2. Environment Requirements​

3. Deployment Steps​

Add Helm Repository​

Create Namespace (Optional)​

Deploy Runner​

Execute Deployment​

4. Configuration Details​

4.1 Global Configuration​

4.2 Service Configuration​

4.3 Package and Image Management​

4.4 GPU Configuration​

4.5 Knative Serving Configuration​

4.6 RBAC Configuration​

4.7 Logging and Monitoring​

4.8 Persistent Storage​

4.9 External Resource Configuration​

4.10 Resource and Scheduling Configuration​

5. Verify Deployment​

6. Upgrade and Uninstallation​

6.1 Upgrade Chart​

Pre-upgrade Operations​

Execute Upgrade​

6.2 Uninstall Chart​

7. FAQ​