高级特性
一、算力卡参数说明
1.1 界面参数说明
# 整个交互界面参数说明
1. 打开开关显示下面配置
2. 选中算力卡，意思安装该算力卡
2.1 天数GPU：ix-device-plugin.enabled=true
2.2 海光DCU:hygon-device-plugin.enabled=true
2.3 昇腾NPU:ascend-device-plugin.enabled=true
2.4 NVIDIA:gpu-operator.enabled=true
2.5 如果安装NVIDIA时,node-feature-discovery.enabled=false,其他算力卡都要选择true
3. 分布式打开设置：lws.emabled=true,否则就是false
4. 模式选择
4.1 选择整卡模式无插件选择
4.2 仅选中nvidia 才显示MIG模式
4.3 选中虚拟化模式
4.3.1 选中hami, hami.enabled=true
4.3.1.1 打开 numa亲和：vocano.custom.scheduler_config_override 的第二个plugins 覆盖或者增加
- name: numa-aware
arguments:
weight: 10
关闭时去掉
4.3.1.2 打开 优先级抢占调度：vocano.custom.scheduler_config_override，tiers所有的.plugins 里面所有的 enablePreemptable 设置为true
4.3.1.3 打开 公平调度：vocano.custom.scheduler_config_override 修改
- name: drf
enablePreemptable: false // 如果4.3.1.2 开启这里要设置为true
4.3.1.4 打开组调度：vocano.custom.scheduler_config_override tiers[0].plugins 里面查找如果没有就增加，关闭就就删除
- name:
enablePreemptable: false // 如果4.3.1.2 开启这里要设置为true
4.3.1.5 选择volcano调度策略
4.3.1.5.1 选择binpack：
第二plugins 里面增加 - name: binpack 删除 spread
4.3.1.5.2 spread：
第二plugins 里面增加 - name: spread 删除 binpack
4.3.1.6 打开重调度 vocano.custom.descheduler_enable 设置为true
4.3.1.6 打开在离线混部署 vocano.custom.colocation_enable 设置为true
增加labels [{nodename:"10-30-100-155","patch_labels":{"volcano.sh/colocation":"true"}}]
4.3.1.6 打开资源超卖 增加laebs [{nodename:"10-30-100-155","patch_labels":{"volcano.sh/colocation":"true","volcano.sh/oversubscription":"true"}}]
4.3.1.6 设置出口网络带宽 增加laebs [{nodename:"10-30-100-155","patch_labels":{"gpu_core_mode":"whole"，"volcano.sh/colocation":"true","volcano.sh/oversubscription":"true"}},"patch_annotations":{"volcano.sh/network-bandwidth-rate":1000}]
1.2 模式和插件参数说明
#模式和插件选择参数说明
#模式1：选择整卡模式
gpu_core_mode:whole
hami-dcgm:on
--------------------------------------
#模式2： 选择虚拟化模式
必填标签：
gpu_core_mode:hami-core
选填标签：
如果插件选择：hami-gpu
hami-gpu:on
gpu:on
hami-dcgm:on
volcano-gpu:null
如果插件选择：volcano-gpu
volcano-gpu:on
gpu:null
hami-dcgm:null
hami-gpu:null
----
------hami要修改的配置文件values中----devicePlugin.nodeConfiguration.config 是一个字符串---
{ // 多个节点需要多个对象组成数组并压缩成字符串
"nodeconfig": [
{
"name": "10-30-100-155",
"operatingmode": "hami-core",
"devicememoryscaling": 1.5,
"devicesplitcount": 10,
"migstrategy": "none",
"filterdevices": {
"uuid": [],
"index": []
},
}
]
}
------vocano要修改的配置文件values中----nodeConfig.configs 是一个yaml 对象数组---
nodeConfig:
enabled: true
configs:
- name: "10-30-100-155"
operatingmode: "hami-core"
devicememoryscaling: 1.5
devicesplitcount: 10
migstrategy: "none"
filterdevices:
uuid: []
index: []
----------------------------------------------------------------------------------------------------
#模式3：选择MIG模式
gpu_core_mode:mig
选填标签：
如果插件选择：hami-gpu
hami-gpu:on
gpu:on
hami-dcgm:on
volcano-gpu:null
如果插件选择：volcano-gpu
volcano-gpu:on
gpu:null
hami-dcgm:null
hami-gpu:null
------hami要修改的配置文件values中----devicePlugin.nodeConfiguration.config 是一个字符串---
{ // 多个节点需要多个对象组成数组并压缩成字符串
"nodeconfig": [
{
"name": "10-30-100-155",
"operatingmode": "hami-core",
"devicememoryscaling": 1.5,
"devicesplitcount": 10,
"migstrategy": "none",
"filterdevices": {
"uuid": [],
"index": []
},
}
]
}
------vocano要修改的配置文件values中----nodeConfig.configs 是一个yaml 对象数组---
nodeConfig:
enabled: true
configs:
- name: "10-30-100-155"
operatingmode: "hami-core"
devicememoryscaling: 1.5
devicesplitcount: 10
migstrategy: "none"
filterdevices:
uuid: []
index: []
---------------
device device-config.yaml 配置 volcano-vgpu-device-plugin/hami 下deviceConfig.nvidia.knownMigGeometries //举例
deviceConfig:
nvidia:
knownMigGeometries: // 数据来源于https://10.30.100.155:8443/v1/topke/node/gpu/migInstance
- models: ["A30"] // gpu_type 唯一值，有就修改，没有就新增，选择一个节点就选择一种类型 有就替换，没有就添加
allowedGeometries:
-geometries:
- name: 1g.6gb // Name
memory: 6144 // Memory
count: 4 // InstanceCount
----
二、volcano高级调度配置
#旧配置
apiVersion: v1
data:
volcano-scheduler.conf: |
actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: priority
- name: gang
enablePreemptable: false
- name: conformance
- plugins:
- name: overcommit
- name: drf
enablePreemptable: false
- name: predicates
- name: proportion
- name: nodeorder
- name: binpack
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: volcano
meta.helm.sh/release-namespace: topke-system
creationTimestamp: "2026-06-22T08:41:05Z"
labels:
app.kubernetes.io/managed-by: Helm
name: volcano-scheduler-configmap
namespace: topke-system
resourceVersion: "6480926"
uid: b3c2d7c6-cdad-4404-8d62-4b0afc67ccc2
---------------------
#新配置
apiVersion: v1
data:
volcano-scheduler.conf: |
actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: priority
- name: gang
enablePreemptable: true
- name: conformance
- plugins:
- name: overcommit
- name: drf
enablePreemptable: true
- name: predicates
- name: proportion
- name: nodeorder
- name: spread
- name: numa-aware
arguments:
weight: 10
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: volcano
meta.helm.sh/release-namespace: topke-system
topke.cluster.name: xggg
topke.cluster.uuid: 131b128d-5e11-43eb-91e4-8183b514a503
topke.createtime.unix: "1782117665"
topke.tenant.name: ken
topke.tenant.uuid: debe920e-5cc5-4e85-bd8b-a17cfc574602
creationTimestamp: "2026-06-22T08:41:05Z"
labels:
app.kubernetes.io/managed-by: Helm
name: volcano-scheduler-configmap
namespace: topke-system
resourceVersion: "6999765"
uid: b3c2d7c6-cdad-4404-8d62-4b0afc67ccc2
-----------------------------------
apiVersion: v1
data:
volcano-scheduler.conf: |
actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: priority
- name: gang
enablePreemptable: true
- name: conformance
- plugins:
- name: overcommit
- name: drf
enablePreemptable: true
- name: predicates
- name: proportion
- name: nodeorder
- name: numa-aware
arguments:
weight: 10
- name: binpack
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: volcano
meta.helm.sh/release-namespace: topke-system
topke.cluster.name: xggg
topke.cluster.uuid: 131b128d-5e11-43eb-91e4-8183b514a503
topke.createtime.unix: "1782117665"
topke.tenant.name: ken
topke.tenant.uuid: debe920e-5cc5-4e85-bd8b-a17cfc574602
creationTimestamp: "2026-06-22T08:41:05Z"
labels:
app.kubernetes.io/managed-by: Helm
name: volcano-scheduler-configmap
namespace: topke-system
resourceVersion: "7000997"
uid: b3c2d7c6-cdad-4404-8d62-4b0afc67ccc2
自动伸缩
cat << EOF > /tmp/autoscapimage.txt
registry.k8s.io/autoscaling/vpa-admission-controller:1.7.0
registry.k8s.io/ingress-nginx/kube-webhook-certgen:v20231011-8b53cabe0
registry.k8s.io/autoscaling/vpa-recommender:1.7.0
registry.k8s.io/autoscaling/vpa-updater:1.7.0
registry.aliyuncs.com/acs/kubernetes-cronhpa-controller:v1.4.3-2f290b2-aliyun
registry.k8s.io/metrics-server/metrics-server:v0.8.1
EOF
while read -r IMAGE; do
MY_IMAGE=$(echo "$IMAGE" | sed 's|^registry\.k8s\.io/|k8s\.flyingtang\.com/|')
FILE_NAME="${IMAGE##*/}"
FILE_NAME="${FILE_NAME/:/_}.tar"
nerdctl pull $MY_IMAGE
nerdctl tag $MY_IMAGE "10.30.13.175:5000/topke-system/$IMAGE"
nerdctl save "10.30.13.175:5000/topke-system/$IMAGE" | gzip > $FILE_NAME
done < /tmp/autoscapimage.txt
Roce-Link
cat << EOF > /tmp/autoscapimage.txt
ghcr.io/k8snetworkplumbingwg/sriov-network-operator:v1.6.0
ghcr.io/k8snetworkplumbingwg/sriov-network-operator-config-daemon:v1.6.0
ghcr.io/k8snetworkplumbingwg/sriov-cni:v2.10.0
ghcr.io/k8snetworkplumbingwg/ib-sriov-cni:v1.3.0
ghcr.io/k8snetworkplumbingwg/ovs-cni-plugin:v0.39.0
ghcr.io/k8snetworkplumbingwg/rdma-cni:v1.6.0
ghcr.io/k8snetworkplumbingwg/sriov-network-device-plugin:v3.11.0
ghcr.io/k8snetworkplumbingwg/network-resources-injector
ghcr.io/k8snetworkplumbingwg/sriov-network-operator-webhook
ghcr.io/k8snetworkplumbingwg/sriov-network-metrics-exporter
registry.k8s.io/kubebuilder/kube-rbac-proxy:v0.15.0
ghcr.io/k8snetworkplumbingwg/multus-cni:snapshot
ghcr.io/k8snetworkplumbingwg/whereabouts:v0.9.3
EOF
while read -r IMAGE; do
MY_IMAGE=$(echo "$IMAGE" | sed 's|^ghcr\.io/|ghcr\.flyingtang\.com/|')
FILE_NAME="${IMAGE##*/}"
FILE_NAME="${FILE_NAME/:/_}.tar"
nerdctl pull $MY_IMAGE
nerdctl tag $MY_IMAGE "10.30.13.175:5000/topke-system/$IMAGE"
nerdctl push "10.30.13.175:5000/topke-system/$IMAGE"
nerdctl save "10.30.13.175:5000/topke-system/$IMAGE" | gzip > $FILE_NAME
done < /tmp/autoscapimage.txt
while read -r IMAGE; do
MY_IMAGE=$(echo "$IMAGE" | sed 's|^registry\.k8s\.io/|k8s\.flyingtang\.com/|')
FILE_NAME="${IMAGE##*/}"
FILE_NAME="${FILE_NAME/:/_}.tar"
nerdctl pull $MY_IMAGE
nerdctl tag $MY_IMAGE "10.30.13.175:5000/topke-system/$IMAGE"
nerdctl push "10.30.13.175:5000/topke-system/$IMAGE"
nerdctl save "10.30.13.175:5000/topke-system/$IMAGE" | gzip > $FILE_NAME
done < /tmp/autoscapimage.txt
