[proposal] ElasticQuota supports GPU sharing

[ZiMengSheng]

What is your proposal:

ElasticQuota supports GPU sharing

Why is this needed:

When users declare ElasticQuota, they may specify resources such as koordinator.sh/gpu-memory-ratio as GPU quotas. However, when users apply for a full-GPU Pod, they may specify nvidia.com/gpu. In this case, ElasticQuota needs to be able to recognize both the pod's nvidia.com/gpu and koordinator.sh/gpu-memory-ratio applications and perform some internal conversion so that the GPU quota limit can be applied to both the full-GPU Pod and the split Pod.

Is there a suggested solution, if so, please add it:

[undertaker86001]

ElasticQuota GPU Sharing Architecture Design Proposal

1. Current Situation Analysis

Currently, the ElasticQuota plugin implements quota-based resource scheduling management but does not support unified handling of GPU resources. From the existing code, it can be seen that ElasticQuota uses a resource mask to handle quota restrictions for different resource types.

Meanwhile, Koordinator has already implemented a GPU resource transformation mechanism in its DeviceShare plugin, supporting the conversion of nvidia.com/gpu into koordinator.sh/gpu-core and koordinator.sh/gpu-memory-ratio.

2. Architecture Design Proposal

2.1 Core Design Concept

Add a GPU resource identification and conversion layer to the ElasticQuota plugin to uniformly handle both full GPU (nvidia.com/gpu) and GPU fragment (koordinator.sh/gpu-memory-ratio) resource requests, ensuring that quota restrictions are correctly applied to both resource types.

2.2 Detailed Design

2.2.1 Enhanced PodRequests Processing Logic

Extend the existing PodRequests function by adding GPU resource conversion capabilities:

During the PreFilter phase of ElasticQuota, add standardized GPU resource handling:

1. Resource Identification: Detect whether the Pod request includes nvidia.com/gpu or koordinator.sh/gpu-memory-ratio
2. Resource Conversion: Convert nvidia.com/gpu into a standardized GPU resource representation according to the DeviceShare plugin's rules
3. Quota Verification: Use the converted resource to perform quota limitation checks

2.2.2 QuotaInfo Resource Management Extension

Add GPU resource conversion and management capabilities to QuotaInfo:

Extend the NewQuotaInfoFromQuota function to support standardized processing for GPU resource quotas.

2.2.3 Resource Mask and Conversion Mechanism

Utilize the existing resource mask mechanism to ensure that GPU quotas only affect relevant resource types:

2.3 Implementation Details

2.3.1 GPU Resource Conversion Mapping

Refer to the resource conversion logic of the DeviceShare plugin:

Implement a similar conversion mechanism in ElasticQuota to ensure:

• 1 nvidia.com/gpu = 100 koordinator.sh/gpu-memory-ratio
• Correct handling of hybrid resource requests

2.3.2 Quota Restriction Application

Apply GPU quota restrictions during the PreFilter phase:

Enhance the quota checking logic to ensure:

• Both full GPU Pods and GPU fragment Pods are correctly subject to quota restrictions
• Accuracy of resource usage statistics
• Proper rejection when quotas are exceeded

2.3.3 Collaboration with DeviceShare Plugin

Ensure consistency between ElasticQuota’s GPU resource conversion and the DeviceShare plugin:

3. Implementation Steps

3.1 Phase One: Basic Conversion Mechanism

1. Add GPU resource conversion functions in the ElasticQuota core package
2. Modify PodRequests processing logic to support standardized GPU resources
3. Update QuotaInfo’s resource management logic

3.2 Phase Two: Enhanced Quota Verification

1. Enhance GPU resource quota checking during the PreFilter phase
2. Update resource mask processing to correctly handle GPU resource types
3. Ensure accuracy of resource statistics

3.3 Phase Three: Compatibility and Optimization

1. Optimize collaboration with the DeviceShare plugin
2. Ensure backward compatibility
3. Performance optimization and testing

4. Expected Outcomes

1. Unified Resource Management: ElasticQuota can uniformly manage both full GPU and GPU fragment resources
2. Precise Quota Control: Ensure GPU quota restrictions are correctly applied to both resource types
3. Seamless Compatibility: Existing GPU workloads can benefit from quota management without modification
4. Resource Utilization Improvement: Precise quota management improves GPU resource utilization efficiency

[undertaker86001]

@ZiMengSheng What do you think of this plan?

[qinfustu]

The nvidia.com/gpu and koordinator.sh/gpu-memory-ratio represent distinct resources and should inherently remain decoupled. While temporary compatibility layers may exist during solution transitions to support legacy configurations, long-term design must enforce complete separation. Maintaining intertwined logic imposes unsustainable complexity across all dependent systems (e.g., monitoring, scheduling), forcing users to invest significant effort in understanding conversion mechanics.​​

​​Recommendation​​:
​​Isolate resources at the reporting phase:​​
Nodes ​​with GPU partitioning enabled​​ exclusively report koordinator.sh/gpu-memory-ratio.
Nodes ​​with GPU partitioning disabled​​ exclusively report nvidia.com/gpu.

[ZiMengSheng]

@ZiMengSheng What do you think of this plan?

Currently, koordinator supports resource allocation for both full- and sharded Pods using the koordinator.sh/gpu-memory-ratio parameter. It only uses the "if memory-ratio < 100" check to determine whether to shard. Both sharded and full-node Pods can be allocated on the same node. From this perspective, the koordinator.sh/gpu-memory-ratio parameter is sufficient and does not require conversion.

So, i kind of agree more with @qinfustu now

[qinfustu]

Can we implement the following adjustments?

1. ​​Add a label node.koordinator.sh/gpu-shared="true" to the node​​ to indicate that the node's GPU sharing mode is enabled.

2. ​​koordlet detects the node.koordinator.sh/gpu-shared="true" label​​:

   • If the label exists, koordlet will ​​report​​ the resources koordinator.sh/gpu-core, koordinator.sh/gpu-memory, and koordinator.sh/gpu-memory-ratio, and ​​remove​​ the nvidia.com/gpu resource.

   • If the label ​​does not exist​​, koordlet will ​​only report​​ the nvidia.com/gpu resource.

3. ​​koord-manager detects the node.koordinator.sh/gpu-shared="true" label​​:

   • If the label exists, koord-manager will ​​convert​​ the GPU information from the devices into koordinator.sh/gpu and koordinator.sh/gpu.shared resources, and ​​synchronize​​ them to the node status.

4. ​Additionally​​: Cluster-wide installation of the nvidia-device-plugin ​​is not required​​.

[ZiMengSheng]

If the purpose is to prevent nvidia.com/gpu and koordinator.sh/gpu-memory-ratio from coexisting on the same node, then it is sufficient to not deploy koordlet on the node where nvidia DP is located?

[qinfustu]

If the purpose is to prevent nvidia.com/gpu and koordinator.sh/gpu-memory-ratio from coexisting on the same node, then it is sufficient to not deploy koordlet on the node where nvidia DP is located?

koordlet is required when utilizing load-aware and fine-grained scheduling capabilities. Can koordlet fully replace the NVIDIA DP?

[ZiMengSheng]

Now, it can