[DisaggregatedSet] Add field to scale disaggregatedSet unit

[Edwinhr716]

What would you like to be added: Add a new field to make it possible to scale DisaggregatedSet unit

type DisaggregatedSetSpec struct {
	// +optional
	// +kubebuilder:default=1
	Slices *int32 `json:"replicas,omitempty"`
}

Why is this needed: Analogous to LWS replica, we need a way to scale the unit of DisaggregatedSet to deploy the same P/D combination multiple times, instead of needing to deploy a new yaml for the same combination.

Completion requirements:

This enhancement requires the following artifacts:

• Design doc
• API change
• Docs update

The artifacts should be linked in subsequent comments.

[yankay]

Strongly +1 — scaling a DisaggregatedSet as a unit feels like a very natural extension of the LWS Replicas model.

Should we wait until #836 (and the legacy-folder cleanup tracked in #789) lands before implementing this? The slice semantics will touch the planner, executor, workload and service managers introduced by #836, so it feels safer to build on a stable base. WDYT?

[hasB4K]

I think we should wait for #836 at the very least yes. Also, we could have floating replicas field. Let's say that the perfect ratio is 7:9 (7 Prefill, 9 Decode). Scaling up using an Integer a single unit doesn't make a lot of sense. BUT, if we'll use a floating replicas, it magically works:

  ┌──────────┬──────────────┬─────────────┬────────────┐
  │ replicas │ Prefill (×7) │ Decode (×9) │ Total Pods │
  ├──────────┼──────────────┼─────────────┼────────────┤
  │ 0.5      │ 3.5 → 4      │ 4.5 → 5     │ 9          │
  ├──────────┼──────────────┼─────────────┼────────────┤
  │ 0.75     │ 5.25 → 5     │ 6.75 → 7    │ 12         │
  ├──────────┼──────────────┼─────────────┼────────────┤
  │ 1.0      │ 7            │ 9           │ 16         │
  ├──────────┼──────────────┼─────────────┼────────────┤
  │ 1.5      │ 10.5 → 11    │ 13.5 → 14   │ 25         │
  ├──────────┼──────────────┼─────────────┼────────────┤
  │ 2.0      │ 14           │ 18          │ 32         │
  └──────────┴──────────────┴─────────────┴────────────┘

Also, we could simplify the planner.go logic (that I feel too complex), by just handling scales.
But I think this work should be done in a second stage.

[Edwinhr716]

Scaling up using an Integer a single unit doesn't make a lot of sense

Can you expand on this? I think it makes sense for features like #848. Also, I don't think there's a precedent in Kubernetes for having a floating point value for replicas. cc @ahg-g

[hasB4K]

Can you expand on this? I think it makes sense for features like #848. Also, I don't think there's a precedent in Kubernetes for having a floating point value for replicas. cc @ahg-g

Well, apparently we were not talking about the same thing. But let me explain.

Disaggregated Deployment are usually quite big, but the right P:D ratio is hard to find (a topic outside of this issue). So you can imagine having a nice P:D ratio of 11:19 or whatever, that can be quite big if you are deploying this on a small deployment. Scaling up with a fixed integer is not great, because you can quickly exhaust the number of nodes that have GPUs.

Let's take this example, you actually have a 11p19d. Let's say that Prefill and Decode take only one node for simplicity of this example. So you are using 30 nodes for this setup. You want to increase the entire setup by 25%, but you can't because DisaggregatedSet doesn't have a floating scaling factor, but a fixed integer replicas. So you'll end up scaling Prefill and Decode roles replicas manually instead of just doing something like this:

  spec:
    scalingFactor: 1.25
    roles:
    - name: prefill
      spec:
        replicas: 11  # effective: ceil(11 * 1.25) = 14
        leaderWorkerTemplate: { ... }
    - name: decode
      spec:
        replicas: 19  # effective: ceil(19 * 1.25) = 24

That said, I agree there's no k8s precedent for float replicas, and it would conflict with topology-aware placement (#848) which needs discrete slice. So maybe integer replicas is the right approach and an additional scaling factor could be explored later as a separate mechanism. I guess we could combine both later (since it's not mutually exclusive):

  spec:
    slices: 2           # 2 independent units (for placement policies)
    scalingFactor: 1.25  # each unit scales its role replicas by 1.25x
    roles:
    - name: prefill
      spec:
        replicas: 11  # per unit: ceil(11 * 1.25) = 14, total: 14 * 2 = 28
        leaderWorkerTemplate: { ... }
    - name: decode
      spec:
        replicas: 19  # per unit: ceil(19 * 1.25) = 24, total: 24 * 2 = 48
        leaderWorkerTemplate: { ... }
  # Total: 76 nodes across 2 topology domains

So in that context slices would be how many copies (integer, for topology placement per #848).
And scalingFactor would how big each copy is.

The DisaggregatedSetSpec would look like this:

type DisaggregatedSetSpec struct {
        // Slices is the number of independent units of the DisaggregatedSet.
        // Each slice creates a full set of role workloads (e.g., prefill + decode).
        // Used for topology-aware placement (#848).
        // +optional
        // +kubebuilder:default=1
        Slices *int32 `json:"replicas,omitempty"`

        // ScalingFactor multiplies each role's replicas within a slice.
        // Enables fine-grained scaling without changing the P:D ratio.
        // Effective replicas = ceil(role.replicas * scalingFactor).
        // +optional
        // +kubebuilder:default="1.0"
        ScalingFactor *resource.Quantity `json:"scalingFactor,omitempty"`

        // Roles defines the list of roles (at least 2 required).
        // +listType=map
        // +listMapKey=name
        // +kubebuilder:validation:MinItems=2
        // +kubebuilder:validation:MaxItems=10
        // +required
        Roles []DisaggregatedRoleSpec `json:"roles"`
  }

But this is probably over-engineered for now. Integer slices first, scaling factor later if needed.

[BenjaminBraunDev]

/assign @BenjaminBraunDev