Header Ad Placement

Rethinking MoE Load Balancing as Feedback Control: Why a Pure Integral Bias Beats Aux-Loss and Loss-Free Under a Mid-Run Data Shift

ARCHITECTURE

Mixture-of-Experts balancers were tuned on stationary data, but real training switches mixtures. A new paper recasts the auxiliary loss and the loss-free bias update as members of one PID family, argues that only the integral term both clears persistent imbalance and ignores batch noise, and tests it through a hard domain switch.

By Ossam Hashim

KEY FINDINGS

  • Treating each expert's deviation from its fair share as a control error and its routing bias as the control input, the paper defines a PID-Balancer family. The auxiliary balancing loss is read as a proportional mechanism acting through gate-weight gradients, with a steady-state error that shrinks as its coefficient grows but never vanishes. The loss-free bias update (ALF-LB) is read as a relay integrator acting on the sign of the error. The authors flag the first reading as an interpretation rather than an exact instance of their update equation.
  • The analysis rests on a split of the load error into a slow part (an expert above or below its share for many steps) and a fast part (step-to-step fluctuation from each batch being different). A dead expert, with error fixed at 1, makes the integral term ramp by the gain every step until the expert re-enters selection, while the proportional term supplies a constant offset that may never close the gap. Under an error alternating in sign every step, the integral bias stays within a narrow band while proportional and derivative biases flip with the error. The authors therefore adopt PID (I), the pure-integral member with a single gain.
  • Pretraining results come from OLMoE-style models with 16, 32 and 64 experts at constant one-eighth activation, trained 33,000 steps with one hard switch at step 16,500 to GitHub alone, six seeds each, over three pre-shift mixtures. PID (I) posts the lowest post-shift MaxVio in every setting. At 64 experts in the Books-to-GitHub case: 0.40 for PID (I), 0.80 for ALF-LB, 1.27 for the auxiliary loss and 6.08 for a no-balancing baseline. Perplexity stays comparable across methods, within about 0.03 to 0.05 in the pretraining table.
  • Continual pretraining of the released OLMoE-1B-7B on code (codeparrot) reaches post-shift MaxVio of 0.90 cold-start and 0.87 warm-start for PID (I), against about 1.0 for ALF-LB, 4.4 to 4.5 for the auxiliary loss and 5.8 for no balancing. Balance is not free: after continual pretraining, perplexity on the original annealing corpus rises from 5.46 to 6.07-7.39, and MMLU drops from 0.55 to 0.46-0.51 across methods, with the balancers at the lower end.
  • The authors' own limits are plain: largest model 6.9B total and 1.3B active parameters, three expert counts, a single hard domain switch, one backbone with a modifiable router, and an untested question of whether the margin over ALF-LB continues to widen at production scale.

Balancers Tuned on Stationary Data

Every expert-balancing method in common use was evaluated on a fixed data mixture. Training does not stay on one. Curriculum stages, continual pretraining into a new domain and corpus refreshes all change which tokens the router sees, and the load it had converged to stops being fair. The paper treats this as a disturbance to a control loop: the measured error is each expert's deviation from its ideal share, normalized so it is dimensionless and bounded, the actuator is a per-expert bias added to the router scores before top-K selection (applied with a one-step delay and affecting only which experts are chosen, not the gate weight), and the router is a memoryless, monotone map from bias to load. That framing puts the auxiliary loss, the loss-free bias update and a family of untried alternatives on one axis for the first time.

Two Famous Balancers, One Family

The auxiliary loss works through gradients: working through the softmax Jacobian, the correction on a token's logit for an expert is proportional to that expert's error relative to the token's softmax-weighted mean error, with no memory of earlier steps. The paper reads that as proportional action, and its test is the steady state: at a stationary point the auxiliary gradient has to cancel a persistent language-modelling gradient that favours imbalance, which leaves a nonzero error scaling as one over the coefficient. An error that shrinks with the gain but never disappears is the signature of proportional control. The loss-free method keeps a bias per expert and nudges it by a fixed step in the direction of the error's sign, which is integral action on the sign alone. It stops drifting only when the error is positive as often as negative, regardless of magnitude, so integral action survives but the sign relay weakens the guarantee. The paper notes that the original loss-free authors also tried a linear update on a sigmoid-gated model at one scale and reported better balance at slightly higher perplexity.

Why Only the Integral Term

The argument is deliberately small. A persistent error such as a dead expert holds the error at 1: the integral term turns that into a ramp that eventually beats any score gap, the proportional term gives a fixed offset that may fall short, and the derivative term sees no change at all. In the opposite extreme, an error that flips sign every step, the integral term cancels it and its bias barely moves, while proportional and derivative biases flip with the error. Because the bias acts one step late, chasing that fast fluctuation is wasted motion. The paper argues the integral term is the only one with both properties, and its appendix experiments find that adding proportional or derivative terms to the per-step error does not beat the integral term alone. The authors do not rule out that a smoothed or proxy error would let the other terms earn their place; they name it the most promising direction for the full family.

What the Shift Experiments Show

Pretraining runs use a SlimPajama subset split into seven source domains, three scales (16, 32, 64 experts) with constant sparsity, and a hard switch at the midpoint to GitHub alone, from Books alone, from six domains without GitHub, or from all seven. Balance is scored with MaxVio (relative overload of the busiest expert) plus two tail metrics, the dead-expert and under-trained-expert ratios, because MaxVio alone can look fine while the tail starves. In every scale and scenario, all three balancers beat the no-balancing baseline by a wide margin and leave essentially no dead experts; the auxiliary loss leaves a few under-trained ones in some runs; ALF-LB falls further behind PID (I) as expert count grows; PID (I) is lowest before and after the shift. The pattern in the post-shift table is monotone: at 16 experts the gap between PID (I) and ALF-LB is about 0.03 to 0.08, at 64 experts PID (I) sits at 0.40 to 0.47 versus 0.80 to 1.20, with standard deviations of 0.01 to 0.20 across seeds. The paper declines to call the growth a firm trend from three points.

Balance Has a Price in Continual Pretraining

Starting from OLMoE-1B-7B, the authors continue training on code under two start conditions: straight onto the code corpus, or first on the annealing corpus so the balancer reaches steady state before the switch. Post-shift MaxVio tells the same story, with PID (I) at 0.90 and 0.87 and the unbalanced baseline near 5.8. Task scores tell a more mixed one. Held-out perplexity on the new corpus is flat across methods, but on the annealing corpus it rises sharply from the base checkpoint's 5.46 to 6.07-7.39, and MMLU falls from 0.55 to 0.46-0.51, HellaSwag from 0.79 to 0.71-0.77. The bias-based methods sit at the weak end of those drops, and the authors report that raising the auxiliary coefficient buys balance only at a cost in perplexity and accuracy. The paper also discusses a specialization trade-off: bias methods override the gate's domain preference only at selection time, while gate weights themselves keep most of it.

Assumptions and limitations

All results are the authors' own, on OLMoE-architecture models up to 6.9B total and 1.3B active parameters, three expert counts, a single hard switch between two domains and one backbone whose router can be modified, so transfer to expert-choice routing, capacity-limited routers or production scale is unmeasured. Gradual drift and repeated switches are not covered. The proportional reading of the auxiliary loss is the authors' interpretation, not an exact instance of their update equation, and the integral-only conclusion is conditional on the per-step error definition they chose. In continual pretraining the balancers lose ground on the old corpus and on downstream scores relative to the base checkpoint; the relation of that loss to balance, rather than to the new data itself, is not isolated here. The worked example is an invented toy with illustrative gains; all MaxVio, perplexity and benchmark figures above are the authors' measurements, not this blog's.

Worked Example: Integral Versus Proportional

The toy below uses a single expert and a gain of 0.1 for both terms. First it feeds an error that flips sign every step for 40 steps and records the range of each controller's bias; the proportional bias swings across 0.2 while the integral bias stays within 0.1. Then it holds a dead expert's error at 1 for ten steps: the integral bias ramps to 1.0 while the proportional bias stays at 0.1. Invented gains and errors, illustrative only; not the paper's code or measurements.

import math

kp, ki = 0.1, 0.1
ui = 0.0
lo_p, hi_p = 9, -9
lo_i, hi_i = 9, -9
for t in range(1, 41):
    e = -1 if t % 2 else 1
    ui += ki * e
    up = kp * e
    lo_p = min(lo_p, up)
    hi_p = max(hi_p, up)
    lo_i = min(lo_i, ui)
    hi_i = max(hi_i, ui)
sw = hi_p - lo_p
print('P swing', round(sw, 2))
sw = hi_i - lo_i
print('I swing', round(sw, 2))

dead = [ki] * 10
ud = math.fsum(dead)
print('dead, I bias', round(ud, 2))
print('dead, P bias', kp)

Sources & Provenance

  • Rethinking MoE Load Balancing as Feedback Control (9 Nov 2026) - arXiv abstract
  • Full paper (PDF): PID-Balancer definition, load-error analysis, post-shift tables, continual-pretraining results, limitations - arXiv full text

All figures are attributed to the preprint as published. Sources accessed 9 November 2026. This analysis is independent editorial content for GlobalTechDigest.

Comments

Kubernetes Liveness Is Not Readiness: Why a Health Check Can Amplify an Outage

KUBERNETES FAILURE RECOVERY A failed health check is an observation. Restarting a container and withholding traffic are different responses, and choosing the wrong one can amplify an outage. By Hossam The important question is what failure should trigger A web application loses access to its shared database. Its processes are still running, but requests that require database access fail. A health endpoint reports the dependency problem. Should Kubernetes restart every application container, or stop directing those requests to it? The endpoint alone cannot choose the right recovery action. Kubernetes separates startup, liveness and readiness probes because they answer different operational questions. Startup protects initialization, liveness can trigger container termination and restart handling, and readiness controls eligibility for traffic from matching Services. A green check is meaningful only within the check and response that were designed. This is a documentation-based de...

Editorial Governance & Pages