Learning Hysteretic Routing Policies for Heterogeneous Queues with Switching Costs

Routing jobs efficiently in heterogeneous server pools requires balancing queue performance against the cost of switching servers between active and idle states. Recent actor–critic methods learn effective threshold routing policies for heterogeneous queues without model knowledge, but optimise queue occupancy alone and treat switching as cost-free. We propose HACHQ, which augments the learning objective with explicit per-event switching penalties and replaces the singlethreshold policy with a two-threshold hysteresis parameterisation. The dead zone between the two thresholds suppresses unnecessary switching, while the switching-cost-aware objective adapts the dead-zone width to the cost magnitude. On two-server systems, HACHQ attains nearoptimal cost, within one percent of the exact hysteretic optimum computed by relative value iteration, while reducing switching. On multiserver systems the gains grow with scale, reaching up to 44% reduction in total cost and 74% reduction in switching rate, scaling monotonically with switching-cost magnitude. We analyse the mechanism that produces these gains.