Adaptive Neural Network Scaling Guided by Contrastive Loss

Abstract

Automatically scaling the capacity of artificial neural networks (ANNs) to match task difficulty still lacks a low-cost and robust general solution: most existing methods rely on prediction loss, which yields an unpredictable width-performance landscape and unstable search. Inspired by continuous neurogenesis in the dentate gyrus, we propose to use contrastive loss as a proxy for pattern-separation quality and to let it drive neuron generation or pruning. We propose a one-dimensional “exponential expansion + golden-section refinement” algorithm: network width is doubled while contrastive loss keeps decreasing and is trimmed once the loss rises, taking advantage of the markedly smoother contrastive curve. On eight public benchmarks, the resulting search procedure (Contra) attains a mean prediction error of 8.94%±7.13%, about 25% lower than the same search guided by prediction loss (Pred; 11.91%±11.82%), and reduces run-to-run variance by 40-50% on small, noisy tabular tasks. The contrastive loss-width curve is 54% smoother than that of plain prediction loss, yet overall wall time overhead is negligible because fewer probes are required. These findings demonstrate that contrastive loss offers a stable control signal for capacity, yielding an immediately “plug-and-play” upgrade for width, channel- embedding-dimension search in dynamic networks and neural-architecture search.