Local competition among neighboring neurons is common in biological neural networks (NNs). In this paper, we apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.
|Original language||English (US)|
|Title of host publication||Advances in Neural Information Processing Systems|
|Publisher||Neural information processing systems foundation|
|State||Published - Jan 1 2013|