|Deep Leaky Single-Peaked Triangle Neural Networks
Chuanhui Shan, Xirong Guo*, and Jun Ou
International Journal of Control, Automation, and Systems, vol. 17, no. 10, pp.2693-2701, 2019
Abstract : Recently, Deep learning has made a great deal of success in processing images, audios, and natural languages and so on. The activation function is one of the key factors in Deep learning. In this paper, according to characteristics of biological neurons, an improved Leaky Single-Peaked Triangle Linear Unit (LSPTLU) activation function is presented for the right-hand response unbounded of Rectified Linear Unit (ReLU) and Leaky ReLU(LReLU). LSPTLU is more in line with the biological neuron essence and achieves the excellent performance of equivalent or beyond ReLU and LReLU on different datsets, e.g., MNIST, Fashion-MNIST, SVHN, IMAGENET, CALTECH101 and CIFAR10 datasets.
Keyword : Comparative experiment, leaky single-peaked triangle linear unit (LSPTLU), LSPTLU network, LSPTLU neuron.