Design and Analysis of activation functions used in deep learning models

Abstract

One of the important challenges in handling real world data is that most of the real-world data have complex data patterns. Another challenge in handling real world data is that data can be highly non-linear. An important element of neural network is an activation function. Activation functions help us to learn complex data patterns that coexist in real world datasets by facilitating us to introduce non-linearity. They help to decide whether a neural node under consideration must be activated or not. In this paper, an activation function is proposed which can be applied in neural networks to build better learning models.