Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning
International Journal of Advanced Computer Science and Applications · مج 14 (2) · 2023
الملخّص
A convolutional neural network (CNN) is a subset of machine learning as well as one of the different types of artificial neural networks that are used for different applications and data types. Activation functions (AFs) are used in this type of network to determine whether or not its neurons are activated. One non-linear AF named as Rectified Linear Units (ReLU) which involves a simple mathematical operations and it gives better performance. It avoids rectifying vanishing gradient problem that inherents older AFs like tanh and sigmoid. Additionally, it has less computational cost. Despite these advantages, it suffers from a problem called Dying problem. Several modifications have been appeared to address this problem, for example; Leaky ReLU (LReLU). The main concept of our algorithm is to improve the current LReLU activation functions in mitigating the dying problem on deep learning by using the readjustment of values (changing and decreasing value) of the loss function or cost function while number of epochs are increased. The model was trained on the MNIST dataset with 20 epochs and achieved lowest misclassification rate by 1.2%. While optimizing our proposed methods, we received comparatively better results in terms of simplicity, low computational cost, and with no hyperparameters.
التفاصيل
- المجلة
- International Journal of Advanced Computer Science and Applications
- الناشر
- Science and Information Organization
- المجلد
- 14
- العدد
- 2
- الصفحات
- —
- الترقيم الدولي ISSN
- 2156-5570
- السنة
- 2023
- اللغة
- en
- الترخيص
- cc-by
- الوصول المفتوح
- diamond
- الاستشهادات
- 4
- المصدر
- scopus
الكلمات المفتاحية
الموضوعات
Neural Networks and Applications (Computer Science) · Advanced Neural Network Applications (Computer Science) · Stochastic Gradient Optimization Techniques (Computer Science)
استشهد بهذا
Atoum, I. A. (2023). Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning. International Journal of Advanced Computer Science and Applications, 14(2). https://doi.org/10.14569/ijacsa.2023.0140212
Atoum, I. A. 2023, 'Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning', International Journal of Advanced Computer Science and Applications, vol. 14(2). Available at: https://doi.org/10.14569/ijacsa.2023.0140212
Atoum IA. Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning. International Journal of Advanced Computer Science and Applications. 2023;14(2).
@article{atoum20232089,
title = {Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning},
author = {Ibrahim A. Atoum},
year = {2023},
journal = {International Journal of Advanced Computer Science and Applications},
volume = {14},
number = {2},
publisher = {Science and Information Organization},
doi = {10.14569/ijacsa.2023.0140212},
url = {http://dx.doi.org/10.14569/ijacsa.2023.0140212}
}TY - JOUR AU - Ibrahim A. Atoum TI - Adaptive Rectified Linear Unit (Arelu) for Classification Problems to Solve Dying Problem in Deep Learning PY - 2023 JO - International Journal of Advanced Computer Science and Applications VL - 14 IS - 2 DO - 10.14569/ijacsa.2023.0140212 UR - http://dx.doi.org/10.14569/ijacsa.2023.0140212 ER -
