Classification of Indonesian Sign Language Using Convolutional Neural Network

Abstract

Indonesian Sign Language is word signs initially taken from the signs conveyed by deaf children. Sign language is common for the deaf and mute, but it is no stranger to ordinary people. For this reason, alternative intermediaries are needed who can become translators between deaf and speech impaired sufferers and ordinary people. This study aims to classify the Indonesian sign system using the Convolutional Neural Network method with VGG-16 and Alexnet architecture. The data divided by each letter from the letter A to the letter Z is 320 test data, 1600 train data, and 320 validation data, and the data will be resized to a size of 224 x 224 pixels, followed by grayscale and augmentation. The results of the VGG-16 test show that the classification using VGG-16 with the Adam optimizer gets the highest level of accuracy, which is 99.32% for each letter, 91.18% for the whole. While the classification results using VGG-16 with the SGD optimizer get the lowest level of accuracy, which is 98.85% for each letter and 84.96% for the whole. Meanwhile, from the AlexNet test results, it can be seen that the results of the classification using AlexNet with the Adam optimizer get the highest level of accuracy, which is 99.16% for each letter and 89.04% for the whole. While the classification results using AlexNet with the SGD optimizer get the lowest level of accuracy, which is 97.33% for each letter and 68.33% for the whole.