Implementasi Mediapipe Holistic dengan Artificial Neural Network untuk Pengenalan Kata dalam Sistem Isyarat Bahasa Indonesia

Ahmad, Fayyadh (2024) Implementasi Mediapipe Holistic dengan Artificial Neural Network untuk Pengenalan Kata dalam Sistem Isyarat Bahasa Indonesia. Undergraduate Thesis thesis, Institut Teknologi Telkom Purwokerto.

	Text Cover.pdf Download (1MB)
	Text Abstrak.pdf Download (177kB)
	Text Abstract.pdf Download (175kB)
	Text BAB I.pdf Download (250kB)
	Text BAB II.pdf Download (765kB)
	Text BAB III.pdf Download (1MB)
	Text BAB IV.pdf Restricted to Registered users only Download (1MB)
	Text BAB V.pdf Download (240kB)
	Text Daftar Pustaka.pdf Download (253kB)
	Text Lampiran.pdf Restricted to Registered users only Download (294kB)

Abstract

Communication is an important part of human social life because it involves the exchange of messages both verbal and non-verbal. For individuals with hearing and speech impairment, the challenges in communicating can be overcome through sign language. This research aims to develop a word recognition model in the Indonesian Sign Language System (SIBI) to help communicate between individuals with hearing impairment and speaking to normal. Although a number of previous studies focused on the recognition of alphabetic letters in the Indonesia Sign Language (BISINDO) and SIBI, there are still shortcomings in the understanding of vocabulary in both sign language systems. The word recognition method used involved video image data of sign language that was converted into image image and then extracted its features through Mediapipe, then converted to tabular data with hundreds of features and 21 labels and then classified with one of the Artificial Neural Network (ANN) architectures named Multi Layer Perceptron. (MLP). The results of this study show that MLP using 8 layer parameters, 1072 neurons (225-200-175-150-125- 100-75-22), ReLU function in hidden layer, learning rate 0.001, 16 batch size, 100 epochs, adam optimizer, and sparse categorical crossentropy function obtained an accuracy score of 97.95% and a loss of 0.0598. However, the model experienced a decrease in performance when predicting new video data indicating overfitting. This is shown from a model that got an average accuracy score of 58%. Keywords: Word Recognition, Indonesian Sign Language System, Mediapipe, Artficial Neural Network

Actions (login required)

View Item