Sistem Deteksi Bahasa Isyarat Alfabet Menggunakan Dataset American Sign Language (ASL) dan Algoritma Random Forest

Siti Farah Fakhirah; Muhammad Fillah Alfatih; Hasna Nabiilah Widiani; Thoriq Muhammad Pasya; Endang Purnama Giri; Gema Parasti Mindara

doi:10.59581/jusiik-widyakarya.v2i4.4321

Authors

Siti Farah Fakhirah Sekolah Vokasi IPB University
Muhammad Fillah Alfatih Sekolah Vokasi IPB University
Hasna Nabiilah Widiani Sekolah Vokasi IPB University
Thoriq Muhammad Pasya Sekolah Vokasi IPB University
Endang Purnama Giri IPB University
Gema Parasti Mindara Sekolah Vokasi IPB University

DOI:

https://doi.org/10.59581/jusiik-widyakarya.v2i4.4321

Keywords:

American Sign Language, Mediapipe, OpenCV, Random Forest, Image Processing

Abstract

Introducing alphabetical sign language is necessary to bridge communication between deaf and hard-of-hearing people and their surrounding environment. This research aims to develop a sign language alphabet letter detection system based on American Sign Language (ASL). The research methods include data collection, feature extraction with OpenCV and Mediapipe, model development with Random Forest algorithm, and real-time system testing. The test results show that the developed system can achieve 97% prediction accuracy in recognizing hand patterns that represent ASL letters. The system uses a webcam as real-time input, providing accurate responses in various environmental conditions. This research contributes significantly to developing communication support technology for the deaf community, with implications for increased inclusivity and social engagement.

References

Abhigna, K., Damisetty, R., & D N, V. K. (2024). Integrating OpenCV and Pandas for Enhanced Image Filtering and Color Detection. ResearchGate. https://doi.org/10.55041/IJSREM36213

Abidin, I. (2024). Penerapan Mediapipe dan Algoritma Random Forest Terhadap Sistem Penerjemah Bahasa Isyarat Sibi [Skripsi, Universitas Muhammadiyah Makassar]. Retrieved from https://digilibadmin.unismuh.ac.id/upload/42089-Full_Text.pdf

Alsharif, B., Alanazi, M., & Ilyas, M. (2023). Machine Learning Technology to Recognize American Sign Language Alphabet. 2023 IEEE 20th International Conference on Smart Communities: Improving Quality of Life Using AI, Robotics and IoT (HONET), 173–178. https://doi.org/10.1109/HONET59747.2023.10374964

Alviando, M. R., Al Rivan, M. E., & Yoannita, Y. (2020). Klasifikasi American Sign Language Menggunakan Fitur Scale Invariant Feature Transform dan Jaringan Saraf Tiruan. Jurnal Algoritme, 1(1), 1–11. https://doi.org/10.35957/algoritme.v1i1.403

Amalia, V. F., & Dewi, R. R. (2024). Penilaian Kesegaran Ikan dengan Metode K-Nearest Neighbor dan Pengolahan Citra Digital. JATI (Jurnal Mahasiswa Teknik Informatika), 8(4), Article 4. https://doi.org/10.36040/jati.v8i4.10441

Amiarrahman, M. R., & Handhika, T. (2018). Analisis dan Implementasi Algoritma Klasifikasi Random Forest Dalam Pengenalan Bahasa Isyarat Indonesia (BISINDO). In Prosiding SEMNAS INOTEK (Seminar Nasional Inovasi Teknologi), 2(1), 083–088. https://doi.org/10.29407/inotek.v2i1.461

Anjum, R., Sadaf, A. N., Sami, M., & Siddiqui, K. A. (2023). An Efficient Approach for Interpretation of Indian Sign Language using Machine Learning. International Journal of Innovative Research in Engineering & Multidisciplinary Physical Sciences (IJIRMPS), 11(1). https://doi.org/10.37082/IJIRMPS.v11.i1.230316

Bhushan, S., Alshehri, M., Keshta, I., Chakraverti, A. K., Rajpurohit, J., & Abugabah, A. (2022). An experimental analysis of various machine learning algorithms for hand gesture recognition. Electronics, 11(6), 968. https://doi.org/10.3390/electronics11060968

Budiman, S. N., Lestanti, S., Evvandri, S. M., & Putri, R. K. (2022). PENGENALAN GESTUR GERAKAN JARI UNTUK MENGONTROL VOLUME DI KOMPUTER MENGGUNAKAN LIBRARY OPENCV DAN MEDIAPIPE. Antivirus : Jurnal Ilmiah Teknik Informatika, 16(2), Article 2. https://doi.org/10.35457/antivirus.v16i2.2508

Jyothi, Mrs. N. K. (2024). Image Processing Model for Sign Language Recognition System. International Journal for Research in Applied Science and Engineering Technology, 12(5), 2235–2237. https://doi.org/10.22214/ijraset.2024.61671

Koleilat, A., Schimmenti, L. A., & Muthusamy, K. (2024). Hearing loss. In Neurogenetics for the Practitioner (pp. 305-325). Academic Press. https://doi.org/10.1016/B978-0-323-99417-0.00031-8

Mavi, A. (2021). A New Dataset and Proposed Convolutional Neural Network Architecture for Classification of American Sign Language Digits (No. arXiv:2011.08927). arXiv. https://doi.org/10.48550/arXiv.2011.08927

Mishra, S. R., Sahoo, J. P., & Sahoo, S. P. (2024). Visual Static Hand Gesture Recognition for ASL using Multi-Feature Fusion Technique. 2024 3rd International Conference on Artificial Intelligence For Internet of Things (AIIoT), 1–6. https://doi.org/10.1109/AIIoT58432.2024.10574668

Muthalib, M. A., Irfan, Kartika, & Meliala, S. (2023). PENGIRAAN POSE MODEL MANUSIA PADA REPETISI KEBUGARAN AI PEMOGRAMAN PYTHON BERBASIS KOMPUTERISASI. INFOTECH Journal, 9(1), 11–19. https://doi.org/10.31949/infotech.v9i1.4233

Piperakis, S., Papatsimouli, M., Argyriou, V., Sarigiannidis, P., & Fragulis, G. F. (2024). SignTrack: Advancements in Real-Time Sign Language Processing for Inclusive Computing with optimized AI. 2024 13th International Conference on Modern Circuits and Systems Technologies (MOCAST), 1–4. https://doi.org/10.1109/MOCAST61810.2024.10615749

Prabu, S., K, S. T., S, S., D, S., & J, R. (2024). Revolutionizing Communication: A Hybrid Deep Learning Framework for Enhanced Sign Language Recognition. 2024 International Conference on Data Science and Network Security (ICDSNS), 1–6. https://doi.org/10.1109/ICDSNS62112.2024.10690996

Reddy, A. (2023). Alphabet based Sign Language Translator for Two Way Communication. International Journal of Science and Research (IJSR), 12(8), 1427–1432. https://doi.org/10.21275/SR23815104133

Salman, H. A., Kalakech, A., & Steiti, A. (2024). Random Forest Algorithm Overview. Babylonian Journal of Machine Learning, 2024, 69–79. https://doi.org/10.58496/BJML/2024/007

Sheffield, A. M., & Smith, R. J. (2019). The epidemiology of deafness. Cold Spring Harbor perspectives in medicine, 9(9). https://doi.org/10.1101/cshperspect.a033258

Tama, N. S., & Novita, R. (2024). Visualization of Indonesian Sign Language System using Image Classification for Special Needs Students in Special Schools. In 2024 International Conference on Circuit, Systems and Communication (ICCSC), 6, 1–5. https://doi.org/10.1109/iccsc62074.2024.10616895

Widiastuti, T., Karsa, K., & Juliane, C. (2023). Evaluasi Tingkat Kepuasan Mahasiswa Terhadap Pelayanan Akademik Menggunakan Metode Klasifikasi Algoritma C4.5. Technomedia Journal, 7(3), 364–380. https://doi.org/10.33050/tmj.v7i3.1932

Wiraswendro, P. E., & Soetanto, H. (2022). Application of Random Forest Classifier Algorithm in Indonesian Sign Language System (Sibi) Detection System. Bit (Fakultas Teknologi Informasi Universitas Budi Luhur), 19(2), 75. https://doi.org/10.36080/bit.v19i2.2043

World Health Organization. (2024). Deafness and hearing loss. World Health Organization. Available at: https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss, diakses tanggal 28 November 2024.