Mahidol University Logo
Faculty of ICT, Mahidol University
 

Admissions

 

SIGN2SOUND

PiyanuchEangpayung5188036ITCS/B
PongsakornLertkiatmongkol5188070ITCS/B
RaksitPolohakul5188149ITCS/B
NatdanaiSrithongkul5188221ITCS/B

B.Sc.(INFORMATION AND COMMUNICATION TECHNOLOGY)

Project Advisor: Asst .Prof. Dr.Sukanya Phongsuphap

Abstract

Up until now, there are problems when common people communicate withdisability people who are deaf- mutes. In this project, Thai sign language has been studied and we developed an application that can translate the gestures or movements into words and voices. Gesture Recognition and Sound Synthesis are the techniques used to develop this application. UsingKinectas a camera sensor, developed by Microsoft, it can detect any part of human body movement such as arms, legs, feet, headand etc. It can also detect in the form of depth image and skeleton. Moreover, this application was integrated with the VajaAPI, developed by NECTEC, which can be used to synthesize voices from words.

There are two main parts in Sign2Sound,Sign language recognition and Voice synthesis. For the former part, we use theKinect Sensor to find depth information of the user, extract joint positions and store in the joint collections. Upon completion, all of the joint data will be used to compare with rules in the rule-base system to find the related meaning for the corresponding words. In the latter part, we use Vaja API to receive words from Sign language recognition and synthesize them into voices. Finally, the performance has been evaluated by repeatedly running the application and record the statistic of the translated sign language. If an accuracy to translate a word is more than eighty percent, the result is guaranteed to be satisfied.

As a result, we implemented an application that can be used to translate Thai sign Language and produce Thai speech.

 

KEYWORDS: SIGN LANGUAGE / GESTURE RECOGNITION / VAJA API / KINECT / CAMERA SENSOR / SPEECH SYNTHESES

 

Go to Top

 

ICT Building, Mahidol University, 999 Phuttamonthon 4 Road, Salaya, Nakhonpathom 73170 Tel. +66 02 441-0909 Fax. +66 02 849-6099
Mahidol University Computing Center, The Faculty of ICT, Mahidol University , Rama 6 Road, Rajathevi, Bangkok 10400 Tel. +66 02 354-4333 Fax. +66 02 354-7333