Abstract
In recent years, new technology developments have been proposed and implemented to support people with hearing impairment and speech loss. It is a severe disability to be unable to speak and communicate. This group of people needs a device to help them use a smartphone in the same way as the rest of the population. This idea has pushed technologists to create new tools to help hearing-impaired and speechless people interact more intelligently and effectively with hearing-impaired and visually impaired people. These tools essentially combine all available new ideas behind speech-to-sign conversion, image processing, gesture extraction and sign-to-speech conversion techniques. Speech-to-sign conversion is processed using template-based recognition with peak modulation. In comparison, many algorithms convert speech-to-sign language, and the template-based peak modulated speech recognition method is superior over other methods, because it considers all components/inputs of speech. Since the highest and lowest peaks are calculated, the accuracy of the method is also high. Converted signs are captured, gesture extraction is performed, and gestures are converted to speech using tri-spectral gesture extraction. The proposed model has a very high accuracy of 98.4% and 98.8% in converting speech to sign and sign to speech, respectively, which is a significant difference from the existing neural network methods. These modules (speech to sign to speech) together compose a Triple-S algorithm that can also be used in wireless communication where hard-of-hearing and speechless people may involve themselves in remote communication as any other people would. The proposed algorithm has been trained and tested with outstanding results with 98.6% accuracy in effective communication through AI, where speech and sign recognition are combined.