Using Modality Replacement to Facilitate Communication between Visually and Hearing-Impaired People

Using sign language, speech, and haptics as communication modalities, a virtual treasure-hunting game serves as an entertainment and educational tool for visually and hearing impaired users. In recent years, there has been an increasing interest in human-computer interaction (HCI) for multimodal interfaces. Since Sutherland's SketchPad in 1961 and Xerox's Alto in 1973, computer users have long been acquainted with technol ogies other than the traditional keyboard for interacting with a system. Recently, with the desire for increased productivity, seamless in teraction, immersion, and e-inclusion of people with disabilities, along with progress in fields such as multimedia, multimodal signal analy sis, and HCI, multimodal interaction has emerged as an active field of research