An investigation into non-verbal sound-based modes of human-to-computer communication with rehabilitation applications

During the course of our work in the National Rehabilitation Hospital in Ireland we have encountered patients who, while unable to speak, are capable of making reproducible utterances. We present techniques used to harness such utterances, in addition to whistling, as a means of communication and control. A simple technique for identifying the phonemes /o/ and /s/ (in single-symbol ARPAbet notation) is presented with applications. The use of pitch variation as a means of controlling a continuously variable parameter is described with two applications - a microcontroller based light dimmer switch and a computer program which facilitates mouse pointer control. Finally, a technique for the recognition of short note sequences is presented. A program is described which allows arbitrary commands to be executed in response to tunes either sung or whistled by the user. These commands may be used to switch on or off electrical appliances in the home