Towards Diverse Media Augmented E-Book Reader Platform

In order to leverage the use of various modalities such as audio-visual-touch in instilling learning behaviour, we present an intuitive approach of annotation based hapto-audio-visual interaction with the traditional digital learning materials such as eBooks. By integrating the traditional home entertainment system and respective media in the user's reading experience combined with haptic interfaces, we examine whether such augmentation of modalities influence the user's reading experience in terms of attention, entertainment and retention. The proposed Haptic E-Book (HE-Book) system leverages the haptic jacket, haptic arm band as well as haptic sofa interfaces to receive haptic emotive signals wirelessly in the form of patterned vibrations of the actuators and expresses the learning material by incorporating audio-video based augmentation in order to pave ways for intimate reading experience in the popular eBook platform. We have designed and developed desktop, mobile/tablet based HE-Book system as well as a semi-automated annotation authoring tool. Our system also supports multimedia based diverse quiz augmentations, which can help in learning tracking. We have conducted quantitative and qualitative tests using the developed prototype systems. We have adopted the indirect objective based performance analysis methodology, which is commonly used for multimedia based learning investigation. The user study shows that, there is a positive tendency of accepting multimodal interactions including haptics with traditional eBook reading experience. Though our limited number of laboratory tests reveal, that haptics can be an influencing media in eBook reading experience, but it requires large scale real life tests to provide a concluding remarks