Eslambolchilar, Parisa and Murray-Smith, Roderick (2006) Model-Based, Multimodal Interaction in Document Browsing. In: Machine Learning for Multimodal Interaction. Third InternationalWorkshop, MLMI 2006 Bethesda, MD, USA, May 1-4, 2006 Revised Selected Papers. Lecture Notes in Computer Science (LNCS), 4299/2006 . Springer Berlin / Heidelberg, pp. 1-12. ISBN 978-3-540-69267-6

Download (400kB)

Abstract

In this paper we introduce a dynamic system approach to the design of multimodal interactive systems. We use an example where we support human behavior in browsing a document, by adapting the dynamics of navigation and the visual feedback (using a focus-in-context (F+C) method) to support the current inferred task. We also demonstrate non-speech audio feedback, based on a language model. We argue that to design interaction we need models of key aspects of the process, here for example, we need models for the dynamic system, language model and sonification. We show how the user's intention is coupled to the visualization technique via the dynamic model, and how the focus-in-context method couples details in context to audio samples via the language identifcation system. We present probabilistic audio feedback as an example of a multimodal approach to sensing different languages in a multilingual text. This general approach is well suited to mobile and wearable applications, and shared displays.

Item Type:	Book Section
Additional Information:	The original publication is available at http://www.springerlink.com/content/xn316362050640w7/fulltext.pdf
Keywords:	Multimodal Interaction; Document Browsing; Continuous interaction; User behavioral models; Language identification system; Model-based interactive method; LNCS; MLMI’06; Hamilton Institute.
Academic Unit:	Faculty of Science and Engineering > Computer Science Faculty of Science and Engineering > Research Institutes > Hamilton Institute
Item ID:	1774
Identification Number:	https://doi.org/10.1007/11965152_1
Depositing User:	Hamilton Editor
Date Deposited:	11 Jan 2010 14:49
Publisher:	Springer Berlin / Heidelberg
Refereed:	No
URI:	Publisher
Use Licence:	This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here

Repository Staff Only(login required)

Item control page

Downloads

Origin of downloads