Sept. 14, 2014
Dude, where’s my interface?: How natural interactions and new media are helping us forget about the traditional interface
Abstract: Humans’ natural abilities have allowed us to interact with the surrounding environment in complex ways, from easily manipulating physical objects to communicating with others through speech. Yet when we interact with technology, we largely do so by ceding direct control and instead projecting our interactions in a virtual environment. As such, the holy grail of human-computer interaction remains that of enabling us to interact with technology and information in natural ways: touching the virtual representation of objects, speaking to virtual presences, and receiving feedback in same manners. It is now time to reach for that grail. While speech and multimodal interaction are often challenging computationally, I have shown in my research that, despite their inherent lack of accuracy, they can empower the users and allow them to interact in almost material ways with digital artefacts (such as automatically-generated transcripts). In this talk, I will first discuss how speech processing works, what are its limitations, where can it be used in interactive systems, and how can it be used to enhance current interaction paradigms. I will then argue that such natural interactions can enable humans to break the barriers of conventional interactions. To support this, I will present two examples of speech-based interactions: an immersive gaming environment for public safety training, in which users interact naturally and materially with virtual avatars through speech, and a mobile language assistant that empowers and increases the independence of low-literacy adults by enabling access to interactive digital dictionaries and language resources.
Bio: Cosmin Munteanu is an Assistant Professor at the Institute for Communication, Culture, Information, and Technology (University of Toronto at Mississauga), and Associate Director of the Technologies for Ageing Gracefully lab. Until 2014 he was a Research Officer with the National Research Council of Canada. His area of expertise is at the intersection of Human-Computer Interaction, Automatic Speech Recognition, Natural Language Processing, Mobile Computing, and Assistive Technologies. He has extensively studied the human factors of using imperfect speech recognition systems, and has designed and evaluated systems that improve humans’ access to and interaction with information-rich media and technologies through natural language.
Cosmin’s multidisciplinary interests include speech and natural language interaction for mobile devices, mixed reality systems, learning technologies for marginalized users, assistive technologies for older adults, and ethics in human-computer interaction research.