Dual-Task Multimodal Web Browsing in Semi-Autonomous Vehicles

Abstract

Drivers should remain alert when operating semi-autonomous vehicles as human intervention is required when the automated systems fail, or an unusual situation is encountered. However, drivers do not always focus solely on the road and sometimes divert their attention to look at cell phones, laptops, tablet computers, or in-dash infotainment systems. In this research, software was created to enable operators of a simulated self-driving car to browse the Human-Centered Design Institute (HCDI) website using multimodal interaction. The software used synthesized speech to read the website aloud and visually highlighted the content on the screen as it was read. Web analytics were also used to optimize the reading order of the website content. The audio modality of the software was tested in a single-task experiment. The final software prototype with audible and visual output was tested by drivers performing a dual-task in a driving simulator. Workload, usability, and emotion were measured after using the software. Usability increased and workload decreased with added visualization. Driver’s looked at the screen more often when highlighting was added to the screen, but the duration of the glances was shorter. The results suggest the software creates more task switching but allows drivers to shorten the duration of their screen glances when browsing the HCDI website in a simulated semi-autonomous vehicle.