Experimental evaluation of auditory cognition's effects on visual cognition of video

Abstract

This paper presents the experimental evaluation of auditory cognition's effects on visual cognition of video. The influences of seven auditory stimuli on visual recognition are investigated based on experimental data of key-down operations. The key-down operations for locating a moving target by visual and auditory images are monitored by an experiment system originally made by devices including VTR, CRT, Data Recorder, etc.. Regression analysis and EM algorithm are applied to analyzing the experiment data of 350 key-down operations, made with 50 people and 7 auditory stimulus types. The following characteristic results about the influence of auditory stimulus on visual recognition are derived. Firstly, seven people responded too early for every experiment. The average of and the standard deviation of their response times are 439[ms] and 231[ms] respectively. Secondly, the other forty three people responded about 10[ms] after at cases, in which auditory images were presented 30[ms] or 60[ms] before visual images. Also they responded about 10[ms] early at the other cases. Thirdly, as the visual image was dominant information used for the key-down decision making, apparent effects of auditory images on the key-down operation were not measured. Averages and standard deviations of distributions measured by EM algorithm, regarding to 7 auditory stimulus types, are considered and verified with the Card's MHP model of human response. © 2009 The Institute of Electrical Engineers of Japan.