Multiple scales pilot action pattern recognition during flight task using video surveillance

Abstract

Pilot action analysis is one of the most important aspects in objective measurement of flight crew workload. Pilot action patterns include distribution of action area, action duration, time and space interval between each of the actions, body posture, cross actions, action path and operation procedures. There are two main problems of pilot action pattern recognition. One is the issue of multiple scales in both time and space. The other is to locate action accurately. This paper presents a method of analyzing pilot action patterns during flight tasks by using video surveillance. Video data is obtained by setting up a single camera in real flight mission. We consider the cockpit as an intelligent environment and develop a new method to analyze pilot action patterns. The concept of intelligent environment is included. First, vision based pattern recognition method is used to locate moving targets in video. Then we use logical based method to recognition actions. The experiment results show that our approach is effective in workload assessment.