Air Traffic Controller's Behavior Recognition from Still Images by Transfer Learning

Abstract

The fatigue problem of air traffic controllers is one of the important reasons that en-danger the safety of civil aviation. In this paper, we introduce behavior recognition into the field of fatigue monitoring of civil air traffic controllers. We investigate the method of automatic recognition of controller's behaviors by using transfer learning and model fusion. The task of behavior recognition for controllers can be regarded as a multi-classification problem (Such as putting heads on desks for a nap, lay with face upward, nod to sleep, talking with others, directing planes normally, Hold up chin on left hand, writing notes on strips radio check, Resting hand on forehead ). Our study solve this problem from three aspects: (1) Set up a data set of controller behavior, including 18247 pictures of 10 actions; (2) Employ two-stream CNN(Resnet50,Inception V3) to extract different features; (3) We investigate different fusion strategy to improve the accuracy of action recognition of controllers. Experiments demonstrate that our method achieves a recognition accuracy of 94.2% and 93.7% on the CAFUC-Controller and Kaggle-Driving datasets.