Performance evaluation in video-surveillance systems: The EventVideo project evaluation protocols

Abstract

During recent years, automatic video-surveillance systems have experienced a great development driven by the growing need for security. Many approaches exist whose performance is not clear for a large variety of available scenarios. To precisely identify which ones operate better for each scenario, empirical performance evaluation has been widely used for determining their strengths and weaknesses through their results. This approach requires defining two aspects (usually named as the evaluation protocol): the dataset (representative sequences) and the metrics (performance estimators). Common empirical approaches use metrics based on ground-truth data that define an ideal result, but there are also some novel approaches that do not require such data. Furthermore, the existence of several metrics and the growing availability of video data increase the complexity of the protocol design as well as require us to automate the whole evaluation process. In this chapter, considering the main analysis stages of a typical video-surveillance system (video object segmentation, people detection, video object tracking and event recognition), we introduce their evaluation protocols within the scope of the EventVideo project.