Abstract
Most of existing benchmarking tools for service robots are basically qualitative, in which a robot’s performance on a task is evaluated based on completion/incompletion of actions contained in the task. In the effort reported in this paper, we tried to implement a synthetical benchmarking system on domestic mobile platforms. Synthetical benchmarking consists of both qualitative and quantitative aspects, such as task completion, accuracy of task completions and efficiency of task completions, about performance of a robot. The system includes a set of algorithms for collecting, recording and analyzing measurement data from a MoCap system. It was used as the evaluator in a competition called the BSR challenge, in which 10 teams participated, at RoboCup 2015. The paper presents our motivations behind synthetical benchmarking, the design considerations on the synthetical benchmarking system, the realization of the competition as a comparative study on performance evaluation of domestic mobile platforms, and an analysis of the teams’ performance.
Cite
CITATION STYLE
Cheng, M., Chen, X., Tang, K., Wu, F., Kupcsik, A., Iocchi, L., … Hsu, D. (2015). Synthetical benchmarking of service robots: A first effort on domestic mobile platforms. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 9513, pp. 377–388). Springer Verlag. https://doi.org/10.1007/978-3-319-29339-4_32
Register to see more suggestions
Mendeley helps you to discover research relevant for your work.
