ML vs DL: Accuracy and Testing Runtime Trade-offs in BCI

Abstract

Trade-offs between accuracy and runtime are a common phenomenon in the field of computer science, but particularly pose a challenge for online Brain-computer interface (BCI) applications, such as control interfaces for paralysis patients. However, research evaluating the testing runtime of various classifiers is extremely limited. In this study, we assess trade-offs between accuracy and runtime (total and testing) of the classifiers benchmarked in Kastrati et al.’s introduction of EEGEyeNet, as well as sLDA, logistic regression, variants of SVM, and a shallow CNN. For simple BCI tasks requiring binary classification, we find that both simple and ensemble ML algorithms, especially tree-based models, can achieve accuracies comparable to DL networks’ while achieving remarkably faster total and testing runtimes. Namely, DecisionTree, RandomForest, and GradientBoost were particularly impressive, and we consider these highly efficient classifiers to be promising machine learning alternatives to slower deep learning classifiers such as CNN in binary motor imagery classification.