Comparative performance analysis of three machine learning algorithms applied to sensor data registered by a leg-attached accelerometer to predict metritis events in dairy cattle

Abstract

Routinely collected sensor data could be used in metritis predictive modeling but a better understanding of its potential is needed. Our objectives were 1) to compare the performance of k-nearest neighbors (k-NN), random forest (RF), and support vector machine (SVM) classifiers on the detection of behavioral patterns associated with metritis events measured by a leg-attached accelerometer (TrackaCow, ENGS, Hampshire, UK); 2) to study the impact of farm scheduled activities on model performance; and 3) to identify which behaviors yield the highest F1 score for metritis prediction as a function of the number of time window and time-lags. A total of 239 metritis events (188 non-metritis and 51 metritis events) were retrospectively created based on changes in two consecutive uterine evaluations from a dataset containing sensor and clinical data during the first 21 days postpartum between June 2014 and May 2017. These events were associated with a total of 10,874 - 14,138 data points corresponding to hourly measurements of lying time, lying bouts, steps, intake, and intake visits. Sensor data corresponding to the 3 days before each metritis event were aggregated every 24-, 12-, 6-, and 3-hour time windows. Multiple time-lags were also used to determine the optimal number of past observations needed for optimal classification. Similarly, different decision thresholds were compared. Depending on the classifier, algorithm hyperparameters were optimized using grid search (RF, k-NN, SVM) and random search (RF). All behaviors changed throughout the study period and showed distinct daily patterns. From the three algorithms, RF had the highest F1 score, with no impact of scheduled farm activities on classifier performance. Furthermore, 3- and 6-hour time windows had the best balance between F1 scores and number of time-lags. We concluded that steps and lying time can be used to predict metritis using data from 2 to 3 days before a metritis event. Findings from this study will be used to develop more complex prediction models that could identify cows at higher risk of experiencing metritis.