Free-living evaluation of laboratory-based activity classifiers in preschoolers

Machine learning classification models for accelerometer data are potentially more accurate methods to measure physical activity in young children than traditional cut point methods. However, existing algorithms have been trained on laboratory-based activity trials, and their performance has not been investigated under free-living conditions. PURPOSE: This study aimed to evaluate the accuracy of laboratory-trained hip and wrist random forest and support vector machine classifiers for the automatic recognition of five activity classes: sedentary (SED), light-intensity activities and games (LIGHT_AG), walking (WALK), running (RUN), and moderate to vigorous activities and games (MV_AG) in preschool-age children under free-living conditions. METHODS: Thirty-one children (4.0 ± 0.9 yr) were video recorded during a 20-min free-living play session while wearing an ActiGraph GT3X+ on their right hip and nondominant wrist. Direct observation was used to continuously code ground truth activity class and specific activity types occurring within each class using a bespoke two-stage coding scheme. Performance was assessed by calculating overall classification accuracy and extended confusion matrices summarizing class-level accuracy and the frequency of specific activities observed within each class. RESULTS: Accuracy values for the hip and wrist random forest algorithms were 69.4% and 59.1%, respectively. Accuracy values for hip and wrist support vector machine algorithms were 66.4% and 59.3%, respectively. Compared with the laboratory cross validation, accuracy decreased by 11%-15% for the hip classifiers and 19%-21% for the wrist classifiers. Classification accuracy values were 72%-78% for SED, 58%-79% for LIGHT_AG, 71%-84% for MV_AG, 9%-15% for WALK, and 66%-75% for RUN. CONCLUSION: The accuracy of laboratory-based activity classifiers for preschool-age children was attenuated when tested on new data collected under free-living conditions. Future studies should train and test machine learning activity recognition algorithms using accelerometer data collected under free-living conditions.