How humans initiate energy optimization and converge on their optimal gaits

Abstract

A central principle in motor control is that the coordination strategies learned by our nervous system are often optimal. Here, we combined human experiments with computational reinforcement learning models to study how the nervous system navigates possible movements to arrive at an optimal coordination. Our experiments used robotic exoskeletons to reshape the relationship between how participants walk and how much energy they consume. We found that while some participants used their relatively high natural gait variability to explore the new energetic landscape and spontaneously initiate energy optimization, most participants preferred to exploit their originally preferred, but now suboptimal, gait. We could nevertheless reliably initiate optimization in these exploiters by providing them with the experience of lower cost gaits, suggesting that the nervous system benefits from cues about the relevant dimensions along which to reoptimize its coordination. Once optimization was initiated, we found that the nervous system employed a local search process to converge on the new optimum gait over tens of seconds. Once optimization was completed, the nervous system learned to predict this new optimal gait and rapidly returned to it within a few steps if perturbed away. We then used our data to develop reinforcement learning models that can predict experimental behaviours, and applied these models to inductively reason about how the nervous system optimizes coordination. We conclude that the nervous system optimizes for energy using a prediction of the optimal gait, and then refines this prediction with the cost of each new walking step.