MotorNet
MotorNet is an open-source Python toolbox for creating arbitrarily complex, differentiable, and biomechanically realistic effectors that can be trained on user-defined motor tasks using ANNs.
Paper
Codol O, Michaels JA, Kashefi M, Pruszynski JA, Gribble PL (2023) MotorNet: a Python toolbox for controlling differentiable biomechanical effectors with artificial neural networks. bioRxiv 2023.02.17.528969
GitHub repository
https://github.com/OlivierCodol/MotorNet
Documentation
https://oliviercodol.github.io/MotorNet/build/html/index.html
Abstract
Artificial neural networks (ANNs) are a powerful class of computational models for unravelling neural mechanisms of brain function. However, for neural control of movement, they currently must be integrated with software simulating biomechanical effectors, leading to limiting impracticalities: (1) researchers must rely on two different platforms and (2) biomechanical effectors are not generally differentiable, constraining researchers to reinforcement learning algorithms despite the existence and potential biological relevance of faster training methods. To address these limitations, we developed MotorNet, an open-source Python toolbox for creating arbitrarily complex, differentiable, and biomechanically realistic effectors that can be trained on user-defined motor tasks using ANNs. MotorNet is designed to meet several goals: ease of installation, ease of use, a high-level user-friendly API, and a modular architecture to allow for flexibility in model building. MotorNet requires no dependencies outside Python, making it easy to get started with. For instance, it allows training ANNs on typically used motor control models such as a two joint, six muscle, planar arm within minutes on a typical desktop computer. MotorNet is built on TensorFlow and therefore can implement any network architecture that is possible using the TensorFlow framework. Consequently, it will immediately benefit from advances in artificial intelligence through TensorFlow updates. Finally, it is open source, enabling users to create and share their own improvements, such as new effector and network architectures or custom task designs. MotorNet's focus on higher order model and task design will alleviate overhead cost to initiate computational projects for new researchers by providing a standalone, ready-to-go framework, and speed up efforts of established computational teams by enabling a focus on concepts and ideas over implementation.
Selected Figures
Figure 1: Controlling an arm-like effector in a centre-out reaching task with a curl field. (a) Schematic of the effector and controller. (b) Endpoint trajectories of centre-out reaching movements in a null and curl field, for an RNN controller that is untrained (naive) and then trained to reach in that curl field. The effector was as defined in Kistemaker et al. (2010) (c) Different variables over time during a rightward reaching movement.
Figure 2: Conceptual organization of a MotorNet model. An ANN receives arbitrary input as well as recurrent connections from itself, and sends motor command outputs to an effector, which in turn sends sensory feedback information. Typically, this feedback will be visual and proprioceptive, and can contain feedback-specific time delays Δp and Δv. Gaussian noise can be added to the recurrent connection, motor commands, and proprioceptive and visual feedback, with specific standard deviation σh, σ u, σ p, and σ v
Figure 3: Implementation of MotorNet. (a) Information flow of a MotorNet model during runtime. (b) Declarative structure of a MotorNet object. Each object instance is held in memory as an attribute of another according to this hierarchical representation, except for the Muscle, and Skeleton instances
Figure 6: Replicating Lillicrap & Scott (2013)—The distribution of preferential movement direction tuning is sensitive to the geometry of the plant. (a) Schematic of the two models compared. The RNNs and their architecture were identical, but the plant differed, with one RNN controlling a two-joint arm26 (left) and the other controlling a point-mass (right). (b) Centre-out reaching trajectories to 24 targets for the arm26 (left) and point-mass (right) model. (c) Distribution of preferential movement directions (PMDs) for the arm26 (left) and point-mass (right) model. The PMDs were determined by regression of each GRU’s hidden activity averaged over time against reach angle (see Methods for details). (d) Normalized muscle activations across reaching angles and for the 300 ms following the “go” cue for the arm26 model. (e) Normalized β coefficients of the regression models used for (c). The GRUs were ordered according to the angle of their maximum β value. Note that the “ridge” of maximum β yields roughly a straight line for the point-mass model, while it yields a crooked line for the arm26, indicative of a representation bias. (f) Hidden activity over time and across reaching angles for a random sample of GRUs in the arm26 model.