The brain dynamically transforms cognitive information. Here the authors build task-performing, functioning neural network models of sensorimotor transformations constrained by human brain data without the use of typical deep learning techniques.