Cornell University researchers have created an interface that allows users to handwrite and sketch within computer code—a challenge to conventional coding, which typically relies on typing.

The pen-based interface, called Notate, lets users of computational, digital notebooks open drawing canvases and handwrite diagrams within lines of traditional, digitized computer code.

Powered by a deep learning model, the interface bridges handwritten and textual programming contexts: notation in the handwritten diagram can reference textual code and vice versa. For instance, Notate recognizes handwritten programming symbols, like “n”, and then links them up to their typewritten equivalents.