Real-time contextual feedback for closed-loop control of navigation

PolyTouch: Tracking animal navigation for rapid, closed-loop control of brain and behavior

PolyTouch is an open-source software written in JAVA that enables animal tracking with real-time elementary behavioral classification while providing rapid feedback at millisecond resolution (average communication latency = 1ms). The software consists of a tracking and feedback module, where contact points are continuously tracked as computerized X, Y monitor screen coordinates. The user can run the software as a standalone program or call it in the MATLAB (Mathworks) environment. It can be deployed in conjunction with any touch input device, let it be a touch screen with a fixed screen resolution, mouse, touch pen or any interface that utilizes USB Touchscreen Controller (Universal), a driver. Commercially, IR sensor frames are sold as “IR touch over-frame”, “touch frames”, “multi-touch frames” by various manufacturers. Note that PolyTouch is not designed for notebook computers (laptops) with touch screen displays whose resolution can be adjusted with multi-touch gestures. This software is under continuous development.  For feature requests and to report bugs contact t.celikel@inavigate.eu

Close-loop control of neural activity will help to dissect the neural circuits of navigation.  However, real-time detection of behavioral events in temporal resolutions relevant to rapidly evolving behaviors is not a trivial task. In animal experiments, this is typically achieved by using sparsely distributed (embedded) sensors that detect animal presence in select regions of interest. High-speed cameras provide high-density sampling across large arenas, capturing the richness of animal behavior, however, the image processing bottleneck prohibits real-time feedback in the context of rapidly evolving behaviors. 

To overcome these limitations, we are developing an open-source software, named PolyTouch, to track animal behavior in large arenas and provide rapid closed-loop feedback in ~5.7 ms, ie. average latency from the detection of an event to analog stimulus delivery, e.g. auditory tone, TTL pulse, when tracking a single body. The first version of this stand-alone software is already released.  The software is written in JAVA. The included wrapper for MATLAB provides experimental flexibility for data acquisition, analysis, and visualization. The next generation of the software will identify and track multiple animals while providing closed-loop neural and behavioral feedback to one or many animals with a temporal resolution of ~2 ms.  PolyTouch is freely available online, and the paper describing the PolyTouch can be accessed freely on Biorxiv: https://www.biorxiv.org/content/10.1101/473108v3