A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones

Really interesting paper by Daniele Palossi on using GAP8 to autonomously navigate a microdrone. This is a great example of porting a significant CNN to GAP8. Eric Flamand, GreenWave’s CTO assisted with the CNN model creation and use of the AutoTiler CNN generators.

Abstract

Fully-autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI) based visual navigation capabilities, are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nano-drones with a size of a few cm2 . In this work, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on board resourceconstrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultra-low-power computing platform, and a 27 g commercial, open-source CrazyFlie 2.0 nano-quadrotor. As part of our general methodology, we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed aboard within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner, it achieves 18 fps while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

Conclusion

Nano- and pico-sized UAVs are ideal IoT nodes; due totheir size and physical footprint, they can act as mobile IoThubs, smart sensors and data collectors for tasks such assurveillance, inspection, etc. However, to be able to performthese tasks, they must be capable of autonomous navigationof environments such as urban streets, industrial facilities andother hazardous or otherwise challenging areas. In this work,we present a complete deployment methodology targeted at enabling execution of complex deep learning algorithms directlyaboard resource-constrained milliwatt-scale nodes. We providethe first (to the best of our knowledge) completely verticallyintegrated hardware/software visual navigation engine for autonomous nano-UAVs with completely onboard computation –and thus potentially able to operate in conditions in which thelatency or the additional power cost of a wirelessly-connectedcentralized solution.

Our system, based on a GREENWAVES Technologies GAP8 SoC used as an accelerator coupled with the STM32 MCU onthe CrazyFlie 2.0 nano-UAV, supports real-time computationof DroNet, an advanced CNN-based autonomous navigationalgorithm. Experimental results show a performance of 6 fps@ 64 mW selecting the most energy-efficient SoC configura-tion, that can scale up to 18 fps within an average power budgetfor computation of 284 mW. This is achieved without qualityof-results loss with respect to the baseline system on whichDroNet was deployed: a COTS standard-size UAV connectedwith a remote PC, on which the CNN was running at 20 fps.Our results show that both systems can detect obstacles fastenough to be able to safely fly at high speed, 4 m/s in the caseof the CrazyFlie 2.0. To further paving the way for a vastnumber of advanced use-cases of autonomous nano-UAVs asIoT-connected mobile smart sensors, we release open-sourceour PULP-Shield design and all code running on it, as wellas datasets and trained networks.

Detail paper click https://arxiv.org/pdf/1805.01831.