A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones

时间： 2024-09-30 来源：GREENWAVES Official Website

技术问答

Really interesting paper by Daniele Palossi on using GAP8 to autonomously navigate a microdrone. This is a great example of porting a significant CNN to GAP8. Eric Flamand, GreenWave’s CTO assisted with the CNN model creation and use of the AutoTiler CNN generators.

Abstract

Fully-autonomous miniaturized robots (e.g., drones), with artificial intelligence (AI) based visual navigation capabilities, are extremely challenging drivers of Internet-of-Things edge intelligence capabilities. Visual navigation based on AI approaches, such as deep neural networks (DNNs) are becoming pervasive for standard-size drones, but are considered out of reach for nano-drones with a size of a few cm2 . In this work, we present the first (to the best of our knowledge) demonstration of a navigation engine for autonomous nano-drones capable of closed-loop end-to-end DNN-based visual navigation. To achieve this goal we developed a complete methodology for parallel execution of complex DNNs directly on board resourceconstrained milliwatt-scale nodes. Our system is based on GAP8, a novel parallel ultra-low-power computing platform, and a 27 g commercial, open-source CrazyFlie 2.0 nano-quadrotor. As part of our general methodology, we discuss the software mapping techniques that enable the state-of-the-art deep convolutional neural network presented in [1] to be fully executed aboard within a strict 6 fps real-time constraint with no compromise in terms of flight results, while all processing is done with only 64 mW on average. Our navigation engine is flexible and can be used to span a wide performance range: at its peak performance corner, it achieves 18 fps while still consuming on average just 3.5% of the power envelope of the deployed nano-aircraft. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.

Conclusion

Nano- and pico-sized UAVs are ideal IoT nodes; due totheir size and physical footprint, they can act as mobile IoThubs, smart sensors and data collectors for tasks such assurveillance, inspection, etc. However, to be able to performthese tasks, they must be capable of autonomous navigationof environments such as urban streets, industrial facilities andother hazardous or otherwise challenging areas. In this work,we present a complete deployment methodology targeted at enabling execution of complex deep learning algorithms directlyaboard resource-constrained milliwatt-scale nodes. We providethe first (to the best of our knowledge) completely verticallyintegrated hardware/software visual navigation engine for autonomous nano-UAVs with completely onboard computation –and thus potentially able to operate in conditions in which thelatency or the additional power cost of a wirelessly-connectedcentralized solution.

Our system, based on a GREENWAVES Technologies GAP8 SoC used as an accelerator coupled with the STM32 MCU onthe CrazyFlie 2.0 nano-UAV, supports real-time computationof DroNet, an advanced CNN-based autonomous navigationalgorithm. Experimental results show a performance of 6 fps@ 64 mW selecting the most energy-efficient SoC configura-tion, that can scale up to 18 fps within an average power budgetfor computation of 284 mW. This is achieved without qualityof-results loss with respect to the baseline system on whichDroNet was deployed: a COTS standard-size UAV connectedwith a remote PC, on which the CNN was running at 20 fps.Our results show that both systems can detect obstacles fastenough to be able to safely fly at high speed, 4 m/s in the caseof the CrazyFlie 2.0. To further paving the way for a vastnumber of advanced use-cases of autonomous nano-UAVs asIoT-connected mobile smart sensors, we release open-sourceour PULP-Shield design and all code running on it, as wellas datasets and trained networks.

Detail paper click https://arxiv.org/pdf/1805.01831.

发送到邮箱 |
+1 赞 0
收藏
评论 0
| 转发至：

本文由JWM转载自GREENWAVES Official Website，原文标题为:A 64mW DNN-based Visual Navigation Engine for Autonomous Nano-Drones，本站所有转载文章系出于传递更多信息之目的，且明确注明来源，不希望被转载的媒体或个人可与我们联系，我们将立即进行删除处理。

全部评论（0）

暂无评论

GAP-8: A RISC-V SoC for AI at the Edge of the IoT

Current ultra-low power smart sensing edge devices, operating for years on small batteries, are limited to low-bandwidth sensors, such as temperature or pressure. Enabling the next generation of edge devices to process data from richer sensors such as image, video, audio, or multi-axial motion/vibration has huge application potential.

应用方案发布时间 : 2024-08-10

Enabling ANC in Open-Ear Earbuds Beyond Apple with GreenWaves GAP9 AI+DSP Processor

By uniquely meeting this sub-millisecond latency requirement, the ultra-low-latency shared memory AI+DSP processor GAP9 enables brands, beyond Apple, to implement active noise cancellation (ANC) in open-ear designs.

应用方案发布时间 : 2024-10-22

GreenWaves Building Occupancy Management Solution Using the TensorFlow Object Detection API

This paper mainly introduces GreenWaves‘ introduction of a head count solution for intelligent building system occupancy management, and introduces its application scenarios and product advantages.

应用方案发布时间 : 2024-09-12

纳芯微电子（NOVOSENSE）信号感知芯片/隔离与接口芯片/驱动与采样芯片选型指南（英文）

描述- NOVOSENSE Microelectronics (NOVOSENSE, SSE Stock Code 688052) is a highly robust & reliable analog and mixed signal Chip company. Since its establishment in 2013, the company has been focusing on sensor, signal chain, and power management, providing comprehensive semiconductor products and solutions, which are widely used in automotive, industrial, information communication and consumer electronic

型号- NSI6602NA-DLAR,NSL21924-Q1,NCA1051,NSI6602VC-Q1SWR,NSI6622NB-Q1SWKR,NCA1051N-Q1SPR,NSM2017,NSI6602NC-Q1SWR,NSM2019,NSM211X,NSI6602A-Q1SWKR,NSM2013,NIRS485,NSIP83086,NSI6622VB-Q1SPOR,NSM2015,NSM2016,NSC9260X,NSHT30-CLAR,NSD1624-Q1SPR,NSM2011,NSM2012,NSI8231,NSI8230,NSD2621C-DQAGR,NSR33XXX,NSC9262,NSC9260,NSI6801LC-DDBR,NSI6602VD-Q1SWKR,NSC9264,NSP1830,NSG65N15K-DQAFR,NST112X,NCA1042C-Q1DNR,NCA1042,NCA1042A-Q1DNHR,NSI8220NX,NSI6622A-DSWKR,NSI6801TB-DDBR,NSD1624-DSPKR,NSP1833,NST7719,NSP1831,NSP1832,NSM1052,NSM1053,NSI8222,NSA2860X,NSI8221,NSI8220,NSE5701 SERIES,NSI6642C-DSPNR,NSM1051,NSI822XC,NSIP8921W1-DSWR,NSI6602MNF-Q1SWTR,NSD11416-Q1STBR,NSI6602VB-Q1SPNR,NCA3491,NSI6642A-DLAR,NSI6602NB-Q1SWKR,NSA2860_TSSOP,NSD1026V,NSI8221CX-DSWR,NSE34050,NCA1043B-Q1DNKR,NCA1044N-Q1SPR,NSIP8944W1-DSWR,NSL21912-Q1,NSM201X,NSI6622A-Q1SPNR,NCA1051N-Q1,NSM2031,NSI6602MNB-Q1SWTR,NSD7310A-DHSPR,NSM2032,NCA1145-Q1SPKR,NSM2033,NSI8210,NSM2034,NST112-DSTR,NSI6601MB-DSPR,NSD1015T,NSI8266CX-DSWR,NSI6602VD-DSWKR,NSI8222CX-DSPR,NSM105X SERIES,NSD1015MT-DSPR,NSC2860X_DQNR,NCA1021S-Q1SPR,NSI8262CX-DSWR,NSPGS2,NCA34XX,NCA3485,NCA1051C,NSI68011C-DSWAR,NSI8221NX,NSI6602NC-DLAMR,NST1002,NSPGS5,NSM211X SERIES,NSI6602VC-DLAMR,NST1001,NCA1042C-Q1,NSI6602A-DLAR,NSI6602ND-DSWKR,NSM105X,NSD16241-Q1SPR,NSI1303M0X,NSM3011,NSPAS3M SERIES,NSI8240CX-DSPR,NSD7312-Q1,NSM3013,NSM3012,NSA2860_SSOP16,NSA2862X,NSI6642B-DSWKR,NSM1071,NSM1072,NSI823XC,NSI6642A-DLAMR,NSD5604N,NSI319X,NSI6642D-DLAR,NSI8100NC,NSI6602B-Q1SWR,NSI6622ND-DSPNR,NSI6601B-DSPR,NSA2860X-QQNR,NSI8210NX,NSI6642D-DSWKR,NSI6602MF-DSWTR,NSD5604E,NIRSP31,NSI6602MNB-DSWTR,NST175,NST1075,NSI6602A-DSPNR,NCA9617A SERIES,NSI68010B-Q1SWAR,NSI1312D,NSI6642B-DSWR,NSI8222NX,NSI6601C-DSPR,NSI6622NB-DLAR,NSIP8841W1-DSWR,NSP1631,NSI8221CX-DSWVR,NSP1632,NSP1630,NSD56008-Q1HTSPR,NSI6622C-DLAR,NSM301X,NSM107X,NSI824XC,NSI6602VA-Q1SPNR,NSI6602ND-DLAR,NSI6801C-DSWFR,NSD3604-Q1,NCA3176,NSI6801MB-DSWVR,NSD3608-Q1QAJR,NSI1312S,NSI6622VB-Q1SWR,NSI6622B-Q1SPNR,NSD5604N-DHTSPR,NSD1624-DLAJR,NSI68011C-Q1SWAR,NSI6602VA-DLAMR,NSI6602NA-DLAMR,NSI6602NB-DSWR,NSD5604E-DHTSTR,NSR35XXX,NSI6622NB-Q1SWR,NSD5604N-Q1HTSPR,NSD1224X,NSI6602C-Q1SPNR,NSI8100W,NSI6622NA-DSPNR,NSIP8844W1-DSWR,NSI6622NB-DSWKR,NSE5702,NSE5701,NSI8100N,NSP183X,NCA1044-Q1DNR,NSI6622NC-DLAR,NCA1021S-Q1DNR,NSI6622VB-Q1SPNR,NCA1044N-Q1DNR,NSD16241-DSPKR,NSI3190,NSI6602VD-DSWR,NPC060N120A-QTOOT,NSI6622NA-Q1SWKR,NSE34050Q,NSD1224LA-DAFR,NSI8241SX,NSI6622NA-Q1SWR,NSI6622NC-Q1SPNR,NSI8231CX-DSWR,NSI1052-DSWR,NSPGL1 SERIES,NSI6622VA-Q1SPOR,NSA2860X_QQNR,NSI6801B-DSWFR,NSI6602B-Q1SPNR,NSI6601WC-DSWVR,NSI6651ASC-Q1SWR,NSC6272,NSI8263CX-DSWR,NSD8310-Q1HTSXR,NSC6273,NSREF31XX,NSI6622ND-DSWR,NSI6622NC-DSPNR,NSI6602NB-DLAR,NSI6602NB-DSPNR,NSA5312,NSI6622VC-Q1SWKR,NSPGL1,NSI6602A-Q1SWR,NSI6622NC-Q1SPOR,NSPGS2 SERIES,NSI6642C-DLAMR,NSR7808,NSI8220WX,NSI8266WX,NSD56008-Q1,NSD5604NE,NSI6622VA-Q1SPNR,NSI6602VB-DSWKR,NSL2161X-Q1,NCA1042BN-Q1SPR,NCA1057-Q1SPR,NCA1043B-Q1,NCA9511,NSI6602A-DSWR,NSC6280,NSI6602NA-Q1SWKR,NSD8308-Q1,NSI6602NC-DSWR,NSI6801TC-DDBR,NSI8242SX,NSI68515AC-DSWR,NSI6602B-DSWKR,NSI6602VD-DLAMR,NPCO60N120A-QTOIT,NSI8220CX-DSWVR,NSI6622NB-DSWR,NST235,NST117,NSI6602VC-DSWKR,NSI6602VC-Q1SWKR,NSI6602MB-DSWDR,NST118,NCA954X SERIES,NSD16242-Q1SPR,NCA9306 SERIES,NCA1044-Q1SPR,NSI6651,NSI6642B-DLAMR,NSI6602C-DSWKR,NSI8240NX,NSE34050 SERIES,NCA1057N-Q1DNR,NSP163X,NSI6602NC-DSPNR,NCA1042BN-Q1,NSI8221WX,NSI66X2,NSI6631ASC-Q1SWR,NSI6602MNC-DSWTR,NCA1042B-Q1DNR,NSI6642,NSI6601C-DSWVR,NIRS2X,NSI6601MB-DSWR,NSI8266SX,NSI6602NB-Q1SPOR,NCA34XX SERIES,NSI6602NC-Q1SPOR,NSD7310A,NSD8306-Q1,NSI6622ND-DLAMR,NSG65N15K,NSD12409-Q1SPR,NSE4250 SERIES,NCA1042CN-DSPR,NSI6602VA-DSWKR,NSI6642D-DLAMR,NCA1051N,NSI6642A-DSWKR,NIRS20N1-DSPR,NIRS21N1-DSPR,NSI6622NA-DLAR,NSC2860X-DQNR,NSI1042-DSWVR,NCA1042C-Q1SPR,NSI6602C-DSWR,NCA1057-Q1DNR,NSI8221CX-DSPR,NPD010N120A-DTOGT,NSI6602VB-Q1SWR,NSI8210CX-DSPR,NSD1026V-QISPR,NSD12416‒Q1,NSI1050-DDBR,NSI8241NX,NSI6602NC-Q1SPNR,NCA1042BN-Q1DNR,NSR10AXX SERIES,NST461,NSD2621X,NCA1042A-Q1,NSI8222WX,NST103,NSI1303D0X,NSI1303E2X,NSI6622A-Q1SWKR,NSI6602VC-DLAR,NSR31 SERIES,NSD16242-DSPR,NCA9555,NSI1052,NSC6360,NSC6362,NSI6622NB-DLAMR,NSI6651ALC-DSWR,NSC6364,NSI6602NA-DSPNR,NSI6622A-DLAR,NSI6622C-Q1SPNR,NCA1042C,NSI8220CX-DSWR,NSI6622C-DSPNR,NSA3166,NCA1042A-Q1SPR,NSIP8941W0-DSWR,NSI6642B-DLAR,NCA1042B-Q1,NSPGD1M,NSPAS3 SERIES,NSD1624-DSPR,NSI6602VA-Q1SWR,NSD1026V-Q1HMSR,NSI8242CX-DSWR,NSI6602MNC-Q1SWTR,NSL2163X-Q1,NST20,NIRS31,NSREF30XX,NCA9545,NSI6622NA-DLAMR,NSI1042,NSI6611,NCA9546,NSI6622B-DSPNR,NCA9306,NCA9548,NSD12416-Q1SPR,NSI6602MC-Q1SWTR,NSI8242NX,NSD8381-Q1QAIR,NSD8308-Q1HTSXR,NCA9617A,NST1001HA,NSI6602VA-DSWR,NSI1303D2X,NSI1050,NSI6601B-DSWVR,NSD7310-DHSPR,NSI6601,NSD731X-Q1,NCA1057N-Q1SPR,NSR7808GXX,NSD7312-Q1HSPR,NSI6602VB-DLAMR,NSI6602MNF-DSWTR,NSD1026V-DHMSR,NSA3300,NSI6602C-Q1SWKR,NSC2860X,NSD11416‒Q1,NSI6622NB-Q1SPNR,NCA1051N-DSPR,NSI6611ASC-DSWR,NSI22C11,NSI8230CX-DSWR,NSI6801B-DSPR,NSI6622A-DSPNR,NSD12409‒Q1,NSD5604NE-DHTSTR,NSPDSX SERI

选型指南 - 纳芯微电子 - April 2024 PDF 英文下载

WAYON 集成电路产品介绍

型号- WR1117A 系列,WR0332A,WJ1043-AT,WY32M0133CCT3,WP101X,WR0338,WY32F1233RBT7,WR78L系列,WJ5201,WP250152XX-B,WR0343,WY32F1221K8T7,WR0340,WP560X,WD2305,WD2308,WD2306,WR78L,WR1117AN,WR431,WY32F1233K8T7,WR432,WP25XX,WP2106,WY32F1235VCT7,WP2101,WR0114,WY8S9003F4D7,WP111X,WY8S8003,WR1117A,WD2316,WY8S8001,WA8199-AT,WY32M系列,WD230X,WY32F1133CBU7,WP8312,WA3321,WY32M4510RET3,WP5602,WA3324,WP5601,WR0116,WR0115,1117,WJ1040&WJ1050,WP110X,WY32A4770IGT9,WD3302A,WP8421,WY32M4771IGT7,WP1230,WA8201,WA3552,WP26XX,WA3551,WJ3157,WY32L系列,WD2XXX,WY8L8006F3P7,WP8412,WA3581,WR78DXX,WY32M1331G6T7,WR0120-AT,WA3582,WY8S8003Y3D7,WY8M9031G6U7,WP8XXX,WR03XX,WP580X,WJ485,WR03XX 系列,WY32F1133C8T7,WY32A1552RCT9,WD1016,WR78LXX,WP25P21XX-B,WY8S9005K6T7,WP8521,WJ1021-AT,WP5X02,WR1241,WA3331,LMV331,WP5X01,WY32F1791RET7,WP8526,WY32F1235RCT7,WP2301,WA3339,WJ1040,WY32M1331C8T7,WP143X,WP8512,WY32F4771ZGT7,WY32M4510VET3,WP5801,WR1010,WP2XXX,WP5802,WR1006,WJ3485,LMV339,WR1008,WJ1050,WY32F4710VCT7,WD2326,WP25P04XTX,WR78MXX,WD231X,WA8240,WD3302,WA3233,WJ1044-AT,WR0603,WA3358,WP1430,WY8L9006Y4P7,WP1432,WY32F1735VCT7,WP250152,WP25P041T5-B,WA3321A,WJ1028-AT,WR1271,WY32F4771IGT7,WP3883CL,WY8S9005G6U7,WY32A系列,WR0115-AT,WA34310,WA3358A,WJ1057-AT,WY32A1552VCT9,WP5X01 系列,WP8721,WY32F1221C8T7,WP6422,WY32F1235CCT7,WP6420,WY32F1791VET7,WY32M1331F6T7,WY8M9031K6T7,WA3133,WY8S8003F3P7,WP520X,WP21XX,WY32F1233C8T7,WJ102X-AT,WA34320,WR033,WJ485X,WY8S系列,WY32F1235CCU7,WP5X02 系列,WY32F1221C8U7,WY32M1333RCT3,WY32A1591VET9,WJ1051-AT,WP8716,WR0516,WY32F4711VCT7,WP8714,WR0514,WA3721,WP8713,WR0513,WR0512,WA3722,WA3724,WY32A,WY8S,WY32L,WA3324A,WY32F4710VET7,WY8L,WY32M,WY8M,WA3393,WY32F1233CBU7,WY8S8003F4P7,LMV393,WR3006,WY8L8006Y3P7,WY8S8003系列,WJ1040-AT,WR0332-AT,WY32M4511VET3,WA8199,WA3XX,WA32177,WA3188,WR4XX,WP311X,WP11XX,WR0301,WY32F1735RCT7,WP540X,WY32F1133K8T7,WY8S9003F4P7,WP3116,WY32M4771ZGT7,WR0308,WP3118,WR0307,WR0306,WR0305,WR0303,WR0302,WP6420系列,WJ1042-AT,WY32F1133RBT7,WY8M系列,78XX,WP101X 系列,78XX SERIES,WY32W1691VET7,WP8111,WY32A1591RET9,WY32F4711VET7,WA3521,WP5401,1117 SERIES,WA3522,WA3OP07,WY8S8003F4U7,WY8M9031E6U7,WY8S8003Y4M7,WP1X1X,WP311X&WP3883CL,WP111X 系列,WD9006,WR4132,WA3199,WA3631,WD1082,WP8108,WA3632,WR78XX,WA3634,WR0331,WP210X,WY8L系列,WY32F1735CCT7,WY8L9006F4P7,WY32A4770VET9,J1040&WJ1050

商品及供应商介绍 - WAYON - V1.0 - 2023/7/27 PDF 中文下载

Building a battery-operated smart camera in five steps using a multi-core microcontroller

In this post, we demonstrate how to train and deploy a deep learning model for image recognition on GAP8—the first generation of ultra-low power IoT application processors. Thanks to the power-optimized MCU-class architecture tailored for intensive AI workloads, GAP8 is the perfect solution when coupled with low-power cameras.

设计经验发布时间 : 2024-11-12

GreenWaves Technologies Partners with Open-Silicon to develop Industry’s First IoT Processor Based on PULP and RISC-V

Open-Silicon, a system-optimized ASIC solution provider, today announced it was selected by GreenWaves Technologies to develop GAP8, the industry’s first IoT processor. GAP8 is built on the open source Parallel Ultra Low Power (PULP) and RISC-V ISA projects. Open-Silicon is providing GreenWaves Technologies with the complete RTL-to-physical design custom SoC implementation that is required to transform this smart IoT concept into working silicon in volume production.

产品发布时间 : 2024-08-20

GAP8 Performance Versus ARM M7 on Embedded CNNs

ARM recently published a new CMSIS library for embedded convolutional neural networks (CNNs) CMSIS-NN. Firstly, it was great to see ARM supporting the market that GreenWaves and GAP8 are focused on. We particularly liked their statement that: “Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication.”

产品发布时间 : 2024-09-10

Silicon Labs Announces New Bluetooth SoC xG27 Family and MCU BB50 Ideal for Small Form-Factor Devices

AUSTIN, Texas, March 14, 2023 /PRNewswire/ -- Silicon Labs (NASDAQ: SLAB), a leader in secure, intelligent wireless technology for a more connected world, today announced two new integrated circuit families designed for the smallest form factor IoT devices: the xG27 family of Bluetooth systems on chips (SoCs) and the BB50 microcontroller unit (MCU).

产品发布时间 : 2024-03-16

GreenWaves Technologies Licenses Intrinsic ID Hardware Root of Trust for RISC-V AI Application Processor

GreenWaves’ pioneering RISC-V-based IoT application processors enable the cost-effective development, deployment and autonomous operation of intelligent, battery-operated sensing devices that capture, analyze, classify and act on the fusion of rich data sources such as images, sounds or vibrations at the very edge of the network.

产品发布时间 : 2024-08-20

Lynred and GreenWaves collaborate on New Occupancy Management Reference Platform for People Counting Sensor

GreenWaves and Lynred have collaborated on an open-source workspace management platform that allows quick deployment of sensors collecting accurate occupancy data. This platform combines Lynred‘s low-power IR sensors with GreenWaves‘ GAP8 processor to create battery-operated people counting devices, released under open source licenses. The platform ensures occupant anonymity using infrared technology and will be demonstrated at Embedded World in Nuremburg, Germany.

产品发布时间 : 2024-09-07

New GAP8 SDK V2.1 Was Released from GreenWaves Technologies

New GAP8 SDK release from GreenWaves Technologies. This article will show the headline changes in this SDK.

产品发布时间 : 2024-08-22

GreenWaves Technologies Won the Silver Golden Mousetrap Award 2019

Grenoble, France, Feb 5, 2019 – GreenWaves Technologies, a fabless semiconductor startup designing disruptive ultra-low-power embedded solutions for image, sound, and vibration artificial intelligence processing in sensing devices, announced today that it has been selected as a winner of a silver Golden Mousetrap award 2019.

原厂动态发布时间 : 2024-08-13