Introducing Three examples of AI MPU RZ/V2M real-life applications

RENESAS released into the RZ/V2M in 2020 to be adopted in Vision AI applications. It is equipped with a high-performance ISP and an original AI-accelerator "DRP-AI". After its release, many customers started developing AI using this product.

Today, I would like to introduce the following three examples of RZ/V2M real-life applications.

• Security camera (using Object Recognition)

• Fitness application (using Pose Estimation)

• AI edge gateway (using Object Recognition, Pose Estimation, and so on)

1. Security camera (using Object Recognition)

The typical application for RZ/V2M is a security camera. ISP is supported up to 4K resolution, the captured image can be encoded to H.265 or H.264. At the same time, AI inference can be treated using DRP-AI. A security camera also requires several special features such as Wide Dynamic Range(WDR), and 2D and/or 3D noise reduction. RZ/V2M supports such requirements.

The AI inference function adapted for security cameras is human detection and recognition, human count, face recognition, suspicious person detection, and so on. DRP-AI is a hardware accelerator, but it can also change the program dynamically. So, you can change the AI module after installation, or change several times the AI algorithms during run time. The key success factor of this application is not only the performance such as AI and/or ISP but also the top level of power efficiency.

2. Fitness application (using Pose Estimation)

Utilizes the pose estimation technology, estimates the pose, and advises the form correction, pose, and so on for fitness service, dance school, and sports analysis. Usually, such application uses cloud AI systems, but it has a delay due to the image cloud upload. This customer installed RZ/V2M in the camera, so its product can get the pose estimation result more easily.

The key success factor is the 4K resolution from the CMOS sensor and low power consumption which does not need any fan and heatsink.

3. AI edge gateway (using Object Recognition, Pose Estimation, and so on)

This case is a kind of edge gateway that inputs several video streams and treats AI function internally. The functionality is almost the same as a security camera, it only differs by the number of video inputs.

When this gateway is set on the edge, it can reduce the delay compared to the cloud. Also, the image data does not need to be uploaded to the cloud, so privacy issues can be solved. They used FPGA in former products, but they had power consumption and heat generation problems. The key success factor is the video codec capability which supports 4K video decoding, and low power consumption. Also, the heatsink could be removed saving costs for the system.

RZ/V2M has potential not only for the introduced application but also for many other kinds of applications.