NTT announced the launch of a large-scale integration chip designed for real-time AI inference processing of ultra-high-definition video up to 4K resolution at 30 frames per second. This innovative technology is tailored for edge devices and terminals with stringent power constraints, marking a significant advancement in inferencing capabilities.
Real-Time AI Inference: The new LSI enables real-time AI inference processing from ultra-high-definition video on edge devices, overcoming the limitations of conventional AI inferencing that requires video compression for real-time processing.
Low Power Consumption: Utilizing an NTT-created AI inference engine, the LSI achieves high computing efficiency with power consumption of less than 20 watts, making it ideal for power-constrained environments.
Enhanced Drone Operations: When installed on drones, the LSI allows for object detection from up to 150 meters above ground, significantly extending the operational range compared to conventional technologies.
Diverse Applications: The technology holds immense potential for various use cases, including infrastructure inspection, public safety, and live sporting events. For example, when this chip is installed on a drone, the drone can detect passersby and objects such as car over a wide area from 150m above the ground (conventionally around 30m), which enables infrastructure inspection while the drone navigates safely and beyond visual line of sight, helping to reduce labor and costs.
Kazu Gomi, President and CEO of NTT Research, stated, "The combination of low-power AI inferencing with ultra-high-definition video holds enormous potential. NTT’s LSI, which we believe to be the first of its kind to achieve such results, represents an important step forward in enabling AI inference at the edge and for power-constrained terminals."
NTT plans to commercialize this LSI within fiscal year 2025 through its operating company, NTT Innovative Devices, leveraging the high-speed and low-latency capabilities of the IOWN All-Photonics Network.
Additionally, NTT researchers are collaborating with NTT DATA to advance this LSI in relation to its proprietary Attribute-Based Encryption technologies, enabling secure data sharing with fine-grained access control.
More Tech details:
High-definition AI inference technology
The basic processing flow of this technology is shown in Figure 2(a). First, to perform the AI inference without image shrinking, the input image is divided, and the AI inference is performed on each divided image. This enables detecting small objects. In parallel, the AI inference in the shrunk whole image is also done to detect large objects that straddle the divided images. The final detection results are output by combining the results obtained from the divided images and those obtained from the whole shrunk image. As a result, both small and large objects can be detected in 4K video. In addition, all detection processes can be executed independently of each other, which enables parallel execution with hardware.
NTT-original AI inference engine
The high-definition technology mentioned above requires performing inferences from a large number of divided images for 4K video, significantly increasing the computational complexity. This makes real-time processing difficult even when parallel calculations are performed using hardware. Therefore, the NTT-original AI inference engine (Figure 2(b)) reduces the computational complexity while ensuring detection accuracy by applying our technologies such as improved computing efficiency using inter-frame correlation and dynamic bit-precision control. This achieves 4K real-time AI inference at low power.
High-definition AI inference technology
The basic processing flow of this technology is shown in Figure 2(a). First, to perform the AI inference without image shrinking, the input image is divided, and the AI inference is performed on each divided image. This enables detecting small objects. In parallel, the AI inference in the shrunk whole image is also done to detect large objects that straddle the divided images. The final detection results are output by combining the results obtained from the divided images and those obtained from the whole shrunk image. As a result, both small and large objects can be detected in 4K video. In addition, all detection processes can be executed independently of each other, which enables parallel execution with hardware.
NTT-original AI inference engine
The high-definition technology mentioned above requires performing inferences from a large number of divided images for 4K video, significantly increasing the computational complexity. This makes real-time processing difficult even when parallel calculations are performed using hardware. Therefore, the NTT-original AI inference engine (Figure 2(b)) reduces the computational complexity while ensuring detection accuracy by applying our technologies such as improved computing efficiency using inter-frame correlation and dynamic bit-precision control. This achieves 4K real-time AI inference at low power.
For more information, Visit https://ntt-research.com
