VeriSilicon’s NPU IP enables over 40 TOPS for on-device LLM inference in mobile devices

On June 9, 2025, VeriSilicon announced that its Neural Network Processing Unit (NPU) IP, designed for ultra-low energy consumption and high performance, supports on-device inference of large language models (LLMs) with a computing performance exceeding 40 tera operations per second (TOPS). The NPU is tailored for generative AI on mobile platforms, including AI phones and AI PCs, addressing energy efficiency requirements.
The NPU features a configurable and scalable architecture with support for mixed-precision computation, advanced sparsity optimization, and parallel processing. It incorporates efficient memory management and sparsity-aware acceleration to reduce computational overhead and latency. The NPU supports numerous AI algorithms, including AI-NR and AI-SR, and models such as Stable Diffusion and LLaMA-7B. It can integrate with VeriSilicon’s other processing IPs for heterogeneous computing, enabling system-on-chip (SoC) designers to create AI solutions for various applications.
The NPU is compatible with AI frameworks like TensorFlow Lite, ONNX, and PyTorch, facilitating deployment and integration across different AI use cases. Weijin Dai, Chief Strategy Officer, Executive Vice President, and General Manager of the IP Division at VeriSilicon, stated that mobile devices are increasingly functioning as personal AI servers, with growing demand for AI computing driven by advancements in generative AI and multi-modal LLMs. He noted that energy consumption is a significant challenge for high AI computing workloads and that VeriSilicon has collaborated with SoC partners to implement this NPU in next-generation AI phones and AI PCs.