Qualcomm Launches AI200 and AI250: Rack-Scale Inference Solutions Target Generative AI Efficiency in 2025 Data Centers

Qualcomm  announced the Qualcomm AI200 and AI250 chip-based accelerator cards and racks for data center AI inference. These solutions provide rack-scale performance and memory capacity for generative AI inference, with the AI250 featuring a near-memory computing architecture that increases effective memory bandwidth by more than 10 times and reduces power consumption compared to prior generations.

The AI200 supports 768 GB of LPDDR per card for large language model (LLM) and large multimodal model (LMM) inference and other AI workloads. Both solutions incorporate direct liquid cooling for thermal management, PCIe connectivity for scale-up, Ethernet for scale-out, and confidential computing for secure AI processing. Rack-level power consumption is 160 kW.

The products integrate with a software stack covering application to system layers, compatible with machine learning frameworks, inference engines, generative AI frameworks, and techniques such as disaggregated serving. It enables model onboarding and one-click deployment of Hugging Face models through the Efficient Transformers Library and Qualcomm AI Inference Suite, along with tools, libraries, APIs, and services for AI applications and agents.

Durga Malladi, sen...