Tachyum announced the release of a white paper titled “Tachyum Successfully Quantized DeepSeek LLM to its 2-bit TAI2,” detailing its approach to scaling Large Language Model (LLM) training and inference using the Mixture of Experts (MoE) method. The company applied a DeepSeekMoE architecture with 4-bit FP4 data types for activation quantization and 2-bit Tachyum AI (TAI2) sparse weights quantization.
The white paper explains that MoE can achieve performance comparable to dense models while using approximately four times less computing power and memory bandwidth, though memory capacity requirements increase by about four times. This ratio is expected to grow. Tachyum’s proprietary high-performance memory eliminates the need for high-bandwidth memory (HBM) solutions. Quantizing DeepSeek LLM to 2-bit TAI2 doubles the efficiency benefits of the DeepSeekMoE architecture compared to other architectures.
Tachyum’s AI researchers implemented FP4 activation quantization and 2-bit TAI2 sparse weights quantization on DeepSeekMoE and Llama 3.1 models. Benchmark testing showed inference speeds up to 25 times faster and a 20-fold cost reduction per token.
Dr. Radoslav Danilak, Tachyum’s founder and CEO, stated that the DeepSeek approach could make next-generation models 10 times more efficient at current costs, addressing exponential scaling challenges. He noted that the Prodigy platform supports this efficiency for AI applications globally.
The white paper highlights the role of Tachyum’s Prodigy Universal Processor, which supports AI workloads. The processor integrates 256 custom-designed 64-bit compute cores, enabling data center servers to switch between AI/ML, HPC, and cloud workloads within a single architecture. This eliminates the need for dedicated AI hardware, reducing capital and operational expenses. The Prodigy processor delivers up to 18 times the performance of the highest-performing GPU for AI applications, three times that of the top x86 processors for cloud workloads, and up to eight times that of the highest-performing GPU for HPC.
The white paper is available for download on Tachyum’s website.
