Tachyum Advances AI Training with FP4 Data Type for Large Language Models

Tachyum announced that its AI team has developed an algorithm for training large language models (LLMs) using the 4-bit FP4 data type, as detailed in the white paper, “Tachyum demonstrates supercharged LLM training in only 4 bits.” The FP4 format, a 4-bit floating-point representation, reduces memory and compute requirements compared to FP32 or BF16 formats, aiming to lower costs and energy use for LLM training while maintaining model accuracy and downstream task performance.

The FP4 data type offers up to 4x better memory efficiency than 16-bit formats and up to 8x better than 32-bit formats. By quantizing numerical values to lower precision, FP4 enables greater model compression, reducing demands on compute, storage, and memory. This addresses challenges posed by LLMs, which can grow 10x in size per generation, leading to extended training times and increased costs.

Tachyum’s Prodigy processor, featuring 256 custom-designed 64-bit compute cores, supports dynamic switching between AI/ML, HPC, and cloud workloads within a single architecture. It targets up to 18x the performance of the highest-performing GPU for AI tasks, 3x that of top x86 processors for cloud workloads, and up to 8x that of leading GPUs for HPC.