MangoBoost demonstrates multi-node LLM training on AMD GPUs in MLPerf training v5.0

MangoBoost announced its MLPerf Training v5.0 submission, validating the scalability of large-scale AI training on AMD Instinct MI300X GPUs. Using 32 GPUs across four nodes, MangoBoost fine-tuned the Llama2-70B-LoRA model in 10.91 minutes, achieving 95–100% scaling efficiency. The submission marks the first MLPerf Training result on AMD GPUs across multiple nodes.
The system integrates MangoBoost’s LLMBoost AI Enterprise software, which supports model parallelism, automatic tuning, batch scheduling, and memory management, with its GPUBoost RoCEv2 NIC for low-latency, high-throughput inter-GPU communication. Internal benchmarks confirmed compatibility with additional models, including Llama2-7B and Llama3.1-8B.
The submission utilized AMD’s ROCm software ecosystem and MI300X GPUs. Meena Arunachalam from AMD noted the collaboration’s role in demonstrating training results across 32 GPUs. David Kanter from MLCommons highlighted the importance of MangoBoost’s software stack in optimizing AMD GPU performance. CEO Jangwoo Kim stated that the result demonstrates the platform’s readiness for enterprise-scale AI training without vendor lock-in.
MangoBoost is developing further optimizations in communication, hybrid parallelism, topology-aware scheduling, and domain-specific acceleration for distributed AI workloads.