Cerebras WSE 3 chip with 4 trillion transistors deliver 2x performance compared WSE 2
Date: 13/03/2024
Cerebras Systems break its own record of designing a chip with larger transistor count compared to its previous Cerebras WSE-2. The new Wafer Scale Engine 3 (WSE-3), delivers double the performance without compromising on power consumption or cost. Engineered specifically for training the most extensive AI models, the WSE-3, built on a 5nm process, boasts 4 trillion transistors and 900,000 AI cores, enabling the Cerebras CS-3 AI supercomputer to achieve an impressive 125 petaflops of peak AI performance.
PIC: WSE 3
Key Features of WSE 3:
4 trillion transistors
900,000 AI cores
125 petaflops of peak AI performance
44GB on-chip SRAM
5nm TSMC process
External memory: 1.5TB, 12TB, or 1.2PB
Trains AI models up to 24 trillion parameters
Cluster size of up to 2048 CS-3 systems
Let's compare this with WSE 2: WSE-2 packs 2.6 trillion transistors and 40 gigabytes of on-chip memory with an active area of 46,225 mm2 fabricated on 300mm silicon wafers using TSMC's 7nm complementary metal-oxide semiconductor (CMOS) technology.
With a vast memory capacity of up to 1.2 petabytes, the CS-3 empowers the training of next-generation AI models, surpassing previous benchmarks such as GPT-4 and Gemini by tenfold. Its capability to handle models with up to 24 trillion parameters in a single logical memory space streamlines training workflows and enhances developer productivity. The CS-3 accommodates training from compact four-system configurations to full-scale operations with 2048 systems, enabling unprecedented advancements in generative AI.
The latest Cerebras Software Framework integrates seamlessly with PyTorch 2.0, facilitating the adoption of advanced AI models and techniques. Unique hardware acceleration for dynamic and unstructured sparsity accelerates training by up to 8x, underscoring Cerebras' commitment to innovation. Notably, the CS-3's superior power efficiency and streamlined software architecture require significantly less code compared to GPUs, reducing complexity and enhancing user experience.
Cerebras has garnered substantial interest across enterprise, government, and international cloud sectors, with a significant backlog of CS-3 orders. The strategic partnership between Cerebras and G42 has yielded remarkable results, culminating in the construction of Condor Galaxy 3, set to be one of the largest AI supercomputers globally. With its unparalleled capabilities and collaborative ventures, Cerebras is poised to revolutionize the AI landscape and drive the industry forward.
By embracing cutting-edge technology and fostering strategic partnerships, Cerebras continues to push the boundaries of AI innovation, paving the way for transformative advancements in machine learning and computational intelligence.
WSE 2 and WSE 2 are the world's largest integrated circuit chips for complex artificial intelligence (AI) computation applications training with very large AI databases as a single data core.
Cerebras Systems making these big chips by having multiple semiconductor wafers integrated using advanced semiconductor packaging and system design.
Pic above: WSE-2
