Azure Maia 100 and Cobalt 100 to arrive in 2024 as Microsoft's first in-house AI chips

After months of rumors it’s been confirmed that Microsoft will begin building its own AI chips. The first of these, the Azure Maia 100 and Cobalt 100 chips, will arrive sometime in 2024.

This move on the part of Microsoft is likely intended to avoid an overreliance on Nvidia, whose H100 GPUs are commonly used to operate AI image generation and language model tools.

The Azure Cobalt 100 chip is custom designed for Microsoft to power Azure cloud-based services, and its design will reportedly allow for control of performance and power consumption per core.

To this end, Microsoft is looking to overhaul the Azure cloud infrastructure according to statements made by head of Azure hardware systems and infrastructure at Microsoft Rani Borker in a recent interview. Borker said in part, “We are rethinking the cloud infrastructure for the era of AI, and literally optimizing every layer of that infrastructure.”

The Maia 100 AI accelerator chip is, as its name suggests, designed to run cloud-based AI operations such as language model training. The Maia 100 chip is currently being tested on GPT 3.5 Turbo. As Borkar added,

“Maia is the first complete liquid cooled server processor built by Microsoft. The goal here was to enable higher density of servers at higher efficiencies. Because we’re reimagining the entire stack we purposely think through every layer, so these systems are actually going to fit in our current data center footprint.”

At this point full specs and performance benchmarks for the Maia 100 and Cobalt 100 chips have not been made public. However, it’s already known that these chips are merely the first of a series, and that the second generation of the Maia 100 and Cobalt 100 chips are already in the design phase.