Move over Nvidia: AMD unveils Instinct MI300 AI accelerators and Dell is ready to take orders right now

Along with the AMD Ryzen 8040 series for laptops, AMD today announced the Instinct MI300. A direct rival to Nvidia’s H100 AI accelerators, AMD’s units promise roughly 10% to 20% faster times when inferencing and equivalent speeds for training.

While AMD wouldn’t quote a price — we overheard one VP jokily say “we price it just right” in a chat immediately after a briefing — we know that the AMD Instinct MI300 is already in production.

Indeed, whilst on stage at the launch, Arthur Lewis, Dell’s President of Core Business Operations, Global Infrastructure Group, announced that it was already ready to take orders. “Effective today, Lisa, we’re open for business, we’re ready to quote and we’re taking orders,” he said.

There are two versions of the new AMD Instinct range: the MI300X and the MI300A. The MI300X is essentially a graphics-based accelerator while the MI300A includes 24 of AMD’s Zen 4 cores to add conventional processing power.

Both come in a package that includes eight units, each of which consumes around 750W of power at peak. As the design is an industry standard, target users — primarily data centres — can upgrade existing AI accelerators as a slot-in replacement.

Nvidia has already announced that its H200 accelerators are coming, with delivery expected in roughly six months. We don’t yet know how fast these will be, but an AMD spokesperson said that if Nvidia “leapfrogged us then we will leapfrog them right back”.

Comprehensive coverage

The AMD Instinct MI300 is the follow-up to Instinct MI100 and Instinct MI200. Each has received a jump in AMD’s CDNA neural accelerator generations, so the AMD Ryzen MI300 is up to CDNA 3.

AMD describes CDNA 3 (rather vaguely) as “next generation AI accelerator architecture”:

You may wonder why you should care about AMD’s new Instinct range at all, and the answer is twofold. One is availability: Nvidia has struggled to meet demand for its AI accelerators, which is one reason why some generative AI services are so slow to accept new customers.

The second is speed. There are some workloads where the MI300 will make a dramatic difference to processing time for a generative AI task, with AMD suggesting that it can be up to 60% faster with AI models such as Bloom.

And the third is due to power demands. We need generative AI tasks to consume less power per task, not only for cost reasons (electricity costs remain a huge factor) but also to keep control of generative AI’s ecological footprint.

Introducing AMD Instinct MI300A

One of the biggest changes is for AMD to create an all-in-one accelerator that packs CPU cores along with the graphical power. So, rather than partner an AMD Epyc processor with an AMD Instinct MI300X, you can buy an AMD Instinct MI300A that does it all.

Aside from cost savings, there are performance benefits too. By having unified memory shared between the 24 Zen 4 cores and the graphics chips, latency becomes far less of an issue.

AMD claims this “APU advantage” extends to dynamic power sharing and easier programming, while delivering huge performance boosts in certain scenarios. For example, it claimed the MI300A was four times faster than the Nvidia H100 in OpenFOAM’s HPCMotorbike test. Much of this is due to the shared memory.

It also reckons the MI300A will deliver twice the peak HPC (high performance computing) performance per watt of the Nvidia GH200. And here are its HPC performance leadership claims overall:

Reaction to the AMD Instinct AI300 range

Informally, the feeling from analysts attending the show is “just get it out there and it will sell”. That AMD didn’t even need to produce a fantastic product, such is the pent-up demand for AI accelerators.

And there was obvious appetite from the various vendors. Senior executives from Dell, Lenovo and Supermicro all joined Dr Lisa Su on stage to extol the benefits not only of AMD’s latest releases but also to explain how businesses were ready to take advantage.

“So we’re incredibly excited today about the addition of the Impact MI300X to the Lenovo ThinkSystem platform,” said Kirk Skaugen, President of Lenovo’s Infrastructure Solutions Group. “So, bottom line from bench to cloud, we are incredibly excited about what’s ahead for us.”