Trending Articles
Trending Topics
Trending
Last month, I caught up with Elpida Tzortzatos, Chief Technology Officer for AI across the IBM Z series mainframes and LinuxONE. If you haven’t been concentrating at the back, this has been a big few months for both product lines: we’ve seen the launch of the z17 mainframes and IBM’s pre-configured LinuxONE 5 rackmount servers.
These updated product lines have been built with artificial intelligence in mind, and as the CTO of AI across both Elpida is in a unique position to explain exactly what that means for businesses. And explain she did.
If you’re a CEO, CTO, CIO, IT director, IT manager or you simply have an interest in what AI can do for your business then I think it makes for fascinating reading.
For ease of navigation, I’ve broken our conversation into four main sections. Predictive AI vs Generative AI, to provide a grounding of what the terms mean. Some specifics on IBM’s Telum II processor and Spyre accelerator. And a wider discussion about the use of AI by enterprises.
I began by asking how comfortable Elpida was about the use of the term “AI” in such a generic fashion, when in reality it means so many different things?
“I am not that comfortable,” she replied. “I mean, everybody talks about Gen AI today, and certainly Gen AI has a lot of potential and is important and critical to enterprises. But to me, predictive AI is not going away for a lot of use cases.”
To quickly define the terms, you can think of predictive AI as a way to extract insights from historical data, while the likes of ChatGPT and Midjourney fall under generative AI. So, Gen AI literally generates responses – images, videos, text – in response to your instructions.
“As you’ve seen during our launch [of the z17 and LinuxONE 5], we invest in two different types of AI hardware accelerators. And you might ask yourself, why?” The answer boils down to the varying types of usage organisations need them for.
Let’s jump straight to IBM’s two different types of AI acceleration. The first is an AI co-processor that’s embedded into its Telum II chip, which both the z17 and LinuxONE 5 are based on. The Telum II is the successor to the Telum, which debuted with the z16 and included a more basic on-chip AI accelerator.
The second is a discrete Spyre accelerator, which hasn’t yet landed. It’s an incredibly modest chip in terms of power demands: at 75W, it consumes less electricity than most Nvidia graphics chips found in gaming laptops.
So, IBM isn’t attempting to grab market share from Nvidia. “The card, initially, will be enabled for inference,” said Elpida. “We’re not looking to provide industry-leading performance for training because our clients are telling us that acceptable performance for training is good enough for them.”
Those clients – and we’ll come on to who we mean in a moment – are more concerned about “power efficiency, security and actually delivering optimal performance on the AI deployment side”.
So, don’t expect Spyre accelerators to create images of cats from a handful of words.
The best way to understand IBM’s approach to AI is to consider that the z17 isn’t some random server sitting in a data centre. It’s a mainframe: to use Elpida’s definition, “an enterprise server where the world’s economy runs”.
“You know, over 70% of the world transactions by volume run on IBM z [mainframes],” she said. “All of the biggest banks in the world run the transactions on ‘Z’. The same for the top global credit card companies, the same for a lot of health insurance enterprises, a lot of governmental agencies around the world, not just here in the US.
“Really the backbone of the world’s economy, I would say, runs on IBM Z servers.”
IBM is now naturally keen for the world’s economy to upgrade to the latest versions. Ones that are protected against quantum threats, for instance. “We are a quantum safe system, which means that everywhere in our Z stack, starting from the firmware up, we leverage state of the art encryption algorithms that are quantum safe.”
During our conversation, Elpida repeatedly mentioned the need for enterprises to guard against fraud.
“When I talk about fraud, there are several different types and categories of frauds across many industries,” she said. The problem isn’t simply about detecting it: it’s about doing so in a timely fashion and at volume, such as the financial industry. If you’re a card payment provider, you want to minimize fraudulent transactions while ensuring that timeouts don’t occur – or for the process to simply take too long and for customers to walk away.
“A lot of our clients run high volume transactional workloads on our platforms, and one of the things that they wanted… is to embed AI into every transaction, especially when you think about payment transactions and debit card transactions and instant payment and credit card transactions.”
What happened if the anti-fraud measures didn’t complete in time? They had to let the transactions go through as the lesser of two evils. “They weren’t getting the response times and the throughput that they needed for those type of workloads [without AI]”, Elpida explained.
This is where the on-chip AI accelerator built into the Telum II comes in. “On a fully loaded z17 server, you can have up to 32 of these chips, therefore 32 of these on chip AI accelerators,” she said.
“Now, when you think about predictive AI and… you’re making a credit card purchase, and that requires single-digit millisecond response times, the fact that this accelerator is right on that chip means that we share the memory and the Level 4 cache with the host.”
And that means there’s no need to waste time sending the data from the chip to a separate AI accelerator such as IBM’s Spyre or the Nvidia equivalent.
So, where does that leave the Spyre accelerator? “Spyre is really targeted for Gen AI use cases,” said Elpida. “Gen AI use cases require more compute, but also have different SLA requirements. You talk about tokens per second per user, versus single-digit millisecond response times. So Spire, we’re targeting for clients who want to sustainably scale their Gen AI deployments.”
As already touched upon, there’s no way that Spyre can compete with Nvidia for compute power. Instead, Elpida believes it wins for “enterprise-grade qualities in terms of reliability, scalability, availability and security to deploy those Gen AI uses cases that are relevant to them”.
But out of all those reasons, security comes top. “Security is the number one priority, especially when they’re fine tuning some of those large language models with their own private data,” she said.
“Being able to have control over their data and their model IP was important to them, but also for clients who have to comply with data sovereignty right and data residency requirements, it’s very important to them that they’ll be able to deploy Gen AI models and use cases securely on prem.”
Sustainability is important too, she adds. “[Clients] can also say that they’re meeting their main ESG strategic initiatives that may be undertaking as well. So security and power, efficiency and sustainability. These are really the key differentiators that we’re bringing to the market with this card.”
You might expect IBM to be defensive when asked about Nvidia, but quite the opposite. When I asked Elpida why an organisation looking to create generative AI solutions would even consider IBM when Nvidia’s solutions are so much more powerful, her eyes positively lit up.
“Great question. Absolutely great question.” Who was I to disagree? “We understood even even back in the z16 timeframe that some clients might have made investments in training and developing models [using Nvidia hardware],” she said.
“And we wanted to make sure that we built a very robust AI ecosystem on top of that hardware acceleration [the z16’s] that would allow clients to bring AI models – even when those models are not developed on Z – and be able to deploy those models on Z very easily in a consumable manner.”
To that end, Elpida explained, they had “instrumented a lot of the popular AI frameworks such as PyTorch, both for the on-chip accelerator and for Spyre”.
She added: “So we actually integrated into the back end of PyTorch. So even if clients take an open-source model from, let’s say HuggingFace Leaderboard, they can deploy it on Spyre without any changes.” IBM has taken the same approach for vLLM, “the de facto inference server for large language models and Gen AI models”.
Similarly for the on-chip accelerator, which supports Red Hat OpenShift AI.
“Now I’m not saying that the Spyre card is, for example, suited for use cases where you want to discover a new drug and you’re doing a lot of training,” said Elpida. “[We are focused on] AI deployments, when you take those models and deploy them into your transactions, into your applications, into your core business processes, because that’s where our clients told us there were mainly challenged from a fine tuning perspective.”
Hopefully this article has helped to set the scene for IBM’s approach to AI. I must confess that it clarified many things for me. But I’ll leave the final words to Elpida.
“One of the most important decisions that organisations are going to have to make is the right fit infrastructure to deploy [their] AI use cases,” she said. “And if you think of data as the fuel for AI, then infrastructure is the engine that would allow those organisations to actually succeed with their AI deployments and journeys.
“Now, when we look at our AI strategy, we take a holistic view, and we’ve rooted our strategy across two key pillars. One is building an AI infrastructure that allows our clients to easily embed AI into their applications, into their core business processes, into their transactions, and that’s what we’ve talked about mainly here.”
The software ecosystem forms part of this first pillar, whether that’s watsonx.governance, watsonx.ai, Red Hat OpenShift AI or open-source tools. “We embraced open source, as you see here, and our clients who have mature open source skills, they’re certainly welcome to leverage hardware acceleration and our infrastructure that way, because we’ve integrated into this popular AI frameworks and tooling,” said Elpida.
“The second pillar of our strategy is us being able to leverage AI throughout the Z stack to deliver value to our clients right out of the box, whether it’s in the AI ops area, whether it’s in data privacy and data security areas… whether it’s for dev labs, for example, for application modernization.”
But Elpida was keen to point out that IBM wasn’t attempting to be “everything for everybody. “Our focus is on enterprise clients and use cases that are relevant to enterprise clients. And we optimise for those use cases.
“Now I know that there are other choices of accelerators out there. If you want to be, for example, designing robotic equipment that helps in surgeries [then] I’m not saying that the Z will be the right place for those type of use cases. We’re really focused on enterprise use cases, and optimising for those use cases from an infrastructure perspective, both on the business side, but also on the IT operation side.
“We maniacally focus on [those pillars], and do that really, really, really well. We know business workloads. We know transactional workloads. That’s our strength, and we continue to bring the latest and state of the art technology such as AI to those workloads.”
