Trending Articles
Trending Topics
Trending
This article is part of our Opinions section, where we invite industry professionals to share their views on the most pressing technology questions of our time.
As AI development speeds up, it’s becoming more and more attractive to host a model yourself, on hardware you own, or at least run. There are a lot of very good reasons for doing so, but there are some unpleasant downsides, too, which aren’t always discussed.
Most AI models now run in the cloud, which brings with it a host of issues. While things like running the model are cheaper – or at least exchange steep startup costs for a monthly payment – and you don’t have any of the headaches associated with running the hardware, the tradeoff is that you’re pretty much surrendering your data to the company running the AI.
It’s no secret that the models running in the cloud, especially freely accessible ones like Claude or ChatGPT, use all the data you enter for their own ends – Stanford published a paper detailing all the challenges. ChatGPT chats can even show up in Google Search. This means that if you’re using AI in your business, you are potentially revealing sensitive information to anybody with access to the backend of the bot, and it may even seep out to other users.
Not having ownership of your data is an issue for anybody, let alone for a business. You could potentially be exposing the data that makes your company successful, like a customer database or whatever digital vault you keep your code in. Most AI bots simply aren’t secure.
The most straightforward solution for this issue is to keep your data within your own network by deploying and hosting AI models on-premise, potentially even in an air-gapped setup. This way, they cannot share data inadvertently, and you retain full control.
Doing so is a lot easier than it was just a few years ago, thanks to the increased know-how among IT professionals and the greater availability of people who know how to work with AI and its hardware. On top of that, great strides have been made in the development of new hardware that make it possible to create the advanced systems that can host AI models. Think of innovations like AI chips, which should, on paper at least, make it relatively easy to put together new systems.
The cost, however, is prohibitive. As AI has heated up, so has the price of everything associated with it. CPUs, GPUs, they’ve all skyrocketed in price, with some of Nvidia’s more advanced GPUs, to give but one example, costing north of $10,000. We can only imagine the cost of AI chips.
Even if you have the money, there’s a chance you’ll be waiting a long time to get your hardware. There’s a backlog for, well, all of it. Though manufacturers aren’t exactly advertising how long it will take for orders to be filled, we can only expect it will be a while, especially considering the turmoil of the Trump-fueled trade wars.
A final issue is that even if you have built a perfect home for your AI, you’re going to have to train both it and the people who will use and maintain it. This isn’t cheap or easy. Training an AI is hard work and can’t be done overnight. You’ll likely need to hire people to do it for you, too, and recruiting them is again a very expensive proposition.
Even when your AI is working the way you want, you’ll have to get people ready to use it. With specialist models, training non-specialist staff takes time and resources. The more you plan to rely on your newly built model, the more staff you’ll need to teach how to use it, and the more they’ll need to know.
It may seem like you’re caught between a rock and a hard place: on the one hand, you may be revealing your data using cloud AI, but preventing this from happening may bankrupt you. Even if it doesn’t, you have no guarantee that your new AI and the hardware it runs on won’t be outdated by the time you finally get it ready.
The problem may seem intractable, but if you look again, you’ll see that the solution is quite simple. When it comes to things like AI, that move fast with no day going by without a major shakeup, don’t overinvest. Be conservative with AI; don’t invest in it unless you know it will do some good. There’s just no point in putting thousands into technology that’s always changing.
On top of that, you may also want to question this idea of on-prem vs. cloud-based AI; in many ways, it’s a false dichotomy. While data security does need to be at the forefront of your mind, the solution could be as simple as having two systems in place.
You could have a cloud-based AI that runs all your non-sensitive data, and a smaller on-prem one that works with anything you want to keep secured. You could even decide to only work on important stuff by hand and keep the machines away from it entirely.
You would probably need some kind of hybrid universal layer to run all this, but that’s something any IT firm worth its salt can put together. While that’s an investment in itself, it’s probably a lot cheaper and less hassle than building an entire on-prem system.
In the end, AI infrastructure choices are not about ideology, where one solution is better than the other, but about aligning compliance, cost, and capability. The winners will be enterprises that adopt AI incrementally, guided by transparent frameworks and hard facts rather than hype.
