Generative AI has taken the world by storm, from creating stories to images based on the user’s prompts – all within mere seconds. But companies are now shedding light on the cost of AI and what it takes to maintain these platforms and keep them running.

Nick Walton, CEO of Latitude – the company behind AI Dungeon – shared that it cost them nearly $200,000 a month to keep the game running at its peak in 2021.

“We joked that we had human employees and we had AI employees, and we spent about as much on each of them,” Walton said, as cited by CNBC. “We spent hundreds of thousands of dollars a month on AI and we are not a big startup, so it was a very massive cost.”

At the start of its journey, Latitude was using text-generation software GPT, created by the artificial intelligence (AI) lab OpenAI under Microsoft. However, as the company bills were skyrocketing Latitude decided to shift to a ‘cheaper but still capable language software offered by startup AI21 Labs,’ CNBC said.

So what makes generative AI the big spender?

While machine learning is an appealing industry that may be worth even trillions of dollars, the harsh reality regarding the cost of AI is making venture capitalists take a step back and reconsider.

AI applications have a permanently smaller margin than their predecessors due to the high cost of computing – as they require substantial computational power and data storage, Hyperstack explained.

Upwards of ‘70% of global corporate investment in AI is spent on infrastructure – that’s a global spend of about $64.4 billion on computing in 2022 alone,’ Hyperstack added citing Statista, adding that ‘deploying ChatGPT into every search conducted by Google, for instance, would require 512,820.51 A100 HGX servers, totaling 4,102,568 A100 GPUs.’

Apart from using up enormous amounts of energy, generative AI systems also require large quantities of water to cool down their processors and to generate electricity, Nature explains in a recent article.

Speaking at the World Economic Forum in Davos, Switzerland, Sam Altman, CEO of OpenAI, pointed to the cost of AI, saying that the industry is heading for an energy crisis.

“There’s no way to get there without a breakthrough,” he noted.

Altman is betting on nuclear fusion to help power AI and potentially make the systems more sustainable, and he’s already making investments too – in 2021 he started investing in fusion company Helion Energy in Everett, Washington.

Whether the cost of AI will go down in the future is uncertain.

“I think it’s fair to say that it’s definitely a huge change we’re excited to see happen in the industry and we’re constantly evaluating how we can deliver the best experience to users,” a Latitude spokesperson said. “Latitude is going to continue to evaluate all AI models to be sure we have the best game out there.”