Advanced computing: Niche or indispensable?

The latest rumour about the most rumour-prone figure in technology is that Elon Musk has acquired 10,000 data centre-grade graphics processing units (GPUs) as he observes that the chips used to run artificial intelligence (AI) systems are “considerably harder to get than drugs.”

Several outlets, including Business Insider, claim to have confirmed the order, tying the move to murky plans for a generative AI programme at Twitter. It highlights a narrowing supply-demand balance for computing power as data-intensive technologies related to ideas such as large language models (LLMs), video streaming, and metaverse applications go mainstream.

“Everyone wants to know how demand will be surging with the emergence of LLMs, autonomous driving, and simulations nowadays. So everyone wants GPUs,” said Sota Nagano, a general partner at Abies Ventures. “The largest GPU cloud service is Lambda, and there’s no vacancy for the H100 GPU capacity. The price of a GPU has climbed by 50% in the last couple of weeks.”

This is not a pinch that investors in these areas are feeling quite yet, but by most accounts, the pinch is coming. The notion of software and algorithm development outpacing its support hardware is as old as the invention of the computer itself. But the dilemma is riding on a parabolic curve that could soon come to a head.

The latest version of ChatGPT is reportedly as much as 500x larger than its predecessor with 100trn parameters. As the models get bigger, enterprises are likely to retrain them more often, whether that’s to cope with dynamic data, respond to feedback, or align with a software release cycle.

According to a recent survey by UK-based technology consultancy Omdia, 76% of global enterprises consider the scale of their AI workloads to be large or very large in customer experience functions. That figure is 75% for quality monitoring, 72% for process optimisation, and 58% for chatbots and virtual assistants.

“As companies and countries try to outdo each other, building more sophisticated foundational models to feed into their LLMs and platforms, they’re going to start to see limitations,” said Hsien-Hui Tong, executive director of investments at SGInnovate.

“We have not hit a critical juncture yet. We probably will not hit it for the next couple of years. But the way that LLM adoption is going, it may be much sooner than that.”

Start-up inroads

Abies and SGInnovate have various investments that play into this theme. Abies’ most direct computing acceleration play is Japan-based Mawari, a scalable 3D content delivery platform that uses cloud rendering and streaming technologies for virtual reality and augmented reality content. It raised USD 6.5m in February.

SGInnovate’s key asset in the space is Singapore-based Blaize, a GPU-style chipmaker and edge hardware provider that claims to deliver accelerated processing speeds for AI, computer vision, and natural language processing. The company raised a USD 71m Series D round in 2021.

It remains an open question, however, to what extent such services can help companies facing pressures around accessing sufficient compute capacity to run big data constructs such as generative AI. For now, the advice is to lean on the most established and reliable technologies, even as their procurement becomes difficult.

“We try not to invest in companies that don’t have a prima facie edge in an arms race. That being said, if one of our companies finds itself a victim of a compute arms race through no fault of their own, we’d advise: do whatever you can to have more compute than your competitors, including painful stockpile purchases in advance,” said James Hardiman, a general partner at US-based Data Collective (DCVC).

“If that is not possible then: focus, focus, focus. Constrain what you’re doing until you can do it with what you have while still delivering a critical economic or product outcome for key customers.”

DCVC’s exposure to advanced computing in the region includes New Zealand’s Nyriad, a GPU-accelerated storage device supplier that has raised about USD 10m since its inception in 2014. But there is scepticism that hardware is the right place for start-ups.

“There’s been a huge bloom of AI hardware start-ups, but to be honest, it’s difficult to see any evidence that they’re getting much traction,” said Alexander Harrowell, a principal analyst focused on advanced computing at Omdia.

“People are responding to [rising compute demand] by essentially buying GPUs, and overwhelmingly that expenditure is going to Nvidia. It’s even difficult to see any evidence that companies like AMD or Intel are getting much traction in this space.”

Harrowell is particularly wary of too much VC money going into coarse-grained reconfigurable architecture (CGRA) chips, which he sees as a less scalable domain versus edge-related applications. Blaize uses CGRAs but is differentiated by various technical advantages, as well as being optimised for streaming.

Capacity conundrum

Meanwhile, there is a sense that the major data centre operators and cloud service providers will be able to simply continue scaling out to meet demand as LLMs and other data-intensive business functions take flight. This is in spite of the excessive power consumption that will be required and other environmental concerns.

Monojit Mazumdar, Deloitte’s Asia lead for cloud, is tracking rapid uptake of cloud services, even in historically hesitant industries such as financial services and pharma. He believes compute capacity shortages, if there are any, will be short-lived phenomena and quickly bridged as the major players build new supply. But this does not mean user costs won’t increase.

As companies move compute capacity to the cloud, there is a tendency to embrace the seemingly unlimited capacity by over-consuming and ramping up fees unnecessarily. This coincides with a tendency among companies to store data in the cloud rather than on-premises; Mazumdar describes storage costs in the cloud as an order of magnitude greater than compute costs.

“Almost every CIO and CTO we talk to has told us that the promise of cloud in terms of getting the cost benefit is only partially realised,” he said. “The costs have gone up because in some cases, users have not been behaving responsibly or they’re simply doing things the old way in a new environment, thereby losing the clear economic advantage of it.”

Industrial compute capacity challenges are not limited to the realm of big data in its most commonly appreciated context. In some instances, the bottleneck is the sheer complexity of the mathematical problem-solving required.

For example, even in a seemingly straightforward optimisation equation like organising a shipping schedule, the number of variables influencing the vessels and their counterparties can be overwhelming. The best algorithms in contemporary computing will take days to come up with a schedule that is still far from optimal.

“Problems that classical computing cannot tackle are coming in different industries, from route optimization in supply chain management to drug design in the pharmaceutical industry, to optimization problems in artificial intelligence and machine learning,” said Alexey Bocharnikov, EY’s Asia Pacific quantum technology leader.

“There are use cases where the complexity of the problem is so demanding that the industry won’t progress unless you have a new computational model. That’s one of the reasons people are investing so much in quantum – they see so many problems in their industries that they believe cannot efficiently progress relying solely on classical computing.”

Atomic-level innovation

Quantum computing handles exponentially larger data sets than classical computing at speed by operating according to the laws of physics at the atomic level. This means the ones and zeros of classical computing are augmented by quantum bits, or qubits, which can represent ones, zeros, or both at the same time.

Abies, SGInnovate, and DCVC all have significant exposure to quantum, with at least three, five, and four portfolio companies in the space, respectively. All three are investors in Singapore’s Horizon Quantum Computing, a company that enables hardware-independent quantum computer software development in conventional computer languages.

They are increasingly being joined by investors with less deep tech-oriented mandates. The latest round for Horizon, for example, a USD 18.1m Series A last April, brought in Peak XV Partners (formerly Sequoia Capital India). Australia’s Q-Ctrl, an investee of DCVC, has raised more than USD 50m in the past two years from the likes of Square Peg Capital and Salesforce Ventures.

Most of the business activity of these start-ups involves supplying other quantum technology developers, universities, or research organisations, and much of the underlying technology remains on the blackboard. As Mark Mattingley-Scott, chief revenue officer at Australia’s Quantum Brilliance, put it: “Right now, nobody can use a quantum computer to do anything useful.”

This bluntness belies significant progress, however. Quantum Brilliance, which raised a USD 17.7m round in February, claims to have developed the first portable quantum computer that operates at room temperature. They typically require extremely low temperatures to operate, making it difficult to roll out in small, inexpensive formats.

The secret to achieving this is the use of synthetic diamonds to create qubits. This material is also expected to support efficient iterating as the business moves into mass production. The company hesitates to nail down timelines, but it is generally hoped that this will happen within a few years.

“We potentially have something that can outperform the entire supercomputer centre, and that’s going to happen quite quickly,” Mattingley-Scott said, referring to a prototype machine in use at the Pawsey Supercomputing Centre in Australia.

“What they have, computationally, is the first step in a hopefully small number of steps to something that will completely change their ability to do engineering, scientific, and commercial computations.”

Practical applications

Evidence is beginning to accumulate that the broader business sector also believes this horizon is fast approaching. In a survey from June last year, EY found that 33% of enterprises and organisations expect quantum to reach sufficient maturity to play a significant role in their industry by 2026. In the next two years, 44% said they would hire a quantum computing leader.

Matt Ocko, a managing partner and co-founder at DCVC, compares the risk-reward balance of quantum start-ups and other accelerated compute start-ups to the building of a supercar versus a space rocket. Both help you go faster, but only one helps you go places that are currently practically unreachable.

“You can figure out who does a lot of driving on tight timelines and sell them something faster – like accelerated compute. By contrast, lots of people think they might want to go to the moon – have routine quantum supremacy for hard compute problems – but no one is currently doing it and so you need to figure out who can actually do something useful once you have a rocket,” he said.

“The first person to figure out you can make a new product with a rocket ship – well, they’re going to make a lot of money.”

Q-Ctrl is one of DCVC’s standout bets in this sense. The company, which primarily supplies other technology companies with tools to facilitate quantum research (EY and Deloitte are both customers), has increasingly moved into defence and industrial applications related to quantum sensors.

CEO Michael Biercuk believes these machines – capable of measuring gravity, magnetic fields, and movement – will be the first practical application of quantum technology. Q-Ctrl has several military and industrial contracts to this effect and is field-testing sensors used to navigate without GPS and detect objects underground. Commercial products are expected to be in the field within two years.

“The pace of progress has accelerated in delivery, and the opportunities to develop economic returns have emerged as near-term,” Biercuk said. “The big motivator is to be an early adopter. Just look at how people were caught so flatfooted with the rise of ChatGPT. There were some organisations that knew about it, and they were already well prepared for the development of LLMs.”

• Tweet  
• Facebook  
• LinkedIn  
• Google plus  
• Save this article  
• Send to  

• Topics
• Technology
• Early-stage
• Australasia
• Southeast Asia
• North America
• Singapore
• Australia
• New Zealand
• Abies Venture
• Data Collective (DCVC)
• data center
• quantum computing