OODA Loop – AI Energy Crisis Boosts Interest in Chips That Do It All

To understand a key reason artificial intelligence requires so much energy, imagine a computer chip serving as a branch of the local library and an AI algorithm as a researcher with borrowing privileges. Every time the algorithm needs data, it goes to the library, known as a memory chip, checks out the data and takes it to another chip, known as a processor, to carry out a function.
AI requires massive amounts of data, which means there are the equivalent of billions of books being trucked back and forth between these two chips, a process that burns through lots of electricity. For at least a decade, researchers have tried to save power by building chips that could process data where it’s stored. “Instead of bringing the book from the library to home, you’re going to the library to do your work,” says Stanford University professor Philip Wong, a top expert in memory chips who’s also a consultant to Taiwan Semiconductor Manufacturing Co. This process, often referred to as in-memory computing, has faced technical challenges and is just now moving beyond the research stage. And with AI’s electricity use raising serious questions about its economic viability and environmental impact, techniques that could make AI more energy efficient might pay off big. This has made in-memory computing the subject of increased excitement—and even meant it’s begun to get caught up in the broader geopolitical wrangling over semiconductors. Major chip manufacturers such as TSMC, Intel and Samsung are all researching in-memory computing. Intel Corp. has produced some of the chips to conduct research, says Ram Krishnamurthy, senior principal engineer at Intel Labs, the company’s research arm, though he declined to say how in-memory computing could fit into Intel’s product lineup.

Full report : How energy-hungry AI chips are drawing new funding—and even inspiring some geopolitical maneuvering.