Optical AI processor to cut data centre power consumption ...

Researchers at MIT in the US and the Technische Universitat Berlin have built a optical neural network processor with embedded lasers that slashes the power consumption of large language model AI such as GPT4.0.

The 3D architecture is based on state-of-the-art arrays of vertical surface-emitting lasers (VCSELs) developed by the Reitzenstein group at Technische Universitat Berlin. This gives a 100-fold improvement in energy efficiency and a 20-fold improvement in compute density,

“This was a collaborative project that would not have been possible without them,” says researcher Ryan Hamerly at the MIT’s Research Laboratory of Electronics (RLE), who has patented the design with colleagues.

The system can reach an energy efficiency of 7 femtojoules per operation (OP) with a compute density of 6 TOP/mm2/s1, representing 100-fold and 20-fold improvements, respectively, over state-of-the-art digital processors. Near-term development could improve these metrics by two more orders of magnitude, opening up machine learning tasks from data centres to decentralized devices.

“We expect that it could be scaled for commercial use in a few years. For example, the laser arrays involved are widely used in cell-phone face ID and data communication,” says Zaijun Chen, first author, who conducted the work while a postdoc at MIT at RLE and is now an assistant professor at the University of Southern California.

“ChatGPT is limited in its size by the power of today’s supercomputers. It’s just not economically viable to train models that are much bigger. Our new technology could make it possible to leapfrog to machine-learning models that otherwise would not be reachable in the near future,” said Dirk Englund, an associate professor in MIT’s Department of Electrical Engineering and Computer Science and leader of the work.

“We don’t know what capabilities the next-generation ChatGPT will have if it is 100 times more powerful, but that’s the regime of discovery that this kind of technology can allow.” Englund is also leader of MIT’s Quantum Photonics Laboratory and is affiliated with the RLE and the Materials Research Laboratory.

Additional coauthors of the current Nature Photonics paper are Alexander Sludds, Ronald Davis, Ian Christen, Liane Bernstein, and Lamia Ateshian, all of RLE; and Tobias Heuser, Niels Heermeier, James A. Lott, and Stephan Reitzensttein of Technische Universitat Berlin.

Chen, Hamerly, and Englund have filed for a patent on the work, which was sponsored by the US Army Research Office and NTT Research in Japan as well as the Volkswagen Foundation in Germany.