Scientists propose a new method to implement a neural network with an optical system, which could make machine learning more sustainable in the future. Researchers from the Max Planck Institute for the Science of Light have published their new method in Nature Physics, demonstrating a much simpler method than previous approaches.
Machine learning and artificial intelligence are becoming increasingly prevalent, with applications ranging from computer vision to text generation, as demonstrated by ChatGPT. However, these complex tasks require increasingly complex neural networks, some with billions of parameters. This rapid growth in the size of neural networks has put these technologies on an unsustainable trajectory due to their energy consumption and exponentially increasing training times. For example, it is estimated that training GPT-3 consumed over 1,000 MWh of energy, which is equivalent to the daily electricity consumption of a small city. This trend has created a need for faster, more energy-efficient, and cost-effective alternatives, leading to the burgeoning field of neuromorphic computing. The goal of this field is to replace the neural networks in our digital computers with physical neural networks. These are designed to physically perform the required mathematical operations in a potentially faster and more energy-efficient manner.
Optics and photonics are particularly promising platforms for neuromorphic computing, as energy consumption can be reduced to a minimum. Computations can be performed in parallel at very high speeds, limited only by the speed of light. However, two major challenges have arisen so far: first, performing the necessary complex mathematical calculations requires high laser powers. Second, the lack of an efficient general training method for such physical neural networks.
Both challenges can be overcome with the new method proposed by Clara Wanjura and Florian Marquardt from the Max Planck Institute for the Science of Light in their new paper in Nature Physics. “Normally, the input data is imprinted on the light field. However, in our new methods, we propose to imprint the input by changing the transmission of the light,” explains Florian Marquardt, director of the institute. In this way, the input signal can be processed arbitrarily. This is true even if the light field itself behaves in the simplest possible way, in which the waves interfere without influencing each other. Therefore, their approach avoids complicated physical interactions to realize the required mathematical functions that would otherwise require high-power light fields. The evaluation and training of this physical neural network would then become very simple: “We would actually only have to send light through the system and observe the transmitted light. This would allow us to evaluate the output of the network.” At the same time, this would allow us to measure all the information relevant to training,” explains Clara Wanjura, first author of the study. The authors demonstrated through simulations that their approach can be used to perform image classification tasks with the same accuracy as digital neural networks.
In the future, the authors plan to collaborate with experimental groups to study the implementation of their method. Since their proposal significantly alleviates the experimental requirements, it can be applied to many physically very different systems. This opens new possibilities for neuromorphic devices enabling physical training on a wide range of platforms.
/Public dissemination. This content from the original organization/authors may be of a timely nature and edited for clarity, style, and length. Mirage.News takes no institutional position or bias, and all views, positions, and conclusions expressed herein are solely those of the author(s). See the full story here.