A multichannel optical neural network for advanced machine vision

Abstract Endowed with the superior computing speed and energy efficiency, all-optical and optoelectronics neural networks (ONNs) have attracted ever-growing attention in recent years. Previous studies lacking viable way for multichannel optical processing, mainly implement single-channel optical computing to solve simple tasks including hand-written digit classification, saliency detection, etc. Aiming for more powerful ONNs to solve complex tasks, we innovatively develop Monet: a multichannel optical neural network for advanced machine vision. The inter- and intra- channel connections are mapped to optical interference and diffraction. By further designing a novel projection-interference-prediction framework on the basis of these connections, optical interference patterns are generated by projecting and interfering the multichannel inputs in a shared domain. These patterns encoding the correspondences together with feature embeddings are iteratively produced through the projection-interference process to predict the final output. Unlike existing ONNs trying to inherit ANN architectures, the framework of Monet is initially designed following the innate characters of light propagation and interaction, to fully explore the potential of wave optics for optical computing. For the first time, we validate that Monet is capable for advanced machine vision tasks such as 3D depth estimation and moving objection detection. Moreover, a prototype system of Monet is developed to accomplish 3D perception for real-world scenarios. We anticipate that the proposed technique will accelerate the development of more powerful optical AI as critical support for modern advanced machine vision.