A Multimodal Hypertensor Architecture for Association Formation

Several fields of study have developed sophisticated techniques to process signals of varying types. For example, pattern recognition techniques may seek to convert an audio waveform to a text transcription or to process an image and classify the represented object. However, much of this research focuses upon a single signal type. While such an approach has many useful applications, there are many domains which receive an influx of differing signal types. The brain receives a variety of sensory input signals and although each input stream does receive its share of individual processing, additional insight also comes from the converged processing of all input modalities. Such an occurrence takes place within the medial temporal lobe (MTL) and hippocampus. Consequently, we have designed a neural network architecture inspired by hippocampus connectivity and functionality which associates multimodal inputs.