Event segmentation in continuous, naturalistic videos from model-based, data-driven, and human perspectives

Human experience is characterised by an apparently continuous stream of information. However, when recalling our past, we commonly experience somewhat discrete episodes. Event segmentation has been suggested to provide a basis for episodic memory formation and recognition/recall. Previous work has suggested that event boundaries are reflected in shifts in stable patterns of neural activity across processing hierarchies, and that these boundaries can be detected using neuroimaging techniques. From a different direction, a recent series of studies in the domain of human time perception have shown that tracking event boundaries as "surprise" in network activity - in human perceptual processing or neural network proxies - successfully describes subjective reports of time on the scale of seconds to minutes. This project aims to understand how these methods of obtaining “event boundaries” perform relative to human-provided event annotations, in situations of life-like, naturalistic sensory stimulation. We compared the boundaries estimated from our neural-network-based model of time perception and a data-driven event segmentation approach (Greedy State Boundary Search on EEG data), against annotations from 131 human raters using two different inferential processes. In both methods, event boundaries from our model-based and the EEG data-driven approach performed similarly well, and were both indistinguishable from human provided boundaries in at least ~89% of cases examined. A random boundary generator was similar to human raters in only approximately 13% of cases. This work demonstrates the potential to reconcile the fields of time perception and episodic memory through a common foundation in the processes underlying event segmentation.