Memory that lasts

Catastrophic forgetting, the tendency of learning systems to lose previously acquired knowledge when trained on new information, remains one of the most fundamental challenges in continual learning. This dissertation contributes to a deeper understanding of why catastrophic forgetting occurs and investigates the mechanisms that enable learning models to maintain stability while acquiring new knowledge over time. The core of this work centers on the Cobweb framework, a psychologically inspired concept formation model that incrementally builds a hierarchy of concepts. A novel extension of this framework, Cobweb/4V, is introduced and systematically evaluated for its capacity to learn visual concepts in a continual manner without suffering from catastrophic forgetting. Motivated by the observed robustness of Cobweb/4V under continual learning, this research examines three hypotheses regarding factors that may contribute to such stability: (1) the adaptive structure hypothesis, which suggests that the ability to dynamically restructure the concept hierarchy as new data arrive supports continual learning by flexibly allocating representational capacity; (2) the sparse update hypothesis, which suggests that restricting parameter updates to localized regions of the model, rather than applying global changes, helps preserve prior knowledge by reducing representational overlap; and (3) the information-theoretic hypothesis, which proposes that employing closed-form updates based on sufficiency statistics enables precise, incremental learning without revisiting past data, offering potential advantages over gradient-based methods when previous data are unavailable. These hypotheses are evaluated through comparative experiments on datasets of varying complexity. By combining cognitive inspiration with controlled empirical evaluations, this dissertation offers new insights into the dynamics of memory retention and interference. These contributions not only clarify the foundations of catastrophic forgetting, but also point toward promising directions for developing continual learning systems that are both stable and adaptable.