Project P5
Computational modeling of generative episodic memory
Despite the large number of experimental and conceptual studies that have suggested that episodic memory is generative, computational models almost exclusively adopt the storage view. In this project, we develop a generative model for the encoding and retrieval of personally experienced episodes, which describes the interplay between hippocampus and neocortex.
The model consists of (a) a perceptual-semantic network that is hierarchically structured and gradually transforms perceived images into a more semantic representation, and (b) a semantic network that is able to complement incomplete semantic representations in a plausible way in a recurrent process. The former is realized by a 'vector quantized variational autoencoder (VQ-VAE)', the latter by a 'pixel convolutional neural network (PixelCNN)'. When an episode is encoded, the VQ-VAE first converts it into a semantic representation, part of which is then stored. When attention is high, a large part is stored; when attention is low, only a small part is stored. During recall, this part is read out again and plausibly completed by the PixelCNN. The VQ-VAE can then be applied backwards and reconstruct a concrete episode from the complete semantic representation.
So far, we use single images of handwritten digits on different backgrounds as episodes; the digits represent objects in different variants, the backgrounds represent the context, e.g. the room in which the object can be found. Objects or digits are preferably found in certain contexts or in front of certain backgrounds, e.g. a toaster in the kitchen (congruent context) and not in the bathroom (incongruent context) or in our simulation a '2' in front of a background with triangles and not squares. We have already reproduced the following experimental results with the model: (i) higher attention improves episodic memory, (ii) objects in congruent context are better remembered than in incongruent context, and (iii) if the correct context is not remembered for an object, at least a semantically congruent context is usually remembered.
Episodic memory is not reliable and can be modified by many influences. For example, we do not like to remember situations that were embarrassing to us, and we like to bring our memories more in line with the image we have of ourselves in retrospect. Conversely, our memories naturally influence our self-image. It is also known that our episodic memory can be altered by social interaction. In particular, we tend to align memories with opinions of our interaction partners when we feel connected to them. These aspects are the subject of further research on our model in cooperation with philosophers who are thinking about the self-model and with psychologists who are doing experiments on the influence of social interaction on memory.
Principal Investigator
Members
-
Analysis of a Generative Model of Episodic Memory Based on Hierarchical VQ-VAE and TransformerReyhanian, S., Fayyaz, Z., & Wiskott, L.In Proceedings of the 33rd International Conference on Artificial Neural Networks (ICANN 2024), Lugano, Switzerland Springer Nature SwitzerlandHebbian Descent: A Unified View on Log-Likelihood LearningMelchior, J., Schiewer, R., & Wiskott, L.Neural Computation, 36(9), 1669–1712A neural network model for online one-shot storage of pattern sequencesMelchior, J., Altamimi, A., Bayati, M., Cheng, S., & Wiskott, L.PLOS ONE, 19(6), 1–28Interpretable Brain-Inspired Representations Improve RL Performance on Visual Navigation TasksLange, M., Engelhardt, R. C., Konen, W., & Wiskott, L.In eXplainable AI approaches for Deep Reinforcement LearningModularity in Nervous Systems - a Key to Efficient Adaptivity for Deep Reinforcement LearningSchilling, M., Hammer, B., Ohl, F. W., Ritter, H. J., & Wiskott, L.Cogn. Comput., 16(5), 2358–2373
2023
A Tutorial on the Spectral Theory of Markov ChainsSeabrook, E., & Wiskott, L.Neural Computation, 35(11), 1713–1796Learning to predict future locations with internally generated theta sequencesParra-Barrero, E., & Cheng, S.PLOS Computational Biology, 19(5), e1011101A model of hippocampal replay driven by experience and environmental structure facilitates spatial learningDiekmann, N., & Cheng, S.eLife, 12, e82301Solidity Meets Surprise: Cerebral and Behavioral Effects of Learning from Episodic Prediction ErrorsSiestrup, S., Jainta, B., Cheng, S., & Schubotz, R. I.Journal of Cognitive Neuroscience, 35(2), 291–313CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learningDiekmann, N., Vijayabaskaran, S., Zeng, X., Kappel, D., Menezes, M. C., & Cheng, S.Frontiers in Neuroinformatics, 17Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A ComparisonLange, M., Krystiniak, N., Engelhardt, R. C., Konen, W., & Wiskott, L.In Proc. 9th International Conference on Machine Learning, Optimization and Data science (LOD)A map of spatial navigation for neuroscienceParra-Barrero, E., Vijayabaskaran, S., Seabrook, E., Wiskott, L., & Cheng, S.Neuroscience & Biobehavioral Reviews, 152, 105200Modeling the function of episodic memory in spatial learningZeng, X., Diekmann, N., Wiskott, L., & Cheng, S.Frontiers in Psychology, 142022
Where was the toaster? A systematic investigation of semantic construction in a new virtual episodic memory paradigmZoellner, C., Klein, N., Cheng, S., Schubotz, R. I., Axmacher, N., & Wolf, O. T.Quarterly Journal of Experimental Psychology, 174702182211166A Model of Semantic Completion in Generative Episodic MemoryFayyaz, Z., Altamimi, A., Zoellner, C., Klein, N., Wolf, O. T., Cheng, S., & Wiskott, L.Neural Computation, 34(9), 1841–1870What Happened When? Cerebral Processing of Modified Structure and Content in Episodic CueingSiestrup, S., Jainta, B., El-Sourani, N., Trempler, I., Wurm, M. F., Wolf, O. T., et al.Journal of Cognitive Neuroscience, 34(7), 1287–1305Seeing What I Did (Not): Cerebral and Behavioral Effects of Agency and Perspective on Episodic Memory Re-activationJainta, B., Siestrup, S., El-Sourani, N., Trempler, I., Wurm, M. F., Werning, M., et al.Frontiers in Behavioral Neuroscience, 152021
The computational benefits of episodic memory in spatial learningNeuronal sequences during theta rely on behavior-dependent spatial mapsParra-Barrero, E., Diba, K., & Cheng, S.eLife, 10Self-referential false associations: A self-enhanced constructive effect for verbal but not pictorial stimuliThe research unit FOR 2812 "Constructing scenarios of the past: A new framework in episodic memory" is a project funded by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG). The research unit studies the cognitive and neuronal mechanisms underlying scenario construction in episodic memory. We employ and integrate approaches from Philosophy, Psychology, and Experimental and Computational Neuroscience.
Universitätsstr. 150,
44801 Bochum, GermanyTel: +49 (0)234 32 27996
Fax: +49 (0)234 32 14210
