Project P2
Modeling the functional role of episodic memory in spatial learning
Memory retains information that is currently not available in the environment, but memory is not a purpose in itself for a living being. The stored information confer an evolutionary benefit on the organism. The function of memory in general, and episodic memory (EM) in particular, has received far less attention than the properties the memory processes and their underlying neural mechanisms. In this subproject, we focus on directive functions, which refers to cases where the retrieved memory is used to solve a practical problem. In the first funding period, we adopted reinforcement learning for modeling the functional role of EM in spatial learning. We considered three learning paradigms: 1. EM is retrieved to learn from single experiences (one-shot learning), 2. EM is replayed to facilitate learning of statistical regularities (replay learning), and 3. learning occurs online as experiences occur, but does not have access to past experiences (online learning). We found that whether an agent is able to solve a task by relying on the three learning paradigms depends differently on the number of learning trials and the complexity of the task. EM can confer a major benefit on spatial learning, but does not always do so, and its effect differs for the two modes of accessing episodic information. One-shot learning is initially faster than replay learning, but the latter reaches a better asymptotic performance. In the second funding period, we will build on this approach, incorporating more cognitive processes associated with EM and extending it to study memory encoding and consolidation. First, by separating image processing from the reinforcement learning component, we will study how the representational format of the inputs influences learning. In addition, we will integrate a model of semantic completion to study the functional role of the generativity of EM. Second, we will test our hypothesis that attentional selection of the most informative parts of the input speeds up learning and makes behavior more robust. We will also study how attentional selection of different representational formats might make learning more flexible. Third, we will use replay prioritization to generate a variety of replay types reported in the literature to study how replay might be optimized for different task demands. Fourth, we will investigate the functional implications of EM updating. We expect that, in a changing environment, it is beneficial to increase the impact of recent memories relative to more remote memories, which gives rise to memory updating. In summary, our research will advance our understanding of the function of EM, i.e., how it drives behavior. This is an important step towards elucidating the nature of EM.
Principal Investigator
Members
Former Members
-
Analysis of a Generative Model of Episodic Memory Based on Hierarchical VQ-VAE and TransformerReyhanian, S., Fayyaz, Z., & Wiskott, L.In Proceedings of the 33rd International Conference on Artificial Neural Networks (ICANN 2024), Lugano, Switzerland Springer Nature SwitzerlandHebbian Descent: A Unified View on Log-Likelihood LearningMelchior, J., Schiewer, R., & Wiskott, L.Neural Computation, 36(9), 1669–1712A neural network model for online one-shot storage of pattern sequencesMelchior, J., Altamimi, A., Bayati, M., Cheng, S., & Wiskott, L.PLOS ONE, 19(6), 1–28Interpretable Brain-Inspired Representations Improve RL Performance on Visual Navigation TasksLange, M., Engelhardt, R. C., Konen, W., & Wiskott, L.In eXplainable AI approaches for Deep Reinforcement LearningModularity in Nervous Systems - a Key to Efficient Adaptivity for Deep Reinforcement LearningSchilling, M., Hammer, B., Ohl, F. W., Ritter, H. J., & Wiskott, L.Cogn. Comput., 16(5), 2358–2373
2023
A Tutorial on the Spectral Theory of Markov ChainsSeabrook, E., & Wiskott, L.Neural Computation, 35(11), 1713–1796Learning to predict future locations with internally generated theta sequencesParra-Barrero, E., & Cheng, S.PLOS Computational Biology, 19(5), e1011101A model of hippocampal replay driven by experience and environmental structure facilitates spatial learningDiekmann, N., & Cheng, S.eLife, 12, e82301Solidity Meets Surprise: Cerebral and Behavioral Effects of Learning from Episodic Prediction ErrorsSiestrup, S., Jainta, B., Cheng, S., & Schubotz, R. I.Journal of Cognitive Neuroscience, 35(2), 291–313CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learningDiekmann, N., Vijayabaskaran, S., Zeng, X., Kappel, D., Menezes, M. C., & Cheng, S.Frontiers in Neuroinformatics, 17Improving Reinforcement Learning Efficiency with Auxiliary Tasks in Non-Visual Environments: A ComparisonLange, M., Krystiniak, N., Engelhardt, R. C., Konen, W., & Wiskott, L.In Proc. 9th International Conference on Machine Learning, Optimization and Data science (LOD)A map of spatial navigation for neuroscienceParra-Barrero, E., Vijayabaskaran, S., Seabrook, E., Wiskott, L., & Cheng, S.Neuroscience & Biobehavioral Reviews, 152, 105200Modeling the function of episodic memory in spatial learningZeng, X., Diekmann, N., Wiskott, L., & Cheng, S.Frontiers in Psychology, 142022
Where was the toaster? A systematic investigation of semantic construction in a new virtual episodic memory paradigmZoellner, C., Klein, N., Cheng, S., Schubotz, R. I., Axmacher, N., & Wolf, O. T.Quarterly Journal of Experimental Psychology, 174702182211166A Model of Semantic Completion in Generative Episodic MemoryFayyaz, Z., Altamimi, A., Zoellner, C., Klein, N., Wolf, O. T., Cheng, S., & Wiskott, L.Neural Computation, 34(9), 1841–1870What Happened When? Cerebral Processing of Modified Structure and Content in Episodic CueingSiestrup, S., Jainta, B., El-Sourani, N., Trempler, I., Wurm, M. F., Wolf, O. T., et al.Journal of Cognitive Neuroscience, 34(7), 1287–1305Seeing What I Did (Not): Cerebral and Behavioral Effects of Agency and Perspective on Episodic Memory Re-activationJainta, B., Siestrup, S., El-Sourani, N., Trempler, I., Wurm, M. F., Werning, M., et al.Frontiers in Behavioral Neuroscience, 152021
The computational benefits of episodic memory in spatial learningNeuronal sequences during theta rely on behavior-dependent spatial mapsParra-Barrero, E., Diba, K., & Cheng, S.eLife, 10Self-referential false associations: A self-enhanced constructive effect for verbal but not pictorial stimuliThe research unit FOR 2812 "Constructing scenarios of the past: A new framework in episodic memory" is a project funded by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG). The research unit studies the cognitive and neuronal mechanisms underlying scenario construction in episodic memory. We employ and integrate approaches from Philosophy, Psychology, and Experimental and Computational Neuroscience.
Universitätsstr. 150,
44801 Bochum, GermanyTel: +49 (0)234 32 27996
Fax: +49 (0)234 32 14210
