We use computational models to study the direct and indirect function of episodic memory in inference and prediction.
Memory retains information that is otherwise not available at the moment, but surprisingly little is known about how that information is used. Due to its reconstructive nature, episodic memory has been suggested to drive mental time travel into the future. But why are specific experiences stored in episodic memory helpful for predicting the future? Why can general, or semantic, information not serve the same purpose? These and further questions arise also because there are virtually no computational studies of the function of episodic memory. Here, we hypothesize that scenario construction during episodic memory retrieval aids inference and prediction. We view inference as the problem of finding objects, people or locations that we cannot sense at the moment, e.g. keys that we cannot see but need to take before leaving the house. We view prediction as forecasting a future state of the world based on past and present states of the world. Furthermore, we hypothesize that inference and prediction are also possible based on semantic information. So, we will contrast episodic with semantic strategies for inference and prediction. We will develop abstract computational models to study four operating modes. An artificial agent moves through a virtual environment or observes moving objects. Episodic memory will be modeled as sequences of feature frames extracted from input images that were taken from the agent's point of view. Inference based on episodic memory will be modeled as template matching: given the current input, the agent finds a feature frame in memory that is most similar to the input, then searches the memory for the desired information in temporal proximity. Prediction is modeled similarly, except a sequence of immediate past and current inputs has to be matched to memory sequences, and the next element in the memory sequence is returned as a prediction. To perform inference based on semantic information, first, an explicit model between the input and target information has to be learned. Given such a model, the target information can be computed for any given input. To perform prediction, a model of the dynamics of the environment has to be learned and evaluated to compute a prediction of the next state. We will study the memory requirements, test the role of the particularity and sequentiality of episodic memory, study under which conditions the episodic strategy is better than the semantic one and vice versa. We hypothesize that when there is little data, it is advantageous to use episodic memory, but the balance shifts when more and more data become available. This prediction will be tested in the model as well as experimentally in collaboration with projects in the research unit.