A Deep Generative Model imitating Predictive Coding in the Human Brain


In recent years, the development of deep generative models for predic- tion has been attracting attention. In our study, we focus on predictive coding, a concept from the neuroscience literature that hypothesizes the brain is constantly making predictions of sensory input, and attempt to develop a human brain-like prediction model. PredNet is a deep learning model based on the concept of predictive coding, however, it can predict the next image with the same prediction interval, but not with any intervals. On the other hand, Temporal Differential Variational Auto-Encoder (TD-VAE) can predict the next images with any predic- tion intervals, although it is not a model reflecting human brain function. In this work, we develop a new human brain-like prediction model by unifying PredNet and TD-VAE, combining both predictive coding and flexible interval prediction abilities in one single model. Through experiments on the KITTI Vision Bench- mark, we confirmed that our proposed model can predict the next images correctly with flexible prediction intervals. We also investigated the correlation between the feature values of representation layers in the model architecture and human brain activity data evoked under natural video stimulation.

2021-12-15 — 2021-12-18