Abstract: A large body of work supports the idea that there are two mutually complementary control systems in the brain constantly competing for executive control: 1) a procedural control system, involving the dorsal striatum, which can be understood in terms of the mathematics of a value function, the Bellman equation, dynamic programming, value back-propagation, TD learning, etc. And 2) a deliberative control system, involving prefrontal cortex (PFC) and the ventral striatum, which bases its decisions on the sequential consideration of alternatives. Deliberative control is noteworthy for being model-based, fast to learn, and thus flexible; making it superior to procedural control in volatile environments.
In this talk I’ll discuss a model in which the hippocampus (HC) forms an intimate part of the deliberative control system; namely as part of a hierarchical planning system. In this model, PFC produces a high-level plan, and then consults with HC in order to sequentially refine each step of the plan. The model formalizes the computational task of HC as an optimal control problem. I will review various kinds of "representational sequences" observed in HC, and describe the computational role the model ascribes them in solving the two-point boundary value problem which emerges from Pontryagin's maximum principle. The model has the potential to explain in a unified manner an array of disparate results, ranging from the effects of learning on hippocampal sequences, to an assortment of lesion studies in HC and PFC, to the role of HC in enabling flexible behavior.