Cognitive control in reinforcement learning : brain structure and function
Author: Schiffler, Björn C
Date: 2017-12-15
Location: Hillarpsalen, Retzius väg 8, Karolinska institutet, Solna
Time: 09.00
Department: Inst för klinisk neurovetenskap / Dept of Clinical Neuroscience
View/ Open:
Thesis (5.065Mb)
Abstract
Adapting behaviour according to internal or external feedback is a fundamental property of cognitive control. For example, humans tend to slow down when they make mistakes, a process called post-error slowing (PES), which has previously received extensive attention in research on response inhibition. However, whether PES is actually an adaptive process which helps avoid future mistakes or a maladaptive one which siphons cognitive resources is still not clear. The overall aim of the work in this thesis was to investigate how post-error slowing contributes to the stabilization of performance after errors and which brain areas are involved in this response inhibition process. We did this by combining behavioural experiments, computational modelling and neuroimaging techniques to provide a comprehensive analysis of latent decision processes and their neural correlates.
Specifically, in Study I, we analyzed data from a probabilistic reinforcement learning task in combination with functional Magnetic Resonance Imaging to explore which brain regions signalled enhanced future post-error slowing when receiving negative feedback. On a behavioural level, we studied whether PES was associated with how well participants learned, as assessed in a later test phase. We showed that post-error slowing was associated with brain activity in a central cognitive control region, the right inferior frontal gyrus (rIFG) as well as brain regions in occipital cortex which overlapped with the representation of absolute prediction errors, a measure reflecting deviance from expectations, i.e., surprise at feedback. In Study II, we found that cortical thickness in rIFG as a measure of grey matter integrity was related to inter-individual differences in post-error slowing, both for direct next trials and trials further apart in time. This analysis was supported by a drift diffusion model of the underlying decision processes, which demonstrated that an increased decision boundary after an error, indicating enhanced response caution, was related to cortical thickness variability, particularly in anterior parts of the rIFG. Finally, in Study III we used drift diffusion modelling on a large-scale behavioural dataset during a visual search task to illuminate decision processes of up to five trials after an error and how post-error adaptation benefits accuracy recovery several trials after the error. Post-error slowing was marked by both adaptive and non-adaptive decision processes which changed dynamically over several trials after an error. While adaptive increases in decision threshold were sustained for several trials after an error, reductions in evidence accumulation only transiently affected performance on the next trial after the error. Further, post-error increases in response caution and evidence accumulation were also associated with better performance on future trials.
These studies illustrate that there is valuable information to be gained about response inhibition processes beyond looking at the simple relation of post-error slowing and accuracy. Computational modelling allowed us to compartmentalize various decision processes and relate adaptations in these processes directly to brain anatomy. We hope the results from the studies presented in this thesis can provide a framework for future work on how the brain learns from mistakes and adapts to a continually changing environment.
Specifically, in Study I, we analyzed data from a probabilistic reinforcement learning task in combination with functional Magnetic Resonance Imaging to explore which brain regions signalled enhanced future post-error slowing when receiving negative feedback. On a behavioural level, we studied whether PES was associated with how well participants learned, as assessed in a later test phase. We showed that post-error slowing was associated with brain activity in a central cognitive control region, the right inferior frontal gyrus (rIFG) as well as brain regions in occipital cortex which overlapped with the representation of absolute prediction errors, a measure reflecting deviance from expectations, i.e., surprise at feedback. In Study II, we found that cortical thickness in rIFG as a measure of grey matter integrity was related to inter-individual differences in post-error slowing, both for direct next trials and trials further apart in time. This analysis was supported by a drift diffusion model of the underlying decision processes, which demonstrated that an increased decision boundary after an error, indicating enhanced response caution, was related to cortical thickness variability, particularly in anterior parts of the rIFG. Finally, in Study III we used drift diffusion modelling on a large-scale behavioural dataset during a visual search task to illuminate decision processes of up to five trials after an error and how post-error adaptation benefits accuracy recovery several trials after the error. Post-error slowing was marked by both adaptive and non-adaptive decision processes which changed dynamically over several trials after an error. While adaptive increases in decision threshold were sustained for several trials after an error, reductions in evidence accumulation only transiently affected performance on the next trial after the error. Further, post-error increases in response caution and evidence accumulation were also associated with better performance on future trials.
These studies illustrate that there is valuable information to be gained about response inhibition processes beyond looking at the simple relation of post-error slowing and accuracy. Computational modelling allowed us to compartmentalize various decision processes and relate adaptations in these processes directly to brain anatomy. We hope the results from the studies presented in this thesis can provide a framework for future work on how the brain learns from mistakes and adapts to a continually changing environment.
List of papers:
I. Schiffler BC, Almeida R, Granqvist M, Bengtsson SL. (2016). Memory-reliant post-error slowing is associated with successful learning and fronto-occipital activity. Journal of Cognitive Neuroscience. 28, 1539-1552.
Fulltext (DOI)
Pubmed
View record in Web of Science®
II. Schiffler BC, Abé C, & Bengtsson SL. (2017). Cortical thickness of right inferior frontal cortex is associated with memory-based post-error slowing. [Manuscript]
III. Schiffler BC, Bengtsson SL†, & Lundqvist D†. (2017). The Sustained Influence of an Error on Future Decision-Making. Frontiers in Psychology. 8, 1077.
Fulltext (DOI)
Pubmed
View record in Web of Science®
I. Schiffler BC, Almeida R, Granqvist M, Bengtsson SL. (2016). Memory-reliant post-error slowing is associated with successful learning and fronto-occipital activity. Journal of Cognitive Neuroscience. 28, 1539-1552.
Fulltext (DOI)
Pubmed
View record in Web of Science®
II. Schiffler BC, Abé C, & Bengtsson SL. (2017). Cortical thickness of right inferior frontal cortex is associated with memory-based post-error slowing. [Manuscript]
III. Schiffler BC, Bengtsson SL†, & Lundqvist D†. (2017). The Sustained Influence of an Error on Future Decision-Making. Frontiers in Psychology. 8, 1077.
Fulltext (DOI)
Pubmed
View record in Web of Science®
Institution: Karolinska Institutet
Supervisor: Bengtsson, Sara L.
Co-supervisor: Ingvar, Martin; Fransson, Peter
Issue date: 2017-11-23
Rights:
Publication year: 2017
ISBN: 978-91-7676-883-9
Statistics
Total Visits
Views | |
---|---|
Cognitive ...(legacy) | 393 |
Cognitive ... | 254 |
Total Visits Per Month
October 2023 | November 2023 | December 2023 | January 2024 | February 2024 | March 2024 | April 2024 | |
---|---|---|---|---|---|---|---|
Cognitive ... | 3 | 5 | 2 | 0 | 1 | 3 | 2 |
File Visits
Views | |
---|---|
Thesis_Bjoern_C_Schiffler.pdf(legacy) | 434 |
Thesis_Bjoern_C_Schiffler.pdf | 274 |
Top country views
Views | |
---|---|
United States | 178 |
Sweden | 126 |
Germany | 47 |
China | 38 |
United Kingdom | 21 |
Netherlands | 20 |
India | 15 |
South Korea | 12 |
France | 10 |
Ireland | 10 |
Top cities views
Views | |
---|---|
Ashburn | 47 |
Woodbridge | 14 |
Stockholm | 13 |
Houston | 11 |
Beijing | 10 |
Seoul | 9 |
Dublin | 8 |
Amsterdam | 5 |
Solna | 5 |
Wilmington | 5 |