On learning in mice and machines : continuous population codes in natural and artificial neural networks
Author: Wärnberg, Emil
Date: 2023-11-24
Location: Sal E1, KTH Royal Institute of Technology, Lindstedtsvägen 3, Stockholm
Time: 13.00
Department: Inst för neurovetenskap / Dept of Neuroscience
View/ Open:
Thesis (5.260Mb)
Abstract
Neural networks, whether artificial in a computer or natural in the brain, could
represent information either using discrete symbols or continuous vector
spaces. In this thesis, I explore how neural networks can represent continuous
vector spaces, using both simulated neural networks and analysis of real neural
population data recorded from mice. A special focus is on the networks of the
basal ganglia circuit and on reinforcement learning, i.e., learning from rewards and
punishments.
The thesis includes four scientific papers: two theoretical/computational (Papers I and IV) and two with analysis of real data (Papers II and III).
In Paper I, we explore methods for implementing continuous vector spaces in networks of spiking neurons using multidimensional attractors, and propose an explanation for why it is hard to escape the neural manifolds created by such attractors.
In Paper II, we analyze experimental data from dorsomedial striatum collected using 1-photon calcium imaging of transgenic mice with celltype-specific markers for the striatal direct, indirect and patch pathways, as the mice were gathering rewards in a 2-choice task. In line with extensive previous results, our data analysis revealed a number of neural signatures of reinforcement learning, but no apparent difference between the pathways.
In Paper III, we present a new software tool for tracking neurons across weeks of 1-photon calcium imaging, and employ it to follow patch-specific striatal projection neurons from the dorsomedial striatum across two weeks of daily recordings.
In Paper IV, we propose a model for how the nigrostriatal dopaminergic projection could, in a biologically plausible way, convey a vector-valued error gradient to the dorsal striatum, as required for backpropagation.
Based on the results of the papers and a review of existing literature, I argue that while the basal ganglia indeed make up a circuit for reinforcement learning as previously thought, this circuit represents reinforcement learning states, actions and policies using a continuous population code and not using discrete symbols.
The thesis includes four scientific papers: two theoretical/computational (Papers I and IV) and two with analysis of real data (Papers II and III).
In Paper I, we explore methods for implementing continuous vector spaces in networks of spiking neurons using multidimensional attractors, and propose an explanation for why it is hard to escape the neural manifolds created by such attractors.
In Paper II, we analyze experimental data from dorsomedial striatum collected using 1-photon calcium imaging of transgenic mice with celltype-specific markers for the striatal direct, indirect and patch pathways, as the mice were gathering rewards in a 2-choice task. In line with extensive previous results, our data analysis revealed a number of neural signatures of reinforcement learning, but no apparent difference between the pathways.
In Paper III, we present a new software tool for tracking neurons across weeks of 1-photon calcium imaging, and employ it to follow patch-specific striatal projection neurons from the dorsomedial striatum across two weeks of daily recordings.
In Paper IV, we propose a model for how the nigrostriatal dopaminergic projection could, in a biologically plausible way, convey a vector-valued error gradient to the dorsal striatum, as required for backpropagation.
Based on the results of the papers and a review of existing literature, I argue that while the basal ganglia indeed make up a circuit for reinforcement learning as previously thought, this circuit represents reinforcement learning states, actions and policies using a continuous population code and not using discrete symbols.
List of papers:
I. Wärnberg & Kumar. (2019). Perturbing low dimensional activity manifolds in spiking neuronal networks. PLoS Computational Biology. 15(5).
Fulltext (DOI)
Pubmed
View record in Web of Science®
II. Weglage*, Wärnberg*, Lazaridis*, Calvigioni, Tzortzi & Meletis. (2021). Complete representation of action space and value in all dorsal striatal pathways. Cell Reports 36. *Equal contribution.
Fulltext (DOI)
Pubmed
View record in Web of Science®
III. Wärnberg, Skara, Mantas & Meletis. Tracking activity of neurons across weeks from 1-photon calcium imaging. [Manuscript]
IV. Wärnberg & Kumar. (2023). Feasibility of dopamine as a vectorvalued feedback signal in the basal ganglia. Proceedings of the National Academy of Sciences (PNAS). 120(32).
Fulltext (DOI)
Pubmed
I. Wärnberg & Kumar. (2019). Perturbing low dimensional activity manifolds in spiking neuronal networks. PLoS Computational Biology. 15(5).
Fulltext (DOI)
Pubmed
View record in Web of Science®
II. Weglage*, Wärnberg*, Lazaridis*, Calvigioni, Tzortzi & Meletis. (2021). Complete representation of action space and value in all dorsal striatal pathways. Cell Reports 36. *Equal contribution.
Fulltext (DOI)
Pubmed
View record in Web of Science®
III. Wärnberg, Skara, Mantas & Meletis. Tracking activity of neurons across weeks from 1-photon calcium imaging. [Manuscript]
IV. Wärnberg & Kumar. (2023). Feasibility of dopamine as a vectorvalued feedback signal in the basal ganglia. Proceedings of the National Academy of Sciences (PNAS). 120(32).
Fulltext (DOI)
Pubmed
Institution:
- Karolinska Institutet
- KTH Royal Institute of Technology
Supervisor: Meletis, Konstantinos
Co-supervisor: Kumar, Arvind; Silberberg, Gilad
Issue date: 2023-10-30
Rights:
Publication year: 2023
ISBN: 978-91-8017-152-6
Statistics
Total Visits
Views | |
---|---|
On ... | 222 |
Total Visits Per Month
October 2023 | November 2023 | December 2023 | January 2024 | February 2024 | March 2024 | April 2024 | |
---|---|---|---|---|---|---|---|
On ... | 54 | 113 | 14 | 15 | 14 | 8 | 4 |
File Visits
Views | |
---|---|
Thesis_Emil_Wärnberg.pdf | 315 |
Top country views
Views | |
---|---|
Sweden | 92 |
United States | 39 |
China | 19 |
Germany | 8 |
South Korea | 5 |
Singapore | 4 |
Austria | 3 |
United Kingdom | 3 |
Japan | 3 |
Netherlands | 3 |
Top cities views
Views | |
---|---|
Stockholm | 20 |
Ashburn | 6 |
Sundbyberg | 6 |
Bandhagen | 5 |
Malmo | 5 |
Beijing | 4 |
Eskilstuna | 4 |
Hamburg | 4 |
Taby | 4 |
Bromma | 3 |