Since the NeuroMatch Academy computational neuroscience 2021 has ended, I think it’d be nice to wrap up what we have done for the team project. Together with brilliant Ziyan Huang & Ming-Chin Chiang, mentored by Josh Siegle, Gaurang Mahajan and Jixing Li.
This was originally planned to be a twitter thread but I obviously over-doing it a bit...so blog it is!
For the team project of the NMA CN, our goal is to find a dataset that you like (eg. the Stringer dataset on Ca2+ imaging in visual cortex, Allen Institute dataset on Ca2+ imaging in many cortices, the Steinmetz neuropixel dataset...etc. There are also other human related datasets in fMRI & EEG but I am not very familiar with them) and formulate a question/hypothesis that you want to answer/test (which IS the most important part!). And based on your question, design a computational model that can help you (hopefully) answer it.
Since we three are both systems neuroscientists working with neuronal data, we chose to work on the Steinmetz neuropixel dataset (btw it’s super well-organized!), which describes a visual task that mice have to compare the visual stimuli and perform 1 of 3 behaviors, turn clockwise/turn counter-CW/standstill. In most cases, mice’s performance is great (~65-90% accuracy, depending on the task difficulty), but there are quite a few trials in which mice received the “Go” visual stim but didn’t do anything. In the original paper, the authors have illustrated the neural responses in different trial types (Go/Missed/passive), and there are differences in baselines (engagement level) and action onsets (distributed action coding). However, the difference in the instantaneous visual stimulus responses was not focused, which can be clearly observed in some brain reigns (eg. Caudoputamen [basal ganglia]).
Fig. 1 Steinmetz sup fig (modified), visual responses are different in Go vs Missed trials (V1, CP, M1), highlighted with instantaneous visual responses.
We found it very interesting and wondered if there is something fundamentally different when the mouse received the same/similar visual stimuli; means neural responses to the visual stimuli are already different (eg. activating different sets of neurons or difference in firing rates). So we have our question/hypothesis now: whether neural responses upon the same visual cue can indicate behavioral outcomes (Go/Miss trials)
So how do we test the hypothesis?
We started with pooling the neurons by different brain areas and matched their behavioral outcomes: “Go trials”: turn counter-clockwise (responding to the right>left stimuli) or “Missed trials” that had no movement. We focused on the “Right” trials where the contrast of right stimulus is higher than the left stimulus, since the recordings were performed on the left hemisphere, so we will recover more of instant visual responses than action coding.
Fig. 2 Cartoon for "Right" trials, that the contrast on the right is higher than the left ones/
And to test if we can use neural activity to predict the behavior, we design an SVM with a sliding window. Ideally, we should see something like this (please bear with my terrible hand-drawing):
Fig. 3 Imaginary decoder results
We may see that in some brain regions decoder can already predict the actions based on the responses to visual stimuli. And also we should see the accuracy rising when the action onset happens, which can be used as a good positive control.
Fig. 4 Design matrix for our SVM decoder
Our design matrix would look like this for each brain area: Features are the mean individual neural firing rate in the given time bin, and observations are trials. We pooled all recording sessions with >30 Right Go trials and >15 Right Missed trials, and to balance the trials numbers, we randomly select 15 Go trials for the SVM. The SVM scores are averaged firstly with 5-fold cross-validation, and then repeat 100 times to reduce the random selection biases. We also perform shuffled controls to make sure our code is working correctly and there is no internal biases being introduced.
Fig. 5 Average firing rate across different brain regions (pooled among recording sessions)
By looking at the average firing rate, not surprisingly, we can’t see much difference during the stimulus responses period (0.1 sec after stimulus onset, highlighted in the fig). But can we distinguish them from individual neurons?
Fig. 6 SVM result: Almost all brain regions encode future behaviors (but why is that?)
And here is the result!
Yes, we can, but it’s a bit weird. As predicted, we can see really high decoding accuracy after-action onset happens (after 0.1 sec of visual stimulus). And we also observed that neural responses right after the stimulus onset (0-0.1 sec) can predict whether mice will Go or Miss. But wait, what’s wrong with the baseline period??? Why the accuracy is so high???
Fig. 7 Top/bottom coefficient neurons in baseline period show high correlations with future actions.
We extracted those neurons with the highest/lowest SVM coefficients (2 std away from mean) during the baseline period and found that the top/bottom coef neurons have very different firing rates during the baseline period already. This resonates with the “engagement index” in the original paper, that when mice are more engaged with the task, the cortex is more silent, and subcortical regions are more activated. But there is more. In the original paper, the authors compared the baseline firing rate when mice were engaged with the task but missed the trial or just passively receiving the visual stimuli (not engaged), and they found a general trend that when engaged, the baseline firing rate is lower in cortices, but higher in subcortical regions. (see fig below with highlights)
Fig. 8 Baseline firing rate difference when mice are (not) engaged with visual task, adapted from Steinmetz Nature 2019.
The authors suggested that when mice are engaged, a lower baseline firing rate can increase the SNR in the cortex, and higher baseline firing rates make closer to thresholds for relaying vis inputs to the cortex. In our analysis, we are comparing a slightly difference conditions (Missed vs Passive or Go vs Missed) than in what the original paper did, we can also find this low-when-engaged cortical neurons and high-when-engaged subcortical neurons, that potentially indicates the engagement level of the animal. But we can also observed an opposite trend here: neurons in the bottom coef cortical neurons, that showed almost opposite baseline features of firing rates (that is high-when-engaged in cortical neurons). And in the high coef subcortical neurons, we can find low-when-engaged neurons as well.
Additionally, in some brain regions, like basal ganglia, the top coef neurons are not really responding to the visual cues, resulting in almost two flat lines that are hard to explain their contribution to the engagement levels. Despite we found them difficult to interpret how these Top/Bottom coef baseline neurons contributed to the visual task, one thing we can try is to drop them completely and see if the remaining neurons can still have the ability to determine future actions. If so, then maybe they are not as important as them seem.
So, to further concentrate our testing hypothesis (visual neural responses during stimulus onset for predicting behaviors), we excluded Top/Bottom coef neurons and re-analyzed the non-selected neurons with the SVM approach. Note that the Top/Bottom coef neurons are roughly only ~5% of the original neuron population, so after exclusion we still preserved ~95% of them.
Fig. 9 SVM again without baseline neurons
This time, we can observe the baseline decoding accuracy, not surprisingly, dropped to close chance level, and only started rising when stimulus onset happened. This finding is supporting our hypothesis that the difference in visual responses can be used to predict whether mice will perform the task or just let it go. Surprisingly, this decision encoding can be found in many brain regions, similar to the distributed action coding from the original paper.
Fig. 10 Top & bottom coef neurons upon visual stimulus onset
We can pick up the responding neurons again based on their coef of the SVM during the stimulus onset period, and identify neurons that are similar in their baseline firing rate, but responded differently in Go and Missed trials. Again this difference in vis responses can be observed in either direction; higher/lower in Go trials and lower/higher in Missed trials. One thing we also notice from the analysis is that those Top/Bottom neurons usually have a higher firing rate (peak at ~25 Hz) compared to the non-selected neurons which were ~ 3 Hz, which potentailly suggests those Top/Bottom coef neurons are fast-spiking interneurons, but we didn't check the spike autocorrelogram for each neurons so we can't be sure about this.
Fig. 11 Zoom-in comparison (decoder with and without baseline top/bottom coef neurons), which we can see the decoding accuracy only rise when the visual stimuli comes in.
Now we have proved our hypothesis, the story ends, right?
Well not exactly.
How do we make sense of this result? Why similar stimuli can activate different sets of neurons and having influences on the future behaviors?
Brain states. This is very possible that when the animal is in a different brain state determines a lot of their potential actions. The engagement level is one explanation here that the baseline firing rates have limited the potential outcomes way before the stimulus comes in. Although by selecting the top/bottom coef neurons during the baselines, we can’t observe any major difference for the vis stimuli responses in different trial types, it’s still possible that the baseline differences are supporting/thresholding other neurons responses, despite being rather neutral to the vis stimuli.
Trial-to-trial variability. In our analysis, we pooled trials with similar conditions together and examine what are the features differences caused/correlated with the actions. In another way of speaking, we are trying to found the trial-to-trial differences in neural activity that reflect the future actions in those similar trials. In a recent paper (Nieh & Schottdorf et al., Nature, 2021, see Fig. 11 below), they describe that CA1 neuron activity can form a cognitive map of mice position vs. accumulated evidence in the visual task. In similar trials when mice made the decision to turn right (when evidence is right > left) at the end of the virtual track, different sets of neurons are activated corresponding to the level of evidence they have received. Here, we proposed that this can be a similar mechanism to what we observed in our analysis. The subset of neurons that are activated during the vis onset, determines the probability of future actions. This learned task has its own cognitive map and when different sets of neurons are activated, it’s “trapped” to go down the path that’s sort of predefined. While in the Nieh & Schottdorf paper the map has axes of the mouse position and evidence, in the Steinmetz dataset it’s probably axes of evidence (or difficulty) and the engagement level. It would be very cool to visualize that, tho we didn’t manage to do further analysis during the summer school.
It’s all just correlation, the real cause is among other factors. Here, we provided some evidence suggesting that neural activity upon vis stim can already influence the future action, but it is however hardly causal. We can find rich and informative representations in neural activities, but this does not mean the brain has access to them. There are obviously some other factors that could be involved in this visual task, for example, the arousal level (pupil size), brain state (LFP), reward history, wheel speed...etc can all contribute to Go or Missed. Ideally, we should be able to combine these factors together with the neural activities & behavioral outcomes and fit a GLM model to assess whether these factors play important roles in the visual task. This is however still not a causal explanation, but by doing so we can evaluate the contribution of other factors, and re-assess whether our claim on the difference of the visual response is suitable. In several attempts, our GLM model, unfortunately, failed to bright us a good level of accuracy, which might be due to the averaging neural activities by brain regions already diminished the distinct feature of the individual neurons. It can also be due to the fact that other factors here are not a good indicator of the animals’ decisions, as the authors mentioned in their supplementary figures that these factors are both not good indicators for the engagement level.
Fig. 12 Evidence vs position map in the mouse CA1 (from Nieh & Schottdorf, Nature, 2021)
So here we have it, this is our team project from the NeuroMatch Academy. This is so far one of the greatest experiences of learning and working with other fellow scientists together. I hope you guys also find it interesting (despite super long!), and all comments are welcome. Kudos to all the people who organized and contributed to this course, really appreciate that.
Code can be view/run publicly by the following colab link, but notice the SVM part can take ~20 mins to finish due to the repeats:
Comments