| Title: |
Generalization in data-driven models of primary visual cortex |
| Authors: |
Lurz, Konstantin-Klemens; Bashiri, Mohammad; Willeke, Konstantin; Jagadish, Akshay; Wang, Eric; Walker, Edgar Y.; Cadena, Santiago A.; Muhammad, Taliah; Cobos, Erick; Tolias, Andreas S.; Ecker, Alexander S.; Sinz, Fabian H. |
| Contributors: |
Lurz, Konstantin-Klemens; Bashiri, Mohammad; Willeke, Konstantin; Jagadish, Akshay; Wang, Eric; Walker, Edgar Y.; Cadena, Santiago A.; Muhammad, Taliah; Cobos, Erick; Tolias, Andreas S.; Ecker, Alexander S.; Sinz, Fabian H. |
| Publication Year: |
2020 |
| Collection: |
Georg-August-Universität Göttingen: GoeScholar |
| Description: |
Deep neural networks (DNN) have set new standards at predicting responses of neural populations to visual input. Most such DNNs consist of a convolutional network (core) shared across all neurons which learns a representation of neural computation in visual cortex and a neuron-specific readout that linearly combines the relevant features in this representation. The goal of this paper is to test whether such a representation is indeed generally characteristic for visual cortex, i.e. generalizes between animals of a species, and what factors contribute to obtaining such a generalizing core. To push all non-linear computations into the core where the generalizing cortical features should be learned, we devise a novel readout that reduces the number of parameters per neuron in the readout by up to two orders of magnitude compared to the previous state-of-the-art. It does so by taking advantage of retinotopy and learns a Gaussian distribution over the neuron’s receptive field position. With this new readout we train our network on neural responses from mouse primary visual cortex (V1) and obtain a gain in performance of 7% compared to the previous state-of-the-art network. We then investigate whether the convolutional core indeed captures general cortical features by using the core in transfer learning to a different animal. When transferring a core trained on thousands of neurons from various animals and scans we exceed the performance of training directly on that animal by 12%, and outperform a commonly used VGG16 core pre-trained on imagenet by 33%. In addition, transfer learning with our data-driven core is more data-efficient than direct training, achieving the same performance with only 40% of the data. Our model with its novel readout thus sets a new state-of-the-art for neural response prediction in mouse visual cortex from natural images, generalizes between animals, and captures better characteristic cortical features than current task-driven pre-training approaches such as VGG16. |
| Document Type: |
conference object |
| Language: |
English |
| Availability: |
https://resolver.sub.uni-goettingen.de/purl?gro-2/122313 |
| Rights: |
info:eu-repo/semantics/openAccess |
| Accession Number: |
edsbas.64F93CC |
| Database: |
BASE |