crude2refined t1_iy9lbuv wrote
To be fair, the SVD of any network architecture trained on a dataset will exhibit such properties. See, for example, the emergence of “V1 features” in MLPs, CNNs, etc when training on image datasets
literum t1_iy9spar wrote
Could you give a reference for "V1 features"? I couldn't find anything googling.
beezlebub33 t1_iy9tozq wrote
V1 refers the the mammalian primary visual cortex. http://www.scholarpedia.org/article/Area_V1
Cells in V1 respond to simple features such as lines of various orientations, certain simple frequencies, colors, etc. the article discusses it more.
The first layer of a CNN does something similar.
SlayahhEUW t1_iya6l1f wrote
Roughly it's the base mammalian feature extractors. This can also be found by performing principal component analysis of the data, the first layer of a CNN will after training have the same representation as the PCA of the data.
Viewing a single comment thread. View all comments