Submitted by Sadness24_7 t3_y72mzl in deeplearning
Hello there,
im working on neural network and i have a lot of input variables which i would like to minimize, based on couple experiments i did i found out that most of the data are corelated. Performing training for each combination for each input size makes no sence since that would take ages. What are the techniques to select variable which have the most inpact on neural network performance ?
Pca for moderate dimension reduction. Straight up throwing away half of the highly correlated dimensions for very high dimension numbers.
Youd reject the worst dimensions until thw size is low enought to use pda then use pda to reduce to a size your network can handle.
I dont think PCA will help me, i need to reduce the number of feature in order to simplify the system im working with. those removed feature will no longer be aquired and thus i cant retrain the model in the future. i need to somehow pick 2-10 features out of 38 for which i can finetune the model and deploy it. only those selected features will be logged for future.
Maybe you can do the PCA and then check the loadings?
whats PDA ?
But what am i looking for tho. i've been looking at loadings matrix for couple minutes but cant really figure out the connections. Lets say i want to select 7 feature out of 38, so i performa pca for 7 components and im looking at loading matrix (correlation between 38 feature's and 7 pca's . do i just look at the component with best correlation with the input features and the 7 highest correlation with that pca component ?
I can’t be sure that it would work, but I would try this:
-PCA for N components
-Plot a graph with the 2 or 3 first principal components (depending on the cumulative explained variance, if 2 is enough, a 2D plot)
-Plot the magnitude of the variables and see which are the most impactful. Pick the X features you want.
-Train the network with those X features.
Pda is clearly a typo
I see, i was not sure about that since you wrote it twice... and also you phrased the reply into two paragraphs.
oh, this looks promising, i'll give it a try and see what comes up.
Yea dure, I'm just grumpy rn
Maybe you could try some feature selection for example, tree based feature selection?
_triszt t1_issbxym wrote
pca