Building ResNet for Tabular Data Regression Problem

Submitted by eternalmathstudent t3_z7z9gg in deeplearning

I have studied different variants of resnet and I'm aware that it is primarily used for image classification. I've even used pretrained resnet models for some projects.

However I'm now very curious to know if there's any variant of resnet that can be used on typical tabular data, that too for a regression task.

Or is there any convenient residual blocks (preferably tensorflow) available that I can plug and play?

8

Comments

You must log in or register to comment.

carbocation t1_iyb3f36 wrote

While convolution is a bit funky with tabular data (what locality are you exploiting?), I think that attention is a mechanism that might make sense in the deep learning context for tabular data. For example, take a look at recent work such as https://openreview.net/forum?id=i_Q1yrOegLY (code and PDF linked from there).

6

eternalmathstudent OP t1_iyb3ny8 wrote

I did not want to use resnet as is, I'm not requiring the convolutional layer itself. I'm looking for general purpose residual blocks with skip connections

2

carbocation t1_iybb5a8 wrote

Yes, which is why I think you’ll find that link of particular interest since they comment on it (and attention).

3

rjog74 t1_iybbf02 wrote

Any particular reason why resnet only and looking for general purpose residual blocks

3

Redditagonist t1_iybkfcc wrote

Friends don’t let friends use deep learning on tabular data

4

suflaj t1_iy8xun6 wrote

If you have tabular data, the solution is to use XGBoost. Resnets are pretrained on imagenet, meaning if you need it pretrained for any other task, you'll have to do it yourself. I do not see how the task would benefit from a ResNet.

1

majinLawliet2 t1_iyb97za wrote

Resnet is just the architecture. It has nothing to do with pertaining data. You can have skip connections with linear layers as well.

1

suflaj t1_iyc4hvj wrote

I know, however try finding a resnet pretrained on something other than a CV dataset.

2

majinLawliet2 t1_iyb9sec wrote

You need to understand why you want to use resnet architecture. The key reason for using resnet is that as NNs get deeper the inputs to each successive layers become smaller and smaller. This can be circumvented by adding the input data to the output of some layers later.

So in the case of tabular data you need to see why you want a NN and if it is the only thing that works for you. Next question is whether you want to have a very deep NN necessarily. What if you transformed the inputs? If yes, then you should be able to build a skip connections trivially.

1