I am planning on moving from Pytorch to Lightning for more structured research. Is lightning + Wandb a good combination in the long run for research experimentations? Which tech stack do you use for research? And for code version control, what do you use? Also, for Kaggle, is it a good choice to use lightning and Wandb? Thank you

Comments

You must log in or register to comment.

ri212 t1_jbsohhm wrote on March 11, 2023 at 12:10 PM

#2,207,793

I can't say for sure whether it is the best combination for research in the long run, but if you do go down that route I have found this template very useful

AerysSk t1_jbt5j4l wrote on March 11, 2023 at 2:49 PM

#2,208,505

My main problem is Lightning itself. I don’t find the flexibility of it like pytorch. Tried migrate an old code, gave up along the way, and still using pytorch now.

maxToTheJ t1_jbtjn8w wrote on March 11, 2023 at 4:31 PM

#2,209,032

Replying to AerysSk (#2,208,505)

The structure of pytorch-lightning makes sense for like 99% of workflows which is why other similar libraries use similar structures for what the generic DL process is like. If you are having trouble migrating you are either in a rare use case or really should think about how you structured your code

koolaidman123 t1_jbtkuif wrote on March 11, 2023 at 4:40 PM

#2,209,068

Replying to maxToTheJ (#2,209,032)

there are some fairly annoying things with pytorch lightning, and somethings are definitely harder to do in lightning due to how it's structured. but overall i find for practical purposes i've been liking lightning a lot more than pytorch + accelerate, especially now you can basically use colossal ai with lightning over deepspeed

shahaff32 t1_jbu0zmr wrote on March 11, 2023 at 6:33 PM

#2,209,634

In our research we ran into issues with Lightning. It is especially annoying when designing non-trivial layers or optimizers. Also, it is much harder to convert the code back to pure pytorch.

For example, in a recent peoject, Lightning caused each forward-backward to operate twice on each batch because we used a combination of two optimizers for a specific reason. And now we are working on rewriting everything without Lightning.

ghostfuckbuddy t1_jbwobij wrote on March 12, 2023 at 8:11 AM

#2,213,068

Replying to shahaff32 (#2,209,634)

That just sounds like a bug. It might take a lot less effort to report it for patching than rewrite all your own code.

shahaff32 t1_jbxy6vj wrote on March 12, 2023 at 4:14 PM

#2,214,649

Replying to ghostfuckbuddy (#2,213,068)

It might be a bug, and it might also be the case that we are the "idiot users" that don't get how to use the package correctly.

Either way, we decided to continue without Lightning (because of that reason and a few others), and therefore we need the code in pure PyTorch for future works that rely on it.

paulgavrikov t1_jc4gnml wrote on March 13, 2023 at 11:38 PM

#2,223,932

I ditched lightning after a while because I spent more too much time fixing it than doing productive stuff. But I regularly use wandb to track experiments and it’s mostly great!