I am kind of facing the same issue. I partially want to try some moderate deep learning project, while also interested in other aspect of the GPU, for example, video editing, GPU acceleration for data processing (Matlab, python, etc), etc. just got a computer with i7-12700k, to upgrade GPU.

now 3090 Ti FE is available at $1099, while 4080 costs about $1199 (but less availability). it seems 4080 is better than 3090 Ti in almost every aspect excluding unknown performance in deep learning (didn't find any benchmark/comparison on DL).

so not sure how to choose. worth it to trade VRAM/deep learning for other performance?