Viewing a single comment thread. View all comments

Red-Portal t1_j6edkjk wrote

One of the bull's eye contributions of signal processing to deep learning was this paper. From a signal processing perspective, naive pooling is obviously problematic because you're decimating without limiting the signal bandwidth. That paper showed that in 2019. Shows how much computer vision has changed from an EE-dominant field to a CS field, where signal processing is not common knowledge.

7

a_khalid1999 OP t1_j6eev77 wrote

Interesting perspective. I did not know Computer Vision was a EE-dominant field at one point, I mean I knew Image Processing is a EE thing, but Vision just gave the ... CS vibe, I mean when I took it in my Bachelor's it was labelled as a CS course.

So basically one way of looking at things could be, and as a EE I'm obviously biased, it's not the Signal Process Engineers moving into ML, it's the CS guys starting to use Signal Processing, cuz where I've been the impression is always that AI is completely a CS thing and the EE's coming in this field are coming due to the lack of job opportunities

3

Red-Portal t1_j6efht6 wrote

That's a more recent trend. Until the late 2000s, computer vision was basically combining machine learning techniques with image processing: Design filters to extract features, and slap them into a classifier. Naturally, lots of Fourier, wavelets, and other weird bases. Very different times.

8

a_khalid1999 OP t1_j6efynn wrote

Sounds fun. During my Bachelors I took ML courses along with Signal Processing and other EE stuff wanting to somehow specialize in some intersection, now ending up kinda in this identity crisis of being a CS or a EE. Make EE great again! :D

1

MrAcurite t1_j6gr1n8 wrote

I would argue that EE is actually a better major than CS for ML. It beefs up your Math and Statistics chops with DiffEQ, Quantum, and the like, and also includes enough Linear Algebra and Statistics to get you sorted. As a Math major doing ML research, I'm kind of embarrassed by how weak my background in Signal Processing is, and am working through a textbook on DSP in my spare time to fix that.

5

a_khalid1999 OP t1_j6h1uli wrote

Guess we shouldn't be taking the EE background for granted

2

adam_jc t1_j6eqggn wrote

That’s a great paper! I like this paper too that looks at ViT’s through a signal processing lens and points out some potential flaws in the architecture for vision applications

1