Viewing a single comment thread. View all comments

99posse t1_j2pyap8 wrote

> Any suggestions on pros and cons of ML research in academia vs a ML industry job of a masters grad would be really helpful!

I am very opinionated on this, so you may want to weigh this accordingly.

PhD: I wish I could get my time back. I would be comfortably retired by now. 4 or 5 years industry experience working for a FAANG gives you a >USD250K/year advantage in the US (plus 4/5 years of being in a highly paid position). The only way you can compensate for this is if you get a PhD is a super relevant area that happens to be exactly what they are looking for (rare nowadays).

Theoretical ML: It's BS, literally. Pretty much everything being published is circlejerk by a small number of people playing with toy models and using plenty of math to hide the fact that they do not know what's going on. One interesting, recent observation is that as you scale models up, the specifics of the architecture no longer matter much and pretty much anything reasonable will work well enough. Which brings me to

Academia vs. industry: There are a few industries that do applied research at a scale that makes a difference. This is because (1) they have access to unlimited training data (2) they have access to nearly unlimited resources (3) they have urgent, real problems to solve (usually, advertising :-) ). What happens in academia (when they not partner with one of these companies) is laughable

18

Farconion t1_j2qh718 wrote

> PhD: I wish I could get my time back. I would be comfortably retired by now. 4 or 5 years industry experience working for a FAANG gives you a >USD250K/year advantage in the US (plus 4/5 years of being in a highly paid position). The only way you can compensate for this is if you get a PhD is a super relevant area that happens to be exactly what they are looking for (rare nowadays).

maybe in the US, OP is in Germany which is a diff ballgame salarywise

11

99posse t1_j2xatk7 wrote

Google in Zurich or London pays more than in Mountain View (this is the one I know, not sure if Meta does the same)

1

Farconion t1_j2yk4sf wrote

that's a few companies in like two cities in Europe, definitely the exception to the trend there :p

2

99posse t1_j2ynomd wrote

I see (and agree with) your point (I am a European citizen working in the US) but ML work at that scale is not that common. If you want to stay local, academia and theoretical work are your best shots (with all the salary implications of the case).

1

anonymousTestPoster t1_j2r4kyj wrote

> Theoretical ML: It's BS, literally

You say this, but I would argue most of the best people in the ML industry that I have personally witness posses a strongly rich theoretical background. Of course to answer very practical questions, I wouldnt maybe go to a theory book, or look into reading a texbook on algebraic geometry..... But the minds of those well-versed in theory tend to better understand novel situations and problems very quickly, and have a very adaptable mind.

So If a theoretician can correctly transition their "post-PHD" personalities to industry, I think they stand the best chance to be one of the most valuable team players, because for example "everyone" can code, or so they say, but not everyone can understand models in the depth of level as a theoretician.

For example if something isn't working, I would rather first seek the counsel of someone with a theoretical background working in industry, rather than someone who has only ever worked in industry, unless that person is exceptionally talented, and has something like 10 years of experience.

2

jokokokok t1_j2q5htz wrote

>One interesting, recent observation is that as you scale models up, the specifics of the architecture no longer matter much and pretty much anything reasonable will work well enough

Could you share some more information on this - is it from a paper? Would like to read more

1

notyourregularnerd OP t1_j2qxeub wrote

  1. I agree with you from financials perspective. I see my peers having joined SWE posts in faang straight after bachelor's have been promoted to SDE2 and equivalent and live a very comfortable life.

  2. I somewhat agree, my thesis also on very well defined properties of robustness in ML models but no explanation for larger models. We give theory for toy 2-3 layer networks with activations on hidden layers. However, recent progress in neural tangent kernels to explain DL is real.

  3. I understand the faang argument. I also had a soft offer to join Amazon as a Applied Scientist I with just MS upon my graduation. However, that ship sailed due to Alexa hiring freeze. My prospective PhD program is completely rooted in academia. Maybe I can do some internship but my program doesn't have a industry collaboration.

1

crispin1 t1_j2qzztx wrote

> What happens in academia (when they not partner with one of these companies) is laughable

Then again, what happens in industry (if there isn't at least a medium term path to financial gain) is non-existent. And some of us think there just might be other worthwhile things for humanity besides money.

1

99posse t1_j2smjnr wrote

Sure, that's why you can spend the rest of your life working on enumerative combinatorics (which, BTW, I just love). Just don't fool yourself thinking you are making the world a better place with a bunch of papers.

I am not disputing the morality of this, just stating what I see/know having worked in both academia and industry.

1

crispin1 t1_j2wxsts wrote

Haha

Looking at my own experience, I'd agree that my journal papers don't *directly* make the world a better place. I do get to do a fair bit of real-world-impact work though, like consult on sustainable transport systems - which sounds suspiciously commercial doesn't it? But I'm only in that position because academia let me build up a base of techniques and software that none of the commercial operators saw a business case for 10 years ago. (And also, because I've built some credibility based on those papers).

1

klop2031 t1_j2rklwy wrote

Would you actually be retired tho? 250k for 4 years is 1m and in 10 you got 2.5m (of course you will get taxed to hell) cutting you down to ~~1m. Sure its good money but not enough to retire on.

1

99posse t1_j2sm1tr wrote

Not just with that, but take a look at levels.fyi and check how much a 1-level advantage at Google or Meta gives you as you progress (the 250K/year I mentioned is just comparing MS + 4Yrs to PhD)

1