Viewing a single comment thread. View all comments

Red-Portal t1_isrrljo wrote

ICML, NeurIPS, ICLR all have theory papers. But theory people tend to complain that they often get anti-theory reviews. It's possible since the reviewers are so random that you might end up with strict empiricists. COLT on the other hand is pure theory. JMLR is also the most theory heavy journal in machine learning. Optimization people sometimes veer towards SIAM journals, while stuff closer to statistics would be fit for statistics journals like AoS, Bernoulli, JRSS etc.

47

master3243 t1_iss7aiv wrote

I remember trying to publish a theory paper (statistical learning theory) in ICML and got criticized by two reviewers that complained the paper had no experimental justification (despite being pure information theoretic lower bound of any learnt algorithm which was impossible to justify experimentally??) and my professor and I doubt they understood what was happening.

The third reviewer was extremely knowledgeable in this area and we truly appreciated their comments which definitely helped better the paper.

20

mietminderung t1_iss2l8t wrote

> But theory people tend to complain that they often get anti-theory reviews.

How the turn tables! A couple of decades ago - the boot was on the other foot.

12

Normal_Flan_1269 t1_isvwxli wrote

Hey, I’m actually interested in statistical learning theory in grad school. What kind of math prerequisites do I need to be able to research statistical learning in a phd program?

2

mietminderung t1_iswnzea wrote

A basic course in statistics and the desire for the PhD grind.

1

Normal_Flan_1269 t1_isx9q5s wrote

Dude cmon stop playing. I’ve read some of these papers and the math is crazy. What math courses do you need

2

Red-Portal t1_it5wz61 wrote

I kindda agree that you don't necessarily need a "math course" other than the usual requirements of CS undergrads. You somewhat pick up the rest on the way. I asked the same thing to theory grad students and they said the same thing. One fella that had a math BS actually said he didn't find his undergrad experience to be terribly useful, which kindda says a lot. Taking actual learning theory classes and reading textbooks will be necessary though.

2

Normal_Flan_1269 t1_it5yhim wrote

That doesn’t make sense tho. Math majors have more theory than cs majors. Statistical learning definitely requires high levels of mathematics, like functional analysis. It’s a huge area in statistics.

1

Red-Portal t1_it5zev6 wrote

The thing is, you can't learn everything in advance. And you don't need everything all the time. Some works in learning theory might be mathematically very deep, but taking a few undergrad math courses definitely won't prepare you for those. Although they might help you develop mathematical thinking. But as per the knowledge itself, I don't think taking math major courses is the most efficient way to do it.

2

Normal_Flan_1269 t1_it641pn wrote

So then your saying a cs major is the best major? All they learn how to do is code? How would you not need mathematical maturity to even make contributions to statistical learning theory. Like I would even argue a statistics major is more prepared than cs cause they have the background in stats with computing experience as well. Cs is just a software dev major.

0

Red-Portal t1_it649eh wrote

Because you do learn theory in undergrad CS...? On the contrary, no course teaches you how to code.

2

Normal_Flan_1269 t1_ithut4x wrote

Yeah you learn systems design, not functional analysis, measure theory, and actual mathematics to do the derivations in statistical learning theory. Statistical learning theory is mathematics. Not just coding.

1

Red-Portal t1_ithvm6b wrote

You learn algorithm analysis, computational complexity theory, discrete mathematics, automata, cryptography and whatnot. Do these seem like coding to you?

1

Normal_Flan_1269 t1_iti0m49 wrote

None of those are useful for creating new statistical learning methods or pushing the boundaries of statistical learning as functional analysis, measure theory, real analysis, and statistics. Like cryptography is useless for developing new regularized regression methods, who gives a shit about complexity theory? Like you guys think ML theory and statistical learning is a CS branch. Like you guys coin the term machine learning and think it’s a branch of CS… very far from the truth. Mathematicians and statisticians have been running circles around you guys doing this for decades. Know your place

1

Red-Portal t1_iti0ug1 wrote

Wait what? The core of learning theory is algorithm analysis and complexity theory! Please take any learning theory textbook or course first before making such groundless judgements. God the freakin definition of "PAC learnable" is algorithm-theoric.

1

Normal_Flan_1269 t1_iti100x wrote

It’s literally called statistical learning theory

1

Red-Portal t1_iti13ms wrote

Yes that's why you need at least "undergraduate" statistics knowledge.

1

Normal_Flan_1269 t1_iti2r8e wrote

Lol that’s so false. Undergrad stats is not nearly enough, not even undergrad math. You can get away with just knowing how to code a little bit but it’s way more math and stats. Cs majors are just trained to be software engineers and nothing else. I’m a math and stats major and run circles around them in AI courses because they don’t have any technical depth mathematically.

1