Viewing a single comment thread. View all comments

vhu9644 t1_j9de7c9 wrote

I have two bachelors, one in Bioengineering (focused on mechanical engineering), one in pure mathematics (with enough classes taken in CS to have a minor if that were allowed at my school). I currently am doing an MD/PhD with that PhD being computational and systems biology. ML and AI are things I want to apply to my field, and I have enough in my background to understand some of the seminal papers in the field. I say this because I have studied core ideas in all of the majors you have put out there.

My recommendation between CS, Math, Neuroscience, and Cog Sci is, in order of priority, Computer science, then applied math, then pure math, then cognitive science, then neuroscience.

Neural networks now borrow nearly nothing from Neuroscience and Cognitive science. The relevant equations in Neuroscience and Cognitive science are intractable to do actual computation on, and while cognitive science (and some neuroscience) does try to use some SOTA stuff, it isn't where the ideas really come from. Also, the perceptron is from the 1960s. ConvNets are from the 1980s. So was backprop. What made these old things actually work was advances in hardware, and what brought them further was educated recursion and iteration. People had ideas mostly driven by deep mathematical and empirical understanding of what they were working with, and then iterated on them until it worked.

That said, If we went through a more formalism-driven proof based conception of machine learning and AI, then math would be more useful. This is not the case. While the ideas in mathematics can be helpful (for example, there is deep mathematical theory for understanding neural networks) many of these ideas are generally applied post-hoc. To my knowledge, we have basically one important theorem in play here, which is the universal approximation theorem. It doesn't say much other than 2 hidden layers is sufficient for approximation of functions by densely connected neural networks. I'm not giving this much justice because the math behind it is deep and hard and beyond pre-collegiate mathematics (hard enough that this subject is the first math class to make me physically cry). This is to illustrate how ill-equipped the mathematical world is at understanding SOTA neural networks.

This isn't to say knowledge of mathematics will not help you. For example, we know things like how the landscape of VAEs loss functions are similar to that of PCA. There is a cool math trick to make diffusion models a tractable training problem. There is work in trying to bring self-attention down to more tractable memory sizes that involves some numerical analysis. This means that if your goal really is to help with AGI, you will need to know some math.

What is important for actual AGI are scientific insights (what is sentience? How can a machine generate new ideas? How can a machine learn about the world?) and engineering solutions (How can we make machine learning tractable? How can we fit the processing power into our current hardware?). Computer science teaches you both. You will learn how to analyze algorithms in how they scale (important for fitting things into hardware), and you'll have electives that teach you how we have concepted machine learning and Artificial Intelligence. What you should supplement is solid numerical and continuous mathematics. Learn some numerical analysis. Learn some control theory. Learn some statistics. These are the core ideas and problems we want AGI to currently solve. Neuroscience won't care about making AGI work (and neither will CogSci). Mathematics is deeply beautiful and useful, but the reliance of proofs make mathematics generally a bit behind on the empirical fields.

If you have any questions, I've chosen a very different path in life, but I'll be happy to answer stuff from my perspective. Best of luck with your major choice.

9