...wtf?

When science sounds like bullshit but it's still scientific

Are you trying to describe optimization techniques using category theory or am I missing something?

If yes, then what's the point? You won't get anything out of renaming "mapping" with "functor".

In one way it's more of a more systematic view of architectures than a simple renaming.with a framework with a mathematical theory behind it investigation into architectures and also it'relationship with data.simplest case is two morphisms in the supervised case m,d:x->y where m is the model and d is the 'hidden' relationship between input and desired output.

In another view,it could be seen as a look into NNs themselves,size the well known (in the CT crowd) adage that colimits are a way to put many things in various relationships with each other into a bigger thing is basically what perceptron does only with concrete numbers and objective functions instead of existence and uniqueness.I see a melding of fuzzy PAC learning and GOFAI structuralism as a way forward in AI,and this is a haphazard proposal of a way to do it.I choose categories because they seem way more flexible than any logic (which by the curry-howard-lambek correspondence you can model in a category with sufficient structure).I mean,you can model categories in categories and it's used in foundations of mathematics and it's kind of intuitive if your mind is the right type of broken :)

I'm bullish on CT in ML. I predict that in 10 years, that the word "functor" will be commonplace, similar to how the word "tensor" is now.

I also predict that we will misuse the word "functor" to mean "function", the way "tensor" is currently misused to mean "array", and mathematicians will continue to curse us for screwing up yet another one of their terms.

Thanks for sharing! Are you in a PhD program?

Long sob story,don't bother :) Working my way up though,currently a wannabe citizen scientist.Thanks for the implication though,appreciate it :)

I'm someone with a PhD in a field that uses category theory extensively and I now work in DS. I'm finding a lot of ideas in this post unmotivated, I guess I'm pretty bearish on CT being applied to ML. Can you explain what problems you see that category theory will be used to solve?

I stumbled on this thesis some time ago, where the author formulates a category of causal model, where arrows are structure-preserving transformations between models. Seems like it would be useful for causal model discovery.

That looks interesting, and I'm definitely not saying there's no intersection between CT and DS. There's some cool things I've seen with CT and probability theory recently. But to me it often seems like theory in search of a problem, and a far cry from functor becoming a common word among ML practitioners.

CT is kinda good for ML if you have a complex topology of solution spaces. When programmers try to implement categories from a naive view, instead of applying sophisticated categorical constraints on their models, I definitely feel a sort of way about it. With that said, LLMs with analytic modules should be able to do categorical constructions in the not too distant future, which will be nice as hell. Optimizing functors might be a thing someday too, but its definitely not there mathematically yet

Im bullish on deriving (co)homologies using ML but it will be some time before we get there i think.

I, for one, can't wait until abstract nonsense takes over probability theory*.

ML/DS definitely has problems with papers sometimes playing fast and loose with the theory and instead focusing on getting +0.1% on your favorite leader board. "We make model bigger and big model make good predictions." In all seriousness, I think stuff like this is interesting, but I don't think it's very useful as of now. The fact that everything is just an array in Python has always irked me a bit, but maybe there can be some way to leverage type theory in some way to make the ML code more robust. Or perhaps CT can be used to make it easier to compose probabilistic models.

> * From Disintegration and Bayesian Inversion via String Diagrams

I guess I'm separating purely categorical applications from TDA applications, which I agree things like persistent homology will probably be useful.

Persistent homology for real data is developing plenty of classical techniques, but we probably need a very good LLM with some very good HoTT to derive non-simplicial (co)homology for some given category -> some given abelian category

This sounds really interesting. Can you expand here or link to any resources related to this? I'm most interested in where you would apply these cohomology theories.

Resources are quite scarce I'm afraid. Emily Riehl and company are working on (inf,1) categories to establish homotopy between derived functors, for applications in univalent foundations. For a computer algebra system or proof assistant, type equivalence is required to abstract away implementation details. To actually compute homotopy equivalence, it's better to compute cohomology equivalence, but simplicial cohomology is often too expensive to compute. So it's an open problem whether we can optimize a derived homology functor between a derived (enriched) functor and an abelian (enriched) category (which still lacks proper definition afaik). But its a goal I heard at a HoTT talk once to get non-simplicial cohomology of types instead of computing homotopy (which is computationally impossible at scale). Feel free to steal and spread this idea but it's kinda original and speculative.

tl;dr application is computing homotopy equivalence of types at a reasonable expense

Sure. I see the potential of CT as being a language for expressing, reasoning about, and ultimately designing, AI/ML architectures abstractly.

In software development, we have the concept of "design patterns" that provide a common vocabulary with which we can describe common recurring patterns in software design at an abstract level. It cuts past the implementation details and allows us to focus on larger concerns, such as the composition and coupling of components, and the flow of information. This maturity in software development has allowed us to grow past brute forcing spagetti-code programs to developing robust enterprise-sized systems.

I believe that AI/ML is still in it's spagetti-code infancy. We have no idea how to build and compose AI/ML components into a system a disciplined way. To scale up to larger and more complex AI/ML systems, we're going to need to step back and look at AI/ML architectures more abstractly the way that software did with design patterns. I think CT may be able to help with that.

I'd take a week off from it, your unconscious will continue to process things and you'll be rested when you come back. Even if it's all wrong somehow it's unlikely that you won't get something out of this process as long as you put in more effort than others are putting into their comments in this thread.

So I'm pretty familiar with homotopy theory but don't know any type theory, homotopy or otherwise. What does determining whether types are homotopy equivalent get you in terms of ML applications?

In "laymans" terms, it gives you a) an environment for ML models to verify their proofs and b) and rich space for ML to study relations between different fields of mathematics, logic, philosophy, ethics, and everything else by default at that point.

propositions are implementations of types so just being able to say when propositions are equivalent in like a rigorous way is good for science anyways.

I am reminded and I think, I have come across this idea,

You might like this,

This Mathematician is working on it.

She worked on a paper using category theory & language models.

Link: https://www.math3ma.com/categories

Math3ma is Tai-Danae Bradley, the first author of the paper linked by the OP.

Are your works similar to this series of lectures on Categories for AI: https://cats.for.ai/ ?

Overall I think this trend is quite nice, and I look forward to new ideas from the trend.

Does category theory continue to be as insanely mind-blowing once you actually understand some of it?

Yes and no. The fundamental ideas, once they start to sink in, show clear parallels between vastly different fields of analytic thought. The more you understand the framework though, the more its limitations can be concerning. Dependence on the axiom of choice, for example, and the naturality of choice in the field itself, leads some to speculate that if contradiction can be chosen true, the theory's implementation (with the vast majority of categorical proofs appealing to choice) is completely broken.

It's overwhelming at times how applicable category theory is from the right perspective, but underwhelming how its implementation in set theory can be expected to pan out.

tl;dr: category theory is dope but aoc is sus