Most infrastructure code like computer vision code, device drivers, etc are either not culturally relevant or have little cultural relevance.

I don't think it makes any sense to prioritize them when things like twitter have much more direct cultural impact. It would be great if my network card driver was open source, but does it really matter? Is it worth prioritizing? Will it likely have any cultural relevance? To most people the answer to all these questions is no.

I was way more impressed by mu zero when it came out. I feel crazy for not being that impressed by these LLMs. I do think they are changing the world, but I don't see this as some huge advancement in ML as much as an advanced ingestion and regurgitation machine. All the "intelligence" is downstream from the humans that generated the data.

Honestly I think the reason it made a huge splash is because the RLHF fine tuning made the models especially good at fooling humans. It feels like more of a hack than a big step in AI. My biggest worry is people will expect too much out of these things, too soon. There seems to be a lot of fear and exuberance going around.

I don't like that they're calling themselves OpenAI when they aren't open.

Genetic optimization might be dead in most cases. I think a lot of the ideas aside from optimization algorithms are still relevant.

I've found GP techniques can yield parsimonious models. A lot of the big research these days is on big models, but GP seems good for small, parsimonious, and elegant models. Good for low data regimes, specialized problems, and problems where you have expert knowledge you can encode. Generally speaking I like working with GP becuase you end up with a parsimonious and interpretable model (opposite of a lot of NN research).

In practice I've found importance sampling methods to work about as good as genetic optimization for optimizing GP trees/grammars for the small amount of work I did with them. I haven't found either method to edge out by much, but it could depend on the problem.

I don't know if this is considered GP (or GA) without a genetic optimization method. However I think we can say that the notion of optimizing a symbolic tree or grammar was heavily developed within GP, even if today you may use some monte carlo optimization method in practice.

I don't know anything about him when it comes to alignment. Seems like a lot of unrigorous wasted effort at first glance, but I haven't really had the time or desire to look into it.

The overbearing smugness of Inadequate Equilibria was nauseating. It was unreadable, even for poop reading. The guy is really impressed with himself for believing he came up with theories that have existed for a long time, but that he was too lazy and too disrespectful to research. I will admit there were a couple good snippets in the book (but given the general lack of originality, can we really be sure those snippets were original?).

>When things suck, they usually suck in a way that's a Nash Equilibrium.

There you go, I just saved you a couple hours.

What has EY actually done or built? He seems like one of those guys that wants to be seen as technical or intellectual but hasn't actually built anything or done anything other than nebulously / unrigorously / long-windedly discuss ideas to make himself sound impressive. Kinda like the Yuval Noah Harari of AI.

Flux.jl is the most popular deep learning library in Julia. I've played around with it a little, it's quite nice and easy to use. It is amazing how much more elegant the implementations become in julia compared to python.

There is also the less known Lux.jl package that is essentially an explicitly parameterized Flux (less mature than Flux though).

Now I use Julia most of the time. It's great, not just for the speed. The combination of the type system and multiple dispatch lead to much better implementations. I find the same implementations take less code and are easier to understand. Also using a language without a native array type for data science work always seemed crazy to me. There are also a number of smaller things about Julia that are nice compared to Python and reduce friction for me (broadcast a function fn(...) withfn.(...), much better/cleaner package management than Python). I still have to use Python for a lot of work but I'm hoping more people try Julia so that the Python hedgemony can end.