I am curious how a deep learning system, while learning to perform prediction and classifation is any different than our own brains. It seems increasingly evident that while the goals used to guide training are different but the mechanisms of learning effectively the same. Of course there are differences in mechanism and complexity but what this last year is teaching us is the artificial deep learning systems work to do the same type of modeling we undergo when learning. Messy at first but definitely capable of learning and sophistication down the line. Linguists argue for genetically wired language rules but really this isn't needed - the system will figure out what it needs and create them like the good blank slates they are.

There are a lot of ChatGPT misconceptions going around. For example that it just blindly memorizes patterns. It is a deep learning system (very deep) that, if it helps with classification and prediction, ends up creating rather complex and functional models of how things work. These actually perform computation of a pretty sophisticated nature (any function can be modeled by a neural network). And this does include creativity and reasoning as the inputs flow into and through the system. Creativity as a phenomena might need a fitness function which scores creative solutions higher (be nice to model that one so the AI can score itself) and of course will take awhile to get down but not outside the capabilities of these types of systems.

Anyways, just wanted to chime in as this has been on my mind. I am still on the fence whether I believe any of this. The last point is that people criticize ChatGPT for giving incorrect answers but it is human nature to 'approximate' knowledge and thus incredibly messy. Partially why it takes so long.