Yes, I'm not sure we can speak of creativity when, in fact, from the datasets the machine is trained on to the infrastructure to the programming to the electricity "it" needs to be provided- everything, actually. And the "output" can't even be considered a "new" creation (even if it tricks us by having not "existed prior") in the sense as that I think it would be *derivative * definitionally. It cannot create anything that isn't a parrot-with- additional-steps, rehash, except wherein we give it IChing/magic 8-ball/ouiji nudges.

The tech is undoubtedly powerful, and the ramifications cannot be understated, but the anthropomorphization (understandably, pareidolia by another sense) going on as far as what people are willing to ascribe to "it", I think is being overstated often (if I'm allowed to make a generalized and spurious statement).

Is the Abacus doing math, if math is "done o it/using it", and in its "end state" it looks like it represents a number?

It's a simulation if we ascribe it any "entity", but since the simulation is being done with language, it is invariably degrees of difficulty harder for most people to "counteract" the illusion ("it says!" [but then, we are talking about a people who casually ascribe that manner of agency to even a collection of unrelated books, ze bible sez]).

So it's like making yourself dizzy and saying bloody Mary three times before a candle-lit mirror, it might seem spooky if your mind is playing tricks on you, but you are alone in the bathroom.