The first part. That's what it is. That's all it is. But that's also all we need it to be. LLMs are just token weighting machine run on silicon. They can still output poetry. Our self-reflexive cognition is meaningfuly different but still the same kind of thing, inputs and calculation cascades which is how we decide to do A or B, which is free will.

You open an old school thermostat and you find a coil of metal that expands and contracts depending on the temp ans you say "wait, it's not really telling the temparature, it's just a piece of non-thinking non-calculating metal". But it is telling the temparature in the way we mean a thermostat tells the temparature, which is the only meaningful and important way we mean thermostats tell the temparature. You being sad their isn't a fairy in the box doesn't lessen what the metal coil is doing.