The entire crust, including the volcano, sort of “floats” on the Mantle.

The Mantle isn’t quite liquid, but it’s not quite solid either. It’s highly viscous molten rock under considerable pressure. It “flows” over very long time scales but you couldn’t swim in it.

Think of it like really thick tar. You could walk on it, but it’s not solid.

The diagrams you're talking about are simplifications. In reality, magma usually rises through a series of cracks in the rock, filling out hollows within it as it goes. (These hollows mostly weren't actually empty to begin with, but they were weaker areas that cracked open under the high pressure of the intruding magma.) If the magma hardens underground, it forms one of a variety of intrusive rocks in one of a variety of shapes, depending on how much space it could make for itself by cracking open the surrounding rock, and these shapes show us the variety of ways magma can force its way through rock.

In some cases, the cracks are more or less vertical, and the pictures in your textbook are (while still simplified) more or less accurate, with a vertical-ish shaft descending through the crust and down to the source of the magma in the upper mantle. This is the case for most hotspot volcanoes, for example. In other cases, the network of chambers is exceptionally complicated with lots of back-and-forth - you can imagine something like an anthill - and would form a complex network of caves if drained of magma.

It's worth keeping in mind, though, that the mantle isn't actually liquid for the most part. We used to think that it was, because, well, it's where magma (which is a liquid) comes from, but it turns out that the mantle is mostly a squishy hot-wax-like solid. It only melts under volcanoes because there's some unusual condition (more water in the mantle -> lower melting point, less pressure above it -> lower melting point) present.

The asthenosphere, which makes up most of the mantle, is mostly solid. It's squishy, but solid.

The lava comes from the lower crust/upper mantel, not the center of the earth. The inner core of the earth is extremely hot but solid iron and nickle metal, because it is under so much pressure. Then, the outer core is a liquid metal. Around that is the mantel, which is a mix of metals and minerals. The area where the crust and mantel meet is where magma comes from.

It's liquid enough to circulate somewhat, dragging the more-solid slabs of the lithosphere around the surface of the Earth.