Space isn't matter, it is what matter exists inside of. And the space is expanding in all directions simultaneously, irrespective of a single point. An analogy would be like zooming in on the cartesian coordinate system and seeing more points appearing between the grid lines. So the distance between "1" and "2" increase despite our ruler staying the same length.

As for what space is expanding into, it's a bit of semantics and a bit of scientific definitions/limitations. There is no proof of there being more than 3 spacial dimensions and the universe is defined as the thing containing all of the space, matter, and energy in existence. So as far as we can see there is no "outside" of the universe. It's an all encompassing, self contained thing.