Just hold it all in memory. My mental arithmetic and manipulation is actually rather decent, despite not being able to visualise it. You actually find that this applies to most people with aphantasia. There's lots of interesting things about it if you search and read up on people's perceptions and experience of it.

It's strange to describe.

Because I know exactly what something like a graph looks like, without being able to visualise it. Just by holding all the information about a graph in memory. I can manipulate that information by simply changing the information of the graph.

However, this ability does break down with more complex systems. If I try and hold an entire chess board in memory and manipulate it, I just fail completely. It's too much information for me to keep in memory and work out accurately without a visual aid.