This is a pretty good question, and it's also good a pretty sensible answer--

Sound is just compression waves, they're what happens when one thing smacks into another thing. Feel something vibrating but can't hear it? That's just because our ears and brains don't pick up those frequencies very well, etc. so that would sound like a concern here, but it's actually the reason we chose it.

A phonograph record is quite literally a micro-scale carving of those physical vibrations. When you make a record, you often first etch it into a hard wax platter with a device that works in a manner that's almost exactly the same as how they are played, only instead of dragging a tiny stylus attached to come electronics which turn the movement of that head with the groove into sound; those electronics pick up sound and move the needle around, carving out a groove that oscillates 1:1 with the sound itself. (That platter is then used to make a master out of something like gold for mass producing pressing vinyl with.)

Even if the aliens have a completely different sense of hearing and very different senses, if they're advanced enough to intercept a rogue space probe, they're going to have the technology necessary to examine the parts of the probe with devices similar in function to a microscope. Once they've seen the grooves at that scale, they will understand that there's something on there, and they're also probably pretty likely to understand that it's sound waves because they literally look like sound waves.

We can only guess as to what their overall technical ability might be, but basically any planet that knows enough to know how sound works and happens to examine it closely would recognize that there's a modulated audio signal of some kind present. If they've got 1980's/90's levels of technical sophistication, they probably won't have to even bother building a machine to extract the audio, because it's actually very possible to extract analog audio from a high resolution digital scan of a record.

Regarding the math present, it's not really that sophisticated as much as it's trying to communicate without a common language. The goal is to try to establish communication if some kind using zero cultural references. Numbers in binary, for example, are easy to identify for a civilization that understands the concept of numerical zero and counting. Humans, largely, use base ten because we've got ten fingers, though other civilizations have used base 8, 12, and 60 for example. Binary is best here because you don't need symbols that "mean anything by themselves" and instead only need to mean something in relation to each other, and if you combine that with geometry, you end up with probably the closest thing to "universal language" we can muster.

A circle or triangle works the same 100 light years from here as it does here. Pi still works, and they know about it if they've gotten as far as examining the disc at all, etc etc.

So that's really the gist of it--the things we put on there were designed for basically the hypothetical alien recipient's equivalent of NASA to be able to figure out. We didn't send those out there expecting them to be able to make immediate sense of them, they were attached because the probes in question were going to leave our solar system, which we'd never done before.