Yes, it's all binary. I believe there were a few experimental computers that used other system, but everything else is binary.

I should point out that outside of computer chips not everything is "binary" in the sense of a signal can be only in one of two states. For instance Ethernet cables can pass many different signal states. How many states depends on the speed of the connection, but it might be something like 64 or 256 different states. This is what makes it so fast. Instead of sending just a 1 or 0, you might send a "34", which actually means 01101, for example, so you are sending five binary values at once instead of just one. I'm hand waving a bit, but that's the idea. When them signal comes into your computer, it ultimately ends up being 0s and 1s again.