It's not just about learning different categories.

Imagine you're trying to study a social network of people, take twitter users for example, the individual nodes will probably be the users and the data associated with them (past tweets, bio, etc) while the edges would be the connection between users that you care about (e.g. A follows B, or A tweeted at B, or A retweeted post by B, etc.) and you can see how each of those connections carries information other than just a binary yes or no (e.g. When did A follow B? How many previous tweets did A see of B? How many followers did B have at the time? How many tweets did B have at that time?)

You can see how an individual edge can carry an extremely rich feature vector between nodes A and B where those features are separate from the features belonging to either node A and B themselves. Thus, it's possible that a binary adjacency matrix would not be enough to capture the intrinsic properties of that system.