The original paper explains the main ideas.

Thank you! 😁😁

Yes. Idk what libraries are popular now but I've used Pytorch Geometric, and it takes as input V which is NxF where N is number of nodes in the graph and F features of the nodes. And E which is 2xnum_edges, where each 2d element is a directed edge between the index of node in V to another index of node in V. E is basically the sparse representation of the adjacency matrix.

Thanks a lot!

Yes, though connections may carry data themselves, in which case an adjacency matrix isn't enough. Some problem domains can be modeled in graphs where nodes are simply connected, but others may require a connection type to be defined (eg: "friend of" / "worked with" / "parent of") and even more data besides just a type (eg: if representing physical objects as a graph, you could encode distances as edge attributes).

So there's multiple ways to represent the graph structure, adj matrix being just one of them but that will have to be part of the input to the network.

Thank you! It was very helpful! ☺️

You can also have a look at these later surveys that give an idea of the different types of GNNs. Also if you prefer Tensorflow you can use the Graph Nets library.

Theory of Graph Neural Networks: Representation and Learning

>though connections may carry data themselves, in which case an adjacency matrix isn't enough

does using pointwise correlation as the adj matrix not give enough information to learn the different categories?

I like Deep Graph Lib for GNN. Documentation is good and their Github is full of examples

Maybe useful

A Gentle Introduction to Graph Neural Networks https://distill.pub/2021/gnn-intro/

Thanks!

Thanks!

It's not just about learning different categories.

Imagine you're trying to study a social network of people, take twitter users for example, the individual nodes will probably be the users and the data associated with them (past tweets, bio, etc) while the edges would be the connection between users that you care about (e.g. A follows B, or A tweeted at B, or A retweeted post by B, etc.) and you can see how each of those connections carries information other than just a binary yes or no (e.g. When did A follow B? How many previous tweets did A see of B? How many followers did B have at the time? How many tweets did B have at that time?)

You can see how an individual edge can carry an extremely rich feature vector between nodes A and B where those features are separate from the features belonging to either node A and B themselves. Thus, it's possible that a binary adjacency matrix would not be enough to capture the intrinsic properties of that system.

Yes, most GNN variants need node features and the graph adjacency, some also take edge features.

This repo has a nice hands-on walkthrough: https://github.com/gordicaleksa/pytorch-GAT

If you want an overview that gives a unifying picture I found this talk really helpful: https://www.youtube.com/watch?v=uF53xsT7mjc

Without edges, you don't have a graph

Have you ever encountered data or a scenario where the underlying process might be well represented by two types of edges? e.g. "Twitter replies to agree/support" vs "Twitter replies to disagree/condemn." Two phase networks have like this are great for representing all kinds of resonance modes (like plasmon), but I haven't quite found a good data science or applied stat application. My intuition is that two party political discourse might be well represented in this style.

What you are describing is often referred to as heterogeneous graphs or knowledge graph, plenty of examples out there. Model wise there is a full literature dedicated to the topic.

I loved petar velickovic's introduction to them. They are on YouTube.

Hshahahsb

Thank you!

Thanks!

Thanks! I’m gonna look!

> I need to give in input to the neural network also info about connections among data (an adjacency matrix) in addition to the data themselves.

Yup :). In a nutshell, you can think of the forward pass as

  def forward(self, X, A):
    potential_msgs = torch.mm(X, self.W2)
    propagated_msgs = torch.mm(A, potential_msgs)
    root_update = torch.mm(X, self.W1)
    output = propagated_msgs + root_update + self.bias
    return output

where A is the adjacency matrix.

PS: I have a code notebook on coding a simple graph neural net from scratch if useful: https://github.com/rasbt/machine-learning-book/blob/main/ch18/ch18_part1.ipynb

I see you've posted a GitHub link to a Jupyter Notebook! GitHub doesn't render large Jupyter Notebooks, so just in case, here is an nbviewer link to the notebook:

https://nbviewer.jupyter.org/url/github.com/rasbt/machine-learning-book/blob/main/ch18/ch18%5C_part1.ipynb

Want to run the code yourself? Here is a binder link to start your own Jupyter server and try it out!

https://mybinder.org/v2/gh/rasbt/machine-learning-book/main?filepath=ch18%2Fch18%5C_part1.ipynb

^(I am a bot.) ^(Feedback) ^(|) ^(GitHub) ^(|) ^(Author)

You may find this introduction by code useful: https://pytorch-geometric.readthedocs.io/en/latest/notes/introduction.html#data-handling-of-graphs

In this example, the connection information is specified as an adjacency list instead of an adjacency matrix.

Thanks!

Thanks a lot!!!