Graph ML in Industry Workshop

When I wrote top applications of GNNs at the beginning of this year, I had a feeling that graph ML community is mature enough to start being used in industrial companies. Nine months ahead we decided to gather researchers, engineers, and industry professionals to talk about applications of graphs in the companies. Please, join us on 23rd Sept, 17-00 Paris time (free, online, ~3 hours) by registering at the link.

Review: Deep Learning on Sets

A new blog post by Fabian Fuchs and others about recent approaches of applying deep learning on sets. It digests several paradigms such as permuting & averaging, sorting, approximating invariance, and learning on graphs as a way to overcome permutation invariance of machine learning algorithms.

Organizational Update

I've been running this channel alone for almost two years but it's been more challenging recently to keep the previous pace. To help me, Michael Galkin generously accepted to be one of the admins of this channel, who has been already involved in several posts here.

Michael Galkin is a postdoc at Mila & McGill and you can know him by the amazing digests of knowledge graphs papers, contributions to the open-source projects, and strong research works. Please, welcome Michael and subscribe to his twitter.

P.S. Also I will use this opportunity to remind that if you have something to share with a graph community, do not hesitate to contact us.

PyG 2.0 (PyTorch Geometric 2.0) Release

One of the most prominent libraries in the world of GNNs and Geometric DL got a major update (and a small re-branding to a shorter "PyG")! Now with a website and Slack channel.

In addition to a constantly growing number of supported GNN architectures, the 2.0 version features:

1. Heterogeneous graph support with models, mini-batching, sampling, and a one-line conversion of homogeneous models to heterogeneous.

2. GraphGym - a whole platform for designing and experimenting with GNN architectures where you can fine-tune the nitty-gritty details of your model and find the best hyperparams. Based on the NeurIPS'20 paper

3. Pre-defined models - before, you'd usually build a GNN model from a collection of layers by yourself (trying to not forget to put that non-linearity after the GCN layer). Now, the library includes 25 well-known models!

4. Half-precision support and other smaller improvements to make your GNN journey easier.

Fresh picks from ArXiv
This week on ArXiv: GNN link with causal models, augmenting data, and using knowledge graphs with BERT 🧸

If I forgot to mention your paper, please shoot me a message and I will update the post.

GNNs
* Relating Graph Neural Networks to Structural Causal Models with Petar Veličković, Kristian Kersting
* A Study of Joint Graph Inference and Forecasting with Stephan Günnemann
* Is Heterophily A Real Nightmare For Graph Neural Networks To Do Node Classification?
* Local Augmentation for Graph Neural Networks
* Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT EMNLP 2021

Math
* There does not exist a strongly regular graph with parameters (1911,270,105,27)
* On the Fundamental Limits of Matrix Completion: Leveraging Hierarchical Similarity Graphs

Stanford Graph Learning Workshop

A great online workshop will be organized by Stanford, on Thursday, Sept 16 2021, 08:00 - 17:00 Pacific Time. It includes talks from Jure Leskovec, Matthias Fey, Weihua Hu, Jiaxuan You, as well as a series of talks on applications of GNNs, and two industry panels.

Modeling Intelligence via Graph Neural Networks: slides

The slides of the thesis by Keyulu Xu: Modeling Intelligence via Graph Neural Networks. Keyulu is one of the authors of GIN and other notable works in GML.

Geometric Deep Learning @ML Street Talk

Michael Bronstein, Petar Veličković, Taco Cohen and Joan Bruna are special guests in the new 3.5 hours (👀) episode of ML Street Talk talking Geometric DL and explaining the concepts covered in their recent book and pretty much all the current state of the art in the field. Available on YT as a video and as a podcast on all major platforms.

Fresh picks from ArXiv
This week on ArXiv: demystifying performance of hyperbolic embeddings, complex question answering, and emotion chatbots 👧

If I forgot to mention your paper, please shoot me a message and I will update the post.

Knowledge graphs
* Benchmarking the Combinatorial Generalizability of Complex Query Answering on Knowledge Graphs
* Complex Temporal Question Answering on Knowledge Graphs
* Emily: Developing An Emotion-affective Open-Domain Chatbot with Knowledge Graph-based Persona

Benchmarking
* Comparing Euclidean and Hyperbolic Embeddings on the WordNet Nouns Hypernymy Graph

GNNs
* Releasing Graph Neural Networks with Differential Privacy Guarantees

OGB Large-Scale Challenge Workshop - Presentations of the Winners

OGB LSC is a KDD'21 challenge organized by the OGB team and known for the largest-to-date benchmarking datasets in node-level (240M nodes / 1.7B edges), link-level (90M nodes, 500M edges), and graph-level (4M molecules) tasks. Surely, not all academic labs can afford such compute, but the more interesting are the approaches taken by the winners! Are there any smart tricks or merely "more layers - more ensembles - GPUs go brrr"?

Finally, the recordings of the LSC workshop are available! (~3 hours long, so the Graph ML channel editors assume you've already successfully digested the ML Street Talk for breakfast)

The 2nd day of the workshop features (videos are available):

- Invited talks by Viktor Prasanna (USC), Marinka Zitnik (Harvard), and Larry Zitnick (Facebook AI)
- Panel discussion on the future of Graph ML with Yizhou Sun (UCLA), Zheng Zhang (NYU / Amazon), Shuiwang Ji (Texas A&M), and Jian Tang (MILA)

GML Express: Graph ML in Industry Workshop, Geometric Deep Learning, and New Software.

In case you missed recent most popular events in graph ML, here is a fresh newsletter with recent videos, courses, books, trends, and future events.

Graph Machine Learning in Industry workshop live

Our workshop starts in one hour and I'm excited about our speakers and talks that are ahead (something I would like to attend even if I didn't organize it). You can join us on YouTube or Zoom and we encourage you to ask questions.

The topics are:

0. Me (17:00 Paris time): opening remarks
1. James Zhang (AWS) (17:15): Challenges and Thinking in Go-production of GNN + DGL.
2. Charles Tapley Hoyt (Harvard) (17:45): Current Issues in Theory, Reproducibility, and Utility of Graph Machine Learning in the Life Sciences.
3. Anton Tsitsulin (Google) (18:15): Graph Learning for Billion Scale Graphs.
4. Cheng Ye (AstraZeneca) (19:00): Predicting Potential Drug Targets Using Tensor Factorisation and Knowledge Graph Embeddings.
5: Rocío Mercado (MIT) (19:30): Accelerating Molecular Design Using Graph-Based Deep Generative Models.
6. Lingfei Wu (JD.com) (20:00): Deep Learning On Graphs for Natural Language Processing.

Kite: An interactive visualization tool for graph theory

Another tool called Kite to draw simple graphs and run some graph algorithms.

Fresh picks from ArXiv
This week on ArXiv: generalization of graph embeddings, approximate message passing, and GNNs for hadron collider 🚇

If I forgot to mention your paper, please shoot me a message and I will update the post.

Knowledge graphs
* How Does Knowledge Graph Embedding Extrapolate to Unseen Data: a Semantic Evidence View

GNNs
* Graph-based Approximate Message Passing Iterations
* Orthogonal Graph Neural Networks
* Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limits

Applications
* Hybrid Quantum Classical Graph Neural Networks for Particle Track Reconstruction
* GeomGCL: Geometric Graph Contrastive Learning for Molecular Property Prediction

Feed-forward neural networks for graph processing: video

In this video, Charu Aggarwal discusses the simplest approach of using feedforward neural networks for graph processing. Much simpler than convolutional neural networks, they can do surprisingly well for creating node representations. The presentation is closely related to node2vec, but simplifies the presentation in many respects.

Graph Neural Networks for Point Cloud Processing: meeting

An online talk on 4th October by Mahdi Saleh on their recent work Graphite: GRAPH-Induced feaTure Extraction for Point Cloud Registration. In this presentation he discusses how graphs can be utilized to describe point cloud patches, detect salient points and use them in downstream tasks such as 3D registration.

Scalable Algorithms for Semi-supervised and Unsupervised Learning

A great event coming from Google, Oct 5-7 on unsupervised learning, which includes many great speakers from graph community (Andreas Krause, Piotr Indyk, David Woodruff, David Gleich, Stefanie Jegelka, Leman Akoglu, Danai Koutra, Andreas Loukas, Marinka Zitnik, and many others).

Fresh picks from ArXiv
This week on ArXiv: reconstruction conjecture for higher expressivity, decision graphs, and control in robots 🤖

If I forgot to mention your paper, please shoot me a message and I will update the post.

NeurIPS
* Motif-based Graph Self-Supervised Learning for Molecular Property Prediction NeurIPS 2021
* Reconstruction for Powerful Graph Representations NeurIPS 2021
* Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration NeurIPS 2021

GNNs
* Graph Pointer Neural Networks
* Equivariant Neural Network for Factor Graphs
* Tree in Tree: from Decision Trees to Decision Graphs

Applications
* Deep Fraud Detection on Non-attributed Graph
* How Neural Processes Improve Graph Link Prediction
* Coverage Control in Multi-Robot Systems via Graph Neural Networks
* Molecule3D: A Benchmark for Predicting 3D Geometries from Molecular Graphs

Graph Representation Learning Reading Group @ Mila

The more reading groups on Graph ML in different regions and timezones - the better!
This one is organized by Mila postdocs and open for participation via Zoom. RG starts this Thursday. The lineup for next weeks is published, check the website for more details.

Graph Neural Network for Lagrangian Simulation: video

A presentation by Zijie Li (CMU) on modeling fluid dynamics with GNNs.

ICLR 2022 Submissions

Attached is the list of all submissions for ICLR 2022. In total there are 3712 submissions, while there are ~270 graph papers. About 40% are resubmissions from previous conferences and 75% have first author as student.