[Model] Add `dgl.nn.CuGraphGATConv` model #5168

tingyu66 · 2023-01-12T21:06:07Z

Description

This PR adds a GATConv model Add dgl.nn.CuGraphGATConv that uses the accelerated sparse aggregation primitives in cugraph-ops. It requires pylibcugraphops >= 23.02.

Checklist

Please feel free to remove inapplicable items for your PR.

The PR title starts with [$CATEGORY] (such as [NN], [Model], [Doc], [Feature]])
I've leverage the tools to beautify the python and c++ code.
The PR is complete and small, read the Google eng practice (CL equals to PR) to understand more about small PR. In DGL, we consider PRs with less than 200 lines of core code change are small (example, test and documentation could be exempted).
All changes have test coverage
Code is well-documented
To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
Related issue is referred in this PR
If the PR is for a new model/paper, I've updated the example index here.

Changes

new nn.Module CuGraphGATConv
test to verify CuGraphGATConv

Notes

Fixes rapidsai/cugraph-ops#181.

dgl-bot · 2023-01-12T21:06:35Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-01-12T21:06:48Z

Commit ID: 77190b5

Build ID: 1

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-01-12T21:09:30Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-01-12T21:09:43Z

Commit ID: 0a38b87

Build ID: 2

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

… to forward()

dgl-bot · 2023-01-19T22:39:16Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-01-19T22:39:29Z

Commit ID: 491556ea28f3af0d3e5666d6606a82945a849c4f

Build ID: 3

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-01-20T05:18:02Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-01-20T05:18:15Z

Commit ID: 3de8a1162c329228359c46bf417ec82d94cddc3c

Build ID: 4

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-01-26T17:46:27Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-01-26T17:46:40Z

Commit ID: 2b1466f96f7e896a10f0a071563ca8e1aa0a6641

Build ID: 5

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-02-02T02:21:09Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-02T02:21:21Z

Commit ID: 6cc6d9da8c9133983d58918ea8c2041297dbcf00

Build ID: 6

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-02-22T03:32:12Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-22T03:32:24Z

Commit ID: 00caff3

Build ID: 7

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-02-22T03:52:06Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-22T03:52:19Z

Commit ID: 8772917

Build ID: 8

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-02-22T15:19:16Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

mufeili · 2023-02-27T09:00:45Z

@dgl-bot

python/dgl/nn/pytorch/conv/cugraph_gatconv.py

mufeili · 2023-02-27T09:13:39Z

@mufeili This PR for CuGraphGATConv is also ready for review.

Compared to GATConv, the cugraph variant lacks support for:

attn_drop option to drop out attention weights

different feature sizes for source and destination nodes

Performance-wise, in the examples/pytorch/gat/train.py (full-graph) example with pubmed dataset, CuGraphGATConv takes 4e-03s per epoch, while GATConv takes 1.2e-02s. Runs with other datasets demonstrated similar speedup numbers.

How about model accuracy?

dgl-bot · 2023-02-27T11:12:22Z

Commit ID: b1882bb9c5937c882c40758664fb88bc0f78fb49

Build ID: 14

Status: ❌ CI test failed in Stage [Tensorflow CPU Unit test].

Report path: link

Full logs path: link

mufeili · 2023-02-28T08:45:48Z

python/dgl/nn/pytorch/conv/cugraph_gatconv.py

+        If not None, applies an activation function to the updated node features.
+        Default: ``None``.
+    bias : bool, optional
+        If True, learns a bias term. Defaults: ``True``.


What if the graph has zero-degree nodes?

Their output features would be zero.

Is this consistent with the behavior of GATConv?

Yes, GATConv also outputs zeros for zero-degree nodes.

python/dgl/nn/pytorch/conv/cugraph_gatconv.py

mufeili

done a pass

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

dgl-bot · 2023-02-28T23:07:41Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-28T23:07:54Z

Commit ID: a86fb29131898b12330787d8ae7d843015e6f7ad

Build ID: 15

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

dgl-bot · 2023-02-28T23:08:38Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-28T23:08:50Z

Commit ID: 6a69aa2a7e98219e0c9a5aa2dfb2bc9608d6abc6

Build ID: 16

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-02-28T23:43:33Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-28T23:43:45Z

Commit ID: fd1dd5d3dc293c239cbc1d45f9bdd2d19cdf1544

Build ID: 17

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

tingyu66 · 2023-02-28T23:49:22Z

@mufeili This PR for CuGraphGATConv is also ready for review.
Compared to GATConv, the cugraph variant lacks support for:

attn_drop option to drop out attention weights

different feature sizes for source and destination nodes

Performance-wise, in the examples/pytorch/gat/train.py (full-graph) example with pubmed dataset, CuGraphGATConv takes 4e-03s per epoch, while GATConv takes 1.2e-02s. Runs with other datasets demonstrated similar speedup numbers.

How about model accuracy?

We get the same accuracy number as the DGL in those examples.

dgl-bot · 2023-03-01T21:19:18Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-03-01T21:19:31Z

Commit ID: 23d4485

Build ID: 18

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

mufeili

I'm good.

dgl-bot · 2023-03-02T07:23:44Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-03-02T07:23:56Z

Commit ID: 0d6f3c4

Build ID: 19

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

mufeili · 2023-03-02T07:24:42Z

@dgl-bot

dgl-bot · 2023-03-02T08:18:24Z

Commit ID: 0d6f3c4

Build ID: 20

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

* add CuGraphGATConv model * lintrunner * update model to reflect changes in make_mfg_csr(), move max_in_degree to forward() * simplify pytest markers * fall back to FG option for large fanout * update error msg * add feat_drop and activation options * add residual option * Update python/dgl/nn/pytorch/conv/cugraph_gatconv.py Co-authored-by: Mufei Li <mufeili1996@gmail.com> * Update python/dgl/nn/pytorch/conv/cugraph_gatconv.py Co-authored-by: Mufei Li <mufeili1996@gmail.com> * reset res_fc --------- Co-authored-by: Mufei Li <mufeili1996@gmail.com>

add CuGraphGATConv model

77190b5

lintrunner

0a38b87

tingyu66 marked this pull request as draft January 12, 2023 21:12

update model to reflect changes in make_mfg_csr(), move max_in_degree…

82e049b

… to forward()

tingyu66 mentioned this pull request Jan 20, 2023

Add/Update cugraph-ops models #5218

Closed

tingyu66 marked this pull request as ready for review January 20, 2023 03:57

simplify pytest markers

8bf50b7

tingyu66 changed the title ~~[DO NOT MERGE][Model] Add dgl.nn.CuGraphGATConv model~~ [Model] Add dgl.nn.CuGraphGATConv model Feb 2, 2023

Merge branch 'master' into cugraphops-gatconv

00caff3

tingyu66 marked this pull request as draft February 22, 2023 03:45

fall back to FG option for large fanout

8772917

tingyu66 added 2 commits February 22, 2023 10:18

Merge branch 'master' into cugraphops-gatconv

04ff458

update error msg

eee2855

mufeili reviewed Feb 27, 2023

View reviewed changes

python/dgl/nn/pytorch/conv/cugraph_gatconv.py Outdated Show resolved Hide resolved

mufeili reviewed Feb 28, 2023

View reviewed changes

python/dgl/nn/pytorch/conv/cugraph_gatconv.py Show resolved Hide resolved

mufeili reviewed Feb 28, 2023

View reviewed changes

python/dgl/nn/pytorch/conv/cugraph_gatconv.py Outdated Show resolved Hide resolved

mufeili requested changes Feb 28, 2023

View reviewed changes

Update python/dgl/nn/pytorch/conv/cugraph_gatconv.py

80773ed

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

Update python/dgl/nn/pytorch/conv/cugraph_gatconv.py

8534f61

Co-authored-by: Mufei Li <mufeili1996@gmail.com>

reset res_fc

e4e189c

Merge branch 'master' into cugraphops-gatconv

23d4485

mufeili approved these changes Mar 2, 2023

View reviewed changes

Merge branch 'master' into cugraphops-gatconv

0d6f3c4

mufeili merged commit bfd411d into dmlc:master Mar 4, 2023

tingyu66 deleted the cugraphops-gatconv branch March 4, 2023 10:16

[Model] Add dgl.nn.CuGraphGATConv model #5168

[Model] Add dgl.nn.CuGraphGATConv model #5168

Conversation

tingyu66 commented Jan 12, 2023 • edited Loading

Description

Checklist

Changes

Notes

dgl-bot commented Jan 12, 2023

dgl-bot commented Jan 12, 2023

dgl-bot commented Jan 12, 2023

dgl-bot commented Jan 12, 2023

dgl-bot commented Jan 19, 2023

dgl-bot commented Jan 19, 2023

dgl-bot commented Jan 20, 2023

dgl-bot commented Jan 20, 2023

dgl-bot commented Jan 26, 2023

dgl-bot commented Jan 26, 2023

dgl-bot commented Feb 2, 2023

dgl-bot commented Feb 2, 2023

dgl-bot commented Feb 22, 2023

dgl-bot commented Feb 22, 2023

dgl-bot commented Feb 22, 2023

dgl-bot commented Feb 22, 2023

dgl-bot commented Feb 22, 2023

mufeili commented Feb 27, 2023

mufeili commented Feb 27, 2023

dgl-bot commented Feb 27, 2023

mufeili Feb 28, 2023

Choose a reason for hiding this comment

tingyu66 Feb 28, 2023 • edited Loading

Choose a reason for hiding this comment

mufeili Mar 2, 2023

Choose a reason for hiding this comment

tingyu66 Mar 4, 2023

Choose a reason for hiding this comment

mufeili left a comment

Choose a reason for hiding this comment

dgl-bot commented Feb 28, 2023

dgl-bot commented Feb 28, 2023

dgl-bot commented Feb 28, 2023

dgl-bot commented Feb 28, 2023

dgl-bot commented Feb 28, 2023

dgl-bot commented Feb 28, 2023

tingyu66 commented Feb 28, 2023

dgl-bot commented Mar 1, 2023

dgl-bot commented Mar 1, 2023

mufeili left a comment

Choose a reason for hiding this comment

dgl-bot commented Mar 2, 2023

dgl-bot commented Mar 2, 2023

mufeili commented Mar 2, 2023

dgl-bot commented Mar 2, 2023

[Model] Add `dgl.nn.CuGraphGATConv` model #5168

[Model] Add `dgl.nn.CuGraphGATConv` model #5168

tingyu66 commented Jan 12, 2023 •

edited

Loading

tingyu66 Feb 28, 2023 •

edited

Loading