Official implementation of paper "OED: Towards One-stage End-to-End Dynamic Scene Graph Generation".
We use the dataset Action Genome to train/evaluate our method.
Please process the downloaded dataset with the Toolkit and put the processed annotation files with COCO style into annotations folder.
The directories of the dataset should look like:
|-- action-genome
|-- annotations # gt annotations
|-- ag_train_coco_style.json
|-- ag_test_coco_style.json
|-- ...
|-- frames # sampled frames
|-- videos # original videos
Please download the checkpoints used in the paper and put it into exps folder.
You can use the scripts below to evaluate the performance of OED.
In order to reduce the training cost, we firstly train the spatial module and then train the temporal module after loading trained spatial paramters.
- For SGDET task
python scripts/eval_spatial_sgdet.py # spatial module
python scripts/eval_temporal_sgdet.py # temporal module
- For PredCLS task
python scripts/eval_spatial_predcls.py # spatial module
python scripts/eval_temporal_predcls.py # temporal module
| Task | Module | W/R@10 | W/R@20 | W/R@50 | N/R@10 | N/R@20 | N/R@50 | weight |
|---|---|---|---|---|---|---|---|---|
| SGDET | spatial | 31.5 | 37.7 | 43.7 | 33.4 | 41.6 | 49.0 | link |
| SGDET | temporal | 33.5 | 40.9 | 48.9 | 35.3 | 44.0 | 51.8 | link |
| PredCLS | spatial | 72.9 | 76.0 | 76.1 | 83.3 | 95.3 | 99.2 | link |
| PredCLS | temporal | 73.0 | 76.1 | 76.1 | 83.3 | 95.3 | 99.2 | link |
You can follow the scripts below to train OED in both SGDET and PredCLS tasks.
Notably, manually tuning LR drop may be needed to obtain the best performance.
- For SGDET task
python scripts/train_spatial_sgdet.py # spatial module
python scripts/trian_temporal_sgdet.py # temporal module
- For PredCLS task
python scripts/train_spatial_predcls.py # spatial module
python scripts/train_temporal_predcls.py # temporal module