CVPR 2024
Mustafa Munir, William Avery, Md Mostafijur Rahman, and Radu Marculescu
This repository contains the source code for GreedyViG: Dynamic Axial Graph Construction for Efficient Vision GNNs
Weights trained on ImageNet-1K can be downloaded here.
Weights trained on COCO 2017 Object Detection and Instance Segmentation can be downloaded here.
Weights trained on ADE20K Semantic Segmentation can be downloaded here.
Contains all of the object detection and instance segmentation results, backbone code, and config.
Contains all of the semantic segmentation results, backbone code, and config.
Contains the main GreedyViG model code.
Contains utility scripts used in GreedyViG.
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6 -c pytorch -c conda-forge
conda install mpi4py
pip install -r requirements.txt
python -m torch.distributed.launch --nproc_per_node=num_GPUs --nnodes=num_nodes --use_env main.py --data-path /path/to/imagenet --model greedyvig_model --output_dir greedyvig_results
For example:
python -m torch.distributed.launch --nproc_per_node=1 --nnodes=1 --use_env main.py --data-path ../../Datasets/ILSVRC/Data/CLS-LOC/ --model GreedyViG_M --output_dir greedyvig_test_results
python -m torch.distributed.launch --nproc_per_node=num_GPUs --nnodes=num_nodes --use_env main.py --data-path /path/to/imagenet --model greedyvig_model --resume pretrained_model --eval
For example:
python -m torch.distributed.launch --nproc_per_node=1 --nnodes=1 --use_env main.py --data-path ../../Datasets/ILSVRC/Data/CLS-LOC/ --model GreedyViG_S --resume Pretrained_Models_GreedyViG/S_GreedyViG_81_1.pth --eval
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6 -c pytorch -c conda-forge
pip install timm
pip install submitit
pip install -U openmim
mim install mmcv-full
mim install mmdet==2.28
Detection and instance segmentation on MS COCO 2017 is implemented based on MMDetection. We follow settings and hyper-parameters of PVT, PoolFormer, and EfficientFormer for comparison.
All commands for object detection and instance segmentation should be run from the GreedyViG/detection/ directory.
Prepare COCO 2017 dataset according to the instructions in MMDetection.
Put ImageNet-1K pretrained weights of backbone as
GreedyViG
├── Final_Results
│ ├── model
│ │ ├── model.pth
│ │ ├── ...
python -m torch.distributed.launch --nproc_per_node num_GPUs --nnodes=num_nodes --node_rank 0 main.py configs/mask_rcnn_greedyvig_model --greedyvig_model greedyvig_model --work-dir Output_Directory --launcher pytorch > Output_Directory/log_file.txt
For example:
python -m torch.distributed.launch --nproc_per_node 2 --nnodes 1 --node_rank 0 main.py configs/mask_rcnn_greedyvig_s_fpn_1x_coco.py --greedyvig_model greedyvig_s --work-dir detection_results/ --launcher pytorch > detection_results/greedyvig_s_run.txt
python -m torch.distributed.launch --nproc_per_node=num_GPUs --nnodes=num_nodes --node_rank 0 test.py configs/mask_rcnn_greedyvig_model --checkpoint Pretrained_Model --eval {bbox or segm} --work-dir Output_Directory --launcher pytorch > log_file.txt
For example:
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank 0 test.py configs/mask_rcnn_greedyvig_s_fpn_1x_coco.py --checkpoint ../Pretrained_Models_GreedyViG/Detection/GreedyViG_S_Det.pth --eval bbox --work-dir detection_results/ --launcher pytorch > detection_results/greedyvig_s_run_evaluation.txt
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6 -c pytorch -c conda-forge
pip install -U openmim
mim install mmengine
mim install mmcv-full
mim install "mmsegmentation <=0.30.0"
Semantic segmentation on ADE20K is implemented based on MMSegmentation. We follow settings and hyper-parameters of PVT, PoolFormer, and EfficientFormer for comparison.
All commands for semantic segmentation should be run from the GreedyViG/segmentation/ directory.
8 GPUs, 40K Iterations
python -m torch.distributed.launch --nproc_per_node 8 --nnodes 1 --node_rank 0 train.py configs/sem_fpn/fpn_greedyvig_s_ade20k_40k.py --greedyvig_model greedyvig_s --work-dir semantic_results/ --launcher pytorch > semantic_results/greedyvig_s_run_semantic.txt
If our code or models help your work, please cite MobileViG (CVPRW 2023), MobileViGv2 (CVPRW 2024), and GreedyViG (CVPR 2024):
@InProceedings{GreedyViG_2024_CVPR,
author = {Munir, Mustafa and Avery, William and Rahman, Md Mostafijur and Marculescu, Radu},
title = {GreedyViG: Dynamic Axial Graph Construction for Efficient Vision GNNs},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024},
pages = {6118-6127}
}
@InProceedings{mobilevig2023,
author = {Munir, Mustafa and Avery, William and Marculescu, Radu},
title = {MobileViG: Graph-Based Sparse Attention for Mobile Vision Applications},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2023},
pages = {2211-2219}
}
@InProceedings{MobileViGv2_2024,
author = {Avery, William and Munir, Mustafa and Marculescu, Radu},
title = {Scaling Graph Convolutions for Mobile Vision},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2024},
pages = {5857-5865}
}