By Cheng Shi, Yulin Zhang, Bin Yang, Jiajin Tang, Yuexin Ma and Sibei Yang
The official PyTorch implementation of the "Part2Object: Hierarchical Unsupervised 3D Instance Segmentation".
- Installation
- Data download and Preprocessing
- Pseudo Mask Generation
- Upload Pseudo Mask Result
- Self-Training
- Upload Pretrained Models
We follow Mask3D to install our environment.
The main dependencies of the project are the following:
python: 3.10.9
cuda: 11.3You can set up a conda environment as follows
# Some users experienced issues on Ubuntu with an AMD CPU
# Install libopenblas-dev (issue #115, thanks WindWing)
# sudo apt-get install libopenblas-dev
export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6"
conda env create -f environment.yml
conda activate part2object
pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
pip3 install torch-scatter -f https://data.pyg.org/whl/torch-1.12.1+cu113.html
pip3 install 'git+https://github.com/facebookresearch/detectron2.git@710e7795d0eeadf9def0e7ef957eea13532e34cf' --no-deps
mkdir third_party
cd third_party
git clone --recursive "https://github.com/NVIDIA/MinkowskiEngine"
cd MinkowskiEngine
git checkout 02fc608bea4c0549b0a7b00ca1bf15dee4a0b228
python setup.py install --force_cuda --blas=openblas
cd ..
git clone https://github.com/ScanNet/ScanNet.git
cd ScanNet/Segmentator
git checkout 3e5726500896748521a6ceb81271b0f5b2c0e7d2
make
cd ../../pointnet2
python setup.py install
cd ../../
pip3 install pytorch-lightning==1.7.2
You can download our generated pseudo-labels here or generate by yourself with our code.
To train or test the results of Part2Object, modify the file paths appropriately and run the following scripts.
sh scripts/scannet/scannet_val.shAfter getting the base model trained with pseudo-labeling, you can train the data efficient model by modifying the following script appropriately.
sh scripts/scannet/scannet_df.sh| Methods | AP25 | AP50 | mAP | |
|---|---|---|---|---|
| Part2Object | 55.1 | 26.8 | 12.6 | result |
| Methods | AP50 / (0% data) | AP50 / 1% data | AP50 / 5% data | AP50 / 10% data | AP50 / 20% data | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Part2Object | 32.6 | weight | 44.1 | weight | 64.2 | weight | 68.0 | weight | 72.1 | weight |
We thank Mask3D for their valuable code bases.
If you find Part2Object useful in your research, please consider citing:
@article{shi2024part2object,
title={Part2Object: Hierarchical Unsupervised 3D Instance Segmentation},
author={Shi, Cheng and Zhang, Yulin and Yang, Bin and Tang, Jiajin and Ma, Yuexin and Yang, Sibei},
journal={arXiv preprint arXiv:2407.10084},
year={2024}
}