Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework （ICLR 2022）

open review | arXiv | Primary contact: Xu Ma

Overview of one stage in PointMLP. Given an input point cloud, PointMLP progressively extract local features using residual point MLP blocks. In each stage, we first transform local point using a geometric affine module, then local points are are extracted before and after aggregation respectively. By repeating multiple stages, PointMLP progressively enlarge the receptive field and model entire point cloud geometric information.

BibTeX

@inproceedings{
    ma2022rethinking,
    title={Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual {MLP} Framework},
    author={Xu Ma and Can Qin and Haoxuan You and Haoxi Ran and Yun Fu},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=3Pbra-_u76D}
}

Model Zoo

• The codes/models/logs for submission version (without bug fixed) can be found here commit:d2b8dbaa.

• On ModelNet40, fixed pointMLP achieves a result of 91.5% mAcc and 94.1% OA without voting, logs and pretrained models can be found [here].

• On ScanObjectNN, fixed pointMLP achieves a result of 84.4% mAcc and 86.1% OA without voting, logs and pretrained models can be found [here]. Fixed pointMLP-elite achieves a result of 81.7% mAcc and 84.1% OA without voting, logs and pretrained models can be found [here].

• Stay tuned. More elite versions and voting results will be uploaded.

News & Updates:

• fix the uncomplete utils in partseg by Mar/10, caused by error uplaoded folder.
• upload test code for ModelNet40
• project page
• update std bug (unstable testing in previous version)
• paper/codes release

👉👉👉NOTE: The codes/models/logs for submission version (without bug fixed) can be found here commit:d2b8dbaa.

⏩ Questions on ModelNet40 classification reproduction

The performance on ModelNet40 of almost all methods are not stable, see (https://github.com/CVMI-Lab/PAConv/issues/9#issuecomment-873371422). If you run the same codes for several times, you will get different results. Also, the randomness of ModelNet40 is our motivation to experiment on ScanObjectNN benchmark, and to report the mean/std results of several runs.

Install

# 1. clone this repo
git clone https://github.com/ma-xu/pointMLP-pytorch.git
cd pointMLP-pytorch

# 2. create a conda virtual environment and activate it
conda create -n pointmlp python=3.7 -y
conda activate pointmlp

# 3. install required libs, pytorch1.7+, torchvision, etc.
conda install pytorch==1.10.1 torchvision==0.11.2 cudatoolkit=10.2 -c pytorch -y
pip install cycler einops h5py pyyaml==5.4.1 scikit-learn==0.24.2 scipy tqdm matplotlib==3.4.2

# 4. install CUDA kernels
pip install pointnet2_ops_lib/.

Useage

Classification ModelNet40

Train: The dataset will be automatically downloaded, run following command to train.

By default, it will create a folder named "checkpoints/{modelName}-{msg}-{randomseed}", which includes args.txt, best_checkpoint.pth, last_checkpoint.pth, log.txt, out.txt.

cd classification_ModelNet40
# train pointMLP
python main.py --model pointMLP
# train pointMLP-elite
python main.py --model pointMLPElite
# please add other paramemters as you wish.

To conduct voting testing, run

# please modify the msg accrodingly
python voting.py --model pointMLP --msg demo

Classification ScanObjectNN

The dataset will be automatically downloaded

• Train pointMLP/pointMLPElite

cd classification_ScanObjectNN
# train pointMLP
python main.py --model pointMLP
# train pointMLP-elite
python main.py --model pointMLPElite
# please add other paramemters as you wish.

By default, it will create a fold named "checkpoints/{modelName}-{msg}-{randomseed}", which includes args.txt, best_checkpoint.pth, last_checkpoint.pth, log.txt, out.txt.

Part segmentation

• Make data folder and download the dataset

cd part_segmentation
mkdir data
cd data
wget https://shapenet.cs.stanford.edu/media/shapenetcore_partanno_segmentation_benchmark_v0_normal.zip --no-check-certificate
unzip shapenetcore_partanno_segmentation_benchmark_v0_normal.zip

• Train pointMLP

# train pointMLP
python main.py --model pointMLP
# please add other paramemters as you wish.

Acknowledgment

Our implementation is mainly based on the following codebases. We gratefully thank the authors for their wonderful works.

CurveNet, PAConv, GDANet, Pointnet2_PyTorch

LICENSE

PointMLP is under the Apache-2.0 license. Please contact the authors for commercial use.