PointMLP/README.md

171 lines
6.8 KiB
Markdown
Raw Normal View History

2022-02-16 02:28:12 +00:00
# Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework ICLR 2022
2022-02-04 01:07:56 +00:00
2021-10-04 07:26:41 +00:00
2022-02-04 05:31:59 +00:00
2022-02-16 06:40:12 +00:00
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/rethinking-network-design-and-local-geometry-1/3d-point-cloud-classification-on-modelnet40)](https://paperswithcode.com/sota/3d-point-cloud-classification-on-modelnet40?p=rethinking-network-design-and-local-geometry-1)
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/rethinking-network-design-and-local-geometry-1/3d-point-cloud-classification-on-scanobjectnn)](https://paperswithcode.com/sota/3d-point-cloud-classification-on-scanobjectnn?p=rethinking-network-design-and-local-geometry-1)
2022-02-04 05:31:59 +00:00
2022-02-04 00:55:26 +00:00
2022-04-25 00:40:43 +00:00
[![github](https://img.shields.io/github/stars/ma-xu/pointMLP-pytorch?style=social)](https://github.com/ma-xu/pointMLP-pytorch)
2022-02-16 02:28:12 +00:00
<div align="left">
<a><img src="images/smile.png" height="70px" ></a>
<a><img src="images/neu.png" height="70px" ></a>
<a><img src="images/columbia.png" height="70px" ></a>
</div>
2022-02-04 01:08:27 +00:00
2022-03-25 13:32:44 +00:00
[open review](https://openreview.net/forum?id=3Pbra-_u76D) | [arXiv](https://arxiv.org/abs/2202.07123) | Primary contact: [Xu Ma](mailto:ma.xu1@northeastern.edu)
2022-02-04 01:07:56 +00:00
2021-10-05 20:46:48 +00:00
<div align="center">
2022-02-16 02:28:12 +00:00
<img src="images/overview.png" width="650px" height="300px">
2021-10-05 20:46:48 +00:00
</div>
Overview of one stage in PointMLP. Given an input point cloud, PointMLP progressively extract local features using residual point MLP blocks. In each stage, we first transform local point using a geometric affine module, then local points are are extracted before and after aggregation respectively. By repeating multiple stages, PointMLP progressively enlarge the receptive field and model entire point cloud geometric information.
2021-10-04 07:26:41 +00:00
2022-01-31 21:05:09 +00:00
2022-02-16 02:28:12 +00:00
## BibTeX
2022-01-31 21:05:09 +00:00
2022-06-07 22:30:03 +00:00
@article{ma2022rethinking,
title={Rethinking network design and local geometry in point cloud: A simple residual MLP framework},
author={Ma, Xu and Qin, Can and You, Haoxuan and Ran, Haoxi and Fu, Yun},
journal={arXiv preprint arXiv:2202.07123},
year={2022}
2022-02-16 02:28:12 +00:00
}
2021-11-13 08:49:30 +00:00
2022-02-16 06:36:20 +00:00
## Model Zoo
2022-11-23 02:19:32 +00:00
**Questions on ModelNet40 classification results (a common issue for ModelNet40 dataset in the community)**
The performance on ModelNet40 of almost all methods are not stable, see (https://github.com/CVMI-Lab/PAConv/issues/9#issuecomment-873371422).<br>
If you run the same codes for several times, you will get different results (even with fixed seed).<br>
The best way to reproduce the results is to test with a pretrained model for ModelNet40. <br>
Also, the randomness of ModelNet40 is our motivation to experiment on ScanObjectNN, and to report the mean/std results of several runs.
------
The codes/models/logs for submission version (without bug fixed) can be found here [commit:d2b8dbaa](http://github.com/13952522076/pointMLP-pytorch/tree/d2b8dbaa06eb6176b222dcf2ad248f8438582026).
On ModelNet40, fixed pointMLP achieves a result of **91.5% mAcc** and **94.1% OA** without voting, logs and pretrained models can be found [[here]](https://web.northeastern.edu/smilelab/xuma/pointMLP/checkpoints/fixstd/modelnet40/pointMLP-20220209053148-404/).
On ScanObjectNN, fixed pointMLP achieves a result of **84.4% mAcc** and **86.1% OA** without voting, logs and pretrained models can be found [[here]](https://web.northeastern.edu/smilelab/xuma/pointMLP/checkpoints/fixstd/scanobjectnn/pointMLP-20220204021453/). Fixed pointMLP-elite achieves a result of **81.7% mAcc** and **84.1% OA** without voting, logs and pretrained models can be found [[here]](https://web.northeastern.edu/smilelab/xuma/pointMLP/checkpoints/fixstd/scanobjectnn/model313Elite-20220220015842-2956/).
Stay tuned. More elite versions and voting results will be uploaded.
2022-02-16 06:36:20 +00:00
2021-11-13 08:49:30 +00:00
2021-11-18 09:43:32 +00:00
2022-02-16 02:28:12 +00:00
## News & Updates:
2022-02-16 06:36:20 +00:00
2022-03-11 20:10:53 +00:00
- [x] fix the uncomplete utils in partseg by Mar/10, caused by error uplaoded folder.
2022-03-08 20:56:09 +00:00
- [x] upload test code for ModelNet40
2022-02-16 02:28:12 +00:00
- [ ] project page
2022-02-16 06:36:20 +00:00
- [x] update std bug (unstable testing in previous version)
- [x] paper/codes release
2021-11-19 02:36:57 +00:00
2022-02-16 02:28:12 +00:00
:point_right::point_right::point_right:**NOTE:** The codes/models/logs for submission version (without bug fixed) can be found here [commit:d2b8dbaa](http://github.com/13952522076/pointMLP-pytorch/tree/d2b8dbaa06eb6176b222dcf2ad248f8438582026).
2021-11-13 08:49:30 +00:00
2021-10-04 17:15:58 +00:00
2021-10-04 07:26:41 +00:00
## Install
2021-10-04 07:48:33 +00:00
```bash
2022-05-26 23:18:07 +00:00
# step 1. clone this repo
2022-02-16 02:28:12 +00:00
git clone https://github.com/ma-xu/pointMLP-pytorch.git
cd pointMLP-pytorch
2022-05-26 23:18:07 +00:00
# step 2. create a conda virtual environment and activate it
2022-05-13 18:49:42 +00:00
conda env create
2022-02-16 02:28:12 +00:00
conda activate pointmlp
2021-10-04 07:48:33 +00:00
```
2021-10-04 07:26:41 +00:00
2022-05-26 23:18:07 +00:00
```bash
# Optional solution for step 2: install libs step by step
conda create -n pointmlp python=3.7 -y
conda activate pointmlp
conda install pytorch==1.10.1 torchvision==0.11.2 cudatoolkit=10.2 -c pytorch -y
2022-06-07 06:16:07 +00:00
# if you are using Ampere GPUs (e.g., A100 and 30X0), please install compatible Pytorch and CUDA versions, like:
2022-06-07 06:00:53 +00:00
# pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
2022-05-26 23:18:07 +00:00
pip install cycler einops h5py pyyaml==5.4.1 scikit-learn==0.24.2 scipy tqdm matplotlib==3.4.2
pip install pointnet2_ops_lib/.
```
2022-02-16 02:28:12 +00:00
## Useage
### Classification ModelNet40
**Train**: The dataset will be automatically downloaded, run following command to train.
2022-04-20 15:53:52 +00:00
By default, it will create a folder named "checkpoints/{modelName}-{msg}-{randomseed}", which includes args.txt, best_checkpoint.pth, last_checkpoint.pth, log.txt, out.txt.
2021-10-04 16:11:38 +00:00
```bash
2022-04-25 05:38:08 +00:00
cd classification_ModelNet40
2021-10-04 16:11:38 +00:00
# train pointMLP
python main.py --model pointMLP
# train pointMLP-elite
python main.py --model pointMLPElite
# please add other paramemters as you wish.
```
2021-10-04 16:15:43 +00:00
2022-02-16 02:28:12 +00:00
To conduct voting testing, run
2021-10-04 16:15:43 +00:00
```bash
# please modify the msg accrodingly
python voting.py --model pointMLP --msg demo
```
2021-10-04 16:11:38 +00:00
2021-10-04 07:26:41 +00:00
2022-02-16 02:28:12 +00:00
### Classification ScanObjectNN
2021-10-04 07:26:41 +00:00
2022-02-16 19:28:22 +00:00
The dataset will be automatically downloaded
2021-10-04 16:48:24 +00:00
2021-10-04 17:15:58 +00:00
- Train pointMLP/pointMLPElite
2021-10-04 16:48:24 +00:00
```bash
2022-04-25 05:38:08 +00:00
cd classification_ScanObjectNN
2021-10-04 16:48:24 +00:00
# train pointMLP
python main.py --model pointMLP
# train pointMLP-elite
python main.py --model pointMLPElite
# please add other paramemters as you wish.
```
By default, it will create a fold named "checkpoints/{modelName}-{msg}-{randomseed}", which includes args.txt, best_checkpoint.pth, last_checkpoint.pth, log.txt, out.txt.
2021-10-04 07:26:41 +00:00
2022-02-16 02:28:12 +00:00
### Part segmentation
2021-10-04 16:48:24 +00:00
- Make data folder and download the dataset
```bash
2022-04-25 05:38:08 +00:00
cd part_segmentation
2021-10-04 16:48:24 +00:00
mkdir data
cd data
wget https://shapenet.cs.stanford.edu/media/shapenetcore_partanno_segmentation_benchmark_v0_normal.zip --no-check-certificate
unzip shapenetcore_partanno_segmentation_benchmark_v0_normal.zip
```
2021-10-04 17:15:58 +00:00
- Train pointMLP
2021-10-04 16:48:24 +00:00
```bash
# train pointMLP
python main.py --model pointMLP
# please add other paramemters as you wish.
```
2022-02-16 02:28:12 +00:00
## Acknowledgment
Our implementation is mainly based on the following codebases. We gratefully thank the authors for their wonderful works.
[CurveNet](https://github.com/tiangexiang/CurveNet),
[PAConv](https://github.com/CVMI-Lab/PAConv),
[GDANet](https://github.com/mutianxu/GDANet),
[Pointnet2_PyTorch](https://github.com/erikwijmans/Pointnet2_PyTorch)
## LICENSE
PointMLP is under the Apache-2.0 license.