Corrections

This commit is contained in:
HuguesTHOMAS 2020-04-27 18:20:35 -04:00
parent c303bcbbce
commit 4dc6b7a225
8 changed files with 297 additions and 1 deletions

View file

@ -1,5 +1,5 @@
![Intro figure](https://github.com/HuguesTHOMAS/KPConv/blob/master/doc/Github_intro.png)
![Intro figure](https://github.com/HuguesTHOMAS/KPConv-PyTorch/blob/master/doc/Github_intro.png)
Created by Hugues THOMAS

BIN
doc/Github_intro.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 199 KiB

76
doc/new_dataset_guide.md Normal file
View file

@ -0,0 +1,76 @@
## Creating your own dataset
### Overview of the pipeline
The training script initiates a bunch of variables and classes before starting the training on a dataset. Here are the
initialization steps:
* Create an instance of the `Config` class. This instance will hold all the parameters defining the network.
* Create an instance of your dataset class. This instance will handle the data, and the input pipeline. **This is the
class you have to implement to train our network on your own data**.
* Load the input point cloud in memory. Most datasets will fit in a 32GB RAM computer. If you don't have enough memory
for your dataset, you will have to redesign the input pipeline.
* Initialize the tensorflow input pipeline, which is a `tf.dataset` object that will create and feed the input batches
to the network.
* Create an instance of the network model class. This class contains the tensorflow operations defining the network.
* Create an instance of our generic `ModelTrainer` class. This class handles the training of the model
Then the training can start.
### The dataset class
This class has several roles. First this is where you define your dataset parameters (class names, data path, nature
of the data...). Then this class will hold the point clouds loaded in memory. Eventually, it also defines the
Tensorflow input pipeline. For efficiency, our implementation uses a parallel input queue, feeding batches to the
network.
Here we give you a description of each essential method that need to be implemented in your new dataset class. For more
details, follow the implementation of the current datasets, which contains a lot of indications as comments.
* The **\_\_init\_\_** method: Here you have to define the parameters of your dataset. Notice that your dataset class
has to be a child of the common `Dataset` class, where generic methods are implemented. Their are a few thing that has
to be defined here:
- The labels: define a dictionary `self.label_to_names`, call the `self.init_labels()` method, and define which
label should be ignored in `self.ignored_labels`.
- The network model: the type of model that will be used on this dataset ("classification", "segmentation",
"multi_segmentation" or "cloud_segmentation").
- The number of CPU threads used in the parallel input queue.
- Data paths and splits: you can manage your data as you wish, these variables are only used in methods that you
will implement, so you do not have to follow exactly the notations of the other dataset classes.
* The **load_subsampled_clouds** method: Here you load your data in memory. Depending on your dataset (if this is a
classification or segmentation task, 3D scenes or 3D models) you will not have to load the same variables. Just follow
the implementation of the existing datasets.
* The **get_batch_gen** method: This method should return a python generator. This will be the base generator for the
`tf.dataset` object. It is called in the generic `self.init_input_pipeline` or `self.init_test_input_pipeline` methods.
Along with the generator, it also has to return the generated types and shapes. You can redesign the generators or used
the ones we implemented. The generator returns np.arrays, but from this point of the pipeline, they will be converted
to tensorflow tensors.
* The **get_tf_mapping** method: This method return a mapping function that takes the generated batches and creates all
the variables for the network. Remember that from this point we are defining a tensorflow graph of operations. There is
not much to implement here as most of the work is done by two generic function `self.tf_augment_input` and
`self.tf_xxxxxxxxx_inputs` where xxxxxxxxx can be "classification" of "segmentation" depending on the task. The only
important thing to do here is to define the features that will be fed to the network.
### The training script and configuration class
In the training script you have to create a class that inherits from the `Config` class. This is where you will define
all the network parameters by overwriting the attributes

View file

@ -0,0 +1,31 @@
## Object classification on ModelNet40
### Data
Regularly sampled clouds from ModelNet40 dataset can be downloaded <a href="https://shapenet.cs.stanford.edu/media/modelnet40_normal_resampled.zip">here (1.6 GB)</a>. Uncompress the folder and move it to `Data/ModelNet40/modelnet40_normal_resampled`.
N.B. If you want to place your data anywhere else, you just have to change the variable `self.path` of `ModelNet40Dataset` class (in the file `datasets/ModelNet40.py`).
### Training a model
Simply run the following script to start the training:
python3 training_ModelNet40.py
This file contains a configuration subclass `ModelNet40Config`, inherited from the general configuration class `Config` defined in `utils/config.py`. The value of every parameter can be modified in the subclass. The first run of this script will precompute structures for the dataset which might take some time.
### Plot a logged training
When you start a new training, it is saved in a `results` folder. A dated log folder will be created, containing many information including loss values, validation metrics, model snapshots, etc.
In `plot_convergence.py`, you will find detailed comments explaining how to choose which training log you want to plot. Follow them and then run the script :
python3 plot_convergence.py
### Test the trained model
The test script is the same for all models (segmentation or classification). In `test_any_model.py`, you will find detailed comments explaining how to choose which logged trained model you want to test. Follow them and then run the script :
python3 test_any_model.py

View file

@ -0,0 +1,29 @@
## Object Part Segmentation on ShapeNetPart
### Data
ShapeNetPart dataset can be downloaded <a href="https://shapenet.cs.stanford.edu/ericyi/shapenetcore_partanno_segmentation_benchmark_v0.zip">here (635 MB)</a>. Uncompress the folder and move it to `Data/ShapeNetPart/shapenetcore_partanno_segmentation_benchmark_v0`.
### Training
Simply run the following script to start the training:
python3 training_ShapeNetPart.py
Similarly to ModelNet40 training, the parameters can be modified in a configuration subclass called `ShapeNetPartConfig`, and the first run of this script might take some time to precompute dataset structures.
### Plot a logged training
When you start a new training, it is saved in a `results` folder. A dated log folder will be created, containing many information including loss values, validation metrics, model snapshots, etc.
In `plot_convergence.py`, you will find detailed comments explaining how to choose which training log you want to plot. Follow them and then run the script :
python3 plot_convergence.py
### Test the trained model
The test script is the same for all models (segmentation or classification). In `test_any_model.py`, you will find detailed comments explaining how to choose which logged trained model you want to test. Follow them and then run the script :
python3 test_any_model.py

View file

@ -0,0 +1,19 @@
## Test a pretrained network
### Data
We provide two examples of pretrained models:
- A network with rigid KPConv trained on S3DIS: <a href="https://drive.google.com/open?id=1h9xlfPhbcThFVhVsNV3ocd8bjxrWXARV">link (50 MB)</a>
- A network with deformable KPConv trained on NPM3D: <a href="https://drive.google.com/open?id=1U87KtFfK8RcgDKXNstwMxapNDOJ6DrNi">link (54 MB)</a>
Unzip the log folder anywhere.
### Test model
In `test_any_model.py`, choose the path of the log you just unzipped with the `chosen_log` variable:
chosen_log = 'path_to_pretrained_log'

View file

@ -0,0 +1,58 @@
## Scene Segmentation on S3DIS
### Data
S3DIS dataset can be downloaded <a href="https://goo.gl/forms/4SoGp4KtH1jfRqEj2">here (4.8 GB)</a>. Download the file named `Stanford3dDataset_v1.2.zip`, uncompress the folder and move it to `Data/S3DIS/Stanford3dDataset_v1.2`.
### Training
Simply run the following script to start the training:
python3 training_S3DIS.py
Similarly to ModelNet40 training, the parameters can be modified in a configuration subclass called `S3DISConfig`, and the first run of this script might take some time to precompute dataset structures.
## Scene Segmentation on Scannet
Incoming
## Scene Segmentation on Semantic3D
### Data
Semantic3D dataset can be found <a href="http://www.semantic3d.net/view_dbase.php?chl=2">here</a>. Download and unzip every point cloud as ascii files and place them in a folder called `Data/Semantic3D/original_data`. You also have to download and unzip the groundthruth labels as ascii files in the same folder
### Training
Simply run the following script to start the training:
python3 training_Semantic3D.py
Similarly to ModelNet40 training, the parameters can be modified in a configuration subclass called `Semantic3DConfig`, and the first run of this script might take some time to precompute dataset structures.
## Scene Segmentation on NPM3D
Incoming
## Plot and test trained models
### Plot a logged training
When you start a new training, it is saved in a `results` folder. A dated log folder will be created, containing many information including loss values, validation metrics, model snapshots, etc.
In `plot_convergence.py`, you will find detailed comments explaining how to choose which training log you want to plot. Follow them and then run the script :
python3 plot_convergence.py
### Test the trained model
The test script is the same for all models (segmentation or classification). In `test_any_model.py`, you will find detailed comments explaining how to choose which logged trained model you want to test. Follow them and then run the script :
python3 test_any_model.py

View file

@ -0,0 +1,83 @@
## Visualize learned features
### Intructions
In order to visualize features you need a dataset and a pretrained model. You can use one of our pretrained models
provided in the [pretrained models guide](./pretrained_models_guide.md), and the corresponding dataset.
To start this visualization run the script:
python3 visualize_features.py
### Details
The visualization script has to main parts, separated in two different methods of the visualizer class in
`visualizer.py`.
* In the first part, implemented in the method `top_relu_activations`, the script runs the model on test examples
(forward pass). At the chosen Relu layer, you have N output features that are going to be visualized. For each feature,
the script keeps the top 5 examples that activated it the most, and saves them in a `visu` folder.
* In the second part, implemented in the method `top_relu_activations`, the script just shows the saved examples for
each feature with the level of activation as color. You can navigate through examples with keys 'g' and 'h'.
N.B. This second part of the code can be started without doing the first part again if the top examples have already
been computed. See details in the code. Alternatively you can visualize the saved example with a point cloud software
like CloudCompare.
## Visualize kernel deformations
### Intructions
In order to visualize features you need a dataset and a pretrained model that uses deformable KPConv. You can use our
NPM3D pretrained model provided in the [pretrained models guide](./pretrained_models_guide.md).
To start this visualization run the script:
python3 visualize_deformations.py
### Details
The visualization script runs the model runs the model on a batch of test examples (forward pass), and then show these
examples in an interactive window. Here is a list of all keyborad shortcuts:
- 'b' / 'n': smaller or larger point size.
- 'g' / 'h': previous or next example in current batch.
- 'k': switch between the rigid kenrel (original kernel points positions) and the deformed kernel (position of the
kernel points after shift are applied)
- 'z': Switch between the points displayed (input points, current layer points or both).
- '0': Saves the example and deformed kernel as ply files.
- mouse left click: select a point and show kernel at its location.
- exit window: compute next batch.
## visualize Effective Receptive Fields
### Intructions
In order to visualize features you need a dataset and a pretrained model. You can use one of our pretrained models
provided in the [pretrained models guide](./pretrained_models_guide.md), and the corresponding dataset.
To start this visualization run the script:
python3 visualize_ERFs.py
**Warning: This cript currently only works on the following datasets: NPM3D, Semantic3D, S3DIS, Scannet**
### Details
The visualization script show the Effective receptive fields of a network layer at one location. If you chose another
location (with left click), it has to rerun the model on the whole input point cloud to get new gradient values. Here a
list of all keyborad shortcuts:
- 'b' / 'n': smaller or larger point size.
- 'g' / 'h': lower or higher ceiling limit. A functionality that remove points from the ceiling. Very handy for indoor
point clouds.
- 'z': Switch between the points displayed (input points, current layer points or both).
- 'x': Go to the next input point cloud.
- '0': Saves the input point cloud with ERF values and the center point used as origin of the ERF.
- mouse left click: select a point and show ERF at its location.
- exit window: End script.