Corrections
This commit is contained in:
parent
c303bcbbce
commit
4dc6b7a225
|
@ -1,5 +1,5 @@
|
|||
|
||||
![Intro figure](https://github.com/HuguesTHOMAS/KPConv/blob/master/doc/Github_intro.png)
|
||||
![Intro figure](https://github.com/HuguesTHOMAS/KPConv-PyTorch/blob/master/doc/Github_intro.png)
|
||||
|
||||
Created by Hugues THOMAS
|
||||
|
||||
|
|
BIN
doc/Github_intro.png
Normal file
BIN
doc/Github_intro.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 199 KiB |
76
doc/new_dataset_guide.md
Normal file
76
doc/new_dataset_guide.md
Normal file
|
@ -0,0 +1,76 @@
|
|||
|
||||
|
||||
## Creating your own dataset
|
||||
|
||||
### Overview of the pipeline
|
||||
|
||||
The training script initiates a bunch of variables and classes before starting the training on a dataset. Here are the
|
||||
initialization steps:
|
||||
|
||||
* Create an instance of the `Config` class. This instance will hold all the parameters defining the network.
|
||||
|
||||
* Create an instance of your dataset class. This instance will handle the data, and the input pipeline. **This is the
|
||||
class you have to implement to train our network on your own data**.
|
||||
|
||||
* Load the input point cloud in memory. Most datasets will fit in a 32GB RAM computer. If you don't have enough memory
|
||||
for your dataset, you will have to redesign the input pipeline.
|
||||
|
||||
* Initialize the tensorflow input pipeline, which is a `tf.dataset` object that will create and feed the input batches
|
||||
to the network.
|
||||
|
||||
* Create an instance of the network model class. This class contains the tensorflow operations defining the network.
|
||||
|
||||
* Create an instance of our generic `ModelTrainer` class. This class handles the training of the model
|
||||
|
||||
Then the training can start.
|
||||
|
||||
|
||||
### The dataset class
|
||||
|
||||
This class has several roles. First this is where you define your dataset parameters (class names, data path, nature
|
||||
of the data...). Then this class will hold the point clouds loaded in memory. Eventually, it also defines the
|
||||
Tensorflow input pipeline. For efficiency, our implementation uses a parallel input queue, feeding batches to the
|
||||
network.
|
||||
|
||||
Here we give you a description of each essential method that need to be implemented in your new dataset class. For more
|
||||
details, follow the implementation of the current datasets, which contains a lot of indications as comments.
|
||||
|
||||
|
||||
* The **\_\_init\_\_** method: Here you have to define the parameters of your dataset. Notice that your dataset class
|
||||
has to be a child of the common `Dataset` class, where generic methods are implemented. Their are a few thing that has
|
||||
to be defined here:
|
||||
- The labels: define a dictionary `self.label_to_names`, call the `self.init_labels()` method, and define which
|
||||
label should be ignored in `self.ignored_labels`.
|
||||
- The network model: the type of model that will be used on this dataset ("classification", "segmentation",
|
||||
"multi_segmentation" or "cloud_segmentation").
|
||||
- The number of CPU threads used in the parallel input queue.
|
||||
- Data paths and splits: you can manage your data as you wish, these variables are only used in methods that you
|
||||
will implement, so you do not have to follow exactly the notations of the other dataset classes.
|
||||
|
||||
|
||||
* The **load_subsampled_clouds** method: Here you load your data in memory. Depending on your dataset (if this is a
|
||||
classification or segmentation task, 3D scenes or 3D models) you will not have to load the same variables. Just follow
|
||||
the implementation of the existing datasets.
|
||||
|
||||
|
||||
* The **get_batch_gen** method: This method should return a python generator. This will be the base generator for the
|
||||
`tf.dataset` object. It is called in the generic `self.init_input_pipeline` or `self.init_test_input_pipeline` methods.
|
||||
Along with the generator, it also has to return the generated types and shapes. You can redesign the generators or used
|
||||
the ones we implemented. The generator returns np.arrays, but from this point of the pipeline, they will be converted
|
||||
to tensorflow tensors.
|
||||
|
||||
|
||||
* The **get_tf_mapping** method: This method return a mapping function that takes the generated batches and creates all
|
||||
the variables for the network. Remember that from this point we are defining a tensorflow graph of operations. There is
|
||||
not much to implement here as most of the work is done by two generic function `self.tf_augment_input` and
|
||||
`self.tf_xxxxxxxxx_inputs` where xxxxxxxxx can be "classification" of "segmentation" depending on the task. The only
|
||||
important thing to do here is to define the features that will be fed to the network.
|
||||
|
||||
|
||||
### The training script and configuration class
|
||||
|
||||
In the training script you have to create a class that inherits from the `Config` class. This is where you will define
|
||||
all the network parameters by overwriting the attributes
|
||||
|
||||
|
||||
|
31
doc/object_classification_guide.md
Normal file
31
doc/object_classification_guide.md
Normal file
|
@ -0,0 +1,31 @@
|
|||
|
||||
## Object classification on ModelNet40
|
||||
|
||||
### Data
|
||||
|
||||
Regularly sampled clouds from ModelNet40 dataset can be downloaded <a href="https://shapenet.cs.stanford.edu/media/modelnet40_normal_resampled.zip">here (1.6 GB)</a>. Uncompress the folder and move it to `Data/ModelNet40/modelnet40_normal_resampled`.
|
||||
|
||||
N.B. If you want to place your data anywhere else, you just have to change the variable `self.path` of `ModelNet40Dataset` class (in the file `datasets/ModelNet40.py`).
|
||||
|
||||
### Training a model
|
||||
|
||||
Simply run the following script to start the training:
|
||||
|
||||
python3 training_ModelNet40.py
|
||||
|
||||
This file contains a configuration subclass `ModelNet40Config`, inherited from the general configuration class `Config` defined in `utils/config.py`. The value of every parameter can be modified in the subclass. The first run of this script will precompute structures for the dataset which might take some time.
|
||||
|
||||
### Plot a logged training
|
||||
|
||||
When you start a new training, it is saved in a `results` folder. A dated log folder will be created, containing many information including loss values, validation metrics, model snapshots, etc.
|
||||
|
||||
In `plot_convergence.py`, you will find detailed comments explaining how to choose which training log you want to plot. Follow them and then run the script :
|
||||
|
||||
python3 plot_convergence.py
|
||||
|
||||
|
||||
### Test the trained model
|
||||
|
||||
The test script is the same for all models (segmentation or classification). In `test_any_model.py`, you will find detailed comments explaining how to choose which logged trained model you want to test. Follow them and then run the script :
|
||||
|
||||
python3 test_any_model.py
|
29
doc/object_segmentation_guide.md
Normal file
29
doc/object_segmentation_guide.md
Normal file
|
@ -0,0 +1,29 @@
|
|||
|
||||
## Object Part Segmentation on ShapeNetPart
|
||||
|
||||
### Data
|
||||
|
||||
ShapeNetPart dataset can be downloaded <a href="https://shapenet.cs.stanford.edu/ericyi/shapenetcore_partanno_segmentation_benchmark_v0.zip">here (635 MB)</a>. Uncompress the folder and move it to `Data/ShapeNetPart/shapenetcore_partanno_segmentation_benchmark_v0`.
|
||||
|
||||
### Training
|
||||
|
||||
Simply run the following script to start the training:
|
||||
|
||||
python3 training_ShapeNetPart.py
|
||||
|
||||
Similarly to ModelNet40 training, the parameters can be modified in a configuration subclass called `ShapeNetPartConfig`, and the first run of this script might take some time to precompute dataset structures.
|
||||
|
||||
### Plot a logged training
|
||||
|
||||
When you start a new training, it is saved in a `results` folder. A dated log folder will be created, containing many information including loss values, validation metrics, model snapshots, etc.
|
||||
|
||||
In `plot_convergence.py`, you will find detailed comments explaining how to choose which training log you want to plot. Follow them and then run the script :
|
||||
|
||||
python3 plot_convergence.py
|
||||
|
||||
|
||||
### Test the trained model
|
||||
|
||||
The test script is the same for all models (segmentation or classification). In `test_any_model.py`, you will find detailed comments explaining how to choose which logged trained model you want to test. Follow them and then run the script :
|
||||
|
||||
python3 test_any_model.py
|
19
doc/pretrained_models_guide.md
Normal file
19
doc/pretrained_models_guide.md
Normal file
|
@ -0,0 +1,19 @@
|
|||
|
||||
|
||||
## Test a pretrained network
|
||||
|
||||
### Data
|
||||
|
||||
We provide two examples of pretrained models:
|
||||
- A network with rigid KPConv trained on S3DIS: <a href="https://drive.google.com/open?id=1h9xlfPhbcThFVhVsNV3ocd8bjxrWXARV">link (50 MB)</a>
|
||||
- A network with deformable KPConv trained on NPM3D: <a href="https://drive.google.com/open?id=1U87KtFfK8RcgDKXNstwMxapNDOJ6DrNi">link (54 MB)</a>
|
||||
|
||||
|
||||
|
||||
Unzip the log folder anywhere.
|
||||
|
||||
### Test model
|
||||
|
||||
In `test_any_model.py`, choose the path of the log you just unzipped with the `chosen_log` variable:
|
||||
|
||||
chosen_log = 'path_to_pretrained_log'
|
58
doc/scene_segmentation_guide.md
Normal file
58
doc/scene_segmentation_guide.md
Normal file
|
@ -0,0 +1,58 @@
|
|||
|
||||
|
||||
## Scene Segmentation on S3DIS
|
||||
|
||||
### Data
|
||||
|
||||
S3DIS dataset can be downloaded <a href="https://goo.gl/forms/4SoGp4KtH1jfRqEj2">here (4.8 GB)</a>. Download the file named `Stanford3dDataset_v1.2.zip`, uncompress the folder and move it to `Data/S3DIS/Stanford3dDataset_v1.2`.
|
||||
|
||||
### Training
|
||||
|
||||
Simply run the following script to start the training:
|
||||
|
||||
python3 training_S3DIS.py
|
||||
|
||||
Similarly to ModelNet40 training, the parameters can be modified in a configuration subclass called `S3DISConfig`, and the first run of this script might take some time to precompute dataset structures.
|
||||
|
||||
|
||||
## Scene Segmentation on Scannet
|
||||
|
||||
Incoming
|
||||
|
||||
## Scene Segmentation on Semantic3D
|
||||
|
||||
### Data
|
||||
|
||||
Semantic3D dataset can be found <a href="http://www.semantic3d.net/view_dbase.php?chl=2">here</a>. Download and unzip every point cloud as ascii files and place them in a folder called `Data/Semantic3D/original_data`. You also have to download and unzip the groundthruth labels as ascii files in the same folder
|
||||
|
||||
|
||||
### Training
|
||||
|
||||
Simply run the following script to start the training:
|
||||
|
||||
python3 training_Semantic3D.py
|
||||
|
||||
Similarly to ModelNet40 training, the parameters can be modified in a configuration subclass called `Semantic3DConfig`, and the first run of this script might take some time to precompute dataset structures.
|
||||
|
||||
|
||||
## Scene Segmentation on NPM3D
|
||||
|
||||
Incoming
|
||||
|
||||
|
||||
## Plot and test trained models
|
||||
|
||||
### Plot a logged training
|
||||
|
||||
When you start a new training, it is saved in a `results` folder. A dated log folder will be created, containing many information including loss values, validation metrics, model snapshots, etc.
|
||||
|
||||
In `plot_convergence.py`, you will find detailed comments explaining how to choose which training log you want to plot. Follow them and then run the script :
|
||||
|
||||
python3 plot_convergence.py
|
||||
|
||||
|
||||
### Test the trained model
|
||||
|
||||
The test script is the same for all models (segmentation or classification). In `test_any_model.py`, you will find detailed comments explaining how to choose which logged trained model you want to test. Follow them and then run the script :
|
||||
|
||||
python3 test_any_model.py
|
83
doc/visualization_guide.md
Normal file
83
doc/visualization_guide.md
Normal file
|
@ -0,0 +1,83 @@
|
|||
|
||||
|
||||
## Visualize learned features
|
||||
|
||||
### Intructions
|
||||
|
||||
In order to visualize features you need a dataset and a pretrained model. You can use one of our pretrained models
|
||||
provided in the [pretrained models guide](./pretrained_models_guide.md), and the corresponding dataset.
|
||||
|
||||
To start this visualization run the script:
|
||||
|
||||
python3 visualize_features.py
|
||||
|
||||
### Details
|
||||
|
||||
The visualization script has to main parts, separated in two different methods of the visualizer class in
|
||||
`visualizer.py`.
|
||||
|
||||
* In the first part, implemented in the method `top_relu_activations`, the script runs the model on test examples
|
||||
(forward pass). At the chosen Relu layer, you have N output features that are going to be visualized. For each feature,
|
||||
the script keeps the top 5 examples that activated it the most, and saves them in a `visu` folder.
|
||||
|
||||
* In the second part, implemented in the method `top_relu_activations`, the script just shows the saved examples for
|
||||
each feature with the level of activation as color. You can navigate through examples with keys 'g' and 'h'.
|
||||
|
||||
N.B. This second part of the code can be started without doing the first part again if the top examples have already
|
||||
been computed. See details in the code. Alternatively you can visualize the saved example with a point cloud software
|
||||
like CloudCompare.
|
||||
|
||||
|
||||
## Visualize kernel deformations
|
||||
|
||||
### Intructions
|
||||
|
||||
In order to visualize features you need a dataset and a pretrained model that uses deformable KPConv. You can use our
|
||||
NPM3D pretrained model provided in the [pretrained models guide](./pretrained_models_guide.md).
|
||||
|
||||
To start this visualization run the script:
|
||||
|
||||
python3 visualize_deformations.py
|
||||
|
||||
### Details
|
||||
|
||||
The visualization script runs the model runs the model on a batch of test examples (forward pass), and then show these
|
||||
examples in an interactive window. Here is a list of all keyborad shortcuts:
|
||||
|
||||
- 'b' / 'n': smaller or larger point size.
|
||||
- 'g' / 'h': previous or next example in current batch.
|
||||
- 'k': switch between the rigid kenrel (original kernel points positions) and the deformed kernel (position of the
|
||||
kernel points after shift are applied)
|
||||
- 'z': Switch between the points displayed (input points, current layer points or both).
|
||||
- '0': Saves the example and deformed kernel as ply files.
|
||||
- mouse left click: select a point and show kernel at its location.
|
||||
- exit window: compute next batch.
|
||||
|
||||
|
||||
## visualize Effective Receptive Fields
|
||||
|
||||
### Intructions
|
||||
|
||||
In order to visualize features you need a dataset and a pretrained model. You can use one of our pretrained models
|
||||
provided in the [pretrained models guide](./pretrained_models_guide.md), and the corresponding dataset.
|
||||
|
||||
To start this visualization run the script:
|
||||
|
||||
python3 visualize_ERFs.py
|
||||
|
||||
**Warning: This cript currently only works on the following datasets: NPM3D, Semantic3D, S3DIS, Scannet**
|
||||
|
||||
### Details
|
||||
|
||||
The visualization script show the Effective receptive fields of a network layer at one location. If you chose another
|
||||
location (with left click), it has to rerun the model on the whole input point cloud to get new gradient values. Here a
|
||||
list of all keyborad shortcuts:
|
||||
|
||||
- 'b' / 'n': smaller or larger point size.
|
||||
- 'g' / 'h': lower or higher ceiling limit. A functionality that remove points from the ceiling. Very handy for indoor
|
||||
point clouds.
|
||||
- 'z': Switch between the points displayed (input points, current layer points or both).
|
||||
- 'x': Go to the next input point cloud.
|
||||
- '0': Saves the input point cloud with ERF values and the center point used as origin of the ERF.
|
||||
- mouse left click: select a point and show ERF at its location.
|
||||
- exit window: End script.
|
Loading…
Reference in a new issue