Please tell me how to prepocess my point clouds for this repo?

[luoxiaoliaolan]

Hi!
I'm very interesting in this repo, and want to train my own model with this repo. I have some LiDAR point clouds.
Each txt file represents a street scene point clouds, and includs x,y,z,r,g,b,label.
How should I use this repo to preprocess these point cloud data?
I checked out the preprocessing_sem3D folder, but now the README.md doesn't provide specific steps for data preprocessing. Can you provide me a pre-processing process for my point cloud data? I need more detailed steps.

Thank you very much, I am looking forward to receiving a reply！

[mathieuorhan]

Hi @luoxiaoliaolan,

Sorry for answering this late, I'll try to guide you.

First, make sure to sample your dataset, to make the calculation tractable. We used different methods to keep the details but reduce the amount of points. Tell me if you want more details.

So you need to copy semantic.py in the dataset folder, and adapt it to your dataset. Unfortunately there are different convention and yours is different, but it is easy to adapt. Just load, parse and store the data inside the right arrays in the init. dataset.py generates the batches : export some of them, visualize them with meshlab, and if it's good, you are almost done.
You should change the config file in the main folder too.

[mathieuorhan]

To sample the dataset with the C++ code, @KeyserGuillaume could help you more than me, that is his code. But you can sample it with other tools.

[luoxiaoliaolan]

Thank you a lot! Mr. @mathieuorhan :
At the moment， I am running a point cloud data segmentation program based on this repo. And I am processing the semantic8 dataset. The data processing results I got initially are not very good. The output I want is that the number of point clouds is the same as the number of test point clouds entered. Can this be done?How should I set the parameters?
@KeyserGuillaume I want to know more about the role of sampling, interpolation, grouping. Can you explain it in more detail? About those C++ code.
I hope to maintain closer communication with you. Is it convenient to use email to communicate? And my email is lybluoxiao@qq.com. I am looking forward to your communication with you and I want to share my recent research results with you.

[luoxiaoliaolan]

Hi! mathieu:
Thank you very much for the detailed content of the email, which helped me a lot. I tried to train my own data a few days ago, and got the training model. I loaded this training model, and used predict.py to process the test data.
The result I got was composed of some sparse point clouds， and the number of the sparse point clouds were less than the raw point clouds . If I want to get the predicted result, the number of point clouds is the same as the raw point cloud, I need to run interpolate.sh in interpolation_sem3D folder to interpolate the predicted result. But I can't use interpolate.sh normally. Could you tell me about the detailed steps of compiling and running this interpolation? Another question is visu.py, this file is used to visualize the results，can you introduce the general principles? There is a parameter "config.json" in this file , I don't know where this config.json is.
I look forward to your reply as soon as possible, I hope to share the research progress with you.
Best wishes!

[luoxiaoliaolan]

@mathieuorhan I can't send an email to your email right now, it seems to have been blocked.

[mathieuorhan]

Hi @luoxiaoliaolan, the email was not mine, but @KeyserGuillaume's. I reckon a more comprehensive documentation is lacking, and I cannot work on this project now. @KeyserGuillaume can explain you how interpolation work by e-mail.
You can use visu.py to visualize predictions and plot some stats about your dataset (class balance, etc.). config.json is a the default value to be replace with your own config file.

[luoxiaoliaolan]

@mathieuorhan If it is convenient, could you give me his(@KeyserGuillaume ) email?

[KeyserGuillaume]

Hello @luoxiaoliaolan ,
The file visu.py serves different purposes. The config.json you want to run it with is semantic.json. Then what it does depends on the arguments you give it. If you give stats = true, it will print histograms with class statistics, and give you a way of visualising whether the point clouds are explored uniformly. Before you do interpolation, you need to use predict.py (like it is said in the README). Did that go all right ? Then interpolate.sh builds some simple C++ code that can do the interpolation work and executes it. If you want me to help you further, maybe you could send me an error message ?
I also thought it would be simpler to communicate by email, thus I sent you the one you referred to, so you should have my email address. Then again I guess it's also possible to communicate on this channel.

[luoxiaoliaolan]

@KeyserGuillaume @mathieuorhan
It is my great pleasure for me to ask you for your advice. I used this repo to train my own LiDAR data， and then loaded the training model to predict the test data and got the point cloud semantic segmentation results as shown：

The point cloud of this result is very sparse， because the raw point cloud was sampled when predict.py was executed. Now I want to get the same segmentation result as the raw scene point cloud.
In this repo's README.md:
"For interpolating results, first use predict.py --cloud=true --n=100 --ckpt=your_ckpt --dataset=semantic --set=test for example. Files will be created in visu/semantic_test/full_scenes_predictions and will contain predictions on sparse point clouds. The actual interpolation is done in interpolation directory with the command:
./interpolate path/to/raw/data visu/semantic_test/full_scenes_predictions /path/to/where/to/put/results 'voxel_size'
(with the voxel_size you want, in m. default is 0.1)"
I tried to interpolate the sparse predicted results, I installed PCL1.8.1，but there is a problem with the compilation of main.cpp and CMakeLists.txt, The screenshot of this error is as follows:


And I executed the shell (interpolate.sh) to interpolate the predicted results, but it always showed error.
(./interpolate.sh: line 50: 23455 Segmentation fault (core dumped) ./interpolate $1 $2 $3 $voxel_size 0
)
Could you please help me see this problem? Please be as detailed as possible in the compilation and use of this interpolation part. I really want to get the desired result.
I would also like to ask, does this value (npoints (int): Defaults to 8192. The number of point in each input) affect the prediction result?
I also really want to maintain efficient communication with you on the email. My email address is lybluoxiao@qq.com. Could I have your email?

[mathieuorhan]

Hi @luoxiaoliaolan
Congrats, your results look very nice !!
npoints is the number of points used in the input of the network. I don't recommend you augment it. With interpolation your results should be fine, @KeyserGuillaume can help you. I don't want to disclose my email publicly but you can reach me through GitHub of course !

[luoxiaoliaolan]

Hi!@mathieuorhan @KeyserGuillaume
Thank you very much for your patience in explaining my questions. Our team has been working on the processing and modeling of LiDAR 3D point cloud data. The problem we have to solve is to analyze and process the point cloud data of large-scale outdoor scenes. Therefore, we believe that a large number of high-quality raw point cloud training data is the basis for point cloud semantic segmentation. In the annotation of point cloud training data, we intend to set up a special team to do. Since using your repo during this time, we have gradually achieved better results. I think your repo code is modular and has different processing options for different datasets. And I also want to mention a few questions that can continue to optimize the program.
(1) Accelerated calculation. I noticed that in your repo code, you used some methods to speed up data processing and training. (e.g. train.py: multiprocessing ,train_multi with GPU). I would like to ask if there is any other way to further speed up the calculation. Can you elaborate on it?
(2) Point cloud segmentation accuracy. I recently did some semantic segmentation processing of point cloud data in street scenes. The size of point cloud data is not large. After the training model is obtained through training, the test data is processed,and interpolated them. The results are as follows:

As can be seen from the image, the ground and houses with more pointclouds are classified better, but the trees and power lines classes with less pointclouds are classified poorly. There are still some categories in which the point cloud is misclassified. I would like to know how to further improve the accuracy of point cloud classification and reduce the number of point cloudes misclassified by adjusting the network model and training parameters without increasing the training data.
(3) Test data problem. In your project, all data is classified and labeled.(Both training data and testing data), I want to test the transfer ability of the training model, and test some unlabeled raw point cloud data as test data. Please implement this code that needs to be modified.
I hope to keep in close communication with you on GitHub and look forward to sharing our research progress with you. My email address is lybluoxiao@qq.com

                   Thank your very much!

[NKNguyenLX]

hi @luoxiaoliaolan, i am also interested in this repo too, can you share me the data preprocessing step that @mathieuorhan sent you. Thank you a lot. My email is knguyentnh98@gmail.com

[medif]

please help me why the code runs only EPOCH 000

please help me

[ali-mz]

Hi @luoxiaoliaolan, could you please also send me the pre-processing steps that @KeyserGuillaume sent you? I really appreciate your help. My email is ali93mz@gmail.com. Thanks in advance.

[ssteele-kraken]

@KeyserGuillaume Looking for help with the interpolation. I tried running interpolate.sh with the build argument, which successfully creates the .build folder but I still end up with an error "no such file or directory" for ./build/interpolate.