About

Implementation of

• Learning Geometric Phase Field representations (Yannick Kees 2022)

File	Description
3Dvisualization.ipynb	Coarse rendering of Neural Networks in Jupyter Notebook
dataset.py	Creates and Visualises Datasets with the shapes from the shapemaker file
error_decomposition.py	Plot of different contributions of Loss functional
different_networksizes.py	Measure accuracy of NN while increasing networks
learn_shape_space_ellipse.py	Training shape space network for ellipsoids
learn_shapespace.py	Training shape space network for Metaballs
loss_functionals.py	Computes Modica-Mortola and Ambrosio-Tortorelli
misc.py	Handles import of different file formates, enables CUDA and shows progress on console
networks.py	Neural Networks
packages.py	All used third party packages
pointclouds.py	Creates or changes point clouds
run.py	Solves the 2D reconstruction problem. Can be executed on any computer
Shapemaker.py	Programm that can produce random point clouds in 2D or 3D form natural looking objects
test_autoencoder.py	Plot inputs and outputs of Autoencoder for differnt shapes of dataset
test_shape_space.py	Make plots of elements of shape space after training
test.py	Ignore this..
train_autoencoder.py	Train PointNet - Autoencoder for the different datasets
visualizing.py	Handles visualization of input and output data
volta.py	Solves the 3D reconstruction problem. Should only be executed on high performance computer

How to install:

1. ssh .... & enter password
2. install conda using wget URL, bash~/Anaconda, conda env list Then type

source ~/anaconda3/bin/activate
conda create -n pytorch3d python=3.10
conda activate pytorch3d
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install -c bottler nvidiacub
conda install pytorch3d -c pytorch3d
pip install matplotlib
pip install random-fourier-features-pytorch 
pip install k3d
git clone https://github.com/paulo-herrera/PyEVTK
cd PyEVTK
python setup.py install
git clone https://github.com/Yannick-Kees/Masterarbeit
cd Masterarbeit

Get files from external Computer using

scp IP_ADRESS:~\Masterarbeit\structured2560.vts C:\Users\Yannick\Desktop

External packages:

• Random Fourier Features Pytorch
• K3D Jupyter
• EVTK (Export VTK)
• PointNet Autoencoder
• [Park]
• PyTorch3D

Literatur:

Quellen und so

Classical surface reconstruction problem

• Grids: Poisson Surface Reconstruction (Michael Kazhdan, Matthew Bolitho and Hugues Hoppe 2006)
• Grids: Screened poisson surface reconstruction (Michael Kazhdan Hugues Hoppe 2013) <- Video
• Grids: Fast Surface Reconstruction Using the Level Set Method (Zhao, Osher, Fedkiw 2013)
• RBF: Reconstruction and Representation of 3D Objects with Radial Basis Functions (Carr, Beatson, Cherrie, ... 2001)
• Polynomials: Multi-level Partition of Unity Implicits (Ohtake, Belyaev, .. 2003)
• Polynomials: Smoothing of partition of unity implicit surfaces for noise robust surface reconstruction (Nagai et all 2009)

Surface reconstruction via occupancy function:

• Occupancy Networks: Learning 3D Reconstruction in Function Space (Mescheder et al. 2019)
• IM-Net: Learning Implicit Fields for Generative Shape Modeling (Chen et al. 2018)
• SAL: Sign Agnostic Learning of Shapes from Raw Data (Atzmon, Lipman 2020)
• NEURAL UNSIGNED DISTANCE FIELDS FOR IMPLICIT FUNCTION LEARNING (Chibane, 2020)
• Convolutional Occupancy Networks (Niemeyer et al. 2018)
• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (Mildenhall et al. 2020)

Surface reconstruction via SDF:

• Phase Transitions, Distance Functions, and Implicit Neural Representations (Yaron Lipman 2021)
• DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation (Park et al. 2019)
• MetaSDF: Meta-Learning Signed Distance Functions (Sitzman et al. 2019)
• Implicit Geometric Regularization for Learning Shapes (Gropp, Lipman et al. 2019)
• Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction (Chabra et al. 2019)
• Curriculum DeepSDF (Duan et al. 2019)
• Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance (Yariv, Lipman et al. 2019)
• ON THE EFFECTIVENESS OF WEIGHT-ENCODED NEURAL IMPLICIT 3D SHAPES (Davies, Nowrouzezahrai, Jacobson 2021)

Application

• PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization (Saito et al. 2019)
• Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes (Takikawa et al. 2021)
• PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting (Zhang et al. 2021)
• RGB-D Local Implicit Function for Depth Completion of Transparent Objects (Zhu et al. 2021)

PvJ

• MARCHING CUBES IN AN UNSIGNED DISTANCE FIELD FOR SURFACE RECONSTRUCTION FROM UNORGANIZED POINT SETS (Congote 2010)
• Shape-Aware Matching of Implicit Surfaces Based on Thin Shell Energies (Iglesias, Rumpf 2015)
• A Thin Shell Approach to the Registration of Implicit Surfaces (Iglesias, Rumpf 2013)
• Geometry Processing with Neural Fields (Yang 2021)

Other important papers

• A constructive geometry for computer graphics (Ricci 1974)
• Representation and Rendering of Implicit Surfaces (Sigg 2006 PhD)
• Metaballs (Kenwright T.A. 2014)
• Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysis (Sharp 2022)
• DeepCurrents: Learning Implicit Representations of Shapes with Boundaries (Palmer 2022)
• SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization (Hertz 2021)
• Learned Initializations for Optimizing Coordinate-Based Neural Representations (Tancik 2021)

MtbA

• Spelunking the Deep: Guaranteed Queries on General Neural Implicit Surfaces via Range Analysis (SHARP 2022)
• MIP-plicits: Level of Detail Factorization of Neural Implicits Sphere Tracing (Silva 2022)
• Seeing Implicit Neural Representations as Fourier Series (Benbarka 2021)
• COIN: COmpression with Implicit Neural representations (Emilien Dupont 2021)
• DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction (Wang 2021)