nn2++: Training two-layer neural networks

This small package provides utilities to simulate learning in fully connected two-layer neural networks. It was used to perform all the experiments of our recent paper on the dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup [1].

Install & usage information

To build the simulator, simply type

make nn2pp.exe

which will create a file nn2pp.exe. Run ./nn2pp.exe -h for usage information.

Likewise, type

make nn2pp_ode_erf.exe

to build the ODE integrator. Run ./nn2pp_ode_erf.exe -h for usage information.

Requirements

• All linear algebra operations are implemented using Armadillo, a C++ library for linear algebra & scientific computing
• Unit tests are implemented using Google Test

References

• [1] S. Goldt, M.S. Advani, A.M. Saxe, F. Krzakala, L. Zdeborová, NeurIPS 2019 (forthcoming), arXiv:1906.08632