Skip to content

Project Meeting 2018.11.02

Ben Stabler edited this page Nov 4, 2018 · 5 revisions

Multiprocessing

  • The primary bottleneck is memory usage
  • In addition, the Portland server, which has more processors than the San Diego server, actually has slower processors, and so the runtime results weren't much better
  • Working on revisions to the code to avoid spikes in memory usage
  • Adding more logging / features related to tracking memory / CPU usage
  • Plan to kickoff another full scale run on a server with lots of processors and memory next week
  • A future improvement to significantly reduce memory usage would be to replace strings with factors (codes) in order to trim memory requirements for tables
  • Plan to wrap up task within the next week or so, including code documentation
Clone this wiki locally