You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What's the problem this feature will solve?
Incredible speed-up of grid-initialisation. An empty grid created with say 10,000 x 10,000 cells, takes up many gigabytes of memory for a structure that has nothing of interest yet embedded
Describe the solution you'd like
I swapped out the default list-of-lists implementation underlying Grid and MultiGrid, with pythons collections.defaultdict and saw a memory footprint 40x smaller than the original implementation.
Here's the simple sub-class I created to override the datastructure:
classSparseMultiGrid(MultiGrid):
def__init__(self, width: int, height: int, torus: bool) ->None:
"""Create a new grid. Args: width, height: The width and height of the grid torus: Boolean whether the grid wraps or not. """self.height=heightself.width=widthself.torus=torusself.num_cells=height*width# Internal list-of-lists which holds the grid cells themselvesself._grid: list[list[GridContent]]
self._grid=defaultdict(lambda: defaultdict(dict))
# Flag to check if the empties set has been created. Better than initializing# _empties as set() because in this case it would become impossible to discern# if the set hasn't still being built or if it has become empty after creation.self._empties_built=False# Neighborhood Cacheself._neighborhood_cache: dict[Any, Sequence[Coordinate]] = {}
# Cutoff used inside self.move_to_empty. The parameters are fitted on Python# 3.11 and it was verified that they are roughly the same for 3.10. Refer to# the code in PR#1565 to check for their stability when a new release gets out.self.cutoff_empties=7.953*self.num_cells**0.384defplace_agent(self, agent: Agent, pos: Coordinate) ->None:
"""Place the agent at the specified location, and set its pos variable."""x, y=posifagent.posisNoneoragentnotinself._grid[x][y]:
iftype(self._grid[x][y]) isnotlist:
self._grid[x][y] = [] # this is needed to initialise empty grid cells before we place the first agent thereself._grid[x][y].append(agent)
agent.pos=posifself._empties_built:
self._empties.discard(pos)
self._empty_mask[agent.pos] =True
Additional context
I'd like to hear the opinions of the Mesa devs on this, perhaps there was some reason that the more memory intensive data-structure was used, but as far as I can tell, there is a lot of wasted overhead being used to initialise the models currently
What's the problem this feature will solve?
Incredible speed-up of grid-initialisation. An empty grid created with say 10,000 x 10,000 cells, takes up many gigabytes of memory for a structure that has nothing of interest yet embedded
Describe the solution you'd like
I swapped out the default list-of-lists implementation underlying Grid and MultiGrid, with pythons
collections.defaultdict
and saw a memory footprint 40x smaller than the original implementation.Here's the simple sub-class I created to override the datastructure:
Additional context
I'd like to hear the opinions of the Mesa devs on this, perhaps there was some reason that the more memory intensive data-structure was used, but as far as I can tell, there is a lot of wasted overhead being used to initialise the models currently
With MultiGrid (Before):
With SparseMultiGrid (After):
The text was updated successfully, but these errors were encountered: