Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pruning to maximum edge size doesn't necessarily meet target. #271

Open
rwest opened this issue Aug 30, 2012 · 3 comments
Open

Pruning to maximum edge size doesn't necessarily meet target. #271

rwest opened this issue Aug 30, 2012 · 3 comments

Comments

@rwest
Copy link
Member

rwest commented Aug 30, 2012

A recent run, which was set to prune the model edge to a maximum of 20,000 species, had this edge size behaviour:
Edge size (species number) versus iteration number, with post... on Twitpic

We had set the limit at 20,000 because we knew it would run out of memory around 40,000.
When it first exceeded 20,000, it was held around the 20k-25k mark for quite a while, but gradually got out of control.
Sure enough, the job ran out of memory and died, when it reached about 40,000. It seems that sometimes there are a lot of species that, for some reason or another, are not, or cannot be, pruned with the current implementation.

@DetlevCM
Copy link

Just because I see this: From my impression, the maximum edge is not a "hard limit". Setting RMG to prune very aggressively, RMG has often exceeded the target edge species count in my runs.
Spontaneously I would say the edge flux can override the maximum edge species setting.

@rwest
Copy link
Member Author

rwest commented Sep 26, 2012

@faribas and I have also seen this behaviour. I had a plot of edge size against iteration, showing the pruning fighting a losing battle to get it back down to the limit, but I can't find the file now. We didn't look into the cause, but did note that the size was not monotonically increasing (occasionally the pruning would get it back down again, just not far enough often enough).

@rwest
Copy link
Member Author

rwest commented Sep 26, 2012

Haha. The graph I was looking for is above. I feel so stupid sometimes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants