Clean up dead units from internal cache of units

[alkurbatov]

Currently, the API tracks every unit created in the game (no matter who owns it). The cache is stored in the tag_to_unit_ map and never cleaned up. As a result we may have quite large memory consumption for significantly large games (e.g. ~40 min) when many units are created and destroyed.

The suggestion is to remove dead units from the cache e.g. after 50 frames so that we can avoid increase in memory consumption.

[mboedigh]

I'm not sure this is a bug. tag_to_units_ was is for all units ever, and tags_to_existing_units_ was for units in vision on a particular observation.
I'm not sure the cleanup is necessary, since the total number of units created isn't huge. It is limited by the total resources on the map (~15000 minerals) and thus only around 1000 units. Moreover, tag_to_units_ only has a pointer to the unit. unit_pool_ stores the unit, but I rely on units existing in memory even if they are dead.

There are another small issue to consider. There isn't a reliable way to know if a unit is dead. Only units that are in vision get passed to the client during an observation if they are in vision. So if an enemy dies outside of vision we won't hear about it again; There is another bug where death of our own units doesn't reliably trigger an event.

[alkurbatov]

So if an enemy dies outside of vision we won't hear about it again

This seems to be ok as humans also don't receive this info.

I'm not sure the cleanup is necessary, since the total number of units created isn't huge.

The problem is that it tracks every units (larva, eggs, different summons, neutrals) so this could be significantly large number.

[mboedigh]

So if an enemy dies outside of vision we won't hear about it again This seems to be ok as humans also don't receive this info. Yes, it is ok, but you can’t reclaim that part of memory /\/\/\/\/\/\/\/\/\ The problem is that it tracks every units (larva, eggs, different gammons, neutrals) so this could be significantly large number. Even so, the number is still limited, because zerg town halls limit the number of larvae. Players can’t create neutral objects (but, I guess yes they can throw down infinite auto-turrets) In practice though this is small savings because you only the pointer to a unit is stored there. Have you encountered a problem due to this, or seen it becoming one? Keep in mind that you can’t delete items from the UnitPool.unit_pool_ because all the pointers used elsewhere would change (e.g. tag_to_unit_). Not worth the effort IMO. But, if you go down that road, maybe we can consider using shared_ptr in the unit pool so they won’t get deleted out from under my bot … ? From: Alexander Kurbatov <notifications@github.com> Sent: Sunday, January 12, 2020 11:35 AM To: alkurbatov/cpp-sc2 <cpp-sc2@noreply.github.com> Cc: M3 <mboedigh@outlook.com>; Comment <comment@noreply.github.com> Subject: Re: [alkurbatov/cpp-sc2] Clean up dead units from internal cache of units (#48) So if an enemy dies outside of vision we won't hear about it again This seems to be ok as humans also don't receive this info. I'm not sure the cleanup is necessary, since the total number of units created isn't huge. The problem is that it tracks every units (larva, eggs, different gammons, neutrals) so this could be significantly large number. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Falkurbatov%2Fcpp-sc2%2Fissues%2F48%3Femail_source%3Dnotifications%26email_token%3DACHT4D4TK6JMBW3JTND2CRTQ5NWG3A5CNFSM4KEGPX32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIXCIKI%23issuecomment-573449257&data=02%7C01%7C%7C7600fbdd1d3e45e8a37e08d797968940%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637144545104150805&sdata=HOM%2BJNiOZMM4dNiXIS1AA0temmlFoB%2FMBT7NrGA4cjw%3D&reserved=0>, or unsubscribe<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACHT4DYQL5C7SQR3MA2YIPTQ5NWG3ANCNFSM4KEGPX3Q&data=02%7C01%7C%7C7600fbdd1d3e45e8a37e08d797968940%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637144545104150805&sdata=P4YZVfxLAsGfbejBNkyRLLff2DJUOVIfsSvodY28zVI%3D&reserved=0>.

[alkurbatov]

So far we have issues with MicroMachine getting out of memory during long games (>=40 minutes). There are no clues that the reason is this logic but it looks suspicious. I think I should spend more time investigating the current implementation.