-
We have a large application with over 100 EF models, and similarly significant automappings. Please note that this is working exactly as expected and there are no complaints or issues with how Automapper performs its magic. This is a question about whether it is really beyond what AM is designed to do, and how alternative approaches affect memory allocation vs. AM Our issue is simply a question of scale. We are at the point where the application at rest is consuming significant memory with the EF mappings and Automapper projections, especially the caching of query plans on first projection. We are aware that the extent of our mappings are likely outside the expectations and probably recommendations, and are looking at ways to implement at least some of these things in an alternate way. I am hoping to get some information about approaches that would result in a successful reduction in required memory. Process memory starts at about 500Mb, with object memory at 50Mb. After first project it is 980Mb/85Mb. Without the mappings it drops by a third (roughly). Again - this seems reasonable considering what is going on. As I understand it, the two significant memory uses with a large config as below will be the storage of the config itself, and then the caching of execution expressions on ProjectTo. First question is - with .Net Core 3.1+, is caching expression trees as important? Looking at https://stackoverflow.com/questions/35805609/performance-of-expression-compile-vs-lambda-direct-vs-virtual-calls, it appears performance has improved dramatically - if memory is taking a big hit, then can we maybe treat the expression composition as transient? How would I go about testing this, because at the moment it appears the entire config is composed on first projection. For this to work, it would need to be kind of 'on-demand'. Is it even possible? Breaking up the config would be difficult (but not impossible) because each of the smaller configs would need to duplicate some models as they are shared between object graphs, possibly exarcerbating the problem. Secondly, relating to alternatives to Automapping at least some of the complex objects: Finally, how much does the complexity of mappings add to memory allocation, or is it just the sheer volume of fields being mapped that is at play? This is the config. I expect that it is too complex. It is part of the problem we are trying to solve, hoping for some insight as to the best approach to do so. This is obviously an immense task to rewrite if we need to, and I would really like to understand your thoughts on how this would best be achieved - or is this really well within the scope of how you expect Automapper could be used.
|
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 2 replies
-
I'm not sure this is too complex - it's pretty close to what I see in production. In terms of expression tree compilation, yeah in the past it was CPU intensive. We cached it because it was slow to compile multiple times, but obviously there's a tradeoff. We defer that compilation for the first time you map, though. With |
Beta Was this translation helpful? Give feedback.
-
Thanks for your comments on this. I am trying to understand where the work is happening here. Does your comment mean that
Ideally, if (1) is true, I will simply prevent any direct mapping, and this should mean I don't end up with the expression tree compilation at all. Nearly my entire application is Projections and I do very little direct mapping. This will improve my memory footprint after first map which is currently causing me some grief (death of a million cuts - EF, AM and model size) Sorry my questions may be a little uninformed, I known my way around expressions and linq well, but I have difficulty understanding what happens where and when in the AM code base :) |
Beta Was this translation helpful? Give feedback.
-
Oh - so its the creation of the Expression Tree which takes all the resources then? Not its compilation? And it would be the Expression Tree which is consuming memory rather that the compilation? If I am only using ProjectTo, does the expression tree of the entire config need to be created? It is the memory associated with this that seems to be causing some grief. It may not be the AM ops, but the corresponding Linq caching as a result of the AM, but it would be great to be able to limit the hit to just the actual resources required, rather than the whole config - any options here? |
Beta Was this translation helpful? Give feedback.
I'm not sure this is too complex - it's pretty close to what I see in production. In terms of expression tree compilation, yeah in the past it was CPU intensive. We cached it because it was slow to compile multiple times, but obviously there's a tradeoff. We defer that compilation for the first time you map, though.
With
ProjectTo
, there is no expression compilation, so you do save something there. I try to default toProjectTo
anyway because it's more efficient all around.