IfcConvert - Out-of-core conversion of large .ifc files #3824
Replies: 1 comment 1 reply
-
So, the good news is that the OBJ or glTF serializers are streaming, DAE isn't. But my naive assumption is that if you're trying to parse a 7GB file into 64GB of RAM you likely didn't get to actually interpreting and serializing the geometry. The other good news is that theoretically, IfcOpenShell has somewhat of a basis for unloading geometry definitions. In an ancient branch we even had an option for that https://github.com/IfcOpenShell/IfcOpenShell/blob/_ifc_hdf5/src/ifcparse/IfcParse.h#L274 and it's trivial to implement anyway. The bad news is twofold: (a) this can only unload the attribute values (which is considerable, but not everything in the memory footprint of an instance; something needs to stay alive to reload the attribute values when requested) (b) determining when and what to unload can be tricky considering we're dealing with a complex graph structure If you're interested you can try after interpreting the geometry of an element [0]. Do IfcFile::traverse() on the representation to do a traversal of all instances that make up the traversal and implement a similar unload. IfcOpenShell would be clever enough to reload the instance if it might be necessary for another representation as well. (we might have to double check if inverse relationships do not get counted twice). And then you'd have to benchmark this on a smaller model (but not too small) first to see if and to what extent this actually reduced memory usage. I find it really hard to predict. If that doesn't cut it, you'd have to come up with a way to segment the file without the IfcOpenShell parser. I think you could use something like https://github.com/IfcOpenShell/step-file-parser to stream through the file to build an adjacency matrix of instances and break up the file in small chunks using text processing. |
Beta Was this translation helpful? Give feedback.
-
Hi,
I’ve been using IfcConvert for a while both distributed binaries and compiled version from source (on Windows Visual Studio 2019). Most of the time, the converter manages to convert my input files to .xml, .glb and .obj quite robustly, which I am grateful for. Recently, I tried to convert a very large ifc file (approx. 7GBs). During parsing, the memory usage was quite high (reaching the limit of my workstation which is 64 GBs) but the conversion was not successful. Unfortunately, I can not provide that file.
I would like to ask if it is possible to convert large .ifc files in an out-of-core fashion. For example by:
If this is not supported by IfcConvert at the moment, how feasible is it to modify the source code to add this functionality (at least for ifc to obj conversion)?
Looking at the source code, I am aware that such a modification might affect many different parts of the project. I've already briefly looked at IfcConvert.cpp and IfcParse.cpp. I would appreciate, if anyone could give me some hints. For example, in which parts of IfcOpenShell source code I might need to focus for such a modification.
Thank you for your time.
Beta Was this translation helpful? Give feedback.
All reactions