Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rtipsy.pro: handle files larger than 2GB. #3

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

trquinn
Copy link
Member

@trquinn trquinn commented Sep 1, 2017

File size calculations were done with 32bit integers, which won't work for large files. Changed to use 64 "LL" ints.

@trquinn
Copy link
Member Author

trquinn commented Sep 1, 2017

This addresses issue #2. @mtremmel could you check that it works?

@mtremmel
Copy link

mtremmel commented Sep 9, 2017

So it still doesn't work. I tested and on these particular files (which are not extremely large) it seems that the header reading only works with "L", but still the end size is still very slightly off (1137874924 expected and 1137875008 is what it is). I have no idea what's going on with this... clearly a bigger issue is that L and LL seem like they are needed for different sized files which is extremely annoying.

@mtremmel
Copy link

mtremmel commented Sep 9, 2017

I can confirm that the header is being read correctly. The number of particles is accurate, but the sizes associated with each are off I think? Not sure why.

@mtremmel
Copy link

mtremmel commented Sep 9, 2017

On one of my runs (a larger one than the one where this is failing) it seems to work fine with "L" in the header and then comparing to "28LL+...". I am very confused.

@mtremmel
Copy link

mtremmel commented Sep 9, 2017

So in other words, the tipsy file from a smaller run is failing with the current fix and that of a larger file is working fine.

@trquinn
Copy link
Member Author

trquinn commented Sep 11, 2017

For the file that fails, what does it report? That is, after, "RTIPSY ERROR: Header and file size inconsistent Estimates: Header bytes: 28 or 32 (either is OK)", what are the numbers that it reports, and what should those numbers be, given the numbers in the header?

@mtremmel
Copy link

Here is an example:
RTIPSY ERROR: Header and file size inconsistent
Estimates: Header bytes: 28 or 32 (either is OK)
ngas: 1532498 bytes: 73559904
ndark: 29140809 bytes: 1049069124
nstar: 346497 bytes: 15245868
Actual File bytes: 1137875008 not one of: 1137874928 1137874924

@mtremmel
Copy link

rtipsy works fine on the previous and next snapshots of the same simulation. This is rtipsy as it is right now in this branch. On all prior steps it works fine. I think it fails on the following steps 2 more times. This must have something to do with either 1) the number of particles just barely crossing some threshold or 2) some kind of rounding error?

@trquinn
Copy link
Member Author

trquinn commented Sep 11, 2017

Would the problem file have been overwritten in a restart?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants