Problem doing ACCEL_sift - sifting:candlist_from_candfile

[jandrovins]

Hi! I am having trouble getting PRESTO to run. I am running a pipeline not developed by me that runs various PRESTO commands.

The traceback:

Traceback (most recent call last):
  File "./pipeline.py", line 293, in <module>
    cands = ACCEL_sift(zmax)
  File "./pipeline.py", line 260, in ACCEL_sift
    cands = sifting.read_candidates(candfiles)
  File "/share/apps/intel/intelpython/latest/envs/presto4-env/lib/python3.7/site-packages/presto/sifting.py", line 1198, in read_candidates
    curr_candlist = candlist_from_candfile(filenm, trackbad=track, trackdupes=track)
  File "/share/apps/intel/intelpython/latest/envs/presto4-env/lib/python3.7/site-packages/presto/sifting.py", line 1085, in candlist_from_candfile
    tobs = numsamp * dt
UnboundLocalError: local variable 'numsamp' referenced before assignment

I see in file presto/sifting.py in line begins the function candlist_from_candfile that generates the error. The code snippet:

def candlist_from_candfile(filename, trackbad=False, trackdupes=False):
    candfile = open(filename, 'r')
    # First identify the length of the observation searched
    for line in candfile:
        if line.startswith(" Number of bins in the time series"):
            numsamp = int(line.split()[-1])
        if line.startswith(" Width of each time series bin (sec)"):
            dt = float(line.split()[-1])
    tobs = numsamp * dt

I have checked, and candfile has no line containing " Number of bins in the time series", so the variable numsamp never gets initialized.

Do you have any idea on how to debug this problem? It seems like a problem in a previous step (a previous PRESTO command maybe?)

[scottransom]

Yeah, this definitely seem like a problem with a previous step. Because when ACCEL files are created, the candidates are written to them, and then the related .inf file is appended to the end of the file. It is that appended .inf file where numsamp should get set. So if you don't have that at the end of the ACCEL file, there is or was some issue. When you view the ACCEL file, are the candidates in there? And is it otherwise human-readable?

[jandrovins]

I see.

In the ACCEL file there are candidates and it is human readable. It is indeed missing the .inf part at the end of the file. I see there is the .inf file, but for some reason it does not get appended to the ACCEL file.

I have no astronomy nor physics background. Where should be the code that is supposed to append the .inf file to the ACCEL file?

[jandrovins]

I have actually used the same pipeline.py script with an installation of presto 3.0.1 and it succesfully created the ACCEL file with the .inf appended at the end.

More info about the environment differences:

• The failing environment uses Presto 4.0 compiled with Intel 2021.3 (icx and ifort).
• The seemingly working environment uses Presto 3.0.1 compiled with GCC 11.2
• Both environments use the Intel distribution for Python 3.

[scottransom]

So it happens with a very clunky Unix "cat" command call in accel_utils.c (as part of running accelsearch). The only way that I can see that this wouldn't have worked is if the pipeline somehow moved files around during the search process.
accel_utils.c
964: sprintf(command, "cat %s.inf >> %s", obs->rootfilenm, obs->accelnm);

Note that you could "fix" the problematic files by simply doing the "cat" yourself (as long as the .inf files exist).