Problems handling errors in reference genomes

[snayfach]

I'm trying to run PHIST on a single viral contig vs a large database of gzipped bacterial genomes. The program finishes without errors, but stops after the first ~715 reference genomes and reports these in the *_common_kmers.csv output file. Additionally, the number of reference genomes processed by PHIST is non-deterministic. Sometimes it stops after 715, 720, or 730 genomes processed.

Update: I unzipped the reference genomes. Now the program is printing a warning for certain genomes and stalling in what appears to be an infinite loop. After CTRL+C, here's the error message:

File "PHIST/phist.py", line 107, in
subprocess.run(cmd)
File "/usr/lib64/python3.6/subprocess.py", line 425, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/usr/lib64/python3.6/subprocess.py", line 855, in communicate
self.wait()
File "/usr/lib64/python3.6/subprocess.py", line 1477, in wait
(pid, sts) = self._try_wait(0)
File "/usr/lib64/python3.6/subprocess.py", line 1424, in _try_wait
(pid, sts) = os.waitpid(self.pid, wait_flags)
KeyboardInterrupt

Turns out these genomes were empty files. After uncompressing and deleting these, the program finished without errors.

It would be great if the behavior of the program was improved when encountering these corrupted reference genomes. Either a warning that skips over them, or an error with informative error message

[agudys]

Hello!
Thank you for the update. We will try to make the next release of PHIST more error resistant.

Adam