Improve pdb_merge to include TERs and END lines

[joaomcteixeira]

Closes #149

[amjjbonvin]

But are END lines first filtered out? There should not be multiple END statements

[joaomcteixeira]

That's right. However, I feel we are adding too much to the purpose of merge. All that should be done by tidy. Maybe if we exchange the -strict option for tidy we don't need to touch the merge. What do you think? Because the merge was really designed to be a concatenator

[amjjbonvin]

Yes - but why does it remove the END statements then? Other code don’t do that (e.g. pdb_chain).

[joaomcteixeira]

The current version of pdb_merge does not remove any lines from the input files. It's a straight concatenation.

pdb-tools/pdbtools/pdb_merge.py

Lines 84 to 87 in 2a070bb

	for fhandle in flist:
	for line in fhandle:
	yield line
	fhandle.close()

[amjjbonvin]

So a simple cat command effectively. Meaning pdb_tidy should always be run to correct things. A very different behaviour than `pdb_mkensemble` for example.

[joaomcteixeira]

Yes. mkensemble has the clear purpose of making a correct ensemble of structures. While merge assumes the user knows what he/she is doing when concatenating the files. Likely the user cleaned the PDBs before using merge.

[amjjbonvin]

hmmm…. assumptions… this is asking for troubles… It is this make clear to the users?

…

Yes. mkensemble has the clear purpose of making a correct ensemble of structures. While merge assumes the user knows what he/she is doing when concatenating the files. Likely the user cleaned the PDBs before using merge.

[joaomcteixeira]

Good point. Let's first clarify that to users before changing the behavior of pdb_merge.

[joaomcteixeira]

I was reviewing this now, and I still agree we should not touch pdb_merge for this purpose and pipe the results to pdb_tidy as the logic is not that straightforward. It is stated in the docs.

The contents are not sorted and no lines are deleted (e.g. END, TER
statements) so we recommend piping the results through `pdb_tidy.py`.

[amjjbonvin]

How will you explain users that pdb_merge does require pdb_tidy to make a correct PDB while pdb_mkensemble not...

[joaomcteixeira]

You are right. I am addressing this. It's not as straightforward as adding an END because also TER was not accounted for. If TERs exist in the output is because they were already present in the input. I am working on it.

[joaomcteixeira]

When doing pdb_merge, should atom numbers be renumbered starting from 1? This would change the original atom numbers but avoid repeated numbers.

[amjjbonvin]

Probably good indeed

[joaomcteixeira]

Hi @amjjbonvin

I have addressed this issue, plus some other details. Now, pdb_merge operates as follows:

• The merged PDB file will represent a single MODEL.
• Comment lines in input PDBs will be ignored (REMARK, ...)
• Atom numbers are restarted from 1.
• CONECT lines are yield at the end. CONECT numbers are updated to
  the new atom numbers.
• Missing TER and END statements are placed accordingly. Original
  TER and END statements are maintained.

Inside tests/data/ there are three PDBs, dummy_merge_A/B/C.pdb that you can use to test. Test also with others you may know, and let me know.