Can't append new data to existing cool file.

[1944498970]

Hello!
When I use cooler.create_cooler(cool, bins=df, pixels=df1, dtypes={'count': 'float64'},mode='a') to add new data to my existing COOL file (where the new data and old data are from the same chromosome but different regions), the old data in my COOL file gets deleted. How can I use this method to add new data without deleting the old data.
Thanks.

[1944498970]

This code snippet can illustrate the issue:
import numpy as np
import cooler
chrom=['chr1','chr1']
start=[0,100]
end=[100,200]
bins=pd.DataFrame({'chrom':chrom,'start':start,'end':end})
pixels=pd.DataFrame({'bin1_id':[0],'bin2_id':[1],'count':[1]})
pixels
cooler.create_cooler('test.cool', bins=bins, pixels=pixels, dtypes={'count': 'float64'},mode='w')

new_pixels=pd.DataFrame({'bin1_id':[1],'bin2_id':[1],'count':[2]})
cooler.create_cooler('test.cool', bins=bins, pixels=new_pixels, dtypes={'count': 'float64'},mode='a')

clr = cooler.Cooler('test.cool')
a=clr.matrix(balance=False).fetch('chr1:0-200')
print(a)

the output is [[0. 0.][0. 2.]] the old data( pixels=pd.DataFrame({'bin1_id':[0],'bin2_id':[1],'count':[1]}) ) can't fetch.

[nvictus]

It's not meant to be used that way. a mode allows you to insert a complete cooler data collection (i.e. contact matrix) into a file with other cooler data collections. To build a single data collection incrementally, you can either:

1. Provide any sort of iterator of dataframes to cooler.create_cooler, like a generator, e.g.:

def iter_pixels():
    ...
    yield chunk1
    ...
    yield chunk2
    ...

cooler.create_cooler('test.cool', bins=bins, pixels=iter_pixels(), dtypes={'count': 'float64'})

2. Create separate cooler data collections, and then merge them.

cooler.create_cooler('test1.cool', bins=bins, pixels=pixels1, dtypes={'count': 'float64'})
cooler.create_cooler('test2.cool', bins=bins, pixels=pixels2, dtypes={'count': 'float64'})

cooler.merge_coolers('test.cool', ['test1.cool', 'test2.cool'])

[1944498970]

Thank you for your response. This answer truly resolved my issue.