How implement rbd export/import using go-ceph?

[mysugar]

I did not found export func on rbd.Image stuct , any idea to implement this?

[agarwal-mudit]

Are you talking about this?

[nixpanic]

What kind of Export() interface do you expect? You already noticed you can use Read() and Write() to the image in #318.

[mysugar]

I mean the rbd export on linux cli

[root@nceph ~]# rbd info test/disk01
rbd image 'disk01':
	size 10GiB in 2560 objects
	order 22 (4MiB objects)
	block_name_prefix: rbd_data.10286b8b4567
	format: 2
	features: layering
	flags: 
	create_timestamp: Wed Jun 17 15:40:08 2020
[root@nceph ~]# rbd export test/disk01  /tmp/disk01.rbd
Exporting image: 100% complete...done.
[root@nceph ~]# du -sh /tmp/disk01.rbd 
639M	/tmp/disk01.rbd

If use Read() the size of /tmp/disk01.rbd will be 10G.

[nixpanic]

Ah, that probably uses rbd_read_iterate2() to skip writing the zero-filled areas of the image. We do not have that in go-ceph yet.

Or it could be used in combination with SeekHole and SeekData, but we do not have that yet either.

[phlogistonjohn]

We have issue #279 filed for rbd_read_iterate2. I don't think seek style interfaces are implemented (in librdb). Where they do exist, in cephfs, the Seek(Hole|Data) whence values don't actually seek the holes or data, they're just stubs (see issue #227).

In the short term I suggest doing a simple copy loop as @nixpanic suggests with read calls and perhaps do hole detection yourself (eg. read N-size blocks and if the block is all zeros seek forward in your destination file). In the future, when rbd_read_iterate2 or other hole sensitive calls are available use it as an optimization. It's not pretty but that's today's situation AFAICT.

[tareksha]

Hi @nixpanic, @phlogistonjohn, the export command in rbd cli does not only dump the sparse contents. It produces a file in a special format that also contains the image metadata and diffs.
https://docs.ceph.com/en/pacific/dev/rbd-export/
Is it within go-ceph's scope to have a full implementation of export/import?

[phlogistonjohn]

I'd say no, a full implementation of a cli/function call that fully replicates the rbd command's import and export would be out of scope for go-ceph. I see go-ceph as a set of building blocks that you can use to interact with ceph using public APIs. I'd happily work to add any functions that are missing from go-ceph that are used to implement import/export. Additionally, I don't know how well standardized/documented the file format you mention is. I don't know if the file is constructed by exported library calls or is just private to the rbd cli tool.

I would be open to having discussions [1] about how to implement such a thing and helping investigate what calls are used but I don't think go-ceph would be the right place for a full blown implementation.

[1] - Thus I have converted the original issue to a discussion. :-)

[tareksha]

As it seems the rbd export format is stable, documented and versioned. Currently it is at version v2. The documentation link I sent earlier details the structure of this file format so implementing it in go should be straight forward imho.

I think it makes sense to have some interoperability with the rbd cli since it is the de-facto standard command line tool for working with RBDs in ceph. I can think more extreme and wonder if we can actually modernize the rbd cli and rewrite it using go on top of go-ceph:)