/opt/ceph-container/bin/osd_disk_prepare.sh: line 46: ceph-disk: command not found

[benapetr]

There is already similar bug #1713 which was closed without solution.

If this is something deprecated, then please delete it from official documentation and from the osd_disk_prepare.sh so that it doesn't confuse people. Following the docs at https://github.com/ceph/ceph-container/tree/main/src/daemon and the info in the previous bug report, I was able to create and activate OSD, using these 2 commands

$ docker run --rm --privileged --net=host --ipc=host \
                    -v /run/lock/lvm:/run/lock/lvm:z \
                    -v /var/run/udev/:/var/run/udev/:z \
                    -v /dev:/dev -v /etc/ceph:/etc/ceph:z \
                    -v /run/lvm/:/run/lvm/ \
                    -v /var/lib/ceph/:/var/lib/ceph/:z \
                    -v /var/log/ceph/:/var/log/ceph/:z \
                    --entrypoint=ceph-volume \
                    docker.io/ceph/daemon\
                    --cluster ceph lvm prepare --bluestore --data /dev/xxxxxx
// assuming the OSD id created is 0
$ docker run --rm --privileged --net=host --pid=host --ipc=host \
                    -v /dev:/dev \
                    -v /etc/localtime:/etc/localtime:ro \
                    -v /var/lib/ceph:/var/lib/ceph:z \
                    -v /etc/ceph:/etc/ceph:z \
                    -v /var/run/ceph:/var/run/ceph:z \
                    -v /var/run/udev/:/var/run/udev/ \
                    -v /var/log/ceph:/var/log/ceph:z \
                    -v /run/lvm/:/run/lvm/ \
                    -e CLUSTER=ceph \
                    -e CEPH_DAEMON=OSD_CEPH_VOLUME_ACTIVATE \
                    -e CONTAINER_IMAGE=docker.io/ceph/daemon \
                    -e OSD_ID=0 \
                    --name=ceph-osd-0 \
                    docker.io/ceph/daemon

However, the second container mounts the tmpfs /var/lib/ceph/osd/ceph-N/ folder with the configs extracted using ceph-bluestore-tool, spawns OSD for a while (that works OK) and after a while it terminates (I assume that's as it's supposed to be, since it's just activate command?).

Problem is that ever since, I am unable to start a persistent OSD daemon, only thing that works is to run that OSD_CEPH_VOLUME_ACTIVATE again, which restarts the OSD for few minutes and then it shuts down again. Any other combination of OSD_TYPE with CEPH_DAEMON being OSD doesn't work. Either it fails with something like

2023-07-05 14:00:47  /opt/ceph-container/bin/entrypoint.sh: Device detected, assuming ceph-disk scenario is desired
2023-07-05 14:00:47  /opt/ceph-container/bin/entrypoint.sh: Preparing and activating /dev/sdb
HEALTH_OK
Error: /dev/sdb: unrecognised disk label
/opt/ceph-container/bin/osd_disk_prepare.sh: line 46: ceph-disk: command not found

Or it complains that /var/lib/ceph/osd/ceph-N doesn't contain any data (which it doesn't because OSD_CEPH_VOLUME_ACTIVATE only mounts temporary tmpfs there and gets the files using bluestore tool).

What is the actual working way to spawn a persistent OSD container then? I don't want to reverse-engineer cephadm to figure this out, isn't there some working documentation? Also please if any of these steps are deprecated, then delete them from the code and respective documentation. Telling people to use something deprecated is a bug of its own.

[benapetr]

I would like to stretch out that as of now, not a single OSD related command mentioned in https://github.com/ceph/ceph-container/tree/main/src/daemon is actually working. Most of them fail with this error, others fail with different errors.

[ppoektos]

Would it be fixed or no?

[VenkateswaranJ]

Any update?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

[benapetr]

@github-actions LOL

this is really a way to deal with issue reports. Just close them, because despite the entire project is completely defunct, nobody cares.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.