Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for modern libcamera Raspberry Pi camera stack #2425

Open
scul81 opened this issue Apr 23, 2022 · 34 comments · May be fixed by #2765
Open

Add support for modern libcamera Raspberry Pi camera stack #2425

scul81 opened this issue Apr 23, 2022 · 34 comments · May be fixed by #2765

Comments

@scul81
Copy link

scul81 commented Apr 23, 2022

Hi,

I am using the latest version Raspbian (2022-04-04) and the latest version of Motioneye (0.42.1)

My infrared nightvision camera is connected to the internal Raspberry Pi camera port.
When I run libcamera-hello I can see that the camera is working
The devices are connected under Unicam /dev/media1 and ISP /dev/media0
The stream is 1296x972-YUV420
When I load Motioneye I cannot seem to add the camera in any way possible. I have tried all 20 or so devices in the list and none of them work.

@MichaIng
Copy link
Member

The cameras I deal with are shown as /dev/video*. Can you check whether they are listed here:

v4l2-ctl --list-devices

It may be related to RPi MMAL camera detection with legacy camera stack vs modern camera stack. Currently we use vcgencmd get_camera to detect the RPi camera module, and I'm not sure whether this works when using KMS + modern camera driver stack with V4L2 and libcamera. I guess you have camera_auto_detect=1 and dtoverlay=vc4-kms-v3d in your /boot/config.txt?

@scul81
Copy link
Author

scul81 commented Apr 23, 2022

Hi the output of that command gives

bcm2835-codec-decode (platform:bcm2835-codec):
	/dev/video10
	/dev/video11
	/dev/video12
	/dev/video18
	/dev/video31
	/dev/media3

bcm2835-isp (platform:bcm2835-isp):
	/dev/video13
	/dev/video14
	/dev/video15
	/dev/video16
	/dev/video20
	/dev/video21
	/dev/video22
	/dev/video23
	/dev/media0
	/dev/media2

unicam (platform:fe801000.csi):
	/dev/video0
	/dev/media1

vcgencmd get_camera
supported=0 detected=0, libcamera interfaces=0

and yes those commands are in the /boot/config.txt

@scul81
Copy link
Author

scul81 commented Apr 23, 2022

I have done some more reading, this thread is helpful.
https://forums.raspberrypi.com/viewtopic.php?t=325259

The output of "libcamera-hello --list-cameras" equals
[1:19:22.870166766] [1172]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3424-e68e0f1e
[1:19:22.897339457] [1174] ERROR CameraSensor camera_sensor.cpp:551 'ov5647 10-0036': Camera sensor does not support test pattern modes.
[1:19:22.929102130] [1174]  INFO RPI raspberrypi.cpp:1317 Registered camera /base/soc/i2c0mux/i2c@1/ov5647@36 to Unicam device /dev/media1 and ISP device /dev/media0
Available cameras
-----------------
0 : ov5647 [2592x1944] (/base/soc/i2c0mux/i2c@1/ov5647@36)
    Modes: 'SGBRG10_CSI2P' : 640x480 1296x972 1920x1080 2592x1944

@MichaIng MichaIng changed the title Infrared camera (OV5647) not recognised Add support for modern libcamera Raspberry Pi camera stack Apr 24, 2022
@MichaIng
Copy link
Member

I'm not sure how such infrared camera is commonly provided. E.g. based on v4l2-ctl --list-devices output it seems like /dev/media1 can also be obtained from /dev/video0, but /dev/media0 is associated with a larger number of /dev/video* devices, so not sure if/how this may work.

However, you added it as MMAL camera, right? In this case vcgencmd get_camera is used, and this only works with the legacy camera stack where the output needs to start with supported=1 detected=1. Since you have the modern camera stack enabled (which is perfectly fine, in general, the default on modern RPi OS), this isn't recognised. For this we need to implement libcamera utilities support. I hence adjusted this issue to reflect it being a feature request for supporting libcamera MMAL cameras.

But it would be interesting to know whether the cameras an be accessed via V4L2. It should show the large list of devices from your v4l2-ctl --list-devices output. Probably you can try whether any of those is working, probably even /dev/media0 and /dev/media1?

... https://www.raspberrypi.com/documentation/accessories/camera.html#v4l2-drivers

Actually to me it seems like MMAL is generally the legacy driver, V4L2 the one used with libcamera. Probably the libcamera- utilities are not necessary at all. The question is whether motion is able to catch the camera stream with modern stack, e.g.: #1857

Related issue on motion project: Motion-Project/motion#1434
Seems like currently motion is not able to handle the modern RPi camera stack. Solution would be to either switch to legacy stack, or providing the stream with other software, like mjpg-streamer, then adding it as local network camera.

@scul81
Copy link
Author

scul81 commented Apr 26, 2022

I have not added it as an MMAL camera, as that does not work.

image

The interface produces an error, the camera box remains black and you can click the OK button a million times but it will never proceed.

Under V4L2 you get the following options:
image

I have tried adding every single one of them but none of them work.

When I select unicam I get video0, also doesn't work.
image

I have tried switching the camera back by enabling "legacy camera support" in raspi-config and the V4L2 output then becomes different, but the issue remains that none of the cameras selected work.

@MichaIng
Copy link
Member

Ah right, makes sense that it doesn't work as MMAL with modern camera stack. I'm wondering why the cameras are spread across so many /dev devices. CLI can only access one at a time, confusing. Sadly I have no DSI (?) camera module to test whether this is expected, or special for your ISP camera.

After you switched to legacy camera stack, could you retry adding it as MMAL? Respectively does it show "supported" then in:

vcgencmd get_camera

?

@scul81
Copy link
Author

scul81 commented Apr 26, 2022

I have now switched the configuration back to legacy stack. Under the MMAL selection the problem remains the same. The block remains blank and you can click OK but it never proceeds.

Under V4L2 I have more options.
image

I will try them out and update this post.
UPDATE. I tried all of them and none work.

The result of the command is:
supported=1 detected=1, libcamera interfaces=0

The result of "v4l2-ctl --list-devices" is:
image

None of them work in motioneye. ;(
By the way this is the camera:
https://www.aliexpress.com/item/1005001586246249.html

@MichaIng
Copy link
Member

Looks like we should add the device path to the camera name, for such cases where multiple devices have the same name, or one camera provides multiple devices for whatever reason.

If vcgencmd get_camera now reports supported=1 detected=1, libcamera interfaces=0, you should see the camera(s) in the MMAL list, isn't it? ... ah

Under the MMAL selection the problem remains the same. The block remains blank and you can click OK but it never proceeds.

This is unexpected. We check for MMAL cameras via if support.startswith('supported=1 detected=1'):, which should be true as of your output 🤔. Can you verify in shell:

[[ $(vcgencmd get_camera) == 'supported=1 detected=1'* ]] && echo 'Yes it does!' || echo 'It does NOT!'

@scul81
Copy link
Author

scul81 commented Apr 28, 2022

Well

Camera detected with that command
image

No camera detected in motioneye
image

@MichaIng
Copy link
Member

Strange, are there any errors reported?

journalctl -u motioneye

@scul81
Copy link
Author

scul81 commented Apr 30, 2022

image

@MichaIng
Copy link
Member

MichaIng commented Apr 30, 2022

Ah right, you are using the Python 2 version, which contains a nowadays buggy check for vcgencmd get_camera output, so not the one I posted above. Sorry I should have seen this earlier. Please try the Python 3 version, following these install instructions: https://github.com/motioneye-project/motioneye/tree/dev#installation

You could then also retry with the modern camera stack, adding the camera as V4L2 one. But the motionEye version of course doesn't affect the many /dev camera device paths, still now sure whether/how the camera is supposed to be accessed then.

@scul81
Copy link
Author

scul81 commented May 16, 2022

I just tried this, installation got stuck on this
image

@MichaIng
Copy link
Member

MichaIng commented May 16, 2022

Is this Raspberry Pi OS 64-bit (aarch64) or 32-bit (armv7l)?

uname -m

If it's 64-bit, you need to install the additional development headers, which can be skipped only on 32-bit RPi OS:

apt install python3-dev libcurl4-openssl-dev gcc libssl-dev

If it's indeed 32-bit, then assure that you followed the step to add piwheels as pip source, from where the pre-compiled pycurl wheel can be pulled:

printf '%b' '[global]\nextra-index-url=https://www.piwheels.org/simple/\n' | sudo tee /etc/pip.conf > /dev/null

@kaikus
Copy link

kaikus commented May 31, 2022

raspberry 4
bullseye lite 64 bit
fresh installation following https://github.com/motioneye-project/motioneye/tree/dev#installation
including add. headers

I can only get a video (as MMAL Camera) if a switch on legacy support with raspi-config. Is there no way to use motioneye without legacy ?

@MichaIng
Copy link
Member

Not yet. Based on other reports the existing /dev/videoN devices cannot be added as V4L2 camera. I guess we'd need to add support for libcamera tools.

@scul81
Copy link
Author

scul81 commented Jun 6, 2022

Just to let you know I have managed to get it working on the previous version of Raspbian (Buster) through Python3.
The camera will only work as MMAL and not as V4L under any circumstance

@zizumara
Copy link

zizumara commented Jul 6, 2022

I'm having similar problems running Motion with the Arducam 12.3 MP HQ camera (IMX477) with automatic IR-cut filter using 32-bit Bullseye on a Pi Zero 2 W. I did not select the legacy stack in raspi-config, but when I run Motion with libcamerify, it DOES find and use the camera. Unfortunately, none of the mmalcam_params in the motion.conf file are being acted upon.

I'm actually attempting to replicate an existing project that is already working. In the working project, I use an earlier revision of the same camera using Buster on a Pi Zero W, and that project properly responds to the mmalcam_params settings (see https://github.com/zizumara/LPRCam). I would use Buster for the new project, but unfortunately Arducam has changed the camera design so that it ONLY runs under Bullseye (at least that's what their product info says). The older revision of that camera is no longer available.

@GenerallyClumsy
Copy link

GenerallyClumsy commented Jan 10, 2023

Arducam has changed the camera design so that it ONLY runs under Bullseye (at least that's what their product info says).

I'm in the same boat with an Arducam IMX327. I need something to do motion detection and streaming. It's looking hard to find anything that supports libcamera.

@zizumara
Copy link

Arducam has changed the camera design so that it ONLY runs under Bullseye (at least that's what their product info says).

I'm in the same boat with an Arducam IMX327. I need something to do motion detection and streaming. It's looking hard to find anything that supports libcamera.

Actually, you may still be able to get the older version of 12.3 MP camera that runs with Buster. I was able to do that a few months ago and they sold me the older Buster-compatible camera for $20 more. Send your inquiry to sales@arducam.com. Note that if they do have it, the ordering process is a little confusing. They give you a product code that represents a $1 increment to your order, so you have to order 20 of these to increase your order by $20. Just make sure that you are very specific about wanting the camera that is compatible with Buster. I'm not sure if they are still doing this, but it may be worth a shot.

@GenerallyClumsy
Copy link

GenerallyClumsy commented Jan 10, 2023

Arducam has changed the camera design so that it ONLY runs under Bullseye (at least that's what their product info says).

I'm in the same boat with an Arducam IMX327. I need something to do motion detection and streaming. It's looking hard to find anything that supports libcamera.

Actually, you may still be able to get the older version of 12.3 MP camera that runs with Buster. I was able to do that a few months ago and they sold me the older Buster-compatible camera for $20 more. Send your inquiry to sales@arducam.com. Note that if they do have it, the ordering process is a little confusing. They give you a product code that represents a $1 increment to your order, so you have to order 20 of these to increase your order by $20. Just make sure that you are very specific about wanting the camera that is compatible with Buster. I'm not sure if they are still doing this, but it may be worth a shot.

Only catch is I have a box of 8 if these cameras already. I feel like there will be huge demand for motioneyes and similar projects to support libcamera as more people buy them.

Would it work for motioneyes to listen to a stream on 127.0.0.1 established natively using a libcamera command?

@axeljerabek
Copy link

@sebPomme
Copy link

sebPomme commented Feb 6, 2023

Maybe using https://github.com/aler9/rtsp-simple-server/ can be useful to retrieve the camera flux and add a rtsp camera in motioneye ?

@ISJ-439
Copy link

ISJ-439 commented Mar 15, 2023

Hello,

I'm having an issue with the dev branch. Should the Dev python 3 branch support the Pi Camera Module 3?

Thanks!

@MichaIng
Copy link
Member

No, otherwise this issue would be closed as completed.

@axeljerabek
Copy link

In my post above, I described an rtmp solution to make the cam module 3 working with motion. maybe you check that again.

@ISJ-439
Copy link

ISJ-439 commented Mar 16, 2023

In my post above, I described an rtmp solution to make the cam module 3 working with motion. maybe you check that again.

I did attempt it however I think the shell script dosnt terminate properly and it caused a crash before I could pkill it. So I sort of gave up on that method as it cost me an hour resetting the OS and installing all the other software like SNMP, ect.

I appreciate the attempt to help.

@raziel420
Copy link

Uh, for shits and giggles, I asked Bard for a solution.

MotionEye uses Motion as its backend for motion detection. MotionEye does not support libcamera directly. However, you can use a workaround to get MotionEye to use libcamera.

The workaround is to use the libcameraify library to create a wrapper around the Motion application. The libcameraify library allows you to use libcamera with applications that were not designed to use libcamera.

To use the libcameraify library with MotionEye, you will need to do the following:

  1. Install the libcameraify library.
  2. Create a wrapper script for the Motion application.
  3. Configure MotionEye to use the wrapper script.

The following instructions will show you how to install the libcameraify library and create a wrapper script for the Motion application.

  1. Install the libcameraify library.

To install the libcameraify library, you can use the following command:

sudo apt-get install libcamerify
  1. Create a wrapper script for the Motion application.

The following is an example of a wrapper script for the Motion application:

#!/bin/bash

# This is a wrapper script for the Motion application.

# The first argument is the path to the Motion application.
MOTION_PATH=$1

# The second argument is the path to the Motion configuration file.
MOTION_CONFIG_FILE=$2

# The third argument is the path to the libcameraify library.
LIBCAMERAIFY_PATH=$3

# Start the Motion application.
$MOTION_PATH $MOTION_CONFIG_FILE

# Start the libcameraify library.
$LIBCAMERAIFY_PATH
  1. Configure MotionEye to use the wrapper script.

To configure MotionEye to use the wrapper script, you will need to edit the motioneye.conf file. The following is an example of a motioneye.conf file that has been configured to use the wrapper script:

# This is an example of a motioneye.conf file that has been configured to use the wrapper script.

# The first argument is the path to the wrapper script.
MOTION_WRAPPER_SCRIPT=/path/to/motion_wrapper.sh

# The second argument is the path to the Motion configuration file.
MOTION_CONFIG_FILE=/path/to/motion.conf

# The third argument is the path to the libcameraify library.
LIBCAMERAIFY_PATH=/path/to/libcameraify

Once you have made the changes to the motioneye.conf file, you can start MotionEye with the following command:

motioneye

MotionEye will start using the wrapper script to start the Motion application. The Motion application will use libcamera to capture frames from the camera and check for motion.

@MichaIng
Copy link
Member

MichaIng commented May 2, 2023

@raziel420
Thanks for the info about libcamerify. However, your instructions are wrong in multiple places, no idea where you got them from?

  1. You mix the wording: it is called libcamerify, not libcameraify (no extra a).
  2. There is not libcamerify APT package, it is called libcamera-tools:
    apt install libcamera-tools
    Only available on Raspberry Pi since Bullseye (via their kernel repository) and on plain Debian since Bookworm.
  3. The wrapper script you use cannot work like that:
    • motionEye does not call the defined motion/wrapper command with the used 3rd argument. How should it even know about it and the path of libcamerify?
    • You call plain motion first, a foreground process which blocks further commands in the script. Additionally, the config file would need to be added via -c option.
    • Even if you fork motion into background, you call libcamerify afterwards without any argument, which is not how it works or can work. If this was the case, no wrapper would be required at all but simply an additional service/call of libcamerify.
    • libcamerify really needs to wrap the executable in a single call, like this: libcamerify /usr/bin/motion
    • motionEye calls motion with a dedicated own config file for each camera, so passing a hardcoded motion.conf breaks it, respectively in practice would double the config file argument.
  4. The three settings you add to motioneye.conf are not valid/interpreted. It is even wrong syntax. There is only one setting which is all we need: https://github.com/motioneye-project/motioneye/blob/ad21146/motioneye/extra/motioneye.conf.sample#L23-L24
  5. There is no motioneye command, it is meyectl and should not be started manually but via systemd service:
    systemctl start motioneye

This is how it should work:

apt install libcamera-tools
echo -e "#!/bin/sh\n$(command -v libcamerify) $(command -v motion) \"\$@\"" > /etc/motioneye/motion_wrapper.sh
chmod +x /etc/motioneye/motion_wrapper.sh
sed -i '/motion_binary/cmotion_binary /etc/motioneye/motion_wrapper.sh' /etc/motioneye/motioneye.conf
systemctl restart motioneye

However, this does not solve the problem with the multiple /dev/video* devices when trying to add an RPi camera module as V4L2 camera. Although, @scul81 @kaikus do the above steps probably allow using /dev/video0 as V4L2 camera (should be the first shown "unicam" in the list) with modern camera stack enabled, or any other of the shown camera devices? It's still that many in the list with recent RPi kernel (Linux 6.1)?

@Jonas1312
Copy link

The instructions above are from Bard, the google's chatgpt.

@MichaIng
Copy link
Member

MichaIng commented May 2, 2023

Ah lol, I missed the re-read the first line. Okay this makes sense and is a perfectly example why one should never rely on any answer one of these AI bots give you for questions without explicit input. Great to shorten text, write summaries, stories and all this, but it does not give you correct/true answers based on research or logic, and it is not even meant to do so.

@raziel420 sorry for thinking this was from you, I was already wondering whether it was mixed up with solution for other software or so 😄. But bringing this topic up probably leads to a solution, at least partly: https://motion-project.github.io/motion_config.html

In the Raspbian version based upon Debian release of 'bullseye' (and presumably later releases), Raspbian allows access to the PI camera via an application called libcamera. libcamera provides access to the camera as a v4l2 device but this interface is only available when using a special application. Users must run Motion using the command libcamerify motion and then specify /dev/video0 in the Motion configuration file.

Matches to what I wrote above (luckily). The other part we need to add in motionEye is to have another list of cameras based on libcamera-hello --list-cameras output, of does this only double with v4l2-ctl --list-devices?

@strips
Copy link

strips commented May 21, 2023

I made a PR for implementing libcamerify support in MotionEye here #2765. Please bear with me as this is my first PR in a public open source project and I'm not a Pyhon developer. Anyways, this patch works for me and has been running since the 19. May without issues.

@don-dolarson
Copy link

don-dolarson commented Jun 11, 2023

Check this guys.

@harakka
Copy link

harakka commented May 22, 2024

Is there anything that people with access to hardware but not that much experience with python could do to help this move forward? Motioneye is still the best open source solution and it would be great to get past this hurdle. The various libcamerify tricks posted in various threads are unfortunately hacks that break down very easily.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.