Add support for OV5640 camera #3063
Conversation
|
what is the impact on the RAM usage? |
|
@caco3 There's no impact on RAM usage at all. It works the same way as ov2640 where cropping is done on the camera module itself. |
|
Process:
However, further changes to the implementation are necessary, e.g. at the sharpness.... Ablauf:
Allerdings sind noch weitere Änderungen an der Implementierung notwendig, z.B. bei sharpness..... |
@SybexX May I know what you would like to change? |
|
e.g. so: I made a few quick adjustments, but I don't know if it works because I don't have an OV5640. |
|
@SybexX I've adjusted the sharpness handling to your version and it's working fine on my OV5640 system. I changed the sharpness range because the officially supported sharpness range is from -3 to +3. Your |
|
I specifically set the sharpnessLevel to 2. I know that the OV5640 can do more, but we should adapt everything to the OV2640, Sometimes less is more^^ |
|
|
|
@SybexX Thank you for reviewing. I've refactored my code based on your sample code but had to make a few changes to make it work with the OV5640.
My apologies, it is not clear to me that OV3660 (which is the other supported camera) behaves the same way as OV2640 and parts of the code are hitting the OV2640 registers directly (presumably the OV3660 has the same registers as OV2640?). Personally, I prefer to be more explicit in the code when it comes to handling hardware specific behaviour, like Either way, I am not particular with how the code is written as long as it is well documented via comments or code itself so other developers understand the intention of the author. |
|
The registers don't matter, the properties of the camera are more important. |
That was an oversight on my part as well. We should probably test sharpness against OV3660 and update the code accordingly? |
|
If something is not available on a camera model, the esp32-camera library actually returns a -1 https://github.com/espressif/esp32-camera/blob/master/sensors/ov2640.c |
|
I propose this sharpness handling code: void CCamera::SetCamSharpness(bool _autoSharpnessEnabled, int _sharpnessLevel)
{
sensor_t *s = esp_camera_sensor_get();
if (s != NULL)
{
_sharpnessLevel = min(2, max(-2, _sharpnessLevel));
camera_sensor_info_t *sensor_info = esp_camera_sensor_get_info(&(s->id));
if (sensor_info != NULL)
{
if (sensor_info->model == CAMERA_OV5640 || sensor_info->model == CAMERA_OV3660)
{
if (_autoSharpnessEnabled)
{
// autoSharpness is not supported, default to zero
s->set_sharpness(s, 0);
}
else
{
s->set_sharpness(s, _sharpnessLevel);
}
}
else if (sensor_info->model == CAMERA_OV2640)
{
// The OV2640 does not officially support sharpness, so the detour is made with the ov2640_sharpness.cpp.
if (_autoSharpnessEnabled)
{
s->set_sharpness(s, 0);
ov2640_enable_auto_sharpness(s);
}
else
{
ov2640_set_sharpness(s, _sharpnessLevel);
}
}
}
}
else
{
LogFile.WriteToFile(ESP_LOG_ERROR, TAG, "SetCamSharpness, Failed to get Cam control structure");
}
}
|
|
@SybexX Looks like OV3660 |
|
or so, then the ESP is spared the second comparison ^^ void CCamera::SetCamSharpness(bool _autoSharpnessEnabled, int _sharpnessLevel) } |
|
|
@SybexX I have updated the code to your suggested sharpness handling and also updated zoom handling code to better deal with OV3660 but I don't have an OV3660 so can't test. I have only tested with OV5640 so far. |
|
|
@SybexX The new I've also increased the OV5640 full frame size to match the datasheet. |
|
still necessary adjustment in the edit_reference.html: still necessary adjustment in the edit_config_template.html: |
|
@SybexX Thank you for testing with OV2640. I have been testing with OV5640 and haven't got time to test against OV2640. I have incorporated your recommended changes. Thanks for fixing it and adding comments to explain how I have tested your multi-step zoom and it is a very nice idea. I have to use I have also been testing the OV5640 in less than perfect lighting conditions and have to bump the max jpeg quality down to 18 because the camera is unstable below 18. I noticed you changed the max jpeg quality to 6 for OV2640. Is the OV2640 stable at 6 in less than perfect lightning conditions? |
|
I always test on the water meter and there is no problem with the quality = 6. |
The OV5640 datasheet recommends 24MHz and I tried that but it didn't work properly. The image was always half black with some ghosting and the other half of the image was normal. |
|
@SybexX I've checked other examples of running OV5640 on ESP32 and they are running 20MHz clock from what I can see. Not sure what's the maximum the ESP32 + OV5640 combo can handle. I have just found out that both OV3660 and OV5640 are different from OV2640 in these areas as well:
What would you like to do? I'm leaning towards increasing the sharpness range to -3 to +3 (clip it to -2 to +2 for OV2640 with a comment saying -3 or +3 are unstable), increasing the auto-exposure range to -5 to +5 but clip it internally for OV2640, and add de-noise option on the web interface. |
|
Personally, I would limit myself to the functions of the OV2640. |
|
I personnaly think that the AutoSharpness (if that is the same as AutoFocus) is most important, as a lot of users have difficulties with the manual lens adjustment. So it could be better to change to the new camera as a default hardware. |
|
AutoSharpness and AutoFocus are two different functions. |
|
For my gas meter and water meter set up, I can't have a fully enclosed housing because meter readers come around every couple of months to read the meters and they need to be able to look at the digits on the meters. What this means is my image is highly subjected to ambient lighting (sun light), so auto-exposure needs to be on. Other auto functions may need to be turned on to get an optimal image. I have also turned on negative effect (I used to have both grayscale and negative). I'm quite impressed that the neural net has been getting 95% of the readings correctly. Long story short, auto features are important if lighting is not constant. Regarding the focus issue, I agree that it's hard to get the focus right, which is why I have been researching the OV5640 with autofocus. I think it will be a more user friendly solution. The user adjusts the focus through the web interface during edit-reference step and leave the focus at the configured level during normal operation (no auto-focus). We would need to proof that focus can be adjusted manually first. I have shared my research here: #2162 (comment) As for the OV3660 and OV5640 specific features (de-noise and bigger auto-exposure range), I'm happy to just have them on the config page as expert settings. |
|
I adjusted a few things and also added Denoise(Config-Page), all the changed files are in the zip. |
|
@SybexX I've incorporated your changes and cleaned the code up a little. My OV5640 is still working, which is good. The higher level of auto-exposure is definitely having an impact on the image. I haven't got time to test de-noise yet and how much impact it has on image quality. |
|
@jasaw I bought an OV5640 for testing and found that the colors red and blue swapped in the zoom function^^ With this change they are back to normal for me: Does your camera get so warm that the temperature of the ESP rises by around 6 to 10 degrees? |
|
@SybexX I don't have swapped red blue colour problem on both my OV5640. I don't understand how The OV5640 does run a little warmer than the OV2640, so I stuck the camera against the microSD card holder using a thermal tape, using the ESP's PCB as a heatsink. I have 2 OV5640 systems, one of them running at 85 degrees (under direct sunlight), and another running at 37 degrees (shaded). I have just purchased a few heatsinks to put on the ESP32 metal can. Could your camera red blue colour swap problem be caused by high temperature? How good is your power supply? The OV5640 is more sensitive to power supply fluctuation. I power my OV5640 systems from 12V to 5V DC-DC converter, capable of 3A output with 220uF output capacitor. |
|
It won't be the temperature, maybe the version of the camera, because I can set the quality to 8 on this camera without any problems (in very poor lighting conditions). With the OV2640 it's the other way around, there's a color swap if it's not divisible by 8 without a remainder. |
|
@SybexX I can't get the This is my OV5640 as reported by the software: I noticed these differences between my ov2640 and ov5640:
|
|
s->set_res_raw(s, xOffset + 1, yOffset + 1, xOffset + xTotal + 1, yOffset + yTotal + 1, 0, 0, frameSizeX, frameSizeY, xOutput, yOutput, scale, binning); works too. I (4955) cam_hal: cam config ok
|
|
Interesting... Update: Any good ideas on how we can narrow down the difference between your camera and mine? How should we handle this? I have only found this so far espressif/esp32-camera#447 This is my OV5640 camera:
|
|
I've tried a few things, but as soon as the values are divisible by 2 without a remainder, the colors are swapped. |
|
@SybexX What's your Espressif SDK version? Mine is this: |
|
|
@SybexX I'm not sure what else we can do to track down the root cause of your camera red blue colour swap.
I have shared my firmware here for you to test to help isolate the problem. Other than this, I'm not sure what else we can do to track this issue down, then the next step is to probably have a red blue colour swap workaround flag that can be controlled via the web interface? |
|
me too^^ I assume it's due to the camera's firmware, your number has an 8 and mine has a B. Since there are no differences when querying the version and revision, the only way to do this is via the settings page. maybe like this: AI-on-the-edge-device-ov5640_support_changes.zip |
|
@SybexX I've pushed your changes, however I did not comment out the code that limits the jpeg quality range. Was it commented out on purpose? if (CCstatus.CamSensor_id == OV5640_PID)
{
qual = min(63, max(18, qual)); // Limit quality from 18..63 (values lower than 20 tent to be unstable)
}
else
{
qual = min(63, max(8, qual)); // Limit quality from 8..63 (values lower than 8 tent to be unstable)
} |
|
We already query the validity of the value in MainFlowControl.cpp and ClassFlowTakeImage.cpp and my OV5640 works with a value of 8 without any problems(even in poor lighting conditions), so I commented it out. case OV5640_PID:
if (CCstatus.isImageColorSwaped == true)
{
qual = min(63, max(8, qual));
}
else
{
qual = min(63, max(18, qual)); // Limit quality from 18..63 (values lower than 20 tent to be unstable)
}
frameSizeX = 2592;
frameSizeY = 1944;
// max imageSize = ((frameSizeX - CCstatus.ImageWidth) / 8 / 4) - 1
// 59 = ((2560 - 640) / 8 / 4) - 1
if (imageSize < 59)
{
_imageSize_temp = (59 - imageSize);
}
SanitizeZoomParams(_imageSize_temp, frameSizeX, frameSizeY, _imageWidth, _imageHeight, _offsetx, _offsety);
SetCamWindow(s, frameSizeX, frameSizeY, _offsetx, _offsety, _imageWidth, _imageHeight, CCstatus.ImageWidth, CCstatus.ImageHeight);
break; |
|
@SybexX OK, I have updated the jpeg limits for our 2 variants of OV5640 and removed the OV2640 jpeg limit. Do you know whether OV3660 has a limit? I guess we just update the OV3660 limit when someone can actually test it? For now we just handle it the same as OV2640? |
|
So far I don't know anyone who has the OV3660, and if anyone does, to my knowledge there have never been any complaints about problems with the camera. |
Add support for OV5640 5MP camera. The camera has a resolution of 2560 x 1920 but the output will be scaled down to the configured resolution e.g. 640 x 480. When zoom is enabled, a 640 x 480 window is cropped from the full 2560 x 1920 frame, therefore giving double the magnification compared to OV2640.
This is a zoom comparison between OV5640 and OV2640.
OV5640:
OV2640:
