Feature request: 16-bit support for Bayer demosaicing

[s-trinh]

I think it could be useful to have Bayer demosaicing support for 16-bit datatype.
If other people agree, please add a +1 to this post.

---

Also, what is the demosaicing method? I believe it is bilinear demosaicing. Could the doc be updated with some details/reference about the algorithm used?

[ermig1979]

Hi!
Is the output 16-bit per chanel BGR or BGRA?

[s-trinh]

Hi.

For my use case, it would be more like 16-bit per channel RGBA but I believe for consistency in the lib supporting both BGR and BGRA mode would be great.

Also, the camera SDK allows retrieving 10-bit and 12-bit Bayer images.

[ermig1979]

Could you give any reference to description of these (10-bit and 12-bit Bayer) image format?

[ermig1979]

As to description of algorithm used in Simd unfortunately I can't remember original paper or reference code which I used to implement it. I remember that the algorithm is smarter than simple bilinear interpolation - it uses information from farer pixels for correction of corners coloring.

[s-trinh]

What I am getting is txt files with header description. Something like mode (10 bits per pixel or 12 per pixel), width/height (2048), etc. I don't have much more information. I don't have access to the camera / software anymore.

Something like uint16_t in input (e.g. 2048x2048) and uint16_t in output (2048x2048x4 or 2048x2048x3) should do the trick for my use case. Conversion from uint16_t to uint8_t for visualization + vertical flipping (I am getting data vertically flipped) if not already implemented would also be useful.

Too bad for the reference.
Matlab uses this reference High-Quality Linear Interpolation for Demosaicing of Bayer-Patterned Color Images.

[s-trinh]

Looks like I am not the only one interested into Bayer conversion 😃

#135

[ermig1979]

There are fast and slow ways to implement 16-bit Bayer to BGR conversion: 1) I use current algorithm for 8-bit and update it to 16-bit. 2) I try to implement algorithms from your paper - it may take much more time.

[ermig1979]

There is third way: you implement Base (scalar) desired algorithm of 16-bit Bayer to RGB and then I will perform it optimization for different SIMD.

[s-trinh]

Thanks.
I will see what I can do. There is no urgent need from my side.

[s-trinh]

@ermig1979
My attempt at Bayer demosaicing (still WIP): https://github.com/lagadic/visp/pull/936/files
There are two implemented methods, bilinear and Malvar interpolation.
I am interested into 16-bit (10/12 bits or even more) and 8-bit handling.
Border handling is tricky. I tried to do something but this is certainly not conventional.

---

There is this paper that could be interesting: Efficient, High-Quality Bayer Demosaic Filtering on GPUs
It eliminates branches for GPU architecture, but not sure if the mentioned optimizations are relevant for CPU since CPU should have advanced branch predictors able to handle that?
From my limited knowledge, SIMD and minimizing cache misses are the keys. So tiling is the key if it is applicable?

---

Just found also this paper: Malvar-He-Cutler Linear Image Demosaicking

[ermig1979]

Hello!
I have seen your implementation of demosaic algorithms:
https://github.com/s-trinh/visp/blob/feat_Bayer_demosaic/modules/core/src/image/vpImageConvert.cpp
I can add these functions to Simd (with some modifications of course) and make their SIMD optimizations.

But this will be a great amount of work. So I have to be shure that it is exactly that you need. In other words: is the quality of these demosaic algorithms is enough to you?

[s-trinh]

To summarize for my use case, ideally:

• a very fast demosaic algorithm, e.g. bilinear demosaicing (this should be what OpenCV have) or similar
• a more advanced algorithm, e.g. the Malvar method like in Matlab
• 16-bit and 8-bit image format support

[ermig1979]

I found a small project which implements many debayering algorithms:
https://github.com/jdthomas/bayer2rgb
It may be useful in order to choose best algorithm.