Decoding bitstream to image but without frequency and colour transforms

[likewise]

I would like to decode a bitstream to a full-size image, but without the reverse frequency and colour transforms in effect. So as to show the LL, HL, LH, HH and further frequencies.

As I think the wavelet transform does also copy from src to dst, altogether not calling them would not work, but I could use some help to understand what does need to be done.

Thanks

[likewise]

Exploring the code further, it seems a good possibility is to create a fourth set of (backward) transforms implementations (besides the generic C and SSE/AVX variant sets) that do not perform the actual wavelet transform/decomposition.

As the code is not documented, I could use a little guidance. I will fork the code for my experiments.

[aous72]

Hi Leon,

Happy to help.
I assume this is to help you understand how the JPEG2000 algorithm works.
The code is designed for efficiency, and therefore might be hard to follow. It tries to work with one row of data at a time, buffering the smallest possible number of rows.

One way, as you mentioned, is to modify the wavelet transform and color transform subroutines. You also need to stop the code from converting data back to its original format, which are in integers. It might be worth noting that the reversible 5/3 wavelet employs an integer pipeline while the irreversible 9/7 wavelet employs a floating point pipeline.
This can work if you produce interleaved output data; all subband samples (or coefficients) are interleaved. For example, if you have 2 levels of decomposition, you would have the following arrangement in each subband:

LL2 HL1 HL2 HL1
LH1 HH1 LH1 HH1
LH2 HL1 HH2 HL1
LH1 HH1 LH1 HH1

Generating a de-interleave image requires allocating many more buffers than what the code allocates.

What I would recommend -- still requires work on your side, but it is different to what you think -- is to:

1. Disable SIMD instructions, compiling only C++ code. Use -DOJPH_DISABLE_INTEL_SIMD=ON at cmake, or define this symbol during compilation.
2. Extract, using your own code, one subband line (or row) at a time, and store it in a buffer you also create, perhaps you have a statically declared class or something. The member function line_buf *subband::pull_line() in ojph_codestream.cpp is where these lines are produced; in fact the last line return lines; returns this line in lines[0]. Most of the code in this subroutine calculates the size of the codeblock, because, once one row of codeblocks is consumed, the codeblock objects are re-initialized to be used for the next row of codeblocks.
3. the buffer lines[0] has one row of one subband. The lines are produced in raster order. You can tell the subband and resolution, by looking at this->res_num and this->band_num. For 5 levels of decomposition, we have res_num from 0 to 5, inclusively. Resolution 0 has only band_num = 0, while other resolutions have band_num from 1 to 3, inclusively. You also have this->band_rect to tell you the width and height of the subband.
4. You need to figure out which color component it is. Pass the comp_num to subband::finalize_alloc() and store it as a member variable in subband for your own usage. This can be done in the implementation of resolution::finalize_alloc().

Hope this helps.

Let me know if you have any issues.

Kind regards,
Aous.

[likewise]

Hello Aous,

Thanks for your generous answers.

My goals are indeed to understand J2K as well as the HT part and then to explore acceleration on FPGA architectures.

1. I already used -DOJPH_DISABLE_INTEL_SIMD=ON.
2. I have no clue how to iterate over the subbands yet. My current approach is to use ojph_expand to decode the image (leaving it in working order), assuming all subbands are iterated over eventually (albeit in non scan order). Then as you suggest, pull out the frequency domain data just before "return lines" and write it to a separate file (for now, proof of concept).
3. I would like to know the line number of the subband is being pulled in subband::pull_line()?. It seems this->cur_line counts down rather than up, so I have added this->line_num for this.
4. Thanks, implemented.

I will put my code in a fork, and I may surely be asking further questions, thanks.

With kind regards,

Leon.

[aous72]

Hello Leon,

I have no clue how to iterate over the subbands yet.

The code pulls a line from a subband when it needs one; the code would then use it to perform the wavelet transform or whatever. So, the code has its own mind when to pull a line. 😃

I would like to know the line number of the subband is being pulled in subband::pull_line()?. It seems this->cur_line counts down rather than up, so I have added this->line_num for this.

This is reasonable. 👍

Let me know if you have other questions.

Cheers,
Aous.

[likewise]

Hello Aous,

Thanks, based on your suggestions I have something initially working where I see what I expect (with some artifacts I am looking into), i.e. I see the DC components in top-left corner and further H/V decompositions as expected.

The code pulls a line from a subband when it needs one

Is there a reference to the (non-naive) algorithm used?

I will push my code into a branch of my fork today or Monday.

Kind regards,

Leon.

[aous72]

Is there a reference to the (non-naive) algorithm used?

I am not very sure if this is explicitly documented, but the paper "Multithreaded processing paradigms for JPEG2000" by David Taubman; let me know if you cannot find it.
The code tries to keep things as cache-local as possible. That's why it tries to consume data as soon as they are produced.

On a second note: This is a list of HTJ2K resources
https://github.com/chafey/HTJ2KResources
There is a discord server near the bottom of this list.