Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filters: Add Power of Blackman and Power of Garamond windows #220

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

garamond13
Copy link

Libplacebo supports 2 free parameters for windows and in my research (https://github.com/garamond13/Finding-the-best-methods-for-image-scaling) more customizable windows have proven to be superior in quality when scaling images. Also compared to Said window and GNW, for these 2 windows we don't need to do any extra modifications to the code in order to implement them.

Reference for Power of Blackman window: https://github.com/garamond13/power-of-blackman

Reference for Power of Garamond window: https://github.com/garamond13/power-of-garamond-window

src/filters.c Outdated Show resolved Hide resolved
src/filters.c Outdated Show resolved Hide resolved
@haasn
Copy link
Owner

haasn commented Nov 17, 2023

Just a random thought, but if it truly is useful to take the power of a window in a more general context, it would also be possible to just add it to pl_filter_config. This would make using it a bit harder in practice (basically requiring scale=custom), but would provide more flexibility.

From a mathematical PoV, what effect does taking the power of a window have on its frequency response?

Copy link
Owner

@haasn haasn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Missing definition in src/include/libplacebo/filters.h for new public symbol
  • Missing entry in meson.build for new public API

@garamond13
Copy link
Author

Just a random thought, but if it truly is useful to take the power of a window in a more general context, it would also be possible to just add it to pl_filter_config. This would make using it a bit harder in practice (basically requiring scale=custom), but would provide more flexibility.

From a mathematical PoV, what effect does taking the power of a window have on its frequency response?

It would basically control the main lobe width. I don't think that it would be very useful for all the other windows.

@haasn
Copy link
Owner

haasn commented Nov 19, 2023

Looks good to me now, can you squash your commits into two? (One commit to add garamond window, one commit to change blackman window)

A highly customizable window with 2 free parameters.

Reference: https://github.com/garamond13/power-of-garamond-window
Convert existing Blackman window into Power of Blackman window, a highly customizable window with one more free parameter.

Reference: https://github.com/garamond13/power-of-blackman
@garamond13
Copy link
Author

Looks good to me now, can you squash your commits into two? (One commit to add garamond window, one commit to change blackman window)

Done, I think.

@estrogently
Copy link
Contributor

estrogently commented Nov 21, 2023

I'm working on my own script for comparing the performance of windowing functions across a dataset, I'll check how much the additional parameter helps Blackman and how well Garamond performs and post the results back here in a day or so a few hours from this edit.

@estrogently
Copy link
Contributor

Both the powers in Garamond and the power added to Blackman should always be nonnegative, else they produce very silly windows. Additionally, if 0 < Blackman's power < 1, it NaNs for most of the window's radius, and if the inner Garamond power = 0, it zeros the whole filter.

For my testing, I'm using a derivative of the RealSRv3 dataset (I'd also like to try with Manga109 but I'm waiting on access). I'm testing them as windows for ewa_lanczos with 3, 4, and 5 lobes.
A derivative-free optimization algorithm picks the antiringing, sigmoid center, and window parameters. From those I use Robidoux's criteria to derive the blur1 and sigmoid slope2.
The LR dataset is then enlarged with vf_libplacebo, and Butteraugli scores are computed of the enlarged images vs their HR ground truth counterparts. The mean score is then returned to the optimization algorithm, and the optimal score found is taken as the score for that combination of kernel + window + lobes.

Compared to the other adjustable filters, Garamond scores slightly better than Kaiser at 3 and 5 lobes with 0.70% and 0.17% lower scores respectively. At 4 lobes, original Blackman beats it with a 0.16% lower score. The optimal parameters for Garamond seem pretty unstable between lobe counts (with other windows the best parameters for e.g. 3 and 5 lobes are fairly close). Note the additional parameter does result in more overfitting in the optimization.

I wasn't able to obtain results for Power of Blackman. For some reason the resulting function was ill-behaved enough the integration just completely stalled out, which is a first for the windows I've tested. I suspect it can probably get slightly better results than the original Blackman (an extra parameter will just always let it overfit better), though you can't raise the power much before it starts zeroing everything past 20-40% of its radius.

Footnotes

  1. One run does "sharp", another "sharpest"

  2. Enlarge a single white pixel on a black background infinitely using the filter, and pick sigmoidization parameters so that the average luminance remains the same as before enlarging. I have implemented this using numerical integration

@haasn
Copy link
Owner

haasn commented Nov 22, 2023

@estrogently it sounds like we should leave original blackman untouched, and simply add the tunable garamond window?

@estrogently
Copy link
Contributor

@estrogently it sounds like we should leave original blackman untouched, and simply add the tunable garamond window?

that sounds good, I'd also add an fmax with 0.0 to both garamond tunables

@garamond13
Copy link
Author

garamond13 commented Nov 22, 2023

@estrogently could you please retest Power of Blackman with alpha in range [-inf, 0.16]?

For Power of Garamond we should return 1 if n is 0.

- Limit parameters to only positive numbers since we have to use them as exponents in power function.
- Return 1 on n=0, cause entire window will be 0 everywhere otherwise.
@estrogently
Copy link
Contributor

estrogently commented Nov 24, 2023

I capped the alpha at 0.16 and let it run overnight. At 3 and 4 jinc lobes the power converged to 1, at 5 lobes it didn't and did score better than original Blackman, though not as well as Kaiser or Garamond

Power of Blackman function may not be defined for all x when alpha is larger than 0.16, so in that case just use Blackman function.
@garamond13
Copy link
Author

Power of Blackman may not be defined for all x if alpha > 0.16. This was probably causing issues before to @estrogently when doing tests. If you're ok with this solution I can squash this into 2 commits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants