Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Freeimage plugin with PluginV3 support. #1064

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

michael080808
Copy link

@michael080808 michael080808 commented Feb 26, 2024

Hi, guys! I just spent 2 days to add a small PluginV3 support for the Freeimage.
I need Freeimage as my backend to handle with very complex image datasets including 16-bit/32-bit PNG and others.

I tried my best to follow the code from
https://github.com/imageio/imageio/blob/master/imageio/plugins/pillow.py
https://github.com/imageio/imageio/blob/master/imageio/plugins/freeimage.py
https://github.com/imageio/imageio/blob/master/imageio/plugins/freeimagemulti.py
but I only tested some JPEG and PNG from imageio url images.
GIF and other formats are not fully tested because of my poor understand of the code.

Another thing is that I did not understand how to achieve exclude_applied in the PluginV3.metadata.
Docs are also needed to add to these codes. These parts need your guys help.

Here are the examples of the new 'freeimage' plugin:

import time

import imageio
from matplotlib import pyplot as plt

uris = [
    # 'chelsea.bsdf',
    'newtonscradle.gif',
    'bricks.jpg',
    'meadow_cube.jpg',
    'wood.jpg',
    # 'cockatoo.mp4',
    # 'stent.npz',
    'astronaut.png',
    'camera.png',
    'checkerboard.png',
    'chelsea.png',
    'clock.png',
    'coffee.png',
    'coins.png',
    'horse.png',
    'hubble_deep_field.png',
    'immunohistochemistry.png',
    'moon.png',
    'page.png',
    'text.png',
    'wikkie.png',
    # 'chelsea.zip',
]

if __name__ == '__main__':
    for uri in uris:
        with imageio.v3.imopen('imageio:' + uri, io_mode='r', plugin='freeimage', flags=0, test=None) as image_file:
            for index, image in enumerate(image_file.iter()):
                plt.title(uri + ' %d' % index), plt.imshow(image), plt.show(), time.sleep(1)
        img = imageio.v3.imread('imageio:' + uri, index=0, plugin='freeimage', flags=0, test=None)
        plt.title(uri), plt.imshow(img), plt.show(), time.sleep(1)
        imageio.v3.imwrite(uri, img, plugin='freeimage', flags=0)

@michael080808 michael080808 changed the title Add a Freeimage plugin with PluginV3 support. Add a Freeimage plugin with PluginV3 support. Feb 26, 2024
@michael080808 michael080808 changed the title Add a Freeimage plugin with PluginV3 support. Add a Freeimage plugin with PluginV3 support. Feb 26, 2024
Copy link
Contributor

@FirefoxMetzger FirefoxMetzger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome stuff 🤩 a rewrite of the free image plugin is very needed. Thanks for taking the time :)

I left detailed feedback for the first 100-ish lines and will have to come back for the rest once I can dedicate another hour or two to reviewing. The code so far looks good but is a little too branch heavy to merge as is. This is an easy fix though and I left some comments where I saw opportunities to refactor. I think we shouldn't need a lot of rounds before we can merge :)

Once again, great job and great initiative 🚀

Comment on lines +54 to +62
self,
name,
class_name,
module_name,
*,
is_legacy=False,
package_name=None,
install_name=None,
legacy_args=None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use black as our default formatter. I don't have a strong preference on how code should be formatted (as long as it passes linting), but I suspect that our linter does 😇 .

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use PyCharm IDE to code the python file. It's a little annoying with its formmatter and auto wrap line. So, it's freely to change the format to meet the black.


def iter(self, *, flags: int = 0, **kwargs) -> Iterator[np.ndarray]:
self.update_flags(flags, **kwargs)
# print(inspect.currentframe().f_code.co_name, self._kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a residue from debugging and should be removed?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I forgot to remove it. That's only for debugging.

if isinstance(self._bm, FIBitmap):
for index in range(1):
yield self._bm.get_image_data()
raise StopIteration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have to explicitly raise StopIteration here. Since we are using yield inside the function it will become a generator and Python will automatically raise this exception when the function returns.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe my Python writting style is a little redundant and obsessive to the lines and indents. So there may be a lot unreasonable codes to understand. Thanks for your advise!

elif isinstance(self._bm, FIMultipageBitmap):
for index in range(len(self._bm)):
yield self._bm.get_page(index).get_image_data()
raise StopIteration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. We can just let the function return as usual :)

yield self._bm.get_page(index).get_image_data()
raise StopIteration
else:
raise InitializationError('Can not iterate with current settings.')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise InitializationError('Can not iterate with current settings.')
raise ValueError('Can not iterate with current settings.')

This should not be an InitializationError. We only raise InitializationError during initialization of a plugin to communicate with plugin selection that the plugin is not able to read images of this type.

If possible, we should instead try and use standard exceptions and I think the one that fits best here is a ValueError:

Raised when an operation or function receives an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as IndexError.

self._bm, self._io_mode = None, request.mode.io_mode

_fif = -1
if fi.has_lib():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should reduce nested branches as much as possible. I think we can write the following here:

if not fi.has_lib():
    raise ImportError("FreeImage Library not installed.")

try:
    ...

if request._uri_type == URI_BYTES:
raise InitializationError(f"Freeimage can not {self._io_mode.name} the provided bytes.") from None
else:
raise InitializationError(f"Freeimage can not {self._io_mode.name} {request.raw_uri}.") from None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise InitializationError(f"Freeimage can not {self._io_mode.name} {request.raw_uri}.") from None
raise InitializationError(f"Freeimage can not {self._io_mode.name} `{request.raw_uri}`.") from None

We typically add ticks around the URI to visually separate it from the exception message.

Comment on lines +106 to +107
except Exception as e:
_fif = -1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we expecting a specific exception here? Maybe we don't want to change anything, but I am always a bit wary when adding a generic "catch all exceptions" block, because it might swallow something that we should have raised instead.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember that when _fif = -1 means FreeImage can not recognize the image format. Because after this Exception some procedures should do. I'll double check the v2 code to corfirm it.


_flags = self.flags(_fif, self._io_mode, flags, **kwargs)
self._bm = (fi.create_multipage_bitmap if _fif in [1, 6, 25] else fi.create_bitmap)(request.filename, _fif, _flags)
self._kwargs = kwargs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come you are storing the kwargs here? Is it required to know all arguments used during initialization when later reading/writing data?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's true. I just found that imopen and imread/imwrite give arguments from different places. Is there any better solution?

self._bm.set_meta_data({'ANIMATION': {'Loop': np.array([kwargs.get('loop', 0)]).astype(np.uint32)}})
else:
self._bm.set_meta_data({})
# print(inspect.currentframe().f_code.co_name, self._kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# print(inspect.currentframe().f_code.co_name, self._kwargs)

Left over debugging comment?

@FirefoxMetzger
Copy link
Contributor

FirefoxMetzger commented Mar 4, 2024

Another thing is that I did not understand how to achieve exclude_applied in the PluginV3.metadata.

As for your question, the exclude_applied flag removes all metadata that the reader has applied to the image. The classic example is a file that sets the EXIF flag rotation. Most readers (but not all) will auto-rotate the image when reading and will also return the rotation flag when reading the metadata. This can be confusing for the user as they can't easily tell if they have to manually rotate the image (reader doesn't auto-rotate) or not (reader auto-rotates). The same is true for other metadata, e.g., auto-applied gamma correction, white balance, etc.

To differentiate this, we exclude any metadata that the reader applies by default. This way a reader that auto-rotates won't report the rotation metadata and the user can have a clear feedback on whether they should rotate manually or not.

Another way to think about it is to view exclude_applied as a flag that allows the user to choose between "raw" metadata and "actual" metadata. "raw" metadata (exclude_applied=False) is the metadata that is stored in the file whereas "actual" metadata (exclude_applied=True) is metadata that is still relevant for the image that is served by read/iter.

Docs are also needed to add to these codes. These parts need your guys help.

As for the docs we should add them (and I will help you get them into the correct state). I would like to do a full review first though, and then maybe discuss adding some tests for the new code 🙃 . This way we avoid having to rewrite documentation because we might do slight changes to the signature of some functions as part of the review.

@michael080808
Copy link
Author

Thanks for your patient review! This is my first time to pull request on Github in fact. I am just a little worried about my code's quality. It's very useful for your advise. I just copied a lot V2 code to make sure the logic of procedures right. (Maybe there are some historical codes which do not make sense.) As I said I did not learn the operations of FreeImage very well but I'll try my best to learn it and make the code better.

By the way, this little code is used for my graduation thesis for postgraduate degree. I'm not sure I'll be free in the following months. However, I'll try my best to catch up.

@FirefoxMetzger
Copy link
Contributor

This is my first time to pull request on Github in fact.

Awesome 🚀 thank you for contributing to open source :) If you are unsure how things work or have questions about the process, feel free to ask and I will explain.

I am just a little worried about my code's quality. It's very useful for your advise.

Thanks! Don't stress too much about the initial code quality. The code you see in FOSS repos is quite refined, but that's the result of thorough peer review. A PR usually starts out in a state quite close to what you have submitted, then people provide their feedback, and we incrementally improve what's there. This process allows sharing best practices and knowledge, e.g., you can learn some of the tricks I've picked up over the years and I can learn from some of your ideas.

I'm not sure I'll be free in the following months. However, I'll try my best to catch up.

No problem. The one decision we need to make then is if you want to keep working on this PR or if we should open it to outside contributions. I.e., should I try to find someone else how adds what's missing before we can merge (documentation, unit tests, some refactor) or do you want to do it when you are more free again?

@michael080808
Copy link
Author

michael080808 commented Mar 6, 2024

No problem. The one decision we need to make then is if you want to keep working on this PR or if we should open it to outside contributions. I.e., should I try to find someone else how adds what's missing before we can merge (documentation, unit tests, some refactor) or do you want to do it when you are more free again?

You can find some other contributers to finish the jobs. That is OK for me. I think it can accelerate the procedures and make more people use it ASAP. Best wishes for all the contributers and the project! 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants