Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplified Linux clipboard support #38

Closed
stephen-huan opened this issue May 9, 2023 · 3 comments
Closed

Simplified Linux clipboard support #38

stephen-huan opened this issue May 9, 2023 · 3 comments

Comments

@stephen-huan
Copy link

stephen-huan commented May 9, 2023

According to the README,

Reading images from clipboard works only on Windows and macOS, on Linux you should read from a directory instead.

Checking the logic, there's two distinct ways manga_ocr handles clipboard:
On Wayland, the pyperclip library is used to get clipboard contents

pyperclip.set_clipboard("wl-clipboard")

while on Windows and MacOS it's through Pillow's PIL.ImageGrab.grabclipboard
img = ImageGrab.grabclipboard()

Although the documentation for grabclipboard() says

Only macOS and Windows are currently supported.

it's undocumented but the Pillow source shows that Linux support was added in 9.4.0 (using wl-paste and xclip).

Thus one can change the existing code from

if sys.platform not in ('darwin', 'win32') and write_to == 'clipboard':
# Check if the system is using Wayland
import os
if os.environ.get('WAYLAND_DISPLAY'):
# Check if the wl-clipboard package is installed
if os.system("which wl-copy > /dev/null") == 0:
pyperclip.set_clipboard("wl-clipboard")
else:
msg = 'Your session uses wayland and does not have wl-clipboard installed. ' \
'Install wl-clipboard for write in clipboard to work.'
raise NotImplementedError(msg)
if read_from == 'clipboard':
if sys.platform not in ('darwin', 'win32'):
msg = 'Reading images from clipboard works only on macOS and Windows. ' \
'On Linux, run "manga_ocr /path/to/screenshot/folder" to read images from a folder instead.'
raise NotImplementedError(msg)
from PIL import ImageGrab
logger.info('Reading from clipboard')

to

        # if sys.platform not in ('darwin', 'win32') and write_to == 'clipboard':
        #     # Check if the system is using Wayland
        #     import os
        #     if os.environ.get('WAYLAND_DISPLAY'):
        #         # Check if the wl-clipboard package is installed
        #         if os.system("which wl-copy > /dev/null") == 0:
        #             pyperclip.set_clipboard("wl-clipboard")
        #         else:
        #             msg = 'Your session uses wayland and does not have wl-clipboard installed. ' \
        #                 'Install wl-clipboard for write in clipboard to work.'
        #             raise NotImplementedError(msg)

        if read_from == 'clipboard':

            # if sys.platform not in ('darwin', 'win32'):
            #     msg = 'Reading images from clipboard works only on macOS and Windows. ' \
            #         'On Linux, run "manga_ocr /path/to/screenshot/folder" to read images from a folder instead.'
            #     raise NotImplementedError(msg)

            from PIL import ImageGrab
            logger.info('Reading from clipboard')

and preserve support for Wayland while adding support for Linux systems running X.
This requires Pillow>=9.4.0 and pyperclip can probably be removed.
(see master...stephen-huan:manga-ocr:linux-clipboard for an implementation)

If neither wl-paste nor clip are available, the NotImplementedError should propagate through the try catch.

Note that the implementation of grabclipboard() on Linux is currently a bit wonky, see python-pillow/Pillow#7147.
In particular the current implementation will (a) generate a lot of unnecessary temporary files and (b) raise either ChildProcessError from xclip reporting Error: target image/png not available or UnidentifiedImageError from when Pillow tries to parse non-image data, e.g. plaintext when the other operating systems return None on invalid data.

It may be worthwhile to temporarily patch

            try:
                img = ImageGrab.grabclipboard()
            except OSError:
                logger.warning('Error while reading from clipboard')

to

           try:
               img = ImageGrab.grabclipboard()
           except (ChildProcessError, UnidentifiedImageError) as error:
               logger.trace(error)
           except OSError:
               logger.warning('Error while reading from clipboard')

since the default level of loguru is debug, using trace for the parse errors that occur every delay_secs prevents the user from being spammed when running with the default command manga_ocr. However, using the LOGURU_LEVEL environmental variable allows seeing every logging message, for debugging reasons.

LOGURU_LEVEL=TRACE manga_ocr

It may also be worthwhile to temporarily maintain a patched implementation of grabclipboard() from Pillow, at least until the next quarterly release of Pillow comes out (on July 1st). One should also be sure to copy the license.

@kha-white
Copy link
Owner

Thanks for letting me know about the Pillow update. I feel like it's not worth it to keep the separate patched implementation so I might just add the temporary logging patch like you proposed and use the official release.

Just one correction:

On Wayland, the pyperclip library is used to get clipboard contents

This is not exactly true, pyperclip is used only for returning the text, not for grabbing the image

@stephen-huan
Copy link
Author

I see; I misunderstood the purpose of that code.

I also figured out that one can use clipnotify to block until the clipboard is updated, instead of polling.

Replacing

            time.sleep(delay_secs)

with

        has_clipnotify = os.system("which clipnotify > /dev/null") == 0
        ...

            # on linux with the clipnotify package
            if sys.platform == "linux" and has_clipnotify:
                # block until clipboard updates
                with subprocess.Popen(
                    ["clipnotify", "-s", "clipboard"],
                    stdout=subprocess.PIPE,
                ) as process:
                    process.stdout.read()  # type: ignore
            # poll in a platform-agonistic manner
            else:
                time.sleep(delay_secs)

is all that is necessary. The solution is Linux and X11 specific however, and requires clipnotify as a runtime dependency.

@stephen-huan
Copy link
Author

Addressed by #44.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants