converts bad photo file extensions (supercedes 404/412)

[ssempervirens]

Resolves #369 and supercedes PRs #404 and #412. I made a real mess of this issue, and I figured a new branch is the cleanest fix since that was a while ago.

Status

WIP

Description of Changes

Fixes # 369.

Changes proposed in this pull request:

• Convert psd and tiff files to jpeg
• Test that tiff and psd files can be uploaded
• Note: avoided using os.open, I don't believe this is necessary and avoids dealing with safety warnings

Notes for Deployment

Screenshots (if appropriate)

Tests and linting

• I have rebased my changes on current develop

• pytests pass in the development environment on my local machine
  Note: I get 'TypeError: attrib() got an unexpected keyword argument 'convert' when I make test. This seems to be an issue with the package attrs rather than anything I changed, but please let me know if this is not correct.

• flake8 checks pass

[redshiftzero]

does this convert e.g. tiff and psd files to jpegs or just rename them? we can use imagemagick for the conversion, i think it should work for both tiff and psds (though we should test that, each are good candidates for unit tests)

[ssempervirens]

Right, so I didn't realize imagemagick had a good python binding? Should I just use PIL?

(So there is no way to avoid the os.open/decompression bomb issue?)

[redshiftzero]

oh right right, Pillow (Python 3 version of PIL) should do the trick! there's also a decent imagemagick API called wand if you find that Pillow is lacking support for a filetype that you want to support.

But yeah I'm not seeing a way to convert these images without using something like Pillow (short of using a third-party service).

[brianmwaters]

According to the Pillow docs for this method (https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.save),

"Saves this image under the given filename. If no format is specified, the format to use is determined from the filename extension, if possible."

So it would appear this is actually converting the files to the new format, not just changing the name.

However, maybe we should have a test for this if we don't already?

[McEileen]

Hey @ssempervirens, I'm following up about the error message you got, TypeError: attrib() got an unexpected keyword argument 'convert' , after you ran make test. I didn't hit this error when I ran the tests on my own machine, but I did see it in the Travis build for another PR. I read up on stackoverflow and realized that the error was caused by OO using an older version of pytest, 4.0.2. (I had a more recent version of pytest installed on my local machine, and I think that's why I didn't get this error locally.)

I updated pytest to 5.2.2 in dev-requirements.txt and it fixed the error for me. Let me know if this fixes the error for you, too.

Also, since this PR is superseding #412, could you please close #412? (I checked and saw that you already closed #404.) Thanks!

[brianmwaters]

For what it's worth, I'm seeing the same thing as @McEileen. Namely, I was getting TypeError: attrib() got an unexpected keyword argument 'convert', when I ran make test on @ssempervirens' 369-convert-photo-extension branch. However, when I rebased this branch onto the current develop, which contains @McEileen's patch bumping the pytest version, it worked.

I think if we ran this through Travis CI again, the tests would probably pass.