-contentSize 0 --> Out of memory

[MattKleinsmith]

Goal

To make the output image size the same as as the content image size.

Attempt

th test.lua -content mushka_cat0159.jpg -style dark_matter_bw.png -contentSize 0

Error

Out of memory

THCudaCheck FAIL file=/root/torch/extra/cutorch/lib/THC/generic/THCStorage.cu line=66 error=2 : out of memory
/root/torch/install/bin/luajit: /root/torch/install/share/lua/5.1/nn/Container.lua:67:
In 29 module of nn.Sequential:
/root/torch/install/share/lua/5.1/nn/THNN.lua:110: cuda runtime error (2) : out of memory at /root/torch/extra/cutorch/lib/THC/generic/THCStorage.cu:66

[stack traceback omitted]

Setup

Content image
1280x720
25,840 bytes
Download

Style image
512x512
469,165 bytes
Download

GPU
Titan X Pascal

Other attempts

th test.lua -content mushka_cat0159.jpg -style dark_matter_bw.png

Result: output image size: 912x512 (i.e. different than the content image size, 1280x720)

---

th test.lua -content mushka_cat0159.jpg -style dark_matter_bw.png -contentSize 720

Result: out of memory

[MattKleinsmith]

If anyone tried to reproduce this, please tell me what happened. If you could reproduce it, then we can see what's going on in the code. If you couldn't reproduce it, then I'll know to focus on my system's idiosyncrasies.

Content image

Style image

Copy and paste the following for an easier test, after downloading the images:

th test.lua -content 425c7aec-13fd-11e7-979e-cc8868c6ad96.jpg -style 4260dd94-13fd-11e7-883f-8edc52b992dc.png -contentSize 0

[gsssrao]

Hi @MatthewKleinsmith, Yes, I tried out your images and was able to reproduce the error.

GPU: Titan X Maxwell

On debugging a bit, I found out that Line118 of test.lua causes the error:

return decoder:forward(targetFeature)

Next, I tried resizing the content image and -contentSize 0 works in some cases and in other cases it throws a similar error. On an image with greater number of pixels, you get this error and on an image with lesser pixels, you don't get this error.

On thinking a bit, this makes perfect sense. It is just as it says, your GPU ran out of memory. Your content image is too big for the decoder CNN.

If you read the paper Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization it says that the authors used first few layers of VGG19 as encoder and its inverse for decoder. Hence, the decoder/encoder is a fully convolutional network and thus can adapt itself to input of any arbitrary size. The original paper on FCN has more details regarding this. So, when your image becomes too big like in your case, torch is unable to allocate memory for your network and it throws an error.

[armoreal]

Use -gpu -1 option. I was able to use images up to 1600px on my server with 64GB memory (and it takes about 3min to style image)

[gsssrao]

@armoreal you are using CPU right? I think on a 12GB GPU (Titan X), max of 500,000 pixels can be processed for this decoder.

[armoreal]

Yes, i use GPU for resolution lower than 512x512px and CPU for resolution up to 1600x1600px.

[gsssrao]

@armoreal 👍
Btw, the use of an AdaIN layer made it really fast. Even videos can be made in minutes with this.
I made a small script to add video support using ffmpeg. Try it out using this repo here if you are interested.
I have posted some results in this video.

[armoreal]

Thanks, I'll look at it later.
AdaIN is really great in terms of speed and overall quallity. But need to tune decoder for face processing. Right now small human face becomes ugly So I'd like to train decoder with different settings (content/style layers weights).

[dovanchan]

@gsssrao
Hi,can you show us more result from different content (no just cartoon video)，I dont know why my computer can not install this implement successful ,I am trying again.

[gsssrao]

@dovanchan In this video I have results with various videos not just cartoon. The only additional dependency to run the video stylization is ffmpeg . If you have installed this, you should not experience any other problems if AdaIN style for images was working.