Skip to content
This repository has been archived by the owner on Sep 25, 2023. It is now read-only.

Applying coreml conversation on other style transfer torch models #8

Open
engahmed1190 opened this issue Sep 26, 2017 · 4 comments
Open

Applying coreml conversation on other style transfer torch models #8

engahmed1190 opened this issue Sep 26, 2017 · 4 comments

Comments

@engahmed1190
Copy link

engahmed1190 commented Sep 26, 2017 •
edited

I have tried the coreml conversation on other repository
here is the link of the repository :- link
they provide a pre trained model :- model.t7

By running the perpare_model.lua on this model an error is thrown
adding some changes to make it load the model

require 'InstanceNormalization'
require 'src/utils'
require 'src/descriptor_net'
require 'src/preprocess_criterion'

error :-

index local 'x' (a nil value)
stack traceback:
	perpare_model.lua:12: in function 'replaceModule'
	perpare_model.lua:32: in function 'main'
	perpare_model.lua:46: in main chunk
	[C]: in function 'dofile'
	.../src/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x00406670

running on convert-fast-neural-style.py

getting this error

Traceback (most recent call last):
  File "convert-fast-neural-style.py", line 176, in <module>
    main()
  File "convert-fast-neural-style.py", line 162, in main
    unknown_layer_converter_fn=convert_instance_norm
  File "/usr/local/lib/python2.7/dist-packages/torch2coreml/_torch_converter.py", line 195, in convert
    torch_model.evaluate()
  File "/usr/local/lib/python2.7/dist-packages/torch/legacy/nn/Container.py", line 39, in evaluate
    self.applyToModules(lambda m: m.evaluate())
  File "/usr/local/lib/python2.7/dist-packages/torch/legacy/nn/Container.py", line 26, in applyToModules
    func(module)
  File "/usr/local/lib/python2.7/dist-packages/torch/legacy/nn/Container.py", line 39, in <lambda>
    self.applyToModules(lambda m: m.evaluate())
TypeError: 'NoneType' object is not callable
@engahmed1190
Copy link
Author

this might help , this is the model layers and the main problem is with evaluate() method

nn.Sequential {
  [input -> (0) -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> (7) -> (8) -> (9) -> (10) -> (11) -> (12) -> (13) -> (14) -> (15) -> (16) -> (17) -> (18) -> (19) -> (20) -> (21) -> (22) -> (23) -> output]
  (0): TorchObject(nn.TVLoss, {'_type': 'torch.FloatTensor', 'strength': 0, 'x_diff': 
  ( 0 ,.,.) = 
    0.0000  0.0000  0.0000  ...  -0.0039  0.0078  0.0000
    0.0000  0.0000  0.0000  ...   0.0039  0.0000  0.0000
    0.0000  0.0000  0.0000  ...   0.0078 -0.0157  0.0000
             ...             ⋱             ...          
   -0.0196  0.0157 -0.0196  ...   0.0078  0.0588  0.0392
   -0.0196  0.0196 -0.0235  ...   0.0235  0.0745  0.1451
   -0.0078  0.0000  0.0118  ...   0.0431  0.1882  0.1569
  
  ( 1 ,.,.) = 
    0.0000  0.0000  0.0000  ...  -0.0039  0.0078  0.0000
    0.0000  0.0000  0.0000  ...   0.0039  0.0000  0.0000
    0.0000  0.0000  0.0000  ...   0.0078 -0.0078  0.0000
             ...             ⋱             ...          
   -0.0196  0.0157 -0.0196  ...   0.0078  0.0431  0.0235
   -0.0196  0.0196 -0.0235  ...   0.0196  0.0588  0.1373
   -0.0078  0.0000  0.0118  ...   0.0353  0.1765  0.1451
  
  ( 2 ,.,.) = 
    0.0000  0.0000  0.0000  ...  -0.0039  0.0078  0.0000
    0.0000  0.0000  0.0000  ...   0.0039  0.0000  0.0000
    0.0000  0.0000  0.0000  ...   0.0078 -0.0078  0.0000
             ...             ⋱             ...          
   -0.0196  0.0157 -0.0196  ...   0.0078  0.0471  0.0235
   -0.0196  0.0196 -0.0235  ...   0.0118  0.0510  0.1216
   -0.0078  0.0039  0.0118  ...   0.0157  0.1647  0.1176
  [torch.FloatTensor of size 3x511x511]
  , 'gradInput': [torch.FloatTensor with no dimension]
  , 'y_diff': 
  ( 0 ,.,.) = 
    0.0000  0.0000  0.0000  ...   0.0000  0.0078  0.0000
    0.0000  0.0000  0.0000  ...   0.0078  0.0118 -0.0039
    0.0000  0.0000  0.0000  ...  -0.0118 -0.0078  0.0000
             ...             ⋱             ...          
    0.0039  0.0039  0.0078  ...   0.0157  0.0314  0.0471
   -0.0157 -0.0039 -0.0235  ...   0.0863  0.1059  0.2196
    0.0039  0.0039  0.0039  ...   0.1020  0.1961  0.1451
  
  ( 1 ,.,.) = 
    0.0000  0.0000  0.0000  ...   0.0000  0.0078  0.0000
    0.0000  0.0000  0.0000  ...   0.0000  0.0039 -0.0039
    0.0000  0.0000  0.0000  ...  -0.0118 -0.0078  0.0000
             ...             ⋱             ...          
    0.0039  0.0039  0.0078  ...   0.0196  0.0314  0.0471
   -0.0157 -0.0039 -0.0235  ...   0.0706  0.0863  0.2039
    0.0039  0.0039  0.0039  ...   0.0941  0.1922  0.1451
  
  ( 2 ,.,.) = 
    0.0000  0.0000  0.0000  ...   0.0000  0.0078  0.0000
    0.0000  0.0000  0.0000  ...   0.0000  0.0039 -0.0039
    0.0000  0.0000  0.0000  ...  -0.0196 -0.0078  0.0000
             ...             ⋱             ...          
    0.0039  0.0039  0.0078  ...   0.0039  0.0078  0.0118
   -0.0196 -0.0078 -0.0235  ...   0.0275  0.0314  0.1451
    0.0039  0.0039  0.0039  ...   0.0745  0.1725  0.0980
  [torch.FloatTensor of size 3x511x511]
  , 'train': True, 'output': [torch.FloatTensor with no dimension]
  })
  (1): nn.SpatialReplicationPadding(4, 4, 4, 4)
  (2): nn.SpatialConvolution(3 -> 32, 9x9)
  (3): nn.InstanceNormalization
  (4): nn.ReLU
  (5): nn.SpatialConvolution(32 -> 64, 3x3, 2, 2, 1, 1)
  (6): nn.InstanceNormalization
  (7): nn.ReLU
  (8): nn.SpatialConvolution(64 -> 128, 3x3, 2, 2, 1, 1)
  (9): nn.InstanceNormalization
  (10): nn.ReLU
  (11): nn.Sequential {
    [input -> (0) -> (1) -> output]
    (0): torch.legacy.nn.ConcatTable.ConcatTable {
      input
        |`-> (0): nn.Identity
        |`-> (1): nn.Sequential {
               [input -> (0) -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
               (0): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (1): nn.SpatialConvolution(128 -> 128, 3x3)
               (2): nn.InstanceNormalization
               (3): nn.ReLU
               (4): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (5): nn.SpatialConvolution(128 -> 128, 3x3)
               (6): nn.InstanceNormalization
             }
         +. -> output
    }
    (1): nn.CAddTable
  }
  (12): nn.Sequential {
    [input -> (0) -> (1) -> output]
    (0): torch.legacy.nn.ConcatTable.ConcatTable {
      input
        |`-> (0): nn.Identity
        |`-> (1): nn.Sequential {
               [input -> (0) -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
               (0): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (1): nn.SpatialConvolution(128 -> 128, 3x3)
               (2): nn.InstanceNormalization
               (3): nn.ReLU
               (4): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (5): nn.SpatialConvolution(128 -> 128, 3x3)
               (6): nn.InstanceNormalization
             }
         +. -> output
    }
    (1): nn.CAddTable
  }
  (13): nn.Sequential {
    [input -> (0) -> (1) -> output]
    (0): torch.legacy.nn.ConcatTable.ConcatTable {
      input
        |`-> (0): nn.Identity
        |`-> (1): nn.Sequential {
               [input -> (0) -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
               (0): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (1): nn.SpatialConvolution(128 -> 128, 3x3)
               (2): nn.InstanceNormalization
               (3): nn.ReLU
               (4): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (5): nn.SpatialConvolution(128 -> 128, 3x3)
               (6): nn.InstanceNormalization
             }
         +. -> output
    }
    (1): nn.CAddTable
  }
  (14): nn.Sequential {
    [input -> (0) -> (1) -> output]
    (0): torch.legacy.nn.ConcatTable.ConcatTable {
      input
        |`-> (0): nn.Identity
        |`-> (1): nn.Sequential {
               [input -> (0) -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
               (0): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (1): nn.SpatialConvolution(128 -> 128, 3x3)
               (2): nn.InstanceNormalization
               (3): nn.ReLU
               (4): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (5): nn.SpatialConvolution(128 -> 128, 3x3)
               (6): nn.InstanceNormalization
             }
         +. -> output
    }
    (1): nn.CAddTable
  }
  (15): nn.Sequential {
    [input -> (0) -> (1) -> output]
    (0): torch.legacy.nn.ConcatTable.ConcatTable {
      input
        |`-> (0): nn.Identity
        |`-> (1): nn.Sequential {
               [input -> (0) -> (1) -> (2) -> (3) -> (4) -> (5) -> (6) -> output]
               (0): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (1): nn.SpatialConvolution(128 -> 128, 3x3)
               (2): nn.InstanceNormalization
               (3): nn.ReLU
               (4): nn.SpatialReplicationPadding(1, 1, 1, 1)
               (5): nn.SpatialConvolution(128 -> 128, 3x3)
               (6): nn.InstanceNormalization
             }
         +. -> output
    }
    (1): nn.CAddTable
  }
  (16): nn.SpatialFullConvolution(128 -> 64, 3x3, 2, 2, 1, 1, 1, 1)
  (17): nn.InstanceNormalization
  (18): nn.ReLU
  (19): nn.SpatialFullConvolution(64 -> 32, 3x3, 2, 2, 1, 1, 1, 1)
  (20): nn.InstanceNormalization
  (21): nn.ReLU
  (22): nn.SpatialReplicationPadding(1, 1, 1, 1)
  (23): nn.SpatialConvolution(32 -> 3, 3x3)
}

the model is loaded perfectly fine but when i am trying to evaluate model , i get this error

 Traceback (most recent call last):
  File "convert-fast-neural-style.py", line 176, in <module>
    main()
  File "convert-fast-neural-style.py", line 162, in main
    unknown_layer_converter_fn=convert_instance_norm
  File "/usr/local/lib/python2.7/dist-packages/torch2coreml/_torch_converter.py", line 194, in convert
    print (model.evaluate())
  File "/usr/local/lib/python2.7/dist-packages/torch/legacy/nn/Container.py", line 39, in evaluate
    self.applyToModules(lambda m: m.evaluate())
  File "/usr/local/lib/python2.7/dist-packages/torch/legacy/nn/Container.py", line 26, in applyToModules
    func(module)
  File "/usr/local/lib/python2.7/dist-packages/torch/legacy/nn/Container.py", line 39, in <lambda>
    self.applyToModules(lambda m: m.evaluate())
TypeError: 'NoneType' object is not callable

what is the main reason behind the evaluation to bring this NoneType error.
is their any other may that is equivalent to m.evaluate()

for reference also this is the Model i am trying to evaluate

@opedge
Copy link
Member

opedge commented Oct 3, 2017

To use this repo and models with custom layers from it you need to implement corresponding layers using pytorch (legacy.nn.Module subclasses) and replace them in parsed model.

@engahmed1190
Copy link
Author

@opedge do you have any example for that

@opedge
Copy link
Member

opedge commented Nov 2, 2017

There is an example of implementing custom InstanceNormalization layer using pytorch.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@opedge @engahmed1190