[Docs] How to train a model with different inputs from different model feature and videos

[zeynepgokce]

The doc issue

Hello,

I need a model training code which takes two different inputs, first one comes from pre-trained another model feature and second input from video inputs. I need to train the my main model using these two different input. I could not handle this problem in mmaction2.
I dont know how to start it? Is there any sample code for this purpose? Please help me.

Thank you.

Suggest a potential alternative/fix

Is there any sample code for this purpose? If it does not exist, could you please provide a documentation about the customization of a model with various inputs or traning codes?

[valentin-fngr]

Not an active contributor but here is how I would deal with it for now (as you can see, it can take up to weeks before getting an answer from the devs ...).

I would suggest to make a new class using

mmaction2/mmaction/models/recognizers/recognizer2d.py

Line 11 in 4d6c934

class Recognizer2D(BaseRecognizer):

as template.

Modify this class such that it take your input feature as well.
Additionally, look into models that also deal with optical flow as they use the optical flow as additional features.