Service interfaces for ML accelerators

[AutonomicPerfectionist]

With the prevalence of hardware machine learning accelerators, we should have services that can take advantage of them. Currently, however, interfaces don't exist to support the generic implementation of such services.

We should discuss the overall requirements of such interfaces. The following are what I believe is the bare minimum:

1. Model loading from either the filesystem or as a byte stream. Alternatively, have the service scan for models in a specified directory and list the available models via a method call
2. List types of supported models, such as tensorflow or caffe.
3. Raw inputs and outputs of the model execution. I believe a float list should suffice for this
4. Inputs and outputs represented as common object types, such as images or strings. This could be selected based on the model set. For example: instead of passing a raw filename, we could pass a composite object that contains the filename as well as the model input type and output type. Input methods would just be overloaded based on input type but output/publishing methods would need a separate publishOutput{type}() method for each type. Only one input overload/publishing method pair would function at a time

[supertick]

Sounds great 👍
breadth scan at a known relative location data/... sounds good
The last one sounds good to .. having the ability to route by type is something downstream apps could utilize.
The rest I don't know much, as I haven't worked much with dnns tensorflow or caffe models