What is the best way to code this Service ?

[jiyer2016]

Here are two possible implementations of a "score_items" API

@bentoml.env(auto_pip_dependencies=True)
@bentoml.artifacts([SklearnModelArtifact('model')])
class Service(bentoml.BentoService):

    # Implementation 1
    @bentoml.api(input=JsonInput())
    def score_items(self, rqst):
        print(type(rqst), rqst)
        w1 = float(rqst['weights']['w1'])
        w2 = float(rqst['weights']['w2'])

        resp = rqst.copy()

        for item in resp['items']:
            f1, f2 = float(item['f1']), float(item['f2'])
            item['score'] = w1*f1 + w2*f2

        resp['items'] = sorted(resp['items'], key=lambda x: x['score'], reverse=True)

        resp['version'] = self.version

        return resp

    # Implementation 2
    @bentoml.api(input=JsonInput())
    def score_items2(self, rqsts):
        print(type(rqsts), rqsts)
        resps = []
        for rqst in rqsts:
            w1 = float(rqst['weights']['w1'])
            w2 = float(rqst['weights']['w2'])

            resp = rqst.copy()

            for item in resp['items']:
                f1, f2 = float(item['f1']), float(item['f2'])
                item['score'] = w1*f1 + w2*f2

            resp['items'] = sorted(resp['items'], key=lambda x: x['score'], reverse=True)

            resp['version'] = self.version
            resps.append(resp)

        return resps

Implementation 1 works well with Integration Test Cases (see test case below) but fails when served via flask because of the behavior described in #1067 for which Implementation 2 above is required.

    def test_rank_items(self):
        service = ...

        rqst = {
            "weights": {
                "w1" : 0.6,
                "w2" : 0.4
            },
            "items": [
                {
                    "id": 1,
                    "f1": 0.5,
                    "f2": 0.5
                },
                {
                    "id": 2,
                    "f1": 0.6,
                    "f2": 0.4
                }
            ]
        }
        resp = service.score_items(rqst)

        self.assertIsInstance(resp, dict)
        self.assertDictEqual(rqst['weights'], resp['weights'])
        self.assertEqual(resp['version'], service.version)

        self.assertEqual(len(rqst['items']), len(resp['items']))
        self.assertEqual(resp['items'][0]['id'], rqst['items'][1]['id'])
        self.assertEqual(resp['items'][0]['score'], 0.52)
        self.assertEqual(resp['items'][1]['id'], rqst['items'][0]['id'])
        self.assertEqual(resp['items'][1]['score'], 0.5)

How do I write code that is compatible with both test cases and runtime simultaneously ?

[jiyer2016]

The reason these test cases are important is because I want to validate the execution pipeline via tests before even proceeding to save into the yatai repository.

[jiyer2016]

Based on other discussions threads on similar topics - I shall use "Implementation 2" - i.e. score_items2() and then adjust the integration test cases such that:

• request object (rqst) is packed into a List (i.e. emulate what the InputAdapter does)
• expect the response object (resp) wrapped in a list for test cases alone - with knowledge that the real production Client won't see the outer list

[resp] = service.score_items([rqst])