Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add end-to-end testing for Bot Framework with NLU.DevOps #190

Open
rozele opened this issue Sep 28, 2019 · 0 comments
Open

Add end-to-end testing for Bot Framework with NLU.DevOps #190

rozele opened this issue Sep 28, 2019 · 0 comments

Comments

@rozele
Copy link
Contributor

rozele commented Sep 28, 2019

With the upcoming changes to NLU.DevOps.ModelPerformace, we'll be able to support arbitrary testing of JSON results from NLU. We should add a providers, NLU.DevOps.BotService, that supports both direct line via BotFramework and direct line speech via the Cognitive Service Speech SDK. The scenario would be as follows, consider the following tests:

[
  {
      "text": "add two avocados to my cart",
      "conversationID": 1,
      "userID": 1,
      "attachments": [
        [
           {
              "name": "avocado",
              "quantity": 2
           }
        ]
      ]
  },
  {
      "text": "remove one avocado and add two dozen eggs",
      "conversationID": 1,
      "userID": 1,
      "attachments": [
        [
           {
              "name": "avocado",
              "quantity": 1
           },
           {
              "name": "12 eggs",
              "quantity": "2"
           }
        ]
      ]
  }
]

In this case, we will set up the direct line channel to the bot and send two user queries "add two avocados to my cart" and "remove one avocado and add two dozen eggs".

The output may not contain any LUIS or other NLU provider specific info, but instead focuses on the end state contained in the activity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant