added testcases for LazyVideos and the corresponding .ccz as well #2568

kbo001 · 2022-07-13T12:53:43Z

Summary

Automated the testcases for VideosWithValidReferences and VideosWithNoReference in the Lazy Videos Tab of the Mobile Test Plan:
https://docs.google.com/spreadsheets/d/1X9YAf_g1Cq9vZZdCkcBLnRSFrhZuk_vEyCSM3sD5QvM/edit#gid=1556040007
Also added it corresponding .ccz file

Feature Flag

Product Description

Safety Assurance

If the PR is high risk, "High Risk" label is set
I have confidence that this PR will not introduce a regression for the reasons below
Do we need to enhance manual QA test coverage ? If yes, "QA Note" label is set correctly

Automated test coverage

This testcase covers the below scenarios:

opens the form Video with valid reference , checks if the video is already downloaded while installing the app (if not already present then downloads it), plays the video, moves to next page and verifies that the video player is present and then submits the form.
opens the form Video with no reference , checks the video is not downloaded while installing the app, tries to download the video, gets error "Media not found in the application", moves to next page and verifies that the video player is missing and then submits the form.

Safety story

Executed in local AVD, ran successfully.

dimagimon · 2022-07-29T10:53:12Z

	1 Warning
⚠️	This PR does not contain any JIRA issue keys in the PR title or commit messages (e.g. KEY-123)

Generated by 🚫 Danger

shubham1g5

@kbo001 I was reading the steps on manual test plan for this and it seems to state that we need to repeat the steps for all questions in the form (steps 3.6 and 5.3 in test plan). In comparison this test only seem to test the lazy download for the first question in the form.

shubham1g5 · 2022-07-29T10:54:50Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

+            InstrumentationUtility.stubIntentWithAction(Intent.ACTION_VIEW)
+            onView(withId(R.id.video_button)).perform(ViewActions.click())
+            InstrumentationUtility.nextPage()


Should this code(L75-77) be outside else as it looks valid even when we downloaded the video as part of the test ?

yes. Removing the else statement

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

shubham1g5 · 2022-07-30T02:45:10Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

+                R.drawable.update_download_icon
+            )
+        ).isPresent()
+        InstrumentationUtility.stubIntentWithAction(Intent.ACTION_SEND)


I don't think we need to do this as Intent.ACTION_SEND never gets fired during this workflow.

@kbo001 FYI

damagatchi · 2022-08-19T11:36:10Z

Can one of the admins verify this patch?

…he form

shubham1g5 · 2022-09-06T11:18:07Z

@damagatchi retest this please

shubham1g5 · 2022-09-06T11:27:52Z

@damagatchi retest this please

shubham1g5 · 2022-09-06T11:51:31Z

@damagatchi retest this please

shubham1g5 · 2022-09-06T12:06:49Z

@damagatchi retest this please

shubham1g5 · 2022-09-15T06:12:22Z

@kbo001 Are the changes here complete now ?
@damagatchi retest this please

kbo001 · 2022-09-15T06:23:38Z

@shubham1g5 I did not get the time yesterday, working on it today.

….kt, updated test steps in testVideosWithValidReference()

kbo001 · 2022-09-15T10:23:39Z

@shubham1g5 I changed the ccz file and updated the test steps according to the testplan.

shubham1g5 · 2022-09-15T11:03:04Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

+        InstrumentationUtility.login("test1", "123")
+        InstrumentationUtility.openForm(1, 0)
+        InstrumentationUtility.waitForView(withId(R.id.video_button))
+        if ((onView(withTagValue(CoreMatchers.`is`(R.drawable.update_download_icon))).isPresent())


this should always be present now right ?

we should not put this in if condition then

But if we keep it, then even if the .ccz is updated then also this will work. Thats what I thought of. If you think it should be removed, I can do that.

We should remove it as we don't want to paas this test unless it goes through the download workflow.

…ion from testVideosWithValidReference()

shubham1g5 · 2022-09-15T12:32:34Z

lazy_videos_test.ccz can be removed now ?

shubham1g5 · 2022-09-15T12:34:28Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

+
+    @Before
+    fun setup() {
+        if (CommCareApplication.instance().currentApp == null) {


curious why are not we using the norma installApp()method as other tests ?

if installApp() is used then the app is not uninstalled everytime and the videos remains in the app, hence failing the tests.

got it, it would be good to add a comment above this line stating this as It's not very obvious.

shubham1g5 · 2022-09-15T12:35:41Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

+        InstrumentationUtility.waitForView(withSubstring("Download in Progress"))
+        InstrumentationUtility.waitForView(withSubstring("Download complete"))
+        InstrumentationUtility.waitForView(withTagValue(CoreMatchers.`is`(android.R.drawable.ic_media_play)))
+//        }


shubham1g5

Approved given the test pass on BS

shubham1g5 · 2022-09-16T05:20:49Z

@kbo001 Seems like DateWidgetTests are failing again repeatedly blocking pass on this test. Would you be able to take a look.
@damagatchi retest this please

…deo code for the remaining questions

kbo001 · 2022-09-16T07:16:53Z

@shubham1g5 the dateWidgetTests seems to be passing but the lazy videos is failing as in the third question the video was still not available. I have added some wait time during the video download and also added the video download code for the remaining questions. Can you execute it once on BS as its already passing at my end.

kbo001 · 2022-09-16T09:28:44Z

I see that the test is passing now

shubham1g5 · 2022-09-16T08:46:20Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

@@ -126,6 +138,15 @@ class LazyVideosTests : BaseTest() {
                R.drawable.icon_info_outline_lightcool
            )
        ).perform(click())
+        if(onView(withTagValue(CoreMatchers.`is`(R.drawable.update_download_icon))).isPresent()){


is the download icon not always present ?

The test was failing earlier because of this, instead of the play icon there was the download icon, maybe the video for this question was not downloaded properly. This time when the test is executed the download icon was not present as earlier. This might happen again so I kept this condition for both the last questions.

If we put these in a if we are bypassing the normal test steps to download media and a pass here won't guarantee that the underlying download workflow works properly risking a bug into the mobile release.

shubham1g5 · 2022-09-16T09:41:46Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

@@ -93,14 +93,17 @@ class LazyVideosTests : BaseTest() {
        onView(withTagValue(CoreMatchers.`is`(R.drawable.update_download_icon))).check(matches(isDisplayed()))
        InstrumentationUtility.stubIntentWithAction(Intent.ACTION_SEND)
        onView(withTagValue(CoreMatchers.`is`(R.drawable.update_download_icon))).perform(click())
-        onView(withId(R.id.video_button)).perform(click())
+        InstrumentationUtility.sleep(1)
+        InstrumentationUtility.waitForView(withSubstring("Download started"))


this should be immediate after click, wait before and after this should not be required.

This video button will be available only when the download is completed. L104 is doing that task here. Once the download is complete we click on the video button and verify the intent. We don't need it here as we will be clicking on R.drawable.update_download_icon and start the download.

I am talking about InstrumentationUtility.sleep() statements, I don't think they should be required before and after this statement since we are already using waitForView here.

shubham1g5 · 2022-09-16T09:41:59Z

app/instrumentation-tests/src/org/commcare/androidTests/LazyVideosTests.kt

@@ -114,6 +117,15 @@ class LazyVideosTests : BaseTest() {
                R.drawable.icon_info_outline_lightcool
            )
        ).perform(click())
+        if(onView(withTagValue(CoreMatchers.`is`(R.drawable.update_download_icon))).isPresent()){


why do we have a if here ?

Same reason as the one before. Sometimes the videos for the other questions may not get downloaded properly.

kbo001 · 2022-09-21T11:44:04Z

@shubham1g5 I have commented the sleep and if statements, can you verify if the test is passing in BS?

added testcases for LazyVideos and the corresponding .ccz as well

1cfc4f3

kbo001 added the product/invisible label Jul 13, 2022

kbo001 requested a review from shubham1g5 July 13, 2022 12:53

shubham1g5 and others added 6 commits July 13, 2022 19:59

match drawables

1d3d1fd

fix drawable matcher

12ca21f

Adds drawable matching for image button

fa7911c

cleaned up unused codes and imports

9cd7ef2

Merge branch 'master' into kb/lazy_videos

b10e271

formating

415c1be

shubham1g5 requested changes Jul 29, 2022

View reviewed changes

spacing

c6abf00

shubham1g5 reviewed Jul 30, 2022

View reviewed changes

fixes drawable matcher to be able to perform clicks

9eb3beb

added steps to validate video player for all remaining questions in t…

219c4f9

…he form

Merge branch 'master' into kb/lazy_videos

9da6232

match drawables based on tag

9cc5425

updated lazy_videos_tests.ccz, added new functions in LazyVideosTests…

7d1d220

….kt, updated test steps in testVideosWithValidReference()

kbo001 requested a review from shubham1g5 September 15, 2022 10:22

shubham1g5 reviewed Sep 15, 2022

View reviewed changes

removed Intent.ACTION_SEND steps from testVideosWithNoReference()

a765570

kbo001 requested a review from shubham1g5 September 15, 2022 11:41

added lazy_videos_no_viedo.ccz, updated setup() and removed if condit…

8667c49

…ion from testVideosWithValidReference()

shubham1g5 requested changes Sep 15, 2022

View reviewed changes

kbo001 added 2 commits September 15, 2022 18:20

deleted lazy_videos_tests.ccz

452ad55

removed commented }

aa4d59d

kbo001 requested a review from shubham1g5 September 15, 2022 13:07

shubham1g5 previously approved these changes Sep 15, 2022

View reviewed changes

added some wait time for downloading the videos and added download vi…

236dd72

…deo code for the remaining questions

kbo001 dismissed shubham1g5’s stale review via 236dd72 September 16, 2022 07:15

kbo001 requested a review from shubham1g5 September 16, 2022 07:17

shubham1g5 reviewed Sep 16, 2022

View reviewed changes

kbo001 requested a review from shubham1g5 September 16, 2022 10:04

commented the sleep and if statements

df0a4c6

added testcases for LazyVideos and the corresponding .ccz as well #2568

Are you sure you want to change the base?

added testcases for LazyVideos and the corresponding .ccz as well #2568

Conversation

kbo001 commented Jul 13, 2022

Summary

Feature Flag

Product Description

Safety Assurance

Automated test coverage

Safety story

dimagimon commented Jul 29, 2022

shubham1g5 left a comment

Choose a reason for hiding this comment

shubham1g5 Jul 29, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damagatchi commented Aug 19, 2022

shubham1g5 commented Sep 6, 2022

shubham1g5 commented Sep 6, 2022

shubham1g5 commented Sep 6, 2022

shubham1g5 commented Sep 6, 2022

shubham1g5 commented Sep 15, 2022

kbo001 commented Sep 15, 2022

kbo001 commented Sep 15, 2022

Choose a reason for hiding this comment

kbo001 Sep 15, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shubham1g5 commented Sep 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shubham1g5 left a comment • edited

Choose a reason for hiding this comment

shubham1g5 commented Sep 16, 2022

kbo001 commented Sep 16, 2022

kbo001 commented Sep 16, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbo001 commented Sep 21, 2022

shubham1g5 Jul 29, 2022 •

edited

kbo001 Sep 15, 2022 •

edited

shubham1g5 left a comment •

edited