review of practices #30

sergio-sastre · 2021-12-07T20:57:34Z

For now this includes a review of:

State clearing
Test Runners
Emulators vs devices
Emulator setup
PageObject pattern
...

more to come soon :)

sergio-sastre · 2021-12-07T20:58:32Z

docs/content/practices/state_clearing.md

@@ -58,10 +63,10 @@ them [here](https://github.com/AdevintaSpain/Barista/tree/master/library/src/mai

 !!! warning

-    This solution won't in 100% of cases:
+    This solution won't work in 100% of cases:

    1. You may have runtime cache, which can also affect your tests


I did not get this point with the runtime cache. What is stored in there and how it affects? Maybe an example would help

Imagine if we store our data in databases and in runtime cache, like hashmap
Clearing of internal storage won't clear it. We have to clear it somehow, the only way -- use some production code, which is a bottleneck

sergio-sastre · 2021-12-07T21:00:22Z

docs/content/practices/state_clearing.md

-uses [special services.](https://github.com/android/android-test/tree/master/services)
-It's just a shell client and should be installed to the device.
+Orchestrator also needs to execute adb commands. For that it uses [special services.](https://github.com/android/android-test/tree/master/services) under the hood.
+It's just a shell client and should be installed on the device.


After reading this, I was doubting whether I have to install it on my own, or the Orchestrator does it for me (this one right?). This should be clarified

sergio-sastre · 2021-12-07T21:02:20Z

docs/content/practices/state_clearing.md

@@ -118,15 +119,15 @@ It's just a shell client and should be installed to the device.
    Despite the fact that it does the job, this solution looks overcomplicated:

    1. We need to install +2 different apk to each emulator
-    2. We delegate this job to the device instead of host machine. 
+    2. We delegate this job to the device instead of the host machine. 
    <br/>Devices are less reliable than host pc


Less reliably in the sense of "they do not lose connection to adb server, issues with battery, internet, system dialogs," or sth like that?

sergio-sastre · 2021-12-07T21:22:00Z

docs/content/practices/state_clearing.md

 <br/>
-➖ Slow execution _(can take 10+ seconds and depends on apk size)_ <br/>


10+ was for ALL the tests together due to adb pm clear right? This I did not understand very well

sergio-sastre · 2021-12-07T21:26:24Z

docs/content/practices/state_clearing.md

 ➕ Fast execution in the same process<br/>
 <br/>
 ➖ Don't give you any guarantee that your app will be cleared properly<br/>
 ➖ Application or Test process killing will break tests execution <br/>
-➖ Can be a bottleneck<br/>
+➖ Can be a bottleneck<br/> 


this needs some short hint of why

sergio-sastre · 2021-12-08T23:13:30Z

docs/content/practices/test_runners_review.md



 * Impossibility to scale
  <br>
-  _As soon as you started your tests, it's impossible to connect more devices to tests run on fly_
+  _As soon as you started your tests, it's impossible to let more devices join the test execution on the fly_


I think explaining why it can help to add test on the fly and what is the case would help.

I was thinking about running test on a device cloud? There could be more available after a while? @NoNews what is the case exactly?

sergio-sastre · 2021-12-08T23:14:25Z

docs/content/practices/test_runners_review.md

 ➕ adb client `ddmlib` replacement: [Adam](https://github.com/Malinskiy/adam) <br>
 ➕ Cross-platform (iOS support) <br>
 ➕ Fragmented test execution (similar to AOSP's sharding): split large testing suites into multiple CI builds <br>
 ➕ Parallel execution of parameterised tests <br>
-➕ Interactions with adb/emulator from within a test (e.g. fake fingerprint or GPS) <br>
+➕ Interactions with adb/emulator from within a test (e.g. fake fingerprint or GPS) <br> 


Looks like Kaspresso does already half of these things :P

sergio-sastre · 2021-12-08T23:16:49Z

docs/content/practices/test_runners_review.md

 ➕ Pull files from the device after test run,
 e.g. [allure-kotlin](https://github.com/allure-framework/allure-kotlin) <br>
-➕ Basic [Allure](https://github.com/allure-framework) support out of the box <br>
+➕ Basic [Allure](https://github.com/allure-framework) support out of the box <br> 


It is not clear what Allure does for someone who has not used it:
video (what is mentioned above), reports...
I'd rather mention the most relevant, that have not been mentioned above, or exclude video form above and put it here (can Marathon record videos without Allure?) @NoNews

sergio-sastre · 2021-12-10T22:01:03Z

docs/content/practices/emulator_setup.md


-After that, you can run your emulator by `avd manager`, which is a part of android sdk manager. After your device
-creation, you need change default generated ini file to custom one. You may see an example below:
+After that, you can run your emulator via `avd manager`, which is part of the android sdk manager. After creating the emulator, you need to switch the default generated `ini` file to the custom one we defined previously. 


I rephrased this. I believe this is what is intended after reading the code in the script

sergio-sastre · 2021-12-10T22:04:20Z

docs/content/practices/emulator_setup.md

-Talking about [Avito emulator](https://github.com/google/android-emulator-container-scripts), it also patches your
-emulator with adb commands to prevent tests flakiness and to speed them up
+Talking about the [Avito emulator](https://github.com/google/android-emulator-container-scripts), it also patches your
+emulator with adb commands to prevent tests flakiness and to speed them up.


would be good to quickly mention how it does it.
From the description here, I take for granted that Avito emulator is the best out of the 3 here? If yes, should we stress the point in here?

What are the differences among them?

As a learner, I want to use the best for my needs

sergio-sastre · 2021-12-10T22:08:26Z

docs/content/practices/emulator_setup.md

-After that, you can run your emulator by `avd manager`, which is a part of android sdk manager. After your device
-creation, you need change default generated ini file to custom one. You may see an example below:
+After that, you can run your emulator via `avd manager`, which is part of the android sdk manager. After creating the emulator, you need to switch the default generated `ini` file to the custom one we defined previously. 
+You can achieve that with a script like this one:

 ```bash
 function define_android_sdk_environment_if_needed() {


I strongly recommend to have a "ready" bash file for this that we upload in this article (maybe extra subsection here to make it more prominent) and indicate how to execute -> ./bashFile.sh -args
The command can actually differ on windows though

sergio-sastre · 2021-12-10T22:12:10Z

docs/content/practices/emulator_setup.md

@@ -109,22 +109,28 @@ define_path_environment_if_needed
 create_and_patch_emulator
 ```

-Pay your attention that you also need to wait until your emulator is fully booted.
+Keep in mind that the emulator must fully boot before running any test. Otherwise the tests will fail because there is still no device ready


Actually, for snapshot testing I was using swarmer from Juno, which is deprecated but works pretty fine for starting several emulators and wait for all of them to be ready before running the tests.
Another new thing is Gradle Virtual Managed Devices...

Maybe worth mentioning later, after releasing the MVP?

review of state clearing

6e0b591

sergio-sastre commented Dec 7, 2021

View reviewed changes

Sergio Sastre Flórez added 2 commits December 7, 2021 22:20

some more corrections

f94a984

rewording

166d8d7

sergio-sastre commented Dec 7, 2021

View reviewed changes

sergio-sastre requested review from NoNews and v1sar December 7, 2021 21:27

Sergio Sastre Flórez added 2 commits December 7, 2021 22:36

better wording for suggestion

1fb3bf5

Review of "Test Runners review"

88d0741

sergio-sastre commented Dec 8, 2021

View reviewed changes

Sergio Sastre Flórez added 5 commits December 10, 2021 22:18

Emulator vs real device review

497d515

Merge branch 'main' into practices_review

2f24e61

emulator setup review

1815a54

second review emulator setup

3e9b9aa

remove unnecessary "a"

eab7416

sergio-sastre commented Dec 10, 2021

View reviewed changes

Sergio Sastre Flórez and others added 3 commits December 27, 2021 22:19

Merge branch 'main' into practices_review

c7f7570

Review page_object

8124fa0

Update flakiness

3398f94

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

review of practices #30

review of practices #30

sergio-sastre commented Dec 7, 2021 •

edited

sergio-sastre Dec 7, 2021

NoNews Dec 10, 2021

sergio-sastre Dec 7, 2021

sergio-sastre Dec 7, 2021

sergio-sastre Dec 7, 2021

sergio-sastre Dec 7, 2021

sergio-sastre Dec 8, 2021

sergio-sastre Dec 8, 2021

sergio-sastre Dec 8, 2021

sergio-sastre Dec 10, 2021

sergio-sastre Dec 10, 2021

sergio-sastre Dec 10, 2021

sergio-sastre Dec 10, 2021

		<br/>
		➖ Slow execution _(can take 10+ seconds and depends on apk size)_ <br/>

review of practices #30

Are you sure you want to change the base?

review of practices #30

Conversation

sergio-sastre commented Dec 7, 2021 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sergio-sastre commented Dec 7, 2021 •

edited