AllReady.ScenarioTest Issues

[c0g1t8]

.net framework 4.6.1 is the last default version of the framework installed with Visual Studio Community 2017 15.9.7. AllReady.ScenarioTest is looking for 4.6.2.

Unless there is a very specific reason for using 4.6.2, I would suggest returning to 4.6.1.

[c0g1t8]

@dchristensen - thoughts? I see that you were the last one to touch AllReady.ScenarioTest.fsproj.

[c0g1t8]

Found another issue related to AllReady.ScenarioTest project. It is missing a dependency on the allReady project.

Fixed - PR to come.

[dchristensen]

@c0g1t8 I don't know that its worthwhile to spend a lot of time working on the AllReady.ScenarioTest project. They are not well maintained and there isn't really anyone who is actively working on the project who is familiar with F# or canopy. I know @stevejgordon was working to replace them with tests written in C# at one point, but I don't know how far the effort got.

[jonparker]

I'm not sure if having no tests is a better option than converting them to c# or learning how f# work. Just something to weigh up.

…

On Sun., 17 Feb. 2019, 2:17 pm Derek Christensen, ***@***.***> wrote: @c0g1t8 <https://github.com/c0g1t8> I don't know that its worthwhile to spend a lot of time working on the AllReady.ScenarioTest project. They are not well maintained and there isn't really anyone who is actively working on the project who is familiar with F# or canopy. I know @stevejgordon <https://github.com/stevejgordon> was working to replace them with tests written in C# at one point, but I don't know how far the effort got. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2345 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJSQ6gKro1UjWBWMpCQ-QcSehUedh-Xks5vOMm9gaJpZM4a_MYk> .

[c0g1t8]

@dchristensen I agree with you. My primary goal right now is simply to get the web part of the solution building and passing unit tests. I messaged @stevejgordon on Slack a little over a week ago. He indicated that he and @MisterJames were working on some new plans.

[c0g1t8]

I've changed the description of this issue so it encompasses all the issues I've discovered. I was initially just having issues getting AllReady.ScenarioTest to build consistently.

1. AllReady.ScenarioTest built before AllReady was built

• added a dependency in AllReadyWebOnly.sln

2. Was unable to build since 4.6.2 is not installed by default by VS2017 Community

• changed to use 4.6.1 which is the latest installed by default with VS2017 Community

3. AppVeyor build was using ChromeDriver 2.45 but local build was using ChromeDriver 2.33

• updated to ChromeDriver 2.45 for local testing
• necessary to be work with current version of Chrome Version 72.0.3626.119 (Official Build) (64-bit)

Even with these changes the test results are inconsistent. That is, tests that may have passed before may or may not pass when the test is run again. Two tests do fail consistently.

Test: Admin can create task
Couldn't find any elements with selector 'Create Task', did you mean:
Campaigns
.alert
.alert-success
.caret
.alert-danger
Error:
can't find element Create Task
Url: http://localhost:48408/Admin/Event/Details/8011
Exception details:
canopy.types+CanopyElementNotFoundException: can't find element Create Task
at canopy.wait.waitResults[a](Double timeout, FSharpFunc2 f) in C:\projects\canopy\src\canopy\wait.fs:line 30 at canopy.core.click[a](a item) in C:\projects\canopy\src\canopy\canopy.fs:line 670 at Pages.AdminEventDetails.CreateNewTask[a](a _arg1) in C:\repos\allReady\AllReadyApp\AllReady.IntegrationTest\Pages.fs:line 216 at CreatingOrgsAndCampaigns.All@79-6.Invoke(Unit _arg7) in C:\repos\allReady\AllReadyApp\AllReady.IntegrationTest\CreatingOrgsAndCampaigns.fs:line 81 at canopy.runner.tryTest[a,b](a test, b suite, FSharpFunc2 func) in C:\projects\canopy\src\canopy\runner.fs:line 134

Test: The Org Admin can create task
Couldn't find any elements with selector 'Create Task', did you mean:
Campaigns
.alert
.alert-success
.caret
.alert-danger
Error:
can't find element Create Task
Url: http://localhost:48408/Admin/Event/Details/8012
Exception details:
canopy.types+CanopyElementNotFoundException: can't find element Create Task
at canopy.wait.waitResults[a](Double timeout, FSharpFunc2 f) in C:\projects\canopy\src\canopy\wait.fs:line 30 at canopy.core.click[a](a item) in C:\projects\canopy\src\canopy\canopy.fs:line 670 at Pages.AdminEventDetails.CreateNewTask[a](a _arg1) in C:\repos\allReady\AllReadyApp\AllReady.IntegrationTest\Pages.fs:line 216 at CreatingActivitiesForCampaigns.All@61-12.Invoke(Unit _arg5) in C:\repos\allReady\AllReadyApp\AllReady.IntegrationTest\CreatingActivitiesForCampaigns.fs:line 63 at canopy.runner.tryTest[a,b](a test, b suite, FSharpFunc2 func) in C:\projects\canopy\src\canopy\runner.fs:line 134

Resolving this locally is next on my list. Again the goal is to simply to get an acceptable build in AppVeyor.

[c0g1t8]

Attempted to isolate a fix for this issue alone - PR #2350.

Build test showed

The initial error is a timeout. This did not occur when the test was run locally. Did get the two test case failures as noted above locally.

Could it be that the build server needs more time to warm up?

[c0g1t8]

Temporarily disabled the two broken tests and committed -3bec725. The results still show a timeout at line 646.

Now looking into extending the timeouts in Canopy.

[c0g1t8]

After multiple trial an error, I was able to reproduce the problem found in AppVeyor. The cause appears to be because a critical NPM package was not being restored in the AppVeyor build - jquery.

On AppVeyor it's always a clean build straight from the repository and is done through the command line. It appears that the cause of the problem is due some javacript libraries are not being restored. This causes the browser tests fits. The normal build and test cycles in VS2017 seems to have gotten them restored.

There appears to be some linkage with #2346.

[c0g1t8]

After multiple trial an error, I was able to reproduce the problem found in AppVeyor. The cause appears to be because a critical NPM package was not being restored in the AppVeyor build - jquery.
On AppVeyor it's always a clean build straight from the repository and is done through the command line. It appears that the cause of the problem is due some javacript libraries are not being restored. This causes the browser tests fits. The normal build and test cycles in VS2017 seems to have gotten them restored.
There appears to be some linkage with #2346.

I thought I would provide an update on this issue.

• I can confirm that one of the primary reasons for ScenarioTest not passing is that the client side packages are not being restored fully on the AppVeyor build server. I believe this is all related to using Node 10 by default. Moving forward this will be addressed Gulp 3.9.1 Issues with Node 10 #2346 - see notes there.
• On the two repeatable failure cases, the root cause is due to the event upon which the task test is being executed has both the event start and end date being populated with the current date as of the execution of the test. The event to be created was to start 1 day after creation and end 5 days after creation.
  
  Canopy is not interacting with the datetimepicker control as expected.

[c0g1t8]

Finally figured it out.

Will create a PR with the fix.

[c0g1t8]

I've abandoned a possible fix in PR #2350 and closed. I did learn a few things and included the fix as part of PR #2352.

• Upgraded the Scenario.Test project to Canopy 2 and Core 2.1.
• Fixed a fewer minor bugs in the tests themselves

The tests are now passing.

After spending many hours on this issue, I think moving the testing to a testing framework with C#/Selenium would be a better long term goal.

[c0g1t8]

I've done some further testing. The underlying issue with this implementation appears to be timing. An action on a browser page may not be complete before the next action on the browser page is executed.

In PR #2352, I had removed knockout-bootstrap to address a warning caused by some jQuery version conflict. It was suppose to be a temporary fix.

When I added knockout-bootstrap back, I had sporadic failures locally and more consistent failure on the AppVeyor build server.

I was not able to reproduce the problem interactively. Thus my suspicions.