Take screenshots during system tests, upload to Google Cloud storage, and append to Dangerbot output #5320

jywarren · 2019-03-31T17:09:53Z

Re #5316 screenshots of tests

Also we might try: bin/rails test:system test to get the coverage to not run twice (we went down from 91-44% but that's because we're running tests twice and 44 is just for system tests i guess?)

https://api.rubyonrails.org/classes/ActionDispatch/SystemTestCase.html

Attempting to generate inline screenshots with https://iterm2.com/documentation-images.html

But fallback could be to generate them as files, then use Dangerfile to leave them as comments, but collasped in a <details> el:

<details>
    <summary>Screenshots 📸 (click to expand)</summary>
    <h3>Front page:</h3>
    <p><img src="dataurl" /></p>
</details>

plotsbot · 2019-03-31T17:22:59Z

	1 Error
🚫	There was an error with Danger bot’s Junit parsing: No JUnit file was found at output.xml

	1 Warning
⚠️	It looks like you merged from master in this pull request. Please rebase to get rid of the merge commits – you may want to rewind the master branch and rebase instead of merging in from master, which can cause problems when accepting new code!

	2 Messages
📖	@jywarren Thank you for your pull request! I’m here to help with some tips and recommendations. Please take a look at the list provided and help us review and accept your contribution! And don’t be discouraged if you see errors – we’re here to help.
📖	#

Screenshots 📸 (click to expand)

5320-test_front_page_with_navbar_search_autocomplete.png

5320-test_viewing_the_dashboard.png

Generated by 🚫 Danger

jywarren · 2019-03-31T18:28:50Z

Great -- [Screenshot]: tmp/screenshots/test_front_page_with_navbar_search_autocomplete.png

Now we just have to figure out how to use Danger to paste it into a comment, or display it inline.

jywarren · 2019-03-31T18:49:58Z

plots2/Dangerfile

Lines 60 to 68 in 3d2b068

    
           images = [] 
        
           Dir.foreach('/tmp/screenshots') do |item| 
        
             next if item == '.' or item == '..' 
        
             dataurl = "data:image/png;base64," + Base64.strict_encode64(File.read(item)) 
        
             images << "<h3>#{item}</h3><p><img src='#{dataurl}' /></p>" 
        
           end 
        
           screenshots = "<details><summary>Screenshots 📸 (click to expand)</summary>" + images.join + "</details>" 
        
           markdown(screenshots)

jywarren · 2019-03-31T19:47:42Z

Huh, so i see:

<details><summary>Screenshots <g-emoji class="g-emoji" alias="camera_flash" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f4f8.png">📸</g-emoji> (click to expand)</summary><h3>test_front_page_with_navbar_search_autocomplete.png</h3><p><a target="_blank" rel="noopener noreferrer" href=""><img style="max-width:100%;"></a></p></details>

However, later in the source code of this page, I see this dataurl which is correct!

jywarren · 2019-03-31T19:49:08Z

Aha -- ok, so the dataurl gets stripped out of comments. Makes sense i guess. The image is:

And by editing the Dangerbot comment you can see it in there -- it's just sanitized out!

jywarren · 2019-03-31T20:19:11Z

I messaged GitHub help desk to ask about limitations of dataurls. Unfortunately I can't think of another way around this except to abandon dataurls and store in Google cloud maybe.

jywarren · 2019-03-31T20:25:26Z

https://cloud.google.com/storage/docs/uploading-objects#storage-upload-object-ruby

jywarren · 2019-03-31T20:26:48Z

# project_id        = "Your Google Cloud project ID"
# bucket_name       = "Your Google Cloud Storage bucket name"
# local_file_path   = "Path to local file to upload"
# storage_file_path = "Path to store the file in Google Cloud Storage"

require "google/cloud/storage"

storage = Google::Cloud::Storage.new project_id: project_id
bucket  = storage.bucket bucket_name

file = bucket.create_file local_file_path, storage_file_path

puts "Uploaded #{file.name}"

jywarren · 2019-03-31T22:07:20Z

data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAQMAAAAlPW0iAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAABlBMVEX+AAD///+KQee0AAAAAWJLR0QB/wIt3gAAAAd0SU1FB+EGHRIVAvrm6EMAAAAMSURBVAjXY2AgDQAAADAAAceqhY4AAAAldEVYdGRhdGU6Y3JlYXRlADIwMTctMDYtMjlUMTg6MjE6MDIrMDI6MDDGD83DAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDE3LTA2LTI5VDE4OjIxOjAyKzAyOjAwt1J1fwAAAABJRU5ErkJggg==

dataurl

jywarren · 2019-04-01T00:00:51Z

test/system/screenshots_test.rb

+
+    # apparently this will wait for ajax to complete? 
+    # also, not sure we can nest css like this:
+    assert_select "ul.dropdown-menu li", count: 2 #, wait: 5


can use:
page.assert_selector('li', text: 'Horse', visible: true)
too

https://www.rubydoc.info/github/teamcapybara/capybara/master/Capybara/Node/Matchers#assert_selector-instance_method

jywarren · 2019-04-01T00:32:20Z

Hmm, seeing:

The previous command failed, possibly due to a malformed secure environment variable.
Please be sure to escape special characters such as ' ' and '$'.
For more information, see https://docs.travis-ci.com/user/encryption-keys.

Probably need to escape keys

jywarren · 2019-04-01T01:16:10Z

Hmm, two issues -- how do we require from Dangerfile?

[!] Invalid Dangerfile file: cannot load such file -- google/cloud/storage

And, we seem to be getting:

$ export RAILS_SYSTEM_TESTING_SCREENSHOT=inline
$ export GOOGLE_CLOUD_KEYFILE_JSON=[secure]
The previous command failed, possibly due to a malformed secure environment variable.
Please be sure to escape special characters such as ' ' and '$'.
For more information, see https://docs.travis-ci.com/user/encryption-keys.

jywarren · 2019-04-01T01:17:13Z

Although i tested these Google Cloud upload functions locally so I know they work! https://storage.googleapis.com/plots2-screenshots/test.png for example was uploaded from Ruby using the same GOOGLE_CLOUD_KEYFILE_JSON env variable...

jywarren · 2019-04-01T01:18:55Z

I'm going to put an encoded JSON value into a dummy JSON_TEST env variable in Travis to see how that appears and if it errors too.

…hat works

jywarren · 2019-04-01T03:08:06Z

@icarito any idea how to properly require the Google cloud gem in Dangerfile?

jywarren · 2019-06-10T22:23:21Z

Tried a couple things!

jywarren · 2019-06-10T22:45:28Z

Hmm, still /usr/local/bundle/gems/multi_json-1.13.1/lib/multi_json/adapters/json_common.rb:19:in encode': "\xE2" on US-ASCII (Encoding::InvalidByteSequenceError)`

jywarren · 2019-06-10T22:45:51Z

Aha, it's in: from /usr/local/bundle/gems/octokit-4.14.0/lib/octokit/client/issues.rb:292:in update_comment'
from /usr/local/bundle/gems/danger-6.0.9/lib/danger/request_sources/github/github.rb:198:in update_pull_request!'

jywarren · 2019-06-10T22:53:42Z

I dunno where that oddly encoded character is from! A typical Google Cloud Storage url might be something like:

https://00e9e64bacdf753409d0085a47f281709c1fabb9adcb05fa11-apidata.googleusercontent.com/download/storage/v1/b/mapknitter-exports-warps/o/1556868737%2Fsupermandala.png?qk=AD5uMEupiA1DaBwQ0B8OQaS78_KrzrERbxQ-UUd_kytYF1mFRtJ-y0BxXPYgunyc-pc2SsgBlE5tTt34Piy9sSe8SOd1BmNFyhqn1iUw6gDNtlp29UlW-eaaTJUwupdLMVflQxg6hg00rm8hl-VeHkYrt6qCxCJCuXhQi_C-HkE2j0buH8dK5J-C9jt-PhiZnrzbhOsvN84rWw67nY5i1tX2HXCm-4kxoQYwpD3iNCCYFETjZGzgEjbSbA44lVRzPhG2aSPvwBUg-W2wEt--m9kTn0XFZ8zYK2LbjibCtGoxa9kQGYhIvTq7ynvyMdSXy1uMyhdo-5cqy9GUkYfyW5ma_o1tkmuQZZFElaYDl6OxjZhnZGN2LPhEkeH_p78v2QB0FeeRCSWd0WVq10a855EqCLemqb3HbffarZasvV0SaUxUDV2IIbsJBpOPOsltZ2lZBXmNmfNyqbspttU7LWnwbzLtNzHPgs6-_j59KgsfnKQYGITA8_t__yIdgV8P-Lil6MDEnbD8ey0YDNiVfRbTJLdyeOd6D1W5qOuoMM4ZkIfku8gDT5eYrfVPlRKMCyAG_potBbFFv74OncXgySyqcgpfpJAiqVyAbGXzoI0XpNA3O2wdy1jG1bsBNYAnMfVtzYB1lsbSAPH71Gr6NGVc74yVV7q0tZBiy8LaFKQBbaqkE2YE4GT1N0JngkCrPt1m8hMALOnJxToAb-eFlLGObtyNsBqSEqibD1RXmpbruyhg-jsis7jm1pk4x58W3BaWw6-ggigcFQQtEXFWA4NUxzcOIFZCCM6ZWrhZ3GE25nT-sMScOPo

Or a public one:

https://storage.googleapis.com/plots2-screenshots/test.png

jywarren · 2019-06-10T23:10:34Z

Ahhhh! I bet it's the emoji camera! !!!!! 📸

Too bad camera... not this time!

jywarren · 2019-06-10T23:27:54Z

Hmm. Well that's odd. It wasn't the emoji, i guess? /usr/local/bundle/gems/multi_json-1.13.1/lib/multi_json/adapters/json_common.rb:19:in encode': "\xE2" on US-ASCII (Encoding::InvalidByteSequenceError)`

jywarren · 2019-06-10T23:28:55Z

They had the same issue here! stevekinney/pizza#103

jywarren · 2019-06-10T23:32:25Z

Dangerfile

+require "google/cloud/storage"
+storage = Google::Cloud::Storage.new project_id: "public-lab"
+bucket  = storage.bucket "plots2-screenshots"
+Encoding.default_external = 'UTF-8'


This may need: # encoding: utf-8 as per https://stackoverflow.com/questions/20521371/set-utf-8-as-default-for-ruby-1-9-3#20523482

Encoding.default_internal = Encoding::UTF_8

Ruby 2.0 and above: default string encoding is UTF-8.

Thus, if you use Ruby 2.0, you could skip the encoding comment and correctly assume UTF-8 encoding everywhere by default.

Yes maybe as a follow-up?

jywarren · 2019-06-10T23:45:06Z

THIS WORKED!!!!!! I'm changing the filenames, but this should be good to merge!!!!!!

jywarren · 2019-06-11T00:01:11Z

Hmm, why isn't it finding output.xml... @alaxalves was this somehow related to the splitting of the system tests into a parallel process? Junit should generate in both, I'd think though...

jywarren · 2019-06-11T00:02:10Z

Maybe this needs a rebase?

jywarren · 2019-06-11T00:16:00Z

Made a clean squashed version of this at #5868 that is also rebased... let's see!

jywarren · 2019-06-12T16:47:17Z

This is complete!!!!! #5868 worked perfectly after some tweaks!!!! Check it out 📸

Re #5320 #5868 #5316

* add screenshots for grids Re #5320 #5868 #5316 * adjust screenshot path

* add screenshots for grids Re publiclab#5320 publiclab#5868 publiclab#5316 * adjust screenshot path

* add screenshots for grids Re #5320 #5868 #5316 * adjust screenshot path

* add screenshots for grids Re publiclab#5320 publiclab#5868 publiclab#5316 * adjust screenshot path

Create screenshots_test

cd852cf

jywarren added 2 commits March 31, 2019 13:23

Rename screenshots_test to screenshots_test.rb

05d3f19

Update screenshots_test.rb

c217fcc

jywarren mentioned this pull request Mar 31, 2019

Brainstorm a list of critical full-stack system tests we should add #5316

Closed

13 tasks

Update Dangerfile

3d2b068

jywarren changed the title ~~Create screenshots_test~~ Take screenshots during system tests, convert to data-urls, and append to Dangerbot output Mar 31, 2019

jywarren added 2 commits March 31, 2019 15:06

relative path to screenshots

4394add

Update Dangerfile

97667d8

jywarren added 2 commits March 31, 2019 19:29

gem "google-cloud-storage"

f8eecc1

try assert_select "ul.dropdown-menu li", count: 2

79981f6

jywarren commented Apr 1, 2019

View reviewed changes

jywarren added 2 commits March 31, 2019 20:22

attempt gcloud upload

d871f8e

file.acl.public!

9ea5c1e

Update Dangerfile

ee6d328

Also trying to comment out require "google/cloud/storage" to see if t…

d1da598

…hat works

jywarren changed the title ~~Take screenshots during system tests, convert to data-urls, and append to Dangerbot output~~ Take screenshots during system tests, upload to Google Cloud storage, and append to Dangerbot output Apr 1, 2019

jywarren added 2 commits June 10, 2019 18:20

add ".png"

0aa9c20

change puts to message and output only file.name and file.public_url

508f133

Update Dangerfile

a32d000

jywarren added 2 commits June 10, 2019 19:11

remove true emoji

4f65112

Update Dangerfile

442b96d

jywarren added 2 commits June 10, 2019 19:30

Encoding.default_external = 'UTF-8'

b784006

Update Dangerfile

53d2f28

jywarren commented Jun 10, 2019

View reviewed changes

item.split('/').last

bcbecce

Update screenshots_test.rb

cb5a489

jywarren mentioned this pull request Jun 11, 2019

screenshots in system tests #5868

Merged

jywarren closed this Jun 12, 2019

jywarren added a commit that referenced this pull request Jun 12, 2019

add screenshots for grids

d00aa5d

Re #5320 #5868 #5316

jywarren mentioned this pull request Jun 12, 2019

add screenshots for grids #5878

Merged

jywarren added a commit that referenced this pull request Jun 12, 2019

add screenshots for grids (#5878)

b1f46a3

* add screenshots for grids Re #5320 #5868 #5316 * adjust screenshot path

rarrunategu1 pushed a commit to rarrunategu1/plots2 that referenced this pull request Jun 14, 2019

add screenshots for grids (publiclab#5878)

f960571

* add screenshots for grids Re publiclab#5320 publiclab#5868 publiclab#5316 * adjust screenshot path

sagarpreet-chadha pushed a commit that referenced this pull request Jun 29, 2019

add screenshots for grids (#5878)

1aa9062

* add screenshots for grids Re #5320 #5868 #5316 * adjust screenshot path

enviro3 pushed a commit to enviro3/plots2 that referenced this pull request Aug 15, 2019

add screenshots for grids (publiclab#5878)

1f1ccf1

* add screenshots for grids Re publiclab#5320 publiclab#5868 publiclab#5316 * adjust screenshot path

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Take screenshots during system tests, upload to Google Cloud storage, and append to Dangerbot output #5320

Take screenshots during system tests, upload to Google Cloud storage, and append to Dangerbot output #5320

jywarren commented Mar 31, 2019 •

edited

plotsbot commented Mar 31, 2019 •

edited

5320-test_front_page_with_navbar_search_autocomplete.png

5320-test_viewing_the_dashboard.png

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019 •

edited

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren Apr 1, 2019

jywarren Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren Jun 10, 2019

jywarren Jun 10, 2019

icarito Jun 10, 2019

jywarren Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 11, 2019

jywarren commented Jun 11, 2019

jywarren commented Jun 11, 2019

jywarren commented Jun 12, 2019

Take screenshots during system tests, upload to Google Cloud storage, and append to Dangerbot output #5320

Take screenshots during system tests, upload to Google Cloud storage, and append to Dangerbot output #5320

Conversation

jywarren commented Mar 31, 2019 • edited

plotsbot commented Mar 31, 2019 • edited

5320-test_front_page_with_navbar_search_autocomplete.png

5320-test_viewing_the_dashboard.png

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019 • edited

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren commented Mar 31, 2019

jywarren Apr 1, 2019

Choose a reason for hiding this comment

jywarren Apr 1, 2019

Choose a reason for hiding this comment

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Apr 1, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren commented Jun 10, 2019

jywarren Jun 10, 2019

Choose a reason for hiding this comment

jywarren Jun 10, 2019

Choose a reason for hiding this comment

icarito Jun 10, 2019

Choose a reason for hiding this comment

jywarren Jun 10, 2019

Choose a reason for hiding this comment

jywarren commented Jun 10, 2019

jywarren commented Jun 11, 2019

jywarren commented Jun 11, 2019

jywarren commented Jun 11, 2019

jywarren commented Jun 12, 2019

jywarren commented Mar 31, 2019 •

edited

plotsbot commented Mar 31, 2019 •

edited

jywarren commented Mar 31, 2019 •

edited