Do not download vendor JS files in released plugin versions #1025

nylen · 2017-06-05T17:33:34Z

This PR introduces the first example of logic needing to vary between the development and release versions of the plugin. In development mode, we download and cache "nightly" vendor JavaScript files, but this isn't appropriate for release - we should bundle these with the plugin instead.

This is accomplished by generating a slightly different gutenberg.php file during the plugin release process, then only downloading vendor files if the GUTENBERG_DEVELOPMENT_MODE constant is truthy.

We respect the SCRIPT_DEBUG constant for these vendor scripts, but not yet for our own JavaScript. I'll create a separate issue for this (#1026).

Follow-up to #985.

Addresses #953 (comment), after this we should be ready to release Gutenberg to the WP plugins directory.

Fixes #921 at the same time, since I was making changes in that area.

aduth

I'm not going to pretend I'm not hopelessly lost trying to understand some of the Bash wizardry here. A bit worrying from maintainability perspective, but then again, probably not a permanent solution I'd imagine. Ideally we'd move off the nightlies. I think we can already for TinyMCE. React 16's latest alpha was released back in February, so not as promising there.

Should we add some of the temporary files to gitignore in case the script is aborted or otherwise exists mid-execution? Thinking *.tmp.js.

aduth · 2017-06-06T18:05:30Z

bin/build-plugin-zip.sh

+	php bin/get-vendor-scripts.php debug
+)
+while IFS='|' read -u 3 url filename; do
+	wget -nv "$url" -O "vendor/$$.tmp.js"


wget is not stock Mac friendly:

./bin/build-plugin-zip.sh: line 41: wget: command not found

Maybe just curl: https://stackoverflow.com/a/4572158

TIL $$, but also TIL apparently it's unwise for temporary files?

https://stackoverflow.com/questions/78493/what-does-mean-in-the-shell

The issues with $$ and other "weak" methods of temporary filename generation are related to privileged processes that can be tricked into writing unexpected files that a user would not otherwise be able to write to. Thankfully, this set of issues does not apply to us.

Still, I've removed the use of $$ to avoid promoting this practice.

aduth · 2017-06-06T18:09:34Z

bin/build-plugin-zip.sh

 # Run the build
 npm install
 npm run build

 # Remove any existing zip file
 rm -f gutenberg.zip

+# Temporarily modify `gutenberg.php` with production constants defined
+php bin/generate-gutenberg-php.php > gutenberg.$$.php


Can we write directly into gutenberg.php here?

I wouldn't recommend it, since the php script is reading line-by-line from the same file at the same time. This might work due to kernel buffering or something like that, but it seems like undefined behavior to me. Added a comment to clarify.

nylen · 2017-06-08T14:20:39Z

I'm not going to pretend I'm not hopelessly lost trying to understand some of the Bash wizardry here. A bit worrying from maintainability perspective, but then again, probably not a permanent solution I'd imagine.

I added a couple more comments to try to address this - but yes, I agree this is not a permanent solution, and the way we handle vendor scripts needs some reworking. Nightlies are part of the issue, our current setup does what we need it to do without having to be configured, but could probably be redesigned a bit.

Should we add some of the temporary files to gitignore in case the script is aborted or otherwise exists mid-execution? Thinking *.tmp.js.

Everything in vendor/ is already ignored.

nylen · 2017-06-09T15:19:07Z

I'd like to merge this today so that we can move forward with a release to the plugin directory ahead of WCEU.

aduth

Should I expect that this downloads both minified and unminified versions of dependencies?

https://unpkg.com/react@next/umd/react.production.min.js
 > vendor/react.min.c7397a88.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom.production.min.js
 > vendor/react-dom.min.912dc6cf.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom-server.production.min.js
 > vendor/react-dom-server.min.538e07c8.js ... done!
https://unpkg.com/moment@2.18.1/min/moment.min.js
 > vendor/moment.min.0fc04ee9.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/tinymce.min.js
 > vendor/tinymce.min.04a1d751.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/plugins/lists/plugin.min.js
 > vendor/tinymce-plugin-lists.min.e95f0005.js ... done!
https://unpkg.com/react@next/umd/react.development.js
 > vendor/react.3a8927d9.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom.development.js
 > vendor/react-dom.e5cfcfc9.js ... done!
https://unpkg.com/react-dom@next/umd/react-dom-server.development.js
 > vendor/react-dom-server.e84b9fa6.js ... done!
https://unpkg.com/moment@2.18.1/moment.js
 > vendor/moment.7eefd5c2.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/tinymce.js
 > vendor/tinymce.c847083c.js ... done!
https://fiddle.azurewebsites.net/tinymce/nightly/plugins/lists/plugin.js
 > vendor/tinymce-plugin-lists.557da104.js ... done!

aduth · 2017-06-09T15:26:52Z

bin/generate-gutenberg-php.php

+function print_production_defines() {
+	global $plugin_version;
+
+	echo "define( 'GUTENBERG_VERSION', '$plugin_version' );\n";


Can you explain why we want these defines? They don't appear to be used? Is it just because we need to replace the development mode constant so we might as well? Wondering if it'd help simplify this at all if we just moved to stripping out the define section during packaging.

Is it just because we need to replace the development mode constant so we might as well?

There are a couple of other reasons as well:

This is a simpler approach to Use get_plugin_data for production script versioning (cachebust) #440. See Add asset cache version helper function. #585 for another way to do it which I would not be as comfortable merging.

Tying each plugin release to a particular git commit is a good idea to make sure that we can reliably identify where each build came from. This should also be done using git tags, for example, but a bit of redundancy here is a good thing.

aduth · 2017-06-09T15:33:15Z

Observing persistent 500 error after trying to activate plugin generated by this branch on a test site. Will follow-up when I learn more.

Edit: Appears to be because the plugin is trying to require newly-introduced blocks/library/latest-posts/index.php (#870) and is not finding it. Also an issue on master.

aduth · 2017-06-09T15:40:50Z

bin/build-plugin-zip.sh

+while IFS='|' read -u 3 url filename; do
+	echo "$url"
+	echo -n " > vendor/$filename ... "
+	curl -s "$url" -o "vendor/_download.tmp.js"


I think we need to follow redirects here (-L).

Finding I have vendor dependencies files written with the content like the following:

<p>You are being redirected to <a href="/react-dom@16.0.0-alpha.13/umd/react-dom.development.js">/react-dom@16.0.0-alpha.13/umd/react-dom.development.js</a>

Indeed, I am not sure how I missed this 😬 Fixed in latest push.

nylen · 2017-06-09T19:54:02Z

Should I expect that this downloads both minified and unminified versions of dependencies?

Yes, so that we respect (at least partly) the SCRIPT_DEBUG setting in users' WP installations - see #1026.

nylen · 2017-06-09T21:11:00Z

Observing persistent 500 error after trying to activate plugin generated by this branch on a test site.

Fixed in #1095 and master; rebased this PR afterwards. This branch is testing out well for me again now.

aduth

Working well now 👍

mtias · 2017-06-12T15:30:07Z

Thanks for working on this one.

nylen · 2017-06-12T19:37:53Z

Thanks for reviewing. I've been thinking about how to simplify this, and probably (when we decide we're ready to stabilize dependency versions a bit) we can do something like this:

Move the code to download vendor dependencies out of PHP and into npm run download-deps or similar. Make the PHP code simply reference these downloaded files.
Make other npm run commands (dev, build, package-plugin) depend on this step.

We would lose the current behavior where we automatically download new versions of these dependencies every day, so we probably shouldn't do this until we are reasonably confident in our versions of React and TinyMCE in particular.

aduth reviewed Jun 6, 2017

View reviewed changes

nylen force-pushed the add/vendor-js-to-plugin-zip branch from 9f9d49b to 7be1f75 Compare June 8, 2017 14:16

nylen force-pushed the add/vendor-js-to-plugin-zip branch from 7be1f75 to 67d6c42 Compare June 8, 2017 14:25

aduth reviewed Jun 9, 2017

View reviewed changes

Do not download vendor JS files in released plugin versions

171f9c5

nylen force-pushed the add/vendor-js-to-plugin-zip branch from 791d72a to 171f9c5 Compare June 9, 2017 21:03

aduth approved these changes Jun 12, 2017

View reviewed changes

mtias added [Type] Build Tooling Issues or PRs related to build tooling Gutenberg Plugin Issues or PRs related to Gutenberg Plugin management related efforts labels Jun 12, 2017

nylen merged commit 16680dd into master Jun 12, 2017

nylen deleted the add/vendor-js-to-plugin-zip branch June 12, 2017 19:35

This was referenced Jun 12, 2017

Releasing to the plugin directory #1147

Closed

Add asset cache version helper function. #585

Closed

shivapoudel mentioned this pull request Jul 6, 2021

Tools: Husky upgrade and git ignorance improved #33183

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not download vendor JS files in released plugin versions #1025

Do not download vendor JS files in released plugin versions #1025

nylen commented Jun 5, 2017 •

edited

aduth left a comment

aduth Jun 6, 2017

aduth Jun 6, 2017

nylen Jun 8, 2017

aduth Jun 6, 2017

nylen Jun 8, 2017

nylen commented Jun 8, 2017

nylen commented Jun 9, 2017

aduth left a comment

aduth Jun 9, 2017

nylen Jun 9, 2017

aduth commented Jun 9, 2017 •

edited

aduth Jun 9, 2017

nylen Jun 9, 2017

nylen commented Jun 9, 2017

nylen commented Jun 9, 2017

aduth left a comment

mtias commented Jun 12, 2017

nylen commented Jun 12, 2017

Do not download vendor JS files in released plugin versions #1025

Do not download vendor JS files in released plugin versions #1025

Conversation

nylen commented Jun 5, 2017 • edited

aduth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nylen commented Jun 8, 2017

nylen commented Jun 9, 2017

aduth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aduth commented Jun 9, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nylen commented Jun 9, 2017

nylen commented Jun 9, 2017

aduth left a comment

Choose a reason for hiding this comment

mtias commented Jun 12, 2017

nylen commented Jun 12, 2017

nylen commented Jun 5, 2017 •

edited

aduth commented Jun 9, 2017 •

edited