Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify how updates work #446

Closed
adrianhopebailie opened this issue Apr 5, 2016 · 45 comments · Fixed by #1011
Closed

Specify how updates work #446

adrianhopebailie opened this issue Apr 5, 2016 · 45 comments · Fixed by #1011
Assignees

Comments

@adrianhopebailie
Copy link

adrianhopebailie commented Apr 5, 2016

Is there a reason that the spec only defines a declarative way to request that a user agent obtain and process a manifest?

Likewise, it would be useful to have an explicit way to request that the user-agent update its version of the manifest.

@marcoscaceres
Copy link
Member

@adrianhopebailie yeah, we might need to add an explicit means to request an update to mitigate security issues: that is, we want to be able to show what's changed and give the end-user a chance to review or dismiss the update.

Excuse the crappy mock-up I slapped together below, but want to illustrate the point:

screenshot 2016-04-06 14 20 41

The UA would also need to show more info, like if the display mode or start URL changed, etc. Basically, anything that could be a concern to a user.

@marcoscaceres marcoscaceres changed the title Support a way for an app to be installed explicitly? Support a way to update explicilty Apr 6, 2016
@marcoscaceres
Copy link
Member

@adrianhopebailie, about "installing explicitly", most browser vendors would like to pursue the installabiltiy signals route. If that doesn't work out, we might need to add an .requestInstall() API. However, we have resisted that option because we don't want install buttons all over the web.

@adrianhopebailie
Copy link
Author

What is the logic a user agent should apply to decide if the installation of one manifest is actually a new install or an update/replacement of an existing one? It would seem that this should be related to scope but I'm not sure if that's explicitly defined anywhere?

Put differently, why would you say this?

we want to be able to show what's changed and give the end-user a chance to review or dismiss the update.

If I visit the same URL a week apart I get no warnings that the content has changed since the last time I visited?

If the "effect" of a manifest is limited to functions that will not impact security or privacy then the normal controls for managing those risks should suffice surely?

@marcoscaceres
Copy link
Member

If I visit the same URL a week apart I get no warnings that the content has changed since the last time I visited?

But you totally do. For instance, Safari shows you the following when you go to bbc.com/news:

screenshot_2016-04-08_17_57_08

And Chrome:
screenshot_2016-04-08_18_07_19

There are a lot of indicators built into browsers to make users aware of what is going on. And, if for instance, you retuned to the bbc.com and their cert had expired, the browser would warn you.

The same with native apps, like providing either, or both, an accessible history showing what's being updated:

screenshot 2016-04-08 18 02 27

The reason it feels different than traditional software updates is that the Web's same origin policy and browser sandboxing protects web pages from meddling with the underlying OS. Web manifest changes that:

  1. it can add an icon to the user's homescreen.
  2. it can set the name of the application.
  3. it can take away the browser's chrome.
  4. it can redirect the browser somewhere unexpected (via start_url)
  5. it can hide information about the user in the start URL (or other URLs - if those URLs are rewritten and captured via a service worker, for instance)

If the "effect" of a manifest is limited to functions that will not impact security or privacy then the normal controls for managing those risks should suffice surely?

But they would impact security and privacy (see all the recently added security and privacy notes in the spec). So doing a "sneaky" switcharoo of the icons and name of the icons that the user is not aware of would be really bad. At least, at Mozilla I know we would be absolutely against just allowing applications to switch any part of the manifest without the user knowing about it (and modifying them).

@adrianhopebailie
Copy link
Author

I'm sorry but I think of myself as a pretty "experienced" web user and I definitely don't consider, the fact that a browser indicates to me a page is being fetched and loaded, as a hint that that the underlying content has changed. I think that is over-reaching a little.

Are you suggesting that an average user of the web understands the difference between a page loaded from the cache and one fetched from the Web and ALSO understands that the latter MAY mean the code of the Web page is different to the last time they loaded that page?

The fact that start_url can change is the only reason all this complex change control is required at all. Wouldn't it be better to bind a manifest to a start_url?

Perhaps if you could answer my initial question it would help me understand this better:

What is the logic a user agent should apply to decide if the installation of one manifest is actually a new install or an update/replacement of an existing one? It would seem that this should be related to scope but I'm not sure if that's explicitly defined anywhere?

It seems to me that it's possible to have a URL that falls under overlapping scopes of two different manifests. In which case, if I land on that URL, which application-context, if any, is loaded?

@marcoscaceres
Copy link
Member

It seems to me that it's possible to have a URL that falls under overlapping scopes of two different manifests. In which case, if I land on that URL, which application-context, if any, is loaded?

We haven't worked out what to do about overlapping scopes. Hope to get to that soon.

However, the problem of requesting an explicit update remains. Relates to #384

@boyofgreen
Copy link

Binding to a start URL doesn't quite fit the grander idea of scope either. Since I may "deep link" into an app, the scope needs to cover all that would be part of the app. The idea we are trying to solve in when you have an application like gmail, where mail.google.com is the scope, but contacts.google.com should also be in scope. In contrast, docs.google.com and www.google.com should not be in scope.

@kenchris kenchris changed the title Support a way to update explicilty Support a way to update explicitly Apr 26, 2016
@benfrancis
Copy link
Member

So now that the update text has been removed, as far as the spec is currently concerned a manifest gets set at install time and can never be updated? Although I'm aware of the risk of app identity theft, I'm not sure this is better than what was there before. Seems like a high priority to address updates.

@marcoscaceres
Copy link
Member

On 27 Apr 2016, at 9:13 PM, Ben Francis notifications@github.com wrote:

So now that the update text has been removed, as far as the spec is currently concerned a manifest gets set at install time and can never be updated?

Correct.
Although I'm aware of the risk of app identity theft, I'm not sure this is better than what was there before. Seems like a high priority to address updates.

Yes. I agree.


You are receiving this because you commented.
Reply to this email directly or view it on GitHub

@marcoscaceres marcoscaceres added this to the Level 2 milestone May 31, 2016
@cheshrkat
Copy link

I just ran into this while spiking the use of app manifest. It's great to get onto the device, but the spec definitely needs to talk about the way(s) you can update the content.

@RobDolinMS
Copy link
Contributor

I'm thinking there are two possible scenarios here:
A) Every time the UA boots the app (and is connected to the Internet), the browser checks for an updated Web App Manifest
B) The Web App Manifest includes something like a time-to-live (TTL) value and UAs should not run the app if the TTL has expired

Is one (or both) of these what you're interested in @adrianhopebailie ?

@kenchris
Copy link
Collaborator

AFAIU, the Android team is looking into updating the app data given manifest changes as part of the work on WebAPK. It would be great to have their feedback. @PaulKinlan ?

@marcoscaceres
Copy link
Member

I think we should limit the declarative features of the manifest as much as possible, particularly ones where we can do better (for developers) by making them programmatic. I believe there is, nevertheless, and as @RobDolinMS suggests, the opportunity for a good mix of collaboration between the browser and the web application: where the browser can automatically check for an update (as service workers do every 24 hours) - or the developer can request an update, through an API.

As I argued in #446 (comment), the end user needs to be in control of the update process (similarly to how they go through the installation process), because we are asking the user agent to make a pretty scary change at the OS level (the application icon, name, start_url, and possibly other things can change).

@marcoscaceres
Copy link
Member

@PaulKinlan, feedback would be tremendously useful, particularly if there is going to be some kind of impact on the API surface. It may be we reuse the onbeforeinstall infrastructure here to perform the update - which would make most sense.

@nunoperalta
Copy link

It is really unfortunate we can't update our PWA. This can be considered a stopper in many projects.

manifest.json should have a "version" that can be updated, and if browsers detect that that version number has changed, they can ask the user whether they would like to receive the update or not.

I was asking about this in a StackOverflow question... https://stackoverflow.com/questions/47762022/apply-changes-to-web-app-manifest-on-an-installed-pwa

@marcoscaceres
Copy link
Member

Version is orthogonal to updating. Nonetheless, update is important and something I hope we solve in 2018.

@dominickng
Copy link
Collaborator

Note that Chrome now will poll the web manifest of an installed PWA on Android from time to time and automatically update if changes are detected. A more explicit updating mechanism may be better though since Chrome's process is entirely user-transparent.

@nunoperalta
Copy link

I had tried yesterday to make an update to the app name in the manifest and then change the link to "manifest.json?v=2", which ensures the browser will download the new version. However, the app name did not change.

@dominickng
Copy link
Collaborator

Changing the manifest URL shouldn't be required (and it may even interfere with the process).

Assuming you have the app installed via the improved add to home screen (i.e. it appears in the Android app list), when you launch the app Chrome will fetch the manifest, parse it, and compare its data to the data we have baked into the app. If things differ, an update is requested and will go through when possible. Hence this being entirely user-transparent, it should just happen in the background.

@ewsizeit
Copy link

What URL is Chrome using to fetch the manifest? Is it the original url the manifest was fetched from when it was installed or the start_url member?

I see a danger in updating using the manifest presented by the page in start_url. What if you make a mistake in an update, for example a misspelled domain name. All installed apps would update and then have no way of fetching the right manifest again. Basically locking yourself out of your installed app base.

Irregardless of the implementation details, that's great news. We can start producing apps without worrying that they will be impossible to update. The ideal would be to specify this in the standard.

@madmoizo
Copy link

madmoizo commented Jan 30, 2018

when you launch the app Chrome will fetch the manifest, parse it, and compare its data to the data we have baked into the app. If things differ, an update is requested

@dominickng I'm afraid it does not work or please provide more details about the update process. I tried with a static name manifest.json and with a dynamic one manifest.[hash].json and Chrome does not update the manifest.
I opened an issue with a screencast example (even if it's more a spec issue...) https://bugs.chromium.org/p/chromium/issues/detail?id=806518#c4

[Edit] comment from blink team:

We check for updates when a webapk is launched, but throttle to once every 3 days (IIRC)

@mgiuca
Copy link
Collaborator

mgiuca commented May 20, 2020

@alancutter has been working on the Chrome manifest update logic, and should be able to answer questions on this.

@alancutter
Copy link
Contributor

Chrome compares the latest manifest against what is currently installed throttled to once per day. If changes are found it will wait until all app windows are closed then update its web app installation to reflect those changes (app name excluded).

I can imagine sites that want control over this behaviour would like a checkForManifestUpdates() function and a beforemanifestupdate event that can be event.preventDefault()ed.

@g-ortuno
Copy link

g-ortuno commented May 20, 2020

I suspect we do something different on Android though.

@alancutter
Copy link
Contributor

Android does the same except the check is done on a central server and manifest updates are pushed out globally. This would require major changes to support any kind of update API.

@dominickng
Copy link
Collaborator

@alancutter @g-ortuno the manifest fetching and parsing for updates is still done on-device on Android, with parsed values and icons being sent to the server to compute whether or not an update is needed based on those. Supporting explicit updating would probably just require some extra flag to "force an update", which may already exist.

The implementation side should be orthogona though - here we should be answering the question of whether this is a sensible and desirable functionality that should be supported.

@marcoscaceres
Copy link
Member

Some documentation for Chrome for Android: https://web.dev/manifest-updates/

@marcoscaceres marcoscaceres changed the title Support a way to update explicitly Specify how updates work Aug 9, 2021
@evanstade
Copy link

Version is orthogonal to updating.

Can you explain what you mean by this @marcoscaceres ? It seems straightforward enough to define a rule such as, "if the user agent finds that the version in the manifest is higher than the version at last install/update time, it MAY apply changes found in the manifest to the installed app"

I imagine a version field would also be useful for orthogonal things, like helping developers identify the code that's being run (e.g. on a user's system when a bug is reported).

Chrome compares the latest manifest against what is currently installed throttled to once per day. If changes are found it will wait until all app windows are closed then update its web app installation to reflect those changes

(Desktop) Chrome is forced to not just compare the manifest to the last seen manifest, but also re-download and compare all images every time, which is a fairly intensive operation. Hence the throttling. But if Chrome were able to skip the update by avoiding this step and only updating after an explicit update signal (via version comparison), the throttling might not be necessary. The throttling is a known/common source of confusion, to the extent that we have documentation that we frequently point confused developers to.

@dmurph
Copy link
Collaborator

dmurph commented Aug 20, 2021

Quick thoughts on version:

  • Would be a nice hint
  • Relying on it would obviously break existing use cases, but we could throttle non-hint cases to reduce resources, and guarantee resources
  • Possibly a privacy risk - version could be a user id. Would have to be not accessible, and even then, it reveals some info about the value when the manifest updates?
    • but I guess we could reset version on browsing data cleared?

I see value as an improvement of update predictability.

@aarongustafson
Copy link
Collaborator

@marcoscaceres
Copy link
Member

@evanstade wrote:

Can you explain what you mean by this @marcoscaceres ? It seems straightforward enough to define a rule such as, "if the user agent finds that the version in the manifest is higher than the version at last install/update time, it MAY apply changes found in the manifest to the installed app"

Sure, the problem is: how do you compute "higher"? Do you do a simple ascii comparison? Or would you require implementing something like sem-ver, which would lead to a lot of complexity (e.g., "01.01.10-beta-1" GT "1.1.10-beta-1""). The best we can do is "is it different"?. What if people don't want to use sem-ver? and so on...

I imagine a version field would also be useful for orthogonal things, like helping developers identify the code that's being run (e.g. on a user's system when a bug is reported).

Sure, but that's equally handled by "version" being an opaque string (as in, "oh, looks like that's good ol' version 'vva.234fad' at it again").

@aspiers
Copy link

aspiers commented Aug 24, 2021

I'm strongly in favour of a versioning scheme to enable predictable updates. As we all know, versioning is a solution to this problem which is used and well-understood across the whole industry, and it should be easy to retain backwards compatibility.

I would suggest using semver unless there are compelling alternatives or reasons not to (which seem unlikely to me at this point). Semver is a very well-trodden path and avoids reinventing wheels. I don't think it's compelling to say that semver leads to a lot of complexity, because it's essentially a very simple concept which is already well understood by many developers and already has multiple implementations.

In other words, update mechanisms via versioning is already a solved problem, so I think it makes sense to treat it as one rather than looking for a new solution. Reinventing wheels would mean a whole new updates and/or versioning scheme for developers to learn, even though it would only exist in this one place.

@evanstade
Copy link

The best we can do is "is it different"?

I think we'd be OK with this and still using it to help manage updates. A lower version number might indicate that a user ought to be down-graded and I can't think of a reason not to support that.

We are concerned about maintaining functionality (at least for now) for apps that don't specify a version at all. Thus, Chrome's ideal behavior would be something like:

  • store the version specified during most recent update/install
  • when processing a manifest for update
    • if version is not found, use current update procedure (i.e. compare all manifest fields and image contents to determine need for updating, but throttle this to ~once/day)
    • if version is found
      • and matches saved version, do not update
      • and does not match saved version, do update, without throttling (probably -- might need to consider some amount of throttling still)
  • (possibly) display version in user-visible app info surface(s), to assist with debugging in the field

If I'm not mistaken, the only part of the above behavior which turns into actual spec is

if version is found, and matches saved version, do not update

App developers should be motivated to use version because it helps reduce unnecessary bandwidth usage (i.e. the user agent does not need to download all images again to make sure they haven't changed) and makes updating more predictable (reduces need for user agent to throttle).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.