Integration with service workers

[marcoscaceres]

Specify the relationship between this spec and service workers.

Or get service worker to define the member and hook in through the extension point in the spec...

[marcoscaceres]

Rough proposal:

{
    "service_workers": [{
        "src": "foo.js",
        "scope": "/allthefoo*"
    }, {
        "src": "bar.js",
        "scope": "/the/bar/just-the-bar/"
    }]
}

[marcoscaceres]

@slightlyoff proposes just doing:

{
"service_worker": "...", 
"service_worker_options": { ... }
}

That keeps it simple, but limits only one SW registration per manifest.

[domenic]

What about:

{
  "service_worker": {
    "src": "foo.js",
    "scope": "/allthefoo"
  }
}

with the option of adding "service_workers" later if it turns out to be a valid use case?

Just cosmetically, I'm not a fan of two separate options for a single concept. But that's certainly minor.

[tobie]

I don't really understand why it's OK to have:

{
    "service_worker": { "src": "foo.js", "scope": "/allthefoo" }
}

But not:

navigator.serviceWorker.register({ src: "foo.js", scope: "/allthefoo" });

[tobie]

As mentioned in #manifest, I'd be very much in favor of being very lenient with input à la npm package.json, but I'm concerned this might be 1) hard to spec, 2) source of interop issues. Thoughts?

[marcoscaceres]

So, I'm going to spec what @domenic suggested. I trust @slightlyoff's judgement about only having one service worker allowed to be declared per manifest - though I find it somewhat limiting. On the other hand, I can see that it plays a nice conceptual "master service worker" role even if in practice it's just another service worker.

@tobie, sorry, discussion about navigator.serviceWorker.register is out of scope for this bug.

[tobie]

@tobie https://github.com/tobie, sorry, discussion about
navigator.serviceWorker.register is out of scope for this bug

Yeah, sorry. That was a poorly formulated way of asking for consistency
between both APIs.

[marcoscaceres]

Spec'ing it is not that hard. But it will definitely make things more complicated as it makes testing and interop way more of a challenge. I would be inclined to initially keep this strict and see how we go. We can always allow more types if we find a lot of people making authoring errors.

[marcoscaceres]

Going to get scope #114 working independently of SWs first. Still probably about 1-2 year till SWs are stable.

[marcoscaceres]

I'm still wondering if we can live without having an SW declared in the manifest at all. I'm probably missing something, but I'm still unsure as to the value of having a SW declared in the manifest is instead of, say, the start_url?

[kenchris]

Well that would mean that the service worker might not have been installed when the app has been added to the launcher and thus might not work offline until launched second time.

With it in the manifest, the UA could make sure that the service worker is installed (which in return could depend on fetching of resources to make offline work etc) when added to the home screen and fail adding it in case the service worker couldn't be installed.

I think that it makes sense

[marcoscaceres]

Well that would mean that the service worker might not have been installed when the app has been added to the launcher and thus might not work offline until launched second time.

True. But there is no guarantee that an app will ever be installed at all, and the app should work offline regardless (because 1. the UA might not support manifests, and 2. the user declined to install it, but still wants to access the app when offline - for example, the user is on vacation, they visited a restaurant's website, restarted their phone, and now want to view the website but offline).

In other words, installing an application should never be a precondition for getting offline functionality.

With it in the manifest, the UA could make sure that the service worker is installed (which in return could depend on fetching of resources to make offline work etc) when added to the home screen and fail adding it in case the service worker couldn't be installed.

I think that it makes sense

I agree. It makes sense. But this is predicated on some assumptions that might not hold in practice: the service worker might not actually provide any offline capabilities. The service worker, and its cache, might have been evicted. Having a service worker in the manifest shouldn't grant that service worker any additional rights (or longevity) when compared to a SW declared in script. Otherwise, we will just end up with everyone declaring SWs in the manifest and then problems if the manifest doesn't get used by the UA.

The more I think about this, the more I want to separate them: I feel like what the manifest provides and what SWs provide are separate concerns.

[kenchris]

The offline etc won't depend on the manifest support, it just means that the first launch after added to the home screen will already have the service worker ready and if the app developer decides, work offline, have push messages etc. Without this, the user might first have it on second launch.

Is there another way do fix that issue? Another option would be to have some event to be called when an app is added to the home screen, which then would allow the developer to ensure the SW is installed, or fail "installing" the app.

[marcoscaceres]

The offline etc won't depend on the manifest support, it just means that the first launch after added to the home screen will already have the service worker ready and if the app developer decides, work offline, have push messages etc. Without this, the user might first have it on second launch.

My working assumption is that installing just reopens the browser but, possibly, in a different display mode. Can you help me understand the issue a bit? This is what I assume happens:

1. user goes to https://foo.com.
2. foo.com registers a SW.
3. Time passes, pages are reloaded. SW is happily doing its thing.
4. Browser or user decides "this is install worthy! I'm going to keep this".
5. foo.com is added to homescreen.
6. foo.com is relaunched - but new display mode is applied. SW is still doing its thing. Everyone is happy?

What am I missing? :(

[marcoscaceres]

Let me put it a different way: let's see how we go initially without having this in the spec... at least allow Chrome and Gecko to catch up by supporting the same basic set of features.

It's going to be a few years before service workers are in all browsers. We have a better chance of enabling the manifest's basic functionality across a larger set of browsers if we don't put in a dependency on SWs at this point (IMO).

Unless people thing this is a critical feature, I would suggest we drop it for now.

[marcoscaceres]

Not hearing any objections, I'm closing for now.

[benfrancis]

@marcoscaceres I'd like to propose re-opening this, I didn't realise it had been closed. I think it's fine if some implementations don't get around to implementing this property for a while, nobody supports the scope property yet either.

that would mean that the service worker might not have been installed when the app has been added to the launcher and thus might not work offline until launched second time.

I think this is the key point. The service_worker property would not be the only way for a given Service Worker to be registered, it would just be a way of ensuring that it gets registered the moment a user installs an app. Otherwise, the app will only work offline the second time the app is launched.

I think being able to use a Service Worker as an installation script for an app by hooking into the worker's "install" event could be hugely powerful. Not just for downloading resources of the app into a cache for offline use, but for all kinds of purposes.

@slightlyoff, what do you think?

[benfrancis]

My working assumption is that installing just reopens the browser but, possibly, in a different display mode. Can you help me understand the issue a bit?

I don't think this is quite how it would work on all operating systems. There's no guarantee that things like cookies, IndexedDB databases and Service Worker registrations would survive the transition from the browsing context in the browser app to the application context in the OS. The OS might even use a different rendering engine to the browser the app was installed from.

[tobie]

Removed my knee-jerk reaction as it's a distraction to this thread. Apologies.

[marcoscaceres]

Maybe @jakearchibald can help here with an illustrative example... or at least tell me how wrong I am :)

@marcoscaceres I'd like to propose re-opening this, I didn't realise it had been closed. I think it's fine if some implementations don't get around to implementing this property for a while, nobody supports the scope property yet either.

The problem is not one of support, it's about use case and functionality.

I think this is the key point. The service_worker property would not be the only way for a given Service Worker to be registered, it would just be a way of ensuring that it gets registered the moment a user installs an app. Otherwise, the app will only work offline the second time the app is launched.

This is the bit I really don't understand - and I need some enlightenment here. As far as I can see, it means throwing away all the registration machinery that a developer gets with navigator.serviceWorker.register(). Would they, and us, really want to do that?

Additionally, there is no guarantee that the SW will be registered/ready in time before the user re-launches the app. The same machinery that governs installation of SW still applies, so installation can fail or be rejected for all the standard reasons. A user could also open the application while the UA is still waiting for the SW script to d/l, etc.... unless the web application is blocked from launching until the SW becomes ready, which might also be kinda sad (as it might never finish installing).

I don't understand why developers wouldn't just do the following prior to installation (or what advantage the manifest brings over just doing the following prior to installation of the web app):

//Register the SW when the page first loads. Done. 
if ('serviceWorker' in navigator) {
  window.addEventListener("DOMContentLoaded", (e) => {
    navigator.serviceWorker.register('/app/sw.js', {
      scope: '/app/'
    })
    .then(reg => console.log('Yey!', reg))
    .catch(err => console.log('Boo!', err));
  });
}

I'm still hoping someone can explain this to me. If possible, a real-life example would be great.

I think being able to use a Service Worker as an installation script for an app by hooking into the worker's "install" event could be hugely powerful. Not just for downloading resources of the app into a cache for offline use, but for all kinds of purposes.

Sure, I agree: but that's a general statement that applies to service workers, and the apps that use them, at large. By having this in manifest, what you are implying is that those capabilities only be made available to manifest-supporting user agents (as it would be only those user agents that would install that particular SW). At least, there is a risk that a developer could unwittingly exclusively target manifest supporting UAs - and I think that's pretty sad.

[jakearchibald]

I haven't really been involved in this proposal so far, so I'm coming at this fresh, apologies if I'm retreading old ground.

Uses

• If the added-to-home-screen site has a different instance of origin storage & ServiceWorkers, a declaration in the manifest means these storage instances can be populated by the SW at the time the user adds to home screen, rather than on first run. However, this would be better fixed by maintaining existing origin storage & SW. Losing/resetting data on adding-to-home-screen sounds like a bug
• The browser could provide a system to add sites to home screen without actually visiting them, eg a web app store. In this case it's desirable for a SW to set up before first visit. Also, the lifecycle of the SW can be used by the app store to indicate installation progress or signal failure
• The declarative SW form could be used by search engines to hint at app-like capability, although it's a very weak signal

Cons

I'm assuming that the SW details in the manifest are only actually used on add-to-homescreen or similar, as in not during regular page load.

• Duplication. You'll still need registration in the page

The only usage that feels particularly strong to me is the 2nd one, enabling an install flow for app-store like things. Personally I'd love to see search engines provide this functionality. Given Mozilla has a web app store, is it something they're keen on?

[kenchris]

im more thinking whether it might be more interesting and future proof to be able to run some JS code when "installing/adding to homescreen". This code should then succeed or fail (meaning failure would mean failure to add to homescreen). That code could then install the service worker (or fail)

[kenchris]

Yes, good idea :-) Then I can also see if I can understand it all. Marcos unfortunately won't be around.

[RobDolinMS]

@kenchris @jakearchibald @marcoscaceres @adrianhopebailie I won't be at TPAC but am supportive of Service Worker being included in the Web App Manifest.

Two use cases of Service Workers mentioned in an earlier comment:

• New Device / Back-up and Restore - When migrating to a new device, a user may "back-up" the list of apps from their old device and then "restore" their apps on the new device.
• Install now, first run later - Similar to above. I personally hit this over the weekend while traveling with my two-year-old child. I browsed a list of apps/games for age < 5 and installed a few of these. Then, when on the 4.5 hour cross-country flight (without Wi-Fi), I tried to run these apps.

[marcoscaceres]

The use cases are very compelling - and would like to explore how they would work in practice. Open questions:

• should we allow multiple service work registrations? Or just a single one?
• what happens when registration fails?

[adrianhopebailie]

Yes, good idea. Apologies if my frustrations with understanding the different boundaries between cookies, apps, sw's etc turned things sour. I have a long standing peev with the number of moving parts in W3C specs that don't seem to inter-operate well unless you have some of the tribal knowledge that helps you understand the nuances.

We had a payment app task force call today and this was discussed so I think I can provide some more clarity:

• Payment apps will be ServiceWorkers (the bleeding edge spec from @adamroach provides more detail: https://adamroach.github.io/webpayments-payment-apps-api/#payment-app-manifest)
• The current proposal is to define our own data model that would be input into the ServiceWorker registration process to provide meta-data about the app. As you can see from some of the issue markers we need to deal with many of the same issues that appmanifest has already dealt with like "What is an icon, how do we have multiple icons of different sizes etc".
• A case has been made for putting this data into a file so it can be re-used outside the scope of the ServiceWorker. See: The relationship between payment apps and service workers payment-handler#33 (comment)
• Another reason for having this meta-data available prior to executing the SW registration is that the browser needs some of this meta-data in providing UI to the user to, for example, prompt them to install a payment app they don't have but which is known to support a specific payment method.

So payment apps would benefit from the data model and algorithms defined in app manifest for things like specifying icons but we don't want to buy into the life-cycle management of appmanifest we want to use SW for that.

What could change to make this work "better":

1. Appmanifest defines it's data model using WebIDL and separate algorithms for parsing the JSON and processing the data. This AppManifest could probably replace the PaymentAppManifest or just be extended by it?
2. Appmanifest describes it's current lifecycle management stuff in terms of it's specific use case and in a way that explicitly says "use this data model in other ways if you want to but this is how we use manifest files via link tags and this is what to expect if that's what you do"
3. Appmanifest includes the possibility of defining a SW instead of a start_url and the "installation" of the app follows the SW lifecycle where the WebIDL defined JS object version of the manifest data is available to the SW during the registration process. This mirros the pattern in other platforms of having a common code base that can be executed differently based on context and runtime parameters (ENV variables, config files etc)

With the following changes a payment app would simply be a specialization of SW that uses appmanifests to define it's meta-data. I may be missing some key considerations here so invite a conversation to clarify at TPAC.

[marcoscaceres]

Yes, good idea. Apologies if my frustrations with understanding the different boundaries between cookies, apps, sw's etc turned things sour. I have a long standing peev with the number of moving parts in W3C specs that don't seem to inter-operate well unless you have some of the tribal knowledge that helps you understand the nuances.

Welcome to the club! :)

So payment apps would benefit from the data model and algorithms defined in app manifest for things like specifying icons but we don't want to buy into the life-cycle management of appmanifest we want to use SW for that.

Re: icons, we might be able to push that stuff down the stack to HTML. It's where "responsive images" are defined. The manifest spec leverages HTML and fetch where it can, because we want both the security assurances and the consistent behavior.

Another spec dealing with icons is Notifications - so it also provides a good place to look (though it currently doesn't do responsive images for icons, AFAICT).

1. Appmanifest defines it's data model using WebIDL and separate algorithms for parsing the JSON and processing the data. This AppManifest could probably replace the PaymentAppManifest or just be extended by it?

Need to do this regardless. I've had similar requests internally. However, we still need to go through the processing step to clean up the manifest's data to get something that can be fed to the IDL machinery.

1. Appmanifest describes it's current lifecycle management stuff in terms of it's specific use case and in a way that explicitly says "use this data model in other ways if you want to but this is how we use manifest files via link tags and this is what to expect if that's what you do"

Yep?

1. Appmanifest includes the possibility of defining a SW instead of a start_url

That doesn't make sense to me. But it could define an SW in addition to the start_url.

and the "installation" of the app follows the SW lifecycle where the WebIDL defined JS object version of the manifest data is available to the SW during the registration process.

If installation is triggered from the manifest, then sure. But why can't you just fetch it? Why do you need the clean object?

This mirros the pattern in other platforms of having a common code base that can be executed differently based on context and runtime parameters (ENV variables, config files etc)

Let's not jump the shark. Remember you can still get the manifest simply by:

addEventListener("install", (ev)=>{
    ev.waitUntil(async function(){
        const json = await fetch("manifest.json").then(r => r.json());
       // ok now what?
    })
}); 

The question is, what do you then want to do with the manifest data?

[marcoscaceres]

(updated code above, added await and comment)

[jakearchibald]

I think the payments stuff is a bit of a distraction here, because I still think their proposed reliance on manifest is a step against the extensible web, and @marcoscaceres example above shows that developers could choose to use JSON (or YAML, or whatever) really easily.

That aside, I'm open to service worker being referenced in the manifest. @RobDolinMS:

New Device / Back-up and Restore - When migrating to a new device, a user may "back-up" the list of apps from their old device and then "restore" their apps on the new device.

The service worker scope & script URL could just be backed up along with the app here right?

Install now, first run later - Similar to above. I personally hit this over the weekend while traveling with my two-year-old child. I browsed a list of apps/games for age < 5 and installed a few of these. Then, when on the 4.5 hour cross-country flight (without Wi-Fi), I tried to run these apps.

I agree this requires a declarative service worker registration, but we already have <link rel="serviceworker">, is this enough?

[kenchris]

I would say yes, it works for <link and Link header, so why shouldn't it be enough for manifest?

How do we handle today if there is a Link header with SW, a tag and a JS registration?

[jakearchibald]

How do we handle today if there is a Link header with SW, a tag and a JS registration?

If they have different scopes, you get three registrations. If they have the same scope and script URL, they coalesce. If they have different script URLs, last one wins.

[marcoscaceres]

Ok, I'm convinced. Let's do this.

[benfrancis]

\o/

To be fair it is now nearly two years since you said:

Still probably about 1-2 year till SWs are stable.

[marcoscaceres]

On 20 Sep. 2016, at 11:39 pm, Ben Francis notifications@github.com wrote:

\o/

To be fair it is now nearly two years since you said:

Still probably about 1-2 year till SWs are stable.

Was I wrong? ;) well, maybe 50% wrong.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[RobDolinMS]

@marcoscaceres wrote:

Ok, I'm convinced. Let's do this.

Woo hoo! /cc @seksenov

[marcoscaceres]

Relevant discussion in the SW repo:
w3c/ServiceWorker#979

    "service_workers": [{
        "src": "sw.js",
        "scope": "/foo",
         "options": {} 
    }

[RobDolinMS]

@marcoscaceres Would you like me (or a colleague) to take a stab at a PR for your proposal?

[marcoscaceres]

@RobDolinMS, sure! That would be great! I'd suggest starting small tho, because integration with the service worker spec is potentially tricky (e.g., there might not be a client for the installation, etc.).

[kenchris]

Any update on this?

[marcoscaceres]

On 20 Oct. 2016, at 10:38 pm, Kenneth Rohde Christiansen notifications@github.com wrote:

Any update on this?

After we finish BIP I'll get onto it, unless the MS folks beat me to it... or do you want to get it started?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[kenchris]

I can do it

[anssiko]

A fix landed in #507, closing this issue.

Opened #542 to track a related issue.

[nuxodin]

Maybe we should add a "type" member for "service_workers"

{ "type": "module" }

Will be needed once Browsers support serviceWorkers as js modules.
https://html.spec.whatwg.org/#module-worker-example

[kenchris]

@nuxodin can you open a separate issue for this?

[nuxodin]

done: #594