Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Presentation Testing API #440

Open
mfoltzgoogle opened this issue Sep 21, 2017 · 10 comments
Open

Presentation Testing API #440

mfoltzgoogle opened this issue Sep 21, 2017 · 10 comments
Assignees

Comments

@mfoltzgoogle
Copy link
Contributor

Currently a good portion of our web platform tests require manual intervention. This task tracks the work to implement a testing API to allow the tests to:

  1. Create and remove a fake presentation screen
  2. Select or cancel presentation
  3. Start presentation from the browser
  4. Implement the "stash" functionality?

We should consider prior work by WebVR and other device APIs here around testing APIs.

@anssiko
Copy link
Member

anssiko commented Oct 24, 2017

Looping in @foolip with whom I've discussed the challenges of manual w-p-t testing of device APIs, particularly Vibration API and Battery Status API.

@Honry has been championing the said test suites recently, and may also have some ideas for improvement to the manual testing of Presentation API.

This might be another good concrete case to discuss in the web-platform-tests breakout session at TPAC on Wednesday. Or perhaps manual testing challenges would warrant a separate breakout session.

@tomoyukilabs
Copy link
Contributor

  1. Implement the "stash" functionality?

+1. The reasons are:

  • Both the controller and the receiver use pre-defined constant UUIDs, which might cause conflict between simultaneous access to stash.
  • Stash requires HTTP polling mechanism for asynchronous communication, which might cause impact on WPT server's CPU load and network traffic.

@louaybassbouss
Copy link
Contributor

@mfoltzgoogle just one question if we allow to create a fake presentation for testing purposes it is still the same underlying implementation of the API in Browser? we need to make sure that the behaviour of the API when presenting on real display is the same as when presenting on a fake display.

@mfoltzgoogle
Copy link
Contributor Author

@louaybassbouss The Presentation API implementation in Chrome consists of multiple layers spread across different process types. Chrome runs Web Platform Tests on the content/ layer, which (in Chrome) excludes any device-specific logic. The "fake" screen would have to be implemented in content/; so parts of the implementation of the Presentation API would be shared between "real" screens and "fake" screens. Others would not.

As part of supporting "fake" screens, we may decide to move more of the implementation into content/ to avoid having to duplicate the logic between "real" and "fake" screens. This would improve test coverage for those aspects of the implementation.

I can provide more concrete feedback once we have a proposal for the functionality of a "fake" screen.

@mfoltzgoogle
Copy link
Contributor Author

I took a quick look at the status of similar APIs that require some device mocking behavior.

  • WebUSB
    • WebUSB Testing API
    • Allows creation of fake devices and choice of a specific one to simulate user permission
  • WebVR
    • Microsoft proposed a headset simulation API, but it appears to have never landed upstream; and the WPT suite appears to be starting over from scratch for WebVR 2.0.
  • Web Bluetooth

This thread from blink-dev is also relevant to the status of these tests from May of this year. There was some more discussion and a follow-up call in earlier this year on the Test Infrastructure list, specifically about the WebUSB proposal.

@mfoltzgoogle
Copy link
Contributor Author

I think the logical next step here is to define the requirements and IDL for a testing API, similar to WebUSB. The initial scope I will look at is supporting cross-browser automation of Web Platform Tests. I'll just post it to this issue for discussion (not starting a formal document or anything.)

@mfoltzgoogle
Copy link
Contributor Author

@foolip @Honry

Here is a draft of what I am thinking to start discussion. I think they will allow conversion of the manual Presentation API tests to automated, but I may have missed something. PTAL.

https://github.com/mfoltzgoogle/presentation-test-api/blob/master/presentation_test_api.md

I am not committed to any particular implementation; there has been some discussion of implementing testing APIs on top of WebDriver, versus bindings to native implementations.

@mfoltzgoogle mfoltzgoogle added P1 v2 and removed v1 labels Nov 1, 2017
@mfoltzgoogle
Copy link
Contributor Author

As we're talking about a new API (or WebDriver) surface, I'm marking this as a v2 feature. It's also a higher priority work item for us.

@foolip
Copy link
Member

foolip commented Nov 4, 2017

@mfoltzgoogle, as of yesterday http://web-platform-tests.org/writing-tests/testdriver.html is an option as well. That requires first defining a WebDriver API, and then wrapping it in something that's ergonomic when writing tests.

The upside is that web developers will also get the capabilities. That being said, a JS API that's enabled by WebDriver is also an option, if the API is too complex to represent as a WebDriver API. @JKereliuk might be able to give further advise.

@mfoltzgoogle
Copy link
Contributor Author

From https://www.w3.org/2017/11/06-webscreens-minutes.html#x13:

  • ACTION: @mfoltzgoogle to refine the Presentation API Testing API and come up with a WebDriver compatible spec
  • ACTION: @mfoltzgoogle to file issue about whether "fake" screen is a "real" screen or can be a mock screen that responds to messages

@mfoltzgoogle mfoltzgoogle self-assigned this Nov 8, 2017
@mfoltzgoogle mfoltzgoogle removed the F2F label Mar 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants