989448 - (compat-data) Compatibility Data tracking bug

Status: RESOLVED WONTFIX

(developer.mozilla.org Graveyard :: BrowserCompat, enhancement)

Description

[Jeremie Patonnier :Jeremie]

This is the tracking bug for the compatibility data project.

Comment 1

[Jeremie Patonnier :Jeremie]

Technical Kick-off meetings has been done, here's the note: https://etherpad.mozilla.org/cdp-tech-kick-off-2014-03-28

Comment 2

[Jeremie Patonnier :Jeremie]

Technical Kick-off meetings has been done, here's the note:
https://etherpad.mozilla.org/cd-ux-kick-off

Comment 3

[Robin Berjon]

I have some information that you might find useful. I think there's potential room for interesting collaboration in here.

As you may or may not know, a bunch of us have been working on a project called "Web Platform Tests" (https://github.com/w3c/web-platform-tests/). The goal is to have a single one-stop shop for the testing of all (standardised) Web technologies. We're not completely there yet (notably CSS, JS, and WebGL are tested elsewhere) but we nevertheless have a lot of tests, that have gone through quality review, for HTML, the DOM, and a bunch of other APIs and things.

James Graham has been developing a tool (wptrunner) that can automatically run all of those tests (or nearly all) against a browser engine. I know he's integrated that with Servo and it is now running as part of the Gecko CI tests.

One thing that is cool with wptrunner is that it uses WebDriver. Another thing that uses WebDriver is SauceLabs (where we have an account). Putting these two things together means that we ought to be getting raw result data for a lot of tests on a *lot* of browsers and device configurations.

Parallel to this, there is ongoing work to vastly improve the links between individual test results, what they are testing in specifications, and "features" (the granularity that usually makes sense when developing). We have part of the infrastructure for this, but it's probably at least a few months away. We could certainly use your input about what type of granularity would make sense here. (We're also digging around for funds to make the guy working on this join us face to face for some high-bandwidth hacking later this year — if you have suggestions for that we'd love to hear.)

All in all I reckon that the data we produce with this could be very useful for this project. We'd certainly like to hear your ideas about how we could make the data more useful to you, the idea being that we want it to be very reusable by third-parties.

Comment 4

[Luke Crouch [:groovecoder]]

That sounds really promising. :hoosteeno - want to follow up with this?

Comment 5

[Jeremie Patonnier :Jeremie]

(In reply to Robin Berjon from comment #3)
> As you may or may not know, a bunch of us have been working on a project
> called "Web Platform Tests" (https://github.com/w3c/web-platform-tests/).

Yup, I'm aware of this :)
We listed this as one of the tests suit we are considering to fill our data base
https://wiki.mozilla.org/MDN/Projects/Development/CompatibilityTables#tests_suite

> James Graham has been developing a tool (wptrunner) that can automatically
> run all of those tests (or nearly all) against a browser engine. I know he's
> integrated that with Servo and it is now running as part of the Gecko CI
> tests.

That's great, as it means that we should be able to use it quite easily to get raw Firefox compatibility information

> One thing that is cool with wptrunner is that it uses WebDriver. Another
> thing that uses WebDriver is SauceLabs (where we have an account).

Oh! I wasn't aware of that, Selenium FTW \o/

> Parallel to this, there is ongoing work to vastly improve the links between
> individual test results, what they are testing in specifications, and
> "features" (the granularity that usually makes sense when developing). We
> have part of the infrastructure for this, but it's probably at least a few
> months away. We could certainly use your input about what type of
> granularity would make sense here. (We're also digging around for funds to
> make the guy working on this join us face to face for some high-bandwidth
> hacking later this year — if you have suggestions for that we'd love to
> hear.)

As first though, I suggest you or W3C people involved in this project take a look at:
https://wiki.mozilla.org/MDN/Development/CompatibilityTables/Data_Requirements

Then, as :groovecoder suggest it, :hoosteeno could follow up with this (Justin, tell me if you need some help on this)

> All in all I reckon that the data we produce with this could be very useful
> for this project. We'd certainly like to hear your ideas about how we could
> make the data more useful to you, the idea being that we want it to be very
> reusable by third-parties.

:groovecoder and John Whitlock are working on the API to access our data base, I guess that I some point it will be interesting to get feedback from James Graham about it.

Comment 6

[Justin Crawford [:hoosteeno] [:jcrawford]]

(In reply to Luke Crouch [:groovecoder] from comment #4)
> That sounds really promising. :hoosteeno - want to follow up with this?

Yes! I'll reach out to Jeremie & Robin for a chat.

Comment 8

[Jeremie Patonnier :Jeremie]

Project milestones: https://wiki.mozilla.org/MDN/Projects/Development/CompatibilityTables/Milestones

Comment 9

[John Whitlock [:jwhitlock]]

The BrowserCompat project is cancelled.  See https://github.com/mdn/browser-compat-data for the current effort.