Closed Bug 1391735 Opened 7 years ago Closed 7 years ago

annotate more of web-platform-tests moz.build files for BUGZILLA_COMPONENTS

Categories

(Testing :: web-platform-tests, enhancement)

Product:
Testing
Component:
web-platform-tests
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox57 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla57
Tracking Flags:
Tracking Status
firefox57 --- fixed

People

(Reporter: jmaher, Assigned: jmaher)

References

Details

(Whiteboard: [PI:September])

Attachments

(2 files, 3 obsolete files)

annotate many of the wpt tests
13.75 KB, patch
jmaher
: review+
Details | Diff | Splinter Review
All of the web-platform/tests/* will be annotated with this
4.36 KB, patch
jgraham
: review+
Details | Diff | Splinter Review
currently we have about half of the test cases in testing/web-platform/tests/* annotated in moz.build files. We should finish the work. ./mach file-info bugzilla-component testing/web-platform/tests/** UNKNOWN testing/web-platform/tests/2dcontext testing/web-platform/tests/CONTRIBUTING.md testing/web-platform/tests/FileAPI testing/web-platform/tests/IndexedDB testing/web-platform/tests/LICENSE testing/web-platform/tests/LICENSE.md testing/web-platform/tests/README.md testing/web-platform/tests/WebCryptoAPI testing/web-platform/tests/WebIDL testing/web-platform/tests/XMLHttpRequest testing/web-platform/tests/accelerometer testing/web-platform/tests/ambient-light testing/web-platform/tests/annotation-model testing/web-platform/tests/annotation-protocol testing/web-platform/tests/annotation-vocab testing/web-platform/tests/apng testing/web-platform/tests/app-uri testing/web-platform/tests/assumptions testing/web-platform/tests/auxclick testing/web-platform/tests/background-fetch testing/web-platform/tests/battery-status testing/web-platform/tests/beacon testing/web-platform/tests/bluetooth testing/web-platform/tests/check_stability.ini testing/web-platform/tests/clear-site-data testing/web-platform/tests/clipboard testing/web-platform/tests/clipboard-apis testing/web-platform/tests/common testing/web-platform/tests/compat testing/web-platform/tests/config.default.json testing/web-platform/tests/conformance-checkers testing/web-platform/tests/console testing/web-platform/tests/content-security-policy testing/web-platform/tests/cookies testing/web-platform/tests/cors testing/web-platform/tests/credential-management testing/web-platform/tests/css testing/web-platform/tests/css-cascade testing/web-platform/tests/css-font-display testing/web-platform/tests/css-font-loading testing/web-platform/tests/css-fonts testing/web-platform/tests/css-paint-api testing/web-platform/tests/css-timing testing/web-platform/tests/css-typed-om testing/web-platform/tests/css-values testing/web-platform/tests/cssom testing/web-platform/tests/cssom-view testing/web-platform/tests/custom-elements testing/web-platform/tests/docs testing/web-platform/tests/dom testing/web-platform/tests/domparsing testing/web-platform/tests/domxpath testing/web-platform/tests/dpub-aam testing/web-platform/tests/dpub-aria testing/web-platform/tests/editing testing/web-platform/tests/encoding testing/web-platform/tests/encrypted-media testing/web-platform/tests/eventsource testing/web-platform/tests/ext-xhtml-pubid testing/web-platform/tests/fonts testing/web-platform/tests/hr-time testing/web-platform/tests/html testing/web-platform/tests/html-imports testing/web-platform/tests/html-longdesc testing/web-platform/tests/html-media-capture testing/web-platform/tests/http testing/web-platform/tests/image-decodes testing/web-platform/tests/imagebitmap-renderingcontext testing/web-platform/tests/images testing/web-platform/tests/infrastructure testing/web-platform/tests/innerText testing/web-platform/tests/input-events testing/web-platform/tests/interfaces testing/web-platform/tests/intersection-observer testing/web-platform/tests/js testing/web-platform/tests/keyboard-lock testing/web-platform/tests/lint testing/web-platform/tests/lint.whitelist testing/web-platform/tests/longtask-timing testing/web-platform/tests/magnetometer testing/web-platform/tests/manifest testing/web-platform/tests/mathml testing/web-platform/tests/media-capabilities testing/web-platform/tests/mediacapture-fromelement testing/web-platform/tests/mediacapture-image testing/web-platform/tests/mediasession testing/web-platform/tests/netinfo testing/web-platform/tests/old-tests testing/web-platform/tests/orientation-sensor testing/web-platform/tests/payment-method-id testing/web-platform/tests/remote-playback testing/web-platform/tests/resources testing/web-platform/tests/scroll-anchoring testing/web-platform/tests/serve testing/web-platform/tests/serve.py testing/web-platform/tests/server-side.md testing/web-platform/tests/server-timing testing/web-platform/tests/staticrange testing/web-platform/tests/storage testing/web-platform/tests/testharness_runner.html testing/web-platform/tests/update-built-tests.sh testing/web-platform/tests/viewport testing/web-platform/tests/wai-aria testing/web-platform/tests/wasm testing/web-platform/tests/web-nfc testing/web-platform/tests/web-share testing/web-platform/tests/webusb testing/web-platform/tests/wpt testing/web-platform/tests/wptrun testing/web-platform/tests/x-frame-options
I have a few questions on: testing/web-platform/tests/annotation-model testing/web-platform/tests/annotation-protocol testing/web-platform/tests/annotation-vocab testing/web-platform/tests/app-uri testing/web-platform/tests/assumptions testing/web-platform/tests/auxclick testing/web-platform/tests/bluetooth (firefoxOS::bluetooth ?) :jgraham, can you help me figure out which bugzilla components these specific tests should map to?
Flags: needinfo?(james)
looking over the css related tests, I see these files in testing/web-platform/tests/ : testing/web-platform/tests/css testing/web-platform/tests/css-cascade testing/web-platform/tests/css-font-display testing/web-platform/tests/css-font-loading testing/web-platform/tests/css-fonts testing/web-platform/tests/css-paint-api testing/web-platform/tests/css-timing testing/web-platform/tests/css-typed-om testing/web-platform/tests/css-values testing/web-platform/tests/cssom testing/web-platform/tests/cssom-view I assume these are split between "Core::DOM: CSS Object Model" and "Core::CSS Parsing and Comp", :jet can you help confirm or lead me in the right direction?
Flags: needinfo?(bugs)
:jgraham, two more (last request for tonight): testing/web-platform/tests/console testing/web-platform/tests/credential-management
Attached patch annotate many of the wpt tests (obsolete) — — Splinter Review
asking for 3 reviews here, each person please review your area of expertise. three are some notes on some of the edits, it should be self explanatory. I left some commented out with TODO statements- I am not sure what bugzilla component those go into (i.e. the needinfo's on the bug) When this lands, there are 25 remaining annotations to complete wpt tests: $ ./mach file-info missing-bugzilla testing/web-platform/tests/** testing/web-platform/tests/FileAPI testing/web-platform/tests/annotation-model testing/web-platform/tests/annotation-protocol testing/web-platform/tests/annotation-vocab testing/web-platform/tests/app-uri testing/web-platform/tests/console testing/web-platform/tests/custom-elements testing/web-platform/tests/docs testing/web-platform/tests/dom testing/web-platform/tests/domparsing testing/web-platform/tests/domxpath testing/web-platform/tests/dpub-aam testing/web-platform/tests/dpub-aria testing/web-platform/tests/editing testing/web-platform/tests/encoding testing/web-platform/tests/encrypted-media testing/web-platform/tests/eventsource testing/web-platform/tests/ext-xhtml-pubid testing/web-platform/tests/fonts testing/web-platform/tests/html testing/web-platform/tests/image-decodes testing/web-platform/tests/keyboard-lock testing/web-platform/tests/old-tests testing/web-platform/tests/payment-method-id testing/web-platform/tests/staticrange
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Flags: needinfo?(james)
Flags: needinfo?(bugs)
Attachment #8902764 - Flags: review?(overholt)
Attachment #8902764 - Flags: review?(james)
Attachment #8902764 - Flags: review?(bugs)
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #3) > :jgraham, two more (last request for tonight): > testing/web-platform/tests/console DOM > testing/web-platform/tests/credential-management We haven't implemented this yet but I guess it'd be DOM: Security based on bug 1156047. (In reply to Joel Maher ( :jmaher) (UTC-5) from comment #1) > testing/web-platform/tests/bluetooth (firefoxOS::bluetooth ?) DOM: Device Interfaces (not that we have plans to implement the Bluetooth API)
Comment on attachment 8902764 [details] [diff] [review] annotate many of the wpt tests Review of attachment 8902764 [details] [diff] [review]: ----------------------------------------------------------------- I made some suggested change but most of the DOM-ish pieces look pretty good! Thanks. ::: testing/web-platform/moz.build @@ +72,5 @@ > +with Files("tests/WebIDL/**"): > + BUG_COMPONENT = ("Core", "DOM") > + > +with Files("tests/XMLHttpRequest/**"): > + BUG_COMPONENT = ("Core", "XML") Core: DOM @@ +75,5 @@ > +with Files("tests/XMLHttpRequest/**"): > + BUG_COMPONENT = ("Core", "XML") > + > +with Files("tests/accelerometer/**"): > + BUG_COMPONENT = ("Core", "DOM") DOM: Device Interfaces @@ +99,5 @@ > +with Files("tests/assumptions/**"): > + BUG_COMPONENT = ("Testing", "web-platform-tests") > + > +with Files("tests/auxclick/**"): > + BUG_COMPONENT = ("Core", "Event Handling") Probably more DOM: Events @@ +112,5 @@ > + BUG_COMPONENT = ("Core", "DOM") > + > +#TODO: this is not a managed product anymore. > +with Files("tests/bluetooth/**"): > + BUG_COMPONENT = ("Firefox OS", "Bluetooth") DOM: Device Interfaces @@ +118,5 @@ > +with Files("tests/clear-site-data/**"): > + BUG_COMPONENT = ("Core", "DOM: Security") > + > +with Files("tests/clipboard/**"): > + BUG_COMPONENT = ("Core", "Widget") Probably DOM (note that we don't have an implementation of the nascent Clipboard API) @@ +121,5 @@ > +with Files("tests/clipboard/**"): > + BUG_COMPONENT = ("Core", "Widget") > + > +with Files("tests/clipboard-apis/**"): > + BUG_COMPONENT = ("Core", "Widget") Oh, maybe *this* should be DOM and clipboard/** should be elsewhere? @@ +135,5 @@ > + BUG_COMPONENT = ("Testing", "web-platform-tests") > + > +#TODO > +#with Files("tests/console/**"): > +# BUG_COMPONENT = ("", "") Developer Tools: Console @@ +231,5 @@ > + BUG_COMPONENT = ("Core", "DOM") > + > +# no mention in bugzilla or source tree about .decode() > +#with Files("tests/image-decodes/**"): > +# BUG_COMPONENT = ("", "") We haven't done this yet. Imagelib? @@ +263,5 @@ > +# No reference to requestKeyboardLock in bugzilla or codebase > +#with Files("tests/keyboard-lock/**"): > +# BUG_COMPONENT = ("", "") > + > +# No reference to longtask in bugzilla or codebase It's bug 1348405 @@ +265,5 @@ > +# BUG_COMPONENT = ("", "") > + > +# No reference to longtask in bugzilla or codebase > +with Files("tests/longtask-timing/**"): > + BUG_COMPONENT = ("Core", "DOM") Correct @@ +372,5 @@ > BUG_COMPONENT = ("Core", "DOM") > > +# No tests in here > +#with Files("tests/payment-method-id/**"): > +# BUG_COMPONENT = ("Core", "DOM") DOM: Web Payments @@ +505,5 @@ > with Files("tests/webmessaging/**"): > BUG_COMPONENT = ("Core", "DOM") > > +with Files("tests/web-nfc/**"): > + BUG_COMPONENT = ("Firefox OS", "NFC") DOM: Device Interfaces is better despite us not shipping this @@ +520,5 @@ > with Files("tests/webstorage/**"): > BUG_COMPONENT = ("Core", "DOM") > > +with Files("tests/webusb/**"): > + BUG_COMPONENT = ("Testing", "web-platform-tests") You could say DOM: Device Interfaces
Attachment #8902764 - Flags: review?(overholt) → review+
(In reply to Andrew Overholt [:overholt] from comment #5) > (In reply to Joel Maher ( :jmaher) (UTC-5) from comment #3) > > :jgraham, two more (last request for tonight): > > testing/web-platform/tests/console > > DOM I was wrong. It should be Developer Tools: Console.
Attached patch annotate many of the wpt tests (obsolete) — — Splinter Review
thanks for the review :overholt, updated the patch with all your recommended changes.
Attachment #8902764 - Attachment is obsolete: true
Attachment #8902764 - Flags: review?(james)
Attachment #8902764 - Flags: review?(bugs)
Attachment #8902785 - Flags: review?(james)
Attachment #8902785 - Flags: review?(bugs)
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #2) > :jet can you help confirm or lead me in the > right direction? "Core::Layout" > testing/web-platform/tests/css > testing/web-platform/tests/css-timing "Core::Layout: Text" > testing/web-platform/tests/css-font-display > testing/web-platform/tests/css-font-loading > testing/web-platform/tests/css-fonts "Core::Layout: Web Painting" > testing/web-platform/tests/css-paint-api "Core::CSS Parsing and Comp" > testing/web-platform/tests/css-cascade "Core::DOM: CSS Object Model" > testing/web-platform/tests/css-typed-om > testing/web-platform/tests/css-values > testing/web-platform/tests/cssom > testing/web-platform/tests/cssom-view
Attached patch annotate many of the wpt tests (obsolete) — — Splinter Review
Thanks :jet for the comment! I have updated the patch for css related tests
Attachment #8902785 - Attachment is obsolete: true
Attachment #8902785 - Flags: review?(james)
Attachment #8902785 - Flags: review?(bugs)
Attachment #8902806 - Flags: review?(james)
Attachment #8902806 - Flags: review?(bugs)
Attachment #8902806 - Flags: review?(bugs) → review+
Comment on attachment 8902806 [details] [diff] [review] annotate many of the wpt tests Review of attachment 8902806 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/web-platform/moz.build @@ +40,5 @@ > + > +with Files("tests/LICENSE"): > + BUG_COMPONENT = ("Testing", "web-platform-tests") > + > +with Files("tests/check_stability.ini"): It would be much more useful if we could annotate files in the top level directory as all being Testing::web-platform-tests. This setup is incredibly fragile and will lead to build bustage. @@ +116,5 @@ > + > +with Files("tests/clear-site-data/**"): > + BUG_COMPONENT = ("Core", "DOM: Security") > + > +with Files("tests/clipboard/**"): Pretty sure this should be the same as tests/clipboard-apis/. The fact that there's two different directories there seems like a bug, and this one is basically empty. @@ +147,5 @@ > + > +with Files("tests/credential-management/**"): > + BUG_COMPONENT = ("Core", "DOM: Security") > + > +with Files("tests/css/**"): It seems unlikely that all CSS tests should be in Core::Layout rather than more specific things for each feature, but this is OK for a first pass. @@ +212,5 @@ > + BUG_COMPONENT = ("Core", "DOM") > + > +# TODO > +#with Files("tests/html/**"): > +# BUG_COMPONENT = ("", "") Core :: DOM would be OK for a first pass, but it certainly wouldn't be right always. @@ +248,5 @@ > +with Files("tests/input-events/**"): > + BUG_COMPONENT = ("Core", "General") > + > +with Files("tests/interfaces/**"): > + BUG_COMPONENT = ("Testing", "web-platform-tests") Core :: DOM @@ +356,5 @@ > + BUG_COMPONENT = ("Testing", "web-platform-tests") > + > +# TODO > +with Files("tests/old-tests/webdriver/**"): > + BUG_COMPONENT = ("Testing", "web-platform-tests") Testing :: geckodriver @@ +434,5 @@ > > with Files("tests/shadow-dom/**"): > BUG_COMPONENT = ("Core", "DOM") > > +# No test files in here This comment seems unlikely to remain true, and is confusing since you also annotated many other things that aren't test files. @@ +528,5 @@ > > with Files("tests/workers/**"): > BUG_COMPONENT = ("Core", "DOM: Workers") > + > +with Files("tests/wpt/**"): Hmm, that isn't a directory. Do you mean tests/wpt*
Attachment #8902806 - Flags: review?(james) → review+
updated patch with :jgraham's suggestions :) Thanks for the reviews! as for the single files in testing/web-platform/tests/*, I am not sure how to capture those and ignore the subdirs, I would prefer to do that. My goal is to not have a new directory show up and get a blanket bugzilla component. thanks for pointing out css/* and html/*, I will work on making those more complete in another pass, but as a starting point those are good places.
Attachment #8902806 - Attachment is obsolete: true
Attachment #8903137 - Flags: review+
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/cf2763028159 annotate more of web-platform-tests moz.build. r=jgraham,jet,overholt
https://hg.mozilla.org/mozilla-central/rev/cf2763028159
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
status-firefox57: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
we still have ~20 directories to annotate, I will work on another patch today/next week
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Whiteboard: [PI:September]
some blanket annotations: tests/css/ tests/html/ tests/dom/ overall, I think these are 90% correct
Attachment #8903747 - Flags: review?(james)
Comment on attachment 8903747 [details] [diff] [review] All of the web-platform/tests/* will be annotated with this Review of attachment 8903747 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/web-platform/moz.build @@ +230,5 @@ > > with Files("tests/fetch/**"): > BUG_COMPONENT = ("Core", "DOM") > > +with Files("tests/fonts/**"): Tesing :: web-platform-tests These are font files, not actual tests.
Attachment #8903747 - Flags: review?(james) → review+
Pushed by jmaher@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/b4f0f1294959 Annotate the rest of the web-platform-tests. r=jgraham
https://hg.mozilla.org/mozilla-central/rev/b4f0f1294959
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Resolution: --- → FIXED
Blocks: 1481831
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: