Closed Bug 1391735 Opened 2 years ago Closed 2 years ago

annotate more of web-platform-tests moz.build files for BUGZILLA_COMPONENTS

Categories

(Testing :: web-platform-tests, enhancement)

enhancement
Not set

Tracking

(firefox57 fixed)

RESOLVED FIXED
mozilla57
Tracking Status
firefox57 --- fixed

People

(Reporter: jmaher, Assigned: jmaher)

References

Details

(Whiteboard: [PI:September])

Attachments

(2 files, 3 obsolete files)

currently we have about half of the test cases in testing/web-platform/tests/* annotated in moz.build files.  We should finish the work.

./mach file-info bugzilla-component testing/web-platform/tests/**
UNKNOWN
  testing/web-platform/tests/2dcontext
  testing/web-platform/tests/CONTRIBUTING.md
  testing/web-platform/tests/FileAPI
  testing/web-platform/tests/IndexedDB
  testing/web-platform/tests/LICENSE
  testing/web-platform/tests/LICENSE.md
  testing/web-platform/tests/README.md
  testing/web-platform/tests/WebCryptoAPI
  testing/web-platform/tests/WebIDL
  testing/web-platform/tests/XMLHttpRequest
  testing/web-platform/tests/accelerometer
  testing/web-platform/tests/ambient-light
  testing/web-platform/tests/annotation-model
  testing/web-platform/tests/annotation-protocol
  testing/web-platform/tests/annotation-vocab
  testing/web-platform/tests/apng
  testing/web-platform/tests/app-uri
  testing/web-platform/tests/assumptions
  testing/web-platform/tests/auxclick
  testing/web-platform/tests/background-fetch
  testing/web-platform/tests/battery-status
  testing/web-platform/tests/beacon
  testing/web-platform/tests/bluetooth
  testing/web-platform/tests/check_stability.ini
  testing/web-platform/tests/clear-site-data
  testing/web-platform/tests/clipboard
  testing/web-platform/tests/clipboard-apis
  testing/web-platform/tests/common
  testing/web-platform/tests/compat
  testing/web-platform/tests/config.default.json
  testing/web-platform/tests/conformance-checkers
  testing/web-platform/tests/console
  testing/web-platform/tests/content-security-policy
  testing/web-platform/tests/cookies
  testing/web-platform/tests/cors
  testing/web-platform/tests/credential-management
  testing/web-platform/tests/css
  testing/web-platform/tests/css-cascade
  testing/web-platform/tests/css-font-display
  testing/web-platform/tests/css-font-loading
  testing/web-platform/tests/css-fonts
  testing/web-platform/tests/css-paint-api
  testing/web-platform/tests/css-timing
  testing/web-platform/tests/css-typed-om
  testing/web-platform/tests/css-values
  testing/web-platform/tests/cssom
  testing/web-platform/tests/cssom-view
  testing/web-platform/tests/custom-elements
  testing/web-platform/tests/docs
  testing/web-platform/tests/dom
  testing/web-platform/tests/domparsing
  testing/web-platform/tests/domxpath
  testing/web-platform/tests/dpub-aam
  testing/web-platform/tests/dpub-aria
  testing/web-platform/tests/editing
  testing/web-platform/tests/encoding
  testing/web-platform/tests/encrypted-media
  testing/web-platform/tests/eventsource
  testing/web-platform/tests/ext-xhtml-pubid
  testing/web-platform/tests/fonts
  testing/web-platform/tests/hr-time
  testing/web-platform/tests/html
  testing/web-platform/tests/html-imports
  testing/web-platform/tests/html-longdesc
  testing/web-platform/tests/html-media-capture
  testing/web-platform/tests/http
  testing/web-platform/tests/image-decodes
  testing/web-platform/tests/imagebitmap-renderingcontext
  testing/web-platform/tests/images
  testing/web-platform/tests/infrastructure
  testing/web-platform/tests/innerText
  testing/web-platform/tests/input-events
  testing/web-platform/tests/interfaces
  testing/web-platform/tests/intersection-observer
  testing/web-platform/tests/js
  testing/web-platform/tests/keyboard-lock
  testing/web-platform/tests/lint
  testing/web-platform/tests/lint.whitelist
  testing/web-platform/tests/longtask-timing
  testing/web-platform/tests/magnetometer
  testing/web-platform/tests/manifest
  testing/web-platform/tests/mathml
  testing/web-platform/tests/media-capabilities
  testing/web-platform/tests/mediacapture-fromelement
  testing/web-platform/tests/mediacapture-image
  testing/web-platform/tests/mediasession
  testing/web-platform/tests/netinfo
  testing/web-platform/tests/old-tests
  testing/web-platform/tests/orientation-sensor
  testing/web-platform/tests/payment-method-id
  testing/web-platform/tests/remote-playback
  testing/web-platform/tests/resources
  testing/web-platform/tests/scroll-anchoring
  testing/web-platform/tests/serve
  testing/web-platform/tests/serve.py
  testing/web-platform/tests/server-side.md
  testing/web-platform/tests/server-timing
  testing/web-platform/tests/staticrange
  testing/web-platform/tests/storage
  testing/web-platform/tests/testharness_runner.html
  testing/web-platform/tests/update-built-tests.sh
  testing/web-platform/tests/viewport
  testing/web-platform/tests/wai-aria
  testing/web-platform/tests/wasm
  testing/web-platform/tests/web-nfc
  testing/web-platform/tests/web-share
  testing/web-platform/tests/webusb
  testing/web-platform/tests/wpt
  testing/web-platform/tests/wptrun
  testing/web-platform/tests/x-frame-options
I have a few questions on:
  testing/web-platform/tests/annotation-model
  testing/web-platform/tests/annotation-protocol
  testing/web-platform/tests/annotation-vocab
  testing/web-platform/tests/app-uri
  testing/web-platform/tests/assumptions
  testing/web-platform/tests/auxclick
  testing/web-platform/tests/bluetooth (firefoxOS::bluetooth ?)




:jgraham, can you help me figure out which bugzilla components these specific tests should map to?
Flags: needinfo?(james)
looking over the css related tests, I see these files in testing/web-platform/tests/ :

  testing/web-platform/tests/css
  testing/web-platform/tests/css-cascade
  testing/web-platform/tests/css-font-display
  testing/web-platform/tests/css-font-loading
  testing/web-platform/tests/css-fonts
  testing/web-platform/tests/css-paint-api
  testing/web-platform/tests/css-timing
  testing/web-platform/tests/css-typed-om
  testing/web-platform/tests/css-values
  testing/web-platform/tests/cssom
  testing/web-platform/tests/cssom-view


I assume these are split between "Core::DOM: CSS Object Model" and "Core::CSS Parsing and Comp", :jet can you help confirm or lead me in the right direction?
Flags: needinfo?(bugs)
:jgraham, two more (last request for tonight):
  testing/web-platform/tests/console
  testing/web-platform/tests/credential-management
Attached patch annotate many of the wpt tests (obsolete) — Splinter Review
asking for 3 reviews here, each person please review your area of expertise.

three are some notes on some of the edits, it should be self explanatory.  I left some commented out with TODO statements- I am not sure what bugzilla component those go into (i.e. the needinfo's on the bug)

When this lands, there are 25 remaining annotations to complete wpt tests:
$ ./mach file-info missing-bugzilla testing/web-platform/tests/**
testing/web-platform/tests/FileAPI
testing/web-platform/tests/annotation-model
testing/web-platform/tests/annotation-protocol
testing/web-platform/tests/annotation-vocab
testing/web-platform/tests/app-uri
testing/web-platform/tests/console
testing/web-platform/tests/custom-elements
testing/web-platform/tests/docs
testing/web-platform/tests/dom
testing/web-platform/tests/domparsing
testing/web-platform/tests/domxpath
testing/web-platform/tests/dpub-aam
testing/web-platform/tests/dpub-aria
testing/web-platform/tests/editing
testing/web-platform/tests/encoding
testing/web-platform/tests/encrypted-media
testing/web-platform/tests/eventsource
testing/web-platform/tests/ext-xhtml-pubid
testing/web-platform/tests/fonts
testing/web-platform/tests/html
testing/web-platform/tests/image-decodes
testing/web-platform/tests/keyboard-lock
testing/web-platform/tests/old-tests
testing/web-platform/tests/payment-method-id
testing/web-platform/tests/staticrange
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Flags: needinfo?(james)
Flags: needinfo?(bugs)
Attachment #8902764 - Flags: review?(overholt)
Attachment #8902764 - Flags: review?(james)
Attachment #8902764 - Flags: review?(bugs)
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #3)
> :jgraham, two more (last request for tonight):
>   testing/web-platform/tests/console

DOM

>   testing/web-platform/tests/credential-management

We haven't implemented this yet but I guess it'd be DOM: Security based on bug 1156047.

(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #1)
>   testing/web-platform/tests/bluetooth (firefoxOS::bluetooth ?)

DOM: Device Interfaces (not that we have plans to implement the Bluetooth API)
Comment on attachment 8902764 [details] [diff] [review]
annotate many of the wpt tests

Review of attachment 8902764 [details] [diff] [review]:
-----------------------------------------------------------------

I made some suggested change but most of the DOM-ish pieces look pretty good! Thanks.

::: testing/web-platform/moz.build
@@ +72,5 @@
> +with Files("tests/WebIDL/**"):
> +    BUG_COMPONENT = ("Core", "DOM")
> +
> +with Files("tests/XMLHttpRequest/**"):
> +    BUG_COMPONENT = ("Core", "XML")

Core: DOM

@@ +75,5 @@
> +with Files("tests/XMLHttpRequest/**"):
> +    BUG_COMPONENT = ("Core", "XML")
> +
> +with Files("tests/accelerometer/**"):
> +    BUG_COMPONENT = ("Core", "DOM")

DOM: Device Interfaces

@@ +99,5 @@
> +with Files("tests/assumptions/**"):
> +    BUG_COMPONENT = ("Testing", "web-platform-tests")
> +
> +with Files("tests/auxclick/**"):
> +    BUG_COMPONENT = ("Core", "Event Handling")

Probably more DOM: Events

@@ +112,5 @@
> +    BUG_COMPONENT = ("Core", "DOM")
> +
> +#TODO: this is not a managed product anymore.
> +with Files("tests/bluetooth/**"):
> +    BUG_COMPONENT = ("Firefox OS", "Bluetooth")

DOM: Device Interfaces

@@ +118,5 @@
> +with Files("tests/clear-site-data/**"):
> +    BUG_COMPONENT = ("Core", "DOM: Security")
> +
> +with Files("tests/clipboard/**"):
> +    BUG_COMPONENT = ("Core", "Widget")

Probably DOM (note that we don't have an implementation of the nascent Clipboard API)

@@ +121,5 @@
> +with Files("tests/clipboard/**"):
> +    BUG_COMPONENT = ("Core", "Widget")
> +
> +with Files("tests/clipboard-apis/**"):
> +    BUG_COMPONENT = ("Core", "Widget")

Oh, maybe *this* should be DOM and clipboard/** should be elsewhere?

@@ +135,5 @@
> +    BUG_COMPONENT = ("Testing", "web-platform-tests")
> +
> +#TODO
> +#with Files("tests/console/**"):
> +#    BUG_COMPONENT = ("", "")

Developer Tools: Console

@@ +231,5 @@
> +    BUG_COMPONENT = ("Core", "DOM")
> +
> +# no mention in bugzilla or source tree about .decode()
> +#with Files("tests/image-decodes/**"):
> +#    BUG_COMPONENT = ("", "")

We haven't done this yet. Imagelib?

@@ +263,5 @@
> +# No reference to requestKeyboardLock in bugzilla or codebase
> +#with Files("tests/keyboard-lock/**"):
> +#    BUG_COMPONENT = ("", "")
> +
> +# No reference to longtask in bugzilla or codebase

It's bug 1348405

@@ +265,5 @@
> +#    BUG_COMPONENT = ("", "")
> +
> +# No reference to longtask in bugzilla or codebase
> +with Files("tests/longtask-timing/**"):
> +    BUG_COMPONENT = ("Core", "DOM")

Correct

@@ +372,5 @@
>      BUG_COMPONENT = ("Core", "DOM")
>  
> +# No tests in here
> +#with Files("tests/payment-method-id/**"):
> +#    BUG_COMPONENT = ("Core", "DOM")

DOM: Web Payments

@@ +505,5 @@
>  with Files("tests/webmessaging/**"):
>      BUG_COMPONENT = ("Core", "DOM")
>  
> +with Files("tests/web-nfc/**"):
> +    BUG_COMPONENT = ("Firefox OS", "NFC")

DOM: Device Interfaces is better despite us not shipping this

@@ +520,5 @@
>  with Files("tests/webstorage/**"):
>      BUG_COMPONENT = ("Core", "DOM")
>  
> +with Files("tests/webusb/**"):
> +    BUG_COMPONENT = ("Testing", "web-platform-tests")

You could say DOM: Device Interfaces
Attachment #8902764 - Flags: review?(overholt) → review+
(In reply to Andrew Overholt [:overholt] from comment #5)
> (In reply to Joel Maher ( :jmaher) (UTC-5) from comment #3)
> > :jgraham, two more (last request for tonight):
> >   testing/web-platform/tests/console
> 
> DOM

I was wrong. It should be Developer Tools: Console.
Attached patch annotate many of the wpt tests (obsolete) — Splinter Review
thanks for the review :overholt, updated the patch with all your recommended changes.
Attachment #8902764 - Attachment is obsolete: true
Attachment #8902764 - Flags: review?(james)
Attachment #8902764 - Flags: review?(bugs)
Attachment #8902785 - Flags: review?(james)
Attachment #8902785 - Flags: review?(bugs)
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #2)
> :jet can you help confirm or lead me in the
> right direction?

"Core::Layout"
>   testing/web-platform/tests/css
>   testing/web-platform/tests/css-timing

"Core::Layout: Text"
>   testing/web-platform/tests/css-font-display
>   testing/web-platform/tests/css-font-loading
>   testing/web-platform/tests/css-fonts

"Core::Layout: Web Painting"
>   testing/web-platform/tests/css-paint-api

"Core::CSS Parsing and Comp"
>   testing/web-platform/tests/css-cascade

"Core::DOM: CSS Object Model"
>   testing/web-platform/tests/css-typed-om
>   testing/web-platform/tests/css-values
>   testing/web-platform/tests/cssom
>   testing/web-platform/tests/cssom-view
Attached patch annotate many of the wpt tests (obsolete) — Splinter Review
Thanks :jet for the comment!  I have updated the patch for css related tests
Attachment #8902785 - Attachment is obsolete: true
Attachment #8902785 - Flags: review?(james)
Attachment #8902785 - Flags: review?(bugs)
Attachment #8902806 - Flags: review?(james)
Attachment #8902806 - Flags: review?(bugs)
Attachment #8902806 - Flags: review?(bugs) → review+
Comment on attachment 8902806 [details] [diff] [review]
annotate many of the wpt tests

Review of attachment 8902806 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/web-platform/moz.build
@@ +40,5 @@
> +
> +with Files("tests/LICENSE"):
> +    BUG_COMPONENT = ("Testing", "web-platform-tests")
> +
> +with Files("tests/check_stability.ini"):

It would be much more useful if we could annotate files in the top level directory as all being Testing::web-platform-tests. This setup is incredibly fragile and will lead to build bustage.

@@ +116,5 @@
> +
> +with Files("tests/clear-site-data/**"):
> +    BUG_COMPONENT = ("Core", "DOM: Security")
> +
> +with Files("tests/clipboard/**"):

Pretty sure this should be the same as tests/clipboard-apis/. The fact that there's two different directories there seems like a bug, and this one is basically empty.

@@ +147,5 @@
> +
> +with Files("tests/credential-management/**"):
> +    BUG_COMPONENT = ("Core", "DOM: Security")
> +
> +with Files("tests/css/**"):

It seems unlikely that all CSS tests should be in Core::Layout rather than more specific things for each feature, but this is OK for a first pass.

@@ +212,5 @@
> +    BUG_COMPONENT = ("Core", "DOM")
> +
> +# TODO
> +#with Files("tests/html/**"):
> +#    BUG_COMPONENT = ("", "")

Core :: DOM would be OK for a first pass, but it certainly wouldn't be right always.

@@ +248,5 @@
> +with Files("tests/input-events/**"):
> +    BUG_COMPONENT = ("Core", "General")
> +
> +with Files("tests/interfaces/**"):
> +    BUG_COMPONENT = ("Testing", "web-platform-tests")

Core :: DOM

@@ +356,5 @@
> +    BUG_COMPONENT = ("Testing", "web-platform-tests")
> +
> +# TODO
> +with Files("tests/old-tests/webdriver/**"):
> +    BUG_COMPONENT = ("Testing", "web-platform-tests")

Testing :: geckodriver

@@ +434,5 @@
>  
>  with Files("tests/shadow-dom/**"):
>      BUG_COMPONENT = ("Core", "DOM")
>  
> +# No test files in here

This comment seems unlikely to remain true, and is confusing since you also annotated many other things that aren't test files.

@@ +528,5 @@
>  
>  with Files("tests/workers/**"):
>      BUG_COMPONENT = ("Core", "DOM: Workers")
> +
> +with Files("tests/wpt/**"):

Hmm, that isn't a directory. Do you mean tests/wpt*
Attachment #8902806 - Flags: review?(james) → review+
updated patch with :jgraham's suggestions :)

Thanks for the reviews!

as for the single files in testing/web-platform/tests/*, I am not sure how to capture those and ignore the subdirs, I would prefer to do that.  My goal is to not have a new directory show up and get a blanket bugzilla component.

thanks for pointing out css/* and html/*, I will work on making those more complete in another pass, but as a starting point those are good places.
Attachment #8902806 - Attachment is obsolete: true
Attachment #8903137 - Flags: review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cf2763028159
annotate more of web-platform-tests moz.build. r=jgraham,jet,overholt
https://hg.mozilla.org/mozilla-central/rev/cf2763028159
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
we still have ~20 directories to annotate, I will work on another patch today/next week
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Whiteboard: [PI:September]
some blanket annotations:
tests/css/
tests/html/
tests/dom/

overall, I think these are 90% correct
Attachment #8903747 - Flags: review?(james)
Comment on attachment 8903747 [details] [diff] [review]
All of the web-platform/tests/* will be annotated with this

Review of attachment 8903747 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/web-platform/moz.build
@@ +230,5 @@
>  
>  with Files("tests/fetch/**"):
>      BUG_COMPONENT = ("Core", "DOM")
>  
> +with Files("tests/fonts/**"):

Tesing :: web-platform-tests

These are font files, not actual tests.
Attachment #8903747 - Flags: review?(james) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b4f0f1294959
Annotate the rest of the web-platform-tests. r=jgraham
https://hg.mozilla.org/mozilla-central/rev/b4f0f1294959
Status: REOPENED → RESOLVED
Closed: 2 years ago2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.