1771587 - Collect ESM-ification progress

Status: RESOLVED FIXED

(Core :: XPConnect, task)

Description

[Tooru Fujisawa [:arai]]

To track the progress of ESM-ification, count the number of remaining JSM files and already-ESM-ified files

Comment 1

[Tooru Fujisawa [:arai]]

Attachment: areweesmifiedyet.py

This script collects status and prints JSON file.
jsm and esm property are the number of files inside the directory and sub-directories.

{
  "jsm": 1335,
  "esm": 7,
  "subdir": {
    "browser": {
      "jsm": 340,
      "esm": 0,
      "subdir": {
        "actors": {
          "jsm": 56,
          "esm": 0
        },
        "base": {
          "jsm": 3,
          "esm": 0,
...

Comment 2

[Brian Grinstead [:bgrins]]

I forgot how we've typically wired this type of thing up to CI to make sure it's running automatically so that we can i.e. fetch the value from a given day and build a chart to show progress. I believe this can be done with taskcluster artifacts and running the script on a cron but I'm having trouble finding a good example. Marco or Andrew, do you have any thoughts on how we might do that?

Comment 3

[Tooru Fujisawa [:arai]]

some TODO for the script:

• perhaps split the tree/count into test and non-test
• exclude tests that's not going to be ESM-ified (tests for JSM loader)

Comment 4

[Tooru Fujisawa [:arai]]

Attachment: areweesmifiedyet.py

Added excluded files list.

Comment 5

[Marco Castelluccio [:marco]]

(In reply to Brian Grinstead [:bgrins] from comment #2)

I forgot how we've typically wired this type of thing up to CI to make sure it's running automatically so that we can i.e. fetch the value from a given day and build a chart to show progress. I believe this can be done with taskcluster artifacts and running the script on a cron but I'm having trouble finding a good example. Marco or Andrew, do you have any thoughts on how we might do that?

I can think of two options:

• add it in the bugbug cron hook (https://github.com/mozilla/bugbug/blob/master/infra/landings-pipeline.yml), which also generates the dashboards at https://changes.moz.tools/team.html?grouping=monthly&teams=DOM;
• create a new cron hook in m-c, adding it to the list we have at https://searchfox.org/mozilla-central/source/.cron.yml. There are some docs at https://searchfox.org/mozilla-central/source/taskcluster/docs/cron.rst which are not super comprehensive as far as I can see, but you can use one of the items in the list in the .cron.yml to figure out how it works.

Comment 6

[Andrew Halberstadt [:ahal]]

I don't know anything about the bugbug approach, but can help with setting up a normal cron task if needed. But as Marco mentioned, you should be able to copy how other cron tasks are implemented.

Comment 7

[Brian Grinstead [:bgrins]]

Thanks Marco and Andrew! I'd say if it's easy enough to run some arbitrary python and save an artifact from a cron in-tree that's probably where I'd start (because then you could i.e. edit the script later on if you want to update it in parallel with changes in m-c). But bugbug's definitely a good option too if cron isn't a good fit for whatever reason.

Comment 8

[Yulia Startsev [:yulia]]

Attachment: Bug 1771587 - Implement are-we-esmified-yet check on ci; r=marco!,arai!

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7d6ca1460d77
Implement are-we-esmified-yet check on ci; r=marco,jmaher

Comment 10

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/7d6ca1460d77