Closed Bug 1637463 Opened 3 months ago Closed 2 months ago

Split writeruntimes data by suite

Categories

(Firefox Build System :: Task Configuration, task)

task

Tracking

(firefox78 fixed)

RESOLVED FIXED
mozilla78
Tracking Status
firefox78 --- fixed

People

(Reporter: egao, Assigned: egao)

References

(Regressed 1 open bug)

Details

Attachments

(4 files)

Currently, all results from the writeruntimes script gets dumped into one big dictionary. One file is generated per platform, of which there are three:

['unix', 'win', 'android']

There are 24 distinct 'suites' reported as defined by testing/mozbase/moztest/moztest/resolve.py.

With the addition of web-platform-tests and its associated subsuites into the mix, this introduces two problems:

  1. dictionary is getting large, which is not a reason to refactor in itself, but more importantly;
  2. web-platform-tests and its subsuites share the same directory structure, often with reftests and vanilla web-platform-tests mixed into the same directory.

Point 2 means that if runtime data is obtained for both web-platform-tests and web-platform-tests-reftest, the normalization that occurs in the writeruntimes script means results for both of these get mixed together, skewing the final runtime value that is used in chunking.py.

Scope of work:

  • do not modify the definition of suites from resolver.py.
  • splitting out to multiple files is acceptable.
Assignee: nobody → egao
Duplicate of this bug: 1633001
Attachment #9149442 - Attachment description: Bug 1637463 - update manifest runtime information for all platforms → Bug 1637463 - update manifest runtime information with new schema for all platforms
Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a5fc1b604238
writeruntimes script to output runtime information into nested dictionary r=ahal
https://hg.mozilla.org/integration/autoland/rev/5c8ccda5f1b1
update manifest runtime information with new schema for all platforms r=ahal
https://hg.mozilla.org/integration/autoland/rev/93efb53380c2
support loading of runtime information for a specific suite r=ahal
Backout by malexandru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c83c52541dd2
Backed out 3 changesets for causing bc jobs to timeout.

Taking a look. The test paths scheduled for windows10-64-asan/opt bc1 chunk seems to differ after the halfway point pre- and post-patch, which I would expect since I updated the runtime information file.

Flags: needinfo?(egao)
Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/588155fb515a
writeruntimes script to output runtime information into nested dictionary r=ahal
https://hg.mozilla.org/integration/autoland/rev/ffa30311305c
update manifest runtime information with new schema for all platforms r=ahal
https://hg.mozilla.org/integration/autoland/rev/7ce1fc0e8368
support loading of runtime information for a specific suite r=ahal
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e31d66f10e6d
update manifest runtime information with web-platform-tests-reftest information r=jmaher

Reopening since my previous push to update the manifest-runtime data for each platform didn't include the web-platform-tests-reftest for some reason.

Status: REOPENED → RESOLVED
Closed: 2 months ago2 months ago
Resolution: --- → FIXED
Regressions: 1641824
You need to log in before you can comment on or make changes to this bug.