Closed Bug 1476053 Opened Last year Closed Last year

Make wpt metadata update faster and use less memory

Categories

(Testing :: web-platform-tests, enhancement)

Version 3
enhancement
Not set

Tracking

(firefox63 fixed)

RESOLVED FIXED
mozilla63
Tracking Status
firefox63 --- fixed

People

(Reporter: jgraham, Assigned: jgraham)

References

Details

Attachments

(5 files, 1 obsolete file)

The wpt metadata update is currently super-slow and can use a lot of memory. The basic design at the moment is that we read all the data in to the metadata ini files, update each ini file and write out the results. But that's largely unnecessary.

Instead, we only need to read/update ini files where something changed (this will also avoid unwanted serialization changes to unaffected files). It's still necessary to load all the result data upfront since we need to convert from result-per-run to result-per-test, which is effectively a transpose of the input data. However this can be made much more efficient than the current setup.

In addition we can use the wptreport.json files to do the metadata updates. This will mean parsing far less json data, which should be a notable performance improvement, and certainly reduce the time to download the log files.
Comment on attachment 8993729 [details]
Bug 1476053 - Add an option not to store state when running wpt-update,

https://reviewboard.mozilla.org/r/258434/#review265530
Attachment #8993729 - Flags: review?(josh) → review+
Comment on attachment 8993730 [details]
Bug 1476053 - Add lsan leak reports to wptreport output,

https://reviewboard.mozilla.org/r/258436/#review265560
Attachment #8993730 - Flags: review?(ato) → review+
Comment on attachment 8993731 [details]
Bug 1476053 - Optimise wpt metadata updates,

https://reviewboard.mozilla.org/r/258438/#review265562

It would be great to see tests for this code, but I realise that
might be easier said than done…

::: testing/web-platform/tests/tools/wptrunner/wptrunner/metadata.py:383
(Diff revision 1)
> +    def set_requires_update(self):
> +        self._requires_update = True

Unsure what the benefit of this function is as opposed to writing
to the mutable class property:

> self.requires_update = True
Attachment #8993731 - Flags: review?(ato) → review+
Comment on attachment 8993732 [details]
Bug 1476053 - Enabling updating wpt metadata from wptreport.json files,

https://reviewboard.mozilla.org/r/258440/#review265564
Attachment #8993732 - Flags: review?(ato) → review+
Comment on attachment 8994269 [details]
Bug 1476053 - Handle invalid json in metadata updates,

https://reviewboard.mozilla.org/r/258844/#review266062
Attachment #8994269 - Flags: review?(ato) → review+
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/bc1c54e5dcea
Add an option not to store state when running wpt-update, r=jdm
https://hg.mozilla.org/integration/autoland/rev/1380819f4aa5
Add lsan leak reports to wptreport output, r=ato
https://hg.mozilla.org/integration/autoland/rev/fcf3f7f2079a
Optimise wpt metadata updates, r=ato
https://hg.mozilla.org/integration/autoland/rev/bfb7d92282e8
Enabling updating wpt metadata from wptreport.json files, r=ato
https://hg.mozilla.org/integration/autoland/rev/ab1b54921f26
Handle invalid json in metadata updates, r=ato
Assignee: nobody → james
Blocks: 1479290
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/12246 for changes under testing/web-platform/tests
Depends on: 1528870
You need to log in before you can comment on or make changes to this bug.