[QM_TRY] Missing revisions cause long execution time of analysis scripts
Categories
(Core :: Storage: Quota Manager, task)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox99 | --- | fixed |
People
(Reporter: jan.rio.krause, Assigned: jan.rio.krause)
References
(Blocks 1 open bug)
Details
Attachments
(1 file)
Missing revisions could cause wget extremely long execution times of the QM_TRY analysis python scripts located in dom/quota/scripts. This can be observed with GNU Wget 1.21.2 and seems to be macOS specific.
When a revision to a build id could not be found, the argument passed to wget is an invalid URI and the numerical sequence of the build id will be address resolved.
In its default configuration, wget retries 20x when a host is unavailable. With around 1:15 min per single retry, a waiting time of 25 min per missing revision is caused. If many could not be found, the analysis script could run for several hours. Depending on the build id and the resolved address, wget sometimes already exits after the first try, so the long execution time is intermittent.
Based on a build id from Attachment 9262116 [details] causing 20 retries:
$ wget "20220131101405/dom/localstorage/ActorsParent.cpp"
--2022-02-14 21:00:00-- http://20220131101405/dom/localstorage/ActorsParent.cpp
Resolving 20220131101405 (20220131101405)... 221.187.74.221
Connecting to 20220131101405 (20220131101405)|221.187.74.221|:80... failed: Operation timed out.
Retrying.
--2022-02-14 21:01:16-- (try: 2) http://20220131101405/dom/localstorage/ActorsParent.cpp
Connecting to 20220131101405 (20220131101405)|221.187.74.221|:80... failed: Operation timed out.
Retrying.
[...]
Based on a build id from Attachment 9263751 [details] causing immediately exit:
$ wget "20220210131556/dom/quota/ActorsParent.cpp"
--2022-02-14 21:00:00-- http://20220210131556/dom/quota/ActorsParent.cpp
Resolving 20220210131556 (20220210131556)... 226.113.50.100
Connecting to 20220210131556 (20220210131556)|226.113.50.100|:80... failed: Address family not supported by protocol family.
Both commands on a Debian GNU/Linux will be exited immediately causing no extra time:
$ wget "20220131101405/dom/localstorage/ActorsParent.cpp"
--2022-02-14 21:00:00-- http://20220131101405/dom/localstorage/ActorsParent.cpp
Resolving 20220131101405 (20220131101405)... failed: Name or service not known.
wget: unable to resolve host address ‘20220131101405’
|
Assignee | |
Comment 1•2 years ago
|
||
Pushed by jkrause@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/1f4303fcab36 Skip fetching source of missing revisions. r=dom-storage-reviewers,jstutte
|
||
Comment 3•2 years ago
|
||
| bugherder | ||
Description
•