Intermittent [tier2] raptor-browsertime Critical: [chrome] Exception [Errno 2] No such file or directory: '/var/folders/0y/lyz4b4sx50dg5sbmmsj0tn3r000014/T/tmpzv56t98k.mozrunner/Default/user.js' while removing mozprofile delimiters
Categories
(Testing :: Raptor, defect, P5)
Tracking
(firefox96 wontfix)
Tracking | Status | |
---|---|---|
firefox96 | --- | wontfix |
People
(Reporter: intermittent-bug-filer, Unassigned)
References
Details
(Keywords: intermittent-failure, Whiteboard: [stockwell unknown])
Attachments
(2 files)
Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=335507079&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Yd8bNRlbT-CLitFo8MSnog/runs/0/artifacts/public/logs/live_backing.log
[task 2021-04-06T00:39:11.314Z] 00:39:11 INFO - raptor-perftest Info: Merging profile: /opt/worker/tasks/task_161766945739418/build/tests/raptor/raptor/profile_data/raptor
[task 2021-04-06T00:39:11.315Z] 00:39:11 INFO - raptor-perftest Info: Local browser profile: /var/folders/0y/lyz4b4sx50dg5sbmmsj0tn3r000014/T/tmpzv56t98k.mozrunner/Default
[task 2021-04-06T00:39:11.315Z] 00:39:11 INFO - raptor-browsertime Info: Removing mozprofile delimiters from browser profile
[task 2021-04-06T00:39:11.315Z] 00:39:11 CRITICAL - raptor-browsertime Critical: [chromium] Exception [Errno 2] No such file or directory: '/var/folders/0y/lyz4b4sx50dg5sbmmsj0tn3r000014/T/tmpzv56t98k.mozrunner/Default/user.js' while removing mozprofile delimiters
[task 2021-04-06T00:39:11.316Z] 00:39:11 INFO - raptor-browsertime Info: cwd: '/opt/worker/tasks/task_161766945739418/build'
[task 2021-04-06T00:39:11.316Z] 00:39:11 INFO - raptor-browsertime Info: browsertime_node: /opt/worker/tasks/task_161766945739418/fetches/node/bin/node
[task 2021-04-06T00:39:11.316Z] 00:39:11 INFO - raptor-browsertime Info: browsertime_node: os.stat_result(st_mode=33261, st_ino=31538611, st_dev=16777221, st_nlink=1, st_uid=36, st_gid=20, st_size=42289088, st_atime=1617669464, st_mtime=1609766268, st_ctime=1617669466)
[task 2021-04-06T00:39:11.317Z] 00:39:11 INFO - raptor-browsertime Info: browsertime_browsertimejs: /opt/worker/tasks/task_161766945739418/fetches/browsertime/node_modules/browsertime/bin/browsertime.js
[task 2021-04-06T00:39:11.317Z] 00:39:11 INFO - raptor-browsertime Info: browsertime_browsertimejs: os.stat_result(st_mode=33261, st_ino=31541327, st_dev=16777221, st_nlink=1, st_uid=36, st_gid=20, st_size=4376, st_atime=1617669463, st_mtime=1610746082, st_ctime=1617669466)
[task 2021-04-06T00:39:11.317Z] 00:39:11 INFO - raptor-browsertime Info: browsertime_geckodriver: /opt/worker/tasks/task_161766945739418/fetches/geckodriver```
Comment 1•3 years ago
|
||
This looks to be osx 10.15 specific, backfilling does not show any conclusive results, so it might've been failing on osx 10.15 since it was enabled intree.
Sparky, what is your take on this?
Comment hidden (Intermittent Failures Robot) |
Comment 3•3 years ago
|
||
Oh those tests shouldn't be running there. :bebe/:alexandrui can either of you disable these benchmark tests on autoland?
Comment 4•3 years ago
|
||
Updated•3 years ago
|
Updated•3 years ago
|
Pushed by aionescu@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/3eb017d9f3fa Disable chrome/chroium browsertime benchmarks on autoland r=perftest-reviewers,sparky
Comment 7•3 years ago
|
||
bugherder |
Comment hidden (Intermittent Failures Robot) |
Comment 9•3 years ago
|
||
Alexandru, this is happening again.
Should we reopen this bug to keep track of this issue, or make a new one?
Comment 10•3 years ago
|
||
Bebe, can you take a look?
Comment hidden (Intermittent Failures Robot) |
Comment 12•3 years ago
|
||
Looks like this is not reproducing anymore.
I will take a look if we see this again
Comment 14•3 years ago
|
||
:Sparky can you take a look over this please
Comment 15•3 years ago
|
||
No I don't have time, you'll have to look into it. We could decrease this test to tier 3 since it isn't very important.
Comment hidden (Intermittent Failures Robot) |
Comment 17•3 years ago
|
||
Tested this locally and it works as expected on my setup.
Note: I have a Linux setup and this issue is mac specific.
Also from the logs it looks like this fails on a random cycle. So the browser and chromedriver should work but at some point a error is triggered in browser.
UnexpectedAlertOpenError: unexpected alert open: {Alert text : WebGL context lost, please reload the page}
This error looks something browser/WebGL specific
Can we get someone with a MAC setup to run this locally to see what happens.
Comment 18•3 years ago
|
||
:bebe, before trying to look into more locally, can you check to see if an older version of chromium works? Alternatively, if there isn't a version which passes, we could drop it to a tier-3 test.
Updated•3 years ago
|
Comment 19•3 years ago
|
||
(In reply to Florin Strugariu [:Bebe] (needinfo me) from comment #17)
Tested this locally and it works as expected on my setup.
Note: I have a Linux setup and this issue is mac specific.Also from the logs it looks like this fails on a random cycle. So the browser and chromedriver should work but at some point a error is triggered in browser.
UnexpectedAlertOpenError: unexpected alert open: {Alert text : WebGL context lost, please reload the page}
This error looks something browser/WebGL specific
Can we get someone with a MAC setup to run this locally to see what happens.
I'm happy to try to run something on my MacBook Pro if you can provide a command line for me, and details of any other prerequisites.
Comment 20•3 years ago
|
||
It's looking good now, I think a chromium update fixed it. :bebe, no need for looking into this for now: https://treeherder.mozilla.org/jobs?repo=mozilla-central&searchStr=chromium
Updated•3 years ago
|
Comment 21•3 years ago
|
||
Comment hidden (Intermittent Failures Robot) |
Updated•3 years ago
|
Updated•3 years ago
|
Comment 23•3 years ago
|
||
The last of these browsertime tests with Chrome Release failed.
Comment 24•3 years ago
|
||
:Sparky can you take a look over this?
Comment 25•3 years ago
|
||
It looks like Chrome Release isn't updating on MacOSX 10.15 Shippable
. :dividehex, would you able to look into this issue? On these machines, chrome should autoupdate and the version should be at 91 but it's currently at 88:
[task 2021-05-28T10:44:57.767Z] 10:44:57 INFO - raptor-perftest Info: Attempting to get version through fallback method...
[task 2021-05-28T10:44:57.802Z] 10:44:57 INFO - Google Chrome 88.0.4324.192
[task 2021-05-28T10:44:57.894Z] 10:44:57 INFO - raptor-perftest Info: Browser name: chrome
[task 2021-05-28T10:44:57.894Z] 10:44:57 INFO - raptor-perftest Info: Browser version: 88.0.4324.192
Comment hidden (Intermittent Failures Robot) |
Comment 27•3 years ago
|
||
:dhouse/:aerickson could you look into this issue?
Comment 28•3 years ago
|
||
I'll check these. Our auto-update of chrome may have broken when we upgraded to MacOS Catalina, on the R8 macminis.
Comment 29•3 years ago
|
||
the macminis are running Chrome 91.0.4472.77 now. I re-triggered keystone and reset the permissions on /Applications/Google.
I think because chrome was installed on the R8's under root, the file permissions blocked the user execution of keystone (gets set as by chrome as a launchagent. so it runs when the user logs into the display console) from overwriting the chrome binaries and resources.
keystone was giving errors like:
Exit code: 8. Standard error output: \"rsync: failed to set times on \\\"/Applications/Google Chrome.app/.\\\": Operation not permitted (1)\\nrsync: failed to set times on \\\"/Applications/Google Chrome.app/Contents\\\"
Comment 30•3 years ago
|
||
Thanks :dhouse!
Comment hidden (Intermittent Failures Robot) |
Updated•3 years ago
|
Comment 32•3 years ago
|
||
No need to do anything here, the failures were caused by an outdated chrome version on the macs.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 36•3 years ago
|
||
One worker macmini had not updated (macmini-r8-257 was on chrome v88). I manually triggered the upgrade on it and it pulled v91
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Updated•3 years ago
|
Comment 40•3 years ago
|
||
:dhouse, it looks like some machines are having issues updating chrome to the latest version (should be 91). Could you take a look? I'm seeing some machines with version 81, and others with version 88.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Updated•3 years ago
|
Comment 51•3 years ago
|
||
:sparky
Hi, can you please take a look at those perma failures?
https://treeherder.mozilla.org/jobs?repo=mozilla-central&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Cusercancel&selectedTaskRun=B38bjDFeRNaHvRI87FhJqg.0&searchStr=Linux%2C18.04%2Cx64%2CWebRender%2CShippable%2Copt%2CBrowsertime%2Cperformance%2Ctests%2Con%2CGoogle%2CChrome%2CRelease%2Ctest-linux1804-64-shippable-qr%2Fopt-browsertime-benchmark-chrome-ares6-e10s%2Cares6
Comment 52•3 years ago
|
||
The real error here is that chromedriver can't be located:
Exception: Cannot find the chromedriver for the chrome version being tested: /home/cltbld/tasks/task_163229241112151/fetches/94chromedriver
This could be due to the version of Chrome updating (v94 appears to have been released today). Cristian: please can you open a separate bug for this issue?
Comment 53•3 years ago
|
||
:davehunt Hi, as per your request, logged https://bugzilla.mozilla.org/show_bug.cgi?id=1732226
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 67•3 years ago
|
||
There have been 79 failures in the last 7 days:
- 35 failures on linux1804-64-shippable-qr opt
- 20 failures on macosx1014-64-shippable-qr opt
- 9 failures on macosx1015-64-shippable-qr opt
- 15 failures on windows10-64-shippable-qr opt
Recent failure log: https://treeherder.mozilla.org/logviewer?job_id=358595498&repo=mozilla-central&lineNumber=2306
[task 2021-11-19T11:59:56.720Z] 11:59:56 INFO - raptor-perftest Info: Local browser profile: /tmp/tmpt74z27ri.mozrunner/Default
[task 2021-11-19T11:59:56.721Z] 11:59:56 INFO - raptor-browsertime Info: Removing mozprofile delimiters from browser profile
[task 2021-11-19T11:59:56.721Z] 11:59:56 CRITICAL - raptor-browsertime Critical: [chrome] Exception [Errno 2] No such file or directory: '/tmp/tmpt74z27ri.mozrunner/Default/user.js' while removing mozprofile delimiters
[task 2021-11-19T11:59:56.721Z] 11:59:56 INFO - raptor-browsertime Info: cwd: '/home/cltbld/tasks/task_163732313428545/build'
[task 2021-11-19T11:59:56.721Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_node: /home/cltbld/tasks/task_163732313428545/fetches/node/bin/node
[task 2021-11-19T11:59:56.722Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_node: os.stat_result(st_mode=33261, st_ino=7379503, st_dev=64768, st_nlink=1, st_uid=1006, st_gid=1007, st_size=48928552, st_atime=1637323136, st_mtime=1617721616, st_ctime=1637323136)
[task 2021-11-19T11:59:56.722Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_browsertimejs: /home/cltbld/tasks/task_163732313428545/fetches/browsertime/node_modules/browsertime/bin/browsertime.js
[task 2021-11-19T11:59:56.722Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_browsertimejs: os.stat_result(st_mode=33261, st_ino=10497085, st_dev=64768, st_nlink=1, st_uid=1006, st_gid=1007, st_size=4988, st_atime=1637323136, st_mtime=1633607361, st_ctime=1637323136)
[task 2021-11-19T11:59:56.723Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_geckodriver: /home/cltbld/tasks/task_163732313428545/fetches/geckodriver
[task 2021-11-19T11:59:56.723Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_geckodriver: os.stat_result(st_mode=33261, st_ino=6988416, st_dev=64768, st_nlink=1, st_uid=1006, st_gid=1007, st_size=8106856, st_atime=1637323135, st_mtime=1636390340, st_ctime=1637323135)
[task 2021-11-19T11:59:56.723Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_chromedriver: /home/cltbld/tasks/task_163732313428545/fetches/{}chromedriver
[task 2021-11-19T11:59:56.724Z] 11:59:56 INFO - raptor-browsertime Info: browsertime_chromedriver: [Errno 2] No such file or directory: '/home/cltbld/tasks/task_163732313428545/fetches/{}chromedriver'
Greg, as the owner of the component could you help us assign this to someone?
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 70•3 years ago
|
||
Comment 71•3 years ago
|
||
This issue will be fixed when the patch under review lands.
Comment hidden (Intermittent Failures Robot) |
Comment 74•3 years ago
|
||
Pushed by gmierz2@outlook.com: https://hg.mozilla.org/integration/autoland/rev/6c90d21f400c Add chromedriver for v96. r=perftest-reviewers,AlexandruIonescu
Comment 75•3 years ago
|
||
bugherder |
Comment hidden (Intermittent Failures Robot) |
Comment 77•3 years ago
|
||
That failure bump yesterday was expected as the new chromedrive hadn't landed yet.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 80•3 years ago
|
||
This has started appearing again on central, Greg, should we make a new bug for it?
Comment 81•3 years ago
|
||
I've reopened this one, thanks!
:dhouse, it looks like the mac machines aren't updating correctly, can you take a look?
Updated•3 years ago
|
Updated•3 years ago
|
Updated•3 years ago
|
Comment 82•3 years ago
|
||
Sorry for the mass ni? but could either of you take a look at what's happening with the macs please? They aren't updating to the new Chrome version (96) for some reason. The versions seem to be stuck at 88 and 92.
Comment 83•3 years ago
|
||
here is a task from 2 days ago that shows fetch-chromium:
https://firefox-ci-tc.services.mozilla.com/tasks/Ptd7eftwQgKh6wTOcjvWVA (ends up with revision: 945744)
I am not able to easily figure out a chromium version. at least 88/92/96. I did see in the original task referenced here:
https://firefox-ci-tc.services.mozilla.com/tasks/fBwFXYN1TC6wz1NZsjKzMg/runs/0/logs/public/logs/live.log
this:
{'name': 'chromium', 'version': '98.0.4735.0'}
The script calls fetch-chromium.py --platform mac
, and we have had some recent changes to fetch-chromium:
https://hg.mozilla.org/mozilla-central/log/tip/taskcluster/scripts/misc/fetch-chromium.py
when the script runs locally it finds a revision via:
https://www.googleapis.com/download/storage/v1/b/chromium-browser-snapshots/o/Mac%2FLAST_CHANGE?alt=media
this results in revision=946511.
if I unpack and ls -laR | grep -i Version
, I see files like:
Chromium.app/Contents/Frameworks/Chromium Framework.framework/Versions/98.0.4740.0/Resources/
so this indicates that we are using version 98- should we be using a different version?
possibly this is resolved?
Comment 84•3 years ago
|
||
Sorry, this isssue is unrelated to chromium now. It's a chrome failure. Here's a link to one of the most recent failures: https://treeherder.mozilla.org/jobs?repo=mozilla-central&selectedJob=359548232&revision=d03f875556391582e06abbf647835af8ca59f94b&searchStr=mac%2Cchrome&selectedTaskRun=bQ_MlsMoS9mrmRHMCPdGDw.0
Note the error line that suggests chrome 92 is installed on the mac:
Exception: Cannot find the chromedriver for the chrome version being tested: /opt/worker/tasks/task_163818342129411/fetches/92chromedriver
I think we should probably rename this bug or make a new one.
Updated•3 years ago
|
Comment 85•3 years ago
|
||
could chromedriver be different from chromium version?
Comment 86•3 years ago
|
||
odd, that specific task linked:
https://firefox-ci-tc.services.mozilla.com/tasks/bQ_MlsMoS9mrmRHMCPdGDw
depends on task fetch chromedriver 96:
https://firefox-ci-tc.services.mozilla.com/tasks/AZdzONqnQ6u6EGdW4ModaQ
and in the log of the failing task it shows chromedriver 96 being downloaded and unpacked:
https://firefox-ci-tc.services.mozilla.com/tasks/bQ_MlsMoS9mrmRHMCPdGDw/runs/0/logs/public/logs/live.log
it also shows chromedriver 94 and 95, but not 92. Maybe there is a hardcoded version on a path somewhere?
given the error message, we must have chrome v.92 if it is looking for 92chromedriver (I am making an assumption).
looking for chrome.app in the failing task, I see:
https://firefox-ci-tc.services.mozilla.com/tasks/bQ_MlsMoS9mrmRHMCPdGDw/runs/0/logs/public/logs/live.log
/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
one would assume this is hardcoded to v.92? do we need to update our preinstalled version of google chrome on osx 10.15 mac minis?
Comment 87•3 years ago
|
||
For chromium, we get the chromedriver it needs through the fetch-task. But for chrome, we have these fetch-tasks for the chromedrivers: https://searchfox.org/mozilla-central/source/taskcluster/ci/fetch/browsertime.yml#32-138
The way we have the chromedriver thing setup in raptor is that we find the current chrome version and then use the correct chromedriver for it in the moz_fetches directory (assuming it's there). We leave the last 3 versions of chrome in-tree because the machines don't update instantaneously.
I thought we had auto-updating on these macs but maybe it's been hardcoded to v92 and if so, then we'd need to update it.
Comment 88•3 years ago
|
||
I think :dhouse could determine if the R8 minis have the preinstalled google chrome set to auto update?
Comment hidden (Intermittent Failures Robot) |
Comment 90•3 years ago
|
||
I have forced an update across the mac workers (r8's), and I'm investigating why they got stuck (again, this happened in https://bugzilla.mozilla.org/show_bug.cgi?id=1722015 when we needed v92).
I'll re-check their chrome version this afternoon and manually fix ones that don't get it this morning.
Comment 91•3 years ago
|
||
I found that the r7 macs were auto-updating (so the autoupdate was fixed there) but only some of the r8's were (I think ones that had been new/reimaged).
They're all updated to chrome v96 now (I quarantined the ones I am manually updating, and I will un-quarantine them when they're updated), and I plan to reimage a set of r8's to see if a new install fixes the auto-update.
Updated•3 years ago
|
Comment hidden (Intermittent Failures Robot) |
Updated•3 years ago
|
Comment 93•3 years ago
|
||
Thanks :dhouse! It's working now based on the Monday test run: https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=chrome%2Cbrowsertime&revision=422c7594bf65d1e930d52165356b0098402ff6fe
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 99•2 years ago
|
||
There have been 37 total failures in the last 7 days (recent failure log).
Affected platforms are:
- linux1804-64-shippable-qr
- macosx1014-64-shippable-qr
- macosx1015-64-shippable-qr
- windows10-64-shippable-qr
Greg, could you please take a look?
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 102•2 years ago
|
||
This should be resolved by https://phabricator.services.mozilla.com/D135251 which landed yesterday
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 139•2 years ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.
Reporter | ||
Comment 140•1 year ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=403762354&repo=mozilla-central
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 143•1 year ago
|
||
The bug assignee is inactive on Bugzilla, so the assignee is being reset.
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 149•1 year ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 150•10 months ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=423386248&repo=mozilla-central
Comment 151•9 months ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 152•9 months ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=426148730&repo=mozilla-central
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 158•7 months ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Reporter | ||
Comment 159•3 months ago
|
||
treeherder |
New failure instance: https://treeherder.mozilla.org/logviewer?job_id=447624614&repo=mozilla-central
Comment hidden (Intermittent Failures Robot) |
Comment 161•2 months ago
|
||
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment hidden (Intermittent Failures Robot) |
Comment 166•27 days ago
|
||
https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.
Description
•