Intermittent OSX 10.7 mochitest failures that seem related to Bluetooth Setup Assistant crashing

RESOLVED FIXED

Status

Release Engineering
Platform Support
P3
critical
RESOLVED FIXED
4 years ago
3 years ago

People

(Reporter: RyanVM, Assigned: kmoir)

Tracking

({intermittent-failure})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 2 obsolete attachments)

(Reporter)

Description

4 years ago
+++ This bug was initially created as a clone of Bug #743594 +++

We were hit by a rash of OSX 10.7 mochitest failures across different slaves where the common thread is mass bustage and the Bluetooth Setup Assistant showing a crashed window on screen (*NOT* the setup wizard itself). Can we take another look at bug 743594 as I think this shows that the wizard popping up isn't always innocuous.

https://tbpl.mozilla.org/php/getParsedLog.php?id=19933482&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19940899&tree=Mozilla-Inbound
(Reporter)

Comment 1

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=19940749&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19944791&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19951035&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19951746&tree=Firefox
Does that "restore running apps after a reboot" misfeature, the one that explains why iCal is running even during suites that aren't the one with the test that opens it, also restore the crash reporter, so we might have a set of slaves which always have that crash report rather than always having the setup assistant?
(Reporter)

Comment 5

4 years ago
This is a pretty frequent failure. Can we please get someone on the relenge side to look at this?

https://tbpl.mozilla.org/php/getParsedLog.php?id=19952759&tree=Mozilla-Inbound
Severity: normal → critical
(Assignee)

Updated

4 years ago
Assignee: nobody → kmoir
(Assignee)

Comment 6

4 years ago
I tried all the steps in bug 743594 to disable the bluetooth setup assistant on 10.7 and they didn't work.  I also tried many more steps that I found on various Apple forums to no avail.  Still looking...
Assignee: kmoir → nobody
(Assignee)

Comment 7

4 years ago
Still haven't found a solution to fix this. I looked at tbpl and this doesn't seem to be happening recently, but as mentioned, it's an intermittent problem.
Assignee: nobody → kmoir
https://tbpl.mozilla.org/php/getParsedLog.php?id=19966616&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19968449&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19944899&tree=Fx-Team
https://tbpl.mozilla.org/php/getParsedLog.php?id=19975109&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19975895&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=19965889&tree=Cedar
(Reporter)

Comment 14

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=19981029&tree=Mozilla-Inbound
Summary: Intermittent OSX mochitest failures that seem related to Bluetooth Setup Assistant crashing → Intermittent OSX 10.7 mochitest failures that seem related to Bluetooth Setup Assistant crashing
(Reporter)

Comment 15

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=19982058&tree=Firefox
(Assignee)

Comment 16

4 years ago
The command to disable it is
defaults write "/Library/Preferences/com.apple.Bluetooth" BluetoothAutoSeekPointingDevice -bool NO
defaults write "/Library/Preferences/com.apple.Bluetooth" BluetoothAutoSeekKeyboard -bool NO

I've tested it on a slave and it does indeed disable the Bluetooth setup assistant for keyboard and mouse.

I'll write a patch for puppet
(Reporter)

Comment 17

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=19987786&tree=Mozilla-Inbound
(Reporter)

Comment 18

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=19989498&tree=Firefox
https://tbpl.mozilla.org/php/getParsedLog.php?id=19991524&tree=Mozilla-Inbound
(Assignee)

Comment 20

4 years ago
Created attachment 717222 [details] [diff] [review]
patch

I've tested in staging
Attachment #717222 - Flags: review?(rail)
(Assignee)

Comment 21

4 years ago
Actually the previous patch just disabled the bluetooth setup assistant.  The resume state should also be disabled, so the crashed applications don't start up again.  I'll attach another patch.
Attachment #717222 - Flags: review?(rail) → review+
Comment on attachment 717222 [details] [diff] [review]
patch

Ooops, wanted to remove r?
Attachment #717222 - Flags: review+
(Reporter)

Comment 23

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=19996842&tree=Firefox
(Assignee)

Comment 24

4 years ago
Created attachment 717268 [details] [diff] [review]
patch

Okay this patch does two things
1) disables the find a keyboard or mouse via bluetooth setup assistant
2) disables the restoring apps upon relogin

I've tested this in staging on talos-r4-lion-001 and it works after a sudo reboot.
Attachment #717222 - Attachment is obsolete: true
(Assignee)

Comment 25

4 years ago
Created attachment 717277 [details] [diff] [review]
patch

to make bool value consistent as false compared to previous patch
Attachment #717268 - Attachment is obsolete: true
Attachment #717277 - Flags: review+
(Assignee)

Comment 26

4 years ago
Comment on attachment 717277 [details] [diff] [review]
patch

and deployed to production.  I'm watching some machines that have puppetized since then and are running mochitests.
Attachment #717277 - Flags: checked-in+
(Reporter)

Comment 27

4 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=20002131&tree=Mozilla-Inbound
(Assignee)

Updated

4 years ago
Duplicate of this bug: 743594
(Assignee)

Comment 29

4 years ago
If the error in comment 27 is still occurring, then it's not related to 1) apps being restored after a restart or 2) the bluetooth setup assistant randomly starting. 

I took talos-r4-lion-008, the machine from comment 27, disabled it in slavealloc, rebooted it, and then started a bunch of applications. Then I rebooted it via sudo reboot, and it didn't restart any applications upon reboot, just like the slave I used in staging.  One thing I did notice is that there are 150+ icons on the desktop for text files with different timestamps on them.  Perhaps they are causing problems when the tests try to find focus?  Just a guess.
It's always hard to tell whether something intermittent has gone away on a weekend, but given the relative frequency Thursday and Friday and the number of jobs we ran Saturday, I would have expected around five Saturday if it was still happening, and we saw zero. I just figured comment 27 was the last one that managed to sneak in before you told it to stop.

The text files are a fun, but separate problem: we think that the test which was leaving them no longer leaves them, but we also think that it's possible that the desktop background (the combination of the actual background and the hundreds of text file icons) affects perf test numbers. You would think that would make us want to remove them all at once, leaving a clear non-code mark that the numbers changed from that, but in fact we're reimaging slaves without them (well, I presume we are, unless we reimage from a refimage that has a desktop-full?) and presuming that they aren't coming back on those reimaged slaves, letting the hypothetical perf improvement roll out as noise. For bonus fun, I've seen screenshots with a pile of .part files, so we probably have other tests which clean up fine when they pass, but leave a .part when they fail, so even post-reimage we should expect the desktop to slowly fill up again, with parts instead of text.
(Assignee)

Comment 31

4 years ago
Okay thanks for the clarification regarding the text and .part files.  I guess this bug can be closed now :-)
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED

Comment 32

4 years ago
talos-r4-lion-001 has a note saying that is being used for this bug.
Is the machine ready to go back to the pool?
Does it need re-imaging?

Thanks in advance!
(Assignee)

Comment 33

4 years ago
Armen: I'm using it for bug 786679 now, I've updated slavealloc.
(In reply to Phil Ringnalda (:philor) from comment #30)
> The text files are a fun, but separate problem: we think that the test which
> was leaving them no longer leaves them, but we also think that it's possible
> that the desktop background (the combination of the actual background and
> the hundreds of text file icons) affects perf test numbers...

Filed bug 851123 for this.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.