1339492 - Mozregression fails to launch OSX builds older than 2016-07-12

Status: NEW

(Testing :: mozregression, defect, P3)

Description

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Using mozregression 2.3.9.

STR:

★ ~/moz/mozilla-central $ mozregression --good 2016-07-11 --bad 2017-02-14
 0:05.51 INFO: Testing good and bad builds to ensure that they are really good and bad...
 0:05.51 INFO: Downloading build from: https://archive.mozilla.org/pub/firefox/nightly/2016/07/2016-07-11-14-37-35-mozilla-central/firefox-50.0a1.en-US.mac.dmg
===== Downloaded 100% =====
 0:21.75 INFO: Running mozilla-central build for 2016-07-11
 0:49.82 INFO: Launching /private/var/folders/mf/7kfd8gkj3mlcv2_14yxmwh2r0000gp/T/tmp9FtHCk/FirefoxNightly.app/Contents/MacOS/firefox
 0:49.82 INFO: Application command: /private/var/folders/mf/7kfd8gkj3mlcv2_14yxmwh2r0000gp/T/tmp9FtHCk/FirefoxNightly.app/Contents/MacOS/firefox -foreground -profile /var/folders/mf/7kfd8gkj3mlcv2_14yxmwh2r0000gp/T/tmpRf6m64.mozrunner
 0:49.83 INFO: application_buildid: 20160711143735
 0:49.83 INFO: application_changeset: 214884d507ee369c1cf14edb26527c4f9a97bf48
 0:49.83 INFO: application_name: Firefox
 0:49.83 INFO: application_repository: https://hg.mozilla.org/mozilla-central
 0:49.83 INFO: application_version: 50.0a1
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry' or 'exit' and press Enter): 

Expected result: Firefox appears.

Actual result: Firefox never appears.

Workaround: mozregression --good 2016-07-12 --bad 2017-02-14

Comment 1

[Matthew N. [:MattN]]

I just got burned by this and waste time figuring out what was going on. Console.app didn't seem to show anything relevant at quick glance. It would be good to announce this issue, especially if we don't intend to fix it.

I would guess this has something to do with GateKeeper.

Comment 2

[William Lachance (:wlach)]

Paul, would this be something you'd be able to look into? If not I can try to find some time to investigate.

Comment 3

[Paul Yang [:pyang] (away)]

I tried to run 2016-07-11 directly and it showed "firefox(72123,0x7fffce8893c0 malloc: *** malloc_zone_unregister() failed for 9x7fffce87f000", looks like something wrong entirely.

As a followup I think we can get the message and display properly.

Comment 4

[Paul Yang [:pyang] (away)]

I tried 2016-01-01 from 
https://ftp.mozilla.org/pub/firefox/nightly/2016/01/2016-01-01-03-03-30-mozilla-central/firefox-46.0a1.en-US.mac.dmg
on my latest macbook pro. FirefoxNightly can't be launched and hung so we don't have a chance to know if gecko worked properly.
It might be machine/os version dependent issue if so we may try all combination.

moreover, mozregression actually capture log in comment 3 right after prompt.

For gecko side, I think if there is a welcome message/log will be nice but then we might still have edge case such like failed right after welcome log flushed.
In the mean time, what I can do immediately is to adjust position of prompt so that firefox's output message will be showed.

Will, what do you think?

Comment 5

[William Lachance (:wlach)]

If the problem is firefox-specific, I think as long as we show something in the console indicating that the launch failed we'd be ok here. Ideally we'd also log some kind of specific message in mozregression (in addition to the gecko log) like:

 0:49.83 ERROR: Application failed to launch

Does that seem doable?

Comment 6

[Paul Yang [:pyang] (away)]

After discussing with wlach, we can let user not to get build by certain date/period.  I'll check if we're able to set this config by platform.

Comment 7

[Kamil Jozwiak [:kjozwiak]]

Unfortunately I'm running into the same issue with my macOS 10.12.4 machine which is really hindering my ability to do my job right now. I've resorted back to using my Ubuntu VM machine as my main platform for mozregressions as I've wasted way to much time trying to figure out what was going on for the past few days.

As mentioned in comment#3, I'm getting the following error via the terminal when using --build-type debug (happens on regular builds as well) with builds from 2016-07-12 or older:

* firefox(45209,0x7fffbfdd43c0) malloc: *** malloc_zone_unregister() failed for 0x7fffbfdca000

Turning off the macOS gatekeeper via "sudo spctl --master-disable" under macOS Sierra didn't help either.

The main issue is that if a developer needs a regression range and the issue only occurs on macOS and is older than 2016-07-12, there's nothing QA can do right now other than maybe doing it manually which takes a very long time.

Comment 8

[MarjaE]

It often freezes without explanation, but not always.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months.
:zeid, could you have a look please?
For more information, please visit auto_nag documentation.

Comment 10

[:glob ✱]

The malloc issue is bug 1284677; it's likely that all builds between Firefox 31 (2014-04-28) and Firefox 50 (2016-08-01) won't run on macOS 10.12 or higher.