Closed Bug 1062255 Opened 9 years ago Closed 9 years ago

Crash while opening IDB Database prevents any further tests from running

Categories

(Core :: DOM: Device Interfaces, defect)

Product:
Core
Component:
DOM: Device Interfaces
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
b2g-v2.1 --- unaffected
b2g-v2.2 --- affected

People

(Reporter: Bebe, Unassigned)

References

Details

(Keywords: qablocker, regression)

Attachments

(5 files, 1 obsolete file)

logcat from first bad build
5.74 MB, text/plain
Details
logcat from local indexeddb failure
238.59 KB, text/plain
Details
timeout_console.txt
14.21 KB, text/plain
Details
stuff.zip
45.71 KB, application/zip
Details
mac- logcat taken while running cleopatra
460.77 KB, text/plain
Details
Attached file Logcat (obsolete) — 
Description:
When running tests on the grid marionette can's start a session on the grid 

Environment:
application_buildid: 20140903010251
application_changeset: e5d7e65f7087
application_display_name: B2G
application_name: B2G
application_repository: https://hg.mozilla.org/integration/b2g-inbound
application_version: 35.0a1
build_changeset: 74465af039d2809454afdfef285285bb63146e1b
device_firmware_date: 1409732525
device_firmware_version_incremental: eng.cltbld.20140903.042155
device_firmware_version_release: 4.3
device_id: flame
gaia_changeset: b5c782b267e9ef4587c2d76a4d96d5eeda952e97
gaia_date: 1409730158
platform_buildid: 20140903010251
platform_changeset: e5d7e65f7087
platform_repository: https://hg.mozilla.org/integration/b2g-inbound


Traceback (most recent call last):
  File "/var/jenkins/2/workspace/flame.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette_test.py", line 153, in run
    self.setUp()
  File "/var/jenkins/2/workspace/flame.b2g-inbound.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/tests/functional/dialer/test_dialer_receive_call_with_locked_screen.py", line 19, in setUp
    GaiaTestCase.setUp(self)
  File "/var/jenkins/2/workspace/flame.b2g-inbound.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 744, in setUp
    self.device.start_b2g()
  File "/var/jenkins/2/workspace/flame.b2g-inbound.ui.functional.smoke/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 562, in start_b2g
    self.marionette.start_session()
  File "/var/jenkins/2/workspace/flame.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 815, in start_session
    self.session = self._send_message('newSession', 'value')
  File "/var/jenkins/2/workspace/flame.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/decorators.py", line 35, in _
    return func(*args, **kwargs)
  File "/var/jenkins/2/workspace/flame.b2g-inbound.ui.functional.smoke/.env/local/lib/python2.7/site-packages/marionette_client-0.8.3-py2.7.egg/marionette/marionette.py", line 621, in _send_message
    "Connection timed out", status=errors.ErrorCodes.TIMEOUT)


Reproducible manually: NO

STR:

I can't reproduce the issue locally but we can see it constantly on the grid


http://jenkins1.qa.scl3.mozilla.com/job/flame.b2g-inbound.ui.functional.smoke/1252/consoleFull
I have seen it locally but very rarely.

B2G just completely fails to start up, the screen is black and the device needs an `adb reboot` to get it going again.

I did b2g-ps and saw that b2g process was running but the device is unusable.
This is causing us lots of issues in automation at the moment. We started doing full flashes, which coincides with this, so the regression may not be in gaia or gecko. Our flame.b2g-inbound.ui.functional.smoke shows the following regression range:

last good:
application_buildid: 20140902072512
application_changeset: 2f65aa582f06
application_version: 34.0a1
build_changeset: 74465af039d2809454afdfef285285bb63146e1b
device_firmware_date: 1409239766
device_firmware_version_incremental: eng.cltbld.20140828.112916
device_firmware_version_release: 4.3
gaia_changeset: a081afab97fe2d1a25b08f870aa5007ff929dfbb
gaia_date: 1409666241

first bad:
application_buildid: 20140902113703
application_changeset: 1d3b0ec6e32d
application_version: 35.0a1
build_changeset: 74465af039d2809454afdfef285285bb63146e1b
device_firmware_date: 1409685070
device_firmware_version_incremental: eng.cltbld.20140902.151100
device_firmware_version_release: 4.3
gaia_changeset: 7e469783859785a8bd4bf02a802f32561c84be7b
gaia_date: 1409669127

I also noticed the following in the logcat, which looks suspicious to me:

E/GeckoConsole(12048): [JavaScript Error: "IndexedDB UnknownErr: OpenDatabaseHelper.cpp:1901"]
E/GeckoConsole(12048): [JavaScript Error: "NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIInputStream.available]" {file: "resource://gre/modules/SettingsDB.jsm" line: 81}]
E/GeckoConsole(12052): Failed to load native module at path '/system/b2g/components/libxpcomsample.so': (80004005) dlopen failed: library "/system/b2g/components/libxpcomsample.so" not found
I/Gecko   (12048): -*- SettingsRequestManager: SETTINGS DATABASE ERROR: Cannot make DB connection!

        Attachment #8483444 -
        Attachment is obsolete: true

    
    

    
  
I've been running tests against the following build without any crashes so far. It's not conclusive but I can pick this up again tomorrow if needed.

application_buildid: 20140902072512
application_changeset: 2f65aa582f06
application_version: 34.0a1
device_firmware_date: 1409671774
device_firmware_version_incremental: eng.cltbld.20140902.112925
device_firmware_version_release: 4.3
gaia_changeset: a081afab97fe2d1a25b08f870aa5007ff929dfbb
gaia_date: 1409666241

Jason: Could you keep an eye out for anything similar, and maybe have a look over the logcat to see if something jumps out at you? Thanks!
Flags: needinfo?(jsmith)
Summary: Marionette can't start a session on the device → Crash prevents any further tests from running

    
    

    
  
(In reply to Dave Hunt (:davehunt) from comment #2)
> This is causing us lots of issues in automation at the moment. We started
> doing full flashes, which coincides with this, so the regression may not be
> in gaia or gecko. Our flame.b2g-inbound.ui.functional.smoke shows the
> following regression range:
> 
> last good:
> application_buildid: 20140902072512
> application_changeset: 2f65aa582f06
> application_version: 34.0a1
> build_changeset: 74465af039d2809454afdfef285285bb63146e1b
> device_firmware_date: 1409239766
> device_firmware_version_incremental: eng.cltbld.20140828.112916
> device_firmware_version_release: 4.3
> gaia_changeset: a081afab97fe2d1a25b08f870aa5007ff929dfbb
> gaia_date: 1409666241
> 
> first bad:
> application_buildid: 20140902113703
> application_changeset: 1d3b0ec6e32d
> application_version: 35.0a1
> build_changeset: 74465af039d2809454afdfef285285bb63146e1b
> device_firmware_date: 1409685070
> device_firmware_version_incremental: eng.cltbld.20140902.151100
> device_firmware_version_release: 4.3
> gaia_changeset: 7e469783859785a8bd4bf02a802f32561c84be7b
> gaia_date: 1409669127
> 
> I also noticed the following in the logcat, which looks suspicious to me:
> 
> E/GeckoConsole(12048): [JavaScript Error: "IndexedDB UnknownErr:
> OpenDatabaseHelper.cpp:1901"]
> E/GeckoConsole(12048): [JavaScript Error: "NS_ERROR_FAILURE: Component
> returned failure code: 0x80004005 (NS_ERROR_FAILURE)
> [nsIInputStream.available]" {file: "resource://gre/modules/SettingsDB.jsm"
> line: 81}]
> E/GeckoConsole(12052): Failed to load native module at path
> '/system/b2g/components/libxpcomsample.so': (80004005) dlopen failed:
> library "/system/b2g/components/libxpcomsample.so" not found
> I/Gecko   (12048): -*- SettingsRequestManager: SETTINGS DATABASE ERROR:
> Cannot make DB connection!

Kyle - Any ideas on what's failing here?
Flags: needinfo?(jsmith) → needinfo?(kyle)

    
    

    
  
Wow, ok, not being able to open the database is REALLY catastrophic. That's like "ran out of disk space or memory" level issues.

I think in this case settings might be our canary for this particular coal mine, as it may be the first thing on the system to try and open/create a file. It's rigged to yell like crazy if anything goes wrong at that point because, well, catastrophic. So I wouldn't blame the API for this, but something is definitely wrong.
Flags: needinfo?(kyle)

    
    

    
  
Bebe - How often are we seeing this failure?
Flags: needinfo?(florin.strugariu)

    
    

    
  
(In reply to Jason Smith [:jsmith] from comment #8)
> Bebe - How often are we seeing this failure?

According to No-Jun this is consistently happening.

    
    

    
  
Can we please pull one of these devices out of the pool so that a dev can take a look?

    
    

    
  
Dunno if we need a full device. We should just be able to pull the image, since they're just flames. Just takes being on the VPN and I haven't had time to set that up yet.

    
    

    
  
(In reply to Jason Smith [:jsmith] from comment #8)
> Bebe - How often are we seeing this failure?

It's affecting most (if not all) of our test runs.

(In reply to ben turner [:bent] (use the needinfo? flag!) from comment #10)
> Can we please pull one of these devices out of the pool so that a dev can
> take a look?

We've just seen this on b2g-23.2, which I've taken temporarily offline so we can investigate. Let me know what you need me to pull from this device (and how) and I will do what I can.
Flags: needinfo?(bent.mozilla)

    
    

    
  
The incident mentioned in comment 12 was after performing a full flash from the 31st August and then a shallow flash on top. This implies that the issue was either present in an earlier full flash build than the previous range suggests, or that it is not related to it being a full flash.
Flags: needinfo?(florin.strugariu)

    
    

    
  
[Blocking Requested - why for this release]:

Critical QA blocking regression causing all on device automation to be down.
blocking-b2g: --- → 2.1?
Component: Gaia::UI Tests → DOM: Device Interfaces
Keywords: qablocker, 
          
            regression
Product: Firefox OS → Core
blocking-b2g: 2.1? → 2.2?

    
    

    
  
(In reply to Dave Hunt (:davehunt) from comment #13)
> The incident mentioned in comment 12 was after performing a full flash from
> the 31st August and then a shallow flash on top. This implies that the issue
> was either present in an earlier full flash build than the previous range
> suggests, or that it is not related to it being a full flash.

Hey guys, so what can we do here?   seems the flashing part is not the culprit.  Kyle, any luck on getting your VPN setup yet?   do you still need a physical device for debugging this?

    
    

    
  
This is proving difficult to get a regression range on. We've been able to reproduce the crash by running a single test 20 times, and if it's going to happen it tends to do so in that time. That said, we've had at least one 'good' build that on a rerun turned out to be not so good. Our best effort at a current regression range is below.

Last good:
application_buildid: 20140902132849
application_changeset: aad03f3096f4
build_changeset: 74465af039d2809454afdfef285285bb63146e1b
device_firmware_date: 1409517083
device_firmware_version_incremental: eng.cltbld.20140831.163113
device_firmware_version_release: 4.3
gaia_changeset: 9f09cbb3d5aaafdf2252442f82f2fc84392fb137
gaia_date: 1409684437

First bad:
application_buildid: 20140903075351
application_changeset: 142047b2389e
build_changeset: 74465af039d2809454afdfef285285bb63146e1b
device_firmware_date: 1409517083
device_firmware_version_incremental: eng.cltbld.20140831.163113
device_firmware_version_release: 4.3
gaia_changeset: d2a1710b3336050a427aca197b44d5821eb68266
gaia_date: 1409753617

Pushlog:
http://hg.mozilla.org/integration/b2g-inbound/pushloghtml?fromchange=aad03f3096f4&tochange=142047b2389e

As we get closer we may find this range shifts a little, but I hope this is at least helpful. Please let me know if there's anything more valuable I can provide.

    
    

    
  
The most helpful thing somebody could do is step through http://mxr.mozilla.org/mozilla-central/source/dom/indexedDB/OpenDatabaseHelper.cpp#1969 in gdb and see which early return we are taking.

    
    

    
  
I have my personal python marionette test environment running daily cronjob on flame device, and also saw this timeout issue over the weekend. 

Following is what i got from the zipped archive of the test result:

Last good build for me:
20140901160203 in pvtbuild mozilla central

First bad build for me: 
20140902160202, in pvtbuild mozilla central

    
    

    
  
(In reply to Tony Chung [:tchung] from comment #15)
> (In reply to Dave Hunt (:davehunt) from comment #13)
> > The incident mentioned in comment 12 was after performing a full flash from
> > the 31st August and then a shallow flash on top. This implies that the issue
> > was either present in an earlier full flash build than the previous range
> > suggests, or that it is not related to it being a full flash.
> 
> Hey guys, so what can we do here?   seems the flashing part is not the
> culprit.  Kyle, any luck on getting your VPN setup yet?   do you still need
> a physical device for debugging this?

Based on No-Jun's comment above, Kyle could use this build below to analyze this bug:

https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/nightly/mozilla-central-flame/2014/09/2014-09-02-16-02-02/

    
    

    
  
(In reply to No-Jun Park [:njpark] from comment #18)
> I have my personal python marionette test environment running daily cronjob
> on flame device, and also saw this timeout issue over the weekend. 
> 
> Following is what i got from the zipped archive of the test result:
> 
> Last good build for me:
> 20140901160203 in pvtbuild mozilla central
> 
> First bad build for me: 
> 20140902160202, in pvtbuild mozilla central

just to note that this is m-c nightly build in pvtbuilds.

    
    

    
  
Here's the latest range according to our automation results:

Last good:
application_buildid: 20140902211838
application_changeset: 01962a67b5d8
build_changeset: 74465af039d2809454afdfef285285bb63146e1b
device_firmware_date: 1409517083
device_firmware_version_incremental: eng.cltbld.20140831.163113
device_firmware_version_release: 4.3
gaia_changeset: e1f6d88db39084c98a7d4e361ba30447a8b4264e
gaia_date: 1409716490

First bad:
application_buildid: 20140903010251
application_changeset: e5d7e65f7087
build_changeset: 74465af039d2809454afdfef285285bb63146e1b
device_firmware_date: 1409517083
device_firmware_version_incremental: eng.cltbld.20140831.163113
device_firmware_version_release: 4.3
gaia_changeset: b5c782b267e9ef4587c2d76a4d96d5eeda952e97
gaia_date: 1409730158

Pushlog: http://hg.mozilla.org/integration/b2g-inbound/pushloghtml?fromchange=01962a67b5d8&tochange=e5d7e65f7087

Again, we can't be 100% confident on the 'good' because of the intermittent behaviour of the crash, but we're not considering a build to be 'bad' unless we see a crash detected by Marionette. There's just one build left to test within this range, however if the pushlog looks unlikely to have caused this regression then we can continue to test some of the builds marked as good, to see if we can improve the range.

    
    

    
  
(In reply to Jason Smith [:jsmith] from comment #19)
> (In reply to Tony Chung [:tchung] from comment #15)
> > (In reply to Dave Hunt (:davehunt) from comment #13)
> > > The incident mentioned in comment 12 was after performing a full flash from
> > > the 31st August and then a shallow flash on top. This implies that the issue
> > > was either present in an earlier full flash build than the previous range
> > > suggests, or that it is not related to it being a full flash.
> > 
> > Hey guys, so what can we do here?   seems the flashing part is not the
> > culprit.  Kyle, any luck on getting your VPN setup yet?   do you still need
> > a physical device for debugging this?
> 
> Based on No-Jun's comment above, Kyle could use this build below to analyze
> this bug:
> 
> https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/nightly/mozilla-
> central-flame/2014/09/2014-09-02-16-02-02/

Do we have a minidump anywhere when the crash happens that can be attached to this bug?

    
    

    
  
(In reply to Jason Smith [:jsmith] from comment #19)

> https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/nightly/mozilla-
> central-flame/2014/09/2014-09-02-16-02-02/

Just flashed this to the phone. Boots fine here.

    
    

    
  
And hit enter too early on that last comment. Now running automation tests to see if I can get a repro.

    
    

    
  
So do we know if this has something to do with running tests on the phone, or is it just something that happens when you reboot a whole bunch? Maybe someone should just try running "adb reboot" in a loop for a while?

    
    

    
  


  
Ok, just hit this myself, but my logcat looks different. We aren't even getting to the point of settings complaining it can't open its database now. Lots of errors from indexeddb.

    
    

    
  
Reboot is NOT required to fix this. Simply bringing the b2g process down and back up using stop/start fixes it and B2G comes up fine.

    
    

    
  

      
      
      
      

        Attached file
          
        timeout_console.txt
        — 
      

    


  
Ran into the same problem while running the smoketest on my local flame, with v123 base + fullflash of mozilla-eng from today. Attaching stack trace in case it can provide any more clues.

    
    

    
  
(In reply to Robert Wood [:rwood] from comment #28)
> Created attachment 8485918 [details]
> timeout_console.txt
> 
> Ran into the same problem while running the smoketest on my local flame,
> with v123 base + fullflash of mozilla-eng from today. Attaching stack trace
> in case it can provide any more clues.

did restarting b2g process fix things for you?

    
    

    
  
I'm running a build right now with instrumentation and debug messages that :ehsan put in. That said, I have some thoughts on what might be causing this. Note that this is a pure guess until I actually get some proof. :)

Between the automation tests, we just run something equivilent to adb stop b2g/adb start b2g. We don't actually reboot the phone. This just kills the b2g process. This means that we're basically simulating a crash in b2g, then bringing the process back up. While we've done that for a long time, it could be that our changes to how things work with settings and IDB mean we're accessing the IDB files before OS level exclusivity locks have cleared. Hence OpenDatabaseHelper gets stuck trying to open the DB.

    
    

    
  
What's happening here is |transaction.Commit();| here <http://mxr.mozilla.org/mozilla-central/source/dom/indexedDB/OpenDatabaseHelper.cpp#2160> fails with 0x80630002 which is NS_ERROR_STORAGE_IOERR.  That is raised from <http://mxr.mozilla.org/mozilla-central/source/storage/src/mozStoragePrivateHelpers.cpp#57> which is a result of SQLITE_IOERR.  I don't know exactly what that means, but google suggests that it has something to do with the db being locked.  Comment 30 might be on to something.

    
    

    
  

      
      
      
      

        Attached file
          
        stuff.zip
        — 
      

    


  
Content of idb directory at the time of freeze

    
    

    
  
https://pastebin.mozilla.org/6390073 contains the dmesg output of the device.

    
    

    
  
I'm still following up on Dave's comment #21 but we keep getting the range changed due to false positives in early regression range finding. I'll update later today.

    
    

    
  
(In reply to Jason Smith [:jsmith] from comment #6)
> > I also noticed the following in the logcat, which looks suspicious to me:
> > 
> > E/GeckoConsole(12048): [JavaScript Error: "IndexedDB UnknownErr:
> > OpenDatabaseHelper.cpp:1901"]
> > E/GeckoConsole(12048): [JavaScript Error: "NS_ERROR_FAILURE: Component
> > returned failure code: 0x80004005 (NS_ERROR_FAILURE)
> > [nsIInputStream.available]" {file: "resource://gre/modules/SettingsDB.jsm"
> > line: 81}]
> > E/GeckoConsole(12052): Failed to load native module at path
> > '/system/b2g/components/libxpcomsample.so': (80004005) dlopen failed:
> > library "/system/b2g/components/libxpcomsample.so" not found
> > I/Gecko   (12048): -*- SettingsRequestManager: SETTINGS DATABASE ERROR:
> > Cannot make DB connection!
> 
> Kyle - Any ideas on what's failing here?

This is also mentioned in bug 1064718.

    
    

    
  
If the harness (or a component of it or the way we run it) changed, could the range of interest be in that code and not in gecko?

    
    

    
  
My regression hunting today on b2g-i range narrowed it down to a m-c -> b2g-i merge commit.

The m-c commits that it included are:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=504e18c45b86&tochange=1d3b0ec6e32d

(I know this range is older than those cited but we found that going back in builds this became more difficult to replicate and thus led to several false 'last good builds' in the range).

    
    

    
  
Shouldn't be http://hg.mozilla.org/mozilla-central/rev/9678f9a596d8 because that got backed out later and has yet to reland

    
    

    
  
Absolutely no clue if this is related, but today I've started to receive complaints of phones slowing down after about an hour of usage. See bug 1064624, bug 1064718.

mac- on IRC ran cleopatra profiles against their phone, seeing things dying in places (the system browser history module) there, mostly on IDB calls.

http://people.mozilla.org/~bgirard/cleopatra/#report=9444aa143c5f1b100a76f4602edd2339176309fa
http://people.mozilla.org/~bgirard/cleopatra/#report=5ec6ddc4d4087d8c82d96a806cbf951780d137e2

Attaching logcat to this bug, but I'm not seeing much in the way of IDB errors in it.

    
    

    
  
Making title more descriptive.
Summary: Crash prevents any further tests from running → Crash while opening IDB Database prevents any further tests from running

    
    

    
  
Zac, have you bisected this on the gaia side?  Given the fact that many other tests examine indexed db and this is the only thing that is broken, it's extremely likely that the regression is on the gaia side (either in gaia code, or the test harness, etc.)
Flags: needinfo?(zcampbell)

    
    

    
  
Fabrice mentioned that backing out http://hg.mozilla.org/mozilla-central/rev/0a76964b8d8d cleared up error messages for him on Tarako v2.2 testing. This might be worth doing. I'd do it now except it's midnight. Zac, if you could try a build with this commit backed out, that'd be great.

    
    

    
  
(In reply to :Ehsan Akhgari (not reading bugmail, needinfo? me!) from comment #44)
> Zac, have you bisected this on the gaia side?  Given the fact that many
> other tests examine indexed db and this is the only thing that is broken,
> it's extremely likely that the regression is on the gaia side (either in
> gaia code, or the test harness, etc.)

While testing the b2g-i range we just used the releng/tbpl built Gaia profile and the range also initially included a lot of Bumperbot commits, so yes we have, but for the tests we've always just run from the tip. I don't feel the latter has affected my ability to replicate the failure. I agree though, it smells like a Gaia problem and I'll be upset if I find my regression range wrong again!
Flags: needinfo?(zcampbell)

    
    

    
  
(In reply to Kyle Machulis [:kmachulis] [:qdot] (USE NEEDINFO?) from comment #45)
> Fabrice mentioned that backing out
> http://hg.mozilla.org/mozilla-central/rev/0a76964b8d8d cleared up error
> messages for him on Tarako v2.2 testing. This might be worth doing. I'd do
> it now except it's midnight. Zac, if you could try a build with this commit
> backed out, that'd be great.

This is in my m-i target so I'll run with the tbpl builds before and after (easier for me to just use the pre-built ones).

    
    

    
  
Running locally with gaiatest/test framework and packages on a static commit (to eliminate comment #45) I've narrowed down the last good and first bad mozilla-inbound builds.

I only used TBPL builds owing to time/capacity which has given us a range of 3 commits:

First failing
https://tbpl.mozilla.org/?showall=1&tree=Mozilla-Inbound&rev=e567693fa55f
Crashed after 3 tests run on two successive attempts.

Last successful 
https://tbpl.mozilla.org/?showall=1&tree=Mozilla-Inbound&rev=9678f9a596d8
On the successful build there were 100 tests run without crash whereas in previous regression hunting a crash can be found in <60.

The range includes bug 1048011 that Kyle referred to in comment #45 and which Fabrice also validated.
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=9678f9a596d8&tochange=e567693fa55f

I'll ask for a backout now.
Depends on: 1048011

    
    

    
  
Once we confirm post backout that this no longer reproduces, let's get this bug closed out.

    
    

    
  
(In reply to Jason Smith [:jsmith] from comment #49)
> Once we confirm post backout that this no longer reproduces, let's get this
> bug closed out.

Zac, thank you for the deep investigation!

Kyle, Fabrice, can you help with the backout per comment 48?
Flags: needinfo?(kyle)
Flags: needinfo?(fabrice)

    
    

    
  
(In reply to Tony Chung [:tchung] from comment #50)
> (In reply to Jason Smith [:jsmith] from comment #49)
> > Once we confirm post backout that this no longer reproduces, let's get this
> > bug closed out.
> 
> Zac, thank you for the deep investigation!
> 
> Kyle, Fabrice, can you help with the backout per comment 48?

sorry, ignore me.   We'll have QA confirm the backout fixes as soon as we can.
Flags: needinfo?(kyle)
Flags: needinfo?(florin.strugariu)
Flags: needinfo?(fabrice)

    
    

    
  
Once we've verified that the backout fixes this, if you need assistance with the backout from mozilla-central, ping me on IRC please.

    
    

    
  
(In reply to :Ehsan Akhgari (not reading bugmail, needinfo? me!) from comment #52)
> Once we've verified that the backout fixes this, if you need assistance with
> the backout from mozilla-central, ping me on IRC please.

Nevermind, it has already been backed out!

    
    

    
  
The bug that caused this was backed out in https://bugzilla.mozilla.org/show_bug.cgi?id=1048011

Device tests are back up and running again!
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME

    
    

    
  
As Zac said looks like the builds are coming back to normal.
Flags: needinfo?(florin.strugariu)
Flags: needinfo?(bent.mozilla)

    
    

    
  
After opened a few webpages I've got again same as in Comment 41, the new profile:

http://people.mozilla.org/~bgirard/cleopatra/#report=30441400a456e1d2a6fb6aff7349a0fcda7b0338

taken on:

Alcatel One Touch Fire production (got from T-mobile Poland)
B2G version: 2.2.0.0-prerelease master
Platform version: 35.0a1
Build Identifier: 20140914040220 
Git commit info: 2014-09-13 09:23:34 e5da0e46

    
    

    
  
(In reply to mac from comment #56)
> After opened a few webpages I've got again same as in Comment 41, the new
> profile:
> 
> http://people.mozilla.org/~bgirard/cleopatra/
> #report=30441400a456e1d2a6fb6aff7349a0fcda7b0338
> 
> taken on:
> 
> Alcatel One Touch Fire production (got from T-mobile Poland)
> B2G version: 2.2.0.0-prerelease master
> Platform version: 35.0a1
> Build Identifier: 20140914040220 
> Git commit info: 2014-09-13 09:23:34 e5da0e46

Please file a new bug with more information if this is still reproducing for you.  Developers request a crash log or mini dump if possible.   This original bug is now marked resolved after backing out the culprit landing from comment 45.

    
    

    
  
Removing blocking nom
blocking-b2g: 2.2? → ---

          You need to log in
          before you can comment on or make changes to this bug.