841836 - Update the b2g emulator for bug 716859

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Jeff Muizelaar [:jrmuizel]]

There's a bad bug in the qemu emulator that's causing crashes when trying to land bug 716859.

Comment 1

[Jonathan Griffin (:jgriffin)]

According to Jeff, the change that needs to get picked up is just landing.  Jeff, which commit do we need for this?

Comment 2

[Jeff Muizelaar [:jrmuizel]]

We need this commit in the android-development repo

https://github.com/mozilla-b2g/android-development/commit/5a439240bc3d4143981b9e690c41ccafe9f67537

Comment 3

[Jonathan Griffin (:jgriffin)]

I'm starting a new emulator build now, will post the url in this bug once it's ready (~1 hr)

Comment 4

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to Jonathan Griffin (:jgriffin) from comment #3)
> I'm starting a new emulator build now, will post the url in this bug once
> it's ready (~1 hr)

that won't include the fix, unless your b2g-manifest repo has c0d63f006c810942716dcdf48584c5f1947c2ba9

Comment 5

[Jeff Muizelaar [:jrmuizel]]

The manifests have changed:
https://github.com/mozilla-b2g/b2g-manifest/commit/c0d63f006c810942716dcdf48584c5f1947c2ba9

Comment 6

[Jonathan Griffin (:jgriffin)]

The manifest being used in the current build contains this line:

<project path="development" name="android-development" remote="b2ggithub" revision="5a439240bc3d4143981b9e690c41ccafe9f67537"/>

so I think it's ok.

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

Releng, I don't know if you are building the emulator and no one could help me figure that out, so please make sure it's with sources that are pulled down by running against b2g-manifest@c0d63f006c810942716dcdf48584c5f1947c2ba9 if you do build the emulator.

Comment 8

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to Jonathan Griffin (:jgriffin) from comment #6)
> The manifest being used in the current build contains this line:
> 
> <project path="development" name="android-development" remote="b2ggithub"
> revision="5a439240bc3d4143981b9e690c41ccafe9f67537"/>
> 
> so I think it's ok.

yup, it's k

Comment 9

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Dumb question: why do we not use a branch name there?  Especially on our github version?  That way we don't have to screw with commit IDs..

Comment 10

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #9)
> Dumb question: why do we not use a branch name there?  Especially on our
> github version?  That way we don't have to screw with commit IDs..

Because it's not a dumb question.  I'm going to change that at the same time as switching to git.mozilla.org, b4da957f393674849b5b9e21411aabc9f52420f6

Comment 11

[Jonathan Griffin (:jgriffin)]

Updated emulator available:  http://ec2-107-20-108-245.compute-1.amazonaws.com/jenkins/job/b2g-build/1545/artifact/package.zip

aki or armenzg, can you upload to tooltool?

Comment 12

[Armen [:armenzg]]

Do I assume that a code landing needs to happen? (Passing it back)


[[e-mail]3 ~]$ curl -u b2g -o emulator.zip http://ec2-107-20-108-245.compute-1.amazonaws.com/jenkins/job/b2g-build/1545/artifact/package.zip
Enter host password for user 'b2g':
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  605M  100  605M    0     0  32.6M      0  0:00:18  0:00:18 --:--:-- 37.1M
[[e-mail]3 ~]$ export SHA512=`openssl sha512 emulator.zip | cut -d' ' -f2`
[[e-mail]3 ~]$ sudo mv ~/emulator.zip /var/www/html/runtime-binaries/tooltool/sha512/${SHA512}
[[e-mail]3 ~]$ chmod 644 /var/www/html/runtime-binaries/tooltool/sha512/${SHA512}
[[e-mail]3 ~]$ ls -l /var/www/html/runtime-binaries/tooltool/sha512/${SHA512}
-rw-r--r-- 1 armenzg armenzg 634898569 Feb 15 13:02 /var/www/html/runtime-binaries/tooltool/sha512/3b143cbf9093f81b33e36f3c74819ef52e77d4c0304da0d55b700c97cc8f52870c29f143ff9a638a746f4c15acaf476de66de03ec104c45cc30b6bf85fdfaf5a


Armens-MacBook-Air ~ $ curl -I http://runtime-binaries.pvt.build.mozilla.org/tooltool/sha512/3b143cbf9093f81b33e36f3c74819ef52e77d4c0304da0d55b700c97cc8f52870c29f143ff9a638a746f4c15acaf476de66de03ec104c45cc30b6bf85fdfaf5a
HTTP/1.1 200 OK
Date: Fri, 15 Feb 2013 21:03:55 GMT
Server: Apache
Last-Modified: Fri, 15 Feb 2013 21:02:56 GMT
ETag: "25d7c889"
Accept-Ranges: bytes
Content-Length: 634898569
Connection: close
Content-Type: text/plain; charset=UTF-8

Comment 13

[Jonathan Griffin (:jgriffin)]

I'll update the in-tree emulator manifest now.

Comment 14

[Jonathan Griffin (:jgriffin)]

Attachment: Update emulator,

Comment 15

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/727072808d9f

Jeff, do we need this same emulator used in the b2g18 branches?

Comment 16

[Jeff Muizelaar [:jrmuizel]]

(In reply to Jonathan Griffin (:jgriffin) from comment #15)
> https://hg.mozilla.org/integration/mozilla-inbound/rev/727072808d9f
> 
> Jeff, do we need this same emulator used in the b2g18 branches?

It wouldn't hurt, but we don't need it right away.

Comment 17

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

per  irc w/jgriffin:

1) jgriffin has built new emulator
2) armenzg has landed the new emulator in tooltool, so production RelEng systems can use it.
3) jgriffin has landed this change in mozilla-inbound. Once this change lands on mozilla-central, we'll be all done here, and jgriffin will close this bug.

Comment 18

[Phil Ringnalda (:philor)]

Backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/25f6e7b84a8c for the mochitest-1 failures in https://tbpl.mozilla.org/php/getParsedLog.php?id=19793312&tree=Mozilla-Inbound (he said casually, as though that didn't take two people six hours to retrigger down to the cause).

Comment 19

[Jonathan Griffin (:jgriffin)]

We need someone (overholt, can you recommend a person?) to look at the failures in https://tbpl.mozilla.org/php/getParsedLog.php?id=19793312&tree=Mozilla-Inbound.  We'll either need to fix these or disable these tests in order to update the emulator.

Comment 20

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

So ignoring the webgl failure, what we seem to have (talking with jeff):

- emulator built from master revision WITHOUT jeff's patch
-- shows permissions failures in tests

- emulator built from master revision WITH jeff's patch
-- shows permissions failures in tests

- emulator built from v1-train manifest WITHOUT jeff's patch
-- does NOT show permissions failures in tests

- emulator built from v1-train manifests WITH jeff's patch
-- (pretty sure) does NOT show permissions failures in tests

We know that the emulator that's being used *right now* on m-c for b2g testing is not the one specified by the emulator.xml manifest in master, because that one still points to one with Jeff's fix.  (That's bad, but irrelevant.)

Can we confirm that the emulator that is being used on m-c is from the manifest immediately previous to Jeff's fix?  The manifest just specifies "master" for a lot of things, so any repo could have been updated since the last time the emulator was built, giving us an environment broken in some way that we haven't detected since we haven't changed it on the tinderboxes.  That seems like the most likely scenario currently (it's not qemu, we checked that one).

Comment 21

[Rail Aliiev [:rail]]

ATM we are using the emulator updated in bug 836644, and it doesn't look like android-development is in the zip file (no such entry in b2g-distro/out/target/product/generic/system/sources.xml), while the latest emulator build has this:

<project name="android-development" path="development" remote="b2g" revision="5a439240bc3d4143981b9e690c41ccafe9f67537"/>

Comment 22

[Jeff Muizelaar [:jrmuizel]]

Does updating the emulator also update the associated gaia?

Comment 23

[Jeff Muizelaar [:jrmuizel]]

It seems like the permissions problems come from a gaia change. i.e. If I run with gaia master locally I can reproduce the permissions problems. If I run against v1-train gaia I can not.

Comment 24

[Rail Aliiev [:rail]]

Attachment: current sources.xml

Currently used emulator's source.xml file

Comment 25

[Jonathan Griffin (:jgriffin)]

I've built another emulator, this one using the same Gaia as the previous one.  Aki or armenzg, can you upload to tooltool?

http://ec2-107-20-108-245.compute-1.amazonaws.com/jenkins/job/b2g-build/1564/artifact/package.zip

Comment 26

[Justin Wood (:Callek)]

(In reply to Jonathan Griffin (:jgriffin) from comment #25)
> I've built another emulator, this one using the same Gaia as the previous
> one.  Aki or armenzg, can you upload to tooltool?
> 
> http://ec2-107-20-108-245.compute-1.amazonaws.com/jenkins/job/b2g-build/1564/
> artifact/package.zip

Done:

[[e-mail]3 ~]$ ls -l /var/www/html/runtime-binaries/tooltool/sha512/${SHA512}
-rw-r--r-- 1 jwood jwood 635042567 Feb 19 12:09 /var/www/html/runtime-binaries/tooltool/sha512/483f049896bf0828b9c0dbe0c387e10cc28cfbd6cdda7ced2d02e9bf690ebe47ef51507ec5cc8bc5e31705b7d4b7ace8b1266c0a283dc93047c8229829f6eda1

Comment 27

[Jonathan Griffin (:jgriffin)]

Attachment: Update emulator for bug 716859,

Comment 28

[Justin Wood (:Callek)]

Comment on attachment 715675 [details] [diff] [review]
Update emulator for bug 716859,

Suggest a run through try before inbound/etc.

Comment 29

[Jonathan Griffin (:jgriffin)]

I'll let jrmuizel make the call.  Do we have time for a try run?  It won't be all in until late tonight or tomorrow, probably.

Comment 30

[Jonathan Griffin (:jgriffin)]

Pushed to try anyhow:  https://tbpl.mozilla.org/?tree=Try&rev=98cf95a9c11e

Comment 31

[Jonathan Griffin (:jgriffin)]

Attachment: Update emulator for bug 716859,

Here's a version that applies cleanly to current inbound.

Comment 32

[Jonathan Griffin (:jgriffin)]

This emulator build somehow picked up the latest gaia, which wasn't intended.  I'm manually rebuilding another...

Comment 33

[Jonathan Griffin (:jgriffin)]

Updated emulator:  http://ec2-107-20-108-245.compute-1.amazonaws.com/jenkins/job/b2g-build/1565/artifact/package.zip

Callek, aki, or armenzg, can you upload to tooltool?

Comment 34

[Aki Sasaki (not active)]

Uploaded.

606070399 
3d3899e2537bee5e3f969bb6d0e90fed93345512123e3133b1de793d59a8993d8174d9456cb92d86a880f6813d6049933de924cd522a433ef26c4bfa997777a7

Comment 35

[Jonathan Griffin (:jgriffin)]

Attachment: Update emulator for bug 716859,

Updated emulator

Comment 36

[Jonathan Griffin (:jgriffin)]

pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=f0c5c9078ad8

Comment 37

[Jeff Muizelaar [:jrmuizel]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/6a14e4c15aa6

Comment 38

[Ryan VanderMeulen [:RyanVM]]

Guess what? We're still failing B2G tests! Since this and bug 716859 landed so close together, I can't separate which actually broke them, so I backed both out.
https://hg.mozilla.org/integration/mozilla-inbound/rev/f04919e7789f

Comment 39

[Jonathan Griffin (:jgriffin)]

The try run above was green, so it may have been something related to the patch for bug 716859.

Comment 40

[Jeff Muizelaar [:jrmuizel]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/fec94a52b86f

Comment 41

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/fec94a52b86f