1116074 - B2G Emulator: lollipop build

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Emulator, defect)

Description

[Vicamo Yang [:vicamo][:vyang]]

+++ This bug was initially created as a clone of Bug #957526 +++

Would be nice to have something sweet before the end of year :)

Comment 1

[Vicamo Yang [:vicamo][:vyang]]

Working branches:
https://github.com/vicamo/B2G/tree/bugzilla/1116074/emulator-lollipop
https://github.com/vicamo/b2g_manifest/tree/bugzilla/1116074/emulator-lollipop
https://github.com/vicamo/b2g_platform_external_qemu/tree/bugzilla/1116074/emulator-lollipop
https://github.com/vicamo/b2g_platform_hardware_ril/tree/bugzilla/1116074/emulator-lollipop
https://github.com/vicamo/b2g_device_generic_goldfish/tree/bugzilla/1116074/emulator-lollipop

Since we don't have necessary branches created in some repositories, one has to cherry-pick following change to manifest in order to have a successful `repo sync`:
https://github.com/vicamo/b2g_manifest/tree/bugzilla/1116074/use-vicamo-s-repositories-temporarily

Comment 2

[Vicamo Yang [:vicamo][:vyang]]

Need sdk as well:
https://github.com/vicamo/b2g_platform_sdk/tree/bugzilla/1116074/emulator-lollipop

Comment 3

[Vicamo Yang [:vicamo][:vyang]]

From Android Lollipop and on, emulator binaries are provided by a new repository android-emulator [1]. Emulator specific products are mini_emulator_{arm,arm64,x86,x86_64}. Only mini_emulator_x86_64 compiles emulator binaries by setting BUILD_EMULATOR to true.

[1]: https://android.googlesource.com/platform/prebuilts/android-emulator/

Comment 4

[Vicamo Yang [:vicamo][:vyang]]

Correct: |aosp_arm.mk| includes |full.mk|, and |full.mk| still includes |emulator.mk| in lollipop. So we can still use full-eng as Mozilla's emulator product target, but it won't include emulator binaries built from source :(

Comment 5

[Vicamo Yang [:vicamo][:vyang]]

Finished first run external/qemu merge (https://github.com/vicamo/b2g_platform_external_qemu/tree/bugzilla/1116074/emulator-lollipop, and https://github.com/vicamo/b2g_platform_external_qemu/tree/next/aosp/master for aosp/master). Need an lollipop environment to verify it.

Comment 6

[Vicamo Yang [:vicamo][:vyang]]

(In reply to Vicamo Yang [:vicamo][:vyang] from comment #5)
> Need an lollipop environment to verify it.

Verified merge commits are kept in this branch: https://github.com/vicamo/b2g_platform_external_qemu/tree/bugzilla/1116074/successfully-built
This was done by building all against existing emulator-kk source tree. However, the next to-be-verified commit failed due to toolchain compatibilities. Need another way to make further progress.

Comment 7

[Vicamo Yang [:vicamo][:vyang]]

https://github.com/vicamo/b2g_platform_external_qemu/tree/bugzilla/1116074/emulator-lollipop passes built agains kitkat-dev tree, to try lollipop later.

Comment 8

[Vicamo Yang [:vicamo][:vyang]]

Both https://github.com/vicamo/b2g_platform_external_qemu/tree/next/aosp/master and https://github.com/vicamo/b2g_platform_external_qemu/tree/bugzilla/1116074/emulator-lollipop are ready. May go on to merge hardware/ril :)

Comment 9

[Vicamo Yang [:vicamo][:vyang]]

xpcom/ds/TimeStamp_posix.cpp:274:78: error: call to 'sprintf' declared with attribute warning: sprintf is often misused; please use snprintf [-Werror]

A lot of sprintf errors when compiling Gecko :(

Comment 10

[Vicamo Yang [:vicamo][:vyang]]

Attachment: Github PR for B2G

This pr add two new emulator targets emulator-l and emulator-x86-l.

Comment 11

[Vicamo Yang [:vicamo][:vyang]]

Attachment: Github PR for b2g-manifest

This pr add two new emulator targets emulator-l and emulator-x86-l.

Note that before all necessary heads being created for new projects added in this change, you may need https://github.com/vicamo/b2g_manifest/tree/bugzilla/1116074/use-vicamo-s-repositories-temporarily to fetch the whole source tree.

Comment 12

[Vicamo Yang [:vicamo][:vyang]]

(In reply to Vicamo Yang [:vicamo][:vyang] from comment #11)
> Note that before all necessary heads being created for new projects added in
> this change, you may need
> https://github.com/vicamo/b2g_manifest/tree/bugzilla/1116074/use-vicamo-s-
> repositories-temporarily to fetch the whole source tree.

All the necessary new branch heads are:

1. https://github.com/mozilla-b2g/platform_external_qemu/ : "b2g-lollipop" => refs/tags/android-5.0.0_r6 (006a174)
2. https://github.com/mozilla-b2g/platform_hardware_ril/ : "b2g-lollipop" => refs/tags/android-5.0.0_r6 (5ffec08)
3. https://github.com/mozilla-b2g/device_generic_goldfish : "b2g-5.0.0_r6" => refs/tags/android-5.0.0_r6 (82a4dc7)
4. https://github.com/mozilla-b2g/android-sdk : "b2g-5.0.0_r6" => refs/tags/android-5.0.0_r6 (d4ce15e)

Comment 13

[Vicamo Yang [:vicamo][:vyang]]

per comment 9, blocked by bug 92064.

Comment 14

[Vicamo Yang [:vicamo][:vyang]]

TODO:

0. Figure out how to compile gecko on Mac OS X 10.10, or any other viable platforms.

1. Pull requests for repositories listed in comment 12 can only be sent after the branches being created. So far only external/qemu is ready.

2. Merge hardware/ril. Not really blocked by anything else but item 0. Need a way to test merged RIL functions.

3. Kernel changes. Lollipop x86_64 emulator is using Android goldfish kernel v3.10, but others are still on v3.4. So probably we may reuse v3.4 until we're going to go x86_64, then we'll need manifest change as well as a new branch?

Comment 15

[Vicamo Yang [:vicamo][:vyang]]

(In reply to Vicamo Yang [:vicamo][:vyang] from comment #12)
> All the necessary new branch heads are:
> 
> 4. https://github.com/mozilla-b2g/android-sdk : "b2g-5.0.0_r6" =>
> refs/tags/android-5.0.0_r6 (d4ce15e)

I'm not quite sure with this one, but due to bug 92064 I can't not have a successful build to make sure that.

Comment 16

[Vicamo Yang [:vicamo][:vyang]]

(In reply to Vicamo Yang [:vicamo][:vyang] from comment #12)
> 1. https://github.com/mozilla-b2g/platform_external_qemu/ : "b2g-lollipop"
> => refs/tags/android-5.0.0_r6 (006a174)
> 2. https://github.com/mozilla-b2g/platform_hardware_ril/ : "b2g-lollipop" =>
> refs/tags/android-5.0.0_r6 (5ffec08)

Depends on b2g-kitkat merge on external/qemu, hardware/ril. See https://bugzilla.mozilla.org/show_bug.cgi?id=1083650#c21

Comment 17

[Vicamo Yang [:vicamo][:vyang]]

(In reply to Vicamo Yang [:vicamo][:vyang] from comment #14)
> TODO:
> 
> 0. Figure out how to compile gecko on Mac OS X 10.10, or any other viable
> platforms.

Gecko compiles on Linux.

Besides, it seems we may no longer compile qemu along with lollipop source tree. At least two errors found. The first one, "no input source" when compiling gdbstub-xml.o, can be resolved with a patch to Makefile.common [1]. The second one, "PIC register clobbered" in function no_gs_ioctl, takes some time to analyze what happened. Any way, I think I should go the AOSP way -- prebuild emulator binaries and save them in prebuilts/android-emulator [2] as AOSP does.

[1]: https://github.com/vicamo/b2g_platform_external_qemu/tree/bugzilla/1116074/fix-gdbstub-xml-o-no-input-source
[2]: https://android.googlesource.com/platform/prebuilts/android-emulator/

Comment 18

[Edgar Chen [:edgar]]

(In reply to Vicamo Yang [:vicamo][:vyang] from comment #16)
> (In reply to Vicamo Yang [:vicamo][:vyang] from comment #12)
> > 1. https://github.com/mozilla-b2g/platform_external_qemu/ : "b2g-lollipop"
> > => refs/tags/android-5.0.0_r6 (006a174)
> > 2. https://github.com/mozilla-b2g/platform_hardware_ril/ : "b2g-lollipop" =>
> > refs/tags/android-5.0.0_r6 (5ffec08)
> 
> Depends on b2g-kitkat merge on external/qemu, hardware/ril. See
> https://bugzilla.mozilla.org/show_bug.cgi?id=1083650#c21

I will help on the merging thing. Will keep you posted here. :)

Comment 19

[Edgar Chen [:edgar]]

(In reply to Edgar Chen [:edgar][:echen] from comment #18)
> (In reply to Vicamo Yang [:vicamo][:vyang] from comment #16)
> > (In reply to Vicamo Yang [:vicamo][:vyang] from comment #12)
> > > 1. https://github.com/mozilla-b2g/platform_external_qemu/ : "b2g-lollipop"
> > > => refs/tags/android-5.0.0_r6 (006a174)
> > > 2. https://github.com/mozilla-b2g/platform_hardware_ril/ : "b2g-lollipop" =>
> > > refs/tags/android-5.0.0_r6 (5ffec08)
> > 
> > Depends on b2g-kitkat merge on external/qemu, hardware/ril. See
> > https://bugzilla.mozilla.org/show_bug.cgi?id=1083650#c21
> 
> I will help on the merging thing. Will keep you posted here. :)

Done,
https://github.com/mozilla-b2g/platform_external_qemu/commits/b2g-kitkat
https://github.com/mozilla-b2g/platform_hardware_ril/commits/b2g-kitkat

Comment 20

[Vicamo Yang [:vicamo][:vyang]]

(In reply to Edgar Chen [:edgar][:echen] from comment #19)
> Done,
> https://github.com/mozilla-b2g/platform_external_qemu/commits/b2g-kitkat
> https://github.com/mozilla-b2g/platform_hardware_ril/commits/b2g-kitkat

Done? You call this "Done"???? I've told you I've prepare those merged branches in 
https://github.com/vicamo/b2g_platform_external_qemu/tree/next/b2g/b2g-jellybean
https://github.com/vicamo/b2g_platform_external_qemu/tree/next/b2g/b2g-kitkat
https://github.com/vicamo/b2g_platform_hardware_ril/tree/next/b2g/b2g-jellybean
https://github.com/vicamo/b2g_platform_hardware_ril/tree/next/b2g/b2g-kitkat
Now you pushed another commit that has NOTHING changed but just a new commit. What do you mean? Do you know I had to re-merge all since the merged commit is different?

Comment 21

[Vicamo Yang [:vicamo][:vyang]]

Since you have other opinion on how this bug should be done, and it has never been assigned to me, good luck.

Comment 22

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Github PR for b2g-manifest

Thomas, Could you help me review this patch? Thanks!

I think we need to have emulator-l build ok and on try server to avoid b2g build base on lollipop build break by other commit.

As comment in manifest, hardware/ril and qemu need to be updated.

Comment 23

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Fix qemu compile error

Thomas,

This could be a temporary solution for emulator-l build successfully. I think we need wait bug 1121378 resolved fixed or move all changes on b2g-kitkat to lollipop.

Could you review this patch? Thanks!

Comment 24

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Update goldfish to 5.0.0_r6

Thomas, I rebase our changes to 5.0.0_r6. Could you review this pull request? Thanks!

Comment 25

[Kai-Zhen Li [:seinlin][:kli]]

Comment on attachment 8552344 [details] [review]
Github PR for B2G

Looks good to me.

Comment 26

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Oh cool!

I have a bunch of other things on the top of my to-do list. I'll take a look during the next days. Sorry for the delay.

Comment 27

[Thomas Zimmermann [:tzimmermann] [:tdz]]

I get the following build error:

Import includes file: out/target/product/generic/obj/SHARED_LIBRARIES/libsqlite_intermediates/import_includes
hardware/ril/reference-ril/atchannel.c:38:29: fatal error: linux/omap_csmi.h: No such file or directory
 #include <linux/omap_csmi.h>
                             ^
compilation terminated.

Comment 28

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8562615 [details] [review]
Fix qemu compile error

Rubber-stamping.

Comment 29

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8562616 [details] [review]
Update goldfish to 5.0.0_r6

Rubber-stamping.

Comment 30

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8562612 [details] [review]
Github PR for b2g-manifest

I'm not sure if I can just r+ this, but it works for me with the remaining nit fixed.

Make sure you fix the latest comment in the pull request before landing this change!

Comment 31

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Thanks for working on this bug!

Comment 32

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #30)
> Comment on attachment 8562612 [details] [review]
> Github PR for b2g-manifest
> 
> I'm not sure if I can just r+ this, but it works for me with the remaining
> nit fixed.
> 
> Make sure you fix the latest comment in the pull request before landing this
> change!

Thomas, Thanks! 

I updated my latest patch to PR. To use repo from caf, the name should be platform/hardware/ril. As the comment in manifest, let's update qemu and hardware/ril in corresponding bugs.

Comment 33

[Kai-Zhen Li [:seinlin][:kli]]

v2.2 support lollipop, I think we need emulator-l on it.

Comment 34

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Build emulators locally

Thomas, We also need to update build to have emulators get built. Could you have a look to this patch? Thanks!

Comment 35

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8565080 [details] [review]
Build emulators locally

I don't generally object this patch, but could we pull prebuilds/android-emulator instead? It seems the 'canonical' way of getting an emulator binary.

For future versions of our emulator, we would just update this repository. I guess that's also nice for reducing workload on our test machines.

Comment 36

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8565080 [details] [review]
Build emulators locally

I left some suggestions in the pull request.

Comment 37

[Thomas Zimmermann [:tzimmermann] [:tdz]]

I was able to build emulator-l locally. Thank you very much for working on this bug.

Comment 38

[Stephany Wilkes]

Triage is taking as a blocker given Lollipop on 2.2. And if it's blocking testing, it will block the release.

Also ni? to Thomas to see if he can resolve this as WORKSFORME.

Comment 39

[Kai-Zhen Li [:seinlin][:kli]]

https://github.com/mozilla-b2g/B2G/commit/e8f04c8d613937ad6ab22b552cf71357d20f7d98
https://github.com/mozilla-b2g/b2g-manifest/commit/a89b4174200a9a49b3b87801ea2c739917c9b347
https://github.com/mozilla-b2g/platform_external_qemu/commit/7639c5c496ffd207bb627f2a59b2c5203ae6fefc
https://github.com/mozilla-b2g/device_generic_goldfish/commit/1b9f134d62536c5727575d1f0a5bd4eeb4747d66
https://github.com/mozilla-b2g/platform_build/commit/52775e03a2d8532429dff579cb2cd56718e488c3

Comment 40

[Kai-Zhen Li [:seinlin][:kli]]

Now emulator can be built, but it will not function properly before bug 1113054 and bug 1121378 get fixed.

Comment 41

[Kai-Zhen Li [:seinlin][:kli]]

To correct comment 40, hardware/ril should be upgraded in bug 1134982 and not bug 1113054.

Comment 42

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: [v2.2] Github PR for b2g-manifest

Comment 43

[Thomas Zimmermann [:tzimmermann] [:tdz]]

(In reply to Kai-Zhen Li [:kli][:seinlin] from comment #40)
> Now emulator can be built, but it will not function properly before bug
> 1113054 and bug 1121378 get fixed.

Bug 1121378 is purely experimental and we should not let anything depend on it. My work on it rather shows that it is probably not possible to use a single QEMU binary, because the guest's OpenGL driver depends on specific QEMU versions.

Comment 44

[Thomas Zimmermann [:tzimmermann] [:tdz]]

(In reply to Stephany Wilkes from comment #38)
> Triage is taking as a blocker given Lollipop on 2.2. And if it's blocking
> testing, it will block the release.
> 
> Also ni? to Thomas to see if he can resolve this as WORKSFORME.

I'm not sure what the exact criteria for blockers are. With these patches landed, we can only build emulator-l. If you what to run actual tests on it, that will still be a long way.

Comment 45

[Thomas Zimmermann [:tzimmermann] [:tdz]]

'Building' allows us to verify that a specific revision of Gecko compiles in an environment of Android L.

Comment 46

[Ryan VanderMeulen [:RyanVM]]

Given that this may soon be blocking Nexus 5-L job visibility, I think blocking status is warranted.

Comment 47

[Pete Moore [:pmoore][:pete]]

I've had to backout the b2g-manifest change as this broke b2g bumper in production:
https://github.com/mozilla-b2g/b2g-manifest/commit/6d817977d2c5404b03a096be16eae2bb383c6fea

Please note the travis tests failed, which indicates that b2g bumper will break:
https://travis-ci.org/mozilla-b2g/b2g-manifest/builds/51473586

I'm not sure if there is a way to configure the b2g-manifest github project to only allow merges from forks with passing tests, but this might be something we could add to help increase stability.

Thanks,
Pete

Comment 48

[Pete Moore [:pmoore][:pete]]

Last lines of log file buildbot-master66.bb.releng.usw2.mozilla.com:/builds/b2g_bumper/master.log:

05:09:43  WARNING - https://git.mozilla.org/external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8:refs/tags/android-5.0.0_r6 - got output: fatal: remote error: FATAL: R any external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8 gitweb DENIED by fallthru
05:09:43  WARNING - (or you mis-spelled the reponame)
05:09:43     INFO - https://git.mozilla.org/external/caf/platform/hardware/qcom/wlan:refs/tags/android-5.0.0_r6 - got output: 2e54754cc0529d26ccac37ed291600048adbf6c0	refs/tags/android-5.0.0_r6
05:09:44     INFO - https://git.mozilla.org/external/caf/platform/hardware/ril:refs/tags/android-5.0.0_r6 - got output: 71dfa8228ad0d6cdf6bac0426ac59404ab74b7f3	refs/tags/android-5.0.0_r6
05:09:45     INFO - https://git.mozilla.org/external/caf/platform/hardware/qcom/sensors:refs/tags/android-5.0.0_r6 - got output: 3724fd91ef5183684d97e2bf1d7ff948faabe090	refs/tags/android-5.0.0_r6
05:09:46     INFO - https://git.mozilla.org/external/caf/platform/hardware/qcom/media:refs/tags/android-5.0.0_r6 - got output: 758a80fbb178b5663d4edbb46944b2dc553cb1ca	refs/tags/android-5.0.0_r6
05:10:14  WARNING - Returned 128 - sleeping and retrying
05:10:14  WARNING - https://git.mozilla.org/external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8:refs/tags/android-5.0.0_r6 - got output: fatal: remote error: FATAL: R any external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8 gitweb DENIED by fallthru
05:10:14  WARNING - (or you mis-spelled the reponame)
05:10:44  WARNING - Returned 128 - sleeping and retrying
05:10:44  WARNING - https://git.mozilla.org/external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8:refs/tags/android-5.0.0_r6 - got output: fatal: remote error: FATAL: R any external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8 gitweb DENIED by fallthru
05:10:44  WARNING - (or you mis-spelled the reponame)
05:11:15  WARNING - Returned 128 - sleeping and retrying
05:11:15  WARNING - https://git.mozilla.org/external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8:refs/tags/android-5.0.0_r6 - got output: fatal: remote error: FATAL: R any external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8 gitweb DENIED by fallthru
05:11:15  WARNING - (or you mis-spelled the reponame)
05:11:45    ERROR - Couldn't resolve reference https://git.mozilla.org/external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8 refs/tags/android-5.0.0_r6
05:11:45     INFO - needs sync? git://codeaurora.org//platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8 -> https://git.mozilla.org/external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8
05:11:45    FATAL - couldn't resolve some refs; exiting
05:11:45    FATAL - Running post_fatal callback...
05:11:45    FATAL - Exiting -1

Comment 49

[Pete Moore [:pmoore][:pete]]

It is not clear why nagios did not alert, I only discovered this by coincidence. Raising a separate bug for that...

Comment 50

[Thomas Zimmermann [:tzimmermann] [:tdz]]

It looks like some CAF-based repo is not available. CAF was down earlier today. Could this be the cause of this problem?

Otherwise, how can this be fixed? Do we need to mirror these repos explicitly before landing the patch set? (Sorry for asking, I don't know enough about the infrastructure.)

Comment 51

[Pete Moore [:pmoore][:pete]]

Hi Thomas,

It looks like we are mirroring to:
http://git.mozilla.org/?p=external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.6.git;a=summary

but not to:
http://git.mozilla.org/?p=external/caf/platform/prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8.git;a=summary

I'm guessing the problem is just that we need to set up the mirroring first, before landing the b2g-manifest change.

If the travis tests for b2g-manifest pass, b2g bumper should run fine in production.

Are you ok to request the mirroring? Maybe there is an existing bug for the previous x86_64-linux-glibc2.11-4.6 version which can be adjusted slightly to change to x86_64-linux-glibc2.11-4.8.

Ideally, b2g bumper would mirror a repo that it spotted it is missing, but we're not there yet, so we have to manually set up the mirroring first, unfortunately.

Pete

Comment 52

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Thanks, I'll request mirroring of branches, according to the changes in the manifest.

Comment 53

[Edgar Chen [:edgar]]

(In reply to Kai-Zhen Li [:kli][:seinlin] from comment #40)
> Now emulator can be built, but it will not function properly before bug
> 1113054 and bug 1121378 get fixed.

Correct the dependence based on above comment.

Comment 54

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #52)
> Thanks, I'll request mirroring of branches, according to the changes in the
> manifest.

Sorry, I forgot to mirror the new added repos before merge the manifest.

Comment 55

[Michael Wu [:mwu]]

b2g-manifest PRs should not be merged if tests fail. Be sure to check the test status next time.

Comment 56

[Kai-Zhen Li [:seinlin][:kli]]

bug 1135041 is resolved fixed and test result is green.  Merge again,

https://github.com/mozilla-b2g/b2g-manifest/commit/4f43590df7257b1646c8eb8d74a75faef7897d07

Comment 57

[Ryan VanderMeulen [:RyanVM]]

v2.2: https://github.com/mozilla-b2g/b2g-manifest/commit/9852a31075443f525bb7d869f7cd1694ebb440cd