1121378 - Use single QEmu branch for all platforms

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Emulator, defect)

Description

[Thomas Zimmermann [:tzimmermann] [:tdz]]

We currently only support ICS emulation. Once we get KK or others working, we'll have multiple QEmu branches with diverging feature sets.

The guest system does not depend on the version of the QEmu emulator, so ideally we'd run all platforms on the same QEmu source code. That will allow us to focus emulator maintenance on a single branch.

Comment 1

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Hi Jonathan,

I hope you're the right person to ask about automated-tests infrastructure.

We'd like to get emulator-kk working at some point and we're looking for ways to reduce our maintenance overhead of the different emulators. Currently we have one emulator branch per platform (ICS, JB, KK). So one idea is to use a single branch for all emulators.

But just switching branches will probably break automates tests completely. Is there a playground for testing infrastructure changes without interfering with automated tests?

Comment 2

[Thomas Zimmermann [:tzimmermann] [:tdz]]

FYI: I was able to build and run the ICS emulator on top of qemu's b2g-kitkat branch. It seemed to work as for the regular branch. I did not run any tests, though.

Comment 3

[Thomas Zimmermann [:tzimmermann] [:tdz]]

> We'd like to get emulator-kk working at some point and we're looking for
> ways to reduce our maintenance overhead of the different emulators.
> Currently we have one emulator branch per platform (ICS, JB, KK). So one
> idea is to use a single branch for all emulators.

To be more specific: I'd like to switch the branch in external/qemu and push the change into automated tests to test against all current tests.

Comment 4

[Jonathan Griffin (:jgriffin)]

I think you may be able to get what you need in a try push.

You can probably modify the files in this directory, and then push it to try, to see what effect it has on emulator tests:

http://hg.mozilla.org/mozilla-central/file/5438e3f74848/b2g/config/emulator

The try syntax you'd want is:  -b o -p emulator -u all -t none

Comment 5

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Thank you so much! I didn't know this is possible. Try run is at

 https://treeherder.mozilla.org/#/jobs?repo=try&revision=2ded314e2a54

Comment 6

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Jonathan,

Sorry for bothering you again. I changed the revision in sources.xml, but it seems as if the emulator was not even build at all. At least the log at [1] does not contain any signs of its build process.

Is there a flag for cleaning up temporary data, or something similar?

[1] https://treeherder.mozilla.org/logviewer.html#?job_id=4325750&repo=try

Comment 7

[Jonathan Griffin (:jgriffin)]

I agree, it seems to be missing.  I don't know why.   All try jobs get clobbered by default, so this shouldn't be a case of some crud left over from a previous job.  Does that qemu commit build for you locally?

Comment 8

[Vicamo Yang [:vicamo][:vyang]]

AOSP no longer compiles emulator host binaries along with the rest of the system since lollipop. Instead, a new repository, prebuilts/android-emulator, was setup to contain them because compiling emulator host binaries relates to so many other repositories that it can be impossible to compile new version along with out-dated tree or to compile old version on newer systems. Maybe you'll feel like to adopt this method, so all Firefox OS emulator variants share the same host binary set, and only those who are going to patch emulator need to compile qemu manually.

https://android.googlesource.com/platform/prebuilts/android-emulator/

Comment 9

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Thanks for this information, Vicamo.

It seems to me that the host binaries are also host-dependent. I guess the best long-term approach is to automatically rebuild the pre-builts when the source repository changes. There's already QEMU and the Linux kernel, who knows what comes next.

Comment 10

[Thomas Zimmermann [:tzimmermann] [:tdz]]

(In reply to Jonathan Griffin (:jgriffin) from comment #7)
> I agree, it seems to be missing.  I don't know why.   All try jobs get
> clobbered by default, so this shouldn't be a case of some crud left over
> from a previous job.  Does that qemu commit build for you locally?

I figured out that ICS needs a small change to build KK's QEMU. I pushed a new try

  https://treeherder.mozilla.org/#/jobs?repo=try&revision=8ba49f5224ff

Comment 11

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Trying again

  - Set 'optional' tag for all QEMU modules

  https://treeherder.mozilla.org/#/jobs?repo=try&revision=53c5fc7a2244

Comment 12

[Thomas Zimmermann [:tzimmermann] [:tdz]]

I would have liked to work on this, but Mercurial broke my checkout of try and cloning takes forever. :( I'll try again tomorrow.

Comment 13

[Jonathan Griffin (:jgriffin)]

You don't need to clone try.  You can push to it from m-c or inbound or whatever.  See https://wiki.mozilla.org/ReleaseEngineering/TryServer#Pushing_to_try

Comment 14

[Thomas Zimmermann [:tzimmermann] [:tdz]]

(In reply to Jonathan Griffin (:jgriffin) from comment #13)
> You don't need to clone try.  You can push to it from m-c or inbound or
> whatever.  See
> https://wiki.mozilla.org/ReleaseEngineering/TryServer#Pushing_to_try

Thank you for your reply. When I read this page, I always thought that I have to clone try first.

It's still too late today. :)

Comment 15

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Trying again

  - updated sdk/ to KK version to use correct GL libs

  https://treeherder.mozilla.org/#/jobs?repo=try&revision=778ad63902ff

Comment 16

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Trying again

  - set LOCAL_MODULE_TAGS for emulator OpenGL modules

  https://treeherder.mozilla.org/#/jobs?repo=try&revision=5d3dcad954a5

Comment 17

[Thomas Zimmermann [:tzimmermann] [:tdz]]

I think a hit a wall here. I'm able to build the ICS emulator with KK's qemu and sdk, and some fixes in build/. But building requires the host tool 'emugen' and I cannot figure out why it's not compiled by the build scripts.

Comment 18

[Thomas Zimmermann [:tzimmermann] [:tdz]]

FYI: This bug is completely experimental. *Nothing* should ever depend in it.

Comment 19

[Thomas Zimmermann [:tzimmermann] [:tdz]]

s/in/on

Comment 20

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Attachment: Changes to platform_build

Comment 21

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Attachment: Changes to android-development

Comment 22

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Attachment: Changes to android-sdk

Comment 23

[BMO Automation]

Firefox OS is not being worked on