1184418 - [emulator-l][external/qemu] build standalone emulator host binary and integrate it into full system build

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Emulator, defect)

Description

[Edgar Chen [:edgar]]

+++ This bug was initially created as a clone of Bug #1181481 +++

external/qemu provides a way to build standalone emulator host binary [1]. Use this way to build host binary [2] can fix bug #1181481 and bug #1171774. However we have to figure out how to integrate the standalone build into full system build.

[1] See bug #1171774 comment #1 for the details.
[2] With studio-1.3-release branch, https://github.com/android/platform_external_qemu/tree/studio-1.3-release.

Comment 1

[Samael Wang [:freesamael] (away for now)]

Attachment: Experimental prototype patch for platform_build

Comment 2

[Samael Wang [:freesamael] (away for now)]

Attachment: Experimental prototype patch for platform_external_qemu

Comment 3

[Edgar Chen [:edgar]]

Thanks for your experimental patch, Samael.

Comment 4

[Edgar Chen [:edgar]]

Attachment: [platform_external_qemu:b2g-studio-1.3] PR #153, Integrate standalone emulator build into full system build

Comment 5

[Edgar Chen [:edgar]]

Attachment: [platform_build:b2g-5.1.0_r1] PR#119, Build standalone emulator host binary

Comment 6

[Edgar Chen [:edgar]]

A new branch, https://github.com/mozilla-b2g/platform_external_qemu/tree/b2g-studio-1.3, is created from https://github.com/android/platform_external_qemu/tree/studio-1.3-release.

Comment 7

[Edgar Chen [:edgar]]

Attachment: [gonk-misc:master] PR#242, Add emulator64-* to emulator package

Comment 8

[Edgar Chen [:edgar]]

Comment on attachment 8635151 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #153, Integrate standalone emulator build into full system build

Hi Thomas, 

We found a way to integrate standalone emulator build into full system build. In short, we create a target calls `emulator-standalone` and add this target to full system build, then copy building results (the host binary and library) to the proper place in out folder.

B2g can boot up success on emulator-l with using this way to build emulator. Try result for your reference: https://treeherder.mozilla.org/#/jobs?repo=try&revision=683baa8b76b6. You can download the emulator package (emulator.tar.gz) from task-inspector, https://tools.taskcluster.net/task-inspector/#42ejT8hlQLu0lgrYnSKHVw/0

May I have your review for this? Thank you.

BTW, I am now working on merging our patches from b2g-kitkat to b2g-studio-1.3, will provide a separated pull request for that.

Comment 9

[Edgar Chen [:edgar]]

Comment on attachment 8635152 [details] [review]
[platform_build:b2g-5.1.0_r1] PR#119, Build standalone emulator host binary

Please see comment #8.

Comment 10

[Edgar Chen [:edgar]]

Attachment: [b2g-manifest:master] PR #361, Switch external/qemu to b2g-studio-1.3 branch

Comment 11

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8635151 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #153, Integrate standalone emulator build into full system build

Thanks! I just hope this isn't too hard to port to future releases.

Comment 12

[Edgar Chen [:edgar]]

(In reply to Edgar Chen [:edgar][:echen] from comment #8)
> BTW, I am now working on merging our patches from b2g-kitkat to
> b2g-studio-1.3, will provide a separated pull request for that.

The first round merge is done: https://github.com/EdgarChen/platform_external_qemu/commits/bug_1181481/porting. I will do some simple test and send a pull request then.

Comment 13

[Edgar Chen [:edgar]]

Comment on attachment 8636473 [details] [review]
[b2g-manifest:master] PR #361, Switch external/qemu to b2g-studio-1.3 branch

Switch external/qemu to b2g-studio-1.3 branch. Could you help to review this, Thomas? Thank you.

Comment 14

[Edgar Chen [:edgar]]

Attachment: [platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

Comment 15

[Edgar Chen [:edgar]]

(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #11)
> Comment on attachment 8635151 [details] [review]
> [platform_external_qemu:b2g-studio-1.3] PR #153, Integrate standalone
> emulator build into full system build
> 
> Thanks! I just hope this isn't too hard to port to future releases.

Thank you, Thomas.
https://github.com/mozilla-b2g/platform_external_qemu/commit/85f91439c854061bda3c6228d98381ea8867170c

Comment 16

[Edgar Chen [:edgar]]

Comment on attachment 8637781 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

This pull request is to perform the first round merge to bring all our patches from b2g-kitkat to b2g-studio-1.3 [1]. I also provide some patches to adopt the upstream changes regarding to directory hierarchy and function names in our codes.

Basic function works good, e.g. system bootup, ftu, screen on, screen off, open settings ...,  but we still need to do more works to fully port RIL/Wifi/BT/NFC functions. The status will be tracking on following bugs,
- RIL: https://bugzilla.mozilla.org/show_bug.cgi?id=1134982
- WIFI: https://bugzilla.mozilla.org/show_bug.cgi?id=1179182
- BT: https://bugzilla.mozilla.org/show_bug.cgi?id=1182992
- NFC: https://bugzilla.mozilla.org/show_bug.cgi?id=1136190

Thank you.

[1] Try result for your reference: https://treeherder.mozilla.org/#/jobs?repo=try&revision=fdde17421e53.

Comment 17

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8637781 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

Hi Edgar,

Thanks! Just one question: can you rebase instead of merge our commits? There are 89 hits when searching for 'Merge' in the pull request's web page. Rebasing would remove the merge commits from the history.

Comment 18

[Edgar Chen [:edgar]]

Comment on attachment 8637781 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #17)
> Comment on attachment 8637781 [details] [review]
> [platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from
> b2g-kitkat to b2g-studio-1.3
> 
> Hi Edgar,
> 
> Thanks! Just one question: can you rebase instead of merge our commits?
> There are 89 hits when searching for 'Merge' in the pull request's web page.
> Rebasing would remove the merge commits from the history.

Sure, I will try to use rebase, cancel review first. Thank you.

Comment 19

[Edgar Chen [:edgar]]

Comment on attachment 8635152 [details] [review]
[platform_build:b2g-5.1.0_r1] PR#119, Build standalone emulator host binary

https://github.com/mozilla-b2g/platform_build/commit/07c383a786f188904311a37f6062c2cb84c9b61d

Comment 20

[Edgar Chen [:edgar]]

Comment on attachment 8636473 [details] [review]
[b2g-manifest:master] PR #361, Switch external/qemu to b2g-studio-1.3 branch

https://github.com/mozilla-b2g/b2g-manifest/commit/c3dd018015b6efea983585a62d2b962a391a39e0

Comment 23

[Edgar Chen [:edgar]]

Comment on attachment 8637781 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

I have finished the rebase and updated the pull request. Could you take a look first? I didn't test it yet, but will do it soon and send review request then. Thank you.

Just for your reference in the case you are interested in the difference between merge and rebase.
- merge version: https://github.com/EdgarChen/platform_external_qemu/tree/bug_1181481/porting_merge
- rebase version: https://github.com/EdgarChen/platform_external_qemu/tree/bug_1181481/porting_rebase

Comment 24

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8637781 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

Looks good. Thanks a lot!

Comment 25

[Josh Cheng [:josh]]

Ride on train of v2.5. Removing feature-b2g:2.5+ flag

Comment 26

[Edgar Chen [:edgar]]

Comment on attachment 8637781 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

Basic function works good, e.g. b2g boot up, ftu, screen on/off, open settings.
Try result for your reference: https://treeherder.mozilla.org/#/jobs?repo=try&revision=01572b9fd4b8.

Comment 27

[Ryan VanderMeulen [:RyanVM]]

(In reply to Edgar Chen [:edgar][:echen] from comment #19)
> Comment on attachment 8635152 [details] [review]
> [platform_build:b2g-5.1.0_r1] PR#119, Build standalone emulator host binary
> 
> https://github.com/mozilla-b2g/platform_build/commit/
> 07c383a786f188904311a37f6062c2cb84c9b61d

This broke b2g37. Assuming that the work in this bug is intended for master only at this point, I've gone ahead and pinned the v2.2 branch to the last good platform_build rev prior to this (rev 61e82f99bb8bc78d52b5717e9a2481ec7267fa33). If/when we find ourselves wanting to do more L-based work on v2.2, we'll need to revert this and fix the bustage.

v2.2: https://github.com/mozilla-b2g/b2g-manifest/commit/c4f99578a02051391a421c0e22996873a7601056

Comment 28

[Ryan VanderMeulen [:RyanVM]]

I link to a broken log would have been helpful I suppose...
https://treeherder.mozilla.org/logviewer.html#?job_id=161513&repo=mozilla-b2g37_v2_2

Comment 29

[Edgar Chen [:edgar]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #27)
> (In reply to Edgar Chen [:edgar][:echen] from comment #19)
> > Comment on attachment 8635152 [details] [review]
> > [platform_build:b2g-5.1.0_r1] PR#119, Build standalone emulator host binary
> > 
> > https://github.com/mozilla-b2g/platform_build/commit/
> > 07c383a786f188904311a37f6062c2cb84c9b61d
> 
> This broke b2g37. Assuming that the work in this bug is intended for master
> only at this point, I've gone ahead and pinned the v2.2 branch to the last

Exactly, this bug is intended for the master only at this point.

> good platform_build rev prior to this (rev
> 61e82f99bb8bc78d52b5717e9a2481ec7267fa33). If/when we find ourselves wanting
> to do more L-based work on v2.2, we'll need to revert this and fix the
> bustage.

I didn't aware this change will affect v2.2. Thanks for fixing the bustage, Ryan.
Filed bug 1188714 for l-base work on v2.2 if needed.

> 
> v2.2:
> https://github.com/mozilla-b2g/b2g-manifest/commit/
> c4f99578a02051391a421c0e22996873a7601056

Comment 30

[Edgar Chen [:edgar]]

Comment on attachment 8637781 [details] [review]
[platform_external_qemu:b2g-studio-1.3] PR #154, Bring our changes from b2g-kitkat to b2g-studio-1.3

Thank you, Thomas.

https://github.com/mozilla-b2g/platform_external_qemu/commit/59e434cbecc02653f44cedeb2ef5cc88dc8bb61b