B2G Emulator-x86: failed to run xpcshell tests: zip not available

RESOLVED WONTFIX

Status

Firefox OS
Emulator
RESOLVED WONTFIX
5 years ago
4 years ago

People

(Reporter: vicamo, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

5 years ago
Created attachment 752095 [details]
err.log

We installs prebuilt busybox binary for running xpcshell tests. However, that binary comes with only ARM version and surely doesn't run on x86 platform.
(Reporter)

Comment 1

5 years ago
Found we have already busybox port on GitHub, maybe build busybox from source and deprecate prebuilt binary in Gaia. @mwu, what do you think?
Flags: needinfo?(mwu)

Comment 2

5 years ago
Busybox doesn't build on OSX.

I wonder why we're using busybox for tests though. It would be nice to use things which are always available on the phone.
Flags: needinfo?(mwu)
(Reporter)

Comment 3

5 years ago
Like the title suggests, it seems we need zip/unzip for pushing local files to device in mozbase/mozdevice/mozdevice/devicemanagerADB.py.[1]  However, the code shows we can fallback to `adb push` when zip is not available, but somehow it just raise an error instead.[2]

[1]: https://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozdevice/mozdevice/devicemanagerADB.py#198
[2]: https://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozdevice/mozdevice/devicemanagerADB.py#743
(Reporter)

Comment 4

5 years ago
Created attachment 753621 [details] [diff] [review]
Github PR for B2G.git

The prebuilt busybox executable is only for ARM. Pushing such executable to emulator-x86 is meaningless and causes unexpected errors in python scripts.

Github PR link: https://github.com/mozilla-b2g/B2G/pull/248
Attachment #753621 - Flags: review?(mwu)
Attachment #753621 - Flags: review?(jgriffin)
(Reporter)

Comment 5

5 years ago
With busybox installation skipped, xpcshell tests run on emualtor-x86, but test data uploading is extremely slow.  Maybe we can use `gzip` instead in this case.  Gzip should be available in every Android device.
Yes, we use busybox because pushing all the xpcshell test files to device takes up to ~20-30 minutes, whereas zipping them up and pushing a single zip file takes ~5.

My understanding was that we didn't have any form of compression on the emulators which is why we needed busybox, though if I'm mistaken, then yes let's just use that.

Updated

5 years ago
Attachment #753621 - Flags: review?(mwu) → review?(dhylands)

Comment 7

5 years ago
FWIW, we shouldn't bother adb pushing anything for emulator tests if at all possible. We can put files into out/target/product/generic/data and generate a partition which already has all the files we need.
Right now the emulator isn't getting built per push, so we'd need to wait for bug 872765 to roll out to all the branches first.

Comment 9

5 years ago
We don't need to build the emulator to generate a custom userdata image. Having the standard build machinery makes it easier, but we can just grab the image generating tools and generate the image ourselves. Though, if bug 872765 is rolling out soon, it doesn't matter too much.
There is a gzip command in the emulator, but it seems to only work with files, not directories.  mwu, is there a way to compress/decompress directories without busybox?

Building a custom userdata image would require substantial changes to the automation, so is definitely out of scope for this bug.
Comment on attachment 753621 [details] [diff] [review]
Github PR for B2G.git

It would probably be better to add an x86 version of busybox to gaia (from http://busybox.net/downloads/binaries/1.19.0/), or download it from within this script.
Attachment #753621 - Flags: review?(jgriffin) → review-
(In reply to Jonathan Griffin (:jgriffin) from comment #11)
> It would probably be better to add an x86 version of busybox to gaia (from
> http://busybox.net/downloads/binaries/1.19.0/), or download it from within
> this script.

Probably want v1.20.0 though:
http://busybox.net/downloads/binaries/latest/
Comment on attachment 753621 [details] [diff] [review]
Github PR for B2G.git

Review of attachment 753621 [details] [diff] [review]:
-----------------------------------------------------------------

Not sure I can add much more. You really need something like tar or zip to be able to bundle multiple files into one. gzip is a single file compresser.

I also discovered that with adb and real devices, you can push about 5 or 6 streams in parallel and get significantly better results. I didn't test that with the emulator.
Attachment #753621 - Flags: review?(dhylands)
(Reporter)

Comment 14

5 years ago
Gaia PR for busybox: https://github.com/mozilla-b2g/gaia/pull/2116

Any suggestion to add another busybox binary of x86 arch, or compile it from source like https://github.com/mozilla-b2g/busybox did?

Comment 15

5 years ago
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #14)
> Any suggestion to add another busybox binary of x86 arch, or compile it from
> source like https://github.com/mozilla-b2g/busybox did?

The main problem with building busybox is that it used to fail to build on OSX. If you can get it building on OSX, I don't mind building from source. Otherwise, we'll probably want another busybox binary.
(Reporter)

Updated

5 years ago
Blocks: 905098
(Reporter)

Updated

5 years ago
Component: General → Emulator
(Reporter)

Comment 16

4 years ago
If I push http://www.busybox.net/downloads/binaries/latest/busybox-i686 to "gaia/build/busybox-armv6l", then xpcshell tests run and pass successfully.
(Reporter)

Comment 17

4 years ago
Bug 821012 fetches busybox binary from internet instead, bug 932362 deprecates `./test.sh xpcshell`, and bug 982159 is to remove it.  As a result, importing busybox to emulator becomes not so meaningful and we should file another bug to assure `./mach xpcshell` also works for emulator-x86.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.