932562 - Allow running gtests with a b2g build

Status: RESOLVED WONTFIX

(Testing :: General, defect)

Description

[Botond Ballo [:botond]]

If I understand correctly, running gtests (such as gfx/tests/gtest/TestAsyncPanZoomController.cpp) requires a desktop build. I think this is unfortunate if you're doing B2G development and otherwise have no reason to maintain a desktop build.

It would be great if we could run gtests using a B2G build.

Comment 1

[Jonathan Griffin (:jgriffin)]

Is running gtests on a B2G build merely a convenience, or would we be testing things that we don't have coverage with if we just run with desktop builds?

Comment 2

[Botond Ballo [:botond]]

(In reply to Jonathan Griffin (:jgriffin) from comment #1)
> Is running gtests on a B2G build merely a convenience, or would we be
> testing things that we don't have coverage with if we just run with desktop
> builds?

For now, it's just a convenience (a significant one IMO given how long builds take), but b2g-specific gtests could conceivably be written in the future.

Comment 3

[Andrew Halberstadt [:ahal]]

I think Josh was looking into this already. Not sure if he made any progress or not.

Comment 4

[Josh Aas]

I was looking into it, made some progress, but it turns out to be unnecessary for my work to continue now so I'm not working on it any more. Sorry.

Comment 5

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

We are implementing b2g specific video encoding module and it needs to have a b2g emulator environment to test our modules.

Comment 6

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

Hi Benoit, 
Do we have plan to enable it? 
We want to write some c++ test cases for modules work on b2g platform.

Comment 7

[Benoit Girard (:BenWa)]

No, either you would have to staff this or maybe ask ATeam to look at this.

Comment 8

[JW Wang [:jwwang] [:jw_wang]]

Attachment: part1_install_gtest_files.patch

Here is the manual way to run gtest on b2g (tested on the emulator):

1. apply the patch to install gtest files when packing b2g
2. go to the object directory of gecko (ex: objdir-gecko)
3. make -C toolkit/library gtestxul (build gtest libraries)
4. make package (pack gtest files)
5. launch emulator or device
6. adb remount && adb push dist/b2g/dependentlibs.list.gtest /system/b2g && adb push dist/b2g/gtest/libxul.so /system/b2g/gtest/libxul.so (deploy gtest files onto the device/emulator)
7. adb shell stop b2g && adb shell "export MOZ_RUN_GTEST=1 && /system/bin/b2g.sh" (restart b2g process and run gtest)

Comment 9

[Randy Lin [:rlin] - Be Mozillian on 2015/01/26]

Hi Benwa, 
Sorry to bother you. 
Do you have bandwith to do this task?

Comment 10

[JW Wang [:jwwang] [:jw_wang]]

Hi Benwa,
Please see comment 8. We can now run gtest on b2g following the steps listed in comment 8. We need someone familiar with test scripts to integrate these manual steps into automation. Thanks.

Comment 11

[Benoit Girard (:BenWa)]

The steps in Comment 8 look great. Having this run in automation shouldn't be too much effort if the current tests don't fail on b2g. But I currently have other bugs I'm looking into so I don't have time to look at this.

Comment 12

[(not currently active) Ted Mielczarek]

I think dminor had been thinking about doing the work to run gtests from packaged builds (bug 992983), he might be interested in picking this up. Most of the work in comment 8 is just the work necessary to get gtests running from packaged builds, which is desirable for other reasons.

Comment 13

[JW Wang [:jwwang] [:jw_wang]]

I think it is more convenient to build gtest files into libxul when an env variable is on (ex: export ENALBE_GTEST=1 in .userconfig) instead of building another libxul which contains gtest files. By doing so, we can just call |./flash.sh gecko| to deploy the binaries to the device.  Btw, I don't see much difference between the file size of stripped libxul with/without gtest files.

Comment 14

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

David, I am close to getting GTests working on Windows, but I won't have time to get it working on B2G. Could you try to drive this from your end by letting your management know that it is important for mozilla::pkix and other PSM stuff?

Comment 15

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

David, never mind. I have another idea for getting these tests working on B2G and solving other (performance) problems. I will talk to you about it via email later this week.

Comment 16

[StevenLee[:slee]]

Hi Brian,

Is there any updates about this bug?
Thanks.

Comment 17

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to StevenLee[:slee] from comment #16)
> Is there any updates about this bug?

No, I'm not working on it.

Comment 18

[StevenLee[:slee]]

This is my plan.

1. Making B2G build system can compile gtest by check an extra flag, B2G_ENABLE_GTEST.
2. Adding one script, run-gtest.sh, to check whether libxul.so with gtest existing then run b2g process with "MOZ_RUN_GTEST=1"

Comment 19

[StevenLee[:slee]]

Attachment: part1: build gtest from gonk-misc

Hi mwu,

Can you help review this patch?
Thanks

Comment 20

[StevenLee[:slee]]

Attachment: part2: helper script to run gtest

Hi mwu, 
This is another patch for adding helper script to run gtest.

Comment 21

[Michael Wu [:mwu]]

Comment on attachment 8469886 [details] [review]
part1: build gtest from gonk-misc

You need to make this change on the gecko build system side.

Comment 22

[StevenLee[:slee]]

Attachment: part 1 - build gtest from mach

From comment 21, mwu suggested building gtest from gecko side.

Hi ted,

Could you help review this patch?
Thanks

Comment 23

[(not currently active) Ted Mielczarek]

Comment on attachment 8471324 [details] [diff] [review]
part 1 - build gtest from mach

Review of attachment 8471324 [details] [diff] [review]:
-----------------------------------------------------------------

I'm going to punt this to gps as he has more insight on mach commands.

Comment 24

[Gregory Szorc [:gps]]

Comment on attachment 8471324 [details] [diff] [review]
part 1 - build gtest from mach

Review of attachment 8471324 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not a huge fan of this approach.

First, the command name "build-gtest" is a bit misleading. While it does "build," that's not the main intent of the command and that's confusing.

The approach I'd like to see is |mach gtest| doing the right thing, preferably automatically. If we can detect that the build is intended for b2g and an emulator is available, and there's no reason to do a local gtest run (I'm guessing that's not even possible since you'd be cross-compiling, right?), then we should do all the steps in this patch automatically. In other words, mach gtest should have separate code paths depending on the environment. It should abstract away implementation details and "just work." If we can't automatically do the right thing, we should introduce a --remote flag to force remote execution. Worst case, we add |mach gtest-remote| to mirror the existing convention.

What do you think?

Comment 25

[Michael Wu [:mwu]]

Comment on attachment 8469887 [details] [review]
part2: helper script to run gtest

Clearing review for now until there is agreement on how to build gtest.

Comment 26

[Botond Ballo [:botond]]

Steven, do you plan on continuing to work on this?

I'd really like to see this fixed because I do my day-to-day development on a b2g build, and not being able to run gtests on that build means I have to choose between:

  - also maintaining a desktop build and updating it all the time
  - always pushing to Try (overkill for small, localized patches)
  - landing on inbound without confidence that gtests pass, risking a backout

Comment 27

[Botond Ballo [:botond]]

(I personally have stopped caring about this as I now mostly work with desktop builds, but this ability is still likely to be useful for others.)