878122 - Please create Nightly B2G Unagi engineering build against mozilla-central

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Zac C (:zac)]

For ui-test automation can you create a Nightly engineering build against mozilla-central

Please include a generic zip file name and in a `latest/` directory.

Basically we need the same as this build but with the Engineering flag:
https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-central-unagi/latest/

Comment 1

[Aki Sasaki (not active)]

Attachment: unagi-eng on m-c

Comment 2

[Rail Aliiev [:rail]]

Comment on attachment 756664 [details] [diff] [review]
unagi-eng on m-c

lgtm

Comment 3

[Aki Sasaki (not active)]

Comment on attachment 756664 [details] [diff] [review]
unagi-eng on m-c

https://hg.mozilla.org/build/buildbot-configs/rev/7ce252a672fa

Comment 4

[Zac C (:zac)]

That was quick, thanks!

Comment 5

[Rail Aliiev [:rail]]

in production

Comment 6

[Zac C (:zac)]

I can't find it; where are you hiding it?

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

I see two pending 'b2g_mozilla-central_unagi_eng_nightly' jobs on 
  https://tbpl.mozilla.org/?showall=1&jobname=eng
but not any builders on the buildbot masters to carry those out.

Comment 8

[Aki Sasaki (not active)]

Looks like a 2nd reconfig has gotten these running.

Comment 9

[Aki Sasaki (not active)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=23728943&tree=Mozilla-Central#error1

Not sure why we're trying to use mozdevice here?

Comment 10

[Aki Sasaki (not active)]

I think this is dying when trying to create the flash zip for unagi-eng on m-c.
Marshall: is the appropriate code checked into m-c for this?

Comment 11

[Aki Sasaki (not active)]

[15:01]	<jgriffin>	it looks like there's a script which is used for staging updates that uses a part of Marionette

[15:02]	<jgriffin>	aki: I think the in-tree version of Marionette in mozilla-b2g doesn't have that dependency

[15:02]	<aki>	jgriffin: ok. that would make sense since we're building unagi-eng on b2g18* but not m-c yet

[15:03]	<aki>	jgriffin: what should i do? ask for that dependency to be removed, or add a mozdevice/marionette virtualenv to the build, or ?
[15:03]	<jgriffin>	the latter, if possible

[15:05]	<aki>	jgriffin: should build.sh handle that, or do we need that logic in mozharness

[15:05]	<jgriffin>	I don't know anything about build.sh, but theoretically it should be easy to get mozharness to do this
[15:06]	<aki>	yeah, i need to know where to find those bits, what modules are required, and then call the flashing target with the appropriate python
[15:06]	<aki>	and hopefully it would work across all branches, so i don't have to special case m-c
[15:07]	<jgriffin>	I think the same logic should work across trees
[15:08]	<jgriffin>	basically it needs testing/marionette/client, and the mozbase packages (from testing/mozbase) that are in setup.py tehre

Comment 12

[Aki Sasaki (not active)]

Comment 11 is complicated by the fact that we will have to create the virtualenv inside mock.

Comment 13

[Aki Sasaki (not active)]

We don't currently have setup_development.py support in mozharness, but I can point to each mozbase module individually.

Comment 14

[Aki Sasaki (not active)]

(In reply to Aki Sasaki [:aki] from comment #13)
> We don't currently have setup_development.py support in mozharness, but I
> can point to each mozbase module individually.

(v)deathduck:/src/unagi-eng-mc [16:34:58] 
610$ pip install /src/clean/mozilla-central/testing/mozbase
("Directory %r is not installable. File 'setup.py' not found.", '/src/clean/mozilla-central/testing/mozbase')
Storing complete log in /Users/asasaki/.pip/pip.log

Moving setup_development.py to setup.py could fix this.  Proceeding with a verbose listing of mozbase modules for now.

Comment 15

[Aki Sasaki (not active)]

Attachment: [needs testing] create venv for unagi-eng

It seems more likely I'm going to break something than fix this bug + not affect any other branches, so I'm going to do some testing first.

Comment 16

[Phil Ringnalda (:philor)]

Hidden on tbpl so we don't have to stare at its glaring red face.

Comment 17

[Aki Sasaki (not active)]

Looks like we're creating the virtualenv now, but it's not in mock.
I noticed we're creating a ./_virtualenv/bin/python in mock through build.sh; I'm trying to use that now.

Comment 18

[Aki Sasaki (not active)]

... and this won't work on b2g18 anyway, since moznetwork doesn't exist there:
http://hg.mozilla.org/releases/mozilla-b2g18/file/997cdbf5d012/testing/mozbase

Could we remove the dependency for marionette/mozdevice in m-c, please?

Comment 19

[Aki Sasaki (not active)]

Needinfo?ing :jgriffin for comment 18.

Comment 20

[Aki Sasaki (not active)]

Attachment: venv take 2

Completely bypasses mozharness.base.python.VirtualenvMixin.create_virtualenv(), since we're in a mock environment and have to deal with setup_development.py which doesn't behave with pip.

Comment 21

[Jonathan Griffin (:jgriffin)]

(In reply to Aki Sasaki [:aki] from comment #18)
> ... and this won't work on b2g18 anyway, since moznetwork doesn't exist
> there:
> http://hg.mozilla.org/releases/mozilla-b2g18/file/997cdbf5d012/testing/
> mozbase
> 
> Could we remove the dependency for marionette/mozdevice in m-c, please?

I'd be more inclined to land moznetwork in mozilla-b2g18; would that be a viable alternative?

Comment 22

[Aki Sasaki (not active)]

Let me test my latest patch; if that works we may not need to.
Fingers crossed.

Comment 23

[Jonathan Griffin (:jgriffin)]

Or do you mean, remove the marionette dependency from stage-update.py?

Comment 24

[Aki Sasaki (not active)]

(In reply to Jonathan Griffin (:jgriffin) from comment #23)
> Or do you mean, remove the marionette dependency from stage-update.py?

If that happened, unagi-eng nightlies on m-c would Just Work without additional b2g_build.py changes, so yes, that would be nice.

I'm not sure what that dependency is giving us, however, so I can't say whether we need it.

Comment 25

[Jonathan Griffin (:jgriffin)]

It's not giving us much, tbh, I can make a hack to remove it.

Comment 26

[Aki Sasaki (not active)]

Cool.  dev-master01 just died, so I still don't know if my latest update to attachment 758039 [details] [diff] [review] helps or not.

Comment 27

[Aki Sasaki (not active)]

Attachment: working mozharness virtualenv patch

If we still want to create a mozbase virtualenv inside mock, this patch will do it.  I think we're good once m-i merges into m-c, though.

Comment 28

[Aki Sasaki (not active)]

(Testing that patch against m-b2g18 just in case.)

Comment 29

[Aki Sasaki (not active)]

(In reply to Aki Sasaki [:aki] from comment #28)
> (Testing that patch against m-b2g18 just in case.)

This went green in staging.

Jonathan's patch merged to m-c; re-kicked the unagi-eng m-c nightly.

Comment 30

[Aki Sasaki (not active)]

https://pvtbuilds.mozilla.org/pub/mozilla.org/b2g/nightly/mozilla-central-unagi-eng/latest/
https://tbpl.mozilla.org/php/getParsedLog.php?id=23831038&tree=Mozilla-Central&full=1

\o/

Comment 31

[Zac C (:zac)]

Thanks Aki, so this is ready for human consumption?

I've flashed it and it's alive but I just want to know whether it's OK for me to start raising bugs.