Provide nightly builds of Ignite for Aries device

RESOLVED FIXED

Status

defect
P1
normal
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: wcosta, Assigned: wcosta)

Tracking

Details

(Whiteboard: [spark])

Attachments

(1 attachment, 1 obsolete attachment)

(Assignee)

Description

4 years ago
Aries device will be used for v3 dogfooding. Provide builds and OTA updates for it.
(Assignee)

Updated

4 years ago
Depends on: 1134722
Blocks: spark
Whiteboard: [lightsaber]
See Also: → 1112260
(Assignee)

Updated

4 years ago
Assignee: nobody → wcosta
Status: NEW → ASSIGNED
(Assignee)

Updated

4 years ago
Depends on: 1139066
(Assignee)

Updated

4 years ago
Depends on: 1141030
Wander and I talked about this on IRC for a while. We're facing a problem because the lightsaber build [1] process has some extra steps, using some node modules, which mozharness and the B2G repo [2] don't have support for. [3]

The way we see it, we have two options to solve this:

1. We overwrite the B2G repo's copy of Gaia with the un-built Lightsaber Gaia branch + pre-built custom apps, then run |B2G: ./build.sh| afterwards. We would provide a script that does this overwriting and pre-building. This is more work on mozharness and less for us.
2. We fork the B2G repo and move all of our custom build processes to the manifest, |./build.sh| script, etc. This is more work for Lightsaber people (probably me) as it will require basically redoing our build process and also maintaining a fork of the B2G repo.

We agreed that maintaining a fork of B2G sounds like a really bad idea. Thus we're going to do option 1.

I'm going to write this script to do the pre-building and overriding and provide it to Wander, who is working on adding support in mozharness for adding a script between the fetch->build steps.

[1] https://github.com/fxos/lightsaber
[2] https://github.com/mozilla-b2g/B2G
[3] https://github.com/fxos/lightsaber/blob/master/Makefile#L3-L18
Also note that we will need added support to mozharness for setting prefs. [1] Wander says that he will work on this as well. Perhaps we should file a separate bug for this.

[1] https://github.com/fxos/lightsaber#set-the-javascriptoptionsdiscardsystemsource-pref
Depends on: 1141191
(Assignee)

Updated

4 years ago
Depends on: 1145583
(Assignee)

Updated

4 years ago
Depends on: 1146294
(Assignee)

Updated

4 years ago
Blocks: 1146469
(Assignee)

Updated

4 years ago
Depends on: 1147815
(Assignee)

Updated

4 years ago
Depends on: 1147190
(Assignee)

Comment 3

4 years ago
/r/6559 - Bug 1134226: Add lightsaber nightly builds.

Pull down this commit:

hg pull -r e08708ab6ea7212c4cbb3d21d64e0941e6af3be5 https://reviewboard-hg.mozilla.org/gecko/
Attachment #8587598 - Flags: review?(garndt)
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

https://reviewboard.mozilla.org/r/6557/#review5445

::: testing/taskcluster/scripts/phone-builder/build-lightsaber-nightly.sh
(Diff revision 1)
> +  --complete-mar-url https://queue.taskcluster.net/v1/task/$TASK_ID/runs/0/artifacts/public/build/$mar_file \

So this is pulling the artifact from the first run (0), what happens if this run fails? Admittedly I see this issue in the other phone builder scripts as well and not something new that was introduced here.

::: testing/taskcluster/scripts/phone-builder/build-lightsaber-nightly.sh
(Diff revision 1)
> +  --config balrog/staging.py \

When would this be changed to prod? When the vpn proxy is setup? (I'm still working on that and hopeuflly will have that soon)

::: testing/taskcluster/scripts/phone-builder/build-lightsaber-nightly.sh
(Diff revision 1)
> +./mozharness/scripts/b2g_lightsaber.py \

Does this clone gaia at all?  I know that there were some changes elsewhere to mozharness commands to specify something like "no-pull" and let tc-vcs pull gaia.

::: testing/taskcluster/tasks/builds/b2g_aries_lightsaber_nightly.yml
(Diff revision 1)
> +  workerType: flame-kk

Is there any value in moving this into it's own "lightsaber" worker type?

::: testing/docker/phone-builder/Dockerfile
(Diff revision 1)
> +RUN           pip install awscli

So each one of these lines starting with "RUN" will create a layer inthe image.  Would it be better to either combine them all in one line "npm install -g bower gulp ..." or create a package.json that will get installed in one layer?
Attachment #8587598 - Flags: review?(garndt)
Flag me again for review once we sort out the few questions in comment 4
(Assignee)

Comment 6

4 years ago
https://reviewboard.mozilla.org/r/6557/#review5459

> Does this clone gaia at all?  I know that there were some changes elsewhere to mozharness commands to specify something like "no-pull" and let tc-vcs pull gaia.

mozharness itself doesn't, but config.sh does. But this is useless, because lightsaber will clone its own gaia branch.

> When would this be changed to prod? When the vpn proxy is setup? (I'm still working on that and hopeuflly will have that soon)

Yep.

> So this is pulling the artifact from the first run (0), what happens if this run fails? Admittedly I see this issue in the other phone builder scripts as well and not something new that was introduced here.

Nop, this is used when submiting to balrog. You tell balrog where to find the generated mar file.

> Is there any value in moving this into it's own "lightsaber" worker type?

Sooner or later, lightsaber will be merged into gaia (I guess) and we won't need specific stuff to build it. So when this time come, having things in separate files will make the cleanup easier.
(Assignee)

Comment 7

4 years ago
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

/r/6559 - Bug 1134226: Add lightsaber nightly builds.

Pull down this commit:

hg pull -r 5a44195264d5ce6be261ef57152b5676b300d2d0 https://reviewboard-hg.mozilla.org/gecko/
Attachment #8587598 - Flags: review?(garndt)
https://reviewboard.mozilla.org/r/6557/#review5469

> mozharness itself doesn't, but config.sh does. But this is useless, because lightsaber will clone its own gaia branch.

So is there no value in having tc-vcs do this so that it's cached?

> Nop, this is used when submiting to balrog. You tell balrog where to find the generated mar file.

But if run 0 failed, there will be no mar file....
(Assignee)

Updated

4 years ago
Blocks: 1148630
(Assignee)

Comment 9

4 years ago
https://reviewboard.mozilla.org/r/6557/#review5471

> So is there no value in having tc-vcs do this so that it's cached?

Nop.
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

https://reviewboard.mozilla.org/r/6557/#review5473

::: testing/taskcluster/scripts/phone-builder/build-lightsaber-nightly.sh
(Diff revision 2)
> +  --complete-mar-url https://queue.taskcluster.net/v1/task/$TASK_ID/runs/0/artifacts/public/build/$mar_file \

So here we might be able to do something similar to: 
https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/mach_commands.py#35

Where if you call this endpoint without a runId, it will grab the latest artifact for that task.
https://github.com/taskcluster/taskcluster-queue/blob/master/routes/artifacts.js#L428

I think this might ensure that if for some reason run 0 was aborted (worker-shutdown, etc) that when run 1 succeeds, this task will know where to find the artifact for the successful run (run 1) instead of hard code to 0.
Attachment #8587598 - Flags: review?(garndt)
(Assignee)

Comment 11

4 years ago
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

/r/6559 - Bug 1134226: Add lightsaber nightly builds.

Pull down this commit:

hg pull -r f8bb075117a6b89e3ecf5cb41b3dfad6f6fe3caf https://reviewboard-hg.mozilla.org/gecko/
Attachment #8587598 - Flags: review?(garndt)
Just to be clear, when the nightly builds are enabled am I going to get a normal build or a lightsaber one?
(Assignee)

Comment 13

4 years ago
(In reply to Dale Harvey (:daleharvey) from comment #12)
> Just to be clear, when the nightly builds are enabled am I going to get a
> normal build or a lightsaber one?

Lightsaber, but if you need normal builds, just file a bug and I can provide you.
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

r+ provided a link to a successful aries build using this configuration is added to the bug.
Attachment #8587598 - Flags: review?(garndt) → review+
(Assignee)

Comment 15

4 years ago
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

/r/6559 - Bug 1134226: Add lightsaber nightly builds.

Pull down this commit:

hg pull -r b4749202f328bb2375f45ac300912348c1afb3bb https://reviewboard-hg.mozilla.org/gecko/
Attachment #8587598 - Flags: review+ → review?(garndt)
(Assignee)

Comment 16

4 years ago
(In reply to Greg Arndt [:garndt] from comment #14)
> Comment on attachment 8587598 [details]
> MozReview Request: bz://1134226/wcosta
> 
> r+ provided a link to a successful aries build using this configuration is
> added to the bug.

I needed apply some fixes to the original patch. Would you mind reviewing again?
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

https://reviewboard.mozilla.org/r/6557/#review5719

::: testing/taskcluster/scripts/phone-builder/build-lightsaber-nightly.sh
(Diff revision 4)
> +#mv $WORKSPACE/B2G/upload/b2g-*.crashreporter-symbols.zip $HOME/artifacts/b2g-crashreporter-symbols.zip

Do we not need crashreporter symbols?  There was some work that James was just doing to make sure our builds have them.  Didn't know if this was the case for nightlies.

::: testing/taskcluster/scripts/phone-builder/pre-build.sh
(Diff revision 4)
> +rm -rf $WORKSPACE/B2G/objdir*

So it appears that we are using device specific caches in the task (build-aries-lightsaber-nightly) but we're going to remove obj caches for every device?  Aries nightly builds can't use cached obj dirs?  Interested to know what the problem might be.

Also, if the aires nightly is the only place we want to blow away the cache, maybe there is an ENV flag we set in the nightly task that the pre-build script looks for before blowing it away.
Attachment #8587598 - Flags: review?(garndt)
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

r+ with the comment about caches being addressed.  Would not like to see the caches removed for all phone builds if not necessary.

Also, could you add a comment with a taskcluster task using this new diff?
Attachment #8587598 - Flags: review+
(Assignee)

Comment 19

4 years ago
https://reviewboard.mozilla.org/r/6557/#review5721

> Do we not need crashreporter symbols?  There was some work that James was just doing to make sure our builds have them.  Didn't know if this was the case for nightlies.

This is because mozharness tries to upload symbols in nightly builds, and this case the build would fail. Bug 1148630 will address this issue.

> So it appears that we are using device specific caches in the task (build-aries-lightsaber-nightly) but we're going to remove obj caches for every device?  Aries nightly builds can't use cached obj dirs?  Interested to know what the problem might be.
> 
> Also, if the aires nightly is the only place we want to blow away the cache, maybe there is an ENV flag we set in the nightly task that the pre-build script looks for before blowing it away.

I was reported that caching aries binaries causes wifi issues. I decided to play safe and not caching binaries for all devices, because I don't know if there are issues for other devices (current or future). Not sure what is the better approach, however.
(Assignee)

Comment 20

4 years ago
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

/r/6559 - Bug 1134226: Add lightsaber nightly builds.

Pull down this commit:

hg pull -r 264e390b0eb0abb87bce410ceb44e56a8ef8940c https://reviewboard-hg.mozilla.org/gecko/
Attachment #8587598 - Flags: review+ → review?(garndt)
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

https://reviewboard.mozilla.org/r/6557/#review5751

::: testing/taskcluster/scripts/phone-builder/build-lightsaber-nightly.sh
(Diff revision 5)
> +  rm -rf $WORKSPACE/B2G/objdir*

I think this would be ok, if there is no other way we could accomplish it.  Is there a way you can tell mozharness to clobber things?  does that take care of both directories?
Attachment #8587598 - Flags: review?(garndt)
(Assignee)

Comment 23

4 years ago
https://reviewboard.mozilla.org/r/6557/#review5775

> I think this would be ok, if there is no other way we could accomplish it.  Is there a way you can tell mozharness to clobber things?  does that take care of both directories?

mozharness has clobbering stuff, but I gave it a try and it doesn't work as expected, it will delete the whole tree.
(Assignee)

Comment 24

4 years ago
Comment on attachment 8587598 [details]
MozReview Request: bz://1134226/wcosta

/r/6559 - Bug 1134226: Add lightsaber nightly builds.

Pull down this commit:

hg pull -r 252ce8ca5a9b75e33d69e05f7ac45149e3a9d020 https://reviewboard-hg.mozilla.org/gecko/
Attachment #8587598 - Flags: review?(garndt)
(Assignee)

Comment 25

4 years ago
Added nightly to cypress branch.
(Assignee)

Comment 26

4 years ago
Also configured indexing.
Attachment #8587598 - Flags: review?(garndt) → review+
https://hg.mozilla.org/mozilla-central/rev/c37edf9eaedc
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla40
Whiteboard: [lightsaber] → [ignite]
See Also: → 1154767
Summary: Provide nightly builds for aries device → Provide nightly builds of Ignite for Aries device
Whiteboard: [ignite] → [spark]
Blocks: spark-ota
No longer blocks: spark
(Assignee)

Comment 29

4 years ago
Attachment #8587598 - Attachment is obsolete: true
Attachment #8619511 - Flags: review+
Component: TaskCluster → General
Product: Testing → Taskcluster
Target Milestone: mozilla40 → mozilla41
Version: unspecified → Trunk
Resetting Version and Target Milestone that accidentally got changed...
Target Milestone: mozilla41 → ---
Version: Trunk → unspecified
You need to log in before you can comment on or make changes to this bug.