Closed Bug 1204800 Opened 8 years ago Closed 7 years ago

Install tooltool packages for B2G Device Image builds

Categories

(Testing :: General, defect)

Product:
Testing
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox44 affected)

Status:
RESOLVED DUPLICATE of bug 1282093
Milestone:
mozilla44
Tracking Flags:
Tracking Status
firefox44 --- affected

People

(Reporter: jdai, Unassigned)

References

Details

Attachments

(1 file, 6 obsolete files)

Add tooltool dependencies to phone builds. r=wcosta
29.28 KB, patch
jdai
: review+
Details | Diff | Splinter Review 
According to bug 1146713 comment 150, we should run install-packages.sh for b2g device image builds as we do for b2g desktop.
Comment on attachment 8661695 [details] [diff] [review]
Add tooltool dependencies to phone builds.

Review of attachment 8661695 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm
Attachment #8661695 - Flags: review?(wcosta) → review+
Keywords: checkin-needed
I saw you have 3 commit in a run [1]. After I saw error message, I think the error is came from other patch, I will make sure my patch is works fine [2]. 

[1] https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=4717107189fe
[2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=b1ffeab8b6c5
Flags: needinfo?(jdai)
All the phone build are pass. 

Try link:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b1ffeab8b6c5
Keywords: checkin-needed
sorry had to back this out for build bustage like https://treeherder.mozilla.org/logviewer.html#?job_id=14418785&repo=mozilla-inbound

01:03 < nigelb|sheriffduty> those oranges are pretty consistent.
01:04 < nigelb|sheriffduty> test -n "$TOOLTOOL_MANIFEST"
01:04 < nigelb|sheriffduty> aha
01:04 < nigelb|sheriffduty> the $TOOLTOOL_MANIFEST is empty there.
Flags: needinfo?(jdai)
(In reply to Carsten Book [:Tomcat] from comment #9)
> sorry had to back this out for build bustage like
> https://treeherder.mozilla.org/logviewer.html#?job_id=14418785&repo=mozilla-
> inbound
> 
> 01:03 < nigelb|sheriffduty> those oranges are pretty consistent.
> 01:04 < nigelb|sheriffduty> test -n "$TOOLTOOL_MANIFEST"
> 01:04 < nigelb|sheriffduty> aha
> 01:04 < nigelb|sheriffduty> the $TOOLTOOL_MANIFEST is empty there.

Hi :Tomcat,
Thank you catch this error. May I ask why only mozilla-inbound has Nexus 4 and Nexus 5-L phone build? Do we miss to add them in try build?
Flags: needinfo?(jdai) → needinfo?(cbook)
Add tooltool dependencies in Nexus 4 and Nexus 5-L phone build.
Attachment #8661695 - Attachment is obsolete: true
Attachment #8664010 - Flags: review+
hey john, actually a good question. I don't know but maybe the guys in #b2g or #developers know how to run try runs on device builds
Flags: needinfo?(cbook)
Keywords: checkin-needed
Remove backup-flame.tar.xz configuration to avoid extracting a file which does not exist.

Try link:
https://treeherder.allizom.org/#/jobs?repo=try&revision=429e45e73de8
Attachment #8664783 - Attachment is obsolete: true
Flags: needinfo?(jdai)
Attachment #8667848 - Flags: review+
Keywords: checkin-needed
(In reply to Nigel Babu [:nigelb] from comment #22)
> Backed out for bustage in Nexus 4 KK builds in:
> 
> https://hg.mozilla.org/integration/mozilla-inbound/rev/14031459c9af
> 
> Bustage logs:
> 
> https://treeherder.mozilla.org/logviewer.html#?job_id=15026621&repo=mozilla-
> inbound

I didn't notice that we add a Nexus 4 KK on TaskCluster(Bug 1206368) that's why it cause this backed out. I will fix in next patch.
Add Nexus 4 KK configuration.

Try link:
https://treeherder.allizom.org/#/jobs?repo=try&revision=4bd9933b3254
Attachment #8667848 - Attachment is obsolete: true
Attachment #8668411 - Flags: review+
Keywords: checkin-needed
Hi, this failed to apply:

adding 1204800 to series file
renamed 1204800 -> bug_1204800_phone.patch
applying bug_1204800_phone.patch
patching file testing/taskcluster/scripts/phone-builder/pre-build.sh
Hunk #1 FAILED at 20
1 out of 1 hunks FAILED -- saving rejects to file testing/taskcluster/scripts/phone-builder/pre-build.sh.rej

can you take a look, thanks!
Flags: needinfo?(jdai)
rebase
Attachment #8668411 - Attachment is obsolete: true
Flags: needinfo?(jdai)
Attachment #8670102 - Flags: review+
Flags: needinfo?(cbook)
done :)
Flags: needinfo?(cbook)
maybe we would need a full try before this gets checked in again
I saw flame kitkat device image build (engineering) is a buildbot build and there has some conflict with TC build[1]. Do we really need this in mozilla-inbound? If we need to keep this, how to avoid the conflict?

[1] https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=e58169d5f96a&filter-searchStr=B2G%20Device%20Image%20opt%20Flame%20KitKat%20Device%20Image%20Flame%20KitKat%20Device%20Image%20Build%20%28Engineering%29%20Flame-KK%28Be%29
Flags: needinfo?(jdai) → needinfo?(wcosta)
From Comment #22 to Comment #27, I only modified Nexus 4 kk configuration and rebased. It should not have different situation which is Comment #22 Flame-KK engineering build success, but Comment #27 Flame-KK engineering build failed.
(In reply to John Dai[:johnz][:jdai] from comment #33)
> How to use try testing Flame-KK engineering build on Treeherder[1]? Because
> I didn't find out any configuration in testcluster[2].
> 
> [1]
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> inbound&revision=e58169d5f96a&filter-
> searchStr=B2G%20Device%20Image%20opt%20Flame%20KitKat%20Device%20Image%20Flam
> e%20KitKat%20Device%20Image%20Build%20%28Engineering%29%20Flame-KK%28Be%29
> [2]
> https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/
> branches/base_jobs.yml?from=base_jobs.yml

"try: -p flame-kk-eng -b o" should work
Flags: needinfo?(wcosta)
(In reply to Wander Lairson Costa [:wcosta] from comment #34)
> (In reply to John Dai[:johnz][:jdai] from comment #33)
> > How to use try testing Flame-KK engineering build on Treeherder[1]? Because
> > I didn't find out any configuration in testcluster[2].
> > 
> > [1]
> > https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> > inbound&revision=e58169d5f96a&filter-
> > searchStr=B2G%20Device%20Image%20opt%20Flame%20KitKat%20Device%20Image%20Flam
> > e%20KitKat%20Device%20Image%20Build%20%28Engineering%29%20Flame-KK%28Be%29
> > [2]
> > https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/tasks/
> > branches/base_jobs.yml?from=base_jobs.yml
> 
> "try: -p flame-kk-eng -b o" should work

It shows [tc] b2g flame kk eng not flame kitkat device image build (engineering)[1].

Hi :jgriffin,
Do you know how to enable flame kitkat device image build (engineering)[2] on try? How can I know this build is passed? Thank you.

[1] https://treeherder.allizom.org/#/jobs?repo=try&revision=893cb67ce9f3
[2] https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=e58169d5f96a&filter-searchStr=B2G%20Device%20Image%20opt%20Flame%20KitKat%20Device%20Image%20Flame%20KitKat%20Device%20Image%20Build%20%28Engineering%29%20Flame-KK%28Be%29
Flags: needinfo?(jgriffin)
Passing in the flags that Wander explained before should work.  Here is a push I just did:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c3df8f9a9ab0

The Be symbol is for the flame-kk engineering build.
Flags: needinfo?(garndt)
Looks like garndt answered, so clearing ni.
Flags: needinfo?(jgriffin)
Hey John, do you have the necessary information to move forward ?
Flags: needinfo?(jdai)
No longer blocks: 1146713
(In reply to Julien Wajsberg [:julienw] from comment #38)
> Hey John, do you have the necessary information to move forward ?

I already have a build pass patch on try[1]. According to bug 1213798 comment 6, I know that even try build is passed, I can't guarantee it can pass on mozilla-inbound phone build. 

[1]https://treeherder.mozilla.org/#/jobs?repo=try&revision=a865e68b5bcd
Flags: needinfo?(jdai)
Maybe land on b2g-inbound so that mozilla-inbound is still OK for gecko developers, even in case of failures ?
(In reply to Julien Wajsberg [:julienw] from comment #40)
> Maybe land on b2g-inbound so that mozilla-inbound is still OK for gecko
> developers, even in case of failures ?

B2g-inbound also has flame kitkat device image build (engineering)[1].
[1]https://treeherder.mozilla.org/#/jobs?repo=b2g-inbound&filter-searchStr=Flame-KK
I mean, if it breaks and requires a backout, it doesn't impair other gecko developers...

Of course it's better to land on an inbound that has an image build so that we see breakage before it arrives in Central ;)

Or maybe I misunderstood you ?

So what's missing to move forward here ?
(In reply to Wes Kocher (:KWierso) from comment #29)
> Backed out alongside bug 1207548 for Flame-KK build failures:
> https://hg.mozilla.org/integration/mozilla-inbound/rev/b42ed97d98cc
> 
> https://treeherder.mozilla.org/logviewer.html#?job_id=15255656&repo=mozilla-
> inbound

According to the fail log[1], I found that it is because I removed backup-flame.tar.xz in releng-flame-kk.tt file[2], and it cause builtbot flame kitkat(engineering) build failed[3]. If I add backup-flame.tar.xz back, TC will build fail[4]. Also, we already download the backup file in pre-build.sh[5], can we reuse this file in builtbot environment?


[1] http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-inbound-flame-kk_eng/1444137536/b2g_mozilla-inbound_flame-kk_eng_dep-bm91-build1-build1438.txt.gz
[2] https://hg.mozilla.org/integration/mozilla-inbound/diff/e58169d5f96a/b2g/config/flame-kk/releng-flame-kk.tt
[3] https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=e58169d5f96a&filter-searchStr=B2G%20Device%20Image 
[4] https://treeherder.allizom.org/#/jobs?repo=try&revision=e684c7512e27
[5] https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/scripts/phone-builder/pre-build.sh?from=pre-build.sh#39
Flags: needinfo?(wcosta)
Flags: needinfo?(catlee)
(In reply to John Dai[:johnz][:jdai] from comment #43)
> (In reply to Wes Kocher (:KWierso) from comment #29)
> > Backed out alongside bug 1207548 for Flame-KK build failures:
> > https://hg.mozilla.org/integration/mozilla-inbound/rev/b42ed97d98cc
> > 
> > https://treeherder.mozilla.org/logviewer.html#?job_id=15255656&repo=mozilla-
> > inbound
> 
> According to the fail log[1], I found that it is because I removed
> backup-flame.tar.xz in releng-flame-kk.tt file[2], and it cause builtbot
> flame kitkat(engineering) build failed[3]. If I add backup-flame.tar.xz
> back, TC will build fail[4]. Also, we already download the backup file in
> pre-build.sh[5], can we reuse this file in builtbot environment?
> 
> 
> [1]
> http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-inbound-
> flame-kk_eng/1444137536/b2g_mozilla-inbound_flame-kk_eng_dep-bm91-build1-
> build1438.txt.gz
> [2]
> https://hg.mozilla.org/integration/mozilla-inbound/diff/e58169d5f96a/b2g/
> config/flame-kk/releng-flame-kk.tt
> [3]
> https://treeherder.mozilla.org/#/jobs?repo=mozilla-
> inbound&revision=e58169d5f96a&filter-searchStr=B2G%20Device%20Image 
> [4] https://treeherder.allizom.org/#/jobs?repo=try&revision=e684c7512e27
> [5]
> https://dxr.mozilla.org/mozilla-central/source/testing/taskcluster/scripts/
> phone-builder/pre-build.sh?from=pre-build.sh#39

The question is why TC cannot download backup-flame.tar.xz. Aries is understandable, as it never ran on buildbot so probably there is no backup file on tooltool, but flame should work.
Flags: needinfo?(wcosta)
Yes, I think the TC build should be downloading backup-flame from tooltool as well. In particular, the tasks need the docker-worker:relengapi-proxy:tooltool.download.internal scope and the relengapiProxy feature. These are added by the decision task.
Flags: needinfo?(catlee)
In this patch, I add back backup-flame.tar.xz configuration, and provide docker-worker:relengapi-proxy:tooltool.download.internal scope for TC tasks.

Try result:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f59c289df2a0
Attachment #8670102 - Attachment is obsolete: true
Attachment #8678067 - Flags: review+
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/3cc789ec2bf0
Status: NEW → RESOLVED
Closed: 8 years ago
status-firefox44: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
I still see issues on try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f3fdc799c749
Flags: needinfo?(jdai)
(In reply to Wander Lairson Costa [:wcosta] from comment #49)
> I still see issues on try:
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=f3fdc799c749

Yes, you are right. Please help me back it out. Thanks.
Status: RESOLVED → REOPENED
Flags: needinfo?(jdai) → needinfo?(wkocher)
Resolution: FIXED → ---
Depends on: 1220093
No longer work on this.
Assignee: jdai → nobody
I have been fixing something close to this in bug 1282093, can you tell me what would be missing to mark this bug as resolved or should I dupe it against bug 1282093 ?

At least in bug 1282093 I am able to pull NDK from tooltool, but I don't make use of the install-packages.sh script. Is this mandatory ? Now that tooltool fetch works with NDK (internal) should we also switch to tooltool from S3 to host the backup file of blobs for z3c device ?
Flags: needinfo?(wcosta)
(In reply to Alexandre LISSY :gerard-majax from comment #53)
> I have been fixing something close to this in bug 1282093, can you tell me
> what would be missing to mark this bug as resolved or should I dupe it
> against bug 1282093 ?
> 

I am more inclined to dup this bug and solve it at bug 1282093.

> At least in bug 1282093 I am able to pull NDK from tooltool, but I don't
> make use of the install-packages.sh script. Is this mandatory ? Now that
> tooltool fetch works with NDK (internal) should we also switch to tooltool
> from S3 to host the backup file of blobs for z3c device ?

Moving S3 backups to tooltool would definitely be much appreciated (one less secret shipping in phone-builder).
Flags: needinfo?(wcosta)
Status: REOPENED → RESOLVED
Closed: 8 years ago → 7 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.