697112 - add more twigs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

I was talking to ehsan this morning who was asking getting nightly updates on his user repository. I told him that we could do this, but that we should probably just add more twigs instead.

Ehsan and Brian are both working on different changes to nightly updates, and Ehsan says that it would be best for them to have their own twigs, each with nightly updates enabled.

I'm going to suggest that we add at least 3 more. I'll see if I can write the patch for it.

Comment 1

[Lukas Blakk [:lsblakk] use ?needinfo]

This should help you make sure you catch everything: https://wiki.mozilla.org/ReleaseEngineering:ProjectBranchPlanning#Release_Engineering_Checklist

Sorry the graphserver branch/machine adding API/webapp is _very_ close but not ready yet so you still have to file a bug to get IT to insert the branch & machine names.

Comment 2

[[e-mail] (:bhearsum)]

Attachment: add 4 more twigs

Here's the config I think we need for setting up 4 more twigs, 3 of which will be pre-customized per the following specs:
1 for Ehsan's work (bug 307181). It requires all platform (including mobile), unittests, but no Talos. Nightly updates need to be turned on.
1 for Brian's work (bug 481815). It only requires Windows builds with unittests, but no talos. Nightly  updates need to be turned on.
1 for integration of the two. This should be configured the same as Ehsan's branch.

Ehsan, Brian, does the above sound right to you?

Lukas, the configuration was pretty straightforward, but I'm not 100% sure about lock_platforms. Do we need that?

Comment 3

[Brian R. Bondy [:bbondy]]

Sounds correct to me.

Comment 4

[[e-mail] (:bhearsum)]

Attachment: aus config updates for twigs

elm only has Windows builds, so it doesn't need to get added to Fennec.

Comment 5

[[e-mail] (:bhearsum)]

Attachment: add new twigs to TBPL

Comment 6

[[e-mail] (:bhearsum)]

Attachment: add new twigs to graph server config

Comment 7

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 569390 [details] [diff] [review]
add new twigs to graph server config

This is a valid patch for the branches portion so r+ but you also need to add in the machine names in the machines table

Comment 8

[[e-mail] (:bhearsum)]

Comment on attachment 569383 [details] [diff] [review]
aus config updates for twigs

Landed, and tag updated:
bhearsum@voot ...aus/xml/inc % cvs tag AUS2_PRODUCTION config-dist.php
W config-dist.php : AUS2_PRODUCTION already exists on version 1.165 : NOT MOVING tag to version 1.166
bhearsum@voot ...aus/xml/inc % cvs tag -F AUS2_PRODUCTION config-dist.php
T config-dist.php

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 569380 [details] [diff] [review]
add 4 more twigs

Landed on default.

Comment 10

[[e-mail] (:bhearsum)]

Attachment: updated graphs patch

Sorry Lukas, I missed your comments about the graphs patch. I believe this fixes it up. Also, I moved the branches to the bottom of the list because I wasn't sure if things would break if branch IDs changed in the future....

Comment 11

[[e-mail] (:bhearsum)]

Attachment: set priorities for new twigs

Sorry Lukas, I forgot to add these to the priorities.

Comment 12

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 569435 [details] [diff] [review]
set priorities for new twigs

good catch!

Comment 13

[[e-mail] (:bhearsum)]

Attachment: fix up aus

Nick pointed out that I missed a trailing comma in the Fennec arrays. While I was fixing that I noticed that I forgot to update the nightlyChannels variable. This should fix both up.

Comment 14

[[e-mail] (:bhearsum)]

Comment on attachment 569491 [details] [diff] [review]
fix up aus

bhearsum@voot ...aus/xml/inc % cvs commit -m "bug 697112: add more twigs - add alm, esh, and oak to AUS config (fixups). r=nthomas" config-dist.php 
cvs tag Checking in config-dist.php;
/cvsroot/mozilla/webtools/aus/xml/inc/config-dist.php,v  <--  config-dist.php
new revision: 1.167; previous revision: 1.166
done
bhearsum@voot ...aus/xml/inc % cvs tag AUS2_PRODUCTION config-dist.php 
W config-dist.php : AUS2_PRODUCTION already exists on version 1.166 : NOT MOVING tag to version 1.167
bhearsum@voot ...aus/xml/inc % cvs tag -F AUS2_PRODUCTION config-dist.php
T config-dist.php

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

While we have bug 694809, we need to do this sort of thing for new branches:

for b in ash elm oak pine; do
  for p in linux linux64; do
    mkdir $b-$p 
    touch $b-$p/codesize-auto.log
  done
  for p in linux linux64 macosx64 macosx win32; do
    mkdir $b-$p-debug
    touch $b-$p-debug/{malloc.log,sdleak.tree}
  done
done

On ash we were getting multiple builds for linux & linux64 opt, and debug on all platforms. They got far enough to compile, upload and sendchange, then would fail to pull a previous log and end up retried. The backlog due to bug 697374 is bigger as a result.

Comment 16

[(no longer active)]

Here are a few problems that I noticed with ash yesterday:

1. There are tons of test runs triggered on OS X and Linux.
2. There are no build/test runs triggered on Windows, Android, and Maemo.
3. I triggered a nightly yesterday on 257400b4fddb, but it seems like it didn't work.  I don't see the N builds in TBPL, and <http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-ash/> doesn't exist yet.

Comment 17

[[e-mail] (:bhearsum)]

(In reply to Ehsan Akhgari [:ehsan] from comment #16)
> Here are a few problems that I noticed with ash yesterday:
> 
> 1. There are tons of test runs triggered on OS X and Linux.

This should be fixed by Nick's actions in comment #15.

> 2. There are no build/test runs triggered on Windows, Android, and Maemo.
> 3. I triggered a nightly yesterday on 257400b4fddb, but it seems like it
> didn't work.  I don't see the N builds in TBPL, and
> <http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-ash/> doesn't
> exist yet.

I'll look into these today.

Comment 18

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #17)
> (In reply to Ehsan Akhgari [:ehsan] from comment #16)
> > 2. There are no build/test runs triggered on Windows, Android, and Maemo.

This seems to have fixed itself.

Comment 19

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #17)
> (In reply to Ehsan Akhgari [:ehsan] from comment #16)
> > 3. I triggered a nightly yesterday on 257400b4fddb, but it seems like it
> > didn't work.  I don't see the N builds in TBPL, and
> > <http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-ash/> doesn't
> > exist yet.

Catlee just theorized that self serve doesn't work correctly for triggering nightlies when there are no previous nightly builds. If that's true, self serve _should_ work again after the round of nightlies that I triggered directly through Buildbot have finished.

Comment 20

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #19)
> (In reply to Ben Hearsum [:bhearsum] from comment #17)
> > (In reply to Ehsan Akhgari [:ehsan] from comment #16)
> > > 3. I triggered a nightly yesterday on 257400b4fddb, but it seems like it
> > > didn't work.  I don't see the N builds in TBPL, and
> > > <http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-ash/> doesn't
> > > exist yet.
> 
> Catlee just theorized that self serve doesn't work correctly for triggering
> nightlies when there are no previous nightly builds. If that's true, self
> serve _should_ work again after the round of nightlies that I triggered
> directly through Buildbot have finished.

I just tried triggering nightlies through self serve, and they worked! Interestingly, I also got a second Windows nightly pending...so maybe only one of the builders needs to finish for all of them to work?