542910 - Create a project branch for Add-ons Manager Rewrite

Status: VERIFIED FIXED

(Release Engineering :: General, defect)

Description

[Dave Townsend [:mossop]]

*do you want builds?

Yes, all OS supported by trunk

** incremental-build-on-checkin?

Yes

** nightlies?

No (assuming we still get to download regular builds)

** Are en-US builds enough, with no l10n? y/n

Just en-US is fine

* do you want unittests?

All unittests run on mozilla-central

* do you want talos?

Talos on all OS like mozilla-central but we probably don't need sunspider, dhtml, svg or gfx testing. Startup, shutdown and page load are the main things we are interested in monitoring.

*name of branch owner

Mossop

*timeline:
** when can we start project branch

As soon as practical

** approx expected life span of project branch - if known?

The main project should be complete and merged to mozilla-central by April however we may decide to keep the branch open at that point to work on other large scale changes.

*misc:
** need any changes to toolchain used in m-c?

Nope

** need any changes to the compile/link/repack steps used in m-c?

Nope

** preference on tinderbox waterfall name?

ExtensibleMonkey?

** preference on where to put builds on ftp.m.o?

extensible-monkey?

** preference on name of project branch in hg?

projects/extensible-monkey

** what unofficial projectname do you want on this project branch?

Just keeping Minefield should be fine I think

Comment 1

[Dave Townsend [:mossop]]

From talking with joduinn, just use Addons as the name for this branch.

Comment 2

[Blair McBride [:Unfocused] (UNAVAILABLE)]

Any ETA on this?

Comment 3

[Lukas Blakk [:lsblakk] use ?needinfo]

What repo will this branch be using? If it doesn't already exist, we will need to file a bug asking for the repo to be created.  Let's use projects/addons if there's not already one set up.

Comment 4

[Lukas Blakk [:lsblakk] use ?needinfo]

(In reply to comment #2)
> Any ETA on this?

I'll get patches together to test this in staging, only hold up here is a repo.

Comment 5

[Lukas Blakk [:lsblakk] use ?needinfo]

(In reply to comment #2)
> Any ETA on this?

So more specifically - it generally takes about 2 weeks end-to-end with a branch that has no special needs (such as this one).  I'll do my best to improve on that but some of it is just procedural, testing and whatnot, before going live in production.

Comment 6

[Lukas Blakk [:lsblakk] use ?needinfo]

Tinderbox tree "Addons" has been created for this branch.

Comment 7

[Reed Loden [:reed]]

(In reply to comment #1)
> From talking with joduinn, just use Addons as the name for this branch.

Addons seems way too vague for something dealing with the Add-ons Manager... How about AddonsMgr or something?

Comment 8

[Lukas Blakk [:lsblakk] use ?needinfo]

It's not vague compared to other branch names in our configs - Addons is short and succinct and works just fine in a list with "Places", "Electrolysis" and "Tracemonkey" so let's not bikeshed on the project name.

Comment 9

[Reed Loden [:reed]]

(In reply to comment #8)
> It's not vague compared to other branch names in our configs - Addons is short
> and succinct and works just fine in a list with "Places", "Electrolysis" and
> "Tracemonkey" so let's not bikeshed on the project name.

Places, Electrolysis, and Tracemonkey all refer to specific things... Addons, however, refers to a ton of different things. It's just going to be confusing.

Comment 10

[Lukas Blakk [:lsblakk] use ?needinfo]

This is an internal-facing config naming that only matters for the repo, the tinderbox tree, and the build/unittest waterfalls.  There's nothing to be "confused" about here since there is no other addon related work in those places. 
As we learned with the recent Firefox-Lorentz/Lorentz branch, we need consistent naming across the board and "addons" is short and clear. 

No other build or performance testing is related to addons and if at some point another project branch is created it can use addons-{new branch's purpose}.

Comment 11

[Lukas Blakk [:lsblakk] use ?needinfo]

Okay, we'll go with "addonsmgr" across the board.  /bikeshed

Comment 12

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Add addonsmgr to graphserver

Comment 13

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [untested] staging configs for addonsmgr

will test this once repo exists.

Comment 14

[Dave Townsend [:mossop]]

(In reply to comment #13)
> Created an attachment (id=429609) [details]
> [untested] staging configs for addonsmgr
> 
> will test this once repo exists.

The repo exists, shall I push a copy of mozilla-central into it?

Comment 15

[Lukas Blakk [:lsblakk] use ?needinfo]

Yes, please do that so there's something to check out for staging tests.

Comment 16

[Dave Townsend [:mossop]]

I've pushed a copy of mozilla-central to the repository

Comment 17

[Lukas Blakk [:lsblakk] use ?needinfo]

Requesting the following be added to staging-graphserver for testing:

insert into branches values (NULL,"Addonsmgr");
insert into machines values (NULL,6,0,NULL,"Linux_addonsmgr",1,unix_timestamp());
insert into machines values (NULL,6,0,NULL,"Linux_addonsmgr_leak_test",1,unix_timestamp());
insert into machines values (NULL,7,0,NULL,"OS_X_10.5.2_addonsmgr",1,unix_timestamp());
insert into machines values (NULL,7,0,NULL,"OS_X_10.5.2_addonsmgr_leak_test",1,unix_timestamp());
insert into machines values (NULL,8,0,NULL,"WINNT_5.2_addonsmgr",1,unix_timestamp());
insert into machines values (NULL,8,0,NULL,"WINNT_5.2_addonsmgr_leak_test",1,unix_timestamp());
insert into machines values (NULL,18,0,NULL,"Linux_x86-64_addonsmgr",1,unix_timestamp());
insert into machines values (NULL,18,0,NULL,"Linux_x86-64_addonsmgr_leak_test",1,unix_timestamp());

Comment 18

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: insert statements for graphServer db

Comment 19

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: insert sql for addonsmgr to staging-graphserver

Comment 20

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 430468 [details]
insert sql for addonsmgr to staging-graphserver

oops - dupe.

Comment 21

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [Tested] staging configs for addonsmgr

Comment 22

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Production configs for addonsmgr

Comment 23

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 431226 [details] [diff] [review]
[Tested] staging configs for addonsmgr

>diff --git a/mozilla2-staging/config.py b/mozilla2-staging/config.py
>+BRANCHES['addonsmgr']['platforms']['win32']['profiled_build'] = False

I think we should leave this on so that win32 talos results are comparable to m-c, so just remove this line.

>+# Enable unit tests
>+BRANCHES['addonsmgr']['platforms']['linux']['enable_unittests'] = True
>+BRANCHES['addonsmgr']['platforms']['linux64']['enable_unittests'] = True
>+BRANCHES['addonsmgr']['platforms']['macosx']['enable_unittests'] = True
>+BRANCHES['addonsmgr']['platforms']['win32']['enable_unittests'] = True

We should just use the opt and debug packaged tests if we want to match m-c, so False for these. And add the 4 lines like
 BRANCHES['addonsmgr']['platforms']['linux-debug']['enable_unittests'] = True

>+BRANCHES['addonsmgr']['platforms']['linux']['update_platform'] = 'fake'
>+BRANCHES['addonsmgr']['platforms']['linux64']['update_platform'] = 'fake'
>+BRANCHES['addonsmgr']['platforms']['win32']['update_platform'] = 'fake'
>+BRANCHES['addonsmgr']['platforms']['macosx']['update_platform'] = 'fake'

These can be left out, the values set in PLATFORM_VARS are fine and won't be used anyway.

>diff --git a/mozilla2-staging/win32/addonsmgr/unittest/mozconfig b/mozilla2-staging/win32/addonsmgr/unittest/mozconfig

Don't need this one given the above.

>diff --git a/talos-staging-pool/config.py b/talos-staging-pool/config.py

Not really my area of knowledge, seek review from alice or catlee.

Comment 24

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 431227 [details] [diff] [review]
Production configs for addonsmgr

The comments for attachment 431226 [details] [diff] [review] apply here too.

>diff --git a/mozilla2/config.py b/mozilla2/config.py
>diff --git a/mozilla2/linux/addonsmgr b/mozilla2/linux/addonsmgr
>diff --git a/mozilla2/linux64/addonsmgr b/mozilla2/linux64/addonsmgr
>diff --git a/mozilla2/macosx/addonsmgr b/mozilla2/macosx/addonsmgr
>+mozilla-central/

Remove the trailing /. 

Mossop, does 'Yes, all OS supported by trunk' (in comment #0) include Maemo/WinMo/kitchen sink ?

Comment 25

[Dave Townsend [:mossop]]

(In reply to comment #24)
> (From update of attachment 431227 [details] [diff] [review])
> The comments for attachment 431226 [details] [diff] [review] apply here too.
> 
> >diff --git a/mozilla2/config.py b/mozilla2/config.py
> >diff --git a/mozilla2/linux/addonsmgr b/mozilla2/linux/addonsmgr
> >diff --git a/mozilla2/linux64/addonsmgr b/mozilla2/linux64/addonsmgr
> >diff --git a/mozilla2/macosx/addonsmgr b/mozilla2/macosx/addonsmgr
> >+mozilla-central/
> 
> Remove the trailing /. 
> 
> Mossop, does 'Yes, all OS supported by trunk' (in comment #0) include
> Maemo/WinMo/kitchen sink ?

I'm expected performance impact so having data from Maemo and WinMo would be extremely useful yes.

Comment 26

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Production configs for addonsmgr

Adding alice to review this for talos

Comment 27

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [Tested] staging configs for addonsmgr

Made the same changes as in production configs, adding alice for talos review.

Comment 28

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 431260 [details] [diff] [review]
[Tested] staging configs for addonsmgr

This is the same as attachment 431226 [details] [diff] [review].

Comment 29

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [Tested] staging configs for addonsmgr

Sorry about that.  Here's the updated patch.

Comment 30

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [tested] staging mobile configs for addonsmgr

Comment 31

[Aki Sasaki (not active)]

Comment on attachment 431283 [details] [diff] [review]
[tested] staging mobile configs for addonsmgr

Looks like you're missing the enable_l10n_onchange option (which should be False as well).

r=me with that added.

Comment 32

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: [tested] staging mobile configs for addonsmgr

setting create_snippet to False

Comment 33

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Production configs for mobile-addonsmgr

Comment 34

[Aki Sasaki (not active)]

Comment on attachment 431284 [details] [diff] [review]
[tested] staging mobile configs for addonsmgr

Same as above, looks like you're missing the enable_l10n_onchange option (which should be
False as well).

(You'll need to update your buildbot-configs and buildbotcustom to see the appropriate changes changes from today)

Comment 35

[Aki Sasaki (not active)]

Comment on attachment 431285 [details] [diff] [review]
Production configs for mobile-addonsmgr

>diff --git a/mozilla2-staging/mobile_config.py b/mozilla2-staging/mobile_config.py
>--- a/mozilla2-staging/mobile_config.py
>+++ b/mozilla2-staging/mobile_config.py
>@@ -415,37 +415,37 @@ 
<snip>
>-MOBILE_BRANCHES['mobile-addonsmgr']['tinderbox_tree'] = 'MozillaTest'
>-MOBILE_BRANCHES['mobile-addonsmgr']['l10n_tinderbox_tree'] = 'MozillaStaging'
>+MOBILE_BRANCHES['mobile-addonsmgr']['tinderbox_tree'] = 'AddonsMgr'
>+MOBILE_BRANCHES['mobile-addonsmgr']['l10n_tinderbox_tree'] = 'Mozilla-l10n'

Looks like you're changing the staging configs to send to the production tinderbox trees?

>diff --git a/mozilla2/mobile_config.py b/mozilla2/mobile_config.py
>--- a/mozilla2/mobile_config.py
>+++ b/mozilla2/mobile_config.py
>@@ -20,16 +20,17 @@ mobile_slaves = {

>+
>+### mozilla-central

Comment nit!

>+MOBILE_BRANCHES['mobile-addonsmgr']['enable_l10n'] = False
>+MOBILE_BRANCHES['mobile-addonsmgr']['enable_multi_locale'] = False

Ok.
Looks like you're enabling l10n on staging, but not production? (checking to see if that's what you wanted.)

If you meant to enable, please set these to True, but set enable_l10n_onchange to False until we track down the weirdness related to bug 550940 and bug 548178.

>+MOBILE_BRANCHES['mobile-addonsmgr']['tinderbox_tree'] = 'MozillaTest'
>+MOBILE_BRANCHES['mobile-addonsmgr']['l10n_tinderbox_tree'] = 'MozillaStaging'

Aha, I think you edited the wrong file =)

I'm going to chalk this one up to the late hour.

Comment 36

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 431274 [details] [diff] [review]
[Tested] staging configs for addonsmgr

>diff --git a/mozilla2-staging/config.py b/mozilla2-staging/config.py
>+BRANCHES['addonsmgr']['unittest_build_space'] = 6

Nit: No need to set this for builds we're not doing.

Comment 37

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 431258 [details] [diff] [review]
Production configs for addonsmgr

>diff --git a/mozilla2/config.py b/mozilla2/config.py
>+BRANCHES['addonsmgr']['platforms']['win32']['profiled_build'] = False

This should be removed to match staging.

>+BRANCHES['addonsmgr']['unittest_build_space'] = 6

Nit: No need to set this for builds we're not doing.

There's no change to mozilla2/master2.cfg in this patch (adding addonsmgr to the ACTIVE_BRANCHES list).

>diff --git a/talos-pool/config.py b/talos-pool/config.py

This should be against talos-r3. These days talos-pool is only doing Tiger jobs and so not useful for mozilla-central based project branch.

Comment 38

[Armen [:armenzg]]

BTW I am assuming that we are only enabling l10n for release branches (mc, m191 and m192) and not for project branches (to the exception of lorentz).
Has this change?

Comment 39

[tymerkaev]

(In reply to comment #38)
> BTW I am assuming that we are only enabling l10n for release branches (mc, m191
> and m192) and not for project branches (to the exception of lorentz).
> Has this change?

What about create an additional branch in l10n-central repo for addonsmgr?

Comment 40

[Armen [:armenzg]]

(In reply to comment #39)
> (In reply to comment #38)
> > BTW I am assuming that we are only enabling l10n for release branches (mc, m191
> > and m192) and not for project branches (to the exception of lorentz).
> > Has this change?
> 
> What about create an additional branch in l10n-central repo for addonsmgr?
I am sorry Tymerkaev but we are only putting machine time into release branches and lorentz for now. This addons project branch is mainly for developers to add features in their own branch and at certain points to port their features to trunk and eventually to release branches. Then and only then, if there are strings changes, it makes sense for localizers to localize and expect results from our build machines.

I hope this makes sense.

Comment 41

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Production configs for addonsmgr

Cleaned up production, and updated talos-r3

Comment 42

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Production configs for addonsmgr

Sorry for the bug spam, did refresh my queue and was missing the master2.cfg change.

Comment 43

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 431274 [details] [diff] [review]
[Tested] staging configs for addonsmgr

http://hg.mozilla.org/build/buildbot-configs/rev/680b69e5e4e2

Comment 44

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 431284 [details] [diff] [review]
[tested] staging mobile configs for addonsmgr

checked in with enable_l10n_onchange set to False

http://hg.mozilla.org/build/buildbot-configs/rev/27156a96f2b7

Comment 45

[Lukas Blakk [:lsblakk] use ?needinfo]

(In reply to comment #38)
> BTW I am assuming that we are only enabling l10n for release branches (mc, m191
> and m192) and not for project branches (to the exception of lorentz).
> Has this change?

Armen, good catch - there will not be 110n enabled for this.

Comment 46

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Production configs for mobile-addonsmgr

Fixed it, on production no less, according to Aki's comments.  Turned off l10n as well since we don't need it on a project (non-release) branch.

Comment 47

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Mobile-staging clean-up - removing mobile l10n in config

Comment 48

[Aki Sasaki (not active)]

Comment on attachment 431422 [details] [diff] [review]
Production configs for mobile-addonsmgr

This doesn't checkconfig out of the box, but that's only because it's dependent on config.BRANCHES['addonsmgr'] existing.

so r=me but it's dependent on the other production patch. (you probably know this already)

Comment 49

[Lukas Blakk [:lsblakk] use ?needinfo]

> so r=me but it's dependent on the other production patch. (you probably know
> this already)


Yes, both production patches will be checked in during the downtime on Thursday, March 11th.

Comment 50

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 431395 [details] [diff] [review]
Production configs for addonsmgr

r+ for the non-talos parts.

>diff --git a/mozilla2/config.py b/mozilla2/config.py
>-
>+ 

Remove the space added here on checkin.

Comment 51

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 431395 [details] [diff] [review]
Production configs for addonsmgr

http://hg.mozilla.org/build/buildbot-configs/rev/d1048144f9ac

Comment 52

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 431422 [details] [diff] [review]
Production configs for mobile-addonsmgr

http://hg.mozilla.org/build/buildbot-configs/rev/c6646140a6cb

Comment 53

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 429863 [details]
insert statements for graphServer db

mysql> insert into branches values (NULL,"Addonsmgr");
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,6,0,NULL,"Linux_addonsmgr",1,unix_timestamp());
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,6,0,NULL,"Linux_addonsmgr_leak_test",1,unix_timestamp());
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,7,0,NULL,"OS_X_10.5.2_addonsmgr",1,unix_timestamp());
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,7,0,NULL,"OS_X_10.5.2_addonsmgr_leak_test",1,unix_timestamp());
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,8,0,NULL,"WINNT_5.2_addonsmgr",1,unix_timestamp());
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,8,0,NULL,"WINNT_5.2_addonsmgr_leak_test",1,unix_timestamp());
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,18,0,NULL,"Linux_x86-64_addonsmgr",1,unix_timestamp());
Query OK, 1 row affected (0.00 sec)

mysql> insert into machines values (NULL,18,0,NULL,"Linux_x86-64_addonsmgr_leak_test",1,unix_timestamp());
Query OK, 1 row affected (0.01 sec)

Comment 54

[Lukas Blakk [:lsblakk] use ?needinfo]

Turned on this branch for production in this morning's downtime.  Leaving the bug open as we sort out nagios monitoring and just in case any issues arise in the first few days of operating.

Comment 55

[Dave Townsend [:mossop]]

Both the linux x86-64 and linux leak test builds seem to be failing on something relating to the setup. See this from the leak test:

======== BuildStep started ========
'wget -O ...' failed
=== Output ===
wget -O bloat.log.old http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/addonsmgr-linux//bloat.log
 in dir /builds/slave/addonsmgr-linux-debug/. (timeout 1200 secs)
 watching logfiles {}
 argv: ['wget', '-O', 'bloat.log.old', 'http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/addonsmgr-linux//bloat.log']
 environment:
  CC=/tools/gcc/bin/gcc
  CCACHE_DIR=/builds/ccache
  CCACHE_UMASK=002
  CVS_RSH=ssh
  CXX=/tools/gcc/bin/g++
  DISPLAY=:2
  G_BROKEN_FILENAMES=1
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=moz2-linux-slave43.build.mozilla.org
  INPUTRC=/etc/inputrc
  JAVA_HOME=/builds/jdk
  LANG=en_US.UTF-8
  LD_LIBRARY_PATH=obj-firefox/dist/bin
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
  MAIL=/var/spool/mail/cltbld
  MOZ_CRASHREPORTER_NO_REPORT=1
  MOZ_OBJDIR=obj-firefox
  PATH=/opt/local/bin:/tools/buildbot/bin:/tools/twisted/bin:/tools/twisted-core/bin:/tools/python/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/cltbld/bin
  PWD=/builds/slave/addonsmgr-linux-debug
  PYTHONHOME=/tools/python
  PYTHONPATH=/tools/buildbotcustom:/tools/buildbot/lib/python2.5/site-packages:/tools/twisted/lib/python2.5/site-packages:/tools/twisted-core/lib/python2.5/site-packages:/tools/zope-interface/lib/python2.5/site-packages/
  SHELL=/bin/bash
  SHLVL=1
  SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
  TBOX_CLIENT_CVS_DIR=/builds/tinderbox/mozilla/tools
  TERM=linux
  USER=cltbld
  XPCOM_DEBUG_BREAK=stack-and-abort
  _=/tools/buildbot/bin/buildbot
 closing stdin
 using PTY: True
--20:05:45--  http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/addonsmgr-linux//bloat.log
Resolving stage.mozilla.org... 10.2.74.116
Connecting to stage.mozilla.org|10.2.74.116|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
20:05:45 ERROR 404: Not Found.

program finished with exit code 1
elapsedTime=0.114581
=== Output ended ===

And this on the x86-64:

======== BuildStep ended ========
======== BuildStep started ========
'wget -O ...' failed
=== Output ===
wget -O codesize-auto-old.log http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/addonsmgr-linux64//codesize-auto.log
 in dir /builds/slave/addonsmgr-linux64/. (timeout 1200 secs)
 watching logfiles {}
 argv: ['wget', '-O', 'codesize-auto-old.log', 'http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/addonsmgr-linux64//codesize-auto.log']
 environment:
  CC=/tools/gcc/bin/gcc
  CCACHE_DIR=/builds/ccache
  CCACHE_UMASK=002
  CVS_RSH=ssh
  CXX=/tools/gcc/bin/g++
  G_BROKEN_FILENAMES=1
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=moz2-linux64-slave11.build.mozilla.org
  INPUTRC=/etc/inputrc
  JAVA_HOME=/builds/jdk
  LANG=en_US.UTF-8
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
  MAIL=/var/spool/mail/cltbld
  MOZ_CRASHREPORTER_NO_REPORT=1
  MOZ_OBJDIR=obj-firefox
  MOZ_SYMBOLS_EXTRA_BUILDID=linux64-addonsmgr
  PATH=/opt/local/bin:/tools/buildbot/bin:/tools/twisted/bin:/tools/twisted-core/bin:/tools/python/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/cltbld/bin
  PWD=/builds/slave/addonsmgr-linux64
  PYTHONHOME=/tools/python
  PYTHONPATH=/tools/buildbotcustom:/tools/buildbot/lib/python2.5/site-packages:/tools/twisted/lib/python2.5/site-packages:/tools/twisted-core/lib/python2.5/site-packages:/tools/zope-interface/lib/python2.5/site-packages/
  SHELL=/bin/bash
  SHLVL=1
  SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
  SYMBOL_SERVER_HOST=dm-symbolpush01.mozilla.org
  SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_ffx/
  SYMBOL_SERVER_SSH_KEY=/home/cltbld/.ssh/ffxbld_dsa
  SYMBOL_SERVER_USER=ffxbld
  TBOX_CLIENT_CVS_DIR=/builds/tinderbox/mozilla/tools
  TERM=linux
  TINDERBOX_OUTPUT=1
  USER=cltbld
  _=/tools/buildbot/bin/buildbot
 closing stdin
 using PTY: True
--19:39:56--  http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/addonsmgr-linux64//codesize-auto.log
Resolving stage.mozilla.org... 10.2.74.116
Connecting to stage.mozilla.org|10.2.74.116|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
19:39:56 ERROR 404: Not Found.

program finished with exit code 1
elapsedTime=0.189046
=== Output ended ===
======== BuildStep ended ========

Comment 56

[Chris AtLee [:catlee]]

Just enabled scraping on a bajillion columns on the addons tree.

Comment 57

[Chris AtLee [:catlee]]

Dave, I've kicked off another build to see if it was just the lack of previous logs to compare against that is the issue here.

Comment 58

[Dave Townsend [:mossop]]

(In reply to comment #57)
> Dave, I've kicked off another build to see if it was just the lack of previous
> logs to compare against that is the issue here.

That seems to have solved it. A couple of other issues appear to be showing:

There are tests running for linux 64bit, apparently this isn't normal and they are always failing (for expected reasons).
WinMo Addonsmgr build is consistently failing with errors about missing pthread.h and freetype2.lib2, perhaps this is because I have a slightly outdated mozilla-central in the repository right now?
WinMo Addonsmgr nightly is consistently timing out cloning the repository, not sure why it is there though I didn't ask for nightlies.

Also one oddity, the nightly directory is getting things added to it, not sure if that is just an expected side effect.

Comment 59

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #58)
> There are tests running for linux 64bit, apparently this isn't normal and they
> are always failing (for expected reasons).

gah review fail. Lukas, we should remove these:
BRANCHES['addonsmgr']['platforms']['linux64']['enable_opt_unittests'] = True
BRANCHES['addonsmgr']['platforms']['linux64-debug']['enable_unittests'] = True

> WinMo Addonsmgr nightly is consistently timing out cloning the repository, not
> sure why it is there though I didn't ask for nightlies.

Mobile config probably doesn't know about the enable_nightly parameter.

> Also one oddity, the nightly directory is getting things added to it, not sure
> if that is just an expected side effect.

That's the shark build, which isn't controlled by the enable_nightly parameter either. We can turn that off if you like.

Comment 60

[Dave Townsend [:mossop]]

(In reply to comment #59)
> > Also one oddity, the nightly directory is getting things added to it, not sure
> > if that is just an expected side effect.
> 
> That's the shark build, which isn't controlled by the enable_nightly parameter
> either. We can turn that off if you like.

Not really bothered, just thought I'd point it out as an oddity.

Comment 61

[Lukas Blakk [:lsblakk] use ?needinfo]

Attachment: Turn off linux64 opt_unittests/debug_unittests for addonsmgr

Comment 62

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 432596 [details] [diff] [review]
Turn off linux64 opt_unittests/debug_unittests for addonsmgr

>diff --git a/mozilla2-staging/config.py b/mozilla2-staging/config.py
>-BRANCHES['tryserver']['platforms']['linux64']['enable_opt_unittests'] = True
>+BRANCHES['tryserver']['platforms']['linux64']['enable_opt_unittests'] = False

We seem to have this turned on elsewhere in staging, so you could leave this out.

>diff --git a/mozilla2/config.py b/mozilla2/config.py
>+BRANCHES['enable_shark'] = False

We should make this change to the staging config to keep it in sync as much as possible.

r+ to fix on landing.

Comment 63

[Lukas Blakk [:lsblakk] use ?needinfo]

Comment on attachment 432596 [details] [diff] [review]
Turn off linux64 opt_unittests/debug_unittests for addonsmgr

http://hg.mozilla.org/build/buildbot-configs/rev/f28ed9851441

matched on staging/production to turn off shark, and opt/debug unittest for linux64

Comment 64

[Lukas Blakk [:lsblakk] use ?needinfo]

The turning off of opt/debug tests for linux 64 went into a reconfig of pm02 today.

Comment 65

[Lukas Blakk [:lsblakk] use ?needinfo]

CLosing this bug since the branch is now live - please file new bugs if issues arise.

Comment 66

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thanks Lukas! Looks good so far. QA can already use the builds for testing. Marking as verified fixed.

Comment 67

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Shall we file a new bug to get the project branch deleted? We don't have to use it anymore.