1151964 - No nightly larch builds since March 21

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[:fabrice] Fabrice Desré]

See http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-larch/

The last graphene builds are from 21-Mar-2015 12:35

Comment 1

[Justin Wood (:Callek)]

This was purposeful, c.f. Bug 1139943

We can re-enable any as needed (or delete from ftp's latest dir) at your discretion.

Comment 2

[Justin Wood (:Callek)]

err misread.... /me looks again...

Comment 3

[Justin Wood (:Callek)]

from todays nightly:

========= Started '/tools/buildbot/bin/python /tools/checkouts/mozharness/scripts/fx_desktop_build.py ...' warnings (results: 1, elapsed: 0 secs) (at 2015-04-06 04:42:38.943429) =========
/tools/buildbot/bin/python /tools/checkouts/mozharness/scripts/fx_desktop_build.py --config builds/releng_base_linux_64_builds.py --custom-build-variant-cfg graphene--config balrog/production.py --branch larch --build-pool production --enable-nightly
 in dir /builds/slave/l-l64_graphene-ntly-0000000000/. (timeout 10800 secs) (maxTime 19800 secs)
 watching logfiles {}
 argv: ['/tools/buildbot/bin/python', '/tools/checkouts/mozharness/scripts/fx_desktop_build.py', '--config', 'builds/releng_base_linux_64_builds.py', '--custom-build-variant-cfg', 'graphene--config', 'balrog/production.py', '--branch', 'larch', '--build-pool', 'production', '--enable-nightly']
 environment:
  CCACHE_HASHDIR=
  CVS_RSH=ssh
...
  USER=cltbld
  _=/tools/buildbot/bin/python
 using PTY: False
Whoops!
'--custom-build-variant' was passed but an appropriate config file could not be determined. Tried using: 'graphene--config' but it was either not:
	-- a valid shortname: ['mulet', 'debug', 'stat-and-debug', 'b2g-debug', 'asan', 'graphene', 'code-coverage', 'asan-and-debug'] 
	-- a valid path in ['.', '/tools/checkouts/mozharness/scripts/../configs', '/tools/checkouts/mozharness/scripts/../../configs'] 
	-- a valid variant for the given platform and bits.
program finished with exit code 1
elapsedTime=0.169767


Jordan can you identify the path forward here?

Comment 4

[Justin Wood (:Callek)]

Attachment: [configs] fix typo

So the issue here is that python was implicitly reading an escaped newline (since we were inside of an unterminated statement) and then concating two strings (like python allows) `" foo" "bar "` === `" foobar "`.

This was all due to a missing comma....

I'm not certain why this didn't break non-nightly builds, but I don't feel looking into that why is a worthwhile time spend atm.

Comment 5

[Jordan Lund (:jlund)]

Comment on attachment 8589248 [details] [diff] [review]
[configs] fix typo

Review of attachment 8589248 [details] [diff] [review]:
-----------------------------------------------------------------

it broke non-nightlies too: https://treeherder.mozilla.org/logviewer.html#?job_id=3896&repo=larch

happened after the 18th of March whenever there was a reconfig: http://hg.mozilla.org/build/buildbot-configs/rev/1d12367da5cb

Comment 6

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/f414f97c22bb

Comment 7

[Paul Rouget [:paul]]

Is this fixed?
I don't see any new build in http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-larch/

Comment 8

[Justin Wood (:Callek)]

Last set of builds was from before I fixed this:

https://treeherder.mozilla.org/#/jobs?repo=larch

Might need to re-build or re-push to get a new run.

Comment 9

[[:fabrice] Fabrice Desré]

I pushed again yesterday evening, and we still don't have build triggered on latest-larch.

Comment 10

[Justin Wood (:Callek)]

There are builds now, and not with (afaict) a releng error:

https://treeherder.mozilla.org/#/jobs?repo=larch&filter-searchStr=nightly

```gmake[1]: *** No rule to make target `sdk'. Stop.```

Prior issue was:

'--custom-build-variant' was passed but an appropriate config file could not be determined. Tried using: 'graphene--config' but it was either not:
	-- a valid shortname: ['mulet', 'debug', 'stat-and-debug', 'b2g-debug', 'asan', 'graphene', 'code-coverage', 'asan-and-debug'] 
	-- a valid path in ['.', '/tools/checkouts/mozharness/scripts/../configs', '/tools/checkouts/mozharness/scripts/../../configs'] 
	-- a valid variant for the given platform and bits.

Comment 11

[Justin Wood (:Callek)]

please file the issue in c#10 as its own bug in the proper component

Comment 12

[Paul Rouget [:paul]]

Follow-up: bug 1157632