If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

[B2G] Support Emulator-L on buildbot

RESOLVED FIXED in Firefox OS v2.2

Status

Firefox OS
Emulator
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: hsinyi, Assigned: tzimmermann)

Tracking

(Depends on: 1 bug, Blocks: 1 bug)

unspecified
2.2 S8 (20mar)
All
Gonk (Firefox OS)
Dependency tree / graph

Firefox Tracking Flags

(b2g-v2.2 fixed, b2g-master fixed)

Details

Attachments

(7 attachments, 6 obsolete attachments)

2.19 KB, patch
catlee
: review+
Details | Diff | Splinter Review
978 bytes, patch
catlee
: review+
Details | Diff | Splinter Review
4.49 KB, patch
philor
: review+
Details | Diff | Splinter Review
54 bytes, text/x-github-pull-request
emorley
: review+
Details | Review | Splinter Review
49 bytes, text/x-github-pull-request
emorley
: review+
Details | Review | Splinter Review
13.12 KB, patch
massimo
: review+
Details | Diff | Splinter Review
6.73 KB, patch
massimo
: review+
Details | Diff | Splinter Review
(Reporter)

Description

3 years ago
Bug 1116074 is to support emulator-lollipop local build. Eventually, we will want to have automatic tests for emulator-lollipop on treeherder. And this is the bug to track tasks to make emulator-L run on TPBL.
Can we bump the priority on this please? At the moment, Nexus 5-L builds are the only Lollipop-based builds we run in automation. However, since B2G device image builds can't be run on Try, that leaves developers with no way to test their patches for possible L-only bustage short of pushing to inbound and crossing their fingers. This isn't a hypothetical concern either, such a situation happened today.

Note that getting tests running on the builds is orthogonal to this discussion. I just think that we need to give developers an ability to verify their changes on Try before pushing, and an emulator build would go a long way to satisfying that.
(Reporter)

Comment 2

3 years ago
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #1)
> Can we bump the priority on this please? At the moment, Nexus 5-L builds are
> the only Lollipop-based builds we run in automation. However, since B2G
> device image builds can't be run on Try, that leaves developers with no way
> to test their patches for possible L-only bustage short of pushing to
> inbound and crossing their fingers. This isn't a hypothetical concern
> either, such a situation happened today.
> 
> Note that getting tests running on the builds is orthogonal to this
> discussion. I just think that we need to give developers an ability to
> verify their changes on Try before pushing, and an emulator build would go a
> long way to satisfying that.

Hi Ryan,
Currently Seinlin and Edgar are working on bug 1116074. Once that gets fixed, we will try to make a L build on try as soon as possible; and as you said, that doesn't need to happen after tests on L are ready.
Created attachment 8565024 [details] [diff] [review]
[01] Bug 1126695: Support emulator-l in Cedar
Assignee: nobody → tzimmermann
Status: NEW → ASSIGNED
Created attachment 8567057 [details] [diff] [review]
Bug 1126695: Added B2G config for emulator-l (on m-c) [checkin: comment 39]
Attachment #8567057 - Flags: review?(ryanvm)
Comment on attachment 8567057 [details] [diff] [review]
Bug 1126695: Added B2G config for emulator-l (on m-c) [checkin: comment 39]

Sorry, I'm not a peer on this code.
Attachment #8567057 - Flags: review?(ryanvm) → review?(catlee)
Created attachment 8567058 [details] [diff] [review]
Bug 1126695: Added emulator-l to B2G bumper script (on mozharness) [checkin: comment 41]
Attachment #8567058 - Flags: review?(bhearsum)
Created attachment 8567059 [details] [diff] [review]
Bug 1126695: Added B2G L emulator (on tbpl)
Attachment #8567059 - Flags: review?(ryanvm)
Created attachment 8567061 [details] [diff] [review]
Bug 1126695: Added emulator-l to build configs (on buildbot_configs)
Attachment #8567061 - Flags: review?(mgervasini)
Created attachment 8567063 [details] [diff] [review]
Bug 1126695: Added emulator-l to trychooser (on tools) [checkin: comment 40]
Attachment #8567063 - Flags: review?(philringnalda)
I looked at bug 979450 and bug 984066, and copied what was done there.
Comment on attachment 8567059 [details] [diff] [review]
Bug 1126695: Added B2G L emulator (on tbpl)

Review of attachment 8567059 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM, but it's far more critical to get this fixed on Treeherder than TBPL at this point.

To patch Treeherder, you need to modify:
https://github.com/mozilla/treeherder-service/blob/master/treeherder/etl/buildbot.py#L273
https://github.com/mozilla/treeherder-ui/blob/master/webapp/app/js/values.js (both thPlatformNameMap and thPlatformOrder)

You can ask edmorley to review. Also, if I can ask a favor of you, while you're editing values.js, can you also add android-4-4-armv7-api11 to thPlatformOrder in the right spot? I forgot to do that when I added support for Android 4.4 tests recently.
Attachment #8567059 - Flags: review?(ryanvm) → review+
Comment on attachment 8567058 [details] [diff] [review]
Bug 1126695: Added emulator-l to B2G bumper script (on mozharness) [checkin: comment 41]

I don't think I'm an appropriate reviewer for this.
Attachment #8567058 - Flags: review?(bhearsum) → review?(catlee)
OK, I'll supply a TH patch as well.
Comment on attachment 8567061 [details] [diff] [review]
Bug 1126695: Added emulator-l to build configs (on buildbot_configs)

Review of attachment 8567061 [details] [diff] [review]:
-----------------------------------------------------------------

I've applied this patch to my local buildbot, and it fails to "make checkconfigure" with this error:

File ".../master/b2g_config.py", line 1945, in <module>
    BRANCHES['try']['platforms']['emulator-l']['slaves'] = TRY_SLAVES['mock']
KeyError: 'emulator-l'

You should probably add 'emulator-l' and 'emulator-l-debug' in 'try' section, around line 1639
Attachment #8567061 - Flags: review?(mgervasini) → review-
Created attachment 8567090 [details] [review]
Github pull request for treeherder-service [checkin: comment 25]
Attachment #8567090 - Flags: review?(emorley)
Created attachment 8567091 [details] [review]
Github pull request for treeherder-ui [checkin: comment 28]
Attachment #8567091 - Flags: review?(emorley)

Updated

3 years ago
Attachment #8567057 - Flags: review?(catlee) → review+
Comment on attachment 8567063 [details] [diff] [review]
Bug 1126695: Added emulator-l to trychooser (on tools) [checkin: comment 40]

Should work.
Attachment #8567063 - Flags: review?(philringnalda) → review+
Created attachment 8567099 [details] [diff] [review]
[01] Bug 1126695: Added emulator-l to build configs (on buildbot_configs) (v2)

Changes since v1:

  - added emulator-l to try branches (near line 1650)
  - enabled nightly builds with m-c and v2.2 (near line 1760)
  - remove emulator-{l,l-debug} for Gecko versions before 37 (near line 1950)

I talked to RyanVM about the nightly builds.

The Gecko version cleanups near line 1950 are similar to emulator-kk's cleanups on anything before b2g 1.4. emulator-l should only be build on b2g 2.2 or later.
Attachment #8567099 - Flags: review?(mgervasini)
Attachment #8567061 - Attachment is obsolete: true
Attachment #8567099 - Attachment description: Bug 1126695: Added emulator-l to build configs (on buildbot_configs) (v2) → [01] Bug 1126695: Added emulator-l to build configs (on buildbot_configs) (v2)
Created attachment 8567112 [details] [diff] [review]
[02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically

As discussed with RyanVM, JB/L opt builds shall run on every push; while JB/L debug builds shall only run periodically. I hope this patch already does the job.
Attachment #8567112 - Flags: review?(mgervasini)
Comment on attachment 8567058 [details] [diff] [review]
Bug 1126695: Added emulator-l to B2G bumper script (on mozharness) [checkin: comment 41]

Review of attachment 8567058 [details] [diff] [review]:
-----------------------------------------------------------------

the only thing missing is the b2g-lollipop tag on the platform_hardware_ril repo. please make sure to create that branch on that repo before landing this patch
Attachment #8567058 - Flags: review?(catlee) → review+
Comment on attachment 8567112 [details] [diff] [review]
[02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically

This needs changes to the Try config or we're gonna have a bad time.
https://hg.mozilla.org/build/buildbot-configs/file/tip/mozilla/b2g_config.py#l1883
Attachment #8567112 - Flags: feedback-

Updated

3 years ago
Summary: [B2G] Support Emulator-L on TPBL → [B2G] Support Emulator-L on buildbot
Comment on attachment 8567059 [details] [diff] [review]
Bug 1126695: Added B2G L emulator (on tbpl)

TBPL is code-frozen now, so we'll just want the Treeherder piece, not the TBPL one.
Attachment #8567059 - Attachment is obsolete: true
Attachment #8567059 - Flags: review+

Updated

3 years ago
Attachment #8567090 - Flags: review?(emorley) → review+
Created attachment 8567124 [details] [diff] [review]
[02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically (v2)

Changes since v1:

  - override try to build pushes but not periodically; as advised by Ryan
Attachment #8567112 - Attachment is obsolete: true
Attachment #8567112 - Flags: review?(mgervasini)
Attachment #8567124 - Flags: review?(mgervasini)
Comment on attachment 8567091 [details] [review]
Github pull request for treeherder-ui [checkin: comment 28]

These Treeherder PRs look great - I'll merge the service one straight away. If you could make the addition that Ryan mentioned on the UI one (adding a missing entry from a previous bug), squash/rebase/force-push, and comment on the PR (so I get notified; commits themselves don't trigger email) I'll merge that too :-)
Attachment #8567091 - Flags: review?(emorley) → review+

Comment 25

3 years ago
Commits pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/9bcc8aaa743e4a215107e25bc7548233fb7f3f83
Bug 1126695: Added emulator-l to build bot

https://github.com/mozilla/treeherder-service/commit/442b5cc086b7acf80bd46f29ee9f8f38106a91b3
Merge pull request #390 from tdz/bug-1226695

Bug 1126695 - Add support for B2G emulator-l jobs
(In reply to Chris AtLee [:catlee] from comment #20)
> Comment on attachment 8567058 [details] [diff] [review]
> Bug 1126695: Added emulator-l to B2G bumper script (on mozharness)
> 
> Review of attachment 8567058 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> the only thing missing is the b2g-lollipop tag on the platform_hardware_ril
> repo. please make sure to create that branch on that repo before landing
> this patch

Edgar, can you take care of this?
Flags: needinfo?(echen)
Comment on attachment 8567091 [details] [review]
Github pull request for treeherder-ui [checkin: comment 28]

Updated Github pull request

  - added entries for android-{4-4,5-0}-armv7-api11
Flags: needinfo?(emorley)

Comment 28

3 years ago
Commits pushed to master at https://github.com/mozilla/treeherder-ui

https://github.com/mozilla/treeherder-ui/commit/a628187e9f94b8fb9b0b7012058f8b0acec38993
Bug 1126695: Added android-{4-4,5-0}-armv7-api11 and emulator-l to Treeherder UI

The entries for android-{4-4,5-0}-armv7-api11 were not added in bug 1112693, so
we do that now.

https://github.com/mozilla/treeherder-ui/commit/a50c62524c3db0e477052c1c2afcbc7ffc91c8cd
Merge pull request #380 from tdz/bug-1126696

Bug 1126695: Added emulator-l to Treeherder UI

Updated

3 years ago
Flags: needinfo?(emorley)
Comment on attachment 8567124 [details] [diff] [review]
[02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically (v2)

Review of attachment 8567124 [details] [diff] [review]:
-----------------------------------------------------------------

I cannot apply this patch locally...


hg pull -u; hg update -C default; hg import --no-commit https://bugzilla.mozilla.org/attachment.cgi?id=8567124
pulling from ssh://hg.mozilla.org/build/buildbot-configs
searching for changes
no changes found
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
applying https://bugzilla.mozilla.org/attachment.cgi?id=8567124
patching file mozilla/b2g_config.py
Hunk #2 FAILED at 1287
Hunk #3 FAILED at 1355
Hunk #5 FAILED at 1937

I cannot apply this patch in fresh clone of buildbot-configs, could you regenerate it? 


Thanks!
Comment on attachment 8567099 [details] [diff] [review]
[01] Bug 1126695: Added emulator-l to build configs (on buildbot_configs) (v2)

Looks like the wrong patch got uploaded. This is the m-c patch (attachment 8567057 [details] [diff] [review]).
Attachment #8567099 - Attachment is obsolete: true
Attachment #8567099 - Flags: review?(mgervasini)
Attachment #8567124 - Flags: review?(mgervasini) → review-
Created attachment 8567823 [details] [diff] [review]
[01] Bug 1126695: Added emulator-l to build configs (v2) [checkin: comment 42]

Changes since v1:

  - please see comment #18

There are too many repos here and I'm getting confused. This should be the correct patch.
Attachment #8567823 - Flags: review?(mgervasini)
Created attachment 8567824 [details] [diff] [review]
[02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically (v3) [checkin: comment 42]

Changes since v2:

  - rebased onto [01]
Attachment #8567124 - Attachment is obsolete: true
Attachment #8567824 - Flags: review?(mgervasini)
Attachment #8567823 - Flags: review?(mgervasini) → review+
Attachment #8567824 - Flags: review?(mgervasini) → review+
(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #26)
> (In reply to Chris AtLee [:catlee] from comment #20)
> > Comment on attachment 8567058 [details] [diff] [review]
> > Bug 1126695: Added emulator-l to B2G bumper script (on mozharness)
> > 
> > Review of attachment 8567058 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > the only thing missing is the b2g-lollipop tag on the platform_hardware_ril
> > repo. please make sure to create that branch on that repo before landing
> > this patch
> 
> Edgar, can you take care of this?

Hi Chris and Thomas,

platform_hardware_ril is moved to new branching model [1], we are going to branch according to ril version, so there will be no b2g-lollipop, but b2g-ril_v* for example.
And currently platform_hardware_ril is only supporting ril version used in kitkat, there is a bug tracking lollipop status [2].
Before the platform_hardware_ril is ready for lollipop, emulator-l fetches hardware/ril codes from aosp directly in b2g-manifest [3].

1. Is it okay for B2G bumper script with above configuration?
2. Once platform_hardware_ril is ready for lollipop, b2g-manifest needs a revise to fetches code back from platform_hardware_ril, do we need to do anything in B2G bumper?

Thank you.

[1] https://wiki.mozilla.org/B2G/Emulator#Branch_Model
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1134982
[3] Please see https://github.com/mozilla-b2g/b2g-manifest/pull/286
Flags: needinfo?(tzimmermann)
Flags: needinfo?(echen)
Flags: needinfo?(catlee)
The bumper only requires that all the repositories / references in the manifests be mirrored onto git.m.o. It doesn't care about the actual branch names.

When I looked at your initial patch, the bumper process failed because the branch specified in the manifest didn't exist yet in the repositories.
Flags: needinfo?(catlee)
Nexus 5-L builds were enabled on b2g37_v2_2 today. Can we please make sure that Emulator-L builds are also run there? Same setup as trunk should be fine.
(In reply to Chris AtLee [:catlee] from comment #34)
> The bumper only requires that all the repositories / references in the
> manifests be mirrored onto git.m.o. It doesn't care about the actual branch
> names.
> 
> When I looked at your initial patch, the bumper process failed because the
> branch specified in the manifest didn't exist yet in the repositories.

I see, thanks for this clarification.
(In reply to Edgar Chen [:edgar][:echen] from comment #33)
> 
> platform_hardware_ril is moved to new branching model [1], we are going to
> branch according to ril version, so there will be no b2g-lollipop, but
> b2g-ril_v* for example.
> And currently platform_hardware_ril is only supporting ril version used in
> kitkat, there is a bug tracking lollipop status [2].
> Before the platform_hardware_ril is ready for lollipop, emulator-l fetches
> hardware/ril codes from aosp directly in b2g-manifest [3].
                          ^^^^
Sorry it should be `caf` not `aosp`.
And the caf/platform/hardware/ril is mirrored onto git.m.o already, http://git.mozilla.org/?p=external/caf/platform/hardware/ril.git;a=summary.

Thank you.

> 
> [1] https://wiki.mozilla.org/B2G/Emulator#Branch_Model
> [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1134982
> [3] Please see https://github.com/mozilla-b2g/b2g-manifest/pull/286
Flags: needinfo?(tzimmermann)
Depends on: 1116074
Now that bug 1116074 landed, emulator-l builds are supported.
Keywords: checkin-needed
OS: Linux → Gonk (Firefox OS)
Hardware: x86_64 → All
Attachment #8567090 - Attachment description: Github pull request for treeherder-service → Github pull request for treeherder-service [checkin: comment 25]
Attachment #8567091 - Attachment description: Github pull request for treeherder-ui → Github pull request for treeherder-ui [checkin: comment 28]
https://hg.mozilla.org/integration/b2g-inbound/rev/ea2b7d6407c4
https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/94530180141f
Keywords: checkin-needed
Comment on attachment 8567063 [details] [diff] [review]
Bug 1126695: Added emulator-l to trychooser (on tools) [checkin: comment 40]

https://hg.mozilla.org/build/tools/rev/82e3589bc31f
Attachment #8567063 - Attachment description: Bug 1126695: Added emulator-l to trychooser (on tools) → Bug 1126695: Added emulator-l to trychooser (on tools) [checkin: comment 40]
Comment on attachment 8567058 [details] [diff] [review]
Bug 1126695: Added emulator-l to B2G bumper script (on mozharness) [checkin: comment 41]

https://hg.mozilla.org/build/mozharness/rev/4e0a080517ed
Attachment #8567058 - Attachment description: Bug 1126695: Added emulator-l to B2G bumper script (on mozharness) → Bug 1126695: Added emulator-l to B2G bumper script (on mozharness) [checkin: comment 41]
Attachment #8567057 - Attachment description: Bug 1126695: Added B2G config for emulator-l (on m-c) → Bug 1126695: Added B2G config for emulator-l (on m-c) [checkin: comment 39]
Attachment #8565024 - Attachment is obsolete: true
Comment on attachment 8567823 [details] [diff] [review]
[01] Bug 1126695: Added emulator-l to build configs (v2) [checkin: comment 42]

https://hg.mozilla.org/build/buildbot-configs/rev/89a7634d0a6f
Attachment #8567823 - Attachment description: [01] Bug 1126695: Added emulator-l to build configs (v2) → [01] Bug 1126695: Added emulator-l to build configs (v2) [checkin: comment 42]
Comment on attachment 8567824 [details] [diff] [review]
[02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically (v3) [checkin: comment 42]

https://hg.mozilla.org/build/buildbot-configs/rev/fbc297beb01e
Attachment #8567824 - Attachment description: [02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically (v3) → [02] Bug 1126695: Run run JB,L opt on pushes; and JB,L debug periodically (v3) [checkin: comment 42]
Comment on attachment 8567058 [details] [diff] [review]
Bug 1126695: Added emulator-l to B2G bumper script (on mozharness) [checkin: comment 41]

v2.2 as well:
http://hg.mozilla.org/build/mozharness/rev/a28d8f302d52
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #41)
> https://hg.mozilla.org/build/mozharness/rev/4e0a080517ed

(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #44)
> http://hg.mozilla.org/build/mozharness/rev/a28d8f302d52

In production.
https://hg.mozilla.org/mozilla-central/rev/ea2b7d6407c4
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
status-firefox39: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 2.2 S7 (6mar)
Darn it, I meant to put a leave-open on this until the buildbot parts were in production and then forgot to.
Status: RESOLVED → REOPENED
status-b2g-v2.2: --- → affected
status-b2g-master: --- → affected
Resolution: FIXED → ---
The jobs got enabled in production today and we ran into bug 1120680. Easy fix.
https://hg.mozilla.org/mozilla-central/rev/600f44fd317c
Blocks: 1137359
At least on b2g37, these are hitting bustage. There's nobody around to fix it right now, so I've hidden them across all trees.
https://treeherder.mozilla.org/logviewer.html#?job_id=52453&repo=mozilla-b2g37_v2_2
Depends on: 1137651
Depends on: 1137654
With bug 1137651 landed, we're hitting this bustage now:
https://treeherder.mozilla.org/logviewer.html#?job_id=1419459&repo=b2g-inbound
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #50)
> With bug 1137651 landed, we're hitting this bustage now:
> https://treeherder.mozilla.org/logviewer.html#?job_id=1419459&repo=b2g-
> inbound

This should get fixed by bug 1137654. IIRC :mwu is at the MWC this week. I'll look for another reviewer.
Getting closer:
https://treeherder.mozilla.org/logviewer.html#?job_id=1430153&repo=b2g-inbound
Let's go through the stack trace

09:03:16     INFO - Running command: ['./repo', 'forall', '-c', 'git show-ref -q --head HEAD || rm -rfv $PWD'] in /builds/slave/b2g_b2g-in_emu-l_dep-000000000/build
09:03:16     INFO - Copy/paste: ./repo forall -c "git show-ref -q --head HEAD || rm -rfv $PWD"

This command fails. repo checks the heads of a git repository exists or otherwise deletes the local checkout (?)

09:03:16     INFO -  Traceback (most recent call last):
09:03:16     INFO -    File "/builds/slave/b2g_b2g-in_emu-l_dep-000000000/build/.repo/repo/main.py", line 506, in <module>
09:03:16     INFO -      _Main(sys.argv[1:])
09:03:16     INFO -    File "/builds/slave/b2g_b2g-in_emu-l_dep-000000000/build/.repo/repo/main.py", line 482, in _Main
09:03:16     INFO -      result = repo._Run(argv) or 0
09:03:16     INFO -    File "/builds/slave/b2g_b2g-in_emu-l_dep-000000000/build/.repo/repo/main.py", line 161, in _Run
09:03:16     INFO -      result = cmd.Execute(copts, cargs)
09:03:16     INFO -    File "/builds/git-shared/repo/repo/subcmds/forall.py", line 200, in Execute
09:03:16     INFO -      mirror = self.manifest.IsMirror
09:03:16     INFO -    File "/builds/git-shared/repo/repo/manifest_xml.py", line 358, in IsMirror
09:03:16     INFO -      return self.manifestProject.config.GetBoolean('repo.mirror')

Here, repo tries to figure out if the manifest repository mirrors an existing git repository.

09:03:16     INFO -    File "/builds/git-shared/repo/repo/git_config.py", line 112, in GetBoolean
09:03:16     INFO -      v = self.GetString(name)
09:03:16     INFO -    File "/builds/git-shared/repo/repo/git_config.py", line 129, in GetString
09:03:16     INFO -      v = self._cache[_key(name)]
09:03:16     INFO -    File "/builds/git-shared/repo/repo/git_config.py", line 246, in _cache
09:03:16     INFO -      self._cache_dict = self._Read()
09:03:16     INFO -    File "/builds/git-shared/repo/repo/git_config.py", line 253, in _Read
09:03:16     INFO -      self._SaveJson(d)
09:03:16     INFO -    File "/builds/git-shared/repo/repo/git_config.py", line 283, in _SaveJson
09:03:16     INFO -      if os.path.exists(self.json):
09:03:16     INFO -  AttributeError: 'GitConfig' object has no attribute 'json'
09:03:16    ERROR - Return code: 1

Repo tries to read the 'mirror' property from a cache file. As the cache doesn't exist, it tries to create the file, which fails. I ran the repo command locally and grepped for repo configs. The result is

  ./manifests.git/.repo_config.json
  ./repo/.git/.repo_config.json
Running

  ./repo forall -c "git show-ref --head HEAD"

locally doesn't produce any errors. The only info I could find online is [1]. It talks about incorrect file permissions.

Ryan, I think this is a problem on the build server. Could you check if the cache files are there and manifest directories are writable by the build job?

[1] https://code.google.com/p/git-repo/issues/detail?id=190
Flags: needinfo?(ryanvm)
(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #54)
> Ryan, I think this is a problem on the build server. Could you check if the
> cache files are there and manifest directories are writable by the build job?

I don't have access to the slaves, so no. Try asking in #releng?
Flags: needinfo?(ryanvm)
(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #55)
> (In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #54)
> > Ryan, I think this is a problem on the build server. Could you check if the
> > cache files are there and manifest directories are writable by the build job?
> 
> I don't have access to the slaves, so no. Try asking in #releng?

Peter, could you take a look? See comment 53 and below for the context.
Flags: needinfo?(pmoore)
-> buildduty
Flags: needinfo?(pmoore) → needinfo?(coop)
Sorry, yesterday was a nightmare. I'll have a look today to see if it's something obvious. 

Standard proviso: I can't debug an actual build problem, but am happy to loan an instance to someone who can. (https://wiki.mozilla.org/ReleaseEngineering/How_To/Request_a_slave)
Flags: needinfo?(coop)
(In reply to Chris Cooper [:coop] from comment #58)
> Sorry, yesterday was a nightmare. I'll have a look today to see if it's
> something obvious. 

Hi Chris, any news here?
Flags: needinfo?(coop)
I had trouble tracking down a slave that still had the build directory intact. The spot instances often get recycled before investigation is possible.

I just found this Emulator-L build from today:

https://treeherder.mozilla.org/#/jobs?repo=b2g-inbound&revision=ab7f61f08c42&exclusion_profile=false

The build is still failing, but not on an Attribute Error.

That particular check-in is a manifest bump, so maybe the AttributeError is fixed in one of the new revs included in the bump?
Flags: needinfo?(coop)
Emulator-L builds have a lot of logspam like the below as well:
16:47:00     INFO -  prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-ar: /lib64/libz.so.1: no version information available (required by prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-ar)
Ted,

the makefile rules around tests names 'steeplechase' fail on emulator L in the build of comment 60. Can you help debugging? The full log is at [1]. The failed command is [2].

[1] https://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/b2g-inbound-emulator-l-debug/1425605419/b2g_b2g-inbound_emulator-l-debug_dep-bm72-build1-build2.txt.gz
[2] https://mxr.mozilla.org/mozilla-central/source/testing/testsuite-targets.mk#531
Flags: needinfo?(ted)
I'm guessing you must have --disable-webrtc in the mozconfig right now? The only steeplechase tests we have are behind a MOZ_WEBRTC check:
https://dxr.mozilla.org/mozilla-central/source/dom/media/moz.build#73

You could safely wrap the guts of the stage-steeplechase target in ifdef MOZ_WEBRTC to work around this:
stage-steeplechase: make-stage-dir
ifdef MOZ_WEBRTC
    <...>
endif
Flags: needinfo?(ted)
Bug 1137515 literally landed and bounced on inbound today. So maybe we just wait for that to stick and see where that gets us. Though it'd probably be a decent safety move to do the ifdef wrap for any future issues that might arise.
Depends on: 1142495
Opt builds are green now that bug 1137515 has landed, so webrtc being disabled was indeed the issue. I filed bug 1142494 for only packaging steeplechase tests if webrtc is enabled.
Depends on: 1137515
Depends on: 1143053
Emulator L builds are green and running on all expected trees. I've unhidden them. I think we're done here! :)
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
status-b2g-v2.2: affected → fixed
status-b2g-master: affected → fixed
status-firefox39: fixed → ---
Resolution: --- → FIXED
Target Milestone: 2.2 S7 (6mar) → 2.2 S8 (20mar)

Comment 67

2 years ago
Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/27c1608e6a38de306176edcef0fc21189606dd30
Bug 1126695: Added android-{4-4,5-0}-armv7-api11 and emulator-l to Treeherder UI

The entries for android-{4-4,5-0}-armv7-api11 were not added in bug 1112693, so
we do that now.

https://github.com/mozilla/treeherder/commit/056a128a53a992fafc6afd42740d2a90504ad813
Merge pull request #380 from tdz/bug-1126696

Bug 1126695: Added emulator-l to Treeherder UI
You need to log in before you can comment on or make changes to this bug.