945208 - B2G-manifests were pointing to a repo that is not yet mirrored at git.m.o

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Carsten Book [:Tomcat]]

seems on all trees that build B2g Builds the build fail with 03:50:29    FATAL - caught OS error 2: No such file or directory while running ['./gonk-misc/add-revision.py', '-o', 'sources.xml', '--force', '.repo/manifest.xml']

a example is https://tbpl.mozilla.org/php/getParsedLog.php?id=31312633&full=1&branch=mozilla-central#error0 and seems it was trying to pull data from
https://git.mozilla.org/external/caf/platform/development but failed 

Closing all affected trees for this

Comment 1

[Carsten Book [:Tomcat]]

seems at least a issue for nexus and B2G JB Emulator builds

Comment 2

[Carsten Book [:Tomcat]]

also now errors like https://tbpl.mozilla.org/php/getParsedLog.php?id=31313022&tree=B2g-Inbound is this the Ec2 issue like bug 940356 again ?

Comment 3

[Pete Moore [:pmoore][:pete]]

The first error I see is:

03:49:07     INFO -  fatal: remote error: FATAL: R any external/caf/platform/prebuilts/gcc/linux-x86/x86/i686-linux-android-4.7 gitweb DENIED by fallthru

I will have a look at the repo manifest to find out which repos seem to be exhibiting the problem at the moment (e.g. it could be a github issue or a git.m.o issue). The repo command that is running is fetching data from lots of other repositories.

It looks like https://git.mozilla.org/external/caf/platform/development is actually ok - in a web browser you get an HTTP 404, but if you clone it (git clone https://git.mozilla.org/external/caf/platform/development) it works. This is expected behaviour. I'll update this ticket when I know which git repo(s) is/are causing the problem.

Comment 4

[Pete Moore [:pmoore][:pete]]

Running
./repo sync
from /builds/slave/b2g_m-cen_emu-jb-d_dep-0000000/build on bld-linux64-ec2-192 as cltbld...

Comment 5

[Pete Moore [:pmoore][:pete]]

Was just investigating when .....

[[e-mail] build]$ ./repo list | while read repo xxx; do ./repo sync "${repo}" >/dev/null 2>/dev/null || echo "$repo"; done

Broadcast message from [e-mail]
	(unknown) at 5:11 ...

The system is going down for power off NOW!
Connection to bld-linux64-ec2-192 closed by remote host.
Connection to bld-linux64-ec2-192 closed.



I will either run this from a different host or find a way to start this instance back up...

Comment 6

[Carsten Book [:Tomcat]]

(In reply to Carsten Book [:Tomcat] from comment #2)
> also now errors like
> https://tbpl.mozilla.org/php/getParsedLog.php?id=31313022&tree=B2g-Inbound
> is this the Ec2 issue like bug 940356 again ?

according to bhearsum on irc this should be fixed by https://bugzilla.mozilla.org/show_bug.cgi?id=792836

Comment 7

[Armen [:armenzg]]

I backed out the change:
https://github.com/mozilla-b2g/b2g-manifest/commit/67bb04e0ef58d92857c9dd37b72616b508e2d3b3

We have to wait until git.m.o mirrors github before re-opening the trees.

The problem that happened in here is due to adding a new repo to the b2g manifests which has not yet been setup to be mirrored.
See https://bugzilla.mozilla.org/show_bug.cgi?id=941478.

See this:
  https://wiki.mozilla.org/ReleaseEngineering/RepositoryCreationRequest#Requester.27s_Actions

Comment 8

[Carsten Book [:Tomcat]]

trees reopned now that the root cause is fixed (thanks everyone!) in case it takes time that git.m.o need to pick this change we can retrigger failed jobs