1007259 - Manage repo checkout directly from b2g_build.py

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Chris AtLee [:catlee]]

b2g_build.py should be able to directly manage the repo tool, so we can control which repository it comes from, and which revision of it we use.

Comment 1

[Chris AtLee [:catlee]]

Attachment: set origin remote when cloning

'repo sync' will try and sync the local checkout of 'repo', which will fail if we clone it right now with gittool since the 'origin' remote doesn't get set.

this patch makes sure we have the 'origin' remote set for our cloned repos.

Comment 2

[Chris AtLee [:catlee]]

Attachment: use gittool to checkout repo

Comment 3

[Aki Sasaki (not active)]

Comment on attachment 8418900 [details] [diff] [review]
use gittool to checkout repo

This will work, but:

a) do you want a cleanup method in case the clone goes south, or do you want to error out purple and have the next job clobber?

   self.retry(self.checkout_repotool, cleanup=self.clobber_repotool, args=(repo_dir,))

b) There's no error checking here; do you want to error out on either the retry or the ./repo --version if unsuccessful?  Ideally we exit 3 (infrastructure) on failure; looks like we might not yet have support for retry() to set the self.return_code on failure.

Comment 4

[Rail Aliiev [:rail]]

Comment on attachment 8418894 [details] [diff] [review]
set origin remote when cloning

Review of attachment 8418894 [details] [diff] [review]:
-----------------------------------------------------------------

::: lib/python/util/git.py
@@ +152,5 @@
> +    cmd = ['git', 'remote', '-v']
> +    proc = subprocess.Popen(cmd, cwd=repo, stdout=subprocess.PIPE)
> +    proc.wait()
> +    for line in proc.stdout.readlines():
> +        m = re.match("%s\s+(\S+) \(fetch\)$" % re.escape(remote_name), line)

It'd be better to use "raw" string (r"%s....") here.

Comment 5

[Chris AtLee [:catlee]]

Attachment: gittool-puppet.patch

update dependency-free version of gittool in puppet

Comment 6

[Rail Aliiev [:rail]]

Comment on attachment 8419616 [details] [diff] [review]
gittool-puppet.patch

O.K.! :D

Comment 7

[Rail Aliiev [:rail]]

(In reply to Chris AtLee [:catlee] from comment #5)
> Created attachment 8419616 [details] [diff] [review]
> gittool-puppet.patch
> 
> update dependency-free version of gittool in puppet

merged to production in https://hg.mozilla.org/build/puppet/rev/c702366a2164

Comment 8

[Chris AtLee [:catlee]]

Attachment: use gittool to checkout repo

Comment 9

[Aki Sasaki (not active)]

Comment on attachment 8422390 [details] [diff] [review]
use gittool to checkout repo

>+        # self.vcs_checkout already retries, so no need to wrap it in
>+        # self.retry. We set the error_level to ERROR to prevent it going fatal
>+        # so we can do our own handling here.
>+        retval = self.vcs_checkout_repos(repos, error_level=ERROR)
>+        if not retval:
>+            self.rmtree(repo_dir)
>+            self.fatal("Automation Error: couldn't clone repo", exit_code=5)

I would have thought 3, here: http://hg.mozilla.org/build/mozharness/file/e3fb9a135242/mozharness/mozilla/buildbot.py#l40
What's 5?

>+                cmd = ['./repo', '--version']
>+                if not self.run_command(cmd, cwd=dirs['work_dir']) == 0:
>+                    # Set return code to RETRY
>+                    self.fatal("repo is broken", exit_code=5)

Here too.

I guess makeMHFactory doesn't have a log_eval_func, so any non-zero is red, but we may want to match the standard at some point.

Comment 10

[Chris AtLee [:catlee]]

Comment on attachment 8422390 [details] [diff] [review]
use gittool to checkout repo

checked-in with s/5/4/