767745 - release l10n repacks not using releng hg mirrors

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect, P2)

Description

[hwine]

Affected FF 13.0.2 chemspill. Bug 767739 was presenting issue, but Shyam pointed out we would have avoided any impact to build vpn (including releases) if we'd been using the hg mirrors set up for us.

Most of the build infra was done via bug 745299, but repacks don't use hgtool, so only use the public hg web servers. When the public hg web servers are having troubles, repacks fail.

Comment 1

[[e-mail] (:bhearsum)]

Is this a dupe of bug 734141?

Comment 2

[Rail Aliiev [:rail]]

I think that it's related to release l10n repacks.

Comment 3

[hwine]

This isn't a dupe - it would be a reopen of bug 745299 (change wasn't made everywhere needed). (It could be resolved by changing the repository listed in the config files to the internal server, but that would obfuscate the audit trail.)

Comment 4

[Chris Cooper [:coop] (he/him)]

This was brought up at the relops mtg last week, so I've cc-ing Shyam here who's providing the impetus from the IT side.

AIUI, Shyam would like us to switch *all* build network hg traffic to the dedicated releng internal hg mirrors. I _think* that means using hg-internal.dmz.scl3.mozilla.com. Many/most hg requests from the build network are already using this set of hosts, but l10n is one of the standout examples that still are not.

Shyam: is that an accurate assessment?

Comment 5

[[e-mail] (:bhearsum)]

Coop, I'm not volunteering you to do all of the work here, but since you're working with Shyam on it right now I'm assigning to you. Throw it back if you think that's unfair.

Comment 6

[Shyam Mani [:fox2mike]]

(In reply to Chris Cooper [:coop] from comment #4)
> This was brought up at the relops mtg last week, so I've cc-ing Shyam here
> who's providing the impetus from the IT side.
> 
> AIUI, Shyam would like us to switch *all* build network hg traffic to the
> dedicated releng internal hg mirrors. I _think* that means using
> hg-internal.dmz.scl3.mozilla.com. Many/most hg requests from the build
> network are already using this set of hosts, but l10n is one of the standout
> examples that still are not.
> 
> Shyam: is that an accurate assessment?

Yes sir, this is correct.

Comment 7

[Shyam Mani [:fox2mike]]

(In reply to Shyam Mani [:fox2mike] from comment #6)
> Yes sir, this is correct.

To clarify : I'm not sure what part of releng infra is using this and what part isn't. I would like to make sure that _all_ of releng's build infra does their hg operations on hg-internal.dmz.scl3.mozilla.com vs hg.mozilla.org.

Comment 8

[Chris Cooper [:coop] (he/him)]

Attachment: Set mirrors for l10n dep/nightly repacks

I discovered that we were purposefully avoiding mirrors for locales with hgtool  by setting mirrors=[] for regular dep/nightly repacks. This patch fixes that, assuming there wasn't some reason for avoiding the mirrors in the first place.

Switching release repacks to use hgtool is going to be more invasive.

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 657333 [details] [diff] [review]
Set mirrors for l10n dep/nightly repacks

Review of attachment 657333 [details] [diff] [review]:
-----------------------------------------------------------------

::: process/factory.py
@@ +3178,5 @@
>                                   self.l10nRepoPath + "/%(locale)s"),
>                  workdir='%s/%s' % (self.baseWorkDir, self.l10nRepoPath),
>                  locks=[hg_l10n_lock.access('counting')],
>                  use_properties=False,
> +                mirrors = mirrors,

Please get rid of the whitespace around the '=' when landing, for consistency.

Comment 10

[Chris Cooper [:coop] (he/him)]

Comment on attachment 657333 [details] [diff] [review]
Set mirrors for l10n dep/nightly repacks

https://hg.mozilla.org/build/buildbotcustom/rev/c246b38678be

This is in production now.

Comment 11

[[e-mail] (:bhearsum)]

I think we're done here?