1254401 - (operation_ysnp) [Tracking Bug] Migrate SeaMonkey off aus2 to balrog.

Status: RESOLVED WONTFIX

(SeaMonkey :: Release Engineering, defect)

Description

[Edmund Wong (:ewong)]

We need to migrate off Aus2-community and use balrog as soon as possible.

Comment 1

[Rainer Bielefeld]

User information: Details concerning Application Update Service "AUS2" and Balrog see here: <https://wiki.mozilla.org/AUS>

Comment 2

[[e-mail] (:bhearsum)]

Here's an off-the-top-of-my-head summary of what needs to happen for this:
* Responsible humans need access to aus4-admin.mozilla.org. This request needs to go through IT, and will require usage of the VPN.
* Submit data for Seamonkey builds to Balrog. This can be done by hand, or through automation. Firefox uses the scripts talked about on https://wiki.mozilla.org/Balrog#Scripts to do this. If automation is doing this, you'll need a system account (like "seabld") and netflows that allow you to get to aus4-admin.mozilla.org.
* Once you have data, you'll need to create a rule or rules to route your updates. You can look at how Firefox or Thunderbird updates are set-up for inspiration.

Comment 3

[Edmund Wong (:ewong)]

(In reply to Ben Hearsum (:bhearsum) from comment #2)
> Here's an off-the-top-of-my-head summary of what needs to happen for this:
> * Responsible humans need access to aus4-admin.mozilla.org. This request
> needs to go through IT, and will require usage of the VPN.
> * Submit data for Seamonkey builds to Balrog. This can be done by hand, or
> through automation. Firefox uses the scripts talked about on
> https://wiki.mozilla.org/Balrog#Scripts to do this. If automation is doing
> this, you'll need a system account (like "seabld") and netflows that allow
> you to get to aus4-admin.mozilla.org.
> * Once you have data, you'll need to create a rule or rules to route your
> updates. You can look at how Firefox or Thunderbird updates are set-up for
> inspiration.

I will get on this right now.  I'm wondering if it's even possible to
do this with the current release? (i.e. 2.40)

Comment 4

[[e-mail] (:bhearsum)]

(In reply to Edmund Wong (:ewong) from comment #3)
> (In reply to Ben Hearsum (:bhearsum) from comment #2)
> > Here's an off-the-top-of-my-head summary of what needs to happen for this:
> > * Responsible humans need access to aus4-admin.mozilla.org. This request
> > needs to go through IT, and will require usage of the VPN.
> > * Submit data for Seamonkey builds to Balrog. This can be done by hand, or
> > through automation. Firefox uses the scripts talked about on
> > https://wiki.mozilla.org/Balrog#Scripts to do this. If automation is doing
> > this, you'll need a system account (like "seabld") and netflows that allow
> > you to get to aus4-admin.mozilla.org.
> > * Once you have data, you'll need to create a rule or rules to route your
> > updates. You can look at how Firefox or Thunderbird updates are set-up for
> > inspiration.
> 
> I will get on this right now.  I'm wondering if it's even possible to
> do this with the current release? (i.e. 2.40)

You can certainly create the data by hand for the current release and submit it.

One thing I forgot to mention above is that when we think Balrog is ready, aus2-community needs to be redirected to it. That might be complicated due to the SSL pinning requirements. For MoCo AUS we did this by terminating SSL at the load balancer the same as before, and then fulfilling the requests by Balrog instead of AUS. I'm not sure if this will be possible for SeaMonkey.

Comment 5

[Edmund Wong (:ewong)]

With bug 1271761 and bug 1274722, I think updating with aus2-community is going
to be a pain.  The faster we move to balrog, the better.  :)

I remembered that I was working on this and I have access to aus4 (re:
bug 1254535).

Next, I need to (iirc) create a seabld update user on aus4, and then
(hand-waving) do some stuff on the buildbot-config/buildbotcustom code.

Am I right, bhearsum?  If so, I might as well get on it now.

Comment 6

[[e-mail] (:bhearsum)]

(In reply to Edmund Wong (:ewong) from comment #5)
> With bug 1271761 and bug 1274722, I think updating with aus2-community is
> going
> to be a pain.  The faster we move to balrog, the better.  :)
> 
> I remembered that I was working on this and I have access to aus4 (re:
> bug 1254535).
> 
> Next, I need to (iirc) create a seabld update user on aus4, and then
> (hand-waving) do some stuff on the buildbot-config/buildbotcustom code.
> 
> Am I right, bhearsum?  If so, I might as well get on it now.

If you're submitting through automation, you'll need to get the seabld user allowed to talk to aus4-admin.mozilla.org. This will require at least one group change in LDAP, and possibly some netflow work. These parts need to be done by IT. Once those are done, I'll need to grant some permissions to it within Balrog.

The submission scripts we use are in https://github.com/mozilla/build-tools/tree/master/scripts/updates. How you use this with your automation is up to you. You can find example invocations in Nightly and Release build logs. For releases, submission happens during en-US builds, l10n repacks, and the "updates" builder.

Comment 7

[Edmund Wong (:ewong)]

(In reply to Ben Hearsum (:bhearsum) from comment #6)
> (In reply to Edmund Wong (:ewong) from comment #5)
> > With bug 1271761 and bug 1274722, I think updating with aus2-community is
> > going
> > to be a pain.  The faster we move to balrog, the better.  :)
> > 
> > I remembered that I was working on this and I have access to aus4 (re:
> > bug 1254535).
> > 
> > Next, I need to (iirc) create a seabld update user on aus4, and then
> > (hand-waving) do some stuff on the buildbot-config/buildbotcustom code.
> > 
> > Am I right, bhearsum?  If so, I might as well get on it now.
> 
> If you're submitting through automation, you'll need to get the seabld user
> allowed to talk to aus4-admin.mozilla.org. This will require at least one
> group change in LDAP, and possibly some netflow work. These parts need to be
> done by IT. Once those are done, I'll need to grant some permissions to it
> within Balrog.

Thanks :bhearsum.  (Sorry for the delay..)

So iiuc:

1) get seabld some access credentials to aus4-admin.mozilla.org 
2) get some netflow permissions set for the machines that need
   access to aus4. 
3) once #1 and #2 are done, I speak to you re: permissions on 
  seabld's user?
(for both #1 and #2, I need to chat with IT?)

> 
> The submission scripts we use are in
> https://github.com/mozilla/build-tools/tree/master/scripts/updates. How you
> use this with your automation is up to you. You can find example invocations
> in Nightly and Release build logs. For releases, submission happens during
> en-US builds, l10n repacks, and the "updates" builder.

Thanks!  Very much appreciate the help.

Edmund

Comment 8

[Edmund Wong (:ewong)]

ftr (just in case I forget),

Access to both admin pages have been granted.  Next steps:

1) Adapt (hand-waving) the mozharness balrog code to our usage
  1a) need to figure out what credentials are needed to use the api.
2) test out manually the balrog submission code
3) Add necessary code to buildbot-config
4) Add necessary code to buildbotcustom
5) Add the necessary credentials to the puppetmaster.

Trees to work on in order of not-screwing-things-up-royally:
 1) Nightly
 2) beta/release

that is the plan.

Comment 9

[Edmund Wong (:ewong)]

Access to the stage host is ok.

The next 'easy' step is to get the credentials for seabld and stage-seabld.

Comment 10

[Edmund Wong (:ewong)]

I believe I have both credentials.

Next step:

Import aus2 stuff to balrog-admin.  [hand-wavey]

Comment 11

[Edmund Wong (:ewong)]

Closing as we won't be using balrog.