1163759 - Deploy dynamic UA file (ua-update.json) to Dynamic UA CDN for Fennec

Status: VERIFIED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Mike Taylor [:miketaylr]]

In Bug 1162099 we're going to start using the dynamic user agent mechanism that was implemented in Bug 897221. B2G currently uses this.

This bug is to ensure that the Fennec ua-update.json file is synchronized to our https://dynamicua.cdn.mozilla.net CDN. Looking at the deploy script [1], it will need to be extended to handle the ua-update.json file for Fennec in addition to B2G.

I plan on submitting patches today with the "general.useragent.updates.enabled" pref set to false until we're ready to move to the testing phase from y'all end.

[1] https://github.com/mozilla/dynamicua_update_script

Comment 1

[Mike Taylor [:miketaylr]]

When I inquired about this bug, I was given the following advice:

"If you have timeline requirements for "is staged for testing", "is
deployed but not live", and/or "is live to all clients", please note
those in the bug so we're aware."

Karen or Shell, could you give some insight into that please? (I'm  alsohappy to guess at things).

Comment 2

[Karen Rudnitski [:kar]]

I would like 'staged for testing' ASAP in order to show tangible process in time for a May 20 check-in (so likely a couple of days prior?). If not feasible (since that's next week!), shortly thereafter.

I would also love to have 'live to all clients' by June 20 if possible, as this is a high-vis item that (again) shows progress for an important check-point with more influential stakeholders involved.

Let me know if these are at all possible.

Thanks!

Comment 3

[Shyam Mani [:fox2mike]]

Mike,

I believe we can push out the changes asap (when they're ready to go). I don't think we have a stage environment here so we will be making these changes directly to production. So basically, we're ready to go when you are, once the changes are made on github we can update the site and you should be good to go.

Comment 4

[Mike Taylor [:miketaylr]]

Thanks Shyam! When 1162099 lands, with "general.useragent.updates.enabled" set to false, we can proceed then. We can then manually flip the pref to true for testing before pulling the trigger for everyone.

Comment 5

[Shyam Mani [:fox2mike]]

(In reply to Mike Taylor [:miketaylr] from comment #4)
> Thanks Shyam! When 1162099 lands, with "general.useragent.updates.enabled"
> set to false, we can proceed then. We can then manually flip the pref to
> true for testing before pulling the trigger for everyone.

Cool. Just poke us here when it's ready for the update.

Comment 6

[Mike Taylor [:miketaylr]]

Hi Shyam, we're ready for a test deploy now. Thanks!

Comment 7

[:Atoll]

So, :miketaylr, we're looking at this bug now and it's not clear how we would *correctly* extend the update script to support Fennec.


1. For this URL for b2g:

> https://hg.mozilla.org/mozilla-central/raw-file/tip/b2g/app/ua-update.json.in

What is the appropriate fennec URL to use?


2. The filename for B2G is a GUID.

> B2G_APP_ID="{3c2e2abc-06d4-11e1-ac3b-374f68613e61}"
> FILENAME=$B2G_APP_ID

What is the appropriate fennec GUID to use?

Comment 8

[Mike Taylor [:miketaylr]]

Hi Richard,

(In reply to Richard Soderberg [:atoll] from comment #7)
> So, :miketaylr, we're looking at this bug now and it's not clear how we
> would *correctly* extend the update script to support Fennec.
> 
> 
> 1. For this URL for b2g:
> 
> > https://hg.mozilla.org/mozilla-central/raw-file/tip/b2g/app/ua-update.json.in
> 
> What is the appropriate fennec URL to use?

https://hg.mozilla.org/mozilla-central/raw-file/tip/mobile/android/app/ua-update.json.in

Note that we're using the #slashslash preprocessor directive to remove the comments from the JSON, rather than sed in a Makefile (which B2G does currently). So that might complicate your script a tiny bit. 

I plan on doing the same thing for B2G in Bug 1164580.

> 2. The filename for B2G is a GUID.
> 
> > B2G_APP_ID="{3c2e2abc-06d4-11e1-ac3b-374f68613e61}"
> > FILENAME=$B2G_APP_ID
> 
> What is the appropriate fennec GUID to use?

"{aa3c5121-dab2-40e2-81ca-7ea25febc110}"

(from https://dxr.mozilla.org/mozilla-central/source/mobile/android/confvars.sh#51)

Comment 9

[:Atoll]

(In reply to Mike Taylor [:miketaylr] from comment #8)
> Note that we're using the #slashslash preprocessor directive to remove the
> comments from the JSON, rather than sed in a Makefile (which B2G does
> currently). So that might complicate your script a tiny bit. 

I'm not sure we're the best people to be altering this script - you seem more familiar with how it works than we are, and I have no idea what you mean by the above comment at all.

I do see a sed command in this script, and maybe that's what you're talking about. What command would replace it, for the Fennec URL?

Would you like commit access to the script's repo, so that you can manage issued and commits and ask us to pull updates from the repo when you're ready for us to do so?

(If not, we'll keep trying to maintain this, but it feels like an unnecessary 'man in the middle' that will eventually introduce bugs or outages.)

Comment 10

[Mike Taylor [:miketaylr]]

(In reply to Richard Soderberg [:atoll] from comment #9)
> Would you like commit access to the script's repo, so that you can manage
> issued and commits and ask us to pull updates from the repo when you're
> ready for us to do so?

Sure, that sounds fine. I'm not the best shell scripter, but there's plenty of smart people around to ask for help. :)

Just so we're on the same page, I assume we're talking about giving me commit access to <https://github.com/mozilla/dynamicua_update_script> and then I ping y'all when I think it's ready to go. My github account is at https://github.com/miketaylr/, (unless I need to ask someone else to add me to the Mozilla org...).

Comment 11

[:Atoll]

If it helps any, I'm a great shell scripter, I just have no idea what's going on with this process beyond what I've worked out so far :) So if you can propose a vague sort of diff that alters the script to work correctly for Fennec, I can take that diff and integrate it into the script in a way that lets the old B2G and the new Fennec method coexist peacefully.

Comment 12

[Mike Taylor [:miketaylr]]

Alright, sounds good. :)

I'll open an issue on the GitHub repo and we can follow up there until it's ready to actually be deployed.

Comment 13

[Mike Taylor [:miketaylr]]

Here's the issue: https://github.com/mozilla/dynamicua_update_script/issues/2

Comment 14

[:shell escalante]

Hi Richard,  Looks like everything has landed or is there that was blocking... do you know when you'll have a chance to look at this?

Comment 15

[:shell escalante]

Hi Richard,  10 top priority Japanese sites can be fixed to display when this lands.  When does this fall in your schedule to fix? (i apologize if you gave info in the webcompat site directly).  I can keep updating as there are new sites identified that this patch would impact


http://yahoo.co.jp
*.yahoo.co.jp sites
rakuten.co.jp
mixi.jp 
* www.aeon.co.jp 
uniqlo.com mobile site is http://www.uniqlo.com/jp/sp/
nicovideo.jp
nhk.or.jp 
* www.nhk.or.jp/sp/ 
navitime.co.jp Needs Android version # + "AppleWebKit" to get to touch site.
* www.navitime.co.jp mobile at http://touch.navitime.co.jp. "AppleWebKit" in UA is minimal requirement to get 302 to touch site.

Comment 16

[:Atoll]

I'm testing the updated script today (there's a pull request at Github with the commits that were neceessary to add support). I'm behind several days from my intended schedule due to illness last week, but I *think* we'll be able to deploy it tomorrow (presuming folks are available to help me test the result).

Comment 17

[:Atoll]

I did a before/after test with my new code [1] and it has the desired effect, with the following changes occurring:

- blank lines are removed from the JSON output file (by the new sed command)
- the '#filter slashslash' header is removed (by the new sed command)
- the file 'file_revision.txt' is split into 'b2g_revision.txt' and 'fennec_revision.txt' (is this acceptable?)

So, presuming these are all acceptable changes, I'm ready to deploy this.

[1] https://github.com/mozilla/dynamicua_update_script/pull/3

Comment 18

[:Atoll]

I verified in the weblogs that nothing is fetching file_revision.txt, which is (as far as I can tell) used only for version-caching by the updater script. So if the b2g and fennec files split into different revisions someday, it'll continue working correctly.

We receive dynamicua requests for two GUIDs that are not supported by the updater script:

ec8030f7-c20a-464f-9b0e-13a3a9e97384
d1bfe7d9-c01e-4237-998b-7b5f960a4314

The two supported GUIDs will be:

3c2e2abc-06d4-11e1-ac3b-374f68613e61  b2g
aa3c5121-dab2-40e2-81ca-7ea25febc110  fennec

As indicated in the updater script [2]:

  # update the files on disk
  update_app  "b2g"     "b2g/app"             "{3c2e2abc-06d4-11e1-ac3b-374f68613e61}"
  update_app  "fennec"  "mobile/android/app"  "{aa3c5121-dab2-40e2-81ca-7ea25febc110}"

[2] https://github.com/floatingatoll/dynamicua_update_script/blob/pull-1/update#L114

Comment 19

[Mike Taylor [:miketaylr]]

(In reply to Richard Soderberg [:atoll] from comment #18)
> We receive dynamicua requests for two GUIDs that are not supported by the
> updater script:
> 
> ec8030f7-c20a-464f-9b0e-13a3a9e97384
> d1bfe7d9-c01e-4237-998b-7b5f960a4314

The first is Firefox desktop--no idea what the second is. If somebody needs dynamic updates to Desktop, they can file a bug. ^_^

Comment 20

[:Atoll]

Okay. Deployed and verified creating the two expected GUIDs with the expected content. Let me know if you have any problems!

Comment 21

[Mike Taylor [:miketaylr]]

Just verified locally, thanks!