1315878 - blocklist update failing: script not able to detect the additional valid encoding

Status: RESOLVED FIXED

(SeaMonkey :: Release Engineering, defect)

Description

[Edmund Wong (:ewong)]

HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://blocklists.settings.services.mozilla.com/v1/blocklist/3/%7B92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a%7D/2.49a1/SeaMonkey/20090105024647/blocklist-sync/en-US/nightly/blocklist-sync/default/default/ [following]
--2016-11-05 03:02:09--  https://blocklists.settings.services.mozilla.com/v1/blocklist/3/%7B92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a%7D/2.49a1/SeaMonkey/20090105024647/blocklist-sync/en-US/nightly/blocklist-sync/default/default/
Resolving blocklists.settings.services.mozilla.com... 35.161.11.107, 54.68.119.170, 52.35.43.119
Connecting to blocklists.settings.services.mozilla.com|35.161.11.107|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 208123 (203K) [application/xml]
Saving to: ‘blocklist_amo.xml’

     0K .......... .......... .......... .......... .......... 24%  715K 0s
    50K .......... .......... .......... .......... .......... 49% 2.07M 0s
   100K .......... .......... .......... .......... .......... 73% 2.11M 0s
   150K .......... .......... .......... .......... .......... 98% 50.6M 0s
   200K ...                                                   100% 6190G=0.1s

2016-11-05 03:02:09 (1.69 MB/s) - ‘blocklist_amo.xml’ saved [208123/208123]

--2016-11-05 03:02:09--  http://hg.mozilla.org/comm-central/raw-file/default/suite/app/blocklist.xml
Resolving hg.mozilla.org... 63.245.215.25, 63.245.215.102
Connecting to hg.mozilla.org|63.245.215.25|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 230629 (225K) [text/xml]
Saving to: ‘blocklist_hg.xml’

     0K .......... .......... .......... .......... .......... 22% 37.4M 0s
    50K .......... .......... .......... .......... .......... 44% 68.9M 0s
   100K .......... .......... .......... .......... .......... 66% 85.8M 0s
   150K .......... .......... .......... .......... .......... 88% 68.4M 0s
   200K .......... .......... .....                           100%  123M=0.003s

2016-11-05 03:02:09 (62.9 MB/s) - ‘blocklist_hg.xml’ saved [230629/230629]

AMO blocklist does not appear to be an XML file. wget error?
program finished with exit code 1

Comment 1

[Edmund Wong (:ewong)]

Attachment: proposed patch

apparently, three weeks or so ago, the xml header for the amo blocklist includes 
the encoding, which our script doesn't check for.

Comment 2

[Justin Wood (:Callek)]

Comment on attachment 8808492 [details] [diff] [review]
proposed patch

Review of attachment 8808492 [details] [diff] [review]:
-----------------------------------------------------------------

I should have thought about this for SeaMonkey when I saw coop fix it, we can probably just take the updates from:

https://bugzilla.mozilla.org/show_bug.cgi?id=1312518  and be fixed.

Comment 3

[Edmund Wong (:ewong)]

Attachment: proposed patch (v2)

Comment 4

[Edmund Wong (:ewong)]

Attachment: [tools] prev. patch + changed tab to space in line 28

Comment 6

[Frank-Rainer Grahl (:frg)]

Comment on attachment 8822102 [details] [diff] [review]
[tools] prev. patch + changed tab to space in line 28

Stealing review as per irc.

I assume this goes into https://hg.mozilla.org/users/Callek_gmail.com/tools/ and is SeaMonkey only.

Should the 
  HGREPO="http://${HGHOST}/${BRANCH}" 
not be changed to 
  HGREPO="https://${HGHOST}/${BRANCH}"
?

r+ with question answered.

Comment 7

[Edmund Wong (:ewong)]

(In reply to Frank-Rainer Grahl from comment #6)
> Comment on attachment 8822102 [details] [diff] [review]
> [tools] prev. patch + changed tab to space in line 28
> 
> Stealing review as per irc.
> 
> I assume this goes into https://hg.mozilla.org/users/Callek_gmail.com/tools/
> and is SeaMonkey only.

Yes, it's SeaMonkey only; but I'm hoping to migrate the stuff from
Callek's repo to https://hg.mozilla.org/SeaMonkey/tools.   I have
merged some recent changes from Callek's tools repo to that one.
I just haven't had the time to file the necessary bug and migrate
since it *will* bitrot whatever else that's in the pipeline for
us.

> 
> Should the 
>   HGREPO="http://${HGHOST}/${BRANCH}" 
> not be changed to 
>   HGREPO="https://${HGHOST}/${BRANCH}"
> ?
> 
> r+ with question answered.

That works too.  It's a good start to bug 1305911. :)

Comment 8

[Edmund Wong (:ewong)]

Pushed to tools:
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/ba64841508e7

Comment 9

[Frank-Rainer Grahl (:frg)]

c-c and c-a got a blocklist update so I would consider it fixed.

https://hg.mozilla.org/comm-central/rev/24a3ded7409d
https://hg.mozilla.org/releases/comm-aurora/rev/08cab745d354