Closed Bug 630526 Opened 10 years ago Closed 10 years ago

Port |Bug 426214 - Automatically update blocklist.xml in cvs/hg from blocklist service| to SeaMonkey

Categories

(SeaMonkey :: Release Engineering, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: sgautherie, Assigned: Callek)

References

Details

Attachments

(3 files)

I'm not very familiar with this feature, but I'm assuming that SeaMonkey blocklist should be (automatically) updated from time to times too, shouldn't it?

NB: And, in the meantime, manually.
taking so I don't forget about it.

Can you also possibly file a two bugs, one blocked by this..

"Update blocklist.xml for seamonkey" and "Ensure blocklist.xml is up to date when we ship SeaMonkey 2.1" the latter will be blocking, the former I want to do "asap". -- Thanks.
Assignee: nobody → bugspam.Callek
Ok, in another bug I noticed our buildbot automation has some code already to do this, though the code runs a build-tools script that is Firefox only. [http://hg.mozilla.org/build/tools/file/2059f9f3f47b/scripts/blocklist]

The solution is to copy the script for Comm-Central apps, and use it here instead.

CC-ing gozer and jhopkins incase one of them want to take on the script part.

We do have two options here, relating to this, #1, do we want each [c-c] app to be responsible for their own blocklist updates, or should we have the script do them all, and only one of us run the builder?

I would prefer each-app separately, mostly due to the option for each app to choose to cancel the process if something is going against plan, etc. but all-in-one might be easier from a script and automation side.
(In reply to comment #2)
> The solution is to copy the script for Comm-Central apps, and use it here
> instead.

Actually, I disagree here, it would be much better to ask Mozilla releng if they would agree with us adding parameters to the existing script that make that one works for us and fall back to the Firefox values when the new params don't exist. Actually, they might one day use the additional options for mobile as well.
This modifies only the tools script, being sure to keep it working in firefox if the -p param doesn't get added to the buildbotcustom instance before this is deployed.

I ran a dry-run of the script for both instances locally as follows (Note; I am not updating SeaMonkey's blocklist yet, as I want to be sure that once deployed this _all_ works fine on our slaves)

Justin@DELLXPS400-1 ~/Sources/Mozilla/tools/scripts/blocklist
$ sh sync-hg-blocklist.sh -d -b comm-central -u "Callek@gmail.com" -k "~/.ssh/id_dsa" -c -p "s
eamonkey" -n
--23:32:45--  http://hg.mozilla.org/comm-central/raw-file/default/suite/config/version.txt
           => `version.txt'
Resolving hg.mozilla.org... 63.245.208.189
Connecting to hg.mozilla.org|63.245.208.189|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 8 [text/plain]

100%[===================================================>] 8             --.--K/s

23:32:45 (378.22 KB/s) - `version.txt' saved [8/8]

--23:32:46--  https://addons.mozilla.org/blocklist/3/%7B92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a%7
/2.1b3/SeaMonkey/20090105024647/blocklist-sync/en-US/nightly/blocklist-sync/default/default/
           => `blocklist_amo.xml'
Resolving addons.mozilla.org... 63.245.217.40
Connecting to addons.mozilla.org|63.245.217.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2,834 (2.8K) [text/xml]

100%[===================================================>] 2,834         --.--K/s

23:32:47 (24.66 MB/s) - `blocklist_amo.xml' saved [2834/2834]

--23:32:47--  http://hg.mozilla.org/comm-central/raw-file/default/suite/app/blocklist.xml
           => `blocklist_hg.xml'
Resolving hg.mozilla.org... 63.245.208.189
Connecting to hg.mozilla.org|63.245.208.189|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 1,426 (1.4K) [text/xml]

100%[===================================================>] 1,426         --.--K/s

23:32:48 (42.28 MB/s) - `blocklist_hg.xml' saved [1426/1426]

Blocklist files differ, but not updating hg in dry-run mode.

Justin@DELLXPS400-1 ~/Sources/Mozilla/tools/scripts/blocklist
$ sh sync-hg-blocklist.sh -d -b mozilla-central -u "Callek@gmail.com" -k "~/.ssh/id_dsa" -c -p
 "firefox" -n
--23:33:30--  http://hg.mozilla.org/mozilla-central/raw-file/default/browser/config/version.tx

           => `version.txt'
Resolving hg.mozilla.org... 63.245.208.189
Connecting to hg.mozilla.org|63.245.208.189|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: 10 [text/plain]

100%[===================================================>] 10            --.--K/s

23:33:30 (421.30 KB/s) - `version.txt' saved [10/10]

--23:33:30--  https://addons.mozilla.org/blocklist/3/%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7
/4.0b12/Firefox/20090105024647/blocklist-sync/en-US/nightly/blocklist-sync/default/default/
           => `blocklist_amo.xml'
Resolving addons.mozilla.org... 63.245.209.91
Connecting to addons.mozilla.org|63.245.209.91|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4,872 (4.8K) [text/xml]
Attachment #513023 - Flags: review?(nrthomas)
Comment on attachment 513023 [details] [diff] [review]
update sync-hg-blocklist.sh

>-HOST=`/bin/hostname -s`
>+#HOST=`/bin/hostname -s`
>+HOST="\"Callek's home computer\""

err this was debugging cruft from windows testing; removed locally.
Comment on attachment 513023 [details] [diff] [review]
update sync-hg-blocklist.sh

>+case "$PRODUCT" in
>+    firefox)
>+        APP_DIR="browser";
>+        APP_ID="%7Bec8030f7-c20a-464f-9b0e-13a3a9e97384%7D";
>+        APP_NAME="Firefox";;
>+    seamonkey)
>+        APP_DIR="suite";
>+        APP_ID="%7B92650c4d-4b8e-4d2a-b7eb-24ecf4f6b63a%7D";
>+        APP_NAME="SeaMonkey";;
>+    *)
>+        echo >&2 "Invalid Product was passed to $0. Passed value was: $PRODUCT";
>+        echo >&2 $USAGE;
>+        exit 1;;
>+esac

Why not just pass all of that data in via arguments? Then it is very configurable for any product that comes along.
(In reply to comment #1)
> Can you also possibly file a two bugs, one blocked by this..
> 
> "Update blocklist.xml for seamonkey" and "Ensure blocklist.xml is up to date
> when we ship SeaMonkey 2.1" the latter will be blocking, the former I want to
> do "asap". -- Thanks.

Let the former be this very bug, per your comment 4.
Wrt the latter, let's reopen bug 525465 if the current bug is not fixed before release time.

(In reply to comment #4)
> I am
> not updating SeaMonkey's blocklist yet, as I want to be sure that once deployed
> this _all_ works fine on our slaves)
blocking-seamonkey2.1: --- → ?
Comment on attachment 513023 [details] [diff] [review]
update sync-hg-blocklist.sh

>-HOST=`/bin/hostname -s`
>+#HOST=`/bin/hostname -s`
>+HOST="\"Callek's home computer\""

Looks fine except for this hunk, r+ with that left out.
Attachment #513023 - Flags: review?(nrthomas) → review+
This is the patch to support this change in misc.py.

This must only be deployed after the script change, otherwise the [current] script will error out on not knowing what |-p| is.

Alternatively I can do this addition without the |if| if desired.
Attachment #513357 - Flags: review?(nrthomas)
KaiRo, I if you approve this, I'll also take it as rs+=you to enable blocklist updates in our master config [for trunk]
Attachment #513358 - Flags: review?(kairo)
Comment on attachment 513358 [details] [diff] [review]
misc.py uncomment out blocklist builder (from generateCCBranchObjects)

Looks good to me.

We possibly should take another look on our blocklist and if something from the Firefox blocklist should also be applied to our side, but that's a different bug.
Attachment #513358 - Flags: review?(kairo) → review+
Comment on attachment 513357 [details] [diff] [review]
misc.py changes to support new argument

Looks OK to me.

Thunderbird will have issues with
 http://hg.mozilla.org/build/buildbot-configs/file/default/thunderbird/config.py#l390
if they want to use this stuff too.
Attachment #513357 - Flags: review?(nrthomas) → review+
(In reply to comment #13)
> Comment on attachment 513357 [details] [diff] [review]
> misc.py changes to support new argument
> Looks OK to me.
> Thunderbird will have issues with

The switch in the script above can easily use Thunderbird here. So I don't anticipate this being a problem yet.
Landed:
https://hg.mozilla.org/build/buildbotcustom/rev/56e21ad2fb5d (main misc.py patch)
https://hg.mozilla.org/build/buildbotcustom/rev/1c993575142b (CC misc.py patch, + comment removal)
https://hg.mozilla.org/build/buildbot-configs/rev/39927ec27bcc (enable for trunk)

I only enabled for trunk since I forgot comm-1.9.1 uses a version-1.9.1.txt file instead of the normal version file (due to the way we used one repo to support two mozilla-* repos) I'll attack the 1.9.1 side in another bug.

Triggered a build, which resulted in Green but failed since I forgot one small piece in the script.

http://hg.mozilla.org/build/tools/rev/6f0cdf2ce441 (got r+=dustin over IRC)

Retriggered the manual update and came out a success:
http://hg.mozilla.org/comm-central/rev/f655ed11a31b

Of note is that the hostname call seems to have resulted in an empty string on our box, will look into that separately as well.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
blocking-seamonkey2.1: ? → ---
No longer blocks: FF2SM
You need to log in before you can comment on or make changes to this bug.