399865 - prepare files to push on aus2.mozilla.org for sunbird 0.5 -> 0.7 update

Status: VERIFIED FIXED

(Calendar :: Build Config, defect)

Description

[ause]

Attachment: first draft of config file for patcher2 - filedates and urls not yet final

attach files to be reviewed or pushed to AUS2 here

Comment 1

[ause]

Attachment: set of AUS2 update snippets 0.5->0.7 en-US only

created with the attached configuration

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 284941 [details]
first draft of config file for patcher2 - filedates and urls not yet final

><app>
>	<Sunbird>
>		<current-update>
>			from 0.5
>			to 0.7
>			channel release
>			testchannel betatest releasetest
>			details http://www.mozilla.org/projects/calendar/releases/sunbird0.7.html
>			<partial>
>				path calendar/sunbird/release/0.7/updates/sunbird-0.5-0.7.%locale%.%platform%.partial.mar

betatest-url needs to point to a non-release location on the ftp, otherwise you can't test updates before you ship. That location is set by the path declaration. The analogue of what we'd use is calendar/sunbird/nightly/0.7-candidates/, where you'd also rsync the en-US and localized builds so they don't get lost in the YYYYMMDDHH-$branch dirs.

(Aside: technically, you could push testing snippets out into the releases dir, but you don't need them on the mirrors for bandwidth, and if you have to change them later it's a serious pain, eg how do you know if you pulled the update from a fresh mirror rather than a stale one ?)

>				url http://download.mozilla.org/?product=sunbird-0.7-partial-0.&os=%bouncer-platform%&lang=%locale%

Looks good.

>				betatest-url http://stage.mozilla.org/pub/calendar/sunbird/release/0.7/updates/sunbird-0.5-0.7.%locale%.%platform%.partial.mar

* needs to match path (see above)
* missing mozilla.org after pub/, ie ../pub/mozilla.org/calendar/...

>			</partial>
>			<complete>
>				path calendar/sunbird/release/0.7/updates/sunbird-0.7.%locale%.%platform%.complete.mar
>				url http://download.mozilla.org/?product=sunbird-0.7-complete&os=%bouncer-platform%&lang=%locale%
>				betatest-url http://stage.mozilla.org/pub/calendar/sunbird/release/0.7/updates/sunbird-0.7.%locale%.%platform%.complete.mar

Same comments as for the partial block.

>			</complete>
>		</current-update>
>
>		<release>
>			<0.7>
>				version 0.7
>				extension-version 0.7
>				completemarurl http://stage.mozilla.org/pub/mozilla.org/calendar/sunbird/nightly/latest-mozilla1.8/sunbird-0.7.%locale%.%platform%.complete.mar

The files here will get overwritten, see comments above about a candidates dir. If you set "$ReleaseToDated = 1" in the tinder-config, then you'll get a proper YYYY-MM-DD-HH-mozilla1.8-l10n/ dir, but don't forget to set it back to 0 after the run though (or it'll start chewing up lots of disk; we actually use separate configs for releases).

>				schema 1
>				<platforms>
>					linux-i686 2007
>					mac 2007
>					win32 2007

Don't forget to fix these, once you have the en-US builds :-)

>				</platforms>
>				locales en-US
>				<exceptions>
>				</exceptions>
>			</0.7>
>			<0.5>
>				version 0.5
>				extension-version 0.5
>				completemarurl http://stage.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.5/updates/%platform%/%locale%/sunbird-0.5.%locale%.%platform%.complete.mar
>				schema 1
>				<platforms>
>					linux-i686 2007061403
>					mac 2007061404
>					win32 2007061404
>				</platforms>
>				locales en-US de
>				<exceptions>
>                    de   linux-i686, win32, mac
>				</exceptions>

This is an experiment ? For the same effect, just leave de off the locale list and have an empty exceptions block.

The snippets look fine, given this config. While follow up with info on how to force complete updates for locales.

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #2)
> betatest-url needs to point to a non-release location on the ftp, otherwise you
> can't test updates before you ship. That location is set by the path
> declaration. The analogue of what we'd use is
> calendar/sunbird/nightly/0.7-candidates/, where you'd also rsync the en-US and
> localized builds so they don't get lost in the YYYYMMDDHH-$branch dirs.

calendar/sunbird/nightly/0.7-candidates/rc2/ would be even better.
 
> The snippets look fine, given this config. While follow up with info on how to
> force complete updates for locales.
 
s/While/Will/

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

I can help out with advice, but I'm not on the hook for this work.

Comment 6

[Stefan Sitter [:ssitter]]

I just did a first test using the instruction from <http://wiki.mozilla.org/Software_Update:Manually_Installing_a_MAR_file> and the *.partial.mar files from <http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/nightly/0.7-candidates/rc2/>.

For win32 Sunbird 0.5 en-US build it worked fine. 

For win32 Sunbird 0.5 de build it failed with status 8 (whatever this means). The *.complete.mar worked fine.

Comment 7

[Omar Bajraszewski]

(In reply to comment #6)
> For win32 Sunbird 0.5 de build it failed with status 8 (whatever this means).

The same with pl build

Comment 8

[Daniel Boelzle [:dbo]]

Putting on blocker list to keep track of it. It would be really really nice if we could resolve it for 0.7 release.

Comment 9

[ause]

Attachment: latest of patcher2 config for sunbird - used to generate files for 0.7rc2

Comment 10

[ause]

Attachment: latest update snippets all locales except mn

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #10)
> Created an attachment (id=285362) [details]
> latest update snippets all locales except mn

Snippets for betatest and releasetest are now live, though only betatest will work at the moment. (It points to stage, while releasetest uses the not-yet configured bouncer).

To test, you'll need to change the channel of a 0.5 install from release to betatest. Don't forget that you can't do this in about:config, but should edit
  <sunbird install dir>/defaults/pref/channel-prefs.js
with a text editor (while Sunbird is closed).

If betatest works out well, then the v2 partials for win32 can get pushed into releases/0.7rc2/update/. It'll take a few hours for all the mirrors to catch up, but I'll configure bouncer so that it's ready to go.

Comment 12

[Nick Thomas [:nthomas] (UTC+12)]

Looks like BuildID's for 0.5 got swapped around, they need to be:

				<platforms>
					linux-i686 2007061404
					mac 2007061403
					win32 2007061404
				</platforms>

If you fix the config, and move aside the existing snippets, then you can call patcher with --create-patchinfo. Please attach new snippets here, and I'll publish them.

Comment 13

[ause]

Attachment: different (fixed?) buildids

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #13)
> Created an attachment (id=285373) [details]
> different (fixed?) buildids

betatest & releasetest snippets from this are live (the rest staged). Linux & Mac updates are available on betatest now. 

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

I did a diff between an installed & updated 0.5, and an installed 0.7, which gave:

> Only in 0.7/installed/chrome: chromelist.txt
This looks like an pre-processed file that hasn't been processed.

>Only in 0.7/installed/components: nsSetDefaultBrowser.js
>Only in 0.7/installed/components: nsSetDefaultMail.js
>Only in 0.7/installed/components: nsUnsetDefaultMail.js

The files are installed by 0.5 and then removed by the updater, because they are in 
 http://mxr.mozilla.org/mozilla1.8/source/calendar/installer/removed-files.in

They were added to that file in bug 356740, you might have a packaging problem with the installer if they are still present in it.

>Only in 0.7/installed/extensions: {e2fda1a4-762b-4020-b5ad-a41df1933103}

not installed by 0.5, and is a patch-if in the update manifest. New for 0.7 ?

>Only in 0.5/updated/: .autoreg
>Only in 0.5/updated/: active-update.xml
>Only in 0.5/updated/: removed-files
>Files 0.5/updated/uninstall/uninstall.log and 0.7/installed/uninstall/uninstall.log differ
>Only in 0.5/updated/: updates
>Only in 0.5/updated/: updates.xml

Harmless or expected differences.

Comment 16

[Stefan Sitter [:ssitter]]

> sunbird\chrome\chromelist.txt

Listed in removed-files.in but in a #ifdef XP_WIN block

> nsSetDefaultBrowser.js, nsSetDefaultMail.js, nsUnsetDefaultMail.js

Already filed as Bug 399809

> sunbird\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}\install.rdf

Yes, this are new files from Bug 379005 that must be added during update.

Comment 17

[ause]

got a problem when using releasetest channel:
---
Sunbird was unable to verifiy the integrity of the incremental update it downloaded, so it is now downloading the complete update package.
---
and nothing more happens...

Comment 18

[ause]

fetching the urls from the existing update.xml gives "file not found" for partial  and complete update file

Comment 19

[Omar Bajraszewski]

(In reply to comment #11)
> Snippets for betatest and releasetest are now live, though only betatest will
> work at the moment. (It points to stage, while releasetest uses the not-yet
> configured bouncer).

I switched to betatest channel and Sunbird 0.5 pl (windows) updated to 0.7 without any problems. Releasetest channel works too

Comment 20

[Stefan Sitter [:ssitter]]

I successfully performed the partial update on win32 for en-US and de locale. But the folder "sunbird\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}" mentioned in Comment #15 is still missing after update.

--> FAILED

Comment 21

[Omar Bajraszewski]

The folder ("sunbird\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}") wasn't created in pl version too

Comment 22

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #21)
> The folder ("sunbird\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}") wasn't
> created in pl version too

This will require a slight tweak to the update generation (partials, snippets). Will advise once I figure it out, but it'll be something like 
   force = "extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/install.rdf"
in the <current update> block.

Comment 23

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: remove the override for extensions/

This patch removes the special casing that doesn't apply for this release (it would if you added a new file to Talkback or other existing optional extension, then you'd only want it added if the extension was already installed).

As a result, you get this type of change in a (sorted) update.manifest:

--- rc2/unpack-en-US-win32/update.manifest.sort 2007-10-22 13:23:12.000000000 -0700
+++ mine/unpack-en-US-win32/update.manifest.sort        2007-10-22 13:23:21.000000000 -0700
@@ -1,10 +1,10 @@

 add ".autoreg"
+add "extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/install.rdf"
 add "freebl3.chk"
 add "js/calAlarmMonitor.js"
 add "js/calFreeBusyService.js"
 add "removed-files"
 add "softokn3.chk"
-add-if "extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}" "extensions/{e2fda1a4-762b-4020-b5ad-a41df1933103}/install.rdf"
 patch "AccessibleMarshal.dll.patch" "AccessibleMarshal.dll"
 patch "chrome/calendar-en-US.jar.patch" "chrome/calendar-en-US.jar"
 patch "chrome/calendar.jar.patch" "chrome/calendar.jar"

ie always add the new dir and install.rdf. The mac also adds a chrome.manifest in that dir, which the dmg doesn't have, hope that's alright.

NB: this is a once-off hack. I wouldn't recommend using it for the next release, or leaving files around in a way that you might accidentally do so.

Comment 24

[ause]

Attachment: update snippets for rc3 build

Comment 25

[ause]

using the patch the "add-if" changed to "add" as expected

Comment 26

[Omar Bajraszewski]

(In reply to comment #17)
> got a problem when using releasetest channel:
> ---
> Sunbird was unable to verifiy the integrity of the incremental update it
> downloaded, so it is now downloading the complete update package.
> ---
> and nothing more happens...
> 

I've got the same problem now

Comment 27

[Nick Thomas [:nthomas] (UTC+12)]

Guess ause didn't put the symlink in from 0.7/update/ from 0.7rc3/update/. I've done that now.

Comment 28

[Omar Bajraszewski]

Ok. I have successfully performed the partial update on win32 for pl locale using betatest and releasetest channels. The folder ("sunbird\extensions\{e2fda1a4-762b-4020-b5ad-a41df1933103}") was also successfully created.

Comment 29

[ause]

sunbird 0.7 is release and update works

Comment 30

[Omar Bajraszewski]

Verified per comment #29