401474 - Need a software update appcast (or generation script)

Status: RESOLVED FIXED

(Camino Graveyard :: Product Site, defect)

Description

[Stuart Morgan]

We need in the short term a placeholder appcast with a single release (1.5.3, probably--just something to make it happy, since sparkle doesn't like empty appcasts), and in the longer term a script that can dynamically build an appcast appropriate to the configuration the user is running. I'll provide both.

Comment 1

[Stuart Morgan]

Attachment: placeholder

Here's the placeholder with 1.5.3. This needs to live at
http://caminobrowser.org/update-check

Comment 2

[Samuel Sidler (old account; do not CC)]

cb.o/update-check or cbo.o/update-check/ or does it matter? I'd prefer the latter.

Comment 3

[Samuel Sidler (old account; do not CC)]

Oh, I also thought we were using /update/, but that works too.

Comment 4

[Stuart Morgan]

The current patch uses:
https://caminobrowser.org/update-check
Nothing has landed though, so if you want something else just comment in bug 185436 and I'll change it on checkin.

It shouldn't have a trailing slash though, as it will be a script, fetched as:
https://caminobrowser.org/update-check?abunchofargumentshere

Comment 5

[Samuel Sidler (old account; do not CC)]

I'd prefer the trailing slash anyway. You can put arguments after it the trailing slash. (e.g. cb.o/update-check/?abunchofargumentshere) This allows us to have other scripts underneath it if needed instead of having them all at the top level.

/update-check/ is fine with me.

Comment 6

[Stuart Morgan]

(In reply to comment #5)
> This allows us to have other scripts underneath it if needed instead of having
> them all at the top level.

I don't follow... how does having a slash at the end of the path portion of the request change anything about how we later choose to subdivide the URL namespace?

Comment 7

[Samuel Sidler (old account; do not CC)]

Comment on attachment 289455 [details]
placeholder

This is live at https://caminobrowser.org/update-check

Comment 8

[Stuart Morgan]

I'm an idiot; the url for the 'enclosure' element should be
http://download.mozilla.org/?product=camino-1.5.3&os=osx&lang=en-US
for us to test with (not the page that starts the download, which is what I put).

Comment 9

[Samuel Sidler (old account; do not CC)]

Fixed.

Comment 10

[Stuart Morgan]

Attachment: 1.6a1 placeholder

This is a new placeholder that will update the people with version of 1.6a1pre where the update system worked to 1.6a1. I stole the blurb from part of the release notes on the wiki and reworded the beginning slightly; if anyone wants to re-work this feel free, just be really careful about escaping any html.

Comment 11

[Samuel Sidler (old account; do not CC)]

Comment on attachment 292497 [details]
1.6a1 placeholder

><p>For details, see <a href="http://caminobrowser.org/releases/1.6a1/">the release notes</a></p>

Will be changed to:

><p>For details, see <a href="http://preview.caminobrowser.org/">our preview site</a>.</p>

Note the URL/text change, as well as the addition of a period.

This will go live tomorrow.

Comment 13

[Markus Amalthea Magnuson]

So, how will the English build and the multilingual build specify different update feeds? By modifying the app.update.url pref? Or sending different queries to a feed generation script? This is a bit unclear at the moment.

Comment 14

[Stuart Morgan]

There is a parameter in the request that indicates whether the user is running the ML version (as detected by a non-zero-length general.useragent.extra.multilang pref). The ML version shouldn't need any further changes to have the right thing happen.

You can check to be sure that it is working by launching an ML build and running
defaults read org.mozilla.camino | grep SUFeedURL
The last argument should be "&intl=1"

Comment 15

[Markus Amalthea Magnuson]

Couldn't test this since the pref "general.useragent.extra.multilang" is added by the ML build script (buildint.sh) and all I have are localized versions of nightly builds. Guess it works though.

Comment 16

[Stuart Morgan]

+'ing for 1.6b3, since we don't want to auto-break anyone crazy enough to be running 10.3.[0-8].

Comment 17

[Stuart Morgan]

This will need to support the new lang= param as well, to allow localizable release notes.

Comment 18

[Stuart Morgan]

Attachment: full script setup

Here's the script, with the support files to serve the same 1.6b2 appcast we currently serve. Here's how it works:

There is a folder of update definitions, each one a file named foo.update (where foo can be anything that helps us keep track of what is what) which has a series of key = value lines describing things like the min OS version, the download URL, the signature, etc.--everything but the description (the Sparkle-displayed release notes). The keys are:
- Version: the CFBundleVersion
- VersionString: the human-readable version
- MinOSVersion: the min OS version required to run it (For something like "Tiger+", either 10.4 or 10.4.0 is fine)
- Arch: a list of supported architectures (i.e., "ppc,i386", or just "i386" if we someday drop ppc)
- Intl: whether or not this update is international ("1") or not ("0"), the idea being that we would have two .update files for a given release version, one with 0 and the English-only download, and one with 1 and the Multilingual download. The script supports the value being "0,1", which means that if we merge them into one version at some point, we can upgrade everyone without having to continue to make two files.
- FileURL: the download URL (not escaped for XML; the script will handle escaping &)
- FileSignature: the Sparkle signature for the download
- FileSize: the size in bytes of the download
- Date: the release date (this doesn't have to match anything, it's just for display)

There is another folder of descriptions, which contains files of the form humanversion-lang (e.g., 1.6b3-en). These files just contain the mini release notes as HTML (no extra XML escaping required). When building an appcast, it takes the language sent in the request URL, and uses the notes that best match:
1) Exact match (1.6b3-fr, and they sent fr as the language), or
2) Locale match (1.6b3-fr, and they sent fr_CA as the language), or
3) English
I don't do full accept-language negotiation, as I think that's overkill here.


So the use is that whenever we have a new update, we add a definition for it (or two, if it's a release version, with a multilingual build), and then description files in one or more languages. The naming of the description files is designed so that there's no need to have two sets of descriptions to support the two build flavors (English and multilingual). The script assumes the folders live next to where-ever the script is on the server, but that could be changed.

We'll want to clean out older builds over time, but one important note is that since Sparkle currently doesn't like empty appcast (we'll presumably ship a version of Sparkle with that fixed at some point, but this will apply until so few people are still using an older version than that that we don't mind breaking their update checks), we'll always need to leave the files for the latest version that meets any possible set of requirements (e.g., 1.6b2 will need to remain indefinitely, as the last build that supports 10.3.[0-8]), so that anyone doing a check will have a match. That's not a big deal though, as we'd probably want to do that anyway to make sure that people who run updates manually, and do so rarely, get the latest version they could run.

Comment 19

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to comment #18)

> - Arch: a list of supported architectures (i.e., "ppc,i386", or just "i386" 

This is now x86, since that's what the app end emits.

We have an updated version of this package running at https://caminobrowser.org/cgi-bin/update-check-test and serving 1.6b2.  You can test with your 1.6a1 or b1 by setting your app.update.url.override user_pref to that url.

We hope to get the final url (and the final test url) back to cbo/update*, but we'll have to see if we can have scripts live outside cgi-bin/.

I'm working on getting an update set for "1.6b3" (so we can test against nightlies that have the post-b2 Sparkle fixes) that'll point to a recent nightly build or something, though at the moment I still need the file signature.

> - Intl: whether or not this update is international ("1") or not ("0"), the
> idea being that we would have two .update files for a given release version,
> one with 0 and the English-only download, and one with 1 and the Multilingual
> download.

Stuart, what sort of filename do we give the ML version of the definition file?  I wasn't able to glean that from my perusal of the script, but my perl is worse than my non-existent python (I've at least copy/pasted chunks of python before!) ;)

Markus, if you want to attach some Swedish versions of the description files, I'll post those for your testing.  Ideally we'll think of some round-about way to test you actually getting a ML download, at least before 1.6 final.

Comment 20

[Stuart Morgan]

Definition files can be named anything (the script just reads all of them), so whatever makes sense to you.

Comment 21

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: diff between package and production

Just for reference/future/etc., this is a diff between what was in "full script setup" attachment and what we have running on the server as update-check-test.

Comment 22

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

This seems to be working well, so FIXED.  Thanks Stuart!

Comment 23

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Attachment: diff between March 23 production version and current production version

The script as it was today (last modified April 17 2008, according to the server) was eventually checked into cvs at camino/tools/autoupdate/update-check

Attached is the diff between the script as of March 23 (diff in attachment 311321 [details] [diff] [review]) and what landed in cvs (aka the April 17 revision).