1168780 - Integrity error for partial update from 38.0.5 build3 to build4

Status: RESOLVED FIXED

(Release Engineering :: Release Requests, defect)

Description

[Nick Thomas [:nthomas] (UTC+12)]

I had 38.0.5 build3 and was offered build4, but got an integrity error for the partial update. This was actually a misrepresented 404. The failover to the complete update worked as expected, so I got there in the end.

Query:
https://aus4.mozilla.org/update/3/Firefox/38.0.5/20150521175336/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/beta/Darwin%2014.3.0/default/default/update.xml

Response:
<?xml version="1.0"?>
<updates>
    <update type="minor" displayVersion="38.0.5" appVersion="38.0.5" platformVersion="38.0.5" buildID="20150525141253" detailsURL="https://www.mozilla.org/en-US/firefox/38.0.5/releasenotes/">
        <patch type="complete" URL="http://download.mozilla.org/?product=firefox-38.0.5build4-complete&amp;os=osx&amp;lang=en-US" hashFunction="sha512" hashValue="771df147cad17a024e55cebb45d3ee218ce849c543721513af9835900ac02107adace3e5d28990f7d92045118b8a54df00f4acb9305c56a4e1311fddfb91915c" size="72851689"/>
        <patch type="partial" URL="http://download.mozilla.org/?product=firefox-38.0.5-partial-38.0.5&amp;os=osx&amp;lang=en-US" hashFunction="sha512" hashValue="561b1048b9774727c6777dac39fc5213179938702a1c0293190f08d2e8fdb26b6c2e164cc4ebb8d9a68e27521abbd56050fd266665a1aa0e82e445f48c6a4a3e" size="2499675"/>
    </update>
</updates>

But 
 http://download.mozilla.org/?product=firefox-38.0.5-partial-38.0.5&os=osx&lang=en-US
is a release style product, and bouncer returns a 404 because we haven't pushed to mirrors. The product should be firefox-38.0.5build4-partial-38.0.5build3.

The Firefox-38.0.5-build4 blob has this:
  "fileUrls": {
...
    "beta": {
      "partials": {
        "Firefox-38.0-build3": "http://download.mozilla.org/?product=firefox-38.0.5build4-partial-38.0build3&os=%OS_BOUNCER%&lang=%LOCALE%"
      },
      "completes": {
        "*": "http://download.mozilla.org/?product=firefox-38.0.5build4-complete&os=%OS_BOUNCER%&lang=%LOCALE%"
      }
    },
    "*": {
      "partials": {
        "Firefox-37.0.2-build1": "http://download.mozilla.org/?product=firefox-38.0.5-partial-37.0.2&os=%OS_BOUNCER%&lang=%LOCALE%",
        "Firefox-36.0.4-build1": "http://download.mozilla.org/?product=firefox-38.0.5-partial-36.0.4&os=%OS_BOUNCER%&lang=%LOCALE%",
        "Firefox-38.0.5-build3": "http://download.mozilla.org/?product=firefox-38.0.5-partial-38.0.5&os=%OS_BOUNCER%&lang=%LOCALE%",
        "Firefox-38.0-build3": "http://download.mozilla.org/?product=firefox-38.0.5-partial-38.0&os=%OS_BOUNCER%&lang=%LOCALE%",
        "Firefox-38.0.1-build1": "http://download.mozilla.org/?product=firefox-38.0.5-partial-38.0.1&os=%OS_BOUNCER%&lang=%LOCALE%"
      },
      "completes": {
        "*": "http://download.mozilla.org/?product=firefox-38.0.5-complete&os=%OS_BOUNCER%&lang=%LOCALE%"
      }
    },

So the issue is that 38.0.5 build3 isn't in the list of partials for beta, and Balrog falls back to the * block. This is a consequence of only specifying 38.0 at http://hg.mozilla.org/build/buildbot-configs/rev/49bf3a08f570#l1.20 (yay me). Presumably this wasn't caught by update verify because the same regex is applied when bumping the patcher config. Not sure why QE tests missed it.

So
* we should probably fix that up by extending the fileUrls block
* we should consider if we shouldn't fallback if the channel is a key in fileUrls

Comment 1

[Rail Aliiev [:rail]]

Probably we need to remove Firefox-38.0-build3 from the partials section - we don't usually build partials for previous buildN builds.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

I've added Firefox-38.0.5-build3 to the fileUrls for beta-localtest, beta-cdntest, and beta channels, so that we get people onto build4 asap. Verified on mac with the be locale.

Comment 3

[Rail Aliiev [:rail]]

Any actions needed here?

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Nick Thomas [:nthomas] from comment #0)
> So
> * we should probably fix that up by extending the fileUrls block
> * we should consider if we shouldn't fallback if the channel is a key in
> fileUrls

Ben, what do you think of the second point ?  It seems weird to me that if something is missing in the beta partials block then we go and look in the * block, which otherwise is for releases.

Comment 5

[[e-mail] (:bhearsum)]

(In reply to Nick Thomas [:nthomas] from comment #4)
> (In reply to Nick Thomas [:nthomas] from comment #0)
> > So
> > * we should probably fix that up by extending the fileUrls block
> > * we should consider if we shouldn't fallback if the channel is a key in
> > fileUrls
> 
> Ben, what do you think of the second point ?  It seems weird to me that if
> something is missing in the beta partials block then we go and look in the *
> block, which otherwise is for releases.

I totally agree with you, falling back to * when the channel is _explicitly_ specified is wrong.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

The 38.0.5 case is fixed. Split the general fix out to bug 1170919.