balrog serves partial when it shouldn't if fromRelease doesn't exist

RESOLVED FIXED

Status

Release Engineering
Balrog: Backend
P3
normal
RESOLVED FIXED
3 years ago
11 months ago

People

(Reporter: bhearsum, Assigned: nthomas, Mentored)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [lang=python])

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
I noticed this while looking at something with Massimo on aus4-dev (where our data is much more sparse). This block is the problem:
if patch["from"] != "*" and fromRelease and not fromRelease.matchesUpdateQuery(updateQuery):
            return None


In cases where patch["from"] is not *, and fromRelease _doesn't_ exist, an update will be returned. In practical terms, this means that if a rule points to a release, and that release has a partial that references a release that doesn't exist, a partial update will be returned anyways.

We're unlikely to hit this in production because we never delete release blobs, and nightly blobs are kept around for 90 days. The only time I would expect to see this is if a branch doesn't have any new updates for over 90 days.
(Assignee)

Comment 1

3 years ago
I've seen this too. IIRC it's not a problem for schema 2, but is for 3 (and presumably 4).

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1923]
(Reporter)

Comment 2

3 years ago
The changes from bug 671488 might have fixed this, haven't verified that though.
Sounds like we are going to hit this a lot in staging, where we proxy previous release files, but don't have them in the Balrog DB.
(Reporter)

Updated

2 years ago
Mentor: bhearsum@mozilla.com
(Reporter)

Comment 4

2 years ago
This is still an issue. It can be reproduced locally with the latest sample data in the Docker with URLs such as http://localhost:9090/update/3/Firefox/32.0/20121005155445/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/default/default/default/update.xml?force=1
Mentor: bhearsum@mozilla.com

Comment 5

2 years ago
(In reply to Ben Hearsum (:bhearsum) from comment #4)

Hi Ben,

I would like to work on this issue, could you assign it to me please? thank you : )
(Reporter)

Comment 6

2 years ago
Here you go, thanks!
Assignee: nobody → sabergeass

Updated

2 years ago
Assignee: sabergeass → nobody

Comment 7

2 years ago
Hey Ben, Sorry for the late reply. I would like to unassign this bug because I'm busy with the school things. :(
(Reporter)

Comment 8

2 years ago
(In reply to MikeLing from comment #7)
> Hey Ben, Sorry for the late reply. I would like to unassign this bug because
> I'm busy with the school things. :(

No worries. If you have time in the future feel free to come back to it!
(Assignee)

Comment 9

2 years ago
The code was rearranged quite a bit since this was originally filed. I think the new location of interest is at
  https://github.com/mozilla/balrog/blob/master/auslib/blobs/apprelease.py#L86
and not returning none at line 89. (Line references for rev 937acd5)
(Assignee)

Comment 10

2 years ago
Created attachment 8737657 [details] [review]
PR to handle missing release
Assignee: nobody → nthomas
Status: NEW → ASSIGNED
Attachment #8737657 - Flags: review?(bhearsum)
(Reporter)

Comment 11

2 years ago
Comment on attachment 8737657 [details] [review]
PR to handle missing release

r=me with a direct test for the blob class added.
Attachment #8737657 - Flags: review?(bhearsum) → review+

Comment 12

a year ago
Commit pushed to master at https://github.com/mozilla/balrog

https://github.com/mozilla/balrog/commit/2a49ba36c1714c4d448b04eca15ec5400716e8c9
Bug 1074362 - balrog serves partial when it shouldn't when fromRelease doesn't exist (#68) r=bhearsum

* Adjust tests to catch case where release is missing from db

* Handle the second case where _getFromRelease() returns None
(Reporter)

Updated

a year ago
Depends on: 1297407
(Reporter)

Comment 13

a year ago
Looks like this is working in stage, which recently reimported the sample data. https://aus4.stage.mozaws.net/update/3/Firefox/32.0/20121005155445/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/default/default/default/update.xml?force=1 maps to Firefox-43.0.2-build1, which contains a few partials (none of which exist in the db). On my local container running 2.5, I get:
<updates>
  <update type="minor" displayVersion="43.0.2" appVersion="43.0.2" platformVersion="43.0.2" buildID="20151221130713" detailsURL="https://www.mozilla.org/en-US/firefox/43.0.2/releasenotes/">
    <patch type="complete" URL="http://download.mozilla.org/?product=firefox-43.0.2-complete&os=osx&lang=en-US&force=1" hashFunction="sha512" hashValue="781478556846b719ebc906a8a9613a421e24449b4456c4ccee990e878b3be9fb0478a78821a499a4c1f1a76d75078acf3fdfa3d0be69d2f6c94e3b6340fc935b" size="80329415"/>
    <patch type="partial" URL="http://download.mozilla.org/?product=firefox-43.0.2-partial-41.0.2&os=osx&lang=en-US&force=1" hashFunction="sha512" hashValue="6edd0803e36a03117e12a36e9fc8941e8f6321071fb00c7e8489f67b332d1cbfa95d00218e5c1b61115752fc0aecde8b2535424c521d45530455a4c5d571f889" size="39520883"/>
  </update>
</updates>

On stage, I get:
<updates>
  <update type="minor" displayVersion="43.0.2" appVersion="43.0.2" platformVersion="43.0.2" buildID="20151221130713" detailsURL="https://www.mozilla.org/en-US/firefox/43.0.2/releasenotes/">
    <patch type="complete" URL="http://download.mozilla.org/?product=firefox-43.0.2-complete&os=osx&lang=en-US&force=1" hashFunction="sha512" hashValue="781478556846b719ebc906a8a9613a421e24449b4456c4ccee990e878b3be9fb0478a78821a499a4c1f1a76d75078acf3fdfa3d0be69d2f6c94e3b6340fc935b" size="80329415"/>
  </update>
</updates>
(Reporter)

Comment 14

a year ago
Looking good in prod, too. I found an old nightly watershed to verify against. https://aus5.mozilla.org/update/3/Firefox/41.0a2/20141209095500/WINNT_x86-msvc/en-US/nightly/Windows_NT%2015.3.0/default/default/update.xml?force=1 was getting a partial prior to the push, and is now getting just a complete.
(Reporter)

Comment 15

11 months ago
Based on recent comments, and a test I just did now, this looks fixed to me.
Status: ASSIGNED → RESOLVED
Last Resolved: 11 months ago
Priority: -- → P3
Resolution: --- → FIXED
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1923] → [lang=python]
You need to log in before you can comment on or make changes to this bug.