1217828 - Wrong version in crash details for Firefox Beta

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[u421692]

Tested on Firefox 42 Beta 8 and Beta 9 for Android
Device: Samsung Galaxy S5(Android 4.4.2)

1. Crash Firefox
2. Go to about:crashes and open the crash report
3. Check the version of Firefox in Details

Expected result:
The correct version should be displayed

Actual result:
Version displayed is 42.0b0 instead of 42.0b8
May affect other versions or platforms

Comment 1

[Robert Kaiser]

The issue there is that we are missing the FennecAndroid 42.0b8 version and the 42.0b9 for both Firefox and FennecAndroid. This is probably an ftpscraper fail because of the move from FTP to S3 for builds.

Peter, I'm not sure who in the team need to take this, but this is urgent and actually is a blocker for our crash analysis activities.

Comment 2

[Peter Bengtsson [:peterbe]]

Nick or Rail,
Do you know anything about this? Are there not FennecAndroid 42.0b8 and FennecAndroid 42.0b9 on the FTP server? 

I noticed that there was no release of 42.0b8 (or 42.0b7) but there was one for 42.0b9
https://www.dropbox.com/s/nvp4huk9vt370pk/Screenshot%202015-10-23%2008.46.24.png?dl=0 

Basically, I'm trying to understand if we're failing to scrape or because the releases are not there at all.

Comment 3

[Rail Aliiev [:rail]]

I think, this is related to the FTP->S3 migration. Not all files are synced yet.

Jeremy, can you make sure that files under mobile/releases/42* are in sync?

Comment 4

[Robert Kaiser]

What URL is the Socorro code actually scraping right now? We probably should make sure we do archive.m.o though I heard from a different scraping service that the HTML might have changed there.

Comment 5

[Robert Kaiser]

On IRC, peterbe pointed me to https://github.com/mozilla/socorro/blob/master/socorro/cron/jobs/ftpscraper.py#L329 which has 'http://ftp.mozilla.org/pub/mozilla.org' - we should switch to 'https://archive.mozilla.org/pub/' instead in the future, but the other seems to give results.

That said, 42.0b9 is listed there for both Firefox and FennecAndroid (mobile/) but we also have not added it to Socorro yet, which will become a problem for our analysis within hours from now.

Comment 6

[Jeremy Orem [:oremj]]

syncing 42.0b8 now.

Comment 7

[Robert Kaiser]

Just some notes from IRC, so they are archived here:

<KaiRo> peterbe: releng is or has been moving from in-house to S3
<KaiRo> peterbe: see https://groups.google.com/forum/#!topic/mozilla.dev.platform/FnAn_4a7Ohc
<KaiRo> peterbe: and for added fun, the HTTP listing format changed, but only for the directories that were moved (like firefox and mobile) but not the ones that didn't (like seamonkey) and Socorro needs to support all those directories
<KaiRo> compare https://archive.mozilla.org/pub/firefox/releases/ and https://archive.mozilla.org/pub/mozilla.org/seamonkey/releases/
<KaiRo> my guess is that the listing format change is causing issues for us
<KaiRo> as it did for others, like mozdownload
<KaiRo> peterbe: FWIW, the changes to mozdownload to parse the new listing format are at https://github.com/mozilla/mozdownload/commit/ea76769ef8cb62091ea85bf76d79df8098b0e635
<KaiRo> peterbe: in case that helps
<KaiRo> peterbe: note that missing the / at the end of a directory now breaks the listings completely (just in case we might do that, whimboo pointed me to that)

Comment 8

[Jeremy Orem [:oremj]]

The sync is complete.

Comment 9

[Peter Bengtsson [:peterbe]]

I'm changing from http://ftp.m.o to https://archive.m.o and investigating how our ftpscraping is coping with the new HTML output. 

Also, Kairo pointed me to https://github.com/mozilla/mozdownload/commit/ea76769ef8cb62091ea85bf76d79df8098b0e635 which might help.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

I've put the files the scraper is expecting in place, eg
  pub/mobile/candidates/42.0b8-candidates/build1/android-api-11_info.txt
  pub/mobile/candidates/42.0b8-candidates/build1/android-api-9_info.txt
  pub/mobile/candidates/42.0b8-candidates/build1/android-x86_info.txt

If it understands the html spoken by archive.m.o it should pick them up OK and insert the buildIDs into the Socorro db.

Comment 11

[[DEACTIVATED] Adrian Gaudebert]

Once the expected versions have been added to Socorro's database, we will have to reprocess those crashes with a version ending in -b0. The version number is now computed in the processors, so reprocessing is needed to update it.

Comment 12

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/d939a3e6d4732a128f775b22e69103a46296ee15
fixes bug 1217828 - make ftpscraper work on https://archive.mozilla.org

https://github.com/mozilla/socorro/commit/e90285f995bb1eff4dfb34eefdf094b9e5d02eb0
Merge pull request #3063 from peterbe/bug-1217828-wrong-version-in-crash-details-for-firefox-beta

fixes bug 1217828 - make ftpscraper work on https://archive.mozilla.org