888840 - Crash reports from Ubuntu builds of Firefox beta are not listed in aggregations

Status: RESOLVED FIXED

(Socorro :: General, task)

Description

[Chris Coulson]

Hi,

We have had the crash reporter enabled in Ubuntu builds of Firefox (release and beta) for some time now so that our users can submit reports directly to https://crash-stats.mozilla.com/ (we also have access to push our symbols to the symbol server).

However, we can no longer access crash reports from users of our Firefox beta builds. This actually started to become difficult when the beta milestone was dropped from the product version number in Firefox - eg, for 22.0b1, our builds are identified as 22.0 as opposed to 22.0b1 (which seems to be the same as the official Mozilla builds too). However, the official Mozilla builds appear in Socorro with the correct version number (including the beta milestone). I assume that there must be some mapping on the server side from the build ID to version number for the official Mozilla Firefox builds.

Anyway, the consequence of this is that there has been no way to find our crash reports in the Socorro interface other than doing an advanced search and then editing the version number in the URL (because 22.0 is not a recognized version number during beta).

However, even this no longer works. If I try to search for reports for 23.0 (which is what our 23.0b1 is identified as), then I get "Select a valid choice. Firefox:23.0 is not one of the available choices."

Is there anything we can do to resolve this? I've disabled the crash reporter for Firefox in Ubuntu for now so that crashes from our builds will go to https://errors.ubuntu.com/ instead.

Comment 1

[Robert Kaiser]

(In reply to Chris Coulson from comment #0)
> 've disabled the crash
> reporter for Firefox in Ubuntu for now

Well, in this case I'd call this a WONTFIX right away. If search isn't good enough, we should just ignore Ubuntu, esp. as it looks that Ubuntu doesn't want Firefox anyhow (change of default browser, which I know you also aren't happy with).

Comment 2

[Robert Kaiser]

And that said, you stated above that you would be able to access the crash reports themselves if they still would be sent. What isn't done is aggregating them into the topcrash views etc. and IMHO, unless you can guarantee that you build the exact changesets we ship as certain betas, your beta builds shouldn't be aggregated with ours as we look very carefully into how patches in certain builds help us or not.

Comment 3

[Chris Coulson]

I've turned it off now because I don't think there is a way for me to search for them anymore. Previously, I could use a URL like this to find them (by doing an advanced search and manually editing the URL to specify the version number without the beta milestone, which isn't in the list of featured versions), but this no longer works:

https://crash-stats.mozilla.com/query/?product=Firefox&version=Firefox%3A23.0&platform=lin&range_value=1&range_unit=weeks&date=07%2F01%2F2013+12%3A00%3A00&query_search=signature&query_type=contains&query=&reason=&release_channels=&build_id=&process_type=browser&hang_type=crash

... this just gives me "Select a valid choice. Firefox:23.0 is not one of the available choices.", which is a recent change in behaviour.

Anyway, resolving WONTFIX is understandable given the circumstances, I guess. But it's disappointing for me to be caught in the middle of this, especially as I'm basically trying to support Firefox in my own time now

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to Chris Coulson from comment #0)
> However, even this no longer works. If I try to search for reports for 23.0
> (which is what our 23.0b1 is identified as), then I get "Select a valid
> choice. Firefox:23.0 is not one of the available choices."
> 
> Is there anything we can do to resolve this? I've disabled the crash
> reporter for Firefox in Ubuntu for now so that crashes from our builds will
> go to https://errors.ubuntu.com/ instead.

Since this hasn't been WONTFIX'd yet ;) here's an explanation of what's going on:

We recently launched a new version of crash-stats which takes more care with input validation, which is why this no longer works. We get the list of valid releases version/buildid/etc. by scraping ftp.mozilla.org so we'd need some way of getting this info into Socorro, or lift this restriction in order for this to work.

Comment 5

[Robert Kaiser]

(In reply to Chris Coulson from comment #3)
> But it's disappointing for me to be caught in the middle of this,
> especially as I'm basically trying to support Firefox in my own time now

Completely understandable.


Rob, I was just about to theorize about something like that being the cause, but we mid-aired ;-)
Do you know if we can do something easily enough to enable at least search there?

Comment 6

[Robert Helmer [:rhelmer]]

(In reply to Robert Kaiser (:[e-mail]) from comment #5)
> (In reply to Chris Coulson from comment #3)
> > But it's disappointing for me to be caught in the middle of this,
> > especially as I'm basically trying to support Firefox in my own time now
> 
> Completely understandable.
> 
> 
> Rob, I was just about to theorize about something like that being the cause,
> but we mid-aired ;-)
> Do you know if we can do something easily enough to enable at least search
> there?

We might be able to add this via the admin panel (not sure if it supports this yet), or remove the restriction that we added to search (that the buildid/product/version combo is valid according to info scraped from FTP)

We are using a lot of shared code to do the latter and it was done on purpose :) So something more like the former would be preferred. I don't think this is possible right now via the admin panel, though.

Maybe you could publish something equivalent to:

https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/23.0b1-candidates/build1/linux_info.txt

In that example we get the build ID from the file and the version/beta info from the URL. Although we refer to it as "ftp scraping" technically we are crawling http://ftp.m.o - we already do some special backflips for B2G and could add one for Ubuntu builds if we want them to show up throughout the UI.

After typing this out, I am pretty ambivalent about it... I really don't like having magical hidden versions that can be selected by manipulating the search URL, though.

Comment 7

[Robert Kaiser]

I think I'd prefer if we would be able to search for the raw version field in the reports as well as the raw release channel field, no matter what FTP scraper "verified". We definitely want to be able to search for things that FTP scraper does not cover - one example there is that any builds created by partners, be it Linux distros or B2G distributors, do no get indexed by the FTP scraper.

Comment 8

[Robert Helmer [:rhelmer]]

(In reply to Robert Kaiser (:[e-mail]) from comment #7)
> I think I'd prefer if we would be able to search for the raw version field
> in the reports as well as the raw release channel field, no matter what FTP
> scraper "verified". We definitely want to be able to search for things that
> FTP scraper does not cover - one example there is that any builds created by
> partners, be it Linux distros or B2G distributors, do no get indexed by the
> FTP scraper.

OK this sounds fine to me - maybe we should have a free-form field in the advanced search UI?

Adrian, what do you think? We should still do some basic input validation I think (e.g. check against a regex since we have some idea of what a "version number" should look like, that is enforced throughout the system in various ways already e.g. urls.py)

Comment 9

[[DEACTIVATED] Adrian Gaudebert]

This fits perfectly in my current work on the new search UI. So if it can wait for the end of Q3, then I'll do that in bug 678101.

Comment 10

[Georg Fritzsche [:gfritzsche]]

Kairo, can you give an update on the current status here - do we want Ubuntu pre-release crash data?

Comment 11

[Robert Kaiser]

We *want* pre-release data from all platforms and distros.

Comment 12

[Lonnen :lonnen]

we've moved on from advanced search. If this is still a problem, please reopen.