869564 - ftpscraper CRITICAL today: "default is not a valid release channel"

Status: RESOLVED FIXED

(Socorro :: Backend, task)

Description

[Selena Deckelmann :selenamarie :selena]

2013-05-07 10:48:23,891 DEBUG - MainThread - error when running <class 'socorro.cron.jobs.ftpscraper.FTPScraperCronApp'> on None
Traceback (most recent call last):
  File "/data/socorro/application/socorro/cron/crontabber.py", line 701, in _run_one
    for last_success in self._run_job(job_class, config, info):
  File "/data/socorro/application/socorro/cron/base.py", line 173, in main
    function(when)
  File "/data/socorro/application/socorro/cron/base.py", line 212, in _run_proxy
    self.run(connection, date)
  File "/data/socorro/application/socorro/cron/jobs/ftpscraper.py", line 200, in run
    self.scrapeB2G(connection, product_name, date)
  File "/data/socorro/application/socorro/cron/jobs/ftpscraper.py", line 311, in scrapeB2G
    ignore_duplicates=True
  File "/data/socorro/application/socorro/lib/buildutil.py", line 53, in insert_build
    cursor.execute(sql, params)
InternalError: default is not a valid release channel
CONTEXT:  SQL statement "SELECT validate_lookup('release_channels','release_channel',release_channel,'release channel')"
PL/pgSQL function add_new_release(citext,citext,citext,numeric,citext,integer,text,boolean,boolean) line 29 at PERFORM

Comment 1

[Selena Deckelmann :selenamarie :selena]

Trying to work out what to do about this with help from releng.

Comment 2

[Selena Deckelmann :selenamarie :selena]

Hi Alexander,

Here's an abbreviated version of the email I sent yesterday: 

Yesterday, the channel got changed for B2G from "nightly" to "default". We don't know what default means or how to map it to our stability groupings.

Here's a JSON file that we use to identify B2G releases:

from ftp://ftp.mozilla.org/pub/mozilla.org/b2g/manifests/1.1.0/2013-05-07-07/socorro_inari_2013-05-07-07.json:
{

    "buildid": "20130507070204",
    "update_channel": "nightly",
    "version": "18.0"

}

Here's what we're seeing now: 
ftp://ftp.mozilla.org/pub/mozilla.org/b2g/manifests/1.1.0/2013-05-07-07/socorro_leo-eng_2013-05-07-07.json
{

    "buildid": "20130507070204",
    "update_channel": "default",
    "version": "18.0"

}

What do you think a good representation of stability would be, if it was to be used in a similar fashion - to group bugs that might gate a release?

* Gecko version
* Some combination of versions
* Something else entirely

And, do we need more than 4 levels of stability for B2G?

Comment 3

[Robert Kaiser]

"default" is by established definition a channel for custom builds that do not get updates. We get crash reports with those but don't do a lot with them (unless they match a release version number, when we just count them against release, as many Linux distro builds end up with the default channel setting of "default").

Comment 4

[Alex Keybl [:akeybl]]

We discussed this during the Socorro team meeting today - we'll start using nightly-device-version or similar for all internal builds. So for instance nightly-unagi-11 or similar for nightly Unagi 1.1 builds. 

In general, we should have the flexibility to bucket based upon channel and build ID. Standardizing those may prove difficult with partners, when we don't have a final sign-off opportunity on the phone and its preferences. I'd suggest remaining as flexible as possible with what defines a tracked B2G version for stability.

Comment 5

[Chris AtLee [:catlee]]

Actually, we agreed to use the regular desktop channel names "nightly", "beta", "release" for the channel name. That's tied to how updates work ATM, so I'm working on untangling the two.

Comment 6

[Selena Deckelmann :selenamarie :selena]

(In reply to Chris AtLee [:catlee] from comment #5)
> Actually, we agreed to use the regular desktop channel names "nightly",
> "beta", "release" for the channel name. That's tied to how updates work ATM,
> so I'm working on untangling the two.

Thanks, Chris. 

If you would like releases that currently have 'default' in their update_channel to have crashes processed, I need a way of mapping into one of 'nightly/aurora/beta/release/esr". Should I just map into 'nightly' for now?

I'll create a separate bug for how we make this system more flexible moving forward.

Comment 7

[Robert Kaiser]

Selena, they probably will continue to use "default" as the channel for per-checkin, non-nightly builds, which is usual process. In that case, please make the FTP scraper just ignore (probably with a log message so we can debug, just in case) any build with a "default" channel.

Comment 8

[Selena Deckelmann :selenamarie :selena]

(In reply to Robert Kaiser (:[e-mail]) from comment #7)
> Selena, they probably will continue to use "default" as the channel for
> per-checkin, non-nightly builds, which is usual process. In that case,
> please make the FTP scraper just ignore (probably with a log message so we
> can debug, just in case) any build with a "default" channel.

Ok. Thanks!  Doing this now.

Comment 9

[[github robot]]

Commits pushed to master at https://github.com/mozilla/socorro

https://github.com/mozilla/socorro/commit/7b75abdce57190042fe508335a07bb200152b485
bug 869564 ignore 'default' in B2G json

https://github.com/mozilla/socorro/commit/c641f8e54804619ea33d89924da927b233d02778
Merge pull request #1237 from selenamarie/bug869564-ftpscraper-default

bug 869564 ignore 'default' as a channel for b2g

Comment 10

[Selena Deckelmann :selenamarie :selena]

Recommending that we cherry pick this soon, since it is currently busted in prod.