1185787 - Unable to bisect on preview releases of Android

Status: RESOLVED FIXED

(Testing :: mozregression, defect)

Description

[Kevin Brosnan [Ex-Mozilla]]

mozregression --bad-release 42 --good-release 41 --app fennec
 0:00.28 LOG: MainThread INFO Using 'bad' date 2015-08-10 for release 42
 0:00.28 LOG: MainThread INFO Using 'good' date 2015-06-29 for release 41
 0:01.25 LOG: MainThread Build Finder WARNING Got HTTPError - retrying
 0:01.25 LOG: MainThread Build Finder WARNING 404 Client Error: Not Found
 0:01.68 LOG: MainThread Build Finder WARNING Got HTTPError - retrying
 0:01.68 LOG: MainThread Build Finder WARNING 404 Client Error: Not Found
Retrieving valid builds from datetime.date(2015, 8, 10) generated an exception: 
404 Client Error: Not Found
[kbrosnan@host-5-240 ~]$ mozregression --good 2015-05-11 --app fennec           
 0:00.32 LOG: MainThread INFO No 'bad' date specified, using 2015-07-20
 0:02.90 LOG: MainThread Bisector INFO Downloading build from: http://ftp.mozill
a.org/pub/mozilla.org/mobile/nightly/2015/06/2015-06-15-03-02-04-mozilla-central
-android-api-11/fennec-41.0a1.multi.android-arm.apk
===== Downloaded 100% =====
 0:04.70 LOG: MainThread Test Runner INFO Running nightly for 2015-06-15
 0:04.75 LOG: MainThread mozversion INFO application_buildid: 20150615030204
 0:04.75 LOG: MainThread mozversion INFO application_changeset: 3c26bef95d54
 0:04.75 LOG: MainThread mozversion INFO application_display_name: Nightly
 0:04.75 LOG: MainThread mozversion INFO application_id: {aa3c5121-dab2-40e2-81c
a-7ea25febc110}
 0:04.75 LOG: MainThread mozversion INFO application_name: Fennec
 0:04.75 LOG: MainThread mozversion INFO application_remotingname: fennec
 0:04.75 LOG: MainThread mozversion INFO application_repository: https://hg.mozi
lla.org/mozilla-central
 0:04.75 LOG: MainThread mozversion INFO application_vendor: Mozilla
 0:04.75 LOG: MainThread mozversion INFO application_version: 41.0a1
 0:04.75 LOG: MainThread mozversion INFO package_name: org.mozilla.fennec
 0:04.75 LOG: MainThread mozversion INFO platform_buildid: 20150615030204
 0:04.75 LOG: MainThread mozversion INFO platform_changeset: 3c26bef95d54
 0:04.75 LOG: MainThread mozversion INFO platform_repository: https://hg.mozilla
.org/mozilla-central
 0:04.75 LOG: MainThread mozversion INFO platform_version: 41.0a1
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry
' or 'exit' and press Enter): good
Traceback (most recent call last):
  File "/usr/bin/mozregression", line 9, in <module>
    load_entry_point('mozregression==0.39', 'console_scripts', 'mozregression')(
)
  File "/usr/lib/python2.7/site-packages/mozregression/main.py", line 384, in cl
i
    sys.exit(bisect(runner, logger))
  File "/usr/lib/python2.7/site-packages/mozregression/main.py", line 278, in bi
sect_nightlies
    return runner.bisect_nightlies(good_date, bad_date)
  File "/usr/lib/python2.7/site-packages/mozregression/bisector.py", line 518, i
n bisect_nightlies
    result = self.do_bisect(handler, good_date, bad_date)
  File "/usr/lib/python2.7/site-packages/mozregression/main.py", line 212, in do
_bisect
    return BisectRunner.do_bisect(self, handler, good, bad, **kwargs)
  File "/usr/lib/python2.7/site-packages/mozregression/bisector.py", line 514, i
n do_bisect
    return self.bisector.bisect(handler, good, bad, **kwargs)
  File "/usr/lib/python2.7/site-packages/mozregression/bisector.py", line 458, i
n bisect
    return self._bisect(handler, build_data)
  File "/usr/lib/python2.7/site-packages/mozregression/bisector.py", line 488, i
n _bisect
    verdict, app_info = bisection.evaluate(build_infos)
  File "/usr/lib/python2.7/site-packages/mozregression/bisector.py", line 373, i
n evaluate
    allow_back=bool(self.previous_data))
  File "/usr/lib/python2.7/site-packages/mozregression/test_runner.py", line 114
, in evaluate
    launcher.stop()
  File "/usr/lib/python2.7/site-packages/mozregression/launchers.py", line 60, i
n stop
    self._stop()
  File "/usr/lib/python2.7/site-packages/mozregression/launchers.py", line 201, 
in _stop
    self.adb.stop_application(self.package_name)
  File "/usr/lib/python2.7/site-packages/mozdevice/adb_android.py", line 320, in
 stop_application
    if StrictVersion(version) >= StrictVersion('3.0'):
  File "/usr/lib64/python2.7/distutils/version.py", line 40, in __init__
    self.parse(vstring)
  File "/usr/lib64/python2.7/distutils/version.py", line 107, in parse
    raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number 'M'
Exception ValueError: ValueError("invalid version number 'M'",) in <bound method
 FennecLauncher.__del__ of <mozregression.launchers.FennecLauncher object at 0x7
f3b5047ea90>> ignored

Comment 1

[Kevin Brosnan [Ex-Mozilla]]

This is the developer preview Android M - http://developer.android.com/preview/index.html

Comment 2

[Julien Pagès (:parkouss)]

Thanks for the bug report. This looks like a mozdevice bug looking at the stack trace and the error, I just created bug 1185969 to track this.

Comment 3

[Sebastian Kaspari (:sebastian; :pocmo)]

I wonder if this is still a problem with the final release of the Android 6.0 SDK? The version number should now be 23 instead of 'M'. But maybe the intention is to fix this for upcoming preview versions too?

Comment 4

[Kevin Brosnan [Ex-Mozilla]]

Yes I was able to use the reelase canidate 3 with MozRegression. Though I expect this bug to be valid for the next preview releaes of Android 'N?'.

Comment 5

[Julien Pagès (:parkouss)]

Fixed upstream in bug 1185969.

Landed new required version of mozdevice in https://github.com/mozilla/mozregression/commit/78498e50b037b3815d98f85c2b9244a482d5e538.