Closed Bug 1601481 Opened 4 years ago Closed 4 years ago

Fix inadventently broken Raptor unittest: test_build_profile[RaptorDesktopFirefox

Categories

(Testing :: Raptor, defect)

Product:
Testing
Component:
Raptor
Version:
Version 3
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: stephend, Assigned: stephend)

References

(
URL
)

Details

I've inadvertently broken test_build_profile[RaptorDesktopFirefox-firefox] (see https://firefoxci.taskcluster-artifacts.net/Pyv_s90DTqK-RlXP_KmIkw/0/public/logs/live_backing.log from https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&author=sdonner%40mozilla.com&selectedJob=279656233)

Noting and making sure I've addressed this before posting patch for full review.

Stack here:

[task 2019-12-04T22:27:36.914Z] 1:24.96 =================================== FAILURES ===================================
[task 2019-12-04T22:27:36.914Z] 1:24.96 _______________ test_build_profile[RaptorDesktopFirefox-firefox] _______________
[task 2019-12-04T22:27:36.914Z] 1:24.96
[task 2019-12-04T22:27:36.914Z] 1:24.96 options = {'app': 'firefox', 'binary': 'path/to/dummy/browser'}
[task 2019-12-04T22:27:36.914Z] 1:24.96 perftest_class = <class 'raptor.raptor.RaptorDesktopFirefox'>
[task 2019-12-04T22:27:36.914Z] 1:24.96 app_name = 'firefox', get_prefs = None
[task 2019-12-04T22:27:36.914Z] 1:24.96
[task 2019-12-04T22:27:36.914Z] 1:24.96 @pytest.mark.parametrize("perftest_class, app_name", [
[task 2019-12-04T22:27:36.914Z] 1:24.96 [RaptorDesktopFirefox, "firefox"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [RaptorDesktopChrome, "chrome"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [RaptorDesktopChrome, "chromium"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [RaptorAndroid, "fennec"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [RaptorAndroid, "geckoview"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [BrowsertimeDesktop, "firefox"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [BrowsertimeDesktop, "chrome"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [BrowsertimeDesktop, "chromium"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [BrowsertimeAndroid, "fennec"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 [BrowsertimeAndroid, "geckoview"],
[task 2019-12-04T22:27:36.914Z] 1:24.96 ])
[task 2019-12-04T22:27:36.914Z] 1:24.96 def test_build_profile(options, perftest_class, app_name, get_prefs):
[task 2019-12-04T22:27:36.914Z] 1:24.96 options['app'] = app_name
[task 2019-12-04T22:27:36.914Z] 1:24.96 > perftest_instance = perftest_class(**options)
[task 2019-12-04T22:27:36.914Z] 1:24.96
[task 2019-12-04T22:27:36.914Z] 1:24.96 testing/raptor/test/test_raptor.py:58:
[task 2019-12-04T22:27:36.914Z] 1:24.96 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-12-04T22:27:36.914Z] 1:24.96 testing/raptor/raptor/raptor.py:1043: in init
[task 2019-12-04T22:27:36.914Z] 1:24.96 super(RaptorDesktop, self).init(*args, **kwargs)
[task 2019-12-04T22:27:36.914Z] 1:24.96 testing/raptor/raptor/raptor.py:848: in init
[task 2019-12-04T22:27:36.914Z] 1:24.96 super(Raptor, self).init(*args, **kwargs)
[task 2019-12-04T22:27:36.914Z] 1:24.96 testing/raptor/raptor/raptor.py:173: in init
[task 2019-12-04T22:27:36.914Z] 1:24.96 self.build_browser_profile()
[task 2019-12-04T22:27:36.914Z] 1:24.96 testing/raptor/raptor/raptor.py:945: in build_browser_profile
[task 2019-12-04T22:27:36.915Z] 1:24.96 super(Raptor, self).build_browser_profile()
[task 2019-12-04T22:27:36.915Z] 1:24.96 testing/raptor/raptor/raptor.py:210: in build_browser_profile
[task 2019-12-04T22:27:36.915Z] 1:24.96 self.get_conditioned_profile()
[task 2019-12-04T22:27:36.915Z] 1:24.96 testing/raptor/raptor/raptor.py:184: in get_conditioned_profile
[task 2019-12-04T22:27:36.915Z] 1:24.96 cond_prof_target_dir = get_profile(temp_download_dir, platform, "cold")
[task 2019-12-04T22:27:36.915Z] 1:24.96 testing/condprofile/condprof/client.py:80: in get_profile
[task 2019-12-04T22:27:36.915Z] 1:24.96 size = len(list(tar))
[task 2019-12-04T22:27:36.915Z] 1:24.96 obj-x86_64-apple-darwin18.6.0/_virtualenvs/gecko-C4HgOhW0-/usr/local/bin/python3/lib/python3.7/tarfile.py:2405: in iter
[task 2019-12-04T22:27:36.915Z] 1:24.96 tarinfo = self.next()
[task 2019-12-04T22:27:36.915Z] 1:24.96 obj-x86_64-apple-darwin18.6.0/_virtualenvs/gecko-C4HgOhW0-/usr/local/bin/python3/lib/python3.7/tarfile.py:2281: in next
[task 2019-12-04T22:27:36.915Z] 1:24.96 self.fileobj.seek(self.offset - 1)
[task 2019-12-04T22:27:36.915Z] 1:24.96 /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/gzip.py:368: in seek
[task 2019-12-04T22:27:36.915Z] 1:24.96 return self._buffer.seek(offset, whence)
[task 2019-12-04T22:27:36.915Z] 1:24.96 /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_compression.py:143: in seek
[task 2019-12-04T22:27:36.915Z] 1:24.96 data = self.read(min(io.DEFAULT_BUFFER_SIZE, offset))
[task 2019-12-04T22:27:36.915Z] 1:24.96 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-12-04T22:27:36.915Z] 1:24.96
[task 2019-12-04T22:27:36.915Z] 1:24.96 self = <gzip._GzipReader object at 0x10bb268d0>, size = 8192
[task 2019-12-04T22:27:36.915Z] 1:24.96
[task 2019-12-04T22:27:36.915Z] 1:24.96 def read(self, size=-1):
[task 2019-12-04T22:27:36.915Z] 1:24.96 if size < 0:
[task 2019-12-04T22:27:36.915Z] 1:24.96 return self.readall()
[task 2019-12-04T22:27:36.915Z] 1:24.96 # size=0 is special because decompress(max_length=0) is not supported
[task 2019-12-04T22:27:36.915Z] 1:24.96 if not size:
[task 2019-12-04T22:27:36.915Z] 1:24.96 return b""
[task 2019-12-04T22:27:36.915Z] 1:24.96
[task 2019-12-04T22:27:36.915Z] 1:24.96 # For certain input data, a single
[task 2019-12-04T22:27:36.915Z] 1:24.96 # call to decompress() may not return
[task 2019-12-04T22:27:36.915Z] 1:24.96 # any data. In this case, retry until we get some data or reach EOF.
[task 2019-12-04T22:27:36.915Z] 1:24.96 while True:
[task 2019-12-04T22:27:36.915Z] 1:24.96 if self._decompressor.eof:
[task 2019-12-04T22:27:36.915Z] 1:24.96 # Ending case: we've come to the end of a member in the file,
[task 2019-12-04T22:27:36.915Z] 1:24.96 # so finish up this member, and read a new gzip header.
[task 2019-12-04T22:27:36.915Z] 1:24.96 # Check the CRC and file size, and set the flag so we read
[task 2019-12-04T22:27:36.915Z] 1:24.96 # a new member
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._read_eof()
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._new_member = True
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._decompressor = self._decomp_factory(
[task 2019-12-04T22:27:36.916Z] 1:24.96 **self._decomp_args)
[task 2019-12-04T22:27:36.916Z] 1:24.96
[task 2019-12-04T22:27:36.916Z] 1:24.96 if self._new_member:
[task 2019-12-04T22:27:36.916Z] 1:24.96 # If the _new_member flag is set, we have to
[task 2019-12-04T22:27:36.916Z] 1:24.96 # jump to the next member, if there is one.
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._init_read()
[task 2019-12-04T22:27:36.916Z] 1:24.96 if not self._read_gzip_header():
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._size = self._pos
[task 2019-12-04T22:27:36.916Z] 1:24.96 return b""
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._new_member = False
[task 2019-12-04T22:27:36.916Z] 1:24.96
[task 2019-12-04T22:27:36.916Z] 1:24.96 # Read a chunk of data from the file
[task 2019-12-04T22:27:36.916Z] 1:24.96 buf = self._fp.read(io.DEFAULT_BUFFER_SIZE)
[task 2019-12-04T22:27:36.916Z] 1:24.96
[task 2019-12-04T22:27:36.916Z] 1:24.96 uncompress = self._decompressor.decompress(buf, size)
[task 2019-12-04T22:27:36.916Z] 1:24.96 if self._decompressor.unconsumed_tail != b"":
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._fp.prepend(self._decompressor.unconsumed_tail)
[task 2019-12-04T22:27:36.916Z] 1:24.96 elif self._decompressor.unused_data != b"":
[task 2019-12-04T22:27:36.916Z] 1:24.96 # Prepend the already read bytes to the fileobj so they can
[task 2019-12-04T22:27:36.916Z] 1:24.96 # be seen by _read_eof() and _read_gzip_header()
[task 2019-12-04T22:27:36.916Z] 1:24.96 self._fp.prepend(self._decompressor.unused_data)
[task 2019-12-04T22:27:36.916Z] 1:24.96
[task 2019-12-04T22:27:36.916Z] 1:24.96 if uncompress != b"":
[task 2019-12-04T22:27:36.916Z] 1:24.96 break
[task 2019-12-04T22:27:36.916Z] 1:24.96 if buf == b"":
[task 2019-12-04T22:27:36.916Z] 1:24.96 > raise EOFError("Compressed file ended before the "
[task 2019-12-04T22:27:36.916Z] 1:24.96 "end-of-stream marker was reached")
[task 2019-12-04T22:27:36.916Z] 1:24.96 E EOFError: Compressed file ended before the end-of-stream marker was reached
[task 2019-12-04T22:27:36.916Z] 1:24.96
[task 2019-12-04T22:27:36.916Z] 1:24.96 /usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/gzip.py:482: EOFError
[task 2019-12-04T22:27:36.916Z] 1:24.96 ----------------------------- Captured stdout call -----------------------------
[task 2019-12-04T22:27:36.917Z] 1:24.96 Getting https://index.taskcluster.net/v1/task/gecko.v2.try.latest.firefox.condprof-macosx64/artifacts/public/condprof/profile-macosx64-cold-default.tgz
[task 2019-12-04T22:27:36.917Z] 1:24.96 Extracting the tarball content in /var/folders/70/_p63h8417253ysc_spcrn3rh000017/T/tmpffk2v1cm
[task 2019-12-04T22:27:36.917Z] 1:24.96 ----------------------------- Captured stderr call -----------------------------
[task 2019-12-04T22:27:36.917Z] 1:24.96 ['path/to/dummy/browser', '--version']
[task 2019-12-04T22:27:36.917Z] 1:24.96 ['path/to/dummy/browser', '--version']
[task 2019-12-04T22:27:36.917Z] 1:24.96 _____________ test_profile_class_assignation[firefox-None-firefox] _____________
[task 2019-12-04T22:27:36.917Z] 1:24.96
[task 2019-12-04T22:27:36.917Z] 1:24.96 ConcretePerftest = <class 'test.conftest.ConcretePerftest.<locals>.PerftestImplementation'>
[task 2019-12-04T22:27:36.917Z] 1:24.96 options = {'app': None, 'binary': 'path/to/dummy/browser'}
[task 2019-12-04T22:27:36.917Z] 1:24.96 profile_class = 'firefox', app = None, expected_profile = 'firefox'
[task 2019-12-04T22:27:36.917Z] 1:24.96
[task 2019-12-04T22:27:36.917Z] 1:24.96 @pytest.mark.parametrize('profile_class, app, expected_profile', [
[task 2019-12-04T22:27:36.917Z] 1:24.96 ['firefox', 'firefox', 'firefox'],
[task 2019-12-04T22:27:36.917Z] 1:24.96 [None, 'firefox', 'firefox'],
[task 2019-12-04T22:27:36.917Z] 1:24.96 ['firefox', None, 'firefox'],
[task 2019-12-04T22:27:36.917Z] 1:24.96 ['firefox', 'fennec', 'firefox'],
[task 2019-12-04T22:27:36.917Z] 1:24.96 ])
[task 2019-12-04T22:27:36.917Z] 1:24.96 def test_profile_class_assignation(ConcretePerftest,
[task 2019-12-04T22:27:36.917Z] 1:24.96 options,
[task 2019-12-04T22:27:36.917Z] 1:24.96 profile_class,
[task 2019-12-04T22:27:36.917Z] 1:24.96 app,
[task 2019-12-04T22:27:36.917Z] 1:24.96 expected_profile):
[task 2019-12-04T22:27:36.917Z] 1:24.96 options['app'] = app
[task 2019-12-04T22:27:36.917Z] 1:24.96 > perftest = ConcretePerftest(profile_class=profile_class, **options)
[task 2019-12-04T22:27:36.917Z] 1:24.96
[task 2019-12-04T22:27:36.917Z] 1:24.96 ../../test/test_raptor.py:118:
[task 2019-12-04T22:27:36.917Z] 1:24.96 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-12-04T22:27:36.917Z] 1:24.96 ../raptor.py:173: in init
[task 2019-12-04T22:27:36.917Z] 1:24.96 self.build_browser_profile()
[task 2019-12-04T22:27:36.917Z] 1:24.96 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[task 2019-12-04T22:27:36.917Z] 1:24.96
[task 2019-12-04T22:27:36.917Z] 1:24.96 self = <test.conftest.ConcretePerftest.<locals>.PerftestImplementation object at 0x10d983e50>
[task 2019-12-04T22:27:36.917Z] 1:24.96
[task 2019-12-04T22:27:36.918Z] 1:24.96 def build_browser_profile(self):
[task 2019-12-04T22:27:36.918Z] 1:24.96 if self.config['platform'] == 'win' and self.config['processor'] == 'aarch64':
[task 2019-12-04T22:27:36.918Z] 1:24.96 LOG.info("No conditioned profiles yet for aarch64; see bug 1582757")
[task 2019-12-04T22:27:36.918Z] 1:24.96 LOG.info("Falling back to mozrunner-created profiles")
[task 2019-12-04T22:27:36.918Z] 1:24.96 self.profile = create_profile(self.profile_class)
[task 2019-12-04T22:27:36.918Z] 1:24.96 # https://searchfox.org/mozilla-central/rev/8bc24752246aeac8a9aed566cf1caccf88d97d11/testing/raptor/raptor/cmdline.py#12-40
[task 2019-12-04T22:27:36.918Z] 1:24.96 # we should be able to key off self.config['app'] or self.profile_class
[task 2019-12-04T22:27:36.918Z] 1:24.96 # https://searchfox.org/mozilla-central/rev/8bc24752246aeac8a9aed566cf1caccf88d97d11/testing/raptor/raptor/raptor.py#137-139
[task 2019-12-04T22:27:36.918Z] 1:24.96 > if 'firefox' not in self.config.get('app', []):
[task 2019-12-04T22:27:36.918Z] 1:24.96 E TypeError: argument of type 'NoneType' is not iterable
[task 2019-12-04T22:27:36.918Z] 1:24.96
[task 2019-12-04T22:27:36.918Z] 1:24.96 ../raptor.py:207: TypeError
[task 2019-12-04T22:27:36.918Z] 1:24.96 ========== 2 failed, 40 passed, 1 skipped, 2 xfailed in 49.77 seconds ==========
[task 2019-12-04T22:27:36.918Z] 1:24.96 Setting retcode to 1 from /Users/cltbld/tasks/task_1575498169/checkouts/gecko/testing/raptor/test/test_raptor.py
[task 2019-12-04T22:27:36.928Z] 1:24.97 Return code from mach python-test: 1
[taskcluster 2019-12-04T22:27:37.093Z] Exit Code: 1

This was fixed and pushed to https://phabricator.services.mozilla.com/D55833 as elif self.config['app'] in self.firefox_android_apps:

As seen in the original traceback, it was if 'firefox' not in self.config.get('app', []): prior.

Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.