Open Bug 1985198 Opened 5 months ago Updated 1 day ago

Intermittent raptor-browsertime Info: TypeError: Cannot read properties of null (reading 'transfer')

Categories

(Testing :: Raptor, defect, P5)

defect

Tracking

(Not tracked)

People

(Reporter: intermittent-bug-filer, Unassigned)

Details

(Keywords: intermittent-failure)

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=524154197&repo=autoland&task=CDIgYkP5QL2LbSTcGQIQbA.0
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CDIgYkP5QL2LbSTcGQIQbA/runs/0/artifacts/public/logs/live_backing.log


[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.319Z adb:command Send '001Ahost:transport:RZCY2011M7N'
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.319Z adb:command Send '0005sync:'
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.326Z adb:sync RECV /sdcard/browsertime.mp4
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.347Z adb:command Send '001Ahost:transport:RZCY2011M7N'
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.347Z adb:command Send '002Dshell:rm -- "/sdcard//sdcard/browsertime.mp4"'
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: CPU / Power usage: 7283
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: Browsertime pageload ended.
[task 2025-08-26T05:46:43.920+00:00] 05:46:43    ERROR -  raptor-browsertime Info: TypeError: Cannot read properties of null (reading 'transfer')
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info:     at /home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/fetches/browsertime/node_modules/usb-power-profiling/usb-power-profiling.js:65:17
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info:     at new Promise (<anonymous>)
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info:     at sendBuffer (/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/fetches/browsertime/node_modules/usb-power-profiling/usb-power-profiling.js:64:10)
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info:     at ShizukuDevice.sendCommand (/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/fetches/browsertime/node_modules/usb-power-profiling/usb-power-profiling.js:343:11)
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info:     at ShizukuDevice.startSampling (/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/fetches/browsertime/node_modules/usb-power-profiling/usb-power-profiling.js:484:16)
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info:     at async tryDevice (/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/fetches/browsertime/node_modules/usb-power-profiling/usb-power-profiling.js:1105:7)
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info:     at async Object.startSampling (/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/fetches/browsertime/node_modules/usb-power-profiling/usb-power-profiling.js:1131:5)
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.859Z Found device: YK-Lab — Korona YK003C in Application Mode Serial Number: 5A5AA5A74E313530FF015030FFFF6232 Vendor Id: 0x483 Product Id: 0xfffe Address: 1:4
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: Not sampling this power meter as its serial number is not 5A5AA5A74E313530FF015030FFFF7217
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.861Z Found device: YK-Lab — Korona YK003C in Application Mode Serial Number: 5A5AA5A74E313530FF025030FFFF1204 Vendor Id: 0x483 Product Id: 0xfffe Address: 1:38
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: Not sampling this power meter as its serial number is not 5A5AA5A74E313530FF015030FFFF7217
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.903Z Found device: YK-Lab — Korona YK003C in Application Mode Serial Number: 5A5AA5A74E313530FF015030FFFF670F Vendor Id: 0x483 Product Id: 0xfffe Address: 1:44
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: Not sampling this power meter as its serial number is not 5A5AA5A74E313530FF015030FFFF7217
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.908Z Found device: YK-Lab — Korona YK003C in Application Mode Serial Number: 5A5AA5A74E313530FF035030FFFF4042 Vendor Id: 0x483 Product Id: 0xfffe Address: 1:46
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: Not sampling this power meter as its serial number is not 5A5AA5A74E313530FF015030FFFF7217
[task 2025-08-26T05:46:43.920+00:00] 05:46:43     INFO -  raptor-browsertime Info: 2025-08-26T05:46:43.910Z Found device: YK-Lab — Korona YK003C in Application Mode Serial Number: 5A5AA5A74E313530FF035030FFFF4012 Vendor Id: 0x483 Product Id: 0xfffe Address: 1:45
[task 2025-08-26T05:50:54.624+00:00] 05:46:43     INFO -  raptor-browsertime Info: Not sampling this power meter as its serial number is not 5A5AA5A74E313530FF015030FFFF7217
[task 2025-08-26T05:50:54.624+00:00] 05:50:45     INFO -  adb Setting test_root to /data/local/tmp/test_root
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  TEST-INFO | screencap: exit 0
[task 2025-08-26T05:50:54.624+00:00] 05:50:46 CRITICAL -  raptor-browsertime Critical: Browsertime process timed out after waiting 240 seconds for output
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  raptor-browsertime-android Info: removing reverse socket connections
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  raptor-mitmproxy Info: Mitmproxy stop!!
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  raptor-mitmproxy Info: Stopping mitmproxy playback, killing process 60565
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  raptor-mitmproxy Info: Successfully killed the mitmproxy playback process
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  raptor-browsertime Info: Test support class is cleaning up...
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  raptor-perftest Info: Removing temporary directory: /tmp/tmpicrzs1qa
[task 2025-08-26T05:50:54.624+00:00] 05:50:46    ERROR -  Traceback (most recent call last):
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -    File "/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/workspace/build/tests/raptor/raptor/raptor.py", line 189, in <module>
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -      main()
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -    File "/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/workspace/build/tests/raptor/raptor/raptor.py", line 138, in main
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -      success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -    File "/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/workspace/build/tests/raptor/raptor/browsertime/android.py", line 251, in run_tests
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -      return super(BrowsertimeAndroid, self).run_tests(tests, test_names)
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -    File "/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/workspace/build/tests/raptor/raptor/perftest.py", line 494, in run_tests
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -      self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -    File "/home/ltuser/foreman/mozilla-lt-devicepool-job-dir.a55-perf.1756186779728164310/tasks/task_175618694597623/workspace/build/tests/raptor/raptor/browsertime/base.py", line 1099, in run_test
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -      raise Exception(
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO -  Exception: Browsertime process timed out after waiting 240 seconds for output
[task 2025-08-26T05:50:54.624+00:00] 05:50:46     INFO - Return code: 1

seems to just be one device with serial R5CXC1AMMNK

:aerickson would you be able to ask lambdatest to take a look?

thanks!

Flags: needinfo?(aerickson)

Yes, definitely. Quarantined for now.

Tracking resolution of these problems in https://mozilla-hub.atlassian.net/browse/RELOPS-1821.

Flags: needinfo?(aerickson)

Quarantined worker RZCY203N75Z for causing almost all failures

Flags: needinfo?(aerickson)

R5CXC1AHXYD is also failing a lot, can we quarantine that one as well?

Flags: needinfo?(amarc)

Added R5CXC1AHXYD to quarantine.

Tracking the issue with lambdatest android devices in https://mozilla-hub.atlassian.net/browse/RELOPS-1969.

Flags: needinfo?(amarc)
Flags: needinfo?(aerickson)

:aerickson, looks like the device RZCX821GYPX is also broken and needs to be quarantined. I see it looking through the list of power meters, but it doesn't seem like it can connect to the one that the device is assigned to (no "Sampling..." found in the log). I noticed that it got some bad from it too.

Flags: needinfo?(aerickson)

Quarantined RZCX821GYPX.

Flags: needinfo?(aerickson)

aerickson, looks like the device RZCXC189JSJ and R5CXC1AP2KT is also broken and needs to be quarantined.

Flags: needinfo?(aerickson)

Agreed. I quarantined those 4 days ago.

Flags: needinfo?(aerickson)
Flags: needinfo?(aerickson)

LT has checked out these devices. Nothing was noticeable. They have rebooted the devices and their associated power meters.

I'll add these to my queue of hosts to retest.

Flags: needinfo?(aerickson)
You need to log in before you can comment on or make changes to this bug.