1678588 - Perma fb-scroll mozperftest.test.browsertime.runner.NodeException: 1

Status: RESOLVED FIXED

(Testing :: mozperftest, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: archaeopteryx [at] coole-files.de
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=322392362&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VWZGmur-QmKaEFAqspFgTA/runs/0/artifacts/public/logs/live_backing.log

---

This fails permanently since bug 1668494 landed.

[task 2020-11-20T05:26:57.851Z] Successfully installed pyssim-0.4
[task 2020-11-20T05:26:58.218Z] Looking in links: https://pypi.pub.build.mozilla.org/pub/
[task 2020-11-20T05:26:58.637Z] ERROR: Could not find a version that satisfies the requirement influxdb==5.3.0 (from versions: none)
[task 2020-11-20T05:26:58.637Z] ERROR: No matching distribution found for influxdb==5.3.0
[task 2020-11-20T05:26:59.001Z] Looking in links: https://pypi.pub.build.mozilla.org/pub/
[task 2020-11-20T05:26:59.422Z] ERROR: Could not find a version that satisfies the requirement grafana_api==1.0.3 (from versions: none)
[task 2020-11-20T05:26:59.422Z] ERROR: No matching distribution found for grafana_api==1.0.3
[task 2020-11-20T05:26:59.455Z]  0:09.35 Created profile at /tmp/tmpw3035wqm.mozrunner
[task 2020-11-20T05:26:59.984Z] Looking in links: https://pypi.pub.build.mozilla.org/pub/
[task 2020-11-20T05:26:59.984Z] Requirement already satisfied: Pillow==7.2.0 in ./obj-x86_64-pc-linux-gnu/_virtualenvs/init_py3/lib/python3.8/site-packages (7.2.0)
[task 2020-11-20T05:27:00.403Z] Looking in links: https://pypi.pub.build.mozilla.org/pub/
[task 2020-11-20T05:27:00.826Z] ERROR: Could not find a version that satisfies the requirement influxdb==5.3.0 (from versions: none)
[task 2020-11-20T05:27:00.826Z] ERROR: No matching distribution found for influxdb==5.3.0
[task 2020-11-20T05:27:01.194Z] Looking in links: https://pypi.pub.build.mozilla.org/pub/
[task 2020-11-20T05:27:01.610Z] ERROR: Could not find a version that satisfies the requirement grafana_api==1.0.3 (from versions: none)
[task 2020-11-20T05:27:01.610Z] ERROR: No matching distribution found for grafana_api==1.0.3
[task 2020-11-20T05:27:01.645Z]  0:11.54 Running browsertime with this command /home/cltbld/.mozbuild/browsertime/node_modules/browsertime/bin/browsertime.js -b firefox --skipHar --viewPort 1366x695 --firefox.binaryPath /home/cltbld/tasks/task_1605849689/fetches/firefox/firefox-bin --firefox.geckodriverPath /home/cltbld/tasks/task_1605849689/fetches/geckodriver --resultDir /home/cltbld/tasks/task_1605849689/artifacts/browsertime-results-1 --firefox.profileTemplate /tmp/tmpw3035wqm.mozrunner --iterations 10 /home/cltbld/tasks/task_1605849689/fetches/netwerk/test/perf/perftest_http3_facebook_scroll.js --firefox.preference mozperftest:true --video true --firefox.windowRecorder true --browsertime.waitTime 10000
[task 2020-11-20T05:27:01.645Z]  0:11.54 /home/cltbld/tasks/task_1605849689/fetches/node/bin/node /home/cltbld/.mozbuild/browsertime/node_modules/browsertime/bin/browsertime.js -b firefox --skipHar --viewPort 1366x695 --firefox.binaryPath /home/cltbld/tasks/task_1605849689/fetches/firefox/firefox-bin --firefox.geckodriverPath /home/cltbld/tasks/task_1605849689/fetches/geckodriver --resultDir /home/cltbld/tasks/task_1605849689/artifacts/browsertime-results-1 --firefox.profileTemplate /tmp/tmpw3035wqm.mozrunner --iterations 10 /home/cltbld/tasks/task_1605849689/fetches/netwerk/test/perf/perftest_http3_facebook_scroll.js --firefox.preference mozperftest:true --video true --firefox.windowRecorder true --browsertime.waitTime 10000
[task 2020-11-20T05:27:02.223Z] [2020-11-20 05:27:02] INFO: Running tests using Firefox - 10 iteration(s)
[task 2020-11-20T05:27:03.953Z] [2020-11-20 05:27:03] INFO: Navigating to url https://www.facebook.com/lambofgod/ iteration 1
[task 2020-11-20T05:27:12.008Z] [2020-11-20 05:27:12] INFO: Start to measure pageload
[task 2020-11-20T05:27:12.009Z] [2020-11-20 05:27:12] INFO: Start firefox window recorder.
[task 2020-11-20T05:27:12.450Z] [2020-11-20 05:27:12] INFO: Navigating to url https://www.facebook.com/lambofgod/ iteration 1
[task 2020-11-20T05:27:20.505Z] [2020-11-20 05:27:20] INFO: Stop firefox window recorder.
[task 2020-11-20T05:27:28.855Z] [2020-11-20 05:27:28] INFO: Bad iteration, redoing...
[task 2020-11-20T05:27:31.357Z] [2020-11-20 05:27:31] INFO: Bad iteration, redoing...
[task 2020-11-20T05:27:33.834Z] [2020-11-20 05:27:33] INFO: Bad iteration, redoing...
[task 2020-11-20T05:27:36.293Z] [2020-11-20 05:27:36] INFO: Bad iteration, redoing...
[task 2020-11-20T05:27:38.731Z] [2020-11-20 05:27:38] INFO: Bad iteration, redoing...
[task 2020-11-20T05:27:38.731Z] [2020-11-20 05:27:38] ERROR: Error: Too many bad scroll iterations occurred
[task 2020-11-20T05:27:38.731Z]     at test (/home/cltbld/tasks/task_1605849689/fetches/netwerk/test/perf/perftest_http3_facebook_scroll.js:139:17)
[task 2020-11-20T05:27:38.732Z]     at process._tickCallback (internal/process/next_tick.js:68:7)
[task 2020-11-20T05:27:39.100Z] [2020-11-20 05:27:39] ERROR: Error running browsertime TypeError: Cannot read property 'length' of undefined
[task 2020-11-20T05:27:39.100Z]     at Collector.perIteration (/home/cltbld/.mozbuild/browsertime/node_modules/browsertime/lib/core/engine/collector.js:116:31)
[task 2020-11-20T05:27:39.100Z]     at Engine.runByScript (/home/cltbld/.mozbuild/browsertime/node_modules/browsertime/lib/core/engine/index.js:178:25)
[task 2020-11-20T05:27:39.100Z]     at process._tickCallback (internal/process/next_tick.js:68:7)
[task 2020-11-20T05:27:39.104Z] Traceback (most recent call last):
[task 2020-11-20T05:27:39.105Z]   File "python/mozperftest/mozperftest/runner.py", line 220, in <module>
[task 2020-11-20T05:27:39.105Z]     sys.exit(main())
[task 2020-11-20T05:27:39.105Z]   File "python/mozperftest/mozperftest/runner.py", line 216, in main
[task 2020-11-20T05:27:39.105Z]     run_tests(mach_cmd, args, user_args)
[task 2020-11-20T05:27:39.105Z]   File "python/mozperftest/mozperftest/runner.py", line 163, in run_tests
[task 2020-11-20T05:27:39.105Z]     e.run(metadata)
[task 2020-11-20T05:27:39.105Z]   File "/home/cltbld/tasks/task_1605849689/fetches/python/mozperftest/mozperftest/environment.py", line 93, in run
[task 2020-11-20T05:27:39.105Z]     metadata = testlayer(syslayer(metadata))
[task 2020-11-20T05:27:39.105Z]   File "/home/cltbld/tasks/task_1605849689/fetches/python/mozperftest/mozperftest/layers.py", line 156, in __call__
[task 2020-11-20T05:27:39.105Z]     metadata = layer(metadata)
[task 2020-11-20T05:27:39.105Z]   File "/home/cltbld/tasks/task_1605849689/fetches/python/mozperftest/mozperftest/layers.py", line 68, in __call__
[task 2020-11-20T05:27:39.105Z]     metadata = self.run(metadata)
[task 2020-11-20T05:27:39.105Z]   File "/home/cltbld/tasks/task_1605849689/fetches/python/mozperftest/mozperftest/test/browsertime/runner.py", line 340, in run
[task 2020-11-20T05:27:39.105Z]     metadata = self._one_cycle(metadata, result_dir)
[task 2020-11-20T05:27:39.105Z]   File "/home/cltbld/tasks/task_1605849689/fetches/python/mozperftest/mozperftest/test/browsertime/runner.py", line 409, in _one_cycle
[task 2020-11-20T05:27:39.105Z]     raise NodeException(exit_code)
[task 2020-11-20T05:27:39.105Z] mozperftest.test.browsertime.runner.NodeException: 1```

Comment 1

[Intermittent Failures Robot]

1 failures in 4810 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• linux64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1678588&startday=2020-11-16&endday=2020-11-22&tree=all

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1668494

Comment 3

[Ryan VanderMeulen [:RyanVM]]

I only see the one instance of this in orangefactor. Is this still an issue?

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Yes, the tasks still fail permanently. They run as tier 3 and are not seen by sheriffs but only show up in perma fail dashboard.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 6

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Please triage this group of permanent failures.

Comment 7

[Greg Mierzwinski [:sparky]]

I'm looking into this issue now. Leaving the ni? open for myself.

Comment 8

[Greg Mierzwinski [:sparky]]

Hmm, it seems like this is only a problem in CI - it's only ever passing for me locally. Getting some more info on them on try: https://treeherder.mozilla.org/jobs?repo=try&revision=5759ec7a6e201c58775cbf2753f0395be7670ad2

Here's where it started completely permafailing:
https://treeherder.mozilla.org/jobs?repo=mozilla-central&group_state=expanded&tier=1%2C2%2C3&searchStr=fb-scroll&revision=a08abfa374b684e27774e5179c1a50d6a92ecfb3

Comment 9

[Greg Mierzwinski [:sparky]]

:dragana, looking at the logs for the test run above, it seems like the images are failing to load very often when we scroll. Would you have some time to look into this?

Comment 10

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: Bug 1678588 - turn off facebook scroll (fb-scroll) perftests. r=sparky DONTBUILD

They were failing permanently since bug 1688494 landed.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b872661bad29
turn off facebook scroll (fb-scroll) perftests. r=sparky DONTBUILD

Comment 12

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/b872661bad29

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.