1488468 - Resolve CI test failures for the telemetry-tests-client suite

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Raphael Aurich [:raphael] UTC+02:00]

Currently the telemetry-tests-client suite fails on CI due to an issue with installing Python dependencies.

See try build https://treeherder.mozilla.org/#/jobs?repo=try&revision=ba593ba8f3c602ae79e1b0d410218a515a541d61&filter-tier=1&filter-tier=2&filter-tier=3&selectedJob=197340728

Comment 1

[Raphael Aurich [:raphael] UTC+02:00]

Resolved an issue with installing "mozterm" from telemetry_tests_requirements.txt by updating "../tools/mozterm" to "../../python/mozterm".

Comment 2

[Raphael Aurich [:raphael] UTC+02:00]

It appears marionette/harness/marionette_harness/runner/httpd.py is coded against "WebTestHttpd" of wptserve 2.0, however the environment has 1.4 installed which is the latest release on PyPI and Mozilla's index server. The path to wptserve in config/marionette_requirements.txt is "../tools/wptserve", but probably needs to be "../web-platform/tests/tools/wptserve".

Comment 3

[Georg Fritzsche [:gfritzsche]]

Setting as P1 (which is how we track bugs that are actively worked on).

Comment 4

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: [telemetry] Resolve CI test failures in telemetry-tests-client suite (Bug 1488468).

Currently the telemetry-tests-client suite fails on CI. This patch addresses a series of problems with Python dependencies and the test implementation.

Comment 5

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 9011804 [details]
[telemetry] Resolve CI test failures in telemetry-tests-client suite (Bug 1488468).

Dave Hunt [:davehunt] ⌚️UTC+1 has approved the revision.

Comment 6

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Bump wptserve in marionette_requirements.txt to 2.0.0; r?davehunt

Comment 7

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Move ping_list to setUp of TelemetryTestCase; r?davehunt

Depends on D7773

Comment 8

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Fix ValueError in telemetry_client.py; r?davehunt

Depends on D7774

Comment 9

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Remove wait.until calls and improve formatting in test_main_tab_scalar; r?davehunt

Depends on D7775

Comment 10

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Rename telemetry_tests_requirements.txt file; r?davehunt

Depends on D7776

Comment 11

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Update requirements files for telemetry tests; r?davehunt

Depends on D7777

Comment 12

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Update telemetry_client.py to expect three return values for evaluate_parser(); r?davehunt

Depends on D7774

Comment 13

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Update requirements files for telemetry tests; r?davehunt

Depends on D7776

Comment 14

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Bump wptserve in marionette_requirements.txt to 2.0.0; r?davehunt

Comment 15

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Do not share ping_list of TelemetryTestCase between instances; r?davehunt

Depends on D7902

Comment 16

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Update telemetry_client.py to expect three return values for evaluate_parser(); r?davehunt

Depends on D7903

Comment 17

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Remove wait.until calls and improve formatting in test_main_tab_scalar; r?davehunt

Depends on D7904

Comment 18

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Update requirements files for telemetry tests; r?davehunt

Depends on D7905

Comment 19

[Raphael Aurich [:raphael] UTC+02:00]

Try build for telemetry-tests-client
https://treeherder.mozilla.org/#/jobs?repo=try&revision=93f829cf058a833bfe8356a8a3cd3d41ba9c0156

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/246160f26d96
Bump wptserve in marionette_requirements.txt to 2.0.0; r=davehunt
https://hg.mozilla.org/integration/autoland/rev/af05b1808f36
Do not share ping_list of TelemetryTestCase between instances; r=davehunt
https://hg.mozilla.org/integration/autoland/rev/5084bccfd9ac
Update telemetry_client.py to expect three return values for evaluate_parser(); r=davehunt
https://hg.mozilla.org/integration/autoland/rev/b6f68f56cc7f
Remove wait.until calls and improve formatting in test_main_tab_scalar; r=davehunt
https://hg.mozilla.org/integration/autoland/rev/b2a9499e2c3e
Update requirements files for telemetry tests; r=davehunt

Comment 21

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/246160f26d96
https://hg.mozilla.org/mozilla-central/rev/af05b1808f36
https://hg.mozilla.org/mozilla-central/rev/5084bccfd9ac
https://hg.mozilla.org/mozilla-central/rev/b6f68f56cc7f
https://hg.mozilla.org/mozilla-central/rev/b2a9499e2c3e

Comment 22

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Filter ping_list in Wait function of TelemetryTestCase; r?davehunt

Comment 23

[Raphael Aurich [:raphael] UTC+02:00]

Attachment: Bug 1488468 - Move HTTP POST request to wait_for_ping in TestPingServer; r?davehunt

Depends on D8531

Comment 24

[Dave Hunt [:davehunt] [he/him] ⌚BST]

:raphael as this bug is closed, we typically take care of follow ups by opening new bugs. I reviewed the patches before realising they're attached to a closed bug. Could you open a new bug and submit your patches again, and you can carry across my r+ or request it again and I'll approve/land them.

Comment 25

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Reopening the bug this time for the follow-ups. Next time please really create a new bug for that.

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bf88501c1f8a
Filter ping_list in Wait function of TelemetryTestCase; r=davehunt
https://hg.mozilla.org/integration/autoland/rev/51b87be56bca
Move HTTP POST request to wait_for_ping in TestPingServer; r=davehunt

Comment 27

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/bf88501c1f8a
https://hg.mozilla.org/mozilla-central/rev/51b87be56bca