1655869 - Perma [tier 2] condprof arsenic.errors.SessionStartError: unknown error: Invalid Content-Type

Status: RESOLVED FIXED

(Testing :: Condprofile, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=311331540&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JrAbifJ-Qb6U7KNdlsfxww/runs/0/artifacts/public/logs/live_backing.log

---

task 2020-07-28T23:10:27.815Z] condprof INFO | Removing pref marionette.enabled: True
[task 2020-07-28T23:10:27.816Z] condprof INFO | Success, we have a profile to work with
[task 2020-07-28T23:10:27.816Z] condprof INFO | Updating profile located at '/tmp/tmplu0636bw/profile'
[task 2020-07-28T23:10:27.817Z] condprof INFO | Reading existing metadata at /tmp/tmplu0636bw/profile/condprofile.json
[task 2020-07-28T23:10:27.817Z] condprof INFO | Starting the Gecko app...
[task 2020-07-28T23:10:27.817Z] condprof INFO | Writing geckodriver logs in /builds/worker/archive/geckodriver-settled-default.log
[task 2020-07-28T23:10:27.820Z] condprof INFO | Running Webdriver on port 37623
[task 2020-07-28T23:10:27.820Z] condprof INFO | Running Marionette on port 2828
[task 2020-07-28T23:10:27.826Z] condprof ERROR | settled scenario broke!
[task 2020-07-28T23:10:27.826Z] condprof INFO | Could not start the session, check /builds/worker/archive/geckodriver-settled-default.log first
[task 2020-07-28T23:10:27.826Z] condprof ERROR | settled x condprof/../condprof/customization/default.json failed.
[task 2020-07-28T23:10:27.826Z] Traceback (most recent call last):
[task 2020-07-28T23:10:27.826Z] File "condprof/../condprof/runner.py", line 155, in run_all
[task 2020-07-28T23:10:27.826Z] res.append(await self.one_run(scenario, customization))
[task 2020-07-28T23:10:27.826Z] File "condprof/../condprof/runner.py", line 130, in one_run
[task 2020-07-28T23:10:27.826Z] ).run(not self.visible)
[task 2020-07-28T23:10:27.826Z] File "condprof/../condprof/creator.py", line 78, in run
[task 2020-07-28T23:10:27.826Z] metadata = await self.build_profile(device, headless)
[task 2020-07-28T23:10:27.826Z] File "condprof/../condprof/creator.py", line 153, in build_profile
[task 2020-07-28T23:10:27.826Z] async with get_session(geckodriver, firefox_instance) as session:
[task 2020-07-28T23:10:27.826Z] File "/builds/worker/fetches/condprofile/lib/python3.7/site-packages/arsenic/init.py", line 16, in aenter
[task 2020-07-28T23:10:27.826Z] self.session = await start_session(self.service, self.browser, self.bind)
[task 2020-07-28T23:10:27.826Z] File "/builds/worker/fetches/condprofile/lib/python3.7/site-packages/arsenic/init.py", line 29, in start_session
[task 2020-07-28T23:10:27.826Z] return await driver.new_session(browser, bind=bind)
[task 2020-07-28T23:10:27.826Z] File "/builds/worker/fetches/condprofile/lib/python3.7/site-packages/arsenic/webdriver.py", line 56, in new_session
[task 2020-07-28T23:10:27.826Z] err_resp["error"], err_resp.get("message", ""), original_response
[task 2020-07-28T23:10:27.826Z] arsenic.errors.SessionStartError: unknown error: Invalid Content-Type
[task 2020-07-28T23:10:27.827Z] Traceback (most recent call last):
[task 2020-07-28T23:10:27.827Z] File "condprof/main.py", line 78, in <module>
[task 2020-07-28T23:10:27.827Z] main()
[task 2020-07-28T23:10:27.827Z] File "condprof/main.py", line 73, in main
[task 2020-07-28T23:10:27.827Z] args.device_name,
[task 2020-07-28T23:10:27.827Z] File "condprof/../condprof/runner.py", line 187, in run
[task 2020-07-28T23:10:27.827Z] failures, results = loop.run_until_complete(runner.run_all())
[task 2020-07-28T23:10:27.827Z] File "/usr/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
[task 2020-07-28T23:10:27.827Z] return future.result()
[task 2020-07-28T23:10:27.827Z] File "condprof/../condprof/runner.py", line 158, in run_all
[task 2020-07-28T23:10:27.827Z] self.display_error(scenario, customization)
[task 2020-07-28T23:10:27.827Z] File "condprof/../condprof/runner.py", line 155, in run_all
[task 2020-07-28T23:10:27.827Z] res.append(await self.one_run(scenario, customization))
[task 2020-07-28T23:10:27.828Z] File "condprof/../condprof/runner.py", line 130, in one_run
[task 2020-07-28T23:10:27.828Z] ).run(not self.visible)
[task 2020-07-28T23:10:27.828Z] File "condprof/../condprof/creator.py", line 78, in run
[task 2020-07-28T23:10:27.828Z] metadata = await self.build_profile(device, headless)
[task 2020-07-28T23:10:27.828Z] File "condprof/../condprof/creator.py", line 153, in build_profile
[task 2020-07-28T23:10:27.828Z] async with get_session(geckodriver, firefox_instance) as session:
[task 2020-07-28T23:10:27.828Z] File "/builds/worker/fetches/condprofile/lib/python3.7/site-packages/arsenic/init.py", line 16, in aenter
[task 2020-07-28T23:10:27.828Z] self.session = await start_session(self.service, self.browser, self.bind)
[task 2020-07-28T23:10:27.828Z] File "/builds/worker/fetches/condprofile/lib/python3.7/site-packages/arsenic/init.py", line 29, in start_session
[task 2020-07-28T23:10:27.828Z] return await driver.new_session(browser, bind=bind)
[task 2020-07-28T23:10:27.828Z] File "/builds/worker/fetches/condprofile/lib/python3.7/site-packages/arsenic/webdriver.py", line 56, in new_session
[task 2020-07-28T23:10:27.828Z] err_resp["error"], err_resp.get("message", ""), original_response
[task 2020-07-28T23:10:27.828Z] arsenic.errors.SessionStartError: unknown error: Invalid Content-Type
[task 2020-07-28T23:10:27.847Z] Unclosed client session
[task 2020-07-28T23:10:27.847Z] client_session: <aiohttp.client.ClientSession object at 0x7fd76ddb9510>
[task 2020-07-28T23:10:27.848Z] Unclosed connector
[task 2020-07-28T23:10:27.848Z] connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7fd76de04ad0>, 768.07716192)]']
[task 2020-07-28T23:10:27.848Z] connector: <aiohttp.connector.TCPConnector object at 0x7fd76ddb95d0>
[fetches 2020-07-28T23:10:27.867Z] removing /builds/worker/fetches
[fetches 2020-07-28T23:10:27.937Z] finished

Comment 1

[Cosmin Sabou [:CosminS]]

James, could you please fix asap these condprof bustages on central? They look related to changes in Bug 1648964 https://hg.mozilla.org/mozilla-central/rev/9568347f186b5b3e999cf4784fe7d10b37294678

TH link: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception&revision=7b8c4f32cdde947c1b1f942905fbc5039b6d1c65&searchStr=condprof&group_state=expanded

Comment 2

[Tarek Ziadé (:tarek)]

is that the desiredCapability deprecation ?

this is the call:

https://github.com/HDE/arsenic/blob/master/src/arsenic/webdriver.py#L40-L44

Arsenic is still doing that old -style. I can patch it locally and see if that fixes, unless this is something else James?

Comment 3

[Tarek Ziadé (:tarek)]

oh no it's just the content type whitelisting. trying a fix..

Comment 4

[James Graham [:jgraham]]

Yeah, I suppose it's using text/plain or something as a content-type. We should prefer application/json if possible.

Comment 5

[Tarek Ziadé (:tarek)]

Attachment: Bug 1655869 - set the right content-type when calling geckodriver

This patch sets the right content-type when calling geckodriver,
in the arsenic client. Long term fix is to use a new version
of Arsenic with the fix. See https://github.com/HDE/arsenic/issues/85

Comment 6

[Tarek Ziadé (:tarek)]

try run https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=6cffa8b8a0dcddaa929cfe748eab44c08bedf77e

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b99ccc15ef01
set the right content-type when calling geckodriver r=Bebe

Comment 8

[Intermittent Failures Robot]

27 failures in 846 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 27

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 9
• android-hw-g5-7-0-arm7-api-16: 9
• macosx1014-64: 3
• windows2012-64: 3
• linux64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1655869&startday=2020-07-29&endday=2020-07-29&tree=all

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/b99ccc15ef01

Comment 10

[Intermittent Failures Robot]

38 failures in 4289 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #31 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 38

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 12
• android-hw-g5-7-0-arm7-api-16: 14
• linux64: 4
• macosx1014-64: 4
• windows2012-64: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1655869&startday=2020-07-27&endday=2020-08-02&tree=all