1032114 - TPS fails to decode Windows file paths in config file

Status: RESOLVED FIXED

(Testing Graveyard :: TPS, defect)

Description

[Cosmin Malutan, [:cosmin-malutan]]

While running tps testruns on a windows slave I've got the following error:
The exact log I've got was:
>21:11:36     config = json.loads(configcontent)
>21:11:36   File "C:\Python27\Lib\json\__init__.py", line 338, in loads
>21:11:36     return _default_decoder.decode(s)
>21:11:36   File "C:\Python27\Lib\json\decoder.py", line 365, in decode
>21:11:36     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>21:11:36   File "C:\Python27\Lib\json\decoder.py", line 381, in raw_decode
>21:11:36     obj, end = self.scan_once(s, idx)
This was because on config file we had escaped characters (backslash) "\" instead of slash ('/').
As a workaround I replaced the '\' with '//' in tps cli, but I guess this could be also fixed in create_venv.py script where we writhe the JSON file.
config = json.loads(configcontent.replace("\\", "//"))
https://github.com/mozilla/gecko-dev/blob/b1328768a61c298a0c78c6309f574f9c25ccf792/testing/tps/tps/cli.py#L87

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Since when does this problem occur? I mean what regressed that functionality?

Comment 2

[Cosmin Malutan, [:cosmin-malutan]]

Attachment: patch v1.0

I don't know if this regressed somewhere or if I had different settings when I created the create_venv.py script.
Best would be replace the backslashes before we write down the config file, as they escape characters. Also we don't have to check the os because only on windows we have backslashes in config file. I ran this on mac and windows and now I'm running on linux.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 8447990 [details] [diff] [review]
patch v1.0

Review of attachment 8447990 [details] [diff] [review]:
-----------------------------------------------------------------

I only call this a pretty bad workaround. What we actually should do here is to properly load and save the config via the json module. That will ensure that we will get proper paths. Given that we have to get the framework working, we need this immediately. Please file a follow-up bug for the real fix.

Also this is one more situation when a patch has not been tested. In this case the one from bug 1009004.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

https://hg.mozilla.org/mozilla-central/rev/606848e8adfc

Don't forget to set the status flags which versions are affected.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The patch works fine on Windows. I backported those to Aurora and Beta, so we get tests running on those branches.

https://hg.mozilla.org/releases/mozilla-aurora/rev/5d619b544f5a
https://hg.mozilla.org/releases/mozilla-beta/rev/4098e00458ca