1714688 - Remove `if PY2`/`if PY3` logic now that we never use Python 2 with Mach

Status: RESOLVED FIXED

(Firefox Build System :: General, task, P3)

Description

[Mitchell Hentges [:mhentges] 🦀]

When run with ./mach, if PY2 will always be False, and if PY3 will always be True. Let's simplify accordingly.

Note: some Python scripts may be run via mozharness, and such tasks may still be using Python 2. When that's the case, report (or find) a bug to run the task with Python 3 instead, resolve that first, then come back to removing PY2/PY3 usages.

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

Note that, in all the cases where PY2 and PY3 come from six, they are able to be automatically handled by pyupgrade.

Comment 2

[Surajeet Das]

Can you tell me where should I begin @mhentges ?

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Sure thing Surajeet:
First, you can see the list of if PY2/if PY3 instances on SearchFox here. Note that anything under third_party/python and testing/web-platform/tests/ are 3rd-party code and shouldn't be changed in your patches. (EDIT as mentioned by ahal, tooltool.py is 3rd-party code too, and also shouldn't be updated).

Next, I'd recommend breaking down your patches to be specific to components: e.g. update testing/mozharness in one patch, and testing/mozbase in another, and python/mozbuild in another, and so on. Each time you complete a patch, send it in, and I can start reviewing.
Does that make sense?

Comment 4

[Surajeet Das]

Yes, it does @mhentges :)
Let me start ,I will let you know if I have any doubts on the way :)
Thank you !

Comment 5

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2' logic from mozglue r=mhentges

Comment 6

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2'/'if PY3' logic from python/mozbuild r=mhentges

Python2 is no longer used in Mozbuild.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1d869c7669e6
Removed 'if PY2' logic from mozglue r=mhentges

Comment 8

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/1d869c7669e6

Comment 9

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2' logic from python/mozrelease r=mhentges

Python2 is no longer used in Mozrelease

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/15c6a16f5828
Removed 'if PY2'/'if PY3' logic from python/mozbuild r=mhentges

Comment 11

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2' logic from js/src r=mhentges

Python 2 is no longer used in js/src

Comment 12

[Natalia Csoregi [:nataliaCs]]

Backed out for docker images bustage

• backout: https://hg.mozilla.org/integration/autoland/rev/d9ce7bf4cebb533721cb204ac41a87a182bef766

• push: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&revision=15c6a16f5828fcab1634dd647c4650170dfdc25f&searchStr=Docker%2CImages%2Copt&selectedTaskRun=HZmKvugESaunrgeOXiI0WQ.0

• failure log: https://treeherder.mozilla.org/logviewer?job_id=342728634&repo=autoland&lineNumber=1925

${PIP_PATH} install mercurial-4.8.1.tar.gz
else
echo "Do not know how to install Mercurial on this OS"
exit 1
fi
Traceback (most recent call last):
File "/tmp/tooltool.py", line 66, in <module>
import urllib.request as urllib2
ImportError: No module named request
error building image: error building stage: failed to execute command: waiting for process to exit: exit status 1
Error: Could not build image.
[taskcluster 2021-06-14 17:31:10.673Z] === Task Finished ===
[taskcluster 2021-06-14 17:31:10.750Z] Artifact "public/image.tar.zst" not found at "/workspace/image.tar.zst"
[taskcluster 2021-06-14 17:31:10.905Z] Unsuccessful task run with exit code: 1 completed in 31.675 seconds

Comment 13

[Justin Wood (:Callek)]

(In reply to Natalia Csoregi [:nataliaCs] from comment #12)

Backed out for docker images bustage

Looks like it was indeed being called with py2:
https://searchfox.org/mozilla-central/source/taskcluster/docker/decision/system-setup.sh#21
https://searchfox.org/mozilla-central/source/taskcluster/docker/periodic-updates/setup.sh#8
https://searchfox.org/mozilla-central/source/taskcluster/docker/recipes/common.sh#6

Might also have py2 issues with:
https://searchfox.org/mozilla-central/source/taskcluster/docker/lint/system-setup.sh#38
https://searchfox.org/mozilla-central/source/taskcluster/docker/update-verify/system-setup.sh#34

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9f0fbb143172
Removed 'if PY2' logic from js/src r=mhentges

Comment 15

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2' logic from python/mozrelease r=mhentges

Python2 is no longer used in Mozrelease

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/177ac92fb734
Fix lint black failure r=fix CLOSED TREE

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8c07e39f8dbd
Removed 'if PY2' logic from python/mozrelease r=mhentges

Comment 18

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/9f0fbb143172
https://hg.mozilla.org/mozilla-central/rev/177ac92fb734

Comment 19

[Surajeet Das]

https://phabricator.services.mozilla.com/D117583
Its closed but still not reflected here : https://searchfox.org/mozilla-central/source/python/mozrelease/mozrelease/versions.py

Comment 20

[Surajeet Das]

Removed the changes for tooltool.py

Comment 21

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/8c07e39f8dbd

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ceac5a7590de
Removed 'if PY2'/'if PY3' logic from python/mozbuild r=mhentges

Comment 23

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY3' logic from testing/condprofile r=mhentges

Python2 is no longer used in testing/condprofile

Comment 24

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/ceac5a7590de

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7cd82eea9af2
Removed 'if PY3' logic from testing/condprofile r=mhentges

Comment 26

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2' logic from testing/marionette r=mhentges

Python2 is no longer used in testing/marionette

Comment 27

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY3' logic from testing/mozbase r=mhentges

Python2 is no longer used in testing/mozbase

Comment 28

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/74fd8bb17de0
Removed 'if PY3' logic from testing/mozbase r=mhentges

Comment 29

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY3' logic from testing/tools r=mhentges

Python2 is no longer used in testing/tools

Comment 30

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2' logic from testing/mozharness r=mhentges

Python2 is no longer used in testing/mozharness

Comment 31

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/52c33043778a
Removed 'if PY2' logic from testing/marionette r=mhentges,webdriver-reviewers,jgraham

Comment 32

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/7cd82eea9af2
https://hg.mozilla.org/mozilla-central/rev/74fd8bb17de0
https://hg.mozilla.org/mozilla-central/rev/52c33043778a

Comment 33

[Natalia Csoregi [:nataliaCs]]

Backed out changeset 74fd8bb17de0 (bug 1714688) for causing Bug 1716860.

backout: https://hg.mozilla.org/integration/autoland/rev/5bb44754f8119fefe1206bd0bf5e614b6452288d

Comment 34

[Surajeet Das]

Attachment: Bug 1714688 - Removed unnecessary postfix '_3' from 'websocketprocessbridge_requirements_3.txt' r=mhentges

The file 'websocketprocessbridge_requirements_3.txt' has been renamed to 'websocketprocessbridge_requirements.txt' as Python2 is no longer used.

Comment 35

[Surajeet Das]

Attachment: Bug 1714688 - Removed 'if PY2' logic from testing/mozharness r=mhentges

Python2 is no longer used in testing/mozharness. Hence,all the instances of file 'websocketprocessbridge_requirements.txt' are removed as it is obsolete now.

Comment 36

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backout merged: https://hg.mozilla.org/mozilla-central/rev/5bb44754f811

Comment 37

[Surajeet Das]

Attachment: Bug 1714688 - Removed unnecessary postfix '_3' from 'websocketprocessbridge_requirements_3.txt' r=mhentges

Now that only the file 'websocketprocessbridge_requirements.txt' is used, so the qualifier '_3' is redundant.

Comment 38

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/941f6c6c51e3
Removed 'if PY2' logic from testing/mozharness r=mhentges
https://hg.mozilla.org/integration/autoland/rev/997f00815e6b
Removed unnecessary postfix '_3' from 'websocketprocessbridge_requirements_3.txt' r=mhentges

Comment 39

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/941f6c6c51e3
https://hg.mozilla.org/mozilla-central/rev/997f00815e6b

Comment 40

[Mitchell Hentges [:mhentges] 🦀]

Let's mark this as fixed since all remaining if PY2/PY3 code is in:

• third_party/python/
• tooltool
• testing/web-platform/tests/
• Mozilla tests that may still be using Python 2.

Great work Surajeet in making the possible cleanups! Big 🎉 :)

Comment 41

[Natalia Csoregi [:nataliaCs]]

Backed out for causing Bug 1717578

backout: https://hg.mozilla.org/integration/autoland/rev/9f610f245248d3ed84e470279b6e42bcfada2efa

Comment 42

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backout merged: https://hg.mozilla.org/mozilla-central/rev/9f610f245248

Comment 43

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/050c73c280e8
Removed 'if PY3' logic from testing/tools r=mhentges

Comment 44

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/050c73c280e8