1752381 - substitute-local-geckoview.gradle fails (because it parses mach's python warning message?)

Status: RESOLVED FIXED

(GeckoView :: General, defect, P3)

Description

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

When I try to do a fenix -> local GV substitution build using fenix/local.properties (i.e. adding dependencySubstitutions.geckoviewTopsrcdir=/Users/mcomella/dev/mozilla-central), I got the following error message:

FAILURE: Build failed with an exception.

* Where:
Script '/Users/mcomella/dev/mozilla-central/substitute-local-geckoview.gradle' line: 66

* What went wrong:
A problem occurred evaluating script.
> Unable to determine the current character, it is not a string, number, array, or object
  
  The current character read is '/' with an int value of 47
  Unable to determine the current character, it is not a string, number, array, or object
  line number 1
  index number 0
  /usr/local/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  ^

In the current mc, every mach command starts with a setup tools warning so I think substitute-local-geckoview is parsing that output instead of something else.

Comment 1

[Nick Alexander :nalexander [he/him]]

I think this is https://searchfox.org/mozilla-central/rev/66e59131c1c76fe486424dc37f0a8a399ca874d4/settings.gradle#7 failing to handle the warnings, but I don't see quite how. Will test locally.

Comment 2

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

I ran with --stacktrace:

        at org.apache.groovy.json.internal.JsonParserCharArray.decodeValueInternal(JsonParserCharArray.java:202)
        at org.apache.groovy.json.internal.JsonParserCharArray.decodeValue(JsonParserCharArray.java:153)
        at org.apache.groovy.json.internal.JsonParserCharArray.decodeFromChars(JsonParserCharArray.java:43)
        at org.apache.groovy.json.internal.JsonParserCharArray.parse(JsonParserCharArray.java:380)
        at org.apache.groovy.json.internal.BaseJsonParser.parse(BaseJsonParser.java:113)
        at substitute_local_geckoview_a2bofz12hl1ja4poc43irat2n.loadMozconfig(/Users/mcomella/dev/mozilla-central/substitute-local-geckoview.gradle:66)
        at substitute_local_geckoview_a2bofz12hl1ja4poc43irat2n.run(/Users/mcomella/dev/mozilla-central/substitute-local-geckoview.gradle:100)
        at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:91)
        ... 164 more

which points to https://searchfox.org/mozilla-central/rev/b70bc09685763c44a8e56e4e04cb741fa020701a/substitute-local-geckoview.gradle#66 as the problem.

Comment 3

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Got it:

    def standardOutput = new ByteArrayOutputStream()
    proc.consumeProcessOutput(standardOutput, standardOutput)
    // ...

    def slurper = new JsonSlurper()
    def mozconfig = slurper.parseText(standardOutput.toString())

We put both stdout and stderr into the same stream, which we then try to parse as json. I worked around this issue by changing the one line to this:

proc.consumeProcessOutput(standardOutput, null)

But think that's a good long term solution. :D

Comment 4

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Attachment: Bug 1752381 - only parse stdout in substitute-local-geckoview. r=nalexander

Root cause: in the current tree, mach commands in mozilla-central output a
warning to stderr. The substitute-local-geckoview script was calling a mach
command and combining the stdout and stderr streams of the process, parsing
it as JSON. This warning is not JSON so the script crashed.

The crashing code was copied from settings.gradle:
https://searchfox.org/mozilla-central/rev/b70bc09685763c44a8e56e4e04cb741fa020701a/settings.gradle#26

The code in settings.gradle does an intuitive thing - capture stderr separately
but only print it on subprocess non-zero exit value - so we also copy that
solution. I'm not sure what an appropriate place to store code shared between
these two files would be so I didn't try to deduplicate it.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/113560f6f02b
only parse stdout in substitute-local-geckoview. r=nalexander

Comment 6

[Cristina Cozmuta (:CrissCozmuta)]

https://hg.mozilla.org/mozilla-central/rev/113560f6f02b