1071012 - BuildReaderError: 'SharedLibrary' is not defined when running './mach build-docs'

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Andrew Halberstadt [:ahal]]

STR:
./mach build-docs

Result:
BuildReaderError: 
==============================
ERROR PROCESSING MOZBUILD FILE
==============================

The error occurred while processing the following file:

    /home/ahal/hg/mozilla-central/accessible/interfaces/ia2/moz.build

The error was triggered on line 7 of this file:

    SharedLibrary('IA2Marshal')

An error was encountered as part of executing the file itself. The error appears to be the fault of the script.

The error as reported by Python is:

    ["NameError: name 'SharedLibrary' is not defined\n"]


  File "/home/ahal/hg/mozilla-central/tools/docs/mach_commands.py", line 49, in build_docs
    for context in reader.walk_topsrcdir():
  File "/home/ahal/hg/mozilla-central/python/mozbuild/mozbuild/frontend/reader.py", line 803, in walk_topsrcdir
    filesystem_absolute=True, read_tiers=True):
  File "/home/ahal/hg/mozilla-central/python/mozbuild/mozbuild/frontend/reader.py", line 848, in read_mozbuild
    sys.exc_info()[2], sandbox_exec_error=se)

Comment 1

[(not currently active) Ted Mielczarek]

This was broken by bug 1041941.

Comment 2

[Gregory Szorc [:gps]]

I'm going to reopen this because generating docs from templates is hard and I want a quick win to make doc building work again.

Comment 3

[Gregory Szorc [:gps]]

Attachment: Extract Sphinx variables via AST reading

The in-tree Sphinx docs have been broken since bug 1041941 because
processing moz.build files outside their context doesn't work.
Specifically, templates aren't loaded (because this information usually
comes from a parent moz.build file). A new execution mode is needed.

I tried to implement a proper execution mode. However, I kept running
into walls. While we should strive for a proper execution mode, this can
be a follow-up, tracked in bug 1058359.

This patch implements extraction of Sphinx variables from ast walking.
It is extremely low-level and definitely a one-off. But it solves the
problem at hand: |mach build-docs| will work after this patch is
applied.

Reviewer notes: I feel a bit dirty for writing this code. I would really
prefer a proper solution that doesn't rely on AST. However, templates
made this more difficult. I know you (glandium) have more ideas for how
execution should work. I figured I'd leave implementing a proper
processing mode up to you.

Comment 4

[Mike Hommey [:glandium]]

Comment on attachment 8501208 [details] [diff] [review]
Extract Sphinx variables via AST reading

Review of attachment 8501208 [details] [diff] [review]:
-----------------------------------------------------------------

::: python/mozbuild/mozbuild/frontend/reader.py
@@ +737,5 @@
>          path = mozpath.join(self.config.topsrcdir, 'moz.build')
>          return self.read_mozbuild(path, self.config, read_tiers=True)
>  
> +    def all_mozbuild_paths(self):
> +        """Iterator over all available moz.build files."""

This code has not much that depends on the reader anymore (only self.config). It seems to me it would be better in a separate module.

@@ +777,5 @@
> +        def assigned_variable(node):
> +            # This is not correct, but we don't care yet.
> +            if hasattr(node, 'targets'):
> +                if len(node.targets) > 1:
> +                    return None, None

maybe raise an exception when one of the targets is relevant?

@@ +783,5 @@
> +                target = node.targets[0]
> +            else:
> +                target = node.target
> +
> +            if isinstance(target, ast.Subscript):

Could you add a comment specifying what kind of syntax those tests cover?

Comment 5

[Gregory Szorc [:gps]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e93c40d4344f

With added comments.

Comment 6

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/e93c40d4344f