Closed Bug 1099345 Opened 10 years ago Closed 10 years ago

Intermittent "make[5]: *** [MediaCodec.o] Error 1" from "MediaCodec.cpp:17:5: error: 'MediaCodec' has not been declared" (and similar) errors

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox34 wontfix, firefox35 fixed, firefox36 fixed, firefox-esr31 unaffected)

Status:
RESOLVED FIXED
Milestone:
mozilla36
Tracking Flags:
Tracking Status
firefox34 --- wontfix
firefox35 --- fixed
firefox36 --- fixed
firefox-esr31 --- unaffected

People

(Reporter: RyanVM, Assigned: snorp)

References

Details

(Keywords: intermittent-failure)

Attachments

(2 files, 1 obsolete file)

Add more descriptive debugging output to the Android SDK code generator
3.60 KB, patch
nalexander
: review+
Details | Diff | Splinter Review
Do not run the Android SDK code generator tool in parallel
1.69 KB, patch
nalexander
: review+
lsblakk
: approval-mozilla-beta+
Details | Diff | Splinter Review 
Some sort of race? This was on a clobber build in case that matters.

/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings/MediaCodec.cpp: In function 'void mozilla::widget::android::sdk::InitMediaCodecStubs(JNIEnv*)':
/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings/MediaCodec.cpp:17:5: error: 'MediaCodec' has not been declared
/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings/MediaCodec.cpp:18:5: error: 'BufferInfo' has not been declared
/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings/MediaCodec.cpp:19:5: error: 'MediaFormat' has not been declared
make[5]: *** [MediaCodec.o] Error 1
make[5]: Leaving directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings'
make[4]: *** [widget/android/bindings/target] Error 2
sorry ignore the last comments, TH copy and paste fail - should be belong to backout fixed by 5ee39194ac67
Nick, do you have any idea what's going on here? It looks like something is nuking all the headers, but the generation isn't happening because the cpp still exists.
Flags: needinfo?(nalexander)
Yeah, this looks like an unexpected race condition.  I think I just did this incorrectly; looks like www.gnu.org/software/make/manual/make.html#Pattern-Examples is the right way to achieve this goal.  I'll dig in.
Assignee: nobody → nalexander
Status: NEW → ASSIGNED
Flags: needinfo?(nalexander)
https://hg.mozilla.org/mozilla-central/rev/640564ff2b2a
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla36
Reopening, since it looks like this isn't fixed. (per comment 30)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Actually, reading the comment 30 log further shows that this looks like a legitimate failure in the generator.  I know that wesj just revved the Android SDK; it's possible the new tooltool package is missing something we depend on.  wesj, snorp?

make[5]: Entering directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings'
/usr/bin/java -Dcom.android.tools.lint.bindir='/builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/tools' -classpath ../../../build/annotationProcessors/annotationProcessors.jar:/builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/tools/lib/lint.jar:/builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/tools/lib/lint-checks.jar org.mozilla.gecko.annotationProcessors.SDKProcessor /builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/platforms/android-21/android.jar /builds/slave/m-in-and-000000000000000000000/build/widget/android/bindings/MediaCodec-classes.txt /builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings MediaCodec 16
/usr/bin/java -Dcom.android.tools.lint.bindir='/builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/tools' -classpath ../../../build/annotationProcessors/annotationProcessors.jar:/builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/tools/lib/lint.jar:/builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/tools/lib/lint-checks.jar org.mozilla.gecko.annotationProcessors.SDKProcessor /builds/slave/m-in-and-000000000000000000000/build/android-sdk-linux/platforms/android-21/android.jar /builds/slave/m-in-and-000000000000000000000/build/widget/android/bindings/SurfaceTexture-classes.txt /builds/slave/m-in-and-000000000000000000000/build/obj-firefox/widget/android/bindings SurfaceTexture 16
Processing platform bindings...
Processing platform bindings...
make[5]: Leaving directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/ipc/ipdl'
make[5]: Leaving directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/dom/bindings'
make[5]: Entering directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/dom/bindings'
make[5]: Nothing to be done for `export'.
make[5]: Leaving directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/dom/bindings'
make[5]: Entering directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/dom/bindings/test'
make[5]: Nothing to be done for `export'.
make[5]: Leaving directory `/builds/slave/m-in-and-000000000000000000000/build/obj-firefox/dom/bindings/test'

The API database file /home/mock_mozilla/.android/cache/api-versions-6-21.bin does not exist
Looking up: android.media.MediaCodec
Failed to generate class android.media.MediaCodec: java.lang.NullPointerException
Looking up: android.media.MediaCodec$BufferInfo
Failed to generate class android.media.MediaCodec$BufferInfo: java.lang.NullPointerException
Looking up: android.media.MediaFormat
Failed to generate class android.media.MediaFormat: java.lang.NullPointerException
SDK processing complete in 10ms
../../../config/nsinstall -R -m 644 'MediaCodec.h' '../../../dist/include'
Flags: needinfo?(wjohnston)
Flags: needinfo?(snorp)
We need some more information about what's going wrong here.
Assignee: nalexander → snorp
Flags: needinfo?(snorp)
Attachment #8529107 - Flags: review?(nalexander)
Comment on attachment 8529107 [details] [diff] [review]
Add more descriptive debugging output to the Android SDK code generator

Review of attachment 8529107 [details] [diff] [review]:
-----------------------------------------------------------------

::: build/annotationProcessors/SDKProcessor.java
@@ +113,5 @@
>                                stubInitializer,
>                                implementationFile,
>                                headerFile);
>              } catch (Exception e) {
> +                System.err.println("Failed to generate class " + className + ": " + e);

Why catch at all?  Why not just go out in flames and kill the build immediately?  If you return non-0, the build will fail.  You might also want to FORCE the relevant files so that we find the problem faster.
Attachment #8529107 - Flags: review?(nalexander) → review+
(In reply to Nick Alexander :nalexander from comment #38)
> Comment on attachment 8529107 [details] [diff] [review]
> Add more descriptive debugging output to the Android SDK code generator
> 
> Review of attachment 8529107 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: build/annotationProcessors/SDKProcessor.java
> @@ +113,5 @@
> >                                stubInitializer,
> >                                implementationFile,
> >                                headerFile);
> >              } catch (Exception e) {
> > +                System.err.println("Failed to generate class " + className + ": " + e);
> 
> Why catch at all?  Why not just go out in flames and kill the build
> immediately?

That's a good point. Lets do that instead.

>  If you return non-0, the build will fail.  You might also want
> to FORCE the relevant files so that we find the problem faster.

What is FORCE?
Flags: needinfo?(wjohnston) → needinfo?(nalexander)
Comment on attachment 8529153 [details] [diff] [review]
Add more descriptive debugging output to the Android SDK code generator

Review of attachment 8529153 [details] [diff] [review]:
-----------------------------------------------------------------

Ship it.

::: build/annotationProcessors/SDKProcessor.java
@@ +110,2 @@
>  
> +            generateClass(Class.forName(className, true, loader),

Why do you not use |c|?
Attachment #8529153 - Flags: review?(nalexander) → review+
(In reply to Nick Alexander :nalexander from comment #41)
> Comment on attachment 8529153 [details] [diff] [review]
> Add more descriptive debugging output to the Android SDK code generator
> 
> Review of attachment 8529153 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Ship it.
> 
> ::: build/annotationProcessors/SDKProcessor.java
> @@ +110,2 @@
> >  
> > +            generateClass(Class.forName(className, true, loader),
> 
> Why do you not use |c|?

LOL. It looks like I intended to, but maybe it got hosed in a refactor? I'll fix.
So the lint stuff is complaining about things in ~/.android being corrupt. I wonder if we have an old one or something? Can we nuke that in the setup on the builders? Failure output below:

Failure reading binary cache file /home/mock_mozilla/.android/cache/api-versions-6-21.bin
Please delete the file and restart the IDE/lint: /home/mock_mozilla/.android/cache/api-versions-6-21.bin
java.io.FileNotFoundException: /home/mock_mozilla/.android/cache/api-versions-6-21.bin (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
at com.google.common.io.Files.map(Files.java:857)
at com.google.common.io.Files.map(Files.java:826)
at com.android.tools.lint.checks.ApiLookup.readData(ApiLookup.java:301)
at com.android.tools.lint.checks.ApiLookup.<init>(ApiLookup.java:256)
at com.android.tools.lint.checks.ApiLookup.get(ApiLookup.java:219)
at com.android.tools.lint.checks.ApiLookup.get(ApiLookup.java:129)
at org.mozilla.gecko.annotationProcessors.SDKProcessor.main(SDKProcessor.java:62)
Flags: needinfo?(nalexander)
I've found that the problem is actually caused by running two instances of the generator in parallel. They both try to create this cache, and boom. We need to serialize this.
Comment on attachment 8529345 [details] [diff] [review]
Do not run the Android SDK code generator tool in parallel

Review of attachment 8529345 [details] [diff] [review]:
-----------------------------------------------------------------

::: widget/android/bindings/Makefile.in
@@ +2,5 @@
>  # License, v. 2.0. If a copy of the MPL was not distributed with this
>  # file, You can obtain one at http://mozilla.org/MPL/2.0/.
>  
> +# Bug 1099345 - The SDK's lint code (used by the code generator) does not enjoy
> +# concurrent access a cache that it generates.

... access *to* a cache that it generates.
Attachment #8529345 - Flags: review?(nalexander) → review+
Comment on attachment 8529345 [details] [diff] [review]
Do not run the Android SDK code generator tool in parallel

Approval Request Comment
[Feature/regressing bug #]: 1086693
[User impact if declined]: intermittent build failures
[Describe test coverage new/current, TBPL]: it's building in 36+
[Risks and why]: no risk
[String/UUID change made/needed]: none
Attachment #8529345 - Flags: approval-mozilla-beta?
Attachment #8529345 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
https://hg.mozilla.org/releases/mozilla-beta/rev/de387e9cb38e
https://hg.mozilla.org/releases/mozilla-beta/rev/c8504ff4aa48
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
status-firefox35: affected → fixed
Keywords: leave-open
Resolution: --- → FIXED
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: