Closed Bug 917562 Opened 7 years ago Closed 6 years ago

Intermittent androidx86 test_media_queries.html,test_value_storage.html,test_property_syntax_errors.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant] on UCOMISS instruction due to emulator misinterpreting amount of memory needed

Categories

(Testing :: General, defect, critical)

x86
Android
defect
Not set
critical

Tracking

(firefox26 unaffected, firefox27 unaffected, firefox28 fixed, firefox-esr24 unaffected)

RESOLVED FIXED
mozilla28
Tracking Status
firefox26 --- unaffected
firefox27 --- unaffected
firefox28 --- fixed
firefox-esr24 --- unaffected

People

(Reporter: philor, Assigned: froydnj)

References

Details

(Keywords: crash, intermittent-failure, Whiteboard: Patch in comment 26, status in comment 60)

https://tbpl.mozilla.org/php/getParsedLog.php?id=27987627&tree=Fx-Team
Android 4.2 x86 Emulator fx-team opt test androidx86-set-2 on 2013-09-17 11:27:31 PDT for push 7a710c502b49
slave: talos-linux64-ix-003

11:43:36     INFO - 24404 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | expression device-width: -0cm should be parseable
11:43:36     INFO - 
11:43:36     INFO - INFO | automation.py | Application ran for: 0:08:26.986738
11:43:36     INFO - INFO | zombiecheck | Reading PID log: /tmp/tmpGjfqV4pidlog
11:43:36     INFO - mozcrash INFO | Downloading symbols from: http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/fx-team-android-x86/1379438713/fennec-27.0a1.en-US.android-i386.crashreporter-symbols.zip
11:43:36     INFO - /data/anr/traces.txt not found
11:43:36     INFO - PROCESS-CRASH | /tests/layout/style/test/test_media_queries.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant]
11:43:36     INFO - Crash dump filename: /tmp/tmpakSqlD/3302882b-5dae-730f-64fe5107-044f477d.dmp
11:43:36     INFO - Operating system: Android
11:43:36     INFO -                   0.0.0 Linux 2.6.29 #2 PREEMPT Thu Nov 15 13:38:07 CST 2012 i686 generic_x86/sdk_x86/generic_x86:4.2/JOP40C/eng.android-build.20121231.103448:eng/test-keys
11:43:36     INFO - CPU: x86
11:43:36     INFO -      GenuineIntel family 6 model 3 stepping 3
11:43:36     INFO -      1 CPU
11:43:36     INFO - 
11:43:36     INFO - Crash reason:  SIGSEGV
11:43:36     INFO - Crash address: 0x96300000
11:43:36     INFO - 
11:43:36     INFO - Thread 26 (crashed)
11:43:36     INFO -  0  libxul.so!CSSParserImpl::ParseNonNegativeVariant [nsCSSParser.cpp:7a710c502b49 : 5010 + 0x3]
11:43:36     INFO -     eip = 0xa0bb9933   esp = 0xa8af5e10   ebp = 0xa8af5e28   ebx = 0xa2b6dfd4
11:43:36     INFO -     esi = 0x9ac4f000   edi = 0x962ffff0   eax = 0x000002bf   ecx = 0x002c0000
11:43:36     INFO -     edx = 0x00000387   efl = 0x00200283
11:43:36     INFO -     Found by: given as instruction pointer in context
11:43:36     INFO -  1  libxul.so!CSSParserImpl::ParseMediaQueryExpression [nsCSSParser.cpp:7a710c502b49 : 1899 + 0xf]
11:43:36     INFO -     eip = 0xa0bbd7cd   esp = 0xa8af5e30   ebp = 0xa8af5e88   ebx = 0xa2b6dfd4
11:43:36     INFO -     esi = 0x9ac4f000   edi = 0x962fffe8
11:43:36     INFO -     Found by: call frame info
11:43:36     INFO -  2  libxul.so!CSSParserImpl::GatherMedia [nsCSSParser.cpp:7a710c502b49 : 1765 + 0x9]
11:43:36     INFO -     eip = 0xa0bc4125   esp = 0xa8af5e90   ebp = 0xa8af5ed8   ebx = 0xa2b6dfd4
11:43:36     INFO -     esi = 0x94296e40   edi = 0xa2063e7b
11:43:36     INFO -     Found by: call frame info
11:43:36     INFO -  3  libxul.so!nsCSSParser::ParseMediaList(nsAString_internal const&, nsIURI*, unsigned int, nsMediaList*, bool) [nsCSSParser.cpp:7a710c502b49 : 1196 + 0x10]
11:43:36     INFO -     eip = 0xa0bc4706   esp = 0xa8af5ee0   ebp = 0xa8af6018   ebx = 0xa2b6dfd4
11:43:36     INFO -     esi = 0x9ac4f000   edi = 0xa8af5f04
11:43:36     INFO -     Found by: call frame info
11:43:36     INFO -  4  libxul.so!mozilla::css::Loader::PrepareSheet(nsCSSStyleSheet*, nsAString_internal const&, nsAString_internal const&, nsMediaList*, mozilla::dom::Element*, bool) [Loader.cpp:7a710c502b49 : 1260 + 0x14]
11:43:36     INFO -     eip = 0xa0ba8ec7   esp = 0xa8af6020   ebp = 0xa8af6078   ebx = 0xa2b6dfd4
11:43:36     INFO -     esi = 0x962fffa0   edi = 0xa8af605c
11:43:36     INFO -     Found by: call frame info
11:43:36     INFO -  5  libxul.so!mozilla::css::Loader::LoadInlineStyle(nsIContent*, nsAString_internal const&, unsigned int, nsAString_internal const&, nsAString_internal const&, mozilla::dom::Element*, nsICSSLoaderObserver*, bool*, bool*) [Loader.cpp:7a710c502b49 : 1882 + 0x1b]
11:43:36     INFO -     eip = 0xa0bac013   esp = 0xa8af6080   ebp = 0xa8af60d8   ebx = 0xa2b6dfd4
11:43:36     INFO -     esi = 0x971de2b0   edi = 0x00000000
11:43:36     INFO -     Found by: call frame info
11:43:36     INFO -  6  libxul.so!nsStyleLinkElement::DoUpdateStyleSheet(nsIDocument*, nsICSSLoaderObserver*, bool*, bool*, bool) [nsStyleLinkElement.cpp:7a710c502b49 : 371 + 0x11]
11:43:36     INFO -     eip = 0xa0d047c0   esp = 0xa8af60e0   ebp = 0xa8af63b8   ebx = 0xa2b6dfd4
11:43:36     INFO -     esi = 0x9614a92c   edi = 0x00000000
Is the compiler responsible known to be competent at compiling?
android-ndk/toolchains/x86-4.7/prebuilt/linux-x86_64/bin/i686-linux-android-gcc?

Well, we've been testing the results of our builds on it for... several hours now. Maybe twenty. Longer if you count https://tbpl.mozilla.org/?tree=Ash&jobname=androidx86, which I don't.
https://tbpl.mozilla.org/php/getParsedLog.php?id=28160553&tree=Mozilla-Inbound
Summary: Intermittent androidx86 test_media_queries.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant] → Intermittent androidx86 test_media_queries.html,test_value_storage.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant]
https://tbpl.mozilla.org/php/getParsedLog.php?id=28349540&tree=Mozilla-Inbound
Summary: Intermittent androidx86 test_media_queries.html,test_value_storage.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant] → Intermittent androidx86 test_media_queries.html,test_value_storage.html,test_property_syntax_errors.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant]
So every single crash so far is a SEGV with a crash address ending in five 0s.  (In one case, six 0s.)  And in every case the EDI register value ends in ffff0, exactly 0x10 below the crash address.

At this point my suspicion is that the mozilla::IsNaN call inside GetFloatValue is compiling (incorrectly, IMO) into something that reads a few bytes too far or some similar problem, and that we're crashing intermittently when the double's address is right before a page boundary.

(I think nsCSSValue should only be 8 bytes here.  And the mozilla::IsNaN() call is a little interesting in that it's passing a float (pulled out of a tagged union) to mozilla::IsNaN(double), so the union plus float->double conversion plus going into another union inside IsNan could, perhaps, confuse a poor compiler.)

I didn't look at any disassembly, though.  Somebdy probably should.
Anybody up to checking the disassembly here to see what the compiler is doing?
Flags: needinfo?(nfroyd)
Flags: needinfo?(jwalden+bmo)
The disassembly (opt build, so the MOZ_ASSERTs being referenced in comment 11 get compiled out):

005cd8e2 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi>:
  5cd8e2:	55                   	push   %ebp
  5cd8e3:	89 e5                	mov    %esp,%ebp
  5cd8e5:	57                   	push   %edi
  5cd8e6:	89 d7                	mov    %edx,%edi
  5cd8e8:	56                   	push   %esi
  5cd8e9:	89 c6                	mov    %eax,%esi
  5cd8eb:	53                   	push   %ebx
  5cd8ec:	8d 64 24 ec          	lea    -0x14(%esp),%esp
  5cd8f0:	ff 75 0c             	pushl  0xc(%ebp)
  5cd8f3:	ff 75 08             	pushl  0x8(%ebp)
  5cd8f6:	e8 fd fb ff ff       	call   5cd4f8 <_ZN12_GLOBAL__N_113CSSParserImpl12ParseVariantER10nsCSSValueiPKi>
  5cd8fb:	83 c4 10             	add    $0x10,%esp
  5cd8fe:	84 c0                	test   %al,%al
  5cd900:	74 2a                	je     5cd92c <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x4a>
  5cd902:	8b 17                	mov    (%edi),%edx
  5cd904:	83 fa 5b             	cmp    $0x5b,%edx
  5cd907:	74 27                	je     5cd930 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x4e>
  5cd909:	8d 82 38 ff ff ff    	lea    -0xc8(%edx),%eax
  5cd90f:	3d c1 02 00 00       	cmp    $0x2c1,%eax
  5cd914:	76 1a                	jbe    5cd930 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x4e>
  5cd916:	eb 25                	jmp    5cd93d <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x5b>
  5cd918:	83 fa 46             	cmp    $0x46,%edx
  5cd91b:	b0 01                	mov    $0x1,%al
  5cd91d:	75 25                	jne    5cd944 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x62>
  5cd91f:	83 7f 04 00          	cmpl   $0x0,0x4(%edi)
  5cd923:	79 1f                	jns    5cd944 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x62>
  5cd925:	80 8e cc 00 00 00 01 	orb    $0x1,0xcc(%esi)
  5cd92c:	31 c0                	xor    %eax,%eax
  5cd92e:	eb 14                	jmp    5cd944 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x62>
  5cd930:	0f 57 c0             	xorps  %xmm0,%xmm0
  5cd933:	0f 2e 47 04          	ucomiss 0x4(%edi),%xmm0
  5cd937:	77 ec                	ja     5cd925 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x43>
  5cd939:	b0 01                	mov    $0x1,%al
  5cd93b:	eb 07                	jmp    5cd944 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x62>
  5cd93d:	83 fa 5a             	cmp    $0x5a,%edx
  5cd940:	75 d6                	jne    5cd918 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x36>
  5cd942:	eb ec                	jmp    5cd930 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x4e>
  5cd944:	8d 65 f4             	lea    -0xc(%ebp),%esp
  5cd947:	5b                   	pop    %ebx
  5cd948:	5e                   	pop    %esi
  5cd949:	5f                   	pop    %edi
  5cd94a:	5d                   	pop    %ebp
  5cd94b:	c3                   	ret    

I don't see anything too unusual and I don't see anyplace we'd be referencing 16(%edi)...
Flags: needinfo?(nfroyd)
Can we not get proper offsets into the disassembly, so we know what instruction it might be crashing at?
Oops, sorry, missed that it was an opt build and the code I was fingering was in a MOZ_ASSERT.

(In reply to Nathan Froyd (:froydnj) from comment #14)
> Can we not get proper offsets into the disassembly, so we know what
> instruction it might be crashing at?

You mean bug 466026?
Flags: needinfo?(jwalden+bmo)
(In reply to David Baron [:dbaron] (needinfo? me) from comment #15)
> (In reply to Nathan Froyd (:froydnj) from comment #14)
> > Can we not get proper offsets into the disassembly, so we know what
> > instruction it might be crashing at?
> 
> You mean bug 466026?

Yes, exactly that bug.  Guess we'll have to wait a while.
Armed with the knowledge about how things are formatted in the dump file, and cross-referencing with the symbols provided by breakpad, the crashing instruction is indeed:

  5cd933:	0f 2e 47 04          	ucomiss 0x4(%edi),%xmm0

(5cd930 belongs to line 5010; +0x3 from there gets us to the crashing instruction)

which seems really strange, as to get there, we'd need to come through this path:

  5cd902:	8b 17                	mov    (%edi),%edx
  5cd904:	83 fa 5b             	cmp    $0x5b,%edx
  5cd907:	74 27                	je     5cd930 <_ZN12_GLOBAL__N_113CSSParserImpl23ParseNonNegativeVariantER10nsCSSValueiPKi+0x4e>
  5cd909:	8d 82 38 ff ff ff    	lea    -0xc8(%edx),%eax
  5cd90f:	3d c1 02 00 00       	cmp    $0x2c1,%eax
  5cd914:	76 1a                	jbe    5cd930 

which means that we've already dereferenced %edi; dereferencing 4(%edi) oughtn't to be a problem.

David's suspicion about page boundaries in comment 11 sounds plausible, especially since this is all being run on a emulator.  I would not be surprised to find that the emulator is doing some eager decoding of SSE instructions with memory operands, only to find that the 16-byte quantity 4(%edi) straddles a page boundary in the crashing cases.  Of course, that's incorrect, since UCOMISS only references 32-bit memory operands, not the 128-bit memory operands that most SSE instructions use.
That seems pretty convincing, and the line number and disassembly certainly seem to match.
Summary: Intermittent androidx86 test_media_queries.html,test_value_storage.html,test_property_syntax_errors.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant] → Intermittent androidx86 test_media_queries.html,test_value_storage.html,test_property_syntax_errors.html | application crashed [@ CSSParserImpl::ParseNonNegativeVariant] on UCOMISS instruction due to emulator misinterpreting amount of memory needed
Do you know where Android-x86 emulator bugs should go?
Flags: needinfo?(gbrown)
(In reply to David Baron [:dbaron] (needinfo? me) from comment #19)
> Do you know where Android-x86 emulator bugs should go?

Maybe http://source.android.com/source/report-bugs.html?

:blassey -- any thoughts?
Flags: needinfo?(gbrown) → needinfo?(blassey.bugs)
Running a simple testcase through qemu shows that it's loading too much data for ucomiss.  Fixing it shouldn't be too hard.
(In reply to Geoff Brown [:gbrown] (PTO Sep 26 - summit) from comment #20)
> (In reply to David Baron [:dbaron] (needinfo? me) from comment #19)
> > Do you know where Android-x86 emulator bugs should go?
> 
> Maybe http://source.android.com/source/report-bugs.html?
> 
> :blassey -- any thoughts?

that would be right if it is the android emulator. If it is actually qemu, then I'd suggest:
https://bugs.launchpad.net/qemu/
Flags: needinfo?(blassey.bugs)
I have a patch that I've sent to qemu-devel, though I can't remember if you need to be subscribed or not to send mail there.  If it doesn't show up tonight, I'll subscribe and resend the patch in the morning.

Not sure whether that addresses the emulator/qemu in our builds being broken, though.
We are running the emulator found in the Android SDK (18).
Guessing a more appropriate Bugzilla component, and assigning to patch author (thanks!).
Assignee: nobody → nfroyd
Component: CSS Parsing and Computation → General
Product: Core → Testing
(Though getting this fix deployed sounds like it might be significantly more work than the patch.)
Graydon has been doing some work in bug 910092 on rebuilding emulator images, he might be able to point someone in the right direction. (Hopefully rebuilding an x86 image with a new QEMU is simpler than the ARM work he's been doing there.)
Blocks: 892688
It seems that the fix got reviewed in the mailing list:
http://lists.gnu.org/archive/html/qemu-devel/2013-09/msg04254.html

What are the next steps?
Whiteboard: Status in comment 59
(In reply to Armen Zambrano [:armenzg] (Release Engineering) (EDT/UTC-4) from comment #59)
> It seems that the fix got reviewed in the mailing list:
> http://lists.gnu.org/archive/html/qemu-devel/2013-09/msg04254.html
> 
> What are the next steps?

The next step for getting it pulled into the main tree are to bug the maintainer enough until he decides to include it in his once-a-week pulls.  (The emulation side of QEMU, AFAICT, is not high-priority; the virtualization side gets much more attention.)

The next steps for getting rid of this orange is to figure out what qemu build/binary we are using, whether we can rebuild it, rebuild if so, and deploy.  If we can't rebuild, I guess we need to go bug Google or whomever to include it in their next sdk/ndk.
The one in aosp seems like quite an old fork:

  "QEMU PC emulator version 0.10.50Android, Copyright (c) 2003-2008 Fabrice Bellard"

I haven't looked into the problem of upgrading qemu within aosp yet as I'm still trying to find an effective armv6 fork. But will look further. Feel free to chase ahead on this issue; no idea if aosp is interested in upgrading theirs yet or if we'd do better in a fork.
(In reply to Graydon Hoare :graydon from comment #61)
> The one in aosp seems like quite an old fork:
> 
>   "QEMU PC emulator version 0.10.50Android, Copyright (c) 2003-2008 Fabrice
> Bellard"
> 
> I haven't looked into the problem of upgrading qemu within aosp yet as I'm
> still trying to find an effective armv6 fork. But will look further. Feel
> free to chase ahead on this issue; no idea if aosp is interested in
> upgrading theirs yet or if we'd do better in a fork.

The good news is that the patch applies without changes (well, OK, with ~600 lines of fuzz) to the aosp version.  What do you mean when you say "find an effective armv6 fork"?
Flags: needinfo?(graydon)
(In reply to Nathan Froyd (:froydnj) from comment #62)
> (In reply to Graydon Hoare :graydon from comment #61)
> > The one in aosp seems like quite an old fork:
> > 
> >   "QEMU PC emulator version 0.10.50Android, Copyright (c) 2003-2008 Fabrice
> > Bellard"
> > 
> > I haven't looked into the problem of upgrading qemu within aosp yet as I'm
> > still trying to find an effective armv6 fork. But will look further. Feel
> > free to chase ahead on this issue; no idea if aosp is interested in
> > upgrading theirs yet or if we'd do better in a fork.
> 
> The good news is that the patch applies without changes (well, OK, with ~600
> lines of fuzz) to the aosp version.  What do you mean when you say "find an
> effective armv6 fork"?

Sorry, not a very-related comment: in bug 910092 I'm trying to work out exactly which pieces of the aosp build need upgrades or patching to support an emulator that thinks it's armv6 (in order to test armv6 binaries on something suitably convincing), so am looking at some other aosp forks (such as http://code.google.com/p/androidarmv6/) who have done most of the work already. I was just pointing out in passing that there's more than one reason why we might wind up running emulators derived from non-stock-aosp releases.
Flags: needinfo?(graydon)
Whiteboard: Status in comment 59 → Status in comment 60
(In reply to Nathan Froyd (:froydnj) from comment #60)
> The next steps for getting rid of this orange is to figure out what qemu
> build/binary we are using, whether we can rebuild it, rebuild if so, and
> deploy.  If we can't rebuild, I guess we need to go bug Google or whomever
> to include it in their next sdk/ndk.

For these tests, mozharness launches "emulator" from the Android 4.3 (API 18) SDK. "emulator64-x86" is the process that ends up being spawned.

In my (outdated) copy of the AOSP source, I see helpful info in external/qemu/INSTALL which says that the Android emulator is a fork of qemu and claims that the emulator can be re-built easily.

It is not clear to me which android-4.3_rXX tag matches up with the Android 4.3 SDK. Hopefully it doesn't matter too much?

My computer is not in a good state to attempt another aosp build right now (low on disk space), but if some kind soul would patch an android-4.3_rXX external/qemu/target-i386/translate.c and send me the resulting emulator and emulator64-x86 binaries, I would be happy to test them.
(In reply to Geoff Brown [:gbrown] from comment #66)
> In my (outdated) copy of the AOSP source, I see helpful info in
> external/qemu/INSTALL which says that the Android emulator is a fork of qemu
> and claims that the emulator can be re-built easily.

"easily"...

Anyway, there are emulator* binaries and qemu-android-* binaries at:

http://people.mozilla.org/~nfroyd/qemu-binaries/

compiled for x86-64 Linux, built from the 'aosp/tools_17' tag.  Ideally the library versions used (Ubuntu 12.04 LTS) will be sufficient to run on any system you please.  I do not know why the emulator binary is not named emulator64-x86; I built x86 and x86-64 binaries and they were named identically.  It's possible that I had to build a newer tag to get differently named binaries...
Setting ni? on gbrown for comment 67.
Flags: needinfo?(gbrown)
I have verified that the emulator and emulator-x86 binaries execute on the loander and I have run a couple of tests successfully using those binaries. I will run several full batches of tests today/tomorrow to get a better idea of stability.
Mochitests and Robocop generally run well; there are a few more failing tests with the new emulator, but nothing too serious -- I wouldn't be surprised if these are simply timing differences.

However, all reftests fail consistently with:

INFO -  REFTEST TEST-UNEXPECTED-FAIL | | EXCEPTION: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIDOMWindowUtils.layerManagerType]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: chrome://reftest/content/reftest.jsm :: BuildConditionSandbox :: line 564"  data: no

Logcat also shows:

16:51:22     INFO -  10-15 23:51:01.274 W/GeckoGLController( 2334): GLController::surfaceChanged, creating compositor; mComposi     torCreated=false, mSurfaceValid=true
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334): Unable to create window surface
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334): java.lang.IllegalArgumentException
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at com.google.android.gles_jni.EGLImpl._eglCreateWindow     Surface(Native Method)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at com.google.android.gles_jni.EGLImpl.eglCreateWindowS     urface(EGLImpl.java:92)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at org.mozilla.gecko.gfx.GLController$1.run(GLControlle     r.java:160)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at android.os.Handler.handleCallback(Handler.java:725)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at android.os.Handler.dispatchMessage(Handler.java:92)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at android.os.Looper.loop(Looper.java:137)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at android.app.ActivityThread.main(ActivityThread.java:     5039)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at java.lang.reflect.Method.invokeNative(Native Method)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at java.lang.reflect.Method.invoke(Method.java:511)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at com.android.internal.os.ZygoteInit$MethodAndArgsCall     er.run(ZygoteInit.java:793)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.j     ava:560)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334):     at dalvik.system.NativeStart.main(Native Method)
16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334): EGL window surface could not be created: Error 12288
Flags: needinfo?(gbrown)
(In reply to Geoff Brown [:gbrown] from comment #70)
> Mochitests and Robocop generally run well; there are a few more failing
> tests with the new emulator, but nothing too serious -- I wouldn't be
> surprised if these are simply timing differences.

Wouldn't surprise me either if a slightly newer version of the emulator would fix that.  I used the version from tools-17...tools-22 probably has various fixes in various places.

> However, all reftests fail consistently with:
> 
> INFO -  REFTEST TEST-UNEXPECTED-FAIL | | EXCEPTION: [Exception... "Component
> returned failure code: 0x80004005 (NS_ERROR_FAILURE)
> [nsIDOMWindowUtils.layerManagerType]"  nsresult: "0x80004005
> (NS_ERROR_FAILURE)"  location: "JS frame ::
> chrome://reftest/content/reftest.jsm :: BuildConditionSandbox :: line 564" 
> data: no
> 
> Logcat also shows:
> 
> 16:51:22     INFO -  10-15 23:51:01.274 W/GeckoGLController( 2334):
> GLController::surfaceChanged, creating compositor; mComposi    
> torCreated=false, mSurfaceValid=true
> 16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334): Unable
> to create window surface

That would probably be because I configured QEMU (and/or SDL?) wihout gles support.  I could try to fix that.
(In reply to Nathan Froyd (:froydnj) from comment #71)
> > 16:51:22     INFO -  10-15 23:51:01.274 W/GeckoGLController( 2334):
> > GLController::surfaceChanged, creating compositor; mComposi    
> > torCreated=false, mSurfaceValid=true
> > 16:51:22     INFO -  10-15 23:51:01.274 E/GeckoGLController( 2334): Unable
> > to create window surface
> 
> That would probably be because I configured QEMU (and/or SDL?) wihout gles
> support.  I could try to fix that.

This is somewhat unpleasant to do (AFAICS, it requires building a significant chunk of AOSP, which may in fact be "all of it" since I don't think there's a good way to do a partial build).  If I can build this emulator and it passes tests, are we just going to drop it in to what we have now?  And are we going to keep rebuilding if future versions of the SDK don't fix the problems either?
If we can build an emulator that works for our tests, I think we would use your binaries to over-write those in the copy of the Android SDK distributed to the test machines. We would probably want (and rel-eng might insist on) a copy of your source tree and/or instructions on how to re-create the emulator in case we need to update it in future. Eventually, a future version of the SDK should have a usable emulator; at that point I'm pretty sure we would switch back to the SDK's emulator.

Armen -- your perspective?
Flags: needinfo?(armenzg)
We can overwrite the current emulator, however, I would prefer to change the name from being "android-sdk18".
We also need to have documentation to rebuild it if possible.
I assume we would only need to rebuild if we find some crashiness that we can fix.
At a later point we can easily switch to use a newer version of the SDK.

I see on the people account some binaries which are not on the sdk18 (emulator-ui and qemu files).
[ ]	emulator		11-Oct-2013 10:50 	119K	 
[ ]	emulator-arm		11-Oct-2013 10:51 	11M	 
[ ]	emulator-ui		11-Oct-2013 10:51 	1.9M	 
[ ]	emulator-x86		11-Oct-2013 10:51 	12M	 
[ ]	qemu-android-arm	11-Oct-2013 10:52 	9.8M	 
[ ]	qemu-android-x86	11-Oct-2013 10:52 	11M	 

Are the qemu files supposed to be included in the avd definition files?

This is what we have on disk.
[cltbld@talos-linux64-ix-004.test.releng.scl3.mozilla.com android-sdk18]$ pwd
/tools/android-sdk18
[cltbld@talos-linux64-ix-004.test.releng.scl3.mozilla.com android-sdk18]$ find . -name "emulator*" -type f
./tools/emulator
./tools/emulator-mips
./tools/emulator64-x86
./tools/emulator64-mips
./tools/emulator-arm
./tools/emulator-x86
./tools/emulator64-arm

[cltbld@talos-linux64-ix-004.test.releng.scl3.mozilla.com android-sdk18]$ tools/emulator -version
Android emulator version 22.0.5.0 (build_id OPENMASTER-757759)
Copyright (C) 2006-2011 The Android Open Source Project and many others.
This program is a derivative of the QEMU CPU emulator (www.qemu.org).

  This software is licensed under the terms of the GNU General Public
  License version 2, as published by the Free Software Foundation, and
  may be copied, distributed, and modified under those terms.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

[cltbld@talos-linux64-ix-004.test.releng.scl3.mozilla.com android-sdk18]$ tools/emulator-arm -version
Android emulator version 22.0.5.0 (build_id OPENMASTER-757759)
Copyright (C) 2006-2011 The Android Open Source Project and many others.
This program is a derivative of the QEMU CPU emulator (www.qemu.org).

  This software is licensed under the terms of the GNU General Public
  License version 2, as published by the Free Software Foundation, and
  may be copied, distributed, and modified under those terms.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

[cltbld@talos-linux64-ix-004.test.releng.scl3.mozilla.com android-sdk18]$ tools/emulator-x86 -version
Android emulator version 22.0.5.0 (build_id OPENMASTER-757759)
Copyright (C) 2006-2011 The Android Open Source Project and many others.
This program is a derivative of the QEMU CPU emulator (www.qemu.org).

  This software is licensed under the terms of the GNU General Public
  License version 2, as published by the Free Software Foundation, and
  may be copied, distributed, and modified under those terms.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.
Flags: needinfo?(armenzg)
I can build aosp, after some gnashing of teeth, but the build doesn't have emulator64* binaries; just emulator-* versions.  The aosp build instructions don't provide any details on how to build 64-bit host tools, either.
I am a little troubled by the discrepancy (64-bit binaries in the sdk/none in the aosp build), but when I tested Nathan's 2013-10-11 build, I only had emulator and emulator-x86 in my $PATH and that seemed to be enough. (More precisely, I removed $ANDROID_SDK/* from $PATH, created a new directory and added it to $PATH, populated the new directory with emulator, emulator-x86, and a copy of $ANDROID_SDK/tools/lib, then ran our tests normally from mozharness.)
I've updated the emulator* binaries at:

http://people.mozilla.org/~nfroyd/qemu-binaries/

They were built in the course of a full AOSP build, so they should have all the necessary GL bits to make reftests work.  Geoff, can you test the new binaries, please?

I have found bits and pieces of what look like a way to build 64-bit emulator binaries, but no luck yet.
Flags: needinfo?(gbrown)
I will run a new set of tests over the weekend and report back here by Monday.
Reftests still fail, just like the previous emulator build -- same error as Comment 70.
Flags: needinfo?(gbrown)
If I start the emulator with -debug all, I get lots of output. Here's the bit around the OpenGL warning:

emulator: android_qemud_init
emulator: Registered QEMUD service boot-properties
emulator: registered 'boot-properties' qemud service
emulator: Adding boot property: 'dalvik.vm.heapsize' = '64m'
emulator: Adding boot property: 'qemu.sf.lcd_density' = '160'
emulator: Adding boot property: 'qemu.hw.mainkeys' = '0'
emulator: android_qemud_init
emulator: Registered QEMUD service gsm
emulator: android_qemud_init
emulator: Registered QEMUD service gps
emulator: Adding boot property: 'qemu.sf.fake_camera' = 'none'
emulator: android_qemud_init
emulator: Registered QEMUD service camera
emulator: android_camera_service_init: Registered 'camera' qemud service
emulator: IP address of your DNS(s): 10.26.75.40, 10.26.75.41
emulator: nand_add_dev: cache,size=0x4200000,file=/home/cltbld/.android/avd/test-x86-1.avd/cache.img
emulator: WARNING: Could not initialize OpenglES emulation, using software renderer.
emulator: KVM mode auto-enabled!
emulator: Kernel parameters: qemu.gles=0 qemu=1 console=ttyS0 android.qemud=ttyS1 androidboot.hardware=goldfish clocksource=pit android.checkjni=1 ndns=2
emulator:     trying to find: /home/cltbld/emu4/bios.bin

emulator:     trying to find: /home/cltbld/emu4/lib/pc-bios/bios.bin

emulator:     trying to find: /home/cltbld/emu4/vgabios-cirrus.bin

emulator:     trying to find: /home/cltbld/emu4/lib/pc-bios/vgabios-cirrus.bin

emulator: qpa_audio_init: entering
emulator: qpa_audio_init: exiting
emulator: autoconfig: -scale 1
emulator: android_qemud_init
emulator: Registered QEMUD service sensors


Looking at the emulator code, I see lots of ways for "Could not initialize OpenglES emulation" to be printed after a more specific error message (like "Could not load OpenGLES emulation library: %s")...but I don't see how to get that error *without* a more specific error message. 

As a long-shot, I wonder if we should be replacing some of the associated libs, like libOpenglRender.so, libEGL_translator.so, etc.?
(In reply to Geoff Brown [:gbrown] from comment #80)
> As a long-shot, I wonder if we should be replacing some of the associated
> libs, like libOpenglRender.so, libEGL_translator.so, etc.?

Replacing them with what?  I don't see libOpenglRender.so and associated libraries in my build tree.  Which seems a little odd.  Does strace'ing the new emulator and seeing what (if any) libraries it's trying to load help?  I'm wondering if it's trying to load the libraries relative to the new emulator and not finding them, whereas the old emulators have the libraries in the proper relative location.
Flags: needinfo?(gbrown)
(In reply to Nathan Froyd (:froydnj) from comment #81)
> Replacing them with what?  I don't see libOpenglRender.so and associated
> libraries in my build tree.  

I thought/hoped you would have those libs in your build.

> Does strace'ing the
> new emulator and seeing what (if any) libraries it's trying to load help? 

See http://people.mozilla.org/~gbrown/strace.out

> I'm wondering if it's trying to load the libraries relative to the new
> emulator and not finding them, whereas the old emulators have the libraries
> in the proper relative location.

The normal Android sdk is at /tools/android-sdk18. I copied /tools/android-sdk18/tools to emu4, copied your emulator and emulator-x86 to emu4, deleted emu4/emulator64-x86, and set:

PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:/tools/android-sdk18/platform-tools:/home/cltbld/emu4

I have not noticed any suspicious library accesses in strace.out.
Flags: needinfo?(gbrown)
(In reply to Geoff Brown [:gbrown] from comment #82)
> (In reply to Nathan Froyd (:froydnj) from comment #81)
> > Replacing them with what?  I don't see libOpenglRender.so and associated
> > libraries in my build tree.  
> 
> I thought/hoped you would have those libs in your build.

I did too!  I do not know why they are not there.

> I have not noticed any suspicious library accesses in strace.out.

Hm, I see:

24883 access("./emulator64-x86", F_OK)  = -1 ENOENT (No such file or directory)
24883 access("./emulator-x86", F_OK)    = -1 ENOENT (No such file or directory)
24883 access("/usr/local/bin/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/local/sbin/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/local/bin/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/sbin/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/bin/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/sbin/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/bin/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/bin/X11/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/games/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/tools/android-sdk18/platform-tools/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/home/cltbld/emu4/emulator64-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/local/bin/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/local/sbin/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/local/bin/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/sbin/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/bin/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/sbin/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/bin/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/bin/X11/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/usr/games/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/tools/android-sdk18/platform-tools/emulator-x86", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/home/cltbld/emu4/emulator-x86", F_OK) = 0
24883 access("/home/cltbld/emu4/emulator-x86", X_OK) = 0
24883 access("/home/cltbld/emu4/lib64OpenglRender.so", F_OK) = -1 ENOENT (No such file or directory)
24883 access("/home/cltbld/emu4/lib/lib64OpenglRender.so", F_OK) = 0
24883 execve("/home/cltbld/emu4/emulator-x86", ["/home/cltbld/emu4/emulator-x86", "-avd", "test-x86-1", "-port", "5554", "-kernel", "/home/cltbld/.android/avd/kernel"..., "-system", "/home/cltbld/.android/avd/system"..., "-ramdisk", "/home/cltbld/.android/avd/ramdis"...], [/* 18 vars */]) = 0

I guess we really need |strace -f|.

What are the contents of /home/cltbld/emu4/lib/?
Flags: needinfo?(gbrown)
Actually, that was strace -f -- I don't know why I'm not getting traces from the child process.

Contents of /home/cltbld/emu4/lib:

.:
total 33940
-rw-r--r-- 1 cltbld cltbld     4737 Oct 20 10:38 android.el
-rw-r--r-- 1 cltbld cltbld   102021 Oct 20 10:38 ant-tasks.jar
-rw-r--r-- 1 cltbld cltbld     1462 Oct 20 10:38 archquery.jar
-rw-r--r-- 1 cltbld cltbld    46022 Oct 20 10:38 asm-4.0.jar
-rw-r--r-- 1 cltbld cltbld    19614 Oct 20 10:38 asm-analysis-4.0.jar
-rw-r--r-- 1 cltbld cltbld    21982 Oct 20 10:38 asm-tree-4.0.jar
-rw-r--r-- 1 cltbld cltbld   643876 Oct 20 10:38 asset-studio.jar
-rw-r--r-- 1 cltbld cltbld   585181 Oct 20 10:38 bcpkix-jdk15on-1.48.jar
-rw-r--r-- 1 cltbld cltbld  2318161 Oct 20 10:38 bcprov-jdk15on-1.48.jar
-rw-r--r-- 1 cltbld cltbld     3934 Oct 20 10:38 build.template
-rw-r--r-- 1 cltbld cltbld    51885 Oct 20 10:38 chimpchat.jar
-rw-r--r-- 1 cltbld cltbld    55006 Oct 20 10:38 common.jar
-rw-r--r-- 1 cltbld cltbld    58160 Oct 20 10:38 commons-codec-1.4.jar
-rw-r--r-- 1 cltbld cltbld   143847 Oct 20 10:38 commons-compress-1.0.jar
-rw-r--r-- 1 cltbld cltbld    60686 Oct 20 10:38 commons-logging-1.1.1.jar
-rw-r--r-- 1 cltbld cltbld   251928 Oct 20 10:38 ddmlib.jar
-rw-r--r-- 1 cltbld cltbld   123767 Oct 20 10:38 ddms.jar
-rw-r--r-- 1 cltbld cltbld   579266 Oct 20 10:38 ddmuilib.jar
-rw-r--r-- 1 cltbld cltbld    23798 Oct 20 10:38 devices.xml
-rw-r--r-- 1 cltbld cltbld    69295 Oct 20 10:38 draw9patch.jar
-rw-r--r-- 1 cltbld cltbld    11083 Oct 20 10:38 dvlib.jar
-rw-r--r-- 1 cltbld cltbld    38232 Oct 20 10:38 emma_ant.jar
-rw-r--r-- 1 cltbld cltbld   379516 Oct 20 10:38 emma_device.jar
-rw-r--r-- 1 cltbld cltbld   428218 Oct 20 10:38 emma.jar
drwxr-xr-x 3 cltbld cltbld     4096 Oct 20 10:38 emulator
-rw-r--r-- 1 cltbld cltbld    71904 Oct 20 10:38 fat32lib.jar
-rw-r--r-- 1 cltbld cltbld  1891110 Oct 20 10:38 guava-13.0.1.jar
-rw-r--r-- 1 cltbld cltbld    10882 Oct 20 10:38 hardware-properties.ini
-rw-r--r-- 1 cltbld cltbld    32576 Oct 20 10:38 hierarchyviewer2.jar
-rw-r--r-- 1 cltbld cltbld   371371 Oct 20 10:38 hierarchyviewer2lib.jar
-rw-r--r-- 1 cltbld cltbld   351132 Oct 20 10:38 httpclient-4.1.1.jar
-rw-r--r-- 1 cltbld cltbld   181041 Oct 20 10:38 httpcore-4.1.jar
-rw-r--r-- 1 cltbld cltbld    26813 Oct 20 10:38 httpmime-4.1.jar
-rw-r--r-- 1 cltbld cltbld   309245 Oct 20 10:38 jcommon-1.0.12.jar
-rw-r--r-- 1 cltbld cltbld  1291498 Oct 20 10:38 jfreechart-1.0.9.jar
-rw-r--r-- 1 cltbld cltbld    73010 Oct 20 10:38 jfreechart-swt-1.0.9.jar
-rw-r--r-- 1 cltbld cltbld    36332 Oct 20 10:38 jobb.jar
-rw-r--r-- 1 cltbld cltbld   545459 Oct 20 10:38 jsilver-1.0.0.jar
-rw-r--r-- 1 cltbld cltbld 14340135 Oct 20 10:38 jython-standalone-2.5.3.jar
-rw-r--r-- 1 cltbld cltbld    43858 Oct 20 10:38 kxml2-2.3.0.jar
-rw-r--r-- 1 cltbld cltbld    64643 Oct 20 10:38 layoutlib-api.jar
-rw-r--r-- 1 cltbld cltbld   238764 Oct 20 10:38 lib64EGL_translator.so
-rw-r--r-- 1 cltbld cltbld   315643 Oct 20 10:38 lib64GLES_CM_translator.so
-rw-r--r-- 1 cltbld cltbld   302966 Oct 20 10:38 lib64GLES_V2_translator.so
-rw-r--r-- 1 cltbld cltbld   382109 Oct 20 10:38 lib64OpenglRender.so
-rw-r--r-- 1 cltbld cltbld   216798 Oct 20 10:38 libEGL_translator.so
-rw-r--r-- 1 cltbld cltbld   295201 Oct 20 10:38 libGLES_CM_translator.so
-rw-r--r-- 1 cltbld cltbld   282688 Oct 20 10:38 libGLES_V2_translator.so
-rw-r--r-- 1 cltbld cltbld   344133 Oct 20 10:38 libOpenglRender.so
-rw-r--r-- 1 cltbld cltbld   137514 Oct 20 10:38 lint-api.jar
-rw-r--r-- 1 cltbld cltbld   457647 Oct 20 10:38 lint-checks.jar
-rw-r--r-- 1 cltbld cltbld    61685 Oct 20 10:38 lint.jar
-rw-r--r-- 1 cltbld cltbld  1060474 Oct 20 10:38 lombok-ast-0.2.1.jar
-rw-r--r-- 1 cltbld cltbld     1076 Oct 20 10:38 lombok-ast-NOTICE.txt
-rw-r--r-- 1 cltbld cltbld    33623 Oct 20 10:38 manifest-merger.jar
drwxr-xr-x 5 cltbld cltbld     4096 Oct 20 10:38 monitor-x86
drwxr-xr-x 5 cltbld cltbld     4096 Oct 20 10:38 monitor-x86_64
-rw-r--r-- 1 cltbld cltbld   101642 Oct 20 10:38 monkeyrunner.jar
-rw-r--r-- 1 cltbld cltbld    11384 Oct 20 10:38 ninepatch.jar
-rw-r--r-- 1 cltbld cltbld   107209 Oct 20 10:38 org-eclipse-core-commands-3.6.0.jar
-rw-r--r-- 1 cltbld cltbld   101958 Oct 20 10:38 org-eclipse-equinox-common-3.6.0.jar
-rw-r--r-- 1 cltbld cltbld  1069483 Oct 20 10:38 org-eclipse-jface-3.6.2.jar
-rw-r--r-- 1 cltbld cltbld  1831843 Oct 20 10:38 osgi-4.0.0.jar
drwxr-xr-x 2 cltbld cltbld     4096 Oct 20 10:38 pc-bios
-rw-r--r-- 1 cltbld cltbld       59 Oct 20 10:38 plugin.prop
-rw-r--r-- 1 cltbld cltbld      781 Oct 20 10:38 proguard-project.txt
-rw-r--r-- 1 cltbld cltbld    39656 Oct 20 10:38 rule-api.jar
-rw-r--r-- 1 cltbld cltbld     5379 Oct 20 10:38 screenshot2.jar
-rw-r--r-- 1 cltbld cltbld   233048 Oct 20 10:38 sdk-common.jar
-rw-r--r-- 1 cltbld cltbld   570315 Oct 20 10:38 sdklib.jar
-rw-r--r-- 1 cltbld cltbld    26742 Oct 20 10:38 sdkmanager.jar
-rw-r--r-- 1 cltbld cltbld    16840 Oct 20 10:38 sdkstats.jar
-rw-r--r-- 1 cltbld cltbld   462638 Oct 20 10:38 sdkuilib.jar
-rw-r--r-- 1 cltbld cltbld    18577 Oct 20 10:38 swtmenubar.jar
-rw-r--r-- 1 cltbld cltbld   137238 Oct 20 10:38 traceview.jar
-rw-r--r-- 1 cltbld cltbld    51636 Oct 20 10:38 uiautomatorviewer.jar
-rw-r--r-- 1 cltbld cltbld     3936 Oct 20 10:38 uibuild.template
drwxr-xr-x 2 cltbld cltbld     4096 Oct 20 10:38 x86
drwxr-xr-x 2 cltbld cltbld     4096 Oct 20 10:38 x86_64

./emulator:
total 260
drwxr-xr-x 3 cltbld cltbld   4096 Oct 20 10:38 skins
-rw-r--r-- 1 cltbld cltbld 262144 Oct 20 10:38 snapshots.img

./emulator/skins:
total 4
drwxr-xr-x 2 cltbld cltbld 4096 Oct 20 10:38 dynamic

./emulator/skins/dynamic:
total 92
-rw-r--r-- 1 cltbld cltbld  449 Oct 20 10:38 arrow_down.png
-rw-r--r-- 1 cltbld cltbld  825 Oct 20 10:38 arrow_left.png
-rw-r--r-- 1 cltbld cltbld  795 Oct 20 10:38 arrow_right.png
-rw-r--r-- 1 cltbld cltbld  453 Oct 20 10:38 arrow_up.png
-rw-r--r-- 1 cltbld cltbld 5322 Oct 20 10:38 basic.png
-rw-r--r-- 1 cltbld cltbld  592 Oct 20 10:38 button.png
-rw-r--r-- 1 cltbld cltbld 9826 Oct 20 10:38 dpad_off.png
-rw-r--r-- 1 cltbld cltbld 9065 Oct 20 10:38 dpad_on.png
-rw-r--r-- 1 cltbld cltbld 9159 Oct 20 10:38 hwkeys_off.png
-rw-r--r-- 1 cltbld cltbld 8259 Oct 20 10:38 hwkeys_on.png
-rw-r--r-- 1 cltbld cltbld 3958 Oct 20 10:38 keyboard_off.png
-rw-r--r-- 1 cltbld cltbld 3374 Oct 20 10:38 keyboard_on.png
-rw-r--r-- 1 cltbld cltbld 2971 Oct 20 10:38 layout
-rw-r--r-- 1 cltbld cltbld  384 Oct 20 10:38 select.png

./monitor-x86:
total 344
drwxr-xr-x 2 cltbld cltbld   4096 Oct 20 10:38 about_files
-rw-r--r-- 1 cltbld cltbld    577 Oct 20 10:38 about.html
drwxr-xr-x 2 cltbld cltbld   4096 Oct 20 10:38 configuration
-rw-r--r-- 1 cltbld cltbld     17 Oct 20 10:38 eclipse.ini
-rw-r--r-- 1 cltbld cltbld 266168 Oct 20 10:38 libcairo-swt.so
-rwxr-xr-x 1 cltbld cltbld  62814 Oct 20 10:38 monitor
drwxr-xr-x 4 cltbld cltbld   4096 Oct 20 10:38 plugins

./monitor-x86/about_files:
total 40
-rw-r--r-- 1 cltbld cltbld  2071 Oct 20 10:38 about_cairo.html
-rw-r--r-- 1 cltbld cltbld 25755 Oct 20 10:38 mpl-v11.txt
-rw-r--r-- 1 cltbld cltbld  4544 Oct 20 10:38 pixman-licenses.txt

./monitor-x86/configuration:
total 4
-rw-r--r-- 1 cltbld cltbld 1664 Oct 20 10:38 config.ini

./monitor-x86/plugins:
total 33804
-rw-r--r-- 1 cltbld cltbld 5840555 Oct 20 10:38 com.android.ide.eclipse.base_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld 2532793 Oct 20 10:38 com.android.ide.eclipse.ddms_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  399205 Oct 20 10:38 com.android.ide.eclipse.gldebugger_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  387524 Oct 20 10:38 com.android.ide.eclipse.hierarchyviewer_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  603722 Oct 20 10:38 com.android.ide.eclipse.monitor_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  139146 Oct 20 10:38 com.android.ide.eclipse.traceview_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld 6701203 Oct 20 10:38 com.ibm.icu_4.4.2.v20110823.jar
-rw-r--r-- 1 cltbld cltbld   61780 Oct 20 10:38 org.eclipse.compare.core_3.5.200.I20110208-0800.jar
-rw-r--r-- 1 cltbld cltbld  107303 Oct 20 10:38 org.eclipse.core.commands_3.6.0.I20110111-0800.jar
-rw-r--r-- 1 cltbld cltbld   91961 Oct 20 10:38 org.eclipse.core.contenttype_3.4.100.v20110423-0524.jar
-rw-r--r-- 1 cltbld cltbld  200967 Oct 20 10:38 org.eclipse.core.databinding_1.4.0.I20110111-0800.jar
-rw-r--r-- 1 cltbld cltbld  296139 Oct 20 10:38 org.eclipse.core.databinding.observable_1.4.0.I20110222-0800.jar
-rw-r--r-- 1 cltbld cltbld  167964 Oct 20 10:38 org.eclipse.core.databinding.property_1.4.0.I20110222-0800.jar
-rw-r--r-- 1 cltbld cltbld   86992 Oct 20 10:38 org.eclipse.core.expressions_3.4.300.v20110228.jar
-rw-r--r-- 1 cltbld cltbld   53218 Oct 20 10:38 org.eclipse.core.filesystem_1.3.100.v20110423-0524.jar
-rw-r--r-- 1 cltbld cltbld   91069 Oct 20 10:38 org.eclipse.core.jobs_3.5.101.v20120113-1953.jar
-rw-r--r-- 1 cltbld cltbld  807241 Oct 20 10:38 org.eclipse.core.resources_3.7.101.v20120125-1505.jar
-rw-r--r-- 1 cltbld cltbld   70526 Oct 20 10:38 org.eclipse.core.runtime_3.7.0.v20110110.jar
drwxr-xr-x 3 cltbld cltbld    4096 Oct 20 10:38 org.eclipse.core.runtime.compatibility.registry_3.5.0.v20110505
-rw-r--r-- 1 cltbld cltbld   33061 Oct 20 10:38 org.eclipse.core.variables_3.2.500.v20110928-1503.jar
-rw-r--r-- 1 cltbld cltbld   93248 Oct 20 10:38 org.eclipse.ecf_3.1.300.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   50502 Oct 20 10:38 org.eclipse.ecf.filetransfer_5.0.0.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   51439 Oct 20 10:38 org.eclipse.ecf.identity_3.1.100.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld  126827 Oct 20 10:38 org.eclipse.ecf.provider.filetransfer_3.2.0.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld    8460 Oct 20 10:38 org.eclipse.ecf.provider.filetransfer.ssl_1.0.0.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   11558 Oct 20 10:38 org.eclipse.ecf.ssl_1.0.100.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   85218 Oct 20 10:38 org.eclipse.equinox.app_1.3.100.v20110321.jar
-rw-r--r-- 1 cltbld cltbld  105624 Oct 20 10:38 org.eclipse.equinox.common_3.6.0.v20110523.jar
-rw-r--r-- 1 cltbld cltbld   47262 Oct 20 10:38 org.eclipse.equinox.launcher_1.2.0.v20110502.jar
drwxr-xr-x 3 cltbld cltbld    4096 Oct 20 10:38 org.eclipse.equinox.launcher.gtk.linux.x86_1.1.100.v20110505
-rw-r--r-- 1 cltbld cltbld   71555 Oct 20 10:38 org.eclipse.equinox.p2.core_2.1.1.v20120113-1346.jar
-rw-r--r-- 1 cltbld cltbld  184966 Oct 20 10:38 org.eclipse.equinox.p2.engine_2.1.1.R37x_v20111003.jar
-rw-r--r-- 1 cltbld cltbld  339099 Oct 20 10:38 org.eclipse.equinox.p2.metadata_2.1.0.v20110815-1419.jar
-rw-r--r-- 1 cltbld cltbld  115721 Oct 20 10:38 org.eclipse.equinox.p2.metadata.repository_1.2.0.v20110815-1419.jar
-rw-r--r-- 1 cltbld cltbld  127900 Oct 20 10:38 org.eclipse.equinox.p2.repository_2.1.1.v20120113-1346.jar
-rw-r--r-- 1 cltbld cltbld  113791 Oct 20 10:38 org.eclipse.equinox.preferences_3.4.2.v20120111-2020.jar
-rw-r--r-- 1 cltbld cltbld  183035 Oct 20 10:38 org.eclipse.equinox.registry_3.5.101.R37x_v20110810-1611.jar
-rw-r--r-- 1 cltbld cltbld  106105 Oct 20 10:38 org.eclipse.equinox.security_1.1.1.R37x_v20110822-1018.jar
-rw-r--r-- 1 cltbld cltbld  257403 Oct 20 10:38 org.eclipse.help_3.5.100.v20110426.jar
-rw-r--r-- 1 cltbld cltbld 1077135 Oct 20 10:38 org.eclipse.jface_3.7.0.v20110928-1505.jar
-rw-r--r-- 1 cltbld cltbld  275458 Oct 20 10:38 org.eclipse.jface.databinding_1.5.0.I20100907-0800.jar
-rw-r--r-- 1 cltbld cltbld  981513 Oct 20 10:38 org.eclipse.jface.text_3.7.2.v20111213-1208.jar
-rw-r--r-- 1 cltbld cltbld 1364409 Oct 20 10:38 org.eclipse.osgi_3.7.2.v20120110-1415.jar
-rw-r--r-- 1 cltbld cltbld   71157 Oct 20 10:38 org.eclipse.osgi.services_3.3.0.v20110513.jar
-rw-r--r-- 1 cltbld cltbld   17332 Oct 20 10:38 org.eclipse.swt_3.7.2.v3740f.jar
-rw-r--r-- 1 cltbld cltbld 2305649 Oct 20 10:38 org.eclipse.swt.gtk.linux.x86_3.7.2.v3740f.jar
-rw-r--r-- 1 cltbld cltbld  248183 Oct 20 10:38 org.eclipse.text_3.5.101.v20110928-1504.jar
-rw-r--r-- 1 cltbld cltbld  153179 Oct 20 10:38 org.eclipse.ui_3.7.0.v20110928-1505.jar
-rw-r--r-- 1 cltbld cltbld  159986 Oct 20 10:38 org.eclipse.ui.console_3.5.100.v20111007_r372.jar
-rw-r--r-- 1 cltbld cltbld 2339393 Oct 20 10:38 org.eclipse.ui.ide_3.7.0.v20110928-1505.jar
-rw-r--r-- 1 cltbld cltbld 4166247 Oct 20 10:38 org.eclipse.ui.workbench_3.7.1.v20120104-1859.jar
-rw-r--r-- 1 cltbld cltbld  581960 Oct 20 10:38 org.eclipse.ui.workbench.texteditor_3.7.0.v20110928-1504.jar

./monitor-x86/plugins/org.eclipse.core.runtime.compatibility.registry_3.5.0.v20110505:
total 24
-rw-r--r-- 1 cltbld cltbld  1432 Oct 20 10:38 about.html
-rw-r--r-- 1 cltbld cltbld   603 Oct 20 10:38 fragment.properties
drwxr-xr-x 2 cltbld cltbld  4096 Oct 20 10:38 META-INF
-rw-r--r-- 1 cltbld cltbld 12213 Oct 20 10:38 runtime_registry_compatibility.jar

./monitor-x86/plugins/org.eclipse.core.runtime.compatibility.registry_3.5.0.v20110505/META-INF:
total 20
-rw-r--r-- 1 cltbld cltbld 5638 Oct 20 10:38 ECLIPSEF.RSA
-rw-r--r-- 1 cltbld cltbld  547 Oct 20 10:38 ECLIPSEF.SF
-rw-r--r-- 1 cltbld cltbld   76 Oct 20 10:38 eclipse.inf
-rw-r--r-- 1 cltbld cltbld  981 Oct 20 10:38 MANIFEST.MF

./monitor-x86/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.1.100.v20110505:
total 176
-rw-r--r-- 1 cltbld cltbld   1445 Oct 20 10:38 about.html
-rw-r--r-- 1 cltbld cltbld 165747 Oct 20 10:38 eclipse_1407.so
-rw-r--r-- 1 cltbld cltbld    617 Oct 20 10:38 launcher.gtk.linux.x86.properties
drwxr-xr-x 2 cltbld cltbld   4096 Oct 20 10:38 META-INF

./monitor-x86/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.1.100.v20110505/META-INF:
total 20
-rw-r--r-- 1 cltbld cltbld 5639 Oct 20 10:38 ECLIPSEF.RSA
-rw-r--r-- 1 cltbld cltbld  473 Oct 20 10:38 ECLIPSEF.SF
-rw-r--r-- 1 cltbld cltbld   76 Oct 20 10:38 eclipse.inf
-rw-r--r-- 1 cltbld cltbld  956 Oct 20 10:38 MANIFEST.MF

./monitor-x86_64:
total 420
drwxr-xr-x 2 cltbld cltbld   4096 Oct 20 10:38 about_files
-rw-r--r-- 1 cltbld cltbld    577 Oct 20 10:38 about.html
drwxr-xr-x 2 cltbld cltbld   4096 Oct 20 10:38 configuration
-rw-r--r-- 1 cltbld cltbld     17 Oct 20 10:38 eclipse.ini
-rw-r--r-- 1 cltbld cltbld 335360 Oct 20 10:38 libcairo-swt.so
-rwxr-xr-x 1 cltbld cltbld  71023 Oct 20 10:38 monitor
drwxr-xr-x 4 cltbld cltbld   4096 Oct 20 10:38 plugins

./monitor-x86_64/about_files:
total 40
-rw-r--r-- 1 cltbld cltbld  2071 Oct 20 10:38 about_cairo.html
-rw-r--r-- 1 cltbld cltbld 25755 Oct 20 10:38 mpl-v11.txt
-rw-r--r-- 1 cltbld cltbld  4544 Oct 20 10:38 pixman-licenses.txt

./monitor-x86_64/configuration:
total 4
-rw-r--r-- 1 cltbld cltbld 1708 Oct 20 10:38 config.ini

./monitor-x86_64/plugins:
total 33992
-rw-r--r-- 1 cltbld cltbld 5840555 Oct 20 10:38 com.android.ide.eclipse.base_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld 2532793 Oct 20 10:38 com.android.ide.eclipse.ddms_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  399205 Oct 20 10:38 com.android.ide.eclipse.gldebugger_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  387524 Oct 20 10:38 com.android.ide.eclipse.hierarchyviewer_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  603722 Oct 20 10:38 com.android.ide.eclipse.monitor_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld  139146 Oct 20 10:38 com.android.ide.eclipse.traceview_22.0.5.201307292138.jar
-rw-r--r-- 1 cltbld cltbld 6701203 Oct 20 10:38 com.ibm.icu_4.4.2.v20110823.jar
-rw-r--r-- 1 cltbld cltbld   61780 Oct 20 10:38 org.eclipse.compare.core_3.5.200.I20110208-0800.jar
-rw-r--r-- 1 cltbld cltbld  107303 Oct 20 10:38 org.eclipse.core.commands_3.6.0.I20110111-0800.jar
-rw-r--r-- 1 cltbld cltbld   91961 Oct 20 10:38 org.eclipse.core.contenttype_3.4.100.v20110423-0524.jar
-rw-r--r-- 1 cltbld cltbld  200967 Oct 20 10:38 org.eclipse.core.databinding_1.4.0.I20110111-0800.jar
-rw-r--r-- 1 cltbld cltbld  296139 Oct 20 10:38 org.eclipse.core.databinding.observable_1.4.0.I20110222-0800.jar
-rw-r--r-- 1 cltbld cltbld  167964 Oct 20 10:38 org.eclipse.core.databinding.property_1.4.0.I20110222-0800.jar
-rw-r--r-- 1 cltbld cltbld   86992 Oct 20 10:38 org.eclipse.core.expressions_3.4.300.v20110228.jar
-rw-r--r-- 1 cltbld cltbld   53218 Oct 20 10:38 org.eclipse.core.filesystem_1.3.100.v20110423-0524.jar
-rw-r--r-- 1 cltbld cltbld    9549 Oct 20 10:38 org.eclipse.core.filesystem.linux.x86_64_1.2.0.v20110423-0524.jar
-rw-r--r-- 1 cltbld cltbld   91069 Oct 20 10:38 org.eclipse.core.jobs_3.5.101.v20120113-1953.jar
-rw-r--r-- 1 cltbld cltbld  807241 Oct 20 10:38 org.eclipse.core.resources_3.7.101.v20120125-1505.jar
-rw-r--r-- 1 cltbld cltbld   70526 Oct 20 10:38 org.eclipse.core.runtime_3.7.0.v20110110.jar
drwxr-xr-x 3 cltbld cltbld    4096 Oct 20 10:38 org.eclipse.core.runtime.compatibility.registry_3.5.0.v20110505
-rw-r--r-- 1 cltbld cltbld   33061 Oct 20 10:38 org.eclipse.core.variables_3.2.500.v20110928-1503.jar
-rw-r--r-- 1 cltbld cltbld   93248 Oct 20 10:38 org.eclipse.ecf_3.1.300.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   50502 Oct 20 10:38 org.eclipse.ecf.filetransfer_5.0.0.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   51439 Oct 20 10:38 org.eclipse.ecf.identity_3.1.100.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld  126827 Oct 20 10:38 org.eclipse.ecf.provider.filetransfer_3.2.0.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld    8460 Oct 20 10:38 org.eclipse.ecf.provider.filetransfer.ssl_1.0.0.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   11558 Oct 20 10:38 org.eclipse.ecf.ssl_1.0.100.v20110531-2218.jar
-rw-r--r-- 1 cltbld cltbld   85218 Oct 20 10:38 org.eclipse.equinox.app_1.3.100.v20110321.jar
-rw-r--r-- 1 cltbld cltbld  105624 Oct 20 10:38 org.eclipse.equinox.common_3.6.0.v20110523.jar
-rw-r--r-- 1 cltbld cltbld   47262 Oct 20 10:38 org.eclipse.equinox.launcher_1.2.0.v20110502.jar
drwxr-xr-x 3 cltbld cltbld    4096 Oct 20 10:38 org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.100.v20110505
-rw-r--r-- 1 cltbld cltbld   71555 Oct 20 10:38 org.eclipse.equinox.p2.core_2.1.1.v20120113-1346.jar
-rw-r--r-- 1 cltbld cltbld  184966 Oct 20 10:38 org.eclipse.equinox.p2.engine_2.1.1.R37x_v20111003.jar
-rw-r--r-- 1 cltbld cltbld  339099 Oct 20 10:38 org.eclipse.equinox.p2.metadata_2.1.0.v20110815-1419.jar
-rw-r--r-- 1 cltbld cltbld  115721 Oct 20 10:38 org.eclipse.equinox.p2.metadata.repository_1.2.0.v20110815-1419.jar
-rw-r--r-- 1 cltbld cltbld  127900 Oct 20 10:38 org.eclipse.equinox.p2.repository_2.1.1.v20120113-1346.jar
-rw-r--r-- 1 cltbld cltbld  113791 Oct 20 10:38 org.eclipse.equinox.preferences_3.4.2.v20120111-2020.jar
-rw-r--r-- 1 cltbld cltbld  183035 Oct 20 10:38 org.eclipse.equinox.registry_3.5.101.R37x_v20110810-1611.jar
-rw-r--r-- 1 cltbld cltbld  106105 Oct 20 10:38 org.eclipse.equinox.security_1.1.1.R37x_v20110822-1018.jar
-rw-r--r-- 1 cltbld cltbld  257403 Oct 20 10:38 org.eclipse.help_3.5.100.v20110426.jar
-rw-r--r-- 1 cltbld cltbld 1077135 Oct 20 10:38 org.eclipse.jface_3.7.0.v20110928-1505.jar
-rw-r--r-- 1 cltbld cltbld  275458 Oct 20 10:38 org.eclipse.jface.databinding_1.5.0.I20100907-0800.jar
-rw-r--r-- 1 cltbld cltbld  981513 Oct 20 10:38 org.eclipse.jface.text_3.7.2.v20111213-1208.jar
-rw-r--r-- 1 cltbld cltbld 1364409 Oct 20 10:38 org.eclipse.osgi_3.7.2.v20120110-1415.jar
-rw-r--r-- 1 cltbld cltbld   71157 Oct 20 10:38 org.eclipse.osgi.services_3.3.0.v20110513.jar
-rw-r--r-- 1 cltbld cltbld   17332 Oct 20 10:38 org.eclipse.swt_3.7.2.v3740f.jar
-rw-r--r-- 1 cltbld cltbld 2485855 Oct 20 10:38 org.eclipse.swt.gtk.linux.x86_64_3.7.2.v3740f.jar
-rw-r--r-- 1 cltbld cltbld  248183 Oct 20 10:38 org.eclipse.text_3.5.101.v20110928-1504.jar
-rw-r--r-- 1 cltbld cltbld  153179 Oct 20 10:38 org.eclipse.ui_3.7.0.v20110928-1505.jar
-rw-r--r-- 1 cltbld cltbld  159986 Oct 20 10:38 org.eclipse.ui.console_3.5.100.v20111007_r372.jar
-rw-r--r-- 1 cltbld cltbld 2339393 Oct 20 10:38 org.eclipse.ui.ide_3.7.0.v20110928-1505.jar
-rw-r--r-- 1 cltbld cltbld 4166247 Oct 20 10:38 org.eclipse.ui.workbench_3.7.1.v20120104-1859.jar
-rw-r--r-- 1 cltbld cltbld  581960 Oct 20 10:38 org.eclipse.ui.workbench.texteditor_3.7.0.v20110928-1504.jar

./monitor-x86_64/plugins/org.eclipse.core.runtime.compatibility.registry_3.5.0.v20110505:
total 24
-rw-r--r-- 1 cltbld cltbld  1432 Oct 20 10:38 about.html
-rw-r--r-- 1 cltbld cltbld   603 Oct 20 10:38 fragment.properties
drwxr-xr-x 2 cltbld cltbld  4096 Oct 20 10:38 META-INF
-rw-r--r-- 1 cltbld cltbld 12213 Oct 20 10:38 runtime_registry_compatibility.jar

./monitor-x86_64/plugins/org.eclipse.core.runtime.compatibility.registry_3.5.0.v20110505/META-INF:
total 20
-rw-r--r-- 1 cltbld cltbld 5638 Oct 20 10:38 ECLIPSEF.RSA
-rw-r--r-- 1 cltbld cltbld  547 Oct 20 10:38 ECLIPSEF.SF
-rw-r--r-- 1 cltbld cltbld   76 Oct 20 10:38 eclipse.inf
-rw-r--r-- 1 cltbld cltbld  981 Oct 20 10:38 MANIFEST.MF

./monitor-x86_64/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.100.v20110505:
total 196
-rw-r--r-- 1 cltbld cltbld   1445 Oct 20 10:38 about.html
-rw-r--r-- 1 cltbld cltbld 187935 Oct 20 10:38 eclipse_1407.so
-rw-r--r-- 1 cltbld cltbld    620 Oct 20 10:38 launcher.gtk.linux.x86_64.properties
drwxr-xr-x 2 cltbld cltbld   4096 Oct 20 10:38 META-INF

./monitor-x86_64/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.100.v20110505/META-INF:
total 20
-rw-r--r-- 1 cltbld cltbld 5639 Oct 20 10:38 ECLIPSEF.RSA
-rw-r--r-- 1 cltbld cltbld  476 Oct 20 10:38 ECLIPSEF.SF
-rw-r--r-- 1 cltbld cltbld   76 Oct 20 10:38 eclipse.inf
-rw-r--r-- 1 cltbld cltbld  971 Oct 20 10:38 MANIFEST.MF

./pc-bios:
total 164
-rw-r--r-- 1 cltbld cltbld 131072 Oct 20 10:38 bios.bin
-rw-r--r-- 1 cltbld cltbld  35840 Oct 20 10:38 vgabios-cirrus.bin

./x86:
total 1364
-rw-r--r-- 1 cltbld cltbld 1395637 Oct 20 10:38 swt.jar

./x86_64:
total 1464
-rw-r--r-- 1 cltbld cltbld 1495154 Oct 20 10:38 swt.jar
Flags: needinfo?(gbrown)
Hm, looks like the 32-bit libraries are there...I wonder why they're not getting picked up?
I *think* I have built a useful emulator. I am having some trouble verifying the stability of the new emulator, but I think that is because of other factors. I should have another update on progress tomorrow.
This got complicated:

Beginning around Oct 19, several new crashes and intermittent failures started appearing on Cedar's x86 jobs; to correct for this, I am now testing against the Oct 18 nightly.

I can build an emulator -- really a whole SDK -- using something like this on Ubuntu 12.04 with appropriate supporting packages:

(make clean)
repo init -u https://android.googlesource.com/platform/manifest -b <tag>
repo sync
source build/envsetup.sh
lunch sdk-eng
make sdk -j4

The resulting output at out/host/linux-x86/sdk/android-sdk_eng.mozdev_linux-x86/tools mirrors the tools directory found in the Android 18 SDK, including 64 bit binaries like emulator64-x86. I found that installing the whole re-built tools directory seemed more stable than just replacing the emulator binaries.

This works for several (but not all) instances of <tag>, but I'm not sure which <tag> aligns with the Android 18 SDK. Some of these seem like possibilities:

android-4.3_r1
android-4.3_r1.1
android-4.3_r2
android-4.3_r2.1
android-4.3_r2.2
android-4.3_r2.2-cts
android-4.3_r2.3
android-4.3_r3
android-4.3_r3.1
tools_r22
tools_r22.2

I was not able to build tools_r22.2.

tools_r22 built fine but crashed intermittently.

android-4.3_r1.1, with froydnj's patch, appears to provide a stable emulator that works at least as well as the Android 18 SDK version.

The android-4.3_r1.1 emulator seems to be a slightly earlier version than the Android 18 SDK:

[cltbld@talos-linux64-ix-001.test.releng.scl3.mozilla.com ~]$ emulator -version
Android emulator version 22.0 (build_id JWR66Y-eng.mozdev.20131029.110127)
Copyright (C) 2006-2011 The Android Open Source Project and many others.
This program is a derivative of the QEMU CPU emulator (www.qemu.org).

  This software is licensed under the terms of the GNU General Public
  License version 2, as published by the Free Software Foundation, and
  may be copied, distributed, and modified under those terms.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

[cltbld@talos-linux64-ix-001.test.releng.scl3.mozilla.com ~]$ emulator64-x86 -version
Android emulator version 22.0 (build_id JWR66Y-eng.mozdev.20131029.110127)
Copyright (C) 2006-2011 The Android Open Source Project and many others.
This program is a derivative of the QEMU CPU emulator (www.qemu.org).

  This software is licensed under the terms of the GNU General Public
  License version 2, as published by the Free Software Foundation, and
  may be copied, distributed, and modified under those terms.

  This program is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

I will test a couple of newer versions. If they do not work out, we can fall back to android-4.3_r1.1.
So actually, once I corrected for some complicating factors and tested each version "fairly", all of the versions that I tested - 4.3_r1.1, 4.3_r2.3, 4.3_r3.1, and tools_r22 - produced working emulators that were as reliable as the Android 18 SDK. (Ignore my concern about tools_r22 in Comment 87 -- I wasn't testing properly.)

As noted above, the Android 18 SDK emulator reports "Android emulator version 22.0.5.0 (build_id OPENMASTER-757759)". This seems to be tools_r22. (4.3_r1.1, 4.3_r2.3, and 4.3_r3.1 all report "Android emulator version 22.0").

I will file a Release Engineering bug to distribute a new SDK tools directory based on tools_r22, with the emulator patch from Comment 59, of course.
Depends on: 933918
What is the status for this bug after fixing bug 933918?
Blocks: 936226
I think it's fixed. Nathan, ok to resolve this?
Flags: needinfo?(nfroyd)
(In reply to Geoff Brown [:gbrown] from comment #90)
> I think it's fixed. Nathan, ok to resolve this?

Works for me.
Flags: needinfo?(nfroyd)
Patch in comment 26 was integrated with an AOSP build of the emulator and the result distributed for use by the Android x86 emu test environment (bug 933918).
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Whiteboard: Status in comment 60 → Patch in comment 26, status in comment 60
Target Milestone: --- → mozilla28
You need to log in before you can comment on or make changes to this bug.