Closed
Bug 987406
Opened 11 years ago
Closed 11 years ago
Add ssltunnel to xre.zip bundle used for b2g tests
Categories
(Release Engineering :: General, defect, P2)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: mt, Assigned: jgriffin)
References
()
Details
(Keywords: ateam-b2g-task, Whiteboard: [leave open])
Attachments
(4 files)
1.27 KB,
patch
|
ahal
:
review+
jgriffin
:
checked-in+
|
Details | Diff | Splinter Review |
1.28 KB,
patch
|
ahal
:
review+
jgriffin
:
checked-in+
|
Details | Diff | Splinter Review |
1.20 KB,
patch
|
mozilla
:
review+
jgriffin
:
checked-in+
|
Details | Diff | Splinter Review |
895 bytes,
patch
|
jgriffin
:
review+
|
Details | Diff | Splinter Review |
Running tests that use https resources means that ssltunnel needs to be run. It is not present in the current build of the xre.zip, so cannot run.
Bug 907770 includes the gecko-side changes for this. That requires the xre bundle, currently at http://runtime-binaries.pvt.build.mozilla.org/tooltool/sha512/cba263cef46b57585334f4b71fbbd15ce740fa4b7260571a9f7a76f8f0d6b492b93b01523cb01ee54697cc9b1de1ccc8e89ad64da95a0ea31e0f119fe744c09f to be updated.
Bug 742597 caused a bundle to be created that would probably suffice, I just want to make sure that there aren't other things here that need to be considered.
Assignee | ||
Comment 1•11 years ago
|
||
The tegra bundle won't work for B2G, because it's too old, and its httpd.js won't match in-tree server.js which is currently bundled in tests.zip.
I think we should add ssltunnel from a linux32 desktop build to the existing xre.zip from tooltool.
Assignee | ||
Comment 2•11 years ago
|
||
Actually, we'll need linux32, linux64, and osx versions of this.
Assignee | ||
Comment 3•11 years ago
|
||
The most recent copy of xre.zip comes from xulrunner-sdk-26. I'll take this and add ssltunnel from here: http://ftp.mozilla.org/pub/mozilla.org/firefox/releases/26.0/
Assignee | ||
Updated•11 years ago
|
Assignee: nobody → jgriffin
Assignee | ||
Comment 4•11 years ago
|
||
For the record, the xre.zip in bug 987957 is the combination of http://runtime-binaries.pvt.build.mozilla.org/tooltool/sha512/d4a0da54e75c27cd2f535e66b586f119ef08b3bde4a9eee03662d296b3434189c542c0a7e7a75954030c04396a9823e22e1f884f5d87c0f4017944cd50ff38de (which we're currently using on linux64 gaia unit tests) with ssltunnel from http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/26.0-candidates/build2/linux-x86_64/en-US/firefox-26.0.tests.zip.
If this works, I'll create linux32 and osx packages as well.
Comment 5•11 years ago
|
||
Should be fine, ssltunnel hasn't changed in a while. (I wish we just had implemented this all in Python.)
Assignee | ||
Comment 6•11 years ago
|
||
I'll update desktop configs in a separate patch.
Attachment #8398074 -
Flags: review?(ahalberstadt)
Comment 7•11 years ago
|
||
Comment on attachment 8398074 [details] [diff] [review]
Use updated xre.zip in emulator unittests,
Review of attachment 8398074 [details] [diff] [review]:
-----------------------------------------------------------------
Lgtm.. I guess :)
Attachment #8398074 -
Flags: review?(ahalberstadt) → review+
Assignee | ||
Comment 8•11 years ago
|
||
Comment on attachment 8398074 [details] [diff] [review]
Use updated xre.zip in emulator unittests,
https://hg.mozilla.org/build/mozharness/rev/bce52213ed6b
Will watch cedar for bustage.
Attachment #8398074 -
Flags: checked-in+
Assignee | ||
Comment 9•11 years ago
|
||
That definitely didn't work:
15:34:32 INFO - MochitestServer : launching [u'/builds/slave/test/build/xre/bin/xpcshell', '-g', '/builds/slave/test/build/xre/bin', '-v', '170', '-f', '/builds/slave/test/build/xre/bin/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmp60hZKU'; const _SERVER_PORT = '8888'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', './server.js']
15:34:32 INFO - runtests.py | Server pid: 2392
15:36:02 WARNING - TEST-UNEXPECTED-FAIL | runtests.py | Timed out while waiting for server startup.
15:37:06 INFO - Traceback (most recent call last):
15:37:06 INFO - File "runtestsb2g.py", line 339, in run_remote_mochitests
15:37:06 INFO - retVal = mochitest.run_tests(options)
15:37:06 INFO - File "runtestsb2g.py", line 117, in run_tests
15:37:06 INFO - self.startWebServer(options)
15:37:06 INFO - File "runtestsb2g.py", line 217, in startWebServer
15:37:06 INFO - self.server.ensureReady(90)
15:37:06 INFO - File "/builds/slave/test/build/tests/mochitest/runtests.py", line 190, in ensureReady
15:37:06 INFO - sys.exit(1)
15:37:06 INFO - SystemExit: 1
15:38:09 INFO - Automation Error: Exception caught while running tests
I've backed out and will investigate.
Assignee | ||
Comment 10•11 years ago
|
||
This failed because I failed to add httpd.js to the new xre.zip. :( I've fixed this, but it will require another tooltool upload.
Comment 11•11 years ago
|
||
in production.
Assignee | ||
Comment 12•11 years ago
|
||
ok, trying again: https://hg.mozilla.org/build/mozharness/rev/e2da84d8c986
Reporter | ||
Comment 13•11 years ago
|
||
No screams yet? Trying this out: https://tbpl.mozilla.org/?tree=Try&rev=e2aa932bebb7
Assignee | ||
Comment 14•11 years ago
|
||
You won't see it in a try job yet; mozharness changes have to be explicitly moved into production before they take effect globally. I'm watching some jobs on cedar (which works on pre-production changes), and if they go green, I'll put this patch into production.
Assignee | ||
Comment 15•11 years ago
|
||
This patch works 80% of the time!
Seriously.
It works for most emulator tests, but unfortunately it fails on reftests running on the old r3 mac minis. Apparently those machines are 32-bit, since this version of xre.zip fails with:
OSError: [Errno 8] Exec format error
We're phasing tests out on r3 mac minis, but we're not quite there, so I'll have to back this out and try again with a 32-bit version.
Assignee | ||
Comment 16•11 years ago
|
||
(The good news is that the 64-bit xre.zip works fine and I'll use that for linux64 b2g desktop builds.)
Assignee | ||
Comment 17•11 years ago
|
||
Landed the 32-bit xre.zip change: https://hg.mozilla.org/build/mozharness/rev/52636b92cbfd
Watching cypress...
Assignee | ||
Comment 18•11 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #17)
> Landed the 32-bit xre.zip change:
> https://hg.mozilla.org/build/mozharness/rev/52636b92cbfd
>
> Watching cypress...
Great success! I'll merge to production on Monday a.m.
Comment 19•11 years ago
|
||
in production.
Assignee | ||
Comment 20•11 years ago
|
||
So, the emulators are done, I now need to fix the B2G desktop build tests.
Reporter | ||
Comment 21•11 years ago
|
||
The desktop builds should be OK, since they are compiling xpcshell and ssltunnel in a compatible form.
Assignee | ||
Comment 22•11 years ago
|
||
Ah, right. In that case, I'll close this; re-open if needed.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 23•11 years ago
|
||
Encountering issues with the b2g emulator: https://tbpl.mozilla.org/?tree=Try&rev=2b5bb0340054
The actual problem seems consistent:
10:35:59 INFO - INFO | runtests.py | SSL tunnel pid: 2413
10:35:59 INFO - Failed to init NSS:
This suggests that the xre bundle (http://runtime-binaries.pvt.build.mozilla.org/tooltool/sha512/ce1ad86b21d534bd706cd70f1d763b9ca1cfd0a1ea4d519864c961e202c36ccf48bcfa9c572113076ee8b3d0770f60b8967b7620b4adaad531f47b993eb57912) doesn't contain a version of ssltunnel that runs on the emulator machines. Given that it's a 64bit Ubuntu machine, I'm at a loss to explain this.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Updated•11 years ago
|
Keywords: ateam-b2g-task
Priority: -- → P2
Assignee | ||
Comment 24•11 years ago
|
||
We're off the mac minis now, so will try bumping our xre.zip to 64-bit.
Assignee | ||
Comment 25•11 years ago
|
||
Attachment #8407778 -
Flags: review?(ahalberstadt)
Updated•11 years ago
|
Attachment #8407778 -
Flags: review?(ahalberstadt) → review+
Assignee | ||
Comment 26•11 years ago
|
||
Comment on attachment 8407778 [details] [diff] [review]
Use a 64-bit version of xre.zip on emulator unittests,
https://hg.mozilla.org/build/mozharness/rev/2a89305f4a67
Attachment #8407778 -
Flags: checked-in+
Comment 27•11 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #26)
> Comment on attachment 8407778 [details] [diff] [review]
> Use a 64-bit version of xre.zip on emulator unittests,
>
> https://hg.mozilla.org/build/mozharness/rev/2a89305f4a67
In production
Assignee | ||
Comment 28•11 years ago
|
||
Martin, can you try again? The 64-bit version is now live.
Flags: needinfo?(martin.thomson)
Reporter | ||
Comment 29•11 years ago
|
||
Bug 907770 needs a rebase. It seems like these files see a lot of churn. Let me see if I can't get a try run overnight.
Flags: needinfo?(martin.thomson)
Reporter | ||
Comment 30•11 years ago
|
||
Reporter | ||
Comment 31•11 years ago
|
||
Looks like our old friend: https://tbpl.mozilla.org/php/getParsedLog.php?id=37968603&tree=Try&full=1
17:39:41 INFO - INFO | runtests.py | SSL tunnel pid: 2420
17:39:41 INFO - Failed to init NSS: None
A few builds seem to be burning here, but it doesn't look like anything else is broken.
Assignee | ||
Comment 32•11 years ago
|
||
I can reproduce this locally; it doesn't seem to be a problem with ssltunnel itself, but how we're invoking. Digging more...
Reporter | ||
Comment 33•11 years ago
|
||
I don't know if this helps, but there's a possibility that the emulator needs to use a different type of certificate DB. Android insists on having the DB in the "new" form, the desktop tests in the "old" form. Maybe my intelligence on this was wrong:
if self.certdbNew:
android and b2g use the new DB formats exclusively
certdbPath = "sql:" + options.profilePath
else:
# desktop seems to use the old
certdbPath = options.profilePath
Reporter | ||
Comment 34•11 years ago
|
||
I hope that the missing comment in the above is obvious - I have to remember that tab thing.
Assignee | ||
Comment 35•11 years ago
|
||
At least one of the problems (hopefully the only one) is that we're producing an invalid ssltunnel.cfg:
httpproxy:1
certdbdir:None
forward:127.0.0.1:8888
websocketserver:10.0.2.2:9988
listen:*:4443:pgo server certificate
websocketserver:10.0.2.2:9988
listen:*:4443:pgo server certificate
listen:self-signed.example.com:443:4443:selfsigned
...
We don't include a real certdbdir because we're not passing --certificate-path to runtestsb2g.py. This will need a mozharness fix, and then an in-tree config fix.
With these things fixed locally, everything works for me on my machine, so we'll hope the same is true for buildbot.
Assignee | ||
Comment 36•11 years ago
|
||
This mozharness part of the fix.
Attachment #8408735 -
Flags: review?(ahalberstadt)
Assignee | ||
Comment 37•11 years ago
|
||
The in-tree part of the fix.
Attachment #8408736 -
Flags: review?(ahalberstadt)
Assignee | ||
Comment 38•11 years ago
|
||
Comment on attachment 8408735 [details] [diff] [review]
Set up certificate_path processing,
Review of attachment 8408735 [details] [diff] [review]:
-----------------------------------------------------------------
ahal is on Easter break today
Attachment #8408735 -
Flags: review?(ahalberstadt) → review?(aki)
Assignee | ||
Updated•11 years ago
|
Attachment #8408736 -
Flags: review?(ahalberstadt) → review+
Updated•11 years ago
|
Attachment #8408735 -
Flags: review?(aki) → review+
Assignee | ||
Comment 39•11 years ago
|
||
Comment on attachment 8408735 [details] [diff] [review]
Set up certificate_path processing,
https://hg.mozilla.org/build/mozharness/rev/d92a223fe008
Attachment #8408735 -
Flags: checked-in+
Reporter | ||
Comment 40•11 years ago
|
||
Should a respin of the try run work at this point?
Assignee | ||
Comment 41•11 years ago
|
||
(In reply to Martin Thomson [:mt] from comment #40)
> Should a respin of the try run work at this point?
No. I need to verify the above landing in staging on cypress, push it to production, then land the gecko bits. I'll ping you when it's ready to re-test.
Assignee | ||
Comment 42•11 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #39)
> Comment on attachment 8408735 [details] [diff] [review]
> Set up certificate_path processing,
>
> https://hg.mozilla.org/build/mozharness/rev/d92a223fe008
in production
Assignee | ||
Comment 43•11 years ago
|
||
Martin, the mozharness piece is in production now. I haven't been able to land the gecko piece, because m-i is closed, but if you add this patch to your patch queue, you could try it out with a try job anyway:
https://bug987406.bugzilla.mozilla.org/attachment.cgi?id=8408736
Flags: needinfo?(martin.thomson)
Assignee | ||
Comment 44•11 years ago
|
||
Ah, m-i just opened and I landed it: https://hg.mozilla.org/integration/mozilla-inbound/rev/5d9fb05d9d12
You'll still need to include this patch in your queue, or rebase on top of m-i before you push to try.
Assignee | ||
Updated•11 years ago
|
Whiteboard: [leave open]
Reporter | ||
Comment 45•11 years ago
|
||
Rebase, rebase, rebase...
https://tbpl.mozilla.org/?tree=Try&rev=f7c3e60b8541
Flags: needinfo?(martin.thomson)
Comment 46•11 years ago
|
||
Reporter | ||
Comment 47•11 years ago
|
||
Good news, it looks like ssltunnel runs. Bad news, the test still times out. But it does seem to at least make progress, so it might be a test issue. Signs are good for this bug.
Assignee | ||
Comment 48•11 years ago
|
||
I also saw that we're producing an ssltunnel.cfg in which a couple of lines are repeated:
websocketserver:10.0.2.2:9988
listen:*:4443:pgo server certificate
websocketserver:10.0.2.2:9988
listen:*:4443:pgo server certificate
Not sure if this would cause any problem, though.
Comment 49•11 years ago
|
||
mozharness patch is in production: http://hg.mozilla.org/build/mozharness/rev/c9615cf1645a :)
Reporter | ||
Comment 50•11 years ago
|
||
(In reply to Jonathan Griffin (:jgriffin) from comment #48)
> I also saw that we're producing an ssltunnel.cfg in which a couple of lines
> are repeated:
>
> websocketserver:10.0.2.2:9988
> listen:*:4443:pgo server certificate
> websocketserver:10.0.2.2:9988
> listen:*:4443:pgo server certificate
>
> Not sure if this would cause any problem, though.
I wouldn't have thought so. I don't see how that would happen using either trunk or my modified code unless there is a python bug or something equally perverse. Especially if those are the only lines duplicated, see: https://hg.mozilla.org/try/rev/5328e2ef6ad2#l2.197 or for trunk: https://hg.mozilla.org/try/rev/5328e2ef6ad2#l2.740
Assignee | ||
Comment 51•11 years ago
|
||
You're right; this was an artifact of how I was retrieving a copy of the file during testing.
Reporter | ||
Comment 52•11 years ago
|
||
Dammit, I should have noticed this more quickly. The test that fails fails because it doesn't run on e10s. Running on desktop with --e10s causes it to time out in exactly the same way. I'll need to disable that test on b2g for that reason. ssltunnel is working perfectly.
Reporter | ||
Comment 53•11 years ago
|
||
Reporter | ||
Comment 54•11 years ago
|
||
https://tbpl.mozilla.org/?tree=Try&rev=4f9132738b08
Yes, it works.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 11 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 55•11 years ago
|
||
\o/
Updated•7 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•