Closed Bug 834266 Opened 7 years ago Closed 7 years ago

B2G panda tests failing with TimeoutException: socket.timeout OR MarionetteException: Please start a session OR error: [Errno 113] No route to host

Categories

(Testing :: General, defect)

x86
macOS
defect
Not set

Tracking

(firefox21 fixed, b2g18 fixed, b2g18-v1.0.1 fixed)

RESOLVED FIXED
mozilla21
Tracking Status
firefox21 --- fixed
b2g18 --- fixed
b2g18-v1.0.1 --- fixed

People

(Reporter: armenzg, Assigned: jgriffin)

References

Details

Attachments

(2 files)

It is happening intermittently.

I will change the parser so at least it auto-retries.

07:27:39     INFO -  TEST-UNEXPECTED-FAIL | test_kill.py TestKill.test_kill | TimeoutException: socket.timeout
07:27:39     INFO -  TEST-UNEXPECTED-FAIL | test_kill.py TestKill.test_kill | MarionetteException: Please start a session
07:27:39     INFO -  TEST-UNEXPECTED-FAIL | test_kill.py TestKill.test_kill_multiple | TimeoutException: socket.timeout
07:27:39     INFO -  TEST-UNEXPECTED-FAIL | test_kill.py TestKill.test_kill_multiple | MarionetteException: Please start a session
07:27:57     INFO -  TEST-UNEXPECTED-FAIL | test_killall.py TestKillAll.test_kill_all | error: [Errno 113] No route to host
07:27:57     INFO -  TEST-UNEXPECTED-FAIL | test_killall.py TestKillAll.test_kill_all_twice | error: [Errno 113] No route to host
07:27:57     INFO -  TEST-UNEXPECTED-FAIL | test_killall.py TestKillAll.test_kill_all_with_no_apps_running | error: [Errno 113] No route to host
07:28:00     INFO -  TEST-UNEXPECTED-FAIL | test_launch_entrypoint.py TestLaunchEntrypoint.test_launch_entrypoint | error: [Errno 113] No route to host
07:28:03     INFO -  TEST-UNEXPECTED-FAIL | test_lock_screen.py TestLockScreen.test_lock_unlock | error: [Errno 113] No route to host
07:28:06     INFO -  TEST-UNEXPECTED-FAIL | test_browser_lan.py TestBrowserLAN.test_browser_lan | error: [Errno 113] No route to host
07:28:09     INFO -  TEST-UNEXPECTED-FAIL | test_add_new_contact.py TestContacts.test_add_new_contact | error: [Errno 113] No route to host
07:28:12     INFO -  TEST-UNEXPECTED-FAIL | test_edit_contact.py TestContacts.test_edit_contact | error: [Errno 113] No route to host
07:28:15     INFO -  TEST-UNEXPECTED-FAIL | test_clock_all_items_present_new_alarm.py TestClockTestAllItemsPresentNewAlarm.test_all_items_present_new_alarm | error: [Errno 113] No route to host
07:28:21     INFO -  TEST-UNEXPECTED-FAIL | test_clock_create_new_alarm.py TestClockCreateNewAlarm.test_clock_create_new_alarm | error: [Errno 113] No route to host
07:28:21     INFO -  TEST-UNEXPECTED-FAIL | test_clock_create_new_alarm.py TestClockCreateNewAlarm.test_clock_set_alarm_label | error: [Errno 113] No route to host
07:28:24     INFO -  TEST-UNEXPECTED-FAIL | test_clock_delete_alarm.py TestClockDeleteAlarm.test_clock_delete_alarm | error: [Errno 113] No route to host
07:28:30     INFO -  TEST-UNEXPECTED-FAIL | test_clock_switch_clock_type.py TestClockSwitchClockType.test_clock_show_time_date | error: [Errno 113] No route to host
07:28:30     INFO -  TEST-UNEXPECTED-FAIL | test_clock_switch_clock_type.py TestClockSwitchClockType.test_clock_switch_clock_type | error: [Errno 113] No route to host
07:28:33     INFO -  TEST-UNEXPECTED-FAIL | test_clock_turn_on_off_alarm.py TestClockTurnOnOffAlarm.test_clock_turn_on_off_alarm | error: [Errno 113] No route to host
07:28:42     INFO -  TEST-UNEXPECTED-FAIL | test_cards_view.py TestCardsView.test_cards_view | error: [Errno 113] No route to host
07:28:42     INFO -  TEST-UNEXPECTED-FAIL | test_cards_view.py TestCardsView.test_kill_app_from_cards_view | error: [Errno 113] No route to host
07:28:42     INFO -  TEST-UNEXPECTED-FAIL | test_cards_view.py TestCardsView.test_that_app_can_be_launched_from_cards_view | error: [Errno 113] No route to host
07:28:45     INFO -  TEST-UNEXPECTED-FAIL | test_delete_app.py TestDeleteApp.test_delete_app | error: [Errno 113] No route to host
07:28:48     INFO -  TEST-UNEXPECTED-FAIL | test_keyboard.py TestKeyboard.test_keyboard_basic | error: [Errno 113] No route to host
07:28:51     INFO -  TEST-UNEXPECTED-FAIL | test_launch_app.py TestLaunchApp.test_launch_app | error: [Errno 113] No route to host
07:28:54     INFO -  TEST-UNEXPECTED-FAIL | test_wallpaper.py TestWallpaper.test_change_wallpaper | error: [Errno 113] No route to host
07:28:54    ERROR - Return code: 10
Blocks: 834268
Switching to new keyword (see https://groups.google.com/d/msg/mozilla.dev.platform/dy-4uHJ8p0Y/4tkpVjGWaYMJ)
Whiteboard: [orange][intermittent]
I can reproduce this locally with an older gaia, which implies it's a gecko change that caused it.  The logcat error I see when this happens is:

E/GeckoConsole(   43): [JavaScript Error: "uncaught exception: [object Object]"]
E/GeckoConsole(  198): Could not read chrome manifest 'file:///system/b2g/chrome.manifest'.
This is looking like a probable failure in Marionette in switch_to_frame.
Blocks: 802317
I've narrowed the problem down to element comparison in getKnownElements. We have two HTMLDocument references which point to the same object, but a == b returns false.  a.isEqualsNode(b) return true, however.
Some gecko change has caused this behavior; it doesn't appear to be a Marionette bug.  I'm trying to bisect, but it will be slow.

A workaround is to change a == b to a.isEqualsNode(b), however, that performs a less strict comparison, and it would probably be good to get to the bottom of whatever has changed.
I've seen this problem in builds all the way back to Jan 15th, although it isn't consistent.  I'm going to make a patch with the isEqualsNode fix instead of spending any more time trying to hunt it down.
Assignee: nobody → jgriffin
I am unable to launch Marketplace from within a gaia-ui-test. Every time I run the test I get a NoSuchFrameException which seems to be coming from switch_to_frame. Full stack trace available at: http://pastebin.mozilla.org/2093056.

The test I am running is: https://github.com/mozilla/gaia-ui-tests/blob/master/gaiatest/tests/marketplace/test_search_marketplace_and_install_app.py

I am on a build downloaded from https://releases.mozilla.com/b2g/latest/unagi_2013-01-28_eng.zip, and the latest Gaia just built from master.

Note that I seem to be able to reproduce this every time, although the frame id changes from run to run.
This fixes the problem locally; I'll run it through try.  Theoretically, this could cause problems, if we had two identical but different frames.  In Gaia, this won't happen though so I think this is OK for now.
Attachment #707318 - Flags: review?(mdas)
Attachment #707318 - Flags: review?(mdas) → review+
I thought I had removed "intermittent" in the past.

Results (soon):
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&jobname=b2g_&noignore=1&rev=abfd67b9c024
Summary: [intermittent] B2G panda tests failing with TimeoutException: socket.timeout OR MarionetteException: Please start a session OR error: [Errno 113] No route to host → B2G panda tests failing with TimeoutException: socket.timeout OR MarionetteException: Please start a session OR error: [Errno 113] No route to host
https://hg.mozilla.org/mozilla-central/rev/abfd67b9c024
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla21
> We have two HTMLDocument references which point to the same object, but a == b returns
> false. 

Why do you think they're references to the same object?

There's a good chance at least for the first hunk that one of these is a now-unloaded document, no?
(In reply to Boris Zbarsky (:bz) from comment #15)
> > We have two HTMLDocument references which point to the same object, but a == b returns
> > false. 
> 
> Why do you think they're references to the same object?
> 
> There's a good chance at least for the first hunk that one of these is a
> now-unloaded document, no?

In this case, no...they both point to the one and only Gaia system app, which is never unloaded/reloaded.  The code with == used to work up until late January, when something changed, but bisecting this is very time consuming, since it requires local B2G builds.
> they both point to the one and only Gaia system app

Is it possible that one of them points to a temporary about:blank?

Alternately, is it possible that one is an Xray and one isn't?  Those are the only ways I can think of == testing false there: either Xray vs not, or different underlying C++ objects...
Or a serious bug, in which case this really needs to be diagnosed and fixed (as opposed to worked around). We really need to know either what regressed this or why it's happening.
(In reply to Boris Zbarsky (:bz) from comment #17)
> > they both point to the one and only Gaia system app
> 
> Is it possible that one of them points to a temporary about:blank?

No, both references are created long after the doc is fully loaded.

> 
> Alternately, is it possible that one is an Xray and one isn't?  Those are
> the only ways I can think of == testing false there: either Xray vs not, or
> different underlying C++ objects...

It's possible.  How could I test for this?
> It's possible.  How could I test for this?

At the point when the == check fails, try also comparing these:

  LHS.wrappedJSObject == RHS
  LHS == RHS.wrappedJSObject

and seeing if either of those tests true?
LHS == RHS.wrappedJSObject works, at least in this particular case.

Is there any way I can determine that I need to use wrappedJSObject, or should I just:

if (LHS == RHS || LHS.wrappedJSObject == RHS || LHS == RHS.wrappedJSObject)
> Is there any way I can determine that I need to use wrappedJSObject

Components.utils.isXrayWrapper.

But I think you can also XPCNativeWrapper.unwrap(LHS) == XPCNativeWrapper.unwrap(RHS) and that should work too...
Will put up a new patch.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
A better fix, courtesy of :bz.  This passes locally; I'll run it through try.
Attachment #709311 - Flags: review?(mdas)
Comment on attachment 709311 [details] [diff] [review]
Use XPCNativeWrapper to compare elements,

Review of attachment 709311 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, we might want to add a comment explaining the reason we're using XPCNativeWrapper, pointing to this bug, so we don't accidentally reintroduce this bug later.
Attachment #709311 - Flags: review?(mdas) → review+
https://hg.mozilla.org/mozilla-central/rev/92970e076a4d
Status: REOPENED → RESOLVED
Closed: 7 years ago7 years ago
Resolution: --- → FIXED
Batch edit: Bugs fixed on b2g18 after 1/25 merge to v1.0 branch are fixed on v1.0.1 branch.
You need to log in before you can comment on or make changes to this bug.