Closed Bug 1456209 Opened 2 years ago Closed 2 years ago

Make GeckoSessionTestRule more reliable on automation

Categories

(GeckoView :: General, defect, P1)

All
Android
defect

Tracking

(firefox61 fixed)

RESOLVED FIXED
mozilla61
Tracking Status
firefox61 --- fixed

People

(Reporter: jchen, Assigned: jchen)

References

Details

Attachments

(3 files)

Fix several aspects of GeckoSessionTestRule to make it more reliable when running under the ARM emulators on automation.
Comment on attachment 8970342 [details]
Bug 1456209 - 3. Scale timeouts based on device;

https://reviewboard.mozilla.org/r/239134/#review244802

::: mobile/android/geckoview/src/androidTest/java/org/mozilla/geckoview/test/rule/GeckoSessionTestRule.java:80
(Diff revision 1)
>  public class GeckoSessionTestRule extends UiThreadTestRule {
>      private static final String LOGTAG = "GeckoSessionTestRule";
>  
>      private static final long DEFAULT_TIMEOUT_MILLIS = 10000;
> +    private static final long DEFAULT_ARM_DEVICE_MILLIS = 30000;
> +    private static final long DEFAULT_ARM_EMULATOR_MILLIS = 120000;

Seems like a long time. If you can reliably use something shorter, I encourage it.

::: mobile/android/geckoview/src/androidTest/java/org/mozilla/geckoview/test/rule/GeckoSessionTestRule.java:436
(Diff revision 1)
>          public boolean isDebugging() {
>              return Debug.isDebuggerConnected();
>          }
> +
> +        public boolean isEmulator() {
> +            return "generic".equals(Build.DEVICE) || Build.DEVICE.startsWith("generic_");

Why not startsWith("generic")?
Attachment #8970342 - Flags: review?(gbrown) → review+
(In reply to Geoff Brown [:gbrown] from comment #4)
> Comment on attachment 8970342 [details]
> Bug 1456209 - 3. Scale timeouts based on device;
> 
> >      private static final long DEFAULT_TIMEOUT_MILLIS = 10000;
> > +    private static final long DEFAULT_ARM_DEVICE_MILLIS = 30000;
> > +    private static final long DEFAULT_ARM_EMULATOR_MILLIS = 120000;
> 
> Seems like a long time. If you can reliably use something shorter, I
> encourage it.

I measured a debug test run on ARM emulator, and the actual maximum wait was about 60s, so a 120s timeout seems to be a safe choice.

> :::
> mobile/android/geckoview/src/androidTest/java/org/mozilla/geckoview/test/
> rule/GeckoSessionTestRule.java:436
> (Diff revision 1)
> >          public boolean isDebugging() {
> >              return Debug.isDebuggerConnected();
> >          }
> > +
> > +        public boolean isEmulator() {
> > +            return "generic".equals(Build.DEVICE) || Build.DEVICE.startsWith("generic_");
> 
> Why not startsWith("generic")?

Just to be safe I guess. The emulator devices are either "generic", or "generic_foo".
Comment on attachment 8970341 [details]
Bug 1456209 - 2. Miscellaneous GeckoSessionTestRule changes;

https://reviewboard.mozilla.org/r/239132/#review244880
Attachment #8970341 - Flags: review+
Pushed by nchen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/041289f66ad9
1. Fix test license headers; r=jchen
https://hg.mozilla.org/integration/autoland/rev/2a11c1864d44
2. Miscellaneous GeckoSessionTestRule changes; r=jchen
https://hg.mozilla.org/integration/autoland/rev/b89f665eed0f
3. Scale timeouts based on device; r=gbrown
Backed out 3 changesets (bug 1456209) for bustage in /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoSession.java:22 on a CLOSED TREE

Problematic push: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=d3dc4f7a2080215ba14b0cc8e15a4e323c34a88a&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception
Bustage: https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&selectedJob=175266838
Backout: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=d3dc4f7a2080215ba14b0cc8e15a4e323c34a88a&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception
Log: https://treeherder.mozilla.org/logviewer.html#?job_id=175266838&repo=autoland&lineNumber=34678

[task 2018-04-24T05:57:09.732Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoSession.java:22: error: cannot find symbol
[task 2018-04-24T05:57:09.732Z] 05:57:09     INFO -  import org.mozilla.gecko.IGeckoEditableParent;
[task 2018-04-24T05:57:09.732Z] 05:57:09     INFO -                          ^
[task 2018-04-24T05:57:09.732Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.732Z] 05:57:09     INFO -    location: package org.mozilla.gecko
[task 2018-04-24T05:57:09.772Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/SessionTextInput.java:10: error: cannot find symbol
[task 2018-04-24T05:57:09.772Z] 05:57:09     INFO -  import org.mozilla.gecko.IGeckoEditableParent;
[task 2018-04-24T05:57:09.773Z] 05:57:09     INFO -                          ^
[task 2018-04-24T05:57:09.773Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.774Z] 05:57:09     INFO -    location: package org.mozilla.gecko
[task 2018-04-24T05:57:09.793Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:16: error: cannot find symbol
[task 2018-04-24T05:57:09.793Z] 05:57:09     INFO -  import org.mozilla.gecko.IGeckoEditableChild;
[task 2018-04-24T05:57:09.793Z] 05:57:09     INFO -                          ^
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -    symbol:   class IGeckoEditableChild
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -    location: package org.mozilla.gecko
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:17: error: cannot find symbol
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -  import org.mozilla.gecko.IGeckoEditableParent;
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -                          ^
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -    location: package org.mozilla.gecko
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:53: error: package IGeckoEditableParent does not exist
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -      extends IGeckoEditableParent.Stub
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -                                  ^
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/GeckoEditableChild.java:24: error: cannot find symbol
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -  public final class GeckoEditableChild extends JNIObject implements IGeckoEditableChild {
[task 2018-04-24T05:57:09.796Z] 05:57:09     INFO -                                                                     ^
[task 2018-04-24T05:57:09.797Z] 05:57:09     INFO -    symbol: class IGeckoEditableChild
[task 2018-04-24T05:57:09.803Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoSession.java:577: error: cannot find symbol
[task 2018-04-24T05:57:09.803Z] 05:57:09     INFO -          public native void attachEditable(IGeckoEditableParent parent,
[task 2018-04-24T05:57:09.803Z] 05:57:09     INFO -                                            ^
[task 2018-04-24T05:57:09.804Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.804Z] 05:57:09     INFO -    location: class Window
[task 2018-04-24T05:57:09.813Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:76: error: cannot find symbol
[task 2018-04-24T05:57:09.814Z] 05:57:09     INFO -      /* package */ IGeckoEditableChild mDefaultChild; // Used by IC thread.
[task 2018-04-24T05:57:09.815Z] 05:57:09     INFO -                    ^
[task 2018-04-24T05:57:09.815Z] 05:57:09     INFO -    symbol:   class IGeckoEditableChild
[task 2018-04-24T05:57:09.816Z] 05:57:09     INFO -    location: class GeckoEditable
[task 2018-04-24T05:57:09.816Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:78: error: cannot find symbol
[task 2018-04-24T05:57:09.816Z] 05:57:09     INFO -      /* package */ IGeckoEditableChild mFocusedChild; // Used by IC thread.
[task 2018-04-24T05:57:09.817Z] 05:57:09     INFO -                    ^
[task 2018-04-24T05:57:09.817Z] 05:57:09     INFO -    symbol:   class IGeckoEditableChild
[task 2018-04-24T05:57:09.817Z] 05:57:09     INFO -    location: class GeckoEditable
[task 2018-04-24T05:57:09.818Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:108: error: cannot find symbol
[task 2018-04-24T05:57:09.818Z] 05:57:09     INFO -      private void onKeyEvent(final IGeckoEditableChild child, KeyEvent event, int action,
[task 2018-04-24T05:57:09.818Z] 05:57:09     INFO -                                    ^
[task 2018-04-24T05:57:09.819Z] 05:57:09     INFO -    symbol:   class IGeckoEditableChild
[task 2018-04-24T05:57:09.819Z] 05:57:09     INFO -    location: class GeckoEditable
[task 2018-04-24T05:57:09.819Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:626: error: cannot find symbol
[task 2018-04-24T05:57:09.819Z] 05:57:09     INFO -      public void setDefaultEditableChild(final IGeckoEditableChild child) {
[task 2018-04-24T05:57:09.820Z] 05:57:09     INFO -                                                ^
[task 2018-04-24T05:57:09.820Z] 05:57:09     INFO -    symbol:   class IGeckoEditableChild
[task 2018-04-24T05:57:09.820Z] 05:57:09     INFO -    location: class GeckoEditable
[task 2018-04-24T05:57:09.820Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoEditable.java:1129: error: cannot find symbol
[task 2018-04-24T05:57:09.820Z] 05:57:09     INFO -      public void notifyIME(final IGeckoEditableChild child, final int type) {
[task 2018-04-24T05:57:09.820Z] 05:57:09     INFO -                                  ^
[task 2018-04-24T05:57:09.821Z] 05:57:09     INFO -    symbol:   class IGeckoEditableChild
[task 2018-04-24T05:57:09.821Z] 05:57:09     INFO -    location: class GeckoEditable
[task 2018-04-24T05:57:09.821Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/GeckoEditableChild.java:73: error: cannot find symbol
[task 2018-04-24T05:57:09.821Z] 05:57:09     INFO -      private final IGeckoEditableParent mEditableParent;
[task 2018-04-24T05:57:09.821Z] 05:57:09     INFO -                    ^
[task 2018-04-24T05:57:09.821Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.821Z] 05:57:09     INFO -    location: class GeckoEditableChild
[task 2018-04-24T05:57:09.822Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/GeckoEditableChild.java:74: error: cannot find symbol
[task 2018-04-24T05:57:09.822Z] 05:57:09     INFO -      private final IGeckoEditableChild mEditableChild;
[task 2018-04-24T05:57:09.822Z] 05:57:09     INFO -                    ^
[task 2018-04-24T05:57:09.822Z] 05:57:09     INFO -    symbol:   class IGeckoEditableChild
[task 2018-04-24T05:57:09.822Z] 05:57:09     INFO -    location: class GeckoEditableChild
[task 2018-04-24T05:57:09.823Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/GeckoEditableChild.java:79: error: cannot find symbol
[task 2018-04-24T05:57:09.823Z] 05:57:09     INFO -      public GeckoEditableChild(final IGeckoEditableParent editableParent) {
[task 2018-04-24T05:57:09.823Z] 05:57:09     INFO -                                      ^
[task 2018-04-24T05:57:09.823Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.824Z] 05:57:09     INFO -    location: class GeckoEditableChild
[task 2018-04-24T05:57:09.945Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/GeckoEditableChild.java:31: error: package IGeckoEditableChild does not exist
[task 2018-04-24T05:57:09.945Z] 05:57:09     INFO -      private final class RemoteChild extends IGeckoEditableChild.Stub {
[task 2018-04-24T05:57:09.946Z] 05:57:09     INFO -                                                                 ^
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/process/GeckoProcessManager.java:9: error: cannot find symbol
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -  import org.mozilla.gecko.IGeckoEditableParent;
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -                          ^
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -    location: package org.mozilla.gecko
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/process/GeckoProcessManager.java:28: error: package IProcessManager does not exist
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -  public final class GeckoProcessManager extends IProcessManager.Stub {
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -                                                                ^
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/process/GeckoProcessManager.java:37: error: cannot find symbol
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -      private static native IGeckoEditableParent nativeGetEditableParent(long contentId,
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -                            ^
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -    location: class GeckoProcessManager
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/process/GeckoProcessManager.java:41: error: cannot find symbol
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -      public IGeckoEditableParent getEditableParent(final long contentId, final long tabId) {
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -             ^
[task 2018-04-24T05:57:09.959Z] 05:57:09     INFO -    symbol:   class IGeckoEditableParent
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -    location: class GeckoProcessManager
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/process/GeckoProcessManager.java:51: error: cannot find symbol
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -          private IChildProcess mChild;
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -                  ^
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -    symbol:   class IChildProcess
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -    location: class ChildConnection
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -  /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/process/GeckoProcessManager.java:69: error: cannot find symbol
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -          public synchronized IChildProcess bind() {
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -                              ^
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -    symbol:   class IChildProcess
[task 2018-04-24T05:57:09.960Z] 05:57:09     INFO -    location: class ChildConnection
[task 2018-04-24T05:57:10.972Z] 05:57:10     INFO -  22 warnings
[task 2018-04-24T05:57:11.023Z] 05:57:11     INFO -  :geckoview:javadocJarOfficialWithGeckoBinariesNoMinApiRelease
[task 2018-04-24T05:57:11.164Z] 05:57:11     INFO -  :geckoview:sourcesJarOfficialWithGeckoBinariesNoMinApiRelease
[task 2018-04-24T05:57:11.869Z] 05:57:11     INFO -  :geckoview:uploadArchivesCould not find metadata org.mozilla:geckoview-default-arm64-v8a/maven-metadata.xml in remote (file:///builds/worker/workspace/build/src/obj-firefox/gradle/build/mobile/android/geckoview/maven)
[task 2018-04-24T05:57:11.890Z] 05:57:11     INFO -  FAILURE: Build failed with an exception.
[task 2018-04-24T05:57:11.890Z] 05:57:11     INFO -  * What went wrong:
[task 2018-04-24T05:57:11.890Z] 05:57:11     INFO -  Execution failed for task ':geckoview:compileOfficialWithGeckoBinariesNoMinApiDebugAndroidTestKotlin'.
[task 2018-04-24T05:57:11.890Z] 05:57:11     INFO -  > Compilation error. See log for more details
[task 2018-04-24T05:57:11.890Z] 05:57:11     INFO -  * Try:
[task 2018-04-24T05:57:11.891Z] 05:57:11     INFO -  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
[task 2018-04-24T05:57:11.891Z] 05:57:11     INFO -  * Get more help at https://help.gradle.org
[task 2018-04-24T05:57:11.891Z] 05:57:11     INFO -  BUILD FAILED in 37s
[task 2018-04-24T05:57:11.891Z] 05:57:11     INFO -  123 actionable tasks: 110 executed, 13 up-to-date
[task 2018-04-24T05:57:12.504Z] 05:57:12    ERROR - Return code: 1
[task 2018-04-24T05:57:12.504Z] 05:57:12    ERROR - 1 not in success codes: [0]
[task 2018-04-24T05:57:12.504Z] 05:57:12  WARNING - setting return code to 2
[task 2018-04-24T05:57:12.504Z] 05:57:12    FATAL - Halting on failure while running ['/usr/bin/python2.7', 'mach', '--log-no-times', 'android', 'archive-geckoview']
[task 2018-04-24T05:57:12.505Z] 05:57:12    FATAL - Running post_fatal callback...
[task 2018-04-24T05:57:12.505Z] 05:57:12    FATAL - Exiting 2
[task 2018-04-24T05:57:12.505Z] 05:57:12     INFO - [mozharness: 2018-04-24 05:57:12.505277Z] Finished build step (failed)
[task 2018-04-24T05:57:12.505Z] 05:57:12     INFO - Running post-run listener: _summarize
[task 2018-04-24T05:57:12.505Z] 05:57:12    ERROR - # TBPL FAILURE #
[task 2018-04-24T05:57:12.505Z] 05:57:12     INFO - [mozharness: 2018-04-24 05:57:12.505590Z] FxDesktopBuild summary:
[task 2018-04-24T05:57:12.505Z] 05:57:12    ERROR - # TBPL FAILURE #
Flags: needinfo?(nchen)
Pushed by nchen@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/65fb23b3c5e0
1. Fix test license headers; r=jchen
https://hg.mozilla.org/integration/autoland/rev/f6dd07476c05
2. Miscellaneous GeckoSessionTestRule changes; r=jchen
https://hg.mozilla.org/integration/autoland/rev/bb4f5395f6dc
3. Scale timeouts based on device; r=gbrown
Flags: needinfo?(nchen)
Product: Firefox for Android → GeckoView
Target Milestone: Firefox 61 → mozilla61
You need to log in before you can comment on or make changes to this bug.