Closed Bug 1538430 Opened 7 months ago Closed 5 months ago

Intermittent org.mozilla.geckoview.test.RuntimeSettingsTest.automaticFontSize | status -2

Categories

(GeckoView :: General, defect, P3)

Unspecified
All
defect

Tracking

(firefox-esr60 unaffected, firefox67 wontfix, firefox68 fixed, firefox69 fixed)

RESOLVED FIXED
mozilla69
Tracking Status
firefox-esr60 --- unaffected
firefox67 --- wontfix
firefox68 --- fixed
firefox69 --- fixed

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [stockwell disabled])

Attachments

(2 files)

#[markdown(off)]
Filed by: csabou [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=235615697&repo=autoland

https://queue.taskcluster.net/v1/task/LgHutoabSeiUsRBWU3-q7A/runs/0/artifacts/public/logs/live_backing.log

[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - TEST-PASS | org.mozilla.geckoview.test.PromptDelegateTest.popupTest | took 9322ms
[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: current=94
[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: class=org.mozilla.geckoview.test.RuntimeSettingsTest
[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: stream=
[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - org.mozilla.geckoview.test | org.mozilla.geckoview.test.RuntimeSettingsTest:
[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: numtests=151
[task 2019-03-23T02:13:56.026Z] 02:13:56 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: test=automaticFontSize
[task 2019-03-23T02:13:56.027Z] 02:13:56 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS_CODE: 1
[task 2019-03-23T02:13:56.027Z] 02:13:56 INFO - TEST-START | org.mozilla.geckoview.test.RuntimeSettingsTest.automaticFontSize
[task 2019-03-23T02:15:58.497Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: id=AndroidJUnitRunner
[task 2019-03-23T02:15:58.498Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: current=94
[task 2019-03-23T02:15:58.498Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: class=org.mozilla.geckoview.test.RuntimeSettingsTest
[task 2019-03-23T02:15:58.498Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: stream=
[task 2019-03-23T02:15:58.498Z] 02:15:58 INFO - org.mozilla.geckoview.test | Error in automaticFontSize(org.mozilla.geckoview.test.RuntimeSettingsTest):
[task 2019-03-23T02:15:58.498Z] 02:15:58 INFO - org.mozilla.geckoview.test | org.mozilla.geckoview.test.util.UiThreadUtils$TimeoutException: Timed out after 120000ms
[task 2019-03-23T02:15:58.498Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.util.UiThreadUtils$TimeoutRunnable.run(UiThreadUtils.java:56)
[task 2019-03-23T02:15:58.499Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.handleCallback(Handler.java:730)
[task 2019-03-23T02:15:58.499Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.dispatchMessage(Handler.java:92)
[task 2019-03-23T02:15:58.499Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.util.UiThreadUtils.loopUntilIdle(UiThreadUtils.java:154)
[task 2019-03-23T02:15:58.499Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.waitForInitialLoad(GeckoSessionTestRule.java:1314)
[task 2019-03-23T02:15:58.500Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.waitForOpenSession(GeckoSessionTestRule.java:1263)
[task 2019-03-23T02:15:58.500Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.openSession(GeckoSessionTestRule.java:1259)
[task 2019-03-23T02:15:58.501Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.prepareStatement(GeckoSessionTestRule.java:1241)
[task 2019-03-23T02:15:58.501Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule$3$1.run(GeckoSessionTestRule.java:1421)
[task 2019-03-23T02:15:58.502Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.app.Instrumentation$SyncRunnable.run(Instrumentation.java:1719)
[task 2019-03-23T02:15:58.502Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.handleCallback(Handler.java:730)
[task 2019-03-23T02:15:58.503Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.dispatchMessage(Handler.java:92)
[task 2019-03-23T02:15:58.503Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Looper.loop(Looper.java:137)
[task 2019-03-23T02:15:58.503Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.app.ActivityThread.main(ActivityThread.java:5103)
[task 2019-03-23T02:15:58.504Z] 02:15:58 INFO - org.mozilla.geckoview.test | at java.lang.reflect.Method.invokeNative(Native Method)
[task 2019-03-23T02:15:58.504Z] 02:15:58 INFO - org.mozilla.geckoview.test | at java.lang.reflect.Method.invoke(Method.java:525)
[task 2019-03-23T02:15:58.504Z] 02:15:58 INFO - org.mozilla.geckoview.test | at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
[task 2019-03-23T02:15:58.505Z] 02:15:58 INFO - org.mozilla.geckoview.test | at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
[task 2019-03-23T02:15:58.505Z] 02:15:58 INFO - org.mozilla.geckoview.test | at dalvik.system.NativeStart.main(Native Method)
[task 2019-03-23T02:15:58.506Z] 02:15:58 INFO - org.mozilla.geckoview.test |
[task 2019-03-23T02:15:58.506Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: numtests=151
[task 2019-03-23T02:15:58.506Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: stack=org.mozilla.geckoview.test.util.UiThreadUtils$TimeoutException: Timed out after 120000ms
[task 2019-03-23T02:15:58.507Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.util.UiThreadUtils$TimeoutRunnable.run(UiThreadUtils.java:56)
[task 2019-03-23T02:15:58.507Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.handleCallback(Handler.java:730)
[task 2019-03-23T02:15:58.508Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.dispatchMessage(Handler.java:92)
[task 2019-03-23T02:15:58.508Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.util.UiThreadUtils.loopUntilIdle(UiThreadUtils.java:154)
[task 2019-03-23T02:15:58.508Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.waitForInitialLoad(GeckoSessionTestRule.java:1314)
[task 2019-03-23T02:15:58.509Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.waitForOpenSession(GeckoSessionTestRule.java:1263)
[task 2019-03-23T02:15:58.509Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.openSession(GeckoSessionTestRule.java:1259)
[task 2019-03-23T02:15:58.510Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.prepareStatement(GeckoSessionTestRule.java:1241)
[task 2019-03-23T02:15:58.510Z] 02:15:58 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule$3$1.run(GeckoSessionTestRule.java:1421)
[task 2019-03-23T02:15:58.511Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.app.Instrumentation$SyncRunnable.run(Instrumentation.java:1719)
[task 2019-03-23T02:15:58.511Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.handleCallback(Handler.java:730)
[task 2019-03-23T02:15:58.511Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Handler.dispatchMessage(Handler.java:92)
[task 2019-03-23T02:15:58.512Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.os.Looper.loop(Looper.java:137)
[task 2019-03-23T02:15:58.512Z] 02:15:58 INFO - org.mozilla.geckoview.test | at android.app.ActivityThread.main(ActivityThread.java:5103)
[task 2019-03-23T02:15:58.512Z] 02:15:58 INFO - org.mozilla.geckoview.test | at java.lang.reflect.Method.invokeNative(Native Method)
[task 2019-03-23T02:15:58.513Z] 02:15:58 INFO - org.mozilla.geckoview.test | at java.lang.reflect.Method.invoke(Method.java:525)
[task 2019-03-23T02:15:58.514Z] 02:15:58 INFO - org.mozilla.geckoview.test | at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
[task 2019-03-23T02:15:58.514Z] 02:15:58 INFO - org.mozilla.geckoview.test | at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
[task 2019-03-23T02:15:58.514Z] 02:15:58 INFO - org.mozilla.geckoview.test | at dalvik.system.NativeStart.main(Native Method)
[task 2019-03-23T02:15:58.514Z] 02:15:58 INFO - org.mozilla.geckoview.test |
[task 2019-03-23T02:15:58.515Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS: test=automaticFontSize
[task 2019-03-23T02:15:58.515Z] 02:15:58 INFO - org.mozilla.geckoview.test | INSTRUMENTATION_STATUS_CODE: -2
[task 2019-03-23T02:15:58.515Z] 02:15:58 WARNING - TEST-UNEXPECTED-FAIL | org.mozilla.geckoview.test.RuntimeSettingsTest.automaticFontSize | status -2

Priority: P5 → P3

[bulk change --> 69 fix-optional b/c intermittent]

Removed disable recommended since we have activity.

Whiteboard: [stockwell disable-recommended]

Green try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ee0b2d44e6fddd8384abdcc5280fd1c48a343e5c&selectedJob=249661638
Took the example of bug 1519591 https://hg.mozilla.org/integration/mozilla-inbound/rev/22ed69977c45
Looking in the log found this line: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=249661638&repo=try&lineNumber=3052
TEST-FAIL | org.mozilla.geckoview.test.RuntimeSettingsTest.automaticFontSize | took 2811ms
The job is green but the test has failed; it's like that syntax doesn't skip the test but renders it as expected fail and it doesn't turn orange anymore. Chris, could this be expected behavior for geckoview tests?

Flags: needinfo?(cpeterson)

(In reply to Cosmin Sabou [:CosminS] from comment #19)

The job is green but the test has failed; it's like that syntax doesn't skip the test but renders it as expected fail and it doesn't turn orange anymore. Chris, could this be expected behavior for geckoview tests?

Sorry, I don't know anything about the junit test syntax.

Agi, do you know what the expected behavior is for skipping junit tests with assumeThat?

Flags: needinfo?(cpeterson) → needinfo?(agi)
Whiteboard: [stockwell disable-recommended] → [stockwell needswork]
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
Pushed by ncsoregi@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/c7fd51c8101b
Disable org.mozilla.geckoview.test.RuntimeSettingsTest.automaticFontSize on Android pgo for frequent failures. r=jmaher
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69

Did that check disable on debug builds instead of non-debug builds?

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: mozilla69 → ---

The test is disabled correctly on non-debug builds, and the test is intermittently skipped on Android 4.3 pgo.

I think the failure is happening before the test is initialized -- before the assumeThat is hit:

[task 2019-06-05T12:24:15.288Z] 12:24:15 INFO - org.mozilla.geckoview.test | Error in automaticFontSize(org.mozilla.geckoview.test.RuntimeSettingsTest):
[task 2019-06-05T12:24:15.289Z] 12:24:15 INFO - org.mozilla.geckoview.test | org.mozilla.geckoview.test.util.UiThreadUtils$TimeoutException: Timed out after 120000ms
[task 2019-06-05T12:24:15.289Z] 12:24:15 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.util.UiThreadUtils$TimeoutRunnable.run(UiThreadUtils.java:56)
[task 2019-06-05T12:24:15.289Z] 12:24:15 INFO - org.mozilla.geckoview.test | at android.os.Handler.handleCallback(Handler.java:730)
[task 2019-06-05T12:24:15.290Z] 12:24:15 INFO - org.mozilla.geckoview.test | at android.os.Handler.dispatchMessage(Handler.java:92)
[task 2019-06-05T12:24:15.290Z] 12:24:15 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.util.UiThreadUtils.loopUntilIdle(UiThreadUtils.java:154)
[task 2019-06-05T12:24:15.290Z] 12:24:15 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.waitForInitialLoad(GeckoSessionTestRule.java:1314)
[task 2019-06-05T12:24:15.290Z] 12:24:15 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.waitForOpenSession(GeckoSessionTestRule.java:1263)
[task 2019-06-05T12:24:15.291Z] 12:24:15 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.openSession(GeckoSessionTestRule.java:1259)
[task 2019-06-05T12:24:15.291Z] 12:24:15 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule.prepareStatement(GeckoSessionTestRule.java:1241)
[task 2019-06-05T12:24:15.291Z] 12:24:15 INFO - org.mozilla.geckoview.test | at org.mozilla.geckoview.test.rule.GeckoSessionTestRule$3$1.run(GeckoSessionTestRule.java:1421)
[task 2019-06-05T12:24:15.291Z] 12:24:15 INFO - org.mozilla.geckoview.test | at android.app.Instrumentation$SyncRunnable.run(Instrumentation.java:1719)
[task 2019-06-05T12:24:15.292Z] 12:24:15 INFO - org.mozilla.geckoview.test | at android.os.Handler.handleCallback(Handler.java:730)
[task 2019-06-05T12:24:15.292Z] 12:24:15 INFO - org.mozilla.geckoview.test | at android.os.Handler.dispatchMessage(Handler.java:92)
[task 2019-06-05T12:24:15.293Z] 12:24:15 INFO - org.mozilla.geckoview.test | at android.os.Looper.loop(Looper.java:137)
[task 2019-06-05T12:24:15.293Z] 12:24:15 INFO - org.mozilla.geckoview.test | at android.app.ActivityThread.main(ActivityThread.java:5103)
[task 2019-06-05T12:24:15.293Z] 12:24:15 INFO - org.mozilla.geckoview.test | at java.lang.reflect.Method.invokeNative(Native Method)
[task 2019-06-05T12:24:15.294Z] 12:24:15 INFO - org.mozilla.geckoview.test | at java.lang.reflect.Method.invoke(Method.java:525)
[task 2019-06-05T12:24:15.294Z] 12:24:15 INFO - org.mozilla.geckoview.test | at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
[task 2019-06-05T12:24:15.295Z] 12:24:15 INFO - org.mozilla.geckoview.test | at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
[task 2019-06-05T12:24:15.296Z] 12:24:15 INFO - org.mozilla.geckoview.test | at dalvik.system.NativeStart.main(Native Method)

In this case @Ignore works where assumeThat did not.

Flags: needinfo?(gbrown)
Flags: needinfo?(agi)
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6e227ad50bd9
Skip junit test automaticFontSize more forcefully; r=aryx
Status: REOPENED → RESOLVED
Closed: 5 months ago5 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
You need to log in before you can comment on or make changes to this bug.