1528494 - Perma /browser_as_render.js | Got the search handoff button - null == true - when Gecko 67 merges to beta on 2019-03-11

Status: VERIFIED FIXED

(Firefox :: New Tab Page, defect, P1)

Description

[Narcis Beleuzu [:NarcisB]]

Central as Beta simulation:
https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Crunnable&revision=7cc2d6c073df873d0a8990f70eefbda1cf86ecc8&selectedJob=228835648

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=228835648&repo=try&lineNumber=2510

Log snippet:

15:36:46 INFO - TEST-START | browser/components/newtab/test/browser/browser_as_render.js
15:36:46 INFO - TEST-INFO | started process screenshot
15:36:46 INFO - TEST-INFO | screenshot: exit 0
15:36:46 INFO - Buffered messages logged at 15:36:46
15:36:46 INFO - Entering test bound test_render_search
15:36:46 INFO - TEST-PASS | browser/components/newtab/test/browser/browser_as_render.js | Got the search box - {} == true -
15:36:46 INFO - TEST-PASS | browser/components/newtab/test/browser/browser_as_render.js | Search box is localized - "Search the Web" != "search_web_placeholder" -
15:36:46 INFO - Leaving test bound test_render_search
15:36:46 INFO - Entering test bound test_render_search_handoff
15:36:46 INFO - Buffered messages finished
15:36:46 INFO - TEST-UNEXPECTED-FAIL | browser/components/newtab/test/browser/browser_as_render.js | Got the search handoff button - null == true -
15:36:46 INFO - Stack trace:
15:36:46 INFO - resource://testing-common/content-task.js line 59 > eval:test_render_search_handoff:5
15:36:46 INFO - resource://testing-common/content-task.js:null:60
15:36:46 INFO - Leaving test bound test_render_search_handoff
15:36:46 INFO - Entering test bound test_render_topsites
15:36:47 INFO - TEST-PASS | browser/components/newtab/test/browser/browser_as_render.js | Got the top sites section - {} == true -

:k88hudson , could this be caused by https://bugzilla.mozilla.org/show_bug.cgi?id=1528410#c2 ?

Comment 1

[Ed Lee :Mardak]

Is this a recent regression or there hasn't been a central-as-beta until now?

The test should be setting the pref explicitly:

https://searchfox.org/mozilla-central/rev/e6520f0a4dd5d7474c32a1164744953ea59be0d0/browser/components/newtab/test/browser/browser_as_render.js#15-20

But the pref is default true only for Nightly:

https://searchfox.org/mozilla-central/rev/e6520f0a4dd5d7474c32a1164744953ea59be0d0/browser/app/profile/firefox.js#1301-1305

Comment 2

[Ricky Rosario [:rrosario, :r1cky]]

Yes, this is puzzling because I dont think there are any relevant differences between what is in Beta 66 and Nightly 67.

Comment 3

[Ricky Rosario [:rrosario, :r1cky]]

Looks like this was reported as intermittent before - Bug 1519829. So I guess something else changed that made it permanent. Probably a timing kind of issue? Are we waiting long enough for page to get rendered?

Comment 4

[Ed Lee :Mardak]

Oh right, and the code and pref are already on mozilla-beta, so I guess something must have changed recently for nightly 67 then.

The screenshot from test failure shows the -> arrow button, so it's showing the non-handoff version…

https://taskcluster-artifacts.net/F_U8PivvSmyZ8f22de412A/0/public/test_info/mozilla-test-fail-screenshot_r6qxjy.png

And curiously only fails on beta, so something with the pref being default false then switching to true.

Comment 5

[Ricky Rosario [:rrosario, :r1cky]]

This was caused by the patch for bug 1527702. Maybe should be uplifted together?

PR coming in a few.

Comment 6

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4803

Comment 7

[Ed Lee :Mardak]

This won't need to be uplifted to 66 beta for bug 1527702. This bug only affects 67.

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

Merged to master https://github.com/mozilla/activity-stream/commit/de302156cfeae3585f3d06f3f51819ec9ae9c112

Comment 9

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/f98b2b564191

Comment 10

[Cosmin Sabou [:CosminS]]

Fixed verified in today's beta-sims: https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=success%2Ctestfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=78e8be5e2e6573bb1ce027625ff96205c6efe765&searchStr=browser&group_state=expanded