Closed Bug 1315953 Opened 3 years ago Closed 8 months ago

browser_google.js and browser_google_behavior.js fail after 'mach build faster' and on artifact builds

Categories

(Firefox :: Search, task, P3)

task
Points:
1

Tracking

()

RESOLVED FIXED
Firefox 68
Iteration:
69.1 - May 13 - 26
Tracking Status
firefox52 --- wontfix
firefox68 --- fixed

People

(Reporter: Gijs, Assigned: standard8)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

STR:

1. ./mach build
2. ./mach mochitest browser/components/search/test/browser_google_behavior.js 

(pass)
3. ./mach build faster
4. ./mach mochitest browser/components/search/test/browser_google_behavior.js 

(fail)

including:


55 INFO TEST-UNEXPECTED-FAIL | browser/components/search/test/browser_google_behavior.js | Check search URL for 'foo' - Got https://www.google.com/search?q=foo&ie=utf-8&oe=utf-8&client=firefox-b, expected https://www.google.com/search?q=foo&ie=utf-8&oe=utf-8


Feels like there's some kind of initialization that happens that makes things dirty and then fails the test. Or something.
Mike, do you know why this doesn't work in this case?
Flags: needinfo?(mozilla)
This happens with artifact builds for me as well, opt builds are fine.
Blocks: 1278699
So the specific reason this doesn't work is because in the artifact build case, we use the packaged list.json as the default and it has google in the list, not google-nocodes (which it should).

What I really want is for the search engines to be rebuild in the artifact case.

They are front end and should be treated as such. I'm investigating that.
Flags: needinfo?(mozilla)
This affects artifact builds, locally as well as on try.
Summary: browser_google.js and browser_google_behavior.js fail (sometimes?) outside of the US → browser_google.js and browser_google_behavior.js fail after 'mach build faster' and on artifact builds
Apparently there are other tests that fail in artifact builds as well.

Mike Hommey is working on something that might fix this.

What I'd really like is for the search engine stuff to happen as part of front end. I'm not entirely sure how to do that.
Mike (Hommey) is the problem here that we're using the libs: target to install the searchplugins:

https://dxr.mozilla.org/mozilla-central/rev/8103c612b79c2587ea4ca1b0a9f9f82db4b185b8/browser/locales/Makefile.in#76

Is there an easy way around this?
Flags: needinfo?(mh+mozilla)
These are the only remaining tests I know of that are failing against artifact builds, we should disable them there until we're able to figure this out.
Comment on attachment 8835065 [details]
Bug 1315953 - Skip browser_google_nocodes.js on artifact builds.

https://reviewboard.mozilla.org/r/110760/#review112148
Attachment #8835065 - Flags: review?(mozilla) → review+
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6c97dfdfe6f9
Skip search related browser chrome tests on artifact builds. r=mkaply
Attachment #8835065 - Flags: review+ → review?(mozilla)
Comment on attachment 8835065 [details]
Bug 1315953 - Skip browser_google_nocodes.js on artifact builds.

I thought this was already r+ed in mozreview...
Attachment #8835065 - Flags: review?(mozilla) → review+
(In reply to Mike Kaply [:mkaply] from comment #13)
> Comment on attachment 8835065 [details]
> Bug 1315953 - Skip browser_google_nocodes.js on artifact builds.
> 
> I thought this was already r+ed in mozreview...

It was... this is a different patch for another failing test I found this week.
Pushed by cmanchester@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f68ac7a260c1
Skip browser_google_nocodes.js on artifact builds. r=mkaply
Mass wontfix for bugs affecting firefox 52.
One thing I noticed when logged into a one-click loaner to debug this was that if I run

./mach mochitest -f browser --run-until-failure browser/components/search/test/browser_google_codes.js

it never fails.


However, if I do

./mach mochitest -f browser --run-until-failure browser/components/search/test

it quickly hits a failure.


After doing a bunch of test disabling by editing browser.ini, the minimal necessary thing to cause failure is to have browser.ini set up to only run browser_google_codes.js and browser_about_searchReset.js.  Removing browser_about_searchReset.js from browser.ini makes the problem go away.

That said, I have yet to reproduce this on a machine that was not one of the taskcluster loaners.
Component: Search → Build Config
Flags: needinfo?(mh+mozilla)
Depends on: 1408730
Assignee: nobody → mozilla
Status: NEW → ASSIGNED
The problems the google tests are all fixed, so this particular bug can probably be closed.

I did find that:

./mach mochitest -f browser --run-until-failure browser/components/search/test/browser_searchEngine_behaviors.js

Always fails on artificate and non artifact builds, so that seems unrelated. I'm guessing that test doesn't properly clean up?
I opened

https://bugzilla.mozilla.org/show_bug.cgi?id=1473710

for the browser_searchEngine_behaviors.js failure. 

This bug is fixed (I removed the artifact only stuff when I checked in bug 1437942).
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1437942
Component: Build Config → General
Product: Firefox → Firefox Build System

I just found a couple of instances of skips still referencing this bug, and we no longer need them, so lets use this to clean those up.

Assignee: mozilla → standard8
Status: RESOLVED → REOPENED
Component: General → Search
Depends on: 1437942
Product: Firefox Build System → Firefox
Resolution: DUPLICATE → ---
Type: defect → task
Points: --- → 1
Priority: -- → P3
Pushed by mbanner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a68c01cb895d
Allow more browser search tests to run in artifact mode now that the underlying issues are fixed. r=daleharvey
Status: REOPENED → RESOLVED
Closed: 2 years ago8 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 68
Iteration: --- → 69.1 - May 13 - 26
You need to log in before you can comment on or make changes to this bug.