Created attachment 654610 [details]
reftest-sanity log with reftest.browser.iframe.enabled = true
With the landing of bug 778072 we switched from <browser> to <iframe mozbrowser> since that is what will be shipping. This causes some additional test failures.
I attached a reftest-sanity log with 9 failures (where previously there were only 3).
The six new failures are:
== test-async.xul test-async-ref.xul
== test-async.html test-async-ref.html
== zoom-invalidation.html zoom-invalidation-ref.html
== corners-2.html corners-2-ref.html
== corners-3.html corners-3-ref.html
== corners-4.html corners-4-ref.html
There are many other failures in other manifests too, but fixing these will probably fix the rest.
Blocking so we can investigate and determine whether or not there is a deeper issue.
Jet, can the layout team investigate?
Instructions for running reftests on b2g at: https://developer.mozilla.org/en-US/docs/Mozilla/Boot_to_Gecko/B2G_Reftests
I'd recommend downloading a nightly emulator build.
To Emmanuelle for investigation...
(In reply to Andrew Halberstadt [:ahal] from comment #3)
> Instructions for running reftests on b2g at:
> I'd recommend downloading a nightly emulator build.
I tried by building b2g from mercurial (+ gaia from git), which worked pretty much okay, but the wiki is a bit scant on details on how to go from that to running the reftests python script; I'll try running from a nightly build, but it'd probably be worth adding details on the wiki page, like where should the B2G_HOME directory point to, and what "--emulator arm" should do.
yeah, I *could* test a nightly build, if there was one for x86_64 on Linux. I'll either have to test under Mac or I will need some guidance on how to go from a build of b2g+gaia to running the runreftestb2g.py script.
The script is run from objdir-gecko/tests/reftest
--b2gpath can point to the B2G repo or an extracted emulator nightly build.
--emulator arm just identifies the type of emulator being used, it shouldn't change.
I'll add these details to the wiki.
thanks for the explanation; sadly, I still get this error:
Exception: File not found: /home/ebassi/mozilla/b2g/out/host/linux-x86/bin/emulator
do I need some specific flag to build mozilla-central apart from the ones listed on the wiki?
I have noticed the reftest extension wasn't getting installed anymore for a period last week. I haven't had a chance to look into it and I'm not sure if it's still a problem (or reproducible to others). So it might be better to punt on this for a little while.
If you do want to look into it, did you run ./config.sh emulator before ./build.sh? Depending on the device you are building for the ./config.sh script will pull down all of the required libraries.
(In reply to Andrew Halberstadt [:ahal] from comment #9)
> If you do want to look into it, did you run ./config.sh emulator before
> ./build.sh? Depending on the device you are building for the ./config.sh
> script will pull down all of the required libraries.
okay, I'm completely missing this step on the wiki. also, and I'm sorry for not saying this earlier, I'm trying to build the desktop client given that I don't have a b2g device to test with.
(In reply to Emmanuele Bassi (:ebassi) from comment #10)
> okay, I'm completely missing this step on the wiki. also, and I'm sorry for
> not saying this earlier, I'm trying to build the desktop client given that I
> don't have a b2g device to test with.
Yeah, I just linked to the normal build instructions in the prerequisites. You'll want to build the emulator (not the b2g desktop) since this is where I noticed these test failures occurring.
But like I said, the reftests might not be working at all with the latest build. My aim is to investigate/fix by the end of the week.
:ebassi, fyi the extension being installed issue is fixed (on m-c, I don't think it made it to aurora yet).
So if you end up tackling this let me know if you need any help. If you are having trouble building I can also give you an m-c based emulator.
(In reply to Andrew Halberstadt [:ahal] from comment #12)
> :ebassi, fyi the extension being installed issue is fixed (on m-c, I don't
> think it made it to aurora yet).
thanks for looking into it; I'll have a look ASAP.
> So if you end up tackling this let me know if you need any help. If you are
> having trouble building I can also give you an m-c based emulator.
thanks! I have a build set up, but if I have any issue I'll poke you.
Re-nominating. It was only blocking for investigation, which isn't enough to hold the entire release for.
I spoke about this with Andrew via IRC. We talked about it and decided that since not many reftests are enabled ATM, it's probably not worth blocking on this. As always, people should feel free to re-nom with a reason as to why something should block and we'll discuss it during triage.
See the differences in reftest failures with that last commit (ignore debug).
Created attachment 719640 [details] [diff] [review]
Enables <iframe mozbrowser remote="true"> in the b2g reftest harness
This enables both <iframe mozbrowser> and remote="true". Note that tests still fail even with remote="false", so in the interest of separating concerns you may want to leave it at false to begin with.