832702 - Reenable building testpilot on mozilla-aurora when it no longer causes test failures

Status: RESOLVED WONTFIX

(Mozilla Labs Graveyard :: Test Pilot, defect)

Description

[Phil Ringnalda (:philor)]

Bug 823989 disabled building testpilot on aurora nightlies because it was causing multiple failures in browser-chrome tests. Once that's fixed, we need to reenable it.

Comment 1

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

What is the current status of Test Pilot?  

1)  Building, but not testing
2)  Completely disabled / Not shipping.

If 2, this is blocking lots of projects going forward, for security, labs, ux and others, and I want to do whatever I can to fix it, but this seems impossibly thorny, and I am not sophisticated to resolve it.

Comment 2

[Ed Morley [:emorley]]

(In reply to Gregg Lind (User Research - Test Pilot) from comment #1)
> What is the current status of Test Pilot?  
> 
> 1)  Building, but not testing
> 2)  Completely disabled / Not shipping.

Not building in {mozilla-central,aurora,release}.

Building (no idea what tests there are), in mozilla-beta.

It was disabled in:
https://hg.mozilla.org/mozilla-central/rev/4919e8091542

And hasn't yet been re-enabled:
https://hg.mozilla.org/mozilla-central/file/tip/browser/installer/Makefile.in#l80
https://hg.mozilla.org/mozilla-central/file/tip/browser/app/profile/extensions/Makefile.in#l16

Comment 3

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

emorley, I am not sure what I can do to get it re-enabled, honestly :)  I am way out of my league here, but willing to do quite a lot of work.

Comment 4

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

So what I can do to fix this?

So far I can:

* build Fx on osx
* know the Test Pilot Code Base

I need help with:

* interpreting the tbpl failure log
* reproducing the failures
* fixing whatever needs fixing

Ask:

* an assigned helper to allow me to fix this / guide me through the process


Having this out is blocking lots of downstream consumers

Comment 5

[(no longer active)]

For starters, you should enable this extension in your local tree and rebuild and then run one of the failing test suites on your machine to see if you can reproduce the failures locally.  If you don't know what the list of all failures includes, you need to push a patch to the try server which enables this extension and run the tests on all platforms.  I don't remember the full list of tests that were failing off the top of my head.

Comment 6

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

Can we just turn off Test Pilot in linux for now (where it is causing the weirdest testing errors) while I work on landing TP2 (which is a jetpack, has no toolbar button, and shouldn't cause any of these issues)?

Comment 7

[Phil Ringnalda (:philor)]

It's not at all a "just Linux" problem. Not at all.

Comment 8

[Jinghua zhang]

Hi there, is there anything we could do to make it moving forward? 

A group of research requests asked by different teams have been piled up in last months, it is crucial for us to enable Test Pilot to unlock the progress for these projects, including Social API, Search and others. 

Please let us know what we shall do, and who can help us out.

Comment 9

[(no longer active)]

(In reply to comment #8)
> Hi there, is there anything we could do to make it moving forward? 
> 
> A group of research requests asked by different teams have been piled up in
> last months, it is crucial for us to enable Test Pilot to unlock the progress
> for these projects, including Social API, Search and others. 
> 
> Please let us know what we shall do, and who can help us out.

Seeing if you can reproduce this on a build on a revision prior to my hack to work around it would be a huge help.

Comment 10

[Phil Ringnalda (:philor)]

Your hack to work around it worked around a teeny tiny little itty bitty part of it, the fact that it screwed up the focus on Linux.

It's disabled because it leaks and OOMs on every platform, not just Linux, and because nobody has tried to investigate it other than Gregg's abortive attempt to get a build setup set up, we have absolutely no reason to believe that it doesn't also leak and OOM for users. In fact, I'm a little puzzled by why we haven't disabled building it on beta, and blocklisted it.

Comment 11

[:Gavin Sharp [email: [e-mail]]]

The right way forward is to fix bug 840108. I've set up a meeting with Gregg to see how Dolske or I can help.

Comment 12

[:Gavin Sharp [email: [e-mail]]]

Bug 867445 will remove the current version of test pilot, bug 840108 will re-introduce a version that doesn't have these problems.