Closed Bug 532445 Opened 10 years ago Closed 2 years ago

Create new Unittest suite: unit test with Firebug installed

Categories

(Testing :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: ted, Unassigned)

References

Details

(Whiteboard: [firebug-p2])

There are a lot of unknowns about how extensions impact our codebase. We should work on ways to make this less scary. For a start, let's take Firebug, which is one of our most popular extensions (and critical to developers). We should try running our unit test suites (at least the Mochitest variants, reftest and crashtest, and probably the js browser tests) with Firebug installed. Probably once a day or so would be fine for starting out. We could grab the nightly build + test package, the latest Firebug XPI (or Firebug development XPI even, if there is such a thing), and tweak the command lines to run the tests with Firebug installed.

I'll need to sort out the precise details of how to run the test suites in this way, I'll put this back in the pool once I'm done with that.
We should also run leak tests in this configuration, and ideally talos (at least the rss tests).
Should probably file a separate bug for Talos. Leak tests we can probably do, although we don't currently run them in a packaged-build configuration.
thanks for the CC. At a bare minimum, this might reveal some weird incompatibility regressions early on.

Agree talos should be filed separately so I will do so.
We discussed this bug in our Firebug conf call today
https://wiki.mozilla.org/Firebug/WeeklyUpdates/2010-03-23

Running the Firefox test suite with Firebug installed, active on all tabs, and active for all pages, would go a long ways to address crashing bugs on trunk for Firebug (tho these have become quite a bit rarer). 

Lots of Firebug function only runs on demand. We have a test system (user-interface focused of course) which we use for our releases, with about 120 test files. This system opens a XUL window and drives Firefox to load pages, apply  Firebug operations, and check results. 

Ideally we should run this test suite on FF nightly builds. However this is very time consuming: if the test fails we have to debug Firefox and Firebug. One new suggestion: we could work with someone from the platform team for a set time slot each week to triage any problems.

At present we have one user who tests Firebug latest on Firefox nightly builds. Well they did it once, and hopefully again.
Whiteboard: [firebug-p1]
alternatively, someone in QA might be able to help out with running these things as part of a weekly checkup.

ctalbert: anybody on your team with a couple of cycles to help with this?
Here is a wiki page about how to run automated tests for Firebug.
http://getfirebug.com/wiki/index.php/Running_Automated_Test_Suite

Quick summary of how to test Firebug 1.6:
1) Install latest Firebug 1.6: http://getfirebug.com/releases/firebug/1.6X/
2) Install latest FBTest 1.6: http://getfirebug.com/releases/fbtest/1.6/
3) Open FBTest window using "Firebug -> Open Test Console" menu item.
4) Click Run All button in the FBTest window.

Honza
People appear to be talking about two different things here. Comment 0 talks about running the fire*fox* tests with firebug installed. Comment 6 talks about running the fire*bug* tests.

I filed bug 559466 on running the fire*bug* tests automatically.
I think we have made good progress on bug 559466 - Integrate firebug test suite with automated tests.

Any progress on this one? Can I help somehow to have firebug installed in testing profiles?

Honza
Bugcleanup.

This seems to be about developing a unittest-suite running firefox-with-firebug-installed. This is very similar to bug#532637, which is about developing a Talos suite measuring firefox-with-firebug-installed. Tweaked summary and moved this bug to Testing:General for suite development. 

Once this new suite is developed, please file a separate bug in
mozilla.org/ReleaseEngineering to enable running the new suite on builds
automatically.
Component: Release Engineering → General
Product: mozilla.org → Testing
QA Contact: release → general
Summary: run unit tests (daily?) with Firebug installed in testing profiles → Create new Unittest suite: unit test with Firebug installed
Version: other → unspecified
Whiteboard: [firebug-p1] → [firebug-p2]
Assignee: ted.mielczarek → nobody
Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.