393410 - reftest should be able to create a (temporary) profile with appropriate prefs

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Rob Campbell [:rc] (:robcee)]

the reftest portion of the unittest suite relies on certain options being present in the profile. Currently, we've been setting these up manually, but we need a buildstep in buildbot to create the profile on first run. Bonus points: Destroy the profile after use.

Required profile options:

user_pref("browser.dom.window.dump.enabled", true);
user_pref("browser.sessionstore.enabled", false);
user_pref("javascript.options.showInConsole", true);
user_pref("layout.fire_onload_after_image_background_loads", true);

Currently, these are being run from the "default" profile. If this is changed to a new profile name, MozillaReftest step will have to be updated to reflect that.

Comment 1

[Ray Kiddy]

I have a perl script that does this for me now. Let me know if you are interested in seeing it. If you already have yours done, no worries.

Comment 2

[Rob Campbell [:rc] (:robcee)]

Sure, can't hurt. Paste it in as an attachment if you like and we can enpatchify it.

Comment 3

[Ray Kiddy]

Attachment: patch for newbrowser script

Here is the script I wrote. Issues with it are:

- yo no hablo Windows! How can I make this more cross-platform?
- extra cruftiness ends up in one's profile.ini file
- any others?

Comment 4

[Chris Cooper [:coop] (he/him)]

robcee: you mentioned yesterday that one of the frameworks already did this (mochitest?) -> where can I find that script?

Comment 5

[Nickolay_Ponomarev]

http://mxr.mozilla.org/seamonkey/source/testing/mochitest/runtests.pl.in#390 ?

Comment 6

[Rob Campbell [:rc] (:robcee)]

yep, that's the one. I'm not sure if we want to rip that out and include the profile creation as a separate perl script or write some buildsteps to produce the same thing in python buildbot code. Take your pick. :)

Comment 7

[Ray Kiddy]

Well, this may be just an issue I have. If I look at two methods for constructing profile directories, ie:

- using -createprofile, part of Firefox, in the way that FF probably uses it
- using extra code to artificially reproduce what we think Firefox is doing

I see the former as being significantly more useful to testing than the latter. Institutionally, as a group, the people more "in the community" than I seem to prefer the latter. I am not sure what to make of this.

My approach often brings out architectural issues with the apps I work with, highlighting their fragilities. But then, I think that is the point. But the latter approach gets you something today. So, I guess, take your pick. :-)

Comment 8

[Nickolay_Ponomarev]

Ray, tests need a special profile and most (if not all) of the mochitest's runtests.pl code creating the profile would still be needed even if it started with running firefox -createprofile.

Comment 9

[Jeff Walden [:Waldo]]

I'd love it if runtests.pl were really a Python script (and perhaps if the basic running stuff were abstracted into some sort of class, probably as a followup -- just hacking a copy of it would be enough to start).  Every time I have to touch that I curse myself for not rewriting it in Python the first time I touched it, back when it was a good bit smaller and somewhat more hackish.

Comment 10

[Chris Cooper [:coop] (he/him)]

I've actually started writing some of this as part of the bug 397724 (leak+reftest). robcee has a basic list above, but tinderbox also has a pretty complete list after many years:

http://mxr.mozilla.org/seamonkey/source/tools/tinderbox/build-seamonkey-util.pl#2129

Please let me know if there are any others I should be including.

Comment 11

[Chris Cooper [:coop] (he/him)]

Attachment: python script for finding/creating a new firefox profile and populating relevant prefs for testing

People seemed to be wanting a python version, so here it is. This will make it easier to fold it into buildbot sometime in the future.

Not sure where this should live. We can add whatever MOZ_CO_MODULE we want to the mozconfigs, so we have some choices.

Comment 12

[Ray Kiddy]

When I create a profile with this, on an intel mac with 10.4.10, I get a problem with the launched executable. It is the same problem as:

http://bugzilla.mozilla.org/show_bug.cgi?id=381283
errors from launching Firefox on Mac with wrong capitalization

So, something is wrong with the chrome initialization here (AFAIK).

Why do you require the executable as a parameter here? I can see that the contents of the profile may be dependent on the _version_ of the executable, but I can think of no reason that the profile must depend on the location of the executable.

Have you tested on a Mac? How do you invoke it? Does NO_EM_RESTART=1 poison the soup?

Comment 13

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 290585 [details]
python script for finding/creating a new firefox profile and populating relevant prefs for testing

I am nitless. This looks like it should do the job.

Comment 14

[Chris Cooper [:coop] (he/him)]

(In reply to comment #12)
> Why do you require the executable as a parameter here? I can see that the
> contents of the profile may be dependent on the _version_ of the executable,
> but I can think of no reason that the profile must depend on the location of
> the executable.

Many of these build machines can have many builds slaves (or tinderbox trees) on the same box, and hence multiple executables.

The script works on Linux, and I have it working on Windows after a small patch. Testing Mac now.

Comment 15

[Ray Kiddy]

Does this script need to create the profile in the default location?

I think it would make sense to have _either_ a --profileName flag, and have it behave as it does now, or a --profileDir flag. If this flag is used, the profile would be created at the path given and the profile name would _not_ be added to the profile.ini file.

Would you accept a patch for this?

Comment 16

[Chris Cooper [:coop] (he/him)]

(In reply to comment #15)
> Does this script need to create the profile in the default location?

The rest of the reftest/buildbot harness doesn't do anything fancy with the profiles, i.e. we don't supply the optional --profileName arg and just go with the default.
 
> I think it would make sense to have _either_ a --profileName flag, and have it
> behave as it does now, or a --profileDir flag. If this flag is used, the
> profile would be created at the path given and the profile name would _not_ be
> added to the profile.ini file.
> 
> Would you accept a patch for this?
 
Sure, I'll take a patch. We won't likely use the functionality for reftesting, but having the option won't hurt.

Comment 17

[Chris Cooper [:coop] (he/him)]

Checking in profiles/createTestingProfile.py;
/cvsroot/mozilla/testing/tools/profiles/createTestingProfile.py,v  <--  createTestingProfile.py
initial revision: 1.1
done

Comment 18

[Chris Cooper [:coop] (he/him)]

Changes are currently baking in the staging environment. This required changes to:

* master.cfg -> replaced sqllite removal steps with createProfile* calls;
* mozbuild.py -> addition of createProfile* classes; and,
* mozconfig files -> needed to add MOZ_CO_MODULE="mozilla/testing/tools" to grab profile creation script from CVS 

Comment 19

[Chris Cooper [:coop] (he/him)]

Attachment: (re)create profiles on slaves for each run rather than culling places.sqlite [checked in]

This is a patch against the current production master.cfg et al., and as such also contains lines to disable winxp and leopard.

Comment 20

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 307724 [details] [diff] [review]
(re)create profiles on slaves for each run rather than culling places.sqlite [checked in]

one nit: change class-names "createProfile(Win)" to "CreateProfile(Win)" to be consistent with class-naming conventions. Otherwise looks great!

Comment 21

[Chris Cooper [:coop] (he/him)]

Comment on attachment 307724 [details] [diff] [review]
(re)create profiles on slaves for each run rather than culling places.sqlite [checked in]

Nits fixed.

Comment 22

[Chris Cooper [:coop] (he/him)]

Pushed to production. Monitoring.

Comment 23

[Chris Cooper [:coop] (he/him)]

Unrelated Win2K3 bustage, but otherwise looks good.