Closed Bug 520722 Opened 10 years ago Closed 9 years ago

run unittests on linux64

Categories

(Release Engineering :: General, defect, P3)

x86
Linux
defect

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: joduinn, Assigned: armenzg)

References

Details

(Whiteboard: [linux64][unittest])

Attachments

(6 files, 6 obsolete files)

17.97 KB, patch
bhearsum
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
3.77 KB, patch
bhearsum
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
158 bytes, patch
bhearsum
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
6.51 KB, patch
bhearsum
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
16.77 KB, patch
armenzg
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
996 bytes, patch
bhearsum
: review+
armenzg
: checked-in+
Details | Diff | Splinter Review
Tracking bug, as we need unittests here to claim full support of this OS. 

I'm inclined to think we would do this after creating linux64 refimages, but could be convinced otherwise. See bug#519074c0 for details.
moving to future until dependent bug is resolved
Component: Release Engineering → Release Engineering: Future
No longer blocks: support-L64
Depends on: support-L64
Duplicate of this bug: 464027
Blocks: 540404
Whiteboard: [linux64]
Assignee: nobody → armenzg
Status: NEW → ASSIGNED
Attachment #425992 - Flags: review?(bhearsum)
Attachment #425993 - Flags: review?(bhearsum)
Priority: -- → P2
Comment on attachment 425993 [details] [diff] [review]
make linux64 on staging to run unittests (2 of 2)

>diff --git a/env.py b/env.py
>--- a/env.py
>+++ b/env.py
>@@ -71,16 +71,23 @@ MozillaEnvironments['win32-ref-platform'
> 
> MozillaEnvironments['linux-unittest'] = {
>     "MOZ_NO_REMOTE": '1',
>     "CVS_RSH": 'ssh',
>     "DISPLAY": ':2',
>     "NO_FAIL_ON_TEST_ERRORS": '1'   
> }
> 
>+MozillaEnvironments['linux64-unittest'] = {
>+    "MOZ_NO_REMOTE": '1',
>+    "CVS_RSH": 'ssh',
>+    "DISPLAY": ':2',
>+    "NO_FAIL_ON_TEST_ERRORS": '1'   
}

Just make this a copy of the 32-bit linux env:
..... = MozillaEnironments['linux-unittest'].copy()

>         'linux': WithProperties('%(toolsdir:-)s/breakpad/linux/minidump_stackwalk'),
>+        'linux64': WithProperties('%(toolsdir:-)s/breakpad/linux/minidump_stackwalk'),

You should point to 'linux64' here, and add a symlink in the tools repository. Just in case we need different versions for the two in the future.
Attachment #425993 - Flags: review?(bhearsum) → review-
Comment on attachment 425992 [details] [diff] [review]
make linux64 on staging to run unittests (1 of 2)

>diff --git a/mozilla2-staging/config.py b/mozilla2-staging/config.py
>+        'linux64-debug': {
>+            'base_name': 'Linux x86-64 %(branch)s leak test',
>+            'mozconfig': 'linux64/%(branch)s/debug',
>+            'profiled_build': False,
>+            'builds_before_reboot': 5,
>+            'build_space': 7,
>+            'slaves': SLAVES['linux64'],
>+            'platform_objdir': OBJDIR,
>+            'env': {
>+                'MOZ_OBJDIR': OBJDIR,
>+                'DISPLAY': ':2',
>+                'LD_LIBRARY_PATH': '%s/dist/bin' % OBJDIR,
>+                'XPCOM_DEBUG_BREAK': 'stack-and-abort',
>+                'MOZ_CRASHREPORTER_NO_REPORT': '1',
>+            },
>+        },


>-BRANCHES['electrolysis']['platforms']['linux']['env']['MOZ_SYMBOLS_EXTRA_BUILDID'] = 'electrolysis'
>-BRANCHES['electrolysis']['platforms']['linux']['env']['LD_LIBRARY_PATH'] = '/tools/gcc-4.3.3/installed/lib'
>-BRANCHES['electrolysis']['platforms']['linux']['unittest-env'] = {
>-    'LD_LIBRARY_PATH': '/tools/gcc-4.3.3/installed/lib',
>+BRANCHES['electrolysis']['platforms']['linux'] = {
>+    'env': {
>+        'MOZ_SYMBOLS_EXTRA_BUILDID': 'electrolysis',
>+        'LD_LIBRARY_PATH':           '/tools/gcc-4.3.3/installed/lib'
>+    },
>+    'unittest-env': {
>+        'LD_LIBRARY_PATH': '/tools/gcc-4.3.3/installed/lib',
>+    }
>+}
>+BRANCHES['electrolysis']['platforms']['linux64'] = {
>+    'env': {
>+        'MOZ_SYMBOLS_EXTRA_BUILDID': 'electrolysis',
>+        'LD_LIBRARY_PATH':           '/tools/gcc-4.3.3/installed/lib'
>+    },
>+    'unittest-env': {
>+        'LD_LIBRARY_PATH': '/tools/gcc-4.3.3/installed/lib',
>+    }
> }

You can't do this like this -- you're overwriting the existing environment. You need to do it like the existing code, or by something like:
BRANCHES['electrolysis']['platforms']['linux']['env'].update({stuff})
BRANCHES['electrolysis']['platforms']['linux']['unittest-env'].update({stuff})
Attachment #425992 - Flags: review?(bhearsum) → review-
Attachment #425992 - Attachment is obsolete: true
Attachment #426299 - Flags: review?(bhearsum)
Attachment #425993 - Attachment is obsolete: true
Attachment #426300 - Flags: review?(bhearsum)
Attachment #426299 - Attachment description: [staging] make linux64 on staging to run unittests (1 of 2) → [staging] make linux64 to run unittests on staging (1 of 3)
Comment on attachment 426299 [details] [diff] [review]
[staging] make linux64 to run unittests on staging (1 of 3) 

Looks good to me
Attachment #426299 - Flags: review?(bhearsum) → review+
Comment on attachment 426300 [details] [diff] [review]
[buildbotcustom] make linux64 to run unittests on staging (2 of 3) 

>diff --git a/misc.py b/misc.py
>--- a/misc.py
>+++ b/misc.py
>@@ -239,16 +239,19 @@ def generateBranchObjects(config, name):
>     # These dicts provides mapping between en-US dep and  nightly scheduler names
>     # to l10n dep and l10n nightly scheduler names. It's filled out just below here.
>     l10nBuilders = {}
>     l10nNightlyBuilders = {}
>     # generate a list of builders, nightly builders (names must be different)
>     # for easy access
>     for platform in config['platforms'].keys():
>         pf = config['platforms'][platform]
>+        print name
>+        print platform
>+        print dir(pf)

You probably want to get rid of these ;-). Looks fine otherwise.
Attachment #426300 - Flags: review?(bhearsum) → review+
Attachment #426301 - Flags: review?(bhearsum) → review+
Comment on attachment 426301 [details] [diff] [review]
[tools] make linux64 to run unittests on staging (3 of 3) 

http://hg.mozilla.org/build/tools/rev/60964b213cda
Attachment #426301 - Flags: checked-in+
We are not landing the patches yet.
We are going to run moz2-linux-slave09 on sm03 using these puppet manifests:
http://hg.mozilla.org/users/bhearsum_mozilla.com/puppet-manifests-linux64
Assignee: armenzg → nobody
Component: Release Engineering: Future → Release Engineering
Attachment #426712 - Flags: review?(bhearsum)
Attachment #426713 - Flags: review?(bhearsum)
Assignee: nobody → armenzg
Attachment #426712 - Flags: review?(bhearsum)
Comment on attachment 426713 [details] [diff] [review]
[custom] disable downloading symbols for Linux 64 (2 of 2)

You missed the second 'UnittestPackagedBuildFactory'. Also, it would be nice to update the generateCCTestBuilder function at the same time -- can you add the downloadSymbols part to all of those?
Attachment #426713 - Flags: review?(bhearsum) → review-
Attachment #426713 - Attachment is obsolete: true
Attachment #426728 - Flags: review?
This also includes the flag in production.
Attachment #426712 - Attachment is obsolete: true
Attachment #426729 - Flags: review?(bhearsum)
Attachment #426729 - Flags: review?(bhearsum) → review+
Attachment #426728 - Flags: review?
Attachment #426728 - Attachment is obsolete: true
Attachment #426738 - Flags: review?(bhearsum)
As I mentioned in IRC I found a one line problem. Reattached and carrying forward the positive review.
-BRANCHES['electrolysis']['platforms']['linux64']['download_symbols'] = False
Attachment #426729 - Attachment is obsolete: true
Attachment #426748 - Flags: review+
Attachment #426738 - Flags: review?(bhearsum) → review+
Comment on attachment 426299 [details] [diff] [review]
[staging] make linux64 to run unittests on staging (1 of 3) 

http://hg.mozilla.org/build/buildbot-configs/rev/299ecb8cbe12
Attachment #426299 - Flags: checked-in+
Comment on attachment 426748 [details] [diff] [review]
[buildbot-configs] disable downloading symbols for Linux 64 (1 of 2) (v1)

http://hg.mozilla.org/build/buildbot-configs/rev/23a90ba9945b
Attachment #426748 - Flags: checked-in+
Comment on attachment 426300 [details] [diff] [review]
[buildbotcustom] make linux64 to run unittests on staging (2 of 3) 

http://hg.mozilla.org/build/buildbotcustom/rev/c731d2bf6b14
Attachment #426300 - Flags: checked-in+
Comment on attachment 426738 [details] [diff] [review]
[custom] disable downloading symbols for Linux 64 (2 of 2) (v2)

http://hg.mozilla.org/build/buildbotcustom/rev/98d1d71282ae
Attachment #426738 - Flags: checked-in+
Attachment #427168 - Flags: review?(bhearsum)
Attachment #427168 - Flags: review?(bhearsum) → review+
Comment on attachment 427168 [details] [diff] [review]
[staging] bustage fix for staging

http://hg.mozilla.org/build/buildbot-configs/rev/5726db2625e5
Attachment #427168 - Flags: checked-in+
Depends on: 547029
Depends on: 547056
Whiteboard: [linux64] → [linux64][unittest]
moz2-linux-slave09 is running unit tests on sm03 since yesterday.
Builds are looking good. The only builds that are orange so far are:
* "mozilla-central opt test mochitest-other"
* "mozilla-central debug test mochitest-other"
I will look again tomorrow morning.
These are the other oranges
* Linux x86-64 places opt test mochitest-other 
* Linux x86-64 places debug test mochitest-other
* Linux x86-64 mozilla-central opt test mochitest-other
* Linux x86-64 mozilla-central debug test mochitests-1/5

It seems that mochitest-other is consistent orange and 1/5 only happened once.
Depends on: 548565
To fully support Linux 64 we have to run unit tests rather than the other way around. Reversed the dependency/blockage.
Blocks: support-L64
No longer depends on: support-L64
I have put slave09 back into the pool.

Latest updates with this bug is that we are going to be running unittests on talos machines (bug 548768). I will wait for next week's meeting to find out where do I fit in all the remaining work.

Lowering priority until then.
Priority: P2 → P3
talos-r3-fed64-001 has been set aside for this work.

Please note when you are done with this box so that it can be put back in the production talos pool.
Reversing the dependency.

Unit tests for Linux 64 bits will be run on user desktop (talos - bug 548768) instead of build machines.
No longer blocks: 548768
Depends on: 548768
Priority: P3 → P2
Depends on: 552430
Adjusting the priority.

We are first deploying unit tests on Leopard and Snow Leopard.
Later we will deploy unit tests on Fedora and Fedora 64.

There is really nothing being worked on this bug but waiting for bug 548768 to have unit tests running on Fedora 64.
Priority: P2 → P3
Depends on: 557918
No longer depends on: 548768
There is nothing really to be done in this bug.

We are currently running Fedora 64 bit unit tests on production.
We will enable unit tests on more branches once all perma-oranges are dealt with and we have the capacity of Rev3 testing machines that we need.

For anyone that wants to follow when all oranges will be done follow bug 554934.
If you want to know on which branches we will enable the unit tests follow bug 557918.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.