Closed Bug 386338 Opened 13 years ago Closed 8 years ago

Diff Release mozconfig with depend/nightly mozconfigs at start of release automation

Categories

(Release Engineering :: General, defect, P5)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: joduinn, Assigned: lsblakk)

References

Details

(Whiteboard: [simple][automation])

Attachments

(1 file, 10 obsolete files)

21.39 KB, patch
catlee
: review+
lsblakk
: checked-in+
Details | Diff | Splinter Review
Currently, we bring forward a version of mozconfig from a release branch, and manually merge in any changes from what is used for nightlies.

Can we just use what is in nightly, and not manually change/update anything?
Priority: -- → P3
1) I'd like to review whats used by nightlies vs release, and for any differences, confirm if they are really intentional differences.

2) for any differences that are really needed, if we can think of some way to deal with them programatically, and not keep needing manual reviews, that would be great.
(In reply to comment #1)
> 1) I'd like to review whats used by nightlies vs release, and for any
> differences, confirm if they are really intentional differences.

We should do this on every release.

> 2) for any differences that are really needed, if we can think of some way to
> deal with them programatically, and not keep needing manual reviews, that would
> be great.

There are several things that must be different, e.g. for an alpha:

ac_add_options --enable-update-channel=beta
ac_add_options --with-branding=browser/branding/unofficial
mk_add_options MOZ_MAKE_FLAGS="-j1"

For a nightly:

# tests, e.g.:
ac_add_options --enable-codesighs
ac_add_options --enable-update-channel=nightly
mk_add_options MOZ_MAKE_FLAGS="-j5"


We have automated the whole tinder-config/version bumping process (using Bootstrap/Config->Bump()), so it's certainly doable. So there are intentional differences, and it doesn't take a lot of time so I'm not sure how worth automating it is right now, compared to the higher-profile items on our plate.

Just to be clear though, it was a conscious decision to split the release and nightly configs. The old way of doing this was to edit the nightly config, make some changes, and change it back when done (of course something would always be missed and the first nightly would always be busted; or we'd miss something in the release).
Because mozconfig is just a shell script, we could theoretically do something like this:

if test -n "$MOZ_RELEASE_FOR_REAL";
mk_add_options MOZ_MAKE_FLAGS="-j1"
else
mk_add_options MOZ_MAKE_FLAGS="-j4"
fi

However, you still have the problem then of telling "something" to set MOZ_RELEASE_FOR_REAL.
Assignee: build → nobody
QA Contact: mozpreed → build
During F3.0b3, we just now discovered that rc1 was built with a mozconfig out of sync with the nightly mozconfig. The nightly mozconfig had been modified on 13dec07 as part of bug#407794, and we never noticed to make the corresponding change to the release machines.

For now, we have added a manual "diff the mozconfig" step to the manual build instructions before starting a build. However, generating release mozconfig programatically from the nightly mozconfig would have avoided this problem, and avoided the entire respin.
Summary: Can Release build process use default mozconfig? → Can Release mozconfig be generated from default/nightly mozconfig?
From triage meeting, we're uncomfortable with auto-generating mozconfig. 

However, it seems useful to:
- diff release mozconfig with default/nightly mozconfig
- exclude any expected/known differences
- fail out if there are any remaining differences, as these are by definition unexpected.

Tweaking summary to match.
Component: Release Engineering → Release Engineering: Future
QA Contact: build → release
Summary: Can Release mozconfig be generated from default/nightly mozconfig? → Can Release mozconfig be automatically diff'd with default/nightly mozconfig?
Found during triage. Even though we dont hit this often, its bad news when we
do, so this still feels like a valid bug.
Tweaked summary to clarify.

Basically, we want to make sure that any changes to depend/nightly mozconfigs
are also in the release mozconfigs. There are some known diffs, but unexpected
diffs should be flagged before the release builds start.
Summary: Can Release mozconfig be automatically diff'd with default/nightly mozconfig? → Diff Release mozconfig with depend/nightly mozconfigs at start of release automation
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Whiteboard: [simple]
triaged - p5
Priority: P3 → P5
Whiteboard: [simple] → [simple][automation]
Assignee: nobody → jhford
This program takes two files and diffs them.  It ignores things that are be known to be different on a line by line basis.  This script does not currently understand sourcing other mozconfigs and I don't know if it should because that would require moving both mozconfigs to the topsrcdir in order to source relative file paths properly or having a full mozilla-central checkout present.  For mozconfigs that have logic (e.g. tryserver and universal build mozconfigs) it will do a line by line diff.  Two methods for accounting for this are to set the allowed_differences list to have the entire blob of script, or run it in a shell.  If we do decide to run the mozconfig through a shell, we can define 

function ac_add_option {
  echo ac_add_option $@
}

function mk_add_option {
  echo mk_add_option $@
}

to get the processed mozconfig output.  This is out of scope for the purpose of this bug (nightly vs release).

Sample output:
python diff-mozconfig.py buildbot-configs/mozilla2/win32/mozilla-central/nightly/mozconfig buildbot-configs/mozilla2/win32/mozilla-central/release/mozconfig 
There are issues that need investigation:
  release: |. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags|


Things that still need to be done:
-OptionParser to make the script nicer to use
-Unit tests -- seems important
-hook into release process
-potentially break the large diff_files into smaller functions (low priority)
Attachment #464878 - Attachment mime type: text/x-python-script → text/plain
Attached file updates (obsolete) —
jhford-wifi:diff-mozconfigs jhford$ python diff-mozconfig.py buildbot-configs/mozilla2/linux/mozilla-central/nightly/mozconfig buildbot-configs/mozilla2/linux/mozilla-central/release/mozconfig -l nightly -r release
There are issues that need investigation:
  only in nightly: |ac_add_options --enable-codesighs|
  only in nightly: |CC="ccache /tools/gcc-4.3.3/installed/bin/gcc"|
  only in nightly: |CXX="ccache /tools/gcc-4.3.3/installed/bin/g++"|
  only in nightly: |# more options for 1.9 vs mozilla-central perf comparisons|
  only in nightly: |# shouldn't do anything - we don't do profiled builds on linux|
  only in nightly: |# Enable parallel compiling|
  only in release: |CC=/tools/gcc-4.3.3/installed/bin/gcc|
  only in release: |CXX=/tools/gcc-4.3.3/installed/bin/g++|

We could fix this by using something like:

REAL_CC=/tools/gcc-4.3.3/installed/bin/gcc
CC="ccache $REAL_CC"
REAL_CXX=/tools/gcc-4.3.3/installed/bin/g++
CXX="ccache $REAL_CXX"

in the nightly, and

REAL_CC=/tools/gcc-4.3.3/installed/bin/gcc
CC="$REAL_CC"
REAL_CXX=/tools/gcc-4.3.3/installed/bin/g++
CXX="$REAL_CXX"

in the release mozconfig.  This means that we can ignore the difference between CC="REAL_CC" and CC="ccache REAL_CC" and the same for CXX.  This seperates the differences on a line by line basis between which compiler we use and how it is invoked.
Attachment #464878 - Attachment is obsolete: true
Here are some issues that I have found (diffing release vs nightly):

|--enable-codesighs| is only on linux,linux64,macosx,macosx64 nightlies for mozilla-central, mozilla-1.9.2 and mozilla-1.9.1

|--disable-install-strip| is only on macosx,macosx64 nightlies for mozilla-central, mozilla-1.9.2

|. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags| is only on win32 nightlies for mozilla-central,mozilla-1.9.2 and mozilla-1.9.1

|ac_add_options --enable-debug-symbols="-gdwarf-2"| is only on macosx mozilla-1.9.2 nightlies

|ac_add_options --with-crashreporter-enable-percent=10| is only on mozilla-1.9.1 win32 release builds

|ac_add_options --disable-tests| is only on win32 mozilla-1.9.1 release builds.

Not sure which of these differences are harmless and which (if any) are actual differences.  I can account for the differences in my patch if they are valid differences.
also

|export ac_cv_have_visibility_class_bug=yes| is only on mozilla-central linux64 nightly builds
Attached patch buildbot-configs patch v1 (obsolete) — Splinter Review
This patch adds a mozconfig verification script to the buildbot-configs repository.  It is currently only useful for comparing nightly and release mozconfigs.  If desired, a new bug to do the same for tryserver and nightly/dep builds could be done.

I am fine with moving the ok_diffs*.dat files somewhere else if it is felt to be an issue.  Ideally, the reason for the ok_diffs*.dat files should be resolved.  I can write a README document if desired.
Attachment #464904 - Attachment is obsolete: true
Attachment #465009 - Flags: review?(nrthomas)
thanks aki, for pointing out the change to let the build system use ccache

http://mxr.mozilla.org/mozilla-central/source/configure.in#7428
which is defined by http://mxr.mozilla.org/mozilla-central/source/build/autoconf/altoptions.m4#108 and looking at configure.in looks like there is a safe fallback for ccache.

filed bug 590607 to track switching our mozconfigs to use --with-cache=<wherever it is> instead of CC='ccache gcc...'
Comment on attachment 465009 [details] [diff] [review]
buildbot-configs patch v1

These comments apply to verify_mozconfigs.py only, since the configs and mozconfigs are going to change for ccache.

>diff --git a/verify_mozconfigs.py b/verify_mozconfigs.py
>+        elif i.startswith('? '):
>+            not_in_either.append(i)

Uh, what's this ?

>+    def find_allowed(file_list, file_title):
>+        for f in file_list:

These are really matching lines, so using file_list and f is a little confusing.

>+    allowed_differences = []
>+    allowed_differences.extend(parse_dat_file('ok_diffs.dat'))
>+    for b in branches:
>+        allowed_differences.extend(parse_dat_file('ok_diffs_%s.dat' % b))
>+    for p in platforms:
>+        allowed_differences.extend(parse_dat_file('ok_diffs_%s.dat' % p))
>+    for b in branches:
>+        for p in platforms:
>+            allowed_differences.extend(parse_dat_file('ok_diffs_%s_%s.dat' % (b,p)))

This appears to remove the connection with the specific branch or platform, so that any rule may apply to any mozconfig. Surely not what you intended.

>+    except AssertionError, e:
>+        print 'ERROR: ', e.args[0]

Never actually write out the differences in non-verbose mode ?
Attachment #465009 - Flags: review?(nrthomas) → feedback-
Assignee: jhford → lsblakk
New approach - run this as part of release_sanity. Checks nightly/release mozconfigs and compares the diff to a whitelist per platform (see attached for example).

This can be tested locally using the following:

PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u lsblakk -V 7.0b4 --branch mozilla-beta --build-number 1 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py --dryrun --products firefox,fennec localhost:9018 -b -m -w mozconfig_whitelist
Attachment #465009 - Attachment is obsolete: true
Attachment #557720 - Flags: review?(catlee)
Attached file sample mozconfig whitelist (obsolete) —
Comment on attachment 557720 [details] [diff] [review]
checks mozconfigs against whitelist (see attached) as part of release_sanity

Review of attachment 557720 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildbot-helpers/release_sanity.py
@@ +79,5 @@
>          log.error("Repo does not exist with required revision. Check again, or use -b to bypass")
>          success = False
>      return success
>  
> +def verify_mozconfigs(branch, version, hghost, product, platforms, whitelist=None):

is None really a valid option? won't readConfig(None, ['whitelist']) fail?

@@ +97,5 @@
> +            mozconfigs.append(urllib2.urlopen(url).readlines())
> +        diffInstance = difflib.Differ()
> +        diffList = list(diffInstance.compare(mozconfigs[0],mozconfigs[1]))
> +        for line in diffList:
> +            if line[0] == '-' and len(line.strip()) > 1 and not '#' in line:

should we be checking new lines as well as deleted lines?

also, I think you need to be more careful about ignoring lines with comments. it's possible for a line to have a comment after real code.

@@ +101,5 @@
> +            if line[0] == '-' and len(line.strip()) > 1 and not '#' in line:
> +                # compare these to the whitelist
> +                line = line.replace('-','', 1).strip()
> +                if mozconfigWhitelist.has_key(platform) and line in mozconfigWhitelist[platform]:
> +                    pass

nit: use continue instead of pass here. then you can un-indent the block below
Attachment #557720 - Flags: review?(catlee) → review-
will attach an updated whitelist too.
Attachment #557720 - Attachment is obsolete: true
Attachment #557722 - Attachment is obsolete: true
Attachment #557898 - Flags: review?(catlee)
Attached file (v.2) mozconfig_whitelist (obsolete) —
added an else: mozconfigWhitelist = {} so that it doesn't fail out later on has_key()
Attachment #557898 - Attachment is obsolete: true
Attachment #557902 - Flags: review?(catlee)
Attachment #557898 - Flags: review?(catlee)
Comment on attachment 557902 [details] [diff] [review]
(v.2) checks mozconfigs against whitelist (see attached) as part of release_sanity

Review of attachment 557902 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildbot-helpers/release_sanity.py
@@ +100,5 @@
> +            mozconfigs.append(urllib2.urlopen(url).readlines())
> +        diffInstance = difflib.Differ()
> +        diffList = list(diffInstance.compare(mozconfigs[0],mozconfigs[1]))
> +        for line in diffList:
> +            clean_line = re.sub(r'[-+]','',line,1).strip()

how about
clean_line = line[1:].strip()

what you have will strip out the first +/- from the line, regardless of the line starts with it or not.

@@ +111,5 @@
> +                    and line.replace('-','',1).strip() in mozconfigWhitelist['release'][platform]:
> +                        continue
> +                if line.startswith('+') and mozconfigWhitelist['nightly'].has_key(platform) \
> +                    and line.replace('+','',1).strip() in mozconfigWhitelist['nightly'][platform]:
> +                        continue

and then these can be replaced with, e.g.

if line[0] == '-' and mozconfigWhitelist.get('release', {}).has_key(platform) and clean_line in mozconfigWhitelist['release'][platform]:

what you've got will fail with KeyError in the case where no mozconfigWhitelist is passed in.
Attachment #557902 - Flags: review?(catlee) → review-
thanks for the tip - this one runs through with or without a whitelist file now.
Attachment #557902 - Attachment is obsolete: true
Attachment #558509 - Flags: review?(catlee)
Comment on attachment 558509 [details] [diff] [review]
(v.3) checks mozconfigs against whitelist (see attached) as part of release_sanity

Review of attachment 558509 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildbot-helpers/release_sanity.py
@@ +111,5 @@
> +                    and line[1:].strip() in mozconfigWhitelist['release'][platform]:
> +                        continue
> +                if line[0] == '+' and mozconfigWhitelist.get('release', {}).has_key(platform) \
> +                    and line[1:].strip() in mozconfigWhitelist['nightly'][platform]:
> +                        continue

I think you can use clean_line here and above instead of line[1:].strip()

also, the second if should use 'nightly' in both places instead of 'release', right?

@@ +273,5 @@
>              help="specify the release-config files (the first is primary)")
>      parser.add_option("-p", "--products", dest="products",
>              help="coma separated list of products")
> +    parser.add_option("-m", "--mozconfig", dest="mozconfig", action="store_true",
> +            help="just do a diff of mozconfigs")

looks like there's no way to disable this? it defaults to True above.
Output from running with/without a whitelist in testing:

Running with whitelist...
2011-09-08 18:01:44,396 : INFO : Comparing fennec nightly/release mozconfigs...
2011-09-08 18:01:44,830 : INFO : Comparing firefox nightly/release mozconfigs...
2011-09-08 18:01:45,318 : INFO : command: START
2011-09-08 18:01:45,319 : INFO : command: buildbot --version
2011-09-08 18:01:45,319 : INFO : command: cwd: /Users/lsb/mozilla/code/build-master/master
2011-09-08 18:01:45,320 : INFO : command: output:
Buildbot version: 0.8.1
Twisted version: 8.2.0
2011-09-08 18:01:45,526 : INFO : command: END (0.21s elapsed)
2011-09-08 18:01:45,527 : INFO : Tests Passed! Did not run reconfig/sendchange. Rerun without `-d`


Running withOUT whitelist...
2011-09-08 18:01:45,873 : INFO : Comparing fennec nightly/release mozconfigs...
2011-09-08 18:01:45,961 : ERROR : FAIL mozconfig diff: not in whitelist: linux-maemo5-gtk/release --mk_add_options MOZ_MAKE_FLAGS=-j4
2011-09-08 18:01:45,961 : ERROR : FAIL mozconfig diff: not in whitelist: linux-maemo5-gtk/nightly --ac_add_options --enable-official-branding
2011-09-08 18:01:46,214 : ERROR : FAIL mozconfig diff: not in whitelist: linux-android/release --mk_add_options MOZ_MAKE_FLAGS=-j4
2011-09-08 18:01:46,214 : ERROR : FAIL mozconfig diff: not in whitelist: linux-android/nightly --ac_add_options --enable-update-channel=beta
2011-09-08 18:01:46,214 : ERROR : FAIL mozconfig diff: not in whitelist: linux-android/release --ac_add_options --enable-update-channel=nightly
2011-09-08 18:01:46,215 : ERROR : FAIL mozconfig diff: not in whitelist: linux-android/nightly --ac_add_options --enable-official-branding
2011-09-08 18:01:46,296 : ERROR : FAIL mozconfig diff: not in whitelist: linux-mobile/nightly --CC=/tools/gcc-4.3.3/installed/bin/gcc
2011-09-08 18:01:46,296 : ERROR : FAIL mozconfig diff: not in whitelist: linux-mobile/release --CC=/tools/gcc-4.5/bin/gcc
2011-09-08 18:01:46,296 : ERROR : FAIL mozconfig diff: not in whitelist: linux-mobile/nightly --CXX=/tools/gcc-4.3.3/installed/bin/g++
2011-09-08 18:01:46,296 : ERROR : FAIL mozconfig diff: not in whitelist: linux-mobile/release --CXX=/tools/gcc-4.5/bin/g++
2011-09-08 18:01:46,297 : ERROR : FAIL mozconfig diff: not in whitelist: linux-mobile/release --ac_add_options --enable-stdcxx-compat
2011-09-08 18:01:46,434 : ERROR : Error verifying mozconfigs
2011-09-08 18:01:46,436 : INFO : Comparing firefox nightly/release mozconfigs...
2011-09-08 18:01:46,504 : ERROR : FAIL mozconfig diff: not in whitelist: linux/nightly --ac_add_options --enable-update-channel=beta
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/release --ac_add_options --enable-update-channel=nightly
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/nightly --ac_add_options --enable-official-branding
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/release --ac_add_options --enable-codesighs
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/release --export MOZILLA_OFFICIAL=1
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/release --export MOZ_TELEMETRY_REPORTING=1
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/nightly --export MOZILLA_OFFICIAL=1
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/release --mk_add_options MOZ_MAKE_FLAGS="-j4"
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/nightly --export MOZ_TELEMETRY_REPORTING=1
2011-09-08 18:01:46,505 : ERROR : FAIL mozconfig diff: not in whitelist: linux/release --ac_add_options --with-ccache=/usr/bin/ccache
2011-09-08 18:01:46,583 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/nightly --ac_add_options --enable-update-channel=beta
2011-09-08 18:01:46,583 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/release --ac_add_options --enable-update-channel=nightly
2011-09-08 18:01:46,583 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/nightly --ac_add_options --enable-official-branding
2011-09-08 18:01:46,583 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/release --ac_add_options --enable-codesighs
2011-09-08 18:01:46,583 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/release --export MOZILLA_OFFICIAL=1
2011-09-08 18:01:46,584 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/release --export MOZ_TELEMETRY_REPORTING=1
2011-09-08 18:01:46,584 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/nightly --export MOZILLA_OFFICIAL=1
2011-09-08 18:01:46,584 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/release --mk_add_options MOZ_MAKE_FLAGS="-j4"
2011-09-08 18:01:46,584 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/nightly --export MOZ_TELEMETRY_REPORTING=1
2011-09-08 18:01:46,584 : ERROR : FAIL mozconfig diff: not in whitelist: linux64/release --ac_add_options --with-ccache=/usr/bin/ccache
2011-09-08 18:01:46,653 : ERROR : FAIL mozconfig diff: not in whitelist: win32/nightly --ac_add_options --enable-update-channel=beta
2011-09-08 18:01:46,653 : ERROR : FAIL mozconfig diff: not in whitelist: win32/release --ac_add_options --enable-update-channel=nightly
2011-09-08 18:01:46,653 : ERROR : FAIL mozconfig diff: not in whitelist: win32/nightly --ac_add_options --enable-official-branding
2011-09-08 18:01:46,653 : ERROR : FAIL mozconfig diff: not in whitelist: win32/release --. $topsrcdir/configs/mozilla2/win32/include/choose-make-flags
2011-09-08 18:01:46,743 : ERROR : FAIL mozconfig diff: not in whitelist: macosx64/nightly --ac_add_options --enable-update-channel=beta
2011-09-08 18:01:46,743 : ERROR : FAIL mozconfig diff: not in whitelist: macosx64/release --ac_add_options --enable-update-channel=nightly
2011-09-08 18:01:46,744 : ERROR : FAIL mozconfig diff: not in whitelist: macosx64/nightly --ac_add_options --enable-official-branding
2011-09-08 18:01:46,744 : ERROR : FAIL mozconfig diff: not in whitelist: macosx64/release --ac_add_options --enable-codesighs
2011-09-08 18:01:46,744 : ERROR : FAIL mozconfig diff: not in whitelist: macosx64/release --ac_add_options --disable-install-strip
2011-09-08 18:01:46,744 : ERROR : FAIL mozconfig diff: not in whitelist: macosx64/release --mk_add_options MOZ_MAKE_FLAGS="-j4"
2011-09-08 18:01:46,744 : ERROR : Error verifying mozconfigs
2011-09-08 18:01:46,744 : INFO : command: START
2011-09-08 18:01:46,746 : INFO : command: buildbot --version
2011-09-08 18:01:46,746 : INFO : command: cwd: /Users/lsb/mozilla/code/build-master/master
2011-09-08 18:01:46,746 : INFO : command: output:
Buildbot version: 0.8.1
Twisted version: 8.2.0
2011-09-08 18:01:46,961 : INFO : command: END (0.21s elapsed)

2011-09-08 18:01:46,961 : CRITICAL : Tests Failed! Not running sendchange!
Attachment #558509 - Attachment is obsolete: true
Attachment #559357 - Flags: review?(catlee)
Attachment #558509 - Flags: review?(catlee)
Attachment #559357 - Flags: review?(catlee)
Updated to include the whitelist in tools/buildbot-helpers/mozconfig_whitelist so that this is all one patch.  I have tested this with the current production release configs for all three release branches with the following script:

echo "Running for a beta...\n++++++++++++++\n"
PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u lsblakk -V 8.0b3 --branch mozilla-beta --build-number 1 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py --dryrun --products firefox,fennec localhost:9018 -w ../tools/buildbot-helpers/mozconfig_whitelist
echo "Running for a release whitelist...\n++++++++++++++\n"
PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u lsblakk -V 7.0.1 --branch mozilla-release --build-number 1 -c release-firefox-mozilla-release.py -c release-fennec-mozilla-release.py --dryrun --products firefox,fennec localhost:9018  -w mozconfig_whitelist
echo "Running for a 3.6 release whitelist...\n++++++++++++++\n"
PYTHONPATH=. ../bin/python ../tools/buildbot-helpers/release_sanity.py -u lsblakk -V 3.6.23 --branch mozilla-1.9.2 --build-number 1 -c release-firefox-mozilla-1.9.2.py --dryrun --products firefox localhost:9018  -w ../tools/buildbot-helpers/mozconfig_whitelist

I also added in an error_tally set because the amount of info scrolling by is so much that it would be great to have a summary at the end of which test steps failed.
Attachment #557899 - Attachment is obsolete: true
Attachment #559357 - Attachment is obsolete: true
Attachment #568143 - Flags: review?(catlee)
Attachment #568143 - Flags: review?(catlee) → review+
Comment on attachment 568143 [details] [diff] [review]
(v.5) checks mozconfigs against whitelist (included) as part of release_sanity

http://hg.mozilla.org/build/tools/rev/40a4916e512c

updated default for whitelist='mozconfig_whitelist'
Attachment #568143 - Flags: checked-in+
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Depends on: 697611
Blocks: 697611
No longer depends on: 697611
release_sanity failed with

==================================
(build1)[cltbld@buildbot-master08 master]$ PYTHONPATH=. python ../tools/buildbot-helpers/release_sanity.py -u rail -pfirefox,fennec -V 8.0b5 --branch mozilla-beta --build-number 1 -c release-firefox-mozilla-beta.py -c release-fennec-mozilla-beta.py --dryrun localhost:9001Traceback (most recent call last):
  File "../tools/buildbot-helpers/release_sanity.py", line 349, in <module>
    options.whitelist
  File "../tools/buildbot-helpers/release_sanity.py", line 93, in verify_mozconfigs
    mozconfigWhitelist = readConfig(whitelist, ['whitelist'])
  File "/builds/buildbot/build1/tools/lib/python/release/info.py", line 71, in readConfig
    execfile(configfile, c)
IOError: [Errno 2] No such file or directory: 'mozconfig_whitelist'

==================================

As a workaround I did this:

(build1)[cltbld@buildbot-master08 master]$ ln -s ../tools/buildbot-helpers/mozconfig_whitelist ./
Thanks for pointing that out, I have updated the default path (and tested to be sure it still worked). http://hg.mozilla.org/build/tools/rev/afc184e156cd
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.