Migrate SeaMonkey's release process off CVS

RESOLVED FIXED

Status

defect
--
blocker
RESOLVED FIXED
5 years ago
3 years ago

People

(Reporter: ewong, Assigned: ewong)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(10 attachments, 17 obsolete attachments)

386.00 KB, patch
Callek
: review+
Details | Diff | Splinter Review
36.35 KB, patch
ewong
: review+
Details | Diff | Splinter Review
9.39 KB, patch
Callek
: review+
Details | Diff | Splinter Review
1.16 KB, patch
Details | Diff | Splinter Review
777 bytes, patch
Details | Diff | Splinter Review
823 bytes, patch
Details | Diff | Splinter Review
1.00 KB, patch
Details | Diff | Splinter Review
2.61 KB, patch
Details | Diff | Splinter Review
2.34 KB, patch
ewong
: review+
Details | Diff | Splinter Review
1.73 KB, patch
Details | Diff | Splinter Review
Assignee

Description

5 years ago
SeaMonkey's Release process depends on CVS for the patcher-configs
update.  As per bug 1057920, we need to move away from CVS.  

RelEng-wise, the patcher-configs has to be moved to Mercurial (if it
hasn't been done so... ) and the buildbotcustom code needs to be
modified to reflect this change.

This needs to be done prior to CVS' shutdown in 2015 Q1. (which is soon.)
Status? End of March is approaching quickly, when CVS will shut down (see also bug 1057920 comment 7)
Flags: needinfo?(ewong)
Assignee

Comment 2

4 years ago
Hi hal,

Thanks for poking me on this. 

I haven't done anything yet on this as I was working on the release and
buildbot code bustage.  

Will get this done next.
Flags: needinfo?(ewong)
Assignee

Updated

4 years ago
Assignee: nobody → ewong
Status: NEW → ASSIGNED
Assignee

Updated

4 years ago
Depends on: 1142868
Assignee

Updated

4 years ago
No longer depends on: 1142868
Assignee

Comment 3

4 years ago
this is the easy part.
Attachment #8577096 - Flags: review?(bugspam.Callek)
As Laura commented in bug 1057920, CVS is scheduled to be decommissioned by today. We can hold on a little while longer but we need to close out the SeaMonkey bugs.

Edmund/Callek - Can you please work to get this bug resolved.
Flags: needinfo?(ewong)
Flags: needinfo?(bugspam.Callek)
(In reply to Lawrence Mandel [:lmandel] (use needinfo) from comment #4)
> As Laura commented in bug 1057920, CVS is scheduled to be decommissioned by
> today. We can hold on a little while longer but we need to close out the
> SeaMonkey bugs.
> 
> Edmund/Callek - Can you please work to get this bug resolved.

Per talk with Lawrence, while this is a blocker for SeaMonkey to be off CVS, there is a blocker to us testing/validating whatever we do here.

And that is something that is also blocking our releases at present.

Current Plan:

* If we have that other blocker fixed before CVS is down, we'll do a release using the CVS process that exists, and work diligently to move off CVS in that regard
* We'll declare this bug (here) no longer a blocker to CVS decom as long as we have at least 1-2 business days notice, that allows us to archive the required data for the move somewhere.
* We'll archive the data and use that when it does come time for us to migrate.
Flags: needinfo?(bugspam.Callek)
Assignee

Updated

4 years ago
Flags: needinfo?(ewong)
Assignee

Comment 6

4 years ago
I'm wondering if we need the following cvs repos converted to hg as well?

mozilla/tools/patcher,
mozilla/tools/patcher-configs
mozilla/tools/release/MozBuild
mozilla/tools/release/Bootstrap/Util.pm

reason for the patcher one is because of the use of
UPDATES_RELEASE_R18 tag that we use.   Right now, it's just
a straight push-the-current-contents-of-patcher tools to
Callek's tools repo, but that doesn't include the UPDATES_RELEASE_R18
tag.   Or do we just use the default tag?
Flags: needinfo?(bugspam.Callek)
Assignee

Comment 7

4 years ago
Including the UPDATE_PACKAGING_R18 tag.
Attachment #8577096 - Attachment is obsolete: true
Attachment #8577096 - Flags: review?(bugspam.Callek)
Flags: needinfo?(bugspam.Callek)
Attachment #8593181 - Flags: review?(bugspam.Callek)
Assignee

Comment 8

4 years ago
Comment on attachment 8593181 [details] [diff] [review]
Adding all related patcher files to one repo  (v1) [option #1]

Actually... this is the wrong approach.
Attachment #8593181 - Flags: review?(bugspam.Callek)
Assignee

Comment 9

4 years ago
I'm wondering if it's a better idea if we put the patcher stuff into
the hg.mo/SeaMonkey/patcher repo.  

The reason why I suggest this is because if I were to put the patcher
files into Callek's tools repo, and tag it with UPDATE_PACKAGING_R18,
during the patcher steps, the UPDATE_PACKAGING_R18 tag will be before
all those SEAMONKEY_2_??_RELEASE tags and I'm concerned it might
affect the process.

i.e. in hg.mo/users/Callek_gmail.com/tools,

  <cset-latest>  tagged with SEAMONKEY_<ver>_RELEASE  
   .
   .
   <cset ###>  tagged with UPDATE_PACKAGING_R18

so anything between cset-latest and cset ### during the patcher step is
ignored..  

In any event,  I'll have two patches made.
1) if it's under hg.mo/SeaMonkey/patcher
2) if it's under hg.mo/users/Callek_gmail.com/tools/release/updates/patcher
   (and remove the current files in ./updates/patcher)
Assignee

Updated

4 years ago
Attachment #8593181 - Attachment description: Adding all related patcher files to one repo (v1) → Adding all related patcher files to one repo (v1) [option #1]
Assignee

Comment 10

4 years ago
Attachment #8593183 - Flags: review?(bugspam.Callek)
Assignee

Updated

4 years ago
Attachment #8593181 - Flags: review?(bugspam.Callek)
Assignee

Comment 11

4 years ago
Once the option has been selected, I'll need to update the buildbotcustom code.
Assignee

Comment 12

4 years ago
After speaking with Callek over irc, the best bet are both options with
the following changes:

1) a new repo is created under hg.mozilla.org/SeaMonkey called tools.
   i.e. hg.mozilla.org/SeaMonkey/tools

2) Callek's tools repo is migrated/moved to it.

3) the patcher files and patcher configs are moved to a directory under
   the tools/ repo.  (Where?  the option#2 patch above puts them 
   in release/updates, but there are already files in there (patcher-config
   files...*, so I'm going to reorganize the release/updates folder.)


* - apparently (from what I remember), we actually update both the CVS
    patcher-config files and Callek's tools/release/updates/ patcher-config
    files. (which is why when something chokes, we need to back out both
    CVS and Callek's Tools changes)  I think (and I'll need Callek's
    confirmation) that we can even do away with the cvs patcher and have
    whatever reading off the CVS patcher-config changes to use Callek's tools
    patcher-config changes..  ) (or rather, the new tools/ repo.)

any opinions/questions?
Flags: needinfo?(bugspam.Callek)
Assignee

Updated

4 years ago
Depends on: 1186269
Assignee

Comment 13

4 years ago
Additional thing I noticed.

The patcher files refer to CVS and therefore, they must be modified
to refer to mercurial.
Assignee

Updated

4 years ago
Depends on: SM2.35
Assignee

Updated

4 years ago
No longer depends on: SM2.35
Assignee

Updated

4 years ago
Blocks: SM2.35
Assignee

Comment 14

4 years ago
Posted patch tools patch #1 (v1) (obsolete) — Splinter Review
This patch does the following:

1) Adds BootStrap, MozBuild, patcher folders to release/updates
2) Moves the original patcher-config files in releases/updates to
   release/updates/patcher-configs
Attachment #8593181 - Attachment is obsolete: true
Attachment #8593183 - Attachment is obsolete: true
Attachment #8593181 - Flags: review?(bugspam.Callek)
Attachment #8593183 - Flags: review?(bugspam.Callek)
Assignee

Updated

4 years ago
Attachment #8637024 - Flags: review?(bugspam.Callek)
Assignee

Updated

4 years ago
Attachment #8637024 - Attachment description: proposed patch(v1) → patch #1 (v1)
Assignee

Updated

4 years ago
Attachment #8637024 - Attachment description: patch #1 (v1) → tools patch #1 (v1)
Assignee

Updated

4 years ago
Depends on: 1189650
Assignee

Updated

4 years ago
Severity: normal → blocker
Assignee

Comment 15

4 years ago
Attachment #8649128 - Flags: review?(bugspam.Callek)
Assignee

Comment 16

4 years ago
Posted patch buildbot-config changes (v1) (obsolete) — Splinter Review
Attachment #8649129 - Flags: review?(bugspam.Callek)
Assignee

Comment 17

4 years ago
Comment on attachment 8649128 [details] [diff] [review]
buildbotcustom code changes (v1)

This is a WIP:

I had replaced the CVS checkout steps with copying the patcher/MozBuild/Bootstrap/patcher-configs to the build folder; but now, I realize that they must be within their original positions as they need to be diffed (particularly the patcher-configs) and the differences pushed to the repo.
Assignee

Comment 18

4 years ago
This patch leaves all those patcher/MozBuild/Bootstrap/patcher-configs where they are and basically run the whole patcher process in tools/release/updates.
Attachment #8649128 - Attachment is obsolete: true
Attachment #8649128 - Flags: review?(bugspam.Callek)
Attachment #8649159 - Flags: feedback?(bugspam.Callek)
Assignee

Comment 19

4 years ago
(In reply to Edmund Wong (:ewong) from comment #18)
> Created attachment 8649159 [details] [diff] [review]
> buildbotcustom code changes (v2)
> 
> This patch leaves all those patcher/MozBuild/Bootstrap/patcher-configs where
> they are and basically run the whole patcher process in
> tools/release/updates.

this is said being ignorant of what goes on in the patcher scripts (in PERL,
no less), but a quick scan of patcher2.pl, it doesn't look like it uses
the build/ stuff..   but not having touched perl that much, I'll need to
really look at those scripts.  (/me wonders if we can make those into
python scripts?)
Assignee

Comment 20

4 years ago
Posted patch tools patch (v2) (obsolete) — Splinter Review
Needed to fix the path to the required sh files from within verify.sh
Attachment #8637024 - Attachment is obsolete: true
Attachment #8637024 - Flags: review?(bugspam.Callek)
Attachment #8650465 - Flags: review?(bugspam.Callek)
Assignee

Comment 21

4 years ago
Comment on attachment 8650465 [details] [diff] [review]
tools patch (v2)

I bungled this patch.  Need to redo this patch with the proper patcher-configs
files.
Attachment #8650465 - Attachment is obsolete: true
Attachment #8650465 - Flags: review?(bugspam.Callek)
Assignee

Comment 22

4 years ago
Posted patch tools patch (v3) (obsolete) — Splinter Review
Attachment #8650501 - Flags: review?(bugspam.Callek)
Assignee

Comment 23

4 years ago
Attachment #8649159 - Attachment is obsolete: true
Attachment #8649159 - Flags: feedback?(bugspam.Callek)
Attachment #8650503 - Flags: review?(bugspam.Callek)
Comment on attachment 8650501 [details] [diff] [review]
tools patch (v3)

Review of attachment 8650501 [details] [diff] [review]:
-----------------------------------------------------------------

r+ if you do the following before checkin:

cd <tools-repo-directory>
find release/updates/patcher-configs/ ! -name *sea*
# Confirm file list does not include any SeaMonkey patcher configs
find release/updates/patcher-configs/ ! -name *sea* -delete
# Update patch with this result
Attachment #8650501 - Flags: review?(bugspam.Callek) → review+
Assignee

Comment 25

4 years ago
Attachment #8650501 - Attachment is obsolete: true
Attachment #8655449 - Flags: review+
Assignee

Updated

4 years ago
Attachment #8655449 - Flags: review+ → review?(bugspam.Callek)
Attachment #8655449 - Flags: review?(bugspam.Callek) → review+
Comment on attachment 8650503 [details] [diff] [review]
buildbotcustom code changes (v3)

Review of attachment 8650503 [details] [diff] [review]:
-----------------------------------------------------------------

I *think* this is all good, lets cross our fingers

::: process/factory.py
@@ +5400,5 @@
>          ))
>          self.addStep(TinderboxMockCommand(
>           name='diff_patcher_config',
>           command=['bash', '-c',
> +                  '(hg diff "%s") && (grep \

instead of dropping -u use -U8

@@ +5506,5 @@
>           env={'HGROOT': self.mozRepository,
>                'PATH': '/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin'},
>           haltOnFailure=True,
>           timeout=3600,
> +         workdir='tools/release/updates/patcher',

'tools/release/updates' not +/patcher

@@ +5593,5 @@
>           name='patcher_download_builds',
>           command=command,
>           description=['patcher:', 'download builds'],
>           haltOnFailure=True,
> +         workdir='tools/release/updates/patcher',

'tools/release/updates' not +/patcher

@@ +5611,5 @@
>           name='patcher_create_patches',
>           command=command,
>           description=['patcher:', 'create patches'],
>           haltOnFailure=True,
> +         workdir='tools/release/updates/patcher',

'tools/release/updates' not +/patcher
Attachment #8650503 - Flags: review?(bugspam.Callek) → review+
Comment on attachment 8649129 [details] [diff] [review]
buildbot-config changes (v1)

Review of attachment 8649129 [details] [diff] [review]:
-----------------------------------------------------------------

r- for the two mentioned issues.

::: seamonkey/config.py
@@ +96,5 @@
>      'stage_server': 'stage.mozilla.org',
>      'aus2_host': 'aus2-community.mozilla.org',
>      'download_base_url': 'http://ftp.mozilla.org/pub/mozilla.org/seamonkey',
>      'graph_server': 'graphs.mozilla.org',
> +    'build_tools_repo_path': 'SeaMonkey/tools',

don't change this *yet*

::: seamonkey/release_master.py
@@ +244,5 @@
>      buildNumber=releaseConfig['buildNumber'],
>      mozRepoPath=releaseConfig['mozillaRepoPath'],
>      inspectorRepoPath=releaseConfig['inspectorRepoPath'],
>      venkmanRepoPath=releaseConfig['venkmanRepoPath'],
>      chatzillaRepoPath=releaseConfig['chatzillaRepoPath'],

there are more cases of cvsroot in this file you need to cull
Attachment #8649129 - Flags: review?(bugspam.Callek) → review-
Assignee

Comment 28

4 years ago
Attachment #8650503 - Attachment is obsolete: true
Attachment #8655468 - Flags: review+
Assignee

Comment 29

4 years ago
Attachment #8649129 - Attachment is obsolete: true
Attachment #8655472 - Flags: review?(bugspam.Callek)
Comment on attachment 8655472 [details] [diff] [review]
[configs] buildbot-configs changes (v2) [checked-in]

Review of attachment 8655472 [details] [diff] [review]:
-----------------------------------------------------------------

Lets give this a go!
Attachment #8655472 - Flags: review?(bugspam.Callek) → review+
Assignee

Comment 31

4 years ago
Comment on attachment 8655472 [details] [diff] [review]
[configs] buildbot-configs changes (v2) [checked-in]

Pushed to buildbot-configs:
https://hg.mozilla.org/build/buildbot-configs/rev/5dc684b8f949
Assignee

Comment 32

4 years ago
Comment on attachment 8655468 [details] [diff] [review]
[custom] buildbotcustom code changes (v4) [checked-in]

Pushed to buildbotcustom:
https://hg.mozilla.org/build/buildbotcustom/rev/f6f4e701119e
Assignee

Comment 33

4 years ago
Comment on attachment 8655449 [details] [diff] [review]
[tools] patch (v4) [checked-in]

Pushed to users/Callek_gmail.com/tools:
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/4ae5a2886153
Assignee

Comment 34

4 years ago
OTR, looking through the patches again, I realized last night that it's not enough.  factory.py needs some path reference changes (patchConfigFile != patch-
configs).  patchConfigFile refers to the file which was previously
in tools/release/updates/mozRelease*-<platform>.cfg.

That's only just one thing.  So back to the drawing board...
Assignee

Comment 35

4 years ago
(In reply to Edmund Wong (:ewong) from comment #34)
> OTR, looking through the patches again, I realized last night that it's not
> enough.  factory.py needs some path reference changes (patchConfigFile !=
> patch-
> configs).  patchConfigFile refers to the file which was previously
> in tools/release/updates/mozRelease*-<platform>.cfg.
> 
correction: tools/release/updates/mozRelease-seamonkey-branch-patcher2.cfg
Assignee

Comment 36

4 years ago
Attachment #8655776 - Flags: review?(bugspam.Callek)
Assignee

Comment 37

4 years ago
Attachment #8655776 - Attachment is obsolete: true
Attachment #8655776 - Flags: review?(bugspam.Callek)
Attachment #8655778 - Flags: review?(bugspam.Callek)
Assignee

Comment 38

4 years ago
(for post-land review)
Attachment #8655813 - Flags: review?(bugspam.Callek)
Assignee

Comment 39

4 years ago
Comment on attachment 8655813 [details] [diff] [review]
[tools] correct path to retry.py command [checked-in]

Pushed to tools:
 https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/53913d27c44c
Assignee

Comment 41

4 years ago
Comment on attachment 8656001 [details] [diff] [review]
[tools] common/unpack.sh's mentions of common/ scripts needs changing.

Pushed to tools:
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/57632c74b44e
Assignee

Comment 42

4 years ago
Comment on attachment 8656001 [details] [diff] [review]
[tools] common/unpack.sh's mentions of common/ scripts needs changing.

Backed out for a better solution:
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/297721d736e4
Attachment #8656001 - Attachment is obsolete: true
Attachment #8656001 - Flags: review?(bugspam.Callek)
Assignee

Comment 43

4 years ago
Attachment #8656008 - Flags: review?(bugspam.Callek)
Assignee

Updated

4 years ago
Attachment #8656008 - Flags: review?(bugspam.Callek)
Assignee

Comment 44

4 years ago
Attachment #8656032 - Flags: review?(bugspam.Callek)
Assignee

Comment 45

4 years ago
Comment on attachment 8656032 [details] [diff] [review]
[tools] verify.sh's references to common/ paths needs changing. (*)

Pushed to tools:
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/6353d7525521
Assignee

Comment 47

4 years ago
Attachment #8656032 - Attachment is obsolete: true
Attachment #8656032 - Flags: review?(bugspam.Callek)
Attachment #8656043 - Flags: review?(bugspam.Callek)
Assignee

Comment 48

4 years ago
(In reply to Edmund Wong (:ewong) from comment #45)
> Comment on attachment 8656032 [details] [diff] [review]
> [tools] verify.sh's references to common/ paths needs changing. (*)
> 
> Pushed to tools:
> https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/6353d7525521

Backed this out. Pushed replacing patch:
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/59ef785b7317
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/306ffea7d128
Assignee

Updated

4 years ago
Attachment #8656332 - Attachment description: [custom] buildbotcustom code changes to fix final verification step → [custom] buildbotcustom code changes to fix final verification step (used with **)
Assignee

Comment 51

4 years ago
Comment on attachment 8656333 [details] [diff] [review]
[tools] fix for final verification step (**) [checked-in]

Pushed to tools:
https://hg.mozilla.org/users/Callek_gmail.com/tools/rev/618386cc6361
Assignee

Comment 53

4 years ago
Attachment #8659120 - Flags: review?(bugspam.Callek)
Assignee

Updated

4 years ago
Attachment #8655449 - Attachment description: tools patch (v4) → [tools] patch (v4) [checked-in]
Assignee

Updated

4 years ago
Attachment #8655472 - Attachment description: buildbot-configs changes (v2) → [configs] buildbot-configs changes (v2) [checked-in]
Assignee

Updated

4 years ago
Attachment #8655468 - Attachment description: buildbotcustom code changes (v4) → [custom] buildbotcustom code changes (v4) [checked-in]
Assignee

Updated

4 years ago
Attachment #8655813 - Attachment description: [tools] correct path to retry.py command → [tools] correct path to retry.py command [checked-in]
Assignee

Updated

4 years ago
Attachment #8656008 - Attachment is obsolete: true
Assignee

Comment 55

4 years ago
Comment on attachment 8659782 [details] [diff] [review]
[custom] Add -I to patcher command to get it to recognize the patcher files. [checked-in]

Pushed to buildbotcustom:
https://hg.mozilla.org/build/buildbotcustom/rev/97be4923e7fb
Assignee

Comment 56

4 years ago
Comment on attachment 8655778 [details] [diff] [review]
[custom] Use right user for commit during commit_patcher_config [checked-in]

Pushed to buildbotcustom:
https://hg.mozilla.org/build/buildbotcustom/rev/935f9d8784db
Attachment #8655778 - Attachment description: [custom] Use right user for commit during commit_patcher_config → [custom] Use right user for commit during commit_patcher_config [checked-in]
Assignee

Updated

4 years ago
Attachment #8656043 - Attachment description: [tools] verify.sh's references to common/ paths needs changing. (*) → [tools] verify.sh's references to common/ paths needs changing. (*) [checked-in]
Assignee

Updated

4 years ago
Attachment #8656333 - Attachment description: [tools] fix for final verification step (**) → [tools] fix for final verification step (**) [checked-in]
Assignee

Comment 57

4 years ago
Attachment #8656036 - Attachment is obsolete: true
Attachment #8656036 - Flags: review?(bugspam.Callek)
Attachment #8659789 - Flags: review?(bugspam.Callek)
Assignee

Comment 58

4 years ago
Comment on attachment 8659789 [details] [diff] [review]
[custom] buildbotcustom code changes to fix verification step. (in conjunction with patch (*)) [checked-in]

Pushed to buildbotcustom:
https://hg.mozilla.org/build/buildbotcustom/rev/81b1838ceb86
Attachment #8659789 - Attachment description: [custom] buildbotcustom code changes to fix verification step. (in conjunction with patch (*)) → [custom] buildbotcustom code changes to fix verification step. (in conjunction with patch (*)) [checked-in]
Assignee

Comment 59

4 years ago
Comment on attachment 8656332 [details] [diff] [review]
[custom] buildbotcustom code changes to fix final verification step (used with **)

This has been merged with this push:
https://bugzilla.mozilla.org/attachment.cgi?id=8659789
Attachment #8656332 - Attachment is obsolete: true
Attachment #8656332 - Flags: review?(bugspam.Callek)
Assignee

Comment 60

4 years ago
Comment on attachment 8656341 [details] [diff] [review]
[custom] buildbotcustom code changes to fix final verification step (v2)

This has been merged into:
https://bugzilla.mozilla.org/attachment.cgi?id=8659789
Attachment #8656341 - Attachment is obsolete: true
Attachment #8656341 - Flags: review?(bugspam.Callek)
Closing this as resolved, its been for some time! At this point I dont reasonably feel I will be reviewing these patches, either way we are off from CVS \o/
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Flags: needinfo?(bugspam.Callek)
Resolution: --- → FIXED
Comment on attachment 8655778 [details] [diff] [review]
[custom] Use right user for commit during commit_patcher_config [checked-in]

Clearing review since this bug is now done (and has been for some time)
Attachment #8655778 - Flags: review?(bugspam.Callek)
Comment on attachment 8655813 [details] [diff] [review]
[tools] correct path to retry.py command [checked-in]

Clearing review since this bug is now done (and has been for some time)
Attachment #8655813 - Flags: review?(bugspam.Callek)
Comment on attachment 8656043 [details] [diff] [review]
[tools] verify.sh's references to common/ paths needs changing. (*) [checked-in]

Clearing review since this bug is now done (and has been for some time)
Attachment #8656043 - Flags: review?(bugspam.Callek)
Comment on attachment 8656333 [details] [diff] [review]
[tools] fix for final verification step (**) [checked-in]

Clearing review since this bug is now done (and has been for some time)
Attachment #8656333 - Flags: review?(bugspam.Callek)
Comment on attachment 8659120 [details] [diff] [review]
remove cvs, rsh package from mock package list (v1)

Clearing review since this bug is now done (and has been for some time)
Attachment #8659120 - Flags: review?(bugspam.Callek)
Comment on attachment 8659789 [details] [diff] [review]
[custom] buildbotcustom code changes to fix verification step. (in conjunction with patch (*)) [checked-in]

Clearing review since this bug is now done (and has been for some time)
Attachment #8659789 - Flags: review?(bugspam.Callek)
You need to log in before you can comment on or make changes to this bug.