Upgrade Mercurial clients to 3.2.1

RESOLVED FIXED

Status

Release Engineering
Platform Support
P3
normal
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: gps, Assigned: Callek)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/621] )

Attachments

(6 attachments, 4 obsolete attachments)

(Reporter)

Description

3 years ago
AFAICT we are still running Mercurial 2.5(.4?) throughout automation. I recommend we upgrade to 2.7.2 or even 2.8.2+ at our earliest convenience.

Please note this request is limited to just the clients, not hg.mozilla.org.

Mercurial's wire protocol is designed to be forward and backward compatible. 2.8 clients can talk to 2.5 servers (current hg.mozilla.org) just fine. 2.2 clients can talk to 2.8 servers just fine. Now, older clients and servers may miss out on newer features, but things will still talk to each other.

Reasons for upgrading the clients:

* 2.6 introduced parallel working directory updates (OS X and Linux). Instead of 1 I/O thread during `hg up`, you have as many as you do cores. On my machine (with an SSD), `hg up` operations are much faster, especially for large m-c updates. This will likely make initial clones and initial updates significantly faster.

* We encountered weird failures with `hg purge` in bug 851270. I wouldn't be surprised if this bug was fixed in a recent Mercurial release.

* 2.7 fixes an issue with an interrupted `hg up` (http://bz.selenic.com/show_bug.cgi?id=3113). We might be seeing this in automation (it could even be the purge issue).

* Each new version typically contains speed improvements. Every little bit helps.

FWIW, Mercurial is on a monthly release cadence. Major X.Y release every 3 months. Minor X.Y.Z release every month (or as needed if there are major issues discovered). 2.9 is due out around February 1. The test suite is comprehensive and I'd feel comfortable with Mozilla regularly picking up releases as needs warrant them.
Thanks gps for letting us know!
I have added it to the list of "next" projects to take on.
I don't know when that will be but we will let you know as soon as we know.
Component: Buildduty → Platform Support
QA Contact: armenzg → coop
(Reporter)

Comment 2

3 years ago
2.9 is out. Let's not fall too far behind. Bumping this to 2.8+.
Summary: Upgrade Mercurial clients to 2.7+ → Upgrade Mercurial clients to 2.8+

Updated

3 years ago
Depends on: 997971

Comment 3

3 years ago
Part of the Mac platform support bundle: https://releng.etherpad.mozilla.org/platform-support-bundles
Assignee: nobody → hwine
Priority: -- → P3

Comment 4

3 years ago
Didn't mean to assign this to Hal in its entirety.

I'd like Hal to tackle the Mac part (and possibly Linux, because...puppet).

I'm handling Windows myself.
Assignee: hwine → coop

Updated

3 years ago
Assignee: coop → sbruno
(Reporter)

Comment 5

3 years ago
At this point, 3.0.2 is out. We should use that.
Summary: Upgrade Mercurial clients to 2.8+ → Upgrade Mercurial clients to 3.0.2+
Callek I had a look to https://bugzilla.mozilla.org/show_bug.cgi?id=867470#c1, in which you updated hg version for CentOS; how were these packages created? Are instructions in https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/HowTo/Build_RPMs accurate?
Flags: needinfo?(bugspam.Callek)
(Assignee)

Comment 7

3 years ago
Sadly I can't find things that document it well, thats my fault and I apologize. I surely didn't follow the exact puppetagain howto there (because we didn't have that process then) [But it does look like it may be what I did anyway]

I do however think using that process should be the way we do it now. There *is* an srpm in our repo http://puppetagain.pub.build.mozilla.org/data/repos/yum/releng/public/CentOS/6/x86_64/mozilla-python27-mercurial-2.5.4-1.el6.src.rpm
Flags: needinfo?(bugspam.Callek)
(Reporter)

Comment 8

3 years ago
You could download a source archive and run |make install PREFIX=/where/to/install|. After that, you'll have /where/to/install/bin/hg.

The gotcha with the self-install method is that if you don't have the Python development headers and libraries, you won't get the C extension and HG will be slow. You could always install to an empty directory and tarball that for distribution.
I built the packages for CentOS, but I don't understand what the purpose of spec files in puppet repo is (e.g.: http://hg.mozilla.org/build/puppet/file/b4dc56b3106a/modules/packages/manifests/mozilla/py27_mercurial.spec) (they are mentioned in https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/HowTo/Build_RPMs)

For example, I am pretty sure that file has never been used to actually build a package since it refers date "Tue Apr 31 2013" in the change log, which would cause mock to fail (since the date does not exist).

Shy do we store the spec in puppet as well? Since spec files can be extracted from srpm's in our actual repos, isn't this a duplicated information?
Flags: needinfo?(bugspam.Callek)
Maybe the spec file changes are the way we specify in puppet the version of mercurial to be installed? (This would explain why we should keep the spec file in mercurial with the one used to build packages...)
What I actually meant is: "This would explain why we should keep the spec file in the puppet repo in sync with the one actually used to build the package... "
Yes, that's about it -- also, mercurial gives us a nice diff view so we can see what changed in the spec.

I think we built the mercurial packages before we had mock set up, so I guess we got away with a fake date (but that's weird, all the same!)
(Assignee)

Updated

3 years ago
Flags: needinfo?(bugspam.Callek)

Updated

3 years ago
Depends on: 1056981
(Reporter)

Comment 13

3 years ago
As I just mentioned in bug 1056981, I think upgrading to 3.1 today is not prudent. Instead, we should target 3.0.2. Or, if we wait until September 1, we can go with 3.1.1.
Per bug 1056981, let's go with 2.9.2 here as well.

Updated

3 years ago
Summary: Upgrade Mercurial clients to 3.0.2+ → Upgrade Mercurial clients to 2.9.2
simone, how's this going?  As you'll recall, I have a patch that only requires a tweak to the version number and a rebuild, over in bug 895995.  I haven't landed that, as I know this bug is coming up next.
hi dustin, I have been working on other bugs + buildduty this week, I will focus on this in the next few days and provide updates.
Status update:
- srpm and rpm packages have been built for CentOS6 (not uploaded anywhere yet, currently they are just in rpmpackager1.srv.releng.use1.mozilla.com:/home/sbruno/mercurial)
- I am now using py27_mercurial-dmg.sh (after applying patch https://bug895995.bugzilla.mozilla.org/attachment.cgi?id=8483558 and opportunely changing revision numbers) to create the dmg files on bld-lion-r5-062.
Created attachment 8497565 [details]
modified_py27_mercurial-dmg.sh

I used a modified version of py27_mercurial-dmg.sh https://bug895995.bugzilla.mozilla.org/attachment.cgi?id=8483558 to create the pkg and dmg for mac (Lion).

Apart from version numbers, the main difference between dustin's original script and the one I used here is that I used pkgbuild instead of packagemaker (pkgbuild is already installed on lion boxes so there's no need to install any other dependencies).
< packagemaker -r $ROOT -v -i com.mozilla.$pyrealname-$realname -o $pkg -l /
---
> pkgbuild --root $ROOT --identifier com.mozilla.$pyrealname-$realname $pkg
The generated package installs correctly on lion:

[root@bld-lion-r5-062.build.releng.scl3.mozilla.com ~]# ls -d /tools/python27*
/tools/python27			/tools/python27_mercurial
[root@bld-lion-r5-062.build.releng.scl3.mozilla.com ~]# installer -target / -package /Users/cltbld/mercurial_test/build/dmg/python27-mercurial-2.9.2-1.pkg
installer: Package name is python27-mercurial-2.9.2-1
installer: Upgrading at base path /
installer: The upgrade was successful.
[root@bld-lion-r5-062.build.releng.scl3.mozilla.com ~]# ls -d /tools/python27*
/tools/python27			/tools/python27-mercurial
[root@bld-lion-r5-062.build.releng.scl3.mozilla.com ~]# /usr/local/bin/hg --version
Mercurial Distributed SCM (version 2.9.2)
(see http://mercurial.selenic.com for more information)

Copyright (C) 2005-2014 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[root@bld-lion-r5-062.build.releng.scl3.mozilla.com ~]# ls -l /usr/local/bin/hg
lrwxr-xr-x  1 root  wheel  32 Sep 30 07:54 /usr/local/bin/hg -> /tools/python27-mercurial/bin/hg
[root@bld-lion-r5-062.build.releng.scl3.mozilla.com ~]# hg --version
Mercurial Distributed SCM (version 2.9.2)
(see http://mercurial.selenic.com for more information)

Copyright (C) 2005-2014 Matt Mackall and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[root@bld-lion-r5-062.build.releng.scl3.mozilla.com ~]#

This still needs to be tested on the other OSX platforms.
Looks good, but can you post it as a patch (along with the puppet changes)?

Updated

3 years ago
Blocks: 1055298
(Reporter)

Comment 21

3 years ago
This was added as a blocker to headless try but the real requirement for headless try will be Mercurial 3.2, which won't get released until around November 1. I'll leave the dependency here. But we may file a new bug to request 3.2 be deployed. If 3.2 can't be deployed in a timely manner, the fallback will involve installing a client-side extension against 3.1.

While I'm here, Mercurial 3.1.2 fixes the performance regression that required us to only upgrade to 2.9.2 instead of 3.1. If you want to hedge your bets, installing 3.1.2 now may be less overall work.
OK, let's go with the new hotness then.
Summary: Upgrade Mercurial clients to 2.9.2 → Upgrade Mercurial clients to 3.1.2
3.1.2 srpm and rpm packages have been built for CentOS6 (rpmpackager1.srv.releng.use1.mozilla.com:/home/sbruno/mercurial_3.1.2)
3.1.2 pkg and dmg have been created and tested for OSX on a lion box.
Next steps:
- test the dmg on other OSX versions.
- upload in (puppet) patch form the script I used for mac package creation (which is a modified version of https://bug895995.bugzilla.mozilla.org/attachment.cgi?id=8483558)
- add packages and dmg's to proper repositories
- create patches to actually use the newly created mercurial packages while running slave setup with puppet
Just a heads up - some of our tests of our util.hg library (in the tools repo) tests fail when run with hg 3.1.2.
Created attachment 8504713 [details] [diff] [review]
puppet patch for linux 64 - WIP

WIP. Tested in my puppet env after adding the newly created package to the CentOS repo.
Comment on attachment 8504713 [details] [diff] [review]
puppet patch for linux 64 - WIP

What is the change to pyrel for?
Oh, that change is actually not needed. I think it ended up here as a mistaken attempt to change the %{release} setting. pyrel will not be changed in the final patch.
Created attachment 8504874 [details] [diff] [review]
tools_v1 - changes needed when hg 3.1.2 is used

This patch addresses what catlee raised in Comment 25.

tests "testApplyAndPushFail" and "testApplyAndPushRebaseFails" (tools repo) are failing when run with mercurial 3.1.2 (everything is ok with 2.6).

Apparently, with 3.1.2, an explicit "hg rebase --abort" command needs to be issued when a "hg rebase" fails, otherwise the subsequent "hg update -C" command will fail with the message "abort: rebase in progress".

With this patch, all tests run green with both 2.6 and 3.1 hg version.

"tools" tests have been run using tox, after applying the patches mentioned in https://bugzilla.mozilla.org/show_bug.cgi?id=1073553#c1 and https://bugzilla.mozilla.org/show_bug.cgi?id=1073553#c2; different mercurial versions have been tested running "tox -e py27-hg2.6" and "tox -e py27-hg3.1".
Attachment #8504874 - Flags: review?(catlee)
Comment on attachment 8504874 [details] [diff] [review]
tools_v1 - changes needed when hg 3.1.2 is used

Review of attachment 8504874 [details] [diff] [review]:
-----------------------------------------------------------------

nicely done!
Attachment #8504874 - Flags: review?(catlee) → review+
I tested the dmg's on all OSX flavours and built deb packages for Ubuntu as well (thanks rail for your detailed instructions!).

I created the deb packages on ubuntu64packager1.srv.releng.use1.mozilla.com with the followings commands:

# create pbuilder image for 64 bit, see https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/HowTo/Build_DEBs#Create_pbuilder_images
sudo pbuilder --create --distribution precise --architecture amd64 \
    --components "main restricted universe multiverse" \
    --debootstrapopts --variant=buildd \
    --basetgz /var/cache/pbuilder/base-precise-amd64.tgz \
    --mirror http://archive.ubuntu.com/ubuntu \
    --othermirror "deb http://archive.ubuntu.com/ubuntu precise-updates main restricted universe multiverse" \
    --othermirror "deb http://archive.ubuntu.com/ubuntu precise-security main restricted universe multiverse"
mkdir /tmp/hg
cd /tmp/hg
# get the latest package from Ubuntu
dget http://archive.ubuntu.com/ubuntu/pool/universe/m/mercurial/mercurial_3.1.1-1.dsc
# get the latest mercurial
wget -O mercurial_3.1.2.orig.tar.gz http://mercurial.selenic.com/release/mercurial-3.1.2.tar.gz
# unpack it
tar xf mercurial_3.1.2.orig.tar.gz
# apply the packaging changes from the previous version
cd mercurial-3.1.2 && tar xf ../mercurial_3.1.1-1.debian.tar.xz
# let dch guess the current version from the directory name (-d)
DEBFULLNAME="Simone Bruno" DEBEMAIL="sbruno@mozilla.com" dch -d --distribution precise "New upstream version"
# make sure to mark the version mozilla-specific
DEBFULLNAME="Simone Bruno" DEBEMAIL="sbruno@mozilla.com" dch -l mozilla  --distribution precise "Packaged for mozilla"
# create source package
dpkg-source -b .
cd ../
# build using mercurial_3.1.2-1mozilla1.dsc, including the source (-sa)
sudo pbuilder --build --distribution precise --architecture amd64 \
--basetgz /var/cache/pbuilder/base-precise-amd64.tgz --buildresult amd64 --debbuildopts "-sa" mercurial_3.1.2-1mozilla1.dsc
# create pbuilder image for 32 bit, see https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/HowTo/Build_DEBs#Create_pbuilder_images
sudo pbuilder --create --distribution precise --architecture i386 \
    --components "main restricted universe multiverse" \
    --debootstrapopts --variant=buildd \
    --basetgz /var/cache/pbuilder/base-precise-i386.tgz \
    --mirror http://archive.ubuntu.com/ubuntu \
    --othermirror "deb http://archive.ubuntu.com/ubuntu precise-updates main restricted universe multiverse" \
    --othermirror "deb http://archive.ubuntu.com/ubuntu precise-security main restricted universe multiverse"
# build for i386, no source required
sudo pbuilder --build --distribution precise --architecture i386 --binary-arch \
--basetgz /var/cache/pbuilder/base-precise-i386.tgz --buildresult i386 mercurial_3.1.2-1mozilla1.dsc
# amd64 and i386 directories contain the packages now, including mercurial-common_3.1.2-1_all.deb

Since a brand new version of the script needed to create the dmg's for OSX has just been created by Dustin in Bug 1081313, what's missing here is just to copy packages to the appropriate repos and upload the puppet patches to roll out them.
Comment on attachment 8504874 [details] [diff] [review]
tools_v1 - changes needed when hg 3.1.2 is used

tools patch checked in: https://hg.mozilla.org/build/tools/rev/5e1839d1441c
Attachment #8504874 - Flags: checked-in+
(In reply to Simone Bruno [:simone] from comment #31)
> I tested the dmg's on all OSX flavours and built deb packages for Ubuntu as
> well (thanks rail for your detailed instructions!).

Is this something we should add to the wiki? Are wiki instructions good enough if someone else comes along and needs to create a DMG now?

Thanks,
Pete
Flags: needinfo?(sbruno)
Good point pete!
Docs exist alread to create DMG's, but in order prepare the DMG's for Mercurial I also needed to do some bug archeology and draw from the almost infinite knowledge of Dustin about everything.
I will update the docs and add a section with a "use case" with all I did for this specific case.
Regarding the deb, I put here the full sequence of commands I used, and I added a link to them in the wiki page https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/HowTo/Build_DEBs (section "Other approaches")
Flags: needinfo?(sbruno)
Created attachment 8507884 [details]
Packages for all OS's temporary stored in tooltool
Packages have been uploaded to the relevant repos:

python27-mercurial-3.1.2-1.dmg
-> releng-puppet2.srv.releng.scl3.mozilla.com:/data/repos/DMGs
http://puppetagain.pub.build.mozilla.org/data/repos/DMGs/

mozilla-python27-mercurial-3.1.2-1.el6.i686.rpm
mozilla-python27-mercurial-3.1.2-1.el6.src.rpm
-> releng-puppet2.srv.releng.scl3.mozilla.com:/data/repos/yum/releng/public/CentOS/6/i386
http://puppetagain.pub.build.mozilla.org/data/repos/yum/releng/public/CentOS/6/i386/

mozilla-python27-mercurial-3.1.2-1.el6.src.rpm
mozilla-python27-mercurial-3.1.2-1.el6.x86_64.rpm
mozilla-python27-mercurial-debuginfo-3.1.2-1.el6.x86_64.rpm
-> releng-puppet2.srv.releng.scl3.mozilla.com:/data/repos/yum/releng/public/CentOS/6/x86_64
http://puppetagain.pub.build.mozilla.org/data/repos/yum/releng/public/CentOS/6/x86_64/

mercurial-common_3.1.2-1mozilla1_all.deb
mercurial_3.1.2-1mozilla1_amd64.deb
mercurial_3.1.2-1mozilla1_i386.deb
-> releng-puppet2.srv.releng.scl3.mozilla.com:/data/repos/apt/releng/pool/main/m/mercurial
http://puppetagain.pub.build.mozilla.org/data/repos/apt/releng/precise/pool/main/m/mercurial/

After scp'ing the files, I have run in each of the mentioned folders on releng-puppet2.srv.releng.scl3.mozilla.com the following two commands:
$puppetmaster-fixperms # to make sure permissions are correctly set
$createrepo --update . # to update the repo metadata

See also https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Packages#CentOS:_Adding_New_Packages
DMGs need to be uploaded to the following folders since the packages repo path used by puppet is os-version dependent:

python27-mercurial-3.1.2-1.dmg
-> releng-puppet2.srv.releng.scl3.mozilla.com:/data/repos/DMGs/10.6
-> releng-puppet2.srv.releng.scl3.mozilla.com:/data/repos/DMGs/10.7
-> releng-puppet2.srv.releng.scl3.mozilla.com:/data/repos/DMGs/10.8
http://puppetagain.pub.build.mozilla.org/data/repos/DMGs/10.6
http://puppetagain.pub.build.mozilla.org/data/repos/DMGs/10.7
http://puppetagain.pub.build.mozilla.org/data/repos/DMGs/10.8

After copying the file, I run "puppetmaster-fixperms" in each of the mentioned folders.

Finally, I run "puppetmaster-sync" on releng-puppet1.srv.releng.scl3.mozilla.com to sync with releng-puppet1
Created attachment 8510330 [details]
update apt index for ubuntu packages

Hi Dustin,
I am going to use this script to re-generate the Packages files for Ubuntu packages in releng repo, since the one in https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Packages#Ubuntu:_Adding_New_Packages seems to be using the wrong folder structure.
The commands to backup the existing ones have been commented out because I already run that step.

I am not sure of the dpkg-scanpackages command, since it does not take into account the $dist variable, so it will generate identical Packages files for i386 ad amd64...
Attachment #8510330 - Flags: review?(dustin)
Errata corrige: "so it will generate identical Packages files for trusty and precise"
Comment on attachment 8510330 [details]
update apt index for ubuntu packages

I spoke to Dustin and we decided to move the new mercurial packages to a new custom repo insteading of touching the deprecated releng one.
Attachment #8510330 - Attachment is obsolete: true
Attachment #8510330 - Flags: review?(dustin)
deb packages for mercurial 3.1.2 have been moved to http://puppetagain-apt.pvt.build.mozilla.org/repos/apt/custom/mercurial/

I also fixed the permissions and update the apt-indexes by creating the Packages and Packages.bz2 files following the procedure described in https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Packages#Ubuntu:_Adding_New_Packages.
Created attachment 8511922 [details] [diff] [review]
mercurial 3.1.2 wip

Finally, I was able to test the mercurial upgrade on all platforms using boxes pinned to my puppet environment.
This is the patch I used in my puppet environment.
I need to clean it removing references to pinned boxes and choosing correct values for repoflags to make sure that apt-update commands are correctly issued when the patch is rolled out to prod.

Asking Dustin for a quick look before I upload the actual patch: can you spot anything macroscopically incorrect or missing here?
Attachment #8511922 - Flags: feedback?(dustin)
Attachment #8511922 - Attachment is patch: true
Comment on attachment 8511922 [details] [diff] [review]
mercurial 3.1.2 wip

Review of attachment 8511922 [details] [diff] [review]:
-----------------------------------------------------------------

::: modules/packages/manifests/mercurial.pp
@@ +14,3 @@
>              package {
>                  ["mercurial", "mercurial-common"]:
> +                    ensure => "3.1.2-1mozilla1";

I'm not convinced we want to upgrade packages::mercurial too -- that's used for aws_manager, puppetmasters, and buildmasters, none of which run on Ubuntu.  The idea for packages::mercurial is to just install the mercurial that ships with the OS, rather than the latest-and-greatest-for-builds version.

::: modules/packages/manifests/mozilla/py27_mercurial.pp
@@ +21,5 @@
>                      require => Class['packages::mozilla::python27'];
>              } -> Anchor['packages::mozilla::py27_mercurial::end']
>          }
>          Ubuntu: {
>              include packages::mercurial

pacakges::mozilla::py27_mercurial used to just symlink to the mercurial installed by packages::mercurial, but it doesn't anymore -- so you don't need this include anymore.

::: modules/packages/manifests/mozilla/py27_mercurial.spec
@@ +3,5 @@
>  %define realname mercurial
>  # This is the 'real' name of the python to build with e.g. python26
>  %define pyrealname python27
>  %define pyver 2.7
> +%define pyrel 1

This is specifying Python-2.7.1, which isn't what we have installed.  But it's not actually used, so it doesn't really matter.  Just delete this line?
Attachment #8511922 - Flags: feedback?(dustin) → feedback+
(In reply to Dustin J. Mitchell [:dustin] from comment #43)
> Comment on attachment 8511922 [details] [diff] [review]
> mercurial 3.1.2 wip
> 
> Review of attachment 8511922 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: modules/packages/manifests/mercurial.pp
> @@ +14,3 @@
> >              package {
> >                  ["mercurial", "mercurial-common"]:
> > +                    ensure => "3.1.2-1mozilla1";
> 
> I'm not convinced we want to upgrade packages::mercurial too -- that's used
> for aws_manager, puppetmasters, and buildmasters, none of which run on
> Ubuntu.  The idea for packages::mercurial is to just install the mercurial
> that ships with the OS, rather than the latest-and-greatest-for-builds
> version.

Mmh... Without the two lines I changed in modules/packages/manifests/mercurial.pp, puppet is not installing 3.1.2 on tst-linux64-ec2-simone.test.releng.use1.mozilla.com (the box I am using for aws puppet test) so I thought this was the right place to change the mercurial version for aws slaves... if not here, where?

> 
> ::: modules/packages/manifests/mozilla/py27_mercurial.pp
> @@ +21,5 @@
> >                      require => Class['packages::mozilla::python27'];
> >              } -> Anchor['packages::mozilla::py27_mercurial::end']
> >          }
> >          Ubuntu: {
> >              include packages::mercurial
> 
> pacakges::mozilla::py27_mercurial used to just symlink to the mercurial
> installed by packages::mercurial, but it doesn't anymore -- so you don't
> need this include anymore.

This line is not part of my patch, should I just delete it?

> ::: modules/packages/manifests/mozilla/py27_mercurial.spec
> @@ +3,5 @@
> >  %define realname mercurial
> >  # This is the 'real' name of the python to build with e.g. python26
> >  %define pyrealname python27
> >  %define pyver 2.7
> > +%define pyrel 1
> 
> This is specifying Python-2.7.1, which isn't what we have installed.  But
> it's not actually used, so it doesn't really matter.  Just delete this line?

Right! I'll delete this line.
The testers use packages::mozilla::py27_mercurial, which installs hg under /tools.

Yes, just delete the line.
Created attachment 8512534 [details] [diff] [review]
mercurial_3.1.2 rollout v1

Here is the patch!
Dustin, please note that I did *not* remove the creation of the symlink ["/tools/python27-mercurial/bin/hg" -> "/usr/bin/hg"] on Ubuntu boxes, since the newly created deb file does not create anything under /tools (and we may be wanting to keep/python27-mercurial/bin/hg since it is probably referenced somewhere).
Attachment #8511922 - Attachment is obsolete: true
Attachment #8512534 - Flags: review?(dustin)
I think there are still bits missing here, since I don't see any py27_mercurial-debian directory being added in this patch.  Let me see if I can lay out the story.

In general, packages::mozilla::py27_mercurial installs /tools/python27-mercurial/bin/hg, with a symlink from /usr/local/bin, which is where build-system scripts expect to find it.  That copy of hg uses the Mozilla-compiled Python-2.7 in /tools/python27/bin/python2.7.

For Ubuntu, we took advantage of the fact that the version we wanted happened to be the version that Ubuntu packaged, so we cheated a little.  In order to install packages::mozilla::py27_mercurial, we installed packages::mercurial (the upstream Ubuntu package) and then symlinked /tools/python27-mercurial/bin/hg to /usr/bin/hg.

Now we want a different version, so we must undo that cheat.  So packages::mozilla::py27_mercurial needs a *new* debian package created, with its debian directory included in the patch.  That package should contain a mercurial install to /tools/python27-mercurial/bin/hg, using /tools/python27/bin/python2.7.  It should also contain a symlink from /usr/local/bin.

For CentOS, we had to write a .spec for 2.5.4 and as you see upgrading that to 3.1.2 is pretty straightforward.  I think this part of the patch is correct.

For Darwin, recall bug 895995 changes /tools/python27_mercurial to /tools/python27-mercurial (`_` -> `-`) to match the other OS's, and adds an updated py27_mercurial-dmg.sh.  All of that needs to be rolled into the patch on this bug, as well.

Hopefully that clarifies what we're trying to do here.  There's still some work to do for Ubuntu and Darwin.

Updated

3 years ago
Whiteboard: [kanban:engops:https://kanbanize.com/ctrl_board/6/621]

Updated

3 years ago
Whiteboard: [kanban:engops:https://kanbanize.com/ctrl_board/6/621] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1106] [kanban:engops:https://kanbanize.com/ctrl_board/6/621]

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1106] [kanban:engops:https://kanbanize.com/ctrl_board/6/621] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1116] [kanban:engops:https://kanbanize.com/ctrl_board/6/621]

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1116] [kanban:engops:https://kanbanize.com/ctrl_board/6/621] [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1117] [kanban:engops:https://kanbanize.com/ctrl_board/6/621] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1117] [kanban:engops:https://kanbanize.com/ctrl_board/6/621] [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1118] [kanban:engops:https://kanbanize.com/ctrl_board/6/621]
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/1118] [kanban:engops:https://kanbanize.com/ctrl_board/6/621] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/621] [kanban:engops:https://kanbanize.com/ctrl_board/6/621]
simone, do you think you'll get this wrapped up in the next few weeks?
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/621] [kanban:engops:https://kanbanize.com/ctrl_board/6/621] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/621]
Attachment #8512534 - Flags: review?(dustin) → review-
I'm going to tentatively re-assign this to Callek, with the understanding that he needs to finish up bug 927199 first.
Assignee: sbruno → bugspam.Callek
(Reporter)

Comment 50

3 years ago
If you need my Mercurial domain expertise here, I'd be glad to provide assistance. What I don't know is how packaging works on your machines.
(Reporter)

Comment 51

3 years ago
It is looking like the new Try server may require clients that `hg pull` from it to use Mercurial 3.2.1. So, bumping summary to 3.2.1.

FWIW, it's looking like we will either require 3.2.1 or the existence of a client-side extension. I'd prefer a vanilla 3.2.1 rather than a custom client-side extension. But feel free to push back of the custom extension route is preferred.
Summary: Upgrade Mercurial clients to 3.1.2 → Upgrade Mercurial clients to 3.2.1
(In reply to Gregory Szorc [:gps] from comment #51)
> It is looking like the new Try server may require clients that `hg pull`
> from it to use Mercurial 3.2.1. So, bumping summary to 3.2.1.
> 
> FWIW, it's looking like we will either require 3.2.1 or the existence of a
> client-side extension. I'd prefer a vanilla 3.2.1 rather than a custom
> client-side extension. But feel free to push back of the custom extension
> route is preferred.

I would also prefer the vanilla install.

I've made it clear to Callek that the the key part of this bug is getting us to a state where we can deploy future updates to hg easily, i.e. < 1 week. The delay here would be comical if it weren't holding us back so much.
Agreed, and fwiw most of the delay here has been to support doing future updates quickly and across all platforms.  I think we're within a week of shipping this, assuming Callek can spare the time.
(Assignee)

Comment 54

3 years ago
Created attachment 8528135 [details]
mozilla-python27-mercurial-3.2.1.deb.tar.gz

Ok, this is the debian/ files for this attempt. I'm looking for *any* feedback/comments before I upload this and proceed.

Based on the debian from http://puppetagain.pub.build.mozilla.org/data/repos/apt/releng/pool/main/m/mercurial/mercurial_2.5.4-0mozilla1.debian.tar.gz

Following roughly https://wiki.debian.org/IntroDebianPackaging (because I'm creating a new package, not updating an existing one, - this should work side-by-side with the systemwide stuff)

The list of installed files from a built version of this is at http://callek.pastebin.mozilla.org/7528137 (pastebin saved for 1 month)
Attachment #8528135 - Flags: review?(dustin)
Attachment #8528135 - Flags: feedback?(rail)
Attachment #8528135 - Flags: feedback?(mh+mozilla)
The changelog's signoff should probably be a person, not "Created by Puppet"

It should depend on mozilla-python27, not python27, I think?  That may require some changes to `rules` to use that Python for the setup.py invocations.

Looks good otherwise!
Attachment #8528135 - Flags: review?(dustin) → feedback+
(Assignee)

Comment 56

3 years ago
(In reply to Dustin J. Mitchell [:dustin] from comment #55)
> It should depend on mozilla-python27, not python27, I think?  That may
> require some changes to `rules` to use that Python for the setup.py
> invocations.

AFAICT not true,

[root@ubuntu64packager1.srv.releng.use1.mozilla.com ~]# apt-get install mozilla-python27
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package mozilla-python27

Puppet creates a symlink for us though. I'm not about to make an apt package depend on file layout from puppet though.
Comment on attachment 8528135 [details]
mozilla-python27-mercurial-3.2.1.deb.tar.gz

The packaging files LGTM, however :)

I'd prefer to have a single mercurial package installed:
* less packages to maintain
* a one liner in puppet to upgrade/downgrade
* mercurial is not like python, which used by OS, so it's hard to upgrade it globally

Is there any particular reason why we want 2 Mercurial packages installed in parallel?

Not sure whether it's f+ or f-, probably f0 :)
Attachment #8528135 - Flags: feedback?(rail)
Rail: we want to have (a) exactly the same version as on CentOS, Darwin, etc., and (b) exactly the same path across POSIXes anyway.

Regarding dependency: that's odd, since `control` lists
  Depends: python27
I'm not sure what you mean about puppet making a symlink.  You can't symlink packages..

At any rate, I'm pretty confident that it doesn't matter which built version of Python Mercurial runs in, so I'll drop that part of my objection.
(Reporter)

Comment 59

3 years ago
bkero has produced an rpm for use on hg.mozilla.org. He could probably weigh in here.

Also, the contrib/ directory in the Mercurial repository has some scripts for building rpms.

I agree with Rail that having 2 Mercurials installed is kind of funky. Mercurial has strong backwards compatibility guarantees. There should be little to no problem replacing the system install.
(Assignee)

Comment 60

3 years ago
Created attachment 8528548 [details] [diff] [review]
[puppet] v1 - untested

f? to glandium for the deb files (*-debian/**)

This is *untested* so far, in any capacity, has not even run in a puppet --noop mode yet.

I also need to test this against the small set of hg client compliance stuff we have.

I followed the puppetAgain docs for all of these, and future upgrades should be easy (I'll flesh out the docs before I close this bug)
Attachment #8528135 - Attachment is obsolete: true
Attachment #8528135 - Flags: feedback?(mh+mozilla)
Attachment #8528548 - Flags: review?(dustin)
Attachment #8528548 - Flags: feedback?(mh+mozilla)
Comment on attachment 8528548 [details] [diff] [review]
[puppet] v1 - untested

Review of attachment 8528548 [details] [diff] [review]:
-----------------------------------------------------------------

::: modules/packages/manifests/mozilla/py27_mercurial.pp
@@ +38,5 @@
>                      version => "2.5.4-2";
> +            } ->
> +            file {
> +                # This link can go away in a few weeks after verification
> +                # that nothing is using it.

How are we going to verify that?
Attachment #8528548 - Flags: review?(dustin) → review+
(Assignee)

Comment 62

3 years ago
Just deployed the files to:

* releng/ repo for YUM
* a new custom mozilla-mercurial/ repo for APT
* OS-specific DMG directories.
(Assignee)

Comment 63

3 years ago
Created attachment 8528818 [details] [diff] [review]
[puppet] v1.5 - partially tested

porting forward r+, has some slight modifications from previous patch.

Tested:
* on b-linux64
 \- old and new mercurial version tools repo tests pass (for hg util)
  - new package installs
  - puppet can apply
* no other testing yet.


:coop, :pmoore, I'm off the rest of this week, and then will be with you both in portland....

That said, I setup my user env on releng-puppet2.srv.releng.scl3.mozilla.com and setup all hosts that use it to be pinned to my user env. And I *think* this patch is ready for a deploy.

You can do a subset of hosts manually, pointing at my user env, or land this patch outright and merge to prod. *or* you can wait for me to arrive back from portland, at your discretion.

A reversion is as simple as a backout (or a reimage if merely pinned to my user env)

I'm going to copy you both on this email to let you decide if you want to deploy in my absense
Attachment #8528548 - Attachment is obsolete: true
Attachment #8528548 - Flags: feedback?(mh+mozilla)
Attachment #8528818 - Flags: review+
Nice work Callek! :)
(In reply to Justin Wood (:Callek) from comment #63) 
> :coop, :pmoore, I'm off the rest of this week, and then will be with you
> both in portland....
> 
> That said, I setup my user env on releng-puppet2.srv.releng.scl3.mozilla.com
> and setup all hosts that use it to be pinned to my user env. And I *think*
> this patch is ready for a deploy.
> 
> You can do a subset of hosts manually, pointing at my user env, or land this
> patch outright and merge to prod. *or* you can wait for me to arrive back
> from portland, at your discretion.
> 
> A reversion is as simple as a backout (or a reimage if merely pinned to my
> user env)
> 
> I'm going to copy you both on this email to let you decide if you want to
> deploy in my absense

Didn't get to this today due to bug 1105826. Maybe Pete can pick it up when he gets in tomorrow, otherwise I'll try to deploy in the morning ET.
(Assignee)

Comment 66

3 years ago
https://hg.mozilla.org/build/puppet/rev/b0c44024ab17
https://hg.mozilla.org/build/puppet/rev/486c0af97eca
Comment on attachment 8528818 [details] [diff] [review]
[puppet] v1.5 - partially tested

backed out:

Thu Dec 04 10:48:20 -0800 2014 /Stage[main]/Packages::Mozilla::Py27_mercurial/File[/tools/python27_mercurial]/ensure (err): change from directory to link failed: Could not remove existing file

http://hg.mozilla.org/build/puppet/rev/25768760c9b4
Attachment #8528818 - Flags: checked-in-
Blocks: 994395
(Assignee)

Comment 68

3 years ago
relanded

https://hg.mozilla.org/build/puppet/rev/6e9fafb754bd
https://hg.mozilla.org/build/puppet/rev/1e187a938951
(Assignee)

Comment 69

3 years ago
(In reply to Justin Wood (:Callek) from comment #68)
> relanded
> 
> https://hg.mozilla.org/build/puppet/rev/6e9fafb754bd
> https://hg.mozilla.org/build/puppet/rev/1e187a938951

(to be clear, with some minor tweaks to the prior patch)
(Assignee)

Comment 70

3 years ago
Had to do a bustage fix for APT

https://hg.mozilla.org/build/puppet/rev/2bd4bea820fa
https://hg.mozilla.org/build/puppet/rev/2aa162f6e196
(Assignee)

Comment 71

3 years ago
....and finally another backout

https://hg.mozilla.org/build/puppet/rev/8f09a3e7d126
https://hg.mozilla.org/build/puppet/rev/0659500a5460

The package is depending on "python27" while the deb is "python2.7" instead :/

Mon Dec 08 15:29:43 -0800 2014 Puppet (err): Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install mozilla-python27-mercurial=3.2.1' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 mozilla-python27-mercurial : Depends: python27 but it is not installable
E: Unable to correct problems, you have held broken packages.
Wrapped exception:
Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install mozilla-python27-mercurial=3.2.1' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 mozilla-python27-mercurial : Depends: python27 but it is not installable
E: Unable to correct problems, you have held broken packages.
Mon Dec 08 15:29:43 -0800 2014 /Stage[main]/Packages::Mozilla::Py27_mercurial/Package[mozilla-python27-mercurial]/ensure (err): change from purged to 3.2.1 failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install mozilla-python27-mercurial=3.2.1' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 mozilla-python27-mercurial : Depends: python27 but it is not installable
E: Unable to correct problems, you have held broken packages.
(Assignee)

Comment 72

3 years ago
relanded, after rebuilding the deb package:

https://hg.mozilla.org/build/puppet/rev/05e2b6b69948
https://hg.mozilla.org/build/puppet/rev/d805e4f51aeb
(Assignee)

Comment 73

3 years ago
fyi, I broke mac-v2-signing puppetizing, (since I didn't build any 10.9 based package)

I copied the 10.8 built one, and all-was-well (10.10's failed).

After I got the 10.8 one in place I ran, as root:

rm -f /var/db/.puppet_pkgdmg_installed_python27-mercurial-3.2*
puppet agent --test

on all v2 signing servers, for sanity.

Manually checked that they could clone our tools repo.
(Assignee)

Updated

3 years ago
Depends on: 1110304
(Assignee)

Comment 74

2 years ago
Doc created at:

https://wiki.mozilla.org/ReleaseEngineering/How_To/Update_Mercurial

n-i to dustin and coop to see if either of you feel there is something missing in that doc.

If not, this bug is free to close. We'll test that doc sometime in january with someone else running an upgrade.
Flags: needinfo?(dustin)
Flags: needinfo?(coop)
Looks good, with one caveat. In places like the 10.8 (probably easiest to search for 'jwood' in your doc to see everywhere it's used) section:

https://wiki.mozilla.org/ReleaseEngineering/How_To/Update_Mercurial#10.8_.28Mountain_Lion.29

... can you please put the hostname and username behind a var, e.g.

HOST=talos-mtnlion-r5-001
USER=jwood
ssh root@${HOST}
puppet agent --server foo --test --environment ${USER}

This makes it easier for me when cutting-and-pasting multiple command sets, which (let's face it) is what we end up doing until we automate more fully. You do this elsewhere, so I just want it consistent.

I also prefer explicitly setting the USER like this rather than relying on `whoami`, because my user name is coop in some places and ccooper in others (and cltbld gets used in still others).
Flags: needinfo?(coop)
(Assignee)

Comment 76

2 years ago
(In reply to Chris Cooper [:coop] from comment #75)
> Looks good, with one caveat. In places like the 10.8 (probably easiest to
> search for 'jwood' in your doc to see everywhere it's used) section:
> 
> https://wiki.mozilla.org/ReleaseEngineering/How_To/Update_Mercurial#10.8_.
> 28Mountain_Lion.29
> 
> ... can you please put the hostname and username behind a var, e.g.
> ...
> I also prefer explicitly setting the USER like this rather than relying on
> `whoami`, because my user name is coop in some places and ccooper in others
> (and cltbld gets used in still others).

Done this all:
https://wiki.mozilla.org/index.php?title=ReleaseEngineering%2FHow_To%2FUpdate_Mercurial&diff=1044330&oldid=1044175
(Assignee)

Comment 77

2 years ago
[12:35:31]	dustin	I had a look - it looked oK to me
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Flags: needinfo?(dustin)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.