520562 - Linux build slaves need to install libhildonfm2-dev

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Mark Finkle (:mfinkle) (use needinfo?)]

Bug 511290 will require an update to the Linux build slaves used to build Maemo Fennec builds:

fakeroot apt-get install libhildonfm2-dev

Comment 1

[[e-mail] (:bhearsum)]

(In reply to comment #0)
> Bug 511290 will require an update to the Linux build slaves used to build Maemo
> Fennec builds:
> 
> fakeroot apt-get install libhildonfm2-dev

This needs to be run inside of scratchbox, for the record.

Comment 2

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

mfinkle: beta4 is going out this week - when is this library needed by? 

bhearsum: is this something we could do using puppet?

Comment 3

[[e-mail] (:bhearsum)]

(In reply to comment #2)
> bhearsum: is this something we could do using puppet?

Yes

Comment 4

[Armen [:armenzg]]

(In reply to comment #2)
> mfinkle: beta4 is going out this week - when is this library needed by? 
> 
beta4 was built last week (and is out at this time) and the bug that needs it has not yet landed (maybe because this has not yet been deployed?)
> bhearsum: is this something we could do using puppet?
bhearsum ask cshields to help me with this while he is away. This is a little different that when I did it with Ben but he gave me instructions on how to do it.

I will try this on staging and comment as I progress

Comment 5

[Armen [:armenzg]]

Attachment: [WIP] [puppet-manifests] deploy libhildonfm2-dev on staging slaves

I have been fighting with many bugs today so I wasn't able to finish this for staging.

cshields has been helping me to get started. The problem that I run to is that "include scratchbox" in http://hg.mozilla.org/build/puppet-manifests/file/tip/classes/staging-buildslave.pp#l12 is commented out (http://hg.mozilla.org/build/puppet-manifests/rev/4ef503445a54) since the checks were not working properly at that time.

If I ask to comment out that package I will have to test that nothing else declared on strachbox.pp can affect one of our production slaves. I will need Ben's input for this.

As far as the patch goes it checks that ./users/cltbld/targets/CHINOOK-ARMEL-2007/var/lib/dpkg/info/libhildonfm2-dev.list exists and if it does not exist it runs the following command:
 /scratchbox/moz_scratchbox -p apt-get --yes install libhildonfm2-dev

I will continue this tomorrow.

NOTE: To check the existence of the package in a slave, login unto scratchbox and type this: "pkg-config --libs hildon-fm-2"

Comment 6

[Armen [:armenzg]]

Attachment: [WIP] [puppet-manifests] deploy libhildonfm2-dev on staging slaves (v2)

I am not using the scratchbox.pp package but instead creating one called "extras.pp" which is included by "staging-buildslave.pp" class. This comes after a conversation with bhearsum.

I am having trouble when the command "/scratchbox/moz_scratchbox -p apt-get --yes --force-yes install libhildonfm2-dev" is run by puppet.

When I run this as root on the slave I get this:

err: //Node[moz2-linux-slave03.build.mozilla.org]/staging-buildslave/extras/Exec[/scratchbox/moz_scratchbox -p apt-get --yes --force-yes install libhildonfm2-dev]/returns: change from notrun to 0 failed: /scratchbox/moz_scratchbox -p apt-get --yes --force-yes install libhildonfm2-dev returned 1 instead of 0 at /etc/puppet/manifests/packages/extras.pp:11

What am I doing wrong? Is puppet running the commands as cltbld?

Comment 7

[Armen [:armenzg]]

Attachment: [puppet-manifests] deploy libhildonfm2-dev on staging slaves (v3)

This patch has worked for linux slaves 3 and 4 on staging.

puppet runs as root and scratchbox does not permit that. That is why I use "su - cltbld -c <command>"

I need to declare the path variable otherwise we get this message:
"err: Could not create su - cltbld -c '/scratchbox/moz_scratchbox -p apt-get --yes --force-yes install libhildonfm2-dev': 'su - cltbld -c '/scratchbox/moz_scratchbox -p apt-get --yes --force-yes install libhildonfm2-dev'' is both unqualifed and specified no search path at /etc/puppet/manifests/packages/extras.pp:11"

This patch is for production and for staging slaves

Comment 8

[Chris Cooper [:coop] (he/him)]

(In reply to comment #7)
> I need to declare the path variable otherwise we get this message:
> "err: Could not create su - cltbld -c '/scratchbox/moz_scratchbox -p apt-get
> --yes --force-yes install libhildonfm2-dev': 'su - cltbld -c
> '/scratchbox/moz_scratchbox -p apt-get --yes --force-yes install
> libhildonfm2-dev'' is both unqualifed and specified no search path at
> /etc/puppet/manifests/packages/extras.pp:11"

What's the path for here exactly? apt-get?

Can we can apt-get via the full path (bin/apt-get or whatever) to avoid the path call if no other puppet packages are using it?

Comment 9

[Corey Shields [:cshields]]

You can either declare a path or use absolute paths in your commands.  Either way works.

Comment 10

[Chris Cooper [:coop] (he/him)]

Comment on attachment 405338 [details] [diff] [review]
[puppet-manifests] deploy libhildonfm2-dev on staging slaves (v3)

http://hg.mozilla.org/build/puppet-manifests/rev/545a4a1bb876

Comment 11

[Chris Cooper [:coop] (he/him)]

Is it possible to specify an exact version of the libhildonfm2-dev package to install via apt-get? 

I don't know how much churn there is in that package, but in the interest of having all our slaves in the same, known state, it would be good to: 

a) start specifying a version for the package; and,
b) download that package version and store it in our binaries repo so we can have a copy of it locally if it's ever not available via apt-get.

Comment 12

[Armen [:armenzg]]

Cool! The production slaves can pick up the change.
I am not sure how many hours (maybe minutes?) it will take for the slaves to pick up the change but all I have to do is check few slaves in an hour to have an idea.

[root@moz2-linux-slave10 ~]# puppetd --test --server production-puppet.build.mozilla.org
info: mount[localhost]: Mounted /
info: mount[modules]: Mounted 
info: mount[plugins]: Mounted 
info: Caching catalog at /var/lib/puppet/localconfig.yaml
notice: Starting catalog run
notice: //Node[moz2-linux-slave10.build.mozilla.org]/base/centos5/Service[auditd]/ensure: ensure changed 'running' to 'stopped'
notice: //Node[moz2-linux-slave10.build.mozilla.org]/buildslave/extras/Exec[su - cltbld -c '/scratchbox/moz_scratchbox -p apt-get --yes --force-yes install libhildonfm2-dev']/returns: executed successfully
notice: //Node[moz2-linux-slave10.build.mozilla.org]/base/centos5/Service[acpid]/ensure: ensure changed 'running' to 'stopped'
notice: //Node[moz2-linux-slave10.build.mozilla.org]/base/centos5/Service[atd]/ensure: ensure changed 'running' to 'stopped'
notice: //Node[moz2-linux-slave10.build.mozilla.org]/cltbld/Exec[/usr/bin/crontab -u cltbld /home/cltbld/crontab]/returns: executed successfully
notice: Finished catalog run in 23.69 seconds

Comment 13

[Armen [:armenzg]]

I have checked that all slaves (production and try) have lib-hildon-fm-2 installed.

Mark you are safe to land. You can use the try server if you want to try it before landing it.

I will resolve this FIXED when I know that the patch has landed properly.

Comment 14

[Armen [:armenzg]]

For reference the lib is installed inside of scratchbox but not outside:

[cltbld@moz2-linux-slave17 ~]$ pkg-config --libs hildon-fm-2
Package hildon-fm-2 was not found in the pkg-config search path.
Perhaps you should add the directory containing `hildon-fm-2.pc'
to the PKG_CONFIG_PATH environment variable
No package 'hildon-fm-2' found
[cltbld@moz2-linux-slave17 ~]$ /scratchbox/login 

Welcome to Scratchbox, the cross-compilation toolkit!

Use 'sb-menu' to change your compilation target.
See /scratchbox/doc/ for documentation.

[sbox-CHINOOK-ARMEL-2007: ~] > pkg-config --libs hildon-fm-2
-lhildonfm -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgdk_pixbuf-2.0 -lm -lpangocairo-1.0 -lpango-1.0 -lcairo -lgobject-2.0 -lgmodule-2.0 -ldl -lglib-2.0