855492 - Install GStreamer 0.10 on linux build machines.

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

We're looking to enable gstreamer by default in the not-too-distant future, so we need it on the build/test boxes.

The centos packages are:
 * gstreamer-devel-0.10.29
 * gstreamer-plugins-base-devel-0.10.29

Comment 1

[Amy Rich [:arr] [:arich]]

Handing this over to release engineering since they handle software with puppet.

Comment 2

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

(clarification: we're looking to enable gstreamer when running tests for now; not yet releasing with gstreamer enabled)

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

It's easy to install those extra two packages (see http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla/config.py#l114, specifially mock_packages for linux, linux64 and probably the matching asan keys). It's global by default, but we can set that up so that it's only some branches if necessary (eg line 1854). You could then do builds on try to see how it goes.

The bigger question is what you're trying to achieve (CCing sheriffs). From our IRC conversation the situation is
* we don't build with --enable-gstreamer for MoCo builds right now, but will when developers have cycles for 'handling working with an arbitrary gstreamer version installed on an arbitrary person's system'
* some linux distros are going to enable gstreamer, and we want to make sure we don't break them in the meantime
* the suggestion is to have tests on TBPL, which means you have to tangle with https://wiki.mozilla.org/Sheriffing/Job_Visibility_Policy
* we don't want to have gstreamer in any of our nightly/aurora/etc builds for now, which implies either
 * different mozconfig for dep and nightly builds (seems unlikely to meet sheriff approval)
 * adding new compile jobs which --enable-gstreamer for linux32 & linux64, which trigger the appropriate test suite (this will be non-trivial to setup on the RelEng side)

Comment 4

[Chris Pearce [:cpearce (Not reading bugmail)]]

I think to start with we should not build with enable-gstreamer by default on Try or elsewhere, and when we want to test gstreamer on Try we can include changes to configure.in in our push to enable gstreamer. That means we can at least test in the tbpl environment, but we don't add much releng overhead.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [buildbot-configs] Add devel packages for Try

I think this is less-fragile than copying the definition into the section for try overrides, but loop is separated a ways from the platform definitions. The other option is just to do this on all branches.

Comment 6

[[e-mail] (:bhearsum)]

Comment on attachment 733148 [details] [diff] [review]
[buildbot-configs] Add devel packages for Try

Review of attachment 733148 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Nick Thomas [:nthomas] from comment #5)
> Created attachment 733148 [details] [diff] [review]
> [buildbot-configs] Add devel packages for Try
> 
> I think this is less-fragile than copying the definition into the section
> for try overrides, but loop is separated a ways from the platform
> definitions. The other option is just to do this on all branches.

Unless I'm severely misunderstanding, it looks like this patch does add the packages to all branches. That seems fine to me though, as they're not going to do any harm....

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 733148 [details] [diff] [review]
[buildbot-configs] Add devel packages for Try

There's the block at the end which makes it try only.
https://hg.mozilla.org/build/buildbot-configs/rev/bebe9f85590b

edmin/cpearce, this will go live when we deploy the next set of changes to buildbot production. There will be a comment here when that happens. At the point you want to enable gstreamer on m-c & co we will need to adjust buildbot again.

Comment 8

[Chris Cooper [:coop] (he/him)]

Comment on attachment 733148 [details] [diff] [review]
[buildbot-configs] Add devel packages for Try

This is in production now.

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

The packages being installed on try runs.

Comment 10

[Edwin Flores [inactive from 2016-12-01] [:eflores] [:edwin]]

Seems to work for opt builds, but not debug. See e.g. [1] (opt failures are my fault :\).

[1] https://tbpl.mozilla.org/php/getParsedLog.php?id=22887793&tree=Try&full=1

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [buildbot-configs] Linux debug platforms too

Oops, this'll be what was nagging me about the previous patch. Leaves out linux64-dbg-st-an, any opinion on that Ben ?

Comment 12

[[e-mail] (:bhearsum)]

Comment on attachment 749641 [details] [diff] [review]
[buildbot-configs] Linux debug platforms too

Review of attachment 749641 [details] [diff] [review]:
-----------------------------------------------------------------

Seems like we should upgrade across the board. If we don't upgrade the static analysis builds, they won't be as useful as companions to the shipping builds.

Comment 13

[Nick Thomas [:nthomas] (UTC+12)]

This fixes up linux64-debug and linux64-dbg-st-an:
https://hg.mozilla.org/build/buildbot-configs/rev/c6e4d48e9221 

Bug 857697 accidentally fixed linux-debug.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: [buildbot-configs] Fix linux32 for non-try

Bug 857697 added .i686 suffixes to the package names for 32bit linux compiles in 64bit env, so we're installing them at the moment. This restores try-only for gstreamer.

Comment 15

[[e-mail] (:bhearsum)]

Comment on attachment 733148 [details] [diff] [review]
[buildbot-configs] Add devel packages for Try

This patch is in production.

Comment 16

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 750845 [details] [diff] [review]
[buildbot-configs] Fix linux32 for non-try

Note to self - actually requesting reviews is more effective. See comment #14 for details on this patch.

Comment 17

[[e-mail] (:bhearsum)]

Comment on attachment 750845 [details] [diff] [review]
[buildbot-configs] Fix linux32 for non-try

Review of attachment 750845 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla/config.py
@@ +2015,5 @@
>          for p, pc in BRANCHES[b]['platforms'].items():
>              if 'mock_packages' in pc:
>                  BRANCHES[b]['platforms'][p]['mock_packages'] = \
> +                    [x for x in BRANCHES[b]['platforms'][p]['mock_packages'] if x not in (
> +                        'gstreamer-devel', 'gstreamer-plugins-base-devel',

Why install these in addition to the .i686 versions?

Comment 18

[Nick Thomas [:nthomas] (UTC+12)]

This is actually removing packages for non-try branches. The .i686 packages are for the linux32-on-linux64-host case, without an arch it's for linux64. There's a whitespace change, but mostly it's adding the i686.

Comment 19

[[e-mail] (:bhearsum)]

Comment on attachment 750845 [details] [diff] [review]
[buildbot-configs] Fix linux32 for non-try

Review of attachment 750845 [details] [diff] [review]:
-----------------------------------------------------------------

Aaaaah, OK.

Comment 20

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 750845 [details] [diff] [review]
[buildbot-configs] Fix linux32 for non-try

https://hg.mozilla.org/build/buildbot-configs/rev/71454b7b456f

Comment 21

[Nick Thomas [:nthomas] (UTC+12)]

Should be good here again.