Closed
Bug 973274
Opened 11 years ago
Closed 10 years ago
Install GStreamer 1.x on linux build slaves
Categories
(Infrastructure & Operations Graveyard :: CIDuty, task)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: alessandro.d, Assigned: rail)
References
Details
Attachments
(2 files)
236.75 KB,
application/octet-stream
|
Details | |
1.58 KB,
patch
|
Details | Diff | Splinter Review |
gstreamer 1.x support has recently landed in m-c. Currently there's no way to test it on try as the build slaves don't have gstreamer 1.x packages. gstreamer 0.10.x and 1.x can be installed side to side, so that shouldn't be a problem. The packages needed would be: gstreamer, gst-plugins-base, gst-plugins-good, gst-plugins-bad, gst-libav. The latest gst version is 1.2.2, but any 1.x should be enough.
Updated•11 years ago
|
Component: Buildduty → Platform Support
QA Contact: armenzg → coop
Reporter | ||
Comment 1•11 years ago
|
||
Ping? Getting this fixed would make me this happy: http://imgur.com/p9CH1
Blocks: 947287
This could be a bit problematic, since all of our linux slaves are a bit... "stable"... AFAICT, CentOS 6 (as on the build slaves) still only has the ancient 0.10.29 in the latest version. Ubuntu 12.04 (as on the test slaves) is still on 0.10.36. The more recent Ubuntu 13.04 has 1.x. Nick, is there a plan to upgrade these any time soon? The test slaves at least -- missing headers at build time can be worked around. (I suspect the answer starts with "if it ain't broke...") Failing that, is there something we can do for not-in-repo packages? (I suspect the answer ends with "...dependency hell")
No longer blocks: 947287
Flags: needinfo?(nthomas)
Summary: Install GStreamer 1.x on linux build machines → Install GStreamer 1.x on linux build and test slaves
Blocks: 947287
Comment 3•11 years ago
|
||
I defer to coop on the scheduling question. It would probably help him if you were to describe the pros/cons of this upgrade, and what is unblocked by it. Others in RelEng have experience with compiling packages where the OS repos don't have recent enough versions.
Flags: needinfo?(nthomas) → needinfo?(coop)
Reporter | ||
Comment 4•11 years ago
|
||
https://bugzilla.mozilla.org/show_bug.cgi?id=851290 anyone? :P (that bug has gst 1.2.2 integrated in gecko-dev). Anyway, the gstreamer modules used by the ffox backend don't have that many dependencies. We would have to recompile liborc, glib, gstreamer, gst-plugins-base, gst-plugins-good, gst-plugins-bad, and gst-libav.
Comment 5•11 years ago
|
||
If a side-by-side install is indeed possible as suggested in comment #0, this will be less work. Alessandro: would you be willing to take a loaner slave (build first, then test) to investigate the upgrade yourself? Keep notes of packages you need to install and env changes made and then we can hopefully get those added to puppet more easily. I'm down two people in platform support at the moment, so that would be really helpful since you seem to have incentive here. https://wiki.mozilla.org/ReleaseEngineering/How_To/Request_a_slave
Flags: needinfo?(coop) → needinfo?(alessandro.d)
Comment 6•10 years ago
|
||
Alessandro: ping re: comment #5?
Updated•10 years ago
|
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → INCOMPLETE
Comment 7•10 years ago
|
||
Is there any plan to update the test slaves to a more recent distro?
Updated•10 years ago
|
Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Comment 8•10 years ago
|
||
Can you install gstreamer-1.0 from the gstreamer PPA? https://launchpad.net/~gstreamer-developers/+archive/ubuntu/ppa
Flags: needinfo?(coop)
Comment 9•10 years ago
|
||
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #7) > Is there any plan to update the test slaves to a more recent distro? Right now, no. That's a quarter-long project for multiple teams: releng, relops, ateam. Not saying it's not needed, or it won't be done, but there's no plan to bootstrap a new linux testing platform before 2015. (In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #8) > Can you install gstreamer-1.0 from the gstreamer PPA? > > https://launchpad.net/~gstreamer-developers/+archive/ubuntu/ppa It's not a simple question of installing a package and walking away. We need developer help to make sure the new package is working as intended for (at the very least) the builds you are concerned about. Releng can then run that modified configuration against other builds and/or tests to make sure it doesn't regress something elsewhere. I offered a loaner slave to Alessandro in comment #5 but never heard back.
Flags: needinfo?(coop)
Comment 10•10 years ago
|
||
(In reply to Chris Cooper [:coop] from comment #9) > (In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #7) > > Is there any plan to update the test slaves to a more recent distro? > > Right now, no. That's a quarter-long project for multiple teams: releng, > relops, ateam. I suggest you schedule some time in Q1 2016. > (In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #8) > It's not a simple question of installing a package and walking away. > > We need developer help to make sure the new package is working as intended > for (at the very least) the builds you are concerned about. Releng can then > run that modified configuration against other builds and/or tests to make > sure it doesn't regress something elsewhere. gstreamer-1.x can be installed alongside gstreamer-0.10. The code in gecko is separate for each API version. Right now most of the distros ship 1.0 but we don't test the 1.0 code path at all. > I offered a loaner slave to Alessandro in comment #5 but never heard back. He has given up a lot of his own time to help us.
Flags: needinfo?(alessandro.d)
Comment 11•10 years ago
|
||
Does that mean gstreamer 1.0 won't be enabled by default before 2016?
Reporter | ||
Comment 12•10 years ago
|
||
In the last year mozilla has gone from being very interested in GStreamer and wanting to use it as the media abstraction for at least linux and mac, to wanting to replace GStreamer with stagefright in linux. All this without involving me at all in the decisions, which I guess is fair enough, but yeah forgive me if I've lost interest a bit :) That said, i'm willing to help to setup 1.x on linux slaves. I just had a chat with simone in #developers who explained me a little how the slaves work. If someone can give me access to a centos 6.2 slave, I can build binaries for gst and deps. Then I would need some assistance to setup tooltool/mock envs to use them. How do I get ssh access?
Comment 13•10 years ago
|
||
(In reply to Alessandro Decina from comment #12) > In the last year mozilla has gone from being very interested in GStreamer > and wanting to use it as the media abstraction for at least linux and mac, > to wanting to replace GStreamer with stagefright in linux. All this without > involving me at all in the decisions, which I guess is fair enough, but yeah > forgive me if I've lost interest a bit :) > > That said, i'm willing to help to setup 1.x on linux slaves. I just had a > chat with simone in #developers who explained me a little how the slaves > work. If someone can give me access to a centos 6.2 slave, I can build > binaries for gst and deps. Then I would need some assistance to setup > tooltool/mock envs to use them. How do I get ssh access? https://wiki.mozilla.org/ReleaseEngineering/How_To/Request_a_slave
Comment 14•10 years ago
|
||
Any updates on this? Is anyone currently working on a slave? I'm guessing the gtk3 port would also like a more recent platform to build on.. I'm happy to help with an Ubuntu 12.04 slave (PPA option) or even setting up a new 14.04 slave if that's what we need to do..
Comment 15•10 years ago
|
||
(In reply to Bryan Quigley from comment #14) > Any updates on this? Is anyone currently working on a slave? I'm guessing > the gtk3 port would also like a more recent platform to build on.. > > I'm happy to help with an Ubuntu 12.04 slave (PPA option) or even setting up > a new 14.04 slave if that's what we need to do.. I'm not aware of anyone working on this.
Comment 16•10 years ago
|
||
Bryan was able to create builds with gstreamer1: https://bugzilla.mozilla.org/show_bug.cgi?id=1122859#c9 There's still a lot of packaging work to do in order to clean up the install process. If the system package upgrades are required, we'll also need to validate other linux builds to make sure they still work.
Assignee | ||
Comment 18•10 years ago
|
||
I just looked at the PPA for packages we'd need https://launchpad.net/~gstreamer-developers/+archive/ubuntu/ppa/+packages?field.name_filter=&field.status_filter=published&field.series_filter=precise 3 of them are broken at the moment. gst-plugins-{good,bad} require libvpx-dev (>= 1.1.0) which requires gcc-4.8 which not a thing in 12.04. I can try to use at https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test/+packages?field.name_filter=&field.status_filter=published&field.series_filter=precise, but it may take some time. I haven't looked at the build side yet.
Assignee | ||
Comment 19•10 years ago
|
||
The dependency on gcc-4.8 was added to fix some solaris issues. I downgraded the requirement and libvpx built normally. Then I found that gst-plugins-good1.0 requires libpulse 2.0+, but the default one for 12.04 is 1.1. It sounds like it may be easier just to upgrade the test machines :) $ ls *.deb | wc -l 86 I'll poke this again tomorrow or next week.
Comment 20•10 years ago
|
||
Also that PPA is abandoned, if anybody wants to take that over let me know. I'm not using Ubuntu anymore since many years, and once they started diverging too much from the Debian packages (and upstream tarballs...) I stopped updating it as I couldn't test the things anymore without installing Ubuntu anywhere. Just upgrade to a less ancient Ubuntu version, or build the required packages manually :)
Assignee | ||
Comment 21•10 years ago
|
||
Upgrading gstreamer will upgrade a lot of system libraries required by it. This is almost similar to upgrading the reference platform to a fresher Ubuntu version, what would cause a fair amount of work. I briefly talked to coop and jmaher about upgrading the reference platform and one of the ideas was upgrade and switch to docker images at the same time. Per IRC chat with arr, we may start the upgrade in 2015q2. I'll update the dependency when we have bugs on file.
Comment 22•10 years ago
|
||
Can we just install the gstreamer-1.0 headers so we can compile against them?
Assignee | ||
Comment 23•10 years ago
|
||
What about tests?
Comment 24•10 years ago
|
||
(In reply to Rail Aliiev [:rail] from comment #23) > What about tests? We aren't running any now. We're not testing 1.0 now and it is a different code path from 0.10. We need to drop support for 0.10 and most of the Linux distros use 1.0 now anyway. Tests starting in 2015q2, I guess.
Assignee | ||
Comment 25•10 years ago
|
||
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #22) > Can we just install the gstreamer-1.0 headers so we can compile against them? This is the easiest thing we can do! :) I have a couple of follow up questions. 1) Do we need any libraries as well? 2) How to specify what headers to use? (parameters for mozconfig) 3) How to verify builds?
Comment 26•10 years ago
|
||
(In reply to Rail Aliiev [:rail] from comment #25) > This is the easiest thing we can do! :) I have a couple of follow up > questions. It is far from ideal, but it is better than nothing. > 1) Do we need any libraries as well? I don't believe so. > 2) How to specify what headers to use? (parameters for mozconfig) I don't think we have a mechanism for that so we'll have to add one. > 3) How to verify builds? Gstreamer is dynamically linked if available. It just won't be available. We're already not testing gstreamer-1.0 so we won't be any worse off.
Assignee | ||
Comment 27•10 years ago
|
||
One more question: do we care about the *.pc files or having just the headers is enough?
Assignee | ||
Comment 28•10 years ago
|
||
I created a script which uses docker to create a tarball with headers from gstreamer gst-plugins-base gst-plugins-good gst-plugins-bad gst-libav packages (1.4.5): https://github.com/rail/gstreamer-docker-builder See attached tarball with generated headers. If the attached headers are enough to generate a build, I can upload it to tooltool and we can use it on try to play with actual builds.
Comment 29•10 years ago
|
||
Ralph - can you see if these headers are sufficient?
Flags: needinfo?(giles)
Comment 30•10 years ago
|
||
Do you need anything else from me or can I return the build slave (1122859)? Thanks!
Assignee | ||
Comment 31•10 years ago
|
||
(In reply to Bryan Quigley from comment #30) > Do you need anything else from me or can I return the build slave (1122859)? > Thanks! I think you can return it. If the headers look OK, we can continue using try.
Assignee | ||
Comment 32•10 years ago
|
||
...And thanks for your help!
Assignee | ||
Comment 33•10 years ago
|
||
Can anyone else verify that the attached archive with gstreamer headers is sufficient?
Comment 34•10 years ago
|
||
It does include all the files we include directly, so this should work fine.
Flags: needinfo?(giles)
Assignee | ||
Comment 35•10 years ago
|
||
Great! I'll upload the files and add some information on how to find them as a part of a build tomorrow.
Assignee | ||
Comment 36•10 years ago
|
||
If you apply this patch to your m-c tree, then you can access the headers in $topsrcdir (for example $topsrcdir/gstreamer-1.0/gst/gstbuffer.h) in your mozconfigs. Let me know if you need to add an extra directory to the file hierarchy in the archive to avoid header collisions using $topsrcdir. Maybe to something like $topsrcdir/gstreamer-headers/gstreamer-1.0/gst/gstbuffer.h.
Assignee | ||
Comment 37•10 years ago
|
||
Morphing the bug into build-only. The test part depends on OS upgrade. From what I've heard this can happen later this year.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Summary: Install GStreamer 1.x on linux build and test slaves → Install GStreamer 1.x on linux build slaves
Comment 38•9 years ago
|
||
Is there any ETA when this can actually happen?
Assignee | ||
Comment 39•9 years ago
|
||
The headers are available already, see comment #36. I'm not sure if there is a plan to switch to using them though.
Comment 40•9 years ago
|
||
Who would know about it? Without this switch, Firefox will be built against obsolete gstreamer. Practically all distros switched to 1.x and many already don't ship 0.10 at all.
Comment 41•9 years ago
|
||
(In reply to Shmerl from comment #40) > Who would know about it? Without this switch, Firefox will be built against > obsolete gstreamer. Practically all distros switched to 1.x and many already > don't ship 0.10 at all. I think you might want to ask in bug 947287 instead, considering that one is marked as closed.
Comment 42•9 years ago
|
||
(In reply to AnAkkk from comment #41) > I think you might want to ask in bug 947287 instead, considering that one is > marked as closed. Well, bug 947287 references this one as a prerequisite (since it's about installing gstreamer 1.x on Linux build machines). But since this one is closed, there is not tracking point for this dependency. This bug really should not be closed yet, since it didn't happen.
Comment 43•9 years ago
|
||
Or may be there is some separate bug for the test part. Then it should be added as a dependency for #947287.
Updated•6 years ago
|
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
Updated•5 years ago
|
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•