Closed
Bug 1087010
Opened 10 years ago
Closed 10 years ago
[Air Mozilla] Install ffmpeg
Categories
(Infrastructure & Operations :: IT-Managed Tools, task)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: peterbe, Assigned: gozer)
References
Details
(Whiteboard: [kanban:webops:https://kanbanize.com/ctrl_board/4/1807] )
Attachments
(1 file)
379.69 KB,
image/png
|
Details |
As of https://bugzilla.mozilla.org/show_bug.cgi?id=1085679 we have code that, on a cron job, periodically queries our videos to figure out how long they are using ffmpeg. The command is something like:: ffmpeg -i "https://vid.ly/0b3w5j?content=video&format=hd_mp4" That works here on my OSX where I have ffmpeg installed with these flags: https://gist.github.com/peterbe/d75785d696b88ee60e8d The code is ready to use it. The code assumes it can simply call out to `ffmpeg` on the command line but if we need to change it to an absolute path you can add something like this to the settings/local.py:: FFMPEG_LOCATION = '/opt/ffmpeg-1.0/bin/ffmpeg'
Comment 1•10 years ago
|
||
to clarify, will this cron be running on the admin node, or webheads?
Flags: needinfo?(peterbe)
Reporter | ||
Comment 2•10 years ago
|
||
(In reply to Chris Turra [:cturra] from comment #1) > to clarify, will this cron be running on the admin node, or webheads? Where all the other cron jobs are run. Is that the admin node? It won't need to call out to ffmpeg in runtime.
Flags: needinfo?(peterbe)
Assignee | ||
Updated•10 years ago
|
Assignee: server-ops-webops → gozer
Assignee | ||
Comment 3•10 years ago
|
||
r95208 | cturra@mozilla.com | 2014-10-21 18:40:05 -0400 (Tue, 21 Oct 2014) | 1 line [root@genericadm.private.phx1 ]# which ffmpeg /usr/bin/ffmpeg [root@genericadm.private.phx1 ]# ffmpeg -version FFmpeg version 0.6.5, Copyright (c) 2000-2010 the FFmpeg developers built on Apr 5 2012 10:23:20 with gcc 4.4.5 20110214 (Red Hat 4.4.5-6) configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --incdir=/usr/include --disable-avisynth --extra-cflags='-O2 -g -fPIC' --enable-avfilter --enable-avfilter-lavf --enable-libdc1394 --enable-libdirac --enable-libfaac --enable-libfaad --enable-libfaadbin --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libx264 --enable-gpl --enable-nonfree --enable-postproc --enable-pthreads --enable-shared --enable-swscale --enable-vdpau --enable-version3 --enable-x11grab libavutil 50.15. 1 / 50.15. 1 libavcodec 52.72. 2 / 52.72. 2 libavformat 52.64. 2 / 52.64. 2 libavdevice 52. 2. 0 / 52. 2. 0 libavfilter 1.19. 0 / 1.19. 0 libswscale 0.11. 0 / 0.11. 0 libpostproc 51. 2. 0 / 51. 2. 0 FFmpeg 0.6.5 libavutil 50.15. 1 / 50.15. 1 libavcodec 52.72. 2 / 52.72. 2 libavformat 52.64. 2 / 52.64. 2 libavdevice 52. 2. 0 / 52. 2. 0 libavfilter 1.19. 0 / 1.19. 0 libswscale 0.11. 0 / 0.11. 0 libpostproc 51. 2. 0 / 51. 2. 0
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 4•10 years ago
|
||
Maybe I just don't know what it's called when installed on linux, but can you check that it supports HTTPS? E.g. ffmpeg -i "https://vid.ly/0b3w5j?content=video&format=hd_mp4"
Assignee | ||
Comment 5•10 years ago
|
||
Nope, that version isn't built with OpenSSL, but http works $> ffmpeg -i 'http://vid.ly/0b3w5j?content=video&format=hd_mp4' FFmpeg version 0.6.5, Copyright (c) 2000-2010 the FFmpeg developers built on Apr 5 2012 10:23:20 with gcc 4.4.5 20110214 (Red Hat 4.4.5-6) configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --mandir=/usr/share/man --incdir=/usr/include --disable-avisynth --extra-cflags='-O2 -g -fPIC' --enable-avfilter --enable-avfilter-lavf --enable-libdc1394 --enable-libdirac --enable-libfaac --enable-libfaad --enable-libfaadbin --enable-libgsm --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libx264 --enable-gpl --enable-nonfree --enable-postproc --enable-pthreads --enable-shared --enable-swscale --enable-vdpau --enable-version3 --enable-x11grab libavutil 50.15. 1 / 50.15. 1 libavcodec 52.72. 2 / 52.72. 2 libavformat 52.64. 2 / 52.64. 2 libavdevice 52. 2. 0 / 52. 2. 0 libavfilter 1.19. 0 / 1.19. 0 libswscale 0.11. 0 / 0.11. 0 libpostproc 51. 2. 0 / 51. 2. 0 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x1a89670]multiple edit list entries, a/v desync might occur, patch welcome Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'http://vid.ly/0b3w5j?content=video&format=hd_mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf54.29.104 Duration: 00:38:24.39, start: 0.000000, bitrate: 3625 kb/s Stream #0.0(eng): Video: h264, yuv420p, 1280x720 [PAR 1:1 DAR 16:9], 3507 kb/s, 30 fps, 30 tbr, 30 tbn, 60 tbc Stream #0.1(eng): Audio: aac, 48000 Hz, stereo, s16, 109 kb/s At least one output file must be specified
Reporter | ||
Comment 6•10 years ago
|
||
Private events on vid.ly (e.g MoCo meetings) have to be accessed with HTTPS because we put a short term token into the URL. How hard would it be?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 7•10 years ago
|
||
(In reply to Peter Bengtsson [:peterbe] from comment #6) > Private events on vid.ly (e.g MoCo meetings) have to be accessed with HTTPS > because we put a short term token into the URL. > > How hard would it be? Well, it would require rebuilding a custom version of ffmpeg, and I'd rather avoid that. Couldn't you just use curl or similar to download the mp4 file and then do the ffmpeg invocation locally?
Reporter | ||
Comment 8•10 years ago
|
||
(In reply to Philippe M. Chiasson (:gozer) from comment #7) > (In reply to Peter Bengtsson [:peterbe] from comment #6) > > Private events on vid.ly (e.g MoCo meetings) have to be accessed with HTTPS > > because we put a short term token into the URL. > > > > How hard would it be? > > Well, it would require rebuilding a custom version of ffmpeg, and I'd rather > avoid that. > > Couldn't you just use curl or similar to download the mp4 file and then do > the ffmpeg invocation locally? Ok. I can try that.
Comment 9•10 years ago
|
||
Commit pushed to master at https://github.com/mozilla/airmozilla https://github.com/mozilla/airmozilla/commit/9f5778e118f1269074ee5b3170d6127170dd1a9e bug 1087010 - download then run ffmpeg on local disk file
Assignee | ||
Comment 10•10 years ago
|
||
Great, thanks!
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 11•10 years ago
|
||
It works! On dev only so far.
Reporter | ||
Comment 12•10 years ago
|
||
After a long discussion on IRC we concluded that this is not working very well on prod. The cron job is really struggling and causes a lot of IO downloading so much. We suspect it gets stuck sometimes causing the cron job to bunch up and ultimately not executing as often as we'd like. The reason we switched to first downloading; then doing `ffmpeg -i /local/file.mp4` was because ffmpeg doesn't have SSL support unless custom built. Hacking it to only download a partial file could work but not a sustainable solution because we want to expand the current functionality to download screencaptures into PNGs (and possibly animated gifs) and then we're going to need the whole file. If our ffmpeg supports SSL we'll be able to capture screencaptures by URL too. In the meantime I'm going to change it so that it does `ffmpeg -i http://...` for non private events (which can be reached over HTTP) and only does download for private events.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Comment 13•10 years ago
|
||
Commit pushed to master at https://github.com/mozilla/airmozilla https://github.com/mozilla/airmozilla/commit/91c5e75b3493a410626295d39d4f75aff86533bc bug 1087010 - only save private events locally for ffmpeg command
Reporter | ||
Comment 14•10 years ago
|
||
So last night I optimized (aka. "hacked") the code a bunch. Because ffmpeg can't do SSL we have to download it. However, Vidly allows us to drop the S in https and still can access the public videos. So, what it does now is that it downloads only those videos that are privacy!=public. Two thirds of all videos are public. So it means two thirds *less downloading*. Overnight it appears to have done better. There are now only 638 events left to process of the backlog. However, some day we'll do this all over again with not just fetching the video duration but also screenshots from the video. In other words, I guess, don't kill yourself over making a non-standard package of ffmpeg if it's too much pain, blood and tears.
Assignee | ||
Comment 15•10 years ago
|
||
Built and deployed a new version of ffmpeg (0.11) on genericadm that has https support, enjoy! It can be used/found here: /opt/air.mozilla.org/bin/ffmpeg Let me know if you encounter problems with it. Just make sure to either fix your PATH or to call *that* ffmpeg directly.
Status: REOPENED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Reporter | ||
Comment 16•10 years ago
|
||
That's great! I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1096570 as a followup. Thank you.
You need to log in
before you can comment on or make changes to this bug.
Description
•