The default bug view has changed. See this FAQ.


Release Engineering
General Automation
3 years ago
2 years ago


(Reporter: catlee, Assigned: Callek)


(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)


(Whiteboard: [rollup status in c#18] [known next-steps in c#19])


(1 attachment)



3 years ago
We need some kind of automation around doing openh264 builds.

First pass of this should be a set of mozharness scripts/configs to generate the builds.

If required, we can hook this up into CI (task cluster perhaps).

Repo is here:

We'll need win32, osx, linux32, linux64 to start with. win64 is nice bonus. android and b2g are likely to be required at some point as well.

Comment 1

3 years ago
Per mtg with coop, taking this bug.

Unclear of requirements/steps atm, so I'll start by gathering those and dumping state here as I go.
Assignee: nobody → bugspam.Callek

Comment 2

3 years ago
Hi Chris,

We now have the GMP plugin build landed on  Here's how to build:

For Linux and OSX

32bit - 'make ENABLE64BIT=No plugin'
64bit - 'make ENABLE64BIT=Yes plugin'

By default these will make release builds.  Adding 'BUILDTYPE=Debug' will make debug builds.

The output of this will be  This combined with the file is what we will be distributing as the GMP plugin.    They will need to end up in a directory named gmp-gmpopenh264 in the interenet plugins directory which is '/usr/lib/mozilla/plugins' on Linux and '~/Libarary/Internet Plug-Ins' on OSX.

For Windows there is build info in the README.  It is essentially the deveoper shell with msys.  I'm assuming we will want to build with msvc instead of gcc so the command would be:

'make OS=msvc plugin'

Its output is gmpopenh264.dll and where it goes on disk involves the registry.  The code that looks for it can be found in the moz source here -

Comment 3

3 years ago
Ok, so had a chat with catlee, for some additional info:

He wrote a quick mozharness script to test this with:

Which mostly just worked "on his laptop".

Linux builds need (at least):
nasm, gtest, svn

Mac is one of the preferred starting platforms for exploratory testing (although all of mac, windows, linux are important)

There is no requirement on us to provide installer/update mechanics with the openh264 pieces itself (all of that will be part of Firefox)

We will not be hosting these files for our users, cisco will. However they do not need to be private and can be on ftp, or s3, etc.

I theorized that windows might be slightly problematic with regard to which MSVC version we choose (I expected that we'd need the same MSVC version as the Firefox version we ship, unless we want to additionally ship extra MSVC runtime dll's. Catlee is not so sure, but at a first-go its not necessarily the most important piece to find out)

Desire to have this built and fully tested by end of week, preferably "yesterday"

Comment 4

3 years ago
I'm testing mac builds ~now with the loaner I stole in Bug 1036254
Depends on: 1036254

Comment 5

3 years ago
Touchpoint, with default mozharness script as catlee provided: 

22:07:59     INFO -  nasm -DUNIX64 -DPREFIX -f macho64 -Icodec/common/x86/   -o codec/encoder/core/x86/coeff.o codec/encoder/core
22:07:59     INFO -  nasm: fatal: unrecognised output format `macho64' - use -hf for a list
22:07:59     INFO -  type `nasm -h' for help
22:07:59     INFO -  make: *** [codec/encoder/core/x86/coeff.o] Error 1
22:07:59    ERROR - Return code: 2
22:07:59    FATAL - couldn't build plugin
22:07:59    FATAL - Running post_fatal callback...
22:07:59    FATAL - Exiting -1
[ openh264]$ nasm -hf
usage: nasm [-@ response file] [-o outfile] [-f format] [-l listfile]
            [options...] [--] filename
    or nasm -r   for version info (obsolete)
    or nasm -v   for version info (preferred)

    -t          assemble in SciTech TASM compatible mode
    -g          generate debug information in selected format.
    -e          preprocess only (writes output to stdout by default)
    -a          don't preprocess (assemble only)
    -M          generate Makefile dependencies on stdout

    -E<file>    redirect error messages to file
    -s          redirect error messages to stdout

    -F format   select a debugging format

    -I<path>    adds a pathname to the include file path
    -O<digit>   optimize branch offsets (-O0 disables, default)
    -P<file>    pre-includes a file
    -D<macro>[=<value>] pre-defines a macro
    -U<macro>   undefines a macro
    -X<format>  specifies error reporting format (gnu or vc)
    -w+foo      enables warnings about foo; -w-foo disables them
where foo can be:
    macro-params            macro calls with wrong no. of params (default off)
    macro-selfref           cyclic macro self-references (default off)
    orphan-labels           labels alone on lines without trailing `:' (default off)
    number-overflow         numeric constants greater than 0xFFFFFFFF (default on)
    gnu-elf-extensions      using 8- or 16-bit relocation in ELF, a GNU extension (default off)

response files should contain command line parameters, one per line.

valid output formats for -f are (`*' denotes default):
  * bin       flat-form binary files (e.g. DOS .COM, .SYS)
    aout      Linux a.out object files
    aoutb     NetBSD/FreeBSD a.out object files
    coff      COFF (i386) object files (e.g. DJGPP for DOS)
    elf       ELF32 (i386) object files (e.g. Linux)
    as86      Linux as86 (bin86 version 0.3) object files
    obj       MS-DOS 16-bit/32-bit OMF object files
    win32     Microsoft Win32 (i386) object files
    rdf       Relocatable Dynamic Object File Format v2.0
    ieee      IEEE-695 (LADsoft variant) object file format
    macho     NeXTstep/OpenStep/Rhapsody/Darwin/MacOS X object files
[ openh264]$

Comment 6

3 years ago
Also failed with --32

per from repo:
NASM needed to be installed for assembly code: workable version 2.07 or above, nasm can downloaded from

[ openh264]$ nasm -v
NASM version 0.98.40 (Apple Computer, Inc. build 11) compiled on May 16 2011

updated nasm ...

[ ~]$ wget
--2014-07-08 22:14:37--
Resolving, 2001:1868:205::10
Connecting to||:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250711 (1.2M) [application/zip]
Saving to: `'

100%[=======================================================================================>] 1,250,711   --.-K/s   in 0.06s

2014-07-08 22:14:37 (19.5 MB/s) - `' saved [1250711/1250711]

[ ~]$ unzip
  inflating: nasm-2.11.05/ldrdf
  inflating: nasm-2.11.05/README
  inflating: nasm-2.11.05/nasm
  inflating: nasm-2.11.05/rdfdump
  inflating: nasm-2.11.05/LICENSE
  inflating: nasm-2.11.05/rdf2ihx
  inflating: nasm-2.11.05/ndisasm
  inflating: nasm-2.11.05/nasmdoc.pdf
  inflating: nasm-2.11.05/rdflib
  inflating: nasm-2.11.05/rdf2bin
  inflating: nasm-2.11.05/rdf2ith
  inflating: nasm-2.11.05/man1/rdf2bin.1
 extracting: nasm-2.11.05/man1/rdf2srec.1
 extracting: nasm-2.11.05/man1/rdf2ihx.1
 extracting: nasm-2.11.05/man1/rdf2ith.1
  inflating: nasm-2.11.05/man1/ldrdf.1
  inflating: nasm-2.11.05/man1/rdflib.1
 extracting: nasm-2.11.05/man1/rdf2com.1
  inflating: nasm-2.11.05/man1/rdfdump.1
  inflating: nasm-2.11.05/man1/ndisasm.1
  inflating: nasm-2.11.05/man1/nasm.1
  inflating: nasm-2.11.05/man1/rdx.1
  inflating: nasm-2.11.05/rdf2srec
  inflating: nasm-2.11.05/rdx
  inflating: nasm-2.11.05/rdf2com
[ ~]$ ls
Desktop                 Downloads               Movies                  Pictures                nasm-2.11.05
Documents               Library                 Music                   Public        
[ ~]$ rm -rf
[ ~]$ popd
[ openh264]$ export PATH=~/nasm-2.11.05/
LICENSE      ldrdf        nasm         ndisasm      rdf2com      rdf2ith      rdfdump      rdx
README       man1/        nasmdoc.pdf  rdf2bin      rdf2ihx      rdf2srec     rdflib
[ openh264]$ export PATH=~/nasm-2.11.05/:$PATH
[ openh264]$ echo $PATH

and things just worked with default script

Comment 7

3 years ago
Running gtest from mozharness failed due to the following:
[ openh264]$ svn co gtest
Error validating server certificate for '':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: *
 - Valid: from Wed, 04 Jun 2014 08:37:35 GMT until Tue, 02 Sep 2014 00:00:00 GMT
 - Issuer: Google Inc, US
 - Fingerprint: 98:19:0d:20:e0:da:1f:e8:ff:c4:06:f2:7d:13:2f:83:dc:7b:d5:eb
(R)eject, accept (t)emporarily or accept (p)ermanently? p

I permanently accepted it, and then was able to run from mozharness without issue. All tests pass.

Repeating with --32 now....

Comment 8

3 years ago
--32 bit OSX built fine, however gtest failed with the mozharness script as exists.

(I suspect the issue is that we need to pass the ENABLE64BIT=no to the other make calls as well)

22:28:04     INFO -  ld: symbol(s) not found for architecture x86_64
22:28:04     INFO -  collect2: ld returned 1 exit status
22:28:04     INFO -  make: *** [codec_unittest] Error 1


As a touchpoint I uploaded the referenced (needed) files to people.m.c for anyone who wants to play with them in any way. Since I'm told they are not private.


  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
 OSX  |   64    |    GOOD   |    PASS   |
 OSX  |   32    |    GOOD   |    FAIL   |
Whiteboard: [status in c#8]

Comment 9

3 years ago
Thanks Callek!

I've uploaded linux builds here:


You're right re: the test failures with 32-bit. I've fixed up the script to set the ENABLE64BIT=no variable properly for 32-bit tests, and now they also work properly for me.


3 years ago
Depends on: 1036768

Comment 10

3 years ago
Per c#9 the new script is

Windows is having problems (I'll detail at end of this comment)


  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
 OSX  |   64    |    GOOD   |    PASS   |
 OSX  |   32    |    GOOD   |    PASS   |

  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
 OSX  |   64    |    GOOD   |    PASS   | <not uploaded, but I can>
 OSX  |   32    |    GOOD   |    PASS   | <not uploaded, but I can>


So, I had lots of trouble connecting properly to windows (ironically since I use windows locally). Eventually I was able to connect as administrator (root) on my loaner...

I had to install Git as in Bug 963617

Following that, running this mozharness on windows is giving me mozharness complete and utter failure....

23:55:44    ERROR - caught OS error 2: The system cannot find the file specified while running ['', '-r', 'master', '', 'c:\\builds\\moz2_slave\\openh
23:55:44     INFO - rmtree: c:\builds\moz2_slave\openh264\build\src
23:55:44     INFO - Using _rmtree_windows ...
23:55:44     INFO - retry: Calling <bound method OpenH264Build._rmtree_windows of <__main__.OpenH
Build object at 0x028BEB50>> with args: ('c:\\builds\\moz2_slave\\openh264\\build\\src',), kwargs
}, attempt #1
23:55:44     INFO - retry: Failed, sleeping 60 seconds before retrying

This was done after launching start-msvc10.bat from /c/mozilla-build

And the following history:

$ history
    1  cd /c/builds/moz2_slave/
    2  cd openh264/
    3  export PATH=/C/mozilla-build/nsis-2.46u:/C/mozilla-build/python27:/C/mozilla-build/buildbotve
m\ Files\ \(x86\)/Microsoft\ SQL\ Server/100/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/10
0/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/100/DTS/Binn/:/c/Program\ Files\ \(x86\)/Wind
ows\ Kits/8.0/Windows\ Performance\ Toolkit/:/c/mozilla-build/moztools-x64/bin:/c/mozilla-build/vim/
    4  ls
    5  rm -rf build
    6  ls
    7  python scripts/scripts/ --os msvc
    8  wget -O /c/mozilla-build/Git/
    9  mv /c/mozilla-build/Git/{,bin/}
   10  python scripts/scripts/ --os msvc
   11  mv /c/mozilla-build/Git/bin/{,real_}
   12  echo "python ./ %*" > /c/mozilla-build/Git/bin/
   13  cat /c/mozilla-build/Git/bin/
   14  python scripts/scripts/ --os msvc
   15  python scripts/scripts/ --os msvc
   16  python scripts/scripts/ --os msvc
   17  history

Apologies for the delay here, but I'm gonna need :catlee, or :aki's help deciphering how to fix this Git issue with mozharness (first thing tomorrow)
Whiteboard: [status in c#8] → [status in c#10]

Comment 11

3 years ago
Changing Requirement (Based on My Understandings thus far):

The two files should be shipped in a .zip instead of flat.
 \- unspecified is compression amount, /me will choose default
 |- unspecified is required structure/layout of the zip internals.

Per bsmedberg in

Ben, is there any requirement around zip file layout?

If not I will be doing
  |- ./
   \- ./{libprefix}gmpopenh264.{libsuffix}
   |- ./
Flags: needinfo?(benjamin)
Just the two files in the toplevel of the ZIP. I presume we should use the best compression.
Flags: needinfo?(benjamin)

Comment 13

3 years ago
Per :catlee, proxying legal. I've restricted my people directory of these files:

SSLRequireSSL On
AuthType Persona
Require persona-idp

tl;dr is you need to load via https:// and its restricted to login via persona.

Comment 14

3 years ago

Windows [non-debug] bulds success!

Did not yet try debug

 * Had to explicitly pass --32, (ENABLE64BIT=Yes failed at linker step)
 * Had to manually install Git on our systems
   \- needed to be last in PATH, to not conflict MSYS libs/binaries with our mozilla-build
 * Had to manually grab nasm ( and add to path
   \- used the zip, and simply unzipped
 * Test failed, (gtest checked out, then promptly tries to invoke g++, which fails)
 * Had to manually checkout the git sources, instead of relying on mozharness

Comment 15

3 years ago
*** [Current] Commands for [re-]Building on Windows ***

current builds used, which has all prerequisites installed

* Git installed to C:\mozilla-build\Git\  per Bug 963617
* nasm-2.11.05 installed [currently] to C:/builds/moz2_slave/openh264/nasm-2.11.05
* mozharness scripts checked out at C:/builds/moz2_slave/openh264/scripts
 \- from

--> launch start-msvc10.bat from C:\mozilla-build
* cd /c/builds/moz2_slave/openh264
* export PATH=/C/mozilla-build/nsis-2.46u:/C/mozilla-build/python27:/C/mozilla-build/buildbotve/scripts:/C/mozilla-build/msys/local/bin:/c/mozilla-build/wget:/c/mozilla-build/7zip:/c/mozilla-build/blat261/full:/c/mozilla-build/python:/c/mozilla-build/svn-win32-1.6.3/bin:/c/mozilla-build/upx203w:/c/mozilla-build/emacs-22.3/bin:/c/mozilla-build/info-zip:/c/mozilla-build/nsis-2.22:/c/mozilla-build/nsis-2.33u:/c/mozilla-build/nsis-2.46u:/c/mozilla-build/wix-351728:/c/mozilla-build/hg:/c/mozilla-build/python/Scripts:/c/mozilla-build/kdiff3:/c/mozilla-build/yasm:.:/C/mozilla-build/msys/local/bin:/C/mozilla-build/msys/mingw/bin:/C/mozilla-build/msys/bin:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0/:/c/mozilla-build:/c/mozilla-build/python27:/c/mozilla-build/python27/Scripts:/C/mozilla-build/msys/bin:/c/mozilla-build/vim/vim72:/c/mozilla-build/wget:/c/mozilla-build/info-zip:/c/CoreUtils/bin:/c/mozilla-build/buildbotve/scripts:/c/Program\ Files\ \(x86\)/Microsoft\ SQL\ Server/100/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/100/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/100/DTS/Binn/:/c/Program\ Files\ \(x86\)/Windows\ Kits/8.0/Windows\ Performance\ Toolkit/:/c/mozilla-build/moztools-x64/bin:/c/mozilla-build/vim/vim72:${PATH}:/c/mozilla-build/Git/bin/
* rm -rf output
* mkdir -p output/win32
* rm -rf build
* python scripts/scripts/ --os msvc --no-checkout-sources --no-build
* git clone build/src
* pushd build/src/; git show > /c/builds/moz2_slave/openh264/output/win32/git_show.txt; popd
* python scripts/scripts/ --os msvc --32 --no-checkout-sources
* cp build/src/gmpopenh264.{dll,info} output/win32/
* pushd output/win32; zip ./ gmpopenh264.{dll,info}; popd

---> from a local computer:
* sftp$/builds/moz2_slave/openh264/output/win32/* ./

Then you can transfer those files around per normal

Comment 16

3 years ago
*** [Current] Commands for [re-]Building on OSX ***

current builds used, which has all prerequisites installed

* nasm-2.11.05 installed [currently] to ~cltbld/nasm-2.11.05
* mozharness scripts checked out at /builds/slave/openh264/scripts
 \- from

--> ssh/launch terminal
* cd /builds/slave/openh264/
* svn co gtest
 \- <press "p">
 |- # This is explicitly to accept the googlecode cert fingerprint so the script will run fine

*  export   PATH=/tools/python/bin:/tools/buildbot/bin:/opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
* export   HG_SHARE_BASE_DIR=/builds/hg-shared
* export   CCACHE_COMPRESS=1
* export   CCACHE_DIR=/builds/ccache
* export   CCACHE_UMASK=002
* export PATH=~/nasm-2.11.05/:$PATH
* rm -rf output
* rm -rf build
* mkdir -p output/macosx64
* ./scripts/scripts/ --64
* pushd build/src/; git show > /builds/slave/openh264/output/macosx64/git_show.txt; popd
* cp build/src/libgmpopenh264.dylib output/macosx64/
* cp build/src/ output/macosx64/
* rm -rf build/
* mkdir -p output/macosx32
* ./scripts/scripts/ --32
* pushd build/src/; git show > /builds/slave/openh264/output/macosx32/git_show.txt; popd
* cp build/src/ output/macosx32/
* cp build/src/libgmpopenh264.dylib output/macosx32/
* cd output/macosx64/
* zip libgmpopenh264.dylib
* cd ../macosx32/
* zip libgmpopenh264.dylib

---> from a local computer:
use scp to get these files to where they need to be

Comment 17

3 years ago
Commands for generating shasums/md5sums on people:


* cd openh264
* echo "# 5th column is size" > filelist.txt;
* find . -mindepth 2 -type f -exec ls -la {} \; >> filelist.txt
* md5sum filelist.txt > MD5SUMS
* find . -mindepth 2 -type f -exec md5sum {} \; >> MD5SUMS
* sha1sum filelist.txt  > SHA1SUMS
* find . -mindepth 2 -type f -exec sha1sum {} \; >> SHA1SUMS


Comment 18

3 years ago
Ok Status Rollup!

OSX and Windows Built by me, see following table.

Windows Build Instructions in: c#15
OSX  Build Instructions in: c#16
Instructions for SHA/MD5 Sum generations in: c#17

Linux Builds were by :catlee, I did not reproduce myself yet, so not part of my SUMS/etc on people.m.c

Windows Tests won't run, due to attempts to compile with g++


  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL (1)
 OSX  |   64    |    GOOD   |    PASS   |
 OSX  |   32    |    GOOD   |    PASS   |
 WIN  |   32    |    GOOD   | <NOT_RUN> |
 LIN  |   64    |    GOOD   |    PASS?  |
 LIN  |   32    |    GOOD   |    PASS?  |
         MD5SUMS                        |
         SHA1SUMS                       |
    File List (includes file size)      |
   1 --- people url protected by persona. per :catlee/legal

  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
 OSX  |   64    |    GOOD   |    PASS   | <not uploaded, but I can>
 OSX  |   32    |    GOOD   |    PASS   | <not uploaded, but I can>
 WIN  |   32    |    GOOD   | <NOT_RUN> | <not uploaded, but I can>
 LIN  |   64    |    GOOD   |    PASS?  | <not uploaded, but I can>
 LIN  |   32    |    GOOD   |    PASS?  | <not uploaded, but I can>
Whiteboard: [status in c#10] → [rollup status in c#18]

Comment 19

3 years ago

* possibly rebuild the binaries per :catlee telling me theres an incomming API change.
* Integrate Linux Builds/Binaries to my own People publish, to include sha1sums/etc
* Update Mozharness to either checkout using git binary on windows, or coerce gittool to work on windows [and install it]
* Update mozharness script to upload these binaries somewhere other than people.m.c
* Get mozharness script reviewed and checked in.
* Make official mozharness script configs to correspond to paths/etc our machines need.
* Get Git properly deployed across windows [Bug 963617]


* CI to to the above
* Convince <others> how its not a good idea to svn checkout from tip of a repo (e.g. google) for testing
* mirror openh264 to inhouse ( and use that from the mozharness script
* Get [newer, on OSX] nasm deployed across our infra.
   \- only maybe due to the fact that as a .zip we could theoretically just have the mozharness grab it as a script pre-req and add to path
Whiteboard: [rollup status in c#18] → [rollup status in c#18] [known next-steps in c#19]

Comment 20

3 years ago
I have windows builds based on the v1.1 branch:

I did not gen shasum's for that yet, but will do so once I unblock the builds for osx and get linux builds out in general.

I'm also told the "rebuild request" for the master branch hasn't come in yet, so those are still as they sat from yesterday.


3 years ago
Blocks: 1038108


3 years ago
No longer blocks: 1038108
Depends on: 1038108

Comment 21

3 years ago
(In reply to Justin Wood (:Callek) from comment #19)
> ToDo's:
> * possibly rebuild the binaries per :catlee telling me theres an incomming
> API change.

:bhearsum told me over the weekend that we are "ready" for new binaries.

They exist for windows/mac now, Linux maybe later this evening  (SHA1SUMS, SHA512SUMS, filelist, etc all up a level)

Comment 22

3 years ago
to restate here, I have mac and win builds for firefox33 at

feel free to reach out if I have anything missing. (catlee sent his firefox33 builds along to cisco, so afaik I'm not blocking anything atm)

Comment 23

3 years ago
Created attachment 8481293 [details] [diff] [review]
mozharnes script for builds

We should get this landed.

Did you have any extra changes from what I had?
Attachment #8481293 - Flags: review?(bugspam.Callek)

Comment 24

3 years ago
Comment on attachment 8481293 [details] [diff] [review]
mozharnes script for builds

Review of attachment 8481293 [details] [diff] [review]:

no extras yet, I've been doing all steps outside of this manually
Attachment #8481293 - Flags: review?(bugspam.Callek) → review+


3 years ago
Attachment #8481293 - Flags: checked-in+
Merged to production, and deployed.
What's the status here?

Comment 27

3 years ago
We have tools written to reliably build the openh264 bits. Still TODO:
- Hook up to automation if desired
- Implement uploading logic

Comment 28

2 years ago
Catlee, I'm thinking I should give up these loaners that were established to do these builds, given the infrequency of them.

Do you agree the above steps should be enough to allow me to give them all up, rather than force their indefinite hold?
Flags: needinfo?(catlee)

Comment 29

2 years ago
Definitely, no need to keep hold of the loaners.
Flags: needinfo?(catlee)

Comment 30

2 years ago
we've since done followup builds in new bugs, and catlee has updated the script. r/f
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.