1024140 - Do open264 builds

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Chris AtLee [:catlee]]

We need some kind of automation around doing openh264 builds.

First pass of this should be a set of mozharness scripts/configs to generate the builds.

If required, we can hook this up into CI (task cluster perhaps).

Repo is here: https://github.com/cisco/openh264

We'll need win32, osx, linux32, linux64 to start with. win64 is nice bonus. android and b2g are likely to be required at some point as well.

Comment 1

[Justin Wood (:Callek)]

Per mtg with coop, taking this bug.

Unclear of requirements/steps atm, so I'll start by gathering those and dumping state here as I go.

Comment 2

[Chris AtLee [:catlee]]

Hi Chris,

We now have the GMP plugin build landed on github.com/cisco/openh264.  Here's how to build:

For Linux and OSX

32bit - 'make ENABLE64BIT=No plugin'
64bit - 'make ENABLE64BIT=Yes plugin'

By default these will make release builds.  Adding 'BUILDTYPE=Debug' will make debug builds.

The output of this will be libgmpopenh264.so/dylib.  This combined with the gmpopenh264.info file is what we will be distributing as the GMP plugin.    They will need to end up in a directory named gmp-gmpopenh264 in the interenet plugins directory which is '/usr/lib/mozilla/plugins' on Linux and '~/Libarary/Internet Plug-Ins' on OSX.

For Windows there is build info in the README.  It is essentially the deveoper shell with msys.  I'm assuming we will want to build with msvc instead of gcc so the command would be:

'make OS=msvc plugin'

Its output is gmpopenh264.dll and where it goes on disk involves the registry.  The code that looks for it can be found in the moz source here - http://mxr.mozilla.org/mozilla-central/source/content/media/gmp/GMPService.cpp#324

Comment 3

[Justin Wood (:Callek)]

Ok, so had a chat with catlee, for some additional info:

He wrote a quick mozharness script to test this with: http://hg.mozilla.org/users/catlee_mozilla.com/mozharness/file/4554900be7d1/scripts/openh264_build.py

Which mostly just worked "on his laptop".

Linux builds need (at least):
nasm, gtest, svn

Mac is one of the preferred starting platforms for exploratory testing (although all of mac, windows, linux are important)

There is no requirement on us to provide installer/update mechanics with the openh264 pieces itself (all of that will be part of Firefox)

We will not be hosting these files for our users, cisco will. However they do not need to be private and can be on ftp, or s3, etc.

I theorized that windows might be slightly problematic with regard to which MSVC version we choose (I expected that we'd need the same MSVC version as the Firefox version we ship, unless we want to additionally ship extra MSVC runtime dll's. Catlee is not so sure, but at a first-go its not necessarily the most important piece to find out)

Desire to have this built and fully tested by end of week, preferably "yesterday"

Comment 4

[Justin Wood (:Callek)]

I'm testing mac builds ~now with the loaner I stole in Bug 1036254

Comment 5

[Justin Wood (:Callek)]

Touchpoint, with default mozharness script as catlee provided: 

22:07:59     INFO -  nasm -DUNIX64 -DPREFIX -f macho64 -Icodec/common/x86/   -o codec/encoder/core/x86/coeff.o codec/encoder/core
/x86/coeff.asm
22:07:59     INFO -  nasm: fatal: unrecognised output format `macho64' - use -hf for a list
22:07:59     INFO -  type `nasm -h' for help
22:07:59     INFO -  make: *** [codec/encoder/core/x86/coeff.o] Error 1
22:07:59    ERROR - Return code: 2
22:07:59    FATAL - couldn't build plugin
22:07:59    FATAL - Running post_fatal callback...
22:07:59    FATAL - Exiting -1
[[e-mail] openh264]$ nasm -hf
usage: nasm [-@ response file] [-o outfile] [-f format] [-l listfile]
            [options...] [--] filename
    or nasm -r   for version info (obsolete)
    or nasm -v   for version info (preferred)

    -t          assemble in SciTech TASM compatible mode
    -g          generate debug information in selected format.
    -e          preprocess only (writes output to stdout by default)
    -a          don't preprocess (assemble only)
    -M          generate Makefile dependencies on stdout

    -E<file>    redirect error messages to file
    -s          redirect error messages to stdout

    -F format   select a debugging format

    -I<path>    adds a pathname to the include file path
    -O<digit>   optimize branch offsets (-O0 disables, default)
    -P<file>    pre-includes a file
    -D<macro>[=<value>] pre-defines a macro
    -U<macro>   undefines a macro
    -X<format>  specifies error reporting format (gnu or vc)
    -w+foo      enables warnings about foo; -w-foo disables them
where foo can be:
    macro-params            macro calls with wrong no. of params (default off)
    macro-selfref           cyclic macro self-references (default off)
    orphan-labels           labels alone on lines without trailing `:' (default off)
    number-overflow         numeric constants greater than 0xFFFFFFFF (default on)
    gnu-elf-extensions      using 8- or 16-bit relocation in ELF, a GNU extension (default off)

response files should contain command line parameters, one per line.

valid output formats for -f are (`*' denotes default):
  * bin       flat-form binary files (e.g. DOS .COM, .SYS)
    aout      Linux a.out object files
    aoutb     NetBSD/FreeBSD a.out object files
    coff      COFF (i386) object files (e.g. DJGPP for DOS)
    elf       ELF32 (i386) object files (e.g. Linux)
    as86      Linux as86 (bin86 version 0.3) object files
    obj       MS-DOS 16-bit/32-bit OMF object files
    win32     Microsoft Win32 (i386) object files
    rdf       Relocatable Dynamic Object File Format v2.0
    ieee      IEEE-695 (LADsoft variant) object file format
    macho     NeXTstep/OpenStep/Rhapsody/Darwin/MacOS X object files
[[e-mail] openh264]$

Comment 6

[Justin Wood (:Callek)]

Also failed with --32

per README.md from repo:
NASM needed to be installed for assembly code: workable version 2.07 or above, nasm can downloaded from http://www.nasm.us/


[[e-mail] openh264]$ nasm -v
NASM version 0.98.40 (Apple Computer, Inc. build 11) compiled on May 16 2011


updated nasm ...

[[e-mail] ~]$ wget http://www.nasm.us/pub/nasm/releasebuilds/2.11.05/macosx/nasm-2.1
1.05-macosx.zip
--2014-07-08 22:14:37--  http://www.nasm.us/pub/nasm/releasebuilds/2.11.05/macosx/nasm-2.11.05-macosx.zip
Resolving www.nasm.us... 198.137.202.10, 2001:1868:205::10
Connecting to www.nasm.us|198.137.202.10|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1250711 (1.2M) [application/zip]
Saving to: `nasm-2.11.05-macosx.zip'

100%[=======================================================================================>] 1,250,711   --.-K/s   in 0.06s

2014-07-08 22:14:37 (19.5 MB/s) - `nasm-2.11.05-macosx.zip' saved [1250711/1250711]

[[e-mail] ~]$ unzip nasm-2.11.05-macosx.zip
Archive:  nasm-2.11.05-macosx.zip
  inflating: nasm-2.11.05/ldrdf
  inflating: nasm-2.11.05/README
  inflating: nasm-2.11.05/nasm
  inflating: nasm-2.11.05/rdfdump
  inflating: nasm-2.11.05/LICENSE
  inflating: nasm-2.11.05/rdf2ihx
  inflating: nasm-2.11.05/ndisasm
  inflating: nasm-2.11.05/nasmdoc.pdf
  inflating: nasm-2.11.05/rdflib
  inflating: nasm-2.11.05/rdf2bin
  inflating: nasm-2.11.05/rdf2ith
  inflating: nasm-2.11.05/man1/rdf2bin.1
 extracting: nasm-2.11.05/man1/rdf2srec.1
 extracting: nasm-2.11.05/man1/rdf2ihx.1
 extracting: nasm-2.11.05/man1/rdf2ith.1
  inflating: nasm-2.11.05/man1/ldrdf.1
  inflating: nasm-2.11.05/man1/rdflib.1
 extracting: nasm-2.11.05/man1/rdf2com.1
  inflating: nasm-2.11.05/man1/rdfdump.1
  inflating: nasm-2.11.05/man1/ndisasm.1
  inflating: nasm-2.11.05/man1/nasm.1
  inflating: nasm-2.11.05/man1/rdx.1
  inflating: nasm-2.11.05/rdf2srec
  inflating: nasm-2.11.05/rdx
  inflating: nasm-2.11.05/rdf2com
[[e-mail] ~]$ ls
Desktop                 Downloads               Movies                  Pictures                nasm-2.11.05
Documents               Library                 Music                   Public                  nasm-2.11.05-macosx.zip
[[e-mail] ~]$ rm -rf nasm-2.11.05-macosx.zip
[[e-mail] ~]$ popd
/builds/slave/openh264
[[e-mail] openh264]$ export PATH=~/nasm-2.11.05/
LICENSE      ldrdf        nasm         ndisasm      rdf2com      rdf2ith      rdfdump      rdx
README       man1/        nasmdoc.pdf  rdf2bin      rdf2ihx      rdf2srec     rdflib
[[e-mail] openh264]$ export PATH=~/nasm-2.11.05/:$PATH
[[e-mail] openh264]$ echo $PATH
/Users/cltbld/nasm-2.11.05/:/tools/python/bin:/tools/buildbot/bin:/opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/us
r/X11/bin

.....
and things just worked with default script

Comment 7

[Justin Wood (:Callek)]

Running gtest from mozharness failed due to the following:
[[e-mail] openh264]$ svn co https://googletest.googlecode.com/svn/trunk/ gtest
Error validating server certificate for 'https://googletest.googlecode.com:443':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: *.googlecode.com
 - Valid: from Wed, 04 Jun 2014 08:37:35 GMT until Tue, 02 Sep 2014 00:00:00 GMT
 - Issuer: Google Inc, US
 - Fingerprint: 98:19:0d:20:e0:da:1f:e8:ff:c4:06:f2:7d:13:2f:83:dc:7b:d5:eb
(R)eject, accept (t)emporarily or accept (p)ermanently? p

I permanently accepted it, and then was able to run from mozharness without issue. All tests pass.

Repeating with --32 now....

Comment 8

[Justin Wood (:Callek)]

--32 bit OSX built fine, however gtest failed with the mozharness script as exists.

(I suspect the issue is that we need to pass the ENABLE64BIT=no to the other make calls as well)

22:28:04     INFO -  ld: symbol(s) not found for architecture x86_64
22:28:04     INFO -  collect2: ld returned 1 exit status
22:28:04     INFO -  make: *** [codec_unittest] Error 1

-------

As a touchpoint I uploaded the referenced (needed) files to people.m.c for anyone who wants to play with them in any way. Since I'm told they are not private.


STATUS:

  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
======================================================================
 OSX  |   64    |    GOOD   |    PASS   | http://tinyurl.com/mm2dh9t
----------------------------------------------------------------------
 OSX  |   32    |    GOOD   |    FAIL   | http://tinyurl.com/m5mevf5
----------------------------------------------------------------------
.....TODO....

Comment 9

[Chris AtLee [:catlee]]

Thanks Callek!

I've uploaded linux builds here:

Linux32: http://people.mozilla.org/~catlee/openh264/linux32/
Linux64: http://people.mozilla.org/~catlee/openh264/linux64/

You're right re: the test failures with 32-bit. I've fixed up the script to set the ENABLE64BIT=no variable properly for 32-bit tests, and now they also work properly for me.

Comment 10

[Justin Wood (:Callek)]

Per c#9 the new script is http://hg.mozilla.org/users/catlee_mozilla.com/mozharness/file/4554900be7d1/scripts/openh264_build.py

Windows is having problems (I'll detail at end of this comment)

PRIOR STATUS:

  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
======================================================================
 OSX  |   64    |    GOOD   |    PASS   | http://tinyurl.com/mm2dh9t
----------------------------------------------------------------------
 OSX  |   32    |    GOOD   |    PASS   | http://tinyurl.com/m5mevf5
----------------------------------------------------------------------


DEBUG:
  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
======================================================================
 OSX  |   64    |    GOOD   |    PASS   | <not uploaded, but I can>
----------------------------------------------------------------------
 OSX  |   32    |    GOOD   |    PASS   | <not uploaded, but I can>
----------------------------------------------------------------------



===============================================================================
Windows:

So, I had lots of trouble connecting properly to windows (ironically since I use windows locally). Eventually I was able to connect as administrator (root) on my loaner...

I had to install Git as in Bug 963617

Following that, running this mozharness on windows is giving me mozharness gittool.py complete and utter failure....

(((
23:55:44    ERROR - caught OS error 2: The system cannot find the file specified while running ['
tool.py', '-r', 'master', 'https://github.com/cisco/openh264.git', 'c:\\builds\\moz2_slave\\openh
\\build\\src']
23:55:44     INFO - rmtree: c:\builds\moz2_slave\openh264\build\src
23:55:44     INFO - Using _rmtree_windows ...
23:55:44     INFO - retry: Calling <bound method OpenH264Build._rmtree_windows of <__main__.OpenH
Build object at 0x028BEB50>> with args: ('c:\\builds\\moz2_slave\\openh264\\build\\src',), kwargs
}, attempt #1
23:55:44     INFO - retry: Failed, sleeping 60 seconds before retrying
)))

This was done after launching start-msvc10.bat from /c/mozilla-build

And the following history:

$ history
    1  cd /c/builds/moz2_slave/
    2  cd openh264/
    3  export PATH=/C/mozilla-build/nsis-2.46u:/C/mozilla-build/python27:/C/mozilla-build/buildbotve
/scripts:/C/mozilla-build/msys/local/bin:/c/mozilla-build/wget:/c/mozilla-build/7zip:/c/mozilla-buil
d/blat261/full:/c/mozilla-build/python:/c/mozilla-build/svn-win32-1.6.3/bin:/c/mozilla-build/upx203w
:/c/mozilla-build/emacs-22.3/bin:/c/mozilla-build/info-zip:/c/mozilla-build/nsis-2.22:/c/mozilla-bui
ld/nsis-2.33u:/c/mozilla-build/nsis-2.46u:/c/mozilla-build/wix-351728:/c/mozilla-build/hg:/c/mozilla
-build/python/Scripts:/c/mozilla-build/kdiff3:/c/mozilla-build/yasm:.:/C/mozilla-build/msys/local/bi
n:/C/mozilla-build/msys/mingw/bin:/C/mozilla-build/msys/bin:/c/windows/system32:/c/windows:/c/window
s/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0/:/c/mozilla-build:/c/mozilla-build/python
27:/c/mozilla-build/python27/Scripts:/C/mozilla-build/msys/bin:/c/mozilla-build/vim/vim72:/c/mozilla
-build/wget:/c/mozilla-build/info-zip:/c/CoreUtils/bin:/c/mozilla-build/buildbotve/scripts:/c/Progra
m\ Files\ \(x86\)/Microsoft\ SQL\ Server/100/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/10
0/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/100/DTS/Binn/:/c/Program\ Files\ \(x86\)/Wind
ows\ Kits/8.0/Windows\ Performance\ Toolkit/:/c/mozilla-build/moztools-x64/bin:/c/mozilla-build/vim/
vim72:${PATH}:/c/mozilla-build/Git/bin/
    4  ls
    5  rm -rf build
    6  ls
    7  python scripts/scripts/openh264_build.py --os msvc
    8  wget -O /c/mozilla-build/Git/gittool.py http://mxr.mozilla.org/build/source/puppet/modules/pa
ckages/files/gittool.py\?raw=1
    9  mv /c/mozilla-build/Git/{,bin/}gittool.py
   10  python scripts/scripts/openh264_build.py --os msvc
   11  mv /c/mozilla-build/Git/bin/{,real_}gittool.py
   12  echo "python ./gittool.py %*" > /c/mozilla-build/Git/bin/gittool.py.bat
   13  cat /c/mozilla-build/Git/bin/gittool.py.bat
   14  python scripts/scripts/openh264_build.py --os msvc
   15  python scripts/scripts/openh264_build.py --os msvc
   16  python scripts/scripts/openh264_build.py --os msvc
   17  history


Apologies for the delay here, but I'm gonna need :catlee, or :aki's help deciphering how to fix this Git issue with mozharness (first thing tomorrow)

Comment 11

[Justin Wood (:Callek)]

Changing Requirement (Based on My Understandings thus far):

The two files should be shipped in a .zip instead of flat.
 \- unspecified is compression amount, /me will choose default
 |- unspecified is required structure/layout of the zip internals.

Per bsmedberg in https://bugzilla.mozilla.org/show_bug.cgi?id=1009816#c53

===
Ben, is there any requirement around zip file layout?

If not I will be doing

-foo.zip
  |- ./
   \- ./{libprefix}gmpopenh264.{libsuffix}
   |- ./gmpopenh264.info

Comment 12

[Benjamin Smedberg]

Just the two files in the toplevel of the ZIP. I presume we should use the best compression.

Comment 13

[Justin Wood (:Callek)]

Per :catlee, proxying legal. I've restricted my people directory of these files:

SSLRequireSSL On
AuthType Persona
Require persona-idp login.mozilla.org


tl;dr is you need to load via https:// and its restricted to login via persona.

Comment 14

[Justin Wood (:Callek)]

touchpoint:

Windows [non-debug] bulds success!

Did not yet try debug

Notes:
 * Had to explicitly pass --32, (ENABLE64BIT=Yes failed at linker step)
 * Had to manually install Git on our systems
   \- needed to be last in PATH, to not conflict MSYS libs/binaries with our mozilla-build
 * Had to manually grab nasm (http://www.nasm.us/pub/nasm/releasebuilds/2.11.05/win32/) and add to path
   \- used the zip, and simply unzipped
 * Test failed, (gtest checked out, then promptly tries to invoke g++, which fails)
 * Had to manually checkout the git sources, instead of relying on mozharness

Comment 15

[Justin Wood (:Callek)]

*** [Current] Commands for [re-]Building on Windows ***

current builds used b-2008-ix-0108.build.mozilla.org, which has all prerequisites installed

(pre-requisites)
* Git installed to C:\mozilla-build\Git\  per Bug 963617
* nasm-2.11.05 installed [currently] to C:/builds/moz2_slave/openh264/nasm-2.11.05
* mozharness scripts checked out at C:/builds/moz2_slave/openh264/scripts
 \- from http://hg.mozilla.org/users/catlee_mozilla.com/mozharness/rev/9020372db38c

--> launch start-msvc10.bat from C:\mozilla-build
* cd /c/builds/moz2_slave/openh264
* export PATH=/C/mozilla-build/nsis-2.46u:/C/mozilla-build/python27:/C/mozilla-build/buildbotve/scripts:/C/mozilla-build/msys/local/bin:/c/mozilla-build/wget:/c/mozilla-build/7zip:/c/mozilla-build/blat261/full:/c/mozilla-build/python:/c/mozilla-build/svn-win32-1.6.3/bin:/c/mozilla-build/upx203w:/c/mozilla-build/emacs-22.3/bin:/c/mozilla-build/info-zip:/c/mozilla-build/nsis-2.22:/c/mozilla-build/nsis-2.33u:/c/mozilla-build/nsis-2.46u:/c/mozilla-build/wix-351728:/c/mozilla-build/hg:/c/mozilla-build/python/Scripts:/c/mozilla-build/kdiff3:/c/mozilla-build/yasm:.:/C/mozilla-build/msys/local/bin:/C/mozilla-build/msys/mingw/bin:/C/mozilla-build/msys/bin:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0/:/c/mozilla-build:/c/mozilla-build/python27:/c/mozilla-build/python27/Scripts:/C/mozilla-build/msys/bin:/c/mozilla-build/vim/vim72:/c/mozilla-build/wget:/c/mozilla-build/info-zip:/c/CoreUtils/bin:/c/mozilla-build/buildbotve/scripts:/c/Program\ Files\ \(x86\)/Microsoft\ SQL\ Server/100/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/100/Tools/Binn/:/c/Program\ Files/Microsoft\ SQL\ Server/100/DTS/Binn/:/c/Program\ Files\ \(x86\)/Windows\ Kits/8.0/Windows\ Performance\ Toolkit/:/c/mozilla-build/moztools-x64/bin:/c/mozilla-build/vim/vim72:${PATH}:/c/mozilla-build/Git/bin/
* rm -rf output
* mkdir -p output/win32
* rm -rf build
* python scripts/scripts/openh264_build.py --os msvc --no-checkout-sources --no-build
* git clone https://github.com/cisco/openh264.git build/src
* pushd build/src/; git show > /c/builds/moz2_slave/openh264/output/win32/git_show.txt; popd
* python scripts/scripts/openh264_build.py --os msvc --32 --no-checkout-sources
* cp build/src/gmpopenh264.{dll,info} output/win32/
* pushd output/win32; zip ./gmpopenh264.zip gmpopenh264.{dll,info}; popd

---> from a local computer:
* sftp [e-mail]:/C$/builds/moz2_slave/openh264/output/win32/* ./

Then you can transfer those files around per normal

Comment 16

[Justin Wood (:Callek)]

*** [Current] Commands for [re-]Building on OSX ***

current builds used bld-lion-r5-013.build.mozilla.org, which has all prerequisites installed

(pre-requisites)
* nasm-2.11.05 installed [currently] to ~cltbld/nasm-2.11.05
* mozharness scripts checked out at /builds/slave/openh264/scripts
 \- from http://hg.mozilla.org/users/catlee_mozilla.com/mozharness/rev/9020372db38c

--> ssh/launch terminal
* cd /builds/slave/openh264/
* svn co https://googletest.googlecode.com/svn/trunk/ gtest
 \- <press "p">
 |- # This is explicitly to accept the googlecode cert fingerprint so the script will run fine

*  export   PATH=/tools/python/bin:/tools/buildbot/bin:/opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin
* export   VERSIONER_PYTHON_PREFER_32_BIT=no
* export   VERSIONER_PYTHON_VERSION=2.7
* export   HG_SHARE_BASE_DIR=/builds/hg-shared
* export   CCACHE_COMPRESS=1
* export   CCACHE_DIR=/builds/ccache
* export   CCACHE_UMASK=002
* export PATH=~/nasm-2.11.05/:$PATH
* rm -rf output
* rm -rf build
* mkdir -p output/macosx64
* ./scripts/scripts/openh264_build.py --64
* pushd build/src/; git show > /builds/slave/openh264/output/macosx64/git_show.txt; popd
* cp build/src/libgmpopenh264.dylib output/macosx64/
* cp build/src/gmpopenh264.info output/macosx64/
* rm -rf build/
* mkdir -p output/macosx32
* ./scripts/scripts/openh264_build.py --32
* pushd build/src/; git show > /builds/slave/openh264/output/macosx32/git_show.txt; popd
* cp build/src/gmpopenh264.info output/macosx32/
* cp build/src/libgmpopenh264.dylib output/macosx32/
* cd output/macosx64/
* zip gmpopenh264.zip gmpopenh264.info libgmpopenh264.dylib
* cd ../macosx32/
* zip gmpopenh264.zip gmpopenh264.info libgmpopenh264.dylib

---> from a local computer:
use scp to get these files to where they need to be

Comment 17

[Justin Wood (:Callek)]

Commands for generating shasums/md5sums on people:

ssh [e-mail]

* cd openh264
* echo "# 5th column is size" > filelist.txt;
* find . -mindepth 2 -type f -exec ls -la {} \; >> filelist.txt
* md5sum filelist.txt > MD5SUMS
* find . -mindepth 2 -type f -exec md5sum {} \; >> MD5SUMS
* sha1sum filelist.txt  > SHA1SUMS
* find . -mindepth 2 -type f -exec sha1sum {} \; >> SHA1SUMS

MUCH REJOICE

Comment 18

[Justin Wood (:Callek)]

Ok Status Rollup!

OSX and Windows Built by me, see following table.

Windows Build Instructions in: c#15
OSX  Build Instructions in: c#16
Instructions for SHA/MD5 Sum generations in: c#17

Linux Builds were by :catlee, I did not reproduce myself yet, so not part of my SUMS/etc on people.m.c

Windows Tests won't run, due to attempts to compile with g++

STATUS:

  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL (1)
======================================================================
 OSX  |   64    |    GOOD   |    PASS   | http://tinyurl.com/nb2qv7e
----------------------------------------------------------------------
 OSX  |   32    |    GOOD   |    PASS   | http://tinyurl.com/nstqe7j
----------------------------------------------------------------------
 WIN  |   32    |    GOOD   | <NOT_RUN> | http://tinyurl.com/ky4ngb7
----------------------------------------------------------------------
 LIN  |   64    |    GOOD   |    PASS?  | http://tinyurl.com/mb2yrwj
----------------------------------------------------------------------
 LIN  |   32    |    GOOD   |    PASS?  | http://tinyurl.com/lwjkjfk
======================================================================
         MD5SUMS                        | http://tinyurl.com/mvlpzp4
----------------------------------------------------------------------
         SHA1SUMS                       | http://tinyurl.com/pyanuoy
----------------------------------------------------------------------
    File List (includes file size)      | http://tinyurl.com/odc9jqb
----------------------------------------------------------------------
   1 --- people url protected by persona. per :catlee/legal


DEBUG:
  OS  |  Arch   |   Build   |   Tests   |    People.m.c URL
======================================================================
 OSX  |   64    |    GOOD   |    PASS   | <not uploaded, but I can>
----------------------------------------------------------------------
 OSX  |   32    |    GOOD   |    PASS   | <not uploaded, but I can>
----------------------------------------------------------------------
 WIN  |   32    |    GOOD   | <NOT_RUN> | <not uploaded, but I can>
----------------------------------------------------------------------
 LIN  |   64    |    GOOD   |    PASS?  | <not uploaded, but I can>
----------------------------------------------------------------------
 LIN  |   32    |    GOOD   |    PASS?  | <not uploaded, but I can>
----------------------------------------------------------------------

Comment 19

[Justin Wood (:Callek)]

ToDo's:

* possibly rebuild the binaries per :catlee telling me theres an incomming API change.
* Integrate Linux Builds/Binaries to my own People publish, to include sha1sums/etc
* Update Mozharness to either checkout using git binary on windows, or coerce gittool to work on windows [and install it]
* Update mozharness script to upload these binaries somewhere other than people.m.c
* Get mozharness script reviewed and checked in.
* Make official mozharness script configs to correspond to paths/etc our machines need.
* Get Git properly deployed across windows [Bug 963617]

Maybe:

* CI to to the above
* Convince <others> how its not a good idea to svn checkout from tip of a repo (e.g. google) for testing
* mirror openh264 to inhouse (git.mozilla.org) and use that from the mozharness script
* Get [newer, on OSX] nasm deployed across our infra.
   \- only maybe due to the fact that as a .zip we could theoretically just have the mozharness grab it as a script pre-req and add to path

Comment 20

[Justin Wood (:Callek)]

I have windows builds based on the v1.1 branch:

https://people.mozilla.org/~jwood/openh264/branches/v1.1-Firefox32/win32/

I did not gen shasum's for that yet, but will do so once I unblock the builds for osx and get linux builds out in general.

I'm also told the "rebuild request" for the master branch hasn't come in yet, so those are still as they sat from yesterday.

Comment 21

[Justin Wood (:Callek)]

(In reply to Justin Wood (:Callek) from comment #19)
> ToDo's:
> 
> * possibly rebuild the binaries per :catlee telling me theres an incomming
> API change.

:bhearsum told me over the weekend that we are "ready" for new binaries.

They exist for windows/mac now, Linux maybe later this evening

https://people.mozilla.org/~jwood/openh264/2014-07-14.0/  (SHA1SUMS, SHA512SUMS, filelist, etc all up a level)

Comment 22

[Justin Wood (:Callek)]

to restate here, I have mac and win builds for firefox33 at https://people.mozilla.org/~jwood/openh264/branches/v1.1-Firefox33/

feel free to reach out if I have anything missing. (catlee sent his firefox33 builds along to cisco, so afaik I'm not blocking anything atm)

Comment 23

[Chris AtLee [:catlee]]

Attachment: mozharnes script for builds

We should get this landed.

Did you have any extra changes from what I had?

Comment 24

[Justin Wood (:Callek)]

Comment on attachment 8481293 [details] [diff] [review]
mozharnes script for builds

Review of attachment 8481293 [details] [diff] [review]:
-----------------------------------------------------------------

no extras yet, I've been doing all steps outside of this manually

Comment 25

[Chris Cooper [:coop] (he/him)]

Merged to production, and deployed.

Comment 26

[Laura Thomson :laura]

What's the status here?

Comment 27

[Chris AtLee [:catlee]]

We have tools written to reliably build the openh264 bits. Still TODO:
- Hook up to automation if desired
- Implement uploading logic

Comment 28

[Justin Wood (:Callek)]

Catlee, I'm thinking I should give up these loaners that were established to do these builds, given the infrequency of them.

Do you agree the above steps should be enough to allow me to give them all up, rather than force their indefinite hold?

Comment 29

[Chris AtLee [:catlee]]

Definitely, no need to keep hold of the loaners.

Comment 30

[Justin Wood (:Callek)]

we've since done followup builds in new bugs, and catlee has updated the script. r/f