1435620 - Basic support for H.264 / AVC I-frame (Intra-coded picture) in HEIF container file

Status: RESOLVED WONTFIX

(Core :: Audio/Video: Playback, defect)

Description

[s6oxu780lkkj]

Attachment: kodim19.zip contain .htm file, .mp4 and .heif image file

User Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0
Build ID: 20180128191252

Steps to reproduce:

Please provide basic support for .heif [1] and H.264/AVC Coding format within. The main intention of the HEIF standard is to deliver HEVC format for the still images instead of JPEG/WebP/JPEG-XR/Jpeg2000 etc. The standard also provided for placing the AVC format in the .heif container.

In the current situation, you can put the avc codec in the mp4 container but only in the <video> tag, even if it's a single frame of the image.

The solution to this problem would be to add .heif container (H.264 inside) support for the <img> tag.

Firefox already supports the AVC codec, so it would not be a big problem to use it for the <img> tag.

HEIF is based on the ISO Base Media File Format (ISO/IEC 14496-12).


Advantages:
- h.264 is highly optimized for both writing (x264) and reading (DXVA)
- heif is based upon a .mp4 container
- users can save images <img> as .heif's and edit, preview etc. on PC, instead of watch on browser single frame (<video>)
- h.264 is a trade-off between JPEG/JPEG-2000/JPEG-XR/WebP on the one hand and BPG/full HEIF (HEVC --> .heic)/expected AV1 [2] on the other hand. [3]

Disdvantages:
- .264 files must be mod2 (x264 encoder)
- only 8-bit image, i420 colorspace is displayed properly in most browsers (.mp4 in <video> tag)
- there may be a problem with displaying the image at a resolution of more than 1920x1080 on mobile phones or in a profile larger than 4.1



According to "Table I. Brands, MIME subtypes, and file extensions for HEIF." [1] .heif file must be:

Brand: mif1
Coding format: Any
Image or sequence?: image
MIME Type: image
MIME subtype: heif
File extension: .heif





"Do we really want to use Flash to display images?
I mean really?" [4]

It will now be possible to use the normal <img> tag.

----

Similar bug/feature request:
Bug 1402293 - HEIF / HEVC support
https://bugzilla.mozilla.org/show_bug.cgi?id=1402293





////////////////////////////////////////////////////////////////////////



User Agent:
Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0



Steps to reproduce:
===================


Image source "kodim19.png" must be mod2 [5]
ffmpeg build "ffmpeg.exe" [6]
x264 build "x264.exe" [7]


1. convert .png to raw .264 Advanced Video Codec stream
-------------------------------------------------------

****
ffmpeg.exe -i kodim19.png -an -pix_fmt yuv420p -f yuv4mpegpipe - | x264.exe --level 4.1 --preset placebo --aq-mode 3 --vbv-bufsize 78125 --vbv-maxrate 62500 --demuxer y4m --frames 1 --output "kodim19.264" - --crf 18
****

|
V

kodim19.png  656 kB
kodim19.264   56 kB


2. mux .264 stream into .heif [8]
---------------------------------

mp4box build "mp4box.exe" [9]

****
mp4box.exe -version
    MP4Box - GPAC version 0.7.2-DEV-rev378-g052b8a9b-master
****

****
mp4box.exe -add-image kodim19.264:primary -ab mif1 -new kodim19.heif
    AVC-H264 import - frame size 512 x 768 at 25.000 FPS
    AVC Import results: 1 samples (4 NALUs) - Slices: 1 I 0 P 0 B - 1 SEI - 1 IDR
    Adding sample from time 0.000 as item 1
    Saving to R:\kodim19.heif: 0.500 secs Interleaving
****


3. display .heif file
---------------------

mp4client.exe from mp4box build

****
mp4client.exe kodim19.heif
****


4. display .heif info
---------------------

****
mp4box.exe -info kodim19.heif
    Root Meta type: "pict" - 2 resource item(s)
    Primary Item - ID 1
    Item #1 - ID 1 - type avc1  - Name: Image
    File has no movie (moov) - static data container
****


5. You may dump raw .264 stream from .heif file [10]
----------------------------------------------------

****
mp4box.exe -dump-item 1:path=dump1.264 kodim19.heif
****


6. Create .mp4 file for insertion in <video> tag
------------------------------------------------

****
mp4box.exe -add "kodim19.264#trackID=1:name=" -new "kodim19.mp4"
    AVC-H264 import - frame size 512 x 768 at 25.000 FPS
    AVC Import results: 1 samples (4 NALUs) - Slices: 1 I 0 P 0 B - 1 SEI - 1 IDR
    Saving R:\kodim19.mp4: 0.500 secs Interleaving
****



Actual results:
===============

.mp4 in <video> tag is displaying properly.
.heif in <img> tag it's not displayed at all.



Expected results:
=================

.heif in <img> tag is displayed.



////////////////////////////////////////////////////////////////////////



[1]
HEIF Technical information
http://nokiatech.github.io/heif/technical.html

[2]
Photo format from Google and Mozilla could leave JPEG in the dust
https://www.cnet.com/news/google-mozilla-av1-photo-format-could-outdo-aging-jpeg/

[3]
H.264 for image compression
https://sonnati.wordpress.com/2010/10/19/h-264-for-image-compression/

[4]
Dark Shikari (doom9.org)
https://forum.doom9.org/showthread.php?p=1214731#post1214731

[5]
Kodak Lossless True Color Image Suite
http://r0k.us/graphics/kodak/kodim19.html

[6]
FFmpeg Builds
https://ffmpeg.zeranoe.com/builds/

[7]
x264/binaries
http://download.videolan.org/pub/videolan/x264/binaries/

[8]
GPAC support for HEIF
https://gpac.wp.imt.fr/2017/06/09/gpac-support-for-heif/

[9]
GPAC Nightly Builds
https://gpac.wp.imt.fr/downloads/gpac-nightly-builds/

[10]
Creating JPEG file from .heic files
https://github.com/gpac/gpac/issues/901

Comment 1

[s6oxu780lkkj]

Attachment: kodim19.zip contain .htm file, .mp4 and .heif image file

This is the right version, htm + mp4 (not .x264) + heif

Comment 2

[s6oxu780lkkj]

It is worth noting that the available tools for JPEG compression do not give such results as using x264.

Example:

guetzli_windows_x86-64.exe --verbose --quality 84 kodim19.png kodim19.jpg
took about 30sec
.jpg is 68 kB and quality is worse than x264


.x264 from above took about 2sec


https://github.com/google/guetzli/releases

Comment 3

[Ralph Giles (:rillian)]

We have no plans to support images of this type. You might consider decoding them yourself in Wasm. If you remux AVC data into an mp4 container, you can use the native video decoder implementation and extract the frame data that way.

Thanks for your interest.