Closed Bug 1546836 Opened 5 years ago Closed 5 years ago

Video Playback Issues on macOS Version 10.14.5 Beta

Categories

(Core :: Security: Process Sandboxing, defect, P1)

68 Branch
Unspecified
macOS
defect

Tracking

()

VERIFIED FIXED
mozilla68
Tracking Status
firefox-esr60 67+ verified
firefox66 --- wontfix
firefox67 blocking verified
firefox68 + verified

People

(Reporter: haik, Assigned: haik)

References

Details

Attachments

(2 files)

We've received a report that Firefox on the (yet to be released) macOS 10.14.5 is not displaying video correctly due to our sandbox rules not allowing access to com.apple.MTLCompilerService. Specifically:

  1. Launch Firefox (latest) on 10.14.5 latest build
  2. Head to Netflix, cnn.com or YouTube.com. We used https://www.cnn.com/2019/04/23/us/brooklyn-man-sentenced-in-joggers-death/index.html but it shouldn’t matter.
  3. Playback a video, see green or orange depending on the hardware used
Assignee: nobody → haftandilian
Priority: -- → P1

On a 2015 MacBook Air (11-inch, Early 2015) I can reproduce the problem on the CNN video in comment 0, on Firefox Release/66, but not on Netflix or YouTube so far. Netflix serves Widevine video content which is decoded in our GMP process and YouTube is currently serving AV1 content which is decoded in our RDD process.

[Tracking Requested - why for this release]: We should keep an eye on this before 67 goes to release in case 10.14.5 gets shipped while it's out.

I am running 10.14.5 ((18F118d) on a Macbook Pro (13 inch, 2017). I see the issue on 66 release very clearly. I see it sometimes on 67b13 (there is a flash of green and then the video plays) and haven't seen it yet on 68 nightly. In all three instances I was testing with the CNN video above.

See Also: → 1547095

I will post a fix that adds an allow rule for com.apple.MTLCompilerService in the content sandbox policy when we are running on 10.14+. The com.apple.MTLCompilerService service's own sandbox policy includes system.sb but is otherwise very minimal. Sandbox violation debugging is very limited on newer macOS versions and at this point I don't know exactly which code path is triggering the connection to the compiler service, but it appears we need the service in 10.14.5 for shader compilation to draw UI elements per bug 1546435 which is also addressed by this fix.

I haven't been able to reproduce this problem on Netflix, YouTube, or Amazon Prime Video.

Allow access to Apple's Metal shader language compiler in our content process sandbox. Limit the sandbox policy change to 10.14 and newer OS versions to reduce risk given that problems have only been reported on 10.14.5.

Attachment #9060834 - Attachment description: Bug 1546836 - Video Playback Issues on macOS Version 10.14.5 Beta r?jmathies → Bug 1546836 - Video Playback and Form Rendering Issues on macOS Version 10.14.5 Beta r?jmathies

[Tracking Requested - why for this release]:
Any supported release running on macOS 10.14.5 is affected. 10.14.5 is expected to be released on May 14th before the next ESR ships.

Pushed by haftandilian@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/434766cf660f
Video Playback and Form Rendering Issues on macOS Version 10.14.5 Beta r=jmathies
Severity: major → critical

It seems the test case has been pretty well established, but in case some more observation is helpful:

I see this on some, but not all, YouTube videos.
I can 100% reproduce this when trying to view https://i.kinja-img.com/gawker-media/image/upload/qdatwtyaeimf5byhi8gk.mp4

This behavior appeared when macOS 10.14.5 Beta (18F118d)was installed. I have reproduced it with FF 66.0.3 and with FF 67.0b13

(In reply to Joe McCabe from comment #9)

It seems the test case has been pretty well established, but in case some more observation is helpful:

I see this on some, but not all, YouTube videos.
I can 100% reproduce this when trying to view https://i.kinja-img.com/gawker-media/image/upload/qdatwtyaeimf5byhi8gk.mp4

This behavior appeared when macOS 10.14.5 Beta (18F118d)was installed. I have reproduced it with FF 66.0.3 and with FF 67.0b13

Thanks for the test case. The fix should be tomorrow's Nightly build and we will work on getting this fixed in 67 and possibly 66.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68

Haik, don't hesitate to do the uplift request as soon as we can (we build beta 15 on Monday). I set the qe-verify+ flag so as that it is on our QA radar that we want it manually tested on both nightly and beta.

Flags: qe-verify+
Flags: needinfo?(haftandilian)
Flags: needinfo?(haftandilian)

Comment on attachment 9061168 [details] [diff] [review]
Bug 1546836 - Patch for build 67 uplift

Beta/Release Uplift Approval Request

  • User impact if declined: User will experience multiple issues with Firefox on macOS 10.14.15 (in Beta, expected to be released mid-May) including videos not playing and form controls not being visible.
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: Yes
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: See video link in the description and comment 9. See form controls issue described in bug 1546435.
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): The fix only affects macOS 10.14 and the change is just a sandbox change to allow access to an additional service which is typically low risk.
  • String changes made/needed: None
Attachment #9061168 - Flags: approval-mozilla-beta?
Comment on attachment 9061168 [details] [diff] [review]
Bug 1546836 - Patch for build 67 uplift

Blocker and only affects the next macOS 10.14.5 release, uplift approved for 67 beta 15 thanks!
Attachment #9061168 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Just installed macOS 10.14.5 Beta (18F127a) (released April 29). The video playback issue can not be reproduced with the above test case now. I have NOT updated to a nightly of FF that contains your fix.

(In reply to Joe McCabe from comment #19)

Just installed macOS 10.14.5 Beta (18F127a) (released April 29). The video playback issue can not be reproduced with the above test case now. I have NOT updated to a nightly of FF that contains your fix.

Thanks. I am seeing similar results. I can not reproduce the problem with today's 10.14.5 update (18F127a).

10.14.5 Beta (18F127a) - Not reproducible
10.14.5 Beta (18F118d) - Reproducible

I will follow up with our Apple contact to see if more information can be provided.

This fix is already in Firefox 67 Beta build 15 (yet to be released). I think we should continue with that, but pushing for a Firefox 66 uplift is off the table.

Haik, do you want to uplift this to ESR 60.7? If so, can you request uplift?

I would think that users of ESR are unlikely to upgrade their OS quickly.
On the other hand, if they do, it's a bad issue. So, I'm happy to take the patch.

Flags: needinfo?(haftandilian)

(In reply to Liz Henry (:lizzard) (use needinfo) from comment #21)

Haik, do you want to uplift this to ESR 60.7? If so, can you request uplift?

I would think that users of ESR are unlikely to upgrade their OS quickly.
On the other hand, if they do, it's a bad issue. So, I'm happy to take the patch.

OK. I'll request uplift.

Even though the bug is not reproducible with the latest 10.14.5 beta, I think uplifting to ESR is a good idea. We don't have a good understanding of what changed in the different 10.14.5 betas to make the problem appear and then go away. And we have a low risk fix ready to go.

Comment on attachment 9061168 [details] [diff] [review]
Bug 1546836 - Patch for build 67 uplift

ESR Uplift Approval Request

  • If this is not a sec:{high,crit} bug, please state case for ESR consideration: When running on macOS 10.14.5 beta (yet to be released) this bug causes major problems which can prevent videos from playing and form UI controls from displaying. Although Apple has released an updated 10.14.5 beta where the problem is no longer reproducible, we don't know what changed and hence how likely the problem is to return.
  • User impact if declined: Video playback for some videos will not work and some form UI controls will not be visible.
  • Fix Landed on Version: 68, 67
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): It's a Mac-specific, low risk sandboxing change to allow access to a system service on 10.14.x OS versions only.
  • String or UUID changes made by this patch: None
Flags: needinfo?(haftandilian)
Attachment #9061168 - Flags: approval-mozilla-esr60?

I haven't requested uplift for build 66 under the assumption that we will ship 67 before macOS 10.14.5 ships. If 10.14.5 ships earlier and the bug is reproducible on the release version of 10.14.5, users would be impacted until 67 is available.

Comment on attachment 9061168 [details] [diff] [review]
Bug 1546836 - Patch for build 67 uplift

Fix for video functionality with upcoming version of macOS.
Attachment #9061168 - Flags: approval-mozilla-esr60? → approval-mozilla-esr60+
QA Whiteboard: [qa-triaged]

I've tested on macOS 10.14.5 Beta 4(18F127a) using Firefox 66.0.3 and Firefox 67 Beta 13, but I can't reproduce the initial issue. I tested on Youtube, Netflix, CNN and on the links from Description and from comment 9, but without success, the videos seem to work properly.

I've tested also on Firefox 67 Beta 15 and latest Nightly 68.0a1 (05-06-2019) and the videos seem to work properly.

  • tested on MacBook Pro(Retina, 15-inch, Late 2013) - Intel Iris Pro 1536 MB

Since I can't reproduce the initial issue, could you please verify if this issue is fixed on Firefox 67 Beta 18 (https://archive.mozilla.org/pub/firefox/candidates/67.0b18-candidates/build1/mac/) and on ESR 60.7 (https://queue.taskcluster.net/v1/task/CXixG9dISXCJmIrEbW93fg/runs/0/artifacts/public/build/target.dmg)?

Thank you!

Flags: needinfo?(haftandilian)

I no longer have the original configuration. I am unable to reproduce the behavior with:

macOS: 10.14.5 Beta (18F131a)
Firefox: 67.0b18 (64-bit)

I don't know which of these being updated officially fixed the problem. I DO know that the macOS 10.14.5 Beta (18F127a) was installed, but with no changes to Firefox, I was no longer able to reproduce the behavior.

(In reply to Camelia Badau [:cbadau], Release Desktop QA from comment #28)

Since I can't reproduce the initial issue, could you please verify if this issue is fixed on Firefox 67 Beta 18 (https://archive.mozilla.org/pub/firefox/candidates/67.0b18-candidates/build1/mac/) and on ESR 60.7 (https://queue.taskcluster.net/v1/task/CXixG9dISXCJmIrEbW93fg/runs/0/artifacts/public/build/target.dmg)?

Thank you!

After this bug was filed and fixed in central, Apple released an 10.14.5 Beta update (18F127a) making the problem no longer reproducible. We don't expect the problem to be reproducible anymore. We should make sure the problem is not reproducible with the fix. We don't have more details about what changed in the 18F127a update at this time.

Flags: needinfo?(haftandilian)

To make sure that the problem is not reproducible with the fix we need someone who still have 10.14.5 Beta (18F118d).
Does anyone know if we can download 10.14.5 Beta (18F118d) now? If yes, where exactly?

By pure coincidence, I still have 10.14.5 Beta (18F118d) installed. I will take a look.

Flags: needinfo?(spohl.mozilla.bugs)

I was not able to reproduce the original issue with the CNN video in comment 0. However, I was able to reproduce the issue and verify the fix with https://i.kinja-img.com/gawker-media/image/upload/qdatwtyaeimf5byhi8gk.mp4 from comment 9.

Results:
10.14.5 Beta (18F118d), Firefox 67 Beta 13: Green screen
10.14.5 Beta (18F118d), Firefox 67 Beta 19: Video plays back successfully

Status: RESOLVED → VERIFIED
Flags: needinfo?(spohl.mozilla.bugs)

Stephen, do you still have macOS 10.14.5 Beta 18F118d? Could you please verify that the issue is fixed on latest Nightly 68.0a1 (https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/firefox-68.0a1.en-US.mac.dmg) and on 60.7.0ESR (https://archive.mozilla.org/pub/firefox/candidates/60.7.0esr-candidates/build1/mac/en-US/Firefox%2060.7.0esr.dmg)? Just to make sure that everything is good everywhere.

Thank you!

Flags: needinfo?(spohl.mozilla.bugs)

Apologies about the delay. I have just tested 60.7.0ESR and today's Nightly 68.0a1. The issue is fixed everywhere.

(In reply to Stephen A Pohl [:spohl] from comment #35)

Apologies about the delay. I have just tested 60.7.0ESR and today's Nightly 68.0a1. The issue is fixed everywhere.

Thanks, Stephen!

Flags: qe-verify+

Please specify a root cause for this bug. See :tmaity for more information.

Root Cause: --- → ?
Root Cause: ? → External Software Affecting Firefox
You need to log in before you can comment on or make changes to this bug.