971618 - Support direct link for delivering OMA DRM 1.0 content (Forward Lock)

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia, defect)

Description

[Noemí Freire (:noemi)]

While testing DRM FL support with v1.3 branch:
Device: hamachi v1.3 (2/11 build)
BuildId: 20140211065801
Gecko: 06f88fa
Gaia: 00cd1ae
Platform: v28.0

We have seen the following behavior depending on how the content is offered:

*OMA download based on a descriptor file (dd) - OK
The content is downloaded, user can open it and it fails when trying to forward it (expected behavior)

*Direct link to protected content - KO
 ...
<a href=“http://www.server.com/drm/xxx.dm”>DRM content link</a>
...

The content is downloaded to the SD card but cannot be opened and no message is shown to the user


ni to David in order to confirm direct link support in the current implementation. Thanks!

Comment 1

[David Flanagan [:djf]]

There is no support for direct links to DM files in the current implementation.  From my reading of the OMA Download specification I did not think that was a requirement.

What is the behavior you would expect in this case?

And why would any vendor expose direct links to .dm files? Direct downloads would not report errors or installation success, which would make customer billing impossible, wouldn't it?

Comment 2

[Noemí Freire (:noemi)]

Hi,

TEF BUs offer the content through direct link so we would need this method to be supported. The current implementation is like not having DRM support due to the way the content is offered so it will cause an issue during the certification process. Because of this asking for 1.3?.
ni to David in order to confirm the feasibility for adding this support in v1.3. Thanks!

Comment 3

[Jason Smith [:jsmith]]

We are way too late in the release cycle to consider this. Why is this being nominated?

Comment 4

[bhavana bajaj [:bajaj]]

(In reply to Jason Smith [:jsmith] from comment #3)
> We are way too late in the release cycle to consider this. Why is this being
> nominated?

Release Managment is of the same notion. Can we move this 1.4?

Comment 5

[Preeti Raghunath(:Preeti)]

Based on Comment 1:

This looks like a new requirement. Please help us understand the requirement so we can decide if this can be taken up or not. Right now, this does seem impossible to take in 1.3. Will move it to 1.4 for now.

Comment 6

[David Flanagan [:djf]]

When this feature was being planned, we gathered product requirements from two other carriers, but as far as I know, TEF was not involved in the planning process. The feature has been available and ready for testing for 4 months or more, so it is frustrating to see requests like this come up so late in the cycle.  Setting needinfo for Adam to loop him in on this issue.

I have to agree with Jason that this is a feature request, not a bug, and I really don't think this is something that Mozilla can hold the release for.  So I think it is unlikely that this will be marked 1.3+.

Furthermore, the Media team is already fully committed for the 1.4 release and cannot take on any new features.

Does TEF have any developers who could work on this? I'd be happy to offer pointers to help them get started.

The difficulty I see in supporting direct links to forward locked files is that I don't know what the UX should be.  With the two step download process via .dd files, the user sees a confirmation dialog that displays content metadata (name, type, vendor, etc.) notifies them that the content they are downloading is locked, and gives them the opportunity to cancel the download before paying. To me, this is an important user-centric feature of Mozilla's implementation. If it was just a matter of doing direct downloads of .dm files through this same confirmation step, that would be technically straightforward.

I don't know how you handle billing in the direct download case, but I'm assuming that by the time the user has clicked on the link to the .dm file, they've already paid for the content, so offering a cancel option doesn't do any good. But I really don't like the idea of installing locked content on the user's phone without notifying the user properly.

If you know what UX you want in this case, and have developers who can implement that, we can offer guidance. I think it is fair for me to say that the forward lock app (apps/fl/) is not something that Mozilla considers a core feature of the OS. We created it to help meet urgent partner needs, and to demonstrate that forward lock can be securely implemented at the Gaia level without adding any DRM primitives to Gecko.  All along, we expected that partners would customize it to meet their specific needs. (But I'm sorry that we completely missed your specific need for direct links to .dm files!)

Comment 7

[Daniel Coloma:dcoloma]

(In reply to David Flanagan [:djf] from comment #6)
> When this feature was being planned, we gathered product requirements from
> two other carriers, but as far as I know, TEF was not involved in the
> planning process. 

This is extremely unfortunate as TEF OBs have been asking Mozilla for this feature since 1.0.1. Who was responsible of the planning so we can checked what happened?

> The feature has been available and ready for testing for 4
> months or more, so it is frustrating to see requests like this come up so
> late in the cycle.  

Absolutely, this is quite true. Our apologies for that. 

> Setting needinfo for Adam to loop him in on this issue.
> 
> I have to agree with Jason that this is a feature request, not a bug, and I
> really don't think this is something that Mozilla can hold the release for. 
> So I think it is unlikely that this will be marked 1.3+.
> 

OK, the point is that so far OEMs have been communicating to TEF OBs that 1.3 devices were going to support OMA DRM FL. With current approach, that is not going to be case as the current DRM implementation is useless for them. What carriers are going to receive is different to what they expect, in a feature they have been requesting for about 9 months... this is going to be difficult or impossible to handle.

> Furthermore, the Media team is already fully committed for the 1.4 release
> and cannot take on any new features.
> 
> Does TEF have any developers who could work on this? I'd be happy to offer
> pointers to help them get started.
> 

Yes, we could take care of this. Please feel free us to send me those links.

> The difficulty I see in supporting direct links to forward locked files is
> that I don't know what the UX should be.  With the two step download process
> via .dd files, the user sees a confirmation dialog that displays content
> metadata (name, type, vendor, etc.) notifies them that the content they are
> downloading is locked, and gives them the opportunity to cancel the download
> before paying. To me, this is an important user-centric feature of Mozilla's
> implementation. If it was just a matter of doing direct downloads of .dm
> files through this same confirmation step, that would be technically
> straightforward.
> 

Yes, this is exactly the case, just downloading the file.

> I don't know how you handle billing in the direct download case, but I'm
> assuming that by the time the user has clicked on the link to the .dm file,
> they've already paid for the content, so offering a cancel option doesn't do
> any good. But I really don't like the idea of installing locked content on
> the user's phone without notifying the user properly.
> 

Key concept here is that most of this content is not pay-per-item but all-you-can-eat, so users can download any item from the store as it is included as part of their subscription.

> If you know what UX you want in this case, and have developers who can
> implement that, we can offer guidance. I think it is fair for me to say that
> the forward lock app (apps/fl/) is not something that Mozilla considers a
> core feature of the OS. We created it to help meet urgent partner needs, and
> to demonstrate that forward lock can be securely implemented at the Gaia
> level without adding any DRM primitives to Gecko.  All along, we expected
> that partners would customize it to meet their specific needs. (But I'm
> sorry that we completely missed your specific need for direct links to .dm
> files!)

For me, the most important thing right now is implementing this for whatever version it is. Obviously, if the risk is low, I would prefer it to land it for 1.3. If that is not possible, a workaround would be handling the patch over to OEMs so they patch their versions: as I pointed out, I am really concerned not supporting this is going to cause the rejection of the device during IOT.

Comment 8

[David Flanagan [:djf]]

(In reply to Daniel Coloma:dcoloma from comment #7)

> > Does TEF have any developers who could work on this? I'd be happy to offer
> > pointers to help them get started.
> > 
> 
> Yes, we could take care of this. Please feel free us to send me those links.

I don't have any specific links. But if you have engineers who can start working on this, I can meet with them to explain how the current code works and discuss options for supporting direct links.

> 
> > The difficulty I see in supporting direct links to forward locked files is
> > that I don't know what the UX should be.  With the two step download process
> > via .dd files, the user sees a confirmation dialog that displays content
> > metadata (name, type, vendor, etc.) notifies them that the content they are
> > downloading is locked, and gives them the opportunity to cancel the download
> > before paying. To me, this is an important user-centric feature of Mozilla's
> > implementation. If it was just a matter of doing direct downloads of .dm
> > files through this same confirmation step, that would be technically
> > straightforward.
> > 
> 
> Yes, this is exactly the case, just downloading the file.

I don't think it is "just downloading" though.  Something has to happen on the screen when the user clicks on a link to a .dm file.  What should that be? Is it suitable in this case to display the same confirmation dialog that we do for the .dd case?  If so, that will probably simplify the implementation considerably.  At a minimum, for audio files, we'll need UX asking the user whether it is a ringtone or a song, since those two cases are handled separately.

> 
> Key concept here is that most of this content is not pay-per-item but
> all-you-can-eat, so users can download any item from the store as it is
> included as part of their subscription.
> 

It is helpful to know that. Thanks!

> 
> For me, the most important thing right now is implementing this for whatever
> version it is. Obviously, if the risk is low, I would prefer it to land it
> for 1.3. If that is not possible, a workaround would be handling the patch
> over to OEMs so they patch their versions: as I pointed out, I am really
> concerned not supporting this is going to cause the rejection of the device
> during IOT.

Comment 9

[Jason Smith [:jsmith]]

FYI to Daniel - I've pinged the product team offline about this. They are going to followup with you offline on this to discuss this more.

Comment 10

[Jason Smith [:jsmith]]

1.4 has moved to a model where we're only blocking on features that are either:

1. DSDS 1.4 required features
2. QC feature complete blockers

On that regard, this feature doesn't hit either of those two criteria, so this isn't a blocker.

Comment 11

[Carmen Jimenez Cabezas]

Attachment: Proposed patch

This is what we're using on our version. The changes you requested are not in, can you take a look before I continue?

Comment 12

[David Flanagan [:djf]]

Carmen,

I'm happy that you've got something that works for you.  I'll look at this as soon as I can, but expect to have a lot of high-priority reviews this week, so it might be a while before I can get back to you on it.

Comment 13

[David Bialer [:dbialer]]

As I am tracking this bug for Marketplace and content partners I just wanted to ask a couple of questions to ensure I understand:
1. This only affects forward-lock ringtones (not non-forward lock ringtones?)?  Is that a correct understanding?
2. As a result, users will not be able to install Forward-lock ringtones until this bug lands?
3. This is out-of-scope for 1.4?
Thanks

Comment 14

[Daniel Coloma:dcoloma]

(In reply to David Bialer [:dbialer] from comment #13)
> As I am tracking this bug for Marketplace and content partners I just wanted
> to ask a couple of questions to ensure I understand:
> 1. This only affects forward-lock ringtones (not non-forward lock
> ringtones?)?  Is that a correct understanding?

This affects any content (not only ringtones) protected with forward-lock. Currently FFOS supports FL protected Ringtones, Music and Wallpapers.

> 2. As a result, users will not be able to install Forward-lock ringtones
> until this bug lands?

There are two ways to expose Forward Lock protected content:

1/ Via a Descriptor File that points to the Download itself. The device downloads first the descriptor that contains metadata and information about the download file.
2/ Exposing the download file directly

Currently FFOS just supports 1, and unfortunately this is not enough for TEF (cannot speak on behalf of other operators)

> 3. This is out-of-scope for 1.4?

That is not my call, but it seems currently it is. We are working directly with OEMs so they can apply directly the patches attached to this bug. 

> Thanks

Comment 15

[Beatriz Rodríguez [:brg]]

David, could you please provide feedback to the proposed patch? We had executed the certification process with it and it passed our testing. Our target is to land this feature in master soon.

Comment 16

[David Flanagan [:djf]]

Beatriz,

I'm really sorry that I haven't been able to pay proper attention to this bug. I'm very happy that you've been able to resolve the certification blocker on your end.  I'm under extreme deadline pressure this week so will not have time to give feedback right away, but will try to make it happen next week.

Sorry!

Comment 17

[Beatriz Rodríguez [:brg]]

David, I guess you are still super busy but can you have some time to provide feedback in the proposed patch or tell us if some else could do it instead of you? Thanks in advance.

Comment 18

[David Flanagan [:djf]]

Comment on attachment 8384532 [details]
Proposed patch

Sorry that it has taken me so long to look this over.

I've left a lot of comments on the github commit. Most of my comments are about ways to make the changes clearer.

I have not tried actually running the code yet (if you create a PR, I'll try running it) so I don't understand what you're doing with the HTML changes... From the code it looks to me like you're displaying two different titles and progress indicators when downloading, which doesn't seem right.

Also the confirmOrPreview() function seems very confusing, and I'd like to have that code simplified before we land this.

Comment 19

[David Bialer [:dbialer]]

added whiteboard [dependency marketplace] since ringtone app developers will depend on this

Comment 20

[Antonio Manuel Amaya Calvo (:amac)]

Attachment: V2, including previous review comments

The patch also fixes the download components needed for OMA files sent with Content-Disposition: attachment to work.
Requesting review from David for the FL app and Francisco for the download components.

Comment 21

[Antonio Manuel Amaya Calvo (:amac)]

Stolen this from Carmen since she's busy with the vertical homescreen and this should land on 2.0, hopefully.

Comment 22

[Francisco Jordano [:arcturus] [:francisco]]

Comment on attachment 8429970 [details] [review]
V2, including previous review comments

Looking good to me the DM part.

Left some suggestions in github to try to make it a bit more versatile and be sure than in the future we don't shoot ourself on the foot.

Also could be nice to have unit tests for this, could be attached to the settings app where we use this shared component.

Thanks,
F.

Comment 23

[Beatriz Rodríguez [:brg]]

David, could you please review the patch proposal done by Antonio? We would like to land it in 2.0. Thanks in advance.

Comment 24

[David Flanagan [:djf]]

Comment on attachment 8429970 [details] [review]
V2, including previous review comments

I don't understand the integration between the download manager and the fl app here. Are there UX specs for how this is supposed to work?  Will the dd+xml type still go directly to the fl app?  Will direct download always go through the download manager and then be transferred to the fl app?  The code makes it look like sometimes it might go through the download manager and sometimes not, which confuses me.

I've left various comments on github. The main one, and the reason for the r-, is that using the view activity to pass locked content from the download manager to the fl app violates locking and exposes the locked content to any app that implements a view activity for that type.

Is download manager integration really necessary here? Can we have direct downloads always go through the FL app as in the previous version of the patch?  That would avoid the problem.

Comment 25

[Beatriz Rodríguez [:brg]]

Nominating to land in 2.0 to have the feature working fine.

Comment 26

[Jason Smith [:jsmith]]

(In reply to Beatriz Rodríguez [:brg] from comment #25)
> Nominating to land in 2.0 to have the feature working fine.

See https://groups.google.com/forum/#!topic/mozilla.dev.b2g/JuZHqK7P5a4. Features won't ever block the release - they should get approval if it's past FL to land on 2.0.

Comment 27

[Antonio Manuel Amaya Calvo (:amac)]

Now that bug 1021156 has a r+'ed patch, I've retaken this. I've updated the PR with the needed changes (and I hope included the previous reviews comments). Can you take another look, David?

Comment 28

[David Flanagan [:djf]]

I see that the patches in 1021156 landed yesterday. I have flashed a nightly build created after those patches landed, but can't get my test cases (at http://djf.net/drmtests/) to invoke the FL app. They are still going to the download manager.  So I'm not able to test and review the patch here yet.

Comment 29

[David Flanagan [:djf]]

I've verified that the .dd files from djf.net/drmtests/ are being served with the correct Content-Type header.

Antonio: are the patches from 1021156 working for you?

Comment 30

[Antonio Manuel Amaya Calvo (:amac)]

Yes. Then again I've only tested them with dm files. The content disposition header is only ignored for dm files but the view activity should be registered for both. 

Bar that. I've just rechecked and there's one contract missing on B2Gmanifest. And so only the dm files are registered. Can you file a bug with that and assign it to me? I'll do it first thing in the morning (it's just adding a contract line to that file...). Sorry about that.

Comment 31

[bhavana bajaj [:bajaj]]

Please note that this involves string changes and must be landed by tomorrow in order to make it into 2.0.

Comment 32

[Antonio Manuel Amaya Calvo (:amac)]

Actually the new string is a leftover from the previous version (when it was using the download manager). I'll remove it.

Comment 33

[Antonio Manuel Amaya Calvo (:amac)]

(In reply to Antonio Manuel Amaya Calvo (:amac) from comment #32)
> Actually the new string is a leftover from the previous version (when it was
> using the download manager). I'll remove it.

Ok, bar that, there's one new string on the FL app :(

Comment 34

[Antonio Manuel Amaya Calvo (:amac)]

I created bug 1028042 to reenable the support for DD files also. I'm testing DM file download using a small node script, that's available at https://github.com/AntonioMA/omadrmserver. With the patch in bug1028042 applied the DD files from http://djf.net/drmtests work perfectly also.

Comment 35

[David Flanagan [:djf]]

Thanks, Antonio, I will review this today.

Comment 36

[David Flanagan [:djf]]

Comment on attachment 8429970 [details] [review]
V2, including previous review comments

Thanks for all your work on this, Antionio. The code looks good and seems to work in all my testing.  Let's get this landed.

The FL app has never really been subject to much UX review at all, and it shows.  One issue with direct downloads in this version of the app is that when you download an image, the user is shown an image preview and an "Install" button, but nothing tells them that the image is going to be installed as wallpaper. In some contexts this might be perfectly clear, but in others it could be quite confusing.  If TEF cares about this, it is something that could be improved for 2.1.

Similarly, I'd love it if there was a step in the direct download process where the user saw the "locked content advisory" string that tells them that they will not be allowed to share the media file.

But neither of those things need to happen for this bug. For now we're just trying to acheive parity with what TEF shipped privately with the 1.3 release.

Comment 37

[David Flanagan [:djf]]

Comment on attachment 8429970 [details] [review]
V2, including previous review comments

I'd like to uplift this to 2.0. It is required for certification by TEFs operators. TEF has already shipped a version of this patch as part of their 1.3 tree. 

Originally we thought there were going to be new strings in this patch, but it turned out that they were not needed, so that should make this easier to uplift.

[Approval Request Comment]

[Bug caused by] (feature/regressing bug #): feature required for certification

[User impact] if declined: users won't be able to download "forward locked" songs, ringtones and wallpapers from the stores of carriers that use "direct download".

[Testing completed]: yes

[Risk to taking this patch] (and alternatives if risky): not risky. TEF shipped an earlier version of this patch in their 1.3 tree. We're only now finally syncing up the Mozilla tree with what the implemented.

[String changes made]: none

Comment 38

[bhavana bajaj [:bajaj]]

Comment on attachment 8429970 [details] [review]
V2, including previous review comments

Contained regression risk and is needed for certification. Approving the patch!
:amac can you please help with verification once this lands on 2.0

Comment 39

[Antonio Manuel Amaya Calvo (:amac)]

Travis isn't passing but it's a couple of unrelated fails on the homescreen... I believe we can check in this (specially since at least one of those failures has appeared before, and it doesn't touch anything outside of the FL app). I would have relaunched the Travis to see if the error goes away (as it shoud) but turns out that you can only do that if you have commit permissions on Gaia, which I don't :).

Comment 40

[Antonio Manuel Amaya Calvo (:amac)]

I'm going to take a look to the tests that fail (which seem to be homescreen related) but requesting checkin for this also since I don't think they're related with this patch in any form.

Comment 41

[Ryan VanderMeulen [:RyanVM]]

Master: https://github.com/mozilla-b2g/gaia/commit/bb1d5663149c5a00bf2e4e1480934f04162b0126

Comment 42

[isabelrios]

Already checked this issue with today's master buri build:
Gecko-fda21c3
Gaia-ed5231f

It is working fine.
Could you please uplift it to 2.0 since it already has the approval‑gaia‑v2.0+?

Thanks

Comment 43

[Ryan VanderMeulen [:RyanVM]]

v2.0: https://github.com/mozilla-b2g/gaia/commit/1de0bede7bf423ee7d02fc1b73d6b32cf99d5d7e

Comment 44

[isabelrios]

Also checked this on today's (06/25) v1.2 buri build:
Gecko-bc848c9
Gaia-7260d58

It is working fine so will change the status to Verified.

Thanks!

Comment 45

[isabelrios]

Sorry for my previous comment, I meant v2.0

Comment 46

[Jeremiah Lee]

Verified fixed on Flame 2.0 and 2.1.

User is able to download "forward locked"/DRM songs, ringtones and wallpapers (using link: http://djf.net/drmtests).

Device: Flame 2.1 (319mb)(Kitkat Base)(Shallow Flash)
BuildID: 20141126001202
Gaia: db2e84860f5a7cc334464618c6ea9e92ff82e9dd
Gecko: 211eae88f119
Version: 34.0 (2.1)
Firmware Version: v188-1
User Agent: Mozilla/5.0 (Mobile; rv:34.0) Gecko/34.0 Firefox/34.0

Device: Flame 2.0 (319mb)(Kitkat Base)(Shallow Flash)
Build ID: 20141126000203
Gaia: f9d6e3d83c3922e9399a6c27f5ce4cdd27bdfd05
Gecko: 45112935086f
Version: 32.0 (2.0)
Firmware Version: v188-1
User Agent: Mozilla/5.0 (Mobile; rv:32.0) Gecko/32.0 Firefox/32.0