971324 - Ensure Apps Marked Android Phone and Tablet are for real

Status: RESOLVED WONTFIX

(Marketplace Graveyard :: General, defect, P2)

Description

[David Bialer [:dbialer]]

We have a lot of apps that just look horrible on tablets or Android devices.  We should scrub the catalog - which means have a tedious task of checking that they really work on Android phones and tablets and adjusting discrepancies.  Otherwise the experience will not be Awesome.

Comment 1

[Christopher Van Wiemeersch [:cvan]]

This is a content issue. CC'ing content folks.

Comment 2

[Andrew Williamson [:eviljeff]]

Are there particular high profile apps we're worried about?  Checking 1100 apps on both a Android Mobile and Tablet manually will obviously be a large undertaking we'd have to find resources for.

Comment 3

[Lisa Brewster [:adora]]

Can you give examples of apps that are broken on Android phones or tablets?  The app review criteria does not require that apps be well designed, we only require that they work.

Comment 4

[Lisa Brewster [:adora]]

And to add to that...many Android apps, especially tablet apps, probably do suck and are barely above the minimum quality required for acceptance.  But we need to give developers incentives and reward good behavior rather than have a stricter review process.

Comment 5

[David Bialer [:dbialer]]

But the problem is that having apps that 'sort-of' work discredits the Marketplace on Android.  About a month ago I installed a whole bunch of apps on my Android tablet and maybe half of the ones I chose displayed properly (i.e. a 320x480 screen).  It left me with a very bad impression. Or at least the impression that a lot of apps were classified as compatible with Android Tablets and functioned poorly (to the point where they weren't usable).

I am not suggesting how we go about checking 1100 apps - only that we have a serious credibility problem if we call some of these apps compatible with Android phones and tablets.  If we agree that this is a problem, then we can figure out how we fix it.  Clearly non-trivial to go through 1100 apps.

I would have to get some examples from my device (which I don't have with me) - but will investigate.

Comment 6

[Andy McKay]

Any progress on this?

Comment 7

[Thomas Elin [:relaas]]

I hear you David. Clearly we do not want to have apps the are 'unusable' please provide some examples for this thread to chew on.

There's also a problem of overall quality concerns if the only apps available for Android are those that 'sort of' work. To fix this, we need to arm out Content team with clear GTM plans and strategies so they can motivate developers to submit better content.

We should not raise the minimum bar of acceptance beyond that of FFOS, Desktop, or any other apps submitted to Marketplace.

RE: assessing the catalog against Android tablets

I would only advocate a scrub of the catalog if a significant number of apps did not meet the minimum bar of acceptance. As an example; we test 20 random apps and 18 of those fail.

Comment 8

[David Bialer [:dbialer]]

I will assign to me to try to judge overall problem: I'll download 20 random apps to my tablet and judge the quality - and see if I can come up with criteria for judging whether an app is not tablet compatible.  We can then discuss if this.

Comment 9

[[e-mail]]

(In reply to David Bialer [:dbialer] from comment #8)
> I will assign to me to try to judge overall problem: I'll download 20 random
> apps to my tablet and judge the quality - and see if I can come up with
> criteria for judging whether an app is not tablet compatible.  We can then
> discuss if this.

David any updates on this? Wondering about making this not a blocker to bug 968380.

Comment 10

[David Bialer [:dbialer]]

It shouldn't block, but as a side comment, I've tested 8 apps so far (out of a random sample of 20).  6 of those 'pass' defined by me as Installs, Works, Formats correctly-enough.  I also give a subjective opinion of 'quality' but this didn't influence whether it passes.  I had intended to sample 20, but am not able to complete this within the next few weeks as I don't have my Android devices with me.

Comment 11

[Andrew Williamson [:eviljeff]]

(In reply to David Bialer [:dbialer] from comment #10)
> It shouldn't block, but as a side comment, I've tested 8 apps so far (out of
> a random sample of 20).  

which apps?  can you post your results here - or link to the doc?

Comment 12

[Andrew Williamson [:eviljeff]]

ni'ing for:

(In reply to Andrew Williamson [:eviljeff] from comment #11)
> which apps?  can you post your results here - or link to the doc?

Comment 13

[Caitlin Galimidi [:c8o]]

Thomas - 
When you get a sec, what's the status of the Android Apps Assessment?

Comment 14

[Caitlin Galimidi [:c8o]]

Target Date for Android Apps Assessment completion : 10/14/2014

Comment 15

[Thomas Elin [:relaas]]

@Caitlin - The android/desktop content assessment may not resolve the concerns listed in this bug.

What I understand from this bug - There's a quality concern around the apps currently approved for Android. I had proposed reviewing the Android content quality by looking at a very small number of android apps and making sure they fit our minimum review guidelines. 

What is committed for 10/14/14 - A review of the most downloaded Marketplace applications that ARE NOT currently available on Desktop/Android. This project will not verify the quality of applications currently available for Android but rather add additional quality apps from our pool of FFOS only apps.

Perhaps we can touch base about this tomorrow and clarify.

Comment 16

[Caitlin Galimidi [:c8o]]

Team - 

The content team is currently working to raise the quality of experience for Android apps. 

Rather than re-examine apps that pass our minimum quality bar, but don't really highlight or provide a quality experience, the content team is examining each of the Top Most Downloaded FxOS apps and noting ones that perform well and pass the test on Android as well.

Those apps, now known to be both performant and popular, will be called out to Content / Editorial for addition to collections and feed, thereby improving the quality of the Android experience through our content programs.

Comment 17

[Caitlin Galimidi [:c8o]]

Lisa and Andrew - 

It sounds like you need to know, very specifically, if an app currently approved for Android is simply horrible and should not have passed our quality bar.

Do you want bugs filed individually on that?

Comment 18

[Caitlin Galimidi [:c8o]]

Work in progress on review of top FxOS apps
How do they perform on Android? on Desktop?

https://docs.google.com/a/mozilla.com/spreadsheets/d/1Zq3QVkBPlIfmTG2ow6wnBgKWc3O5jUy7Y5oQSMd3H3M/edit?pli=1#gid=0 

Due for completion: 14-Oct

Comment 19

[Caitlin Galimidi [:c8o]]

Thomas - 
Has the review of the most popular FxOS and their suitability for use on Android (and desktop) has been completed?

What are the next steps to review the results and flag apps as available for the platform?
- stakeholders: scott, adora, andrew

Comment 20

[Andrew Williamson [:eviljeff]]

(In reply to Caitlin Galimidi [:c8o] from comment #18)
> Work in progress on review of top FxOS apps
> How do they perform on Android? on Desktop?
> 
> https://docs.google.com/a/mozilla.com/spreadsheets/d/
> 1Zq3QVkBPlIfmTG2ow6wnBgKWc3O5jUy7Y5oQSMd3H3M/edit?pli=1#gid=0 

Note it was a re-review of apps that were previously NOT marked for Android and Desktop, to see if they should be; whereas the original comment of this bug was that apps that were /already/ marked for Android weren't good enough to be.  No results/examples were provided so its difficult to assess next steps here.

Comment 21

[Caitlin Galimidi [:c8o]]

clearing old ni requests