Created attachment 579824 [details]
Short report in OpenOffice Impress format
(I'm not sure if this the right project; please feel free to move elsewhere if necessary)
This is tracking bug to review data and summary presentation of a review of the use of vendor-specific (moz,webkit,o,ms,khtml) CSS prefixes across the web.
See the attached short summary document (in ODP and PDF formats), or more detailed results in the attachments that will be coming shortly.
Created attachment 579826 [details]
Report in PDF format
Created attachment 579827 [details]
Summary data in text format
Created attachment 579830 [details]
Updated summary with better descriptions of some of the tables
These numbers used a traditional desktop UA for the crawl?
It would be useful to have the same reports for a crawl that pretends to be a mobile browser getting the "mobile version" of sites that have one.
Yes, it was the desktop Firefox 8.0 for the crawl.
I am currently setting up instances to crawl the web using the latest UAs for:
1) iOS Mobile Safari
3) Android Browser
4) Desktop Firefox
In addition to parsing the static .css files, it will also parse inline style sheets and tags in HTML documents, where, anecdotally, more of the mobile-specific CSS can be found.
I took the 'most common properties' section of the report in comment 3 and sorted just the counts where webkit and moz properties are reported to get a list like below that shows for a lot of the top used properties the use of moz-* prefix outnumbers the -webkit-* prefix. there are some exceptions like:
but this is the kind of thing that we want to get at, right? places where use of the -webkit prefix exceeds the -moz prefix, or the use of the moz prefix doesn't register at all... If we first did that for the accumulation of all the data like in the list below, then tried to do that on a more site specific basis, then we have a list of properties that provided the most impact, and a list of the sites that might "just start working" if we make some changes to accept webkit- properties.
On radii, see:
18981 -moz-border-radius-topright - the only value supported in Firefox 3.6 and below; now
we also support the standard border-top-right-radius.
275 -moz-border-top-right-radius - does not and has never existed; presumably someone
copied the Webkit property and just changed the prefix
However, for this property, we need to know which sites specify a webkit-border-radius without the standard version, or with an invalid moz version by mistake. Only then can we know which sites are broken.
The same applies to box-shadow - we implemented this unprefixed in 4.0, so we need to know how many sites are not using the unprefixed version.
webkit-opacity was in Safari 1.1, but has been opacity since then, so it's not surprising that we see very few instances of it. By contrast, we only unprefixed opacity in 4.0.
"transition" is currently unprefixed in neither browser, so we need to know how many sites use the webkit version without using the moz version. It looks like about 5/6 of them use both.
"box-sizing" - everyone else has it unprefixed; but our version is buggy (see bug 243412) so we haven't unprefixed it yet. It seems many sites are using it regardless...
Anyway: I guess what we need is more detailed info on how many sites use certain constructs but not other constructs (and therefore have some chance of starting to work if we support the alternative syntax).
I have put the summary data file on a public URL for download:
Note that it is 620MB compressed, 7.2GB uncompressed
(In reply to Gervase Markham [:gerv] from comment #7)
> webkit-opacity was in Safari 1.1, but has been opacity since then, so it's
> not surprising that we see very few instances of it. By contrast, we only
> unprefixed opacity in 4.0.
opacity has been supported _ un-prefixed _ since Firefox 1.0 (the prefix code was dropped in the Gecko 2.0 timespan, though)
(In reply to John Jensen from comment #8)
> I have put the summary data file on a public URL for download:
> Note that it is 620MB compressed, 7.2GB uncompressed
This file was corrupt; it has now been replaced with a fixed version.
...and has been moved to: http://people.mozilla.com/~jjensen/css.csv.gz
The wiki discussion referencing this ticket can be found at
I've produced some raw datasets suitable for better assessing the issue of vendor-specific prefixes in content presented to mobile browsers. I am in the process of creating some reports on these but thought I would share the raw data for anyone else who is interested.
Data was collected from approximately 1.1m URLs from 18,025 of the top global sites on the internet, using either the latest Android Browser User-Agent string, or that for the latest version of Mobile Safari.
All CSS from separate .css files, from inline <style> tags, or inline style attributes ("<p style='something'>") was parsed.
There are two CSV files, compressed using "xz":
downloaded using the Android ICS Browser User Agent
10.3 GB uncompressed
424 MB compressed
downloaded using the Mobile Safari User Agent
10.4 GB uncompressed
432 MB compressed
The format of the files is as follows. I've tried to remove as much non-essential data to reduce size.
COL TYPE DESCRIPTION
1 VARCHAR(255) domain
2 VARCHAR(255) rule id (based on md5 of domain, url, and selector)
3 VARCHAR(255) CSS property
5 VARCHAR(255) unprefixed name of the property -- eg if col 3 is '-moz-background-color',
this column is 'background-color'
6 BOOLEAN whether this property has one of -moz-, -webkit-, -o-, -ms-, or -khtml- as a vendor prefix
7 BOOLEAN whether this property has been seen prefixed with vendor-specific variants
I'll update this ticket with some more reports once I have them.
Created attachment 596728 [details]
Summary of use of vendor-specific prefixes from CSS collected with Safari and Android Browser UA strings
Please find attached a short text file summarizing the use of vendor-prefixed CSS properties from a sample of pages from the top 18,000 websites globally. It's a short file with content like this:
2427 49.34% -webkit-transition-duration
904 18.38% -moz-transition-duration
753 15.31% -o-transition-duration
442 8.99% -ms-transition-duration
0 0.00% -khtml-transition-duration
393 7.99% unprefixed
This output should be self-explanatory but please let me know if you have any questions.
I've been reviewing your summary file, and I have a few questions:
(In reply to John Jensen from comment #14)
> 4919 transition-duration
Is this number the total number of occurrences across all the CSS files downloaded from the 18,000 sites? Are you able to say what percentage of sites employ each particular construct?
> 2427 49.34% -webkit-transition-duration
> 904 18.38% -moz-transition-duration
> 753 15.31% -o-transition-duration
> 442 8.99% -ms-transition-duration
> 0 0.00% -khtml-transition-duration
> 393 7.99% unprefixed
These percentages appear to be percentages of occurrences, and so don't tell us the vital information about how many or what percentage of sites are using -webkit but not -moz or unprefixed. Are you able to do some analysis to tell us that?
In particular, I'd love to get the data that goes in this table:
: # of sites with a rule using this property, with or without vendor prefix
: # of sites with a rule using ONLY the -webkit- prefix for this property
: # of sites with a rule using the -webkit- prefix AND a bare property
  
animation-duration: 73 65 4
animation-iteration-count: 51 44 3
animation-name: 72 64 4
animation-timing-function: 51 49 2
appearance: 431 386 11
background-size: 504 159 134
border-bottom-left-radius: 1359 641 652
border-bottom-right-radius: 1317 580 643
border-radius: 4022 454 2592
border-top-left-radius: 1353 587 679
border-top-right-radius: 1370 656 647
box-shadow: 2446 355 1888
box-sizing: 447 60 197
font-smoothing: 81 78 0
padding-start: 132 16 0
tap-highlight-color: 300 298 1
text-size-adjust: 779 725 3
transform: 405 225 79
transition-duration: 208 149 42
transition-property: 156 116 26
John: thank you! That's awesome!
I just started some analysis on that data, and I realised I underspecified slightly. Can you confirm that  is "Sites using -webkit- and -bare- but NOT using -moz-"? Or did you not check the presence or absence of "-moz-" for column ?
Secondly, I have now come across a much more complete list of variant CSS properties:
Would you be able to do a similar analysis on all the properties listed there? Here are the column requests for each line in that table:
 # of sites using any listed variant of this property
 # of occurrences of any listed variant of property (these two together give an idea of
 # of sites using one or more (non-moz) prefixed versions but no bare version and no
 # of sites using one or more (non-moz) prefixed versions and a bare version but no
This does, of course, assume that all uses within a site are consistent. I'm not sure what to do if you find that's not the case. Suggestions?
If we can get this data, we should have a very good idea of where we can get the most bang for the buck in terms of implementing properties.
If  is high, then we may be forced to implement/alias the -webkit- (or whatever) property in order to work. If  is high, we can get gains from unprefixing our own implementation, if we have one. Looking at  and , together with our own knowledge of the sort of breakage that missing a property results in, will tell us how much impact we'd have making any fixes for a given property at all.
I will report my simple analysis of the data you've given above in just a moment.
Created attachment 599084 [details]
WebKit CSS - brief broken/working analysis
This spreadsheet uses the data just given by John to calculate what numbers and percentages of sites would not work fully in today's Mobile Firefox.
- It _seems_ that the prevalence of prefixed-CSS-related brokenness, at least in the sample of properties given above and in the sample of sites used, is not that high. text-size-adjust is the property with the most sites using it where we wouldn't respect the CSS rule, and that's only 4% of sites.
- Next in the list are the border-radius properties. There's a big difference here in the % brokennesses for border-radius as opposed to border-*-*-radius. That may be an artifact of the fact that the -moz-prefixed version of this property is actually spelt differently - e.g. -moz-border-radius-bottomright, not -moz-border-bottom-right-radius - and perhaps John's checking only looked for a property spelt the same. We supported the moz-prefixed version up to Firefox 2, though, which is quite a long time ago.
After that, it's "appearance", which does have a moz-prefixed version, although most sites don't use it - but I suspect that might be a good thing, because it seems to me that this particular property is not well standardized, the last attempt being in 2004. So just copying syntax from -webkit- to -moz- probably wouldn't help.
Someone with better knowledge of the CSS landscape can perhaps interpret this data better than me :-)
> Can you confirm that  is "Sites using -webkit- and -bare- but
> NOT using -moz-"? Or did you not check the presence or absence of "-moz-"
> for column ?
I did not check for the presence or absence of -moz- at all. -moz- properties (or -o-, -ms-, etc properties) could (and likely are) included in some of the rules and domains in column .
> Secondly, I have now come across a much more complete list of variant CSS
> Would you be able to do a similar analysis on all the properties listed
Yup, will start it today.
>  # of sites using any listed variant of this property
>  # of occurrences of any listed variant of property (these two together
> give an idea of
>  # of sites using one or more (non-moz) prefixed versions but no bare
> version and no
> -moz-prefixed version
>  # of sites using one or more (non-moz) prefixed versions and a bare
> version but no
> -moz-prefixed version
John: if ,  and  are easier to do in terms of "rules featuring" instead of "occurrences" or "sites" respectively, then that would be fine too.
Doing a possibilities matrix, I've just realised I think we need:
 # of sites using a bare version only (no prefixed versions of any kind)
Then any site not in ,  or  must be using a moz-prefixed version.
(In reply to Gervase Markham [:gerv] from comment #18)
> After that, it's "appearance", which does have a moz-prefixed version,
> although most sites don't use it - but I suspect that might be a good thing,
> because it seems to me that this particular property is not well
> standardized, the last attempt being in 2004. So just copying syntax from
> -webkit- to -moz- probably wouldn't help.
Actually most (or at least a lot) of -webkit-appearance is probably just -webkit-appearance:none to disable theming, which maps exactly to -moz-appearance none.
> Doing a possibilities matrix, I've just realised I think we need:
>  # of sites using a bare version only (no prefixed versions of any kind)
 through  are running now; I'll update the ticket when done.
roc: that's useful to know. I think that when we get the full data set back from John soon, then we'll want to start looking at property-specific solutions in each of the prominent cases, and that will mean we may need to ask more detailed questions about particular properties (e.g. "What percentage of uses of -webkit-appearance have the value 'none'?").
Created attachment 599705 [details]
More detailed summary of vendor-specific prefixes
See the attached .txt file.
It turns out that column #5 is inaccurate for some properties. The other columns are correct. I'll update it ASAP.
John: while you are checking, some other things to look at:
animation-play-state 2 15 1 2 0
This suggests there are 2 sites using this property, with 1 using a non-moz prefix only, and 2 using a non-moz prefix + a bare property. 2 + 1 != 2 :-)
The same issue occurs, more obviously, with box-shadow:
box-shadow 2446 153283 696 1890 0
My analysis is looking interesting; I just need those accurate values for column 5 (they are currently all 0!).
Created attachment 601449 [details]
More detailed summary of vendor-specific prefixes
Sorry for the delay -- a hard drive failure interrupted processing. I've updated the data, see attached.
OK; please find attached my initial analysis. It needs to be taken on from here by someone with a better understanding of the individual CSS properties, their effects and uses, so that they can ask property-specific questions such as "what percentage of uses of "appearance" have the value "none", and so on.
But, looking at the headline figures, it seems to me that the problem of vendor-prefix-only CSS is not yet too widespread in the mobile web. The most prevalent property is "filter", which I guess might be lots of people using -ms-filter to enable PNG transparency or other make-IE-work hacks. The next is "zoom", which is actually implemented unprefixed in WebKit despite being an Apple extension; not sure what people are using that for. After that, the next most popular properties only break us on 4% of sites in the top 18,000.
John: now we've identified some properties of particular interest, it would be good to look at the most commonly-used range of values. In some cases, it may turn out that the property is only used to turn a default feature off, or that we don't have to support the full range for most sites to work.
Can you please develop a script which gives a summary of commonly-used values for a property, both overall and broken down by prefix/no-prefix? And then run it for the top 40 properties in the attached sheet (those with the greatest values for "Sites Not Working")?
Note that we changed the name of the border-radius properties when we unprefixed them, which may be throwing off the analysis and making it look like more sites are broken than actually are. For a sample border radius property, say border-top-right-radius, can you check and see if those sites which appear to be only using prefixes and "not" using moz prefixes, are in fact using the moz-prefixed version which is spelled differently, i.e. in this case -moz-border-radius-topright? Does that change the picture?
Also, can you run exactly the same analysis you just ran, but only over the top 100 mobile sites, and produce an identical sheet? That will give us some idea of whether problems are skewed at the top end (which may be pushing the envelope more), and therefore whether we might be seeing problems more often than these results would suggest.
Created attachment 601566 [details]
CSS vendor-prefixed property analysis (18,000 sites)
> Can you please develop a script which gives a summary of commonly-used values for a property,
> both overall and broken down by prefix/no-prefix? And then run it for the top 40 properties
> in the attached sheet (those with the greatest values for "Sites Not Working")?
> Also, can you run exactly the same analysis you just ran, but only over the top 100 mobile
> sites, and produce an identical sheet?
OK; will take a few days, am behind on a few other projects.
Created attachment 602769 [details]
Value summary of subset of vendor-prefixed properties
> Can you please develop a script which gives a summary of commonly-used values
> for a property,both overall and broken down by prefix/no-prefix? And then run it
> for the top 40 properties in the attached sheet (those with the greatest values
> for "Sites Not Working")?
See attached. Note that all values have been simplified in the following three ways:
1) All numeric values translated to "X". Eg "123px" -> "Xpx"
2) All hex color codes translated to "#XXXX". Eg "#303050" -> "#XXXX"
3) All values between brackets removed. Eg "url(http://this.com)" -> "url()"
Awesome - thanks John :-)
roc: seems like the next step is for you and your team to look at the data and come up with a plan (or more questions).
I think you mean Jet.
To quote from an email I wrote on February 21:
> In particular, the data I've seen so far
> weren't quite measured the right way to answer the question "how
> much will implementing property X with a -webkit- prefix help site
> compatibility?". (In particular, we need to look for how many sites
> have -webkit- prefixed properties and *don't* have equivalent -moz-
> prefixed properties -- and possibly ask the same question again for
> equivalent -moz-prefixed properties or unprefixed properties.)
> Then once we have that data we should check a sample of the sites
> that the data predict will be fixed against a build with the
> prefixed properties to see that the benefit is what we think it is.
So what I'd like to see is:
1. a list of the properties that we ought to support either unprefixed or with -webkit- prefixes
2. for each such property, percentages of mobile sites in the sample that we think each would help (as predicted by the data)
3. for each such property, a small random sample of the sites expected to be fixed that we can actually look at the effects on (with a test build) to know what it is we're fixing, both to check that we're measuring the right thing and to characterize how broken the sites are
(Also, 'filter' is probably most likely the hack to support an 'opacity'-equivalent on old versions of IE.)
Also, to continue from the same email:
> Additionally, the plan I'd like to use for (2) is that if we
> determine something is bad enough that we need to support -webkit-
> for mobile, that we:
> a) add -webkit- prefixed support for mobile only
> b) add unprefixed support everywhere
> but I haven't discussed that much with other Mozilla folks. I think
> we're likely to be able to get some other browsers (likely WebKit
> and Opera) to follow on any unprefixing decisions -- but the more
> and better tests we have, the more likely we are to be able to get
> them to follow.
CCing Jet. Someone take this forward! :-)
The "ought" in dbaron's point 1) is the key question. How do we decide that? We need CSS and web experts to look at the data, the possible effects of support and non-support, the variance between implementations, the likelihood of further change, the effort it would be to support the prefixed version, etc. etc. and then come up with a suggested set of properties to implement which give the most bang for the buck.
We are taking this forward. We know that it's not enough to come with a list of properties to emulate. We're going to prioritize our efforts sorted using a simple equation:
affected_users * pain_level^2
affected_users = (vendor_specific_occurrences - moz_specific_occurrences)
pain_level = (busted_property_multiplier * site_usability)
We square the pain_level to add weight to the relative suffering that a missing property inflicts on affected_users. Using this admittedly subjective formula, we'll find properties that are cosmetic (eg. border-radius) sort after properties that break sites (eg. transform.)
We are working through the W3C to unprefix the worst offenders while we fix bugs in our implementations. At this point, the top of that list includes these properties:
John: please see  and [3} from comment 34. Can we run this analysis on just the properties I listed?
> John: please see  and [3} from comment 34.
> Can we run this analysis on just the properties I listed?
Yes, but sorry, I need a bit more clarity on what that means:
> 2. for each such property, percentages of mobile sites in the sample
> that we think each would help (as predicted by the data)
a) By "mobile site", do you mean "HTML returned to a GET request made with an Android mobile browser UA"? If so, great, that's what I have. If not, please define.
b) Could you/roc define exactly what "that we think would help" means in terms of identifying properties or sites.
> 3. for each such property, a small random sample of the sites expected to be fixed
> that we can actually look at the effects on (with a test build) to know what it
> is we're fixing, both to check that we're measuring the right thing and to characterize
> how broken the sites are
a) I assume this means "produce a tarball of the HTML from the sites identified above". Correct?
(In reply to John Jensen from comment #37)
> > John: please see  and [3} from comment 34.
> > Can we run this analysis on just the properties I listed?
> Yes, but sorry, I need a bit more clarity on what that means:
> > 2. for each such property, percentages of mobile sites in the sample
> > that we think each would help (as predicted by the data)
> a) By "mobile site", do you mean "HTML returned to a GET request made with
> an Android mobile browser UA"? If so, great, that's what I have. If not,
> please define.
Sounds good enough.
> b) Could you/roc define exactly what "that we think would help" means in
> terms of identifying properties or sites.
I think measuring whether implementing something unprefixed would help involves, for properties we currently implement with -moz-prefixes, looking at the number of sites that use an unprefixed form of that property without the -moz-prefixed form.
I think measuring whether implementing a -webkit- prefix would help involves looking at, for properties we currently implement either with -moz- prefixes or unprefixed, looking at the number of sites that use a -webkit-prefixed form of that property without the unprefixed form or the -moz-prefixed form.
That said, you've had a lot of discussion above about what to measure; perhaps you have better ideas based on that discussion?
> > 3. for each such property, a small random sample of the sites expected to be fixed
> > that we can actually look at the effects on (with a test build) to know what it
> > is we're fixing, both to check that we're measuring the right thing and to characterize
> > how broken the sites are
> a) I assume this means "produce a tarball of the HTML from the sites
> identified above". Correct?
I think URLs would be better.
Created attachment 608547 [details]
ODS Spreadsheet summarizing per-domain and per-property prefixed and unprefixed statistics
Gerv, Jet, dbaron, please see attached spreadsheet in OpenDocument format. The meat of it is as below:
base_property num_domains num_rules pct_no_unprefixed_and_no_moz
animation-count 1 1 100.0
animation-delay 5 137 80.0
animation-direction 8 10 62.5
animation-duration 73 324 87.9
animation-fill-mode 2 3 50.0
animation-iteration-count 51 78 84.7
animation-name 72 756 87.6
animation-play-state 2 3 0.0
animation-timing-function 51 100 94.5
text-size-adjust 779 6352 99.5
transform-origin 68 196 56.9
transform-origin-y 2 3 0.0
transform-style 35 50 100.0
transition-delay 19 53 63.2
transition-duration 208 853 71.5
transition-property 156 491 76.2
transition-timing 1 2 0.0
transition-timing-function 45 111 58.9
Sample explanation for text-size adjust:
A total of 779 of 24,510 domains had at least one rule that included “text-size-adjust”.
There were a total of 6,352 such rules across all the 24,510 domains surveyed.
Of the 779 domains, on average, 99.5% CSS rules that had a “text-size-adjust” property did not also include an unprefixed or -moz- version.
Just wanted to add that we have also 2 non-standard, but implemented unprefixed (in WebKit and MS), to consider: background-position-x and background-position-y.
It is using to achieve spriting images and when it is used, non-supported browsers often render the page in an unusable way.
According this http://lists.w3.org/Archives/Public/www-style/2012Feb/0623.html , it breaks several mobile web sites.
In that case a quick specification (in CSS4 B&B) and an implementation (even prefixed) would help.
Though slightly different for what we are tracking here, this is also something we need to track.
There's a been a lot of analysis on this bug, but the end of when the report is "finished" is not clear. What's left to closeout this bug?
We can probably close this now!