528240 - Instructions for tracking external download sources needs more detail

Status: RESOLVED FIXED

(addons.mozilla.org Graveyard :: Statistics, enhancement, P5)

Description

[Daniel Einspanjer [:dre] [:deinspanjer]]

The instructions don't indicate the limitations which could cause add-on developers frustration if they run a campaign and get back bad or non-useful data.
http://screencast.com/t/MDBlYjJjNT

The limit in the metrics processing of the src parameter is 70 characters.  Currently, if the string is more than 70 characters, it is replaced with the string "invalid".  If necessary, we could change it to do truncation instead.

Only URL safe characters are allowed. This includes the unreserved set of characters from RFC3986 section 2.3 (alphanumeric - . _ ~) and the reserved characters % and +. Percent encoded strings are not decoded and fully count against the 70 character limit.

If any invalid characters are found in the string, the value is replaced with the string "invalid".

Note that since "external-" is part of the value, the effective limit of customizable strings the add-on developer can use is 61 (e.g. &src=external-this_is-an~absurdly~%20long+example.with_all_allowed_chars%21 )

Comment 1

[Justin Scott [:fligtar]]

I wasn't aware of these limitations either. I'm thinking in these cases we should truncate or remove the invalid characters instead of changing it to invalid.

Comment 2

[Daniel Einspanjer [:dre] [:deinspanjer]]

The only reason I didn't default to truncation is because the chances are much greater that if the string is over 70 characters, it is the result of a bogus request (i.e. a very long trash string) rather than a legitimate attempt to have a tracking string that long.  If I don't replace it with the "invalid" marker, I have to keep all those bogus strings in my data warehouse which could greatly increase the cardinality of that field and hence increase the storage requirements and decrease the query performance.
If you guys can't live with the possibility that someone out there might want to be using long prose (or dual GUIDs) for tracking values, and you are okay with the fact that there could be value collisions if we implement truncation, then file a metrics feature request for it.  I'd encourage you to poll your users first though.


I'd really rather not deal with sanitizing characters though.  It will slow down my processing to have to decide character by character whether to include it or not.  Note that if there are invalid characters in the value, again, it is much more likely that this is a bogus request with noise/trash characters in the request rather than an intentional decision by an add-on developer to disregarded protocol and spec by not URL encoding them in the first place.

Comment 3

[Wil Clouser [:clouserw]]

-> fligtar for decisions and/or wording

Comment 4

[Wil Clouser [:clouserw]]

kicking to 5.5.  Still need a decision here.

Comment 5

[Justin Scott [:fligtar]]

Will fix this as part of the new stats dashboard in bug 543548.

Comment 6

[Pratibha]

Attachment: screenshot

Tracking external sources information is as follows:

- Load https://addons-dev.allizom.org/en-US/firefox/addon/adblock-plus/statistics/downloads/sources/?last=30
- Click on the link "About tracking external sources..."

pop-up shows the following information:

If you link to your add-on's details page or directly to its file from an external site, such as your blog or website, you can append a parameter to be tracked as an additional download source on this page. For example, the following links would appear as sourced by your blog:

Add-on Details Page
    https://addons.mozilla.org/addon/adblock-plus?src=external-blog
Direct File Link
    https://addons.mozilla.org/downloads/latest/1865/addon-1865-latest.xpi?src=external-blog

Only src parameters that begin with "external-" will be tracked, up to 61 additional characters. Any text after "external-" can be used to describe the source, such as "external-blog", "external-sidebar", "external-campaign225", etc. The following URL-safe characters are allowed: a-z A-Z - . _ ~ % + 

see attached screenshot.jpg

Note: 61 character limit and URL-safe characters are included in the information.Pls update if this makes the issue Fixed.

Comment 7

[Wil Clouser [:clouserw]]

closing according to comment 6