Closed Bug 1312016 Opened 4 years ago Closed 3 years ago

Implement highlights ranking/scoring

Categories

(Firefox for Android :: General, defect, P1)

All
Android
defect

Tracking

()

RESOLVED FIXED
Firefox 54
Iteration:
1.17
Tracking Status
firefox54 --- fixed

People

(Reporter: sebastian, Assigned: sebastian)

References

(Depends on 3 open bugs)

Details

(Whiteboard: [MobileAS])

Attachments

(1 file, 1 obsolete file)

The desktop add-on for Activity Stream implemented a scoring algorithm on top of the highlights query:
https://github.com/mozilla/activity-stream/blob/0a91220a2a14cdbbcc28d3741c2875375e1e6deb/common/recommender/Baseline.js#L162
Priority: -- → P3
Whiteboard: [MobileAS]
Priority: P3 → P2
Iteration: --- → 1.13
Priority: P2 → P1
Assignee: nobody → s.kaspari
Status: NEW → ASSIGNED
Depends on: 1330973
Iteration: 1.13 → 1.14
Duplicate of this bug: 1318549
Depends on: 1335817
Depends on: 1335819
Depends on: 1335824
Depends on: 1336037
Depends on: 1336040
Attachment #8830690 - Attachment is obsolete: true
Okay. So this is is a first implementation that seems to generate much better results than what we have right now (but we will see when it's in Nightly and it works with "real" browsing history). There are a bunch of things to improve and add - I filed follow-up bugs for them. Overall this seems to be good enough to test it in the wild before working on the missing parts.

I considered storing the highlights in the database (at least temporarily) but for now everything is fast enough so that this is not needed. But it's something to consider as the algorithm gets more complex. I re-added the histogram, so let's see how it performs on our small test group.
See Also: → 1337031
Iteration: 1.14 → 1.15
Comment on attachment 8830691 [details]
Bug 1312016 - Activity Stream: Implement highlights ranking to mirror desktop add-on behavior.

https://reviewboard.mozilla.org/r/107426/#review111746

Apologies for taking so long, it was hard to get to this. I'll do another pass in the Morning!

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:39
(Diff revision 3)
> +            FEATURE_BOOKMARK_AGE_IN_MILLISECONDS, FEATURE_DESCRIPTION_LENGTH, FEATURE_PATH_LENGTH,
> +            FEATURE_QUERY_LENGTH, FEATURE_IMAGE_SIZE})
> +    public @interface Feature {}
> +
> +    private Highlight highlight;
> +    private Map<String, Double> features;

final

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:41
(Diff revision 3)
> +    public @interface Feature {}
> +
> +    private Highlight highlight;
> +    private Map<String, Double> features;
> +    private String host;
> +    private String imageUrl;

This might be `null`. Perhaps annotate it w/ `@Nullable`?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:66
(Diff revision 3)
> +     * Extract and assign features that will be used for ranking.
> +     */
> +    private static void extractFeatures(HighlightCandidate candidate, Cursor cursor) {
> +        candidate.features.put(
> +                FEATURE_AGE_IN_DAYS,
> +                System.currentTimeMillis() - cursor.getDouble(cursor.getColumnIndexOrThrow(BrowserContract.History.DATE_LAST_VISITED))

You seem to be doing:
`nowMillis - ageMillis/millisInADay`

But you probably intended to do:
`(nowMillis - ageMillis) / millisInADay`

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:90
(Diff revision 3)
> +        // The desktop add-on used the number of images returned form Embed.ly here. This is not the
> +        // same as total images on the page (think of small icons or the famous spacer.gif). So for
> +        // now this value will only be 1 or 0 depending on whether we found a good image. The desktop
> +        // team will face the same issue when switching from Embed.ly to the metadata-page-parser.
> +        // At this point we can try to find a fathom rule for determining a good value here.
> +        candidate.features.put(

It's not obvious to me that more than one good image is a useful heuristic here - unless we're doing some kind of a carousel thingy - but OK.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:105
(Diff revision 3)
> +
> +        // This value is not really the time at which the bookmark was created by the user. Especially
> +        // synchronized bookmarks can have a recent value but have been bookmarked a long time ago.
> +        // But we are sourcing highlights from the recent visited history - so in order to show up
> +        // this bookmark need to have been visited recently too.
> +        final int bookmarkDateColumnIndex = cursor.getColumnIndexOrThrow(BrowserContract.Bookmarks.DATE_CREATED);

Bug 1335198 is relevant here, might be useful to mention it in the comment.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:162
(Diff revision 3)
> +
> +    /* package-private */ Highlight getHighlight() {
> +        return highlight;
> +    }
> +
> +    /* package-private */ double getFeature(@Feature String feature) {

So `Feature` here means `feature name` but also `feature value`... Maybe `getFeatureValue`? Not sure if you'll care to be so verbose.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:163
(Diff revision 3)
> +    /* package-private */ Highlight getHighlight() {
> +        return highlight;
> +    }
> +
> +    /* package-private */ double getFeature(@Feature String feature) {
> +        return features.get(feature);

Can we ever be missing a feature from `features`? if not, perhaps throw if this gets a `null`, to fail early?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:169
(Diff revision 3)
> +    }
> +
> +    /* package-private */ Map<String, Double> getFilteredFeatures(RankingUtils.Func1<String, Boolean> filter) {
> +        Map<String, Double> filteredFeatures = new HashMap<>();
> +
> +        for (Map.Entry<String, Double> entry : features.entrySet()) {

Why re-implement RankingUtils.filter? Can't you do something like

`return RankingUtils.filter(new HashMap(features), filterFunction))`?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:178
(Diff revision 3)
> +        }
> +
> +        return filteredFeatures;
> +    }
> +
> +    /* package-private */ void normalize(@Feature String feature, double min, double max) {

This doesn't seem like a particularly good method name. It has side-effects, but the name implies it's pure and might even return something.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:182
(Diff revision 3)
> +
> +    /* package-private */ void normalize(@Feature String feature, double min, double max) {
> +        features.put(feature, normalize(features.get(feature), min, max));
> +    }
> +
> +    private double normalize(double value, double min, double max) {

Seems like a static util method.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightCandidate.java:183
(Diff revision 3)
> +    /* package-private */ void normalize(@Feature String feature, double min, double max) {
> +        features.put(feature, normalize(features.get(feature), min, max));
> +    }
> +
> +    private double normalize(double value, double min, double max) {
> +        if (max > min) {

Is there a valid reason to have min > max here? If it's only result of a programming error (and not bad data that we need to deal with), we should throw here.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:35
(Diff revision 3)
> +import static org.mozilla.gecko.activitystream.ranking.RankingUtils.map;
> +import static org.mozilla.gecko.activitystream.ranking.RankingUtils.mapCursor;
> +import static org.mozilla.gecko.activitystream.ranking.RankingUtils.mapWithLimit;
> +import static org.mozilla.gecko.activitystream.ranking.RankingUtils.reduce;
> +
> +public class HighlightsRanking {

Perhaps explain on a high level how ranking works as a class comment?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:73
(Diff revision 3)
> +
> +        scoreEntries(highlights);
> +
> +        sortDescendingByScore(highlights);
> +
> +        filterOutItemsWithNoScore(highlights);

this doesn't seem to take advantage of the fact that list is sorted, so why not do it before sorting?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:111
(Diff revision 3)
> +            @Override
> +            public void call(HighlightCandidate candidate, String feature) {
> +                double[] minMaxForFeature = minMaxValues.get(feature);
> +
> +                if (minMaxForFeature == null) {
> +                    minMaxForFeature = new double[] { 1, 0 };

Maybe it's wednesday evening talking, but this whole transformation a bit hard to read.

iiuc:
values: [3, 5, 15, 75] will produce min/max: [1, 75]

Which doesn't seem correct?

How about doing something super straightforward:
(I miss python generators...)
`
Map featureValues <string, double[]>;
for each normalizationFeature:
 for each candidate:
  featureValues[feature].add(candidate.getFeature(feature))

// (your apply2D might help here? ^^)

Map featureMinMax <string, double[]>;
for feature, values in featureValues:
 featureMinMax.put(feature, [Collections.min(values), Collections.max(values)])
`

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:116
(Diff revision 3)
> +                    minMaxForFeature = new double[] { 1, 0 };
> +                    minMaxValues.put(feature, minMaxForFeature);
> +                }
> +
> +                minMaxForFeature[0] = Math.min(minMaxForFeature[0], candidate.getFeature(feature));
> +                minMaxForFeature[1] = Math.max(minMaxForFeature[1], candidate.getFeature(feature));

Did you forget to put it to minMaxValues?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:126
(Diff revision 3)
> +        apply2D(candidates, NORMALIZATION_FEATURES, new Action2<HighlightCandidate, String>() {
> +            @Override
> +            public void call(HighlightCandidate candidate, String feature) {
> +                double[] minMaxForFeature = minMaxValues.get(feature);
> +
> +                candidate.normalize(feature, minMaxForFeature[0], minMaxForFeature[1]);

For simplicity's sake, this should really be something like `candidate.setFeatureValue(feature, Utils.normalize(feature, min, max))`...

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:138
(Diff revision 3)
> +     */
> +    @VisibleForTesting static void scoreEntries(List<HighlightCandidate> highlights) {
> +        apply(highlights, new Action1<HighlightCandidate>() {
> +            @Override
> +            public void call(HighlightCandidate candidate) {
> +                final Map<String, Double> featuresForWeighting = candidate.getFilteredFeatures(new Func1<String, Boolean>() {

So, weight every feature which isn't part of `adjustment_features` list?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:227
(Diff revision 3)
> +        final Set<String> knownHosts = new HashSet<String>();
> +
> +        filter(candidates, new Func1<HighlightCandidate, Boolean>() {
> +            @Override
> +            public Boolean call(HighlightCandidate candidate) {
> +                if (knownHosts.contains(candidate.getHost())) {

knownHosts is a set, so you could just do
`return !knownHosts.add(candidate.getHost())`

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:249
(Diff revision 3)
> +                return candidate.getHighlight();
> +            }
> +        }, limit);
> +    }
> +
> +    private static double decay(double initialScore, Map<String, Double> features, final Map<String, Double> weights) {

I feel like this needs a sample spreadsheet with visualizations of how this function plays out with some sample data.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:255
(Diff revision 3)
> +        if (features.size() != weights.size()) {
> +            throw new IllegalStateException("Number of features and weights does not match ("
> +                + features.size() + " != " + weights.size());
> +        }
> +
> +        double exp = reduce(features.entrySet(), new Func2<Map.Entry<String, Double>, Double, Double>() {

Maybe name `exp` something like `weightedAverage` for clarity?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:262
(Diff revision 3)
> +            public Double call(Map.Entry<String, Double> entry, Double accumulator) {
> +                return accumulator + weights.get(entry.getKey()) * entry.getValue();
> +            }
> +        }, 0d);
> +
> +        return initialScore * Math.pow(Math.E, -exp);

Math.exp(-exp)

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:276
(Diff revision 3)
> +        // could consider just lowering the score significantly because we support displaying
> +        // highlights without images too. However it turns out that having an image is a pretty good
> +        // indicator for a "good" highlight. So completely ignoring items without images is a good
> +        // strategy for now.
> +        if (candidate.getFeature(HighlightCandidate.FEATURE_IMAGE_COUNT) == 0) {
> +            newScore = 0;

I don't like this.

How much "highlight" images matter seems to depend on one's browsing habits... If I'm reading a lot of text-heavy pages, blogs, online books, or whatever else, images will often be irrelevant and/or missing.

This approach seems to punish certain segments of the web, particularly that precious part of the web which doesn't optimize for being linked to from a facebook post or a tweet.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/HighlightsRanking.java:284
(Diff revision 3)
> +        if (candidate.getFeature(HighlightCandidate.FEATURE_PATH_LENGTH) == 0
> +                || candidate.getFeature(HighlightCandidate.FEATURE_DESCRIPTION_LENGTH) == 0) {
> +            newScore *= 0.2;
> +        }
> +
> +        // TODO: Consider adding a penalty for items without an icon or with a low quality icon (Bug 1335824).

See my comment regarding lack of highlight images above...

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/RankingUtils.java:19
(Diff revision 3)
> +import java.util.Iterator;
> +import java.util.List;
> +
> +/**
> + * Some helper methods that make processing lists in a pipeline easier. This wouldn't be needed with
> + * Java 8 streams or RxJava. But we haven't anything like that available.

Oh boy, our very own semi-FP shim class.
Calling this RankingLodash would have been funnier ;-)

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/RankingUtils.java:21
(Diff revision 3)
> +
> +/**
> + * Some helper methods that make processing lists in a pipeline easier. This wouldn't be needed with
> + * Java 8 streams or RxJava. But we haven't anything like that available.
> + */
> +/* package-private */ class RankingUtils {

This really needs tests.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/RankingUtils.java:51
(Diff revision 3)
> +    interface Func2<T1, T2, R> {
> +        R call(T1 t, T2 a);
> +    }
> +
> +    /**
> +     * Filter a list of items. Items for which the provided function returns true are removed from

comment is wrong, or code is wrong (but i like code's version better): items are removed if function returns false.

or in other words, this keeps every item for which filter function returns true.

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/RankingUtils.java:57
(Diff revision 3)
> +     * the list. This method will modify the list in place and not create a copy.
> +     */
> +    static <T> void filter(List<T> items, Func1<T, Boolean> filter) {
> +        final Iterator<T> iterator = items.iterator();
> +
> +        while (iterator.hasNext()) {

I take it we trust that iterator is not going to change while we're iterating through it?

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/RankingUtils.java:149
(Diff revision 3)
> +        }
> +
> +        return newItems;
> +    }
> +
> +    //

cleanup

::: mobile/android/base/java/org/mozilla/gecko/activitystream/ranking/RankingUtils.java:175
(Diff revision 3)
> +
> +    /**
> +     * Transform a cursor into a list of items by calling a function for every cursor position to
> +     * return one item.
> +     */
> +    static <T> List<T> mapCursor(Cursor cursor, Func1<Cursor, T> func) {

Might be either worth fixing or at least noting in a comment that this method has a funny side effect of actually modifying the cursor you give it (changing its position).

I'd vote for "fix it", but then again, cursors seem to be fair game in our code base once passed anywhere :)

::: mobile/android/base/java/org/mozilla/gecko/db/BrowserProvider.java:1201
(Diff revision 3)
> -                DBUtils.qualifyColumn(Bookmarks.TABLE_NAME, Bookmarks.POSITION) + " AS " + Bookmarks.POSITION + ", " +
> -                DBUtils.qualifyColumn(History.TABLE_NAME, History.URL) + ", " +
> -                DBUtils.qualifyColumn(History.TABLE_NAME, History.TITLE) + ", " +
> -                DBUtils.qualifyColumn(History.TABLE_NAME, History.DATE_LAST_VISITED) + " AS " + Highlights.DATE + ", " +
> -                DBUtils.qualifyColumn(PageMetadata.TABLE_NAME, PageMetadata.JSON) + " AS " + Highlights.METADATA + " " +
>                  "FROM " + History.TABLE_NAME + " " +

So we're explicitely ignoring bookmarks without associated history records, correct?

::: mobile/android/base/java/org/mozilla/gecko/db/LocalBrowserDB.java:1977
(Diff revision 3)
>  
>          return new MergeCursor(new Cursor[] {topSitesCursor, blanksCursor});
>      }
>  
>      @Override
> -    public CursorLoader getHighlights(Context context, int limit) {
> +    public Cursor getHighlightCandidates(ContentResolver contentResolver, int limit) {

.query may return `null`, so perhaps tell callers to deal with it via @Nullable?

::: mobile/android/base/java/org/mozilla/gecko/db/LocalBrowserDB.java:1982
(Diff revision 3)
> -    public CursorLoader getHighlights(Context context, int limit) {
> -        final Uri uri = mHighlightsUriWithProfile.buildUpon()
> +    public Cursor getHighlightCandidates(ContentResolver contentResolver, int limit) {
> +        final Uri uri = mHighlightCandidatesUriWithProfile.buildUpon()
>                  .appendQueryParameter(BrowserContract.PARAM_LIMIT, String.valueOf(limit))
>                  .build();
>  
> -        return new TelemetrisedCursorLoader(context, uri, null, null, null, null,
> +        return contentResolver.query(uri, null, null, null, null, null);

Any particular reason you've de-instrumented this query? Keeping an eye out on time-to-load still seems useful.

::: mobile/android/base/java/org/mozilla/gecko/home/activitystream/model/Metadata.java:48
(Diff revision 3)
>  
>      public boolean hasProvider() {
>          return !TextUtils.isEmpty(provider);
>      }
>  
> +    public String getImageUrl() {

@Nullable
Comment on attachment 8830691 [details]
Bug 1312016 - Activity Stream: Implement highlights ranking to mirror desktop add-on behavior.

https://reviewboard.mozilla.org/r/107426/#review111746

> Why re-implement RankingUtils.filter? Can't you do something like
> 
> `return RankingUtils.filter(new HashMap(features), filterFunction))`?

The difference is that RankingUtils.filter modifies the list in-place while we need a "copy" here.

> Is there a valid reason to have min > max here? If it's only result of a programming error (and not bad data that we need to deal with), we should throw here.

Yeah, that's the initial value (min=1, max=0) and will be the case if all feature values are the same (or do not exist).

> Did you forget to put it to minMaxValues?

I'm updating the array object in-place.

> So, weight every feature which isn't part of `adjustment_features` list?

yep!

> Any particular reason you've de-instrumented this query? Keeping an eye out on time-to-load still seems useful.

I wrapped this around the whole loader: db + ranking.
Added some first tests. More to come in bug 1336040.
Comment on attachment 8830691 [details]
Bug 1312016 - Activity Stream: Implement highlights ranking to mirror desktop add-on behavior.

https://reviewboard.mozilla.org/r/107426/#review114450

Looking great! With a caveat that 1) i haven't tried the patch yet, and 2) I've only glanced through tests.

::: mobile/android/base/java/org/mozilla/gecko/db/BrowserProvider.java:1210
(Diff revision 5)
> -                "AND " + DBUtils.qualifyColumn(History.TABLE_NAME, History.TITLE) + " NOT NULL AND " + DBUtils.qualifyColumn(History.TABLE_NAME, History.TITLE) + " != '' " +
> -                "AND " + DBUtils.qualifyColumn(History.TABLE_NAME, History.IS_DELETED) + " = 0 " +
> -                "AND " + DBUtils.qualifyColumn(History.TABLE_NAME, History.URL) + " NOT IN (SELECT " + ActivityStreamBlocklist.URL + " FROM " + ActivityStreamBlocklist.TABLE_NAME + " )" +
> -                // TODO: Implement domain black list (bug 1298786)
> -                // TODO: Group by host (bug 1298785)
>                  "ORDER BY " + DBUtils.qualifyColumn(History.TABLE_NAME, History.DATE_LAST_VISITED) + " DESC " +

Not sure about this, but maybe it makes sense to order by `Highlights.DATE` instead, to account for the case statement above?

::: mobile/android/base/java/org/mozilla/gecko/db/BrowserProvider.java:1211
(Diff revision 5)
> -                "AND " + DBUtils.qualifyColumn(History.TABLE_NAME, History.IS_DELETED) + " = 0 " +
> -                "AND " + DBUtils.qualifyColumn(History.TABLE_NAME, History.URL) + " NOT IN (SELECT " + ActivityStreamBlocklist.URL + " FROM " + ActivityStreamBlocklist.TABLE_NAME + " )" +
> -                // TODO: Implement domain black list (bug 1298786)
> -                // TODO: Group by host (bug 1298785)
>                  "ORDER BY " + DBUtils.qualifyColumn(History.TABLE_NAME, History.DATE_LAST_VISITED) + " DESC " +
> -                "LIMIT " + historyLimit + ")";
> +                "LIMIT " + limit;

Currently the limit used is 500, right? I suppose a downside to this is that we'll have (wildly) varying results for users who browse a lot vs. those who do not.

iirc, we don't really do anything with dates afterwards (other than sorting).

Do you feel this might be a problem? On a phone-heavy day, I might make something like 100-200 visits, so for me 500 is roughly "over the past few days", whereas for a very heavy phone user it might be "today and maybe yesterday".

For an infrequent user, this might mean we'll be showing highlights from a long time ago (i suppose ranking is supposed to take care of that currently to an extent?).

maybe limit by time?
```
where date < 1.week.ago &&
order by date &&
limit 500
```
but then, picking a good time interval is tricky.

Perhaps the current approach of "simply get bunch of recent history items and let ranking do all of the heavy lifting" is the way to go. It might be worth considering _increasing_ the limit to something larger to account for a wider variety of usage patterns.

::: mobile/android/base/java/org/mozilla/gecko/home/activitystream/ActivityStream.java:40
(Diff revision 5)
>      private static final int LOADER_ID_TOPSITES = 1;
>  
> +    /**
> +     * Number of database entries to consider and rank for finding highlights.
> +     */
> +    private static final int HIGHLIGHTS_CANDIDATES = 500;

As I said in the comment for the query, perhaps consider increasing this limit?

::: mobile/android/base/java/org/mozilla/gecko/home/activitystream/StreamRecyclerAdapter.java:149
(Diff revision 5)
> -        onUrlOpenListener.onUrlOpen(url, EnumSet.of(HomePager.OnUrlOpenListener.Flags.ALLOW_SWITCH_TO_TAB));
> +        onUrlOpenListener.onUrlOpen(highlight.getUrl(), EnumSet.of(HomePager.OnUrlOpenListener.Flags.ALLOW_SWITCH_TO_TAB));
>      }
>  
>      @Override
>      public int getItemCount() {
> -        final int highlightsCount;
> +        return highlights.size() + 3;

Would be nice to have a description of what "3" is (and we use it elsewhere in the class as well).
Attachment #8830691 - Flags: review?(gkruglov) → review+
Iteration: 1.15 → 1.16
Pushed by s.kaspari@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/69f111a76667
Activity Stream: Implement highlights ranking to mirror desktop add-on behavior. r=Grisha
https://hg.mozilla.org/mozilla-central/rev/69f111a76667
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 54
Iteration: 1.16 → 1.17
Depends on: 1353001
Depends on: 1367024
Depends on: 1369604
You need to log in before you can comment on or make changes to this bug.