1259775 - Running web-platform-tests-create adds lots of unrelated gunk to manifest again

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect)

Description

[Boris Zbarsky [:bzbarsky]]

Attachment: test.patch

I just ran 

  mach web-platform-tests-create testing/web-platform/tests/html/webappapis/scripting/processing-model-2/window-onerror-with-cross-frame-event-listeners-1.html

I'm attaching the resulting diff for testing/web-platform/meta/MANIFEST.json.  What happened here, exactly?  Should those entries be in there and were incorrectly missing (and if so why)?  Should they not be there and were incorrectly added?

Comment 1

[Boris Zbarsky [:bzbarsky]]

So I took a look at one of the added things: the entry for the "2dcontext/building-paths/canvas_complexshapes_arcto_001.htm" reftest.  It's present around line 3000 of the file and again around line 34800, the latter under "local_changes"....  The latter entry is the one the mach command added.

Oh, and just to make this easier to reproduce, I was running on top of inbound revision 4907a0296ff9.

Comment 2

[James Graham [:jgraham]]

As discussed on irc, the problem is that when you add a reftest the manifest update code copies all the reftest data into locla changes rather than diffing. It seems like someone added such a test and updated the manifest by hand, so now automatic additions are adding gunk. There are then two parts to the solution; update the checked-in manifest to be correct per the current tool behaviour and update the behaviour of the tool to be smarter and copy less data.

Comment 3

[Jeremy Chen ([e-mail])]

I've had this problem in Bug 1247777 a while ago.

STR could be simplified as follows:

1. add any reftest files
2. run "./mach web-platform-tests --manifest-update"

I've filed https://github.com/w3c/wpt-tools/issues/54 for this, but not sure if anyone would take care of that. Maybe file it here would be more apporpriate. Since I'm adding wpt reftest recently, my work around is comment the following line while updating manifest automatically:

https://dxr.mozilla.org/mozilla-central/rev/f5a97eb5c89a2b5133ff8bba915acba6344fa7cc/testing/web-platform/tests/tools/manifest/manifest.py#166

Comment 4

[Brian Birtles (:birtles, Feb 26~ parental leave)]

I'm encountering this every time I run --manifest-update even when not adding reftest files -- in fact, even when not adding any files at all.

Comment 6

[Jeremy Chen ([e-mail])]

This bug has been fixed for a while. See https://github.com/w3c/wpt-tools/pull/55 and https://github.com/w3c/wpt-tools/pull/61 for more detail.