Closed Bug 1352128 Opened 7 years ago Closed 7 years ago

Web platform test failures for Windows 10 on TaskCluster

Categories

(Testing :: General, enhancement)

Product:
Testing
Component:
General
Version:
Version 3
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox56 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla56
Tracking Flags:
Tracking Status
firefox56 --- fixed

People

(Reporter: armenzg, Assigned: jmaher)

References

Details

(Whiteboard: [stockwell disabled])

Attachments

(3 files, 1 obsolete file)

enable wpt tests for m-c and try
1.51 KB, patch
grenade
: review+
Details | Diff | Splinter Review
update wpt manifests for win10
335.87 KB, patch
jmaher
: review+
Details | Diff | Splinter Review
windows 10 manifest changes for web-platform-tests
340.47 KB, patch
jgraham
: review+
Details | Diff | Splinter Review 
We have a bunch of web platform test failures for Windows 10 and Windows 7 on TaskCluster. We're trying to green them up:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=253b7b9da0f75142f4650d4fbce6d6ca45597e3e

From jmaher:
Most likely we need to update the expectation files for wpt; typically these are as specific as possible to have os, version, flavor, etc.
For example, this is windows 8 specifically: https://dxr.mozilla.org/mozilla-central/source/testing/web-platform/meta/2dcontext/path-objects/2d.path.arc.selfintersect.1.html.ini#5
You can see windows 10 versions: (version == "10.0.10240") but we updated that to 10.0.14393 recently- a search replace should work, maybe 10.0.* would be more appropriate assuming 10.0.* works
Summary: Web platform test failures for Windows 10 and Windows 7 on TaskCluster → Web platform test failures for Windows 10 on TaskCluster
Depends on: 1360503, 1352137
You can also go the more generic route and drop the OS version check and skip on all win64. I've found more often than not that it's what we often want to do. I find wpt's default annotations to be overly-restrictive.
yes, that is what I will do- bug I expect some tests to pass/fail differently when we get there, so this will be a single chance to wpt vs. a few changes.
Attached patch update wpt manifests for win10 (obsolete) — — Splinter Review 
Assignee: nobody → jmaher
Status: NEW → ASSIGNED

        Attachment #8874504 -
        Flags: review+

    
    

    
  
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/9745b4fb9ac4
Update web-platform-tests expected data. r=jmaher, a=testonly

        Attachment #8874507 -
        Flags: review?(rthijssen) → review+

    
    

    
  


  
removed the one change to conftest.py.ini and verified on try:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=try&originalRevision=5492bf1b91c9&newProject=try&newRevision=fd685976b2ac64e987c73609842525e5d4a841ae&framework=1&showOnlyImportant=0

I will land both patches when I am not in |afk mode.

        Attachment #8874504 -
        Attachment is obsolete: true
Flags: needinfo?(jmaher)

        Attachment #8874750 -
        Flags: review+

    
    

    
  
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/0e098111552f
enabled web-platform-tests for win10 on m-c and try. r=grenade
https://hg.mozilla.org/integration/mozilla-inbound/rev/98c867096a06
Update web-platform-tests expected data. r=jmaher, a=testonly

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/0e098111552f
https://hg.mozilla.org/mozilla-central/rev/98c867096a06
Status: ASSIGNED → RESOLVED
Closed: 7 years ago

          status-firefox55:
          --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55

    
    

    
      
  

    
    

    
      
  

    
    

    
  
joel since this is perma failing shoudn't we just hide them on central till this is fixed ?
Flags: needinfo?(jmaher)

    
    

    
  
oh we should, I haven't been able to get a fix with the toolchain
Flags: needinfo?(jmaher)

    
    

    
  
opt is working, but not debug.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

    
  
Whiteboard: [stockwell disabled]

    
    

    
      
  

    
    

    
      
  

    
    

    
  


  
all changes outside of win10:
$ hg qdiff | grep ^\+ | grep -v \+\+\+ | grep -v 10.0.15063
+
+
+
+
+
+  [contexts.py::test_window_resize]
+    expected:
+      if not debug and not stylo and e10s and (os == "linux") and (version == "Ubuntu 16.04") and (processor == "x86_64") and (bits == 64): FAIL
+      if not debug and not stylo and not e10s and (os == "linux") and (version == "Ubuntu 16.04") and (processor == "x86_64") and (bits == 64): FAIL
+

        Attachment #8879669 -
        Flags: review?(james)

    
    

    
  
Comment on attachment 8879669 [details] [diff] [review]
windows 10 manifest changes for web-platform-tests

Review of attachment 8879669 [details] [diff] [review]:
-----------------------------------------------------------------

The one non-Windows 10 change is a known intermittent that ato is working on, please revert.

        Attachment #8879669 -
        Flags: review?(james) → review+

    
    

    
  
got it, done

    
    

    
  
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/89775519af6b
Web platform test failures for Windows 10 debug on TaskCluster. r=jgraham

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/89775519af6b
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago

          status-firefox56:
          --- → fixed
Resolution: --- → FIXED

          status-firefox55:
          fixed → ---
Target Milestone: mozilla55 → mozilla56

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: