Automated mcom-tests produces FAILS

RESOLVED INVALID

Status

www.mozilla.org
General
P3
normal
RESOLVED INVALID
2 years ago
2 years ago

People

(Reporter: ithompson4, Unassigned, NeedInfo)

Tracking

({reproducible, testcase})

Production
reproducible, testcase

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(1 attachment)

17.92 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document
Details
(Reporter)

Description

2 years ago
Created attachment 8702086 [details]
Automated test Results for MCOM-TESTS 26 December 2015.docx

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0
Build ID: 20151221130713

Steps to reproduce:

1.	Navigate to mozilla/mcom-tests on GitHub  
2.	Fork a copy to your GitHub account (in my case ithompson4/mcom-tests)
3.	Clone the GitHub copy onto a PC running Windows 7 
4.	mkvitualenv to create a  virtual environment on Windows 7 PC within which to run mcom-tests
5.	setprojectdir to create path to GitHub directory on Windows 7 PC containing mcom-tests
6.	cdvirtualenv to link virtual environment to project directory on Windows 7 PC containing mcom-tests
7.	py.test –driver=Firefox   to run the automated tests  (note: the –destructive argument was not recognised when I tried to include this)



Actual results:

The test results include the following FAILS:
FAIL Lib/site-packages/wheel/test/test_basic.py::test_pydist
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_default_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_explicit_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_universal_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_universal_beats_explicit_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_universal_in_setup_cfg
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_pythontag_in_setup_cfg
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_legacy_wheel_section_in_setup_cfg
FAIL Lib/site-packages/wheel/test/test_tool.py::test_keygen



Expected results:

No FAILS should have been produced. A file with the information available from the command page plus the full test results report is attached. 

Note: The "ERRORS" in the test report do not look like real errors, but a summary of the sections of python code tests executed. They have therefore been ignored for the purposes of filing this bug report.
(Reporter)

Updated

2 years ago
Keywords: reproducible, testcase
Priority: -- → P3
Rebecca, would this go to your team (or maybe to Dave?) Random triagebot driveby from me.
Component: Untriaged → General
Flags: needinfo?(rbillings)
Flags: needinfo?(dave.hunt)
Product: Firefox → www.mozilla.org
Version: 43 Branch → Production
This is definitely in the Web QA team wheelhouse. I am unable to debug with the information provided, and am unable to repro- but I'm also working from a different OS, and I know it works with all of them.

I would highly recommend emailing the mozwebqa@mozilla.com mailing list with the info above, or popping into the #mozwebqa IRC channel to get individualized help with this. 

The first question that should be answered is to share what you get from a "pip list" command. I'm wondering if pip is actually installing correctly as the errors relate to the wheel package.
Flags: needinfo?(rbillings) → needinfo?(ithompson4)
It looks like pytest is discovering and running tests from the lib/site-packages/wheel/test/ directory, which is not part of mcom-tests. We're in the process of moving the mcom-tests into the bedrock repository. If you're interested in running the new tests, documentation can be found here: http://bedrock.readthedocs.org/en/latest/testing.html
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Flags: needinfo?(dave.hunt)
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.