1235227 - Automated mcom-tests produces FAILS

Status: RESOLVED INVALID

(www.mozilla.org :: General, defect, P3)

Description

[ithompson4]

Attachment: Automated test Results for MCOM-TESTS 26 December 2015.docx

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0
Build ID: 20151221130713

Steps to reproduce:

1.	Navigate to mozilla/mcom-tests on GitHub  
2.	Fork a copy to your GitHub account (in my case ithompson4/mcom-tests)
3.	Clone the GitHub copy onto a PC running Windows 7 
4.	mkvitualenv to create a  virtual environment on Windows 7 PC within which to run mcom-tests
5.	setprojectdir to create path to GitHub directory on Windows 7 PC containing mcom-tests
6.	cdvirtualenv to link virtual environment to project directory on Windows 7 PC containing mcom-tests
7.	py.test –driver=Firefox   to run the automated tests  (note: the –destructive argument was not recognised when I tried to include this)



Actual results:

The test results include the following FAILS:
FAIL Lib/site-packages/wheel/test/test_basic.py::test_pydist
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_default_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_explicit_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_universal_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_universal_beats_explicit_tag
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_universal_in_setup_cfg
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_pythontag_in_setup_cfg
FAIL Lib/site-packages/wheel/test/test_tagopt.py::test_legacy_wheel_section_in_setup_cfg
FAIL Lib/site-packages/wheel/test/test_tool.py::test_keygen



Expected results:

No FAILS should have been produced. A file with the information available from the command page plus the full test results report is attached. 

Note: The "ERRORS" in the test report do not look like real errors, but a summary of the sections of python code tests executed. They have therefore been ignored for the purposes of filing this bug report.

Comment 1

[Liz Henry (:lizzard) (relman/hg->git project)]

Rebecca, would this go to your team (or maybe to Dave?) Random triagebot driveby from me.

Comment 2

[Rebecca Billings [:rbillings]]

This is definitely in the Web QA team wheelhouse. I am unable to debug with the information provided, and am unable to repro- but I'm also working from a different OS, and I know it works with all of them.

I would highly recommend emailing the [e-mail] mailing list with the info above, or popping into the #mozwebqa IRC channel to get individualized help with this. 

The first question that should be answered is to share what you get from a "pip list" command. I'm wondering if pip is actually installing correctly as the errors relate to the wheel package.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

It looks like pytest is discovering and running tests from the lib/site-packages/wheel/test/ directory, which is not part of mcom-tests. We're in the process of moving the mcom-tests into the bedrock repository. If you're interested in running the new tests, documentation can be found here: http://bedrock.readthedocs.org/en/latest/testing.html