1021634 - [Moztrap] Add new environment to the latest product version, only partial cases apply it

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: MozTrap, defect, P1)

Description

[Mike Lien[:mlien]]

Add new environment to Firefox OS 2.0 and create a test run.
Only partial cases apply new environment.
Total 6500+ cases, test run only show 4000+ cases with new environment.

Comment 1

[Cameron Dawson [:camd]]

Mike - Can you point me at the specific test run?  I want to be sure I'm investigating what you're seeing.  I have an idea about what may be happening.  thanks.

Comment 2

[Mike Lien[:mlien]]

(In reply to Cameron Dawson [:camd] from comment #1)
> Mike - Can you point me at the specific test run?  I want to be sure I'm
> investigating what you're seeing.  I have an idea about what may be
> happening.  thanks.

product:  Firefox OS
name: test create a run 2

Comment 3

[Cameron Dawson [:camd]]

Thanks.  I believe I understand at least part of the problem here.  I am working to identify the full scope of the bug and identify a fix.

Thanks for the info.

Comment 4

[Cameron Dawson [:camd]]

Hey :mbrandt -- cc'ing you on this bug because I think it's high priority.  I would consider it "critical" but I'll let you make that final call.

I would like to nominate this as a high priority work item on our next moztrap dev push.

Comment 5

[Peter Bengtsson [:peterbe]]

I've wrapped it all in a transaction. 
Now, calling the `productversion = form.save_if_valid()` takes, on my laptop, 313 seconds!
It seems that these lines [0] take 276 seconds. 
I'm not sure where the other (313-276) 38 seconds are spent!

I measured every SQL command for doing a single add productversion for "Firefox" which causes it to do clones for 755 caseversions. 
https://gist.github.com/peterbe/23736e5ccd04f419404f
Basically, the most time is spent doing a select statement!

After some investigation, it seems that 99% of the time is spent in this block of code:
https://github.com/mozilla/moztrap/blob/8acd8273882e9285a956b7c5492e41ab124f4cdc/moztrap/model/mtmodel.py#L223-L282
Yep, that's the deepest and scariest black nook of Moztrap. 

[0] https://github.com/mozilla/moztrap/blob/8acd8273882e9285a956b7c5492e41ab124f4cdc/moztrap/view/manage/productversions/forms.py#L94-L95

Comment 6

[Peter Bengtsson [:peterbe]]

One option would be to NOT use `cvinstance.clone(...)` but to instead do, manually, something like this:

```
for cvinstance in case_versions_to_clone:
  cv = CaseVersion(thing=cvinstance.thing, other=cvinstance.other, ... etc)
  cvs.append(cv)
...

CaseVersion.objects.bulk_create(cvs)
```
However, we still need to solve how to do all the ManyToManyFields for those with a manual clone too.

Comment 7

[Matt Brandt [:mbrandt]]

+1 on critical for the teams workflows -- thank you for the jumping on this one.

Comment 8

[Peter Bengtsson [:peterbe]]

This'll make it tighter and less likely to get hung half-way. 
https://github.com/mozilla/moztrap/pull/71

We still have some severe optimization problems of things taking minutes to complete.

Comment 9

[Stephen Donner [:stephend] Not actively reading bugmail]

Mass-closing remaining MozTrap bugs as WONTFIX, due to 1) the Mozilla-hosted instance being decommissioned (see https://wiki.mozilla.org/TestEngineering/Testrail), and, for now, 2) the still-up code archived at its GitHub page: https://github.com/mozilla/moztrap (we'll decide what's next for that, in the near future).

See also the history and more-detailed discussion which led us here, at https://groups.google.com/forum/#!topic/mozilla.dev.quality/Sa75hV8Ywvk

(If you'd like, you should be able to filter these notification emails using at least the unique string of "Sa75hV8Ywvk" in the message body.

Thanks!