1154060 - firefox-ui-tests should use marionette driver 0.5 to keep /tmp tidy

Status: RESOLVED FIXED

(Testing :: Firefox UI Tests, defect)

Description

[Armen [:armenzg]]

The loaner I have is running out of space when I try to run update_verify.
After investigating it, I noticed that /tmp directory was too full.

There are 867 directories which say mozrunner [1]
Before clean up, /tmp had 26GB used [2]
After clean up, we only use 1.2GB [3]

Morgan, can we ask mozrunner to clean /tmp more aggresively?

[1]
[[e-mail] updates]$ ls /tmp | grep "mozrunner" | wc -l
867

[2]
26G     /tmp

[3]
[[e-mail] updates]$ rm -rf /tmp/*mozrunner
[[e-mail] updates]$ du -sh /tmp
du: cannot read directory `/tmp/tmprsNTVO': Permission denied
du: cannot read directory `/tmp/tmpBhNBWV': Permission denied
du: cannot read directory `/tmp/tmpAUnOy5': Permission denied
du: cannot read directory `/tmp/tmpnyHhzK': Permission denied
du: cannot read directory `/tmp/tmpkFCIGW': Permission denied
du: cannot read directory `/tmp/tmptwdEUX': Permission denied
du: cannot read directory `/tmp/tmp2MacTY': Permission denied
1.2G    /tmp

Comment 1

[Armen [:armenzg]]

We should keep it tidy regardless of mozrunner dirs.

Comment 2

[Armen [:armenzg]]

We have to disable mozrunner for loaners.
I cleaned the mozrunner dirs yesterday and I already have 11GB today.

[[e-mail] updates]$ ls -lrt /tmp/*mozrunner | wc -l
14957
[[e-mail] updates]$ du -sh /tmp
11G     /tmp
[[e-mail] updates]$ rm -rf /tmp/*mozrunner
[[e-mail] updates]$ du -sh /tmp
6.7M    /tmp

Comment 3

[Armen [:armenzg]]

ping

Comment 4

[Morgan Phillips [:mrrrgn]]

(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #3)
> ping

Sorry for the delay! Yes, we can create a runner task that tries to do this.

Comment 5

[Morgan Phillips [:mrrrgn]]

If this is a mozrunner specific thing, I can look at that too; but I think that's more bhearsum's area of expertise.

Comment 6

[Justin Wood (:Callek)]

fwiw runner is not enabled on loaners (we rm -rf /opt/runner

What is "mozrunner", as in when/where/how does it run. That would give me a better understanding of when/where/how this cleanup should happen.

Comment 7

[[e-mail] (:bhearsum)]

(In reply to Justin Wood (:Callek) from comment #6)
> fwiw runner is not enabled on loaners (we rm -rf /opt/runner
> 
> What is "mozrunner", as in when/where/how does it run. That would give me a
> better understanding of when/where/how this cleanup should happen.

Sounds to me like this wasn't done on Armen's machine. Armen, can you see if /opt/runner exists on yours, and delete it if it does? It sounds like that simple action should fix your problem.

Comment 8

[Armen [:armenzg]]

There is no /opt/runner.

I figured out that I got confused between runner and mozrunner [1]
It seems that is on me to discover what the harness I'm executing is doing.

[1] http://hg.mozilla.org/mozilla-central/file/default/testing/mozbase/mozrunner

Comment 9

[Armen [:armenzg]]

Attachment: wip - If we create a profile, we should remove it

This has proven to work for me.

I will probably have to generate a temporary mozprofile and upload it to the internal pypi (with temporary version .dev0).

I will have to pin mozprofile to a specific version in firefox_ui_requirements.txt.

py.test seems to pass, however, I should test it on treeherder as well:
armenzg@armenzg-thinkpad:~/repos/branches/mozilla-central/testing/mozbase/mozprofile/tests$ py.test
============================================================================================ test session starts ============================================================================================
platform linux2 -- Python 2.7.6 -- pytest-2.5.1
collected 39 items 

test_addons.py .s.....s.......
test_clone_cleanup.py ..
test_nonce.py .
test_preferences.py .............
test_profile_view.py ...
test_webapps.py .....

=================================================================================== 37 passed, 2 skipped in 2.29 seconds ====================================================================================

Comment 10

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8596792 [details] [diff] [review]
wip - If we create a profile, we should remove it

Review of attachment 8596792 [details] [diff] [review]:
-----------------------------------------------------------------

I was able to land a fix for bug 1083131 that should be equivalent. I should have communicated this more actively, sorry about that!

Comment 11

[Armen [:armenzg]]

(In reply to Chris Manchester [:chmanchester] from comment #10)
> Comment on attachment 8596792 [details] [diff] [review]
> wip - If we create a profile, we should remove it
> 
> Review of attachment 8596792 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I was able to land a fix for bug 1083131 that should be equivalent. I should
> have communicated this more actively, sorry about that!

Not a problem.
Can we make a new release of marionette driver and use that on firefox-ui-tests?

Comment 13

[Armen [:armenzg]]

Attachment: PR request to use marionette driver 0.5

Comment 14

[Chris Manchester (limited bugmail, email directly)]

We need a new client for travis to pass (bug 1159816).

Comment 15

[Armen [:armenzg]]

I'm retrying the Travis runs now that marionette client 0.12 has been uploaded.

What a churn!

Comment 16

[Chris Manchester (limited bugmail, email directly)]

(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #15)
> I'm retrying the Travis runs now that marionette client 0.12 has been
> uploaded.
> 
> What a churn!

Please also update the "==" dep on marionette client to .12 in the requirements for firefox-ui-tests.

Comment 17

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8599389 [details] [review]
PR request to use marionette driver 0.5

Hurrah, passing build! Please squash and update the commit message before merging.

Comment 18

[Armen [:armenzg]]

https://github.com/mozilla/firefox-ui-tests/commit/c909645b8671654e2cafa9660fa18c4753864c2f