firefox-ui-tests should use marionette driver 0.5 to keep /tmp tidy

RESOLVED FIXED

Status

Testing
Firefox UI Tests
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: armenzg, Assigned: armenzg)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Assignee)

Description

3 years ago
The loaner I have is running out of space when I try to run update_verify.
After investigating it, I noticed that /tmp directory was too full.

There are 867 directories which say mozrunner [1]
Before clean up, /tmp had 26GB used [2]
After clean up, we only use 1.2GB [3]

Morgan, can we ask mozrunner to clean /tmp more aggresively?

[1]
[cltbld@dev-linux64-ec2-armenzg.dev.releng.use1.mozilla.com updates]$ ls /tmp | grep "mozrunner" | wc -l
867

[2]
26G     /tmp

[3]
[cltbld@dev-linux64-ec2-armenzg.dev.releng.use1.mozilla.com updates]$ rm -rf /tmp/*mozrunner
[cltbld@dev-linux64-ec2-armenzg.dev.releng.use1.mozilla.com updates]$ du -sh /tmp
du: cannot read directory `/tmp/tmprsNTVO': Permission denied
du: cannot read directory `/tmp/tmpBhNBWV': Permission denied
du: cannot read directory `/tmp/tmpAUnOy5': Permission denied
du: cannot read directory `/tmp/tmpnyHhzK': Permission denied
du: cannot read directory `/tmp/tmpkFCIGW': Permission denied
du: cannot read directory `/tmp/tmptwdEUX': Permission denied
du: cannot read directory `/tmp/tmp2MacTY': Permission denied
1.2G    /tmp
(Assignee)

Comment 1

3 years ago
We should keep it tidy regardless of mozrunner dirs.
Summary: Be more aggressive at removing mozrunner from /tmp → Be more aggressive at cleaning up /tmp
(Assignee)

Comment 2

3 years ago
We have to disable mozrunner for loaners.
I cleaned the mozrunner dirs yesterday and I already have 11GB today.

[cltbld@dev-linux64-ec2-armenzg.dev.releng.use1.mozilla.com updates]$ ls -lrt /tmp/*mozrunner | wc -l
14957
[cltbld@dev-linux64-ec2-armenzg.dev.releng.use1.mozilla.com updates]$ du -sh /tmp
11G     /tmp
[cltbld@dev-linux64-ec2-armenzg.dev.releng.use1.mozilla.com updates]$ rm -rf /tmp/*mozrunner
[cltbld@dev-linux64-ec2-armenzg.dev.releng.use1.mozilla.com updates]$ du -sh /tmp
6.7M    /tmp
(Assignee)

Comment 3

3 years ago
ping
Flags: needinfo?(winter2718)
(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #3)
> ping

Sorry for the delay! Yes, we can create a runner task that tries to do this.
Flags: needinfo?(winter2718)
If this is a mozrunner specific thing, I can look at that too; but I think that's more bhearsum's area of expertise.
fwiw runner is not enabled on loaners (we rm -rf /opt/runner

What is "mozrunner", as in when/where/how does it run. That would give me a better understanding of when/where/how this cleanup should happen.
(In reply to Justin Wood (:Callek) from comment #6)
> fwiw runner is not enabled on loaners (we rm -rf /opt/runner
> 
> What is "mozrunner", as in when/where/how does it run. That would give me a
> better understanding of when/where/how this cleanup should happen.

Sounds to me like this wasn't done on Armen's machine. Armen, can you see if /opt/runner exists on yours, and delete it if it does? It sounds like that simple action should fix your problem.
Flags: needinfo?(armenzg)
(Assignee)

Comment 8

3 years ago
There is no /opt/runner.

I figured out that I got confused between runner and mozrunner [1]
It seems that is on me to discover what the harness I'm executing is doing.

[1] http://hg.mozilla.org/mozilla-central/file/default/testing/mozbase/mozrunner
Flags: needinfo?(armenzg)
(Assignee)

Updated

3 years ago
Component: Platform Support → Firefox UI Tests
Product: Release Engineering → Mozilla QA
QA Contact: coop → hskupin
Summary: Be more aggressive at cleaning up /tmp → firefox-ui-updates fills up /tmp thanks to mozrunner
Version: unspecified → Firefox 38
(Assignee)

Updated

3 years ago
Assignee: nobody → armenzg
(Assignee)

Updated

3 years ago
Component: Firefox UI Tests → Mozbase
Product: Mozilla QA → Testing
QA Contact: hskupin
Summary: firefox-ui-updates fills up /tmp thanks to mozrunner → mozprofile does not clean profiles that have been created
Version: Firefox 38 → Trunk
(Assignee)

Updated

3 years ago
Blocks: 1148546
Summary: mozprofile does not clean profiles that have been created → [mozprofile] mozprofile does not clean profiles that have been created
(Assignee)

Comment 9

3 years ago
Created attachment 8596792 [details] [diff] [review]
wip - If we create a profile, we should remove it

This has proven to work for me.

I will probably have to generate a temporary mozprofile and upload it to the internal pypi (with temporary version .dev0).

I will have to pin mozprofile to a specific version in firefox_ui_requirements.txt.

py.test seems to pass, however, I should test it on treeherder as well:
armenzg@armenzg-thinkpad:~/repos/branches/mozilla-central/testing/mozbase/mozprofile/tests$ py.test
============================================================================================ test session starts ============================================================================================
platform linux2 -- Python 2.7.6 -- pytest-2.5.1
collected 39 items 

test_addons.py .s.....s.......
test_clone_cleanup.py ..
test_nonce.py .
test_preferences.py .............
test_profile_view.py ...
test_webapps.py .....

=================================================================================== 37 passed, 2 skipped in 2.29 seconds ====================================================================================
Attachment #8596792 - Flags: feedback?(cmanchester)
Comment on attachment 8596792 [details] [diff] [review]
wip - If we create a profile, we should remove it

Review of attachment 8596792 [details] [diff] [review]:
-----------------------------------------------------------------

I was able to land a fix for bug 1083131 that should be equivalent. I should have communicated this more actively, sorry about that!
Attachment #8596792 - Flags: feedback?(cmanchester)
(Assignee)

Comment 11

3 years ago
(In reply to Chris Manchester [:chmanchester] from comment #10)
> Comment on attachment 8596792 [details] [diff] [review]
> wip - If we create a profile, we should remove it
> 
> Review of attachment 8596792 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I was able to land a fix for bug 1083131 that should be equivalent. I should
> have communicated this more actively, sorry about that!

Not a problem.
Can we make a new release of marionette driver and use that on firefox-ui-tests?
(Assignee)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1083131
(Assignee)

Updated

3 years ago
Component: Mozbase → Firefox UI Tests
Product: Testing → Mozilla QA
QA Contact: hskupin
Summary: [mozprofile] mozprofile does not clean profiles that have been created → firefox-ui-updates should use marionette driver 0.5 to keep /tmp tidy
Version: Trunk → unspecified
(Assignee)

Updated

3 years ago
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---
Summary: firefox-ui-updates should use marionette driver 0.5 to keep /tmp tidy → firefox-ui-tests should use marionette driver 0.5 to keep /tmp tidy
(Assignee)

Comment 13

3 years ago
Created attachment 8599389 [details] [review]
PR request to use marionette driver 0.5
Attachment #8599389 - Flags: review?(cmanchester)
We need a new client for travis to pass (bug 1159816).
(Assignee)

Updated

3 years ago
Depends on: 1159816
(Assignee)

Updated

3 years ago
Attachment #8599389 - Flags: review?(cmanchester)
(Assignee)

Comment 15

3 years ago
I'm retrying the Travis runs now that marionette client 0.12 has been uploaded.

What a churn!
(In reply to Armen Zambrano G. (:armenzg - Toronto) from comment #15)
> I'm retrying the Travis runs now that marionette client 0.12 has been
> uploaded.
> 
> What a churn!

Please also update the "==" dep on marionette client to .12 in the requirements for firefox-ui-tests.
Comment on attachment 8599389 [details] [review]
PR request to use marionette driver 0.5

Hurrah, passing build! Please squash and update the commit message before merging.
Attachment #8599389 - Flags: review+
(Assignee)

Comment 18

3 years ago
https://github.com/mozilla/firefox-ui-tests/commit/c909645b8671654e2cafa9660fa18c4753864c2f
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
Product: Mozilla QA → Testing
You need to log in before you can comment on or make changes to this bug.