640829 - Install xperf tools on win7 talos slaves

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Shawn Wilsher :sdwilsh]

We need to install the xperf tools (part of the windows SDK) on all windows 7 slaves for bug 609111.

(sorry if I put this bug in the wrong component; not sure which one is correct)

Comment 1

[Armen [:armenzg]]

This is the right component.
We won't be able to take this before end of Q1 though.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

What can I do to help get this done?  Is there anything releng needs?

Comment 3

[Armen [:armenzg]]

Are these the instructions to get this installed?
https://developer.mozilla.org/En/Profiling_with_Xperf
Is this for win7 only, correct?

Getting this done relies on having someone (probably me) to have time to deploy it manually through VNC/RDP one by one in all ~50 win7 slaves (probably 2-3 hours). Before that we would have to deploy it on staging and making sure that it runs as expected and yields the information you need.

jmaher is this a Q1 goal running late? or a very awesome goodness that we should have had since day 1?

Would you have time at the all hands to go together over it and maybe get it done on staging in an hour or so? :)

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

this is something that would be really nice to get done in Q1, but it is a bit too late for that now.  Next week I would like to resolve the remaining two issues outlined above in person; an hour or two should be plenty.

Comment 5

[Armen [:armenzg]]

jmaher how are you doing on your side about this?

I have had my eyes set on fixing a bunch of stuff before the May 17th refresh.
I hope to tackle the week after that date.

Setting priority to P3 meaning that I would like to tackle this in the next 2 weeks.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

the next 2 weeks is great.  My patches for talos proper are just about ready, then we need to sort out the buildbot config changes.

Comment 7

[Armen [:armenzg]]

Now that sdwilsh is gone who is driving this in his place?

Comment 8

[Shawn Wilsher :sdwilsh]

(In reply to comment #7)
> Now that sdwilsh is gone who is driving this in his place?
Me still (and I'm not quite gone yet).

Comment 9

[Armen [:armenzg]]

triagefollowup: I am slowing the process by trying to get to it when more higher priority keeps on falling on me. Could you please find someone who could deploy this manually to the Win7 slaves?

jmaher what was again the path to deploy this? C:\perftools?

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

yes, c:\perftools would be great.  As long as the path doesn't have a ' ' in it, that will keep the scripts and other tools easier to work with.

We are waiting for the new test talos master vm to run staging runs before checking in the code to turn this on.

Comment 11

[Armen [:armenzg]]

Thanks jmaher.
Would you like us to hold on deploying it until this get tested in your systems?

Comment 12

[Joel Maher ( :jmaher ) (UTC -8)]

if you can get these tools deployed it would be one less dependency.  The good news is the patches are at a r+ state, so we are getting closer to making this a reality.

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

Can we revisit this and get this wrapped up?  I finally was able to test my patches in staging and and ready to land.

Comment 14

[Armen [:armenzg]]

coop will be on holidays next week and I will be on buildduty.
If the week is not bad I will be doing few slaves every morning.

> Getting this done relies on having someone (probably me) to have time to
> deploy it manually through VNC/RDP one by one in all ~50 win7 slaves
> (probably 2-3 hours). Before that we would have to deploy it on staging and
> making sure that it runs as expected and yields the information you need.

Comment 15

[Armen [:armenzg]]

Returning to coop as I will be away next week and bug 629759 is being more pressing.

Comment 16

[Chris Cooper [:coop] (he/him)]

Going to follow same plan as Armen: update a few slaves everyday until we're done.

Comment 17

[Joel Maher ( :jmaher ) (UTC -8)]

what is the status of this?  I know we have a lot of slaves, but a few/day and by now we should be pretty close to rolled out?

Comment 18

[Armen [:armenzg]]

Not sure where coop got to (he's gone for holidays).
I will deal with this in the next few days.

Comment 19

[Joel Maher ( :jmaher ) (UTC -8)]

update?

Comment 20

[Armen [:armenzg]]

The rest of my day will be dedicated to this besides small tasks.
I will update at my EOD.

Comment 21

[Armen [:armenzg]]

I have been able to generate the wpt_x86.msi and place it on:
http://dev-stage01.build.mozilla.org/pub/mozilla.org/mozilla/libraries/win32/wpt_x86.msi

I am trying to figure out what the correct command lines are to deploy it.

Comment 22

[Armen [:armenzg]]

Attachment: bat file to install Windows Performance Toolkit

It seems that the following command does the job but a UAC message comes up:
msiexec /passive /i “C:\Users\cltbld\Desktop\wpt_x86.msi” /log wpt_x86.log WPFPERFDIR=C:\perftools

jmaher does it look good to you?

Comment 23

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 553532 [details]
bat file to install Windows Performance Toolkit

this looks like the correct .msi and WFPPERFDIR!

Thanks!

Comment 24

[Armen [:armenzg]]

I deployed this to half the win7 slaves. I will do the other half tomorrow.

Something to notice is that I will have to do the following slaves once they are rebooted by IT:
talos-r3-w7-011
talos-r3-w7-019
talos-r3-w7-036

I also have to deploy this to talos-r3-w7-ref and ask IT to take a snapshot.

This is the commands I have been following:
> wget --no-check-certificate -Owpt.bat https://bugzilla.mozilla.org/attachment.cgi?id=553532
> wpt.bat
> click "Yes" on UAC prompt

Comment 25

[Armen [:armenzg]]

I did few more slaves in the morning but the network is bad again.

Tonight IT will have a downtime for the network state so tomorrow should be easier.

Comment 26

[Armen [:armenzg]]

I have done all slaves but the ones mentioned in comment 24.

IT will kick the ref image in bug 680445 and the remaining slaves in bug 678883.

Once they are up I will deployed unto them.

Documentation updated as well:
https://wiki.mozilla.org/index.php?title=ReferencePlatforms/Test/Win7#Xperf

Comment 27

[Joel Maher ( :jmaher ) (UTC -8)]

I suspect we cannot talk to 10.2.76.100 from the talos slave.

can we try:
wget 'http://10.2.76.100'
wget 'http://brasstacks.mozilla.com'

I suspect the first wget will result in a timeout, error or something like that (404?) and the second one will immediately return with a downloaded index.html.

If this is the case, we can fix the hardcoded servername in the xperf log parser to use brasstacks.mozilla.com and we can start logging our data there.

Comment 28

[Armen [:armenzg]]

Actually both return an index.html with the same contents :S

Comment 29

[Armen [:armenzg]]

FTR I can ping to the first address but not the second one:

C:\Users\cltbld>ping 10.2.76.100
                
Pinging 10.2.76.100 with 32 bytes of data:                     
Reply from 10.2.76.100: bytes=32 time=3ms TTL=60
Reply from 10.2.76.100: bytes=32 time=2ms TTL=60               
Reply from 10.2.76.100: bytes=32 time=2ms TTL=60
                
Ping statistics for 10.2.76.100:               
    Packets: Sent = 3, Received = 3, Lost = 0 (0% loss),       
Approximate round trip times in milli-seconds:.215
    Minimum = 2ms, Maximum = 3ms, Average = 2ms.215|:80... 

C:\Users\cltbld>ping brasstacks.mozilla.com
                
Pinging brasstacks.sj.mozilla.com [63.245.208.215] with 32 bytes of data:
Request timed out.

Perhaps arr and dustin can notice something we are doing wrong right away.

Comment 30

[Dustin J. Mitchell [:dustin] (he/him)]

Joel, let me know what you're seeing, and what's expected, and I'll be happy to take a look.

Comment 31

[Armen [:armenzg]]

To reboot talos-r3-w7-036 see bug 682698.

Comment 32

[Armen [:armenzg]]

I deployed this to slaves #11 and #19.

We are waiting on a reboot on bug 682698 for #36.

I can't access the ref machine even after the reboot in bug 680445. I will ask IT.

Comment 33

[Armen [:armenzg]]

I took care of slave #36 as well.

We are now waiting on bug 680445.