1042750 - [b2ghaystack] Convert push time to Pacific timezone before comparing with target directories

Status: RESOLVED INCOMPLETE

(Testing :: General, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

+++ This bug was initially created as a clone of Bug #1018907 +++

b2ghaystack uses the push timestamp to find tinderbox builds within a range, however bug 1042119 demonstrates valid builds not being detected. On investigation I discovered that the directory timestamps appear to be offset by 7 hours.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

For example:
https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-inbound-flame-eng/20140721062720/sources.xml

This has a revision of: 008d526c1b98
This was pushed at: 1405949190
The actual directory is: 20140721062720
The expected directory is: 20140721132630

Comment 2

[(not currently active) Ted Mielczarek]

Actually, maybe the problem is just that we didn't update post_upload.py on pvtbuilds? bug 1025165 did it for upload{1,2}.

Comment 3

[Chris Turra [:cturra]]

pvtbuilds.mozilla.org is served out of the ftp cluster. Ted - are you saying the post_upload script be on these nodes as well? i was under the impression that the files being served off of the ftp cluster were uploaded in a different location.

Comment 4

[(not currently active) Ted Mielczarek]

I don't actually know what host is used to upload things to pvtbuilds, I'm assuming there's a different host and that's our issue. Maybe catlee knows (or can point us at someone who does).

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Dave Hunt (:davehunt) from comment #1)
> For example:
> https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-
> inbound-flame-eng/20140721062720/sources.xml
> 
> This has a revision of: 008d526c1b98
> This was pushed at: 1405949190

a.k.a Mon Jul 21 06:26:19 2014 -0700

> The actual directory is: 20140721062720

The upload directory is set using the buildid, which is set by buildbot when creating the job. It's in Pacific time and there is a variable delay after the push. In this case about a minute, which is pretty good considering the assorted 1 minute polling intervals in play. 

FWIW, b2g builds are done using mozharness, which uses rsync to upload instead of post-upload.py, so bug 1018907 is not directly related.
eg:
06:40:00     INFO - Running command: ['rsync', '-e', 'ssh -oIdentityFile=/home/cltbld/.ssh/b2gbld_dsa', '-azv', '.', u'[e-mail]:/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-inbound-flame-eng/20140721062720/'] in /builds/slave/b2g_b2g-in_flame_eng_dep-00000/build/upload
[Full log is at https://tbpl.mozilla.org/php/getParsedLog.php?id=44272335&tree=B2g-Inbound&full=1]

> The expected directory is: 20140721132630

This seems to be the push time converted to UTC, rather than as described above. Beware that the timestamps returned by Apache on pvtbuilds are in UTC.

Does that clear up the confusion ?  You should be able to reduce your 12 hour window if the timezone is taken into account.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Okay, that makes sense to me. So we can safely assume that the directories are close to push time converted from UTC to Pacific. Once done, we should be able to reduce the window back to four hours.

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I'm going to close this as INCOMPLETE as b2ghaystack is not in active use.