[b2ghaystack] Convert push time to Pacific timezone before comparing with target directories

RESOLVED INCOMPLETE

Status

Testing
General
RESOLVED INCOMPLETE
3 years ago
3 years ago

People

(Reporter: davehunt, Unassigned, Mentored)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
+++ This bug was initially created as a clone of Bug #1018907 +++

b2ghaystack uses the push timestamp to find tinderbox builds within a range, however bug 1042119 demonstrates valid builds not being detected. On investigation I discovered that the directory timestamps appear to be offset by 7 hours.
(Reporter)

Comment 1

3 years ago
For example:
https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-inbound-flame-eng/20140721062720/sources.xml

This has a revision of: 008d526c1b98
This was pushed at: 1405949190
The actual directory is: 20140721062720
The expected directory is: 20140721132630
Actually, maybe the problem is just that we didn't update post_upload.py on pvtbuilds? bug 1025165 did it for upload{1,2}.
Flags: needinfo?(cturra)
pvtbuilds.mozilla.org is served out of the ftp cluster. Ted - are you saying the post_upload script be on these nodes as well? i was under the impression that the files being served off of the ftp cluster were uploaded in a different location.
Flags: needinfo?(cturra) → needinfo?(ted)
I don't actually know what host is used to upload things to pvtbuilds, I'm assuming there's a different host and that's our issue. Maybe catlee knows (or can point us at someone who does).
Flags: needinfo?(ted) → needinfo?(catlee)
(In reply to Dave Hunt (:davehunt) from comment #1)
> For example:
> https://pvtbuilds.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-
> inbound-flame-eng/20140721062720/sources.xml
> 
> This has a revision of: 008d526c1b98
> This was pushed at: 1405949190

a.k.a Mon Jul 21 06:26:19 2014 -0700

> The actual directory is: 20140721062720

The upload directory is set using the buildid, which is set by buildbot when creating the job. It's in Pacific time and there is a variable delay after the push. In this case about a minute, which is pretty good considering the assorted 1 minute polling intervals in play. 

FWIW, b2g builds are done using mozharness, which uses rsync to upload instead of post-upload.py, so bug 1018907 is not directly related.
eg:
06:40:00     INFO - Running command: ['rsync', '-e', 'ssh -oIdentityFile=/home/cltbld/.ssh/b2gbld_dsa', '-azv', '.', u'b2gbld@pvtbuilds.pvt.build.mozilla.org:/pvt/mozilla.org/b2gotoro/tinderbox-builds/b2g-inbound-flame-eng/20140721062720/'] in /builds/slave/b2g_b2g-in_flame_eng_dep-00000/build/upload
[Full log is at https://tbpl.mozilla.org/php/getParsedLog.php?id=44272335&tree=B2g-Inbound&full=1]

> The expected directory is: 20140721132630

This seems to be the push time converted to UTC, rather than as described above. Beware that the timestamps returned by Apache on pvtbuilds are in UTC.

Does that clear up the confusion ?  You should be able to reduce your 12 hour window if the timezone is taken into account.

Updated

3 years ago
Flags: needinfo?(catlee)
(Reporter)

Comment 6

3 years ago
Okay, that makes sense to me. So we can safely assume that the directories are close to push time converted from UTC to Pacific. Once done, we should be able to reduce the window back to four hours.
Mentor: dave.hunt@gmail.com
Component: General Automation → General
Product: Release Engineering → Testing
QA Contact: catlee
Summary: timestamps for B2G build directories are off by timezone offset (7 hours) → [b2ghaystack] Convert push time to Pacific timezone before comparing with target directories
(Reporter)

Comment 7

3 years ago
I'm going to close this as INCOMPLETE as b2ghaystack is not in active use.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.