Closed Bug 1079144 Opened 10 years ago Closed 10 years ago

File 20141005-modulelist.txt didn't get generated

Categories

(Infrastructure & Operations Graveyard :: WebOps: Socorro, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Socorro
Platform:
x86
macOS
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: Usul, Unassigned)

References

(
URL
)

Details

I've followed the runbook - but it failed. When I tried again I got :

[socorro@sp-admin01.phx1 ~]$ export JAVA_HOME=/usr/lib/jvm/java-1.6.0; export HADOOP_GW="cherry-gw.cherry.metrics.phx1.mozilla.com"; /data/socorro/application/scripts/crons/cron_modulelist.sh
Fatal exit code: 6
pig run failed
The run triggered from cron was almost done but was killed:

2014-10-07 08:40:36,941 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 96% complete
Killed by signal 1.

Manual run failed because the output file already exists:

2014-10-07 08:57:55,025 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 6000: 
<file ./socorro/analysis/modulelist.pig, line 37, column 0> Output Location Validation Failed for: 'hdfs://nameservice1/user/socorro/modulelist-141006-141006 More info to follow:
Output directory hdfs://nameservice1/user/socorro/modulelist-141006-141006 already exists
Details at logfile: socorro-modulelist.log

I'll clean up the output file and try another manual run.
Removed:

[socorro@sp-admin01.phx1 ~]$ export JAVA_HOME=/usr/lib/jvm/java-1.6.0; export HADOOP_GW="cherry-gw.cherry.metrics.phx1.mozilla.com";
[socorro@sp-admin01.phx1 ~]$ DATE=`date -d 'yesterday' +%y%m%d`
[socorro@sp-admin01.phx1 ~]$ ssh $HADOOP_GW "hadoop fs -rmr modulelist-${DATE}-${DATE}"
rmr: DEPRECATED: Please use 'rm -r' instead.
14/10/07 09:09:29 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 1440 minutes, Emptier interval = 0 minutes.
Moved: 'hdfs://nameservice1/user/socorro/modulelist-141006-141006' to trash at: hdfs://nameservice1/user/socorro/.Trash/Current


Running manually now (same as comment 0)
of course today's file didn't get generated : 20141005-modulelist.tx :(
(In reply to Robert Helmer [:rhelmer] from comment #2)
> Removed:
> 
> [socorro@sp-admin01.phx1 ~]$ export JAVA_HOME=/usr/lib/jvm/java-1.6.0;
> export HADOOP_GW="cherry-gw.cherry.metrics.phx1.mozilla.com";
> [socorro@sp-admin01.phx1 ~]$ DATE=`date -d 'yesterday' +%y%m%d`

Oops this should have been:
DATE=141005

> [socorro@sp-admin01.phx1 ~]$ ssh $HADOOP_GW "hadoop fs -rmr
> modulelist-${DATE}-${DATE}"

Did this again, rerunning comment 0 now.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.