Closed
Bug 1249992
Opened 8 years ago
Closed 8 years ago
ftpscraper fails with IndexError on mobile
Categories
(Socorro :: General, task)
Socorro
General
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: peterbe, Unassigned)
References
Details
See https://sentry.prod.mozaws.net/operations/socorro-prod/group/237348/ I can reproduce this locally by running: python socorro/cron/jobs/ftpscraper.py --product=mobile --date=2016-02-21 Here's the error: Traceback (most recent call last): File "socorro/cron/jobs/ftpscraper.py", line 731, in <module> sys.exit(main(FTPScraperCronAppDryRunner)) File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/app/socorro_app.py", line 506, in main values_source_list=values_source_list File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/app/socorro_app.py", line 191, in run values_source_list=values_source_list File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/app/socorro_app.py", line 289, in _do_run return_code = fix_exit_code(app_to_run.main()) File "socorro/cron/jobs/ftpscraper.py", line 727, in main self.ftpscraper.run(self.config.date) File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/cron/jobs/ftpscraper.py", line 383, in run date File "/Users/peterbe/virtualenvs/socorro/lib/python2.7/site-packages/crontabber/transaction_executor.py", line 46, in __call__ result = function(connection, *args, **kwargs) File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/cron/jobs/ftpscraper.py", line 388, in _scrape_releases_and_nightlies self.scrape_nightlies(connection, product_name, date) File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/cron/jobs/ftpscraper.py", line 611, in scrape_nightlies for info in self.get_nightly(nightly, dirname): File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/cron/jobs/ftpscraper.py", line 295, in get_nightly kvpairs, bad_lines = self.parse_info_file(url, nightly=True) File "/Users/peterbe/dev/MOZILLA/SOCORRO/socorro/socorro/cron/jobs/ftpscraper.py", line 146, in parse_info_file results = {'buildID': contents[0], 'rev': contents[1]} IndexError: list index out of range
Reporter | ||
Comment 1•8 years ago
|
||
Kairo, As you can see, our ftpscraper is repeatedly stuck and it's holding up crontabber. The problem seems to be that releng has put a non-nightly _info.txt file under a nightly release. I can reproduce this locally so it wouldn't be particularly hard to hack around this. But I'm not sure if we should. Hoping for some feedback from you and Rail.
Comment 2•8 years ago
|
||
I prefer the FTP scraper working and just ignoring a file it can't deal with so at least the rest would work. For example, I need Firefox 45 beta 8 to be picked up Monday morning, otherwise the already-in-jeopardy Firefox 45 release will be in even more trouble than it already is.
Comment 3•8 years ago
|
||
What's the status here ? If it's still a problem it would be good to get some urls which are causing problems.
Reporter | ||
Comment 4•8 years ago
|
||
The reason there were two bugs was 1) One to track changing the code over so it starts reading the .json files instead of _info.txt 2) Being impatient and waiting for 1) let's fix what's broken "right now". I think all is well now. We're happily reading those .json files for mobile in prod now and have for some time.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•