Created a more sophisticated submitter.py. This one is in scripts and will eventually replace the one in .../socorro/collector. For now, I'm leaving that one in place. New options: -s, --delay pause between submission queing in milliseconds (default: 1) -D, --dryrun don't actually submit, just print product/version -N, --numberOfSubmissions the number items to submit (all, forever, 1...) (default: all) -n, --numberOfThreads the number of threads (default: 4) the --numberOfSubmissions option allows the submitter to get a specific number of crashes from the source (specified by --search or -s), or loop through a search over and over. The default behavior is to just submit all the crashes found in a search (this mimics the behavior of the old submitter). Also includes some rudimentary statistics. After one minute, it starts to print the average number of submissions attempted per minute over the last five minutes. This app is BETA. There are known defects: * the --numberOfSubmissions option: all and forever work fine, integers do not. * khan seems to have a defective pycurl module for python26. Actual submissions are not tested. * stats don't settle down until after five minutes of running.
Assignee: nobody → lars
Severity: normal → critical
Target Milestone: --- → 1.7.6
examples of use: python26 .../scripts/submitter.py -N forever -s ~lars/breakpad17/testData/500 this will loop through the 500 crashes in my testData directory over and over forever. It will submit as fast as it can. It will only stop with ^C, SIGTERM or SIGKILL python26 .../scripts/submitter.py --delay=200 -N all -s ~lars/breakpad17/testData/1013 this will loop through the 1013 crashes in one of my testData directories. It will submit only 5 per second.
enhancements and corrections are now checked in. * minor modification to the command line parameters * all features of --numberOfSubmissions now work * integrated into the syslog and config system - submitterconfig.py.dist * working components moved into the .../socorro/collector/submitter module * original submitter still intact ** pycurl for python26 is still broken on khan. It works on my workstation at home. we'll modify this app on the fly as our actual needs become more known. (py26)[lars@khan scripts]$ python submitter.py --help submitter 2.0 --delay pause between submission queing in milliseconds (default: 0) -D, --dryrun don't actually submit, just print product/version (default: False) -d, --dumpfile the pathname of a dumpfile to submit (default: None) -?, --help print this list -j, --jsonfile the pathname of a json file to submit (default: None) -n, --numberOfSubmissions the number of crashes to submit (all, forever, 1...) (default: all) --numberOfThreads the number of threads to use (default: 4) -s, --searchRoot a filesystem location to begin a search for json/dump pairs (default: None) --stderrErrorLoggingLevel logging level for the logging to stderr (10 - DEBUG, 20 - INFO, 30 - WARNING, 40 - ERROR, 50 - CRITICAL) (default: 10) --stderrLineFormatString python logging system format for logging to stderr (default: %(asctime)s %(levelname)s - %(threadName)s - %(message)s) --syslogErrorLoggingLevel logging level for the log file (10 - DEBUG, 20 - INFO, 30 - WARNING, 40 - ERROR, 50 - CRITICAL) (default: 10) --syslogFacilityString syslog facility string ("user", "local0", etc) (default: user) --syslogHost syslog hostname (default: localhost) --syslogLineFormatString python logging system format for syslog entries (default: Socorro Submitter (pid %(process)d): %(asctime)s %(levelname)s - %(threadName)s - %(message)s) --syslogPort syslog port (default: 514) --uniqueHangId cache and uniquify hangids (default: True) -u, --url The url of the server to load test (default: https://crash-reports.stage.mozilla.com/submit)
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Component: Socorro → General
Product: Webtools → Socorro
You need to log in before you can comment on or make changes to this bug.