Closed
Bug 1111733
Opened 10 years ago
Closed 10 years ago
Test local UDP transfers when bigger payloads are sent by Telemetry Experiments
Categories
(Content Services Graveyard :: Tiles: Data Processing, defect)
Content Services Graveyard
Tiles: Data Processing
Tracking
(Not tracked)
RESOLVED
FIXED
Iteration:
37.2
People
(Reporter: oyiptong, Assigned: kthiessen)
References
Details
(Whiteboard: .002a)
User Story
Tiles webheads need to be tested to make sure there is no loss of data due to the increased payload sizes. We use UDP as the transport to rsyslog within the same machine. There are potential issues of message sizes growing beyond buffer sizes, therefore, this should be tested I will write a script to help kthiessen test
No description provided.
Reporter | ||
Updated•10 years ago
|
User Story: (updated)
Reporter | ||
Comment 1•10 years ago
|
||
Code to generate data in review: https://github.com/oyiptong/onyx/pull/12
kthiessen: to run this, you'd type:
ONYX_SETTINGS=onyx_config.OnyxConfig manage.py log flood_user_event 1000 -s 10
1000 here stands for the number of messages
the parameter to -s is the number of kilobytes one message should be
Reporter | ||
Comment 2•10 years ago
|
||
will update when the code is deployed in staging
Comment 3•10 years ago
|
||
The code is deployed in staging. The staging box is at: ec2-54-172-198-19.compute-1.amazonaws.com.
How do we run manage.py from the stage environment?
I tried:
> cd /data/onyx
> ONYX_SETTINGS=onyx_config.OnyxConfig ./bin/python scripts/manage.py log flood_user_event 1 -s 10
Traceback (most recent call last):
File "scripts/manage.py", line 3, in <module>
from onyx.utils import environment_manager_create, GunicornServerCommand, LogCommand
ImportError: No module named onyx.utils
Reporter | ||
Comment 4•10 years ago
|
||
The correct command to execute it is:
> $ ONYX_SETTINGS=onyx_config.OnyxConfig /data/onyx/bin/manage.py log flood_user_event 1 -s 10
To make that work in the current environment, onyx had to be installed in the virtualenv's site-packages.
To do so:
> cd /data/onyx
> sudo ./bin/python setup.py install
Reporter | ||
Comment 5•10 years ago
|
||
Actually, that won't work.
This will:
> $ PYTHONPATH=/data/onyx ONYX_SETTINGS=onyx_config.OnyxConfig /data/onyx/bin/manage.py log flood_user_event 1 -s 10
This can run from any cwd
Comment 6•10 years ago
|
||
Conclusions from testing:
Max safe message size is 3KB. When message payloads are larger we start seeing message drops. Message loss starts happening around 4KB.
We generated 20,000 3KB messages and got 100% delivery. The delivery rate was ~330 messages/second. The rate was limited by CPU (c3.large).
As long as the new telemetry messages are <3KB we should see no affect on message delivery at the onyx layer.
Updated•10 years ago
|
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•