Need Real Time Metrics for Encoder -> CDN Ingest Connections

RESOLVED WONTFIX

Status

Cloud Services
Operations: Metrics/Monitoring
--
major
RESOLVED WONTFIX
3 years ago
2 years ago

People

(Reporter: richard, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
How do I go about getting real-time network metrics on connections between:

encoder1.commons.av.mtv2.mozilla.com -> fso.lax.1237.taucdn.net

encoder2.commons.av.mtv2.mozilla.com -> fso.lax.1237.taucdn.net

These are TCP RTMP connections on port 1935
(Reporter)

Updated

3 years ago
Assignee: network-operations → server-ops-webops
Component: NetOps: Other → WebOps: Product Delivery
QA Contact: jbarnell → smani

Updated

3 years ago
Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/772]
On the theory that this is a question for the CDN folks, I'm going to hand this over to CloudOps so they can look into it further. If that doesn't pan out, you might try speaking with Netops next, or it's possible that either application-level or system-level monitoring could be designed to support this data collection. (I'm not sure that Webops can be of any useful assistance, though, sorry.)
Assignee: server-ops-webops → nobody
Component: WebOps: Product Delivery → Operations: Metrics/Monitoring
Product: Infrastructure & Operations → Mozilla Services
QA Contact: smani
Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/772]
(Reporter)

Comment 2

3 years ago
Edgecast Support requested we get mtr stats for the connection.   It would appear from those collected before this week's Project Meeting that the problem is on our lan, between the encoder and the gateway:



                                      My traceroute  [v0.82]
encoder1.commons.av.mtv2.mozilla.com (0.0.0.0)                           Mon Mar 23 09:34:49 2015
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                         Packets               Pings
 Host                                                  Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 10.252.48.1                                        56.1%  1117    0.5   1.1   0.3 166.6   8.8
 2. v-1077.border1.sjc2.mozilla.net                     0.0%  1117    2.0   3.0   1.7  88.4   6.1
 3. xe-5-0-0.mpr4.sjc7.us.above.net                     0.0%  1116    2.0   3.4   1.7 123.5   6.9
 4. ae11.mpr3.sjc7.us.zip.zayo.com                      0.0%  1116    5.3   2.9   1.6 110.1   4.7
 5. zayo-ntt.sjc7.us.zip.zayo.com                       0.0%  1116    2.5   2.6   2.1  74.3   3.1
 6. ae-4.r22.snjsca04.us.bb.gin.ntt.net                 0.0%  1116    2.2   2.6   1.9  92.6   4.0
 7. ae-7.r21.lsanca03.us.bb.gin.ntt.net                 0.0%  1116   10.5  13.2  10.1 134.9   9.6
 8. ae-2.r05.lsanca03.us.bb.gin.ntt.net                 0.0%  1116   11.2  11.0  10.3  96.9   3.3
 9. xe-0-2-0-29.r05.lsanca03.us.ce.gin.ntt.net          0.0%  1116   12.5  13.1  12.3  89.3   4.2
10. 192.229.130.10                                      0.0%  1116   12.7  13.0  12.5  73.1   3.3



                                      My traceroute  [v0.82]
encoder1.commons.av.mtv2.mozilla.com (0.0.0.0)                           Mon Mar 23 09:35:37 2015
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                        Packets               Pings
 Host                                                 Drop   Rcv    Avg Gmean Jttr Javg Jmax Jint
 1. 10.252.48.1                                        650   513    1.1   0.4  0.1  1.6 166.  8.4
 2. v-1077.border1.sjc2.mozilla.net                      0  1164    3.0   2.0  0.2  2.6 86.5 41.1
 3. xe-5-0-0.mpr4.sjc7.us.above.net                      0  1164    3.3   2.1  0.2  3.3 121. 11.5
 4. ae11.mpr3.sjc7.us.zip.zayo.com                       0  1164    2.9   2.2  0.8  2.1 108. 11.6
 5. zayo-ntt.sjc7.us.zip.zayo.com                        0  1164    2.6   2.3  0.3  1.2 72.0  4.6
 6. ae-4.r22.snjsca04.us.bb.gin.ntt.net                  0  1164    2.6   2.1  0.2  1.3 90.5  4.2
 7. ae-7.r21.lsanca03.us.bb.gin.ntt.net                  0  1164   13.2  12.1  3.2  5.1 124. 59.1
 8. ae-2.r05.lsanca03.us.bb.gin.ntt.net                  0  1164   11.0  10.8  0.0  1.1 86.1  8.9
 9. xe-0-2-0-29.r05.lsanca03.us.ce.gin.ntt.net           0  1164   13.0  12.7  0.3  1.4 55.0 18.9
10. 192.229.130.10                                       0  1164   12.9  12.7  0.0  1.1 43.7  1.3


Will do another sample during the Project Meeting today to confirm.
Severity: normal → major
(Reporter)

Comment 3

3 years ago
Status for the duration of the meeting:

                                            My traceroute  [v0.82]
encoder1.commons.av.mtv2.mozilla.com (0.0.0.0)                                        Mon Mar 23 11:20:42 2015
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                      Packets               Pings
 Host                                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 10.252.48.1                                                     49.2%  2218    0.4   0.4   0.3  61.0   2.0
 2. v-1077.border1.sjc2.mozilla.net                                  0.0%  2217    1.9   1.9   1.7  51.7   3.4
 3. xe-5-0-0.mpr4.sjc7.us.above.net                                  0.0%  2217    1.8   2.2   1.4  52.1   5.1
 4. ae11.mpr3.sjc7.us.zip.zayo.com                                   0.1%  2217    1.9   2.7   1.5  52.2   3.1
 5. zayo-ntt.sjc7.us.zip.zayo.com                                    0.0%  2217    2.3   2.2   2.1  16.7   0.7
 6. ae-4.r22.snjsca04.us.bb.gin.ntt.net                              0.0%  2217    2.2   2.8   1.8  79.2   6.2
 7. ae-7.r21.lsanca03.us.bb.gin.ntt.net                              0.0%  2217   14.8  12.7  10.2 101.1   7.2
 8. ae-2.r05.lsanca03.us.bb.gin.ntt.net                              0.0%  2217   11.2  12.5  11.0  92.2   5.9
 9. xe-0-2-0-29.r05.lsanca03.us.ce.gin.ntt.net                       0.0%  2217   12.5  12.5  12.3  48.5   2.7
10. 192.229.130.10                                                   0.0%  2217   12.8  12.6  12.4  35.6   0.8



                                            My traceroute  [v0.82]
encoder1.commons.av.mtv2.mozilla.com (0.0.0.0)                                        Mon Mar 23 11:16:58 2015
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                     Packets               Pings
 Host                                                              Drop   Rcv    Avg Gmean Jttr Javg Jmax Jint
 1. 10.252.48.1                                                     971  1022    0.4   0.3  0.1  0.6 60.6 16.9
 2. v-1077.border1.sjc2.mozilla.net                                   0  1994    2.0   1.9  0.3  1.8 49.8 37.1
 3. xe-5-0-0.mpr4.sjc7.us.above.net                                   0  1994    2.4   1.7  0.0  2.6 47.5 16.1
 4. ae11.mpr3.sjc7.us.zip.zayo.com                                    1  1993    2.8   2.3  1.0  1.9 49.5 13.4
 5. zayo-ntt.sjc7.us.zip.zayo.com                                     0  1994    2.2   2.2  0.1  0.7 14.4  7.3
 6. ae-4.r22.snjsca04.us.bb.gin.ntt.net                               0  1994    3.0   2.2  0.1  2.6 76.9  6.9
 7. ae-7.r21.lsanca03.us.bb.gin.ntt.net                               0  1993   12.3  11.7  0.1  3.7 90.6 49.9
 8. ae-2.r05.lsanca03.us.bb.gin.ntt.net                               1  1992   12.7  12.1  1.8  3.0 58.7  5.4
 9. xe-0-2-0-29.r05.lsanca03.us.ce.gin.ntt.net                        0  1993   12.5  12.5  0.0  1.5 36.0 39.2
10. 192.229.130.10                                                    0  1993   12.6  12.6  0.0  0.5 22.9  1.5

Comment 4

2 years ago
Edgecast is no longer in use.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.