- mat-wg - mailman.ripe.net

Re: [mat-wg] [atlas] Proposal: Measure well-known CDNs,[CDN-HTTP]
by Massimo Candela 17 Dec '22

17 Dec '22

On 17/12/2022 00:19, Barry Raveendran Greene wrote: > > >> On Dec 16, 2022, at 23:29, Stephane Bortzmeyer <bortzmeyer(a)nic.fr> wrote: >> >> There is a larger problem here, a more strategic one: such a feature >> would contribute to the centralisation of the Internet, which is >> already too important. Tagging some targets are "important" and >> "worthy of measurements" would mean that we consider some HTTP servers >> to be more useful than others. That would be a bad message from RIPE. > > We’ve come full circle - we started with centralized PTTs - moved to a decentralized ASN/Paul Baran model - now Re-centralized based on marketing domination. > > +1 with Stephane’s observation. The selection of who to measure is a statement. +1 Also, while the data would be useful, I don't think the role of the ripe ncc is to grade commercial services. Let other companies or individual researchers do that.

1 0

RIPE Atlas, RIPEstat and RIS Quarterly Planning Q1 2023
by Robert Kisteleki 16 Dec '22

16 Dec '22

Dear colleagues, We’ve now published our quarterly planning for RIPE Atlas, RIPEstat and RIS. These are available at: https://www.ripe.net/support/documentation/quarterly-planning The goal of our quarterly planning is to communicate the work we’re doing for the coming months, get your input and start a discussion on other developments we should work on. If you have any input on our plans or want to let us know what you would like to see us work on, please let us know. Kind regards, Robert Kisteleki Research and Development Manager RIPE NCC

1 0

Fwd: [atlas] Proposal: Add support for STARTTLS measurements [STARTTLS]
by Stephen Strowes 14 Dec '22

14 Dec '22

Hi all, I'm forwarding these five threads into the MAT WG because they're all probably of interest to folks here. Please share your feedback with the NCC! Cheers, S. -------- Forwarded Message -------- Subject: [atlas] Proposal: Add support for STARTTLS measurements [STARTTLS] Date: Wed, 14 Dec 2022 15:04:06 +0100 From: Robert Kisteleki <robert(a)ripe.net> Organization: RIPE NCC To: ripe-atlas(a)ripe.net <ripe-atlas(a)ripe.net> Dear RIPE Atlas users, We recently published a RIPE Labs article containing a few proposals: https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. We'd like to encourage you to express your comments about this proposal (if you'd like to share them) here. Regards, Robert Kisteleki For the RIPE Atlas team -- ripe-atlas mailing list ripe-atlas(a)ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas

1 0

Fwd: [atlas] Proposal: Generic HTTP measurements [GENERIC-HTTP]
by Stephen Strowes 14 Dec '22

14 Dec '22

Hi all, I'm forwarding these five threads into the MAT WG because they're all probably of interest to folks here. Please share your feedback with the NCC! Cheers, S. -------- Forwarded Message -------- Subject: [atlas] Proposal: Generic HTTP measurements [GENERIC-HTTP] Date: Wed, 14 Dec 2022 15:03:32 +0100 From: Robert Kisteleki <robert(a)ripe.net> Organization: RIPE NCC To: ripe-atlas(a)ripe.net <ripe-atlas(a)ripe.net> Dear RIPE Atlas users, We recently published a RIPE Labs article containing a few proposals: https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. We'd like to encourage you to express your comments about this proposal (if you'd like to share them) here. Regards, Robert Kisteleki For the RIPE Atlas team -- ripe-atlas mailing list ripe-atlas(a)ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas

1 0

Fwd: [atlas] Proposal: Measure well-known CDNs, [CDN-HTTP]
by Stephen Strowes 14 Dec '22

14 Dec '22

Hi all, I'm forwarding these five threads into the MAT WG because they're all probably of interest to folks here. Please share your feedback with the NCC! Cheers, S. -------- Forwarded Message -------- Subject: [atlas] Proposal: Measure well-known CDNs,[CDN-HTTP] Date: Wed, 14 Dec 2022 15:02:46 +0100 From: Robert Kisteleki <robert(a)ripe.net> Organization: RIPE NCC To: ripe-atlas(a)ripe.net <ripe-atlas(a)ripe.net> Dear RIPE Atlas users, We recently published a RIPE Labs article containing a few proposals: https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. We'd like to encourage you to express your comments about this proposal (if you'd like to share them) here. Regards, Robert Kisteleki For the RIPE Atlas team -- ripe-atlas mailing list ripe-atlas(a)ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas

1 0

Fwd: [atlas] Proposal: remove per-hop “late” packets from traceroute [LATE-PACKETS]
by Stephen Strowes 14 Dec '22

14 Dec '22

Hi all, I'm forwarding these five threads into the MAT WG because they're all probably of interest to folks here. Please share your feedback with the NCC! Cheers, S. -------- Forwarded Message -------- Subject: [atlas] Proposal: remove per-hop “late” packets from traceroute [LATE-PACKETS] Date: Wed, 14 Dec 2022 15:02:13 +0100 From: Robert Kisteleki <robert(a)ripe.net> Organization: RIPE NCC To: ripe-atlas(a)ripe.net <ripe-atlas(a)ripe.net> Dear RIPE Atlas users, We recently published a RIPE Labs article containing a few proposals: https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. We'd like to encourage you to express your comments about this proposal (if you'd like to share them) here. Regards, Robert Kisteleki For the RIPE Atlas team -- ripe-atlas mailing list ripe-atlas(a)ripe.net https://lists.ripe.net/mailman/listinfo/ripe-atlas

1 0

MAT Working Group Minutes - RIPE 85
by Massimo Candela 29 Nov '22

29 Nov '22

Dear all, The MAT Working Group Minutes for RIPE 85 are available at https://www.ripe.net/participate/ripe/wg/active-wg/mat/minutes/mat-working-… Ciao, Massimo (on behalf of the MAT Chairs)

1 0

Format change for index pages of RIS MRT data files and RIPE Atlas daily dataset dumps
by Oleg Muravskiy 24 Nov '22

24 Nov '22

Dear RIS and RIPE Atlas users, On Wednesday, 30 November 2022 we are planning to migrate the web server at <https://data.ris.ripe.net/> as well as <https://data-store.ripe.net> from Apache httpd to nginx. This will affect how the file listings in HTML index pages are generated, including the following: * RIS MRT data files (updates and dumps) * RIPE Atlas daily dataset dumps If you rely on the format of these pages, you need to update your software. Before we do the upgrade, you could check the new format of the index pages at: https://test-data.ris.ripe.net/rrc00/ We would like to ask you to avoid relying on the specific format of the index.html files. If you are fetching BGP dump files, please use the REST API that is described at: https://ris.ripe.net/docs/40_Prototypes/15_per_rrc_dumps.html You might also consider using per-peer dump files instead: https://ris.ripe.net/docs/40_Prototypes/10_per_peer_dumps.html We want to encourage you to provide your feedback on the API, so that we could improve it and build a similar API for the update files. If you are fetching update files, please use direct URLs to files as described at: https://ris.ripe.net/docs/20_raw_data_mrt.html#name-and-location Kind regards, Oleg Muravskiy, Robert Kisteleki RIPE NCC

1 0

streaming link MAT session ripe85
by Massimo Candela 27 Oct '22

27 Oct '22

The MAT session started at RIPE85. Streaming here: https://ripe85.ripe.net/live/side/

1 0

MAT agenda for RIPE85
by Massimo Candela 26 Oct '22

26 Oct '22

Dear all, I'm pleased to announce that we have an agenda for RIPE85! https://ripe85.ripe.net/programme/meeting-plan/mat-wg/ Ciao, Massimo (on behalf of the MAT chairs)

2 1

Re: [mat-wg] [ippm] [Rpm] [M-Lab-Discuss] misery metrics & consequences
by J Ignacio Alvarez-Hamelin 26 Oct '22

26 Oct '22

Dear all, After some time in silence on the IPPM list, I like to make some comments here. As we presented in the draft-ietf-ippm-route-00 (now the RFC9198), the main problem is the traffic follows heavy-tailed distributions when it is seen from the end-to-end points: the origin of most of the issues in that video. Therefore, treating it as parametric distribution is not possible, unless you are dealing with a complex distribution like the Stable distribution: • B. Mandelbrot, “New methods in statistical economics,” Journal of political economy, vol. 71, no. 5, pp. 421–440, 1963. • ——, “The variation of certain speculative prices,” The journal of business, vol. 36, no. 4, pp. 394–419, 1963. (and so it will be extremely complex a high computing demands.) This is why we propose to use quartiles to characterize delays in the RFC9198. Then, I am doing some research to understand how the delay can change with network load, using the quartiles. I attach some measurements done during the pandemic, showing the congestion as a function of the time. Best, Ignacio _______________________________________________________________ Dr. Ing. José Ignacio Alvarez-Hamelin CONICET and Facultad de Ingeniería, Universidad de Buenos Aires Av. Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina +54 (11) 5285 0716 / 5285 0705 e-mail: ihameli(a)cnet.fi.uba.ar web: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/ _______________________________________________________________ > On 23 Oct 2022, at 08:57, Sebastian Moeller <moeller0(a)gmx.de> wrote: > > Hi Glenn, > > >> On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm <rpm(a)lists.bufferbloat.net> wrote: >> >> As a classic died in the wool empiricist, granted that you can identify "misery" factors, given a population of 1,000 users, how do you propose deriving a misery index for that population? >> >> We can measure download, upload, ping, jitter pretty much without user intervention. For the measurements you hypothesize, how you you automatically extract those indecies without subjective user contamination. >> >> I.e. my download speed sucks. Measure the download speed. >> >> My isp doesn't fix my problem. Measure what? How? >> >> Human survey technology is 70+ years old and it still has problems figuring out how to correlate opinion with fact. >> >> Without an objective measurement scheme that doesn't require human interaction, the misery index is a cool hypothesis with no way to link to actual data. What objective measurements can be made? Answer that and the index becomes useful. Otherwise it's just consumer whining. >> >> Not trying to be combative here, in fact I like the concept you support, but I'm hard pressed to see how the concept can lead to data, and the data lead to policy proposals. > > [SM] So it seems that outside of seemingly simple to test throughput numbers*, the next most important quality number (or the most important depending on subjective ranking) is how does latency change under "load". Absolute latency is also important albeit static high latency can be worked around within limits so the change under load seems more relevant. > All of flent's RRUL test, apple's networkQuality/RPM, and iperf2's bounceback test offer methods to asses latency change under load**, as do waveforms bufferbloat tests and even to a degree Ookla's speedtest.net. IMHO something like latency increase under load or apple's responsiveness measure RPM (basically the inverse of the latency under load calculated on a per minute basis, so it scales in the typical higher numbers are better way, unlike raw latency under load numbers where smaller is better). > IMHO what networkQuality is missing ATM is to measure and report the unloaded RPM as well as the loaded the first gives a measure over the static latency the second over how well things keep working if capacity gets tight. They report the base RTT which can be converted to RPM. As an example: > > macbook:~ user$ networkQuality -v > ==== SUMMARY ==== > Upload capacity: 24.341 Mbps > Download capacity: 91.951 Mbps > Upload flows: 20 > Download flows: 16 > Responsiveness: High (2123 RPM) > Base RTT: 16 > Start: 10/23/22, 13:44:39 > End: 10/23/22, 13:44:53 > OS Version: Version 12.6 (Build 21G115) > > Here RPM 2133 corresponds to 60000/2123 = 28.26 ms latency under load, while the Base RTT of 16ms corresponds to 60000/16 = 3750 RPM, son on this link load reduces the responsiveness by 3750-2123 = 1627 RPM a reduction by 100-100*2123/3750 = 43.4%, and that is with competent AQM and scheduling on the router. > > Without competent AQM/shaping I get: > ==== SUMMARY ==== > Upload capacity: 15.101 Mbps > Download capacity: 97.664 Mbps > Upload flows: 20 > Download flows: 12 > Responsiveness: Medium (427 RPM) > Base RTT: 16 > Start: 10/23/22, 13:51:50 > End: 10/23/22, 13:52:06 > OS Version: Version 12.6 (Build 21G115) > latency under load: 60000/427 = 140.52 ms > base RPM: 60000/16 = 3750 RPM > reduction RPM: 100-100*427/3750 = 88.6% > > > I understand apple's desire to have a single reported number with a single qualifier medium/high/... because in the end a link is only reliably usable if responsiveness under load stays acceptable, but with two numbers it is easier to see what one's ISP could do to help. (I guess some ISPs might already be unhappy with the single number, so this needs some diplomacy/tact) > > Regards > Sebastian > > > > *) Seemingly as quite some ISPs operate their own speedtest servers in their network and ignore customers not reaching the contracted rates into speedtest-servers located in different ASs. As the product is called internet access I a inclined to expect that my ISP maintains sufficient peering/transit capacity to reach the next tier of AS at my contracted rate (the EU legislative seems to agree, see EU directive 2015/2120). > > **) Most do by creating load themselves and measuring throughput at the same time, bounceback IIUC will focus on the latency measurement and leave the load generation optional (so offers a mode to measure responsiveness of a live network with minimal measurement traffic). @Bob, please correct me if this is wrong. > > >> >> >> On Fri, Oct 21, 2022, 5:20 PM Dave Taht <dave.taht(a)gmail.com> wrote: >> One of the best talks I've ever seen on how to measure customer >> satisfaction properly just went up after the P99 Conference. >> >> It's called Misery Metrics. >> >> After going through a deep dive as to why and how we think and act on >> percentiles, bins, and other statistical methods as to how we use the >> web and internet are *so wrong* (well worth watching and thinking >> about if you are relying on or creating network metrics today), it >> then points to the real metrics that matter to users and the ultimate >> success of an internet business: Timeouts, retries, misses, failed >> queries, angry phone calls, abandoned shopping carts and loss of >> engagement. >> >> https://www.p99conf.io/session/misery-metrics-consequences/ >> >> The ending advice was - don't aim to make a specific percentile >> acceptable, aim for an acceptable % of misery. >> >> I enjoyed the p99 conference more than any conference I've attended in years. >> >> -- >> This song goes out to all the folk that thought Stadia would work: >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656… >> Dave Täht CEO, TekLibre, LLC >> >> -- >> You received this message because you are subscribed to the Google Groups "discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe(a)measurementlab.net. >> To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAA93jw4w27a…. >> _______________________________________________ >> Rpm mailing list >> Rpm(a)lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/rpm > > _______________________________________________ > ippm mailing list > ippm(a)ietf.org > https://www.ietf.org/mailman/listinfo/ippm

2 1

Re: [mat-wg] [Rpm] [ippm] [M-Lab-Discuss] misery metrics & consequences
by J Ignacio Alvarez-Hamelin 25 Oct '22

25 Oct '22

Dear Bob, I answer inline. Thank you for your comments. P.D. It seems that I have no rights to send emails to the Measurement Analysis and Tools Working Group: how I can join it? _______________________________________________________________ Dr. Ing. José Ignacio Alvarez-Hamelin CONICET and Facultad de Ingeniería, Universidad de Buenos Aires Av. Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina +54 (11) 5285 0716 / 5285 0705 e-mail: ihameli(a)cnet.fi.uba.ar web: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/ _______________________________________________________________ > On 25 Oct 2022, at 14:02, rjmcmahon <rjmcmahon(a)rjmcmahon.com> wrote: > > I don't understand the information in the link. It looks like lines on a map to some form of dials which are too small to read. I suppose that depends on the zoom of your browser, try to put 100% (command+ to zoom-in in chrome). You will see a 24 hour clock where some hours are green (no congestion), yellow (some congestion) or red (congestion). But all of that is just an example. > > One can sample and create a Gaussian per the central limit theorem (CLT) if the underlying process probability density functions converge, i.e. can be integrated to 1. With that said, normalizing does lose information and doesn't say much about tails, outliers, etc. The Stable distribution is a generalization over all the distributions. Consider that Gaussian has finite moments, and heavy-tailed distribution not, this is where Stable one enters as a generalization. > > We should be careful in assuming only the tails matter and that all traffic follows heavy-tailed distributions. With bufferbloat it's the minimum of the latency PDF that shifts. Codel watches a minimum. "Jacobson suggested that average queue length actually contains no information at all about packet demand or network load.[3][5] He suggested that a better metric might be the minimum queue length during a sliding time window." Well, it depends on which point of the network you really are, but there a lot where this is true. In the other hand, no heavy tailed distributions do not cause problems if you use statistical tools that suport anything. For example, the median (Q2) converges to the mean (fist statistical moment) for Poisson distributions. > > We need statistical tools that also allow for the analysis of non-parametric distributions too. Hotelling T2 assumes the multivariate distributions are Gaussian. Kolmogorov-Smirnov tests can be used for non-parametric distributions. We find both are needed for SPC used by our automation systems. Yes, I did a lot of this kind of test, and I found very interesting things, even in the very important links (important ISP with important CDN), verifying Stable distributions with certain parameters (even some ones with heavy-tailed distributions). The problem is how to do this a low computable cost, and this why you can use quartiles. > > Sample subgroups of one really don't give sufficient information about any type of distribution, parametric or non-parametric. Here I cannot understand: which are the subgroups? > > Bob >> Dear all, >> After some time in silence on the IPPM list, I like to make some >> comments here. As we presented in the draft-ietf-ippm-route-00 (now >> the RFC9198), the main problem is the traffic follows heavy-tailed >> distributions when it is seen from the end-to-end points: the origin >> of most of the issues in that video. Therefore, treating it as >> parametric distribution is not possible, unless you are dealing with a >> complex distribution like the Stable distribution: >> • B. Mandelbrot, “New methods in statistical economics,” Journal of >> political economy, vol. 71, no. 5, pp. 421–440, 1963. >> • ——, “The variation of certain speculative prices,” The journal of >> business, vol. 36, no. 4, pp. 394–419, 1963. >> (and so it will be extremely complex a high computing demands.) >> This is why we propose to use quartiles to characterize delays in the >> RFC9198. Then, I am doing some research to understand how the delay >> can change with network load, using the quartiles. >> You can see some measurements done during the pandemic, showing the >> congestion as a function of the time (24 hours maximum): >> https://cnet.fi.uba.ar/ignacio.alvarez-hamelin/RIPE-Atlas-measurement-24681… >> [you can zoom in and out, pan it, clicking on the Xs you can close >> dialogs, to reopen them click on the link] >> Best, >> Ignacio >> ___________________________________ >> _______________________________________________________________ >> Dr. Ing. José Ignacio Alvarez-Hamelin >> CONICET and Facultad de Ingeniería, Universidad de Buenos Aires >> Av. Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina >> +54 (11) 5285 0716 / 5285 0705 >> e-mail: ihameli(a)cnet.fi.uba.ar >> web: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/ >> _______________________________________________________________ >>> On 23 Oct 2022, at 08:57, Sebastian Moeller <moeller0(a)gmx.de> wrote: >>> Hi Glenn, >>>> On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm <rpm(a)lists.bufferbloat.net> wrote: >>>> As a classic died in the wool empiricist, granted that you can identify "misery" factors, given a population of 1,000 users, how do you propose deriving a misery index for that population? >>>> We can measure download, upload, ping, jitter pretty much without user intervention. For the measurements you hypothesize, how you you automatically extract those indecies without subjective user contamination. >>>> I.e. my download speed sucks. Measure the download speed. >>>> My isp doesn't fix my problem. Measure what? How? >>>> Human survey technology is 70+ years old and it still has problems figuring out how to correlate opinion with fact. >>>> Without an objective measurement scheme that doesn't require human interaction, the misery index is a cool hypothesis with no way to link to actual data. What objective measurements can be made? Answer that and the index becomes useful. Otherwise it's just consumer whining. >>>> Not trying to be combative here, in fact I like the concept you support, but I'm hard pressed to see how the concept can lead to data, and the data lead to policy proposals. >>> [SM] So it seems that outside of seemingly simple to test throughput numbers*, the next most important quality number (or the most important depending on subjective ranking) is how does latency change under "load". Absolute latency is also important albeit static high latency can be worked around within limits so the change under load seems more relevant. >>> All of flent's RRUL test, apple's networkQuality/RPM, and iperf2's bounceback test offer methods to asses latency change under load**, as do waveforms bufferbloat tests and even to a degree Ookla's speedtest.net. IMHO something like latency increase under load or apple's responsiveness measure RPM (basically the inverse of the latency under load calculated on a per minute basis, so it scales in the typical higher numbers are better way, unlike raw latency under load numbers where smaller is better). >>> IMHO what networkQuality is missing ATM is to measure and report the unloaded RPM as well as the loaded the first gives a measure over the static latency the second over how well things keep working if capacity gets tight. They report the base RTT which can be converted to RPM. As an example: >>> macbook:~ user$ networkQuality -v >>> ==== SUMMARY ==== >>> Upload capacity: 24.341 Mbps >>> Download capacity: 91.951 Mbps >>> Upload flows: 20 >>> Download flows: 16 >>> Responsiveness: High (2123 RPM) >>> Base RTT: 16 >>> Start: 10/23/22, 13:44:39 >>> End: 10/23/22, 13:44:53 >>> OS Version: Version 12.6 (Build 21G115) >>> Here RPM 2133 corresponds to 60000/2123 = 28.26 ms latency under load, while the Base RTT of 16ms corresponds to 60000/16 = 3750 RPM, son on this link load reduces the responsiveness by 3750-2123 = 1627 RPM a reduction by 100-100*2123/3750 = 43.4%, and that is with competent AQM and scheduling on the router. >>> Without competent AQM/shaping I get: >>> ==== SUMMARY ==== >>> Upload capacity: 15.101 Mbps >>> Download capacity: 97.664 Mbps >>> Upload flows: 20 >>> Download flows: 12 >>> Responsiveness: Medium (427 RPM) >>> Base RTT: 16 >>> Start: 10/23/22, 13:51:50 >>> End: 10/23/22, 13:52:06 >>> OS Version: Version 12.6 (Build 21G115) >>> latency under load: 60000/427 = 140.52 ms >>> base RPM: 60000/16 = 3750 RPM >>> reduction RPM: 100-100*427/3750 = 88.6% >>> I understand apple's desire to have a single reported number with a single qualifier medium/high/... because in the end a link is only reliably usable if responsiveness under load stays acceptable, but with two numbers it is easier to see what one's ISP could do to help. (I guess some ISPs might already be unhappy with the single number, so this needs some diplomacy/tact) >>> Regards >>> Sebastian >>> *) Seemingly as quite some ISPs operate their own speedtest servers in their network and ignore customers not reaching the contracted rates into speedtest-servers located in different ASs. As the product is called internet access I a inclined to expect that my ISP maintains sufficient peering/transit capacity to reach the next tier of AS at my contracted rate (the EU legislative seems to agree, see EU directive 2015/2120). >>> **) Most do by creating load themselves and measuring throughput at the same time, bounceback IIUC will focus on the latency measurement and leave the load generation optional (so offers a mode to measure responsiveness of a live network with minimal measurement traffic). @Bob, please correct me if this is wrong. >>>> On Fri, Oct 21, 2022, 5:20 PM Dave Taht <dave.taht(a)gmail.com> wrote: >>>> One of the best talks I've ever seen on how to measure customer >>>> satisfaction properly just went up after the P99 Conference. >>>> It's called Misery Metrics. >>>> After going through a deep dive as to why and how we think and act on >>>> percentiles, bins, and other statistical methods as to how we use the >>>> web and internet are *so wrong* (well worth watching and thinking >>>> about if you are relying on or creating network metrics today), it >>>> then points to the real metrics that matter to users and the ultimate >>>> success of an internet business: Timeouts, retries, misses, failed >>>> queries, angry phone calls, abandoned shopping carts and loss of >>>> engagement. >>>> https://www.p99conf.io/session/misery-metrics-consequences/ >>>> The ending advice was - don't aim to make a specific percentile >>>> acceptable, aim for an acceptable % of misery. >>>> I enjoyed the p99 conference more than any conference I've attended in years. >>>> -- >>>> This song goes out to all the folk that thought Stadia would work: >>>> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656… >>>> Dave Täht CEO, TekLibre, LLC >>>> -- >>>> You received this message because you are subscribed to the Google Groups "discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe(a)measurementlab.net. >>>> To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAA93jw4w27a…. >>>> _______________________________________________ >>>> Rpm mailing list >>>> Rpm(a)lists.bufferbloat.net >>>> https://lists.bufferbloat.net/listinfo/rpm >>> _______________________________________________ >>> ippm mailing list >>> ippm(a)ietf.org >>> https://www.ietf.org/mailman/listinfo/ippm >> _______________________________________________ >> Rpm mailing list >> Rpm(a)lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/rpm

1 0

Re: [mat-wg] [ippm] [Rpm] [M-Lab-Discuss] misery metrics & consequences
by J Ignacio Alvarez-Hamelin 25 Oct '22

25 Oct '22

Dear all, After some time in silence on the IPPM list, I like to make some comments here. As we presented in the draft-ietf-ippm-route-00 (now the RFC9198), the main problem is the traffic follows heavy-tailed distributions when it is seen from the end-to-end points: the origin of most of the issues in that video. Therefore, treating it as parametric distribution is not possible, unless you are dealing with a complex distribution like the Stable distribution: • B. Mandelbrot, “New methods in statistical economics,” Journal of political economy, vol. 71, no. 5, pp. 421–440, 1963. • ——, “The variation of certain speculative prices,” The journal of business, vol. 36, no. 4, pp. 394–419, 1963. (and so it will be extremely complex a high computing demands.) This is why we propose to use quartiles to characterize delays in the RFC9198. Then, I am doing some research to understand how the delay can change with network load, using the quartiles. You can see some measurements done during the pandemic, showing the congestion as a function of the time (24 hours maximum): https://cnet.fi.uba.ar/ignacio.alvarez-hamelin/RIPE-Atlas-measurement-24681… [you can zoom in and out, pan it, clicking on the Xs you can close dialogs, to reopen them click on the link] Best, Ignacio ___________________________________ _______________________________________________________________ Dr. Ing. José Ignacio Alvarez-Hamelin CONICET and Facultad de Ingeniería, Universidad de Buenos Aires Av. Paseo Colón 850 - C1063ACV - Buenos Aires - Argentina +54 (11) 5285 0716 / 5285 0705 e-mail: ihameli(a)cnet.fi.uba.ar web: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/ _______________________________________________________________ > On 23 Oct 2022, at 08:57, Sebastian Moeller <moeller0(a)gmx.de> wrote: > > Hi Glenn, > > >> On Oct 23, 2022, at 02:17, Glenn Fishbine via Rpm <rpm(a)lists.bufferbloat.net> wrote: >> >> As a classic died in the wool empiricist, granted that you can identify "misery" factors, given a population of 1,000 users, how do you propose deriving a misery index for that population? >> >> We can measure download, upload, ping, jitter pretty much without user intervention. For the measurements you hypothesize, how you you automatically extract those indecies without subjective user contamination. >> >> I.e. my download speed sucks. Measure the download speed. >> >> My isp doesn't fix my problem. Measure what? How? >> >> Human survey technology is 70+ years old and it still has problems figuring out how to correlate opinion with fact. >> >> Without an objective measurement scheme that doesn't require human interaction, the misery index is a cool hypothesis with no way to link to actual data. What objective measurements can be made? Answer that and the index becomes useful. Otherwise it's just consumer whining. >> >> Not trying to be combative here, in fact I like the concept you support, but I'm hard pressed to see how the concept can lead to data, and the data lead to policy proposals. > > [SM] So it seems that outside of seemingly simple to test throughput numbers*, the next most important quality number (or the most important depending on subjective ranking) is how does latency change under "load". Absolute latency is also important albeit static high latency can be worked around within limits so the change under load seems more relevant. > All of flent's RRUL test, apple's networkQuality/RPM, and iperf2's bounceback test offer methods to asses latency change under load**, as do waveforms bufferbloat tests and even to a degree Ookla's speedtest.net. IMHO something like latency increase under load or apple's responsiveness measure RPM (basically the inverse of the latency under load calculated on a per minute basis, so it scales in the typical higher numbers are better way, unlike raw latency under load numbers where smaller is better). > IMHO what networkQuality is missing ATM is to measure and report the unloaded RPM as well as the loaded the first gives a measure over the static latency the second over how well things keep working if capacity gets tight. They report the base RTT which can be converted to RPM. As an example: > > macbook:~ user$ networkQuality -v > ==== SUMMARY ==== > Upload capacity: 24.341 Mbps > Download capacity: 91.951 Mbps > Upload flows: 20 > Download flows: 16 > Responsiveness: High (2123 RPM) > Base RTT: 16 > Start: 10/23/22, 13:44:39 > End: 10/23/22, 13:44:53 > OS Version: Version 12.6 (Build 21G115) > > Here RPM 2133 corresponds to 60000/2123 = 28.26 ms latency under load, while the Base RTT of 16ms corresponds to 60000/16 = 3750 RPM, son on this link load reduces the responsiveness by 3750-2123 = 1627 RPM a reduction by 100-100*2123/3750 = 43.4%, and that is with competent AQM and scheduling on the router. > > Without competent AQM/shaping I get: > ==== SUMMARY ==== > Upload capacity: 15.101 Mbps > Download capacity: 97.664 Mbps > Upload flows: 20 > Download flows: 12 > Responsiveness: Medium (427 RPM) > Base RTT: 16 > Start: 10/23/22, 13:51:50 > End: 10/23/22, 13:52:06 > OS Version: Version 12.6 (Build 21G115) > latency under load: 60000/427 = 140.52 ms > base RPM: 60000/16 = 3750 RPM > reduction RPM: 100-100*427/3750 = 88.6% > > > I understand apple's desire to have a single reported number with a single qualifier medium/high/... because in the end a link is only reliably usable if responsiveness under load stays acceptable, but with two numbers it is easier to see what one's ISP could do to help. (I guess some ISPs might already be unhappy with the single number, so this needs some diplomacy/tact) > > Regards > Sebastian > > > > *) Seemingly as quite some ISPs operate their own speedtest servers in their network and ignore customers not reaching the contracted rates into speedtest-servers located in different ASs. As the product is called internet access I a inclined to expect that my ISP maintains sufficient peering/transit capacity to reach the next tier of AS at my contracted rate (the EU legislative seems to agree, see EU directive 2015/2120). > > **) Most do by creating load themselves and measuring throughput at the same time, bounceback IIUC will focus on the latency measurement and leave the load generation optional (so offers a mode to measure responsiveness of a live network with minimal measurement traffic). @Bob, please correct me if this is wrong. > > >> >> >> On Fri, Oct 21, 2022, 5:20 PM Dave Taht <dave.taht(a)gmail.com> wrote: >> One of the best talks I've ever seen on how to measure customer >> satisfaction properly just went up after the P99 Conference. >> >> It's called Misery Metrics. >> >> After going through a deep dive as to why and how we think and act on >> percentiles, bins, and other statistical methods as to how we use the >> web and internet are *so wrong* (well worth watching and thinking >> about if you are relying on or creating network metrics today), it >> then points to the real metrics that matter to users and the ultimate >> success of an internet business: Timeouts, retries, misses, failed >> queries, angry phone calls, abandoned shopping carts and loss of >> engagement. >> >> https://www.p99conf.io/session/misery-metrics-consequences/ >> >> The ending advice was - don't aim to make a specific percentile >> acceptable, aim for an acceptable % of misery. >> >> I enjoyed the p99 conference more than any conference I've attended in years. >> >> -- >> This song goes out to all the folk that thought Stadia would work: >> https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656… >> Dave Täht CEO, TekLibre, LLC >> >> -- >> You received this message because you are subscribed to the Google Groups "discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an email to discuss+unsubscribe(a)measurementlab.net. >> To view this discussion on the web visit https://groups.google.com/a/measurementlab.net/d/msgid/discuss/CAA93jw4w27a…. >> _______________________________________________ >> Rpm mailing list >> Rpm(a)lists.bufferbloat.net >> https://lists.bufferbloat.net/listinfo/rpm > > _______________________________________________ > ippm mailing list > ippm(a)ietf.org > https://www.ietf.org/mailman/listinfo/ippm

1 0

misery metrics & consequences
by Dave Taht 22 Oct '22

22 Oct '22

One of the best talks I've ever seen on how to measure customer satisfaction properly just went up after the P99 Conference. It's called Misery Metrics. After going through a deep dive as to why and how we think and act on percentiles, bins, and other statistical methods as to how we use the web and internet are *so wrong* (well worth watching and thinking about if you are relying on or creating network metrics today), it then points to the real metrics that matter to users and the ultimate success of an internet business: Timeouts, retries, misses, failed queries, angry phone calls, abandoned shopping carts and loss of engagement. https://www.p99conf.io/session/misery-metrics-consequences/ The ending advice was - don't aim to make a specific percentile acceptable, aim for an acceptable % of misery. I enjoyed the p99 conference more than any conference I've attended in years. -- This song goes out to all the folk that thought Stadia would work: https://www.linkedin.com/posts/dtaht_the-mushroom-song-activity-69813666656… Dave Täht CEO, TekLibre, LLC

2 2

RIPE Atlas, RIPEstat and RIS Quarterly Planning Q4 2022
by Robert Kisteleki 27 Sep '22

27 Sep '22

Dear colleagues, We’ve now published our quarterly planning for RIPE Atlas, RIPEstat and RIS. These are available at: https://www.ripe.net/support/documentation/quarterly-planning The goal of our quarterly planning is to communicate the work we’re doing for the coming months, get your input and start a discussion on other developments we should work on. If you have any input on our plans or want to let us know what you would like to see us work on, please let us know. Kind regards, Robert Kisteleki Research and Development Manager RIPE NCC

1 0

Open House: Exploring RIPEstat
by Christian Teuschel 21 Sep '22

21 Sep '22

Dear colleagues, A quick note to remind you that the RIPE NCC Open House: Exploring RIPEstat is taking place on Wednesday, 28 September at 10:00 (UTC). During this online event, we will talk about what makes this tool a favourite for network operators, CERTs and a wide group of Internet stakeholders. I will be joined by guests who will speak in detail about RIPEstat datasets, a number of new features and the new UI. Check out the details and register for free at: https://www.ripe.net/participate/meetings/open-house/ripe-ncc-open-house-ex… Best regards, Christian Teuschel

1 0

Internet Measurement Day - Tomorrow, 13 September, Online
by Gergana Petrova 12 Sep '22

12 Sep '22

Dear colleagues, A quick note to remind you that the Internet Measurement Day Serbia is taking place tomorrow, 13 September, online. During this online event we will talk about tools on how to measure the Internet, such as RIPE Atlas, the Identifier Technology Health Indicators (ITHI Metrics), Routing Information System (RIS), Resource Public Key Infrastructure (RPKI) and the RIPE NCC’s open data platform RIPEstat. You will also hear from a number of local and international organisations, such as RATEL, SOX, CAIDA and more. Even though the particular focus is on Serbia, a lot of the presentations would be general. Perhaps even the more local presentations would be relevant for those of you who are doing business in neighbouring countries. Check out the agenda and if interested, register for free here: https://www.ripe.net/participate/meetings/internet-measurement-day/internet… Best regards, Gergana -- Gergana Petrova Community Development Manager RIPE NCC

1 0

Fwd: Join Virtual En-ROADS Climate Workshop on September 8!
by Vesna Manojlovic 07 Sep '22

07 Sep '22

Dear MAT-WG, here's an invitation to en_ROADS workshop: https://www.climateinteractive.org/get-involved/webinars/ This is a great showcase for the visualisation of complex & open data, measured over long time periods, analysed and fit into models that are predicting outcomes in the future. I find it highly relevant for the operations of Internet infrastructure, since it has elements of: - water availability https://www.datacenterdynamics.com/en/opinions/why-data-centers-need-to-tal… - sea-level rise https://labs.ripe.net/author/ggm/surviving-the-flood-how-data-centres-and-t… - energy consumption https://www.consultancy.uk/news/28772/10-ways-businesses-can-reduce-their-d… I hope you find it useful, regards, Vesna

1 0

samknows reports 853 ms of latency under load for US DSL.
by Dave Taht 26 Aug '22

26 Aug '22

US DSL, according to their measurements, has 853 ms of latency under load, as reported to the FCC. pretty good podcast and blog at: https://samknows.com/blog/bufferbloat - where they also discuss their current and planned methods of measurement. Also, speedtest.net is finally reporting on "working latency", and their new speedtest apps showing it to the user. https://www.ookla.com/articles/introducing-loaded-latency It would be really great if y'all could join in on helping us finally solve the bufferbloat problem internet wide. -- FQ World Domination pending: https://blog.cerowrt.org/post/state_of_fq_codel/ Dave Täht CEO, TekLibre, LLC

1 0

New on RIPE Labs: 240/4 As Seen by RIPE Atlas
by Qasim Lone 23 Aug '22

23 Aug '22

Dear colleagues, In this new article on RIPE Labs, we use data from RIPE Atlas probes to investigate the usage of 240/4, a block of IPv4 addresses 'reserved for future use' and formally known as Class E in the wild. https://labs.ripe.net/author/qasim-lone/2404-as-seen-by-ripe-atlas/ Kind regards, Qasim

1 0

Call for presentations MAT WG at RIPE85
by Massimo Candela 15 Aug '22

15 Aug '22

Dear MAT WG, We should start getting ready for RIPE85! This year we have a 1.5 hours slot on Thursday 27 October at 09:00 CEST (draft, things may change). Please, submit your presentation proposal by September 9th by sending an email at mat-wg-chairs(a)ripe.net. We prefer talks of 15min. Include in your email a title, a draft of the presentation, and a paragraph describing what you will talk about and why the topic is of interest for the community. Please, let us know if you are planning to present in person or remotely. Thanks! Cheers, Nina, Stephen, Massimo

1 1

Announcement of new RIS prototype services
by Oleg Muravskiy 05 Aug '22

05 Aug '22

Dear RIS users, Over the past few months, we have created several prototype services that expose RIS data in different ways. These services provide a REST API, and the data they expose is closer to real-time, as compared to our traditional "bview" and "updates" files. These services are: - Per-peer dumps https://ris.ripe.net/docs/40_Prototypes/10_per_peer_dumps.html - Per-RRC dumps https://ris.ripe.net/docs/40_Prototypes/15_per_rrc_dumps.html - Peer metadata https://ris.ripe.net/docs/40_Prototypes/20_peer_metadata.html Please take a look at the documentation (links provided above), try out these new services, and tell us what we can improve! We would love to hear your feedback. Since we're posting this message across multiple mailing lists, we ask you to please share your feedback on the ris-users(a)ripe.net mailing list, or write to us directly at ris(a)ripe.net rather than on this list. Kind regards, Oleg Muravskiy RIPE NCC

1 0

New on RIPE Labs: Measuring Encrypted-DNS Censorship Using OONI Probe
by Anastasiya Pak 04 Aug '22

04 Aug '22

Dear colleagues, The Open Observatory Network Interference (OONI) is a non-profit free software project that aims to empower decentralised efforts in documenting Internet censorship around the world. In collaboration with different partners, Simone Basso and colleagues publish research reports and research papers documenting cases of Internet censorship around the world. In this new article on RIPE Labs, Simone talks through the preliminary results of an encrypted-DNS censorship study currently being conducted by the OONI. Read now on Labs: https://labs.ripe.net/author/bassosimone/measuring-encrypted-dns-censorship… Best regards, Anastasiya Pak Marketing & Communications Officer RIPE NCC

1 0

New on RIPE Labs: Dealing with the Undercurrent of Unwanted Traffic
by Alun Davies 06 Jul '22

06 Jul '22

Dear colleagues, Leslie Daigle and her colleagues over at the Global Cyber Alliance have been collecting data from hundreds of sensors to get a clearer picture of how much unwanted traffic there is out there. How much counts as too much? Read more on RIPE Labs: https://labs.ripe.net/author/leslie-daigle/dealing-with-the-undercurrent-of… Best regards, Alun Davies RIPE Labs Editor RIPE NCC

3 3

RIPE Atlas, RIPEstat and RIS Quarterly Planning Q3 2022
by Robert Kisteleki 30 Jun '22

30 Jun '22

Dear colleagues, We’ve now published our quarterly for RIPE Atlas, RIPEstat and RIS. These are available at: https://www.ripe.net/support/documentation/quarterly-planning We hope that by providing these plans, we can better inform the community of the ongoing work on the services we operate and get feedback on our work. If you have any input on our plans or want to let us know what you would like to see us work on, please let us know. Kind regards, Robert Kisteleki Research and Development Manager RIPE NCC

1 0